2023-09-13T02:43:56.156 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-09-13T02:43:56.157 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} --archive /home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395256 --owner scheduled_pdonnell@teuthology --verbose --name pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi -- /home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395256/orig.config.yaml 2023-09-13T02:43:56.228 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T02:43:56.331 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395256 branch: wip-batrick-testing-20230912.122437 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} email: pdonnell@redhat.com first_in_suite: false job_id: '7395256' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-batrick-testing-20230912.122437 ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: global: ms die on skipped message: false mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 2110e00747b31a0d73768c9e5229da18b26b8aa0 subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 2110e00747b31a0d73768c9e5229da18b26b8aa0 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - wsync syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20230912.122437 sha1: 3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 9684 sha1: 2110e00747b31a0d73768c9e5229da18b26b8aa0 sleep_before_teardown: 0 subset: 236/256 suite: fs suite_branch: wip-batrick-testing-20230912.122437 suite_path: /home/teuthworker/src/github.com_batrick_ceph_3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f/qa suite_relpath: qa suite_repo: https://github.com/batrick/ceph.git suite_sha1: 3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi064.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi142.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi170.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1M - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6M3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-09-12_14:07:50 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.1394 2023-09-13T02:43:56.331 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_batrick_ceph_3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f/qa; will attempt to use it 2023-09-13T02:43:56.332 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_batrick_ceph_3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f/qa/tasks 2023-09-13T02:43:56.333 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-09-13T02:43:56.335 INFO:teuthology.task.internal:Checking packages... 2023-09-13T02:43:56.359 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '2110e00747b31a0d73768c9e5229da18b26b8aa0' 2023-09-13T02:43:56.359 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-09-13T02:43:56.360 INFO:teuthology.packaging:ref: None 2023-09-13T02:43:56.360 INFO:teuthology.packaging:tag: None 2023-09-13T02:43:56.360 INFO:teuthology.packaging:branch: wip-batrick-testing-20230912.122437 2023-09-13T02:43:56.361 INFO:teuthology.packaging:sha1: 2110e00747b31a0d73768c9e5229da18b26b8aa0 2023-09-13T02:43:56.361 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-batrick-testing-20230912.122437 2023-09-13T02:43:56.494 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-6088.g2110e007 2023-09-13T02:43:56.494 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-09-13T02:43:56.501 INFO:teuthology.task.internal:no buildpackages task found 2023-09-13T02:43:56.501 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-09-13T02:43:56.527 INFO:teuthology.task.internal:Saving configuration 2023-09-13T02:43:56.550 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-09-13T02:43:56.561 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-13T02:43:56.586 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi064.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395256', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-13 02:38:41.933685', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-13T02:43:56.611 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi142.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395256', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-13 02:38:41.938062', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-13T02:43:56.631 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi170.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395256', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-13 02:38:41.935152', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-13T02:43:56.631 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-09-13T02:43:56.638 INFO:teuthology.task.internal:roles: ubuntu@smithi064.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-09-13T02:43:56.638 INFO:teuthology.task.internal:roles: ubuntu@smithi142.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-09-13T02:43:56.639 INFO:teuthology.task.internal:roles: ubuntu@smithi170.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-09-13T02:43:56.639 INFO:teuthology.run_tasks:Running task console_log... 2023-09-13T02:43:56.761 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3dcd22aca0>, signals=[15]) 2023-09-13T02:43:56.763 INFO:teuthology.run_tasks:Running task internal.connect... 2023-09-13T02:43:56.769 INFO:teuthology.task.internal:Opening connections... 2023-09-13T02:43:56.769 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi064.front.sepia.ceph.com 2023-09-13T02:43:56.771 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T02:43:56.840 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi142.front.sepia.ceph.com 2023-09-13T02:43:56.842 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T02:43:56.908 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi170.front.sepia.ceph.com 2023-09-13T02:43:56.909 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T02:43:56.975 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-09-13T02:43:56.982 DEBUG:teuthology.orchestra.run.smithi064:> uname -m 2023-09-13T02:43:57.003 INFO:teuthology.orchestra.run.smithi064.stdout:x86_64 2023-09-13T02:43:57.004 DEBUG:teuthology.orchestra.run.smithi064:> cat /etc/os-release 2023-09-13T02:43:57.062 INFO:teuthology.orchestra.run.smithi064.stdout:NAME="Red Hat Enterprise Linux" 2023-09-13T02:43:57.063 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION="8.6 (Ootpa)" 2023-09-13T02:43:57.063 INFO:teuthology.orchestra.run.smithi064.stdout:ID="rhel" 2023-09-13T02:43:57.063 INFO:teuthology.orchestra.run.smithi064.stdout:ID_LIKE="fedora" 2023-09-13T02:43:57.063 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION_ID="8.6" 2023-09-13T02:43:57.064 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM_ID="platform:el8" 2023-09-13T02:43:57.064 INFO:teuthology.orchestra.run.smithi064.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-09-13T02:43:57.064 INFO:teuthology.orchestra.run.smithi064.stdout:ANSI_COLOR="0;31" 2023-09-13T02:43:57.064 INFO:teuthology.orchestra.run.smithi064.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-09-13T02:43:57.064 INFO:teuthology.orchestra.run.smithi064.stdout:HOME_URL="https://www.redhat.com/" 2023-09-13T02:43:57.065 INFO:teuthology.orchestra.run.smithi064.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-13T02:43:57.065 INFO:teuthology.orchestra.run.smithi064.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-13T02:43:57.065 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T02:43:57.065 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-13T02:43:57.066 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-09-13T02:43:57.066 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-13T02:43:57.066 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-09-13T02:43:57.067 INFO:teuthology.lock.ops:Updating smithi064.front.sepia.ceph.com on lock server 2023-09-13T02:43:57.087 DEBUG:teuthology.orchestra.run.smithi142:> uname -m 2023-09-13T02:43:57.108 INFO:teuthology.orchestra.run.smithi142.stdout:x86_64 2023-09-13T02:43:57.108 DEBUG:teuthology.orchestra.run.smithi142:> cat /etc/os-release 2023-09-13T02:43:57.168 INFO:teuthology.orchestra.run.smithi142.stdout:NAME="Red Hat Enterprise Linux" 2023-09-13T02:43:57.168 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION="8.6 (Ootpa)" 2023-09-13T02:43:57.168 INFO:teuthology.orchestra.run.smithi142.stdout:ID="rhel" 2023-09-13T02:43:57.169 INFO:teuthology.orchestra.run.smithi142.stdout:ID_LIKE="fedora" 2023-09-13T02:43:57.169 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION_ID="8.6" 2023-09-13T02:43:57.169 INFO:teuthology.orchestra.run.smithi142.stdout:PLATFORM_ID="platform:el8" 2023-09-13T02:43:57.169 INFO:teuthology.orchestra.run.smithi142.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-09-13T02:43:57.169 INFO:teuthology.orchestra.run.smithi142.stdout:ANSI_COLOR="0;31" 2023-09-13T02:43:57.170 INFO:teuthology.orchestra.run.smithi142.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-09-13T02:43:57.170 INFO:teuthology.orchestra.run.smithi142.stdout:HOME_URL="https://www.redhat.com/" 2023-09-13T02:43:57.170 INFO:teuthology.orchestra.run.smithi142.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-13T02:43:57.170 INFO:teuthology.orchestra.run.smithi142.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-13T02:43:57.170 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-09-13T02:43:57.171 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-13T02:43:57.171 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-09-13T02:43:57.171 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-13T02:43:57.171 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-09-13T02:43:57.171 INFO:teuthology.lock.ops:Updating smithi142.front.sepia.ceph.com on lock server 2023-09-13T02:43:57.195 DEBUG:teuthology.orchestra.run.smithi170:> uname -m 2023-09-13T02:43:57.215 INFO:teuthology.orchestra.run.smithi170.stdout:x86_64 2023-09-13T02:43:57.216 DEBUG:teuthology.orchestra.run.smithi170:> cat /etc/os-release 2023-09-13T02:43:57.275 INFO:teuthology.orchestra.run.smithi170.stdout:NAME="Red Hat Enterprise Linux" 2023-09-13T02:43:57.275 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION="8.6 (Ootpa)" 2023-09-13T02:43:57.276 INFO:teuthology.orchestra.run.smithi170.stdout:ID="rhel" 2023-09-13T02:43:57.276 INFO:teuthology.orchestra.run.smithi170.stdout:ID_LIKE="fedora" 2023-09-13T02:43:57.276 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_ID="8.6" 2023-09-13T02:43:57.276 INFO:teuthology.orchestra.run.smithi170.stdout:PLATFORM_ID="platform:el8" 2023-09-13T02:43:57.276 INFO:teuthology.orchestra.run.smithi170.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-09-13T02:43:57.277 INFO:teuthology.orchestra.run.smithi170.stdout:ANSI_COLOR="0;31" 2023-09-13T02:43:57.277 INFO:teuthology.orchestra.run.smithi170.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-09-13T02:43:57.277 INFO:teuthology.orchestra.run.smithi170.stdout:HOME_URL="https://www.redhat.com/" 2023-09-13T02:43:57.277 INFO:teuthology.orchestra.run.smithi170.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-13T02:43:57.277 INFO:teuthology.orchestra.run.smithi170.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-13T02:43:57.278 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-09-13T02:43:57.278 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-13T02:43:57.278 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-09-13T02:43:57.279 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-13T02:43:57.279 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-09-13T02:43:57.279 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server 2023-09-13T02:43:57.295 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-09-13T02:43:57.306 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-09-13T02:43:57.370 INFO:teuthology.task.internal:Checking for old test directory... 2023-09-13T02:43:57.370 DEBUG:teuthology.orchestra.run.smithi064:> test '!' -e /home/ubuntu/cephtest 2023-09-13T02:43:57.373 DEBUG:teuthology.orchestra.run.smithi142:> test '!' -e /home/ubuntu/cephtest 2023-09-13T02:43:57.375 DEBUG:teuthology.orchestra.run.smithi170:> test '!' -e /home/ubuntu/cephtest 2023-09-13T02:43:57.394 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-09-13T02:43:57.400 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-09-13T02:43:57.400 DEBUG:teuthology.orchestra.run.smithi064:> test -z $(ls -A /var/lib/ceph) 2023-09-13T02:43:57.433 DEBUG:teuthology.orchestra.run.smithi142:> test -z $(ls -A /var/lib/ceph) 2023-09-13T02:43:57.435 DEBUG:teuthology.orchestra.run.smithi170:> test -z $(ls -A /var/lib/ceph) 2023-09-13T02:43:57.461 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-09-13T02:43:57.542 INFO:teuthology.run_tasks:Running task kernel... 2023-09-13T02:43:57.556 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2023-09-13T02:43:57.557 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2023-09-13T02:43:57.557 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2023-09-13T02:43:57.557 DEBUG:teuthology.orchestra.run.smithi064:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-13T02:43:57.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-13T02:43:57.577 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-09-13T02:43:57.861 DEBUG:teuthology.task.kernel:sha1 for client.0 is fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-09-13T02:43:57.861 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c"... 2023-09-13T02:43:57.862 DEBUG:teuthology.orchestra.run.smithi064:> uname -r 2023-09-13T02:43:57.882 INFO:teuthology.orchestra.run.smithi064.stdout:4.18.0-372.9.1.el8.x86_64 2023-09-13T02:43:57.882 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-09-13T02:43:57.882 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-09-13T02:43:57.883 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum upgrade -y linux-firmware 2023-09-13T02:43:59.111 INFO:teuthology.orchestra.run.smithi064.stdout:Updating Subscription Management repositories. 2023-09-13T02:43:59.111 INFO:teuthology.orchestra.run.smithi064.stdout:Unable to read consumer identity 2023-09-13T02:44:00.493 INFO:teuthology.orchestra.run.smithi064.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2023-09-13T02:44:01.190 INFO:teuthology.orchestra.run.smithi064.stdout:Extra Packages for Enterprise Linux 23 MB/s | 16 MB 00:00 2023-09-13T02:44:04.894 INFO:teuthology.orchestra.run.smithi064.stdout:lab-extras 431 kB/s | 24 kB 00:00 2023-09-13T02:44:05.649 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2023-09-13T02:44:05.650 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2023-09-13T02:44:05.650 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2023-09-13T02:44:05.800 INFO:teuthology.task.kernel:Downloading kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on client.0... 2023-09-13T02:44:05.801 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-09-13T02:44:06.015 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/ 2023-09-13T02:44:06.015 DEBUG:teuthology.orchestra.run.smithi064:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/ --input-file=- 2023-09-13T02:44:06.018 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi064.front.sepia.ceph.com to complete... 2023-09-13T02:44:19.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13 02:44:19 URL:https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm [380759012/380759012] -> "/tmp/kernel.x86_64.rpm" [1] 2023-09-13T02:44:19.066 INFO:teuthology.orchestra.run.smithi064.stderr:FINISHED --2023-09-13 02:44:19-- 2023-09-13T02:44:19.066 INFO:teuthology.orchestra.run.smithi064.stderr:Total wall clock time: 13s 2023-09-13T02:44:19.066 INFO:teuthology.orchestra.run.smithi064.stderr:Downloaded: 1 files, 363M in 13s (28.6 MB/s) 2023-09-13T02:44:19.068 INFO:teuthology.task.kernel:Installing kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on client.0... 2023-09-13T02:44:19.068 DEBUG:teuthology.orchestra.run.smithi064:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-09-13T02:44:20.727 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying... ######################################## 2023-09-13T02:44:20.781 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing... ######################################## 2023-09-13T02:44:20.867 INFO:teuthology.orchestra.run.smithi064.stdout:Updating / installing... 2023-09-13T02:45:01.569 INFO:teuthology.orchestra.run.smithi064.stdout:kernel-6.5.0_rc7_gfef64f93da2b-1 ######################################## 2023-09-13T02:45:27.158 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi064.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-09-13T02:45:27.158 DEBUG:teuthology.orchestra.run.smithi064:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-09-13T02:45:27.216 INFO:teuthology.orchestra.run.smithi064.stdout:/boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-09-13T02:45:27.217 INFO:teuthology.orchestra.run.smithi064.stdout:/boot/config-6.5.0-rc7-gfef64f93da2b 2023-09-13T02:45:27.217 INFO:teuthology.orchestra.run.smithi064.stdout:/boot/vmlinuz-6.5.0-rc7-gfef64f93da2b 2023-09-13T02:45:27.217 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b 2023-09-13T02:45:27.217 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel 2023-09-13T02:45:27.217 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch 2023-09-13T02:45:27.218 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86 2023-09-13T02:45:27.218 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto 2023-09-13T02:45:27.218 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-09-13T02:45:27.218 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-09-13T02:45:27.218 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-09-13T02:45:27.219 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-09-13T02:45:27.219 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-09-13T02:45:27.219 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-09-13T02:45:27.219 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-09-13T02:45:27.219 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32c-intel.ko 2023-09-13T02:45:27.220 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-09-13T02:45:27.220 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-09-13T02:45:27.220 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-09-13T02:45:27.220 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-09-13T02:45:27.221 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-09-13T02:45:27.221 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-09-13T02:45:27.221 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-09-13T02:45:27.221 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-09-13T02:45:27.221 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-09-13T02:45:27.222 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-09-13T02:45:27.222 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-09-13T02:45:27.222 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-09-13T02:45:27.222 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel 2023-09-13T02:45:27.222 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu 2023-09-13T02:45:27.223 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce 2023-09-13T02:45:27.223 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-09-13T02:45:27.223 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm 2023-09-13T02:45:27.223 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-amd.ko 2023-09-13T02:45:27.223 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-intel.ko 2023-09-13T02:45:27.224 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm.ko 2023-09-13T02:45:27.224 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto 2023-09-13T02:45:27.224 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ansi_cprng.ko 2023-09-13T02:45:27.224 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/anubis.ko 2023-09-13T02:45:27.224 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/arc4.ko 2023-09-13T02:45:27.225 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx 2023-09-13T02:45:27.225 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_memcpy.ko 2023-09-13T02:45:27.225 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_pq.ko 2023-09-13T02:45:27.225 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-09-13T02:45:27.225 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_tx.ko 2023-09-13T02:45:27.225 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_xor.ko 2023-09-13T02:45:27.226 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/raid6test.ko 2023-09-13T02:45:27.226 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authenc.ko 2023-09-13T02:45:27.226 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authencesn.ko 2023-09-13T02:45:27.226 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blake2b_generic.ko 2023-09-13T02:45:27.226 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_common.ko 2023-09-13T02:45:27.227 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_generic.ko 2023-09-13T02:45:27.227 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/camellia_generic.ko 2023-09-13T02:45:27.227 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast5_generic.ko 2023-09-13T02:45:27.227 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast6_generic.ko 2023-09-13T02:45:27.227 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast_common.ko 2023-09-13T02:45:27.228 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ccm.ko 2023-09-13T02:45:27.228 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cmac.ko 2023-09-13T02:45:27.228 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crc32_generic.ko 2023-09-13T02:45:27.228 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_engine.ko 2023-09-13T02:45:27.228 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_user.ko 2023-09-13T02:45:27.229 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cts.ko 2023-09-13T02:45:27.229 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/des_generic.ko 2023-09-13T02:45:27.229 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/dh_generic.ko 2023-09-13T02:45:27.229 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/echainiv.ko 2023-09-13T02:45:27.229 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/essiv.ko 2023-09-13T02:45:27.230 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/fcrypt.ko 2023-09-13T02:45:27.230 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/khazad.ko 2023-09-13T02:45:27.230 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4.ko 2023-09-13T02:45:27.230 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4hc.ko 2023-09-13T02:45:27.230 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/md4.ko 2023-09-13T02:45:27.231 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/michael_mic.ko 2023-09-13T02:45:27.231 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcbc.ko 2023-09-13T02:45:27.231 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcrypt.ko 2023-09-13T02:45:27.231 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/rmd160.ko 2023-09-13T02:45:27.231 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/seed.ko 2023-09-13T02:45:27.232 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/serpent_generic.ko 2023-09-13T02:45:27.232 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tcrypt.ko 2023-09-13T02:45:27.232 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tea.ko 2023-09-13T02:45:27.232 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_common.ko 2023-09-13T02:45:27.232 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_generic.ko 2023-09-13T02:45:27.233 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/vmac.ko 2023-09-13T02:45:27.233 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/wp512.ko 2023-09-13T02:45:27.233 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xcbc.ko 2023-09-13T02:45:27.233 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xor.ko 2023-09-13T02:45:27.233 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xxhash_generic.ko 2023-09-13T02:45:27.234 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers 2023-09-13T02:45:27.234 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi 2023-09-13T02:45:27.234 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_ipmi.ko 2023-09-13T02:45:27.234 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_pad.ko 2023-09-13T02:45:27.234 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/custom_method.ko 2023-09-13T02:45:27.235 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/ec_sys.ko 2023-09-13T02:45:27.235 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/platform_profile.ko 2023-09-13T02:45:27.235 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbs.ko 2023-09-13T02:45:27.235 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbshc.ko 2023-09-13T02:45:27.235 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/video.ko 2023-09-13T02:45:27.235 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata 2023-09-13T02:45:27.236 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/acard-ahci.ko 2023-09-13T02:45:27.236 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ahci_platform.ko 2023-09-13T02:45:27.236 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ata_generic.ko 2023-09-13T02:45:27.236 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/libahci_platform.ko 2023-09-13T02:45:27.236 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_acpi.ko 2023-09-13T02:45:27.237 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ali.ko 2023-09-13T02:45:27.237 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_amd.ko 2023-09-13T02:45:27.237 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_artop.ko 2023-09-13T02:45:27.237 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atiixp.ko 2023-09-13T02:45:27.237 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atp867x.ko 2023-09-13T02:45:27.238 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd640.ko 2023-09-13T02:45:27.238 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd64x.ko 2023-09-13T02:45:27.238 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cypress.ko 2023-09-13T02:45:27.238 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_efar.ko 2023-09-13T02:45:27.238 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt366.ko 2023-09-13T02:45:27.239 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt37x.ko 2023-09-13T02:45:27.239 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-09-13T02:45:27.239 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x3.ko 2023-09-13T02:45:27.239 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it8213.ko 2023-09-13T02:45:27.239 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it821x.ko 2023-09-13T02:45:27.240 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_jmicron.ko 2023-09-13T02:45:27.240 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_marvell.ko 2023-09-13T02:45:27.240 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_mpiix.ko 2023-09-13T02:45:27.240 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_netcell.ko 2023-09-13T02:45:27.240 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ninja32.ko 2023-09-13T02:45:27.241 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87410.ko 2023-09-13T02:45:27.241 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87415.ko 2023-09-13T02:45:27.241 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_oldpiix.ko 2023-09-13T02:45:27.241 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_opti.ko 2023-09-13T02:45:27.241 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_optidma.ko 2023-09-13T02:45:27.242 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pcmcia.ko 2023-09-13T02:45:27.242 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc2027x.ko 2023-09-13T02:45:27.242 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-09-13T02:45:27.242 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_piccolo.ko 2023-09-13T02:45:27.242 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_rdc.ko 2023-09-13T02:45:27.243 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sch.ko 2023-09-13T02:45:27.243 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_serverworks.ko 2023-09-13T02:45:27.243 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sil680.ko 2023-09-13T02:45:27.243 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sis.ko 2023-09-13T02:45:27.243 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sl82c105.ko 2023-09-13T02:45:27.244 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_triflex.ko 2023-09-13T02:45:27.244 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_via.ko 2023-09-13T02:45:27.244 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pdc_adma.ko 2023-09-13T02:45:27.244 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_inic162x.ko 2023-09-13T02:45:27.244 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_mv.ko 2023-09-13T02:45:27.244 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_nv.ko 2023-09-13T02:45:27.245 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_promise.ko 2023-09-13T02:45:27.245 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_qstor.ko 2023-09-13T02:45:27.245 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil.ko 2023-09-13T02:45:27.245 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil24.ko 2023-09-13T02:45:27.245 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sis.ko 2023-09-13T02:45:27.246 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_svw.ko 2023-09-13T02:45:27.246 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sx4.ko 2023-09-13T02:45:27.246 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_uli.ko 2023-09-13T02:45:27.246 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_via.ko 2023-09-13T02:45:27.246 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_vsc.ko 2023-09-13T02:45:27.247 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay 2023-09-13T02:45:27.247 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay/ks0108.ko 2023-09-13T02:45:27.247 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base 2023-09-13T02:45:27.248 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap 2023-09-13T02:45:27.249 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-i2c.ko 2023-09-13T02:45:27.249 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-mmio.ko 2023-09-13T02:45:27.249 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma 2023-09-13T02:45:27.249 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma/bcma.ko 2023-09-13T02:45:27.249 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block 2023-09-13T02:45:27.250 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe 2023-09-13T02:45:27.250 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe/aoe.ko 2023-09-13T02:45:27.250 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/brd.ko 2023-09-13T02:45:27.250 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd 2023-09-13T02:45:27.251 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd/drbd.ko 2023-09-13T02:45:27.251 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/floppy.ko 2023-09-13T02:45:27.251 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/loop.ko 2023-09-13T02:45:27.251 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx 2023-09-13T02:45:27.251 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-09-13T02:45:27.251 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/nbd.ko 2023-09-13T02:45:27.252 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk 2023-09-13T02:45:27.252 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk/null_blk.ko 2023-09-13T02:45:27.252 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/pktcdvd.ko 2023-09-13T02:45:27.252 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/rbd.ko 2023-09-13T02:45:27.252 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/virtio_blk.ko 2023-09-13T02:45:27.253 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback 2023-09-13T02:45:27.253 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-09-13T02:45:27.253 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkfront.ko 2023-09-13T02:45:27.253 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram 2023-09-13T02:45:27.253 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram/zram.ko 2023-09-13T02:45:27.254 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char 2023-09-13T02:45:27.254 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hangcheck-timer.ko 2023-09-13T02:45:27.254 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random 2023-09-13T02:45:27.254 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/amd-rng.ko 2023-09-13T02:45:27.254 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/intel-rng.ko 2023-09-13T02:45:27.255 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-09-13T02:45:27.255 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/via-rng.ko 2023-09-13T02:45:27.255 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-09-13T02:45:27.255 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi 2023-09-13T02:45:27.255 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-09-13T02:45:27.256 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-09-13T02:45:27.256 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-09-13T02:45:27.256 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-09-13T02:45:27.256 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-09-13T02:45:27.256 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-09-13T02:45:27.257 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/lp.ko 2023-09-13T02:45:27.257 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave 2023-09-13T02:45:27.257 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave/mwave.ko 2023-09-13T02:45:27.257 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ppdev.ko 2023-09-13T02:45:27.257 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tlclk.ko 2023-09-13T02:45:27.258 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm 2023-09-13T02:45:27.258 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm.ko 2023-09-13T02:45:27.258 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-09-13T02:45:27.258 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-09-13T02:45:27.258 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-09-13T02:45:27.259 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis.ko 2023-09-13T02:45:27.259 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-09-13T02:45:27.259 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/uv_mmtimer.ko 2023-09-13T02:45:27.259 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/virtio_console.ko 2023-09-13T02:45:27.259 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq 2023-09-13T02:45:27.260 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-09-13T02:45:27.260 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-09-13T02:45:27.260 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-09-13T02:45:27.260 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-09-13T02:45:27.260 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/powernow-k8.ko 2023-09-13T02:45:27.261 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-09-13T02:45:27.261 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto 2023-09-13T02:45:27.261 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp 2023-09-13T02:45:27.261 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-09-13T02:45:27.261 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp.ko 2023-09-13T02:45:27.262 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel 2023-09-13T02:45:27.262 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat 2023-09-13T02:45:27.262 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common 2023-09-13T02:45:27.262 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-09-13T02:45:27.262 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-09-13T02:45:27.263 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-09-13T02:45:27.263 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-aes.ko 2023-09-13T02:45:27.263 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-sha.ko 2023-09-13T02:45:27.263 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio 2023-09-13T02:45:27.263 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-09-13T02:45:27.263 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca 2023-09-13T02:45:27.264 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca/dca.ko 2023-09-13T02:45:27.264 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq 2023-09-13T02:45:27.264 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-09-13T02:45:27.264 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma 2023-09-13T02:45:27.264 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw 2023-09-13T02:45:27.265 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw/dw_dmac.ko 2023-09-13T02:45:27.265 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat 2023-09-13T02:45:27.265 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat/ioatdma.ko 2023-09-13T02:45:27.265 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac 2023-09-13T02:45:27.265 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/amd64_edac.ko 2023-09-13T02:45:27.266 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/e752x_edac.ko 2023-09-13T02:45:27.266 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/edac_mce_amd.ko 2023-09-13T02:45:27.266 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3000_edac.ko 2023-09-13T02:45:27.266 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3200_edac.ko 2023-09-13T02:45:27.266 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5100_edac.ko 2023-09-13T02:45:27.267 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5400_edac.ko 2023-09-13T02:45:27.267 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7300_edac.ko 2023-09-13T02:45:27.267 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7core_edac.ko 2023-09-13T02:45:27.267 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i82975x_edac.ko 2023-09-13T02:45:27.267 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/ie31200_edac.ko 2023-09-13T02:45:27.268 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/sb_edac.ko 2023-09-13T02:45:27.268 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/x38_edac.ko 2023-09-13T02:45:27.268 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware 2023-09-13T02:45:27.268 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/edd.ko 2023-09-13T02:45:27.268 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/iscsi_ibft.ko 2023-09-13T02:45:27.269 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio 2023-09-13T02:45:27.269 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-ich.ko 2023-09-13T02:45:27.269 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-viperboard.ko 2023-09-13T02:45:27.269 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid 2023-09-13T02:45:27.269 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-appleir.ko 2023-09-13T02:45:27.270 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-aureal.ko 2023-09-13T02:45:27.270 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-axff.ko 2023-09-13T02:45:27.270 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-dr.ko 2023-09-13T02:45:27.270 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elecom.ko 2023-09-13T02:45:27.270 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elo.ko 2023-09-13T02:45:27.271 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-emsff.ko 2023-09-13T02:45:27.271 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gaff.ko 2023-09-13T02:45:27.271 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gt683r.ko 2023-09-13T02:45:27.271 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gyration.ko 2023-09-13T02:45:27.271 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-09-13T02:45:27.272 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-09-13T02:45:27.272 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtekff.ko 2023-09-13T02:45:27.272 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-hyperv.ko 2023-09-13T02:45:27.272 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-icade.ko 2023-09-13T02:45:27.272 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-keytouch.ko 2023-09-13T02:45:27.273 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-kye.ko 2023-09-13T02:45:27.273 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lcpower.ko 2023-09-13T02:45:27.273 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-led.ko 2023-09-13T02:45:27.273 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lenovo.ko 2023-09-13T02:45:27.274 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-dj.ko 2023-09-13T02:45:27.274 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-09-13T02:45:27.274 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-multitouch.ko 2023-09-13T02:45:27.274 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-ortek.ko 2023-09-13T02:45:27.274 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-penmount.ko 2023-09-13T02:45:27.274 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-petalynx.ko 2023-09-13T02:45:27.275 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-picolcd.ko 2023-09-13T02:45:27.275 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-pl.ko 2023-09-13T02:45:27.275 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-plantronics.ko 2023-09-13T02:45:27.275 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-primax.ko 2023-09-13T02:45:27.276 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-rmi.ko 2023-09-13T02:45:27.276 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-09-13T02:45:27.276 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-common.ko 2023-09-13T02:45:27.276 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-isku.ko 2023-09-13T02:45:27.276 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kone.ko 2023-09-13T02:45:27.276 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-09-13T02:45:27.277 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-09-13T02:45:27.277 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-09-13T02:45:27.277 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-lua.ko 2023-09-13T02:45:27.277 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-09-13T02:45:27.277 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-09-13T02:45:27.278 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-savu.ko 2023-09-13T02:45:27.278 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat.ko 2023-09-13T02:45:27.278 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-saitek.ko 2023-09-13T02:45:27.278 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-samsung.ko 2023-09-13T02:45:27.278 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sensor-hub.ko 2023-09-13T02:45:27.279 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sjoy.ko 2023-09-13T02:45:27.279 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sony.ko 2023-09-13T02:45:27.279 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-speedlink.ko 2023-09-13T02:45:27.279 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-steelseries.ko 2023-09-13T02:45:27.279 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sunplus.ko 2023-09-13T02:45:27.280 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-thrustmaster.ko 2023-09-13T02:45:27.280 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tivo.ko 2023-09-13T02:45:27.280 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tmff.ko 2023-09-13T02:45:27.280 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-topseed.ko 2023-09-13T02:45:27.280 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-twinhan.ko 2023-09-13T02:45:27.281 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-uclogic.ko 2023-09-13T02:45:27.281 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-waltop.ko 2023-09-13T02:45:27.281 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-wiimote.ko 2023-09-13T02:45:27.281 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-xinmo.ko 2023-09-13T02:45:27.281 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zpff.ko 2023-09-13T02:45:27.282 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zydacron.ko 2023-09-13T02:45:27.282 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/uhid.ko 2023-09-13T02:45:27.282 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/wacom.ko 2023-09-13T02:45:27.282 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv 2023-09-13T02:45:27.283 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_balloon.ko 2023-09-13T02:45:27.283 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_utils.ko 2023-09-13T02:45:27.283 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_vmbus.ko 2023-09-13T02:45:27.283 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon 2023-09-13T02:45:27.283 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru.ko 2023-09-13T02:45:27.284 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru3.ko 2023-09-13T02:45:27.284 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-09-13T02:45:27.284 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7414.ko 2023-09-13T02:45:27.284 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7418.ko 2023-09-13T02:45:27.284 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adc128d818.ko 2023-09-13T02:45:27.284 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1025.ko 2023-09-13T02:45:27.285 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1026.ko 2023-09-13T02:45:27.285 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1029.ko 2023-09-13T02:45:27.285 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1031.ko 2023-09-13T02:45:27.285 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm9240.ko 2023-09-13T02:45:27.285 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ads7828.ko 2023-09-13T02:45:27.286 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7410.ko 2023-09-13T02:45:27.286 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7411.ko 2023-09-13T02:45:27.286 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7462.ko 2023-09-13T02:45:27.286 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7470.ko 2023-09-13T02:45:27.286 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7475.ko 2023-09-13T02:45:27.287 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7x10.ko 2023-09-13T02:45:27.287 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/amc6821.ko 2023-09-13T02:45:27.287 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/applesmc.ko 2023-09-13T02:45:27.287 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asb100.ko 2023-09-13T02:45:27.287 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asc7621.ko 2023-09-13T02:45:27.288 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asus_atk0110.ko 2023-09-13T02:45:27.288 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/atxp1.ko 2023-09-13T02:45:27.288 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/coretemp.ko 2023-09-13T02:45:27.288 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-09-13T02:45:27.288 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dme1737.ko 2023-09-13T02:45:27.289 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds1621.ko 2023-09-13T02:45:27.289 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds620.ko 2023-09-13T02:45:27.289 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc1403.ko 2023-09-13T02:45:27.289 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc6w201.ko 2023-09-13T02:45:27.289 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71805f.ko 2023-09-13T02:45:27.290 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71882fg.ko 2023-09-13T02:45:27.290 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f75375s.ko 2023-09-13T02:45:27.290 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fam15h_power.ko 2023-09-13T02:45:27.290 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fschmd.ko 2023-09-13T02:45:27.290 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g760a.ko 2023-09-13T02:45:27.291 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g762.ko 2023-09-13T02:45:27.291 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl518sm.ko 2023-09-13T02:45:27.291 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl520sm.ko 2023-09-13T02:45:27.291 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/hwmon-vid.ko 2023-09-13T02:45:27.291 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5500_temp.ko 2023-09-13T02:45:27.292 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5k_amb.ko 2023-09-13T02:45:27.292 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmaem.ko 2023-09-13T02:45:27.292 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmpex.ko 2023-09-13T02:45:27.292 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina209.ko 2023-09-13T02:45:27.292 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina2xx.ko 2023-09-13T02:45:27.293 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/it87.ko 2023-09-13T02:45:27.293 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k10temp.ko 2023-09-13T02:45:27.293 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k8temp.ko 2023-09-13T02:45:27.293 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lineage-pem.ko 2023-09-13T02:45:27.293 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm63.ko 2023-09-13T02:45:27.293 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm73.ko 2023-09-13T02:45:27.294 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm75.ko 2023-09-13T02:45:27.294 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm77.ko 2023-09-13T02:45:27.294 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm78.ko 2023-09-13T02:45:27.294 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm80.ko 2023-09-13T02:45:27.294 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm83.ko 2023-09-13T02:45:27.295 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm85.ko 2023-09-13T02:45:27.295 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm87.ko 2023-09-13T02:45:27.295 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm90.ko 2023-09-13T02:45:27.295 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm92.ko 2023-09-13T02:45:27.295 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm93.ko 2023-09-13T02:45:27.296 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95234.ko 2023-09-13T02:45:27.296 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95241.ko 2023-09-13T02:45:27.296 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95245.ko 2023-09-13T02:45:27.296 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc2945.ko 2023-09-13T02:45:27.296 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4151.ko 2023-09-13T02:45:27.297 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4215.ko 2023-09-13T02:45:27.297 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4222.ko 2023-09-13T02:45:27.297 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4245.ko 2023-09-13T02:45:27.297 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4260.ko 2023-09-13T02:45:27.297 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4261.ko 2023-09-13T02:45:27.298 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max16065.ko 2023-09-13T02:45:27.298 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1619.ko 2023-09-13T02:45:27.298 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1668.ko 2023-09-13T02:45:27.298 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max197.ko 2023-09-13T02:45:27.298 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6639.ko 2023-09-13T02:45:27.299 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6650.ko 2023-09-13T02:45:27.299 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6697.ko 2023-09-13T02:45:27.299 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/mcp3021.ko 2023-09-13T02:45:27.299 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6683.ko 2023-09-13T02:45:27.299 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775-core.ko 2023-09-13T02:45:27.299 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775.ko 2023-09-13T02:45:27.300 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct7802.ko 2023-09-13T02:45:27.300 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87360.ko 2023-09-13T02:45:27.300 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87427.ko 2023-09-13T02:45:27.300 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pcf8591.ko 2023-09-13T02:45:27.300 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus 2023-09-13T02:45:27.301 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-09-13T02:45:27.301 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-09-13T02:45:27.301 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-09-13T02:45:27.301 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-09-13T02:45:27.301 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-09-13T02:45:27.302 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-09-13T02:45:27.302 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-09-13T02:45:27.302 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-09-13T02:45:27.302 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-09-13T02:45:27.302 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-09-13T02:45:27.303 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-09-13T02:45:27.303 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-09-13T02:45:27.303 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/powr1220.ko 2023-09-13T02:45:27.303 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5627.ko 2023-09-13T02:45:27.303 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5636.ko 2023-09-13T02:45:27.304 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch56xx-common.ko 2023-09-13T02:45:27.304 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht15.ko 2023-09-13T02:45:27.304 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht21.ko 2023-09-13T02:45:27.304 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/shtc1.ko 2023-09-13T02:45:27.304 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sis5595.ko 2023-09-13T02:45:27.305 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47b397.ko 2023-09-13T02:45:27.305 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m1.ko 2023-09-13T02:45:27.305 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m192.ko 2023-09-13T02:45:27.305 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/thmc50.ko 2023-09-13T02:45:27.305 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp102.ko 2023-09-13T02:45:27.305 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp103.ko 2023-09-13T02:45:27.306 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp401.ko 2023-09-13T02:45:27.306 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp421.ko 2023-09-13T02:45:27.306 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via-cputemp.ko 2023-09-13T02:45:27.306 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via686a.ko 2023-09-13T02:45:27.306 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt1211.ko 2023-09-13T02:45:27.307 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt8231.ko 2023-09-13T02:45:27.307 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627ehf.ko 2023-09-13T02:45:27.307 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627hf.ko 2023-09-13T02:45:27.307 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83781d.ko 2023-09-13T02:45:27.307 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83791d.ko 2023-09-13T02:45:27.308 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83792d.ko 2023-09-13T02:45:27.308 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83793.ko 2023-09-13T02:45:27.308 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83795.ko 2023-09-13T02:45:27.308 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l785ts.ko 2023-09-13T02:45:27.308 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l786ng.ko 2023-09-13T02:45:27.309 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c 2023-09-13T02:45:27.309 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos 2023-09-13T02:45:27.309 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-09-13T02:45:27.309 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-09-13T02:45:27.309 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses 2023-09-13T02:45:27.310 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-09-13T02:45:27.310 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-09-13T02:45:27.310 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-09-13T02:45:27.310 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-09-13T02:45:27.311 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-09-13T02:45:27.311 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-09-13T02:45:27.311 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-09-13T02:45:27.311 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-09-13T02:45:27.311 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-09-13T02:45:27.312 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-09-13T02:45:27.312 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-09-13T02:45:27.312 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-09-13T02:45:27.312 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-09-13T02:45:27.312 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-09-13T02:45:27.313 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-09-13T02:45:27.313 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-09-13T02:45:27.313 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-09-13T02:45:27.313 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-09-13T02:45:27.313 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-09-13T02:45:27.314 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-09-13T02:45:27.314 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-via.ko 2023-09-13T02:45:27.314 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-09-13T02:45:27.314 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-09-13T02:45:27.314 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-dev.ko 2023-09-13T02:45:27.315 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-mux.ko 2023-09-13T02:45:27.315 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-09-13T02:45:27.315 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-smbus.ko 2023-09-13T02:45:27.315 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-stub.ko 2023-09-13T02:45:27.315 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband 2023-09-13T02:45:27.316 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core 2023-09-13T02:45:27.316 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_cm.ko 2023-09-13T02:45:27.316 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_core.ko 2023-09-13T02:45:27.316 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_umad.ko 2023-09-13T02:45:27.316 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-09-13T02:45:27.317 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/iw_cm.ko 2023-09-13T02:45:27.317 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-09-13T02:45:27.317 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-09-13T02:45:27.317 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw 2023-09-13T02:45:27.317 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4 2023-09-13T02:45:27.318 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-09-13T02:45:27.318 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4 2023-09-13T02:45:27.318 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-09-13T02:45:27.318 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5 2023-09-13T02:45:27.318 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-09-13T02:45:27.318 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca 2023-09-13T02:45:27.319 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-09-13T02:45:27.319 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp 2023-09-13T02:45:27.319 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib 2023-09-13T02:45:27.319 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-09-13T02:45:27.319 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser 2023-09-13T02:45:27.320 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-09-13T02:45:27.320 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert 2023-09-13T02:45:27.320 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-09-13T02:45:27.320 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp 2023-09-13T02:45:27.320 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-09-13T02:45:27.321 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt 2023-09-13T02:45:27.321 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-09-13T02:45:27.321 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input 2023-09-13T02:45:27.321 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport 2023-09-13T02:45:27.321 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-09-13T02:45:27.322 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/fm801-gp.ko 2023-09-13T02:45:27.322 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/gameport.ko 2023-09-13T02:45:27.322 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/lightning.ko 2023-09-13T02:45:27.322 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/ns558.ko 2023-09-13T02:45:27.323 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/joydev.ko 2023-09-13T02:45:27.323 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard 2023-09-13T02:45:27.323 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-09-13T02:45:27.323 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse 2023-09-13T02:45:27.323 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/appletouch.ko 2023-09-13T02:45:27.324 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/bcm5974.ko 2023-09-13T02:45:27.324 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/cyapatp.ko 2023-09-13T02:45:27.324 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/elan_i2c.ko 2023-09-13T02:45:27.324 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/sermouse.ko 2023-09-13T02:45:27.324 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-09-13T02:45:27.325 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-09-13T02:45:27.325 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-09-13T02:45:27.325 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4 2023-09-13T02:45:27.325 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4/rmi_core.ko 2023-09-13T02:45:27.325 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio 2023-09-13T02:45:27.326 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/altera_ps2.ko 2023-09-13T02:45:27.326 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/arc_ps2.ko 2023-09-13T02:45:27.326 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-09-13T02:45:27.326 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/serio_raw.ko 2023-09-13T02:45:27.326 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/sparse-keymap.ko 2023-09-13T02:45:27.327 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu 2023-09-13T02:45:27.327 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd 2023-09-13T02:45:27.327 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-09-13T02:45:27.327 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds 2023-09-13T02:45:27.327 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-blinkm.ko 2023-09-13T02:45:27.328 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lm3530.ko 2023-09-13T02:45:27.328 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lp3944.ko 2023-09-13T02:45:27.328 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-ss4200.ko 2023-09-13T02:45:27.328 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger 2023-09-13T02:45:27.328 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-09-13T02:45:27.328 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-09-13T02:45:27.329 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-09-13T02:45:27.329 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-09-13T02:45:27.329 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-09-13T02:45:27.329 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-09-13T02:45:27.329 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-09-13T02:45:27.330 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-09-13T02:45:27.330 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md 2023-09-13T02:45:27.330 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache 2023-09-13T02:45:27.330 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache/bcache.ko 2023-09-13T02:45:27.330 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-bio-prison.ko 2023-09-13T02:45:27.331 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache-smq.ko 2023-09-13T02:45:27.331 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache.ko 2023-09-13T02:45:27.331 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-crypt.ko 2023-09-13T02:45:27.331 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-delay.ko 2023-09-13T02:45:27.331 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-flakey.ko 2023-09-13T02:45:27.332 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-log-userspace.ko 2023-09-13T02:45:27.332 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-multipath.ko 2023-09-13T02:45:27.332 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-queue-length.ko 2023-09-13T02:45:27.332 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-raid.ko 2023-09-13T02:45:27.332 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-round-robin.ko 2023-09-13T02:45:27.333 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-service-time.ko 2023-09-13T02:45:27.333 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-switch.ko 2023-09-13T02:45:27.333 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-thin-pool.ko 2023-09-13T02:45:27.333 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-verity.ko 2023-09-13T02:45:27.333 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/faulty.ko 2023-09-13T02:45:27.334 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/linear.ko 2023-09-13T02:45:27.334 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/multipath.ko 2023-09-13T02:45:27.334 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data 2023-09-13T02:45:27.334 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-09-13T02:45:27.334 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid0.ko 2023-09-13T02:45:27.335 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid1.ko 2023-09-13T02:45:27.335 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid10.ko 2023-09-13T02:45:27.335 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid456.ko 2023-09-13T02:45:27.335 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message 2023-09-13T02:45:27.335 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion 2023-09-13T02:45:27.336 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptbase.ko 2023-09-13T02:45:27.336 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptctl.ko 2023-09-13T02:45:27.336 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptfc.ko 2023-09-13T02:45:27.336 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptlan.ko 2023-09-13T02:45:27.336 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptsas.ko 2023-09-13T02:45:27.337 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptscsih.ko 2023-09-13T02:45:27.337 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptspi.ko 2023-09-13T02:45:27.337 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd 2023-09-13T02:45:27.337 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_ich.ko 2023-09-13T02:45:27.337 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_sch.ko 2023-09-13T02:45:27.337 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/mfd-core.ko 2023-09-13T02:45:27.338 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/sm501.ko 2023-09-13T02:45:27.338 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/viperboard.ko 2023-09-13T02:45:27.338 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/vx855.ko 2023-09-13T02:45:27.338 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/wl1273-core.ko 2023-09-13T02:45:27.338 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc 2023-09-13T02:45:27.339 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl 2023-09-13T02:45:27.339 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-09-13T02:45:27.339 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds9802als.ko 2023-09-13T02:45:27.339 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds990x.ko 2023-09-13T02:45:27.339 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/bh1770glc.ko 2023-09-13T02:45:27.340 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710 2023-09-13T02:45:27.340 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710/cb710.ko 2023-09-13T02:45:27.340 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom 2023-09-13T02:45:27.340 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/at24.ko 2023-09-13T02:45:27.340 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom.ko 2023-09-13T02:45:27.341 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-09-13T02:45:27.341 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/max6875.ko 2023-09-13T02:45:27.341 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/enclosure.ko 2023-09-13T02:45:27.341 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/hpilo.ko 2023-09-13T02:45:27.341 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29003.ko 2023-09-13T02:45:27.342 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29020.ko 2023-09-13T02:45:27.342 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d 2023-09-13T02:45:27.342 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-09-13T02:45:27.342 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-09-13T02:45:27.342 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei 2023-09-13T02:45:27.343 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-me.ko 2023-09-13T02:45:27.343 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-txe.ko 2023-09-13T02:45:27.343 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei.ko 2023-09-13T02:45:27.343 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru 2023-09-13T02:45:27.343 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru/gru.ko 2023-09-13T02:45:27.344 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp 2023-09-13T02:45:27.344 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xp.ko 2023-09-13T02:45:27.344 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpc.ko 2023-09-13T02:45:27.344 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-09-13T02:45:27.344 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_7xx1.ko 2023-09-13T02:45:27.345 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_core.ko 2023-09-13T02:45:27.345 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tsl2550.ko 2023-09-13T02:45:27.345 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_balloon.ko 2023-09-13T02:45:27.345 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci 2023-09-13T02:45:27.345 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-09-13T02:45:27.345 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net 2023-09-13T02:45:27.346 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding 2023-09-13T02:45:27.346 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding/bonding.ko 2023-09-13T02:45:27.346 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa 2023-09-13T02:45:27.346 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53 2023-09-13T02:45:27.346 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-09-13T02:45:27.347 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-09-13T02:45:27.347 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6060.ko 2023-09-13T02:45:27.347 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx 2023-09-13T02:45:27.347 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-09-13T02:45:27.347 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dummy.ko 2023-09-13T02:45:27.348 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/eql.ko 2023-09-13T02:45:27.348 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet 2023-09-13T02:45:27.348 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com 2023-09-13T02:45:27.348 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-09-13T02:45:27.348 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-09-13T02:45:27.349 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-09-13T02:45:27.349 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-09-13T02:45:27.349 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390 2023-09-13T02:45:27.349 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/8390.ko 2023-09-13T02:45:27.349 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-09-13T02:45:27.350 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-09-13T02:45:27.350 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-09-13T02:45:27.350 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec 2023-09-13T02:45:27.350 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-09-13T02:45:27.350 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere 2023-09-13T02:45:27.351 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-09-13T02:45:27.351 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon 2023-09-13T02:45:27.351 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-09-13T02:45:27.351 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera 2023-09-13T02:45:27.351 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-09-13T02:45:27.351 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd 2023-09-13T02:45:27.352 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-09-13T02:45:27.352 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-09-13T02:45:27.352 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-09-13T02:45:27.352 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros 2023-09-13T02:45:27.352 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx 2023-09-13T02:45:27.353 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-09-13T02:45:27.353 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c 2023-09-13T02:45:27.353 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-09-13T02:45:27.353 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e 2023-09-13T02:45:27.353 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-09-13T02:45:27.354 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx 2023-09-13T02:45:27.354 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-09-13T02:45:27.354 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-09-13T02:45:27.354 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom 2023-09-13T02:45:27.354 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-09-13T02:45:27.355 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-09-13T02:45:27.355 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-09-13T02:45:27.355 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-09-13T02:45:27.355 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-09-13T02:45:27.355 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet 2023-09-13T02:45:27.356 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-09-13T02:45:27.356 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-09-13T02:45:27.356 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade 2023-09-13T02:45:27.356 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna 2023-09-13T02:45:27.356 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-09-13T02:45:27.357 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio 2023-09-13T02:45:27.357 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb 2023-09-13T02:45:27.357 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-09-13T02:45:27.357 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-09-13T02:45:27.357 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-09-13T02:45:27.357 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-09-13T02:45:27.358 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-09-13T02:45:27.358 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-09-13T02:45:27.358 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-09-13T02:45:27.358 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-09-13T02:45:27.358 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-09-13T02:45:27.359 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco 2023-09-13T02:45:27.359 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic 2023-09-13T02:45:27.359 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-09-13T02:45:27.359 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec 2023-09-13T02:45:27.359 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip 2023-09-13T02:45:27.360 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-09-13T02:45:27.360 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-09-13T02:45:27.360 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-09-13T02:45:27.360 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-09-13T02:45:27.360 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-09-13T02:45:27.361 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-09-13T02:45:27.361 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink 2023-09-13T02:45:27.361 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-09-13T02:45:27.361 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-09-13T02:45:27.361 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dnet.ko 2023-09-13T02:45:27.362 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex 2023-09-13T02:45:27.362 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet 2023-09-13T02:45:27.362 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-09-13T02:45:27.362 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ethoc.ko 2023-09-13T02:45:27.362 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/fealnx.ko 2023-09-13T02:45:27.363 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel 2023-09-13T02:45:27.363 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e100.ko 2023-09-13T02:45:27.363 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000 2023-09-13T02:45:27.363 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-09-13T02:45:27.363 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e 2023-09-13T02:45:27.363 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-09-13T02:45:27.364 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k 2023-09-13T02:45:27.364 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-09-13T02:45:27.364 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e 2023-09-13T02:45:27.364 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-09-13T02:45:27.364 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf 2023-09-13T02:45:27.365 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-09-13T02:45:27.365 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb 2023-09-13T02:45:27.365 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-09-13T02:45:27.365 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf 2023-09-13T02:45:27.365 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-09-13T02:45:27.366 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe 2023-09-13T02:45:27.366 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-09-13T02:45:27.366 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf 2023-09-13T02:45:27.366 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-09-13T02:45:27.366 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/jme.ko 2023-09-13T02:45:27.367 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell 2023-09-13T02:45:27.367 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-09-13T02:45:27.367 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-09-13T02:45:27.367 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-09-13T02:45:27.367 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox 2023-09-13T02:45:27.368 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4 2023-09-13T02:45:27.368 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-09-13T02:45:27.368 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-09-13T02:45:27.368 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5 2023-09-13T02:45:27.368 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-09-13T02:45:27.369 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-09-13T02:45:27.369 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel 2023-09-13T02:45:27.369 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-09-13T02:45:27.369 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom 2023-09-13T02:45:27.369 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge 2023-09-13T02:45:27.370 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-09-13T02:45:27.370 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi 2023-09-13T02:45:27.370 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-09-13T02:45:27.370 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-09-13T02:45:27.370 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion 2023-09-13T02:45:27.371 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-09-13T02:45:27.371 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia 2023-09-13T02:45:27.371 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-09-13T02:45:27.371 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines 2023-09-13T02:45:27.371 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-09-13T02:45:27.371 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-09-13T02:45:27.372 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic 2023-09-13T02:45:27.372 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen 2023-09-13T02:45:27.372 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-09-13T02:45:27.372 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-09-13T02:45:27.372 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-09-13T02:45:27.373 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-09-13T02:45:27.373 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc 2023-09-13T02:45:27.373 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-09-13T02:45:27.373 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek 2023-09-13T02:45:27.373 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-09-13T02:45:27.374 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-09-13T02:45:27.374 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-09-13T02:45:27.374 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-09-13T02:45:27.374 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker 2023-09-13T02:45:27.374 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-09-13T02:45:27.375 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc 2023-09-13T02:45:27.375 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-09-13T02:45:27.375 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan 2023-09-13T02:45:27.375 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-09-13T02:45:27.375 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis 2023-09-13T02:45:27.376 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-09-13T02:45:27.376 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-09-13T02:45:27.376 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc 2023-09-13T02:45:27.376 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-09-13T02:45:27.376 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-09-13T02:45:27.376 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-09-13T02:45:27.377 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-09-13T02:45:27.377 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro 2023-09-13T02:45:27.377 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac 2023-09-13T02:45:27.377 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-09-13T02:45:27.377 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-09-13T02:45:27.378 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun 2023-09-13T02:45:27.378 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-09-13T02:45:27.378 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/niu.ko 2023-09-13T02:45:27.378 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-09-13T02:45:27.378 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-09-13T02:45:27.379 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti 2023-09-13T02:45:27.379 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-09-13T02:45:27.379 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti 2023-09-13T02:45:27.379 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-09-13T02:45:27.379 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via 2023-09-13T02:45:27.380 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-09-13T02:45:27.380 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-09-13T02:45:27.380 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet 2023-09-13T02:45:27.380 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-09-13T02:45:27.380 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-09-13T02:45:27.381 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom 2023-09-13T02:45:27.381 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-09-13T02:45:27.381 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/geneve.ko 2023-09-13T02:45:27.381 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv 2023-09-13T02:45:27.381 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-09-13T02:45:27.382 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154 2023-09-13T02:45:27.382 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154/fakelb.ko 2023-09-13T02:45:27.382 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ifb.ko 2023-09-13T02:45:27.382 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan 2023-09-13T02:45:27.382 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-09-13T02:45:27.382 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvlan.ko 2023-09-13T02:45:27.383 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvtap.ko 2023-09-13T02:45:27.383 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio 2023-09-13T02:45:27.383 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio.ko 2023-09-13T02:45:27.383 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-09-13T02:45:27.383 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-09-13T02:45:27.384 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-09-13T02:45:27.384 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mii.ko 2023-09-13T02:45:27.384 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/net_failover.ko 2023-09-13T02:45:27.384 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/netconsole.ko 2023-09-13T02:45:27.384 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/nlmon.ko 2023-09-13T02:45:27.385 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs 2023-09-13T02:45:27.385 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-09-13T02:45:27.385 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-09-13T02:45:27.385 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy 2023-09-13T02:45:27.385 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/amd.ko 2023-09-13T02:45:27.386 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-09-13T02:45:27.386 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-09-13T02:45:27.386 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm7xxx.ko 2023-09-13T02:45:27.386 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm87xx.ko 2023-09-13T02:45:27.386 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/broadcom.ko 2023-09-13T02:45:27.387 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/cicada.ko 2023-09-13T02:45:27.387 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/davicom.ko 2023-09-13T02:45:27.387 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/dp83640.ko 2023-09-13T02:45:27.387 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/et1011c.ko 2023-09-13T02:45:27.387 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/icplus.ko 2023-09-13T02:45:27.388 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/lxt.ko 2023-09-13T02:45:27.388 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/marvell.ko 2023-09-13T02:45:27.388 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/micrel.ko 2023-09-13T02:45:27.388 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/national.ko 2023-09-13T02:45:27.388 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/phylink.ko 2023-09-13T02:45:27.388 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/qsemi.ko 2023-09-13T02:45:27.389 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/realtek.ko 2023-09-13T02:45:27.389 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/smsc.ko 2023-09-13T02:45:27.389 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/ste10Xp.ko 2023-09-13T02:45:27.389 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/vitesse.ko 2023-09-13T02:45:27.389 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp 2023-09-13T02:45:27.390 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/bsd_comp.ko 2023-09-13T02:45:27.390 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_async.ko 2023-09-13T02:45:27.390 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-09-13T02:45:27.390 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_generic.ko 2023-09-13T02:45:27.390 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-09-13T02:45:27.391 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-09-13T02:45:27.391 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppoe.ko 2023-09-13T02:45:27.391 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppox.ko 2023-09-13T02:45:27.391 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pptp.ko 2023-09-13T02:45:27.391 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip 2023-09-13T02:45:27.392 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slhc.ko 2023-09-13T02:45:27.392 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slip.ko 2023-09-13T02:45:27.392 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/sungem_phy.ko 2023-09-13T02:45:27.392 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tap.ko 2023-09-13T02:45:27.392 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team 2023-09-13T02:45:27.393 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team.ko 2023-09-13T02:45:27.393 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-09-13T02:45:27.393 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-09-13T02:45:27.393 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-09-13T02:45:27.393 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_random.ko 2023-09-13T02:45:27.393 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-09-13T02:45:27.394 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tun.ko 2023-09-13T02:45:27.394 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/veth.ko 2023-09-13T02:45:27.394 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/virtio_net.ko 2023-09-13T02:45:27.394 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3 2023-09-13T02:45:27.394 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-09-13T02:45:27.395 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan 2023-09-13T02:45:27.395 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan/vxlan.ko 2023-09-13T02:45:27.395 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback 2023-09-13T02:45:27.395 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-09-13T02:45:27.395 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netfront.ko 2023-09-13T02:45:27.396 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb 2023-09-13T02:45:27.396 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb/ntb.ko 2023-09-13T02:45:27.396 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme 2023-09-13T02:45:27.396 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host 2023-09-13T02:45:27.396 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme-core.ko 2023-09-13T02:45:27.397 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme.ko 2023-09-13T02:45:27.397 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport 2023-09-13T02:45:27.397 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport.ko 2023-09-13T02:45:27.397 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_cs.ko 2023-09-13T02:45:27.397 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_pc.ko 2023-09-13T02:45:27.398 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_serial.ko 2023-09-13T02:45:27.398 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci 2023-09-13T02:45:27.398 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug 2023-09-13T02:45:27.398 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-09-13T02:45:27.398 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie 2023-09-13T02:45:27.399 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie/aer_inject.ko 2023-09-13T02:45:27.399 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/xen-pcifront.ko 2023-09-13T02:45:27.399 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia 2023-09-13T02:45:27.399 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/i82092.ko 2023-09-13T02:45:27.399 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/pd6729.ko 2023-09-13T02:45:27.399 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/yenta_socket.ko 2023-09-13T02:45:27.400 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl 2023-09-13T02:45:27.400 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel 2023-09-13T02:45:27.400 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-09-13T02:45:27.400 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform 2023-09-13T02:45:27.400 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86 2023-09-13T02:45:27.401 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acer-wmi.ko 2023-09-13T02:45:27.401 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acerhdf.ko 2023-09-13T02:45:27.401 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/apple-gmux.ko 2023-09-13T02:45:27.401 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-laptop.ko 2023-09-13T02:45:27.401 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-09-13T02:45:27.402 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-wmi.ko 2023-09-13T02:45:27.402 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-09-13T02:45:27.402 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-09-13T02:45:27.402 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-09-13T02:45:27.402 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-09-13T02:45:27.402 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-09-13T02:45:27.403 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/hdaps.ko 2023-09-13T02:45:27.403 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel 2023-09-13T02:45:27.403 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-09-13T02:45:27.403 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel_ips.ko 2023-09-13T02:45:27.403 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/msi-wmi.ko 2023-09-13T02:45:27.404 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-09-13T02:45:27.404 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-09-13T02:45:27.404 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-09-13T02:45:27.404 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-q10.ko 2023-09-13T02:45:27.404 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-09-13T02:45:27.405 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-09-13T02:45:27.405 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-09-13T02:45:27.405 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-09-13T02:45:27.405 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi.ko 2023-09-13T02:45:27.405 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap 2023-09-13T02:45:27.405 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_common.ko 2023-09-13T02:45:27.406 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-09-13T02:45:27.406 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps 2023-09-13T02:45:27.406 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients 2023-09-13T02:45:27.406 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-gpio.ko 2023-09-13T02:45:27.406 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-09-13T02:45:27.407 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps_parport.ko 2023-09-13T02:45:27.407 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/pps_core.ko 2023-09-13T02:45:27.407 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp 2023-09-13T02:45:27.407 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp.ko 2023-09-13T02:45:27.407 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp_kvm.ko 2023-09-13T02:45:27.408 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc 2023-09-13T02:45:27.408 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq32k.ko 2023-09-13T02:45:27.408 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq4802.ko 2023-09-13T02:45:27.408 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1286.ko 2023-09-13T02:45:27.408 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1307.ko 2023-09-13T02:45:27.408 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1374.ko 2023-09-13T02:45:27.409 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1511.ko 2023-09-13T02:45:27.409 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1553.ko 2023-09-13T02:45:27.409 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1672.ko 2023-09-13T02:45:27.409 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1742.ko 2023-09-13T02:45:27.409 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds2404.ko 2023-09-13T02:45:27.410 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds3232.ko 2023-09-13T02:45:27.410 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-em3027.ko 2023-09-13T02:45:27.410 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-fm3130.ko 2023-09-13T02:45:27.410 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl12022.ko 2023-09-13T02:45:27.410 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl1208.ko 2023-09-13T02:45:27.411 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m41t80.ko 2023-09-13T02:45:27.411 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t35.ko 2023-09-13T02:45:27.411 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t59.ko 2023-09-13T02:45:27.411 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max6900.ko 2023-09-13T02:45:27.411 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-msm6242.ko 2023-09-13T02:45:27.411 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-09-13T02:45:27.412 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-09-13T02:45:27.412 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-09-13T02:45:27.412 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-09-13T02:45:27.412 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-09-13T02:45:27.412 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-09-13T02:45:27.413 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-09-13T02:45:27.413 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-09-13T02:45:27.413 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8025.ko 2023-09-13T02:45:27.413 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8581.ko 2023-09-13T02:45:27.413 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-09-13T02:45:27.414 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-x1205.ko 2023-09-13T02:45:27.414 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi 2023-09-13T02:45:27.414 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-9xxx.ko 2023-09-13T02:45:27.414 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-sas.ko 2023-09-13T02:45:27.414 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-xxxx.ko 2023-09-13T02:45:27.415 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/BusLogic.ko 2023-09-13T02:45:27.415 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/a100u2w.ko 2023-09-13T02:45:27.415 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid 2023-09-13T02:45:27.415 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-09-13T02:45:27.415 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/advansys.ko 2023-09-13T02:45:27.415 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx 2023-09-13T02:45:27.416 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-09-13T02:45:27.416 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-09-13T02:45:27.416 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/am53c974.ko 2023-09-13T02:45:27.416 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr 2023-09-13T02:45:27.416 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-09-13T02:45:27.417 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/atp870u.ko 2023-09-13T02:45:27.417 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi 2023-09-13T02:45:27.417 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-09-13T02:45:27.417 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa 2023-09-13T02:45:27.417 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa/bfa.ko 2023-09-13T02:45:27.418 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc 2023-09-13T02:45:27.418 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-09-13T02:45:27.418 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i 2023-09-13T02:45:27.418 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-09-13T02:45:27.418 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ch.ko 2023-09-13T02:45:27.418 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor 2023-09-13T02:45:27.419 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-09-13T02:45:27.419 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi 2023-09-13T02:45:27.419 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i 2023-09-13T02:45:27.419 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-09-13T02:45:27.419 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i 2023-09-13T02:45:27.420 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-09-13T02:45:27.420 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-09-13T02:45:27.420 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dc395x.ko 2023-09-13T02:45:27.420 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler 2023-09-13T02:45:27.420 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-09-13T02:45:27.421 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-09-13T02:45:27.421 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-09-13T02:45:27.421 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-09-13T02:45:27.421 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dmx3191d.ko 2023-09-13T02:45:27.421 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r 2023-09-13T02:45:27.422 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-09-13T02:45:27.422 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esp_scsi.ko 2023-09-13T02:45:27.422 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe 2023-09-13T02:45:27.422 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-09-13T02:45:27.422 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-09-13T02:45:27.423 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic 2023-09-13T02:45:27.423 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic/fnic.ko 2023-09-13T02:45:27.423 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hpsa.ko 2023-09-13T02:45:27.423 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hptiop.ko 2023-09-13T02:45:27.423 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hv_storvsc.ko 2023-09-13T02:45:27.423 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/initio.ko 2023-09-13T02:45:27.424 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ipr.ko 2023-09-13T02:45:27.424 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ips.ko 2023-09-13T02:45:27.424 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci 2023-09-13T02:45:27.424 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci/isci.ko 2023-09-13T02:45:27.424 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-09-13T02:45:27.425 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_tcp.ko 2023-09-13T02:45:27.425 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc 2023-09-13T02:45:27.425 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc/libfc.ko 2023-09-13T02:45:27.425 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi.ko 2023-09-13T02:45:27.425 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-09-13T02:45:27.425 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas 2023-09-13T02:45:27.426 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas/libsas.ko 2023-09-13T02:45:27.426 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc 2023-09-13T02:45:27.426 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-09-13T02:45:27.426 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid 2023-09-13T02:45:27.426 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid.ko 2023-09-13T02:45:27.427 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-09-13T02:45:27.427 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-09-13T02:45:27.427 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-09-13T02:45:27.427 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas 2023-09-13T02:45:27.427 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-09-13T02:45:27.428 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas 2023-09-13T02:45:27.428 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-09-13T02:45:27.428 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvumi.ko 2023-09-13T02:45:27.428 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001 2023-09-13T02:45:27.428 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-09-13T02:45:27.429 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pmcraid.ko 2023-09-13T02:45:27.429 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla1280.ko 2023-09-13T02:45:27.429 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx 2023-09-13T02:45:27.429 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-09-13T02:45:27.429 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-09-13T02:45:27.429 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx 2023-09-13T02:45:27.430 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-09-13T02:45:27.430 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/raid_class.ko 2023-09-13T02:45:27.430 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_debug.ko 2023-09-13T02:45:27.430 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-09-13T02:45:27.430 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-09-13T02:45:27.431 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-09-13T02:45:27.431 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-09-13T02:45:27.431 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-09-13T02:45:27.431 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ses.ko 2023-09-13T02:45:27.431 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/st.ko 2023-09-13T02:45:27.432 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/stex.ko 2023-09-13T02:45:27.432 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2 2023-09-13T02:45:27.432 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-09-13T02:45:27.432 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/virtio_scsi.ko 2023-09-13T02:45:27.432 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-09-13T02:45:27.433 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/wd719x.ko 2023-09-13T02:45:27.433 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb 2023-09-13T02:45:27.433 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb/ssb.ko 2023-09-13T02:45:27.433 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target 2023-09-13T02:45:27.433 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi 2023-09-13T02:45:27.433 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-09-13T02:45:27.434 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback 2023-09-13T02:45:27.434 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback/tcm_loop.ko 2023-09-13T02:45:27.434 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_file.ko 2023-09-13T02:45:27.434 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_iblock.ko 2023-09-13T02:45:27.434 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_mod.ko 2023-09-13T02:45:27.435 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_pscsi.ko 2023-09-13T02:45:27.435 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_user.ko 2023-09-13T02:45:27.435 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc 2023-09-13T02:45:27.435 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-09-13T02:45:27.435 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal 2023-09-13T02:45:27.436 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel 2023-09-13T02:45:27.436 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal 2023-09-13T02:45:27.436 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-09-13T02:45:27.436 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-09-13T02:45:27.436 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-09-13T02:45:27.437 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-09-13T02:45:27.437 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-09-13T02:45:27.437 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-09-13T02:45:27.437 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-09-13T02:45:27.437 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-09-13T02:45:27.438 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-09-13T02:45:27.438 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-09-13T02:45:27.438 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-09-13T02:45:27.438 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-09-13T02:45:27.438 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-09-13T02:45:27.439 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-09-13T02:45:27.439 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-09-13T02:45:27.439 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty 2023-09-13T02:45:27.439 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless 2023-09-13T02:45:27.439 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-09-13T02:45:27.440 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_gsm.ko 2023-09-13T02:45:27.440 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_hdlc.ko 2023-09-13T02:45:27.440 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/nozomi.ko 2023-09-13T02:45:27.440 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial 2023-09-13T02:45:27.440 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250 2023-09-13T02:45:27.441 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-09-13T02:45:27.441 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/arc_uart.ko 2023-09-13T02:45:27.441 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm 2023-09-13T02:45:27.441 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-09-13T02:45:27.441 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/synclink_gt.ko 2023-09-13T02:45:27.442 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs 2023-09-13T02:45:27.442 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core 2023-09-13T02:45:27.442 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-09-13T02:45:27.442 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host 2023-09-13T02:45:27.442 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-09-13T02:45:27.443 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio 2023-09-13T02:45:27.443 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio.ko 2023-09-13T02:45:27.443 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_aec.ko 2023-09-13T02:45:27.443 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_cif.ko 2023-09-13T02:45:27.443 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_pci_generic.ko 2023-09-13T02:45:27.443 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_sercos3.ko 2023-09-13T02:45:27.444 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb 2023-09-13T02:45:27.444 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class 2023-09-13T02:45:27.444 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-acm.ko 2023-09-13T02:45:27.444 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-wdm.ko 2023-09-13T02:45:27.444 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usblp.ko 2023-09-13T02:45:27.445 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usbtmc.ko 2023-09-13T02:45:27.445 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host 2023-09-13T02:45:27.445 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/sl811-hcd.ko 2023-09-13T02:45:27.445 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image 2023-09-13T02:45:27.445 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/mdc800.ko 2023-09-13T02:45:27.446 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/microtek.ko 2023-09-13T02:45:27.446 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc 2023-09-13T02:45:27.446 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/adutux.ko 2023-09-13T02:45:27.446 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/appledisplay.ko 2023-09-13T02:45:27.446 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi26.ko 2023-09-13T02:45:27.446 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi62.ko 2023-09-13T02:45:27.447 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ezusb.ko 2023-09-13T02:45:27.447 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/idmouse.ko 2023-09-13T02:45:27.447 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/iowarrior.ko 2023-09-13T02:45:27.447 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/isight_firmware.ko 2023-09-13T02:45:27.447 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ldusb.ko 2023-09-13T02:45:27.448 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/legousbtower.ko 2023-09-13T02:45:27.448 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga 2023-09-13T02:45:27.448 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-09-13T02:45:27.448 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/trancevibrator.ko 2023-09-13T02:45:27.448 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usb3503.ko 2023-09-13T02:45:27.449 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usblcd.ko 2023-09-13T02:45:27.449 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usbsevseg.ko 2023-09-13T02:45:27.449 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/uss720.ko 2023-09-13T02:45:27.449 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/yurex.ko 2023-09-13T02:45:27.449 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy 2023-09-13T02:45:27.449 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/phy-generic.ko 2023-09-13T02:45:27.450 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial 2023-09-13T02:45:27.450 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/aircable.ko 2023-09-13T02:45:27.450 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ark3116.ko 2023-09-13T02:45:27.450 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/belkin_sa.ko 2023-09-13T02:45:27.450 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ch341.ko 2023-09-13T02:45:27.451 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cp210x.ko 2023-09-13T02:45:27.451 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cyberjack.ko 2023-09-13T02:45:27.451 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cypress_m8.ko 2023-09-13T02:45:27.451 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-09-13T02:45:27.451 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/empeg.ko 2023-09-13T02:45:27.452 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-09-13T02:45:27.452 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/garmin_gps.ko 2023-09-13T02:45:27.452 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_edgeport.ko 2023-09-13T02:45:27.452 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_ti.ko 2023-09-13T02:45:27.452 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipaq.ko 2023-09-13T02:45:27.452 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipw.ko 2023-09-13T02:45:27.453 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ir-usb.ko 2023-09-13T02:45:27.453 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-09-13T02:45:27.453 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan.ko 2023-09-13T02:45:27.453 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-09-13T02:45:27.453 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-09-13T02:45:27.454 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kobil_sct.ko 2023-09-13T02:45:27.454 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mct_u232.ko 2023-09-13T02:45:27.454 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7720.ko 2023-09-13T02:45:27.454 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7840.ko 2023-09-13T02:45:27.454 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/navman.ko 2023-09-13T02:45:27.455 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/omninet.ko 2023-09-13T02:45:27.455 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/opticon.ko 2023-09-13T02:45:27.457 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/option.ko 2023-09-13T02:45:27.457 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/oti6858.ko 2023-09-13T02:45:27.457 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/pl2303.ko 2023-09-13T02:45:27.457 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcaux.ko 2023-09-13T02:45:27.458 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcserial.ko 2023-09-13T02:45:27.458 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/quatech2.ko 2023-09-13T02:45:27.458 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/safe_serial.ko 2023-09-13T02:45:27.458 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/sierra.ko 2023-09-13T02:45:27.458 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/spcp8x5.ko 2023-09-13T02:45:27.459 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ssu100.ko 2023-09-13T02:45:27.459 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/symbolserial.ko 2023-09-13T02:45:27.459 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-09-13T02:45:27.459 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-09-13T02:45:27.459 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_debug.ko 2023-09-13T02:45:27.459 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_wwan.ko 2023-09-13T02:45:27.460 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/visor.ko 2023-09-13T02:45:27.460 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/whiteheat.ko 2023-09-13T02:45:27.460 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/xsens_mt.ko 2023-09-13T02:45:27.460 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage 2023-09-13T02:45:27.460 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/uas.ko 2023-09-13T02:45:27.461 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-alauda.ko 2023-09-13T02:45:27.461 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-cypress.ko 2023-09-13T02:45:27.461 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-datafab.ko 2023-09-13T02:45:27.461 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-09-13T02:45:27.461 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-freecom.ko 2023-09-13T02:45:27.462 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-isd200.ko 2023-09-13T02:45:27.462 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-09-13T02:45:27.462 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-karma.ko 2023-09-13T02:45:27.462 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-09-13T02:45:27.462 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-realtek.ko 2023-09-13T02:45:27.463 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-09-13T02:45:27.463 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-09-13T02:45:27.463 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-usbat.ko 2023-09-13T02:45:27.463 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/usb-storage.ko 2023-09-13T02:45:27.463 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip 2023-09-13T02:45:27.463 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-core.ko 2023-09-13T02:45:27.464 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-host.ko 2023-09-13T02:45:27.464 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-09-13T02:45:27.464 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio 2023-09-13T02:45:27.464 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci 2023-09-13T02:45:27.464 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-09-13T02:45:27.465 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-09-13T02:45:27.465 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio.ko 2023-09-13T02:45:27.465 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-09-13T02:45:27.465 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost 2023-09-13T02:45:27.465 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost.ko 2023-09-13T02:45:27.466 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_iotlb.ko 2023-09-13T02:45:27.466 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_net.ko 2023-09-13T02:45:27.466 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_scsi.ko 2023-09-13T02:45:27.466 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video 2023-09-13T02:45:27.466 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight 2023-09-13T02:45:27.466 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/apple_bl.ko 2023-09-13T02:45:27.467 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/lcd.ko 2023-09-13T02:45:27.467 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/platform_lcd.ko 2023-09-13T02:45:27.467 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio 2023-09-13T02:45:27.467 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio.ko 2023-09-13T02:45:27.467 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_balloon.ko 2023-09-13T02:45:27.468 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_mmio.ko 2023-09-13T02:45:27.468 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci.ko 2023-09-13T02:45:27.468 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-09-13T02:45:27.468 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-09-13T02:45:27.468 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_ring.ko 2023-09-13T02:45:27.469 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1 2023-09-13T02:45:27.469 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters 2023-09-13T02:45:27.469 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2482.ko 2023-09-13T02:45:27.469 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2490.ko 2023-09-13T02:45:27.469 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves 2023-09-13T02:45:27.469 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-09-13T02:45:27.470 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-09-13T02:45:27.470 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-09-13T02:45:27.470 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-09-13T02:45:27.470 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-09-13T02:45:27.470 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-09-13T02:45:27.471 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-09-13T02:45:27.471 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-09-13T02:45:27.471 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-09-13T02:45:27.471 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_smem.ko 2023-09-13T02:45:27.471 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_therm.ko 2023-09-13T02:45:27.472 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/wire.ko 2023-09-13T02:45:27.472 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog 2023-09-13T02:45:27.472 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-09-13T02:45:27.472 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-09-13T02:45:27.472 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-09-13T02:45:27.472 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/hpwdt.ko 2023-09-13T02:45:27.473 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/i6300esb.ko 2023-09-13T02:45:27.473 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-09-13T02:45:27.473 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-09-13T02:45:27.473 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ib700wdt.ko 2023-09-13T02:45:27.473 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ibmasr.ko 2023-09-13T02:45:27.474 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-09-13T02:45:27.474 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-09-13T02:45:27.474 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it87_wdt.ko 2023-09-13T02:45:27.474 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/machzwd.ko 2023-09-13T02:45:27.474 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/nv_tco.ko 2023-09-13T02:45:27.475 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_pci.ko 2023-09-13T02:45:27.475 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_usb.ko 2023-09-13T02:45:27.475 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-09-13T02:45:27.475 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-09-13T02:45:27.475 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/softdog.ko 2023-09-13T02:45:27.476 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sp5100_tco.ko 2023-09-13T02:45:27.476 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/via_wdt.ko 2023-09-13T02:45:27.476 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-09-13T02:45:27.476 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-09-13T02:45:27.476 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-09-13T02:45:27.476 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/wdt_pci.ko 2023-09-13T02:45:27.477 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/xen_wdt.ko 2023-09-13T02:45:27.477 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen 2023-09-13T02:45:27.477 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-acpi-processor.ko 2023-09-13T02:45:27.477 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-evtchn.ko 2023-09-13T02:45:27.477 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntalloc.ko 2023-09-13T02:45:27.478 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntdev.ko 2023-09-13T02:45:27.478 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback 2023-09-13T02:45:27.478 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-09-13T02:45:27.478 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-privcmd.ko 2023-09-13T02:45:27.478 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs 2023-09-13T02:45:27.479 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs/xenfs.ko 2023-09-13T02:45:27.479 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs 2023-09-13T02:45:27.479 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p 2023-09-13T02:45:27.479 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p/9p.ko 2023-09-13T02:45:27.479 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs 2023-09-13T02:45:27.480 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs/affs.ko 2023-09-13T02:45:27.480 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs 2023-09-13T02:45:27.480 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs/befs.ko 2023-09-13T02:45:27.480 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/binfmt_misc.ko 2023-09-13T02:45:27.480 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs 2023-09-13T02:45:27.481 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs/btrfs.ko 2023-09-13T02:45:27.481 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles 2023-09-13T02:45:27.481 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles/cachefiles.ko 2023-09-13T02:45:27.481 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph 2023-09-13T02:45:27.481 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph/ceph.ko 2023-09-13T02:45:27.481 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda 2023-09-13T02:45:27.482 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda/coda.ko 2023-09-13T02:45:27.482 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs 2023-09-13T02:45:27.482 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs/cramfs.ko 2023-09-13T02:45:27.482 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm 2023-09-13T02:45:27.482 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm/dlm.ko 2023-09-13T02:45:27.483 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs 2023-09-13T02:45:27.483 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs/ecryptfs.ko 2023-09-13T02:45:27.483 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs 2023-09-13T02:45:27.483 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs/f2fs.ko 2023-09-13T02:45:27.483 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat 2023-09-13T02:45:27.484 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/fat.ko 2023-09-13T02:45:27.484 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/msdos.ko 2023-09-13T02:45:27.484 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/vfat.ko 2023-09-13T02:45:27.484 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache 2023-09-13T02:45:27.484 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache/fscache.ko 2023-09-13T02:45:27.485 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse 2023-09-13T02:45:27.485 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/cuse.ko 2023-09-13T02:45:27.485 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/fuse.ko 2023-09-13T02:45:27.485 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2 2023-09-13T02:45:27.485 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2/gfs2.ko 2023-09-13T02:45:27.485 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs 2023-09-13T02:45:27.486 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs/hfs.ko 2023-09-13T02:45:27.486 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus 2023-09-13T02:45:27.486 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus/hfsplus.ko 2023-09-13T02:45:27.486 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs 2023-09-13T02:45:27.486 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs/isofs.ko 2023-09-13T02:45:27.487 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs 2023-09-13T02:45:27.487 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs/jfs.ko 2023-09-13T02:45:27.487 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd 2023-09-13T02:45:27.487 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd/lockd.ko 2023-09-13T02:45:27.487 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix 2023-09-13T02:45:27.488 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix/minix.ko 2023-09-13T02:45:27.488 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs 2023-09-13T02:45:27.488 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs/netfs.ko 2023-09-13T02:45:27.488 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs 2023-09-13T02:45:27.488 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout 2023-09-13T02:45:27.488 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-09-13T02:45:27.489 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout 2023-09-13T02:45:27.489 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-09-13T02:45:27.489 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout 2023-09-13T02:45:27.489 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-09-13T02:45:27.489 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfs.ko 2023-09-13T02:45:27.490 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv3.ko 2023-09-13T02:45:27.490 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv4.ko 2023-09-13T02:45:27.490 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common 2023-09-13T02:45:27.490 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/grace.ko 2023-09-13T02:45:27.490 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/nfs_acl.ko 2023-09-13T02:45:27.491 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd 2023-09-13T02:45:27.491 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd/nfsd.ko 2023-09-13T02:45:27.491 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2 2023-09-13T02:45:27.491 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2/nilfs2.ko 2023-09-13T02:45:27.491 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls 2023-09-13T02:45:27.492 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-celtic.ko 2023-09-13T02:45:27.492 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-centeuro.ko 2023-09-13T02:45:27.492 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-croatian.ko 2023-09-13T02:45:27.492 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-cyrillic.ko 2023-09-13T02:45:27.492 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-gaelic.ko 2023-09-13T02:45:27.493 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-greek.ko 2023-09-13T02:45:27.493 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-iceland.ko 2023-09-13T02:45:27.493 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-inuit.ko 2023-09-13T02:45:27.493 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-roman.ko 2023-09-13T02:45:27.493 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-romanian.ko 2023-09-13T02:45:27.493 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-turkish.ko 2023-09-13T02:45:27.494 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1250.ko 2023-09-13T02:45:27.494 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1251.ko 2023-09-13T02:45:27.494 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1255.ko 2023-09-13T02:45:27.494 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp737.ko 2023-09-13T02:45:27.494 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp775.ko 2023-09-13T02:45:27.495 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp850.ko 2023-09-13T02:45:27.495 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp852.ko 2023-09-13T02:45:27.495 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp855.ko 2023-09-13T02:45:27.495 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp857.ko 2023-09-13T02:45:27.495 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp860.ko 2023-09-13T02:45:27.496 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp861.ko 2023-09-13T02:45:27.496 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp862.ko 2023-09-13T02:45:27.496 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp863.ko 2023-09-13T02:45:27.496 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp864.ko 2023-09-13T02:45:27.496 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp865.ko 2023-09-13T02:45:27.497 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp866.ko 2023-09-13T02:45:27.497 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp869.ko 2023-09-13T02:45:27.497 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp874.ko 2023-09-13T02:45:27.497 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp932.ko 2023-09-13T02:45:27.497 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp936.ko 2023-09-13T02:45:27.498 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp949.ko 2023-09-13T02:45:27.498 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp950.ko 2023-09-13T02:45:27.498 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_euc-jp.ko 2023-09-13T02:45:27.498 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-1.ko 2023-09-13T02:45:27.498 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-13.ko 2023-09-13T02:45:27.499 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-14.ko 2023-09-13T02:45:27.499 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-15.ko 2023-09-13T02:45:27.499 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-2.ko 2023-09-13T02:45:27.499 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-3.ko 2023-09-13T02:45:27.499 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-4.ko 2023-09-13T02:45:27.500 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-5.ko 2023-09-13T02:45:27.500 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-6.ko 2023-09-13T02:45:27.500 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-7.ko 2023-09-13T02:45:27.500 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-9.ko 2023-09-13T02:45:27.500 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-r.ko 2023-09-13T02:45:27.500 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-ru.ko 2023-09-13T02:45:27.501 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-u.ko 2023-09-13T02:45:27.501 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_utf8.ko 2023-09-13T02:45:27.501 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2 2023-09-13T02:45:27.501 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster 2023-09-13T02:45:27.501 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-09-13T02:45:27.502 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm 2023-09-13T02:45:27.502 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-09-13T02:45:27.502 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs 2023-09-13T02:45:27.502 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-09-13T02:45:27.502 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2.ko 2023-09-13T02:45:27.503 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-09-13T02:45:27.503 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-09-13T02:45:27.503 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-09-13T02:45:27.503 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs 2023-09-13T02:45:27.503 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs/overlay.ko 2023-09-13T02:45:27.504 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore 2023-09-13T02:45:27.504 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore/ramoops.ko 2023-09-13T02:45:27.504 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs 2023-09-13T02:45:27.504 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs/reiserfs.ko 2023-09-13T02:45:27.504 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs 2023-09-13T02:45:27.505 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs/romfs.ko 2023-09-13T02:45:27.505 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb 2023-09-13T02:45:27.505 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client 2023-09-13T02:45:27.505 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client/cifs.ko 2023-09-13T02:45:27.505 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common 2023-09-13T02:45:27.506 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_arc4.ko 2023-09-13T02:45:27.506 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_md4.ko 2023-09-13T02:45:27.506 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs 2023-09-13T02:45:27.506 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs/squashfs.ko 2023-09-13T02:45:27.506 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv 2023-09-13T02:45:27.507 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv/sysv.ko 2023-09-13T02:45:27.507 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf 2023-09-13T02:45:27.507 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf/udf.ko 2023-09-13T02:45:27.507 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs 2023-09-13T02:45:27.507 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs/ufs.ko 2023-09-13T02:45:27.508 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs 2023-09-13T02:45:27.508 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs/xfs.ko 2023-09-13T02:45:27.508 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel 2023-09-13T02:45:27.508 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu 2023-09-13T02:45:27.508 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu/rcutorture.ko 2023-09-13T02:45:27.509 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/torture.ko 2023-09-13T02:45:27.509 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace 2023-09-13T02:45:27.509 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-09-13T02:45:27.509 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib 2023-09-13T02:45:27.509 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/asn1_encoder.ko 2023-09-13T02:45:27.510 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc-itu-t.ko 2023-09-13T02:45:27.510 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc8.ko 2023-09-13T02:45:27.510 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto 2023-09-13T02:45:27.510 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libarc4.ko 2023-09-13T02:45:27.510 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libdes.ko 2023-09-13T02:45:27.510 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/libcrc32c.ko 2023-09-13T02:45:27.511 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lru_cache.ko 2023-09-13T02:45:27.511 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4 2023-09-13T02:45:27.511 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4_compress.ko 2023-09-13T02:45:27.511 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4hc_compress.ko 2023-09-13T02:45:27.511 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math 2023-09-13T02:45:27.512 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math/cordic.ko 2023-09-13T02:45:27.512 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6 2023-09-13T02:45:27.512 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6/raid6_pq.ko 2023-09-13T02:45:27.512 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon 2023-09-13T02:45:27.512 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon/reed_solomon.ko 2023-09-13T02:45:27.513 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_bm.ko 2023-09-13T02:45:27.513 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_fsm.ko 2023-09-13T02:45:27.513 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_kmp.ko 2023-09-13T02:45:27.513 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd 2023-09-13T02:45:27.513 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd/zstd_compress.ko 2023-09-13T02:45:27.513 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm 2023-09-13T02:45:27.514 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm/hwpoison-inject.ko 2023-09-13T02:45:27.514 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net 2023-09-13T02:45:27.514 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan 2023-09-13T02:45:27.514 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/6lowpan.ko 2023-09-13T02:45:27.514 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_dest.ko 2023-09-13T02:45:27.515 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_fragment.ko 2023-09-13T02:45:27.515 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_hop.ko 2023-09-13T02:45:27.515 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_ipv6.ko 2023-09-13T02:45:27.515 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_mobility.ko 2023-09-13T02:45:27.515 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_routing.ko 2023-09-13T02:45:27.516 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_udp.ko 2023-09-13T02:45:27.516 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802 2023-09-13T02:45:27.516 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/garp.ko 2023-09-13T02:45:27.516 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/mrp.ko 2023-09-13T02:45:27.516 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/p8022.ko 2023-09-13T02:45:27.516 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/psnap.ko 2023-09-13T02:45:27.517 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/stp.ko 2023-09-13T02:45:27.517 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q 2023-09-13T02:45:27.517 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q/8021q.ko 2023-09-13T02:45:27.517 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p 2023-09-13T02:45:27.517 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet.ko 2023-09-13T02:45:27.518 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_fd.ko 2023-09-13T02:45:27.518 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_rdma.ko 2023-09-13T02:45:27.518 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_virtio.ko 2023-09-13T02:45:27.518 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv 2023-09-13T02:45:27.518 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv/batman-adv.ko 2023-09-13T02:45:27.519 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge 2023-09-13T02:45:27.519 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/br_netfilter.ko 2023-09-13T02:45:27.519 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/bridge.ko 2023-09-13T02:45:27.519 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter 2023-09-13T02:45:27.519 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-09-13T02:45:27.519 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_among.ko 2023-09-13T02:45:27.520 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-09-13T02:45:27.520 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-09-13T02:45:27.520 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-09-13T02:45:27.520 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-09-13T02:45:27.520 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-09-13T02:45:27.521 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-09-13T02:45:27.521 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_log.ko 2023-09-13T02:45:27.521 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-09-13T02:45:27.521 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-09-13T02:45:27.521 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-09-13T02:45:27.522 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-09-13T02:45:27.522 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-09-13T02:45:27.522 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-09-13T02:45:27.522 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-09-13T02:45:27.522 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-09-13T02:45:27.522 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-09-13T02:45:27.523 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-09-13T02:45:27.523 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-09-13T02:45:27.523 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtables.ko 2023-09-13T02:45:27.523 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-09-13T02:45:27.523 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-09-13T02:45:27.524 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-09-13T02:45:27.524 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph 2023-09-13T02:45:27.524 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph/libceph.ko 2023-09-13T02:45:27.524 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core 2023-09-13T02:45:27.524 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/failover.ko 2023-09-13T02:45:27.525 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/pktgen.ko 2023-09-13T02:45:27.525 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp 2023-09-13T02:45:27.525 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp.ko 2023-09-13T02:45:27.525 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_diag.ko 2023-09-13T02:45:27.525 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv4.ko 2023-09-13T02:45:27.525 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv6.ko 2023-09-13T02:45:27.526 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver 2023-09-13T02:45:27.526 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver/dns_resolver.ko 2023-09-13T02:45:27.526 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa 2023-09-13T02:45:27.526 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/dsa_core.ko 2023-09-13T02:45:27.526 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_brcm.ko 2023-09-13T02:45:27.527 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_dsa.ko 2023-09-13T02:45:27.527 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_none.ko 2023-09-13T02:45:27.527 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_trailer.ko 2023-09-13T02:45:27.527 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154 2023-09-13T02:45:27.527 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan 2023-09-13T02:45:27.528 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-09-13T02:45:27.528 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154.ko 2023-09-13T02:45:27.528 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154_socket.ko 2023-09-13T02:45:27.528 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4 2023-09-13T02:45:27.528 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ah4.ko 2023-09-13T02:45:27.529 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/esp4.ko 2023-09-13T02:45:27.529 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/fou.ko 2023-09-13T02:45:27.529 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/gre.ko 2023-09-13T02:45:27.529 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/inet_diag.ko 2023-09-13T02:45:27.529 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_gre.ko 2023-09-13T02:45:27.529 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_tunnel.ko 2023-09-13T02:45:27.530 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_vti.ko 2023-09-13T02:45:27.530 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipcomp.ko 2023-09-13T02:45:27.530 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipip.ko 2023-09-13T02:45:27.530 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter 2023-09-13T02:45:27.530 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-09-13T02:45:27.531 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-09-13T02:45:27.531 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-09-13T02:45:27.531 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-09-13T02:45:27.531 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-09-13T02:45:27.531 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-09-13T02:45:27.532 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-09-13T02:45:27.532 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-09-13T02:45:27.532 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-09-13T02:45:27.532 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-09-13T02:45:27.532 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-09-13T02:45:27.532 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-09-13T02:45:27.533 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-09-13T02:45:27.533 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-09-13T02:45:27.533 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-09-13T02:45:27.533 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-09-13T02:45:27.533 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-09-13T02:45:27.534 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-09-13T02:45:27.534 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-09-13T02:45:27.534 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-09-13T02:45:27.534 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-09-13T02:45:27.534 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_bic.ko 2023-09-13T02:45:27.535 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_dctcp.ko 2023-09-13T02:45:27.535 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_diag.ko 2023-09-13T02:45:27.535 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_highspeed.ko 2023-09-13T02:45:27.535 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_htcp.ko 2023-09-13T02:45:27.535 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_hybla.ko 2023-09-13T02:45:27.536 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_illinois.ko 2023-09-13T02:45:27.536 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_lp.ko 2023-09-13T02:45:27.536 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_scalable.ko 2023-09-13T02:45:27.536 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_vegas.ko 2023-09-13T02:45:27.536 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_veno.ko 2023-09-13T02:45:27.537 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_westwood.ko 2023-09-13T02:45:27.537 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_yeah.ko 2023-09-13T02:45:27.537 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tunnel4.ko 2023-09-13T02:45:27.537 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_diag.ko 2023-09-13T02:45:27.537 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_tunnel.ko 2023-09-13T02:45:27.537 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-09-13T02:45:27.538 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6 2023-09-13T02:45:27.538 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ah6.ko 2023-09-13T02:45:27.538 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/esp6.ko 2023-09-13T02:45:27.538 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/fou6.ko 2023-09-13T02:45:27.538 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_tunnel.ko 2023-09-13T02:45:27.539 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-09-13T02:45:27.539 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_vti.ko 2023-09-13T02:45:27.539 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ipcomp6.ko 2023-09-13T02:45:27.539 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter 2023-09-13T02:45:27.539 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-09-13T02:45:27.540 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-09-13T02:45:27.540 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-09-13T02:45:27.540 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-09-13T02:45:27.540 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-09-13T02:45:27.540 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-09-13T02:45:27.541 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-09-13T02:45:27.541 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-09-13T02:45:27.541 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-09-13T02:45:27.541 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-09-13T02:45:27.541 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-09-13T02:45:27.542 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-09-13T02:45:27.542 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-09-13T02:45:27.542 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-09-13T02:45:27.542 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-09-13T02:45:27.542 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-09-13T02:45:27.543 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-09-13T02:45:27.543 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-09-13T02:45:27.543 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-09-13T02:45:27.543 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-09-13T02:45:27.543 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-09-13T02:45:27.543 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-09-13T02:45:27.544 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-09-13T02:45:27.544 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-09-13T02:45:27.544 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/sit.ko 2023-09-13T02:45:27.544 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/tunnel6.ko 2023-09-13T02:45:27.544 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-09-13T02:45:27.545 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key 2023-09-13T02:45:27.545 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key/af_key.ko 2023-09-13T02:45:27.545 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp 2023-09-13T02:45:27.545 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_core.ko 2023-09-13T02:45:27.545 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_debugfs.ko 2023-09-13T02:45:27.546 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_eth.ko 2023-09-13T02:45:27.546 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip.ko 2023-09-13T02:45:27.546 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip6.ko 2023-09-13T02:45:27.546 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_netlink.ko 2023-09-13T02:45:27.546 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ppp.ko 2023-09-13T02:45:27.547 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc 2023-09-13T02:45:27.547 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc/llc.ko 2023-09-13T02:45:27.547 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154 2023-09-13T02:45:27.547 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154/mac802154.ko 2023-09-13T02:45:27.547 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls 2023-09-13T02:45:27.547 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls/mpls_gso.ko 2023-09-13T02:45:27.548 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter 2023-09-13T02:45:27.548 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset 2023-09-13T02:45:27.548 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set.ko 2023-09-13T02:45:27.548 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-09-13T02:45:27.548 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-09-13T02:45:27.549 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-09-13T02:45:27.549 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-09-13T02:45:27.549 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-09-13T02:45:27.549 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-09-13T02:45:27.549 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-09-13T02:45:27.550 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-09-13T02:45:27.550 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-09-13T02:45:27.550 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-09-13T02:45:27.550 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-09-13T02:45:27.550 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-09-13T02:45:27.551 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-09-13T02:45:27.551 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-09-13T02:45:27.551 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-09-13T02:45:27.551 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs 2023-09-13T02:45:27.551 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-09-13T02:45:27.551 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-09-13T02:45:27.552 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-09-13T02:45:27.552 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-09-13T02:45:27.552 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-09-13T02:45:27.552 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-09-13T02:45:27.552 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-09-13T02:45:27.553 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-09-13T02:45:27.553 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-09-13T02:45:27.553 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-09-13T02:45:27.553 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-09-13T02:45:27.553 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-09-13T02:45:27.554 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-09-13T02:45:27.554 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-09-13T02:45:27.554 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conncount.ko 2023-09-13T02:45:27.554 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack.ko 2023-09-13T02:45:27.554 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-09-13T02:45:27.555 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-09-13T02:45:27.555 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-09-13T02:45:27.555 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-09-13T02:45:27.555 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-09-13T02:45:27.555 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-09-13T02:45:27.555 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-09-13T02:45:27.556 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-09-13T02:45:27.556 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-09-13T02:45:27.556 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-09-13T02:45:27.556 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-09-13T02:45:27.556 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-09-13T02:45:27.557 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_dup_netdev.ko 2023-09-13T02:45:27.557 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table.ko 2023-09-13T02:45:27.557 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-09-13T02:45:27.557 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_log_syslog.ko 2023-09-13T02:45:27.557 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat.ko 2023-09-13T02:45:27.558 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_amanda.ko 2023-09-13T02:45:27.558 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_ftp.ko 2023-09-13T02:45:27.558 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_irc.ko 2023-09-13T02:45:27.558 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_sip.ko 2023-09-13T02:45:27.558 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_tftp.ko 2023-09-13T02:45:27.558 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_synproxy_core.ko 2023-09-13T02:45:27.559 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_tables.ko 2023-09-13T02:45:27.559 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink.ko 2023-09-13T02:45:27.559 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_acct.ko 2023-09-13T02:45:27.559 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_log.ko 2023-09-13T02:45:27.559 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_osf.ko 2023-09-13T02:45:27.560 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_queue.ko 2023-09-13T02:45:27.560 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_chain_nat.ko 2023-09-13T02:45:27.560 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_compat.ko 2023-09-13T02:45:27.560 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_ct.ko 2023-09-13T02:45:27.560 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_dup_netdev.ko 2023-09-13T02:45:27.561 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fib.ko 2023-09-13T02:45:27.561 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-09-13T02:45:27.561 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_hash.ko 2023-09-13T02:45:27.561 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_limit.ko 2023-09-13T02:45:27.561 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_log.ko 2023-09-13T02:45:27.561 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_masq.ko 2023-09-13T02:45:27.562 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_nat.ko 2023-09-13T02:45:27.562 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_numgen.ko 2023-09-13T02:45:27.562 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_queue.ko 2023-09-13T02:45:27.562 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_quota.ko 2023-09-13T02:45:27.562 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_redir.ko 2023-09-13T02:45:27.563 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject.ko 2023-09-13T02:45:27.563 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject_inet.ko 2023-09-13T02:45:27.563 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_socket.ko 2023-09-13T02:45:27.563 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_tunnel.ko 2023-09-13T02:45:27.563 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_AUDIT.ko 2023-09-13T02:45:27.564 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-09-13T02:45:27.564 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-09-13T02:45:27.564 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-09-13T02:45:27.564 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CT.ko 2023-09-13T02:45:27.564 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_DSCP.ko 2023-09-13T02:45:27.565 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HL.ko 2023-09-13T02:45:27.565 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HMARK.ko 2023-09-13T02:45:27.565 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-09-13T02:45:27.565 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LED.ko 2023-09-13T02:45:27.565 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LOG.ko 2023-09-13T02:45:27.566 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-09-13T02:45:27.566 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NETMAP.ko 2023-09-13T02:45:27.566 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFLOG.ko 2023-09-13T02:45:27.566 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-09-13T02:45:27.566 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_RATEEST.ko 2023-09-13T02:45:27.566 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_REDIRECT.ko 2023-09-13T02:45:27.567 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_SECMARK.ko 2023-09-13T02:45:27.567 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPMSS.ko 2023-09-13T02:45:27.567 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-09-13T02:45:27.567 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TEE.ko 2023-09-13T02:45:27.567 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TPROXY.ko 2023-09-13T02:45:27.568 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TRACE.ko 2023-09-13T02:45:27.568 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_addrtype.ko 2023-09-13T02:45:27.568 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_bpf.ko 2023-09-13T02:45:27.568 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cgroup.ko 2023-09-13T02:45:27.568 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cluster.ko 2023-09-13T02:45:27.569 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_comment.ko 2023-09-13T02:45:27.569 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connbytes.ko 2023-09-13T02:45:27.569 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlabel.ko 2023-09-13T02:45:27.569 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlimit.ko 2023-09-13T02:45:27.569 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connmark.ko 2023-09-13T02:45:27.569 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_conntrack.ko 2023-09-13T02:45:27.570 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cpu.ko 2023-09-13T02:45:27.570 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dccp.ko 2023-09-13T02:45:27.570 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_devgroup.ko 2023-09-13T02:45:27.570 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dscp.ko 2023-09-13T02:45:27.570 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ecn.ko 2023-09-13T02:45:27.571 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_esp.ko 2023-09-13T02:45:27.571 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hashlimit.ko 2023-09-13T02:45:27.571 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_helper.ko 2023-09-13T02:45:27.571 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hl.ko 2023-09-13T02:45:27.571 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipcomp.ko 2023-09-13T02:45:27.572 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_iprange.ko 2023-09-13T02:45:27.572 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipvs.ko 2023-09-13T02:45:27.572 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_l2tp.ko 2023-09-13T02:45:27.572 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_length.ko 2023-09-13T02:45:27.572 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_limit.ko 2023-09-13T02:45:27.572 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mac.ko 2023-09-13T02:45:27.573 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mark.ko 2023-09-13T02:45:27.573 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_multiport.ko 2023-09-13T02:45:27.573 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nat.ko 2023-09-13T02:45:27.573 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nfacct.ko 2023-09-13T02:45:27.573 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_osf.ko 2023-09-13T02:45:27.574 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_owner.ko 2023-09-13T02:45:27.574 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_physdev.ko 2023-09-13T02:45:27.574 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_pkttype.ko 2023-09-13T02:45:27.574 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_policy.ko 2023-09-13T02:45:27.574 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_quota.ko 2023-09-13T02:45:27.575 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_rateest.ko 2023-09-13T02:45:27.575 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_realm.ko 2023-09-13T02:45:27.575 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_recent.ko 2023-09-13T02:45:27.575 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_sctp.ko 2023-09-13T02:45:27.575 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_set.ko 2023-09-13T02:45:27.576 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_socket.ko 2023-09-13T02:45:27.576 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_state.ko 2023-09-13T02:45:27.576 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_statistic.ko 2023-09-13T02:45:27.576 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_string.ko 2023-09-13T02:45:27.576 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_tcpmss.ko 2023-09-13T02:45:27.577 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_time.ko 2023-09-13T02:45:27.577 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_u32.ko 2023-09-13T02:45:27.577 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink 2023-09-13T02:45:27.577 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink/netlink_diag.ko 2023-09-13T02:45:27.577 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh 2023-09-13T02:45:27.578 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh/nsh.ko 2023-09-13T02:45:27.578 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch 2023-09-13T02:45:27.578 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/openvswitch.ko 2023-09-13T02:45:27.578 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-geneve.ko 2023-09-13T02:45:27.578 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-gre.ko 2023-09-13T02:45:27.578 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-vxlan.ko 2023-09-13T02:45:27.579 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet 2023-09-13T02:45:27.579 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet/af_packet_diag.ko 2023-09-13T02:45:27.579 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds 2023-09-13T02:45:27.579 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds.ko 2023-09-13T02:45:27.579 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_rdma.ko 2023-09-13T02:45:27.580 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_tcp.ko 2023-09-13T02:45:27.580 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched 2023-09-13T02:45:27.580 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_csum.ko 2023-09-13T02:45:27.580 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_gact.ko 2023-09-13T02:45:27.580 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_ipt.ko 2023-09-13T02:45:27.581 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_mirred.ko 2023-09-13T02:45:27.581 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_nat.ko 2023-09-13T02:45:27.581 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_pedit.ko 2023-09-13T02:45:27.581 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_police.ko 2023-09-13T02:45:27.581 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_simple.ko 2023-09-13T02:45:27.582 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_skbedit.ko 2023-09-13T02:45:27.582 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_vlan.ko 2023-09-13T02:45:27.582 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_basic.ko 2023-09-13T02:45:27.582 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_bpf.ko 2023-09-13T02:45:27.582 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_flow.ko 2023-09-13T02:45:27.582 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_fw.ko 2023-09-13T02:45:27.583 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_route.ko 2023-09-13T02:45:27.583 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_u32.ko 2023-09-13T02:45:27.583 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_cmp.ko 2023-09-13T02:45:27.583 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_ipset.ko 2023-09-13T02:45:27.583 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_meta.ko 2023-09-13T02:45:27.584 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_nbyte.ko 2023-09-13T02:45:27.584 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_text.ko 2023-09-13T02:45:27.584 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_u32.ko 2023-09-13T02:45:27.584 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_choke.ko 2023-09-13T02:45:27.584 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_codel.ko 2023-09-13T02:45:27.584 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_drr.ko 2023-09-13T02:45:27.585 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_fq.ko 2023-09-13T02:45:27.585 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_gred.ko 2023-09-13T02:45:27.585 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hfsc.ko 2023-09-13T02:45:27.585 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hhf.ko 2023-09-13T02:45:27.585 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_htb.ko 2023-09-13T02:45:27.586 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_ingress.ko 2023-09-13T02:45:27.586 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio.ko 2023-09-13T02:45:27.586 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio_lib.ko 2023-09-13T02:45:27.586 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_multiq.ko 2023-09-13T02:45:27.586 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_netem.ko 2023-09-13T02:45:27.587 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_pie.ko 2023-09-13T02:45:27.587 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_plug.ko 2023-09-13T02:45:27.587 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_prio.ko 2023-09-13T02:45:27.587 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_qfq.ko 2023-09-13T02:45:27.587 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_red.ko 2023-09-13T02:45:27.588 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfb.ko 2023-09-13T02:45:27.588 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfq.ko 2023-09-13T02:45:27.588 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_tbf.ko 2023-09-13T02:45:27.588 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_teql.ko 2023-09-13T02:45:27.588 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp 2023-09-13T02:45:27.588 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp.ko 2023-09-13T02:45:27.589 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp_diag.ko 2023-09-13T02:45:27.589 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc 2023-09-13T02:45:27.589 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss 2023-09-13T02:45:27.589 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-09-13T02:45:27.589 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-09-13T02:45:27.590 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/sunrpc.ko 2023-09-13T02:45:27.590 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma 2023-09-13T02:45:27.590 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-09-13T02:45:27.590 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc 2023-09-13T02:45:27.590 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/diag.ko 2023-09-13T02:45:27.591 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/tipc.ko 2023-09-13T02:45:27.591 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix 2023-09-13T02:45:27.591 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix/unix_diag.ko 2023-09-13T02:45:27.591 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock 2023-09-13T02:45:27.591 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-09-13T02:45:27.592 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-09-13T02:45:27.592 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock.ko 2023-09-13T02:45:27.592 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_diag.ko 2023-09-13T02:45:27.592 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-09-13T02:45:27.592 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm 2023-09-13T02:45:27.592 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-09-13T02:45:27.593 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security 2023-09-13T02:45:27.593 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys 2023-09-13T02:45:27.593 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys 2023-09-13T02:45:27.593 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-09-13T02:45:27.593 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys 2023-09-13T02:45:27.594 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys/trusted.ko 2023-09-13T02:45:27.594 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt 2023-09-13T02:45:27.594 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib 2023-09-13T02:45:27.594 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib/irqbypass.ko 2023-09-13T02:45:27.594 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias 2023-09-13T02:45:27.594 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias.bin 2023-09-13T02:45:27.595 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin 2023-09-13T02:45:27.595 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.bin 2023-09-13T02:45:27.595 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.modinfo 2023-09-13T02:45:27.595 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep 2023-09-13T02:45:27.595 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep.bin 2023-09-13T02:45:27.596 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.devname 2023-09-13T02:45:27.596 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.order 2023-09-13T02:45:27.596 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.softdep 2023-09-13T02:45:27.596 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols 2023-09-13T02:45:27.596 INFO:teuthology.orchestra.run.smithi064.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols.bin 2023-09-13T02:45:27.597 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc7-gfef64f93da2b 2023-09-13T02:45:27.597 DEBUG:teuthology.orchestra.run.smithi064:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-09-13T02:45:27.626 INFO:teuthology.orchestra.run.smithi064.stdout:postinstall scriptlet (using /bin/sh): 2023-09-13T02:45:27.626 INFO:teuthology.orchestra.run.smithi064.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b -a -r /boot/System.map-6.5.0-rc7-gfef64f93da2b ]; then 2023-09-13T02:45:27.627 INFO:teuthology.orchestra.run.smithi064.stdout:cp /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm 2023-09-13T02:45:27.627 INFO:teuthology.orchestra.run.smithi064.stdout:cp /boot/System.map-6.5.0-rc7-gfef64f93da2b /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-09-13T02:45:27.627 INFO:teuthology.orchestra.run.smithi064.stdout:rm -f /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-09-13T02:45:27.627 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/installkernel 6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-09-13T02:45:27.627 INFO:teuthology.orchestra.run.smithi064.stdout:rm -f /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-09-13T02:45:27.628 INFO:teuthology.orchestra.run.smithi064.stdout:fi 2023-09-13T02:45:27.628 INFO:teuthology.orchestra.run.smithi064.stdout:preuninstall scriptlet (using /bin/sh): 2023-09-13T02:45:27.628 INFO:teuthology.orchestra.run.smithi064.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-09-13T02:45:27.628 INFO:teuthology.orchestra.run.smithi064.stdout:new-kernel-pkg --remove 6.5.0-rc7-gfef64f93da2b --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc7-gfef64f93da2b.img 2023-09-13T02:45:27.628 INFO:teuthology.orchestra.run.smithi064.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-09-13T02:45:27.629 INFO:teuthology.orchestra.run.smithi064.stdout:kernel-install remove 6.5.0-rc7-gfef64f93da2b 2023-09-13T02:45:27.629 INFO:teuthology.orchestra.run.smithi064.stdout:fi 2023-09-13T02:45:27.629 INFO:teuthology.orchestra.run.smithi064.stdout:postuninstall scriptlet (using /bin/sh): 2023-09-13T02:45:27.629 INFO:teuthology.orchestra.run.smithi064.stdout:if [ -x /sbin/update-bootloader ]; then 2023-09-13T02:45:27.629 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/update-bootloader --remove 6.5.0-rc7-gfef64f93da2b 2023-09-13T02:45:27.629 INFO:teuthology.orchestra.run.smithi064.stdout:fi 2023-09-13T02:45:27.631 DEBUG:teuthology.orchestra.run.smithi064:> sudo rpm -qi grub2-tools 2023-09-13T02:45:27.711 INFO:teuthology.orchestra.run.smithi064.stdout:Name : grub2-tools 2023-09-13T02:45:27.711 INFO:teuthology.orchestra.run.smithi064.stdout:Epoch : 1 2023-09-13T02:45:27.712 INFO:teuthology.orchestra.run.smithi064.stdout:Version : 2.02 2023-09-13T02:45:27.712 INFO:teuthology.orchestra.run.smithi064.stdout:Release : 123.el8 2023-09-13T02:45:27.712 INFO:teuthology.orchestra.run.smithi064.stdout:Architecture: x86_64 2023-09-13T02:45:27.712 INFO:teuthology.orchestra.run.smithi064.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-09-13T02:45:27.712 INFO:teuthology.orchestra.run.smithi064.stdout:Group : System Environment/Base 2023-09-13T02:45:27.713 INFO:teuthology.orchestra.run.smithi064.stdout:Size : 9265450 2023-09-13T02:45:27.713 INFO:teuthology.orchestra.run.smithi064.stdout:License : GPLv3+ 2023-09-13T02:45:27.713 INFO:teuthology.orchestra.run.smithi064.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-09-13T02:45:27.713 INFO:teuthology.orchestra.run.smithi064.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-09-13T02:45:27.714 INFO:teuthology.orchestra.run.smithi064.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-09-13T02:45:27.714 INFO:teuthology.orchestra.run.smithi064.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-09-13T02:45:27.714 INFO:teuthology.orchestra.run.smithi064.stdout:Relocations : (not relocatable) 2023-09-13T02:45:27.714 INFO:teuthology.orchestra.run.smithi064.stdout:Packager : Red Hat, Inc. 2023-09-13T02:45:27.715 INFO:teuthology.orchestra.run.smithi064.stdout:Vendor : Red Hat, Inc. 2023-09-13T02:45:27.715 INFO:teuthology.orchestra.run.smithi064.stdout:URL : http://www.gnu.org/software/grub/ 2023-09-13T02:45:27.715 INFO:teuthology.orchestra.run.smithi064.stdout:Summary : Support tools for GRUB. 2023-09-13T02:45:27.715 INFO:teuthology.orchestra.run.smithi064.stdout:Description : 2023-09-13T02:45:27.716 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T02:45:27.716 INFO:teuthology.orchestra.run.smithi064.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-09-13T02:45:27.716 INFO:teuthology.orchestra.run.smithi064.stdout:customizable bootloader with modular architecture. It supports a rich 2023-09-13T02:45:27.717 INFO:teuthology.orchestra.run.smithi064.stdout:variety of kernel formats, file systems, computer architectures and 2023-09-13T02:45:27.717 INFO:teuthology.orchestra.run.smithi064.stdout:hardware devices. 2023-09-13T02:45:27.717 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T02:45:27.717 INFO:teuthology.orchestra.run.smithi064.stdout:This subpackage provides tools for support of all platforms. 2023-09-13T02:45:27.719 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-09-13T02:45:27.719 INFO:teuthology.task.kernel:Updating grub on smithi064 to boot 6.5.0-rc7-gfef64f93da2b 2023-09-13T02:45:27.720 DEBUG:teuthology.orchestra.run.smithi064:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-09-13T02:45:30.335 INFO:teuthology.orchestra.run.smithi064.stderr:Generating grub configuration file ... 2023-09-13T02:45:32.488 INFO:teuthology.orchestra.run.smithi064.stderr:done 2023-09-13T02:45:32.490 DEBUG:teuthology.orchestra.run.smithi064:> mktemp 2023-09-13T02:45:32.507 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.zA2MTjBEbc 2023-09-13T02:45:32.508 DEBUG:teuthology.orchestra.run.smithi064:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.zA2MTjBEbc 2023-09-13T02:45:32.579 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0666 /tmp/tmp.zA2MTjBEbc 2023-09-13T02:45:32.724 DEBUG:teuthology.orchestra.remote:smithi064:/tmp/tmp.zA2MTjBEbc is 6KB 2023-09-13T02:45:32.773 DEBUG:teuthology.orchestra.run.smithi064:> rm -fr /tmp/tmp.zA2MTjBEbc 2023-09-13T02:45:32.790 DEBUG:teuthology.orchestra.run.smithi064:> sudo /bin/ls /boot/loader/entries || true 2023-09-13T02:45:32.863 INFO:teuthology.orchestra.run.smithi064.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-09-13T02:45:32.863 INFO:teuthology.orchestra.run.smithi064.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-09-13T02:45:32.863 INFO:teuthology.orchestra.run.smithi064.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc7-gfef64f93da2b.conf 2023-09-13T02:45:32.864 DEBUG:teuthology.orchestra.run.smithi064:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc7-gfef64f93da2b 2023-09-13T02:45:33.159 DEBUG:teuthology.orchestra.run.smithi064:> sudo shutdown -r now 2023-09-13T02:46:03.162 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-09-13T02:46:03.163 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2023-09-13T02:46:03.164 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T02:47:03.166 DEBUG:teuthology.orchestra.remote:timed out 2023-09-13T02:47:12.168 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2023-09-13T02:47:12.170 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T02:47:15.051 DEBUG:teuthology.orchestra.run.smithi064:> true 2023-09-13T02:47:15.524 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com' 2023-09-13T02:47:15.525 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.5.0-rc7-gfef64f93da2b"... 2023-09-13T02:47:15.525 DEBUG:teuthology.orchestra.run.smithi064:> uname -r 2023-09-13T02:47:15.568 INFO:teuthology.orchestra.run.smithi064.stdout:6.5.0-rc7-gfef64f93da2b 2023-09-13T02:47:15.568 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc7-gfef64f93da2b vs 6.5.0-rc7-gfef64f93da2b 2023-09-13T02:47:15.568 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-09-13T02:47:15.569 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-09-13T02:47:16.570 DEBUG:teuthology.parallel:result is None 2023-09-13T02:47:16.571 INFO:teuthology.run_tasks:Running task internal.base... 2023-09-13T02:47:16.582 INFO:teuthology.task.internal:Creating test directory... 2023-09-13T02:47:16.583 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-13T02:47:16.588 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-13T02:47:16.591 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-13T02:47:16.628 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-09-13T02:47:16.636 INFO:teuthology.run_tasks:Running task internal.archive... 2023-09-13T02:47:16.679 INFO:teuthology.task.internal:Creating archive directory... 2023-09-13T02:47:16.680 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-13T02:47:16.684 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-13T02:47:16.687 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-13T02:47:16.743 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-09-13T02:47:16.753 INFO:teuthology.task.internal:Enabling coredump saving... 2023-09-13T02:47:16.753 DEBUG:teuthology.orchestra.run.smithi142:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-13T02:47:16.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-13T02:47:16.782 DEBUG:teuthology.orchestra.run.smithi170:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-13T02:47:16.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-13T02:47:16.801 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-13T02:47:16.806 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-13T02:47:16.824 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-13T02:47:16.868 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-13T02:47:16.884 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-13T02:47:16.886 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-13T02:47:16.932 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-13T02:47:16.958 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-13T02:47:16.998 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-13T02:47:17.003 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-09-13T02:47:17.011 INFO:teuthology.task.internal:Configuring sudo... 2023-09-13T02:47:17.011 DEBUG:teuthology.orchestra.run.smithi064:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-13T02:47:17.015 DEBUG:teuthology.orchestra.run.smithi142:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-13T02:47:17.018 DEBUG:teuthology.orchestra.run.smithi170:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-13T02:47:17.081 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-09-13T02:47:17.090 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-09-13T02:47:17.090 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-13T02:47:17.094 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-13T02:47:17.096 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-13T02:47:17.133 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-13T02:47:17.244 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-13T02:47:17.335 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-13T02:47:17.408 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-13T02:47:17.492 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-13T02:47:17.492 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-13T02:47:17.563 DEBUG:teuthology.orchestra.run.smithi142:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-13T02:47:17.611 DEBUG:teuthology.orchestra.run.smithi142:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-13T02:47:17.670 DEBUG:teuthology.orchestra.run.smithi142:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-13T02:47:17.756 DEBUG:teuthology.orchestra.run.smithi142:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-13T02:47:17.802 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-09-13T02:47:17.803 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-13T02:47:17.872 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-13T02:47:17.919 DEBUG:teuthology.orchestra.run.smithi170:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-13T02:47:17.973 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-13T02:47:18.059 DEBUG:teuthology.orchestra.run.smithi170:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-13T02:47:18.102 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-09-13T02:47:18.102 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-13T02:47:18.192 DEBUG:teuthology.orchestra.run.smithi064:> sudo service rsyslog restart 2023-09-13T02:47:18.196 DEBUG:teuthology.orchestra.run.smithi142:> sudo service rsyslog restart 2023-09-13T02:47:18.199 DEBUG:teuthology.orchestra.run.smithi170:> sudo service rsyslog restart 2023-09-13T02:47:18.263 INFO:teuthology.orchestra.run.smithi142.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-13T02:47:18.299 INFO:teuthology.orchestra.run.smithi170.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-13T02:47:18.317 INFO:teuthology.orchestra.run.smithi064.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-13T02:47:18.671 INFO:teuthology.run_tasks:Running task internal.timer... 2023-09-13T02:47:18.680 INFO:teuthology.task.internal:Starting timer... 2023-09-13T02:47:18.680 INFO:teuthology.run_tasks:Running task pcp... 2023-09-13T02:47:18.724 INFO:teuthology.run_tasks:Running task selinux... 2023-09-13T02:47:18.773 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-09-13T02:47:18.774 DEBUG:teuthology.orchestra.run.smithi064:> sudo service auditd rotate 2023-09-13T02:47:18.879 INFO:teuthology.orchestra.run.smithi064.stdout:Rotating logs: 2023-09-13T02:47:18.882 DEBUG:teuthology.orchestra.run.smithi142:> sudo service auditd rotate 2023-09-13T02:47:18.953 INFO:teuthology.orchestra.run.smithi142.stdout:Rotating logs: 2023-09-13T02:47:18.955 DEBUG:teuthology.orchestra.run.smithi170:> sudo service auditd rotate 2023-09-13T02:47:19.017 INFO:teuthology.orchestra.run.smithi170.stdout:Rotating logs: 2023-09-13T02:47:19.018 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-09-13T02:47:19.018 DEBUG:teuthology.orchestra.run.smithi064:> /usr/sbin/getenforce 2023-09-13T02:47:19.070 INFO:teuthology.orchestra.run.smithi064.stdout:Permissive 2023-09-13T02:47:19.071 DEBUG:teuthology.orchestra.run.smithi142:> /usr/sbin/getenforce 2023-09-13T02:47:19.090 INFO:teuthology.orchestra.run.smithi142.stdout:Permissive 2023-09-13T02:47:19.091 DEBUG:teuthology.orchestra.run.smithi170:> /usr/sbin/getenforce 2023-09-13T02:47:19.114 INFO:teuthology.orchestra.run.smithi170.stdout:Permissive 2023-09-13T02:47:19.115 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi064.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi142.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi170.front.sepia.ceph.com': 'permissive'} 2023-09-13T02:47:19.117 DEBUG:teuthology.orchestra.run.smithi064:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-13T02:47:19.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-13T02:47:19.192 DEBUG:teuthology.orchestra.run.smithi142:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-13T02:47:19.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-13T02:47:19.220 DEBUG:teuthology.orchestra.run.smithi170:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-13T02:47:19.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-13T02:47:19.247 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-09-13T02:47:19.247 DEBUG:teuthology.orchestra.run.smithi064:> sudo /usr/sbin/setenforce permissive 2023-09-13T02:47:19.317 DEBUG:teuthology.orchestra.run.smithi142:> sudo /usr/sbin/setenforce permissive 2023-09-13T02:47:19.343 DEBUG:teuthology.orchestra.run.smithi170:> sudo /usr/sbin/setenforce permissive 2023-09-13T02:47:19.370 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-09-13T02:47:19.381 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-09-13T02:47:19.382 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-09-13T02:47:19.398 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-09-13T02:47:19.400 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi064.front.sepia.ceph.com,smithi142.front.sepia.ceph.com,smithi170.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-09-13T02:59:30.788 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi064.front.sepia.ceph.com'), Remote(name='ubuntu@smithi142.front.sepia.ceph.com'), Remote(name='ubuntu@smithi170.front.sepia.ceph.com')] 2023-09-13T02:59:30.807 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2023-09-13T02:59:30.809 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T02:59:30.901 DEBUG:teuthology.orchestra.run.smithi064:> true 2023-09-13T02:59:31.016 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com' 2023-09-13T02:59:31.017 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2023-09-13T02:59:31.018 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T02:59:31.086 DEBUG:teuthology.orchestra.run.smithi142:> true 2023-09-13T02:59:31.158 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi142.front.sepia.ceph.com' 2023-09-13T02:59:31.159 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2023-09-13T02:59:31.159 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T02:59:31.227 DEBUG:teuthology.orchestra.run.smithi170:> true 2023-09-13T02:59:31.296 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi170.front.sepia.ceph.com' 2023-09-13T02:59:31.298 INFO:teuthology.run_tasks:Running task clock... 2023-09-13T02:59:31.309 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-09-13T02:59:31.310 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-13T02:59:31.311 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-13T02:59:31.316 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-13T02:59:31.316 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-13T02:59:31.319 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-13T02:59:31.319 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-13T02:59:31.352 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-09-13T02:59:31.369 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-09-13T02:59:31.369 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-09-13T02:59:31.384 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-09-13T02:59:31.386 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-09-13T02:59:31.422 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-09-13T02:59:31.440 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: ntpd: command not found 2023-09-13T02:59:31.446 INFO:teuthology.orchestra.run.smithi170.stderr:sudo: ntpd: command not found 2023-09-13T02:59:31.453 INFO:teuthology.orchestra.run.smithi142.stdout:506 Cannot talk to daemon 2023-09-13T02:59:31.458 INFO:teuthology.orchestra.run.smithi170.stdout:506 Cannot talk to daemon 2023-09-13T02:59:31.468 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-09-13T02:59:31.473 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-09-13T02:59:31.483 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-09-13T02:59:31.487 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-09-13T02:59:31.517 INFO:teuthology.orchestra.run.smithi064.stderr:sudo: ntpd: command not found 2023-09-13T02:59:31.547 INFO:teuthology.orchestra.run.smithi064.stdout:506 Cannot talk to daemon 2023-09-13T02:59:31.554 INFO:teuthology.orchestra.run.smithi170.stderr:bash: ntpq: command not found 2023-09-13T02:59:31.558 INFO:teuthology.orchestra.run.smithi170.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-13T02:59:31.558 INFO:teuthology.orchestra.run.smithi170.stdout:=============================================================================== 2023-09-13T02:59:31.558 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-13T02:59:31.558 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-13T02:59:31.559 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-13T02:59:31.559 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-13T02:59:31.571 INFO:teuthology.orchestra.run.smithi142.stderr:bash: ntpq: command not found 2023-09-13T02:59:31.574 INFO:teuthology.orchestra.run.smithi142.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-13T02:59:31.574 INFO:teuthology.orchestra.run.smithi142.stdout:=============================================================================== 2023-09-13T02:59:31.575 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-13T02:59:31.575 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-13T02:59:31.575 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-13T02:59:31.575 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-13T02:59:31.581 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-09-13T02:59:31.616 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-09-13T02:59:31.706 INFO:teuthology.orchestra.run.smithi064.stderr:bash: ntpq: command not found 2023-09-13T02:59:31.715 INFO:teuthology.orchestra.run.smithi064.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-13T02:59:31.716 INFO:teuthology.orchestra.run.smithi064.stdout:=============================================================================== 2023-09-13T02:59:31.716 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-13T02:59:31.716 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-13T02:59:31.716 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-13T02:59:31.716 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-13T02:59:31.717 INFO:teuthology.run_tasks:Running task pexec... 2023-09-13T02:59:31.727 INFO:teuthology.task.pexec:Executing custom commands... 2023-09-13T02:59:31.728 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi064.front.sepia.ceph.com 2023-09-13T02:59:31.728 DEBUG:teuthology.orchestra.run.smithi064:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-09-13T02:59:31.729 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi142.front.sepia.ceph.com 2023-09-13T02:59:31.729 DEBUG:teuthology.orchestra.run.smithi142:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-09-13T02:59:31.729 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi170.front.sepia.ceph.com 2023-09-13T02:59:31.730 DEBUG:teuthology.orchestra.run.smithi170:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-09-13T02:59:32.099 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2023-09-13T02:59:32.103 INFO:teuthology.orchestra.run.smithi142.stdout:Updating Subscription Management repositories. 2023-09-13T02:59:32.246 INFO:teuthology.orchestra.run.smithi064.stdout:Updating Subscription Management repositories. 2023-09-13T02:59:32.952 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:04:49 ago on Wed 13 Sep 2023 02:54:43 AM UTC. 2023-09-13T02:59:32.956 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:04:48 ago on Wed 13 Sep 2023 02:54:44 AM UTC. 2023-09-13T02:59:33.220 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:48 ago on Wed 13 Sep 2023 02:57:44 AM UTC. 2023-09-13T02:59:34.817 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2023-09-13T02:59:34.818 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2023-09-13T02:59:34.818 INFO:teuthology.orchestra.run.smithi170.stdout: Package Architecture Version Repository Size 2023-09-13T02:59:34.818 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2023-09-13T02:59:34.818 INFO:teuthology.orchestra.run.smithi170.stdout:Resetting modules: 2023-09-13T02:59:34.818 INFO:teuthology.orchestra.run.smithi170.stdout: container-tools 2023-09-13T02:59:34.819 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-09-13T02:59:34.819 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2023-09-13T02:59:34.819 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2023-09-13T02:59:34.819 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-09-13T02:59:34.837 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2023-09-13T02:59:34.838 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2023-09-13T02:59:34.838 INFO:teuthology.orchestra.run.smithi142.stdout: Package Architecture Version Repository Size 2023-09-13T02:59:34.839 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2023-09-13T02:59:34.839 INFO:teuthology.orchestra.run.smithi142.stdout:Resetting modules: 2023-09-13T02:59:34.839 INFO:teuthology.orchestra.run.smithi142.stdout: container-tools 2023-09-13T02:59:34.839 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-09-13T02:59:34.839 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2023-09-13T02:59:34.840 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2023-09-13T02:59:34.840 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-09-13T02:59:35.271 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2023-09-13T02:59:35.272 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2023-09-13T02:59:35.272 INFO:teuthology.orchestra.run.smithi064.stdout: Package Architecture Version Repository Size 2023-09-13T02:59:35.273 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2023-09-13T02:59:35.273 INFO:teuthology.orchestra.run.smithi064.stdout:Resetting modules: 2023-09-13T02:59:35.273 INFO:teuthology.orchestra.run.smithi064.stdout: container-tools 2023-09-13T02:59:35.273 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T02:59:35.274 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2023-09-13T02:59:35.274 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2023-09-13T02:59:35.274 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T02:59:35.773 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:04:52 ago on Wed 13 Sep 2023 02:54:43 AM UTC. 2023-09-13T02:59:35.813 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:04:51 ago on Wed 13 Sep 2023 02:54:44 AM UTC. 2023-09-13T02:59:36.556 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:52 ago on Wed 13 Sep 2023 02:57:44 AM UTC. 2023-09-13T02:59:38.488 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2023-09-13T02:59:38.663 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2023-09-13T02:59:38.965 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2023-09-13T02:59:39.133 INFO:teuthology.orchestra.run.smithi142.stdout:Updating Subscription Management repositories. 2023-09-13T02:59:39.140 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2023-09-13T02:59:39.767 INFO:teuthology.orchestra.run.smithi064.stdout:Updating Subscription Management repositories. 2023-09-13T02:59:39.770 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:04:56 ago on Wed 13 Sep 2023 02:54:43 AM UTC. 2023-09-13T02:59:39.906 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:04:55 ago on Wed 13 Sep 2023 02:54:44 AM UTC. 2023-09-13T02:59:40.745 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:56 ago on Wed 13 Sep 2023 02:57:44 AM UTC. 2023-09-13T02:59:41.464 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2023-09-13T02:59:41.556 INFO:teuthology.orchestra.run.smithi170.stdout:============================================================================================================ 2023-09-13T02:59:41.557 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repository Size 2023-09-13T02:59:41.557 INFO:teuthology.orchestra.run.smithi170.stdout:============================================================================================================ 2023-09-13T02:59:41.557 INFO:teuthology.orchestra.run.smithi170.stdout:Installing group/module packages: 2023-09-13T02:59:41.557 INFO:teuthology.orchestra.run.smithi170.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-09-13T02:59:41.558 INFO:teuthology.orchestra.run.smithi170.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-09-13T02:59:41.558 INFO:teuthology.orchestra.run.smithi170.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-09-13T02:59:41.558 INFO:teuthology.orchestra.run.smithi170.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-09-13T02:59:41.558 INFO:teuthology.orchestra.run.smithi170.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-09-13T02:59:41.558 INFO:teuthology.orchestra.run.smithi170.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-09-13T02:59:41.559 INFO:teuthology.orchestra.run.smithi170.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-09-13T02:59:41.559 INFO:teuthology.orchestra.run.smithi170.stdout:Installing dependencies: 2023-09-13T02:59:41.559 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-09-13T02:59:41.560 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-09-13T02:59:41.560 INFO:teuthology.orchestra.run.smithi170.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-09-13T02:59:41.560 INFO:teuthology.orchestra.run.smithi170.stdout:Installing module profiles: 2023-09-13T02:59:41.560 INFO:teuthology.orchestra.run.smithi170.stdout: container-tools/common 2023-09-13T02:59:41.560 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-09-13T02:59:41.561 INFO:teuthology.orchestra.run.smithi170.stdout:Enabling module streams: 2023-09-13T02:59:41.561 INFO:teuthology.orchestra.run.smithi170.stdout: container-tools rhel8 2023-09-13T02:59:41.561 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-09-13T02:59:41.561 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2023-09-13T02:59:41.562 INFO:teuthology.orchestra.run.smithi170.stdout:============================================================================================================ 2023-09-13T02:59:41.562 INFO:teuthology.orchestra.run.smithi170.stdout:Install 10 Packages 2023-09-13T02:59:41.562 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-09-13T02:59:41.563 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 18 M 2023-09-13T02:59:41.563 INFO:teuthology.orchestra.run.smithi170.stdout:Installed size: 65 M 2023-09-13T02:59:41.564 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2023-09-13T02:59:41.613 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2023-09-13T02:59:41.637 INFO:teuthology.orchestra.run.smithi170.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.2 MB/s | 94 kB 00:00 2023-09-13T02:59:41.662 INFO:teuthology.orchestra.run.smithi170.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 241 kB/s | 25 kB 00:00 2023-09-13T02:59:41.687 INFO:teuthology.orchestra.run.smithi170.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 318 kB/s | 41 kB 00:00 2023-09-13T02:59:41.706 INFO:teuthology.orchestra.run.smithi142.stdout:============================================================================================================ 2023-09-13T02:59:41.707 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2023-09-13T02:59:41.707 INFO:teuthology.orchestra.run.smithi142.stdout:============================================================================================================ 2023-09-13T02:59:41.708 INFO:teuthology.orchestra.run.smithi142.stdout:Installing group/module packages: 2023-09-13T02:59:41.708 INFO:teuthology.orchestra.run.smithi142.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-09-13T02:59:41.708 INFO:teuthology.orchestra.run.smithi142.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-09-13T02:59:41.708 INFO:teuthology.orchestra.run.smithi142.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-09-13T02:59:41.708 INFO:teuthology.orchestra.run.smithi142.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-09-13T02:59:41.709 INFO:teuthology.orchestra.run.smithi142.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-09-13T02:59:41.709 INFO:teuthology.orchestra.run.smithi142.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-09-13T02:59:41.709 INFO:teuthology.orchestra.run.smithi142.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-09-13T02:59:41.709 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2023-09-13T02:59:41.710 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-09-13T02:59:41.710 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-09-13T02:59:41.710 INFO:teuthology.orchestra.run.smithi142.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-09-13T02:59:41.710 INFO:teuthology.orchestra.run.smithi142.stdout:Installing module profiles: 2023-09-13T02:59:41.711 INFO:teuthology.orchestra.run.smithi142.stdout: container-tools/common 2023-09-13T02:59:41.711 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-09-13T02:59:41.711 INFO:teuthology.orchestra.run.smithi142.stdout:Enabling module streams: 2023-09-13T02:59:41.711 INFO:teuthology.orchestra.run.smithi142.stdout: container-tools rhel8 2023-09-13T02:59:41.712 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-09-13T02:59:41.712 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2023-09-13T02:59:41.712 INFO:teuthology.orchestra.run.smithi142.stdout:============================================================================================================ 2023-09-13T02:59:41.712 INFO:teuthology.orchestra.run.smithi142.stdout:Install 10 Packages 2023-09-13T02:59:41.712 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-09-13T02:59:41.713 INFO:teuthology.orchestra.run.smithi170.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 655 kB/s | 49 kB 00:00 2023-09-13T02:59:41.714 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 18 M 2023-09-13T02:59:41.714 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 65 M 2023-09-13T02:59:41.714 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2023-09-13T02:59:41.746 INFO:teuthology.orchestra.run.smithi170.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-09-13T02:59:41.771 INFO:teuthology.orchestra.run.smithi170.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-09-13T02:59:41.807 INFO:teuthology.orchestra.run.smithi170.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 3.5 MB/s | 213 kB 00:00 2023-09-13T02:59:41.890 INFO:teuthology.orchestra.run.smithi142.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 519 kB/s | 94 kB 00:00 2023-09-13T02:59:41.907 INFO:teuthology.orchestra.run.smithi142.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 125 kB/s | 25 kB 00:00 2023-09-13T02:59:41.914 INFO:teuthology.orchestra.run.smithi170.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 20 MB/s | 2.2 MB 00:00 2023-09-13T02:59:41.924 INFO:teuthology.orchestra.run.smithi142.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 190 kB/s | 41 kB 00:00 2023-09-13T02:59:41.941 INFO:teuthology.orchestra.run.smithi142.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 979 kB/s | 49 kB 00:00 2023-09-13T02:59:41.975 INFO:teuthology.orchestra.run.smithi142.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 7.6 MB/s | 524 kB 00:00 2023-09-13T02:59:42.000 INFO:teuthology.orchestra.run.smithi142.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-09-13T02:59:42.025 INFO:teuthology.orchestra.run.smithi142.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.1 MB/s | 213 kB 00:00 2023-09-13T02:59:42.039 INFO:teuthology.orchestra.run.smithi170.stdout:(9/10): buildah-1.26.2-1.module+el8.6.0+16771+2 23 MB/s | 8.0 MB 00:00 2023-09-13T02:59:42.122 INFO:teuthology.orchestra.run.smithi170.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 20 MB/s | 7.0 MB 00:00 2023-09-13T02:59:42.123 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2023-09-13T02:59:42.123 INFO:teuthology.orchestra.run.smithi170.stdout:Total 32 MB/s | 18 MB 00:00 2023-09-13T02:59:42.134 INFO:teuthology.orchestra.run.smithi142.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 20 MB/s | 2.2 MB 00:00 2023-09-13T02:59:42.251 INFO:teuthology.orchestra.run.smithi142.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 28 MB/s | 7.0 MB 00:00 2023-09-13T02:59:42.322 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2023-09-13T02:59:42.341 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2023-09-13T02:59:42.341 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2023-09-13T02:59:42.351 INFO:teuthology.orchestra.run.smithi142.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 19 MB/s | 8.0 MB 00:00 2023-09-13T02:59:42.351 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2023-09-13T02:59:42.352 INFO:teuthology.orchestra.run.smithi142.stdout:Total 28 MB/s | 18 MB 00:00 2023-09-13T02:59:42.524 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2023-09-13T02:59:42.528 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2023-09-13T02:59:42.552 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2023-09-13T02:59:42.571 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2023-09-13T02:59:42.571 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2023-09-13T02:59:42.754 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2023-09-13T02:59:42.758 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2023-09-13T02:59:42.872 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2023-09-13T02:59:42.977 INFO:teuthology.orchestra.run.smithi064.stdout:============================================================================================================ 2023-09-13T02:59:42.978 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2023-09-13T02:59:42.978 INFO:teuthology.orchestra.run.smithi064.stdout:============================================================================================================ 2023-09-13T02:59:42.978 INFO:teuthology.orchestra.run.smithi064.stdout:Installing group/module packages: 2023-09-13T02:59:42.979 INFO:teuthology.orchestra.run.smithi064.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-09-13T02:59:42.979 INFO:teuthology.orchestra.run.smithi064.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-09-13T02:59:42.979 INFO:teuthology.orchestra.run.smithi064.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-09-13T02:59:42.979 INFO:teuthology.orchestra.run.smithi064.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-09-13T02:59:42.979 INFO:teuthology.orchestra.run.smithi064.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-09-13T02:59:42.980 INFO:teuthology.orchestra.run.smithi064.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-09-13T02:59:42.980 INFO:teuthology.orchestra.run.smithi064.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-09-13T02:59:42.980 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2023-09-13T02:59:42.980 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-09-13T02:59:42.980 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-09-13T02:59:42.981 INFO:teuthology.orchestra.run.smithi064.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-09-13T02:59:42.981 INFO:teuthology.orchestra.run.smithi064.stdout:Installing module profiles: 2023-09-13T02:59:42.981 INFO:teuthology.orchestra.run.smithi064.stdout: container-tools/common 2023-09-13T02:59:42.981 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T02:59:42.981 INFO:teuthology.orchestra.run.smithi064.stdout:Enabling module streams: 2023-09-13T02:59:42.982 INFO:teuthology.orchestra.run.smithi064.stdout: container-tools rhel8 2023-09-13T02:59:42.982 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T02:59:42.982 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2023-09-13T02:59:42.982 INFO:teuthology.orchestra.run.smithi064.stdout:============================================================================================================ 2023-09-13T02:59:42.982 INFO:teuthology.orchestra.run.smithi064.stdout:Install 10 Packages 2023-09-13T02:59:42.983 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T02:59:42.983 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 18 M 2023-09-13T02:59:42.984 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 65 M 2023-09-13T02:59:42.984 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2023-09-13T02:59:43.044 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2023-09-13T02:59:43.053 INFO:teuthology.orchestra.run.smithi064.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.3 MB/s | 94 kB 00:00 2023-09-13T02:59:43.078 INFO:teuthology.orchestra.run.smithi064.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 256 kB/s | 25 kB 00:00 2023-09-13T02:59:43.104 INFO:teuthology.orchestra.run.smithi064.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 333 kB/s | 41 kB 00:00 2023-09-13T02:59:43.129 INFO:teuthology.orchestra.run.smithi064.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 655 kB/s | 49 kB 00:00 2023-09-13T02:59:43.179 INFO:teuthology.orchestra.run.smithi064.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 5.1 MB/s | 524 kB 00:00 2023-09-13T02:59:43.203 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-09-13T02:59:43.204 INFO:teuthology.orchestra.run.smithi064.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 1.9 MB/s | 149 kB 00:00 2023-09-13T02:59:43.238 INFO:teuthology.orchestra.run.smithi064.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 3.6 MB/s | 213 kB 00:00 2023-09-13T02:59:43.296 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2023-09-13T02:59:43.372 INFO:teuthology.orchestra.run.smithi064.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 16 MB/s | 2.2 MB 00:00 2023-09-13T02:59:43.395 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-13T02:59:43.518 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-09-13T02:59:43.547 INFO:teuthology.orchestra.run.smithi064.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 21 MB/s | 7.0 MB 00:00 2023-09-13T02:59:43.565 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-09-13T02:59:43.640 INFO:teuthology.orchestra.run.smithi064.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 15 MB/s | 8.0 MB 00:00 2023-09-13T02:59:43.640 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2023-09-13T02:59:43.641 INFO:teuthology.orchestra.run.smithi064.stdout:Total 28 MB/s | 18 MB 00:00 2023-09-13T02:59:43.682 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-13T02:59:43.774 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-09-13T02:59:43.926 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-09-13T02:59:43.982 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2023-09-13T02:59:44.006 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2023-09-13T02:59:44.007 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2023-09-13T02:59:44.115 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-09-13T02:59:44.120 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-09-13T02:59:44.493 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-09-13T02:59:44.516 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2023-09-13T02:59:44.520 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2023-09-13T02:59:44.848 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-09-13T02:59:45.084 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2023-09-13T02:59:45.258 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-09-13T02:59:45.281 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-09-13T02:59:45.403 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-13T02:59:45.616 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-09-13T02:59:45.710 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-09-13T02:59:45.839 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-09-13T02:59:45.923 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-09-13T02:59:46.077 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-09-13T02:59:46.183 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-09-13T02:59:46.219 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-09-13T02:59:46.237 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-13T02:59:46.417 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-09-13T02:59:46.504 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-13T02:59:46.504 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-09-13T02:59:46.504 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-13T02:59:46.505 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-09-13T02:59:46.505 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-09-13T02:59:46.505 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-09-13T02:59:46.505 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-09-13T02:59:46.506 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-09-13T02:59:46.506 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-09-13T02:59:46.506 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-09-13T02:59:46.605 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-09-13T02:59:46.824 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-13T02:59:47.059 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-09-13T02:59:47.167 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-13T02:59:47.167 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-09-13T02:59:47.167 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-13T02:59:47.168 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-09-13T02:59:47.168 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-09-13T02:59:47.168 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-09-13T02:59:47.168 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-09-13T02:59:47.169 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-09-13T02:59:47.169 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-09-13T02:59:47.169 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-09-13T02:59:47.626 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-09-13T02:59:47.627 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:05:04 ago on Wed 13 Sep 2023 02:54:43 AM UTC. 2023-09-13T02:59:48.013 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-09-13T02:59:48.266 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-09-13T02:59:48.341 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-09-13T02:59:48.341 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:05:03 ago on Wed 13 Sep 2023 02:54:44 AM UTC. 2023-09-13T02:59:48.425 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-09-13T02:59:48.623 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-13T02:59:49.011 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-13T02:59:49.011 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-09-13T02:59:49.011 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-13T02:59:49.011 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-09-13T02:59:49.012 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-09-13T02:59:49.012 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-09-13T02:59:49.012 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-09-13T02:59:49.012 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-09-13T02:59:49.012 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-09-13T02:59:49.013 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-09-13T02:59:50.434 INFO:teuthology.orchestra.run.smithi170.stdout:Installed products updated. 2023-09-13T02:59:50.542 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-09-13T02:59:50.543 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:02:05 ago on Wed 13 Sep 2023 02:57:44 AM UTC. 2023-09-13T02:59:50.638 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-09-13T02:59:50.638 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2023-09-13T02:59:50.639 INFO:teuthology.orchestra.run.smithi170.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-13T02:59:50.639 INFO:teuthology.orchestra.run.smithi170.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-09-13T02:59:50.639 INFO:teuthology.orchestra.run.smithi170.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-13T02:59:50.640 INFO:teuthology.orchestra.run.smithi170.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-09-13T02:59:50.640 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-09-13T02:59:50.640 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-09-13T02:59:50.640 INFO:teuthology.orchestra.run.smithi170.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-13T02:59:50.641 INFO:teuthology.orchestra.run.smithi170.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-09-13T02:59:50.641 INFO:teuthology.orchestra.run.smithi170.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-09-13T02:59:50.641 INFO:teuthology.orchestra.run.smithi170.stdout: yajl-2.1.0-10.el8.x86_64 2023-09-13T02:59:50.641 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-09-13T02:59:50.641 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2023-09-13T02:59:50.843 DEBUG:teuthology.parallel:result is None 2023-09-13T02:59:51.025 INFO:teuthology.orchestra.run.smithi142.stdout:Installed products updated. 2023-09-13T02:59:51.266 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-09-13T02:59:51.266 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2023-09-13T02:59:51.267 INFO:teuthology.orchestra.run.smithi142.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-13T02:59:51.267 INFO:teuthology.orchestra.run.smithi142.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-09-13T02:59:51.267 INFO:teuthology.orchestra.run.smithi142.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-13T02:59:51.267 INFO:teuthology.orchestra.run.smithi142.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-09-13T02:59:51.267 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-09-13T02:59:51.268 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-09-13T02:59:51.268 INFO:teuthology.orchestra.run.smithi142.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-13T02:59:51.268 INFO:teuthology.orchestra.run.smithi142.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-09-13T02:59:51.268 INFO:teuthology.orchestra.run.smithi142.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-09-13T02:59:51.269 INFO:teuthology.orchestra.run.smithi142.stdout: yajl-2.1.0-10.el8.x86_64 2023-09-13T02:59:51.269 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-09-13T02:59:51.269 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2023-09-13T02:59:51.429 DEBUG:teuthology.parallel:result is None 2023-09-13T02:59:53.439 INFO:teuthology.orchestra.run.smithi064.stdout:Installed products updated. 2023-09-13T02:59:53.654 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T02:59:53.655 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2023-09-13T02:59:53.655 INFO:teuthology.orchestra.run.smithi064.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-13T02:59:53.655 INFO:teuthology.orchestra.run.smithi064.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-09-13T02:59:53.656 INFO:teuthology.orchestra.run.smithi064.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-13T02:59:53.656 INFO:teuthology.orchestra.run.smithi064.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-09-13T02:59:53.656 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-09-13T02:59:53.656 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-09-13T02:59:53.656 INFO:teuthology.orchestra.run.smithi064.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-13T02:59:53.657 INFO:teuthology.orchestra.run.smithi064.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-09-13T02:59:53.657 INFO:teuthology.orchestra.run.smithi064.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-09-13T02:59:53.657 INFO:teuthology.orchestra.run.smithi064.stdout: yajl-2.1.0-10.el8.x86_64 2023-09-13T02:59:53.657 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T02:59:53.657 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2023-09-13T02:59:53.876 DEBUG:teuthology.parallel:result is None 2023-09-13T02:59:53.877 INFO:teuthology.run_tasks:Running task install... 2023-09-13T02:59:53.886 DEBUG:teuthology.task.install:project ceph 2023-09-13T02:59:53.886 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2110e00747b31a0d73768c9e5229da18b26b8aa0'}} 2023-09-13T02:59:53.887 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2110e00747b31a0d73768c9e5229da18b26b8aa0'} 2023-09-13T02:59:53.887 INFO:teuthology.task.install:Using flavor: default 2023-09-13T02:59:53.894 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-09-13T02:59:53.894 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-09-13T02:59:53.895 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2110e00747b31a0d73768c9e5229da18b26b8aa0', 'tag': None, 'wait_for_package': False} 2023-09-13T02:59:53.896 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2110e00747b31a0d73768c9e5229da18b26b8aa0 2023-09-13T02:59:53.897 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2110e00747b31a0d73768c9e5229da18b26b8aa0', 'tag': None, 'wait_for_package': False} 2023-09-13T02:59:53.898 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2110e00747b31a0d73768c9e5229da18b26b8aa0 2023-09-13T02:59:53.899 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2110e00747b31a0d73768c9e5229da18b26b8aa0', 'tag': None, 'wait_for_package': False} 2023-09-13T02:59:53.899 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2110e00747b31a0d73768c9e5229da18b26b8aa0 2023-09-13T02:59:54.112 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/centos/8/flavors/default/ 2023-09-13T02:59:54.113 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6088.g2110e007 2023-09-13T02:59:54.119 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/centos/8/flavors/default/ 2023-09-13T02:59:54.120 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6088.g2110e007 2023-09-13T02:59:54.127 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/centos/8/flavors/default/ 2023-09-13T02:59:54.128 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6088.g2110e007 2023-09-13T02:59:54.235 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-09-13T02:59:54.235 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-09-13T02:59:54.235 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-09-13T02:59:54.263 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-09-13T02:59:54.263 DEBUG:teuthology.orchestra.run.smithi142:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2110e00747b31a0d73768c9e5229da18b26b8aa0/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-09-13T02:59:54.287 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-09-13T02:59:54.288 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-13T02:59:54.288 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-09-13T02:59:54.296 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-09-13T02:59:54.296 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-09-13T02:59:54.296 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-09-13T02:59:54.322 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-09-13T02:59:54.323 DEBUG:teuthology.orchestra.run.smithi170:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2110e00747b31a0d73768c9e5229da18b26b8aa0/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-09-13T02:59:54.330 DEBUG:teuthology.orchestra.run.smithi142:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-09-13T02:59:54.358 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-09-13T02:59:54.358 DEBUG:teuthology.orchestra.run.smithi064:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2110e00747b31a0d73768c9e5229da18b26b8aa0/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-09-13T02:59:54.388 DEBUG:teuthology.orchestra.run.smithi170:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-09-13T02:59:54.406 DEBUG:teuthology.orchestra.run.smithi142:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-09-13T02:59:54.467 DEBUG:teuthology.orchestra.run.smithi170:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-09-13T02:59:54.468 DEBUG:teuthology.orchestra.run.smithi064:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-09-13T02:59:54.472 INFO:teuthology.orchestra.run.smithi142.stdout:check_obsoletes = 1 2023-09-13T02:59:54.473 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum clean all 2023-09-13T02:59:54.532 INFO:teuthology.orchestra.run.smithi170.stdout:check_obsoletes = 1 2023-09-13T02:59:54.534 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum clean all 2023-09-13T02:59:54.566 DEBUG:teuthology.orchestra.run.smithi064:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-09-13T02:59:54.635 INFO:teuthology.orchestra.run.smithi064.stdout:check_obsoletes = 1 2023-09-13T02:59:54.638 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum clean all 2023-09-13T02:59:54.915 INFO:teuthology.orchestra.run.smithi142.stdout:Updating Subscription Management repositories. 2023-09-13T02:59:54.922 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2023-09-13T02:59:55.119 INFO:teuthology.orchestra.run.smithi064.stdout:Updating Subscription Management repositories. 2023-09-13T02:59:55.198 INFO:teuthology.orchestra.run.smithi142.stdout:44 files removed 2023-09-13T02:59:55.208 INFO:teuthology.orchestra.run.smithi170.stdout:44 files removed 2023-09-13T02:59:55.254 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-09-13T02:59:55.262 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-09-13T02:59:55.479 INFO:teuthology.orchestra.run.smithi064.stdout:44 files removed 2023-09-13T02:59:55.542 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-09-13T02:59:55.617 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2023-09-13T02:59:55.621 INFO:teuthology.orchestra.run.smithi142.stdout:Updating Subscription Management repositories. 2023-09-13T02:59:56.026 INFO:teuthology.orchestra.run.smithi064.stdout:Updating Subscription Management repositories. 2023-09-13T02:59:56.423 INFO:teuthology.orchestra.run.smithi170.stdout:ceph packages for x86_64 313 kB/s | 81 kB 00:00 2023-09-13T02:59:56.499 INFO:teuthology.orchestra.run.smithi142.stdout:ceph packages for x86_64 252 kB/s | 81 kB 00:00 2023-09-13T02:59:56.708 INFO:teuthology.orchestra.run.smithi170.stdout:ceph noarch packages 37 kB/s | 9.4 kB 00:00 2023-09-13T02:59:56.742 INFO:teuthology.orchestra.run.smithi142.stdout:ceph noarch packages 45 kB/s | 9.4 kB 00:00 2023-09-13T02:59:56.782 INFO:teuthology.orchestra.run.smithi064.stdout:ceph packages for x86_64 301 kB/s | 81 kB 00:00 2023-09-13T02:59:57.026 INFO:teuthology.orchestra.run.smithi170.stdout:ceph source packages 5.0 kB/s | 1.5 kB 00:00 2023-09-13T02:59:57.127 INFO:teuthology.orchestra.run.smithi142.stdout:ceph source packages 4.1 kB/s | 1.5 kB 00:00 2023-09-13T02:59:57.150 INFO:teuthology.orchestra.run.smithi064.stdout:ceph noarch packages 28 kB/s | 9.4 kB 00:00 2023-09-13T02:59:57.219 INFO:teuthology.orchestra.run.smithi170.stdout:Copr repo for python3-asyncssh owned by ceph 20 kB/s | 3.5 kB 00:00 2023-09-13T02:59:57.277 INFO:teuthology.orchestra.run.smithi142.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2023-09-13T02:59:57.442 INFO:teuthology.orchestra.run.smithi064.stdout:ceph source packages 5.5 kB/s | 1.5 kB 00:00 2023-09-13T02:59:57.584 INFO:teuthology.orchestra.run.smithi064.stdout:Copr repo for python3-asyncssh owned by ceph 29 kB/s | 3.5 kB 00:00 2023-09-13T02:59:58.171 INFO:teuthology.orchestra.run.smithi142.stdout:Extra Packages for Enterprise Linux 18 MB/s | 16 MB 00:00 2023-09-13T02:59:59.008 INFO:teuthology.orchestra.run.smithi170.stdout:Extra Packages for Enterprise Linux 9.0 MB/s | 16 MB 00:01 2023-09-13T03:00:01.897 INFO:teuthology.orchestra.run.smithi064.stdout:Extra Packages for Enterprise Linux 3.7 MB/s | 16 MB 00:04 2023-09-13T03:00:01.909 INFO:teuthology.orchestra.run.smithi142.stdout:lab-extras 279 kB/s | 24 kB 00:00 2023-09-13T03:00:02.730 INFO:teuthology.orchestra.run.smithi170.stdout:lab-extras 500 kB/s | 24 kB 00:00 2023-09-13T03:00:03.176 INFO:teuthology.orchestra.run.smithi142.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 45 MB/s | 53 MB 00:01 2023-09-13T03:00:03.690 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 53 MB/s | 47 MB 00:00 2023-09-13T03:00:06.202 INFO:teuthology.orchestra.run.smithi064.stdout:lab-extras 332 kB/s | 24 kB 00:00 2023-09-13T03:00:07.379 INFO:teuthology.orchestra.run.smithi064.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 43 MB/s | 47 MB 00:01 2023-09-13T03:00:11.467 INFO:teuthology.orchestra.run.smithi142.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 48 MB/s | 47 MB 00:00 2023-09-13T03:00:11.740 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 50 MB/s | 53 MB 00:01 2023-09-13T03:00:16.847 INFO:teuthology.orchestra.run.smithi064.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 35 MB/s | 53 MB 00:01 2023-09-13T03:00:22.486 INFO:teuthology.orchestra.run.smithi142.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-13T03:00:22.487 INFO:teuthology.orchestra.run.smithi142.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-13T03:00:22.501 INFO:teuthology.orchestra.run.smithi142.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-09-13T03:00:22.502 INFO:teuthology.orchestra.run.smithi142.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-09-13T03:00:22.503 INFO:teuthology.orchestra.run.smithi142.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-09-13T03:00:22.505 INFO:teuthology.orchestra.run.smithi142.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-09-13T03:00:22.507 INFO:teuthology.orchestra.run.smithi142.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-09-13T03:00:22.510 INFO:teuthology.orchestra.run.smithi142.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-09-13T03:00:22.512 INFO:teuthology.orchestra.run.smithi142.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-09-13T03:00:22.645 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2023-09-13T03:00:22.657 INFO:teuthology.orchestra.run.smithi142.stdout:=========================================================================================================================== 2023-09-13T03:00:22.658 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2023-09-13T03:00:22.658 INFO:teuthology.orchestra.run.smithi142.stdout:=========================================================================================================================== 2023-09-13T03:00:22.658 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2023-09-13T03:00:22.659 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-09-13T03:00:22.659 INFO:teuthology.orchestra.run.smithi142.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-09-13T03:00:22.659 INFO:teuthology.orchestra.run.smithi142.stdout: ceph x86_64 2:18.0.0-6088.g2110e007.el8 ceph 6.4 k 2023-09-13T03:00:22.659 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-base x86_64 2:18.0.0-6088.g2110e007.el8 ceph 5.4 M 2023-09-13T03:00:22.660 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-fuse x86_64 2:18.0.0-6088.g2110e007.el8 ceph 907 k 2023-09-13T03:00:22.660 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6088.g2110e007.el8 ceph 167 k 2023-09-13T03:00:22.660 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr x86_64 2:18.0.0-6088.g2110e007.el8 ceph 1.6 M 2023-09-13T03:00:22.661 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 132 k 2023-09-13T03:00:22.661 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 1.7 M 2023-09-13T03:00:22.661 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 7.4 M 2023-09-13T03:00:22.661 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-rook noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 47 k 2023-09-13T03:00:22.662 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-radosgw x86_64 2:18.0.0-6088.g2110e007.el8 ceph 14 M 2023-09-13T03:00:22.662 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-test x86_64 2:18.0.0-6088.g2110e007.el8 ceph 49 M 2023-09-13T03:00:22.662 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-volume noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 261 k 2023-09-13T03:00:22.662 INFO:teuthology.orchestra.run.smithi142.stdout: cephadm noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 177 k 2023-09-13T03:00:22.663 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-mirror x86_64 2:18.0.0-6088.g2110e007.el8 ceph 261 k 2023-09-13T03:00:22.663 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-top noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 18 k 2023-09-13T03:00:22.663 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-09-13T03:00:22.663 INFO:teuthology.orchestra.run.smithi142.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-09-13T03:00:22.664 INFO:teuthology.orchestra.run.smithi142.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-09-13T03:00:22.664 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs-devel x86_64 2:18.0.0-6088.g2110e007.el8 ceph 31 k 2023-09-13T03:00:22.664 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs2 x86_64 2:18.0.0-6088.g2110e007.el8 ceph 827 k 2023-09-13T03:00:22.664 INFO:teuthology.orchestra.run.smithi142.stdout: librados-devel x86_64 2:18.0.0-6088.g2110e007.el8 ceph 139 k 2023-09-13T03:00:22.664 INFO:teuthology.orchestra.run.smithi142.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-09-13T03:00:22.665 INFO:teuthology.orchestra.run.smithi142.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-09-13T03:00:22.665 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-09-13T03:00:22.665 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-09-13T03:00:22.665 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-09-13T03:00:22.666 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-09-13T03:00:22.666 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cephfs x86_64 2:18.0.0-6088.g2110e007.el8 ceph 197 k 2023-09-13T03:00:22.666 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rados x86_64 2:18.0.0-6088.g2110e007.el8 ceph 371 k 2023-09-13T03:00:22.666 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rbd x86_64 2:18.0.0-6088.g2110e007.el8 ceph 364 k 2023-09-13T03:00:22.667 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rgw x86_64 2:18.0.0-6088.g2110e007.el8 ceph 106 k 2023-09-13T03:00:22.667 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-fuse x86_64 2:18.0.0-6088.g2110e007.el8 ceph 94 k 2023-09-13T03:00:22.667 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-mirror x86_64 2:18.0.0-6088.g2110e007.el8 ceph 3.6 M 2023-09-13T03:00:22.667 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-nbd x86_64 2:18.0.0-6088.g2110e007.el8 ceph 179 k 2023-09-13T03:00:22.668 INFO:teuthology.orchestra.run.smithi142.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-09-13T03:00:22.668 INFO:teuthology.orchestra.run.smithi142.stdout:Upgrading: 2023-09-13T03:00:22.668 INFO:teuthology.orchestra.run.smithi142.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-09-13T03:00:22.668 INFO:teuthology.orchestra.run.smithi142.stdout: librados2 x86_64 2:18.0.0-6088.g2110e007.el8 ceph 3.7 M 2023-09-13T03:00:22.669 INFO:teuthology.orchestra.run.smithi142.stdout: librbd1 x86_64 2:18.0.0-6088.g2110e007.el8 ceph 3.7 M 2023-09-13T03:00:22.669 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2023-09-13T03:00:22.669 INFO:teuthology.orchestra.run.smithi142.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-09-13T03:00:22.669 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-common x86_64 2:18.0.0-6088.g2110e007.el8 ceph 22 M 2023-09-13T03:00:22.669 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 23 k 2023-09-13T03:00:22.670 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mds x86_64 2:18.0.0-6088.g2110e007.el8 ceph 2.3 M 2023-09-13T03:00:22.670 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 242 k 2023-09-13T03:00:22.670 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mon x86_64 2:18.0.0-6088.g2110e007.el8 ceph 4.6 M 2023-09-13T03:00:22.670 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-osd x86_64 2:18.0.0-6088.g2110e007.el8 ceph 18 M 2023-09-13T03:00:22.670 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 15 k 2023-09-13T03:00:22.671 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-selinux x86_64 2:18.0.0-6088.g2110e007.el8 ceph 24 k 2023-09-13T03:00:22.671 INFO:teuthology.orchestra.run.smithi142.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-09-13T03:00:22.671 INFO:teuthology.orchestra.run.smithi142.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-09-13T03:00:22.671 INFO:teuthology.orchestra.run.smithi142.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-09-13T03:00:22.671 INFO:teuthology.orchestra.run.smithi142.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-09-13T03:00:22.672 INFO:teuthology.orchestra.run.smithi142.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-09-13T03:00:22.672 INFO:teuthology.orchestra.run.smithi142.stdout: libcephsqlite x86_64 2:18.0.0-6088.g2110e007.el8 ceph 184 k 2023-09-13T03:00:22.672 INFO:teuthology.orchestra.run.smithi142.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-09-13T03:00:22.672 INFO:teuthology.orchestra.run.smithi142.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-09-13T03:00:22.673 INFO:teuthology.orchestra.run.smithi142.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-09-13T03:00:22.673 INFO:teuthology.orchestra.run.smithi142.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-09-13T03:00:22.673 INFO:teuthology.orchestra.run.smithi142.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-09-13T03:00:22.673 INFO:teuthology.orchestra.run.smithi142.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-09-13T03:00:22.674 INFO:teuthology.orchestra.run.smithi142.stdout: libradosstriper1 x86_64 2:18.0.0-6088.g2110e007.el8 ceph 519 k 2023-09-13T03:00:22.674 INFO:teuthology.orchestra.run.smithi142.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-09-13T03:00:22.674 INFO:teuthology.orchestra.run.smithi142.stdout: librgw2 x86_64 2:18.0.0-6088.g2110e007.el8 ceph 6.8 M 2023-09-13T03:00:22.674 INFO:teuthology.orchestra.run.smithi142.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-09-13T03:00:22.674 INFO:teuthology.orchestra.run.smithi142.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-09-13T03:00:22.675 INFO:teuthology.orchestra.run.smithi142.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-09-13T03:00:22.675 INFO:teuthology.orchestra.run.smithi142.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-09-13T03:00:22.675 INFO:teuthology.orchestra.run.smithi142.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-09-13T03:00:22.676 INFO:teuthology.orchestra.run.smithi142.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-09-13T03:00:22.676 INFO:teuthology.orchestra.run.smithi142.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-09-13T03:00:22.676 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-09-13T03:00:22.676 INFO:teuthology.orchestra.run.smithi142.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-09-13T03:00:22.676 INFO:teuthology.orchestra.run.smithi142.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-09-13T03:00:22.677 INFO:teuthology.orchestra.run.smithi142.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-09-13T03:00:22.677 INFO:teuthology.orchestra.run.smithi142.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-09-13T03:00:22.677 INFO:teuthology.orchestra.run.smithi142.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-09-13T03:00:22.677 INFO:teuthology.orchestra.run.smithi142.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-09-13T03:00:22.678 INFO:teuthology.orchestra.run.smithi142.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-09-13T03:00:22.678 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-09-13T03:00:22.678 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-argparse x86_64 2:18.0.0-6088.g2110e007.el8 ceph 45 k 2023-09-13T03:00:22.678 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-common x86_64 2:18.0.0-6088.g2110e007.el8 ceph 125 k 2023-09-13T03:00:22.679 INFO:teuthology.orchestra.run.smithi142.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-09-13T03:00:22.679 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-09-13T03:00:22.679 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-09-13T03:00:22.679 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-09-13T03:00:22.680 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-09-13T03:00:22.680 INFO:teuthology.orchestra.run.smithi142.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-09-13T03:00:22.680 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-09-13T03:00:22.680 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-09-13T03:00:22.681 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-09-13T03:00:22.681 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-09-13T03:00:22.681 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-09-13T03:00:22.681 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-13T03:00:22.682 INFO:teuthology.orchestra.run.smithi142.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-09-13T03:00:22.682 INFO:teuthology.orchestra.run.smithi142.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-09-13T03:00:22.682 INFO:teuthology.orchestra.run.smithi142.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-09-13T03:00:22.682 INFO:teuthology.orchestra.run.smithi142.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-09-13T03:00:22.683 INFO:teuthology.orchestra.run.smithi142.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-09-13T03:00:22.683 INFO:teuthology.orchestra.run.smithi142.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-09-13T03:00:22.683 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-09-13T03:00:22.683 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-09-13T03:00:22.683 INFO:teuthology.orchestra.run.smithi142.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-09-13T03:00:22.684 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-09-13T03:00:22.684 INFO:teuthology.orchestra.run.smithi142.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-09-13T03:00:22.684 INFO:teuthology.orchestra.run.smithi142.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-09-13T03:00:22.684 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-09-13T03:00:22.685 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-09-13T03:00:22.685 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-09-13T03:00:22.685 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-09-13T03:00:22.685 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-09-13T03:00:22.686 INFO:teuthology.orchestra.run.smithi142.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-09-13T03:00:22.686 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-13T03:00:22.686 INFO:teuthology.orchestra.run.smithi142.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-09-13T03:00:22.686 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-09-13T03:00:22.686 INFO:teuthology.orchestra.run.smithi142.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-09-13T03:00:22.687 INFO:teuthology.orchestra.run.smithi142.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-09-13T03:00:22.687 INFO:teuthology.orchestra.run.smithi142.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-09-13T03:00:22.687 INFO:teuthology.orchestra.run.smithi142.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-09-13T03:00:22.687 INFO:teuthology.orchestra.run.smithi142.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-09-13T03:00:22.687 INFO:teuthology.orchestra.run.smithi142.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-09-13T03:00:22.688 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-09-13T03:00:22.688 INFO:teuthology.orchestra.run.smithi142.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-09-13T03:00:22.688 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-09-13T03:00:22.688 INFO:teuthology.orchestra.run.smithi142.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-09-13T03:00:22.689 INFO:teuthology.orchestra.run.smithi142.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-09-13T03:00:22.689 INFO:teuthology.orchestra.run.smithi142.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-09-13T03:00:22.689 INFO:teuthology.orchestra.run.smithi142.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-09-13T03:00:22.689 INFO:teuthology.orchestra.run.smithi142.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-09-13T03:00:22.689 INFO:teuthology.orchestra.run.smithi142.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-09-13T03:00:22.690 INFO:teuthology.orchestra.run.smithi142.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-09-13T03:00:22.690 INFO:teuthology.orchestra.run.smithi142.stdout:Enabling module streams: 2023-09-13T03:00:22.690 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql 10 2023-09-13T03:00:22.690 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-09-13T03:00:22.691 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2023-09-13T03:00:22.691 INFO:teuthology.orchestra.run.smithi142.stdout:=========================================================================================================================== 2023-09-13T03:00:22.691 INFO:teuthology.orchestra.run.smithi142.stdout:Install 128 Packages 2023-09-13T03:00:22.691 INFO:teuthology.orchestra.run.smithi142.stdout:Upgrade 3 Packages 2023-09-13T03:00:22.691 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-09-13T03:00:22.692 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 208 M 2023-09-13T03:00:22.692 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2023-09-13T03:00:22.796 INFO:teuthology.orchestra.run.smithi170.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-13T03:00:22.797 INFO:teuthology.orchestra.run.smithi170.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-13T03:00:22.812 INFO:teuthology.orchestra.run.smithi170.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-09-13T03:00:22.813 INFO:teuthology.orchestra.run.smithi170.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-09-13T03:00:22.814 INFO:teuthology.orchestra.run.smithi170.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-09-13T03:00:22.815 INFO:teuthology.orchestra.run.smithi170.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-09-13T03:00:22.817 INFO:teuthology.orchestra.run.smithi170.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-09-13T03:00:22.821 INFO:teuthology.orchestra.run.smithi170.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-09-13T03:00:22.822 INFO:teuthology.orchestra.run.smithi170.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-09-13T03:00:22.955 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2023-09-13T03:00:22.967 INFO:teuthology.orchestra.run.smithi170.stdout:=========================================================================================================================== 2023-09-13T03:00:22.969 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repository Size 2023-09-13T03:00:22.969 INFO:teuthology.orchestra.run.smithi170.stdout:=========================================================================================================================== 2023-09-13T03:00:22.970 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2023-09-13T03:00:22.970 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-09-13T03:00:22.970 INFO:teuthology.orchestra.run.smithi170.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-09-13T03:00:22.970 INFO:teuthology.orchestra.run.smithi170.stdout: ceph x86_64 2:18.0.0-6088.g2110e007.el8 ceph 6.4 k 2023-09-13T03:00:22.971 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-base x86_64 2:18.0.0-6088.g2110e007.el8 ceph 5.4 M 2023-09-13T03:00:22.971 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-fuse x86_64 2:18.0.0-6088.g2110e007.el8 ceph 907 k 2023-09-13T03:00:22.971 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6088.g2110e007.el8 ceph 167 k 2023-09-13T03:00:22.971 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr x86_64 2:18.0.0-6088.g2110e007.el8 ceph 1.6 M 2023-09-13T03:00:22.971 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 132 k 2023-09-13T03:00:22.972 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 1.7 M 2023-09-13T03:00:22.972 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 7.4 M 2023-09-13T03:00:22.972 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-rook noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 47 k 2023-09-13T03:00:22.972 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-radosgw x86_64 2:18.0.0-6088.g2110e007.el8 ceph 14 M 2023-09-13T03:00:22.972 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-test x86_64 2:18.0.0-6088.g2110e007.el8 ceph 49 M 2023-09-13T03:00:22.973 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-volume noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 261 k 2023-09-13T03:00:22.973 INFO:teuthology.orchestra.run.smithi170.stdout: cephadm noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 177 k 2023-09-13T03:00:22.973 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-mirror x86_64 2:18.0.0-6088.g2110e007.el8 ceph 261 k 2023-09-13T03:00:22.973 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-top noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 18 k 2023-09-13T03:00:22.974 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-09-13T03:00:22.974 INFO:teuthology.orchestra.run.smithi170.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-09-13T03:00:22.974 INFO:teuthology.orchestra.run.smithi170.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-09-13T03:00:22.974 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs-devel x86_64 2:18.0.0-6088.g2110e007.el8 ceph 31 k 2023-09-13T03:00:22.974 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs2 x86_64 2:18.0.0-6088.g2110e007.el8 ceph 827 k 2023-09-13T03:00:22.975 INFO:teuthology.orchestra.run.smithi170.stdout: librados-devel x86_64 2:18.0.0-6088.g2110e007.el8 ceph 139 k 2023-09-13T03:00:22.975 INFO:teuthology.orchestra.run.smithi170.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-09-13T03:00:22.975 INFO:teuthology.orchestra.run.smithi170.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-09-13T03:00:22.975 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-09-13T03:00:22.975 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-09-13T03:00:22.975 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-09-13T03:00:22.976 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-09-13T03:00:22.976 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cephfs x86_64 2:18.0.0-6088.g2110e007.el8 ceph 197 k 2023-09-13T03:00:22.976 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rados x86_64 2:18.0.0-6088.g2110e007.el8 ceph 371 k 2023-09-13T03:00:22.976 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rbd x86_64 2:18.0.0-6088.g2110e007.el8 ceph 364 k 2023-09-13T03:00:22.976 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rgw x86_64 2:18.0.0-6088.g2110e007.el8 ceph 106 k 2023-09-13T03:00:22.977 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-fuse x86_64 2:18.0.0-6088.g2110e007.el8 ceph 94 k 2023-09-13T03:00:22.977 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-mirror x86_64 2:18.0.0-6088.g2110e007.el8 ceph 3.6 M 2023-09-13T03:00:22.977 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-nbd x86_64 2:18.0.0-6088.g2110e007.el8 ceph 179 k 2023-09-13T03:00:22.977 INFO:teuthology.orchestra.run.smithi170.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-09-13T03:00:22.977 INFO:teuthology.orchestra.run.smithi170.stdout:Upgrading: 2023-09-13T03:00:22.978 INFO:teuthology.orchestra.run.smithi170.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-09-13T03:00:22.978 INFO:teuthology.orchestra.run.smithi170.stdout: librados2 x86_64 2:18.0.0-6088.g2110e007.el8 ceph 3.7 M 2023-09-13T03:00:22.978 INFO:teuthology.orchestra.run.smithi170.stdout: librbd1 x86_64 2:18.0.0-6088.g2110e007.el8 ceph 3.7 M 2023-09-13T03:00:22.978 INFO:teuthology.orchestra.run.smithi170.stdout:Installing dependencies: 2023-09-13T03:00:22.979 INFO:teuthology.orchestra.run.smithi170.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-09-13T03:00:22.979 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-common x86_64 2:18.0.0-6088.g2110e007.el8 ceph 22 M 2023-09-13T03:00:22.979 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 23 k 2023-09-13T03:00:22.979 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mds x86_64 2:18.0.0-6088.g2110e007.el8 ceph 2.3 M 2023-09-13T03:00:22.979 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 242 k 2023-09-13T03:00:22.979 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mon x86_64 2:18.0.0-6088.g2110e007.el8 ceph 4.6 M 2023-09-13T03:00:22.980 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-osd x86_64 2:18.0.0-6088.g2110e007.el8 ceph 18 M 2023-09-13T03:00:22.980 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 15 k 2023-09-13T03:00:22.980 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-selinux x86_64 2:18.0.0-6088.g2110e007.el8 ceph 24 k 2023-09-13T03:00:22.980 INFO:teuthology.orchestra.run.smithi170.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-09-13T03:00:22.980 INFO:teuthology.orchestra.run.smithi170.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-09-13T03:00:22.981 INFO:teuthology.orchestra.run.smithi170.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-09-13T03:00:22.981 INFO:teuthology.orchestra.run.smithi170.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-09-13T03:00:22.981 INFO:teuthology.orchestra.run.smithi170.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-09-13T03:00:22.981 INFO:teuthology.orchestra.run.smithi170.stdout: libcephsqlite x86_64 2:18.0.0-6088.g2110e007.el8 ceph 184 k 2023-09-13T03:00:22.982 INFO:teuthology.orchestra.run.smithi170.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-09-13T03:00:22.982 INFO:teuthology.orchestra.run.smithi170.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-09-13T03:00:22.982 INFO:teuthology.orchestra.run.smithi170.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-09-13T03:00:22.982 INFO:teuthology.orchestra.run.smithi170.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-09-13T03:00:22.982 INFO:teuthology.orchestra.run.smithi170.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-09-13T03:00:22.983 INFO:teuthology.orchestra.run.smithi170.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-09-13T03:00:22.983 INFO:teuthology.orchestra.run.smithi170.stdout: libradosstriper1 x86_64 2:18.0.0-6088.g2110e007.el8 ceph 519 k 2023-09-13T03:00:22.983 INFO:teuthology.orchestra.run.smithi170.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-09-13T03:00:22.983 INFO:teuthology.orchestra.run.smithi170.stdout: librgw2 x86_64 2:18.0.0-6088.g2110e007.el8 ceph 6.8 M 2023-09-13T03:00:22.984 INFO:teuthology.orchestra.run.smithi170.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-09-13T03:00:22.984 INFO:teuthology.orchestra.run.smithi170.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-09-13T03:00:22.984 INFO:teuthology.orchestra.run.smithi170.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-09-13T03:00:22.984 INFO:teuthology.orchestra.run.smithi170.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-09-13T03:00:22.985 INFO:teuthology.orchestra.run.smithi170.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-09-13T03:00:22.985 INFO:teuthology.orchestra.run.smithi170.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-09-13T03:00:22.985 INFO:teuthology.orchestra.run.smithi170.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-09-13T03:00:22.985 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-09-13T03:00:22.985 INFO:teuthology.orchestra.run.smithi170.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-09-13T03:00:22.986 INFO:teuthology.orchestra.run.smithi170.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-09-13T03:00:22.986 INFO:teuthology.orchestra.run.smithi170.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-09-13T03:00:22.986 INFO:teuthology.orchestra.run.smithi170.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-09-13T03:00:22.986 INFO:teuthology.orchestra.run.smithi170.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-09-13T03:00:22.986 INFO:teuthology.orchestra.run.smithi170.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-09-13T03:00:22.987 INFO:teuthology.orchestra.run.smithi170.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-09-13T03:00:22.987 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-09-13T03:00:22.987 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-argparse x86_64 2:18.0.0-6088.g2110e007.el8 ceph 45 k 2023-09-13T03:00:22.987 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-common x86_64 2:18.0.0-6088.g2110e007.el8 ceph 125 k 2023-09-13T03:00:22.987 INFO:teuthology.orchestra.run.smithi170.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-09-13T03:00:22.988 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-09-13T03:00:22.988 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-09-13T03:00:22.988 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-09-13T03:00:22.988 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-09-13T03:00:22.988 INFO:teuthology.orchestra.run.smithi170.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-09-13T03:00:22.989 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-09-13T03:00:22.989 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-09-13T03:00:22.989 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-09-13T03:00:22.989 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-09-13T03:00:22.989 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-09-13T03:00:22.990 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-13T03:00:22.990 INFO:teuthology.orchestra.run.smithi170.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-09-13T03:00:22.990 INFO:teuthology.orchestra.run.smithi170.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-09-13T03:00:22.990 INFO:teuthology.orchestra.run.smithi170.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-09-13T03:00:22.990 INFO:teuthology.orchestra.run.smithi170.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-09-13T03:00:22.991 INFO:teuthology.orchestra.run.smithi170.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-09-13T03:00:22.991 INFO:teuthology.orchestra.run.smithi170.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-09-13T03:00:22.991 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-09-13T03:00:22.991 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-09-13T03:00:22.991 INFO:teuthology.orchestra.run.smithi170.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-09-13T03:00:22.992 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-09-13T03:00:22.992 INFO:teuthology.orchestra.run.smithi170.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-09-13T03:00:22.992 INFO:teuthology.orchestra.run.smithi170.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-09-13T03:00:22.993 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-09-13T03:00:22.993 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-09-13T03:00:22.993 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-09-13T03:00:22.993 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-09-13T03:00:22.993 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-09-13T03:00:22.994 INFO:teuthology.orchestra.run.smithi170.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-09-13T03:00:22.994 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-13T03:00:22.994 INFO:teuthology.orchestra.run.smithi170.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-09-13T03:00:22.994 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-09-13T03:00:22.994 INFO:teuthology.orchestra.run.smithi170.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-09-13T03:00:22.995 INFO:teuthology.orchestra.run.smithi170.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-09-13T03:00:22.995 INFO:teuthology.orchestra.run.smithi170.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-09-13T03:00:22.995 INFO:teuthology.orchestra.run.smithi170.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-09-13T03:00:22.995 INFO:teuthology.orchestra.run.smithi170.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-09-13T03:00:22.996 INFO:teuthology.orchestra.run.smithi170.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-09-13T03:00:22.996 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-09-13T03:00:22.996 INFO:teuthology.orchestra.run.smithi170.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-09-13T03:00:22.996 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-09-13T03:00:22.997 INFO:teuthology.orchestra.run.smithi170.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-09-13T03:00:22.997 INFO:teuthology.orchestra.run.smithi170.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-09-13T03:00:22.997 INFO:teuthology.orchestra.run.smithi170.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-09-13T03:00:22.997 INFO:teuthology.orchestra.run.smithi170.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-09-13T03:00:22.998 INFO:teuthology.orchestra.run.smithi170.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-09-13T03:00:22.998 INFO:teuthology.orchestra.run.smithi170.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-09-13T03:00:22.998 INFO:teuthology.orchestra.run.smithi170.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-09-13T03:00:22.998 INFO:teuthology.orchestra.run.smithi170.stdout:Enabling module streams: 2023-09-13T03:00:22.999 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql 10 2023-09-13T03:00:22.999 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-09-13T03:00:22.999 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2023-09-13T03:00:22.999 INFO:teuthology.orchestra.run.smithi170.stdout:=========================================================================================================================== 2023-09-13T03:00:22.999 INFO:teuthology.orchestra.run.smithi170.stdout:Install 128 Packages 2023-09-13T03:00:22.999 INFO:teuthology.orchestra.run.smithi170.stdout:Upgrade 3 Packages 2023-09-13T03:00:23.000 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-09-13T03:00:23.001 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 208 M 2023-09-13T03:00:23.001 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2023-09-13T03:00:23.168 INFO:teuthology.orchestra.run.smithi142.stdout:(1/131): ceph-18.0.0-6088.g2110e007.el8.x86_64. 35 kB/s | 6.4 kB 00:00 2023-09-13T03:00:24.260 INFO:teuthology.orchestra.run.smithi142.stdout:(2/131): ceph-common-18.0.0-6088.g2110e007.el8. 18 MB/s | 22 MB 00:01 2023-09-13T03:00:24.294 INFO:teuthology.orchestra.run.smithi142.stdout:(3/131): ceph-fuse-18.0.0-6088.g2110e007.el8.x8 806 kB/s | 907 kB 00:01 2023-09-13T03:00:24.344 INFO:teuthology.orchestra.run.smithi142.stdout:(4/131): ceph-immutable-object-cache-18.0.0-608 2.0 MB/s | 167 kB 00:00 2023-09-13T03:00:24.462 INFO:teuthology.orchestra.run.smithi142.stdout:(5/131): ceph-mgr-18.0.0-6088.g2110e007.el8.x86 14 MB/s | 1.6 MB 00:00 2023-09-13T03:00:24.500 INFO:teuthology.orchestra.run.smithi170.stdout:(1/131): ceph-18.0.0-6088.g2110e007.el8.x86_64. 44 kB/s | 6.4 kB 00:00 2023-09-13T03:00:24.744 INFO:teuthology.orchestra.run.smithi142.stdout:(6/131): ceph-mon-18.0.0-6088.g2110e007.el8.x86 16 MB/s | 4.6 MB 00:00 2023-09-13T03:00:24.920 INFO:teuthology.orchestra.run.smithi142.stdout:(7/131): ceph-base-18.0.0-6088.g2110e007.el8.x8 2.8 MB/s | 5.4 MB 00:01 2023-09-13T03:00:25.317 INFO:teuthology.orchestra.run.smithi170.stdout:(2/131): ceph-fuse-18.0.0-6088.g2110e007.el8.x8 1.1 MB/s | 907 kB 00:00 2023-09-13T03:00:25.412 INFO:teuthology.orchestra.run.smithi142.stdout:(8/131): ceph-mds-18.0.0-6088.g2110e007.el8.x86 2.1 MB/s | 2.3 MB 00:01 2023-09-13T03:00:25.459 INFO:teuthology.orchestra.run.smithi170.stdout:(3/131): ceph-immutable-object-cache-18.0.0-608 1.2 MB/s | 167 kB 00:00 2023-09-13T03:00:25.520 INFO:teuthology.orchestra.run.smithi142.stdout:(9/131): ceph-selinux-18.0.0-6088.g2110e007.el8 224 kB/s | 24 kB 00:00 2023-09-13T03:00:25.874 INFO:teuthology.orchestra.run.smithi142.stdout:(10/131): ceph-osd-18.0.0-6088.g2110e007.el8.x8 16 MB/s | 18 MB 00:01 2023-09-13T03:00:25.954 INFO:teuthology.orchestra.run.smithi142.stdout:(11/131): cephfs-mirror-18.0.0-6088.g2110e007.e 3.2 MB/s | 261 kB 00:00 2023-09-13T03:00:26.013 INFO:teuthology.orchestra.run.smithi142.stdout:(12/131): libcephfs-devel-18.0.0-6088.g2110e007 534 kB/s | 31 kB 00:00 2023-09-13T03:00:26.130 INFO:teuthology.orchestra.run.smithi142.stdout:(13/131): libcephfs2-18.0.0-6088.g2110e007.el8. 6.9 MB/s | 827 kB 00:00 2023-09-13T03:00:26.151 INFO:teuthology.orchestra.run.smithi170.stdout:(4/131): ceph-mds-18.0.0-6088.g2110e007.el8.x86 3.4 MB/s | 2.3 MB 00:00 2023-09-13T03:00:26.188 INFO:teuthology.orchestra.run.smithi142.stdout:(14/131): libcephsqlite-18.0.0-6088.g2110e007.e 3.1 MB/s | 184 kB 00:00 2023-09-13T03:00:26.264 INFO:teuthology.orchestra.run.smithi142.stdout:(15/131): librados-devel-18.0.0-6088.g2110e007. 1.8 MB/s | 139 kB 00:00 2023-09-13T03:00:26.372 INFO:teuthology.orchestra.run.smithi142.stdout:(16/131): libradosstriper1-18.0.0-6088.g2110e00 4.7 MB/s | 519 kB 00:00 2023-09-13T03:00:26.476 INFO:teuthology.orchestra.run.smithi170.stdout:(5/131): ceph-base-18.0.0-6088.g2110e007.el8.x8 2.5 MB/s | 5.4 MB 00:02 2023-09-13T03:00:26.684 INFO:teuthology.orchestra.run.smithi170.stdout:(6/131): ceph-mgr-18.0.0-6088.g2110e007.el8.x86 3.0 MB/s | 1.6 MB 00:00 2023-09-13T03:00:26.748 INFO:teuthology.orchestra.run.smithi142.stdout:(17/131): ceph-radosgw-18.0.0-6088.g2110e007.el 7.5 MB/s | 14 MB 00:01 2023-09-13T03:00:26.856 INFO:teuthology.orchestra.run.smithi142.stdout:(18/131): python3-ceph-argparse-18.0.0-6088.g21 415 kB/s | 45 kB 00:00 2023-09-13T03:00:27.006 INFO:teuthology.orchestra.run.smithi142.stdout:(19/131): librgw2-18.0.0-6088.g2110e007.el8.x86 11 MB/s | 6.8 MB 00:00 2023-09-13T03:00:27.032 INFO:teuthology.orchestra.run.smithi142.stdout:(20/131): python3-ceph-common-18.0.0-6088.g2110 716 kB/s | 125 kB 00:00 2023-09-13T03:00:27.066 INFO:teuthology.orchestra.run.smithi142.stdout:(21/131): python3-cephfs-18.0.0-6088.g2110e007. 3.3 MB/s | 197 kB 00:00 2023-09-13T03:00:27.116 INFO:teuthology.orchestra.run.smithi142.stdout:(22/131): python3-rados-18.0.0-6088.g2110e007.e 4.3 MB/s | 371 kB 00:00 2023-09-13T03:00:27.118 INFO:teuthology.orchestra.run.smithi170.stdout:(7/131): ceph-mon-18.0.0-6088.g2110e007.el8.x86 7.2 MB/s | 4.6 MB 00:00 2023-09-13T03:00:27.168 INFO:teuthology.orchestra.run.smithi142.stdout:(23/131): python3-rbd-18.0.0-6088.g2110e007.el8 3.5 MB/s | 364 kB 00:00 2023-09-13T03:00:27.199 INFO:teuthology.orchestra.run.smithi142.stdout:(24/131): python3-rgw-18.0.0-6088.g2110e007.el8 1.2 MB/s | 106 kB 00:00 2023-09-13T03:00:27.225 INFO:teuthology.orchestra.run.smithi142.stdout:(25/131): rbd-fuse-18.0.0-6088.g2110e007.el8.x8 1.6 MB/s | 94 kB 00:00 2023-09-13T03:00:27.309 INFO:teuthology.orchestra.run.smithi142.stdout:(26/131): rbd-nbd-18.0.0-6088.g2110e007.el8.x86 2.1 MB/s | 179 kB 00:00 2023-09-13T03:00:27.392 INFO:teuthology.orchestra.run.smithi142.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6088.g 278 kB/s | 23 kB 00:00 2023-09-13T03:00:27.484 INFO:teuthology.orchestra.run.smithi142.stdout:(28/131): ceph-mgr-cephadm-18.0.0-6088.g2110e00 1.4 MB/s | 132 kB 00:00 2023-09-13T03:00:27.768 INFO:teuthology.orchestra.run.smithi142.stdout:(29/131): ceph-mgr-dashboard-18.0.0-6088.g2110e 6.2 MB/s | 1.7 MB 00:00 2023-09-13T03:00:27.826 INFO:teuthology.orchestra.run.smithi142.stdout:(30/131): rbd-mirror-18.0.0-6088.g2110e007.el8. 5.7 MB/s | 3.6 MB 00:00 2023-09-13T03:00:27.935 INFO:teuthology.orchestra.run.smithi142.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6088.g21 2.2 MB/s | 242 kB 00:00 2023-09-13T03:00:28.026 INFO:teuthology.orchestra.run.smithi142.stdout:(32/131): ceph-mgr-rook-18.0.0-6088.g2110e007.e 517 kB/s | 47 kB 00:00 2023-09-13T03:00:28.119 INFO:teuthology.orchestra.run.smithi142.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6088.g2 158 kB/s | 15 kB 00:00 2023-09-13T03:00:28.227 INFO:teuthology.orchestra.run.smithi142.stdout:(34/131): ceph-volume-18.0.0-6088.g2110e007.el8 2.4 MB/s | 261 kB 00:00 2023-09-13T03:00:28.371 INFO:teuthology.orchestra.run.smithi142.stdout:(35/131): cephadm-18.0.0-6088.g2110e007.el8.noa 1.2 MB/s | 177 kB 00:00 2023-09-13T03:00:28.453 INFO:teuthology.orchestra.run.smithi142.stdout:(36/131): cephfs-top-18.0.0-6088.g2110e007.el8. 219 kB/s | 18 kB 00:00 2023-09-13T03:00:28.510 INFO:teuthology.orchestra.run.smithi170.stdout:(8/131): ceph-radosgw-18.0.0-6088.g2110e007.el8 9.8 MB/s | 14 MB 00:01 2023-09-13T03:00:28.552 INFO:teuthology.orchestra.run.smithi170.stdout:(9/131): ceph-selinux-18.0.0-6088.g2110e007.el8 587 kB/s | 24 kB 00:00 2023-09-13T03:00:28.553 INFO:teuthology.orchestra.run.smithi142.stdout:(37/131): ceph-mgr-diskprediction-local-18.0.0- 9.4 MB/s | 7.4 MB 00:00 2023-09-13T03:00:28.795 INFO:teuthology.orchestra.run.smithi142.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 248 kB/s | 59 kB 00:00 2023-09-13T03:00:28.863 INFO:teuthology.orchestra.run.smithi142.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 1.1 MB/s | 75 kB 00:00 2023-09-13T03:00:28.888 INFO:teuthology.orchestra.run.smithi142.stdout:(40/131): gperftools-libs-2.7-9.el8.x86_64.rpm 704 kB/s | 306 kB 00:00 2023-09-13T03:00:28.955 INFO:teuthology.orchestra.run.smithi142.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 654 kB/s | 44 kB 00:00 2023-09-13T03:00:28.997 INFO:teuthology.orchestra.run.smithi142.stdout:(42/131): python3-asyncssh-2.7.0-2.el8.noarch.r 3.5 MB/s | 476 kB 00:00 2023-09-13T03:00:29.022 INFO:teuthology.orchestra.run.smithi142.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 2.7 MB/s | 185 kB 00:00 2023-09-13T03:00:29.064 INFO:teuthology.orchestra.run.smithi142.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 492 kB/s | 33 kB 00:00 2023-09-13T03:00:29.074 INFO:teuthology.orchestra.run.smithi064.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-13T03:00:29.075 INFO:teuthology.orchestra.run.smithi064.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-13T03:00:29.089 INFO:teuthology.orchestra.run.smithi142.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 244 kB/s | 16 kB 00:00 2023-09-13T03:00:29.090 INFO:teuthology.orchestra.run.smithi064.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-09-13T03:00:29.091 INFO:teuthology.orchestra.run.smithi064.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-09-13T03:00:29.093 INFO:teuthology.orchestra.run.smithi064.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-09-13T03:00:29.094 INFO:teuthology.orchestra.run.smithi064.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-09-13T03:00:29.096 INFO:teuthology.orchestra.run.smithi064.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-09-13T03:00:29.099 INFO:teuthology.orchestra.run.smithi064.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-09-13T03:00:29.101 INFO:teuthology.orchestra.run.smithi064.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-09-13T03:00:29.149 INFO:teuthology.orchestra.run.smithi142.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.0 MB/s | 173 kB 00:00 2023-09-13T03:00:29.198 INFO:teuthology.orchestra.run.smithi142.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 3.5 MB/s | 384 kB 00:00 2023-09-13T03:00:29.243 INFO:teuthology.orchestra.run.smithi142.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 856 kB/s | 81 kB 00:00 2023-09-13T03:00:29.247 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2023-09-13T03:00:29.261 INFO:teuthology.orchestra.run.smithi064.stdout:=========================================================================================================================== 2023-09-13T03:00:29.261 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2023-09-13T03:00:29.262 INFO:teuthology.orchestra.run.smithi064.stdout:=========================================================================================================================== 2023-09-13T03:00:29.262 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2023-09-13T03:00:29.262 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-09-13T03:00:29.262 INFO:teuthology.orchestra.run.smithi064.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-09-13T03:00:29.262 INFO:teuthology.orchestra.run.smithi064.stdout: ceph x86_64 2:18.0.0-6088.g2110e007.el8 ceph 6.4 k 2023-09-13T03:00:29.263 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base x86_64 2:18.0.0-6088.g2110e007.el8 ceph 5.4 M 2023-09-13T03:00:29.263 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-fuse x86_64 2:18.0.0-6088.g2110e007.el8 ceph 907 k 2023-09-13T03:00:29.263 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6088.g2110e007.el8 ceph 167 k 2023-09-13T03:00:29.263 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr x86_64 2:18.0.0-6088.g2110e007.el8 ceph 1.6 M 2023-09-13T03:00:29.263 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 132 k 2023-09-13T03:00:29.264 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 1.7 M 2023-09-13T03:00:29.264 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 7.4 M 2023-09-13T03:00:29.264 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-rook noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 47 k 2023-09-13T03:00:29.264 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-radosgw x86_64 2:18.0.0-6088.g2110e007.el8 ceph 14 M 2023-09-13T03:00:29.264 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-test x86_64 2:18.0.0-6088.g2110e007.el8 ceph 49 M 2023-09-13T03:00:29.265 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-volume noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 261 k 2023-09-13T03:00:29.265 INFO:teuthology.orchestra.run.smithi064.stdout: cephadm noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 177 k 2023-09-13T03:00:29.265 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-mirror x86_64 2:18.0.0-6088.g2110e007.el8 ceph 261 k 2023-09-13T03:00:29.265 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-top noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 18 k 2023-09-13T03:00:29.266 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-09-13T03:00:29.266 INFO:teuthology.orchestra.run.smithi064.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-09-13T03:00:29.266 INFO:teuthology.orchestra.run.smithi064.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-09-13T03:00:29.267 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs-devel x86_64 2:18.0.0-6088.g2110e007.el8 ceph 31 k 2023-09-13T03:00:29.267 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs2 x86_64 2:18.0.0-6088.g2110e007.el8 ceph 827 k 2023-09-13T03:00:29.267 INFO:teuthology.orchestra.run.smithi064.stdout: librados-devel x86_64 2:18.0.0-6088.g2110e007.el8 ceph 139 k 2023-09-13T03:00:29.267 INFO:teuthology.orchestra.run.smithi064.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-09-13T03:00:29.267 INFO:teuthology.orchestra.run.smithi064.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-09-13T03:00:29.268 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-09-13T03:00:29.268 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-09-13T03:00:29.268 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-09-13T03:00:29.268 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-09-13T03:00:29.268 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cephfs x86_64 2:18.0.0-6088.g2110e007.el8 ceph 197 k 2023-09-13T03:00:29.269 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rados x86_64 2:18.0.0-6088.g2110e007.el8 ceph 371 k 2023-09-13T03:00:29.269 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rbd x86_64 2:18.0.0-6088.g2110e007.el8 ceph 364 k 2023-09-13T03:00:29.269 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rgw x86_64 2:18.0.0-6088.g2110e007.el8 ceph 106 k 2023-09-13T03:00:29.269 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-fuse x86_64 2:18.0.0-6088.g2110e007.el8 ceph 94 k 2023-09-13T03:00:29.269 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-mirror x86_64 2:18.0.0-6088.g2110e007.el8 ceph 3.6 M 2023-09-13T03:00:29.270 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-nbd x86_64 2:18.0.0-6088.g2110e007.el8 ceph 179 k 2023-09-13T03:00:29.270 INFO:teuthology.orchestra.run.smithi064.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-09-13T03:00:29.270 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrading: 2023-09-13T03:00:29.270 INFO:teuthology.orchestra.run.smithi064.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-09-13T03:00:29.270 INFO:teuthology.orchestra.run.smithi064.stdout: librados2 x86_64 2:18.0.0-6088.g2110e007.el8 ceph 3.7 M 2023-09-13T03:00:29.271 INFO:teuthology.orchestra.run.smithi064.stdout: librbd1 x86_64 2:18.0.0-6088.g2110e007.el8 ceph 3.7 M 2023-09-13T03:00:29.271 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2023-09-13T03:00:29.271 INFO:teuthology.orchestra.run.smithi064.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-09-13T03:00:29.271 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-common x86_64 2:18.0.0-6088.g2110e007.el8 ceph 22 M 2023-09-13T03:00:29.271 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 23 k 2023-09-13T03:00:29.272 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mds x86_64 2:18.0.0-6088.g2110e007.el8 ceph 2.3 M 2023-09-13T03:00:29.272 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 242 k 2023-09-13T03:00:29.272 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mon x86_64 2:18.0.0-6088.g2110e007.el8 ceph 4.6 M 2023-09-13T03:00:29.273 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-osd x86_64 2:18.0.0-6088.g2110e007.el8 ceph 18 M 2023-09-13T03:00:29.273 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6088.g2110e007.el8 ceph-noarch 15 k 2023-09-13T03:00:29.273 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-selinux x86_64 2:18.0.0-6088.g2110e007.el8 ceph 24 k 2023-09-13T03:00:29.273 INFO:teuthology.orchestra.run.smithi064.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-09-13T03:00:29.273 INFO:teuthology.orchestra.run.smithi064.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-09-13T03:00:29.274 INFO:teuthology.orchestra.run.smithi064.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-09-13T03:00:29.274 INFO:teuthology.orchestra.run.smithi064.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-09-13T03:00:29.274 INFO:teuthology.orchestra.run.smithi064.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-09-13T03:00:29.274 INFO:teuthology.orchestra.run.smithi064.stdout: libcephsqlite x86_64 2:18.0.0-6088.g2110e007.el8 ceph 184 k 2023-09-13T03:00:29.274 INFO:teuthology.orchestra.run.smithi064.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-09-13T03:00:29.275 INFO:teuthology.orchestra.run.smithi064.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-09-13T03:00:29.275 INFO:teuthology.orchestra.run.smithi064.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-09-13T03:00:29.275 INFO:teuthology.orchestra.run.smithi064.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-09-13T03:00:29.275 INFO:teuthology.orchestra.run.smithi064.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-09-13T03:00:29.275 INFO:teuthology.orchestra.run.smithi064.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-09-13T03:00:29.276 INFO:teuthology.orchestra.run.smithi064.stdout: libradosstriper1 x86_64 2:18.0.0-6088.g2110e007.el8 ceph 519 k 2023-09-13T03:00:29.276 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-09-13T03:00:29.276 INFO:teuthology.orchestra.run.smithi064.stdout: librgw2 x86_64 2:18.0.0-6088.g2110e007.el8 ceph 6.8 M 2023-09-13T03:00:29.276 INFO:teuthology.orchestra.run.smithi064.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-09-13T03:00:29.276 INFO:teuthology.orchestra.run.smithi064.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-09-13T03:00:29.277 INFO:teuthology.orchestra.run.smithi064.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-09-13T03:00:29.277 INFO:teuthology.orchestra.run.smithi064.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-09-13T03:00:29.277 INFO:teuthology.orchestra.run.smithi064.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-09-13T03:00:29.277 INFO:teuthology.orchestra.run.smithi064.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-09-13T03:00:29.278 INFO:teuthology.orchestra.run.smithi064.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-09-13T03:00:29.278 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-09-13T03:00:29.278 INFO:teuthology.orchestra.run.smithi064.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-09-13T03:00:29.278 INFO:teuthology.orchestra.run.smithi064.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-09-13T03:00:29.278 INFO:teuthology.orchestra.run.smithi064.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-09-13T03:00:29.279 INFO:teuthology.orchestra.run.smithi064.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-09-13T03:00:29.279 INFO:teuthology.orchestra.run.smithi064.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-09-13T03:00:29.279 INFO:teuthology.orchestra.run.smithi064.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-09-13T03:00:29.279 INFO:teuthology.orchestra.run.smithi064.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-09-13T03:00:29.279 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-09-13T03:00:29.280 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse x86_64 2:18.0.0-6088.g2110e007.el8 ceph 45 k 2023-09-13T03:00:29.280 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-common x86_64 2:18.0.0-6088.g2110e007.el8 ceph 125 k 2023-09-13T03:00:29.280 INFO:teuthology.orchestra.run.smithi064.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-09-13T03:00:29.280 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-09-13T03:00:29.281 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-09-13T03:00:29.281 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-09-13T03:00:29.281 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-09-13T03:00:29.281 INFO:teuthology.orchestra.run.smithi064.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-09-13T03:00:29.281 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-09-13T03:00:29.282 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-09-13T03:00:29.282 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-09-13T03:00:29.282 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-09-13T03:00:29.282 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-09-13T03:00:29.282 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-13T03:00:29.283 INFO:teuthology.orchestra.run.smithi064.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-09-13T03:00:29.283 INFO:teuthology.orchestra.run.smithi064.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-09-13T03:00:29.283 INFO:teuthology.orchestra.run.smithi064.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-09-13T03:00:29.283 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-09-13T03:00:29.284 INFO:teuthology.orchestra.run.smithi064.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-09-13T03:00:29.284 INFO:teuthology.orchestra.run.smithi064.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-09-13T03:00:29.284 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-09-13T03:00:29.284 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-09-13T03:00:29.284 INFO:teuthology.orchestra.run.smithi064.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-09-13T03:00:29.285 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-09-13T03:00:29.285 INFO:teuthology.orchestra.run.smithi064.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-09-13T03:00:29.285 INFO:teuthology.orchestra.run.smithi064.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-09-13T03:00:29.285 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-09-13T03:00:29.285 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-09-13T03:00:29.286 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-09-13T03:00:29.286 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-09-13T03:00:29.286 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-09-13T03:00:29.286 INFO:teuthology.orchestra.run.smithi064.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-09-13T03:00:29.286 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-13T03:00:29.287 INFO:teuthology.orchestra.run.smithi064.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-09-13T03:00:29.287 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-09-13T03:00:29.287 INFO:teuthology.orchestra.run.smithi064.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-09-13T03:00:29.287 INFO:teuthology.orchestra.run.smithi064.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-09-13T03:00:29.287 INFO:teuthology.orchestra.run.smithi064.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-09-13T03:00:29.288 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-09-13T03:00:29.288 INFO:teuthology.orchestra.run.smithi064.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-09-13T03:00:29.288 INFO:teuthology.orchestra.run.smithi064.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-09-13T03:00:29.288 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-09-13T03:00:29.288 INFO:teuthology.orchestra.run.smithi064.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-09-13T03:00:29.288 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-09-13T03:00:29.289 INFO:teuthology.orchestra.run.smithi064.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-09-13T03:00:29.289 INFO:teuthology.orchestra.run.smithi064.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-09-13T03:00:29.289 INFO:teuthology.orchestra.run.smithi064.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-09-13T03:00:29.289 INFO:teuthology.orchestra.run.smithi064.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-09-13T03:00:29.290 INFO:teuthology.orchestra.run.smithi064.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-09-13T03:00:29.290 INFO:teuthology.orchestra.run.smithi064.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-09-13T03:00:29.290 INFO:teuthology.orchestra.run.smithi064.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-09-13T03:00:29.291 INFO:teuthology.orchestra.run.smithi064.stdout:Enabling module streams: 2023-09-13T03:00:29.291 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql 10 2023-09-13T03:00:29.291 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:00:29.291 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2023-09-13T03:00:29.291 INFO:teuthology.orchestra.run.smithi064.stdout:=========================================================================================================================== 2023-09-13T03:00:29.292 INFO:teuthology.orchestra.run.smithi064.stdout:Install 128 Packages 2023-09-13T03:00:29.292 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrade 3 Packages 2023-09-13T03:00:29.292 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:00:29.293 INFO:teuthology.orchestra.run.smithi142.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 120 kB/s | 11 kB 00:00 2023-09-13T03:00:29.293 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 208 M 2023-09-13T03:00:29.293 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2023-09-13T03:00:29.316 INFO:teuthology.orchestra.run.smithi142.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 243 kB/s | 18 kB 00:00 2023-09-13T03:00:29.374 INFO:teuthology.orchestra.run.smithi142.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 838 kB/s | 49 kB 00:00 2023-09-13T03:00:29.416 INFO:teuthology.orchestra.run.smithi142.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 9.3 MB/s | 1.2 MB 00:00 2023-09-13T03:00:29.442 INFO:teuthology.orchestra.run.smithi142.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 876 kB/s | 59 kB 00:00 2023-09-13T03:00:29.484 INFO:teuthology.orchestra.run.smithi142.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 903 kB/s | 60 kB 00:00 2023-09-13T03:00:29.509 INFO:teuthology.orchestra.run.smithi142.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 4.1 MB/s | 283 kB 00:00 2023-09-13T03:00:29.551 INFO:teuthology.orchestra.run.smithi142.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 233 kB/s | 16 kB 00:00 2023-09-13T03:00:29.576 INFO:teuthology.orchestra.run.smithi142.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 502 kB/s | 34 kB 00:00 2023-09-13T03:00:29.619 INFO:teuthology.orchestra.run.smithi142.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 2.9 MB/s | 196 kB 00:00 2023-09-13T03:00:29.644 INFO:teuthology.orchestra.run.smithi142.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 960 kB/s | 64 kB 00:00 2023-09-13T03:00:29.672 INFO:teuthology.orchestra.run.smithi064.stdout:(1/131): ceph-18.0.0-6088.g2110e007.el8.x86_64. 27 kB/s | 6.4 kB 00:00 2023-09-13T03:00:29.702 INFO:teuthology.orchestra.run.smithi142.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 224 kB/s | 19 kB 00:00 2023-09-13T03:00:29.727 INFO:teuthology.orchestra.run.smithi142.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 286 kB/s | 24 kB 00:00 2023-09-13T03:00:29.761 INFO:teuthology.orchestra.run.smithi142.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 500 kB/s | 29 kB 00:00 2023-09-13T03:00:29.786 INFO:teuthology.orchestra.run.smithi142.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 471 kB/s | 27 kB 00:00 2023-09-13T03:00:29.828 INFO:teuthology.orchestra.run.smithi142.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 3.7 MB/s | 255 kB 00:00 2023-09-13T03:00:29.854 INFO:teuthology.orchestra.run.smithi142.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 3.7 MB/s | 251 kB 00:00 2023-09-13T03:00:29.887 INFO:teuthology.orchestra.run.smithi142.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 1.0 MB/s | 61 kB 00:00 2023-09-13T03:00:29.930 INFO:teuthology.orchestra.run.smithi142.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.1 MB/s | 85 kB 00:00 2023-09-13T03:00:29.946 INFO:teuthology.orchestra.run.smithi142.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 391 kB/s | 23 kB 00:00 2023-09-13T03:00:29.964 INFO:teuthology.orchestra.run.smithi142.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 4.1 MB/s | 70 kB 00:00 2023-09-13T03:00:30.014 INFO:teuthology.orchestra.run.smithi142.stdout:(70/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 939 kB/s | 47 kB 00:00 2023-09-13T03:00:30.064 INFO:teuthology.orchestra.run.smithi142.stdout:(71/131): thrift-0.13.0-2.el8.x86_64.rpm 13 MB/s | 1.7 MB 00:00 2023-09-13T03:00:30.114 INFO:teuthology.orchestra.run.smithi142.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 1.9 MB/s | 200 kB 00:00 2023-09-13T03:00:30.157 INFO:teuthology.orchestra.run.smithi142.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 463 kB/s | 43 kB 00:00 2023-09-13T03:00:30.181 INFO:teuthology.orchestra.run.smithi142.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 3.5 MB/s | 220 kB 00:00 2023-09-13T03:00:30.224 INFO:teuthology.orchestra.run.smithi142.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 716 kB/s | 48 kB 00:00 2023-09-13T03:00:30.249 INFO:teuthology.orchestra.run.smithi142.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.3 MB/s | 87 kB 00:00 2023-09-13T03:00:30.275 INFO:teuthology.orchestra.run.smithi142.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 853 kB/s | 43 kB 00:00 2023-09-13T03:00:30.291 INFO:teuthology.orchestra.run.smithi142.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 2.5 MB/s | 109 kB 00:00 2023-09-13T03:00:30.326 INFO:teuthology.orchestra.run.smithi142.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.5 MB/s | 229 kB 00:00 2023-09-13T03:00:30.368 INFO:teuthology.orchestra.run.smithi142.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 7.3 MB/s | 560 kB 00:00 2023-09-13T03:00:30.392 INFO:teuthology.orchestra.run.smithi142.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 280 kB/s | 19 kB 00:00 2023-09-13T03:00:30.417 INFO:teuthology.orchestra.run.smithi142.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.0 MB/s | 155 kB 00:00 2023-09-13T03:00:30.451 INFO:teuthology.orchestra.run.smithi142.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 9.3 MB/s | 559 kB 00:00 2023-09-13T03:00:30.476 INFO:teuthology.orchestra.run.smithi142.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 306 kB/s | 18 kB 00:00 2023-09-13T03:00:30.482 INFO:teuthology.orchestra.run.smithi064.stdout:(2/131): ceph-fuse-18.0.0-6088.g2110e007.el8.x8 1.1 MB/s | 907 kB 00:00 2023-09-13T03:00:30.543 INFO:teuthology.orchestra.run.smithi142.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 6.4 MB/s | 605 kB 00:00 2023-09-13T03:00:30.568 INFO:teuthology.orchestra.run.smithi142.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.5 MB/s | 238 kB 00:00 2023-09-13T03:00:30.594 INFO:teuthology.orchestra.run.smithi142.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 2.8 MB/s | 72 kB 00:00 2023-09-13T03:00:30.644 INFO:teuthology.orchestra.run.smithi142.stdout:(88/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 23 MB/s | 2.3 MB 00:00 2023-09-13T03:00:30.648 INFO:teuthology.orchestra.run.smithi064.stdout:(3/131): ceph-immutable-object-cache-18.0.0-608 1.0 MB/s | 167 kB 00:00 2023-09-13T03:00:30.670 INFO:teuthology.orchestra.run.smithi142.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.1 MB/s | 83 kB 00:00 2023-09-13T03:00:30.694 INFO:teuthology.orchestra.run.smithi142.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.2 MB/s | 60 kB 00:00 2023-09-13T03:00:30.720 INFO:teuthology.orchestra.run.smithi142.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-09-13T03:00:30.745 INFO:teuthology.orchestra.run.smithi142.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-09-13T03:00:30.770 INFO:teuthology.orchestra.run.smithi142.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 768 kB/s | 39 kB 00:00 2023-09-13T03:00:30.806 INFO:teuthology.orchestra.run.smithi142.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 10 MB/s | 644 kB 00:00 2023-09-13T03:00:30.831 INFO:teuthology.orchestra.run.smithi142.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.8 MB/s | 171 kB 00:00 2023-09-13T03:00:30.865 INFO:teuthology.orchestra.run.smithi142.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.3 MB/s | 138 kB 00:00 2023-09-13T03:00:30.949 INFO:teuthology.orchestra.run.smithi142.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.3 MB/s | 110 kB 00:00 2023-09-13T03:00:30.981 INFO:teuthology.orchestra.run.smithi064.stdout:(4/131): ceph-base-18.0.0-6088.g2110e007.el8.x8 3.5 MB/s | 5.4 MB 00:01 2023-09-13T03:00:30.991 INFO:teuthology.orchestra.run.smithi142.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-09-13T03:00:31.033 INFO:teuthology.orchestra.run.smithi142.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-09-13T03:00:31.077 INFO:teuthology.orchestra.run.smithi170.stdout:(10/131): ceph-osd-18.0.0-6088.g2110e007.el8.x8 4.1 MB/s | 18 MB 00:04 2023-09-13T03:00:31.083 INFO:teuthology.orchestra.run.smithi142.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2023-09-13T03:00:31.133 INFO:teuthology.orchestra.run.smithi142.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 777 kB/s | 39 kB 00:00 2023-09-13T03:00:31.252 INFO:teuthology.orchestra.run.smithi170.stdout:(11/131): cephfs-mirror-18.0.0-6088.g2110e007.e 1.5 MB/s | 261 kB 00:00 2023-09-13T03:00:31.257 INFO:teuthology.orchestra.run.smithi064.stdout:(5/131): ceph-mgr-18.0.0-6088.g2110e007.el8.x86 5.9 MB/s | 1.6 MB 00:00 2023-09-13T03:00:31.283 INFO:teuthology.orchestra.run.smithi142.stdout:(102/131): python3-scipy-1.0.0-21.module+el8.5. 30 MB/s | 14 MB 00:00 2023-09-13T03:00:31.319 INFO:teuthology.orchestra.run.smithi170.stdout:(12/131): libcephfs-devel-18.0.0-6088.g2110e007 466 kB/s | 31 kB 00:00 2023-09-13T03:00:31.325 INFO:teuthology.orchestra.run.smithi142.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 3.5 MB/s | 688 kB 00:00 2023-09-13T03:00:31.384 INFO:teuthology.orchestra.run.smithi142.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 5.3 MB/s | 538 kB 00:00 2023-09-13T03:00:31.409 INFO:teuthology.orchestra.run.smithi142.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 1.8 MB/s | 157 kB 00:00 2023-09-13T03:00:31.434 INFO:teuthology.orchestra.run.smithi142.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2023-09-13T03:00:31.468 INFO:teuthology.orchestra.run.smithi142.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-09-13T03:00:31.510 INFO:teuthology.orchestra.run.smithi142.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 637 kB/s | 27 kB 00:00 2023-09-13T03:00:31.585 INFO:teuthology.orchestra.run.smithi142.stdout:(109/131): python3-babel-2.5.1-7.el8.noarch.rpm 32 MB/s | 4.8 MB 00:00 2023-09-13T03:00:31.610 INFO:teuthology.orchestra.run.smithi142.stdout:(110/131): flex-2.6.1-9.el8.x86_64.rpm 3.1 MB/s | 320 kB 00:00 2023-09-13T03:00:31.619 INFO:teuthology.orchestra.run.smithi170.stdout:(13/131): libcephfs2-18.0.0-6088.g2110e007.el8. 2.7 MB/s | 827 kB 00:00 2023-09-13T03:00:31.635 INFO:teuthology.orchestra.run.smithi142.stdout:(111/131): python3-prettytable-0.7.2-14.el8.noa 881 kB/s | 44 kB 00:00 2023-09-13T03:00:31.674 INFO:teuthology.orchestra.run.smithi064.stdout:(6/131): ceph-mds-18.0.0-6088.g2110e007.el8.x86 2.3 MB/s | 2.3 MB 00:01 2023-09-13T03:00:31.686 INFO:teuthology.orchestra.run.smithi142.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 8.9 MB/s | 457 kB 00:00 2023-09-13T03:00:31.711 INFO:teuthology.orchestra.run.smithi170.stdout:(14/131): libcephsqlite-18.0.0-6088.g2110e007.e 2.0 MB/s | 184 kB 00:00 2023-09-13T03:00:31.775 INFO:teuthology.orchestra.run.smithi142.stdout:(113/131): python3-numpy-1.14.3-10.el8.x86_64.r 23 MB/s | 3.7 MB 00:00 2023-09-13T03:00:31.803 INFO:teuthology.orchestra.run.smithi142.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 1.9 MB/s | 225 kB 00:00 2023-09-13T03:00:31.811 INFO:teuthology.orchestra.run.smithi170.stdout:(15/131): librados-devel-18.0.0-6088.g2110e007. 1.4 MB/s | 139 kB 00:00 2023-09-13T03:00:31.829 INFO:teuthology.orchestra.run.smithi142.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2023-09-13T03:00:31.854 INFO:teuthology.orchestra.run.smithi142.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-09-13T03:00:31.879 INFO:teuthology.orchestra.run.smithi142.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 398 kB/s | 20 kB 00:00 2023-09-13T03:00:31.904 INFO:teuthology.orchestra.run.smithi142.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-09-13T03:00:31.946 INFO:teuthology.orchestra.run.smithi142.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2023-09-13T03:00:31.982 INFO:teuthology.orchestra.run.smithi064.stdout:(7/131): ceph-mon-18.0.0-6088.g2110e007.el8.x86 6.4 MB/s | 4.6 MB 00:00 2023-09-13T03:00:31.996 INFO:teuthology.orchestra.run.smithi142.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 3.9 MB/s | 198 kB 00:00 2023-09-13T03:00:32.080 INFO:teuthology.orchestra.run.smithi142.stdout:(121/131): openblas-threads-0.3.15-3.el8.x86_64 28 MB/s | 4.9 MB 00:00 2023-09-13T03:00:32.130 INFO:teuthology.orchestra.run.smithi142.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.0 MB/s | 202 kB 00:00 2023-09-13T03:00:32.239 INFO:teuthology.orchestra.run.smithi142.stdout:(123/131): openblas-0.3.15-3.el8.x86_64.rpm 20 MB/s | 4.7 MB 00:00 2023-09-13T03:00:32.272 INFO:teuthology.orchestra.run.smithi142.stdout:(124/131): librdkafka-0.11.4-3.el8.x86_64.rpm 2.4 MB/s | 354 kB 00:00 2023-09-13T03:00:32.340 INFO:teuthology.orchestra.run.smithi142.stdout:(125/131): postgresql-contrib-10.21-2.module+el 12 MB/s | 808 kB 00:00 2023-09-13T03:00:32.415 INFO:teuthology.orchestra.run.smithi142.stdout:(126/131): postgresql-server-10.21-2.module+el8 29 MB/s | 5.1 MB 00:00 2023-09-13T03:00:32.465 INFO:teuthology.orchestra.run.smithi142.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 12 MB/s | 1.5 MB 00:00 2023-09-13T03:00:32.504 INFO:teuthology.orchestra.run.smithi170.stdout:(16/131): ceph-test-18.0.0-6088.g2110e007.el8.x 12 MB/s | 49 MB 00:03 2023-09-13T03:00:32.546 INFO:teuthology.orchestra.run.smithi170.stdout:(17/131): libradosstriper1-18.0.0-6088.g2110e00 707 kB/s | 519 kB 00:00 2023-09-13T03:00:32.621 INFO:teuthology.orchestra.run.smithi170.stdout:(18/131): python3-ceph-argparse-18.0.0-6088.g21 598 kB/s | 45 kB 00:00 2023-09-13T03:00:32.796 INFO:teuthology.orchestra.run.smithi170.stdout:(19/131): python3-ceph-common-18.0.0-6088.g2110 718 kB/s | 125 kB 00:00 2023-09-13T03:00:33.272 INFO:teuthology.orchestra.run.smithi170.stdout:(20/131): python3-cephfs-18.0.0-6088.g2110e007. 414 kB/s | 197 kB 00:00 2023-09-13T03:00:33.407 INFO:teuthology.orchestra.run.smithi142.stdout:(128/131): librbd1-18.0.0-6088.g2110e007.el8.x8 3.9 MB/s | 3.7 MB 00:00 2023-09-13T03:00:33.481 INFO:teuthology.orchestra.run.smithi170.stdout:(21/131): python3-rados-18.0.0-6088.g2110e007.e 1.7 MB/s | 371 kB 00:00 2023-09-13T03:00:33.507 INFO:teuthology.orchestra.run.smithi142.stdout:(129/131): librados2-18.0.0-6088.g2110e007.el8. 3.4 MB/s | 3.7 MB 00:01 2023-09-13T03:00:33.631 INFO:teuthology.orchestra.run.smithi170.stdout:(22/131): python3-rbd-18.0.0-6088.g2110e007.el8 2.4 MB/s | 364 kB 00:00 2023-09-13T03:00:33.690 INFO:teuthology.orchestra.run.smithi170.stdout:(23/131): python3-rgw-18.0.0-6088.g2110e007.el8 1.8 MB/s | 106 kB 00:00 2023-09-13T03:00:33.741 INFO:teuthology.orchestra.run.smithi142.stdout:(130/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 2.3 MB/s | 798 kB 00:00 2023-09-13T03:00:33.782 INFO:teuthology.orchestra.run.smithi170.stdout:(24/131): librgw2-18.0.0-6088.g2110e007.el8.x86 5.3 MB/s | 6.8 MB 00:01 2023-09-13T03:00:33.823 INFO:teuthology.orchestra.run.smithi170.stdout:(25/131): rbd-fuse-18.0.0-6088.g2110e007.el8.x8 701 kB/s | 94 kB 00:00 2023-09-13T03:00:33.940 INFO:teuthology.orchestra.run.smithi170.stdout:(26/131): rbd-nbd-18.0.0-6088.g2110e007.el8.x86 1.5 MB/s | 179 kB 00:00 2023-09-13T03:00:33.999 INFO:teuthology.orchestra.run.smithi170.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6088.g 396 kB/s | 23 kB 00:00 2023-09-13T03:00:34.074 INFO:teuthology.orchestra.run.smithi170.stdout:(28/131): ceph-mgr-cephadm-18.0.0-6088.g2110e00 1.7 MB/s | 132 kB 00:00 2023-09-13T03:00:34.249 INFO:teuthology.orchestra.run.smithi064.stdout:(8/131): ceph-radosgw-18.0.0-6088.g2110e007.el8 6.0 MB/s | 14 MB 00:02 2023-09-13T03:00:34.316 INFO:teuthology.orchestra.run.smithi170.stdout:(29/131): rbd-mirror-18.0.0-6088.g2110e007.el8. 6.7 MB/s | 3.6 MB 00:00 2023-09-13T03:00:34.324 INFO:teuthology.orchestra.run.smithi064.stdout:(9/131): ceph-selinux-18.0.0-6088.g2110e007.el8 325 kB/s | 24 kB 00:00 2023-09-13T03:00:34.533 INFO:teuthology.orchestra.run.smithi170.stdout:(30/131): ceph-mgr-dashboard-18.0.0-6088.g2110e 3.8 MB/s | 1.7 MB 00:00 2023-09-13T03:00:34.591 INFO:teuthology.orchestra.run.smithi064.stdout:(10/131): ceph-osd-18.0.0-6088.g2110e007.el8.x8 6.1 MB/s | 18 MB 00:02 2023-09-13T03:00:34.625 INFO:teuthology.orchestra.run.smithi170.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6088.g21 2.6 MB/s | 242 kB 00:00 2023-09-13T03:00:34.684 INFO:teuthology.orchestra.run.smithi170.stdout:(32/131): ceph-mgr-rook-18.0.0-6088.g2110e007.e 812 kB/s | 47 kB 00:00 2023-09-13T03:00:34.734 INFO:teuthology.orchestra.run.smithi170.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6088.g2 291 kB/s | 15 kB 00:00 2023-09-13T03:00:34.741 INFO:teuthology.orchestra.run.smithi064.stdout:(11/131): cephfs-mirror-18.0.0-6088.g2110e007.e 1.7 MB/s | 261 kB 00:00 2023-09-13T03:00:34.834 INFO:teuthology.orchestra.run.smithi064.stdout:(12/131): libcephfs-devel-18.0.0-6088.g2110e007 339 kB/s | 31 kB 00:00 2023-09-13T03:00:34.842 INFO:teuthology.orchestra.run.smithi170.stdout:(34/131): ceph-volume-18.0.0-6088.g2110e007.el8 2.4 MB/s | 261 kB 00:00 2023-09-13T03:00:34.960 INFO:teuthology.orchestra.run.smithi170.stdout:(35/131): cephadm-18.0.0-6088.g2110e007.el8.noa 1.5 MB/s | 177 kB 00:00 2023-09-13T03:00:34.984 INFO:teuthology.orchestra.run.smithi064.stdout:(13/131): libcephfs2-18.0.0-6088.g2110e007.el8. 5.4 MB/s | 827 kB 00:00 2023-09-13T03:00:35.010 INFO:teuthology.orchestra.run.smithi170.stdout:(36/131): cephfs-top-18.0.0-6088.g2110e007.el8. 361 kB/s | 18 kB 00:00 2023-09-13T03:00:35.085 INFO:teuthology.orchestra.run.smithi064.stdout:(14/131): libcephsqlite-18.0.0-6088.g2110e007.e 1.8 MB/s | 184 kB 00:00 2023-09-13T03:00:35.168 INFO:teuthology.orchestra.run.smithi064.stdout:(15/131): librados-devel-18.0.0-6088.g2110e007. 1.6 MB/s | 139 kB 00:00 2023-09-13T03:00:35.194 INFO:teuthology.orchestra.run.smithi170.stdout:(37/131): ceph-mgr-diskprediction-local-18.0.0- 8.4 MB/s | 7.4 MB 00:00 2023-09-13T03:00:35.310 INFO:teuthology.orchestra.run.smithi064.stdout:(16/131): libradosstriper1-18.0.0-6088.g2110e00 3.6 MB/s | 519 kB 00:00 2023-09-13T03:00:35.710 INFO:teuthology.orchestra.run.smithi170.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 115 kB/s | 59 kB 00:00 2023-09-13T03:00:35.861 INFO:teuthology.orchestra.run.smithi170.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 360 kB/s | 306 kB 00:00 2023-09-13T03:00:35.969 INFO:teuthology.orchestra.run.smithi170.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 290 kB/s | 75 kB 00:00 2023-09-13T03:00:36.103 INFO:teuthology.orchestra.run.smithi170.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 327 kB/s | 44 kB 00:00 2023-09-13T03:00:36.128 INFO:teuthology.orchestra.run.smithi170.stdout:(42/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.7 MB/s | 476 kB 00:00 2023-09-13T03:00:36.193 INFO:teuthology.orchestra.run.smithi064.stdout:(17/131): librgw2-18.0.0-6088.g2110e007.el8.x86 7.7 MB/s | 6.8 MB 00:00 2023-09-13T03:00:36.254 INFO:teuthology.orchestra.run.smithi170.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 263 kB/s | 33 kB 00:00 2023-09-13T03:00:36.285 INFO:teuthology.orchestra.run.smithi064.stdout:(18/131): python3-ceph-argparse-18.0.0-6088.g21 490 kB/s | 45 kB 00:00 2023-09-13T03:00:36.329 INFO:teuthology.orchestra.run.smithi170.stdout:(44/131): python3-beautifulsoup4-4.6.3-2.el8.1. 820 kB/s | 185 kB 00:00 2023-09-13T03:00:36.371 INFO:teuthology.orchestra.run.smithi170.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 139 kB/s | 16 kB 00:00 2023-09-13T03:00:36.377 INFO:teuthology.orchestra.run.smithi064.stdout:(19/131): python3-ceph-common-18.0.0-6088.g2110 1.3 MB/s | 125 kB 00:00 2023-09-13T03:00:36.469 INFO:teuthology.orchestra.run.smithi064.stdout:(20/131): python3-cephfs-18.0.0-6088.g2110e007. 2.1 MB/s | 197 kB 00:00 2023-09-13T03:00:36.488 INFO:teuthology.orchestra.run.smithi170.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.1 MB/s | 173 kB 00:00 2023-09-13T03:00:36.514 INFO:teuthology.orchestra.run.smithi170.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.6 MB/s | 384 kB 00:00 2023-09-13T03:00:36.544 INFO:teuthology.orchestra.run.smithi064.stdout:(21/131): python3-rados-18.0.0-6088.g2110e007.e 4.8 MB/s | 371 kB 00:00 2023-09-13T03:00:36.605 INFO:teuthology.orchestra.run.smithi170.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 695 kB/s | 81 kB 00:00 2023-09-13T03:00:36.620 INFO:teuthology.orchestra.run.smithi064.stdout:(22/131): python3-rbd-18.0.0-6088.g2110e007.el8 4.8 MB/s | 364 kB 00:00 2023-09-13T03:00:36.631 INFO:teuthology.orchestra.run.smithi170.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 95 kB/s | 11 kB 00:00 2023-09-13T03:00:36.687 INFO:teuthology.orchestra.run.smithi064.stdout:(23/131): python3-rgw-18.0.0-6088.g2110e007.el8 1.6 MB/s | 106 kB 00:00 2023-09-13T03:00:36.706 INFO:teuthology.orchestra.run.smithi170.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 175 kB/s | 18 kB 00:00 2023-09-13T03:00:36.754 INFO:teuthology.orchestra.run.smithi064.stdout:(24/131): rbd-fuse-18.0.0-6088.g2110e007.el8.x8 1.4 MB/s | 94 kB 00:00 2023-09-13T03:00:36.823 INFO:teuthology.orchestra.run.smithi170.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 417 kB/s | 49 kB 00:00 2023-09-13T03:00:36.940 INFO:teuthology.orchestra.run.smithi170.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 503 kB/s | 59 kB 00:00 2023-09-13T03:00:36.973 INFO:teuthology.orchestra.run.smithi170.stdout:(53/131): python3-kubernetes-11.0.0-6.el8.noarc 3.4 MB/s | 1.2 MB 00:00 2023-09-13T03:00:37.049 INFO:teuthology.orchestra.run.smithi170.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 555 kB/s | 60 kB 00:00 2023-09-13T03:00:37.108 INFO:teuthology.orchestra.run.smithi170.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.1 MB/s | 283 kB 00:00 2023-09-13T03:00:37.158 INFO:teuthology.orchestra.run.smithi170.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 144 kB/s | 16 kB 00:00 2023-09-13T03:00:37.171 INFO:teuthology.orchestra.run.smithi064.stdout:(25/131): rbd-mirror-18.0.0-6088.g2110e007.el8. 8.6 MB/s | 3.6 MB 00:00 2023-09-13T03:00:37.182 INFO:teuthology.orchestra.run.smithi142.stdout:(131/131): ceph-test-18.0.0-6088.g2110e007.el8. 4.2 MB/s | 49 MB 00:11 2023-09-13T03:00:37.187 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2023-09-13T03:00:37.188 INFO:teuthology.orchestra.run.smithi142.stdout:Total 14 MB/s | 208 MB 00:14 2023-09-13T03:00:37.208 INFO:teuthology.orchestra.run.smithi170.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 336 kB/s | 34 kB 00:00 2023-09-13T03:00:37.246 INFO:teuthology.orchestra.run.smithi064.stdout:(26/131): rbd-nbd-18.0.0-6088.g2110e007.el8.x86 2.3 MB/s | 179 kB 00:00 2023-09-13T03:00:37.283 INFO:teuthology.orchestra.run.smithi170.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 1.5 MB/s | 196 kB 00:00 2023-09-13T03:00:37.305 INFO:teuthology.orchestra.run.smithi064.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6088.g 393 kB/s | 23 kB 00:00 2023-09-13T03:00:37.317 INFO:teuthology.orchestra.run.smithi170.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 592 kB/s | 64 kB 00:00 2023-09-13T03:00:37.372 INFO:teuthology.orchestra.run.smithi064.stdout:(28/131): ceph-mgr-cephadm-18.0.0-6088.g2110e00 1.9 MB/s | 132 kB 00:00 2023-09-13T03:00:37.392 INFO:teuthology.orchestra.run.smithi170.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 172 kB/s | 19 kB 00:00 2023-09-13T03:00:37.418 INFO:teuthology.orchestra.run.smithi170.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 238 kB/s | 24 kB 00:00 2023-09-13T03:00:37.501 INFO:teuthology.orchestra.run.smithi170.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 269 kB/s | 29 kB 00:00 2023-09-13T03:00:37.518 INFO:teuthology.orchestra.run.smithi170.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 275 kB/s | 27 kB 00:00 2023-09-13T03:00:37.564 INFO:teuthology.orchestra.run.smithi064.stdout:(29/131): ceph-mgr-dashboard-18.0.0-6088.g2110e 9.1 MB/s | 1.7 MB 00:00 2023-09-13T03:00:37.719 INFO:teuthology.orchestra.run.smithi170.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 1.1 MB/s | 255 kB 00:00 2023-09-13T03:00:37.836 INFO:teuthology.orchestra.run.smithi170.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 792 kB/s | 251 kB 00:00 2023-09-13T03:00:37.861 INFO:teuthology.orchestra.run.smithi170.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 431 kB/s | 61 kB 00:00 2023-09-13T03:00:37.944 INFO:teuthology.orchestra.run.smithi170.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 787 kB/s | 85 kB 00:00 2023-09-13T03:00:37.978 INFO:teuthology.orchestra.run.smithi170.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 196 kB/s | 23 kB 00:00 2023-09-13T03:00:38.003 INFO:teuthology.orchestra.run.smithi170.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 70 kB 00:00 2023-09-13T03:00:38.038 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2023-09-13T03:00:38.159 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2023-09-13T03:00:38.159 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2023-09-13T03:00:38.179 INFO:teuthology.orchestra.run.smithi170.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 7.4 MB/s | 1.7 MB 00:00 2023-09-13T03:00:38.229 INFO:teuthology.orchestra.run.smithi170.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.2 MB/s | 110 kB 00:00 2023-09-13T03:00:38.239 INFO:teuthology.orchestra.run.smithi064.stdout:(30/131): ceph-mgr-diskprediction-local-18.0.0- 11 MB/s | 7.4 MB 00:00 2023-09-13T03:00:38.331 INFO:teuthology.orchestra.run.smithi064.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6088.g21 2.6 MB/s | 242 kB 00:00 2023-09-13T03:00:38.363 INFO:teuthology.orchestra.run.smithi170.stdout:(72/131): python3-scipy-1.0.0-21.module+el8.5.0 38 MB/s | 14 MB 00:00 2023-09-13T03:00:38.388 INFO:teuthology.orchestra.run.smithi170.stdout:(73/131): python3-pytz-2017.2-9.el8.noarch.rpm 342 kB/s | 54 kB 00:00 2023-09-13T03:00:38.398 INFO:teuthology.orchestra.run.smithi064.stdout:(32/131): ceph-mgr-rook-18.0.0-6088.g2110e007.e 711 kB/s | 47 kB 00:00 2023-09-13T03:00:38.413 INFO:teuthology.orchestra.run.smithi170.stdout:(74/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.1 MB/s | 259 kB 00:00 2023-09-13T03:00:38.439 INFO:teuthology.orchestra.run.smithi170.stdout:(75/131): socat-1.7.4.1-1.el8.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2023-09-13T03:00:38.456 INFO:teuthology.orchestra.run.smithi064.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6088.g2 250 kB/s | 15 kB 00:00 2023-09-13T03:00:38.464 INFO:teuthology.orchestra.run.smithi170.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 772 kB/s | 39 kB 00:00 2023-09-13T03:00:38.506 INFO:teuthology.orchestra.run.smithi170.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 10 MB/s | 688 kB 00:00 2023-09-13T03:00:38.540 INFO:teuthology.orchestra.run.smithi170.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 7.0 MB/s | 538 kB 00:00 2023-09-13T03:00:38.540 INFO:teuthology.orchestra.run.smithi064.stdout:(34/131): ceph-volume-18.0.0-6088.g2110e007.el8 3.1 MB/s | 261 kB 00:00 2023-09-13T03:00:38.565 INFO:teuthology.orchestra.run.smithi170.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-09-13T03:00:38.590 INFO:teuthology.orchestra.run.smithi170.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2023-09-13T03:00:38.623 INFO:teuthology.orchestra.run.smithi064.stdout:(35/131): cephadm-18.0.0-6088.g2110e007.el8.noa 2.1 MB/s | 177 kB 00:00 2023-09-13T03:00:38.624 INFO:teuthology.orchestra.run.smithi170.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-09-13T03:00:38.657 INFO:teuthology.orchestra.run.smithi170.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 797 kB/s | 27 kB 00:00 2023-09-13T03:00:38.699 INFO:teuthology.orchestra.run.smithi064.stdout:(36/131): cephfs-top-18.0.0-6088.g2110e007.el8. 240 kB/s | 18 kB 00:00 2023-09-13T03:00:38.708 INFO:teuthology.orchestra.run.smithi170.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 6.2 MB/s | 320 kB 00:00 2023-09-13T03:00:38.785 INFO:teuthology.orchestra.run.smithi170.stdout:(84/131): python3-babel-2.5.1-7.el8.noarch.rpm 25 MB/s | 4.8 MB 00:00 2023-09-13T03:00:38.808 INFO:teuthology.orchestra.run.smithi170.stdout:(85/131): python3-prettytable-0.7.2-14.el8.noar 438 kB/s | 44 kB 00:00 2023-09-13T03:00:38.851 INFO:teuthology.orchestra.run.smithi170.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 11 MB/s | 457 kB 00:00 2023-09-13T03:00:38.857 INFO:teuthology.orchestra.run.smithi064.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.9 MB/s | 306 kB 00:00 2023-09-13T03:00:38.891 INFO:teuthology.orchestra.run.smithi064.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 1.7 MB/s | 59 kB 00:00 2023-09-13T03:00:38.892 INFO:teuthology.orchestra.run.smithi170.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 5.3 MB/s | 225 kB 00:00 2023-09-13T03:00:38.950 INFO:teuthology.orchestra.run.smithi064.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 1.3 MB/s | 75 kB 00:00 2023-09-13T03:00:38.959 INFO:teuthology.orchestra.run.smithi170.stdout:(88/131): python3-numpy-1.14.3-10.el8.x86_64.rp 21 MB/s | 3.7 MB 00:00 2023-09-13T03:00:38.984 INFO:teuthology.orchestra.run.smithi170.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 696 kB/s | 64 kB 00:00 2023-09-13T03:00:39.010 INFO:teuthology.orchestra.run.smithi170.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-09-13T03:00:39.017 INFO:teuthology.orchestra.run.smithi064.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 7.0 MB/s | 476 kB 00:00 2023-09-13T03:00:39.027 INFO:teuthology.orchestra.run.smithi170.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 476 kB/s | 20 kB 00:00 2023-09-13T03:00:39.044 INFO:teuthology.orchestra.run.smithi170.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 3.0 MB/s | 103 kB 00:00 2023-09-13T03:00:39.051 INFO:teuthology.orchestra.run.smithi064.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 1.3 MB/s | 44 kB 00:00 2023-09-13T03:00:39.077 INFO:teuthology.orchestra.run.smithi170.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 14 MB/s | 709 kB 00:00 2023-09-13T03:00:39.101 INFO:teuthology.orchestra.run.smithi064.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 3.6 MB/s | 185 kB 00:00 2023-09-13T03:00:39.119 INFO:teuthology.orchestra.run.smithi170.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 4.7 MB/s | 198 kB 00:00 2023-09-13T03:00:39.151 INFO:teuthology.orchestra.run.smithi064.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 661 kB/s | 33 kB 00:00 2023-09-13T03:00:39.211 INFO:teuthology.orchestra.run.smithi170.stdout:(95/131): openblas-threads-0.3.15-3.el8.x86_64. 29 MB/s | 4.9 MB 00:00 2023-09-13T03:00:39.218 INFO:teuthology.orchestra.run.smithi064.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 245 kB/s | 16 kB 00:00 2023-09-13T03:00:39.253 INFO:teuthology.orchestra.run.smithi170.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-09-13T03:00:39.260 INFO:teuthology.orchestra.run.smithi064.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 4.1 MB/s | 173 kB 00:00 2023-09-13T03:00:39.293 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2023-09-13T03:00:39.297 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2023-09-13T03:00:39.362 INFO:teuthology.orchestra.run.smithi170.stdout:(97/131): openblas-0.3.15-3.el8.x86_64.rpm 19 MB/s | 4.7 MB 00:00 2023-09-13T03:00:39.388 INFO:teuthology.orchestra.run.smithi170.stdout:(98/131): librdkafka-0.11.4-3.el8.x86_64.rpm 2.6 MB/s | 354 kB 00:00 2023-09-13T03:00:39.446 INFO:teuthology.orchestra.run.smithi170.stdout:(99/131): postgresql-contrib-10.21-2.module+el8 14 MB/s | 808 kB 00:00 2023-09-13T03:00:39.477 INFO:teuthology.orchestra.run.smithi064.stdout:(46/131): ceph-common-18.0.0-6088.g2110e007.el8 2.2 MB/s | 22 MB 00:10 2023-09-13T03:00:39.511 INFO:teuthology.orchestra.run.smithi064.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.5 MB/s | 384 kB 00:00 2023-09-13T03:00:39.538 INFO:teuthology.orchestra.run.smithi170.stdout:(100/131): postgresql-server-10.21-2.module+el8 29 MB/s | 5.1 MB 00:00 2023-09-13T03:00:39.544 INFO:teuthology.orchestra.run.smithi064.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 390 kB/s | 11 kB 00:00 2023-09-13T03:00:39.578 INFO:teuthology.orchestra.run.smithi064.stdout:(49/131): python3-jaraco-functools-2.0-4.el8.no 529 kB/s | 18 kB 00:00 2023-09-13T03:00:39.580 INFO:teuthology.orchestra.run.smithi170.stdout:(101/131): postgresql-10.21-2.module+el8.6.0+15 11 MB/s | 1.5 MB 00:00 2023-09-13T03:00:39.605 INFO:teuthology.orchestra.run.smithi064.stdout:(50/131): python3-google-auth-1.1.1-10.el8.noar 648 kB/s | 81 kB 00:00 2023-09-13T03:00:39.606 INFO:teuthology.orchestra.run.smithi170.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 701 kB/s | 47 kB 00:00 2023-09-13T03:00:39.631 INFO:teuthology.orchestra.run.smithi170.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 3.9 MB/s | 200 kB 00:00 2023-09-13T03:00:39.656 INFO:teuthology.orchestra.run.smithi170.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 845 kB/s | 43 kB 00:00 2023-09-13T03:00:39.662 INFO:teuthology.orchestra.run.smithi064.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 837 kB/s | 49 kB 00:00 2023-09-13T03:00:39.681 INFO:teuthology.orchestra.run.smithi170.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-09-13T03:00:39.696 INFO:teuthology.orchestra.run.smithi064.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 9.9 MB/s | 1.2 MB 00:00 2023-09-13T03:00:39.721 INFO:teuthology.orchestra.run.smithi064.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 1.0 MB/s | 59 kB 00:00 2023-09-13T03:00:39.746 INFO:teuthology.orchestra.run.smithi064.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.2 MB/s | 60 kB 00:00 2023-09-13T03:00:39.780 INFO:teuthology.orchestra.run.smithi064.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 469 kB/s | 16 kB 00:00 2023-09-13T03:00:39.805 INFO:teuthology.orchestra.run.smithi064.stdout:(56/131): python3-pecan-1.3.2-9.el8.noarch.rpm 3.3 MB/s | 283 kB 00:00 2023-09-13T03:00:39.830 INFO:teuthology.orchestra.run.smithi064.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 671 kB/s | 34 kB 00:00 2023-09-13T03:00:39.864 INFO:teuthology.orchestra.run.smithi064.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 3.3 MB/s | 196 kB 00:00 2023-09-13T03:00:39.889 INFO:teuthology.orchestra.run.smithi064.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 1.1 MB/s | 64 kB 00:00 2023-09-13T03:00:39.914 INFO:teuthology.orchestra.run.smithi064.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 374 kB/s | 19 kB 00:00 2023-09-13T03:00:39.915 INFO:teuthology.orchestra.run.smithi170.stdout:(106/131): ceph-common-18.0.0-6088.g2110e007.el 1.4 MB/s | 22 MB 00:15 2023-09-13T03:00:39.940 INFO:teuthology.orchestra.run.smithi064.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 474 kB/s | 24 kB 00:00 2023-09-13T03:00:39.941 INFO:teuthology.orchestra.run.smithi170.stdout:(107/131): keyutils-libs-devel-1.5.10-9.el8.x86 169 kB/s | 48 kB 00:00 2023-09-13T03:00:39.965 INFO:teuthology.orchestra.run.smithi064.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 583 kB/s | 29 kB 00:00 2023-09-13T03:00:39.974 INFO:teuthology.orchestra.run.smithi170.stdout:(108/131): libsepol-devel-2.9-3.el8.x86_64.rpm 298 kB/s | 87 kB 00:00 2023-09-13T03:00:39.991 INFO:teuthology.orchestra.run.smithi064.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 549 kB/s | 27 kB 00:00 2023-09-13T03:00:39.999 INFO:teuthology.orchestra.run.smithi170.stdout:(109/131): python3-requests-oauthlib-1.0.0-1.el 511 kB/s | 43 kB 00:00 2023-09-13T03:00:40.016 INFO:teuthology.orchestra.run.smithi064.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 5.0 MB/s | 255 kB 00:00 2023-09-13T03:00:40.024 INFO:teuthology.orchestra.run.smithi170.stdout:(110/131): python3-pycparser-2.14-14.el8.noarch 1.4 MB/s | 109 kB 00:00 2023-09-13T03:00:40.041 INFO:teuthology.orchestra.run.smithi064.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 4.9 MB/s | 251 kB 00:00 2023-09-13T03:00:40.049 INFO:teuthology.orchestra.run.smithi170.stdout:(111/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.0 MB/s | 229 kB 00:00 2023-09-13T03:00:40.066 INFO:teuthology.orchestra.run.smithi064.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 1.2 MB/s | 61 kB 00:00 2023-09-13T03:00:40.075 INFO:teuthology.orchestra.run.smithi170.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 373 kB/s | 19 kB 00:00 2023-09-13T03:00:40.100 INFO:teuthology.orchestra.run.smithi170.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.0 MB/s | 155 kB 00:00 2023-09-13T03:00:40.101 INFO:teuthology.orchestra.run.smithi064.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.4 MB/s | 85 kB 00:00 2023-09-13T03:00:40.126 INFO:teuthology.orchestra.run.smithi170.stdout:(114/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 4.3 MB/s | 560 kB 00:00 2023-09-13T03:00:40.126 INFO:teuthology.orchestra.run.smithi064.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 389 kB/s | 23 kB 00:00 2023-09-13T03:00:40.151 INFO:teuthology.orchestra.run.smithi170.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 356 kB/s | 18 kB 00:00 2023-09-13T03:00:40.151 INFO:teuthology.orchestra.run.smithi064.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-09-13T03:00:40.193 INFO:teuthology.orchestra.run.smithi170.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 4.6 MB/s | 559 kB 00:00 2023-09-13T03:00:40.218 INFO:teuthology.orchestra.run.smithi170.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.5 MB/s | 238 kB 00:00 2023-09-13T03:00:40.235 INFO:teuthology.orchestra.run.smithi064.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 13 MB/s | 1.7 MB 00:00 2023-09-13T03:00:40.243 INFO:teuthology.orchestra.run.smithi170.stdout:(118/131): pcre2-devel-10.32-2.el8.x86_64.rpm 5.0 MB/s | 605 kB 00:00 2023-09-13T03:00:40.269 INFO:teuthology.orchestra.run.smithi170.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.4 MB/s | 72 kB 00:00 2023-09-13T03:00:40.286 INFO:teuthology.orchestra.run.smithi170.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.9 MB/s | 83 kB 00:00 2023-09-13T03:00:40.293 INFO:teuthology.orchestra.run.smithi064.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.8 MB/s | 110 kB 00:00 2023-09-13T03:00:40.302 INFO:teuthology.orchestra.run.smithi170.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.7 MB/s | 60 kB 00:00 2023-09-13T03:00:40.328 INFO:teuthology.orchestra.run.smithi170.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 2.3 MB/s | 98 kB 00:00 2023-09-13T03:00:40.335 INFO:teuthology.orchestra.run.smithi064.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-09-13T03:00:40.345 INFO:teuthology.orchestra.run.smithi170.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.3 MB/s | 58 kB 00:00 2023-09-13T03:00:40.362 INFO:teuthology.orchestra.run.smithi170.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 1.1 MB/s | 39 kB 00:00 2023-09-13T03:00:40.377 INFO:teuthology.orchestra.run.smithi064.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.2 MB/s | 259 kB 00:00 2023-09-13T03:00:40.395 INFO:teuthology.orchestra.run.smithi170.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 12 MB/s | 644 kB 00:00 2023-09-13T03:00:40.427 INFO:teuthology.orchestra.run.smithi064.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2023-09-13T03:00:40.429 INFO:teuthology.orchestra.run.smithi170.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.5 MB/s | 171 kB 00:00 2023-09-13T03:00:40.454 INFO:teuthology.orchestra.run.smithi170.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.3 MB/s | 138 kB 00:00 2023-09-13T03:00:40.461 INFO:teuthology.orchestra.run.smithi064.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 1.1 MB/s | 39 kB 00:00 2023-09-13T03:00:40.513 INFO:teuthology.orchestra.run.smithi170.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 7.3 MB/s | 2.3 MB 00:00 2023-09-13T03:00:40.545 INFO:teuthology.orchestra.run.smithi064.stdout:(76/131): bison-3.0.4-10.el8.x86_64.rpm 8.1 MB/s | 688 kB 00:00 2023-09-13T03:00:40.580 INFO:teuthology.orchestra.run.smithi170.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 12 MB/s | 798 kB 00:00 2023-09-13T03:00:40.595 INFO:teuthology.orchestra.run.smithi064.stdout:(77/131): python3-jinja2-2.10.1-3.el8.noarch.rp 11 MB/s | 538 kB 00:00 2023-09-13T03:00:40.637 INFO:teuthology.orchestra.run.smithi064.stdout:(78/131): python3-mako-1.0.6-13.el8.noarch.rpm 3.7 MB/s | 157 kB 00:00 2023-09-13T03:00:40.704 INFO:teuthology.orchestra.run.smithi064.stdout:(79/131): oniguruma-6.8.2-2.el8.x86_64.rpm 2.7 MB/s | 187 kB 00:00 2023-09-13T03:00:40.771 INFO:teuthology.orchestra.run.smithi064.stdout:(80/131): automake-1.16.1-7.el8.noarch.rpm 10 MB/s | 713 kB 00:00 2023-09-13T03:00:40.880 INFO:teuthology.orchestra.run.smithi170.stdout:(130/131): librbd1-18.0.0-6088.g2110e007.el8.x8 8.7 MB/s | 3.7 MB 00:00 2023-09-13T03:00:41.013 INFO:teuthology.orchestra.run.smithi064.stdout:(81/131): python3-scipy-1.0.0-21.module+el8.5.0 16 MB/s | 14 MB 00:00 2023-09-13T03:00:41.038 INFO:teuthology.orchestra.run.smithi064.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 1.0 MB/s | 27 kB 00:00 2023-09-13T03:00:41.113 INFO:teuthology.orchestra.run.smithi064.stdout:(83/131): python3-babel-2.5.1-7.el8.noarch.rpm 14 MB/s | 4.8 MB 00:00 2023-09-13T03:00:41.147 INFO:teuthology.orchestra.run.smithi064.stdout:(84/131): flex-2.6.1-9.el8.x86_64.rpm 2.9 MB/s | 320 kB 00:00 2023-09-13T03:00:41.172 INFO:teuthology.orchestra.run.smithi170.stdout:(131/131): librados2-18.0.0-6088.g2110e007.el8. 5.0 MB/s | 3.7 MB 00:00 2023-09-13T03:00:41.172 INFO:teuthology.orchestra.run.smithi064.stdout:(85/131): python3-prettytable-0.7.2-14.el8.noar 751 kB/s | 44 kB 00:00 2023-09-13T03:00:41.176 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2023-09-13T03:00:41.176 INFO:teuthology.orchestra.run.smithi170.stdout:Total 11 MB/s | 208 MB 00:18 2023-09-13T03:00:41.222 INFO:teuthology.orchestra.run.smithi064.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 9.0 MB/s | 457 kB 00:00 2023-09-13T03:00:41.264 INFO:teuthology.orchestra.run.smithi064.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 5.3 MB/s | 225 kB 00:00 2023-09-13T03:00:41.306 INFO:teuthology.orchestra.run.smithi064.stdout:(88/131): uuid-1.6.2-43.el8.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2023-09-13T03:00:41.340 INFO:teuthology.orchestra.run.smithi064.stdout:(89/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 3.7 MB/s | 126 kB 00:00 2023-09-13T03:00:41.407 INFO:teuthology.orchestra.run.smithi064.stdout:(90/131): python3-numpy-1.14.3-10.el8.x86_64.rp 14 MB/s | 3.7 MB 00:00 2023-09-13T03:00:41.432 INFO:teuthology.orchestra.run.smithi064.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 217 kB/s | 20 kB 00:00 2023-09-13T03:00:41.458 INFO:teuthology.orchestra.run.smithi064.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2023-09-13T03:00:41.508 INFO:teuthology.orchestra.run.smithi064.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 9.2 MB/s | 709 kB 00:00 2023-09-13T03:00:41.550 INFO:teuthology.orchestra.run.smithi064.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-09-13T03:00:42.010 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2023-09-13T03:00:42.118 INFO:teuthology.orchestra.run.smithi064.stdout:(95/131): ceph-test-18.0.0-6088.g2110e007.el8.x 6.2 MB/s | 49 MB 00:07 2023-09-13T03:00:42.129 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2023-09-13T03:00:42.129 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2023-09-13T03:00:42.184 INFO:teuthology.orchestra.run.smithi064.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 3.0 MB/s | 202 kB 00:00 2023-09-13T03:00:42.235 INFO:teuthology.orchestra.run.smithi064.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 6.9 MB/s | 354 kB 00:00 2023-09-13T03:00:42.318 INFO:teuthology.orchestra.run.smithi064.stdout:(98/131): openblas-threads-0.3.15-3.el8.x86_64. 5.6 MB/s | 4.9 MB 00:00 2023-09-13T03:00:42.427 INFO:teuthology.orchestra.run.smithi064.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 5.4 MB/s | 4.7 MB 00:00 2023-09-13T03:00:42.477 INFO:teuthology.orchestra.run.smithi064.stdout:(100/131): postgresql-contrib-10.21-2.module+el 5.0 MB/s | 808 kB 00:00 2023-09-13T03:00:42.636 INFO:teuthology.orchestra.run.smithi064.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 296 kB/s | 47 kB 00:00 2023-09-13T03:00:42.677 INFO:teuthology.orchestra.run.smithi064.stdout:(102/131): libselinux-devel-2.9-5.el8.x86_64.rp 4.7 MB/s | 200 kB 00:00 2023-09-13T03:00:42.719 INFO:teuthology.orchestra.run.smithi064.stdout:(103/131): postgresql-10.21-2.module+el8.6.0+15 5.1 MB/s | 1.5 MB 00:00 2023-09-13T03:00:42.745 INFO:teuthology.orchestra.run.smithi064.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 638 kB/s | 43 kB 00:00 2023-09-13T03:00:42.778 INFO:teuthology.orchestra.run.smithi064.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 3.7 MB/s | 220 kB 00:00 2023-09-13T03:00:42.804 INFO:teuthology.orchestra.run.smithi064.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 821 kB/s | 48 kB 00:00 2023-09-13T03:00:42.829 INFO:teuthology.orchestra.run.smithi064.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-09-13T03:00:42.855 INFO:teuthology.orchestra.run.smithi064.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 850 kB/s | 43 kB 00:00 2023-09-13T03:00:42.880 INFO:teuthology.orchestra.run.smithi064.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-09-13T03:00:42.913 INFO:teuthology.orchestra.run.smithi064.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.8 MB/s | 229 kB 00:00 2023-09-13T03:00:42.963 INFO:teuthology.orchestra.run.smithi064.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 6.6 MB/s | 560 kB 00:00 2023-09-13T03:00:42.980 INFO:teuthology.orchestra.run.smithi064.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 281 kB/s | 19 kB 00:00 2023-09-13T03:00:43.006 INFO:teuthology.orchestra.run.smithi064.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.6 MB/s | 155 kB 00:00 2023-09-13T03:00:43.039 INFO:teuthology.orchestra.run.smithi064.stdout:(114/131): libverto-devel-0.3.0-5.el8.x86_64.rp 539 kB/s | 18 kB 00:00 2023-09-13T03:00:43.073 INFO:teuthology.orchestra.run.smithi064.stdout:(115/131): python3-cryptography-3.2.1-5.el8.x86 5.9 MB/s | 559 kB 00:00 2023-09-13T03:00:43.148 INFO:teuthology.orchestra.run.smithi064.stdout:(116/131): postgresql-server-10.21-2.module+el8 5.5 MB/s | 5.1 MB 00:00 2023-09-13T03:00:43.182 INFO:teuthology.orchestra.run.smithi064.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.1 MB/s | 238 kB 00:00 2023-09-13T03:00:43.224 INFO:teuthology.orchestra.run.smithi064.stdout:(118/131): pcre2-devel-10.32-2.el8.x86_64.rpm 3.2 MB/s | 605 kB 00:00 2023-09-13T03:00:43.241 INFO:teuthology.orchestra.run.smithi064.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.2 MB/s | 72 kB 00:00 2023-09-13T03:00:43.266 INFO:teuthology.orchestra.run.smithi064.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.9 MB/s | 83 kB 00:00 2023-09-13T03:00:43.291 INFO:teuthology.orchestra.run.smithi064.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-09-13T03:00:43.304 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2023-09-13T03:00:43.317 INFO:teuthology.orchestra.run.smithi064.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-09-13T03:00:43.334 INFO:teuthology.orchestra.run.smithi064.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.3 MB/s | 58 kB 00:00 2023-09-13T03:00:43.335 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2023-09-13T03:00:43.351 INFO:teuthology.orchestra.run.smithi064.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 1.1 MB/s | 39 kB 00:00 2023-09-13T03:00:43.401 INFO:teuthology.orchestra.run.smithi064.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 9.4 MB/s | 644 kB 00:00 2023-09-13T03:00:43.426 INFO:teuthology.orchestra.run.smithi064.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.2 MB/s | 171 kB 00:00 2023-09-13T03:00:43.451 INFO:teuthology.orchestra.run.smithi064.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-09-13T03:00:43.519 INFO:teuthology.orchestra.run.smithi064.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 6.3 MB/s | 2.3 MB 00:00 2023-09-13T03:00:43.586 INFO:teuthology.orchestra.run.smithi064.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 12 MB/s | 798 kB 00:00 2023-09-13T03:00:44.061 INFO:teuthology.orchestra.run.smithi064.stdout:(130/131): librados2-18.0.0-6088.g2110e007.el8. 5.9 MB/s | 3.7 MB 00:00 2023-09-13T03:00:44.286 INFO:teuthology.orchestra.run.smithi064.stdout:(131/131): librbd1-18.0.0-6088.g2110e007.el8.x8 4.4 MB/s | 3.7 MB 00:00 2023-09-13T03:00:44.290 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2023-09-13T03:00:44.290 INFO:teuthology.orchestra.run.smithi064.stdout:Total 14 MB/s | 208 MB 00:15 2023-09-13T03:00:44.409 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2023-09-13T03:00:44.700 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-09-13T03:00:44.892 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-09-13T03:00:45.093 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-09-13T03:00:45.209 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-09-13T03:00:45.409 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-09-13T03:00:45.848 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-09-13T03:00:45.974 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2023-09-13T03:00:46.084 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : librados2-2:18.0.0-6088.g2110e007.el8.x86_64 5/134 2023-09-13T03:00:46.122 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2023-09-13T03:00:46.123 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2023-09-13T03:00:46.496 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librados2-2:18.0.0-6088.g2110e007.el8.x86_64 5/134 2023-09-13T03:00:46.667 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : librbd1-2:18.0.0-6088.g2110e007.el8.x86_64 6/134 2023-09-13T03:00:46.836 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librbd1-2:18.0.0-6088.g2110e007.el8.x86_64 6/134 2023-09-13T03:00:47.025 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephfs2-2:18.0.0-6088.g2110e007.el8.x86_64 7/134 2023-09-13T03:00:47.111 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libcephfs2-2:18.0.0-6088.g2110e007.el8.x86_64 7/134 2023-09-13T03:00:47.305 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rados-2:18.0.0-6088.g2110e007.el8.x86_64 8/134 2023-09-13T03:00:47.490 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-09-13T03:00:47.754 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-09-13T03:00:47.917 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2023-09-13T03:00:47.922 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2023-09-13T03:00:47.954 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-09-13T03:00:48.306 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2023-09-13T03:00:48.351 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-09-13T03:00:48.470 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-09-13T03:00:48.616 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-09-13T03:00:48.624 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-09-13T03:00:48.767 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephsqlite-2:18.0.0-6088.g2110e007.el8.x86_64 14/134 2023-09-13T03:00:48.812 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-09-13T03:00:48.878 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6088.g2110e007.el8.x86_64 14/134 2023-09-13T03:00:49.033 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libradosstriper1-2:18.0.0-6088.g2110e007.el8.x86 15/134 2023-09-13T03:00:49.183 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6088.g2110e007.el8.x86 15/134 2023-09-13T03:00:49.420 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-09-13T03:00:49.455 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-09-13T03:00:49.573 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-09-13T03:00:49.606 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-09-13T03:00:49.805 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-09-13T03:00:49.825 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-09-13T03:00:50.206 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-09-13T03:00:50.438 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : librados2-2:18.0.0-6088.g2110e007.el8.x86_64 6/134 2023-09-13T03:00:50.600 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-09-13T03:00:50.629 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-09-13T03:00:50.808 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-09-13T03:00:50.829 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librados2-2:18.0.0-6088.g2110e007.el8.x86_64 6/134 2023-09-13T03:00:51.021 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : librbd1-2:18.0.0-6088.g2110e007.el8.x86_64 7/134 2023-09-13T03:00:51.151 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librbd1-2:18.0.0-6088.g2110e007.el8.x86_64 7/134 2023-09-13T03:00:51.330 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephfs2-2:18.0.0-6088.g2110e007.el8.x86_64 8/134 2023-09-13T03:00:51.431 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-09-13T03:00:51.436 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libcephfs2-2:18.0.0-6088.g2110e007.el8.x86_64 8/134 2023-09-13T03:00:51.622 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rados-2:18.0.0-6088.g2110e007.el8.x86_64 9/134 2023-09-13T03:00:51.633 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-09-13T03:00:51.841 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-09-13T03:00:52.049 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-09-13T03:00:52.230 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-09-13T03:00:52.443 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-09-13T03:00:52.450 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-09-13T03:00:52.695 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-09-13T03:00:52.842 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2023-09-13T03:00:52.846 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephsqlite-2:18.0.0-6088.g2110e007.el8.x86_64 14/134 2023-09-13T03:00:52.957 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6088.g2110e007.el8.x86_64 14/134 2023-09-13T03:00:53.024 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-09-13T03:00:53.088 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libradosstriper1-2:18.0.0-6088.g2110e007.el8.x86 15/134 2023-09-13T03:00:53.138 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-09-13T03:00:53.171 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6088.g2110e007.el8.x86 15/134 2023-09-13T03:00:53.188 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-09-13T03:00:53.304 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-09-13T03:00:53.387 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-09-13T03:00:53.415 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-09-13T03:00:53.431 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-09-13T03:00:53.550 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-09-13T03:00:53.588 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-09-13T03:00:53.588 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-09-13T03:00:53.693 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-09-13T03:00:53.844 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-09-13T03:00:53.942 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-09-13T03:00:54.006 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ceph-common-2:18.0.0-6088.g2110e007.el8. 26/134 2023-09-13T03:00:54.161 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ceph-argparse-2:18.0.0-6088.g2110e007.el 27/134 2023-09-13T03:00:54.188 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-09-13T03:00:54.362 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cephfs-2:18.0.0-6088.g2110e007.el8.x86_6 28/134 2023-09-13T03:00:54.362 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-09-13T03:00:54.438 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-09-13T03:00:54.566 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-09-13T03:00:54.674 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : librados2-2:18.0.0-6088.g2110e007.el8.x86_64 6/134 2023-09-13T03:00:54.946 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-09-13T03:00:55.137 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-09-13T03:00:55.170 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados2-2:18.0.0-6088.g2110e007.el8.x86_64 6/134 2023-09-13T03:00:55.333 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : librbd1-2:18.0.0-6088.g2110e007.el8.x86_64 7/134 2023-09-13T03:00:55.511 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librbd1-2:18.0.0-6088.g2110e007.el8.x86_64 7/134 2023-09-13T03:00:55.675 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephfs2-2:18.0.0-6088.g2110e007.el8.x86_64 8/134 2023-09-13T03:00:55.798 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephfs2-2:18.0.0-6088.g2110e007.el8.x86_64 8/134 2023-09-13T03:00:55.829 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-09-13T03:00:55.980 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rados-2:18.0.0-6088.g2110e007.el8.x86_64 9/134 2023-09-13T03:00:56.127 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-09-13T03:00:56.219 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-09-13T03:00:56.312 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-09-13T03:00:56.329 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-09-13T03:00:56.414 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-09-13T03:00:56.566 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-09-13T03:00:56.760 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-09-13T03:00:56.834 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-09-13T03:00:56.951 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-09-13T03:00:57.068 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-09-13T03:00:57.189 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-09-13T03:00:57.207 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephsqlite-2:18.0.0-6088.g2110e007.el8.x86_64 14/134 2023-09-13T03:00:57.342 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6088.g2110e007.el8.x86_64 14/134 2023-09-13T03:00:57.349 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-09-13T03:00:57.351 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-ceph-common-2:18.0.0-6088.g2110e007.el8. 26/134 2023-09-13T03:00:57.473 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-ceph-argparse-2:18.0.0-6088.g2110e007.el 27/134 2023-09-13T03:00:57.662 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-09-13T03:00:57.674 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libradosstriper1-2:18.0.0-6088.g2110e007.el8.x86 15/134 2023-09-13T03:00:57.777 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6088.g2110e007.el8.x86 15/134 2023-09-13T03:00:57.907 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-09-13T03:00:57.942 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-09-13T03:00:58.088 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-09-13T03:00:58.155 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-09-13T03:00:58.249 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-09-13T03:00:58.373 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cephfs-2:18.0.0-6088.g2110e007.el8.x86_6 28/134 2023-09-13T03:00:58.401 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-09-13T03:00:58.568 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-09-13T03:00:58.677 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-09-13T03:00:58.762 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rbd-2:18.0.0-6088.g2110e007.el8.x86_64 36/134 2023-09-13T03:00:58.905 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librados-devel-2:18.0.0-6088.g2110e007.el8.x86_6 37/134 2023-09-13T03:00:58.913 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-09-13T03:00:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-09-13T03:00:59.047 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-09-13T03:00:59.139 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-09-13T03:00:59.191 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-09-13T03:00:59.224 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-09-13T03:00:59.320 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-09-13T03:00:59.365 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-09-13T03:00:59.439 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-09-13T03:00:59.702 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-09-13T03:00:59.909 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-09-13T03:00:59.918 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-09-13T03:01:00.099 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-09-13T03:01:00.099 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-09-13T03:01:00.175 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-09-13T03:01:00.394 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-09-13T03:01:00.610 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-09-13T03:01:00.780 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-09-13T03:01:00.983 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-09-13T03:01:00.986 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-09-13T03:01:01.066 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-09-13T03:01:01.142 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-09-13T03:01:01.240 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-09-13T03:01:01.318 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-09-13T03:01:01.378 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-09-13T03:01:01.515 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-09-13T03:01:01.531 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-09-13T03:01:01.598 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-09-13T03:01:01.723 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-09-13T03:01:01.763 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-09-13T03:01:01.772 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-09-13T03:01:01.984 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rbd-2:18.0.0-6088.g2110e007.el8.x86_64 36/134 2023-09-13T03:01:01.986 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-09-13T03:01:02.131 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-09-13T03:01:02.141 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librados-devel-2:18.0.0-6088.g2110e007.el8.x86_6 37/134 2023-09-13T03:01:02.196 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-09-13T03:01:02.335 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-09-13T03:01:02.424 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-09-13T03:01:02.447 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-09-13T03:01:02.487 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-09-13T03:01:02.684 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-09-13T03:01:02.705 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-09-13T03:01:02.848 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-09-13T03:01:02.881 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ceph-common-2:18.0.0-6088.g2110e007.el8. 26/134 2023-09-13T03:01:02.885 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-09-13T03:01:03.013 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ceph-argparse-2:18.0.0-6088.g2110e007.el 27/134 2023-09-13T03:01:03.059 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-09-13T03:01:03.144 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-09-13T03:01:03.242 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-09-13T03:01:03.497 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-09-13T03:01:03.596 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-09-13T03:01:03.697 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-09-13T03:01:03.872 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-09-13T03:01:03.923 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-09-13T03:01:04.070 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-09-13T03:01:04.097 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cephfs-2:18.0.0-6088.g2110e007.el8.x86_6 28/134 2023-09-13T03:01:04.135 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-09-13T03:01:04.272 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-09-13T03:01:04.343 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-09-13T03:01:04.487 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-09-13T03:01:04.517 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-09-13T03:01:04.532 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-09-13T03:01:04.768 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-09-13T03:01:04.843 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-09-13T03:01:04.849 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-09-13T03:01:05.033 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-09-13T03:01:05.069 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-09-13T03:01:05.110 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-09-13T03:01:05.215 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-09-13T03:01:05.241 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-09-13T03:01:05.344 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-09-13T03:01:05.449 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-09-13T03:01:05.582 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-09-13T03:01:05.607 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-09-13T03:01:05.730 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-09-13T03:01:05.764 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-09-13T03:01:05.908 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-09-13T03:01:06.060 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-09-13T03:01:06.172 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-09-13T03:01:06.365 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-09-13T03:01:06.420 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-09-13T03:01:06.568 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-09-13T03:01:06.598 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-09-13T03:01:06.715 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-09-13T03:01:06.842 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-09-13T03:01:06.921 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-09-13T03:01:06.992 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-09-13T03:01:07.115 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-09-13T03:01:07.120 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-09-13T03:01:07.294 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-09-13T03:01:07.320 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-09-13T03:01:07.367 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-09-13T03:01:07.400 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-09-13T03:01:07.483 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-09-13T03:01:07.594 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-09-13T03:01:07.654 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-09-13T03:01:07.732 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-09-13T03:01:07.846 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-09-13T03:01:07.847 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-09-13T03:01:07.946 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-09-13T03:01:07.993 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-09-13T03:01:08.058 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-09-13T03:01:08.070 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-09-13T03:01:08.139 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-09-13T03:01:08.252 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-09-13T03:01:08.309 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-09-13T03:01:08.339 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rbd-2:18.0.0-6088.g2110e007.el8.x86_64 36/134 2023-09-13T03:01:08.470 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-09-13T03:01:08.478 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-09-13T03:01:08.520 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librados-devel-2:18.0.0-6088.g2110e007.el8.x86_6 37/134 2023-09-13T03:01:08.628 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-09-13T03:01:08.691 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-09-13T03:01:08.704 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-09-13T03:01:08.815 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-09-13T03:01:08.860 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-09-13T03:01:08.861 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-09-13T03:01:09.026 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-09-13T03:01:09.041 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-09-13T03:01:09.075 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-09-13T03:01:09.171 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-09-13T03:01:09.242 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-09-13T03:01:09.372 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-09-13T03:01:09.373 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-09-13T03:01:09.505 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-09-13T03:01:09.590 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-09-13T03:01:09.704 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-09-13T03:01:09.906 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-09-13T03:01:09.989 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-09-13T03:01:10.095 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-09-13T03:01:10.140 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-09-13T03:01:10.263 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-09-13T03:01:10.290 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-09-13T03:01:10.383 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-09-13T03:01:10.450 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-09-13T03:01:10.497 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-09-13T03:01:10.582 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-09-13T03:01:10.729 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-09-13T03:01:10.756 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-09-13T03:01:10.806 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-09-13T03:01:10.982 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-09-13T03:01:10.993 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-09-13T03:01:11.090 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-09-13T03:01:11.266 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-09-13T03:01:11.316 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-09-13T03:01:11.354 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-09-13T03:01:11.484 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-09-13T03:01:11.499 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-09-13T03:01:11.598 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-09-13T03:01:11.633 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-09-13T03:01:11.708 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-09-13T03:01:11.832 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-09-13T03:01:11.927 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-09-13T03:01:12.115 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-09-13T03:01:12.288 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-09-13T03:01:12.374 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-09-13T03:01:12.529 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librgw2-2:18.0.0-6088.g2110e007.el8.x86_64 92/134 2023-09-13T03:01:12.586 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librgw2-2:18.0.0-6088.g2110e007.el8.x86_64 92/134 2023-09-13T03:01:12.627 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-09-13T03:01:12.780 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rgw-2:18.0.0-6088.g2110e007.el8.x86_64 93/134 2023-09-13T03:01:12.905 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-09-13T03:01:12.925 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-09-13T03:01:13.154 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-09-13T03:01:13.191 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-09-13T03:01:13.389 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-09-13T03:01:13.402 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-09-13T03:01:13.573 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-09-13T03:01:13.735 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-09-13T03:01:13.760 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-09-13T03:01:13.930 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-09-13T03:01:14.006 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-09-13T03:01:14.018 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-09-13T03:01:14.225 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-09-13T03:01:14.266 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-09-13T03:01:14.470 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-09-13T03:01:14.647 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-09-13T03:01:14.743 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-09-13T03:01:14.943 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-09-13T03:01:15.005 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-09-13T03:01:15.108 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-09-13T03:01:15.209 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librgw2-2:18.0.0-6088.g2110e007.el8.x86_64 92/134 2023-09-13T03:01:15.257 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-09-13T03:01:15.284 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librgw2-2:18.0.0-6088.g2110e007.el8.x86_64 92/134 2023-09-13T03:01:15.440 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-09-13T03:01:15.459 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rgw-2:18.0.0-6088.g2110e007.el8.x86_64 93/134 2023-09-13T03:01:15.710 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-09-13T03:01:15.922 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-09-13T03:01:16.094 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-09-13T03:01:16.253 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-09-13T03:01:16.465 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-09-13T03:01:16.493 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-common-2:18.0.0-6088.g2110e007.el8.x86_64 94/134 2023-09-13T03:01:16.665 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-09-13T03:01:16.746 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-common-2:18.0.0-6088.g2110e007.el8.x86_64 94/134 2023-09-13T03:01:16.847 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-09-13T03:01:17.026 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-09-13T03:01:17.154 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-09-13T03:01:17.349 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-common-2:18.0.0-6088.g2110e007.el8.x86_64 94/134 2023-09-13T03:01:17.384 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-09-13T03:01:17.610 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-base-2:18.0.0-6088.g2110e007.el8.x86_64 95/134 2023-09-13T03:01:17.620 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-09-13T03:01:17.695 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-base-2:18.0.0-6088.g2110e007.el8.x86_64 95/134 2023-09-13T03:01:17.813 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-09-13T03:01:17.862 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-selinux-2:18.0.0-6088.g2110e007.el8.x86_64 96/134 2023-09-13T03:01:18.130 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-09-13T03:01:18.339 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-09-13T03:01:18.516 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-09-13T03:01:18.741 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-09-13T03:01:19.168 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-common-2:18.0.0-6088.g2110e007.el8.x86_64 94/134 2023-09-13T03:01:19.409 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-common-2:18.0.0-6088.g2110e007.el8.x86_64 94/134 2023-09-13T03:01:20.008 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-common-2:18.0.0-6088.g2110e007.el8.x86_64 94/134 2023-09-13T03:01:20.172 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-09-13T03:01:20.444 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-09-13T03:01:20.703 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-09-13T03:01:21.066 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-base-2:18.0.0-6088.g2110e007.el8.x86_64 95/134 2023-09-13T03:01:21.087 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-09-13T03:01:21.148 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-base-2:18.0.0-6088.g2110e007.el8.x86_64 95/134 2023-09-13T03:01:21.283 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-selinux-2:18.0.0-6088.g2110e007.el8.x86_64 96/134 2023-09-13T03:01:21.366 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-09-13T03:01:21.568 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-09-13T03:01:22.430 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-09-13T03:01:22.587 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librgw2-2:18.0.0-6088.g2110e007.el8.x86_64 92/134 2023-09-13T03:01:22.686 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librgw2-2:18.0.0-6088.g2110e007.el8.x86_64 92/134 2023-09-13T03:01:22.846 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rgw-2:18.0.0-6088.g2110e007.el8.x86_64 93/134 2023-09-13T03:01:27.952 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-common-2:18.0.0-6088.g2110e007.el8.x86_64 94/134 2023-09-13T03:01:28.638 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-common-2:18.0.0-6088.g2110e007.el8.x86_64 94/134 2023-09-13T03:01:29.384 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-common-2:18.0.0-6088.g2110e007.el8.x86_64 94/134 2023-09-13T03:01:29.665 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-base-2:18.0.0-6088.g2110e007.el8.x86_64 95/134 2023-09-13T03:01:29.800 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-base-2:18.0.0-6088.g2110e007.el8.x86_64 95/134 2023-09-13T03:01:29.953 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-selinux-2:18.0.0-6088.g2110e007.el8.x86_64 96/134 2023-09-13T03:01:36.573 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6088.g2110e007.el8.x86_64 96/134 2023-09-13T03:01:36.574 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /sys 2023-09-13T03:01:36.574 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /proc 2023-09-13T03:01:36.574 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /dev 2023-09-13T03:01:36.574 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /run 2023-09-13T03:01:36.575 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /mnt 2023-09-13T03:01:36.575 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /var/tmp 2023-09-13T03:01:36.575 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /home 2023-09-13T03:01:36.575 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /tmp 2023-09-13T03:01:36.575 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /dev 2023-09-13T03:01:36.576 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-09-13T03:01:36.960 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-volume-2:18.0.0-6088.g2110e007.el8.noarch 97/134 2023-09-13T03:01:38.526 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-volume-2:18.0.0-6088.g2110e007.el8.noarch 97/134 2023-09-13T03:01:38.732 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-osd-2:18.0.0-6088.g2110e007.el8.x86_64 98/134 2023-09-13T03:01:38.955 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-osd-2:18.0.0-6088.g2110e007.el8.x86_64 98/134 2023-09-13T03:01:39.159 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mds-2:18.0.0-6088.g2110e007.el8.x86_64 99/134 2023-09-13T03:01:39.614 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mds-2:18.0.0-6088.g2110e007.el8.x86_64 99/134 2023-09-13T03:01:39.776 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mon-2:18.0.0-6088.g2110e007.el8.x86_64 100/134 2023-09-13T03:01:39.793 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mon-2:18.0.0-6088.g2110e007.el8.x86_64 100/134 2023-09-13T03:01:40.107 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6088.g2110e007.el8.x86_64 96/134 2023-09-13T03:01:40.107 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /sys 2023-09-13T03:01:40.107 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /proc 2023-09-13T03:01:40.107 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /dev 2023-09-13T03:01:40.107 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /run 2023-09-13T03:01:40.108 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /mnt 2023-09-13T03:01:40.108 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /var/tmp 2023-09-13T03:01:40.108 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /home 2023-09-13T03:01:40.108 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /tmp 2023-09-13T03:01:40.108 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /dev 2023-09-13T03:01:40.109 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-09-13T03:01:40.455 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-volume-2:18.0.0-6088.g2110e007.el8.noarch 97/134 2023-09-13T03:01:41.471 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: cephadm-2:18.0.0-6088.g2110e007.el8.noarch 101/134 2023-09-13T03:01:41.692 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephadm-2:18.0.0-6088.g2110e007.el8.noarch 101/134 2023-09-13T03:01:41.806 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6088.g2110e007.e 102/134 2023-09-13T03:01:42.021 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-volume-2:18.0.0-6088.g2110e007.el8.noarch 97/134 2023-09-13T03:01:42.026 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6088.g2110e007. 103/134 2023-09-13T03:01:42.191 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6088.g2110e007.el8.noa 104/134 2023-09-13T03:01:42.212 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-osd-2:18.0.0-6088.g2110e007.el8.x86_64 98/134 2023-09-13T03:01:42.407 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6088.g2110e007.el8.noa 104/134 2023-09-13T03:01:42.435 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-osd-2:18.0.0-6088.g2110e007.el8.x86_64 98/134 2023-09-13T03:01:42.622 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mds-2:18.0.0-6088.g2110e007.el8.x86_64 99/134 2023-09-13T03:01:42.706 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6088.g2110e007.el8.n 105/134 2023-09-13T03:01:43.076 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mds-2:18.0.0-6088.g2110e007.el8.x86_64 99/134 2023-09-13T03:01:43.247 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mon-2:18.0.0-6088.g2110e007.el8.x86_64 100/134 2023-09-13T03:01:43.264 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mon-2:18.0.0-6088.g2110e007.el8.x86_64 100/134 2023-09-13T03:01:43.665 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6088.g2110e007.el8.n 105/134 2023-09-13T03:01:43.858 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6088.g211 106/134 2023-09-13T03:01:43.949 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6088.g211 106/134 2023-09-13T03:01:44.315 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6088.g2110e007.el 107/134 2023-09-13T03:01:44.535 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-2:18.0.0-6088.g2110e007.el8.x86_64 108/134 2023-09-13T03:01:44.571 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6088.g2110e007.el8.x86_64 108/134 2023-09-13T03:01:44.733 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-rook-2:18.0.0-6088.g2110e007.el8.noarch 109/134 2023-09-13T03:01:44.744 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6088.g2110e007.el8.noarch 109/134 2023-09-13T03:01:44.962 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: cephadm-2:18.0.0-6088.g2110e007.el8.noarch 101/134 2023-09-13T03:01:45.156 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephadm-2:18.0.0-6088.g2110e007.el8.noarch 101/134 2023-09-13T03:01:45.269 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6088.g2110e007.e 102/134 2023-09-13T03:01:45.453 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6088.g2110e007. 103/134 2023-09-13T03:01:45.671 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6088.g2110e007.el8.noa 104/134 2023-09-13T03:01:45.890 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6088.g2110e007.el8.noa 104/134 2023-09-13T03:01:46.063 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-2:18.0.0-6088.g2110e007.el8.x86_64 110/134 2023-09-13T03:01:46.203 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6088.g2110e007.el8.n 105/134 2023-09-13T03:01:46.271 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-radosgw-2:18.0.0-6088.g2110e007.el8.x86_64 111/134 2023-09-13T03:01:46.305 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6088.g2110e007.el8.x86_64 111/134 2023-09-13T03:01:46.441 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6088.g2110e 112/134 2023-09-13T03:01:46.488 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6088.g2110e 112/134 2023-09-13T03:01:46.691 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephfs-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 113/134 2023-09-13T03:01:47.052 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 113/134 2023-09-13T03:01:47.163 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6088.g2110e007.el8.n 105/134 2023-09-13T03:01:47.224 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 114/134 2023-09-13T03:01:47.362 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6088.g211 106/134 2023-09-13T03:01:47.454 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6088.g211 106/134 2023-09-13T03:01:47.786 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6088.g2110e007.el 107/134 2023-09-13T03:01:47.989 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-2:18.0.0-6088.g2110e007.el8.x86_64 108/134 2023-09-13T03:01:48.025 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6088.g2110e007.el8.x86_64 108/134 2023-09-13T03:01:48.195 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-rook-2:18.0.0-6088.g2110e007.el8.noarch 109/134 2023-09-13T03:01:48.207 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6088.g2110e007.el8.noarch 109/134 2023-09-13T03:01:49.522 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-2:18.0.0-6088.g2110e007.el8.x86_64 110/134 2023-09-13T03:01:49.698 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-radosgw-2:18.0.0-6088.g2110e007.el8.x86_64 111/134 2023-09-13T03:01:49.732 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6088.g2110e007.el8.x86_64 111/134 2023-09-13T03:01:49.871 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6088.g2110e 112/134 2023-09-13T03:01:49.918 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6088.g2110e 112/134 2023-09-13T03:01:50.112 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephfs-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 113/134 2023-09-13T03:01:50.470 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 113/134 2023-09-13T03:01:50.654 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 114/134 2023-09-13T03:01:51.065 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6088.g2110e007.el8.x86_64 96/134 2023-09-13T03:01:51.065 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /sys 2023-09-13T03:01:51.065 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /proc 2023-09-13T03:01:51.065 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /dev 2023-09-13T03:01:51.066 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /run 2023-09-13T03:01:51.066 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /mnt 2023-09-13T03:01:51.066 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /var/tmp 2023-09-13T03:01:51.066 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /home 2023-09-13T03:01:51.066 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /tmp 2023-09-13T03:01:51.067 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /dev 2023-09-13T03:01:51.067 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:01:51.426 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-volume-2:18.0.0-6088.g2110e007.el8.noarch 97/134 2023-09-13T03:01:51.894 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 114/134 2023-09-13T03:01:52.782 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-test-2:18.0.0-6088.g2110e007.el8.x86_64 115/134 2023-09-13T03:01:53.118 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-09-13T03:01:53.272 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-volume-2:18.0.0-6088.g2110e007.el8.noarch 97/134 2023-09-13T03:01:53.292 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-fuse-2:18.0.0-6088.g2110e007.el8.x86_64 117/134 2023-09-13T03:01:53.470 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-osd-2:18.0.0-6088.g2110e007.el8.x86_64 98/134 2023-09-13T03:01:53.572 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-09-13T03:01:53.715 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-09-13T03:01:53.779 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-osd-2:18.0.0-6088.g2110e007.el8.x86_64 98/134 2023-09-13T03:01:53.916 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-09-13T03:01:53.959 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mds-2:18.0.0-6088.g2110e007.el8.x86_64 99/134 2023-09-13T03:01:54.127 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-09-13T03:01:54.327 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephfs-devel-2:18.0.0-6088.g2110e007.el8.x86_ 121/134 2023-09-13T03:01:54.491 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-09-13T03:01:54.520 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mds-2:18.0.0-6088.g2110e007.el8.x86_64 99/134 2023-09-13T03:01:54.725 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mon-2:18.0.0-6088.g2110e007.el8.x86_64 100/134 2023-09-13T03:01:54.768 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mon-2:18.0.0-6088.g2110e007.el8.x86_64 100/134 2023-09-13T03:01:55.272 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 114/134 2023-09-13T03:01:55.558 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-test-2:18.0.0-6088.g2110e007.el8.x86_64 115/134 2023-09-13T03:01:55.694 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-09-13T03:01:55.885 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-09-13T03:01:56.748 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-09-13T03:01:56.773 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-13T03:01:57.023 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-13T03:01:57.041 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-13T03:01:57.063 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-09-13T03:01:57.213 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-fuse-2:18.0.0-6088.g2110e007.el8.x86_64 119/134 2023-09-13T03:01:57.274 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephfs-top-2:18.0.0-6088.g2110e007.el8.noarch 124/134 2023-09-13T03:01:57.373 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-09-13T03:01:57.514 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-fuse-2:18.0.0-6088.g2110e007.el8.x86_64 125/134 2023-09-13T03:01:57.531 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephfs-devel-2:18.0.0-6088.g2110e007.el8.x86_ 121/134 2023-09-13T03:01:57.670 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-09-13T03:01:57.728 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: cephadm-2:18.0.0-6088.g2110e007.el8.noarch 101/134 2023-09-13T03:01:57.878 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-nbd-2:18.0.0-6088.g2110e007.el8.x86_64 126/134 2023-09-13T03:01:57.914 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephadm-2:18.0.0-6088.g2110e007.el8.noarch 101/134 2023-09-13T03:01:58.041 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6088.g2110e007.e 102/134 2023-09-13T03:01:58.187 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-09-13T03:01:58.250 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6088.g2110e007. 103/134 2023-09-13T03:01:58.372 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-09-13T03:01:58.433 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6088.g2110e007.el8.noa 104/134 2023-09-13T03:01:58.468 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-09-13T03:01:58.672 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-09-13T03:01:58.685 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-09-13T03:01:58.713 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6088.g2110e007.el8.noa 104/134 2023-09-13T03:01:58.830 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-09-13T03:01:58.838 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-09-13T03:01:58.910 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6088.g2110e007.el8.n 105/134 2023-09-13T03:01:58.991 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-09-13T03:01:58.991 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-13T03:01:59.158 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-13T03:01:59.159 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-13T03:01:59.411 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-13T03:01:59.604 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-13T03:01:59.916 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-13T03:02:00.094 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6088.g2110e007.el8.n 105/134 2023-09-13T03:02:00.134 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-09-13T03:02:00.135 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-2:18.0.0-6088.g2110e007.el8.x86_64 1/134 2023-09-13T03:02:00.135 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-base-2:18.0.0-6088.g2110e007.el8.x86_64 2/134 2023-09-13T03:02:00.135 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-common-2:18.0.0-6088.g2110e007.el8.x86_64 3/134 2023-09-13T03:02:00.136 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-fuse-2:18.0.0-6088.g2110e007.el8.x86_64 4/134 2023-09-13T03:02:00.136 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6088.g2110e 5/134 2023-09-13T03:02:00.136 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mds-2:18.0.0-6088.g2110e007.el8.x86_64 6/134 2023-09-13T03:02:00.136 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-2:18.0.0-6088.g2110e007.el8.x86_64 7/134 2023-09-13T03:02:00.136 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mon-2:18.0.0-6088.g2110e007.el8.x86_64 8/134 2023-09-13T03:02:00.137 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-osd-2:18.0.0-6088.g2110e007.el8.x86_64 9/134 2023-09-13T03:02:00.137 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-radosgw-2:18.0.0-6088.g2110e007.el8.x86_64 10/134 2023-09-13T03:02:00.137 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-selinux-2:18.0.0-6088.g2110e007.el8.x86_64 11/134 2023-09-13T03:02:00.137 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-test-2:18.0.0-6088.g2110e007.el8.x86_64 12/134 2023-09-13T03:02:00.137 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephfs-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 13/134 2023-09-13T03:02:00.138 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephfs-devel-2:18.0.0-6088.g2110e007.el8.x86_ 14/134 2023-09-13T03:02:00.138 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephfs2-2:18.0.0-6088.g2110e007.el8.x86_64 15/134 2023-09-13T03:02:00.138 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephsqlite-2:18.0.0-6088.g2110e007.el8.x86_64 16/134 2023-09-13T03:02:00.138 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados-devel-2:18.0.0-6088.g2110e007.el8.x86_6 17/134 2023-09-13T03:02:00.138 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libradosstriper1-2:18.0.0-6088.g2110e007.el8.x86 18/134 2023-09-13T03:02:00.139 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librgw2-2:18.0.0-6088.g2110e007.el8.x86_64 19/134 2023-09-13T03:02:00.139 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6088.g2110e007.el 20/134 2023-09-13T03:02:00.139 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ceph-common-2:18.0.0-6088.g2110e007.el8. 21/134 2023-09-13T03:02:00.139 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cephfs-2:18.0.0-6088.g2110e007.el8.x86_6 22/134 2023-09-13T03:02:00.139 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rados-2:18.0.0-6088.g2110e007.el8.x86_64 23/134 2023-09-13T03:02:00.139 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rbd-2:18.0.0-6088.g2110e007.el8.x86_64 24/134 2023-09-13T03:02:00.140 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rgw-2:18.0.0-6088.g2110e007.el8.x86_64 25/134 2023-09-13T03:02:00.140 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-fuse-2:18.0.0-6088.g2110e007.el8.x86_64 26/134 2023-09-13T03:02:00.140 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 27/134 2023-09-13T03:02:00.149 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-nbd-2:18.0.0-6088.g2110e007.el8.x86_64 28/134 2023-09-13T03:02:00.149 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6088.g2110e007. 29/134 2023-09-13T03:02:00.150 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6088.g2110e007.el8.noa 30/134 2023-09-13T03:02:00.150 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6088.g2110e007.el8.n 31/134 2023-09-13T03:02:00.150 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6088.g211 32/134 2023-09-13T03:02:00.150 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6088.g2110e007.el 33/134 2023-09-13T03:02:00.150 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6088.g2110e007.el8.noarch 34/134 2023-09-13T03:02:00.151 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6088.g2110e007.e 35/134 2023-09-13T03:02:00.151 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-volume-2:18.0.0-6088.g2110e007.el8.noarch 36/134 2023-09-13T03:02:00.151 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephadm-2:18.0.0-6088.g2110e007.el8.noarch 37/134 2023-09-13T03:02:00.151 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephfs-top-2:18.0.0-6088.g2110e007.el8.noarch 38/134 2023-09-13T03:02:00.151 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-09-13T03:02:00.152 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-09-13T03:02:00.152 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-09-13T03:02:00.152 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-09-13T03:02:00.152 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-09-13T03:02:00.152 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-09-13T03:02:00.153 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-09-13T03:02:00.153 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-09-13T03:02:00.153 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-09-13T03:02:00.153 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-09-13T03:02:00.153 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-09-13T03:02:00.154 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-09-13T03:02:00.154 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-09-13T03:02:00.154 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-09-13T03:02:00.154 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-09-13T03:02:00.154 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-09-13T03:02:00.155 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-09-13T03:02:00.155 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-09-13T03:02:00.155 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-09-13T03:02:00.155 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-09-13T03:02:00.155 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-09-13T03:02:00.155 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-09-13T03:02:00.156 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-09-13T03:02:00.156 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-09-13T03:02:00.156 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-09-13T03:02:00.156 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-09-13T03:02:00.156 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-09-13T03:02:00.157 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-09-13T03:02:00.157 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-09-13T03:02:00.157 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-09-13T03:02:00.157 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-09-13T03:02:00.157 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-09-13T03:02:00.158 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-09-13T03:02:00.158 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-09-13T03:02:00.158 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-09-13T03:02:00.158 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-09-13T03:02:00.158 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-09-13T03:02:00.158 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-09-13T03:02:00.159 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-09-13T03:02:00.159 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-09-13T03:02:00.159 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-09-13T03:02:00.159 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-09-13T03:02:00.159 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-09-13T03:02:00.160 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-09-13T03:02:00.160 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-09-13T03:02:00.160 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-09-13T03:02:00.160 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-09-13T03:02:00.160 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-09-13T03:02:00.161 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-09-13T03:02:00.161 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-09-13T03:02:00.161 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-09-13T03:02:00.161 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-09-13T03:02:00.161 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-09-13T03:02:00.162 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-09-13T03:02:00.162 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-09-13T03:02:00.162 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-09-13T03:02:00.162 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-09-13T03:02:00.162 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-09-13T03:02:00.163 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-09-13T03:02:00.163 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-09-13T03:02:00.163 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-09-13T03:02:00.163 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-09-13T03:02:00.163 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-09-13T03:02:00.164 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-09-13T03:02:00.164 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-09-13T03:02:00.164 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-09-13T03:02:00.164 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-09-13T03:02:00.164 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-09-13T03:02:00.164 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-09-13T03:02:00.165 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-09-13T03:02:00.165 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-09-13T03:02:00.165 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-09-13T03:02:00.165 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-09-13T03:02:00.165 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-09-13T03:02:00.166 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-09-13T03:02:00.166 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-09-13T03:02:00.166 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-09-13T03:02:00.166 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-09-13T03:02:00.166 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-09-13T03:02:00.167 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-09-13T03:02:00.167 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-09-13T03:02:00.167 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-09-13T03:02:00.167 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-09-13T03:02:00.167 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-09-13T03:02:00.168 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-09-13T03:02:00.168 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-09-13T03:02:00.168 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-09-13T03:02:00.168 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-09-13T03:02:00.168 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-09-13T03:02:00.169 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-09-13T03:02:00.169 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados2-2:18.0.0-6088.g2110e007.el8.x86_64 129/134 2023-09-13T03:02:00.169 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-09-13T03:02:00.169 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librbd1-2:18.0.0-6088.g2110e007.el8.x86_64 131/134 2023-09-13T03:02:00.169 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-13T03:02:00.170 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-09-13T03:02:00.185 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-13T03:02:00.203 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-13T03:02:00.291 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6088.g211 106/134 2023-09-13T03:02:00.402 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephfs-top-2:18.0.0-6088.g2110e007.el8.noarch 124/134 2023-09-13T03:02:00.426 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6088.g211 106/134 2023-09-13T03:02:00.676 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-fuse-2:18.0.0-6088.g2110e007.el8.x86_64 125/134 2023-09-13T03:02:00.809 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6088.g2110e007.el 107/134 2023-09-13T03:02:00.967 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-nbd-2:18.0.0-6088.g2110e007.el8.x86_64 126/134 2023-09-13T03:02:01.030 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-2:18.0.0-6088.g2110e007.el8.x86_64 108/134 2023-09-13T03:02:01.097 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6088.g2110e007.el8.x86_64 108/134 2023-09-13T03:02:01.229 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-09-13T03:02:01.274 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-rook-2:18.0.0-6088.g2110e007.el8.noarch 109/134 2023-09-13T03:02:01.303 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6088.g2110e007.el8.noarch 109/134 2023-09-13T03:02:01.360 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-09-13T03:02:01.369 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-09-13T03:02:01.557 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-09-13T03:02:01.768 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-09-13T03:02:01.850 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-09-13T03:02:02.027 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-09-13T03:02:02.040 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-09-13T03:02:02.040 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-13T03:02:02.246 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-13T03:02:02.247 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-13T03:02:02.490 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-13T03:02:02.626 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-13T03:02:02.626 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:01:52 ago on Wed 13 Sep 2023 03:00:10 AM UTC. 2023-09-13T03:02:02.701 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-13T03:02:02.786 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-2:18.0.0-6088.g2110e007.el8.x86_64 110/134 2023-09-13T03:02:02.973 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-radosgw-2:18.0.0-6088.g2110e007.el8.x86_64 111/134 2023-09-13T03:02:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6088.g2110e007.el8.x86_64 111/134 2023-09-13T03:02:03.174 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6088.g2110e 112/134 2023-09-13T03:02:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6088.g2110e 112/134 2023-09-13T03:02:03.260 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-09-13T03:02:03.260 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-2:18.0.0-6088.g2110e007.el8.x86_64 1/134 2023-09-13T03:02:03.260 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-base-2:18.0.0-6088.g2110e007.el8.x86_64 2/134 2023-09-13T03:02:03.261 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-common-2:18.0.0-6088.g2110e007.el8.x86_64 3/134 2023-09-13T03:02:03.261 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-fuse-2:18.0.0-6088.g2110e007.el8.x86_64 4/134 2023-09-13T03:02:03.261 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6088.g2110e 5/134 2023-09-13T03:02:03.261 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mds-2:18.0.0-6088.g2110e007.el8.x86_64 6/134 2023-09-13T03:02:03.261 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-2:18.0.0-6088.g2110e007.el8.x86_64 7/134 2023-09-13T03:02:03.262 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mon-2:18.0.0-6088.g2110e007.el8.x86_64 8/134 2023-09-13T03:02:03.262 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-osd-2:18.0.0-6088.g2110e007.el8.x86_64 9/134 2023-09-13T03:02:03.262 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-radosgw-2:18.0.0-6088.g2110e007.el8.x86_64 10/134 2023-09-13T03:02:03.262 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-selinux-2:18.0.0-6088.g2110e007.el8.x86_64 11/134 2023-09-13T03:02:03.262 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-test-2:18.0.0-6088.g2110e007.el8.x86_64 12/134 2023-09-13T03:02:03.263 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephfs-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 13/134 2023-09-13T03:02:03.263 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephfs-devel-2:18.0.0-6088.g2110e007.el8.x86_ 14/134 2023-09-13T03:02:03.263 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephfs2-2:18.0.0-6088.g2110e007.el8.x86_64 15/134 2023-09-13T03:02:03.263 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephsqlite-2:18.0.0-6088.g2110e007.el8.x86_64 16/134 2023-09-13T03:02:03.263 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados-devel-2:18.0.0-6088.g2110e007.el8.x86_6 17/134 2023-09-13T03:02:03.263 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libradosstriper1-2:18.0.0-6088.g2110e007.el8.x86 18/134 2023-09-13T03:02:03.264 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librgw2-2:18.0.0-6088.g2110e007.el8.x86_64 19/134 2023-09-13T03:02:03.264 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6088.g2110e007.el 20/134 2023-09-13T03:02:03.264 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-ceph-common-2:18.0.0-6088.g2110e007.el8. 21/134 2023-09-13T03:02:03.264 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cephfs-2:18.0.0-6088.g2110e007.el8.x86_6 22/134 2023-09-13T03:02:03.264 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rados-2:18.0.0-6088.g2110e007.el8.x86_64 23/134 2023-09-13T03:02:03.265 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rbd-2:18.0.0-6088.g2110e007.el8.x86_64 24/134 2023-09-13T03:02:03.265 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rgw-2:18.0.0-6088.g2110e007.el8.x86_64 25/134 2023-09-13T03:02:03.274 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-fuse-2:18.0.0-6088.g2110e007.el8.x86_64 26/134 2023-09-13T03:02:03.274 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 27/134 2023-09-13T03:02:03.274 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-nbd-2:18.0.0-6088.g2110e007.el8.x86_64 28/134 2023-09-13T03:02:03.274 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6088.g2110e007. 29/134 2023-09-13T03:02:03.275 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6088.g2110e007.el8.noa 30/134 2023-09-13T03:02:03.275 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6088.g2110e007.el8.n 31/134 2023-09-13T03:02:03.275 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6088.g211 32/134 2023-09-13T03:02:03.275 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6088.g2110e007.el 33/134 2023-09-13T03:02:03.275 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6088.g2110e007.el8.noarch 34/134 2023-09-13T03:02:03.276 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6088.g2110e007.e 35/134 2023-09-13T03:02:03.276 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-volume-2:18.0.0-6088.g2110e007.el8.noarch 36/134 2023-09-13T03:02:03.276 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephadm-2:18.0.0-6088.g2110e007.el8.noarch 37/134 2023-09-13T03:02:03.276 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephfs-top-2:18.0.0-6088.g2110e007.el8.noarch 38/134 2023-09-13T03:02:03.276 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-09-13T03:02:03.277 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-09-13T03:02:03.277 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-09-13T03:02:03.277 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-09-13T03:02:03.277 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-09-13T03:02:03.277 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-09-13T03:02:03.278 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-09-13T03:02:03.278 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-09-13T03:02:03.278 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-09-13T03:02:03.278 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-09-13T03:02:03.278 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-09-13T03:02:03.279 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-09-13T03:02:03.279 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-09-13T03:02:03.279 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-09-13T03:02:03.279 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-09-13T03:02:03.279 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-09-13T03:02:03.279 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-09-13T03:02:03.280 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-09-13T03:02:03.280 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-09-13T03:02:03.280 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-09-13T03:02:03.280 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-09-13T03:02:03.280 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-09-13T03:02:03.281 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-09-13T03:02:03.281 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-09-13T03:02:03.281 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-09-13T03:02:03.281 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-09-13T03:02:03.281 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-09-13T03:02:03.282 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-09-13T03:02:03.282 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-09-13T03:02:03.282 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-09-13T03:02:03.282 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-09-13T03:02:03.282 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-09-13T03:02:03.283 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-09-13T03:02:03.283 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-09-13T03:02:03.283 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-09-13T03:02:03.283 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-09-13T03:02:03.283 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-09-13T03:02:03.284 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-09-13T03:02:03.284 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-09-13T03:02:03.284 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-09-13T03:02:03.284 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-09-13T03:02:03.284 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-09-13T03:02:03.284 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-09-13T03:02:03.285 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-09-13T03:02:03.285 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-09-13T03:02:03.285 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-09-13T03:02:03.285 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-09-13T03:02:03.285 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-09-13T03:02:03.286 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-09-13T03:02:03.286 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-09-13T03:02:03.286 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-09-13T03:02:03.286 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-09-13T03:02:03.286 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-09-13T03:02:03.287 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-09-13T03:02:03.287 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-09-13T03:02:03.287 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-09-13T03:02:03.287 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-09-13T03:02:03.287 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-09-13T03:02:03.288 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-09-13T03:02:03.288 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-09-13T03:02:03.288 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-09-13T03:02:03.288 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-09-13T03:02:03.288 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-09-13T03:02:03.289 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-09-13T03:02:03.289 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-09-13T03:02:03.289 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-09-13T03:02:03.289 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-09-13T03:02:03.289 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-09-13T03:02:03.289 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-09-13T03:02:03.290 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-09-13T03:02:03.290 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-09-13T03:02:03.290 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-09-13T03:02:03.290 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-09-13T03:02:03.290 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-09-13T03:02:03.291 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-09-13T03:02:03.291 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-09-13T03:02:03.291 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-09-13T03:02:03.291 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-09-13T03:02:03.291 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-09-13T03:02:03.292 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-09-13T03:02:03.292 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-09-13T03:02:03.292 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-09-13T03:02:03.292 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-09-13T03:02:03.292 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-09-13T03:02:03.292 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-09-13T03:02:03.293 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-09-13T03:02:03.293 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-09-13T03:02:03.293 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-09-13T03:02:03.293 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-09-13T03:02:03.293 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-09-13T03:02:03.294 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados2-2:18.0.0-6088.g2110e007.el8.x86_64 129/134 2023-09-13T03:02:03.294 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-09-13T03:02:03.294 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librbd1-2:18.0.0-6088.g2110e007.el8.x86_64 131/134 2023-09-13T03:02:03.294 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-13T03:02:03.294 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-09-13T03:02:03.424 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephfs-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 113/134 2023-09-13T03:02:03.873 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 113/134 2023-09-13T03:02:04.024 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 114/134 2023-09-13T03:02:05.463 INFO:teuthology.orchestra.run.smithi142.stdout:Installed products updated. 2023-09-13T03:02:05.676 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-09-13T03:02:05.677 INFO:teuthology.orchestra.run.smithi142.stdout:Upgraded: 2023-09-13T03:02:05.677 INFO:teuthology.orchestra.run.smithi142.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-09-13T03:02:05.677 INFO:teuthology.orchestra.run.smithi142.stdout: librados2-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:05.677 INFO:teuthology.orchestra.run.smithi142.stdout: librbd1-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:05.677 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2023-09-13T03:02:05.678 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-09-13T03:02:05.678 INFO:teuthology.orchestra.run.smithi142.stdout: automake-1.16.1-7.el8.noarch 2023-09-13T03:02:05.678 INFO:teuthology.orchestra.run.smithi142.stdout: bison-3.0.4-10.el8.x86_64 2023-09-13T03:02:05.678 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:05.679 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-base-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:05.679 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-common-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:05.679 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-fuse-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:05.679 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-grafana-dashboards-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-13T03:02:05.679 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-immutable-object-cache-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:05.680 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mds-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:05.680 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:05.680 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-cephadm-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-13T03:02:05.680 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-dashboard-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-13T03:02:05.680 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-13T03:02:05.681 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-modules-core-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-13T03:02:05.681 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-rook-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-13T03:02:05.681 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mon-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:05.681 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-osd-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:05.681 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-prometheus-alerts-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-13T03:02:05.681 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-radosgw-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:05.682 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-selinux-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:05.682 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-test-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:05.682 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-volume-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-13T03:02:05.682 INFO:teuthology.orchestra.run.smithi142.stdout: cephadm-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-13T03:02:05.682 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:05.683 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-top-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-13T03:02:05.683 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-09-13T03:02:05.683 INFO:teuthology.orchestra.run.smithi142.stdout: flex-2.6.1-9.el8.x86_64 2023-09-13T03:02:05.683 INFO:teuthology.orchestra.run.smithi142.stdout: fuse-2.9.7-15.el8.x86_64 2023-09-13T03:02:05.683 INFO:teuthology.orchestra.run.smithi142.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-09-13T03:02:05.684 INFO:teuthology.orchestra.run.smithi142.stdout: jq-1.6-3.el8.x86_64 2023-09-13T03:02:05.684 INFO:teuthology.orchestra.run.smithi142.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-09-13T03:02:05.684 INFO:teuthology.orchestra.run.smithi142.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-09-13T03:02:05.684 INFO:teuthology.orchestra.run.smithi142.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-09-13T03:02:05.684 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs-devel-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:05.685 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs2-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:05.685 INFO:teuthology.orchestra.run.smithi142.stdout: libcephsqlite-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:05.685 INFO:teuthology.orchestra.run.smithi142.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-09-13T03:02:05.685 INFO:teuthology.orchestra.run.smithi142.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-09-13T03:02:05.685 INFO:teuthology.orchestra.run.smithi142.stdout: liboath-2.6.2-3.el8.x86_64 2023-09-13T03:02:05.686 INFO:teuthology.orchestra.run.smithi142.stdout: libpq-13.5-1.el8.x86_64 2023-09-13T03:02:05.686 INFO:teuthology.orchestra.run.smithi142.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-09-13T03:02:05.686 INFO:teuthology.orchestra.run.smithi142.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-09-13T03:02:05.686 INFO:teuthology.orchestra.run.smithi142.stdout: librados-devel-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:05.687 INFO:teuthology.orchestra.run.smithi142.stdout: libradosstriper1-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:05.687 INFO:teuthology.orchestra.run.smithi142.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-09-13T03:02:05.687 INFO:teuthology.orchestra.run.smithi142.stdout: librgw2-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:05.687 INFO:teuthology.orchestra.run.smithi142.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-09-13T03:02:05.687 INFO:teuthology.orchestra.run.smithi142.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-09-13T03:02:05.688 INFO:teuthology.orchestra.run.smithi142.stdout: libtool-2.4.6-25.el8.x86_64 2023-09-13T03:02:05.688 INFO:teuthology.orchestra.run.smithi142.stdout: libunwind-1.3.1-3.el8.x86_64 2023-09-13T03:02:05.688 INFO:teuthology.orchestra.run.smithi142.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-09-13T03:02:05.688 INFO:teuthology.orchestra.run.smithi142.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-09-13T03:02:05.688 INFO:teuthology.orchestra.run.smithi142.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-09-13T03:02:05.689 INFO:teuthology.orchestra.run.smithi142.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-09-13T03:02:05.689 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-0.3.15-3.el8.x86_64 2023-09-13T03:02:05.689 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-09-13T03:02:05.689 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-09-13T03:02:05.689 INFO:teuthology.orchestra.run.smithi142.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-09-13T03:02:05.690 INFO:teuthology.orchestra.run.smithi142.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-09-13T03:02:05.690 INFO:teuthology.orchestra.run.smithi142.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-09-13T03:02:05.690 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-13T03:02:05.690 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-13T03:02:05.690 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-13T03:02:05.690 INFO:teuthology.orchestra.run.smithi142.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-09-13T03:02:05.691 INFO:teuthology.orchestra.run.smithi142.stdout: python3-babel-2.5.1-7.el8.noarch 2023-09-13T03:02:05.691 INFO:teuthology.orchestra.run.smithi142.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-09-13T03:02:05.691 INFO:teuthology.orchestra.run.smithi142.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-09-13T03:02:05.691 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-09-13T03:02:05.691 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-argparse-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:05.692 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-common-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:05.692 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cephfs-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:05.692 INFO:teuthology.orchestra.run.smithi142.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-09-13T03:02:05.692 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-09-13T03:02:05.692 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-09-13T03:02:05.692 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-09-13T03:02:05.693 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-09-13T03:02:05.693 INFO:teuthology.orchestra.run.smithi142.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-09-13T03:02:05.693 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-6.2-6.el8.noarch 2023-09-13T03:02:05.693 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-09-13T03:02:05.693 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-09-13T03:02:05.694 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-09-13T03:02:05.694 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-09-13T03:02:05.694 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-09-13T03:02:05.694 INFO:teuthology.orchestra.run.smithi142.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-09-13T03:02:05.694 INFO:teuthology.orchestra.run.smithi142.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-09-13T03:02:05.695 INFO:teuthology.orchestra.run.smithi142.stdout: python3-mako-1.0.6-13.el8.noarch 2023-09-13T03:02:05.695 INFO:teuthology.orchestra.run.smithi142.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-09-13T03:02:05.695 INFO:teuthology.orchestra.run.smithi142.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-09-13T03:02:05.695 INFO:teuthology.orchestra.run.smithi142.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-09-13T03:02:05.695 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-09-13T03:02:05.696 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-09-13T03:02:05.696 INFO:teuthology.orchestra.run.smithi142.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-09-13T03:02:05.696 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-09-13T03:02:05.696 INFO:teuthology.orchestra.run.smithi142.stdout: python3-portend-2.6-1.el8.noarch 2023-09-13T03:02:05.696 INFO:teuthology.orchestra.run.smithi142.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-09-13T03:02:05.696 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-09-13T03:02:05.697 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-09-13T03:02:05.697 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-09-13T03:02:05.697 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pycparser-2.14-14.el8.noarch 2023-09-13T03:02:05.697 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytz-2017.2-9.el8.noarch 2023-09-13T03:02:05.698 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rados-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:05.698 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rbd-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:05.698 INFO:teuthology.orchestra.run.smithi142.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-09-13T03:02:05.698 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-09-13T03:02:05.698 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rgw-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:05.699 INFO:teuthology.orchestra.run.smithi142.stdout: python3-routes-2.4.1-12.el8.noarch 2023-09-13T03:02:05.699 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rsa-4.9-2.el8.noarch 2023-09-13T03:02:05.699 INFO:teuthology.orchestra.run.smithi142.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-09-13T03:02:05.699 INFO:teuthology.orchestra.run.smithi142.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-09-13T03:02:05.699 INFO:teuthology.orchestra.run.smithi142.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-09-13T03:02:05.700 INFO:teuthology.orchestra.run.smithi142.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-09-13T03:02:05.700 INFO:teuthology.orchestra.run.smithi142.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-09-13T03:02:05.700 INFO:teuthology.orchestra.run.smithi142.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-09-13T03:02:05.700 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-09-13T03:02:05.700 INFO:teuthology.orchestra.run.smithi142.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-09-13T03:02:05.701 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-09-13T03:02:05.701 INFO:teuthology.orchestra.run.smithi142.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-09-13T03:02:05.701 INFO:teuthology.orchestra.run.smithi142.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-09-13T03:02:05.701 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-fuse-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:05.701 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:05.702 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-nbd-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:05.702 INFO:teuthology.orchestra.run.smithi142.stdout: socat-1.7.4.1-1.el8.x86_64 2023-09-13T03:02:05.702 INFO:teuthology.orchestra.run.smithi142.stdout: thrift-0.13.0-2.el8.x86_64 2023-09-13T03:02:05.702 INFO:teuthology.orchestra.run.smithi142.stdout: uuid-1.6.2-43.el8.x86_64 2023-09-13T03:02:05.702 INFO:teuthology.orchestra.run.smithi142.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-09-13T03:02:05.703 INFO:teuthology.orchestra.run.smithi142.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-09-13T03:02:05.703 INFO:teuthology.orchestra.run.smithi142.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-09-13T03:02:05.704 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-09-13T03:02:05.704 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2023-09-13T03:02:05.745 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-13T03:02:05.745 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:01:54 ago on Wed 13 Sep 2023 03:00:11 AM UTC. 2023-09-13T03:02:05.888 DEBUG:teuthology.parallel:result is None 2023-09-13T03:02:09.330 INFO:teuthology.orchestra.run.smithi170.stdout:Installed products updated. 2023-09-13T03:02:09.472 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 114/134 2023-09-13T03:02:09.548 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-09-13T03:02:09.548 INFO:teuthology.orchestra.run.smithi170.stdout:Upgraded: 2023-09-13T03:02:09.549 INFO:teuthology.orchestra.run.smithi170.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-09-13T03:02:09.549 INFO:teuthology.orchestra.run.smithi170.stdout: librados2-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:09.549 INFO:teuthology.orchestra.run.smithi170.stdout: librbd1-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:09.549 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2023-09-13T03:02:09.549 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-09-13T03:02:09.550 INFO:teuthology.orchestra.run.smithi170.stdout: automake-1.16.1-7.el8.noarch 2023-09-13T03:02:09.550 INFO:teuthology.orchestra.run.smithi170.stdout: bison-3.0.4-10.el8.x86_64 2023-09-13T03:02:09.550 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:09.550 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-base-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:09.550 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-common-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:09.551 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-fuse-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:09.551 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-grafana-dashboards-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-13T03:02:09.551 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-immutable-object-cache-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:09.551 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mds-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:09.551 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:09.552 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-cephadm-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-13T03:02:09.552 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-dashboard-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-13T03:02:09.552 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-13T03:02:09.552 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-modules-core-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-13T03:02:09.552 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-rook-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-13T03:02:09.553 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mon-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:09.553 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-osd-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:09.553 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-prometheus-alerts-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-13T03:02:09.553 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-radosgw-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:09.553 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-selinux-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:09.554 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-test-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:09.554 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-volume-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-13T03:02:09.554 INFO:teuthology.orchestra.run.smithi170.stdout: cephadm-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-13T03:02:09.554 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:09.554 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-top-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-13T03:02:09.555 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-09-13T03:02:09.555 INFO:teuthology.orchestra.run.smithi170.stdout: flex-2.6.1-9.el8.x86_64 2023-09-13T03:02:09.555 INFO:teuthology.orchestra.run.smithi170.stdout: fuse-2.9.7-15.el8.x86_64 2023-09-13T03:02:09.555 INFO:teuthology.orchestra.run.smithi170.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-09-13T03:02:09.555 INFO:teuthology.orchestra.run.smithi170.stdout: jq-1.6-3.el8.x86_64 2023-09-13T03:02:09.555 INFO:teuthology.orchestra.run.smithi170.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-09-13T03:02:09.556 INFO:teuthology.orchestra.run.smithi170.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-09-13T03:02:09.556 INFO:teuthology.orchestra.run.smithi170.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-09-13T03:02:09.556 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs-devel-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:09.556 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs2-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:09.556 INFO:teuthology.orchestra.run.smithi170.stdout: libcephsqlite-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:09.557 INFO:teuthology.orchestra.run.smithi170.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-09-13T03:02:09.557 INFO:teuthology.orchestra.run.smithi170.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-09-13T03:02:09.557 INFO:teuthology.orchestra.run.smithi170.stdout: liboath-2.6.2-3.el8.x86_64 2023-09-13T03:02:09.557 INFO:teuthology.orchestra.run.smithi170.stdout: libpq-13.5-1.el8.x86_64 2023-09-13T03:02:09.557 INFO:teuthology.orchestra.run.smithi170.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-09-13T03:02:09.558 INFO:teuthology.orchestra.run.smithi170.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-09-13T03:02:09.558 INFO:teuthology.orchestra.run.smithi170.stdout: librados-devel-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:09.558 INFO:teuthology.orchestra.run.smithi170.stdout: libradosstriper1-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:09.558 INFO:teuthology.orchestra.run.smithi170.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-09-13T03:02:09.558 INFO:teuthology.orchestra.run.smithi170.stdout: librgw2-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:09.558 INFO:teuthology.orchestra.run.smithi170.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-09-13T03:02:09.559 INFO:teuthology.orchestra.run.smithi170.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-09-13T03:02:09.559 INFO:teuthology.orchestra.run.smithi170.stdout: libtool-2.4.6-25.el8.x86_64 2023-09-13T03:02:09.559 INFO:teuthology.orchestra.run.smithi170.stdout: libunwind-1.3.1-3.el8.x86_64 2023-09-13T03:02:09.559 INFO:teuthology.orchestra.run.smithi170.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-09-13T03:02:09.560 INFO:teuthology.orchestra.run.smithi170.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-09-13T03:02:09.560 INFO:teuthology.orchestra.run.smithi170.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-09-13T03:02:09.560 INFO:teuthology.orchestra.run.smithi170.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-09-13T03:02:09.560 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-0.3.15-3.el8.x86_64 2023-09-13T03:02:09.560 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-09-13T03:02:09.560 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-09-13T03:02:09.561 INFO:teuthology.orchestra.run.smithi170.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-09-13T03:02:09.561 INFO:teuthology.orchestra.run.smithi170.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-09-13T03:02:09.561 INFO:teuthology.orchestra.run.smithi170.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-09-13T03:02:09.561 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-13T03:02:09.561 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-13T03:02:09.562 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-13T03:02:09.562 INFO:teuthology.orchestra.run.smithi170.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-09-13T03:02:09.562 INFO:teuthology.orchestra.run.smithi170.stdout: python3-babel-2.5.1-7.el8.noarch 2023-09-13T03:02:09.562 INFO:teuthology.orchestra.run.smithi170.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-09-13T03:02:09.562 INFO:teuthology.orchestra.run.smithi170.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-09-13T03:02:09.563 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-09-13T03:02:09.563 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-argparse-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:09.563 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-common-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:09.563 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cephfs-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:09.563 INFO:teuthology.orchestra.run.smithi170.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-09-13T03:02:09.563 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-09-13T03:02:09.564 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-09-13T03:02:09.564 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-09-13T03:02:09.564 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-09-13T03:02:09.564 INFO:teuthology.orchestra.run.smithi170.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-09-13T03:02:09.564 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-6.2-6.el8.noarch 2023-09-13T03:02:09.565 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-09-13T03:02:09.565 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-09-13T03:02:09.565 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-09-13T03:02:09.565 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-09-13T03:02:09.565 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-09-13T03:02:09.566 INFO:teuthology.orchestra.run.smithi170.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-09-13T03:02:09.566 INFO:teuthology.orchestra.run.smithi170.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-09-13T03:02:09.566 INFO:teuthology.orchestra.run.smithi170.stdout: python3-mako-1.0.6-13.el8.noarch 2023-09-13T03:02:09.566 INFO:teuthology.orchestra.run.smithi170.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-09-13T03:02:09.566 INFO:teuthology.orchestra.run.smithi170.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-09-13T03:02:09.567 INFO:teuthology.orchestra.run.smithi170.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-09-13T03:02:09.567 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-09-13T03:02:09.567 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-09-13T03:02:09.567 INFO:teuthology.orchestra.run.smithi170.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-09-13T03:02:09.567 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-09-13T03:02:09.567 INFO:teuthology.orchestra.run.smithi170.stdout: python3-portend-2.6-1.el8.noarch 2023-09-13T03:02:09.568 INFO:teuthology.orchestra.run.smithi170.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-09-13T03:02:09.568 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-09-13T03:02:09.568 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-09-13T03:02:09.568 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-09-13T03:02:09.568 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pycparser-2.14-14.el8.noarch 2023-09-13T03:02:09.569 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytz-2017.2-9.el8.noarch 2023-09-13T03:02:09.569 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rados-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:09.569 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rbd-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:09.569 INFO:teuthology.orchestra.run.smithi170.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-09-13T03:02:09.569 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-09-13T03:02:09.570 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rgw-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:09.570 INFO:teuthology.orchestra.run.smithi170.stdout: python3-routes-2.4.1-12.el8.noarch 2023-09-13T03:02:09.570 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rsa-4.9-2.el8.noarch 2023-09-13T03:02:09.570 INFO:teuthology.orchestra.run.smithi170.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-09-13T03:02:09.570 INFO:teuthology.orchestra.run.smithi170.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-09-13T03:02:09.571 INFO:teuthology.orchestra.run.smithi170.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-09-13T03:02:09.571 INFO:teuthology.orchestra.run.smithi170.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-09-13T03:02:09.571 INFO:teuthology.orchestra.run.smithi170.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-09-13T03:02:09.571 INFO:teuthology.orchestra.run.smithi170.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-09-13T03:02:09.571 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-09-13T03:02:09.572 INFO:teuthology.orchestra.run.smithi170.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-09-13T03:02:09.572 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-09-13T03:02:09.572 INFO:teuthology.orchestra.run.smithi170.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-09-13T03:02:09.572 INFO:teuthology.orchestra.run.smithi170.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-09-13T03:02:09.572 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-fuse-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:09.573 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:09.573 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-nbd-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:09.573 INFO:teuthology.orchestra.run.smithi170.stdout: socat-1.7.4.1-1.el8.x86_64 2023-09-13T03:02:09.573 INFO:teuthology.orchestra.run.smithi170.stdout: thrift-0.13.0-2.el8.x86_64 2023-09-13T03:02:09.573 INFO:teuthology.orchestra.run.smithi170.stdout: uuid-1.6.2-43.el8.x86_64 2023-09-13T03:02:09.574 INFO:teuthology.orchestra.run.smithi170.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-09-13T03:02:09.574 INFO:teuthology.orchestra.run.smithi170.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-09-13T03:02:09.574 INFO:teuthology.orchestra.run.smithi170.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-09-13T03:02:09.575 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-09-13T03:02:09.575 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2023-09-13T03:02:09.732 DEBUG:teuthology.parallel:result is None 2023-09-13T03:02:09.783 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-test-2:18.0.0-6088.g2110e007.el8.x86_64 115/134 2023-09-13T03:02:09.922 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-09-13T03:02:10.192 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-09-13T03:02:11.492 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-09-13T03:02:11.822 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-09-13T03:02:11.996 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-fuse-2:18.0.0-6088.g2110e007.el8.x86_64 119/134 2023-09-13T03:02:12.186 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-09-13T03:02:12.361 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephfs-devel-2:18.0.0-6088.g2110e007.el8.x86_ 121/134 2023-09-13T03:02:12.507 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-09-13T03:02:16.028 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-13T03:02:16.281 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-13T03:02:16.319 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-13T03:02:16.540 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephfs-top-2:18.0.0-6088.g2110e007.el8.noarch 124/134 2023-09-13T03:02:16.741 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-fuse-2:18.0.0-6088.g2110e007.el8.x86_64 125/134 2023-09-13T03:02:17.072 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-nbd-2:18.0.0-6088.g2110e007.el8.x86_64 126/134 2023-09-13T03:02:17.383 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-09-13T03:02:17.572 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-09-13T03:02:17.595 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-09-13T03:02:17.816 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-09-13T03:02:18.005 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-09-13T03:02:18.119 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-09-13T03:02:18.296 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-09-13T03:02:18.325 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-09-13T03:02:18.326 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-13T03:02:18.554 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-13T03:02:18.554 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-13T03:02:18.796 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-13T03:02:19.005 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-13T03:02:19.270 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-09-13T03:02:19.270 INFO:teuthology.orchestra.run.smithi064.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-09-13T03:02:19.270 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:02:19.919 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-2:18.0.0-6088.g2110e007.el8.x86_64 1/134 2023-09-13T03:02:19.919 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-base-2:18.0.0-6088.g2110e007.el8.x86_64 2/134 2023-09-13T03:02:19.919 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-common-2:18.0.0-6088.g2110e007.el8.x86_64 3/134 2023-09-13T03:02:19.920 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-fuse-2:18.0.0-6088.g2110e007.el8.x86_64 4/134 2023-09-13T03:02:19.920 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6088.g2110e 5/134 2023-09-13T03:02:19.920 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mds-2:18.0.0-6088.g2110e007.el8.x86_64 6/134 2023-09-13T03:02:19.920 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-2:18.0.0-6088.g2110e007.el8.x86_64 7/134 2023-09-13T03:02:19.920 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mon-2:18.0.0-6088.g2110e007.el8.x86_64 8/134 2023-09-13T03:02:19.921 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-osd-2:18.0.0-6088.g2110e007.el8.x86_64 9/134 2023-09-13T03:02:19.921 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-radosgw-2:18.0.0-6088.g2110e007.el8.x86_64 10/134 2023-09-13T03:02:19.921 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-selinux-2:18.0.0-6088.g2110e007.el8.x86_64 11/134 2023-09-13T03:02:19.921 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-test-2:18.0.0-6088.g2110e007.el8.x86_64 12/134 2023-09-13T03:02:19.921 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephfs-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 13/134 2023-09-13T03:02:19.922 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephfs-devel-2:18.0.0-6088.g2110e007.el8.x86_ 14/134 2023-09-13T03:02:19.922 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephfs2-2:18.0.0-6088.g2110e007.el8.x86_64 15/134 2023-09-13T03:02:19.922 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephsqlite-2:18.0.0-6088.g2110e007.el8.x86_64 16/134 2023-09-13T03:02:19.922 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados-devel-2:18.0.0-6088.g2110e007.el8.x86_6 17/134 2023-09-13T03:02:19.922 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libradosstriper1-2:18.0.0-6088.g2110e007.el8.x86 18/134 2023-09-13T03:02:19.923 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librgw2-2:18.0.0-6088.g2110e007.el8.x86_64 19/134 2023-09-13T03:02:19.923 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6088.g2110e007.el 20/134 2023-09-13T03:02:19.923 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ceph-common-2:18.0.0-6088.g2110e007.el8. 21/134 2023-09-13T03:02:19.923 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cephfs-2:18.0.0-6088.g2110e007.el8.x86_6 22/134 2023-09-13T03:02:19.923 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rados-2:18.0.0-6088.g2110e007.el8.x86_64 23/134 2023-09-13T03:02:19.924 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rbd-2:18.0.0-6088.g2110e007.el8.x86_64 24/134 2023-09-13T03:02:19.924 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rgw-2:18.0.0-6088.g2110e007.el8.x86_64 25/134 2023-09-13T03:02:19.924 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-fuse-2:18.0.0-6088.g2110e007.el8.x86_64 26/134 2023-09-13T03:02:19.924 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 27/134 2023-09-13T03:02:19.924 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-nbd-2:18.0.0-6088.g2110e007.el8.x86_64 28/134 2023-09-13T03:02:19.925 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6088.g2110e007. 29/134 2023-09-13T03:02:19.925 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6088.g2110e007.el8.noa 30/134 2023-09-13T03:02:19.925 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6088.g2110e007.el8.n 31/134 2023-09-13T03:02:19.925 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6088.g211 32/134 2023-09-13T03:02:19.925 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6088.g2110e007.el 33/134 2023-09-13T03:02:19.926 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6088.g2110e007.el8.noarch 34/134 2023-09-13T03:02:19.926 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6088.g2110e007.e 35/134 2023-09-13T03:02:19.926 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-volume-2:18.0.0-6088.g2110e007.el8.noarch 36/134 2023-09-13T03:02:19.926 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephadm-2:18.0.0-6088.g2110e007.el8.noarch 37/134 2023-09-13T03:02:19.926 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephfs-top-2:18.0.0-6088.g2110e007.el8.noarch 38/134 2023-09-13T03:02:19.927 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-09-13T03:02:19.927 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-09-13T03:02:19.927 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-09-13T03:02:19.927 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-09-13T03:02:19.927 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-09-13T03:02:19.928 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-09-13T03:02:19.928 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-09-13T03:02:19.928 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-09-13T03:02:19.928 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-09-13T03:02:19.928 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-09-13T03:02:19.928 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-09-13T03:02:19.929 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-09-13T03:02:19.929 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-09-13T03:02:19.929 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-09-13T03:02:19.929 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-09-13T03:02:19.929 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-09-13T03:02:19.930 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-09-13T03:02:19.930 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-09-13T03:02:19.930 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-09-13T03:02:19.930 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-09-13T03:02:19.931 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-09-13T03:02:19.931 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-09-13T03:02:19.931 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-09-13T03:02:19.931 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-09-13T03:02:19.931 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-09-13T03:02:19.931 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-09-13T03:02:19.932 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-09-13T03:02:19.932 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-09-13T03:02:19.932 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-09-13T03:02:19.932 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-09-13T03:02:19.932 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-09-13T03:02:19.933 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-09-13T03:02:19.933 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-09-13T03:02:19.933 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-09-13T03:02:19.933 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-09-13T03:02:19.933 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-09-13T03:02:19.934 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-09-13T03:02:19.934 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-09-13T03:02:19.934 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-09-13T03:02:19.934 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-09-13T03:02:19.934 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-09-13T03:02:19.935 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-09-13T03:02:19.935 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-09-13T03:02:19.935 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-09-13T03:02:19.935 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-09-13T03:02:19.935 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-09-13T03:02:19.936 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-09-13T03:02:19.936 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-09-13T03:02:19.936 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-09-13T03:02:19.936 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-09-13T03:02:19.936 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-09-13T03:02:19.936 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-09-13T03:02:19.937 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-09-13T03:02:19.937 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-09-13T03:02:19.937 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-09-13T03:02:19.937 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-09-13T03:02:19.937 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-09-13T03:02:19.938 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-09-13T03:02:19.938 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-09-13T03:02:19.938 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-09-13T03:02:19.938 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-09-13T03:02:19.938 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-09-13T03:02:19.939 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-09-13T03:02:19.939 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-09-13T03:02:19.939 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-09-13T03:02:19.939 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-09-13T03:02:19.939 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-09-13T03:02:19.940 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-09-13T03:02:19.940 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-09-13T03:02:19.940 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-09-13T03:02:19.940 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-09-13T03:02:19.940 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-09-13T03:02:19.941 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-09-13T03:02:19.941 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-09-13T03:02:19.941 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-09-13T03:02:19.941 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-09-13T03:02:19.941 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-09-13T03:02:19.942 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-09-13T03:02:19.942 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-09-13T03:02:19.942 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-09-13T03:02:19.942 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-09-13T03:02:19.942 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-09-13T03:02:19.943 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-09-13T03:02:19.943 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-09-13T03:02:19.943 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-09-13T03:02:19.943 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-09-13T03:02:19.943 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-09-13T03:02:19.943 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-09-13T03:02:19.944 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-09-13T03:02:19.944 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-09-13T03:02:19.944 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados2-2:18.0.0-6088.g2110e007.el8.x86_64 129/134 2023-09-13T03:02:19.944 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-09-13T03:02:19.944 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librbd1-2:18.0.0-6088.g2110e007.el8.x86_64 131/134 2023-09-13T03:02:19.945 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-13T03:02:19.945 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-09-13T03:02:22.611 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-13T03:02:22.611 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:02:06 ago on Wed 13 Sep 2023 03:00:16 AM UTC. 2023-09-13T03:02:25.412 INFO:teuthology.orchestra.run.smithi064.stdout:Installed products updated. 2023-09-13T03:02:25.656 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:02:25.656 INFO:teuthology.orchestra.run.smithi064.stdout:Upgraded: 2023-09-13T03:02:25.656 INFO:teuthology.orchestra.run.smithi064.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-09-13T03:02:25.657 INFO:teuthology.orchestra.run.smithi064.stdout: librados2-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:25.657 INFO:teuthology.orchestra.run.smithi064.stdout: librbd1-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:25.657 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2023-09-13T03:02:25.657 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-09-13T03:02:25.658 INFO:teuthology.orchestra.run.smithi064.stdout: automake-1.16.1-7.el8.noarch 2023-09-13T03:02:25.658 INFO:teuthology.orchestra.run.smithi064.stdout: bison-3.0.4-10.el8.x86_64 2023-09-13T03:02:25.658 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:25.658 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:25.658 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-common-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:25.659 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-fuse-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:25.659 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-grafana-dashboards-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-13T03:02:25.659 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-immutable-object-cache-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:25.659 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mds-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:25.659 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:25.660 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-cephadm-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-13T03:02:25.660 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-dashboard-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-13T03:02:25.660 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-13T03:02:25.660 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-modules-core-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-13T03:02:25.660 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-rook-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-13T03:02:25.661 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mon-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:25.661 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-osd-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:25.661 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-prometheus-alerts-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-13T03:02:25.661 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-radosgw-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:25.661 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-selinux-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:25.661 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-test-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:25.662 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-volume-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-13T03:02:25.662 INFO:teuthology.orchestra.run.smithi064.stdout: cephadm-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-13T03:02:25.662 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:25.662 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-top-2:18.0.0-6088.g2110e007.el8.noarch 2023-09-13T03:02:25.662 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-09-13T03:02:25.663 INFO:teuthology.orchestra.run.smithi064.stdout: flex-2.6.1-9.el8.x86_64 2023-09-13T03:02:25.663 INFO:teuthology.orchestra.run.smithi064.stdout: fuse-2.9.7-15.el8.x86_64 2023-09-13T03:02:25.663 INFO:teuthology.orchestra.run.smithi064.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-09-13T03:02:25.663 INFO:teuthology.orchestra.run.smithi064.stdout: jq-1.6-3.el8.x86_64 2023-09-13T03:02:25.664 INFO:teuthology.orchestra.run.smithi064.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-09-13T03:02:25.664 INFO:teuthology.orchestra.run.smithi064.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-09-13T03:02:25.664 INFO:teuthology.orchestra.run.smithi064.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-09-13T03:02:25.664 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs-devel-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:25.664 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs2-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:25.665 INFO:teuthology.orchestra.run.smithi064.stdout: libcephsqlite-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:25.665 INFO:teuthology.orchestra.run.smithi064.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-09-13T03:02:25.665 INFO:teuthology.orchestra.run.smithi064.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-09-13T03:02:25.665 INFO:teuthology.orchestra.run.smithi064.stdout: liboath-2.6.2-3.el8.x86_64 2023-09-13T03:02:25.665 INFO:teuthology.orchestra.run.smithi064.stdout: libpq-13.5-1.el8.x86_64 2023-09-13T03:02:25.666 INFO:teuthology.orchestra.run.smithi064.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-09-13T03:02:25.666 INFO:teuthology.orchestra.run.smithi064.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-09-13T03:02:25.666 INFO:teuthology.orchestra.run.smithi064.stdout: librados-devel-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:25.666 INFO:teuthology.orchestra.run.smithi064.stdout: libradosstriper1-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:25.666 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-09-13T03:02:25.667 INFO:teuthology.orchestra.run.smithi064.stdout: librgw2-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:25.667 INFO:teuthology.orchestra.run.smithi064.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-09-13T03:02:25.667 INFO:teuthology.orchestra.run.smithi064.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-09-13T03:02:25.667 INFO:teuthology.orchestra.run.smithi064.stdout: libtool-2.4.6-25.el8.x86_64 2023-09-13T03:02:25.668 INFO:teuthology.orchestra.run.smithi064.stdout: libunwind-1.3.1-3.el8.x86_64 2023-09-13T03:02:25.668 INFO:teuthology.orchestra.run.smithi064.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-09-13T03:02:25.668 INFO:teuthology.orchestra.run.smithi064.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-09-13T03:02:25.668 INFO:teuthology.orchestra.run.smithi064.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-09-13T03:02:25.668 INFO:teuthology.orchestra.run.smithi064.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-09-13T03:02:25.668 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-0.3.15-3.el8.x86_64 2023-09-13T03:02:25.669 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-09-13T03:02:25.669 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-09-13T03:02:25.669 INFO:teuthology.orchestra.run.smithi064.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-09-13T03:02:25.669 INFO:teuthology.orchestra.run.smithi064.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-09-13T03:02:25.669 INFO:teuthology.orchestra.run.smithi064.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-09-13T03:02:25.670 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-13T03:02:25.670 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-13T03:02:25.670 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-13T03:02:25.670 INFO:teuthology.orchestra.run.smithi064.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-09-13T03:02:25.670 INFO:teuthology.orchestra.run.smithi064.stdout: python3-babel-2.5.1-7.el8.noarch 2023-09-13T03:02:25.671 INFO:teuthology.orchestra.run.smithi064.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-09-13T03:02:25.671 INFO:teuthology.orchestra.run.smithi064.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-09-13T03:02:25.671 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-09-13T03:02:25.671 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:25.672 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-common-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:25.672 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cephfs-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:25.672 INFO:teuthology.orchestra.run.smithi064.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-09-13T03:02:25.672 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-09-13T03:02:25.672 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-09-13T03:02:25.673 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-09-13T03:02:25.673 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-09-13T03:02:25.673 INFO:teuthology.orchestra.run.smithi064.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-09-13T03:02:25.673 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-6.2-6.el8.noarch 2023-09-13T03:02:25.673 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-09-13T03:02:25.674 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-09-13T03:02:25.674 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-09-13T03:02:25.674 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-09-13T03:02:25.674 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-09-13T03:02:25.674 INFO:teuthology.orchestra.run.smithi064.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-09-13T03:02:25.675 INFO:teuthology.orchestra.run.smithi064.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-09-13T03:02:25.675 INFO:teuthology.orchestra.run.smithi064.stdout: python3-mako-1.0.6-13.el8.noarch 2023-09-13T03:02:25.675 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-09-13T03:02:25.675 INFO:teuthology.orchestra.run.smithi064.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-09-13T03:02:25.675 INFO:teuthology.orchestra.run.smithi064.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-09-13T03:02:25.676 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-09-13T03:02:25.676 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-09-13T03:02:25.676 INFO:teuthology.orchestra.run.smithi064.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-09-13T03:02:25.676 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-09-13T03:02:25.676 INFO:teuthology.orchestra.run.smithi064.stdout: python3-portend-2.6-1.el8.noarch 2023-09-13T03:02:25.676 INFO:teuthology.orchestra.run.smithi064.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-09-13T03:02:25.677 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-09-13T03:02:25.677 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-09-13T03:02:25.677 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-09-13T03:02:25.677 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pycparser-2.14-14.el8.noarch 2023-09-13T03:02:25.677 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytz-2017.2-9.el8.noarch 2023-09-13T03:02:25.678 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rados-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:25.678 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rbd-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:25.678 INFO:teuthology.orchestra.run.smithi064.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-09-13T03:02:25.678 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-09-13T03:02:25.679 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rgw-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:25.679 INFO:teuthology.orchestra.run.smithi064.stdout: python3-routes-2.4.1-12.el8.noarch 2023-09-13T03:02:25.679 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rsa-4.9-2.el8.noarch 2023-09-13T03:02:25.679 INFO:teuthology.orchestra.run.smithi064.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-09-13T03:02:25.679 INFO:teuthology.orchestra.run.smithi064.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-09-13T03:02:25.680 INFO:teuthology.orchestra.run.smithi064.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-09-13T03:02:25.680 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-09-13T03:02:25.680 INFO:teuthology.orchestra.run.smithi064.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-09-13T03:02:25.680 INFO:teuthology.orchestra.run.smithi064.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-09-13T03:02:25.680 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-09-13T03:02:25.681 INFO:teuthology.orchestra.run.smithi064.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-09-13T03:02:25.681 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-09-13T03:02:25.681 INFO:teuthology.orchestra.run.smithi064.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-09-13T03:02:25.681 INFO:teuthology.orchestra.run.smithi064.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-09-13T03:02:25.681 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-fuse-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:25.682 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-mirror-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:25.682 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-nbd-2:18.0.0-6088.g2110e007.el8.x86_64 2023-09-13T03:02:25.682 INFO:teuthology.orchestra.run.smithi064.stdout: socat-1.7.4.1-1.el8.x86_64 2023-09-13T03:02:25.682 INFO:teuthology.orchestra.run.smithi064.stdout: thrift-0.13.0-2.el8.x86_64 2023-09-13T03:02:25.682 INFO:teuthology.orchestra.run.smithi064.stdout: uuid-1.6.2-43.el8.x86_64 2023-09-13T03:02:25.683 INFO:teuthology.orchestra.run.smithi064.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-09-13T03:02:25.683 INFO:teuthology.orchestra.run.smithi064.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-09-13T03:02:25.683 INFO:teuthology.orchestra.run.smithi064.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-09-13T03:02:25.684 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:02:25.684 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2023-09-13T03:02:25.995 DEBUG:teuthology.parallel:result is None 2023-09-13T03:02:25.995 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2110e00747b31a0d73768c9e5229da18b26b8aa0 2023-09-13T03:02:26.135 DEBUG:teuthology.orchestra.run.smithi064:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-09-13T03:02:26.189 INFO:teuthology.orchestra.run.smithi064.stdout:18.0.0-6088.g2110e007.el8 2023-09-13T03:02:26.190 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6088.g2110e007.el8 2023-09-13T03:02:26.190 INFO:teuthology.task.install:The correct ceph version 18.0.0-6088.g2110e007 is installed. 2023-09-13T03:02:26.191 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2110e00747b31a0d73768c9e5229da18b26b8aa0 2023-09-13T03:02:26.328 DEBUG:teuthology.orchestra.run.smithi142:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-09-13T03:02:26.357 INFO:teuthology.orchestra.run.smithi142.stdout:18.0.0-6088.g2110e007.el8 2023-09-13T03:02:26.358 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6088.g2110e007.el8 2023-09-13T03:02:26.358 INFO:teuthology.task.install:The correct ceph version 18.0.0-6088.g2110e007 is installed. 2023-09-13T03:02:26.359 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2110e00747b31a0d73768c9e5229da18b26b8aa0 2023-09-13T03:02:26.496 DEBUG:teuthology.orchestra.run.smithi170:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-09-13T03:02:26.524 INFO:teuthology.orchestra.run.smithi170.stdout:18.0.0-6088.g2110e007.el8 2023-09-13T03:02:26.525 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6088.g2110e007.el8 2023-09-13T03:02:26.525 INFO:teuthology.task.install:The correct ceph version 18.0.0-6088.g2110e007 is installed. 2023-09-13T03:02:26.525 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-09-13T03:02:26.526 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-13T03:02:26.526 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-13T03:02:26.617 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-09-13T03:02:26.617 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-13T03:02:26.666 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-09-13T03:02:26.667 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-13T03:02:26.714 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-09-13T03:02:26.715 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-13T03:02:26.716 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/daemon-helper 2023-09-13T03:02:26.787 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-13T03:02:26.898 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-09-13T03:02:26.898 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/daemon-helper 2023-09-13T03:02:26.930 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-13T03:02:27.000 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-09-13T03:02:27.000 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/daemon-helper 2023-09-13T03:02:27.034 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-13T03:02:27.105 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-09-13T03:02:27.105 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-13T03:02:27.106 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-13T03:02:27.181 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-13T03:02:27.292 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-09-13T03:02:27.292 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-13T03:02:27.324 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-13T03:02:27.394 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-09-13T03:02:27.395 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-13T03:02:27.427 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-13T03:02:27.499 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-09-13T03:02:27.500 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-13T03:02:27.500 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/stdin-killer 2023-09-13T03:02:27.572 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-13T03:02:27.684 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-09-13T03:02:27.684 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/stdin-killer 2023-09-13T03:02:27.716 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-13T03:02:27.787 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-09-13T03:02:27.788 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/stdin-killer 2023-09-13T03:02:27.820 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-13T03:02:27.891 INFO:teuthology.run_tasks:Running task cephadm... 2023-09-13T03:02:27.969 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'global': {'ms die on skipped message': False}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '2110e00747b31a0d73768c9e5229da18b26b8aa0', 'subvols': {'create': 2}} 2023-09-13T03:02:27.970 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 2023-09-13T03:02:27.971 INFO:tasks.cephadm:Cluster fsid is f8dbced4-51e1-11ee-9ab7-7b867c8bd7da 2023-09-13T03:02:27.971 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-09-13T03:02:27.971 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.64', 'mon.b': '172.21.15.142', 'mon.c': '172.21.15.170'} 2023-09-13T03:02:27.972 INFO:tasks.cephadm:First mon is mon.a on smithi064 2023-09-13T03:02:27.972 INFO:tasks.cephadm:First mgr is x 2023-09-13T03:02:27.972 INFO:tasks.cephadm:Normalizing hostnames... 2023-09-13T03:02:27.973 DEBUG:teuthology.orchestra.run.smithi064:> sudo hostname $(hostname -s) 2023-09-13T03:02:28.045 DEBUG:teuthology.orchestra.run.smithi142:> sudo hostname $(hostname -s) 2023-09-13T03:02:28.078 DEBUG:teuthology.orchestra.run.smithi170:> sudo hostname $(hostname -s) 2023-09-13T03:02:28.109 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-09-13T03:02:28.109 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2110e00747b31a0d73768c9e5229da18b26b8aa0 2023-09-13T03:02:28.387 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '2110e00747b31a0d73768c9e5229da18b26b8aa0', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/72559/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-6088-g2110e007', 'node_name': '172.21.5.32+adami02', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-6088.g2110e007'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-09-12 14:00:53.539150', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-batrick-testing-20230912.122437', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2023-09-13T03:02:28.736 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-batrick-testing-20230912.122437, sha1 2110e00747b31a0d73768c9e5229da18b26b8aa0 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=2110e00747b31a0d73768c9e5229da18b26b8aa0 2023-09-13T03:02:28.737 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/centos/8/x86_64/flavors/default/cephadm 2023-09-13T03:02:28.738 DEBUG:teuthology.orchestra.run.smithi064:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-09-13T03:02:29.158 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-rw-r--. 1 ubuntu ubuntu 728366 Sep 13 03:02 /home/ubuntu/cephtest/cephadm 2023-09-13T03:02:29.159 DEBUG:teuthology.orchestra.run.smithi142:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-09-13T03:02:29.527 INFO:teuthology.orchestra.run.smithi142.stdout:-rw-rw-r--. 1 ubuntu ubuntu 728366 Sep 13 03:02 /home/ubuntu/cephtest/cephadm 2023-09-13T03:02:29.527 DEBUG:teuthology.orchestra.run.smithi170:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20230912.122437/2110e00747b31a0d73768c9e5229da18b26b8aa0/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-09-13T03:02:30.035 INFO:teuthology.orchestra.run.smithi170.stdout:-rw-rw-r--. 1 ubuntu ubuntu 728366 Sep 13 03:02 /home/ubuntu/cephtest/cephadm 2023-09-13T03:02:30.036 DEBUG:teuthology.orchestra.run.smithi064:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-09-13T03:02:30.248 DEBUG:teuthology.orchestra.run.smithi142:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-09-13T03:02:30.270 DEBUG:teuthology.orchestra.run.smithi170:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-09-13T03:02:30.307 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 on all hosts... 2023-09-13T03:02:30.307 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 pull 2023-09-13T03:02:30.313 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 pull 2023-09-13T03:02:30.315 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 pull 2023-09-13T03:02:31.753 INFO:teuthology.orchestra.run.smithi170.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0... 2023-09-13T03:02:32.296 INFO:teuthology.orchestra.run.smithi142.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0... 2023-09-13T03:02:34.862 INFO:teuthology.orchestra.run.smithi064.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0... 2023-09-13T03:03:08.036 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-09-13T03:03:08.037 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_version": "ceph version 18.0.0-6088-g2110e007 (2110e00747b31a0d73768c9e5229da18b26b8aa0) reef (dev)", 2023-09-13T03:03:08.037 INFO:teuthology.orchestra.run.smithi170.stdout: "image_id": "bf15e540d61151b5ec5ff58d59c42648a8ab7c32d2849aa41b0b97aca329cfed", 2023-09-13T03:03:08.037 INFO:teuthology.orchestra.run.smithi170.stdout: "repo_digests": [ 2023-09-13T03:03:08.037 INFO:teuthology.orchestra.run.smithi170.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:245225109b90f7de349b7e93f44579ca2c2de61e498376ad1f95f5caf5575b65" 2023-09-13T03:03:08.038 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2023-09-13T03:03:08.038 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-09-13T03:03:09.508 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2023-09-13T03:03:09.508 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_version": "ceph version 18.0.0-6088-g2110e007 (2110e00747b31a0d73768c9e5229da18b26b8aa0) reef (dev)", 2023-09-13T03:03:09.508 INFO:teuthology.orchestra.run.smithi142.stdout: "image_id": "bf15e540d61151b5ec5ff58d59c42648a8ab7c32d2849aa41b0b97aca329cfed", 2023-09-13T03:03:09.509 INFO:teuthology.orchestra.run.smithi142.stdout: "repo_digests": [ 2023-09-13T03:03:09.509 INFO:teuthology.orchestra.run.smithi142.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:245225109b90f7de349b7e93f44579ca2c2de61e498376ad1f95f5caf5575b65" 2023-09-13T03:03:09.509 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2023-09-13T03:03:09.509 INFO:teuthology.orchestra.run.smithi142.stdout:} 2023-09-13T03:03:13.364 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-13T03:03:13.365 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 18.0.0-6088-g2110e007 (2110e00747b31a0d73768c9e5229da18b26b8aa0) reef (dev)", 2023-09-13T03:03:13.365 INFO:teuthology.orchestra.run.smithi064.stdout: "image_id": "bf15e540d61151b5ec5ff58d59c42648a8ab7c32d2849aa41b0b97aca329cfed", 2023-09-13T03:03:13.365 INFO:teuthology.orchestra.run.smithi064.stdout: "repo_digests": [ 2023-09-13T03:03:13.365 INFO:teuthology.orchestra.run.smithi064.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:245225109b90f7de349b7e93f44579ca2c2de61e498376ad1f95f5caf5575b65" 2023-09-13T03:03:13.366 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-09-13T03:03:13.366 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-13T03:03:13.395 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /etc/ceph 2023-09-13T03:03:13.468 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /etc/ceph 2023-09-13T03:03:13.505 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /etc/ceph 2023-09-13T03:03:13.542 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 777 /etc/ceph 2023-09-13T03:03:13.610 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 777 /etc/ceph 2023-09-13T03:03:13.641 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 777 /etc/ceph 2023-09-13T03:03:13.673 INFO:tasks.cephadm:Writing seed config... 2023-09-13T03:03:13.674 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-09-13T03:03:13.675 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-09-13T03:03:13.675 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2023-09-13T03:03:13.675 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-09-13T03:03:13.676 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-09-13T03:03:13.676 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-09-13T03:03:13.676 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-09-13T03:03:13.676 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-09-13T03:03:13.677 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2023-09-13T03:03:13.677 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2023-09-13T03:03:13.677 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2023-09-13T03:03:13.677 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-09-13T03:03:13.678 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-09-13T03:03:13.678 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2023-09-13T03:03:13.678 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2023-09-13T03:03:13.678 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-09-13T03:03:13.679 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-09-13T03:03:13.679 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2023-09-13T03:03:13.679 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-09-13T03:03:13.679 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-09-13T03:03:13.680 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-09-13T03:03:13.680 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-09-13T03:03:13.680 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-09-13T03:03:13.681 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-09-13T03:03:13.681 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2023-09-13T03:03:13.682 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-13T03:03:13.682 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-09-13T03:03:13.725 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = f8dbced4-51e1-11ee-9ab7-7b867c8bd7da ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2023-09-13T03:03:13.726 DEBUG:teuthology.orchestra.run.smithi064:mon.a> sudo journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mon.a.service 2023-09-13T03:03:13.770 DEBUG:teuthology.orchestra.run.smithi064:mgr.x> sudo journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mgr.x.service 2023-09-13T03:03:13.815 INFO:tasks.cephadm:Bootstrapping... 2023-09-13T03:03:13.816 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 -v bootstrap --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.64 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-09-13T03:03:13.838 INFO:journalctl@ceph.mon.a.smithi064.stdout:-- Logs begin at Wed 2023-09-13 02:46:56 UTC. -- 2023-09-13T03:03:13.884 INFO:journalctl@ceph.mgr.x.smithi064.stdout:-- Logs begin at Wed 2023-09-13 02:46:56 UTC. -- 2023-09-13T03:03:14.050 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2023-09-13T03:03:14.050 INFO:teuthology.orchestra.run.smithi064.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0', '-v', 'bootstrap', '--fsid', 'f8dbced4-51e1-11ee-9ab7-7b867c8bd7da', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.64', '--skip-admin-label'] 2023-09-13T03:03:14.224 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stdout 4.1.1 2023-09-13T03:03:14.225 INFO:teuthology.orchestra.run.smithi064.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-09-13T03:03:14.225 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying podman|docker is present... 2023-09-13T03:03:14.397 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stdout 4.1.1 2023-09-13T03:03:14.397 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying lvm2 is present... 2023-09-13T03:03:14.397 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying time synchronization is in place... 2023-09-13T03:03:14.411 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-09-13T03:03:14.411 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-09-13T03:03:14.424 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-09-13T03:03:14.425 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2023-09-13T03:03:14.438 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout enabled 2023-09-13T03:03:14.451 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout active 2023-09-13T03:03:14.452 INFO:teuthology.orchestra.run.smithi064.stdout:Unit chronyd.service is enabled and running 2023-09-13T03:03:14.452 INFO:teuthology.orchestra.run.smithi064.stdout:Repeating the final host check... 2023-09-13T03:03:14.622 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stdout 4.1.1 2023-09-13T03:03:14.623 INFO:teuthology.orchestra.run.smithi064.stdout:podman (/bin/podman) version 4.1.1 is present 2023-09-13T03:03:14.623 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl is present 2023-09-13T03:03:14.623 INFO:teuthology.orchestra.run.smithi064.stdout:lvcreate is present 2023-09-13T03:03:14.636 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-09-13T03:03:14.636 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-09-13T03:03:14.648 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-09-13T03:03:14.649 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2023-09-13T03:03:14.662 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout enabled 2023-09-13T03:03:14.675 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout active 2023-09-13T03:03:14.676 INFO:teuthology.orchestra.run.smithi064.stdout:Unit chronyd.service is enabled and running 2023-09-13T03:03:14.676 INFO:teuthology.orchestra.run.smithi064.stdout:Host looks OK 2023-09-13T03:03:14.676 INFO:teuthology.orchestra.run.smithi064.stdout:Cluster fsid: f8dbced4-51e1-11ee-9ab7-7b867c8bd7da 2023-09-13T03:03:14.677 INFO:teuthology.orchestra.run.smithi064.stdout:Acquiring lock 139905671431168 on /run/cephadm/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da.lock 2023-09-13T03:03:14.677 INFO:teuthology.orchestra.run.smithi064.stdout:Lock 139905671431168 acquired on /run/cephadm/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da.lock 2023-09-13T03:03:14.677 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying IP 172.21.15.64 port 3300 ... 2023-09-13T03:03:14.678 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying IP 172.21.15.64 port 6789 ... 2023-09-13T03:03:14.678 INFO:teuthology.orchestra.run.smithi064.stdout:Base mon IP(s) is [172.21.15.64:3300, 172.21.15.64:6789], mon addrv is [v2:172.21.15.64:3300,v1:172.21.15.64:6789] 2023-09-13T03:03:14.685 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp metric 100 2023-09-13T03:03:14.685 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.64 metric 100 2023-09-13T03:03:14.692 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-09-13T03:03:14.692 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2023-09-13T03:03:14.692 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2023-09-13T03:03:14.698 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-09-13T03:03:14.699 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-09-13T03:03:14.699 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-09-13T03:03:14.699 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2023-09-13T03:03:14.699 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout inet6 fe80::7de2:f971:923f:f167/64 scope link noprefixroute 2023-09-13T03:03:14.699 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-09-13T03:03:14.700 INFO:teuthology.orchestra.run.smithi064.stdout:Mon IP `172.21.15.64` is in CIDR network `172.21.0.0/20` 2023-09-13T03:03:14.700 INFO:teuthology.orchestra.run.smithi064.stdout:Mon IP `172.21.15.64` is in CIDR network `172.21.0.0/20` 2023-09-13T03:03:14.701 INFO:teuthology.orchestra.run.smithi064.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-09-13T03:03:14.701 INFO:teuthology.orchestra.run.smithi064.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-09-13T03:03:14.702 INFO:teuthology.orchestra.run.smithi064.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0... 2023-09-13T03:03:15.175 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stdout bf15e540d61151b5ec5ff58d59c42648a8ab7c32d2849aa41b0b97aca329cfed 2023-09-13T03:03:15.175 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0... 2023-09-13T03:03:15.176 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stderr Getting image source signatures 2023-09-13T03:03:15.176 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stderr Copying blob sha256:d0e1bfbb8440979edf72f0bc146ed182f1efda6b57c3af4c34dad6d07bc7592b 2023-09-13T03:03:15.176 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stderr Copying blob sha256:aba40bcce7ff9e06f5f6d056b94d79b3c5f9fa5aba1a4d6d95b68b2cadf32b62 2023-09-13T03:03:15.176 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stderr Copying config sha256:bf15e540d61151b5ec5ff58d59c42648a8ab7c32d2849aa41b0b97aca329cfed 2023-09-13T03:03:15.176 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stderr Writing manifest to image destination 2023-09-13T03:03:15.177 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stderr Storing signatures 2023-09-13T03:03:16.050 INFO:teuthology.orchestra.run.smithi064.stdout:ceph: stdout ceph version 18.0.0-6088-g2110e007 (2110e00747b31a0d73768c9e5229da18b26b8aa0) reef (dev) 2023-09-13T03:03:16.051 INFO:teuthology.orchestra.run.smithi064.stdout:Ceph version: ceph version 18.0.0-6088-g2110e007 (2110e00747b31a0d73768c9e5229da18b26b8aa0) reef (dev) 2023-09-13T03:03:16.051 INFO:teuthology.orchestra.run.smithi064.stdout:Extracting ceph user uid/gid from container image... 2023-09-13T03:03:17.229 INFO:teuthology.orchestra.run.smithi064.stdout:stat: stdout 167 167 2023-09-13T03:03:17.230 INFO:teuthology.orchestra.run.smithi064.stdout:Creating initial keys... 2023-09-13T03:03:18.006 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-authtool: stdout AQB1JgFl3A3lJBAAEBXOx2EPTreIfNMyrxehCQ== 2023-09-13T03:03:18.783 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-authtool: stdout AQB2JgFls2N5GBAA1CAiAYh59/o1pz8FNRwaJA== 2023-09-13T03:03:19.560 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-authtool: stdout AQB3JgFlZdsQCxAAbJQp51IZEnfa57UyO3HwhA== 2023-09-13T03:03:19.561 INFO:teuthology.orchestra.run.smithi064.stdout:Creating initial monmap... 2023-09-13T03:03:20.336 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-09-13T03:03:20.336 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-09-13T03:03:20.336 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to f8dbced4-51e1-11ee-9ab7-7b867c8bd7da 2023-09-13T03:03:20.336 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-09-13T03:03:20.337 INFO:teuthology.orchestra.run.smithi064.stdout:monmaptool for a [v2:172.21.15.64:3300,v1:172.21.15.64:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-09-13T03:03:20.337 INFO:teuthology.orchestra.run.smithi064.stdout:setting min_mon_release = pacific 2023-09-13T03:03:20.337 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: set fsid to f8dbced4-51e1-11ee-9ab7-7b867c8bd7da 2023-09-13T03:03:20.337 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-09-13T03:03:20.337 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:03:20.338 INFO:teuthology.orchestra.run.smithi064.stdout:Creating mon... 2023-09-13T03:03:21.213 INFO:teuthology.orchestra.run.smithi064.stdout:create mon.a on 2023-09-13T03:03:21.709 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-09-13T03:03:21.919 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-09-13T03:03:22.852 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da.target → /etc/systemd/system/ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da.target. 2023-09-13T03:03:22.853 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da.target → /etc/systemd/system/ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da.target. 2023-09-13T03:03:23.219 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mon.a 2023-09-13T03:03:23.219 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Failed to reset failed state of unit ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mon.a.service: Unit ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mon.a.service not loaded. 2023-09-13T03:03:23.422 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da.target.wants/ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mon.a.service → /etc/systemd/system/ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@.service. 2023-09-13T03:03:23.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:23 smithi064 systemd[1]: Starting Ceph mon.a for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:03:24.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:23 smithi064 podman[97128]: 2023-09-13T03:03:24.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:24 smithi064 bash[97128]: c9545ddd48b82b1fe58f51d65ef7396299ea2c2fb3a4358f593ae9f9a8e28bc7 2023-09-13T03:03:24.225 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-09-13T03:03:24.225 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout disabled 2023-09-13T03:03:24.241 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-09-13T03:03:24.242 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2023-09-13T03:03:24.242 INFO:teuthology.orchestra.run.smithi064.stdout:firewalld.service is not enabled 2023-09-13T03:03:24.242 INFO:teuthology.orchestra.run.smithi064.stdout:Not possible to enable service . firewalld.service is not available 2023-09-13T03:03:24.243 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mon to start... 2023-09-13T03:03:24.243 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mon... 2023-09-13T03:03:24.518 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:24 smithi064 ceph-mon[97168]: mkfs f8dbced4-51e1-11ee-9ab7-7b867c8bd7da 2023-09-13T03:03:24.518 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:24 smithi064 ceph-mon[97168]: mon.a is new leader, mons a in quorum (ranks 0) 2023-09-13T03:03:24.518 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:24 smithi064 systemd[1]: Started Ceph mon.a for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da. 2023-09-13T03:03:25.224 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout cluster: 2023-09-13T03:03:25.226 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout id: f8dbced4-51e1-11ee-9ab7-7b867c8bd7da 2023-09-13T03:03:25.227 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-09-13T03:03:25.227 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2023-09-13T03:03:25.227 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout services: 2023-09-13T03:03:25.227 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.697591s) 2023-09-13T03:03:25.227 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-09-13T03:03:25.228 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-09-13T03:03:25.228 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2023-09-13T03:03:25.228 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout data: 2023-09-13T03:03:25.228 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-09-13T03:03:25.229 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-09-13T03:03:25.229 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-09-13T03:03:25.229 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout pgs: 2023-09-13T03:03:25.229 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2023-09-13T03:03:25.229 INFO:teuthology.orchestra.run.smithi064.stdout:mon is available 2023-09-13T03:03:25.230 INFO:teuthology.orchestra.run.smithi064.stdout:Assimilating anything we can from ceph.conf... 2023-09-13T03:03:25.450 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:25 smithi064 ceph-mon[97168]: mon.a is new leader, mons a in quorum (ranks 0) 2023-09-13T03:03:25.450 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:25 smithi064 ceph-mon[97168]: monmap e1: 1 mons at {a=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0]} removed_ranks: {} 2023-09-13T03:03:25.450 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:25 smithi064 ceph-mon[97168]: fsmap 2023-09-13T03:03:25.451 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:25 smithi064 ceph-mon[97168]: osdmap e1: 0 total, 0 up, 0 in 2023-09-13T03:03:25.451 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:25 smithi064 ceph-mon[97168]: mgrmap e1: no daemons active 2023-09-13T03:03:25.451 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:25 smithi064 ceph-mon[97168]: from='client.? 172.21.15.64:0/1419031749' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-09-13T03:03:26.204 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2023-09-13T03:03:26.205 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [global] 2023-09-13T03:03:26.205 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout fsid = f8dbced4-51e1-11ee-9ab7-7b867c8bd7da 2023-09-13T03:03:26.205 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.64:3300,v1:172.21.15.64:6789] 2023-09-13T03:03:26.205 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-09-13T03:03:26.206 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-09-13T03:03:26.206 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-09-13T03:03:26.206 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-09-13T03:03:26.206 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2023-09-13T03:03:26.206 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [mgr] 2023-09-13T03:03:26.207 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-09-13T03:03:26.207 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2023-09-13T03:03:26.208 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [osd] 2023-09-13T03:03:26.208 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-09-13T03:03:26.208 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-09-13T03:03:26.208 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-09-13T03:03:26.209 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-09-13T03:03:26.209 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-09-13T03:03:26.209 INFO:teuthology.orchestra.run.smithi064.stdout:Generating new minimal ceph.conf... 2023-09-13T03:03:26.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:26 smithi064 ceph-mon[97168]: from='client.? 172.21.15.64:0/3168483838' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-09-13T03:03:26.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:26 smithi064 ceph-mon[97168]: from='client.? 172.21.15.64:0/3168483838' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-09-13T03:03:27.184 INFO:teuthology.orchestra.run.smithi064.stdout:Restarting the monitor... 2023-09-13T03:03:27.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:27 smithi064 ceph-mon[97168]: from='client.? 172.21.15.64:0/3203270525' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:03:27.652 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:27 smithi064 systemd[1]: Stopping Ceph mon.a for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:03:27.914 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:27 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mon-a[97144]: 2023-09-13T03:03:27.650+0000 7fa92cc0f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T03:03:27.914 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:27 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mon-a[97144]: 2023-09-13T03:03:27.650+0000 7fa92cc0f700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-09-13T03:03:28.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:27 smithi064 bash[97506]: ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mon-a 2023-09-13T03:03:28.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:28 smithi064 systemd[1]: ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mon.a.service: Succeeded. 2023-09-13T03:03:28.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:28 smithi064 systemd[1]: Stopped Ceph mon.a for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da. 2023-09-13T03:03:28.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:28 smithi064 systemd[1]: Starting Ceph mon.a for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:03:29.040 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:28 smithi064 podman[97635]: 2023-09-13T03:03:29.041 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 bash[97635]: 27338a39719e34e5ba3caa1d222174e7b7f9377e651efee541810f15e0db9b54 2023-09-13T03:03:29.041 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: set uid:gid to 167:167 (ceph:ceph) 2023-09-13T03:03:29.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: ceph version 18.0.0-6088-g2110e007 (2110e00747b31a0d73768c9e5229da18b26b8aa0) reef (dev), process ceph-mon, pid 7 2023-09-13T03:03:29.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: pidfile_write: ignore empty --pid-file 2023-09-13T03:03:29.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: load: jerasure load: lrc 2023-09-13T03:03:29.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: RocksDB version: 7.9.2 2023-09-13T03:03:29.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Git sha 0 2023-09-13T03:03:29.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Compile date 2023-09-12 13:00:11 2023-09-13T03:03:29.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: DB SUMMARY 2023-09-13T03:03:29.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: DB Session ID: UEB6MG67CB98OBS5VZWV 2023-09-13T03:03:29.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: CURRENT file: CURRENT 2023-09-13T03:03:29.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: IDENTITY file: IDENTITY 2023-09-13T03:03:29.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-09-13T03:03:29.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-09-13T03:03:29.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 89924 ; 2023-09-13T03:03:29.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.error_if_exists: 0 2023-09-13T03:03:29.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.create_if_missing: 0 2023-09-13T03:03:29.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.paranoid_checks: 1 2023-09-13T03:03:29.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.flush_verify_memtable_count: 1 2023-09-13T03:03:29.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-09-13T03:03:29.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-09-13T03:03:29.122 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.env: 0x559a92742da0 2023-09-13T03:03:29.122 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.fs: PosixFileSystem 2023-09-13T03:03:29.122 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.info_log: 0x559a94e914e0 2023-09-13T03:03:29.122 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.max_file_opening_threads: 16 2023-09-13T03:03:29.123 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.statistics: (nil) 2023-09-13T03:03:29.123 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.use_fsync: 0 2023-09-13T03:03:29.123 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.max_log_file_size: 0 2023-09-13T03:03:29.123 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-09-13T03:03:29.123 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.log_file_time_to_roll: 0 2023-09-13T03:03:29.124 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.keep_log_file_num: 1000 2023-09-13T03:03:29.124 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.recycle_log_file_num: 0 2023-09-13T03:03:29.124 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.allow_fallocate: 1 2023-09-13T03:03:29.124 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.allow_mmap_reads: 0 2023-09-13T03:03:29.125 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.allow_mmap_writes: 0 2023-09-13T03:03:29.125 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.use_direct_reads: 0 2023-09-13T03:03:29.125 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-09-13T03:03:29.125 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.create_missing_column_families: 0 2023-09-13T03:03:29.125 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.db_log_dir: 2023-09-13T03:03:29.126 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.wal_dir: 2023-09-13T03:03:29.126 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.table_cache_numshardbits: 6 2023-09-13T03:03:29.126 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.WAL_ttl_seconds: 0 2023-09-13T03:03:29.126 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.WAL_size_limit_MB: 0 2023-09-13T03:03:29.127 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-09-13T03:03:29.127 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-09-13T03:03:29.127 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.is_fd_close_on_exec: 1 2023-09-13T03:03:29.127 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.advise_random_on_open: 1 2023-09-13T03:03:29.127 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.db_write_buffer_size: 0 2023-09-13T03:03:29.128 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.write_buffer_manager: 0x559a94e4b4a0 2023-09-13T03:03:29.128 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-09-13T03:03:29.128 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-09-13T03:03:29.128 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.use_adaptive_mutex: 0 2023-09-13T03:03:29.128 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.rate_limiter: (nil) 2023-09-13T03:03:29.129 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-09-13T03:03:29.129 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.wal_recovery_mode: 2 2023-09-13T03:03:29.129 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.enable_thread_tracking: 0 2023-09-13T03:03:29.129 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.enable_pipelined_write: 0 2023-09-13T03:03:29.130 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.unordered_write: 0 2023-09-13T03:03:29.130 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-09-13T03:03:29.130 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-09-13T03:03:29.130 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-09-13T03:03:29.130 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-09-13T03:03:29.131 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.row_cache: None 2023-09-13T03:03:29.131 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.wal_filter: None 2023-09-13T03:03:29.131 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-09-13T03:03:29.131 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.allow_ingest_behind: 0 2023-09-13T03:03:29.131 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.two_write_queues: 0 2023-09-13T03:03:29.132 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.manual_wal_flush: 0 2023-09-13T03:03:29.132 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.wal_compression: 0 2023-09-13T03:03:29.132 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.atomic_flush: 0 2023-09-13T03:03:29.132 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-09-13T03:03:29.133 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.persist_stats_to_disk: 0 2023-09-13T03:03:29.133 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.write_dbid_to_manifest: 0 2023-09-13T03:03:29.133 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.log_readahead_size: 0 2023-09-13T03:03:29.133 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-09-13T03:03:29.133 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.best_efforts_recovery: 0 2023-09-13T03:03:29.134 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-09-13T03:03:29.134 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-09-13T03:03:29.134 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.allow_data_in_errors: 0 2023-09-13T03:03:29.134 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.db_host_id: __hostname__ 2023-09-13T03:03:29.134 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.enforce_single_del_contracts: true 2023-09-13T03:03:29.136 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.max_background_jobs: 2 2023-09-13T03:03:29.136 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.max_background_compactions: -1 2023-09-13T03:03:29.136 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.max_subcompactions: 1 2023-09-13T03:03:29.136 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-09-13T03:03:29.137 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-09-13T03:03:29.137 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.delayed_write_rate : 16777216 2023-09-13T03:03:29.137 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.max_total_wal_size: 0 2023-09-13T03:03:29.137 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-09-13T03:03:29.137 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.stats_dump_period_sec: 600 2023-09-13T03:03:29.138 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.stats_persist_period_sec: 600 2023-09-13T03:03:29.138 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-09-13T03:03:29.138 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.max_open_files: -1 2023-09-13T03:03:29.138 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.bytes_per_sync: 0 2023-09-13T03:03:29.139 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.wal_bytes_per_sync: 0 2023-09-13T03:03:29.139 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.strict_bytes_per_sync: 0 2023-09-13T03:03:29.139 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.compaction_readahead_size: 0 2023-09-13T03:03:29.139 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.max_background_flushes: -1 2023-09-13T03:03:29.139 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Compression algorithms supported: 2023-09-13T03:03:29.140 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-09-13T03:03:29.140 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: kZSTD supported: 0 2023-09-13T03:03:29.140 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: kXpressCompression supported: 0 2023-09-13T03:03:29.140 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: kLZ4HCCompression supported: 1 2023-09-13T03:03:29.140 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: kZlibCompression supported: 1 2023-09-13T03:03:29.141 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: kSnappyCompression supported: 1 2023-09-13T03:03:29.141 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: kLZ4Compression supported: 1 2023-09-13T03:03:29.141 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: kBZip2Compression supported: 0 2023-09-13T03:03:29.141 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Fast CRC32 supported: Supported on x86 2023-09-13T03:03:29.141 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: DMutex implementation: pthread_mutex_t 2023-09-13T03:03:29.141 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-09-13T03:03:29.142 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-09-13T03:03:29.142 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-09-13T03:03:29.142 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.merge_operator: 2023-09-13T03:03:29.142 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.compaction_filter: None 2023-09-13T03:03:29.143 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.compaction_filter_factory: None 2023-09-13T03:03:29.143 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.sst_partitioner_factory: None 2023-09-13T03:03:29.143 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.memtable_factory: SkipListFactory 2023-09-13T03:03:29.143 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.table_factory: BlockBasedTable 2023-09-13T03:03:29.143 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559a94e915e0) 2023-09-13T03:03:29.144 INFO:journalctl@ceph.mon.a.smithi064.stdout: cache_index_and_filter_blocks: 1 2023-09-13T03:03:29.144 INFO:journalctl@ceph.mon.a.smithi064.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-09-13T03:03:29.144 INFO:journalctl@ceph.mon.a.smithi064.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-09-13T03:03:29.144 INFO:journalctl@ceph.mon.a.smithi064.stdout: pin_top_level_index_and_filter: 1 2023-09-13T03:03:29.144 INFO:journalctl@ceph.mon.a.smithi064.stdout: index_type: 0 2023-09-13T03:03:29.145 INFO:journalctl@ceph.mon.a.smithi064.stdout: data_block_index_type: 0 2023-09-13T03:03:29.145 INFO:journalctl@ceph.mon.a.smithi064.stdout: index_shortening: 1 2023-09-13T03:03:29.145 INFO:journalctl@ceph.mon.a.smithi064.stdout: data_block_hash_table_util_ratio: 0.750000 2023-09-13T03:03:29.145 INFO:journalctl@ceph.mon.a.smithi064.stdout: checksum: 4 2023-09-13T03:03:29.145 INFO:journalctl@ceph.mon.a.smithi064.stdout: no_block_cache: 0 2023-09-13T03:03:29.146 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_cache: 0x559a94ee91f0 2023-09-13T03:03:29.146 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_cache_name: BinnedLRUCache 2023-09-13T03:03:29.146 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_cache_options: 2023-09-13T03:03:29.146 INFO:journalctl@ceph.mon.a.smithi064.stdout: capacity : 536870912 2023-09-13T03:03:29.146 INFO:journalctl@ceph.mon.a.smithi064.stdout: num_shard_bits : 4 2023-09-13T03:03:29.147 INFO:journalctl@ceph.mon.a.smithi064.stdout: strict_capacity_limit : 0 2023-09-13T03:03:29.147 INFO:journalctl@ceph.mon.a.smithi064.stdout: high_pri_pool_ratio: 0.000 2023-09-13T03:03:29.147 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_cache_compressed: (nil) 2023-09-13T03:03:29.147 INFO:journalctl@ceph.mon.a.smithi064.stdout: persistent_cache: (nil) 2023-09-13T03:03:29.147 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_size: 4096 2023-09-13T03:03:29.148 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_size_deviation: 10 2023-09-13T03:03:29.148 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_restart_interval: 16 2023-09-13T03:03:29.148 INFO:journalctl@ceph.mon.a.smithi064.stdout: index_block_restart_interval: 1 2023-09-13T03:03:29.148 INFO:journalctl@ceph.mon.a.smithi064.stdout: metadata_block_size: 4096 2023-09-13T03:03:29.148 INFO:journalctl@ceph.mon.a.smithi064.stdout: partition_filters: 0 2023-09-13T03:03:29.149 INFO:journalctl@ceph.mon.a.smithi064.stdout: use_delta_encoding: 1 2023-09-13T03:03:29.149 INFO:journalctl@ceph.mon.a.smithi064.stdout: filter_policy: bloomfilter 2023-09-13T03:03:29.149 INFO:journalctl@ceph.mon.a.smithi064.stdout: whole_key_filtering: 1 2023-09-13T03:03:29.149 INFO:journalctl@ceph.mon.a.smithi064.stdout: verify_compression: 0 2023-09-13T03:03:29.149 INFO:journalctl@ceph.mon.a.smithi064.stdout: read_amp_bytes_per_bit: 0 2023-09-13T03:03:29.150 INFO:journalctl@ceph.mon.a.smithi064.stdout: format_version: 5 2023-09-13T03:03:29.150 INFO:journalctl@ceph.mon.a.smithi064.stdout: enable_index_compression: 1 2023-09-13T03:03:29.150 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_align: 0 2023-09-13T03:03:29.150 INFO:journalctl@ceph.mon.a.smithi064.stdout: max_auto_readahead_size: 262144 2023-09-13T03:03:29.150 INFO:journalctl@ceph.mon.a.smithi064.stdout: prepopulate_block_cache: 0 2023-09-13T03:03:29.151 INFO:journalctl@ceph.mon.a.smithi064.stdout: initial_auto_readahead_size: 8192 2023-09-13T03:03:29.151 INFO:journalctl@ceph.mon.a.smithi064.stdout: num_file_reads_for_auto_readahead: 2 2023-09-13T03:03:29.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.write_buffer_size: 33554432 2023-09-13T03:03:29.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.max_write_buffer_number: 2 2023-09-13T03:03:29.151 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.compression: NoCompression 2023-09-13T03:03:29.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.bottommost_compression: Disabled 2023-09-13T03:03:29.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.prefix_extractor: nullptr 2023-09-13T03:03:29.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-09-13T03:03:29.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.num_levels: 7 2023-09-13T03:03:29.152 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-09-13T03:03:29.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-09-13T03:03:29.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-09-13T03:03:29.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-09-13T03:03:29.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-09-13T03:03:29.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-09-13T03:03:29.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-09-13T03:03:29.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-09-13T03:03:29.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-09-13T03:03:29.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-09-13T03:03:29.155 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-09-13T03:03:29.155 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-09-13T03:03:29.155 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.compression_opts.window_bits: -14 2023-09-13T03:03:29.155 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.compression_opts.level: 32767 2023-09-13T03:03:29.155 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.compression_opts.strategy: 0 2023-09-13T03:03:29.156 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-09-13T03:03:29.156 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-09-13T03:03:29.156 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-09-13T03:03:29.156 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-09-13T03:03:29.156 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.compression_opts.enabled: false 2023-09-13T03:03:29.157 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-09-13T03:03:29.157 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-09-13T03:03:29.157 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-09-13T03:03:29.157 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-09-13T03:03:29.157 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.target_file_size_base: 67108864 2023-09-13T03:03:29.158 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.target_file_size_multiplier: 1 2023-09-13T03:03:29.158 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-09-13T03:03:29.158 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-09-13T03:03:29.158 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-09-13T03:03:29.158 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-09-13T03:03:29.159 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-09-13T03:03:29.159 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-09-13T03:03:29.159 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-09-13T03:03:29.159 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-09-13T03:03:29.159 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-09-13T03:03:29.160 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-09-13T03:03:29.160 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-09-13T03:03:29.160 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-09-13T03:03:29.160 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-09-13T03:03:29.160 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.arena_block_size: 1048576 2023-09-13T03:03:29.161 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-09-13T03:03:29.161 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-09-13T03:03:29.161 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.disable_auto_compactions: 0 2023-09-13T03:03:29.161 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-09-13T03:03:29.161 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-09-13T03:03:29.162 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-09-13T03:03:29.162 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-09-13T03:03:29.162 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-09-13T03:03:29.162 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-09-13T03:03:29.162 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-09-13T03:03:29.163 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-09-13T03:03:29.163 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-09-13T03:03:29.163 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-09-13T03:03:29.163 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.table_properties_collectors: 2023-09-13T03:03:29.163 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.inplace_update_support: 0 2023-09-13T03:03:29.164 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.inplace_update_num_locks: 10000 2023-09-13T03:03:29.164 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-09-13T03:03:29.164 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-09-13T03:03:29.164 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.memtable_huge_page_size: 0 2023-09-13T03:03:29.164 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.bloom_locality: 0 2023-09-13T03:03:29.165 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.max_successive_merges: 0 2023-09-13T03:03:29.165 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.optimize_filters_for_hits: 0 2023-09-13T03:03:29.165 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.paranoid_file_checks: 0 2023-09-13T03:03:29.165 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.force_consistency_checks: 1 2023-09-13T03:03:29.165 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.report_bg_io_stats: 0 2023-09-13T03:03:29.166 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.ttl: 2592000 2023-09-13T03:03:29.166 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.periodic_compaction_seconds: 0 2023-09-13T03:03:29.166 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-09-13T03:03:29.166 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-09-13T03:03:29.166 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.enable_blob_files: false 2023-09-13T03:03:29.167 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.min_blob_size: 0 2023-09-13T03:03:29.167 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.blob_file_size: 268435456 2023-09-13T03:03:29.167 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.blob_compression_type: NoCompression 2023-09-13T03:03:29.167 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.enable_blob_garbage_collection: false 2023-09-13T03:03:29.167 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-09-13T03:03:29.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-09-13T03:03:29.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-09-13T03:03:29.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.blob_file_starting_level: 0 2023-09-13T03:03:29.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-09-13T03:03:29.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-09-13T03:03:29.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-09-13T03:03:29.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: d93d9870-9620-4f64-b723-97a31a2398cf 2023-09-13T03:03:29.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: EVENT_LOG_v1 {"time_micros": 1694574209054599, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-09-13T03:03:29.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-09-13T03:03:29.170 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: EVENT_LOG_v1 {"time_micros": 1694574209057370, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 86101, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 276, "table_properties": {"data_size": 84189, "index_size": 258, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11427, "raw_average_key_size": 48, "raw_value_size": 77765, "raw_average_value_size": 330, "num_data_blocks": 11, "num_entries": 235, "num_filter_entries": 235, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1694574209, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d93d9870-9620-4f64-b723-97a31a2398cf", "db_session_id": "UEB6MG67CB98OBS5VZWV", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-09-13T03:03:29.170 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: EVENT_LOG_v1 {"time_micros": 1694574209057565, "job": 1, "event": "recovery_finished"} 2023-09-13T03:03:29.170 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-09-13T03:03:29.170 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-09-13T03:03:29.170 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x559a94f84000 2023-09-13T03:03:29.171 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: DB pointer 0x559a94f70000 2023-09-13T03:03:29.171 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-09-13T03:03:29.171 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-09-13T03:03:29.171 INFO:journalctl@ceph.mon.a.smithi064.stdout: ** DB Stats ** 2023-09-13T03:03:29.171 INFO:journalctl@ceph.mon.a.smithi064.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-13T03:03:29.172 INFO:journalctl@ceph.mon.a.smithi064.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-09-13T03:03:29.172 INFO:journalctl@ceph.mon.a.smithi064.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-13T03:03:29.172 INFO:journalctl@ceph.mon.a.smithi064.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-13T03:03:29.172 INFO:journalctl@ceph.mon.a.smithi064.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-09-13T03:03:29.172 INFO:journalctl@ceph.mon.a.smithi064.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-13T03:03:29.173 INFO:journalctl@ceph.mon.a.smithi064.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-13T03:03:29.173 INFO:journalctl@ceph.mon.a.smithi064.stdout: 2023-09-13T03:03:29.173 INFO:journalctl@ceph.mon.a.smithi064.stdout: ** Compaction Stats [default] ** 2023-09-13T03:03:29.173 INFO:journalctl@ceph.mon.a.smithi064.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-13T03:03:29.173 INFO:journalctl@ceph.mon.a.smithi064.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-09-13T03:03:29.174 INFO:journalctl@ceph.mon.a.smithi064.stdout: L0 2/0 85.92 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 46.2 0.00 0.00 1 0.002 0 0 0.0 0.0 2023-09-13T03:03:29.174 INFO:journalctl@ceph.mon.a.smithi064.stdout: Sum 2/0 85.92 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 46.2 0.00 0.00 1 0.002 0 0 0.0 0.0 2023-09-13T03:03:29.174 INFO:journalctl@ceph.mon.a.smithi064.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 46.2 0.00 0.00 1 0.002 0 0 0.0 0.0 2023-09-13T03:03:29.174 INFO:journalctl@ceph.mon.a.smithi064.stdout: 2023-09-13T03:03:29.175 INFO:journalctl@ceph.mon.a.smithi064.stdout: ** Compaction Stats [default] ** 2023-09-13T03:03:29.175 INFO:journalctl@ceph.mon.a.smithi064.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-13T03:03:29.175 INFO:journalctl@ceph.mon.a.smithi064.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-09-13T03:03:29.175 INFO:journalctl@ceph.mon.a.smithi064.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 46.2 0.00 0.00 1 0.002 0 0 0.0 0.0 2023-09-13T03:03:29.175 INFO:journalctl@ceph.mon.a.smithi064.stdout: 2023-09-13T03:03:29.176 INFO:journalctl@ceph.mon.a.smithi064.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-09-13T03:03:29.176 INFO:journalctl@ceph.mon.a.smithi064.stdout: 2023-09-13T03:03:29.176 INFO:journalctl@ceph.mon.a.smithi064.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-13T03:03:29.176 INFO:journalctl@ceph.mon.a.smithi064.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-09-13T03:03:29.177 INFO:journalctl@ceph.mon.a.smithi064.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-09-13T03:03:29.178 INFO:journalctl@ceph.mon.a.smithi064.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-09-13T03:03:29.178 INFO:journalctl@ceph.mon.a.smithi064.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-09-13T03:03:29.179 INFO:journalctl@ceph.mon.a.smithi064.stdout: AddFile(Keys): cumulative 0, interval 0 2023-09-13T03:03:29.179 INFO:journalctl@ceph.mon.a.smithi064.stdout: Cumulative compaction: 0.00 GB write, 8.23 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-13T03:03:29.179 INFO:journalctl@ceph.mon.a.smithi064.stdout: Interval compaction: 0.00 GB write, 8.23 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-13T03:03:29.179 INFO:journalctl@ceph.mon.a.smithi064.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-09-13T03:03:29.179 INFO:journalctl@ceph.mon.a.smithi064.stdout: Block cache BinnedLRUCache@0x559a94ee91f0#7 capacity: 512.00 MB usage: 1.30 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2023-09-13T03:03:29.180 INFO:journalctl@ceph.mon.a.smithi064.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.83 KB,0.000157952%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2023-09-13T03:03:29.180 INFO:journalctl@ceph.mon.a.smithi064.stdout: 2023-09-13T03:03:29.180 INFO:journalctl@ceph.mon.a.smithi064.stdout: ** File Read Latency Histogram By Level [default] ** 2023-09-13T03:03:29.180 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: starting mon.a rank 0 at public addrs [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] at bind addrs [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da 2023-09-13T03:03:29.180 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: mon.a@-1(???) e1 preinit fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da 2023-09-13T03:03:29.181 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: mon.a@-1(???).mds e1 new map 2023-09-13T03:03:29.182 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: mon.a@-1(???).mds e1 print_map 2023-09-13T03:03:29.182 INFO:journalctl@ceph.mon.a.smithi064.stdout: e1 2023-09-13T03:03:29.182 INFO:journalctl@ceph.mon.a.smithi064.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-09-13T03:03:29.182 INFO:journalctl@ceph.mon.a.smithi064.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-13T03:03:29.183 INFO:journalctl@ceph.mon.a.smithi064.stdout: legacy client fscid: -1 2023-09-13T03:03:29.183 INFO:journalctl@ceph.mon.a.smithi064.stdout: 2023-09-13T03:03:29.183 INFO:journalctl@ceph.mon.a.smithi064.stdout: No filesystems configured 2023-09-13T03:03:29.183 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-09-13T03:03:29.183 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-09-13T03:03:29.184 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-09-13T03:03:29.184 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-09-13T03:03:29.184 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-09-13T03:03:29.184 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: expand_channel_meta expand map: {default=false} 2023-09-13T03:03:29.187 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: expand_channel_meta from 'false' to 'false' 2023-09-13T03:03:29.187 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: expand_channel_meta expanded map: {default=false} 2023-09-13T03:03:29.188 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: expand_channel_meta expand map: {default=info} 2023-09-13T03:03:29.188 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: expand_channel_meta from 'info' to 'info' 2023-09-13T03:03:29.188 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: expand_channel_meta expanded map: {default=info} 2023-09-13T03:03:29.188 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: expand_channel_meta expand map: {default=daemon} 2023-09-13T03:03:29.188 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: expand_channel_meta from 'daemon' to 'daemon' 2023-09-13T03:03:29.189 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: expand_channel_meta expanded map: {default=daemon} 2023-09-13T03:03:29.189 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: expand_channel_meta expand map: {default=debug} 2023-09-13T03:03:29.189 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: expand_channel_meta from 'debug' to 'debug' 2023-09-13T03:03:29.189 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: expand_channel_meta expanded map: {default=debug} 2023-09-13T03:03:29.190 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 systemd[1]: Started Ceph mon.a for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da. 2023-09-13T03:03:29.190 INFO:teuthology.orchestra.run.smithi064.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-09-13T03:03:29.372 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: mon.a is new leader, mons a in quorum (ranks 0) 2023-09-13T03:03:29.372 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: monmap e1: 1 mons at {a=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0]} removed_ranks: {} 2023-09-13T03:03:29.373 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: fsmap 2023-09-13T03:03:29.373 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: osdmap e1: 0 total, 0 up, 0 in 2023-09-13T03:03:29.373 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:29 smithi064 ceph-mon[97675]: mgrmap e1: no daemons active 2023-09-13T03:03:30.105 INFO:teuthology.orchestra.run.smithi064.stdout:Wrote config to /etc/ceph/ceph.conf 2023-09-13T03:03:30.130 INFO:teuthology.orchestra.run.smithi064.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-09-13T03:03:30.130 INFO:teuthology.orchestra.run.smithi064.stdout:Creating mgr... 2023-09-13T03:03:30.130 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying port 0.0.0.0:9283 ... 2023-09-13T03:03:30.131 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying port 0.0.0.0:8765 ... 2023-09-13T03:03:30.507 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mgr.x 2023-09-13T03:03:30.508 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Failed to reset failed state of unit ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mgr.x.service: Unit ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mgr.x.service not loaded. 2023-09-13T03:03:30.717 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da.target.wants/ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mgr.x.service → /etc/systemd/system/ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@.service. 2023-09-13T03:03:30.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:30 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/345844894' entity='client.admin' 2023-09-13T03:03:30.918 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:30 smithi064 systemd[1]: Starting Ceph mgr.x for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:03:31.500 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:31 smithi064 podman[97906]: 2023-09-13T03:03:31.501 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:31 smithi064 bash[97906]: b19027cebd1d683226f897c2eafc964c70efb3ff3caeb7f8f9b511cc08528a31 2023-09-13T03:03:31.524 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-09-13T03:03:31.524 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout disabled 2023-09-13T03:03:31.541 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-09-13T03:03:31.541 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2023-09-13T03:03:31.541 INFO:teuthology.orchestra.run.smithi064.stdout:firewalld.service is not enabled 2023-09-13T03:03:31.541 INFO:teuthology.orchestra.run.smithi064.stdout:Not possible to enable service . firewalld.service is not available 2023-09-13T03:03:31.559 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-09-13T03:03:31.559 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout disabled 2023-09-13T03:03:31.576 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-09-13T03:03:31.576 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2023-09-13T03:03:31.576 INFO:teuthology.orchestra.run.smithi064.stdout:firewalld.service is not enabled 2023-09-13T03:03:31.576 INFO:teuthology.orchestra.run.smithi064.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-09-13T03:03:31.577 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mgr to start... 2023-09-13T03:03:31.577 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mgr... 2023-09-13T03:03:31.848 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:31 smithi064 systemd[1]: Started Ceph mgr.x for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da. 2023-09-13T03:03:31.849 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:31 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:31.611+0000 7f9b3ed98200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-13T03:03:31.849 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:31 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:31.723+0000 7f9b3ed98200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-13T03:03:31.849 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:31 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:31.848+0000 7f9b3ed98200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-13T03:03:32.168 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:31 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:31.977+0000 7f9b3ed98200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-13T03:03:32.560 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2023-09-13T03:03:32.563 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2023-09-13T03:03:32.563 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsid": "f8dbced4-51e1-11ee-9ab7-7b867c8bd7da", 2023-09-13T03:03:32.563 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "health": { 2023-09-13T03:03:32.564 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-09-13T03:03:32.564 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-09-13T03:03:32.564 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-09-13T03:03:32.564 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-09-13T03:03:32.564 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-09-13T03:03:32.565 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-09-13T03:03:32.565 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 0 2023-09-13T03:03:32.565 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2023-09-13T03:03:32.565 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-09-13T03:03:32.565 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "a" 2023-09-13T03:03:32.566 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2023-09-13T03:03:32.566 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2023-09-13T03:03:32.566 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "monmap": { 2023-09-13T03:03:32.566 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-13T03:03:32.566 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-09-13T03:03:32.567 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-09-13T03:03:32.567 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-09-13T03:03:32.567 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-09-13T03:03:32.567 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-13T03:03:32.567 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-09-13T03:03:32.568 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-09-13T03:03:32.568 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-09-13T03:03:32.568 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-09-13T03:03:32.571 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-09-13T03:03:32.571 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-09-13T03:03:32.572 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-09-13T03:03:32.572 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-09-13T03:03:32.572 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-09-13T03:03:32.572 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-09-13T03:03:32.572 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-09-13T03:03:32.573 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-09-13T03:03:32.573 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-09-13T03:03:32.573 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-09-13T03:03:32.573 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-09-13T03:03:32.573 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-09-13T03:03:32.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-09-13T03:03:32.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-09-13T03:03:32.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-13T03:03:32.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-09-13T03:03:32.574 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-09-13T03:03:32.575 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-09-13T03:03:32.575 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-09-13T03:03:32.575 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": false, 2023-09-13T03:03:32.575 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-09-13T03:03:32.575 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modules": [ 2023-09-13T03:03:32.576 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "iostat", 2023-09-13T03:03:32.576 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "nfs", 2023-09-13T03:03:32.576 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "restful" 2023-09-13T03:03:32.576 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2023-09-13T03:03:32.576 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-13T03:03:32.577 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-09-13T03:03:32.577 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-09-13T03:03:32.577 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-13T03:03:32.577 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modified": "2023-09-13T03:03:24.167702+0000", 2023-09-13T03:03:32.577 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-13T03:03:32.577 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-09-13T03:03:32.578 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-09-13T03:03:32.578 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2023-09-13T03:03:32.578 INFO:teuthology.orchestra.run.smithi064.stdout:mgr not available, waiting (1/15)... 2023-09-13T03:03:32.579 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:32 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2756355111' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-09-13T03:03:32.579 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:32 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:32.213+0000 7f9b3ed98200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-13T03:03:32.883 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:32 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:32.715+0000 7f9b3ed98200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-13T03:03:33.168 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:32 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:32.882+0000 7f9b3ed98200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-13T03:03:33.168 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:33 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:33.002+0000 7f9b3ed98200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-13T03:03:33.168 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:33 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:33.130+0000 7f9b3ed98200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-13T03:03:34.168 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:33 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:33.690+0000 7f9b3ed98200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-13T03:03:34.918 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:34 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:34.608+0000 7f9b3ed98200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-13T03:03:34.918 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:34 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:34.727+0000 7f9b3ed98200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-13T03:03:35.652 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2023-09-13T03:03:35.653 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2023-09-13T03:03:35.653 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsid": "f8dbced4-51e1-11ee-9ab7-7b867c8bd7da", 2023-09-13T03:03:35.653 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "health": { 2023-09-13T03:03:35.653 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-09-13T03:03:35.653 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-09-13T03:03:35.654 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-09-13T03:03:35.654 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-09-13T03:03:35.654 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-09-13T03:03:35.654 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-09-13T03:03:35.654 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 0 2023-09-13T03:03:35.655 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2023-09-13T03:03:35.655 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-09-13T03:03:35.655 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "a" 2023-09-13T03:03:35.655 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2023-09-13T03:03:35.655 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2023-09-13T03:03:35.656 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "monmap": { 2023-09-13T03:03:35.656 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-13T03:03:35.656 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-09-13T03:03:35.656 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-09-13T03:03:35.656 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-09-13T03:03:35.657 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-09-13T03:03:35.657 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-13T03:03:35.657 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-09-13T03:03:35.657 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-09-13T03:03:35.657 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-09-13T03:03:35.658 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-09-13T03:03:35.661 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-09-13T03:03:35.661 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-09-13T03:03:35.661 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-09-13T03:03:35.661 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-09-13T03:03:35.662 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-09-13T03:03:35.662 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-09-13T03:03:35.662 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-09-13T03:03:35.662 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-09-13T03:03:35.663 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-09-13T03:03:35.663 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-09-13T03:03:35.663 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-09-13T03:03:35.663 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-09-13T03:03:35.663 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-09-13T03:03:35.664 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-09-13T03:03:35.664 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-13T03:03:35.664 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-09-13T03:03:35.664 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-09-13T03:03:35.664 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-09-13T03:03:35.665 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-09-13T03:03:35.665 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": false, 2023-09-13T03:03:35.665 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-09-13T03:03:35.665 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modules": [ 2023-09-13T03:03:35.665 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "iostat", 2023-09-13T03:03:35.665 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "nfs", 2023-09-13T03:03:35.666 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "restful" 2023-09-13T03:03:35.666 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2023-09-13T03:03:35.666 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-13T03:03:35.666 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-09-13T03:03:35.667 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-09-13T03:03:35.667 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-13T03:03:35.667 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modified": "2023-09-13T03:03:24.167702+0000", 2023-09-13T03:03:35.667 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-13T03:03:35.667 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-09-13T03:03:35.667 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-09-13T03:03:35.668 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2023-09-13T03:03:35.668 INFO:teuthology.orchestra.run.smithi064.stdout:mgr not available, waiting (2/15)... 2023-09-13T03:03:35.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:35 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/1078176491' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-09-13T03:03:36.110 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:35 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:35.757+0000 7f9b3ed98200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-13T03:03:36.110 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:35 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:35.880+0000 7f9b3ed98200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-13T03:03:36.418 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:36 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:36.109+0000 7f9b3ed98200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-13T03:03:36.785 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:36 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:36.483+0000 7f9b3ed98200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-13T03:03:37.168 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:36 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:36.785+0000 7f9b3ed98200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-13T03:03:37.168 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:36 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:36.984+0000 7f9b3ed98200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-13T03:03:37.668 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:37 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:37.338+0000 7f9b3ed98200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-13T03:03:38.668 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:38 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:38.299+0000 7f9b3ed98200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-13T03:03:38.668 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:38 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:38.481+0000 7f9b3ed98200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-13T03:03:39.418 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:38 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:38.737+0000 7f9b3ed98200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-13T03:03:39.418 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:38 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:38.888+0000 7f9b3ed98200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-13T03:03:39.418 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:39 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:39.174+0000 7f9b3ed98200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-13T03:03:40.668 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:40 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:40.283+0000 7f9b3ed98200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-13T03:03:40.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:40 smithi064 ceph-mon[97675]: Activating manager daemon x 2023-09-13T03:03:40.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:40 smithi064 ceph-mon[97675]: mgrmap e2: x(active, starting, since 0.00436443s) 2023-09-13T03:03:40.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:40 smithi064 ceph-mon[97675]: from='mgr.14100 172.21.15.64:0/4232124918' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-13T03:03:40.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:40 smithi064 ceph-mon[97675]: from='mgr.14100 172.21.15.64:0/4232124918' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-13T03:03:40.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:40 smithi064 ceph-mon[97675]: from='mgr.14100 172.21.15.64:0/4232124918' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-13T03:03:40.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:40 smithi064 ceph-mon[97675]: from='mgr.14100 172.21.15.64:0/4232124918' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-13T03:03:40.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:40 smithi064 ceph-mon[97675]: from='mgr.14100 172.21.15.64:0/4232124918' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-13T03:03:40.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:40 smithi064 ceph-mon[97675]: Manager daemon x is now available 2023-09-13T03:03:41.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:41 smithi064 ceph-mon[97675]: from='mgr.14100 172.21.15.64:0/4232124918' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-13T03:03:41.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:41 smithi064 ceph-mon[97675]: from='mgr.14100 172.21.15.64:0/4232124918' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-13T03:03:41.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:41 smithi064 ceph-mon[97675]: from='mgr.14100 172.21.15.64:0/4232124918' entity='mgr.x' 2023-09-13T03:03:41.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:41 smithi064 ceph-mon[97675]: from='mgr.14100 172.21.15.64:0/4232124918' entity='mgr.x' 2023-09-13T03:03:41.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:41 smithi064 ceph-mon[97675]: from='mgr.14100 172.21.15.64:0/4232124918' entity='mgr.x' 2023-09-13T03:03:41.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:41 smithi064 ceph-mon[97675]: mgrmap e3: x(active, since 1.00981s) 2023-09-13T03:03:43.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:43 smithi064 ceph-mon[97675]: mgrmap e4: x(active, since 2s) 2023-09-13T03:03:45.650 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2023-09-13T03:03:45.650 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2023-09-13T03:03:45.650 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsid": "f8dbced4-51e1-11ee-9ab7-7b867c8bd7da", 2023-09-13T03:03:45.651 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "health": { 2023-09-13T03:03:45.651 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-09-13T03:03:45.651 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-09-13T03:03:45.651 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-09-13T03:03:45.651 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-09-13T03:03:45.652 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-09-13T03:03:45.652 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-09-13T03:03:45.652 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 0 2023-09-13T03:03:45.652 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2023-09-13T03:03:45.652 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-09-13T03:03:45.653 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "a" 2023-09-13T03:03:45.653 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2023-09-13T03:03:45.653 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_age": 16, 2023-09-13T03:03:45.653 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "monmap": { 2023-09-13T03:03:45.654 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-13T03:03:45.654 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-09-13T03:03:45.654 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-09-13T03:03:45.654 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-09-13T03:03:45.654 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-09-13T03:03:45.655 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-13T03:03:45.655 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-09-13T03:03:45.655 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-09-13T03:03:45.655 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-09-13T03:03:45.655 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-09-13T03:03:45.659 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-09-13T03:03:45.659 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-09-13T03:03:45.659 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-09-13T03:03:45.659 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-09-13T03:03:45.660 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-09-13T03:03:45.660 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-09-13T03:03:45.660 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-09-13T03:03:45.660 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-09-13T03:03:45.660 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-09-13T03:03:45.661 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-09-13T03:03:45.661 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-09-13T03:03:45.661 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-09-13T03:03:45.661 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-09-13T03:03:45.661 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-09-13T03:03:45.662 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-13T03:03:45.662 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-09-13T03:03:45.662 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-09-13T03:03:45.662 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-09-13T03:03:45.662 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-09-13T03:03:45.663 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": true, 2023-09-13T03:03:45.663 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-09-13T03:03:45.663 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modules": [ 2023-09-13T03:03:45.663 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "iostat", 2023-09-13T03:03:45.663 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "nfs", 2023-09-13T03:03:45.664 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "restful" 2023-09-13T03:03:45.664 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2023-09-13T03:03:45.664 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-13T03:03:45.664 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-09-13T03:03:45.664 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-09-13T03:03:45.665 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-13T03:03:45.665 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modified": "2023-09-13T03:03:24.167702+0000", 2023-09-13T03:03:45.665 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-13T03:03:45.665 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2023-09-13T03:03:45.665 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-09-13T03:03:45.666 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2023-09-13T03:03:45.666 INFO:teuthology.orchestra.run.smithi064.stdout:mgr is available 2023-09-13T03:03:45.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:45 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/3596208123' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-09-13T03:03:46.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:46 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/354563584' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-09-13T03:03:46.728 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2023-09-13T03:03:46.728 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [global] 2023-09-13T03:03:46.729 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout fsid = f8dbced4-51e1-11ee-9ab7-7b867c8bd7da 2023-09-13T03:03:46.729 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-09-13T03:03:46.729 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-09-13T03:03:46.729 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-09-13T03:03:46.729 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-09-13T03:03:46.730 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2023-09-13T03:03:46.730 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [mgr] 2023-09-13T03:03:46.730 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-09-13T03:03:46.730 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2023-09-13T03:03:46.731 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [osd] 2023-09-13T03:03:46.731 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-09-13T03:03:46.731 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-09-13T03:03:46.731 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-09-13T03:03:46.731 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-09-13T03:03:46.732 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-09-13T03:03:46.732 INFO:teuthology.orchestra.run.smithi064.stdout:Enabling cephadm module... 2023-09-13T03:03:47.829 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:47 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2522799535' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-09-13T03:03:47.829 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:47 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: ignoring --setuser ceph since I am not root 2023-09-13T03:03:47.830 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:47 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: ignoring --setgroup ceph since I am not root 2023-09-13T03:03:47.830 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:47 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:47.828+0000 7f39a6af8200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-13T03:03:48.168 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:47 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:47.944+0000 7f39a6af8200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-13T03:03:48.168 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:48 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:48.073+0000 7f39a6af8200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-13T03:03:48.450 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:48 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:48.202+0000 7f39a6af8200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-13T03:03:48.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:48 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2522799535' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-09-13T03:03:48.763 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:48 smithi064 ceph-mon[97675]: mgrmap e5: x(active, since 7s) 2023-09-13T03:03:48.763 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:48 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:48.450+0000 7f39a6af8200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-13T03:03:49.108 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:48 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:48.938+0000 7f39a6af8200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-13T03:03:49.192 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2023-09-13T03:03:49.192 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-09-13T03:03:49.192 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": true, 2023-09-13T03:03:49.192 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-09-13T03:03:49.193 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-09-13T03:03:49.193 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2023-09-13T03:03:49.193 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for the mgr to restart... 2023-09-13T03:03:49.193 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mgr epoch 5... 2023-09-13T03:03:49.369 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:49 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:49.107+0000 7f39a6af8200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-13T03:03:49.370 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:49 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:49.230+0000 7f39a6af8200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-13T03:03:49.370 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:49 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:49.369+0000 7f39a6af8200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-13T03:03:49.637 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:49 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/4025875885' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-09-13T03:03:50.418 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:49 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:49.958+0000 7f39a6af8200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-13T03:03:51.168 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:50 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:50.838+0000 7f39a6af8200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-13T03:03:51.168 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:50 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:50.948+0000 7f39a6af8200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-13T03:03:52.292 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:51 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:51.938+0000 7f39a6af8200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-13T03:03:52.293 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:52 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:52.061+0000 7f39a6af8200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-13T03:03:52.666 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:52 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:52.291+0000 7f39a6af8200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-13T03:03:52.918 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:52 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:52.665+0000 7f39a6af8200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-13T03:03:53.418 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:52 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:52.947+0000 7f39a6af8200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-13T03:03:53.418 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:53 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:53.148+0000 7f39a6af8200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-13T03:03:53.918 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:53 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:53.500+0000 7f39a6af8200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-13T03:03:54.898 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:54 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:54.461+0000 7f39a6af8200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-13T03:03:54.899 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:54 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:54.643+0000 7f39a6af8200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-13T03:03:55.168 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:54 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:54.898+0000 7f39a6af8200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-13T03:03:55.168 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:55 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:55.050+0000 7f39a6af8200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-13T03:03:55.668 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:55 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:55.333+0000 7f39a6af8200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-13T03:03:56.918 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:03:56 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:03:56.439+0000 7f39a6af8200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-13T03:03:56.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:56 smithi064 ceph-mon[97675]: Active manager daemon x restarted 2023-09-13T03:03:56.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:56 smithi064 ceph-mon[97675]: Activating manager daemon x 2023-09-13T03:03:56.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:56 smithi064 ceph-mon[97675]: osdmap e2: 0 total, 0 up, 0 in 2023-09-13T03:03:56.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:56 smithi064 ceph-mon[97675]: mgrmap e6: x(active, starting, since 0.0039577s) 2023-09-13T03:03:56.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:56 smithi064 ceph-mon[97675]: from='mgr.14118 172.21.15.64:0/1600004161' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-13T03:03:56.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:56 smithi064 ceph-mon[97675]: from='mgr.14118 172.21.15.64:0/1600004161' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-13T03:03:56.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:56 smithi064 ceph-mon[97675]: from='mgr.14118 172.21.15.64:0/1600004161' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-13T03:03:56.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:56 smithi064 ceph-mon[97675]: from='mgr.14118 172.21.15.64:0/1600004161' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-13T03:03:56.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:56 smithi064 ceph-mon[97675]: from='mgr.14118 172.21.15.64:0/1600004161' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-13T03:03:56.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:56 smithi064 ceph-mon[97675]: Manager daemon x is now available 2023-09-13T03:03:56.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:56 smithi064 ceph-mon[97675]: from='mgr.14118 172.21.15.64:0/1600004161' entity='mgr.x' 2023-09-13T03:03:56.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:56 smithi064 ceph-mon[97675]: from='mgr.14118 172.21.15.64:0/1600004161' entity='mgr.x' 2023-09-13T03:03:56.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:56 smithi064 ceph-mon[97675]: from='mgr.14118 172.21.15.64:0/1600004161' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:03:56.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:56 smithi064 ceph-mon[97675]: from='mgr.14118 172.21.15.64:0/1600004161' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:03:57.878 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:57 smithi064 ceph-mon[97675]: Found migration_current of "None". Setting to last migration. 2023-09-13T03:03:57.879 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:57 smithi064 ceph-mon[97675]: from='mgr.14118 172.21.15.64:0/1600004161' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:03:57.879 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:57 smithi064 ceph-mon[97675]: from='mgr.14118 172.21.15.64:0/1600004161' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-13T03:03:57.879 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:57 smithi064 ceph-mon[97675]: from='mgr.14118 172.21.15.64:0/1600004161' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-13T03:03:57.879 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:57 smithi064 ceph-mon[97675]: from='mgr.14118 172.21.15.64:0/1600004161' entity='mgr.x' 2023-09-13T03:03:57.880 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:57 smithi064 ceph-mon[97675]: from='mgr.14118 172.21.15.64:0/1600004161' entity='mgr.x' 2023-09-13T03:03:57.880 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:57 smithi064 ceph-mon[97675]: mgrmap e7: x(active, since 1.00757s) 2023-09-13T03:03:57.884 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2023-09-13T03:03:57.884 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-09-13T03:03:57.885 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "initialized": true 2023-09-13T03:03:57.885 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2023-09-13T03:03:57.885 INFO:teuthology.orchestra.run.smithi064.stdout:mgr epoch 5 is available 2023-09-13T03:03:57.885 INFO:teuthology.orchestra.run.smithi064.stdout:Setting orchestrator backend to cephadm... 2023-09-13T03:03:58.754 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:58 smithi064 ceph-mon[97675]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-09-13T03:03:58.755 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:58 smithi064 ceph-mon[97675]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-09-13T03:03:59.787 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:59 smithi064 ceph-mon[97675]: [13/Sep/2023:03:03:58] ENGINE Bus STARTING 2023-09-13T03:03:59.787 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:59 smithi064 ceph-mon[97675]: [13/Sep/2023:03:03:58] ENGINE Serving on https://172.21.15.64:7150 2023-09-13T03:03:59.787 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:59 smithi064 ceph-mon[97675]: mgrmap e8: x(active, since 2s) 2023-09-13T03:03:59.788 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:59 smithi064 ceph-mon[97675]: [13/Sep/2023:03:03:58] ENGINE Serving on http://172.21.15.64:8765 2023-09-13T03:03:59.788 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:59 smithi064 ceph-mon[97675]: [13/Sep/2023:03:03:58] ENGINE Bus STARTED 2023-09-13T03:03:59.788 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:59 smithi064 ceph-mon[97675]: from='mgr.14118 172.21.15.64:0/1600004161' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:03:59.788 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:59 smithi064 ceph-mon[97675]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:03:59.788 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:59 smithi064 ceph-mon[97675]: from='mgr.14118 172.21.15.64:0/1600004161' entity='mgr.x' 2023-09-13T03:03:59.789 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:03:59 smithi064 ceph-mon[97675]: from='mgr.14118 172.21.15.64:0/1600004161' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:04:00.036 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout value unchanged 2023-09-13T03:04:00.036 INFO:teuthology.orchestra.run.smithi064.stdout:Generating ssh key... 2023-09-13T03:04:00.997 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:00 smithi064 ceph-mon[97675]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:04:01.412 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:00 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: Generating public/private rsa key pair. 2023-09-13T03:04:01.412 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:00 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: Your identification has been saved in /tmp/tmpjr7xd_om/key. 2023-09-13T03:04:01.413 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:00 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: Your public key has been saved in /tmp/tmpjr7xd_om/key.pub. 2023-09-13T03:04:01.413 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:00 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: The key fingerprint is: 2023-09-13T03:04:01.413 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:00 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: SHA256:53odq+xcaTVL+H66hjChKHe79iMN2Q1rX8Xj/diizIk ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da 2023-09-13T03:04:01.413 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:00 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: The key's randomart image is: 2023-09-13T03:04:01.413 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:00 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: +---[RSA 3072]----+ 2023-09-13T03:04:01.414 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:00 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: | | 2023-09-13T03:04:01.414 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:00 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: | | 2023-09-13T03:04:01.414 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:00 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: | . | 2023-09-13T03:04:01.414 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:00 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: | o . .o| 2023-09-13T03:04:01.414 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:00 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: | .S+.=. =.o| 2023-09-13T03:04:01.415 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:00 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: | . o =o* o=.+.| 2023-09-13T03:04:01.415 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:00 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: | o . =.=+=oo.| 2023-09-13T03:04:01.415 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:00 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: | +++*=o+ +| 2023-09-13T03:04:01.415 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:00 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: | .o*Eo=o+= | 2023-09-13T03:04:01.415 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:00 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: +----[SHA256]-----+ 2023-09-13T03:04:02.269 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:02 smithi064 ceph-mon[97675]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:04:02.269 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:02 smithi064 ceph-mon[97675]: Generating ssh key... 2023-09-13T03:04:02.270 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:02 smithi064 ceph-mon[97675]: from='mgr.14118 172.21.15.64:0/1600004161' entity='mgr.x' 2023-09-13T03:04:02.270 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:02 smithi064 ceph-mon[97675]: from='mgr.14118 172.21.15.64:0/1600004161' entity='mgr.x' 2023-09-13T03:04:02.489 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtlx9VL3WXVuJralvCTbaJpxSzYKU1eOyKFpdY9Zcjpx6WEq7QMEdzMcQ5csSXLy9PSnaBEHAHpq8lSgw9416Fgjew4AC4bmIDGeiwe2BM8A60ZMhepl0BbrDHZ/5pNwhJ4KBVph0R9DX6ztNSoRQclainsLNt+vTSZHGIeMIxidHExBEs3dlXPF63eppcEWRR93hqIr7lSeSUFqm5Nv47ic7/CKc3PUPL4XhfZsEQdEt4CCm6n0tjQ5g+/S4VDs1yyuChu9cPndcubHd/JiWFB52rxmhtxLtNXknKqXwsfRDNYmKd01L9vyEd87e+aBa3bdpgMLMB973u53OqDO0r5rG6neqbgT9ZrdCD3JtIXFM/MyZhbIBqHIJHrbiITFeE7kt60WeX+TnDbQzwr3aUw5s+xJN8/E2K3k2m2ki4N2/qjlQaleNaHpRihdFfbjUiiviY0VMwS6e0Z0WPegfPhFZwFimqYwPmPHe7E1BgzsjKH/ZPvZFGvDqqlYFXA/k= ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da 2023-09-13T03:04:02.489 INFO:teuthology.orchestra.run.smithi064.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-09-13T03:04:02.490 INFO:teuthology.orchestra.run.smithi064.stdout:Adding key to root@localhost authorized_keys... 2023-09-13T03:04:02.491 INFO:teuthology.orchestra.run.smithi064.stdout:Adding host smithi064... 2023-09-13T03:04:03.263 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:03 smithi064 ceph-mon[97675]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:04:04.311 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:04 smithi064 ceph-mon[97675]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi064", "addr": "172.21.15.64", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:04:05.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:05 smithi064 ceph-mon[97675]: Deploying cephadm binary to smithi064 2023-09-13T03:04:06.883 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout Added host 'smithi064' with addr '172.21.15.64' 2023-09-13T03:04:06.883 INFO:teuthology.orchestra.run.smithi064.stdout:Deploying unmanaged mon service... 2023-09-13T03:04:07.819 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:07 smithi064 ceph-mon[97675]: from='mgr.14118 172.21.15.64:0/1600004161' entity='mgr.x' 2023-09-13T03:04:07.819 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:07 smithi064 ceph-mon[97675]: Added host smithi064 2023-09-13T03:04:07.819 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:07 smithi064 ceph-mon[97675]: from='mgr.14118 172.21.15.64:0/1600004161' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:04:08.129 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-09-13T03:04:08.129 INFO:teuthology.orchestra.run.smithi064.stdout:Deploying unmanaged mgr service... 2023-09-13T03:04:08.819 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:08 smithi064 ceph-mon[97675]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:04:08.819 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:08 smithi064 ceph-mon[97675]: Saving service mon spec with placement count:5 2023-09-13T03:04:08.820 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:08 smithi064 ceph-mon[97675]: from='mgr.14118 172.21.15.64:0/1600004161' entity='mgr.x' 2023-09-13T03:04:09.460 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-09-13T03:04:10.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:09 smithi064 ceph-mon[97675]: from='mgr.14118 172.21.15.64:0/1600004161' entity='mgr.x' 2023-09-13T03:04:10.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:09 smithi064 ceph-mon[97675]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:04:10.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:09 smithi064 ceph-mon[97675]: Saving service mgr spec with placement count:2 2023-09-13T03:04:10.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:09 smithi064 ceph-mon[97675]: from='mgr.14118 172.21.15.64:0/1600004161' entity='mgr.x' 2023-09-13T03:04:10.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:09 smithi064 ceph-mon[97675]: from='mgr.14118 172.21.15.64:0/1600004161' entity='mgr.x' 2023-09-13T03:04:11.534 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:11 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/881088685' entity='client.admin' 2023-09-13T03:04:11.826 INFO:teuthology.orchestra.run.smithi064.stdout:Enabling the dashboard module... 2023-09-13T03:04:12.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:12 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2411234822' entity='client.admin' 2023-09-13T03:04:12.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:12 smithi064 ceph-mon[97675]: from='mgr.14118 172.21.15.64:0/1600004161' entity='mgr.x' 2023-09-13T03:04:13.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:13 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2843805038' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-09-13T03:04:13.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:13 smithi064 ceph-mon[97675]: from='mgr.14118 172.21.15.64:0/1600004161' entity='mgr.x' 2023-09-13T03:04:13.670 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:13 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: ignoring --setuser ceph since I am not root 2023-09-13T03:04:13.670 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:13 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: ignoring --setgroup ceph since I am not root 2023-09-13T03:04:14.100 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:13 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:04:13.727+0000 7fd1a87c6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-13T03:04:14.100 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:13 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:04:13.838+0000 7fd1a87c6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-13T03:04:14.100 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:13 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:04:13.970+0000 7fd1a87c6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-13T03:04:14.352 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:14 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:04:14.099+0000 7fd1a87c6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-13T03:04:14.353 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:14 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:04:14.352+0000 7fd1a87c6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-13T03:04:14.618 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:14 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2843805038' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-09-13T03:04:14.618 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:14 smithi064 ceph-mon[97675]: mgrmap e9: x(active, since 16s) 2023-09-13T03:04:15.168 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:14 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:04:14.875+0000 7fd1a87c6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-13T03:04:15.168 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:15 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:04:15.048+0000 7fd1a87c6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-13T03:04:15.294 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2023-09-13T03:04:15.294 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-09-13T03:04:15.294 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": true, 2023-09-13T03:04:15.294 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-09-13T03:04:15.295 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-09-13T03:04:15.295 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2023-09-13T03:04:15.295 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for the mgr to restart... 2023-09-13T03:04:15.295 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mgr epoch 9... 2023-09-13T03:04:15.583 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:15 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/229921685' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-09-13T03:04:15.583 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:15 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:04:15.168+0000 7fd1a87c6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-13T03:04:15.583 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:15 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:04:15.302+0000 7fd1a87c6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-13T03:04:16.168 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:15 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:04:15.889+0000 7fd1a87c6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-13T03:04:17.168 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:16 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:04:16.765+0000 7fd1a87c6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-13T03:04:17.168 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:16 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:04:16.874+0000 7fd1a87c6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-13T03:04:18.168 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:17 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:04:17.856+0000 7fd1a87c6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-13T03:04:18.168 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:17 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:04:17.980+0000 7fd1a87c6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-13T03:04:18.585 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:18 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:04:18.211+0000 7fd1a87c6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-13T03:04:18.867 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:18 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:04:18.584+0000 7fd1a87c6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-13T03:04:19.168 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:18 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:04:18.866+0000 7fd1a87c6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-13T03:04:19.168 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:19 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:04:19.065+0000 7fd1a87c6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-13T03:04:19.918 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:19 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:04:19.418+0000 7fd1a87c6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-13T03:04:20.668 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:20 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:04:20.375+0000 7fd1a87c6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-13T03:04:20.668 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:20 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:04:20.556+0000 7fd1a87c6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-13T03:04:21.168 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:20 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:04:20.811+0000 7fd1a87c6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-13T03:04:21.168 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:20 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:04:20.963+0000 7fd1a87c6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-13T03:04:21.668 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:21 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:04:21.248+0000 7fd1a87c6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-13T03:04:22.668 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:04:22 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:04:22.353+0000 7fd1a87c6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-13T03:04:22.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:22 smithi064 ceph-mon[97675]: Active manager daemon x restarted 2023-09-13T03:04:22.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:22 smithi064 ceph-mon[97675]: Activating manager daemon x 2023-09-13T03:04:22.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:22 smithi064 ceph-mon[97675]: osdmap e3: 0 total, 0 up, 0 in 2023-09-13T03:04:22.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:22 smithi064 ceph-mon[97675]: mgrmap e10: x(active, starting, since 0.00537459s) 2023-09-13T03:04:22.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:22 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-13T03:04:22.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:22 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-13T03:04:22.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:22 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-13T03:04:22.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:22 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-13T03:04:22.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:22 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-13T03:04:22.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:22 smithi064 ceph-mon[97675]: Manager daemon x is now available 2023-09-13T03:04:22.671 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:22 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:04:22.671 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:22 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:04:23.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:23 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-13T03:04:23.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:23 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-13T03:04:23.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:23 smithi064 ceph-mon[97675]: mgrmap e11: x(active, since 1.00799s) 2023-09-13T03:04:23.685 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2023-09-13T03:04:23.686 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-09-13T03:04:23.686 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "initialized": true 2023-09-13T03:04:23.687 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2023-09-13T03:04:23.687 INFO:teuthology.orchestra.run.smithi064.stdout:mgr epoch 9 is available 2023-09-13T03:04:23.687 INFO:teuthology.orchestra.run.smithi064.stdout:Generating a dashboard self-signed certificate... 2023-09-13T03:04:24.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:24 smithi064 ceph-mon[97675]: [13/Sep/2023:03:04:23] ENGINE Bus STARTING 2023-09-13T03:04:24.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:24 smithi064 ceph-mon[97675]: [13/Sep/2023:03:04:23] ENGINE Serving on https://172.21.15.64:7150 2023-09-13T03:04:24.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:24 smithi064 ceph-mon[97675]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-09-13T03:04:24.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:24 smithi064 ceph-mon[97675]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-09-13T03:04:24.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:24 smithi064 ceph-mon[97675]: [13/Sep/2023:03:04:23] ENGINE Serving on http://172.21.15.64:8765 2023-09-13T03:04:24.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:24 smithi064 ceph-mon[97675]: [13/Sep/2023:03:04:23] ENGINE Bus STARTED 2023-09-13T03:04:24.967 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-09-13T03:04:24.967 INFO:teuthology.orchestra.run.smithi064.stdout:Creating initial admin user... 2023-09-13T03:04:25.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:25 smithi064 ceph-mon[97675]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:04:25.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:25 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:04:25.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:25 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:04:25.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:25 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:04:25.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:25 smithi064 ceph-mon[97675]: mgrmap e12: x(active, since 2s) 2023-09-13T03:04:26.347 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$n4QTROG7PUks4cFPiKte2e6johNnP07Z69kXlpcSlFxmLvpC.pWCm", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1694574266, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-09-13T03:04:26.347 INFO:teuthology.orchestra.run.smithi064.stdout:Fetching dashboard port number... 2023-09-13T03:04:27.321 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:27 smithi064 ceph-mon[97675]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:04:27.322 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:27 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:04:27.322 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:27 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:04:27.634 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 8443 2023-09-13T03:04:27.656 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-09-13T03:04:27.657 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout disabled 2023-09-13T03:04:27.676 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-09-13T03:04:27.676 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2023-09-13T03:04:27.676 INFO:teuthology.orchestra.run.smithi064.stdout:firewalld.service is not enabled 2023-09-13T03:04:27.677 INFO:teuthology.orchestra.run.smithi064.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-09-13T03:04:27.679 INFO:teuthology.orchestra.run.smithi064.stdout:Ceph Dashboard is now available at: 2023-09-13T03:04:27.679 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:04:27.679 INFO:teuthology.orchestra.run.smithi064.stdout: URL: https://smithi064.front.sepia.ceph.com:8443/ 2023-09-13T03:04:27.680 INFO:teuthology.orchestra.run.smithi064.stdout: User: admin 2023-09-13T03:04:27.680 INFO:teuthology.orchestra.run.smithi064.stdout: Password: uqqhcggx97 2023-09-13T03:04:27.680 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:04:27.680 INFO:teuthology.orchestra.run.smithi064.stdout:Saving cluster configuration to /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config directory 2023-09-13T03:04:27.681 INFO:teuthology.orchestra.run.smithi064.stdout:Enabling autotune for osd_memory_target 2023-09-13T03:04:28.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:28 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/4242906277' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-09-13T03:04:30.388 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-09-13T03:04:30.388 INFO:teuthology.orchestra.run.smithi064.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-09-13T03:04:30.388 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:04:30.389 INFO:teuthology.orchestra.run.smithi064.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-09-13T03:04:30.389 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:04:30.389 INFO:teuthology.orchestra.run.smithi064.stdout:Or, if you are only running a single cluster on this host: 2023-09-13T03:04:30.389 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:04:30.389 INFO:teuthology.orchestra.run.smithi064.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-09-13T03:04:30.390 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:04:30.390 INFO:teuthology.orchestra.run.smithi064.stdout:Please consider enabling telemetry to help improve Ceph: 2023-09-13T03:04:30.390 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:04:30.390 INFO:teuthology.orchestra.run.smithi064.stdout: ceph telemetry on 2023-09-13T03:04:30.391 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:04:30.391 INFO:teuthology.orchestra.run.smithi064.stdout:For more information see: 2023-09-13T03:04:30.391 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:04:30.391 INFO:teuthology.orchestra.run.smithi064.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-09-13T03:04:30.392 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:04:30.392 INFO:teuthology.orchestra.run.smithi064.stdout:Bootstrap complete. 2023-09-13T03:04:30.411 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:30 smithi064 ceph-mon[97675]: mgrmap e13: x(active, since 6s) 2023-09-13T03:04:30.412 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:30 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/3739644736' entity='client.admin' 2023-09-13T03:04:30.447 INFO:tasks.cephadm:Fetching config... 2023-09-13T03:04:30.448 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-13T03:04:30.448 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-09-13T03:04:30.494 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-09-13T03:04:30.494 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-13T03:04:30.495 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-09-13T03:04:30.577 INFO:tasks.cephadm:Fetching mon keyring... 2023-09-13T03:04:30.578 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-13T03:04:30.578 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/keyring of=/dev/stdout 2023-09-13T03:04:30.688 INFO:tasks.cephadm:Fetching pub ssh key... 2023-09-13T03:04:30.688 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-13T03:04:30.689 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-09-13T03:04:30.771 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-09-13T03:04:30.772 DEBUG:teuthology.orchestra.run.smithi064:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtlx9VL3WXVuJralvCTbaJpxSzYKU1eOyKFpdY9Zcjpx6WEq7QMEdzMcQ5csSXLy9PSnaBEHAHpq8lSgw9416Fgjew4AC4bmIDGeiwe2BM8A60ZMhepl0BbrDHZ/5pNwhJ4KBVph0R9DX6ztNSoRQclainsLNt+vTSZHGIeMIxidHExBEs3dlXPF63eppcEWRR93hqIr7lSeSUFqm5Nv47ic7/CKc3PUPL4XhfZsEQdEt4CCm6n0tjQ5g+/S4VDs1yyuChu9cPndcubHd/JiWFB52rxmhtxLtNXknKqXwsfRDNYmKd01L9vyEd87e+aBa3bdpgMLMB973u53OqDO0r5rG6neqbgT9ZrdCD3JtIXFM/MyZhbIBqHIJHrbiITFeE7kt60WeX+TnDbQzwr3aUw5s+xJN8/E2K3k2m2ki4N2/qjlQaleNaHpRihdFfbjUiiviY0VMwS6e0Z0WPegfPhFZwFimqYwPmPHe7E1BgzsjKH/ZPvZFGvDqqlYFXA/k= ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-09-13T03:04:30.915 INFO:teuthology.orchestra.run.smithi064.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtlx9VL3WXVuJralvCTbaJpxSzYKU1eOyKFpdY9Zcjpx6WEq7QMEdzMcQ5csSXLy9PSnaBEHAHpq8lSgw9416Fgjew4AC4bmIDGeiwe2BM8A60ZMhepl0BbrDHZ/5pNwhJ4KBVph0R9DX6ztNSoRQclainsLNt+vTSZHGIeMIxidHExBEs3dlXPF63eppcEWRR93hqIr7lSeSUFqm5Nv47ic7/CKc3PUPL4XhfZsEQdEt4CCm6n0tjQ5g+/S4VDs1yyuChu9cPndcubHd/JiWFB52rxmhtxLtNXknKqXwsfRDNYmKd01L9vyEd87e+aBa3bdpgMLMB973u53OqDO0r5rG6neqbgT9ZrdCD3JtIXFM/MyZhbIBqHIJHrbiITFeE7kt60WeX+TnDbQzwr3aUw5s+xJN8/E2K3k2m2ki4N2/qjlQaleNaHpRihdFfbjUiiviY0VMwS6e0Z0WPegfPhFZwFimqYwPmPHe7E1BgzsjKH/ZPvZFGvDqqlYFXA/k= ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da 2023-09-13T03:04:30.948 DEBUG:teuthology.orchestra.run.smithi142:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtlx9VL3WXVuJralvCTbaJpxSzYKU1eOyKFpdY9Zcjpx6WEq7QMEdzMcQ5csSXLy9PSnaBEHAHpq8lSgw9416Fgjew4AC4bmIDGeiwe2BM8A60ZMhepl0BbrDHZ/5pNwhJ4KBVph0R9DX6ztNSoRQclainsLNt+vTSZHGIeMIxidHExBEs3dlXPF63eppcEWRR93hqIr7lSeSUFqm5Nv47ic7/CKc3PUPL4XhfZsEQdEt4CCm6n0tjQ5g+/S4VDs1yyuChu9cPndcubHd/JiWFB52rxmhtxLtNXknKqXwsfRDNYmKd01L9vyEd87e+aBa3bdpgMLMB973u53OqDO0r5rG6neqbgT9ZrdCD3JtIXFM/MyZhbIBqHIJHrbiITFeE7kt60WeX+TnDbQzwr3aUw5s+xJN8/E2K3k2m2ki4N2/qjlQaleNaHpRihdFfbjUiiviY0VMwS6e0Z0WPegfPhFZwFimqYwPmPHe7E1BgzsjKH/ZPvZFGvDqqlYFXA/k= ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-09-13T03:04:30.993 INFO:teuthology.orchestra.run.smithi142.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtlx9VL3WXVuJralvCTbaJpxSzYKU1eOyKFpdY9Zcjpx6WEq7QMEdzMcQ5csSXLy9PSnaBEHAHpq8lSgw9416Fgjew4AC4bmIDGeiwe2BM8A60ZMhepl0BbrDHZ/5pNwhJ4KBVph0R9DX6ztNSoRQclainsLNt+vTSZHGIeMIxidHExBEs3dlXPF63eppcEWRR93hqIr7lSeSUFqm5Nv47ic7/CKc3PUPL4XhfZsEQdEt4CCm6n0tjQ5g+/S4VDs1yyuChu9cPndcubHd/JiWFB52rxmhtxLtNXknKqXwsfRDNYmKd01L9vyEd87e+aBa3bdpgMLMB973u53OqDO0r5rG6neqbgT9ZrdCD3JtIXFM/MyZhbIBqHIJHrbiITFeE7kt60WeX+TnDbQzwr3aUw5s+xJN8/E2K3k2m2ki4N2/qjlQaleNaHpRihdFfbjUiiviY0VMwS6e0Z0WPegfPhFZwFimqYwPmPHe7E1BgzsjKH/ZPvZFGvDqqlYFXA/k= ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da 2023-09-13T03:04:31.006 DEBUG:teuthology.orchestra.run.smithi170:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtlx9VL3WXVuJralvCTbaJpxSzYKU1eOyKFpdY9Zcjpx6WEq7QMEdzMcQ5csSXLy9PSnaBEHAHpq8lSgw9416Fgjew4AC4bmIDGeiwe2BM8A60ZMhepl0BbrDHZ/5pNwhJ4KBVph0R9DX6ztNSoRQclainsLNt+vTSZHGIeMIxidHExBEs3dlXPF63eppcEWRR93hqIr7lSeSUFqm5Nv47ic7/CKc3PUPL4XhfZsEQdEt4CCm6n0tjQ5g+/S4VDs1yyuChu9cPndcubHd/JiWFB52rxmhtxLtNXknKqXwsfRDNYmKd01L9vyEd87e+aBa3bdpgMLMB973u53OqDO0r5rG6neqbgT9ZrdCD3JtIXFM/MyZhbIBqHIJHrbiITFeE7kt60WeX+TnDbQzwr3aUw5s+xJN8/E2K3k2m2ki4N2/qjlQaleNaHpRihdFfbjUiiviY0VMwS6e0Z0WPegfPhFZwFimqYwPmPHe7E1BgzsjKH/ZPvZFGvDqqlYFXA/k= ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-09-13T03:04:31.050 INFO:teuthology.orchestra.run.smithi170.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtlx9VL3WXVuJralvCTbaJpxSzYKU1eOyKFpdY9Zcjpx6WEq7QMEdzMcQ5csSXLy9PSnaBEHAHpq8lSgw9416Fgjew4AC4bmIDGeiwe2BM8A60ZMhepl0BbrDHZ/5pNwhJ4KBVph0R9DX6ztNSoRQclainsLNt+vTSZHGIeMIxidHExBEs3dlXPF63eppcEWRR93hqIr7lSeSUFqm5Nv47ic7/CKc3PUPL4XhfZsEQdEt4CCm6n0tjQ5g+/S4VDs1yyuChu9cPndcubHd/JiWFB52rxmhtxLtNXknKqXwsfRDNYmKd01L9vyEd87e+aBa3bdpgMLMB973u53OqDO0r5rG6neqbgT9ZrdCD3JtIXFM/MyZhbIBqHIJHrbiITFeE7kt60WeX+TnDbQzwr3aUw5s+xJN8/E2K3k2m2ki4N2/qjlQaleNaHpRihdFfbjUiiviY0VMwS6e0Z0WPegfPhFZwFimqYwPmPHe7E1BgzsjKH/ZPvZFGvDqqlYFXA/k= ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da 2023-09-13T03:04:31.063 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-09-13T03:04:31.801 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:04:34.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:33 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:04:34.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:33 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:04:34.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:33 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi064", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:04:34.170 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:33 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:04:34.170 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:33 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:04:34.170 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:33 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:04:34.171 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:33 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/562099123' entity='client.admin' 2023-09-13T03:04:34.394 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-09-13T03:04:34.395 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-09-13T03:04:35.161 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:34 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:04:35.162 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:34 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:04:35.379 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:04:37.559 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi142 2023-09-13T03:04:37.560 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-09-13T03:04:37.560 DEBUG:teuthology.orchestra.run.smithi142:> dd of=/etc/ceph/ceph.conf 2023-09-13T03:04:37.581 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-09-13T03:04:37.582 DEBUG:teuthology.orchestra.run.smithi142:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-13T03:04:37.641 INFO:tasks.cephadm:Adding host smithi142 to orchestrator... 2023-09-13T03:04:37.641 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph orch host add smithi142 2023-09-13T03:04:38.391 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:04:38.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:38 smithi064 ceph-mon[97675]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:04:38.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:38 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:04:38.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:38 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:04:38.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:38 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:04:38.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:38 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:04:38.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:38 smithi064 ceph-mon[97675]: Updating smithi064:/etc/ceph/ceph.conf 2023-09-13T03:04:39.213 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:39 smithi064 ceph-mon[97675]: Updating smithi064:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-13T03:04:39.214 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:39 smithi064 ceph-mon[97675]: Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2023-09-13T03:04:40.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:40 smithi064 ceph-mon[97675]: Updating smithi064:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-13T03:04:40.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:40 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:04:40.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:40 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:04:40.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:40 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:04:41.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:41 smithi064 ceph-mon[97675]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi142", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:04:42.510 INFO:teuthology.orchestra.run.smithi064.stdout:Added host 'smithi142' with addr '172.21.15.142' 2023-09-13T03:04:42.536 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:42 smithi064 ceph-mon[97675]: Deploying cephadm binary to smithi142 2023-09-13T03:04:42.980 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph orch host ls --format=json 2023-09-13T03:04:43.715 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:04:43.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:43 smithi064 ceph-mon[97675]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:04:43.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:43 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:04:43.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:43 smithi064 ceph-mon[97675]: Added host smithi142 2023-09-13T03:04:43.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:43 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:04:43.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:43 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:04:44.840 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:44 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:04:45.320 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:04:45.320 INFO:teuthology.orchestra.run.smithi064.stdout:[{"addr": "172.21.15.64", "hostname": "smithi064", "labels": [], "status": ""}, {"addr": "172.21.15.142", "hostname": "smithi142", "labels": [], "status": ""}] 2023-09-13T03:04:45.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:45 smithi064 ceph-mon[97675]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:04:45.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:45 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:04:45.702 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi170 2023-09-13T03:04:45.703 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-09-13T03:04:45.703 DEBUG:teuthology.orchestra.run.smithi170:> dd of=/etc/ceph/ceph.conf 2023-09-13T03:04:45.724 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-09-13T03:04:45.725 DEBUG:teuthology.orchestra.run.smithi170:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-13T03:04:45.784 INFO:tasks.cephadm:Adding host smithi170 to orchestrator... 2023-09-13T03:04:45.784 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph orch host add smithi170 2023-09-13T03:04:46.513 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:04:46.824 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:46 smithi064 ceph-mon[97675]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T03:04:47.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:47 smithi064 ceph-mon[97675]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:04:49.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:49 smithi064 ceph-mon[97675]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi170", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:04:49.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:49 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:04:49.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:49 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:04:49.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:49 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:04:49.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:49 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:04:49.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:49 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi142", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:04:49.421 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:49 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:04:49.421 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:49 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:04:49.421 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:49 smithi064 ceph-mon[97675]: Updating smithi142:/etc/ceph/ceph.conf 2023-09-13T03:04:49.421 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:49 smithi064 ceph-mon[97675]: Updating smithi142:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-13T03:04:49.421 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:49 smithi064 ceph-mon[97675]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:04:49.422 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:49 smithi064 ceph-mon[97675]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2023-09-13T03:04:49.422 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:49 smithi064 ceph-mon[97675]: Deploying cephadm binary to smithi170 2023-09-13T03:04:50.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:50 smithi064 ceph-mon[97675]: Updating smithi142:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-13T03:04:50.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:50 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:04:50.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:50 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:04:50.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:50 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:04:50.456 INFO:teuthology.orchestra.run.smithi064.stdout:Added host 'smithi170' with addr '172.21.15.170' 2023-09-13T03:04:51.002 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph orch host ls --format=json 2023-09-13T03:04:51.729 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:04:51.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:51 smithi064 ceph-mon[97675]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:04:51.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:51 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:04:51.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:51 smithi064 ceph-mon[97675]: Added host smithi170 2023-09-13T03:04:51.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:51 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:04:51.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:51 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:04:52.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:52 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:04:53.334 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:04:53.334 INFO:teuthology.orchestra.run.smithi064.stdout:[{"addr": "172.21.15.64", "hostname": "smithi064", "labels": [], "status": ""}, {"addr": "172.21.15.142", "hostname": "smithi142", "labels": [], "status": ""}, {"addr": "172.21.15.170", "hostname": "smithi170", "labels": [], "status": ""}] 2023-09-13T03:04:53.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:53 smithi064 ceph-mon[97675]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:04:53.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:53 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:04:53.702 INFO:tasks.cephadm:Setting crush tunables to default 2023-09-13T03:04:53.703 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph osd crush tunables default 2023-09-13T03:04:54.431 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:04:54.818 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:54 smithi064 ceph-mon[97675]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T03:04:55.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:55 smithi064 ceph-mon[97675]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:04:56.773 INFO:teuthology.orchestra.run.smithi064.stderr:adjusted tunables profile to default 2023-09-13T03:04:57.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:56 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:04:57.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:56 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:04:57.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:56 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:04:57.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:56 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:04:57.170 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:56 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi170", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:04:57.170 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:56 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:04:57.170 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:56 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:04:57.170 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:56 smithi064 ceph-mon[97675]: Updating smithi170:/etc/ceph/ceph.conf 2023-09-13T03:04:57.170 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:56 smithi064 ceph-mon[97675]: Updating smithi170:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-13T03:04:57.171 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:56 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/4293718870' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-09-13T03:04:57.171 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:56 smithi064 ceph-mon[97675]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2023-09-13T03:04:57.171 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:56 smithi064 ceph-mon[97675]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:04:57.171 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:56 smithi064 ceph-mon[97675]: Updating smithi170:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-13T03:04:57.171 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:56 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:04:57.172 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:56 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:04:57.172 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:56 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:04:57.208 INFO:tasks.cephadm:Adding mon.a on smithi064 2023-09-13T03:04:57.208 INFO:tasks.cephadm:Adding mon.b on smithi142 2023-09-13T03:04:57.208 INFO:tasks.cephadm:Adding mon.c on smithi170 2023-09-13T03:04:57.209 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph orch apply mon '3;smithi064:172.21.15.64=a;smithi142:172.21.15.142=b;smithi170:172.21.15.170=c' 2023-09-13T03:04:57.792 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-13T03:04:58.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:57 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/4293718870' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-09-13T03:04:58.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:57 smithi064 ceph-mon[97675]: osdmap e4: 0 total, 0 up, 0 in 2023-09-13T03:04:59.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:04:58 smithi064 ceph-mon[97675]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:04:59.232 INFO:teuthology.orchestra.run.smithi170.stdout:Scheduled mon update... 2023-09-13T03:04:59.664 DEBUG:teuthology.orchestra.run.smithi142:mon.b> sudo journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mon.b.service 2023-09-13T03:04:59.667 DEBUG:teuthology.orchestra.run.smithi170:mon.c> sudo journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mon.c.service 2023-09-13T03:04:59.669 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-09-13T03:04:59.669 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph mon dump -f json 2023-09-13T03:04:59.701 INFO:journalctl@ceph.mon.c.smithi170.stdout:-- Logs begin at Wed 2023-09-13 02:42:51 UTC. -- 2023-09-13T03:04:59.703 INFO:journalctl@ceph.mon.b.smithi142.stdout:-- Logs begin at Wed 2023-09-13 02:43:02 UTC. -- 2023-09-13T03:05:00.502 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-13T03:05:00.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:00 smithi064 ceph-mon[97675]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi064:172.21.15.64=a;smithi142:172.21.15.142=b;smithi170:172.21.15.170=c", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:05:00.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:00 smithi064 ceph-mon[97675]: Saving service mon spec with placement smithi064:172.21.15.64=a;smithi142:172.21.15.142=b;smithi170:172.21.15.170=c;count:3 2023-09-13T03:05:00.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:00 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:00.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:00 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:05:00.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:00 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:00.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:00 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:05:00.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:00 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:00.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:00 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-13T03:05:00.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:00 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:00.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:00 smithi064 ceph-mon[97675]: Deploying daemon mon.c on smithi170 2023-09-13T03:05:01.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:01 smithi064 ceph-mon[97675]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:02.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 systemd[1]: Starting Ceph mon.c for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:05:02.645 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-09-13T03:05:02.646 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":1,"fsid":"f8dbced4-51e1-11ee-9ab7-7b867c8bd7da","modified":"2023-09-13T03:03:19.966576Z","created":"2023-09-13T03:03:19.966576Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-09-13T03:05:02.647 INFO:teuthology.orchestra.run.smithi170.stderr:dumped monmap epoch 1 2023-09-13T03:05:02.770 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 podman[134278]: 2023-09-13T03:05:02.770 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: set uid:gid to 167:167 (ceph:ceph) 2023-09-13T03:05:02.771 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: ceph version 18.0.0-6088-g2110e007 (2110e00747b31a0d73768c9e5229da18b26b8aa0) reef (dev), process ceph-mon, pid 7 2023-09-13T03:05:02.771 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: pidfile_write: ignore empty --pid-file 2023-09-13T03:05:02.771 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: load: jerasure load: lrc 2023-09-13T03:05:02.771 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: RocksDB version: 7.9.2 2023-09-13T03:05:02.772 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Git sha 0 2023-09-13T03:05:02.772 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Compile date 2023-09-12 13:00:11 2023-09-13T03:05:02.772 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: DB SUMMARY 2023-09-13T03:05:02.773 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: DB Session ID: 79TVHFS9ES40KCHEG6ON 2023-09-13T03:05:02.773 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: CURRENT file: CURRENT 2023-09-13T03:05:02.773 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: IDENTITY file: IDENTITY 2023-09-13T03:05:02.773 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-09-13T03:05:02.774 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2023-09-13T03:05:02.774 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2023-09-13T03:05:02.774 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.error_if_exists: 0 2023-09-13T03:05:02.774 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.create_if_missing: 0 2023-09-13T03:05:02.775 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.paranoid_checks: 1 2023-09-13T03:05:02.775 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.flush_verify_memtable_count: 1 2023-09-13T03:05:02.775 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-09-13T03:05:02.775 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-09-13T03:05:02.776 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.env: 0x555e86832da0 2023-09-13T03:05:02.776 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.fs: PosixFileSystem 2023-09-13T03:05:02.776 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.info_log: 0x555e881a14e0 2023-09-13T03:05:02.776 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.max_file_opening_threads: 16 2023-09-13T03:05:02.777 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.statistics: (nil) 2023-09-13T03:05:02.777 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.use_fsync: 0 2023-09-13T03:05:02.777 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.max_log_file_size: 0 2023-09-13T03:05:02.777 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-09-13T03:05:02.777 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.log_file_time_to_roll: 0 2023-09-13T03:05:02.778 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.keep_log_file_num: 1000 2023-09-13T03:05:02.778 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.recycle_log_file_num: 0 2023-09-13T03:05:02.778 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.allow_fallocate: 1 2023-09-13T03:05:02.778 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.allow_mmap_reads: 0 2023-09-13T03:05:02.779 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.allow_mmap_writes: 0 2023-09-13T03:05:02.779 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.use_direct_reads: 0 2023-09-13T03:05:02.779 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-09-13T03:05:02.779 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.create_missing_column_families: 0 2023-09-13T03:05:02.779 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.db_log_dir: 2023-09-13T03:05:02.780 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.wal_dir: 2023-09-13T03:05:02.781 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.table_cache_numshardbits: 6 2023-09-13T03:05:02.782 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.WAL_ttl_seconds: 0 2023-09-13T03:05:02.782 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.WAL_size_limit_MB: 0 2023-09-13T03:05:02.782 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-09-13T03:05:02.783 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-09-13T03:05:02.783 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.is_fd_close_on_exec: 1 2023-09-13T03:05:02.783 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.advise_random_on_open: 1 2023-09-13T03:05:02.784 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.db_write_buffer_size: 0 2023-09-13T03:05:02.784 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.write_buffer_manager: 0x555e8815b4a0 2023-09-13T03:05:02.784 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-09-13T03:05:02.784 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-09-13T03:05:02.784 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.use_adaptive_mutex: 0 2023-09-13T03:05:02.785 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.rate_limiter: (nil) 2023-09-13T03:05:02.785 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-09-13T03:05:02.785 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.wal_recovery_mode: 2 2023-09-13T03:05:02.785 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.enable_thread_tracking: 0 2023-09-13T03:05:02.786 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.enable_pipelined_write: 0 2023-09-13T03:05:02.786 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.unordered_write: 0 2023-09-13T03:05:02.786 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-09-13T03:05:02.786 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-09-13T03:05:02.787 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-09-13T03:05:02.787 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-09-13T03:05:02.787 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.row_cache: None 2023-09-13T03:05:02.787 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.wal_filter: None 2023-09-13T03:05:02.788 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-09-13T03:05:02.788 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.allow_ingest_behind: 0 2023-09-13T03:05:02.788 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.two_write_queues: 0 2023-09-13T03:05:02.788 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.manual_wal_flush: 0 2023-09-13T03:05:02.788 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.wal_compression: 0 2023-09-13T03:05:02.789 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.atomic_flush: 0 2023-09-13T03:05:02.789 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-09-13T03:05:02.789 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.persist_stats_to_disk: 0 2023-09-13T03:05:02.789 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.write_dbid_to_manifest: 0 2023-09-13T03:05:02.789 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.log_readahead_size: 0 2023-09-13T03:05:02.790 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-09-13T03:05:02.790 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.best_efforts_recovery: 0 2023-09-13T03:05:02.790 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-09-13T03:05:02.790 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-09-13T03:05:02.791 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.allow_data_in_errors: 0 2023-09-13T03:05:02.791 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.db_host_id: __hostname__ 2023-09-13T03:05:02.791 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.enforce_single_del_contracts: true 2023-09-13T03:05:02.791 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.max_background_jobs: 2 2023-09-13T03:05:02.791 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.max_background_compactions: -1 2023-09-13T03:05:02.792 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.max_subcompactions: 1 2023-09-13T03:05:02.792 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-09-13T03:05:02.792 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-09-13T03:05:02.793 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.delayed_write_rate : 16777216 2023-09-13T03:05:02.793 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.max_total_wal_size: 0 2023-09-13T03:05:02.793 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-09-13T03:05:02.794 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.stats_dump_period_sec: 600 2023-09-13T03:05:02.794 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.stats_persist_period_sec: 600 2023-09-13T03:05:02.794 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-09-13T03:05:02.794 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.max_open_files: -1 2023-09-13T03:05:02.795 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.bytes_per_sync: 0 2023-09-13T03:05:02.795 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.wal_bytes_per_sync: 0 2023-09-13T03:05:02.795 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.strict_bytes_per_sync: 0 2023-09-13T03:05:02.795 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.compaction_readahead_size: 0 2023-09-13T03:05:02.796 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.max_background_flushes: -1 2023-09-13T03:05:02.796 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Compression algorithms supported: 2023-09-13T03:05:02.796 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-09-13T03:05:02.796 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: kZSTD supported: 0 2023-09-13T03:05:02.797 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: kXpressCompression supported: 0 2023-09-13T03:05:02.797 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: kLZ4HCCompression supported: 1 2023-09-13T03:05:02.797 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: kZlibCompression supported: 1 2023-09-13T03:05:02.797 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: kSnappyCompression supported: 1 2023-09-13T03:05:02.798 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: kLZ4Compression supported: 1 2023-09-13T03:05:02.798 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: kBZip2Compression supported: 0 2023-09-13T03:05:02.798 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Fast CRC32 supported: Supported on x86 2023-09-13T03:05:02.798 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: DMutex implementation: pthread_mutex_t 2023-09-13T03:05:02.799 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2023-09-13T03:05:02.799 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-09-13T03:05:02.799 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-09-13T03:05:02.799 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.merge_operator: 2023-09-13T03:05:02.799 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.compaction_filter: None 2023-09-13T03:05:02.800 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.compaction_filter_factory: None 2023-09-13T03:05:02.800 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.sst_partitioner_factory: None 2023-09-13T03:05:02.800 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.memtable_factory: SkipListFactory 2023-09-13T03:05:02.800 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.table_factory: BlockBasedTable 2023-09-13T03:05:02.800 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x555e881a15e0) 2023-09-13T03:05:02.801 INFO:journalctl@ceph.mon.c.smithi170.stdout: cache_index_and_filter_blocks: 1 2023-09-13T03:05:02.801 INFO:journalctl@ceph.mon.c.smithi170.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-09-13T03:05:02.801 INFO:journalctl@ceph.mon.c.smithi170.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-09-13T03:05:02.801 INFO:journalctl@ceph.mon.c.smithi170.stdout: pin_top_level_index_and_filter: 1 2023-09-13T03:05:02.802 INFO:journalctl@ceph.mon.c.smithi170.stdout: index_type: 0 2023-09-13T03:05:02.802 INFO:journalctl@ceph.mon.c.smithi170.stdout: data_block_index_type: 0 2023-09-13T03:05:02.802 INFO:journalctl@ceph.mon.c.smithi170.stdout: index_shortening: 1 2023-09-13T03:05:02.802 INFO:journalctl@ceph.mon.c.smithi170.stdout: data_block_hash_table_util_ratio: 0.750000 2023-09-13T03:05:02.803 INFO:journalctl@ceph.mon.c.smithi170.stdout: checksum: 4 2023-09-13T03:05:02.803 INFO:journalctl@ceph.mon.c.smithi170.stdout: no_block_cache: 0 2023-09-13T03:05:02.803 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_cache: 0x555e881f91f0 2023-09-13T03:05:02.803 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_cache_name: BinnedLRUCache 2023-09-13T03:05:02.804 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_cache_options: 2023-09-13T03:05:02.804 INFO:journalctl@ceph.mon.c.smithi170.stdout: capacity : 536870912 2023-09-13T03:05:02.804 INFO:journalctl@ceph.mon.c.smithi170.stdout: num_shard_bits : 4 2023-09-13T03:05:02.805 INFO:journalctl@ceph.mon.c.smithi170.stdout: strict_capacity_limit : 0 2023-09-13T03:05:02.805 INFO:journalctl@ceph.mon.c.smithi170.stdout: high_pri_pool_ratio: 0.000 2023-09-13T03:05:02.805 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_cache_compressed: (nil) 2023-09-13T03:05:02.805 INFO:journalctl@ceph.mon.c.smithi170.stdout: persistent_cache: (nil) 2023-09-13T03:05:02.806 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_size: 4096 2023-09-13T03:05:02.806 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_size_deviation: 10 2023-09-13T03:05:02.806 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_restart_interval: 16 2023-09-13T03:05:02.806 INFO:journalctl@ceph.mon.c.smithi170.stdout: index_block_restart_interval: 1 2023-09-13T03:05:02.806 INFO:journalctl@ceph.mon.c.smithi170.stdout: metadata_block_size: 4096 2023-09-13T03:05:02.807 INFO:journalctl@ceph.mon.c.smithi170.stdout: partition_filters: 0 2023-09-13T03:05:02.807 INFO:journalctl@ceph.mon.c.smithi170.stdout: use_delta_encoding: 1 2023-09-13T03:05:02.807 INFO:journalctl@ceph.mon.c.smithi170.stdout: filter_policy: bloomfilter 2023-09-13T03:05:02.807 INFO:journalctl@ceph.mon.c.smithi170.stdout: whole_key_filtering: 1 2023-09-13T03:05:02.808 INFO:journalctl@ceph.mon.c.smithi170.stdout: verify_compression: 0 2023-09-13T03:05:02.808 INFO:journalctl@ceph.mon.c.smithi170.stdout: read_amp_bytes_per_bit: 0 2023-09-13T03:05:02.808 INFO:journalctl@ceph.mon.c.smithi170.stdout: format_version: 5 2023-09-13T03:05:02.809 INFO:journalctl@ceph.mon.c.smithi170.stdout: enable_index_compression: 1 2023-09-13T03:05:02.809 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_align: 0 2023-09-13T03:05:02.809 INFO:journalctl@ceph.mon.c.smithi170.stdout: max_auto_readahead_size: 262144 2023-09-13T03:05:02.809 INFO:journalctl@ceph.mon.c.smithi170.stdout: prepopulate_block_cache: 0 2023-09-13T03:05:02.810 INFO:journalctl@ceph.mon.c.smithi170.stdout: initial_auto_readahead_size: 8192 2023-09-13T03:05:02.810 INFO:journalctl@ceph.mon.c.smithi170.stdout: num_file_reads_for_auto_readahead: 2 2023-09-13T03:05:02.810 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.write_buffer_size: 33554432 2023-09-13T03:05:02.810 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.max_write_buffer_number: 2 2023-09-13T03:05:02.810 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.compression: NoCompression 2023-09-13T03:05:02.811 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.bottommost_compression: Disabled 2023-09-13T03:05:02.811 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.prefix_extractor: nullptr 2023-09-13T03:05:02.811 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-09-13T03:05:02.811 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.num_levels: 7 2023-09-13T03:05:02.811 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-09-13T03:05:02.812 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-09-13T03:05:02.812 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-09-13T03:05:02.812 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-09-13T03:05:02.812 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-09-13T03:05:02.812 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-09-13T03:05:02.813 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-09-13T03:05:02.813 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-09-13T03:05:02.813 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-09-13T03:05:02.813 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-09-13T03:05:02.814 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-09-13T03:05:02.814 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-09-13T03:05:02.814 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.compression_opts.window_bits: -14 2023-09-13T03:05:02.815 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.compression_opts.level: 32767 2023-09-13T03:05:02.815 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.compression_opts.strategy: 0 2023-09-13T03:05:02.815 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-09-13T03:05:02.815 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-09-13T03:05:02.816 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-09-13T03:05:02.816 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-09-13T03:05:02.816 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.compression_opts.enabled: false 2023-09-13T03:05:02.816 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-09-13T03:05:02.817 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-09-13T03:05:02.817 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-09-13T03:05:02.817 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-09-13T03:05:02.817 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.target_file_size_base: 67108864 2023-09-13T03:05:02.818 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.target_file_size_multiplier: 1 2023-09-13T03:05:02.818 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-09-13T03:05:02.818 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-09-13T03:05:02.818 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-09-13T03:05:02.819 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-09-13T03:05:02.819 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-09-13T03:05:02.819 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-09-13T03:05:02.819 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-09-13T03:05:02.820 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-09-13T03:05:02.820 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-09-13T03:05:02.820 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-09-13T03:05:02.820 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-09-13T03:05:02.820 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-09-13T03:05:02.821 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-09-13T03:05:02.821 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.arena_block_size: 1048576 2023-09-13T03:05:02.821 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-09-13T03:05:02.821 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-09-13T03:05:02.821 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.disable_auto_compactions: 0 2023-09-13T03:05:02.822 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-09-13T03:05:02.822 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-09-13T03:05:02.822 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-09-13T03:05:02.822 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-09-13T03:05:02.823 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-09-13T03:05:02.823 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-09-13T03:05:02.823 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-09-13T03:05:02.823 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-09-13T03:05:02.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-09-13T03:05:02.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-09-13T03:05:02.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.table_properties_collectors: 2023-09-13T03:05:02.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.inplace_update_support: 0 2023-09-13T03:05:02.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.inplace_update_num_locks: 10000 2023-09-13T03:05:02.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-09-13T03:05:02.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-09-13T03:05:02.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.memtable_huge_page_size: 0 2023-09-13T03:05:02.827 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.bloom_locality: 0 2023-09-13T03:05:02.827 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.max_successive_merges: 0 2023-09-13T03:05:02.827 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.optimize_filters_for_hits: 0 2023-09-13T03:05:02.827 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.paranoid_file_checks: 0 2023-09-13T03:05:02.828 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.force_consistency_checks: 1 2023-09-13T03:05:02.828 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.report_bg_io_stats: 0 2023-09-13T03:05:02.828 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.ttl: 2592000 2023-09-13T03:05:02.828 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.periodic_compaction_seconds: 0 2023-09-13T03:05:02.828 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-09-13T03:05:02.829 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-09-13T03:05:02.829 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.enable_blob_files: false 2023-09-13T03:05:02.829 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.min_blob_size: 0 2023-09-13T03:05:02.829 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.blob_file_size: 268435456 2023-09-13T03:05:02.830 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.blob_compression_type: NoCompression 2023-09-13T03:05:02.830 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.enable_blob_garbage_collection: false 2023-09-13T03:05:02.830 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-09-13T03:05:02.830 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-09-13T03:05:02.831 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-09-13T03:05:02.831 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.blob_file_starting_level: 0 2023-09-13T03:05:02.831 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-09-13T03:05:02.831 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-09-13T03:05:02.832 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-09-13T03:05:02.832 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 79820d9c-f09d-4708-bcaa-e498bac90651 2023-09-13T03:05:02.832 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: EVENT_LOG_v1 {"time_micros": 1694574302749308, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-09-13T03:05:02.832 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-09-13T03:05:02.832 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: EVENT_LOG_v1 {"time_micros": 1694574302749869, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1694574302, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "79820d9c-f09d-4708-bcaa-e498bac90651", "db_session_id": "79TVHFS9ES40KCHEG6ON", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-09-13T03:05:02.833 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: EVENT_LOG_v1 {"time_micros": 1694574302749961, "job": 1, "event": "recovery_finished"} 2023-09-13T03:05:02.833 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-09-13T03:05:02.833 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-09-13T03:05:02.833 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x555e88294000 2023-09-13T03:05:02.834 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: DB pointer 0x555e88280000 2023-09-13T03:05:02.834 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-09-13T03:05:02.834 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-09-13T03:05:02.834 INFO:journalctl@ceph.mon.c.smithi170.stdout: ** DB Stats ** 2023-09-13T03:05:02.834 INFO:journalctl@ceph.mon.c.smithi170.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-13T03:05:02.835 INFO:journalctl@ceph.mon.c.smithi170.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-09-13T03:05:02.835 INFO:journalctl@ceph.mon.c.smithi170.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-13T03:05:02.835 INFO:journalctl@ceph.mon.c.smithi170.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-13T03:05:02.835 INFO:journalctl@ceph.mon.c.smithi170.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-09-13T03:05:02.836 INFO:journalctl@ceph.mon.c.smithi170.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-13T03:05:02.836 INFO:journalctl@ceph.mon.c.smithi170.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-13T03:05:02.836 INFO:journalctl@ceph.mon.c.smithi170.stdout: 2023-09-13T03:05:02.836 INFO:journalctl@ceph.mon.c.smithi170.stdout: ** Compaction Stats [default] ** 2023-09-13T03:05:02.837 INFO:journalctl@ceph.mon.c.smithi170.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-13T03:05:02.837 INFO:journalctl@ceph.mon.c.smithi170.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-09-13T03:05:02.837 INFO:journalctl@ceph.mon.c.smithi170.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-13T03:05:02.837 INFO:journalctl@ceph.mon.c.smithi170.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-13T03:05:02.837 INFO:journalctl@ceph.mon.c.smithi170.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-13T03:05:02.838 INFO:journalctl@ceph.mon.c.smithi170.stdout: 2023-09-13T03:05:02.838 INFO:journalctl@ceph.mon.c.smithi170.stdout: ** Compaction Stats [default] ** 2023-09-13T03:05:02.838 INFO:journalctl@ceph.mon.c.smithi170.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-13T03:05:02.838 INFO:journalctl@ceph.mon.c.smithi170.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-09-13T03:05:02.839 INFO:journalctl@ceph.mon.c.smithi170.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-13T03:05:02.839 INFO:journalctl@ceph.mon.c.smithi170.stdout: 2023-09-13T03:05:02.839 INFO:journalctl@ceph.mon.c.smithi170.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-09-13T03:05:02.839 INFO:journalctl@ceph.mon.c.smithi170.stdout: 2023-09-13T03:05:02.840 INFO:journalctl@ceph.mon.c.smithi170.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-13T03:05:02.840 INFO:journalctl@ceph.mon.c.smithi170.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-09-13T03:05:02.840 INFO:journalctl@ceph.mon.c.smithi170.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-09-13T03:05:02.840 INFO:journalctl@ceph.mon.c.smithi170.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-09-13T03:05:02.841 INFO:journalctl@ceph.mon.c.smithi170.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-09-13T03:05:02.841 INFO:journalctl@ceph.mon.c.smithi170.stdout: AddFile(Keys): cumulative 0, interval 0 2023-09-13T03:05:02.841 INFO:journalctl@ceph.mon.c.smithi170.stdout: Cumulative compaction: 0.00 GB write, 0.45 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-13T03:05:02.841 INFO:journalctl@ceph.mon.c.smithi170.stdout: Interval compaction: 0.00 GB write, 0.45 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-13T03:05:02.841 INFO:journalctl@ceph.mon.c.smithi170.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-09-13T03:05:02.842 INFO:journalctl@ceph.mon.c.smithi170.stdout: Block cache BinnedLRUCache@0x555e881f91f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-09-13T03:05:02.842 INFO:journalctl@ceph.mon.c.smithi170.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-09-13T03:05:02.842 INFO:journalctl@ceph.mon.c.smithi170.stdout: 2023-09-13T03:05:02.842 INFO:journalctl@ceph.mon.c.smithi170.stdout: ** File Read Latency Histogram By Level [default] ** 2023-09-13T03:05:02.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: mon.c does not exist in monmap, will attempt to join an existing cluster 2023-09-13T03:05:02.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: using public_addr v2:172.21.15.170:0/0 -> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] 2023-09-13T03:05:02.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: starting mon.c rank -1 at public addrs [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] at bind addrs [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da 2023-09-13T03:05:02.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: mon.c@-1(???) e0 preinit fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da 2023-09-13T03:05:02.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: mon.c@-1(synchronizing).mds e1 new map 2023-09-13T03:05:02.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: mon.c@-1(synchronizing).mds e1 print_map 2023-09-13T03:05:02.844 INFO:journalctl@ceph.mon.c.smithi170.stdout: e1 2023-09-13T03:05:02.844 INFO:journalctl@ceph.mon.c.smithi170.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-09-13T03:05:02.844 INFO:journalctl@ceph.mon.c.smithi170.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-13T03:05:02.845 INFO:journalctl@ceph.mon.c.smithi170.stdout: legacy client fscid: -1 2023-09-13T03:05:02.845 INFO:journalctl@ceph.mon.c.smithi170.stdout: 2023-09-13T03:05:02.845 INFO:journalctl@ceph.mon.c.smithi170.stdout: No filesystems configured 2023-09-13T03:05:02.845 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-09-13T03:05:02.845 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-09-13T03:05:02.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-09-13T03:05:02.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-09-13T03:05:02.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-09-13T03:05:02.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-09-13T03:05:02.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-09-13T03:05:02.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-13T03:05:02.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-13T03:05:02.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-13T03:05:02.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T03:05:02.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:02.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi170", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:05:02.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:02.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:02.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:02.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:02.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi142", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:05:02.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:02.850 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:05:02.850 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: Updating smithi142:/etc/ceph/ceph.conf 2023-09-13T03:05:02.850 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: Updating smithi142:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-13T03:05:02.851 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:02.851 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2023-09-13T03:05:02.851 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: Deploying cephadm binary to smithi170 2023-09-13T03:05:02.851 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: Updating smithi142:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-13T03:05:02.852 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:02.852 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:02.852 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:02.852 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:02.852 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:02.853 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: Added host smithi170 2023-09-13T03:05:02.853 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:05:02.853 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:02.853 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:02.853 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:02.854 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:02.854 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T03:05:02.854 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:02.854 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:02.855 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:02.855 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:02.855 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:02.855 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi170", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:05:02.856 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:02.856 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:05:02.856 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: Updating smithi170:/etc/ceph/ceph.conf 2023-09-13T03:05:02.857 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: Updating smithi170:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-13T03:05:02.857 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/4293718870' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-09-13T03:05:02.857 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2023-09-13T03:05:02.858 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:02.858 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: Updating smithi170:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-13T03:05:02.858 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:02.858 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:02.859 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:02.859 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/4293718870' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-09-13T03:05:02.859 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: osdmap e4: 0 total, 0 up, 0 in 2023-09-13T03:05:02.859 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:02.859 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi064:172.21.15.64=a;smithi142:172.21.15.142=b;smithi170:172.21.15.170=c", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:05:02.860 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: Saving service mon spec with placement smithi064:172.21.15.64=a;smithi142:172.21.15.142=b;smithi170:172.21.15.170=c;count:3 2023-09-13T03:05:02.860 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:02.860 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:05:02.860 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:02.861 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:05:02.861 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:02.861 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-13T03:05:02.861 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:02.862 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: Deploying daemon mon.c on smithi170 2023-09-13T03:05:02.862 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:02.862 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-09-13T03:05:02.862 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: expand_channel_meta expand map: {default=false} 2023-09-13T03:05:02.863 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: expand_channel_meta from 'false' to 'false' 2023-09-13T03:05:02.863 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: expand_channel_meta expanded map: {default=false} 2023-09-13T03:05:02.863 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: expand_channel_meta expand map: {default=info} 2023-09-13T03:05:02.863 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: expand_channel_meta from 'info' to 'info' 2023-09-13T03:05:02.863 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: expand_channel_meta expanded map: {default=info} 2023-09-13T03:05:02.864 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: expand_channel_meta expand map: {default=daemon} 2023-09-13T03:05:02.864 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: expand_channel_meta from 'daemon' to 'daemon' 2023-09-13T03:05:02.864 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: expand_channel_meta expanded map: {default=daemon} 2023-09-13T03:05:02.864 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: expand_channel_meta expand map: {default=debug} 2023-09-13T03:05:02.864 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: expand_channel_meta from 'debug' to 'debug' 2023-09-13T03:05:02.865 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: expand_channel_meta expanded map: {default=debug} 2023-09-13T03:05:02.865 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:02 smithi170 ceph-mon[134328]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2023-09-13T03:05:03.031 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:03 smithi170 bash[134278]: ab82ebb517afa18bfa09221d0dbf28d31019d8ac5597a499ad74140feec3d864 2023-09-13T03:05:03.409 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:03 smithi170 systemd[1]: Started Ceph mon.c for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da. 2023-09-13T03:05:04.166 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-09-13T03:05:04.167 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph mon dump -f json 2023-09-13T03:05:04.735 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-13T03:05:08.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:07 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-13T03:05:08.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:07 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T03:05:08.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:07 smithi170 ceph-mon[134328]: mon.a calling monitor election 2023-09-13T03:05:08.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:07 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T03:05:08.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:07 smithi170 ceph-mon[134328]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:08.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:07 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T03:05:08.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:07 smithi170 ceph-mon[134328]: mon.c calling monitor election 2023-09-13T03:05:08.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:07 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T03:05:08.162 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:07 smithi170 ceph-mon[134328]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:08.162 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:07 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T03:05:08.162 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:07 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T03:05:08.162 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:07 smithi170 ceph-mon[134328]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-13T03:05:08.162 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:07 smithi170 ceph-mon[134328]: monmap e2: 2 mons at {a=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} 2023-09-13T03:05:08.163 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:07 smithi170 ceph-mon[134328]: fsmap 2023-09-13T03:05:08.163 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:07 smithi170 ceph-mon[134328]: osdmap e4: 0 total, 0 up, 0 in 2023-09-13T03:05:08.163 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:07 smithi170 ceph-mon[134328]: mgrmap e13: x(active, since 45s) 2023-09-13T03:05:08.163 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:07 smithi170 ceph-mon[134328]: overall HEALTH_OK 2023-09-13T03:05:08.163 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:07 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:08.164 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:07 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:08.164 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:07 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:08.164 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:07 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-13T03:05:08.164 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:07 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:08.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:07 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-13T03:05:08.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:07 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T03:05:08.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:07 smithi064 ceph-mon[97675]: mon.a calling monitor election 2023-09-13T03:05:08.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:07 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T03:05:08.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:07 smithi064 ceph-mon[97675]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:08.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:07 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T03:05:08.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:07 smithi064 ceph-mon[97675]: mon.c calling monitor election 2023-09-13T03:05:08.170 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:07 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T03:05:08.170 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:07 smithi064 ceph-mon[97675]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:08.170 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:07 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T03:05:08.171 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:07 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T03:05:08.171 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:07 smithi064 ceph-mon[97675]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-13T03:05:08.171 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:07 smithi064 ceph-mon[97675]: monmap e2: 2 mons at {a=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} 2023-09-13T03:05:08.171 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:07 smithi064 ceph-mon[97675]: fsmap 2023-09-13T03:05:08.171 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:07 smithi064 ceph-mon[97675]: osdmap e4: 0 total, 0 up, 0 in 2023-09-13T03:05:08.172 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:07 smithi064 ceph-mon[97675]: mgrmap e13: x(active, since 45s) 2023-09-13T03:05:08.172 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:07 smithi064 ceph-mon[97675]: overall HEALTH_OK 2023-09-13T03:05:08.172 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:07 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:08.172 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:07 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:08.172 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:07 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:08.172 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:07 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-13T03:05:08.173 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:07 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:09.159 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:08 smithi170 ceph-mon[134328]: Deploying daemon mon.b on smithi142 2023-09-13T03:05:09.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:08 smithi170 ceph-mon[134328]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:09.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:08 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T03:05:09.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:08 smithi064 ceph-mon[97675]: Deploying daemon mon.b on smithi142 2023-09-13T03:05:09.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:08 smithi064 ceph-mon[97675]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:09.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:08 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T03:05:09.213 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-09-13T03:05:09.213 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":2,"fsid":"f8dbced4-51e1-11ee-9ab7-7b867c8bd7da","modified":"2023-09-13T03:05:02.775622Z","created":"2023-09-13T03:03:19.966576Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:3300","nonce":0},{"type":"v1","addr":"172.21.15.170:6789","nonce":0}]},"addr":"172.21.15.170:6789/0","public_addr":"172.21.15.170:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-09-13T03:05:09.215 INFO:teuthology.orchestra.run.smithi170.stderr:dumped monmap epoch 2 2023-09-13T03:05:10.088 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:05:09 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:05:09.776+0000 7fd161ea3700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-09-13T03:05:10.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:10 smithi170 ceph-mon[134328]: from='client.? 172.21.15.170:0/553491245' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-13T03:05:10.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:10 smithi064 ceph-mon[97675]: from='client.? 172.21.15.170:0/553491245' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-13T03:05:10.675 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 systemd[1]: Starting Ceph mon.b for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:05:10.686 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-09-13T03:05:10.687 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph mon dump -f json 2023-09-13T03:05:10.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 podman[134042]: 2023-09-13T03:05:10.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: set uid:gid to 167:167 (ceph:ceph) 2023-09-13T03:05:10.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: ceph version 18.0.0-6088-g2110e007 (2110e00747b31a0d73768c9e5229da18b26b8aa0) reef (dev), process ceph-mon, pid 7 2023-09-13T03:05:10.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: pidfile_write: ignore empty --pid-file 2023-09-13T03:05:10.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 bash[134042]: 920fce1721710813a51898af5d2ec1dacf2db01fe72d699ea2a3d35c97881082 2023-09-13T03:05:10.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: load: jerasure load: lrc 2023-09-13T03:05:10.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: RocksDB version: 7.9.2 2023-09-13T03:05:10.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Git sha 0 2023-09-13T03:05:10.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Compile date 2023-09-12 13:00:11 2023-09-13T03:05:10.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: DB SUMMARY 2023-09-13T03:05:10.998 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: DB Session ID: UI2D1OSHWH6YALK66DWW 2023-09-13T03:05:10.998 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: CURRENT file: CURRENT 2023-09-13T03:05:10.998 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: IDENTITY file: IDENTITY 2023-09-13T03:05:10.998 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-09-13T03:05:10.998 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-09-13T03:05:10.999 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-09-13T03:05:10.999 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.error_if_exists: 0 2023-09-13T03:05:10.999 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.create_if_missing: 0 2023-09-13T03:05:10.999 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.paranoid_checks: 1 2023-09-13T03:05:10.999 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.flush_verify_memtable_count: 1 2023-09-13T03:05:11.000 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-09-13T03:05:11.000 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-09-13T03:05:11.000 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.env: 0x556e1108cda0 2023-09-13T03:05:11.000 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.fs: PosixFileSystem 2023-09-13T03:05:11.000 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.info_log: 0x556e1195f4e0 2023-09-13T03:05:11.001 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.max_file_opening_threads: 16 2023-09-13T03:05:11.001 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.statistics: (nil) 2023-09-13T03:05:11.001 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.use_fsync: 0 2023-09-13T03:05:11.001 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.max_log_file_size: 0 2023-09-13T03:05:11.001 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-09-13T03:05:11.002 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.log_file_time_to_roll: 0 2023-09-13T03:05:11.002 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.keep_log_file_num: 1000 2023-09-13T03:05:11.002 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.recycle_log_file_num: 0 2023-09-13T03:05:11.002 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.allow_fallocate: 1 2023-09-13T03:05:11.002 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.allow_mmap_reads: 0 2023-09-13T03:05:11.003 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.allow_mmap_writes: 0 2023-09-13T03:05:11.003 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.use_direct_reads: 0 2023-09-13T03:05:11.003 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-09-13T03:05:11.003 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.create_missing_column_families: 0 2023-09-13T03:05:11.003 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.db_log_dir: 2023-09-13T03:05:11.004 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.wal_dir: 2023-09-13T03:05:11.004 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.table_cache_numshardbits: 6 2023-09-13T03:05:11.004 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.WAL_ttl_seconds: 0 2023-09-13T03:05:11.004 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.WAL_size_limit_MB: 0 2023-09-13T03:05:11.004 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-09-13T03:05:11.005 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-09-13T03:05:11.005 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.is_fd_close_on_exec: 1 2023-09-13T03:05:11.005 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.advise_random_on_open: 1 2023-09-13T03:05:11.005 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.db_write_buffer_size: 0 2023-09-13T03:05:11.005 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.write_buffer_manager: 0x556e119194a0 2023-09-13T03:05:11.006 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-09-13T03:05:11.006 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-09-13T03:05:11.006 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.use_adaptive_mutex: 0 2023-09-13T03:05:11.006 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.rate_limiter: (nil) 2023-09-13T03:05:11.006 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-09-13T03:05:11.007 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.wal_recovery_mode: 2 2023-09-13T03:05:11.007 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.enable_thread_tracking: 0 2023-09-13T03:05:11.007 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.enable_pipelined_write: 0 2023-09-13T03:05:11.007 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.unordered_write: 0 2023-09-13T03:05:11.007 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-09-13T03:05:11.008 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-09-13T03:05:11.008 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-09-13T03:05:11.008 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-09-13T03:05:11.008 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.row_cache: None 2023-09-13T03:05:11.008 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.wal_filter: None 2023-09-13T03:05:11.008 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-09-13T03:05:11.009 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.allow_ingest_behind: 0 2023-09-13T03:05:11.009 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.two_write_queues: 0 2023-09-13T03:05:11.009 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.manual_wal_flush: 0 2023-09-13T03:05:11.009 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.wal_compression: 0 2023-09-13T03:05:11.009 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.atomic_flush: 0 2023-09-13T03:05:11.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-09-13T03:05:11.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.persist_stats_to_disk: 0 2023-09-13T03:05:11.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.write_dbid_to_manifest: 0 2023-09-13T03:05:11.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.log_readahead_size: 0 2023-09-13T03:05:11.010 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-09-13T03:05:11.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.best_efforts_recovery: 0 2023-09-13T03:05:11.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-09-13T03:05:11.011 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-09-13T03:05:11.013 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.allow_data_in_errors: 0 2023-09-13T03:05:11.013 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.db_host_id: __hostname__ 2023-09-13T03:05:11.013 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.enforce_single_del_contracts: true 2023-09-13T03:05:11.013 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.max_background_jobs: 2 2023-09-13T03:05:11.013 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.max_background_compactions: -1 2023-09-13T03:05:11.014 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.max_subcompactions: 1 2023-09-13T03:05:11.014 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-09-13T03:05:11.014 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-09-13T03:05:11.014 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.delayed_write_rate : 16777216 2023-09-13T03:05:11.014 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.max_total_wal_size: 0 2023-09-13T03:05:11.015 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-09-13T03:05:11.015 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.stats_dump_period_sec: 600 2023-09-13T03:05:11.015 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.stats_persist_period_sec: 600 2023-09-13T03:05:11.015 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-09-13T03:05:11.015 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.max_open_files: -1 2023-09-13T03:05:11.016 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.bytes_per_sync: 0 2023-09-13T03:05:11.016 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.wal_bytes_per_sync: 0 2023-09-13T03:05:11.016 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.strict_bytes_per_sync: 0 2023-09-13T03:05:11.016 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.compaction_readahead_size: 0 2023-09-13T03:05:11.016 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.max_background_flushes: -1 2023-09-13T03:05:11.017 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Compression algorithms supported: 2023-09-13T03:05:11.017 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-09-13T03:05:11.017 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: kZSTD supported: 0 2023-09-13T03:05:11.017 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: kXpressCompression supported: 0 2023-09-13T03:05:11.017 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: kLZ4HCCompression supported: 1 2023-09-13T03:05:11.018 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: kZlibCompression supported: 1 2023-09-13T03:05:11.018 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: kSnappyCompression supported: 1 2023-09-13T03:05:11.018 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: kLZ4Compression supported: 1 2023-09-13T03:05:11.018 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: kBZip2Compression supported: 0 2023-09-13T03:05:11.018 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Fast CRC32 supported: Supported on x86 2023-09-13T03:05:11.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: DMutex implementation: pthread_mutex_t 2023-09-13T03:05:11.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-09-13T03:05:11.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-09-13T03:05:11.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-09-13T03:05:11.019 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.merge_operator: 2023-09-13T03:05:11.020 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.compaction_filter: None 2023-09-13T03:05:11.020 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.compaction_filter_factory: None 2023-09-13T03:05:11.020 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.sst_partitioner_factory: None 2023-09-13T03:05:11.020 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.memtable_factory: SkipListFactory 2023-09-13T03:05:11.020 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.table_factory: BlockBasedTable 2023-09-13T03:05:11.021 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556e1195f5e0) 2023-09-13T03:05:11.021 INFO:journalctl@ceph.mon.b.smithi142.stdout: cache_index_and_filter_blocks: 1 2023-09-13T03:05:11.021 INFO:journalctl@ceph.mon.b.smithi142.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-09-13T03:05:11.021 INFO:journalctl@ceph.mon.b.smithi142.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-09-13T03:05:11.021 INFO:journalctl@ceph.mon.b.smithi142.stdout: pin_top_level_index_and_filter: 1 2023-09-13T03:05:11.022 INFO:journalctl@ceph.mon.b.smithi142.stdout: index_type: 0 2023-09-13T03:05:11.022 INFO:journalctl@ceph.mon.b.smithi142.stdout: data_block_index_type: 0 2023-09-13T03:05:11.022 INFO:journalctl@ceph.mon.b.smithi142.stdout: index_shortening: 1 2023-09-13T03:05:11.022 INFO:journalctl@ceph.mon.b.smithi142.stdout: data_block_hash_table_util_ratio: 0.750000 2023-09-13T03:05:11.022 INFO:journalctl@ceph.mon.b.smithi142.stdout: checksum: 4 2023-09-13T03:05:11.023 INFO:journalctl@ceph.mon.b.smithi142.stdout: no_block_cache: 0 2023-09-13T03:05:11.023 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_cache: 0x556e119b71f0 2023-09-13T03:05:11.023 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_cache_name: BinnedLRUCache 2023-09-13T03:05:11.023 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_cache_options: 2023-09-13T03:05:11.023 INFO:journalctl@ceph.mon.b.smithi142.stdout: capacity : 536870912 2023-09-13T03:05:11.024 INFO:journalctl@ceph.mon.b.smithi142.stdout: num_shard_bits : 4 2023-09-13T03:05:11.024 INFO:journalctl@ceph.mon.b.smithi142.stdout: strict_capacity_limit : 0 2023-09-13T03:05:11.024 INFO:journalctl@ceph.mon.b.smithi142.stdout: high_pri_pool_ratio: 0.000 2023-09-13T03:05:11.024 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_cache_compressed: (nil) 2023-09-13T03:05:11.024 INFO:journalctl@ceph.mon.b.smithi142.stdout: persistent_cache: (nil) 2023-09-13T03:05:11.025 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_size: 4096 2023-09-13T03:05:11.025 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_size_deviation: 10 2023-09-13T03:05:11.025 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_restart_interval: 16 2023-09-13T03:05:11.025 INFO:journalctl@ceph.mon.b.smithi142.stdout: index_block_restart_interval: 1 2023-09-13T03:05:11.025 INFO:journalctl@ceph.mon.b.smithi142.stdout: metadata_block_size: 4096 2023-09-13T03:05:11.025 INFO:journalctl@ceph.mon.b.smithi142.stdout: partition_filters: 0 2023-09-13T03:05:11.026 INFO:journalctl@ceph.mon.b.smithi142.stdout: use_delta_encoding: 1 2023-09-13T03:05:11.026 INFO:journalctl@ceph.mon.b.smithi142.stdout: filter_policy: bloomfilter 2023-09-13T03:05:11.026 INFO:journalctl@ceph.mon.b.smithi142.stdout: whole_key_filtering: 1 2023-09-13T03:05:11.026 INFO:journalctl@ceph.mon.b.smithi142.stdout: verify_compression: 0 2023-09-13T03:05:11.026 INFO:journalctl@ceph.mon.b.smithi142.stdout: read_amp_bytes_per_bit: 0 2023-09-13T03:05:11.027 INFO:journalctl@ceph.mon.b.smithi142.stdout: format_version: 5 2023-09-13T03:05:11.027 INFO:journalctl@ceph.mon.b.smithi142.stdout: enable_index_compression: 1 2023-09-13T03:05:11.027 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_align: 0 2023-09-13T03:05:11.027 INFO:journalctl@ceph.mon.b.smithi142.stdout: max_auto_readahead_size: 262144 2023-09-13T03:05:11.027 INFO:journalctl@ceph.mon.b.smithi142.stdout: prepopulate_block_cache: 0 2023-09-13T03:05:11.028 INFO:journalctl@ceph.mon.b.smithi142.stdout: initial_auto_readahead_size: 8192 2023-09-13T03:05:11.028 INFO:journalctl@ceph.mon.b.smithi142.stdout: num_file_reads_for_auto_readahead: 2 2023-09-13T03:05:11.028 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.write_buffer_size: 33554432 2023-09-13T03:05:11.028 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.max_write_buffer_number: 2 2023-09-13T03:05:11.028 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.compression: NoCompression 2023-09-13T03:05:11.029 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.bottommost_compression: Disabled 2023-09-13T03:05:11.029 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.prefix_extractor: nullptr 2023-09-13T03:05:11.029 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-09-13T03:05:11.029 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.num_levels: 7 2023-09-13T03:05:11.029 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-09-13T03:05:11.030 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-09-13T03:05:11.030 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-09-13T03:05:11.030 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-09-13T03:05:11.030 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-09-13T03:05:11.030 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-09-13T03:05:11.031 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-09-13T03:05:11.031 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-09-13T03:05:11.031 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-09-13T03:05:11.031 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-09-13T03:05:11.031 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-09-13T03:05:11.031 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-09-13T03:05:11.032 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.compression_opts.window_bits: -14 2023-09-13T03:05:11.032 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.compression_opts.level: 32767 2023-09-13T03:05:11.032 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.compression_opts.strategy: 0 2023-09-13T03:05:11.032 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-09-13T03:05:11.032 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-09-13T03:05:11.033 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-09-13T03:05:11.033 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-09-13T03:05:11.033 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.compression_opts.enabled: false 2023-09-13T03:05:11.033 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-09-13T03:05:11.033 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-09-13T03:05:11.034 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-09-13T03:05:11.034 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-09-13T03:05:11.034 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.target_file_size_base: 67108864 2023-09-13T03:05:11.034 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.target_file_size_multiplier: 1 2023-09-13T03:05:11.034 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-09-13T03:05:11.035 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-09-13T03:05:11.035 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-09-13T03:05:11.035 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-09-13T03:05:11.037 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-09-13T03:05:11.037 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-09-13T03:05:11.037 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-09-13T03:05:11.037 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-09-13T03:05:11.037 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-09-13T03:05:11.038 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-09-13T03:05:11.038 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-09-13T03:05:11.038 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-09-13T03:05:11.038 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-09-13T03:05:11.038 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.arena_block_size: 1048576 2023-09-13T03:05:11.039 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-09-13T03:05:11.039 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-09-13T03:05:11.039 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.disable_auto_compactions: 0 2023-09-13T03:05:11.039 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-09-13T03:05:11.039 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-09-13T03:05:11.040 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-09-13T03:05:11.040 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-09-13T03:05:11.040 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-09-13T03:05:11.040 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-09-13T03:05:11.040 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-09-13T03:05:11.041 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-09-13T03:05:11.041 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-09-13T03:05:11.041 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-09-13T03:05:11.041 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.table_properties_collectors: 2023-09-13T03:05:11.041 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.inplace_update_support: 0 2023-09-13T03:05:11.041 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.inplace_update_num_locks: 10000 2023-09-13T03:05:11.042 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-09-13T03:05:11.042 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-09-13T03:05:11.042 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.memtable_huge_page_size: 0 2023-09-13T03:05:11.042 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.bloom_locality: 0 2023-09-13T03:05:11.042 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.max_successive_merges: 0 2023-09-13T03:05:11.043 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.optimize_filters_for_hits: 0 2023-09-13T03:05:11.043 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.paranoid_file_checks: 0 2023-09-13T03:05:11.043 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.force_consistency_checks: 1 2023-09-13T03:05:11.043 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.report_bg_io_stats: 0 2023-09-13T03:05:11.043 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.ttl: 2592000 2023-09-13T03:05:11.044 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.periodic_compaction_seconds: 0 2023-09-13T03:05:11.044 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-09-13T03:05:11.044 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-09-13T03:05:11.044 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.enable_blob_files: false 2023-09-13T03:05:11.044 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.min_blob_size: 0 2023-09-13T03:05:11.044 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.blob_file_size: 268435456 2023-09-13T03:05:11.045 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.blob_compression_type: NoCompression 2023-09-13T03:05:11.045 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.enable_blob_garbage_collection: false 2023-09-13T03:05:11.045 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-09-13T03:05:11.045 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-09-13T03:05:11.045 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-09-13T03:05:11.046 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.blob_file_starting_level: 0 2023-09-13T03:05:11.046 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-09-13T03:05:11.046 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-09-13T03:05:11.046 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-09-13T03:05:11.046 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: f96afd6d-a538-4de6-85f1-ad97daf0d99d 2023-09-13T03:05:11.047 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: EVENT_LOG_v1 {"time_micros": 1694574310879031, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-09-13T03:05:11.047 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-09-13T03:05:11.047 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: EVENT_LOG_v1 {"time_micros": 1694574310879609, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1694574310, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "f96afd6d-a538-4de6-85f1-ad97daf0d99d", "db_session_id": "UI2D1OSHWH6YALK66DWW", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-09-13T03:05:11.047 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: EVENT_LOG_v1 {"time_micros": 1694574310879699, "job": 1, "event": "recovery_finished"} 2023-09-13T03:05:11.047 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-09-13T03:05:11.048 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-09-13T03:05:11.048 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556e11a52000 2023-09-13T03:05:11.048 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: DB pointer 0x556e11a3e000 2023-09-13T03:05:11.048 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-09-13T03:05:11.049 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-09-13T03:05:11.049 INFO:journalctl@ceph.mon.b.smithi142.stdout: ** DB Stats ** 2023-09-13T03:05:11.049 INFO:journalctl@ceph.mon.b.smithi142.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-13T03:05:11.049 INFO:journalctl@ceph.mon.b.smithi142.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-09-13T03:05:11.049 INFO:journalctl@ceph.mon.b.smithi142.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-13T03:05:11.049 INFO:journalctl@ceph.mon.b.smithi142.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-13T03:05:11.050 INFO:journalctl@ceph.mon.b.smithi142.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-09-13T03:05:11.050 INFO:journalctl@ceph.mon.b.smithi142.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-13T03:05:11.050 INFO:journalctl@ceph.mon.b.smithi142.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-13T03:05:11.050 INFO:journalctl@ceph.mon.b.smithi142.stdout: 2023-09-13T03:05:11.051 INFO:journalctl@ceph.mon.b.smithi142.stdout: ** Compaction Stats [default] ** 2023-09-13T03:05:11.051 INFO:journalctl@ceph.mon.b.smithi142.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-13T03:05:11.051 INFO:journalctl@ceph.mon.b.smithi142.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-09-13T03:05:11.051 INFO:journalctl@ceph.mon.b.smithi142.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-13T03:05:11.051 INFO:journalctl@ceph.mon.b.smithi142.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-13T03:05:11.052 INFO:journalctl@ceph.mon.b.smithi142.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-13T03:05:11.052 INFO:journalctl@ceph.mon.b.smithi142.stdout: 2023-09-13T03:05:11.052 INFO:journalctl@ceph.mon.b.smithi142.stdout: ** Compaction Stats [default] ** 2023-09-13T03:05:11.052 INFO:journalctl@ceph.mon.b.smithi142.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-13T03:05:11.052 INFO:journalctl@ceph.mon.b.smithi142.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-09-13T03:05:11.053 INFO:journalctl@ceph.mon.b.smithi142.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-13T03:05:11.053 INFO:journalctl@ceph.mon.b.smithi142.stdout: 2023-09-13T03:05:11.053 INFO:journalctl@ceph.mon.b.smithi142.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-09-13T03:05:11.053 INFO:journalctl@ceph.mon.b.smithi142.stdout: 2023-09-13T03:05:11.053 INFO:journalctl@ceph.mon.b.smithi142.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-13T03:05:11.054 INFO:journalctl@ceph.mon.b.smithi142.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-09-13T03:05:11.054 INFO:journalctl@ceph.mon.b.smithi142.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-09-13T03:05:11.054 INFO:journalctl@ceph.mon.b.smithi142.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-09-13T03:05:11.054 INFO:journalctl@ceph.mon.b.smithi142.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-09-13T03:05:11.054 INFO:journalctl@ceph.mon.b.smithi142.stdout: AddFile(Keys): cumulative 0, interval 0 2023-09-13T03:05:11.055 INFO:journalctl@ceph.mon.b.smithi142.stdout: Cumulative compaction: 0.00 GB write, 0.50 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-13T03:05:11.055 INFO:journalctl@ceph.mon.b.smithi142.stdout: Interval compaction: 0.00 GB write, 0.50 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-13T03:05:11.055 INFO:journalctl@ceph.mon.b.smithi142.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-09-13T03:05:11.055 INFO:journalctl@ceph.mon.b.smithi142.stdout: Block cache BinnedLRUCache@0x556e119b71f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-09-13T03:05:11.055 INFO:journalctl@ceph.mon.b.smithi142.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-09-13T03:05:11.056 INFO:journalctl@ceph.mon.b.smithi142.stdout: 2023-09-13T03:05:11.056 INFO:journalctl@ceph.mon.b.smithi142.stdout: ** File Read Latency Histogram By Level [default] ** 2023-09-13T03:05:11.056 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-09-13T03:05:11.056 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: using public_addr v2:172.21.15.142:0/0 -> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] 2023-09-13T03:05:11.056 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: starting mon.b rank -1 at public addrs [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] at bind addrs [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da 2023-09-13T03:05:11.057 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: mon.b@-1(???) e0 preinit fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da 2023-09-13T03:05:11.057 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 systemd[1]: Started Ceph mon.b for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da. 2023-09-13T03:05:11.057 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: mon.b@-1(synchronizing).mds e1 new map 2023-09-13T03:05:11.057 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: mon.b@-1(synchronizing).mds e1 print_map 2023-09-13T03:05:11.057 INFO:journalctl@ceph.mon.b.smithi142.stdout: e1 2023-09-13T03:05:11.057 INFO:journalctl@ceph.mon.b.smithi142.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-09-13T03:05:11.058 INFO:journalctl@ceph.mon.b.smithi142.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-13T03:05:11.058 INFO:journalctl@ceph.mon.b.smithi142.stdout: legacy client fscid: -1 2023-09-13T03:05:11.058 INFO:journalctl@ceph.mon.b.smithi142.stdout: 2023-09-13T03:05:11.058 INFO:journalctl@ceph.mon.b.smithi142.stdout: No filesystems configured 2023-09-13T03:05:11.058 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-09-13T03:05:11.059 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-09-13T03:05:11.059 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-09-13T03:05:11.059 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-09-13T03:05:11.059 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-09-13T03:05:11.059 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-09-13T03:05:11.060 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-09-13T03:05:11.060 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-13T03:05:11.060 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-13T03:05:11.060 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-13T03:05:11.060 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T03:05:11.061 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:11.061 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi170", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:05:11.061 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:11.061 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:11.061 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:11.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:11.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi142", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:05:11.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:11.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:05:11.062 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: Updating smithi142:/etc/ceph/ceph.conf 2023-09-13T03:05:11.063 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: Updating smithi142:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-13T03:05:11.063 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:11.063 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2023-09-13T03:05:11.063 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: Deploying cephadm binary to smithi170 2023-09-13T03:05:11.063 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: Updating smithi142:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-13T03:05:11.064 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:11.064 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:11.064 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:11.064 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:11.064 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:11.065 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: Added host smithi170 2023-09-13T03:05:11.065 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:05:11.065 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:11.065 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:11.065 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:11.066 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:11.066 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T03:05:11.066 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:11.066 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:11.066 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:11.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:11.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:11.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi170", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:05:11.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:11.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:05:11.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: Updating smithi170:/etc/ceph/ceph.conf 2023-09-13T03:05:11.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: Updating smithi170:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-13T03:05:11.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/4293718870' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-09-13T03:05:11.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2023-09-13T03:05:11.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:11.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: Updating smithi170:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-13T03:05:11.069 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:11.069 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:11.069 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:11.069 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/4293718870' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-09-13T03:05:11.069 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: osdmap e4: 0 total, 0 up, 0 in 2023-09-13T03:05:11.070 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:11.070 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi064:172.21.15.64=a;smithi142:172.21.15.142=b;smithi170:172.21.15.170=c", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:05:11.070 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: Saving service mon spec with placement smithi064:172.21.15.64=a;smithi142:172.21.15.142=b;smithi170:172.21.15.170=c;count:3 2023-09-13T03:05:11.070 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:11.071 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:05:11.071 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:11.071 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:05:11.071 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:11.071 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-13T03:05:11.072 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:11.072 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: Deploying daemon mon.c on smithi170 2023-09-13T03:05:11.072 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:11.072 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-13T03:05:11.072 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T03:05:11.073 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: mon.a calling monitor election 2023-09-13T03:05:11.073 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T03:05:11.073 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:11.073 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T03:05:11.073 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: mon.c calling monitor election 2023-09-13T03:05:11.074 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T03:05:11.074 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:11.074 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T03:05:11.074 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T03:05:11.074 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-13T03:05:11.075 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: monmap e2: 2 mons at {a=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} 2023-09-13T03:05:11.075 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: fsmap 2023-09-13T03:05:11.075 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: osdmap e4: 0 total, 0 up, 0 in 2023-09-13T03:05:11.075 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: mgrmap e13: x(active, since 45s) 2023-09-13T03:05:11.075 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: overall HEALTH_OK 2023-09-13T03:05:11.076 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:11.076 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:11.076 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:11.076 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-13T03:05:11.076 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:11.077 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: Deploying daemon mon.b on smithi142 2023-09-13T03:05:11.077 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:11.077 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T03:05:11.077 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: from='client.? 172.21.15.170:0/553491245' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-13T03:05:11.077 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-09-13T03:05:11.078 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: expand_channel_meta expand map: {default=false} 2023-09-13T03:05:11.078 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: expand_channel_meta from 'false' to 'false' 2023-09-13T03:05:11.078 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: expand_channel_meta expanded map: {default=false} 2023-09-13T03:05:11.078 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: expand_channel_meta expand map: {default=info} 2023-09-13T03:05:11.078 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: expand_channel_meta from 'info' to 'info' 2023-09-13T03:05:11.078 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: expand_channel_meta expanded map: {default=info} 2023-09-13T03:05:11.079 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: expand_channel_meta expand map: {default=daemon} 2023-09-13T03:05:11.079 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: expand_channel_meta from 'daemon' to 'daemon' 2023-09-13T03:05:11.079 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: expand_channel_meta expanded map: {default=daemon} 2023-09-13T03:05:11.079 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: expand_channel_meta expand map: {default=debug} 2023-09-13T03:05:11.079 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: expand_channel_meta from 'debug' to 'debug' 2023-09-13T03:05:11.080 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: expand_channel_meta expanded map: {default=debug} 2023-09-13T03:05:11.080 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:10 smithi142 ceph-mon[134082]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-09-13T03:05:11.252 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-13T03:05:16.232 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-09-13T03:05:16.232 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":3,"fsid":"f8dbced4-51e1-11ee-9ab7-7b867c8bd7da","modified":"2023-09-13T03:05:10.912287Z","created":"2023-09-13T03:03:19.966576Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:3300","nonce":0},{"type":"v1","addr":"172.21.15.170:6789","nonce":0}]},"addr":"172.21.15.170:6789/0","public_addr":"172.21.15.170:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:3300","nonce":0},{"type":"v1","addr":"172.21.15.142:6789","nonce":0}]},"addr":"172.21.15.142:6789/0","public_addr":"172.21.15.142:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-09-13T03:05:16.234 INFO:teuthology.orchestra.run.smithi170.stderr:dumped monmap epoch 3 2023-09-13T03:05:16.254 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:15 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-13T03:05:16.255 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:15 smithi170 ceph-mon[134328]: mon.c calling monitor election 2023-09-13T03:05:16.255 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:15 smithi170 ceph-mon[134328]: mon.a calling monitor election 2023-09-13T03:05:16.256 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:15 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T03:05:16.256 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:15 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T03:05:16.257 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:15 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T03:05:16.257 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:15 smithi170 ceph-mon[134328]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:16.257 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:15 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T03:05:16.258 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:15 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T03:05:16.258 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:15 smithi170 ceph-mon[134328]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:16.259 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:15 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T03:05:16.259 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:15 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T03:05:16.259 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:15 smithi170 ceph-mon[134328]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-13T03:05:16.260 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:15 smithi170 ceph-mon[134328]: monmap e3: 3 mons at {a=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0],b=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} 2023-09-13T03:05:16.260 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:15 smithi170 ceph-mon[134328]: fsmap 2023-09-13T03:05:16.261 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:15 smithi170 ceph-mon[134328]: osdmap e4: 0 total, 0 up, 0 in 2023-09-13T03:05:16.261 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:15 smithi170 ceph-mon[134328]: mgrmap e13: x(active, since 53s) 2023-09-13T03:05:16.261 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:15 smithi170 ceph-mon[134328]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-09-13T03:05:16.262 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:15 smithi170 ceph-mon[134328]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-09-13T03:05:16.262 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:15 smithi170 ceph-mon[134328]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-09-13T03:05:16.263 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:15 smithi170 ceph-mon[134328]: mon.b (rank 2) addr [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] is down (out of quorum) 2023-09-13T03:05:16.263 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:15 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:16.263 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:15 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:16.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:15 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-13T03:05:16.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:15 smithi064 ceph-mon[97675]: mon.c calling monitor election 2023-09-13T03:05:16.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:15 smithi064 ceph-mon[97675]: mon.a calling monitor election 2023-09-13T03:05:16.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:15 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T03:05:16.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:15 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T03:05:16.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:15 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T03:05:16.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:15 smithi064 ceph-mon[97675]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:16.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:15 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T03:05:16.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:15 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T03:05:16.421 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:15 smithi064 ceph-mon[97675]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:16.421 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:15 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T03:05:16.421 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:15 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T03:05:16.421 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:15 smithi064 ceph-mon[97675]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-13T03:05:16.421 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:15 smithi064 ceph-mon[97675]: monmap e3: 3 mons at {a=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0],b=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} 2023-09-13T03:05:16.422 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:15 smithi064 ceph-mon[97675]: fsmap 2023-09-13T03:05:16.422 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:15 smithi064 ceph-mon[97675]: osdmap e4: 0 total, 0 up, 0 in 2023-09-13T03:05:16.422 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:15 smithi064 ceph-mon[97675]: mgrmap e13: x(active, since 53s) 2023-09-13T03:05:16.422 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:15 smithi064 ceph-mon[97675]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-09-13T03:05:16.423 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:15 smithi064 ceph-mon[97675]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-09-13T03:05:16.423 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:15 smithi064 ceph-mon[97675]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-09-13T03:05:16.423 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:15 smithi064 ceph-mon[97675]: mon.b (rank 2) addr [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] is down (out of quorum) 2023-09-13T03:05:16.423 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:15 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:16.423 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:15 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:16.603 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-09-13T03:05:16.604 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph config generate-minimal-conf 2023-09-13T03:05:17.349 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:05:17.409 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:16 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:17.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:16 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:17.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:16 smithi170 ceph-mon[134328]: from='client.? 172.21.15.170:0/4070733412' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-13T03:05:17.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:16 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:05:17.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:16 smithi170 ceph-mon[134328]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:17.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:16 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T03:05:17.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:16 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:17.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:16 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:17.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:16 smithi064 ceph-mon[97675]: from='client.? 172.21.15.170:0/4070733412' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-13T03:05:17.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:16 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:05:17.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:16 smithi064 ceph-mon[97675]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:17.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:16 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T03:05:18.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:17 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-13T03:05:18.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:17 smithi142 ceph-mon[134082]: mon.c calling monitor election 2023-09-13T03:05:18.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:17 smithi142 ceph-mon[134082]: mon.a calling monitor election 2023-09-13T03:05:18.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:17 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T03:05:18.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:17 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T03:05:18.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:17 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T03:05:18.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:17 smithi142 ceph-mon[134082]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:18.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:17 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T03:05:18.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:17 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T03:05:18.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:17 smithi142 ceph-mon[134082]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:18.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:17 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T03:05:18.248 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:17 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T03:05:18.248 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:17 smithi142 ceph-mon[134082]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-13T03:05:18.248 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:17 smithi142 ceph-mon[134082]: monmap e3: 3 mons at {a=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0],b=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} 2023-09-13T03:05:18.248 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:17 smithi142 ceph-mon[134082]: fsmap 2023-09-13T03:05:18.249 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:17 smithi142 ceph-mon[134082]: osdmap e4: 0 total, 0 up, 0 in 2023-09-13T03:05:18.249 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:17 smithi142 ceph-mon[134082]: mgrmap e13: x(active, since 53s) 2023-09-13T03:05:18.249 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:17 smithi142 ceph-mon[134082]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-09-13T03:05:18.249 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:17 smithi142 ceph-mon[134082]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-09-13T03:05:18.249 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:17 smithi142 ceph-mon[134082]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-09-13T03:05:18.250 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:17 smithi142 ceph-mon[134082]: mon.b (rank 2) addr [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] is down (out of quorum) 2023-09-13T03:05:18.250 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:17 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:18.250 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:17 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:18.250 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:17 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:18.250 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:17 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:18.251 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:17 smithi142 ceph-mon[134082]: from='client.? 172.21.15.170:0/4070733412' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-13T03:05:18.251 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:17 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:05:18.251 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:17 smithi142 ceph-mon[134082]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:18.251 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:17 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T03:05:18.953 INFO:teuthology.orchestra.run.smithi064.stdout:# minimal ceph.conf for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da 2023-09-13T03:05:18.953 INFO:teuthology.orchestra.run.smithi064.stdout:[global] 2023-09-13T03:05:18.954 INFO:teuthology.orchestra.run.smithi064.stdout: fsid = f8dbced4-51e1-11ee-9ab7-7b867c8bd7da 2023-09-13T03:05:18.954 INFO:teuthology.orchestra.run.smithi064.stdout: mon_host = [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] 2023-09-13T03:05:19.068 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:18 smithi064 ceph-mon[97675]: mon.b calling monitor election 2023-09-13T03:05:19.068 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:18 smithi064 ceph-mon[97675]: Updating smithi064:/etc/ceph/ceph.conf 2023-09-13T03:05:19.069 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:18 smithi064 ceph-mon[97675]: Updating smithi142:/etc/ceph/ceph.conf 2023-09-13T03:05:19.069 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:18 smithi064 ceph-mon[97675]: Updating smithi170:/etc/ceph/ceph.conf 2023-09-13T03:05:19.069 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:18 smithi064 ceph-mon[97675]: Updating smithi142:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-13T03:05:19.069 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:18 smithi064 ceph-mon[97675]: Updating smithi170:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-13T03:05:19.069 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:18 smithi064 ceph-mon[97675]: mon.b calling monitor election 2023-09-13T03:05:19.070 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:18 smithi064 ceph-mon[97675]: mon.c calling monitor election 2023-09-13T03:05:19.070 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:18 smithi064 ceph-mon[97675]: mon.a calling monitor election 2023-09-13T03:05:19.070 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:18 smithi064 ceph-mon[97675]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-09-13T03:05:19.070 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:18 smithi064 ceph-mon[97675]: monmap e3: 3 mons at {a=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0],b=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} 2023-09-13T03:05:19.070 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:18 smithi064 ceph-mon[97675]: fsmap 2023-09-13T03:05:19.071 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:18 smithi064 ceph-mon[97675]: osdmap e4: 0 total, 0 up, 0 in 2023-09-13T03:05:19.071 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:18 smithi064 ceph-mon[97675]: mgrmap e13: x(active, since 55s) 2023-09-13T03:05:19.071 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:18 smithi064 ceph-mon[97675]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-09-13T03:05:19.071 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:18 smithi064 ceph-mon[97675]: Cluster is now healthy 2023-09-13T03:05:19.071 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:18 smithi064 ceph-mon[97675]: overall HEALTH_OK 2023-09-13T03:05:19.072 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:18 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:19.072 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:18 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:19.072 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:18 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:19.072 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:18 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:19.072 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:18 smithi064 ceph-mon[97675]: Updating smithi064:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-13T03:05:19.072 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:18 smithi064 ceph-mon[97675]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:19.073 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:18 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:19.073 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:18 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:19.073 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:18 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:19.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:18 smithi170 ceph-mon[134328]: mon.b calling monitor election 2023-09-13T03:05:19.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:18 smithi170 ceph-mon[134328]: Updating smithi064:/etc/ceph/ceph.conf 2023-09-13T03:05:19.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:18 smithi170 ceph-mon[134328]: Updating smithi142:/etc/ceph/ceph.conf 2023-09-13T03:05:19.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:18 smithi170 ceph-mon[134328]: Updating smithi170:/etc/ceph/ceph.conf 2023-09-13T03:05:19.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:18 smithi170 ceph-mon[134328]: Updating smithi142:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-13T03:05:19.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:18 smithi170 ceph-mon[134328]: Updating smithi170:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-13T03:05:19.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:18 smithi170 ceph-mon[134328]: mon.b calling monitor election 2023-09-13T03:05:19.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:18 smithi170 ceph-mon[134328]: mon.c calling monitor election 2023-09-13T03:05:19.162 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:18 smithi170 ceph-mon[134328]: mon.a calling monitor election 2023-09-13T03:05:19.162 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:18 smithi170 ceph-mon[134328]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-09-13T03:05:19.162 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:18 smithi170 ceph-mon[134328]: monmap e3: 3 mons at {a=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0],b=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} 2023-09-13T03:05:19.162 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:18 smithi170 ceph-mon[134328]: fsmap 2023-09-13T03:05:19.163 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:18 smithi170 ceph-mon[134328]: osdmap e4: 0 total, 0 up, 0 in 2023-09-13T03:05:19.163 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:18 smithi170 ceph-mon[134328]: mgrmap e13: x(active, since 55s) 2023-09-13T03:05:19.163 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:18 smithi170 ceph-mon[134328]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-09-13T03:05:19.163 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:18 smithi170 ceph-mon[134328]: Cluster is now healthy 2023-09-13T03:05:19.163 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:18 smithi170 ceph-mon[134328]: overall HEALTH_OK 2023-09-13T03:05:19.163 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:18 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:19.164 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:18 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:19.164 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:18 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:19.164 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:18 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:19.164 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:18 smithi170 ceph-mon[134328]: Updating smithi064:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-13T03:05:19.164 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:18 smithi170 ceph-mon[134328]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:19.165 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:18 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:19.165 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:18 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:19.165 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:18 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:19.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:18 smithi142 ceph-mon[134082]: mon.b calling monitor election 2023-09-13T03:05:19.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:18 smithi142 ceph-mon[134082]: Updating smithi064:/etc/ceph/ceph.conf 2023-09-13T03:05:19.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:18 smithi142 ceph-mon[134082]: Updating smithi142:/etc/ceph/ceph.conf 2023-09-13T03:05:19.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:18 smithi142 ceph-mon[134082]: Updating smithi170:/etc/ceph/ceph.conf 2023-09-13T03:05:19.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:18 smithi142 ceph-mon[134082]: Updating smithi142:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-13T03:05:19.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:18 smithi142 ceph-mon[134082]: Updating smithi170:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-13T03:05:19.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:18 smithi142 ceph-mon[134082]: mon.b calling monitor election 2023-09-13T03:05:19.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:18 smithi142 ceph-mon[134082]: mon.c calling monitor election 2023-09-13T03:05:19.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:18 smithi142 ceph-mon[134082]: mon.a calling monitor election 2023-09-13T03:05:19.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:18 smithi142 ceph-mon[134082]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-09-13T03:05:19.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:18 smithi142 ceph-mon[134082]: monmap e3: 3 mons at {a=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0],b=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} 2023-09-13T03:05:19.248 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:18 smithi142 ceph-mon[134082]: fsmap 2023-09-13T03:05:19.248 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:18 smithi142 ceph-mon[134082]: osdmap e4: 0 total, 0 up, 0 in 2023-09-13T03:05:19.248 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:18 smithi142 ceph-mon[134082]: mgrmap e13: x(active, since 55s) 2023-09-13T03:05:19.248 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:18 smithi142 ceph-mon[134082]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-09-13T03:05:19.248 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:18 smithi142 ceph-mon[134082]: Cluster is now healthy 2023-09-13T03:05:19.249 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:18 smithi142 ceph-mon[134082]: overall HEALTH_OK 2023-09-13T03:05:19.249 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:18 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:19.249 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:18 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:19.249 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:18 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:19.249 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:18 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:19.249 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:18 smithi142 ceph-mon[134082]: Updating smithi064:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-13T03:05:19.250 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:18 smithi142 ceph-mon[134082]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:19.250 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:18 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:19.250 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:18 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:19.250 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:18 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:19.333 INFO:tasks.cephadm:['ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'] 2023-09-13T03:05:19.334 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph config set client client_mount_timeout 600 2023-09-13T03:05:19.339 INFO:tasks.cephadm:['ceph', 'config', 'set', 'client', 'debug_client', '20'] 2023-09-13T03:05:19.339 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph config set client debug_client 20 2023-09-13T03:05:19.384 INFO:tasks.cephadm:['ceph', 'config', 'set', 'client', 'debug_ms', '1'] 2023-09-13T03:05:19.385 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph config set client debug_ms 1 2023-09-13T03:05:19.430 INFO:tasks.cephadm:['ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'] 2023-09-13T03:05:19.431 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph config set client rados_mon_op_timeout 900 2023-09-13T03:05:19.478 INFO:tasks.cephadm:['ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'] 2023-09-13T03:05:19.479 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph config set client rados_osd_op_timeout 900 2023-09-13T03:05:19.524 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'debug_mds', '20'] 2023-09-13T03:05:19.525 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph config set mds debug_mds 20 2023-09-13T03:05:19.572 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'] 2023-09-13T03:05:19.572 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph config set mds debug_mds_balancer 20 2023-09-13T03:05:19.623 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'debug_ms', '1'] 2023-09-13T03:05:19.624 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph config set mds debug_ms 1 2023-09-13T03:05:19.676 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'] 2023-09-13T03:05:19.677 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph config set mds mds_bal_fragment_size_max 10000 2023-09-13T03:05:19.729 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'] 2023-09-13T03:05:19.730 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph config set mds mds_bal_merge_size 5 2023-09-13T03:05:19.789 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'] 2023-09-13T03:05:19.789 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph config set mds mds_bal_split_bits 3 2023-09-13T03:05:19.859 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'] 2023-09-13T03:05:19.859 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph config set mds mds_bal_split_size 100 2023-09-13T03:05:19.932 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'] 2023-09-13T03:05:19.933 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph config set mds mds_debug_frag True 2023-09-13T03:05:19.934 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:19 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T03:05:19.934 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:19 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/441968423' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:19.934 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:19 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:19.934 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:19 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:19.935 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:19 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:19.935 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:19 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:19.935 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:19 smithi064 ceph-mon[97675]: Reconfiguring mon.a (unknown last config time)... 2023-09-13T03:05:19.935 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:19 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-13T03:05:19.935 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:19 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-13T03:05:19.936 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:19 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:19.936 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:19 smithi064 ceph-mon[97675]: Reconfiguring daemon mon.a on smithi064 2023-09-13T03:05:19.987 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'] 2023-09-13T03:05:19.987 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph config set mds mds_debug_scatterstat True 2023-09-13T03:05:20.066 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'] 2023-09-13T03:05:20.067 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph config set mds mds_op_complaint_time 180 2023-09-13T03:05:20.132 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'] 2023-09-13T03:05:20.132 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph config set mds mds_verify_scatter True 2023-09-13T03:05:20.159 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:19 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T03:05:20.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:19 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/441968423' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:20.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:19 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:20.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:19 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:20.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:19 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:20.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:19 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:20.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:19 smithi170 ceph-mon[134328]: Reconfiguring mon.a (unknown last config time)... 2023-09-13T03:05:20.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:19 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-13T03:05:20.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:19 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-13T03:05:20.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:19 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:20.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:19 smithi170 ceph-mon[134328]: Reconfiguring daemon mon.a on smithi064 2023-09-13T03:05:20.163 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:05:19 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:05:19.922+0000 7fd161ea3700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-09-13T03:05:20.172 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'] 2023-09-13T03:05:20.172 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph config set mds osd_op_complaint_time 180 2023-09-13T03:05:20.244 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'] 2023-09-13T03:05:20.244 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph config set mds rados_mon_op_timeout 900 2023-09-13T03:05:20.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:19 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T03:05:20.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:19 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/441968423' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:20.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:19 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:20.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:19 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:20.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:19 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:20.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:19 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:20.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:19 smithi142 ceph-mon[134082]: Reconfiguring mon.a (unknown last config time)... 2023-09-13T03:05:20.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:19 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-13T03:05:20.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:19 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-13T03:05:20.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:19 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:20.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:19 smithi142 ceph-mon[134082]: Reconfiguring daemon mon.a on smithi064 2023-09-13T03:05:20.331 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'] 2023-09-13T03:05:20.332 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph config set mds rados_osd_op_timeout 900 2023-09-13T03:05:20.382 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'] 2023-09-13T03:05:20.383 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph config set mon mon_op_complaint_time 120 2023-09-13T03:05:20.445 INFO:tasks.cephadm:['ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'] 2023-09-13T03:05:20.446 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph config set osd osd_op_complaint_time 180 2023-09-13T03:05:20.500 DEBUG:tasks.cephadm:set 21 configs 2023-09-13T03:05:21.057 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:05:21.081 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:05:21.159 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:20 smithi170 ceph-mon[134328]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:21.170 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:20 smithi064 ceph-mon[97675]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:21.178 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:05:21.190 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:05:21.200 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:05:21.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:20 smithi142 ceph-mon[134082]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:21.458 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:05:21.490 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:05:21.524 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:05:21.543 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:05:21.591 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:05:22.106 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:05:22.674 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:22 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:22.674 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:22 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:22.674 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:22 smithi064 ceph-mon[97675]: Reconfiguring mon.b (monmap changed)... 2023-09-13T03:05:22.675 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:22 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-13T03:05:22.675 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:22 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-13T03:05:22.675 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:22 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:22.675 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:22 smithi064 ceph-mon[97675]: Reconfiguring daemon mon.b on smithi142 2023-09-13T03:05:22.715 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:05:22.716 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:05:22.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:22 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:22.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:22 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:22.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:22 smithi142 ceph-mon[134082]: Reconfiguring mon.b (monmap changed)... 2023-09-13T03:05:22.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:22 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-13T03:05:22.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:22 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-13T03:05:22.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:22 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:22.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:22 smithi142 ceph-mon[134082]: Reconfiguring daemon mon.b on smithi142 2023-09-13T03:05:22.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:22 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:22.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:22 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:22.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:22 smithi170 ceph-mon[134328]: Reconfiguring mon.b (monmap changed)... 2023-09-13T03:05:22.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:22 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-13T03:05:22.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:22 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-13T03:05:22.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:22 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:22.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:22 smithi170 ceph-mon[134328]: Reconfiguring daemon mon.b on smithi142 2023-09-13T03:05:23.586 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:23 smithi170 ceph-mon[134328]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:23.586 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:23 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:23.586 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:23 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:23.586 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:23 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-13T03:05:23.587 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:23 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-13T03:05:23.587 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:23 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:23.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:23 smithi064 ceph-mon[97675]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:23.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:23 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:23.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:23 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:23.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:23 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-13T03:05:23.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:23 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-13T03:05:23.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:23 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:23.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:23 smithi142 ceph-mon[134082]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:23.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:23 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:23.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:23 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:23.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:23 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-13T03:05:23.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:23 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-13T03:05:23.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:23 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:24.597 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:24 smithi064 ceph-mon[97675]: Reconfiguring mon.c (monmap changed)... 2023-09-13T03:05:24.597 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:24 smithi064 ceph-mon[97675]: Reconfiguring daemon mon.c on smithi170 2023-09-13T03:05:24.597 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:24 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:24.598 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:24 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:24.598 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:24 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:05:24.598 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:24 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:24.598 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:24 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:05:24.599 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:24 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:24.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:24 smithi142 ceph-mon[134082]: Reconfiguring mon.c (monmap changed)... 2023-09-13T03:05:24.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:24 smithi142 ceph-mon[134082]: Reconfiguring daemon mon.c on smithi170 2023-09-13T03:05:24.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:24 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:24.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:24 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:24.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:24 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:05:24.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:24 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:24.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:24 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:05:24.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:24 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:24.772 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:05:24.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:24 smithi170 ceph-mon[134328]: Reconfiguring mon.c (monmap changed)... 2023-09-13T03:05:24.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:24 smithi170 ceph-mon[134328]: Reconfiguring daemon mon.c on smithi170 2023-09-13T03:05:24.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:24 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:24.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:24 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:24.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:24 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:05:24.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:24 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:24.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:24 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:05:24.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:24 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:25.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:25 smithi142 ceph-mon[134082]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:25.828 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:25 smithi064 ceph-mon[97675]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:25.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:25 smithi170 ceph-mon[134328]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:27.430 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:05:27.730 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:27 smithi064 ceph-mon[97675]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:27.731 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:27 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/3611038662' entity='client.admin' 2023-09-13T03:05:27.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:27 smithi142 ceph-mon[134082]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:27.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:27 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/3611038662' entity='client.admin' 2023-09-13T03:05:27.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:27 smithi170 ceph-mon[134328]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:27.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:27 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/3611038662' entity='client.admin' 2023-09-13T03:05:28.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:28.395+0000 7f0349910700 1 -- 172.21.15.64:0/1139439271 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f032c038630 msgr2=0x7f032c03aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:28.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:28.395+0000 7f0349910700 1 --2- 172.21.15.64:0/1139439271 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f032c038630 0x7f032c03aaf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0334005fd0 tx=0x7f0334009310 comp rx=0 tx=0).stop 2023-09-13T03:05:28.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:28.395+0000 7f0349910700 1 -- 172.21.15.64:0/1139439271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0344100fe0 msgr2=0x7f0344198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:28.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:28.395+0000 7f0349910700 1 --2- 172.21.15.64:0/1139439271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0344100fe0 0x7f0344198400 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0338009810 tx=0x7f033800b790 comp rx=0 tx=0).stop 2023-09-13T03:05:28.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:28.395+0000 7f0349910700 1 -- 172.21.15.64:0/1139439271 shutdown_connections 2023-09-13T03:05:28.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:28.395+0000 7f0349910700 1 --2- 172.21.15.64:0/1139439271 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f032c038630 0x7f032c03aaf0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:28.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:28.395+0000 7f0349910700 1 --2- 172.21.15.64:0/1139439271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0344100620 0x7f0344197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:28.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:28.395+0000 7f0349910700 1 --2- 172.21.15.64:0/1139439271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0344106620 0x7f034419c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:28.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:28.395+0000 7f0349910700 1 --2- 172.21.15.64:0/1139439271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0344100fe0 0x7f0344198400 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:28.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:28.395+0000 7f0349910700 1 -- 172.21.15.64:0/1139439271 >> 172.21.15.64:0/1139439271 conn(0x7f03440fc0f0 msgr2=0x7f034410ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:05:28.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:28.396+0000 7f0349910700 1 -- 172.21.15.64:0/1139439271 shutdown_connections 2023-09-13T03:05:28.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:28.396+0000 7f0349910700 1 -- 172.21.15.64:0/1139439271 wait complete. 2023-09-13T03:05:29.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.148+0000 7f6e9af82700 1 -- 172.21.15.64:0/1839755553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e94102870 msgr2=0x7f6e94102cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:29.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.148+0000 7f6e9af82700 1 --2- 172.21.15.64:0/1839755553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e94102870 0x7f6e94102cf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6e80009990 tx=0x7f6e80009ca0 comp rx=0 tx=0).stop 2023-09-13T03:05:29.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.148+0000 7f6e9af82700 1 -- 172.21.15.64:0/1839755553 shutdown_connections 2023-09-13T03:05:29.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.148+0000 7f6e9af82700 1 --2- 172.21.15.64:0/1839755553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e94108870 0x7f6e94108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:29.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.148+0000 7f6e9af82700 1 --2- 172.21.15.64:0/1839755553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e94103230 0x7f6e9410f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:29.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.148+0000 7f6e9af82700 1 --2- 172.21.15.64:0/1839755553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e94102870 0x7f6e94102cf0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:29.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.148+0000 7f6e9af82700 1 -- 172.21.15.64:0/1839755553 >> 172.21.15.64:0/1839755553 conn(0x7f6e940fe340 msgr2=0x7f6e94100760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:05:29.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.148+0000 7f6e9af82700 1 -- 172.21.15.64:0/1839755553 shutdown_connections 2023-09-13T03:05:29.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.148+0000 7f6e9af82700 1 -- 172.21.15.64:0/1839755553 wait complete. 2023-09-13T03:05:29.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.149+0000 7f6e9af82700 1 Processor -- start 2023-09-13T03:05:29.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.149+0000 7f6e9af82700 1 -- start start 2023-09-13T03:05:29.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.150+0000 7f6e9af82700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e94102870 0x7f6e94197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:29.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.150+0000 7f6e9af82700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e94103230 0x7f6e94198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:29.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.150+0000 7f6e9af82700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e94108870 0x7f6e9419c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:29.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.150+0000 7f6e9af82700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9419ced0 con 0x7f6e94108870 2023-09-13T03:05:29.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.150+0000 7f6e9af82700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9419d040 con 0x7f6e94103230 2023-09-13T03:05:29.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.150+0000 7f6e9af82700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9419d200 con 0x7f6e94102870 2023-09-13T03:05:29.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.150+0000 7f6e98d1e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e94102870 0x7f6e94197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:29.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.150+0000 7f6e98d1e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e94102870 0x7f6e94197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:59754/0 (socket says 172.21.15.64:59754) 2023-09-13T03:05:29.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.150+0000 7f6e98d1e700 1 -- 172.21.15.64:0/3975066532 learned_addr learned my addr 172.21.15.64:0/3975066532 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:05:29.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.150+0000 7f6e93fff700 1 --2- 172.21.15.64:0/3975066532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e94103230 0x7f6e94198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:29.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.150+0000 7f6e9951f700 1 --2- 172.21.15.64:0/3975066532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e94108870 0x7f6e9419c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:29.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.151+0000 7f6e93fff700 1 -- 172.21.15.64:0/3975066532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e94102870 msgr2=0x7f6e94197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:29.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.151+0000 7f6e93fff700 1 --2- 172.21.15.64:0/3975066532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e94102870 0x7f6e94197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:29.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.151+0000 7f6e93fff700 1 -- 172.21.15.64:0/3975066532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e94108870 msgr2=0x7f6e9419c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:29.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.151+0000 7f6e93fff700 1 --2- 172.21.15.64:0/3975066532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e94108870 0x7f6e9419c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:29.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.151+0000 7f6e93fff700 1 -- 172.21.15.64:0/3975066532 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e800095f0 con 0x7f6e94103230 2023-09-13T03:05:29.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.151+0000 7f6e93fff700 1 --2- 172.21.15.64:0/3975066532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e94103230 0x7f6e94198450 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6e800098e0 tx=0x7f6e80003920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:05:29.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.152+0000 7f6e91ffb700 1 -- 172.21.15.64:0/3975066532 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8001d070 con 0x7f6e94103230 2023-09-13T03:05:29.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.152+0000 7f6e91ffb700 1 -- 172.21.15.64:0/3975066532 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(23 keys) v1 ==== 1012+0+0 (secure 0 0 0) 0x7f6e8000bd20 con 0x7f6e94103230 2023-09-13T03:05:29.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.152+0000 7f6e91ffb700 1 -- 172.21.15.64:0/3975066532 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e80022bb0 con 0x7f6e94103230 2023-09-13T03:05:29.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.152+0000 7f6e9af82700 1 -- 172.21.15.64:0/3975066532 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e9419d480 con 0x7f6e94103230 2023-09-13T03:05:29.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.153+0000 7f6e9af82700 1 -- 172.21.15.64:0/3975066532 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e941a4b70 con 0x7f6e94103230 2023-09-13T03:05:29.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.154+0000 7f6e9af82700 1 -- 172.21.15.64:0/3975066532 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e9404e300 con 0x7f6e94103230 2023-09-13T03:05:29.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.154+0000 7f6e91ffb700 1 -- 172.21.15.64:0/3975066532 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7f6e80022d10 con 0x7f6e94103230 2023-09-13T03:05:29.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.155+0000 7f6e91ffb700 1 --2- 172.21.15.64:0/3975066532 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6e84038630 0x7f6e8403aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:29.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.155+0000 7f6e91ffb700 1 -- 172.21.15.64:0/3975066532 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6e8004cee0 con 0x7f6e94103230 2023-09-13T03:05:29.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.162+0000 7f6e98d1e700 1 --2- 172.21.15.64:0/3975066532 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6e84038630 0x7f6e8403aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:29.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.163+0000 7f6e98d1e700 1 --2- 172.21.15.64:0/3975066532 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6e84038630 0x7f6e8403aaf0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6e88009ca0 tx=0x7f6e88009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:05:29.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.166+0000 7f6e91ffb700 1 -- 172.21.15.64:0/3975066532 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6e80020370 con 0x7f6e94103230 2023-09-13T03:05:29.177 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:05:29.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.449+0000 7f6e9af82700 1 -- 172.21.15.64:0/3975066532 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) v1 -- 0x7f6e94199270 con 0x7f6e94103230 2023-09-13T03:05:29.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.454+0000 7f6e91ffb700 1 -- 172.21.15.64:0/3975066532 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v12)=0 v12) v1 ==== 108+0+0 (secure 0 0 0) 0x7f6e80027030 con 0x7f6e94103230 2023-09-13T03:05:29.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.456+0000 7f6e9af82700 1 -- 172.21.15.64:0/3975066532 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6e84038630 msgr2=0x7f6e8403aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:29.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.456+0000 7f6e9af82700 1 --2- 172.21.15.64:0/3975066532 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6e84038630 0x7f6e8403aaf0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6e88009ca0 tx=0x7f6e88009260 comp rx=0 tx=0).stop 2023-09-13T03:05:29.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.456+0000 7f6e9af82700 1 -- 172.21.15.64:0/3975066532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e94103230 msgr2=0x7f6e94198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:29.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.456+0000 7f6e9af82700 1 --2- 172.21.15.64:0/3975066532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e94103230 0x7f6e94198450 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6e800098e0 tx=0x7f6e80003920 comp rx=0 tx=0).stop 2023-09-13T03:05:29.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.456+0000 7f6e9af82700 1 -- 172.21.15.64:0/3975066532 shutdown_connections 2023-09-13T03:05:29.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.456+0000 7f6e9af82700 1 --2- 172.21.15.64:0/3975066532 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6e84038630 0x7f6e8403aaf0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:29.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.456+0000 7f6e9af82700 1 --2- 172.21.15.64:0/3975066532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e94108870 0x7f6e9419c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:29.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.456+0000 7f6e9af82700 1 --2- 172.21.15.64:0/3975066532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e94102870 0x7f6e94197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:29.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.456+0000 7f6e9af82700 1 --2- 172.21.15.64:0/3975066532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e94103230 0x7f6e94198450 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:29.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.456+0000 7f6e9af82700 1 -- 172.21.15.64:0/3975066532 >> 172.21.15.64:0/3975066532 conn(0x7f6e940fe340 msgr2=0x7f6e940fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:05:29.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.457+0000 7f6e9af82700 1 -- 172.21.15.64:0/3975066532 shutdown_connections 2023-09-13T03:05:29.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.457+0000 7f6e9af82700 1 -- 172.21.15.64:0/3975066532 wait complete. 2023-09-13T03:05:29.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:29 smithi170 ceph-mon[134328]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:29.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:29 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/1139439271' entity='client.admin' 2023-09-13T03:05:29.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:29 smithi064 ceph-mon[97675]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:29.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:29 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/1139439271' entity='client.admin' 2023-09-13T03:05:29.675 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:05:29.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:29 smithi142 ceph-mon[134082]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:29.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:29 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/1139439271' entity='client.admin' 2023-09-13T03:05:29.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.628+0000 7fcf32203700 1 -- 172.21.15.64:0/670507153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf2c100fe0 msgr2=0x7fcf2c10d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:29.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.628+0000 7fcf32203700 1 --2- 172.21.15.64:0/670507153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf2c100fe0 0x7fcf2c10d4f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcf2400b0a0 tx=0x7fcf2400b3b0 comp rx=0 tx=0).stop 2023-09-13T03:05:29.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.628+0000 7fcf32203700 1 -- 172.21.15.64:0/670507153 shutdown_connections 2023-09-13T03:05:29.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.628+0000 7fcf32203700 1 --2- 172.21.15.64:0/670507153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf2c100fe0 0x7fcf2c10d4f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:29.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.628+0000 7fcf32203700 1 --2- 172.21.15.64:0/670507153 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf2c106620 0x7fcf2c106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:29.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.628+0000 7fcf32203700 1 --2- 172.21.15.64:0/670507153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf2c100620 0x7fcf2c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:29.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.628+0000 7fcf32203700 1 -- 172.21.15.64:0/670507153 >> 172.21.15.64:0/670507153 conn(0x7fcf2c0fc0f0 msgr2=0x7fcf2c0fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:05:29.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.628+0000 7fcf32203700 1 -- 172.21.15.64:0/670507153 shutdown_connections 2023-09-13T03:05:29.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.628+0000 7fcf32203700 1 -- 172.21.15.64:0/670507153 wait complete. 2023-09-13T03:05:29.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.629+0000 7fcf32203700 1 Processor -- start 2023-09-13T03:05:29.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.629+0000 7fcf32203700 1 -- start start 2023-09-13T03:05:29.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.629+0000 7fcf32203700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf2c100620 0x7fcf2c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:29.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.629+0000 7fcf32203700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf2c100fe0 0x7fcf2c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:29.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.629+0000 7fcf32203700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf2c106620 0x7fcf2c0733c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:29.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.629+0000 7fcf32203700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2c1a4210 con 0x7fcf2c106620 2023-09-13T03:05:29.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.629+0000 7fcf32203700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2c1a4380 con 0x7fcf2c100fe0 2023-09-13T03:05:29.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.629+0000 7fcf32203700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2c1a4540 con 0x7fcf2c100620 2023-09-13T03:05:29.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.630+0000 7fcf2b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf2c100620 0x7fcf2c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:29.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.630+0000 7fcf2bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf2c106620 0x7fcf2c0733c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:29.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.630+0000 7fcf2b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf2c100620 0x7fcf2c072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:59778/0 (socket says 172.21.15.64:59778) 2023-09-13T03:05:29.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.630+0000 7fcf2b7fe700 1 -- 172.21.15.64:0/1080987866 learned_addr learned my addr 172.21.15.64:0/1080987866 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:05:29.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.630+0000 7fcf2affd700 1 --2- 172.21.15.64:0/1080987866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf2c100fe0 0x7fcf2c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:29.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.630+0000 7fcf2b7fe700 1 -- 172.21.15.64:0/1080987866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf2c100fe0 msgr2=0x7fcf2c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:29.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.630+0000 7fcf2b7fe700 1 --2- 172.21.15.64:0/1080987866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf2c100fe0 0x7fcf2c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:29.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.630+0000 7fcf2b7fe700 1 -- 172.21.15.64:0/1080987866 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf2c106620 msgr2=0x7fcf2c0733c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:29.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.630+0000 7fcf2b7fe700 1 --2- 172.21.15.64:0/1080987866 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf2c106620 0x7fcf2c0733c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:29.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.630+0000 7fcf2b7fe700 1 -- 172.21.15.64:0/1080987866 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf24009d00 con 0x7fcf2c100620 2023-09-13T03:05:29.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.630+0000 7fcf2b7fe700 1 --2- 172.21.15.64:0/1080987866 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf2c100620 0x7fcf2c072900 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c00e9a0 tx=0x7fcf1c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:05:29.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.631+0000 7fcf28ff9700 1 -- 172.21.15.64:0/1080987866 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c009790 con 0x7fcf2c100620 2023-09-13T03:05:29.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.631+0000 7fcf28ff9700 1 -- 172.21.15.64:0/1080987866 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(23 keys) v1 ==== 1012+0+0 (secure 0 0 0) 0x7fcf1c00cbd0 con 0x7fcf2c100620 2023-09-13T03:05:29.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.631+0000 7fcf32203700 1 -- 172.21.15.64:0/1080987866 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf2c1a4820 con 0x7fcf2c100620 2023-09-13T03:05:29.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.631+0000 7fcf28ff9700 1 -- 172.21.15.64:0/1080987866 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c018420 con 0x7fcf2c100620 2023-09-13T03:05:29.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.631+0000 7fcf32203700 1 -- 172.21.15.64:0/1080987866 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf2c1a4d70 con 0x7fcf2c100620 2023-09-13T03:05:29.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.632+0000 7fcf28ff9700 1 -- 172.21.15.64:0/1080987866 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7fcf1c018580 con 0x7fcf2c100620 2023-09-13T03:05:29.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.632+0000 7fcf32203700 1 -- 172.21.15.64:0/1080987866 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf2c04e300 con 0x7fcf2c100620 2023-09-13T03:05:29.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.632+0000 7fcf28ff9700 1 --2- 172.21.15.64:0/1080987866 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fcf14038630 0x7fcf1403aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:29.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.632+0000 7fcf28ff9700 1 -- 172.21.15.64:0/1080987866 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fcf1c014070 con 0x7fcf2c100620 2023-09-13T03:05:29.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.632+0000 7fcf2affd700 1 --2- 172.21.15.64:0/1080987866 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fcf14038630 0x7fcf1403aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:29.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.633+0000 7fcf2affd700 1 --2- 172.21.15.64:0/1080987866 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fcf14038630 0x7fcf1403aaf0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c0740f0 tx=0x7fcf20009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:05:29.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.638+0000 7fcf28ff9700 1 -- 172.21.15.64:0/1080987866 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fcf1c01f8e0 con 0x7fcf2c100620 2023-09-13T03:05:29.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.921+0000 7fcf32203700 1 -- 172.21.15.64:0/1080987866 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) v1 -- 0x7fcf2c073e30 con 0x7fcf2c100620 2023-09-13T03:05:29.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.931+0000 7fcf28ff9700 1 -- 172.21.15.64:0/1080987866 <== mon.1 v2:172.21.15.170:3300/0 7 ==== config(24 keys) v1 ==== 1043+0+0 (secure 0 0 0) 0x7fcf1c01f8e0 con 0x7fcf2c100620 2023-09-13T03:05:29.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.932+0000 7fcf28ff9700 1 -- 172.21.15.64:0/1080987866 <== mon.1 v2:172.21.15.170:3300/0 8 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v13)=0 v13) v1 ==== 123+0+0 (secure 0 0 0) 0x7fcf1c01f310 con 0x7fcf2c100620 2023-09-13T03:05:29.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.936+0000 7fcf32203700 1 -- 172.21.15.64:0/1080987866 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fcf14038630 msgr2=0x7fcf1403aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:29.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.936+0000 7fcf32203700 1 --2- 172.21.15.64:0/1080987866 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fcf14038630 0x7fcf1403aaf0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c0740f0 tx=0x7fcf20009260 comp rx=0 tx=0).stop 2023-09-13T03:05:29.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.936+0000 7fcf32203700 1 -- 172.21.15.64:0/1080987866 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf2c100620 msgr2=0x7fcf2c072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:29.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.936+0000 7fcf32203700 1 --2- 172.21.15.64:0/1080987866 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf2c100620 0x7fcf2c072900 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c00e9a0 tx=0x7fcf1c00ecb0 comp rx=0 tx=0).stop 2023-09-13T03:05:29.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.937+0000 7fcf32203700 1 -- 172.21.15.64:0/1080987866 shutdown_connections 2023-09-13T03:05:29.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.937+0000 7fcf32203700 1 --2- 172.21.15.64:0/1080987866 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fcf14038630 0x7fcf1403aaf0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:29.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.937+0000 7fcf32203700 1 --2- 172.21.15.64:0/1080987866 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf2c106620 0x7fcf2c0733c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:29.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.937+0000 7fcf32203700 1 --2- 172.21.15.64:0/1080987866 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf2c100620 0x7fcf2c072900 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:29.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.937+0000 7fcf32203700 1 --2- 172.21.15.64:0/1080987866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf2c100fe0 0x7fcf2c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:29.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.937+0000 7fcf32203700 1 -- 172.21.15.64:0/1080987866 >> 172.21.15.64:0/1080987866 conn(0x7fcf2c0fc0f0 msgr2=0x7fcf2c0fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:05:29.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.937+0000 7fcf32203700 1 -- 172.21.15.64:0/1080987866 shutdown_connections 2023-09-13T03:05:29.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.937+0000 7fcf32203700 1 -- 172.21.15.64:0/1080987866 wait complete. 2023-09-13T03:05:30.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.914+0000 7fcfda573700 1 -- 172.21.15.64:0/1696666149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfd4106620 msgr2=0x7fcfd4106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:30.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.914+0000 7fcfda573700 1 --2- 172.21.15.64:0/1696666149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfd4106620 0x7fcfd4106a00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcfc40098f0 tx=0x7fcfc4009c00 comp rx=0 tx=0).stop 2023-09-13T03:05:30.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.915+0000 7fcfda573700 1 -- 172.21.15.64:0/1696666149 shutdown_connections 2023-09-13T03:05:30.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.915+0000 7fcfda573700 1 --2- 172.21.15.64:0/1696666149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfd4106620 0x7fcfd4106a00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:30.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.915+0000 7fcfda573700 1 --2- 172.21.15.64:0/1696666149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfd4100fe0 0x7fcfd410d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:30.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.915+0000 7fcfda573700 1 --2- 172.21.15.64:0/1696666149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfd4100620 0x7fcfd4100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:30.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.915+0000 7fcfda573700 1 -- 172.21.15.64:0/1696666149 >> 172.21.15.64:0/1696666149 conn(0x7fcfd40fc0f0 msgr2=0x7fcfd40fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:05:30.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.915+0000 7fcfda573700 1 -- 172.21.15.64:0/1696666149 shutdown_connections 2023-09-13T03:05:30.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.915+0000 7fcfda573700 1 -- 172.21.15.64:0/1696666149 wait complete. 2023-09-13T03:05:30.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.917+0000 7fcfda573700 1 Processor -- start 2023-09-13T03:05:30.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.917+0000 7fcfda573700 1 -- start start 2023-09-13T03:05:30.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.918+0000 7fcfda573700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfd4100620 0x7fcfd4198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:30.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.918+0000 7fcfda573700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfd4100fe0 0x7fcfd4198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:30.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.918+0000 7fcfda573700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfd4106620 0x7fcfd410a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:30.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.918+0000 7fcfda573700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd410a850 con 0x7fcfd4100620 2023-09-13T03:05:30.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.918+0000 7fcfda573700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd410a9c0 con 0x7fcfd4106620 2023-09-13T03:05:30.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.918+0000 7fcfda573700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd410ab80 con 0x7fcfd4100fe0 2023-09-13T03:05:30.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.918+0000 7fcfd37fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfd4100fe0 0x7fcfd4198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:30.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.918+0000 7fcfd37fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfd4100fe0 0x7fcfd4198d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:59798/0 (socket says 172.21.15.64:59798) 2023-09-13T03:05:30.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.918+0000 7fcfd37fe700 1 -- 172.21.15.64:0/1372068795 learned_addr learned my addr 172.21.15.64:0/1372068795 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:05:30.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.919+0000 7fcfd37fe700 1 -- 172.21.15.64:0/1372068795 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfd4106620 msgr2=0x7fcfd410a080 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-13T03:05:30.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.919+0000 7fcfd37fe700 1 --2- 172.21.15.64:0/1372068795 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfd4106620 0x7fcfd410a080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:30.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.919+0000 7fcfd37fe700 1 -- 172.21.15.64:0/1372068795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfd4100620 msgr2=0x7fcfd4198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:30.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.919+0000 7fcfd37fe700 1 --2- 172.21.15.64:0/1372068795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfd4100620 0x7fcfd4198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:30.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.919+0000 7fcfd37fe700 1 -- 172.21.15.64:0/1372068795 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc4009520 con 0x7fcfd4100fe0 2023-09-13T03:05:30.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.921+0000 7fcfd37fe700 1 --2- 172.21.15.64:0/1372068795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfd4100fe0 0x7fcfd4198d90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcfc800c9b0 tx=0x7fcfc800cd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:05:30.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.922+0000 7fcfd17fa700 1 -- 172.21.15.64:0/1372068795 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc8007b00 con 0x7fcfd4100fe0 2023-09-13T03:05:30.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.922+0000 7fcfd17fa700 1 -- 172.21.15.64:0/1372068795 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(23 keys) v1 ==== 1012+0+0 (secure 0 0 0) 0x7fcfc8007c60 con 0x7fcfd4100fe0 2023-09-13T03:05:30.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.922+0000 7fcfd17fa700 1 -- 172.21.15.64:0/1372068795 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc8018420 con 0x7fcfd4100fe0 2023-09-13T03:05:30.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.922+0000 7fcfda573700 1 -- 172.21.15.64:0/1372068795 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd410ae60 con 0x7fcfd4100fe0 2023-09-13T03:05:30.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.922+0000 7fcfda573700 1 -- 172.21.15.64:0/1372068795 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd419c840 con 0x7fcfd4100fe0 2023-09-13T03:05:30.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.924+0000 7fcfd17fa700 1 -- 172.21.15.64:0/1372068795 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7fcfc800f450 con 0x7fcfd4100fe0 2023-09-13T03:05:30.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.924+0000 7fcfda573700 1 -- 172.21.15.64:0/1372068795 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfd404e300 con 0x7fcfd4100fe0 2023-09-13T03:05:30.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.924+0000 7fcfd17fa700 1 --2- 172.21.15.64:0/1372068795 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fcfbc038630 0x7fcfbc03aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:30.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.925+0000 7fcfd17fa700 1 -- 172.21.15.64:0/1372068795 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fcfc804af00 con 0x7fcfd4100fe0 2023-09-13T03:05:30.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.925+0000 7fcfd3fff700 1 --2- 172.21.15.64:0/1372068795 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fcfbc038630 0x7fcfbc03aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:30.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.925+0000 7fcfd3fff700 1 --2- 172.21.15.64:0/1372068795 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fcfbc038630 0x7fcfbc03aaf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcfc400c010 tx=0x7fcfc4009f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:05:30.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.930+0000 7fcfd17fa700 1 -- 172.21.15.64:0/1372068795 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fcfc800dd10 con 0x7fcfd4100fe0 2023-09-13T03:05:30.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:29.931+0000 7fcfd17fa700 1 -- 172.21.15.64:0/1372068795 <== mon.1 v2:172.21.15.170:3300/0 7 ==== config(24 keys) v1 ==== 1043+0+0 (secure 0 0 0) 0x7fcfc800dd10 con 0x7fcfd4100fe0 2023-09-13T03:05:30.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.206+0000 7fcfda573700 1 -- 172.21.15.64:0/1372068795 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) v1 -- 0x7fcfd410f9d0 con 0x7fcfd4100fe0 2023-09-13T03:05:30.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.211+0000 7fcfd17fa700 1 -- 172.21.15.64:0/1372068795 <== mon.1 v2:172.21.15.170:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v14)=0 v14) v1 ==== 119+0+0 (secure 0 0 0) 0x7fcfc800eed0 con 0x7fcfd4100fe0 2023-09-13T03:05:30.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.213+0000 7fcfda573700 1 -- 172.21.15.64:0/1372068795 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fcfbc038630 msgr2=0x7fcfbc03aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:30.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.213+0000 7fcfda573700 1 --2- 172.21.15.64:0/1372068795 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fcfbc038630 0x7fcfbc03aaf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcfc400c010 tx=0x7fcfc4009f50 comp rx=0 tx=0).stop 2023-09-13T03:05:30.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.213+0000 7fcfda573700 1 -- 172.21.15.64:0/1372068795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfd4100fe0 msgr2=0x7fcfd4198d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:30.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.213+0000 7fcfda573700 1 --2- 172.21.15.64:0/1372068795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfd4100fe0 0x7fcfd4198d90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcfc800c9b0 tx=0x7fcfc800cd70 comp rx=0 tx=0).stop 2023-09-13T03:05:30.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.213+0000 7fcfda573700 1 -- 172.21.15.64:0/1372068795 shutdown_connections 2023-09-13T03:05:30.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.213+0000 7fcfda573700 1 --2- 172.21.15.64:0/1372068795 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fcfbc038630 0x7fcfbc03aaf0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:30.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.213+0000 7fcfda573700 1 --2- 172.21.15.64:0/1372068795 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfd4106620 0x7fcfd410a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:30.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.213+0000 7fcfda573700 1 --2- 172.21.15.64:0/1372068795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfd4100620 0x7fcfd4198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:30.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.213+0000 7fcfda573700 1 --2- 172.21.15.64:0/1372068795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfd4100fe0 0x7fcfd4198d90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:30.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.213+0000 7fcfda573700 1 -- 172.21.15.64:0/1372068795 >> 172.21.15.64:0/1372068795 conn(0x7fcfd40fc0f0 msgr2=0x7fcfd410b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:05:30.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.213+0000 7fcfda573700 1 -- 172.21.15.64:0/1372068795 shutdown_connections 2023-09-13T03:05:30.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.213+0000 7fcfda573700 1 -- 172.21.15.64:0/1372068795 wait complete. 2023-09-13T03:05:30.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.308+0000 7fc5c18cb700 1 -- 172.21.15.64:0/2527996353 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5bc068120 msgr2=0x7fc5bc0685a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:30.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.308+0000 7fc5c18cb700 1 --2- 172.21.15.64:0/2527996353 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5bc068120 0x7fc5bc0685a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0009990 tx=0x7fc5b0009ca0 comp rx=0 tx=0).stop 2023-09-13T03:05:30.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.308+0000 7fc5c18cb700 1 -- 172.21.15.64:0/2527996353 shutdown_connections 2023-09-13T03:05:30.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.308+0000 7fc5c18cb700 1 --2- 172.21.15.64:0/2527996353 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5bc106770 0x7fc5bc106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:30.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.308+0000 7fc5c18cb700 1 --2- 172.21.15.64:0/2527996353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5bc068ae0 0x7fc5bc10d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:30.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.308+0000 7fc5c18cb700 1 --2- 172.21.15.64:0/2527996353 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5bc068120 0x7fc5bc0685a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:30.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.308+0000 7fc5c18cb700 1 -- 172.21.15.64:0/2527996353 >> 172.21.15.64:0/2527996353 conn(0x7fc5bc076600 msgr2=0x7fc5bc076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:05:30.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.308+0000 7fc5c18cb700 1 -- 172.21.15.64:0/2527996353 shutdown_connections 2023-09-13T03:05:30.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.308+0000 7fc5c18cb700 1 -- 172.21.15.64:0/2527996353 wait complete. 2023-09-13T03:05:30.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.310+0000 7fc5c18cb700 1 Processor -- start 2023-09-13T03:05:30.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.310+0000 7fc5c18cb700 1 -- start start 2023-09-13T03:05:30.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.311+0000 7fc5c18cb700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5bc068120 0x7fc5bc0ff5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:30.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.311+0000 7fc5c18cb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5bc068ae0 0x7fc5bc0ffaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:30.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.311+0000 7fc5c18cb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5bc106770 0x7fc5bc100070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:30.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.311+0000 7fc5c18cb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc1a4a30 con 0x7fc5bc106770 2023-09-13T03:05:30.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.311+0000 7fc5c18cb700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc1a4ba0 con 0x7fc5bc068ae0 2023-09-13T03:05:30.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.311+0000 7fc5c18cb700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc1a4d60 con 0x7fc5bc068120 2023-09-13T03:05:30.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.311+0000 7fc5bb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5bc106770 0x7fc5bc100070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:30.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.311+0000 7fc5ba7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5bc068ae0 0x7fc5bc0ffaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:30.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.311+0000 7fc5ba7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5bc068ae0 0x7fc5bc0ffaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:52814/0 (socket says 172.21.15.64:52814) 2023-09-13T03:05:30.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.311+0000 7fc5baffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5bc068120 0x7fc5bc0ff5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:30.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.311+0000 7fc5ba7fc700 1 -- 172.21.15.64:0/655462074 learned_addr learned my addr 172.21.15.64:0/655462074 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:05:30.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.311+0000 7fc5bb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5bc106770 0x7fc5bc100070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58328/0 (socket says 172.21.15.64:58328) 2023-09-13T03:05:30.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.312+0000 7fc5bb7fe700 1 -- 172.21.15.64:0/655462074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5bc068120 msgr2=0x7fc5bc0ff5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:30.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.312+0000 7fc5bb7fe700 1 --2- 172.21.15.64:0/655462074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5bc068120 0x7fc5bc0ff5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:30.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.312+0000 7fc5bb7fe700 1 -- 172.21.15.64:0/655462074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5bc068ae0 msgr2=0x7fc5bc0ffaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:30.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.312+0000 7fc5bb7fe700 1 --2- 172.21.15.64:0/655462074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5bc068ae0 0x7fc5bc0ffaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:30.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.312+0000 7fc5bb7fe700 1 -- 172.21.15.64:0/655462074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b00095f0 con 0x7fc5bc106770 2023-09-13T03:05:30.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.312+0000 7fc5baffd700 1 --2- 172.21.15.64:0/655462074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5bc068120 0x7fc5bc0ff5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:05:30.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.312+0000 7fc5bb7fe700 1 --2- 172.21.15.64:0/655462074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5bc106770 0x7fc5bc100070 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc5b400efc0 tx=0x7fc5b400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:05:30.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.312+0000 7fc5c08c9700 1 -- 172.21.15.64:0/655462074 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b4018070 con 0x7fc5bc106770 2023-09-13T03:05:30.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.312+0000 7fc5c08c9700 1 -- 172.21.15.64:0/655462074 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(24 keys) v1 ==== 1043+0+0 (secure 0 0 0) 0x7fc5b400f040 con 0x7fc5bc106770 2023-09-13T03:05:30.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.312+0000 7fc5c18cb700 1 -- 172.21.15.64:0/655462074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5bc1a4fe0 con 0x7fc5bc106770 2023-09-13T03:05:30.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.312+0000 7fc5c08c9700 1 -- 172.21.15.64:0/655462074 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b400cae0 con 0x7fc5bc106770 2023-09-13T03:05:30.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.312+0000 7fc5c18cb700 1 -- 172.21.15.64:0/655462074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5bc1a5460 con 0x7fc5bc106770 2023-09-13T03:05:30.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.313+0000 7fc5c08c9700 1 -- 172.21.15.64:0/655462074 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7fc5b400cc60 con 0x7fc5bc106770 2023-09-13T03:05:30.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.314+0000 7fc5c18cb700 1 -- 172.21.15.64:0/655462074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5bc04e300 con 0x7fc5bc106770 2023-09-13T03:05:30.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.314+0000 7fc5c08c9700 1 --2- 172.21.15.64:0/655462074 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc5a4038630 0x7fc5a403aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:30.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.314+0000 7fc5c08c9700 1 -- 172.21.15.64:0/655462074 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc5b404aae0 con 0x7fc5bc106770 2023-09-13T03:05:30.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.314+0000 7fc5baffd700 1 --2- 172.21.15.64:0/655462074 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc5a4038630 0x7fc5a403aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:30.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.315+0000 7fc5baffd700 1 --2- 172.21.15.64:0/655462074 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc5a4038630 0x7fc5a403aaf0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac011490 tx=0x7fc5ac009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:05:30.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.320+0000 7fc5c08c9700 1 -- 172.21.15.64:0/655462074 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fc5b401cbf0 con 0x7fc5bc106770 2023-09-13T03:05:30.587 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:30 smithi064 ceph-mon[97675]: from='client.? ' entity='client.admin' 2023-09-13T03:05:30.588 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:30 smithi064 ceph-mon[97675]: from='client.? ' entity='client.admin' 2023-09-13T03:05:30.588 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:30 smithi064 ceph-mon[97675]: from='client.? ' entity='client.admin' 2023-09-13T03:05:30.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.591+0000 7fc5c18cb700 1 -- 172.21.15.64:0/655462074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) v1 -- 0x7fc5bc1104d0 con 0x7fc5bc106770 2023-09-13T03:05:30.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.606+0000 7fc5c08c9700 1 -- 172.21.15.64:0/655462074 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v15)=0 v15) v1 ==== 127+0+0 (secure 0 0 0) 0x7fc5b400f1b0 con 0x7fc5bc106770 2023-09-13T03:05:30.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.608+0000 7fc5c18cb700 1 -- 172.21.15.64:0/655462074 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc5a4038630 msgr2=0x7fc5a403aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:30.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.608+0000 7fc5c18cb700 1 --2- 172.21.15.64:0/655462074 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc5a4038630 0x7fc5a403aaf0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac011490 tx=0x7fc5ac009260 comp rx=0 tx=0).stop 2023-09-13T03:05:30.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.608+0000 7fc5c18cb700 1 -- 172.21.15.64:0/655462074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5bc106770 msgr2=0x7fc5bc100070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:30.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.608+0000 7fc5c18cb700 1 --2- 172.21.15.64:0/655462074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5bc106770 0x7fc5bc100070 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc5b400efc0 tx=0x7fc5b400c3a0 comp rx=0 tx=0).stop 2023-09-13T03:05:30.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.609+0000 7fc5c18cb700 1 -- 172.21.15.64:0/655462074 shutdown_connections 2023-09-13T03:05:30.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.609+0000 7fc5c18cb700 1 --2- 172.21.15.64:0/655462074 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc5a4038630 0x7fc5a403aaf0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:30.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.609+0000 7fc5c18cb700 1 --2- 172.21.15.64:0/655462074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5bc106770 0x7fc5bc100070 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:30.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.609+0000 7fc5c18cb700 1 --2- 172.21.15.64:0/655462074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5bc068120 0x7fc5bc0ff5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:30.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.609+0000 7fc5c18cb700 1 --2- 172.21.15.64:0/655462074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5bc068ae0 0x7fc5bc0ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:30.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.609+0000 7fc5c18cb700 1 -- 172.21.15.64:0/655462074 >> 172.21.15.64:0/655462074 conn(0x7fc5bc076600 msgr2=0x7fc5bc0fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:05:30.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.609+0000 7fc5c18cb700 1 -- 172.21.15.64:0/655462074 shutdown_connections 2023-09-13T03:05:30.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.609+0000 7fc5c18cb700 1 -- 172.21.15.64:0/655462074 wait complete. 2023-09-13T03:05:30.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:30 smithi142 ceph-mon[134082]: from='client.? ' entity='client.admin' 2023-09-13T03:05:30.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:30 smithi142 ceph-mon[134082]: from='client.? ' entity='client.admin' 2023-09-13T03:05:30.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:30 smithi142 ceph-mon[134082]: from='client.? ' entity='client.admin' 2023-09-13T03:05:30.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:30 smithi170 ceph-mon[134328]: from='client.? ' entity='client.admin' 2023-09-13T03:05:30.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:30 smithi170 ceph-mon[134328]: from='client.? ' entity='client.admin' 2023-09-13T03:05:30.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:30 smithi170 ceph-mon[134328]: from='client.? ' entity='client.admin' 2023-09-13T03:05:31.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.005+0000 7fb2c23cf700 1 -- 172.21.15.64:0/2241730890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2bc1055c0 msgr2=0x7fb2bc103050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:31.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.005+0000 7fb2c23cf700 1 --2- 172.21.15.64:0/2241730890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2bc1055c0 0x7fb2bc103050 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac009990 tx=0x7fb2ac009ca0 comp rx=0 tx=0).stop 2023-09-13T03:05:31.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.005+0000 7fb2c23cf700 1 -- 172.21.15.64:0/2241730890 shutdown_connections 2023-09-13T03:05:31.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.005+0000 7fb2c23cf700 1 --2- 172.21.15.64:0/2241730890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2bc1055c0 0x7fb2bc103050 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:31.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.005+0000 7fb2c23cf700 1 --2- 172.21.15.64:0/2241730890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2bc068120 0x7fb2bc0685a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:31.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.005+0000 7fb2c23cf700 1 --2- 172.21.15.64:0/2241730890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2bc103590 0x7fb2bc103a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:31.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.005+0000 7fb2c23cf700 1 -- 172.21.15.64:0/2241730890 >> 172.21.15.64:0/2241730890 conn(0x7fb2bc076600 msgr2=0x7fb2bc076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:05:31.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.005+0000 7fb2c23cf700 1 -- 172.21.15.64:0/2241730890 shutdown_connections 2023-09-13T03:05:31.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.005+0000 7fb2c23cf700 1 -- 172.21.15.64:0/2241730890 wait complete. 2023-09-13T03:05:31.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.007+0000 7fb2c23cf700 1 Processor -- start 2023-09-13T03:05:31.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.007+0000 7fb2c23cf700 1 -- start start 2023-09-13T03:05:31.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.007+0000 7fb2c23cf700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2bc068120 0x7fb2bc1a28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:31.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.007+0000 7fb2c23cf700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2bc103590 0x7fb2bc1a2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:31.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.007+0000 7fb2c23cf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2bc1055c0 0x7fb2bc19c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:31.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.007+0000 7fb2c23cf700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc19cfa0 con 0x7fb2bc1055c0 2023-09-13T03:05:31.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.007+0000 7fb2c23cf700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc19d110 con 0x7fb2bc103590 2023-09-13T03:05:31.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.007+0000 7fb2c23cf700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc19d2d0 con 0x7fb2bc068120 2023-09-13T03:05:31.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.008+0000 7fb2c096c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2bc1055c0 0x7fb2bc19c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:31.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.008+0000 7fb2bbfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2bc068120 0x7fb2bc1a28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:31.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.008+0000 7fb2bbfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2bc068120 0x7fb2bc1a28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:59872/0 (socket says 172.21.15.64:59872) 2023-09-13T03:05:31.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.008+0000 7fb2bbfff700 1 -- 172.21.15.64:0/1792148298 learned_addr learned my addr 172.21.15.64:0/1792148298 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:05:31.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.008+0000 7fb2bb7fe700 1 --2- 172.21.15.64:0/1792148298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2bc103590 0x7fb2bc1a2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:31.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.008+0000 7fb2c096c700 1 -- 172.21.15.64:0/1792148298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2bc068120 msgr2=0x7fb2bc1a28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:31.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.008+0000 7fb2c096c700 1 --2- 172.21.15.64:0/1792148298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2bc068120 0x7fb2bc1a28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:31.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.008+0000 7fb2c096c700 1 -- 172.21.15.64:0/1792148298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2bc103590 msgr2=0x7fb2bc1a2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:31.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.008+0000 7fb2c096c700 1 --2- 172.21.15.64:0/1792148298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2bc103590 0x7fb2bc1a2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:31.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.008+0000 7fb2c096c700 1 -- 172.21.15.64:0/1792148298 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ac0095f0 con 0x7fb2bc1055c0 2023-09-13T03:05:31.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.009+0000 7fb2c096c700 1 --2- 172.21.15.64:0/1792148298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2bc1055c0 0x7fb2bc19c930 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb2b400cba0 tx=0x7fb2b400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:05:31.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.009+0000 7fb2b97fa700 1 -- 172.21.15.64:0/1792148298 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b400aa60 con 0x7fb2bc1055c0 2023-09-13T03:05:31.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.009+0000 7fb2b97fa700 1 -- 172.21.15.64:0/1792148298 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(24 keys) v1 ==== 1043+0+0 (secure 0 0 0) 0x7fb2b400abc0 con 0x7fb2bc1055c0 2023-09-13T03:05:31.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.009+0000 7fb2b97fa700 1 -- 172.21.15.64:0/1792148298 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4011380 con 0x7fb2bc1055c0 2023-09-13T03:05:31.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.009+0000 7fb2c23cf700 1 -- 172.21.15.64:0/1792148298 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2bc19d5b0 con 0x7fb2bc1055c0 2023-09-13T03:05:31.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.009+0000 7fb2c23cf700 1 -- 172.21.15.64:0/1792148298 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2bc1a98a0 con 0x7fb2bc1055c0 2023-09-13T03:05:31.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.011+0000 7fb2c23cf700 1 -- 172.21.15.64:0/1792148298 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2bc04e300 con 0x7fb2bc1055c0 2023-09-13T03:05:31.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.011+0000 7fb2b97fa700 1 -- 172.21.15.64:0/1792148298 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7fb2b4012420 con 0x7fb2bc1055c0 2023-09-13T03:05:31.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.012+0000 7fb2b97fa700 1 --2- 172.21.15.64:0/1792148298 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fb2a4038630 0x7fb2a403aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:31.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.012+0000 7fb2b97fa700 1 -- 172.21.15.64:0/1792148298 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb2b404aee0 con 0x7fb2bc1055c0 2023-09-13T03:05:31.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.012+0000 7fb2bbfff700 1 --2- 172.21.15.64:0/1792148298 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fb2a4038630 0x7fb2a403aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:31.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.013+0000 7fb2bbfff700 1 --2- 172.21.15.64:0/1792148298 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fb2a4038630 0x7fb2a403aaf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac005cd0 tx=0x7fb2ac005bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:05:31.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.017+0000 7fb2b97fa700 1 -- 172.21.15.64:0/1792148298 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb2b4022d40 con 0x7fb2bc1055c0 2023-09-13T03:05:31.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.273+0000 7fb2c23cf700 1 -- 172.21.15.64:0/1792148298 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) v1 -- 0x7fb2bc19de40 con 0x7fb2bc1055c0 2023-09-13T03:05:31.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.277+0000 7fb2b97fa700 1 -- 172.21.15.64:0/1792148298 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v17)=0 v17) v1 ==== 106+0+0 (secure 0 0 0) 0x7fb2b4016030 con 0x7fb2bc1055c0 2023-09-13T03:05:31.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.280+0000 7fb2c23cf700 1 -- 172.21.15.64:0/1792148298 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fb2a4038630 msgr2=0x7fb2a403aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:31.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.280+0000 7fb2c23cf700 1 --2- 172.21.15.64:0/1792148298 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fb2a4038630 0x7fb2a403aaf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac005cd0 tx=0x7fb2ac005bc0 comp rx=0 tx=0).stop 2023-09-13T03:05:31.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.280+0000 7fb2c23cf700 1 -- 172.21.15.64:0/1792148298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2bc1055c0 msgr2=0x7fb2bc19c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:31.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.280+0000 7fb2c23cf700 1 --2- 172.21.15.64:0/1792148298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2bc1055c0 0x7fb2bc19c930 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb2b400cba0 tx=0x7fb2b400ceb0 comp rx=0 tx=0).stop 2023-09-13T03:05:31.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.280+0000 7fb2c23cf700 1 -- 172.21.15.64:0/1792148298 shutdown_connections 2023-09-13T03:05:31.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.280+0000 7fb2c23cf700 1 --2- 172.21.15.64:0/1792148298 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fb2a4038630 0x7fb2a403aaf0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:31.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.280+0000 7fb2c23cf700 1 --2- 172.21.15.64:0/1792148298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2bc1055c0 0x7fb2bc19c930 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:31.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.280+0000 7fb2c23cf700 1 --2- 172.21.15.64:0/1792148298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2bc068120 0x7fb2bc1a28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:31.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.280+0000 7fb2c23cf700 1 --2- 172.21.15.64:0/1792148298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2bc103590 0x7fb2bc1a2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:31.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.280+0000 7fb2c23cf700 1 -- 172.21.15.64:0/1792148298 >> 172.21.15.64:0/1792148298 conn(0x7fb2bc076600 msgr2=0x7fb2bc0fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:05:31.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.280+0000 7fb2c23cf700 1 -- 172.21.15.64:0/1792148298 shutdown_connections 2023-09-13T03:05:31.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.280+0000 7fb2c23cf700 1 -- 172.21.15.64:0/1792148298 wait complete. 2023-09-13T03:05:31.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:31 smithi142 ceph-mon[134082]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:31.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:31 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/655462074' entity='client.admin' 2023-09-13T03:05:31.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:31 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/4117388592' entity='client.admin' 2023-09-13T03:05:31.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:31 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/1792148298' entity='client.admin' 2023-09-13T03:05:31.829 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:31 smithi064 ceph-mon[97675]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:31.830 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:31 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/655462074' entity='client.admin' 2023-09-13T03:05:31.830 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:31 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/4117388592' entity='client.admin' 2023-09-13T03:05:31.830 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:31 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/1792148298' entity='client.admin' 2023-09-13T03:05:31.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:31 smithi170 ceph-mon[134328]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:31.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:31 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/655462074' entity='client.admin' 2023-09-13T03:05:31.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:31 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/4117388592' entity='client.admin' 2023-09-13T03:05:31.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:31 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/1792148298' entity='client.admin' 2023-09-13T03:05:32.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.786+0000 7f72246f5700 1 -- 172.21.15.64:0/569060950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f721c100fe0 msgr2=0x7f721c10d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:32.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.786+0000 7f72246f5700 1 --2- 172.21.15.64:0/569060950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f721c100fe0 0x7f721c10d4f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f721400b0a0 tx=0x7f721400b3b0 comp rx=0 tx=0).stop 2023-09-13T03:05:32.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.786+0000 7f72246f5700 1 -- 172.21.15.64:0/569060950 shutdown_connections 2023-09-13T03:05:32.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.786+0000 7f72246f5700 1 --2- 172.21.15.64:0/569060950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f721c106620 0x7f721c106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:32.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.786+0000 7f72246f5700 1 --2- 172.21.15.64:0/569060950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f721c100fe0 0x7f721c10d4f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:32.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.786+0000 7f72246f5700 1 --2- 172.21.15.64:0/569060950 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f721c100620 0x7f721c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:32.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.786+0000 7f72246f5700 1 -- 172.21.15.64:0/569060950 >> 172.21.15.64:0/569060950 conn(0x7f721c0fc0f0 msgr2=0x7f721c0fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:05:32.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.786+0000 7f72246f5700 1 -- 172.21.15.64:0/569060950 shutdown_connections 2023-09-13T03:05:32.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.786+0000 7f72246f5700 1 -- 172.21.15.64:0/569060950 wait complete. 2023-09-13T03:05:32.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.787+0000 7f72246f5700 1 Processor -- start 2023-09-13T03:05:32.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.787+0000 7f72246f5700 1 -- start start 2023-09-13T03:05:32.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.788+0000 7f72246f5700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f721c100620 0x7f721c107570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:32.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.788+0000 7f72246f5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f721c100fe0 0x7f721c107ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:32.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.788+0000 7f72246f5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f721c106620 0x7f721c107ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:32.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.788+0000 7f72246f5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721c1a8fe0 con 0x7f721c100fe0 2023-09-13T03:05:32.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.788+0000 7f72246f5700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721c1a9150 con 0x7f721c100620 2023-09-13T03:05:32.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.788+0000 7f72246f5700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721c1a9310 con 0x7f721c106620 2023-09-13T03:05:32.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.788+0000 7f7222491700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f721c100620 0x7f721c107570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:32.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.788+0000 7f7222491700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f721c100620 0x7f721c107570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:52830/0 (socket says 172.21.15.64:52830) 2023-09-13T03:05:32.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.788+0000 7f7221c90700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f721c100fe0 0x7f721c107ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:32.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.788+0000 7f7222491700 1 -- 172.21.15.64:0/4117388592 learned_addr learned my addr 172.21.15.64:0/4117388592 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:05:32.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.788+0000 7f7222c92700 1 --2- 172.21.15.64:0/4117388592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f721c106620 0x7f721c107ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:32.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.789+0000 7f7221c90700 1 -- 172.21.15.64:0/4117388592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f721c106620 msgr2=0x7f721c107ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:32.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.789+0000 7f7221c90700 1 --2- 172.21.15.64:0/4117388592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f721c106620 0x7f721c107ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:32.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.789+0000 7f7221c90700 1 -- 172.21.15.64:0/4117388592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f721c100620 msgr2=0x7f721c107570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:32.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.789+0000 7f7221c90700 1 --2- 172.21.15.64:0/4117388592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f721c100620 0x7f721c107570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:32.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.789+0000 7f7221c90700 1 -- 172.21.15.64:0/4117388592 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7214009d00 con 0x7f721c100fe0 2023-09-13T03:05:32.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.789+0000 7f7222491700 1 --2- 172.21.15.64:0/4117388592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f721c100620 0x7f721c107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:05:32.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.789+0000 7f7221c90700 1 --2- 172.21.15.64:0/4117388592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f721c100fe0 0x7f721c107ab0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f720c00e9f0 tx=0x7f720c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:05:32.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.789+0000 7f72137fe700 1 -- 172.21.15.64:0/4117388592 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c004220 con 0x7f721c100fe0 2023-09-13T03:05:32.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.789+0000 7f72137fe700 1 -- 172.21.15.64:0/4117388592 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(24 keys) v1 ==== 1043+0+0 (secure 0 0 0) 0x7f720c004380 con 0x7f721c100fe0 2023-09-13T03:05:32.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.789+0000 7f72246f5700 1 -- 172.21.15.64:0/4117388592 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f721c1a95f0 con 0x7f721c100fe0 2023-09-13T03:05:32.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.789+0000 7f72137fe700 1 -- 172.21.15.64:0/4117388592 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c00cd10 con 0x7f721c100fe0 2023-09-13T03:05:32.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.790+0000 7f72246f5700 1 -- 172.21.15.64:0/4117388592 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f721c1a9af0 con 0x7f721c100fe0 2023-09-13T03:05:32.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.791+0000 7f72246f5700 1 -- 172.21.15.64:0/4117388592 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f721c04e300 con 0x7f721c100fe0 2023-09-13T03:05:32.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.791+0000 7f72137fe700 1 -- 172.21.15.64:0/4117388592 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7f720c004750 con 0x7f721c100fe0 2023-09-13T03:05:32.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.791+0000 7f72137fe700 1 --2- 172.21.15.64:0/4117388592 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f7208038630 0x7f720803aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:32.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.792+0000 7f72137fe700 1 -- 172.21.15.64:0/4117388592 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f720c014070 con 0x7f721c100fe0 2023-09-13T03:05:32.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.792+0000 7f7222491700 1 --2- 172.21.15.64:0/4117388592 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f7208038630 0x7f720803aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:32.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.792+0000 7f7222491700 1 --2- 172.21.15.64:0/4117388592 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f7208038630 0x7f720803aaf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7218011550 tx=0x7f7218009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:05:32.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:30.797+0000 7f72137fe700 1 -- 172.21.15.64:0/4117388592 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f720c01f990 con 0x7f721c100fe0 2023-09-13T03:05:32.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.042+0000 7f72246f5700 1 -- 172.21.15.64:0/4117388592 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) v1 -- 0x7f721c110390 con 0x7f721c100fe0 2023-09-13T03:05:32.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.046+0000 7f72137fe700 1 -- 172.21.15.64:0/4117388592 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v16)=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7f720c01f3a0 con 0x7f721c100fe0 2023-09-13T03:05:32.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.049+0000 7f72246f5700 1 -- 172.21.15.64:0/4117388592 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f7208038630 msgr2=0x7f720803aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:32.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.049+0000 7f72246f5700 1 --2- 172.21.15.64:0/4117388592 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f7208038630 0x7f720803aaf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7218011550 tx=0x7f7218009260 comp rx=0 tx=0).stop 2023-09-13T03:05:32.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.049+0000 7f72246f5700 1 -- 172.21.15.64:0/4117388592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f721c100fe0 msgr2=0x7f721c107ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:32.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.049+0000 7f72246f5700 1 --2- 172.21.15.64:0/4117388592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f721c100fe0 0x7f721c107ab0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f720c00e9f0 tx=0x7f720c00ed00 comp rx=0 tx=0).stop 2023-09-13T03:05:32.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.049+0000 7f72246f5700 1 -- 172.21.15.64:0/4117388592 shutdown_connections 2023-09-13T03:05:32.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.049+0000 7f72246f5700 1 --2- 172.21.15.64:0/4117388592 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f7208038630 0x7f720803aaf0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:32.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.049+0000 7f72246f5700 1 --2- 172.21.15.64:0/4117388592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f721c100620 0x7f721c107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:32.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.049+0000 7f72246f5700 1 --2- 172.21.15.64:0/4117388592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f721c106620 0x7f721c107ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:32.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.049+0000 7f72246f5700 1 --2- 172.21.15.64:0/4117388592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f721c100fe0 0x7f721c107ab0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:32.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.049+0000 7f72246f5700 1 -- 172.21.15.64:0/4117388592 >> 172.21.15.64:0/4117388592 conn(0x7f721c0fc0f0 msgr2=0x7f721c10baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:05:32.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.049+0000 7f72246f5700 1 -- 172.21.15.64:0/4117388592 shutdown_connections 2023-09-13T03:05:32.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:31.049+0000 7f72246f5700 1 -- 172.21.15.64:0/4117388592 wait complete. 2023-09-13T03:05:32.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.366+0000 7fc1c36e6700 1 -- 172.21.15.64:0/1930807083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1bc10ce60 msgr2=0x7fc1bc10f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:32.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.366+0000 7fc1c36e6700 1 --2- 172.21.15.64:0/1930807083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1bc10ce60 0x7fc1bc10f2d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc1b400b0a0 tx=0x7fc1b400b3b0 comp rx=0 tx=0).stop 2023-09-13T03:05:32.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.366+0000 7fc1c36e6700 1 -- 172.21.15.64:0/1930807083 shutdown_connections 2023-09-13T03:05:32.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.366+0000 7fc1c36e6700 1 --2- 172.21.15.64:0/1930807083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1bc072a30 0x7fc1bc072e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:32.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.366+0000 7fc1c36e6700 1 --2- 172.21.15.64:0/1930807083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1bc10ce60 0x7fc1bc10f2d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.366+0000 7fc1c36e6700 1 --2- 172.21.15.64:0/1930807083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1bc073350 0x7fc1bc10c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.367+0000 7fc1c36e6700 1 -- 172.21.15.64:0/1930807083 >> 172.21.15.64:0/1930807083 conn(0x7fc1bc0fc0f0 msgr2=0x7fc1bc0fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:05:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.367+0000 7fc1c36e6700 1 -- 172.21.15.64:0/1930807083 shutdown_connections 2023-09-13T03:05:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.367+0000 7fc1c36e6700 1 -- 172.21.15.64:0/1930807083 wait complete. 2023-09-13T03:05:32.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.368+0000 7fc1c36e6700 1 Processor -- start 2023-09-13T03:05:32.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.368+0000 7fc1c36e6700 1 -- start start 2023-09-13T03:05:32.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.368+0000 7fc1c36e6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1bc072a30 0x7fc1bc19e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:32.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.368+0000 7fc1c36e6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1bc073350 0x7fc1bc19eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:32.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.369+0000 7fc1c36e6700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1bc10ce60 0x7fc1bc1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:32.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.369+0000 7fc1c36e6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc198d50 con 0x7fc1bc072a30 2023-09-13T03:05:32.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.369+0000 7fc1c36e6700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc198ec0 con 0x7fc1bc10ce60 2023-09-13T03:05:32.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.369+0000 7fc1c36e6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc199080 con 0x7fc1bc073350 2023-09-13T03:05:32.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.369+0000 7fc1c1482700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1bc072a30 0x7fc1bc19e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:32.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.369+0000 7fc1c1482700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1bc072a30 0x7fc1bc19e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58392/0 (socket says 172.21.15.64:58392) 2023-09-13T03:05:32.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.369+0000 7fc1c1482700 1 -- 172.21.15.64:0/3677753502 learned_addr learned my addr 172.21.15.64:0/3677753502 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:05:32.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.369+0000 7fc1c0c81700 1 --2- 172.21.15.64:0/3677753502 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1bc073350 0x7fc1bc19eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:32.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.369+0000 7fc1c1c83700 1 --2- 172.21.15.64:0/3677753502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1bc10ce60 0x7fc1bc1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:32.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.369+0000 7fc1c1482700 1 -- 172.21.15.64:0/3677753502 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1bc073350 msgr2=0x7fc1bc19eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:32.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.369+0000 7fc1c1482700 1 --2- 172.21.15.64:0/3677753502 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1bc073350 0x7fc1bc19eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:32.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.369+0000 7fc1c1482700 1 -- 172.21.15.64:0/3677753502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1bc10ce60 msgr2=0x7fc1bc1986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:32.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.369+0000 7fc1c1482700 1 --2- 172.21.15.64:0/3677753502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1bc10ce60 0x7fc1bc1986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:32.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.369+0000 7fc1c1482700 1 -- 172.21.15.64:0/3677753502 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b8009520 con 0x7fc1bc072a30 2023-09-13T03:05:32.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.369+0000 7fc1c1c83700 1 --2- 172.21.15.64:0/3677753502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1bc10ce60 0x7fc1bc1986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:05:32.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.370+0000 7fc1c1482700 1 --2- 172.21.15.64:0/3677753502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1bc072a30 0x7fc1bc19e660 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc1b800eb30 tx=0x7fc1b800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:05:32.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.370+0000 7fc1b27fc700 1 -- 172.21.15.64:0/3677753502 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8009880 con 0x7fc1bc072a30 2023-09-13T03:05:32.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.370+0000 7fc1c36e6700 1 -- 172.21.15.64:0/3677753502 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1b4009d00 con 0x7fc1bc072a30 2023-09-13T03:05:32.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.370+0000 7fc1b27fc700 1 -- 172.21.15.64:0/3677753502 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(24 keys) v1 ==== 1043+0+0 (secure 0 0 0) 0x7fc1b80044a0 con 0x7fc1bc072a30 2023-09-13T03:05:32.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.370+0000 7fc1b27fc700 1 -- 172.21.15.64:0/3677753502 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8003c50 con 0x7fc1bc072a30 2023-09-13T03:05:32.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.370+0000 7fc1c36e6700 1 -- 172.21.15.64:0/3677753502 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1bc199550 con 0x7fc1bc072a30 2023-09-13T03:05:32.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.372+0000 7fc1c36e6700 1 -- 172.21.15.64:0/3677753502 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1bc04e300 con 0x7fc1bc072a30 2023-09-13T03:05:32.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.372+0000 7fc1b27fc700 1 -- 172.21.15.64:0/3677753502 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7fc1b800cd40 con 0x7fc1bc072a30 2023-09-13T03:05:32.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.373+0000 7fc1b27fc700 1 --2- 172.21.15.64:0/3677753502 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc1a8038630 0x7fc1a803aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:32.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.373+0000 7fc1b27fc700 1 -- 172.21.15.64:0/3677753502 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc1b8014070 con 0x7fc1bc072a30 2023-09-13T03:05:32.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.373+0000 7fc1c0c81700 1 --2- 172.21.15.64:0/3677753502 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc1a8038630 0x7fc1a803aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:32.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.374+0000 7fc1c0c81700 1 --2- 172.21.15.64:0/3677753502 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc1a8038630 0x7fc1a803aaf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc199b50 tx=0x7fc1ac009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:05:32.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.379+0000 7fc1b27fc700 1 -- 172.21.15.64:0/3677753502 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fc1b8020960 con 0x7fc1bc072a30 2023-09-13T03:05:32.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.658+0000 7fc1c36e6700 1 -- 172.21.15.64:0/3677753502 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) v1 -- 0x7fc1bc199bf0 con 0x7fc1bc072a30 2023-09-13T03:05:32.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.662+0000 7fc1b27fc700 1 -- 172.21.15.64:0/3677753502 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v18)=0 v18) v1 ==== 123+0+0 (secure 0 0 0) 0x7fc1b8020390 con 0x7fc1bc072a30 2023-09-13T03:05:32.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.663+0000 7fc1b27fc700 1 -- 172.21.15.64:0/3677753502 <== mon.0 v2:172.21.15.64:3300/0 8 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc1b8020390 con 0x7fc1bc072a30 2023-09-13T03:05:32.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.664+0000 7fc1c36e6700 1 -- 172.21.15.64:0/3677753502 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc1a8038630 msgr2=0x7fc1a803aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:32.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.664+0000 7fc1c36e6700 1 --2- 172.21.15.64:0/3677753502 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc1a8038630 0x7fc1a803aaf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc199b50 tx=0x7fc1ac009260 comp rx=0 tx=0).stop 2023-09-13T03:05:32.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.664+0000 7fc1c36e6700 1 -- 172.21.15.64:0/3677753502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1bc072a30 msgr2=0x7fc1bc19e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:32.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.664+0000 7fc1c36e6700 1 --2- 172.21.15.64:0/3677753502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1bc072a30 0x7fc1bc19e660 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc1b800eb30 tx=0x7fc1b800ee40 comp rx=0 tx=0).stop 2023-09-13T03:05:32.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.665+0000 7fc1c36e6700 1 -- 172.21.15.64:0/3677753502 shutdown_connections 2023-09-13T03:05:32.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.665+0000 7fc1c36e6700 1 --2- 172.21.15.64:0/3677753502 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc1a8038630 0x7fc1a803aaf0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:32.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.665+0000 7fc1c36e6700 1 --2- 172.21.15.64:0/3677753502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1bc10ce60 0x7fc1bc1986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:32.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.665+0000 7fc1c36e6700 1 --2- 172.21.15.64:0/3677753502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1bc072a30 0x7fc1bc19e660 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:32.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.665+0000 7fc1c36e6700 1 --2- 172.21.15.64:0/3677753502 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1bc073350 0x7fc1bc19eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:32.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.665+0000 7fc1c36e6700 1 -- 172.21.15.64:0/3677753502 >> 172.21.15.64:0/3677753502 conn(0x7fc1bc0fc0f0 msgr2=0x7fc1bc10f030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:05:32.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.665+0000 7fc1c36e6700 1 -- 172.21.15.64:0/3677753502 shutdown_connections 2023-09-13T03:05:32.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.665+0000 7fc1c36e6700 1 -- 172.21.15.64:0/3677753502 wait complete. 2023-09-13T03:05:32.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.676+0000 7f04adb6e700 1 -- 172.21.15.64:0/3784233419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04a80732c0 msgr2=0x7f04a81110f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:32.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.676+0000 7f04adb6e700 1 --2- 172.21.15.64:0/3784233419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04a80732c0 0x7f04a81110f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f049c009990 tx=0x7f049c009ca0 comp rx=0 tx=0).stop 2023-09-13T03:05:32.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.677+0000 7f04adb6e700 1 -- 172.21.15.64:0/3784233419 shutdown_connections 2023-09-13T03:05:32.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.677+0000 7f04adb6e700 1 --2- 172.21.15.64:0/3784233419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04a80729a0 0x7f04a8072d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:32.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.677+0000 7f04adb6e700 1 --2- 172.21.15.64:0/3784233419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04a8111630 0x7f04a8113a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:32.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.677+0000 7f04adb6e700 1 --2- 172.21.15.64:0/3784233419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04a80732c0 0x7f04a81110f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:32.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.677+0000 7f04adb6e700 1 -- 172.21.15.64:0/3784233419 >> 172.21.15.64:0/3784233419 conn(0x7f04a80fc0f0 msgr2=0x7f04a80fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:05:32.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.677+0000 7f04adb6e700 1 -- 172.21.15.64:0/3784233419 shutdown_connections 2023-09-13T03:05:32.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.677+0000 7f04adb6e700 1 -- 172.21.15.64:0/3784233419 wait complete. 2023-09-13T03:05:32.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.678+0000 7f04adb6e700 1 Processor -- start 2023-09-13T03:05:32.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.678+0000 7f04adb6e700 1 -- start start 2023-09-13T03:05:32.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.679+0000 7f04adb6e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04a80729a0 0x7f04a819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:32.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.679+0000 7f04adb6e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04a80732c0 0x7f04a819d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:32.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.679+0000 7f04adb6e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04a8111630 0x7f04a81a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:32.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.679+0000 7f04adb6e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a81a0bb0 con 0x7f04a80732c0 2023-09-13T03:05:32.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.679+0000 7f04adb6e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a81a0d20 con 0x7f04a8111630 2023-09-13T03:05:32.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.679+0000 7f04adb6e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a81a0ee0 con 0x7f04a80729a0 2023-09-13T03:05:32.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.679+0000 7f04a6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04a80732c0 0x7f04a819d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:32.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.679+0000 7f04a6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04a80732c0 0x7f04a819d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58402/0 (socket says 172.21.15.64:58402) 2023-09-13T03:05:32.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.679+0000 7f04a6ffd700 1 -- 172.21.15.64:0/499008397 learned_addr learned my addr 172.21.15.64:0/499008397 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:05:32.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.680+0000 7f04a7fff700 1 --2- 172.21.15.64:0/499008397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04a8111630 0x7f04a81a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:32.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.680+0000 7f04a77fe700 1 --2- 172.21.15.64:0/499008397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04a80729a0 0x7f04a819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:32.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.680+0000 7f04a6ffd700 1 -- 172.21.15.64:0/499008397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04a80729a0 msgr2=0x7f04a819ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:32.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.680+0000 7f04a6ffd700 1 --2- 172.21.15.64:0/499008397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04a80729a0 0x7f04a819ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:32.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.680+0000 7f04a6ffd700 1 -- 172.21.15.64:0/499008397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04a8111630 msgr2=0x7f04a81a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:32.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.680+0000 7f04a6ffd700 1 --2- 172.21.15.64:0/499008397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04a8111630 0x7f04a81a05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:32.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.680+0000 7f04a6ffd700 1 -- 172.21.15.64:0/499008397 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f049c0095f0 con 0x7f04a80732c0 2023-09-13T03:05:32.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.680+0000 7f04a77fe700 1 --2- 172.21.15.64:0/499008397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04a80729a0 0x7f04a819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:05:32.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.680+0000 7f04a6ffd700 1 --2- 172.21.15.64:0/499008397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04a80732c0 0x7f04a819d1f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f049c006010 tx=0x7f049c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:05:32.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.680+0000 7f04a4ff9700 1 -- 172.21.15.64:0/499008397 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049c01d070 con 0x7f04a80732c0 2023-09-13T03:05:32.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.681+0000 7f04a4ff9700 1 -- 172.21.15.64:0/499008397 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f049c022470 con 0x7f04a80732c0 2023-09-13T03:05:32.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.681+0000 7f04adb6e700 1 -- 172.21.15.64:0/499008397 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04a81a1160 con 0x7f04a80732c0 2023-09-13T03:05:32.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.681+0000 7f04a4ff9700 1 -- 172.21.15.64:0/499008397 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049c017650 con 0x7f04a80732c0 2023-09-13T03:05:32.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.681+0000 7f04adb6e700 1 -- 172.21.15.64:0/499008397 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04a81a9b10 con 0x7f04a80732c0 2023-09-13T03:05:32.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.682+0000 7f04adb6e700 1 -- 172.21.15.64:0/499008397 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04a8103b10 con 0x7f04a80732c0 2023-09-13T03:05:32.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.682+0000 7f04a4ff9700 1 -- 172.21.15.64:0/499008397 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7f049c022830 con 0x7f04a80732c0 2023-09-13T03:05:32.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.682+0000 7f04a4ff9700 1 --2- 172.21.15.64:0/499008397 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f0490038630 0x7f049003aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:32.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.682+0000 7f04a4ff9700 1 -- 172.21.15.64:0/499008397 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f049c04c0e0 con 0x7f04a80732c0 2023-09-13T03:05:32.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.683+0000 7f04a77fe700 1 --2- 172.21.15.64:0/499008397 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f0490038630 0x7f049003aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:32.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.683+0000 7f04a77fe700 1 --2- 172.21.15.64:0/499008397 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f0490038630 0x7f049003aaf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0498009cb0 tx=0x7f0498009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:05:32.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.688+0000 7f04a4ff9700 1 -- 172.21.15.64:0/499008397 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f049c016790 con 0x7f04a80732c0 2023-09-13T03:05:32.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.923+0000 7f04adb6e700 1 -- 172.21.15.64:0/499008397 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) v1 -- 0x7f04a819e0a0 con 0x7f04a80732c0 2023-09-13T03:05:32.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.926+0000 7f04a4ff9700 1 -- 172.21.15.64:0/499008397 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v19)=0 v19) v1 ==== 117+0+0 (secure 0 0 0) 0x7f049c027020 con 0x7f04a80732c0 2023-09-13T03:05:32.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.929+0000 7f04adb6e700 1 -- 172.21.15.64:0/499008397 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f0490038630 msgr2=0x7f049003aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:32.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.929+0000 7f04adb6e700 1 --2- 172.21.15.64:0/499008397 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f0490038630 0x7f049003aaf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0498009cb0 tx=0x7f0498009260 comp rx=0 tx=0).stop 2023-09-13T03:05:32.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.929+0000 7f04adb6e700 1 -- 172.21.15.64:0/499008397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04a80732c0 msgr2=0x7f04a819d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:32.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.929+0000 7f04adb6e700 1 --2- 172.21.15.64:0/499008397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04a80732c0 0x7f04a819d1f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f049c006010 tx=0x7f049c004730 comp rx=0 tx=0).stop 2023-09-13T03:05:32.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.929+0000 7f04adb6e700 1 -- 172.21.15.64:0/499008397 shutdown_connections 2023-09-13T03:05:32.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.929+0000 7f04adb6e700 1 --2- 172.21.15.64:0/499008397 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f0490038630 0x7f049003aaf0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:32.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.929+0000 7f04adb6e700 1 --2- 172.21.15.64:0/499008397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04a8111630 0x7f04a81a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:32.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.930+0000 7f04adb6e700 1 --2- 172.21.15.64:0/499008397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04a80729a0 0x7f04a819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:32.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.930+0000 7f04adb6e700 1 --2- 172.21.15.64:0/499008397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04a80732c0 0x7f04a819d1f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:32.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.930+0000 7f04adb6e700 1 -- 172.21.15.64:0/499008397 >> 172.21.15.64:0/499008397 conn(0x7f04a80fc0f0 msgr2=0x7f04a80fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:05:32.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.930+0000 7f04adb6e700 1 -- 172.21.15.64:0/499008397 shutdown_connections 2023-09-13T03:05:32.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:32.930+0000 7f04adb6e700 1 -- 172.21.15.64:0/499008397 wait complete. 2023-09-13T03:05:33.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:33 smithi064 ceph-mon[97675]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:33.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:33 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/3677753502' entity='client.admin' 2023-09-13T03:05:33.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:33 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/499008397' entity='client.admin' 2023-09-13T03:05:33.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:33 smithi142 ceph-mon[134082]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:33.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:33 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/3677753502' entity='client.admin' 2023-09-13T03:05:33.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:33 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/499008397' entity='client.admin' 2023-09-13T03:05:33.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:33 smithi170 ceph-mon[134328]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:33.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:33 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/3677753502' entity='client.admin' 2023-09-13T03:05:33.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:33 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/499008397' entity='client.admin' 2023-09-13T03:05:33.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:33.917+0000 7f484a7ca700 1 -- 172.21.15.64:0/259408819 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48440ff520 msgr2=0x7f48440ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:33.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:33.917+0000 7f484a7ca700 1 --2- 172.21.15.64:0/259408819 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48440ff520 0x7f48440ff9a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4838009990 tx=0x7f4838009ca0 comp rx=0 tx=0).stop 2023-09-13T03:05:33.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:33.917+0000 7f484a7ca700 1 -- 172.21.15.64:0/259408819 shutdown_connections 2023-09-13T03:05:33.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:33.917+0000 7f484a7ca700 1 --2- 172.21.15.64:0/259408819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4844104530 0x7f4844104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:33.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:33.917+0000 7f484a7ca700 1 --2- 172.21.15.64:0/259408819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48440ffee0 0x7f484410c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:33.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:33.917+0000 7f484a7ca700 1 --2- 172.21.15.64:0/259408819 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48440ff520 0x7f48440ff9a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:33.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:33.917+0000 7f484a7ca700 1 -- 172.21.15.64:0/259408819 >> 172.21.15.64:0/259408819 conn(0x7f4844076600 msgr2=0x7f4844076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:05:33.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:33.917+0000 7f484a7ca700 1 -- 172.21.15.64:0/259408819 shutdown_connections 2023-09-13T03:05:33.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:33.918+0000 7f484a7ca700 1 -- 172.21.15.64:0/259408819 wait complete. 2023-09-13T03:05:33.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:33.919+0000 7f484a7ca700 1 Processor -- start 2023-09-13T03:05:33.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:33.919+0000 7f484a7ca700 1 -- start start 2023-09-13T03:05:33.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:33.919+0000 7f484a7ca700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48440ff520 0x7f484419e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:33.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:33.919+0000 7f484a7ca700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48440ffee0 0x7f484419eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:33.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:33.919+0000 7f484a7ca700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4844104530 0x7f48441986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:33.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:33.919+0000 7f484a7ca700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4844198d30 con 0x7f48440ff520 2023-09-13T03:05:33.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:33.919+0000 7f484a7ca700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4844198ea0 con 0x7f4844104530 2023-09-13T03:05:33.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:33.919+0000 7f484a7ca700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4844199060 con 0x7f48440ffee0 2023-09-13T03:05:33.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:33.920+0000 7f48437fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48440ffee0 0x7f484419eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:33.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:33.920+0000 7f48437fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48440ffee0 0x7f484419eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:59920/0 (socket says 172.21.15.64:59920) 2023-09-13T03:05:33.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:33.920+0000 7f48437fe700 1 -- 172.21.15.64:0/940294762 learned_addr learned my addr 172.21.15.64:0/940294762 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:05:33.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:33.920+0000 7f4848d67700 1 --2- 172.21.15.64:0/940294762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4844104530 0x7f48441986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:33.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:33.920+0000 7f48437fe700 1 -- 172.21.15.64:0/940294762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4844104530 msgr2=0x7f48441986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:33.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:33.920+0000 7f48437fe700 1 --2- 172.21.15.64:0/940294762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4844104530 0x7f48441986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:33.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:33.920+0000 7f48437fe700 1 -- 172.21.15.64:0/940294762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48440ff520 msgr2=0x7f484419e640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-13T03:05:33.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:33.920+0000 7f48437fe700 1 --2- 172.21.15.64:0/940294762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48440ff520 0x7f484419e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:33.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:33.921+0000 7f48437fe700 1 -- 172.21.15.64:0/940294762 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48380095f0 con 0x7f48440ffee0 2023-09-13T03:05:33.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:33.921+0000 7f48437fe700 1 --2- 172.21.15.64:0/940294762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48440ffee0 0x7f484419eb80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f483800b650 tx=0x7f4838004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:05:33.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:33.921+0000 7f48417fa700 1 -- 172.21.15.64:0/940294762 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483801d070 con 0x7f48440ffee0 2023-09-13T03:05:33.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:33.921+0000 7f48417fa700 1 -- 172.21.15.64:0/940294762 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4838022470 con 0x7f48440ffee0 2023-09-13T03:05:33.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:33.921+0000 7f48417fa700 1 -- 172.21.15.64:0/940294762 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4838017650 con 0x7f48440ffee0 2023-09-13T03:05:33.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:33.921+0000 7f484a7ca700 1 -- 172.21.15.64:0/940294762 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48441992e0 con 0x7f48440ffee0 2023-09-13T03:05:33.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:33.922+0000 7f484a7ca700 1 -- 172.21.15.64:0/940294762 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48441a5630 con 0x7f48440ffee0 2023-09-13T03:05:33.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:33.923+0000 7f484a7ca700 1 -- 172.21.15.64:0/940294762 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f484404e300 con 0x7f48440ffee0 2023-09-13T03:05:33.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:33.924+0000 7f48417fa700 1 -- 172.21.15.64:0/940294762 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7f483800fb10 con 0x7f48440ffee0 2023-09-13T03:05:33.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:33.924+0000 7f48417fa700 1 --2- 172.21.15.64:0/940294762 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f482c038630 0x7f482c03aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:33.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:33.924+0000 7f48417fa700 1 -- 172.21.15.64:0/940294762 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f483804be10 con 0x7f48440ffee0 2023-09-13T03:05:33.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:33.925+0000 7f4843fff700 1 --2- 172.21.15.64:0/940294762 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f482c038630 0x7f482c03aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:33.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:33.925+0000 7f4843fff700 1 --2- 172.21.15.64:0/940294762 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f482c038630 0x7f482c03aaf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4834005950 tx=0x7f48340058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:05:33.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:33.929+0000 7f48417fa700 1 -- 172.21.15.64:0/940294762 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f483800fdc0 con 0x7f48440ffee0 2023-09-13T03:05:34.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.169+0000 7f484a7ca700 1 -- 172.21.15.64:0/940294762 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) v1 -- 0x7f4844199bd0 con 0x7f48440ffee0 2023-09-13T03:05:34.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.175+0000 7f48417fa700 1 -- 172.21.15.64:0/940294762 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v20)=0 v20) v1 ==== 116+0+0 (secure 0 0 0) 0x7f4838027030 con 0x7f48440ffee0 2023-09-13T03:05:34.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.177+0000 7f484a7ca700 1 -- 172.21.15.64:0/940294762 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f482c038630 msgr2=0x7f482c03aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:34.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.177+0000 7f484a7ca700 1 --2- 172.21.15.64:0/940294762 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f482c038630 0x7f482c03aaf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4834005950 tx=0x7f48340058e0 comp rx=0 tx=0).stop 2023-09-13T03:05:34.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.177+0000 7f484a7ca700 1 -- 172.21.15.64:0/940294762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48440ffee0 msgr2=0x7f484419eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:34.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.177+0000 7f484a7ca700 1 --2- 172.21.15.64:0/940294762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48440ffee0 0x7f484419eb80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f483800b650 tx=0x7f4838004730 comp rx=0 tx=0).stop 2023-09-13T03:05:34.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.177+0000 7f484a7ca700 1 -- 172.21.15.64:0/940294762 shutdown_connections 2023-09-13T03:05:34.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.177+0000 7f484a7ca700 1 --2- 172.21.15.64:0/940294762 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f482c038630 0x7f482c03aaf0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:34.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.177+0000 7f484a7ca700 1 --2- 172.21.15.64:0/940294762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4844104530 0x7f48441986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:34.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.177+0000 7f484a7ca700 1 --2- 172.21.15.64:0/940294762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48440ff520 0x7f484419e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:34.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.177+0000 7f484a7ca700 1 --2- 172.21.15.64:0/940294762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48440ffee0 0x7f484419eb80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:34.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.177+0000 7f484a7ca700 1 -- 172.21.15.64:0/940294762 >> 172.21.15.64:0/940294762 conn(0x7f4844076600 msgr2=0x7f48440680c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:05:34.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.178+0000 7f484a7ca700 1 -- 172.21.15.64:0/940294762 shutdown_connections 2023-09-13T03:05:34.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.178+0000 7f484a7ca700 1 -- 172.21.15.64:0/940294762 wait complete. 2023-09-13T03:05:34.716 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:05:35.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.026+0000 7ff4ea0e1700 1 -- 172.21.15.64:0/2114070776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4e4103590 msgr2=0x7ff4e4103a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:35.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.026+0000 7ff4ea0e1700 1 --2- 172.21.15.64:0/2114070776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4e4103590 0x7ff4e4103a10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8009990 tx=0x7ff4d8009ca0 comp rx=0 tx=0).stop 2023-09-13T03:05:35.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.027+0000 7ff4ea0e1700 1 -- 172.21.15.64:0/2114070776 shutdown_connections 2023-09-13T03:05:35.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.027+0000 7ff4ea0e1700 1 --2- 172.21.15.64:0/2114070776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4e4068120 0x7ff4e40685a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:35.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.027+0000 7ff4ea0e1700 1 --2- 172.21.15.64:0/2114070776 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4e41055c0 0x7ff4e4103050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:35.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.027+0000 7ff4ea0e1700 1 --2- 172.21.15.64:0/2114070776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4e4103590 0x7ff4e4103a10 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:35.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.027+0000 7ff4ea0e1700 1 -- 172.21.15.64:0/2114070776 >> 172.21.15.64:0/2114070776 conn(0x7ff4e4076600 msgr2=0x7ff4e4076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:05:35.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.027+0000 7ff4ea0e1700 1 -- 172.21.15.64:0/2114070776 shutdown_connections 2023-09-13T03:05:35.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.027+0000 7ff4ea0e1700 1 -- 172.21.15.64:0/2114070776 wait complete. 2023-09-13T03:05:35.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.028+0000 7ff4ea0e1700 1 Processor -- start 2023-09-13T03:05:35.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.028+0000 7ff4ea0e1700 1 -- start start 2023-09-13T03:05:35.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.029+0000 7ff4ea0e1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4e4068120 0x7ff4e4071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:35.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.029+0000 7ff4ea0e1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4e4103590 0x7ff4e4072000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:35.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.029+0000 7ff4ea0e1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4e41055c0 0x7ff4e406aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:35.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.029+0000 7ff4ea0e1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e406b1b0 con 0x7ff4e41055c0 2023-09-13T03:05:35.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.029+0000 7ff4ea0e1700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e406b320 con 0x7ff4e4103590 2023-09-13T03:05:35.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.029+0000 7ff4ea0e1700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e406b4e0 con 0x7ff4e4068120 2023-09-13T03:05:35.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.029+0000 7ff4e3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4e41055c0 0x7ff4e406aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:35.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.029+0000 7ff4e3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4e41055c0 0x7ff4e406aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58448/0 (socket says 172.21.15.64:58448) 2023-09-13T03:05:35.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.029+0000 7ff4e3fff700 1 -- 172.21.15.64:0/1291736381 learned_addr learned my addr 172.21.15.64:0/1291736381 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:05:35.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.029+0000 7ff4e2ffd700 1 --2- 172.21.15.64:0/1291736381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4e4103590 0x7ff4e4072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:35.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.030+0000 7ff4e2ffd700 1 -- 172.21.15.64:0/1291736381 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4e4068120 msgr2=0x7ff4e4071a80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-13T03:05:35.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.030+0000 7ff4e2ffd700 1 --2- 172.21.15.64:0/1291736381 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4e4068120 0x7ff4e4071a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:35.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.030+0000 7ff4e2ffd700 1 -- 172.21.15.64:0/1291736381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4e41055c0 msgr2=0x7ff4e406aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:35.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.030+0000 7ff4e2ffd700 1 --2- 172.21.15.64:0/1291736381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4e41055c0 0x7ff4e406aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:35.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.030+0000 7ff4e2ffd700 1 -- 172.21.15.64:0/1291736381 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d80095f0 con 0x7ff4e4103590 2023-09-13T03:05:35.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.030+0000 7ff4e3fff700 1 --2- 172.21.15.64:0/1291736381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4e41055c0 0x7ff4e406aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:05:35.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.030+0000 7ff4e2ffd700 1 --2- 172.21.15.64:0/1291736381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4e4103590 0x7ff4e4072000 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8006010 tx=0x7ff4d8005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:05:35.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.031+0000 7ff4e0ff9700 1 -- 172.21.15.64:0/1291736381 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d801d070 con 0x7ff4e4103590 2023-09-13T03:05:35.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.031+0000 7ff4e0ff9700 1 -- 172.21.15.64:0/1291736381 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff4d8003f50 con 0x7ff4e4103590 2023-09-13T03:05:35.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.031+0000 7ff4ea0e1700 1 -- 172.21.15.64:0/1291736381 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4e406b760 con 0x7ff4e4103590 2023-09-13T03:05:35.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.031+0000 7ff4e0ff9700 1 -- 172.21.15.64:0/1291736381 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d8022b00 con 0x7ff4e4103590 2023-09-13T03:05:35.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.031+0000 7ff4ea0e1700 1 -- 172.21.15.64:0/1291736381 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4e406f7f0 con 0x7ff4e4103590 2023-09-13T03:05:35.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.032+0000 7ff4ea0e1700 1 -- 172.21.15.64:0/1291736381 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4e404e300 con 0x7ff4e4103590 2023-09-13T03:05:35.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.033+0000 7ff4e0ff9700 1 -- 172.21.15.64:0/1291736381 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7ff4d8003ab0 con 0x7ff4e4103590 2023-09-13T03:05:35.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.034+0000 7ff4e0ff9700 1 --2- 172.21.15.64:0/1291736381 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ff4cc038630 0x7ff4cc03aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:35.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.034+0000 7ff4e0ff9700 1 -- 172.21.15.64:0/1291736381 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff4d804cc60 con 0x7ff4e4103590 2023-09-13T03:05:35.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.034+0000 7ff4e37fe700 1 --2- 172.21.15.64:0/1291736381 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ff4cc038630 0x7ff4cc03aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:35.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.035+0000 7ff4e37fe700 1 --2- 172.21.15.64:0/1291736381 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ff4cc038630 0x7ff4cc03aaf0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4009710 tx=0x7ff4d4006c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:05:35.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.038+0000 7ff4e0ff9700 1 -- 172.21.15.64:0/1291736381 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ff4d8003d60 con 0x7ff4e4103590 2023-09-13T03:05:35.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.163+0000 7ff4e0ff9700 1 -- 172.21.15.64:0/1291736381 <== mon.2 v2:172.21.15.142:3300/0 7 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4d8003d60 con 0x7ff4e4103590 2023-09-13T03:05:35.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.287+0000 7ff4ea0e1700 1 -- 172.21.15.64:0/1291736381 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) v1 -- 0x7ff4e406bfd0 con 0x7ff4e4103590 2023-09-13T03:05:35.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.294+0000 7ff4e0ff9700 1 -- 172.21.15.64:0/1291736381 <== mon.2 v2:172.21.15.142:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff4d8027020 con 0x7ff4e4103590 2023-09-13T03:05:35.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.297+0000 7ff4ea0e1700 1 -- 172.21.15.64:0/1291736381 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ff4cc038630 msgr2=0x7ff4cc03aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:35.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.297+0000 7ff4ea0e1700 1 --2- 172.21.15.64:0/1291736381 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ff4cc038630 0x7ff4cc03aaf0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4009710 tx=0x7ff4d4006c60 comp rx=0 tx=0).stop 2023-09-13T03:05:35.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.297+0000 7ff4ea0e1700 1 -- 172.21.15.64:0/1291736381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4e4103590 msgr2=0x7ff4e4072000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:35.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.297+0000 7ff4ea0e1700 1 --2- 172.21.15.64:0/1291736381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4e4103590 0x7ff4e4072000 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8006010 tx=0x7ff4d8005740 comp rx=0 tx=0).stop 2023-09-13T03:05:35.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.299+0000 7ff4ea0e1700 1 -- 172.21.15.64:0/1291736381 shutdown_connections 2023-09-13T03:05:35.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.299+0000 7ff4ea0e1700 1 --2- 172.21.15.64:0/1291736381 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ff4cc038630 0x7ff4cc03aaf0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:35.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.299+0000 7ff4ea0e1700 1 --2- 172.21.15.64:0/1291736381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4e41055c0 0x7ff4e406aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:35.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.299+0000 7ff4ea0e1700 1 --2- 172.21.15.64:0/1291736381 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4e4068120 0x7ff4e4071a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:35.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.299+0000 7ff4ea0e1700 1 --2- 172.21.15.64:0/1291736381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4e4103590 0x7ff4e4072000 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:35.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.299+0000 7ff4ea0e1700 1 -- 172.21.15.64:0/1291736381 >> 172.21.15.64:0/1291736381 conn(0x7ff4e4076600 msgr2=0x7ff4e40fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:05:35.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.302+0000 7ff4ea0e1700 1 -- 172.21.15.64:0/1291736381 shutdown_connections 2023-09-13T03:05:35.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.303+0000 7ff4ea0e1700 1 -- 172.21.15.64:0/1291736381 wait complete. 2023-09-13T03:05:35.467 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:35 smithi064 ceph-mon[97675]: from='client.? ' entity='client.admin' 2023-09-13T03:05:35.467 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:35 smithi064 ceph-mon[97675]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:35.467 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:35 smithi064 ceph-mon[97675]: from='client.? ' entity='client.admin' 2023-09-13T03:05:35.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:35 smithi142 ceph-mon[134082]: from='client.? ' entity='client.admin' 2023-09-13T03:05:35.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:35 smithi142 ceph-mon[134082]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:35.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:35 smithi142 ceph-mon[134082]: from='client.? ' entity='client.admin' 2023-09-13T03:05:35.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.882+0000 7fd74eb70700 1 -- 172.21.15.64:0/3510625937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd748068120 msgr2=0x7fd7480685a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:35.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.882+0000 7fd74eb70700 1 --2- 172.21.15.64:0/3510625937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd748068120 0x7fd7480685a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd7340098f0 tx=0x7fd734009c00 comp rx=0 tx=0).stop 2023-09-13T03:05:35.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.882+0000 7fd74eb70700 1 -- 172.21.15.64:0/3510625937 shutdown_connections 2023-09-13T03:05:35.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.882+0000 7fd74eb70700 1 --2- 172.21.15.64:0/3510625937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd748068ae0 0x7fd74810d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:35.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.882+0000 7fd74eb70700 1 --2- 172.21.15.64:0/3510625937 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd748106770 0x7fd748106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:35.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.882+0000 7fd74eb70700 1 --2- 172.21.15.64:0/3510625937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd748068120 0x7fd7480685a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:35.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.882+0000 7fd74eb70700 1 -- 172.21.15.64:0/3510625937 >> 172.21.15.64:0/3510625937 conn(0x7fd748076600 msgr2=0x7fd748076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:05:35.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.882+0000 7fd74eb70700 1 -- 172.21.15.64:0/3510625937 shutdown_connections 2023-09-13T03:05:35.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.882+0000 7fd74eb70700 1 -- 172.21.15.64:0/3510625937 wait complete. 2023-09-13T03:05:35.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.883+0000 7fd74eb70700 1 Processor -- start 2023-09-13T03:05:35.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.883+0000 7fd74eb70700 1 -- start start 2023-09-13T03:05:35.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.884+0000 7fd74eb70700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd748068120 0x7fd74819e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:35.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.884+0000 7fd74eb70700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd748068ae0 0x7fd74819eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:35.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.884+0000 7fd74eb70700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd748106770 0x7fd748198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:35.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.884+0000 7fd74eb70700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd748198df0 con 0x7fd748068120 2023-09-13T03:05:35.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.884+0000 7fd74eb70700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd748198f60 con 0x7fd748106770 2023-09-13T03:05:35.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.884+0000 7fd74eb70700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd748199120 con 0x7fd748068ae0 2023-09-13T03:05:35.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.884+0000 7fd74c90c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd748068120 0x7fd74819e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:35.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.884+0000 7fd74c90c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd748068120 0x7fd74819e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58434/0 (socket says 172.21.15.64:58434) 2023-09-13T03:05:35.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.884+0000 7fd74c90c700 1 -- 172.21.15.64:0/2257956230 learned_addr learned my addr 172.21.15.64:0/2257956230 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:05:35.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.884+0000 7fd747fff700 1 --2- 172.21.15.64:0/2257956230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd748068ae0 0x7fd74819eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:35.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.884+0000 7fd74d10d700 1 --2- 172.21.15.64:0/2257956230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd748106770 0x7fd748198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:35.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.885+0000 7fd747fff700 1 -- 172.21.15.64:0/2257956230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd748106770 msgr2=0x7fd748198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:35.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.885+0000 7fd747fff700 1 --2- 172.21.15.64:0/2257956230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd748106770 0x7fd748198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:35.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.885+0000 7fd747fff700 1 -- 172.21.15.64:0/2257956230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd748068120 msgr2=0x7fd74819e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:35.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.885+0000 7fd747fff700 1 --2- 172.21.15.64:0/2257956230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd748068120 0x7fd74819e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:35.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.885+0000 7fd747fff700 1 -- 172.21.15.64:0/2257956230 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd734009520 con 0x7fd748068ae0 2023-09-13T03:05:35.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.885+0000 7fd747fff700 1 --2- 172.21.15.64:0/2257956230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd748068ae0 0x7fd74819eb90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd734009810 tx=0x7fd734009490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:05:35.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.885+0000 7fd745ffb700 1 -- 172.21.15.64:0/2257956230 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73401c070 con 0x7fd748068ae0 2023-09-13T03:05:35.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.885+0000 7fd745ffb700 1 -- 172.21.15.64:0/2257956230 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd7340054e0 con 0x7fd748068ae0 2023-09-13T03:05:35.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.885+0000 7fd745ffb700 1 -- 172.21.15.64:0/2257956230 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd734003b90 con 0x7fd748068ae0 2023-09-13T03:05:35.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.885+0000 7fd74eb70700 1 -- 172.21.15.64:0/2257956230 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7481993a0 con 0x7fd748068ae0 2023-09-13T03:05:35.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.885+0000 7fd74eb70700 1 -- 172.21.15.64:0/2257956230 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7481a55d0 con 0x7fd748068ae0 2023-09-13T03:05:35.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.887+0000 7fd745ffb700 1 -- 172.21.15.64:0/2257956230 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7fd734003cf0 con 0x7fd748068ae0 2023-09-13T03:05:35.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.887+0000 7fd74eb70700 1 -- 172.21.15.64:0/2257956230 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd748199530 con 0x7fd748068ae0 2023-09-13T03:05:35.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.888+0000 7fd745ffb700 1 --2- 172.21.15.64:0/2257956230 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd738038630 0x7fd73803aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:35.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.888+0000 7fd745ffb700 1 -- 172.21.15.64:0/2257956230 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd73404baf0 con 0x7fd748068ae0 2023-09-13T03:05:35.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.888+0000 7fd74c90c700 1 --2- 172.21.15.64:0/2257956230 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd738038630 0x7fd73803aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:35.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.889+0000 7fd74c90c700 1 --2- 172.21.15.64:0/2257956230 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd738038630 0x7fd73803aaf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd73c005950 tx=0x7fd73c009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:05:35.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:34.892+0000 7fd745ffb700 1 -- 172.21.15.64:0/2257956230 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fd7340158d0 con 0x7fd748068ae0 2023-09-13T03:05:35.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.158+0000 7fd74eb70700 1 -- 172.21.15.64:0/2257956230 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) v1 -- 0x7fd748110560 con 0x7fd748068ae0 2023-09-13T03:05:35.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.163+0000 7fd745ffb700 1 -- 172.21.15.64:0/2257956230 <== mon.1 v2:172.21.15.170:3300/0 7 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7340158d0 con 0x7fd748068ae0 2023-09-13T03:05:35.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.163+0000 7fd745ffb700 1 -- 172.21.15.64:0/2257956230 <== mon.1 v2:172.21.15.170:3300/0 8 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v21)=0 v21) v1 ==== 123+0+0 (secure 0 0 0) 0x7fd734025020 con 0x7fd748068ae0 2023-09-13T03:05:35.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.167+0000 7fd74eb70700 1 -- 172.21.15.64:0/2257956230 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd738038630 msgr2=0x7fd73803aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:35.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.167+0000 7fd74eb70700 1 --2- 172.21.15.64:0/2257956230 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd738038630 0x7fd73803aaf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd73c005950 tx=0x7fd73c009310 comp rx=0 tx=0).stop 2023-09-13T03:05:35.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.167+0000 7fd74eb70700 1 -- 172.21.15.64:0/2257956230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd748068ae0 msgr2=0x7fd74819eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:35.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.167+0000 7fd74eb70700 1 --2- 172.21.15.64:0/2257956230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd748068ae0 0x7fd74819eb90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd734009810 tx=0x7fd734009490 comp rx=0 tx=0).stop 2023-09-13T03:05:35.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.168+0000 7fd74eb70700 1 -- 172.21.15.64:0/2257956230 shutdown_connections 2023-09-13T03:05:35.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.168+0000 7fd74eb70700 1 --2- 172.21.15.64:0/2257956230 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd738038630 0x7fd73803aaf0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:35.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.168+0000 7fd74eb70700 1 --2- 172.21.15.64:0/2257956230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd748106770 0x7fd748198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:35.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.168+0000 7fd74eb70700 1 --2- 172.21.15.64:0/2257956230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd748068120 0x7fd74819e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:35.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.168+0000 7fd74eb70700 1 --2- 172.21.15.64:0/2257956230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd748068ae0 0x7fd74819eb90 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:35.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.168+0000 7fd74eb70700 1 -- 172.21.15.64:0/2257956230 >> 172.21.15.64:0/2257956230 conn(0x7fd748076600 msgr2=0x7fd74810b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:05:35.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.168+0000 7fd74eb70700 1 -- 172.21.15.64:0/2257956230 shutdown_connections 2023-09-13T03:05:35.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.168+0000 7fd74eb70700 1 -- 172.21.15.64:0/2257956230 wait complete. 2023-09-13T03:05:35.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.452+0000 7feb47bcb700 1 -- 172.21.15.64:0/2701510327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb40068120 msgr2=0x7feb400685a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:35.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.452+0000 7feb47bcb700 1 --2- 172.21.15.64:0/2701510327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb40068120 0x7feb400685a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7feb3000faf0 tx=0x7feb3000fe00 comp rx=0 tx=0).stop 2023-09-13T03:05:35.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.453+0000 7feb47bcb700 1 -- 172.21.15.64:0/2701510327 shutdown_connections 2023-09-13T03:05:35.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.453+0000 7feb47bcb700 1 --2- 172.21.15.64:0/2701510327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb40068ae0 0x7feb4010d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:35.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.453+0000 7feb47bcb700 1 --2- 172.21.15.64:0/2701510327 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb40106770 0x7feb40106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:35.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.453+0000 7feb47bcb700 1 --2- 172.21.15.64:0/2701510327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb40068120 0x7feb400685a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:35.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.453+0000 7feb47bcb700 1 -- 172.21.15.64:0/2701510327 >> 172.21.15.64:0/2701510327 conn(0x7feb40076600 msgr2=0x7feb40076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:05:35.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.453+0000 7feb47bcb700 1 -- 172.21.15.64:0/2701510327 shutdown_connections 2023-09-13T03:05:35.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.453+0000 7feb47bcb700 1 -- 172.21.15.64:0/2701510327 wait complete. 2023-09-13T03:05:35.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.454+0000 7feb47bcb700 1 Processor -- start 2023-09-13T03:05:35.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.455+0000 7feb47bcb700 1 -- start start 2023-09-13T03:05:35.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.455+0000 7feb47bcb700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb40068120 0x7feb40194520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:35.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.455+0000 7feb47bcb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb40068ae0 0x7feb40194a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:35.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.455+0000 7feb47bcb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb40106770 0x7feb40198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:35.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.455+0000 7feb47bcb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb401994e0 con 0x7feb40106770 2023-09-13T03:05:35.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.455+0000 7feb47bcb700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb40199650 con 0x7feb40068ae0 2023-09-13T03:05:35.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.455+0000 7feb47bcb700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb40199810 con 0x7feb40068120 2023-09-13T03:05:35.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.456+0000 7feb46168700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb40106770 0x7feb40198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:35.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.456+0000 7feb46168700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb40106770 0x7feb40198ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58468/0 (socket says 172.21.15.64:58468) 2023-09-13T03:05:35.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.456+0000 7feb46168700 1 -- 172.21.15.64:0/2053766209 learned_addr learned my addr 172.21.15.64:0/2053766209 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:05:35.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.456+0000 7feb45166700 1 --2- 172.21.15.64:0/2053766209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb40068ae0 0x7feb40194a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:35.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.456+0000 7feb45967700 1 --2- 172.21.15.64:0/2053766209 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb40068120 0x7feb40194520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:35.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.456+0000 7feb46168700 1 -- 172.21.15.64:0/2053766209 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb40068120 msgr2=0x7feb40194520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:35.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.456+0000 7feb46168700 1 --2- 172.21.15.64:0/2053766209 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb40068120 0x7feb40194520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:35.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.457+0000 7feb46168700 1 -- 172.21.15.64:0/2053766209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb40068ae0 msgr2=0x7feb40194a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:35.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.457+0000 7feb46168700 1 --2- 172.21.15.64:0/2053766209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb40068ae0 0x7feb40194a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:35.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.457+0000 7feb46168700 1 -- 172.21.15.64:0/2053766209 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb3000f750 con 0x7feb40106770 2023-09-13T03:05:35.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.457+0000 7feb46168700 1 --2- 172.21.15.64:0/2053766209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb40106770 0x7feb40198ed0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7feb3800efc0 tx=0x7feb3800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:05:35.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.457+0000 7feb36ffd700 1 -- 172.21.15.64:0/2053766209 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb38018070 con 0x7feb40106770 2023-09-13T03:05:35.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.457+0000 7feb36ffd700 1 -- 172.21.15.64:0/2053766209 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb3800f040 con 0x7feb40106770 2023-09-13T03:05:35.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.457+0000 7feb36ffd700 1 -- 172.21.15.64:0/2053766209 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3800cc10 con 0x7feb40106770 2023-09-13T03:05:35.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.457+0000 7feb47bcb700 1 -- 172.21.15.64:0/2053766209 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb40199a90 con 0x7feb40106770 2023-09-13T03:05:35.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.458+0000 7feb47bcb700 1 -- 172.21.15.64:0/2053766209 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb4006a1f0 con 0x7feb40106770 2023-09-13T03:05:35.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.459+0000 7feb47bcb700 1 -- 172.21.15.64:0/2053766209 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb4004e300 con 0x7feb40106770 2023-09-13T03:05:35.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.460+0000 7feb36ffd700 1 -- 172.21.15.64:0/2053766209 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7feb3800cd90 con 0x7feb40106770 2023-09-13T03:05:35.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.460+0000 7feb36ffd700 1 --2- 172.21.15.64:0/2053766209 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7feb2c038630 0x7feb2c03aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:35.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.460+0000 7feb36ffd700 1 -- 172.21.15.64:0/2053766209 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7feb3804b320 con 0x7feb40106770 2023-09-13T03:05:35.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.461+0000 7feb45967700 1 --2- 172.21.15.64:0/2053766209 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7feb2c038630 0x7feb2c03aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:35.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.462+0000 7feb45967700 1 --2- 172.21.15.64:0/2053766209 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7feb2c038630 0x7feb2c03aaf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7feb3c009c60 tx=0x7feb3c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:05:35.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.466+0000 7feb36ffd700 1 -- 172.21.15.64:0/2053766209 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7feb38004ad0 con 0x7feb40106770 2023-09-13T03:05:35.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:35 smithi170 ceph-mon[134328]: from='client.? ' entity='client.admin' 2023-09-13T03:05:35.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:35 smithi170 ceph-mon[134328]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:35.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:35 smithi170 ceph-mon[134328]: from='client.? ' entity='client.admin' 2023-09-13T03:05:35.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.690+0000 7feb47bcb700 1 -- 172.21.15.64:0/2053766209 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) v1 -- 0x7feb40199c20 con 0x7feb40106770 2023-09-13T03:05:35.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.693+0000 7feb36ffd700 1 -- 172.21.15.64:0/2053766209 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v24)=0 v24) v1 ==== 115+0+0 (secure 0 0 0) 0x7feb38005810 con 0x7feb40106770 2023-09-13T03:05:35.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.696+0000 7feb47bcb700 1 -- 172.21.15.64:0/2053766209 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7feb2c038630 msgr2=0x7feb2c03aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:35.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.696+0000 7feb47bcb700 1 --2- 172.21.15.64:0/2053766209 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7feb2c038630 0x7feb2c03aaf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7feb3c009c60 tx=0x7feb3c009260 comp rx=0 tx=0).stop 2023-09-13T03:05:35.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.696+0000 7feb47bcb700 1 -- 172.21.15.64:0/2053766209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb40106770 msgr2=0x7feb40198ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:35.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.696+0000 7feb47bcb700 1 --2- 172.21.15.64:0/2053766209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb40106770 0x7feb40198ed0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7feb3800efc0 tx=0x7feb3800c3a0 comp rx=0 tx=0).stop 2023-09-13T03:05:35.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.696+0000 7feb47bcb700 1 -- 172.21.15.64:0/2053766209 shutdown_connections 2023-09-13T03:05:35.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.696+0000 7feb47bcb700 1 --2- 172.21.15.64:0/2053766209 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7feb2c038630 0x7feb2c03aaf0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:35.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.696+0000 7feb47bcb700 1 --2- 172.21.15.64:0/2053766209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb40106770 0x7feb40198ed0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:35.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.696+0000 7feb47bcb700 1 --2- 172.21.15.64:0/2053766209 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb40068120 0x7feb40194520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:35.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.696+0000 7feb47bcb700 1 --2- 172.21.15.64:0/2053766209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb40068ae0 0x7feb40194a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:35.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.696+0000 7feb47bcb700 1 -- 172.21.15.64:0/2053766209 >> 172.21.15.64:0/2053766209 conn(0x7feb40076600 msgr2=0x7feb400fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:05:35.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.696+0000 7feb47bcb700 1 -- 172.21.15.64:0/2053766209 shutdown_connections 2023-09-13T03:05:35.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.696+0000 7feb47bcb700 1 -- 172.21.15.64:0/2053766209 wait complete. 2023-09-13T03:05:35.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.191+0000 7fc67621a700 1 -- 172.21.15.64:0/1374949960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc670100620 msgr2=0x7fc670100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:35.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.191+0000 7fc67621a700 1 --2- 172.21.15.64:0/1374949960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc670100620 0x7fc670100aa0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc6640098f0 tx=0x7fc664009c00 comp rx=0 tx=0).stop 2023-09-13T03:05:35.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.191+0000 7fc67621a700 1 -- 172.21.15.64:0/1374949960 shutdown_connections 2023-09-13T03:05:35.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.191+0000 7fc67621a700 1 --2- 172.21.15.64:0/1374949960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc670100fe0 0x7fc67010d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:35.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.191+0000 7fc67621a700 1 --2- 172.21.15.64:0/1374949960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc670106620 0x7fc670106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:35.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.191+0000 7fc67621a700 1 --2- 172.21.15.64:0/1374949960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc670100620 0x7fc670100aa0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:35.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.191+0000 7fc67621a700 1 -- 172.21.15.64:0/1374949960 >> 172.21.15.64:0/1374949960 conn(0x7fc6700fc0f0 msgr2=0x7fc6700fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:05:35.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.191+0000 7fc67621a700 1 -- 172.21.15.64:0/1374949960 shutdown_connections 2023-09-13T03:05:35.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.191+0000 7fc67621a700 1 -- 172.21.15.64:0/1374949960 wait complete. 2023-09-13T03:05:35.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.192+0000 7fc67621a700 1 Processor -- start 2023-09-13T03:05:35.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.193+0000 7fc67621a700 1 -- start start 2023-09-13T03:05:35.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.193+0000 7fc67621a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc670100620 0x7fc670072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:35.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.193+0000 7fc67621a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc670100fe0 0x7fc670072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:35.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.193+0000 7fc67621a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc670106620 0x7fc6700733c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:35.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.193+0000 7fc67621a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6701a4c00 con 0x7fc670106620 2023-09-13T03:05:35.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.193+0000 7fc67621a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6701a4d70 con 0x7fc670100620 2023-09-13T03:05:35.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.193+0000 7fc67621a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6701a4f30 con 0x7fc670100fe0 2023-09-13T03:05:35.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.194+0000 7fc66f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc670100620 0x7fc670072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:35.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.194+0000 7fc66f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc670100620 0x7fc670072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:52956/0 (socket says 172.21.15.64:52956) 2023-09-13T03:05:35.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.194+0000 7fc66f7fe700 1 -- 172.21.15.64:0/4282546507 learned_addr learned my addr 172.21.15.64:0/4282546507 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:05:35.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.194+0000 7fc66effd700 1 --2- 172.21.15.64:0/4282546507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc670100fe0 0x7fc670072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:35.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.194+0000 7fc66ffff700 1 --2- 172.21.15.64:0/4282546507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc670106620 0x7fc6700733c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:35.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.194+0000 7fc66f7fe700 1 -- 172.21.15.64:0/4282546507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc670100fe0 msgr2=0x7fc670072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:35.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.194+0000 7fc66f7fe700 1 --2- 172.21.15.64:0/4282546507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc670100fe0 0x7fc670072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:35.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.194+0000 7fc66f7fe700 1 -- 172.21.15.64:0/4282546507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc670106620 msgr2=0x7fc6700733c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:35.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.194+0000 7fc66f7fe700 1 --2- 172.21.15.64:0/4282546507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc670106620 0x7fc6700733c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:35.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.194+0000 7fc66f7fe700 1 -- 172.21.15.64:0/4282546507 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6600095f0 con 0x7fc670100620 2023-09-13T03:05:35.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.195+0000 7fc66f7fe700 1 --2- 172.21.15.64:0/4282546507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc670100620 0x7fc670072900 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc66000ec10 tx=0x7fc66000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:05:35.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.195+0000 7fc66cff9700 1 -- 172.21.15.64:0/4282546507 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66000cc90 con 0x7fc670100620 2023-09-13T03:05:35.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.195+0000 7fc67621a700 1 -- 172.21.15.64:0/4282546507 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc664009520 con 0x7fc670100620 2023-09-13T03:05:35.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.195+0000 7fc66cff9700 1 -- 172.21.15.64:0/4282546507 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6600044e0 con 0x7fc670100620 2023-09-13T03:05:35.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.195+0000 7fc66cff9700 1 -- 172.21.15.64:0/4282546507 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc660010430 con 0x7fc670100620 2023-09-13T03:05:35.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.195+0000 7fc67621a700 1 -- 172.21.15.64:0/4282546507 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6701a55a0 con 0x7fc670100620 2023-09-13T03:05:35.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.196+0000 7fc66cff9700 1 -- 172.21.15.64:0/4282546507 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7fc660010590 con 0x7fc670100620 2023-09-13T03:05:35.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.196+0000 7fc67621a700 1 -- 172.21.15.64:0/4282546507 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc67004e300 con 0x7fc670100620 2023-09-13T03:05:35.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.197+0000 7fc66cff9700 1 --2- 172.21.15.64:0/4282546507 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc658038630 0x7fc65803aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:35.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.197+0000 7fc66cff9700 1 -- 172.21.15.64:0/4282546507 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc660014070 con 0x7fc670100620 2023-09-13T03:05:35.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.197+0000 7fc66effd700 1 --2- 172.21.15.64:0/4282546507 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc658038630 0x7fc65803aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:35.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.198+0000 7fc66effd700 1 --2- 172.21.15.64:0/4282546507 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc658038630 0x7fc65803aaf0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc664005e90 tx=0x7fc664019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:05:35.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.202+0000 7fc66cff9700 1 -- 172.21.15.64:0/4282546507 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fc66001f340 con 0x7fc670100620 2023-09-13T03:05:35.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.441+0000 7fc67621a700 1 -- 172.21.15.64:0/4282546507 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) v1 -- 0x7fc670073e30 con 0x7fc670100620 2023-09-13T03:05:35.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.448+0000 7fc66cff9700 1 -- 172.21.15.64:0/4282546507 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v23)=0 v23) v1 ==== 116+0+0 (secure 0 0 0) 0x7fc670073e30 con 0x7fc670100620 2023-09-13T03:05:35.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.452+0000 7fc67621a700 1 -- 172.21.15.64:0/4282546507 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc658038630 msgr2=0x7fc65803aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:35.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.452+0000 7fc67621a700 1 --2- 172.21.15.64:0/4282546507 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc658038630 0x7fc65803aaf0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc664005e90 tx=0x7fc664019040 comp rx=0 tx=0).stop 2023-09-13T03:05:35.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.452+0000 7fc67621a700 1 -- 172.21.15.64:0/4282546507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc670100620 msgr2=0x7fc670072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:35.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.452+0000 7fc67621a700 1 --2- 172.21.15.64:0/4282546507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc670100620 0x7fc670072900 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc66000ec10 tx=0x7fc66000ef20 comp rx=0 tx=0).stop 2023-09-13T03:05:35.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.452+0000 7fc67621a700 1 -- 172.21.15.64:0/4282546507 shutdown_connections 2023-09-13T03:05:35.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.452+0000 7fc67621a700 1 --2- 172.21.15.64:0/4282546507 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc658038630 0x7fc65803aaf0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:35.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.452+0000 7fc67621a700 1 --2- 172.21.15.64:0/4282546507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc670100620 0x7fc670072900 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:35.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.452+0000 7fc67621a700 1 --2- 172.21.15.64:0/4282546507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc670106620 0x7fc6700733c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:35.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.452+0000 7fc67621a700 1 --2- 172.21.15.64:0/4282546507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc670100fe0 0x7fc670072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:35.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.452+0000 7fc67621a700 1 -- 172.21.15.64:0/4282546507 >> 172.21.15.64:0/4282546507 conn(0x7fc6700fc0f0 msgr2=0x7fc6700fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:05:35.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.452+0000 7fc67621a700 1 -- 172.21.15.64:0/4282546507 shutdown_connections 2023-09-13T03:05:35.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:35.452+0000 7fc67621a700 1 -- 172.21.15.64:0/4282546507 wait complete. 2023-09-13T03:05:35.815 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:05:36.584 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:36 smithi064 ceph-mon[97675]: from='client.? ' entity='client.admin' 2023-09-13T03:05:36.584 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:36 smithi064 ceph-mon[97675]: from='client.? ' entity='client.admin' 2023-09-13T03:05:36.585 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:36 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2053766209' entity='client.admin' 2023-09-13T03:05:36.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:36 smithi170 ceph-mon[134328]: from='client.? ' entity='client.admin' 2023-09-13T03:05:36.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:36 smithi170 ceph-mon[134328]: from='client.? ' entity='client.admin' 2023-09-13T03:05:36.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:36 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/2053766209' entity='client.admin' 2023-09-13T03:05:36.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.444+0000 7fbd914b2700 1 -- 172.21.15.64:0/947086874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd8c106620 msgr2=0x7fbd8c106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:36.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.444+0000 7fbd914b2700 1 --2- 172.21.15.64:0/947086874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd8c106620 0x7fbd8c106a00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c009990 tx=0x7fbd7c009ca0 comp rx=0 tx=0).stop 2023-09-13T03:05:36.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.444+0000 7fbd914b2700 1 -- 172.21.15.64:0/947086874 shutdown_connections 2023-09-13T03:05:36.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.444+0000 7fbd914b2700 1 --2- 172.21.15.64:0/947086874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd8c106620 0x7fbd8c106a00 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:36.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.444+0000 7fbd914b2700 1 --2- 172.21.15.64:0/947086874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd8c100fe0 0x7fbd8c10d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:36.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.444+0000 7fbd914b2700 1 --2- 172.21.15.64:0/947086874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd8c100620 0x7fbd8c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:36.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.444+0000 7fbd914b2700 1 -- 172.21.15.64:0/947086874 >> 172.21.15.64:0/947086874 conn(0x7fbd8c0fc0f0 msgr2=0x7fbd8c0fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:05:36.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.444+0000 7fbd914b2700 1 -- 172.21.15.64:0/947086874 shutdown_connections 2023-09-13T03:05:36.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.444+0000 7fbd914b2700 1 -- 172.21.15.64:0/947086874 wait complete. 2023-09-13T03:05:36.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.445+0000 7fbd914b2700 1 Processor -- start 2023-09-13T03:05:36.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.445+0000 7fbd914b2700 1 -- start start 2023-09-13T03:05:36.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.446+0000 7fbd914b2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd8c100620 0x7fbd8c19c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:36.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.446+0000 7fbd914b2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd8c100fe0 0x7fbd8c19ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:36.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.446+0000 7fbd914b2700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd8c106620 0x7fbd8c196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:36.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.446+0000 7fbd914b2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8c196bf0 con 0x7fbd8c100fe0 2023-09-13T03:05:36.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.446+0000 7fbd914b2700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8c196d60 con 0x7fbd8c106620 2023-09-13T03:05:36.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.446+0000 7fbd914b2700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8c196f20 con 0x7fbd8c100620 2023-09-13T03:05:36.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.446+0000 7fbd8a7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd8c100fe0 0x7fbd8c19ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:36.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.446+0000 7fbd8affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd8c100620 0x7fbd8c19c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:36.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.446+0000 7fbd8a7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd8c100fe0 0x7fbd8c19ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33068/0 (socket says 172.21.15.64:33068) 2023-09-13T03:05:36.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.446+0000 7fbd8a7fc700 1 -- 172.21.15.64:0/3859952021 learned_addr learned my addr 172.21.15.64:0/3859952021 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:05:36.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.446+0000 7fbd8affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd8c100620 0x7fbd8c19c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:42882/0 (socket says 172.21.15.64:42882) 2023-09-13T03:05:36.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.446+0000 7fbd8b7fe700 1 --2- 172.21.15.64:0/3859952021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd8c106620 0x7fbd8c196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:36.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.447+0000 7fbd8a7fc700 1 -- 172.21.15.64:0/3859952021 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd8c100620 msgr2=0x7fbd8c19c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:36.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.447+0000 7fbd8a7fc700 1 --2- 172.21.15.64:0/3859952021 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd8c100620 0x7fbd8c19c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:36.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.447+0000 7fbd8a7fc700 1 -- 172.21.15.64:0/3859952021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd8c106620 msgr2=0x7fbd8c196580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:36.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.447+0000 7fbd8a7fc700 1 --2- 172.21.15.64:0/3859952021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd8c106620 0x7fbd8c196580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:36.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.447+0000 7fbd8a7fc700 1 -- 172.21.15.64:0/3859952021 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd7c0095f0 con 0x7fbd8c100fe0 2023-09-13T03:05:36.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.447+0000 7fbd8affd700 1 --2- 172.21.15.64:0/3859952021 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd8c100620 0x7fbd8c19c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:05:36.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.447+0000 7fbd8a7fc700 1 --2- 172.21.15.64:0/3859952021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd8c100fe0 0x7fbd8c19ca40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbd8000e9a0 tx=0x7fbd8000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:05:36.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.447+0000 7fbd73fff700 1 -- 172.21.15.64:0/3859952021 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd80009790 con 0x7fbd8c100fe0 2023-09-13T03:05:36.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.447+0000 7fbd73fff700 1 -- 172.21.15.64:0/3859952021 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd80010430 con 0x7fbd8c100fe0 2023-09-13T03:05:36.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.447+0000 7fbd914b2700 1 -- 172.21.15.64:0/3859952021 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd8c197200 con 0x7fbd8c100fe0 2023-09-13T03:05:36.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.447+0000 7fbd73fff700 1 -- 172.21.15.64:0/3859952021 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd80018680 con 0x7fbd8c100fe0 2023-09-13T03:05:36.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.447+0000 7fbd914b2700 1 -- 172.21.15.64:0/3859952021 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd8c1a34f0 con 0x7fbd8c100fe0 2023-09-13T03:05:36.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.448+0000 7fbd73fff700 1 -- 172.21.15.64:0/3859952021 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7fbd800105a0 con 0x7fbd8c100fe0 2023-09-13T03:05:36.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.449+0000 7fbd914b2700 1 -- 172.21.15.64:0/3859952021 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd8c04e300 con 0x7fbd8c100fe0 2023-09-13T03:05:36.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.449+0000 7fbd73fff700 1 --2- 172.21.15.64:0/3859952021 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fbd74038630 0x7fbd7403aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:36.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.449+0000 7fbd73fff700 1 -- 172.21.15.64:0/3859952021 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbd80014070 con 0x7fbd8c100fe0 2023-09-13T03:05:36.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.449+0000 7fbd8affd700 1 --2- 172.21.15.64:0/3859952021 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fbd74038630 0x7fbd7403aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:36.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.450+0000 7fbd8affd700 1 --2- 172.21.15.64:0/3859952021 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fbd74038630 0x7fbd7403aaf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c00b2e0 tx=0x7fbd7c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:05:36.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.454+0000 7fbd73fff700 1 -- 172.21.15.64:0/3859952021 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fbd8001f780 con 0x7fbd8c100fe0 2023-09-13T03:05:36.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.713+0000 7fbd914b2700 1 -- 172.21.15.64:0/3859952021 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) v1 -- 0x7fbd8c110690 con 0x7fbd8c100fe0 2023-09-13T03:05:36.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.717+0000 7fbd73fff700 1 -- 172.21.15.64:0/3859952021 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v25)=0 v25) v1 ==== 121+0+0 (secure 0 0 0) 0x7fbd8001f780 con 0x7fbd8c100fe0 2023-09-13T03:05:36.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.720+0000 7fbd914b2700 1 -- 172.21.15.64:0/3859952021 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fbd74038630 msgr2=0x7fbd7403aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:36.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.720+0000 7fbd914b2700 1 --2- 172.21.15.64:0/3859952021 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fbd74038630 0x7fbd7403aaf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c00b2e0 tx=0x7fbd7c005fd0 comp rx=0 tx=0).stop 2023-09-13T03:05:36.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.720+0000 7fbd914b2700 1 -- 172.21.15.64:0/3859952021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd8c100fe0 msgr2=0x7fbd8c19ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:36.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.720+0000 7fbd914b2700 1 --2- 172.21.15.64:0/3859952021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd8c100fe0 0x7fbd8c19ca40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbd8000e9a0 tx=0x7fbd8000ecb0 comp rx=0 tx=0).stop 2023-09-13T03:05:36.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.720+0000 7fbd914b2700 1 -- 172.21.15.64:0/3859952021 shutdown_connections 2023-09-13T03:05:36.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.720+0000 7fbd914b2700 1 --2- 172.21.15.64:0/3859952021 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fbd74038630 0x7fbd7403aaf0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:36.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.720+0000 7fbd914b2700 1 --2- 172.21.15.64:0/3859952021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd8c106620 0x7fbd8c196580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:36.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.720+0000 7fbd914b2700 1 --2- 172.21.15.64:0/3859952021 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd8c100620 0x7fbd8c19c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:36.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.720+0000 7fbd914b2700 1 --2- 172.21.15.64:0/3859952021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd8c100fe0 0x7fbd8c19ca40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:36.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.720+0000 7fbd914b2700 1 -- 172.21.15.64:0/3859952021 >> 172.21.15.64:0/3859952021 conn(0x7fbd8c0fc0f0 msgr2=0x7fbd8c0fc720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:05:36.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.720+0000 7fbd914b2700 1 -- 172.21.15.64:0/3859952021 shutdown_connections 2023-09-13T03:05:36.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:36.720+0000 7fbd914b2700 1 -- 172.21.15.64:0/3859952021 wait complete. 2023-09-13T03:05:36.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:36 smithi142 ceph-mon[134082]: from='client.? ' entity='client.admin' 2023-09-13T03:05:36.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:36 smithi142 ceph-mon[134082]: from='client.? ' entity='client.admin' 2023-09-13T03:05:36.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:36 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/2053766209' entity='client.admin' 2023-09-13T03:05:37.180 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:05:37.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.325+0000 7f8e5ca75700 1 -- 172.21.15.64:0/1675006260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e580729a0 msgr2=0x7f8e58072d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:37.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.325+0000 7f8e5ca75700 1 --2- 172.21.15.64:0/1675006260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e580729a0 0x7f8e58072d80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8e500098f0 tx=0x7f8e50009c00 comp rx=0 tx=0).stop 2023-09-13T03:05:37.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.325+0000 7f8e5ca75700 1 -- 172.21.15.64:0/1675006260 shutdown_connections 2023-09-13T03:05:37.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.325+0000 7f8e5ca75700 1 --2- 172.21.15.64:0/1675006260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e580729a0 0x7f8e58072d80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:37.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.325+0000 7f8e5ca75700 1 --2- 172.21.15.64:0/1675006260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e58111630 0x7f8e58113a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:37.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.325+0000 7f8e5ca75700 1 --2- 172.21.15.64:0/1675006260 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8e580732c0 0x7f8e581110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:37.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.325+0000 7f8e5ca75700 1 -- 172.21.15.64:0/1675006260 >> 172.21.15.64:0/1675006260 conn(0x7f8e580fc0f0 msgr2=0x7f8e580fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:05:37.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.325+0000 7f8e5ca75700 1 -- 172.21.15.64:0/1675006260 shutdown_connections 2023-09-13T03:05:37.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.326+0000 7f8e5ca75700 1 -- 172.21.15.64:0/1675006260 wait complete. 2023-09-13T03:05:37.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.326+0000 7f8e5ca75700 1 Processor -- start 2023-09-13T03:05:37.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.327+0000 7f8e5ca75700 1 -- start start 2023-09-13T03:05:37.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.327+0000 7f8e5ca75700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e580729a0 0x7f8e58109960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:37.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.327+0000 7f8e5ca75700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8e580732c0 0x7f8e58109ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:37.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.327+0000 7f8e5ca75700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e58111630 0x7f8e581b1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:37.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.327+0000 7f8e5ca75700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e581b1840 con 0x7f8e580729a0 2023-09-13T03:05:37.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.327+0000 7f8e5ca75700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5806a020 con 0x7f8e58111630 2023-09-13T03:05:37.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.327+0000 7f8e5ca75700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5806a1e0 con 0x7f8e580732c0 2023-09-13T03:05:37.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.327+0000 7f8e5659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e580729a0 0x7f8e58109960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:37.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.327+0000 7f8e55d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8e580732c0 0x7f8e58109ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:37.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.327+0000 7f8e56d9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e58111630 0x7f8e581b1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:37.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.327+0000 7f8e55d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8e580732c0 0x7f8e58109ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:42910/0 (socket says 172.21.15.64:42910) 2023-09-13T03:05:37.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.327+0000 7f8e55d9b700 1 -- 172.21.15.64:0/1552104915 learned_addr learned my addr 172.21.15.64:0/1552104915 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:05:37.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.328+0000 7f8e55d9b700 1 -- 172.21.15.64:0/1552104915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e58111630 msgr2=0x7f8e581b1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:37.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.328+0000 7f8e55d9b700 1 --2- 172.21.15.64:0/1552104915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e58111630 0x7f8e581b1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:37.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.328+0000 7f8e55d9b700 1 -- 172.21.15.64:0/1552104915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e580729a0 msgr2=0x7f8e58109960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:37.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.328+0000 7f8e55d9b700 1 --2- 172.21.15.64:0/1552104915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e580729a0 0x7f8e58109960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:37.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.328+0000 7f8e55d9b700 1 -- 172.21.15.64:0/1552104915 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e50009520 con 0x7f8e580732c0 2023-09-13T03:05:37.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.328+0000 7f8e55d9b700 1 --2- 172.21.15.64:0/1552104915 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8e580732c0 0x7f8e58109ea0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000e9c0 tx=0x7f8e4000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:05:37.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.328+0000 7f8e4f7fe700 1 -- 172.21.15.64:0/1552104915 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4000cbd0 con 0x7f8e580732c0 2023-09-13T03:05:37.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.328+0000 7f8e4f7fe700 1 -- 172.21.15.64:0/1552104915 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e4000eeb0 con 0x7f8e580732c0 2023-09-13T03:05:37.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.328+0000 7f8e5ca75700 1 -- 172.21.15.64:0/1552104915 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e5806a4c0 con 0x7f8e580732c0 2023-09-13T03:05:37.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.328+0000 7f8e4f7fe700 1 -- 172.21.15.64:0/1552104915 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e40005580 con 0x7f8e580732c0 2023-09-13T03:05:37.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.328+0000 7f8e5ca75700 1 -- 172.21.15.64:0/1552104915 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e5806a9c0 con 0x7f8e580732c0 2023-09-13T03:05:37.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.330+0000 7f8e5ca75700 1 -- 172.21.15.64:0/1552104915 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e58103b10 con 0x7f8e580732c0 2023-09-13T03:05:37.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.330+0000 7f8e4f7fe700 1 -- 172.21.15.64:0/1552104915 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7f8e40004750 con 0x7f8e580732c0 2023-09-13T03:05:37.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.331+0000 7f8e4f7fe700 1 --2- 172.21.15.64:0/1552104915 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f8e44038630 0x7f8e4403aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:37.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.331+0000 7f8e4f7fe700 1 -- 172.21.15.64:0/1552104915 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8e40014070 con 0x7f8e580732c0 2023-09-13T03:05:37.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.331+0000 7f8e5659c700 1 --2- 172.21.15.64:0/1552104915 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f8e44038630 0x7f8e4403aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:37.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.332+0000 7f8e5659c700 1 --2- 172.21.15.64:0/1552104915 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f8e44038630 0x7f8e4403aaf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8e5000c010 tx=0x7f8e50019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:05:37.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.335+0000 7f8e4f7fe700 1 -- 172.21.15.64:0/1552104915 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f8e4000cd30 con 0x7f8e580732c0 2023-09-13T03:05:37.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.556+0000 7f8e5ca75700 1 -- 172.21.15.64:0/1552104915 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) v1 -- 0x7f8e58116bf0 con 0x7f8e580732c0 2023-09-13T03:05:37.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.561+0000 7f8e4f7fe700 1 -- 172.21.15.64:0/1552104915 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v26)=0 v26) v1 ==== 120+0+0 (secure 0 0 0) 0x7f8e4001f340 con 0x7f8e580732c0 2023-09-13T03:05:37.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.563+0000 7f8e5ca75700 1 -- 172.21.15.64:0/1552104915 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f8e44038630 msgr2=0x7f8e4403aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:37.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.563+0000 7f8e5ca75700 1 --2- 172.21.15.64:0/1552104915 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f8e44038630 0x7f8e4403aaf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8e5000c010 tx=0x7f8e50019040 comp rx=0 tx=0).stop 2023-09-13T03:05:37.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.563+0000 7f8e5ca75700 1 -- 172.21.15.64:0/1552104915 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8e580732c0 msgr2=0x7f8e58109ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:37.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.563+0000 7f8e5ca75700 1 --2- 172.21.15.64:0/1552104915 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8e580732c0 0x7f8e58109ea0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000e9c0 tx=0x7f8e4000ecd0 comp rx=0 tx=0).stop 2023-09-13T03:05:37.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.563+0000 7f8e5ca75700 1 -- 172.21.15.64:0/1552104915 shutdown_connections 2023-09-13T03:05:37.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.563+0000 7f8e5ca75700 1 --2- 172.21.15.64:0/1552104915 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f8e44038630 0x7f8e4403aaf0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:37.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.563+0000 7f8e5ca75700 1 --2- 172.21.15.64:0/1552104915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e58111630 0x7f8e581b1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:37.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.563+0000 7f8e5ca75700 1 --2- 172.21.15.64:0/1552104915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e580729a0 0x7f8e58109960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:37.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.563+0000 7f8e5ca75700 1 --2- 172.21.15.64:0/1552104915 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8e580732c0 0x7f8e58109ea0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:37.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.563+0000 7f8e5ca75700 1 -- 172.21.15.64:0/1552104915 >> 172.21.15.64:0/1552104915 conn(0x7f8e580fc0f0 msgr2=0x7f8e580fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:05:37.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.563+0000 7f8e5ca75700 1 -- 172.21.15.64:0/1552104915 shutdown_connections 2023-09-13T03:05:37.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:37.564+0000 7f8e5ca75700 1 -- 172.21.15.64:0/1552104915 wait complete. 2023-09-13T03:05:37.586 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:37 smithi064 ceph-mon[97675]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:37.586 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:37 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/3859952021' entity='client.admin' 2023-09-13T03:05:37.659 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:37 smithi170 ceph-mon[134328]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:37.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:37 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/3859952021' entity='client.admin' 2023-09-13T03:05:37.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:37 smithi142 ceph-mon[134082]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:37.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:37 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/3859952021' entity='client.admin' 2023-09-13T03:05:37.938 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:05:38.779 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:38 smithi064 ceph-mon[97675]: from='client.? ' entity='client.admin' 2023-09-13T03:05:38.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:38 smithi170 ceph-mon[134328]: from='client.? ' entity='client.admin' 2023-09-13T03:05:38.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:38 smithi142 ceph-mon[134082]: from='client.? ' entity='client.admin' 2023-09-13T03:05:39.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.593+0000 7f10fb319700 1 -- 172.21.15.64:0/2727444725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10f4103590 msgr2=0x7f10f4103a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:39.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.593+0000 7f10fb319700 1 --2- 172.21.15.64:0/2727444725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10f4103590 0x7f10f4103a10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f10e4009990 tx=0x7f10e4009ca0 comp rx=0 tx=0).stop 2023-09-13T03:05:39.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.593+0000 7f10fb319700 1 -- 172.21.15.64:0/2727444725 shutdown_connections 2023-09-13T03:05:39.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.593+0000 7f10fb319700 1 --2- 172.21.15.64:0/2727444725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10f4068120 0x7f10f40685a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:39.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.593+0000 7f10fb319700 1 --2- 172.21.15.64:0/2727444725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10f41055c0 0x7f10f4103050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:39.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.593+0000 7f10fb319700 1 --2- 172.21.15.64:0/2727444725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10f4103590 0x7f10f4103a10 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:39.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.593+0000 7f10fb319700 1 -- 172.21.15.64:0/2727444725 >> 172.21.15.64:0/2727444725 conn(0x7f10f4076600 msgr2=0x7f10f4076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:05:39.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.593+0000 7f10fb319700 1 -- 172.21.15.64:0/2727444725 shutdown_connections 2023-09-13T03:05:39.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.593+0000 7f10fb319700 1 -- 172.21.15.64:0/2727444725 wait complete. 2023-09-13T03:05:39.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.595+0000 7f10fb319700 1 Processor -- start 2023-09-13T03:05:39.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.595+0000 7f10fb319700 1 -- start start 2023-09-13T03:05:39.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.596+0000 7f10fb319700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10f4068120 0x7f10f419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:39.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.596+0000 7f10fb319700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10f4103590 0x7f10f419d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:39.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.596+0000 7f10fb319700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10f41055c0 0x7f10f41a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:39.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.596+0000 7f10fb319700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f41a1da0 con 0x7f10f4103590 2023-09-13T03:05:39.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.596+0000 7f10fb319700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f41a1f10 con 0x7f10f41055c0 2023-09-13T03:05:39.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.596+0000 7f10fb319700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f41a20d0 con 0x7f10f4068120 2023-09-13T03:05:39.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.596+0000 7f10f90b5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10f4068120 0x7f10f419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:39.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.596+0000 7f10f98b6700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10f41055c0 0x7f10f41a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:39.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.596+0000 7f10f90b5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10f4068120 0x7f10f419cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:42924/0 (socket says 172.21.15.64:42924) 2023-09-13T03:05:39.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.596+0000 7f10f98b6700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10f41055c0 0x7f10f41a1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:50920/0 (socket says 172.21.15.64:50920) 2023-09-13T03:05:39.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.596+0000 7f10f90b5700 1 -- 172.21.15.64:0/339143982 learned_addr learned my addr 172.21.15.64:0/339143982 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:05:39.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.596+0000 7f10f88b4700 1 --2- 172.21.15.64:0/339143982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10f4103590 0x7f10f419d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:39.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.597+0000 7f10f88b4700 1 -- 172.21.15.64:0/339143982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10f4068120 msgr2=0x7f10f419cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:39.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.597+0000 7f10f88b4700 1 --2- 172.21.15.64:0/339143982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10f4068120 0x7f10f419cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:39.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.597+0000 7f10f88b4700 1 -- 172.21.15.64:0/339143982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10f41055c0 msgr2=0x7f10f41a1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:39.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.597+0000 7f10f88b4700 1 --2- 172.21.15.64:0/339143982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10f41055c0 0x7f10f41a1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:39.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.597+0000 7f10f88b4700 1 -- 172.21.15.64:0/339143982 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e40095f0 con 0x7f10f4103590 2023-09-13T03:05:39.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.597+0000 7f10f90b5700 1 --2- 172.21.15.64:0/339143982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10f4068120 0x7f10f419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:05:39.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.597+0000 7f10f98b6700 1 --2- 172.21.15.64:0/339143982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10f41055c0 0x7f10f41a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:05:39.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.597+0000 7f10f88b4700 1 --2- 172.21.15.64:0/339143982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10f4103590 0x7f10f419d320 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f10e4000c00 tx=0x7f10e400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:05:39.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.597+0000 7f10ea7fc700 1 -- 172.21.15.64:0/339143982 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e401d070 con 0x7f10f4103590 2023-09-13T03:05:39.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.597+0000 7f10fb319700 1 -- 172.21.15.64:0/339143982 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f41a2350 con 0x7f10f4103590 2023-09-13T03:05:39.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.597+0000 7f10ea7fc700 1 -- 172.21.15.64:0/339143982 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10e4005050 con 0x7f10f4103590 2023-09-13T03:05:39.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.597+0000 7f10ea7fc700 1 -- 172.21.15.64:0/339143982 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e4004970 con 0x7f10f4103590 2023-09-13T03:05:39.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.598+0000 7f10fb319700 1 -- 172.21.15.64:0/339143982 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f4105cd0 con 0x7f10f4103590 2023-09-13T03:05:39.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.599+0000 7f10dffff700 1 -- 172.21.15.64:0/339143982 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10d40052f0 con 0x7f10f4103590 2023-09-13T03:05:39.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.599+0000 7f10ea7fc700 1 -- 172.21.15.64:0/339143982 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7f10e40051c0 con 0x7f10f4103590 2023-09-13T03:05:39.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.600+0000 7f10ea7fc700 1 --2- 172.21.15.64:0/339143982 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f10e0038630 0x7f10e003aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:39.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.600+0000 7f10ea7fc700 1 -- 172.21.15.64:0/339143982 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f10e404bd10 con 0x7f10f4103590 2023-09-13T03:05:39.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.600+0000 7f10f90b5700 1 --2- 172.21.15.64:0/339143982 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f10e0038630 0x7f10e003aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:39.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.601+0000 7f10f90b5700 1 --2- 172.21.15.64:0/339143982 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f10e0038630 0x7f10e003aaf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f10f0005950 tx=0x7f10f000b500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:05:39.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.607+0000 7f10ea7fc700 1 -- 172.21.15.64:0/339143982 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f10e4016ba0 con 0x7f10f4103590 2023-09-13T03:05:39.825 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:39 smithi064 ceph-mon[97675]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:39.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.834+0000 7f10dffff700 1 -- 172.21.15.64:0/339143982 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) v1 -- 0x7f10d4005b20 con 0x7f10f4103590 2023-09-13T03:05:39.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.838+0000 7f10ea7fc700 1 -- 172.21.15.64:0/339143982 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v27)=0 v27) v1 ==== 121+0+0 (secure 0 0 0) 0x7f10e4026070 con 0x7f10f4103590 2023-09-13T03:05:39.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.841+0000 7f10dffff700 1 -- 172.21.15.64:0/339143982 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f10e0038630 msgr2=0x7f10e003aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:39.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.841+0000 7f10dffff700 1 --2- 172.21.15.64:0/339143982 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f10e0038630 0x7f10e003aaf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f10f0005950 tx=0x7f10f000b500 comp rx=0 tx=0).stop 2023-09-13T03:05:39.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.841+0000 7f10dffff700 1 -- 172.21.15.64:0/339143982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10f4103590 msgr2=0x7f10f419d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:39.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.841+0000 7f10dffff700 1 --2- 172.21.15.64:0/339143982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10f4103590 0x7f10f419d320 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f10e4000c00 tx=0x7f10e400f690 comp rx=0 tx=0).stop 2023-09-13T03:05:39.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.841+0000 7f10dffff700 1 -- 172.21.15.64:0/339143982 shutdown_connections 2023-09-13T03:05:39.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.841+0000 7f10dffff700 1 --2- 172.21.15.64:0/339143982 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f10e0038630 0x7f10e003aaf0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:39.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.841+0000 7f10dffff700 1 --2- 172.21.15.64:0/339143982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10f41055c0 0x7f10f41a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:39.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.841+0000 7f10dffff700 1 --2- 172.21.15.64:0/339143982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10f4068120 0x7f10f419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:39.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.841+0000 7f10dffff700 1 --2- 172.21.15.64:0/339143982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10f4103590 0x7f10f419d320 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:39.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.841+0000 7f10dffff700 1 -- 172.21.15.64:0/339143982 >> 172.21.15.64:0/339143982 conn(0x7f10f4076600 msgr2=0x7f10f40fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:05:39.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.841+0000 7f10dffff700 1 -- 172.21.15.64:0/339143982 shutdown_connections 2023-09-13T03:05:39.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.841+0000 7f10dffff700 1 -- 172.21.15.64:0/339143982 wait complete. 2023-09-13T03:05:39.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:39 smithi170 ceph-mon[134328]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:39.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.960+0000 7f98ec67f700 1 -- 172.21.15.64:0/1523021135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98e40729a0 msgr2=0x7f98e4072d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:39.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.960+0000 7f98ec67f700 1 --2- 172.21.15.64:0/1523021135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98e40729a0 0x7f98e4072d80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f98e00098f0 tx=0x7f98e0009c00 comp rx=0 tx=0).stop 2023-09-13T03:05:39.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.960+0000 7f98ec67f700 1 -- 172.21.15.64:0/1523021135 shutdown_connections 2023-09-13T03:05:39.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.960+0000 7f98ec67f700 1 --2- 172.21.15.64:0/1523021135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98e4111630 0x7f98e4113a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:39.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.961+0000 7f98ec67f700 1 --2- 172.21.15.64:0/1523021135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98e40729a0 0x7f98e4072d80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:39.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.961+0000 7f98ec67f700 1 --2- 172.21.15.64:0/1523021135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98e40732c0 0x7f98e41110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:39.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.961+0000 7f98ec67f700 1 -- 172.21.15.64:0/1523021135 >> 172.21.15.64:0/1523021135 conn(0x7f98e40fc0f0 msgr2=0x7f98e40fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:05:39.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.961+0000 7f98ec67f700 1 -- 172.21.15.64:0/1523021135 shutdown_connections 2023-09-13T03:05:39.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.961+0000 7f98ec67f700 1 -- 172.21.15.64:0/1523021135 wait complete. 2023-09-13T03:05:39.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.962+0000 7f98ec67f700 1 Processor -- start 2023-09-13T03:05:39.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.962+0000 7f98ec67f700 1 -- start start 2023-09-13T03:05:39.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.962+0000 7f98ec67f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98e40729a0 0x7f98e419ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:39.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.962+0000 7f98ec67f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98e40732c0 0x7f98e419d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:39.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.963+0000 7f98ec67f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98e4111630 0x7f98e41a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:39.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.963+0000 7f98ec67f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e41a1e20 con 0x7f98e4111630 2023-09-13T03:05:39.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.963+0000 7f98ec67f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e41a1f90 con 0x7f98e40732c0 2023-09-13T03:05:39.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.963+0000 7f98ec67f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e41a2150 con 0x7f98e40729a0 2023-09-13T03:05:39.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.963+0000 7f98ea41b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98e40729a0 0x7f98e419ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:39.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.963+0000 7f98e9c1a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98e40732c0 0x7f98e419d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:39.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.963+0000 7f98ea41b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98e40729a0 0x7f98e419ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:42936/0 (socket says 172.21.15.64:42936) 2023-09-13T03:05:39.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.963+0000 7f98ea41b700 1 -- 172.21.15.64:0/2335260402 learned_addr learned my addr 172.21.15.64:0/2335260402 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:05:39.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.963+0000 7f98eac1c700 1 --2- 172.21.15.64:0/2335260402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98e4111630 0x7f98e41a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:39.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.963+0000 7f98ea41b700 1 -- 172.21.15.64:0/2335260402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98e40732c0 msgr2=0x7f98e419d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:39.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.963+0000 7f98ea41b700 1 --2- 172.21.15.64:0/2335260402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98e40732c0 0x7f98e419d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:39.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.963+0000 7f98ea41b700 1 -- 172.21.15.64:0/2335260402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98e4111630 msgr2=0x7f98e41a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:39.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.963+0000 7f98ea41b700 1 --2- 172.21.15.64:0/2335260402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98e4111630 0x7f98e41a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:39.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.963+0000 7f98ea41b700 1 -- 172.21.15.64:0/2335260402 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d40095f0 con 0x7f98e40729a0 2023-09-13T03:05:39.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.963+0000 7f98eac1c700 1 --2- 172.21.15.64:0/2335260402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98e4111630 0x7f98e41a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:05:39.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.964+0000 7f98ea41b700 1 --2- 172.21.15.64:0/2335260402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98e40729a0 0x7f98e419ce60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f98e0009810 tx=0x7f98e000fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:05:39.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.964+0000 7f98db7fe700 1 -- 172.21.15.64:0/2335260402 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e001c070 con 0x7f98e40729a0 2023-09-13T03:05:39.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.964+0000 7f98db7fe700 1 -- 172.21.15.64:0/2335260402 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98e000bbd0 con 0x7f98e40729a0 2023-09-13T03:05:39.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.964+0000 7f98db7fe700 1 -- 172.21.15.64:0/2335260402 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e00049c0 con 0x7f98e40729a0 2023-09-13T03:05:39.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.964+0000 7f98ec67f700 1 -- 172.21.15.64:0/2335260402 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98e0009520 con 0x7f98e40729a0 2023-09-13T03:05:39.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.964+0000 7f98ec67f700 1 -- 172.21.15.64:0/2335260402 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e41a2690 con 0x7f98e40729a0 2023-09-13T03:05:39.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.965+0000 7f98db7fe700 1 -- 172.21.15.64:0/2335260402 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7f98e0020070 con 0x7f98e40729a0 2023-09-13T03:05:39.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.966+0000 7f98ec67f700 1 -- 172.21.15.64:0/2335260402 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98e4103b10 con 0x7f98e40729a0 2023-09-13T03:05:39.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.966+0000 7f98db7fe700 1 --2- 172.21.15.64:0/2335260402 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f98d0038630 0x7f98d003aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:39.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.966+0000 7f98db7fe700 1 -- 172.21.15.64:0/2335260402 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f98e004bbc0 con 0x7f98e40729a0 2023-09-13T03:05:39.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.966+0000 7f98e9c1a700 1 --2- 172.21.15.64:0/2335260402 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f98d0038630 0x7f98d003aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:39.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.967+0000 7f98e9c1a700 1 --2- 172.21.15.64:0/2335260402 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f98d0038630 0x7f98d003aaf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f98e419e480 tx=0x7f98d4009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:05:39.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:39.971+0000 7f98db7fe700 1 -- 172.21.15.64:0/2335260402 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f98e0020320 con 0x7f98e40729a0 2023-09-13T03:05:39.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:39 smithi142 ceph-mon[134082]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:40.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.215+0000 7f98ec67f700 1 -- 172.21.15.64:0/2335260402 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) v1 -- 0x7f98e404e300 con 0x7f98e40729a0 2023-09-13T03:05:40.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.220+0000 7f98db7fe700 1 -- 172.21.15.64:0/2335260402 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v28)=0 v28) v1 ==== 121+0+0 (secure 0 0 0) 0x7f98e0027070 con 0x7f98e40729a0 2023-09-13T03:05:40.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.223+0000 7f98ec67f700 1 -- 172.21.15.64:0/2335260402 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f98d0038630 msgr2=0x7f98d003aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:40.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.223+0000 7f98ec67f700 1 --2- 172.21.15.64:0/2335260402 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f98d0038630 0x7f98d003aaf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f98e419e480 tx=0x7f98d4009310 comp rx=0 tx=0).stop 2023-09-13T03:05:40.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.223+0000 7f98ec67f700 1 -- 172.21.15.64:0/2335260402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98e40729a0 msgr2=0x7f98e419ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:40.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.223+0000 7f98ec67f700 1 --2- 172.21.15.64:0/2335260402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98e40729a0 0x7f98e419ce60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f98e0009810 tx=0x7f98e000fa00 comp rx=0 tx=0).stop 2023-09-13T03:05:40.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.223+0000 7f98ec67f700 1 -- 172.21.15.64:0/2335260402 shutdown_connections 2023-09-13T03:05:40.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.223+0000 7f98ec67f700 1 --2- 172.21.15.64:0/2335260402 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f98d0038630 0x7f98d003aaf0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:40.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.223+0000 7f98ec67f700 1 --2- 172.21.15.64:0/2335260402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98e4111630 0x7f98e41a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:40.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.223+0000 7f98ec67f700 1 --2- 172.21.15.64:0/2335260402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98e40729a0 0x7f98e419ce60 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:40.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.223+0000 7f98ec67f700 1 --2- 172.21.15.64:0/2335260402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98e40732c0 0x7f98e419d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:40.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.223+0000 7f98ec67f700 1 -- 172.21.15.64:0/2335260402 >> 172.21.15.64:0/2335260402 conn(0x7f98e40fc0f0 msgr2=0x7f98e40fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:05:40.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.223+0000 7f98ec67f700 1 -- 172.21.15.64:0/2335260402 shutdown_connections 2023-09-13T03:05:40.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.223+0000 7f98ec67f700 1 -- 172.21.15.64:0/2335260402 wait complete. 2023-09-13T03:05:40.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.460+0000 7f10f2c9c700 1 -- 172.21.15.64:0/3973907424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10ec100620 msgr2=0x7f10ec100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:40.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.460+0000 7f10f2c9c700 1 --2- 172.21.15.64:0/3973907424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10ec100620 0x7f10ec100aa0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f10d8009990 tx=0x7f10d8009ca0 comp rx=0 tx=0).stop 2023-09-13T03:05:40.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.461+0000 7f10f2c9c700 1 -- 172.21.15.64:0/3973907424 shutdown_connections 2023-09-13T03:05:40.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.461+0000 7f10f2c9c700 1 --2- 172.21.15.64:0/3973907424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10ec106620 0x7f10ec106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:40.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.461+0000 7f10f2c9c700 1 --2- 172.21.15.64:0/3973907424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10ec100fe0 0x7f10ec10d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:40.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.461+0000 7f10f2c9c700 1 --2- 172.21.15.64:0/3973907424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10ec100620 0x7f10ec100aa0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:40.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.461+0000 7f10f2c9c700 1 -- 172.21.15.64:0/3973907424 >> 172.21.15.64:0/3973907424 conn(0x7f10ec0fc0f0 msgr2=0x7f10ec0fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:05:40.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.461+0000 7f10f2c9c700 1 -- 172.21.15.64:0/3973907424 shutdown_connections 2023-09-13T03:05:40.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.461+0000 7f10f2c9c700 1 -- 172.21.15.64:0/3973907424 wait complete. 2023-09-13T03:05:40.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.462+0000 7f10f2c9c700 1 Processor -- start 2023-09-13T03:05:40.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.462+0000 7f10f2c9c700 1 -- start start 2023-09-13T03:05:40.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.462+0000 7f10f2c9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10ec100620 0x7f10ec1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:40.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.462+0000 7f10f2c9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10ec100fe0 0x7f10ec198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:40.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.462+0000 7f10f2c9c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10ec106620 0x7f10ec19d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:40.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.462+0000 7f10f2c9c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ec19d950 con 0x7f10ec100620 2023-09-13T03:05:40.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.463+0000 7f10f2c9c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ec19dac0 con 0x7f10ec100fe0 2023-09-13T03:05:40.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.463+0000 7f10f2c9c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ec19dc80 con 0x7f10ec106620 2023-09-13T03:05:40.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.463+0000 7f10f0a38700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10ec100620 0x7f10ec1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:40.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.463+0000 7f10f0a38700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10ec100620 0x7f10ec1989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33124/0 (socket says 172.21.15.64:33124) 2023-09-13T03:05:40.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.463+0000 7f10f0a38700 1 -- 172.21.15.64:0/371720774 learned_addr learned my addr 172.21.15.64:0/371720774 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:05:40.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.463+0000 7f10ebfff700 1 --2- 172.21.15.64:0/371720774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10ec100fe0 0x7f10ec198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:40.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.463+0000 7f10f1239700 1 --2- 172.21.15.64:0/371720774 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10ec106620 0x7f10ec19d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:40.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.463+0000 7f10f0a38700 1 -- 172.21.15.64:0/371720774 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10ec106620 msgr2=0x7f10ec19d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:40.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.463+0000 7f10f0a38700 1 --2- 172.21.15.64:0/371720774 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10ec106620 0x7f10ec19d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:40.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.463+0000 7f10f0a38700 1 -- 172.21.15.64:0/371720774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10ec100fe0 msgr2=0x7f10ec198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:40.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.463+0000 7f10f0a38700 1 --2- 172.21.15.64:0/371720774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10ec100fe0 0x7f10ec198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:40.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.463+0000 7f10f0a38700 1 -- 172.21.15.64:0/371720774 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d80095f0 con 0x7f10ec100620 2023-09-13T03:05:40.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.464+0000 7f10f0a38700 1 --2- 172.21.15.64:0/371720774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10ec100620 0x7f10ec1989a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f10e000c920 tx=0x7f10e000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:05:40.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.464+0000 7f10e9ffb700 1 -- 172.21.15.64:0/371720774 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e0004a30 con 0x7f10ec100620 2023-09-13T03:05:40.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.464+0000 7f10e9ffb700 1 -- 172.21.15.64:0/371720774 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10e00054e0 con 0x7f10ec100620 2023-09-13T03:05:40.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.464+0000 7f10f2c9c700 1 -- 172.21.15.64:0/371720774 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10ec19df60 con 0x7f10ec100620 2023-09-13T03:05:40.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.464+0000 7f10e9ffb700 1 -- 172.21.15.64:0/371720774 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e000eb80 con 0x7f10ec100620 2023-09-13T03:05:40.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.464+0000 7f10f2c9c700 1 -- 172.21.15.64:0/371720774 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10ec1a5800 con 0x7f10ec100620 2023-09-13T03:05:40.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.465+0000 7f10f2c9c700 1 -- 172.21.15.64:0/371720774 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10ec04e300 con 0x7f10ec100620 2023-09-13T03:05:40.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.466+0000 7f10e9ffb700 1 -- 172.21.15.64:0/371720774 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7f10e000ed20 con 0x7f10ec100620 2023-09-13T03:05:40.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.467+0000 7f10e9ffb700 1 --2- 172.21.15.64:0/371720774 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f10dc038670 0x7f10dc03ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:40.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.467+0000 7f10e9ffb700 1 -- 172.21.15.64:0/371720774 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f10e0053840 con 0x7f10ec100620 2023-09-13T03:05:40.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.467+0000 7f10ebfff700 1 --2- 172.21.15.64:0/371720774 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f10dc038670 0x7f10dc03ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:40.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.468+0000 7f10ebfff700 1 --2- 172.21.15.64:0/371720774 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f10dc038670 0x7f10dc03ab30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f10ec199910 tx=0x7f10d80058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:05:40.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.473+0000 7f10e9ffb700 1 -- 172.21.15.64:0/371720774 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f10e0050060 con 0x7f10ec100620 2023-09-13T03:05:40.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.731+0000 7f10f2c9c700 1 -- 172.21.15.64:0/371720774 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) v1 -- 0x7f10ec199d00 con 0x7f10ec100620 2023-09-13T03:05:40.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.735+0000 7f10e9ffb700 1 -- 172.21.15.64:0/371720774 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v29)=0 v29) v1 ==== 120+0+0 (secure 0 0 0) 0x7f10e0050020 con 0x7f10ec100620 2023-09-13T03:05:40.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.737+0000 7f10f2c9c700 1 -- 172.21.15.64:0/371720774 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f10dc038670 msgr2=0x7f10dc03ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:40.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.737+0000 7f10f2c9c700 1 --2- 172.21.15.64:0/371720774 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f10dc038670 0x7f10dc03ab30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f10ec199910 tx=0x7f10d80058e0 comp rx=0 tx=0).stop 2023-09-13T03:05:40.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.737+0000 7f10f2c9c700 1 -- 172.21.15.64:0/371720774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10ec100620 msgr2=0x7f10ec1989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:40.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.737+0000 7f10f2c9c700 1 --2- 172.21.15.64:0/371720774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10ec100620 0x7f10ec1989a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f10e000c920 tx=0x7f10e000cc30 comp rx=0 tx=0).stop 2023-09-13T03:05:40.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.737+0000 7f10f2c9c700 1 -- 172.21.15.64:0/371720774 shutdown_connections 2023-09-13T03:05:40.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.737+0000 7f10f2c9c700 1 --2- 172.21.15.64:0/371720774 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f10dc038670 0x7f10dc03ab30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:40.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.737+0000 7f10f2c9c700 1 --2- 172.21.15.64:0/371720774 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10ec106620 0x7f10ec19d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:40.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.737+0000 7f10f2c9c700 1 --2- 172.21.15.64:0/371720774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10ec100620 0x7f10ec1989a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:40.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.737+0000 7f10f2c9c700 1 --2- 172.21.15.64:0/371720774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10ec100fe0 0x7f10ec198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:40.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.738+0000 7f10f2c9c700 1 -- 172.21.15.64:0/371720774 >> 172.21.15.64:0/371720774 conn(0x7f10ec0fc0f0 msgr2=0x7f10ec072930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:05:40.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.738+0000 7f10f2c9c700 1 -- 172.21.15.64:0/371720774 shutdown_connections 2023-09-13T03:05:40.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:40.738+0000 7f10f2c9c700 1 -- 172.21.15.64:0/371720774 wait complete. 2023-09-13T03:05:41.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.034+0000 7f6d3bd03700 1 -- 172.21.15.64:0/290052860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d34073350 msgr2=0x7f6d3410c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:41.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.034+0000 7f6d3bd03700 1 --2- 172.21.15.64:0/290052860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d34073350 0x7f6d3410c920 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6d24009990 tx=0x7f6d24009ca0 comp rx=0 tx=0).stop 2023-09-13T03:05:41.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.034+0000 7f6d3bd03700 1 -- 172.21.15.64:0/290052860 shutdown_connections 2023-09-13T03:05:41.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.034+0000 7f6d3bd03700 1 --2- 172.21.15.64:0/290052860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d3410ce60 0x7f6d3410f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:41.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.034+0000 7f6d3bd03700 1 --2- 172.21.15.64:0/290052860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d34072a30 0x7f6d34072e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:41.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.034+0000 7f6d3bd03700 1 --2- 172.21.15.64:0/290052860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d34073350 0x7f6d3410c920 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:41.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.034+0000 7f6d3bd03700 1 -- 172.21.15.64:0/290052860 >> 172.21.15.64:0/290052860 conn(0x7f6d340fc0f0 msgr2=0x7f6d340fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:05:41.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.034+0000 7f6d3bd03700 1 -- 172.21.15.64:0/290052860 shutdown_connections 2023-09-13T03:05:41.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.034+0000 7f6d3bd03700 1 -- 172.21.15.64:0/290052860 wait complete. 2023-09-13T03:05:41.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.035+0000 7f6d3bd03700 1 Processor -- start 2023-09-13T03:05:41.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.035+0000 7f6d3bd03700 1 -- start start 2023-09-13T03:05:41.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.036+0000 7f6d3bd03700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d34072a30 0x7f6d34198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:41.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.036+0000 7f6d3bd03700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d34073350 0x7f6d34198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:41.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.036+0000 7f6d3bd03700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d3410ce60 0x7f6d3419d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:41.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.036+0000 7f6d3bd03700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3419d900 con 0x7f6d34072a30 2023-09-13T03:05:41.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.036+0000 7f6d3bd03700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3419da70 con 0x7f6d34073350 2023-09-13T03:05:41.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.036+0000 7f6d3bd03700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3419dc30 con 0x7f6d3410ce60 2023-09-13T03:05:41.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.036+0000 7f6d39a9f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d34072a30 0x7f6d34198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:41.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.036+0000 7f6d39a9f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d34072a30 0x7f6d34198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33148/0 (socket says 172.21.15.64:33148) 2023-09-13T03:05:41.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.036+0000 7f6d39a9f700 1 -- 172.21.15.64:0/2577536443 learned_addr learned my addr 172.21.15.64:0/2577536443 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:05:41.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.036+0000 7f6d3a2a0700 1 --2- 172.21.15.64:0/2577536443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d3410ce60 0x7f6d3419d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:41.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.036+0000 7f6d3929e700 1 --2- 172.21.15.64:0/2577536443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d34073350 0x7f6d34198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:41.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.037+0000 7f6d39a9f700 1 -- 172.21.15.64:0/2577536443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d3410ce60 msgr2=0x7f6d3419d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:41.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.037+0000 7f6d39a9f700 1 --2- 172.21.15.64:0/2577536443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d3410ce60 0x7f6d3419d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:41.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.037+0000 7f6d39a9f700 1 -- 172.21.15.64:0/2577536443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d34073350 msgr2=0x7f6d34198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:41.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.037+0000 7f6d39a9f700 1 --2- 172.21.15.64:0/2577536443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d34073350 0x7f6d34198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:41.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.037+0000 7f6d39a9f700 1 -- 172.21.15.64:0/2577536443 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d240095f0 con 0x7f6d34072a30 2023-09-13T03:05:41.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.037+0000 7f6d3a2a0700 1 --2- 172.21.15.64:0/2577536443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d3410ce60 0x7f6d3419d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:05:41.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.037+0000 7f6d39a9f700 1 --2- 172.21.15.64:0/2577536443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d34072a30 0x7f6d34198950 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6d3000e9a0 tx=0x7f6d3000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:05:41.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.037+0000 7f6d2affd700 1 -- 172.21.15.64:0/2577536443 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d30009790 con 0x7f6d34072a30 2023-09-13T03:05:41.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.037+0000 7f6d2affd700 1 -- 172.21.15.64:0/2577536443 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d30010430 con 0x7f6d34072a30 2023-09-13T03:05:41.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.037+0000 7f6d3bd03700 1 -- 172.21.15.64:0/2577536443 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d3419df10 con 0x7f6d34072a30 2023-09-13T03:05:41.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.037+0000 7f6d2affd700 1 -- 172.21.15.64:0/2577536443 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d30018680 con 0x7f6d34072a30 2023-09-13T03:05:41.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.037+0000 7f6d3bd03700 1 -- 172.21.15.64:0/2577536443 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d341a57b0 con 0x7f6d34072a30 2023-09-13T03:05:41.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.038+0000 7f6d2affd700 1 -- 172.21.15.64:0/2577536443 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7f6d300105a0 con 0x7f6d34072a30 2023-09-13T03:05:41.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.038+0000 7f6d3bd03700 1 -- 172.21.15.64:0/2577536443 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d3404e300 con 0x7f6d34072a30 2023-09-13T03:05:41.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.039+0000 7f6d2affd700 1 --2- 172.21.15.64:0/2577536443 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6d20038630 0x7f6d2003aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:41.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.039+0000 7f6d2affd700 1 -- 172.21.15.64:0/2577536443 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6d30014070 con 0x7f6d34072a30 2023-09-13T03:05:41.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.039+0000 7f6d3929e700 1 --2- 172.21.15.64:0/2577536443 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6d20038630 0x7f6d2003aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:41.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.040+0000 7f6d3929e700 1 --2- 172.21.15.64:0/2577536443 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6d20038630 0x7f6d2003aaf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6d34199f70 tx=0x7f6d24005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:05:41.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.044+0000 7f6d2affd700 1 -- 172.21.15.64:0/2577536443 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6d3001f780 con 0x7f6d34072a30 2023-09-13T03:05:41.074 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:40 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/339143982' entity='client.admin' 2023-09-13T03:05:41.075 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:40 smithi064 ceph-mon[97675]: from='client.? ' entity='client.admin' 2023-09-13T03:05:41.075 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:40 smithi064 ceph-mon[97675]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:41.075 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:40 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/371720774' entity='client.admin' 2023-09-13T03:05:41.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:40 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/339143982' entity='client.admin' 2023-09-13T03:05:41.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:40 smithi170 ceph-mon[134328]: from='client.? ' entity='client.admin' 2023-09-13T03:05:41.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:40 smithi170 ceph-mon[134328]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:41.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:40 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/371720774' entity='client.admin' 2023-09-13T03:05:41.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:40 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/339143982' entity='client.admin' 2023-09-13T03:05:41.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:40 smithi142 ceph-mon[134082]: from='client.? ' entity='client.admin' 2023-09-13T03:05:41.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:40 smithi142 ceph-mon[134082]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:41.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:40 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/371720774' entity='client.admin' 2023-09-13T03:05:41.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.279+0000 7f6d3bd03700 1 -- 172.21.15.64:0/2577536443 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) v1 -- 0x7f6d34199cb0 con 0x7f6d34072a30 2023-09-13T03:05:41.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.282+0000 7f6d2affd700 1 -- 172.21.15.64:0/2577536443 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v30)=0 v30) v1 ==== 121+0+0 (secure 0 0 0) 0x7f6d3001f780 con 0x7f6d34072a30 2023-09-13T03:05:41.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.285+0000 7f6d3bd03700 1 -- 172.21.15.64:0/2577536443 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6d20038630 msgr2=0x7f6d2003aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:41.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.285+0000 7f6d3bd03700 1 --2- 172.21.15.64:0/2577536443 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6d20038630 0x7f6d2003aaf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6d34199f70 tx=0x7f6d24005fd0 comp rx=0 tx=0).stop 2023-09-13T03:05:41.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.285+0000 7f6d3bd03700 1 -- 172.21.15.64:0/2577536443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d34072a30 msgr2=0x7f6d34198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:41.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.285+0000 7f6d3bd03700 1 --2- 172.21.15.64:0/2577536443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d34072a30 0x7f6d34198950 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6d3000e9a0 tx=0x7f6d3000ecb0 comp rx=0 tx=0).stop 2023-09-13T03:05:41.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.285+0000 7f6d3bd03700 1 -- 172.21.15.64:0/2577536443 shutdown_connections 2023-09-13T03:05:41.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.285+0000 7f6d3bd03700 1 --2- 172.21.15.64:0/2577536443 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6d20038630 0x7f6d2003aaf0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:41.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.285+0000 7f6d3bd03700 1 --2- 172.21.15.64:0/2577536443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d3410ce60 0x7f6d3419d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:41.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.285+0000 7f6d3bd03700 1 --2- 172.21.15.64:0/2577536443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d34072a30 0x7f6d34198950 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:41.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.285+0000 7f6d3bd03700 1 --2- 172.21.15.64:0/2577536443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d34073350 0x7f6d34198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:41.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.285+0000 7f6d3bd03700 1 -- 172.21.15.64:0/2577536443 >> 172.21.15.64:0/2577536443 conn(0x7f6d340fc0f0 msgr2=0x7f6d3410f010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:05:41.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.285+0000 7f6d3bd03700 1 -- 172.21.15.64:0/2577536443 shutdown_connections 2023-09-13T03:05:41.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:41.285+0000 7f6d3bd03700 1 -- 172.21.15.64:0/2577536443 wait complete. 2023-09-13T03:05:41.657 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-09-13T03:05:41.658 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-13T03:05:41.658 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.conf 2023-09-13T03:05:41.732 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-13T03:05:41.733 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-13T03:05:41.841 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-09-13T03:05:41.841 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/ceph/ceph.conf 2023-09-13T03:05:41.878 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-09-13T03:05:41.878 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-13T03:05:41.949 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-09-13T03:05:41.949 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/ceph/ceph.conf 2023-09-13T03:05:41.986 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-09-13T03:05:41.987 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-13T03:05:42.056 INFO:tasks.cephadm:Adding mgr.x on smithi064 2023-09-13T03:05:42.056 INFO:tasks.cephadm:Adding mgr.y on smithi142 2023-09-13T03:05:42.056 INFO:tasks.cephadm:Adding mgr.z on smithi170 2023-09-13T03:05:42.057 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph orch apply mgr '3;smithi064=x;smithi142=y;smithi170=z' 2023-09-13T03:05:42.652 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-13T03:05:42.659 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:42 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/2577536443' entity='client.admin' 2023-09-13T03:05:42.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:42 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2577536443' entity='client.admin' 2023-09-13T03:05:42.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:42 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/2577536443' entity='client.admin' 2023-09-13T03:05:43.456 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:43 smithi170 ceph-mon[134328]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:43.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:43 smithi064 ceph-mon[97675]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:43.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:43 smithi142 ceph-mon[134082]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:43.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:43.956+0000 7fc8efb43700 1 -- 172.21.15.170:0/1554854197 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8e80ff590 msgr2=0x7fc8e80ff9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:43.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:43.956+0000 7fc8efb43700 1 --2- 172.21.15.170:0/1554854197 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8e80ff590 0x7fc8e80ff9b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4009990 tx=0x7fc8e4009ca0 comp rx=0 tx=0).stop 2023-09-13T03:05:43.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:43.956+0000 7fc8efb43700 1 -- 172.21.15.170:0/1554854197 shutdown_connections 2023-09-13T03:05:43.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:43.956+0000 7fc8efb43700 1 --2- 172.21.15.170:0/1554854197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8e8101130 0x7fc8e81079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:43.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:43.956+0000 7fc8efb43700 1 --2- 172.21.15.170:0/1554854197 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8e80ff590 0x7fc8e80ff9b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:43.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:43.956+0000 7fc8efb43700 1 --2- 172.21.15.170:0/1554854197 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8e8100790 0x7fc8e8100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:43.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:43.956+0000 7fc8efb43700 1 -- 172.21.15.170:0/1554854197 >> 172.21.15.170:0/1554854197 conn(0x7fc8e80fab10 msgr2=0x7fc8e80fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:05:43.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:43.956+0000 7fc8efb43700 1 -- 172.21.15.170:0/1554854197 shutdown_connections 2023-09-13T03:05:43.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:43.956+0000 7fc8efb43700 1 -- 172.21.15.170:0/1554854197 wait complete. 2023-09-13T03:05:43.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:43.956+0000 7fc8efb43700 1 Processor -- start 2023-09-13T03:05:43.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:43.956+0000 7fc8efb43700 1 -- start start 2023-09-13T03:05:43.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:43.957+0000 7fc8efb43700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8e80ff590 0x7fc8e8103520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:43.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:43.957+0000 7fc8efb43700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8e8100790 0x7fc8e8101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:43.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:43.957+0000 7fc8efb43700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8e8101130 0x7fc8e81020b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:43.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:43.957+0000 7fc8efb43700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e8102830 con 0x7fc8e8101130 2023-09-13T03:05:43.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:43.957+0000 7fc8efb43700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e81029a0 con 0x7fc8e8100790 2023-09-13T03:05:43.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:43.957+0000 7fc8efb43700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e81a0510 con 0x7fc8e80ff590 2023-09-13T03:05:43.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:43.957+0000 7fc8ed8df700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8e80ff590 0x7fc8e8103520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:43.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:43.957+0000 7fc8ed8df700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8e80ff590 0x7fc8e8103520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45392/0 (socket says 172.21.15.170:45392) 2023-09-13T03:05:43.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:43.957+0000 7fc8ed8df700 1 -- 172.21.15.170:0/384026177 learned_addr learned my addr 172.21.15.170:0/384026177 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-09-13T03:05:43.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:43.957+0000 7fc8ed0de700 1 --2- 172.21.15.170:0/384026177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8e8100790 0x7fc8e8101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:43.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:43.957+0000 7fc8ee0e0700 1 --2- 172.21.15.170:0/384026177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8e8101130 0x7fc8e81020b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:43.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:43.957+0000 7fc8ed8df700 1 -- 172.21.15.170:0/384026177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8e8100790 msgr2=0x7fc8e8101b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:43.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:43.957+0000 7fc8ed8df700 1 --2- 172.21.15.170:0/384026177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8e8100790 0x7fc8e8101b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:43.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:43.957+0000 7fc8ed8df700 1 -- 172.21.15.170:0/384026177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8e8101130 msgr2=0x7fc8e81020b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:43.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:43.957+0000 7fc8ed8df700 1 --2- 172.21.15.170:0/384026177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8e8101130 0x7fc8e81020b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:43.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:43.957+0000 7fc8ed8df700 1 -- 172.21.15.170:0/384026177 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e40095f0 con 0x7fc8e80ff590 2023-09-13T03:05:43.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:43.957+0000 7fc8ed0de700 1 --2- 172.21.15.170:0/384026177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8e8100790 0x7fc8e8101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T03:05:43.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:43.957+0000 7fc8ee0e0700 1 --2- 172.21.15.170:0/384026177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8e8101130 0x7fc8e81020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T03:05:43.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:43.957+0000 7fc8ed8df700 1 --2- 172.21.15.170:0/384026177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8e80ff590 0x7fc8e8103520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc8e400b5c0 tx=0x7fc8e400f7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:05:43.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:43.957+0000 7fc8deffd700 1 -- 172.21.15.170:0/384026177 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e401d070 con 0x7fc8e80ff590 2023-09-13T03:05:43.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:43.957+0000 7fc8deffd700 1 -- 172.21.15.170:0/384026177 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8e4003e20 con 0x7fc8e80ff590 2023-09-13T03:05:43.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:43.957+0000 7fc8efb43700 1 -- 172.21.15.170:0/384026177 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8e81a0790 con 0x7fc8e80ff590 2023-09-13T03:05:43.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:43.957+0000 7fc8deffd700 1 -- 172.21.15.170:0/384026177 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4017650 con 0x7fc8e80ff590 2023-09-13T03:05:43.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:43.957+0000 7fc8efb43700 1 -- 172.21.15.170:0/384026177 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8e81a0c80 con 0x7fc8e80ff590 2023-09-13T03:05:43.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:43.958+0000 7fc8deffd700 1 -- 172.21.15.170:0/384026177 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7fc8e40177b0 con 0x7fc8e80ff590 2023-09-13T03:05:43.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:43.958+0000 7fc8efb43700 1 -- 172.21.15.170:0/384026177 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8e810ae90 con 0x7fc8e80ff590 2023-09-13T03:05:43.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:43.958+0000 7fc8deffd700 1 --2- 172.21.15.170:0/384026177 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc8d4038630 0x7fc8d403aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:43.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:43.958+0000 7fc8deffd700 1 -- 172.21.15.170:0/384026177 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc8e404c5a0 con 0x7fc8e80ff590 2023-09-13T03:05:43.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:43.959+0000 7fc8ed0de700 1 --2- 172.21.15.170:0/384026177 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc8d4038630 0x7fc8d403aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:43.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:43.959+0000 7fc8ed0de700 1 --2- 172.21.15.170:0/384026177 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc8d4038630 0x7fc8d403aaf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8007900 tx=0x7fc8d8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:05:43.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:43.963+0000 7fc8deffd700 1 -- 172.21.15.170:0/384026177 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fc8e4016aa0 con 0x7fc8e80ff590 2023-09-13T03:05:44.163 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:44.162+0000 7fc8efb43700 1 -- 172.21.15.170:0/384026177 --> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi064=x;smithi142=y;smithi170=z", "target": ["mon-mgr", ""]}) v1 -- 0x7fc8e80687f0 con 0x7fc8d4038630 2023-09-13T03:05:44.169 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:44.168+0000 7fc8deffd700 1 -- 172.21.15.170:0/384026177 <== mgr.14150 v2:172.21.15.64:6800/873679308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fc8e80687f0 con 0x7fc8d4038630 2023-09-13T03:05:44.169 INFO:teuthology.orchestra.run.smithi170.stdout:Scheduled mgr update... 2023-09-13T03:05:44.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:44.170+0000 7fc8efb43700 1 -- 172.21.15.170:0/384026177 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc8d4038630 msgr2=0x7fc8d403aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:44.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:44.170+0000 7fc8efb43700 1 --2- 172.21.15.170:0/384026177 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc8d4038630 0x7fc8d403aaf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8007900 tx=0x7fc8d8008040 comp rx=0 tx=0).stop 2023-09-13T03:05:44.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:44.170+0000 7fc8efb43700 1 -- 172.21.15.170:0/384026177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8e80ff590 msgr2=0x7fc8e8103520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:44.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:44.170+0000 7fc8efb43700 1 --2- 172.21.15.170:0/384026177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8e80ff590 0x7fc8e8103520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc8e400b5c0 tx=0x7fc8e400f7b0 comp rx=0 tx=0).stop 2023-09-13T03:05:44.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:44.170+0000 7fc8efb43700 1 -- 172.21.15.170:0/384026177 shutdown_connections 2023-09-13T03:05:44.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:44.170+0000 7fc8efb43700 1 --2- 172.21.15.170:0/384026177 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc8d4038630 0x7fc8d403aaf0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:44.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:44.170+0000 7fc8efb43700 1 --2- 172.21.15.170:0/384026177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8e8101130 0x7fc8e81020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:44.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:44.170+0000 7fc8efb43700 1 --2- 172.21.15.170:0/384026177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8e80ff590 0x7fc8e8103520 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:44.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:44.170+0000 7fc8efb43700 1 --2- 172.21.15.170:0/384026177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8e8100790 0x7fc8e8101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:44.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:44.170+0000 7fc8efb43700 1 -- 172.21.15.170:0/384026177 >> 172.21.15.170:0/384026177 conn(0x7fc8e80fab10 msgr2=0x7fc8e80680e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:05:44.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:44.171+0000 7fc8efb43700 1 -- 172.21.15.170:0/384026177 shutdown_connections 2023-09-13T03:05:44.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:05:44.171+0000 7fc8efb43700 1 -- 172.21.15.170:0/384026177 wait complete. 2023-09-13T03:05:44.593 DEBUG:teuthology.orchestra.run.smithi142:mgr.y> sudo journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mgr.y.service 2023-09-13T03:05:44.596 DEBUG:teuthology.orchestra.run.smithi170:mgr.z> sudo journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mgr.z.service 2023-09-13T03:05:44.599 INFO:tasks.cephadm:Deploying OSDs... 2023-09-13T03:05:44.599 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-13T03:05:44.599 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/scratch_devs of=/dev/stdout 2023-09-13T03:05:44.627 INFO:journalctl@ceph.mgr.y.smithi142.stdout:-- Logs begin at Wed 2023-09-13 02:43:02 UTC. -- 2023-09-13T03:05:44.628 INFO:journalctl@ceph.mgr.z.smithi170.stdout:-- Logs begin at Wed 2023-09-13 02:42:51 UTC. -- 2023-09-13T03:05:44.643 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-13T03:05:44.643 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_1 2023-09-13T03:05:44.728 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-13T03:05:44.728 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-13T03:05:44.729 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 448 Links: 1 2023-09-13T03:05:44.729 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-13T03:05:44.729 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2023-09-13T03:05:44.729 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2023-09-13 03:04:34.177004704 +0000 2023-09-13T03:05:44.729 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2023-09-13 02:58:50.055653237 +0000 2023-09-13T03:05:44.730 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2023-09-13 02:58:50.055653237 +0000 2023-09-13T03:05:44.730 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2023-09-13 02:58:50.028653747 +0000 2023-09-13T03:05:44.730 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-13T03:05:44.840 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2023-09-13T03:05:44.841 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2023-09-13T03:05:44.841 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000309213 s, 1.7 MB/s 2023-09-13T03:05:44.843 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-13T03:05:44.889 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_2 2023-09-13T03:05:44.977 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-13T03:05:44.978 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-13T03:05:44.978 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 454 Links: 1 2023-09-13T03:05:44.978 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-13T03:05:44.979 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2023-09-13T03:05:44.979 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2023-09-13 03:04:34.259003146 +0000 2023-09-13T03:05:44.979 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2023-09-13 02:58:50.888637501 +0000 2023-09-13T03:05:44.979 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2023-09-13 02:58:50.888637501 +0000 2023-09-13T03:05:44.980 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2023-09-13 02:58:50.862637992 +0000 2023-09-13T03:05:44.980 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-13T03:05:45.048 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2023-09-13T03:05:45.048 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2023-09-13T03:05:45.049 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000301867 s, 1.7 MB/s 2023-09-13T03:05:45.051 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-13T03:05:45.096 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_3 2023-09-13T03:05:45.182 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-13T03:05:45.183 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-13T03:05:45.183 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 460 Links: 1 2023-09-13T03:05:45.183 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-13T03:05:45.183 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2023-09-13T03:05:45.183 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2023-09-13 03:04:34.340001608 +0000 2023-09-13T03:05:45.184 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2023-09-13 02:58:51.727621653 +0000 2023-09-13T03:05:45.184 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2023-09-13 02:58:51.727621653 +0000 2023-09-13T03:05:45.184 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2023-09-13 02:58:51.701622144 +0000 2023-09-13T03:05:45.184 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-13T03:05:45.288 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2023-09-13T03:05:45.289 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2023-09-13T03:05:45.289 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000292817 s, 1.7 MB/s 2023-09-13T03:05:45.291 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-13T03:05:45.335 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_4 2023-09-13T03:05:45.419 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-13T03:05:45.419 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-13T03:05:45.419 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 466 Links: 1 2023-09-13T03:05:45.419 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-13T03:05:45.419 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2023-09-13T03:05:45.420 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2023-09-13 03:04:34.430999880 +0000 2023-09-13T03:05:45.420 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2023-09-13 02:58:52.568605766 +0000 2023-09-13T03:05:45.420 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2023-09-13 02:58:52.568605766 +0000 2023-09-13T03:05:45.420 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2023-09-13 02:58:52.542606257 +0000 2023-09-13T03:05:45.421 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-13T03:05:45.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:45 smithi142 ceph-mon[134082]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi064=x;smithi142=y;smithi170=z", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:05:45.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:45 smithi142 ceph-mon[134082]: Saving service mgr spec with placement smithi064=x;smithi142=y;smithi170=z;count:3 2023-09-13T03:05:45.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:45 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:45.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:45 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:05:45.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:45 smithi142 ceph-mon[134082]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:45.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:45 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:45.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:45 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:05:45.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:45 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:45.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:45 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-13T03:05:45.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:45 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-13T03:05:45.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:45 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-13T03:05:45.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:45 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:45.518 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:45 smithi064 ceph-mon[97675]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi064=x;smithi142=y;smithi170=z", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:05:45.519 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:45 smithi064 ceph-mon[97675]: Saving service mgr spec with placement smithi064=x;smithi142=y;smithi170=z;count:3 2023-09-13T03:05:45.519 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:45 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:45.519 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:45 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:05:45.519 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:45 smithi064 ceph-mon[97675]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:45.520 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:45 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:45.520 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:45 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:05:45.520 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:45 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:45.520 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:45 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-13T03:05:45.520 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:45 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-13T03:05:45.521 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:45 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-13T03:05:45.521 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:45 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:45.525 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2023-09-13T03:05:45.525 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2023-09-13T03:05:45.525 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000277721 s, 1.8 MB/s 2023-09-13T03:05:45.528 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-13T03:05:45.574 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-09-13T03:05:45.574 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/scratch_devs of=/dev/stdout 2023-09-13T03:05:45.595 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-13T03:05:45.595 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_1 2023-09-13T03:05:45.655 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:45 smithi170 ceph-mon[134328]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi064=x;smithi142=y;smithi170=z", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:05:45.655 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:45 smithi170 ceph-mon[134328]: Saving service mgr spec with placement smithi064=x;smithi142=y;smithi170=z;count:3 2023-09-13T03:05:45.655 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:45 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:45.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:45 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:05:45.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:45 smithi170 ceph-mon[134328]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:45.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:45 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:45.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:45 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:05:45.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:45 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:45.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:45 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-13T03:05:45.658 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:45 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-13T03:05:45.658 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:45 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-13T03:05:45.658 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:45 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:45.660 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-13T03:05:45.660 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-13T03:05:45.660 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 6h/6d Inode: 383416 Links: 1 2023-09-13T03:05:45.661 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-13T03:05:45.661 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2023-09-13T03:05:45.661 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2023-09-13 03:05:10.357487392 +0000 2023-09-13T03:05:45.662 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2023-09-13 02:55:37.880595749 +0000 2023-09-13T03:05:45.662 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2023-09-13 02:55:37.880595749 +0000 2023-09-13T03:05:45.662 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: - 2023-09-13T03:05:45.663 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-13T03:05:45.730 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2023-09-13T03:05:45.731 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2023-09-13T03:05:45.731 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000278236 s, 1.8 MB/s 2023-09-13T03:05:45.732 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-13T03:05:45.792 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_2 2023-09-13T03:05:45.852 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-13T03:05:45.852 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-13T03:05:45.852 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 6h/6d Inode: 383554 Links: 1 2023-09-13T03:05:45.852 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-13T03:05:45.852 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2023-09-13T03:05:45.853 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2023-09-13 03:05:10.357487392 +0000 2023-09-13T03:05:45.853 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2023-09-13 02:55:38.390586454 +0000 2023-09-13T03:05:45.853 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2023-09-13 02:55:38.390586454 +0000 2023-09-13T03:05:45.853 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: - 2023-09-13T03:05:45.853 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-13T03:05:45.922 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2023-09-13T03:05:45.922 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2023-09-13T03:05:45.922 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000245809 s, 2.1 MB/s 2023-09-13T03:05:45.923 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-13T03:05:45.984 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_3 2023-09-13T03:05:46.045 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-13T03:05:46.045 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-13T03:05:46.045 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 6h/6d Inode: 389739 Links: 1 2023-09-13T03:05:46.046 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-13T03:05:46.046 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2023-09-13T03:05:46.046 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2023-09-13 03:05:10.357487392 +0000 2023-09-13T03:05:46.046 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2023-09-13 02:55:38.890577341 +0000 2023-09-13T03:05:46.047 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2023-09-13 02:55:38.890577341 +0000 2023-09-13T03:05:46.047 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: - 2023-09-13T03:05:46.047 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-13T03:05:46.114 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2023-09-13T03:05:46.114 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2023-09-13T03:05:46.114 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000236607 s, 2.2 MB/s 2023-09-13T03:05:46.116 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-13T03:05:46.175 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_4 2023-09-13T03:05:46.234 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-13T03:05:46.235 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-13T03:05:46.235 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 6h/6d Inode: 383764 Links: 1 2023-09-13T03:05:46.235 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-13T03:05:46.235 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2023-09-13T03:05:46.235 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2023-09-13 03:05:10.357487392 +0000 2023-09-13T03:05:46.236 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2023-09-13 02:55:39.382568374 +0000 2023-09-13T03:05:46.236 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2023-09-13 02:55:39.382568374 +0000 2023-09-13T03:05:46.236 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: - 2023-09-13T03:05:46.236 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-13T03:05:46.302 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2023-09-13T03:05:46.302 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2023-09-13T03:05:46.303 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000183907 s, 2.8 MB/s 2023-09-13T03:05:46.304 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-13T03:05:46.359 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:46 smithi170 ceph-mon[134328]: Deploying daemon mgr.z on smithi170 2023-09-13T03:05:46.363 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-09-13T03:05:46.363 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/scratch_devs of=/dev/stdout 2023-09-13T03:05:46.389 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-13T03:05:46.390 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_1 2023-09-13T03:05:46.446 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-13T03:05:46.447 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-13T03:05:46.447 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 6h/6d Inode: 385622 Links: 1 2023-09-13T03:05:46.447 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-13T03:05:46.447 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2023-09-13T03:05:46.448 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2023-09-13 03:05:46.410396507 +0000 2023-09-13T03:05:46.448 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2023-09-13 02:55:37.695047023 +0000 2023-09-13T03:05:46.448 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2023-09-13 02:55:37.695047023 +0000 2023-09-13T03:05:46.448 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2023-09-13T03:05:46.448 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-13T03:05:46.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:46 smithi142 ceph-mon[134082]: Deploying daemon mgr.z on smithi170 2023-09-13T03:05:46.584 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2023-09-13T03:05:46.584 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2023-09-13T03:05:46.584 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000135703 s, 3.8 MB/s 2023-09-13T03:05:46.585 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-13T03:05:46.605 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_2 2023-09-13T03:05:46.660 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:05:46 smithi170 systemd[1]: Starting Ceph mgr.z for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:05:46.668 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-13T03:05:46.668 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-13T03:05:46.668 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 6h/6d Inode: 389219 Links: 1 2023-09-13T03:05:46.668 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-13T03:05:46.669 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2023-09-13T03:05:46.669 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2023-09-13 03:05:46.536393518 +0000 2023-09-13T03:05:46.669 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2023-09-13 02:55:38.190035386 +0000 2023-09-13T03:05:46.669 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2023-09-13 02:55:38.190035386 +0000 2023-09-13T03:05:46.669 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2023-09-13T03:05:46.670 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-13T03:05:46.671 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:46 smithi064 ceph-mon[97675]: Deploying daemon mgr.z on smithi170 2023-09-13T03:05:46.740 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2023-09-13T03:05:46.741 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2023-09-13T03:05:46.741 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000118802 s, 4.3 MB/s 2023-09-13T03:05:46.742 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-13T03:05:46.800 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_3 2023-09-13T03:05:46.857 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-13T03:05:46.858 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-13T03:05:46.859 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 6h/6d Inode: 380908 Links: 1 2023-09-13T03:05:46.859 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-13T03:05:46.859 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2023-09-13T03:05:46.859 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2023-09-13 03:05:46.536393518 +0000 2023-09-13T03:05:46.859 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2023-09-13 02:55:38.675023985 +0000 2023-09-13T03:05:46.860 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2023-09-13 02:55:38.675023985 +0000 2023-09-13T03:05:46.860 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2023-09-13T03:05:46.860 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-13T03:05:46.922 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2023-09-13T03:05:46.923 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2023-09-13T03:05:46.923 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000103749 s, 4.9 MB/s 2023-09-13T03:05:46.924 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-13T03:05:46.982 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_4 2023-09-13T03:05:47.025 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:05:46 smithi170 podman[136324]: 2023-09-13T03:05:47.039 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-13T03:05:47.040 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-13T03:05:47.040 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 6h/6d Inode: 391337 Links: 1 2023-09-13T03:05:47.040 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-13T03:05:47.040 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2023-09-13T03:05:47.041 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2023-09-13 03:05:46.536393518 +0000 2023-09-13T03:05:47.041 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2023-09-13 02:55:39.198011690 +0000 2023-09-13T03:05:47.041 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2023-09-13 02:55:39.198011690 +0000 2023-09-13T03:05:47.041 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2023-09-13T03:05:47.042 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-13T03:05:47.104 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2023-09-13T03:05:47.104 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2023-09-13T03:05:47.104 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000119235 s, 4.3 MB/s 2023-09-13T03:05:47.105 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-13T03:05:47.165 INFO:tasks.cephadm:Deploying osd.0 on smithi064 with /dev/vg_nvme/lv_4... 2023-09-13T03:05:47.165 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-09-13T03:05:47.363 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:47 smithi170 ceph-mon[134328]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:47.363 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:05:47 smithi170 ceph-mgr[136489]: -- 172.21.15.170:0/1847950952 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5561b75ef1e0 con 0x5561b75c4c00 2023-09-13T03:05:47.363 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:05:47 smithi170 bash[136324]: a56ff4b2796538dd2a97655fcb2f75843f899282742068fa39a70675f502248b 2023-09-13T03:05:47.364 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:05:47 smithi170 systemd[1]: Started Ceph mgr.z for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da. 2023-09-13T03:05:47.364 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:05:47 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:05:47.284+0000 7f4ff0965200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-13T03:05:47.364 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:05:47 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:05:47.362+0000 7f4ff0965200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-13T03:05:47.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:47 smithi142 ceph-mon[134082]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:47.659 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:05:47 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:05:47.449+0000 7f4ff0965200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-13T03:05:47.660 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:05:47 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:05:47.542+0000 7f4ff0965200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-13T03:05:47.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:47 smithi064 ceph-mon[97675]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:47.908 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:05:48.073 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:05:47 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:05:47.710+0000 7f4ff0965200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-13T03:05:48.385 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:05:48 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:05:48.072+0000 7f4ff0965200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-13T03:05:48.385 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:05:48 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:05:48.200+0000 7f4ff0965200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-13T03:05:48.385 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:05:48 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:05:48.289+0000 7f4ff0965200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-13T03:05:48.386 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:48 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:48.386 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:48 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:48.386 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:48 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:48.386 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:48 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-13T03:05:48.386 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:48 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-13T03:05:48.387 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:48 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-13T03:05:48.387 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:48 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:48.387 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:48 smithi170 ceph-mon[134328]: Deploying daemon mgr.y on smithi142 2023-09-13T03:05:48.406 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:48 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:48.407 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:48 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:48.407 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:48 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:48.407 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:48 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-13T03:05:48.407 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:48 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-13T03:05:48.407 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:48 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-13T03:05:48.408 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:48 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:48.408 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:48 smithi064 ceph-mon[97675]: Deploying daemon mgr.y on smithi142 2023-09-13T03:05:48.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:48 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:48.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:48 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:48.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:48 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:48.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:48 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-13T03:05:48.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:48 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-13T03:05:48.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:48 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-13T03:05:48.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:48 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:48.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:48 smithi142 ceph-mon[134082]: Deploying daemon mgr.y on smithi142 2023-09-13T03:05:48.659 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:05:48 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:05:48.384+0000 7f4ff0965200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-13T03:05:48.995 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:05:48 smithi142 systemd[1]: Starting Ceph mgr.y for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:05:49.159 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:05:48 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:05:48.801+0000 7f4ff0965200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-13T03:05:49.308 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:49 smithi142 ceph-mon[134082]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:49.309 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:05:49 smithi142 podman[135553]: 2023-09-13T03:05:49.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:49 smithi064 ceph-mon[97675]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:49.550 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:49 smithi170 ceph-mon[134328]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:49.550 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:05:49 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:05:49.468+0000 7f4ff0965200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-13T03:05:49.608 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:05:49 smithi142 ceph-mgr[135593]: -- 172.21.15.142:0/3681197628 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e6dc7d91e0 con 0x55e6dc7aec00 2023-09-13T03:05:49.608 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:05:49 smithi142 bash[135553]: 255a7d29b9414b2f7b2d91bf11defe22c5517116ecc3a405471265e55e690288 2023-09-13T03:05:49.608 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:05:49 smithi142 systemd[1]: Started Ceph mgr.y for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da. 2023-09-13T03:05:49.609 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:05:49 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:05:49.436+0000 7fa109212200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-13T03:05:49.609 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:05:49 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:05:49.515+0000 7fa109212200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-13T03:05:49.609 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:05:49 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:05:49.607+0000 7fa109212200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-13T03:05:49.773 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-09-13T03:05:49.774 INFO:teuthology.orchestra.run.smithi064.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-09-13T03:05:49.774 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10+0 records in 2023-09-13T03:05:49.774 INFO:teuthology.orchestra.run.smithi064.stderr:10+0 records out 2023-09-13T03:05:49.774 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.036862 s, 284 MB/s 2023-09-13T03:05:49.775 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping successful for: 2023-09-13T03:05:49.888 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:05:49 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:05:49.707+0000 7fa109212200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-13T03:05:49.888 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:05:49 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:05:49.886+0000 7fa109212200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-13T03:05:49.910 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:05:49 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:05:49.549+0000 7f4ff0965200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-13T03:05:50.182 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi064:vg_nvme/lv_4 2023-09-13T03:05:50.617 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:05:50 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:05:50.284+0000 7fa109212200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-13T03:05:50.617 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:05:50 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:05:50.427+0000 7fa109212200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-13T03:05:50.618 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:05:50 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:05:50.516+0000 7fa109212200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-13T03:05:50.618 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:50 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:50.618 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:50 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:50.619 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:50 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:50.619 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:50 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:50.619 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:50 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:05:50.651 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:05:50 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:05:50.377+0000 7f4ff0965200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-13T03:05:50.651 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:05:50 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:05:50.473+0000 7f4ff0965200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-13T03:05:50.652 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:50 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:50.652 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:50 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:50.652 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:50 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:50.652 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:50 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:50.652 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:50 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:05:50.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:50 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:50.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:50 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:50.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:50 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:50.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:50 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:50.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:50 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:05:50.917 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:05:51.330 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:05:50 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:05:50.650+0000 7f4ff0965200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-13T03:05:51.330 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:05:50 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:05:50.945+0000 7f4ff0965200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-13T03:05:51.331 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:05:51 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:05:51.183+0000 7f4ff0965200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-13T03:05:51.433 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:05:50 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:05:50.616+0000 7fa109212200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-13T03:05:51.433 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:05:51 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:05:51.068+0000 7fa109212200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-13T03:05:51.592 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:05:51 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:05:51.330+0000 7f4ff0965200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-13T03:05:51.592 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:51 smithi170 ceph-mon[134328]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:51.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:51 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:51.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:51 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:51.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:51 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:51.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:51 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:51.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:51 smithi064 ceph-mon[97675]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:51.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:51 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:51.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:51 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:51.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:51 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:51.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:51 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:51.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:51 smithi142 ceph-mon[134082]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:51.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:51 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:51.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:51 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:51.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:51 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:51.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:51 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:51.909 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:05:51 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:05:51.592+0000 7f4ff0965200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-13T03:05:52.245 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:05:51 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:05:51.792+0000 7fa109212200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-13T03:05:52.245 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:05:51 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:05:51.875+0000 7fa109212200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-13T03:05:52.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:52.325+0000 7f0835a74700 1 -- 172.21.15.64:0/2661647579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0830101120 msgr2=0x7f0830103510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:52.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:52.325+0000 7f0835a74700 1 --2- 172.21.15.64:0/2661647579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0830101120 0x7f0830103510 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0820009990 tx=0x7f0820009ca0 comp rx=0 tx=0).stop 2023-09-13T03:05:52.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:52.325+0000 7f0835a74700 1 -- 172.21.15.64:0/2661647579 shutdown_connections 2023-09-13T03:05:52.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:52.325+0000 7f0835a74700 1 --2- 172.21.15.64:0/2661647579 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0830106380 0x7f08301087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:52.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:52.325+0000 7f0835a74700 1 --2- 172.21.15.64:0/2661647579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0830101120 0x7f0830103510 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:52.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:52.325+0000 7f0835a74700 1 --2- 172.21.15.64:0/2661647579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0830103a50 0x7f0830105e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:52.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:52.325+0000 7f0835a74700 1 -- 172.21.15.64:0/2661647579 >> 172.21.15.64:0/2661647579 conn(0x7f08300fab10 msgr2=0x7f08300fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:05:52.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:52.325+0000 7f0835a74700 1 -- 172.21.15.64:0/2661647579 shutdown_connections 2023-09-13T03:05:52.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:52.325+0000 7f0835a74700 1 -- 172.21.15.64:0/2661647579 wait complete. 2023-09-13T03:05:52.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:52.326+0000 7f0835a74700 1 Processor -- start 2023-09-13T03:05:52.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:52.327+0000 7f0835a74700 1 -- start start 2023-09-13T03:05:52.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:52.327+0000 7f0835a74700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0830101120 0x7f08301963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:52.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:52.327+0000 7f0835a74700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0830103a50 0x7f08301968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:52.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:52.327+0000 7f0835a74700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0830106380 0x7f083019d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:52.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:52.327+0000 7f0835a74700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083019df40 con 0x7f0830103a50 2023-09-13T03:05:52.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:52.327+0000 7f0835a74700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083019e080 con 0x7f0830106380 2023-09-13T03:05:52.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:52.327+0000 7f0835a74700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083019e210 con 0x7f0830101120 2023-09-13T03:05:52.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:52.327+0000 7f082f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0830101120 0x7f08301963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:52.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:52.328+0000 7f082effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0830103a50 0x7f08301968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:52.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:52.328+0000 7f082f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0830101120 0x7f08301963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:38670/0 (socket says 172.21.15.64:38670) 2023-09-13T03:05:52.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:52.328+0000 7f082f7fe700 1 -- 172.21.15.64:0/452448821 learned_addr learned my addr 172.21.15.64:0/452448821 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:05:52.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:52.328+0000 7f082ffff700 1 --2- 172.21.15.64:0/452448821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0830106380 0x7f083019d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:52.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:52.328+0000 7f082f7fe700 1 -- 172.21.15.64:0/452448821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0830106380 msgr2=0x7f083019d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:52.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:52.328+0000 7f082f7fe700 1 --2- 172.21.15.64:0/452448821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0830106380 0x7f083019d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:52.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:52.328+0000 7f082f7fe700 1 -- 172.21.15.64:0/452448821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0830103a50 msgr2=0x7f08301968f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:05:52.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:52.328+0000 7f082f7fe700 1 --2- 172.21.15.64:0/452448821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0830103a50 0x7f08301968f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:05:52.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:52.328+0000 7f082f7fe700 1 -- 172.21.15.64:0/452448821 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0828009c40 con 0x7f0830101120 2023-09-13T03:05:52.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:52.328+0000 7f082f7fe700 1 --2- 172.21.15.64:0/452448821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0830101120 0x7f08301963b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0820005910 tx=0x7f0820005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:05:52.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:52.328+0000 7f082cff9700 1 -- 172.21.15.64:0/452448821 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082001d070 con 0x7f0830101120 2023-09-13T03:05:52.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:52.328+0000 7f0835a74700 1 -- 172.21.15.64:0/452448821 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08200095f0 con 0x7f0830101120 2023-09-13T03:05:52.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:52.328+0000 7f082cff9700 1 -- 172.21.15.64:0/452448821 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0820022470 con 0x7f0830101120 2023-09-13T03:05:52.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:52.328+0000 7f082cff9700 1 -- 172.21.15.64:0/452448821 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0820017650 con 0x7f0830101120 2023-09-13T03:05:52.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:52.329+0000 7f0835a74700 1 -- 172.21.15.64:0/452448821 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f083019e720 con 0x7f0830101120 2023-09-13T03:05:52.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:52.330+0000 7f082cff9700 1 -- 172.21.15.64:0/452448821 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7f082000fad0 con 0x7f0830101120 2023-09-13T03:05:52.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:52.330+0000 7f0835a74700 1 -- 172.21.15.64:0/452448821 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08300fc690 con 0x7f0830101120 2023-09-13T03:05:52.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:52.330+0000 7f082cff9700 1 --2- 172.21.15.64:0/452448821 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f0818038630 0x7f081803aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:05:52.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:52.330+0000 7f082cff9700 1 -- 172.21.15.64:0/452448821 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f082004c5f0 con 0x7f0830101120 2023-09-13T03:05:52.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:52.330+0000 7f082effd700 1 --2- 172.21.15.64:0/452448821 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f0818038630 0x7f081803aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:05:52.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:52.331+0000 7f082effd700 1 --2- 172.21.15.64:0/452448821 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f0818038630 0x7f081803aaf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0824009c60 tx=0x7f0824009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:05:52.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:52.335+0000 7f082cff9700 1 -- 172.21.15.64:0/452448821 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f08200177b0 con 0x7f0830101120 2023-09-13T03:05:52.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:52.563+0000 7f0835a74700 1 -- 172.21.15.64:0/452448821 --> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f083002cf50 con 0x7f0818038630 2023-09-13T03:05:52.612 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:05:52 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:05:52.291+0000 7f4ff0965200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-13T03:05:52.612 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:05:52 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:05:52.422+0000 7f4ff0965200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-13T03:05:52.613 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:52 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:52.613 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:52 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:05:52.613 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:52 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:52.613 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:52 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-13T03:05:52.614 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:52 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-13T03:05:52.614 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:52 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:52.614 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:52 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:52.722 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:52 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:52.722 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:52 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:05:52.723 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:52 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:52.723 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:52 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-13T03:05:52.724 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:52 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-13T03:05:52.724 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:52 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:52.724 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:52 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:52.737 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:05:52 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:05:52.646+0000 7fa109212200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-13T03:05:52.738 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:52 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:52.738 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:52 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:05:52.739 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:52 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:52.739 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:52 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-13T03:05:52.739 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:52 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-13T03:05:52.740 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:52 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:52.740 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:52 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:52.910 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:05:52 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:05:52.612+0000 7f4ff0965200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-13T03:05:52.910 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:05:52 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:05:52.725+0000 7f4ff0965200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-13T03:05:52.995 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:05:52 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:05:52.736+0000 7fa109212200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-13T03:05:52.995 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:05:52 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:05:52.908+0000 7fa109212200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-13T03:05:53.409 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:05:52 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:05:52.940+0000 7f4ff0965200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-13T03:05:53.437 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:05:53 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:05:53.186+0000 7fa109212200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-13T03:05:53.438 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:05:53 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:05:53.413+0000 7fa109212200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-13T03:05:53.745 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:05:53 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:05:53.562+0000 7fa109212200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-13T03:05:53.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:53 smithi142 ceph-mon[134082]: Reconfiguring mgr.x (unknown last config time)... 2023-09-13T03:05:53.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:53 smithi142 ceph-mon[134082]: Reconfiguring daemon mgr.x on smithi064 2023-09-13T03:05:53.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:53 smithi142 ceph-mon[134082]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:53.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:53 smithi142 ceph-mon[134082]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:05:53.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:53 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T03:05:53.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:53 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T03:05:53.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:53 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:53.763 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:53 smithi064 ceph-mon[97675]: Reconfiguring mgr.x (unknown last config time)... 2023-09-13T03:05:53.763 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:53 smithi064 ceph-mon[97675]: Reconfiguring daemon mgr.x on smithi064 2023-09-13T03:05:53.763 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:53 smithi064 ceph-mon[97675]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:53.764 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:53 smithi064 ceph-mon[97675]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:05:53.764 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:53 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T03:05:53.764 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:53 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T03:05:53.764 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:53 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:53.830 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:53 smithi170 ceph-mon[134328]: Reconfiguring mgr.x (unknown last config time)... 2023-09-13T03:05:53.830 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:53 smithi170 ceph-mon[134328]: Reconfiguring daemon mgr.x on smithi064 2023-09-13T03:05:53.831 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:53 smithi170 ceph-mon[134328]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:53.831 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:53 smithi170 ceph-mon[134328]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:05:53.831 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:53 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T03:05:53.831 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:53 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T03:05:53.832 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:53 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:54.160 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:05:53 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:05:53.830+0000 7f4ff0965200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-13T03:05:54.245 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:05:53 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:05:53.822+0000 7fa109212200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-13T03:05:54.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:54.457+0000 7f082cff9700 1 -- 172.21.15.64:0/452448821 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mgrmap(e 14) v1 ==== 90295+0+0 (secure 0 0 0) 0x7f0820022850 con 0x7f0830101120 2023-09-13T03:05:54.661 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:54 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:54.661 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:54 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:54.662 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:54 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:05:54.662 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:54 smithi064 ceph-mon[97675]: Standby manager daemon z started 2023-09-13T03:05:54.662 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:54 smithi064 ceph-mon[97675]: from='mgr.? 172.21.15.170:0/2094586277' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-13T03:05:54.662 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:54 smithi064 ceph-mon[97675]: from='mgr.? 172.21.15.170:0/2094586277' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-13T03:05:54.663 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:54 smithi064 ceph-mon[97675]: from='mgr.? 172.21.15.170:0/2094586277' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-13T03:05:54.663 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:54 smithi064 ceph-mon[97675]: from='mgr.? 172.21.15.170:0/2094586277' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-13T03:05:54.663 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:54 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:54.663 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:54 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:05:54.663 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:54 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:54.745 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:05:54 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:05:54.529+0000 7fa109212200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-13T03:05:54.745 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:05:54 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:05:54.659+0000 7fa109212200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-13T03:05:54.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:54 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:54.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:54 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:54.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:54 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:05:54.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:54 smithi142 ceph-mon[134082]: Standby manager daemon z started 2023-09-13T03:05:54.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:54 smithi142 ceph-mon[134082]: from='mgr.? 172.21.15.170:0/2094586277' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-13T03:05:54.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:54 smithi142 ceph-mon[134082]: from='mgr.? 172.21.15.170:0/2094586277' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-13T03:05:54.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:54 smithi142 ceph-mon[134082]: from='mgr.? 172.21.15.170:0/2094586277' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-13T03:05:54.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:54 smithi142 ceph-mon[134082]: from='mgr.? 172.21.15.170:0/2094586277' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-13T03:05:54.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:54 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:54.748 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:54 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:05:54.748 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:54 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:54.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:54 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:54.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:54 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:54.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:54 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:05:54.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:54 smithi170 ceph-mon[134328]: Standby manager daemon z started 2023-09-13T03:05:54.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:54 smithi170 ceph-mon[134328]: from='mgr.? 172.21.15.170:0/2094586277' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-13T03:05:54.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:54 smithi170 ceph-mon[134328]: from='mgr.? 172.21.15.170:0/2094586277' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-13T03:05:54.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:54 smithi170 ceph-mon[134328]: from='mgr.? 172.21.15.170:0/2094586277' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-13T03:05:54.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:54 smithi170 ceph-mon[134328]: from='mgr.? 172.21.15.170:0/2094586277' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-13T03:05:54.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:54 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:05:54.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:54 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:05:54.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:54 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:05:55.182 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:05:54 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:05:54.851+0000 7fa109212200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-13T03:05:55.183 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:05:54 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:05:54.963+0000 7fa109212200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-13T03:05:55.459 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:05:55 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:05:55.181+0000 7fa109212200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-13T03:05:55.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:55 smithi142 ceph-mon[134082]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:55.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:55 smithi142 ceph-mon[134082]: mgrmap e14: x(active, since 92s), standbys: z 2023-09-13T03:05:55.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:55 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-13T03:05:55.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:55 smithi170 ceph-mon[134328]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:55.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:55 smithi170 ceph-mon[134328]: mgrmap e14: x(active, since 92s), standbys: z 2023-09-13T03:05:55.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:55 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-13T03:05:55.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:55 smithi064 ceph-mon[97675]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:55.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:55 smithi064 ceph-mon[97675]: mgrmap e14: x(active, since 92s), standbys: z 2023-09-13T03:05:55.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:55 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-13T03:05:56.460 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:05:56 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:05:56.041+0000 7fa109212200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-13T03:05:56.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:56 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/420058793' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3997c114-241b-48c4-8feb-653c99209470"}]: dispatch 2023-09-13T03:05:56.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:56 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/420058793' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3997c114-241b-48c4-8feb-653c99209470"}]': finished 2023-09-13T03:05:56.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:56 smithi142 ceph-mon[134082]: osdmap e5: 1 total, 0 up, 1 in 2023-09-13T03:05:56.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:56 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-13T03:05:56.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:56 smithi142 ceph-mon[134082]: Standby manager daemon y started 2023-09-13T03:05:56.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:56 smithi142 ceph-mon[134082]: from='mgr.? 172.21.15.142:0/3128612081' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-13T03:05:56.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:56 smithi142 ceph-mon[134082]: from='mgr.? 172.21.15.142:0/3128612081' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-13T03:05:56.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:56 smithi142 ceph-mon[134082]: from='mgr.? 172.21.15.142:0/3128612081' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-13T03:05:56.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:56 smithi142 ceph-mon[134082]: from='mgr.? 172.21.15.142:0/3128612081' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-13T03:05:56.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:56 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/3197914089' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T03:05:56.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:05:56.800+0000 7f082cff9700 1 -- 172.21.15.64:0/452448821 <== mon.1 v2:172.21.15.170:3300/0 8 ==== mgrmap(e 15) v1 ==== 135203+0+0 (secure 0 0 0) 0x7f0820016720 con 0x7f0830101120 2023-09-13T03:05:56.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:56 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/420058793' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3997c114-241b-48c4-8feb-653c99209470"}]: dispatch 2023-09-13T03:05:56.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:56 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/420058793' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3997c114-241b-48c4-8feb-653c99209470"}]': finished 2023-09-13T03:05:56.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:56 smithi170 ceph-mon[134328]: osdmap e5: 1 total, 0 up, 1 in 2023-09-13T03:05:56.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:56 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-13T03:05:56.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:56 smithi170 ceph-mon[134328]: Standby manager daemon y started 2023-09-13T03:05:56.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:56 smithi170 ceph-mon[134328]: from='mgr.? 172.21.15.142:0/3128612081' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-13T03:05:56.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:56 smithi170 ceph-mon[134328]: from='mgr.? 172.21.15.142:0/3128612081' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-13T03:05:56.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:56 smithi170 ceph-mon[134328]: from='mgr.? 172.21.15.142:0/3128612081' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-13T03:05:56.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:56 smithi170 ceph-mon[134328]: from='mgr.? 172.21.15.142:0/3128612081' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-13T03:05:56.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:56 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/3197914089' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T03:05:56.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:56 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/420058793' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3997c114-241b-48c4-8feb-653c99209470"}]: dispatch 2023-09-13T03:05:56.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:56 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/420058793' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3997c114-241b-48c4-8feb-653c99209470"}]': finished 2023-09-13T03:05:56.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:56 smithi064 ceph-mon[97675]: osdmap e5: 1 total, 0 up, 1 in 2023-09-13T03:05:56.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:56 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-13T03:05:56.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:56 smithi064 ceph-mon[97675]: Standby manager daemon y started 2023-09-13T03:05:56.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:56 smithi064 ceph-mon[97675]: from='mgr.? 172.21.15.142:0/3128612081' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-13T03:05:56.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:56 smithi064 ceph-mon[97675]: from='mgr.? 172.21.15.142:0/3128612081' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-13T03:05:56.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:56 smithi064 ceph-mon[97675]: from='mgr.? 172.21.15.142:0/3128612081' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-13T03:05:56.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:56 smithi064 ceph-mon[97675]: from='mgr.? 172.21.15.142:0/3128612081' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-13T03:05:56.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:56 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/3197914089' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T03:05:57.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:57 smithi142 ceph-mon[134082]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:57.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:57 smithi142 ceph-mon[134082]: mgrmap e15: x(active, since 94s), standbys: z, y 2023-09-13T03:05:57.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:57 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-13T03:05:57.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:57 smithi170 ceph-mon[134328]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:57.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:57 smithi170 ceph-mon[134328]: mgrmap e15: x(active, since 94s), standbys: z, y 2023-09-13T03:05:57.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:57 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-13T03:05:57.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:57 smithi064 ceph-mon[97675]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:57.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:57 smithi064 ceph-mon[97675]: mgrmap e15: x(active, since 94s), standbys: z, y 2023-09-13T03:05:57.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:57 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-13T03:05:59.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:05:59 smithi142 ceph-mon[134082]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:59.892 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:05:59 smithi064 ceph-mon[97675]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:05:59.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:05:59 smithi170 ceph-mon[134328]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:06:01.645 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:01 smithi064 ceph-mon[97675]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:06:01.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:01 smithi142 ceph-mon[134082]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:06:01.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:01 smithi170 ceph-mon[134328]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:06:03.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:03 smithi142 ceph-mon[134082]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:06:03.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:03 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-09-13T03:06:03.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:03 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:06:03.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:03 smithi170 ceph-mon[134328]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:06:03.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:03 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-09-13T03:06:03.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:03 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:06:03.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:03 smithi064 ceph-mon[97675]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:06:03.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:03 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-09-13T03:06:03.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:03 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:06:04.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:04 smithi142 ceph-mon[134082]: Deploying daemon osd.0 on smithi064 2023-09-13T03:06:04.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:04 smithi170 ceph-mon[134328]: Deploying daemon osd.0 on smithi064 2023-09-13T03:06:04.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:04 smithi064 ceph-mon[97675]: Deploying daemon osd.0 on smithi064 2023-09-13T03:06:05.736 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:05 smithi064 ceph-mon[97675]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:06:05.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:05 smithi142 ceph-mon[134082]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:06:05.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:05 smithi170 ceph-mon[134328]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:06:07.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:07 smithi064 ceph-mon[97675]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:06:07.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:07 smithi142 ceph-mon[134082]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:06:07.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:07 smithi170 ceph-mon[134328]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:06:09.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:09 smithi142 ceph-mon[134082]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:06:09.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:09 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:09.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:09 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:09.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:09 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:06:09.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:09 smithi170 ceph-mon[134328]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:06:09.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:09 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:09.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:09 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:09.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:09 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:06:09.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:09 smithi064 ceph-mon[97675]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:06:09.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:09 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:09.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:09 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:09.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:09 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:06:11.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:11 smithi142 ceph-mon[134082]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:06:11.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:11 smithi170 ceph-mon[134328]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:06:11.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:11 smithi064 ceph-mon[97675]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:06:12.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:12.724+0000 7f082cff9700 1 -- 172.21.15.64:0/452448821 <== mgr.14150 v2:172.21.15.64:6800/873679308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f083002cf50 con 0x7f0818038630 2023-09-13T03:06:12.725 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 0 on host 'smithi064' 2023-09-13T03:06:12.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:12.726+0000 7f0835a74700 1 -- 172.21.15.64:0/452448821 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f0818038630 msgr2=0x7f081803aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:06:12.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:12.726+0000 7f0835a74700 1 --2- 172.21.15.64:0/452448821 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f0818038630 0x7f081803aaf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0824009c60 tx=0x7f0824009260 comp rx=0 tx=0).stop 2023-09-13T03:06:12.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:12.726+0000 7f0835a74700 1 -- 172.21.15.64:0/452448821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0830101120 msgr2=0x7f08301963b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:06:12.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:12.726+0000 7f0835a74700 1 --2- 172.21.15.64:0/452448821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0830101120 0x7f08301963b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0820005910 tx=0x7f0820005740 comp rx=0 tx=0).stop 2023-09-13T03:06:12.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:12.726+0000 7f0835a74700 1 -- 172.21.15.64:0/452448821 shutdown_connections 2023-09-13T03:06:12.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:12.726+0000 7f0835a74700 1 --2- 172.21.15.64:0/452448821 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f0818038630 0x7f081803aaf0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:06:12.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:12.726+0000 7f0835a74700 1 --2- 172.21.15.64:0/452448821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0830106380 0x7f083019d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:06:12.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:12.726+0000 7f0835a74700 1 --2- 172.21.15.64:0/452448821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0830101120 0x7f08301963b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:06:12.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:12.726+0000 7f0835a74700 1 --2- 172.21.15.64:0/452448821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0830103a50 0x7f08301968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:06:12.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:12.726+0000 7f0835a74700 1 -- 172.21.15.64:0/452448821 >> 172.21.15.64:0/452448821 conn(0x7f08300fab10 msgr2=0x7f0830101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:06:12.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:12.726+0000 7f0835a74700 1 -- 172.21.15.64:0/452448821 shutdown_connections 2023-09-13T03:06:12.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:12.726+0000 7f0835a74700 1 -- 172.21.15.64:0/452448821 wait complete. 2023-09-13T03:06:12.836 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:12 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:12.836 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:12 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:12.836 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:12 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:06:12.837 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:12 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:06:12.837 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:12 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:12.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:12 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:12.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:12 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:12.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:12 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:06:12.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:12 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:06:12.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:12 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:12.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:12 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:12.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:12 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:12.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:12 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:06:12.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:12 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:06:12.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:12 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:13.144 DEBUG:teuthology.orchestra.run.smithi064:osd.0> sudo journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@osd.0.service 2023-09-13T03:06:13.149 INFO:tasks.cephadm:Deploying osd.1 on smithi142 with /dev/vg_nvme/lv_4... 2023-09-13T03:06:13.149 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-09-13T03:06:13.233 INFO:journalctl@ceph.osd.0.smithi064.stdout:-- Logs begin at Wed 2023-09-13 02:46:56 UTC. -- 2023-09-13T03:06:13.418 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 13 03:06:13 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-0[120490]: 2023-09-13T03:06:13.232+0000 7f43f805e640 -1 osd.0 0 log_to_monitors true 2023-09-13T03:06:13.716 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-13T03:06:13.850 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:13 smithi064 ceph-mon[97675]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:06:13.850 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:13 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:13.850 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:13 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:13.851 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:13 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:06:13.851 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:13 smithi064 ceph-mon[97675]: from='osd.0 [v2:172.21.15.64:6802/3940892135,v1:172.21.15.64:6803/3940892135]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-09-13T03:06:13.851 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:13 smithi064 ceph-mon[97675]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-09-13T03:06:13.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:13 smithi170 ceph-mon[134328]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:06:13.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:13 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:13.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:13 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:13.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:13 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:06:13.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:13 smithi170 ceph-mon[134328]: from='osd.0 [v2:172.21.15.64:6802/3940892135,v1:172.21.15.64:6803/3940892135]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-09-13T03:06:13.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:13 smithi170 ceph-mon[134328]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-09-13T03:06:13.962 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:13 smithi142 ceph-mon[134082]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:06:13.962 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:13 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:13.962 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:13 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:13.963 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:13 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:06:13.963 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:13 smithi142 ceph-mon[134082]: from='osd.0 [v2:172.21.15.64:6802/3940892135,v1:172.21.15.64:6803/3940892135]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-09-13T03:06:13.963 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:13 smithi142 ceph-mon[134082]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-09-13T03:06:14.862 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:14 smithi142 ceph-mon[134082]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-09-13T03:06:14.862 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:14 smithi142 ceph-mon[134082]: osdmap e6: 1 total, 0 up, 1 in 2023-09-13T03:06:14.863 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:14 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-13T03:06:14.863 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:14 smithi142 ceph-mon[134082]: from='osd.0 [v2:172.21.15.64:6802/3940892135,v1:172.21.15.64:6803/3940892135]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-09-13T03:06:14.863 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:14 smithi142 ceph-mon[134082]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-09-13T03:06:14.863 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:14 smithi142 ceph-mon[134082]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:06:15.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:14 smithi170 ceph-mon[134328]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-09-13T03:06:15.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:14 smithi170 ceph-mon[134328]: osdmap e6: 1 total, 0 up, 1 in 2023-09-13T03:06:15.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:14 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-13T03:06:15.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:14 smithi170 ceph-mon[134328]: from='osd.0 [v2:172.21.15.64:6802/3940892135,v1:172.21.15.64:6803/3940892135]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-09-13T03:06:15.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:14 smithi170 ceph-mon[134328]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-09-13T03:06:15.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:14 smithi170 ceph-mon[134328]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:06:15.168 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 13 03:06:14 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-0[120490]: 2023-09-13T03:06:14.685+0000 7f43eced4700 -1 osd.0 0 waiting for initial osdmap 2023-09-13T03:06:15.169 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 13 03:06:14 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-0[120490]: 2023-09-13T03:06:14.689+0000 7f43e8cc9700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-13T03:06:15.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:14 smithi064 ceph-mon[97675]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-09-13T03:06:15.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:14 smithi064 ceph-mon[97675]: osdmap e6: 1 total, 0 up, 1 in 2023-09-13T03:06:15.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:14 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-13T03:06:15.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:14 smithi064 ceph-mon[97675]: from='osd.0 [v2:172.21.15.64:6802/3940892135,v1:172.21.15.64:6803/3940892135]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-09-13T03:06:15.170 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:14 smithi064 ceph-mon[97675]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-09-13T03:06:15.170 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:14 smithi064 ceph-mon[97675]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:06:15.367 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-09-13T03:06:15.367 INFO:teuthology.orchestra.run.smithi142.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-09-13T03:06:15.367 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10+0 records in 2023-09-13T03:06:15.368 INFO:teuthology.orchestra.run.smithi142.stderr:10+0 records out 2023-09-13T03:06:15.368 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0109896 s, 954 MB/s 2023-09-13T03:06:15.368 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping successful for: 2023-09-13T03:06:15.757 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi142:vg_nvme/lv_4 2023-09-13T03:06:15.944 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:15 smithi064 ceph-mon[97675]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2023-09-13T03:06:15.944 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:15 smithi064 ceph-mon[97675]: osdmap e7: 1 total, 0 up, 1 in 2023-09-13T03:06:15.944 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:15 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-13T03:06:15.945 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:15 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-13T03:06:15.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:15 smithi142 ceph-mon[134082]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2023-09-13T03:06:15.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:15 smithi142 ceph-mon[134082]: osdmap e7: 1 total, 0 up, 1 in 2023-09-13T03:06:15.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:15 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-13T03:06:15.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:15 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-13T03:06:16.159 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:15 smithi170 ceph-mon[134328]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2023-09-13T03:06:16.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:15 smithi170 ceph-mon[134328]: osdmap e7: 1 total, 0 up, 1 in 2023-09-13T03:06:16.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:15 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-13T03:06:16.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:15 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-13T03:06:16.309 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-13T03:06:16.813 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:16 smithi142 ceph-mon[134082]: purged_snaps scrub starts 2023-09-13T03:06:16.813 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:16 smithi142 ceph-mon[134082]: purged_snaps scrub ok 2023-09-13T03:06:16.814 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:16 smithi142 ceph-mon[134082]: osd.0 [v2:172.21.15.64:6802/3940892135,v1:172.21.15.64:6803/3940892135] boot 2023-09-13T03:06:16.814 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:16 smithi142 ceph-mon[134082]: osdmap e8: 1 total, 1 up, 1 in 2023-09-13T03:06:16.814 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:16 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-13T03:06:16.814 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:16 smithi142 ceph-mon[134082]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:06:16.814 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:16 smithi142 ceph-mon[134082]: Detected new or changed devices on smithi064 2023-09-13T03:06:16.815 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:16 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:16.815 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:16 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:16.815 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:16 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:06:16.815 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:16 smithi142 ceph-mon[134082]: Adjusting osd_memory_target on smithi064 to 17267M 2023-09-13T03:06:16.815 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:16 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:16.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:16 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:06:16.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:16 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:06:16.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:16 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:16.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:16 smithi064 ceph-mon[97675]: purged_snaps scrub starts 2023-09-13T03:06:16.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:16 smithi064 ceph-mon[97675]: purged_snaps scrub ok 2023-09-13T03:06:16.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:16 smithi064 ceph-mon[97675]: osd.0 [v2:172.21.15.64:6802/3940892135,v1:172.21.15.64:6803/3940892135] boot 2023-09-13T03:06:16.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:16 smithi064 ceph-mon[97675]: osdmap e8: 1 total, 1 up, 1 in 2023-09-13T03:06:16.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:16 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-13T03:06:16.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:16 smithi064 ceph-mon[97675]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:06:16.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:16 smithi064 ceph-mon[97675]: Detected new or changed devices on smithi064 2023-09-13T03:06:16.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:16 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:16.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:16 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:16.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:16 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:06:16.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:16 smithi064 ceph-mon[97675]: Adjusting osd_memory_target on smithi064 to 17267M 2023-09-13T03:06:16.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:16 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:16.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:16 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:06:16.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:16 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:06:16.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:16 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:17.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:16 smithi170 ceph-mon[134328]: purged_snaps scrub starts 2023-09-13T03:06:17.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:16 smithi170 ceph-mon[134328]: purged_snaps scrub ok 2023-09-13T03:06:17.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:16 smithi170 ceph-mon[134328]: osd.0 [v2:172.21.15.64:6802/3940892135,v1:172.21.15.64:6803/3940892135] boot 2023-09-13T03:06:17.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:16 smithi170 ceph-mon[134328]: osdmap e8: 1 total, 1 up, 1 in 2023-09-13T03:06:17.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:16 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-13T03:06:17.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:16 smithi170 ceph-mon[134328]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-13T03:06:17.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:16 smithi170 ceph-mon[134328]: Detected new or changed devices on smithi064 2023-09-13T03:06:17.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:16 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:17.162 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:16 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:17.162 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:16 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:06:17.162 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:16 smithi170 ceph-mon[134328]: Adjusting osd_memory_target on smithi064 to 17267M 2023-09-13T03:06:17.162 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:16 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:17.162 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:16 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:06:17.163 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:16 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:06:17.163 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:16 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:17.513 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:17.509+0000 7f57142c8700 1 -- 172.21.15.142:0/4250901503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f570c106380 msgr2=0x7f570c1087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:06:17.513 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:17.509+0000 7f57142c8700 1 --2- 172.21.15.142:0/4250901503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f570c106380 0x7f570c1087f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f570400b0a0 tx=0x7f570400b3b0 comp rx=0 tx=0).stop 2023-09-13T03:06:17.514 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:17.510+0000 7f57142c8700 1 -- 172.21.15.142:0/4250901503 shutdown_connections 2023-09-13T03:06:17.514 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:17.510+0000 7f57142c8700 1 --2- 172.21.15.142:0/4250901503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f570c106380 0x7f570c1087f0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:06:17.514 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:17.510+0000 7f57142c8700 1 --2- 172.21.15.142:0/4250901503 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f570c101120 0x7f570c103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:06:17.515 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:17.510+0000 7f57142c8700 1 --2- 172.21.15.142:0/4250901503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f570c103a50 0x7f570c105e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:06:17.515 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:17.510+0000 7f57142c8700 1 -- 172.21.15.142:0/4250901503 >> 172.21.15.142:0/4250901503 conn(0x7f570c0faaf0 msgr2=0x7f570c0fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:06:17.515 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:17.510+0000 7f57142c8700 1 -- 172.21.15.142:0/4250901503 shutdown_connections 2023-09-13T03:06:17.516 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:17.510+0000 7f57142c8700 1 -- 172.21.15.142:0/4250901503 wait complete. 2023-09-13T03:06:17.516 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:17.511+0000 7f57142c8700 1 Processor -- start 2023-09-13T03:06:17.516 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:17.511+0000 7f57142c8700 1 -- start start 2023-09-13T03:06:17.517 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:17.511+0000 7f57142c8700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f570c101120 0x7f570c100a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:06:17.517 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:17.511+0000 7f57142c8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f570c103a50 0x7f570c0ff0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:06:17.517 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:17.511+0000 7f57142c8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f570c106380 0x7f570c0ff610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:06:17.518 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:17.511+0000 7f57142c8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f570c0ffd90 con 0x7f570c106380 2023-09-13T03:06:17.518 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:17.511+0000 7f57142c8700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f570c0fff00 con 0x7f570c101120 2023-09-13T03:06:17.518 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:17.511+0000 7f57142c8700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f570c1a04f0 con 0x7f570c103a50 2023-09-13T03:06:17.519 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:17.511+0000 7f5712064700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f570c101120 0x7f570c100a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:06:17.519 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:17.511+0000 7f5712064700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f570c101120 0x7f570c100a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:40492/0 (socket says 172.21.15.142:40492) 2023-09-13T03:06:17.519 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:17.511+0000 7f5712064700 1 -- 172.21.15.142:0/3182586788 learned_addr learned my addr 172.21.15.142:0/3182586788 (peer_addr_for_me v2:172.21.15.142:0/0) 2023-09-13T03:06:17.520 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:17.511+0000 7f5711863700 1 --2- 172.21.15.142:0/3182586788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f570c103a50 0x7f570c0ff0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:06:17.520 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:17.511+0000 7f5712064700 1 -- 172.21.15.142:0/3182586788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f570c103a50 msgr2=0x7f570c0ff0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:06:17.520 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:17.511+0000 7f5712064700 1 --2- 172.21.15.142:0/3182586788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f570c103a50 0x7f570c0ff0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:06:17.521 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:17.511+0000 7f5712064700 1 -- 172.21.15.142:0/3182586788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f570c106380 msgr2=0x7f570c0ff610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:06:17.521 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:17.511+0000 7f5712064700 1 --2- 172.21.15.142:0/3182586788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f570c106380 0x7f570c0ff610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:06:17.521 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:17.511+0000 7f5712064700 1 -- 172.21.15.142:0/3182586788 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5704009d00 con 0x7f570c101120 2023-09-13T03:06:17.521 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:17.512+0000 7f5712064700 1 --2- 172.21.15.142:0/3182586788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f570c101120 0x7f570c100a80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f56fc00e9a0 tx=0x7f56fc00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:06:17.522 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:17.512+0000 7f57037fe700 1 -- 172.21.15.142:0/3182586788 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56fc009790 con 0x7f570c101120 2023-09-13T03:06:17.522 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:17.512+0000 7f57037fe700 1 -- 172.21.15.142:0/3182586788 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56fc00cbd0 con 0x7f570c101120 2023-09-13T03:06:17.522 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:17.512+0000 7f57142c8700 1 -- 172.21.15.142:0/3182586788 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f570c1a07d0 con 0x7f570c101120 2023-09-13T03:06:17.523 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:17.512+0000 7f57037fe700 1 -- 172.21.15.142:0/3182586788 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56fc018680 con 0x7f570c101120 2023-09-13T03:06:17.523 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:17.512+0000 7f57142c8700 1 -- 172.21.15.142:0/3182586788 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f570c1a0cd0 con 0x7f570c101120 2023-09-13T03:06:17.523 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:17.513+0000 7f57142c8700 1 -- 172.21.15.142:0/3182586788 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f570c0fc670 con 0x7f570c101120 2023-09-13T03:06:17.524 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:17.514+0000 7f57037fe700 1 -- 172.21.15.142:0/3182586788 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 15) v1 ==== 135203+0+0 (secure 0 0 0) 0x7f56fc00cd40 con 0x7f570c101120 2023-09-13T03:06:17.524 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:17.514+0000 7f57037fe700 1 --2- 172.21.15.142:0/3182586788 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f56f80a05b0 0x7f56f80a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:06:17.525 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:17.514+0000 7f57037fe700 1 -- 172.21.15.142:0/3182586788 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f56fc014070 con 0x7f570c101120 2023-09-13T03:06:17.525 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:17.515+0000 7f5711863700 1 --2- 172.21.15.142:0/3182586788 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f56f80a05b0 0x7f56f80a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:06:17.525 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:17.515+0000 7f5711863700 1 --2- 172.21.15.142:0/3182586788 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f56f80a05b0 0x7f56f80a2a70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5708005950 tx=0x7f57080058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:06:17.525 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:17.518+0000 7f57037fe700 1 -- 172.21.15.142:0/3182586788 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f56fc095ed0 con 0x7f570c101120 2023-09-13T03:06:17.713 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:17.713+0000 7f57142c8700 1 -- 172.21.15.142:0/3182586788 --> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f570c02d030 con 0x7f56f80a05b0 2023-09-13T03:06:17.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:17 smithi142 ceph-mon[134082]: osdmap e9: 1 total, 1 up, 1 in 2023-09-13T03:06:18.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:17 smithi170 ceph-mon[134328]: osdmap e9: 1 total, 1 up, 1 in 2023-09-13T03:06:18.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:17 smithi064 ceph-mon[97675]: osdmap e9: 1 total, 1 up, 1 in 2023-09-13T03:06:18.813 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:18 smithi142 ceph-mon[134082]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:06:18.813 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:18 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T03:06:18.813 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:18 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T03:06:18.814 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:18 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:06:18.814 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:18 smithi142 ceph-mon[134082]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T03:06:19.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:18 smithi170 ceph-mon[134328]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:06:19.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:18 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T03:06:19.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:18 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T03:06:19.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:18 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:06:19.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:18 smithi170 ceph-mon[134328]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T03:06:19.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:18 smithi064 ceph-mon[97675]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:06:19.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:18 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T03:06:19.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:18 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T03:06:19.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:18 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:06:19.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:18 smithi064 ceph-mon[97675]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T03:06:20.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:20 smithi170 ceph-mon[134328]: from='client.? 172.21.15.142:0/2287922673' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94cfae85-322e-4ed2-a6e6-e40349bb14bf"}]: dispatch 2023-09-13T03:06:20.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:20 smithi170 ceph-mon[134328]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94cfae85-322e-4ed2-a6e6-e40349bb14bf"}]: dispatch 2023-09-13T03:06:20.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:20 smithi170 ceph-mon[134328]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "94cfae85-322e-4ed2-a6e6-e40349bb14bf"}]': finished 2023-09-13T03:06:20.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:20 smithi170 ceph-mon[134328]: osdmap e10: 2 total, 1 up, 2 in 2023-09-13T03:06:20.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:20 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-13T03:06:20.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:20 smithi064 ceph-mon[97675]: from='client.? 172.21.15.142:0/2287922673' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94cfae85-322e-4ed2-a6e6-e40349bb14bf"}]: dispatch 2023-09-13T03:06:20.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:20 smithi064 ceph-mon[97675]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94cfae85-322e-4ed2-a6e6-e40349bb14bf"}]: dispatch 2023-09-13T03:06:20.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:20 smithi064 ceph-mon[97675]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "94cfae85-322e-4ed2-a6e6-e40349bb14bf"}]': finished 2023-09-13T03:06:20.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:20 smithi064 ceph-mon[97675]: osdmap e10: 2 total, 1 up, 2 in 2023-09-13T03:06:20.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:20 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-13T03:06:20.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:20 smithi142 ceph-mon[134082]: from='client.? 172.21.15.142:0/2287922673' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94cfae85-322e-4ed2-a6e6-e40349bb14bf"}]: dispatch 2023-09-13T03:06:20.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:20 smithi142 ceph-mon[134082]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94cfae85-322e-4ed2-a6e6-e40349bb14bf"}]: dispatch 2023-09-13T03:06:20.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:20 smithi142 ceph-mon[134082]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "94cfae85-322e-4ed2-a6e6-e40349bb14bf"}]': finished 2023-09-13T03:06:20.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:20 smithi142 ceph-mon[134082]: osdmap e10: 2 total, 1 up, 2 in 2023-09-13T03:06:20.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:20 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-13T03:06:21.409 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:21 smithi170 ceph-mon[134328]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T03:06:21.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:21 smithi170 ceph-mon[134328]: from='client.? 172.21.15.142:0/3757607543' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T03:06:21.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:21 smithi064 ceph-mon[97675]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T03:06:21.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:21 smithi064 ceph-mon[97675]: from='client.? 172.21.15.142:0/3757607543' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T03:06:21.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:21 smithi142 ceph-mon[134082]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T03:06:21.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:21 smithi142 ceph-mon[134082]: from='client.? 172.21.15.142:0/3757607543' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T03:06:23.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:23 smithi142 ceph-mon[134082]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T03:06:23.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:23 smithi170 ceph-mon[134328]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T03:06:23.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:23 smithi064 ceph-mon[97675]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T03:06:25.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:25 smithi170 ceph-mon[134328]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T03:06:25.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:25 smithi064 ceph-mon[97675]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T03:06:25.424 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:25 smithi142 ceph-mon[134082]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T03:06:27.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:27 smithi142 ceph-mon[134082]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T03:06:27.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:27 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-09-13T03:06:27.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:27 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:06:27.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:27 smithi170 ceph-mon[134328]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T03:06:27.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:27 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-09-13T03:06:27.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:27 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:06:27.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:27 smithi064 ceph-mon[97675]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T03:06:27.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:27 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-09-13T03:06:27.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:27 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:06:28.713 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:28 smithi142 ceph-mon[134082]: Deploying daemon osd.1 on smithi142 2023-09-13T03:06:28.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:28 smithi170 ceph-mon[134328]: Deploying daemon osd.1 on smithi142 2023-09-13T03:06:28.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:28 smithi064 ceph-mon[97675]: Deploying daemon osd.1 on smithi142 2023-09-13T03:06:29.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:29 smithi142 ceph-mon[134082]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T03:06:29.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:29 smithi170 ceph-mon[134328]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T03:06:29.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:29 smithi064 ceph-mon[97675]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T03:06:31.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:31 smithi142 ceph-mon[134082]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T03:06:31.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:31 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:31.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:31 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:31.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:31 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:06:31.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:31 smithi170 ceph-mon[134328]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T03:06:31.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:31 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:31.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:31 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:31.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:31 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:06:31.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:31 smithi064 ceph-mon[97675]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T03:06:31.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:31 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:31.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:31 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:31.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:31 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:06:33.724 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:33 smithi142 ceph-mon[134082]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T03:06:33.724 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:33 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:33.724 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:33 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:33.725 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:33 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:06:33.725 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:33 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:06:33.725 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:33 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:33.776 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:33.775+0000 7f57037fe700 1 -- 172.21.15.142:0/3182586788 <== mgr.14150 v2:172.21.15.64:6800/873679308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f570c02d030 con 0x7f56f80a05b0 2023-09-13T03:06:33.776 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 1 on host 'smithi142' 2023-09-13T03:06:33.778 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:33.777+0000 7f57142c8700 1 -- 172.21.15.142:0/3182586788 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f56f80a05b0 msgr2=0x7f56f80a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:06:33.778 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:33.777+0000 7f57142c8700 1 --2- 172.21.15.142:0/3182586788 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f56f80a05b0 0x7f56f80a2a70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5708005950 tx=0x7f57080058e0 comp rx=0 tx=0).stop 2023-09-13T03:06:33.779 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:33.777+0000 7f57142c8700 1 -- 172.21.15.142:0/3182586788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f570c101120 msgr2=0x7f570c100a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:06:33.779 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:33.777+0000 7f57142c8700 1 --2- 172.21.15.142:0/3182586788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f570c101120 0x7f570c100a80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f56fc00e9a0 tx=0x7f56fc00ed60 comp rx=0 tx=0).stop 2023-09-13T03:06:33.779 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:33.777+0000 7f57142c8700 1 -- 172.21.15.142:0/3182586788 shutdown_connections 2023-09-13T03:06:33.779 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:33.777+0000 7f57142c8700 1 --2- 172.21.15.142:0/3182586788 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f56f80a05b0 0x7f56f80a2a70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:06:33.779 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:33.777+0000 7f57142c8700 1 --2- 172.21.15.142:0/3182586788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f570c101120 0x7f570c100a80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:06:33.780 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:33.777+0000 7f57142c8700 1 --2- 172.21.15.142:0/3182586788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f570c106380 0x7f570c0ff610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:06:33.780 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:33.777+0000 7f57142c8700 1 --2- 172.21.15.142:0/3182586788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f570c103a50 0x7f570c0ff0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:06:33.780 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:33.777+0000 7f57142c8700 1 -- 172.21.15.142:0/3182586788 >> 172.21.15.142:0/3182586788 conn(0x7f570c0faaf0 msgr2=0x7f570c101d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:06:33.780 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:33.777+0000 7f57142c8700 1 -- 172.21.15.142:0/3182586788 shutdown_connections 2023-09-13T03:06:33.780 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:06:33.777+0000 7f57142c8700 1 -- 172.21.15.142:0/3182586788 wait complete. 2023-09-13T03:06:33.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:33 smithi170 ceph-mon[134328]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T03:06:33.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:33 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:33.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:33 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:33.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:33 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:06:33.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:33 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:06:33.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:33 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:33.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:33 smithi064 ceph-mon[97675]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T03:06:33.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:33 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:33.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:33 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:33.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:33 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:06:33.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:33 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:06:33.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:33 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:34.184 DEBUG:teuthology.orchestra.run.smithi142:osd.1> sudo journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@osd.1.service 2023-09-13T03:06:34.186 INFO:tasks.cephadm:Deploying osd.2 on smithi170 with /dev/vg_nvme/lv_4... 2023-09-13T03:06:34.187 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-09-13T03:06:34.217 INFO:journalctl@ceph.osd.1.smithi142.stdout:-- Logs begin at Wed 2023-09-13 02:43:02 UTC. -- 2023-09-13T03:06:34.755 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-13T03:06:34.978 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:34 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:34.979 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:34 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:34.979 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:34 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:06:34.979 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:34 smithi170 ceph-mon[134328]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T03:06:34.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:34 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:34.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:34 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:34.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:34 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:06:34.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:34 smithi142 ceph-mon[134082]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T03:06:35.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:34 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:35.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:34 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:35.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:34 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:06:35.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:34 smithi064 ceph-mon[97675]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T03:06:35.495 INFO:journalctl@ceph.osd.1.smithi142.stdout:Sep 13 03:06:35 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-1[139226]: 2023-09-13T03:06:35.128+0000 7f7d4a9ff640 -1 osd.1 0 log_to_monitors true 2023-09-13T03:06:36.246 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-09-13T03:06:36.247 INFO:teuthology.orchestra.run.smithi170.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-09-13T03:06:36.247 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10+0 records in 2023-09-13T03:06:36.247 INFO:teuthology.orchestra.run.smithi170.stderr:10+0 records out 2023-09-13T03:06:36.247 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00829135 s, 1.3 GB/s 2023-09-13T03:06:36.247 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping successful for: 2023-09-13T03:06:36.374 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:36 smithi170 ceph-mon[134328]: from='osd.1 [v2:172.21.15.142:6800/4188149333,v1:172.21.15.142:6801/4188149333]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-13T03:06:36.374 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:36 smithi170 ceph-mon[134328]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-13T03:06:36.415 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:36 smithi142 ceph-mon[134082]: from='osd.1 [v2:172.21.15.142:6800/4188149333,v1:172.21.15.142:6801/4188149333]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-13T03:06:36.416 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:36 smithi142 ceph-mon[134082]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-13T03:06:36.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:36 smithi064 ceph-mon[97675]: from='osd.1 [v2:172.21.15.142:6800/4188149333,v1:172.21.15.142:6801/4188149333]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-13T03:06:36.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:36 smithi064 ceph-mon[97675]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-13T03:06:36.609 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi170:vg_nvme/lv_4 2023-09-13T03:06:37.161 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-13T03:06:37.388 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:37 smithi170 ceph-mon[134328]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-09-13T03:06:37.388 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:37 smithi170 ceph-mon[134328]: osdmap e11: 2 total, 1 up, 2 in 2023-09-13T03:06:37.388 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:37 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-13T03:06:37.389 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:37 smithi170 ceph-mon[134328]: from='osd.1 [v2:172.21.15.142:6800/4188149333,v1:172.21.15.142:6801/4188149333]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-09-13T03:06:37.389 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:37 smithi170 ceph-mon[134328]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-09-13T03:06:37.389 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:37 smithi170 ceph-mon[134328]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T03:06:37.389 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:37 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:37.389 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:37 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:37.390 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:37 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:06:37.390 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:37 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:37.390 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:37 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:06:37.390 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:37 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:06:37.390 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:37 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:37.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:37 smithi064 ceph-mon[97675]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-09-13T03:06:37.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:37 smithi064 ceph-mon[97675]: osdmap e11: 2 total, 1 up, 2 in 2023-09-13T03:06:37.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:37 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-13T03:06:37.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:37 smithi064 ceph-mon[97675]: from='osd.1 [v2:172.21.15.142:6800/4188149333,v1:172.21.15.142:6801/4188149333]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-09-13T03:06:37.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:37 smithi064 ceph-mon[97675]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-09-13T03:06:37.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:37 smithi064 ceph-mon[97675]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T03:06:37.421 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:37 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:37.421 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:37 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:37.421 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:37 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:06:37.422 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:37 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:37.422 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:37 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:06:37.422 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:37 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:06:37.422 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:37 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:37.495 INFO:journalctl@ceph.osd.1.smithi142.stdout:Sep 13 03:06:37 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-1[139226]: 2023-09-13T03:06:37.131+0000 7f7d3f875700 -1 osd.1 0 waiting for initial osdmap 2023-09-13T03:06:37.495 INFO:journalctl@ceph.osd.1.smithi142.stdout:Sep 13 03:06:37 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-1[139226]: 2023-09-13T03:06:37.135+0000 7f7d3be6b700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-13T03:06:37.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:37 smithi142 ceph-mon[134082]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-09-13T03:06:37.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:37 smithi142 ceph-mon[134082]: osdmap e11: 2 total, 1 up, 2 in 2023-09-13T03:06:37.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:37 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-13T03:06:37.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:37 smithi142 ceph-mon[134082]: from='osd.1 [v2:172.21.15.142:6800/4188149333,v1:172.21.15.142:6801/4188149333]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-09-13T03:06:37.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:37 smithi142 ceph-mon[134082]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-09-13T03:06:37.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:37 smithi142 ceph-mon[134082]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T03:06:37.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:37 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:37.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:37 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:37.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:37 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:06:37.498 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:37 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:37.498 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:37 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:06:37.498 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:37 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:06:37.498 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:37 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:38.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:38.327+0000 7f4e64aa3700 1 -- 172.21.15.170:0/1407899815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e60101130 msgr2=0x7f4e601079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:06:38.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:38.327+0000 7f4e64aa3700 1 --2- 172.21.15.170:0/1407899815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e60101130 0x7f4e601079e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4e5800b0a0 tx=0x7f4e5800b3b0 comp rx=0 tx=0).stop 2023-09-13T03:06:38.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:38.327+0000 7f4e64aa3700 1 -- 172.21.15.170:0/1407899815 shutdown_connections 2023-09-13T03:06:38.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:38.327+0000 7f4e64aa3700 1 --2- 172.21.15.170:0/1407899815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e60101130 0x7f4e601079e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:06:38.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:38.327+0000 7f4e64aa3700 1 --2- 172.21.15.170:0/1407899815 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e600ff590 0x7f4e600ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:06:38.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:38.327+0000 7f4e64aa3700 1 --2- 172.21.15.170:0/1407899815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e60100790 0x7f4e60100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:06:38.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:38.327+0000 7f4e64aa3700 1 -- 172.21.15.170:0/1407899815 >> 172.21.15.170:0/1407899815 conn(0x7f4e600fab10 msgr2=0x7f4e600fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:06:38.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:38.328+0000 7f4e64aa3700 1 -- 172.21.15.170:0/1407899815 shutdown_connections 2023-09-13T03:06:38.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:38.328+0000 7f4e64aa3700 1 -- 172.21.15.170:0/1407899815 wait complete. 2023-09-13T03:06:38.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:38.328+0000 7f4e64aa3700 1 Processor -- start 2023-09-13T03:06:38.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:38.328+0000 7f4e64aa3700 1 -- start start 2023-09-13T03:06:38.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:38.329+0000 7f4e64aa3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e600ff590 0x7f4e600694d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:06:38.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:38.329+0000 7f4e64aa3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e60100790 0x7f4e6019b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:06:38.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:38.329+0000 7f4e64aa3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e60101130 0x7f4e6019d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:06:38.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:38.329+0000 7f4e64aa3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6019dfb0 con 0x7f4e600ff590 2023-09-13T03:06:38.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:38.329+0000 7f4e64aa3700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6019e120 con 0x7f4e60101130 2023-09-13T03:06:38.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:38.329+0000 7f4e64aa3700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6019e2e0 con 0x7f4e60100790 2023-09-13T03:06:38.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:38.329+0000 7f4e5dd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e60100790 0x7f4e6019b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:06:38.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:38.329+0000 7f4e5dd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e60100790 0x7f4e6019b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45426/0 (socket says 172.21.15.170:45426) 2023-09-13T03:06:38.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:38.329+0000 7f4e5dd9b700 1 -- 172.21.15.170:0/3764384573 learned_addr learned my addr 172.21.15.170:0/3764384573 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-09-13T03:06:38.334 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:38.329+0000 7f4e5ed9d700 1 --2- 172.21.15.170:0/3764384573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e60101130 0x7f4e6019d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:06:38.334 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:38.329+0000 7f4e5dd9b700 1 -- 172.21.15.170:0/3764384573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e60101130 msgr2=0x7f4e6019d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:06:38.334 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:38.329+0000 7f4e5dd9b700 1 --2- 172.21.15.170:0/3764384573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e60101130 0x7f4e6019d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:06:38.334 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:38.329+0000 7f4e5dd9b700 1 -- 172.21.15.170:0/3764384573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e600ff590 msgr2=0x7f4e600694d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:06:38.334 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:38.329+0000 7f4e5e59c700 1 --2- 172.21.15.170:0/3764384573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e600ff590 0x7f4e600694d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:06:38.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:38.329+0000 7f4e5dd9b700 1 --2- 172.21.15.170:0/3764384573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e600ff590 0x7f4e600694d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:06:38.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:38.329+0000 7f4e5dd9b700 1 -- 172.21.15.170:0/3764384573 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e58009d00 con 0x7f4e60100790 2023-09-13T03:06:38.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:38.329+0000 7f4e5e59c700 1 --2- 172.21.15.170:0/3764384573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e600ff590 0x7f4e600694d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T03:06:38.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:38.329+0000 7f4e5dd9b700 1 --2- 172.21.15.170:0/3764384573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e60100790 0x7f4e6019b1c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4e5000c9b0 tx=0x7f4e5000ccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:06:38.336 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:38.329+0000 7f4e4f7fe700 1 -- 172.21.15.170:0/3764384573 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e50007b00 con 0x7f4e60100790 2023-09-13T03:06:38.336 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:38.329+0000 7f4e4f7fe700 1 -- 172.21.15.170:0/3764384573 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e50007c60 con 0x7f4e60100790 2023-09-13T03:06:38.336 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:38.329+0000 7f4e64aa3700 1 -- 172.21.15.170:0/3764384573 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e6019e5c0 con 0x7f4e60100790 2023-09-13T03:06:38.337 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:38.329+0000 7f4e4f7fe700 1 -- 172.21.15.170:0/3764384573 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e50018660 con 0x7f4e60100790 2023-09-13T03:06:38.337 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:38.329+0000 7f4e64aa3700 1 -- 172.21.15.170:0/3764384573 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e6019eb40 con 0x7f4e60100790 2023-09-13T03:06:38.338 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:38.330+0000 7f4e64aa3700 1 -- 172.21.15.170:0/3764384573 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e6010ae40 con 0x7f4e60100790 2023-09-13T03:06:38.338 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:38.331+0000 7f4e4f7fe700 1 -- 172.21.15.170:0/3764384573 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 15) v1 ==== 135203+0+0 (secure 0 0 0) 0x7f4e5000f450 con 0x7f4e60100790 2023-09-13T03:06:38.338 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:38.332+0000 7f4e4f7fe700 1 --2- 172.21.15.170:0/3764384573 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f4e480a05b0 0x7f4e480a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:06:38.339 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:38.332+0000 7f4e4f7fe700 1 -- 172.21.15.170:0/3764384573 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f4e500ca2e0 con 0x7f4e60100790 2023-09-13T03:06:38.339 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:38.332+0000 7f4e5e59c700 1 --2- 172.21.15.170:0/3764384573 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f4e480a05b0 0x7f4e480a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:06:38.339 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:38.333+0000 7f4e5e59c700 1 --2- 172.21.15.170:0/3764384573 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f4e480a05b0 0x7f4e480a2a70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4e601005f0 tx=0x7f4e54008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:06:38.339 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:38.337+0000 7f4e4f7fe700 1 -- 172.21.15.170:0/3764384573 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f4e50095660 con 0x7f4e60100790 2023-09-13T03:06:38.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:38 smithi170 ceph-mon[134328]: Detected new or changed devices on smithi142 2023-09-13T03:06:38.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:38 smithi170 ceph-mon[134328]: Adjusting osd_memory_target on smithi142 to 17168M 2023-09-13T03:06:38.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:38 smithi170 ceph-mon[134328]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2023-09-13T03:06:38.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:38 smithi170 ceph-mon[134328]: osdmap e12: 2 total, 1 up, 2 in 2023-09-13T03:06:38.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:38 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-13T03:06:38.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:38 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-13T03:06:38.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:38 smithi064 ceph-mon[97675]: Detected new or changed devices on smithi142 2023-09-13T03:06:38.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:38 smithi064 ceph-mon[97675]: Adjusting osd_memory_target on smithi142 to 17168M 2023-09-13T03:06:38.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:38 smithi064 ceph-mon[97675]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2023-09-13T03:06:38.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:38 smithi064 ceph-mon[97675]: osdmap e12: 2 total, 1 up, 2 in 2023-09-13T03:06:38.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:38 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-13T03:06:38.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:38 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-13T03:06:38.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:38 smithi142 ceph-mon[134082]: Detected new or changed devices on smithi142 2023-09-13T03:06:38.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:38 smithi142 ceph-mon[134082]: Adjusting osd_memory_target on smithi142 to 17168M 2023-09-13T03:06:38.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:38 smithi142 ceph-mon[134082]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2023-09-13T03:06:38.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:38 smithi142 ceph-mon[134082]: osdmap e12: 2 total, 1 up, 2 in 2023-09-13T03:06:38.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:38 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-13T03:06:38.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:38 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-13T03:06:38.534 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:38.534+0000 7f4e64aa3700 1 -- 172.21.15.170:0/3764384573 --> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e60060a40 con 0x7f4e480a05b0 2023-09-13T03:06:39.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:39 smithi170 ceph-mon[134328]: purged_snaps scrub starts 2023-09-13T03:06:39.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:39 smithi170 ceph-mon[134328]: purged_snaps scrub ok 2023-09-13T03:06:39.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:39 smithi170 ceph-mon[134328]: osd.1 [v2:172.21.15.142:6800/4188149333,v1:172.21.15.142:6801/4188149333] boot 2023-09-13T03:06:39.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:39 smithi170 ceph-mon[134328]: osdmap e13: 2 total, 2 up, 2 in 2023-09-13T03:06:39.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:39 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-13T03:06:39.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:39 smithi170 ceph-mon[134328]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T03:06:39.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:39 smithi170 ceph-mon[134328]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:06:39.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:39 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T03:06:39.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:39 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T03:06:39.412 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:39 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:06:39.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:39 smithi064 ceph-mon[97675]: purged_snaps scrub starts 2023-09-13T03:06:39.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:39 smithi064 ceph-mon[97675]: purged_snaps scrub ok 2023-09-13T03:06:39.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:39 smithi064 ceph-mon[97675]: osd.1 [v2:172.21.15.142:6800/4188149333,v1:172.21.15.142:6801/4188149333] boot 2023-09-13T03:06:39.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:39 smithi064 ceph-mon[97675]: osdmap e13: 2 total, 2 up, 2 in 2023-09-13T03:06:39.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:39 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-13T03:06:39.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:39 smithi064 ceph-mon[97675]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T03:06:39.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:39 smithi064 ceph-mon[97675]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:06:39.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:39 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T03:06:39.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:39 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T03:06:39.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:39 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:06:39.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:39 smithi142 ceph-mon[134082]: purged_snaps scrub starts 2023-09-13T03:06:39.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:39 smithi142 ceph-mon[134082]: purged_snaps scrub ok 2023-09-13T03:06:39.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:39 smithi142 ceph-mon[134082]: osd.1 [v2:172.21.15.142:6800/4188149333,v1:172.21.15.142:6801/4188149333] boot 2023-09-13T03:06:39.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:39 smithi142 ceph-mon[134082]: osdmap e13: 2 total, 2 up, 2 in 2023-09-13T03:06:39.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:39 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-13T03:06:39.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:39 smithi142 ceph-mon[134082]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-13T03:06:39.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:39 smithi142 ceph-mon[134082]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:06:39.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:39 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T03:06:39.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:39 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T03:06:39.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:39 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:06:40.409 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:40 smithi170 ceph-mon[134328]: osdmap e14: 2 total, 2 up, 2 in 2023-09-13T03:06:40.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:40 smithi064 ceph-mon[97675]: osdmap e14: 2 total, 2 up, 2 in 2023-09-13T03:06:40.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:40 smithi142 ceph-mon[134082]: osdmap e14: 2 total, 2 up, 2 in 2023-09-13T03:06:41.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:41 smithi170 ceph-mon[134328]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T03:06:41.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:41 smithi170 ceph-mon[134328]: from='client.? 172.21.15.170:0/1438864029' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3aae17e-96db-4008-8c27-f733c46b30ee"}]: dispatch 2023-09-13T03:06:41.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:41 smithi170 ceph-mon[134328]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3aae17e-96db-4008-8c27-f733c46b30ee"}]: dispatch 2023-09-13T03:06:41.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:41 smithi170 ceph-mon[134328]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b3aae17e-96db-4008-8c27-f733c46b30ee"}]': finished 2023-09-13T03:06:41.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:41 smithi170 ceph-mon[134328]: osdmap e15: 3 total, 2 up, 3 in 2023-09-13T03:06:41.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:41 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-13T03:06:41.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:41 smithi064 ceph-mon[97675]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T03:06:41.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:41 smithi064 ceph-mon[97675]: from='client.? 172.21.15.170:0/1438864029' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3aae17e-96db-4008-8c27-f733c46b30ee"}]: dispatch 2023-09-13T03:06:41.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:41 smithi064 ceph-mon[97675]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3aae17e-96db-4008-8c27-f733c46b30ee"}]: dispatch 2023-09-13T03:06:41.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:41 smithi064 ceph-mon[97675]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b3aae17e-96db-4008-8c27-f733c46b30ee"}]': finished 2023-09-13T03:06:41.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:41 smithi064 ceph-mon[97675]: osdmap e15: 3 total, 2 up, 3 in 2023-09-13T03:06:41.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:41 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-13T03:06:41.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:41 smithi142 ceph-mon[134082]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T03:06:41.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:41 smithi142 ceph-mon[134082]: from='client.? 172.21.15.170:0/1438864029' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3aae17e-96db-4008-8c27-f733c46b30ee"}]: dispatch 2023-09-13T03:06:41.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:41 smithi142 ceph-mon[134082]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3aae17e-96db-4008-8c27-f733c46b30ee"}]: dispatch 2023-09-13T03:06:41.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:41 smithi142 ceph-mon[134082]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b3aae17e-96db-4008-8c27-f733c46b30ee"}]': finished 2023-09-13T03:06:41.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:41 smithi142 ceph-mon[134082]: osdmap e15: 3 total, 2 up, 3 in 2023-09-13T03:06:41.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:41 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-13T03:06:42.409 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:42 smithi170 ceph-mon[134328]: from='client.? 172.21.15.170:0/1062870180' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T03:06:42.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:42 smithi064 ceph-mon[97675]: from='client.? 172.21.15.170:0/1062870180' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T03:06:42.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:42 smithi142 ceph-mon[134082]: from='client.? 172.21.15.170:0/1062870180' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T03:06:43.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:43 smithi170 ceph-mon[134328]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T03:06:43.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:43 smithi064 ceph-mon[97675]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T03:06:43.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:43 smithi142 ceph-mon[134082]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T03:06:45.409 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:45 smithi170 ceph-mon[134328]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T03:06:45.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:45 smithi064 ceph-mon[97675]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T03:06:45.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:45 smithi142 ceph-mon[134082]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T03:06:47.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:47 smithi170 ceph-mon[134328]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T03:06:47.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:47 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-09-13T03:06:47.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:47 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:06:47.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:47 smithi142 ceph-mon[134082]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T03:06:47.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:47 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-09-13T03:06:47.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:47 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:06:47.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:47 smithi064 ceph-mon[97675]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T03:06:47.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:47 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-09-13T03:06:47.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:47 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:06:48.544 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:48 smithi170 ceph-mon[134328]: Deploying daemon osd.2 on smithi170 2023-09-13T03:06:48.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:48 smithi142 ceph-mon[134082]: Deploying daemon osd.2 on smithi170 2023-09-13T03:06:48.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:48 smithi064 ceph-mon[97675]: Deploying daemon osd.2 on smithi170 2023-09-13T03:06:49.533 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:49 smithi170 ceph-mon[134328]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T03:06:49.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:49 smithi142 ceph-mon[134082]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T03:06:49.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:49 smithi064 ceph-mon[97675]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T03:06:51.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:51 smithi142 ceph-mon[134082]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T03:06:51.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:51 smithi170 ceph-mon[134328]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T03:06:51.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:51 smithi064 ceph-mon[97675]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T03:06:52.788 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:52 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:52.788 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:52 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:06:52.789 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:52 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:52.789 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:52 smithi170 ceph-mon[134328]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T03:06:52.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:52 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:52.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:52 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:06:52.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:52 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:52.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:52 smithi142 ceph-mon[134082]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T03:06:53.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:52 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:53.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:52 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:06:53.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:52 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:53.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:52 smithi064 ceph-mon[97675]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T03:06:54.198 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:54.197+0000 7f4e4f7fe700 1 -- 172.21.15.170:0/3764384573 <== mgr.14150 v2:172.21.15.64:6800/873679308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f4e60060a40 con 0x7f4e480a05b0 2023-09-13T03:06:54.198 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 2 on host 'smithi170' 2023-09-13T03:06:54.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:54.199+0000 7f4e64aa3700 1 -- 172.21.15.170:0/3764384573 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f4e480a05b0 msgr2=0x7f4e480a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:06:54.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:54.199+0000 7f4e64aa3700 1 --2- 172.21.15.170:0/3764384573 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f4e480a05b0 0x7f4e480a2a70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4e601005f0 tx=0x7f4e54008040 comp rx=0 tx=0).stop 2023-09-13T03:06:54.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:54.199+0000 7f4e64aa3700 1 -- 172.21.15.170:0/3764384573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e60100790 msgr2=0x7f4e6019b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:06:54.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:54.199+0000 7f4e64aa3700 1 --2- 172.21.15.170:0/3764384573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e60100790 0x7f4e6019b1c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4e5000c9b0 tx=0x7f4e5000ccc0 comp rx=0 tx=0).stop 2023-09-13T03:06:54.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:54.199+0000 7f4e64aa3700 1 -- 172.21.15.170:0/3764384573 shutdown_connections 2023-09-13T03:06:54.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:54.199+0000 7f4e64aa3700 1 --2- 172.21.15.170:0/3764384573 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f4e480a05b0 0x7f4e480a2a70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:06:54.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:54.199+0000 7f4e64aa3700 1 --2- 172.21.15.170:0/3764384573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e60101130 0x7f4e6019d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:06:54.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:54.199+0000 7f4e64aa3700 1 --2- 172.21.15.170:0/3764384573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e600ff590 0x7f4e600694d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:06:54.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:54.199+0000 7f4e64aa3700 1 --2- 172.21.15.170:0/3764384573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e60100790 0x7f4e6019b1c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:06:54.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:54.199+0000 7f4e64aa3700 1 -- 172.21.15.170:0/3764384573 >> 172.21.15.170:0/3764384573 conn(0x7f4e600fab10 msgr2=0x7f4e600fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:06:54.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:54.199+0000 7f4e64aa3700 1 -- 172.21.15.170:0/3764384573 shutdown_connections 2023-09-13T03:06:54.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:06:54.199+0000 7f4e64aa3700 1 -- 172.21.15.170:0/3764384573 wait complete. 2023-09-13T03:06:54.548 DEBUG:teuthology.orchestra.run.smithi170:osd.2> sudo journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@osd.2.service 2023-09-13T03:06:54.551 INFO:tasks.cephadm:Deploying osd.3 on smithi064 with /dev/vg_nvme/lv_3... 2023-09-13T03:06:54.552 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-09-13T03:06:54.580 INFO:journalctl@ceph.osd.2.smithi170.stdout:-- Logs begin at Wed 2023-09-13 02:42:51 UTC. -- 2023-09-13T03:06:55.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:54 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:55.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:54 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:55.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:54 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:06:55.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:54 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:06:55.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:54 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:55.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:54 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:55.170 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:54 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:55.170 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:54 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:06:55.170 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:54 smithi064 ceph-mon[97675]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T03:06:55.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:54 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:55.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:54 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:55.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:54 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:06:55.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:54 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:06:55.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:54 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:55.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:54 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:55.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:54 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:55.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:54 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:06:55.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:54 smithi142 ceph-mon[134082]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T03:06:55.290 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:06:55.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:54 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:55.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:54 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:55.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:54 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:06:55.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:54 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:06:55.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:54 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:55.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:54 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:55.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:54 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:55.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:54 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:06:55.412 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:54 smithi170 ceph-mon[134328]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T03:06:55.904 INFO:journalctl@ceph.osd.2.smithi170.stdout:Sep 13 03:06:55 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-2[140189]: 2023-09-13T03:06:55.699+0000 7f632494f640 -1 osd.2 0 log_to_monitors true 2023-09-13T03:06:56.465 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:56 smithi064 ceph-mon[97675]: from='osd.2 [v2:172.21.15.170:6800/2406669338,v1:172.21.15.170:6801/2406669338]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-13T03:06:56.466 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:56 smithi064 ceph-mon[97675]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-13T03:06:56.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:56 smithi142 ceph-mon[134082]: from='osd.2 [v2:172.21.15.170:6800/2406669338,v1:172.21.15.170:6801/2406669338]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-13T03:06:56.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:56 smithi142 ceph-mon[134082]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-13T03:06:56.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:56 smithi170 ceph-mon[134328]: from='osd.2 [v2:172.21.15.170:6800/2406669338,v1:172.21.15.170:6801/2406669338]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-13T03:06:56.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:56 smithi170 ceph-mon[134328]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-13T03:06:57.151 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-09-13T03:06:57.152 INFO:teuthology.orchestra.run.smithi064.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-09-13T03:06:57.152 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10+0 records in 2023-09-13T03:06:57.152 INFO:teuthology.orchestra.run.smithi064.stderr:10+0 records out 2023-09-13T03:06:57.152 INFO:teuthology.orchestra.run.smithi064.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0300882 s, 349 MB/s 2023-09-13T03:06:57.153 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping successful for: 2023-09-13T03:06:57.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:57 smithi142 ceph-mon[134082]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-09-13T03:06:57.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:57 smithi142 ceph-mon[134082]: osdmap e16: 3 total, 2 up, 3 in 2023-09-13T03:06:57.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:57 smithi142 ceph-mon[134082]: from='osd.2 [v2:172.21.15.170:6800/2406669338,v1:172.21.15.170:6801/2406669338]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-13T03:06:57.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:57 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-13T03:06:57.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:57 smithi142 ceph-mon[134082]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-13T03:06:57.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:57 smithi142 ceph-mon[134082]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T03:06:57.527 INFO:journalctl@ceph.osd.2.smithi170.stdout:Sep 13 03:06:57 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-2[140189]: 2023-09-13T03:06:57.209+0000 7f63197c5700 -1 osd.2 0 waiting for initial osdmap 2023-09-13T03:06:57.527 INFO:journalctl@ceph.osd.2.smithi170.stdout:Sep 13 03:06:57 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-2[140189]: 2023-09-13T03:06:57.215+0000 7f6312db5700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-13T03:06:57.527 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:57 smithi170 ceph-mon[134328]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-09-13T03:06:57.528 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:57 smithi170 ceph-mon[134328]: osdmap e16: 3 total, 2 up, 3 in 2023-09-13T03:06:57.528 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:57 smithi170 ceph-mon[134328]: from='osd.2 [v2:172.21.15.170:6800/2406669338,v1:172.21.15.170:6801/2406669338]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-13T03:06:57.528 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:57 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-13T03:06:57.528 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:57 smithi170 ceph-mon[134328]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-13T03:06:57.528 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:57 smithi170 ceph-mon[134328]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T03:06:57.568 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:57 smithi064 ceph-mon[97675]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-09-13T03:06:57.568 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:57 smithi064 ceph-mon[97675]: osdmap e16: 3 total, 2 up, 3 in 2023-09-13T03:06:57.568 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:57 smithi064 ceph-mon[97675]: from='osd.2 [v2:172.21.15.170:6800/2406669338,v1:172.21.15.170:6801/2406669338]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-13T03:06:57.569 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:57 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-13T03:06:57.569 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:57 smithi064 ceph-mon[97675]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-13T03:06:57.569 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:57 smithi064 ceph-mon[97675]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T03:06:57.571 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi064:vg_nvme/lv_3 2023-09-13T03:06:58.308 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:06:58.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:58 smithi142 ceph-mon[134082]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2023-09-13T03:06:58.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:58 smithi142 ceph-mon[134082]: osdmap e17: 3 total, 2 up, 3 in 2023-09-13T03:06:58.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:58 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-13T03:06:58.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:58 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-13T03:06:58.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:58 smithi142 ceph-mon[134082]: Detected new or changed devices on smithi170 2023-09-13T03:06:58.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:58 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:58.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:58 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:58.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:58 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:06:58.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:58 smithi142 ceph-mon[134082]: Adjusting osd_memory_target on smithi170 to 17168M 2023-09-13T03:06:58.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:58 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:58.498 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:58 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:06:58.498 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:58 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:06:58.498 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:58 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:58.554 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:58 smithi064 ceph-mon[97675]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2023-09-13T03:06:58.554 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:58 smithi064 ceph-mon[97675]: osdmap e17: 3 total, 2 up, 3 in 2023-09-13T03:06:58.554 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:58 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-13T03:06:58.555 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:58 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-13T03:06:58.555 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:58 smithi064 ceph-mon[97675]: Detected new or changed devices on smithi170 2023-09-13T03:06:58.555 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:58 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:58.555 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:58 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:58.555 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:58 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:06:58.556 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:58 smithi064 ceph-mon[97675]: Adjusting osd_memory_target on smithi170 to 17168M 2023-09-13T03:06:58.556 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:58 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:58.556 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:58 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:06:58.556 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:58 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:06:58.557 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:58 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:58.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:58 smithi170 ceph-mon[134328]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2023-09-13T03:06:58.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:58 smithi170 ceph-mon[134328]: osdmap e17: 3 total, 2 up, 3 in 2023-09-13T03:06:58.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:58 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-13T03:06:58.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:58 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-13T03:06:58.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:58 smithi170 ceph-mon[134328]: Detected new or changed devices on smithi170 2023-09-13T03:06:58.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:58 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:58.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:58 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:58.662 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:58 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:06:58.662 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:58 smithi170 ceph-mon[134328]: Adjusting osd_memory_target on smithi170 to 17168M 2023-09-13T03:06:58.662 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:58 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:58.662 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:58 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:06:58.662 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:58 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:06:58.663 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:58 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:06:59.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:59 smithi064 ceph-mon[97675]: purged_snaps scrub starts 2023-09-13T03:06:59.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:59 smithi064 ceph-mon[97675]: purged_snaps scrub ok 2023-09-13T03:06:59.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:59 smithi064 ceph-mon[97675]: osd.2 [v2:172.21.15.170:6800/2406669338,v1:172.21.15.170:6801/2406669338] boot 2023-09-13T03:06:59.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:59 smithi064 ceph-mon[97675]: osdmap e18: 3 total, 3 up, 3 in 2023-09-13T03:06:59.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:59 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-13T03:06:59.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:59 smithi064 ceph-mon[97675]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T03:06:59.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:06:59 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-09-13T03:06:59.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:59 smithi142 ceph-mon[134082]: purged_snaps scrub starts 2023-09-13T03:06:59.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:59 smithi142 ceph-mon[134082]: purged_snaps scrub ok 2023-09-13T03:06:59.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:59 smithi142 ceph-mon[134082]: osd.2 [v2:172.21.15.170:6800/2406669338,v1:172.21.15.170:6801/2406669338] boot 2023-09-13T03:06:59.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:59 smithi142 ceph-mon[134082]: osdmap e18: 3 total, 3 up, 3 in 2023-09-13T03:06:59.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:59 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-13T03:06:59.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:59 smithi142 ceph-mon[134082]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T03:06:59.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:06:59 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-09-13T03:06:59.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:59 smithi170 ceph-mon[134328]: purged_snaps scrub starts 2023-09-13T03:06:59.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:59 smithi170 ceph-mon[134328]: purged_snaps scrub ok 2023-09-13T03:06:59.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:59 smithi170 ceph-mon[134328]: osd.2 [v2:172.21.15.170:6800/2406669338,v1:172.21.15.170:6801/2406669338] boot 2023-09-13T03:06:59.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:59 smithi170 ceph-mon[134328]: osdmap e18: 3 total, 3 up, 3 in 2023-09-13T03:06:59.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:59 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-13T03:06:59.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:59 smithi170 ceph-mon[134328]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-13T03:06:59.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:06:59 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-09-13T03:06:59.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:59.711+0000 7f38d327b700 1 -- 172.21.15.64:0/2982279763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38cc100690 msgr2=0x7f38cc100ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:06:59.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:59.711+0000 7f38d327b700 1 --2- 172.21.15.64:0/2982279763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38cc100690 0x7f38cc100ab0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f38c80099c0 tx=0x7f38c8009cd0 comp rx=0 tx=0).stop 2023-09-13T03:06:59.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:59.711+0000 7f38d327b700 1 -- 172.21.15.64:0/2982279763 shutdown_connections 2023-09-13T03:06:59.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:59.711+0000 7f38d327b700 1 --2- 172.21.15.64:0/2982279763 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38cc102230 0x7f38cc108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:06:59.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:59.711+0000 7f38d327b700 1 --2- 172.21.15.64:0/2982279763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38cc100690 0x7f38cc100ab0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:06:59.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:59.711+0000 7f38d327b700 1 --2- 172.21.15.64:0/2982279763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38cc101890 0x7f38cc101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:06:59.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:59.711+0000 7f38d327b700 1 -- 172.21.15.64:0/2982279763 >> 172.21.15.64:0/2982279763 conn(0x7f38cc0fbbf0 msgr2=0x7f38cc0fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:06:59.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:59.711+0000 7f38d327b700 1 -- 172.21.15.64:0/2982279763 shutdown_connections 2023-09-13T03:06:59.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:59.711+0000 7f38d327b700 1 -- 172.21.15.64:0/2982279763 wait complete. 2023-09-13T03:06:59.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:59.712+0000 7f38d327b700 1 Processor -- start 2023-09-13T03:06:59.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:59.713+0000 7f38d327b700 1 -- start start 2023-09-13T03:06:59.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:59.713+0000 7f38d327b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38cc100690 0x7f38cc0743a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:06:59.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:59.713+0000 7f38d327b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38cc101890 0x7f38cc0729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:06:59.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:59.713+0000 7f38d327b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38cc102230 0x7f38cc072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:06:59.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:59.713+0000 7f38d327b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc0736b0 con 0x7f38cc100690 2023-09-13T03:06:59.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:59.713+0000 7f38d327b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc073820 con 0x7f38cc101890 2023-09-13T03:06:59.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:59.713+0000 7f38d327b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc1a03f0 con 0x7f38cc102230 2023-09-13T03:06:59.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:59.713+0000 7f38d1017700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38cc100690 0x7f38cc0743a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:06:59.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:59.713+0000 7f38d0816700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38cc101890 0x7f38cc0729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:06:59.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:59.713+0000 7f38d1017700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38cc100690 0x7f38cc0743a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38908/0 (socket says 172.21.15.64:38908) 2023-09-13T03:06:59.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:59.713+0000 7f38d1017700 1 -- 172.21.15.64:0/3572790888 learned_addr learned my addr 172.21.15.64:0/3572790888 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:06:59.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:59.713+0000 7f38d1818700 1 --2- 172.21.15.64:0/3572790888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38cc102230 0x7f38cc072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:06:59.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:59.714+0000 7f38d1017700 1 -- 172.21.15.64:0/3572790888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38cc102230 msgr2=0x7f38cc072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:06:59.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:59.714+0000 7f38d1017700 1 --2- 172.21.15.64:0/3572790888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38cc102230 0x7f38cc072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:06:59.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:59.714+0000 7f38d1017700 1 -- 172.21.15.64:0/3572790888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38cc101890 msgr2=0x7f38cc0729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:06:59.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:59.714+0000 7f38d1017700 1 --2- 172.21.15.64:0/3572790888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38cc101890 0x7f38cc0729f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:06:59.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:59.714+0000 7f38d1017700 1 -- 172.21.15.64:0/3572790888 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38bc009520 con 0x7f38cc100690 2023-09-13T03:06:59.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:59.714+0000 7f38d0816700 1 --2- 172.21.15.64:0/3572790888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38cc101890 0x7f38cc0729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:06:59.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:59.714+0000 7f38d1017700 1 --2- 172.21.15.64:0/3572790888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38cc100690 0x7f38cc0743a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f38c8009990 tx=0x7f38c800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:06:59.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:59.714+0000 7f38c27fc700 1 -- 172.21.15.64:0/3572790888 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c801d070 con 0x7f38cc100690 2023-09-13T03:06:59.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:59.714+0000 7f38d327b700 1 -- 172.21.15.64:0/3572790888 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38c80095f0 con 0x7f38cc100690 2023-09-13T03:06:59.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:59.714+0000 7f38c27fc700 1 -- 172.21.15.64:0/3572790888 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38c8005030 con 0x7f38cc100690 2023-09-13T03:06:59.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:59.714+0000 7f38c27fc700 1 -- 172.21.15.64:0/3572790888 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c8004a70 con 0x7f38cc100690 2023-09-13T03:06:59.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:59.714+0000 7f38d327b700 1 -- 172.21.15.64:0/3572790888 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38cc1a0950 con 0x7f38cc100690 2023-09-13T03:06:59.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:59.716+0000 7f38d327b700 1 -- 172.21.15.64:0/3572790888 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38cc10bee0 con 0x7f38cc100690 2023-09-13T03:06:59.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:59.718+0000 7f38c27fc700 1 -- 172.21.15.64:0/3572790888 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 15) v1 ==== 135203+0+0 (secure 0 0 0) 0x7f38c8005410 con 0x7f38cc100690 2023-09-13T03:06:59.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:59.720+0000 7f38c27fc700 1 --2- 172.21.15.64:0/3572790888 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f38b80a0630 0x7f38b80a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:06:59.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:59.721+0000 7f38c27fc700 1 -- 172.21.15.64:0/3572790888 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3226+0+0 (secure 0 0 0) 0x7f38c80cb820 con 0x7f38cc100690 2023-09-13T03:06:59.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:59.721+0000 7f38d0816700 1 --2- 172.21.15.64:0/3572790888 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f38b80a0630 0x7f38b80a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:06:59.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:59.721+0000 7f38d0816700 1 --2- 172.21.15.64:0/3572790888 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f38b80a0630 0x7f38b80a2af0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f38bc009fd0 tx=0x7f38bc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:06:59.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:59.723+0000 7f38c27fc700 1 -- 172.21.15.64:0/3572790888 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f38c8096800 con 0x7f38cc100690 2023-09-13T03:06:59.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:06:59.944+0000 7f38d327b700 1 -- 172.21.15.64:0/3572790888 --> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f38cc060a40 con 0x7f38b80a0630 2023-09-13T03:07:00.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:00 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-09-13T03:07:00.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:00 smithi142 ceph-mon[134082]: osdmap e19: 3 total, 3 up, 3 in 2023-09-13T03:07:00.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:00 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-09-13T03:07:00.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:00 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T03:07:00.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:00 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T03:07:00.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:00 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:07:00.659 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:00 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-09-13T03:07:00.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:00 smithi170 ceph-mon[134328]: osdmap e19: 3 total, 3 up, 3 in 2023-09-13T03:07:00.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:00 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-09-13T03:07:00.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:00 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T03:07:00.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:00 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T03:07:00.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:00 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:07:00.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:00 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-09-13T03:07:00.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:00 smithi064 ceph-mon[97675]: osdmap e19: 3 total, 3 up, 3 in 2023-09-13T03:07:00.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:00 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-09-13T03:07:00.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:00 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T03:07:00.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:00 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T03:07:00.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:00 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:07:01.387 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:01 smithi064 ceph-mon[97675]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:07:01.387 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:01 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-09-13T03:07:01.387 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:01 smithi064 ceph-mon[97675]: osdmap e20: 3 total, 3 up, 3 in 2023-09-13T03:07:01.387 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:01 smithi064 ceph-mon[97675]: pgmap v89: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-13T03:07:01.388 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 13 03:07:01 smithi064 sudo[125907]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-13T03:07:01.388 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 13 03:07:01 smithi064 sudo[125907]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-13T03:07:01.388 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 13 03:07:01 smithi064 sudo[125907]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-13T03:07:01.389 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 13 03:07:01 smithi064 sudo[125907]: pam_unix(sudo:session): session closed for user root 2023-09-13T03:07:01.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:01 smithi142 ceph-mon[134082]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:07:01.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:01 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-09-13T03:07:01.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:01 smithi142 ceph-mon[134082]: osdmap e20: 3 total, 3 up, 3 in 2023-09-13T03:07:01.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:01 smithi142 ceph-mon[134082]: pgmap v89: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-13T03:07:01.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:01 smithi170 ceph-mon[134328]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:07:01.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:01 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-09-13T03:07:01.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:01 smithi170 ceph-mon[134328]: osdmap e20: 3 total, 3 up, 3 in 2023-09-13T03:07:01.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:01 smithi170 ceph-mon[134328]: pgmap v89: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-13T03:07:01.918 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 13 03:07:01 smithi064 sudo[125959]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-13T03:07:01.919 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 13 03:07:01 smithi064 sudo[125959]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-13T03:07:01.919 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 13 03:07:01 smithi064 sudo[125959]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-13T03:07:01.919 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 13 03:07:01 smithi064 sudo[125959]: pam_unix(sudo:session): session closed for user root 2023-09-13T03:07:02.192 INFO:journalctl@ceph.osd.1.smithi142.stdout:Sep 13 03:07:01 smithi142 sudo[144205]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-13T03:07:02.192 INFO:journalctl@ceph.osd.1.smithi142.stdout:Sep 13 03:07:01 smithi142 sudo[144205]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-13T03:07:02.193 INFO:journalctl@ceph.osd.1.smithi142.stdout:Sep 13 03:07:01 smithi142 sudo[144205]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-13T03:07:02.193 INFO:journalctl@ceph.osd.1.smithi142.stdout:Sep 13 03:07:01 smithi142 sudo[144205]: pam_unix(sudo:session): session closed for user root 2023-09-13T03:07:02.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:02 smithi142 ceph-mon[134082]: osdmap e21: 3 total, 3 up, 3 in 2023-09-13T03:07:02.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:02 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:02.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:02 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:02.496 INFO:journalctl@ceph.osd.1.smithi142.stdout:Sep 13 03:07:02 smithi142 sudo[144208]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-13T03:07:02.496 INFO:journalctl@ceph.osd.1.smithi142.stdout:Sep 13 03:07:02 smithi142 sudo[144208]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-13T03:07:02.496 INFO:journalctl@ceph.osd.1.smithi142.stdout:Sep 13 03:07:02 smithi142 sudo[144208]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-13T03:07:02.496 INFO:journalctl@ceph.osd.1.smithi142.stdout:Sep 13 03:07:02 smithi142 sudo[144208]: pam_unix(sudo:session): session closed for user root 2023-09-13T03:07:02.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:02 smithi170 ceph-mon[134328]: osdmap e21: 3 total, 3 up, 3 in 2023-09-13T03:07:02.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:02.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:02.661 INFO:journalctl@ceph.osd.2.smithi170.stdout:Sep 13 03:07:02 smithi170 sudo[144953]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-13T03:07:02.661 INFO:journalctl@ceph.osd.2.smithi170.stdout:Sep 13 03:07:02 smithi170 sudo[144953]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-13T03:07:02.661 INFO:journalctl@ceph.osd.2.smithi170.stdout:Sep 13 03:07:02 smithi170 sudo[144953]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-13T03:07:02.661 INFO:journalctl@ceph.osd.2.smithi170.stdout:Sep 13 03:07:02 smithi170 sudo[144953]: pam_unix(sudo:session): session closed for user root 2023-09-13T03:07:02.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:02 smithi064 ceph-mon[97675]: osdmap e21: 3 total, 3 up, 3 in 2023-09-13T03:07:02.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:02 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:02.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:02 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:03.160 INFO:journalctl@ceph.osd.2.smithi170.stdout:Sep 13 03:07:02 smithi170 sudo[144956]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-13T03:07:03.160 INFO:journalctl@ceph.osd.2.smithi170.stdout:Sep 13 03:07:02 smithi170 sudo[144956]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-13T03:07:03.160 INFO:journalctl@ceph.osd.2.smithi170.stdout:Sep 13 03:07:02 smithi170 sudo[144956]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-13T03:07:03.160 INFO:journalctl@ceph.osd.2.smithi170.stdout:Sep 13 03:07:02 smithi170 sudo[144956]: pam_unix(sudo:session): session closed for user root 2023-09-13T03:07:03.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:03 smithi064 sudo[126047]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-13T03:07:03.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:03 smithi064 sudo[126047]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-13T03:07:03.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:03 smithi064 sudo[126047]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-13T03:07:03.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:03 smithi064 sudo[126047]: pam_unix(sudo:session): session closed for user root 2023-09-13T03:07:03.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:07:03.836+0000 7f38c27fc700 1 -- 172.21.15.64:0/3572790888 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f38c8096bd0 con 0x7f38cc100690 2023-09-13T03:07:04.085 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:03 smithi170 ceph-mon[134328]: pgmap v91: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-13T03:07:04.086 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:03 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:04.086 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:03 smithi170 ceph-mon[134328]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-13T03:07:04.086 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:04 smithi170 sudo[144959]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-13T03:07:04.086 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:04 smithi170 sudo[144959]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-13T03:07:04.087 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:04 smithi170 sudo[144959]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-13T03:07:04.087 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:04 smithi170 sudo[144959]: pam_unix(sudo:session): session closed for user root 2023-09-13T03:07:04.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:03 smithi064 sudo[126050]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-13T03:07:04.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:03 smithi064 sudo[126050]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-13T03:07:04.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:03 smithi064 sudo[126050]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-13T03:07:04.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:03 smithi064 sudo[126050]: pam_unix(sudo:session): session closed for user root 2023-09-13T03:07:04.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:03 smithi064 ceph-mon[97675]: pgmap v91: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-13T03:07:04.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:03 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:04.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:03 smithi064 ceph-mon[97675]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-13T03:07:04.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:03 smithi142 ceph-mon[134082]: pgmap v91: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-13T03:07:04.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:03 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:04.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:03 smithi142 ceph-mon[134082]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-13T03:07:04.409 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:04 smithi170 sudo[144962]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-13T03:07:04.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:04 smithi170 sudo[144962]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-13T03:07:04.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:04 smithi170 sudo[144962]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-13T03:07:04.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:04 smithi170 sudo[144962]: pam_unix(sudo:session): session closed for user root 2023-09-13T03:07:04.822 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:04 smithi142 sudo[144211]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-13T03:07:04.822 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:04 smithi142 sudo[144211]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-13T03:07:04.823 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:04 smithi142 sudo[144211]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-13T03:07:04.823 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:04 smithi142 sudo[144211]: pam_unix(sudo:session): session closed for user root 2023-09-13T03:07:05.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:04 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/2873865662' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fa8d8cb-2ab7-48a7-8b49-5e2903d6779f"}]: dispatch 2023-09-13T03:07:05.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:04 smithi170 ceph-mon[134328]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-13T03:07:05.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:04 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-13T03:07:05.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:04 smithi170 ceph-mon[134328]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fa8d8cb-2ab7-48a7-8b49-5e2903d6779f"}]: dispatch 2023-09-13T03:07:05.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:04 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T03:07:05.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:04 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T03:07:05.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:04 smithi170 ceph-mon[134328]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-13T03:07:05.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:04 smithi170 ceph-mon[134328]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-09-13T03:07:05.162 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:04 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-13T03:07:05.162 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:04 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T03:07:05.162 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:04 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T03:07:05.162 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:04 smithi170 ceph-mon[134328]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-13T03:07:05.163 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:04 smithi170 ceph-mon[134328]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7fa8d8cb-2ab7-48a7-8b49-5e2903d6779f"}]': finished 2023-09-13T03:07:05.163 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:04 smithi170 ceph-mon[134328]: osdmap e22: 4 total, 3 up, 4 in 2023-09-13T03:07:05.163 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:04 smithi170 ceph-mon[134328]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-13T03:07:05.163 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:04 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-13T03:07:05.163 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:04 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-13T03:07:05.164 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:04 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T03:07:05.164 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:04 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T03:07:05.164 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:04 smithi170 ceph-mon[134328]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-13T03:07:05.164 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:04 smithi170 ceph-mon[134328]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-13T03:07:05.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:04 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2873865662' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fa8d8cb-2ab7-48a7-8b49-5e2903d6779f"}]: dispatch 2023-09-13T03:07:05.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:04 smithi064 ceph-mon[97675]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-13T03:07:05.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:04 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-13T03:07:05.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:04 smithi064 ceph-mon[97675]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fa8d8cb-2ab7-48a7-8b49-5e2903d6779f"}]: dispatch 2023-09-13T03:07:05.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:04 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T03:07:05.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:04 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T03:07:05.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:04 smithi064 ceph-mon[97675]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-13T03:07:05.170 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:04 smithi064 ceph-mon[97675]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-09-13T03:07:05.170 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:04 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-13T03:07:05.170 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:04 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T03:07:05.170 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:04 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T03:07:05.170 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:04 smithi064 ceph-mon[97675]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-13T03:07:05.171 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:04 smithi064 ceph-mon[97675]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7fa8d8cb-2ab7-48a7-8b49-5e2903d6779f"}]': finished 2023-09-13T03:07:05.171 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:04 smithi064 ceph-mon[97675]: osdmap e22: 4 total, 3 up, 4 in 2023-09-13T03:07:05.171 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:04 smithi064 ceph-mon[97675]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-13T03:07:05.171 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:04 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-13T03:07:05.171 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:04 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-13T03:07:05.172 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:04 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T03:07:05.172 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:04 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T03:07:05.172 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:04 smithi064 ceph-mon[97675]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-13T03:07:05.172 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:04 smithi064 ceph-mon[97675]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-13T03:07:05.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:04 smithi142 sudo[144214]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-13T03:07:05.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:04 smithi142 sudo[144214]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-13T03:07:05.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:04 smithi142 sudo[144214]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-13T03:07:05.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:04 smithi142 sudo[144214]: pam_unix(sudo:session): session closed for user root 2023-09-13T03:07:05.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:04 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/2873865662' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fa8d8cb-2ab7-48a7-8b49-5e2903d6779f"}]: dispatch 2023-09-13T03:07:05.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:04 smithi142 ceph-mon[134082]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-13T03:07:05.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:04 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-13T03:07:05.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:04 smithi142 ceph-mon[134082]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fa8d8cb-2ab7-48a7-8b49-5e2903d6779f"}]: dispatch 2023-09-13T03:07:05.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:04 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T03:07:05.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:04 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T03:07:05.248 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:04 smithi142 ceph-mon[134082]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-13T03:07:05.249 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:04 smithi142 ceph-mon[134082]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-09-13T03:07:05.249 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:04 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-13T03:07:05.249 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:04 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T03:07:05.249 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:04 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T03:07:05.250 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:04 smithi142 ceph-mon[134082]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-13T03:07:05.250 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:04 smithi142 ceph-mon[134082]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7fa8d8cb-2ab7-48a7-8b49-5e2903d6779f"}]': finished 2023-09-13T03:07:05.250 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:04 smithi142 ceph-mon[134082]: osdmap e22: 4 total, 3 up, 4 in 2023-09-13T03:07:05.250 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:04 smithi142 ceph-mon[134082]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-13T03:07:05.250 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:04 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-13T03:07:05.251 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:04 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-13T03:07:05.251 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:04 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T03:07:05.251 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:04 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T03:07:05.251 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:04 smithi142 ceph-mon[134082]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-13T03:07:05.251 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:04 smithi142 ceph-mon[134082]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-13T03:07:06.159 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:05 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/2635194378' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T03:07:06.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:05 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2635194378' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T03:07:06.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:05 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/2635194378' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T03:07:07.159 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:06 smithi170 ceph-mon[134328]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-13T03:07:07.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:06 smithi064 ceph-mon[97675]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-13T03:07:07.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:06 smithi142 ceph-mon[134082]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-13T03:07:09.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:09 smithi142 ceph-mon[134082]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-13T03:07:09.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:09 smithi170 ceph-mon[134328]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-13T03:07:09.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:09 smithi064 ceph-mon[97675]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-13T03:07:11.728 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:11 smithi064 ceph-mon[97675]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.6 KiB/s rd, 43 KiB/s wr, 4 op/s 2023-09-13T03:07:11.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:11 smithi142 ceph-mon[134082]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.6 KiB/s rd, 43 KiB/s wr, 4 op/s 2023-09-13T03:07:11.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:11 smithi170 ceph-mon[134328]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.6 KiB/s rd, 43 KiB/s wr, 4 op/s 2023-09-13T03:07:12.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:12 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-09-13T03:07:12.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:12 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:07:12.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:12 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-09-13T03:07:12.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:12 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:07:12.915 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:12 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-09-13T03:07:12.915 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:12 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:07:13.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:13 smithi142 ceph-mon[134082]: Deploying daemon osd.3 on smithi064 2023-09-13T03:07:13.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:13 smithi142 ceph-mon[134082]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 32 KiB/s wr, 3 op/s 2023-09-13T03:07:13.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:13 smithi170 ceph-mon[134328]: Deploying daemon osd.3 on smithi064 2023-09-13T03:07:13.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:13 smithi170 ceph-mon[134328]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 32 KiB/s wr, 3 op/s 2023-09-13T03:07:13.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:13 smithi064 ceph-mon[97675]: Deploying daemon osd.3 on smithi064 2023-09-13T03:07:13.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:13 smithi064 ceph-mon[97675]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 32 KiB/s wr, 3 op/s 2023-09-13T03:07:15.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:15 smithi170 ceph-mon[134328]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1017 B/s rd, 26 KiB/s wr, 2 op/s 2023-09-13T03:07:15.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:15 smithi064 ceph-mon[97675]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1017 B/s rd, 26 KiB/s wr, 2 op/s 2023-09-13T03:07:15.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:15 smithi142 ceph-mon[134082]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1017 B/s rd, 26 KiB/s wr, 2 op/s 2023-09-13T03:07:17.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:17 smithi142 ceph-mon[134082]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-09-13T03:07:17.787 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:17 smithi064 ceph-mon[97675]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-09-13T03:07:17.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:17 smithi170 ceph-mon[134328]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-09-13T03:07:19.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:19 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:19.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:19 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:19.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:19 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:07:19.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:19 smithi170 ceph-mon[134328]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-09-13T03:07:19.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:19 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:19.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:19 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:19.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:19 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:07:19.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:19 smithi064 ceph-mon[97675]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-09-13T03:07:19.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:19 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:19.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:19 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:19.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:19 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:07:19.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:19 smithi142 ceph-mon[134082]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-09-13T03:07:21.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:07:21.311+0000 7f38c27fc700 1 -- 172.21.15.64:0/3572790888 <== mgr.14150 v2:172.21.15.64:6800/873679308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f38cc060a40 con 0x7f38b80a0630 2023-09-13T03:07:21.312 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 3 on host 'smithi064' 2023-09-13T03:07:21.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:07:21.315+0000 7f38d327b700 1 -- 172.21.15.64:0/3572790888 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f38b80a0630 msgr2=0x7f38b80a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:07:21.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:07:21.315+0000 7f38d327b700 1 --2- 172.21.15.64:0/3572790888 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f38b80a0630 0x7f38b80a2af0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f38bc009fd0 tx=0x7f38bc009260 comp rx=0 tx=0).stop 2023-09-13T03:07:21.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:07:21.315+0000 7f38d327b700 1 -- 172.21.15.64:0/3572790888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38cc100690 msgr2=0x7f38cc0743a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:07:21.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:07:21.315+0000 7f38d327b700 1 --2- 172.21.15.64:0/3572790888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38cc100690 0x7f38cc0743a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f38c8009990 tx=0x7f38c800f690 comp rx=0 tx=0).stop 2023-09-13T03:07:21.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:07:21.316+0000 7f38d327b700 1 -- 172.21.15.64:0/3572790888 shutdown_connections 2023-09-13T03:07:21.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:07:21.316+0000 7f38d327b700 1 --2- 172.21.15.64:0/3572790888 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f38b80a0630 0x7f38b80a2af0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:07:21.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:07:21.316+0000 7f38d327b700 1 --2- 172.21.15.64:0/3572790888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38cc102230 0x7f38cc072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:07:21.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:07:21.316+0000 7f38d327b700 1 --2- 172.21.15.64:0/3572790888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38cc100690 0x7f38cc0743a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:07:21.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:07:21.316+0000 7f38d327b700 1 --2- 172.21.15.64:0/3572790888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38cc101890 0x7f38cc0729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:07:21.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:07:21.316+0000 7f38d327b700 1 -- 172.21.15.64:0/3572790888 >> 172.21.15.64:0/3572790888 conn(0x7f38cc0fbbf0 msgr2=0x7f38cc0fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:07:21.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:07:21.316+0000 7f38d327b700 1 -- 172.21.15.64:0/3572790888 shutdown_connections 2023-09-13T03:07:21.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:07:21.316+0000 7f38d327b700 1 -- 172.21.15.64:0/3572790888 wait complete. 2023-09-13T03:07:21.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:21 smithi064 ceph-mon[97675]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-09-13T03:07:21.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:21 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:21.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:21 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:21.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:21 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:07:21.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:21 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:07:21.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:21 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:21.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:21 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:21.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:21 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:21.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:21 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:07:21.741 DEBUG:teuthology.orchestra.run.smithi064:osd.3> sudo journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@osd.3.service 2023-09-13T03:07:21.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:21 smithi142 ceph-mon[134082]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-09-13T03:07:21.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:21 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:21.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:21 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:21.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:21 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:07:21.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:21 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:07:21.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:21 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:21.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:21 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:21.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:21 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:21.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:21 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:07:21.747 INFO:tasks.cephadm:Deploying osd.4 on smithi142 with /dev/vg_nvme/lv_3... 2023-09-13T03:07:21.748 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-09-13T03:07:21.819 INFO:journalctl@ceph.osd.3.smithi064.stdout:-- Logs begin at Wed 2023-09-13 02:46:56 UTC. -- 2023-09-13T03:07:21.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:21 smithi170 ceph-mon[134328]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-09-13T03:07:21.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:21 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:21.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:21 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:21.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:21 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:07:21.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:21 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:07:21.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:21 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:21.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:21 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:21.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:21 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:21.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:21 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:07:22.314 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-13T03:07:22.418 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 13 03:07:22 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-3[128330]: 2023-09-13T03:07:22.135+0000 7f864e485640 -1 osd.3 0 log_to_monitors true 2023-09-13T03:07:22.708 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:22 smithi064 ceph-mon[97675]: from='osd.3 [v2:172.21.15.64:6810/3558318707,v1:172.21.15.64:6811/3558318707]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-09-13T03:07:22.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:22 smithi142 ceph-mon[134082]: from='osd.3 [v2:172.21.15.64:6810/3558318707,v1:172.21.15.64:6811/3558318707]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-09-13T03:07:22.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:22 smithi170 ceph-mon[134328]: from='osd.3 [v2:172.21.15.64:6810/3558318707,v1:172.21.15.64:6811/3558318707]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-09-13T03:07:23.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:23 smithi064 ceph-mon[97675]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-09-13T03:07:23.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:23 smithi064 ceph-mon[97675]: from='osd.3 [v2:172.21.15.64:6810/3558318707,v1:172.21.15.64:6811/3558318707]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-09-13T03:07:23.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:23 smithi064 ceph-mon[97675]: osdmap e23: 4 total, 3 up, 4 in 2023-09-13T03:07:23.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:23 smithi064 ceph-mon[97675]: from='osd.3 [v2:172.21.15.64:6810/3558318707,v1:172.21.15.64:6811/3558318707]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-09-13T03:07:23.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:23 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-13T03:07:23.670 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 13 03:07:23 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-3[128330]: 2023-09-13T03:07:23.463+0000 7f86432fb700 -1 osd.3 0 waiting for initial osdmap 2023-09-13T03:07:23.670 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 13 03:07:23 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-3[128330]: 2023-09-13T03:07:23.471+0000 7f863f0f0700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-13T03:07:23.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:23 smithi142 ceph-mon[134082]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-09-13T03:07:23.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:23 smithi142 ceph-mon[134082]: from='osd.3 [v2:172.21.15.64:6810/3558318707,v1:172.21.15.64:6811/3558318707]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-09-13T03:07:23.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:23 smithi142 ceph-mon[134082]: osdmap e23: 4 total, 3 up, 4 in 2023-09-13T03:07:23.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:23 smithi142 ceph-mon[134082]: from='osd.3 [v2:172.21.15.64:6810/3558318707,v1:172.21.15.64:6811/3558318707]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-09-13T03:07:23.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:23 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-13T03:07:23.790 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-09-13T03:07:23.790 INFO:teuthology.orchestra.run.smithi142.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-09-13T03:07:23.790 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10+0 records in 2023-09-13T03:07:23.790 INFO:teuthology.orchestra.run.smithi142.stderr:10+0 records out 2023-09-13T03:07:23.791 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0109362 s, 959 MB/s 2023-09-13T03:07:23.791 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping successful for: 2023-09-13T03:07:23.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:23 smithi170 ceph-mon[134328]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-09-13T03:07:23.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:23 smithi170 ceph-mon[134328]: from='osd.3 [v2:172.21.15.64:6810/3558318707,v1:172.21.15.64:6811/3558318707]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-09-13T03:07:23.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:23 smithi170 ceph-mon[134328]: osdmap e23: 4 total, 3 up, 4 in 2023-09-13T03:07:23.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:23 smithi170 ceph-mon[134328]: from='osd.3 [v2:172.21.15.64:6810/3558318707,v1:172.21.15.64:6811/3558318707]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-09-13T03:07:23.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:23 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-13T03:07:24.272 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi142:vg_nvme/lv_3 2023-09-13T03:07:24.725 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:24 smithi064 ceph-mon[97675]: from='osd.3 [v2:172.21.15.64:6810/3558318707,v1:172.21.15.64:6811/3558318707]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2023-09-13T03:07:24.726 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:24 smithi064 ceph-mon[97675]: osdmap e24: 4 total, 3 up, 4 in 2023-09-13T03:07:24.726 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:24 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-13T03:07:24.726 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:24 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-13T03:07:24.726 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:24 smithi064 ceph-mon[97675]: osd.3 [v2:172.21.15.64:6810/3558318707,v1:172.21.15.64:6811/3558318707] boot 2023-09-13T03:07:24.726 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:24 smithi064 ceph-mon[97675]: osdmap e25: 4 total, 4 up, 4 in 2023-09-13T03:07:24.727 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:24 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-13T03:07:24.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:24 smithi142 ceph-mon[134082]: from='osd.3 [v2:172.21.15.64:6810/3558318707,v1:172.21.15.64:6811/3558318707]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2023-09-13T03:07:24.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:24 smithi142 ceph-mon[134082]: osdmap e24: 4 total, 3 up, 4 in 2023-09-13T03:07:24.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:24 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-13T03:07:24.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:24 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-13T03:07:24.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:24 smithi142 ceph-mon[134082]: osd.3 [v2:172.21.15.64:6810/3558318707,v1:172.21.15.64:6811/3558318707] boot 2023-09-13T03:07:24.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:24 smithi142 ceph-mon[134082]: osdmap e25: 4 total, 4 up, 4 in 2023-09-13T03:07:24.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:24 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-13T03:07:24.827 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-13T03:07:24.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:24 smithi170 ceph-mon[134328]: from='osd.3 [v2:172.21.15.64:6810/3558318707,v1:172.21.15.64:6811/3558318707]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2023-09-13T03:07:24.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:24 smithi170 ceph-mon[134328]: osdmap e24: 4 total, 3 up, 4 in 2023-09-13T03:07:24.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:24 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-13T03:07:24.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:24 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-13T03:07:24.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:24 smithi170 ceph-mon[134328]: osd.3 [v2:172.21.15.64:6810/3558318707,v1:172.21.15.64:6811/3558318707] boot 2023-09-13T03:07:24.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:24 smithi170 ceph-mon[134328]: osdmap e25: 4 total, 4 up, 4 in 2023-09-13T03:07:24.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:24 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-13T03:07:25.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:25 smithi142 ceph-mon[134082]: purged_snaps scrub starts 2023-09-13T03:07:25.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:25 smithi142 ceph-mon[134082]: purged_snaps scrub ok 2023-09-13T03:07:25.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:25 smithi142 ceph-mon[134082]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-09-13T03:07:25.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:25 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:25.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:25 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:25.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:25 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:07:25.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:25 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:07:25.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:25 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:25.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:25 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:07:25.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:25 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:07:25.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:25 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:25.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:25 smithi142 ceph-mon[134082]: osdmap e26: 4 total, 4 up, 4 in 2023-09-13T03:07:25.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:25 smithi170 ceph-mon[134328]: purged_snaps scrub starts 2023-09-13T03:07:25.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:25 smithi170 ceph-mon[134328]: purged_snaps scrub ok 2023-09-13T03:07:25.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:25 smithi170 ceph-mon[134328]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-09-13T03:07:25.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:25 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:25.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:25 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:25.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:25 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:07:25.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:25 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:07:25.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:25 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:25.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:25 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:07:25.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:25 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:07:25.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:25 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:25.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:25 smithi170 ceph-mon[134328]: osdmap e26: 4 total, 4 up, 4 in 2023-09-13T03:07:25.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:25 smithi064 ceph-mon[97675]: purged_snaps scrub starts 2023-09-13T03:07:25.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:25 smithi064 ceph-mon[97675]: purged_snaps scrub ok 2023-09-13T03:07:25.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:25 smithi064 ceph-mon[97675]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-09-13T03:07:25.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:25 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:25.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:25 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:25.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:25 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:07:25.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:25 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:07:25.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:25 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:25.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:25 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:07:25.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:25 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:07:25.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:25 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:25.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:25 smithi064 ceph-mon[97675]: osdmap e26: 4 total, 4 up, 4 in 2023-09-13T03:07:26.025 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:26.025+0000 7f216eb08700 1 -- 172.21.15.142:0/1389093550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2168102260 msgr2=0x7f2168108b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:07:26.026 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:26.025+0000 7f216eb08700 1 --2- 172.21.15.142:0/1389093550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2168102260 0x7f2168108b10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f216000b0d0 tx=0x7f216000b3e0 comp rx=0 tx=0).stop 2023-09-13T03:07:26.026 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:26.025+0000 7f216eb08700 1 -- 172.21.15.142:0/1389093550 shutdown_connections 2023-09-13T03:07:26.026 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:26.025+0000 7f216eb08700 1 --2- 172.21.15.142:0/1389093550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2168102260 0x7f2168108b10 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:07:26.026 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:26.025+0000 7f216eb08700 1 --2- 172.21.15.142:0/1389093550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21681006c0 0x7f2168100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:07:26.026 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:26.025+0000 7f216eb08700 1 --2- 172.21.15.142:0/1389093550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21681018c0 0x7f2168101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:07:26.027 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:26.025+0000 7f216eb08700 1 -- 172.21.15.142:0/1389093550 >> 172.21.15.142:0/1389093550 conn(0x7f21680fbc80 msgr2=0x7f21680fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:07:26.027 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:26.026+0000 7f216eb08700 1 -- 172.21.15.142:0/1389093550 shutdown_connections 2023-09-13T03:07:26.027 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:26.026+0000 7f216eb08700 1 -- 172.21.15.142:0/1389093550 wait complete. 2023-09-13T03:07:26.030 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:26.026+0000 7f216eb08700 1 Processor -- start 2023-09-13T03:07:26.030 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:26.026+0000 7f216eb08700 1 -- start start 2023-09-13T03:07:26.030 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:26.026+0000 7f216eb08700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21681006c0 0x7f2168196500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:07:26.030 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:26.026+0000 7f216eb08700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21681018c0 0x7f2168196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:07:26.030 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:26.026+0000 7f216eb08700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2168102260 0x7f216819dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:07:26.031 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:26.026+0000 7f216eb08700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216819e090 con 0x7f21681018c0 2023-09-13T03:07:26.031 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:26.026+0000 7f216eb08700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216819e1d0 con 0x7f2168102260 2023-09-13T03:07:26.031 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:26.026+0000 7f216eb08700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216819e360 con 0x7f21681006c0 2023-09-13T03:07:26.031 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:26.026+0000 7f216d0a5700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2168102260 0x7f216819dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:07:26.031 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:26.026+0000 7f216d0a5700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2168102260 0x7f216819dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:40562/0 (socket says 172.21.15.142:40562) 2023-09-13T03:07:26.032 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:26.026+0000 7f216d0a5700 1 -- 172.21.15.142:0/1778799342 learned_addr learned my addr 172.21.15.142:0/1778799342 (peer_addr_for_me v2:172.21.15.142:0/0) 2023-09-13T03:07:26.032 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:26.026+0000 7f216c8a4700 1 --2- 172.21.15.142:0/1778799342 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21681006c0 0x7f2168196500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:07:26.032 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:26.027+0000 7f2167fff700 1 --2- 172.21.15.142:0/1778799342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21681018c0 0x7f2168196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:07:26.032 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:26.027+0000 7f216d0a5700 1 -- 172.21.15.142:0/1778799342 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21681006c0 msgr2=0x7f2168196500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:07:26.032 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:26.027+0000 7f216d0a5700 1 --2- 172.21.15.142:0/1778799342 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21681006c0 0x7f2168196500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:07:26.033 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:26.027+0000 7f216d0a5700 1 -- 172.21.15.142:0/1778799342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21681018c0 msgr2=0x7f2168196a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:07:26.033 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:26.027+0000 7f216d0a5700 1 --2- 172.21.15.142:0/1778799342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21681018c0 0x7f2168196a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:07:26.033 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:26.027+0000 7f216d0a5700 1 -- 172.21.15.142:0/1778799342 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2160009d00 con 0x7f2168102260 2023-09-13T03:07:26.033 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:26.027+0000 7f216d0a5700 1 --2- 172.21.15.142:0/1778799342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2168102260 0x7f216819dac0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2160007a00 tx=0x7f2160012620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:07:26.033 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:26.027+0000 7f2165ffb700 1 -- 172.21.15.142:0/1778799342 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216000e040 con 0x7f2168102260 2023-09-13T03:07:26.034 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:26.027+0000 7f216eb08700 1 -- 172.21.15.142:0/1778799342 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f216819e5e0 con 0x7f2168102260 2023-09-13T03:07:26.034 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:26.027+0000 7f2165ffb700 1 -- 172.21.15.142:0/1778799342 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f216000ceb0 con 0x7f2168102260 2023-09-13T03:07:26.034 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:26.027+0000 7f2165ffb700 1 -- 172.21.15.142:0/1778799342 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216001bb30 con 0x7f2168102260 2023-09-13T03:07:26.034 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:26.027+0000 7f216eb08700 1 -- 172.21.15.142:0/1778799342 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f216819eb20 con 0x7f2168102260 2023-09-13T03:07:26.034 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:26.028+0000 7f216eb08700 1 -- 172.21.15.142:0/1778799342 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2168107c40 con 0x7f2168102260 2023-09-13T03:07:26.035 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:26.029+0000 7f2165ffb700 1 -- 172.21.15.142:0/1778799342 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f216001bc90 con 0x7f2168102260 2023-09-13T03:07:26.035 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:26.030+0000 7f2165ffb700 1 --2- 172.21.15.142:0/1778799342 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f21580a0640 0x7f21580a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:07:26.035 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:26.030+0000 7f2165ffb700 1 -- 172.21.15.142:0/1778799342 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f21600cc5a0 con 0x7f2168102260 2023-09-13T03:07:26.035 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:26.030+0000 7f216c8a4700 1 --2- 172.21.15.142:0/1778799342 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f21580a0640 0x7f21580a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:07:26.036 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:26.031+0000 7f216c8a4700 1 --2- 172.21.15.142:0/1778799342 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f21580a0640 0x7f21580a2b00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2168101720 tx=0x7f215c00a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:07:26.036 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:26.034+0000 7f2165ffb700 1 -- 172.21.15.142:0/1778799342 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f21600972c0 con 0x7f2168102260 2023-09-13T03:07:26.233 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:26.233+0000 7f216eb08700 1 -- 172.21.15.142:0/1778799342 --> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f216802cf50 con 0x7f21580a0640 2023-09-13T03:07:26.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:26 smithi142 ceph-mon[134082]: Detected new or changed devices on smithi064 2023-09-13T03:07:26.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:26 smithi142 ceph-mon[134082]: Adjusting osd_memory_target on smithi064 to 8633M 2023-09-13T03:07:26.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:26 smithi142 ceph-mon[134082]: osdmap e27: 4 total, 4 up, 4 in 2023-09-13T03:07:26.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:26 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T03:07:26.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:26 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T03:07:26.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:26 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:07:26.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:26 smithi170 ceph-mon[134328]: Detected new or changed devices on smithi064 2023-09-13T03:07:26.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:26 smithi170 ceph-mon[134328]: Adjusting osd_memory_target on smithi064 to 8633M 2023-09-13T03:07:26.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:26 smithi170 ceph-mon[134328]: osdmap e27: 4 total, 4 up, 4 in 2023-09-13T03:07:26.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:26 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T03:07:26.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:26 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T03:07:26.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:26 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:07:26.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:26 smithi064 ceph-mon[97675]: Detected new or changed devices on smithi064 2023-09-13T03:07:26.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:26 smithi064 ceph-mon[97675]: Adjusting osd_memory_target on smithi064 to 8633M 2023-09-13T03:07:26.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:26 smithi064 ceph-mon[97675]: osdmap e27: 4 total, 4 up, 4 in 2023-09-13T03:07:26.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:26 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T03:07:26.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:26 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T03:07:26.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:26 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:07:27.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:27 smithi142 ceph-mon[134082]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:07:27.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:27 smithi142 ceph-mon[134082]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-13T03:07:27.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:27 smithi142 ceph-mon[134082]: osdmap e28: 4 total, 4 up, 4 in 2023-09-13T03:07:27.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:27 smithi170 ceph-mon[134328]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:07:27.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:27 smithi170 ceph-mon[134328]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-13T03:07:27.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:27 smithi170 ceph-mon[134328]: osdmap e28: 4 total, 4 up, 4 in 2023-09-13T03:07:27.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:27 smithi064 ceph-mon[97675]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:07:27.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:27 smithi064 ceph-mon[97675]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-13T03:07:27.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:27 smithi064 ceph-mon[97675]: osdmap e28: 4 total, 4 up, 4 in 2023-09-13T03:07:29.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:29 smithi170 ceph-mon[134328]: osdmap e29: 4 total, 4 up, 4 in 2023-09-13T03:07:29.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:29 smithi170 ceph-mon[134328]: pgmap v112: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-13T03:07:29.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:29 smithi170 ceph-mon[134328]: from='client.? 172.21.15.142:0/3934661536' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c7a9c9f-f919-48d3-9db2-214a25bb5603"}]: dispatch 2023-09-13T03:07:29.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:29 smithi170 ceph-mon[134328]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c7a9c9f-f919-48d3-9db2-214a25bb5603"}]: dispatch 2023-09-13T03:07:29.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:29 smithi170 ceph-mon[134328]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8c7a9c9f-f919-48d3-9db2-214a25bb5603"}]': finished 2023-09-13T03:07:29.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:29 smithi170 ceph-mon[134328]: osdmap e30: 5 total, 4 up, 5 in 2023-09-13T03:07:29.412 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:29 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-13T03:07:29.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:29 smithi064 ceph-mon[97675]: osdmap e29: 4 total, 4 up, 4 in 2023-09-13T03:07:29.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:29 smithi064 ceph-mon[97675]: pgmap v112: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-13T03:07:29.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:29 smithi064 ceph-mon[97675]: from='client.? 172.21.15.142:0/3934661536' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c7a9c9f-f919-48d3-9db2-214a25bb5603"}]: dispatch 2023-09-13T03:07:29.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:29 smithi064 ceph-mon[97675]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c7a9c9f-f919-48d3-9db2-214a25bb5603"}]: dispatch 2023-09-13T03:07:29.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:29 smithi064 ceph-mon[97675]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8c7a9c9f-f919-48d3-9db2-214a25bb5603"}]': finished 2023-09-13T03:07:29.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:29 smithi064 ceph-mon[97675]: osdmap e30: 5 total, 4 up, 5 in 2023-09-13T03:07:29.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:29 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-13T03:07:29.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:29 smithi142 ceph-mon[134082]: osdmap e29: 4 total, 4 up, 4 in 2023-09-13T03:07:29.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:29 smithi142 ceph-mon[134082]: pgmap v112: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-13T03:07:29.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:29 smithi142 ceph-mon[134082]: from='client.? 172.21.15.142:0/3934661536' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c7a9c9f-f919-48d3-9db2-214a25bb5603"}]: dispatch 2023-09-13T03:07:29.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:29 smithi142 ceph-mon[134082]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c7a9c9f-f919-48d3-9db2-214a25bb5603"}]: dispatch 2023-09-13T03:07:29.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:29 smithi142 ceph-mon[134082]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8c7a9c9f-f919-48d3-9db2-214a25bb5603"}]': finished 2023-09-13T03:07:29.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:29 smithi142 ceph-mon[134082]: osdmap e30: 5 total, 4 up, 5 in 2023-09-13T03:07:29.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:29 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-13T03:07:30.409 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:30 smithi170 ceph-mon[134328]: from='client.? 172.21.15.142:0/1235113685' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T03:07:30.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:30 smithi064 ceph-mon[97675]: from='client.? 172.21.15.142:0/1235113685' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T03:07:30.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:30 smithi142 ceph-mon[134082]: from='client.? 172.21.15.142:0/1235113685' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T03:07:31.409 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:31 smithi170 ceph-mon[134328]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 135 KiB/s, 0 objects/s recovering 2023-09-13T03:07:31.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:31 smithi064 ceph-mon[97675]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 135 KiB/s, 0 objects/s recovering 2023-09-13T03:07:31.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:31 smithi142 ceph-mon[134082]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 135 KiB/s, 0 objects/s recovering 2023-09-13T03:07:33.704 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:33 smithi142 ceph-mon[134082]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-13T03:07:33.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:33 smithi170 ceph-mon[134328]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-13T03:07:33.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:33 smithi064 ceph-mon[97675]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-13T03:07:35.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:35 smithi142 ceph-mon[134082]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2023-09-13T03:07:35.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:35 smithi170 ceph-mon[134328]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2023-09-13T03:07:35.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:35 smithi064 ceph-mon[97675]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2023-09-13T03:07:36.233 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:36 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-09-13T03:07:36.234 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:36 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:07:36.234 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:36 smithi142 ceph-mon[134082]: Deploying daemon osd.4 on smithi142 2023-09-13T03:07:36.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:36 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-09-13T03:07:36.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:36 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:07:36.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:36 smithi170 ceph-mon[134328]: Deploying daemon osd.4 on smithi142 2023-09-13T03:07:36.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:36 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-09-13T03:07:36.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:36 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:07:36.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:36 smithi064 ceph-mon[97675]: Deploying daemon osd.4 on smithi142 2023-09-13T03:07:37.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:37 smithi170 ceph-mon[134328]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-09-13T03:07:37.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:37 smithi064 ceph-mon[97675]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-09-13T03:07:37.475 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:37 smithi142 ceph-mon[134082]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-09-13T03:07:39.714 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:39 smithi142 ceph-mon[134082]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-09-13T03:07:39.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:39 smithi170 ceph-mon[134328]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-09-13T03:07:39.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:39 smithi064 ceph-mon[97675]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-09-13T03:07:40.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:40 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:07:40.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:40 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:40.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:40 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:40.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:40 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:07:40.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:40 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:40.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:40 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:40.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:40 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:07:40.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:40 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:40.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:40 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:41.697 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:41 smithi142 ceph-mon[134082]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-09-13T03:07:41.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:41 smithi170 ceph-mon[134328]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-09-13T03:07:41.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:41 smithi064 ceph-mon[97675]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-09-13T03:07:42.597 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:42.597+0000 7f2165ffb700 1 -- 172.21.15.142:0/1778799342 <== mgr.14150 v2:172.21.15.64:6800/873679308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f216802cf50 con 0x7f21580a0640 2023-09-13T03:07:42.598 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 4 on host 'smithi142' 2023-09-13T03:07:42.600 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:42.599+0000 7f216eb08700 1 -- 172.21.15.142:0/1778799342 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f21580a0640 msgr2=0x7f21580a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:07:42.601 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:42.599+0000 7f216eb08700 1 --2- 172.21.15.142:0/1778799342 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f21580a0640 0x7f21580a2b00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2168101720 tx=0x7f215c00a380 comp rx=0 tx=0).stop 2023-09-13T03:07:42.601 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:42.599+0000 7f216eb08700 1 -- 172.21.15.142:0/1778799342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2168102260 msgr2=0x7f216819dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:07:42.601 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:42.599+0000 7f216eb08700 1 --2- 172.21.15.142:0/1778799342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2168102260 0x7f216819dac0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2160007a00 tx=0x7f2160012620 comp rx=0 tx=0).stop 2023-09-13T03:07:42.601 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:42.599+0000 7f216eb08700 1 -- 172.21.15.142:0/1778799342 shutdown_connections 2023-09-13T03:07:42.601 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:42.599+0000 7f216eb08700 1 --2- 172.21.15.142:0/1778799342 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f21580a0640 0x7f21580a2b00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:07:42.602 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:42.599+0000 7f216eb08700 1 --2- 172.21.15.142:0/1778799342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2168102260 0x7f216819dac0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:07:42.602 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:42.599+0000 7f216eb08700 1 --2- 172.21.15.142:0/1778799342 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21681006c0 0x7f2168196500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:07:42.602 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:42.599+0000 7f216eb08700 1 --2- 172.21.15.142:0/1778799342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21681018c0 0x7f2168196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:07:42.602 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:42.599+0000 7f216eb08700 1 -- 172.21.15.142:0/1778799342 >> 172.21.15.142:0/1778799342 conn(0x7f21680fbc80 msgr2=0x7f2168104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:07:42.602 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:42.599+0000 7f216eb08700 1 -- 172.21.15.142:0/1778799342 shutdown_connections 2023-09-13T03:07:42.603 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:07:42.599+0000 7f216eb08700 1 -- 172.21.15.142:0/1778799342 wait complete. 2023-09-13T03:07:43.019 DEBUG:teuthology.orchestra.run.smithi142:osd.4> sudo journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@osd.4.service 2023-09-13T03:07:43.023 INFO:tasks.cephadm:Deploying osd.5 on smithi170 with /dev/vg_nvme/lv_3... 2023-09-13T03:07:43.023 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-09-13T03:07:43.065 INFO:journalctl@ceph.osd.4.smithi142.stdout:-- Logs begin at Wed 2023-09-13 02:43:02 UTC. -- 2023-09-13T03:07:43.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:43 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:43.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:43 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:43.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:43 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:07:43.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:43 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:07:43.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:43 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:43.412 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:43 smithi170 ceph-mon[134328]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-13T03:07:43.412 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:43 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:43.413 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:43 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:43.413 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:43 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:07:43.414 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:43 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:43.414 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:43 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:43.414 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:43 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:07:43.415 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:43 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:07:43.415 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:43 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:43.415 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:43 smithi142 ceph-mon[134082]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-13T03:07:43.416 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:43 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:43.416 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:43 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:43.416 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:43 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:07:43.608 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-13T03:07:43.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:43 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:43.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:43 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:43.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:43 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:07:43.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:43 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:07:43.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:43 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:43.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:43 smithi064 ceph-mon[97675]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-13T03:07:43.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:43 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:43.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:43 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:43.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:43 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:07:43.995 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 13 03:07:43 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-4[147323]: 2023-09-13T03:07:43.687+0000 7f6d4eed8640 -1 osd.4 0 log_to_monitors true 2023-09-13T03:07:44.314 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:44 smithi142 ceph-mon[134082]: from='osd.4 [v2:172.21.15.142:6808/1811142576,v1:172.21.15.142:6809/1811142576]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-13T03:07:44.314 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:44 smithi142 ceph-mon[134082]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-13T03:07:44.409 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:44 smithi170 ceph-mon[134328]: from='osd.4 [v2:172.21.15.142:6808/1811142576,v1:172.21.15.142:6809/1811142576]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-13T03:07:44.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:44 smithi170 ceph-mon[134328]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-13T03:07:44.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:44 smithi064 ceph-mon[97675]: from='osd.4 [v2:172.21.15.142:6808/1811142576,v1:172.21.15.142:6809/1811142576]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-13T03:07:44.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:44 smithi064 ceph-mon[97675]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-13T03:07:45.059 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-09-13T03:07:45.059 INFO:teuthology.orchestra.run.smithi170.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-09-13T03:07:45.059 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10+0 records in 2023-09-13T03:07:45.060 INFO:teuthology.orchestra.run.smithi170.stderr:10+0 records out 2023-09-13T03:07:45.060 INFO:teuthology.orchestra.run.smithi170.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0123944 s, 846 MB/s 2023-09-13T03:07:45.060 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping successful for: 2023-09-13T03:07:45.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:45 smithi170 ceph-mon[134328]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-09-13T03:07:45.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:45 smithi170 ceph-mon[134328]: osdmap e31: 5 total, 4 up, 5 in 2023-09-13T03:07:45.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:45 smithi170 ceph-mon[134328]: from='osd.4 [v2:172.21.15.142:6808/1811142576,v1:172.21.15.142:6809/1811142576]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-09-13T03:07:45.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:45 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-13T03:07:45.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:45 smithi170 ceph-mon[134328]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-09-13T03:07:45.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:45 smithi170 ceph-mon[134328]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-13T03:07:45.479 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 13 03:07:45 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-4[147323]: 2023-09-13T03:07:45.268+0000 7f6d43d4e700 -1 osd.4 0 waiting for initial osdmap 2023-09-13T03:07:45.480 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 13 03:07:45 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-4[147323]: 2023-09-13T03:07:45.275+0000 7f6d3eb41700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-13T03:07:45.480 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:45 smithi142 ceph-mon[134082]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-09-13T03:07:45.480 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:45 smithi142 ceph-mon[134082]: osdmap e31: 5 total, 4 up, 5 in 2023-09-13T03:07:45.481 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:45 smithi142 ceph-mon[134082]: from='osd.4 [v2:172.21.15.142:6808/1811142576,v1:172.21.15.142:6809/1811142576]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-09-13T03:07:45.481 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:45 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-13T03:07:45.481 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:45 smithi142 ceph-mon[134082]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-09-13T03:07:45.481 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:45 smithi142 ceph-mon[134082]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-13T03:07:45.558 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi170:vg_nvme/lv_3 2023-09-13T03:07:45.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:45 smithi064 ceph-mon[97675]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-09-13T03:07:45.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:45 smithi064 ceph-mon[97675]: osdmap e31: 5 total, 4 up, 5 in 2023-09-13T03:07:45.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:45 smithi064 ceph-mon[97675]: from='osd.4 [v2:172.21.15.142:6808/1811142576,v1:172.21.15.142:6809/1811142576]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-09-13T03:07:45.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:45 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-13T03:07:45.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:45 smithi064 ceph-mon[97675]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-09-13T03:07:45.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:45 smithi064 ceph-mon[97675]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-13T03:07:46.119 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-13T03:07:46.621 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:46 smithi170 ceph-mon[134328]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2023-09-13T03:07:46.621 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:46 smithi170 ceph-mon[134328]: osdmap e32: 5 total, 4 up, 5 in 2023-09-13T03:07:46.621 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:46 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-13T03:07:46.621 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:46 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-13T03:07:46.622 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:46 smithi170 ceph-mon[134328]: Detected new or changed devices on smithi142 2023-09-13T03:07:46.622 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:46 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:46.622 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:46 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:46.622 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:46 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:07:46.623 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:46 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:07:46.623 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:46 smithi170 ceph-mon[134328]: Adjusting osd_memory_target on smithi142 to 8584M 2023-09-13T03:07:46.623 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:46 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:46.623 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:46 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:07:46.623 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:46 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:07:46.623 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:46 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:46.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:46 smithi064 ceph-mon[97675]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2023-09-13T03:07:46.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:46 smithi064 ceph-mon[97675]: osdmap e32: 5 total, 4 up, 5 in 2023-09-13T03:07:46.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:46 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-13T03:07:46.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:46 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-13T03:07:46.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:46 smithi064 ceph-mon[97675]: Detected new or changed devices on smithi142 2023-09-13T03:07:46.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:46 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:46.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:46 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:46.671 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:46 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:07:46.671 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:46 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:07:46.671 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:46 smithi064 ceph-mon[97675]: Adjusting osd_memory_target on smithi142 to 8584M 2023-09-13T03:07:46.671 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:46 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:46.672 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:46 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:07:46.672 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:46 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:07:46.672 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:46 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:46.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:46 smithi142 ceph-mon[134082]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2023-09-13T03:07:46.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:46 smithi142 ceph-mon[134082]: osdmap e32: 5 total, 4 up, 5 in 2023-09-13T03:07:46.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:46 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-13T03:07:46.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:46 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-13T03:07:46.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:46 smithi142 ceph-mon[134082]: Detected new or changed devices on smithi142 2023-09-13T03:07:46.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:46 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:46.748 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:46 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:46.748 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:46 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:07:46.749 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:46 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:07:46.749 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:46 smithi142 ceph-mon[134082]: Adjusting osd_memory_target on smithi142 to 8584M 2023-09-13T03:07:46.749 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:46 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:46.750 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:46 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:07:46.750 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:46 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:07:46.750 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:46 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:07:47.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:07:47.298+0000 7fe63bf47700 1 -- 172.21.15.170:0/1753086279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe634101160 msgr2=0x7fe634107a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:07:47.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:07:47.298+0000 7fe63bf47700 1 --2- 172.21.15.170:0/1753086279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe634101160 0x7fe634107a10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe62c00b0a0 tx=0x7fe62c00b3b0 comp rx=0 tx=0).stop 2023-09-13T03:07:47.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:07:47.298+0000 7fe63bf47700 1 -- 172.21.15.170:0/1753086279 shutdown_connections 2023-09-13T03:07:47.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:07:47.298+0000 7fe63bf47700 1 --2- 172.21.15.170:0/1753086279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6340ff5c0 0x7fe6340ff9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:07:47.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:07:47.298+0000 7fe63bf47700 1 --2- 172.21.15.170:0/1753086279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe634101160 0x7fe634107a10 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:07:47.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:07:47.298+0000 7fe63bf47700 1 --2- 172.21.15.170:0/1753086279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6341007c0 0x7fe634100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:07:47.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:07:47.298+0000 7fe63bf47700 1 -- 172.21.15.170:0/1753086279 >> 172.21.15.170:0/1753086279 conn(0x7fe6340fab80 msgr2=0x7fe6340fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:07:47.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:07:47.298+0000 7fe63bf47700 1 -- 172.21.15.170:0/1753086279 shutdown_connections 2023-09-13T03:07:47.301 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:07:47.298+0000 7fe63bf47700 1 -- 172.21.15.170:0/1753086279 wait complete. 2023-09-13T03:07:47.303 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:07:47.299+0000 7fe63bf47700 1 Processor -- start 2023-09-13T03:07:47.303 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:07:47.299+0000 7fe63bf47700 1 -- start start 2023-09-13T03:07:47.303 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:07:47.299+0000 7fe63bf47700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6340ff5c0 0x7fe634198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:07:47.303 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:07:47.299+0000 7fe63bf47700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6341007c0 0x7fe634198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:07:47.303 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:07:47.299+0000 7fe63bf47700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe634101160 0x7fe63419fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:07:47.304 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:07:47.299+0000 7fe63bf47700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6341a03d0 con 0x7fe6340ff5c0 2023-09-13T03:07:47.304 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:07:47.299+0000 7fe63bf47700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6341a0510 con 0x7fe634101160 2023-09-13T03:07:47.304 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:07:47.299+0000 7fe63bf47700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6341a06a0 con 0x7fe6341007c0 2023-09-13T03:07:47.304 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:07:47.299+0000 7fe6394e2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6341007c0 0x7fe634198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:07:47.304 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:07:47.299+0000 7fe6394e2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6341007c0 0x7fe634198d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45504/0 (socket says 172.21.15.170:45504) 2023-09-13T03:07:47.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:07:47.299+0000 7fe6394e2700 1 -- 172.21.15.170:0/3033687765 learned_addr learned my addr 172.21.15.170:0/3033687765 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-09-13T03:07:47.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:07:47.299+0000 7fe63a4e4700 1 --2- 172.21.15.170:0/3033687765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe634101160 0x7fe63419fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:07:47.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:07:47.299+0000 7fe6394e2700 1 -- 172.21.15.170:0/3033687765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe634101160 msgr2=0x7fe63419fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:07:47.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:07:47.299+0000 7fe6394e2700 1 --2- 172.21.15.170:0/3033687765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe634101160 0x7fe63419fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:07:47.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:07:47.299+0000 7fe6394e2700 1 -- 172.21.15.170:0/3033687765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6340ff5c0 msgr2=0x7fe634198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:07:47.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:07:47.299+0000 7fe6394e2700 1 --2- 172.21.15.170:0/3033687765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6340ff5c0 0x7fe634198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:07:47.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:07:47.299+0000 7fe6394e2700 1 -- 172.21.15.170:0/3033687765 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe62c009d00 con 0x7fe6341007c0 2023-09-13T03:07:47.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:07:47.299+0000 7fe6394e2700 1 --2- 172.21.15.170:0/3033687765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6341007c0 0x7fe634198d80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe63000b560 tx=0x7fe63000b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:07:47.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:07:47.300+0000 7fe62affd700 1 -- 172.21.15.170:0/3033687765 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe630011830 con 0x7fe6341007c0 2023-09-13T03:07:47.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:07:47.300+0000 7fe62affd700 1 -- 172.21.15.170:0/3033687765 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe630011e50 con 0x7fe6341007c0 2023-09-13T03:07:47.307 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:07:47.300+0000 7fe63bf47700 1 -- 172.21.15.170:0/3033687765 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6341a0980 con 0x7fe6341007c0 2023-09-13T03:07:47.307 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:07:47.300+0000 7fe62affd700 1 -- 172.21.15.170:0/3033687765 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe63000f2e0 con 0x7fe6341007c0 2023-09-13T03:07:47.307 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:07:47.300+0000 7fe63bf47700 1 -- 172.21.15.170:0/3033687765 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6341a0f00 con 0x7fe6341007c0 2023-09-13T03:07:47.307 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:07:47.300+0000 7fe63bf47700 1 -- 172.21.15.170:0/3033687765 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe63410af30 con 0x7fe6341007c0 2023-09-13T03:07:47.307 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:07:47.301+0000 7fe62affd700 1 -- 172.21.15.170:0/3033687765 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fe630010580 con 0x7fe6341007c0 2023-09-13T03:07:47.307 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:07:47.302+0000 7fe62affd700 1 --2- 172.21.15.170:0/3033687765 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe6200a0640 0x7fe6200a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:07:47.308 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:07:47.302+0000 7fe62affd700 1 -- 172.21.15.170:0/3033687765 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4101+0+0 (secure 0 0 0) 0x7fe6300ca600 con 0x7fe6341007c0 2023-09-13T03:07:47.308 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:07:47.302+0000 7fe639ce3700 1 --2- 172.21.15.170:0/3033687765 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe6200a0640 0x7fe6200a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:07:47.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:07:47.303+0000 7fe639ce3700 1 --2- 172.21.15.170:0/3033687765 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe6200a0640 0x7fe6200a2b00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe634100620 tx=0x7fe624006d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:07:47.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:07:47.306+0000 7fe62affd700 1 -- 172.21.15.170:0/3033687765 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe6300951a0 con 0x7fe6341007c0 2023-09-13T03:07:47.409 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:47 smithi170 ceph-mon[134328]: purged_snaps scrub starts 2023-09-13T03:07:47.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:47 smithi170 ceph-mon[134328]: purged_snaps scrub ok 2023-09-13T03:07:47.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:47 smithi170 ceph-mon[134328]: osd.4 [v2:172.21.15.142:6808/1811142576,v1:172.21.15.142:6809/1811142576] boot 2023-09-13T03:07:47.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:47 smithi170 ceph-mon[134328]: osdmap e33: 5 total, 5 up, 5 in 2023-09-13T03:07:47.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:47 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-13T03:07:47.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:47 smithi170 ceph-mon[134328]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-13T03:07:47.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:07:47.504+0000 7fe63bf47700 1 -- 172.21.15.170:0/3033687765 --> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fe6340691b0 con 0x7fe6200a0640 2023-09-13T03:07:47.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:47 smithi064 ceph-mon[97675]: purged_snaps scrub starts 2023-09-13T03:07:47.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:47 smithi064 ceph-mon[97675]: purged_snaps scrub ok 2023-09-13T03:07:47.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:47 smithi064 ceph-mon[97675]: osd.4 [v2:172.21.15.142:6808/1811142576,v1:172.21.15.142:6809/1811142576] boot 2023-09-13T03:07:47.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:47 smithi064 ceph-mon[97675]: osdmap e33: 5 total, 5 up, 5 in 2023-09-13T03:07:47.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:47 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-13T03:07:47.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:47 smithi064 ceph-mon[97675]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-13T03:07:47.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:47 smithi142 ceph-mon[134082]: purged_snaps scrub starts 2023-09-13T03:07:47.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:47 smithi142 ceph-mon[134082]: purged_snaps scrub ok 2023-09-13T03:07:47.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:47 smithi142 ceph-mon[134082]: osd.4 [v2:172.21.15.142:6808/1811142576,v1:172.21.15.142:6809/1811142576] boot 2023-09-13T03:07:47.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:47 smithi142 ceph-mon[134082]: osdmap e33: 5 total, 5 up, 5 in 2023-09-13T03:07:47.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:47 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-13T03:07:47.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:47 smithi142 ceph-mon[134082]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-13T03:07:48.587 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:48 smithi170 ceph-mon[134328]: osdmap e34: 5 total, 5 up, 5 in 2023-09-13T03:07:48.588 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:48 smithi170 ceph-mon[134328]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:07:48.588 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:48 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T03:07:48.588 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:48 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T03:07:48.588 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:48 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:07:48.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:48 smithi064 ceph-mon[97675]: osdmap e34: 5 total, 5 up, 5 in 2023-09-13T03:07:48.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:48 smithi064 ceph-mon[97675]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:07:48.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:48 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T03:07:48.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:48 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T03:07:48.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:48 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:07:48.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:48 smithi142 ceph-mon[134082]: osdmap e34: 5 total, 5 up, 5 in 2023-09-13T03:07:48.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:48 smithi142 ceph-mon[134082]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:07:48.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:48 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T03:07:48.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:48 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T03:07:48.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:48 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:07:49.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:49 smithi170 ceph-mon[134328]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T03:07:49.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:49 smithi064 ceph-mon[97675]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T03:07:49.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:49 smithi142 ceph-mon[134082]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T03:07:50.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:50 smithi170 ceph-mon[134328]: from='client.? 172.21.15.170:0/12123145' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c6101c6-2065-4022-a751-d3f216d7f478"}]: dispatch 2023-09-13T03:07:50.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:50 smithi170 ceph-mon[134328]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c6101c6-2065-4022-a751-d3f216d7f478"}]: dispatch 2023-09-13T03:07:50.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:50 smithi170 ceph-mon[134328]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c6101c6-2065-4022-a751-d3f216d7f478"}]': finished 2023-09-13T03:07:50.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:50 smithi170 ceph-mon[134328]: osdmap e35: 6 total, 5 up, 6 in 2023-09-13T03:07:50.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:50 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-13T03:07:50.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:50 smithi064 ceph-mon[97675]: from='client.? 172.21.15.170:0/12123145' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c6101c6-2065-4022-a751-d3f216d7f478"}]: dispatch 2023-09-13T03:07:50.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:50 smithi064 ceph-mon[97675]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c6101c6-2065-4022-a751-d3f216d7f478"}]: dispatch 2023-09-13T03:07:50.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:50 smithi064 ceph-mon[97675]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c6101c6-2065-4022-a751-d3f216d7f478"}]': finished 2023-09-13T03:07:50.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:50 smithi064 ceph-mon[97675]: osdmap e35: 6 total, 5 up, 6 in 2023-09-13T03:07:50.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:50 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-13T03:07:50.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:50 smithi142 ceph-mon[134082]: from='client.? 172.21.15.170:0/12123145' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c6101c6-2065-4022-a751-d3f216d7f478"}]: dispatch 2023-09-13T03:07:50.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:50 smithi142 ceph-mon[134082]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c6101c6-2065-4022-a751-d3f216d7f478"}]: dispatch 2023-09-13T03:07:50.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:50 smithi142 ceph-mon[134082]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c6101c6-2065-4022-a751-d3f216d7f478"}]': finished 2023-09-13T03:07:50.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:50 smithi142 ceph-mon[134082]: osdmap e35: 6 total, 5 up, 6 in 2023-09-13T03:07:50.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:50 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-13T03:07:51.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:51 smithi170 ceph-mon[134328]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T03:07:51.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:51 smithi170 ceph-mon[134328]: from='client.? 172.21.15.170:0/706222051' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T03:07:51.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:51 smithi064 ceph-mon[97675]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T03:07:51.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:51 smithi064 ceph-mon[97675]: from='client.? 172.21.15.170:0/706222051' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T03:07:51.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:51 smithi142 ceph-mon[134082]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T03:07:51.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:51 smithi142 ceph-mon[134082]: from='client.? 172.21.15.170:0/706222051' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T03:07:53.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:53 smithi142 ceph-mon[134082]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T03:07:53.823 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:53 smithi170 ceph-mon[134328]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T03:07:53.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:53 smithi064 ceph-mon[97675]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T03:07:55.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:55 smithi170 ceph-mon[134328]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T03:07:55.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:55 smithi064 ceph-mon[97675]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T03:07:55.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:55 smithi142 ceph-mon[134082]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T03:07:56.659 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:56 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-09-13T03:07:56.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:56 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:07:56.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:56 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-09-13T03:07:56.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:56 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:07:56.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:56 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-09-13T03:07:56.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:56 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:07:57.659 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:57 smithi170 ceph-mon[134328]: Deploying daemon osd.5 on smithi170 2023-09-13T03:07:57.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:57 smithi170 ceph-mon[134328]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T03:07:57.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:57 smithi064 ceph-mon[97675]: Deploying daemon osd.5 on smithi170 2023-09-13T03:07:57.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:57 smithi064 ceph-mon[97675]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T03:07:57.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:57 smithi142 ceph-mon[134082]: Deploying daemon osd.5 on smithi170 2023-09-13T03:07:57.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:57 smithi142 ceph-mon[134082]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T03:07:59.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:07:59 smithi170 ceph-mon[134328]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T03:07:59.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:07:59 smithi142 ceph-mon[134082]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T03:07:59.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:07:59 smithi064 ceph-mon[97675]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T03:08:01.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:01 smithi142 ceph-mon[134082]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T03:08:01.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:01 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:01.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:01 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:08:01.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:01 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:01.791 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:01 smithi170 ceph-mon[134328]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T03:08:01.791 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:01 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:01.791 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:01 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:08:01.791 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:01 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:01.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:01 smithi064 ceph-mon[97675]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T03:08:01.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:01 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:01.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:01 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:08:01.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:01 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:03.422 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 5 on host 'smithi170' 2023-09-13T03:08:03.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:03.421+0000 7fe62affd700 1 -- 172.21.15.170:0/3033687765 <== mgr.14150 v2:172.21.15.64:6800/873679308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe6340691b0 con 0x7fe6200a0640 2023-09-13T03:08:03.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:03.424+0000 7fe63bf47700 1 -- 172.21.15.170:0/3033687765 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe6200a0640 msgr2=0x7fe6200a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:08:03.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:03.424+0000 7fe63bf47700 1 --2- 172.21.15.170:0/3033687765 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe6200a0640 0x7fe6200a2b00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe634100620 tx=0x7fe624006d20 comp rx=0 tx=0).stop 2023-09-13T03:08:03.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:03.424+0000 7fe63bf47700 1 -- 172.21.15.170:0/3033687765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6341007c0 msgr2=0x7fe634198d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:08:03.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:03.424+0000 7fe63bf47700 1 --2- 172.21.15.170:0/3033687765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6341007c0 0x7fe634198d80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe63000b560 tx=0x7fe63000b870 comp rx=0 tx=0).stop 2023-09-13T03:08:03.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:03.424+0000 7fe63bf47700 1 -- 172.21.15.170:0/3033687765 shutdown_connections 2023-09-13T03:08:03.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:03.424+0000 7fe63bf47700 1 --2- 172.21.15.170:0/3033687765 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe6200a0640 0x7fe6200a2b00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:08:03.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:03.424+0000 7fe63bf47700 1 --2- 172.21.15.170:0/3033687765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe634101160 0x7fe63419fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:08:03.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:03.424+0000 7fe63bf47700 1 --2- 172.21.15.170:0/3033687765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6340ff5c0 0x7fe634198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:08:03.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:03.424+0000 7fe63bf47700 1 --2- 172.21.15.170:0/3033687765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6341007c0 0x7fe634198d80 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:08:03.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:03.424+0000 7fe63bf47700 1 -- 172.21.15.170:0/3033687765 >> 172.21.15.170:0/3033687765 conn(0x7fe6340fab80 msgr2=0x7fe634068aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:08:03.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:03.424+0000 7fe63bf47700 1 -- 172.21.15.170:0/3033687765 shutdown_connections 2023-09-13T03:08:03.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:03.424+0000 7fe63bf47700 1 -- 172.21.15.170:0/3033687765 wait complete. 2023-09-13T03:08:03.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:03 smithi170 ceph-mon[134328]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T03:08:03.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:03 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:03.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:03 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:03.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:03 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:03.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:03 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:03.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:03 smithi142 ceph-mon[134082]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T03:08:03.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:03 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:03.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:03 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:03.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:03 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:03.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:03 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:03.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:03 smithi064 ceph-mon[97675]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T03:08:03.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:03 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:03.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:03 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:03.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:03 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:03.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:03 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:03.920 DEBUG:teuthology.orchestra.run.smithi170:osd.5> sudo journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@osd.5.service 2023-09-13T03:08:03.922 INFO:tasks.cephadm:Deploying osd.6 on smithi064 with /dev/vg_nvme/lv_2... 2023-09-13T03:08:03.923 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-09-13T03:08:03.953 INFO:journalctl@ceph.osd.5.smithi170.stdout:-- Logs begin at Wed 2023-09-13 02:42:51 UTC. -- 2023-09-13T03:08:04.659 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 13 03:08:04 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-5[148087]: 2023-09-13T03:08:04.609+0000 7f1a575f0640 -1 osd.5 0 log_to_monitors true 2023-09-13T03:08:04.684 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:08:05.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:05 smithi064 ceph-mon[97675]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T03:08:05.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:05 smithi064 ceph-mon[97675]: from='osd.5 [v2:172.21.15.170:6808/2903841453,v1:172.21.15.170:6809/2903841453]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-13T03:08:05.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:05 smithi064 ceph-mon[97675]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-13T03:08:05.409 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:05 smithi170 ceph-mon[134328]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T03:08:05.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:05 smithi170 ceph-mon[134328]: from='osd.5 [v2:172.21.15.170:6808/2903841453,v1:172.21.15.170:6809/2903841453]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-13T03:08:05.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:05 smithi170 ceph-mon[134328]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-13T03:08:05.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:05 smithi142 ceph-mon[134082]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T03:08:05.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:05 smithi142 ceph-mon[134082]: from='osd.5 [v2:172.21.15.170:6808/2903841453,v1:172.21.15.170:6809/2903841453]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-13T03:08:05.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:05 smithi142 ceph-mon[134082]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-13T03:08:06.410 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 13 03:08:06 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-5[148087]: 2023-09-13T03:08:06.155+0000 7f1a4c466700 -1 osd.5 0 waiting for initial osdmap 2023-09-13T03:08:06.410 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 13 03:08:06 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-5[148087]: 2023-09-13T03:08:06.163+0000 7f1a45a56700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-13T03:08:06.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:06 smithi170 ceph-mon[134328]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-09-13T03:08:06.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:06 smithi170 ceph-mon[134328]: from='osd.5 [v2:172.21.15.170:6808/2903841453,v1:172.21.15.170:6809/2903841453]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-13T03:08:06.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:06 smithi170 ceph-mon[134328]: osdmap e36: 6 total, 5 up, 6 in 2023-09-13T03:08:06.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:06 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-13T03:08:06.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:06 smithi170 ceph-mon[134328]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-13T03:08:06.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:06 smithi064 ceph-mon[97675]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-09-13T03:08:06.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:06 smithi064 ceph-mon[97675]: from='osd.5 [v2:172.21.15.170:6808/2903841453,v1:172.21.15.170:6809/2903841453]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-13T03:08:06.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:06 smithi064 ceph-mon[97675]: osdmap e36: 6 total, 5 up, 6 in 2023-09-13T03:08:06.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:06 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-13T03:08:06.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:06 smithi064 ceph-mon[97675]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-13T03:08:06.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:06 smithi142 ceph-mon[134082]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-09-13T03:08:06.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:06 smithi142 ceph-mon[134082]: from='osd.5 [v2:172.21.15.170:6808/2903841453,v1:172.21.15.170:6809/2903841453]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-13T03:08:06.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:06 smithi142 ceph-mon[134082]: osdmap e36: 6 total, 5 up, 6 in 2023-09-13T03:08:06.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:06 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-13T03:08:06.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:06 smithi142 ceph-mon[134082]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-13T03:08:06.791 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-09-13T03:08:06.791 INFO:teuthology.orchestra.run.smithi064.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-09-13T03:08:06.792 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10+0 records in 2023-09-13T03:08:06.792 INFO:teuthology.orchestra.run.smithi064.stderr:10+0 records out 2023-09-13T03:08:06.792 INFO:teuthology.orchestra.run.smithi064.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.04333 s, 242 MB/s 2023-09-13T03:08:06.793 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 2023-09-13T03:08:06.793 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping successful for: 2023-09-13T03:08:07.159 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi064:vg_nvme/lv_2 2023-09-13T03:08:07.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:07 smithi170 ceph-mon[134328]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2023-09-13T03:08:07.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:07 smithi170 ceph-mon[134328]: osdmap e37: 6 total, 5 up, 6 in 2023-09-13T03:08:07.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:07 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-13T03:08:07.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:07 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-13T03:08:07.412 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:07 smithi170 ceph-mon[134328]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T03:08:07.412 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:07 smithi170 ceph-mon[134328]: Detected new or changed devices on smithi170 2023-09-13T03:08:07.413 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:07 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:07.413 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:07 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:07.413 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:07 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:08:07.414 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:07 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:08:07.414 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:07 smithi170 ceph-mon[134328]: Adjusting osd_memory_target on smithi170 to 8584M 2023-09-13T03:08:07.414 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:07 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:07.415 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:07 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:08:07.415 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:07 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:08:07.416 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:07 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:07.416 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:07 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:08:07.416 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:07 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:08:07.417 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:07 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:08:07.417 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:07 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:07.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:07 smithi064 ceph-mon[97675]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2023-09-13T03:08:07.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:07 smithi064 ceph-mon[97675]: osdmap e37: 6 total, 5 up, 6 in 2023-09-13T03:08:07.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:07 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-13T03:08:07.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:07 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-13T03:08:07.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:07 smithi064 ceph-mon[97675]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T03:08:07.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:07 smithi064 ceph-mon[97675]: Detected new or changed devices on smithi170 2023-09-13T03:08:07.421 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:07 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:07.421 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:07 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:07.421 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:07 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:08:07.422 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:07 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:08:07.422 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:07 smithi064 ceph-mon[97675]: Adjusting osd_memory_target on smithi170 to 8584M 2023-09-13T03:08:07.422 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:07 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:07.423 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:07 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:08:07.423 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:07 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:08:07.423 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:07 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:07.424 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:07 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:08:07.424 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:07 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:08:07.424 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:07 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:08:07.424 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:07 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:07.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:07 smithi142 ceph-mon[134082]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2023-09-13T03:08:07.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:07 smithi142 ceph-mon[134082]: osdmap e37: 6 total, 5 up, 6 in 2023-09-13T03:08:07.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:07 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-13T03:08:07.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:07 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-13T03:08:07.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:07 smithi142 ceph-mon[134082]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-13T03:08:07.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:07 smithi142 ceph-mon[134082]: Detected new or changed devices on smithi170 2023-09-13T03:08:07.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:07 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:07.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:07 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:07.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:07 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:08:07.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:07 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:08:07.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:07 smithi142 ceph-mon[134082]: Adjusting osd_memory_target on smithi170 to 8584M 2023-09-13T03:08:07.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:07 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:07.498 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:07 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:08:07.498 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:07 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:08:07.498 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:07 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:07.498 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:07 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:08:07.499 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:07 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:08:07.499 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:07 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:08:07.499 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:07 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:07.898 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:08:08.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:08 smithi064 ceph-mon[97675]: purged_snaps scrub starts 2023-09-13T03:08:08.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:08 smithi064 ceph-mon[97675]: purged_snaps scrub ok 2023-09-13T03:08:08.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:08 smithi064 ceph-mon[97675]: osd.5 [v2:172.21.15.170:6808/2903841453,v1:172.21.15.170:6809/2903841453] boot 2023-09-13T03:08:08.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:08 smithi064 ceph-mon[97675]: osdmap e38: 6 total, 6 up, 6 in 2023-09-13T03:08:08.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:08 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-13T03:08:08.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:08 smithi142 ceph-mon[134082]: purged_snaps scrub starts 2023-09-13T03:08:08.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:08 smithi142 ceph-mon[134082]: purged_snaps scrub ok 2023-09-13T03:08:08.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:08 smithi142 ceph-mon[134082]: osd.5 [v2:172.21.15.170:6808/2903841453,v1:172.21.15.170:6809/2903841453] boot 2023-09-13T03:08:08.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:08 smithi142 ceph-mon[134082]: osdmap e38: 6 total, 6 up, 6 in 2023-09-13T03:08:08.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:08 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-13T03:08:08.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:08 smithi170 ceph-mon[134328]: purged_snaps scrub starts 2023-09-13T03:08:08.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:08 smithi170 ceph-mon[134328]: purged_snaps scrub ok 2023-09-13T03:08:08.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:08 smithi170 ceph-mon[134328]: osd.5 [v2:172.21.15.170:6808/2903841453,v1:172.21.15.170:6809/2903841453] boot 2023-09-13T03:08:08.662 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:08 smithi170 ceph-mon[134328]: osdmap e38: 6 total, 6 up, 6 in 2023-09-13T03:08:08.662 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:08 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-13T03:08:09.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:09.361+0000 7f64ee003700 1 -- 172.21.15.64:0/479980820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64e8100690 msgr2=0x7f64e8100ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:08:09.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:09.361+0000 7f64ee003700 1 --2- 172.21.15.64:0/479980820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64e8100690 0x7f64e8100ab0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f64d80099c0 tx=0x7f64d8009cd0 comp rx=0 tx=0).stop 2023-09-13T03:08:09.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:09.361+0000 7f64ee003700 1 -- 172.21.15.64:0/479980820 shutdown_connections 2023-09-13T03:08:09.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:09.361+0000 7f64ee003700 1 --2- 172.21.15.64:0/479980820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64e8102230 0x7f64e8108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:08:09.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:09.361+0000 7f64ee003700 1 --2- 172.21.15.64:0/479980820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64e8100690 0x7f64e8100ab0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:08:09.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:09.361+0000 7f64ee003700 1 --2- 172.21.15.64:0/479980820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64e8101890 0x7f64e8101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:08:09.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:09.361+0000 7f64ee003700 1 -- 172.21.15.64:0/479980820 >> 172.21.15.64:0/479980820 conn(0x7f64e80fbbf0 msgr2=0x7f64e80fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:08:09.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:09.361+0000 7f64ee003700 1 -- 172.21.15.64:0/479980820 shutdown_connections 2023-09-13T03:08:09.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:09.361+0000 7f64ee003700 1 -- 172.21.15.64:0/479980820 wait complete. 2023-09-13T03:08:09.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:09.362+0000 7f64ee003700 1 Processor -- start 2023-09-13T03:08:09.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:09.362+0000 7f64ee003700 1 -- start start 2023-09-13T03:08:09.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:09.362+0000 7f64ee003700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64e8100690 0x7f64e80743a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:08:09.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:09.362+0000 7f64ee003700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64e8101890 0x7f64e80729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:08:09.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:09.362+0000 7f64ee003700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64e8102230 0x7f64e8072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:08:09.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:09.362+0000 7f64ee003700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e80736b0 con 0x7f64e8100690 2023-09-13T03:08:09.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:09.362+0000 7f64ee003700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e8073820 con 0x7f64e8102230 2023-09-13T03:08:09.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:09.362+0000 7f64ee003700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e81a03f0 con 0x7f64e8101890 2023-09-13T03:08:09.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:09.363+0000 7f64e77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64e8100690 0x7f64e80743a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:08:09.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:09.363+0000 7f64e77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64e8100690 0x7f64e80743a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56874/0 (socket says 172.21.15.64:56874) 2023-09-13T03:08:09.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:09.363+0000 7f64e77fe700 1 -- 172.21.15.64:0/784925975 learned_addr learned my addr 172.21.15.64:0/784925975 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:08:09.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:09.363+0000 7f64e7fff700 1 --2- 172.21.15.64:0/784925975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64e8102230 0x7f64e8072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:08:09.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:09.363+0000 7f64e6ffd700 1 --2- 172.21.15.64:0/784925975 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64e8101890 0x7f64e80729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:08:09.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:09.363+0000 7f64e77fe700 1 -- 172.21.15.64:0/784925975 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64e8101890 msgr2=0x7f64e80729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:08:09.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:09.363+0000 7f64e77fe700 1 --2- 172.21.15.64:0/784925975 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64e8101890 0x7f64e80729f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:08:09.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:09.363+0000 7f64e77fe700 1 -- 172.21.15.64:0/784925975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64e8102230 msgr2=0x7f64e8072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:08:09.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:09.363+0000 7f64e77fe700 1 --2- 172.21.15.64:0/784925975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64e8102230 0x7f64e8072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:08:09.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:09.363+0000 7f64e77fe700 1 -- 172.21.15.64:0/784925975 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d80095f0 con 0x7f64e8100690 2023-09-13T03:08:09.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:09.363+0000 7f64e77fe700 1 --2- 172.21.15.64:0/784925975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64e8100690 0x7f64e80743a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f64d8009fd0 tx=0x7f64d800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:08:09.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:09.364+0000 7f64e4ff9700 1 -- 172.21.15.64:0/784925975 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d801d070 con 0x7f64e8100690 2023-09-13T03:08:09.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:09.364+0000 7f64ee003700 1 -- 172.21.15.64:0/784925975 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64e81a0670 con 0x7f64e8100690 2023-09-13T03:08:09.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:09.364+0000 7f64e4ff9700 1 -- 172.21.15.64:0/784925975 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64d8005510 con 0x7f64e8100690 2023-09-13T03:08:09.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:09.364+0000 7f64e4ff9700 1 -- 172.21.15.64:0/784925975 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d8004b00 con 0x7f64e8100690 2023-09-13T03:08:09.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:09.364+0000 7f64ee003700 1 -- 172.21.15.64:0/784925975 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e81a0ae0 con 0x7f64e8100690 2023-09-13T03:08:09.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:09.365+0000 7f64ee003700 1 -- 172.21.15.64:0/784925975 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64e810bee0 con 0x7f64e8100690 2023-09-13T03:08:09.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:09.367+0000 7f64e4ff9700 1 -- 172.21.15.64:0/784925975 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f64d8004ca0 con 0x7f64e8100690 2023-09-13T03:08:09.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:09.368+0000 7f64e4ff9700 1 --2- 172.21.15.64:0/784925975 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f64d00a06c0 0x7f64d00a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:08:09.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:09.368+0000 7f64e4ff9700 1 -- 172.21.15.64:0/784925975 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4562+0+0 (secure 0 0 0) 0x7f64d80cbb30 con 0x7f64e8100690 2023-09-13T03:08:09.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:09.368+0000 7f64e6ffd700 1 --2- 172.21.15.64:0/784925975 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f64d00a06c0 0x7f64d00a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:08:09.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:09.369+0000 7f64e6ffd700 1 --2- 172.21.15.64:0/784925975 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f64d00a06c0 0x7f64d00a2b80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f64dc009bf0 tx=0x7f64dc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:08:09.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:09.373+0000 7f64e4ff9700 1 -- 172.21.15.64:0/784925975 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f64d8096610 con 0x7f64e8100690 2023-09-13T03:08:09.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:09 smithi064 ceph-mon[97675]: osdmap e39: 6 total, 6 up, 6 in 2023-09-13T03:08:09.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:09 smithi064 ceph-mon[97675]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-13T03:08:09.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:09 smithi142 ceph-mon[134082]: osdmap e39: 6 total, 6 up, 6 in 2023-09-13T03:08:09.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:09 smithi142 ceph-mon[134082]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-13T03:08:09.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:09.595+0000 7f64ee003700 1 -- 172.21.15.64:0/784925975 --> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f64e81a0d00 con 0x7f64d00a06c0 2023-09-13T03:08:09.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:09 smithi170 ceph-mon[134328]: osdmap e39: 6 total, 6 up, 6 in 2023-09-13T03:08:09.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:09 smithi170 ceph-mon[134328]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-13T03:08:10.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:10 smithi142 ceph-mon[134082]: osdmap e40: 6 total, 6 up, 6 in 2023-09-13T03:08:10.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:10 smithi142 ceph-mon[134082]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:08:10.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T03:08:10.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T03:08:10.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:10 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:08:10.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:10 smithi064 ceph-mon[97675]: osdmap e40: 6 total, 6 up, 6 in 2023-09-13T03:08:10.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:10 smithi064 ceph-mon[97675]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:08:10.653 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:10 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T03:08:10.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:10 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T03:08:10.654 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:10 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:08:10.659 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:10 smithi170 ceph-mon[134328]: osdmap e40: 6 total, 6 up, 6 in 2023-09-13T03:08:10.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:10 smithi170 ceph-mon[134328]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:08:10.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:10 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T03:08:10.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:10 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T03:08:10.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:10 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:08:11.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:11 smithi142 ceph-mon[134082]: osdmap e41: 6 total, 6 up, 6 in 2023-09-13T03:08:11.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:11 smithi142 ceph-mon[134082]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-13T03:08:11.538 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:11 smithi064 ceph-mon[97675]: osdmap e41: 6 total, 6 up, 6 in 2023-09-13T03:08:11.539 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:11 smithi064 ceph-mon[97675]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-13T03:08:11.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:11 smithi170 ceph-mon[134328]: osdmap e41: 6 total, 6 up, 6 in 2023-09-13T03:08:11.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:11 smithi170 ceph-mon[134328]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-13T03:08:12.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:12 smithi142 ceph-mon[134082]: osdmap e42: 6 total, 6 up, 6 in 2023-09-13T03:08:12.659 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:12 smithi170 ceph-mon[134328]: osdmap e42: 6 total, 6 up, 6 in 2023-09-13T03:08:12.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:12 smithi064 ceph-mon[97675]: osdmap e42: 6 total, 6 up, 6 in 2023-09-13T03:08:13.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:13 smithi142 ceph-mon[134082]: pgmap v147: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-13T03:08:13.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:13 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/921746830' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5cf3e52e-d155-41b1-9194-db047d26a953"}]: dispatch 2023-09-13T03:08:13.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:13 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/921746830' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5cf3e52e-d155-41b1-9194-db047d26a953"}]': finished 2023-09-13T03:08:13.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:13 smithi142 ceph-mon[134082]: osdmap e43: 7 total, 6 up, 7 in 2023-09-13T03:08:13.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:13 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-13T03:08:13.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:13 smithi170 ceph-mon[134328]: pgmap v147: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-13T03:08:13.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:13 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/921746830' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5cf3e52e-d155-41b1-9194-db047d26a953"}]: dispatch 2023-09-13T03:08:13.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:13 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/921746830' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5cf3e52e-d155-41b1-9194-db047d26a953"}]': finished 2023-09-13T03:08:13.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:13 smithi170 ceph-mon[134328]: osdmap e43: 7 total, 6 up, 7 in 2023-09-13T03:08:13.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:13 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-13T03:08:13.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:13 smithi064 ceph-mon[97675]: pgmap v147: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-13T03:08:13.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:13 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/921746830' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5cf3e52e-d155-41b1-9194-db047d26a953"}]: dispatch 2023-09-13T03:08:13.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:13 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/921746830' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5cf3e52e-d155-41b1-9194-db047d26a953"}]': finished 2023-09-13T03:08:13.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:13 smithi064 ceph-mon[97675]: osdmap e43: 7 total, 6 up, 7 in 2023-09-13T03:08:13.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:13 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-13T03:08:14.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:14 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/1775258198' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T03:08:14.659 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:14 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/1775258198' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T03:08:14.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:14 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/1775258198' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T03:08:15.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:15 smithi142 ceph-mon[134082]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2023-09-13T03:08:15.659 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:15 smithi170 ceph-mon[134328]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2023-09-13T03:08:15.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:15 smithi064 ceph-mon[97675]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2023-09-13T03:08:17.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:17 smithi142 ceph-mon[134082]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2023-09-13T03:08:17.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:17 smithi170 ceph-mon[134328]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2023-09-13T03:08:17.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:17 smithi064 ceph-mon[97675]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2023-09-13T03:08:19.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:19 smithi142 ceph-mon[134082]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-13T03:08:19.768 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:19 smithi064 ceph-mon[97675]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-13T03:08:19.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:19 smithi170 ceph-mon[134328]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-13T03:08:20.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:20 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-09-13T03:08:20.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:20 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:08:20.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:20 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-09-13T03:08:20.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:20 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:08:20.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:20 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-09-13T03:08:20.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:20 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:08:21.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:21 smithi064 ceph-mon[97675]: Deploying daemon osd.6 on smithi064 2023-09-13T03:08:21.614 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:21 smithi064 ceph-mon[97675]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2023-09-13T03:08:21.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:21 smithi142 ceph-mon[134082]: Deploying daemon osd.6 on smithi064 2023-09-13T03:08:21.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:21 smithi142 ceph-mon[134082]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2023-09-13T03:08:21.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:21 smithi170 ceph-mon[134328]: Deploying daemon osd.6 on smithi064 2023-09-13T03:08:21.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:21 smithi170 ceph-mon[134328]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2023-09-13T03:08:23.738 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:23 smithi064 ceph-mon[97675]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-09-13T03:08:23.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:23 smithi142 ceph-mon[134082]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-09-13T03:08:23.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:23 smithi170 ceph-mon[134328]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-09-13T03:08:25.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:25 smithi170 ceph-mon[134328]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-09-13T03:08:25.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:25 smithi064 ceph-mon[97675]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-09-13T03:08:25.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:25 smithi142 ceph-mon[134082]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-09-13T03:08:27.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:27 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:27.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:27 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:27.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:27 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:08:27.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:27 smithi170 ceph-mon[134328]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-13T03:08:27.411 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:27 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:27.412 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:27 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:27.412 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:27 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:08:27.412 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:27 smithi064 ceph-mon[97675]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-13T03:08:27.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:27 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:27.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:27 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:27.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:27 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:08:27.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:27 smithi142 ceph-mon[134082]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-13T03:08:29.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:29.409+0000 7f64e4ff9700 1 -- 172.21.15.64:0/784925975 <== mgr.14150 v2:172.21.15.64:6800/873679308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f64e81a0d00 con 0x7f64d00a06c0 2023-09-13T03:08:29.411 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 6 on host 'smithi064' 2023-09-13T03:08:29.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:29.413+0000 7f64ee003700 1 -- 172.21.15.64:0/784925975 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f64d00a06c0 msgr2=0x7f64d00a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:08:29.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:29.413+0000 7f64ee003700 1 --2- 172.21.15.64:0/784925975 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f64d00a06c0 0x7f64d00a2b80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f64dc009bf0 tx=0x7f64dc009260 comp rx=0 tx=0).stop 2023-09-13T03:08:29.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:29.413+0000 7f64ee003700 1 -- 172.21.15.64:0/784925975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64e8100690 msgr2=0x7f64e80743a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:08:29.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:29.413+0000 7f64ee003700 1 --2- 172.21.15.64:0/784925975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64e8100690 0x7f64e80743a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f64d8009fd0 tx=0x7f64d800f690 comp rx=0 tx=0).stop 2023-09-13T03:08:29.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:29.413+0000 7f64ee003700 1 -- 172.21.15.64:0/784925975 shutdown_connections 2023-09-13T03:08:29.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:29.413+0000 7f64ee003700 1 --2- 172.21.15.64:0/784925975 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f64d00a06c0 0x7f64d00a2b80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:08:29.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:29.413+0000 7f64ee003700 1 --2- 172.21.15.64:0/784925975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64e8102230 0x7f64e8072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:08:29.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:29.413+0000 7f64ee003700 1 --2- 172.21.15.64:0/784925975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64e8100690 0x7f64e80743a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:08:29.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:29.413+0000 7f64ee003700 1 --2- 172.21.15.64:0/784925975 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64e8101890 0x7f64e80729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:08:29.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:29.413+0000 7f64ee003700 1 -- 172.21.15.64:0/784925975 >> 172.21.15.64:0/784925975 conn(0x7f64e80fbbf0 msgr2=0x7f64e80fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:08:29.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:29.413+0000 7f64ee003700 1 -- 172.21.15.64:0/784925975 shutdown_connections 2023-09-13T03:08:29.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:08:29.413+0000 7f64ee003700 1 -- 172.21.15.64:0/784925975 wait complete. 2023-09-13T03:08:29.577 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:29 smithi064 ceph-mon[97675]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-13T03:08:29.577 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:29 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:29.577 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:29 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:29.578 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:29 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:08:29.578 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:29 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:08:29.578 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:29 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:29.578 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:29 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:29.578 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:29 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:29.579 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:29 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:08:29.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:29 smithi142 ceph-mon[134082]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-13T03:08:29.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:29 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:29.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:29 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:29.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:29 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:08:29.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:29 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:08:29.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:29 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:29.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:29 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:29.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:29 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:29.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:29 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:08:29.827 DEBUG:teuthology.orchestra.run.smithi064:osd.6> sudo journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@osd.6.service 2023-09-13T03:08:29.831 INFO:tasks.cephadm:Deploying osd.7 on smithi142 with /dev/vg_nvme/lv_2... 2023-09-13T03:08:29.832 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-09-13T03:08:29.905 INFO:journalctl@ceph.osd.6.smithi064.stdout:-- Logs begin at Wed 2023-09-13 02:46:56 UTC. -- 2023-09-13T03:08:29.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:29 smithi170 ceph-mon[134328]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-13T03:08:29.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:29 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:29.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:29 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:29.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:29 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:08:29.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:29 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:08:29.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:29 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:29.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:29 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:29.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:29 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:29.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:29 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:08:30.397 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-13T03:08:30.418 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:08:30 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:08:30.124+0000 7fe5fd3e1640 -1 osd.6 0 log_to_monitors true 2023-09-13T03:08:30.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:30 smithi142 ceph-mon[134082]: from='osd.6 [v2:172.21.15.64:6818/1436584490,v1:172.21.15.64:6819/1436584490]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-09-13T03:08:30.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:30 smithi142 ceph-mon[134082]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-09-13T03:08:30.825 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:30 smithi064 ceph-mon[97675]: from='osd.6 [v2:172.21.15.64:6818/1436584490,v1:172.21.15.64:6819/1436584490]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-09-13T03:08:30.826 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:30 smithi064 ceph-mon[97675]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-09-13T03:08:30.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:30 smithi170 ceph-mon[134328]: from='osd.6 [v2:172.21.15.64:6818/1436584490,v1:172.21.15.64:6819/1436584490]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-09-13T03:08:30.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:30 smithi170 ceph-mon[134328]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-09-13T03:08:31.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:31 smithi142 ceph-mon[134082]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-13T03:08:31.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:31 smithi142 ceph-mon[134082]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-09-13T03:08:31.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:31 smithi142 ceph-mon[134082]: from='osd.6 [v2:172.21.15.64:6818/1436584490,v1:172.21.15.64:6819/1436584490]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-09-13T03:08:31.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:31 smithi142 ceph-mon[134082]: osdmap e44: 7 total, 6 up, 7 in 2023-09-13T03:08:31.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:31 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-13T03:08:31.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:31 smithi142 ceph-mon[134082]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-09-13T03:08:31.895 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-09-13T03:08:31.896 INFO:teuthology.orchestra.run.smithi142.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-09-13T03:08:31.896 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10+0 records in 2023-09-13T03:08:31.896 INFO:teuthology.orchestra.run.smithi142.stderr:10+0 records out 2023-09-13T03:08:31.896 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110075 s, 953 MB/s 2023-09-13T03:08:31.897 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping successful for: 2023-09-13T03:08:31.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:31 smithi170 ceph-mon[134328]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-13T03:08:31.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:31 smithi170 ceph-mon[134328]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-09-13T03:08:31.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:31 smithi170 ceph-mon[134328]: from='osd.6 [v2:172.21.15.64:6818/1436584490,v1:172.21.15.64:6819/1436584490]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-09-13T03:08:31.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:31 smithi170 ceph-mon[134328]: osdmap e44: 7 total, 6 up, 7 in 2023-09-13T03:08:31.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:31 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-13T03:08:31.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:31 smithi170 ceph-mon[134328]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-09-13T03:08:31.918 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:08:31 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:08:31.480+0000 7fe5f2257700 -1 osd.6 0 waiting for initial osdmap 2023-09-13T03:08:31.918 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:08:31 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:08:31.492+0000 7fe5ed04a700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-13T03:08:31.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:31 smithi064 ceph-mon[97675]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-13T03:08:31.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:31 smithi064 ceph-mon[97675]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-09-13T03:08:31.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:31 smithi064 ceph-mon[97675]: from='osd.6 [v2:172.21.15.64:6818/1436584490,v1:172.21.15.64:6819/1436584490]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-09-13T03:08:31.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:31 smithi064 ceph-mon[97675]: osdmap e44: 7 total, 6 up, 7 in 2023-09-13T03:08:31.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:31 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-13T03:08:31.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:31 smithi064 ceph-mon[97675]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-09-13T03:08:32.350 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi142:vg_nvme/lv_2 2023-09-13T03:08:32.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:32 smithi142 ceph-mon[134082]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2023-09-13T03:08:32.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:32 smithi142 ceph-mon[134082]: osdmap e45: 7 total, 6 up, 7 in 2023-09-13T03:08:32.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:32 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-13T03:08:32.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:32 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-13T03:08:32.906 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-13T03:08:32.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:32 smithi170 ceph-mon[134328]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2023-09-13T03:08:32.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:32 smithi170 ceph-mon[134328]: osdmap e45: 7 total, 6 up, 7 in 2023-09-13T03:08:32.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:32 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-13T03:08:32.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:32 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-13T03:08:32.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:32 smithi064 ceph-mon[97675]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2023-09-13T03:08:32.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:32 smithi064 ceph-mon[97675]: osdmap e45: 7 total, 6 up, 7 in 2023-09-13T03:08:32.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:32 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-13T03:08:32.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:32 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-13T03:08:33.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:33 smithi064 ceph-mon[97675]: purged_snaps scrub starts 2023-09-13T03:08:33.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:33 smithi064 ceph-mon[97675]: purged_snaps scrub ok 2023-09-13T03:08:33.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:33 smithi064 ceph-mon[97675]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-13T03:08:33.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:33 smithi064 ceph-mon[97675]: osd.6 [v2:172.21.15.64:6818/1436584490,v1:172.21.15.64:6819/1436584490] boot 2023-09-13T03:08:33.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:33 smithi064 ceph-mon[97675]: osdmap e46: 7 total, 7 up, 7 in 2023-09-13T03:08:33.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:33 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-13T03:08:33.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:33 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:33.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:33 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:33.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:33 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:08:33.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:33 smithi142 ceph-mon[134082]: purged_snaps scrub starts 2023-09-13T03:08:33.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:33 smithi142 ceph-mon[134082]: purged_snaps scrub ok 2023-09-13T03:08:33.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:33 smithi142 ceph-mon[134082]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-13T03:08:33.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:33 smithi142 ceph-mon[134082]: osd.6 [v2:172.21.15.64:6818/1436584490,v1:172.21.15.64:6819/1436584490] boot 2023-09-13T03:08:33.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:33 smithi142 ceph-mon[134082]: osdmap e46: 7 total, 7 up, 7 in 2023-09-13T03:08:33.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:33 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-13T03:08:33.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:33 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:33.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:33 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:33.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:33 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:08:33.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:33 smithi170 ceph-mon[134328]: purged_snaps scrub starts 2023-09-13T03:08:33.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:33 smithi170 ceph-mon[134328]: purged_snaps scrub ok 2023-09-13T03:08:33.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:33 smithi170 ceph-mon[134328]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-13T03:08:33.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:33 smithi170 ceph-mon[134328]: osd.6 [v2:172.21.15.64:6818/1436584490,v1:172.21.15.64:6819/1436584490] boot 2023-09-13T03:08:33.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:33 smithi170 ceph-mon[134328]: osdmap e46: 7 total, 7 up, 7 in 2023-09-13T03:08:33.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:33 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-13T03:08:33.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:33 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:33.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:33 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:33.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:33 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:08:34.120 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:34.119+0000 7f0216c34700 1 -- 172.21.15.142:0/1588997514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0210101130 msgr2=0x7f02101079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:08:34.120 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:34.119+0000 7f0216c34700 1 --2- 172.21.15.142:0/1588997514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0210101130 0x7f02101079e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f020800b0d0 tx=0x7f020800b3e0 comp rx=0 tx=0).stop 2023-09-13T03:08:34.120 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:34.119+0000 7f0216c34700 1 -- 172.21.15.142:0/1588997514 shutdown_connections 2023-09-13T03:08:34.121 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:34.119+0000 7f0216c34700 1 --2- 172.21.15.142:0/1588997514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0210101130 0x7f02101079e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:08:34.121 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:34.119+0000 7f0216c34700 1 --2- 172.21.15.142:0/1588997514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02100ff590 0x7f02100ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:08:34.121 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:34.119+0000 7f0216c34700 1 --2- 172.21.15.142:0/1588997514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0210100790 0x7f0210100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:08:34.121 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:34.119+0000 7f0216c34700 1 -- 172.21.15.142:0/1588997514 >> 172.21.15.142:0/1588997514 conn(0x7f02100fab10 msgr2=0x7f02100fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:08:34.121 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:34.119+0000 7f0216c34700 1 -- 172.21.15.142:0/1588997514 shutdown_connections 2023-09-13T03:08:34.122 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:34.119+0000 7f0216c34700 1 -- 172.21.15.142:0/1588997514 wait complete. 2023-09-13T03:08:34.124 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:34.120+0000 7f0216c34700 1 Processor -- start 2023-09-13T03:08:34.124 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:34.120+0000 7f0216c34700 1 -- start start 2023-09-13T03:08:34.125 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:34.120+0000 7f0216c34700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02100ff590 0x7f0210068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:08:34.125 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:34.120+0000 7f0216c34700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0210100790 0x7f021019b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:08:34.125 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:34.120+0000 7f0216c34700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0210101130 0x7f021019d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:08:34.125 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:34.120+0000 7f0216c34700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021019df60 con 0x7f02100ff590 2023-09-13T03:08:34.125 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:34.120+0000 7f0216c34700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021019e0d0 con 0x7f0210100790 2023-09-13T03:08:34.126 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:34.120+0000 7f0216c34700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021019e290 con 0x7f0210101130 2023-09-13T03:08:34.126 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:34.120+0000 7f020ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0210100790 0x7f021019b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:08:34.126 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:34.120+0000 7f020ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0210100790 0x7f021019b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:40670/0 (socket says 172.21.15.142:40670) 2023-09-13T03:08:34.126 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:34.120+0000 7f020ffff700 1 -- 172.21.15.142:0/2146291559 learned_addr learned my addr 172.21.15.142:0/2146291559 (peer_addr_for_me v2:172.21.15.142:0/0) 2023-09-13T03:08:34.126 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:34.120+0000 7f02149d0700 1 --2- 172.21.15.142:0/2146291559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02100ff590 0x7f0210068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:08:34.127 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:34.120+0000 7f02151d1700 1 --2- 172.21.15.142:0/2146291559 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0210101130 0x7f021019d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:08:34.127 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:34.121+0000 7f020ffff700 1 -- 172.21.15.142:0/2146291559 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0210101130 msgr2=0x7f021019d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:08:34.127 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:34.121+0000 7f020ffff700 1 --2- 172.21.15.142:0/2146291559 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0210101130 0x7f021019d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:08:34.127 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:34.121+0000 7f020ffff700 1 -- 172.21.15.142:0/2146291559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02100ff590 msgr2=0x7f0210068c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:08:34.128 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:34.121+0000 7f020ffff700 1 --2- 172.21.15.142:0/2146291559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02100ff590 0x7f0210068c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:08:34.128 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:34.121+0000 7f020ffff700 1 -- 172.21.15.142:0/2146291559 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0208009d00 con 0x7f0210100790 2023-09-13T03:08:34.128 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:34.121+0000 7f02149d0700 1 --2- 172.21.15.142:0/2146291559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02100ff590 0x7f0210068c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:08:34.128 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:34.121+0000 7f020ffff700 1 --2- 172.21.15.142:0/2146291559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0210100790 0x7f021019b170 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f01fc00d950 tx=0x7f01fc00dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:08:34.128 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:34.121+0000 7f020dffb700 1 -- 172.21.15.142:0/2146291559 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01fc009b40 con 0x7f0210100790 2023-09-13T03:08:34.129 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:34.121+0000 7f020dffb700 1 -- 172.21.15.142:0/2146291559 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01fc009ca0 con 0x7f0210100790 2023-09-13T03:08:34.129 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:34.121+0000 7f020dffb700 1 -- 172.21.15.142:0/2146291559 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01fc00f620 con 0x7f0210100790 2023-09-13T03:08:34.129 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:34.121+0000 7f0216c34700 1 -- 172.21.15.142:0/2146291559 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f021019e570 con 0x7f0210100790 2023-09-13T03:08:34.129 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:34.121+0000 7f0216c34700 1 -- 172.21.15.142:0/2146291559 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f021019ea20 con 0x7f0210100790 2023-09-13T03:08:34.129 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:34.122+0000 7f0216c34700 1 -- 172.21.15.142:0/2146291559 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0210106b00 con 0x7f0210100790 2023-09-13T03:08:34.130 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:34.123+0000 7f020dffb700 1 -- 172.21.15.142:0/2146291559 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f01fc010460 con 0x7f0210100790 2023-09-13T03:08:34.130 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:34.123+0000 7f020dffb700 1 --2- 172.21.15.142:0/2146291559 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f02000a0740 0x7f02000a2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:08:34.130 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:34.124+0000 7f020dffb700 1 -- 172.21.15.142:0/2146291559 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f01fc0cb250 con 0x7f0210100790 2023-09-13T03:08:34.131 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:34.124+0000 7f02149d0700 1 --2- 172.21.15.142:0/2146291559 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f02000a0740 0x7f02000a2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:08:34.131 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:34.124+0000 7f02149d0700 1 --2- 172.21.15.142:0/2146291559 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f02000a0740 0x7f02000a2c00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0204005950 tx=0x7f0204005f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:08:34.131 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:34.128+0000 7f020dffb700 1 -- 172.21.15.142:0/2146291559 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f01fc0959f0 con 0x7f0210100790 2023-09-13T03:08:34.327 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:34.326+0000 7f0216c34700 1 -- 172.21.15.142:0/2146291559 --> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f021002d050 con 0x7f02000a0740 2023-09-13T03:08:34.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:34 smithi142 ceph-mon[134082]: Detected new or changed devices on smithi064 2023-09-13T03:08:34.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:34 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:08:34.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:34 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:08:34.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:34 smithi142 ceph-mon[134082]: Adjusting osd_memory_target on smithi064 to 5755M 2023-09-13T03:08:34.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:34 smithi142 ceph-mon[134082]: osdmap e47: 7 total, 7 up, 7 in 2023-09-13T03:08:34.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:34 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:34.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:34 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:08:34.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:34 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:08:34.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:34 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:34.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:34 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T03:08:34.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:34 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T03:08:34.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:34 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:08:34.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:34 smithi170 ceph-mon[134328]: Detected new or changed devices on smithi064 2023-09-13T03:08:34.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:34 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:08:34.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:34 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:08:34.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:34 smithi170 ceph-mon[134328]: Adjusting osd_memory_target on smithi064 to 5755M 2023-09-13T03:08:34.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:34 smithi170 ceph-mon[134328]: osdmap e47: 7 total, 7 up, 7 in 2023-09-13T03:08:34.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:34 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:34.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:34 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:08:34.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:34 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:08:34.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:34 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:34.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:34 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T03:08:34.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:34 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T03:08:34.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:34 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:08:34.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:34 smithi064 ceph-mon[97675]: Detected new or changed devices on smithi064 2023-09-13T03:08:34.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:34 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:08:34.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:34 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:08:34.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:34 smithi064 ceph-mon[97675]: Adjusting osd_memory_target on smithi064 to 5755M 2023-09-13T03:08:34.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:34 smithi064 ceph-mon[97675]: osdmap e47: 7 total, 7 up, 7 in 2023-09-13T03:08:34.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:34 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:34.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:34 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:08:34.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:34 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:08:34.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:34 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:34.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:34 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T03:08:34.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:34 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T03:08:34.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:34 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:08:35.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:35 smithi142 ceph-mon[134082]: from='client.24407 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:08:35.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:35 smithi142 ceph-mon[134082]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T03:08:35.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:35 smithi170 ceph-mon[134328]: from='client.24407 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:08:35.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:35 smithi170 ceph-mon[134328]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T03:08:35.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:35 smithi064 ceph-mon[97675]: from='client.24407 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:08:35.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:35 smithi064 ceph-mon[97675]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T03:08:37.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:37 smithi170 ceph-mon[134328]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T03:08:37.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:37 smithi170 ceph-mon[134328]: from='client.? 172.21.15.142:0/1903027563' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e97323e3-badd-41a7-af9e-e78a81a64046"}]: dispatch 2023-09-13T03:08:37.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:37 smithi170 ceph-mon[134328]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e97323e3-badd-41a7-af9e-e78a81a64046"}]: dispatch 2023-09-13T03:08:37.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:37 smithi170 ceph-mon[134328]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e97323e3-badd-41a7-af9e-e78a81a64046"}]': finished 2023-09-13T03:08:37.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:37 smithi170 ceph-mon[134328]: osdmap e48: 8 total, 7 up, 8 in 2023-09-13T03:08:37.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:37 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-13T03:08:37.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:37 smithi170 ceph-mon[134328]: from='client.? 172.21.15.142:0/2150691649' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T03:08:37.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:37 smithi064 ceph-mon[97675]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T03:08:37.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:37 smithi064 ceph-mon[97675]: from='client.? 172.21.15.142:0/1903027563' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e97323e3-badd-41a7-af9e-e78a81a64046"}]: dispatch 2023-09-13T03:08:37.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:37 smithi064 ceph-mon[97675]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e97323e3-badd-41a7-af9e-e78a81a64046"}]: dispatch 2023-09-13T03:08:37.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:37 smithi064 ceph-mon[97675]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e97323e3-badd-41a7-af9e-e78a81a64046"}]': finished 2023-09-13T03:08:37.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:37 smithi064 ceph-mon[97675]: osdmap e48: 8 total, 7 up, 8 in 2023-09-13T03:08:37.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:37 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-13T03:08:37.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:37 smithi064 ceph-mon[97675]: from='client.? 172.21.15.142:0/2150691649' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T03:08:37.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:37 smithi142 ceph-mon[134082]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T03:08:37.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:37 smithi142 ceph-mon[134082]: from='client.? 172.21.15.142:0/1903027563' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e97323e3-badd-41a7-af9e-e78a81a64046"}]: dispatch 2023-09-13T03:08:37.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:37 smithi142 ceph-mon[134082]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e97323e3-badd-41a7-af9e-e78a81a64046"}]: dispatch 2023-09-13T03:08:37.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:37 smithi142 ceph-mon[134082]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e97323e3-badd-41a7-af9e-e78a81a64046"}]': finished 2023-09-13T03:08:37.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:37 smithi142 ceph-mon[134082]: osdmap e48: 8 total, 7 up, 8 in 2023-09-13T03:08:37.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:37 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-13T03:08:37.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:37 smithi142 ceph-mon[134082]: from='client.? 172.21.15.142:0/2150691649' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T03:08:39.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:39 smithi170 ceph-mon[134328]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T03:08:39.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:39 smithi064 ceph-mon[97675]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T03:08:39.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:39 smithi142 ceph-mon[134082]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T03:08:41.796 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:41 smithi142 ceph-mon[134082]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T03:08:41.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:41 smithi170 ceph-mon[134328]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T03:08:41.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:41 smithi064 ceph-mon[97675]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T03:08:43.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:43 smithi142 ceph-mon[134082]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T03:08:43.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:43 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-09-13T03:08:43.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:43 smithi170 ceph-mon[134328]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T03:08:43.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:43 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-09-13T03:08:43.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:43 smithi064 ceph-mon[97675]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T03:08:43.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:43 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-09-13T03:08:44.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:44 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:08:44.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:44 smithi142 ceph-mon[134082]: Deploying daemon osd.7 on smithi142 2023-09-13T03:08:44.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:44 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:08:44.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:44 smithi170 ceph-mon[134328]: Deploying daemon osd.7 on smithi142 2023-09-13T03:08:44.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:44 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:08:44.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:44 smithi064 ceph-mon[97675]: Deploying daemon osd.7 on smithi142 2023-09-13T03:08:45.872 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:45 smithi142 ceph-mon[134082]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T03:08:45.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:45 smithi170 ceph-mon[134328]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T03:08:45.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:45 smithi064 ceph-mon[97675]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T03:08:47.839 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:47 smithi142 ceph-mon[134082]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T03:08:47.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:47 smithi170 ceph-mon[134328]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T03:08:47.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:47 smithi064 ceph-mon[97675]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T03:08:48.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:48 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:48.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:48 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:48.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:48 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:08:48.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:48 smithi142 ceph-mon[134082]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T03:08:49.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:48 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:49.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:48 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:49.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:48 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:08:49.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:48 smithi170 ceph-mon[134328]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T03:08:49.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:48 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:49.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:48 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:49.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:48 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:08:49.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:48 smithi064 ceph-mon[97675]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T03:08:50.642 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 7 on host 'smithi142' 2023-09-13T03:08:50.643 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:50.641+0000 7f020dffb700 1 -- 172.21.15.142:0/2146291559 <== mgr.14150 v2:172.21.15.64:6800/873679308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f021002d050 con 0x7f02000a0740 2023-09-13T03:08:50.644 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:50.643+0000 7f0216c34700 1 -- 172.21.15.142:0/2146291559 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f02000a0740 msgr2=0x7f02000a2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:08:50.645 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:50.643+0000 7f0216c34700 1 --2- 172.21.15.142:0/2146291559 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f02000a0740 0x7f02000a2c00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0204005950 tx=0x7f0204005f50 comp rx=0 tx=0).stop 2023-09-13T03:08:50.645 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:50.643+0000 7f0216c34700 1 -- 172.21.15.142:0/2146291559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0210100790 msgr2=0x7f021019b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:08:50.645 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:50.643+0000 7f0216c34700 1 --2- 172.21.15.142:0/2146291559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0210100790 0x7f021019b170 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f01fc00d950 tx=0x7f01fc00dd10 comp rx=0 tx=0).stop 2023-09-13T03:08:50.645 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:50.643+0000 7f0216c34700 1 -- 172.21.15.142:0/2146291559 shutdown_connections 2023-09-13T03:08:50.646 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:50.643+0000 7f0216c34700 1 --2- 172.21.15.142:0/2146291559 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f02000a0740 0x7f02000a2c00 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:08:50.646 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:50.643+0000 7f0216c34700 1 --2- 172.21.15.142:0/2146291559 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0210101130 0x7f021019d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:08:50.646 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:50.643+0000 7f0216c34700 1 --2- 172.21.15.142:0/2146291559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02100ff590 0x7f0210068c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:08:50.647 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:50.643+0000 7f0216c34700 1 --2- 172.21.15.142:0/2146291559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0210100790 0x7f021019b170 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:08:50.647 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:50.643+0000 7f0216c34700 1 -- 172.21.15.142:0/2146291559 >> 172.21.15.142:0/2146291559 conn(0x7f02100fab10 msgr2=0x7f0210103c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:08:50.647 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:50.643+0000 7f0216c34700 1 -- 172.21.15.142:0/2146291559 shutdown_connections 2023-09-13T03:08:50.648 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:08:50.643+0000 7f0216c34700 1 -- 172.21.15.142:0/2146291559 wait complete. 2023-09-13T03:08:51.118 DEBUG:teuthology.orchestra.run.smithi142:osd.7> sudo journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@osd.7.service 2023-09-13T03:08:51.122 INFO:tasks.cephadm:Deploying osd.8 on smithi170 with /dev/vg_nvme/lv_2... 2023-09-13T03:08:51.123 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-09-13T03:08:51.153 INFO:journalctl@ceph.osd.7.smithi142.stdout:-- Logs begin at Wed 2023-09-13 02:43:02 UTC. -- 2023-09-13T03:08:51.689 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-13T03:08:51.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:51 smithi142 ceph-mon[134082]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T03:08:51.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:51 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:51.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:51 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:51.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:51 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:08:51.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:51 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:08:51.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:51 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:51.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:51 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:51.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:51 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:51.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:51 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:08:51.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:51 smithi170 ceph-mon[134328]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T03:08:51.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:51 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:51.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:51 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:51.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:51 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:08:51.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:51 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:08:51.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:51 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:51.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:51 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:51.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:51 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:51.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:51 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:08:51.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:51 smithi064 ceph-mon[97675]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T03:08:51.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:51 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:51.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:51 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:51.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:51 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:08:51.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:51 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:08:51.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:51 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:51.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:51 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:51.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:51 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:51.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:51 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:08:52.106 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:08:51 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:08:51.830+0000 7f4364fbc640 -1 osd.7 0 log_to_monitors true 2023-09-13T03:08:52.531 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:52 smithi170 ceph-mon[134328]: from='osd.7 [v2:172.21.15.142:6816/3726793534,v1:172.21.15.142:6817/3726793534]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-13T03:08:52.531 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:52 smithi170 ceph-mon[134328]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-13T03:08:52.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:52 smithi142 ceph-mon[134082]: from='osd.7 [v2:172.21.15.142:6816/3726793534,v1:172.21.15.142:6817/3726793534]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-13T03:08:52.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:52 smithi142 ceph-mon[134082]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-13T03:08:52.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:52 smithi064 ceph-mon[97675]: from='osd.7 [v2:172.21.15.142:6816/3726793534,v1:172.21.15.142:6817/3726793534]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-13T03:08:52.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:52 smithi064 ceph-mon[97675]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-13T03:08:53.257 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-09-13T03:08:53.257 INFO:teuthology.orchestra.run.smithi170.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-09-13T03:08:53.258 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10+0 records in 2023-09-13T03:08:53.258 INFO:teuthology.orchestra.run.smithi170.stderr:10+0 records out 2023-09-13T03:08:53.258 INFO:teuthology.orchestra.run.smithi170.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0124268 s, 844 MB/s 2023-09-13T03:08:53.259 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping successful for: 2023-09-13T03:08:53.612 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:53 smithi170 ceph-mon[134328]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T03:08:53.612 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:53 smithi170 ceph-mon[134328]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-09-13T03:08:53.613 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:53 smithi170 ceph-mon[134328]: osdmap e49: 8 total, 7 up, 8 in 2023-09-13T03:08:53.613 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:53 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-13T03:08:53.613 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:53 smithi170 ceph-mon[134328]: from='osd.7 [v2:172.21.15.142:6816/3726793534,v1:172.21.15.142:6817/3726793534]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-09-13T03:08:53.613 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:53 smithi170 ceph-mon[134328]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-09-13T03:08:53.642 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi170:vg_nvme/lv_2 2023-09-13T03:08:53.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:53 smithi064 ceph-mon[97675]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T03:08:53.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:53 smithi064 ceph-mon[97675]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-09-13T03:08:53.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:53 smithi064 ceph-mon[97675]: osdmap e49: 8 total, 7 up, 8 in 2023-09-13T03:08:53.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:53 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-13T03:08:53.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:53 smithi064 ceph-mon[97675]: from='osd.7 [v2:172.21.15.142:6816/3726793534,v1:172.21.15.142:6817/3726793534]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-09-13T03:08:53.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:53 smithi064 ceph-mon[97675]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-09-13T03:08:53.995 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:08:53 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:08:53.499+0000 7f4359e32700 -1 osd.7 0 waiting for initial osdmap 2023-09-13T03:08:53.995 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:08:53 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:08:53.514+0000 7f4354424700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-13T03:08:53.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:53 smithi142 ceph-mon[134082]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-13T03:08:53.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:53 smithi142 ceph-mon[134082]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-09-13T03:08:53.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:53 smithi142 ceph-mon[134082]: osdmap e49: 8 total, 7 up, 8 in 2023-09-13T03:08:53.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:53 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-13T03:08:53.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:53 smithi142 ceph-mon[134082]: from='osd.7 [v2:172.21.15.142:6816/3726793534,v1:172.21.15.142:6817/3726793534]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-09-13T03:08:53.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:53 smithi142 ceph-mon[134082]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-09-13T03:08:54.195 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-13T03:08:54.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:54 smithi170 ceph-mon[134328]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2023-09-13T03:08:54.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:54 smithi170 ceph-mon[134328]: osdmap e50: 8 total, 7 up, 8 in 2023-09-13T03:08:54.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:54 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-13T03:08:54.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:54 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-13T03:08:54.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:54 smithi170 ceph-mon[134328]: Detected new or changed devices on smithi142 2023-09-13T03:08:54.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:54 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:54.662 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:54 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:54.662 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:54 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:08:54.662 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:54 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:08:54.662 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:54 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:08:54.663 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:54 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:54.663 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:54 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:08:54.663 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:54 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:08:54.663 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:54 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:54.663 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:54 smithi170 ceph-mon[134328]: osd.7 [v2:172.21.15.142:6816/3726793534,v1:172.21.15.142:6817/3726793534] boot 2023-09-13T03:08:54.664 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:54 smithi170 ceph-mon[134328]: osdmap e51: 8 total, 8 up, 8 in 2023-09-13T03:08:54.664 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:54 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-13T03:08:54.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:54 smithi064 ceph-mon[97675]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2023-09-13T03:08:54.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:54 smithi064 ceph-mon[97675]: osdmap e50: 8 total, 7 up, 8 in 2023-09-13T03:08:54.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:54 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-13T03:08:54.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:54 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-13T03:08:54.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:54 smithi064 ceph-mon[97675]: Detected new or changed devices on smithi142 2023-09-13T03:08:54.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:54 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:54.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:54 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:54.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:54 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:08:54.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:54 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:08:54.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:54 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:08:54.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:54 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:54.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:54 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:08:54.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:54 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:08:54.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:54 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:54.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:54 smithi064 ceph-mon[97675]: osd.7 [v2:172.21.15.142:6816/3726793534,v1:172.21.15.142:6817/3726793534] boot 2023-09-13T03:08:54.922 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:54 smithi064 ceph-mon[97675]: osdmap e51: 8 total, 8 up, 8 in 2023-09-13T03:08:54.922 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:54 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-13T03:08:54.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:54 smithi142 ceph-mon[134082]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2023-09-13T03:08:54.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:54 smithi142 ceph-mon[134082]: osdmap e50: 8 total, 7 up, 8 in 2023-09-13T03:08:54.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:54 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-13T03:08:54.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:54 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-13T03:08:54.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:54 smithi142 ceph-mon[134082]: Detected new or changed devices on smithi142 2023-09-13T03:08:54.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:54 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:54.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:54 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:54.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:54 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:08:54.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:54 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:08:54.998 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:54 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:08:54.998 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:54 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:54.998 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:54 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:08:54.999 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:54 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:08:54.999 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:54 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:08:54.999 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:54 smithi142 ceph-mon[134082]: osd.7 [v2:172.21.15.142:6816/3726793534,v1:172.21.15.142:6817/3726793534] boot 2023-09-13T03:08:55.000 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:54 smithi142 ceph-mon[134082]: osdmap e51: 8 total, 8 up, 8 in 2023-09-13T03:08:55.000 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:54 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-13T03:08:55.383 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:55.380+0000 7ffbc4e69700 1 -- 172.21.15.170:0/1681250166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbc00ff590 msgr2=0x7ffbc00ff9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:08:55.383 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:55.380+0000 7ffbc4e69700 1 --2- 172.21.15.170:0/1681250166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbc00ff590 0x7ffbc00ff9b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0009990 tx=0x7ffbb0009ca0 comp rx=0 tx=0).stop 2023-09-13T03:08:55.383 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:55.380+0000 7ffbc4e69700 1 -- 172.21.15.170:0/1681250166 shutdown_connections 2023-09-13T03:08:55.383 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:55.380+0000 7ffbc4e69700 1 --2- 172.21.15.170:0/1681250166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffbc0101130 0x7ffbc01079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:08:55.383 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:55.380+0000 7ffbc4e69700 1 --2- 172.21.15.170:0/1681250166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbc00ff590 0x7ffbc00ff9b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:08:55.384 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:55.380+0000 7ffbc4e69700 1 --2- 172.21.15.170:0/1681250166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffbc0100790 0x7ffbc0100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:08:55.384 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:55.380+0000 7ffbc4e69700 1 -- 172.21.15.170:0/1681250166 >> 172.21.15.170:0/1681250166 conn(0x7ffbc00fab10 msgr2=0x7ffbc00fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:08:55.384 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:55.380+0000 7ffbc4e69700 1 -- 172.21.15.170:0/1681250166 shutdown_connections 2023-09-13T03:08:55.384 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:55.380+0000 7ffbc4e69700 1 -- 172.21.15.170:0/1681250166 wait complete. 2023-09-13T03:08:55.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:55.381+0000 7ffbc4e69700 1 Processor -- start 2023-09-13T03:08:55.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:55.381+0000 7ffbc4e69700 1 -- start start 2023-09-13T03:08:55.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:55.381+0000 7ffbc4e69700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffbc00ff590 0x7ffbc0068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:08:55.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:55.381+0000 7ffbc4e69700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffbc0100790 0x7ffbc019b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:08:55.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:55.381+0000 7ffbc4e69700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbc0101130 0x7ffbc019d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:08:55.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:55.381+0000 7ffbc4e69700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc019def0 con 0x7ffbc00ff590 2023-09-13T03:08:55.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:55.381+0000 7ffbc4e69700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc019e060 con 0x7ffbc0100790 2023-09-13T03:08:55.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:55.381+0000 7ffbc4e69700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc019e220 con 0x7ffbc0101130 2023-09-13T03:08:55.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:55.381+0000 7ffbbed9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbc0101130 0x7ffbc019d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:08:55.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:55.381+0000 7ffbbed9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbc0101130 0x7ffbc019d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45626/0 (socket says 172.21.15.170:45626) 2023-09-13T03:08:55.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:55.381+0000 7ffbbed9d700 1 -- 172.21.15.170:0/127183770 learned_addr learned my addr 172.21.15.170:0/127183770 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-09-13T03:08:55.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:55.381+0000 7ffbbe59c700 1 --2- 172.21.15.170:0/127183770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffbc00ff590 0x7ffbc0068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:08:55.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:55.381+0000 7ffbbdd9b700 1 --2- 172.21.15.170:0/127183770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffbc0100790 0x7ffbc019b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:08:55.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:55.381+0000 7ffbbed9d700 1 -- 172.21.15.170:0/127183770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffbc0100790 msgr2=0x7ffbc019b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:08:55.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:55.381+0000 7ffbbed9d700 1 --2- 172.21.15.170:0/127183770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffbc0100790 0x7ffbc019b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:08:55.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:55.381+0000 7ffbbed9d700 1 -- 172.21.15.170:0/127183770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffbc00ff590 msgr2=0x7ffbc0068c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:08:55.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:55.381+0000 7ffbbed9d700 1 --2- 172.21.15.170:0/127183770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffbc00ff590 0x7ffbc0068c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:08:55.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:55.381+0000 7ffbbed9d700 1 -- 172.21.15.170:0/127183770 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb00095f0 con 0x7ffbc0101130 2023-09-13T03:08:55.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:55.381+0000 7ffbbdd9b700 1 --2- 172.21.15.170:0/127183770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffbc0100790 0x7ffbc019b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:08:55.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:55.381+0000 7ffbbed9d700 1 --2- 172.21.15.170:0/127183770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbc0101130 0x7ffbc019d750 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8007ee0 tx=0x7ffbb800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:08:55.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:55.381+0000 7ffbaf7fe700 1 -- 172.21.15.170:0/127183770 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb8017070 con 0x7ffbc0101130 2023-09-13T03:08:55.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:55.381+0000 7ffbc4e69700 1 -- 172.21.15.170:0/127183770 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbc019e500 con 0x7ffbc0101130 2023-09-13T03:08:55.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:55.381+0000 7ffbaf7fe700 1 -- 172.21.15.170:0/127183770 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffbb800f040 con 0x7ffbc0101130 2023-09-13T03:08:55.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:55.382+0000 7ffbaf7fe700 1 -- 172.21.15.170:0/127183770 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb8005740 con 0x7ffbc0101130 2023-09-13T03:08:55.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:55.382+0000 7ffbc4e69700 1 -- 172.21.15.170:0/127183770 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbc019e9b0 con 0x7ffbc0101130 2023-09-13T03:08:55.396 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:55.382+0000 7ffbc4e69700 1 -- 172.21.15.170:0/127183770 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbc0106b80 con 0x7ffbc0101130 2023-09-13T03:08:55.396 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:55.384+0000 7ffbaf7fe700 1 -- 172.21.15.170:0/127183770 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7ffbb80058a0 con 0x7ffbc0101130 2023-09-13T03:08:55.396 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:55.384+0000 7ffbaf7fe700 1 --2- 172.21.15.170:0/127183770 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ffba80a0640 0x7ffba80a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:08:55.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:55.384+0000 7ffbaf7fe700 1 -- 172.21.15.170:0/127183770 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5426+0+0 (secure 0 0 0) 0x7ffbb80cafe0 con 0x7ffbc0101130 2023-09-13T03:08:55.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:55.385+0000 7ffbbe59c700 1 --2- 172.21.15.170:0/127183770 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ffba80a0640 0x7ffba80a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:08:55.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:55.385+0000 7ffbbe59c700 1 --2- 172.21.15.170:0/127183770 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ffba80a0640 0x7ffba80a2b00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ffbb000b5c0 tx=0x7ffbb0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:08:55.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:55.387+0000 7ffbaf7fe700 1 -- 172.21.15.170:0/127183770 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ffbb80955b0 con 0x7ffbc0101130 2023-09-13T03:08:55.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:08:55.584+0000 7ffbc4e69700 1 -- 172.21.15.170:0/127183770 --> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7ffbc002d090 con 0x7ffba80a0640 2023-09-13T03:08:55.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:55 smithi170 ceph-mon[134328]: purged_snaps scrub starts 2023-09-13T03:08:55.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:55 smithi170 ceph-mon[134328]: purged_snaps scrub ok 2023-09-13T03:08:55.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:55 smithi170 ceph-mon[134328]: Adjusting osd_memory_target on smithi142 to 5722M 2023-09-13T03:08:55.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:55 smithi170 ceph-mon[134328]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-09-13T03:08:55.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:55 smithi170 ceph-mon[134328]: osdmap e52: 8 total, 8 up, 8 in 2023-09-13T03:08:55.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:55 smithi064 ceph-mon[97675]: purged_snaps scrub starts 2023-09-13T03:08:55.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:55 smithi064 ceph-mon[97675]: purged_snaps scrub ok 2023-09-13T03:08:55.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:55 smithi064 ceph-mon[97675]: Adjusting osd_memory_target on smithi142 to 5722M 2023-09-13T03:08:55.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:55 smithi064 ceph-mon[97675]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-09-13T03:08:55.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:55 smithi064 ceph-mon[97675]: osdmap e52: 8 total, 8 up, 8 in 2023-09-13T03:08:55.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:55 smithi142 ceph-mon[134082]: purged_snaps scrub starts 2023-09-13T03:08:55.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:55 smithi142 ceph-mon[134082]: purged_snaps scrub ok 2023-09-13T03:08:55.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:55 smithi142 ceph-mon[134082]: Adjusting osd_memory_target on smithi142 to 5722M 2023-09-13T03:08:55.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:55 smithi142 ceph-mon[134082]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-09-13T03:08:55.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:55 smithi142 ceph-mon[134082]: osdmap e52: 8 total, 8 up, 8 in 2023-09-13T03:08:56.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:56 smithi170 ceph-mon[134328]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:08:56.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:56 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T03:08:56.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:56 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T03:08:56.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:56 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:08:56.662 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:56 smithi170 ceph-mon[134328]: osdmap e53: 8 total, 8 up, 8 in 2023-09-13T03:08:56.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:56 smithi064 ceph-mon[97675]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:08:56.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:56 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T03:08:56.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:56 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T03:08:56.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:56 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:08:56.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:56 smithi064 ceph-mon[97675]: osdmap e53: 8 total, 8 up, 8 in 2023-09-13T03:08:56.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:56 smithi142 ceph-mon[134082]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:08:56.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:56 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T03:08:56.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:56 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T03:08:56.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:56 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:08:56.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:56 smithi142 ceph-mon[134082]: osdmap e53: 8 total, 8 up, 8 in 2023-09-13T03:08:57.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:57 smithi170 ceph-mon[134328]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-09-13T03:08:57.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:57 smithi064 ceph-mon[97675]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-09-13T03:08:57.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:57 smithi142 ceph-mon[134082]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-09-13T03:08:58.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:58 smithi170 ceph-mon[134328]: osdmap e54: 8 total, 8 up, 8 in 2023-09-13T03:08:58.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:58 smithi170 ceph-mon[134328]: from='client.? 172.21.15.170:0/25967506' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83d2835c-ed26-493d-9222-c2894ec0e071"}]: dispatch 2023-09-13T03:08:58.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:58 smithi170 ceph-mon[134328]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83d2835c-ed26-493d-9222-c2894ec0e071"}]: dispatch 2023-09-13T03:08:58.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:58 smithi170 ceph-mon[134328]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83d2835c-ed26-493d-9222-c2894ec0e071"}]': finished 2023-09-13T03:08:58.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:58 smithi170 ceph-mon[134328]: osdmap e55: 9 total, 8 up, 9 in 2023-09-13T03:08:58.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:58 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-13T03:08:58.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:58 smithi064 ceph-mon[97675]: osdmap e54: 8 total, 8 up, 8 in 2023-09-13T03:08:58.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:58 smithi064 ceph-mon[97675]: from='client.? 172.21.15.170:0/25967506' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83d2835c-ed26-493d-9222-c2894ec0e071"}]: dispatch 2023-09-13T03:08:58.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:58 smithi064 ceph-mon[97675]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83d2835c-ed26-493d-9222-c2894ec0e071"}]: dispatch 2023-09-13T03:08:58.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:58 smithi064 ceph-mon[97675]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83d2835c-ed26-493d-9222-c2894ec0e071"}]': finished 2023-09-13T03:08:58.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:58 smithi064 ceph-mon[97675]: osdmap e55: 9 total, 8 up, 9 in 2023-09-13T03:08:58.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:58 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-13T03:08:58.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:58 smithi142 ceph-mon[134082]: osdmap e54: 8 total, 8 up, 8 in 2023-09-13T03:08:58.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:58 smithi142 ceph-mon[134082]: from='client.? 172.21.15.170:0/25967506' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83d2835c-ed26-493d-9222-c2894ec0e071"}]: dispatch 2023-09-13T03:08:58.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:58 smithi142 ceph-mon[134082]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83d2835c-ed26-493d-9222-c2894ec0e071"}]: dispatch 2023-09-13T03:08:58.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:58 smithi142 ceph-mon[134082]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83d2835c-ed26-493d-9222-c2894ec0e071"}]': finished 2023-09-13T03:08:58.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:58 smithi142 ceph-mon[134082]: osdmap e55: 9 total, 8 up, 9 in 2023-09-13T03:08:58.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:58 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-13T03:08:59.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:59 smithi170 ceph-mon[134328]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T03:08:59.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:08:59 smithi170 ceph-mon[134328]: from='client.? 172.21.15.170:0/2148481968' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T03:08:59.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:59 smithi064 ceph-mon[97675]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T03:08:59.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:08:59 smithi064 ceph-mon[97675]: from='client.? 172.21.15.170:0/2148481968' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T03:08:59.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:59 smithi142 ceph-mon[134082]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T03:08:59.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:08:59 smithi142 ceph-mon[134082]: from='client.? 172.21.15.170:0/2148481968' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T03:09:01.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:01 smithi170 ceph-mon[134328]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T03:09:01.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:01 smithi064 ceph-mon[97675]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T03:09:01.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:01 smithi142 ceph-mon[134082]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T03:09:03.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:03 smithi170 ceph-mon[134328]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T03:09:03.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:03 smithi064 ceph-mon[97675]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T03:09:03.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:03 smithi142 ceph-mon[134082]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T03:09:05.233 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:05 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-09-13T03:09:05.233 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:05 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:09:05.234 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:05 smithi170 ceph-mon[134328]: Deploying daemon osd.8 on smithi170 2023-09-13T03:09:05.234 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:05 smithi170 ceph-mon[134328]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T03:09:05.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:05 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-09-13T03:09:05.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:05 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:09:05.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:05 smithi064 ceph-mon[97675]: Deploying daemon osd.8 on smithi170 2023-09-13T03:09:05.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:05 smithi064 ceph-mon[97675]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T03:09:05.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:05 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-09-13T03:09:05.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:05 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:09:05.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:05 smithi142 ceph-mon[134082]: Deploying daemon osd.8 on smithi170 2023-09-13T03:09:05.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:05 smithi142 ceph-mon[134082]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T03:09:07.659 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:07 smithi170 ceph-mon[134328]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T03:09:07.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:07 smithi142 ceph-mon[134082]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T03:09:07.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:07 smithi064 ceph-mon[97675]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T03:09:09.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:09 smithi142 ceph-mon[134082]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T03:09:09.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:09 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:09.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:09 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:09.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:09 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:09:09.837 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:09 smithi170 ceph-mon[134328]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T03:09:09.837 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:09 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:09.837 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:09 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:09.837 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:09 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:09:09.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:09 smithi064 ceph-mon[97675]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T03:09:09.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:09 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:09.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:09 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:09.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:09 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:09:11.542 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:11 smithi170 ceph-mon[134328]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T03:09:11.543 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:11 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:11.543 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:11 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:11.646 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 8 on host 'smithi170' 2023-09-13T03:09:11.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:09:11.645+0000 7ffbaf7fe700 1 -- 172.21.15.170:0/127183770 <== mgr.14150 v2:172.21.15.64:6800/873679308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ffbc002d090 con 0x7ffba80a0640 2023-09-13T03:09:11.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:09:11.647+0000 7ffbc4e69700 1 -- 172.21.15.170:0/127183770 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ffba80a0640 msgr2=0x7ffba80a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:09:11.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:09:11.647+0000 7ffbc4e69700 1 --2- 172.21.15.170:0/127183770 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ffba80a0640 0x7ffba80a2b00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ffbb000b5c0 tx=0x7ffbb0005fb0 comp rx=0 tx=0).stop 2023-09-13T03:09:11.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:09:11.648+0000 7ffbc4e69700 1 -- 172.21.15.170:0/127183770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbc0101130 msgr2=0x7ffbc019d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:09:11.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:09:11.648+0000 7ffbc4e69700 1 --2- 172.21.15.170:0/127183770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbc0101130 0x7ffbc019d750 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8007ee0 tx=0x7ffbb800d3b0 comp rx=0 tx=0).stop 2023-09-13T03:09:11.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:09:11.648+0000 7ffbc4e69700 1 -- 172.21.15.170:0/127183770 shutdown_connections 2023-09-13T03:09:11.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:09:11.648+0000 7ffbc4e69700 1 --2- 172.21.15.170:0/127183770 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ffba80a0640 0x7ffba80a2b00 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:09:11.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:09:11.648+0000 7ffbc4e69700 1 --2- 172.21.15.170:0/127183770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbc0101130 0x7ffbc019d750 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:09:11.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:09:11.648+0000 7ffbc4e69700 1 --2- 172.21.15.170:0/127183770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffbc00ff590 0x7ffbc0068c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:09:11.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:09:11.648+0000 7ffbc4e69700 1 --2- 172.21.15.170:0/127183770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffbc0100790 0x7ffbc019b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:09:11.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:09:11.648+0000 7ffbc4e69700 1 -- 172.21.15.170:0/127183770 >> 172.21.15.170:0/127183770 conn(0x7ffbc00fab10 msgr2=0x7ffbc0103cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:09:11.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:09:11.648+0000 7ffbc4e69700 1 -- 172.21.15.170:0/127183770 shutdown_connections 2023-09-13T03:09:11.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:09:11.648+0000 7ffbc4e69700 1 -- 172.21.15.170:0/127183770 wait complete. 2023-09-13T03:09:11.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:11 smithi142 ceph-mon[134082]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T03:09:11.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:11 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:11.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:11 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:11.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:11 smithi064 ceph-mon[97675]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T03:09:11.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:11 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:11.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:11 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:12.070 DEBUG:teuthology.orchestra.run.smithi170:osd.8> sudo journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@osd.8.service 2023-09-13T03:09:12.072 INFO:tasks.cephadm:Deploying osd.9 on smithi064 with /dev/vg_nvme/lv_1... 2023-09-13T03:09:12.073 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-09-13T03:09:12.103 INFO:journalctl@ceph.osd.8.smithi170.stdout:-- Logs begin at Wed 2023-09-13 02:42:51 UTC. -- 2023-09-13T03:09:12.814 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:09:12.882 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:12 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:12.883 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:12 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:12.883 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:09:12 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:09:12.640+0000 7fa0d73b0640 -1 osd.8 0 log_to_monitors true 2023-09-13T03:09:12.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:12 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:12.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:12 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:12.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:12 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:12.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:12 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:13.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:13 smithi170 ceph-mon[134328]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T03:09:13.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:13 smithi170 ceph-mon[134328]: from='osd.8 [v2:172.21.15.170:6816/4272513873,v1:172.21.15.170:6817/4272513873]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-13T03:09:13.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:13 smithi170 ceph-mon[134328]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-13T03:09:13.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:13 smithi064 ceph-mon[97675]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T03:09:13.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:13 smithi064 ceph-mon[97675]: from='osd.8 [v2:172.21.15.170:6816/4272513873,v1:172.21.15.170:6817/4272513873]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-13T03:09:13.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:13 smithi064 ceph-mon[97675]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-13T03:09:13.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:13 smithi142 ceph-mon[134082]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T03:09:13.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:13 smithi142 ceph-mon[134082]: from='osd.8 [v2:172.21.15.170:6816/4272513873,v1:172.21.15.170:6817/4272513873]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-13T03:09:13.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:13 smithi142 ceph-mon[134082]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-13T03:09:14.656 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-09-13T03:09:14.656 INFO:teuthology.orchestra.run.smithi064.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-09-13T03:09:14.657 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10+0 records in 2023-09-13T03:09:14.657 INFO:teuthology.orchestra.run.smithi064.stderr:10+0 records out 2023-09-13T03:09:14.657 INFO:teuthology.orchestra.run.smithi064.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0322537 s, 325 MB/s 2023-09-13T03:09:14.658 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping successful for: 2023-09-13T03:09:14.910 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:09:14 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:09:14.634+0000 7fa0cc226700 -1 osd.8 0 waiting for initial osdmap 2023-09-13T03:09:14.910 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:09:14 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:09:14.647+0000 7fa0c6017700 -1 osd.8 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-13T03:09:14.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:14 smithi170 ceph-mon[134328]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-09-13T03:09:14.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:14 smithi170 ceph-mon[134328]: from='osd.8 [v2:172.21.15.170:6816/4272513873,v1:172.21.15.170:6817/4272513873]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-13T03:09:14.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:14 smithi170 ceph-mon[134328]: osdmap e56: 9 total, 8 up, 9 in 2023-09-13T03:09:14.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:14 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-13T03:09:14.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:14 smithi170 ceph-mon[134328]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-13T03:09:14.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:14 smithi170 ceph-mon[134328]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T03:09:14.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:14 smithi170 ceph-mon[134328]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2023-09-13T03:09:14.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:14 smithi064 ceph-mon[97675]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-09-13T03:09:14.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:14 smithi064 ceph-mon[97675]: from='osd.8 [v2:172.21.15.170:6816/4272513873,v1:172.21.15.170:6817/4272513873]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-13T03:09:14.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:14 smithi064 ceph-mon[97675]: osdmap e56: 9 total, 8 up, 9 in 2023-09-13T03:09:14.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:14 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-13T03:09:14.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:14 smithi064 ceph-mon[97675]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-13T03:09:14.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:14 smithi064 ceph-mon[97675]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T03:09:14.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:14 smithi064 ceph-mon[97675]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2023-09-13T03:09:14.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:14 smithi142 ceph-mon[134082]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-09-13T03:09:14.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:14 smithi142 ceph-mon[134082]: from='osd.8 [v2:172.21.15.170:6816/4272513873,v1:172.21.15.170:6817/4272513873]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-13T03:09:14.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:14 smithi142 ceph-mon[134082]: osdmap e56: 9 total, 8 up, 9 in 2023-09-13T03:09:14.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:14 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-13T03:09:14.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:14 smithi142 ceph-mon[134082]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-13T03:09:14.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:14 smithi142 ceph-mon[134082]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-13T03:09:14.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:14 smithi142 ceph-mon[134082]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2023-09-13T03:09:15.096 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi064:vg_nvme/lv_1 2023-09-13T03:09:15.835 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:09:15.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:15 smithi170 ceph-mon[134328]: osdmap e57: 9 total, 8 up, 9 in 2023-09-13T03:09:15.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:15 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-13T03:09:15.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:15 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-13T03:09:15.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:15 smithi170 ceph-mon[134328]: Detected new or changed devices on smithi170 2023-09-13T03:09:15.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:15 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:15.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:15 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:15.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:15 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:09:15.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:15 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:09:15.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:15 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:09:15.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:15 smithi170 ceph-mon[134328]: Adjusting osd_memory_target on smithi170 to 5722M 2023-09-13T03:09:15.913 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:15 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:15.913 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:15 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:09:15.913 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:15 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:09:15.913 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:15 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:15.913 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:15 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:09:15.914 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:15 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:09:15.914 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:15 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:09:15.914 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:15 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:15.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:15 smithi064 ceph-mon[97675]: osdmap e57: 9 total, 8 up, 9 in 2023-09-13T03:09:15.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:15 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-13T03:09:15.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:15 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-13T03:09:15.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:15 smithi064 ceph-mon[97675]: Detected new or changed devices on smithi170 2023-09-13T03:09:15.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:15 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:15.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:15 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:15.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:15 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:09:15.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:15 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:09:15.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:15 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:09:15.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:15 smithi064 ceph-mon[97675]: Adjusting osd_memory_target on smithi170 to 5722M 2023-09-13T03:09:15.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:15 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:15.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:15 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:09:15.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:15 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:09:15.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:15 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:15.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:15 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:09:15.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:15 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:09:15.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:15 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:09:15.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:15 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:15.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:15 smithi142 ceph-mon[134082]: osdmap e57: 9 total, 8 up, 9 in 2023-09-13T03:09:15.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:15 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-13T03:09:15.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:15 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-13T03:09:15.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:15 smithi142 ceph-mon[134082]: Detected new or changed devices on smithi170 2023-09-13T03:09:15.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:15 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:15.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:15 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:15.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:15 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:09:15.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:15 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:09:15.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:15 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:09:15.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:15 smithi142 ceph-mon[134082]: Adjusting osd_memory_target on smithi170 to 5722M 2023-09-13T03:09:15.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:15 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:15.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:15 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:09:15.998 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:15 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:09:15.998 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:15 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:15.998 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:15 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:09:15.998 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:15 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:09:15.999 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:15 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:09:15.999 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:15 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:16.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:16 smithi064 ceph-mon[97675]: purged_snaps scrub starts 2023-09-13T03:09:16.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:16 smithi064 ceph-mon[97675]: purged_snaps scrub ok 2023-09-13T03:09:16.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:16 smithi064 ceph-mon[97675]: osd.8 [v2:172.21.15.170:6816/4272513873,v1:172.21.15.170:6817/4272513873] boot 2023-09-13T03:09:16.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:16 smithi064 ceph-mon[97675]: osdmap e58: 9 total, 9 up, 9 in 2023-09-13T03:09:16.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:16 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-13T03:09:16.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:16 smithi064 ceph-mon[97675]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-09-13T03:09:16.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:16 smithi170 ceph-mon[134328]: purged_snaps scrub starts 2023-09-13T03:09:16.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:16 smithi170 ceph-mon[134328]: purged_snaps scrub ok 2023-09-13T03:09:16.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:16 smithi170 ceph-mon[134328]: osd.8 [v2:172.21.15.170:6816/4272513873,v1:172.21.15.170:6817/4272513873] boot 2023-09-13T03:09:16.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:16 smithi170 ceph-mon[134328]: osdmap e58: 9 total, 9 up, 9 in 2023-09-13T03:09:16.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:16 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-13T03:09:16.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:16 smithi170 ceph-mon[134328]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-09-13T03:09:16.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:16 smithi142 ceph-mon[134082]: purged_snaps scrub starts 2023-09-13T03:09:16.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:16 smithi142 ceph-mon[134082]: purged_snaps scrub ok 2023-09-13T03:09:16.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:16 smithi142 ceph-mon[134082]: osd.8 [v2:172.21.15.170:6816/4272513873,v1:172.21.15.170:6817/4272513873] boot 2023-09-13T03:09:16.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:16 smithi142 ceph-mon[134082]: osdmap e58: 9 total, 9 up, 9 in 2023-09-13T03:09:16.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:16 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-13T03:09:16.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:16 smithi142 ceph-mon[134082]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-09-13T03:09:17.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:17.243+0000 7f5a00c8e700 1 -- 172.21.15.64:0/4010437222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59fc101890 msgr2=0x7f59fc101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:09:17.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:17.243+0000 7f5a00c8e700 1 --2- 172.21.15.64:0/4010437222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59fc101890 0x7f59fc101cf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f59f00099c0 tx=0x7f59f0009cd0 comp rx=0 tx=0).stop 2023-09-13T03:09:17.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:17.243+0000 7f5a00c8e700 1 -- 172.21.15.64:0/4010437222 shutdown_connections 2023-09-13T03:09:17.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:17.243+0000 7f5a00c8e700 1 --2- 172.21.15.64:0/4010437222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59fc102230 0x7f59fc108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:09:17.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:17.243+0000 7f5a00c8e700 1 --2- 172.21.15.64:0/4010437222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59fc100690 0x7f59fc100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:09:17.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:17.243+0000 7f5a00c8e700 1 --2- 172.21.15.64:0/4010437222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59fc101890 0x7f59fc101cf0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:09:17.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:17.243+0000 7f5a00c8e700 1 -- 172.21.15.64:0/4010437222 >> 172.21.15.64:0/4010437222 conn(0x7f59fc0fbbf0 msgr2=0x7f59fc0fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:09:17.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:17.243+0000 7f5a00c8e700 1 -- 172.21.15.64:0/4010437222 shutdown_connections 2023-09-13T03:09:17.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:17.243+0000 7f5a00c8e700 1 -- 172.21.15.64:0/4010437222 wait complete. 2023-09-13T03:09:17.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:17.244+0000 7f5a00c8e700 1 Processor -- start 2023-09-13T03:09:17.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:17.244+0000 7f5a00c8e700 1 -- start start 2023-09-13T03:09:17.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:17.245+0000 7f5a00c8e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59fc100690 0x7f59fc198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:09:17.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:17.245+0000 7f5a00c8e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59fc101890 0x7f59fc198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:09:17.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:17.245+0000 7f5a00c8e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59fc102230 0x7f59fc19fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:09:17.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:17.245+0000 7f5a00c8e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc1a00d0 con 0x7f59fc100690 2023-09-13T03:09:17.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:17.245+0000 7f5a00c8e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc1a0210 con 0x7f59fc102230 2023-09-13T03:09:17.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:17.245+0000 7f5a00c8e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc1a03a0 con 0x7f59fc101890 2023-09-13T03:09:17.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:17.245+0000 7f59fa59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59fc100690 0x7f59fc198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:09:17.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:17.245+0000 7f59f9d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59fc101890 0x7f59fc198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:09:17.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:17.245+0000 7f59fa59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59fc100690 0x7f59fc198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44786/0 (socket says 172.21.15.64:44786) 2023-09-13T03:09:17.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:17.245+0000 7f59fa59c700 1 -- 172.21.15.64:0/536533133 learned_addr learned my addr 172.21.15.64:0/536533133 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:09:17.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:17.245+0000 7f59fad9d700 1 --2- 172.21.15.64:0/536533133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59fc102230 0x7f59fc19fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:09:17.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:17.245+0000 7f59f9d9b700 1 -- 172.21.15.64:0/536533133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59fc102230 msgr2=0x7f59fc19fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:09:17.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:17.245+0000 7f59f9d9b700 1 --2- 172.21.15.64:0/536533133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59fc102230 0x7f59fc19fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:09:17.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:17.245+0000 7f59f9d9b700 1 -- 172.21.15.64:0/536533133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59fc100690 msgr2=0x7f59fc198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:09:17.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:17.245+0000 7f59f9d9b700 1 --2- 172.21.15.64:0/536533133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59fc100690 0x7f59fc198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:09:17.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:17.245+0000 7f59f9d9b700 1 -- 172.21.15.64:0/536533133 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f00095f0 con 0x7f59fc101890 2023-09-13T03:09:17.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:17.245+0000 7f59fa59c700 1 --2- 172.21.15.64:0/536533133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59fc100690 0x7f59fc198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:09:17.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:17.246+0000 7f59f9d9b700 1 --2- 172.21.15.64:0/536533133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59fc101890 0x7f59fc198a80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f59f00099c0 tx=0x7f59f000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:09:17.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:17.246+0000 7f59eb7fe700 1 -- 172.21.15.64:0/536533133 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f001c070 con 0x7f59fc101890 2023-09-13T03:09:17.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:17.246+0000 7f5a00c8e700 1 -- 172.21.15.64:0/536533133 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59fc1a0620 con 0x7f59fc101890 2023-09-13T03:09:17.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:17.246+0000 7f59eb7fe700 1 -- 172.21.15.64:0/536533133 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59f000fd40 con 0x7f59fc101890 2023-09-13T03:09:17.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:17.246+0000 7f59eb7fe700 1 -- 172.21.15.64:0/536533133 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f0022da0 con 0x7f59fc101890 2023-09-13T03:09:17.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:17.246+0000 7f5a00c8e700 1 -- 172.21.15.64:0/536533133 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59fc1a0a90 con 0x7f59fc101890 2023-09-13T03:09:17.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:17.247+0000 7f5a00c8e700 1 -- 172.21.15.64:0/536533133 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59fc192c80 con 0x7f59fc101890 2023-09-13T03:09:17.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:17.250+0000 7f59eb7fe700 1 -- 172.21.15.64:0/536533133 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f59f0020070 con 0x7f59fc101890 2023-09-13T03:09:17.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:17.251+0000 7f59eb7fe700 1 --2- 172.21.15.64:0/536533133 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f59e40a06c0 0x7f59e40a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:09:17.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:17.251+0000 7f59eb7fe700 1 -- 172.21.15.64:0/536533133 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5858+0+0 (secure 0 0 0) 0x7f59f00cbfd0 con 0x7f59fc101890 2023-09-13T03:09:17.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:17.251+0000 7f59fa59c700 1 --2- 172.21.15.64:0/536533133 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f59e40a06c0 0x7f59e40a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:09:17.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:17.252+0000 7f59fa59c700 1 --2- 172.21.15.64:0/536533133 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f59e40a06c0 0x7f59e40a2b80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f59ec005950 tx=0x7f59ec009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:09:17.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:17.253+0000 7f59eb7fe700 1 -- 172.21.15.64:0/536533133 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f59f005ce10 con 0x7f59fc101890 2023-09-13T03:09:17.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:17.473+0000 7f5a00c8e700 1 -- 172.21.15.64:0/536533133 --> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f59fc02ced0 con 0x7f59e40a06c0 2023-09-13T03:09:17.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:17 smithi170 ceph-mon[134328]: osdmap e59: 9 total, 9 up, 9 in 2023-09-13T03:09:17.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:17 smithi170 ceph-mon[134328]: from='client.24454 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:09:17.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:17 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T03:09:17.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:17 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T03:09:17.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:17 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:09:17.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:17 smithi064 ceph-mon[97675]: osdmap e59: 9 total, 9 up, 9 in 2023-09-13T03:09:17.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:17 smithi064 ceph-mon[97675]: from='client.24454 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:09:17.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:17 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T03:09:17.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:17 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T03:09:17.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:17 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:09:17.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:17 smithi142 ceph-mon[134082]: osdmap e59: 9 total, 9 up, 9 in 2023-09-13T03:09:17.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:17 smithi142 ceph-mon[134082]: from='client.24454 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:09:17.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:17 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T03:09:17.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:17 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T03:09:17.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:17 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:09:18.818 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:18 smithi064 ceph-mon[97675]: osdmap e60: 9 total, 9 up, 9 in 2023-09-13T03:09:18.818 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:18 smithi064 ceph-mon[97675]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-09-13T03:09:18.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:18 smithi170 ceph-mon[134328]: osdmap e60: 9 total, 9 up, 9 in 2023-09-13T03:09:18.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:18 smithi170 ceph-mon[134328]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-09-13T03:09:18.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:18 smithi142 ceph-mon[134082]: osdmap e60: 9 total, 9 up, 9 in 2023-09-13T03:09:18.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:18 smithi142 ceph-mon[134082]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-09-13T03:09:19.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:19 smithi064 ceph-mon[97675]: osdmap e61: 9 total, 9 up, 9 in 2023-09-13T03:09:19.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:19 smithi142 ceph-mon[134082]: osdmap e61: 9 total, 9 up, 9 in 2023-09-13T03:09:20.159 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:19 smithi170 ceph-mon[134328]: osdmap e61: 9 total, 9 up, 9 in 2023-09-13T03:09:20.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:20 smithi142 ceph-mon[134082]: osdmap e62: 9 total, 9 up, 9 in 2023-09-13T03:09:20.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:20 smithi142 ceph-mon[134082]: pgmap v201: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-13T03:09:20.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:20 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/2179178158' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb936bdd-1bec-4236-aa2e-a24adcac11e3"}]: dispatch 2023-09-13T03:09:20.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:20 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/2179178158' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cb936bdd-1bec-4236-aa2e-a24adcac11e3"}]': finished 2023-09-13T03:09:20.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:20 smithi142 ceph-mon[134082]: osdmap e63: 10 total, 9 up, 10 in 2023-09-13T03:09:20.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:20 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-13T03:09:21.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:20 smithi170 ceph-mon[134328]: osdmap e62: 9 total, 9 up, 9 in 2023-09-13T03:09:21.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:20 smithi170 ceph-mon[134328]: pgmap v201: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-13T03:09:21.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:20 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/2179178158' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb936bdd-1bec-4236-aa2e-a24adcac11e3"}]: dispatch 2023-09-13T03:09:21.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:20 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/2179178158' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cb936bdd-1bec-4236-aa2e-a24adcac11e3"}]': finished 2023-09-13T03:09:21.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:20 smithi170 ceph-mon[134328]: osdmap e63: 10 total, 9 up, 10 in 2023-09-13T03:09:21.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:20 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-13T03:09:21.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:20 smithi064 ceph-mon[97675]: osdmap e62: 9 total, 9 up, 9 in 2023-09-13T03:09:21.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:20 smithi064 ceph-mon[97675]: pgmap v201: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-13T03:09:21.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:20 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2179178158' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb936bdd-1bec-4236-aa2e-a24adcac11e3"}]: dispatch 2023-09-13T03:09:21.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:20 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2179178158' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cb936bdd-1bec-4236-aa2e-a24adcac11e3"}]': finished 2023-09-13T03:09:21.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:20 smithi064 ceph-mon[97675]: osdmap e63: 10 total, 9 up, 10 in 2023-09-13T03:09:21.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:20 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-13T03:09:21.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:21 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/123044974' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T03:09:22.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:21 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/123044974' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T03:09:22.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:21 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/123044974' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T03:09:22.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:22 smithi142 ceph-mon[134082]: pgmap v203: 1 pgs: 1 peering; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 121 KiB/s, 0 objects/s recovering 2023-09-13T03:09:23.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:22 smithi170 ceph-mon[134328]: pgmap v203: 1 pgs: 1 peering; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 121 KiB/s, 0 objects/s recovering 2023-09-13T03:09:23.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:22 smithi064 ceph-mon[97675]: pgmap v203: 1 pgs: 1 peering; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 121 KiB/s, 0 objects/s recovering 2023-09-13T03:09:25.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:25 smithi170 ceph-mon[134328]: pgmap v204: 1 pgs: 1 peering; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-13T03:09:25.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:25 smithi064 ceph-mon[97675]: pgmap v204: 1 pgs: 1 peering; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-13T03:09:25.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:25 smithi142 ceph-mon[134082]: pgmap v204: 1 pgs: 1 peering; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-13T03:09:27.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:27 smithi142 ceph-mon[134082]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2023-09-13T03:09:27.767 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:27 smithi064 ceph-mon[97675]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2023-09-13T03:09:27.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:27 smithi170 ceph-mon[134328]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2023-09-13T03:09:28.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:28 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-09-13T03:09:28.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:28 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:09:28.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:28 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-09-13T03:09:28.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:28 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:09:28.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:28 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-09-13T03:09:28.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:28 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:09:29.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:29 smithi064 ceph-mon[97675]: Deploying daemon osd.9 on smithi064 2023-09-13T03:09:29.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:29 smithi064 ceph-mon[97675]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-13T03:09:29.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:29 smithi142 ceph-mon[134082]: Deploying daemon osd.9 on smithi064 2023-09-13T03:09:29.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:29 smithi142 ceph-mon[134082]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-13T03:09:29.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:29 smithi170 ceph-mon[134328]: Deploying daemon osd.9 on smithi064 2023-09-13T03:09:29.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:29 smithi170 ceph-mon[134328]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-13T03:09:31.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:31 smithi142 ceph-mon[134082]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-13T03:09:31.796 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:31 smithi064 ceph-mon[97675]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-13T03:09:31.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:31 smithi170 ceph-mon[134328]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-13T03:09:33.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:33 smithi142 ceph-mon[134082]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-13T03:09:33.806 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:33 smithi064 ceph-mon[97675]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-13T03:09:33.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:33 smithi170 ceph-mon[134328]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-13T03:09:35.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:35 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:35.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:35 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:35.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:35 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:09:35.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:35 smithi142 ceph-mon[134082]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-13T03:09:35.500 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:35 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:35.500 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:35 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:35.500 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:35 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:09:35.501 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:35 smithi064 ceph-mon[97675]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-13T03:09:35.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:35 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:35.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:35 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:35.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:35 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:09:35.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:35 smithi170 ceph-mon[134328]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-13T03:09:37.238 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 9 on host 'smithi064' 2023-09-13T03:09:37.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:37.237+0000 7f59eb7fe700 1 -- 172.21.15.64:0/536533133 <== mgr.14150 v2:172.21.15.64:6800/873679308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f59fc02ced0 con 0x7f59e40a06c0 2023-09-13T03:09:37.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:37.239+0000 7f5a00c8e700 1 -- 172.21.15.64:0/536533133 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f59e40a06c0 msgr2=0x7f59e40a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:09:37.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:37.239+0000 7f5a00c8e700 1 --2- 172.21.15.64:0/536533133 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f59e40a06c0 0x7f59e40a2b80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f59ec005950 tx=0x7f59ec009260 comp rx=0 tx=0).stop 2023-09-13T03:09:37.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:37.239+0000 7f5a00c8e700 1 -- 172.21.15.64:0/536533133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59fc101890 msgr2=0x7f59fc198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:09:37.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:37.239+0000 7f5a00c8e700 1 --2- 172.21.15.64:0/536533133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59fc101890 0x7f59fc198a80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f59f00099c0 tx=0x7f59f000f690 comp rx=0 tx=0).stop 2023-09-13T03:09:37.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:37.239+0000 7f5a00c8e700 1 -- 172.21.15.64:0/536533133 shutdown_connections 2023-09-13T03:09:37.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:37.239+0000 7f5a00c8e700 1 --2- 172.21.15.64:0/536533133 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f59e40a06c0 0x7f59e40a2b80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:09:37.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:37.239+0000 7f5a00c8e700 1 --2- 172.21.15.64:0/536533133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59fc102230 0x7f59fc19fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:09:37.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:37.240+0000 7f5a00c8e700 1 --2- 172.21.15.64:0/536533133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59fc100690 0x7f59fc198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:09:37.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:37.240+0000 7f5a00c8e700 1 --2- 172.21.15.64:0/536533133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59fc101890 0x7f59fc198a80 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:09:37.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:37.240+0000 7f5a00c8e700 1 -- 172.21.15.64:0/536533133 >> 172.21.15.64:0/536533133 conn(0x7f59fc0fbbf0 msgr2=0x7f59fc1071d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:09:37.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:37.240+0000 7f5a00c8e700 1 -- 172.21.15.64:0/536533133 shutdown_connections 2023-09-13T03:09:37.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:09:37.240+0000 7f5a00c8e700 1 -- 172.21.15.64:0/536533133 wait complete. 2023-09-13T03:09:37.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:37 smithi142 ceph-mon[134082]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-13T03:09:37.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:37 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:37.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:37 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:37.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:37 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:37.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:37 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:37.757 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:37 smithi064 ceph-mon[97675]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-13T03:09:37.757 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:37 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:37.757 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:37 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:37.757 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:37 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:37.758 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:37 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:37.759 DEBUG:teuthology.orchestra.run.smithi064:osd.9> sudo journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@osd.9.service 2023-09-13T03:09:37.764 INFO:tasks.cephadm:Deploying osd.10 on smithi142 with /dev/vg_nvme/lv_1... 2023-09-13T03:09:37.764 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-09-13T03:09:37.842 INFO:journalctl@ceph.osd.9.smithi064.stdout:-- Logs begin at Wed 2023-09-13 02:46:56 UTC. -- 2023-09-13T03:09:37.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:37 smithi170 ceph-mon[134328]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-13T03:09:37.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:37 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:37.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:37 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:37.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:37 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:37.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:37 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:38.333 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-13T03:09:38.418 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:09:38 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:09:38.151+0000 7f408c40a640 -1 osd.9 0 log_to_monitors true 2023-09-13T03:09:38.698 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:38 smithi064 ceph-mon[97675]: from='osd.9 [v2:172.21.15.64:6826/2641175211,v1:172.21.15.64:6827/2641175211]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-09-13T03:09:38.744 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:38 smithi142 ceph-mon[134082]: from='osd.9 [v2:172.21.15.64:6826/2641175211,v1:172.21.15.64:6827/2641175211]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-09-13T03:09:38.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:38 smithi170 ceph-mon[134328]: from='osd.9 [v2:172.21.15.64:6826/2641175211,v1:172.21.15.64:6827/2641175211]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-09-13T03:09:39.668 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:09:39 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:09:39.499+0000 7f4081280700 -1 osd.9 0 waiting for initial osdmap 2023-09-13T03:09:39.668 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:09:39 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:09:39.520+0000 7f407b872700 -1 osd.9 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-13T03:09:39.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:39 smithi064 ceph-mon[97675]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-13T03:09:39.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:39 smithi064 ceph-mon[97675]: from='osd.9 [v2:172.21.15.64:6826/2641175211,v1:172.21.15.64:6827/2641175211]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-09-13T03:09:39.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:39 smithi064 ceph-mon[97675]: osdmap e64: 10 total, 9 up, 10 in 2023-09-13T03:09:39.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:39 smithi064 ceph-mon[97675]: from='osd.9 [v2:172.21.15.64:6826/2641175211,v1:172.21.15.64:6827/2641175211]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-09-13T03:09:39.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:39 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-13T03:09:39.832 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-09-13T03:09:39.832 INFO:teuthology.orchestra.run.smithi142.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-09-13T03:09:39.833 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10+0 records in 2023-09-13T03:09:39.833 INFO:teuthology.orchestra.run.smithi142.stderr:10+0 records out 2023-09-13T03:09:39.833 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0123 s, 852 MB/s 2023-09-13T03:09:39.833 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping successful for: 2023-09-13T03:09:39.877 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:39 smithi142 ceph-mon[134082]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-13T03:09:39.877 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:39 smithi142 ceph-mon[134082]: from='osd.9 [v2:172.21.15.64:6826/2641175211,v1:172.21.15.64:6827/2641175211]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-09-13T03:09:39.877 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:39 smithi142 ceph-mon[134082]: osdmap e64: 10 total, 9 up, 10 in 2023-09-13T03:09:39.877 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:39 smithi142 ceph-mon[134082]: from='osd.9 [v2:172.21.15.64:6826/2641175211,v1:172.21.15.64:6827/2641175211]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-09-13T03:09:39.877 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:39 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-13T03:09:39.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:39 smithi170 ceph-mon[134328]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-13T03:09:39.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:39 smithi170 ceph-mon[134328]: from='osd.9 [v2:172.21.15.64:6826/2641175211,v1:172.21.15.64:6827/2641175211]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-09-13T03:09:39.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:39 smithi170 ceph-mon[134328]: osdmap e64: 10 total, 9 up, 10 in 2023-09-13T03:09:39.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:39 smithi170 ceph-mon[134328]: from='osd.9 [v2:172.21.15.64:6826/2641175211,v1:172.21.15.64:6827/2641175211]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-09-13T03:09:39.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:39 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-13T03:09:40.289 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi142:vg_nvme/lv_1 2023-09-13T03:09:40.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:40 smithi142 ceph-mon[134082]: from='osd.9 [v2:172.21.15.64:6826/2641175211,v1:172.21.15.64:6827/2641175211]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2023-09-13T03:09:40.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:40 smithi142 ceph-mon[134082]: osdmap e65: 10 total, 9 up, 10 in 2023-09-13T03:09:40.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:40 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-13T03:09:40.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:40 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-13T03:09:40.763 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:40 smithi064 ceph-mon[97675]: from='osd.9 [v2:172.21.15.64:6826/2641175211,v1:172.21.15.64:6827/2641175211]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2023-09-13T03:09:40.764 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:40 smithi064 ceph-mon[97675]: osdmap e65: 10 total, 9 up, 10 in 2023-09-13T03:09:40.764 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:40 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-13T03:09:40.764 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:40 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-13T03:09:40.843 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-13T03:09:40.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:40 smithi170 ceph-mon[134328]: from='osd.9 [v2:172.21.15.64:6826/2641175211,v1:172.21.15.64:6827/2641175211]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2023-09-13T03:09:40.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:40 smithi170 ceph-mon[134328]: osdmap e65: 10 total, 9 up, 10 in 2023-09-13T03:09:40.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:40 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-13T03:09:40.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:40 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-13T03:09:41.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:41 smithi142 ceph-mon[134082]: purged_snaps scrub starts 2023-09-13T03:09:41.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:41 smithi142 ceph-mon[134082]: purged_snaps scrub ok 2023-09-13T03:09:41.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:41 smithi142 ceph-mon[134082]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-13T03:09:41.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:41 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-13T03:09:41.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:41 smithi142 ceph-mon[134082]: osd.9 [v2:172.21.15.64:6826/2641175211,v1:172.21.15.64:6827/2641175211] boot 2023-09-13T03:09:41.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:41 smithi142 ceph-mon[134082]: osdmap e66: 10 total, 10 up, 10 in 2023-09-13T03:09:41.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:41 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-13T03:09:41.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:41 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:41.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:41 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:41.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:41 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:09:41.748 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:41 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:09:41.748 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:41 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:09:41.748 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:41 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:09:41.748 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:41 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:41.748 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:41 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:09:41.748 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:41 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:09:41.749 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:41 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:41.749 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:41 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:09:41.749 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:41 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:09:41.749 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:41 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:09:41.749 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:41 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:41.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:41 smithi170 ceph-mon[134328]: purged_snaps scrub starts 2023-09-13T03:09:41.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:41 smithi170 ceph-mon[134328]: purged_snaps scrub ok 2023-09-13T03:09:41.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:41 smithi170 ceph-mon[134328]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-13T03:09:41.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:41 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-13T03:09:41.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:41 smithi170 ceph-mon[134328]: osd.9 [v2:172.21.15.64:6826/2641175211,v1:172.21.15.64:6827/2641175211] boot 2023-09-13T03:09:41.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:41 smithi170 ceph-mon[134328]: osdmap e66: 10 total, 10 up, 10 in 2023-09-13T03:09:41.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:41 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-13T03:09:41.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:41 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:41.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:41 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:41.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:41 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:09:41.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:41 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:09:41.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:41 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:09:41.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:41 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:09:41.913 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:41 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:41.913 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:41 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:09:41.913 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:41 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:09:41.913 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:41 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:41.913 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:41 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:09:41.914 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:41 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:09:41.914 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:41 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:09:41.914 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:41 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:41.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:41 smithi064 ceph-mon[97675]: purged_snaps scrub starts 2023-09-13T03:09:41.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:41 smithi064 ceph-mon[97675]: purged_snaps scrub ok 2023-09-13T03:09:41.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:41 smithi064 ceph-mon[97675]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-13T03:09:41.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:41 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-13T03:09:41.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:41 smithi064 ceph-mon[97675]: osd.9 [v2:172.21.15.64:6826/2641175211,v1:172.21.15.64:6827/2641175211] boot 2023-09-13T03:09:41.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:41 smithi064 ceph-mon[97675]: osdmap e66: 10 total, 10 up, 10 in 2023-09-13T03:09:41.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:41 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-13T03:09:41.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:41 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:41.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:41 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:41.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:41 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:09:41.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:41 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:09:41.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:41 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:09:41.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:41 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:09:41.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:41 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:41.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:41 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:09:41.922 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:41 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:09:41.922 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:41 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:41.922 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:41 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:09:41.922 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:41 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:09:41.922 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:41 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:09:41.923 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:41 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:42.026 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:42.024+0000 7f6f9061d700 1 -- 172.21.15.142:0/445632124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f880fed10 msgr2=0x7f6f880ff130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:09:42.027 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:42.024+0000 7f6f9061d700 1 --2- 172.21.15.142:0/445632124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f880fed10 0x7f6f880ff130 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6f780099c0 tx=0x7f6f78009cd0 comp rx=0 tx=0).stop 2023-09-13T03:09:42.027 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:42.024+0000 7f6f9061d700 1 -- 172.21.15.142:0/445632124 shutdown_connections 2023-09-13T03:09:42.027 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:42.024+0000 7f6f9061d700 1 --2- 172.21.15.142:0/445632124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f880fed10 0x7f6f880ff130 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:09:42.027 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:42.024+0000 7f6f9061d700 1 --2- 172.21.15.142:0/445632124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f881008b0 0x7f6f88107160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:09:42.028 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:42.024+0000 7f6f9061d700 1 --2- 172.21.15.142:0/445632124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f880fff10 0x7f6f88100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:09:42.028 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:42.024+0000 7f6f9061d700 1 -- 172.21.15.142:0/445632124 >> 172.21.15.142:0/445632124 conn(0x7f6f880fa2b0 msgr2=0x7f6f880fc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:09:42.028 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:42.024+0000 7f6f9061d700 1 -- 172.21.15.142:0/445632124 shutdown_connections 2023-09-13T03:09:42.028 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:42.024+0000 7f6f9061d700 1 -- 172.21.15.142:0/445632124 wait complete. 2023-09-13T03:09:42.028 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:42.025+0000 7f6f9061d700 1 Processor -- start 2023-09-13T03:09:42.029 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:42.025+0000 7f6f9061d700 1 -- start start 2023-09-13T03:09:42.029 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:42.025+0000 7f6f9061d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f880fed10 0x7f6f88196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:09:42.029 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:42.025+0000 7f6f9061d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f880fff10 0x7f6f88196850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:09:42.029 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:42.025+0000 7f6f9061d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f881008b0 0x7f6f8819d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:09:42.029 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:42.025+0000 7f6f9061d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8819dea0 con 0x7f6f880fff10 2023-09-13T03:09:42.030 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:42.025+0000 7f6f9061d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8819dfe0 con 0x7f6f880fed10 2023-09-13T03:09:42.030 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:42.025+0000 7f6f9061d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8819e170 con 0x7f6f881008b0 2023-09-13T03:09:42.030 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:42.025+0000 7f6f8e3b9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f880fed10 0x7f6f88196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:09:42.030 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:42.025+0000 7f6f8e3b9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f880fed10 0x7f6f88196310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:40814/0 (socket says 172.21.15.142:40814) 2023-09-13T03:09:42.030 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:42.025+0000 7f6f8e3b9700 1 -- 172.21.15.142:0/2318044924 learned_addr learned my addr 172.21.15.142:0/2318044924 (peer_addr_for_me v2:172.21.15.142:0/0) 2023-09-13T03:09:42.031 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:42.025+0000 7f6f8ebba700 1 --2- 172.21.15.142:0/2318044924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f881008b0 0x7f6f8819d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:09:42.031 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:42.025+0000 7f6f8e3b9700 1 -- 172.21.15.142:0/2318044924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f881008b0 msgr2=0x7f6f8819d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:09:42.031 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:42.025+0000 7f6f8e3b9700 1 --2- 172.21.15.142:0/2318044924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f881008b0 0x7f6f8819d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:09:42.031 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:42.025+0000 7f6f8e3b9700 1 -- 172.21.15.142:0/2318044924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f880fff10 msgr2=0x7f6f88196850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:09:42.031 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:42.025+0000 7f6f8e3b9700 1 --2- 172.21.15.142:0/2318044924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f880fff10 0x7f6f88196850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:09:42.032 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:42.025+0000 7f6f8e3b9700 1 -- 172.21.15.142:0/2318044924 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f780095f0 con 0x7f6f880fed10 2023-09-13T03:09:42.032 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:42.025+0000 7f6f8e3b9700 1 --2- 172.21.15.142:0/2318044924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f880fed10 0x7f6f88196310 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6f78009990 tx=0x7f6f78004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:09:42.032 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:42.026+0000 7f6f7f7fe700 1 -- 172.21.15.142:0/2318044924 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7801d070 con 0x7f6f880fed10 2023-09-13T03:09:42.032 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:42.026+0000 7f6f9061d700 1 -- 172.21.15.142:0/2318044924 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f8819e3f0 con 0x7f6f880fed10 2023-09-13T03:09:42.032 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:42.026+0000 7f6f7f7fe700 1 -- 172.21.15.142:0/2318044924 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f78004090 con 0x7f6f880fed10 2023-09-13T03:09:42.033 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:42.026+0000 7f6f7f7fe700 1 -- 172.21.15.142:0/2318044924 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f78022d30 con 0x7f6f880fed10 2023-09-13T03:09:42.033 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:42.026+0000 7f6f9061d700 1 -- 172.21.15.142:0/2318044924 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f8819e890 con 0x7f6f880fed10 2023-09-13T03:09:42.034 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:42.027+0000 7f6f9061d700 1 -- 172.21.15.142:0/2318044924 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f88109c80 con 0x7f6f880fed10 2023-09-13T03:09:42.034 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:42.027+0000 7f6f7f7fe700 1 -- 172.21.15.142:0/2318044924 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f6f78003bd0 con 0x7f6f880fed10 2023-09-13T03:09:42.034 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:42.028+0000 7f6f7f7fe700 1 --2- 172.21.15.142:0/2318044924 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6f740a0640 0x7f6f740a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:09:42.035 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:42.028+0000 7f6f7f7fe700 1 -- 172.21.15.142:0/2318044924 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f6f780cc720 con 0x7f6f880fed10 2023-09-13T03:09:42.035 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:42.028+0000 7f6f8dbb8700 1 --2- 172.21.15.142:0/2318044924 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6f740a0640 0x7f6f740a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:09:42.035 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:42.029+0000 7f6f8dbb8700 1 --2- 172.21.15.142:0/2318044924 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6f740a0640 0x7f6f740a2b00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6f84005950 tx=0x7f6f840058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:09:42.035 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:42.032+0000 7f6f7f7fe700 1 -- 172.21.15.142:0/2318044924 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6f78096ae0 con 0x7f6f880fed10 2023-09-13T03:09:42.231 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:42.230+0000 7f6f9061d700 1 -- 172.21.15.142:0/2318044924 --> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f6f88060a40 con 0x7f6f740a0640 2023-09-13T03:09:42.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:42 smithi170 ceph-mon[134328]: Detected new or changed devices on smithi064 2023-09-13T03:09:42.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:42 smithi170 ceph-mon[134328]: Adjusting osd_memory_target on smithi064 to 4316M 2023-09-13T03:09:42.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:42 smithi170 ceph-mon[134328]: osdmap e67: 10 total, 10 up, 10 in 2023-09-13T03:09:42.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:42 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T03:09:42.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:42 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T03:09:42.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:42 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:09:42.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:42 smithi064 ceph-mon[97675]: Detected new or changed devices on smithi064 2023-09-13T03:09:42.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:42 smithi064 ceph-mon[97675]: Adjusting osd_memory_target on smithi064 to 4316M 2023-09-13T03:09:42.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:42 smithi064 ceph-mon[97675]: osdmap e67: 10 total, 10 up, 10 in 2023-09-13T03:09:42.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:42 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T03:09:42.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:42 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T03:09:42.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:42 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:09:42.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:42 smithi142 ceph-mon[134082]: Detected new or changed devices on smithi064 2023-09-13T03:09:42.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:42 smithi142 ceph-mon[134082]: Adjusting osd_memory_target on smithi064 to 4316M 2023-09-13T03:09:42.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:42 smithi142 ceph-mon[134082]: osdmap e67: 10 total, 10 up, 10 in 2023-09-13T03:09:42.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:42 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T03:09:42.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:42 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T03:09:42.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:42 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:09:43.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:43 smithi142 ceph-mon[134082]: from='client.24476 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:09:43.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:43 smithi142 ceph-mon[134082]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T03:09:43.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:43 smithi170 ceph-mon[134328]: from='client.24476 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:09:43.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:43 smithi170 ceph-mon[134328]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T03:09:43.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:43 smithi064 ceph-mon[97675]: from='client.24476 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:09:43.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:43 smithi064 ceph-mon[97675]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T03:09:45.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:45 smithi064 ceph-mon[97675]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T03:09:45.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:45 smithi064 ceph-mon[97675]: from='client.? 172.21.15.142:0/1333547938' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85512a1c-6171-43d4-9c15-c588a8b71be0"}]: dispatch 2023-09-13T03:09:45.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:45 smithi064 ceph-mon[97675]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85512a1c-6171-43d4-9c15-c588a8b71be0"}]: dispatch 2023-09-13T03:09:45.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:45 smithi064 ceph-mon[97675]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "85512a1c-6171-43d4-9c15-c588a8b71be0"}]': finished 2023-09-13T03:09:45.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:45 smithi064 ceph-mon[97675]: osdmap e68: 11 total, 10 up, 11 in 2023-09-13T03:09:45.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:45 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-13T03:09:45.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:45 smithi142 ceph-mon[134082]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T03:09:45.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:45 smithi142 ceph-mon[134082]: from='client.? 172.21.15.142:0/1333547938' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85512a1c-6171-43d4-9c15-c588a8b71be0"}]: dispatch 2023-09-13T03:09:45.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:45 smithi142 ceph-mon[134082]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85512a1c-6171-43d4-9c15-c588a8b71be0"}]: dispatch 2023-09-13T03:09:45.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:45 smithi142 ceph-mon[134082]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "85512a1c-6171-43d4-9c15-c588a8b71be0"}]': finished 2023-09-13T03:09:45.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:45 smithi142 ceph-mon[134082]: osdmap e68: 11 total, 10 up, 11 in 2023-09-13T03:09:45.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:45 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-13T03:09:45.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:45 smithi170 ceph-mon[134328]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T03:09:45.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:45 smithi170 ceph-mon[134328]: from='client.? 172.21.15.142:0/1333547938' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85512a1c-6171-43d4-9c15-c588a8b71be0"}]: dispatch 2023-09-13T03:09:45.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:45 smithi170 ceph-mon[134328]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85512a1c-6171-43d4-9c15-c588a8b71be0"}]: dispatch 2023-09-13T03:09:45.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:45 smithi170 ceph-mon[134328]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "85512a1c-6171-43d4-9c15-c588a8b71be0"}]': finished 2023-09-13T03:09:45.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:45 smithi170 ceph-mon[134328]: osdmap e68: 11 total, 10 up, 11 in 2023-09-13T03:09:45.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:45 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-13T03:09:46.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:46 smithi064 ceph-mon[97675]: from='client.? 172.21.15.142:0/3086902093' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T03:09:46.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:46 smithi142 ceph-mon[134082]: from='client.? 172.21.15.142:0/3086902093' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T03:09:46.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:46 smithi170 ceph-mon[134328]: from='client.? 172.21.15.142:0/3086902093' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T03:09:47.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:47 smithi142 ceph-mon[134082]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T03:09:47.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:47 smithi170 ceph-mon[134328]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T03:09:47.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:47 smithi064 ceph-mon[97675]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T03:09:49.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:49 smithi142 ceph-mon[134082]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T03:09:49.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:49 smithi170 ceph-mon[134328]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T03:09:49.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:49 smithi064 ceph-mon[97675]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T03:09:51.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:51 smithi142 ceph-mon[134082]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T03:09:51.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:51 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-09-13T03:09:51.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:51 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:09:51.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:51 smithi170 ceph-mon[134328]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T03:09:51.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:51 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-09-13T03:09:51.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:51 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:09:51.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:51 smithi064 ceph-mon[97675]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T03:09:51.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:51 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-09-13T03:09:51.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:51 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:09:52.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:52 smithi142 ceph-mon[134082]: Deploying daemon osd.10 on smithi142 2023-09-13T03:09:52.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:52 smithi170 ceph-mon[134328]: Deploying daemon osd.10 on smithi142 2023-09-13T03:09:52.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:52 smithi064 ceph-mon[97675]: Deploying daemon osd.10 on smithi142 2023-09-13T03:09:53.699 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:53 smithi142 ceph-mon[134082]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T03:09:53.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:53 smithi170 ceph-mon[134328]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T03:09:53.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:53 smithi064 ceph-mon[97675]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T03:09:55.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:55 smithi064 ceph-mon[97675]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T03:09:55.419 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:55 smithi142 ceph-mon[134082]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T03:09:55.659 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:55 smithi170 ceph-mon[134328]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T03:09:56.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:56 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:56.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:56 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:09:56.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:56 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:56.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:56 smithi142 ceph-mon[134082]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T03:09:57.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:56 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:57.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:56 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:09:57.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:56 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:57.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:56 smithi170 ceph-mon[134328]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T03:09:57.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:56 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:57.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:56 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:09:57.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:56 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:57.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:56 smithi064 ceph-mon[97675]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T03:09:58.497 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:58.497+0000 7f6f7f7fe700 1 -- 172.21.15.142:0/2318044924 <== mgr.14150 v2:172.21.15.64:6800/873679308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f6f88060a40 con 0x7f6f740a0640 2023-09-13T03:09:58.497 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 10 on host 'smithi142' 2023-09-13T03:09:58.499 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:58.499+0000 7f6f9061d700 1 -- 172.21.15.142:0/2318044924 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6f740a0640 msgr2=0x7f6f740a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:09:58.499 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:58.499+0000 7f6f9061d700 1 --2- 172.21.15.142:0/2318044924 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6f740a0640 0x7f6f740a2b00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6f84005950 tx=0x7f6f840058e0 comp rx=0 tx=0).stop 2023-09-13T03:09:58.499 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:58.499+0000 7f6f9061d700 1 -- 172.21.15.142:0/2318044924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f880fed10 msgr2=0x7f6f88196310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:09:58.500 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:58.499+0000 7f6f9061d700 1 --2- 172.21.15.142:0/2318044924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f880fed10 0x7f6f88196310 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6f78009990 tx=0x7f6f78004730 comp rx=0 tx=0).stop 2023-09-13T03:09:58.500 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:58.499+0000 7f6f9061d700 1 -- 172.21.15.142:0/2318044924 shutdown_connections 2023-09-13T03:09:58.500 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:58.499+0000 7f6f9061d700 1 --2- 172.21.15.142:0/2318044924 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6f740a0640 0x7f6f740a2b00 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:09:58.500 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:58.499+0000 7f6f9061d700 1 --2- 172.21.15.142:0/2318044924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f880fed10 0x7f6f88196310 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:09:58.501 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:58.499+0000 7f6f9061d700 1 --2- 172.21.15.142:0/2318044924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f881008b0 0x7f6f8819d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:09:58.501 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:58.499+0000 7f6f9061d700 1 --2- 172.21.15.142:0/2318044924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f880fff10 0x7f6f88196850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:09:58.501 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:58.499+0000 7f6f9061d700 1 -- 172.21.15.142:0/2318044924 >> 172.21.15.142:0/2318044924 conn(0x7f6f880fa2b0 msgr2=0x7f6f880fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:09:58.501 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:58.499+0000 7f6f9061d700 1 -- 172.21.15.142:0/2318044924 shutdown_connections 2023-09-13T03:09:58.501 INFO:teuthology.orchestra.run.smithi142.stderr:2023-09-13T03:09:58.499+0000 7f6f9061d700 1 -- 172.21.15.142:0/2318044924 wait complete. 2023-09-13T03:09:58.950 DEBUG:teuthology.orchestra.run.smithi142:osd.10> sudo journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@osd.10.service 2023-09-13T03:09:58.953 INFO:tasks.cephadm:Deploying osd.11 on smithi170 with /dev/vg_nvme/lv_1... 2023-09-13T03:09:58.953 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-09-13T03:09:58.982 INFO:journalctl@ceph.osd.10.smithi142.stdout:-- Logs begin at Wed 2023-09-13 02:43:02 UTC. -- 2023-09-13T03:09:59.524 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-13T03:09:59.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:59 smithi142 ceph-mon[134082]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T03:09:59.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:59 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:59.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:59 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:59.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:59 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:59.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:09:59 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:59.746 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:09:59 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:09:59.641+0000 7f53ed0ca640 -1 osd.10 0 log_to_monitors true 2023-09-13T03:09:59.797 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:59 smithi170 ceph-mon[134328]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T03:09:59.797 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:59 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:59.798 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:59 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:59.798 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:59 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:59.798 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:09:59 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:59.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:59 smithi064 ceph-mon[97675]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T03:09:59.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:59 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:59.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:59 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:59.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:59 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:09:59.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:09:59 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:00.663 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:00 smithi170 ceph-mon[134328]: from='osd.10 [v2:172.21.15.142:6824/936499217,v1:172.21.15.142:6825/936499217]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-13T03:10:00.663 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:00 smithi170 ceph-mon[134328]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-13T03:10:00.663 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:00 smithi170 ceph-mon[134328]: overall HEALTH_OK 2023-09-13T03:10:00.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:00 smithi142 ceph-mon[134082]: from='osd.10 [v2:172.21.15.142:6824/936499217,v1:172.21.15.142:6825/936499217]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-13T03:10:00.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:00 smithi142 ceph-mon[134082]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-13T03:10:00.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:00 smithi142 ceph-mon[134082]: overall HEALTH_OK 2023-09-13T03:10:00.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:00 smithi064 ceph-mon[97675]: from='osd.10 [v2:172.21.15.142:6824/936499217,v1:172.21.15.142:6825/936499217]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-13T03:10:00.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:00 smithi064 ceph-mon[97675]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-13T03:10:00.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:00 smithi064 ceph-mon[97675]: overall HEALTH_OK 2023-09-13T03:10:01.007 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-09-13T03:10:01.008 INFO:teuthology.orchestra.run.smithi170.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-09-13T03:10:01.008 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10+0 records in 2023-09-13T03:10:01.008 INFO:teuthology.orchestra.run.smithi170.stderr:10+0 records out 2023-09-13T03:10:01.008 INFO:teuthology.orchestra.run.smithi170.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00847913 s, 1.2 GB/s 2023-09-13T03:10:01.009 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping successful for: 2023-09-13T03:10:01.378 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi170:vg_nvme/lv_1 2023-09-13T03:10:01.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:01 smithi170 ceph-mon[134328]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T03:10:01.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:01 smithi170 ceph-mon[134328]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-09-13T03:10:01.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:01 smithi170 ceph-mon[134328]: from='osd.10 [v2:172.21.15.142:6824/936499217,v1:172.21.15.142:6825/936499217]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-09-13T03:10:01.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:01 smithi170 ceph-mon[134328]: osdmap e69: 11 total, 10 up, 11 in 2023-09-13T03:10:01.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:01 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-13T03:10:01.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:01 smithi170 ceph-mon[134328]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-09-13T03:10:01.745 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:10:01 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:10:01.464+0000 7f53e1f40700 -1 osd.10 0 waiting for initial osdmap 2023-09-13T03:10:01.745 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:10:01 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:10:01.480+0000 7f53dcd33700 -1 osd.10 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-13T03:10:01.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:01 smithi142 ceph-mon[134082]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T03:10:01.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:01 smithi142 ceph-mon[134082]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-09-13T03:10:01.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:01 smithi142 ceph-mon[134082]: from='osd.10 [v2:172.21.15.142:6824/936499217,v1:172.21.15.142:6825/936499217]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-09-13T03:10:01.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:01 smithi142 ceph-mon[134082]: osdmap e69: 11 total, 10 up, 11 in 2023-09-13T03:10:01.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:01 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-13T03:10:01.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:01 smithi142 ceph-mon[134082]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-09-13T03:10:01.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:01 smithi064 ceph-mon[97675]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T03:10:01.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:01 smithi064 ceph-mon[97675]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-09-13T03:10:01.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:01 smithi064 ceph-mon[97675]: from='osd.10 [v2:172.21.15.142:6824/936499217,v1:172.21.15.142:6825/936499217]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-09-13T03:10:01.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:01 smithi064 ceph-mon[97675]: osdmap e69: 11 total, 10 up, 11 in 2023-09-13T03:10:01.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:01 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-13T03:10:01.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:01 smithi064 ceph-mon[97675]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2023-09-13T03:10:01.937 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-13T03:10:02.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:02 smithi142 ceph-mon[134082]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2023-09-13T03:10:02.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:02 smithi142 ceph-mon[134082]: osdmap e70: 11 total, 10 up, 11 in 2023-09-13T03:10:02.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:02 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-13T03:10:02.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:02 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-13T03:10:02.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:02 smithi142 ceph-mon[134082]: Detected new or changed devices on smithi142 2023-09-13T03:10:02.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:02 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:02.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:02 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:02.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:02 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:10:02.748 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:02 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:10:02.748 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:02 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:10:02.748 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:02 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:10:02.748 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:02 smithi142 ceph-mon[134082]: Adjusting osd_memory_target on smithi142 to 4292M 2023-09-13T03:10:02.748 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:02 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:02.749 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:02 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:02.749 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:02 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:10:02.749 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:02 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:02.749 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:02 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:10:02.749 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:02 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:02.750 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:02 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:10:02.750 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:02 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:02.750 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:02 smithi142 ceph-mon[134082]: osd.10 [v2:172.21.15.142:6824/936499217,v1:172.21.15.142:6825/936499217] boot 2023-09-13T03:10:02.750 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:02 smithi142 ceph-mon[134082]: osdmap e71: 11 total, 11 up, 11 in 2023-09-13T03:10:02.750 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:02 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-13T03:10:02.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:02 smithi170 ceph-mon[134328]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2023-09-13T03:10:02.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:02 smithi170 ceph-mon[134328]: osdmap e70: 11 total, 10 up, 11 in 2023-09-13T03:10:02.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-13T03:10:02.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-13T03:10:02.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:02 smithi170 ceph-mon[134328]: Detected new or changed devices on smithi142 2023-09-13T03:10:02.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:02.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:02.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:10:02.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:10:02.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:10:02.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:10:02.913 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:02 smithi170 ceph-mon[134328]: Adjusting osd_memory_target on smithi142 to 4292M 2023-09-13T03:10:02.913 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:02.913 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:02.913 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:10:02.913 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:02.913 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:10:02.914 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:02.914 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:10:02.914 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:02.914 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:02 smithi170 ceph-mon[134328]: osd.10 [v2:172.21.15.142:6824/936499217,v1:172.21.15.142:6825/936499217] boot 2023-09-13T03:10:02.915 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:02 smithi170 ceph-mon[134328]: osdmap e71: 11 total, 11 up, 11 in 2023-09-13T03:10:02.915 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-13T03:10:02.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:02 smithi064 ceph-mon[97675]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2023-09-13T03:10:02.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:02 smithi064 ceph-mon[97675]: osdmap e70: 11 total, 10 up, 11 in 2023-09-13T03:10:02.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:02 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-13T03:10:02.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:02 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-13T03:10:02.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:02 smithi064 ceph-mon[97675]: Detected new or changed devices on smithi142 2023-09-13T03:10:02.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:02 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:02.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:02 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:02.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:02 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:10:02.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:02 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:10:02.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:02 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:10:02.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:02 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:10:02.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:02 smithi064 ceph-mon[97675]: Adjusting osd_memory_target on smithi142 to 4292M 2023-09-13T03:10:02.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:02 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:02.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:02 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:02.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:02 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:10:02.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:02 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:02.922 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:02 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:10:02.922 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:02 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:02.922 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:02 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:10:02.922 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:02 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:02.922 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:02 smithi064 ceph-mon[97675]: osd.10 [v2:172.21.15.142:6824/936499217,v1:172.21.15.142:6825/936499217] boot 2023-09-13T03:10:02.923 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:02 smithi064 ceph-mon[97675]: osdmap e71: 11 total, 11 up, 11 in 2023-09-13T03:10:02.923 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:02 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-13T03:10:03.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:03.123+0000 7ff868ae1700 1 -- 172.21.15.170:0/3748729783 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8640ff590 msgr2=0x7ff8640ff9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:03.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:03.123+0000 7ff868ae1700 1 --2- 172.21.15.170:0/3748729783 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8640ff590 0x7ff8640ff9b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff858009990 tx=0x7ff858009ca0 comp rx=0 tx=0).stop 2023-09-13T03:10:03.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:03.124+0000 7ff868ae1700 1 -- 172.21.15.170:0/3748729783 shutdown_connections 2023-09-13T03:10:03.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:03.124+0000 7ff868ae1700 1 --2- 172.21.15.170:0/3748729783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff864101130 0x7ff8641079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:03.126 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:03.124+0000 7ff868ae1700 1 --2- 172.21.15.170:0/3748729783 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8640ff590 0x7ff8640ff9b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:03.126 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:03.124+0000 7ff868ae1700 1 --2- 172.21.15.170:0/3748729783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff864100790 0x7ff864100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:03.126 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:03.124+0000 7ff868ae1700 1 -- 172.21.15.170:0/3748729783 >> 172.21.15.170:0/3748729783 conn(0x7ff8640fab10 msgr2=0x7ff8640fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:10:03.126 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:03.124+0000 7ff868ae1700 1 -- 172.21.15.170:0/3748729783 shutdown_connections 2023-09-13T03:10:03.126 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:03.124+0000 7ff868ae1700 1 -- 172.21.15.170:0/3748729783 wait complete. 2023-09-13T03:10:03.128 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:03.124+0000 7ff868ae1700 1 Processor -- start 2023-09-13T03:10:03.128 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:03.124+0000 7ff868ae1700 1 -- start start 2023-09-13T03:10:03.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:03.125+0000 7ff868ae1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8640ff590 0x7ff8641951d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:03.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:03.125+0000 7ff868ae1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff864100790 0x7ff864195710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:03.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:03.125+0000 7ff868ae1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff864101130 0x7ff86419c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:03.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:03.125+0000 7ff868ae1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff86419cd60 con 0x7ff864100790 2023-09-13T03:10:03.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:03.125+0000 7ff868ae1700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff86419cea0 con 0x7ff8640ff590 2023-09-13T03:10:03.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:03.125+0000 7ff868ae1700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff86419d030 con 0x7ff864101130 2023-09-13T03:10:03.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:03.125+0000 7ff862d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff864101130 0x7ff86419c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:03.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:03.125+0000 7ff86259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8640ff590 0x7ff8641951d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:03.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:03.125+0000 7ff86259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8640ff590 0x7ff8641951d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.170:52324/0 (socket says 172.21.15.170:52324) 2023-09-13T03:10:03.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:03.125+0000 7ff86259c700 1 -- 172.21.15.170:0/1211739013 learned_addr learned my addr 172.21.15.170:0/1211739013 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-09-13T03:10:03.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:03.125+0000 7ff862d9d700 1 -- 172.21.15.170:0/1211739013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8640ff590 msgr2=0x7ff8641951d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:03.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:03.125+0000 7ff862d9d700 1 --2- 172.21.15.170:0/1211739013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8640ff590 0x7ff8641951d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:03.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:03.125+0000 7ff862d9d700 1 -- 172.21.15.170:0/1211739013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff864100790 msgr2=0x7ff864195710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:03.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:03.125+0000 7ff862d9d700 1 --2- 172.21.15.170:0/1211739013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff864100790 0x7ff864195710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:03.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:03.125+0000 7ff862d9d700 1 -- 172.21.15.170:0/1211739013 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8580095f0 con 0x7ff864101130 2023-09-13T03:10:03.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:03.125+0000 7ff862d9d700 1 --2- 172.21.15.170:0/1211739013 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff864101130 0x7ff86419c790 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff85c00cc80 tx=0x7ff85c00cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:10:03.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:03.125+0000 7ff8537fe700 1 -- 172.21.15.170:0/1211739013 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85c00ab30 con 0x7ff864101130 2023-09-13T03:10:03.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:03.125+0000 7ff868ae1700 1 -- 172.21.15.170:0/1211739013 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff86419d310 con 0x7ff864101130 2023-09-13T03:10:03.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:03.125+0000 7ff8537fe700 1 -- 172.21.15.170:0/1211739013 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff85c00ac90 con 0x7ff864101130 2023-09-13T03:10:03.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:03.125+0000 7ff8537fe700 1 -- 172.21.15.170:0/1211739013 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85c0115e0 con 0x7ff864101130 2023-09-13T03:10:03.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:03.125+0000 7ff868ae1700 1 -- 172.21.15.170:0/1211739013 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff86419d860 con 0x7ff864101130 2023-09-13T03:10:03.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:03.126+0000 7ff868ae1700 1 -- 172.21.15.170:0/1211739013 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff86410ada0 con 0x7ff864101130 2023-09-13T03:10:03.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:03.127+0000 7ff8537fe700 1 -- 172.21.15.170:0/1211739013 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7ff85c012420 con 0x7ff864101130 2023-09-13T03:10:03.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:03.128+0000 7ff8537fe700 1 --2- 172.21.15.170:0/1211739013 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ff84c0a0640 0x7ff84c0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:03.134 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:03.128+0000 7ff8537fe700 1 -- 172.21.15.170:0/1211739013 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6727+0+0 (secure 0 0 0) 0x7ff85c0cb6f0 con 0x7ff864101130 2023-09-13T03:10:03.134 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:03.128+0000 7ff86259c700 1 --2- 172.21.15.170:0/1211739013 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ff84c0a0640 0x7ff84c0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:03.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:03.129+0000 7ff86259c700 1 --2- 172.21.15.170:0/1211739013 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ff84c0a0640 0x7ff84c0a2b00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff858000c00 tx=0x7ff858005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:10:03.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:03.132+0000 7ff8537fe700 1 -- 172.21.15.170:0/1211739013 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ff85c095850 con 0x7ff864101130 2023-09-13T03:10:03.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:03.331+0000 7ff868ae1700 1 -- 172.21.15.170:0/1211739013 --> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7ff864068750 con 0x7ff84c0a0640 2023-09-13T03:10:03.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:03 smithi142 ceph-mon[134082]: purged_snaps scrub starts 2023-09-13T03:10:03.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:03 smithi142 ceph-mon[134082]: purged_snaps scrub ok 2023-09-13T03:10:03.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:03 smithi142 ceph-mon[134082]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T03:10:03.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:03 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T03:10:03.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:03 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T03:10:03.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:03 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:03.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:03 smithi170 ceph-mon[134328]: purged_snaps scrub starts 2023-09-13T03:10:03.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:03 smithi170 ceph-mon[134328]: purged_snaps scrub ok 2023-09-13T03:10:03.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:03 smithi170 ceph-mon[134328]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T03:10:03.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:03 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T03:10:03.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:03 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T03:10:03.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:03 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:03.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:03 smithi064 ceph-mon[97675]: purged_snaps scrub starts 2023-09-13T03:10:03.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:03 smithi064 ceph-mon[97675]: purged_snaps scrub ok 2023-09-13T03:10:03.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:03 smithi064 ceph-mon[97675]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-13T03:10:03.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:03 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-13T03:10:03.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:03 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-13T03:10:03.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:03 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:04.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:04 smithi142 ceph-mon[134082]: from='client.24499 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:10:04.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:04 smithi142 ceph-mon[134082]: osdmap e72: 11 total, 11 up, 11 in 2023-09-13T03:10:04.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:04 smithi170 ceph-mon[134328]: from='client.24499 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:10:04.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:04 smithi170 ceph-mon[134328]: osdmap e72: 11 total, 11 up, 11 in 2023-09-13T03:10:04.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:04 smithi064 ceph-mon[97675]: from='client.24499 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:10:04.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:04 smithi064 ceph-mon[97675]: osdmap e72: 11 total, 11 up, 11 in 2023-09-13T03:10:05.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:05 smithi142 ceph-mon[134082]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-13T03:10:05.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:05 smithi142 ceph-mon[134082]: osdmap e73: 11 total, 11 up, 11 in 2023-09-13T03:10:05.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:05 smithi170 ceph-mon[134328]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-13T03:10:05.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:05 smithi170 ceph-mon[134328]: osdmap e73: 11 total, 11 up, 11 in 2023-09-13T03:10:05.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:05 smithi064 ceph-mon[97675]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-13T03:10:05.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:05 smithi064 ceph-mon[97675]: osdmap e73: 11 total, 11 up, 11 in 2023-09-13T03:10:06.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:06 smithi142 ceph-mon[134082]: osdmap e74: 11 total, 11 up, 11 in 2023-09-13T03:10:06.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:06 smithi142 ceph-mon[134082]: from='client.? 172.21.15.170:0/1948977133' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b8417dc-5d6c-4917-94a9-b032c5bc33d6"}]: dispatch 2023-09-13T03:10:06.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:06 smithi142 ceph-mon[134082]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b8417dc-5d6c-4917-94a9-b032c5bc33d6"}]: dispatch 2023-09-13T03:10:06.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:06 smithi142 ceph-mon[134082]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9b8417dc-5d6c-4917-94a9-b032c5bc33d6"}]': finished 2023-09-13T03:10:06.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:06 smithi142 ceph-mon[134082]: osdmap e75: 12 total, 11 up, 12 in 2023-09-13T03:10:06.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:06 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-13T03:10:06.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:06 smithi142 ceph-mon[134082]: from='client.? 172.21.15.170:0/2110953574' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T03:10:06.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:06 smithi170 ceph-mon[134328]: osdmap e74: 11 total, 11 up, 11 in 2023-09-13T03:10:06.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:06 smithi170 ceph-mon[134328]: from='client.? 172.21.15.170:0/1948977133' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b8417dc-5d6c-4917-94a9-b032c5bc33d6"}]: dispatch 2023-09-13T03:10:06.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:06 smithi170 ceph-mon[134328]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b8417dc-5d6c-4917-94a9-b032c5bc33d6"}]: dispatch 2023-09-13T03:10:06.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:06 smithi170 ceph-mon[134328]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9b8417dc-5d6c-4917-94a9-b032c5bc33d6"}]': finished 2023-09-13T03:10:06.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:06 smithi170 ceph-mon[134328]: osdmap e75: 12 total, 11 up, 12 in 2023-09-13T03:10:06.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:06 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-13T03:10:06.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:06 smithi170 ceph-mon[134328]: from='client.? 172.21.15.170:0/2110953574' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T03:10:06.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:06 smithi064 ceph-mon[97675]: osdmap e74: 11 total, 11 up, 11 in 2023-09-13T03:10:06.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:06 smithi064 ceph-mon[97675]: from='client.? 172.21.15.170:0/1948977133' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b8417dc-5d6c-4917-94a9-b032c5bc33d6"}]: dispatch 2023-09-13T03:10:06.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:06 smithi064 ceph-mon[97675]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b8417dc-5d6c-4917-94a9-b032c5bc33d6"}]: dispatch 2023-09-13T03:10:06.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:06 smithi064 ceph-mon[97675]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9b8417dc-5d6c-4917-94a9-b032c5bc33d6"}]': finished 2023-09-13T03:10:06.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:06 smithi064 ceph-mon[97675]: osdmap e75: 12 total, 11 up, 12 in 2023-09-13T03:10:06.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:06 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-13T03:10:06.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:06 smithi064 ceph-mon[97675]: from='client.? 172.21.15.170:0/2110953574' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-13T03:10:07.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:07 smithi170 ceph-mon[134328]: pgmap v237: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2023-09-13T03:10:07.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:07 smithi170 ceph-mon[134328]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2023-09-13T03:10:07.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:07 smithi064 ceph-mon[97675]: pgmap v237: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2023-09-13T03:10:07.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:07 smithi064 ceph-mon[97675]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2023-09-13T03:10:07.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:07 smithi142 ceph-mon[134082]: pgmap v237: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2023-09-13T03:10:07.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:07 smithi142 ceph-mon[134082]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2023-09-13T03:10:09.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:09 smithi170 ceph-mon[134328]: pgmap v238: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2023-09-13T03:10:09.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:09 smithi064 ceph-mon[97675]: pgmap v238: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2023-09-13T03:10:09.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:09 smithi142 ceph-mon[134082]: pgmap v238: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2023-09-13T03:10:10.824 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:10 smithi170 ceph-mon[134328]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2023-09-13T03:10:10.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:10 smithi170 ceph-mon[134328]: Cluster is now healthy 2023-09-13T03:10:10.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:10 smithi064 ceph-mon[97675]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2023-09-13T03:10:10.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:10 smithi064 ceph-mon[97675]: Cluster is now healthy 2023-09-13T03:10:10.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:10 smithi142 ceph-mon[134082]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2023-09-13T03:10:10.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:10 smithi142 ceph-mon[134082]: Cluster is now healthy 2023-09-13T03:10:11.857 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:11 smithi170 ceph-mon[134328]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-13T03:10:11.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:11 smithi064 ceph-mon[97675]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-13T03:10:11.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:11 smithi142 ceph-mon[134082]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-13T03:10:12.812 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:12 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-09-13T03:10:12.812 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:12 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:12.836 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:12 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-09-13T03:10:12.836 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:12 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:12.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:12 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-09-13T03:10:12.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:12 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:13.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:13 smithi170 ceph-mon[134328]: Deploying daemon osd.11 on smithi170 2023-09-13T03:10:13.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:13 smithi170 ceph-mon[134328]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-13T03:10:13.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:13 smithi064 ceph-mon[97675]: Deploying daemon osd.11 on smithi170 2023-09-13T03:10:13.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:13 smithi064 ceph-mon[97675]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-13T03:10:13.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:13 smithi142 ceph-mon[134082]: Deploying daemon osd.11 on smithi170 2023-09-13T03:10:13.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:13 smithi142 ceph-mon[134082]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-13T03:10:15.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:15 smithi170 ceph-mon[134328]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-09-13T03:10:15.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:15 smithi142 ceph-mon[134082]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-09-13T03:10:15.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:15 smithi064 ceph-mon[97675]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-09-13T03:10:17.659 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:17 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:17.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:17 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:17.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:17 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:10:17.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:17 smithi170 ceph-mon[134328]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2023-09-13T03:10:17.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:17 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:17.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:17 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:17.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:17 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:10:17.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:17 smithi142 ceph-mon[134082]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2023-09-13T03:10:17.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:17 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:17.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:17 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:17.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:17 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:10:17.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:17 smithi064 ceph-mon[97675]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2023-09-13T03:10:19.198 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:19.197+0000 7ff8537fe700 1 -- 172.21.15.170:0/1211739013 <== mgr.14150 v2:172.21.15.64:6800/873679308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7ff864068750 con 0x7ff84c0a0640 2023-09-13T03:10:19.199 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 11 on host 'smithi170' 2023-09-13T03:10:19.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:19.199+0000 7ff868ae1700 1 -- 172.21.15.170:0/1211739013 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ff84c0a0640 msgr2=0x7ff84c0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:19.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:19.199+0000 7ff868ae1700 1 --2- 172.21.15.170:0/1211739013 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ff84c0a0640 0x7ff84c0a2b00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff858000c00 tx=0x7ff858005fb0 comp rx=0 tx=0).stop 2023-09-13T03:10:19.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:19.199+0000 7ff868ae1700 1 -- 172.21.15.170:0/1211739013 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff864101130 msgr2=0x7ff86419c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:19.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:19.199+0000 7ff868ae1700 1 --2- 172.21.15.170:0/1211739013 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff864101130 0x7ff86419c790 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff85c00cc80 tx=0x7ff85c00cf90 comp rx=0 tx=0).stop 2023-09-13T03:10:19.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:19.199+0000 7ff868ae1700 1 -- 172.21.15.170:0/1211739013 shutdown_connections 2023-09-13T03:10:19.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:19.199+0000 7ff868ae1700 1 --2- 172.21.15.170:0/1211739013 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ff84c0a0640 0x7ff84c0a2b00 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:19.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:19.199+0000 7ff868ae1700 1 --2- 172.21.15.170:0/1211739013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8640ff590 0x7ff8641951d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:19.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:19.199+0000 7ff868ae1700 1 --2- 172.21.15.170:0/1211739013 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff864101130 0x7ff86419c790 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:19.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:19.199+0000 7ff868ae1700 1 --2- 172.21.15.170:0/1211739013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff864100790 0x7ff864195710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:19.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:19.199+0000 7ff868ae1700 1 -- 172.21.15.170:0/1211739013 >> 172.21.15.170:0/1211739013 conn(0x7ff8640fab10 msgr2=0x7ff8640fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:10:19.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:19.200+0000 7ff868ae1700 1 -- 172.21.15.170:0/1211739013 shutdown_connections 2023-09-13T03:10:19.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:19.200+0000 7ff868ae1700 1 -- 172.21.15.170:0/1211739013 wait complete. 2023-09-13T03:10:19.613 DEBUG:teuthology.orchestra.run.smithi170:osd.11> sudo journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@osd.11.service 2023-09-13T03:10:19.616 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-09-13T03:10:19.617 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph osd stat -f json 2023-09-13T03:10:19.649 INFO:journalctl@ceph.osd.11.smithi170.stdout:-- Logs begin at Wed 2023-09-13 02:42:51 UTC. -- 2023-09-13T03:10:19.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:19 smithi142 ceph-mon[134082]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-09-13T03:10:19.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:19 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:19.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:19 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:19.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:19 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:19.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:19 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:19.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:19 smithi170 ceph-mon[134328]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-09-13T03:10:19.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:19 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:19.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:19 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:19.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:19 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:19.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:19 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:19.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:19 smithi064 ceph-mon[97675]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-09-13T03:10:19.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:19 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:19.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:19 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:19.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:19 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:19.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:19 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:20.363 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:10:20.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:20 smithi142 ceph-mon[134082]: from='osd.11 [v2:172.21.15.170:6824/3605017060,v1:172.21.15.170:6825/3605017060]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-13T03:10:20.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:20 smithi142 ceph-mon[134082]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-13T03:10:20.788 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:20 smithi170 ceph-mon[134328]: from='osd.11 [v2:172.21.15.170:6824/3605017060,v1:172.21.15.170:6825/3605017060]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-13T03:10:20.789 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:20 smithi170 ceph-mon[134328]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-13T03:10:20.789 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:10:20 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:10:20.430+0000 7f89cab2e640 -1 osd.11 0 log_to_monitors true 2023-09-13T03:10:20.801 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:20 smithi064 ceph-mon[97675]: from='osd.11 [v2:172.21.15.170:6824/3605017060,v1:172.21.15.170:6825/3605017060]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-13T03:10:20.801 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:20 smithi064 ceph-mon[97675]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-13T03:10:21.660 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:10:21 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:10:21.501+0000 7f89bf9a4700 -1 osd.11 0 waiting for initial osdmap 2023-09-13T03:10:21.660 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:10:21 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:10:21.517+0000 7f89b9795700 -1 osd.11 77 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-13T03:10:21.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:21 smithi170 ceph-mon[134328]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-09-13T03:10:21.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:21 smithi170 ceph-mon[134328]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-09-13T03:10:21.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:21 smithi170 ceph-mon[134328]: from='osd.11 [v2:172.21.15.170:6824/3605017060,v1:172.21.15.170:6825/3605017060]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-13T03:10:21.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:21 smithi170 ceph-mon[134328]: osdmap e76: 12 total, 11 up, 12 in 2023-09-13T03:10:21.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:21 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-13T03:10:21.662 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:21 smithi170 ceph-mon[134328]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-13T03:10:21.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.706+0000 7f4037eb1700 1 -- 172.21.15.64:0/16775116 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4030101890 msgr2=0x7f4030101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:21.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.706+0000 7f4037eb1700 1 --2- 172.21.15.64:0/16775116 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4030101890 0x7f4030101cf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4020009990 tx=0x7f4020009ca0 comp rx=0 tx=0).stop 2023-09-13T03:10:21.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.706+0000 7f4037eb1700 1 -- 172.21.15.64:0/16775116 shutdown_connections 2023-09-13T03:10:21.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.706+0000 7f4037eb1700 1 --2- 172.21.15.64:0/16775116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4030102230 0x7f4030108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:21.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.706+0000 7f4037eb1700 1 --2- 172.21.15.64:0/16775116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4030100690 0x7f4030100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:21.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.706+0000 7f4037eb1700 1 --2- 172.21.15.64:0/16775116 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4030101890 0x7f4030101cf0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:21.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.706+0000 7f4037eb1700 1 -- 172.21.15.64:0/16775116 >> 172.21.15.64:0/16775116 conn(0x7f40300fbbf0 msgr2=0x7f40300fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:10:21.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.706+0000 7f4037eb1700 1 -- 172.21.15.64:0/16775116 shutdown_connections 2023-09-13T03:10:21.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.706+0000 7f4037eb1700 1 -- 172.21.15.64:0/16775116 wait complete. 2023-09-13T03:10:21.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.707+0000 7f4037eb1700 1 Processor -- start 2023-09-13T03:10:21.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.708+0000 7f4037eb1700 1 -- start start 2023-09-13T03:10:21.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.708+0000 7f4037eb1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4030100690 0x7f40300743a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:21.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.708+0000 7f4037eb1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4030101890 0x7f40300729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:21.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.708+0000 7f4037eb1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4030102230 0x7f4030072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:21.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.708+0000 7f4037eb1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40300736b0 con 0x7f4030102230 2023-09-13T03:10:21.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.708+0000 7f4037eb1700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4030073820 con 0x7f4030100690 2023-09-13T03:10:21.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.708+0000 7f4037eb1700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40301a03d0 con 0x7f4030101890 2023-09-13T03:10:21.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.709+0000 7f403544c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4030101890 0x7f40300729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:21.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.709+0000 7f403544c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4030101890 0x7f40300729f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:44266/0 (socket says 172.21.15.64:44266) 2023-09-13T03:10:21.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.709+0000 7f403544c700 1 -- 172.21.15.64:0/803128474 learned_addr learned my addr 172.21.15.64:0/803128474 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:10:21.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.709+0000 7f4035c4d700 1 --2- 172.21.15.64:0/803128474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4030100690 0x7f40300743a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:21.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.709+0000 7f403644e700 1 --2- 172.21.15.64:0/803128474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4030102230 0x7f4030072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:21.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.709+0000 7f403544c700 1 -- 172.21.15.64:0/803128474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4030100690 msgr2=0x7f40300743a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:21.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.709+0000 7f403544c700 1 --2- 172.21.15.64:0/803128474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4030100690 0x7f40300743a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:21.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.709+0000 7f403544c700 1 -- 172.21.15.64:0/803128474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4030102230 msgr2=0x7f4030072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:21.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.709+0000 7f403544c700 1 --2- 172.21.15.64:0/803128474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4030102230 0x7f4030072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:21.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.709+0000 7f403544c700 1 -- 172.21.15.64:0/803128474 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40200095f0 con 0x7f4030101890 2023-09-13T03:10:21.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.709+0000 7f4035c4d700 1 --2- 172.21.15.64:0/803128474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4030100690 0x7f40300743a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:10:21.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.709+0000 7f403644e700 1 --2- 172.21.15.64:0/803128474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4030102230 0x7f4030072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:10:21.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.709+0000 7f403544c700 1 --2- 172.21.15.64:0/803128474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4030101890 0x7f40300729f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f402000b680 tx=0x7f4020005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:10:21.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.710+0000 7f4026ffd700 1 -- 172.21.15.64:0/803128474 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402001d070 con 0x7f4030101890 2023-09-13T03:10:21.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.710+0000 7f4026ffd700 1 -- 172.21.15.64:0/803128474 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4020022470 con 0x7f4030101890 2023-09-13T03:10:21.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.710+0000 7f4037eb1700 1 -- 172.21.15.64:0/803128474 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40301a0650 con 0x7f4030101890 2023-09-13T03:10:21.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.710+0000 7f4026ffd700 1 -- 172.21.15.64:0/803128474 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4020017650 con 0x7f4030101890 2023-09-13T03:10:21.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.710+0000 7f4037eb1700 1 -- 172.21.15.64:0/803128474 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40301a0ac0 con 0x7f4030101890 2023-09-13T03:10:21.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.711+0000 7f4037eb1700 1 -- 172.21.15.64:0/803128474 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4030107c90 con 0x7f4030101890 2023-09-13T03:10:21.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.713+0000 7f4026ffd700 1 -- 172.21.15.64:0/803128474 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f40200225e0 con 0x7f4030101890 2023-09-13T03:10:21.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.714+0000 7f4026ffd700 1 --2- 172.21.15.64:0/803128474 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f401c0a06c0 0x7f401c0a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:21.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.714+0000 7f4026ffd700 1 -- 172.21.15.64:0/803128474 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f40200cd2e0 con 0x7f4030101890 2023-09-13T03:10:21.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.714+0000 7f4035c4d700 1 --2- 172.21.15.64:0/803128474 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f401c0a06c0 0x7f401c0a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:21.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.715+0000 7f4035c4d700 1 --2- 172.21.15.64:0/803128474 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f401c0a06c0 0x7f401c0a2b80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f40301016f0 tx=0x7f402c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:10:21.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.716+0000 7f4026ffd700 1 -- 172.21.15.64:0/803128474 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f4020097440 con 0x7f4030101890 2023-09-13T03:10:21.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:21 smithi064 ceph-mon[97675]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-09-13T03:10:21.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:21 smithi064 ceph-mon[97675]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-09-13T03:10:21.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:21 smithi064 ceph-mon[97675]: from='osd.11 [v2:172.21.15.170:6824/3605017060,v1:172.21.15.170:6825/3605017060]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-13T03:10:21.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:21 smithi064 ceph-mon[97675]: osdmap e76: 12 total, 11 up, 12 in 2023-09-13T03:10:21.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:21 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-13T03:10:21.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:21 smithi064 ceph-mon[97675]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-13T03:10:21.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.923+0000 7f4037eb1700 1 -- 172.21.15.64:0/803128474 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f403010c130 con 0x7f4030101890 2023-09-13T03:10:21.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.924+0000 7f4026ffd700 1 -- 172.21.15.64:0/803128474 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v77) v1 ==== 74+0+133 (secure 0 0 0) 0x7f402009aa60 con 0x7f4030101890 2023-09-13T03:10:21.925 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:10:21.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.926+0000 7f4037eb1700 1 -- 172.21.15.64:0/803128474 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f401c0a06c0 msgr2=0x7f401c0a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:21.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.926+0000 7f4037eb1700 1 --2- 172.21.15.64:0/803128474 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f401c0a06c0 0x7f401c0a2b80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f40301016f0 tx=0x7f402c009260 comp rx=0 tx=0).stop 2023-09-13T03:10:21.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.926+0000 7f4037eb1700 1 -- 172.21.15.64:0/803128474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4030101890 msgr2=0x7f40300729f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:21.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.926+0000 7f4037eb1700 1 --2- 172.21.15.64:0/803128474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4030101890 0x7f40300729f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f402000b680 tx=0x7f4020005740 comp rx=0 tx=0).stop 2023-09-13T03:10:21.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.926+0000 7f4037eb1700 1 -- 172.21.15.64:0/803128474 shutdown_connections 2023-09-13T03:10:21.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.926+0000 7f4037eb1700 1 --2- 172.21.15.64:0/803128474 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f401c0a06c0 0x7f401c0a2b80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:21.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.926+0000 7f4037eb1700 1 --2- 172.21.15.64:0/803128474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4030100690 0x7f40300743a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:21.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.926+0000 7f4037eb1700 1 --2- 172.21.15.64:0/803128474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4030102230 0x7f4030072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:21.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.926+0000 7f4037eb1700 1 --2- 172.21.15.64:0/803128474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4030101890 0x7f40300729f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:21.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.926+0000 7f4037eb1700 1 -- 172.21.15.64:0/803128474 >> 172.21.15.64:0/803128474 conn(0x7f40300fbbf0 msgr2=0x7f4030104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:10:21.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.926+0000 7f4037eb1700 1 -- 172.21.15.64:0/803128474 shutdown_connections 2023-09-13T03:10:21.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:21.926+0000 7f4037eb1700 1 -- 172.21.15.64:0/803128474 wait complete. 2023-09-13T03:10:21.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:21 smithi142 ceph-mon[134082]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-09-13T03:10:21.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:21 smithi142 ceph-mon[134082]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-09-13T03:10:21.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:21 smithi142 ceph-mon[134082]: from='osd.11 [v2:172.21.15.170:6824/3605017060,v1:172.21.15.170:6825/3605017060]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-13T03:10:21.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:21 smithi142 ceph-mon[134082]: osdmap e76: 12 total, 11 up, 12 in 2023-09-13T03:10:21.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:21 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-13T03:10:21.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:21 smithi142 ceph-mon[134082]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-13T03:10:22.442 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":77,"num_osds":12,"num_up_osds":11,"osd_up_since":1694574602,"num_in_osds":12,"osd_in_since":1694574605,"num_remapped_pgs":0} 2023-09-13T03:10:22.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:22 smithi170 ceph-mon[134328]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2023-09-13T03:10:22.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:22 smithi170 ceph-mon[134328]: osdmap e77: 12 total, 11 up, 12 in 2023-09-13T03:10:22.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:22 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-13T03:10:22.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:22 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-13T03:10:22.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:22 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/803128474' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-13T03:10:22.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:22 smithi064 ceph-mon[97675]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2023-09-13T03:10:22.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:22 smithi064 ceph-mon[97675]: osdmap e77: 12 total, 11 up, 12 in 2023-09-13T03:10:22.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:22 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-13T03:10:22.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:22 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-13T03:10:22.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:22 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/803128474' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-13T03:10:22.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:22 smithi142 ceph-mon[134082]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2023-09-13T03:10:22.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:22 smithi142 ceph-mon[134082]: osdmap e77: 12 total, 11 up, 12 in 2023-09-13T03:10:22.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:22 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-13T03:10:22.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:22 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-13T03:10:22.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:22 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/803128474' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-13T03:10:23.443 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph osd stat -f json 2023-09-13T03:10:23.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:23 smithi170 ceph-mon[134328]: purged_snaps scrub starts 2023-09-13T03:10:23.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:23 smithi170 ceph-mon[134328]: purged_snaps scrub ok 2023-09-13T03:10:23.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:23 smithi170 ceph-mon[134328]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-13T03:10:23.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:23 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-13T03:10:23.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:23 smithi170 ceph-mon[134328]: osd.11 [v2:172.21.15.170:6824/3605017060,v1:172.21.15.170:6825/3605017060] boot 2023-09-13T03:10:23.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:23 smithi170 ceph-mon[134328]: osdmap e78: 12 total, 12 up, 12 in 2023-09-13T03:10:23.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:23 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-13T03:10:23.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:23 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:23.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:23 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:23.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:23 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:10:23.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:23 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:10:23.913 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:23 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:10:23.913 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:23 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:10:23.913 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:23 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:23.913 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:23 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:23.914 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:23 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:10:23.914 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:23 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:23.914 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:23 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:10:23.914 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:23 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:23.915 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:23 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:10:23.915 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:23 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:23.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:23 smithi064 ceph-mon[97675]: purged_snaps scrub starts 2023-09-13T03:10:23.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:23 smithi064 ceph-mon[97675]: purged_snaps scrub ok 2023-09-13T03:10:23.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:23 smithi064 ceph-mon[97675]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-13T03:10:23.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:23 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-13T03:10:23.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:23 smithi064 ceph-mon[97675]: osd.11 [v2:172.21.15.170:6824/3605017060,v1:172.21.15.170:6825/3605017060] boot 2023-09-13T03:10:23.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:23 smithi064 ceph-mon[97675]: osdmap e78: 12 total, 12 up, 12 in 2023-09-13T03:10:23.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:23 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-13T03:10:23.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:23 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:23.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:23 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:23.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:23 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:10:23.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:23 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:10:23.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:23 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:10:23.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:23 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:10:23.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:23 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:23.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:23 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:23.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:23 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:10:23.922 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:23 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:23.922 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:23 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:10:23.922 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:23 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:23.922 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:23 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:10:23.922 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:23 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:23.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:23 smithi142 ceph-mon[134082]: purged_snaps scrub starts 2023-09-13T03:10:23.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:23 smithi142 ceph-mon[134082]: purged_snaps scrub ok 2023-09-13T03:10:23.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:23 smithi142 ceph-mon[134082]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-13T03:10:23.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:23 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-13T03:10:23.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:23 smithi142 ceph-mon[134082]: osd.11 [v2:172.21.15.170:6824/3605017060,v1:172.21.15.170:6825/3605017060] boot 2023-09-13T03:10:23.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:23 smithi142 ceph-mon[134082]: osdmap e78: 12 total, 12 up, 12 in 2023-09-13T03:10:23.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:23 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-13T03:10:23.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:23 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:23.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:23 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:23.998 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:23 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:10:23.998 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:23 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:10:23.998 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:23 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:10:23.999 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:23 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:10:23.999 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:23 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:23.999 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:23 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:24.000 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:23 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:10:24.000 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:23 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:24.000 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:23 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:10:24.000 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:23 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:24.001 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:23 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:10:24.001 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:23 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:24.180 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:10:24.663 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:24 smithi064 ceph-mon[97675]: Detected new or changed devices on smithi170 2023-09-13T03:10:24.664 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:24 smithi064 ceph-mon[97675]: Adjusting osd_memory_target on smithi170 to 4292M 2023-09-13T03:10:24.664 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:24 smithi064 ceph-mon[97675]: osdmap e79: 12 total, 12 up, 12 in 2023-09-13T03:10:24.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:24 smithi170 ceph-mon[134328]: Detected new or changed devices on smithi170 2023-09-13T03:10:24.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:24 smithi170 ceph-mon[134328]: Adjusting osd_memory_target on smithi170 to 4292M 2023-09-13T03:10:24.913 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:24 smithi170 ceph-mon[134328]: osdmap e79: 12 total, 12 up, 12 in 2023-09-13T03:10:24.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:24 smithi142 ceph-mon[134082]: Detected new or changed devices on smithi170 2023-09-13T03:10:24.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:24 smithi142 ceph-mon[134082]: Adjusting osd_memory_target on smithi170 to 4292M 2023-09-13T03:10:24.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:24 smithi142 ceph-mon[134082]: osdmap e79: 12 total, 12 up, 12 in 2023-09-13T03:10:25.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.431+0000 7f0b541d3700 1 -- 172.21.15.64:0/3742594640 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b4c102230 msgr2=0x7f0b4c108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:25.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.431+0000 7f0b541d3700 1 --2- 172.21.15.64:0/3742594640 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b4c102230 0x7f0b4c108ae0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0b4400b0d0 tx=0x7f0b4400b3e0 comp rx=0 tx=0).stop 2023-09-13T03:10:25.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.432+0000 7f0b541d3700 1 -- 172.21.15.64:0/3742594640 shutdown_connections 2023-09-13T03:10:25.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.432+0000 7f0b541d3700 1 --2- 172.21.15.64:0/3742594640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b4c100690 0x7f0b4c100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:25.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.432+0000 7f0b541d3700 1 --2- 172.21.15.64:0/3742594640 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b4c102230 0x7f0b4c108ae0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:25.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.432+0000 7f0b541d3700 1 --2- 172.21.15.64:0/3742594640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b4c101890 0x7f0b4c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:25.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.432+0000 7f0b541d3700 1 -- 172.21.15.64:0/3742594640 >> 172.21.15.64:0/3742594640 conn(0x7f0b4c0fbbf0 msgr2=0x7f0b4c0fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:10:25.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.432+0000 7f0b541d3700 1 -- 172.21.15.64:0/3742594640 shutdown_connections 2023-09-13T03:10:25.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.432+0000 7f0b541d3700 1 -- 172.21.15.64:0/3742594640 wait complete. 2023-09-13T03:10:25.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.433+0000 7f0b541d3700 1 Processor -- start 2023-09-13T03:10:25.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.433+0000 7f0b541d3700 1 -- start start 2023-09-13T03:10:25.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.433+0000 7f0b541d3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b4c100690 0x7f0b4c198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:25.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.433+0000 7f0b541d3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b4c101890 0x7f0b4c198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:25.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.433+0000 7f0b541d3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b4c102230 0x7f0b4c19fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:25.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.434+0000 7f0b541d3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4c1a01d0 con 0x7f0b4c100690 2023-09-13T03:10:25.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.434+0000 7f0b541d3700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4c1a0310 con 0x7f0b4c102230 2023-09-13T03:10:25.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.434+0000 7f0b541d3700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4c1a04a0 con 0x7f0b4c101890 2023-09-13T03:10:25.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.434+0000 7f0b5176e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b4c101890 0x7f0b4c198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:25.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.434+0000 7f0b51f6f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b4c100690 0x7f0b4c198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:25.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.434+0000 7f0b5176e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b4c101890 0x7f0b4c198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:44286/0 (socket says 172.21.15.64:44286) 2023-09-13T03:10:25.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.434+0000 7f0b5176e700 1 -- 172.21.15.64:0/1150596885 learned_addr learned my addr 172.21.15.64:0/1150596885 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:10:25.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.434+0000 7f0b52770700 1 --2- 172.21.15.64:0/1150596885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b4c102230 0x7f0b4c19fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:25.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.434+0000 7f0b5176e700 1 -- 172.21.15.64:0/1150596885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b4c102230 msgr2=0x7f0b4c19fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:25.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.434+0000 7f0b5176e700 1 --2- 172.21.15.64:0/1150596885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b4c102230 0x7f0b4c19fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:25.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.434+0000 7f0b5176e700 1 -- 172.21.15.64:0/1150596885 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b4c100690 msgr2=0x7f0b4c198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:25.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.434+0000 7f0b5176e700 1 --2- 172.21.15.64:0/1150596885 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b4c100690 0x7f0b4c198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:25.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.434+0000 7f0b5176e700 1 -- 172.21.15.64:0/1150596885 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b48009520 con 0x7f0b4c101890 2023-09-13T03:10:25.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.435+0000 7f0b5176e700 1 --2- 172.21.15.64:0/1150596885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b4c101890 0x7f0b4c198b80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c00e9f0 tx=0x7f0b3c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:10:25.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.435+0000 7f0b42ffd700 1 -- 172.21.15.64:0/1150596885 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c0097b0 con 0x7f0b4c101890 2023-09-13T03:10:25.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.435+0000 7f0b541d3700 1 -- 172.21.15.64:0/1150596885 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b44009d00 con 0x7f0b4c101890 2023-09-13T03:10:25.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.435+0000 7f0b42ffd700 1 -- 172.21.15.64:0/1150596885 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b3c00ee90 con 0x7f0b4c101890 2023-09-13T03:10:25.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.435+0000 7f0b42ffd700 1 -- 172.21.15.64:0/1150596885 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c005520 con 0x7f0b4c101890 2023-09-13T03:10:25.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.435+0000 7f0b541d3700 1 -- 172.21.15.64:0/1150596885 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b4c1a0af0 con 0x7f0b4c101890 2023-09-13T03:10:25.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.436+0000 7f0b541d3700 1 -- 172.21.15.64:0/1150596885 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b4c192cb0 con 0x7f0b4c101890 2023-09-13T03:10:25.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.438+0000 7f0b42ffd700 1 -- 172.21.15.64:0/1150596885 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f0b3c004750 con 0x7f0b4c101890 2023-09-13T03:10:25.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.439+0000 7f0b42ffd700 1 --2- 172.21.15.64:0/1150596885 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f0b380a06c0 0x7f0b380a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:25.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.440+0000 7f0b42ffd700 1 -- 172.21.15.64:0/1150596885 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0b3c014070 con 0x7f0b4c101890 2023-09-13T03:10:25.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.440+0000 7f0b51f6f700 1 --2- 172.21.15.64:0/1150596885 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f0b380a06c0 0x7f0b380a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:25.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.440+0000 7f0b51f6f700 1 --2- 172.21.15.64:0/1150596885 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f0b380a06c0 0x7f0b380a2b80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c1016f0 tx=0x7f0b48009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:10:25.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.442+0000 7f0b42ffd700 1 -- 172.21.15.64:0/1150596885 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0b3c095a20 con 0x7f0b4c101890 2023-09-13T03:10:25.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.651+0000 7f0b541d3700 1 -- 172.21.15.64:0/1150596885 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f0b4c10c090 con 0x7f0b4c101890 2023-09-13T03:10:25.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.652+0000 7f0b42ffd700 1 -- 172.21.15.64:0/1150596885 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) v1 ==== 74+0+133 (secure 0 0 0) 0x7f0b3c099040 con 0x7f0b4c101890 2023-09-13T03:10:25.653 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:10:25.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.654+0000 7f0b541d3700 1 -- 172.21.15.64:0/1150596885 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f0b380a06c0 msgr2=0x7f0b380a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:25.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.654+0000 7f0b541d3700 1 --2- 172.21.15.64:0/1150596885 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f0b380a06c0 0x7f0b380a2b80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c1016f0 tx=0x7f0b48009260 comp rx=0 tx=0).stop 2023-09-13T03:10:25.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.654+0000 7f0b541d3700 1 -- 172.21.15.64:0/1150596885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b4c101890 msgr2=0x7f0b4c198b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:25.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.654+0000 7f0b541d3700 1 --2- 172.21.15.64:0/1150596885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b4c101890 0x7f0b4c198b80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c00e9f0 tx=0x7f0b3c00ed00 comp rx=0 tx=0).stop 2023-09-13T03:10:25.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.654+0000 7f0b541d3700 1 -- 172.21.15.64:0/1150596885 shutdown_connections 2023-09-13T03:10:25.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.654+0000 7f0b541d3700 1 --2- 172.21.15.64:0/1150596885 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f0b380a06c0 0x7f0b380a2b80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:25.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.654+0000 7f0b541d3700 1 --2- 172.21.15.64:0/1150596885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b4c102230 0x7f0b4c19fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:25.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.654+0000 7f0b541d3700 1 --2- 172.21.15.64:0/1150596885 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b4c100690 0x7f0b4c198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:25.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.654+0000 7f0b541d3700 1 --2- 172.21.15.64:0/1150596885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b4c101890 0x7f0b4c198b80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:25.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.655+0000 7f0b541d3700 1 -- 172.21.15.64:0/1150596885 >> 172.21.15.64:0/1150596885 conn(0x7f0b4c0fbbf0 msgr2=0x7f0b4c107150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:10:25.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.655+0000 7f0b541d3700 1 -- 172.21.15.64:0/1150596885 shutdown_connections 2023-09-13T03:10:25.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:25.655+0000 7f0b541d3700 1 -- 172.21.15.64:0/1150596885 wait complete. 2023-09-13T03:10:25.843 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:25 smithi064 ceph-mon[97675]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:25.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:25 smithi170 ceph-mon[134328]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:25.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:25 smithi142 ceph-mon[134082]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:26.161 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":79,"num_osds":12,"num_up_osds":12,"osd_up_since":1694574622,"num_in_osds":12,"osd_in_since":1694574605,"num_remapped_pgs":0} 2023-09-13T03:10:26.162 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph osd dump --format=json 2023-09-13T03:10:26.895 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:10:26.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:26 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/1150596885' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-13T03:10:26.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:26 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/1150596885' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-13T03:10:26.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:26 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/1150596885' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-13T03:10:27.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:27 smithi064 ceph-mon[97675]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:27.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:27 smithi170 ceph-mon[134328]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:27.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:27 smithi142 ceph-mon[134082]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:28.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.280+0000 7fbb5ca80700 1 -- 172.21.15.64:0/823750079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb58100fe0 msgr2=0x7fbb5810d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:28.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.280+0000 7fbb5ca80700 1 --2- 172.21.15.64:0/823750079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb58100fe0 0x7fbb5810d4f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbb5000b0a0 tx=0x7fbb5000b3b0 comp rx=0 tx=0).stop 2023-09-13T03:10:28.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.280+0000 7fbb5ca80700 1 -- 172.21.15.64:0/823750079 shutdown_connections 2023-09-13T03:10:28.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.280+0000 7fbb5ca80700 1 --2- 172.21.15.64:0/823750079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb58106620 0x7fbb58106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:28.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.280+0000 7fbb5ca80700 1 --2- 172.21.15.64:0/823750079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb58100fe0 0x7fbb5810d4f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:28.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.280+0000 7fbb5ca80700 1 --2- 172.21.15.64:0/823750079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb58100620 0x7fbb58100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:28.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.280+0000 7fbb5ca80700 1 -- 172.21.15.64:0/823750079 >> 172.21.15.64:0/823750079 conn(0x7fbb580fc0f0 msgr2=0x7fbb580fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:10:28.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.280+0000 7fbb5ca80700 1 -- 172.21.15.64:0/823750079 shutdown_connections 2023-09-13T03:10:28.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.280+0000 7fbb5ca80700 1 -- 172.21.15.64:0/823750079 wait complete. 2023-09-13T03:10:28.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.281+0000 7fbb5ca80700 1 Processor -- start 2023-09-13T03:10:28.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.282+0000 7fbb5ca80700 1 -- start start 2023-09-13T03:10:28.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.282+0000 7fbb5ca80700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb58100620 0x7fbb5819c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:28.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.282+0000 7fbb5ca80700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb58100fe0 0x7fbb5819cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:28.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.282+0000 7fbb5ca80700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb58106620 0x7fbb581966a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:28.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.282+0000 7fbb5ca80700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb58196d10 con 0x7fbb58106620 2023-09-13T03:10:28.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.282+0000 7fbb5ca80700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb58196e80 con 0x7fbb58100620 2023-09-13T03:10:28.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.282+0000 7fbb5ca80700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb58197040 con 0x7fbb58100fe0 2023-09-13T03:10:28.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.282+0000 7fbb56d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb58106620 0x7fbb581966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:28.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.282+0000 7fbb5659c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb58100620 0x7fbb5819c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:28.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.282+0000 7fbb56d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb58106620 0x7fbb581966a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60308/0 (socket says 172.21.15.64:60308) 2023-09-13T03:10:28.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.282+0000 7fbb5659c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb58100620 0x7fbb5819c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:44220/0 (socket says 172.21.15.64:44220) 2023-09-13T03:10:28.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.282+0000 7fbb56d9d700 1 -- 172.21.15.64:0/2780832106 learned_addr learned my addr 172.21.15.64:0/2780832106 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:10:28.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.282+0000 7fbb55d9b700 1 --2- 172.21.15.64:0/2780832106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb58100fe0 0x7fbb5819cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:28.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.283+0000 7fbb5659c700 1 -- 172.21.15.64:0/2780832106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb58100fe0 msgr2=0x7fbb5819cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:28.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.283+0000 7fbb5659c700 1 --2- 172.21.15.64:0/2780832106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb58100fe0 0x7fbb5819cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:28.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.283+0000 7fbb5659c700 1 -- 172.21.15.64:0/2780832106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb58106620 msgr2=0x7fbb581966a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:28.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.283+0000 7fbb5659c700 1 --2- 172.21.15.64:0/2780832106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb58106620 0x7fbb581966a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:28.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.283+0000 7fbb5659c700 1 -- 172.21.15.64:0/2780832106 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb50009d00 con 0x7fbb58100620 2023-09-13T03:10:28.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.283+0000 7fbb56d9d700 1 --2- 172.21.15.64:0/2780832106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb58106620 0x7fbb581966a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:10:28.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.283+0000 7fbb5659c700 1 --2- 172.21.15.64:0/2780832106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb58100620 0x7fbb5819c620 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800e9f0 tx=0x7fbb4800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:10:28.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.283+0000 7fbb477fe700 1 -- 172.21.15.64:0/2780832106 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4800c9c0 con 0x7fbb58100620 2023-09-13T03:10:28.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.283+0000 7fbb5ca80700 1 -- 172.21.15.64:0/2780832106 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb58197320 con 0x7fbb58100620 2023-09-13T03:10:28.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.283+0000 7fbb477fe700 1 -- 172.21.15.64:0/2780832106 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb48010430 con 0x7fbb58100620 2023-09-13T03:10:28.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.283+0000 7fbb477fe700 1 -- 172.21.15.64:0/2780832106 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb48018680 con 0x7fbb58100620 2023-09-13T03:10:28.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.284+0000 7fbb5ca80700 1 -- 172.21.15.64:0/2780832106 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb581a3610 con 0x7fbb58100620 2023-09-13T03:10:28.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.285+0000 7fbb5ca80700 1 -- 172.21.15.64:0/2780832106 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb5804e300 con 0x7fbb58100620 2023-09-13T03:10:28.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.287+0000 7fbb477fe700 1 -- 172.21.15.64:0/2780832106 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fbb48010860 con 0x7fbb58100620 2023-09-13T03:10:28.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.288+0000 7fbb477fe700 1 --2- 172.21.15.64:0/2780832106 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fbb400a0640 0x7fbb400a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:28.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.288+0000 7fbb477fe700 1 -- 172.21.15.64:0/2780832106 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbb48014070 con 0x7fbb58100620 2023-09-13T03:10:28.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.288+0000 7fbb55d9b700 1 --2- 172.21.15.64:0/2780832106 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fbb400a0640 0x7fbb400a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:28.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.289+0000 7fbb55d9b700 1 --2- 172.21.15.64:0/2780832106 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fbb400a0640 0x7fbb400a2b00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbb58105d50 tx=0x7fbb4c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:10:28.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.290+0000 7fbb477fe700 1 -- 172.21.15.64:0/2780832106 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fbb480959c0 con 0x7fbb58100620 2023-09-13T03:10:28.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.499+0000 7fbb5ca80700 1 -- 172.21.15.64:0/2780832106 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb58197510 con 0x7fbb58100620 2023-09-13T03:10:28.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.500+0000 7fbb477fe700 1 -- 172.21.15.64:0/2780832106 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18445 (secure 0 0 0) 0x7fbb58197510 con 0x7fbb58100620 2023-09-13T03:10:28.502 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:10:28.502 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":79,"fsid":"f8dbced4-51e1-11ee-9ab7-7b867c8bd7da","created":"2023-09-13T03:03:24.167202+0000","modified":"2023-09-13T03:10:23.938509+0000","last_up_change":"2023-09-13T03:10:22.503529+0000","last_in_change":"2023-09-13T03:10:05.678720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T03:06:58.987479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3997c114-241b-48c4-8feb-653c99209470","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6803","nonce":3940892135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6805","nonce":3940892135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6809","nonce":3940892135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6807","nonce":3940892135}]},"public_addr":"172.21.15.64:6803/3940892135","cluster_addr":"172.21.15.64:6805/3940892135","heartbeat_back_addr":"172.21.15.64:6809/3940892135","heartbeat_front_addr":"172.21.15.64:6807/3940892135","state":["exists","up"]},{"osd":1,"uuid":"94cfae85-322e-4ed2-a6e6-e40349bb14bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6801","nonce":4188149333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6803","nonce":4188149333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6807","nonce":4188149333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6805","nonce":4188149333}]},"public_addr":"172.21.15.142:6801/4188149333","cluster_addr":"172.21.15.142:6803/4188149333","heartbeat_back_addr":"172.21.15.142:6807/4188149333","heartbeat_front_addr":"172.21.15.142:6805/4188149333","state":["exists","up"]},{"osd":2,"uuid":"b3aae17e-96db-4008-8c27-f733c46b30ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6801","nonce":2406669338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6803","nonce":2406669338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6807","nonce":2406669338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6805","nonce":2406669338}]},"public_addr":"172.21.15.170:6801/2406669338","cluster_addr":"172.21.15.170:6803/2406669338","heartbeat_back_addr":"172.21.15.170:6807/2406669338","heartbeat_front_addr":"172.21.15.170:6805/2406669338","state":["exists","up"]},{"osd":3,"uuid":"7fa8d8cb-2ab7-48a7-8b49-5e2903d6779f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6811","nonce":3558318707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6813","nonce":3558318707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6817","nonce":3558318707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6815","nonce":3558318707}]},"public_addr":"172.21.15.64:6811/3558318707","cluster_addr":"172.21.15.64:6813/3558318707","heartbeat_back_addr":"172.21.15.64:6817/3558318707","heartbeat_front_addr":"172.21.15.64:6815/3558318707","state":["exists","up"]},{"osd":4,"uuid":"8c7a9c9f-f919-48d3-9db2-214a25bb5603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6809","nonce":1811142576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6811","nonce":1811142576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6815","nonce":1811142576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6813","nonce":1811142576}]},"public_addr":"172.21.15.142:6809/1811142576","cluster_addr":"172.21.15.142:6811/1811142576","heartbeat_back_addr":"172.21.15.142:6815/1811142576","heartbeat_front_addr":"172.21.15.142:6813/1811142576","state":["exists","up"]},{"osd":5,"uuid":"0c6101c6-2065-4022-a751-d3f216d7f478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6809","nonce":2903841453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6811","nonce":2903841453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6815","nonce":2903841453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6813","nonce":2903841453}]},"public_addr":"172.21.15.170:6809/2903841453","cluster_addr":"172.21.15.170:6811/2903841453","heartbeat_back_addr":"172.21.15.170:6815/2903841453","heartbeat_front_addr":"172.21.15.170:6813/2903841453","state":["exists","up"]},{"osd":6,"uuid":"5cf3e52e-d155-41b1-9194-db047d26a953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6819","nonce":1436584490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6821","nonce":1436584490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6825","nonce":1436584490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6823","nonce":1436584490}]},"public_addr":"172.21.15.64:6819/1436584490","cluster_addr":"172.21.15.64:6821/1436584490","heartbeat_back_addr":"172.21.15.64:6825/1436584490","heartbeat_front_addr":"172.21.15.64:6823/1436584490","state":["exists","up"]},{"osd":7,"uuid":"e97323e3-badd-41a7-af9e-e78a81a64046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6817","nonce":3726793534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6819","nonce":3726793534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6823","nonce":3726793534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6821","nonce":3726793534}]},"public_addr":"172.21.15.142:6817/3726793534","cluster_addr":"172.21.15.142:6819/3726793534","heartbeat_back_addr":"172.21.15.142:6823/3726793534","heartbeat_front_addr":"172.21.15.142:6821/3726793534","state":["exists","up"]},{"osd":8,"uuid":"83d2835c-ed26-493d-9222-c2894ec0e071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6817","nonce":4272513873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6819","nonce":4272513873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6823","nonce":4272513873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6821","nonce":4272513873}]},"public_addr":"172.21.15.170:6817/4272513873","cluster_addr":"172.21.15.170:6819/4272513873","heartbeat_back_addr":"172.21.15.170:6823/4272513873","heartbeat_front_addr":"172.21.15.170:6821/4272513873","state":["exists","up"]},{"osd":9,"uuid":"cb936bdd-1bec-4236-aa2e-a24adcac11e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6827","nonce":2641175211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6829","nonce":2641175211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6833","nonce":2641175211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6831","nonce":2641175211}]},"public_addr":"172.21.15.64:6827/2641175211","cluster_addr":"172.21.15.64:6829/2641175211","heartbeat_back_addr":"172.21.15.64:6833/2641175211","heartbeat_front_addr":"172.21.15.64:6831/2641175211","state":["exists","up"]},{"osd":10,"uuid":"85512a1c-6171-43d4-9c15-c588a8b71be0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6825","nonce":936499217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6827","nonce":936499217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6831","nonce":936499217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6829","nonce":936499217}]},"public_addr":"172.21.15.142:6825/936499217","cluster_addr":"172.21.15.142:6827/936499217","heartbeat_back_addr":"172.21.15.142:6831/936499217","heartbeat_front_addr":"172.21.15.142:6829/936499217","state":["exists","up"]},{"osd":11,"uuid":"9b8417dc-5d6c-4917-94a9-b032c5bc33d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6825","nonce":3605017060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6827","nonce":3605017060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6831","nonce":3605017060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6829","nonce":3605017060}]},"public_addr":"172.21.15.170:6825/3605017060","cluster_addr":"172.21.15.170:6827/3605017060","heartbeat_back_addr":"172.21.15.170:6831/3605017060","heartbeat_front_addr":"172.21.15.170:6829/3605017060","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:14.229044+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:36.147777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:56.655298+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:07:23.107741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:07:44.663770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:05.577432+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:31.168031+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:52.791055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:09:13.658546+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:09:39.150360+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:10:00.601735+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:10:21.407555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6800/2955227347":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/1373223118":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/271194613":"2023-09-14T03:04:22.355817+0000","172.21.15.64:6801/3998091347":"2023-09-14T03:03:56.442541+0000","172.21.15.64:6800/3998091347":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/920818572":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/735789005":"2023-09-14T03:03:56.442541+0000","172.21.15.64:6801/2955227347":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/1975923178":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/1500151178":"2023-09-14T03:04:22.355817+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T03:10:28.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.503+0000 7fbb5ca80700 1 -- 172.21.15.64:0/2780832106 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fbb400a0640 msgr2=0x7fbb400a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:28.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.503+0000 7fbb5ca80700 1 --2- 172.21.15.64:0/2780832106 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fbb400a0640 0x7fbb400a2b00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbb58105d50 tx=0x7fbb4c009260 comp rx=0 tx=0).stop 2023-09-13T03:10:28.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.503+0000 7fbb5ca80700 1 -- 172.21.15.64:0/2780832106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb58100620 msgr2=0x7fbb5819c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:28.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.503+0000 7fbb5ca80700 1 --2- 172.21.15.64:0/2780832106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb58100620 0x7fbb5819c620 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800e9f0 tx=0x7fbb4800ed00 comp rx=0 tx=0).stop 2023-09-13T03:10:28.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.503+0000 7fbb5ca80700 1 -- 172.21.15.64:0/2780832106 shutdown_connections 2023-09-13T03:10:28.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.503+0000 7fbb5ca80700 1 --2- 172.21.15.64:0/2780832106 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fbb400a0640 0x7fbb400a2b00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:28.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.503+0000 7fbb5ca80700 1 --2- 172.21.15.64:0/2780832106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb58100620 0x7fbb5819c620 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:28.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.503+0000 7fbb5ca80700 1 --2- 172.21.15.64:0/2780832106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb58106620 0x7fbb581966a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:28.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.503+0000 7fbb5ca80700 1 --2- 172.21.15.64:0/2780832106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb58100fe0 0x7fbb5819cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:28.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.503+0000 7fbb5ca80700 1 -- 172.21.15.64:0/2780832106 >> 172.21.15.64:0/2780832106 conn(0x7fbb580fc0f0 msgr2=0x7fbb580fc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:10:28.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.503+0000 7fbb5ca80700 1 -- 172.21.15.64:0/2780832106 shutdown_connections 2023-09-13T03:10:28.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:28.503+0000 7fbb5ca80700 1 -- 172.21.15.64:0/2780832106 wait complete. 2023-09-13T03:10:28.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:28 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/2780832106' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:10:28.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:28 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2780832106' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:10:28.972 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-13T03:06:58.987479+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-13T03:10:28.973 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph osd pool get .mgr pg_num 2023-09-13T03:10:28.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:28 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/2780832106' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:10:29.710 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:10:29.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:29 smithi170 ceph-mon[134328]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:29.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:29 smithi064 ceph-mon[97675]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:29.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:29 smithi142 ceph-mon[134082]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:31.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.075+0000 7f6e94b16700 1 -- 172.21.15.64:0/4232427194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e90106620 msgr2=0x7f6e90106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:31.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.075+0000 7f6e94b16700 1 --2- 172.21.15.64:0/4232427194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e90106620 0x7f6e90106a00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6e80009990 tx=0x7f6e80009ca0 comp rx=0 tx=0).stop 2023-09-13T03:10:31.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.075+0000 7f6e94b16700 1 -- 172.21.15.64:0/4232427194 shutdown_connections 2023-09-13T03:10:31.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.075+0000 7f6e94b16700 1 --2- 172.21.15.64:0/4232427194 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e90100fe0 0x7f6e9010d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:31.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.075+0000 7f6e94b16700 1 --2- 172.21.15.64:0/4232427194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e90106620 0x7f6e90106a00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:31.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.075+0000 7f6e94b16700 1 --2- 172.21.15.64:0/4232427194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e90100620 0x7f6e90100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:31.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.075+0000 7f6e94b16700 1 -- 172.21.15.64:0/4232427194 >> 172.21.15.64:0/4232427194 conn(0x7f6e900fc0f0 msgr2=0x7f6e900fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:10:31.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.075+0000 7f6e94b16700 1 -- 172.21.15.64:0/4232427194 shutdown_connections 2023-09-13T03:10:31.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.075+0000 7f6e94b16700 1 -- 172.21.15.64:0/4232427194 wait complete. 2023-09-13T03:10:31.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.077+0000 7f6e94b16700 1 Processor -- start 2023-09-13T03:10:31.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.077+0000 7f6e94b16700 1 -- start start 2023-09-13T03:10:31.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.077+0000 7f6e94b16700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e90100620 0x7f6e90196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:31.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.077+0000 7f6e94b16700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e90100fe0 0x7f6e90196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:31.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.077+0000 7f6e94b16700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e90106620 0x7f6e9019a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:31.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.077+0000 7f6e94b16700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9019a600 con 0x7f6e90106620 2023-09-13T03:10:31.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.077+0000 7f6e94b16700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9019a770 con 0x7f6e90100620 2023-09-13T03:10:31.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.078+0000 7f6e94b16700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9019a930 con 0x7f6e90100fe0 2023-09-13T03:10:31.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.078+0000 7f6e8e59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e90100620 0x7f6e90196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:31.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.078+0000 7f6e8ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e90106620 0x7f6e9019a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:31.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.078+0000 7f6e8e59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e90100620 0x7f6e90196750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:44246/0 (socket says 172.21.15.64:44246) 2023-09-13T03:10:31.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.078+0000 7f6e8e59c700 1 -- 172.21.15.64:0/644573985 learned_addr learned my addr 172.21.15.64:0/644573985 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:10:31.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.078+0000 7f6e8ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e90106620 0x7f6e9019a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60326/0 (socket says 172.21.15.64:60326) 2023-09-13T03:10:31.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.078+0000 7f6e8dd9b700 1 --2- 172.21.15.64:0/644573985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e90100fe0 0x7f6e90196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:31.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.078+0000 7f6e8ed9d700 1 -- 172.21.15.64:0/644573985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e90100fe0 msgr2=0x7f6e90196c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:31.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.078+0000 7f6e8ed9d700 1 --2- 172.21.15.64:0/644573985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e90100fe0 0x7f6e90196c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:31.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.078+0000 7f6e8ed9d700 1 -- 172.21.15.64:0/644573985 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e90100620 msgr2=0x7f6e90196750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:31.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.078+0000 7f6e8ed9d700 1 --2- 172.21.15.64:0/644573985 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e90100620 0x7f6e90196750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:31.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.078+0000 7f6e8ed9d700 1 -- 172.21.15.64:0/644573985 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e800095f0 con 0x7f6e90106620 2023-09-13T03:10:31.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.079+0000 7f6e8ed9d700 1 --2- 172.21.15.64:0/644573985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e90106620 0x7f6e9019a000 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6e8800efc0 tx=0x7f6e8800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:10:31.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.079+0000 7f6e7f7fe700 1 -- 172.21.15.64:0/644573985 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e88018070 con 0x7f6e90106620 2023-09-13T03:10:31.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.079+0000 7f6e7f7fe700 1 -- 172.21.15.64:0/644573985 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e8800f040 con 0x7f6e90106620 2023-09-13T03:10:31.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.079+0000 7f6e94b16700 1 -- 172.21.15.64:0/644573985 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e9019abb0 con 0x7f6e90106620 2023-09-13T03:10:31.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.079+0000 7f6e7f7fe700 1 -- 172.21.15.64:0/644573985 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8800cd40 con 0x7f6e90106620 2023-09-13T03:10:31.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.079+0000 7f6e94b16700 1 -- 172.21.15.64:0/644573985 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e901a3560 con 0x7f6e90106620 2023-09-13T03:10:31.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.080+0000 7f6e94b16700 1 -- 172.21.15.64:0/644573985 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e9004e300 con 0x7f6e90106620 2023-09-13T03:10:31.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.082+0000 7f6e7f7fe700 1 -- 172.21.15.64:0/644573985 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f6e88003a00 con 0x7f6e90106620 2023-09-13T03:10:31.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.083+0000 7f6e7f7fe700 1 --2- 172.21.15.64:0/644573985 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6e780a0640 0x7f6e780a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:31.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.084+0000 7f6e7f7fe700 1 -- 172.21.15.64:0/644573985 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6e880cbc40 con 0x7f6e90106620 2023-09-13T03:10:31.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.084+0000 7f6e8e59c700 1 --2- 172.21.15.64:0/644573985 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6e780a0640 0x7f6e780a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:31.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.084+0000 7f6e8e59c700 1 --2- 172.21.15.64:0/644573985 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6e780a0640 0x7f6e780a2b00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6e80006010 tx=0x7f6e8000b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:10:31.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.086+0000 7f6e7f7fe700 1 -- 172.21.15.64:0/644573985 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6e88095c00 con 0x7f6e90106620 2023-09-13T03:10:31.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.298+0000 7f6e94b16700 1 -- 172.21.15.64:0/644573985 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6e901106a0 con 0x7f6e90106620 2023-09-13T03:10:31.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.298+0000 7f6e7f7fe700 1 -- 172.21.15.64:0/644573985 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6e88099220 con 0x7f6e90106620 2023-09-13T03:10:31.299 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-09-13T03:10:31.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.301+0000 7f6e94b16700 1 -- 172.21.15.64:0/644573985 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6e780a0640 msgr2=0x7f6e780a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:31.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.301+0000 7f6e94b16700 1 --2- 172.21.15.64:0/644573985 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6e780a0640 0x7f6e780a2b00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6e80006010 tx=0x7f6e8000b260 comp rx=0 tx=0).stop 2023-09-13T03:10:31.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.301+0000 7f6e94b16700 1 -- 172.21.15.64:0/644573985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e90106620 msgr2=0x7f6e9019a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:31.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.301+0000 7f6e94b16700 1 --2- 172.21.15.64:0/644573985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e90106620 0x7f6e9019a000 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6e8800efc0 tx=0x7f6e8800c3a0 comp rx=0 tx=0).stop 2023-09-13T03:10:31.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.301+0000 7f6e94b16700 1 -- 172.21.15.64:0/644573985 shutdown_connections 2023-09-13T03:10:31.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.301+0000 7f6e94b16700 1 --2- 172.21.15.64:0/644573985 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6e780a0640 0x7f6e780a2b00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:31.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.301+0000 7f6e94b16700 1 --2- 172.21.15.64:0/644573985 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e90100620 0x7f6e90196750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:31.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.301+0000 7f6e94b16700 1 --2- 172.21.15.64:0/644573985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e90106620 0x7f6e9019a000 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:31.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.301+0000 7f6e94b16700 1 --2- 172.21.15.64:0/644573985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e90100fe0 0x7f6e90196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:31.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.301+0000 7f6e94b16700 1 -- 172.21.15.64:0/644573985 >> 172.21.15.64:0/644573985 conn(0x7f6e900fc0f0 msgr2=0x7f6e900fc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:10:31.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.301+0000 7f6e94b16700 1 -- 172.21.15.64:0/644573985 shutdown_connections 2023-09-13T03:10:31.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:31.301+0000 7f6e94b16700 1 -- 172.21.15.64:0/644573985 wait complete. 2023-09-13T03:10:31.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:31 smithi064 ceph-mon[97675]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:31.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:31 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/644573985' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T03:10:31.785 INFO:tasks.cephadm:Adding mds.a on smithi064 2023-09-13T03:10:31.786 INFO:tasks.cephadm:Adding mds.d on smithi064 2023-09-13T03:10:31.786 INFO:tasks.cephadm:Adding mds.g on smithi064 2023-09-13T03:10:31.786 INFO:tasks.cephadm:Adding mds.j on smithi064 2023-09-13T03:10:31.787 INFO:tasks.cephadm:Adding mds.b on smithi142 2023-09-13T03:10:31.787 INFO:tasks.cephadm:Adding mds.e on smithi142 2023-09-13T03:10:31.787 INFO:tasks.cephadm:Adding mds.h on smithi142 2023-09-13T03:10:31.788 INFO:tasks.cephadm:Adding mds.k on smithi142 2023-09-13T03:10:31.788 INFO:tasks.cephadm:Adding mds.c on smithi170 2023-09-13T03:10:31.788 INFO:tasks.cephadm:Adding mds.f on smithi170 2023-09-13T03:10:31.788 INFO:tasks.cephadm:Adding mds.i on smithi170 2023-09-13T03:10:31.789 INFO:tasks.cephadm:Adding mds.l on smithi170 2023-09-13T03:10:31.789 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph orch apply mds all '12;smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l' 2023-09-13T03:10:31.822 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:31 smithi170 ceph-mon[134328]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:31.822 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:31 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/644573985' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T03:10:31.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:31 smithi142 ceph-mon[134082]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:31.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:31 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/644573985' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T03:10:32.358 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-13T03:10:33.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.509+0000 7fd37549b700 1 -- 172.21.15.170:0/2192146984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3700ff590 msgr2=0x7fd3700ff9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:33.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.509+0000 7fd37549b700 1 --2- 172.21.15.170:0/2192146984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3700ff590 0x7fd3700ff9b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd364009990 tx=0x7fd364009ca0 comp rx=0 tx=0).stop 2023-09-13T03:10:33.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.509+0000 7fd37549b700 1 -- 172.21.15.170:0/2192146984 shutdown_connections 2023-09-13T03:10:33.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.509+0000 7fd37549b700 1 --2- 172.21.15.170:0/2192146984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd370101130 0x7fd3701079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:33.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.509+0000 7fd37549b700 1 --2- 172.21.15.170:0/2192146984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3700ff590 0x7fd3700ff9b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:33.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.509+0000 7fd37549b700 1 --2- 172.21.15.170:0/2192146984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd370100790 0x7fd370100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:33.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.509+0000 7fd37549b700 1 -- 172.21.15.170:0/2192146984 >> 172.21.15.170:0/2192146984 conn(0x7fd3700fab10 msgr2=0x7fd3700fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:10:33.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.509+0000 7fd37549b700 1 -- 172.21.15.170:0/2192146984 shutdown_connections 2023-09-13T03:10:33.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.509+0000 7fd37549b700 1 -- 172.21.15.170:0/2192146984 wait complete. 2023-09-13T03:10:33.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.510+0000 7fd37549b700 1 Processor -- start 2023-09-13T03:10:33.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.510+0000 7fd37549b700 1 -- start start 2023-09-13T03:10:33.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.510+0000 7fd37549b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3700ff590 0x7fd370069510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:33.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.510+0000 7fd37549b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd370100790 0x7fd37019b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:33.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.510+0000 7fd37549b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd370101130 0x7fd37019d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:33.516 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.510+0000 7fd37549b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37019e040 con 0x7fd3700ff590 2023-09-13T03:10:33.516 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.510+0000 7fd37549b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37019e1b0 con 0x7fd370100790 2023-09-13T03:10:33.516 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.510+0000 7fd37549b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37019e370 con 0x7fd370101130 2023-09-13T03:10:33.516 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.510+0000 7fd36f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd370101130 0x7fd37019d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:33.516 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.510+0000 7fd36f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd370101130 0x7fd37019d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45900/0 (socket says 172.21.15.170:45900) 2023-09-13T03:10:33.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.510+0000 7fd36f7fe700 1 -- 172.21.15.170:0/1787848447 learned_addr learned my addr 172.21.15.170:0/1787848447 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-09-13T03:10:33.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.510+0000 7fd36e7fc700 1 --2- 172.21.15.170:0/1787848447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd370100790 0x7fd37019b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:33.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.510+0000 7fd36effd700 1 --2- 172.21.15.170:0/1787848447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3700ff590 0x7fd370069510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:33.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.511+0000 7fd36f7fe700 1 -- 172.21.15.170:0/1787848447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd370100790 msgr2=0x7fd37019b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:33.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.511+0000 7fd36f7fe700 1 --2- 172.21.15.170:0/1787848447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd370100790 0x7fd37019b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:33.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.511+0000 7fd36f7fe700 1 -- 172.21.15.170:0/1787848447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3700ff590 msgr2=0x7fd370069510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:33.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.511+0000 7fd36f7fe700 1 --2- 172.21.15.170:0/1787848447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3700ff590 0x7fd370069510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:33.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.511+0000 7fd36f7fe700 1 -- 172.21.15.170:0/1787848447 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3640095f0 con 0x7fd370101130 2023-09-13T03:10:33.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.511+0000 7fd36e7fc700 1 --2- 172.21.15.170:0/1787848447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd370100790 0x7fd37019b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:10:33.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.511+0000 7fd36f7fe700 1 --2- 172.21.15.170:0/1787848447 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd370101130 0x7fd37019d8a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd36800b970 tx=0x7fd36800bd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:10:33.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.511+0000 7fd357fff700 1 -- 172.21.15.170:0/1787848447 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36800c9b0 con 0x7fd370101130 2023-09-13T03:10:33.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.511+0000 7fd357fff700 1 -- 172.21.15.170:0/1787848447 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd368014440 con 0x7fd370101130 2023-09-13T03:10:33.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.511+0000 7fd357fff700 1 -- 172.21.15.170:0/1787848447 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3680075d0 con 0x7fd370101130 2023-09-13T03:10:33.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.511+0000 7fd37549b700 1 -- 172.21.15.170:0/1787848447 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd37019e650 con 0x7fd370101130 2023-09-13T03:10:33.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.511+0000 7fd37549b700 1 -- 172.21.15.170:0/1787848447 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd37019eb80 con 0x7fd370101130 2023-09-13T03:10:33.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.512+0000 7fd37549b700 1 -- 172.21.15.170:0/1787848447 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd370106c40 con 0x7fd370101130 2023-09-13T03:10:33.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.513+0000 7fd357fff700 1 -- 172.21.15.170:0/1787848447 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fd3680145b0 con 0x7fd370101130 2023-09-13T03:10:33.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.514+0000 7fd357fff700 1 --2- 172.21.15.170:0/1787848447 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd3580a06c0 0x7fd3580a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:33.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.514+0000 7fd357fff700 1 -- 172.21.15.170:0/1787848447 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd368006e40 con 0x7fd370101130 2023-09-13T03:10:33.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.514+0000 7fd36effd700 1 --2- 172.21.15.170:0/1787848447 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd3580a06c0 0x7fd3580a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:33.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.515+0000 7fd36effd700 1 --2- 172.21.15.170:0/1787848447 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd3580a06c0 0x7fd3580a2b80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd36400b5c0 tx=0x7fd364005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:10:33.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.518+0000 7fd357fff700 1 -- 172.21.15.170:0/1787848447 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fd36804ee10 con 0x7fd370101130 2023-09-13T03:10:33.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:33 smithi170 ceph-mon[134328]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:33.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.716+0000 7fd37549b700 1 -- 172.21.15.170:0/1787848447 --> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l", "target": ["mon-mgr", ""]}) v1 -- 0x7fd37002d0a0 con 0x7fd3580a06c0 2023-09-13T03:10:33.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.725+0000 7fd357fff700 1 -- 172.21.15.170:0/1787848447 <== mgr.14150 v2:172.21.15.64:6800/873679308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fd37002d0a0 con 0x7fd3580a06c0 2023-09-13T03:10:33.726 INFO:teuthology.orchestra.run.smithi170.stdout:Scheduled mds.all update... 2023-09-13T03:10:33.728 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.727+0000 7fd37549b700 1 -- 172.21.15.170:0/1787848447 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd3580a06c0 msgr2=0x7fd3580a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:33.728 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.727+0000 7fd37549b700 1 --2- 172.21.15.170:0/1787848447 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd3580a06c0 0x7fd3580a2b80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd36400b5c0 tx=0x7fd364005fb0 comp rx=0 tx=0).stop 2023-09-13T03:10:33.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.727+0000 7fd37549b700 1 -- 172.21.15.170:0/1787848447 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd370101130 msgr2=0x7fd37019d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:33.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.727+0000 7fd37549b700 1 --2- 172.21.15.170:0/1787848447 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd370101130 0x7fd37019d8a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd36800b970 tx=0x7fd36800bd30 comp rx=0 tx=0).stop 2023-09-13T03:10:33.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.727+0000 7fd37549b700 1 -- 172.21.15.170:0/1787848447 shutdown_connections 2023-09-13T03:10:33.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.727+0000 7fd37549b700 1 --2- 172.21.15.170:0/1787848447 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd3580a06c0 0x7fd3580a2b80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:33.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.727+0000 7fd37549b700 1 --2- 172.21.15.170:0/1787848447 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd370101130 0x7fd37019d8a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:33.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.727+0000 7fd37549b700 1 --2- 172.21.15.170:0/1787848447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3700ff590 0x7fd370069510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:33.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.727+0000 7fd37549b700 1 --2- 172.21.15.170:0/1787848447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd370100790 0x7fd37019b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:33.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.727+0000 7fd37549b700 1 -- 172.21.15.170:0/1787848447 >> 172.21.15.170:0/1787848447 conn(0x7fd3700fab10 msgr2=0x7fd370103d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:10:33.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.727+0000 7fd37549b700 1 -- 172.21.15.170:0/1787848447 shutdown_connections 2023-09-13T03:10:33.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-13T03:10:33.727+0000 7fd37549b700 1 -- 172.21.15.170:0/1787848447 wait complete. 2023-09-13T03:10:33.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:33 smithi064 ceph-mon[97675]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:33.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:33 smithi142 ceph-mon[134082]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:34.113 DEBUG:teuthology.orchestra.run.smithi064:mds.a> sudo journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mds.a.service 2023-09-13T03:10:34.118 DEBUG:teuthology.orchestra.run.smithi064:mds.d> sudo journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mds.d.service 2023-09-13T03:10:34.163 DEBUG:teuthology.orchestra.run.smithi064:mds.g> sudo journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mds.g.service 2023-09-13T03:10:34.203 INFO:journalctl@ceph.mds.a.smithi064.stdout:-- Logs begin at Wed 2023-09-13 02:46:56 UTC. -- 2023-09-13T03:10:34.206 DEBUG:teuthology.orchestra.run.smithi064:mds.j> sudo journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mds.j.service 2023-09-13T03:10:34.240 INFO:journalctl@ceph.mds.d.smithi064.stdout:-- Logs begin at Wed 2023-09-13 02:46:56 UTC. -- 2023-09-13T03:10:34.244 DEBUG:teuthology.orchestra.run.smithi142:mds.b> sudo journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mds.b.service 2023-09-13T03:10:34.246 DEBUG:teuthology.orchestra.run.smithi142:mds.e> sudo journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mds.e.service 2023-09-13T03:10:34.279 INFO:journalctl@ceph.mds.g.smithi064.stdout:-- Logs begin at Wed 2023-09-13 02:46:56 UTC. -- 2023-09-13T03:10:34.282 INFO:journalctl@ceph.mds.b.smithi142.stdout:-- Logs begin at Wed 2023-09-13 02:43:02 UTC. -- 2023-09-13T03:10:34.283 DEBUG:teuthology.orchestra.run.smithi142:mds.h> sudo journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mds.h.service 2023-09-13T03:10:34.312 INFO:journalctl@ceph.mds.j.smithi064.stdout:-- Logs begin at Wed 2023-09-13 02:46:56 UTC. -- 2023-09-13T03:10:34.313 INFO:journalctl@ceph.mds.e.smithi142.stdout:-- Logs begin at Wed 2023-09-13 02:43:02 UTC. -- 2023-09-13T03:10:34.313 DEBUG:teuthology.orchestra.run.smithi142:mds.k> sudo journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mds.k.service 2023-09-13T03:10:34.341 INFO:journalctl@ceph.mds.h.smithi142.stdout:-- Logs begin at Wed 2023-09-13 02:43:02 UTC. -- 2023-09-13T03:10:34.342 DEBUG:teuthology.orchestra.run.smithi170:mds.c> sudo journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mds.c.service 2023-09-13T03:10:34.344 DEBUG:teuthology.orchestra.run.smithi170:mds.f> sudo journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mds.f.service 2023-09-13T03:10:34.368 INFO:journalctl@ceph.mds.k.smithi142.stdout:-- Logs begin at Wed 2023-09-13 02:43:02 UTC. -- 2023-09-13T03:10:34.374 INFO:journalctl@ceph.mds.c.smithi170.stdout:-- Logs begin at Wed 2023-09-13 02:42:51 UTC. -- 2023-09-13T03:10:34.375 DEBUG:teuthology.orchestra.run.smithi170:mds.i> sudo journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mds.i.service 2023-09-13T03:10:34.400 INFO:journalctl@ceph.mds.f.smithi170.stdout:-- Logs begin at Wed 2023-09-13 02:42:51 UTC. -- 2023-09-13T03:10:34.401 DEBUG:teuthology.orchestra.run.smithi170:mds.l> sudo journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mds.l.service 2023-09-13T03:10:34.426 INFO:journalctl@ceph.mds.i.smithi170.stdout:-- Logs begin at Wed 2023-09-13 02:42:51 UTC. -- 2023-09-13T03:10:34.427 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-09-13T03:10:34.427 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-13T03:10:34.452 INFO:journalctl@ceph.mds.l.smithi170.stdout:-- Logs begin at Wed 2023-09-13 02:42:51 UTC. -- 2023-09-13T03:10:34.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.639+0000 7fd46c7ff700 1 -- 172.21.15.64:0/915753769 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd464100610 msgr2=0x7fd464100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:34.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.639+0000 7fd46c7ff700 1 --2- 172.21.15.64:0/915753769 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd464100610 0x7fd464100a30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd4600099c0 tx=0x7fd460009cd0 comp rx=0 tx=0).stop 2023-09-13T03:10:34.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.639+0000 7fd46c7ff700 1 -- 172.21.15.64:0/915753769 shutdown_connections 2023-09-13T03:10:34.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.639+0000 7fd46c7ff700 1 --2- 172.21.15.64:0/915753769 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd464100610 0x7fd464100a30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:34.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.639+0000 7fd46c7ff700 1 --2- 172.21.15.64:0/915753769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4641021a0 0x7fd464108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:34.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.639+0000 7fd46c7ff700 1 --2- 172.21.15.64:0/915753769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd464101800 0x7fd464101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:34.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.639+0000 7fd46c7ff700 1 -- 172.21.15.64:0/915753769 >> 172.21.15.64:0/915753769 conn(0x7fd4640fbb40 msgr2=0x7fd4640fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:10:34.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.639+0000 7fd46c7ff700 1 -- 172.21.15.64:0/915753769 shutdown_connections 2023-09-13T03:10:34.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.639+0000 7fd46c7ff700 1 -- 172.21.15.64:0/915753769 wait complete. 2023-09-13T03:10:34.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.641+0000 7fd46c7ff700 1 Processor -- start 2023-09-13T03:10:34.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.641+0000 7fd46c7ff700 1 -- start start 2023-09-13T03:10:34.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.641+0000 7fd46c7ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd464100610 0x7fd4641984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:34.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.641+0000 7fd46c7ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd464101800 0x7fd464198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:34.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.641+0000 7fd46c7ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4641021a0 0x7fd46419fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:34.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.642+0000 7fd46c7ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4641a0050 con 0x7fd4641021a0 2023-09-13T03:10:34.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.642+0000 7fd46c7ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4641a0190 con 0x7fd464101800 2023-09-13T03:10:34.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.642+0000 7fd46c7ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4641a0320 con 0x7fd464100610 2023-09-13T03:10:34.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.642+0000 7fd46a59b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd464100610 0x7fd4641984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:34.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.642+0000 7fd46a59b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd464100610 0x7fd4641984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:51218/0 (socket says 172.21.15.64:51218) 2023-09-13T03:10:34.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.642+0000 7fd46a59b700 1 -- 172.21.15.64:0/4028922281 learned_addr learned my addr 172.21.15.64:0/4028922281 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:10:34.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.642+0000 7fd46ad9c700 1 --2- 172.21.15.64:0/4028922281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4641021a0 0x7fd46419fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:34.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.642+0000 7fd469d9a700 1 --2- 172.21.15.64:0/4028922281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd464101800 0x7fd464198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:34.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.642+0000 7fd46a59b700 1 -- 172.21.15.64:0/4028922281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd464101800 msgr2=0x7fd464198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:34.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.642+0000 7fd46a59b700 1 --2- 172.21.15.64:0/4028922281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd464101800 0x7fd464198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:34.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.642+0000 7fd46a59b700 1 -- 172.21.15.64:0/4028922281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4641021a0 msgr2=0x7fd46419fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:34.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.642+0000 7fd46a59b700 1 --2- 172.21.15.64:0/4028922281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4641021a0 0x7fd46419fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:34.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.642+0000 7fd46a59b700 1 -- 172.21.15.64:0/4028922281 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4600095f0 con 0x7fd464100610 2023-09-13T03:10:34.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.642+0000 7fd46a59b700 1 --2- 172.21.15.64:0/4028922281 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd464100610 0x7fd4641984d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd4600058e0 tx=0x7fd46000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:10:34.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.643+0000 7fd45b7fe700 1 -- 172.21.15.64:0/4028922281 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46001d070 con 0x7fd464100610 2023-09-13T03:10:34.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.643+0000 7fd46c7ff700 1 -- 172.21.15.64:0/4028922281 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4641a05a0 con 0x7fd464100610 2023-09-13T03:10:34.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.643+0000 7fd45b7fe700 1 -- 172.21.15.64:0/4028922281 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd460005050 con 0x7fd464100610 2023-09-13T03:10:34.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.643+0000 7fd45b7fe700 1 -- 172.21.15.64:0/4028922281 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd460004b30 con 0x7fd464100610 2023-09-13T03:10:34.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.643+0000 7fd46c7ff700 1 -- 172.21.15.64:0/4028922281 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4641a09c0 con 0x7fd464100610 2023-09-13T03:10:34.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.644+0000 7fd46c7ff700 1 -- 172.21.15.64:0/4028922281 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd464192c30 con 0x7fd464100610 2023-09-13T03:10:34.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.646+0000 7fd45b7fe700 1 -- 172.21.15.64:0/4028922281 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fd4600051c0 con 0x7fd464100610 2023-09-13T03:10:34.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.647+0000 7fd45b7fe700 1 --2- 172.21.15.64:0/4028922281 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd4500a0690 0x7fd4500a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:34.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.647+0000 7fd45b7fe700 1 -- 172.21.15.64:0/4028922281 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd4600cca40 con 0x7fd464100610 2023-09-13T03:10:34.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.647+0000 7fd469d9a700 1 --2- 172.21.15.64:0/4028922281 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd4500a0690 0x7fd4500a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:34.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.648+0000 7fd469d9a700 1 --2- 172.21.15.64:0/4028922281 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd4500a0690 0x7fd4500a2b50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd454009c60 tx=0x7fd454009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:10:34.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.649+0000 7fd45b7fe700 1 -- 172.21.15.64:0/4028922281 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fd460096a00 con 0x7fd464100610 2023-09-13T03:10:34.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.863+0000 7fd46c7ff700 1 -- 172.21.15.64:0/4028922281 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd46410be40 con 0x7fd464100610 2023-09-13T03:10:34.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.864+0000 7fd45b7fe700 1 -- 172.21.15.64:0/4028922281 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18445 (secure 0 0 0) 0x7fd46009a020 con 0x7fd464100610 2023-09-13T03:10:34.865 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:10:34.866 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":79,"fsid":"f8dbced4-51e1-11ee-9ab7-7b867c8bd7da","created":"2023-09-13T03:03:24.167202+0000","modified":"2023-09-13T03:10:23.938509+0000","last_up_change":"2023-09-13T03:10:22.503529+0000","last_in_change":"2023-09-13T03:10:05.678720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T03:06:58.987479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3997c114-241b-48c4-8feb-653c99209470","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6803","nonce":3940892135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6805","nonce":3940892135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6809","nonce":3940892135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6807","nonce":3940892135}]},"public_addr":"172.21.15.64:6803/3940892135","cluster_addr":"172.21.15.64:6805/3940892135","heartbeat_back_addr":"172.21.15.64:6809/3940892135","heartbeat_front_addr":"172.21.15.64:6807/3940892135","state":["exists","up"]},{"osd":1,"uuid":"94cfae85-322e-4ed2-a6e6-e40349bb14bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6801","nonce":4188149333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6803","nonce":4188149333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6807","nonce":4188149333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6805","nonce":4188149333}]},"public_addr":"172.21.15.142:6801/4188149333","cluster_addr":"172.21.15.142:6803/4188149333","heartbeat_back_addr":"172.21.15.142:6807/4188149333","heartbeat_front_addr":"172.21.15.142:6805/4188149333","state":["exists","up"]},{"osd":2,"uuid":"b3aae17e-96db-4008-8c27-f733c46b30ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6801","nonce":2406669338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6803","nonce":2406669338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6807","nonce":2406669338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6805","nonce":2406669338}]},"public_addr":"172.21.15.170:6801/2406669338","cluster_addr":"172.21.15.170:6803/2406669338","heartbeat_back_addr":"172.21.15.170:6807/2406669338","heartbeat_front_addr":"172.21.15.170:6805/2406669338","state":["exists","up"]},{"osd":3,"uuid":"7fa8d8cb-2ab7-48a7-8b49-5e2903d6779f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6811","nonce":3558318707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6813","nonce":3558318707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6817","nonce":3558318707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6815","nonce":3558318707}]},"public_addr":"172.21.15.64:6811/3558318707","cluster_addr":"172.21.15.64:6813/3558318707","heartbeat_back_addr":"172.21.15.64:6817/3558318707","heartbeat_front_addr":"172.21.15.64:6815/3558318707","state":["exists","up"]},{"osd":4,"uuid":"8c7a9c9f-f919-48d3-9db2-214a25bb5603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6809","nonce":1811142576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6811","nonce":1811142576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6815","nonce":1811142576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6813","nonce":1811142576}]},"public_addr":"172.21.15.142:6809/1811142576","cluster_addr":"172.21.15.142:6811/1811142576","heartbeat_back_addr":"172.21.15.142:6815/1811142576","heartbeat_front_addr":"172.21.15.142:6813/1811142576","state":["exists","up"]},{"osd":5,"uuid":"0c6101c6-2065-4022-a751-d3f216d7f478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6809","nonce":2903841453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6811","nonce":2903841453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6815","nonce":2903841453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6813","nonce":2903841453}]},"public_addr":"172.21.15.170:6809/2903841453","cluster_addr":"172.21.15.170:6811/2903841453","heartbeat_back_addr":"172.21.15.170:6815/2903841453","heartbeat_front_addr":"172.21.15.170:6813/2903841453","state":["exists","up"]},{"osd":6,"uuid":"5cf3e52e-d155-41b1-9194-db047d26a953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6819","nonce":1436584490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6821","nonce":1436584490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6825","nonce":1436584490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6823","nonce":1436584490}]},"public_addr":"172.21.15.64:6819/1436584490","cluster_addr":"172.21.15.64:6821/1436584490","heartbeat_back_addr":"172.21.15.64:6825/1436584490","heartbeat_front_addr":"172.21.15.64:6823/1436584490","state":["exists","up"]},{"osd":7,"uuid":"e97323e3-badd-41a7-af9e-e78a81a64046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6817","nonce":3726793534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6819","nonce":3726793534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6823","nonce":3726793534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6821","nonce":3726793534}]},"public_addr":"172.21.15.142:6817/3726793534","cluster_addr":"172.21.15.142:6819/3726793534","heartbeat_back_addr":"172.21.15.142:6823/3726793534","heartbeat_front_addr":"172.21.15.142:6821/3726793534","state":["exists","up"]},{"osd":8,"uuid":"83d2835c-ed26-493d-9222-c2894ec0e071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6817","nonce":4272513873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6819","nonce":4272513873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6823","nonce":4272513873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6821","nonce":4272513873}]},"public_addr":"172.21.15.170:6817/4272513873","cluster_addr":"172.21.15.170:6819/4272513873","heartbeat_back_addr":"172.21.15.170:6823/4272513873","heartbeat_front_addr":"172.21.15.170:6821/4272513873","state":["exists","up"]},{"osd":9,"uuid":"cb936bdd-1bec-4236-aa2e-a24adcac11e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6827","nonce":2641175211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6829","nonce":2641175211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6833","nonce":2641175211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6831","nonce":2641175211}]},"public_addr":"172.21.15.64:6827/2641175211","cluster_addr":"172.21.15.64:6829/2641175211","heartbeat_back_addr":"172.21.15.64:6833/2641175211","heartbeat_front_addr":"172.21.15.64:6831/2641175211","state":["exists","up"]},{"osd":10,"uuid":"85512a1c-6171-43d4-9c15-c588a8b71be0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6825","nonce":936499217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6827","nonce":936499217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6831","nonce":936499217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6829","nonce":936499217}]},"public_addr":"172.21.15.142:6825/936499217","cluster_addr":"172.21.15.142:6827/936499217","heartbeat_back_addr":"172.21.15.142:6831/936499217","heartbeat_front_addr":"172.21.15.142:6829/936499217","state":["exists","up"]},{"osd":11,"uuid":"9b8417dc-5d6c-4917-94a9-b032c5bc33d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6825","nonce":3605017060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6827","nonce":3605017060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6831","nonce":3605017060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6829","nonce":3605017060}]},"public_addr":"172.21.15.170:6825/3605017060","cluster_addr":"172.21.15.170:6827/3605017060","heartbeat_back_addr":"172.21.15.170:6831/3605017060","heartbeat_front_addr":"172.21.15.170:6829/3605017060","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:14.229044+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:36.147777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:56.655298+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:07:23.107741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:07:44.663770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:05.577432+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:31.168031+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:52.791055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:09:13.658546+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:09:39.150360+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:10:00.601735+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:10:21.407555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6800/2955227347":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/1373223118":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/271194613":"2023-09-14T03:04:22.355817+0000","172.21.15.64:6801/3998091347":"2023-09-14T03:03:56.442541+0000","172.21.15.64:6800/3998091347":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/920818572":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/735789005":"2023-09-14T03:03:56.442541+0000","172.21.15.64:6801/2955227347":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/1975923178":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/1500151178":"2023-09-14T03:04:22.355817+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T03:10:34.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.867+0000 7fd46c7ff700 1 -- 172.21.15.64:0/4028922281 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd4500a0690 msgr2=0x7fd4500a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:34.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.867+0000 7fd46c7ff700 1 --2- 172.21.15.64:0/4028922281 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd4500a0690 0x7fd4500a2b50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd454009c60 tx=0x7fd454009260 comp rx=0 tx=0).stop 2023-09-13T03:10:34.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.867+0000 7fd46c7ff700 1 -- 172.21.15.64:0/4028922281 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd464100610 msgr2=0x7fd4641984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:34.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.867+0000 7fd46c7ff700 1 --2- 172.21.15.64:0/4028922281 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd464100610 0x7fd4641984d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd4600058e0 tx=0x7fd46000f690 comp rx=0 tx=0).stop 2023-09-13T03:10:34.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.867+0000 7fd46c7ff700 1 -- 172.21.15.64:0/4028922281 shutdown_connections 2023-09-13T03:10:34.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.867+0000 7fd46c7ff700 1 --2- 172.21.15.64:0/4028922281 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd4500a0690 0x7fd4500a2b50 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:34.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.867+0000 7fd46c7ff700 1 --2- 172.21.15.64:0/4028922281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4641021a0 0x7fd46419fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:34.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.867+0000 7fd46c7ff700 1 --2- 172.21.15.64:0/4028922281 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd464100610 0x7fd4641984d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:34.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.867+0000 7fd46c7ff700 1 --2- 172.21.15.64:0/4028922281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd464101800 0x7fd464198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:34.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.867+0000 7fd46c7ff700 1 -- 172.21.15.64:0/4028922281 >> 172.21.15.64:0/4028922281 conn(0x7fd4640fbb40 msgr2=0x7fd4641070e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:10:34.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.867+0000 7fd46c7ff700 1 -- 172.21.15.64:0/4028922281 shutdown_connections 2023-09-13T03:10:34.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:34.867+0000 7fd46c7ff700 1 -- 172.21.15.64:0/4028922281 wait complete. 2023-09-13T03:10:34.885 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:34 smithi064 ceph-mon[97675]: from='client.24550 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:10:34.886 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:34 smithi064 ceph-mon[97675]: Saving service mds.all spec with placement smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l;count:12 2023-09-13T03:10:34.886 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:34 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:34.886 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:34 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:10:34.886 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:34 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:34.887 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:34 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:10:34.887 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:34 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:34.887 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:34 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T03:10:34.887 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:34 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T03:10:34.887 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:34 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:34.887 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:34 smithi064 ceph-mon[97675]: Deploying daemon mds.l on smithi170 2023-09-13T03:10:34.888 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:34 smithi064 ceph-mon[97675]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:34.889 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-13T03:06:58.987479+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-13T03:10:34.890 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-13T03:10:34.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:34 smithi142 ceph-mon[134082]: from='client.24550 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:10:34.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:34 smithi142 ceph-mon[134082]: Saving service mds.all spec with placement smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l;count:12 2023-09-13T03:10:34.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:34 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:34.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:34 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:10:34.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:34 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:34.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:34 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:10:34.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:34 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:34.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:34 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T03:10:34.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:34 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T03:10:34.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:34 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:34.998 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:34 smithi142 ceph-mon[134082]: Deploying daemon mds.l on smithi170 2023-09-13T03:10:34.998 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:34 smithi142 ceph-mon[134082]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:35.024 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:34 smithi170 ceph-mon[134328]: from='client.24550 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:10:35.025 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:34 smithi170 ceph-mon[134328]: Saving service mds.all spec with placement smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l;count:12 2023-09-13T03:10:35.025 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:34 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:35.025 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:34 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:10:35.026 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:34 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:35.026 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:34 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:10:35.026 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:34 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:35.027 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:34 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T03:10:35.027 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:34 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T03:10:35.027 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:34 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:35.028 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:34 smithi170 ceph-mon[134328]: Deploying daemon mds.l on smithi170 2023-09-13T03:10:35.028 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:34 smithi170 ceph-mon[134328]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:35.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.096+0000 7f2a315ff700 1 -- 172.21.15.64:0/2745573137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a2c100610 msgr2=0x7f2a2c100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:35.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.096+0000 7f2a315ff700 1 --2- 172.21.15.64:0/2745573137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a2c100610 0x7f2a2c100a30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c0099c0 tx=0x7f2a1c009cd0 comp rx=0 tx=0).stop 2023-09-13T03:10:35.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.096+0000 7f2a315ff700 1 -- 172.21.15.64:0/2745573137 shutdown_connections 2023-09-13T03:10:35.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.096+0000 7f2a315ff700 1 --2- 172.21.15.64:0/2745573137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a2c1021a0 0x7f2a2c108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:35.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.096+0000 7f2a315ff700 1 --2- 172.21.15.64:0/2745573137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a2c100610 0x7f2a2c100a30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:35.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.096+0000 7f2a315ff700 1 --2- 172.21.15.64:0/2745573137 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a2c101800 0x7f2a2c101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:35.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.096+0000 7f2a315ff700 1 -- 172.21.15.64:0/2745573137 >> 172.21.15.64:0/2745573137 conn(0x7f2a2c0fbb80 msgr2=0x7f2a2c0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:10:35.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.096+0000 7f2a315ff700 1 -- 172.21.15.64:0/2745573137 shutdown_connections 2023-09-13T03:10:35.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.096+0000 7f2a315ff700 1 -- 172.21.15.64:0/2745573137 wait complete. 2023-09-13T03:10:35.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.097+0000 7f2a315ff700 1 Processor -- start 2023-09-13T03:10:35.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.097+0000 7f2a315ff700 1 -- start start 2023-09-13T03:10:35.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.097+0000 7f2a315ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a2c100610 0x7f2a2c196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:35.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.097+0000 7f2a315ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a2c101800 0x7f2a2c196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:35.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.097+0000 7f2a315ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a2c1021a0 0x7f2a2c19d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:35.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.097+0000 7f2a315ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c19dec0 con 0x7f2a2c100610 2023-09-13T03:10:35.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.097+0000 7f2a315ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c19e000 con 0x7f2a2c1021a0 2023-09-13T03:10:35.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.097+0000 7f2a315ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c19e190 con 0x7f2a2c101800 2023-09-13T03:10:35.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.098+0000 7f2a2affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a2c100610 0x7f2a2c196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:35.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.098+0000 7f2a2affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a2c100610 0x7f2a2c196340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60358/0 (socket says 172.21.15.64:60358) 2023-09-13T03:10:35.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.098+0000 7f2a2affd700 1 -- 172.21.15.64:0/3657654173 learned_addr learned my addr 172.21.15.64:0/3657654173 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:10:35.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.098+0000 7f2a2a7fc700 1 --2- 172.21.15.64:0/3657654173 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a2c101800 0x7f2a2c196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:35.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.098+0000 7f2a2b7fe700 1 --2- 172.21.15.64:0/3657654173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a2c1021a0 0x7f2a2c19d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:35.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.098+0000 7f2a2affd700 1 -- 172.21.15.64:0/3657654173 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a2c101800 msgr2=0x7f2a2c196880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:35.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.098+0000 7f2a2affd700 1 --2- 172.21.15.64:0/3657654173 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a2c101800 0x7f2a2c196880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:35.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.098+0000 7f2a2affd700 1 -- 172.21.15.64:0/3657654173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a2c1021a0 msgr2=0x7f2a2c19d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:35.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.098+0000 7f2a2affd700 1 --2- 172.21.15.64:0/3657654173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a2c1021a0 0x7f2a2c19d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:35.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.098+0000 7f2a2affd700 1 -- 172.21.15.64:0/3657654173 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a1c0095f0 con 0x7f2a2c100610 2023-09-13T03:10:35.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.098+0000 7f2a2b7fe700 1 --2- 172.21.15.64:0/3657654173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a2c1021a0 0x7f2a2c19d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:10:35.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.098+0000 7f2a2affd700 1 --2- 172.21.15.64:0/3657654173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a2c100610 0x7f2a2c196340 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c009fd0 tx=0x7f2a1c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:10:35.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.099+0000 7f2a13fff700 1 -- 172.21.15.64:0/3657654173 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c01d070 con 0x7f2a2c100610 2023-09-13T03:10:35.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.099+0000 7f2a315ff700 1 -- 172.21.15.64:0/3657654173 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a2c19e410 con 0x7f2a2c100610 2023-09-13T03:10:35.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.099+0000 7f2a13fff700 1 -- 172.21.15.64:0/3657654173 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a1c005510 con 0x7f2a2c100610 2023-09-13T03:10:35.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.099+0000 7f2a13fff700 1 -- 172.21.15.64:0/3657654173 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c004b00 con 0x7f2a2c100610 2023-09-13T03:10:35.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.099+0000 7f2a315ff700 1 -- 172.21.15.64:0/3657654173 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a2c19e890 con 0x7f2a2c100610 2023-09-13T03:10:35.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.100+0000 7f2a315ff700 1 -- 172.21.15.64:0/3657654173 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a2c10be80 con 0x7f2a2c100610 2023-09-13T03:10:35.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.102+0000 7f2a13fff700 1 -- 172.21.15.64:0/3657654173 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f2a1c004ca0 con 0x7f2a2c100610 2023-09-13T03:10:35.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.103+0000 7f2a13fff700 1 --2- 172.21.15.64:0/3657654173 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f2a140a0640 0x7f2a140a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:35.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.103+0000 7f2a13fff700 1 -- 172.21.15.64:0/3657654173 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2a1c0cc610 con 0x7f2a2c100610 2023-09-13T03:10:35.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.104+0000 7f2a2a7fc700 1 --2- 172.21.15.64:0/3657654173 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f2a140a0640 0x7f2a140a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:35.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.104+0000 7f2a2a7fc700 1 --2- 172.21.15.64:0/3657654173 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f2a140a0640 0x7f2a140a2b00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2a20009c60 tx=0x7f2a20009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:10:35.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.107+0000 7f2a13fff700 1 -- 172.21.15.64:0/3657654173 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f2a1c0965d0 con 0x7f2a2c100610 2023-09-13T03:10:35.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.319+0000 7f2a315ff700 1 -- 172.21.15.64:0/3657654173 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2a2c107560 con 0x7f2a2c100610 2023-09-13T03:10:35.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.320+0000 7f2a13fff700 1 -- 172.21.15.64:0/3657654173 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2a1c099bf0 con 0x7f2a2c100610 2023-09-13T03:10:35.321 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-09-13T03:10:35.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.322+0000 7f2a315ff700 1 -- 172.21.15.64:0/3657654173 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f2a140a0640 msgr2=0x7f2a140a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:35.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.322+0000 7f2a315ff700 1 --2- 172.21.15.64:0/3657654173 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f2a140a0640 0x7f2a140a2b00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2a20009c60 tx=0x7f2a20009260 comp rx=0 tx=0).stop 2023-09-13T03:10:35.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.322+0000 7f2a315ff700 1 -- 172.21.15.64:0/3657654173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a2c100610 msgr2=0x7f2a2c196340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:35.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.322+0000 7f2a315ff700 1 --2- 172.21.15.64:0/3657654173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a2c100610 0x7f2a2c196340 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c009fd0 tx=0x7f2a1c00f690 comp rx=0 tx=0).stop 2023-09-13T03:10:35.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.322+0000 7f2a315ff700 1 -- 172.21.15.64:0/3657654173 shutdown_connections 2023-09-13T03:10:35.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.322+0000 7f2a315ff700 1 --2- 172.21.15.64:0/3657654173 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f2a140a0640 0x7f2a140a2b00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:35.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.322+0000 7f2a315ff700 1 --2- 172.21.15.64:0/3657654173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a2c1021a0 0x7f2a2c19d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:35.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.322+0000 7f2a315ff700 1 --2- 172.21.15.64:0/3657654173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a2c100610 0x7f2a2c196340 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:35.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.322+0000 7f2a315ff700 1 --2- 172.21.15.64:0/3657654173 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a2c101800 0x7f2a2c196880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:35.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.322+0000 7f2a315ff700 1 -- 172.21.15.64:0/3657654173 >> 172.21.15.64:0/3657654173 conn(0x7f2a2c0fbb80 msgr2=0x7f2a2c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:10:35.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.322+0000 7f2a315ff700 1 -- 172.21.15.64:0/3657654173 shutdown_connections 2023-09-13T03:10:35.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.322+0000 7f2a315ff700 1 -- 172.21.15.64:0/3657654173 wait complete. 2023-09-13T03:10:35.342 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T03:10:35.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.545+0000 7f57ea3ff700 1 -- 172.21.15.64:0/3271182385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57e4100600 msgr2=0x7f57e4100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:35.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.545+0000 7f57ea3ff700 1 --2- 172.21.15.64:0/3271182385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57e4100600 0x7f57e4100a20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f57d40099c0 tx=0x7f57d4009cd0 comp rx=0 tx=0).stop 2023-09-13T03:10:35.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.545+0000 7f57ea3ff700 1 -- 172.21.15.64:0/3271182385 shutdown_connections 2023-09-13T03:10:35.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.545+0000 7f57ea3ff700 1 --2- 172.21.15.64:0/3271182385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57e4100600 0x7f57e4100a20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:35.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.545+0000 7f57ea3ff700 1 --2- 172.21.15.64:0/3271182385 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57e4102190 0x7f57e4108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:35.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.545+0000 7f57ea3ff700 1 --2- 172.21.15.64:0/3271182385 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57e41017f0 0x7f57e4101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:35.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.545+0000 7f57ea3ff700 1 -- 172.21.15.64:0/3271182385 >> 172.21.15.64:0/3271182385 conn(0x7f57e40fbb30 msgr2=0x7f57e40fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:10:35.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.545+0000 7f57ea3ff700 1 -- 172.21.15.64:0/3271182385 shutdown_connections 2023-09-13T03:10:35.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.545+0000 7f57ea3ff700 1 -- 172.21.15.64:0/3271182385 wait complete. 2023-09-13T03:10:35.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.546+0000 7f57ea3ff700 1 Processor -- start 2023-09-13T03:10:35.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.547+0000 7f57ea3ff700 1 -- start start 2023-09-13T03:10:35.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.547+0000 7f57ea3ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57e4100600 0x7f57e4198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:35.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.547+0000 7f57ea3ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57e41017f0 0x7f57e4198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:35.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.547+0000 7f57ea3ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57e4102190 0x7f57e419fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:35.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.547+0000 7f57ea3ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e41a0080 con 0x7f57e4100600 2023-09-13T03:10:35.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.547+0000 7f57ea3ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e41a01c0 con 0x7f57e4102190 2023-09-13T03:10:35.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.547+0000 7f57ea3ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e41a0350 con 0x7f57e41017f0 2023-09-13T03:10:35.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.547+0000 7f57e3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57e4100600 0x7f57e4198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:35.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.547+0000 7f57e3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57e4100600 0x7f57e4198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60368/0 (socket says 172.21.15.64:60368) 2023-09-13T03:10:35.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.547+0000 7f57e3fff700 1 -- 172.21.15.64:0/1952875173 learned_addr learned my addr 172.21.15.64:0/1952875173 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:10:35.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.547+0000 7f57e899c700 1 --2- 172.21.15.64:0/1952875173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57e4102190 0x7f57e419fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:35.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.547+0000 7f57e37fe700 1 --2- 172.21.15.64:0/1952875173 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57e41017f0 0x7f57e4198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:35.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.548+0000 7f57e3fff700 1 -- 172.21.15.64:0/1952875173 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57e41017f0 msgr2=0x7f57e4198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:35.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.548+0000 7f57e3fff700 1 --2- 172.21.15.64:0/1952875173 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57e41017f0 0x7f57e4198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:35.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.548+0000 7f57e3fff700 1 -- 172.21.15.64:0/1952875173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57e4102190 msgr2=0x7f57e419fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:35.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.548+0000 7f57e3fff700 1 --2- 172.21.15.64:0/1952875173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57e4102190 0x7f57e419fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:35.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.548+0000 7f57e3fff700 1 -- 172.21.15.64:0/1952875173 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d40095f0 con 0x7f57e4100600 2023-09-13T03:10:35.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.548+0000 7f57e37fe700 1 --2- 172.21.15.64:0/1952875173 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57e41017f0 0x7f57e4198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:10:35.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.548+0000 7f57e899c700 1 --2- 172.21.15.64:0/1952875173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57e4102190 0x7f57e419fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:10:35.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.548+0000 7f57e3fff700 1 --2- 172.21.15.64:0/1952875173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57e4100600 0x7f57e4198500 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f57d4009990 tx=0x7f57d400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:10:35.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.548+0000 7f57e17fa700 1 -- 172.21.15.64:0/1952875173 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d401d070 con 0x7f57e4100600 2023-09-13T03:10:35.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.548+0000 7f57e17fa700 1 -- 172.21.15.64:0/1952875173 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57d4005030 con 0x7f57e4100600 2023-09-13T03:10:35.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.548+0000 7f57e17fa700 1 -- 172.21.15.64:0/1952875173 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d4004a70 con 0x7f57e4100600 2023-09-13T03:10:35.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.548+0000 7f57ea3ff700 1 -- 172.21.15.64:0/1952875173 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57e41a05d0 con 0x7f57e4100600 2023-09-13T03:10:35.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.548+0000 7f57ea3ff700 1 -- 172.21.15.64:0/1952875173 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e41a09a0 con 0x7f57e4100600 2023-09-13T03:10:35.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.549+0000 7f57ea3ff700 1 -- 172.21.15.64:0/1952875173 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57e410be70 con 0x7f57e4100600 2023-09-13T03:10:35.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.551+0000 7f57e17fa700 1 -- 172.21.15.64:0/1952875173 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f57d4005410 con 0x7f57e4100600 2023-09-13T03:10:35.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.553+0000 7f57e17fa700 1 --2- 172.21.15.64:0/1952875173 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f57cc0a0690 0x7f57cc0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:35.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.553+0000 7f57e17fa700 1 -- 172.21.15.64:0/1952875173 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f57d40cc520 con 0x7f57e4100600 2023-09-13T03:10:35.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.553+0000 7f57e37fe700 1 --2- 172.21.15.64:0/1952875173 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f57cc0a0690 0x7f57cc0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:35.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.554+0000 7f57e37fe700 1 --2- 172.21.15.64:0/1952875173 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f57cc0a0690 0x7f57cc0a2b50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f57d8011490 tx=0x7f57d8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:10:35.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.557+0000 7f57e17fa700 1 -- 172.21.15.64:0/1952875173 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f57d40964e0 con 0x7f57e4100600 2023-09-13T03:10:35.638 INFO:journalctl@ceph.mds.l.smithi170.stdout:Sep 13 03:10:35 smithi170 systemd[1]: Starting Ceph mds.l for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:10:35.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.811+0000 7f57ea3ff700 1 -- 172.21.15.64:0/1952875173 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f57e410c0b0 con 0x7f57e4100600 2023-09-13T03:10:35.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.812+0000 7f57e17fa700 1 -- 172.21.15.64:0/1952875173 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7f57d4099b00 con 0x7f57e4100600 2023-09-13T03:10:35.813 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:10:35.813 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-09-13T03:10:35.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.814+0000 7f57ea3ff700 1 -- 172.21.15.64:0/1952875173 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f57cc0a0690 msgr2=0x7f57cc0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:35.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.814+0000 7f57ea3ff700 1 --2- 172.21.15.64:0/1952875173 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f57cc0a0690 0x7f57cc0a2b50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f57d8011490 tx=0x7f57d8009260 comp rx=0 tx=0).stop 2023-09-13T03:10:35.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.814+0000 7f57ea3ff700 1 -- 172.21.15.64:0/1952875173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57e4100600 msgr2=0x7f57e4198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:35.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.814+0000 7f57ea3ff700 1 --2- 172.21.15.64:0/1952875173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57e4100600 0x7f57e4198500 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f57d4009990 tx=0x7f57d400f690 comp rx=0 tx=0).stop 2023-09-13T03:10:35.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.814+0000 7f57ea3ff700 1 -- 172.21.15.64:0/1952875173 shutdown_connections 2023-09-13T03:10:35.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.814+0000 7f57ea3ff700 1 --2- 172.21.15.64:0/1952875173 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f57cc0a0690 0x7f57cc0a2b50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:35.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.814+0000 7f57ea3ff700 1 --2- 172.21.15.64:0/1952875173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57e4102190 0x7f57e419fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:35.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.814+0000 7f57ea3ff700 1 --2- 172.21.15.64:0/1952875173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57e4100600 0x7f57e4198500 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:35.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.814+0000 7f57ea3ff700 1 --2- 172.21.15.64:0/1952875173 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57e41017f0 0x7f57e4198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:35.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.814+0000 7f57ea3ff700 1 -- 172.21.15.64:0/1952875173 >> 172.21.15.64:0/1952875173 conn(0x7f57e40fbb30 msgr2=0x7f57e40fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:10:35.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.815+0000 7f57ea3ff700 1 -- 172.21.15.64:0/1952875173 shutdown_connections 2023-09-13T03:10:35.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:35.815+0000 7f57ea3ff700 1 -- 172.21.15.64:0/1952875173 wait complete. 2023-09-13T03:10:35.817 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 1 2023-09-13T03:10:35.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:35 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/4028922281' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:10:35.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:35 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/3657654173' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T03:10:35.910 INFO:journalctl@ceph.mds.l.smithi170.stdout:Sep 13 03:10:35 smithi170 podman[171264]: 2023-09-13T03:10:35.910 INFO:journalctl@ceph.mds.l.smithi170.stdout:Sep 13 03:10:35 smithi170 ceph-mds[171305]: -- 172.21.15.170:0/929429752 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55efbe7211e0 con 0x55efbe6f7000 2023-09-13T03:10:35.910 INFO:journalctl@ceph.mds.l.smithi170.stdout:Sep 13 03:10:35 smithi170 bash[171264]: be07ceacb357ebdd453ab631ce95f2d983e18d1a6c5650d171d46c1054d9e835 2023-09-13T03:10:35.911 INFO:journalctl@ceph.mds.l.smithi170.stdout:Sep 13 03:10:35 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-l[171280]: starting mds.l at 2023-09-13T03:10:35.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:35 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/4028922281' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:10:35.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:35 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/3657654173' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T03:10:36.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:35 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/4028922281' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:10:36.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:35 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/3657654173' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T03:10:36.410 INFO:journalctl@ceph.mds.l.smithi170.stdout:Sep 13 03:10:35 smithi170 systemd[1]: Started Ceph mds.l for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da. 2023-09-13T03:10:36.970 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:36 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/1952875173' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:10:36.971 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:36 smithi142 ceph-mon[134082]: mds.? [v2:172.21.15.170:6832/1897366227,v1:172.21.15.170:6833/1897366227] up:boot 2023-09-13T03:10:36.971 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:36 smithi142 ceph-mon[134082]: fsmap 1 up:standby 2023-09-13T03:10:36.972 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:36 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-13T03:10:36.972 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:36 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:36.972 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:36 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:36.973 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:36 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:36.973 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:36 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T03:10:36.973 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:36 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T03:10:36.974 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:36 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:36.974 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:36 smithi142 ceph-mon[134082]: Deploying daemon mds.k on smithi142 2023-09-13T03:10:36.975 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:36 smithi142 ceph-mon[134082]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:37.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:36 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/1952875173' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:10:37.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:36 smithi170 ceph-mon[134328]: mds.? [v2:172.21.15.170:6832/1897366227,v1:172.21.15.170:6833/1897366227] up:boot 2023-09-13T03:10:37.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:36 smithi170 ceph-mon[134328]: fsmap 1 up:standby 2023-09-13T03:10:37.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:36 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-13T03:10:37.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:36 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:37.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:36 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:37.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:36 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:37.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:36 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T03:10:37.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:36 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T03:10:37.162 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:36 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:37.162 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:36 smithi170 ceph-mon[134328]: Deploying daemon mds.k on smithi142 2023-09-13T03:10:37.162 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:36 smithi170 ceph-mon[134328]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:37.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:36 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/1952875173' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:10:37.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:36 smithi064 ceph-mon[97675]: mds.? [v2:172.21.15.170:6832/1897366227,v1:172.21.15.170:6833/1897366227] up:boot 2023-09-13T03:10:37.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:36 smithi064 ceph-mon[97675]: fsmap 1 up:standby 2023-09-13T03:10:37.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:36 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-13T03:10:37.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:36 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:37.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:36 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:37.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:36 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:37.170 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:36 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T03:10:37.170 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:36 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T03:10:37.170 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:36 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:37.170 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:36 smithi064 ceph-mon[97675]: Deploying daemon mds.k on smithi142 2023-09-13T03:10:37.171 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:36 smithi064 ceph-mon[97675]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:37.745 INFO:journalctl@ceph.mds.k.smithi142.stdout:Sep 13 03:10:37 smithi142 systemd[1]: Starting Ceph mds.k for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:10:37.834 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T03:10:38.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.049+0000 7fc5619ff700 1 -- 172.21.15.64:0/1219356247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc55c1017f0 msgr2=0x7fc55c101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:38.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.049+0000 7fc5619ff700 1 --2- 172.21.15.64:0/1219356247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc55c1017f0 0x7fc55c101c50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc5500099c0 tx=0x7fc550009cd0 comp rx=0 tx=0).stop 2023-09-13T03:10:38.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.050+0000 7fc5619ff700 1 -- 172.21.15.64:0/1219356247 shutdown_connections 2023-09-13T03:10:38.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.050+0000 7fc5619ff700 1 --2- 172.21.15.64:0/1219356247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc55c102190 0x7fc55c108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:38.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.050+0000 7fc5619ff700 1 --2- 172.21.15.64:0/1219356247 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc55c100600 0x7fc55c100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:38.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.050+0000 7fc5619ff700 1 --2- 172.21.15.64:0/1219356247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc55c1017f0 0x7fc55c101c50 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:38.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.050+0000 7fc5619ff700 1 -- 172.21.15.64:0/1219356247 >> 172.21.15.64:0/1219356247 conn(0x7fc55c0fbb30 msgr2=0x7fc55c0fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:10:38.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.050+0000 7fc5619ff700 1 -- 172.21.15.64:0/1219356247 shutdown_connections 2023-09-13T03:10:38.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.050+0000 7fc5619ff700 1 -- 172.21.15.64:0/1219356247 wait complete. 2023-09-13T03:10:38.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.051+0000 7fc5619ff700 1 Processor -- start 2023-09-13T03:10:38.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.051+0000 7fc5619ff700 1 -- start start 2023-09-13T03:10:38.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.051+0000 7fc5619ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc55c100600 0x7fc55c1962f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:38.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.051+0000 7fc5619ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc55c1017f0 0x7fc55c196830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:38.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.051+0000 7fc5619ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc55c102190 0x7fc55c19d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:38.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.051+0000 7fc5619ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c19de70 con 0x7fc55c102190 2023-09-13T03:10:38.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.051+0000 7fc5619ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c19dfb0 con 0x7fc55c100600 2023-09-13T03:10:38.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.051+0000 7fc5619ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c19e140 con 0x7fc55c1017f0 2023-09-13T03:10:38.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.051+0000 7fc55affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc55c100600 0x7fc55c1962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:38.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.051+0000 7fc55affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc55c100600 0x7fc55c1962f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:54676/0 (socket says 172.21.15.64:54676) 2023-09-13T03:10:38.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.051+0000 7fc55b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc55c102190 0x7fc55c19d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:38.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.051+0000 7fc55affd700 1 -- 172.21.15.64:0/2805659487 learned_addr learned my addr 172.21.15.64:0/2805659487 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:10:38.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.051+0000 7fc55a7fc700 1 --2- 172.21.15.64:0/2805659487 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc55c1017f0 0x7fc55c196830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:38.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.052+0000 7fc55affd700 1 -- 172.21.15.64:0/2805659487 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc55c1017f0 msgr2=0x7fc55c196830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:38.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.052+0000 7fc55affd700 1 --2- 172.21.15.64:0/2805659487 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc55c1017f0 0x7fc55c196830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:38.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.052+0000 7fc55affd700 1 -- 172.21.15.64:0/2805659487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc55c102190 msgr2=0x7fc55c19d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:38.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.052+0000 7fc55affd700 1 --2- 172.21.15.64:0/2805659487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc55c102190 0x7fc55c19d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:38.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.052+0000 7fc55affd700 1 -- 172.21.15.64:0/2805659487 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5500095f0 con 0x7fc55c100600 2023-09-13T03:10:38.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.052+0000 7fc55b7fe700 1 --2- 172.21.15.64:0/2805659487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc55c102190 0x7fc55c19d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:10:38.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.052+0000 7fc55affd700 1 --2- 172.21.15.64:0/2805659487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc55c100600 0x7fc55c1962f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc54c00e970 tx=0x7fc54c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:10:38.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.052+0000 7fc5609fd700 1 -- 172.21.15.64:0/2805659487 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c009790 con 0x7fc55c100600 2023-09-13T03:10:38.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.052+0000 7fc5609fd700 1 -- 172.21.15.64:0/2805659487 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc54c00c7e0 con 0x7fc55c100600 2023-09-13T03:10:38.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.052+0000 7fc5609fd700 1 -- 172.21.15.64:0/2805659487 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c003c70 con 0x7fc55c100600 2023-09-13T03:10:38.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.052+0000 7fc5619ff700 1 -- 172.21.15.64:0/2805659487 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc55c19e420 con 0x7fc55c100600 2023-09-13T03:10:38.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.053+0000 7fc5619ff700 1 -- 172.21.15.64:0/2805659487 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc55c19e8e0 con 0x7fc55c100600 2023-09-13T03:10:38.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.053+0000 7fc5619ff700 1 -- 172.21.15.64:0/2805659487 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc55c10be60 con 0x7fc55c100600 2023-09-13T03:10:38.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.056+0000 7fc5609fd700 1 -- 172.21.15.64:0/2805659487 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fc54c00cc50 con 0x7fc55c100600 2023-09-13T03:10:38.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.057+0000 7fc5609fd700 1 --2- 172.21.15.64:0/2805659487 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc5440a0710 0x7fc5440a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:38.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.057+0000 7fc5609fd700 1 -- 172.21.15.64:0/2805659487 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc54c014070 con 0x7fc55c100600 2023-09-13T03:10:38.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.057+0000 7fc55a7fc700 1 --2- 172.21.15.64:0/2805659487 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc5440a0710 0x7fc5440a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:38.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.058+0000 7fc55a7fc700 1 --2- 172.21.15.64:0/2805659487 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc5440a0710 0x7fc5440a2bd0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc55000b580 tx=0x7fc550005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:10:38.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.059+0000 7fc5609fd700 1 -- 172.21.15.64:0/2805659487 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fc54c095a50 con 0x7fc55c100600 2023-09-13T03:10:38.068 INFO:journalctl@ceph.mds.k.smithi142.stdout:Sep 13 03:10:37 smithi142 podman[169624]: 2023-09-13T03:10:38.069 INFO:journalctl@ceph.mds.k.smithi142.stdout:Sep 13 03:10:38 smithi142 ceph-mds[169664]: -- 172.21.15.142:0/3783588272 <== mon.2 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564218b031e0 con 0x564218ad8800 2023-09-13T03:10:38.069 INFO:journalctl@ceph.mds.k.smithi142.stdout:Sep 13 03:10:38 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-k[169640]: starting mds.k at 2023-09-13T03:10:38.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.329+0000 7fc5619ff700 1 -- 172.21.15.64:0/2805659487 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc55c107530 con 0x7fc55c100600 2023-09-13T03:10:38.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.329+0000 7fc5609fd700 1 -- 172.21.15.64:0/2805659487 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2119 (secure 0 0 0) 0x7fc54c099070 con 0x7fc55c100600 2023-09-13T03:10:38.330 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:10:38.330 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24569,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/101906999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":101906999},{"type":"v1","addr":"172.21.15.142:6833","nonce":101906999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1897366227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1897366227},{"type":"v1","addr":"172.21.15.170:6833","nonce":1897366227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-09-13T03:10:38.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.332+0000 7fc5619ff700 1 -- 172.21.15.64:0/2805659487 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc5440a0710 msgr2=0x7fc5440a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:38.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.332+0000 7fc5619ff700 1 --2- 172.21.15.64:0/2805659487 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc5440a0710 0x7fc5440a2bd0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc55000b580 tx=0x7fc550005fb0 comp rx=0 tx=0).stop 2023-09-13T03:10:38.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.332+0000 7fc5619ff700 1 -- 172.21.15.64:0/2805659487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc55c100600 msgr2=0x7fc55c1962f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:38.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.332+0000 7fc5619ff700 1 --2- 172.21.15.64:0/2805659487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc55c100600 0x7fc55c1962f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc54c00e970 tx=0x7fc54c00ec80 comp rx=0 tx=0).stop 2023-09-13T03:10:38.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.332+0000 7fc5619ff700 1 -- 172.21.15.64:0/2805659487 shutdown_connections 2023-09-13T03:10:38.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.332+0000 7fc5619ff700 1 --2- 172.21.15.64:0/2805659487 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc5440a0710 0x7fc5440a2bd0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:38.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.332+0000 7fc5619ff700 1 --2- 172.21.15.64:0/2805659487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc55c100600 0x7fc55c1962f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:38.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.332+0000 7fc5619ff700 1 --2- 172.21.15.64:0/2805659487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc55c102190 0x7fc55c19d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:38.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.332+0000 7fc5619ff700 1 --2- 172.21.15.64:0/2805659487 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc55c1017f0 0x7fc55c196830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:38.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.332+0000 7fc5619ff700 1 -- 172.21.15.64:0/2805659487 >> 172.21.15.64:0/2805659487 conn(0x7fc55c0fbb30 msgr2=0x7fc55c0fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:10:38.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.332+0000 7fc5619ff700 1 -- 172.21.15.64:0/2805659487 shutdown_connections 2023-09-13T03:10:38.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:38.332+0000 7fc5619ff700 1 -- 172.21.15.64:0/2805659487 wait complete. 2023-09-13T03:10:38.335 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 3 2023-09-13T03:10:38.495 INFO:journalctl@ceph.mds.k.smithi142.stdout:Sep 13 03:10:38 smithi142 bash[169624]: b9ea338403756d0df6990e396c2f18c533bb688272916590ccf323446f77a650 2023-09-13T03:10:38.495 INFO:journalctl@ceph.mds.k.smithi142.stdout:Sep 13 03:10:38 smithi142 systemd[1]: Started Ceph mds.k for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da. 2023-09-13T03:10:39.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:39 smithi170 ceph-mon[134328]: mds.? [v2:172.21.15.142:6832/101906999,v1:172.21.15.142:6833/101906999] up:boot 2023-09-13T03:10:39.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:39 smithi170 ceph-mon[134328]: fsmap 2 up:standby 2023-09-13T03:10:39.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:39 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-13T03:10:39.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:39 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:39.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:39 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:39.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:39 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:39.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:39 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T03:10:39.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:39 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T03:10:39.412 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:39 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:39.412 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:39 smithi170 ceph-mon[134328]: Deploying daemon mds.j on smithi064 2023-09-13T03:10:39.412 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:39 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/2805659487' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:10:39.412 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:39 smithi170 ceph-mon[134328]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:39.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:39 smithi064 ceph-mon[97675]: mds.? [v2:172.21.15.142:6832/101906999,v1:172.21.15.142:6833/101906999] up:boot 2023-09-13T03:10:39.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:39 smithi064 ceph-mon[97675]: fsmap 2 up:standby 2023-09-13T03:10:39.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:39 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-13T03:10:39.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:39 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:39.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:39 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:39.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:39 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:39.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:39 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T03:10:39.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:39 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T03:10:39.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:39 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:39.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:39 smithi064 ceph-mon[97675]: Deploying daemon mds.j on smithi064 2023-09-13T03:10:39.421 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:39 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2805659487' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:10:39.421 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:39 smithi064 ceph-mon[97675]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:39.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:39 smithi142 ceph-mon[134082]: mds.? [v2:172.21.15.142:6832/101906999,v1:172.21.15.142:6833/101906999] up:boot 2023-09-13T03:10:39.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:39 smithi142 ceph-mon[134082]: fsmap 2 up:standby 2023-09-13T03:10:39.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:39 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-13T03:10:39.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:39 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:39.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:39 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:39.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:39 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:39.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:39 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T03:10:39.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:39 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T03:10:39.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:39 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:39.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:39 smithi142 ceph-mon[134082]: Deploying daemon mds.j on smithi064 2023-09-13T03:10:39.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:39 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/2805659487' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:10:39.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:39 smithi142 ceph-mon[134082]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:40.353 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T03:10:40.374 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 13 03:10:40 smithi064 systemd[1]: Starting Ceph mds.j for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:10:40.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.578+0000 7f7ec6bff700 1 -- 172.21.15.64:0/1761589917 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ec0100610 msgr2=0x7f7ec0100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:40.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.578+0000 7f7ec6bff700 1 --2- 172.21.15.64:0/1761589917 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ec0100610 0x7f7ec0100a30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7eb40099c0 tx=0x7f7eb4009cd0 comp rx=0 tx=0).stop 2023-09-13T03:10:40.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.578+0000 7f7ec6bff700 1 -- 172.21.15.64:0/1761589917 shutdown_connections 2023-09-13T03:10:40.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.578+0000 7f7ec6bff700 1 --2- 172.21.15.64:0/1761589917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ec01021a0 0x7f7ec0108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:40.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.578+0000 7f7ec6bff700 1 --2- 172.21.15.64:0/1761589917 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ec0100610 0x7f7ec0100a30 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:40.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.578+0000 7f7ec6bff700 1 --2- 172.21.15.64:0/1761589917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ec0101800 0x7f7ec0101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:40.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.578+0000 7f7ec6bff700 1 -- 172.21.15.64:0/1761589917 >> 172.21.15.64:0/1761589917 conn(0x7f7ec00fbb40 msgr2=0x7f7ec00fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:10:40.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.578+0000 7f7ec6bff700 1 -- 172.21.15.64:0/1761589917 shutdown_connections 2023-09-13T03:10:40.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.578+0000 7f7ec6bff700 1 -- 172.21.15.64:0/1761589917 wait complete. 2023-09-13T03:10:40.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.579+0000 7f7ec6bff700 1 Processor -- start 2023-09-13T03:10:40.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.579+0000 7f7ec6bff700 1 -- start start 2023-09-13T03:10:40.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.579+0000 7f7ec6bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ec0100610 0x7f7ec0198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:40.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.579+0000 7f7ec6bff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ec0101800 0x7f7ec0198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:40.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.579+0000 7f7ec6bff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ec01021a0 0x7f7ec019fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:40.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.579+0000 7f7ec6bff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec01a0080 con 0x7f7ec01021a0 2023-09-13T03:10:40.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.579+0000 7f7ec6bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec01a01c0 con 0x7f7ec0100610 2023-09-13T03:10:40.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.579+0000 7f7ec6bff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec01a0350 con 0x7f7ec0101800 2023-09-13T03:10:40.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.580+0000 7f7ebffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ec0101800 0x7f7ec0198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:40.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.580+0000 7f7ec499b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ec0100610 0x7f7ec0198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:40.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.580+0000 7f7ebffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ec0101800 0x7f7ec0198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:60618/0 (socket says 172.21.15.64:60618) 2023-09-13T03:10:40.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.580+0000 7f7ebffff700 1 -- 172.21.15.64:0/2168655576 learned_addr learned my addr 172.21.15.64:0/2168655576 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:10:40.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.580+0000 7f7ec519c700 1 --2- 172.21.15.64:0/2168655576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ec01021a0 0x7f7ec019fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:40.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.580+0000 7f7ebffff700 1 -- 172.21.15.64:0/2168655576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ec0100610 msgr2=0x7f7ec0198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:40.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.580+0000 7f7ebffff700 1 --2- 172.21.15.64:0/2168655576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ec0100610 0x7f7ec0198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:40.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.580+0000 7f7ebffff700 1 -- 172.21.15.64:0/2168655576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ec01021a0 msgr2=0x7f7ec019fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:40.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.580+0000 7f7ebffff700 1 --2- 172.21.15.64:0/2168655576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ec01021a0 0x7f7ec019fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:40.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.580+0000 7f7ebffff700 1 -- 172.21.15.64:0/2168655576 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb40095f0 con 0x7f7ec0101800 2023-09-13T03:10:40.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.582+0000 7f7ebffff700 1 --2- 172.21.15.64:0/2168655576 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ec0101800 0x7f7ec0198a40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00e9a0 tx=0x7f7eac00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:10:40.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.582+0000 7f7ebdffb700 1 -- 172.21.15.64:0/2168655576 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eac009790 con 0x7f7ec0101800 2023-09-13T03:10:40.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.582+0000 7f7ebdffb700 1 -- 172.21.15.64:0/2168655576 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7eac00cbd0 con 0x7f7ec0101800 2023-09-13T03:10:40.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.582+0000 7f7ebdffb700 1 -- 172.21.15.64:0/2168655576 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eac018680 con 0x7f7ec0101800 2023-09-13T03:10:40.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.582+0000 7f7ec6bff700 1 -- 172.21.15.64:0/2168655576 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ec0102a80 con 0x7f7ec0101800 2023-09-13T03:10:40.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.582+0000 7f7ec6bff700 1 -- 172.21.15.64:0/2168655576 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ec0102f90 con 0x7f7ec0101800 2023-09-13T03:10:40.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.583+0000 7f7ec6bff700 1 -- 172.21.15.64:0/2168655576 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ec0192c70 con 0x7f7ec0101800 2023-09-13T03:10:40.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.585+0000 7f7ebdffb700 1 -- 172.21.15.64:0/2168655576 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f7eac00cd40 con 0x7f7ec0101800 2023-09-13T03:10:40.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.587+0000 7f7ebdffb700 1 --2- 172.21.15.64:0/2168655576 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f7eb00a0710 0x7f7eb00a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:40.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.587+0000 7f7ebdffb700 1 -- 172.21.15.64:0/2168655576 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7eac014070 con 0x7f7ec0101800 2023-09-13T03:10:40.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.587+0000 7f7ec499b700 1 --2- 172.21.15.64:0/2168655576 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f7eb00a0710 0x7f7eb00a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:40.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.589+0000 7f7ec499b700 1 --2- 172.21.15.64:0/2168655576 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f7eb00a0710 0x7f7eb00a2bd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4009990 tx=0x7f7eb4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:10:40.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.589+0000 7f7ebdffb700 1 -- 172.21.15.64:0/2168655576 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f7eac095c80 con 0x7f7ec0101800 2023-09-13T03:10:40.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.857+0000 7f7ec6bff700 1 -- 172.21.15.64:0/2168655576 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7ec010c0d0 con 0x7f7ec0101800 2023-09-13T03:10:40.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.858+0000 7f7ebdffb700 1 -- 172.21.15.64:0/2168655576 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2119 (secure 0 0 0) 0x7f7eac0992a0 con 0x7f7ec0101800 2023-09-13T03:10:40.858 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:10:40.858 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24569,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/101906999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":101906999},{"type":"v1","addr":"172.21.15.142:6833","nonce":101906999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1897366227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1897366227},{"type":"v1","addr":"172.21.15.170:6833","nonce":1897366227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-09-13T03:10:40.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.861+0000 7f7ec6bff700 1 -- 172.21.15.64:0/2168655576 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f7eb00a0710 msgr2=0x7f7eb00a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:40.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.861+0000 7f7ec6bff700 1 --2- 172.21.15.64:0/2168655576 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f7eb00a0710 0x7f7eb00a2bd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4009990 tx=0x7f7eb4005fb0 comp rx=0 tx=0).stop 2023-09-13T03:10:40.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.861+0000 7f7ec6bff700 1 -- 172.21.15.64:0/2168655576 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ec0101800 msgr2=0x7f7ec0198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:40.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.861+0000 7f7ec6bff700 1 --2- 172.21.15.64:0/2168655576 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ec0101800 0x7f7ec0198a40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00e9a0 tx=0x7f7eac00ecb0 comp rx=0 tx=0).stop 2023-09-13T03:10:40.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.861+0000 7f7ec6bff700 1 -- 172.21.15.64:0/2168655576 shutdown_connections 2023-09-13T03:10:40.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.861+0000 7f7ec6bff700 1 --2- 172.21.15.64:0/2168655576 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f7eb00a0710 0x7f7eb00a2bd0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:40.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.861+0000 7f7ec6bff700 1 --2- 172.21.15.64:0/2168655576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ec0100610 0x7f7ec0198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:40.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.861+0000 7f7ec6bff700 1 --2- 172.21.15.64:0/2168655576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ec01021a0 0x7f7ec019fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:40.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.861+0000 7f7ec6bff700 1 --2- 172.21.15.64:0/2168655576 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ec0101800 0x7f7ec0198a40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:40.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.861+0000 7f7ec6bff700 1 -- 172.21.15.64:0/2168655576 >> 172.21.15.64:0/2168655576 conn(0x7f7ec00fbb40 msgr2=0x7f7ec01070f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:10:40.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.861+0000 7f7ec6bff700 1 -- 172.21.15.64:0/2168655576 shutdown_connections 2023-09-13T03:10:40.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:40.861+0000 7f7ec6bff700 1 -- 172.21.15.64:0/2168655576 wait complete. 2023-09-13T03:10:40.863 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 3 2023-09-13T03:10:41.168 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 13 03:10:40 smithi064 podman[151182]: 2023-09-13T03:10:41.168 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 13 03:10:41 smithi064 bash[151182]: 619defdce61dced448c2e2484c02b257f2de0b6a90e10ece7958c4200bd9d106 2023-09-13T03:10:41.169 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 13 03:10:41 smithi064 ceph-mds[151224]: -- 172.21.15.64:0/2933004438 <== mon.0 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555a415551e0 con 0x555a4152ac00 2023-09-13T03:10:41.169 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 13 03:10:41 smithi064 systemd[1]: Started Ceph mds.j for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da. 2023-09-13T03:10:41.169 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 13 03:10:41 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-j[151200]: starting mds.j at 2023-09-13T03:10:41.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:41 smithi170 ceph-mon[134328]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:41.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:41 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/2168655576' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:10:41.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:41 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:41.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:41 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:41.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:41 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:41.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:41 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T03:10:41.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:41 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T03:10:41.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:41 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:41.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:41 smithi064 ceph-mon[97675]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:41.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:41 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2168655576' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:10:41.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:41 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:41.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:41 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:41.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:41 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:41.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:41 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T03:10:41.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:41 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T03:10:41.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:41 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:41.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:41 smithi142 ceph-mon[134082]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:41.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:41 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/2168655576' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:10:41.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:41 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:41.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:41 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:41.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:41 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:41.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:41 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T03:10:41.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:41 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T03:10:41.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:41 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:42.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:42 smithi064 ceph-mon[97675]: Deploying daemon mds.d on smithi064 2023-09-13T03:10:42.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:42 smithi064 ceph-mon[97675]: mds.? [v2:172.21.15.64:6834/2234182985,v1:172.21.15.64:6835/2234182985] up:boot 2023-09-13T03:10:42.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:42 smithi064 ceph-mon[97675]: fsmap 3 up:standby 2023-09-13T03:10:42.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:42 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-13T03:10:42.880 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T03:10:42.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:42 smithi170 ceph-mon[134328]: Deploying daemon mds.d on smithi064 2023-09-13T03:10:42.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:42 smithi170 ceph-mon[134328]: mds.? [v2:172.21.15.64:6834/2234182985,v1:172.21.15.64:6835/2234182985] up:boot 2023-09-13T03:10:42.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:42 smithi170 ceph-mon[134328]: fsmap 3 up:standby 2023-09-13T03:10:42.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:42 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-13T03:10:42.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:42 smithi142 ceph-mon[134082]: Deploying daemon mds.d on smithi064 2023-09-13T03:10:42.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:42 smithi142 ceph-mon[134082]: mds.? [v2:172.21.15.64:6834/2234182985,v1:172.21.15.64:6835/2234182985] up:boot 2023-09-13T03:10:42.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:42 smithi142 ceph-mon[134082]: fsmap 3 up:standby 2023-09-13T03:10:42.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:42 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-13T03:10:43.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.130+0000 7fb1fc9ff700 1 -- 172.21.15.64:0/3735449266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1f810cbd0 msgr2=0x7fb1f810efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:43.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.130+0000 7fb1fc9ff700 1 --2- 172.21.15.64:0/3735449266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1f810cbd0 0x7fb1f810efc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb1f000b0a0 tx=0x7fb1f000b3b0 comp rx=0 tx=0).stop 2023-09-13T03:10:43.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.130+0000 7fb1fc9ff700 1 -- 172.21.15.64:0/3735449266 shutdown_connections 2023-09-13T03:10:43.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.130+0000 7fb1fc9ff700 1 --2- 172.21.15.64:0/3735449266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1f8074640 0x7fb1f8072a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:43.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.130+0000 7fb1fc9ff700 1 --2- 172.21.15.64:0/3735449266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1f810cbd0 0x7fb1f810efc0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:43.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.130+0000 7fb1fc9ff700 1 --2- 172.21.15.64:0/3735449266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1f8073060 0x7fb1f80734e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:43.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.130+0000 7fb1fc9ff700 1 -- 172.21.15.64:0/3735449266 >> 172.21.15.64:0/3735449266 conn(0x7fb1f80fbb30 msgr2=0x7fb1f80fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:10:43.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.130+0000 7fb1fc9ff700 1 -- 172.21.15.64:0/3735449266 shutdown_connections 2023-09-13T03:10:43.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.130+0000 7fb1fc9ff700 1 -- 172.21.15.64:0/3735449266 wait complete. 2023-09-13T03:10:43.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.131+0000 7fb1fc9ff700 1 Processor -- start 2023-09-13T03:10:43.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.131+0000 7fb1fc9ff700 1 -- start start 2023-09-13T03:10:43.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.131+0000 7fb1fc9ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1f8073060 0x7fb1f819c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:43.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.131+0000 7fb1fc9ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1f8074640 0x7fb1f819ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:43.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.131+0000 7fb1fc9ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1f810cbd0 0x7fb1f81a3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:43.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.131+0000 7fb1fc9ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f81a44d0 con 0x7fb1f8074640 2023-09-13T03:10:43.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.131+0000 7fb1fc9ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f81a4610 con 0x7fb1f810cbd0 2023-09-13T03:10:43.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.131+0000 7fb1fc9ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f81a47a0 con 0x7fb1f8073060 2023-09-13T03:10:43.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.132+0000 7fb1f5d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1f8074640 0x7fb1f819ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:43.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.132+0000 7fb1f5d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1f8074640 0x7fb1f819ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47468/0 (socket says 172.21.15.64:47468) 2023-09-13T03:10:43.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.132+0000 7fb1f5d9b700 1 -- 172.21.15.64:0/3322003461 learned_addr learned my addr 172.21.15.64:0/3322003461 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:10:43.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.132+0000 7fb1f659c700 1 --2- 172.21.15.64:0/3322003461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1f8073060 0x7fb1f819c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:43.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.132+0000 7fb1f6d9d700 1 --2- 172.21.15.64:0/3322003461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1f810cbd0 0x7fb1f81a3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:43.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.132+0000 7fb1f5d9b700 1 -- 172.21.15.64:0/3322003461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1f8073060 msgr2=0x7fb1f819c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:43.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.132+0000 7fb1f5d9b700 1 --2- 172.21.15.64:0/3322003461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1f8073060 0x7fb1f819c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:43.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.132+0000 7fb1f5d9b700 1 -- 172.21.15.64:0/3322003461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1f810cbd0 msgr2=0x7fb1f81a3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:43.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.132+0000 7fb1f5d9b700 1 --2- 172.21.15.64:0/3322003461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1f810cbd0 0x7fb1f81a3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:43.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.132+0000 7fb1f5d9b700 1 -- 172.21.15.64:0/3322003461 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f0009d00 con 0x7fb1f8074640 2023-09-13T03:10:43.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.132+0000 7fb1f659c700 1 --2- 172.21.15.64:0/3322003461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1f8073060 0x7fb1f819c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:10:43.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.132+0000 7fb1f6d9d700 1 --2- 172.21.15.64:0/3322003461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1f810cbd0 0x7fb1f81a3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:10:43.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.132+0000 7fb1f5d9b700 1 --2- 172.21.15.64:0/3322003461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1f8074640 0x7fb1f819ce90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec00e970 tx=0x7fb1ec00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:10:43.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.133+0000 7fb1e77fe700 1 -- 172.21.15.64:0/3322003461 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec009790 con 0x7fb1f8074640 2023-09-13T03:10:43.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.133+0000 7fb1e77fe700 1 -- 172.21.15.64:0/3322003461 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1ec00c7e0 con 0x7fb1f8074640 2023-09-13T03:10:43.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.133+0000 7fb1e77fe700 1 -- 172.21.15.64:0/3322003461 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec003c70 con 0x7fb1f8074640 2023-09-13T03:10:43.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.133+0000 7fb1fc9ff700 1 -- 172.21.15.64:0/3322003461 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1f81a4a80 con 0x7fb1f8074640 2023-09-13T03:10:43.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.133+0000 7fb1fc9ff700 1 -- 172.21.15.64:0/3322003461 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1f81a4f90 con 0x7fb1f8074640 2023-09-13T03:10:43.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.134+0000 7fb1fc9ff700 1 -- 172.21.15.64:0/3322003461 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1f8102720 con 0x7fb1f8074640 2023-09-13T03:10:43.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.136+0000 7fb1e77fe700 1 -- 172.21.15.64:0/3322003461 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fb1ec00cc50 con 0x7fb1f8074640 2023-09-13T03:10:43.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.138+0000 7fb1e77fe700 1 --2- 172.21.15.64:0/3322003461 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fb1e00a0710 0x7fb1e00a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:43.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.138+0000 7fb1e77fe700 1 -- 172.21.15.64:0/3322003461 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb1ec014070 con 0x7fb1f8074640 2023-09-13T03:10:43.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.139+0000 7fb1f659c700 1 --2- 172.21.15.64:0/3322003461 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fb1e00a0710 0x7fb1e00a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:43.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.139+0000 7fb1f659c700 1 --2- 172.21.15.64:0/3322003461 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fb1e00a0710 0x7fb1e00a2bd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8011490 tx=0x7fb1e8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:10:43.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.143+0000 7fb1e77fe700 1 -- 172.21.15.64:0/3322003461 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb1ec095820 con 0x7fb1f8074640 2023-09-13T03:10:43.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.419+0000 7fb1fc9ff700 1 -- 172.21.15.64:0/3322003461 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1f81123f0 con 0x7fb1f8074640 2023-09-13T03:10:43.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.420+0000 7fb1e77fe700 1 -- 172.21.15.64:0/3322003461 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2910 (secure 0 0 0) 0x7fb1ec003dd0 con 0x7fb1f8074640 2023-09-13T03:10:43.420 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:10:43.421 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2234182985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2234182985},{"type":"v1","addr":"172.21.15.64:6835","nonce":2234182985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24569,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/101906999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":101906999},{"type":"v1","addr":"172.21.15.142:6833","nonce":101906999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1897366227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1897366227},{"type":"v1","addr":"172.21.15.170:6833","nonce":1897366227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-09-13T03:10:43.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.422+0000 7fb1fc9ff700 1 -- 172.21.15.64:0/3322003461 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fb1e00a0710 msgr2=0x7fb1e00a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:43.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.422+0000 7fb1fc9ff700 1 --2- 172.21.15.64:0/3322003461 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fb1e00a0710 0x7fb1e00a2bd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8011490 tx=0x7fb1e8009260 comp rx=0 tx=0).stop 2023-09-13T03:10:43.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.422+0000 7fb1fc9ff700 1 -- 172.21.15.64:0/3322003461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1f8074640 msgr2=0x7fb1f819ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:43.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.422+0000 7fb1fc9ff700 1 --2- 172.21.15.64:0/3322003461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1f8074640 0x7fb1f819ce90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec00e970 tx=0x7fb1ec00ec80 comp rx=0 tx=0).stop 2023-09-13T03:10:43.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.422+0000 7fb1fc9ff700 1 -- 172.21.15.64:0/3322003461 shutdown_connections 2023-09-13T03:10:43.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.422+0000 7fb1fc9ff700 1 --2- 172.21.15.64:0/3322003461 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fb1e00a0710 0x7fb1e00a2bd0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:43.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.422+0000 7fb1fc9ff700 1 --2- 172.21.15.64:0/3322003461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1f810cbd0 0x7fb1f81a3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:43.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.422+0000 7fb1fc9ff700 1 --2- 172.21.15.64:0/3322003461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1f8073060 0x7fb1f819c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:43.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.422+0000 7fb1fc9ff700 1 --2- 172.21.15.64:0/3322003461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1f8074640 0x7fb1f819ce90 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:43.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.422+0000 7fb1fc9ff700 1 -- 172.21.15.64:0/3322003461 >> 172.21.15.64:0/3322003461 conn(0x7fb1f80fbb30 msgr2=0x7fb1f80fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:10:43.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.422+0000 7fb1fc9ff700 1 -- 172.21.15.64:0/3322003461 shutdown_connections 2023-09-13T03:10:43.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:43.423+0000 7fb1fc9ff700 1 -- 172.21.15.64:0/3322003461 wait complete. 2023-09-13T03:10:43.426 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 4 2023-09-13T03:10:43.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:43 smithi064 ceph-mon[97675]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:43.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:43 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/3322003461' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:10:43.669 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 13 03:10:43 smithi064 systemd[1]: Starting Ceph mds.d for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:10:43.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:43 smithi170 ceph-mon[134328]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:43.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:43 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/3322003461' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:10:43.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:43 smithi142 ceph-mon[134082]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:43.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:43 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/3322003461' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:10:44.089 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 13 03:10:43 smithi064 podman[151572]: 2023-09-13T03:10:44.089 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 13 03:10:43 smithi064 bash[151572]: 6490b80c61b3b978a12f65c8f5042afd1695d89d64e9db84ffdda869a697e14d 2023-09-13T03:10:44.090 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 13 03:10:44 smithi064 ceph-mds[151612]: -- 172.21.15.64:0/168513585 <== mon.2 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56427fe531e0 con 0x56427fe28c00 2023-09-13T03:10:44.090 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 13 03:10:44 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-d[151588]: starting mds.d at 2023-09-13T03:10:44.418 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 13 03:10:44 smithi064 systemd[1]: Started Ceph mds.d for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da. 2023-09-13T03:10:45.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:45 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:45.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:45 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:45.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:45 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:45.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:45 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T03:10:45.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:45 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T03:10:45.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:45 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:45.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:45 smithi170 ceph-mon[134328]: Deploying daemon mds.e on smithi142 2023-09-13T03:10:45.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:45 smithi170 ceph-mon[134328]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:45.412 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:45 smithi170 ceph-mon[134328]: mds.? [v2:172.21.15.64:6836/4053841873,v1:172.21.15.64:6837/4053841873] up:boot 2023-09-13T03:10:45.412 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:45 smithi170 ceph-mon[134328]: fsmap 4 up:standby 2023-09-13T03:10:45.412 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:45 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-13T03:10:45.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:45 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:45.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:45 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:45.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:45 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:45.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:45 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T03:10:45.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:45 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T03:10:45.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:45 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:45.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:45 smithi064 ceph-mon[97675]: Deploying daemon mds.e on smithi142 2023-09-13T03:10:45.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:45 smithi064 ceph-mon[97675]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:45.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:45 smithi064 ceph-mon[97675]: mds.? [v2:172.21.15.64:6836/4053841873,v1:172.21.15.64:6837/4053841873] up:boot 2023-09-13T03:10:45.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:45 smithi064 ceph-mon[97675]: fsmap 4 up:standby 2023-09-13T03:10:45.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:45 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-13T03:10:45.442 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:45 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:45.442 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:45 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:45.442 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:45 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:45.443 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:45 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T03:10:45.443 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:45 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T03:10:45.443 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:45 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:45.443 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:45 smithi142 ceph-mon[134082]: Deploying daemon mds.e on smithi142 2023-09-13T03:10:45.444 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:45 smithi142 ceph-mon[134082]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:45.444 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:45 smithi142 ceph-mon[134082]: mds.? [v2:172.21.15.64:6836/4053841873,v1:172.21.15.64:6837/4053841873] up:boot 2023-09-13T03:10:45.444 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:45 smithi142 ceph-mon[134082]: fsmap 4 up:standby 2023-09-13T03:10:45.444 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:45 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-13T03:10:45.445 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T03:10:45.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.664+0000 7f70b11ff700 1 -- 172.21.15.64:0/3731287219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70ac1017f0 msgr2=0x7f70ac101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:45.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.664+0000 7f70b11ff700 1 --2- 172.21.15.64:0/3731287219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70ac1017f0 0x7f70ac101c50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f70a0009920 tx=0x7f70a0009c30 comp rx=0 tx=0).stop 2023-09-13T03:10:45.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.665+0000 7f70b11ff700 1 -- 172.21.15.64:0/3731287219 shutdown_connections 2023-09-13T03:10:45.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.665+0000 7f70b11ff700 1 --2- 172.21.15.64:0/3731287219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70ac100600 0x7f70ac100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:45.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.665+0000 7f70b11ff700 1 --2- 172.21.15.64:0/3731287219 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70ac102190 0x7f70ac108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:45.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.665+0000 7f70b11ff700 1 --2- 172.21.15.64:0/3731287219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70ac1017f0 0x7f70ac101c50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:45.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.665+0000 7f70b11ff700 1 -- 172.21.15.64:0/3731287219 >> 172.21.15.64:0/3731287219 conn(0x7f70ac0fbb30 msgr2=0x7f70ac0fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:10:45.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.665+0000 7f70b11ff700 1 -- 172.21.15.64:0/3731287219 shutdown_connections 2023-09-13T03:10:45.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.665+0000 7f70b11ff700 1 -- 172.21.15.64:0/3731287219 wait complete. 2023-09-13T03:10:45.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.666+0000 7f70b11ff700 1 Processor -- start 2023-09-13T03:10:45.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.666+0000 7f70b11ff700 1 -- start start 2023-09-13T03:10:45.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.666+0000 7f70b11ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70ac100600 0x7f70ac198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:45.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.666+0000 7f70b11ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70ac1017f0 0x7f70ac198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:45.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.666+0000 7f70b11ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70ac102190 0x7f70ac19fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:45.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.666+0000 7f70b11ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ac1a00b0 con 0x7f70ac100600 2023-09-13T03:10:45.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.666+0000 7f70b11ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ac1a01f0 con 0x7f70ac102190 2023-09-13T03:10:45.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.666+0000 7f70b11ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ac1a0380 con 0x7f70ac1017f0 2023-09-13T03:10:45.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.667+0000 7f70aad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70ac100600 0x7f70ac198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:45.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.667+0000 7f70aad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70ac100600 0x7f70ac198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47496/0 (socket says 172.21.15.64:47496) 2023-09-13T03:10:45.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.667+0000 7f70aad9d700 1 -- 172.21.15.64:0/2202174162 learned_addr learned my addr 172.21.15.64:0/2202174162 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:10:45.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.667+0000 7f70aa59c700 1 --2- 172.21.15.64:0/2202174162 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70ac1017f0 0x7f70ac198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:45.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.667+0000 7f70ab59e700 1 --2- 172.21.15.64:0/2202174162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70ac102190 0x7f70ac19fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:45.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.667+0000 7f70aad9d700 1 -- 172.21.15.64:0/2202174162 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70ac1017f0 msgr2=0x7f70ac198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:45.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.667+0000 7f70aad9d700 1 --2- 172.21.15.64:0/2202174162 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70ac1017f0 0x7f70ac198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:45.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.667+0000 7f70aad9d700 1 -- 172.21.15.64:0/2202174162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70ac102190 msgr2=0x7f70ac19fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:45.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.667+0000 7f70aad9d700 1 --2- 172.21.15.64:0/2202174162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70ac102190 0x7f70ac19fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:45.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.667+0000 7f70aad9d700 1 -- 172.21.15.64:0/2202174162 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a0009520 con 0x7f70ac100600 2023-09-13T03:10:45.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.667+0000 7f70aad9d700 1 --2- 172.21.15.64:0/2202174162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70ac100600 0x7f70ac198530 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f709c00e970 tx=0x7f709c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:10:45.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.668+0000 7f7093fff700 1 -- 172.21.15.64:0/2202174162 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c009790 con 0x7f70ac100600 2023-09-13T03:10:45.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.668+0000 7f7093fff700 1 -- 172.21.15.64:0/2202174162 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f709c00c7e0 con 0x7f70ac100600 2023-09-13T03:10:45.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.668+0000 7f70b11ff700 1 -- 172.21.15.64:0/2202174162 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70ac1a0660 con 0x7f70ac100600 2023-09-13T03:10:45.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.668+0000 7f7093fff700 1 -- 172.21.15.64:0/2202174162 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c003c70 con 0x7f70ac100600 2023-09-13T03:10:45.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.668+0000 7f70b11ff700 1 -- 172.21.15.64:0/2202174162 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70ac1a0bc0 con 0x7f70ac100600 2023-09-13T03:10:45.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.669+0000 7f70b11ff700 1 -- 172.21.15.64:0/2202174162 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70ac192c50 con 0x7f70ac100600 2023-09-13T03:10:45.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.671+0000 7f7093fff700 1 -- 172.21.15.64:0/2202174162 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f709c00cc50 con 0x7f70ac100600 2023-09-13T03:10:45.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.672+0000 7f7093fff700 1 --2- 172.21.15.64:0/2202174162 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f70940a0710 0x7f70940a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:45.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.672+0000 7f7093fff700 1 -- 172.21.15.64:0/2202174162 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f709c014070 con 0x7f70ac100600 2023-09-13T03:10:45.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.672+0000 7f70aa59c700 1 --2- 172.21.15.64:0/2202174162 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f70940a0710 0x7f70940a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:45.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.673+0000 7f70aa59c700 1 --2- 172.21.15.64:0/2202174162 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f70940a0710 0x7f70940a2bd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f70a000fa40 tx=0x7f70a0019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:10:45.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.676+0000 7f7093fff700 1 -- 172.21.15.64:0/2202174162 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f709c095830 con 0x7f70ac100600 2023-09-13T03:10:45.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.933+0000 7f70b11ff700 1 -- 172.21.15.64:0/2202174162 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70ac10be60 con 0x7f70ac100600 2023-09-13T03:10:45.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.934+0000 7f7093fff700 1 -- 172.21.15.64:0/2202174162 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3701 (secure 0 0 0) 0x7f709c003dd0 con 0x7f70ac100600 2023-09-13T03:10:45.935 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:10:45.935 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2234182985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2234182985},{"type":"v1","addr":"172.21.15.64:6835","nonce":2234182985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4053841873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4053841873},{"type":"v1","addr":"172.21.15.64:6837","nonce":4053841873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24569,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/101906999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":101906999},{"type":"v1","addr":"172.21.15.142:6833","nonce":101906999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1897366227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1897366227},{"type":"v1","addr":"172.21.15.170:6833","nonce":1897366227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-09-13T03:10:45.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.937+0000 7f70b11ff700 1 -- 172.21.15.64:0/2202174162 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f70940a0710 msgr2=0x7f70940a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:45.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.937+0000 7f70b11ff700 1 --2- 172.21.15.64:0/2202174162 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f70940a0710 0x7f70940a2bd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f70a000fa40 tx=0x7f70a0019040 comp rx=0 tx=0).stop 2023-09-13T03:10:45.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.937+0000 7f70b11ff700 1 -- 172.21.15.64:0/2202174162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70ac100600 msgr2=0x7f70ac198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:45.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.937+0000 7f70b11ff700 1 --2- 172.21.15.64:0/2202174162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70ac100600 0x7f70ac198530 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f709c00e970 tx=0x7f709c00ec80 comp rx=0 tx=0).stop 2023-09-13T03:10:45.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.937+0000 7f70b11ff700 1 -- 172.21.15.64:0/2202174162 shutdown_connections 2023-09-13T03:10:45.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.937+0000 7f70b11ff700 1 --2- 172.21.15.64:0/2202174162 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f70940a0710 0x7f70940a2bd0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:45.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.937+0000 7f70b11ff700 1 --2- 172.21.15.64:0/2202174162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70ac102190 0x7f70ac19fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:45.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.937+0000 7f70b11ff700 1 --2- 172.21.15.64:0/2202174162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70ac100600 0x7f70ac198530 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:45.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.937+0000 7f70b11ff700 1 --2- 172.21.15.64:0/2202174162 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70ac1017f0 0x7f70ac198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:45.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.937+0000 7f70b11ff700 1 -- 172.21.15.64:0/2202174162 >> 172.21.15.64:0/2202174162 conn(0x7f70ac0fbb30 msgr2=0x7f70ac107170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:10:45.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.937+0000 7f70b11ff700 1 -- 172.21.15.64:0/2202174162 shutdown_connections 2023-09-13T03:10:45.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:45.937+0000 7f70b11ff700 1 -- 172.21.15.64:0/2202174162 wait complete. 2023-09-13T03:10:45.940 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 5 2023-09-13T03:10:45.997 INFO:journalctl@ceph.mds.e.smithi142.stdout:Sep 13 03:10:45 smithi142 systemd[1]: Starting Ceph mds.e for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:10:46.430 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:46 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/2202174162' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:10:46.431 INFO:journalctl@ceph.mds.e.smithi142.stdout:Sep 13 03:10:46 smithi142 podman[169965]: 2023-09-13T03:10:46.431 INFO:journalctl@ceph.mds.e.smithi142.stdout:Sep 13 03:10:46 smithi142 ceph-mds[170005]: -- 172.21.15.142:0/1614758390 <== mon.2 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564439cf31e0 con 0x564439cc9000 2023-09-13T03:10:46.431 INFO:journalctl@ceph.mds.e.smithi142.stdout:Sep 13 03:10:46 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-e[169981]: starting mds.e at 2023-09-13T03:10:46.431 INFO:journalctl@ceph.mds.e.smithi142.stdout:Sep 13 03:10:46 smithi142 bash[169965]: cae6724bca997a9ec7ed6c3fc1e48066649a05723645ecc47731b8768398b1a2 2023-09-13T03:10:46.659 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:46 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/2202174162' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:10:46.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:46 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2202174162' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:10:46.745 INFO:journalctl@ceph.mds.e.smithi142.stdout:Sep 13 03:10:46 smithi142 systemd[1]: Started Ceph mds.e for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da. 2023-09-13T03:10:47.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:47 smithi170 ceph-mon[134328]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:47.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:47 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:47.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:47 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:47.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:47 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:47.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:47 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T03:10:47.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:47 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T03:10:47.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:47 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:47.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:47 smithi170 ceph-mon[134328]: Deploying daemon mds.f on smithi170 2023-09-13T03:10:47.662 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:47 smithi170 ceph-mon[134328]: mds.? [v2:172.21.15.142:6834/2167779918,v1:172.21.15.142:6835/2167779918] up:boot 2023-09-13T03:10:47.662 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:47 smithi170 ceph-mon[134328]: fsmap 5 up:standby 2023-09-13T03:10:47.662 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:47 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-13T03:10:47.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:47 smithi142 ceph-mon[134082]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:47.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:47 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:47.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:47 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:47.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:47 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:47.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:47 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T03:10:47.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:47 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T03:10:47.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:47 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:47.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:47 smithi142 ceph-mon[134082]: Deploying daemon mds.f on smithi170 2023-09-13T03:10:47.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:47 smithi142 ceph-mon[134082]: mds.? [v2:172.21.15.142:6834/2167779918,v1:172.21.15.142:6835/2167779918] up:boot 2023-09-13T03:10:47.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:47 smithi142 ceph-mon[134082]: fsmap 5 up:standby 2023-09-13T03:10:47.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:47 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-13T03:10:47.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:47 smithi064 ceph-mon[97675]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:47.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:47 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:47.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:47 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:47.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:47 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:47.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:47 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T03:10:47.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:47 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T03:10:47.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:47 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:47.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:47 smithi064 ceph-mon[97675]: Deploying daemon mds.f on smithi170 2023-09-13T03:10:47.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:47 smithi064 ceph-mon[97675]: mds.? [v2:172.21.15.142:6834/2167779918,v1:172.21.15.142:6835/2167779918] up:boot 2023-09-13T03:10:47.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:47 smithi064 ceph-mon[97675]: fsmap 5 up:standby 2023-09-13T03:10:47.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:47 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-13T03:10:47.956 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T03:10:48.160 INFO:journalctl@ceph.mds.f.smithi170.stdout:Sep 13 03:10:47 smithi170 systemd[1]: Starting Ceph mds.f for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:10:48.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.171+0000 7f235d5ff700 1 -- 172.21.15.64:0/1514198529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2358100600 msgr2=0x7f2358100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:48.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.171+0000 7f235d5ff700 1 --2- 172.21.15.64:0/1514198529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2358100600 0x7f2358100a20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f23480099c0 tx=0x7f2348009cd0 comp rx=0 tx=0).stop 2023-09-13T03:10:48.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.171+0000 7f235d5ff700 1 -- 172.21.15.64:0/1514198529 shutdown_connections 2023-09-13T03:10:48.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.171+0000 7f235d5ff700 1 --2- 172.21.15.64:0/1514198529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2358102190 0x7f2358108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:48.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.171+0000 7f235d5ff700 1 --2- 172.21.15.64:0/1514198529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2358100600 0x7f2358100a20 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:48.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.171+0000 7f235d5ff700 1 --2- 172.21.15.64:0/1514198529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23581017f0 0x7f2358101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:48.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.171+0000 7f235d5ff700 1 -- 172.21.15.64:0/1514198529 >> 172.21.15.64:0/1514198529 conn(0x7f23580fbb30 msgr2=0x7f23580fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:10:48.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.171+0000 7f235d5ff700 1 -- 172.21.15.64:0/1514198529 shutdown_connections 2023-09-13T03:10:48.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.171+0000 7f235d5ff700 1 -- 172.21.15.64:0/1514198529 wait complete. 2023-09-13T03:10:48.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.173+0000 7f235d5ff700 1 Processor -- start 2023-09-13T03:10:48.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.173+0000 7f235d5ff700 1 -- start start 2023-09-13T03:10:48.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.173+0000 7f235d5ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2358100600 0x7f23580743b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:48.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.173+0000 7f235d5ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23581017f0 0x7f23580729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:48.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.173+0000 7f235d5ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2358102190 0x7f2358072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:48.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.173+0000 7f235d5ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23580736d0 con 0x7f2358100600 2023-09-13T03:10:48.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.173+0000 7f235d5ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2358073840 con 0x7f23581017f0 2023-09-13T03:10:48.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.173+0000 7f235d5ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23581a0370 con 0x7f2358102190 2023-09-13T03:10:48.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.174+0000 7f2356ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2358100600 0x7f23580743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:48.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.174+0000 7f23567fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23581017f0 0x7f23580729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:48.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.174+0000 7f2356ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2358100600 0x7f23580743b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48052/0 (socket says 172.21.15.64:48052) 2023-09-13T03:10:48.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.174+0000 7f2356ffd700 1 -- 172.21.15.64:0/3996956761 learned_addr learned my addr 172.21.15.64:0/3996956761 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:10:48.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.174+0000 7f23567fc700 1 --2- 172.21.15.64:0/3996956761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23581017f0 0x7f23580729f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:39704/0 (socket says 172.21.15.64:39704) 2023-09-13T03:10:48.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.174+0000 7f23577fe700 1 --2- 172.21.15.64:0/3996956761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2358102190 0x7f2358072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:48.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.174+0000 7f2356ffd700 1 -- 172.21.15.64:0/3996956761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2358102190 msgr2=0x7f2358072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:48.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.174+0000 7f2356ffd700 1 --2- 172.21.15.64:0/3996956761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2358102190 0x7f2358072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:48.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.174+0000 7f2356ffd700 1 -- 172.21.15.64:0/3996956761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23581017f0 msgr2=0x7f23580729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:48.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.174+0000 7f2356ffd700 1 --2- 172.21.15.64:0/3996956761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23581017f0 0x7f23580729f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:48.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.174+0000 7f2356ffd700 1 -- 172.21.15.64:0/3996956761 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23480095f0 con 0x7f2358100600 2023-09-13T03:10:48.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.174+0000 7f2356ffd700 1 --2- 172.21.15.64:0/3996956761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2358100600 0x7f23580743b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f23480098e0 tx=0x7f234800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:10:48.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.174+0000 7f233ffff700 1 -- 172.21.15.64:0/3996956761 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234801d070 con 0x7f2358100600 2023-09-13T03:10:48.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.175+0000 7f233ffff700 1 -- 172.21.15.64:0/3996956761 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2348005510 con 0x7f2358100600 2023-09-13T03:10:48.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.175+0000 7f235d5ff700 1 -- 172.21.15.64:0/3996956761 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23581a05f0 con 0x7f2358100600 2023-09-13T03:10:48.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.175+0000 7f233ffff700 1 -- 172.21.15.64:0/3996956761 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2348004c00 con 0x7f2358100600 2023-09-13T03:10:48.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.175+0000 7f235d5ff700 1 -- 172.21.15.64:0/3996956761 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23581a0a10 con 0x7f2358100600 2023-09-13T03:10:48.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.176+0000 7f235d5ff700 1 -- 172.21.15.64:0/3996956761 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f235810be70 con 0x7f2358100600 2023-09-13T03:10:48.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.178+0000 7f233ffff700 1 -- 172.21.15.64:0/3996956761 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f234800fb80 con 0x7f2358100600 2023-09-13T03:10:48.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.179+0000 7f233ffff700 1 --2- 172.21.15.64:0/3996956761 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f23400a0690 0x7f23400a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:48.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.179+0000 7f233ffff700 1 -- 172.21.15.64:0/3996956761 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f23480cca30 con 0x7f2358100600 2023-09-13T03:10:48.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.179+0000 7f23567fc700 1 --2- 172.21.15.64:0/3996956761 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f23400a0690 0x7f23400a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:48.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.180+0000 7f23567fc700 1 --2- 172.21.15.64:0/3996956761 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f23400a0690 0x7f23400a2b50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f234c009c60 tx=0x7f234c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:10:48.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.181+0000 7f233ffff700 1 -- 172.21.15.64:0/3996956761 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f23480969f0 con 0x7f2358100600 2023-09-13T03:10:48.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.436+0000 7f235d5ff700 1 -- 172.21.15.64:0/3996956761 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2358107540 con 0x7f2358100600 2023-09-13T03:10:48.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.436+0000 7f233ffff700 1 -- 172.21.15.64:0/3996956761 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4495 (secure 0 0 0) 0x7f234809a010 con 0x7f2358100600 2023-09-13T03:10:48.437 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:10:48.438 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2234182985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2234182985},{"type":"v1","addr":"172.21.15.64:6835","nonce":2234182985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4053841873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4053841873},{"type":"v1","addr":"172.21.15.64:6837","nonce":4053841873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24569,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/101906999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":101906999},{"type":"v1","addr":"172.21.15.142:6833","nonce":101906999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1897366227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1897366227},{"type":"v1","addr":"172.21.15.170:6833","nonce":1897366227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/2167779918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2167779918},{"type":"v1","addr":"172.21.15.142:6835","nonce":2167779918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2023-09-13T03:10:48.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.439+0000 7f235d5ff700 1 -- 172.21.15.64:0/3996956761 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f23400a0690 msgr2=0x7f23400a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:48.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.439+0000 7f235d5ff700 1 --2- 172.21.15.64:0/3996956761 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f23400a0690 0x7f23400a2b50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f234c009c60 tx=0x7f234c009260 comp rx=0 tx=0).stop 2023-09-13T03:10:48.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.439+0000 7f235d5ff700 1 -- 172.21.15.64:0/3996956761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2358100600 msgr2=0x7f23580743b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:48.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.439+0000 7f235d5ff700 1 --2- 172.21.15.64:0/3996956761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2358100600 0x7f23580743b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f23480098e0 tx=0x7f234800f690 comp rx=0 tx=0).stop 2023-09-13T03:10:48.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.439+0000 7f235d5ff700 1 -- 172.21.15.64:0/3996956761 shutdown_connections 2023-09-13T03:10:48.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.439+0000 7f235d5ff700 1 --2- 172.21.15.64:0/3996956761 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f23400a0690 0x7f23400a2b50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:48.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.439+0000 7f235d5ff700 1 --2- 172.21.15.64:0/3996956761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2358102190 0x7f2358072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:48.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.439+0000 7f235d5ff700 1 --2- 172.21.15.64:0/3996956761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2358100600 0x7f23580743b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:48.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.439+0000 7f235d5ff700 1 --2- 172.21.15.64:0/3996956761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23581017f0 0x7f23580729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:48.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.439+0000 7f235d5ff700 1 -- 172.21.15.64:0/3996956761 >> 172.21.15.64:0/3996956761 conn(0x7f23580fbb30 msgr2=0x7f23580fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:10:48.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.439+0000 7f235d5ff700 1 -- 172.21.15.64:0/3996956761 shutdown_connections 2023-09-13T03:10:48.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:48.439+0000 7f235d5ff700 1 -- 172.21.15.64:0/3996956761 wait complete. 2023-09-13T03:10:48.443 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 6 2023-09-13T03:10:48.559 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:48 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/3996956761' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:10:48.560 INFO:journalctl@ceph.mds.f.smithi170.stdout:Sep 13 03:10:48 smithi170 podman[171605]: 2023-09-13T03:10:48.560 INFO:journalctl@ceph.mds.f.smithi170.stdout:Sep 13 03:10:48 smithi170 ceph-mds[171645]: -- 172.21.15.170:0/2008912151 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e8ad0dd1e0 con 0x55e8ad0b3000 2023-09-13T03:10:48.560 INFO:journalctl@ceph.mds.f.smithi170.stdout:Sep 13 03:10:48 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-f[171621]: starting mds.f at 2023-09-13T03:10:48.560 INFO:journalctl@ceph.mds.f.smithi170.stdout:Sep 13 03:10:48 smithi170 bash[171605]: 3f2e6a45d76a601c2e7881e1540e16f3463401a699c668b9694f9c754d72e08e 2023-09-13T03:10:48.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:48 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/3996956761' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:10:48.910 INFO:journalctl@ceph.mds.f.smithi170.stdout:Sep 13 03:10:48 smithi170 systemd[1]: Started Ceph mds.f for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da. 2023-09-13T03:10:48.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:48 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/3996956761' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:10:49.863 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:49 smithi170 ceph-mon[134328]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:49.863 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:49 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:49.863 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:49 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:49.863 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:49 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:49.864 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:49 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T03:10:49.864 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:49 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T03:10:49.864 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:49 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:49.864 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:49 smithi170 ceph-mon[134328]: Deploying daemon mds.c on smithi170 2023-09-13T03:10:49.864 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:49 smithi170 ceph-mon[134328]: mds.? [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] up:boot 2023-09-13T03:10:49.865 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:49 smithi170 ceph-mon[134328]: fsmap 6 up:standby 2023-09-13T03:10:49.865 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:49 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-13T03:10:49.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:49 smithi064 ceph-mon[97675]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:49.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:49 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:49.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:49 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:49.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:49 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:49.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:49 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T03:10:49.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:49 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T03:10:49.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:49 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:49.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:49 smithi064 ceph-mon[97675]: Deploying daemon mds.c on smithi170 2023-09-13T03:10:49.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:49 smithi064 ceph-mon[97675]: mds.? [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] up:boot 2023-09-13T03:10:49.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:49 smithi064 ceph-mon[97675]: fsmap 6 up:standby 2023-09-13T03:10:49.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:49 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-13T03:10:49.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:49 smithi142 ceph-mon[134082]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:49.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:49 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:49.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:49 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:49.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:49 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:49.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:49 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T03:10:49.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:49 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T03:10:49.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:49 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:49.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:49 smithi142 ceph-mon[134082]: Deploying daemon mds.c on smithi170 2023-09-13T03:10:49.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:49 smithi142 ceph-mon[134082]: mds.? [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] up:boot 2023-09-13T03:10:49.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:49 smithi142 ceph-mon[134082]: fsmap 6 up:standby 2023-09-13T03:10:49.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:49 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-13T03:10:50.460 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T03:10:50.493 INFO:journalctl@ceph.mds.c.smithi170.stdout:Sep 13 03:10:50 smithi170 systemd[1]: Starting Ceph mds.c for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:10:50.494 INFO:journalctl@ceph.mds.c.smithi170.stdout:Sep 13 03:10:50 smithi170 podman[171948]: 2023-09-13T03:10:50.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.674+0000 7f5aab5ff700 1 -- 172.21.15.64:0/781601256 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5aa4100600 msgr2=0x7f5aa4100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:50.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.674+0000 7f5aab5ff700 1 --2- 172.21.15.64:0/781601256 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5aa4100600 0x7f5aa4100a20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0009990 tx=0x7f5aa0009ca0 comp rx=0 tx=0).stop 2023-09-13T03:10:50.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.675+0000 7f5aab5ff700 1 -- 172.21.15.64:0/781601256 shutdown_connections 2023-09-13T03:10:50.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.675+0000 7f5aab5ff700 1 --2- 172.21.15.64:0/781601256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5aa4102190 0x7f5aa4108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:50.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.675+0000 7f5aab5ff700 1 --2- 172.21.15.64:0/781601256 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5aa4100600 0x7f5aa4100a20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:50.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.675+0000 7f5aab5ff700 1 --2- 172.21.15.64:0/781601256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5aa41017f0 0x7f5aa4101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:50.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.675+0000 7f5aab5ff700 1 -- 172.21.15.64:0/781601256 >> 172.21.15.64:0/781601256 conn(0x7f5aa40fbb30 msgr2=0x7f5aa40fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:10:50.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.675+0000 7f5aab5ff700 1 -- 172.21.15.64:0/781601256 shutdown_connections 2023-09-13T03:10:50.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.675+0000 7f5aab5ff700 1 -- 172.21.15.64:0/781601256 wait complete. 2023-09-13T03:10:50.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.676+0000 7f5aab5ff700 1 Processor -- start 2023-09-13T03:10:50.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.676+0000 7f5aab5ff700 1 -- start start 2023-09-13T03:10:50.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.676+0000 7f5aab5ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5aa4100600 0x7f5aa4198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:50.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.676+0000 7f5aab5ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5aa41017f0 0x7f5aa4198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:50.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.676+0000 7f5aab5ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5aa4102190 0x7f5aa419fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:50.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.676+0000 7f5aab5ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa41a0110 con 0x7f5aa41017f0 2023-09-13T03:10:50.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.676+0000 7f5aab5ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa41a0250 con 0x7f5aa4100600 2023-09-13T03:10:50.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.676+0000 7f5aab5ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa41a03e0 con 0x7f5aa4102190 2023-09-13T03:10:50.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.676+0000 7f5aa8b9a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5aa41017f0 0x7f5aa4198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:50.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.677+0000 7f5aa8b9a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5aa41017f0 0x7f5aa4198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48068/0 (socket says 172.21.15.64:48068) 2023-09-13T03:10:50.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.677+0000 7f5aa8b9a700 1 -- 172.21.15.64:0/2340533929 learned_addr learned my addr 172.21.15.64:0/2340533929 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:10:50.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.677+0000 7f5aa9b9c700 1 --2- 172.21.15.64:0/2340533929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5aa4102190 0x7f5aa419fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:50.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.677+0000 7f5aa939b700 1 --2- 172.21.15.64:0/2340533929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5aa4100600 0x7f5aa4198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:50.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.677+0000 7f5aa8b9a700 1 -- 172.21.15.64:0/2340533929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5aa4102190 msgr2=0x7f5aa419fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:50.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.677+0000 7f5aa8b9a700 1 --2- 172.21.15.64:0/2340533929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5aa4102190 0x7f5aa419fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:50.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.677+0000 7f5aa8b9a700 1 -- 172.21.15.64:0/2340533929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5aa4100600 msgr2=0x7f5aa4198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:50.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.677+0000 7f5aa8b9a700 1 --2- 172.21.15.64:0/2340533929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5aa4100600 0x7f5aa4198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:50.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.677+0000 7f5aa8b9a700 1 -- 172.21.15.64:0/2340533929 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa00095f0 con 0x7f5aa41017f0 2023-09-13T03:10:50.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.677+0000 7f5aa9b9c700 1 --2- 172.21.15.64:0/2340533929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5aa4102190 0x7f5aa419fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:10:50.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.677+0000 7f5aa8b9a700 1 --2- 172.21.15.64:0/2340533929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5aa41017f0 0x7f5aa4198ad0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5a9400e9a0 tx=0x7f5a9400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:10:50.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.677+0000 7f5a9a7fc700 1 -- 172.21.15.64:0/2340533929 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a94009790 con 0x7f5aa41017f0 2023-09-13T03:10:50.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.677+0000 7f5aab5ff700 1 -- 172.21.15.64:0/2340533929 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aa41a06c0 con 0x7f5aa41017f0 2023-09-13T03:10:50.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.677+0000 7f5a9a7fc700 1 -- 172.21.15.64:0/2340533929 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a94010430 con 0x7f5aa41017f0 2023-09-13T03:10:50.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.677+0000 7f5a9a7fc700 1 -- 172.21.15.64:0/2340533929 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a94018680 con 0x7f5aa41017f0 2023-09-13T03:10:50.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.678+0000 7f5aab5ff700 1 -- 172.21.15.64:0/2340533929 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aa41a0c20 con 0x7f5aa41017f0 2023-09-13T03:10:50.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.679+0000 7f5aab5ff700 1 -- 172.21.15.64:0/2340533929 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aa4192c60 con 0x7f5aa41017f0 2023-09-13T03:10:50.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.680+0000 7f5a9a7fc700 1 -- 172.21.15.64:0/2340533929 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f5a940105a0 con 0x7f5aa41017f0 2023-09-13T03:10:50.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.682+0000 7f5a9a7fc700 1 --2- 172.21.15.64:0/2340533929 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f5a900a0710 0x7f5a900a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:50.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.682+0000 7f5a9a7fc700 1 -- 172.21.15.64:0/2340533929 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5a94014070 con 0x7f5aa41017f0 2023-09-13T03:10:50.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.682+0000 7f5aa939b700 1 --2- 172.21.15.64:0/2340533929 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f5a900a0710 0x7f5a900a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:50.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.683+0000 7f5aa939b700 1 --2- 172.21.15.64:0/2340533929 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f5a900a0710 0x7f5a900a2bd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5aa000b2e0 tx=0x7f5aa0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:10:50.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.686+0000 7f5a9a7fc700 1 -- 172.21.15.64:0/2340533929 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5a94051e10 con 0x7f5aa41017f0 2023-09-13T03:10:50.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.951+0000 7f5aab5ff700 1 -- 172.21.15.64:0/2340533929 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5aa410be70 con 0x7f5aa41017f0 2023-09-13T03:10:50.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.951+0000 7f5a9a7fc700 1 -- 172.21.15.64:0/2340533929 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5289 (secure 0 0 0) 0x7f5a94095820 con 0x7f5aa41017f0 2023-09-13T03:10:50.953 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:10:50.953 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2234182985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2234182985},{"type":"v1","addr":"172.21.15.64:6835","nonce":2234182985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4053841873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4053841873},{"type":"v1","addr":"172.21.15.64:6837","nonce":4053841873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24569,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/101906999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":101906999},{"type":"v1","addr":"172.21.15.142:6833","nonce":101906999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1897366227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1897366227},{"type":"v1","addr":"172.21.15.170:6833","nonce":1897366227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/2167779918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2167779918},{"type":"v1","addr":"172.21.15.142:6835","nonce":2167779918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24622,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1493371121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1493371121},{"type":"v1","addr":"172.21.15.170:6835","nonce":1493371121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2023-09-13T03:10:50.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.953+0000 7f5aab5ff700 1 -- 172.21.15.64:0/2340533929 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f5a900a0710 msgr2=0x7f5a900a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:50.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.953+0000 7f5aab5ff700 1 --2- 172.21.15.64:0/2340533929 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f5a900a0710 0x7f5a900a2bd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5aa000b2e0 tx=0x7f5aa0005fb0 comp rx=0 tx=0).stop 2023-09-13T03:10:50.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.953+0000 7f5aab5ff700 1 -- 172.21.15.64:0/2340533929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5aa41017f0 msgr2=0x7f5aa4198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:50.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.953+0000 7f5aab5ff700 1 --2- 172.21.15.64:0/2340533929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5aa41017f0 0x7f5aa4198ad0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5a9400e9a0 tx=0x7f5a9400ecb0 comp rx=0 tx=0).stop 2023-09-13T03:10:50.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.954+0000 7f5aab5ff700 1 -- 172.21.15.64:0/2340533929 shutdown_connections 2023-09-13T03:10:50.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.954+0000 7f5aab5ff700 1 --2- 172.21.15.64:0/2340533929 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f5a900a0710 0x7f5a900a2bd0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:50.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.954+0000 7f5aab5ff700 1 --2- 172.21.15.64:0/2340533929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5aa4100600 0x7f5aa4198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:50.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.954+0000 7f5aab5ff700 1 --2- 172.21.15.64:0/2340533929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5aa4102190 0x7f5aa419fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:50.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.954+0000 7f5aab5ff700 1 --2- 172.21.15.64:0/2340533929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5aa41017f0 0x7f5aa4198ad0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:50.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.954+0000 7f5aab5ff700 1 -- 172.21.15.64:0/2340533929 >> 172.21.15.64:0/2340533929 conn(0x7f5aa40fbb30 msgr2=0x7f5aa4107120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:10:50.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.954+0000 7f5aab5ff700 1 -- 172.21.15.64:0/2340533929 shutdown_connections 2023-09-13T03:10:50.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:50.954+0000 7f5aab5ff700 1 -- 172.21.15.64:0/2340533929 wait complete. 2023-09-13T03:10:50.958 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 7 2023-09-13T03:10:51.160 INFO:journalctl@ceph.mds.c.smithi170.stdout:Sep 13 03:10:50 smithi170 ceph-mds[171988]: -- 172.21.15.170:0/2813557328 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b954bdd1e0 con 0x55b954bb3000 2023-09-13T03:10:51.160 INFO:journalctl@ceph.mds.c.smithi170.stdout:Sep 13 03:10:50 smithi170 bash[171948]: ca0afca0c00b077a88a88f371b07b8c023e9e603db321782f27f377c39a4d69b 2023-09-13T03:10:51.160 INFO:journalctl@ceph.mds.c.smithi170.stdout:Sep 13 03:10:50 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-c[171964]: starting mds.c at 2023-09-13T03:10:51.160 INFO:journalctl@ceph.mds.c.smithi170.stdout:Sep 13 03:10:50 smithi170 systemd[1]: Started Ceph mds.c for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da. 2023-09-13T03:10:51.793 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:51 smithi142 ceph-mon[134082]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:51.793 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:51 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:51.793 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:51 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:51.794 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:51 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:51.794 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:51 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T03:10:51.794 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:51 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T03:10:51.794 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:51 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:51.794 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:51 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/2340533929' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:10:51.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:51 smithi170 ceph-mon[134328]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:51.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:51 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:51.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:51 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:51.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:51 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:51.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:51 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T03:10:51.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:51 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T03:10:51.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:51 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:51.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:51 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/2340533929' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:10:51.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:51 smithi064 ceph-mon[97675]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:51.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:51 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:51.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:51 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:51.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:51 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:51.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:51 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T03:10:51.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:51 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T03:10:51.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:51 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:51.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:51 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2340533929' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:10:52.496 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:10:52 smithi142 systemd[1]: Starting Ceph mds.h for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:10:52.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:52 smithi170 ceph-mon[134328]: Deploying daemon mds.h on smithi142 2023-09-13T03:10:52.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:52 smithi170 ceph-mon[134328]: mds.? [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] up:boot 2023-09-13T03:10:52.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:52 smithi170 ceph-mon[134328]: fsmap 7 up:standby 2023-09-13T03:10:52.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:52 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-13T03:10:52.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:52 smithi064 ceph-mon[97675]: Deploying daemon mds.h on smithi142 2023-09-13T03:10:52.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:52 smithi064 ceph-mon[97675]: mds.? [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] up:boot 2023-09-13T03:10:52.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:52 smithi064 ceph-mon[97675]: fsmap 7 up:standby 2023-09-13T03:10:52.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:52 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-13T03:10:52.957 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:10:52 smithi142 podman[170307]: 2023-09-13T03:10:52.958 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:52 smithi142 ceph-mon[134082]: Deploying daemon mds.h on smithi142 2023-09-13T03:10:52.958 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:52 smithi142 ceph-mon[134082]: mds.? [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] up:boot 2023-09-13T03:10:52.958 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:52 smithi142 ceph-mon[134082]: fsmap 7 up:standby 2023-09-13T03:10:52.959 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:52 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-13T03:10:52.973 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T03:10:53.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.191+0000 7f9534bff700 1 -- 172.21.15.64:0/3855954952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9530073060 msgr2=0x7f95300734e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:53.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.191+0000 7f9534bff700 1 --2- 172.21.15.64:0/3855954952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9530073060 0x7f95300734e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9524009990 tx=0x7f9524009ca0 comp rx=0 tx=0).stop 2023-09-13T03:10:53.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.191+0000 7f9534bff700 1 -- 172.21.15.64:0/3855954952 shutdown_connections 2023-09-13T03:10:53.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.191+0000 7f9534bff700 1 --2- 172.21.15.64:0/3855954952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f953010cbe0 0x7f953010efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:53.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.191+0000 7f9534bff700 1 --2- 172.21.15.64:0/3855954952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9530074640 0x7f9530072a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:53.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.191+0000 7f9534bff700 1 --2- 172.21.15.64:0/3855954952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9530073060 0x7f95300734e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:53.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.191+0000 7f9534bff700 1 -- 172.21.15.64:0/3855954952 >> 172.21.15.64:0/3855954952 conn(0x7f95300fbb40 msgr2=0x7f95300fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:10:53.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.191+0000 7f9534bff700 1 -- 172.21.15.64:0/3855954952 shutdown_connections 2023-09-13T03:10:53.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.191+0000 7f9534bff700 1 -- 172.21.15.64:0/3855954952 wait complete. 2023-09-13T03:10:53.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.192+0000 7f9534bff700 1 Processor -- start 2023-09-13T03:10:53.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.192+0000 7f9534bff700 1 -- start start 2023-09-13T03:10:53.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.192+0000 7f9534bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9530073060 0x7f953019c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:53.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.192+0000 7f9534bff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9530074640 0x7f953019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:53.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.192+0000 7f9534bff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f953010cbe0 0x7f95301a3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:53.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.192+0000 7f9534bff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95301a4410 con 0x7f9530074640 2023-09-13T03:10:53.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.192+0000 7f9534bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95301a4550 con 0x7f9530073060 2023-09-13T03:10:53.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.192+0000 7f9534bff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95301a46e0 con 0x7f953010cbe0 2023-09-13T03:10:53.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.193+0000 7f952dd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9530074640 0x7f953019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:53.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.193+0000 7f952e59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9530073060 0x7f953019c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:53.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.193+0000 7f952dd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9530074640 0x7f953019cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48088/0 (socket says 172.21.15.64:48088) 2023-09-13T03:10:53.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.193+0000 7f952dd9b700 1 -- 172.21.15.64:0/3979443094 learned_addr learned my addr 172.21.15.64:0/3979443094 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:10:53.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.193+0000 7f952ed9d700 1 --2- 172.21.15.64:0/3979443094 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f953010cbe0 0x7f95301a3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:53.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.193+0000 7f952e59c700 1 -- 172.21.15.64:0/3979443094 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f953010cbe0 msgr2=0x7f95301a3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:53.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.193+0000 7f952e59c700 1 --2- 172.21.15.64:0/3979443094 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f953010cbe0 0x7f95301a3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:53.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.193+0000 7f952e59c700 1 -- 172.21.15.64:0/3979443094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9530074640 msgr2=0x7f953019cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:53.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.193+0000 7f952e59c700 1 --2- 172.21.15.64:0/3979443094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9530074640 0x7f953019cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:53.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.193+0000 7f952e59c700 1 -- 172.21.15.64:0/3979443094 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95240095f0 con 0x7f9530073060 2023-09-13T03:10:53.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.193+0000 7f952e59c700 1 --2- 172.21.15.64:0/3979443094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9530073060 0x7f953019c890 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f952000e9f0 tx=0x7f952000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:10:53.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.194+0000 7f951f7fe700 1 -- 172.21.15.64:0/3979443094 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952000c9c0 con 0x7f9530073060 2023-09-13T03:10:53.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.194+0000 7f951f7fe700 1 -- 172.21.15.64:0/3979443094 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9520010430 con 0x7f9530073060 2023-09-13T03:10:53.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.194+0000 7f951f7fe700 1 -- 172.21.15.64:0/3979443094 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9520018680 con 0x7f9530073060 2023-09-13T03:10:53.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.194+0000 7f9534bff700 1 -- 172.21.15.64:0/3979443094 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95301a49c0 con 0x7f9530073060 2023-09-13T03:10:53.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.194+0000 7f9534bff700 1 -- 172.21.15.64:0/3979443094 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95301a4e80 con 0x7f9530073060 2023-09-13T03:10:53.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.195+0000 7f9534bff700 1 -- 172.21.15.64:0/3979443094 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9530102730 con 0x7f9530073060 2023-09-13T03:10:53.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.197+0000 7f951f7fe700 1 -- 172.21.15.64:0/3979443094 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f9520010860 con 0x7f9530073060 2023-09-13T03:10:53.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.198+0000 7f951f7fe700 1 --2- 172.21.15.64:0/3979443094 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f95180a0710 0x7f95180a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:53.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.198+0000 7f951f7fe700 1 -- 172.21.15.64:0/3979443094 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9520014070 con 0x7f9530073060 2023-09-13T03:10:53.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.198+0000 7f952dd9b700 1 --2- 172.21.15.64:0/3979443094 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f95180a0710 0x7f95180a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:53.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.199+0000 7f952dd9b700 1 --2- 172.21.15.64:0/3979443094 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f95180a0710 0x7f95180a2bd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f952400b2e0 tx=0x7f9524005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:10:53.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.202+0000 7f951f7fe700 1 -- 172.21.15.64:0/3979443094 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f9520051e10 con 0x7f9530073060 2023-09-13T03:10:53.245 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:10:52 smithi142 ceph-mds[170348]: -- 172.21.15.142:0/1153646325 <== mon.2 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55845d1171e0 con 0x55845d0ecc00 2023-09-13T03:10:53.245 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:10:52 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-h[170323]: starting mds.h at 2023-09-13T03:10:53.246 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:10:52 smithi142 bash[170307]: fc1fa908a50fe8a0d968a3475b484b7ea65e96cbedd7f2cc912c6790f463fe90 2023-09-13T03:10:53.246 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:10:53 smithi142 systemd[1]: Started Ceph mds.h for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da. 2023-09-13T03:10:53.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.457+0000 7f9534bff700 1 -- 172.21.15.64:0/3979443094 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f953002cee0 con 0x7f9530073060 2023-09-13T03:10:53.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.458+0000 7f951f7fe700 1 -- 172.21.15.64:0/3979443094 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6083 (secure 0 0 0) 0x7f95200957d0 con 0x7f9530073060 2023-09-13T03:10:53.459 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:10:53.459 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2234182985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2234182985},{"type":"v1","addr":"172.21.15.64:6835","nonce":2234182985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4053841873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4053841873},{"type":"v1","addr":"172.21.15.64:6837","nonce":4053841873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24569,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/101906999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":101906999},{"type":"v1","addr":"172.21.15.142:6833","nonce":101906999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1897366227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1897366227},{"type":"v1","addr":"172.21.15.170:6833","nonce":1897366227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/2167779918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2167779918},{"type":"v1","addr":"172.21.15.142:6835","nonce":2167779918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24622,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1493371121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1493371121},{"type":"v1","addr":"172.21.15.170:6835","nonce":1493371121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24634,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/1726587275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":1726587275},{"type":"v1","addr":"172.21.15.170:6837","nonce":1726587275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2023-09-13T03:10:53.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.461+0000 7f9534bff700 1 -- 172.21.15.64:0/3979443094 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f95180a0710 msgr2=0x7f95180a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:53.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.461+0000 7f9534bff700 1 --2- 172.21.15.64:0/3979443094 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f95180a0710 0x7f95180a2bd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f952400b2e0 tx=0x7f9524005fb0 comp rx=0 tx=0).stop 2023-09-13T03:10:53.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.461+0000 7f9534bff700 1 -- 172.21.15.64:0/3979443094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9530073060 msgr2=0x7f953019c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:53.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.461+0000 7f9534bff700 1 --2- 172.21.15.64:0/3979443094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9530073060 0x7f953019c890 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f952000e9f0 tx=0x7f952000ed00 comp rx=0 tx=0).stop 2023-09-13T03:10:53.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.461+0000 7f9534bff700 1 -- 172.21.15.64:0/3979443094 shutdown_connections 2023-09-13T03:10:53.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.461+0000 7f9534bff700 1 --2- 172.21.15.64:0/3979443094 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f95180a0710 0x7f95180a2bd0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:53.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.461+0000 7f9534bff700 1 --2- 172.21.15.64:0/3979443094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9530073060 0x7f953019c890 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:53.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.461+0000 7f9534bff700 1 --2- 172.21.15.64:0/3979443094 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f953010cbe0 0x7f95301a3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:53.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.461+0000 7f9534bff700 1 --2- 172.21.15.64:0/3979443094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9530074640 0x7f953019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:53.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.461+0000 7f9534bff700 1 -- 172.21.15.64:0/3979443094 >> 172.21.15.64:0/3979443094 conn(0x7f95300fbb40 msgr2=0x7f95300fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:10:53.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.461+0000 7f9534bff700 1 -- 172.21.15.64:0/3979443094 shutdown_connections 2023-09-13T03:10:53.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:53.461+0000 7f9534bff700 1 -- 172.21.15.64:0/3979443094 wait complete. 2023-09-13T03:10:53.464 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 8 2023-09-13T03:10:53.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:53 smithi170 ceph-mon[134328]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:53.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:53 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:53.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:53 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:53.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:53 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:53.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:53 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T03:10:53.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:53 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T03:10:53.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:53 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:53.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:53 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/3979443094' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:10:53.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:53 smithi064 ceph-mon[97675]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:53.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:53 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:53.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:53 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:53.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:53 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:53.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:53 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T03:10:53.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:53 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T03:10:53.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:53 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:53.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:53 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/3979443094' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:10:53.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:53 smithi142 ceph-mon[134082]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:53.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:53 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:53.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:53 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:53.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:53 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:53.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:53 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T03:10:53.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:53 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T03:10:53.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:53 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:53.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:53 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/3979443094' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:10:54.910 INFO:journalctl@ceph.mds.i.smithi170.stdout:Sep 13 03:10:54 smithi170 systemd[1]: Starting Ceph mds.i for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:10:54.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:54 smithi170 ceph-mon[134328]: Deploying daemon mds.i on smithi170 2023-09-13T03:10:54.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:54 smithi170 ceph-mon[134328]: mds.? [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] up:boot 2023-09-13T03:10:54.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:54 smithi170 ceph-mon[134328]: fsmap 8 up:standby 2023-09-13T03:10:54.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:54 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-13T03:10:54.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:54 smithi064 ceph-mon[97675]: Deploying daemon mds.i on smithi170 2023-09-13T03:10:54.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:54 smithi064 ceph-mon[97675]: mds.? [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] up:boot 2023-09-13T03:10:54.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:54 smithi064 ceph-mon[97675]: fsmap 8 up:standby 2023-09-13T03:10:54.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:54 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-13T03:10:54.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:54 smithi142 ceph-mon[134082]: Deploying daemon mds.i on smithi170 2023-09-13T03:10:54.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:54 smithi142 ceph-mon[134082]: mds.? [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] up:boot 2023-09-13T03:10:54.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:54 smithi142 ceph-mon[134082]: fsmap 8 up:standby 2023-09-13T03:10:54.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:54 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-13T03:10:55.321 INFO:journalctl@ceph.mds.i.smithi170.stdout:Sep 13 03:10:55 smithi170 podman[172285]: 2023-09-13T03:10:55.481 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T03:10:55.632 INFO:journalctl@ceph.mds.i.smithi170.stdout:Sep 13 03:10:55 smithi170 ceph-mds[172326]: -- 172.21.15.170:0/1966320748 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5595052ed1e0 con 0x5595052c2800 2023-09-13T03:10:55.632 INFO:journalctl@ceph.mds.i.smithi170.stdout:Sep 13 03:10:55 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-i[172301]: starting mds.i at 2023-09-13T03:10:55.632 INFO:journalctl@ceph.mds.i.smithi170.stdout:Sep 13 03:10:55 smithi170 bash[172285]: 94b86140e34be055b49d1607d437d913b9ce36f9567909ae27ec6e7d885e9b8d 2023-09-13T03:10:55.633 INFO:journalctl@ceph.mds.i.smithi170.stdout:Sep 13 03:10:55 smithi170 systemd[1]: Started Ceph mds.i for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da. 2023-09-13T03:10:55.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.703+0000 7fb96c3ff700 1 -- 172.21.15.64:0/1238224840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9641017f0 msgr2=0x7fb964101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:55.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.703+0000 7fb96c3ff700 1 --2- 172.21.15.64:0/1238224840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9641017f0 0x7fb964101c50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb9540099c0 tx=0x7fb954009cd0 comp rx=0 tx=0).stop 2023-09-13T03:10:55.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.703+0000 7fb96c3ff700 1 -- 172.21.15.64:0/1238224840 shutdown_connections 2023-09-13T03:10:55.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.703+0000 7fb96c3ff700 1 --2- 172.21.15.64:0/1238224840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb964102190 0x7fb964108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:55.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.703+0000 7fb96c3ff700 1 --2- 172.21.15.64:0/1238224840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb964100600 0x7fb964100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:55.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.703+0000 7fb96c3ff700 1 --2- 172.21.15.64:0/1238224840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9641017f0 0x7fb964101c50 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:55.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.703+0000 7fb96c3ff700 1 -- 172.21.15.64:0/1238224840 >> 172.21.15.64:0/1238224840 conn(0x7fb9640fbb90 msgr2=0x7fb9640fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:10:55.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.703+0000 7fb96c3ff700 1 -- 172.21.15.64:0/1238224840 shutdown_connections 2023-09-13T03:10:55.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.703+0000 7fb96c3ff700 1 -- 172.21.15.64:0/1238224840 wait complete. 2023-09-13T03:10:55.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.704+0000 7fb96c3ff700 1 Processor -- start 2023-09-13T03:10:55.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.704+0000 7fb96c3ff700 1 -- start start 2023-09-13T03:10:55.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.704+0000 7fb96c3ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb964100600 0x7fb9640743b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:55.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.704+0000 7fb96c3ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9641017f0 0x7fb9640729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:55.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.704+0000 7fb96c3ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb964102190 0x7fb964072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:55.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.705+0000 7fb96c3ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9640736d0 con 0x7fb9641017f0 2023-09-13T03:10:55.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.705+0000 7fb96c3ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb964073840 con 0x7fb964102190 2023-09-13T03:10:55.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.705+0000 7fb96c3ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9641a0320 con 0x7fb964100600 2023-09-13T03:10:55.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.705+0000 7fb96999a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9641017f0 0x7fb9640729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:55.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.705+0000 7fb96a19b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb964100600 0x7fb9640743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:55.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.705+0000 7fb96999a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9641017f0 0x7fb9640729f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48102/0 (socket says 172.21.15.64:48102) 2023-09-13T03:10:55.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.705+0000 7fb96999a700 1 -- 172.21.15.64:0/3377035259 learned_addr learned my addr 172.21.15.64:0/3377035259 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:10:55.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.705+0000 7fb96a99c700 1 --2- 172.21.15.64:0/3377035259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb964102190 0x7fb964072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:55.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.705+0000 7fb96999a700 1 -- 172.21.15.64:0/3377035259 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb964100600 msgr2=0x7fb9640743b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:55.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.705+0000 7fb96999a700 1 --2- 172.21.15.64:0/3377035259 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb964100600 0x7fb9640743b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:55.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.705+0000 7fb96999a700 1 -- 172.21.15.64:0/3377035259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb964102190 msgr2=0x7fb964072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:55.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.705+0000 7fb96999a700 1 --2- 172.21.15.64:0/3377035259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb964102190 0x7fb964072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:55.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.705+0000 7fb96999a700 1 -- 172.21.15.64:0/3377035259 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9540095f0 con 0x7fb9641017f0 2023-09-13T03:10:55.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.705+0000 7fb96a19b700 1 --2- 172.21.15.64:0/3377035259 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb964100600 0x7fb9640743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:10:55.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.705+0000 7fb96999a700 1 --2- 172.21.15.64:0/3377035259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9641017f0 0x7fb9640729f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb954009990 tx=0x7fb95400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:10:55.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.706+0000 7fb95b7fe700 1 -- 172.21.15.64:0/3377035259 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95401d070 con 0x7fb9641017f0 2023-09-13T03:10:55.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.706+0000 7fb95b7fe700 1 -- 172.21.15.64:0/3377035259 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb954005030 con 0x7fb9641017f0 2023-09-13T03:10:55.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.706+0000 7fb96c3ff700 1 -- 172.21.15.64:0/3377035259 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9641a05a0 con 0x7fb9641017f0 2023-09-13T03:10:55.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.706+0000 7fb95b7fe700 1 -- 172.21.15.64:0/3377035259 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb954004a70 con 0x7fb9641017f0 2023-09-13T03:10:55.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.706+0000 7fb96c3ff700 1 -- 172.21.15.64:0/3377035259 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9641a0a20 con 0x7fb9641017f0 2023-09-13T03:10:55.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.707+0000 7fb96c3ff700 1 -- 172.21.15.64:0/3377035259 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb964107b40 con 0x7fb9641017f0 2023-09-13T03:10:55.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.709+0000 7fb95b7fe700 1 -- 172.21.15.64:0/3377035259 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fb954005410 con 0x7fb9641017f0 2023-09-13T03:10:55.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.711+0000 7fb95b7fe700 1 --2- 172.21.15.64:0/3377035259 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fb9500a0710 0x7fb9500a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:55.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.711+0000 7fb95b7fe700 1 -- 172.21.15.64:0/3377035259 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb9540cc480 con 0x7fb9641017f0 2023-09-13T03:10:55.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.711+0000 7fb96a19b700 1 --2- 172.21.15.64:0/3377035259 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fb9500a0710 0x7fb9500a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:55.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.712+0000 7fb96a19b700 1 --2- 172.21.15.64:0/3377035259 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fb9500a0710 0x7fb9500a2bd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb964101650 tx=0x7fb960009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:10:55.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.716+0000 7fb95b7fe700 1 -- 172.21.15.64:0/3377035259 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb954096440 con 0x7fb9641017f0 2023-09-13T03:10:55.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:55 smithi170 ceph-mon[134328]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:55.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:55 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:55.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:55 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:55.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:55 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:55.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:55 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T03:10:55.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:55 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T03:10:55.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:55 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:55.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:55 smithi064 ceph-mon[97675]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:55.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:55 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:55.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:55 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:55.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:55 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:55.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:55 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T03:10:55.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:55 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T03:10:55.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:55 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:55.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.975+0000 7fb96c3ff700 1 -- 172.21.15.64:0/3377035259 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb96410be60 con 0x7fb9641017f0 2023-09-13T03:10:55.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.975+0000 7fb95b7fe700 1 -- 172.21.15.64:0/3377035259 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6874 (secure 0 0 0) 0x7fb954099a60 con 0x7fb9641017f0 2023-09-13T03:10:55.976 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:10:55.976 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2234182985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2234182985},{"type":"v1","addr":"172.21.15.64:6835","nonce":2234182985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4053841873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4053841873},{"type":"v1","addr":"172.21.15.64:6837","nonce":4053841873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24569,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/101906999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":101906999},{"type":"v1","addr":"172.21.15.142:6833","nonce":101906999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1897366227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1897366227},{"type":"v1","addr":"172.21.15.170:6833","nonce":1897366227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/2167779918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2167779918},{"type":"v1","addr":"172.21.15.142:6835","nonce":2167779918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24622,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1493371121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1493371121},{"type":"v1","addr":"172.21.15.170:6835","nonce":1493371121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24634,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/1726587275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":1726587275},{"type":"v1","addr":"172.21.15.170:6837","nonce":1726587275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/741173268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":741173268},{"type":"v1","addr":"172.21.15.142:6837","nonce":741173268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2023-09-13T03:10:55.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.977+0000 7fb96c3ff700 1 -- 172.21.15.64:0/3377035259 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fb9500a0710 msgr2=0x7fb9500a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:55.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.977+0000 7fb96c3ff700 1 --2- 172.21.15.64:0/3377035259 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fb9500a0710 0x7fb9500a2bd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb964101650 tx=0x7fb960009260 comp rx=0 tx=0).stop 2023-09-13T03:10:55.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.977+0000 7fb96c3ff700 1 -- 172.21.15.64:0/3377035259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9641017f0 msgr2=0x7fb9640729f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:55.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.977+0000 7fb96c3ff700 1 --2- 172.21.15.64:0/3377035259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9641017f0 0x7fb9640729f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb954009990 tx=0x7fb95400f690 comp rx=0 tx=0).stop 2023-09-13T03:10:55.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.978+0000 7fb96c3ff700 1 -- 172.21.15.64:0/3377035259 shutdown_connections 2023-09-13T03:10:55.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.978+0000 7fb96c3ff700 1 --2- 172.21.15.64:0/3377035259 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fb9500a0710 0x7fb9500a2bd0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:55.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.978+0000 7fb96c3ff700 1 --2- 172.21.15.64:0/3377035259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb964102190 0x7fb964072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:55.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.978+0000 7fb96c3ff700 1 --2- 172.21.15.64:0/3377035259 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb964100600 0x7fb9640743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:55.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.978+0000 7fb96c3ff700 1 --2- 172.21.15.64:0/3377035259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9641017f0 0x7fb9640729f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:55.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.978+0000 7fb96c3ff700 1 -- 172.21.15.64:0/3377035259 >> 172.21.15.64:0/3377035259 conn(0x7fb9640fbb90 msgr2=0x7fb964104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:10:55.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.978+0000 7fb96c3ff700 1 -- 172.21.15.64:0/3377035259 shutdown_connections 2023-09-13T03:10:55.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:55.978+0000 7fb96c3ff700 1 -- 172.21.15.64:0/3377035259 wait complete. 2023-09-13T03:10:55.981 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 9 2023-09-13T03:10:55.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:55 smithi142 ceph-mon[134082]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:55.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:55 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:55.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:55 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:55.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:55 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:55.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:55 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T03:10:55.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:55 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T03:10:55.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:55 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:56.730 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:56 smithi064 ceph-mon[97675]: Deploying daemon mds.g on smithi064 2023-09-13T03:10:56.731 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:56 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/3377035259' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:10:56.731 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:56 smithi064 ceph-mon[97675]: mds.? [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] up:boot 2023-09-13T03:10:56.731 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:56 smithi064 ceph-mon[97675]: fsmap 9 up:standby 2023-09-13T03:10:56.731 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:56 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-13T03:10:56.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:56 smithi170 ceph-mon[134328]: Deploying daemon mds.g on smithi064 2023-09-13T03:10:56.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:56 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/3377035259' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:10:56.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:56 smithi170 ceph-mon[134328]: mds.? [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] up:boot 2023-09-13T03:10:56.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:56 smithi170 ceph-mon[134328]: fsmap 9 up:standby 2023-09-13T03:10:56.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:56 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-13T03:10:56.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:56 smithi142 ceph-mon[134082]: Deploying daemon mds.g on smithi064 2023-09-13T03:10:56.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:56 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/3377035259' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:10:56.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:56 smithi142 ceph-mon[134082]: mds.? [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] up:boot 2023-09-13T03:10:56.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:56 smithi142 ceph-mon[134082]: fsmap 9 up:standby 2023-09-13T03:10:56.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:56 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-13T03:10:57.659 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:57 smithi064 ceph-mon[97675]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:57.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:57 smithi170 ceph-mon[134328]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:57.918 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 13 03:10:57 smithi064 systemd[1]: Starting Ceph mds.g for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:10:57.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:57 smithi142 ceph-mon[134082]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:57.998 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T03:10:58.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.211+0000 7f58483ff700 1 -- 172.21.15.64:0/3253667596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5840100610 msgr2=0x7f5840100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:58.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.211+0000 7f58483ff700 1 --2- 172.21.15.64:0/3253667596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5840100610 0x7f5840100a30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f583c0099c0 tx=0x7f583c009cd0 comp rx=0 tx=0).stop 2023-09-13T03:10:58.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.212+0000 7f58483ff700 1 -- 172.21.15.64:0/3253667596 shutdown_connections 2023-09-13T03:10:58.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.212+0000 7f58483ff700 1 --2- 172.21.15.64:0/3253667596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5840100610 0x7f5840100a30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:58.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.212+0000 7f58483ff700 1 --2- 172.21.15.64:0/3253667596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58401021a0 0x7f5840108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:58.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.212+0000 7f58483ff700 1 --2- 172.21.15.64:0/3253667596 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5840101800 0x7f5840101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:58.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.212+0000 7f58483ff700 1 -- 172.21.15.64:0/3253667596 >> 172.21.15.64:0/3253667596 conn(0x7f58400fbb40 msgr2=0x7f58400fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:10:58.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.212+0000 7f58483ff700 1 -- 172.21.15.64:0/3253667596 shutdown_connections 2023-09-13T03:10:58.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.212+0000 7f58483ff700 1 -- 172.21.15.64:0/3253667596 wait complete. 2023-09-13T03:10:58.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.213+0000 7f58483ff700 1 Processor -- start 2023-09-13T03:10:58.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.213+0000 7f58483ff700 1 -- start start 2023-09-13T03:10:58.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.213+0000 7f58483ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5840100610 0x7f5840196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:58.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.213+0000 7f58483ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5840101800 0x7f5840196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:58.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.213+0000 7f58483ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58401021a0 0x7f584019d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:58.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.213+0000 7f58483ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584019dea0 con 0x7f5840100610 2023-09-13T03:10:58.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.213+0000 7f58483ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584019dfe0 con 0x7f5840101800 2023-09-13T03:10:58.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.213+0000 7f58483ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584019e170 con 0x7f58401021a0 2023-09-13T03:10:58.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.214+0000 7f584619b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5840100610 0x7f5840196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:58.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.214+0000 7f584619b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5840100610 0x7f5840196320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46500/0 (socket says 172.21.15.64:46500) 2023-09-13T03:10:58.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.214+0000 7f584619b700 1 -- 172.21.15.64:0/314126668 learned_addr learned my addr 172.21.15.64:0/314126668 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:10:58.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.214+0000 7f584599a700 1 --2- 172.21.15.64:0/314126668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5840101800 0x7f5840196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:58.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.214+0000 7f584699c700 1 --2- 172.21.15.64:0/314126668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58401021a0 0x7f584019d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:58.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.214+0000 7f584619b700 1 -- 172.21.15.64:0/314126668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58401021a0 msgr2=0x7f584019d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:58.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.214+0000 7f584619b700 1 --2- 172.21.15.64:0/314126668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58401021a0 0x7f584019d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:58.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.214+0000 7f584619b700 1 -- 172.21.15.64:0/314126668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5840101800 msgr2=0x7f5840196860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:58.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.214+0000 7f584619b700 1 --2- 172.21.15.64:0/314126668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5840101800 0x7f5840196860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:58.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.214+0000 7f584619b700 1 -- 172.21.15.64:0/314126668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5830009520 con 0x7f5840100610 2023-09-13T03:10:58.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.214+0000 7f584599a700 1 --2- 172.21.15.64:0/314126668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5840101800 0x7f5840196860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:10:58.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.214+0000 7f584619b700 1 --2- 172.21.15.64:0/314126668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5840100610 0x7f5840196320 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f583c0058e0 tx=0x7f583c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:10:58.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.215+0000 7f58377fe700 1 -- 172.21.15.64:0/314126668 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583c01d070 con 0x7f5840100610 2023-09-13T03:10:58.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.215+0000 7f58377fe700 1 -- 172.21.15.64:0/314126668 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f583c005050 con 0x7f5840100610 2023-09-13T03:10:58.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.215+0000 7f58483ff700 1 -- 172.21.15.64:0/314126668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f583c0095f0 con 0x7f5840100610 2023-09-13T03:10:58.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.215+0000 7f58377fe700 1 -- 172.21.15.64:0/314126668 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583c004a20 con 0x7f5840100610 2023-09-13T03:10:58.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.215+0000 7f58483ff700 1 -- 172.21.15.64:0/314126668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f584019e6e0 con 0x7f5840100610 2023-09-13T03:10:58.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.216+0000 7f58483ff700 1 -- 172.21.15.64:0/314126668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f584010be80 con 0x7f5840100610 2023-09-13T03:10:58.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.219+0000 7f58377fe700 1 -- 172.21.15.64:0/314126668 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f583c005450 con 0x7f5840100610 2023-09-13T03:10:58.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.220+0000 7f58377fe700 1 --2- 172.21.15.64:0/314126668 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f582c0a0790 0x7f582c0a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:10:58.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.220+0000 7f58377fe700 1 -- 172.21.15.64:0/314126668 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f583c0cc310 con 0x7f5840100610 2023-09-13T03:10:58.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.220+0000 7f584599a700 1 --2- 172.21.15.64:0/314126668 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f582c0a0790 0x7f582c0a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:10:58.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.221+0000 7f584599a700 1 --2- 172.21.15.64:0/314126668 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f582c0a0790 0x7f582c0a2c50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5830009fd0 tx=0x7f5830009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:10:58.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.228+0000 7f58377fe700 1 -- 172.21.15.64:0/314126668 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f583c0962d0 con 0x7f5840100610 2023-09-13T03:10:58.369 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 13 03:10:58 smithi064 podman[152137]: 2023-09-13T03:10:58.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.507+0000 7f58483ff700 1 -- 172.21.15.64:0/314126668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f584019e9c0 con 0x7f5840100610 2023-09-13T03:10:58.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.508+0000 7f58377fe700 1 -- 172.21.15.64:0/314126668 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7670 (secure 0 0 0) 0x7f584019e9c0 con 0x7f5840100610 2023-09-13T03:10:58.509 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:10:58.509 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2234182985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2234182985},{"type":"v1","addr":"172.21.15.64:6835","nonce":2234182985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4053841873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4053841873},{"type":"v1","addr":"172.21.15.64:6837","nonce":4053841873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24569,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/101906999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":101906999},{"type":"v1","addr":"172.21.15.142:6833","nonce":101906999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1897366227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1897366227},{"type":"v1","addr":"172.21.15.170:6833","nonce":1897366227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/2167779918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2167779918},{"type":"v1","addr":"172.21.15.142:6835","nonce":2167779918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24622,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1493371121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1493371121},{"type":"v1","addr":"172.21.15.170:6835","nonce":1493371121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24634,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/1726587275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":1726587275},{"type":"v1","addr":"172.21.15.170:6837","nonce":1726587275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/741173268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":741173268},{"type":"v1","addr":"172.21.15.142:6837","nonce":741173268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24649,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6839/2189703596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2189703596},{"type":"v1","addr":"172.21.15.170:6839","nonce":2189703596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-09-13T03:10:58.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.511+0000 7f58483ff700 1 -- 172.21.15.64:0/314126668 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f582c0a0790 msgr2=0x7f582c0a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:58.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.511+0000 7f58483ff700 1 --2- 172.21.15.64:0/314126668 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f582c0a0790 0x7f582c0a2c50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5830009fd0 tx=0x7f5830009260 comp rx=0 tx=0).stop 2023-09-13T03:10:58.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.511+0000 7f58483ff700 1 -- 172.21.15.64:0/314126668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5840100610 msgr2=0x7f5840196320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:10:58.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.511+0000 7f58483ff700 1 --2- 172.21.15.64:0/314126668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5840100610 0x7f5840196320 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f583c0058e0 tx=0x7f583c00f690 comp rx=0 tx=0).stop 2023-09-13T03:10:58.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.511+0000 7f58483ff700 1 -- 172.21.15.64:0/314126668 shutdown_connections 2023-09-13T03:10:58.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.511+0000 7f58483ff700 1 --2- 172.21.15.64:0/314126668 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f582c0a0790 0x7f582c0a2c50 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:58.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.511+0000 7f58483ff700 1 --2- 172.21.15.64:0/314126668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58401021a0 0x7f584019d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:58.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.511+0000 7f58483ff700 1 --2- 172.21.15.64:0/314126668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5840100610 0x7f5840196320 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:58.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.511+0000 7f58483ff700 1 --2- 172.21.15.64:0/314126668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5840101800 0x7f5840196860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:10:58.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.511+0000 7f58483ff700 1 -- 172.21.15.64:0/314126668 >> 172.21.15.64:0/314126668 conn(0x7f58400fbb40 msgr2=0x7f58400fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:10:58.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.511+0000 7f58483ff700 1 -- 172.21.15.64:0/314126668 shutdown_connections 2023-09-13T03:10:58.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:10:58.511+0000 7f58483ff700 1 -- 172.21.15.64:0/314126668 wait complete. 2023-09-13T03:10:58.514 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 10 2023-09-13T03:10:58.668 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 13 03:10:58 smithi064 bash[152137]: 6fdaab75e036a30fe5e7270f6747c17fa3a4ceb3150413939c772e787801099c 2023-09-13T03:10:58.668 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 13 03:10:58 smithi064 ceph-mds[152220]: -- 172.21.15.64:0/3589531816 <== mon.0 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5621914671e0 con 0x56219143c800 2023-09-13T03:10:58.669 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 13 03:10:58 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-g[152178]: starting mds.g at 2023-09-13T03:10:58.669 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 13 03:10:58 smithi064 systemd[1]: Started Ceph mds.g for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da. 2023-09-13T03:10:59.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:59 smithi142 ceph-mon[134082]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:59.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:59 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:59.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:59 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:59.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:59 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/314126668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:10:59.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:59 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:59.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:59 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T03:10:59.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:59 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T03:10:59.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:59 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:59.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:59 smithi142 ceph-mon[134082]: Deploying daemon mds.a on smithi064 2023-09-13T03:10:59.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:59 smithi142 ceph-mon[134082]: mds.? [v2:172.21.15.64:6838/99523374,v1:172.21.15.64:6839/99523374] up:boot 2023-09-13T03:10:59.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:59 smithi142 ceph-mon[134082]: fsmap 10 up:standby 2023-09-13T03:10:59.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:10:59 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-13T03:10:59.829 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:59 smithi064 ceph-mon[97675]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:59.830 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:59 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:59.830 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:59 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:59.830 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:59 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/314126668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:10:59.830 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:59 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:59.831 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:59 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T03:10:59.831 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:59 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T03:10:59.831 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:59 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:59.831 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:59 smithi064 ceph-mon[97675]: Deploying daemon mds.a on smithi064 2023-09-13T03:10:59.831 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:59 smithi064 ceph-mon[97675]: mds.? [v2:172.21.15.64:6838/99523374,v1:172.21.15.64:6839/99523374] up:boot 2023-09-13T03:10:59.832 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:59 smithi064 ceph-mon[97675]: fsmap 10 up:standby 2023-09-13T03:10:59.832 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:10:59 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-13T03:10:59.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:59 smithi170 ceph-mon[134328]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:10:59.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:59 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:59.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:59 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:59.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:59 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/314126668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:10:59.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:59 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:10:59.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:59 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T03:10:59.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:59 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T03:10:59.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:59 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:10:59.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:59 smithi170 ceph-mon[134328]: Deploying daemon mds.a on smithi064 2023-09-13T03:10:59.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:59 smithi170 ceph-mon[134328]: mds.? [v2:172.21.15.64:6838/99523374,v1:172.21.15.64:6839/99523374] up:boot 2023-09-13T03:10:59.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:59 smithi170 ceph-mon[134328]: fsmap 10 up:standby 2023-09-13T03:10:59.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:10:59 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-13T03:11:00.531 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T03:11:00.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:00.783+0000 7fe6a25ff700 1 -- 172.21.15.64:0/1115170918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe69c074640 msgr2=0x7fe69c072a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:00.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:00.783+0000 7fe6a25ff700 1 --2- 172.21.15.64:0/1115170918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe69c074640 0x7fe69c072a90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe68c009990 tx=0x7fe68c009ca0 comp rx=0 tx=0).stop 2023-09-13T03:11:00.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:00.783+0000 7fe6a25ff700 1 -- 172.21.15.64:0/1115170918 shutdown_connections 2023-09-13T03:11:00.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:00.783+0000 7fe6a25ff700 1 --2- 172.21.15.64:0/1115170918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe69c10cbd0 0x7fe69c10efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:00.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:00.783+0000 7fe6a25ff700 1 --2- 172.21.15.64:0/1115170918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe69c074640 0x7fe69c072a90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:00.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:00.783+0000 7fe6a25ff700 1 --2- 172.21.15.64:0/1115170918 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe69c073060 0x7fe69c0734e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:00.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:00.783+0000 7fe6a25ff700 1 -- 172.21.15.64:0/1115170918 >> 172.21.15.64:0/1115170918 conn(0x7fe69c0fbb30 msgr2=0x7fe69c0fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:00.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:00.783+0000 7fe6a25ff700 1 -- 172.21.15.64:0/1115170918 shutdown_connections 2023-09-13T03:11:00.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:00.783+0000 7fe6a25ff700 1 -- 172.21.15.64:0/1115170918 wait complete. 2023-09-13T03:11:00.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:00.784+0000 7fe6a25ff700 1 Processor -- start 2023-09-13T03:11:00.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:00.784+0000 7fe6a25ff700 1 -- start start 2023-09-13T03:11:00.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:00.785+0000 7fe6a25ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe69c073060 0x7fe69c19c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:00.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:00.785+0000 7fe6a25ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe69c074640 0x7fe69c19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:00.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:00.785+0000 7fe6a25ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe69c10cbd0 0x7fe69c1a3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:00.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:00.785+0000 7fe6a25ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69c1a4470 con 0x7fe69c074640 2023-09-13T03:11:00.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:00.785+0000 7fe6a25ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69c1a45b0 con 0x7fe69c073060 2023-09-13T03:11:00.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:00.785+0000 7fe6a25ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69c1a4740 con 0x7fe69c10cbd0 2023-09-13T03:11:00.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:00.785+0000 7fe69b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe69c074640 0x7fe69c19ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:00.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:00.785+0000 7fe69bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe69c073060 0x7fe69c19c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:00.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:00.785+0000 7fe69b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe69c074640 0x7fe69c19ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46532/0 (socket says 172.21.15.64:46532) 2023-09-13T03:11:00.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:00.785+0000 7fe69b7fe700 1 -- 172.21.15.64:0/3288022980 learned_addr learned my addr 172.21.15.64:0/3288022980 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:00.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:00.785+0000 7fe6a0b9c700 1 --2- 172.21.15.64:0/3288022980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe69c10cbd0 0x7fe69c1a3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:00.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:00.785+0000 7fe69bfff700 1 -- 172.21.15.64:0/3288022980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe69c10cbd0 msgr2=0x7fe69c1a3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:00.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:00.785+0000 7fe69bfff700 1 --2- 172.21.15.64:0/3288022980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe69c10cbd0 0x7fe69c1a3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:00.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:00.786+0000 7fe69bfff700 1 -- 172.21.15.64:0/3288022980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe69c074640 msgr2=0x7fe69c19ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:00.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:00.786+0000 7fe69bfff700 1 --2- 172.21.15.64:0/3288022980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe69c074640 0x7fe69c19ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:00.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:00.786+0000 7fe69bfff700 1 -- 172.21.15.64:0/3288022980 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe68c0095f0 con 0x7fe69c073060 2023-09-13T03:11:00.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:00.786+0000 7fe69bfff700 1 --2- 172.21.15.64:0/3288022980 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe69c073060 0x7fe69c19c8f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe68c0098e0 tx=0x7fe68c009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:00.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:00.786+0000 7fe6997fa700 1 -- 172.21.15.64:0/3288022980 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68c01d070 con 0x7fe69c073060 2023-09-13T03:11:00.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:00.786+0000 7fe6997fa700 1 -- 172.21.15.64:0/3288022980 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe68c021410 con 0x7fe69c073060 2023-09-13T03:11:00.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:00.786+0000 7fe6a25ff700 1 -- 172.21.15.64:0/3288022980 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe69c1a49c0 con 0x7fe69c073060 2023-09-13T03:11:00.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:00.786+0000 7fe6997fa700 1 -- 172.21.15.64:0/3288022980 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68c005720 con 0x7fe69c073060 2023-09-13T03:11:00.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:00.786+0000 7fe6a25ff700 1 -- 172.21.15.64:0/3288022980 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe69c1a4e40 con 0x7fe69c073060 2023-09-13T03:11:00.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:00.787+0000 7fe6a25ff700 1 -- 172.21.15.64:0/3288022980 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe69c102720 con 0x7fe69c073060 2023-09-13T03:11:00.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:00.789+0000 7fe6997fa700 1 -- 172.21.15.64:0/3288022980 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fe68c021580 con 0x7fe69c073060 2023-09-13T03:11:00.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:00.791+0000 7fe6997fa700 1 --2- 172.21.15.64:0/3288022980 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe6840a0710 0x7fe6840a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:00.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:00.791+0000 7fe6997fa700 1 -- 172.21.15.64:0/3288022980 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe68c0cd260 con 0x7fe69c073060 2023-09-13T03:11:00.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:00.791+0000 7fe69b7fe700 1 --2- 172.21.15.64:0/3288022980 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe6840a0710 0x7fe6840a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:00.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:00.792+0000 7fe69b7fe700 1 --2- 172.21.15.64:0/3288022980 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe6840a0710 0x7fe6840a2bd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe690005950 tx=0x7fe690009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:00.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:00.793+0000 7fe6997fa700 1 -- 172.21.15.64:0/3288022980 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe68c097220 con 0x7fe69c073060 2023-09-13T03:11:01.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:01.056+0000 7fe6a25ff700 1 -- 172.21.15.64:0/3288022980 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe69c112690 con 0x7fe69c073060 2023-09-13T03:11:01.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:01.058+0000 7fe6997fa700 1 -- 172.21.15.64:0/3288022980 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8456 (secure 0 0 0) 0x7fe68c09a840 con 0x7fe69c073060 2023-09-13T03:11:01.058 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:11:01.059 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2234182985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2234182985},{"type":"v1","addr":"172.21.15.64:6835","nonce":2234182985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4053841873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4053841873},{"type":"v1","addr":"172.21.15.64:6837","nonce":4053841873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/99523374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":99523374},{"type":"v1","addr":"172.21.15.64:6839","nonce":99523374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24569,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/101906999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":101906999},{"type":"v1","addr":"172.21.15.142:6833","nonce":101906999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1897366227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1897366227},{"type":"v1","addr":"172.21.15.170:6833","nonce":1897366227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/2167779918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2167779918},{"type":"v1","addr":"172.21.15.142:6835","nonce":2167779918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24622,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1493371121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1493371121},{"type":"v1","addr":"172.21.15.170:6835","nonce":1493371121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24634,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/1726587275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":1726587275},{"type":"v1","addr":"172.21.15.170:6837","nonce":1726587275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/741173268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":741173268},{"type":"v1","addr":"172.21.15.142:6837","nonce":741173268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24649,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6839/2189703596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2189703596},{"type":"v1","addr":"172.21.15.170:6839","nonce":2189703596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-09-13T03:11:01.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:01.060+0000 7fe6a25ff700 1 -- 172.21.15.64:0/3288022980 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe6840a0710 msgr2=0x7fe6840a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:01.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:01.060+0000 7fe6a25ff700 1 --2- 172.21.15.64:0/3288022980 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe6840a0710 0x7fe6840a2bd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe690005950 tx=0x7fe690009260 comp rx=0 tx=0).stop 2023-09-13T03:11:01.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:01.060+0000 7fe6a25ff700 1 -- 172.21.15.64:0/3288022980 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe69c073060 msgr2=0x7fe69c19c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:01.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:01.060+0000 7fe6a25ff700 1 --2- 172.21.15.64:0/3288022980 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe69c073060 0x7fe69c19c8f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe68c0098e0 tx=0x7fe68c009580 comp rx=0 tx=0).stop 2023-09-13T03:11:01.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:01.060+0000 7fe6a25ff700 1 -- 172.21.15.64:0/3288022980 shutdown_connections 2023-09-13T03:11:01.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:01.060+0000 7fe6a25ff700 1 --2- 172.21.15.64:0/3288022980 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe6840a0710 0x7fe6840a2bd0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:01.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:01.061+0000 7fe6a25ff700 1 --2- 172.21.15.64:0/3288022980 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe69c073060 0x7fe69c19c8f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:01.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:01.061+0000 7fe6a25ff700 1 --2- 172.21.15.64:0/3288022980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe69c10cbd0 0x7fe69c1a3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:01.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:01.061+0000 7fe6a25ff700 1 --2- 172.21.15.64:0/3288022980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe69c074640 0x7fe69c19ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:01.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:01.061+0000 7fe6a25ff700 1 -- 172.21.15.64:0/3288022980 >> 172.21.15.64:0/3288022980 conn(0x7fe69c0fbb30 msgr2=0x7fe69c0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:01.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:01.061+0000 7fe6a25ff700 1 -- 172.21.15.64:0/3288022980 shutdown_connections 2023-09-13T03:11:01.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:01.061+0000 7fe6a25ff700 1 -- 172.21.15.64:0/3288022980 wait complete. 2023-09-13T03:11:01.064 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 11 2023-09-13T03:11:01.413 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 13 03:11:00 smithi064 systemd[1]: Starting Ceph mds.a for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:11:01.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:01 smithi064 ceph-mon[97675]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:11:01.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:01 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/3288022980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:11:01.669 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 13 03:11:01 smithi064 podman[152571]: 2023-09-13T03:11:01.669 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 13 03:11:01 smithi064 bash[152571]: 210febf3b32906d7013b6460382a70ae4183738e67ccc281e835a7b1da2a2016 2023-09-13T03:11:01.670 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 13 03:11:01 smithi064 ceph-mds[152612]: -- 172.21.15.64:0/3690060578 <== mon.0 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5608eeaf11e0 con 0x5608eeac7000 2023-09-13T03:11:01.670 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 13 03:11:01 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-a[152587]: starting mds.a at 2023-09-13T03:11:01.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:01 smithi170 ceph-mon[134328]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:11:01.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:01 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/3288022980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:11:01.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:01 smithi142 ceph-mon[134082]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:11:01.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:01 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/3288022980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:11:02.168 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 13 03:11:01 smithi064 systemd[1]: Started Ceph mds.a for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da. 2023-09-13T03:11:03.083 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T03:11:03.084 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:02 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:03.084 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:02 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:03.084 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:02 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:03.084 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:02 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T03:11:03.085 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:02 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T03:11:03.085 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:02 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:11:03.085 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:02 smithi142 ceph-mon[134082]: Deploying daemon mds.b on smithi142 2023-09-13T03:11:03.085 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:02 smithi142 ceph-mon[134082]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:11:03.085 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:02 smithi142 ceph-mon[134082]: mds.? [v2:172.21.15.64:6840/1866582875,v1:172.21.15.64:6841/1866582875] up:boot 2023-09-13T03:11:03.086 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:02 smithi142 ceph-mon[134082]: fsmap 11 up:standby 2023-09-13T03:11:03.086 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:02 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-13T03:11:03.157 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:02 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:03.157 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:02 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:03.157 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:02 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:03.157 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:02 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T03:11:03.157 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:02 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T03:11:03.158 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:02 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:11:03.158 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:02 smithi064 ceph-mon[97675]: Deploying daemon mds.b on smithi142 2023-09-13T03:11:03.158 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:02 smithi064 ceph-mon[97675]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:11:03.158 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:02 smithi064 ceph-mon[97675]: mds.? [v2:172.21.15.64:6840/1866582875,v1:172.21.15.64:6841/1866582875] up:boot 2023-09-13T03:11:03.159 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:02 smithi064 ceph-mon[97675]: fsmap 11 up:standby 2023-09-13T03:11:03.159 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:02 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-13T03:11:03.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:03.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:03.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:03.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-13T03:11:03.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-13T03:11:03.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:11:03.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:02 smithi170 ceph-mon[134328]: Deploying daemon mds.b on smithi142 2023-09-13T03:11:03.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:02 smithi170 ceph-mon[134328]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:11:03.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:02 smithi170 ceph-mon[134328]: mds.? [v2:172.21.15.64:6840/1866582875,v1:172.21.15.64:6841/1866582875] up:boot 2023-09-13T03:11:03.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:02 smithi170 ceph-mon[134328]: fsmap 11 up:standby 2023-09-13T03:11:03.162 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:02 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-13T03:11:03.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.305+0000 7ffb9e3ff700 1 -- 172.21.15.64:0/1870865464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb98102190 msgr2=0x7ffb98108a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:03.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.305+0000 7ffb9e3ff700 1 --2- 172.21.15.64:0/1870865464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb98102190 0x7ffb98108a40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ffb9000b0d0 tx=0x7ffb9000b3e0 comp rx=0 tx=0).stop 2023-09-13T03:11:03.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.305+0000 7ffb9e3ff700 1 -- 172.21.15.64:0/1870865464 shutdown_connections 2023-09-13T03:11:03.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.305+0000 7ffb9e3ff700 1 --2- 172.21.15.64:0/1870865464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb98100600 0x7ffb98100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:03.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.305+0000 7ffb9e3ff700 1 --2- 172.21.15.64:0/1870865464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb98102190 0x7ffb98108a40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:03.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.305+0000 7ffb9e3ff700 1 --2- 172.21.15.64:0/1870865464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb981017f0 0x7ffb98101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:03.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.305+0000 7ffb9e3ff700 1 -- 172.21.15.64:0/1870865464 >> 172.21.15.64:0/1870865464 conn(0x7ffb980fbb30 msgr2=0x7ffb980fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:03.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.305+0000 7ffb9e3ff700 1 -- 172.21.15.64:0/1870865464 shutdown_connections 2023-09-13T03:11:03.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.305+0000 7ffb9e3ff700 1 -- 172.21.15.64:0/1870865464 wait complete. 2023-09-13T03:11:03.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.306+0000 7ffb9e3ff700 1 Processor -- start 2023-09-13T03:11:03.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.306+0000 7ffb9e3ff700 1 -- start start 2023-09-13T03:11:03.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.307+0000 7ffb9e3ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb98100600 0x7ffb98198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:03.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.307+0000 7ffb9e3ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb981017f0 0x7ffb98198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:03.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.307+0000 7ffb9e3ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb98102190 0x7ffb9819fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:03.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.307+0000 7ffb9e3ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb981a0100 con 0x7ffb98102190 2023-09-13T03:11:03.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.307+0000 7ffb9e3ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb981a0240 con 0x7ffb981017f0 2023-09-13T03:11:03.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.307+0000 7ffb9e3ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb981a03d0 con 0x7ffb98100600 2023-09-13T03:11:03.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.307+0000 7ffb977fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb981017f0 0x7ffb98198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:03.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.307+0000 7ffb977fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb981017f0 0x7ffb98198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:41326/0 (socket says 172.21.15.64:41326) 2023-09-13T03:11:03.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.307+0000 7ffb977fe700 1 -- 172.21.15.64:0/2476536374 learned_addr learned my addr 172.21.15.64:0/2476536374 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:03.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.307+0000 7ffb9c99c700 1 --2- 172.21.15.64:0/2476536374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb98102190 0x7ffb9819fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:03.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.307+0000 7ffb97fff700 1 --2- 172.21.15.64:0/2476536374 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb98100600 0x7ffb98198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:03.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.307+0000 7ffb977fe700 1 -- 172.21.15.64:0/2476536374 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb98100600 msgr2=0x7ffb98198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:03.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.307+0000 7ffb977fe700 1 --2- 172.21.15.64:0/2476536374 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb98100600 0x7ffb98198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:03.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.307+0000 7ffb977fe700 1 -- 172.21.15.64:0/2476536374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb98102190 msgr2=0x7ffb9819fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:03.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.307+0000 7ffb977fe700 1 --2- 172.21.15.64:0/2476536374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb98102190 0x7ffb9819fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:03.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.307+0000 7ffb977fe700 1 -- 172.21.15.64:0/2476536374 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb88009520 con 0x7ffb981017f0 2023-09-13T03:11:03.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.308+0000 7ffb977fe700 1 --2- 172.21.15.64:0/2476536374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb981017f0 0x7ffb98198ac0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c00e9a0 tx=0x7ffb8c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:03.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.308+0000 7ffb957fa700 1 -- 172.21.15.64:0/2476536374 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c009790 con 0x7ffb981017f0 2023-09-13T03:11:03.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.308+0000 7ffb957fa700 1 -- 172.21.15.64:0/2476536374 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb8c010430 con 0x7ffb981017f0 2023-09-13T03:11:03.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.308+0000 7ffb9e3ff700 1 -- 172.21.15.64:0/2476536374 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb90009d00 con 0x7ffb981017f0 2023-09-13T03:11:03.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.308+0000 7ffb957fa700 1 -- 172.21.15.64:0/2476536374 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c018680 con 0x7ffb981017f0 2023-09-13T03:11:03.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.308+0000 7ffb9e3ff700 1 -- 172.21.15.64:0/2476536374 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb981a0a30 con 0x7ffb981017f0 2023-09-13T03:11:03.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.309+0000 7ffb9e3ff700 1 -- 172.21.15.64:0/2476536374 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb9810be20 con 0x7ffb981017f0 2023-09-13T03:11:03.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.311+0000 7ffb957fa700 1 -- 172.21.15.64:0/2476536374 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7ffb8c0105a0 con 0x7ffb981017f0 2023-09-13T03:11:03.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.312+0000 7ffb957fa700 1 --2- 172.21.15.64:0/2476536374 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ffb800a0690 0x7ffb800a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:03.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.312+0000 7ffb957fa700 1 -- 172.21.15.64:0/2476536374 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ffb8c014070 con 0x7ffb981017f0 2023-09-13T03:11:03.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.313+0000 7ffb97fff700 1 --2- 172.21.15.64:0/2476536374 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ffb800a0690 0x7ffb800a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:03.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.313+0000 7ffb97fff700 1 --2- 172.21.15.64:0/2476536374 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ffb800a0690 0x7ffb800a2b50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ffb98101650 tx=0x7ffb88009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:03.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.317+0000 7ffb957fa700 1 -- 172.21.15.64:0/2476536374 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ffb8c095a50 con 0x7ffb981017f0 2023-09-13T03:11:03.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.573+0000 7ffb9e3ff700 1 -- 172.21.15.64:0/2476536374 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb9810c040 con 0x7ffb981017f0 2023-09-13T03:11:03.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.574+0000 7ffb957fa700 1 -- 172.21.15.64:0/2476536374 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9248 (secure 0 0 0) 0x7ffb8c099070 con 0x7ffb981017f0 2023-09-13T03:11:03.575 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:11:03.575 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2234182985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2234182985},{"type":"v1","addr":"172.21.15.64:6835","nonce":2234182985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4053841873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4053841873},{"type":"v1","addr":"172.21.15.64:6837","nonce":4053841873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/99523374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":99523374},{"type":"v1","addr":"172.21.15.64:6839","nonce":99523374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/1866582875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":1866582875},{"type":"v1","addr":"172.21.15.64:6841","nonce":1866582875}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24569,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/101906999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":101906999},{"type":"v1","addr":"172.21.15.142:6833","nonce":101906999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1897366227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1897366227},{"type":"v1","addr":"172.21.15.170:6833","nonce":1897366227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/2167779918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2167779918},{"type":"v1","addr":"172.21.15.142:6835","nonce":2167779918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24622,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1493371121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1493371121},{"type":"v1","addr":"172.21.15.170:6835","nonce":1493371121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24634,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/1726587275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":1726587275},{"type":"v1","addr":"172.21.15.170:6837","nonce":1726587275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/741173268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":741173268},{"type":"v1","addr":"172.21.15.142:6837","nonce":741173268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24649,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6839/2189703596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2189703596},{"type":"v1","addr":"172.21.15.170:6839","nonce":2189703596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-09-13T03:11:03.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.577+0000 7ffb9e3ff700 1 -- 172.21.15.64:0/2476536374 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ffb800a0690 msgr2=0x7ffb800a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:03.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.577+0000 7ffb9e3ff700 1 --2- 172.21.15.64:0/2476536374 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ffb800a0690 0x7ffb800a2b50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ffb98101650 tx=0x7ffb88009260 comp rx=0 tx=0).stop 2023-09-13T03:11:03.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.577+0000 7ffb9e3ff700 1 -- 172.21.15.64:0/2476536374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb981017f0 msgr2=0x7ffb98198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:03.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.577+0000 7ffb9e3ff700 1 --2- 172.21.15.64:0/2476536374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb981017f0 0x7ffb98198ac0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c00e9a0 tx=0x7ffb8c00ecb0 comp rx=0 tx=0).stop 2023-09-13T03:11:03.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.577+0000 7ffb9e3ff700 1 -- 172.21.15.64:0/2476536374 shutdown_connections 2023-09-13T03:11:03.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.577+0000 7ffb9e3ff700 1 --2- 172.21.15.64:0/2476536374 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ffb800a0690 0x7ffb800a2b50 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:03.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.577+0000 7ffb9e3ff700 1 --2- 172.21.15.64:0/2476536374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb98102190 0x7ffb9819fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:03.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.577+0000 7ffb9e3ff700 1 --2- 172.21.15.64:0/2476536374 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb98100600 0x7ffb98198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:03.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.577+0000 7ffb9e3ff700 1 --2- 172.21.15.64:0/2476536374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb981017f0 0x7ffb98198ac0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:03.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.577+0000 7ffb9e3ff700 1 -- 172.21.15.64:0/2476536374 >> 172.21.15.64:0/2476536374 conn(0x7ffb980fbb30 msgr2=0x7ffb98107050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:03.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.577+0000 7ffb9e3ff700 1 -- 172.21.15.64:0/2476536374 shutdown_connections 2023-09-13T03:11:03.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:03.577+0000 7ffb9e3ff700 1 -- 172.21.15.64:0/2476536374 wait complete. 2023-09-13T03:11:03.580 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 12 2023-09-13T03:11:03.745 INFO:journalctl@ceph.mds.b.smithi142.stdout:Sep 13 03:11:03 smithi142 systemd[1]: Starting Ceph mds.b for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:11:03.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:03 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2476536374' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:11:04.013 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:03 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/2476536374' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:11:04.014 INFO:journalctl@ceph.mds.b.smithi142.stdout:Sep 13 03:11:03 smithi142 podman[170646]: 2023-09-13T03:11:04.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:03 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/2476536374' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:11:04.495 INFO:journalctl@ceph.mds.b.smithi142.stdout:Sep 13 03:11:04 smithi142 ceph-mds[170686]: -- 172.21.15.142:0/1189610587 <== mon.2 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560303a5b1e0 con 0x560303a30c00 2023-09-13T03:11:04.495 INFO:journalctl@ceph.mds.b.smithi142.stdout:Sep 13 03:11:04 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-b[170662]: starting mds.b at 2023-09-13T03:11:04.496 INFO:journalctl@ceph.mds.b.smithi142.stdout:Sep 13 03:11:04 smithi142 bash[170646]: 79a5d73f522cd0c0a205ee4ddea08ab6aae9465b4539205e887aaaa8552e7764 2023-09-13T03:11:04.496 INFO:journalctl@ceph.mds.b.smithi142.stdout:Sep 13 03:11:04 smithi142 systemd[1]: Started Ceph mds.b for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da. 2023-09-13T03:11:05.409 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:05 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:05.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:05 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:05.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:05 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:05.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:05 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:05.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:05 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:05.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:05 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:11:05.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:05 smithi170 ceph-mon[134328]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:11:05.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:05 smithi170 ceph-mon[134328]: mds.? [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] up:boot 2023-09-13T03:11:05.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:05 smithi170 ceph-mon[134328]: fsmap 12 up:standby 2023-09-13T03:11:05.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:05 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-13T03:11:05.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:05 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:05.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:05 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:05.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:05 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:05.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:05 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:05.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:05 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:05.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:05 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:11:05.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:05 smithi064 ceph-mon[97675]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:11:05.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:05 smithi064 ceph-mon[97675]: mds.? [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] up:boot 2023-09-13T03:11:05.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:05 smithi064 ceph-mon[97675]: fsmap 12 up:standby 2023-09-13T03:11:05.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:05 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-13T03:11:05.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:05 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:05.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:05 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:05.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:05 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:05.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:05 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:05.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:05 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:05.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:05 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:11:05.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:05 smithi142 ceph-mon[134082]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:11:05.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:05 smithi142 ceph-mon[134082]: mds.? [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] up:boot 2023-09-13T03:11:05.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:05 smithi142 ceph-mon[134082]: fsmap 12 up:standby 2023-09-13T03:11:05.498 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:05 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-13T03:11:05.597 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T03:11:05.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:05.816+0000 7fd368fff700 1 -- 172.21.15.64:0/1401160423 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3641017f0 msgr2=0x7fd364101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:05.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:05.816+0000 7fd368fff700 1 --2- 172.21.15.64:0/1401160423 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3641017f0 0x7fd364101c50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd358009990 tx=0x7fd358009ca0 comp rx=0 tx=0).stop 2023-09-13T03:11:05.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:05.816+0000 7fd368fff700 1 -- 172.21.15.64:0/1401160423 shutdown_connections 2023-09-13T03:11:05.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:05.816+0000 7fd368fff700 1 --2- 172.21.15.64:0/1401160423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd364102190 0x7fd364108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:05.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:05.816+0000 7fd368fff700 1 --2- 172.21.15.64:0/1401160423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd364100600 0x7fd364100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:05.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:05.816+0000 7fd368fff700 1 --2- 172.21.15.64:0/1401160423 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3641017f0 0x7fd364101c50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:05.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:05.816+0000 7fd368fff700 1 -- 172.21.15.64:0/1401160423 >> 172.21.15.64:0/1401160423 conn(0x7fd3640fbb90 msgr2=0x7fd3640fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:05.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:05.816+0000 7fd368fff700 1 -- 172.21.15.64:0/1401160423 shutdown_connections 2023-09-13T03:11:05.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:05.816+0000 7fd368fff700 1 -- 172.21.15.64:0/1401160423 wait complete. 2023-09-13T03:11:05.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:05.817+0000 7fd368fff700 1 Processor -- start 2023-09-13T03:11:05.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:05.817+0000 7fd368fff700 1 -- start start 2023-09-13T03:11:05.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:05.818+0000 7fd368fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd364100600 0x7fd3641984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:05.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:05.818+0000 7fd368fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3641017f0 0x7fd364198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:05.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:05.818+0000 7fd368fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd364102190 0x7fd36419fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:05.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:05.818+0000 7fd368fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3641a0040 con 0x7fd3641017f0 2023-09-13T03:11:05.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:05.818+0000 7fd368fff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3641a0180 con 0x7fd364102190 2023-09-13T03:11:05.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:05.818+0000 7fd368fff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3641a0310 con 0x7fd364100600 2023-09-13T03:11:05.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:05.818+0000 7fd36259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd364100600 0x7fd3641984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:05.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:05.818+0000 7fd36259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd364100600 0x7fd3641984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:41332/0 (socket says 172.21.15.64:41332) 2023-09-13T03:11:05.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:05.818+0000 7fd36259c700 1 -- 172.21.15.64:0/2865198699 learned_addr learned my addr 172.21.15.64:0/2865198699 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:05.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:05.818+0000 7fd361d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3641017f0 0x7fd364198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:05.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:05.818+0000 7fd362d9d700 1 --2- 172.21.15.64:0/2865198699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd364102190 0x7fd36419fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:05.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:05.818+0000 7fd36259c700 1 -- 172.21.15.64:0/2865198699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd364102190 msgr2=0x7fd36419fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:05.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:05.818+0000 7fd36259c700 1 --2- 172.21.15.64:0/2865198699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd364102190 0x7fd36419fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:05.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:05.818+0000 7fd36259c700 1 -- 172.21.15.64:0/2865198699 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3641017f0 msgr2=0x7fd364198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:05.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:05.818+0000 7fd36259c700 1 --2- 172.21.15.64:0/2865198699 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3641017f0 0x7fd364198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:05.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:05.818+0000 7fd36259c700 1 -- 172.21.15.64:0/2865198699 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd35c009c40 con 0x7fd364100600 2023-09-13T03:11:05.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:05.818+0000 7fd362d9d700 1 --2- 172.21.15.64:0/2865198699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd364102190 0x7fd36419fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:11:05.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:05.819+0000 7fd36259c700 1 --2- 172.21.15.64:0/2865198699 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd364100600 0x7fd3641984c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd35400e970 tx=0x7fd35400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:05.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:05.819+0000 7fd3537fe700 1 -- 172.21.15.64:0/2865198699 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd354009790 con 0x7fd364100600 2023-09-13T03:11:05.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:05.819+0000 7fd3537fe700 1 -- 172.21.15.64:0/2865198699 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd35400c7c0 con 0x7fd364100600 2023-09-13T03:11:05.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:05.819+0000 7fd3537fe700 1 -- 172.21.15.64:0/2865198699 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd354003c20 con 0x7fd364100600 2023-09-13T03:11:05.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:05.819+0000 7fd368fff700 1 -- 172.21.15.64:0/2865198699 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3580095f0 con 0x7fd364100600 2023-09-13T03:11:05.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:05.819+0000 7fd368fff700 1 -- 172.21.15.64:0/2865198699 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3641a0920 con 0x7fd364100600 2023-09-13T03:11:05.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:05.820+0000 7fd368fff700 1 -- 172.21.15.64:0/2865198699 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd364192c10 con 0x7fd364100600 2023-09-13T03:11:05.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:05.822+0000 7fd3537fe700 1 -- 172.21.15.64:0/2865198699 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fd35400c930 con 0x7fd364100600 2023-09-13T03:11:05.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:05.823+0000 7fd3537fe700 1 --2- 172.21.15.64:0/2865198699 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd34c0a0690 0x7fd34c0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:05.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:05.824+0000 7fd3537fe700 1 -- 172.21.15.64:0/2865198699 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd354014070 con 0x7fd364100600 2023-09-13T03:11:05.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:05.824+0000 7fd361d9b700 1 --2- 172.21.15.64:0/2865198699 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd34c0a0690 0x7fd34c0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:05.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:05.824+0000 7fd361d9b700 1 --2- 172.21.15.64:0/2865198699 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd34c0a0690 0x7fd34c0a2b50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd35800b2e0 tx=0x7fd358005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:05.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:05.827+0000 7fd3537fe700 1 -- 172.21.15.64:0/2865198699 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fd354095b00 con 0x7fd364100600 2023-09-13T03:11:06.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.111+0000 7fd368fff700 1 -- 172.21.15.64:0/2865198699 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd36410be20 con 0x7fd364100600 2023-09-13T03:11:06.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.112+0000 7fd3537fe700 1 -- 172.21.15.64:0/2865198699 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10043 (secure 0 0 0) 0x7fd354099120 con 0x7fd364100600 2023-09-13T03:11:06.113 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:11:06.113 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2234182985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2234182985},{"type":"v1","addr":"172.21.15.64:6835","nonce":2234182985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4053841873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4053841873},{"type":"v1","addr":"172.21.15.64:6837","nonce":4053841873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/99523374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":99523374},{"type":"v1","addr":"172.21.15.64:6839","nonce":99523374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/1866582875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":1866582875},{"type":"v1","addr":"172.21.15.64:6841","nonce":1866582875}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24569,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/101906999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":101906999},{"type":"v1","addr":"172.21.15.142:6833","nonce":101906999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24574,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1897366227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1897366227},{"type":"v1","addr":"172.21.15.170:6833","nonce":1897366227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24611,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/2167779918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2167779918},{"type":"v1","addr":"172.21.15.142:6835","nonce":2167779918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24622,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1493371121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1493371121},{"type":"v1","addr":"172.21.15.170:6835","nonce":1493371121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24634,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/1726587275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":1726587275},{"type":"v1","addr":"172.21.15.170:6837","nonce":1726587275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24635,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/741173268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":741173268},{"type":"v1","addr":"172.21.15.142:6837","nonce":741173268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24649,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6839/2189703596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2189703596},{"type":"v1","addr":"172.21.15.170:6839","nonce":2189703596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24686,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6839/3645998616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645998616},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645998616}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2023-09-13T03:11:06.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.115+0000 7fd368fff700 1 -- 172.21.15.64:0/2865198699 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd34c0a0690 msgr2=0x7fd34c0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:06.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.115+0000 7fd368fff700 1 --2- 172.21.15.64:0/2865198699 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd34c0a0690 0x7fd34c0a2b50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd35800b2e0 tx=0x7fd358005fb0 comp rx=0 tx=0).stop 2023-09-13T03:11:06.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.115+0000 7fd368fff700 1 -- 172.21.15.64:0/2865198699 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd364100600 msgr2=0x7fd3641984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:06.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.115+0000 7fd368fff700 1 --2- 172.21.15.64:0/2865198699 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd364100600 0x7fd3641984c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd35400e970 tx=0x7fd35400ec80 comp rx=0 tx=0).stop 2023-09-13T03:11:06.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.115+0000 7fd368fff700 1 -- 172.21.15.64:0/2865198699 shutdown_connections 2023-09-13T03:11:06.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.115+0000 7fd368fff700 1 --2- 172.21.15.64:0/2865198699 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd34c0a0690 0x7fd34c0a2b50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:06.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.115+0000 7fd368fff700 1 --2- 172.21.15.64:0/2865198699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd364102190 0x7fd36419fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:06.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.115+0000 7fd368fff700 1 --2- 172.21.15.64:0/2865198699 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd364100600 0x7fd3641984c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:06.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.115+0000 7fd368fff700 1 --2- 172.21.15.64:0/2865198699 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3641017f0 0x7fd364198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:06.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.115+0000 7fd368fff700 1 -- 172.21.15.64:0/2865198699 >> 172.21.15.64:0/2865198699 conn(0x7fd3640fbb90 msgr2=0x7fd3641070d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:06.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.115+0000 7fd368fff700 1 -- 172.21.15.64:0/2865198699 shutdown_connections 2023-09-13T03:11:06.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.115+0000 7fd368fff700 1 -- 172.21.15.64:0/2865198699 wait complete. 2023-09-13T03:11:06.118 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 13 2023-09-13T03:11:06.134 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-13T03:11:06.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.346+0000 7eff20bff700 1 -- 172.21.15.64:0/3902001034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff1c106320 msgr2=0x7eff1c108730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:06.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.346+0000 7eff20bff700 1 --2- 172.21.15.64:0/3902001034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff1c106320 0x7eff1c108730 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7eff14009f90 tx=0x7eff1400b330 comp rx=0 tx=0).stop 2023-09-13T03:11:06.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.346+0000 7eff20bff700 1 -- 172.21.15.64:0/3902001034 shutdown_connections 2023-09-13T03:11:06.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.346+0000 7eff20bff700 1 --2- 172.21.15.64:0/3902001034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff1c106320 0x7eff1c108730 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:06.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.346+0000 7eff20bff700 1 --2- 172.21.15.64:0/3902001034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff1c1010c0 0x7eff1c1034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:06.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.346+0000 7eff20bff700 1 --2- 172.21.15.64:0/3902001034 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff1c1039f0 0x7eff1c105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:06.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.346+0000 7eff20bff700 1 -- 172.21.15.64:0/3902001034 >> 172.21.15.64:0/3902001034 conn(0x7eff1c0faa30 msgr2=0x7eff1c0fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:06.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.346+0000 7eff20bff700 1 -- 172.21.15.64:0/3902001034 shutdown_connections 2023-09-13T03:11:06.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.346+0000 7eff20bff700 1 -- 172.21.15.64:0/3902001034 wait complete. 2023-09-13T03:11:06.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.348+0000 7eff20bff700 1 Processor -- start 2023-09-13T03:11:06.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.348+0000 7eff20bff700 1 -- start start 2023-09-13T03:11:06.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.348+0000 7eff20bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff1c1010c0 0x7eff1c196400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:06.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.348+0000 7eff20bff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff1c1039f0 0x7eff1c196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:06.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.348+0000 7eff20bff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff1c106320 0x7eff1c19d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:06.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.349+0000 7eff20bff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1c19df80 con 0x7eff1c106320 2023-09-13T03:11:06.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.349+0000 7eff20bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1c19e0c0 con 0x7eff1c1010c0 2023-09-13T03:11:06.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.349+0000 7eff20bff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1c19e250 con 0x7eff1c1039f0 2023-09-13T03:11:06.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.349+0000 7eff1ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff1c106320 0x7eff1c19d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:06.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.349+0000 7eff1a59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff1c1010c0 0x7eff1c196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:06.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.349+0000 7eff13fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff1c1039f0 0x7eff1c196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:06.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.349+0000 7eff1a59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff1c1010c0 0x7eff1c196400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:34924/0 (socket says 172.21.15.64:34924) 2023-09-13T03:11:06.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.349+0000 7eff1a59c700 1 -- 172.21.15.64:0/979424839 learned_addr learned my addr 172.21.15.64:0/979424839 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:06.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.349+0000 7eff1a59c700 1 -- 172.21.15.64:0/979424839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff1c1039f0 msgr2=0x7eff1c196940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:06.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.349+0000 7eff1a59c700 1 --2- 172.21.15.64:0/979424839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff1c1039f0 0x7eff1c196940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:06.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.349+0000 7eff1a59c700 1 -- 172.21.15.64:0/979424839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff1c106320 msgr2=0x7eff1c19d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:06.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.349+0000 7eff1a59c700 1 --2- 172.21.15.64:0/979424839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff1c106320 0x7eff1c19d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:06.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.349+0000 7eff1a59c700 1 -- 172.21.15.64:0/979424839 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff0c0095f0 con 0x7eff1c1010c0 2023-09-13T03:11:06.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.349+0000 7eff1a59c700 1 --2- 172.21.15.64:0/979424839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff1c1010c0 0x7eff1c196400 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7eff0c00ec10 tx=0x7eff0c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:06.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.350+0000 7eff137fe700 1 -- 172.21.15.64:0/979424839 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0c00cc90 con 0x7eff1c1010c0 2023-09-13T03:11:06.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.350+0000 7eff137fe700 1 -- 172.21.15.64:0/979424839 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff0c0044e0 con 0x7eff1c1010c0 2023-09-13T03:11:06.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.350+0000 7eff20bff700 1 -- 172.21.15.64:0/979424839 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff14009c40 con 0x7eff1c1010c0 2023-09-13T03:11:06.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.350+0000 7eff137fe700 1 -- 172.21.15.64:0/979424839 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0c010430 con 0x7eff1c1010c0 2023-09-13T03:11:06.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.350+0000 7eff20bff700 1 -- 172.21.15.64:0/979424839 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff1c19e900 con 0x7eff1c1010c0 2023-09-13T03:11:06.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.351+0000 7eff20bff700 1 -- 172.21.15.64:0/979424839 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff1c0fc5b0 con 0x7eff1c1010c0 2023-09-13T03:11:06.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.353+0000 7eff137fe700 1 -- 172.21.15.64:0/979424839 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7eff0c003fb0 con 0x7eff1c1010c0 2023-09-13T03:11:06.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.354+0000 7eff137fe700 1 --2- 172.21.15.64:0/979424839 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7eff080a0710 0x7eff080a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:06.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.355+0000 7eff137fe700 1 -- 172.21.15.64:0/979424839 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7eff0c014070 con 0x7eff1c1010c0 2023-09-13T03:11:06.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.355+0000 7eff13fff700 1 --2- 172.21.15.64:0/979424839 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7eff080a0710 0x7eff080a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:06.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.355+0000 7eff13fff700 1 --2- 172.21.15.64:0/979424839 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7eff080a0710 0x7eff080a2bd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7eff04009c60 tx=0x7eff04009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:06.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.358+0000 7eff137fe700 1 -- 172.21.15.64:0/979424839 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7eff0c095b90 con 0x7eff1c1010c0 2023-09-13T03:11:06.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:06 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2865198699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:11:06.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:06 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/2865198699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:11:06.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.583+0000 7eff20bff700 1 -- 172.21.15.64:0/979424839 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff1c10be30 con 0x7eff1c1010c0 2023-09-13T03:11:06.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.584+0000 7eff137fe700 1 -- 172.21.15.64:0/979424839 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18445 (secure 0 0 0) 0x7eff0c0991b0 con 0x7eff1c1010c0 2023-09-13T03:11:06.585 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:11:06.585 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":79,"fsid":"f8dbced4-51e1-11ee-9ab7-7b867c8bd7da","created":"2023-09-13T03:03:24.167202+0000","modified":"2023-09-13T03:10:23.938509+0000","last_up_change":"2023-09-13T03:10:22.503529+0000","last_in_change":"2023-09-13T03:10:05.678720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T03:06:58.987479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3997c114-241b-48c4-8feb-653c99209470","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6803","nonce":3940892135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6805","nonce":3940892135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6809","nonce":3940892135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6807","nonce":3940892135}]},"public_addr":"172.21.15.64:6803/3940892135","cluster_addr":"172.21.15.64:6805/3940892135","heartbeat_back_addr":"172.21.15.64:6809/3940892135","heartbeat_front_addr":"172.21.15.64:6807/3940892135","state":["exists","up"]},{"osd":1,"uuid":"94cfae85-322e-4ed2-a6e6-e40349bb14bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6801","nonce":4188149333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6803","nonce":4188149333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6807","nonce":4188149333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6805","nonce":4188149333}]},"public_addr":"172.21.15.142:6801/4188149333","cluster_addr":"172.21.15.142:6803/4188149333","heartbeat_back_addr":"172.21.15.142:6807/4188149333","heartbeat_front_addr":"172.21.15.142:6805/4188149333","state":["exists","up"]},{"osd":2,"uuid":"b3aae17e-96db-4008-8c27-f733c46b30ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6801","nonce":2406669338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6803","nonce":2406669338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6807","nonce":2406669338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6805","nonce":2406669338}]},"public_addr":"172.21.15.170:6801/2406669338","cluster_addr":"172.21.15.170:6803/2406669338","heartbeat_back_addr":"172.21.15.170:6807/2406669338","heartbeat_front_addr":"172.21.15.170:6805/2406669338","state":["exists","up"]},{"osd":3,"uuid":"7fa8d8cb-2ab7-48a7-8b49-5e2903d6779f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6811","nonce":3558318707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6813","nonce":3558318707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6817","nonce":3558318707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6815","nonce":3558318707}]},"public_addr":"172.21.15.64:6811/3558318707","cluster_addr":"172.21.15.64:6813/3558318707","heartbeat_back_addr":"172.21.15.64:6817/3558318707","heartbeat_front_addr":"172.21.15.64:6815/3558318707","state":["exists","up"]},{"osd":4,"uuid":"8c7a9c9f-f919-48d3-9db2-214a25bb5603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6809","nonce":1811142576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6811","nonce":1811142576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6815","nonce":1811142576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6813","nonce":1811142576}]},"public_addr":"172.21.15.142:6809/1811142576","cluster_addr":"172.21.15.142:6811/1811142576","heartbeat_back_addr":"172.21.15.142:6815/1811142576","heartbeat_front_addr":"172.21.15.142:6813/1811142576","state":["exists","up"]},{"osd":5,"uuid":"0c6101c6-2065-4022-a751-d3f216d7f478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6809","nonce":2903841453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6811","nonce":2903841453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6815","nonce":2903841453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6813","nonce":2903841453}]},"public_addr":"172.21.15.170:6809/2903841453","cluster_addr":"172.21.15.170:6811/2903841453","heartbeat_back_addr":"172.21.15.170:6815/2903841453","heartbeat_front_addr":"172.21.15.170:6813/2903841453","state":["exists","up"]},{"osd":6,"uuid":"5cf3e52e-d155-41b1-9194-db047d26a953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6819","nonce":1436584490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6821","nonce":1436584490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6825","nonce":1436584490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6823","nonce":1436584490}]},"public_addr":"172.21.15.64:6819/1436584490","cluster_addr":"172.21.15.64:6821/1436584490","heartbeat_back_addr":"172.21.15.64:6825/1436584490","heartbeat_front_addr":"172.21.15.64:6823/1436584490","state":["exists","up"]},{"osd":7,"uuid":"e97323e3-badd-41a7-af9e-e78a81a64046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6817","nonce":3726793534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6819","nonce":3726793534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6823","nonce":3726793534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6821","nonce":3726793534}]},"public_addr":"172.21.15.142:6817/3726793534","cluster_addr":"172.21.15.142:6819/3726793534","heartbeat_back_addr":"172.21.15.142:6823/3726793534","heartbeat_front_addr":"172.21.15.142:6821/3726793534","state":["exists","up"]},{"osd":8,"uuid":"83d2835c-ed26-493d-9222-c2894ec0e071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6817","nonce":4272513873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6819","nonce":4272513873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6823","nonce":4272513873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6821","nonce":4272513873}]},"public_addr":"172.21.15.170:6817/4272513873","cluster_addr":"172.21.15.170:6819/4272513873","heartbeat_back_addr":"172.21.15.170:6823/4272513873","heartbeat_front_addr":"172.21.15.170:6821/4272513873","state":["exists","up"]},{"osd":9,"uuid":"cb936bdd-1bec-4236-aa2e-a24adcac11e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6827","nonce":2641175211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6829","nonce":2641175211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6833","nonce":2641175211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6831","nonce":2641175211}]},"public_addr":"172.21.15.64:6827/2641175211","cluster_addr":"172.21.15.64:6829/2641175211","heartbeat_back_addr":"172.21.15.64:6833/2641175211","heartbeat_front_addr":"172.21.15.64:6831/2641175211","state":["exists","up"]},{"osd":10,"uuid":"85512a1c-6171-43d4-9c15-c588a8b71be0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6825","nonce":936499217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6827","nonce":936499217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6831","nonce":936499217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6829","nonce":936499217}]},"public_addr":"172.21.15.142:6825/936499217","cluster_addr":"172.21.15.142:6827/936499217","heartbeat_back_addr":"172.21.15.142:6831/936499217","heartbeat_front_addr":"172.21.15.142:6829/936499217","state":["exists","up"]},{"osd":11,"uuid":"9b8417dc-5d6c-4917-94a9-b032c5bc33d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6825","nonce":3605017060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6827","nonce":3605017060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6831","nonce":3605017060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6829","nonce":3605017060}]},"public_addr":"172.21.15.170:6825/3605017060","cluster_addr":"172.21.15.170:6827/3605017060","heartbeat_back_addr":"172.21.15.170:6831/3605017060","heartbeat_front_addr":"172.21.15.170:6829/3605017060","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:14.229044+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:36.147777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:56.655298+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:07:23.107741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:07:44.663770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:05.577432+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:31.168031+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:52.791055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:09:13.658546+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:09:39.150360+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:10:00.601735+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:10:21.407555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6800/2955227347":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/1373223118":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/271194613":"2023-09-14T03:04:22.355817+0000","172.21.15.64:6801/3998091347":"2023-09-14T03:03:56.442541+0000","172.21.15.64:6800/3998091347":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/920818572":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/735789005":"2023-09-14T03:03:56.442541+0000","172.21.15.64:6801/2955227347":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/1975923178":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/1500151178":"2023-09-14T03:04:22.355817+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T03:11:06.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.587+0000 7eff20bff700 1 -- 172.21.15.64:0/979424839 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7eff080a0710 msgr2=0x7eff080a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:06.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.587+0000 7eff20bff700 1 --2- 172.21.15.64:0/979424839 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7eff080a0710 0x7eff080a2bd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7eff04009c60 tx=0x7eff04009260 comp rx=0 tx=0).stop 2023-09-13T03:11:06.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.587+0000 7eff20bff700 1 -- 172.21.15.64:0/979424839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff1c1010c0 msgr2=0x7eff1c196400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:06.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.587+0000 7eff20bff700 1 --2- 172.21.15.64:0/979424839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff1c1010c0 0x7eff1c196400 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7eff0c00ec10 tx=0x7eff0c00ef20 comp rx=0 tx=0).stop 2023-09-13T03:11:06.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.587+0000 7eff20bff700 1 -- 172.21.15.64:0/979424839 shutdown_connections 2023-09-13T03:11:06.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.587+0000 7eff20bff700 1 --2- 172.21.15.64:0/979424839 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7eff080a0710 0x7eff080a2bd0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:06.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.587+0000 7eff20bff700 1 --2- 172.21.15.64:0/979424839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff1c1010c0 0x7eff1c196400 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:06.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.587+0000 7eff20bff700 1 --2- 172.21.15.64:0/979424839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff1c106320 0x7eff1c19d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:06.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.587+0000 7eff20bff700 1 --2- 172.21.15.64:0/979424839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff1c1039f0 0x7eff1c196940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:06.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.587+0000 7eff20bff700 1 -- 172.21.15.64:0/979424839 >> 172.21.15.64:0/979424839 conn(0x7eff1c0faa30 msgr2=0x7eff1c101d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:06.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.587+0000 7eff20bff700 1 -- 172.21.15.64:0/979424839 shutdown_connections 2023-09-13T03:11:06.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.587+0000 7eff20bff700 1 -- 172.21.15.64:0/979424839 wait complete. 2023-09-13T03:11:06.607 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-13T03:06:58.987479+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-13T03:11:06.608 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-13T03:11:06.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:06 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/2865198699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:11:06.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.830+0000 7f72bdfff700 1 -- 172.21.15.64:0/1167712358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72b8100630 msgr2=0x7f72b8100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:06.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.830+0000 7f72bdfff700 1 --2- 172.21.15.64:0/1167712358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72b8100630 0x7f72b8100a50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f72a8009990 tx=0x7f72a8009ca0 comp rx=0 tx=0).stop 2023-09-13T03:11:06.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.831+0000 7f72bdfff700 1 -- 172.21.15.64:0/1167712358 shutdown_connections 2023-09-13T03:11:06.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.831+0000 7f72bdfff700 1 --2- 172.21.15.64:0/1167712358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72b81021c0 0x7f72b8108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:06.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.831+0000 7f72bdfff700 1 --2- 172.21.15.64:0/1167712358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72b8100630 0x7f72b8100a50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:06.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.831+0000 7f72bdfff700 1 --2- 172.21.15.64:0/1167712358 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72b8101820 0x7f72b8101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:06.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.831+0000 7f72bdfff700 1 -- 172.21.15.64:0/1167712358 >> 172.21.15.64:0/1167712358 conn(0x7f72b80fbb80 msgr2=0x7f72b80fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:06.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.831+0000 7f72bdfff700 1 -- 172.21.15.64:0/1167712358 shutdown_connections 2023-09-13T03:11:06.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.831+0000 7f72bdfff700 1 -- 172.21.15.64:0/1167712358 wait complete. 2023-09-13T03:11:06.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.832+0000 7f72bdfff700 1 Processor -- start 2023-09-13T03:11:06.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.832+0000 7f72bdfff700 1 -- start start 2023-09-13T03:11:06.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.833+0000 7f72bdfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72b8100630 0x7f72b80743b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:06.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.833+0000 7f72bdfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72b8101820 0x7f72b80729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:06.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.833+0000 7f72bdfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72b81021c0 0x7f72b8072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:06.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.833+0000 7f72bdfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b80736d0 con 0x7f72b8101820 2023-09-13T03:11:06.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.833+0000 7f72bdfff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b8073840 con 0x7f72b8100630 2023-09-13T03:11:06.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.833+0000 7f72bdfff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b81a03b0 con 0x7f72b81021c0 2023-09-13T03:11:06.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.833+0000 7f72b7fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72b81021c0 0x7f72b8072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:06.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.833+0000 7f72b7fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72b81021c0 0x7f72b8072f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:41354/0 (socket says 172.21.15.64:41354) 2023-09-13T03:11:06.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.833+0000 7f72b7fff700 1 -- 172.21.15.64:0/2070346162 learned_addr learned my addr 172.21.15.64:0/2070346162 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:06.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.833+0000 7f72b77fe700 1 --2- 172.21.15.64:0/2070346162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72b8100630 0x7f72b80743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:06.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.833+0000 7f72b6ffd700 1 --2- 172.21.15.64:0/2070346162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72b8101820 0x7f72b80729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:06.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.833+0000 7f72b7fff700 1 -- 172.21.15.64:0/2070346162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72b8100630 msgr2=0x7f72b80743b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:06.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.833+0000 7f72b7fff700 1 --2- 172.21.15.64:0/2070346162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72b8100630 0x7f72b80743b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:06.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.833+0000 7f72b7fff700 1 -- 172.21.15.64:0/2070346162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72b8101820 msgr2=0x7f72b80729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:06.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.833+0000 7f72b7fff700 1 --2- 172.21.15.64:0/2070346162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72b8101820 0x7f72b80729f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:06.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.833+0000 7f72b7fff700 1 -- 172.21.15.64:0/2070346162 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a80095f0 con 0x7f72b81021c0 2023-09-13T03:11:06.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.834+0000 7f72b7fff700 1 --2- 172.21.15.64:0/2070346162 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72b81021c0 0x7f72b8072f30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f72b0007ee0 tx=0x7f72b000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:06.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.834+0000 7f72b4ff9700 1 -- 172.21.15.64:0/2070346162 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b0017070 con 0x7f72b81021c0 2023-09-13T03:11:06.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.834+0000 7f72b4ff9700 1 -- 172.21.15.64:0/2070346162 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72b000f040 con 0x7f72b81021c0 2023-09-13T03:11:06.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.834+0000 7f72b4ff9700 1 -- 172.21.15.64:0/2070346162 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b0005580 con 0x7f72b81021c0 2023-09-13T03:11:06.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.834+0000 7f72bdfff700 1 -- 172.21.15.64:0/2070346162 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72b81a0690 con 0x7f72b81021c0 2023-09-13T03:11:06.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.834+0000 7f72bdfff700 1 -- 172.21.15.64:0/2070346162 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72b81a0b50 con 0x7f72b81021c0 2023-09-13T03:11:06.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.835+0000 7f72bdfff700 1 -- 172.21.15.64:0/2070346162 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72b810be90 con 0x7f72b81021c0 2023-09-13T03:11:06.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.837+0000 7f72b4ff9700 1 -- 172.21.15.64:0/2070346162 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f72b000db20 con 0x7f72b81021c0 2023-09-13T03:11:06.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.839+0000 7f72b4ff9700 1 --2- 172.21.15.64:0/2070346162 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f72a00a06c0 0x7f72a00a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:06.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.840+0000 7f72b4ff9700 1 -- 172.21.15.64:0/2070346162 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f72b00caec0 con 0x7f72b81021c0 2023-09-13T03:11:06.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.840+0000 7f72b77fe700 1 --2- 172.21.15.64:0/2070346162 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f72a00a06c0 0x7f72a00a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:06.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.840+0000 7f72b77fe700 1 --2- 172.21.15.64:0/2070346162 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f72a00a06c0 0x7f72a00a2b80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f72a800b2e0 tx=0x7f72a8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:06.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:06.842+0000 7f72b4ff9700 1 -- 172.21.15.64:0/2070346162 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f72b0094df0 con 0x7f72b81021c0 2023-09-13T03:11:07.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.098+0000 7f72bdfff700 1 -- 172.21.15.64:0/2070346162 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f72b81a0d70 con 0x7f72b81021c0 2023-09-13T03:11:07.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.098+0000 7f72b4ff9700 1 -- 172.21.15.64:0/2070346162 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f72b0098410 con 0x7f72b81021c0 2023-09-13T03:11:07.099 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-09-13T03:11:07.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.101+0000 7f72bdfff700 1 -- 172.21.15.64:0/2070346162 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f72a00a06c0 msgr2=0x7f72a00a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:07.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.101+0000 7f72bdfff700 1 --2- 172.21.15.64:0/2070346162 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f72a00a06c0 0x7f72a00a2b80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f72a800b2e0 tx=0x7f72a8005fb0 comp rx=0 tx=0).stop 2023-09-13T03:11:07.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.101+0000 7f72bdfff700 1 -- 172.21.15.64:0/2070346162 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72b81021c0 msgr2=0x7f72b8072f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:07.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.101+0000 7f72bdfff700 1 --2- 172.21.15.64:0/2070346162 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72b81021c0 0x7f72b8072f30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f72b0007ee0 tx=0x7f72b000d3b0 comp rx=0 tx=0).stop 2023-09-13T03:11:07.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.101+0000 7f72bdfff700 1 -- 172.21.15.64:0/2070346162 shutdown_connections 2023-09-13T03:11:07.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.101+0000 7f72bdfff700 1 --2- 172.21.15.64:0/2070346162 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f72a00a06c0 0x7f72a00a2b80 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:07.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.101+0000 7f72bdfff700 1 --2- 172.21.15.64:0/2070346162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72b8100630 0x7f72b80743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:07.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.101+0000 7f72bdfff700 1 --2- 172.21.15.64:0/2070346162 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72b81021c0 0x7f72b8072f30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:07.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.101+0000 7f72bdfff700 1 --2- 172.21.15.64:0/2070346162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72b8101820 0x7f72b80729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:07.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.101+0000 7f72bdfff700 1 -- 172.21.15.64:0/2070346162 >> 172.21.15.64:0/2070346162 conn(0x7f72b80fbb80 msgr2=0x7f72b80fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:07.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.101+0000 7f72bdfff700 1 -- 172.21.15.64:0/2070346162 shutdown_connections 2023-09-13T03:11:07.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.101+0000 7f72bdfff700 1 -- 172.21.15.64:0/2070346162 wait complete. 2023-09-13T03:11:07.121 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-09-13T03:11:07.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.353+0000 7fd8003ff700 1 -- 172.21.15.64:0/4294372077 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7f8101800 msgr2=0x7fd7f8101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:07.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.353+0000 7fd8003ff700 1 --2- 172.21.15.64:0/4294372077 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7f8101800 0x7fd7f8101c60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd7e80099c0 tx=0x7fd7e8009cd0 comp rx=0 tx=0).stop 2023-09-13T03:11:07.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.353+0000 7fd8003ff700 1 -- 172.21.15.64:0/4294372077 shutdown_connections 2023-09-13T03:11:07.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.353+0000 7fd8003ff700 1 --2- 172.21.15.64:0/4294372077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7f8100610 0x7fd7f8100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:07.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.353+0000 7fd8003ff700 1 --2- 172.21.15.64:0/4294372077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7f81021a0 0x7fd7f8108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:07.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.353+0000 7fd8003ff700 1 --2- 172.21.15.64:0/4294372077 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7f8101800 0x7fd7f8101c60 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:07.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.353+0000 7fd8003ff700 1 -- 172.21.15.64:0/4294372077 >> 172.21.15.64:0/4294372077 conn(0x7fd7f80fbb40 msgr2=0x7fd7f80fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:07.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.354+0000 7fd8003ff700 1 -- 172.21.15.64:0/4294372077 shutdown_connections 2023-09-13T03:11:07.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.354+0000 7fd8003ff700 1 -- 172.21.15.64:0/4294372077 wait complete. 2023-09-13T03:11:07.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.355+0000 7fd8003ff700 1 Processor -- start 2023-09-13T03:11:07.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.355+0000 7fd8003ff700 1 -- start start 2023-09-13T03:11:07.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.355+0000 7fd8003ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7f8100610 0x7fd7f81984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:07.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.355+0000 7fd8003ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7f8101800 0x7fd7f8198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:07.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.355+0000 7fd8003ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7f81021a0 0x7fd7f819fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:07.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.355+0000 7fd8003ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f81a0040 con 0x7fd7f8100610 2023-09-13T03:11:07.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.355+0000 7fd8003ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f81a0180 con 0x7fd7f81021a0 2023-09-13T03:11:07.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.355+0000 7fd8003ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f81a0310 con 0x7fd7f8101800 2023-09-13T03:11:07.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.355+0000 7fd7fe19b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7f8100610 0x7fd7f81984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:07.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.355+0000 7fd7fe99c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7f81021a0 0x7fd7f819fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:07.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.355+0000 7fd7fe19b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7f8100610 0x7fd7f81984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57480/0 (socket says 172.21.15.64:57480) 2023-09-13T03:11:07.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.355+0000 7fd7fe19b700 1 -- 172.21.15.64:0/3436238139 learned_addr learned my addr 172.21.15.64:0/3436238139 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:07.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.355+0000 7fd7fd99a700 1 --2- 172.21.15.64:0/3436238139 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7f8101800 0x7fd7f8198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:07.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.356+0000 7fd7fe99c700 1 -- 172.21.15.64:0/3436238139 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7f8101800 msgr2=0x7fd7f8198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:07.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.356+0000 7fd7fe99c700 1 --2- 172.21.15.64:0/3436238139 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7f8101800 0x7fd7f8198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:07.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.356+0000 7fd7fe99c700 1 -- 172.21.15.64:0/3436238139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7f8100610 msgr2=0x7fd7f81984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:07.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.356+0000 7fd7fe99c700 1 --2- 172.21.15.64:0/3436238139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7f8100610 0x7fd7f81984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:07.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.356+0000 7fd7fe99c700 1 -- 172.21.15.64:0/3436238139 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e80095f0 con 0x7fd7f81021a0 2023-09-13T03:11:07.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.356+0000 7fd7fe19b700 1 --2- 172.21.15.64:0/3436238139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7f8100610 0x7fd7f81984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:11:07.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.356+0000 7fd7fe99c700 1 --2- 172.21.15.64:0/3436238139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7f81021a0 0x7fd7f819fa70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0007f10 tx=0x7fd7f000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:07.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.356+0000 7fd7ef7fe700 1 -- 172.21.15.64:0/3436238139 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f0004db0 con 0x7fd7f81021a0 2023-09-13T03:11:07.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.356+0000 7fd7ef7fe700 1 -- 172.21.15.64:0/3436238139 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7f000f040 con 0x7fd7f81021a0 2023-09-13T03:11:07.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.356+0000 7fd8003ff700 1 -- 172.21.15.64:0/3436238139 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7f81a05f0 con 0x7fd7f81021a0 2023-09-13T03:11:07.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.356+0000 7fd7ef7fe700 1 -- 172.21.15.64:0/3436238139 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f0005990 con 0x7fd7f81021a0 2023-09-13T03:11:07.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.356+0000 7fd8003ff700 1 -- 172.21.15.64:0/3436238139 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f81a0b50 con 0x7fd7f81021a0 2023-09-13T03:11:07.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.357+0000 7fd8003ff700 1 -- 172.21.15.64:0/3436238139 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7f810be70 con 0x7fd7f81021a0 2023-09-13T03:11:07.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.359+0000 7fd7ef7fe700 1 -- 172.21.15.64:0/3436238139 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fd7f000db20 con 0x7fd7f81021a0 2023-09-13T03:11:07.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.361+0000 7fd7ef7fe700 1 --2- 172.21.15.64:0/3436238139 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd7e40a0710 0x7fd7e40a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:07.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.361+0000 7fd7ef7fe700 1 -- 172.21.15.64:0/3436238139 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd7f00cb7e0 con 0x7fd7f81021a0 2023-09-13T03:11:07.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.361+0000 7fd7fe19b700 1 --2- 172.21.15.64:0/3436238139 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd7e40a0710 0x7fd7e40a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:07.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.361+0000 7fd7fe19b700 1 --2- 172.21.15.64:0/3436238139 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd7e40a0710 0x7fd7e40a2bd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4007fd0 tx=0x7fd7f4007480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:07.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.363+0000 7fd7ef7fe700 1 -- 172.21.15.64:0/3436238139 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fd7f00957b0 con 0x7fd7f81021a0 2023-09-13T03:11:07.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:07 smithi064 ceph-mon[97675]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:11:07.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:07 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/979424839' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:11:07.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:07 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:07.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:07 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:07.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:07 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:07.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:07 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:07.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:07 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2070346162' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T03:11:07.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:07 smithi142 ceph-mon[134082]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:11:07.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:07 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/979424839' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:11:07.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:07 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:07.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:07 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:07.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:07 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:07.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:07 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:07.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:07 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/2070346162' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T03:11:07.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.608+0000 7fd8003ff700 1 -- 172.21.15.64:0/3436238139 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd7f810c060 con 0x7fd7f81021a0 2023-09-13T03:11:07.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.608+0000 7fd7ef7fe700 1 -- 172.21.15.64:0/3436238139 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) v1 ==== 84+0+65 (secure 0 0 0) 0x7fd7f0098dd0 con 0x7fd7f81021a0 2023-09-13T03:11:07.609 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:11:07.609 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2023-09-13T03:11:07.610 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:11:07.610 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 1, 2023-09-13T03:11:07.610 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": ".mgr" 2023-09-13T03:11:07.610 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:11:07.611 INFO:teuthology.orchestra.run.smithi064.stdout:] 2023-09-13T03:11:07.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.611+0000 7fd8003ff700 1 -- 172.21.15.64:0/3436238139 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd7e40a0710 msgr2=0x7fd7e40a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:07.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.611+0000 7fd8003ff700 1 --2- 172.21.15.64:0/3436238139 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd7e40a0710 0x7fd7e40a2bd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4007fd0 tx=0x7fd7f4007480 comp rx=0 tx=0).stop 2023-09-13T03:11:07.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.611+0000 7fd8003ff700 1 -- 172.21.15.64:0/3436238139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7f81021a0 msgr2=0x7fd7f819fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:07.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.611+0000 7fd8003ff700 1 --2- 172.21.15.64:0/3436238139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7f81021a0 0x7fd7f819fa70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0007f10 tx=0x7fd7f000d3b0 comp rx=0 tx=0).stop 2023-09-13T03:11:07.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.611+0000 7fd8003ff700 1 -- 172.21.15.64:0/3436238139 shutdown_connections 2023-09-13T03:11:07.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.611+0000 7fd8003ff700 1 --2- 172.21.15.64:0/3436238139 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd7e40a0710 0x7fd7e40a2bd0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:07.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.611+0000 7fd8003ff700 1 --2- 172.21.15.64:0/3436238139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7f81021a0 0x7fd7f819fa70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:07.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.611+0000 7fd8003ff700 1 --2- 172.21.15.64:0/3436238139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7f8100610 0x7fd7f81984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:07.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.611+0000 7fd8003ff700 1 --2- 172.21.15.64:0/3436238139 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7f8101800 0x7fd7f8198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:07.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.611+0000 7fd8003ff700 1 -- 172.21.15.64:0/3436238139 >> 172.21.15.64:0/3436238139 conn(0x7fd7f80fbb40 msgr2=0x7fd7f80fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:07.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.611+0000 7fd8003ff700 1 -- 172.21.15.64:0/3436238139 shutdown_connections 2023-09-13T03:11:07.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.611+0000 7fd8003ff700 1 -- 172.21.15.64:0/3436238139 wait complete. 2023-09-13T03:11:07.630 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-09-13T03:11:07.630 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-09-13T03:11:07.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:07 smithi170 ceph-mon[134328]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:11:07.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:07 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/979424839' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:11:07.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:07 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:07.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:07 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:07.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:07 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:07.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:07 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:07.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:07 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/2070346162' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T03:11:07.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.843+0000 7f81eabff700 1 -- 172.21.15.64:0/3482525209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81e41005f0 msgr2=0x7f81e4100a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:07.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.843+0000 7f81eabff700 1 --2- 172.21.15.64:0/3482525209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81e41005f0 0x7f81e4100a10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f81d80099c0 tx=0x7f81d8009cd0 comp rx=0 tx=0).stop 2023-09-13T03:11:07.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.843+0000 7f81eabff700 1 -- 172.21.15.64:0/3482525209 shutdown_connections 2023-09-13T03:11:07.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.843+0000 7f81eabff700 1 --2- 172.21.15.64:0/3482525209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81e41005f0 0x7f81e4100a10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:07.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.843+0000 7f81eabff700 1 --2- 172.21.15.64:0/3482525209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81e4102180 0x7f81e4108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:07.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.843+0000 7f81eabff700 1 --2- 172.21.15.64:0/3482525209 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81e41017e0 0x7f81e4101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:07.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.843+0000 7f81eabff700 1 -- 172.21.15.64:0/3482525209 >> 172.21.15.64:0/3482525209 conn(0x7f81e40fbb40 msgr2=0x7f81e40fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:07.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.843+0000 7f81eabff700 1 -- 172.21.15.64:0/3482525209 shutdown_connections 2023-09-13T03:11:07.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.843+0000 7f81eabff700 1 -- 172.21.15.64:0/3482525209 wait complete. 2023-09-13T03:11:07.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.844+0000 7f81eabff700 1 Processor -- start 2023-09-13T03:11:07.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.844+0000 7f81eabff700 1 -- start start 2023-09-13T03:11:07.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.845+0000 7f81eabff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81e41005f0 0x7f81e4198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:07.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.845+0000 7f81eabff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81e41017e0 0x7f81e4198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:07.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.845+0000 7f81eabff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81e4102180 0x7f81e419fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:07.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.845+0000 7f81eabff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e41a00d0 con 0x7f81e41005f0 2023-09-13T03:11:07.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.845+0000 7f81eabff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e41a0210 con 0x7f81e4102180 2023-09-13T03:11:07.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.845+0000 7f81eabff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e41a03a0 con 0x7f81e41017e0 2023-09-13T03:11:07.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.845+0000 7f81e899b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81e41005f0 0x7f81e4198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:07.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.845+0000 7f81e899b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81e41005f0 0x7f81e4198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57494/0 (socket says 172.21.15.64:57494) 2023-09-13T03:11:07.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.845+0000 7f81e899b700 1 -- 172.21.15.64:0/2617493779 learned_addr learned my addr 172.21.15.64:0/2617493779 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:07.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.845+0000 7f81e919c700 1 --2- 172.21.15.64:0/2617493779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81e4102180 0x7f81e419fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:07.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.845+0000 7f81e3fff700 1 --2- 172.21.15.64:0/2617493779 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81e41017e0 0x7f81e4198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:07.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.845+0000 7f81e899b700 1 -- 172.21.15.64:0/2617493779 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81e41017e0 msgr2=0x7f81e4198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:07.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.845+0000 7f81e899b700 1 --2- 172.21.15.64:0/2617493779 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81e41017e0 0x7f81e4198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:07.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.846+0000 7f81e899b700 1 -- 172.21.15.64:0/2617493779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81e4102180 msgr2=0x7f81e419fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:07.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.846+0000 7f81e899b700 1 --2- 172.21.15.64:0/2617493779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81e4102180 0x7f81e419fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:07.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.846+0000 7f81e899b700 1 -- 172.21.15.64:0/2617493779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d80095f0 con 0x7f81e41005f0 2023-09-13T03:11:07.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.846+0000 7f81e919c700 1 --2- 172.21.15.64:0/2617493779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81e4102180 0x7f81e419fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:11:07.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.846+0000 7f81e899b700 1 --2- 172.21.15.64:0/2617493779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81e41005f0 0x7f81e4198550 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f81d8009990 tx=0x7f81d800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:07.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.846+0000 7f81e1ffb700 1 -- 172.21.15.64:0/2617493779 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d801d070 con 0x7f81e41005f0 2023-09-13T03:11:07.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.846+0000 7f81e1ffb700 1 -- 172.21.15.64:0/2617493779 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81d8005030 con 0x7f81e41005f0 2023-09-13T03:11:07.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.846+0000 7f81eabff700 1 -- 172.21.15.64:0/2617493779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81e41a0620 con 0x7f81e41005f0 2023-09-13T03:11:07.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.846+0000 7f81e1ffb700 1 -- 172.21.15.64:0/2617493779 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d8004a70 con 0x7f81e41005f0 2023-09-13T03:11:07.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.846+0000 7f81eabff700 1 -- 172.21.15.64:0/2617493779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81e41a0aa0 con 0x7f81e41005f0 2023-09-13T03:11:07.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.847+0000 7f81eabff700 1 -- 172.21.15.64:0/2617493779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81e410be60 con 0x7f81e41005f0 2023-09-13T03:11:07.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.850+0000 7f81e1ffb700 1 -- 172.21.15.64:0/2617493779 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f81d8005410 con 0x7f81e41005f0 2023-09-13T03:11:07.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.852+0000 7f81e1ffb700 1 --2- 172.21.15.64:0/2617493779 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f81d40a06c0 0x7f81d40a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:07.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.852+0000 7f81e1ffb700 1 -- 172.21.15.64:0/2617493779 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f81d80cc870 con 0x7f81e41005f0 2023-09-13T03:11:07.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.852+0000 7f81e3fff700 1 --2- 172.21.15.64:0/2617493779 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f81d40a06c0 0x7f81d40a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:07.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.853+0000 7f81e3fff700 1 --2- 172.21.15.64:0/2617493779 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f81d40a06c0 0x7f81d40a2b80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f81d0009c60 tx=0x7f81d0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:07.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:07.854+0000 7f81e1ffb700 1 -- 172.21.15.64:0/2617493779 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f81d8096830 con 0x7f81e41005f0 2023-09-13T03:11:08.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:08.078+0000 7f81eabff700 1 -- 172.21.15.64:0/2617493779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f81e410c070 con 0x7f81e41005f0 2023-09-13T03:11:08.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:08.903+0000 7f81e1ffb700 1 -- 172.21.15.64:0/2617493779 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) v1 ==== 138+0+0 (secure 0 0 0) 0x7f81d8099e50 con 0x7f81e41005f0 2023-09-13T03:11:08.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:08.909+0000 7f81eabff700 1 -- 172.21.15.64:0/2617493779 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f81d40a06c0 msgr2=0x7f81d40a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:08.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:08.909+0000 7f81eabff700 1 --2- 172.21.15.64:0/2617493779 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f81d40a06c0 0x7f81d40a2b80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f81d0009c60 tx=0x7f81d0009260 comp rx=0 tx=0).stop 2023-09-13T03:11:08.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:08.909+0000 7f81eabff700 1 -- 172.21.15.64:0/2617493779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81e41005f0 msgr2=0x7f81e4198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:08.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:08.909+0000 7f81eabff700 1 --2- 172.21.15.64:0/2617493779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81e41005f0 0x7f81e4198550 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f81d8009990 tx=0x7f81d800f690 comp rx=0 tx=0).stop 2023-09-13T03:11:08.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:08 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:08.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:08 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/3436238139' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-09-13T03:11:08.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:08 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:08.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:08 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:08.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:08 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:11:08.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:08 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:11:08.913 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:08 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:08.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:08.910+0000 7f81eabff700 1 -- 172.21.15.64:0/2617493779 shutdown_connections 2023-09-13T03:11:08.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:08.910+0000 7f81eabff700 1 --2- 172.21.15.64:0/2617493779 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f81d40a06c0 0x7f81d40a2b80 secure :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f81d0009c60 tx=0x7f81d0009260 comp rx=0 tx=0).stop 2023-09-13T03:11:08.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:08.910+0000 7f81eabff700 1 --2- 172.21.15.64:0/2617493779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81e4102180 0x7f81e419fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:08.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:08.910+0000 7f81eabff700 1 --2- 172.21.15.64:0/2617493779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81e41005f0 0x7f81e4198550 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:08.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:08.910+0000 7f81eabff700 1 --2- 172.21.15.64:0/2617493779 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81e41017e0 0x7f81e4198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:08.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:08.910+0000 7f81eabff700 1 -- 172.21.15.64:0/2617493779 >> 172.21.15.64:0/2617493779 conn(0x7f81e40fbb40 msgr2=0x7f81e40fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:08.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:08.912+0000 7f81eabff700 1 -- 172.21.15.64:0/2617493779 shutdown_connections 2023-09-13T03:11:08.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:08.912+0000 7f81eabff700 1 -- 172.21.15.64:0/2617493779 wait complete. 2023-09-13T03:11:08.915 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' created 2023-09-13T03:11:08.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:08 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:08.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:08 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/3436238139' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-09-13T03:11:08.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:08 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:08.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:08 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:08.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:08 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:11:08.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:08 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:11:08.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:08 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:08.932 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-09-13T03:11:08.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:08 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:08.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:08 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/3436238139' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-09-13T03:11:08.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:08 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:08.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:08 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:08.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:08 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:11:08.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:08 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:11:08.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:08 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:09.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.153+0000 7f5ab4bff700 1 -- 172.21.15.64:0/2148172269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ab01021a0 msgr2=0x7f5ab0108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:09.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.153+0000 7f5ab4bff700 1 --2- 172.21.15.64:0/2148172269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ab01021a0 0x7f5ab0108a50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5aa800b0d0 tx=0x7f5aa800b3e0 comp rx=0 tx=0).stop 2023-09-13T03:11:09.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.153+0000 7f5ab4bff700 1 -- 172.21.15.64:0/2148172269 shutdown_connections 2023-09-13T03:11:09.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.153+0000 7f5ab4bff700 1 --2- 172.21.15.64:0/2148172269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ab01021a0 0x7f5ab0108a50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:09.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.153+0000 7f5ab4bff700 1 --2- 172.21.15.64:0/2148172269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ab0100610 0x7f5ab0100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:09.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.153+0000 7f5ab4bff700 1 --2- 172.21.15.64:0/2148172269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ab0101800 0x7f5ab0101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:09.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.153+0000 7f5ab4bff700 1 -- 172.21.15.64:0/2148172269 >> 172.21.15.64:0/2148172269 conn(0x7f5ab00fbb80 msgr2=0x7f5ab00fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:09.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.153+0000 7f5ab4bff700 1 -- 172.21.15.64:0/2148172269 shutdown_connections 2023-09-13T03:11:09.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.153+0000 7f5ab4bff700 1 -- 172.21.15.64:0/2148172269 wait complete. 2023-09-13T03:11:09.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.154+0000 7f5ab4bff700 1 Processor -- start 2023-09-13T03:11:09.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.154+0000 7f5ab4bff700 1 -- start start 2023-09-13T03:11:09.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.154+0000 7f5ab4bff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ab0100610 0x7f5ab0196400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:09.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.154+0000 7f5ab4bff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ab0101800 0x7f5ab0196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.154+0000 7f5ab4bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ab01021a0 0x7f5ab019d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.154+0000 7f5ab4bff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab019df80 con 0x7f5ab0101800 2023-09-13T03:11:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.154+0000 7f5ab4bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab019e0c0 con 0x7f5ab01021a0 2023-09-13T03:11:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.154+0000 7f5ab4bff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab019e250 con 0x7f5ab0100610 2023-09-13T03:11:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.155+0000 7f5aadd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ab0101800 0x7f5ab0196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:09.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.155+0000 7f5aadd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ab0101800 0x7f5ab0196940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57516/0 (socket says 172.21.15.64:57516) 2023-09-13T03:11:09.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.155+0000 7f5aadd9b700 1 -- 172.21.15.64:0/293178864 learned_addr learned my addr 172.21.15.64:0/293178864 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:09.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.155+0000 7f5aae59c700 1 --2- 172.21.15.64:0/293178864 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ab0100610 0x7f5ab0196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:09.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.155+0000 7f5aaed9d700 1 --2- 172.21.15.64:0/293178864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ab01021a0 0x7f5ab019d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:09.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.155+0000 7f5aadd9b700 1 -- 172.21.15.64:0/293178864 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ab0100610 msgr2=0x7f5ab0196400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:09.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.155+0000 7f5aadd9b700 1 --2- 172.21.15.64:0/293178864 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ab0100610 0x7f5ab0196400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:09.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.155+0000 7f5aadd9b700 1 -- 172.21.15.64:0/293178864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ab01021a0 msgr2=0x7f5ab019d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:09.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.155+0000 7f5aadd9b700 1 --2- 172.21.15.64:0/293178864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ab01021a0 0x7f5ab019d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:09.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.155+0000 7f5aadd9b700 1 -- 172.21.15.64:0/293178864 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa8009d00 con 0x7f5ab0101800 2023-09-13T03:11:09.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.155+0000 7f5aadd9b700 1 --2- 172.21.15.64:0/293178864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ab0101800 0x7f5ab0196940 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5aa400d980 tx=0x7f5aa400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:09.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.155+0000 7f5a9f7fe700 1 -- 172.21.15.64:0/293178864 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa4004220 con 0x7f5ab0101800 2023-09-13T03:11:09.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.155+0000 7f5a9f7fe700 1 -- 172.21.15.64:0/293178864 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5aa4007640 con 0x7f5ab0101800 2023-09-13T03:11:09.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.156+0000 7f5ab4bff700 1 -- 172.21.15.64:0/293178864 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ab019e530 con 0x7f5ab0101800 2023-09-13T03:11:09.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.156+0000 7f5a9f7fe700 1 -- 172.21.15.64:0/293178864 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa4005690 con 0x7f5ab0101800 2023-09-13T03:11:09.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.156+0000 7f5ab4bff700 1 -- 172.21.15.64:0/293178864 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ab019ea40 con 0x7f5ab0101800 2023-09-13T03:11:09.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.157+0000 7f5ab4bff700 1 -- 172.21.15.64:0/293178864 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ab010be80 con 0x7f5ab0101800 2023-09-13T03:11:09.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.159+0000 7f5a9f7fe700 1 -- 172.21.15.64:0/293178864 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f5aa4004750 con 0x7f5ab0101800 2023-09-13T03:11:09.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.160+0000 7f5a9f7fe700 1 --2- 172.21.15.64:0/293178864 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f5a980a0640 0x7f5a980a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:09.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.160+0000 7f5a9f7fe700 1 -- 172.21.15.64:0/293178864 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f5aa40cbc10 con 0x7f5ab0101800 2023-09-13T03:11:09.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.160+0000 7f5aae59c700 1 --2- 172.21.15.64:0/293178864 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f5a980a0640 0x7f5a980a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:09.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.161+0000 7f5aae59c700 1 --2- 172.21.15.64:0/293178864 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f5a980a0640 0x7f5a980a2b00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0101660 tx=0x7f5aa0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:09.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.163+0000 7f5a9f7fe700 1 -- 172.21.15.64:0/293178864 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5aa40959d0 con 0x7f5ab0101800 2023-09-13T03:11:09.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.379+0000 7f5ab4bff700 1 -- 172.21.15.64:0/293178864 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5ab0107470 con 0x7f5ab0101800 2023-09-13T03:11:09.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.931+0000 7f5a9f7fe700 1 -- 172.21.15.64:0/293178864 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v81) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5aa4098ff0 con 0x7f5ab0101800 2023-09-13T03:11:09.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.935+0000 7f5ab4bff700 1 -- 172.21.15.64:0/293178864 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f5a980a0640 msgr2=0x7f5a980a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:09.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.935+0000 7f5ab4bff700 1 --2- 172.21.15.64:0/293178864 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f5a980a0640 0x7f5a980a2b00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0101660 tx=0x7f5aa0009260 comp rx=0 tx=0).stop 2023-09-13T03:11:09.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.935+0000 7f5ab4bff700 1 -- 172.21.15.64:0/293178864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ab0101800 msgr2=0x7f5ab0196940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:09.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.935+0000 7f5ab4bff700 1 --2- 172.21.15.64:0/293178864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ab0101800 0x7f5ab0196940 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5aa400d980 tx=0x7f5aa400dc90 comp rx=0 tx=0).stop 2023-09-13T03:11:09.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.940+0000 7f5ab4bff700 1 -- 172.21.15.64:0/293178864 shutdown_connections 2023-09-13T03:11:09.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.940+0000 7f5ab4bff700 1 --2- 172.21.15.64:0/293178864 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f5a980a0640 0x7f5a980a2b00 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:09.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.940+0000 7f5ab4bff700 1 --2- 172.21.15.64:0/293178864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ab01021a0 0x7f5ab019d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:09.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.940+0000 7f5ab4bff700 1 --2- 172.21.15.64:0/293178864 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ab0100610 0x7f5ab0196400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:09.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.940+0000 7f5ab4bff700 1 --2- 172.21.15.64:0/293178864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ab0101800 0x7f5ab0196940 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:09.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.940+0000 7f5ab4bff700 1 -- 172.21.15.64:0/293178864 >> 172.21.15.64:0/293178864 conn(0x7f5ab00fbb80 msgr2=0x7f5ab00fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:09.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.944+0000 7f5ab4bff700 1 -- 172.21.15.64:0/293178864 shutdown_connections 2023-09-13T03:11:09.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:09.944+0000 7f5ab4bff700 1 -- 172.21.15.64:0/293178864 wait complete. 2023-09-13T03:11:09.954 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' created 2023-09-13T03:11:09.975 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-09-13T03:11:10.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.202+0000 7f5b5b1ff700 1 -- 172.21.15.64:0/3084380404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b54100620 msgr2=0x7f5b54100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:10.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.202+0000 7f5b5b1ff700 1 --2- 172.21.15.64:0/3084380404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b54100620 0x7f5b54100a40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5b480099c0 tx=0x7f5b48009cd0 comp rx=0 tx=0).stop 2023-09-13T03:11:10.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.202+0000 7f5b5b1ff700 1 -- 172.21.15.64:0/3084380404 shutdown_connections 2023-09-13T03:11:10.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.202+0000 7f5b5b1ff700 1 --2- 172.21.15.64:0/3084380404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b54100620 0x7f5b54100a40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:10.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.202+0000 7f5b5b1ff700 1 --2- 172.21.15.64:0/3084380404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b541021b0 0x7f5b54108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:10.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.202+0000 7f5b5b1ff700 1 --2- 172.21.15.64:0/3084380404 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b54101810 0x7f5b54101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:10.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.202+0000 7f5b5b1ff700 1 -- 172.21.15.64:0/3084380404 >> 172.21.15.64:0/3084380404 conn(0x7f5b540fbbb0 msgr2=0x7f5b540fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:10.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.202+0000 7f5b5b1ff700 1 -- 172.21.15.64:0/3084380404 shutdown_connections 2023-09-13T03:11:10.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.202+0000 7f5b5b1ff700 1 -- 172.21.15.64:0/3084380404 wait complete. 2023-09-13T03:11:10.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.203+0000 7f5b5b1ff700 1 Processor -- start 2023-09-13T03:11:10.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.203+0000 7f5b5b1ff700 1 -- start start 2023-09-13T03:11:10.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.204+0000 7f5b5b1ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b54100620 0x7f5b54198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:10.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.204+0000 7f5b5b1ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b54101810 0x7f5b54198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:10.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.204+0000 7f5b5b1ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b541021b0 0x7f5b5419fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:10.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.204+0000 7f5b5b1ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b541a00d0 con 0x7f5b54100620 2023-09-13T03:11:10.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.204+0000 7f5b5b1ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b541a0210 con 0x7f5b54101810 2023-09-13T03:11:10.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.204+0000 7f5b5b1ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b541a03a0 con 0x7f5b541021b0 2023-09-13T03:11:10.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.204+0000 7f5b58f9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b54100620 0x7f5b54198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:10.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.204+0000 7f5b53fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b54101810 0x7f5b54198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:10.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.204+0000 7f5b58f9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b54100620 0x7f5b54198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57534/0 (socket says 172.21.15.64:57534) 2023-09-13T03:11:10.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.204+0000 7f5b58f9b700 1 -- 172.21.15.64:0/1438064125 learned_addr learned my addr 172.21.15.64:0/1438064125 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:10.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.204+0000 7f5b5979c700 1 --2- 172.21.15.64:0/1438064125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b541021b0 0x7f5b5419fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:10.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.204+0000 7f5b58f9b700 1 -- 172.21.15.64:0/1438064125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b541021b0 msgr2=0x7f5b5419fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:10.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.204+0000 7f5b58f9b700 1 --2- 172.21.15.64:0/1438064125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b541021b0 0x7f5b5419fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:10.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.204+0000 7f5b58f9b700 1 -- 172.21.15.64:0/1438064125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b54101810 msgr2=0x7f5b54198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:10.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.204+0000 7f5b58f9b700 1 --2- 172.21.15.64:0/1438064125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b54101810 0x7f5b54198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:10.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.204+0000 7f5b58f9b700 1 -- 172.21.15.64:0/1438064125 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b480095f0 con 0x7f5b54100620 2023-09-13T03:11:10.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.204+0000 7f5b53fff700 1 --2- 172.21.15.64:0/1438064125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b54101810 0x7f5b54198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:11:10.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.205+0000 7f5b58f9b700 1 --2- 172.21.15.64:0/1438064125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b54100620 0x7f5b54198550 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5b48009fd0 tx=0x7f5b4800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:10.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.205+0000 7f5b51ffb700 1 -- 172.21.15.64:0/1438064125 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4801d070 con 0x7f5b54100620 2023-09-13T03:11:10.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.205+0000 7f5b5b1ff700 1 -- 172.21.15.64:0/1438064125 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b541a0620 con 0x7f5b54100620 2023-09-13T03:11:10.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.205+0000 7f5b51ffb700 1 -- 172.21.15.64:0/1438064125 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b48005050 con 0x7f5b54100620 2023-09-13T03:11:10.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.205+0000 7f5b51ffb700 1 -- 172.21.15.64:0/1438064125 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b48004a70 con 0x7f5b54100620 2023-09-13T03:11:10.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.205+0000 7f5b5b1ff700 1 -- 172.21.15.64:0/1438064125 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b541a0aa0 con 0x7f5b54100620 2023-09-13T03:11:10.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.206+0000 7f5b5b1ff700 1 -- 172.21.15.64:0/1438064125 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b54192c80 con 0x7f5b54100620 2023-09-13T03:11:10.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.208+0000 7f5b51ffb700 1 -- 172.21.15.64:0/1438064125 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f5b48005450 con 0x7f5b54100620 2023-09-13T03:11:10.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.210+0000 7f5b51ffb700 1 --2- 172.21.15.64:0/1438064125 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f5b440a0640 0x7f5b440a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:10.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.210+0000 7f5b51ffb700 1 -- 172.21.15.64:0/1438064125 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f5b480ccc70 con 0x7f5b54100620 2023-09-13T03:11:10.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.210+0000 7f5b53fff700 1 --2- 172.21.15.64:0/1438064125 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f5b440a0640 0x7f5b440a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:10.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.211+0000 7f5b53fff700 1 --2- 172.21.15.64:0/1438064125 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f5b440a0640 0x7f5b440a2b00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5b40011490 tx=0x7f5b40009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:10.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.213+0000 7f5b51ffb700 1 -- 172.21.15.64:0/1438064125 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5b48096940 con 0x7f5b54100620 2023-09-13T03:11:10.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:09 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/2617493779' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-09-13T03:11:10.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:09 smithi142 ceph-mon[134082]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:11:10.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:09 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:11:10.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:09 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/2617493779' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-09-13T03:11:10.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:09 smithi142 ceph-mon[134082]: osdmap e80: 12 total, 12 up, 12 in 2023-09-13T03:11:10.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:09 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/2617493779' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-09-13T03:11:10.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:09 smithi170 ceph-mon[134328]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:11:10.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:09 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:11:10.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:09 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/2617493779' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-09-13T03:11:10.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:09 smithi170 ceph-mon[134328]: osdmap e80: 12 total, 12 up, 12 in 2023-09-13T03:11:10.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:09 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2617493779' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-09-13T03:11:10.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:09 smithi064 ceph-mon[97675]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:11:10.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:09 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:11:10.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:09 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2617493779' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-09-13T03:11:10.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:09 smithi064 ceph-mon[97675]: osdmap e80: 12 total, 12 up, 12 in 2023-09-13T03:11:10.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.488+0000 7f5b5b1ff700 1 -- 172.21.15.64:0/1438064125 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f5b5410be90 con 0x7f5b54100620 2023-09-13T03:11:10.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.960+0000 7f5b51ffb700 1 -- 172.21.15.64:0/1438064125 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f5b480a0030 con 0x7f5b54100620 2023-09-13T03:11:10.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.963+0000 7f5b5b1ff700 1 -- 172.21.15.64:0/1438064125 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f5b440a0640 msgr2=0x7f5b440a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:10.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.963+0000 7f5b5b1ff700 1 --2- 172.21.15.64:0/1438064125 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f5b440a0640 0x7f5b440a2b00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5b40011490 tx=0x7f5b40009260 comp rx=0 tx=0).stop 2023-09-13T03:11:10.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.964+0000 7f5b5b1ff700 1 -- 172.21.15.64:0/1438064125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b54100620 msgr2=0x7f5b54198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:10.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.964+0000 7f5b5b1ff700 1 --2- 172.21.15.64:0/1438064125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b54100620 0x7f5b54198550 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5b48009fd0 tx=0x7f5b4800f690 comp rx=0 tx=0).stop 2023-09-13T03:11:10.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.964+0000 7f5b5b1ff700 1 -- 172.21.15.64:0/1438064125 shutdown_connections 2023-09-13T03:11:10.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.964+0000 7f5b5b1ff700 1 --2- 172.21.15.64:0/1438064125 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f5b440a0640 0x7f5b440a2b00 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:10.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.964+0000 7f5b5b1ff700 1 --2- 172.21.15.64:0/1438064125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b541021b0 0x7f5b5419fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:10.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.964+0000 7f5b5b1ff700 1 --2- 172.21.15.64:0/1438064125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b54100620 0x7f5b54198550 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:10.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.964+0000 7f5b5b1ff700 1 --2- 172.21.15.64:0/1438064125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b54101810 0x7f5b54198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:10.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.964+0000 7f5b5b1ff700 1 -- 172.21.15.64:0/1438064125 >> 172.21.15.64:0/1438064125 conn(0x7f5b540fbbb0 msgr2=0x7f5b54107140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:10.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.964+0000 7f5b5b1ff700 1 -- 172.21.15.64:0/1438064125 shutdown_connections 2023-09-13T03:11:10.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:10.964+0000 7f5b5b1ff700 1 -- 172.21.15.64:0/1438064125 wait complete. 2023-09-13T03:11:10.967 INFO:teuthology.orchestra.run.smithi064.stderr:new fs with metadata pool 2 and data pool 3 2023-09-13T03:11:10.989 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-13T03:11:11.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.202+0000 7f906cfff700 1 -- 172.21.15.64:0/2020630098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90681010c0 msgr2=0x7f90681034b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:11.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.202+0000 7f906cfff700 1 --2- 172.21.15.64:0/2020630098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90681010c0 0x7f90681034b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f906000b0a0 tx=0x7f906000b3b0 comp rx=0 tx=0).stop 2023-09-13T03:11:11.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.202+0000 7f906cfff700 1 -- 172.21.15.64:0/2020630098 shutdown_connections 2023-09-13T03:11:11.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.202+0000 7f906cfff700 1 --2- 172.21.15.64:0/2020630098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9068106320 0x7f9068108730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:11.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.202+0000 7f906cfff700 1 --2- 172.21.15.64:0/2020630098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90681010c0 0x7f90681034b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:11.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.202+0000 7f906cfff700 1 --2- 172.21.15.64:0/2020630098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90681039f0 0x7f9068105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:11.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.202+0000 7f906cfff700 1 -- 172.21.15.64:0/2020630098 >> 172.21.15.64:0/2020630098 conn(0x7f90680faa30 msgr2=0x7f90680fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:11.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.202+0000 7f906cfff700 1 -- 172.21.15.64:0/2020630098 shutdown_connections 2023-09-13T03:11:11.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.202+0000 7f906cfff700 1 -- 172.21.15.64:0/2020630098 wait complete. 2023-09-13T03:11:11.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.203+0000 7f906cfff700 1 Processor -- start 2023-09-13T03:11:11.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.203+0000 7f906cfff700 1 -- start start 2023-09-13T03:11:11.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.204+0000 7f906cfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90681010c0 0x7f9068198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:11.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.204+0000 7f906cfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90681039f0 0x7f9068198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:11.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.204+0000 7f906cfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9068106320 0x7f906819fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:11.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.204+0000 7f906cfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90681a01e0 con 0x7f90681039f0 2023-09-13T03:11:11.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.204+0000 7f906cfff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90681a0320 con 0x7f90681010c0 2023-09-13T03:11:11.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.204+0000 7f906cfff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90681a04b0 con 0x7f9068106320 2023-09-13T03:11:11.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.204+0000 7f9065d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90681039f0 0x7f9068198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:11.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.204+0000 7f906659c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90681010c0 0x7f9068198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:11.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.204+0000 7f9066d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9068106320 0x7f906819fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:11.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.204+0000 7f9065d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90681039f0 0x7f9068198ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57548/0 (socket says 172.21.15.64:57548) 2023-09-13T03:11:11.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.204+0000 7f9066d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9068106320 0x7f906819fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:41464/0 (socket says 172.21.15.64:41464) 2023-09-13T03:11:11.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.204+0000 7f906659c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90681010c0 0x7f9068198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:35032/0 (socket says 172.21.15.64:35032) 2023-09-13T03:11:11.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.204+0000 7f9066d9d700 1 -- 172.21.15.64:0/607525593 learned_addr learned my addr 172.21.15.64:0/607525593 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:11.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.204+0000 7f9066d9d700 1 -- 172.21.15.64:0/607525593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90681010c0 msgr2=0x7f9068198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:11.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.204+0000 7f9066d9d700 1 --2- 172.21.15.64:0/607525593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90681010c0 0x7f9068198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:11.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.204+0000 7f9066d9d700 1 -- 172.21.15.64:0/607525593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90681039f0 msgr2=0x7f9068198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:11.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.204+0000 7f9066d9d700 1 --2- 172.21.15.64:0/607525593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90681039f0 0x7f9068198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:11.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.204+0000 7f9066d9d700 1 -- 172.21.15.64:0/607525593 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9060009d00 con 0x7f9068106320 2023-09-13T03:11:11.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.204+0000 7f9065d9b700 1 --2- 172.21.15.64:0/607525593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90681039f0 0x7f9068198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:11:11.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.204+0000 7f906659c700 1 --2- 172.21.15.64:0/607525593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90681010c0 0x7f9068198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:11:11.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.205+0000 7f9066d9d700 1 --2- 172.21.15.64:0/607525593 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9068106320 0x7f906819fc10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f905c00e940 tx=0x7f905c00ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:11.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.205+0000 7f90577fe700 1 -- 172.21.15.64:0/607525593 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905c0097b0 con 0x7f9068106320 2023-09-13T03:11:11.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.205+0000 7f90577fe700 1 -- 172.21.15.64:0/607525593 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f905c005020 con 0x7f9068106320 2023-09-13T03:11:11.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.205+0000 7f906cfff700 1 -- 172.21.15.64:0/607525593 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90681a0790 con 0x7f9068106320 2023-09-13T03:11:11.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.205+0000 7f90577fe700 1 -- 172.21.15.64:0/607525593 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905c00cc40 con 0x7f9068106320 2023-09-13T03:11:11.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.205+0000 7f906cfff700 1 -- 172.21.15.64:0/607525593 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90681a0cf0 con 0x7f9068106320 2023-09-13T03:11:11.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.206+0000 7f906cfff700 1 -- 172.21.15.64:0/607525593 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9068107ec0 con 0x7f9068106320 2023-09-13T03:11:11.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.208+0000 7f90577fe700 1 -- 172.21.15.64:0/607525593 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f905c005400 con 0x7f9068106320 2023-09-13T03:11:11.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.209+0000 7f90577fe700 1 --2- 172.21.15.64:0/607525593 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f90500a0710 0x7f90500a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:11.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.210+0000 7f90577fe700 1 -- 172.21.15.64:0/607525593 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f905c014070 con 0x7f9068106320 2023-09-13T03:11:11.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.210+0000 7f906659c700 1 --2- 172.21.15.64:0/607525593 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f90500a0710 0x7f90500a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:11.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.210+0000 7f906659c700 1 --2- 172.21.15.64:0/607525593 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f90500a0710 0x7f90500a2bd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f906000b780 tx=0x7f9060006210 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:11.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.211+0000 7f90577fe700 1 -- 172.21.15.64:0/607525593 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f905c095930 con 0x7f9068106320 2023-09-13T03:11:11.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:10 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/293178864' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-09-13T03:11:11.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:10 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/293178864' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-09-13T03:11:11.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:10 smithi142 ceph-mon[134082]: osdmap e81: 12 total, 12 up, 12 in 2023-09-13T03:11:11.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:10 smithi142 ceph-mon[134082]: pgmap v275: 97 pgs: 9 creating+peering, 87 unknown, 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:11:11.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:10 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/1438064125' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-09-13T03:11:11.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:10 smithi142 ceph-mon[134082]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-09-13T03:11:11.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:10 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/293178864' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-09-13T03:11:11.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:10 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/293178864' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-09-13T03:11:11.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:10 smithi170 ceph-mon[134328]: osdmap e81: 12 total, 12 up, 12 in 2023-09-13T03:11:11.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:10 smithi170 ceph-mon[134328]: pgmap v275: 97 pgs: 9 creating+peering, 87 unknown, 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:11:11.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:10 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/1438064125' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-09-13T03:11:11.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:10 smithi170 ceph-mon[134328]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-09-13T03:11:11.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:10 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/293178864' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-09-13T03:11:11.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:10 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/293178864' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-09-13T03:11:11.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:10 smithi064 ceph-mon[97675]: osdmap e81: 12 total, 12 up, 12 in 2023-09-13T03:11:11.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:10 smithi064 ceph-mon[97675]: pgmap v275: 97 pgs: 9 creating+peering, 87 unknown, 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:11:11.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:10 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/1438064125' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-09-13T03:11:11.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:10 smithi064 ceph-mon[97675]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-09-13T03:11:11.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.458+0000 7f906cfff700 1 -- 172.21.15.64:0/607525593 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f906810be30 con 0x7f9068106320 2023-09-13T03:11:11.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.459+0000 7f90577fe700 1 -- 172.21.15.64:0/607525593 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22155 (secure 0 0 0) 0x7f905c005190 con 0x7f9068106320 2023-09-13T03:11:11.460 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:11:11.461 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":82,"fsid":"f8dbced4-51e1-11ee-9ab7-7b867c8bd7da","created":"2023-09-13T03:03:24.167202+0000","modified":"2023-09-13T03:11:10.489849+0000","last_up_change":"2023-09-13T03:10:22.503529+0000","last_in_change":"2023-09-13T03:10:05.678720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T03:06:58.987479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-13T03:11:08.589202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-13T03:11:09.924659+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3997c114-241b-48c4-8feb-653c99209470","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6803","nonce":3940892135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6805","nonce":3940892135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6809","nonce":3940892135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6807","nonce":3940892135}]},"public_addr":"172.21.15.64:6803/3940892135","cluster_addr":"172.21.15.64:6805/3940892135","heartbeat_back_addr":"172.21.15.64:6809/3940892135","heartbeat_front_addr":"172.21.15.64:6807/3940892135","state":["exists","up"]},{"osd":1,"uuid":"94cfae85-322e-4ed2-a6e6-e40349bb14bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6801","nonce":4188149333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6803","nonce":4188149333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6807","nonce":4188149333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6805","nonce":4188149333}]},"public_addr":"172.21.15.142:6801/4188149333","cluster_addr":"172.21.15.142:6803/4188149333","heartbeat_back_addr":"172.21.15.142:6807/4188149333","heartbeat_front_addr":"172.21.15.142:6805/4188149333","state":["exists","up"]},{"osd":2,"uuid":"b3aae17e-96db-4008-8c27-f733c46b30ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6801","nonce":2406669338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6803","nonce":2406669338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6807","nonce":2406669338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6805","nonce":2406669338}]},"public_addr":"172.21.15.170:6801/2406669338","cluster_addr":"172.21.15.170:6803/2406669338","heartbeat_back_addr":"172.21.15.170:6807/2406669338","heartbeat_front_addr":"172.21.15.170:6805/2406669338","state":["exists","up"]},{"osd":3,"uuid":"7fa8d8cb-2ab7-48a7-8b49-5e2903d6779f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6811","nonce":3558318707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6813","nonce":3558318707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6817","nonce":3558318707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6815","nonce":3558318707}]},"public_addr":"172.21.15.64:6811/3558318707","cluster_addr":"172.21.15.64:6813/3558318707","heartbeat_back_addr":"172.21.15.64:6817/3558318707","heartbeat_front_addr":"172.21.15.64:6815/3558318707","state":["exists","up"]},{"osd":4,"uuid":"8c7a9c9f-f919-48d3-9db2-214a25bb5603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6809","nonce":1811142576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6811","nonce":1811142576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6815","nonce":1811142576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6813","nonce":1811142576}]},"public_addr":"172.21.15.142:6809/1811142576","cluster_addr":"172.21.15.142:6811/1811142576","heartbeat_back_addr":"172.21.15.142:6815/1811142576","heartbeat_front_addr":"172.21.15.142:6813/1811142576","state":["exists","up"]},{"osd":5,"uuid":"0c6101c6-2065-4022-a751-d3f216d7f478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6809","nonce":2903841453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6811","nonce":2903841453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6815","nonce":2903841453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6813","nonce":2903841453}]},"public_addr":"172.21.15.170:6809/2903841453","cluster_addr":"172.21.15.170:6811/2903841453","heartbeat_back_addr":"172.21.15.170:6815/2903841453","heartbeat_front_addr":"172.21.15.170:6813/2903841453","state":["exists","up"]},{"osd":6,"uuid":"5cf3e52e-d155-41b1-9194-db047d26a953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6819","nonce":1436584490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6821","nonce":1436584490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6825","nonce":1436584490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6823","nonce":1436584490}]},"public_addr":"172.21.15.64:6819/1436584490","cluster_addr":"172.21.15.64:6821/1436584490","heartbeat_back_addr":"172.21.15.64:6825/1436584490","heartbeat_front_addr":"172.21.15.64:6823/1436584490","state":["exists","up"]},{"osd":7,"uuid":"e97323e3-badd-41a7-af9e-e78a81a64046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6817","nonce":3726793534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6819","nonce":3726793534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6823","nonce":3726793534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6821","nonce":3726793534}]},"public_addr":"172.21.15.142:6817/3726793534","cluster_addr":"172.21.15.142:6819/3726793534","heartbeat_back_addr":"172.21.15.142:6823/3726793534","heartbeat_front_addr":"172.21.15.142:6821/3726793534","state":["exists","up"]},{"osd":8,"uuid":"83d2835c-ed26-493d-9222-c2894ec0e071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6817","nonce":4272513873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6819","nonce":4272513873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6823","nonce":4272513873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6821","nonce":4272513873}]},"public_addr":"172.21.15.170:6817/4272513873","cluster_addr":"172.21.15.170:6819/4272513873","heartbeat_back_addr":"172.21.15.170:6823/4272513873","heartbeat_front_addr":"172.21.15.170:6821/4272513873","state":["exists","up"]},{"osd":9,"uuid":"cb936bdd-1bec-4236-aa2e-a24adcac11e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6827","nonce":2641175211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6829","nonce":2641175211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6833","nonce":2641175211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6831","nonce":2641175211}]},"public_addr":"172.21.15.64:6827/2641175211","cluster_addr":"172.21.15.64:6829/2641175211","heartbeat_back_addr":"172.21.15.64:6833/2641175211","heartbeat_front_addr":"172.21.15.64:6831/2641175211","state":["exists","up"]},{"osd":10,"uuid":"85512a1c-6171-43d4-9c15-c588a8b71be0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6825","nonce":936499217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6827","nonce":936499217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6831","nonce":936499217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6829","nonce":936499217}]},"public_addr":"172.21.15.142:6825/936499217","cluster_addr":"172.21.15.142:6827/936499217","heartbeat_back_addr":"172.21.15.142:6831/936499217","heartbeat_front_addr":"172.21.15.142:6829/936499217","state":["exists","up"]},{"osd":11,"uuid":"9b8417dc-5d6c-4917-94a9-b032c5bc33d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6825","nonce":3605017060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6827","nonce":3605017060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6831","nonce":3605017060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6829","nonce":3605017060}]},"public_addr":"172.21.15.170:6825/3605017060","cluster_addr":"172.21.15.170:6827/3605017060","heartbeat_back_addr":"172.21.15.170:6831/3605017060","heartbeat_front_addr":"172.21.15.170:6829/3605017060","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:14.229044+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:36.147777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:56.655298+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:07:23.107741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:07:44.663770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:05.577432+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:31.168031+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:52.791055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:09:13.658546+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:09:39.150360+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:10:00.601735+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:10:21.407555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6800/2955227347":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/1373223118":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/271194613":"2023-09-14T03:04:22.355817+0000","172.21.15.64:6801/3998091347":"2023-09-14T03:03:56.442541+0000","172.21.15.64:6800/3998091347":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/920818572":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/735789005":"2023-09-14T03:03:56.442541+0000","172.21.15.64:6801/2955227347":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/1975923178":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/1500151178":"2023-09-14T03:04:22.355817+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T03:11:11.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.462+0000 7f906cfff700 1 -- 172.21.15.64:0/607525593 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f90500a0710 msgr2=0x7f90500a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:11.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.462+0000 7f906cfff700 1 --2- 172.21.15.64:0/607525593 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f90500a0710 0x7f90500a2bd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f906000b780 tx=0x7f9060006210 comp rx=0 tx=0).stop 2023-09-13T03:11:11.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.462+0000 7f906cfff700 1 -- 172.21.15.64:0/607525593 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9068106320 msgr2=0x7f906819fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:11.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.462+0000 7f906cfff700 1 --2- 172.21.15.64:0/607525593 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9068106320 0x7f906819fc10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f905c00e940 tx=0x7f905c00ec50 comp rx=0 tx=0).stop 2023-09-13T03:11:11.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.462+0000 7f906cfff700 1 -- 172.21.15.64:0/607525593 shutdown_connections 2023-09-13T03:11:11.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.462+0000 7f906cfff700 1 --2- 172.21.15.64:0/607525593 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f90500a0710 0x7f90500a2bd0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:11.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.462+0000 7f906cfff700 1 --2- 172.21.15.64:0/607525593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90681010c0 0x7f9068198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:11.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.462+0000 7f906cfff700 1 --2- 172.21.15.64:0/607525593 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9068106320 0x7f906819fc10 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:11.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.462+0000 7f906cfff700 1 --2- 172.21.15.64:0/607525593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90681039f0 0x7f9068198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:11.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.462+0000 7f906cfff700 1 -- 172.21.15.64:0/607525593 >> 172.21.15.64:0/607525593 conn(0x7f90680faa30 msgr2=0x7f90680ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:11.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.462+0000 7f906cfff700 1 -- 172.21.15.64:0/607525593 shutdown_connections 2023-09-13T03:11:11.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.462+0000 7f906cfff700 1 -- 172.21.15.64:0/607525593 wait complete. 2023-09-13T03:11:11.482 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-13T03:11:11.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.697+0000 7ff4857ff700 1 -- 172.21.15.64:0/187535027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff480074640 msgr2=0x7ff480072a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:11.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.697+0000 7ff4857ff700 1 --2- 172.21.15.64:0/187535027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff480074640 0x7ff480072a90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff470009990 tx=0x7ff470009ca0 comp rx=0 tx=0).stop 2023-09-13T03:11:11.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.697+0000 7ff4857ff700 1 -- 172.21.15.64:0/187535027 shutdown_connections 2023-09-13T03:11:11.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.697+0000 7ff4857ff700 1 --2- 172.21.15.64:0/187535027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff48010cbd0 0x7ff48010efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:11.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.697+0000 7ff4857ff700 1 --2- 172.21.15.64:0/187535027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff480074640 0x7ff480072a90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:11.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.697+0000 7ff4857ff700 1 --2- 172.21.15.64:0/187535027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff480073060 0x7ff4800734e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:11.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.697+0000 7ff4857ff700 1 -- 172.21.15.64:0/187535027 >> 172.21.15.64:0/187535027 conn(0x7ff4800fbb30 msgr2=0x7ff4800fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:11.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.698+0000 7ff4857ff700 1 -- 172.21.15.64:0/187535027 shutdown_connections 2023-09-13T03:11:11.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.698+0000 7ff4857ff700 1 -- 172.21.15.64:0/187535027 wait complete. 2023-09-13T03:11:11.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.698+0000 7ff4857ff700 1 Processor -- start 2023-09-13T03:11:11.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.699+0000 7ff4857ff700 1 -- start start 2023-09-13T03:11:11.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.699+0000 7ff4857ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff480073060 0x7ff48019c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:11.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.699+0000 7ff4857ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff480074640 0x7ff48019ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:11.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.699+0000 7ff4857ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff48010cbd0 0x7ff4801a3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:11.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.699+0000 7ff4857ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4801a4480 con 0x7ff480073060 2023-09-13T03:11:11.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.699+0000 7ff4857ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4801a45c0 con 0x7ff480074640 2023-09-13T03:11:11.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.699+0000 7ff4857ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4801a4750 con 0x7ff48010cbd0 2023-09-13T03:11:11.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.699+0000 7ff47effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff480073060 0x7ff48019c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:11.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.699+0000 7ff47effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff480073060 0x7ff48019c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57568/0 (socket says 172.21.15.64:57568) 2023-09-13T03:11:11.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.699+0000 7ff47effd700 1 -- 172.21.15.64:0/1452386291 learned_addr learned my addr 172.21.15.64:0/1452386291 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:11.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.699+0000 7ff47e7fc700 1 --2- 172.21.15.64:0/1452386291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff480074640 0x7ff48019ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:11.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.699+0000 7ff47f7fe700 1 --2- 172.21.15.64:0/1452386291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff48010cbd0 0x7ff4801a3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:11.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.700+0000 7ff47f7fe700 1 -- 172.21.15.64:0/1452386291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff480074640 msgr2=0x7ff48019ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:11.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.700+0000 7ff47f7fe700 1 --2- 172.21.15.64:0/1452386291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff480074640 0x7ff48019ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:11.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.700+0000 7ff47f7fe700 1 -- 172.21.15.64:0/1452386291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff480073060 msgr2=0x7ff48019c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:11.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.700+0000 7ff47f7fe700 1 --2- 172.21.15.64:0/1452386291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff480073060 0x7ff48019c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:11.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.700+0000 7ff47f7fe700 1 -- 172.21.15.64:0/1452386291 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff474009520 con 0x7ff48010cbd0 2023-09-13T03:11:11.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.700+0000 7ff47effd700 1 --2- 172.21.15.64:0/1452386291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff480073060 0x7ff48019c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:11:11.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.700+0000 7ff47f7fe700 1 --2- 172.21.15.64:0/1452386291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff48010cbd0 0x7ff4801a3eb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff47800efc0 tx=0x7ff47800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:11.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.700+0000 7ff467fff700 1 -- 172.21.15.64:0/1452386291 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff478018070 con 0x7ff48010cbd0 2023-09-13T03:11:11.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.700+0000 7ff467fff700 1 -- 172.21.15.64:0/1452386291 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff47800f040 con 0x7ff48010cbd0 2023-09-13T03:11:11.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.700+0000 7ff4857ff700 1 -- 172.21.15.64:0/1452386291 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4700095f0 con 0x7ff48010cbd0 2023-09-13T03:11:11.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.700+0000 7ff467fff700 1 -- 172.21.15.64:0/1452386291 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47800cd40 con 0x7ff48010cbd0 2023-09-13T03:11:11.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.700+0000 7ff4857ff700 1 -- 172.21.15.64:0/1452386291 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4801a4da0 con 0x7ff48010cbd0 2023-09-13T03:11:11.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.701+0000 7ff4857ff700 1 -- 172.21.15.64:0/1452386291 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff480102720 con 0x7ff48010cbd0 2023-09-13T03:11:11.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.703+0000 7ff467fff700 1 -- 172.21.15.64:0/1452386291 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7ff478003a00 con 0x7ff48010cbd0 2023-09-13T03:11:11.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.704+0000 7ff467fff700 1 --2- 172.21.15.64:0/1452386291 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ff4680a0710 0x7ff4680a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:11.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.705+0000 7ff467fff700 1 -- 172.21.15.64:0/1452386291 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff4780cbce0 con 0x7ff48010cbd0 2023-09-13T03:11:11.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.705+0000 7ff47effd700 1 --2- 172.21.15.64:0/1452386291 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ff4680a0710 0x7ff4680a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:11.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.705+0000 7ff47effd700 1 --2- 172.21.15.64:0/1452386291 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ff4680a0710 0x7ff4680a2bd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff47000b2e0 tx=0x7ff470005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:11.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.707+0000 7ff467fff700 1 -- 172.21.15.64:0/1452386291 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ff478095950 con 0x7ff48010cbd0 2023-09-13T03:11:11.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.924+0000 7ff4857ff700 1 -- 172.21.15.64:0/1452386291 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff480112610 con 0x7ff48010cbd0 2023-09-13T03:11:11.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.926+0000 7ff467fff700 1 -- 172.21.15.64:0/1452386291 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22155 (secure 0 0 0) 0x7ff478098f70 con 0x7ff48010cbd0 2023-09-13T03:11:11.927 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:11:11.927 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":82,"fsid":"f8dbced4-51e1-11ee-9ab7-7b867c8bd7da","created":"2023-09-13T03:03:24.167202+0000","modified":"2023-09-13T03:11:10.489849+0000","last_up_change":"2023-09-13T03:10:22.503529+0000","last_in_change":"2023-09-13T03:10:05.678720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T03:06:58.987479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-13T03:11:08.589202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-13T03:11:09.924659+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3997c114-241b-48c4-8feb-653c99209470","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6803","nonce":3940892135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6805","nonce":3940892135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6809","nonce":3940892135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6807","nonce":3940892135}]},"public_addr":"172.21.15.64:6803/3940892135","cluster_addr":"172.21.15.64:6805/3940892135","heartbeat_back_addr":"172.21.15.64:6809/3940892135","heartbeat_front_addr":"172.21.15.64:6807/3940892135","state":["exists","up"]},{"osd":1,"uuid":"94cfae85-322e-4ed2-a6e6-e40349bb14bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6801","nonce":4188149333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6803","nonce":4188149333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6807","nonce":4188149333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6805","nonce":4188149333}]},"public_addr":"172.21.15.142:6801/4188149333","cluster_addr":"172.21.15.142:6803/4188149333","heartbeat_back_addr":"172.21.15.142:6807/4188149333","heartbeat_front_addr":"172.21.15.142:6805/4188149333","state":["exists","up"]},{"osd":2,"uuid":"b3aae17e-96db-4008-8c27-f733c46b30ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6801","nonce":2406669338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6803","nonce":2406669338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6807","nonce":2406669338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6805","nonce":2406669338}]},"public_addr":"172.21.15.170:6801/2406669338","cluster_addr":"172.21.15.170:6803/2406669338","heartbeat_back_addr":"172.21.15.170:6807/2406669338","heartbeat_front_addr":"172.21.15.170:6805/2406669338","state":["exists","up"]},{"osd":3,"uuid":"7fa8d8cb-2ab7-48a7-8b49-5e2903d6779f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6811","nonce":3558318707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6813","nonce":3558318707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6817","nonce":3558318707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6815","nonce":3558318707}]},"public_addr":"172.21.15.64:6811/3558318707","cluster_addr":"172.21.15.64:6813/3558318707","heartbeat_back_addr":"172.21.15.64:6817/3558318707","heartbeat_front_addr":"172.21.15.64:6815/3558318707","state":["exists","up"]},{"osd":4,"uuid":"8c7a9c9f-f919-48d3-9db2-214a25bb5603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6809","nonce":1811142576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6811","nonce":1811142576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6815","nonce":1811142576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6813","nonce":1811142576}]},"public_addr":"172.21.15.142:6809/1811142576","cluster_addr":"172.21.15.142:6811/1811142576","heartbeat_back_addr":"172.21.15.142:6815/1811142576","heartbeat_front_addr":"172.21.15.142:6813/1811142576","state":["exists","up"]},{"osd":5,"uuid":"0c6101c6-2065-4022-a751-d3f216d7f478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6809","nonce":2903841453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6811","nonce":2903841453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6815","nonce":2903841453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6813","nonce":2903841453}]},"public_addr":"172.21.15.170:6809/2903841453","cluster_addr":"172.21.15.170:6811/2903841453","heartbeat_back_addr":"172.21.15.170:6815/2903841453","heartbeat_front_addr":"172.21.15.170:6813/2903841453","state":["exists","up"]},{"osd":6,"uuid":"5cf3e52e-d155-41b1-9194-db047d26a953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6819","nonce":1436584490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6821","nonce":1436584490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6825","nonce":1436584490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6823","nonce":1436584490}]},"public_addr":"172.21.15.64:6819/1436584490","cluster_addr":"172.21.15.64:6821/1436584490","heartbeat_back_addr":"172.21.15.64:6825/1436584490","heartbeat_front_addr":"172.21.15.64:6823/1436584490","state":["exists","up"]},{"osd":7,"uuid":"e97323e3-badd-41a7-af9e-e78a81a64046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6817","nonce":3726793534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6819","nonce":3726793534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6823","nonce":3726793534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6821","nonce":3726793534}]},"public_addr":"172.21.15.142:6817/3726793534","cluster_addr":"172.21.15.142:6819/3726793534","heartbeat_back_addr":"172.21.15.142:6823/3726793534","heartbeat_front_addr":"172.21.15.142:6821/3726793534","state":["exists","up"]},{"osd":8,"uuid":"83d2835c-ed26-493d-9222-c2894ec0e071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6817","nonce":4272513873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6819","nonce":4272513873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6823","nonce":4272513873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6821","nonce":4272513873}]},"public_addr":"172.21.15.170:6817/4272513873","cluster_addr":"172.21.15.170:6819/4272513873","heartbeat_back_addr":"172.21.15.170:6823/4272513873","heartbeat_front_addr":"172.21.15.170:6821/4272513873","state":["exists","up"]},{"osd":9,"uuid":"cb936bdd-1bec-4236-aa2e-a24adcac11e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6827","nonce":2641175211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6829","nonce":2641175211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6833","nonce":2641175211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6831","nonce":2641175211}]},"public_addr":"172.21.15.64:6827/2641175211","cluster_addr":"172.21.15.64:6829/2641175211","heartbeat_back_addr":"172.21.15.64:6833/2641175211","heartbeat_front_addr":"172.21.15.64:6831/2641175211","state":["exists","up"]},{"osd":10,"uuid":"85512a1c-6171-43d4-9c15-c588a8b71be0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6825","nonce":936499217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6827","nonce":936499217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6831","nonce":936499217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6829","nonce":936499217}]},"public_addr":"172.21.15.142:6825/936499217","cluster_addr":"172.21.15.142:6827/936499217","heartbeat_back_addr":"172.21.15.142:6831/936499217","heartbeat_front_addr":"172.21.15.142:6829/936499217","state":["exists","up"]},{"osd":11,"uuid":"9b8417dc-5d6c-4917-94a9-b032c5bc33d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6825","nonce":3605017060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6827","nonce":3605017060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6831","nonce":3605017060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6829","nonce":3605017060}]},"public_addr":"172.21.15.170:6825/3605017060","cluster_addr":"172.21.15.170:6827/3605017060","heartbeat_back_addr":"172.21.15.170:6831/3605017060","heartbeat_front_addr":"172.21.15.170:6829/3605017060","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:14.229044+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:36.147777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:56.655298+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:07:23.107741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:07:44.663770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:05.577432+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:31.168031+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:52.791055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:09:13.658546+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:09:39.150360+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:10:00.601735+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:10:21.407555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6800/2955227347":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/1373223118":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/271194613":"2023-09-14T03:04:22.355817+0000","172.21.15.64:6801/3998091347":"2023-09-14T03:03:56.442541+0000","172.21.15.64:6800/3998091347":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/920818572":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/735789005":"2023-09-14T03:03:56.442541+0000","172.21.15.64:6801/2955227347":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/1975923178":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/1500151178":"2023-09-14T03:04:22.355817+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T03:11:11.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.929+0000 7ff4857ff700 1 -- 172.21.15.64:0/1452386291 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ff4680a0710 msgr2=0x7ff4680a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:11.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.929+0000 7ff4857ff700 1 --2- 172.21.15.64:0/1452386291 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ff4680a0710 0x7ff4680a2bd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff47000b2e0 tx=0x7ff470005fb0 comp rx=0 tx=0).stop 2023-09-13T03:11:11.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.929+0000 7ff4857ff700 1 -- 172.21.15.64:0/1452386291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff48010cbd0 msgr2=0x7ff4801a3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:11.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.929+0000 7ff4857ff700 1 --2- 172.21.15.64:0/1452386291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff48010cbd0 0x7ff4801a3eb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff47800efc0 tx=0x7ff47800c3a0 comp rx=0 tx=0).stop 2023-09-13T03:11:11.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.929+0000 7ff4857ff700 1 -- 172.21.15.64:0/1452386291 shutdown_connections 2023-09-13T03:11:11.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.929+0000 7ff4857ff700 1 --2- 172.21.15.64:0/1452386291 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ff4680a0710 0x7ff4680a2bd0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:11.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.929+0000 7ff4857ff700 1 --2- 172.21.15.64:0/1452386291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff48010cbd0 0x7ff4801a3eb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:11.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.929+0000 7ff4857ff700 1 --2- 172.21.15.64:0/1452386291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff480073060 0x7ff48019c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:11.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.929+0000 7ff4857ff700 1 --2- 172.21.15.64:0/1452386291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff480074640 0x7ff48019ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:11.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.929+0000 7ff4857ff700 1 -- 172.21.15.64:0/1452386291 >> 172.21.15.64:0/1452386291 conn(0x7ff4800fbb30 msgr2=0x7ff4800fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:11.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.929+0000 7ff4857ff700 1 -- 172.21.15.64:0/1452386291 shutdown_connections 2023-09-13T03:11:11.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:11.929+0000 7ff4857ff700 1 -- 172.21.15.64:0/1452386291 wait complete. 2023-09-13T03:11:11.951 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-09-13T03:11:12.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:12.166+0000 7f89db9ff700 1 -- 172.21.15.64:0/3584333261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89d4101800 msgr2=0x7f89d4101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:12.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:12.166+0000 7f89db9ff700 1 --2- 172.21.15.64:0/3584333261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89d4101800 0x7f89d4101c60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f89c40099c0 tx=0x7f89c4009cd0 comp rx=0 tx=0).stop 2023-09-13T03:11:12.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:12.167+0000 7f89db9ff700 1 -- 172.21.15.64:0/3584333261 shutdown_connections 2023-09-13T03:11:12.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:12.167+0000 7f89db9ff700 1 --2- 172.21.15.64:0/3584333261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89d4100610 0x7f89d4100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:12.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:12.167+0000 7f89db9ff700 1 --2- 172.21.15.64:0/3584333261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89d41021a0 0x7f89d4108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:12.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:12.167+0000 7f89db9ff700 1 --2- 172.21.15.64:0/3584333261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89d4101800 0x7f89d4101c60 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:12.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:12.167+0000 7f89db9ff700 1 -- 172.21.15.64:0/3584333261 >> 172.21.15.64:0/3584333261 conn(0x7f89d40fbb80 msgr2=0x7f89d40fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:12.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:12.167+0000 7f89db9ff700 1 -- 172.21.15.64:0/3584333261 shutdown_connections 2023-09-13T03:11:12.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:12.167+0000 7f89db9ff700 1 -- 172.21.15.64:0/3584333261 wait complete. 2023-09-13T03:11:12.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:12.168+0000 7f89db9ff700 1 Processor -- start 2023-09-13T03:11:12.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:12.168+0000 7f89db9ff700 1 -- start start 2023-09-13T03:11:12.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:12.169+0000 7f89db9ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89d4100610 0x7f89d4196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:12.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:12.169+0000 7f89db9ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89d4101800 0x7f89d4196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:12.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:12.169+0000 7f89db9ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89d41021a0 0x7f89d419d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:12.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:12.169+0000 7f89db9ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d419de80 con 0x7f89d41021a0 2023-09-13T03:11:12.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:12.169+0000 7f89db9ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d419dfc0 con 0x7f89d4100610 2023-09-13T03:11:12.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:12.169+0000 7f89db9ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d419e150 con 0x7f89d4101800 2023-09-13T03:11:12.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:12.169+0000 7f89d8f9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89d4101800 0x7f89d4196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:12.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:12.169+0000 7f89d9f9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89d41021a0 0x7f89d419d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:12.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:12.169+0000 7f89d979b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89d4100610 0x7f89d4196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:12.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:12.169+0000 7f89d8f9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89d4101800 0x7f89d4196840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:41500/0 (socket says 172.21.15.64:41500) 2023-09-13T03:11:12.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:12.169+0000 7f89d8f9a700 1 -- 172.21.15.64:0/45244231 learned_addr learned my addr 172.21.15.64:0/45244231 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:12.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:12.169+0000 7f89d8f9a700 1 -- 172.21.15.64:0/45244231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89d4100610 msgr2=0x7f89d4196300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:12.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:12.169+0000 7f89d8f9a700 1 --2- 172.21.15.64:0/45244231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89d4100610 0x7f89d4196300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:12.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:12.169+0000 7f89d8f9a700 1 -- 172.21.15.64:0/45244231 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89d41021a0 msgr2=0x7f89d419d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:12.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:12.169+0000 7f89d8f9a700 1 --2- 172.21.15.64:0/45244231 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89d41021a0 0x7f89d419d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:12.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:12.169+0000 7f89d8f9a700 1 -- 172.21.15.64:0/45244231 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c40095f0 con 0x7f89d4101800 2023-09-13T03:11:12.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:12.169+0000 7f89d979b700 1 --2- 172.21.15.64:0/45244231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89d4100610 0x7f89d4196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:11:12.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:12.170+0000 7f89d8f9a700 1 --2- 172.21.15.64:0/45244231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89d4101800 0x7f89d4196840 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f89c4009990 tx=0x7f89c400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:12.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:12.170+0000 7f89ca7fc700 1 -- 172.21.15.64:0/45244231 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c401d070 con 0x7f89d4101800 2023-09-13T03:11:12.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:12.170+0000 7f89ca7fc700 1 -- 172.21.15.64:0/45244231 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89c4005030 con 0x7f89d4101800 2023-09-13T03:11:12.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:12.170+0000 7f89ca7fc700 1 -- 172.21.15.64:0/45244231 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c4004a70 con 0x7f89d4101800 2023-09-13T03:11:12.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:12.170+0000 7f89db9ff700 1 -- 172.21.15.64:0/45244231 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89d419e3d0 con 0x7f89d4101800 2023-09-13T03:11:12.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:12.170+0000 7f89db9ff700 1 -- 172.21.15.64:0/45244231 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89d419e7a0 con 0x7f89d4101800 2023-09-13T03:11:12.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:12.171+0000 7f89db9ff700 1 -- 172.21.15.64:0/45244231 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89d4107590 con 0x7f89d4101800 2023-09-13T03:11:12.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:12.173+0000 7f89ca7fc700 1 -- 172.21.15.64:0/45244231 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f89c4005410 con 0x7f89d4101800 2023-09-13T03:11:12.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:12.175+0000 7f89ca7fc700 1 --2- 172.21.15.64:0/45244231 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f89c00a0640 0x7f89c00a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:12.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:12.175+0000 7f89ca7fc700 1 -- 172.21.15.64:0/45244231 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f89c40ccd40 con 0x7f89d4101800 2023-09-13T03:11:12.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:12.175+0000 7f89d979b700 1 --2- 172.21.15.64:0/45244231 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f89c00a0640 0x7f89c00a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:12.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:12.176+0000 7f89d979b700 1 --2- 172.21.15.64:0/45244231 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f89c00a0640 0x7f89c00a2b00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f89d4101660 tx=0x7f89d0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:12.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:12.177+0000 7f89ca7fc700 1 -- 172.21.15.64:0/45244231 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f89c40969b0 con 0x7f89d4101800 2023-09-13T03:11:12.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:11 smithi142 ceph-mon[134082]: osdmap e82: 12 total, 12 up, 12 in 2023-09-13T03:11:12.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:11 smithi142 ceph-mon[134082]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-09-13T03:11:12.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:11 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/1438064125' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-09-13T03:11:12.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:11 smithi142 ceph-mon[134082]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-09-13T03:11:12.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:11 smithi142 ceph-mon[134082]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-09-13T03:11:12.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:11 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:12.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:11 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:12.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:11 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:12.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:11 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:12.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:11 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/607525593' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:11:12.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:11 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/1452386291' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:11:12.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:11 smithi170 ceph-mon[134328]: osdmap e82: 12 total, 12 up, 12 in 2023-09-13T03:11:12.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:11 smithi170 ceph-mon[134328]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-09-13T03:11:12.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:11 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/1438064125' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-09-13T03:11:12.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:11 smithi170 ceph-mon[134328]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-09-13T03:11:12.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:11 smithi170 ceph-mon[134328]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-09-13T03:11:12.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:11 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:12.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:11 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:12.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:11 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:12.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:11 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:12.412 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:11 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/607525593' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:11:12.412 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:11 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/1452386291' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:11:12.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:11 smithi064 ceph-mon[97675]: osdmap e82: 12 total, 12 up, 12 in 2023-09-13T03:11:12.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:11 smithi064 ceph-mon[97675]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-09-13T03:11:12.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:11 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/1438064125' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-09-13T03:11:12.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:11 smithi064 ceph-mon[97675]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-09-13T03:11:12.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:11 smithi064 ceph-mon[97675]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-09-13T03:11:12.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:11 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:12.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:11 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:12.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:11 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:12.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:11 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:12.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:11 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/607525593' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:11:12.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:11 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/1452386291' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:11:12.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:12.437+0000 7f89db9ff700 1 -- 172.21.15.64:0/45244231 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f89d40008d0 con 0x7f89d4101800 2023-09-13T03:11:13.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.242+0000 7f89ca7fc700 1 -- 172.21.15.64:0/45244231 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f89c4099fd0 con 0x7f89d4101800 2023-09-13T03:11:13.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.244+0000 7f89db9ff700 1 -- 172.21.15.64:0/45244231 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f89c00a0640 msgr2=0x7f89c00a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:13.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.244+0000 7f89db9ff700 1 --2- 172.21.15.64:0/45244231 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f89c00a0640 0x7f89c00a2b00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f89d4101660 tx=0x7f89d0009260 comp rx=0 tx=0).stop 2023-09-13T03:11:13.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.244+0000 7f89db9ff700 1 -- 172.21.15.64:0/45244231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89d4101800 msgr2=0x7f89d4196840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:13.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.244+0000 7f89db9ff700 1 --2- 172.21.15.64:0/45244231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89d4101800 0x7f89d4196840 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f89c4009990 tx=0x7f89c400f690 comp rx=0 tx=0).stop 2023-09-13T03:11:13.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.245+0000 7f89db9ff700 1 -- 172.21.15.64:0/45244231 shutdown_connections 2023-09-13T03:11:13.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.245+0000 7f89db9ff700 1 --2- 172.21.15.64:0/45244231 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f89c00a0640 0x7f89c00a2b00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:13.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.245+0000 7f89db9ff700 1 --2- 172.21.15.64:0/45244231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89d4100610 0x7f89d4196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:13.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.245+0000 7f89db9ff700 1 --2- 172.21.15.64:0/45244231 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89d41021a0 0x7f89d419d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:13.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.245+0000 7f89db9ff700 1 --2- 172.21.15.64:0/45244231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89d4101800 0x7f89d4196840 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:13.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.245+0000 7f89db9ff700 1 -- 172.21.15.64:0/45244231 >> 172.21.15.64:0/45244231 conn(0x7f89d40fbb80 msgr2=0x7f89d40fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:13.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.245+0000 7f89db9ff700 1 -- 172.21.15.64:0/45244231 shutdown_connections 2023-09-13T03:11:13.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.245+0000 7f89db9ff700 1 -- 172.21.15.64:0/45244231 wait complete. 2023-09-13T03:11:13.248 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:12 smithi142 ceph-mon[134082]: osdmap e83: 12 total, 12 up, 12 in 2023-09-13T03:11:13.248 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:12 smithi142 ceph-mon[134082]: mds.? [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] up:active 2023-09-13T03:11:13.249 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:12 smithi142 ceph-mon[134082]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-13T03:11:13.249 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:12 smithi142 ceph-mon[134082]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-13T03:11:13.249 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:12 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:13.249 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:12 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:13.249 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:12 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:11:13.250 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:12 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:11:13.250 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:12 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:13.250 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:12 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/45244231' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-09-13T03:11:13.250 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:12 smithi142 ceph-mon[134082]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-09-13T03:11:13.250 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:12 smithi142 ceph-mon[134082]: pgmap v278: 97 pgs: 13 creating+peering, 59 unknown, 25 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 1 op/s 2023-09-13T03:11:13.263 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:12 smithi064 ceph-mon[97675]: osdmap e83: 12 total, 12 up, 12 in 2023-09-13T03:11:13.263 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:12 smithi064 ceph-mon[97675]: mds.? [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] up:active 2023-09-13T03:11:13.263 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:12 smithi064 ceph-mon[97675]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-13T03:11:13.263 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:12 smithi064 ceph-mon[97675]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-13T03:11:13.264 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:12 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:13.264 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:12 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:13.264 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:12 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:11:13.264 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:12 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:11:13.265 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:12 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:13.265 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:12 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/45244231' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-09-13T03:11:13.265 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:12 smithi064 ceph-mon[97675]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-09-13T03:11:13.265 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:12 smithi064 ceph-mon[97675]: pgmap v278: 97 pgs: 13 creating+peering, 59 unknown, 25 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 1 op/s 2023-09-13T03:11:13.266 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2023-09-13T03:11:13.267 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2023-09-13T03:11:13.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:12 smithi170 ceph-mon[134328]: osdmap e83: 12 total, 12 up, 12 in 2023-09-13T03:11:13.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:12 smithi170 ceph-mon[134328]: mds.? [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] up:active 2023-09-13T03:11:13.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:12 smithi170 ceph-mon[134328]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-13T03:11:13.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:12 smithi170 ceph-mon[134328]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-13T03:11:13.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:12 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:13.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:12 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:13.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:12 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:11:13.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:12 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:11:13.412 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:12 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:11:13.412 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:12 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/45244231' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-09-13T03:11:13.412 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:12 smithi170 ceph-mon[134328]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-09-13T03:11:13.412 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:12 smithi170 ceph-mon[134328]: pgmap v278: 97 pgs: 13 creating+peering, 59 unknown, 25 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 1 op/s 2023-09-13T03:11:13.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.473+0000 7f0824fff700 1 -- 172.21.15.64:0/288330564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0820074640 msgr2=0x7f0820072a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:13.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.473+0000 7f0824fff700 1 --2- 172.21.15.64:0/288330564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0820074640 0x7f0820072a90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0810009990 tx=0x7f0810009ca0 comp rx=0 tx=0).stop 2023-09-13T03:11:13.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.473+0000 7f0824fff700 1 -- 172.21.15.64:0/288330564 shutdown_connections 2023-09-13T03:11:13.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.473+0000 7f0824fff700 1 --2- 172.21.15.64:0/288330564 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08201084b0 0x7f082010a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:13.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.473+0000 7f0824fff700 1 --2- 172.21.15.64:0/288330564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0820074640 0x7f0820072a90 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:13.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.473+0000 7f0824fff700 1 --2- 172.21.15.64:0/288330564 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0820073060 0x7f08200734e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:13.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.473+0000 7f0824fff700 1 -- 172.21.15.64:0/288330564 >> 172.21.15.64:0/288330564 conn(0x7f08200fbb40 msgr2=0x7f08200fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:13.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.473+0000 7f0824fff700 1 -- 172.21.15.64:0/288330564 shutdown_connections 2023-09-13T03:11:13.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.473+0000 7f0824fff700 1 -- 172.21.15.64:0/288330564 wait complete. 2023-09-13T03:11:13.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.475+0000 7f0824fff700 1 Processor -- start 2023-09-13T03:11:13.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.475+0000 7f0824fff700 1 -- start start 2023-09-13T03:11:13.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.475+0000 7f0824fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0820073060 0x7f0820198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:13.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.476+0000 7f0824fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0820074640 0x7f0820198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:13.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.476+0000 7f0824fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08201084b0 0x7f082019fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:13.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.476+0000 7f0824fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08201a00d0 con 0x7f08201084b0 2023-09-13T03:11:13.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.476+0000 7f0824fff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08201a0210 con 0x7f0820074640 2023-09-13T03:11:13.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.476+0000 7f0824fff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08201a03a0 con 0x7f0820073060 2023-09-13T03:11:13.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.476+0000 7f081ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08201084b0 0x7f082019fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:13.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.476+0000 7f081ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08201084b0 0x7f082019fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57600/0 (socket says 172.21.15.64:57600) 2023-09-13T03:11:13.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.476+0000 7f081ed9d700 1 -- 172.21.15.64:0/2526007096 learned_addr learned my addr 172.21.15.64:0/2526007096 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:13.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.476+0000 7f081dd9b700 1 --2- 172.21.15.64:0/2526007096 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0820074640 0x7f0820198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:13.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.476+0000 7f081e59c700 1 --2- 172.21.15.64:0/2526007096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0820073060 0x7f0820198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:13.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.476+0000 7f081ed9d700 1 -- 172.21.15.64:0/2526007096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0820073060 msgr2=0x7f0820198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:13.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.476+0000 7f081ed9d700 1 --2- 172.21.15.64:0/2526007096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0820073060 0x7f0820198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:13.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.476+0000 7f081ed9d700 1 -- 172.21.15.64:0/2526007096 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0820074640 msgr2=0x7f0820198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:13.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.476+0000 7f081ed9d700 1 --2- 172.21.15.64:0/2526007096 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0820074640 0x7f0820198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:13.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.476+0000 7f081ed9d700 1 -- 172.21.15.64:0/2526007096 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08100095f0 con 0x7f08201084b0 2023-09-13T03:11:13.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.476+0000 7f081dd9b700 1 --2- 172.21.15.64:0/2526007096 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0820074640 0x7f0820198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:11:13.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.477+0000 7f081ed9d700 1 --2- 172.21.15.64:0/2526007096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08201084b0 0x7f082019fb00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f081800ef30 tx=0x7f081800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:13.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.477+0000 7f080f7fe700 1 -- 172.21.15.64:0/2526007096 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0818018070 con 0x7f08201084b0 2023-09-13T03:11:13.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.477+0000 7f080f7fe700 1 -- 172.21.15.64:0/2526007096 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f081800f040 con 0x7f08201084b0 2023-09-13T03:11:13.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.477+0000 7f0824fff700 1 -- 172.21.15.64:0/2526007096 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08201a0620 con 0x7f08201084b0 2023-09-13T03:11:13.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.477+0000 7f080f7fe700 1 -- 172.21.15.64:0/2526007096 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081800cd00 con 0x7f08201084b0 2023-09-13T03:11:13.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.477+0000 7f0824fff700 1 -- 172.21.15.64:0/2526007096 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08201a0b80 con 0x7f08201084b0 2023-09-13T03:11:13.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.478+0000 7f0824fff700 1 -- 172.21.15.64:0/2526007096 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0820106b60 con 0x7f08201084b0 2023-09-13T03:11:13.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.480+0000 7f080f7fe700 1 -- 172.21.15.64:0/2526007096 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f0818003a00 con 0x7f08201084b0 2023-09-13T03:11:13.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.481+0000 7f080f7fe700 1 --2- 172.21.15.64:0/2526007096 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f08080a06c0 0x7f08080a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:13.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.482+0000 7f080f7fe700 1 -- 172.21.15.64:0/2526007096 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f08180cb590 con 0x7f08201084b0 2023-09-13T03:11:13.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.482+0000 7f081e59c700 1 --2- 172.21.15.64:0/2526007096 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f08080a06c0 0x7f08080a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:13.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.482+0000 7f081e59c700 1 --2- 172.21.15.64:0/2526007096 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f08080a06c0 0x7f08080a2b80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f081000b2e0 tx=0x7f0810005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:13.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.486+0000 7f080f7fe700 1 -- 172.21.15.64:0/2526007096 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0818095200 con 0x7f08201084b0 2023-09-13T03:11:13.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:13.744+0000 7f0824fff700 1 -- 172.21.15.64:0/2526007096 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7f082002cf80 con 0x7f08201084b0 2023-09-13T03:11:14.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.185+0000 7f080f7fe700 1 -- 172.21.15.64:0/2526007096 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) v1 ==== 105+0+0 (secure 0 0 0) 0x7f0818098820 con 0x7f08201084b0 2023-09-13T03:11:14.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.188+0000 7f0824fff700 1 -- 172.21.15.64:0/2526007096 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f08080a06c0 msgr2=0x7f08080a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:14.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.188+0000 7f0824fff700 1 --2- 172.21.15.64:0/2526007096 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f08080a06c0 0x7f08080a2b80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f081000b2e0 tx=0x7f0810005fb0 comp rx=0 tx=0).stop 2023-09-13T03:11:14.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.188+0000 7f0824fff700 1 -- 172.21.15.64:0/2526007096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08201084b0 msgr2=0x7f082019fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:14.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.188+0000 7f0824fff700 1 --2- 172.21.15.64:0/2526007096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08201084b0 0x7f082019fb00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f081800ef30 tx=0x7f081800c3a0 comp rx=0 tx=0).stop 2023-09-13T03:11:14.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.188+0000 7f0824fff700 1 -- 172.21.15.64:0/2526007096 shutdown_connections 2023-09-13T03:11:14.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.188+0000 7f0824fff700 1 --2- 172.21.15.64:0/2526007096 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f08080a06c0 0x7f08080a2b80 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:14.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.188+0000 7f0824fff700 1 --2- 172.21.15.64:0/2526007096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08201084b0 0x7f082019fb00 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:14.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.188+0000 7f0824fff700 1 --2- 172.21.15.64:0/2526007096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0820073060 0x7f0820198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:14.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.188+0000 7f0824fff700 1 --2- 172.21.15.64:0/2526007096 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0820074640 0x7f0820198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:14.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.188+0000 7f0824fff700 1 -- 172.21.15.64:0/2526007096 >> 172.21.15.64:0/2526007096 conn(0x7f08200fbb40 msgr2=0x7f08200fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:14.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.188+0000 7f0824fff700 1 -- 172.21.15.64:0/2526007096 shutdown_connections 2023-09-13T03:11:14.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.188+0000 7f0824fff700 1 -- 172.21.15.64:0/2526007096 wait complete. 2023-09-13T03:11:14.210 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:13 smithi064 ceph-mon[97675]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2023-09-13T03:11:14.210 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:13 smithi064 ceph-mon[97675]: Cluster is now healthy 2023-09-13T03:11:14.210 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:13 smithi064 ceph-mon[97675]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-09-13T03:11:14.211 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:13 smithi064 ceph-mon[97675]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-13T03:11:14.211 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:13 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2526007096' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-09-13T03:11:14.212 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-09-13T03:11:14.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:13 smithi142 ceph-mon[134082]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2023-09-13T03:11:14.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:13 smithi142 ceph-mon[134082]: Cluster is now healthy 2023-09-13T03:11:14.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:13 smithi142 ceph-mon[134082]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-09-13T03:11:14.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:13 smithi142 ceph-mon[134082]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-13T03:11:14.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:13 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/2526007096' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-09-13T03:11:14.409 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:13 smithi170 ceph-mon[134328]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2023-09-13T03:11:14.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:13 smithi170 ceph-mon[134328]: Cluster is now healthy 2023-09-13T03:11:14.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:13 smithi170 ceph-mon[134328]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-09-13T03:11:14.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:13 smithi170 ceph-mon[134328]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-13T03:11:14.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:13 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/2526007096' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-09-13T03:11:14.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.417+0000 7f24a7dff700 1 -- 172.21.15.64:0/564384879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24a01021a0 msgr2=0x7f24a0108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:14.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.417+0000 7f24a7dff700 1 --2- 172.21.15.64:0/564384879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24a01021a0 0x7f24a0108a50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f249800b0d0 tx=0x7f249800b3e0 comp rx=0 tx=0).stop 2023-09-13T03:11:14.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.417+0000 7f24a7dff700 1 -- 172.21.15.64:0/564384879 shutdown_connections 2023-09-13T03:11:14.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.417+0000 7f24a7dff700 1 --2- 172.21.15.64:0/564384879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24a01021a0 0x7f24a0108a50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:14.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.417+0000 7f24a7dff700 1 --2- 172.21.15.64:0/564384879 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24a0100610 0x7f24a0100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:14.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.417+0000 7f24a7dff700 1 --2- 172.21.15.64:0/564384879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24a0101800 0x7f24a0101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:14.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.417+0000 7f24a7dff700 1 -- 172.21.15.64:0/564384879 >> 172.21.15.64:0/564384879 conn(0x7f24a00fbb80 msgr2=0x7f24a00fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:14.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.417+0000 7f24a7dff700 1 -- 172.21.15.64:0/564384879 shutdown_connections 2023-09-13T03:11:14.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.417+0000 7f24a7dff700 1 -- 172.21.15.64:0/564384879 wait complete. 2023-09-13T03:11:14.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.418+0000 7f24a7dff700 1 Processor -- start 2023-09-13T03:11:14.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.418+0000 7f24a7dff700 1 -- start start 2023-09-13T03:11:14.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.419+0000 7f24a7dff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24a0100610 0x7f24a01985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:14.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.419+0000 7f24a7dff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24a0101800 0x7f24a0198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:14.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.419+0000 7f24a7dff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24a01021a0 0x7f24a019fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:14.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.419+0000 7f24a7dff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a01a0140 con 0x7f24a0100610 2023-09-13T03:11:14.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.419+0000 7f24a7dff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a01a0280 con 0x7f24a01021a0 2023-09-13T03:11:14.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.419+0000 7f24a7dff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a01a0410 con 0x7f24a0101800 2023-09-13T03:11:14.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.419+0000 7f24a5b9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24a0100610 0x7f24a01985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:14.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.419+0000 7f24a5b9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24a0100610 0x7f24a01985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57610/0 (socket says 172.21.15.64:57610) 2023-09-13T03:11:14.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.419+0000 7f24a5b9b700 1 -- 172.21.15.64:0/1924268439 learned_addr learned my addr 172.21.15.64:0/1924268439 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:14.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.419+0000 7f24a639c700 1 --2- 172.21.15.64:0/1924268439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24a01021a0 0x7f24a019fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:14.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.419+0000 7f24a539a700 1 --2- 172.21.15.64:0/1924268439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24a0101800 0x7f24a0198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:14.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.419+0000 7f24a5b9b700 1 -- 172.21.15.64:0/1924268439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24a0101800 msgr2=0x7f24a0198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:14.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.419+0000 7f24a5b9b700 1 --2- 172.21.15.64:0/1924268439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24a0101800 0x7f24a0198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:14.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.419+0000 7f24a5b9b700 1 -- 172.21.15.64:0/1924268439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24a01021a0 msgr2=0x7f24a019fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:14.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.419+0000 7f24a5b9b700 1 --2- 172.21.15.64:0/1924268439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24a01021a0 0x7f24a019fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:14.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.419+0000 7f24a5b9b700 1 -- 172.21.15.64:0/1924268439 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2498009d00 con 0x7f24a0100610 2023-09-13T03:11:14.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.420+0000 7f24a5b9b700 1 --2- 172.21.15.64:0/1924268439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24a0100610 0x7f24a01985c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f249c00e970 tx=0x7f249c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:14.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.420+0000 7f2496ffd700 1 -- 172.21.15.64:0/1924268439 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249c009790 con 0x7f24a0100610 2023-09-13T03:11:14.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.420+0000 7f2496ffd700 1 -- 172.21.15.64:0/1924268439 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f249c00eec0 con 0x7f24a0100610 2023-09-13T03:11:14.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.420+0000 7f24a7dff700 1 -- 172.21.15.64:0/1924268439 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24a01a06f0 con 0x7f24a0100610 2023-09-13T03:11:14.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.420+0000 7f2496ffd700 1 -- 172.21.15.64:0/1924268439 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249c003cf0 con 0x7f24a0100610 2023-09-13T03:11:14.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.420+0000 7f24a7dff700 1 -- 172.21.15.64:0/1924268439 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24a01a0c50 con 0x7f24a0100610 2023-09-13T03:11:14.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.421+0000 7f24a7dff700 1 -- 172.21.15.64:0/1924268439 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24a010be30 con 0x7f24a0100610 2023-09-13T03:11:14.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.423+0000 7f2496ffd700 1 -- 172.21.15.64:0/1924268439 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f249c00ccd0 con 0x7f24a0100610 2023-09-13T03:11:14.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.424+0000 7f2496ffd700 1 --2- 172.21.15.64:0/1924268439 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f248c0a06c0 0x7f248c0a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:14.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.424+0000 7f2496ffd700 1 -- 172.21.15.64:0/1924268439 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f249c014070 con 0x7f24a0100610 2023-09-13T03:11:14.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.425+0000 7f24a539a700 1 --2- 172.21.15.64:0/1924268439 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f248c0a06c0 0x7f248c0a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:14.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.425+0000 7f24a539a700 1 --2- 172.21.15.64:0/1924268439 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f248c0a06c0 0x7f248c0a2b80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2490009bf0 tx=0x7f2490009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:14.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.429+0000 7f2496ffd700 1 -- 172.21.15.64:0/1924268439 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f249c0958b0 con 0x7f24a0100610 2023-09-13T03:11:14.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:14.688+0000 7f24a7dff700 1 -- 172.21.15.64:0/1924268439 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f24a010c040 con 0x7f24a0100610 2023-09-13T03:11:15.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.195+0000 7f2496ffd700 1 -- 172.21.15.64:0/1924268439 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v19) v1 ==== 121+0+0 (secure 0 0 0) 0x7f249c098ed0 con 0x7f24a0100610 2023-09-13T03:11:15.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.198+0000 7f24a7dff700 1 -- 172.21.15.64:0/1924268439 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f248c0a06c0 msgr2=0x7f248c0a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:15.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.198+0000 7f24a7dff700 1 --2- 172.21.15.64:0/1924268439 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f248c0a06c0 0x7f248c0a2b80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2490009bf0 tx=0x7f2490009260 comp rx=0 tx=0).stop 2023-09-13T03:11:15.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.198+0000 7f24a7dff700 1 -- 172.21.15.64:0/1924268439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24a0100610 msgr2=0x7f24a01985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:15.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.198+0000 7f24a7dff700 1 --2- 172.21.15.64:0/1924268439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24a0100610 0x7f24a01985c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f249c00e970 tx=0x7f249c00ec80 comp rx=0 tx=0).stop 2023-09-13T03:11:15.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.198+0000 7f24a7dff700 1 -- 172.21.15.64:0/1924268439 shutdown_connections 2023-09-13T03:11:15.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.198+0000 7f24a7dff700 1 --2- 172.21.15.64:0/1924268439 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f248c0a06c0 0x7f248c0a2b80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:15.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.198+0000 7f24a7dff700 1 --2- 172.21.15.64:0/1924268439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24a01021a0 0x7f24a019fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:15.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.198+0000 7f24a7dff700 1 --2- 172.21.15.64:0/1924268439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24a0100610 0x7f24a01985c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:15.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.198+0000 7f24a7dff700 1 --2- 172.21.15.64:0/1924268439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24a0101800 0x7f24a0198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:15.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.198+0000 7f24a7dff700 1 -- 172.21.15.64:0/1924268439 >> 172.21.15.64:0/1924268439 conn(0x7f24a00fbb80 msgr2=0x7f24a01070a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:15.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.198+0000 7f24a7dff700 1 -- 172.21.15.64:0/1924268439 shutdown_connections 2023-09-13T03:11:15.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.198+0000 7f24a7dff700 1 -- 172.21.15.64:0/1924268439 wait complete. 2023-09-13T03:11:15.220 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:14 smithi064 ceph-mon[97675]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-09-13T03:11:15.220 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:14 smithi064 ceph-mon[97675]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-09-13T03:11:15.220 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:14 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2526007096' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-09-13T03:11:15.220 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:14 smithi064 ceph-mon[97675]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-09-13T03:11:15.221 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:14 smithi064 ceph-mon[97675]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-09-13T03:11:15.221 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:14 smithi064 ceph-mon[97675]: pgmap v279: 97 pgs: 9 creating+peering, 16 unknown, 72 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 4 op/s 2023-09-13T03:11:15.221 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:14 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/1924268439' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-09-13T03:11:15.222 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-09-13T03:11:15.245 INFO:journalctl@ceph.mds.b.smithi142.stdout:Sep 13 03:11:14 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-b[170662]: 2023-09-13T03:11:14.838+0000 7fe0dc505700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-09-13T03:11:15.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:14 smithi142 ceph-mon[134082]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-09-13T03:11:15.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:14 smithi142 ceph-mon[134082]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-09-13T03:11:15.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:14 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/2526007096' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-09-13T03:11:15.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:14 smithi142 ceph-mon[134082]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-09-13T03:11:15.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:14 smithi142 ceph-mon[134082]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-09-13T03:11:15.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:14 smithi142 ceph-mon[134082]: pgmap v279: 97 pgs: 9 creating+peering, 16 unknown, 72 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 4 op/s 2023-09-13T03:11:15.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:14 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/1924268439' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-09-13T03:11:15.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:14 smithi170 ceph-mon[134328]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-09-13T03:11:15.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:14 smithi170 ceph-mon[134328]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-09-13T03:11:15.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:14 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/2526007096' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-09-13T03:11:15.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:14 smithi170 ceph-mon[134328]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-09-13T03:11:15.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:14 smithi170 ceph-mon[134328]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-09-13T03:11:15.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:14 smithi170 ceph-mon[134328]: pgmap v279: 97 pgs: 9 creating+peering, 16 unknown, 72 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 4 op/s 2023-09-13T03:11:15.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:14 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/1924268439' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-09-13T03:11:15.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.428+0000 7f088d3ff700 1 -- 172.21.15.64:0/3244689305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0888101130 msgr2=0x7f0888103520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:15.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.428+0000 7f088d3ff700 1 --2- 172.21.15.64:0/3244689305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0888101130 0x7f0888103520 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0878009990 tx=0x7f0878009ca0 comp rx=0 tx=0).stop 2023-09-13T03:11:15.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.428+0000 7f088d3ff700 1 -- 172.21.15.64:0/3244689305 shutdown_connections 2023-09-13T03:11:15.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.428+0000 7f088d3ff700 1 --2- 172.21.15.64:0/3244689305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0888106390 0x7f0888108780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:15.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.428+0000 7f088d3ff700 1 --2- 172.21.15.64:0/3244689305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0888101130 0x7f0888103520 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:15.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.428+0000 7f088d3ff700 1 --2- 172.21.15.64:0/3244689305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0888103a60 0x7f0888105e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:15.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.428+0000 7f088d3ff700 1 -- 172.21.15.64:0/3244689305 >> 172.21.15.64:0/3244689305 conn(0x7f08880faa60 msgr2=0x7f08880fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:15.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.428+0000 7f088d3ff700 1 -- 172.21.15.64:0/3244689305 shutdown_connections 2023-09-13T03:11:15.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.428+0000 7f088d3ff700 1 -- 172.21.15.64:0/3244689305 wait complete. 2023-09-13T03:11:15.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.429+0000 7f088d3ff700 1 Processor -- start 2023-09-13T03:11:15.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.430+0000 7f088d3ff700 1 -- start start 2023-09-13T03:11:15.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.430+0000 7f088d3ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0888101130 0x7f0888198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:15.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.430+0000 7f088d3ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0888103a60 0x7f0888198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:15.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.430+0000 7f088d3ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0888106390 0x7f088819fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:15.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.430+0000 7f088d3ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08881a0090 con 0x7f0888106390 2023-09-13T03:11:15.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.430+0000 7f088d3ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08881a01d0 con 0x7f0888103a60 2023-09-13T03:11:15.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.430+0000 7f088d3ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08881a0360 con 0x7f0888101130 2023-09-13T03:11:15.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.430+0000 7f08877fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0888106390 0x7f088819fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:15.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.430+0000 7f08867fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0888103a60 0x7f0888198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:15.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.430+0000 7f0886ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0888101130 0x7f0888198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:15.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.430+0000 7f08877fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0888106390 0x7f088819fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57624/0 (socket says 172.21.15.64:57624) 2023-09-13T03:11:15.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.430+0000 7f08877fe700 1 -- 172.21.15.64:0/2563390044 learned_addr learned my addr 172.21.15.64:0/2563390044 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:15.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.431+0000 7f0886ffd700 1 -- 172.21.15.64:0/2563390044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0888103a60 msgr2=0x7f0888198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:15.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.431+0000 7f0886ffd700 1 --2- 172.21.15.64:0/2563390044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0888103a60 0x7f0888198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:15.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.431+0000 7f0886ffd700 1 -- 172.21.15.64:0/2563390044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0888106390 msgr2=0x7f088819fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:15.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.431+0000 7f0886ffd700 1 --2- 172.21.15.64:0/2563390044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0888106390 0x7f088819fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:15.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.431+0000 7f0886ffd700 1 -- 172.21.15.64:0/2563390044 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08780095f0 con 0x7f0888101130 2023-09-13T03:11:15.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.431+0000 7f08877fe700 1 --2- 172.21.15.64:0/2563390044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0888106390 0x7f088819fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:11:15.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.431+0000 7f0886ffd700 1 --2- 172.21.15.64:0/2563390044 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0888101130 0x7f0888198510 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f08780098e0 tx=0x7f0878004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:15.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.431+0000 7f086ffff700 1 -- 172.21.15.64:0/2563390044 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087801d070 con 0x7f0888101130 2023-09-13T03:11:15.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.431+0000 7f088d3ff700 1 -- 172.21.15.64:0/2563390044 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08881a05e0 con 0x7f0888101130 2023-09-13T03:11:15.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.431+0000 7f086ffff700 1 -- 172.21.15.64:0/2563390044 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08780054f0 con 0x7f0888101130 2023-09-13T03:11:15.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.431+0000 7f086ffff700 1 -- 172.21.15.64:0/2563390044 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0878004b90 con 0x7f0888101130 2023-09-13T03:11:15.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.431+0000 7f088d3ff700 1 -- 172.21.15.64:0/2563390044 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08881a0a00 con 0x7f0888101130 2023-09-13T03:11:15.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.432+0000 7f088d3ff700 1 -- 172.21.15.64:0/2563390044 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0888107f10 con 0x7f0888101130 2023-09-13T03:11:15.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.434+0000 7f086ffff700 1 -- 172.21.15.64:0/2563390044 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f0878004d30 con 0x7f0888101130 2023-09-13T03:11:15.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.436+0000 7f086ffff700 1 --2- 172.21.15.64:0/2563390044 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f08700a0640 0x7f08700a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:15.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.436+0000 7f086ffff700 1 -- 172.21.15.64:0/2563390044 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f08780ccdb0 con 0x7f0888101130 2023-09-13T03:11:15.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.436+0000 7f08867fc700 1 --2- 172.21.15.64:0/2563390044 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f08700a0640 0x7f08700a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:15.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.437+0000 7f08867fc700 1 --2- 172.21.15.64:0/2563390044 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f08700a0640 0x7f08700a2b00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f087c009c60 tx=0x7f087c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:15.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.438+0000 7f086ffff700 1 -- 172.21.15.64:0/2563390044 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0878096a20 con 0x7f0888101130 2023-09-13T03:11:15.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:15.699+0000 7f088d3ff700 1 -- 172.21.15.64:0/2563390044 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f088810c0b0 con 0x7f0888101130 2023-09-13T03:11:16.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.205+0000 7f086ffff700 1 -- 172.21.15.64:0/2563390044 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v21) v1 ==== 115+0+0 (secure 0 0 0) 0x7f087809a040 con 0x7f0888101130 2023-09-13T03:11:16.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.208+0000 7f088d3ff700 1 -- 172.21.15.64:0/2563390044 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f08700a0640 msgr2=0x7f08700a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:16.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.208+0000 7f088d3ff700 1 --2- 172.21.15.64:0/2563390044 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f08700a0640 0x7f08700a2b00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f087c009c60 tx=0x7f087c009260 comp rx=0 tx=0).stop 2023-09-13T03:11:16.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.208+0000 7f088d3ff700 1 -- 172.21.15.64:0/2563390044 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0888101130 msgr2=0x7f0888198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:16.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.208+0000 7f088d3ff700 1 --2- 172.21.15.64:0/2563390044 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0888101130 0x7f0888198510 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f08780098e0 tx=0x7f0878004730 comp rx=0 tx=0).stop 2023-09-13T03:11:16.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.208+0000 7f088d3ff700 1 -- 172.21.15.64:0/2563390044 shutdown_connections 2023-09-13T03:11:16.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.208+0000 7f088d3ff700 1 --2- 172.21.15.64:0/2563390044 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f08700a0640 0x7f08700a2b00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:16.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.208+0000 7f088d3ff700 1 --2- 172.21.15.64:0/2563390044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0888106390 0x7f088819fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:16.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.208+0000 7f088d3ff700 1 --2- 172.21.15.64:0/2563390044 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0888101130 0x7f0888198510 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:16.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.208+0000 7f088d3ff700 1 --2- 172.21.15.64:0/2563390044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0888103a60 0x7f0888198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:16.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.208+0000 7f088d3ff700 1 -- 172.21.15.64:0/2563390044 >> 172.21.15.64:0/2563390044 conn(0x7f08880faa60 msgr2=0x7f08880ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:16.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.208+0000 7f088d3ff700 1 -- 172.21.15.64:0/2563390044 shutdown_connections 2023-09-13T03:11:16.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.208+0000 7f088d3ff700 1 -- 172.21.15.64:0/2563390044 wait complete. 2023-09-13T03:11:16.233 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2023-09-13T03:11:16.233 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-09-13T03:11:16.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.440+0000 7fad3e3ff700 1 -- 172.21.15.64:0/3246169557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad38074640 msgr2=0x7fad38072a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:16.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.440+0000 7fad3e3ff700 1 --2- 172.21.15.64:0/3246169557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad38074640 0x7fad38072a90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fad28009990 tx=0x7fad28009ca0 comp rx=0 tx=0).stop 2023-09-13T03:11:16.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.440+0000 7fad3e3ff700 1 -- 172.21.15.64:0/3246169557 shutdown_connections 2023-09-13T03:11:16.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.440+0000 7fad3e3ff700 1 --2- 172.21.15.64:0/3246169557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad381084d0 0x7fad3810a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:16.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.440+0000 7fad3e3ff700 1 --2- 172.21.15.64:0/3246169557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad38074640 0x7fad38072a90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:16.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.440+0000 7fad3e3ff700 1 --2- 172.21.15.64:0/3246169557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad38073060 0x7fad380734e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:16.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.440+0000 7fad3e3ff700 1 -- 172.21.15.64:0/3246169557 >> 172.21.15.64:0/3246169557 conn(0x7fad380fbb80 msgr2=0x7fad380fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:16.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.440+0000 7fad3e3ff700 1 -- 172.21.15.64:0/3246169557 shutdown_connections 2023-09-13T03:11:16.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.440+0000 7fad3e3ff700 1 -- 172.21.15.64:0/3246169557 wait complete. 2023-09-13T03:11:16.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.441+0000 7fad3e3ff700 1 Processor -- start 2023-09-13T03:11:16.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.441+0000 7fad3e3ff700 1 -- start start 2023-09-13T03:11:16.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.442+0000 7fad3e3ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad38073060 0x7fad38198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:16.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.442+0000 7fad3e3ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad38074640 0x7fad38198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:16.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.442+0000 7fad3e3ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad381084d0 0x7fad3819fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:16.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.442+0000 7fad3e3ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad381a00b0 con 0x7fad38073060 2023-09-13T03:11:16.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.442+0000 7fad3e3ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad381a01f0 con 0x7fad38074640 2023-09-13T03:11:16.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.442+0000 7fad3e3ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad381a0380 con 0x7fad381084d0 2023-09-13T03:11:16.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.442+0000 7fad37fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad38073060 0x7fad38198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:16.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.442+0000 7fad37fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad38073060 0x7fad38198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51140/0 (socket says 172.21.15.64:51140) 2023-09-13T03:11:16.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.442+0000 7fad37fff700 1 -- 172.21.15.64:0/3709683565 learned_addr learned my addr 172.21.15.64:0/3709683565 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:16.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.442+0000 7fad377fe700 1 --2- 172.21.15.64:0/3709683565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad38074640 0x7fad38198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:16.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.443+0000 7fad3c99c700 1 --2- 172.21.15.64:0/3709683565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad381084d0 0x7fad3819fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:16.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.443+0000 7fad37fff700 1 -- 172.21.15.64:0/3709683565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad381084d0 msgr2=0x7fad3819fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:16.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.443+0000 7fad37fff700 1 --2- 172.21.15.64:0/3709683565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad381084d0 0x7fad3819fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:16.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.443+0000 7fad37fff700 1 -- 172.21.15.64:0/3709683565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad38074640 msgr2=0x7fad38198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:16.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.443+0000 7fad37fff700 1 --2- 172.21.15.64:0/3709683565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad38074640 0x7fad38198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:16.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.443+0000 7fad37fff700 1 -- 172.21.15.64:0/3709683565 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad280095f0 con 0x7fad38073060 2023-09-13T03:11:16.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.443+0000 7fad3c99c700 1 --2- 172.21.15.64:0/3709683565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad381084d0 0x7fad3819fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T03:11:16.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.443+0000 7fad37fff700 1 --2- 172.21.15.64:0/3709683565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad38073060 0x7fad38198530 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fad2800b680 tx=0x7fad28005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:16.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.443+0000 7fad357fa700 1 -- 172.21.15.64:0/3709683565 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2801d070 con 0x7fad38073060 2023-09-13T03:11:16.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.443+0000 7fad357fa700 1 -- 172.21.15.64:0/3709683565 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad28022470 con 0x7fad38073060 2023-09-13T03:11:16.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.443+0000 7fad3e3ff700 1 -- 172.21.15.64:0/3709683565 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad381a0600 con 0x7fad38073060 2023-09-13T03:11:16.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.443+0000 7fad357fa700 1 -- 172.21.15.64:0/3709683565 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad28017650 con 0x7fad38073060 2023-09-13T03:11:16.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.443+0000 7fad3e3ff700 1 -- 172.21.15.64:0/3709683565 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad381a0a20 con 0x7fad38073060 2023-09-13T03:11:16.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.444+0000 7fad3e3ff700 1 -- 172.21.15.64:0/3709683565 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad38106b80 con 0x7fad38073060 2023-09-13T03:11:16.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.446+0000 7fad357fa700 1 -- 172.21.15.64:0/3709683565 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fad280225e0 con 0x7fad38073060 2023-09-13T03:11:16.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.448+0000 7fad357fa700 1 --2- 172.21.15.64:0/3709683565 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fad200a0640 0x7fad200a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:16.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.448+0000 7fad357fa700 1 -- 172.21.15.64:0/3709683565 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fad280cd720 con 0x7fad38073060 2023-09-13T03:11:16.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.448+0000 7fad377fe700 1 --2- 172.21.15.64:0/3709683565 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fad200a0640 0x7fad200a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:16.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.449+0000 7fad377fe700 1 --2- 172.21.15.64:0/3709683565 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fad200a0640 0x7fad200a2b00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fad38074360 tx=0x7fad2c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:16.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.450+0000 7fad357fa700 1 -- 172.21.15.64:0/3709683565 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fad28097390 con 0x7fad38073060 2023-09-13T03:11:16.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:16 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/1924268439' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-09-13T03:11:16.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:16 smithi142 ceph-mon[134082]: mds.? [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] up:active 2023-09-13T03:11:16.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:16 smithi142 ceph-mon[134082]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-09-13T03:11:16.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:16 smithi142 ceph-mon[134082]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-09-13T03:11:16.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:16 smithi142 ceph-mon[134082]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2023-09-13T03:11:16.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:16 smithi142 ceph-mon[134082]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-09-13T03:11:16.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:16 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/2563390044' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-09-13T03:11:16.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:16 smithi142 ceph-mon[134082]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-09-13T03:11:16.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:16 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/1924268439' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-09-13T03:11:16.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:16 smithi170 ceph-mon[134328]: mds.? [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] up:active 2023-09-13T03:11:16.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:16 smithi170 ceph-mon[134328]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-09-13T03:11:16.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:16 smithi170 ceph-mon[134328]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-09-13T03:11:16.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:16 smithi170 ceph-mon[134328]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2023-09-13T03:11:16.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:16 smithi170 ceph-mon[134328]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-09-13T03:11:16.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:16 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/2563390044' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-09-13T03:11:16.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:16 smithi170 ceph-mon[134328]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-09-13T03:11:16.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:16 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/1924268439' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-09-13T03:11:16.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:16 smithi064 ceph-mon[97675]: mds.? [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] up:active 2023-09-13T03:11:16.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:16 smithi064 ceph-mon[97675]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-09-13T03:11:16.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:16 smithi064 ceph-mon[97675]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-09-13T03:11:16.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:16 smithi064 ceph-mon[97675]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2023-09-13T03:11:16.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:16 smithi064 ceph-mon[97675]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-09-13T03:11:16.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:16 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2563390044' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-09-13T03:11:16.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:16 smithi064 ceph-mon[97675]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-09-13T03:11:16.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:16.696+0000 7fad3e3ff700 1 -- 172.21.15.64:0/3709683565 --> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7fad3802cfe0 con 0x7fad200a0640 2023-09-13T03:11:17.168 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:11:16 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:11:16.703+0000 7fd10b3c0700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-13T03:11:17.168 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:11:16 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:11:16.703+0000 7fd10b3c0700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-13T03:11:17.169 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:11:16 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:11:16.703+0000 7fd10b3c0700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-13T03:11:17.169 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:11:16 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:11:16.703+0000 7fd10b3c0700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-13T03:11:17.169 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:11:16 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:11:16.703+0000 7fd10b3c0700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-13T03:11:17.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:17 smithi142 ceph-mon[134082]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-09-13T03:11:17.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:17 smithi142 ceph-mon[134082]: mds.? [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] up:active 2023-09-13T03:11:17.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:17 smithi142 ceph-mon[134082]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-09-13T03:11:17.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:17 smithi142 ceph-mon[134082]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2023-09-13T03:11:17.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:17 smithi142 ceph-mon[134082]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2023-09-13T03:11:17.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:17 smithi142 ceph-mon[134082]: daemon mds.c is now active in filesystem cephfs as rank 3 2023-09-13T03:11:17.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:17 smithi142 ceph-mon[134082]: pgmap v280: 97 pgs: 97 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s wr, 10 op/s 2023-09-13T03:11:17.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:17 smithi170 ceph-mon[134328]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-09-13T03:11:17.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:17 smithi170 ceph-mon[134328]: mds.? [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] up:active 2023-09-13T03:11:17.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:17 smithi170 ceph-mon[134328]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-09-13T03:11:17.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:17 smithi170 ceph-mon[134328]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2023-09-13T03:11:17.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:17 smithi170 ceph-mon[134328]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2023-09-13T03:11:17.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:17 smithi170 ceph-mon[134328]: daemon mds.c is now active in filesystem cephfs as rank 3 2023-09-13T03:11:17.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:17 smithi170 ceph-mon[134328]: pgmap v280: 97 pgs: 97 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s wr, 10 op/s 2023-09-13T03:11:17.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:17 smithi064 ceph-mon[97675]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-09-13T03:11:17.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:17 smithi064 ceph-mon[97675]: mds.? [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] up:active 2023-09-13T03:11:17.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:17 smithi064 ceph-mon[97675]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-09-13T03:11:17.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:17 smithi064 ceph-mon[97675]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2023-09-13T03:11:17.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:17 smithi064 ceph-mon[97675]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2023-09-13T03:11:17.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:17 smithi064 ceph-mon[97675]: daemon mds.c is now active in filesystem cephfs as rank 3 2023-09-13T03:11:17.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:17 smithi064 ceph-mon[97675]: pgmap v280: 97 pgs: 97 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s wr, 10 op/s 2023-09-13T03:11:18.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.261+0000 7fad357fa700 1 -- 172.21.15.64:0/3709683565 <== mgr.14150 v2:172.21.15.64:6800/873679308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fad3802cfe0 con 0x7fad200a0640 2023-09-13T03:11:18.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.264+0000 7fad3e3ff700 1 -- 172.21.15.64:0/3709683565 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fad200a0640 msgr2=0x7fad200a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:18.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.264+0000 7fad3e3ff700 1 --2- 172.21.15.64:0/3709683565 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fad200a0640 0x7fad200a2b00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fad38074360 tx=0x7fad2c009260 comp rx=0 tx=0).stop 2023-09-13T03:11:18.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.264+0000 7fad3e3ff700 1 -- 172.21.15.64:0/3709683565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad38073060 msgr2=0x7fad38198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:18.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.264+0000 7fad3e3ff700 1 --2- 172.21.15.64:0/3709683565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad38073060 0x7fad38198530 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fad2800b680 tx=0x7fad28005740 comp rx=0 tx=0).stop 2023-09-13T03:11:18.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.265+0000 7fad3e3ff700 1 -- 172.21.15.64:0/3709683565 shutdown_connections 2023-09-13T03:11:18.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.265+0000 7fad3e3ff700 1 --2- 172.21.15.64:0/3709683565 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fad200a0640 0x7fad200a2b00 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:18.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.265+0000 7fad3e3ff700 1 --2- 172.21.15.64:0/3709683565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad381084d0 0x7fad3819fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:18.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.265+0000 7fad3e3ff700 1 --2- 172.21.15.64:0/3709683565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad38073060 0x7fad38198530 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:18.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.265+0000 7fad3e3ff700 1 --2- 172.21.15.64:0/3709683565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad38074640 0x7fad38198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:18.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.265+0000 7fad3e3ff700 1 -- 172.21.15.64:0/3709683565 >> 172.21.15.64:0/3709683565 conn(0x7fad380fbb80 msgr2=0x7fad380fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:18.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.265+0000 7fad3e3ff700 1 -- 172.21.15.64:0/3709683565 shutdown_connections 2023-09-13T03:11:18.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.265+0000 7fad3e3ff700 1 -- 172.21.15.64:0/3709683565 wait complete. 2023-09-13T03:11:18.287 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-09-13T03:11:18.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:18 smithi142 ceph-mon[134082]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:11:18.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:18 smithi142 ceph-mon[134082]: mds.? [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] up:active 2023-09-13T03:11:18.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:18 smithi142 ceph-mon[134082]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-09-13T03:11:18.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:18 smithi142 ceph-mon[134082]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-09-13T03:11:18.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:18 smithi142 ceph-mon[134082]: Cluster is now healthy 2023-09-13T03:11:18.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:18 smithi142 ceph-mon[134082]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2023-09-13T03:11:18.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:18 smithi142 ceph-mon[134082]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2023-09-13T03:11:18.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:18 smithi142 ceph-mon[134082]: daemon mds.f is now active in filesystem cephfs as rank 4 2023-09-13T03:11:18.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.497+0000 7f9d1fbff700 1 -- 172.21.15.64:0/2114600749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d18073060 msgr2=0x7f9d180734e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:18.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.497+0000 7f9d1fbff700 1 --2- 172.21.15.64:0/2114600749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d18073060 0x7f9d180734e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9d080098f0 tx=0x7f9d08009c00 comp rx=0 tx=0).stop 2023-09-13T03:11:18.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.497+0000 7f9d1fbff700 1 -- 172.21.15.64:0/2114600749 shutdown_connections 2023-09-13T03:11:18.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.497+0000 7f9d1fbff700 1 --2- 172.21.15.64:0/2114600749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d181084b0 0x7f9d1810a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:18.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.497+0000 7f9d1fbff700 1 --2- 172.21.15.64:0/2114600749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d18074640 0x7f9d18072a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:18.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.497+0000 7f9d1fbff700 1 --2- 172.21.15.64:0/2114600749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d18073060 0x7f9d180734e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:18.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.497+0000 7f9d1fbff700 1 -- 172.21.15.64:0/2114600749 >> 172.21.15.64:0/2114600749 conn(0x7f9d180fbb40 msgr2=0x7f9d180fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:18.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.497+0000 7f9d1fbff700 1 -- 172.21.15.64:0/2114600749 shutdown_connections 2023-09-13T03:11:18.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.497+0000 7f9d1fbff700 1 -- 172.21.15.64:0/2114600749 wait complete. 2023-09-13T03:11:18.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.499+0000 7f9d1fbff700 1 Processor -- start 2023-09-13T03:11:18.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.499+0000 7f9d1fbff700 1 -- start start 2023-09-13T03:11:18.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.499+0000 7f9d1fbff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d18073060 0x7f9d18198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:18.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.499+0000 7f9d1fbff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d18074640 0x7f9d18198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:18.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.500+0000 7f9d1fbff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d181084b0 0x7f9d1819fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:18.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.500+0000 7f9d1fbff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d181a0090 con 0x7f9d18073060 2023-09-13T03:11:18.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.500+0000 7f9d1fbff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d181a01d0 con 0x7f9d181084b0 2023-09-13T03:11:18.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.500+0000 7f9d1fbff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d181a0360 con 0x7f9d18074640 2023-09-13T03:11:18.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.500+0000 7f9d1d99b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d18073060 0x7f9d18198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:18.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.500+0000 7f9d1d99b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d18073060 0x7f9d18198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51166/0 (socket says 172.21.15.64:51166) 2023-09-13T03:11:18.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.500+0000 7f9d1d99b700 1 -- 172.21.15.64:0/1516016870 learned_addr learned my addr 172.21.15.64:0/1516016870 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:18.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.500+0000 7f9d1e19c700 1 --2- 172.21.15.64:0/1516016870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d181084b0 0x7f9d1819fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:18.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.500+0000 7f9d1d19a700 1 --2- 172.21.15.64:0/1516016870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d18074640 0x7f9d18198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:18.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.500+0000 7f9d1d99b700 1 -- 172.21.15.64:0/1516016870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d18074640 msgr2=0x7f9d18198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:18.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.500+0000 7f9d1d99b700 1 --2- 172.21.15.64:0/1516016870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d18074640 0x7f9d18198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:18.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.500+0000 7f9d1d99b700 1 -- 172.21.15.64:0/1516016870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d181084b0 msgr2=0x7f9d1819fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:18.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.500+0000 7f9d1d99b700 1 --2- 172.21.15.64:0/1516016870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d181084b0 0x7f9d1819fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:18.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.500+0000 7f9d1d99b700 1 -- 172.21.15.64:0/1516016870 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d08009520 con 0x7f9d18073060 2023-09-13T03:11:18.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.501+0000 7f9d1d99b700 1 --2- 172.21.15.64:0/1516016870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d18073060 0x7f9d18198510 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400e9c0 tx=0x7f9d1400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:18.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.501+0000 7f9d0effd700 1 -- 172.21.15.64:0/1516016870 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d14004a30 con 0x7f9d18073060 2023-09-13T03:11:18.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.501+0000 7f9d1fbff700 1 -- 172.21.15.64:0/1516016870 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d181a0640 con 0x7f9d18073060 2023-09-13T03:11:18.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.501+0000 7f9d0effd700 1 -- 172.21.15.64:0/1516016870 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d1400cc80 con 0x7f9d18073060 2023-09-13T03:11:18.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.501+0000 7f9d0effd700 1 -- 172.21.15.64:0/1516016870 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d140054e0 con 0x7f9d18073060 2023-09-13T03:11:18.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.501+0000 7f9d1fbff700 1 -- 172.21.15.64:0/1516016870 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d181a0b50 con 0x7f9d18073060 2023-09-13T03:11:18.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.502+0000 7f9d1fbff700 1 -- 172.21.15.64:0/1516016870 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d18106b60 con 0x7f9d18073060 2023-09-13T03:11:18.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.504+0000 7f9d0effd700 1 -- 172.21.15.64:0/1516016870 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f9d14004b90 con 0x7f9d18073060 2023-09-13T03:11:18.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.505+0000 7f9d0effd700 1 --2- 172.21.15.64:0/1516016870 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f9d040a0640 0x7f9d040a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:18.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.505+0000 7f9d0effd700 1 -- 172.21.15.64:0/1516016870 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9d14014070 con 0x7f9d18073060 2023-09-13T03:11:18.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.506+0000 7f9d1d19a700 1 --2- 172.21.15.64:0/1516016870 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f9d040a0640 0x7f9d040a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:18.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.506+0000 7f9d1d19a700 1 --2- 172.21.15.64:0/1516016870 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f9d040a0640 0x7f9d040a2b00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9d08009fd0 tx=0x7f9d08019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:18.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.507+0000 7f9d0effd700 1 -- 172.21.15.64:0/1516016870 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f9d14095940 con 0x7f9d18073060 2023-09-13T03:11:18.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:18 smithi170 ceph-mon[134328]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:11:18.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:18 smithi170 ceph-mon[134328]: mds.? [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] up:active 2023-09-13T03:11:18.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:18 smithi170 ceph-mon[134328]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-09-13T03:11:18.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:18 smithi170 ceph-mon[134328]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-09-13T03:11:18.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:18 smithi170 ceph-mon[134328]: Cluster is now healthy 2023-09-13T03:11:18.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:18 smithi170 ceph-mon[134328]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2023-09-13T03:11:18.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:18 smithi170 ceph-mon[134328]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2023-09-13T03:11:18.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:18 smithi170 ceph-mon[134328]: daemon mds.f is now active in filesystem cephfs as rank 4 2023-09-13T03:11:18.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:18 smithi064 ceph-mon[97675]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:11:18.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:18 smithi064 ceph-mon[97675]: mds.? [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] up:active 2023-09-13T03:11:18.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:18 smithi064 ceph-mon[97675]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-09-13T03:11:18.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:18 smithi064 ceph-mon[97675]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-09-13T03:11:18.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:18 smithi064 ceph-mon[97675]: Cluster is now healthy 2023-09-13T03:11:18.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:18 smithi064 ceph-mon[97675]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2023-09-13T03:11:18.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:18 smithi064 ceph-mon[97675]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2023-09-13T03:11:18.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:18 smithi064 ceph-mon[97675]: daemon mds.f is now active in filesystem cephfs as rank 4 2023-09-13T03:11:18.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.753+0000 7f9d1fbff700 1 -- 172.21.15.64:0/1516016870 --> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d1802cfd0 con 0x7f9d040a0640 2023-09-13T03:11:18.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.757+0000 7f9d0effd700 1 -- 172.21.15.64:0/1516016870 <== mgr.14150 v2:172.21.15.64:6800/873679308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f9d1802cfd0 con 0x7f9d040a0640 2023-09-13T03:11:18.757 INFO:teuthology.orchestra.run.smithi064.stdout:/volumes/_nogroup/sv_0/4f43bb84-d68c-4d30-a21d-380ea033387a 2023-09-13T03:11:18.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.759+0000 7f9d1fbff700 1 -- 172.21.15.64:0/1516016870 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f9d040a0640 msgr2=0x7f9d040a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:18.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.759+0000 7f9d1fbff700 1 --2- 172.21.15.64:0/1516016870 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f9d040a0640 0x7f9d040a2b00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9d08009fd0 tx=0x7f9d08019040 comp rx=0 tx=0).stop 2023-09-13T03:11:18.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.759+0000 7f9d1fbff700 1 -- 172.21.15.64:0/1516016870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d18073060 msgr2=0x7f9d18198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:18.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.759+0000 7f9d1fbff700 1 --2- 172.21.15.64:0/1516016870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d18073060 0x7f9d18198510 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400e9c0 tx=0x7f9d1400ecd0 comp rx=0 tx=0).stop 2023-09-13T03:11:18.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.759+0000 7f9d1fbff700 1 -- 172.21.15.64:0/1516016870 shutdown_connections 2023-09-13T03:11:18.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.759+0000 7f9d1fbff700 1 --2- 172.21.15.64:0/1516016870 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f9d040a0640 0x7f9d040a2b00 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:18.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.759+0000 7f9d1fbff700 1 --2- 172.21.15.64:0/1516016870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d181084b0 0x7f9d1819fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:18.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.759+0000 7f9d1fbff700 1 --2- 172.21.15.64:0/1516016870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d18073060 0x7f9d18198510 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:18.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.759+0000 7f9d1fbff700 1 --2- 172.21.15.64:0/1516016870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d18074640 0x7f9d18198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:18.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.759+0000 7f9d1fbff700 1 -- 172.21.15.64:0/1516016870 >> 172.21.15.64:0/1516016870 conn(0x7f9d180fbb40 msgr2=0x7f9d180fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:18.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.759+0000 7f9d1fbff700 1 -- 172.21.15.64:0/1516016870 shutdown_connections 2023-09-13T03:11:18.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.759+0000 7f9d1fbff700 1 -- 172.21.15.64:0/1516016870 wait complete. 2023-09-13T03:11:18.778 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-09-13T03:11:18.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.989+0000 7f60da5ff700 1 -- 172.21.15.64:0/4208806712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60d4074640 msgr2=0x7f60d4072a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:18.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.989+0000 7f60da5ff700 1 --2- 172.21.15.64:0/4208806712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60d4074640 0x7f60d4072a90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f60c4009990 tx=0x7f60c4009ca0 comp rx=0 tx=0).stop 2023-09-13T03:11:18.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.990+0000 7f60da5ff700 1 -- 172.21.15.64:0/4208806712 shutdown_connections 2023-09-13T03:11:18.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.990+0000 7f60da5ff700 1 --2- 172.21.15.64:0/4208806712 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60d410cbe0 0x7f60d410efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:18.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.990+0000 7f60da5ff700 1 --2- 172.21.15.64:0/4208806712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60d4074640 0x7f60d4072a90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:18.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.990+0000 7f60da5ff700 1 --2- 172.21.15.64:0/4208806712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60d4073060 0x7f60d40734e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:18.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.990+0000 7f60da5ff700 1 -- 172.21.15.64:0/4208806712 >> 172.21.15.64:0/4208806712 conn(0x7f60d40fbb80 msgr2=0x7f60d40fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:18.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.990+0000 7f60da5ff700 1 -- 172.21.15.64:0/4208806712 shutdown_connections 2023-09-13T03:11:18.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.990+0000 7f60da5ff700 1 -- 172.21.15.64:0/4208806712 wait complete. 2023-09-13T03:11:18.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.991+0000 7f60da5ff700 1 Processor -- start 2023-09-13T03:11:18.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.991+0000 7f60da5ff700 1 -- start start 2023-09-13T03:11:18.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.991+0000 7f60da5ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60d4073060 0x7f60d419c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:18.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.991+0000 7f60da5ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60d4074640 0x7f60d419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:18.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.991+0000 7f60da5ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60d410cbe0 0x7f60d41a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:18.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.991+0000 7f60da5ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d41a44c0 con 0x7f60d410cbe0 2023-09-13T03:11:18.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.991+0000 7f60da5ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d41a4600 con 0x7f60d4074640 2023-09-13T03:11:18.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.991+0000 7f60da5ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d41a4790 con 0x7f60d4073060 2023-09-13T03:11:18.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.991+0000 7f60d8b9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60d410cbe0 0x7f60d41a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:18.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.991+0000 7f60d3fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60d4073060 0x7f60d419c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:18.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.991+0000 7f60d8b9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60d410cbe0 0x7f60d41a3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51190/0 (socket says 172.21.15.64:51190) 2023-09-13T03:11:18.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.991+0000 7f60d8b9c700 1 -- 172.21.15.64:0/1310211028 learned_addr learned my addr 172.21.15.64:0/1310211028 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:18.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.991+0000 7f60d37fe700 1 --2- 172.21.15.64:0/1310211028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60d4074640 0x7f60d419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:18.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.992+0000 7f60d8b9c700 1 -- 172.21.15.64:0/1310211028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60d4073060 msgr2=0x7f60d419c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:18.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.992+0000 7f60d8b9c700 1 --2- 172.21.15.64:0/1310211028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60d4073060 0x7f60d419c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:18.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.992+0000 7f60d8b9c700 1 -- 172.21.15.64:0/1310211028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60d4074640 msgr2=0x7f60d419ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:18.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.992+0000 7f60d8b9c700 1 --2- 172.21.15.64:0/1310211028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60d4074640 0x7f60d419ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:18.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.992+0000 7f60d8b9c700 1 -- 172.21.15.64:0/1310211028 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c40095f0 con 0x7f60d410cbe0 2023-09-13T03:11:18.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.992+0000 7f60d8b9c700 1 --2- 172.21.15.64:0/1310211028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60d410cbe0 0x7f60d41a3ef0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f60cc00ef30 tx=0x7f60cc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:18.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.992+0000 7f60d17fa700 1 -- 172.21.15.64:0/1310211028 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60cc018070 con 0x7f60d410cbe0 2023-09-13T03:11:18.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.992+0000 7f60d17fa700 1 -- 172.21.15.64:0/1310211028 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60cc00f040 con 0x7f60d410cbe0 2023-09-13T03:11:18.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.992+0000 7f60da5ff700 1 -- 172.21.15.64:0/1310211028 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60d41a4a10 con 0x7f60d410cbe0 2023-09-13T03:11:19.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.992+0000 7f60d17fa700 1 -- 172.21.15.64:0/1310211028 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60cc00cd00 con 0x7f60d410cbe0 2023-09-13T03:11:19.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.992+0000 7f60da5ff700 1 -- 172.21.15.64:0/1310211028 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60d41a4f20 con 0x7f60d410cbe0 2023-09-13T03:11:19.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.994+0000 7f60da5ff700 1 -- 172.21.15.64:0/1310211028 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60d4102730 con 0x7f60d410cbe0 2023-09-13T03:11:19.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.996+0000 7f60d17fa700 1 -- 172.21.15.64:0/1310211028 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f60cc003a00 con 0x7f60d410cbe0 2023-09-13T03:11:19.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.998+0000 7f60d17fa700 1 --2- 172.21.15.64:0/1310211028 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f60bc0a06c0 0x7f60bc0a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:19.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.998+0000 7f60d17fa700 1 -- 172.21.15.64:0/1310211028 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f60cc0cb5c0 con 0x7f60d410cbe0 2023-09-13T03:11:19.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.998+0000 7f60d3fff700 1 --2- 172.21.15.64:0/1310211028 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f60bc0a06c0 0x7f60bc0a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:19.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:18.999+0000 7f60d3fff700 1 --2- 172.21.15.64:0/1310211028 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f60bc0a06c0 0x7f60bc0a2b80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f60c400b2e0 tx=0x7f60c4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:19.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.002+0000 7f60d17fa700 1 -- 172.21.15.64:0/1310211028 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f60cc095230 con 0x7f60d410cbe0 2023-09-13T03:11:19.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.252+0000 7f60d17fa700 1 -- 172.21.15.64:0/1310211028 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f60cc003cb0 con 0x7f60d410cbe0 2023-09-13T03:11:19.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.257+0000 7f60da5ff700 1 -- 172.21.15.64:0/1310211028 --> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f60d402d0c0 con 0x7f60bc0a06c0 2023-09-13T03:11:19.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.285+0000 7f60d17fa700 1 -- 172.21.15.64:0/1310211028 <== mgr.14150 v2:172.21.15.64:6800/873679308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f60d402d0c0 con 0x7f60bc0a06c0 2023-09-13T03:11:19.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.287+0000 7f60da5ff700 1 -- 172.21.15.64:0/1310211028 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f60bc0a06c0 msgr2=0x7f60bc0a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:19.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.287+0000 7f60da5ff700 1 --2- 172.21.15.64:0/1310211028 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f60bc0a06c0 0x7f60bc0a2b80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f60c400b2e0 tx=0x7f60c4005fb0 comp rx=0 tx=0).stop 2023-09-13T03:11:19.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.287+0000 7f60da5ff700 1 -- 172.21.15.64:0/1310211028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60d410cbe0 msgr2=0x7f60d41a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:19.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.287+0000 7f60da5ff700 1 --2- 172.21.15.64:0/1310211028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60d410cbe0 0x7f60d41a3ef0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f60cc00ef30 tx=0x7f60cc00c3a0 comp rx=0 tx=0).stop 2023-09-13T03:11:19.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.287+0000 7f60da5ff700 1 -- 172.21.15.64:0/1310211028 shutdown_connections 2023-09-13T03:11:19.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.287+0000 7f60da5ff700 1 --2- 172.21.15.64:0/1310211028 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f60bc0a06c0 0x7f60bc0a2b80 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:19.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.287+0000 7f60da5ff700 1 --2- 172.21.15.64:0/1310211028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60d410cbe0 0x7f60d41a3ef0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:19.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.287+0000 7f60da5ff700 1 --2- 172.21.15.64:0/1310211028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60d4073060 0x7f60d419c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:19.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.287+0000 7f60da5ff700 1 --2- 172.21.15.64:0/1310211028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60d4074640 0x7f60d419ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:19.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.288+0000 7f60da5ff700 1 -- 172.21.15.64:0/1310211028 >> 172.21.15.64:0/1310211028 conn(0x7f60d40fbb80 msgr2=0x7f60d40fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:19.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.288+0000 7f60da5ff700 1 -- 172.21.15.64:0/1310211028 shutdown_connections 2023-09-13T03:11:19.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.288+0000 7f60da5ff700 1 -- 172.21.15.64:0/1310211028 wait complete. 2023-09-13T03:11:19.307 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-09-13T03:11:19.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:19 smithi142 ceph-mon[134082]: mds.? [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] up:active 2023-09-13T03:11:19.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:19 smithi142 ceph-mon[134082]: fsmap cephfs:5 7 up:standby 5 up:active 2023-09-13T03:11:19.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:19 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:11:19.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:19 smithi142 ceph-mon[134082]: pgmap v281: 97 pgs: 97 active+clean; 583 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s wr, 17 op/s 2023-09-13T03:11:19.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.514+0000 7f90d23ff700 1 -- 172.21.15.64:0/3257913337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90cc100610 msgr2=0x7f90cc100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:19.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.514+0000 7f90d23ff700 1 --2- 172.21.15.64:0/3257913337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90cc100610 0x7f90cc100a30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f90bc0098f0 tx=0x7f90bc009c00 comp rx=0 tx=0).stop 2023-09-13T03:11:19.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.514+0000 7f90d23ff700 1 -- 172.21.15.64:0/3257913337 shutdown_connections 2023-09-13T03:11:19.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.514+0000 7f90d23ff700 1 --2- 172.21.15.64:0/3257913337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90cc1021a0 0x7f90cc108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:19.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.514+0000 7f90d23ff700 1 --2- 172.21.15.64:0/3257913337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90cc100610 0x7f90cc100a30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:19.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.514+0000 7f90d23ff700 1 --2- 172.21.15.64:0/3257913337 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90cc101800 0x7f90cc101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:19.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.514+0000 7f90d23ff700 1 -- 172.21.15.64:0/3257913337 >> 172.21.15.64:0/3257913337 conn(0x7f90cc0fbb80 msgr2=0x7f90cc0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:19.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.514+0000 7f90d23ff700 1 -- 172.21.15.64:0/3257913337 shutdown_connections 2023-09-13T03:11:19.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.514+0000 7f90d23ff700 1 -- 172.21.15.64:0/3257913337 wait complete. 2023-09-13T03:11:19.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.515+0000 7f90d23ff700 1 Processor -- start 2023-09-13T03:11:19.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.515+0000 7f90d23ff700 1 -- start start 2023-09-13T03:11:19.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.516+0000 7f90d23ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90cc100610 0x7f90cc0743b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:19.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.516+0000 7f90d23ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90cc101800 0x7f90cc0729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:19.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.516+0000 7f90d23ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90cc1021a0 0x7f90cc072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:19.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.516+0000 7f90d23ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc0736d0 con 0x7f90cc1021a0 2023-09-13T03:11:19.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.516+0000 7f90d23ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc073840 con 0x7f90cc101800 2023-09-13T03:11:19.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.516+0000 7f90d23ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc1a03f0 con 0x7f90cc100610 2023-09-13T03:11:19.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.516+0000 7f90cbfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90cc100610 0x7f90cc0743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:19.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.516+0000 7f90cbfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90cc100610 0x7f90cc0743b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:35970/0 (socket says 172.21.15.64:35970) 2023-09-13T03:11:19.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.516+0000 7f90cb7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90cc101800 0x7f90cc0729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:19.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.516+0000 7f90cbfff700 1 -- 172.21.15.64:0/478596532 learned_addr learned my addr 172.21.15.64:0/478596532 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:19.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.516+0000 7f90d099c700 1 --2- 172.21.15.64:0/478596532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90cc1021a0 0x7f90cc072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:19.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.516+0000 7f90cbfff700 1 -- 172.21.15.64:0/478596532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90cc101800 msgr2=0x7f90cc0729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:19.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.516+0000 7f90cbfff700 1 --2- 172.21.15.64:0/478596532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90cc101800 0x7f90cc0729f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:19.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.516+0000 7f90cbfff700 1 -- 172.21.15.64:0/478596532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90cc1021a0 msgr2=0x7f90cc072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:19.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.516+0000 7f90cbfff700 1 --2- 172.21.15.64:0/478596532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90cc1021a0 0x7f90cc072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:19.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.516+0000 7f90cbfff700 1 -- 172.21.15.64:0/478596532 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c00095f0 con 0x7f90cc100610 2023-09-13T03:11:19.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.517+0000 7f90cbfff700 1 --2- 172.21.15.64:0/478596532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90cc100610 0x7f90cc0743b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f90bc009fd0 tx=0x7f90bc00b560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:19.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.517+0000 7f90c97fa700 1 -- 172.21.15.64:0/478596532 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90bc01c070 con 0x7f90cc100610 2023-09-13T03:11:19.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.517+0000 7f90c97fa700 1 -- 172.21.15.64:0/478596532 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90bc00bc10 con 0x7f90cc100610 2023-09-13T03:11:19.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.517+0000 7f90d23ff700 1 -- 172.21.15.64:0/478596532 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90bc009520 con 0x7f90cc100610 2023-09-13T03:11:19.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.517+0000 7f90c97fa700 1 -- 172.21.15.64:0/478596532 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90bc003990 con 0x7f90cc100610 2023-09-13T03:11:19.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.517+0000 7f90d23ff700 1 -- 172.21.15.64:0/478596532 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90cc1a0900 con 0x7f90cc100610 2023-09-13T03:11:19.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.518+0000 7f90d23ff700 1 -- 172.21.15.64:0/478596532 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90cc107590 con 0x7f90cc100610 2023-09-13T03:11:19.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.520+0000 7f90c97fa700 1 -- 172.21.15.64:0/478596532 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f90bc020030 con 0x7f90cc100610 2023-09-13T03:11:19.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.521+0000 7f90c97fa700 1 --2- 172.21.15.64:0/478596532 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f90b40a06d0 0x7f90b40a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:19.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.521+0000 7f90c97fa700 1 -- 172.21.15.64:0/478596532 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f90bc0ccc60 con 0x7f90cc100610 2023-09-13T03:11:19.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.522+0000 7f90cb7fe700 1 --2- 172.21.15.64:0/478596532 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f90b40a06d0 0x7f90b40a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:19.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.522+0000 7f90cb7fe700 1 --2- 172.21.15.64:0/478596532 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f90b40a06d0 0x7f90b40a2b90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f90c0005fd0 tx=0x7f90c0009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:19.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.525+0000 7f90c97fa700 1 -- 172.21.15.64:0/478596532 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f90bc096770 con 0x7f90cc100610 2023-09-13T03:11:19.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:19 smithi170 ceph-mon[134328]: mds.? [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] up:active 2023-09-13T03:11:19.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:19 smithi170 ceph-mon[134328]: fsmap cephfs:5 7 up:standby 5 up:active 2023-09-13T03:11:19.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:19 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:11:19.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:19 smithi170 ceph-mon[134328]: pgmap v281: 97 pgs: 97 active+clean; 583 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s wr, 17 op/s 2023-09-13T03:11:19.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:19 smithi064 ceph-mon[97675]: mds.? [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] up:active 2023-09-13T03:11:19.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:19 smithi064 ceph-mon[97675]: fsmap cephfs:5 7 up:standby 5 up:active 2023-09-13T03:11:19.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:19 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:11:19.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:19 smithi064 ceph-mon[97675]: pgmap v281: 97 pgs: 97 active+clean; 583 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s wr, 17 op/s 2023-09-13T03:11:19.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.771+0000 7f90d23ff700 1 -- 172.21.15.64:0/478596532 --> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f90cc02d030 con 0x7f90b40a06d0 2023-09-13T03:11:19.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.776+0000 7f90c97fa700 1 -- 172.21.15.64:0/478596532 <== mgr.14150 v2:172.21.15.64:6800/873679308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f90cc02d030 con 0x7f90b40a06d0 2023-09-13T03:11:19.776 INFO:teuthology.orchestra.run.smithi064.stdout:/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed 2023-09-13T03:11:19.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.778+0000 7f90d23ff700 1 -- 172.21.15.64:0/478596532 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f90b40a06d0 msgr2=0x7f90b40a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:19.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.778+0000 7f90d23ff700 1 --2- 172.21.15.64:0/478596532 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f90b40a06d0 0x7f90b40a2b90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f90c0005fd0 tx=0x7f90c0009310 comp rx=0 tx=0).stop 2023-09-13T03:11:19.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.778+0000 7f90d23ff700 1 -- 172.21.15.64:0/478596532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90cc100610 msgr2=0x7f90cc0743b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:19.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.778+0000 7f90d23ff700 1 --2- 172.21.15.64:0/478596532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90cc100610 0x7f90cc0743b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f90bc009fd0 tx=0x7f90bc00b560 comp rx=0 tx=0).stop 2023-09-13T03:11:19.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.778+0000 7f90d23ff700 1 -- 172.21.15.64:0/478596532 shutdown_connections 2023-09-13T03:11:19.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.778+0000 7f90d23ff700 1 --2- 172.21.15.64:0/478596532 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f90b40a06d0 0x7f90b40a2b90 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:19.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.778+0000 7f90d23ff700 1 --2- 172.21.15.64:0/478596532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90cc1021a0 0x7f90cc072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:19.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.778+0000 7f90d23ff700 1 --2- 172.21.15.64:0/478596532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90cc100610 0x7f90cc0743b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:19.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.778+0000 7f90d23ff700 1 --2- 172.21.15.64:0/478596532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90cc101800 0x7f90cc0729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:19.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.778+0000 7f90d23ff700 1 -- 172.21.15.64:0/478596532 >> 172.21.15.64:0/478596532 conn(0x7f90cc0fbb80 msgr2=0x7f90cc0fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:19.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.779+0000 7f90d23ff700 1 -- 172.21.15.64:0/478596532 shutdown_connections 2023-09-13T03:11:19.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:19.779+0000 7f90d23ff700 1 -- 172.21.15.64:0/478596532 wait complete. 2023-09-13T03:11:19.797 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T03:11:20.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.001+0000 7f98d1bff700 1 -- 172.21.15.64:0/1771882749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98cc101800 msgr2=0x7f98cc101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:20.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.001+0000 7f98d1bff700 1 --2- 172.21.15.64:0/1771882749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98cc101800 0x7f98cc101c60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f98c00099c0 tx=0x7f98c0009cd0 comp rx=0 tx=0).stop 2023-09-13T03:11:20.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.002+0000 7f98d1bff700 1 -- 172.21.15.64:0/1771882749 shutdown_connections 2023-09-13T03:11:20.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.002+0000 7f98d1bff700 1 --2- 172.21.15.64:0/1771882749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98cc100610 0x7f98cc100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:20.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.002+0000 7f98d1bff700 1 --2- 172.21.15.64:0/1771882749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98cc1021a0 0x7f98cc108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:20.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.002+0000 7f98d1bff700 1 --2- 172.21.15.64:0/1771882749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98cc101800 0x7f98cc101c60 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:20.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.002+0000 7f98d1bff700 1 -- 172.21.15.64:0/1771882749 >> 172.21.15.64:0/1771882749 conn(0x7f98cc0fbb40 msgr2=0x7f98cc0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:20.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.002+0000 7f98d1bff700 1 -- 172.21.15.64:0/1771882749 shutdown_connections 2023-09-13T03:11:20.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.002+0000 7f98d1bff700 1 -- 172.21.15.64:0/1771882749 wait complete. 2023-09-13T03:11:20.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.003+0000 7f98d1bff700 1 Processor -- start 2023-09-13T03:11:20.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.003+0000 7f98d1bff700 1 -- start start 2023-09-13T03:11:20.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.003+0000 7f98d1bff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98cc100610 0x7f98cc1962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:20.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.003+0000 7f98d1bff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98cc101800 0x7f98cc1967f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:20.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.003+0000 7f98d1bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98cc1021a0 0x7f98cc19d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:20.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.003+0000 7f98d1bff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc19de30 con 0x7f98cc101800 2023-09-13T03:11:20.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.003+0000 7f98d1bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc19df70 con 0x7f98cc1021a0 2023-09-13T03:11:20.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.003+0000 7f98d1bff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc19e100 con 0x7f98cc100610 2023-09-13T03:11:20.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.003+0000 7f98caffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98cc101800 0x7f98cc1967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:20.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.003+0000 7f98cb7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98cc100610 0x7f98cc1962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:20.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.003+0000 7f98cb7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98cc100610 0x7f98cc1962b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:35990/0 (socket says 172.21.15.64:35990) 2023-09-13T03:11:20.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.003+0000 7f98cb7fe700 1 -- 172.21.15.64:0/526453532 learned_addr learned my addr 172.21.15.64:0/526453532 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:20.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.004+0000 7f98cbfff700 1 --2- 172.21.15.64:0/526453532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98cc1021a0 0x7f98cc19d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:20.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.004+0000 7f98caffd700 1 -- 172.21.15.64:0/526453532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98cc100610 msgr2=0x7f98cc1962b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:20.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.004+0000 7f98caffd700 1 --2- 172.21.15.64:0/526453532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98cc100610 0x7f98cc1962b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:20.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.004+0000 7f98caffd700 1 -- 172.21.15.64:0/526453532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98cc1021a0 msgr2=0x7f98cc19d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:20.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.004+0000 7f98caffd700 1 --2- 172.21.15.64:0/526453532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98cc1021a0 0x7f98cc19d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:20.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.004+0000 7f98caffd700 1 -- 172.21.15.64:0/526453532 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c00095f0 con 0x7f98cc101800 2023-09-13T03:11:20.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.004+0000 7f98caffd700 1 --2- 172.21.15.64:0/526453532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98cc101800 0x7f98cc1967f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f98c00098e0 tx=0x7f98c000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:20.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.004+0000 7f98c8ff9700 1 -- 172.21.15.64:0/526453532 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c001d070 con 0x7f98cc101800 2023-09-13T03:11:20.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.004+0000 7f98c8ff9700 1 -- 172.21.15.64:0/526453532 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98c0005020 con 0x7f98cc101800 2023-09-13T03:11:20.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.004+0000 7f98d1bff700 1 -- 172.21.15.64:0/526453532 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98cc19e380 con 0x7f98cc101800 2023-09-13T03:11:20.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.004+0000 7f98c8ff9700 1 -- 172.21.15.64:0/526453532 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c0004a70 con 0x7f98cc101800 2023-09-13T03:11:20.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.005+0000 7f98d1bff700 1 -- 172.21.15.64:0/526453532 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98cc19e740 con 0x7f98cc101800 2023-09-13T03:11:20.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.006+0000 7f98d1bff700 1 -- 172.21.15.64:0/526453532 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98cc107b80 con 0x7f98cc101800 2023-09-13T03:11:20.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.008+0000 7f98c8ff9700 1 -- 172.21.15.64:0/526453532 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f98c0005400 con 0x7f98cc101800 2023-09-13T03:11:20.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.009+0000 7f98c8ff9700 1 --2- 172.21.15.64:0/526453532 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f98b40a0710 0x7f98b40a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:20.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.009+0000 7f98c8ff9700 1 -- 172.21.15.64:0/526453532 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f98c00cc6d0 con 0x7f98cc101800 2023-09-13T03:11:20.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.009+0000 7f98cb7fe700 1 --2- 172.21.15.64:0/526453532 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f98b40a0710 0x7f98b40a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:20.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.010+0000 7f98cb7fe700 1 --2- 172.21.15.64:0/526453532 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f98b40a0710 0x7f98b40a2bd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f98bc005950 tx=0x7f98bc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:20.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.013+0000 7f98c8ff9700 1 -- 172.21.15.64:0/526453532 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f98c00961e0 con 0x7f98cc101800 2023-09-13T03:11:20.243 INFO:journalctl@ceph.mds.b.smithi142.stdout:Sep 13 03:11:19 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-b[170662]: 2023-09-13T03:11:19.839+0000 7fe0dc505700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2023-09-13T03:11:20.244 INFO:journalctl@ceph.mds.b.smithi142.stdout:Sep 13 03:11:19 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-b[170662]: 2023-09-13T03:11:19.839+0000 7fe0dc505700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2023-09-13T03:11:20.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.275+0000 7f98d1bff700 1 -- 172.21.15.64:0/526453532 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98cc10be70 con 0x7f98cc101800 2023-09-13T03:11:20.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.275+0000 7f98c8ff9700 1 -- 172.21.15.64:0/526453532 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11366 (secure 0 0 0) 0x7f98c0005190 con 0x7f98cc101800 2023-09-13T03:11:20.276 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:11:20.276 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2234182985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2234182985},{"type":"v1","addr":"172.21.15.64:6835","nonce":2234182985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4053841873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4053841873},{"type":"v1","addr":"172.21.15.64:6837","nonce":4053841873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-13T03:11:10.490260+0000","modified":"2023-09-13T03:11:18.230740+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24686,"mds_1":24649,"mds_2":24635,"mds_3":24634,"mds_4":24622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6839/99523374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":99523374},{"type":"v1","addr":"172.21.15.64:6839","nonce":99523374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14649":{"gid":14649,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6841/1866582875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":1866582875},{"type":"v1","addr":"172.21.15.64:6841","nonce":1866582875}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/101906999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":101906999},{"type":"v1","addr":"172.21.15.142:6833","nonce":101906999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24574":{"gid":24574,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/1897366227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1897366227},{"type":"v1","addr":"172.21.15.170:6833","nonce":1897366227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24611":{"gid":24611,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2167779918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2167779918},{"type":"v1","addr":"172.21.15.142:6835","nonce":2167779918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24622":{"gid":24622,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6835/1493371121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1493371121},{"type":"v1","addr":"172.21.15.170:6835","nonce":1493371121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24634":{"gid":24634,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.170:6837/1726587275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":1726587275},{"type":"v1","addr":"172.21.15.170:6837","nonce":1726587275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24635":{"gid":24635,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.142:6837/741173268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":741173268},{"type":"v1","addr":"172.21.15.142:6837","nonce":741173268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24649":{"gid":24649,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.170:6839/2189703596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2189703596},{"type":"v1","addr":"172.21.15.170:6839","nonce":2189703596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24686":{"gid":24686,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/3645998616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645998616},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645998616}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-13T03:11:20.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.278+0000 7f98d1bff700 1 -- 172.21.15.64:0/526453532 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f98b40a0710 msgr2=0x7f98b40a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:20.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.278+0000 7f98d1bff700 1 --2- 172.21.15.64:0/526453532 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f98b40a0710 0x7f98b40a2bd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f98bc005950 tx=0x7f98bc009260 comp rx=0 tx=0).stop 2023-09-13T03:11:20.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.278+0000 7f98d1bff700 1 -- 172.21.15.64:0/526453532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98cc101800 msgr2=0x7f98cc1967f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:20.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.278+0000 7f98d1bff700 1 --2- 172.21.15.64:0/526453532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98cc101800 0x7f98cc1967f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f98c00098e0 tx=0x7f98c000f690 comp rx=0 tx=0).stop 2023-09-13T03:11:20.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.278+0000 7f98d1bff700 1 -- 172.21.15.64:0/526453532 shutdown_connections 2023-09-13T03:11:20.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.278+0000 7f98d1bff700 1 --2- 172.21.15.64:0/526453532 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f98b40a0710 0x7f98b40a2bd0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:20.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.278+0000 7f98d1bff700 1 --2- 172.21.15.64:0/526453532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98cc1021a0 0x7f98cc19d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:20.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.278+0000 7f98d1bff700 1 --2- 172.21.15.64:0/526453532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98cc100610 0x7f98cc1962b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:20.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.278+0000 7f98d1bff700 1 --2- 172.21.15.64:0/526453532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98cc101800 0x7f98cc1967f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:20.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.278+0000 7f98d1bff700 1 -- 172.21.15.64:0/526453532 >> 172.21.15.64:0/526453532 conn(0x7f98cc0fbb40 msgr2=0x7f98cc104cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:20.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.278+0000 7f98d1bff700 1 -- 172.21.15.64:0/526453532 shutdown_connections 2023-09-13T03:11:20.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.278+0000 7f98d1bff700 1 -- 172.21.15.64:0/526453532 wait complete. 2023-09-13T03:11:20.282 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 26 2023-09-13T03:11:20.298 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-13T03:11:20.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:20 smithi142 ceph-mon[134082]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:11:20.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:20 smithi142 ceph-mon[134082]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-09-13T03:11:20.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:20 smithi142 ceph-mon[134082]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:11:20.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.501+0000 7f07e7dff700 1 -- 172.21.15.64:0/1305115592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07e01017f0 msgr2=0x7f07e0101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:20.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.501+0000 7f07e7dff700 1 --2- 172.21.15.64:0/1305115592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07e01017f0 0x7f07e0101c50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f07d00099c0 tx=0x7f07d0009cd0 comp rx=0 tx=0).stop 2023-09-13T03:11:20.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.501+0000 7f07e7dff700 1 -- 172.21.15.64:0/1305115592 shutdown_connections 2023-09-13T03:11:20.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.501+0000 7f07e7dff700 1 --2- 172.21.15.64:0/1305115592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07e0102190 0x7f07e0108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:20.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.501+0000 7f07e7dff700 1 --2- 172.21.15.64:0/1305115592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07e0100600 0x7f07e0100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:20.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.501+0000 7f07e7dff700 1 --2- 172.21.15.64:0/1305115592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07e01017f0 0x7f07e0101c50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:20.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.501+0000 7f07e7dff700 1 -- 172.21.15.64:0/1305115592 >> 172.21.15.64:0/1305115592 conn(0x7f07e00fbb30 msgr2=0x7f07e00fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:20.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.502+0000 7f07e7dff700 1 -- 172.21.15.64:0/1305115592 shutdown_connections 2023-09-13T03:11:20.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.502+0000 7f07e7dff700 1 -- 172.21.15.64:0/1305115592 wait complete. 2023-09-13T03:11:20.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.502+0000 7f07e7dff700 1 Processor -- start 2023-09-13T03:11:20.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.503+0000 7f07e7dff700 1 -- start start 2023-09-13T03:11:20.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.503+0000 7f07e7dff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07e0100600 0x7f07e0196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:20.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.503+0000 7f07e7dff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07e01017f0 0x7f07e01967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:20.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.503+0000 7f07e7dff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07e0102190 0x7f07e019d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:20.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.503+0000 7f07e7dff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e019de00 con 0x7f07e01017f0 2023-09-13T03:11:20.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.503+0000 7f07e7dff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e019df40 con 0x7f07e0100600 2023-09-13T03:11:20.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.503+0000 7f07e7dff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e019e0d0 con 0x7f07e0102190 2023-09-13T03:11:20.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.503+0000 7f07e539a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07e01017f0 0x7f07e01967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:20.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.503+0000 7f07e539a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07e01017f0 0x7f07e01967c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51242/0 (socket says 172.21.15.64:51242) 2023-09-13T03:11:20.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.503+0000 7f07e539a700 1 -- 172.21.15.64:0/2422588256 learned_addr learned my addr 172.21.15.64:0/2422588256 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:20.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.503+0000 7f07e5b9b700 1 --2- 172.21.15.64:0/2422588256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07e0100600 0x7f07e0196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:20.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.503+0000 7f07e639c700 1 --2- 172.21.15.64:0/2422588256 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07e0102190 0x7f07e019d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:20.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.504+0000 7f07e539a700 1 -- 172.21.15.64:0/2422588256 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07e0102190 msgr2=0x7f07e019d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:20.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.504+0000 7f07e539a700 1 --2- 172.21.15.64:0/2422588256 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07e0102190 0x7f07e019d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:20.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.504+0000 7f07e539a700 1 -- 172.21.15.64:0/2422588256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07e0100600 msgr2=0x7f07e0196280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:20.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.504+0000 7f07e539a700 1 --2- 172.21.15.64:0/2422588256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07e0100600 0x7f07e0196280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:20.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.504+0000 7f07e539a700 1 -- 172.21.15.64:0/2422588256 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d8009c40 con 0x7f07e01017f0 2023-09-13T03:11:20.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.504+0000 7f07e539a700 1 --2- 172.21.15.64:0/2422588256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07e01017f0 0x7f07e01967c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f07d000f670 tx=0x7f07d000f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:20.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.504+0000 7f07d6ffd700 1 -- 172.21.15.64:0/2422588256 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d001c070 con 0x7f07e01017f0 2023-09-13T03:11:20.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.504+0000 7f07d6ffd700 1 -- 172.21.15.64:0/2422588256 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07d000fe70 con 0x7f07e01017f0 2023-09-13T03:11:20.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.504+0000 7f07d6ffd700 1 -- 172.21.15.64:0/2422588256 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d0022e80 con 0x7f07e01017f0 2023-09-13T03:11:20.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.504+0000 7f07e7dff700 1 -- 172.21.15.64:0/2422588256 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07d00095f0 con 0x7f07e01017f0 2023-09-13T03:11:20.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.504+0000 7f07e7dff700 1 -- 172.21.15.64:0/2422588256 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e019e590 con 0x7f07e01017f0 2023-09-13T03:11:20.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.505+0000 7f07e7dff700 1 -- 172.21.15.64:0/2422588256 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07e010be20 con 0x7f07e01017f0 2023-09-13T03:11:20.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.507+0000 7f07d6ffd700 1 -- 172.21.15.64:0/2422588256 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f07d0020070 con 0x7f07e01017f0 2023-09-13T03:11:20.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.508+0000 7f07d6ffd700 1 --2- 172.21.15.64:0/2422588256 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f07cc0a0790 0x7f07cc0a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:20.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.509+0000 7f07d6ffd700 1 -- 172.21.15.64:0/2422588256 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f07d00cc960 con 0x7f07e01017f0 2023-09-13T03:11:20.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.509+0000 7f07e5b9b700 1 --2- 172.21.15.64:0/2422588256 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f07cc0a0790 0x7f07cc0a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:20.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.509+0000 7f07e5b9b700 1 --2- 172.21.15.64:0/2422588256 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f07cc0a0790 0x7f07cc0a2c50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f07e0101650 tx=0x7f07dc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:20.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.512+0000 7f07d6ffd700 1 -- 172.21.15.64:0/2422588256 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f07d005ce10 con 0x7f07e01017f0 2023-09-13T03:11:20.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:20 smithi170 ceph-mon[134328]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:11:20.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:20 smithi170 ceph-mon[134328]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-09-13T03:11:20.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:20 smithi170 ceph-mon[134328]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:11:20.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:20 smithi064 ceph-mon[97675]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:11:20.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:20 smithi064 ceph-mon[97675]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-09-13T03:11:20.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:20 smithi064 ceph-mon[97675]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:11:20.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.724+0000 7f07e7dff700 1 -- 172.21.15.64:0/2422588256 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f07e010c060 con 0x7f07e01017f0 2023-09-13T03:11:20.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.726+0000 7f07d6ffd700 1 -- 172.21.15.64:0/2422588256 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22142 (secure 0 0 0) 0x7f07d0096470 con 0x7f07e01017f0 2023-09-13T03:11:20.727 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:11:20.727 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":83,"fsid":"f8dbced4-51e1-11ee-9ab7-7b867c8bd7da","created":"2023-09-13T03:03:24.167202+0000","modified":"2023-09-13T03:11:11.928665+0000","last_up_change":"2023-09-13T03:10:22.503529+0000","last_in_change":"2023-09-13T03:10:05.678720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T03:06:58.987479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-13T03:11:08.589202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-13T03:11:09.924659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3997c114-241b-48c4-8feb-653c99209470","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6803","nonce":3940892135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6805","nonce":3940892135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6809","nonce":3940892135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6807","nonce":3940892135}]},"public_addr":"172.21.15.64:6803/3940892135","cluster_addr":"172.21.15.64:6805/3940892135","heartbeat_back_addr":"172.21.15.64:6809/3940892135","heartbeat_front_addr":"172.21.15.64:6807/3940892135","state":["exists","up"]},{"osd":1,"uuid":"94cfae85-322e-4ed2-a6e6-e40349bb14bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6801","nonce":4188149333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6803","nonce":4188149333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6807","nonce":4188149333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6805","nonce":4188149333}]},"public_addr":"172.21.15.142:6801/4188149333","cluster_addr":"172.21.15.142:6803/4188149333","heartbeat_back_addr":"172.21.15.142:6807/4188149333","heartbeat_front_addr":"172.21.15.142:6805/4188149333","state":["exists","up"]},{"osd":2,"uuid":"b3aae17e-96db-4008-8c27-f733c46b30ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6801","nonce":2406669338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6803","nonce":2406669338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6807","nonce":2406669338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6805","nonce":2406669338}]},"public_addr":"172.21.15.170:6801/2406669338","cluster_addr":"172.21.15.170:6803/2406669338","heartbeat_back_addr":"172.21.15.170:6807/2406669338","heartbeat_front_addr":"172.21.15.170:6805/2406669338","state":["exists","up"]},{"osd":3,"uuid":"7fa8d8cb-2ab7-48a7-8b49-5e2903d6779f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6811","nonce":3558318707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6813","nonce":3558318707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6817","nonce":3558318707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6815","nonce":3558318707}]},"public_addr":"172.21.15.64:6811/3558318707","cluster_addr":"172.21.15.64:6813/3558318707","heartbeat_back_addr":"172.21.15.64:6817/3558318707","heartbeat_front_addr":"172.21.15.64:6815/3558318707","state":["exists","up"]},{"osd":4,"uuid":"8c7a9c9f-f919-48d3-9db2-214a25bb5603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6809","nonce":1811142576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6811","nonce":1811142576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6815","nonce":1811142576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6813","nonce":1811142576}]},"public_addr":"172.21.15.142:6809/1811142576","cluster_addr":"172.21.15.142:6811/1811142576","heartbeat_back_addr":"172.21.15.142:6815/1811142576","heartbeat_front_addr":"172.21.15.142:6813/1811142576","state":["exists","up"]},{"osd":5,"uuid":"0c6101c6-2065-4022-a751-d3f216d7f478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6809","nonce":2903841453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6811","nonce":2903841453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6815","nonce":2903841453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6813","nonce":2903841453}]},"public_addr":"172.21.15.170:6809/2903841453","cluster_addr":"172.21.15.170:6811/2903841453","heartbeat_back_addr":"172.21.15.170:6815/2903841453","heartbeat_front_addr":"172.21.15.170:6813/2903841453","state":["exists","up"]},{"osd":6,"uuid":"5cf3e52e-d155-41b1-9194-db047d26a953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6819","nonce":1436584490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6821","nonce":1436584490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6825","nonce":1436584490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6823","nonce":1436584490}]},"public_addr":"172.21.15.64:6819/1436584490","cluster_addr":"172.21.15.64:6821/1436584490","heartbeat_back_addr":"172.21.15.64:6825/1436584490","heartbeat_front_addr":"172.21.15.64:6823/1436584490","state":["exists","up"]},{"osd":7,"uuid":"e97323e3-badd-41a7-af9e-e78a81a64046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6817","nonce":3726793534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6819","nonce":3726793534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6823","nonce":3726793534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6821","nonce":3726793534}]},"public_addr":"172.21.15.142:6817/3726793534","cluster_addr":"172.21.15.142:6819/3726793534","heartbeat_back_addr":"172.21.15.142:6823/3726793534","heartbeat_front_addr":"172.21.15.142:6821/3726793534","state":["exists","up"]},{"osd":8,"uuid":"83d2835c-ed26-493d-9222-c2894ec0e071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6817","nonce":4272513873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6819","nonce":4272513873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6823","nonce":4272513873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6821","nonce":4272513873}]},"public_addr":"172.21.15.170:6817/4272513873","cluster_addr":"172.21.15.170:6819/4272513873","heartbeat_back_addr":"172.21.15.170:6823/4272513873","heartbeat_front_addr":"172.21.15.170:6821/4272513873","state":["exists","up"]},{"osd":9,"uuid":"cb936bdd-1bec-4236-aa2e-a24adcac11e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6827","nonce":2641175211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6829","nonce":2641175211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6833","nonce":2641175211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6831","nonce":2641175211}]},"public_addr":"172.21.15.64:6827/2641175211","cluster_addr":"172.21.15.64:6829/2641175211","heartbeat_back_addr":"172.21.15.64:6833/2641175211","heartbeat_front_addr":"172.21.15.64:6831/2641175211","state":["exists","up"]},{"osd":10,"uuid":"85512a1c-6171-43d4-9c15-c588a8b71be0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6825","nonce":936499217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6827","nonce":936499217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6831","nonce":936499217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6829","nonce":936499217}]},"public_addr":"172.21.15.142:6825/936499217","cluster_addr":"172.21.15.142:6827/936499217","heartbeat_back_addr":"172.21.15.142:6831/936499217","heartbeat_front_addr":"172.21.15.142:6829/936499217","state":["exists","up"]},{"osd":11,"uuid":"9b8417dc-5d6c-4917-94a9-b032c5bc33d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6825","nonce":3605017060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6827","nonce":3605017060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6831","nonce":3605017060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6829","nonce":3605017060}]},"public_addr":"172.21.15.170:6825/3605017060","cluster_addr":"172.21.15.170:6827/3605017060","heartbeat_back_addr":"172.21.15.170:6831/3605017060","heartbeat_front_addr":"172.21.15.170:6829/3605017060","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:14.229044+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:36.147777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:56.655298+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:07:23.107741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:07:44.663770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:05.577432+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:31.168031+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:52.791055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:09:13.658546+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:09:39.150360+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:10:00.601735+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:10:21.407555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6800/2955227347":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/1373223118":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/271194613":"2023-09-14T03:04:22.355817+0000","172.21.15.64:6801/3998091347":"2023-09-14T03:03:56.442541+0000","172.21.15.64:6800/3998091347":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/920818572":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/735789005":"2023-09-14T03:03:56.442541+0000","172.21.15.64:6801/2955227347":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/1975923178":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/1500151178":"2023-09-14T03:04:22.355817+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T03:11:20.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.728+0000 7f07e7dff700 1 -- 172.21.15.64:0/2422588256 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f07cc0a0790 msgr2=0x7f07cc0a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:20.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.728+0000 7f07e7dff700 1 --2- 172.21.15.64:0/2422588256 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f07cc0a0790 0x7f07cc0a2c50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f07e0101650 tx=0x7f07dc009260 comp rx=0 tx=0).stop 2023-09-13T03:11:20.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.728+0000 7f07e7dff700 1 -- 172.21.15.64:0/2422588256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07e01017f0 msgr2=0x7f07e01967c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:20.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.728+0000 7f07e7dff700 1 --2- 172.21.15.64:0/2422588256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07e01017f0 0x7f07e01967c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f07d000f670 tx=0x7f07d000f6a0 comp rx=0 tx=0).stop 2023-09-13T03:11:20.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.729+0000 7f07e7dff700 1 -- 172.21.15.64:0/2422588256 shutdown_connections 2023-09-13T03:11:20.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.729+0000 7f07e7dff700 1 --2- 172.21.15.64:0/2422588256 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f07cc0a0790 0x7f07cc0a2c50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:20.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.729+0000 7f07e7dff700 1 --2- 172.21.15.64:0/2422588256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07e0100600 0x7f07e0196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:20.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.729+0000 7f07e7dff700 1 --2- 172.21.15.64:0/2422588256 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07e0102190 0x7f07e019d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:20.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.729+0000 7f07e7dff700 1 --2- 172.21.15.64:0/2422588256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07e01017f0 0x7f07e01967c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:20.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.729+0000 7f07e7dff700 1 -- 172.21.15.64:0/2422588256 >> 172.21.15.64:0/2422588256 conn(0x7f07e00fbb30 msgr2=0x7f07e00fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:20.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.729+0000 7f07e7dff700 1 -- 172.21.15.64:0/2422588256 shutdown_connections 2023-09-13T03:11:20.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.729+0000 7f07e7dff700 1 -- 172.21.15.64:0/2422588256 wait complete. 2023-09-13T03:11:20.748 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-09-13T03:11:20.748 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-13T03:11:20.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.951+0000 7f2adafff700 1 -- 172.21.15.64:0/1839923617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ad4101100 msgr2=0x7f2ad41034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:20.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.951+0000 7f2adafff700 1 --2- 172.21.15.64:0/1839923617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ad4101100 0x7f2ad41034f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8009990 tx=0x7f2ac8009ca0 comp rx=0 tx=0).stop 2023-09-13T03:11:20.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.951+0000 7f2adafff700 1 -- 172.21.15.64:0/1839923617 shutdown_connections 2023-09-13T03:11:20.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.951+0000 7f2adafff700 1 --2- 172.21.15.64:0/1839923617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ad4106360 0x7f2ad4108750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:20.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.951+0000 7f2adafff700 1 --2- 172.21.15.64:0/1839923617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ad4101100 0x7f2ad41034f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:20.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.951+0000 7f2adafff700 1 --2- 172.21.15.64:0/1839923617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ad4103a30 0x7f2ad4105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:20.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.951+0000 7f2adafff700 1 -- 172.21.15.64:0/1839923617 >> 172.21.15.64:0/1839923617 conn(0x7f2ad40faa30 msgr2=0x7f2ad40fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:20.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.951+0000 7f2adafff700 1 -- 172.21.15.64:0/1839923617 shutdown_connections 2023-09-13T03:11:20.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.951+0000 7f2adafff700 1 -- 172.21.15.64:0/1839923617 wait complete. 2023-09-13T03:11:20.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.952+0000 7f2adafff700 1 Processor -- start 2023-09-13T03:11:20.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.952+0000 7f2adafff700 1 -- start start 2023-09-13T03:11:20.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.953+0000 7f2adafff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ad4101100 0x7f2ad41984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:20.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.953+0000 7f2adafff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ad4103a30 0x7f2ad4198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:20.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.953+0000 7f2adafff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ad4106360 0x7f2ad419fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:20.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.953+0000 7f2adafff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad41a0060 con 0x7f2ad4103a30 2023-09-13T03:11:20.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.953+0000 7f2adafff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad41a01a0 con 0x7f2ad4101100 2023-09-13T03:11:20.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.953+0000 7f2adafff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad41a0330 con 0x7f2ad4106360 2023-09-13T03:11:20.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.953+0000 7f2ad3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ad4103a30 0x7f2ad4198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:20.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.953+0000 7f2ad3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ad4103a30 0x7f2ad4198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51262/0 (socket says 172.21.15.64:51262) 2023-09-13T03:11:20.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.953+0000 7f2ad8d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ad4101100 0x7f2ad41984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:20.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.953+0000 7f2ad3fff700 1 -- 172.21.15.64:0/2465952608 learned_addr learned my addr 172.21.15.64:0/2465952608 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:20.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.953+0000 7f2ad959c700 1 --2- 172.21.15.64:0/2465952608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ad4106360 0x7f2ad419fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:20.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.953+0000 7f2ad3fff700 1 -- 172.21.15.64:0/2465952608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ad4106360 msgr2=0x7f2ad419fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:20.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.953+0000 7f2ad3fff700 1 --2- 172.21.15.64:0/2465952608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ad4106360 0x7f2ad419fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:20.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.953+0000 7f2ad3fff700 1 -- 172.21.15.64:0/2465952608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ad4101100 msgr2=0x7f2ad41984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:20.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.953+0000 7f2ad3fff700 1 --2- 172.21.15.64:0/2465952608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ad4101100 0x7f2ad41984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:20.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.953+0000 7f2ad3fff700 1 -- 172.21.15.64:0/2465952608 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac80095f0 con 0x7f2ad4103a30 2023-09-13T03:11:20.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.954+0000 7f2ad3fff700 1 --2- 172.21.15.64:0/2465952608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ad4103a30 0x7f2ad4198a20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2ac000d980 tx=0x7f2ac000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:20.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.954+0000 7f2ad1ffb700 1 -- 172.21.15.64:0/2465952608 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0004220 con 0x7f2ad4103a30 2023-09-13T03:11:20.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.954+0000 7f2ad1ffb700 1 -- 172.21.15.64:0/2465952608 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ac0007640 con 0x7f2ad4103a30 2023-09-13T03:11:20.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.954+0000 7f2adafff700 1 -- 172.21.15.64:0/2465952608 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ad41a0610 con 0x7f2ad4103a30 2023-09-13T03:11:20.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.954+0000 7f2ad1ffb700 1 -- 172.21.15.64:0/2465952608 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0005690 con 0x7f2ad4103a30 2023-09-13T03:11:20.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.954+0000 7f2adafff700 1 -- 172.21.15.64:0/2465952608 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ad41a0b20 con 0x7f2ad4103a30 2023-09-13T03:11:20.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.955+0000 7f2adafff700 1 -- 172.21.15.64:0/2465952608 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ad4107ee0 con 0x7f2ad4103a30 2023-09-13T03:11:20.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.957+0000 7f2ad1ffb700 1 -- 172.21.15.64:0/2465952608 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f2ac0004750 con 0x7f2ad4103a30 2023-09-13T03:11:20.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.959+0000 7f2ad1ffb700 1 --2- 172.21.15.64:0/2465952608 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f2ac40a0710 0x7f2ac40a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:20.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.959+0000 7f2ad1ffb700 1 -- 172.21.15.64:0/2465952608 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2ac00cbaa0 con 0x7f2ad4103a30 2023-09-13T03:11:20.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.959+0000 7f2ad8d9b700 1 --2- 172.21.15.64:0/2465952608 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f2ac40a0710 0x7f2ac40a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:20.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.959+0000 7f2ad8d9b700 1 --2- 172.21.15.64:0/2465952608 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f2ac40a0710 0x7f2ac40a2bd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2ac800b2e0 tx=0x7f2ac8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:20.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:20.963+0000 7f2ad1ffb700 1 -- 172.21.15.64:0/2465952608 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f2ac0095520 con 0x7f2ad4103a30 2023-09-13T03:11:21.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.171+0000 7f2adafff700 1 -- 172.21.15.64:0/2465952608 --> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2ad402cf80 con 0x7f2ac40a0710 2023-09-13T03:11:21.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.184+0000 7f2ad1ffb700 1 -- 172.21.15.64:0/2465952608 <== mgr.14150 v2:172.21.15.64:6800/873679308 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+335754 (secure 0 0 0) 0x7f2ad402cf80 con 0x7f2ac40a0710 2023-09-13T03:11:21.190 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:11:21.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.189+0000 7f2adafff700 1 -- 172.21.15.64:0/2465952608 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f2ac40a0710 msgr2=0x7f2ac40a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:21.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.189+0000 7f2adafff700 1 --2- 172.21.15.64:0/2465952608 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f2ac40a0710 0x7f2ac40a2bd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2ac800b2e0 tx=0x7f2ac8005fb0 comp rx=0 tx=0).stop 2023-09-13T03:11:21.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.189+0000 7f2adafff700 1 -- 172.21.15.64:0/2465952608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ad4103a30 msgr2=0x7f2ad4198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:21.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.189+0000 7f2adafff700 1 --2- 172.21.15.64:0/2465952608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ad4103a30 0x7f2ad4198a20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2ac000d980 tx=0x7f2ac000dc90 comp rx=0 tx=0).stop 2023-09-13T03:11:21.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.189+0000 7f2adafff700 1 -- 172.21.15.64:0/2465952608 shutdown_connections 2023-09-13T03:11:21.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.189+0000 7f2adafff700 1 --2- 172.21.15.64:0/2465952608 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f2ac40a0710 0x7f2ac40a2bd0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:21.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.189+0000 7f2adafff700 1 --2- 172.21.15.64:0/2465952608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ad4101100 0x7f2ad41984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:21.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.189+0000 7f2adafff700 1 --2- 172.21.15.64:0/2465952608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ad4106360 0x7f2ad419fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:21.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.189+0000 7f2adafff700 1 --2- 172.21.15.64:0/2465952608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ad4103a30 0x7f2ad4198a20 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:21.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.189+0000 7f2adafff700 1 -- 172.21.15.64:0/2465952608 >> 172.21.15.64:0/2465952608 conn(0x7f2ad40faa30 msgr2=0x7f2ad40ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:21.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.189+0000 7f2adafff700 1 -- 172.21.15.64:0/2465952608 shutdown_connections 2023-09-13T03:11:21.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.189+0000 7f2adafff700 1 -- 172.21.15.64:0/2465952608 wait complete. 2023-09-13T03:11:21.195 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-13T03:11:21.209 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":282,"stamp":"2023-09-13T03:11:20.445881+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598933,"num_objects":89,"num_object_clones":0,"num_object_copies":267,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":0,"num_read":146,"num_read_kb":126,"num_write":368,"num_write_kb":3001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":57,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":300,"ondisk_log_size":300,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":282,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336712,"kb_used_data":12520,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605112,"statfs":{"total":1151940427776,"available":1151595634688,"internally_reserved":0,"allocated":12820480,"data_stored":6938814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19130,"internal_metadata":331855174},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8565,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":10,"num_read_kb":7,"num_write":187,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":57,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.516783"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643523+0000","last_change":"2023-09-13T03:11:10.949449+0000","last_active":"2023-09-13T03:11:12.643523+0000","last_peered":"2023-09-13T03:11:12.643523+0000","last_clean":"2023-09-13T03:11:12.643523+0000","last_became_active":"2023-09-13T03:11:10.948912+0000","last_became_peered":"2023-09-13T03:11:10.948912+0000","last_unstale":"2023-09-13T03:11:12.643523+0000","last_undegraded":"2023-09-13T03:11:12.643523+0000","last_fullsized":"2023-09-13T03:11:12.643523+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:34:27.381352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643150+0000","last_change":"2023-09-13T03:11:10.956817+0000","last_active":"2023-09-13T03:11:12.643150+0000","last_peered":"2023-09-13T03:11:12.643150+0000","last_clean":"2023-09-13T03:11:12.643150+0000","last_became_active":"2023-09-13T03:11:10.955359+0000","last_became_peered":"2023-09-13T03:11:10.955359+0000","last_unstale":"2023-09-13T03:11:12.643150+0000","last_undegraded":"2023-09-13T03:11:12.643150+0000","last_fullsized":"2023-09-13T03:11:12.643150+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:06:01.055910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.642907+0000","last_change":"2023-09-13T03:11:10.959915+0000","last_active":"2023-09-13T03:11:12.642907+0000","last_peered":"2023-09-13T03:11:12.642907+0000","last_clean":"2023-09-13T03:11:12.642907+0000","last_became_active":"2023-09-13T03:11:10.959611+0000","last_became_peered":"2023-09-13T03:11:10.959611+0000","last_unstale":"2023-09-13T03:11:12.642907+0000","last_undegraded":"2023-09-13T03:11:12.642907+0000","last_fullsized":"2023-09-13T03:11:12.642907+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:31:40.075569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:11.938190+0000","last_change":"2023-09-13T03:11:10.944329+0000","last_active":"2023-09-13T03:11:11.938190+0000","last_peered":"2023-09-13T03:11:11.938190+0000","last_clean":"2023-09-13T03:11:11.938190+0000","last_became_active":"2023-09-13T03:11:10.944137+0000","last_became_peered":"2023-09-13T03:11:10.944137+0000","last_unstale":"2023-09-13T03:11:11.938190+0000","last_undegraded":"2023-09-13T03:11:11.938190+0000","last_fullsized":"2023-09-13T03:11:11.938190+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:03:58.300005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643437+0000","last_change":"2023-09-13T03:11:10.951554+0000","last_active":"2023-09-13T03:11:12.643437+0000","last_peered":"2023-09-13T03:11:12.643437+0000","last_clean":"2023-09-13T03:11:12.643437+0000","last_became_active":"2023-09-13T03:11:10.951252+0000","last_became_peered":"2023-09-13T03:11:10.951252+0000","last_unstale":"2023-09-13T03:11:12.643437+0000","last_undegraded":"2023-09-13T03:11:12.643437+0000","last_fullsized":"2023-09-13T03:11:12.643437+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:38:42.890256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643324+0000","last_change":"2023-09-13T03:11:10.953087+0000","last_active":"2023-09-13T03:11:12.643324+0000","last_peered":"2023-09-13T03:11:12.643324+0000","last_clean":"2023-09-13T03:11:12.643324+0000","last_became_active":"2023-09-13T03:11:10.952773+0000","last_became_peered":"2023-09-13T03:11:10.952773+0000","last_unstale":"2023-09-13T03:11:12.643324+0000","last_undegraded":"2023-09-13T03:11:12.643324+0000","last_fullsized":"2023-09-13T03:11:12.643324+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:14:55.203347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:13.443065+0000","last_change":"2023-09-13T03:11:10.956058+0000","last_active":"2023-09-13T03:11:13.443065+0000","last_peered":"2023-09-13T03:11:13.443065+0000","last_clean":"2023-09-13T03:11:13.443065+0000","last_became_active":"2023-09-13T03:11:10.955783+0000","last_became_peered":"2023-09-13T03:11:10.955783+0000","last_unstale":"2023-09-13T03:11:13.443065+0000","last_undegraded":"2023-09-13T03:11:13.443065+0000","last_fullsized":"2023-09-13T03:11:13.443065+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:31:06.810338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.642680+0000","last_change":"2023-09-13T03:11:10.950102+0000","last_active":"2023-09-13T03:11:12.642680+0000","last_peered":"2023-09-13T03:11:12.642680+0000","last_clean":"2023-09-13T03:11:12.642680+0000","last_became_active":"2023-09-13T03:11:10.948877+0000","last_became_peered":"2023-09-13T03:11:10.948877+0000","last_unstale":"2023-09-13T03:11:12.642680+0000","last_undegraded":"2023-09-13T03:11:12.642680+0000","last_fullsized":"2023-09-13T03:11:12.642680+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:32:09.168793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643295+0000","last_change":"2023-09-13T03:11:10.945244+0000","last_active":"2023-09-13T03:11:12.643295+0000","last_peered":"2023-09-13T03:11:12.643295+0000","last_clean":"2023-09-13T03:11:12.643295+0000","last_became_active":"2023-09-13T03:11:10.944951+0000","last_became_peered":"2023-09-13T03:11:10.944951+0000","last_unstale":"2023-09-13T03:11:12.643295+0000","last_undegraded":"2023-09-13T03:11:12.643295+0000","last_fullsized":"2023-09-13T03:11:12.643295+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:42:35.420767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643658+0000","last_change":"2023-09-13T03:11:10.952312+0000","last_active":"2023-09-13T03:11:12.643658+0000","last_peered":"2023-09-13T03:11:12.643658+0000","last_clean":"2023-09-13T03:11:12.643658+0000","last_became_active":"2023-09-13T03:11:10.952077+0000","last_became_peered":"2023-09-13T03:11:10.952077+0000","last_unstale":"2023-09-13T03:11:12.643658+0000","last_undegraded":"2023-09-13T03:11:12.643658+0000","last_fullsized":"2023-09-13T03:11:12.643658+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:57:10.888184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:13.443504+0000","last_change":"2023-09-13T03:11:10.959259+0000","last_active":"2023-09-13T03:11:13.443504+0000","last_peered":"2023-09-13T03:11:13.443504+0000","last_clean":"2023-09-13T03:11:13.443504+0000","last_became_active":"2023-09-13T03:11:10.958922+0000","last_became_peered":"2023-09-13T03:11:10.958922+0000","last_unstale":"2023-09-13T03:11:13.443504+0000","last_undegraded":"2023-09-13T03:11:13.443504+0000","last_fullsized":"2023-09-13T03:11:13.443504+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:08:54.680718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:11.938117+0000","last_change":"2023-09-13T03:11:10.950946+0000","last_active":"2023-09-13T03:11:11.938117+0000","last_peered":"2023-09-13T03:11:11.938117+0000","last_clean":"2023-09-13T03:11:11.938117+0000","last_became_active":"2023-09-13T03:11:10.950633+0000","last_became_peered":"2023-09-13T03:11:10.950633+0000","last_unstale":"2023-09-13T03:11:11.938117+0000","last_undegraded":"2023-09-13T03:11:11.938117+0000","last_fullsized":"2023-09-13T03:11:11.938117+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:37:25.796555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.644173+0000","last_change":"2023-09-13T03:11:10.957712+0000","last_active":"2023-09-13T03:11:12.644173+0000","last_peered":"2023-09-13T03:11:12.644173+0000","last_clean":"2023-09-13T03:11:12.644173+0000","last_became_active":"2023-09-13T03:11:10.957379+0000","last_became_peered":"2023-09-13T03:11:10.957379+0000","last_unstale":"2023-09-13T03:11:12.644173+0000","last_undegraded":"2023-09-13T03:11:12.644173+0000","last_fullsized":"2023-09-13T03:11:12.644173+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:31:17.084185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.642902+0000","last_change":"2023-09-13T03:11:10.952951+0000","last_active":"2023-09-13T03:11:12.642902+0000","last_peered":"2023-09-13T03:11:12.642902+0000","last_clean":"2023-09-13T03:11:12.642902+0000","last_became_active":"2023-09-13T03:11:10.952629+0000","last_became_peered":"2023-09-13T03:11:10.952629+0000","last_unstale":"2023-09-13T03:11:12.642902+0000","last_undegraded":"2023-09-13T03:11:12.642902+0000","last_fullsized":"2023-09-13T03:11:12.642902+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:20:40.802584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.642803+0000","last_change":"2023-09-13T03:11:10.950799+0000","last_active":"2023-09-13T03:11:12.642803+0000","last_peered":"2023-09-13T03:11:12.642803+0000","last_clean":"2023-09-13T03:11:12.642803+0000","last_became_active":"2023-09-13T03:11:10.950500+0000","last_became_peered":"2023-09-13T03:11:10.950500+0000","last_unstale":"2023-09-13T03:11:12.642803+0000","last_undegraded":"2023-09-13T03:11:12.642803+0000","last_fullsized":"2023-09-13T03:11:12.642803+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:24:45.770696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.642539+0000","last_change":"2023-09-13T03:11:10.955141+0000","last_active":"2023-09-13T03:11:12.642539+0000","last_peered":"2023-09-13T03:11:12.642539+0000","last_clean":"2023-09-13T03:11:12.642539+0000","last_became_active":"2023-09-13T03:11:10.954840+0000","last_became_peered":"2023-09-13T03:11:10.954840+0000","last_unstale":"2023-09-13T03:11:12.642539+0000","last_undegraded":"2023-09-13T03:11:12.642539+0000","last_fullsized":"2023-09-13T03:11:12.642539+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:56:08.318784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643646+0000","last_change":"2023-09-13T03:11:10.949199+0000","last_active":"2023-09-13T03:11:12.643646+0000","last_peered":"2023-09-13T03:11:12.643646+0000","last_clean":"2023-09-13T03:11:12.643646+0000","last_became_active":"2023-09-13T03:11:10.948506+0000","last_became_peered":"2023-09-13T03:11:10.948506+0000","last_unstale":"2023-09-13T03:11:12.643646+0000","last_undegraded":"2023-09-13T03:11:12.643646+0000","last_fullsized":"2023-09-13T03:11:12.643646+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:04:07.714033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:11.937809+0000","last_change":"2023-09-13T03:11:10.953207+0000","last_active":"2023-09-13T03:11:11.937809+0000","last_peered":"2023-09-13T03:11:11.937809+0000","last_clean":"2023-09-13T03:11:11.937809+0000","last_became_active":"2023-09-13T03:11:10.951566+0000","last_became_peered":"2023-09-13T03:11:10.951566+0000","last_unstale":"2023-09-13T03:11:11.937809+0000","last_undegraded":"2023-09-13T03:11:11.937809+0000","last_fullsized":"2023-09-13T03:11:11.937809+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:23:06.020880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:11.937663+0000","last_change":"2023-09-13T03:11:10.951695+0000","last_active":"2023-09-13T03:11:11.937663+0000","last_peered":"2023-09-13T03:11:11.937663+0000","last_clean":"2023-09-13T03:11:11.937663+0000","last_became_active":"2023-09-13T03:11:10.951382+0000","last_became_peered":"2023-09-13T03:11:10.951382+0000","last_unstale":"2023-09-13T03:11:11.937663+0000","last_undegraded":"2023-09-13T03:11:11.937663+0000","last_fullsized":"2023-09-13T03:11:11.937663+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:29:05.200980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'11","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:16.219903+0000","last_change":"2023-09-13T03:11:09.956193+0000","last_active":"2023-09-13T03:11:16.219903+0000","last_peered":"2023-09-13T03:11:16.219903+0000","last_clean":"2023-09-13T03:11:16.219903+0000","last_became_active":"2023-09-13T03:11:09.955778+0000","last_became_peered":"2023-09-13T03:11:09.955778+0000","last_unstale":"2023-09-13T03:11:16.219903+0000","last_undegraded":"2023-09-13T03:11:16.219903+0000","last_fullsized":"2023-09-13T03:11:16.219903+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:15:57.425730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643666+0000","last_change":"2023-09-13T03:11:10.944437+0000","last_active":"2023-09-13T03:11:12.643666+0000","last_peered":"2023-09-13T03:11:12.643666+0000","last_clean":"2023-09-13T03:11:12.643666+0000","last_became_active":"2023-09-13T03:11:10.943961+0000","last_became_peered":"2023-09-13T03:11:10.943961+0000","last_unstale":"2023-09-13T03:11:12.643666+0000","last_undegraded":"2023-09-13T03:11:12.643666+0000","last_fullsized":"2023-09-13T03:11:12.643666+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:39:43.928334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643677+0000","last_change":"2023-09-13T03:11:09.952017+0000","last_active":"2023-09-13T03:11:12.643677+0000","last_peered":"2023-09-13T03:11:12.643677+0000","last_clean":"2023-09-13T03:11:12.643677+0000","last_became_active":"2023-09-13T03:11:09.951700+0000","last_became_peered":"2023-09-13T03:11:09.951700+0000","last_unstale":"2023-09-13T03:11:12.643677+0000","last_undegraded":"2023-09-13T03:11:12.643677+0000","last_fullsized":"2023-09-13T03:11:12.643677+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:28:50.687271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643185+0000","last_change":"2023-09-13T03:11:10.960198+0000","last_active":"2023-09-13T03:11:12.643185+0000","last_peered":"2023-09-13T03:11:12.643185+0000","last_clean":"2023-09-13T03:11:12.643185+0000","last_became_active":"2023-09-13T03:11:10.959839+0000","last_became_peered":"2023-09-13T03:11:10.959839+0000","last_unstale":"2023-09-13T03:11:12.643185+0000","last_undegraded":"2023-09-13T03:11:12.643185+0000","last_fullsized":"2023-09-13T03:11:12.643185+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:37:16.868093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:15.209158+0000","last_change":"2023-09-13T03:11:09.954684+0000","last_active":"2023-09-13T03:11:15.209158+0000","last_peered":"2023-09-13T03:11:15.209158+0000","last_clean":"2023-09-13T03:11:15.209158+0000","last_became_active":"2023-09-13T03:11:09.954392+0000","last_became_peered":"2023-09-13T03:11:09.954392+0000","last_unstale":"2023-09-13T03:11:15.209158+0000","last_undegraded":"2023-09-13T03:11:15.209158+0000","last_fullsized":"2023-09-13T03:11:15.209158+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:29:20.333063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.642726+0000","last_change":"2023-09-13T03:11:10.944364+0000","last_active":"2023-09-13T03:11:12.642726+0000","last_peered":"2023-09-13T03:11:12.642726+0000","last_clean":"2023-09-13T03:11:12.642726+0000","last_became_active":"2023-09-13T03:11:10.943971+0000","last_became_peered":"2023-09-13T03:11:10.943971+0000","last_unstale":"2023-09-13T03:11:12.642726+0000","last_undegraded":"2023-09-13T03:11:12.642726+0000","last_fullsized":"2023-09-13T03:11:12.642726+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:55:07.831397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-13T03:11:10.974241+0000","last_change":"2023-09-13T03:11:09.952751+0000","last_active":"2023-09-13T03:11:10.974241+0000","last_peered":"2023-09-13T03:11:10.974241+0000","last_clean":"2023-09-13T03:11:10.974241+0000","last_became_active":"2023-09-13T03:11:09.952447+0000","last_became_peered":"2023-09-13T03:11:09.952447+0000","last_unstale":"2023-09-13T03:11:10.974241+0000","last_undegraded":"2023-09-13T03:11:10.974241+0000","last_fullsized":"2023-09-13T03:11:10.974241+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:55:09.971742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.143629+0000","last_change":"2023-09-13T03:11:10.956427+0000","last_active":"2023-09-13T03:11:12.143629+0000","last_peered":"2023-09-13T03:11:12.143629+0000","last_clean":"2023-09-13T03:11:12.143629+0000","last_became_active":"2023-09-13T03:11:10.956113+0000","last_became_peered":"2023-09-13T03:11:10.956113+0000","last_unstale":"2023-09-13T03:11:12.143629+0000","last_undegraded":"2023-09-13T03:11:12.143629+0000","last_fullsized":"2023-09-13T03:11:12.143629+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:21:48.832314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:14.201886+0000","last_change":"2023-09-13T03:11:09.947207+0000","last_active":"2023-09-13T03:11:14.201886+0000","last_peered":"2023-09-13T03:11:14.201886+0000","last_clean":"2023-09-13T03:11:14.201886+0000","last_became_active":"2023-09-13T03:11:09.946787+0000","last_became_peered":"2023-09-13T03:11:09.946787+0000","last_unstale":"2023-09-13T03:11:14.201886+0000","last_undegraded":"2023-09-13T03:11:14.201886+0000","last_fullsized":"2023-09-13T03:11:14.201886+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:28:47.100723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643289+0000","last_change":"2023-09-13T03:11:10.948427+0000","last_active":"2023-09-13T03:11:12.643289+0000","last_peered":"2023-09-13T03:11:12.643289+0000","last_clean":"2023-09-13T03:11:12.643289+0000","last_became_active":"2023-09-13T03:11:10.948095+0000","last_became_peered":"2023-09-13T03:11:10.948095+0000","last_unstale":"2023-09-13T03:11:12.643289+0000","last_undegraded":"2023-09-13T03:11:12.643289+0000","last_fullsized":"2023-09-13T03:11:12.643289+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:55:07.532772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:16.219456+0000","last_change":"2023-09-13T03:11:09.952335+0000","last_active":"2023-09-13T03:11:16.219456+0000","last_peered":"2023-09-13T03:11:16.219456+0000","last_clean":"2023-09-13T03:11:16.219456+0000","last_became_active":"2023-09-13T03:11:09.951974+0000","last_became_peered":"2023-09-13T03:11:09.951974+0000","last_unstale":"2023-09-13T03:11:16.219456+0000","last_undegraded":"2023-09-13T03:11:16.219456+0000","last_fullsized":"2023-09-13T03:11:16.219456+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:15:50.681751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.143776+0000","last_change":"2023-09-13T03:11:10.948962+0000","last_active":"2023-09-13T03:11:12.143776+0000","last_peered":"2023-09-13T03:11:12.143776+0000","last_clean":"2023-09-13T03:11:12.143776+0000","last_became_active":"2023-09-13T03:11:10.948543+0000","last_became_peered":"2023-09-13T03:11:10.948543+0000","last_unstale":"2023-09-13T03:11:12.143776+0000","last_undegraded":"2023-09-13T03:11:12.143776+0000","last_fullsized":"2023-09-13T03:11:12.143776+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:35:28.789698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:15.212445+0000","last_change":"2023-09-13T03:11:09.957644+0000","last_active":"2023-09-13T03:11:15.212445+0000","last_peered":"2023-09-13T03:11:15.212445+0000","last_clean":"2023-09-13T03:11:15.212445+0000","last_became_active":"2023-09-13T03:11:09.957289+0000","last_became_peered":"2023-09-13T03:11:09.957289+0000","last_unstale":"2023-09-13T03:11:15.212445+0000","last_undegraded":"2023-09-13T03:11:15.212445+0000","last_fullsized":"2023-09-13T03:11:15.212445+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:40:48.615322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643712+0000","last_change":"2023-09-13T03:11:10.947561+0000","last_active":"2023-09-13T03:11:12.643712+0000","last_peered":"2023-09-13T03:11:12.643712+0000","last_clean":"2023-09-13T03:11:12.643712+0000","last_became_active":"2023-09-13T03:11:10.947238+0000","last_became_peered":"2023-09-13T03:11:10.947238+0000","last_unstale":"2023-09-13T03:11:12.643712+0000","last_undegraded":"2023-09-13T03:11:12.643712+0000","last_fullsized":"2023-09-13T03:11:12.643712+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:54:31.360657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:16.218673+0000","last_change":"2023-09-13T03:11:09.947806+0000","last_active":"2023-09-13T03:11:16.218673+0000","last_peered":"2023-09-13T03:11:16.218673+0000","last_clean":"2023-09-13T03:11:16.218673+0000","last_became_active":"2023-09-13T03:11:09.947428+0000","last_became_peered":"2023-09-13T03:11:09.947428+0000","last_unstale":"2023-09-13T03:11:16.218673+0000","last_undegraded":"2023-09-13T03:11:16.218673+0000","last_fullsized":"2023-09-13T03:11:16.218673+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:25:00.421873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:11.937578+0000","last_change":"2023-09-13T03:11:10.951637+0000","last_active":"2023-09-13T03:11:11.937578+0000","last_peered":"2023-09-13T03:11:11.937578+0000","last_clean":"2023-09-13T03:11:11.937578+0000","last_became_active":"2023-09-13T03:11:10.951249+0000","last_became_peered":"2023-09-13T03:11:10.951249+0000","last_unstale":"2023-09-13T03:11:11.937578+0000","last_undegraded":"2023-09-13T03:11:11.937578+0000","last_fullsized":"2023-09-13T03:11:11.937578+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:24:27.715167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:14.199142+0000","last_change":"2023-09-13T03:11:09.954343+0000","last_active":"2023-09-13T03:11:14.199142+0000","last_peered":"2023-09-13T03:11:14.199142+0000","last_clean":"2023-09-13T03:11:14.199142+0000","last_became_active":"2023-09-13T03:11:09.954063+0000","last_became_peered":"2023-09-13T03:11:09.954063+0000","last_unstale":"2023-09-13T03:11:14.199142+0000","last_undegraded":"2023-09-13T03:11:14.199142+0000","last_fullsized":"2023-09-13T03:11:14.199142+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:34:50.057539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643086+0000","last_change":"2023-09-13T03:11:10.956994+0000","last_active":"2023-09-13T03:11:12.643086+0000","last_peered":"2023-09-13T03:11:12.643086+0000","last_clean":"2023-09-13T03:11:12.643086+0000","last_became_active":"2023-09-13T03:11:10.956336+0000","last_became_peered":"2023-09-13T03:11:10.956336+0000","last_unstale":"2023-09-13T03:11:12.643086+0000","last_undegraded":"2023-09-13T03:11:12.643086+0000","last_fullsized":"2023-09-13T03:11:12.643086+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:11:54.186143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:17.233876+0000","last_change":"2023-09-13T03:11:09.945775+0000","last_active":"2023-09-13T03:11:17.233876+0000","last_peered":"2023-09-13T03:11:17.233876+0000","last_clean":"2023-09-13T03:11:17.233876+0000","last_became_active":"2023-09-13T03:11:09.945406+0000","last_became_peered":"2023-09-13T03:11:09.945406+0000","last_unstale":"2023-09-13T03:11:17.233876+0000","last_undegraded":"2023-09-13T03:11:17.233876+0000","last_fullsized":"2023-09-13T03:11:17.233876+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:29:26.677351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643014+0000","last_change":"2023-09-13T03:11:10.951941+0000","last_active":"2023-09-13T03:11:12.643014+0000","last_peered":"2023-09-13T03:11:12.643014+0000","last_clean":"2023-09-13T03:11:12.643014+0000","last_became_active":"2023-09-13T03:11:10.951331+0000","last_became_peered":"2023-09-13T03:11:10.951331+0000","last_unstale":"2023-09-13T03:11:12.643014+0000","last_undegraded":"2023-09-13T03:11:12.643014+0000","last_fullsized":"2023-09-13T03:11:12.643014+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:15:53.741657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:14.199417+0000","last_change":"2023-09-13T03:11:09.959503+0000","last_active":"2023-09-13T03:11:14.199417+0000","last_peered":"2023-09-13T03:11:14.199417+0000","last_clean":"2023-09-13T03:11:14.199417+0000","last_became_active":"2023-09-13T03:11:09.959166+0000","last_became_peered":"2023-09-13T03:11:09.959166+0000","last_unstale":"2023-09-13T03:11:14.199417+0000","last_undegraded":"2023-09-13T03:11:14.199417+0000","last_fullsized":"2023-09-13T03:11:14.199417+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:12:36.179520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643402+0000","last_change":"2023-09-13T03:11:10.941575+0000","last_active":"2023-09-13T03:11:12.643402+0000","last_peered":"2023-09-13T03:11:12.643402+0000","last_clean":"2023-09-13T03:11:12.643402+0000","last_became_active":"2023-09-13T03:11:10.941285+0000","last_became_peered":"2023-09-13T03:11:10.941285+0000","last_unstale":"2023-09-13T03:11:12.643402+0000","last_undegraded":"2023-09-13T03:11:12.643402+0000","last_fullsized":"2023-09-13T03:11:12.643402+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:03:09.377471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'11","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:17.234676+0000","last_change":"2023-09-13T03:11:09.948041+0000","last_active":"2023-09-13T03:11:17.234676+0000","last_peered":"2023-09-13T03:11:17.234676+0000","last_clean":"2023-09-13T03:11:17.234676+0000","last_became_active":"2023-09-13T03:11:09.947785+0000","last_became_peered":"2023-09-13T03:11:09.947785+0000","last_unstale":"2023-09-13T03:11:17.234676+0000","last_undegraded":"2023-09-13T03:11:17.234676+0000","last_fullsized":"2023-09-13T03:11:17.234676+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:46:47.348307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2002,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643222+0000","last_change":"2023-09-13T03:11:10.950709+0000","last_active":"2023-09-13T03:11:12.643222+0000","last_peered":"2023-09-13T03:11:12.643222+0000","last_clean":"2023-09-13T03:11:12.643222+0000","last_became_active":"2023-09-13T03:11:10.950395+0000","last_became_peered":"2023-09-13T03:11:10.950395+0000","last_unstale":"2023-09-13T03:11:12.643222+0000","last_undegraded":"2023-09-13T03:11:12.643222+0000","last_fullsized":"2023-09-13T03:11:12.643222+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:50:08.628777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:18.237723+0000","last_change":"2023-09-13T03:11:09.946049+0000","last_active":"2023-09-13T03:11:18.237723+0000","last_peered":"2023-09-13T03:11:18.237723+0000","last_clean":"2023-09-13T03:11:18.237723+0000","last_became_active":"2023-09-13T03:11:09.945398+0000","last_became_peered":"2023-09-13T03:11:09.945398+0000","last_unstale":"2023-09-13T03:11:18.237723+0000","last_undegraded":"2023-09-13T03:11:18.237723+0000","last_fullsized":"2023-09-13T03:11:18.237723+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:57:21.448431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.644175+0000","last_change":"2023-09-13T03:11:10.955763+0000","last_active":"2023-09-13T03:11:12.644175+0000","last_peered":"2023-09-13T03:11:12.644175+0000","last_clean":"2023-09-13T03:11:12.644175+0000","last_became_active":"2023-09-13T03:11:10.955455+0000","last_became_peered":"2023-09-13T03:11:10.955455+0000","last_unstale":"2023-09-13T03:11:12.644175+0000","last_undegraded":"2023-09-13T03:11:12.644175+0000","last_fullsized":"2023-09-13T03:11:12.644175+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:43:37.127490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:16.219283+0000","last_change":"2023-09-13T03:11:09.947739+0000","last_active":"2023-09-13T03:11:16.219283+0000","last_peered":"2023-09-13T03:11:16.219283+0000","last_clean":"2023-09-13T03:11:16.219283+0000","last_became_active":"2023-09-13T03:11:09.947348+0000","last_became_peered":"2023-09-13T03:11:09.947348+0000","last_unstale":"2023-09-13T03:11:16.219283+0000","last_undegraded":"2023-09-13T03:11:16.219283+0000","last_fullsized":"2023-09-13T03:11:16.219283+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:47:58.408877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:13.443369+0000","last_change":"2023-09-13T03:11:10.958140+0000","last_active":"2023-09-13T03:11:13.443369+0000","last_peered":"2023-09-13T03:11:13.443369+0000","last_clean":"2023-09-13T03:11:13.443369+0000","last_became_active":"2023-09-13T03:11:10.957798+0000","last_became_peered":"2023-09-13T03:11:10.957798+0000","last_unstale":"2023-09-13T03:11:13.443369+0000","last_undegraded":"2023-09-13T03:11:13.443369+0000","last_fullsized":"2023-09-13T03:11:13.443369+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:44:40.208559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'6","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:16.218499+0000","last_change":"2023-09-13T03:11:09.945554+0000","last_active":"2023-09-13T03:11:16.218499+0000","last_peered":"2023-09-13T03:11:16.218499+0000","last_clean":"2023-09-13T03:11:16.218499+0000","last_became_active":"2023-09-13T03:11:09.945214+0000","last_became_peered":"2023-09-13T03:11:09.945214+0000","last_unstale":"2023-09-13T03:11:16.218499+0000","last_undegraded":"2023-09-13T03:11:16.218499+0000","last_fullsized":"2023-09-13T03:11:16.218499+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:46:12.338617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:13.442989+0000","last_change":"2023-09-13T03:11:10.952589+0000","last_active":"2023-09-13T03:11:13.442989+0000","last_peered":"2023-09-13T03:11:13.442989+0000","last_clean":"2023-09-13T03:11:13.442989+0000","last_became_active":"2023-09-13T03:11:10.952302+0000","last_became_peered":"2023-09-13T03:11:10.952302+0000","last_unstale":"2023-09-13T03:11:13.442989+0000","last_undegraded":"2023-09-13T03:11:13.442989+0000","last_fullsized":"2023-09-13T03:11:13.442989+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:13:46.972293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:15.210597+0000","last_change":"2023-09-13T03:11:09.959509+0000","last_active":"2023-09-13T03:11:15.210597+0000","last_peered":"2023-09-13T03:11:15.210597+0000","last_clean":"2023-09-13T03:11:15.210597+0000","last_became_active":"2023-09-13T03:11:09.959150+0000","last_became_peered":"2023-09-13T03:11:09.959150+0000","last_unstale":"2023-09-13T03:11:15.210597+0000","last_undegraded":"2023-09-13T03:11:15.210597+0000","last_fullsized":"2023-09-13T03:11:15.210597+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:33:30.648283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.143398+0000","last_change":"2023-09-13T03:11:10.956610+0000","last_active":"2023-09-13T03:11:12.143398+0000","last_peered":"2023-09-13T03:11:12.143398+0000","last_clean":"2023-09-13T03:11:12.143398+0000","last_became_active":"2023-09-13T03:11:10.955726+0000","last_became_peered":"2023-09-13T03:11:10.955726+0000","last_unstale":"2023-09-13T03:11:12.143398+0000","last_undegraded":"2023-09-13T03:11:12.143398+0000","last_fullsized":"2023-09-13T03:11:12.143398+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:55:45.667752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-13T03:11:10.971586+0000","last_change":"2023-09-13T03:11:09.949951+0000","last_active":"2023-09-13T03:11:10.971586+0000","last_peered":"2023-09-13T03:11:10.971586+0000","last_clean":"2023-09-13T03:11:10.971586+0000","last_became_active":"2023-09-13T03:11:09.949106+0000","last_became_peered":"2023-09-13T03:11:09.949106+0000","last_unstale":"2023-09-13T03:11:10.971586+0000","last_undegraded":"2023-09-13T03:11:10.971586+0000","last_fullsized":"2023-09-13T03:11:10.971586+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:30:30.092007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643320+0000","last_change":"2023-09-13T03:11:10.942255+0000","last_active":"2023-09-13T03:11:12.643320+0000","last_peered":"2023-09-13T03:11:12.643320+0000","last_clean":"2023-09-13T03:11:12.643320+0000","last_became_active":"2023-09-13T03:11:10.941953+0000","last_became_peered":"2023-09-13T03:11:10.941953+0000","last_unstale":"2023-09-13T03:11:12.643320+0000","last_undegraded":"2023-09-13T03:11:12.643320+0000","last_fullsized":"2023-09-13T03:11:12.643320+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:38:55.260227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:11.937747+0000","last_change":"2023-09-13T03:11:09.956042+0000","last_active":"2023-09-13T03:11:11.937747+0000","last_peered":"2023-09-13T03:11:11.937747+0000","last_clean":"2023-09-13T03:11:11.937747+0000","last_became_active":"2023-09-13T03:11:09.955669+0000","last_became_peered":"2023-09-13T03:11:09.955669+0000","last_unstale":"2023-09-13T03:11:11.937747+0000","last_undegraded":"2023-09-13T03:11:11.937747+0000","last_fullsized":"2023-09-13T03:11:11.937747+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:16:25.227447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643490+0000","last_change":"2023-09-13T03:11:10.952421+0000","last_active":"2023-09-13T03:11:12.643490+0000","last_peered":"2023-09-13T03:11:12.643490+0000","last_clean":"2023-09-13T03:11:12.643490+0000","last_became_active":"2023-09-13T03:11:10.951664+0000","last_became_peered":"2023-09-13T03:11:10.951664+0000","last_unstale":"2023-09-13T03:11:12.643490+0000","last_undegraded":"2023-09-13T03:11:12.643490+0000","last_fullsized":"2023-09-13T03:11:12.643490+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:46:52.664524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'5","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:18.243989+0000","last_change":"2023-09-13T03:11:09.952286+0000","last_active":"2023-09-13T03:11:18.243989+0000","last_peered":"2023-09-13T03:11:18.243989+0000","last_clean":"2023-09-13T03:11:18.243989+0000","last_became_active":"2023-09-13T03:11:09.951999+0000","last_became_peered":"2023-09-13T03:11:09.951999+0000","last_unstale":"2023-09-13T03:11:18.243989+0000","last_undegraded":"2023-09-13T03:11:18.243989+0000","last_fullsized":"2023-09-13T03:11:18.243989+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:54:46.910087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643290+0000","last_change":"2023-09-13T03:11:10.950866+0000","last_active":"2023-09-13T03:11:12.643290+0000","last_peered":"2023-09-13T03:11:12.643290+0000","last_clean":"2023-09-13T03:11:12.643290+0000","last_became_active":"2023-09-13T03:11:10.950562+0000","last_became_peered":"2023-09-13T03:11:10.950562+0000","last_unstale":"2023-09-13T03:11:12.643290+0000","last_undegraded":"2023-09-13T03:11:12.643290+0000","last_fullsized":"2023-09-13T03:11:12.643290+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:55:07.333207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.143503+0000","last_change":"2023-09-13T03:11:10.950736+0000","last_active":"2023-09-13T03:11:12.143503+0000","last_peered":"2023-09-13T03:11:12.143503+0000","last_clean":"2023-09-13T03:11:12.143503+0000","last_became_active":"2023-09-13T03:11:10.950164+0000","last_became_peered":"2023-09-13T03:11:10.950164+0000","last_unstale":"2023-09-13T03:11:12.143503+0000","last_undegraded":"2023-09-13T03:11:12.143503+0000","last_fullsized":"2023-09-13T03:11:12.143503+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:22:44.453642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:18.238085+0000","last_change":"2023-09-13T03:11:09.954070+0000","last_active":"2023-09-13T03:11:18.238085+0000","last_peered":"2023-09-13T03:11:18.238085+0000","last_clean":"2023-09-13T03:11:18.238085+0000","last_became_active":"2023-09-13T03:11:09.953735+0000","last_became_peered":"2023-09-13T03:11:09.953735+0000","last_unstale":"2023-09-13T03:11:18.238085+0000","last_undegraded":"2023-09-13T03:11:18.238085+0000","last_fullsized":"2023-09-13T03:11:18.238085+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:36:59.891449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643652+0000","last_change":"2023-09-13T03:11:10.948509+0000","last_active":"2023-09-13T03:11:12.643652+0000","last_peered":"2023-09-13T03:11:12.643652+0000","last_clean":"2023-09-13T03:11:12.643652+0000","last_became_active":"2023-09-13T03:11:10.948149+0000","last_became_peered":"2023-09-13T03:11:10.948149+0000","last_unstale":"2023-09-13T03:11:12.643652+0000","last_undegraded":"2023-09-13T03:11:12.643652+0000","last_fullsized":"2023-09-13T03:11:12.643652+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:45:54.197368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:16.218695+0000","last_change":"2023-09-13T03:11:09.954476+0000","last_active":"2023-09-13T03:11:16.218695+0000","last_peered":"2023-09-13T03:11:16.218695+0000","last_clean":"2023-09-13T03:11:16.218695+0000","last_became_active":"2023-09-13T03:11:09.954094+0000","last_became_peered":"2023-09-13T03:11:09.954094+0000","last_unstale":"2023-09-13T03:11:16.218695+0000","last_undegraded":"2023-09-13T03:11:16.218695+0000","last_fullsized":"2023-09-13T03:11:16.218695+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:08:23.815751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":318,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-13T03:11:10.943859+0000","last_change":"2023-09-13T03:10:05.689061+0000","last_active":"2023-09-13T03:11:10.943859+0000","last_peered":"2023-09-13T03:11:10.943859+0000","last_clean":"2023-09-13T03:11:10.943859+0000","last_became_active":"2023-09-13T03:10:05.688557+0000","last_became_peered":"2023-09-13T03:10:05.688557+0000","last_unstale":"2023-09-13T03:11:10.943859+0000","last_undegraded":"2023-09-13T03:11:10.943859+0000","last_fullsized":"2023-09-13T03:11:10.943859+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:06:59.216900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:06:59.216900+0000","last_clean_scrub_stamp":"2023-09-13T03:06:59.216900+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:10:03.279019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643866+0000","last_change":"2023-09-13T03:11:10.952523+0000","last_active":"2023-09-13T03:11:12.643866+0000","last_peered":"2023-09-13T03:11:12.643866+0000","last_clean":"2023-09-13T03:11:12.643866+0000","last_became_active":"2023-09-13T03:11:10.952193+0000","last_became_peered":"2023-09-13T03:11:10.952193+0000","last_unstale":"2023-09-13T03:11:12.643866+0000","last_undegraded":"2023-09-13T03:11:12.643866+0000","last_fullsized":"2023-09-13T03:11:12.643866+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:31:01.811841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:15.210584+0000","last_change":"2023-09-13T03:11:09.950414+0000","last_active":"2023-09-13T03:11:15.210584+0000","last_peered":"2023-09-13T03:11:15.210584+0000","last_clean":"2023-09-13T03:11:15.210584+0000","last_became_active":"2023-09-13T03:11:09.950057+0000","last_became_peered":"2023-09-13T03:11:09.950057+0000","last_unstale":"2023-09-13T03:11:15.210584+0000","last_undegraded":"2023-09-13T03:11:15.210584+0000","last_fullsized":"2023-09-13T03:11:15.210584+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:57:01.317880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643345+0000","last_change":"2023-09-13T03:11:09.955535+0000","last_active":"2023-09-13T03:11:12.643345+0000","last_peered":"2023-09-13T03:11:12.643345+0000","last_clean":"2023-09-13T03:11:12.643345+0000","last_became_active":"2023-09-13T03:11:09.955212+0000","last_became_peered":"2023-09-13T03:11:09.955212+0000","last_unstale":"2023-09-13T03:11:12.643345+0000","last_undegraded":"2023-09-13T03:11:12.643345+0000","last_fullsized":"2023-09-13T03:11:12.643345+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:15:04.365337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.644116+0000","last_change":"2023-09-13T03:11:10.954255+0000","last_active":"2023-09-13T03:11:12.644116+0000","last_peered":"2023-09-13T03:11:12.644116+0000","last_clean":"2023-09-13T03:11:12.644116+0000","last_became_active":"2023-09-13T03:11:10.953866+0000","last_became_peered":"2023-09-13T03:11:10.953866+0000","last_unstale":"2023-09-13T03:11:12.644116+0000","last_undegraded":"2023-09-13T03:11:12.644116+0000","last_fullsized":"2023-09-13T03:11:12.644116+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:25:12.281218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'7","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:16.220083+0000","last_change":"2023-09-13T03:11:09.956402+0000","last_active":"2023-09-13T03:11:16.220083+0000","last_peered":"2023-09-13T03:11:16.220083+0000","last_clean":"2023-09-13T03:11:16.220083+0000","last_became_active":"2023-09-13T03:11:09.954710+0000","last_became_peered":"2023-09-13T03:11:09.954710+0000","last_unstale":"2023-09-13T03:11:16.220083+0000","last_undegraded":"2023-09-13T03:11:16.220083+0000","last_fullsized":"2023-09-13T03:11:16.220083+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:07:10.256221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643490+0000","last_change":"2023-09-13T03:11:10.949037+0000","last_active":"2023-09-13T03:11:12.643490+0000","last_peered":"2023-09-13T03:11:12.643490+0000","last_clean":"2023-09-13T03:11:12.643490+0000","last_became_active":"2023-09-13T03:11:10.948641+0000","last_became_peered":"2023-09-13T03:11:10.948641+0000","last_unstale":"2023-09-13T03:11:12.643490+0000","last_undegraded":"2023-09-13T03:11:12.643490+0000","last_fullsized":"2023-09-13T03:11:12.643490+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:00:57.563300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:17.234038+0000","last_change":"2023-09-13T03:11:09.946435+0000","last_active":"2023-09-13T03:11:17.234038+0000","last_peered":"2023-09-13T03:11:17.234038+0000","last_clean":"2023-09-13T03:11:17.234038+0000","last_became_active":"2023-09-13T03:11:09.946176+0000","last_became_peered":"2023-09-13T03:11:09.946176+0000","last_unstale":"2023-09-13T03:11:17.234038+0000","last_undegraded":"2023-09-13T03:11:17.234038+0000","last_fullsized":"2023-09-13T03:11:17.234038+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:34:16.055188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643287+0000","last_change":"2023-09-13T03:11:10.957035+0000","last_active":"2023-09-13T03:11:12.643287+0000","last_peered":"2023-09-13T03:11:12.643287+0000","last_clean":"2023-09-13T03:11:12.643287+0000","last_became_active":"2023-09-13T03:11:10.954656+0000","last_became_peered":"2023-09-13T03:11:10.954656+0000","last_unstale":"2023-09-13T03:11:12.643287+0000","last_undegraded":"2023-09-13T03:11:12.643287+0000","last_fullsized":"2023-09-13T03:11:12.643287+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:54:51.142734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:14.199239+0000","last_change":"2023-09-13T03:11:09.959358+0000","last_active":"2023-09-13T03:11:14.199239+0000","last_peered":"2023-09-13T03:11:14.199239+0000","last_clean":"2023-09-13T03:11:14.199239+0000","last_became_active":"2023-09-13T03:11:09.959129+0000","last_became_peered":"2023-09-13T03:11:09.959129+0000","last_unstale":"2023-09-13T03:11:14.199239+0000","last_undegraded":"2023-09-13T03:11:14.199239+0000","last_fullsized":"2023-09-13T03:11:14.199239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:29:26.930910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643075+0000","last_change":"2023-09-13T03:11:10.952588+0000","last_active":"2023-09-13T03:11:12.643075+0000","last_peered":"2023-09-13T03:11:12.643075+0000","last_clean":"2023-09-13T03:11:12.643075+0000","last_became_active":"2023-09-13T03:11:10.952304+0000","last_became_peered":"2023-09-13T03:11:10.952304+0000","last_unstale":"2023-09-13T03:11:12.643075+0000","last_undegraded":"2023-09-13T03:11:12.643075+0000","last_fullsized":"2023-09-13T03:11:12.643075+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:05:49.824053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:17.232352+0000","last_change":"2023-09-13T03:11:09.956947+0000","last_active":"2023-09-13T03:11:17.232352+0000","last_peered":"2023-09-13T03:11:17.232352+0000","last_clean":"2023-09-13T03:11:17.232352+0000","last_became_active":"2023-09-13T03:11:09.956639+0000","last_became_peered":"2023-09-13T03:11:09.956639+0000","last_unstale":"2023-09-13T03:11:17.232352+0000","last_undegraded":"2023-09-13T03:11:17.232352+0000","last_fullsized":"2023-09-13T03:11:17.232352+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:21:37.759130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.642869+0000","last_change":"2023-09-13T03:11:10.947314+0000","last_active":"2023-09-13T03:11:12.642869+0000","last_peered":"2023-09-13T03:11:12.642869+0000","last_clean":"2023-09-13T03:11:12.642869+0000","last_became_active":"2023-09-13T03:11:10.946964+0000","last_became_peered":"2023-09-13T03:11:10.946964+0000","last_unstale":"2023-09-13T03:11:12.642869+0000","last_undegraded":"2023-09-13T03:11:12.642869+0000","last_fullsized":"2023-09-13T03:11:12.642869+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:55:08.542303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:15.329040+0000","last_change":"2023-09-13T03:11:09.952540+0000","last_active":"2023-09-13T03:11:15.329040+0000","last_peered":"2023-09-13T03:11:15.329040+0000","last_clean":"2023-09-13T03:11:15.329040+0000","last_became_active":"2023-09-13T03:11:09.952208+0000","last_became_peered":"2023-09-13T03:11:09.952208+0000","last_unstale":"2023-09-13T03:11:15.329040+0000","last_undegraded":"2023-09-13T03:11:15.329040+0000","last_fullsized":"2023-09-13T03:11:15.329040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:11:41.200401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643016+0000","last_change":"2023-09-13T03:11:10.951305+0000","last_active":"2023-09-13T03:11:12.643016+0000","last_peered":"2023-09-13T03:11:12.643016+0000","last_clean":"2023-09-13T03:11:12.643016+0000","last_became_active":"2023-09-13T03:11:10.950982+0000","last_became_peered":"2023-09-13T03:11:10.950982+0000","last_unstale":"2023-09-13T03:11:12.643016+0000","last_undegraded":"2023-09-13T03:11:12.643016+0000","last_fullsized":"2023-09-13T03:11:12.643016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:55:04.497698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:14.199721+0000","last_change":"2023-09-13T03:11:09.955410+0000","last_active":"2023-09-13T03:11:14.199721+0000","last_peered":"2023-09-13T03:11:14.199721+0000","last_clean":"2023-09-13T03:11:14.199721+0000","last_became_active":"2023-09-13T03:11:09.955107+0000","last_became_peered":"2023-09-13T03:11:09.955107+0000","last_unstale":"2023-09-13T03:11:14.199721+0000","last_undegraded":"2023-09-13T03:11:14.199721+0000","last_fullsized":"2023-09-13T03:11:14.199721+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:34:05.108897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:13.443411+0000","last_change":"2023-09-13T03:11:10.959487+0000","last_active":"2023-09-13T03:11:13.443411+0000","last_peered":"2023-09-13T03:11:13.443411+0000","last_clean":"2023-09-13T03:11:13.443411+0000","last_became_active":"2023-09-13T03:11:10.959157+0000","last_became_peered":"2023-09-13T03:11:10.959157+0000","last_unstale":"2023-09-13T03:11:13.443411+0000","last_undegraded":"2023-09-13T03:11:13.443411+0000","last_fullsized":"2023-09-13T03:11:13.443411+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:37:50.852197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'5","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:18.243289+0000","last_change":"2023-09-13T03:11:09.951208+0000","last_active":"2023-09-13T03:11:18.243289+0000","last_peered":"2023-09-13T03:11:18.243289+0000","last_clean":"2023-09-13T03:11:18.243289+0000","last_became_active":"2023-09-13T03:11:09.950869+0000","last_became_peered":"2023-09-13T03:11:09.950869+0000","last_unstale":"2023-09-13T03:11:18.243289+0000","last_undegraded":"2023-09-13T03:11:18.243289+0000","last_fullsized":"2023-09-13T03:11:18.243289+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:22:53.697803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:13.443650+0000","last_change":"2023-09-13T03:11:10.959129+0000","last_active":"2023-09-13T03:11:13.443650+0000","last_peered":"2023-09-13T03:11:13.443650+0000","last_clean":"2023-09-13T03:11:13.443650+0000","last_became_active":"2023-09-13T03:11:10.958820+0000","last_became_peered":"2023-09-13T03:11:10.958820+0000","last_unstale":"2023-09-13T03:11:13.443650+0000","last_undegraded":"2023-09-13T03:11:13.443650+0000","last_fullsized":"2023-09-13T03:11:13.443650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:20:30.607854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:16.219213+0000","last_change":"2023-09-13T03:11:09.946855+0000","last_active":"2023-09-13T03:11:16.219213+0000","last_peered":"2023-09-13T03:11:16.219213+0000","last_clean":"2023-09-13T03:11:16.219213+0000","last_became_active":"2023-09-13T03:11:09.946156+0000","last_became_peered":"2023-09-13T03:11:09.946156+0000","last_unstale":"2023-09-13T03:11:16.219213+0000","last_undegraded":"2023-09-13T03:11:16.219213+0000","last_fullsized":"2023-09-13T03:11:16.219213+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:42:19.387280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:14.514465+0000","last_change":"2023-09-13T03:11:10.951335+0000","last_active":"2023-09-13T03:11:14.514465+0000","last_peered":"2023-09-13T03:11:14.514465+0000","last_clean":"2023-09-13T03:11:14.514465+0000","last_became_active":"2023-09-13T03:11:10.951034+0000","last_became_peered":"2023-09-13T03:11:10.951034+0000","last_unstale":"2023-09-13T03:11:14.514465+0000","last_undegraded":"2023-09-13T03:11:14.514465+0000","last_fullsized":"2023-09-13T03:11:14.514465+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:57:24.373196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:17.232198+0000","last_change":"2023-09-13T03:11:09.950937+0000","last_active":"2023-09-13T03:11:17.232198+0000","last_peered":"2023-09-13T03:11:17.232198+0000","last_clean":"2023-09-13T03:11:17.232198+0000","last_became_active":"2023-09-13T03:11:09.950574+0000","last_became_peered":"2023-09-13T03:11:09.950574+0000","last_unstale":"2023-09-13T03:11:17.232198+0000","last_undegraded":"2023-09-13T03:11:17.232198+0000","last_fullsized":"2023-09-13T03:11:17.232198+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:10:23.215080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:13.443547+0000","last_change":"2023-09-13T03:11:10.950517+0000","last_active":"2023-09-13T03:11:13.443547+0000","last_peered":"2023-09-13T03:11:13.443547+0000","last_clean":"2023-09-13T03:11:13.443547+0000","last_became_active":"2023-09-13T03:11:10.950028+0000","last_became_peered":"2023-09-13T03:11:10.950028+0000","last_unstale":"2023-09-13T03:11:13.443547+0000","last_undegraded":"2023-09-13T03:11:13.443547+0000","last_fullsized":"2023-09-13T03:11:13.443547+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:06:40.008527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643049+0000","last_change":"2023-09-13T03:11:10.942192+0000","last_active":"2023-09-13T03:11:12.643049+0000","last_peered":"2023-09-13T03:11:12.643049+0000","last_clean":"2023-09-13T03:11:12.643049+0000","last_became_active":"2023-09-13T03:11:10.941893+0000","last_became_peered":"2023-09-13T03:11:10.941893+0000","last_unstale":"2023-09-13T03:11:12.643049+0000","last_undegraded":"2023-09-13T03:11:12.643049+0000","last_fullsized":"2023-09-13T03:11:12.643049+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:26:22.064843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:11.937442+0000","last_change":"2023-09-13T03:11:10.947186+0000","last_active":"2023-09-13T03:11:11.937442+0000","last_peered":"2023-09-13T03:11:11.937442+0000","last_clean":"2023-09-13T03:11:11.937442+0000","last_became_active":"2023-09-13T03:11:10.946902+0000","last_became_peered":"2023-09-13T03:11:10.946902+0000","last_unstale":"2023-09-13T03:11:11.937442+0000","last_undegraded":"2023-09-13T03:11:11.937442+0000","last_fullsized":"2023-09-13T03:11:11.937442+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:26:01.325597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:11.937308+0000","last_change":"2023-09-13T03:11:10.946504+0000","last_active":"2023-09-13T03:11:11.937308+0000","last_peered":"2023-09-13T03:11:11.937308+0000","last_clean":"2023-09-13T03:11:11.937308+0000","last_became_active":"2023-09-13T03:11:10.946196+0000","last_became_peered":"2023-09-13T03:11:10.946196+0000","last_unstale":"2023-09-13T03:11:11.937308+0000","last_undegraded":"2023-09-13T03:11:11.937308+0000","last_fullsized":"2023-09-13T03:11:11.937308+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:37:43.578685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643668+0000","last_change":"2023-09-13T03:11:10.959637+0000","last_active":"2023-09-13T03:11:12.643668+0000","last_peered":"2023-09-13T03:11:12.643668+0000","last_clean":"2023-09-13T03:11:12.643668+0000","last_became_active":"2023-09-13T03:11:10.959324+0000","last_became_peered":"2023-09-13T03:11:10.959324+0000","last_unstale":"2023-09-13T03:11:12.643668+0000","last_undegraded":"2023-09-13T03:11:12.643668+0000","last_fullsized":"2023-09-13T03:11:12.643668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:48:19.245785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.644275+0000","last_change":"2023-09-13T03:11:10.948678+0000","last_active":"2023-09-13T03:11:12.644275+0000","last_peered":"2023-09-13T03:11:12.644275+0000","last_clean":"2023-09-13T03:11:12.644275+0000","last_became_active":"2023-09-13T03:11:10.948325+0000","last_became_peered":"2023-09-13T03:11:10.948325+0000","last_unstale":"2023-09-13T03:11:12.644275+0000","last_undegraded":"2023-09-13T03:11:12.644275+0000","last_fullsized":"2023-09-13T03:11:12.644275+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:59:36.340698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643176+0000","last_change":"2023-09-13T03:11:10.945076+0000","last_active":"2023-09-13T03:11:12.643176+0000","last_peered":"2023-09-13T03:11:12.643176+0000","last_clean":"2023-09-13T03:11:12.643176+0000","last_became_active":"2023-09-13T03:11:10.944776+0000","last_became_peered":"2023-09-13T03:11:10.944776+0000","last_unstale":"2023-09-13T03:11:12.643176+0000","last_undegraded":"2023-09-13T03:11:12.643176+0000","last_fullsized":"2023-09-13T03:11:12.643176+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:22:03.624488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:11.937646+0000","last_change":"2023-09-13T03:11:10.942663+0000","last_active":"2023-09-13T03:11:11.937646+0000","last_peered":"2023-09-13T03:11:11.937646+0000","last_clean":"2023-09-13T03:11:11.937646+0000","last_became_active":"2023-09-13T03:11:10.942203+0000","last_became_peered":"2023-09-13T03:11:10.942203+0000","last_unstale":"2023-09-13T03:11:11.937646+0000","last_undegraded":"2023-09-13T03:11:11.937646+0000","last_fullsized":"2023-09-13T03:11:11.937646+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:46:48.091425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643342+0000","last_change":"2023-09-13T03:11:10.940089+0000","last_active":"2023-09-13T03:11:12.643342+0000","last_peered":"2023-09-13T03:11:12.643342+0000","last_clean":"2023-09-13T03:11:12.643342+0000","last_became_active":"2023-09-13T03:11:10.939622+0000","last_became_peered":"2023-09-13T03:11:10.939622+0000","last_unstale":"2023-09-13T03:11:12.643342+0000","last_undegraded":"2023-09-13T03:11:12.643342+0000","last_fullsized":"2023-09-13T03:11:12.643342+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:43:55.024944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.143263+0000","last_change":"2023-09-13T03:11:10.956896+0000","last_active":"2023-09-13T03:11:12.143263+0000","last_peered":"2023-09-13T03:11:12.143263+0000","last_clean":"2023-09-13T03:11:12.143263+0000","last_became_active":"2023-09-13T03:11:10.956624+0000","last_became_peered":"2023-09-13T03:11:10.956624+0000","last_unstale":"2023-09-13T03:11:12.143263+0000","last_undegraded":"2023-09-13T03:11:12.143263+0000","last_fullsized":"2023-09-13T03:11:12.143263+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:29:41.305999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643108+0000","last_change":"2023-09-13T03:11:10.939240+0000","last_active":"2023-09-13T03:11:12.643108+0000","last_peered":"2023-09-13T03:11:12.643108+0000","last_clean":"2023-09-13T03:11:12.643108+0000","last_became_active":"2023-09-13T03:11:10.938869+0000","last_became_peered":"2023-09-13T03:11:10.938869+0000","last_unstale":"2023-09-13T03:11:12.643108+0000","last_undegraded":"2023-09-13T03:11:12.643108+0000","last_fullsized":"2023-09-13T03:11:12.643108+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:06:44.931165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643340+0000","last_change":"2023-09-13T03:11:10.956704+0000","last_active":"2023-09-13T03:11:12.643340+0000","last_peered":"2023-09-13T03:11:12.643340+0000","last_clean":"2023-09-13T03:11:12.643340+0000","last_became_active":"2023-09-13T03:11:10.956413+0000","last_became_peered":"2023-09-13T03:11:10.956413+0000","last_unstale":"2023-09-13T03:11:12.643340+0000","last_undegraded":"2023-09-13T03:11:12.643340+0000","last_fullsized":"2023-09-13T03:11:12.643340+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:50:04.626111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.644102+0000","last_change":"2023-09-13T03:11:10.952707+0000","last_active":"2023-09-13T03:11:12.644102+0000","last_peered":"2023-09-13T03:11:12.644102+0000","last_clean":"2023-09-13T03:11:12.644102+0000","last_became_active":"2023-09-13T03:11:10.952423+0000","last_became_peered":"2023-09-13T03:11:10.952423+0000","last_unstale":"2023-09-13T03:11:12.644102+0000","last_undegraded":"2023-09-13T03:11:12.644102+0000","last_fullsized":"2023-09-13T03:11:12.644102+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:35:09.869172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643041+0000","last_change":"2023-09-13T03:11:10.941096+0000","last_active":"2023-09-13T03:11:12.643041+0000","last_peered":"2023-09-13T03:11:12.643041+0000","last_clean":"2023-09-13T03:11:12.643041+0000","last_became_active":"2023-09-13T03:11:10.940761+0000","last_became_peered":"2023-09-13T03:11:10.940761+0000","last_unstale":"2023-09-13T03:11:12.643041+0000","last_undegraded":"2023-09-13T03:11:12.643041+0000","last_fullsized":"2023-09-13T03:11:12.643041+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:34:27.389650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":8442,"num_objects":86,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":10,"num_read_kb":7,"num_write":186,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":57,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":69700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":162,"ondisk_log_size":162,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449101,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":427010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]}]},{"osd":10,"up_from":71,"seq":304942678032,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28516,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716636,"statfs":{"total":95995035648,"available":95965835264,"internally_reserved":0,"allocated":1536000,"data_stored":1019500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":1,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.44},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.47099999999999997}]},{"osd":2,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.69599999999999995}]},{"osd":3,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.55900000000000005}]},{"osd":4,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.39000000000000001}]},{"osd":5,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.64100000000000001}]},{"osd":6,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":0.85699999999999998}]},{"osd":7,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.47599999999999998}]},{"osd":8,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.63}]},{"osd":9,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.68400000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]}]},{"osd":9,"up_from":66,"seq":283467841557,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":426726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.376}]},{"osd":1,"last update":"Wed Sep 13 03:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.80000000000000004}]},{"osd":2,"last update":"Wed Sep 13 03:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.53800000000000003}]},{"osd":3,"last update":"Wed Sep 13 03:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.55900000000000005}]},{"osd":4,"last update":"Wed Sep 13 03:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.69599999999999995}]},{"osd":5,"last update":"Wed Sep 13 03:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":0.622},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":4.2590000000000003,"5min":4.2590000000000003,"15min":4.2590000000000003},"last":0.77000000000000002}]},{"osd":6,"last update":"Wed Sep 13 03:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.59099999999999997}]},{"osd":7,"last update":"Wed Sep 13 03:10:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":4.367,"5min":4.367,"15min":4.367},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":4.3520000000000003,"5min":4.3520000000000003,"15min":4.3520000000000003},"last":0.92500000000000004}]},{"osd":8,"last update":"Wed Sep 13 03:10:40 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":4.298,"5min":4.298,"15min":4.298},"last":0.746}]},{"osd":10,"last update":"Wed Sep 13 03:11:05 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.81999999999999995}]}]},{"osd":8,"up_from":58,"seq":249108103193,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":426225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:10:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.71499999999999997}]},{"osd":1,"last update":"Wed Sep 13 03:10:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.57999999999999996}]},{"osd":2,"last update":"Wed Sep 13 03:10:16 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.745},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.65400000000000003}]},{"osd":3,"last update":"Wed Sep 13 03:10:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.80400000000000005}]},{"osd":4,"last update":"Wed Sep 13 03:10:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.626},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.879}]},{"osd":5,"last update":"Wed Sep 13 03:10:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.83999999999999997}]},{"osd":6,"last update":"Wed Sep 13 03:10:16 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.86899999999999999}]},{"osd":7,"last update":"Wed Sep 13 03:10:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.82799999999999996}]},{"osd":9,"last update":"Wed Sep 13 03:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.92200000000000004}]},{"osd":10,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":2.004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]}]},{"osd":7,"up_from":51,"seq":219043332126,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1511424,"data_stored":1019835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:10:55 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.59699999999999998,"15min":0.58599999999999997},"min":{"1min":0.498,"5min":0.438,"15min":0.438},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.378},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60099999999999998,"15min":0.59099999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.41099999999999998}]},{"osd":1,"last update":"Wed Sep 13 03:10:55 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.41399999999999998,"15min":0.40200000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.42099999999999999,"15min":0.41199999999999998},"min":{"1min":0.32000000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.50600000000000001}]},{"osd":2,"last update":"Wed Sep 13 03:10:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.59799999999999998,"15min":0.58999999999999997},"min":{"1min":0.46300000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.57999999999999996,"15min":0.57499999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.58599999999999997}]},{"osd":3,"last update":"Wed Sep 13 03:10:55 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.68000000000000005,"15min":0.628},"min":{"1min":0.496,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":6.2720000000000002,"5min":6.2720000000000002,"15min":6.2720000000000002},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.63600000000000001,"15min":0.622},"min":{"1min":0.54600000000000004,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.66200000000000003}]},{"osd":4,"last update":"Wed Sep 13 03:10:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.52400000000000002,"15min":0.51200000000000001},"min":{"1min":0.35199999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.623},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.54000000000000004,"15min":0.52200000000000002},"min":{"1min":0.36099999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.67700000000000005}]},{"osd":5,"last update":"Wed Sep 13 03:10:55 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.64500000000000002,"15min":0.63700000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.627,"15min":0.61599999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.70099999999999996}]},{"osd":6,"last update":"Wed Sep 13 03:10:55 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.71499999999999997,"15min":0.69699999999999995},"min":{"1min":0.63900000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.73999999999999999,"15min":0.72199999999999998},"min":{"1min":0.67200000000000004,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.85799999999999998}]},{"osd":8,"last update":"Wed Sep 13 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.75},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.76600000000000001}]},{"osd":9,"last update":"Wed Sep 13 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.80500000000000005}]},{"osd":10,"last update":"Wed Sep 13 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.73799999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]}]},{"osd":6,"up_from":46,"seq":197568495651,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":426540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:10:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.501,"15min":0.497},"min":{"1min":0.34200000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53800000000000003,"15min":0.53100000000000003},"min":{"1min":0.37,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.48799999999999999}]},{"osd":1,"last update":"Wed Sep 13 03:10:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.64100000000000001,"15min":0.63100000000000001},"min":{"1min":0.42799999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.63100000000000001,"15min":0.61699999999999999},"min":{"1min":0.47199999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.59699999999999998}]},{"osd":2,"last update":"Wed Sep 13 03:10:36 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.68200000000000005,"15min":0.67200000000000004},"min":{"1min":0.54600000000000004,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.72399999999999998,"15min":0.71799999999999997},"min":{"1min":0.48099999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.96199999999999997,"5min":1.149,"15min":1.149},"last":0.68700000000000006}]},{"osd":3,"last update":"Wed Sep 13 03:10:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.65500000000000003,"15min":0.64800000000000002},"min":{"1min":0.48299999999999998,"5min":0.374,"15min":0.374},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.626},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.66100000000000003,"15min":0.65500000000000003},"min":{"1min":0.5,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.56000000000000005}]},{"osd":4,"last update":"Wed Sep 13 03:10:36 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.73699999999999999,"15min":0.73099999999999998},"min":{"1min":0.51100000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.752},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.77200000000000002,"15min":0.76800000000000002},"min":{"1min":0.56699999999999995,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.66800000000000004}]},{"osd":5,"last update":"Wed Sep 13 03:10:36 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.58899999999999997,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.97499999999999998,"5min":1.115,"15min":1.115},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.79100000000000004,"15min":0.78400000000000003},"min":{"1min":0.625,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.64900000000000002}]},{"osd":7,"last update":"Wed Sep 13 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.77500000000000002,"15min":0.77000000000000002},"min":{"1min":0.53700000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.76400000000000001,"15min":0.751},"min":{"1min":0.57899999999999996,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.85799999999999998}]},{"osd":8,"last update":"Wed Sep 13 03:10:18 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.77400000000000002}]},{"osd":9,"last update":"Wed Sep 13 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.89200000000000002}]},{"osd":10,"last update":"Wed Sep 13 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.92600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574905,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":446274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.56699999999999995,"15min":0.53800000000000003},"min":{"1min":0.35499999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":2.9260000000000002,"5min":2.9260000000000002,"15min":2.9260000000000002},"last":0.435},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.54100000000000004,"15min":0.53100000000000003},"min":{"1min":0.38700000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.63800000000000001}]},{"osd":2,"last update":"Wed Sep 13 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.54400000000000004,"15min":0.52600000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.54900000000000004,"15min":0.52900000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.66700000000000004}]},{"osd":3,"last update":"Wed Sep 13 03:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59899999999999998,"15min":0.58499999999999996},"min":{"1min":0.41699999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84799999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.61899999999999999,"15min":0.60099999999999998},"min":{"1min":0.5,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.45100000000000001}]},{"osd":4,"last update":"Wed Sep 13 03:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.41199999999999998,"15min":0.38600000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.45000000000000001,"15min":0.42799999999999999},"min":{"1min":0.36099999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.49399999999999999}]},{"osd":5,"last update":"Wed Sep 13 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.57199999999999995,"15min":0.53400000000000003},"min":{"1min":0.44900000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.57799999999999996,"15min":0.54500000000000004},"min":{"1min":0.46500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.78300000000000003}]},{"osd":6,"last update":"Wed Sep 13 03:10:36 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.65600000000000003,"15min":0.65000000000000002},"min":{"1min":0.52700000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.69499999999999995,"15min":0.67200000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.9790000000000001,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":0.53700000000000003}]},{"osd":7,"last update":"Wed Sep 13 03:10:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.57099999999999995,"15min":0.56100000000000005},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.625,"5min":0.60299999999999998,"15min":0.59899999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.622}]},{"osd":8,"last update":"Wed Sep 13 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.79700000000000004}]},{"osd":9,"last update":"Wed Sep 13 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.65400000000000003}]},{"osd":10,"last update":"Wed Sep 13 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.72399999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738430,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":427296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 13 03:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.59299999999999997,"15min":0.55300000000000005},"min":{"1min":0.44700000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58299999999999996,"15min":0.55700000000000005},"min":{"1min":0.40799999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.84799999999999998,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.83699999999999997}]},{"osd":2,"last update":"Wed Sep 13 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.64800000000000002,"15min":0.60799999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.621,"15min":0.59099999999999997},"min":{"1min":0.5,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.86599999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.81100000000000005}]},{"osd":3,"last update":"Wed Sep 13 03:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.46800000000000003,"15min":0.45800000000000002},"min":{"1min":0.309,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.56200000000000006,"15min":0.54700000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.60499999999999998}]},{"osd":4,"last update":"Wed Sep 13 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69299999999999995,"15min":0.69899999999999995},"min":{"1min":0.56200000000000006,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0760000000000001,"5min":1.855,"15min":1.855},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.68899999999999995,"15min":0.68200000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.95199999999999996,"5min":1.665,"15min":1.665},"last":0.71599999999999997}]},{"osd":5,"last update":"Wed Sep 13 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.70299999999999996,"15min":0.69299999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.90300000000000002,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68600000000000005,"15min":0.67400000000000004},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.88400000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.95399999999999996}]},{"osd":6,"last update":"Wed Sep 13 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.68100000000000005,"15min":0.66900000000000004},"min":{"1min":0.40999999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.752,"15min":0.74199999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.68899999999999995}]},{"osd":7,"last update":"Wed Sep 13 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.74299999999999999,"15min":0.73899999999999999},"min":{"1min":0.56299999999999994,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.626},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.80600000000000005,"15min":0.79400000000000004},"min":{"1min":0.74399999999999999,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1379999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.77900000000000003}]},{"osd":8,"last update":"Wed Sep 13 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.94199999999999995}]},{"osd":9,"last update":"Wed Sep 13 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":1.0409999999999999}]},{"osd":10,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":1.0900000000000001},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.97499999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411381,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":447157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.50800000000000001,"15min":0.47599999999999998},"min":{"1min":0.35399999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.68000000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.50800000000000001,"15min":0.48299999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73099999999999998,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.83299999999999996}]},{"osd":1,"last update":"Wed Sep 13 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53100000000000003,"15min":0.51400000000000001},"min":{"1min":0.38400000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.69899999999999995,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.52900000000000003,"15min":0.52500000000000002},"min":{"1min":0.40400000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.58599999999999997}]},{"osd":3,"last update":"Wed Sep 13 03:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.58399999999999996,"15min":0.55600000000000005},"min":{"1min":0.375,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.57699999999999996,"15min":0.55800000000000005},"min":{"1min":0.42799999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.90400000000000003,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.77500000000000002}]},{"osd":4,"last update":"Wed Sep 13 03:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.56000000000000005,"15min":0.55200000000000005},"min":{"1min":0.375,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73999999999999999,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56899999999999995,"15min":0.55900000000000005},"min":{"1min":0.42599999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.752,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.48499999999999999}]},{"osd":5,"last update":"Wed Sep 13 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.442,"15min":0.42399999999999999},"min":{"1min":0.33300000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70499999999999996,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.46100000000000002,"15min":0.442},"min":{"1min":0.29699999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.55500000000000005}]},{"osd":6,"last update":"Wed Sep 13 03:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.626,"15min":0.61699999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.874,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.621,"15min":0.61099999999999999},"min":{"1min":0.50700000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.81799999999999995,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.87}]},{"osd":7,"last update":"Wed Sep 13 03:10:58 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.63700000000000001,"15min":0.63600000000000001},"min":{"1min":0.47499999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.84699999999999998,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.66100000000000003,"15min":0.65900000000000003},"min":{"1min":0.51700000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.83399999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.497}]},{"osd":8,"last update":"Wed Sep 13 03:10:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.68100000000000005}]},{"osd":9,"last update":"Wed Sep 13 03:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.73799999999999999}]},{"osd":10,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.71599999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]}]},{"osd":3,"up_from":25,"seq":107374182449,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":426371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.41399999999999998,"15min":0.39900000000000002},"min":{"1min":0.30299999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.443,"15min":0.42899999999999999},"min":{"1min":0.32800000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.54000000000000004}]},{"osd":1,"last update":"Wed Sep 13 03:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.60699999999999998,"15min":0.58799999999999997},"min":{"1min":0.45200000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.61699999999999999,"15min":0.59199999999999997},"min":{"1min":0.48399999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.69799999999999995}]},{"osd":2,"last update":"Wed Sep 13 03:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.64700000000000002,"15min":0.627},"min":{"1min":0.50800000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.66400000000000003,"15min":0.64000000000000001},"min":{"1min":0.48899999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.83399999999999996}]},{"osd":4,"last update":"Wed Sep 13 03:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.65600000000000003,"15min":0.627},"min":{"1min":0.51400000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.387,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.65000000000000002,"15min":0.61899999999999999},"min":{"1min":0.54000000000000004,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.749}]},{"osd":5,"last update":"Wed Sep 13 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.69499999999999995,"15min":0.68000000000000005},"min":{"1min":0.53500000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1,"5min":1.012,"15min":1.012},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.71999999999999997,"15min":0.70199999999999996},"min":{"1min":0.53500000000000003,"5min":0.498,"15min":0.498},"max":{"1min":0.95999999999999996,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":1.0069999999999999}]},{"osd":6,"last update":"Wed Sep 13 03:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.63,"15min":0.62},"min":{"1min":0.45400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":1.117},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.76000000000000001,"15min":0.746},"min":{"1min":0.60299999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.599,"5min":1.599,"15min":1.599},"last":0.93700000000000006}]},{"osd":7,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.86199999999999999,"15min":0.85399999999999998},"min":{"1min":0.63200000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.4890000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.83999999999999997,"15min":0.82999999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.512,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":1.0269999999999999}]},{"osd":8,"last update":"Wed Sep 13 03:10:17 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.82199999999999995}]},{"osd":9,"last update":"Wed Sep 13 03:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":1.135},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":0.96799999999999997}]},{"osd":10,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.708,"5min":1.708,"15min":1.708},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":1.087}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.175}]}]},{"osd":4,"up_from":33,"seq":141733920812,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":426484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.54500000000000004,"15min":0.51500000000000001},"min":{"1min":0.36699999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.55900000000000005,"15min":0.52900000000000003},"min":{"1min":0.35599999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.63800000000000001}]},{"osd":1,"last update":"Wed Sep 13 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.379,"15min":0.35599999999999998},"min":{"1min":0.27000000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.63300000000000001,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.374},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.39100000000000001,"15min":0.36699999999999999},"min":{"1min":0.27000000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.67800000000000005,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.40999999999999998}]},{"osd":2,"last update":"Wed Sep 13 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57199999999999995,"15min":0.56000000000000005},"min":{"1min":0.42399999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.44},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.55800000000000005,"15min":0.53000000000000003},"min":{"1min":0.377,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.90500000000000003}]},{"osd":3,"last update":"Wed Sep 13 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.61499999999999999,"15min":0.55800000000000005},"min":{"1min":0.40200000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.03,"5min":4.2919999999999998,"15min":4.2919999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.59299999999999997,"15min":0.56799999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.56799999999999995}]},{"osd":5,"last update":"Wed Sep 13 03:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59499999999999997,"15min":0.57599999999999996},"min":{"1min":0.42799999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.85199999999999998,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.57999999999999996,"15min":0.56000000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.81299999999999994,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.625}]},{"osd":6,"last update":"Wed Sep 13 03:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.70799999999999996,"15min":0.69999999999999996},"min":{"1min":0.47899999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.753,"5min":0.72199999999999998,"15min":0.71699999999999997},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.73599999999999999}]},{"osd":7,"last update":"Wed Sep 13 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.621,"15min":0.60799999999999998},"min":{"1min":0.38600000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.629,"15min":0.62},"min":{"1min":0.35999999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.69899999999999995}]},{"osd":8,"last update":"Wed Sep 13 03:10:18 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.71899999999999997}]},{"osd":9,"last update":"Wed Sep 13 03:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.874},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.88800000000000001}]},{"osd":10,"last update":"Wed Sep 13 03:11:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.77900000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]}]},{"osd":5,"up_from":38,"seq":163208757289,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28508,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716644,"statfs":{"total":95995035648,"available":95965843456,"internally_reserved":0,"allocated":1527808,"data_stored":1019396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.59699999999999998,"15min":0.61299999999999999},"min":{"1min":0.39000000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78000000000000003,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.65800000000000003,"15min":0.63600000000000001},"min":{"1min":0.375,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":6.0179999999999998,"5min":6.0179999999999998,"15min":6.0179999999999998},"last":0.58499999999999996}]},{"osd":1,"last update":"Wed Sep 13 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.54000000000000004,"15min":0.51800000000000002},"min":{"1min":0.372,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.53300000000000003,"15min":0.51000000000000001},"min":{"1min":0.45200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.83799999999999997}]},{"osd":2,"last update":"Wed Sep 13 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.40100000000000002,"15min":0.38400000000000001},"min":{"1min":0.29099999999999998,"5min":0.193,"15min":0.193},"max":{"1min":0.71299999999999997,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.434,"5min":0.40600000000000003,"15min":0.38700000000000001},"min":{"1min":0.27000000000000002,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.76800000000000002,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.44800000000000001}]},{"osd":3,"last update":"Wed Sep 13 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.40400000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.69099999999999995,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60099999999999998,"15min":0.58599999999999997},"min":{"1min":0.38300000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.85299999999999998,"5min":1.48,"15min":1.48},"last":0.60699999999999998}]},{"osd":4,"last update":"Wed Sep 13 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.59899999999999998,"15min":0.55700000000000005},"min":{"1min":0.40899999999999997,"5min":0.376,"15min":0.376},"max":{"1min":3.3740000000000001,"5min":3.3740000000000001,"15min":3.3740000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56699999999999995,"15min":0.54900000000000004},"min":{"1min":0.39600000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79400000000000004,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.77400000000000002}]},{"osd":6,"last update":"Wed Sep 13 03:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.40999999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.65500000000000003,"15min":0.65000000000000002},"min":{"1min":0.50600000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.73299999999999998}]},{"osd":7,"last update":"Wed Sep 13 03:10:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.84899999999999998,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.79100000000000004}]},{"osd":8,"last update":"Wed Sep 13 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.84299999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61599999999999999,"15min":0.61699999999999999},"min":{"1min":0.38800000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.91100000000000003,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.64000000000000001}]},{"osd":9,"last update":"Wed Sep 13 03:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":5.9829999999999997,"5min":5.9829999999999997,"15min":5.9829999999999997},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.624}]},{"osd":10,"last update":"Wed Sep 13 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.879}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":25342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":25952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-13T03:11:21.219 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-13T03:11:21.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.451+0000 7f7cc91ff700 1 -- 172.21.15.64:0/3101178346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cc4100610 msgr2=0x7f7cc4100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:21.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.451+0000 7f7cc91ff700 1 --2- 172.21.15.64:0/3101178346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cc4100610 0x7f7cc4100a30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7cb40099c0 tx=0x7f7cb4009cd0 comp rx=0 tx=0).stop 2023-09-13T03:11:21.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.452+0000 7f7cc91ff700 1 -- 172.21.15.64:0/3101178346 shutdown_connections 2023-09-13T03:11:21.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.452+0000 7f7cc91ff700 1 --2- 172.21.15.64:0/3101178346 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7cc41021a0 0x7f7cc4108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:21.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.452+0000 7f7cc91ff700 1 --2- 172.21.15.64:0/3101178346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cc4100610 0x7f7cc4100a30 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:21.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.452+0000 7f7cc91ff700 1 --2- 172.21.15.64:0/3101178346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7cc4101800 0x7f7cc4101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:21.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.452+0000 7f7cc91ff700 1 -- 172.21.15.64:0/3101178346 >> 172.21.15.64:0/3101178346 conn(0x7f7cc40fbb40 msgr2=0x7f7cc40fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:21.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.452+0000 7f7cc91ff700 1 -- 172.21.15.64:0/3101178346 shutdown_connections 2023-09-13T03:11:21.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.452+0000 7f7cc91ff700 1 -- 172.21.15.64:0/3101178346 wait complete. 2023-09-13T03:11:21.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.453+0000 7f7cc91ff700 1 Processor -- start 2023-09-13T03:11:21.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.453+0000 7f7cc91ff700 1 -- start start 2023-09-13T03:11:21.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.454+0000 7f7cc91ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7cc4100610 0x7f7cc41962c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:21.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.454+0000 7f7cc91ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7cc4101800 0x7f7cc4196800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:21.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.454+0000 7f7cc91ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cc41021a0 0x7f7cc419d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:21.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.454+0000 7f7cc91ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc419de40 con 0x7f7cc41021a0 2023-09-13T03:11:21.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.454+0000 7f7cc91ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc419df80 con 0x7f7cc4101800 2023-09-13T03:11:21.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.454+0000 7f7cc91ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc419e110 con 0x7f7cc4100610 2023-09-13T03:11:21.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.454+0000 7f7cc259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7cc4101800 0x7f7cc4196800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:21.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.454+0000 7f7cc2d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7cc4100610 0x7f7cc41962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:21.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.454+0000 7f7cc259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7cc4101800 0x7f7cc4196800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:51106/0 (socket says 172.21.15.64:51106) 2023-09-13T03:11:21.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.454+0000 7f7cc259c700 1 -- 172.21.15.64:0/3726132650 learned_addr learned my addr 172.21.15.64:0/3726132650 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:21.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.454+0000 7f7cc359e700 1 --2- 172.21.15.64:0/3726132650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cc41021a0 0x7f7cc419d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:21.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.454+0000 7f7cc259c700 1 -- 172.21.15.64:0/3726132650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7cc4100610 msgr2=0x7f7cc41962c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:21.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.454+0000 7f7cc259c700 1 --2- 172.21.15.64:0/3726132650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7cc4100610 0x7f7cc41962c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:21.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.454+0000 7f7cc259c700 1 -- 172.21.15.64:0/3726132650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cc41021a0 msgr2=0x7f7cc419d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:21.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.454+0000 7f7cc259c700 1 --2- 172.21.15.64:0/3726132650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cc41021a0 0x7f7cc419d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:21.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.454+0000 7f7cc259c700 1 -- 172.21.15.64:0/3726132650 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb40095f0 con 0x7f7cc4101800 2023-09-13T03:11:21.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.455+0000 7f7cc259c700 1 --2- 172.21.15.64:0/3726132650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7cc4101800 0x7f7cc4196800 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800e9a0 tx=0x7f7cb800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:21.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.455+0000 7f7cabfff700 1 -- 172.21.15.64:0/3726132650 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8009790 con 0x7f7cc4101800 2023-09-13T03:11:21.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.455+0000 7f7cc91ff700 1 -- 172.21.15.64:0/3726132650 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cc4072890 con 0x7f7cc4101800 2023-09-13T03:11:21.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.455+0000 7f7cabfff700 1 -- 172.21.15.64:0/3726132650 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7cb8010430 con 0x7f7cc4101800 2023-09-13T03:11:21.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.455+0000 7f7cabfff700 1 -- 172.21.15.64:0/3726132650 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8018680 con 0x7f7cc4101800 2023-09-13T03:11:21.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.455+0000 7f7cc91ff700 1 -- 172.21.15.64:0/3726132650 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cc4072da0 con 0x7f7cc4101800 2023-09-13T03:11:21.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.456+0000 7f7cc91ff700 1 -- 172.21.15.64:0/3726132650 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cc410be30 con 0x7f7cc4101800 2023-09-13T03:11:21.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.458+0000 7f7cabfff700 1 -- 172.21.15.64:0/3726132650 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f7cb80105a0 con 0x7f7cc4101800 2023-09-13T03:11:21.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.460+0000 7f7cabfff700 1 --2- 172.21.15.64:0/3726132650 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f7cac0a0710 0x7f7cac0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:21.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.460+0000 7f7cabfff700 1 -- 172.21.15.64:0/3726132650 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7cb8014070 con 0x7f7cc4101800 2023-09-13T03:11:21.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.460+0000 7f7cc2d9d700 1 --2- 172.21.15.64:0/3726132650 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f7cac0a0710 0x7f7cac0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:21.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.460+0000 7f7cc2d9d700 1 --2- 172.21.15.64:0/3726132650 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f7cac0a0710 0x7f7cac0a2bd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4009fd0 tx=0x7f7cb4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:21.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.462+0000 7f7cabfff700 1 -- 172.21.15.64:0/3726132650 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f7cb80f8e10 con 0x7f7cc4101800 2023-09-13T03:11:21.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:21 smithi170 ceph-mon[134328]: from='client.24790 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:11:21.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:21 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/526453532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:11:21.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:21 smithi170 ceph-mon[134328]: pgmap v282: 97 pgs: 97 active+clean; 585 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 753 B/s rd, 6.6 KiB/s wr, 20 op/s 2023-09-13T03:11:21.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:21 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/2422588256' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:11:21.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.666+0000 7f7cc91ff700 1 -- 172.21.15.64:0/3726132650 --> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7cc4060a40 con 0x7f7cac0a0710 2023-09-13T03:11:21.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:21 smithi064 ceph-mon[97675]: from='client.24790 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:11:21.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:21 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/526453532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:11:21.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:21 smithi064 ceph-mon[97675]: pgmap v282: 97 pgs: 97 active+clean; 585 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 753 B/s rd, 6.6 KiB/s wr, 20 op/s 2023-09-13T03:11:21.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:21 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2422588256' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:11:21.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.677+0000 7f7cabfff700 1 -- 172.21.15.64:0/3726132650 <== mgr.14150 v2:172.21.15.64:6800/873679308 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+335754 (secure 0 0 0) 0x7f7cc4060a40 con 0x7f7cac0a0710 2023-09-13T03:11:21.682 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:11:21.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.682+0000 7f7cc91ff700 1 -- 172.21.15.64:0/3726132650 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f7cac0a0710 msgr2=0x7f7cac0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:21.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.682+0000 7f7cc91ff700 1 --2- 172.21.15.64:0/3726132650 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f7cac0a0710 0x7f7cac0a2bd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4009fd0 tx=0x7f7cb4005fb0 comp rx=0 tx=0).stop 2023-09-13T03:11:21.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.682+0000 7f7cc91ff700 1 -- 172.21.15.64:0/3726132650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7cc4101800 msgr2=0x7f7cc4196800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:21.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.682+0000 7f7cc91ff700 1 --2- 172.21.15.64:0/3726132650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7cc4101800 0x7f7cc4196800 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800e9a0 tx=0x7f7cb800ecb0 comp rx=0 tx=0).stop 2023-09-13T03:11:21.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.682+0000 7f7cc91ff700 1 -- 172.21.15.64:0/3726132650 shutdown_connections 2023-09-13T03:11:21.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.682+0000 7f7cc91ff700 1 --2- 172.21.15.64:0/3726132650 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f7cac0a0710 0x7f7cac0a2bd0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:21.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.682+0000 7f7cc91ff700 1 --2- 172.21.15.64:0/3726132650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cc41021a0 0x7f7cc419d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:21.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.682+0000 7f7cc91ff700 1 --2- 172.21.15.64:0/3726132650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7cc4100610 0x7f7cc41962c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:21.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.682+0000 7f7cc91ff700 1 --2- 172.21.15.64:0/3726132650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7cc4101800 0x7f7cc4196800 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:21.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.682+0000 7f7cc91ff700 1 -- 172.21.15.64:0/3726132650 >> 172.21.15.64:0/3726132650 conn(0x7f7cc40fbb40 msgr2=0x7f7cc40fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:21.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.682+0000 7f7cc91ff700 1 -- 172.21.15.64:0/3726132650 shutdown_connections 2023-09-13T03:11:21.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:21.682+0000 7f7cc91ff700 1 -- 172.21.15.64:0/3726132650 wait complete. 2023-09-13T03:11:21.687 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-13T03:11:21.701 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":282,"stamp":"2023-09-13T03:11:20.445881+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598933,"num_objects":89,"num_object_clones":0,"num_object_copies":267,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":0,"num_read":146,"num_read_kb":126,"num_write":368,"num_write_kb":3001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":57,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":300,"ondisk_log_size":300,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":282,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336712,"kb_used_data":12520,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605112,"statfs":{"total":1151940427776,"available":1151595634688,"internally_reserved":0,"allocated":12820480,"data_stored":6938814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19130,"internal_metadata":331855174},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8565,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":10,"num_read_kb":7,"num_write":187,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":57,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.516783"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643523+0000","last_change":"2023-09-13T03:11:10.949449+0000","last_active":"2023-09-13T03:11:12.643523+0000","last_peered":"2023-09-13T03:11:12.643523+0000","last_clean":"2023-09-13T03:11:12.643523+0000","last_became_active":"2023-09-13T03:11:10.948912+0000","last_became_peered":"2023-09-13T03:11:10.948912+0000","last_unstale":"2023-09-13T03:11:12.643523+0000","last_undegraded":"2023-09-13T03:11:12.643523+0000","last_fullsized":"2023-09-13T03:11:12.643523+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:34:27.381352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643150+0000","last_change":"2023-09-13T03:11:10.956817+0000","last_active":"2023-09-13T03:11:12.643150+0000","last_peered":"2023-09-13T03:11:12.643150+0000","last_clean":"2023-09-13T03:11:12.643150+0000","last_became_active":"2023-09-13T03:11:10.955359+0000","last_became_peered":"2023-09-13T03:11:10.955359+0000","last_unstale":"2023-09-13T03:11:12.643150+0000","last_undegraded":"2023-09-13T03:11:12.643150+0000","last_fullsized":"2023-09-13T03:11:12.643150+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:06:01.055910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.642907+0000","last_change":"2023-09-13T03:11:10.959915+0000","last_active":"2023-09-13T03:11:12.642907+0000","last_peered":"2023-09-13T03:11:12.642907+0000","last_clean":"2023-09-13T03:11:12.642907+0000","last_became_active":"2023-09-13T03:11:10.959611+0000","last_became_peered":"2023-09-13T03:11:10.959611+0000","last_unstale":"2023-09-13T03:11:12.642907+0000","last_undegraded":"2023-09-13T03:11:12.642907+0000","last_fullsized":"2023-09-13T03:11:12.642907+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:31:40.075569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:11.938190+0000","last_change":"2023-09-13T03:11:10.944329+0000","last_active":"2023-09-13T03:11:11.938190+0000","last_peered":"2023-09-13T03:11:11.938190+0000","last_clean":"2023-09-13T03:11:11.938190+0000","last_became_active":"2023-09-13T03:11:10.944137+0000","last_became_peered":"2023-09-13T03:11:10.944137+0000","last_unstale":"2023-09-13T03:11:11.938190+0000","last_undegraded":"2023-09-13T03:11:11.938190+0000","last_fullsized":"2023-09-13T03:11:11.938190+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:03:58.300005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643437+0000","last_change":"2023-09-13T03:11:10.951554+0000","last_active":"2023-09-13T03:11:12.643437+0000","last_peered":"2023-09-13T03:11:12.643437+0000","last_clean":"2023-09-13T03:11:12.643437+0000","last_became_active":"2023-09-13T03:11:10.951252+0000","last_became_peered":"2023-09-13T03:11:10.951252+0000","last_unstale":"2023-09-13T03:11:12.643437+0000","last_undegraded":"2023-09-13T03:11:12.643437+0000","last_fullsized":"2023-09-13T03:11:12.643437+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:38:42.890256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643324+0000","last_change":"2023-09-13T03:11:10.953087+0000","last_active":"2023-09-13T03:11:12.643324+0000","last_peered":"2023-09-13T03:11:12.643324+0000","last_clean":"2023-09-13T03:11:12.643324+0000","last_became_active":"2023-09-13T03:11:10.952773+0000","last_became_peered":"2023-09-13T03:11:10.952773+0000","last_unstale":"2023-09-13T03:11:12.643324+0000","last_undegraded":"2023-09-13T03:11:12.643324+0000","last_fullsized":"2023-09-13T03:11:12.643324+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:14:55.203347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:13.443065+0000","last_change":"2023-09-13T03:11:10.956058+0000","last_active":"2023-09-13T03:11:13.443065+0000","last_peered":"2023-09-13T03:11:13.443065+0000","last_clean":"2023-09-13T03:11:13.443065+0000","last_became_active":"2023-09-13T03:11:10.955783+0000","last_became_peered":"2023-09-13T03:11:10.955783+0000","last_unstale":"2023-09-13T03:11:13.443065+0000","last_undegraded":"2023-09-13T03:11:13.443065+0000","last_fullsized":"2023-09-13T03:11:13.443065+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:31:06.810338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.642680+0000","last_change":"2023-09-13T03:11:10.950102+0000","last_active":"2023-09-13T03:11:12.642680+0000","last_peered":"2023-09-13T03:11:12.642680+0000","last_clean":"2023-09-13T03:11:12.642680+0000","last_became_active":"2023-09-13T03:11:10.948877+0000","last_became_peered":"2023-09-13T03:11:10.948877+0000","last_unstale":"2023-09-13T03:11:12.642680+0000","last_undegraded":"2023-09-13T03:11:12.642680+0000","last_fullsized":"2023-09-13T03:11:12.642680+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:32:09.168793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643295+0000","last_change":"2023-09-13T03:11:10.945244+0000","last_active":"2023-09-13T03:11:12.643295+0000","last_peered":"2023-09-13T03:11:12.643295+0000","last_clean":"2023-09-13T03:11:12.643295+0000","last_became_active":"2023-09-13T03:11:10.944951+0000","last_became_peered":"2023-09-13T03:11:10.944951+0000","last_unstale":"2023-09-13T03:11:12.643295+0000","last_undegraded":"2023-09-13T03:11:12.643295+0000","last_fullsized":"2023-09-13T03:11:12.643295+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:42:35.420767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643658+0000","last_change":"2023-09-13T03:11:10.952312+0000","last_active":"2023-09-13T03:11:12.643658+0000","last_peered":"2023-09-13T03:11:12.643658+0000","last_clean":"2023-09-13T03:11:12.643658+0000","last_became_active":"2023-09-13T03:11:10.952077+0000","last_became_peered":"2023-09-13T03:11:10.952077+0000","last_unstale":"2023-09-13T03:11:12.643658+0000","last_undegraded":"2023-09-13T03:11:12.643658+0000","last_fullsized":"2023-09-13T03:11:12.643658+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:57:10.888184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:13.443504+0000","last_change":"2023-09-13T03:11:10.959259+0000","last_active":"2023-09-13T03:11:13.443504+0000","last_peered":"2023-09-13T03:11:13.443504+0000","last_clean":"2023-09-13T03:11:13.443504+0000","last_became_active":"2023-09-13T03:11:10.958922+0000","last_became_peered":"2023-09-13T03:11:10.958922+0000","last_unstale":"2023-09-13T03:11:13.443504+0000","last_undegraded":"2023-09-13T03:11:13.443504+0000","last_fullsized":"2023-09-13T03:11:13.443504+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:08:54.680718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:11.938117+0000","last_change":"2023-09-13T03:11:10.950946+0000","last_active":"2023-09-13T03:11:11.938117+0000","last_peered":"2023-09-13T03:11:11.938117+0000","last_clean":"2023-09-13T03:11:11.938117+0000","last_became_active":"2023-09-13T03:11:10.950633+0000","last_became_peered":"2023-09-13T03:11:10.950633+0000","last_unstale":"2023-09-13T03:11:11.938117+0000","last_undegraded":"2023-09-13T03:11:11.938117+0000","last_fullsized":"2023-09-13T03:11:11.938117+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:37:25.796555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.644173+0000","last_change":"2023-09-13T03:11:10.957712+0000","last_active":"2023-09-13T03:11:12.644173+0000","last_peered":"2023-09-13T03:11:12.644173+0000","last_clean":"2023-09-13T03:11:12.644173+0000","last_became_active":"2023-09-13T03:11:10.957379+0000","last_became_peered":"2023-09-13T03:11:10.957379+0000","last_unstale":"2023-09-13T03:11:12.644173+0000","last_undegraded":"2023-09-13T03:11:12.644173+0000","last_fullsized":"2023-09-13T03:11:12.644173+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:31:17.084185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.642902+0000","last_change":"2023-09-13T03:11:10.952951+0000","last_active":"2023-09-13T03:11:12.642902+0000","last_peered":"2023-09-13T03:11:12.642902+0000","last_clean":"2023-09-13T03:11:12.642902+0000","last_became_active":"2023-09-13T03:11:10.952629+0000","last_became_peered":"2023-09-13T03:11:10.952629+0000","last_unstale":"2023-09-13T03:11:12.642902+0000","last_undegraded":"2023-09-13T03:11:12.642902+0000","last_fullsized":"2023-09-13T03:11:12.642902+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:20:40.802584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.642803+0000","last_change":"2023-09-13T03:11:10.950799+0000","last_active":"2023-09-13T03:11:12.642803+0000","last_peered":"2023-09-13T03:11:12.642803+0000","last_clean":"2023-09-13T03:11:12.642803+0000","last_became_active":"2023-09-13T03:11:10.950500+0000","last_became_peered":"2023-09-13T03:11:10.950500+0000","last_unstale":"2023-09-13T03:11:12.642803+0000","last_undegraded":"2023-09-13T03:11:12.642803+0000","last_fullsized":"2023-09-13T03:11:12.642803+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:24:45.770696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.642539+0000","last_change":"2023-09-13T03:11:10.955141+0000","last_active":"2023-09-13T03:11:12.642539+0000","last_peered":"2023-09-13T03:11:12.642539+0000","last_clean":"2023-09-13T03:11:12.642539+0000","last_became_active":"2023-09-13T03:11:10.954840+0000","last_became_peered":"2023-09-13T03:11:10.954840+0000","last_unstale":"2023-09-13T03:11:12.642539+0000","last_undegraded":"2023-09-13T03:11:12.642539+0000","last_fullsized":"2023-09-13T03:11:12.642539+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:56:08.318784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643646+0000","last_change":"2023-09-13T03:11:10.949199+0000","last_active":"2023-09-13T03:11:12.643646+0000","last_peered":"2023-09-13T03:11:12.643646+0000","last_clean":"2023-09-13T03:11:12.643646+0000","last_became_active":"2023-09-13T03:11:10.948506+0000","last_became_peered":"2023-09-13T03:11:10.948506+0000","last_unstale":"2023-09-13T03:11:12.643646+0000","last_undegraded":"2023-09-13T03:11:12.643646+0000","last_fullsized":"2023-09-13T03:11:12.643646+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:04:07.714033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:11.937809+0000","last_change":"2023-09-13T03:11:10.953207+0000","last_active":"2023-09-13T03:11:11.937809+0000","last_peered":"2023-09-13T03:11:11.937809+0000","last_clean":"2023-09-13T03:11:11.937809+0000","last_became_active":"2023-09-13T03:11:10.951566+0000","last_became_peered":"2023-09-13T03:11:10.951566+0000","last_unstale":"2023-09-13T03:11:11.937809+0000","last_undegraded":"2023-09-13T03:11:11.937809+0000","last_fullsized":"2023-09-13T03:11:11.937809+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:23:06.020880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:11.937663+0000","last_change":"2023-09-13T03:11:10.951695+0000","last_active":"2023-09-13T03:11:11.937663+0000","last_peered":"2023-09-13T03:11:11.937663+0000","last_clean":"2023-09-13T03:11:11.937663+0000","last_became_active":"2023-09-13T03:11:10.951382+0000","last_became_peered":"2023-09-13T03:11:10.951382+0000","last_unstale":"2023-09-13T03:11:11.937663+0000","last_undegraded":"2023-09-13T03:11:11.937663+0000","last_fullsized":"2023-09-13T03:11:11.937663+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:29:05.200980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'11","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:16.219903+0000","last_change":"2023-09-13T03:11:09.956193+0000","last_active":"2023-09-13T03:11:16.219903+0000","last_peered":"2023-09-13T03:11:16.219903+0000","last_clean":"2023-09-13T03:11:16.219903+0000","last_became_active":"2023-09-13T03:11:09.955778+0000","last_became_peered":"2023-09-13T03:11:09.955778+0000","last_unstale":"2023-09-13T03:11:16.219903+0000","last_undegraded":"2023-09-13T03:11:16.219903+0000","last_fullsized":"2023-09-13T03:11:16.219903+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:15:57.425730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643666+0000","last_change":"2023-09-13T03:11:10.944437+0000","last_active":"2023-09-13T03:11:12.643666+0000","last_peered":"2023-09-13T03:11:12.643666+0000","last_clean":"2023-09-13T03:11:12.643666+0000","last_became_active":"2023-09-13T03:11:10.943961+0000","last_became_peered":"2023-09-13T03:11:10.943961+0000","last_unstale":"2023-09-13T03:11:12.643666+0000","last_undegraded":"2023-09-13T03:11:12.643666+0000","last_fullsized":"2023-09-13T03:11:12.643666+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:39:43.928334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643677+0000","last_change":"2023-09-13T03:11:09.952017+0000","last_active":"2023-09-13T03:11:12.643677+0000","last_peered":"2023-09-13T03:11:12.643677+0000","last_clean":"2023-09-13T03:11:12.643677+0000","last_became_active":"2023-09-13T03:11:09.951700+0000","last_became_peered":"2023-09-13T03:11:09.951700+0000","last_unstale":"2023-09-13T03:11:12.643677+0000","last_undegraded":"2023-09-13T03:11:12.643677+0000","last_fullsized":"2023-09-13T03:11:12.643677+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:28:50.687271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643185+0000","last_change":"2023-09-13T03:11:10.960198+0000","last_active":"2023-09-13T03:11:12.643185+0000","last_peered":"2023-09-13T03:11:12.643185+0000","last_clean":"2023-09-13T03:11:12.643185+0000","last_became_active":"2023-09-13T03:11:10.959839+0000","last_became_peered":"2023-09-13T03:11:10.959839+0000","last_unstale":"2023-09-13T03:11:12.643185+0000","last_undegraded":"2023-09-13T03:11:12.643185+0000","last_fullsized":"2023-09-13T03:11:12.643185+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:37:16.868093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:15.209158+0000","last_change":"2023-09-13T03:11:09.954684+0000","last_active":"2023-09-13T03:11:15.209158+0000","last_peered":"2023-09-13T03:11:15.209158+0000","last_clean":"2023-09-13T03:11:15.209158+0000","last_became_active":"2023-09-13T03:11:09.954392+0000","last_became_peered":"2023-09-13T03:11:09.954392+0000","last_unstale":"2023-09-13T03:11:15.209158+0000","last_undegraded":"2023-09-13T03:11:15.209158+0000","last_fullsized":"2023-09-13T03:11:15.209158+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:29:20.333063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.642726+0000","last_change":"2023-09-13T03:11:10.944364+0000","last_active":"2023-09-13T03:11:12.642726+0000","last_peered":"2023-09-13T03:11:12.642726+0000","last_clean":"2023-09-13T03:11:12.642726+0000","last_became_active":"2023-09-13T03:11:10.943971+0000","last_became_peered":"2023-09-13T03:11:10.943971+0000","last_unstale":"2023-09-13T03:11:12.642726+0000","last_undegraded":"2023-09-13T03:11:12.642726+0000","last_fullsized":"2023-09-13T03:11:12.642726+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:55:07.831397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-13T03:11:10.974241+0000","last_change":"2023-09-13T03:11:09.952751+0000","last_active":"2023-09-13T03:11:10.974241+0000","last_peered":"2023-09-13T03:11:10.974241+0000","last_clean":"2023-09-13T03:11:10.974241+0000","last_became_active":"2023-09-13T03:11:09.952447+0000","last_became_peered":"2023-09-13T03:11:09.952447+0000","last_unstale":"2023-09-13T03:11:10.974241+0000","last_undegraded":"2023-09-13T03:11:10.974241+0000","last_fullsized":"2023-09-13T03:11:10.974241+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:55:09.971742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.143629+0000","last_change":"2023-09-13T03:11:10.956427+0000","last_active":"2023-09-13T03:11:12.143629+0000","last_peered":"2023-09-13T03:11:12.143629+0000","last_clean":"2023-09-13T03:11:12.143629+0000","last_became_active":"2023-09-13T03:11:10.956113+0000","last_became_peered":"2023-09-13T03:11:10.956113+0000","last_unstale":"2023-09-13T03:11:12.143629+0000","last_undegraded":"2023-09-13T03:11:12.143629+0000","last_fullsized":"2023-09-13T03:11:12.143629+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:21:48.832314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:14.201886+0000","last_change":"2023-09-13T03:11:09.947207+0000","last_active":"2023-09-13T03:11:14.201886+0000","last_peered":"2023-09-13T03:11:14.201886+0000","last_clean":"2023-09-13T03:11:14.201886+0000","last_became_active":"2023-09-13T03:11:09.946787+0000","last_became_peered":"2023-09-13T03:11:09.946787+0000","last_unstale":"2023-09-13T03:11:14.201886+0000","last_undegraded":"2023-09-13T03:11:14.201886+0000","last_fullsized":"2023-09-13T03:11:14.201886+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:28:47.100723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643289+0000","last_change":"2023-09-13T03:11:10.948427+0000","last_active":"2023-09-13T03:11:12.643289+0000","last_peered":"2023-09-13T03:11:12.643289+0000","last_clean":"2023-09-13T03:11:12.643289+0000","last_became_active":"2023-09-13T03:11:10.948095+0000","last_became_peered":"2023-09-13T03:11:10.948095+0000","last_unstale":"2023-09-13T03:11:12.643289+0000","last_undegraded":"2023-09-13T03:11:12.643289+0000","last_fullsized":"2023-09-13T03:11:12.643289+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:55:07.532772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:16.219456+0000","last_change":"2023-09-13T03:11:09.952335+0000","last_active":"2023-09-13T03:11:16.219456+0000","last_peered":"2023-09-13T03:11:16.219456+0000","last_clean":"2023-09-13T03:11:16.219456+0000","last_became_active":"2023-09-13T03:11:09.951974+0000","last_became_peered":"2023-09-13T03:11:09.951974+0000","last_unstale":"2023-09-13T03:11:16.219456+0000","last_undegraded":"2023-09-13T03:11:16.219456+0000","last_fullsized":"2023-09-13T03:11:16.219456+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:15:50.681751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.143776+0000","last_change":"2023-09-13T03:11:10.948962+0000","last_active":"2023-09-13T03:11:12.143776+0000","last_peered":"2023-09-13T03:11:12.143776+0000","last_clean":"2023-09-13T03:11:12.143776+0000","last_became_active":"2023-09-13T03:11:10.948543+0000","last_became_peered":"2023-09-13T03:11:10.948543+0000","last_unstale":"2023-09-13T03:11:12.143776+0000","last_undegraded":"2023-09-13T03:11:12.143776+0000","last_fullsized":"2023-09-13T03:11:12.143776+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:35:28.789698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:15.212445+0000","last_change":"2023-09-13T03:11:09.957644+0000","last_active":"2023-09-13T03:11:15.212445+0000","last_peered":"2023-09-13T03:11:15.212445+0000","last_clean":"2023-09-13T03:11:15.212445+0000","last_became_active":"2023-09-13T03:11:09.957289+0000","last_became_peered":"2023-09-13T03:11:09.957289+0000","last_unstale":"2023-09-13T03:11:15.212445+0000","last_undegraded":"2023-09-13T03:11:15.212445+0000","last_fullsized":"2023-09-13T03:11:15.212445+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:40:48.615322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643712+0000","last_change":"2023-09-13T03:11:10.947561+0000","last_active":"2023-09-13T03:11:12.643712+0000","last_peered":"2023-09-13T03:11:12.643712+0000","last_clean":"2023-09-13T03:11:12.643712+0000","last_became_active":"2023-09-13T03:11:10.947238+0000","last_became_peered":"2023-09-13T03:11:10.947238+0000","last_unstale":"2023-09-13T03:11:12.643712+0000","last_undegraded":"2023-09-13T03:11:12.643712+0000","last_fullsized":"2023-09-13T03:11:12.643712+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:54:31.360657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:16.218673+0000","last_change":"2023-09-13T03:11:09.947806+0000","last_active":"2023-09-13T03:11:16.218673+0000","last_peered":"2023-09-13T03:11:16.218673+0000","last_clean":"2023-09-13T03:11:16.218673+0000","last_became_active":"2023-09-13T03:11:09.947428+0000","last_became_peered":"2023-09-13T03:11:09.947428+0000","last_unstale":"2023-09-13T03:11:16.218673+0000","last_undegraded":"2023-09-13T03:11:16.218673+0000","last_fullsized":"2023-09-13T03:11:16.218673+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:25:00.421873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:11.937578+0000","last_change":"2023-09-13T03:11:10.951637+0000","last_active":"2023-09-13T03:11:11.937578+0000","last_peered":"2023-09-13T03:11:11.937578+0000","last_clean":"2023-09-13T03:11:11.937578+0000","last_became_active":"2023-09-13T03:11:10.951249+0000","last_became_peered":"2023-09-13T03:11:10.951249+0000","last_unstale":"2023-09-13T03:11:11.937578+0000","last_undegraded":"2023-09-13T03:11:11.937578+0000","last_fullsized":"2023-09-13T03:11:11.937578+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:24:27.715167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:14.199142+0000","last_change":"2023-09-13T03:11:09.954343+0000","last_active":"2023-09-13T03:11:14.199142+0000","last_peered":"2023-09-13T03:11:14.199142+0000","last_clean":"2023-09-13T03:11:14.199142+0000","last_became_active":"2023-09-13T03:11:09.954063+0000","last_became_peered":"2023-09-13T03:11:09.954063+0000","last_unstale":"2023-09-13T03:11:14.199142+0000","last_undegraded":"2023-09-13T03:11:14.199142+0000","last_fullsized":"2023-09-13T03:11:14.199142+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:34:50.057539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643086+0000","last_change":"2023-09-13T03:11:10.956994+0000","last_active":"2023-09-13T03:11:12.643086+0000","last_peered":"2023-09-13T03:11:12.643086+0000","last_clean":"2023-09-13T03:11:12.643086+0000","last_became_active":"2023-09-13T03:11:10.956336+0000","last_became_peered":"2023-09-13T03:11:10.956336+0000","last_unstale":"2023-09-13T03:11:12.643086+0000","last_undegraded":"2023-09-13T03:11:12.643086+0000","last_fullsized":"2023-09-13T03:11:12.643086+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:11:54.186143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:17.233876+0000","last_change":"2023-09-13T03:11:09.945775+0000","last_active":"2023-09-13T03:11:17.233876+0000","last_peered":"2023-09-13T03:11:17.233876+0000","last_clean":"2023-09-13T03:11:17.233876+0000","last_became_active":"2023-09-13T03:11:09.945406+0000","last_became_peered":"2023-09-13T03:11:09.945406+0000","last_unstale":"2023-09-13T03:11:17.233876+0000","last_undegraded":"2023-09-13T03:11:17.233876+0000","last_fullsized":"2023-09-13T03:11:17.233876+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:29:26.677351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643014+0000","last_change":"2023-09-13T03:11:10.951941+0000","last_active":"2023-09-13T03:11:12.643014+0000","last_peered":"2023-09-13T03:11:12.643014+0000","last_clean":"2023-09-13T03:11:12.643014+0000","last_became_active":"2023-09-13T03:11:10.951331+0000","last_became_peered":"2023-09-13T03:11:10.951331+0000","last_unstale":"2023-09-13T03:11:12.643014+0000","last_undegraded":"2023-09-13T03:11:12.643014+0000","last_fullsized":"2023-09-13T03:11:12.643014+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:15:53.741657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:14.199417+0000","last_change":"2023-09-13T03:11:09.959503+0000","last_active":"2023-09-13T03:11:14.199417+0000","last_peered":"2023-09-13T03:11:14.199417+0000","last_clean":"2023-09-13T03:11:14.199417+0000","last_became_active":"2023-09-13T03:11:09.959166+0000","last_became_peered":"2023-09-13T03:11:09.959166+0000","last_unstale":"2023-09-13T03:11:14.199417+0000","last_undegraded":"2023-09-13T03:11:14.199417+0000","last_fullsized":"2023-09-13T03:11:14.199417+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:12:36.179520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643402+0000","last_change":"2023-09-13T03:11:10.941575+0000","last_active":"2023-09-13T03:11:12.643402+0000","last_peered":"2023-09-13T03:11:12.643402+0000","last_clean":"2023-09-13T03:11:12.643402+0000","last_became_active":"2023-09-13T03:11:10.941285+0000","last_became_peered":"2023-09-13T03:11:10.941285+0000","last_unstale":"2023-09-13T03:11:12.643402+0000","last_undegraded":"2023-09-13T03:11:12.643402+0000","last_fullsized":"2023-09-13T03:11:12.643402+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:03:09.377471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'11","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:17.234676+0000","last_change":"2023-09-13T03:11:09.948041+0000","last_active":"2023-09-13T03:11:17.234676+0000","last_peered":"2023-09-13T03:11:17.234676+0000","last_clean":"2023-09-13T03:11:17.234676+0000","last_became_active":"2023-09-13T03:11:09.947785+0000","last_became_peered":"2023-09-13T03:11:09.947785+0000","last_unstale":"2023-09-13T03:11:17.234676+0000","last_undegraded":"2023-09-13T03:11:17.234676+0000","last_fullsized":"2023-09-13T03:11:17.234676+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:46:47.348307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2002,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643222+0000","last_change":"2023-09-13T03:11:10.950709+0000","last_active":"2023-09-13T03:11:12.643222+0000","last_peered":"2023-09-13T03:11:12.643222+0000","last_clean":"2023-09-13T03:11:12.643222+0000","last_became_active":"2023-09-13T03:11:10.950395+0000","last_became_peered":"2023-09-13T03:11:10.950395+0000","last_unstale":"2023-09-13T03:11:12.643222+0000","last_undegraded":"2023-09-13T03:11:12.643222+0000","last_fullsized":"2023-09-13T03:11:12.643222+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:50:08.628777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:18.237723+0000","last_change":"2023-09-13T03:11:09.946049+0000","last_active":"2023-09-13T03:11:18.237723+0000","last_peered":"2023-09-13T03:11:18.237723+0000","last_clean":"2023-09-13T03:11:18.237723+0000","last_became_active":"2023-09-13T03:11:09.945398+0000","last_became_peered":"2023-09-13T03:11:09.945398+0000","last_unstale":"2023-09-13T03:11:18.237723+0000","last_undegraded":"2023-09-13T03:11:18.237723+0000","last_fullsized":"2023-09-13T03:11:18.237723+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:57:21.448431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.644175+0000","last_change":"2023-09-13T03:11:10.955763+0000","last_active":"2023-09-13T03:11:12.644175+0000","last_peered":"2023-09-13T03:11:12.644175+0000","last_clean":"2023-09-13T03:11:12.644175+0000","last_became_active":"2023-09-13T03:11:10.955455+0000","last_became_peered":"2023-09-13T03:11:10.955455+0000","last_unstale":"2023-09-13T03:11:12.644175+0000","last_undegraded":"2023-09-13T03:11:12.644175+0000","last_fullsized":"2023-09-13T03:11:12.644175+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:43:37.127490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:16.219283+0000","last_change":"2023-09-13T03:11:09.947739+0000","last_active":"2023-09-13T03:11:16.219283+0000","last_peered":"2023-09-13T03:11:16.219283+0000","last_clean":"2023-09-13T03:11:16.219283+0000","last_became_active":"2023-09-13T03:11:09.947348+0000","last_became_peered":"2023-09-13T03:11:09.947348+0000","last_unstale":"2023-09-13T03:11:16.219283+0000","last_undegraded":"2023-09-13T03:11:16.219283+0000","last_fullsized":"2023-09-13T03:11:16.219283+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:47:58.408877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:13.443369+0000","last_change":"2023-09-13T03:11:10.958140+0000","last_active":"2023-09-13T03:11:13.443369+0000","last_peered":"2023-09-13T03:11:13.443369+0000","last_clean":"2023-09-13T03:11:13.443369+0000","last_became_active":"2023-09-13T03:11:10.957798+0000","last_became_peered":"2023-09-13T03:11:10.957798+0000","last_unstale":"2023-09-13T03:11:13.443369+0000","last_undegraded":"2023-09-13T03:11:13.443369+0000","last_fullsized":"2023-09-13T03:11:13.443369+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:44:40.208559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'6","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:16.218499+0000","last_change":"2023-09-13T03:11:09.945554+0000","last_active":"2023-09-13T03:11:16.218499+0000","last_peered":"2023-09-13T03:11:16.218499+0000","last_clean":"2023-09-13T03:11:16.218499+0000","last_became_active":"2023-09-13T03:11:09.945214+0000","last_became_peered":"2023-09-13T03:11:09.945214+0000","last_unstale":"2023-09-13T03:11:16.218499+0000","last_undegraded":"2023-09-13T03:11:16.218499+0000","last_fullsized":"2023-09-13T03:11:16.218499+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:46:12.338617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:13.442989+0000","last_change":"2023-09-13T03:11:10.952589+0000","last_active":"2023-09-13T03:11:13.442989+0000","last_peered":"2023-09-13T03:11:13.442989+0000","last_clean":"2023-09-13T03:11:13.442989+0000","last_became_active":"2023-09-13T03:11:10.952302+0000","last_became_peered":"2023-09-13T03:11:10.952302+0000","last_unstale":"2023-09-13T03:11:13.442989+0000","last_undegraded":"2023-09-13T03:11:13.442989+0000","last_fullsized":"2023-09-13T03:11:13.442989+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:13:46.972293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:15.210597+0000","last_change":"2023-09-13T03:11:09.959509+0000","last_active":"2023-09-13T03:11:15.210597+0000","last_peered":"2023-09-13T03:11:15.210597+0000","last_clean":"2023-09-13T03:11:15.210597+0000","last_became_active":"2023-09-13T03:11:09.959150+0000","last_became_peered":"2023-09-13T03:11:09.959150+0000","last_unstale":"2023-09-13T03:11:15.210597+0000","last_undegraded":"2023-09-13T03:11:15.210597+0000","last_fullsized":"2023-09-13T03:11:15.210597+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:33:30.648283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.143398+0000","last_change":"2023-09-13T03:11:10.956610+0000","last_active":"2023-09-13T03:11:12.143398+0000","last_peered":"2023-09-13T03:11:12.143398+0000","last_clean":"2023-09-13T03:11:12.143398+0000","last_became_active":"2023-09-13T03:11:10.955726+0000","last_became_peered":"2023-09-13T03:11:10.955726+0000","last_unstale":"2023-09-13T03:11:12.143398+0000","last_undegraded":"2023-09-13T03:11:12.143398+0000","last_fullsized":"2023-09-13T03:11:12.143398+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:55:45.667752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-13T03:11:10.971586+0000","last_change":"2023-09-13T03:11:09.949951+0000","last_active":"2023-09-13T03:11:10.971586+0000","last_peered":"2023-09-13T03:11:10.971586+0000","last_clean":"2023-09-13T03:11:10.971586+0000","last_became_active":"2023-09-13T03:11:09.949106+0000","last_became_peered":"2023-09-13T03:11:09.949106+0000","last_unstale":"2023-09-13T03:11:10.971586+0000","last_undegraded":"2023-09-13T03:11:10.971586+0000","last_fullsized":"2023-09-13T03:11:10.971586+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:30:30.092007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643320+0000","last_change":"2023-09-13T03:11:10.942255+0000","last_active":"2023-09-13T03:11:12.643320+0000","last_peered":"2023-09-13T03:11:12.643320+0000","last_clean":"2023-09-13T03:11:12.643320+0000","last_became_active":"2023-09-13T03:11:10.941953+0000","last_became_peered":"2023-09-13T03:11:10.941953+0000","last_unstale":"2023-09-13T03:11:12.643320+0000","last_undegraded":"2023-09-13T03:11:12.643320+0000","last_fullsized":"2023-09-13T03:11:12.643320+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:38:55.260227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:11.937747+0000","last_change":"2023-09-13T03:11:09.956042+0000","last_active":"2023-09-13T03:11:11.937747+0000","last_peered":"2023-09-13T03:11:11.937747+0000","last_clean":"2023-09-13T03:11:11.937747+0000","last_became_active":"2023-09-13T03:11:09.955669+0000","last_became_peered":"2023-09-13T03:11:09.955669+0000","last_unstale":"2023-09-13T03:11:11.937747+0000","last_undegraded":"2023-09-13T03:11:11.937747+0000","last_fullsized":"2023-09-13T03:11:11.937747+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:16:25.227447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643490+0000","last_change":"2023-09-13T03:11:10.952421+0000","last_active":"2023-09-13T03:11:12.643490+0000","last_peered":"2023-09-13T03:11:12.643490+0000","last_clean":"2023-09-13T03:11:12.643490+0000","last_became_active":"2023-09-13T03:11:10.951664+0000","last_became_peered":"2023-09-13T03:11:10.951664+0000","last_unstale":"2023-09-13T03:11:12.643490+0000","last_undegraded":"2023-09-13T03:11:12.643490+0000","last_fullsized":"2023-09-13T03:11:12.643490+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:46:52.664524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'5","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:18.243989+0000","last_change":"2023-09-13T03:11:09.952286+0000","last_active":"2023-09-13T03:11:18.243989+0000","last_peered":"2023-09-13T03:11:18.243989+0000","last_clean":"2023-09-13T03:11:18.243989+0000","last_became_active":"2023-09-13T03:11:09.951999+0000","last_became_peered":"2023-09-13T03:11:09.951999+0000","last_unstale":"2023-09-13T03:11:18.243989+0000","last_undegraded":"2023-09-13T03:11:18.243989+0000","last_fullsized":"2023-09-13T03:11:18.243989+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:54:46.910087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643290+0000","last_change":"2023-09-13T03:11:10.950866+0000","last_active":"2023-09-13T03:11:12.643290+0000","last_peered":"2023-09-13T03:11:12.643290+0000","last_clean":"2023-09-13T03:11:12.643290+0000","last_became_active":"2023-09-13T03:11:10.950562+0000","last_became_peered":"2023-09-13T03:11:10.950562+0000","last_unstale":"2023-09-13T03:11:12.643290+0000","last_undegraded":"2023-09-13T03:11:12.643290+0000","last_fullsized":"2023-09-13T03:11:12.643290+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:55:07.333207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.143503+0000","last_change":"2023-09-13T03:11:10.950736+0000","last_active":"2023-09-13T03:11:12.143503+0000","last_peered":"2023-09-13T03:11:12.143503+0000","last_clean":"2023-09-13T03:11:12.143503+0000","last_became_active":"2023-09-13T03:11:10.950164+0000","last_became_peered":"2023-09-13T03:11:10.950164+0000","last_unstale":"2023-09-13T03:11:12.143503+0000","last_undegraded":"2023-09-13T03:11:12.143503+0000","last_fullsized":"2023-09-13T03:11:12.143503+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:22:44.453642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:18.238085+0000","last_change":"2023-09-13T03:11:09.954070+0000","last_active":"2023-09-13T03:11:18.238085+0000","last_peered":"2023-09-13T03:11:18.238085+0000","last_clean":"2023-09-13T03:11:18.238085+0000","last_became_active":"2023-09-13T03:11:09.953735+0000","last_became_peered":"2023-09-13T03:11:09.953735+0000","last_unstale":"2023-09-13T03:11:18.238085+0000","last_undegraded":"2023-09-13T03:11:18.238085+0000","last_fullsized":"2023-09-13T03:11:18.238085+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:36:59.891449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643652+0000","last_change":"2023-09-13T03:11:10.948509+0000","last_active":"2023-09-13T03:11:12.643652+0000","last_peered":"2023-09-13T03:11:12.643652+0000","last_clean":"2023-09-13T03:11:12.643652+0000","last_became_active":"2023-09-13T03:11:10.948149+0000","last_became_peered":"2023-09-13T03:11:10.948149+0000","last_unstale":"2023-09-13T03:11:12.643652+0000","last_undegraded":"2023-09-13T03:11:12.643652+0000","last_fullsized":"2023-09-13T03:11:12.643652+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:45:54.197368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:16.218695+0000","last_change":"2023-09-13T03:11:09.954476+0000","last_active":"2023-09-13T03:11:16.218695+0000","last_peered":"2023-09-13T03:11:16.218695+0000","last_clean":"2023-09-13T03:11:16.218695+0000","last_became_active":"2023-09-13T03:11:09.954094+0000","last_became_peered":"2023-09-13T03:11:09.954094+0000","last_unstale":"2023-09-13T03:11:16.218695+0000","last_undegraded":"2023-09-13T03:11:16.218695+0000","last_fullsized":"2023-09-13T03:11:16.218695+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:08:23.815751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":318,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-13T03:11:10.943859+0000","last_change":"2023-09-13T03:10:05.689061+0000","last_active":"2023-09-13T03:11:10.943859+0000","last_peered":"2023-09-13T03:11:10.943859+0000","last_clean":"2023-09-13T03:11:10.943859+0000","last_became_active":"2023-09-13T03:10:05.688557+0000","last_became_peered":"2023-09-13T03:10:05.688557+0000","last_unstale":"2023-09-13T03:11:10.943859+0000","last_undegraded":"2023-09-13T03:11:10.943859+0000","last_fullsized":"2023-09-13T03:11:10.943859+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:06:59.216900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:06:59.216900+0000","last_clean_scrub_stamp":"2023-09-13T03:06:59.216900+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:10:03.279019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643866+0000","last_change":"2023-09-13T03:11:10.952523+0000","last_active":"2023-09-13T03:11:12.643866+0000","last_peered":"2023-09-13T03:11:12.643866+0000","last_clean":"2023-09-13T03:11:12.643866+0000","last_became_active":"2023-09-13T03:11:10.952193+0000","last_became_peered":"2023-09-13T03:11:10.952193+0000","last_unstale":"2023-09-13T03:11:12.643866+0000","last_undegraded":"2023-09-13T03:11:12.643866+0000","last_fullsized":"2023-09-13T03:11:12.643866+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:31:01.811841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:15.210584+0000","last_change":"2023-09-13T03:11:09.950414+0000","last_active":"2023-09-13T03:11:15.210584+0000","last_peered":"2023-09-13T03:11:15.210584+0000","last_clean":"2023-09-13T03:11:15.210584+0000","last_became_active":"2023-09-13T03:11:09.950057+0000","last_became_peered":"2023-09-13T03:11:09.950057+0000","last_unstale":"2023-09-13T03:11:15.210584+0000","last_undegraded":"2023-09-13T03:11:15.210584+0000","last_fullsized":"2023-09-13T03:11:15.210584+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:57:01.317880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643345+0000","last_change":"2023-09-13T03:11:09.955535+0000","last_active":"2023-09-13T03:11:12.643345+0000","last_peered":"2023-09-13T03:11:12.643345+0000","last_clean":"2023-09-13T03:11:12.643345+0000","last_became_active":"2023-09-13T03:11:09.955212+0000","last_became_peered":"2023-09-13T03:11:09.955212+0000","last_unstale":"2023-09-13T03:11:12.643345+0000","last_undegraded":"2023-09-13T03:11:12.643345+0000","last_fullsized":"2023-09-13T03:11:12.643345+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:15:04.365337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.644116+0000","last_change":"2023-09-13T03:11:10.954255+0000","last_active":"2023-09-13T03:11:12.644116+0000","last_peered":"2023-09-13T03:11:12.644116+0000","last_clean":"2023-09-13T03:11:12.644116+0000","last_became_active":"2023-09-13T03:11:10.953866+0000","last_became_peered":"2023-09-13T03:11:10.953866+0000","last_unstale":"2023-09-13T03:11:12.644116+0000","last_undegraded":"2023-09-13T03:11:12.644116+0000","last_fullsized":"2023-09-13T03:11:12.644116+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:25:12.281218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'7","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:16.220083+0000","last_change":"2023-09-13T03:11:09.956402+0000","last_active":"2023-09-13T03:11:16.220083+0000","last_peered":"2023-09-13T03:11:16.220083+0000","last_clean":"2023-09-13T03:11:16.220083+0000","last_became_active":"2023-09-13T03:11:09.954710+0000","last_became_peered":"2023-09-13T03:11:09.954710+0000","last_unstale":"2023-09-13T03:11:16.220083+0000","last_undegraded":"2023-09-13T03:11:16.220083+0000","last_fullsized":"2023-09-13T03:11:16.220083+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:07:10.256221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643490+0000","last_change":"2023-09-13T03:11:10.949037+0000","last_active":"2023-09-13T03:11:12.643490+0000","last_peered":"2023-09-13T03:11:12.643490+0000","last_clean":"2023-09-13T03:11:12.643490+0000","last_became_active":"2023-09-13T03:11:10.948641+0000","last_became_peered":"2023-09-13T03:11:10.948641+0000","last_unstale":"2023-09-13T03:11:12.643490+0000","last_undegraded":"2023-09-13T03:11:12.643490+0000","last_fullsized":"2023-09-13T03:11:12.643490+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:00:57.563300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:17.234038+0000","last_change":"2023-09-13T03:11:09.946435+0000","last_active":"2023-09-13T03:11:17.234038+0000","last_peered":"2023-09-13T03:11:17.234038+0000","last_clean":"2023-09-13T03:11:17.234038+0000","last_became_active":"2023-09-13T03:11:09.946176+0000","last_became_peered":"2023-09-13T03:11:09.946176+0000","last_unstale":"2023-09-13T03:11:17.234038+0000","last_undegraded":"2023-09-13T03:11:17.234038+0000","last_fullsized":"2023-09-13T03:11:17.234038+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:34:16.055188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643287+0000","last_change":"2023-09-13T03:11:10.957035+0000","last_active":"2023-09-13T03:11:12.643287+0000","last_peered":"2023-09-13T03:11:12.643287+0000","last_clean":"2023-09-13T03:11:12.643287+0000","last_became_active":"2023-09-13T03:11:10.954656+0000","last_became_peered":"2023-09-13T03:11:10.954656+0000","last_unstale":"2023-09-13T03:11:12.643287+0000","last_undegraded":"2023-09-13T03:11:12.643287+0000","last_fullsized":"2023-09-13T03:11:12.643287+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:54:51.142734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:14.199239+0000","last_change":"2023-09-13T03:11:09.959358+0000","last_active":"2023-09-13T03:11:14.199239+0000","last_peered":"2023-09-13T03:11:14.199239+0000","last_clean":"2023-09-13T03:11:14.199239+0000","last_became_active":"2023-09-13T03:11:09.959129+0000","last_became_peered":"2023-09-13T03:11:09.959129+0000","last_unstale":"2023-09-13T03:11:14.199239+0000","last_undegraded":"2023-09-13T03:11:14.199239+0000","last_fullsized":"2023-09-13T03:11:14.199239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:29:26.930910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643075+0000","last_change":"2023-09-13T03:11:10.952588+0000","last_active":"2023-09-13T03:11:12.643075+0000","last_peered":"2023-09-13T03:11:12.643075+0000","last_clean":"2023-09-13T03:11:12.643075+0000","last_became_active":"2023-09-13T03:11:10.952304+0000","last_became_peered":"2023-09-13T03:11:10.952304+0000","last_unstale":"2023-09-13T03:11:12.643075+0000","last_undegraded":"2023-09-13T03:11:12.643075+0000","last_fullsized":"2023-09-13T03:11:12.643075+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:05:49.824053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:17.232352+0000","last_change":"2023-09-13T03:11:09.956947+0000","last_active":"2023-09-13T03:11:17.232352+0000","last_peered":"2023-09-13T03:11:17.232352+0000","last_clean":"2023-09-13T03:11:17.232352+0000","last_became_active":"2023-09-13T03:11:09.956639+0000","last_became_peered":"2023-09-13T03:11:09.956639+0000","last_unstale":"2023-09-13T03:11:17.232352+0000","last_undegraded":"2023-09-13T03:11:17.232352+0000","last_fullsized":"2023-09-13T03:11:17.232352+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:21:37.759130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.642869+0000","last_change":"2023-09-13T03:11:10.947314+0000","last_active":"2023-09-13T03:11:12.642869+0000","last_peered":"2023-09-13T03:11:12.642869+0000","last_clean":"2023-09-13T03:11:12.642869+0000","last_became_active":"2023-09-13T03:11:10.946964+0000","last_became_peered":"2023-09-13T03:11:10.946964+0000","last_unstale":"2023-09-13T03:11:12.642869+0000","last_undegraded":"2023-09-13T03:11:12.642869+0000","last_fullsized":"2023-09-13T03:11:12.642869+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:55:08.542303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:15.329040+0000","last_change":"2023-09-13T03:11:09.952540+0000","last_active":"2023-09-13T03:11:15.329040+0000","last_peered":"2023-09-13T03:11:15.329040+0000","last_clean":"2023-09-13T03:11:15.329040+0000","last_became_active":"2023-09-13T03:11:09.952208+0000","last_became_peered":"2023-09-13T03:11:09.952208+0000","last_unstale":"2023-09-13T03:11:15.329040+0000","last_undegraded":"2023-09-13T03:11:15.329040+0000","last_fullsized":"2023-09-13T03:11:15.329040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:11:41.200401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643016+0000","last_change":"2023-09-13T03:11:10.951305+0000","last_active":"2023-09-13T03:11:12.643016+0000","last_peered":"2023-09-13T03:11:12.643016+0000","last_clean":"2023-09-13T03:11:12.643016+0000","last_became_active":"2023-09-13T03:11:10.950982+0000","last_became_peered":"2023-09-13T03:11:10.950982+0000","last_unstale":"2023-09-13T03:11:12.643016+0000","last_undegraded":"2023-09-13T03:11:12.643016+0000","last_fullsized":"2023-09-13T03:11:12.643016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:55:04.497698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:14.199721+0000","last_change":"2023-09-13T03:11:09.955410+0000","last_active":"2023-09-13T03:11:14.199721+0000","last_peered":"2023-09-13T03:11:14.199721+0000","last_clean":"2023-09-13T03:11:14.199721+0000","last_became_active":"2023-09-13T03:11:09.955107+0000","last_became_peered":"2023-09-13T03:11:09.955107+0000","last_unstale":"2023-09-13T03:11:14.199721+0000","last_undegraded":"2023-09-13T03:11:14.199721+0000","last_fullsized":"2023-09-13T03:11:14.199721+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:34:05.108897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:13.443411+0000","last_change":"2023-09-13T03:11:10.959487+0000","last_active":"2023-09-13T03:11:13.443411+0000","last_peered":"2023-09-13T03:11:13.443411+0000","last_clean":"2023-09-13T03:11:13.443411+0000","last_became_active":"2023-09-13T03:11:10.959157+0000","last_became_peered":"2023-09-13T03:11:10.959157+0000","last_unstale":"2023-09-13T03:11:13.443411+0000","last_undegraded":"2023-09-13T03:11:13.443411+0000","last_fullsized":"2023-09-13T03:11:13.443411+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:37:50.852197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'5","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:18.243289+0000","last_change":"2023-09-13T03:11:09.951208+0000","last_active":"2023-09-13T03:11:18.243289+0000","last_peered":"2023-09-13T03:11:18.243289+0000","last_clean":"2023-09-13T03:11:18.243289+0000","last_became_active":"2023-09-13T03:11:09.950869+0000","last_became_peered":"2023-09-13T03:11:09.950869+0000","last_unstale":"2023-09-13T03:11:18.243289+0000","last_undegraded":"2023-09-13T03:11:18.243289+0000","last_fullsized":"2023-09-13T03:11:18.243289+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:22:53.697803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:13.443650+0000","last_change":"2023-09-13T03:11:10.959129+0000","last_active":"2023-09-13T03:11:13.443650+0000","last_peered":"2023-09-13T03:11:13.443650+0000","last_clean":"2023-09-13T03:11:13.443650+0000","last_became_active":"2023-09-13T03:11:10.958820+0000","last_became_peered":"2023-09-13T03:11:10.958820+0000","last_unstale":"2023-09-13T03:11:13.443650+0000","last_undegraded":"2023-09-13T03:11:13.443650+0000","last_fullsized":"2023-09-13T03:11:13.443650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:20:30.607854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:16.219213+0000","last_change":"2023-09-13T03:11:09.946855+0000","last_active":"2023-09-13T03:11:16.219213+0000","last_peered":"2023-09-13T03:11:16.219213+0000","last_clean":"2023-09-13T03:11:16.219213+0000","last_became_active":"2023-09-13T03:11:09.946156+0000","last_became_peered":"2023-09-13T03:11:09.946156+0000","last_unstale":"2023-09-13T03:11:16.219213+0000","last_undegraded":"2023-09-13T03:11:16.219213+0000","last_fullsized":"2023-09-13T03:11:16.219213+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:42:19.387280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:14.514465+0000","last_change":"2023-09-13T03:11:10.951335+0000","last_active":"2023-09-13T03:11:14.514465+0000","last_peered":"2023-09-13T03:11:14.514465+0000","last_clean":"2023-09-13T03:11:14.514465+0000","last_became_active":"2023-09-13T03:11:10.951034+0000","last_became_peered":"2023-09-13T03:11:10.951034+0000","last_unstale":"2023-09-13T03:11:14.514465+0000","last_undegraded":"2023-09-13T03:11:14.514465+0000","last_fullsized":"2023-09-13T03:11:14.514465+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:57:24.373196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:17.232198+0000","last_change":"2023-09-13T03:11:09.950937+0000","last_active":"2023-09-13T03:11:17.232198+0000","last_peered":"2023-09-13T03:11:17.232198+0000","last_clean":"2023-09-13T03:11:17.232198+0000","last_became_active":"2023-09-13T03:11:09.950574+0000","last_became_peered":"2023-09-13T03:11:09.950574+0000","last_unstale":"2023-09-13T03:11:17.232198+0000","last_undegraded":"2023-09-13T03:11:17.232198+0000","last_fullsized":"2023-09-13T03:11:17.232198+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:10:23.215080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:13.443547+0000","last_change":"2023-09-13T03:11:10.950517+0000","last_active":"2023-09-13T03:11:13.443547+0000","last_peered":"2023-09-13T03:11:13.443547+0000","last_clean":"2023-09-13T03:11:13.443547+0000","last_became_active":"2023-09-13T03:11:10.950028+0000","last_became_peered":"2023-09-13T03:11:10.950028+0000","last_unstale":"2023-09-13T03:11:13.443547+0000","last_undegraded":"2023-09-13T03:11:13.443547+0000","last_fullsized":"2023-09-13T03:11:13.443547+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:06:40.008527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643049+0000","last_change":"2023-09-13T03:11:10.942192+0000","last_active":"2023-09-13T03:11:12.643049+0000","last_peered":"2023-09-13T03:11:12.643049+0000","last_clean":"2023-09-13T03:11:12.643049+0000","last_became_active":"2023-09-13T03:11:10.941893+0000","last_became_peered":"2023-09-13T03:11:10.941893+0000","last_unstale":"2023-09-13T03:11:12.643049+0000","last_undegraded":"2023-09-13T03:11:12.643049+0000","last_fullsized":"2023-09-13T03:11:12.643049+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:26:22.064843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:11.937442+0000","last_change":"2023-09-13T03:11:10.947186+0000","last_active":"2023-09-13T03:11:11.937442+0000","last_peered":"2023-09-13T03:11:11.937442+0000","last_clean":"2023-09-13T03:11:11.937442+0000","last_became_active":"2023-09-13T03:11:10.946902+0000","last_became_peered":"2023-09-13T03:11:10.946902+0000","last_unstale":"2023-09-13T03:11:11.937442+0000","last_undegraded":"2023-09-13T03:11:11.937442+0000","last_fullsized":"2023-09-13T03:11:11.937442+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:26:01.325597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:11.937308+0000","last_change":"2023-09-13T03:11:10.946504+0000","last_active":"2023-09-13T03:11:11.937308+0000","last_peered":"2023-09-13T03:11:11.937308+0000","last_clean":"2023-09-13T03:11:11.937308+0000","last_became_active":"2023-09-13T03:11:10.946196+0000","last_became_peered":"2023-09-13T03:11:10.946196+0000","last_unstale":"2023-09-13T03:11:11.937308+0000","last_undegraded":"2023-09-13T03:11:11.937308+0000","last_fullsized":"2023-09-13T03:11:11.937308+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:37:43.578685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643668+0000","last_change":"2023-09-13T03:11:10.959637+0000","last_active":"2023-09-13T03:11:12.643668+0000","last_peered":"2023-09-13T03:11:12.643668+0000","last_clean":"2023-09-13T03:11:12.643668+0000","last_became_active":"2023-09-13T03:11:10.959324+0000","last_became_peered":"2023-09-13T03:11:10.959324+0000","last_unstale":"2023-09-13T03:11:12.643668+0000","last_undegraded":"2023-09-13T03:11:12.643668+0000","last_fullsized":"2023-09-13T03:11:12.643668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:48:19.245785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.644275+0000","last_change":"2023-09-13T03:11:10.948678+0000","last_active":"2023-09-13T03:11:12.644275+0000","last_peered":"2023-09-13T03:11:12.644275+0000","last_clean":"2023-09-13T03:11:12.644275+0000","last_became_active":"2023-09-13T03:11:10.948325+0000","last_became_peered":"2023-09-13T03:11:10.948325+0000","last_unstale":"2023-09-13T03:11:12.644275+0000","last_undegraded":"2023-09-13T03:11:12.644275+0000","last_fullsized":"2023-09-13T03:11:12.644275+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:59:36.340698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643176+0000","last_change":"2023-09-13T03:11:10.945076+0000","last_active":"2023-09-13T03:11:12.643176+0000","last_peered":"2023-09-13T03:11:12.643176+0000","last_clean":"2023-09-13T03:11:12.643176+0000","last_became_active":"2023-09-13T03:11:10.944776+0000","last_became_peered":"2023-09-13T03:11:10.944776+0000","last_unstale":"2023-09-13T03:11:12.643176+0000","last_undegraded":"2023-09-13T03:11:12.643176+0000","last_fullsized":"2023-09-13T03:11:12.643176+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:22:03.624488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:11.937646+0000","last_change":"2023-09-13T03:11:10.942663+0000","last_active":"2023-09-13T03:11:11.937646+0000","last_peered":"2023-09-13T03:11:11.937646+0000","last_clean":"2023-09-13T03:11:11.937646+0000","last_became_active":"2023-09-13T03:11:10.942203+0000","last_became_peered":"2023-09-13T03:11:10.942203+0000","last_unstale":"2023-09-13T03:11:11.937646+0000","last_undegraded":"2023-09-13T03:11:11.937646+0000","last_fullsized":"2023-09-13T03:11:11.937646+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:46:48.091425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643342+0000","last_change":"2023-09-13T03:11:10.940089+0000","last_active":"2023-09-13T03:11:12.643342+0000","last_peered":"2023-09-13T03:11:12.643342+0000","last_clean":"2023-09-13T03:11:12.643342+0000","last_became_active":"2023-09-13T03:11:10.939622+0000","last_became_peered":"2023-09-13T03:11:10.939622+0000","last_unstale":"2023-09-13T03:11:12.643342+0000","last_undegraded":"2023-09-13T03:11:12.643342+0000","last_fullsized":"2023-09-13T03:11:12.643342+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:43:55.024944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.143263+0000","last_change":"2023-09-13T03:11:10.956896+0000","last_active":"2023-09-13T03:11:12.143263+0000","last_peered":"2023-09-13T03:11:12.143263+0000","last_clean":"2023-09-13T03:11:12.143263+0000","last_became_active":"2023-09-13T03:11:10.956624+0000","last_became_peered":"2023-09-13T03:11:10.956624+0000","last_unstale":"2023-09-13T03:11:12.143263+0000","last_undegraded":"2023-09-13T03:11:12.143263+0000","last_fullsized":"2023-09-13T03:11:12.143263+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:29:41.305999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643108+0000","last_change":"2023-09-13T03:11:10.939240+0000","last_active":"2023-09-13T03:11:12.643108+0000","last_peered":"2023-09-13T03:11:12.643108+0000","last_clean":"2023-09-13T03:11:12.643108+0000","last_became_active":"2023-09-13T03:11:10.938869+0000","last_became_peered":"2023-09-13T03:11:10.938869+0000","last_unstale":"2023-09-13T03:11:12.643108+0000","last_undegraded":"2023-09-13T03:11:12.643108+0000","last_fullsized":"2023-09-13T03:11:12.643108+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:06:44.931165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643340+0000","last_change":"2023-09-13T03:11:10.956704+0000","last_active":"2023-09-13T03:11:12.643340+0000","last_peered":"2023-09-13T03:11:12.643340+0000","last_clean":"2023-09-13T03:11:12.643340+0000","last_became_active":"2023-09-13T03:11:10.956413+0000","last_became_peered":"2023-09-13T03:11:10.956413+0000","last_unstale":"2023-09-13T03:11:12.643340+0000","last_undegraded":"2023-09-13T03:11:12.643340+0000","last_fullsized":"2023-09-13T03:11:12.643340+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:50:04.626111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.644102+0000","last_change":"2023-09-13T03:11:10.952707+0000","last_active":"2023-09-13T03:11:12.644102+0000","last_peered":"2023-09-13T03:11:12.644102+0000","last_clean":"2023-09-13T03:11:12.644102+0000","last_became_active":"2023-09-13T03:11:10.952423+0000","last_became_peered":"2023-09-13T03:11:10.952423+0000","last_unstale":"2023-09-13T03:11:12.644102+0000","last_undegraded":"2023-09-13T03:11:12.644102+0000","last_fullsized":"2023-09-13T03:11:12.644102+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:35:09.869172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-13T03:11:12.643041+0000","last_change":"2023-09-13T03:11:10.941096+0000","last_active":"2023-09-13T03:11:12.643041+0000","last_peered":"2023-09-13T03:11:12.643041+0000","last_clean":"2023-09-13T03:11:12.643041+0000","last_became_active":"2023-09-13T03:11:10.940761+0000","last_became_peered":"2023-09-13T03:11:10.940761+0000","last_unstale":"2023-09-13T03:11:12.643041+0000","last_undegraded":"2023-09-13T03:11:12.643041+0000","last_fullsized":"2023-09-13T03:11:12.643041+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:34:27.389650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":8442,"num_objects":86,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":10,"num_read_kb":7,"num_write":186,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":57,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":69700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":162,"ondisk_log_size":162,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449101,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":427010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]}]},{"osd":10,"up_from":71,"seq":304942678032,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28516,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716636,"statfs":{"total":95995035648,"available":95965835264,"internally_reserved":0,"allocated":1536000,"data_stored":1019500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":1,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.44},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.47099999999999997}]},{"osd":2,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.69599999999999995}]},{"osd":3,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.55900000000000005}]},{"osd":4,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.39000000000000001}]},{"osd":5,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.64100000000000001}]},{"osd":6,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":0.85699999999999998}]},{"osd":7,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.47599999999999998}]},{"osd":8,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.63}]},{"osd":9,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.68400000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]}]},{"osd":9,"up_from":66,"seq":283467841557,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":426726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.376}]},{"osd":1,"last update":"Wed Sep 13 03:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.80000000000000004}]},{"osd":2,"last update":"Wed Sep 13 03:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.53800000000000003}]},{"osd":3,"last update":"Wed Sep 13 03:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.55900000000000005}]},{"osd":4,"last update":"Wed Sep 13 03:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.69599999999999995}]},{"osd":5,"last update":"Wed Sep 13 03:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":0.622},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":4.2590000000000003,"5min":4.2590000000000003,"15min":4.2590000000000003},"last":0.77000000000000002}]},{"osd":6,"last update":"Wed Sep 13 03:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.59099999999999997}]},{"osd":7,"last update":"Wed Sep 13 03:10:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":4.367,"5min":4.367,"15min":4.367},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":4.3520000000000003,"5min":4.3520000000000003,"15min":4.3520000000000003},"last":0.92500000000000004}]},{"osd":8,"last update":"Wed Sep 13 03:10:40 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":4.298,"5min":4.298,"15min":4.298},"last":0.746}]},{"osd":10,"last update":"Wed Sep 13 03:11:05 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.81999999999999995}]}]},{"osd":8,"up_from":58,"seq":249108103193,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":426225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:10:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.71499999999999997}]},{"osd":1,"last update":"Wed Sep 13 03:10:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.57999999999999996}]},{"osd":2,"last update":"Wed Sep 13 03:10:16 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.745},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.65400000000000003}]},{"osd":3,"last update":"Wed Sep 13 03:10:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.80400000000000005}]},{"osd":4,"last update":"Wed Sep 13 03:10:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.626},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.879}]},{"osd":5,"last update":"Wed Sep 13 03:10:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.83999999999999997}]},{"osd":6,"last update":"Wed Sep 13 03:10:16 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.86899999999999999}]},{"osd":7,"last update":"Wed Sep 13 03:10:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.82799999999999996}]},{"osd":9,"last update":"Wed Sep 13 03:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.92200000000000004}]},{"osd":10,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":2.004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]}]},{"osd":7,"up_from":51,"seq":219043332126,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1511424,"data_stored":1019835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:10:55 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.59699999999999998,"15min":0.58599999999999997},"min":{"1min":0.498,"5min":0.438,"15min":0.438},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.378},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60099999999999998,"15min":0.59099999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.41099999999999998}]},{"osd":1,"last update":"Wed Sep 13 03:10:55 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.41399999999999998,"15min":0.40200000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.42099999999999999,"15min":0.41199999999999998},"min":{"1min":0.32000000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.50600000000000001}]},{"osd":2,"last update":"Wed Sep 13 03:10:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.59799999999999998,"15min":0.58999999999999997},"min":{"1min":0.46300000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.57999999999999996,"15min":0.57499999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.58599999999999997}]},{"osd":3,"last update":"Wed Sep 13 03:10:55 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.68000000000000005,"15min":0.628},"min":{"1min":0.496,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":6.2720000000000002,"5min":6.2720000000000002,"15min":6.2720000000000002},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.63600000000000001,"15min":0.622},"min":{"1min":0.54600000000000004,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.66200000000000003}]},{"osd":4,"last update":"Wed Sep 13 03:10:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.52400000000000002,"15min":0.51200000000000001},"min":{"1min":0.35199999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.623},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.54000000000000004,"15min":0.52200000000000002},"min":{"1min":0.36099999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.67700000000000005}]},{"osd":5,"last update":"Wed Sep 13 03:10:55 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.64500000000000002,"15min":0.63700000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.627,"15min":0.61599999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.70099999999999996}]},{"osd":6,"last update":"Wed Sep 13 03:10:55 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.71499999999999997,"15min":0.69699999999999995},"min":{"1min":0.63900000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.73999999999999999,"15min":0.72199999999999998},"min":{"1min":0.67200000000000004,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.85799999999999998}]},{"osd":8,"last update":"Wed Sep 13 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.75},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.76600000000000001}]},{"osd":9,"last update":"Wed Sep 13 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.80500000000000005}]},{"osd":10,"last update":"Wed Sep 13 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.73799999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]}]},{"osd":6,"up_from":46,"seq":197568495651,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":426540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:10:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.501,"15min":0.497},"min":{"1min":0.34200000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53800000000000003,"15min":0.53100000000000003},"min":{"1min":0.37,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.48799999999999999}]},{"osd":1,"last update":"Wed Sep 13 03:10:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.64100000000000001,"15min":0.63100000000000001},"min":{"1min":0.42799999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.63100000000000001,"15min":0.61699999999999999},"min":{"1min":0.47199999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.59699999999999998}]},{"osd":2,"last update":"Wed Sep 13 03:10:36 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.68200000000000005,"15min":0.67200000000000004},"min":{"1min":0.54600000000000004,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.72399999999999998,"15min":0.71799999999999997},"min":{"1min":0.48099999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.96199999999999997,"5min":1.149,"15min":1.149},"last":0.68700000000000006}]},{"osd":3,"last update":"Wed Sep 13 03:10:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.65500000000000003,"15min":0.64800000000000002},"min":{"1min":0.48299999999999998,"5min":0.374,"15min":0.374},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.626},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.66100000000000003,"15min":0.65500000000000003},"min":{"1min":0.5,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.56000000000000005}]},{"osd":4,"last update":"Wed Sep 13 03:10:36 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.73699999999999999,"15min":0.73099999999999998},"min":{"1min":0.51100000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.752},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.77200000000000002,"15min":0.76800000000000002},"min":{"1min":0.56699999999999995,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.66800000000000004}]},{"osd":5,"last update":"Wed Sep 13 03:10:36 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.58899999999999997,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.97499999999999998,"5min":1.115,"15min":1.115},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.79100000000000004,"15min":0.78400000000000003},"min":{"1min":0.625,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.64900000000000002}]},{"osd":7,"last update":"Wed Sep 13 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.77500000000000002,"15min":0.77000000000000002},"min":{"1min":0.53700000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.76400000000000001,"15min":0.751},"min":{"1min":0.57899999999999996,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.85799999999999998}]},{"osd":8,"last update":"Wed Sep 13 03:10:18 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.77400000000000002}]},{"osd":9,"last update":"Wed Sep 13 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.89200000000000002}]},{"osd":10,"last update":"Wed Sep 13 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.92600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574905,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":446274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.56699999999999995,"15min":0.53800000000000003},"min":{"1min":0.35499999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":2.9260000000000002,"5min":2.9260000000000002,"15min":2.9260000000000002},"last":0.435},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.54100000000000004,"15min":0.53100000000000003},"min":{"1min":0.38700000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.63800000000000001}]},{"osd":2,"last update":"Wed Sep 13 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.54400000000000004,"15min":0.52600000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.54900000000000004,"15min":0.52900000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.66700000000000004}]},{"osd":3,"last update":"Wed Sep 13 03:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59899999999999998,"15min":0.58499999999999996},"min":{"1min":0.41699999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84799999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.61899999999999999,"15min":0.60099999999999998},"min":{"1min":0.5,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.45100000000000001}]},{"osd":4,"last update":"Wed Sep 13 03:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.41199999999999998,"15min":0.38600000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.45000000000000001,"15min":0.42799999999999999},"min":{"1min":0.36099999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.49399999999999999}]},{"osd":5,"last update":"Wed Sep 13 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.57199999999999995,"15min":0.53400000000000003},"min":{"1min":0.44900000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.57799999999999996,"15min":0.54500000000000004},"min":{"1min":0.46500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.78300000000000003}]},{"osd":6,"last update":"Wed Sep 13 03:10:36 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.65600000000000003,"15min":0.65000000000000002},"min":{"1min":0.52700000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.69499999999999995,"15min":0.67200000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.9790000000000001,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":0.53700000000000003}]},{"osd":7,"last update":"Wed Sep 13 03:10:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.57099999999999995,"15min":0.56100000000000005},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.625,"5min":0.60299999999999998,"15min":0.59899999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.622}]},{"osd":8,"last update":"Wed Sep 13 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.79700000000000004}]},{"osd":9,"last update":"Wed Sep 13 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.65400000000000003}]},{"osd":10,"last update":"Wed Sep 13 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.72399999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738430,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":427296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 13 03:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.59299999999999997,"15min":0.55300000000000005},"min":{"1min":0.44700000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58299999999999996,"15min":0.55700000000000005},"min":{"1min":0.40799999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.84799999999999998,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.83699999999999997}]},{"osd":2,"last update":"Wed Sep 13 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.64800000000000002,"15min":0.60799999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.621,"15min":0.59099999999999997},"min":{"1min":0.5,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.86599999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.81100000000000005}]},{"osd":3,"last update":"Wed Sep 13 03:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.46800000000000003,"15min":0.45800000000000002},"min":{"1min":0.309,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.56200000000000006,"15min":0.54700000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.60499999999999998}]},{"osd":4,"last update":"Wed Sep 13 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69299999999999995,"15min":0.69899999999999995},"min":{"1min":0.56200000000000006,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0760000000000001,"5min":1.855,"15min":1.855},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.68899999999999995,"15min":0.68200000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.95199999999999996,"5min":1.665,"15min":1.665},"last":0.71599999999999997}]},{"osd":5,"last update":"Wed Sep 13 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.70299999999999996,"15min":0.69299999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.90300000000000002,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68600000000000005,"15min":0.67400000000000004},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.88400000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.95399999999999996}]},{"osd":6,"last update":"Wed Sep 13 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.68100000000000005,"15min":0.66900000000000004},"min":{"1min":0.40999999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.752,"15min":0.74199999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.68899999999999995}]},{"osd":7,"last update":"Wed Sep 13 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.74299999999999999,"15min":0.73899999999999999},"min":{"1min":0.56299999999999994,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.626},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.80600000000000005,"15min":0.79400000000000004},"min":{"1min":0.74399999999999999,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1379999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.77900000000000003}]},{"osd":8,"last update":"Wed Sep 13 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.94199999999999995}]},{"osd":9,"last update":"Wed Sep 13 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":1.0409999999999999}]},{"osd":10,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":1.0900000000000001},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.97499999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411381,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":447157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.50800000000000001,"15min":0.47599999999999998},"min":{"1min":0.35399999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.68000000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.50800000000000001,"15min":0.48299999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73099999999999998,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.83299999999999996}]},{"osd":1,"last update":"Wed Sep 13 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53100000000000003,"15min":0.51400000000000001},"min":{"1min":0.38400000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.69899999999999995,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.52900000000000003,"15min":0.52500000000000002},"min":{"1min":0.40400000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.58599999999999997}]},{"osd":3,"last update":"Wed Sep 13 03:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.58399999999999996,"15min":0.55600000000000005},"min":{"1min":0.375,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.57699999999999996,"15min":0.55800000000000005},"min":{"1min":0.42799999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.90400000000000003,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.77500000000000002}]},{"osd":4,"last update":"Wed Sep 13 03:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.56000000000000005,"15min":0.55200000000000005},"min":{"1min":0.375,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73999999999999999,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56899999999999995,"15min":0.55900000000000005},"min":{"1min":0.42599999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.752,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.48499999999999999}]},{"osd":5,"last update":"Wed Sep 13 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.442,"15min":0.42399999999999999},"min":{"1min":0.33300000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70499999999999996,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.46100000000000002,"15min":0.442},"min":{"1min":0.29699999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.55500000000000005}]},{"osd":6,"last update":"Wed Sep 13 03:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.626,"15min":0.61699999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.874,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.621,"15min":0.61099999999999999},"min":{"1min":0.50700000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.81799999999999995,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.87}]},{"osd":7,"last update":"Wed Sep 13 03:10:58 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.63700000000000001,"15min":0.63600000000000001},"min":{"1min":0.47499999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.84699999999999998,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.66100000000000003,"15min":0.65900000000000003},"min":{"1min":0.51700000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.83399999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.497}]},{"osd":8,"last update":"Wed Sep 13 03:10:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.68100000000000005}]},{"osd":9,"last update":"Wed Sep 13 03:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.73799999999999999}]},{"osd":10,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.71599999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]}]},{"osd":3,"up_from":25,"seq":107374182449,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":426371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.41399999999999998,"15min":0.39900000000000002},"min":{"1min":0.30299999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.443,"15min":0.42899999999999999},"min":{"1min":0.32800000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.54000000000000004}]},{"osd":1,"last update":"Wed Sep 13 03:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.60699999999999998,"15min":0.58799999999999997},"min":{"1min":0.45200000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.61699999999999999,"15min":0.59199999999999997},"min":{"1min":0.48399999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.69799999999999995}]},{"osd":2,"last update":"Wed Sep 13 03:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.64700000000000002,"15min":0.627},"min":{"1min":0.50800000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.66400000000000003,"15min":0.64000000000000001},"min":{"1min":0.48899999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.83399999999999996}]},{"osd":4,"last update":"Wed Sep 13 03:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.65600000000000003,"15min":0.627},"min":{"1min":0.51400000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.387,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.65000000000000002,"15min":0.61899999999999999},"min":{"1min":0.54000000000000004,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.749}]},{"osd":5,"last update":"Wed Sep 13 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.69499999999999995,"15min":0.68000000000000005},"min":{"1min":0.53500000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1,"5min":1.012,"15min":1.012},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.71999999999999997,"15min":0.70199999999999996},"min":{"1min":0.53500000000000003,"5min":0.498,"15min":0.498},"max":{"1min":0.95999999999999996,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":1.0069999999999999}]},{"osd":6,"last update":"Wed Sep 13 03:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.63,"15min":0.62},"min":{"1min":0.45400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":1.117},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.76000000000000001,"15min":0.746},"min":{"1min":0.60299999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.599,"5min":1.599,"15min":1.599},"last":0.93700000000000006}]},{"osd":7,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.86199999999999999,"15min":0.85399999999999998},"min":{"1min":0.63200000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.4890000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.83999999999999997,"15min":0.82999999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.512,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":1.0269999999999999}]},{"osd":8,"last update":"Wed Sep 13 03:10:17 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.82199999999999995}]},{"osd":9,"last update":"Wed Sep 13 03:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":1.135},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":0.96799999999999997}]},{"osd":10,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.708,"5min":1.708,"15min":1.708},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":1.087}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.175}]}]},{"osd":4,"up_from":33,"seq":141733920812,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":426484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.54500000000000004,"15min":0.51500000000000001},"min":{"1min":0.36699999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.55900000000000005,"15min":0.52900000000000003},"min":{"1min":0.35599999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.63800000000000001}]},{"osd":1,"last update":"Wed Sep 13 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.379,"15min":0.35599999999999998},"min":{"1min":0.27000000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.63300000000000001,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.374},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.39100000000000001,"15min":0.36699999999999999},"min":{"1min":0.27000000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.67800000000000005,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.40999999999999998}]},{"osd":2,"last update":"Wed Sep 13 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57199999999999995,"15min":0.56000000000000005},"min":{"1min":0.42399999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.44},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.55800000000000005,"15min":0.53000000000000003},"min":{"1min":0.377,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.90500000000000003}]},{"osd":3,"last update":"Wed Sep 13 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.61499999999999999,"15min":0.55800000000000005},"min":{"1min":0.40200000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.03,"5min":4.2919999999999998,"15min":4.2919999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.59299999999999997,"15min":0.56799999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.56799999999999995}]},{"osd":5,"last update":"Wed Sep 13 03:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59499999999999997,"15min":0.57599999999999996},"min":{"1min":0.42799999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.85199999999999998,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.57999999999999996,"15min":0.56000000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.81299999999999994,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.625}]},{"osd":6,"last update":"Wed Sep 13 03:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.70799999999999996,"15min":0.69999999999999996},"min":{"1min":0.47899999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.753,"5min":0.72199999999999998,"15min":0.71699999999999997},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.73599999999999999}]},{"osd":7,"last update":"Wed Sep 13 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.621,"15min":0.60799999999999998},"min":{"1min":0.38600000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.629,"15min":0.62},"min":{"1min":0.35999999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.69899999999999995}]},{"osd":8,"last update":"Wed Sep 13 03:10:18 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.71899999999999997}]},{"osd":9,"last update":"Wed Sep 13 03:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.874},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.88800000000000001}]},{"osd":10,"last update":"Wed Sep 13 03:11:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.77900000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]}]},{"osd":5,"up_from":38,"seq":163208757289,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28508,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716644,"statfs":{"total":95995035648,"available":95965843456,"internally_reserved":0,"allocated":1527808,"data_stored":1019396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.59699999999999998,"15min":0.61299999999999999},"min":{"1min":0.39000000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78000000000000003,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.65800000000000003,"15min":0.63600000000000001},"min":{"1min":0.375,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":6.0179999999999998,"5min":6.0179999999999998,"15min":6.0179999999999998},"last":0.58499999999999996}]},{"osd":1,"last update":"Wed Sep 13 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.54000000000000004,"15min":0.51800000000000002},"min":{"1min":0.372,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.53300000000000003,"15min":0.51000000000000001},"min":{"1min":0.45200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.83799999999999997}]},{"osd":2,"last update":"Wed Sep 13 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.40100000000000002,"15min":0.38400000000000001},"min":{"1min":0.29099999999999998,"5min":0.193,"15min":0.193},"max":{"1min":0.71299999999999997,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.434,"5min":0.40600000000000003,"15min":0.38700000000000001},"min":{"1min":0.27000000000000002,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.76800000000000002,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.44800000000000001}]},{"osd":3,"last update":"Wed Sep 13 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.40400000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.69099999999999995,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60099999999999998,"15min":0.58599999999999997},"min":{"1min":0.38300000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.85299999999999998,"5min":1.48,"15min":1.48},"last":0.60699999999999998}]},{"osd":4,"last update":"Wed Sep 13 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.59899999999999998,"15min":0.55700000000000005},"min":{"1min":0.40899999999999997,"5min":0.376,"15min":0.376},"max":{"1min":3.3740000000000001,"5min":3.3740000000000001,"15min":3.3740000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56699999999999995,"15min":0.54900000000000004},"min":{"1min":0.39600000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79400000000000004,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.77400000000000002}]},{"osd":6,"last update":"Wed Sep 13 03:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.40999999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.65500000000000003,"15min":0.65000000000000002},"min":{"1min":0.50600000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.73299999999999998}]},{"osd":7,"last update":"Wed Sep 13 03:10:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.84899999999999998,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.79100000000000004}]},{"osd":8,"last update":"Wed Sep 13 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.84299999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61599999999999999,"15min":0.61699999999999999},"min":{"1min":0.38800000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.91100000000000003,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.64000000000000001}]},{"osd":9,"last update":"Wed Sep 13 03:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":5.9829999999999997,"5min":5.9829999999999997,"15min":5.9829999999999997},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.624}]},{"osd":10,"last update":"Wed Sep 13 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.879}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":25342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":25952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-13T03:11:21.708 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-09-13T03:11:21.709 INFO:tasks.cephadm:Setting up client nodes... 2023-09-13T03:11:21.709 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-09-13T03:11:21.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:21 smithi142 ceph-mon[134082]: from='client.24790 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:11:21.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:21 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/526453532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:11:21.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:21 smithi142 ceph-mon[134082]: pgmap v282: 97 pgs: 97 active+clean; 585 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 753 B/s rd, 6.6 KiB/s wr, 20 op/s 2023-09-13T03:11:21.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:21 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/2422588256' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:11:22.469 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:11:22.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:22 smithi170 ceph-mon[134328]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T03:11:22.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:22 smithi064 ceph-mon[97675]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T03:11:22.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:22 smithi142 ceph-mon[134082]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T03:11:23.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:23 smithi064 ceph-mon[97675]: from='client.24815 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T03:11:23.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:23 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2023-09-13T03:11:23.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:23 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2023-09-13T03:11:23.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:23 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2023-09-13T03:11:23.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:23 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2023-09-13T03:11:23.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:23 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2023-09-13T03:11:23.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:23 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2023-09-13T03:11:23.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:23 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2023-09-13T03:11:23.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:23 smithi064 ceph-mon[97675]: pgmap v283: 97 pgs: 97 active+clean; 586 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 6.3 KiB/s wr, 23 op/s 2023-09-13T03:11:23.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:23 smithi170 ceph-mon[134328]: from='client.24815 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T03:11:23.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:23 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2023-09-13T03:11:23.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:23 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2023-09-13T03:11:23.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:23 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2023-09-13T03:11:23.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:23 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2023-09-13T03:11:23.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:23 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2023-09-13T03:11:23.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:23 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2023-09-13T03:11:23.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:23 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2023-09-13T03:11:23.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:23 smithi170 ceph-mon[134328]: pgmap v283: 97 pgs: 97 active+clean; 586 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 6.3 KiB/s wr, 23 op/s 2023-09-13T03:11:23.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:23 smithi142 ceph-mon[134082]: from='client.24815 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T03:11:23.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:23 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2023-09-13T03:11:23.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:23 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2023-09-13T03:11:23.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:23 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2023-09-13T03:11:23.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:23 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2023-09-13T03:11:23.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:23 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2023-09-13T03:11:23.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:23 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2023-09-13T03:11:23.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:23 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2023-09-13T03:11:23.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:23 smithi142 ceph-mon[134082]: pgmap v283: 97 pgs: 97 active+clean; 586 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 6.3 KiB/s wr, 23 op/s 2023-09-13T03:11:23.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:23.858+0000 7fa59d725700 1 -- 172.21.15.64:0/437057274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5981006c0 msgr2=0x7fa598100ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:23.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:23.858+0000 7fa59d725700 1 --2- 172.21.15.64:0/437057274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5981006c0 0x7fa598100ae0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa5880099c0 tx=0x7fa588009cd0 comp rx=0 tx=0).stop 2023-09-13T03:11:23.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:23.858+0000 7fa59d725700 1 -- 172.21.15.64:0/437057274 shutdown_connections 2023-09-13T03:11:23.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:23.858+0000 7fa59d725700 1 --2- 172.21.15.64:0/437057274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa598102260 0x7fa598108b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:23.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:23.858+0000 7fa59d725700 1 --2- 172.21.15.64:0/437057274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5981006c0 0x7fa598100ae0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:23.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:23.858+0000 7fa59d725700 1 --2- 172.21.15.64:0/437057274 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5981018c0 0x7fa598101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:23.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:23.858+0000 7fa59d725700 1 -- 172.21.15.64:0/437057274 >> 172.21.15.64:0/437057274 conn(0x7fa5980fbc80 msgr2=0x7fa5980fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:23.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:23.858+0000 7fa59d725700 1 -- 172.21.15.64:0/437057274 shutdown_connections 2023-09-13T03:11:23.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:23.858+0000 7fa59d725700 1 -- 172.21.15.64:0/437057274 wait complete. 2023-09-13T03:11:23.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:23.859+0000 7fa59d725700 1 Processor -- start 2023-09-13T03:11:23.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:23.859+0000 7fa59d725700 1 -- start start 2023-09-13T03:11:23.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:23.859+0000 7fa59d725700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5981006c0 0x7fa598196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:23.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:23.859+0000 7fa59d725700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5981018c0 0x7fa598196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:23.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:23.859+0000 7fa59d725700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa598102260 0x7fa59819d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:23.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:23.860+0000 7fa59d725700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59819ded0 con 0x7fa5981006c0 2023-09-13T03:11:23.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:23.860+0000 7fa59d725700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59819e010 con 0x7fa598102260 2023-09-13T03:11:23.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:23.860+0000 7fa59d725700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59819e1a0 con 0x7fa5981018c0 2023-09-13T03:11:23.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:23.860+0000 7fa596ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5981006c0 0x7fa598196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:23.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:23.860+0000 7fa5967fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5981018c0 0x7fa598196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:23.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:23.860+0000 7fa5967fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5981018c0 0x7fa598196880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:36072/0 (socket says 172.21.15.64:36072) 2023-09-13T03:11:23.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:23.860+0000 7fa5967fc700 1 -- 172.21.15.64:0/3350989984 learned_addr learned my addr 172.21.15.64:0/3350989984 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:23.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:23.860+0000 7fa596ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5981006c0 0x7fa598196340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51308/0 (socket says 172.21.15.64:51308) 2023-09-13T03:11:23.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:23.860+0000 7fa5977fe700 1 --2- 172.21.15.64:0/3350989984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa598102260 0x7fa59819d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:23.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:23.860+0000 7fa596ffd700 1 -- 172.21.15.64:0/3350989984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5981018c0 msgr2=0x7fa598196880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:23.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:23.860+0000 7fa596ffd700 1 --2- 172.21.15.64:0/3350989984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5981018c0 0x7fa598196880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:23.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:23.860+0000 7fa596ffd700 1 -- 172.21.15.64:0/3350989984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa598102260 msgr2=0x7fa59819d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:23.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:23.860+0000 7fa596ffd700 1 --2- 172.21.15.64:0/3350989984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa598102260 0x7fa59819d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:23.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:23.860+0000 7fa596ffd700 1 -- 172.21.15.64:0/3350989984 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5880095f0 con 0x7fa5981006c0 2023-09-13T03:11:23.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:23.860+0000 7fa5967fc700 1 --2- 172.21.15.64:0/3350989984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5981018c0 0x7fa598196880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:11:23.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:23.861+0000 7fa596ffd700 1 --2- 172.21.15.64:0/3350989984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5981006c0 0x7fa598196340 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa5880098e0 tx=0x7fa588005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:23.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:23.861+0000 7fa57ffff700 1 -- 172.21.15.64:0/3350989984 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58801d070 con 0x7fa5981006c0 2023-09-13T03:11:23.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:23.861+0000 7fa57ffff700 1 -- 172.21.15.64:0/3350989984 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa588003f90 con 0x7fa5981006c0 2023-09-13T03:11:23.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:23.861+0000 7fa59d725700 1 -- 172.21.15.64:0/3350989984 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa59819e420 con 0x7fa5981006c0 2023-09-13T03:11:23.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:23.861+0000 7fa57ffff700 1 -- 172.21.15.64:0/3350989984 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa588022ba0 con 0x7fa5981006c0 2023-09-13T03:11:23.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:23.861+0000 7fa59d725700 1 -- 172.21.15.64:0/3350989984 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa59819e8e0 con 0x7fa5981006c0 2023-09-13T03:11:23.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:23.862+0000 7fa59d725700 1 -- 172.21.15.64:0/3350989984 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa598107680 con 0x7fa5981006c0 2023-09-13T03:11:23.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:23.864+0000 7fa57ffff700 1 -- 172.21.15.64:0/3350989984 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fa588003ad0 con 0x7fa5981006c0 2023-09-13T03:11:23.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:23.866+0000 7fa57ffff700 1 --2- 172.21.15.64:0/3350989984 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fa5800a0750 0x7fa5800a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:23.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:23.866+0000 7fa57ffff700 1 -- 172.21.15.64:0/3350989984 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa5880cd030 con 0x7fa5981006c0 2023-09-13T03:11:23.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:23.866+0000 7fa5967fc700 1 --2- 172.21.15.64:0/3350989984 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fa5800a0750 0x7fa5800a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:23.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:23.867+0000 7fa5967fc700 1 --2- 172.21.15.64:0/3350989984 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fa5800a0750 0x7fa5800a2c10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa58c005950 tx=0x7fa58c00b500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:23.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:23.870+0000 7fa57ffff700 1 -- 172.21.15.64:0/3350989984 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fa588096ac0 con 0x7fa5981006c0 2023-09-13T03:11:24.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:24.162+0000 7fa59d725700 1 -- 172.21.15.64:0/3350989984 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fa59810bec0 con 0x7fa5981006c0 2023-09-13T03:11:24.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:24.165+0000 7fa57ffff700 1 -- 172.21.15.64:0/3350989984 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7fa58809a0e0 con 0x7fa5981006c0 2023-09-13T03:11:24.166 INFO:teuthology.orchestra.run.smithi064.stdout:[client.0] 2023-09-13T03:11:24.166 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQBcKAFlIEPLCRAAf5aJIkKenWqKJQflQ/vd3g== 2023-09-13T03:11:24.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:24.167+0000 7fa59d725700 1 -- 172.21.15.64:0/3350989984 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fa5800a0750 msgr2=0x7fa5800a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:24.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:24.167+0000 7fa59d725700 1 --2- 172.21.15.64:0/3350989984 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fa5800a0750 0x7fa5800a2c10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa58c005950 tx=0x7fa58c00b500 comp rx=0 tx=0).stop 2023-09-13T03:11:24.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:24.167+0000 7fa59d725700 1 -- 172.21.15.64:0/3350989984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5981006c0 msgr2=0x7fa598196340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:24.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:24.167+0000 7fa59d725700 1 --2- 172.21.15.64:0/3350989984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5981006c0 0x7fa598196340 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa5880098e0 tx=0x7fa588005720 comp rx=0 tx=0).stop 2023-09-13T03:11:24.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:24.168+0000 7fa59d725700 1 -- 172.21.15.64:0/3350989984 shutdown_connections 2023-09-13T03:11:24.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:24.168+0000 7fa59d725700 1 --2- 172.21.15.64:0/3350989984 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fa5800a0750 0x7fa5800a2c10 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:24.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:24.168+0000 7fa59d725700 1 --2- 172.21.15.64:0/3350989984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa598102260 0x7fa59819d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:24.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:24.168+0000 7fa59d725700 1 --2- 172.21.15.64:0/3350989984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5981006c0 0x7fa598196340 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:24.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:24.168+0000 7fa59d725700 1 --2- 172.21.15.64:0/3350989984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5981018c0 0x7fa598196880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:24.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:24.168+0000 7fa59d725700 1 -- 172.21.15.64:0/3350989984 >> 172.21.15.64:0/3350989984 conn(0x7fa5980fbc80 msgr2=0x7fa5980fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:24.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:24.168+0000 7fa59d725700 1 -- 172.21.15.64:0/3350989984 shutdown_connections 2023-09-13T03:11:24.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:24.168+0000 7fa59d725700 1 -- 172.21.15.64:0/3350989984 wait complete. 2023-09-13T03:11:24.499 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:24 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2023-09-13T03:11:24.500 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:24 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2023-09-13T03:11:24.500 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:24 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2023-09-13T03:11:24.500 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:24 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2023-09-13T03:11:24.500 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:24 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2023-09-13T03:11:24.501 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:24 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2023-09-13T03:11:24.501 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:24 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2023-09-13T03:11:24.501 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:24 smithi064 ceph-mon[97675]: osdmap e84: 12 total, 12 up, 12 in 2023-09-13T03:11:24.501 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:24 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/3350989984' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-09-13T03:11:24.501 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:24 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/3350989984' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-09-13T03:11:24.554 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-13T03:11:24.554 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-09-13T03:11:24.555 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-09-13T03:11:24.658 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph config log 1 --format=json 2023-09-13T03:11:24.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:24 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2023-09-13T03:11:24.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:24 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2023-09-13T03:11:24.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:24 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2023-09-13T03:11:24.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:24 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2023-09-13T03:11:24.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:24 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2023-09-13T03:11:24.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:24 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2023-09-13T03:11:24.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:24 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2023-09-13T03:11:24.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:24 smithi170 ceph-mon[134328]: osdmap e84: 12 total, 12 up, 12 in 2023-09-13T03:11:24.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:24 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/3350989984' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-09-13T03:11:24.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:24 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/3350989984' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-09-13T03:11:24.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:24 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2023-09-13T03:11:24.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:24 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2023-09-13T03:11:24.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:24 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2023-09-13T03:11:24.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:24 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2023-09-13T03:11:24.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:24 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2023-09-13T03:11:24.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:24 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2023-09-13T03:11:24.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:24 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2023-09-13T03:11:24.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:24 smithi142 ceph-mon[134082]: osdmap e84: 12 total, 12 up, 12 in 2023-09-13T03:11:24.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:24 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/3350989984' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-09-13T03:11:24.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:24 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/3350989984' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-09-13T03:11:25.431 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:11:25.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:25 smithi170 ceph-mon[134328]: osdmap e85: 12 total, 12 up, 12 in 2023-09-13T03:11:25.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:25 smithi170 ceph-mon[134328]: pgmap v286: 97 pgs: 2 peering, 95 active+clean; 621 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 29 op/s 2023-09-13T03:11:25.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:25 smithi064 ceph-mon[97675]: osdmap e85: 12 total, 12 up, 12 in 2023-09-13T03:11:25.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:25 smithi064 ceph-mon[97675]: pgmap v286: 97 pgs: 2 peering, 95 active+clean; 621 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 29 op/s 2023-09-13T03:11:25.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:25 smithi142 ceph-mon[134082]: osdmap e85: 12 total, 12 up, 12 in 2023-09-13T03:11:25.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:25 smithi142 ceph-mon[134082]: pgmap v286: 97 pgs: 2 peering, 95 active+clean; 621 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 29 op/s 2023-09-13T03:11:27.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.026+0000 7f0fbe523700 1 -- 172.21.15.64:0/1122609529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fb8100fe0 msgr2=0x7f0fb810d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:27.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.026+0000 7f0fbe523700 1 --2- 172.21.15.64:0/1122609529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fb8100fe0 0x7f0fb810d4f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0fb000b0a0 tx=0x7f0fb000b3b0 comp rx=0 tx=0).stop 2023-09-13T03:11:27.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.026+0000 7f0fbe523700 1 -- 172.21.15.64:0/1122609529 shutdown_connections 2023-09-13T03:11:27.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.026+0000 7f0fbe523700 1 --2- 172.21.15.64:0/1122609529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fb8100fe0 0x7f0fb810d4f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:27.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.026+0000 7f0fbe523700 1 --2- 172.21.15.64:0/1122609529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0fb8106620 0x7f0fb8106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:27.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.026+0000 7f0fbe523700 1 --2- 172.21.15.64:0/1122609529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fb8100620 0x7f0fb8100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:27.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.027+0000 7f0fbe523700 1 -- 172.21.15.64:0/1122609529 >> 172.21.15.64:0/1122609529 conn(0x7f0fb80fc0f0 msgr2=0x7f0fb80fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:27.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.027+0000 7f0fbe523700 1 -- 172.21.15.64:0/1122609529 shutdown_connections 2023-09-13T03:11:27.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.027+0000 7f0fbe523700 1 -- 172.21.15.64:0/1122609529 wait complete. 2023-09-13T03:11:27.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.028+0000 7f0fbe523700 1 Processor -- start 2023-09-13T03:11:27.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.028+0000 7f0fbe523700 1 -- start start 2023-09-13T03:11:27.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.028+0000 7f0fbe523700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fb8100620 0x7f0fb819c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:27.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.028+0000 7f0fbe523700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0fb8100fe0 0x7f0fb819caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:27.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.028+0000 7f0fbe523700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fb8106620 0x7f0fb8196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:27.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.028+0000 7f0fbe523700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb8196ca0 con 0x7f0fb8100620 2023-09-13T03:11:27.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.028+0000 7f0fbe523700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb8196e10 con 0x7f0fb8106620 2023-09-13T03:11:27.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.028+0000 7f0fbe523700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb8196fd0 con 0x7f0fb8100fe0 2023-09-13T03:11:27.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.029+0000 7f0fb7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fb8100620 0x7f0fb819c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:27.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.029+0000 7f0fb7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fb8100620 0x7f0fb819c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48774/0 (socket says 172.21.15.64:48774) 2023-09-13T03:11:27.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.029+0000 7f0fb7fff700 1 -- 172.21.15.64:0/2225631254 learned_addr learned my addr 172.21.15.64:0/2225631254 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:27.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.029+0000 7f0fb77fe700 1 --2- 172.21.15.64:0/2225631254 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0fb8100fe0 0x7f0fb819caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:27.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.029+0000 7f0fbcac0700 1 --2- 172.21.15.64:0/2225631254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fb8106620 0x7f0fb8196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:27.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.029+0000 7f0fb7fff700 1 -- 172.21.15.64:0/2225631254 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0fb8100fe0 msgr2=0x7f0fb819caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:27.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.029+0000 7f0fb7fff700 1 --2- 172.21.15.64:0/2225631254 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0fb8100fe0 0x7f0fb819caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:27.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.029+0000 7f0fb7fff700 1 -- 172.21.15.64:0/2225631254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fb8106620 msgr2=0x7f0fb8196630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:27.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.029+0000 7f0fb7fff700 1 --2- 172.21.15.64:0/2225631254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fb8106620 0x7f0fb8196630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:27.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.029+0000 7f0fb7fff700 1 -- 172.21.15.64:0/2225631254 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb0009d00 con 0x7f0fb8100620 2023-09-13T03:11:27.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.029+0000 7f0fbcac0700 1 --2- 172.21.15.64:0/2225631254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fb8106620 0x7f0fb8196630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:11:27.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.029+0000 7f0fb7fff700 1 --2- 172.21.15.64:0/2225631254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fb8100620 0x7f0fb819c5b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0fa800d980 tx=0x7f0fa800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:27.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.030+0000 7f0fb57fa700 1 -- 172.21.15.64:0/2225631254 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa8004220 con 0x7f0fb8100620 2023-09-13T03:11:27.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.030+0000 7f0fb57fa700 1 -- 172.21.15.64:0/2225631254 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fa8007640 con 0x7f0fb8100620 2023-09-13T03:11:27.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.030+0000 7f0fbe523700 1 -- 172.21.15.64:0/2225631254 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fb81972b0 con 0x7f0fb8100620 2023-09-13T03:11:27.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.030+0000 7f0fb57fa700 1 -- 172.21.15.64:0/2225631254 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa8005610 con 0x7f0fb8100620 2023-09-13T03:11:27.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.030+0000 7f0fbe523700 1 -- 172.21.15.64:0/2225631254 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fb81a35a0 con 0x7f0fb8100620 2023-09-13T03:11:27.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.031+0000 7f0fbe523700 1 -- 172.21.15.64:0/2225631254 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fb804e300 con 0x7f0fb8100620 2023-09-13T03:11:27.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.033+0000 7f0fb57fa700 1 -- 172.21.15.64:0/2225631254 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f0fa8004750 con 0x7f0fb8100620 2023-09-13T03:11:27.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.034+0000 7f0fb57fa700 1 --2- 172.21.15.64:0/2225631254 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f0fa00a06d0 0x7f0fa00a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:27.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.034+0000 7f0fb57fa700 1 -- 172.21.15.64:0/2225631254 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f0fa80cbb30 con 0x7f0fb8100620 2023-09-13T03:11:27.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.034+0000 7f0fb77fe700 1 --2- 172.21.15.64:0/2225631254 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f0fa00a06d0 0x7f0fa00a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:27.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.035+0000 7f0fb77fe700 1 --2- 172.21.15.64:0/2225631254 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f0fa00a06d0 0x7f0fa00a2b90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0fb8105db0 tx=0x7f0fac00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:27.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.039+0000 7f0fb57fa700 1 -- 172.21.15.64:0/2225631254 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0fa80954d0 con 0x7f0fb8100620 2023-09-13T03:11:27.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.250+0000 7f0fbe523700 1 -- 172.21.15.64:0/2225631254 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) v1 -- 0x7f0fb8110660 con 0x7f0fb8100620 2023-09-13T03:11:27.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.251+0000 7f0fb57fa700 1 -- 172.21.15.64:0/2225631254 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v43) v1 ==== 86+0+133 (secure 0 0 0) 0x7f0fa8016020 con 0x7f0fb8100620 2023-09-13T03:11:27.252 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:11:27.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.253+0000 7f0fbe523700 1 -- 172.21.15.64:0/2225631254 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f0fa00a06d0 msgr2=0x7f0fa00a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:27.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.253+0000 7f0fbe523700 1 --2- 172.21.15.64:0/2225631254 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f0fa00a06d0 0x7f0fa00a2b90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0fb8105db0 tx=0x7f0fac00b410 comp rx=0 tx=0).stop 2023-09-13T03:11:27.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.253+0000 7f0fbe523700 1 -- 172.21.15.64:0/2225631254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fb8100620 msgr2=0x7f0fb819c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:27.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.253+0000 7f0fbe523700 1 --2- 172.21.15.64:0/2225631254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fb8100620 0x7f0fb819c5b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0fa800d980 tx=0x7f0fa800dc90 comp rx=0 tx=0).stop 2023-09-13T03:11:27.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.253+0000 7f0fbe523700 1 -- 172.21.15.64:0/2225631254 shutdown_connections 2023-09-13T03:11:27.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.253+0000 7f0fbe523700 1 --2- 172.21.15.64:0/2225631254 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f0fa00a06d0 0x7f0fa00a2b90 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:27.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.253+0000 7f0fbe523700 1 --2- 172.21.15.64:0/2225631254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fb8106620 0x7f0fb8196630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:27.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.253+0000 7f0fbe523700 1 --2- 172.21.15.64:0/2225631254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fb8100620 0x7f0fb819c5b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:27.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.253+0000 7f0fbe523700 1 --2- 172.21.15.64:0/2225631254 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0fb8100fe0 0x7f0fb819caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:27.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.253+0000 7f0fbe523700 1 -- 172.21.15.64:0/2225631254 >> 172.21.15.64:0/2225631254 conn(0x7f0fb80fc0f0 msgr2=0x7f0fb80fc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:27.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.253+0000 7f0fbe523700 1 -- 172.21.15.64:0/2225631254 shutdown_connections 2023-09-13T03:11:27.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:27.253+0000 7f0fbe523700 1 -- 172.21.15.64:0/2225631254 wait complete. 2023-09-13T03:11:27.575 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:27 smithi064 ceph-mon[97675]: pgmap v287: 97 pgs: 2 peering, 95 active+clean; 621 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 20 op/s; 0 B/s, 0 objects/s recovering 2023-09-13T03:11:27.576 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:27 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2225631254' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2023-09-13T03:11:27.610 INFO:teuthology.orchestra.run.smithi064.stdout:[{"version":43,"timestamp":"2023-09-13T03:11:04.103150+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2023-09-13T03:11:27.610 INFO:tasks.ceph_manager:config epoch is 43 2023-09-13T03:11:27.611 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-09-13T03:11:27.612 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-09-13T03:11:27.612 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph mgr dump --format=json 2023-09-13T03:11:27.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:27 smithi170 ceph-mon[134328]: pgmap v287: 97 pgs: 2 peering, 95 active+clean; 621 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 20 op/s; 0 B/s, 0 objects/s recovering 2023-09-13T03:11:27.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:27 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/2225631254' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2023-09-13T03:11:27.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:27 smithi142 ceph-mon[134082]: pgmap v287: 97 pgs: 2 peering, 95 active+clean; 621 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 20 op/s; 0 B/s, 0 objects/s recovering 2023-09-13T03:11:27.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:27 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/2225631254' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2023-09-13T03:11:28.345 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:11:29.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:29.813+0000 7fbb0ff5b700 1 -- 172.21.15.64:0/4184419628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb08100620 msgr2=0x7fbb08100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:29.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:29.813+0000 7fbb0ff5b700 1 --2- 172.21.15.64:0/4184419628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb08100620 0x7fbb08100aa0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8009990 tx=0x7fbaf8009ca0 comp rx=0 tx=0).stop 2023-09-13T03:11:29.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:29.813+0000 7fbb0ff5b700 1 -- 172.21.15.64:0/4184419628 shutdown_connections 2023-09-13T03:11:29.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:29.813+0000 7fbb0ff5b700 1 --2- 172.21.15.64:0/4184419628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb08106620 0x7fbb08106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:29.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:29.813+0000 7fbb0ff5b700 1 --2- 172.21.15.64:0/4184419628 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb08100fe0 0x7fbb0810d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:29.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:29.813+0000 7fbb0ff5b700 1 --2- 172.21.15.64:0/4184419628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb08100620 0x7fbb08100aa0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:29.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:29.813+0000 7fbb0ff5b700 1 -- 172.21.15.64:0/4184419628 >> 172.21.15.64:0/4184419628 conn(0x7fbb080fc0f0 msgr2=0x7fbb080fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:29.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:29.813+0000 7fbb0ff5b700 1 -- 172.21.15.64:0/4184419628 shutdown_connections 2023-09-13T03:11:29.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:29.813+0000 7fbb0ff5b700 1 -- 172.21.15.64:0/4184419628 wait complete. 2023-09-13T03:11:29.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:29.814+0000 7fbb0ff5b700 1 Processor -- start 2023-09-13T03:11:29.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:29.815+0000 7fbb0ff5b700 1 -- start start 2023-09-13T03:11:29.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:29.815+0000 7fbb0ff5b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb08100620 0x7fbb08072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:29.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:29.815+0000 7fbb0ff5b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb08100fe0 0x7fbb08072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:29.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:29.815+0000 7fbb0ff5b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb08106620 0x7fbb080733c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:29.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:29.815+0000 7fbb0ff5b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb081a4be0 con 0x7fbb08100fe0 2023-09-13T03:11:29.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:29.815+0000 7fbb0ff5b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb081a4d50 con 0x7fbb08100620 2023-09-13T03:11:29.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:29.815+0000 7fbb0ff5b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb081a4f10 con 0x7fbb08106620 2023-09-13T03:11:29.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:29.815+0000 7fbb0d4f6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb08100fe0 0x7fbb08072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:29.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:29.815+0000 7fbb0d4f6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb08100fe0 0x7fbb08072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48792/0 (socket says 172.21.15.64:48792) 2023-09-13T03:11:29.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:29.815+0000 7fbb0d4f6700 1 -- 172.21.15.64:0/411758079 learned_addr learned my addr 172.21.15.64:0/411758079 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:29.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:29.815+0000 7fbb0dcf7700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb08100620 0x7fbb08072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:29.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:29.815+0000 7fbb0e4f8700 1 --2- 172.21.15.64:0/411758079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb08106620 0x7fbb080733c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:29.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:29.816+0000 7fbb0d4f6700 1 -- 172.21.15.64:0/411758079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb08106620 msgr2=0x7fbb080733c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:29.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:29.816+0000 7fbb0d4f6700 1 --2- 172.21.15.64:0/411758079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb08106620 0x7fbb080733c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:29.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:29.816+0000 7fbb0d4f6700 1 -- 172.21.15.64:0/411758079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb08100620 msgr2=0x7fbb08072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:29.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:29.816+0000 7fbb0d4f6700 1 --2- 172.21.15.64:0/411758079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb08100620 0x7fbb08072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:29.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:29.816+0000 7fbb0d4f6700 1 -- 172.21.15.64:0/411758079 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf80095f0 con 0x7fbb08100fe0 2023-09-13T03:11:29.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:29.816+0000 7fbb0d4f6700 1 --2- 172.21.15.64:0/411758079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb08100fe0 0x7fbb08072e40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8005910 tx=0x7fbaf8005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:29.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:29.816+0000 7fbafeffd700 1 -- 172.21.15.64:0/411758079 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf801d070 con 0x7fbb08100fe0 2023-09-13T03:11:29.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:29.816+0000 7fbb0ff5b700 1 -- 172.21.15.64:0/411758079 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb081a5190 con 0x7fbb08100fe0 2023-09-13T03:11:29.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:29.816+0000 7fbafeffd700 1 -- 172.21.15.64:0/411758079 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbaf8022470 con 0x7fbb08100fe0 2023-09-13T03:11:29.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:29.816+0000 7fbafeffd700 1 -- 172.21.15.64:0/411758079 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8017650 con 0x7fbb08100fe0 2023-09-13T03:11:29.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:29.816+0000 7fbb0ff5b700 1 -- 172.21.15.64:0/411758079 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb081a55b0 con 0x7fbb08100fe0 2023-09-13T03:11:29.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:29.817+0000 7fbb0ff5b700 1 -- 172.21.15.64:0/411758079 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb0804e300 con 0x7fbb08100fe0 2023-09-13T03:11:29.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:29.819+0000 7fbafeffd700 1 -- 172.21.15.64:0/411758079 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fbaf80225e0 con 0x7fbb08100fe0 2023-09-13T03:11:29.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:29.821+0000 7fbafeffd700 1 --2- 172.21.15.64:0/411758079 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fbaf40a06d0 0x7fbaf40a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:29.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:29.821+0000 7fbafeffd700 1 -- 172.21.15.64:0/411758079 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fbaf80ccfe0 con 0x7fbb08100fe0 2023-09-13T03:11:29.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:29.821+0000 7fbb0dcf7700 1 --2- 172.21.15.64:0/411758079 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fbaf40a06d0 0x7fbaf40a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:29.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:29.822+0000 7fbb0dcf7700 1 --2- 172.21.15.64:0/411758079 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fbaf40a06d0 0x7fbaf40a2b90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbb04009be0 tx=0x7fbb04009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:29.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:29.825+0000 7fbafeffd700 1 -- 172.21.15.64:0/411758079 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fbaf8096b90 con 0x7fbb08100fe0 2023-09-13T03:11:29.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:29 smithi170 ceph-mon[134328]: pgmap v288: 97 pgs: 2 peering, 95 active+clean; 631 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.9 KiB/s wr, 18 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-09-13T03:11:29.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:29 smithi064 ceph-mon[97675]: pgmap v288: 97 pgs: 2 peering, 95 active+clean; 631 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.9 KiB/s wr, 18 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-09-13T03:11:29.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:29 smithi142 ceph-mon[134082]: pgmap v288: 97 pgs: 2 peering, 95 active+clean; 631 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.9 KiB/s wr, 18 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-09-13T03:11:30.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:30.093+0000 7fbb0ff5b700 1 -- 172.21.15.64:0/411758079 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fbb08110650 con 0x7fbb08100fe0 2023-09-13T03:11:30.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:30.098+0000 7fbafeffd700 1 -- 172.21.15.64:0/411758079 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+259015 (secure 0 0 0) 0x7fbaf81133e0 con 0x7fbb08100fe0 2023-09-13T03:11:30.103 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:11:30.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:30.102+0000 7fbb0ff5b700 1 -- 172.21.15.64:0/411758079 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fbaf40a06d0 msgr2=0x7fbaf40a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:30.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:30.102+0000 7fbb0ff5b700 1 --2- 172.21.15.64:0/411758079 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fbaf40a06d0 0x7fbaf40a2b90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbb04009be0 tx=0x7fbb04009260 comp rx=0 tx=0).stop 2023-09-13T03:11:30.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:30.102+0000 7fbb0ff5b700 1 -- 172.21.15.64:0/411758079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb08100fe0 msgr2=0x7fbb08072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:30.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:30.102+0000 7fbb0ff5b700 1 --2- 172.21.15.64:0/411758079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb08100fe0 0x7fbb08072e40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8005910 tx=0x7fbaf8005720 comp rx=0 tx=0).stop 2023-09-13T03:11:30.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:30.103+0000 7fbb0ff5b700 1 -- 172.21.15.64:0/411758079 shutdown_connections 2023-09-13T03:11:30.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:30.103+0000 7fbb0ff5b700 1 --2- 172.21.15.64:0/411758079 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fbaf40a06d0 0x7fbaf40a2b90 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:30.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:30.103+0000 7fbb0ff5b700 1 --2- 172.21.15.64:0/411758079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb08100620 0x7fbb08072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:30.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:30.103+0000 7fbb0ff5b700 1 --2- 172.21.15.64:0/411758079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb08106620 0x7fbb080733c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:30.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:30.103+0000 7fbb0ff5b700 1 --2- 172.21.15.64:0/411758079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb08100fe0 0x7fbb08072e40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:30.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:30.103+0000 7fbb0ff5b700 1 -- 172.21.15.64:0/411758079 >> 172.21.15.64:0/411758079 conn(0x7fbb080fc0f0 msgr2=0x7fbb080fc740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:30.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:30.103+0000 7fbb0ff5b700 1 -- 172.21.15.64:0/411758079 shutdown_connections 2023-09-13T03:11:30.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:30.103+0000 7fbb0ff5b700 1 -- 172.21.15.64:0/411758079 wait complete. 2023-09-13T03:11:30.620 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":17,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":873679308},{"type":"v1","addr":"172.21.15.64:6801","nonce":873679308}]},"active_addr":"172.21.15.64:6801/873679308","active_change":"2023-09-13T03:04:22.355934+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24226,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24233,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.64:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":791645251}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":1170163026}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":4083942699}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":3771142265}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":2286078816}]}]} 2023-09-13T03:11:30.626 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-09-13T03:11:30.627 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-09-13T03:11:30.628 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph osd dump --format=json 2023-09-13T03:11:30.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:30 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/411758079' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-09-13T03:11:30.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:30 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/411758079' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-09-13T03:11:30.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:30 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/411758079' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-09-13T03:11:31.364 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:11:31.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:31 smithi170 ceph-mon[134328]: pgmap v289: 97 pgs: 97 active+clean; 631 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 KiB/s wr, 14 op/s; 8 B/s, 2 keys/s, 1 objects/s recovering 2023-09-13T03:11:31.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:31 smithi064 ceph-mon[97675]: pgmap v289: 97 pgs: 97 active+clean; 631 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 KiB/s wr, 14 op/s; 8 B/s, 2 keys/s, 1 objects/s recovering 2023-09-13T03:11:31.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:31 smithi142 ceph-mon[134082]: pgmap v289: 97 pgs: 97 active+clean; 631 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 KiB/s wr, 14 op/s; 8 B/s, 2 keys/s, 1 objects/s recovering 2023-09-13T03:11:32.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:32.920+0000 7f6e47907700 1 -- 172.21.15.64:0/2448824824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e40106620 msgr2=0x7f6e40106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:32.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:32.920+0000 7f6e47907700 1 --2- 172.21.15.64:0/2448824824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e40106620 0x7f6e40106a00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c009990 tx=0x7f6e3c009ca0 comp rx=0 tx=0).stop 2023-09-13T03:11:32.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:32.920+0000 7f6e47907700 1 -- 172.21.15.64:0/2448824824 shutdown_connections 2023-09-13T03:11:32.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:32.920+0000 7f6e47907700 1 --2- 172.21.15.64:0/2448824824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e40100fe0 0x7f6e4010d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:32.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:32.920+0000 7f6e47907700 1 --2- 172.21.15.64:0/2448824824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e40106620 0x7f6e40106a00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:32.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:32.920+0000 7f6e47907700 1 --2- 172.21.15.64:0/2448824824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e40100620 0x7f6e40100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:32.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:32.920+0000 7f6e47907700 1 -- 172.21.15.64:0/2448824824 >> 172.21.15.64:0/2448824824 conn(0x7f6e400fc0f0 msgr2=0x7f6e400fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:32.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:32.920+0000 7f6e47907700 1 -- 172.21.15.64:0/2448824824 shutdown_connections 2023-09-13T03:11:32.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:32.920+0000 7f6e47907700 1 -- 172.21.15.64:0/2448824824 wait complete. 2023-09-13T03:11:32.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:32.921+0000 7f6e47907700 1 Processor -- start 2023-09-13T03:11:32.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:32.921+0000 7f6e47907700 1 -- start start 2023-09-13T03:11:32.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:32.922+0000 7f6e47907700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e40100620 0x7f6e40196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:32.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:32.922+0000 7f6e47907700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e40100fe0 0x7f6e40196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:32.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:32.922+0000 7f6e47907700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e40106620 0x7f6e4019a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:32.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:32.922+0000 7f6e47907700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4019a600 con 0x7f6e40100fe0 2023-09-13T03:11:32.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:32.922+0000 7f6e47907700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4019a770 con 0x7f6e40106620 2023-09-13T03:11:32.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:32.922+0000 7f6e47907700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4019a930 con 0x7f6e40100620 2023-09-13T03:11:32.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:32.922+0000 7f6e44ea2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e40100fe0 0x7f6e40196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:32.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:32.922+0000 7f6e44ea2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e40100fe0 0x7f6e40196c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48796/0 (socket says 172.21.15.64:48796) 2023-09-13T03:11:32.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:32.922+0000 7f6e44ea2700 1 -- 172.21.15.64:0/2560268842 learned_addr learned my addr 172.21.15.64:0/2560268842 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:32.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:32.922+0000 7f6e456a3700 1 --2- 172.21.15.64:0/2560268842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e40100620 0x7f6e40196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:32.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:32.922+0000 7f6e45ea4700 1 --2- 172.21.15.64:0/2560268842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e40106620 0x7f6e4019a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:32.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:32.922+0000 7f6e44ea2700 1 -- 172.21.15.64:0/2560268842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e40100620 msgr2=0x7f6e40196700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:32.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:32.922+0000 7f6e44ea2700 1 --2- 172.21.15.64:0/2560268842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e40100620 0x7f6e40196700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:32.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:32.922+0000 7f6e44ea2700 1 -- 172.21.15.64:0/2560268842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e40106620 msgr2=0x7f6e4019a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:32.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:32.922+0000 7f6e44ea2700 1 --2- 172.21.15.64:0/2560268842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e40106620 0x7f6e4019a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:32.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:32.923+0000 7f6e44ea2700 1 -- 172.21.15.64:0/2560268842 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e3c0095f0 con 0x7f6e40100fe0 2023-09-13T03:11:32.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:32.923+0000 7f6e44ea2700 1 --2- 172.21.15.64:0/2560268842 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e40100fe0 0x7f6e40196c40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6e3000d980 tx=0x7f6e3000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:32.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:32.923+0000 7f6e367fc700 1 -- 172.21.15.64:0/2560268842 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e30004220 con 0x7f6e40100fe0 2023-09-13T03:11:32.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:32.923+0000 7f6e367fc700 1 -- 172.21.15.64:0/2560268842 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e30007640 con 0x7f6e40100fe0 2023-09-13T03:11:32.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:32.923+0000 7f6e47907700 1 -- 172.21.15.64:0/2560268842 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e4019ac10 con 0x7f6e40100fe0 2023-09-13T03:11:32.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:32.923+0000 7f6e367fc700 1 -- 172.21.15.64:0/2560268842 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e30005690 con 0x7f6e40100fe0 2023-09-13T03:11:32.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:32.923+0000 7f6e47907700 1 -- 172.21.15.64:0/2560268842 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e401a3560 con 0x7f6e40100fe0 2023-09-13T03:11:32.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:32.924+0000 7f6e47907700 1 -- 172.21.15.64:0/2560268842 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e4004e300 con 0x7f6e40100fe0 2023-09-13T03:11:32.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:32.926+0000 7f6e367fc700 1 -- 172.21.15.64:0/2560268842 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f6e30004750 con 0x7f6e40100fe0 2023-09-13T03:11:32.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:32.927+0000 7f6e367fc700 1 --2- 172.21.15.64:0/2560268842 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6e2c0a0750 0x7f6e2c0a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:32.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:32.928+0000 7f6e367fc700 1 -- 172.21.15.64:0/2560268842 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f6e300cbd50 con 0x7f6e40100fe0 2023-09-13T03:11:32.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:32.928+0000 7f6e456a3700 1 --2- 172.21.15.64:0/2560268842 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6e2c0a0750 0x7f6e2c0a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:32.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:32.928+0000 7f6e456a3700 1 --2- 172.21.15.64:0/2560268842 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6e2c0a0750 0x7f6e2c0a2c10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c00b2e0 tx=0x7f6e3c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:32.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:32.930+0000 7f6e367fc700 1 -- 172.21.15.64:0/2560268842 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6e300956f0 con 0x7f6e40100fe0 2023-09-13T03:11:33.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:33.139+0000 7f6e47907700 1 -- 172.21.15.64:0/2560268842 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e40197a60 con 0x7f6e40100fe0 2023-09-13T03:11:33.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:33.141+0000 7f6e367fc700 1 -- 172.21.15.64:0/2560268842 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22473 (secure 0 0 0) 0x7f6e3004d7e0 con 0x7f6e40100fe0 2023-09-13T03:11:33.142 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:11:33.142 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":85,"fsid":"f8dbced4-51e1-11ee-9ab7-7b867c8bd7da","created":"2023-09-13T03:03:24.167202+0000","modified":"2023-09-13T03:11:24.258871+0000","last_up_change":"2023-09-13T03:10:22.503529+0000","last_in_change":"2023-09-13T03:10:05.678720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T03:06:58.987479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-13T03:11:08.589202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-13T03:11:09.924659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3997c114-241b-48c4-8feb-653c99209470","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6803","nonce":3940892135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6805","nonce":3940892135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6809","nonce":3940892135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6807","nonce":3940892135}]},"public_addr":"172.21.15.64:6803/3940892135","cluster_addr":"172.21.15.64:6805/3940892135","heartbeat_back_addr":"172.21.15.64:6809/3940892135","heartbeat_front_addr":"172.21.15.64:6807/3940892135","state":["exists","up"]},{"osd":1,"uuid":"94cfae85-322e-4ed2-a6e6-e40349bb14bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6801","nonce":4188149333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6803","nonce":4188149333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6807","nonce":4188149333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6805","nonce":4188149333}]},"public_addr":"172.21.15.142:6801/4188149333","cluster_addr":"172.21.15.142:6803/4188149333","heartbeat_back_addr":"172.21.15.142:6807/4188149333","heartbeat_front_addr":"172.21.15.142:6805/4188149333","state":["exists","up"]},{"osd":2,"uuid":"b3aae17e-96db-4008-8c27-f733c46b30ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6801","nonce":2406669338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6803","nonce":2406669338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6807","nonce":2406669338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6805","nonce":2406669338}]},"public_addr":"172.21.15.170:6801/2406669338","cluster_addr":"172.21.15.170:6803/2406669338","heartbeat_back_addr":"172.21.15.170:6807/2406669338","heartbeat_front_addr":"172.21.15.170:6805/2406669338","state":["exists","up"]},{"osd":3,"uuid":"7fa8d8cb-2ab7-48a7-8b49-5e2903d6779f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6811","nonce":3558318707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6813","nonce":3558318707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6817","nonce":3558318707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6815","nonce":3558318707}]},"public_addr":"172.21.15.64:6811/3558318707","cluster_addr":"172.21.15.64:6813/3558318707","heartbeat_back_addr":"172.21.15.64:6817/3558318707","heartbeat_front_addr":"172.21.15.64:6815/3558318707","state":["exists","up"]},{"osd":4,"uuid":"8c7a9c9f-f919-48d3-9db2-214a25bb5603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6809","nonce":1811142576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6811","nonce":1811142576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6815","nonce":1811142576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6813","nonce":1811142576}]},"public_addr":"172.21.15.142:6809/1811142576","cluster_addr":"172.21.15.142:6811/1811142576","heartbeat_back_addr":"172.21.15.142:6815/1811142576","heartbeat_front_addr":"172.21.15.142:6813/1811142576","state":["exists","up"]},{"osd":5,"uuid":"0c6101c6-2065-4022-a751-d3f216d7f478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6809","nonce":2903841453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6811","nonce":2903841453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6815","nonce":2903841453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6813","nonce":2903841453}]},"public_addr":"172.21.15.170:6809/2903841453","cluster_addr":"172.21.15.170:6811/2903841453","heartbeat_back_addr":"172.21.15.170:6815/2903841453","heartbeat_front_addr":"172.21.15.170:6813/2903841453","state":["exists","up"]},{"osd":6,"uuid":"5cf3e52e-d155-41b1-9194-db047d26a953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6819","nonce":1436584490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6821","nonce":1436584490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6825","nonce":1436584490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6823","nonce":1436584490}]},"public_addr":"172.21.15.64:6819/1436584490","cluster_addr":"172.21.15.64:6821/1436584490","heartbeat_back_addr":"172.21.15.64:6825/1436584490","heartbeat_front_addr":"172.21.15.64:6823/1436584490","state":["exists","up"]},{"osd":7,"uuid":"e97323e3-badd-41a7-af9e-e78a81a64046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6817","nonce":3726793534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6819","nonce":3726793534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6823","nonce":3726793534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6821","nonce":3726793534}]},"public_addr":"172.21.15.142:6817/3726793534","cluster_addr":"172.21.15.142:6819/3726793534","heartbeat_back_addr":"172.21.15.142:6823/3726793534","heartbeat_front_addr":"172.21.15.142:6821/3726793534","state":["exists","up"]},{"osd":8,"uuid":"83d2835c-ed26-493d-9222-c2894ec0e071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6817","nonce":4272513873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6819","nonce":4272513873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6823","nonce":4272513873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6821","nonce":4272513873}]},"public_addr":"172.21.15.170:6817/4272513873","cluster_addr":"172.21.15.170:6819/4272513873","heartbeat_back_addr":"172.21.15.170:6823/4272513873","heartbeat_front_addr":"172.21.15.170:6821/4272513873","state":["exists","up"]},{"osd":9,"uuid":"cb936bdd-1bec-4236-aa2e-a24adcac11e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6827","nonce":2641175211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6829","nonce":2641175211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6833","nonce":2641175211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6831","nonce":2641175211}]},"public_addr":"172.21.15.64:6827/2641175211","cluster_addr":"172.21.15.64:6829/2641175211","heartbeat_back_addr":"172.21.15.64:6833/2641175211","heartbeat_front_addr":"172.21.15.64:6831/2641175211","state":["exists","up"]},{"osd":10,"uuid":"85512a1c-6171-43d4-9c15-c588a8b71be0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6825","nonce":936499217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6827","nonce":936499217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6831","nonce":936499217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6829","nonce":936499217}]},"public_addr":"172.21.15.142:6825/936499217","cluster_addr":"172.21.15.142:6827/936499217","heartbeat_back_addr":"172.21.15.142:6831/936499217","heartbeat_front_addr":"172.21.15.142:6829/936499217","state":["exists","up"]},{"osd":11,"uuid":"9b8417dc-5d6c-4917-94a9-b032c5bc33d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6825","nonce":3605017060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6827","nonce":3605017060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6831","nonce":3605017060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6829","nonce":3605017060}]},"public_addr":"172.21.15.170:6825/3605017060","cluster_addr":"172.21.15.170:6827/3605017060","heartbeat_back_addr":"172.21.15.170:6831/3605017060","heartbeat_front_addr":"172.21.15.170:6829/3605017060","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:14.229044+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:36.147777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:56.655298+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:07:23.107741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:07:44.663770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:05.577432+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:31.168031+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:52.791055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:09:13.658546+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:09:39.150360+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:10:00.601735+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:10:21.407555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6800/2955227347":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/1373223118":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/271194613":"2023-09-14T03:04:22.355817+0000","172.21.15.64:6801/3998091347":"2023-09-14T03:03:56.442541+0000","172.21.15.64:6800/3998091347":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/920818572":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/735789005":"2023-09-14T03:03:56.442541+0000","172.21.15.64:6801/2955227347":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/1975923178":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/1500151178":"2023-09-14T03:04:22.355817+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T03:11:33.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:33.144+0000 7f6e47907700 1 -- 172.21.15.64:0/2560268842 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6e2c0a0750 msgr2=0x7f6e2c0a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:33.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:33.144+0000 7f6e47907700 1 --2- 172.21.15.64:0/2560268842 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6e2c0a0750 0x7f6e2c0a2c10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c00b2e0 tx=0x7f6e3c005fd0 comp rx=0 tx=0).stop 2023-09-13T03:11:33.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:33.144+0000 7f6e47907700 1 -- 172.21.15.64:0/2560268842 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e40100fe0 msgr2=0x7f6e40196c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:33.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:33.144+0000 7f6e47907700 1 --2- 172.21.15.64:0/2560268842 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e40100fe0 0x7f6e40196c40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6e3000d980 tx=0x7f6e3000dc90 comp rx=0 tx=0).stop 2023-09-13T03:11:33.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:33.144+0000 7f6e47907700 1 -- 172.21.15.64:0/2560268842 shutdown_connections 2023-09-13T03:11:33.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:33.144+0000 7f6e47907700 1 --2- 172.21.15.64:0/2560268842 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6e2c0a0750 0x7f6e2c0a2c10 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:33.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:33.144+0000 7f6e47907700 1 --2- 172.21.15.64:0/2560268842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e40106620 0x7f6e4019a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:33.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:33.144+0000 7f6e47907700 1 --2- 172.21.15.64:0/2560268842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e40100620 0x7f6e40196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:33.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:33.144+0000 7f6e47907700 1 --2- 172.21.15.64:0/2560268842 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e40100fe0 0x7f6e40196c40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:33.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:33.144+0000 7f6e47907700 1 -- 172.21.15.64:0/2560268842 >> 172.21.15.64:0/2560268842 conn(0x7f6e400fc0f0 msgr2=0x7f6e40104130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:33.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:33.144+0000 7f6e47907700 1 -- 172.21.15.64:0/2560268842 shutdown_connections 2023-09-13T03:11:33.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:33.144+0000 7f6e47907700 1 -- 172.21.15.64:0/2560268842 wait complete. 2023-09-13T03:11:33.542 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-09-13T03:11:33.542 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph osd dump --format=json 2023-09-13T03:11:33.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:33 smithi170 ceph-mon[134328]: pgmap v290: 97 pgs: 97 active+clean; 631 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s; 7 B/s, 1 keys/s, 1 objects/s recovering 2023-09-13T03:11:33.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:33 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/2560268842' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:11:33.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:33 smithi064 ceph-mon[97675]: pgmap v290: 97 pgs: 97 active+clean; 631 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s; 7 B/s, 1 keys/s, 1 objects/s recovering 2023-09-13T03:11:33.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:33 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2560268842' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:11:33.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:33 smithi142 ceph-mon[134082]: pgmap v290: 97 pgs: 97 active+clean; 631 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s; 7 B/s, 1 keys/s, 1 objects/s recovering 2023-09-13T03:11:33.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:33 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/2560268842' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:11:34.276 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:11:35.479 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:35 smithi064 ceph-mon[97675]: pgmap v291: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.3 KiB/s wr, 10 op/s; 6 B/s, 1 keys/s, 1 objects/s recovering 2023-09-13T03:11:35.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:35 smithi142 ceph-mon[134082]: pgmap v291: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.3 KiB/s wr, 10 op/s; 6 B/s, 1 keys/s, 1 objects/s recovering 2023-09-13T03:11:35.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.655+0000 7f85c2267700 1 -- 172.21.15.64:0/2286821661 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85bc100fe0 msgr2=0x7f85bc10d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:35.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.655+0000 7f85c2267700 1 --2- 172.21.15.64:0/2286821661 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85bc100fe0 0x7f85bc10d4f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f85b400b0a0 tx=0x7f85b400b3b0 comp rx=0 tx=0).stop 2023-09-13T03:11:35.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.655+0000 7f85c2267700 1 -- 172.21.15.64:0/2286821661 shutdown_connections 2023-09-13T03:11:35.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.655+0000 7f85c2267700 1 --2- 172.21.15.64:0/2286821661 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85bc100fe0 0x7f85bc10d4f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:35.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.655+0000 7f85c2267700 1 --2- 172.21.15.64:0/2286821661 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85bc106620 0x7f85bc106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:35.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.655+0000 7f85c2267700 1 --2- 172.21.15.64:0/2286821661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85bc100620 0x7f85bc100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:35.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.655+0000 7f85c2267700 1 -- 172.21.15.64:0/2286821661 >> 172.21.15.64:0/2286821661 conn(0x7f85bc0fc0f0 msgr2=0x7f85bc0fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:35.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.655+0000 7f85c2267700 1 -- 172.21.15.64:0/2286821661 shutdown_connections 2023-09-13T03:11:35.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.655+0000 7f85c2267700 1 -- 172.21.15.64:0/2286821661 wait complete. 2023-09-13T03:11:35.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.657+0000 7f85c2267700 1 Processor -- start 2023-09-13T03:11:35.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.657+0000 7f85c2267700 1 -- start start 2023-09-13T03:11:35.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.658+0000 7f85c2267700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85bc100620 0x7f85bc072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:35.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.658+0000 7f85c2267700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85bc100fe0 0x7f85bc072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:35.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.658+0000 7f85c2267700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85bc106620 0x7f85bc0733c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:35.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.658+0000 7f85c2267700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc1a4c50 con 0x7f85bc106620 2023-09-13T03:11:35.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.658+0000 7f85c2267700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc1a4dc0 con 0x7f85bc100620 2023-09-13T03:11:35.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.658+0000 7f85c2267700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc1a4f80 con 0x7f85bc100fe0 2023-09-13T03:11:35.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.658+0000 7f85c0804700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85bc106620 0x7f85bc0733c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:35.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.658+0000 7f85bbfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85bc100620 0x7f85bc072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:35.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.658+0000 7f85bb7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85bc100fe0 0x7f85bc072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:35.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.658+0000 7f85c0804700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85bc106620 0x7f85bc0733c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48812/0 (socket says 172.21.15.64:48812) 2023-09-13T03:11:35.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.658+0000 7f85bbfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85bc100620 0x7f85bc072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:60074/0 (socket says 172.21.15.64:60074) 2023-09-13T03:11:35.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.658+0000 7f85c0804700 1 -- 172.21.15.64:0/920957211 learned_addr learned my addr 172.21.15.64:0/920957211 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:35.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.658+0000 7f85c0804700 1 -- 172.21.15.64:0/920957211 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85bc100fe0 msgr2=0x7f85bc072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:35.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.658+0000 7f85c0804700 1 --2- 172.21.15.64:0/920957211 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85bc100fe0 0x7f85bc072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:35.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.658+0000 7f85c0804700 1 -- 172.21.15.64:0/920957211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85bc100620 msgr2=0x7f85bc072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:35.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.658+0000 7f85c0804700 1 --2- 172.21.15.64:0/920957211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85bc100620 0x7f85bc072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:35.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.658+0000 7f85c0804700 1 -- 172.21.15.64:0/920957211 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b4009d00 con 0x7f85bc106620 2023-09-13T03:11:35.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.659+0000 7f85c0804700 1 --2- 172.21.15.64:0/920957211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85bc106620 0x7f85bc0733c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f85b400b780 tx=0x7f85b4007990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:35.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.659+0000 7f85b97fa700 1 -- 172.21.15.64:0/920957211 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b400e040 con 0x7f85bc106620 2023-09-13T03:11:35.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.659+0000 7f85c2267700 1 -- 172.21.15.64:0/920957211 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85bc1a5200 con 0x7f85bc106620 2023-09-13T03:11:35.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.659+0000 7f85b97fa700 1 -- 172.21.15.64:0/920957211 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85b401d470 con 0x7f85bc106620 2023-09-13T03:11:35.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.659+0000 7f85b97fa700 1 -- 172.21.15.64:0/920957211 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b40052a0 con 0x7f85bc106620 2023-09-13T03:11:35.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.659+0000 7f85c2267700 1 -- 172.21.15.64:0/920957211 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85bc1a5670 con 0x7f85bc106620 2023-09-13T03:11:35.665 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:35 smithi170 ceph-mon[134328]: pgmap v291: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.3 KiB/s wr, 10 op/s; 6 B/s, 1 keys/s, 1 objects/s recovering 2023-09-13T03:11:35.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.660+0000 7f85c2267700 1 -- 172.21.15.64:0/920957211 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85bc04e300 con 0x7f85bc106620 2023-09-13T03:11:35.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.662+0000 7f85b97fa700 1 -- 172.21.15.64:0/920957211 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f85b401d5e0 con 0x7f85bc106620 2023-09-13T03:11:35.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.665+0000 7f85b97fa700 1 --2- 172.21.15.64:0/920957211 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f85a40a06d0 0x7f85a40a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:35.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.665+0000 7f85b97fa700 1 -- 172.21.15.64:0/920957211 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f85b40cd040 con 0x7f85bc106620 2023-09-13T03:11:35.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.665+0000 7f85bbfff700 1 --2- 172.21.15.64:0/920957211 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f85a40a06d0 0x7f85a40a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:35.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.665+0000 7f85bbfff700 1 --2- 172.21.15.64:0/920957211 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f85a40a06d0 0x7f85a40a2b90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f85ac009be0 tx=0x7f85ac009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:35.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.669+0000 7f85b97fa700 1 -- 172.21.15.64:0/920957211 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f85b4096ad0 con 0x7f85bc106620 2023-09-13T03:11:35.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.881+0000 7f85c2267700 1 -- 172.21.15.64:0/920957211 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f85bc1107c0 con 0x7f85bc106620 2023-09-13T03:11:35.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.883+0000 7f85b97fa700 1 -- 172.21.15.64:0/920957211 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22473 (secure 0 0 0) 0x7f85b409a0f0 con 0x7f85bc106620 2023-09-13T03:11:35.884 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:11:35.884 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":85,"fsid":"f8dbced4-51e1-11ee-9ab7-7b867c8bd7da","created":"2023-09-13T03:03:24.167202+0000","modified":"2023-09-13T03:11:24.258871+0000","last_up_change":"2023-09-13T03:10:22.503529+0000","last_in_change":"2023-09-13T03:10:05.678720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T03:06:58.987479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-13T03:11:08.589202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-13T03:11:09.924659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3997c114-241b-48c4-8feb-653c99209470","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6803","nonce":3940892135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6805","nonce":3940892135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6809","nonce":3940892135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6807","nonce":3940892135}]},"public_addr":"172.21.15.64:6803/3940892135","cluster_addr":"172.21.15.64:6805/3940892135","heartbeat_back_addr":"172.21.15.64:6809/3940892135","heartbeat_front_addr":"172.21.15.64:6807/3940892135","state":["exists","up"]},{"osd":1,"uuid":"94cfae85-322e-4ed2-a6e6-e40349bb14bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6801","nonce":4188149333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6803","nonce":4188149333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6807","nonce":4188149333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6805","nonce":4188149333}]},"public_addr":"172.21.15.142:6801/4188149333","cluster_addr":"172.21.15.142:6803/4188149333","heartbeat_back_addr":"172.21.15.142:6807/4188149333","heartbeat_front_addr":"172.21.15.142:6805/4188149333","state":["exists","up"]},{"osd":2,"uuid":"b3aae17e-96db-4008-8c27-f733c46b30ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6801","nonce":2406669338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6803","nonce":2406669338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6807","nonce":2406669338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6805","nonce":2406669338}]},"public_addr":"172.21.15.170:6801/2406669338","cluster_addr":"172.21.15.170:6803/2406669338","heartbeat_back_addr":"172.21.15.170:6807/2406669338","heartbeat_front_addr":"172.21.15.170:6805/2406669338","state":["exists","up"]},{"osd":3,"uuid":"7fa8d8cb-2ab7-48a7-8b49-5e2903d6779f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6811","nonce":3558318707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6813","nonce":3558318707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6817","nonce":3558318707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6815","nonce":3558318707}]},"public_addr":"172.21.15.64:6811/3558318707","cluster_addr":"172.21.15.64:6813/3558318707","heartbeat_back_addr":"172.21.15.64:6817/3558318707","heartbeat_front_addr":"172.21.15.64:6815/3558318707","state":["exists","up"]},{"osd":4,"uuid":"8c7a9c9f-f919-48d3-9db2-214a25bb5603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6809","nonce":1811142576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6811","nonce":1811142576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6815","nonce":1811142576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6813","nonce":1811142576}]},"public_addr":"172.21.15.142:6809/1811142576","cluster_addr":"172.21.15.142:6811/1811142576","heartbeat_back_addr":"172.21.15.142:6815/1811142576","heartbeat_front_addr":"172.21.15.142:6813/1811142576","state":["exists","up"]},{"osd":5,"uuid":"0c6101c6-2065-4022-a751-d3f216d7f478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6809","nonce":2903841453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6811","nonce":2903841453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6815","nonce":2903841453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6813","nonce":2903841453}]},"public_addr":"172.21.15.170:6809/2903841453","cluster_addr":"172.21.15.170:6811/2903841453","heartbeat_back_addr":"172.21.15.170:6815/2903841453","heartbeat_front_addr":"172.21.15.170:6813/2903841453","state":["exists","up"]},{"osd":6,"uuid":"5cf3e52e-d155-41b1-9194-db047d26a953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6819","nonce":1436584490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6821","nonce":1436584490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6825","nonce":1436584490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6823","nonce":1436584490}]},"public_addr":"172.21.15.64:6819/1436584490","cluster_addr":"172.21.15.64:6821/1436584490","heartbeat_back_addr":"172.21.15.64:6825/1436584490","heartbeat_front_addr":"172.21.15.64:6823/1436584490","state":["exists","up"]},{"osd":7,"uuid":"e97323e3-badd-41a7-af9e-e78a81a64046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6817","nonce":3726793534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6819","nonce":3726793534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6823","nonce":3726793534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6821","nonce":3726793534}]},"public_addr":"172.21.15.142:6817/3726793534","cluster_addr":"172.21.15.142:6819/3726793534","heartbeat_back_addr":"172.21.15.142:6823/3726793534","heartbeat_front_addr":"172.21.15.142:6821/3726793534","state":["exists","up"]},{"osd":8,"uuid":"83d2835c-ed26-493d-9222-c2894ec0e071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6817","nonce":4272513873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6819","nonce":4272513873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6823","nonce":4272513873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6821","nonce":4272513873}]},"public_addr":"172.21.15.170:6817/4272513873","cluster_addr":"172.21.15.170:6819/4272513873","heartbeat_back_addr":"172.21.15.170:6823/4272513873","heartbeat_front_addr":"172.21.15.170:6821/4272513873","state":["exists","up"]},{"osd":9,"uuid":"cb936bdd-1bec-4236-aa2e-a24adcac11e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6827","nonce":2641175211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6829","nonce":2641175211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6833","nonce":2641175211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6831","nonce":2641175211}]},"public_addr":"172.21.15.64:6827/2641175211","cluster_addr":"172.21.15.64:6829/2641175211","heartbeat_back_addr":"172.21.15.64:6833/2641175211","heartbeat_front_addr":"172.21.15.64:6831/2641175211","state":["exists","up"]},{"osd":10,"uuid":"85512a1c-6171-43d4-9c15-c588a8b71be0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6825","nonce":936499217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6827","nonce":936499217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6831","nonce":936499217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6829","nonce":936499217}]},"public_addr":"172.21.15.142:6825/936499217","cluster_addr":"172.21.15.142:6827/936499217","heartbeat_back_addr":"172.21.15.142:6831/936499217","heartbeat_front_addr":"172.21.15.142:6829/936499217","state":["exists","up"]},{"osd":11,"uuid":"9b8417dc-5d6c-4917-94a9-b032c5bc33d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6825","nonce":3605017060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6827","nonce":3605017060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6831","nonce":3605017060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6829","nonce":3605017060}]},"public_addr":"172.21.15.170:6825/3605017060","cluster_addr":"172.21.15.170:6827/3605017060","heartbeat_back_addr":"172.21.15.170:6831/3605017060","heartbeat_front_addr":"172.21.15.170:6829/3605017060","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:14.229044+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:36.147777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:56.655298+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:07:23.107741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:07:44.663770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:05.577432+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:31.168031+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:52.791055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:09:13.658546+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:09:39.150360+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:10:00.601735+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:10:21.407555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6800/2955227347":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/1373223118":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/271194613":"2023-09-14T03:04:22.355817+0000","172.21.15.64:6801/3998091347":"2023-09-14T03:03:56.442541+0000","172.21.15.64:6800/3998091347":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/920818572":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/735789005":"2023-09-14T03:03:56.442541+0000","172.21.15.64:6801/2955227347":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/1975923178":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/1500151178":"2023-09-14T03:04:22.355817+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T03:11:35.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.885+0000 7f85c2267700 1 -- 172.21.15.64:0/920957211 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f85a40a06d0 msgr2=0x7f85a40a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:35.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.885+0000 7f85c2267700 1 --2- 172.21.15.64:0/920957211 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f85a40a06d0 0x7f85a40a2b90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f85ac009be0 tx=0x7f85ac009260 comp rx=0 tx=0).stop 2023-09-13T03:11:35.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.885+0000 7f85c2267700 1 -- 172.21.15.64:0/920957211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85bc106620 msgr2=0x7f85bc0733c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:35.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.885+0000 7f85c2267700 1 --2- 172.21.15.64:0/920957211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85bc106620 0x7f85bc0733c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f85b400b780 tx=0x7f85b4007990 comp rx=0 tx=0).stop 2023-09-13T03:11:35.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.885+0000 7f85c2267700 1 -- 172.21.15.64:0/920957211 shutdown_connections 2023-09-13T03:11:35.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.885+0000 7f85c2267700 1 --2- 172.21.15.64:0/920957211 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f85a40a06d0 0x7f85a40a2b90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:35.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.885+0000 7f85c2267700 1 --2- 172.21.15.64:0/920957211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85bc100620 0x7f85bc072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:35.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.885+0000 7f85c2267700 1 --2- 172.21.15.64:0/920957211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85bc106620 0x7f85bc0733c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:35.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.885+0000 7f85c2267700 1 --2- 172.21.15.64:0/920957211 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85bc100fe0 0x7f85bc072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:35.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.885+0000 7f85c2267700 1 -- 172.21.15.64:0/920957211 >> 172.21.15.64:0/920957211 conn(0x7f85bc0fc0f0 msgr2=0x7f85bc0fc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:35.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.886+0000 7f85c2267700 1 -- 172.21.15.64:0/920957211 shutdown_connections 2023-09-13T03:11:35.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:35.886+0000 7f85c2267700 1 -- 172.21.15.64:0/920957211 wait complete. 2023-09-13T03:11:36.257 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph tell osd.0 flush_pg_stats 2023-09-13T03:11:36.258 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph tell osd.1 flush_pg_stats 2023-09-13T03:11:36.258 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph tell osd.2 flush_pg_stats 2023-09-13T03:11:36.259 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph tell osd.3 flush_pg_stats 2023-09-13T03:11:36.259 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph tell osd.4 flush_pg_stats 2023-09-13T03:11:36.260 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph tell osd.5 flush_pg_stats 2023-09-13T03:11:36.261 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph tell osd.6 flush_pg_stats 2023-09-13T03:11:36.261 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph tell osd.7 flush_pg_stats 2023-09-13T03:11:36.262 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph tell osd.8 flush_pg_stats 2023-09-13T03:11:36.262 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph tell osd.9 flush_pg_stats 2023-09-13T03:11:36.263 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph tell osd.10 flush_pg_stats 2023-09-13T03:11:36.263 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph tell osd.11 flush_pg_stats 2023-09-13T03:11:36.455 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:36 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/920957211' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:11:36.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:36 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/920957211' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:11:36.659 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:36 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/920957211' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:11:37.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:37 smithi142 ceph-mon[134082]: pgmap v292: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.9 KiB/s wr, 9 op/s; 5 B/s, 1 keys/s, 0 objects/s recovering 2023-09-13T03:11:37.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:37 smithi170 ceph-mon[134328]: pgmap v292: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.9 KiB/s wr, 9 op/s; 5 B/s, 1 keys/s, 0 objects/s recovering 2023-09-13T03:11:37.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:37 smithi064 ceph-mon[97675]: pgmap v292: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.9 KiB/s wr, 9 op/s; 5 B/s, 1 keys/s, 0 objects/s recovering 2023-09-13T03:11:38.054 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:11:38.195 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:11:38.232 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:11:38.264 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:11:38.430 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:11:38.449 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:11:38.592 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:11:38.592 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:11:39.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:39 smithi064 ceph-mon[97675]: pgmap v293: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.1 KiB/s wr, 12 op/s; 5 B/s, 1 keys/s, 0 objects/s recovering 2023-09-13T03:11:39.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:39 smithi170 ceph-mon[134328]: pgmap v293: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.1 KiB/s wr, 12 op/s; 5 B/s, 1 keys/s, 0 objects/s recovering 2023-09-13T03:11:39.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:39 smithi142 ceph-mon[134082]: pgmap v293: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.1 KiB/s wr, 12 op/s; 5 B/s, 1 keys/s, 0 objects/s recovering 2023-09-13T03:11:40.931 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:11:41.338 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:11:41.424 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:11:41.585 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:41 smithi064 ceph-mon[97675]: pgmap v294: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 937 B/s wr, 10 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2023-09-13T03:11:41.621 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:11:41.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:41 smithi170 ceph-mon[134328]: pgmap v294: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 937 B/s wr, 10 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2023-09-13T03:11:41.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:41 smithi142 ceph-mon[134082]: pgmap v294: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 937 B/s wr, 10 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2023-09-13T03:11:42.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.668+0000 7f5691394700 1 -- 172.21.15.64:0/3013363698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f568c100fe0 msgr2=0x7f568c10d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:42.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.668+0000 7f5691394700 1 --2- 172.21.15.64:0/3013363698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f568c100fe0 0x7f568c10d4f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f568400b0a0 tx=0x7f568400b3b0 comp rx=0 tx=0).stop 2023-09-13T03:11:42.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.668+0000 7f5691394700 1 -- 172.21.15.64:0/3013363698 shutdown_connections 2023-09-13T03:11:42.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.668+0000 7f5691394700 1 --2- 172.21.15.64:0/3013363698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f568c106620 0x7f568c106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:42.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.668+0000 7f5691394700 1 --2- 172.21.15.64:0/3013363698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f568c100fe0 0x7f568c10d4f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:42.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.668+0000 7f5691394700 1 --2- 172.21.15.64:0/3013363698 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f568c100620 0x7f568c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:42.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.668+0000 7f5691394700 1 -- 172.21.15.64:0/3013363698 >> 172.21.15.64:0/3013363698 conn(0x7f568c0fc0f0 msgr2=0x7f568c0fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:42.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.668+0000 7f5691394700 1 -- 172.21.15.64:0/3013363698 shutdown_connections 2023-09-13T03:11:42.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.668+0000 7f5691394700 1 -- 172.21.15.64:0/3013363698 wait complete. 2023-09-13T03:11:42.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.669+0000 7f5691394700 1 Processor -- start 2023-09-13T03:11:42.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.669+0000 7f5691394700 1 -- start start 2023-09-13T03:11:42.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.670+0000 7f5691394700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f568c100620 0x7f568c19e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:42.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.670+0000 7f5691394700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f568c100fe0 0x7f568c19ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:42.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.670+0000 7f5691394700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f568c106620 0x7f568c1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:42.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.670+0000 7f5691394700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568c198f20 con 0x7f568c100620 2023-09-13T03:11:42.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.670+0000 7f5691394700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568c199090 con 0x7f568c100fe0 2023-09-13T03:11:42.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.670+0000 7f5691394700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568c199250 con 0x7f568c106620 2023-09-13T03:11:42.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.670+0000 7f568affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f568c100620 0x7f568c19e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:42.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.670+0000 7f568b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f568c106620 0x7f568c1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:42.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.670+0000 7f568affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f568c100620 0x7f568c19e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39898/0 (socket says 172.21.15.64:39898) 2023-09-13T03:11:42.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.670+0000 7f568a7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f568c100fe0 0x7f568c19ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:42.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.670+0000 7f568affd700 1 -- 172.21.15.64:0/729354806 learned_addr learned my addr 172.21.15.64:0/729354806 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:42.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.670+0000 7f568b7fe700 1 -- 172.21.15.64:0/729354806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f568c100fe0 msgr2=0x7f568c19ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:42.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.670+0000 7f568b7fe700 1 --2- 172.21.15.64:0/729354806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f568c100fe0 0x7f568c19ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:42.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.670+0000 7f568b7fe700 1 -- 172.21.15.64:0/729354806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f568c100620 msgr2=0x7f568c19e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:42.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.670+0000 7f568b7fe700 1 --2- 172.21.15.64:0/729354806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f568c100620 0x7f568c19e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:42.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.670+0000 7f568b7fe700 1 -- 172.21.15.64:0/729354806 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5684009d00 con 0x7f568c106620 2023-09-13T03:11:42.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.670+0000 7f568a7fc700 1 --2- 172.21.15.64:0/729354806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f568c100fe0 0x7f568c19ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:11:42.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.670+0000 7f568affd700 1 --2- 172.21.15.64:0/729354806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f568c100620 0x7f568c19e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:11:42.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.671+0000 7f568b7fe700 1 --2- 172.21.15.64:0/729354806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f568c106620 0x7f568c1988b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5684009fd0 tx=0x7f5684007930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:42.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.671+0000 7f5673fff700 1 -- 172.21.15.64:0/729354806 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568400e040 con 0x7f568c106620 2023-09-13T03:11:42.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.671+0000 7f5673fff700 1 -- 172.21.15.64:0/729354806 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f568401d470 con 0x7f568c106620 2023-09-13T03:11:42.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.671+0000 7f5673fff700 1 -- 172.21.15.64:0/729354806 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56840052a0 con 0x7f568c106620 2023-09-13T03:11:42.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.671+0000 7f5691394700 1 -- 172.21.15.64:0/729354806 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f568c1994d0 con 0x7f568c106620 2023-09-13T03:11:42.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.671+0000 7f5691394700 1 -- 172.21.15.64:0/729354806 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f568c1a57b0 con 0x7f568c106620 2023-09-13T03:11:42.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.672+0000 7f5691394700 1 -- 172.21.15.64:0/729354806 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f568c04e300 con 0x7f568c106620 2023-09-13T03:11:42.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.674+0000 7f5673fff700 1 -- 172.21.15.64:0/729354806 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f568401d5e0 con 0x7f568c106620 2023-09-13T03:11:42.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.675+0000 7f5673fff700 1 --2- 172.21.15.64:0/729354806 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f56740a0750 0x7f56740a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:42.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.676+0000 7f5673fff700 1 -- 172.21.15.64:0/729354806 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f56840cd310 con 0x7f568c106620 2023-09-13T03:11:42.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.676+0000 7f568affd700 1 --2- 172.21.15.64:0/729354806 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f56740a0750 0x7f56740a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:42.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.676+0000 7f5673fff700 1 --2- 172.21.15.64:0/729354806 >> [v2:172.21.15.170:6824/3605017060,v1:172.21.15.170:6825/3605017060] conn(0x7f56740aa210 0x7f56740ac6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:42.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.676+0000 7f5673fff700 1 -- 172.21.15.64:0/729354806 --> [v2:172.21.15.170:6824/3605017060,v1:172.21.15.170:6825/3605017060] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f56740acd80 con 0x7f56740aa210 2023-09-13T03:11:42.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.676+0000 7f5673fff700 1 -- 172.21.15.64:0/729354806 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56840cd680 con 0x7f568c106620 2023-09-13T03:11:42.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.676+0000 7f568a7fc700 1 --2- 172.21.15.64:0/729354806 >> [v2:172.21.15.170:6824/3605017060,v1:172.21.15.170:6825/3605017060] conn(0x7f56740aa210 0x7f56740ac6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:42.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.676+0000 7f568affd700 1 --2- 172.21.15.64:0/729354806 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f56740a0750 0x7f56740a2c10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f567c011550 tx=0x7f567c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:42.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.677+0000 7f568a7fc700 1 --2- 172.21.15.64:0/729354806 >> [v2:172.21.15.170:6824/3605017060,v1:172.21.15.170:6825/3605017060] conn(0x7f56740aa210 0x7f56740ac6b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:42.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.678+0000 7f5673fff700 1 -- 172.21.15.64:0/729354806 <== osd.11 v2:172.21.15.170:6824/3605017060 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f56740acd80 con 0x7f56740aa210 2023-09-13T03:11:42.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.698+0000 7f5691394700 1 -- 172.21.15.64:0/729354806 --> [v2:172.21.15.170:6824/3605017060,v1:172.21.15.170:6825/3605017060] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f568c0fd8c0 con 0x7f56740aa210 2023-09-13T03:11:42.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.698+0000 7f5673fff700 1 -- 172.21.15.64:0/729354806 <== osd.11 v2:172.21.15.170:6824/3605017060 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f568c0fd8c0 con 0x7f56740aa210 2023-09-13T03:11:42.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.699+0000 7f5691394700 1 -- 172.21.15.64:0/729354806 >> [v2:172.21.15.170:6824/3605017060,v1:172.21.15.170:6825/3605017060] conn(0x7f56740aa210 msgr2=0x7f56740ac6b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:42.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.699+0000 7f5691394700 1 --2- 172.21.15.64:0/729354806 >> [v2:172.21.15.170:6824/3605017060,v1:172.21.15.170:6825/3605017060] conn(0x7f56740aa210 0x7f56740ac6b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:42.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.699+0000 7f5691394700 1 -- 172.21.15.64:0/729354806 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f56740a0750 msgr2=0x7f56740a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:42.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.699+0000 7f5691394700 1 --2- 172.21.15.64:0/729354806 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f56740a0750 0x7f56740a2c10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f567c011550 tx=0x7f567c009260 comp rx=0 tx=0).stop 2023-09-13T03:11:42.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.699+0000 7f5691394700 1 -- 172.21.15.64:0/729354806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f568c106620 msgr2=0x7f568c1988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:42.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.699+0000 7f5691394700 1 --2- 172.21.15.64:0/729354806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f568c106620 0x7f568c1988b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5684009fd0 tx=0x7f5684007930 comp rx=0 tx=0).stop 2023-09-13T03:11:42.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.699+0000 7f568b7fe700 1 -- 172.21.15.64:0/729354806 reap_dead start 2023-09-13T03:11:42.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.699+0000 7f5691394700 1 -- 172.21.15.64:0/729354806 shutdown_connections 2023-09-13T03:11:42.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.699+0000 7f5691394700 1 -- 172.21.15.64:0/729354806 >> 172.21.15.64:0/729354806 conn(0x7f568c0fc0f0 msgr2=0x7f568c10ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:42.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.699+0000 7f5691394700 1 -- 172.21.15.64:0/729354806 shutdown_connections 2023-09-13T03:11:42.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:42.699+0000 7f5691394700 1 -- 172.21.15.64:0/729354806 wait complete. 2023-09-13T03:11:43.524 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:43 smithi064 ceph-mon[97675]: pgmap v295: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 937 B/s wr, 9 op/s 2023-09-13T03:11:43.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.526+0000 7f834ae52700 1 -- 172.21.15.64:0/2087534580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8344068120 msgr2=0x7f83440685a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:43.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.526+0000 7f834ae52700 1 --2- 172.21.15.64:0/2087534580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8344068120 0x7f83440685a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8330009990 tx=0x7f8330009ca0 comp rx=0 tx=0).stop 2023-09-13T03:11:43.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.526+0000 7f834ae52700 1 -- 172.21.15.64:0/2087534580 shutdown_connections 2023-09-13T03:11:43.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.526+0000 7f834ae52700 1 --2- 172.21.15.64:0/2087534580 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8344068ae0 0x7f834410d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:43.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.526+0000 7f834ae52700 1 --2- 172.21.15.64:0/2087534580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8344106770 0x7f8344106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:43.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.526+0000 7f834ae52700 1 --2- 172.21.15.64:0/2087534580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8344068120 0x7f83440685a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:43.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.526+0000 7f834ae52700 1 -- 172.21.15.64:0/2087534580 >> 172.21.15.64:0/2087534580 conn(0x7f8344076600 msgr2=0x7f8344076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:43.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.526+0000 7f834ae52700 1 -- 172.21.15.64:0/2087534580 shutdown_connections 2023-09-13T03:11:43.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.527+0000 7f834ae52700 1 -- 172.21.15.64:0/2087534580 wait complete. 2023-09-13T03:11:43.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.528+0000 7f834ae52700 1 Processor -- start 2023-09-13T03:11:43.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.528+0000 7f834ae52700 1 -- start start 2023-09-13T03:11:43.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.528+0000 7f834ae52700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8344068120 0x7f834419e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:43.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.528+0000 7f834ae52700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8344068ae0 0x7f834419ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:43.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.528+0000 7f834ae52700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8344106770 0x7f83441988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:43.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.528+0000 7f834ae52700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8344198f10 con 0x7f8344068120 2023-09-13T03:11:43.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.528+0000 7f834ae52700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8344199080 con 0x7f8344106770 2023-09-13T03:11:43.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.528+0000 7f834ae52700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8344199240 con 0x7f8344068ae0 2023-09-13T03:11:43.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.529+0000 7f8348bee700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8344068120 0x7f834419e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:43.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.529+0000 7f8348bee700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8344068120 0x7f834419e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39904/0 (socket says 172.21.15.64:39904) 2023-09-13T03:11:43.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.529+0000 7f8348bee700 1 -- 172.21.15.64:0/1849628927 learned_addr learned my addr 172.21.15.64:0/1849628927 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:43.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.529+0000 7f83493ef700 1 --2- 172.21.15.64:0/1849628927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8344106770 0x7f83441988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:43.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.529+0000 7f8343fff700 1 --2- 172.21.15.64:0/1849628927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8344068ae0 0x7f834419ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:43.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.529+0000 7f8348bee700 1 -- 172.21.15.64:0/1849628927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8344068ae0 msgr2=0x7f834419ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:43.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.529+0000 7f8348bee700 1 --2- 172.21.15.64:0/1849628927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8344068ae0 0x7f834419ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:43.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.529+0000 7f8348bee700 1 -- 172.21.15.64:0/1849628927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8344106770 msgr2=0x7f83441988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:43.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.529+0000 7f8348bee700 1 --2- 172.21.15.64:0/1849628927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8344106770 0x7f83441988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:43.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.529+0000 7f8348bee700 1 -- 172.21.15.64:0/1849628927 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83300095f0 con 0x7f8344068120 2023-09-13T03:11:43.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.529+0000 7f8348bee700 1 --2- 172.21.15.64:0/1849628927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8344068120 0x7f834419e780 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f833800e9a0 tx=0x7f833800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:43.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.530+0000 7f8341ffb700 1 -- 172.21.15.64:0/1849628927 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8338009790 con 0x7f8344068120 2023-09-13T03:11:43.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.530+0000 7f8341ffb700 1 -- 172.21.15.64:0/1849628927 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8338010430 con 0x7f8344068120 2023-09-13T03:11:43.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.530+0000 7f834ae52700 1 -- 172.21.15.64:0/1849628927 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8344199520 con 0x7f8344068120 2023-09-13T03:11:43.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.530+0000 7f8341ffb700 1 -- 172.21.15.64:0/1849628927 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8338018680 con 0x7f8344068120 2023-09-13T03:11:43.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.530+0000 7f834ae52700 1 -- 172.21.15.64:0/1849628927 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83441a5770 con 0x7f8344068120 2023-09-13T03:11:43.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.531+0000 7f834ae52700 1 -- 172.21.15.64:0/1849628927 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f83441108e0 con 0x7f8344068120 2023-09-13T03:11:43.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.533+0000 7f8341ffb700 1 -- 172.21.15.64:0/1849628927 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f83380105a0 con 0x7f8344068120 2023-09-13T03:11:43.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.534+0000 7f8341ffb700 1 --2- 172.21.15.64:0/1849628927 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f83340a0750 0x7f83340a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:43.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.534+0000 7f8341ffb700 1 -- 172.21.15.64:0/1849628927 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f8338014070 con 0x7f8344068120 2023-09-13T03:11:43.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.534+0000 7f8343fff700 1 --2- 172.21.15.64:0/1849628927 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f83340a0750 0x7f83340a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:43.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.534+0000 7f8341ffb700 1 --2- 172.21.15.64:0/1849628927 >> [v2:172.21.15.64:6802/3940892135,v1:172.21.15.64:6803/3940892135] conn(0x7f83340aa210 0x7f83340ac6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:43.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.534+0000 7f8341ffb700 1 -- 172.21.15.64:0/1849628927 --> [v2:172.21.15.64:6802/3940892135,v1:172.21.15.64:6803/3940892135] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f83340acd80 con 0x7f83340aa210 2023-09-13T03:11:43.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.535+0000 7f8341ffb700 1 -- 172.21.15.64:0/1849628927 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83380cc500 con 0x7f8344068120 2023-09-13T03:11:43.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.535+0000 7f83493ef700 1 --2- 172.21.15.64:0/1849628927 >> [v2:172.21.15.64:6802/3940892135,v1:172.21.15.64:6803/3940892135] conn(0x7f83340aa210 0x7f83340ac6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:43.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.535+0000 7f8343fff700 1 --2- 172.21.15.64:0/1849628927 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f83340a0750 0x7f83340a2c10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f83440fdcf0 tx=0x7f8330005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:43.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.535+0000 7f83493ef700 1 --2- 172.21.15.64:0/1849628927 >> [v2:172.21.15.64:6802/3940892135,v1:172.21.15.64:6803/3940892135] conn(0x7f83340aa210 0x7f83340ac6b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:43.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.536+0000 7f8341ffb700 1 -- 172.21.15.64:0/1849628927 <== osd.0 v2:172.21.15.64:6802/3940892135 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f83340acd80 con 0x7f83340aa210 2023-09-13T03:11:43.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.553+0000 7f834ae52700 1 -- 172.21.15.64:0/1849628927 --> [v2:172.21.15.64:6802/3940892135,v1:172.21.15.64:6803/3940892135] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f834404e300 con 0x7f83340aa210 2023-09-13T03:11:43.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.553+0000 7f8341ffb700 1 -- 172.21.15.64:0/1849628927 <== osd.0 v2:172.21.15.64:6802/3940892135 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f834404e300 con 0x7f83340aa210 2023-09-13T03:11:43.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.554+0000 7f834ae52700 1 -- 172.21.15.64:0/1849628927 >> [v2:172.21.15.64:6802/3940892135,v1:172.21.15.64:6803/3940892135] conn(0x7f83340aa210 msgr2=0x7f83340ac6b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:43.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.554+0000 7f834ae52700 1 --2- 172.21.15.64:0/1849628927 >> [v2:172.21.15.64:6802/3940892135,v1:172.21.15.64:6803/3940892135] conn(0x7f83340aa210 0x7f83340ac6b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:43.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.554+0000 7f834ae52700 1 -- 172.21.15.64:0/1849628927 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f83340a0750 msgr2=0x7f83340a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:43.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.554+0000 7f834ae52700 1 --2- 172.21.15.64:0/1849628927 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f83340a0750 0x7f83340a2c10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f83440fdcf0 tx=0x7f8330005fd0 comp rx=0 tx=0).stop 2023-09-13T03:11:43.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.554+0000 7f834ae52700 1 -- 172.21.15.64:0/1849628927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8344068120 msgr2=0x7f834419e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:43.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.554+0000 7f834ae52700 1 --2- 172.21.15.64:0/1849628927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8344068120 0x7f834419e780 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f833800e9a0 tx=0x7f833800ecb0 comp rx=0 tx=0).stop 2023-09-13T03:11:43.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.554+0000 7f83493ef700 1 -- 172.21.15.64:0/1849628927 reap_dead start 2023-09-13T03:11:43.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.554+0000 7f834ae52700 1 -- 172.21.15.64:0/1849628927 shutdown_connections 2023-09-13T03:11:43.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.554+0000 7f834ae52700 1 -- 172.21.15.64:0/1849628927 >> 172.21.15.64:0/1849628927 conn(0x7f8344076600 msgr2=0x7f83440fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:43.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.554+0000 7f834ae52700 1 -- 172.21.15.64:0/1849628927 shutdown_connections 2023-09-13T03:11:43.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:43.554+0000 7f834ae52700 1 -- 172.21.15.64:0/1849628927 wait complete. 2023-09-13T03:11:43.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:43 smithi170 ceph-mon[134328]: pgmap v295: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 937 B/s wr, 9 op/s 2023-09-13T03:11:43.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:43 smithi142 ceph-mon[134082]: pgmap v295: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 937 B/s wr, 9 op/s 2023-09-13T03:11:44.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.503+0000 7f43f4a1b700 1 -- 172.21.15.64:0/2236147767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43f0104810 msgr2=0x7f43f0104cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:44.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.503+0000 7f43f4a1b700 1 --2- 172.21.15.64:0/2236147767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43f0104810 0x7f43f0104cc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f43e800b0a0 tx=0x7f43e800b3b0 comp rx=0 tx=0).stop 2023-09-13T03:11:44.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.503+0000 7f43f4a1b700 1 -- 172.21.15.64:0/2236147767 shutdown_connections 2023-09-13T03:11:44.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.503+0000 7f43f4a1b700 1 --2- 172.21.15.64:0/2236147767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43f00ff400 0x7f43f00ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:44.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.503+0000 7f43f4a1b700 1 --2- 172.21.15.64:0/2236147767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43f0104810 0x7f43f0104cc0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:44.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.503+0000 7f43f4a1b700 1 --2- 172.21.15.64:0/2236147767 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43f00ffd20 0x7f43f01041f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:44.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.503+0000 7f43f4a1b700 1 -- 172.21.15.64:0/2236147767 >> 172.21.15.64:0/2236147767 conn(0x7f43f00faea0 msgr2=0x7f43f00fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:44.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.503+0000 7f43f4a1b700 1 -- 172.21.15.64:0/2236147767 shutdown_connections 2023-09-13T03:11:44.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.503+0000 7f43f4a1b700 1 -- 172.21.15.64:0/2236147767 wait complete. 2023-09-13T03:11:44.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.505+0000 7f43f4a1b700 1 Processor -- start 2023-09-13T03:11:44.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.505+0000 7f43f4a1b700 1 -- start start 2023-09-13T03:11:44.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.506+0000 7f43f4a1b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43f00ff400 0x7f43f0071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:44.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.506+0000 7f43f4a1b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43f00ffd20 0x7f43f0072000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:44.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.506+0000 7f43f4a1b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43f0104810 0x7f43f006aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:44.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.506+0000 7f43f4a1b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f006b1b0 con 0x7f43f00ff400 2023-09-13T03:11:44.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.506+0000 7f43f4a1b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f006b320 con 0x7f43f0104810 2023-09-13T03:11:44.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.506+0000 7f43f4a1b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f006b4e0 con 0x7f43f00ffd20 2023-09-13T03:11:44.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.506+0000 7f43eeffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43f00ffd20 0x7f43f0072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:44.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.506+0000 7f43ef7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43f00ff400 0x7f43f0071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:44.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.506+0000 7f43eeffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43f00ffd20 0x7f43f0072000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:58264/0 (socket says 172.21.15.64:58264) 2023-09-13T03:11:44.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.506+0000 7f43eeffd700 1 -- 172.21.15.64:0/1488041086 learned_addr learned my addr 172.21.15.64:0/1488041086 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:44.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.506+0000 7f43effff700 1 --2- 172.21.15.64:0/1488041086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43f0104810 0x7f43f006aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:44.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.507+0000 7f43eeffd700 1 -- 172.21.15.64:0/1488041086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43f0104810 msgr2=0x7f43f006aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:44.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.507+0000 7f43eeffd700 1 --2- 172.21.15.64:0/1488041086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43f0104810 0x7f43f006aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:44.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.507+0000 7f43eeffd700 1 -- 172.21.15.64:0/1488041086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43f00ff400 msgr2=0x7f43f0071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:44.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.507+0000 7f43eeffd700 1 --2- 172.21.15.64:0/1488041086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43f00ff400 0x7f43f0071a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:44.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.507+0000 7f43eeffd700 1 -- 172.21.15.64:0/1488041086 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e8009d00 con 0x7f43f00ffd20 2023-09-13T03:11:44.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.507+0000 7f43eeffd700 1 --2- 172.21.15.64:0/1488041086 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43f00ffd20 0x7f43f0072000 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f43e400e9c0 tx=0x7f43e400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:44.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.508+0000 7f43ecff9700 1 -- 172.21.15.64:0/1488041086 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e40096e0 con 0x7f43f00ffd20 2023-09-13T03:11:44.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.508+0000 7f43ecff9700 1 -- 172.21.15.64:0/1488041086 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43e400c7c0 con 0x7f43f00ffd20 2023-09-13T03:11:44.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.508+0000 7f43f4a1b700 1 -- 172.21.15.64:0/1488041086 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43f006b7c0 con 0x7f43f00ffd20 2023-09-13T03:11:44.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.508+0000 7f43ecff9700 1 -- 172.21.15.64:0/1488041086 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e4005410 con 0x7f43f00ffd20 2023-09-13T03:11:44.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.508+0000 7f43f4a1b700 1 -- 172.21.15.64:0/1488041086 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43f006f8a0 con 0x7f43f00ffd20 2023-09-13T03:11:44.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.509+0000 7f43f4a1b700 1 -- 172.21.15.64:0/1488041086 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f43f0116cd0 con 0x7f43f00ffd20 2023-09-13T03:11:44.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.513+0000 7f43ecff9700 1 -- 172.21.15.64:0/1488041086 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f43e400c930 con 0x7f43f00ffd20 2023-09-13T03:11:44.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.515+0000 7f43ecff9700 1 --2- 172.21.15.64:0/1488041086 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f43d80a0750 0x7f43d80a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:44.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.515+0000 7f43ecff9700 1 -- 172.21.15.64:0/1488041086 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f43e4014070 con 0x7f43f00ffd20 2023-09-13T03:11:44.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.515+0000 7f43ef7fe700 1 --2- 172.21.15.64:0/1488041086 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f43d80a0750 0x7f43d80a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:44.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.515+0000 7f43ecff9700 1 --2- 172.21.15.64:0/1488041086 >> [v2:172.21.15.170:6800/2406669338,v1:172.21.15.170:6801/2406669338] conn(0x7f43d80aa210 0x7f43d80ac6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:44.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.515+0000 7f43ecff9700 1 -- 172.21.15.64:0/1488041086 --> [v2:172.21.15.170:6800/2406669338,v1:172.21.15.170:6801/2406669338] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f43d80acd80 con 0x7f43d80aa210 2023-09-13T03:11:44.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.515+0000 7f43ecff9700 1 -- 172.21.15.64:0/1488041086 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43e40cc4f0 con 0x7f43f00ffd20 2023-09-13T03:11:44.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.516+0000 7f43effff700 1 --2- 172.21.15.64:0/1488041086 >> [v2:172.21.15.170:6800/2406669338,v1:172.21.15.170:6801/2406669338] conn(0x7f43d80aa210 0x7f43d80ac6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:44.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.516+0000 7f43ef7fe700 1 --2- 172.21.15.64:0/1488041086 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f43d80a0750 0x7f43d80a2c10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f43e0005950 tx=0x7f43e000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:44.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.516+0000 7f43effff700 1 --2- 172.21.15.64:0/1488041086 >> [v2:172.21.15.170:6800/2406669338,v1:172.21.15.170:6801/2406669338] conn(0x7f43d80aa210 0x7f43d80ac6b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:44.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.517+0000 7f43ecff9700 1 -- 172.21.15.64:0/1488041086 <== osd.2 v2:172.21.15.170:6800/2406669338 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f43d80acd80 con 0x7f43d80aa210 2023-09-13T03:11:44.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.536+0000 7f43f4a1b700 1 -- 172.21.15.64:0/1488041086 --> [v2:172.21.15.170:6800/2406669338,v1:172.21.15.170:6801/2406669338] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f43f004eb50 con 0x7f43d80aa210 2023-09-13T03:11:44.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.536+0000 7f43ecff9700 1 -- 172.21.15.64:0/1488041086 <== osd.2 v2:172.21.15.170:6800/2406669338 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f43f004eb50 con 0x7f43d80aa210 2023-09-13T03:11:44.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.537+0000 7f43f4a1b700 1 -- 172.21.15.64:0/1488041086 >> [v2:172.21.15.170:6800/2406669338,v1:172.21.15.170:6801/2406669338] conn(0x7f43d80aa210 msgr2=0x7f43d80ac6b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:44.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.537+0000 7f43f4a1b700 1 --2- 172.21.15.64:0/1488041086 >> [v2:172.21.15.170:6800/2406669338,v1:172.21.15.170:6801/2406669338] conn(0x7f43d80aa210 0x7f43d80ac6b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:44.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.537+0000 7f43f4a1b700 1 -- 172.21.15.64:0/1488041086 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f43d80a0750 msgr2=0x7f43d80a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:44.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.537+0000 7f43f4a1b700 1 --2- 172.21.15.64:0/1488041086 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f43d80a0750 0x7f43d80a2c10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f43e0005950 tx=0x7f43e000b410 comp rx=0 tx=0).stop 2023-09-13T03:11:44.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.537+0000 7f43f4a1b700 1 -- 172.21.15.64:0/1488041086 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43f00ffd20 msgr2=0x7f43f0072000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:44.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.537+0000 7f43f4a1b700 1 --2- 172.21.15.64:0/1488041086 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43f00ffd20 0x7f43f0072000 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f43e400e9c0 tx=0x7f43e400ecd0 comp rx=0 tx=0).stop 2023-09-13T03:11:44.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.537+0000 7f43effff700 1 -- 172.21.15.64:0/1488041086 reap_dead start 2023-09-13T03:11:44.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.537+0000 7f43f4a1b700 1 -- 172.21.15.64:0/1488041086 shutdown_connections 2023-09-13T03:11:44.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.537+0000 7f43f4a1b700 1 -- 172.21.15.64:0/1488041086 >> 172.21.15.64:0/1488041086 conn(0x7f43f00faea0 msgr2=0x7f43f0102230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:44.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.537+0000 7f43f4a1b700 1 -- 172.21.15.64:0/1488041086 shutdown_connections 2023-09-13T03:11:44.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:44.537+0000 7f43f4a1b700 1 -- 172.21.15.64:0/1488041086 wait complete. 2023-09-13T03:11:44.592 INFO:teuthology.orchestra.run.smithi064.stdout:34359738436 2023-09-13T03:11:44.593 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.0 2023-09-13T03:11:44.910 INFO:teuthology.orchestra.run.smithi064.stdout:335007449106 2023-09-13T03:11:44.910 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.11 2023-09-13T03:11:45.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.093+0000 7f094af51700 1 -- 172.21.15.64:0/872595199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0944068120 msgr2=0x7f09440685a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:45.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.093+0000 7f094af51700 1 --2- 172.21.15.64:0/872595199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0944068120 0x7f09440685a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0930009990 tx=0x7f0930009ca0 comp rx=0 tx=0).stop 2023-09-13T03:11:45.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.093+0000 7f094af51700 1 -- 172.21.15.64:0/872595199 shutdown_connections 2023-09-13T03:11:45.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.093+0000 7f094af51700 1 --2- 172.21.15.64:0/872595199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0944068ae0 0x7f094410d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:45.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.093+0000 7f094af51700 1 --2- 172.21.15.64:0/872595199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0944106770 0x7f0944106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:45.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.093+0000 7f094af51700 1 --2- 172.21.15.64:0/872595199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0944068120 0x7f09440685a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:45.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.093+0000 7f094af51700 1 -- 172.21.15.64:0/872595199 >> 172.21.15.64:0/872595199 conn(0x7f0944076600 msgr2=0x7f0944076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:45.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.094+0000 7f094af51700 1 -- 172.21.15.64:0/872595199 shutdown_connections 2023-09-13T03:11:45.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.094+0000 7f094af51700 1 -- 172.21.15.64:0/872595199 wait complete. 2023-09-13T03:11:45.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.095+0000 7f094af51700 1 Processor -- start 2023-09-13T03:11:45.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.095+0000 7f094af51700 1 -- start start 2023-09-13T03:11:45.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.096+0000 7f094af51700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0944068120 0x7f09440ff5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:45.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.096+0000 7f094af51700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0944068ae0 0x7f09440ffaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:45.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.096+0000 7f094af51700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0944106770 0x7f0944100070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:45.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.096+0000 7f094af51700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09441a4ba0 con 0x7f0944068120 2023-09-13T03:11:45.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.096+0000 7f094af51700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09441a4d10 con 0x7f0944068ae0 2023-09-13T03:11:45.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.096+0000 7f094af51700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09441a4ed0 con 0x7f0944106770 2023-09-13T03:11:45.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.096+0000 7f09494ee700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0944106770 0x7f0944100070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:45.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.096+0000 7f0948ced700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0944068120 0x7f09440ff5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:45.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.096+0000 7f0948ced700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0944068120 0x7f09440ff5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39946/0 (socket says 172.21.15.64:39946) 2023-09-13T03:11:45.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.096+0000 7f0948ced700 1 -- 172.21.15.64:0/4145031956 learned_addr learned my addr 172.21.15.64:0/4145031956 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:45.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.096+0000 7f0943fff700 1 --2- 172.21.15.64:0/4145031956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0944068ae0 0x7f09440ffaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:45.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.097+0000 7f09494ee700 1 -- 172.21.15.64:0/4145031956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0944068ae0 msgr2=0x7f09440ffaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:45.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.097+0000 7f09494ee700 1 --2- 172.21.15.64:0/4145031956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0944068ae0 0x7f09440ffaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:45.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.097+0000 7f09494ee700 1 -- 172.21.15.64:0/4145031956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0944068120 msgr2=0x7f09440ff5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:45.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.097+0000 7f09494ee700 1 --2- 172.21.15.64:0/4145031956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0944068120 0x7f09440ff5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:45.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.097+0000 7f09494ee700 1 -- 172.21.15.64:0/4145031956 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09300095f0 con 0x7f0944106770 2023-09-13T03:11:45.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.097+0000 7f0948ced700 1 --2- 172.21.15.64:0/4145031956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0944068120 0x7f09440ff5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:11:45.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.097+0000 7f09494ee700 1 --2- 172.21.15.64:0/4145031956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0944106770 0x7f0944100070 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f093c00efc0 tx=0x7f093c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:45.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.097+0000 7f0941ffb700 1 -- 172.21.15.64:0/4145031956 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093c018070 con 0x7f0944106770 2023-09-13T03:11:45.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.097+0000 7f0941ffb700 1 -- 172.21.15.64:0/4145031956 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f093c00f040 con 0x7f0944106770 2023-09-13T03:11:45.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.097+0000 7f0941ffb700 1 -- 172.21.15.64:0/4145031956 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093c00cc80 con 0x7f0944106770 2023-09-13T03:11:45.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.097+0000 7f094af51700 1 -- 172.21.15.64:0/4145031956 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09441a5150 con 0x7f0944106770 2023-09-13T03:11:45.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.098+0000 7f094af51700 1 -- 172.21.15.64:0/4145031956 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09441a5650 con 0x7f0944106770 2023-09-13T03:11:45.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.098+0000 7f094af51700 1 -- 172.21.15.64:0/4145031956 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f09441108e0 con 0x7f0944106770 2023-09-13T03:11:45.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.101+0000 7f0941ffb700 1 -- 172.21.15.64:0/4145031956 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f093c003a00 con 0x7f0944106770 2023-09-13T03:11:45.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.102+0000 7f0941ffb700 1 --2- 172.21.15.64:0/4145031956 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f09340a07d0 0x7f09340a2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:45.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.102+0000 7f0941ffb700 1 -- 172.21.15.64:0/4145031956 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f093c0cbec0 con 0x7f0944106770 2023-09-13T03:11:45.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.102+0000 7f0948ced700 1 --2- 172.21.15.64:0/4145031956 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f09340a07d0 0x7f09340a2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:45.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.102+0000 7f0941ffb700 1 --2- 172.21.15.64:0/4145031956 >> [v2:172.21.15.142:6824/936499217,v1:172.21.15.142:6825/936499217] conn(0x7f09340aa290 0x7f09340ac730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:45.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.103+0000 7f0941ffb700 1 -- 172.21.15.64:0/4145031956 --> [v2:172.21.15.142:6824/936499217,v1:172.21.15.142:6825/936499217] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f09340ace00 con 0x7f09340aa290 2023-09-13T03:11:45.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.103+0000 7f0941ffb700 1 -- 172.21.15.64:0/4145031956 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f093c0cc270 con 0x7f0944106770 2023-09-13T03:11:45.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.103+0000 7f0943fff700 1 --2- 172.21.15.64:0/4145031956 >> [v2:172.21.15.142:6824/936499217,v1:172.21.15.142:6825/936499217] conn(0x7f09340aa290 0x7f09340ac730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:45.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.103+0000 7f0948ced700 1 --2- 172.21.15.64:0/4145031956 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f09340a07d0 0x7f09340a2c90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0938011550 tx=0x7f0938009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:45.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.104+0000 7f0943fff700 1 --2- 172.21.15.64:0/4145031956 >> [v2:172.21.15.142:6824/936499217,v1:172.21.15.142:6825/936499217] conn(0x7f09340aa290 0x7f09340ac730 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:45.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.105+0000 7f0941ffb700 1 -- 172.21.15.64:0/4145031956 <== osd.10 v2:172.21.15.142:6824/936499217 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f09340ace00 con 0x7f09340aa290 2023-09-13T03:11:45.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.122+0000 7f094af51700 1 -- 172.21.15.64:0/4145031956 --> [v2:172.21.15.142:6824/936499217,v1:172.21.15.142:6825/936499217] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f094404e300 con 0x7f09340aa290 2023-09-13T03:11:45.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.122+0000 7f0941ffb700 1 -- 172.21.15.64:0/4145031956 <== osd.10 v2:172.21.15.142:6824/936499217 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f094404e300 con 0x7f09340aa290 2023-09-13T03:11:45.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.123+0000 7f094af51700 1 -- 172.21.15.64:0/4145031956 >> [v2:172.21.15.142:6824/936499217,v1:172.21.15.142:6825/936499217] conn(0x7f09340aa290 msgr2=0x7f09340ac730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:45.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.123+0000 7f094af51700 1 --2- 172.21.15.64:0/4145031956 >> [v2:172.21.15.142:6824/936499217,v1:172.21.15.142:6825/936499217] conn(0x7f09340aa290 0x7f09340ac730 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:45.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.123+0000 7f094af51700 1 -- 172.21.15.64:0/4145031956 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f09340a07d0 msgr2=0x7f09340a2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:45.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.123+0000 7f094af51700 1 --2- 172.21.15.64:0/4145031956 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f09340a07d0 0x7f09340a2c90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0938011550 tx=0x7f0938009260 comp rx=0 tx=0).stop 2023-09-13T03:11:45.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.123+0000 7f094af51700 1 -- 172.21.15.64:0/4145031956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0944106770 msgr2=0x7f0944100070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:45.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.123+0000 7f094af51700 1 --2- 172.21.15.64:0/4145031956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0944106770 0x7f0944100070 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f093c00efc0 tx=0x7f093c00c3a0 comp rx=0 tx=0).stop 2023-09-13T03:11:45.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.123+0000 7f09494ee700 1 -- 172.21.15.64:0/4145031956 reap_dead start 2023-09-13T03:11:45.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.123+0000 7f094af51700 1 -- 172.21.15.64:0/4145031956 shutdown_connections 2023-09-13T03:11:45.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.123+0000 7f094af51700 1 -- 172.21.15.64:0/4145031956 >> 172.21.15.64:0/4145031956 conn(0x7f0944076600 msgr2=0x7f09440fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:45.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.123+0000 7f094af51700 1 -- 172.21.15.64:0/4145031956 shutdown_connections 2023-09-13T03:11:45.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.123+0000 7f094af51700 1 -- 172.21.15.64:0/4145031956 wait complete. 2023-09-13T03:11:45.286 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:45 smithi064 ceph-mon[97675]: pgmap v296: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 937 B/s wr, 11 op/s 2023-09-13T03:11:45.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.326+0000 7f3fc17c3700 1 -- 172.21.15.64:0/3254507967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fbc102870 msgr2=0x7f3fbc102cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:45.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.326+0000 7f3fc17c3700 1 --2- 172.21.15.64:0/3254507967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fbc102870 0x7f3fbc102cf0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0009990 tx=0x7f3fb0009ca0 comp rx=0 tx=0).stop 2023-09-13T03:11:45.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.326+0000 7f3fc17c3700 1 -- 172.21.15.64:0/3254507967 shutdown_connections 2023-09-13T03:11:45.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.326+0000 7f3fc17c3700 1 --2- 172.21.15.64:0/3254507967 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fbc103230 0x7f3fbc10f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:45.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.326+0000 7f3fc17c3700 1 --2- 172.21.15.64:0/3254507967 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3fbc108870 0x7f3fbc108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:45.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.326+0000 7f3fc17c3700 1 --2- 172.21.15.64:0/3254507967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fbc102870 0x7f3fbc102cf0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:45.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.326+0000 7f3fc17c3700 1 -- 172.21.15.64:0/3254507967 >> 172.21.15.64:0/3254507967 conn(0x7f3fbc0fe340 msgr2=0x7f3fbc100760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:45.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.327+0000 7f3fc17c3700 1 -- 172.21.15.64:0/3254507967 shutdown_connections 2023-09-13T03:11:45.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.327+0000 7f3fc17c3700 1 -- 172.21.15.64:0/3254507967 wait complete. 2023-09-13T03:11:45.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.328+0000 7f3fc17c3700 1 Processor -- start 2023-09-13T03:11:45.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.328+0000 7f3fc17c3700 1 -- start start 2023-09-13T03:11:45.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.328+0000 7f3fc17c3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3fbc102870 0x7f3fbc10daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:45.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.328+0000 7f3fc17c3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fbc103230 0x7f3fbc10e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:45.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.329+0000 7f3fc17c3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fbc108870 0x7f3fbc10e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:45.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.329+0000 7f3fc17c3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc1a9000 con 0x7f3fbc108870 2023-09-13T03:11:45.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.329+0000 7f3fc17c3700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc1a9170 con 0x7f3fbc103230 2023-09-13T03:11:45.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.329+0000 7f3fc17c3700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc1a9330 con 0x7f3fbc102870 2023-09-13T03:11:45.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.329+0000 7f3fbaffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3fbc102870 0x7f3fbc10daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:45.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.329+0000 7f3fba7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fbc103230 0x7f3fbc10e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:45.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.329+0000 7f3fba7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fbc103230 0x7f3fbc10e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:40950/0 (socket says 172.21.15.64:40950) 2023-09-13T03:11:45.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.329+0000 7f3fbaffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3fbc102870 0x7f3fbc10daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:58296/0 (socket says 172.21.15.64:58296) 2023-09-13T03:11:45.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.329+0000 7f3fba7fc700 1 -- 172.21.15.64:0/1555898296 learned_addr learned my addr 172.21.15.64:0/1555898296 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:45.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.329+0000 7f3fbb7fe700 1 --2- 172.21.15.64:0/1555898296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fbc108870 0x7f3fbc10e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:45.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.329+0000 7f3fbaffd700 1 -- 172.21.15.64:0/1555898296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fbc103230 msgr2=0x7f3fbc10e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:45.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.329+0000 7f3fbaffd700 1 --2- 172.21.15.64:0/1555898296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fbc103230 0x7f3fbc10e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:45.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.329+0000 7f3fbaffd700 1 -- 172.21.15.64:0/1555898296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fbc108870 msgr2=0x7f3fbc10e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:45.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.329+0000 7f3fbaffd700 1 --2- 172.21.15.64:0/1555898296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fbc108870 0x7f3fbc10e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:45.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.329+0000 7f3fbaffd700 1 -- 172.21.15.64:0/1555898296 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb00095f0 con 0x7f3fbc102870 2023-09-13T03:11:45.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.330+0000 7f3fbaffd700 1 --2- 172.21.15.64:0/1555898296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3fbc102870 0x7f3fbc10daf0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3fac00e9f0 tx=0x7f3fac00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:45.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.330+0000 7f3fa3fff700 1 -- 172.21.15.64:0/1555898296 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fac00c9c0 con 0x7f3fbc102870 2023-09-13T03:11:45.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.330+0000 7f3fc17c3700 1 -- 172.21.15.64:0/1555898296 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fbc1a9610 con 0x7f3fbc102870 2023-09-13T03:11:45.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.330+0000 7f3fa3fff700 1 -- 172.21.15.64:0/1555898296 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fac010430 con 0x7f3fbc102870 2023-09-13T03:11:45.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.330+0000 7f3fa3fff700 1 -- 172.21.15.64:0/1555898296 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fac018680 con 0x7f3fbc102870 2023-09-13T03:11:45.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.330+0000 7f3fc17c3700 1 -- 172.21.15.64:0/1555898296 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fbc1a9b60 con 0x7f3fbc102870 2023-09-13T03:11:45.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.332+0000 7f3fc17c3700 1 -- 172.21.15.64:0/1555898296 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3fbc04e300 con 0x7f3fbc102870 2023-09-13T03:11:45.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.334+0000 7f3fa3fff700 1 -- 172.21.15.64:0/1555898296 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f3fac00cb20 con 0x7f3fbc102870 2023-09-13T03:11:45.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.336+0000 7f3fa3fff700 1 --2- 172.21.15.64:0/1555898296 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f3fa40a0750 0x7f3fa40a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:45.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.336+0000 7f3fa3fff700 1 -- 172.21.15.64:0/1555898296 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f3fac014070 con 0x7f3fbc102870 2023-09-13T03:11:45.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.336+0000 7f3fba7fc700 1 --2- 172.21.15.64:0/1555898296 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f3fa40a0750 0x7f3fa40a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:45.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.336+0000 7f3fa3fff700 1 --2- 172.21.15.64:0/1555898296 >> [v2:172.21.15.64:6810/3558318707,v1:172.21.15.64:6811/3558318707] conn(0x7f3fa40aa210 0x7f3fa40ac6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:45.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.336+0000 7f3fa3fff700 1 -- 172.21.15.64:0/1555898296 --> [v2:172.21.15.64:6810/3558318707,v1:172.21.15.64:6811/3558318707] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3fa40acd80 con 0x7f3fa40aa210 2023-09-13T03:11:45.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.336+0000 7f3fa3fff700 1 -- 172.21.15.64:0/1555898296 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fac0d02a0 con 0x7f3fbc102870 2023-09-13T03:11:45.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.336+0000 7f3fbb7fe700 1 --2- 172.21.15.64:0/1555898296 >> [v2:172.21.15.64:6810/3558318707,v1:172.21.15.64:6811/3558318707] conn(0x7f3fa40aa210 0x7f3fa40ac6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:45.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.337+0000 7f3fba7fc700 1 --2- 172.21.15.64:0/1555898296 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f3fa40a0750 0x7f3fa40a2c10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc10f2e0 tx=0x7f3fb000b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:45.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.337+0000 7f3fbb7fe700 1 --2- 172.21.15.64:0/1555898296 >> [v2:172.21.15.64:6810/3558318707,v1:172.21.15.64:6811/3558318707] conn(0x7f3fa40aa210 0x7f3fa40ac6b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:45.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.339+0000 7f3fa3fff700 1 -- 172.21.15.64:0/1555898296 <== osd.3 v2:172.21.15.64:6810/3558318707 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f3fa40acd80 con 0x7f3fa40aa210 2023-09-13T03:11:45.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.357+0000 7f3fc17c3700 1 -- 172.21.15.64:0/1555898296 --> [v2:172.21.15.64:6810/3558318707,v1:172.21.15.64:6811/3558318707] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3fbc075d60 con 0x7f3fa40aa210 2023-09-13T03:11:45.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.358+0000 7f3fa3fff700 1 -- 172.21.15.64:0/1555898296 <== osd.3 v2:172.21.15.64:6810/3558318707 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3fb400f040 con 0x7f3fa40aa210 2023-09-13T03:11:45.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.359+0000 7f3fc17c3700 1 -- 172.21.15.64:0/1555898296 >> [v2:172.21.15.64:6810/3558318707,v1:172.21.15.64:6811/3558318707] conn(0x7f3fa40aa210 msgr2=0x7f3fa40ac6b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:45.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.359+0000 7f3fc17c3700 1 --2- 172.21.15.64:0/1555898296 >> [v2:172.21.15.64:6810/3558318707,v1:172.21.15.64:6811/3558318707] conn(0x7f3fa40aa210 0x7f3fa40ac6b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:45.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.359+0000 7f3fc17c3700 1 -- 172.21.15.64:0/1555898296 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f3fa40a0750 msgr2=0x7f3fa40a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:45.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.359+0000 7f3fc17c3700 1 --2- 172.21.15.64:0/1555898296 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f3fa40a0750 0x7f3fa40a2c10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc10f2e0 tx=0x7f3fb000b260 comp rx=0 tx=0).stop 2023-09-13T03:11:45.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.359+0000 7f3fc17c3700 1 -- 172.21.15.64:0/1555898296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3fbc102870 msgr2=0x7f3fbc10daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:45.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.359+0000 7f3fc17c3700 1 --2- 172.21.15.64:0/1555898296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3fbc102870 0x7f3fbc10daf0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3fac00e9f0 tx=0x7f3fac00ed00 comp rx=0 tx=0).stop 2023-09-13T03:11:45.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.359+0000 7f3fbb7fe700 1 -- 172.21.15.64:0/1555898296 reap_dead start 2023-09-13T03:11:45.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.359+0000 7f3fc17c3700 1 -- 172.21.15.64:0/1555898296 shutdown_connections 2023-09-13T03:11:45.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.359+0000 7f3fc17c3700 1 -- 172.21.15.64:0/1555898296 >> 172.21.15.64:0/1555898296 conn(0x7f3fbc0fe340 msgr2=0x7f3fbc1002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:45.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.359+0000 7f3fc17c3700 1 -- 172.21.15.64:0/1555898296 shutdown_connections 2023-09-13T03:11:45.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.359+0000 7f3fc17c3700 1 -- 172.21.15.64:0/1555898296 wait complete. 2023-09-13T03:11:45.430 INFO:teuthology.orchestra.run.smithi064.stdout:77309411387 2023-09-13T03:11:45.430 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.2 2023-09-13T03:11:45.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:45 smithi142 ceph-mon[134082]: pgmap v296: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 937 B/s wr, 11 op/s 2023-09-13T03:11:45.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:45 smithi170 ceph-mon[134328]: pgmap v296: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 937 B/s wr, 11 op/s 2023-09-13T03:11:45.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.674+0000 7fca97f8b700 1 -- 172.21.15.64:0/2101874263 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca90100fe0 msgr2=0x7fca9010d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:45.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.674+0000 7fca97f8b700 1 --2- 172.21.15.64:0/2101874263 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca90100fe0 0x7fca9010d4f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fca88009f90 tx=0x7fca8800b330 comp rx=0 tx=0).stop 2023-09-13T03:11:45.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.674+0000 7fca97f8b700 1 -- 172.21.15.64:0/2101874263 shutdown_connections 2023-09-13T03:11:45.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.674+0000 7fca97f8b700 1 --2- 172.21.15.64:0/2101874263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca90106620 0x7fca90106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:45.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.674+0000 7fca97f8b700 1 --2- 172.21.15.64:0/2101874263 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca90100fe0 0x7fca9010d4f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:45.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.674+0000 7fca97f8b700 1 --2- 172.21.15.64:0/2101874263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca90100620 0x7fca90100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:45.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.674+0000 7fca97f8b700 1 -- 172.21.15.64:0/2101874263 >> 172.21.15.64:0/2101874263 conn(0x7fca900fc0f0 msgr2=0x7fca900fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:45.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.674+0000 7fca97f8b700 1 -- 172.21.15.64:0/2101874263 shutdown_connections 2023-09-13T03:11:45.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.675+0000 7fca97f8b700 1 -- 172.21.15.64:0/2101874263 wait complete. 2023-09-13T03:11:45.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.676+0000 7fca97f8b700 1 Processor -- start 2023-09-13T03:11:45.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.676+0000 7fca97f8b700 1 -- start start 2023-09-13T03:11:45.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.677+0000 7fca97f8b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca90100620 0x7fca9019c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:45.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.677+0000 7fca97f8b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca90100fe0 0x7fca9019cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:45.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.677+0000 7fca97f8b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca90106620 0x7fca901966a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:45.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.677+0000 7fca97f8b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca90196d10 con 0x7fca90100620 2023-09-13T03:11:45.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.677+0000 7fca97f8b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca90196e80 con 0x7fca90100fe0 2023-09-13T03:11:45.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.677+0000 7fca97f8b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca90197040 con 0x7fca90106620 2023-09-13T03:11:45.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.678+0000 7fca95d27700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca90100620 0x7fca9019c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:45.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.678+0000 7fca95d27700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca90100620 0x7fca9019c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39984/0 (socket says 172.21.15.64:39984) 2023-09-13T03:11:45.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.678+0000 7fca95d27700 1 -- 172.21.15.64:0/4191797210 learned_addr learned my addr 172.21.15.64:0/4191797210 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:45.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.678+0000 7fca95526700 1 --2- 172.21.15.64:0/4191797210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca90100fe0 0x7fca9019cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:45.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.678+0000 7fca96528700 1 --2- 172.21.15.64:0/4191797210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca90106620 0x7fca901966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:45.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.678+0000 7fca96528700 1 -- 172.21.15.64:0/4191797210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca90100fe0 msgr2=0x7fca9019cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:45.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.678+0000 7fca96528700 1 --2- 172.21.15.64:0/4191797210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca90100fe0 0x7fca9019cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:45.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.678+0000 7fca96528700 1 -- 172.21.15.64:0/4191797210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca90100620 msgr2=0x7fca9019c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:45.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.678+0000 7fca96528700 1 --2- 172.21.15.64:0/4191797210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca90100620 0x7fca9019c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:45.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.678+0000 7fca96528700 1 -- 172.21.15.64:0/4191797210 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca88009c40 con 0x7fca90106620 2023-09-13T03:11:45.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.678+0000 7fca95d27700 1 --2- 172.21.15.64:0/4191797210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca90100620 0x7fca9019c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:11:45.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.679+0000 7fca96528700 1 --2- 172.21.15.64:0/4191797210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca90106620 0x7fca901966a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fca8800b6c0 tx=0x7fca880094b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:45.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.679+0000 7fca86ffd700 1 -- 172.21.15.64:0/4191797210 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8800e040 con 0x7fca90106620 2023-09-13T03:11:45.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.679+0000 7fca86ffd700 1 -- 172.21.15.64:0/4191797210 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca88004e70 con 0x7fca90106620 2023-09-13T03:11:45.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.679+0000 7fca86ffd700 1 -- 172.21.15.64:0/4191797210 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca88005040 con 0x7fca90106620 2023-09-13T03:11:45.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.679+0000 7fca97f8b700 1 -- 172.21.15.64:0/4191797210 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca901972c0 con 0x7fca90106620 2023-09-13T03:11:45.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.679+0000 7fca97f8b700 1 -- 172.21.15.64:0/4191797210 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca90072a00 con 0x7fca90106620 2023-09-13T03:11:45.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.680+0000 7fca97f8b700 1 -- 172.21.15.64:0/4191797210 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fca901108c0 con 0x7fca90106620 2023-09-13T03:11:45.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.683+0000 7fca86ffd700 1 -- 172.21.15.64:0/4191797210 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fca8801d470 con 0x7fca90106620 2023-09-13T03:11:45.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.685+0000 7fca86ffd700 1 --2- 172.21.15.64:0/4191797210 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fca7c0a06d0 0x7fca7c0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:45.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.685+0000 7fca86ffd700 1 -- 172.21.15.64:0/4191797210 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fca880cd360 con 0x7fca90106620 2023-09-13T03:11:45.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.686+0000 7fca86ffd700 1 --2- 172.21.15.64:0/4191797210 >> [v2:172.21.15.64:6818/1436584490,v1:172.21.15.64:6819/1436584490] conn(0x7fca7c0aa190 0x7fca7c0ac630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:45.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.686+0000 7fca86ffd700 1 -- 172.21.15.64:0/4191797210 --> [v2:172.21.15.64:6818/1436584490,v1:172.21.15.64:6819/1436584490] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca7c0acd00 con 0x7fca7c0aa190 2023-09-13T03:11:45.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.686+0000 7fca86ffd700 1 -- 172.21.15.64:0/4191797210 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca880cd6d0 con 0x7fca90106620 2023-09-13T03:11:45.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.686+0000 7fca95d27700 1 --2- 172.21.15.64:0/4191797210 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fca7c0a06d0 0x7fca7c0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:45.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.687+0000 7fca95526700 1 --2- 172.21.15.64:0/4191797210 >> [v2:172.21.15.64:6818/1436584490,v1:172.21.15.64:6819/1436584490] conn(0x7fca7c0aa190 0x7fca7c0ac630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:45.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.688+0000 7fca95d27700 1 --2- 172.21.15.64:0/4191797210 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fca7c0a06d0 0x7fca7c0a2b90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fca8c011550 tx=0x7fca8c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:45.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.688+0000 7fca95526700 1 --2- 172.21.15.64:0/4191797210 >> [v2:172.21.15.64:6818/1436584490,v1:172.21.15.64:6819/1436584490] conn(0x7fca7c0aa190 0x7fca7c0ac630 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:45.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.689+0000 7fca86ffd700 1 -- 172.21.15.64:0/4191797210 <== osd.6 v2:172.21.15.64:6818/1436584490 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fca7c0acd00 con 0x7fca7c0aa190 2023-09-13T03:11:45.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.725+0000 7fca97f8b700 1 -- 172.21.15.64:0/4191797210 --> [v2:172.21.15.64:6818/1436584490,v1:172.21.15.64:6819/1436584490] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fca9004e300 con 0x7fca7c0aa190 2023-09-13T03:11:45.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.726+0000 7fca86ffd700 1 -- 172.21.15.64:0/4191797210 <== osd.6 v2:172.21.15.64:6818/1436584490 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fca9004e300 con 0x7fca7c0aa190 2023-09-13T03:11:45.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.726+0000 7fca97f8b700 1 -- 172.21.15.64:0/4191797210 >> [v2:172.21.15.64:6818/1436584490,v1:172.21.15.64:6819/1436584490] conn(0x7fca7c0aa190 msgr2=0x7fca7c0ac630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:45.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.726+0000 7fca97f8b700 1 --2- 172.21.15.64:0/4191797210 >> [v2:172.21.15.64:6818/1436584490,v1:172.21.15.64:6819/1436584490] conn(0x7fca7c0aa190 0x7fca7c0ac630 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:45.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.727+0000 7fca97f8b700 1 -- 172.21.15.64:0/4191797210 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fca7c0a06d0 msgr2=0x7fca7c0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:45.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.727+0000 7fca97f8b700 1 --2- 172.21.15.64:0/4191797210 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fca7c0a06d0 0x7fca7c0a2b90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fca8c011550 tx=0x7fca8c009260 comp rx=0 tx=0).stop 2023-09-13T03:11:45.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.727+0000 7fca97f8b700 1 -- 172.21.15.64:0/4191797210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca90106620 msgr2=0x7fca901966a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:45.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.727+0000 7fca97f8b700 1 --2- 172.21.15.64:0/4191797210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca90106620 0x7fca901966a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fca8800b6c0 tx=0x7fca880094b0 comp rx=0 tx=0).stop 2023-09-13T03:11:45.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.727+0000 7fca96528700 1 -- 172.21.15.64:0/4191797210 reap_dead start 2023-09-13T03:11:45.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.727+0000 7fca97f8b700 1 -- 172.21.15.64:0/4191797210 shutdown_connections 2023-09-13T03:11:45.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.727+0000 7fca97f8b700 1 -- 172.21.15.64:0/4191797210 >> 172.21.15.64:0/4191797210 conn(0x7fca900fc0f0 msgr2=0x7fca90104130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:45.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.727+0000 7fca97f8b700 1 -- 172.21.15.64:0/4191797210 shutdown_connections 2023-09-13T03:11:45.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.727+0000 7fca97f8b700 1 -- 172.21.15.64:0/4191797210 wait complete. 2023-09-13T03:11:45.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.794+0000 7f6e635fc700 1 -- 172.21.15.64:0/3012984269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e5c068120 msgr2=0x7f6e5c0685a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:45.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.794+0000 7f6e635fc700 1 --2- 172.21.15.64:0/3012984269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e5c068120 0x7f6e5c0685a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c009990 tx=0x7f6e4c009ca0 comp rx=0 tx=0).stop 2023-09-13T03:11:45.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.795+0000 7f6e635fc700 1 -- 172.21.15.64:0/3012984269 shutdown_connections 2023-09-13T03:11:45.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.795+0000 7f6e635fc700 1 --2- 172.21.15.64:0/3012984269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e5c106770 0x7f6e5c106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:45.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.795+0000 7f6e635fc700 1 --2- 172.21.15.64:0/3012984269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e5c068ae0 0x7f6e5c10d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:45.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.795+0000 7f6e635fc700 1 --2- 172.21.15.64:0/3012984269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e5c068120 0x7f6e5c0685a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:45.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.795+0000 7f6e635fc700 1 -- 172.21.15.64:0/3012984269 >> 172.21.15.64:0/3012984269 conn(0x7f6e5c076600 msgr2=0x7f6e5c076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:45.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.795+0000 7f6e635fc700 1 -- 172.21.15.64:0/3012984269 shutdown_connections 2023-09-13T03:11:45.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.795+0000 7f6e635fc700 1 -- 172.21.15.64:0/3012984269 wait complete. 2023-09-13T03:11:45.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.797+0000 7f6e635fc700 1 Processor -- start 2023-09-13T03:11:45.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.797+0000 7f6e635fc700 1 -- start start 2023-09-13T03:11:45.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.798+0000 7f6e635fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e5c068120 0x7f6e5c10ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:45.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.798+0000 7f6e635fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e5c068ae0 0x7f6e5c10d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:45.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.798+0000 7f6e635fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e5c106770 0x7f6e5c109f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:45.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.798+0000 7f6e635fc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5c10a720 con 0x7f6e5c068120 2023-09-13T03:11:45.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.798+0000 7f6e635fc700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5c10a890 con 0x7f6e5c068ae0 2023-09-13T03:11:45.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.798+0000 7f6e635fc700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5c10aa50 con 0x7f6e5c106770 2023-09-13T03:11:45.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.798+0000 7f6e60b97700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e5c068ae0 0x7f6e5c10d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:45.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.798+0000 7f6e61398700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e5c068120 0x7f6e5c10ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:45.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.798+0000 7f6e60b97700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e5c068ae0 0x7f6e5c10d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:38808/0 (socket says 172.21.15.64:38808) 2023-09-13T03:11:45.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.798+0000 7f6e60b97700 1 -- 172.21.15.64:0/3796401392 learned_addr learned my addr 172.21.15.64:0/3796401392 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:45.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.798+0000 7f6e61b99700 1 --2- 172.21.15.64:0/3796401392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e5c106770 0x7f6e5c109f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:45.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.799+0000 7f6e60b97700 1 -- 172.21.15.64:0/3796401392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e5c106770 msgr2=0x7f6e5c109f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:45.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.799+0000 7f6e60b97700 1 --2- 172.21.15.64:0/3796401392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e5c106770 0x7f6e5c109f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:45.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.799+0000 7f6e60b97700 1 -- 172.21.15.64:0/3796401392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e5c068120 msgr2=0x7f6e5c10ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:45.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.799+0000 7f6e60b97700 1 --2- 172.21.15.64:0/3796401392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e5c068120 0x7f6e5c10ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:45.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.799+0000 7f6e60b97700 1 -- 172.21.15.64:0/3796401392 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e58009520 con 0x7f6e5c068ae0 2023-09-13T03:11:45.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.799+0000 7f6e60b97700 1 --2- 172.21.15.64:0/3796401392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e5c068ae0 0x7f6e5c10d370 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c005910 tx=0x7f6e4c005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:45.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.799+0000 7f6e527fc700 1 -- 172.21.15.64:0/3796401392 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c01d070 con 0x7f6e5c068ae0 2023-09-13T03:11:45.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.800+0000 7f6e527fc700 1 -- 172.21.15.64:0/3796401392 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e4c022470 con 0x7f6e5c068ae0 2023-09-13T03:11:45.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.800+0000 7f6e527fc700 1 -- 172.21.15.64:0/3796401392 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c017650 con 0x7f6e5c068ae0 2023-09-13T03:11:45.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.800+0000 7f6e635fc700 1 -- 172.21.15.64:0/3796401392 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e4c0095f0 con 0x7f6e5c068ae0 2023-09-13T03:11:45.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.800+0000 7f6e635fc700 1 -- 172.21.15.64:0/3796401392 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e5c1a9870 con 0x7f6e5c068ae0 2023-09-13T03:11:45.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.802+0000 7f6e635fc700 1 -- 172.21.15.64:0/3796401392 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6e5c101070 con 0x7f6e5c068ae0 2023-09-13T03:11:45.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.805+0000 7f6e527fc700 1 -- 172.21.15.64:0/3796401392 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f6e4c0225e0 con 0x7f6e5c068ae0 2023-09-13T03:11:45.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.807+0000 7f6e527fc700 1 --2- 172.21.15.64:0/3796401392 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6e480a0750 0x7f6e480a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:45.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.807+0000 7f6e527fc700 1 -- 172.21.15.64:0/3796401392 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f6e4c0cdaa0 con 0x7f6e5c068ae0 2023-09-13T03:11:45.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.807+0000 7f6e61398700 1 --2- 172.21.15.64:0/3796401392 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6e480a0750 0x7f6e480a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:45.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.807+0000 7f6e527fc700 1 --2- 172.21.15.64:0/3796401392 >> [v2:172.21.15.142:6816/3726793534,v1:172.21.15.142:6817/3726793534] conn(0x7f6e480aa210 0x7f6e480ac6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:45.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.807+0000 7f6e527fc700 1 -- 172.21.15.64:0/3796401392 --> [v2:172.21.15.142:6816/3726793534,v1:172.21.15.142:6817/3726793534] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6e480acd80 con 0x7f6e480aa210 2023-09-13T03:11:45.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.807+0000 7f6e527fc700 1 -- 172.21.15.64:0/3796401392 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e4c0cde10 con 0x7f6e5c068ae0 2023-09-13T03:11:45.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.808+0000 7f6e61b99700 1 --2- 172.21.15.64:0/3796401392 >> [v2:172.21.15.142:6816/3726793534,v1:172.21.15.142:6817/3726793534] conn(0x7f6e480aa210 0x7f6e480ac6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:45.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.808+0000 7f6e61398700 1 --2- 172.21.15.64:0/3796401392 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6e480a0750 0x7f6e480a2c10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6e58009df0 tx=0x7f6e58009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:45.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.809+0000 7f6e61b99700 1 --2- 172.21.15.64:0/3796401392 >> [v2:172.21.15.142:6816/3726793534,v1:172.21.15.142:6817/3726793534] conn(0x7f6e480aa210 0x7f6e480ac6b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:45.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.809+0000 7f6e527fc700 1 -- 172.21.15.64:0/3796401392 <== osd.7 v2:172.21.15.142:6816/3726793534 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f6e480acd80 con 0x7f6e480aa210 2023-09-13T03:11:45.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.832+0000 7f6e635fc700 1 -- 172.21.15.64:0/3796401392 --> [v2:172.21.15.142:6816/3726793534,v1:172.21.15.142:6817/3726793534] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6e5c04e300 con 0x7f6e480aa210 2023-09-13T03:11:45.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.833+0000 7f6e527fc700 1 -- 172.21.15.64:0/3796401392 <== osd.7 v2:172.21.15.142:6816/3726793534 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f6e5c04e300 con 0x7f6e480aa210 2023-09-13T03:11:45.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.833+0000 7f6e635fc700 1 -- 172.21.15.64:0/3796401392 >> [v2:172.21.15.142:6816/3726793534,v1:172.21.15.142:6817/3726793534] conn(0x7f6e480aa210 msgr2=0x7f6e480ac6b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:45.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.833+0000 7f6e635fc700 1 --2- 172.21.15.64:0/3796401392 >> [v2:172.21.15.142:6816/3726793534,v1:172.21.15.142:6817/3726793534] conn(0x7f6e480aa210 0x7f6e480ac6b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:45.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.833+0000 7f6e635fc700 1 -- 172.21.15.64:0/3796401392 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6e480a0750 msgr2=0x7f6e480a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:45.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.833+0000 7f6e635fc700 1 --2- 172.21.15.64:0/3796401392 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6e480a0750 0x7f6e480a2c10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6e58009df0 tx=0x7f6e58009260 comp rx=0 tx=0).stop 2023-09-13T03:11:45.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.833+0000 7f6e635fc700 1 -- 172.21.15.64:0/3796401392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e5c068ae0 msgr2=0x7f6e5c10d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:45.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.833+0000 7f6e635fc700 1 --2- 172.21.15.64:0/3796401392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e5c068ae0 0x7f6e5c10d370 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c005910 tx=0x7f6e4c005720 comp rx=0 tx=0).stop 2023-09-13T03:11:45.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.833+0000 7f6e61b99700 1 -- 172.21.15.64:0/3796401392 reap_dead start 2023-09-13T03:11:45.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.834+0000 7f6e635fc700 1 -- 172.21.15.64:0/3796401392 shutdown_connections 2023-09-13T03:11:45.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.834+0000 7f6e635fc700 1 -- 172.21.15.64:0/3796401392 >> 172.21.15.64:0/3796401392 conn(0x7f6e5c076600 msgr2=0x7f6e5c104330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:45.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.834+0000 7f6e635fc700 1 -- 172.21.15.64:0/3796401392 shutdown_connections 2023-09-13T03:11:45.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.834+0000 7f6e635fc700 1 -- 172.21.15.64:0/3796401392 wait complete. 2023-09-13T03:11:45.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.653+0000 7f9d1f865700 1 -- 172.21.15.64:0/1598325401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d18068ae0 msgr2=0x7f9d1810d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:45.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.653+0000 7f9d1f865700 1 --2- 172.21.15.64:0/1598325401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d18068ae0 0x7f9d1810d440 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9d1000b0a0 tx=0x7f9d1000b3b0 comp rx=0 tx=0).stop 2023-09-13T03:11:45.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.653+0000 7f9d1f865700 1 -- 172.21.15.64:0/1598325401 shutdown_connections 2023-09-13T03:11:45.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.653+0000 7f9d1f865700 1 --2- 172.21.15.64:0/1598325401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d18068ae0 0x7f9d1810d440 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:45.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.653+0000 7f9d1f865700 1 --2- 172.21.15.64:0/1598325401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d18106770 0x7f9d18106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:45.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.653+0000 7f9d1f865700 1 --2- 172.21.15.64:0/1598325401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d18068120 0x7f9d180685a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:45.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.653+0000 7f9d1f865700 1 -- 172.21.15.64:0/1598325401 >> 172.21.15.64:0/1598325401 conn(0x7f9d18076600 msgr2=0x7f9d18076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:45.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.653+0000 7f9d1f865700 1 -- 172.21.15.64:0/1598325401 shutdown_connections 2023-09-13T03:11:45.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.653+0000 7f9d1f865700 1 -- 172.21.15.64:0/1598325401 wait complete. 2023-09-13T03:11:45.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.654+0000 7f9d1f865700 1 Processor -- start 2023-09-13T03:11:45.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.655+0000 7f9d1f865700 1 -- start start 2023-09-13T03:11:45.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.655+0000 7f9d1f865700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d18068120 0x7f9d1819e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:45.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.655+0000 7f9d1f865700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d18068ae0 0x7f9d1819edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:45.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.655+0000 7f9d1f865700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d18106770 0x7f9d18198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:45.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.655+0000 7f9d1f865700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d18199020 con 0x7f9d18106770 2023-09-13T03:11:45.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.655+0000 7f9d1f865700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d18199190 con 0x7f9d18068ae0 2023-09-13T03:11:45.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.655+0000 7f9d1f865700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d18199350 con 0x7f9d18068120 2023-09-13T03:11:45.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.656+0000 7f9d1d601700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d18068120 0x7f9d1819e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:45.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.656+0000 7f9d1d601700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d18068120 0x7f9d1819e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:58318/0 (socket says 172.21.15.64:58318) 2023-09-13T03:11:45.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.656+0000 7f9d1d601700 1 -- 172.21.15.64:0/1658375996 learned_addr learned my addr 172.21.15.64:0/1658375996 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:45.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.656+0000 7f9d1de02700 1 --2- 172.21.15.64:0/1658375996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d18106770 0x7f9d18198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:45.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.656+0000 7f9d1ce00700 1 --2- 172.21.15.64:0/1658375996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d18068ae0 0x7f9d1819edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:45.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.656+0000 7f9d1d601700 1 -- 172.21.15.64:0/1658375996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d18068ae0 msgr2=0x7f9d1819edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:45.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.656+0000 7f9d1d601700 1 --2- 172.21.15.64:0/1658375996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d18068ae0 0x7f9d1819edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:45.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.656+0000 7f9d1d601700 1 -- 172.21.15.64:0/1658375996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d18106770 msgr2=0x7f9d18198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:45.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.656+0000 7f9d1d601700 1 --2- 172.21.15.64:0/1658375996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d18106770 0x7f9d18198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:45.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.656+0000 7f9d1d601700 1 -- 172.21.15.64:0/1658375996 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d10009d00 con 0x7f9d18068120 2023-09-13T03:11:45.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.656+0000 7f9d1ce00700 1 --2- 172.21.15.64:0/1658375996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d18068ae0 0x7f9d1819edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T03:11:45.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.657+0000 7f9d1d601700 1 --2- 172.21.15.64:0/1658375996 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d18068120 0x7f9d1819e880 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400e9a0 tx=0x7f9d1400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:45.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.657+0000 7f9d0e7fc700 1 -- 172.21.15.64:0/1658375996 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d14009790 con 0x7f9d18068120 2023-09-13T03:11:45.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.657+0000 7f9d0e7fc700 1 -- 172.21.15.64:0/1658375996 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d1400cbd0 con 0x7f9d18068120 2023-09-13T03:11:45.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.657+0000 7f9d0e7fc700 1 -- 172.21.15.64:0/1658375996 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d14018680 con 0x7f9d18068120 2023-09-13T03:11:45.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.657+0000 7f9d1f865700 1 -- 172.21.15.64:0/1658375996 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d18199630 con 0x7f9d18068120 2023-09-13T03:11:45.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.657+0000 7f9d1f865700 1 -- 172.21.15.64:0/1658375996 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d181a5800 con 0x7f9d18068120 2023-09-13T03:11:45.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.658+0000 7f9d1f865700 1 -- 172.21.15.64:0/1658375996 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9d181108e0 con 0x7f9d18068120 2023-09-13T03:11:45.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.662+0000 7f9d0e7fc700 1 -- 172.21.15.64:0/1658375996 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f9d1400cd40 con 0x7f9d18068120 2023-09-13T03:11:45.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.663+0000 7f9d0e7fc700 1 --2- 172.21.15.64:0/1658375996 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f9d040a0750 0x7f9d040a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:45.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.663+0000 7f9d0e7fc700 1 -- 172.21.15.64:0/1658375996 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f9d14014070 con 0x7f9d18068120 2023-09-13T03:11:45.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.663+0000 7f9d1ce00700 1 --2- 172.21.15.64:0/1658375996 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f9d040a0750 0x7f9d040a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:45.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.664+0000 7f9d0e7fc700 1 --2- 172.21.15.64:0/1658375996 >> [v2:172.21.15.170:6816/4272513873,v1:172.21.15.170:6817/4272513873] conn(0x7f9d040aa210 0x7f9d040ac6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:45.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.664+0000 7f9d0e7fc700 1 -- 172.21.15.64:0/1658375996 --> [v2:172.21.15.170:6816/4272513873,v1:172.21.15.170:6817/4272513873] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9d040acd80 con 0x7f9d040aa210 2023-09-13T03:11:45.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.664+0000 7f9d0e7fc700 1 -- 172.21.15.64:0/1658375996 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d140cc770 con 0x7f9d18068120 2023-09-13T03:11:45.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.664+0000 7f9d1de02700 1 --2- 172.21.15.64:0/1658375996 >> [v2:172.21.15.170:6816/4272513873,v1:172.21.15.170:6817/4272513873] conn(0x7f9d040aa210 0x7f9d040ac6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:45.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.664+0000 7f9d1ce00700 1 --2- 172.21.15.64:0/1658375996 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f9d040a0750 0x7f9d040a2c10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9d1819a0a0 tx=0x7f9d08009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:45.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.665+0000 7f9d1de02700 1 --2- 172.21.15.64:0/1658375996 >> [v2:172.21.15.170:6816/4272513873,v1:172.21.15.170:6817/4272513873] conn(0x7f9d040aa210 0x7f9d040ac6b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:45.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.665+0000 7f9d0e7fc700 1 -- 172.21.15.64:0/1658375996 <== osd.8 v2:172.21.15.170:6816/4272513873 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f9d040acd80 con 0x7f9d040aa210 2023-09-13T03:11:45.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.687+0000 7f9d1f865700 1 -- 172.21.15.64:0/1658375996 --> [v2:172.21.15.170:6816/4272513873,v1:172.21.15.170:6817/4272513873] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9d1804e300 con 0x7f9d040aa210 2023-09-13T03:11:45.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.688+0000 7f9d0e7fc700 1 -- 172.21.15.64:0/1658375996 <== osd.8 v2:172.21.15.170:6816/4272513873 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9d1804e300 con 0x7f9d040aa210 2023-09-13T03:11:45.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.689+0000 7f9d1f865700 1 -- 172.21.15.64:0/1658375996 >> [v2:172.21.15.170:6816/4272513873,v1:172.21.15.170:6817/4272513873] conn(0x7f9d040aa210 msgr2=0x7f9d040ac6b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:45.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.689+0000 7f9d1f865700 1 --2- 172.21.15.64:0/1658375996 >> [v2:172.21.15.170:6816/4272513873,v1:172.21.15.170:6817/4272513873] conn(0x7f9d040aa210 0x7f9d040ac6b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:45.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.690+0000 7f9d1f865700 1 -- 172.21.15.64:0/1658375996 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f9d040a0750 msgr2=0x7f9d040a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:45.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.690+0000 7f9d1f865700 1 --2- 172.21.15.64:0/1658375996 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f9d040a0750 0x7f9d040a2c10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9d1819a0a0 tx=0x7f9d08009260 comp rx=0 tx=0).stop 2023-09-13T03:11:45.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.690+0000 7f9d1f865700 1 -- 172.21.15.64:0/1658375996 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d18068120 msgr2=0x7f9d1819e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:45.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.690+0000 7f9d1f865700 1 --2- 172.21.15.64:0/1658375996 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d18068120 0x7f9d1819e880 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400e9a0 tx=0x7f9d1400ecb0 comp rx=0 tx=0).stop 2023-09-13T03:11:45.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.690+0000 7f9d1de02700 1 -- 172.21.15.64:0/1658375996 reap_dead start 2023-09-13T03:11:45.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.690+0000 7f9d1f865700 1 -- 172.21.15.64:0/1658375996 shutdown_connections 2023-09-13T03:11:45.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.690+0000 7f9d1f865700 1 -- 172.21.15.64:0/1658375996 >> 172.21.15.64:0/1658375996 conn(0x7f9d18076600 msgr2=0x7f9d1810b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:45.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.691+0000 7f9d1f865700 1 -- 172.21.15.64:0/1658375996 shutdown_connections 2023-09-13T03:11:45.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:45.691+0000 7f9d1f865700 1 -- 172.21.15.64:0/1658375996 wait complete. 2023-09-13T03:11:46.461 INFO:teuthology.orchestra.run.smithi064.stdout:107374182455 2023-09-13T03:11:46.462 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.3 2023-09-13T03:11:46.868 INFO:teuthology.orchestra.run.smithi064.stdout:304942678038 2023-09-13T03:11:46.869 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.10 2023-09-13T03:11:46.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.952+0000 7fcdf850b700 1 -- 172.21.15.64:0/30620386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcdf0068120 msgr2=0x7fcdf00685a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:46.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.952+0000 7fcdf850b700 1 --2- 172.21.15.64:0/30620386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcdf0068120 0x7fcdf00685a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcde00098f0 tx=0x7fcde0009c00 comp rx=0 tx=0).stop 2023-09-13T03:11:46.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.952+0000 7fcdf850b700 1 -- 172.21.15.64:0/30620386 shutdown_connections 2023-09-13T03:11:46.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.953+0000 7fcdf850b700 1 --2- 172.21.15.64:0/30620386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdf0068ae0 0x7fcdf010d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:46.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.953+0000 7fcdf850b700 1 --2- 172.21.15.64:0/30620386 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdf0106770 0x7fcdf0106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:46.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.953+0000 7fcdf850b700 1 --2- 172.21.15.64:0/30620386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcdf0068120 0x7fcdf00685a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:46.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.953+0000 7fcdf850b700 1 -- 172.21.15.64:0/30620386 >> 172.21.15.64:0/30620386 conn(0x7fcdf0076600 msgr2=0x7fcdf0076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:46.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.953+0000 7fcdf850b700 1 -- 172.21.15.64:0/30620386 shutdown_connections 2023-09-13T03:11:46.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.953+0000 7fcdf850b700 1 -- 172.21.15.64:0/30620386 wait complete. 2023-09-13T03:11:46.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.954+0000 7fcdf850b700 1 Processor -- start 2023-09-13T03:11:46.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.954+0000 7fcdf850b700 1 -- start start 2023-09-13T03:11:46.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.955+0000 7fcdf850b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdf0068120 0x7fcdf019e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:46.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.955+0000 7fcdf850b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdf0068ae0 0x7fcdf019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:46.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.955+0000 7fcdf850b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcdf0106770 0x7fcdf0198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:46.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.955+0000 7fcdf850b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf0198fc0 con 0x7fcdf0068120 2023-09-13T03:11:46.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.955+0000 7fcdf850b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf0199130 con 0x7fcdf0106770 2023-09-13T03:11:46.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.955+0000 7fcdf850b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf01992f0 con 0x7fcdf0068ae0 2023-09-13T03:11:46.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.955+0000 7fcdf62a7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdf0068120 0x7fcdf019e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:46.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.955+0000 7fcdf5aa6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdf0068ae0 0x7fcdf019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:46.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.955+0000 7fcdf5aa6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdf0068ae0 0x7fcdf019ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:41884/0 (socket says 172.21.15.64:41884) 2023-09-13T03:11:46.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.955+0000 7fcdf5aa6700 1 -- 172.21.15.64:0/1798009178 learned_addr learned my addr 172.21.15.64:0/1798009178 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:46.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.955+0000 7fcdf5aa6700 1 -- 172.21.15.64:0/1798009178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcdf0106770 msgr2=0x7fcdf0198810 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-13T03:11:46.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.955+0000 7fcdf5aa6700 1 --2- 172.21.15.64:0/1798009178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcdf0106770 0x7fcdf0198810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:46.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.955+0000 7fcdf5aa6700 1 -- 172.21.15.64:0/1798009178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdf0068120 msgr2=0x7fcdf019e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:46.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.955+0000 7fcdf5aa6700 1 --2- 172.21.15.64:0/1798009178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdf0068120 0x7fcdf019e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:46.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.956+0000 7fcdf5aa6700 1 -- 172.21.15.64:0/1798009178 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdec009520 con 0x7fcdf0068ae0 2023-09-13T03:11:46.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.956+0000 7fcdf62a7700 1 --2- 172.21.15.64:0/1798009178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdf0068120 0x7fcdf019e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:11:46.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.958+0000 7fcdf5aa6700 1 --2- 172.21.15.64:0/1798009178 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdf0068ae0 0x7fcdf019ed60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcde0009810 tx=0x7fcde000fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:46.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.959+0000 7fcde77fe700 1 -- 172.21.15.64:0/1798009178 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde001c070 con 0x7fcdf0068ae0 2023-09-13T03:11:46.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.959+0000 7fcde77fe700 1 -- 172.21.15.64:0/1798009178 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcde000bb50 con 0x7fcdf0068ae0 2023-09-13T03:11:46.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.959+0000 7fcde77fe700 1 -- 172.21.15.64:0/1798009178 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde0004930 con 0x7fcdf0068ae0 2023-09-13T03:11:46.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.959+0000 7fcdf850b700 1 -- 172.21.15.64:0/1798009178 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcde0009520 con 0x7fcdf0068ae0 2023-09-13T03:11:46.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.959+0000 7fcdf850b700 1 -- 172.21.15.64:0/1798009178 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf01a55d0 con 0x7fcdf0068ae0 2023-09-13T03:11:46.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.960+0000 7fcdf850b700 1 -- 172.21.15.64:0/1798009178 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcdf01108a0 con 0x7fcdf0068ae0 2023-09-13T03:11:46.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.964+0000 7fcde77fe700 1 -- 172.21.15.64:0/1798009178 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fcde0020070 con 0x7fcdf0068ae0 2023-09-13T03:11:46.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.965+0000 7fcde77fe700 1 --2- 172.21.15.64:0/1798009178 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fcddc0a0750 0x7fcddc0a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:46.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.965+0000 7fcde77fe700 1 -- 172.21.15.64:0/1798009178 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fcde00cd250 con 0x7fcdf0068ae0 2023-09-13T03:11:46.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.966+0000 7fcdf62a7700 1 --2- 172.21.15.64:0/1798009178 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fcddc0a0750 0x7fcddc0a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:46.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.966+0000 7fcde77fe700 1 --2- 172.21.15.64:0/1798009178 >> [v2:172.21.15.170:6808/2903841453,v1:172.21.15.170:6809/2903841453] conn(0x7fcddc0aa210 0x7fcddc0ac6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:46.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.966+0000 7fcde77fe700 1 -- 172.21.15.64:0/1798009178 --> [v2:172.21.15.170:6808/2903841453,v1:172.21.15.170:6809/2903841453] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcddc0acd80 con 0x7fcddc0aa210 2023-09-13T03:11:46.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.966+0000 7fcde77fe700 1 -- 172.21.15.64:0/1798009178 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcde00cd660 con 0x7fcdf0068ae0 2023-09-13T03:11:46.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.966+0000 7fcdf6aa8700 1 --2- 172.21.15.64:0/1798009178 >> [v2:172.21.15.170:6808/2903841453,v1:172.21.15.170:6809/2903841453] conn(0x7fcddc0aa210 0x7fcddc0ac6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:46.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.966+0000 7fcdf62a7700 1 --2- 172.21.15.64:0/1798009178 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fcddc0a0750 0x7fcddc0a2c10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcdec011630 tx=0x7fcdec009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:46.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.967+0000 7fcdf6aa8700 1 --2- 172.21.15.64:0/1798009178 >> [v2:172.21.15.170:6808/2903841453,v1:172.21.15.170:6809/2903841453] conn(0x7fcddc0aa210 0x7fcddc0ac6b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:46.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.968+0000 7fcde77fe700 1 -- 172.21.15.64:0/1798009178 <== osd.5 v2:172.21.15.170:6808/2903841453 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fcddc0acd80 con 0x7fcddc0aa210 2023-09-13T03:11:46.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:46.986+0000 7fcdf850b700 1 -- 172.21.15.64:0/1798009178 --> [v2:172.21.15.170:6808/2903841453,v1:172.21.15.170:6809/2903841453] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcdf004e300 con 0x7fcddc0aa210 2023-09-13T03:11:47.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:47.001+0000 7fcde77fe700 1 -- 172.21.15.64:0/1798009178 <== osd.5 v2:172.21.15.170:6808/2903841453 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fcdf004e300 con 0x7fcddc0aa210 2023-09-13T03:11:47.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:47.001+0000 7fcdf850b700 1 -- 172.21.15.64:0/1798009178 >> [v2:172.21.15.170:6808/2903841453,v1:172.21.15.170:6809/2903841453] conn(0x7fcddc0aa210 msgr2=0x7fcddc0ac6b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:47.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:47.001+0000 7fcdf850b700 1 --2- 172.21.15.64:0/1798009178 >> [v2:172.21.15.170:6808/2903841453,v1:172.21.15.170:6809/2903841453] conn(0x7fcddc0aa210 0x7fcddc0ac6b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:47.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:47.002+0000 7fcdf850b700 1 -- 172.21.15.64:0/1798009178 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fcddc0a0750 msgr2=0x7fcddc0a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:47.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:47.002+0000 7fcdf850b700 1 --2- 172.21.15.64:0/1798009178 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fcddc0a0750 0x7fcddc0a2c10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcdec011630 tx=0x7fcdec009260 comp rx=0 tx=0).stop 2023-09-13T03:11:47.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:47.002+0000 7fcdf850b700 1 -- 172.21.15.64:0/1798009178 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdf0068ae0 msgr2=0x7fcdf019ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:47.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:47.002+0000 7fcdf850b700 1 --2- 172.21.15.64:0/1798009178 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdf0068ae0 0x7fcdf019ed60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcde0009810 tx=0x7fcde000fa00 comp rx=0 tx=0).stop 2023-09-13T03:11:47.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:47.002+0000 7fcdf6aa8700 1 -- 172.21.15.64:0/1798009178 reap_dead start 2023-09-13T03:11:47.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:47.002+0000 7fcdf850b700 1 -- 172.21.15.64:0/1798009178 shutdown_connections 2023-09-13T03:11:47.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:47.002+0000 7fcdf850b700 1 -- 172.21.15.64:0/1798009178 >> 172.21.15.64:0/1798009178 conn(0x7fcdf0076600 msgr2=0x7fcdf010b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:47.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:47.002+0000 7fcdf850b700 1 -- 172.21.15.64:0/1798009178 shutdown_connections 2023-09-13T03:11:47.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:47.002+0000 7fcdf850b700 1 -- 172.21.15.64:0/1798009178 wait complete. 2023-09-13T03:11:47.009 INFO:teuthology.orchestra.run.smithi064.stdout:219043332132 2023-09-13T03:11:47.009 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.7 2023-09-13T03:11:47.525 INFO:teuthology.orchestra.run.smithi064.stdout:197568495657 2023-09-13T03:11:47.526 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.6 2023-09-13T03:11:47.776 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:47 smithi064 ceph-mon[97675]: pgmap v297: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 9 op/s 2023-09-13T03:11:47.822 INFO:teuthology.orchestra.run.smithi064.stdout:249108103200 2023-09-13T03:11:47.822 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.8 2023-09-13T03:11:47.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:47 smithi170 ceph-mon[134328]: pgmap v297: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 9 op/s 2023-09-13T03:11:47.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:47 smithi142 ceph-mon[134082]: pgmap v297: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 9 op/s 2023-09-13T03:11:48.725 INFO:teuthology.orchestra.run.smithi064.stdout:163208757295 2023-09-13T03:11:48.725 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.5 2023-09-13T03:11:48.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.770+0000 7fe7f2457700 1 -- 172.21.15.64:0/3790126597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7ec119de0 msgr2=0x7fe7ec11c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:48.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.770+0000 7fe7f2457700 1 --2- 172.21.15.64:0/3790126597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7ec119de0 0x7fe7ec11c1d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe7e00098f0 tx=0x7fe7e0009c00 comp rx=0 tx=0).stop 2023-09-13T03:11:48.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.770+0000 7fe7f2457700 1 -- 172.21.15.64:0/3790126597 shutdown_connections 2023-09-13T03:11:48.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.770+0000 7fe7f2457700 1 --2- 172.21.15.64:0/3790126597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7ec119de0 0x7fe7ec11c1d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:48.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.770+0000 7fe7f2457700 1 --2- 172.21.15.64:0/3790126597 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7ec10ee60 0x7fe7ec10c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:48.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.770+0000 7fe7f2457700 1 --2- 172.21.15.64:0/3790126597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7ec10ce80 0x7fe7ec10d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:48.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.770+0000 7fe7f2457700 1 -- 172.21.15.64:0/3790126597 >> 172.21.15.64:0/3790126597 conn(0x7fe7ec06c800 msgr2=0x7fe7ec06cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:48.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.771+0000 7fe7f2457700 1 -- 172.21.15.64:0/3790126597 shutdown_connections 2023-09-13T03:11:48.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.771+0000 7fe7f2457700 1 -- 172.21.15.64:0/3790126597 wait complete. 2023-09-13T03:11:48.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.772+0000 7fe7f2457700 1 Processor -- start 2023-09-13T03:11:48.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.772+0000 7fe7f2457700 1 -- start start 2023-09-13T03:11:48.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.773+0000 7fe7f2457700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7ec10ce80 0x7fe7ec1a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:48.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.773+0000 7fe7f2457700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7ec10ee60 0x7fe7ec1a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:48.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.773+0000 7fe7f2457700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7ec119de0 0x7fe7ec1a9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:48.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.773+0000 7fe7f2457700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec1aa4c0 con 0x7fe7ec10ce80 2023-09-13T03:11:48.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.773+0000 7fe7f2457700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec1aa630 con 0x7fe7ec10ee60 2023-09-13T03:11:48.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.773+0000 7fe7f2457700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec1aa7f0 con 0x7fe7ec119de0 2023-09-13T03:11:48.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.773+0000 7fe7f09f4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7ec119de0 0x7fe7ec1a9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:48.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.773+0000 7fe7f09f4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7ec119de0 0x7fe7ec1a9eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:41910/0 (socket says 172.21.15.64:41910) 2023-09-13T03:11:48.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.773+0000 7fe7f09f4700 1 -- 172.21.15.64:0/2474051537 learned_addr learned my addr 172.21.15.64:0/2474051537 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:48.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.773+0000 7fe7ebfff700 1 --2- 172.21.15.64:0/2474051537 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7ec10ce80 0x7fe7ec1a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:48.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.774+0000 7fe7eb7fe700 1 --2- 172.21.15.64:0/2474051537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7ec10ee60 0x7fe7ec1a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:48.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.774+0000 7fe7f09f4700 1 -- 172.21.15.64:0/2474051537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7ec10ee60 msgr2=0x7fe7ec1a5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:48.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.774+0000 7fe7f09f4700 1 --2- 172.21.15.64:0/2474051537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7ec10ee60 0x7fe7ec1a5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:48.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.774+0000 7fe7f09f4700 1 -- 172.21.15.64:0/2474051537 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7ec10ce80 msgr2=0x7fe7ec1a5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:48.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.774+0000 7fe7f09f4700 1 --2- 172.21.15.64:0/2474051537 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7ec10ce80 0x7fe7ec1a5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:48.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.774+0000 7fe7f09f4700 1 -- 172.21.15.64:0/2474051537 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e4009d00 con 0x7fe7ec119de0 2023-09-13T03:11:48.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.774+0000 7fe7f09f4700 1 --2- 172.21.15.64:0/2474051537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7ec119de0 0x7fe7ec1a9eb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0005950 tx=0x7fe7e000b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:48.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.774+0000 7fe7e97fa700 1 -- 172.21.15.64:0/2474051537 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e001c070 con 0x7fe7ec119de0 2023-09-13T03:11:48.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.774+0000 7fe7f2457700 1 -- 172.21.15.64:0/2474051537 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7e0009520 con 0x7fe7ec119de0 2023-09-13T03:11:48.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.775+0000 7fe7e97fa700 1 -- 172.21.15.64:0/2474051537 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7e000f910 con 0x7fe7ec119de0 2023-09-13T03:11:48.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.775+0000 7fe7e97fa700 1 -- 172.21.15.64:0/2474051537 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0003a00 con 0x7fe7ec119de0 2023-09-13T03:11:48.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.775+0000 7fe7f2457700 1 -- 172.21.15.64:0/2474051537 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7ec1b2160 con 0x7fe7ec119de0 2023-09-13T03:11:48.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.776+0000 7fe7f2457700 1 -- 172.21.15.64:0/2474051537 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe7ec04e300 con 0x7fe7ec119de0 2023-09-13T03:11:48.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.780+0000 7fe7e97fa700 1 -- 172.21.15.64:0/2474051537 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fe7e0020030 con 0x7fe7ec119de0 2023-09-13T03:11:48.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.782+0000 7fe7e97fa700 1 --2- 172.21.15.64:0/2474051537 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe7d40a07d0 0x7fe7d40a2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:48.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.782+0000 7fe7e97fa700 1 -- 172.21.15.64:0/2474051537 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fe7e00cd2c0 con 0x7fe7ec119de0 2023-09-13T03:11:48.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.783+0000 7fe7e97fa700 1 --2- 172.21.15.64:0/2474051537 >> [v2:172.21.15.142:6808/1811142576,v1:172.21.15.142:6809/1811142576] conn(0x7fe7d40aa290 0x7fe7d40ac730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:48.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.783+0000 7fe7e97fa700 1 -- 172.21.15.64:0/2474051537 --> [v2:172.21.15.142:6808/1811142576,v1:172.21.15.142:6809/1811142576] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe7d40ace00 con 0x7fe7d40aa290 2023-09-13T03:11:48.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.783+0000 7fe7e97fa700 1 -- 172.21.15.64:0/2474051537 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7e00cd670 con 0x7fe7ec119de0 2023-09-13T03:11:48.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.783+0000 7fe7ebfff700 1 --2- 172.21.15.64:0/2474051537 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe7d40a07d0 0x7fe7d40a2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:48.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.783+0000 7fe7eb7fe700 1 --2- 172.21.15.64:0/2474051537 >> [v2:172.21.15.142:6808/1811142576,v1:172.21.15.142:6809/1811142576] conn(0x7fe7d40aa290 0x7fe7d40ac730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:48.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.784+0000 7fe7ebfff700 1 --2- 172.21.15.64:0/2474051537 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe7d40a07d0 0x7fe7d40a2c90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4009cd0 tx=0x7fe7e400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:48.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.784+0000 7fe7eb7fe700 1 --2- 172.21.15.64:0/2474051537 >> [v2:172.21.15.142:6808/1811142576,v1:172.21.15.142:6809/1811142576] conn(0x7fe7d40aa290 0x7fe7d40ac730 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:48.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.785+0000 7fe7e97fa700 1 -- 172.21.15.64:0/2474051537 <== osd.4 v2:172.21.15.142:6808/1811142576 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fe7d40ace00 con 0x7fe7d40aa290 2023-09-13T03:11:48.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.807+0000 7fe7f2457700 1 -- 172.21.15.64:0/2474051537 --> [v2:172.21.15.142:6808/1811142576,v1:172.21.15.142:6809/1811142576] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe7ec11f730 con 0x7fe7d40aa290 2023-09-13T03:11:48.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.808+0000 7fe7e97fa700 1 -- 172.21.15.64:0/2474051537 <== osd.4 v2:172.21.15.142:6808/1811142576 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe7dc0099e0 con 0x7fe7d40aa290 2023-09-13T03:11:48.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.809+0000 7fe7f2457700 1 -- 172.21.15.64:0/2474051537 >> [v2:172.21.15.142:6808/1811142576,v1:172.21.15.142:6809/1811142576] conn(0x7fe7d40aa290 msgr2=0x7fe7d40ac730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:48.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.809+0000 7fe7f2457700 1 --2- 172.21.15.64:0/2474051537 >> [v2:172.21.15.142:6808/1811142576,v1:172.21.15.142:6809/1811142576] conn(0x7fe7d40aa290 0x7fe7d40ac730 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:48.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.809+0000 7fe7f2457700 1 -- 172.21.15.64:0/2474051537 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe7d40a07d0 msgr2=0x7fe7d40a2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:48.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.809+0000 7fe7f2457700 1 --2- 172.21.15.64:0/2474051537 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe7d40a07d0 0x7fe7d40a2c90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4009cd0 tx=0x7fe7e400b040 comp rx=0 tx=0).stop 2023-09-13T03:11:48.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.809+0000 7fe7f2457700 1 -- 172.21.15.64:0/2474051537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7ec119de0 msgr2=0x7fe7ec1a9eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:48.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.809+0000 7fe7f2457700 1 --2- 172.21.15.64:0/2474051537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7ec119de0 0x7fe7ec1a9eb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0005950 tx=0x7fe7e000b790 comp rx=0 tx=0).stop 2023-09-13T03:11:48.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.809+0000 7fe7f09f4700 1 -- 172.21.15.64:0/2474051537 reap_dead start 2023-09-13T03:11:48.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.809+0000 7fe7f2457700 1 -- 172.21.15.64:0/2474051537 shutdown_connections 2023-09-13T03:11:48.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.809+0000 7fe7f2457700 1 -- 172.21.15.64:0/2474051537 >> 172.21.15.64:0/2474051537 conn(0x7fe7ec06c800 msgr2=0x7fe7ec06f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:48.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.809+0000 7fe7f2457700 1 -- 172.21.15.64:0/2474051537 shutdown_connections 2023-09-13T03:11:48.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:48.809+0000 7fe7f2457700 1 -- 172.21.15.64:0/2474051537 wait complete. 2023-09-13T03:11:49.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.200+0000 7f0ae598d700 1 -- 172.21.15.64:0/1782935072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ae0068120 msgr2=0x7f0ae00685a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:49.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.200+0000 7f0ae598d700 1 --2- 172.21.15.64:0/1782935072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ae0068120 0x7f0ae00685a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0acc009990 tx=0x7f0acc009ca0 comp rx=0 tx=0).stop 2023-09-13T03:11:49.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.200+0000 7f0ae598d700 1 -- 172.21.15.64:0/1782935072 shutdown_connections 2023-09-13T03:11:49.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.200+0000 7f0ae598d700 1 --2- 172.21.15.64:0/1782935072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ae0068ae0 0x7f0ae010d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:49.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.200+0000 7f0ae598d700 1 --2- 172.21.15.64:0/1782935072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ae0106770 0x7f0ae0106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:49.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.200+0000 7f0ae598d700 1 --2- 172.21.15.64:0/1782935072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ae0068120 0x7f0ae00685a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:49.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.200+0000 7f0ae598d700 1 -- 172.21.15.64:0/1782935072 >> 172.21.15.64:0/1782935072 conn(0x7f0ae0076600 msgr2=0x7f0ae0076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:49.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.200+0000 7f0ae598d700 1 -- 172.21.15.64:0/1782935072 shutdown_connections 2023-09-13T03:11:49.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.201+0000 7f0ae598d700 1 -- 172.21.15.64:0/1782935072 wait complete. 2023-09-13T03:11:49.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.202+0000 7f0ae598d700 1 Processor -- start 2023-09-13T03:11:49.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.202+0000 7f0ae598d700 1 -- start start 2023-09-13T03:11:49.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.202+0000 7f0ae598d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ae0068120 0x7f0ae01945a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:49.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.202+0000 7f0ae598d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ae0068ae0 0x7f0ae0194ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:49.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.202+0000 7f0ae598d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ae0106770 0x7f0ae0198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:49.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.202+0000 7f0ae598d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae0199560 con 0x7f0ae0068120 2023-09-13T03:11:49.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.202+0000 7f0ae598d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae01996d0 con 0x7f0ae0068ae0 2023-09-13T03:11:49.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.202+0000 7f0ae598d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae0199890 con 0x7f0ae0106770 2023-09-13T03:11:49.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.203+0000 7f0adeffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ae0068120 0x7f0ae01945a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:49.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.203+0000 7f0adeffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ae0068120 0x7f0ae01945a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46874/0 (socket says 172.21.15.64:46874) 2023-09-13T03:11:49.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.203+0000 7f0adeffd700 1 -- 172.21.15.64:0/43323604 learned_addr learned my addr 172.21.15.64:0/43323604 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:49.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.203+0000 7f0adf7fe700 1 --2- 172.21.15.64:0/43323604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ae0106770 0x7f0ae0198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:49.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.203+0000 7f0ade7fc700 1 --2- 172.21.15.64:0/43323604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ae0068ae0 0x7f0ae0194ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:49.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.203+0000 7f0adeffd700 1 -- 172.21.15.64:0/43323604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ae0106770 msgr2=0x7f0ae0198f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:49.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.203+0000 7f0adeffd700 1 --2- 172.21.15.64:0/43323604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ae0106770 0x7f0ae0198f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:49.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.203+0000 7f0adeffd700 1 -- 172.21.15.64:0/43323604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ae0068ae0 msgr2=0x7f0ae0194ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:49.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.203+0000 7f0adeffd700 1 --2- 172.21.15.64:0/43323604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ae0068ae0 0x7f0ae0194ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:49.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.203+0000 7f0adeffd700 1 -- 172.21.15.64:0/43323604 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad0009520 con 0x7f0ae0068120 2023-09-13T03:11:49.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.203+0000 7f0adf7fe700 1 --2- 172.21.15.64:0/43323604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ae0106770 0x7f0ae0198f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:11:49.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.203+0000 7f0adeffd700 1 --2- 172.21.15.64:0/43323604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ae0068120 0x7f0ae01945a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0ad000eb30 tx=0x7f0ad000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:49.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.204+0000 7f0ae498b700 1 -- 172.21.15.64:0/43323604 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0009880 con 0x7f0ae0068120 2023-09-13T03:11:49.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.204+0000 7f0ae498b700 1 -- 172.21.15.64:0/43323604 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ad00044e0 con 0x7f0ae0068120 2023-09-13T03:11:49.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.204+0000 7f0ae598d700 1 -- 172.21.15.64:0/43323604 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0acc0095f0 con 0x7f0ae0068120 2023-09-13T03:11:49.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.204+0000 7f0ae498b700 1 -- 172.21.15.64:0/43323604 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0003d80 con 0x7f0ae0068120 2023-09-13T03:11:49.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.204+0000 7f0ae598d700 1 -- 172.21.15.64:0/43323604 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ae01a1200 con 0x7f0ae0068120 2023-09-13T03:11:49.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.205+0000 7f0ae598d700 1 -- 172.21.15.64:0/43323604 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0ae010e760 con 0x7f0ae0068120 2023-09-13T03:11:49.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.207+0000 7f0ae498b700 1 -- 172.21.15.64:0/43323604 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f0ad0003ee0 con 0x7f0ae0068120 2023-09-13T03:11:49.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.208+0000 7f0ae498b700 1 --2- 172.21.15.64:0/43323604 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f0ac40a0750 0x7f0ac40a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:49.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.209+0000 7f0ae498b700 1 -- 172.21.15.64:0/43323604 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f0ad0014070 con 0x7f0ae0068120 2023-09-13T03:11:49.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.209+0000 7f0ade7fc700 1 --2- 172.21.15.64:0/43323604 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f0ac40a0750 0x7f0ac40a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:49.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.209+0000 7f0ae498b700 1 --2- 172.21.15.64:0/43323604 >> [v2:172.21.15.64:6826/2641175211,v1:172.21.15.64:6827/2641175211] conn(0x7f0ac40aa210 0x7f0ac40ac6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:49.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.209+0000 7f0ae498b700 1 -- 172.21.15.64:0/43323604 --> [v2:172.21.15.64:6826/2641175211,v1:172.21.15.64:6827/2641175211] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0ac40acd80 con 0x7f0ac40aa210 2023-09-13T03:11:49.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.209+0000 7f0ae498b700 1 -- 172.21.15.64:0/43323604 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ad00cc5f0 con 0x7f0ae0068120 2023-09-13T03:11:49.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.209+0000 7f0adf7fe700 1 --2- 172.21.15.64:0/43323604 >> [v2:172.21.15.64:6826/2641175211,v1:172.21.15.64:6827/2641175211] conn(0x7f0ac40aa210 0x7f0ac40ac6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:49.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.209+0000 7f0ade7fc700 1 --2- 172.21.15.64:0/43323604 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f0ac40a0750 0x7f0ac40a2c10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0195bc0 tx=0x7f0acc005c00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:49.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.210+0000 7f0adf7fe700 1 --2- 172.21.15.64:0/43323604 >> [v2:172.21.15.64:6826/2641175211,v1:172.21.15.64:6827/2641175211] conn(0x7f0ac40aa210 0x7f0ac40ac6b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:49.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.211+0000 7f0ae498b700 1 -- 172.21.15.64:0/43323604 <== osd.9 v2:172.21.15.64:6826/2641175211 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f0ac40acd80 con 0x7f0ac40aa210 2023-09-13T03:11:49.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.229+0000 7f0ae598d700 1 -- 172.21.15.64:0/43323604 --> [v2:172.21.15.64:6826/2641175211,v1:172.21.15.64:6827/2641175211] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0ae004e300 con 0x7f0ac40aa210 2023-09-13T03:11:49.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.229+0000 7f0ae498b700 1 -- 172.21.15.64:0/43323604 <== osd.9 v2:172.21.15.64:6826/2641175211 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0ae004e300 con 0x7f0ac40aa210 2023-09-13T03:11:49.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.230+0000 7f0ae598d700 1 -- 172.21.15.64:0/43323604 >> [v2:172.21.15.64:6826/2641175211,v1:172.21.15.64:6827/2641175211] conn(0x7f0ac40aa210 msgr2=0x7f0ac40ac6b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:49.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.230+0000 7f0ae598d700 1 --2- 172.21.15.64:0/43323604 >> [v2:172.21.15.64:6826/2641175211,v1:172.21.15.64:6827/2641175211] conn(0x7f0ac40aa210 0x7f0ac40ac6b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:49.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.230+0000 7f0ae598d700 1 -- 172.21.15.64:0/43323604 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f0ac40a0750 msgr2=0x7f0ac40a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:49.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.230+0000 7f0ae598d700 1 --2- 172.21.15.64:0/43323604 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f0ac40a0750 0x7f0ac40a2c10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0195bc0 tx=0x7f0acc005c00 comp rx=0 tx=0).stop 2023-09-13T03:11:49.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.230+0000 7f0ae598d700 1 -- 172.21.15.64:0/43323604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ae0068120 msgr2=0x7f0ae01945a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:49.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.230+0000 7f0ae598d700 1 --2- 172.21.15.64:0/43323604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ae0068120 0x7f0ae01945a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0ad000eb30 tx=0x7f0ad000ee40 comp rx=0 tx=0).stop 2023-09-13T03:11:49.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.230+0000 7f0adf7fe700 1 -- 172.21.15.64:0/43323604 reap_dead start 2023-09-13T03:11:49.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.230+0000 7f0ae598d700 1 -- 172.21.15.64:0/43323604 shutdown_connections 2023-09-13T03:11:49.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.230+0000 7f0ae598d700 1 -- 172.21.15.64:0/43323604 >> 172.21.15.64:0/43323604 conn(0x7f0ae0076600 msgr2=0x7f0ae00fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:49.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.230+0000 7f0ae598d700 1 -- 172.21.15.64:0/43323604 shutdown_connections 2023-09-13T03:11:49.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.230+0000 7f0ae598d700 1 -- 172.21.15.64:0/43323604 wait complete. 2023-09-13T03:11:49.504 INFO:teuthology.orchestra.run.smithi064.stdout:141733920819 2023-09-13T03:11:49.504 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.4 2023-09-13T03:11:49.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:49 smithi064 ceph-mon[97675]: pgmap v298: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 11 op/s 2023-09-13T03:11:49.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.783+0000 7f883fca0700 1 -- 172.21.15.64:0/309076056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8838068120 msgr2=0x7f88380685a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:49.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.783+0000 7f883fca0700 1 --2- 172.21.15.64:0/309076056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8838068120 0x7f88380685a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f88280098f0 tx=0x7f8828009c00 comp rx=0 tx=0).stop 2023-09-13T03:11:49.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.783+0000 7f883fca0700 1 -- 172.21.15.64:0/309076056 shutdown_connections 2023-09-13T03:11:49.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.783+0000 7f883fca0700 1 --2- 172.21.15.64:0/309076056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8838068ae0 0x7f883810d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:49.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.783+0000 7f883fca0700 1 --2- 172.21.15.64:0/309076056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8838106770 0x7f8838106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:49.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.783+0000 7f883fca0700 1 --2- 172.21.15.64:0/309076056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8838068120 0x7f88380685a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:49.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.783+0000 7f883fca0700 1 -- 172.21.15.64:0/309076056 >> 172.21.15.64:0/309076056 conn(0x7f8838076600 msgr2=0x7f8838076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:49.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.784+0000 7f883fca0700 1 -- 172.21.15.64:0/309076056 shutdown_connections 2023-09-13T03:11:49.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.784+0000 7f883fca0700 1 -- 172.21.15.64:0/309076056 wait complete. 2023-09-13T03:11:49.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.785+0000 7f883fca0700 1 Processor -- start 2023-09-13T03:11:49.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.785+0000 7f883fca0700 1 -- start start 2023-09-13T03:11:49.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.786+0000 7f883fca0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8838068120 0x7f88380ff5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:49.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.786+0000 7f883fca0700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8838068ae0 0x7f88380ffaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:49.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.786+0000 7f883fca0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8838106770 0x7f88381a4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:49.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.786+0000 7f883fca0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88381003e0 con 0x7f8838068120 2023-09-13T03:11:49.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.786+0000 7f883fca0700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88381a4e90 con 0x7f8838068ae0 2023-09-13T03:11:49.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.786+0000 7f883fca0700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88381a5000 con 0x7f8838106770 2023-09-13T03:11:49.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.786+0000 7f883da3c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8838068120 0x7f88380ff5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:49.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.786+0000 7f883da3c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8838068120 0x7f88380ff5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46878/0 (socket says 172.21.15.64:46878) 2023-09-13T03:11:49.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.786+0000 7f883da3c700 1 -- 172.21.15.64:0/115493704 learned_addr learned my addr 172.21.15.64:0/115493704 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:49.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.786+0000 7f883d23b700 1 --2- 172.21.15.64:0/115493704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8838068ae0 0x7f88380ffaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:49.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.786+0000 7f883e23d700 1 --2- 172.21.15.64:0/115493704 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8838106770 0x7f88381a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:49.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.787+0000 7f883da3c700 1 -- 172.21.15.64:0/115493704 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8838106770 msgr2=0x7f88381a4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:49.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.787+0000 7f883da3c700 1 --2- 172.21.15.64:0/115493704 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8838106770 0x7f88381a4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:49.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.787+0000 7f883da3c700 1 -- 172.21.15.64:0/115493704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8838068ae0 msgr2=0x7f88380ffaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:49.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.787+0000 7f883da3c700 1 --2- 172.21.15.64:0/115493704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8838068ae0 0x7f88380ffaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:49.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.787+0000 7f883da3c700 1 -- 172.21.15.64:0/115493704 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8830009d00 con 0x7f8838068120 2023-09-13T03:11:49.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.787+0000 7f883d23b700 1 --2- 172.21.15.64:0/115493704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8838068ae0 0x7f88380ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:11:49.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.787+0000 7f883e23d700 1 --2- 172.21.15.64:0/115493704 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8838106770 0x7f88381a4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:11:49.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.787+0000 7f883da3c700 1 --2- 172.21.15.64:0/115493704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8838068120 0x7f88380ff5b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f883400e9a0 tx=0x7f883400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:49.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.787+0000 7f882effd700 1 -- 172.21.15.64:0/115493704 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8834009790 con 0x7f8838068120 2023-09-13T03:11:49.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.787+0000 7f883fca0700 1 -- 172.21.15.64:0/115493704 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8828009520 con 0x7f8838068120 2023-09-13T03:11:49.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.787+0000 7f882effd700 1 -- 172.21.15.64:0/115493704 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f883400cbd0 con 0x7f8838068120 2023-09-13T03:11:49.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.787+0000 7f882effd700 1 -- 172.21.15.64:0/115493704 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8834018680 con 0x7f8838068120 2023-09-13T03:11:49.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.787+0000 7f883fca0700 1 -- 172.21.15.64:0/115493704 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88381a55c0 con 0x7f8838068120 2023-09-13T03:11:49.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.789+0000 7f883fca0700 1 -- 172.21.15.64:0/115493704 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f88381108e0 con 0x7f8838068120 2023-09-13T03:11:49.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.790+0000 7f882effd700 1 -- 172.21.15.64:0/115493704 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f883400cd40 con 0x7f8838068120 2023-09-13T03:11:49.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.792+0000 7f882effd700 1 --2- 172.21.15.64:0/115493704 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f88240a06d0 0x7f88240a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:49.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.792+0000 7f882effd700 1 -- 172.21.15.64:0/115493704 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f8834014070 con 0x7f8838068120 2023-09-13T03:11:49.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.792+0000 7f883d23b700 1 --2- 172.21.15.64:0/115493704 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f88240a06d0 0x7f88240a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:49.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.792+0000 7f882effd700 1 --2- 172.21.15.64:0/115493704 >> [v2:172.21.15.142:6800/4188149333,v1:172.21.15.142:6801/4188149333] conn(0x7f88240aa190 0x7f88240ac630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:49.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.792+0000 7f882effd700 1 -- 172.21.15.64:0/115493704 --> [v2:172.21.15.142:6800/4188149333,v1:172.21.15.142:6801/4188149333] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f88240acd00 con 0x7f88240aa190 2023-09-13T03:11:49.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.792+0000 7f882effd700 1 -- 172.21.15.64:0/115493704 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88340ce3a0 con 0x7f8838068120 2023-09-13T03:11:49.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.793+0000 7f883e23d700 1 --2- 172.21.15.64:0/115493704 >> [v2:172.21.15.142:6800/4188149333,v1:172.21.15.142:6801/4188149333] conn(0x7f88240aa190 0x7f88240ac630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:49.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.793+0000 7f883d23b700 1 --2- 172.21.15.64:0/115493704 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f88240a06d0 0x7f88240a2b90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8828005e90 tx=0x7f8828019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:49.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.793+0000 7f883e23d700 1 --2- 172.21.15.64:0/115493704 >> [v2:172.21.15.142:6800/4188149333,v1:172.21.15.142:6801/4188149333] conn(0x7f88240aa190 0x7f88240ac630 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:49.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.794+0000 7f882effd700 1 -- 172.21.15.64:0/115493704 <== osd.1 v2:172.21.15.142:6800/4188149333 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f88240acd00 con 0x7f88240aa190 2023-09-13T03:11:49.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.821+0000 7f883fca0700 1 -- 172.21.15.64:0/115493704 --> [v2:172.21.15.142:6800/4188149333,v1:172.21.15.142:6801/4188149333] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f883804e300 con 0x7f88240aa190 2023-09-13T03:11:49.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.822+0000 7f882effd700 1 -- 172.21.15.64:0/115493704 <== osd.1 v2:172.21.15.142:6800/4188149333 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f883804e300 con 0x7f88240aa190 2023-09-13T03:11:49.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.823+0000 7f883fca0700 1 -- 172.21.15.64:0/115493704 >> [v2:172.21.15.142:6800/4188149333,v1:172.21.15.142:6801/4188149333] conn(0x7f88240aa190 msgr2=0x7f88240ac630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:49.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.823+0000 7f883fca0700 1 --2- 172.21.15.64:0/115493704 >> [v2:172.21.15.142:6800/4188149333,v1:172.21.15.142:6801/4188149333] conn(0x7f88240aa190 0x7f88240ac630 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:49.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.823+0000 7f883fca0700 1 -- 172.21.15.64:0/115493704 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f88240a06d0 msgr2=0x7f88240a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:49.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.823+0000 7f883fca0700 1 --2- 172.21.15.64:0/115493704 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f88240a06d0 0x7f88240a2b90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8828005e90 tx=0x7f8828019040 comp rx=0 tx=0).stop 2023-09-13T03:11:49.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.823+0000 7f883fca0700 1 -- 172.21.15.64:0/115493704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8838068120 msgr2=0x7f88380ff5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:49.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.823+0000 7f883fca0700 1 --2- 172.21.15.64:0/115493704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8838068120 0x7f88380ff5b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f883400e9a0 tx=0x7f883400ecb0 comp rx=0 tx=0).stop 2023-09-13T03:11:49.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.823+0000 7f883e23d700 1 -- 172.21.15.64:0/115493704 reap_dead start 2023-09-13T03:11:49.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.823+0000 7f883fca0700 1 -- 172.21.15.64:0/115493704 shutdown_connections 2023-09-13T03:11:49.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.823+0000 7f883fca0700 1 -- 172.21.15.64:0/115493704 >> 172.21.15.64:0/115493704 conn(0x7f8838076600 msgr2=0x7f88380fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:49.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.823+0000 7f883fca0700 1 -- 172.21.15.64:0/115493704 shutdown_connections 2023-09-13T03:11:49.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:49.823+0000 7f883fca0700 1 -- 172.21.15.64:0/115493704 wait complete. 2023-09-13T03:11:49.856 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:11:49.881 INFO:teuthology.orchestra.run.smithi064.stdout:283467841564 2023-09-13T03:11:49.881 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.9 2023-09-13T03:11:49.902 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:11:49.904 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:11:49.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:49 smithi170 ceph-mon[134328]: pgmap v298: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 11 op/s 2023-09-13T03:11:49.942 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:11:49.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:49 smithi142 ceph-mon[134082]: pgmap v298: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 11 op/s 2023-09-13T03:11:50.057 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:11:50.260 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:11:51.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:51 smithi064 ceph-mon[97675]: pgmap v299: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2023-09-13T03:11:51.753 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:11:51.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:51 smithi170 ceph-mon[134328]: pgmap v299: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2023-09-13T03:11:51.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:51 smithi142 ceph-mon[134082]: pgmap v299: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2023-09-13T03:11:52.196 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:11:52.233 INFO:teuthology.orchestra.run.smithi064.stdout:55834574912 2023-09-13T03:11:52.234 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.1 2023-09-13T03:11:52.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:52.965+0000 7fcc8ef70700 1 -- 172.21.15.64:0/155623035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc881064f0 msgr2=0x7fcc881068d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:52.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:52.965+0000 7fcc8ef70700 1 --2- 172.21.15.64:0/155623035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc881064f0 0x7fcc881068d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcc840098f0 tx=0x7fcc84009c00 comp rx=0 tx=0).stop 2023-09-13T03:11:52.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:52.965+0000 7fcc8ef70700 1 -- 172.21.15.64:0/155623035 shutdown_connections 2023-09-13T03:11:52.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:52.965+0000 7fcc8ef70700 1 --2- 172.21.15.64:0/155623035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc88100e20 0x7fcc8810d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:52.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:52.965+0000 7fcc8ef70700 1 --2- 172.21.15.64:0/155623035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc881064f0 0x7fcc881068d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:52.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:52.965+0000 7fcc8ef70700 1 --2- 172.21.15.64:0/155623035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc88100460 0x7fcc881008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:52.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:52.965+0000 7fcc8ef70700 1 -- 172.21.15.64:0/155623035 >> 172.21.15.64:0/155623035 conn(0x7fcc880fbfb0 msgr2=0x7fcc880fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:52.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:52.965+0000 7fcc8ef70700 1 -- 172.21.15.64:0/155623035 shutdown_connections 2023-09-13T03:11:52.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:52.965+0000 7fcc8ef70700 1 -- 172.21.15.64:0/155623035 wait complete. 2023-09-13T03:11:52.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:52.967+0000 7fcc8ef70700 1 Processor -- start 2023-09-13T03:11:52.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:52.967+0000 7fcc8ef70700 1 -- start start 2023-09-13T03:11:52.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:52.967+0000 7fcc8ef70700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc88100460 0x7fcc8819e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:52.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:52.968+0000 7fcc8ef70700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc88100e20 0x7fcc8819eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:52.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:52.968+0000 7fcc8ef70700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc881064f0 0x7fcc881985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:52.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:52.968+0000 7fcc8ef70700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc88198d70 con 0x7fcc881064f0 2023-09-13T03:11:52.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:52.968+0000 7fcc8ef70700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc88198ee0 con 0x7fcc88100e20 2023-09-13T03:11:52.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:52.968+0000 7fcc8ef70700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc881990a0 con 0x7fcc88100460 2023-09-13T03:11:52.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:52.968+0000 7fcc8e76f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc881064f0 0x7fcc881985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:52.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:52.968+0000 7fcc8e76f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc881064f0 0x7fcc881985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46902/0 (socket says 172.21.15.64:46902) 2023-09-13T03:11:52.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:52.968+0000 7fcc8e76f700 1 -- 172.21.15.64:0/2128013047 learned_addr learned my addr 172.21.15.64:0/2128013047 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:52.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:52.968+0000 7fcc8df6e700 1 --2- 172.21.15.64:0/2128013047 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc88100460 0x7fcc8819e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:52.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:52.968+0000 7fcc8d76d700 1 --2- 172.21.15.64:0/2128013047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc88100e20 0x7fcc8819eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:52.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:52.969+0000 7fcc8df6e700 1 -- 172.21.15.64:0/2128013047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc88100e20 msgr2=0x7fcc8819eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:52.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:52.969+0000 7fcc8df6e700 1 --2- 172.21.15.64:0/2128013047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc88100e20 0x7fcc8819eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:52.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:52.969+0000 7fcc8df6e700 1 -- 172.21.15.64:0/2128013047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc881064f0 msgr2=0x7fcc881985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:52.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:52.969+0000 7fcc8df6e700 1 --2- 172.21.15.64:0/2128013047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc881064f0 0x7fcc881985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:52.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:52.969+0000 7fcc8df6e700 1 -- 172.21.15.64:0/2128013047 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc84009520 con 0x7fcc88100460 2023-09-13T03:11:52.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:52.969+0000 7fcc8df6e700 1 --2- 172.21.15.64:0/2128013047 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc88100460 0x7fcc8819e5d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcc8400b2e0 tx=0x7fcc84010fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:52.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:52.969+0000 7fcc7effd700 1 -- 172.21.15.64:0/2128013047 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8401c070 con 0x7fcc88100460 2023-09-13T03:11:52.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:52.970+0000 7fcc8ef70700 1 -- 172.21.15.64:0/2128013047 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc88199320 con 0x7fcc88100460 2023-09-13T03:11:52.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:52.970+0000 7fcc7effd700 1 -- 172.21.15.64:0/2128013047 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc84005530 con 0x7fcc88100460 2023-09-13T03:11:52.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:52.970+0000 7fcc7effd700 1 -- 172.21.15.64:0/2128013047 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc84003aa0 con 0x7fcc88100460 2023-09-13T03:11:52.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:52.970+0000 7fcc8ef70700 1 -- 172.21.15.64:0/2128013047 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc881a5550 con 0x7fcc88100460 2023-09-13T03:11:52.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:52.971+0000 7fcc8ef70700 1 -- 172.21.15.64:0/2128013047 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc8804eb50 con 0x7fcc88100460 2023-09-13T03:11:52.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:52.975+0000 7fcc7effd700 1 -- 172.21.15.64:0/2128013047 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fcc84020070 con 0x7fcc88100460 2023-09-13T03:11:52.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:52.976+0000 7fcc7effd700 1 --2- 172.21.15.64:0/2128013047 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fcc740a0750 0x7fcc740a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:52.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:52.976+0000 7fcc7effd700 1 -- 172.21.15.64:0/2128013047 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fcc840cd0a0 con 0x7fcc88100460 2023-09-13T03:11:52.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:52.977+0000 7fcc8d76d700 1 --2- 172.21.15.64:0/2128013047 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fcc740a0750 0x7fcc740a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:52.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:52.977+0000 7fcc7effd700 1 -- 172.21.15.64:0/2128013047 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fcc84096b30 con 0x7fcc88100460 2023-09-13T03:11:52.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:52.977+0000 7fcc8d76d700 1 --2- 172.21.15.64:0/2128013047 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fcc740a0750 0x7fcc740a2c10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcc88199e60 tx=0x7fcc7800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:53.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.203+0000 7fcc8ef70700 1 -- 172.21.15.64:0/2128013047 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fcc88199af0 con 0x7fcc88100460 2023-09-13T03:11:53.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.204+0000 7fcc7effd700 1 -- 172.21.15.64:0/2128013047 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcc8409a150 con 0x7fcc88100460 2023-09-13T03:11:53.205 INFO:teuthology.orchestra.run.smithi064.stdout:219043332133 2023-09-13T03:11:53.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.206+0000 7fcc8ef70700 1 -- 172.21.15.64:0/2128013047 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fcc740a0750 msgr2=0x7fcc740a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:53.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.206+0000 7fcc8ef70700 1 --2- 172.21.15.64:0/2128013047 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fcc740a0750 0x7fcc740a2c10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcc88199e60 tx=0x7fcc7800b410 comp rx=0 tx=0).stop 2023-09-13T03:11:53.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.206+0000 7fcc8ef70700 1 -- 172.21.15.64:0/2128013047 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc88100460 msgr2=0x7fcc8819e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:53.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.206+0000 7fcc8ef70700 1 --2- 172.21.15.64:0/2128013047 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc88100460 0x7fcc8819e5d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcc8400b2e0 tx=0x7fcc84010fa0 comp rx=0 tx=0).stop 2023-09-13T03:11:53.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.206+0000 7fcc8ef70700 1 -- 172.21.15.64:0/2128013047 shutdown_connections 2023-09-13T03:11:53.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.206+0000 7fcc8ef70700 1 --2- 172.21.15.64:0/2128013047 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fcc740a0750 0x7fcc740a2c10 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:53.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.206+0000 7fcc8ef70700 1 --2- 172.21.15.64:0/2128013047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc881064f0 0x7fcc881985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:53.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.206+0000 7fcc8ef70700 1 --2- 172.21.15.64:0/2128013047 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc88100460 0x7fcc8819e5d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:53.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.206+0000 7fcc8ef70700 1 --2- 172.21.15.64:0/2128013047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc88100e20 0x7fcc8819eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:53.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.206+0000 7fcc8ef70700 1 -- 172.21.15.64:0/2128013047 >> 172.21.15.64:0/2128013047 conn(0x7fcc880fbfb0 msgr2=0x7fcc8810b810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:53.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.206+0000 7fcc8ef70700 1 -- 172.21.15.64:0/2128013047 shutdown_connections 2023-09-13T03:11:53.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.206+0000 7fcc8ef70700 1 -- 172.21.15.64:0/2128013047 wait complete. 2023-09-13T03:11:53.649 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:53 smithi064 ceph-mon[97675]: pgmap v300: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-13T03:11:53.649 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:53 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2128013047' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-09-13T03:11:53.710 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:11:53.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.871+0000 7fd107571700 1 -- 172.21.15.64:0/2126447495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd100106620 msgr2=0x7fd100106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:53.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.871+0000 7fd107571700 1 --2- 172.21.15.64:0/2126447495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd100106620 0x7fd100106a00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc009990 tx=0x7fd0fc009ca0 comp rx=0 tx=0).stop 2023-09-13T03:11:53.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.871+0000 7fd107571700 1 -- 172.21.15.64:0/2126447495 shutdown_connections 2023-09-13T03:11:53.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.871+0000 7fd107571700 1 --2- 172.21.15.64:0/2126447495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd100106620 0x7fd100106a00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:53.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.871+0000 7fd107571700 1 --2- 172.21.15.64:0/2126447495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd100100fe0 0x7fd10010d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:53.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.871+0000 7fd107571700 1 --2- 172.21.15.64:0/2126447495 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd100100620 0x7fd100100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:53.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.871+0000 7fd107571700 1 -- 172.21.15.64:0/2126447495 >> 172.21.15.64:0/2126447495 conn(0x7fd1000fc0f0 msgr2=0x7fd1000fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:53.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.871+0000 7fd107571700 1 -- 172.21.15.64:0/2126447495 shutdown_connections 2023-09-13T03:11:53.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.871+0000 7fd107571700 1 -- 172.21.15.64:0/2126447495 wait complete. 2023-09-13T03:11:53.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.872+0000 7fd107571700 1 Processor -- start 2023-09-13T03:11:53.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.872+0000 7fd107571700 1 -- start start 2023-09-13T03:11:53.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.873+0000 7fd107571700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd100100620 0x7fd10019c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:53.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.873+0000 7fd107571700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd100100fe0 0x7fd10019ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:53.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.873+0000 7fd107571700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd100106620 0x7fd100196590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:53.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.873+0000 7fd107571700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd100196c00 con 0x7fd100106620 2023-09-13T03:11:53.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.873+0000 7fd107571700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd100196d70 con 0x7fd100100fe0 2023-09-13T03:11:53.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.873+0000 7fd107571700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd100196f30 con 0x7fd100100620 2023-09-13T03:11:53.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.873+0000 7fd105b0e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd100106620 0x7fd100196590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:53.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.873+0000 7fd10530d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd100100620 0x7fd10019c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:53.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.873+0000 7fd104b0c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd100100fe0 0x7fd10019ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:53.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.873+0000 7fd105b0e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd100106620 0x7fd100196590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46922/0 (socket says 172.21.15.64:46922) 2023-09-13T03:11:53.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.873+0000 7fd10530d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd100100620 0x7fd10019c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:41994/0 (socket says 172.21.15.64:41994) 2023-09-13T03:11:53.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.873+0000 7fd105b0e700 1 -- 172.21.15.64:0/3334479273 learned_addr learned my addr 172.21.15.64:0/3334479273 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:53.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.874+0000 7fd104b0c700 1 -- 172.21.15.64:0/3334479273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd100100620 msgr2=0x7fd10019c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:53.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.874+0000 7fd104b0c700 1 --2- 172.21.15.64:0/3334479273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd100100620 0x7fd10019c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:53.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.874+0000 7fd104b0c700 1 -- 172.21.15.64:0/3334479273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd100106620 msgr2=0x7fd100196590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:53.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.874+0000 7fd104b0c700 1 --2- 172.21.15.64:0/3334479273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd100106620 0x7fd100196590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:53.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.874+0000 7fd104b0c700 1 -- 172.21.15.64:0/3334479273 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0fc0095f0 con 0x7fd100100fe0 2023-09-13T03:11:53.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.874+0000 7fd10530d700 1 --2- 172.21.15.64:0/3334479273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd100100620 0x7fd10019c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:11:53.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.874+0000 7fd104b0c700 1 --2- 172.21.15.64:0/3334479273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd100100fe0 0x7fd10019ca50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd0f000e9f0 tx=0x7fd0f000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:53.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.874+0000 7fd0f67fc700 1 -- 172.21.15.64:0/3334479273 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f000c9c0 con 0x7fd100100fe0 2023-09-13T03:11:53.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.874+0000 7fd107571700 1 -- 172.21.15.64:0/3334479273 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd100197210 con 0x7fd100100fe0 2023-09-13T03:11:53.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.874+0000 7fd0f67fc700 1 -- 172.21.15.64:0/3334479273 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0f0010430 con 0x7fd100100fe0 2023-09-13T03:11:53.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.874+0000 7fd0f67fc700 1 -- 172.21.15.64:0/3334479273 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0018680 con 0x7fd100100fe0 2023-09-13T03:11:53.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.874+0000 7fd107571700 1 -- 172.21.15.64:0/3334479273 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1001a3500 con 0x7fd100100fe0 2023-09-13T03:11:53.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.875+0000 7fd107571700 1 -- 172.21.15.64:0/3334479273 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd10004e300 con 0x7fd100100fe0 2023-09-13T03:11:53.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.877+0000 7fd0f67fc700 1 -- 172.21.15.64:0/3334479273 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fd0f0010860 con 0x7fd100100fe0 2023-09-13T03:11:53.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.879+0000 7fd0f67fc700 1 --2- 172.21.15.64:0/3334479273 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd0ec0a06d0 0x7fd0ec0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:53.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.879+0000 7fd0f67fc700 1 -- 172.21.15.64:0/3334479273 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fd0f0014070 con 0x7fd100100fe0 2023-09-13T03:11:53.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.879+0000 7fd10530d700 1 --2- 172.21.15.64:0/3334479273 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd0ec0a06d0 0x7fd0ec0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:53.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.880+0000 7fd10530d700 1 --2- 172.21.15.64:0/3334479273 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd0ec0a06d0 0x7fd0ec0a2b90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc00b2e0 tx=0x7fd0fc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:53.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.881+0000 7fd0f67fc700 1 -- 172.21.15.64:0/3334479273 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fd0f0095a70 con 0x7fd100100fe0 2023-09-13T03:11:53.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:53 smithi170 ceph-mon[134328]: pgmap v300: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-13T03:11:53.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:53 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/2128013047' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-09-13T03:11:53.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.985+0000 7f1f01f9d700 1 -- 172.21.15.64:0/2634267288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1efc111630 msgr2=0x7f1efc113a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:53.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.985+0000 7f1f01f9d700 1 --2- 172.21.15.64:0/2634267288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1efc111630 0x7f1efc113a20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4009f90 tx=0x7f1ef400b330 comp rx=0 tx=0).stop 2023-09-13T03:11:53.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.985+0000 7f1f01f9d700 1 -- 172.21.15.64:0/2634267288 shutdown_connections 2023-09-13T03:11:53.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.985+0000 7f1f01f9d700 1 --2- 172.21.15.64:0/2634267288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1efc111630 0x7f1efc113a20 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:53.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.985+0000 7f1f01f9d700 1 --2- 172.21.15.64:0/2634267288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1efc0729a0 0x7f1efc072d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:53.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.985+0000 7f1f01f9d700 1 --2- 172.21.15.64:0/2634267288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1efc0732c0 0x7f1efc1110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:53.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.985+0000 7f1f01f9d700 1 -- 172.21.15.64:0/2634267288 >> 172.21.15.64:0/2634267288 conn(0x7f1efc0fc0f0 msgr2=0x7f1efc0fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:53.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.985+0000 7f1f01f9d700 1 -- 172.21.15.64:0/2634267288 shutdown_connections 2023-09-13T03:11:53.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.985+0000 7f1f01f9d700 1 -- 172.21.15.64:0/2634267288 wait complete. 2023-09-13T03:11:53.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.986+0000 7f1f01f9d700 1 Processor -- start 2023-09-13T03:11:53.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.986+0000 7f1f01f9d700 1 -- start start 2023-09-13T03:11:53.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.986+0000 7f1f01f9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1efc0729a0 0x7f1efc19ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:53.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.986+0000 7f1f01f9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1efc0732c0 0x7f1efc19d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:53.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.986+0000 7f1f01f9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1efc111630 0x7f1efc1a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:53.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.987+0000 7f1f01f9d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc1a1e10 con 0x7f1efc111630 2023-09-13T03:11:53.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.987+0000 7f1f01f9d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc1a1f80 con 0x7f1efc0729a0 2023-09-13T03:11:53.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.987+0000 7f1f01f9d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc1a2140 con 0x7f1efc0732c0 2023-09-13T03:11:53.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.987+0000 7f1efbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1efc111630 0x7f1efc1a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:53.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.987+0000 7f1efbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1efc111630 0x7f1efc1a1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46942/0 (socket says 172.21.15.64:46942) 2023-09-13T03:11:53.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.987+0000 7f1efbfff700 1 -- 172.21.15.64:0/1305324609 learned_addr learned my addr 172.21.15.64:0/1305324609 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:53.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.987+0000 7f1efb7fe700 1 --2- 172.21.15.64:0/1305324609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1efc0729a0 0x7f1efc19ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:53.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.987+0000 7f1efaffd700 1 --2- 172.21.15.64:0/1305324609 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1efc0732c0 0x7f1efc19d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:53.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.987+0000 7f1efbfff700 1 -- 172.21.15.64:0/1305324609 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1efc0732c0 msgr2=0x7f1efc19d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:53.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.987+0000 7f1efbfff700 1 --2- 172.21.15.64:0/1305324609 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1efc0732c0 0x7f1efc19d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:53.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.987+0000 7f1efbfff700 1 -- 172.21.15.64:0/1305324609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1efc0729a0 msgr2=0x7f1efc19ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:53.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.987+0000 7f1efbfff700 1 --2- 172.21.15.64:0/1305324609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1efc0729a0 0x7f1efc19ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:53.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.987+0000 7f1efbfff700 1 -- 172.21.15.64:0/1305324609 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef4009c40 con 0x7f1efc111630 2023-09-13T03:11:53.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.988+0000 7f1efbfff700 1 --2- 172.21.15.64:0/1305324609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1efc111630 0x7f1efc1a1800 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1ef40094b0 tx=0x7f1ef40094e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:53.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.988+0000 7f1ef8ff9700 1 -- 172.21.15.64:0/1305324609 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef400e040 con 0x7f1efc111630 2023-09-13T03:11:53.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.988+0000 7f1ef8ff9700 1 -- 172.21.15.64:0/1305324609 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ef4004e70 con 0x7f1efc111630 2023-09-13T03:11:53.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.988+0000 7f1f01f9d700 1 -- 172.21.15.64:0/1305324609 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1efc1a23c0 con 0x7f1efc111630 2023-09-13T03:11:53.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.988+0000 7f1ef8ff9700 1 -- 172.21.15.64:0/1305324609 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4005040 con 0x7f1efc111630 2023-09-13T03:11:53.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.988+0000 7f1f01f9d700 1 -- 172.21.15.64:0/1305324609 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1efc1a9c80 con 0x7f1efc111630 2023-09-13T03:11:53.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.989+0000 7f1f01f9d700 1 -- 172.21.15.64:0/1305324609 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1efc103b10 con 0x7f1efc111630 2023-09-13T03:11:53.999 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:53 smithi142 ceph-mon[134082]: pgmap v300: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-13T03:11:53.999 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:53 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/2128013047' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-09-13T03:11:54.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.991+0000 7f1ef8ff9700 1 -- 172.21.15.64:0/1305324609 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f1ef401d470 con 0x7f1efc111630 2023-09-13T03:11:54.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.994+0000 7f1ef8ff9700 1 --2- 172.21.15.64:0/1305324609 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f1ee40a06d0 0x7f1ee40a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:54.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.994+0000 7f1ef8ff9700 1 -- 172.21.15.64:0/1305324609 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f1ef40cd5f0 con 0x7f1efc111630 2023-09-13T03:11:54.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.994+0000 7f1efb7fe700 1 --2- 172.21.15.64:0/1305324609 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f1ee40a06d0 0x7f1ee40a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:54.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.995+0000 7f1efb7fe700 1 --2- 172.21.15.64:0/1305324609 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f1ee40a06d0 0x7f1ee40a2b90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1eec005fd0 tx=0x7f1eec009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:54.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:53.996+0000 7f1ef8ff9700 1 -- 172.21.15.64:0/1305324609 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1ef4097140 con 0x7f1efc111630 2023-09-13T03:11:54.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:54.108+0000 7fd107571700 1 -- 172.21.15.64:0/3334479273 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd100110690 con 0x7fd100100fe0 2023-09-13T03:11:54.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:54.108+0000 7fd0f67fc700 1 -- 172.21.15.64:0/3334479273 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd0f0099090 con 0x7fd100100fe0 2023-09-13T03:11:54.109 INFO:teuthology.orchestra.run.smithi064.stdout:107374182456 2023-09-13T03:11:54.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:54.111+0000 7fd107571700 1 -- 172.21.15.64:0/3334479273 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd0ec0a06d0 msgr2=0x7fd0ec0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:54.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:54.111+0000 7fd107571700 1 --2- 172.21.15.64:0/3334479273 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd0ec0a06d0 0x7fd0ec0a2b90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc00b2e0 tx=0x7fd0fc005fb0 comp rx=0 tx=0).stop 2023-09-13T03:11:54.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:54.111+0000 7fd107571700 1 -- 172.21.15.64:0/3334479273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd100100fe0 msgr2=0x7fd10019ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:54.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:54.111+0000 7fd107571700 1 --2- 172.21.15.64:0/3334479273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd100100fe0 0x7fd10019ca50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd0f000e9f0 tx=0x7fd0f000ed00 comp rx=0 tx=0).stop 2023-09-13T03:11:54.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:54.111+0000 7fd107571700 1 -- 172.21.15.64:0/3334479273 shutdown_connections 2023-09-13T03:11:54.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:54.111+0000 7fd107571700 1 --2- 172.21.15.64:0/3334479273 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd0ec0a06d0 0x7fd0ec0a2b90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:54.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:54.111+0000 7fd107571700 1 --2- 172.21.15.64:0/3334479273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd100106620 0x7fd100196590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:54.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:54.111+0000 7fd107571700 1 --2- 172.21.15.64:0/3334479273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd100100620 0x7fd10019c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:54.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:54.111+0000 7fd107571700 1 --2- 172.21.15.64:0/3334479273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd100100fe0 0x7fd10019ca50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:54.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:54.111+0000 7fd107571700 1 -- 172.21.15.64:0/3334479273 >> 172.21.15.64:0/3334479273 conn(0x7fd1000fc0f0 msgr2=0x7fd1000fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:54.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:54.111+0000 7fd107571700 1 -- 172.21.15.64:0/3334479273 shutdown_connections 2023-09-13T03:11:54.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:54.111+0000 7fd107571700 1 -- 172.21.15.64:0/3334479273 wait complete. 2023-09-13T03:11:54.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:54.234+0000 7f1f01f9d700 1 -- 172.21.15.64:0/1305324609 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f1efc04e300 con 0x7f1efc111630 2023-09-13T03:11:54.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:54.235+0000 7f1ef8ff9700 1 -- 172.21.15.64:0/1305324609 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f1ef409a760 con 0x7f1efc111630 2023-09-13T03:11:54.236 INFO:teuthology.orchestra.run.smithi064.stdout:335007449108 2023-09-13T03:11:54.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:54.238+0000 7f1f01f9d700 1 -- 172.21.15.64:0/1305324609 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f1ee40a06d0 msgr2=0x7f1ee40a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:54.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:54.238+0000 7f1f01f9d700 1 --2- 172.21.15.64:0/1305324609 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f1ee40a06d0 0x7f1ee40a2b90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1eec005fd0 tx=0x7f1eec009310 comp rx=0 tx=0).stop 2023-09-13T03:11:54.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:54.238+0000 7f1f01f9d700 1 -- 172.21.15.64:0/1305324609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1efc111630 msgr2=0x7f1efc1a1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:54.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:54.238+0000 7f1f01f9d700 1 --2- 172.21.15.64:0/1305324609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1efc111630 0x7f1efc1a1800 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1ef40094b0 tx=0x7f1ef40094e0 comp rx=0 tx=0).stop 2023-09-13T03:11:54.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:54.238+0000 7f1f01f9d700 1 -- 172.21.15.64:0/1305324609 shutdown_connections 2023-09-13T03:11:54.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:54.238+0000 7f1f01f9d700 1 --2- 172.21.15.64:0/1305324609 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f1ee40a06d0 0x7f1ee40a2b90 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:54.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:54.238+0000 7f1f01f9d700 1 --2- 172.21.15.64:0/1305324609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1efc0729a0 0x7f1efc19ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:54.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:54.238+0000 7f1f01f9d700 1 --2- 172.21.15.64:0/1305324609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1efc111630 0x7f1efc1a1800 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:54.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:54.238+0000 7f1f01f9d700 1 --2- 172.21.15.64:0/1305324609 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1efc0732c0 0x7f1efc19d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:54.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:54.238+0000 7f1f01f9d700 1 -- 172.21.15.64:0/1305324609 >> 172.21.15.64:0/1305324609 conn(0x7f1efc0fc0f0 msgr2=0x7f1efc0fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:54.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:54.238+0000 7f1f01f9d700 1 -- 172.21.15.64:0/1305324609 shutdown_connections 2023-09-13T03:11:54.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:54.238+0000 7f1f01f9d700 1 -- 172.21.15.64:0/1305324609 wait complete. 2023-09-13T03:11:54.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:54 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/3334479273' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-09-13T03:11:54.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:54 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/1305324609' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-09-13T03:11:54.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:54 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/3334479273' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-09-13T03:11:54.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:54 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/1305324609' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-09-13T03:11:54.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:54 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/3334479273' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-09-13T03:11:54.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:54 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/1305324609' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-09-13T03:11:55.320 INFO:tasks.cephadm.ceph_manager.ceph:need seq 335007449106 got 335007449108 for osd.11 2023-09-13T03:11:55.320 DEBUG:teuthology.parallel:result is None 2023-09-13T03:11:55.478 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332132 got 219043332133 for osd.7 2023-09-13T03:11:55.478 DEBUG:teuthology.parallel:result is None 2023-09-13T03:11:55.649 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:55 smithi064 ceph-mon[97675]: pgmap v301: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-13T03:11:55.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.651+0000 7f157dadb700 1 -- 172.21.15.64:0/3754388556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1578068ae0 msgr2=0x7f157810d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:55.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.651+0000 7f157dadb700 1 --2- 172.21.15.64:0/3754388556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1578068ae0 0x7f157810d440 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f157000b0a0 tx=0x7f157000b3b0 comp rx=0 tx=0).stop 2023-09-13T03:11:55.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.652+0000 7f157dadb700 1 -- 172.21.15.64:0/3754388556 shutdown_connections 2023-09-13T03:11:55.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.652+0000 7f157dadb700 1 --2- 172.21.15.64:0/3754388556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1578068ae0 0x7f157810d440 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:55.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.652+0000 7f157dadb700 1 --2- 172.21.15.64:0/3754388556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1578106770 0x7f1578106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:55.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.652+0000 7f157dadb700 1 --2- 172.21.15.64:0/3754388556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1578068120 0x7f15780685a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:55.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.652+0000 7f157dadb700 1 -- 172.21.15.64:0/3754388556 >> 172.21.15.64:0/3754388556 conn(0x7f1578076600 msgr2=0x7f1578076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:55.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.652+0000 7f157dadb700 1 -- 172.21.15.64:0/3754388556 shutdown_connections 2023-09-13T03:11:55.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.652+0000 7f157dadb700 1 -- 172.21.15.64:0/3754388556 wait complete. 2023-09-13T03:11:55.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.653+0000 7f157dadb700 1 Processor -- start 2023-09-13T03:11:55.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.654+0000 7f157dadb700 1 -- start start 2023-09-13T03:11:55.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.654+0000 7f157dadb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1578068120 0x7f157819e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:55.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.654+0000 7f157dadb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1578068ae0 0x7f157819ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:55.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.654+0000 7f157dadb700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1578106770 0x7f1578198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:55.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.654+0000 7f157dadb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1578198eb0 con 0x7f1578068120 2023-09-13T03:11:55.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.654+0000 7f157dadb700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1578199020 con 0x7f1578068ae0 2023-09-13T03:11:55.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.654+0000 7f157dadb700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15781991e0 con 0x7f1578106770 2023-09-13T03:11:55.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.655+0000 7f15777fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1578068120 0x7f157819e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:55.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.655+0000 7f15777fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1578068120 0x7f157819e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46958/0 (socket says 172.21.15.64:46958) 2023-09-13T03:11:55.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.655+0000 7f15777fe700 1 -- 172.21.15.64:0/2078965210 learned_addr learned my addr 172.21.15.64:0/2078965210 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:55.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.655+0000 7f1577fff700 1 --2- 172.21.15.64:0/2078965210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1578106770 0x7f1578198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:55.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.655+0000 7f1576ffd700 1 --2- 172.21.15.64:0/2078965210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1578068ae0 0x7f157819ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:55.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.655+0000 7f1577fff700 1 -- 172.21.15.64:0/2078965210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1578068ae0 msgr2=0x7f157819ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:55.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.655+0000 7f1577fff700 1 --2- 172.21.15.64:0/2078965210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1578068ae0 0x7f157819ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:55.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.655+0000 7f1577fff700 1 -- 172.21.15.64:0/2078965210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1578068120 msgr2=0x7f157819e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:55.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.655+0000 7f1577fff700 1 --2- 172.21.15.64:0/2078965210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1578068120 0x7f157819e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:55.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.655+0000 7f1577fff700 1 -- 172.21.15.64:0/2078965210 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1570009d00 con 0x7f1578106770 2023-09-13T03:11:55.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.655+0000 7f15777fe700 1 --2- 172.21.15.64:0/2078965210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1578068120 0x7f157819e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:11:55.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.656+0000 7f1577fff700 1 --2- 172.21.15.64:0/2078965210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1578106770 0x7f1578198840 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1570009fd0 tx=0x7f1570007880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:55.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.656+0000 7f1574ff9700 1 -- 172.21.15.64:0/2078965210 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157000e040 con 0x7f1578106770 2023-09-13T03:11:55.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.656+0000 7f1574ff9700 1 -- 172.21.15.64:0/2078965210 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f157001d470 con 0x7f1578106770 2023-09-13T03:11:55.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.656+0000 7f157dadb700 1 -- 172.21.15.64:0/2078965210 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1578199460 con 0x7f1578106770 2023-09-13T03:11:55.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.656+0000 7f1574ff9700 1 -- 172.21.15.64:0/2078965210 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15700052a0 con 0x7f1578106770 2023-09-13T03:11:55.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.657+0000 7f157dadb700 1 -- 172.21.15.64:0/2078965210 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15781a57b0 con 0x7f1578106770 2023-09-13T03:11:55.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.659+0000 7f157dadb700 1 -- 172.21.15.64:0/2078965210 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f157804e300 con 0x7f1578106770 2023-09-13T03:11:55.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.662+0000 7f1574ff9700 1 -- 172.21.15.64:0/2078965210 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f157001d5e0 con 0x7f1578106770 2023-09-13T03:11:55.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.664+0000 7f1574ff9700 1 --2- 172.21.15.64:0/2078965210 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f15600a0750 0x7f15600a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:55.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.664+0000 7f1574ff9700 1 -- 172.21.15.64:0/2078965210 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f15700cc600 con 0x7f1578106770 2023-09-13T03:11:55.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.664+0000 7f15777fe700 1 --2- 172.21.15.64:0/2078965210 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f15600a0750 0x7f15600a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:55.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.665+0000 7f15777fe700 1 --2- 172.21.15.64:0/2078965210 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f15600a0750 0x7f15600a2c10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1568011550 tx=0x7f1568009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:55.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.668+0000 7f1574ff9700 1 -- 172.21.15.64:0/2078965210 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1570096090 con 0x7f1578106770 2023-09-13T03:11:55.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.908+0000 7f157dadb700 1 -- 172.21.15.64:0/2078965210 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f15781106c0 con 0x7f1578106770 2023-09-13T03:11:55.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:55 smithi170 ceph-mon[134328]: pgmap v301: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-13T03:11:55.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.909+0000 7f1574ff9700 1 -- 172.21.15.64:0/2078965210 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f15700996b0 con 0x7f1578106770 2023-09-13T03:11:55.911 INFO:teuthology.orchestra.run.smithi064.stdout:77309411389 2023-09-13T03:11:55.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.912+0000 7f157dadb700 1 -- 172.21.15.64:0/2078965210 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f15600a0750 msgr2=0x7f15600a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:55.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.912+0000 7f157dadb700 1 --2- 172.21.15.64:0/2078965210 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f15600a0750 0x7f15600a2c10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1568011550 tx=0x7f1568009260 comp rx=0 tx=0).stop 2023-09-13T03:11:55.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.912+0000 7f157dadb700 1 -- 172.21.15.64:0/2078965210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1578106770 msgr2=0x7f1578198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:55.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.912+0000 7f157dadb700 1 --2- 172.21.15.64:0/2078965210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1578106770 0x7f1578198840 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1570009fd0 tx=0x7f1570007880 comp rx=0 tx=0).stop 2023-09-13T03:11:55.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.912+0000 7f157dadb700 1 -- 172.21.15.64:0/2078965210 shutdown_connections 2023-09-13T03:11:55.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.912+0000 7f157dadb700 1 --2- 172.21.15.64:0/2078965210 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f15600a0750 0x7f15600a2c10 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:55.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.912+0000 7f157dadb700 1 --2- 172.21.15.64:0/2078965210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1578106770 0x7f1578198840 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:55.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.912+0000 7f157dadb700 1 --2- 172.21.15.64:0/2078965210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1578068120 0x7f157819e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:55.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.912+0000 7f157dadb700 1 --2- 172.21.15.64:0/2078965210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1578068ae0 0x7f157819ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:55.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.912+0000 7f157dadb700 1 -- 172.21.15.64:0/2078965210 >> 172.21.15.64:0/2078965210 conn(0x7f1578076600 msgr2=0x7f15780fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:55.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.913+0000 7f157dadb700 1 -- 172.21.15.64:0/2078965210 shutdown_connections 2023-09-13T03:11:55.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.913+0000 7f157dadb700 1 -- 172.21.15.64:0/2078965210 wait complete. 2023-09-13T03:11:55.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.991+0000 7f939acb2700 1 -- 172.21.15.64:0/2407842237 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9394106660 msgr2=0x7f9394106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:55.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.991+0000 7f939acb2700 1 --2- 172.21.15.64:0/2407842237 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9394106660 0x7f9394106a40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f93880099c0 tx=0x7f9388009cd0 comp rx=0 tx=0).stop 2023-09-13T03:11:55.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.991+0000 7f939acb2700 1 -- 172.21.15.64:0/2407842237 shutdown_connections 2023-09-13T03:11:55.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.991+0000 7f939acb2700 1 --2- 172.21.15.64:0/2407842237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9394068b20 0x7f939410d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:55.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.991+0000 7f939acb2700 1 --2- 172.21.15.64:0/2407842237 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9394106660 0x7f9394106a40 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:55.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.991+0000 7f939acb2700 1 --2- 172.21.15.64:0/2407842237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9394068160 0x7f93940685e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:55.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.991+0000 7f939acb2700 1 -- 172.21.15.64:0/2407842237 >> 172.21.15.64:0/2407842237 conn(0x7f9394076650 msgr2=0x7f9394076a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:55.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.991+0000 7f939acb2700 1 -- 172.21.15.64:0/2407842237 shutdown_connections 2023-09-13T03:11:55.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.991+0000 7f939acb2700 1 -- 172.21.15.64:0/2407842237 wait complete. 2023-09-13T03:11:55.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.993+0000 7f939acb2700 1 Processor -- start 2023-09-13T03:11:55.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.993+0000 7f939acb2700 1 -- start start 2023-09-13T03:11:55.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.994+0000 7f939acb2700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9394068160 0x7f93940ff540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:55.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.994+0000 7f939acb2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9394068b20 0x7f93940ffa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:55.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.994+0000 7f939acb2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9394106660 0x7f9394100000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:55.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.994+0000 7f939acb2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93941a4ba0 con 0x7f9394106660 2023-09-13T03:11:55.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.994+0000 7f939acb2700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93941a4d10 con 0x7f9394068160 2023-09-13T03:11:55.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.994+0000 7f939acb2700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93941a4ed0 con 0x7f9394068b20 2023-09-13T03:11:55.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.994+0000 7f9398a4e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9394068160 0x7f93940ff540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:55.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.994+0000 7f9398a4e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9394068160 0x7f93940ff540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:41132/0 (socket says 172.21.15.64:41132) 2023-09-13T03:11:55.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.994+0000 7f9398a4e700 1 -- 172.21.15.64:0/3757157133 learned_addr learned my addr 172.21.15.64:0/3757157133 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:55.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.994+0000 7f9393fff700 1 --2- 172.21.15.64:0/3757157133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9394068b20 0x7f93940ffa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:55.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.994+0000 7f939924f700 1 --2- 172.21.15.64:0/3757157133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9394106660 0x7f9394100000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:55.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.994+0000 7f9398a4e700 1 -- 172.21.15.64:0/3757157133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9394068b20 msgr2=0x7f93940ffa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:55.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.994+0000 7f9398a4e700 1 --2- 172.21.15.64:0/3757157133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9394068b20 0x7f93940ffa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:55.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.994+0000 7f9398a4e700 1 -- 172.21.15.64:0/3757157133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9394106660 msgr2=0x7f9394100000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:55.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.994+0000 7f9398a4e700 1 --2- 172.21.15.64:0/3757157133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9394106660 0x7f9394100000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:55.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.994+0000 7f9398a4e700 1 -- 172.21.15.64:0/3757157133 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9380009520 con 0x7f9394068160 2023-09-13T03:11:55.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.995+0000 7f9398a4e700 1 --2- 172.21.15.64:0/3757157133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9394068160 0x7f93940ff540 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f938800b580 tx=0x7f938800bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:55.999 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:55 smithi142 ceph-mon[134082]: pgmap v301: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-13T03:11:56.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.995+0000 7f9391ffb700 1 -- 172.21.15.64:0/3757157133 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938801d070 con 0x7f9394068160 2023-09-13T03:11:56.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.995+0000 7f9391ffb700 1 -- 172.21.15.64:0/3757157133 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9388005050 con 0x7f9394068160 2023-09-13T03:11:56.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.995+0000 7f939acb2700 1 -- 172.21.15.64:0/3757157133 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93880095f0 con 0x7f9394068160 2023-09-13T03:11:56.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.995+0000 7f9391ffb700 1 -- 172.21.15.64:0/3757157133 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9388004a40 con 0x7f9394068160 2023-09-13T03:11:56.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.995+0000 7f939acb2700 1 -- 172.21.15.64:0/3757157133 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93941a5430 con 0x7f9394068160 2023-09-13T03:11:56.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.997+0000 7f939acb2700 1 -- 172.21.15.64:0/3757157133 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f939404e300 con 0x7f9394068160 2023-09-13T03:11:56.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:55.999+0000 7f9391ffb700 1 -- 172.21.15.64:0/3757157133 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f93880051c0 con 0x7f9394068160 2023-09-13T03:11:56.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.001+0000 7f9391ffb700 1 --2- 172.21.15.64:0/3757157133 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f93840a0750 0x7f93840a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:56.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.001+0000 7f9391ffb700 1 -- 172.21.15.64:0/3757157133 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f93880ccb20 con 0x7f9394068160 2023-09-13T03:11:56.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.002+0000 7f9393fff700 1 --2- 172.21.15.64:0/3757157133 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f93840a0750 0x7f93840a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:56.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.002+0000 7f9393fff700 1 --2- 172.21.15.64:0/3757157133 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f93840a0750 0x7f93840a2c10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9394100d30 tx=0x7f9380009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:56.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.004+0000 7f9391ffb700 1 -- 172.21.15.64:0/3757157133 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f9388096550 con 0x7f9394068160 2023-09-13T03:11:56.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.276+0000 7f939acb2700 1 -- 172.21.15.64:0/3757157133 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f9394110580 con 0x7f9394068160 2023-09-13T03:11:56.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.276+0000 7f9391ffb700 1 -- 172.21.15.64:0/3757157133 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9388099b70 con 0x7f9394068160 2023-09-13T03:11:56.277 INFO:teuthology.orchestra.run.smithi064.stdout:197568495659 2023-09-13T03:11:56.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.279+0000 7f939acb2700 1 -- 172.21.15.64:0/3757157133 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f93840a0750 msgr2=0x7f93840a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:56.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.279+0000 7f939acb2700 1 --2- 172.21.15.64:0/3757157133 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f93840a0750 0x7f93840a2c10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9394100d30 tx=0x7f9380009260 comp rx=0 tx=0).stop 2023-09-13T03:11:56.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.279+0000 7f939acb2700 1 -- 172.21.15.64:0/3757157133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9394068160 msgr2=0x7f93940ff540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:56.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.279+0000 7f939acb2700 1 --2- 172.21.15.64:0/3757157133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9394068160 0x7f93940ff540 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f938800b580 tx=0x7f938800bfd0 comp rx=0 tx=0).stop 2023-09-13T03:11:56.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.279+0000 7f939acb2700 1 -- 172.21.15.64:0/3757157133 shutdown_connections 2023-09-13T03:11:56.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.279+0000 7f939acb2700 1 --2- 172.21.15.64:0/3757157133 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f93840a0750 0x7f93840a2c10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:56.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.279+0000 7f939acb2700 1 --2- 172.21.15.64:0/3757157133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9394068160 0x7f93940ff540 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:56.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.279+0000 7f939acb2700 1 --2- 172.21.15.64:0/3757157133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9394106660 0x7f9394100000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:56.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.279+0000 7f939acb2700 1 --2- 172.21.15.64:0/3757157133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9394068b20 0x7f93940ffa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:56.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.279+0000 7f939acb2700 1 -- 172.21.15.64:0/3757157133 >> 172.21.15.64:0/3757157133 conn(0x7f9394076650 msgr2=0x7f9394104320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:56.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.279+0000 7f939acb2700 1 -- 172.21.15.64:0/3757157133 shutdown_connections 2023-09-13T03:11:56.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.279+0000 7f939acb2700 1 -- 172.21.15.64:0/3757157133 wait complete. 2023-09-13T03:11:56.860 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:56 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2078965210' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-09-13T03:11:56.860 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:56 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/3757157133' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-09-13T03:11:56.861 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:56 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/545022662' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-09-13T03:11:56.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:56 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/2078965210' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-09-13T03:11:56.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:56 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/3757157133' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-09-13T03:11:56.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:56 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/545022662' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-09-13T03:11:56.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:56 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/2078965210' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-09-13T03:11:56.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:56 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/3757157133' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-09-13T03:11:56.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:56 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/545022662' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-09-13T03:11:57.078 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411387 got 77309411389 for osd.2 2023-09-13T03:11:57.078 DEBUG:teuthology.parallel:result is None 2023-09-13T03:11:57.108 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182455 got 107374182456 for osd.3 2023-09-13T03:11:57.109 DEBUG:teuthology.parallel:result is None 2023-09-13T03:11:57.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.175+0000 7fb85d500700 1 -- 172.21.15.64:0/893760446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb858100fe0 msgr2=0x7fb85810d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:57.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.175+0000 7fb85d500700 1 --2- 172.21.15.64:0/893760446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb858100fe0 0x7fb85810d4f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb850009f90 tx=0x7fb85000b330 comp rx=0 tx=0).stop 2023-09-13T03:11:57.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.175+0000 7fb85d500700 1 -- 172.21.15.64:0/893760446 shutdown_connections 2023-09-13T03:11:57.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.175+0000 7fb85d500700 1 --2- 172.21.15.64:0/893760446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb858100fe0 0x7fb85810d4f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:57.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.175+0000 7fb85d500700 1 --2- 172.21.15.64:0/893760446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb858106620 0x7fb858106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:57.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.175+0000 7fb85d500700 1 --2- 172.21.15.64:0/893760446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb858100620 0x7fb858100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:57.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.175+0000 7fb85d500700 1 -- 172.21.15.64:0/893760446 >> 172.21.15.64:0/893760446 conn(0x7fb8580fc0f0 msgr2=0x7fb8580fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:57.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.176+0000 7fb85d500700 1 -- 172.21.15.64:0/893760446 shutdown_connections 2023-09-13T03:11:57.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.176+0000 7fb85d500700 1 -- 172.21.15.64:0/893760446 wait complete. 2023-09-13T03:11:57.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.177+0000 7fb85d500700 1 Processor -- start 2023-09-13T03:11:57.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.177+0000 7fb85d500700 1 -- start start 2023-09-13T03:11:57.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.177+0000 7fb85d500700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb858100620 0x7fb858072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:57.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.177+0000 7fb85d500700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb858100fe0 0x7fb858072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:57.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.177+0000 7fb85d500700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb858106620 0x7fb8580733c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:57.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.177+0000 7fb85d500700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8581a4cc0 con 0x7fb858100620 2023-09-13T03:11:57.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.177+0000 7fb85d500700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8581a4e30 con 0x7fb858106620 2023-09-13T03:11:57.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.177+0000 7fb85d500700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8581a4ff0 con 0x7fb858100fe0 2023-09-13T03:11:57.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.178+0000 7fb856ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb858100620 0x7fb858072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:57.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.178+0000 7fb8567fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb858100fe0 0x7fb858072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:57.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.178+0000 7fb8567fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb858100fe0 0x7fb858072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:53110/0 (socket says 172.21.15.64:53110) 2023-09-13T03:11:57.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.178+0000 7fb8567fc700 1 -- 172.21.15.64:0/545022662 learned_addr learned my addr 172.21.15.64:0/545022662 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:57.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.178+0000 7fb856ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb858100620 0x7fb858072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53902/0 (socket says 172.21.15.64:53902) 2023-09-13T03:11:57.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.178+0000 7fb8577fe700 1 --2- 172.21.15.64:0/545022662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb858106620 0x7fb8580733c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:57.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.178+0000 7fb8567fc700 1 -- 172.21.15.64:0/545022662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb858106620 msgr2=0x7fb8580733c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:57.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.178+0000 7fb8567fc700 1 --2- 172.21.15.64:0/545022662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb858106620 0x7fb8580733c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:57.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.178+0000 7fb8567fc700 1 -- 172.21.15.64:0/545022662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb858100620 msgr2=0x7fb858072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:57.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.178+0000 7fb8567fc700 1 --2- 172.21.15.64:0/545022662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb858100620 0x7fb858072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:57.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.178+0000 7fb8567fc700 1 -- 172.21.15.64:0/545022662 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb850009c40 con 0x7fb858100fe0 2023-09-13T03:11:57.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.179+0000 7fb8567fc700 1 --2- 172.21.15.64:0/545022662 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb858100fe0 0x7fb858072e40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00e9a0 tx=0x7fb84c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:57.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.179+0000 7fb83ffff700 1 -- 172.21.15.64:0/545022662 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84c009790 con 0x7fb858100fe0 2023-09-13T03:11:57.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.179+0000 7fb85d500700 1 -- 172.21.15.64:0/545022662 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8581a52d0 con 0x7fb858100fe0 2023-09-13T03:11:57.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.179+0000 7fb83ffff700 1 -- 172.21.15.64:0/545022662 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb84c00ce80 con 0x7fb858100fe0 2023-09-13T03:11:57.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.179+0000 7fb83ffff700 1 -- 172.21.15.64:0/545022662 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84c018680 con 0x7fb858100fe0 2023-09-13T03:11:57.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.179+0000 7fb85d500700 1 -- 172.21.15.64:0/545022662 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8581a57d0 con 0x7fb858100fe0 2023-09-13T03:11:57.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.180+0000 7fb85d500700 1 -- 172.21.15.64:0/545022662 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb85804e300 con 0x7fb858100fe0 2023-09-13T03:11:57.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.182+0000 7fb83ffff700 1 -- 172.21.15.64:0/545022662 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fb84c010430 con 0x7fb858100fe0 2023-09-13T03:11:57.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.185+0000 7fb83ffff700 1 --2- 172.21.15.64:0/545022662 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fb8400a06d0 0x7fb8400a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:57.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.185+0000 7fb83ffff700 1 -- 172.21.15.64:0/545022662 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb84c014070 con 0x7fb858100fe0 2023-09-13T03:11:57.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.185+0000 7fb856ffd700 1 --2- 172.21.15.64:0/545022662 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fb8400a06d0 0x7fb8400a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:57.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.186+0000 7fb83ffff700 1 -- 172.21.15.64:0/545022662 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb84c095ce0 con 0x7fb858100fe0 2023-09-13T03:11:57.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.186+0000 7fb856ffd700 1 --2- 172.21.15.64:0/545022662 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fb8400a06d0 0x7fb8400a2b90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb848005fd0 tx=0x7fb848009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:57.138 INFO:teuthology.orchestra.run.smithi064.stdout:34359738438 2023-09-13T03:11:57.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.442+0000 7fb85d500700 1 -- 172.21.15.64:0/545022662 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb8581106f0 con 0x7fb858100fe0 2023-09-13T03:11:57.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.443+0000 7fb83ffff700 1 -- 172.21.15.64:0/545022662 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb84c099300 con 0x7fb858100fe0 2023-09-13T03:11:57.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.447+0000 7fb85d500700 1 -- 172.21.15.64:0/545022662 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fb8400a06d0 msgr2=0x7fb8400a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:57.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.447+0000 7fb85d500700 1 --2- 172.21.15.64:0/545022662 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fb8400a06d0 0x7fb8400a2b90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb848005fd0 tx=0x7fb848009310 comp rx=0 tx=0).stop 2023-09-13T03:11:57.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.447+0000 7fb85d500700 1 -- 172.21.15.64:0/545022662 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb858100fe0 msgr2=0x7fb858072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:57.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.447+0000 7fb85d500700 1 --2- 172.21.15.64:0/545022662 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb858100fe0 0x7fb858072e40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00e9a0 tx=0x7fb84c00ecb0 comp rx=0 tx=0).stop 2023-09-13T03:11:57.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.447+0000 7fb85d500700 1 -- 172.21.15.64:0/545022662 shutdown_connections 2023-09-13T03:11:57.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.447+0000 7fb85d500700 1 --2- 172.21.15.64:0/545022662 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fb8400a06d0 0x7fb8400a2b90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:57.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.447+0000 7fb85d500700 1 --2- 172.21.15.64:0/545022662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb858106620 0x7fb8580733c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:57.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.447+0000 7fb85d500700 1 --2- 172.21.15.64:0/545022662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb858100620 0x7fb858072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:57.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.447+0000 7fb85d500700 1 --2- 172.21.15.64:0/545022662 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb858100fe0 0x7fb858072e40 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:57.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.447+0000 7fb85d500700 1 -- 172.21.15.64:0/545022662 >> 172.21.15.64:0/545022662 conn(0x7fb8580fc0f0 msgr2=0x7fb858104130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:57.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.447+0000 7fb85d500700 1 -- 172.21.15.64:0/545022662 shutdown_connections 2023-09-13T03:11:57.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.447+0000 7fb85d500700 1 -- 172.21.15.64:0/545022662 wait complete. 2023-09-13T03:11:57.144 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495657 got 197568495659 for osd.6 2023-09-13T03:11:57.145 DEBUG:teuthology.parallel:result is None 2023-09-13T03:11:57.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.610+0000 7ff6a2de3700 1 -- 172.21.15.64:0/2628821825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff69c068120 msgr2=0x7ff69c0685a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:57.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.610+0000 7ff6a2de3700 1 --2- 172.21.15.64:0/2628821825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff69c068120 0x7ff69c0685a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff688009990 tx=0x7ff688009ca0 comp rx=0 tx=0).stop 2023-09-13T03:11:57.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.610+0000 7ff6a2de3700 1 -- 172.21.15.64:0/2628821825 shutdown_connections 2023-09-13T03:11:57.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.610+0000 7ff6a2de3700 1 --2- 172.21.15.64:0/2628821825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff69c068ae0 0x7ff69c10d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:57.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.610+0000 7ff6a2de3700 1 --2- 172.21.15.64:0/2628821825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff69c106770 0x7ff69c106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:57.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.610+0000 7ff6a2de3700 1 --2- 172.21.15.64:0/2628821825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff69c068120 0x7ff69c0685a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:57.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.610+0000 7ff6a2de3700 1 -- 172.21.15.64:0/2628821825 >> 172.21.15.64:0/2628821825 conn(0x7ff69c076600 msgr2=0x7ff69c076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:57.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.610+0000 7ff6a2de3700 1 -- 172.21.15.64:0/2628821825 shutdown_connections 2023-09-13T03:11:57.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.610+0000 7ff6a2de3700 1 -- 172.21.15.64:0/2628821825 wait complete. 2023-09-13T03:11:57.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.611+0000 7ff6a2de3700 1 Processor -- start 2023-09-13T03:11:57.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.611+0000 7ff6a2de3700 1 -- start start 2023-09-13T03:11:57.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.612+0000 7ff6a2de3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff69c068120 0x7ff69c19e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:57.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.612+0000 7ff6a2de3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff69c068ae0 0x7ff69c19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:57.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.612+0000 7ff6a2de3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff69c106770 0x7ff69c198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:57.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.612+0000 7ff6a2de3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c198d80 con 0x7ff69c068ae0 2023-09-13T03:11:57.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.612+0000 7ff6a2de3700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c198ef0 con 0x7ff69c106770 2023-09-13T03:11:57.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.612+0000 7ff6a2de3700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c1990b0 con 0x7ff69c068120 2023-09-13T03:11:57.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.612+0000 7ff69bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff69c068ae0 0x7ff69c19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:57.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.612+0000 7ff69bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff69c068ae0 0x7ff69c19eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53930/0 (socket says 172.21.15.64:53930) 2023-09-13T03:11:57.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.612+0000 7ff69bfff700 1 -- 172.21.15.64:0/1730107939 learned_addr learned my addr 172.21.15.64:0/1730107939 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:57.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.612+0000 7ff6a0b7f700 1 --2- 172.21.15.64:0/1730107939 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff69c068120 0x7ff69c19e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:57.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.612+0000 7ff6a1380700 1 --2- 172.21.15.64:0/1730107939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff69c106770 0x7ff69c198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:57.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.613+0000 7ff69bfff700 1 -- 172.21.15.64:0/1730107939 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff69c068120 msgr2=0x7ff69c19e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:57.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.613+0000 7ff69bfff700 1 --2- 172.21.15.64:0/1730107939 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff69c068120 0x7ff69c19e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:57.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.613+0000 7ff69bfff700 1 -- 172.21.15.64:0/1730107939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff69c106770 msgr2=0x7ff69c198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:57.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.613+0000 7ff69bfff700 1 --2- 172.21.15.64:0/1730107939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff69c106770 0x7ff69c198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:57.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.613+0000 7ff69bfff700 1 -- 172.21.15.64:0/1730107939 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6880095f0 con 0x7ff69c068ae0 2023-09-13T03:11:57.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.613+0000 7ff6a1380700 1 --2- 172.21.15.64:0/1730107939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff69c106770 0x7ff69c198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:11:57.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.613+0000 7ff6a0b7f700 1 --2- 172.21.15.64:0/1730107939 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff69c068120 0x7ff69c19e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:11:57.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.613+0000 7ff69bfff700 1 --2- 172.21.15.64:0/1730107939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff69c068ae0 0x7ff69c19eb60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff688000c00 tx=0x7ff6880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:57.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.613+0000 7ff699ffb700 1 -- 172.21.15.64:0/1730107939 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68801d070 con 0x7ff69c068ae0 2023-09-13T03:11:57.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.614+0000 7ff6a2de3700 1 -- 172.21.15.64:0/1730107939 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff69c199330 con 0x7ff69c068ae0 2023-09-13T03:11:57.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.614+0000 7ff699ffb700 1 -- 172.21.15.64:0/1730107939 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff688022470 con 0x7ff69c068ae0 2023-09-13T03:11:57.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.614+0000 7ff699ffb700 1 -- 172.21.15.64:0/1730107939 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff688017650 con 0x7ff69c068ae0 2023-09-13T03:11:57.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.614+0000 7ff6a2de3700 1 -- 172.21.15.64:0/1730107939 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff69c1a5610 con 0x7ff69c068ae0 2023-09-13T03:11:57.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.615+0000 7ff6a2de3700 1 -- 172.21.15.64:0/1730107939 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff69c04e300 con 0x7ff69c068ae0 2023-09-13T03:11:57.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.619+0000 7ff699ffb700 1 -- 172.21.15.64:0/1730107939 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7ff6880225e0 con 0x7ff69c068ae0 2023-09-13T03:11:57.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.620+0000 7ff699ffb700 1 --2- 172.21.15.64:0/1730107939 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ff68c0a06d0 0x7ff68c0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:57.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.620+0000 7ff699ffb700 1 -- 172.21.15.64:0/1730107939 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff6880cd990 con 0x7ff69c068ae0 2023-09-13T03:11:57.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.620+0000 7ff6a0b7f700 1 --2- 172.21.15.64:0/1730107939 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ff68c0a06d0 0x7ff68c0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:57.236 INFO:teuthology.orchestra.run.smithi064.stdout:304942678040 2023-09-13T03:11:57.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.621+0000 7ff6a0b7f700 1 --2- 172.21.15.64:0/1730107939 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ff68c0a06d0 0x7ff68c0a2b90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff690009cb0 tx=0x7ff690009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:57.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.622+0000 7ff699ffb700 1 -- 172.21.15.64:0/1730107939 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ff6880973c0 con 0x7ff69c068ae0 2023-09-13T03:11:57.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.835+0000 7ff6a2de3700 1 -- 172.21.15.64:0/1730107939 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7ff69c110620 con 0x7ff69c068ae0 2023-09-13T03:11:57.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.835+0000 7ff699ffb700 1 -- 172.21.15.64:0/1730107939 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7ff688027020 con 0x7ff69c068ae0 2023-09-13T03:11:57.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.838+0000 7ff6a2de3700 1 -- 172.21.15.64:0/1730107939 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ff68c0a06d0 msgr2=0x7ff68c0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:57.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.838+0000 7ff6a2de3700 1 --2- 172.21.15.64:0/1730107939 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ff68c0a06d0 0x7ff68c0a2b90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff690009cb0 tx=0x7ff690009260 comp rx=0 tx=0).stop 2023-09-13T03:11:57.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.838+0000 7ff6a2de3700 1 -- 172.21.15.64:0/1730107939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff69c068ae0 msgr2=0x7ff69c19eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:57.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.838+0000 7ff6a2de3700 1 --2- 172.21.15.64:0/1730107939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff69c068ae0 0x7ff69c19eb60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff688000c00 tx=0x7ff6880057f0 comp rx=0 tx=0).stop 2023-09-13T03:11:57.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.838+0000 7ff6a2de3700 1 -- 172.21.15.64:0/1730107939 shutdown_connections 2023-09-13T03:11:57.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.838+0000 7ff6a2de3700 1 --2- 172.21.15.64:0/1730107939 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ff68c0a06d0 0x7ff68c0a2b90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:57.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.838+0000 7ff6a2de3700 1 --2- 172.21.15.64:0/1730107939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff69c106770 0x7ff69c198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:57.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.838+0000 7ff6a2de3700 1 --2- 172.21.15.64:0/1730107939 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff69c068120 0x7ff69c19e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:57.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.838+0000 7ff6a2de3700 1 --2- 172.21.15.64:0/1730107939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff69c068ae0 0x7ff69c19eb60 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:57.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.838+0000 7ff6a2de3700 1 -- 172.21.15.64:0/1730107939 >> 172.21.15.64:0/1730107939 conn(0x7ff69c076600 msgr2=0x7ff69c10b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:57.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.838+0000 7ff6a2de3700 1 -- 172.21.15.64:0/1730107939 shutdown_connections 2023-09-13T03:11:57.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:56.838+0000 7ff6a2de3700 1 -- 172.21.15.64:0/1730107939 wait complete. 2023-09-13T03:11:57.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.275+0000 7f413f879700 1 -- 172.21.15.64:0/3503090305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41380729d0 msgr2=0x7f4138072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:57.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.275+0000 7f413f879700 1 --2- 172.21.15.64:0/3503090305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41380729d0 0x7f4138072db0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4134009990 tx=0x7f4134009ca0 comp rx=0 tx=0).stop 2023-09-13T03:11:57.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.275+0000 7f413f879700 1 -- 172.21.15.64:0/3503090305 shutdown_connections 2023-09-13T03:11:57.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.275+0000 7f413f879700 1 --2- 172.21.15.64:0/3503090305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f413810cec0 0x7f413810f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:57.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.275+0000 7f413f879700 1 --2- 172.21.15.64:0/3503090305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41380729d0 0x7f4138072db0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:57.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.275+0000 7f413f879700 1 --2- 172.21.15.64:0/3503090305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41380732f0 0x7f413810c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:57.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.275+0000 7f413f879700 1 -- 172.21.15.64:0/3503090305 >> 172.21.15.64:0/3503090305 conn(0x7f41380fc0f0 msgr2=0x7f41380fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:57.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.275+0000 7f413f879700 1 -- 172.21.15.64:0/3503090305 shutdown_connections 2023-09-13T03:11:57.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.275+0000 7f413f879700 1 -- 172.21.15.64:0/3503090305 wait complete. 2023-09-13T03:11:57.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.277+0000 7f413f879700 1 Processor -- start 2023-09-13T03:11:57.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.277+0000 7f413f879700 1 -- start start 2023-09-13T03:11:57.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.278+0000 7f413f879700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41380729d0 0x7f4138198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:57.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.278+0000 7f413f879700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41380732f0 0x7f4138198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:57.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.278+0000 7f413f879700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f413810cec0 0x7f413819d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:57.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.278+0000 7f413f879700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413819d910 con 0x7f413810cec0 2023-09-13T03:11:57.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.278+0000 7f413f879700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413819da80 con 0x7f41380729d0 2023-09-13T03:11:57.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.278+0000 7f413d615700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41380729d0 0x7f4138198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:57.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.278+0000 7f413f879700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413819dc40 con 0x7f41380732f0 2023-09-13T03:11:57.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.278+0000 7f413ce14700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41380732f0 0x7f4138198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:57.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.278+0000 7f413d615700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41380729d0 0x7f4138198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:41188/0 (socket says 172.21.15.64:41188) 2023-09-13T03:11:57.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.278+0000 7f413de16700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f413810cec0 0x7f413819d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:57.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.278+0000 7f413d615700 1 -- 172.21.15.64:0/1069741205 learned_addr learned my addr 172.21.15.64:0/1069741205 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:57.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.279+0000 7f413d615700 1 -- 172.21.15.64:0/1069741205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41380732f0 msgr2=0x7f4138198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:57.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.279+0000 7f413d615700 1 --2- 172.21.15.64:0/1069741205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41380732f0 0x7f4138198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:57.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.279+0000 7f413d615700 1 -- 172.21.15.64:0/1069741205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f413810cec0 msgr2=0x7f413819d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:57.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.279+0000 7f413d615700 1 --2- 172.21.15.64:0/1069741205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f413810cec0 0x7f413819d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:57.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.279+0000 7f413d615700 1 -- 172.21.15.64:0/1069741205 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4130009c40 con 0x7f41380729d0 2023-09-13T03:11:57.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.279+0000 7f413de16700 1 --2- 172.21.15.64:0/1069741205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f413810cec0 0x7f413819d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:11:57.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.279+0000 7f413d615700 1 --2- 172.21.15.64:0/1069741205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41380729d0 0x7f4138198960 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4134005910 tx=0x7f4134004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:57.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.279+0000 7f412e7fc700 1 -- 172.21.15.64:0/1069741205 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413401d070 con 0x7f41380729d0 2023-09-13T03:11:57.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.279+0000 7f413f879700 1 -- 172.21.15.64:0/1069741205 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41340095f0 con 0x7f41380729d0 2023-09-13T03:11:57.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.279+0000 7f412e7fc700 1 -- 172.21.15.64:0/1069741205 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4134022470 con 0x7f41380729d0 2023-09-13T03:11:57.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.279+0000 7f412e7fc700 1 -- 172.21.15.64:0/1069741205 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4134017650 con 0x7f41380729d0 2023-09-13T03:11:57.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.280+0000 7f413f879700 1 -- 172.21.15.64:0/1069741205 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41381a55f0 con 0x7f41380729d0 2023-09-13T03:11:57.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.281+0000 7f413f879700 1 -- 172.21.15.64:0/1069741205 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41381080d0 con 0x7f41380729d0 2023-09-13T03:11:57.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.283+0000 7f412e7fc700 1 -- 172.21.15.64:0/1069741205 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f41340225e0 con 0x7f41380729d0 2023-09-13T03:11:57.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.284+0000 7f412e7fc700 1 --2- 172.21.15.64:0/1069741205 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f41240a06d0 0x7f41240a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:57.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.284+0000 7f412e7fc700 1 -- 172.21.15.64:0/1069741205 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f41340cd430 con 0x7f41380729d0 2023-09-13T03:11:57.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.284+0000 7f413ce14700 1 --2- 172.21.15.64:0/1069741205 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f41240a06d0 0x7f41240a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:57.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.285+0000 7f413ce14700 1 --2- 172.21.15.64:0/1069741205 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f41240a06d0 0x7f41240a2b90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4138199f80 tx=0x7f4128009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:57.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.286+0000 7f412e7fc700 1 -- 172.21.15.64:0/1069741205 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f4134096e60 con 0x7f41380729d0 2023-09-13T03:11:57.418 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:11:57.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.512+0000 7f413f879700 1 -- 172.21.15.64:0/1069741205 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f413819de00 con 0x7f41380729d0 2023-09-13T03:11:57.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.512+0000 7f412e7fc700 1 -- 172.21.15.64:0/1069741205 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f413409a480 con 0x7f41380729d0 2023-09-13T03:11:57.513 INFO:teuthology.orchestra.run.smithi064.stdout:249108103202 2023-09-13T03:11:57.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.515+0000 7f413f879700 1 -- 172.21.15.64:0/1069741205 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f41240a06d0 msgr2=0x7f41240a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:57.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.515+0000 7f413f879700 1 --2- 172.21.15.64:0/1069741205 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f41240a06d0 0x7f41240a2b90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4138199f80 tx=0x7f4128009260 comp rx=0 tx=0).stop 2023-09-13T03:11:57.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.515+0000 7f413f879700 1 -- 172.21.15.64:0/1069741205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41380729d0 msgr2=0x7f4138198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:57.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.515+0000 7f413f879700 1 --2- 172.21.15.64:0/1069741205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41380729d0 0x7f4138198960 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4134005910 tx=0x7f4134004710 comp rx=0 tx=0).stop 2023-09-13T03:11:57.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.515+0000 7f413f879700 1 -- 172.21.15.64:0/1069741205 shutdown_connections 2023-09-13T03:11:57.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.515+0000 7f413f879700 1 --2- 172.21.15.64:0/1069741205 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f41240a06d0 0x7f41240a2b90 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:57.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.515+0000 7f413f879700 1 --2- 172.21.15.64:0/1069741205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41380729d0 0x7f4138198960 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:57.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.515+0000 7f413f879700 1 --2- 172.21.15.64:0/1069741205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f413810cec0 0x7f413819d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:57.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.515+0000 7f413f879700 1 --2- 172.21.15.64:0/1069741205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41380732f0 0x7f4138198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:57.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.515+0000 7f413f879700 1 -- 172.21.15.64:0/1069741205 >> 172.21.15.64:0/1069741205 conn(0x7f41380fc0f0 msgr2=0x7f413810f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:57.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.515+0000 7f413f879700 1 -- 172.21.15.64:0/1069741205 shutdown_connections 2023-09-13T03:11:57.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:57.515+0000 7f413f879700 1 -- 172.21.15.64:0/1069741205 wait complete. 2023-09-13T03:11:57.615 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:11:57.836 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:57 smithi064 ceph-mon[97675]: pgmap v302: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-13T03:11:57.836 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:57 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/1730107939' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-09-13T03:11:57.836 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:57 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/1069741205' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-09-13T03:11:57.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:57 smithi170 ceph-mon[134328]: pgmap v302: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-13T03:11:57.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:57 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/1730107939' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-09-13T03:11:57.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:57 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/1069741205' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-09-13T03:11:57.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:57 smithi142 ceph-mon[134082]: pgmap v302: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-13T03:11:57.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:57 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/1730107939' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-09-13T03:11:57.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:57 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/1069741205' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-09-13T03:11:58.436 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678038 got 304942678040 for osd.10 2023-09-13T03:11:58.437 DEBUG:teuthology.parallel:result is None 2023-09-13T03:11:58.437 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738436 got 34359738438 for osd.0 2023-09-13T03:11:58.437 DEBUG:teuthology.parallel:result is None 2023-09-13T03:11:58.901 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103200 got 249108103202 for osd.8 2023-09-13T03:11:58.902 DEBUG:teuthology.parallel:result is None 2023-09-13T03:11:59.045 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:11:59.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.412+0000 7fa211aab700 1 -- 172.21.15.64:0/2515301938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa20c100620 msgr2=0x7fa20c100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:59.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.412+0000 7fa211aab700 1 --2- 172.21.15.64:0/2515301938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa20c100620 0x7fa20c100aa0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa200009990 tx=0x7fa200009ca0 comp rx=0 tx=0).stop 2023-09-13T03:11:59.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.412+0000 7fa211aab700 1 -- 172.21.15.64:0/2515301938 shutdown_connections 2023-09-13T03:11:59.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.412+0000 7fa211aab700 1 --2- 172.21.15.64:0/2515301938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa20c100fe0 0x7fa20c10d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:59.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.412+0000 7fa211aab700 1 --2- 172.21.15.64:0/2515301938 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa20c106620 0x7fa20c106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:59.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.412+0000 7fa211aab700 1 --2- 172.21.15.64:0/2515301938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa20c100620 0x7fa20c100aa0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:59.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.412+0000 7fa211aab700 1 -- 172.21.15.64:0/2515301938 >> 172.21.15.64:0/2515301938 conn(0x7fa20c0fc0f0 msgr2=0x7fa20c0fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:59.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.412+0000 7fa211aab700 1 -- 172.21.15.64:0/2515301938 shutdown_connections 2023-09-13T03:11:59.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.413+0000 7fa211aab700 1 -- 172.21.15.64:0/2515301938 wait complete. 2023-09-13T03:11:59.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.413+0000 7fa211aab700 1 Processor -- start 2023-09-13T03:11:59.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.414+0000 7fa211aab700 1 -- start start 2023-09-13T03:11:59.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.414+0000 7fa211aab700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa20c100620 0x7fa20c19e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:59.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.414+0000 7fa211aab700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa20c100fe0 0x7fa20c19ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:59.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.414+0000 7fa211aab700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa20c106620 0x7fa20c1987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:59.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.414+0000 7fa211aab700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c198e20 con 0x7fa20c100fe0 2023-09-13T03:11:59.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.414+0000 7fa211aab700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c198f60 con 0x7fa20c106620 2023-09-13T03:11:59.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.414+0000 7fa211aab700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c199120 con 0x7fa20c100620 2023-09-13T03:11:59.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.414+0000 7fa20affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa20c100fe0 0x7fa20c19ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:59.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.414+0000 7fa20affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa20c100fe0 0x7fa20c19ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53972/0 (socket says 172.21.15.64:53972) 2023-09-13T03:11:59.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.414+0000 7fa20affd700 1 -- 172.21.15.64:0/4275734951 learned_addr learned my addr 172.21.15.64:0/4275734951 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:11:59.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.414+0000 7fa20b7fe700 1 --2- 172.21.15.64:0/4275734951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa20c100620 0x7fa20c19e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:59.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.414+0000 7fa20bfff700 1 --2- 172.21.15.64:0/4275734951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa20c106620 0x7fa20c1987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:59.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.415+0000 7fa20affd700 1 -- 172.21.15.64:0/4275734951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa20c100620 msgr2=0x7fa20c19e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:59.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.415+0000 7fa20affd700 1 --2- 172.21.15.64:0/4275734951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa20c100620 0x7fa20c19e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:59.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.415+0000 7fa20affd700 1 -- 172.21.15.64:0/4275734951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa20c106620 msgr2=0x7fa20c1987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:59.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.415+0000 7fa20affd700 1 --2- 172.21.15.64:0/4275734951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa20c106620 0x7fa20c1987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:59.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.415+0000 7fa20affd700 1 -- 172.21.15.64:0/4275734951 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa204009c40 con 0x7fa20c100fe0 2023-09-13T03:11:59.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.415+0000 7fa20affd700 1 --2- 172.21.15.64:0/4275734951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa20c100fe0 0x7fa20c19ec20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa200000c00 tx=0x7fa200004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:59.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.415+0000 7fa208ff9700 1 -- 172.21.15.64:0/4275734951 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20001d070 con 0x7fa20c100fe0 2023-09-13T03:11:59.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.415+0000 7fa211aab700 1 -- 172.21.15.64:0/4275734951 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2000095f0 con 0x7fa20c100fe0 2023-09-13T03:11:59.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.415+0000 7fa208ff9700 1 -- 172.21.15.64:0/4275734951 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa200021410 con 0x7fa20c100fe0 2023-09-13T03:11:59.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.415+0000 7fa208ff9700 1 -- 172.21.15.64:0/4275734951 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa200017650 con 0x7fa20c100fe0 2023-09-13T03:11:59.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.415+0000 7fa211aab700 1 -- 172.21.15.64:0/4275734951 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa20c1a55b0 con 0x7fa20c100fe0 2023-09-13T03:11:59.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.416+0000 7fa211aab700 1 -- 172.21.15.64:0/4275734951 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa20c04e300 con 0x7fa20c100fe0 2023-09-13T03:11:59.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.419+0000 7fa208ff9700 1 -- 172.21.15.64:0/4275734951 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fa200021580 con 0x7fa20c100fe0 2023-09-13T03:11:59.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.420+0000 7fa208ff9700 1 --2- 172.21.15.64:0/4275734951 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fa1f40a06d0 0x7fa1f40a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:11:59.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.420+0000 7fa208ff9700 1 -- 172.21.15.64:0/4275734951 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa2000ccdd0 con 0x7fa20c100fe0 2023-09-13T03:11:59.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.420+0000 7fa20b7fe700 1 --2- 172.21.15.64:0/4275734951 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fa1f40a06d0 0x7fa1f40a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:11:59.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.421+0000 7fa20b7fe700 1 --2- 172.21.15.64:0/4275734951 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fa1f40a06d0 0x7fa1f40a2b90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa20c199900 tx=0x7fa1fc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:11:59.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.424+0000 7fa208ff9700 1 -- 172.21.15.64:0/4275734951 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fa200096910 con 0x7fa20c100fe0 2023-09-13T03:11:59.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.640+0000 7fa211aab700 1 -- 172.21.15.64:0/4275734951 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fa20c110570 con 0x7fa20c100fe0 2023-09-13T03:11:59.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.641+0000 7fa208ff9700 1 -- 172.21.15.64:0/4275734951 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa200099f30 con 0x7fa20c100fe0 2023-09-13T03:11:59.642 INFO:teuthology.orchestra.run.smithi064.stdout:163208757298 2023-09-13T03:11:59.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.643+0000 7fa211aab700 1 -- 172.21.15.64:0/4275734951 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fa1f40a06d0 msgr2=0x7fa1f40a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:59.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.643+0000 7fa211aab700 1 --2- 172.21.15.64:0/4275734951 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fa1f40a06d0 0x7fa1f40a2b90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa20c199900 tx=0x7fa1fc009260 comp rx=0 tx=0).stop 2023-09-13T03:11:59.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.643+0000 7fa211aab700 1 -- 172.21.15.64:0/4275734951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa20c100fe0 msgr2=0x7fa20c19ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:11:59.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.643+0000 7fa211aab700 1 --2- 172.21.15.64:0/4275734951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa20c100fe0 0x7fa20c19ec20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa200000c00 tx=0x7fa200004790 comp rx=0 tx=0).stop 2023-09-13T03:11:59.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.644+0000 7fa211aab700 1 -- 172.21.15.64:0/4275734951 shutdown_connections 2023-09-13T03:11:59.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.644+0000 7fa211aab700 1 --2- 172.21.15.64:0/4275734951 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fa1f40a06d0 0x7fa1f40a2b90 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:59.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.644+0000 7fa211aab700 1 --2- 172.21.15.64:0/4275734951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa20c106620 0x7fa20c1987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:59.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.644+0000 7fa211aab700 1 --2- 172.21.15.64:0/4275734951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa20c100620 0x7fa20c19e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:59.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.644+0000 7fa211aab700 1 --2- 172.21.15.64:0/4275734951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa20c100fe0 0x7fa20c19ec20 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:11:59.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.644+0000 7fa211aab700 1 -- 172.21.15.64:0/4275734951 >> 172.21.15.64:0/4275734951 conn(0x7fa20c0fc0f0 msgr2=0x7fa20c072a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:11:59.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.644+0000 7fa211aab700 1 -- 172.21.15.64:0/4275734951 shutdown_connections 2023-09-13T03:11:59.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:11:59.644+0000 7fa211aab700 1 -- 172.21.15.64:0/4275734951 wait complete. 2023-09-13T03:11:59.737 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:11:59 smithi064 ceph-mon[97675]: pgmap v303: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2023-09-13T03:11:59.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:11:59 smithi170 ceph-mon[134328]: pgmap v303: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2023-09-13T03:11:59.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:11:59 smithi142 ceph-mon[134082]: pgmap v303: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2023-09-13T03:12:00.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.037+0000 7f2c5489f700 1 -- 172.21.15.64:0/1665612956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c50100620 msgr2=0x7f2c50100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:00.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.037+0000 7f2c5489f700 1 --2- 172.21.15.64:0/1665612956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c50100620 0x7f2c50100aa0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2c44009990 tx=0x7f2c44009ca0 comp rx=0 tx=0).stop 2023-09-13T03:12:00.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.037+0000 7f2c5489f700 1 -- 172.21.15.64:0/1665612956 shutdown_connections 2023-09-13T03:12:00.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.037+0000 7f2c5489f700 1 --2- 172.21.15.64:0/1665612956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c50100fe0 0x7f2c5010d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:00.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.037+0000 7f2c5489f700 1 --2- 172.21.15.64:0/1665612956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c50106620 0x7f2c50106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:00.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.037+0000 7f2c5489f700 1 --2- 172.21.15.64:0/1665612956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c50100620 0x7f2c50100aa0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:00.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.037+0000 7f2c5489f700 1 -- 172.21.15.64:0/1665612956 >> 172.21.15.64:0/1665612956 conn(0x7f2c500fc0f0 msgr2=0x7f2c500fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:00.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.037+0000 7f2c5489f700 1 -- 172.21.15.64:0/1665612956 shutdown_connections 2023-09-13T03:12:00.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.037+0000 7f2c5489f700 1 -- 172.21.15.64:0/1665612956 wait complete. 2023-09-13T03:12:00.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.038+0000 7f2c5489f700 1 Processor -- start 2023-09-13T03:12:00.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.038+0000 7f2c5489f700 1 -- start start 2023-09-13T03:12:00.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.039+0000 7f2c5489f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c50100620 0x7f2c50198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:00.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.039+0000 7f2c5489f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c50100fe0 0x7f2c50198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:00.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.039+0000 7f2c5489f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c50106620 0x7f2c5019d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:00.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.039+0000 7f2c5489f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5019d900 con 0x7f2c50100620 2023-09-13T03:12:00.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.039+0000 7f2c5489f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5019da70 con 0x7f2c50106620 2023-09-13T03:12:00.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.039+0000 7f2c5489f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5019dc30 con 0x7f2c50100fe0 2023-09-13T03:12:00.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.039+0000 7f2c4e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c50100620 0x7f2c50198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:00.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.039+0000 7f2c4e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c50100620 0x7f2c50198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54002/0 (socket says 172.21.15.64:54002) 2023-09-13T03:12:00.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.039+0000 7f2c4e59c700 1 -- 172.21.15.64:0/2011165413 learned_addr learned my addr 172.21.15.64:0/2011165413 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:00.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.039+0000 7f2c4ed9d700 1 --2- 172.21.15.64:0/2011165413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c50106620 0x7f2c5019d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:00.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.039+0000 7f2c4dd9b700 1 --2- 172.21.15.64:0/2011165413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c50100fe0 0x7f2c50198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:00.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.040+0000 7f2c4e59c700 1 -- 172.21.15.64:0/2011165413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c50100fe0 msgr2=0x7f2c50198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:00.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.040+0000 7f2c4e59c700 1 --2- 172.21.15.64:0/2011165413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c50100fe0 0x7f2c50198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:00.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.040+0000 7f2c4e59c700 1 -- 172.21.15.64:0/2011165413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c50106620 msgr2=0x7f2c5019d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:00.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.040+0000 7f2c4e59c700 1 --2- 172.21.15.64:0/2011165413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c50106620 0x7f2c5019d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:00.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.040+0000 7f2c4e59c700 1 -- 172.21.15.64:0/2011165413 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c440095f0 con 0x7f2c50100620 2023-09-13T03:12:00.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.040+0000 7f2c4ed9d700 1 --2- 172.21.15.64:0/2011165413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c50106620 0x7f2c5019d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:00.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.040+0000 7f2c4dd9b700 1 --2- 172.21.15.64:0/2011165413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c50100fe0 0x7f2c50198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:12:00.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.040+0000 7f2c4e59c700 1 --2- 172.21.15.64:0/2011165413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c50100620 0x7f2c50198950 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2c4000e9a0 tx=0x7f2c4000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:00.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.040+0000 7f2c3f7fe700 1 -- 172.21.15.64:0/2011165413 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c40009790 con 0x7f2c50100620 2023-09-13T03:12:00.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.040+0000 7f2c3f7fe700 1 -- 172.21.15.64:0/2011165413 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c4000cbd0 con 0x7f2c50100620 2023-09-13T03:12:00.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.040+0000 7f2c5489f700 1 -- 172.21.15.64:0/2011165413 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c5019df10 con 0x7f2c50100620 2023-09-13T03:12:00.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.040+0000 7f2c3f7fe700 1 -- 172.21.15.64:0/2011165413 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c40018680 con 0x7f2c50100620 2023-09-13T03:12:00.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.040+0000 7f2c5489f700 1 -- 172.21.15.64:0/2011165413 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c501a57b0 con 0x7f2c50100620 2023-09-13T03:12:00.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.041+0000 7f2c5489f700 1 -- 172.21.15.64:0/2011165413 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c5004e300 con 0x7f2c50100620 2023-09-13T03:12:00.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.043+0000 7f2c3f7fe700 1 -- 172.21.15.64:0/2011165413 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f2c4000cd40 con 0x7f2c50100620 2023-09-13T03:12:00.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.046+0000 7f2c3f7fe700 1 --2- 172.21.15.64:0/2011165413 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f2c380a0750 0x7f2c380a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:00.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.046+0000 7f2c3f7fe700 1 -- 172.21.15.64:0/2011165413 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f2c40014070 con 0x7f2c50100620 2023-09-13T03:12:00.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.046+0000 7f2c4dd9b700 1 --2- 172.21.15.64:0/2011165413 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f2c380a0750 0x7f2c380a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:00.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.047+0000 7f2c4dd9b700 1 --2- 172.21.15.64:0/2011165413 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f2c380a0750 0x7f2c380a2c10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2c50199f70 tx=0x7f2c44005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:00.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.048+0000 7f2c3f7fe700 1 -- 172.21.15.64:0/2011165413 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f2c40095b80 con 0x7f2c50100620 2023-09-13T03:12:00.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.264+0000 7f2c5489f700 1 -- 172.21.15.64:0/2011165413 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f2c50199cb0 con 0x7f2c50100620 2023-09-13T03:12:00.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.264+0000 7f2c3f7fe700 1 -- 172.21.15.64:0/2011165413 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2c400991a0 con 0x7f2c50100620 2023-09-13T03:12:00.265 INFO:teuthology.orchestra.run.smithi064.stdout:141733920821 2023-09-13T03:12:00.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.267+0000 7f2c5489f700 1 -- 172.21.15.64:0/2011165413 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f2c380a0750 msgr2=0x7f2c380a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:00.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.267+0000 7f2c5489f700 1 --2- 172.21.15.64:0/2011165413 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f2c380a0750 0x7f2c380a2c10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2c50199f70 tx=0x7f2c44005fb0 comp rx=0 tx=0).stop 2023-09-13T03:12:00.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.267+0000 7f2c5489f700 1 -- 172.21.15.64:0/2011165413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c50100620 msgr2=0x7f2c50198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:00.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.267+0000 7f2c5489f700 1 --2- 172.21.15.64:0/2011165413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c50100620 0x7f2c50198950 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2c4000e9a0 tx=0x7f2c4000ecb0 comp rx=0 tx=0).stop 2023-09-13T03:12:00.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.268+0000 7f2c5489f700 1 -- 172.21.15.64:0/2011165413 shutdown_connections 2023-09-13T03:12:00.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.268+0000 7f2c5489f700 1 --2- 172.21.15.64:0/2011165413 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f2c380a0750 0x7f2c380a2c10 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:00.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.268+0000 7f2c5489f700 1 --2- 172.21.15.64:0/2011165413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c50106620 0x7f2c5019d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:00.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.268+0000 7f2c5489f700 1 --2- 172.21.15.64:0/2011165413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c50100620 0x7f2c50198950 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:00.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.268+0000 7f2c5489f700 1 --2- 172.21.15.64:0/2011165413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c50100fe0 0x7f2c50198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:00.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.268+0000 7f2c5489f700 1 -- 172.21.15.64:0/2011165413 >> 172.21.15.64:0/2011165413 conn(0x7f2c500fc0f0 msgr2=0x7f2c500fc760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:00.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.268+0000 7f2c5489f700 1 -- 172.21.15.64:0/2011165413 shutdown_connections 2023-09-13T03:12:00.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.268+0000 7f2c5489f700 1 -- 172.21.15.64:0/2011165413 wait complete. 2023-09-13T03:12:00.358 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757295 got 163208757298 for osd.5 2023-09-13T03:12:00.359 DEBUG:teuthology.parallel:result is None 2023-09-13T03:12:00.835 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:00 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/4275734951' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-09-13T03:12:00.835 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:00 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2011165413' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-09-13T03:12:00.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.882+0000 7f9a2919d700 1 -- 172.21.15.64:0/3028470340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a24100620 msgr2=0x7f9a24100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:00.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.882+0000 7f9a2919d700 1 --2- 172.21.15.64:0/3028470340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a24100620 0x7f9a24100aa0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9a18009990 tx=0x7f9a18009ca0 comp rx=0 tx=0).stop 2023-09-13T03:12:00.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.882+0000 7f9a2919d700 1 -- 172.21.15.64:0/3028470340 shutdown_connections 2023-09-13T03:12:00.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.882+0000 7f9a2919d700 1 --2- 172.21.15.64:0/3028470340 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a24100fe0 0x7f9a2410d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:00.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.882+0000 7f9a2919d700 1 --2- 172.21.15.64:0/3028470340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a24106620 0x7f9a24106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:00.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.882+0000 7f9a2919d700 1 --2- 172.21.15.64:0/3028470340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a24100620 0x7f9a24100aa0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:00.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.882+0000 7f9a2919d700 1 -- 172.21.15.64:0/3028470340 >> 172.21.15.64:0/3028470340 conn(0x7f9a240fc0f0 msgr2=0x7f9a240fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:00.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.882+0000 7f9a2919d700 1 -- 172.21.15.64:0/3028470340 shutdown_connections 2023-09-13T03:12:00.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.882+0000 7f9a2919d700 1 -- 172.21.15.64:0/3028470340 wait complete. 2023-09-13T03:12:00.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.883+0000 7f9a2919d700 1 Processor -- start 2023-09-13T03:12:00.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.884+0000 7f9a2919d700 1 -- start start 2023-09-13T03:12:00.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.884+0000 7f9a2919d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a24100620 0x7f9a24072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:00.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.884+0000 7f9a2919d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a24100fe0 0x7f9a24072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:00.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.884+0000 7f9a2919d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a24106620 0x7f9a240733c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:00.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.884+0000 7f9a2919d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a241a4b80 con 0x7f9a24106620 2023-09-13T03:12:00.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.884+0000 7f9a2919d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a241a4cf0 con 0x7f9a24100620 2023-09-13T03:12:00.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.884+0000 7f9a2919d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a241a4eb0 con 0x7f9a24100fe0 2023-09-13T03:12:00.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.884+0000 7f9a22d9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a24100620 0x7f9a24072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:00.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.884+0000 7f9a22d9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a24100620 0x7f9a24072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:41238/0 (socket says 172.21.15.64:41238) 2023-09-13T03:12:00.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.884+0000 7f9a22d9d700 1 -- 172.21.15.64:0/3511102009 learned_addr learned my addr 172.21.15.64:0/3511102009 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:00.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.884+0000 7f9a2359e700 1 --2- 172.21.15.64:0/3511102009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a24106620 0x7f9a240733c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:00.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.884+0000 7f9a2259c700 1 --2- 172.21.15.64:0/3511102009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a24100fe0 0x7f9a24072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:00.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.885+0000 7f9a22d9d700 1 -- 172.21.15.64:0/3511102009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a24100fe0 msgr2=0x7f9a24072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:00.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.885+0000 7f9a22d9d700 1 --2- 172.21.15.64:0/3511102009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a24100fe0 0x7f9a24072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:00.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.885+0000 7f9a22d9d700 1 -- 172.21.15.64:0/3511102009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a24106620 msgr2=0x7f9a240733c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:00.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.885+0000 7f9a22d9d700 1 --2- 172.21.15.64:0/3511102009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a24106620 0x7f9a240733c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:00.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.885+0000 7f9a22d9d700 1 -- 172.21.15.64:0/3511102009 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a180095f0 con 0x7f9a24100620 2023-09-13T03:12:00.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.885+0000 7f9a2359e700 1 --2- 172.21.15.64:0/3511102009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a24106620 0x7f9a240733c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:00.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.885+0000 7f9a22d9d700 1 --2- 172.21.15.64:0/3511102009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a24100620 0x7f9a24072900 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9a1400e9a0 tx=0x7f9a1400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:00.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.885+0000 7f9a0bfff700 1 -- 172.21.15.64:0/3511102009 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a14009790 con 0x7f9a24100620 2023-09-13T03:12:00.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.885+0000 7f9a0bfff700 1 -- 172.21.15.64:0/3511102009 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a14010430 con 0x7f9a24100620 2023-09-13T03:12:00.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.885+0000 7f9a2919d700 1 -- 172.21.15.64:0/3511102009 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a241a5190 con 0x7f9a24100620 2023-09-13T03:12:00.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.885+0000 7f9a0bfff700 1 -- 172.21.15.64:0/3511102009 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a14018680 con 0x7f9a24100620 2023-09-13T03:12:00.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.886+0000 7f9a2919d700 1 -- 172.21.15.64:0/3511102009 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a241a5690 con 0x7f9a24100620 2023-09-13T03:12:00.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.886+0000 7f9a2919d700 1 -- 172.21.15.64:0/3511102009 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a2404e300 con 0x7f9a24100620 2023-09-13T03:12:00.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.888+0000 7f9a0bfff700 1 -- 172.21.15.64:0/3511102009 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f9a140105a0 con 0x7f9a24100620 2023-09-13T03:12:00.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.890+0000 7f9a0bfff700 1 --2- 172.21.15.64:0/3511102009 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f9a0c0a06d0 0x7f9a0c0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:00.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.890+0000 7f9a0bfff700 1 -- 172.21.15.64:0/3511102009 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f9a14014070 con 0x7f9a24100620 2023-09-13T03:12:00.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.890+0000 7f9a2259c700 1 --2- 172.21.15.64:0/3511102009 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f9a0c0a06d0 0x7f9a0c0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:00.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.891+0000 7f9a2259c700 1 --2- 172.21.15.64:0/3511102009 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f9a0c0a06d0 0x7f9a0c0a2b90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9a1800b6f0 tx=0x7f9a1800b280 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:00.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:00.892+0000 7f9a0bfff700 1 -- 172.21.15.64:0/3511102009 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f9a14095b10 con 0x7f9a24100620 2023-09-13T03:12:00.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:00 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/4275734951' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-09-13T03:12:00.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:00 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/2011165413' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-09-13T03:12:00.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:00 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/4275734951' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-09-13T03:12:00.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:00 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/2011165413' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-09-13T03:12:01.104 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920819 got 141733920821 for osd.4 2023-09-13T03:12:01.105 DEBUG:teuthology.parallel:result is None 2023-09-13T03:12:01.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.126+0000 7f9a2919d700 1 -- 172.21.15.64:0/3511102009 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f9a24073e30 con 0x7f9a24100620 2023-09-13T03:12:01.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.126+0000 7f9a0bfff700 1 -- 172.21.15.64:0/3511102009 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9a14099130 con 0x7f9a24100620 2023-09-13T03:12:01.127 INFO:teuthology.orchestra.run.smithi064.stdout:283467841566 2023-09-13T03:12:01.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.129+0000 7f9a2919d700 1 -- 172.21.15.64:0/3511102009 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f9a0c0a06d0 msgr2=0x7f9a0c0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:01.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.129+0000 7f9a2919d700 1 --2- 172.21.15.64:0/3511102009 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f9a0c0a06d0 0x7f9a0c0a2b90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9a1800b6f0 tx=0x7f9a1800b280 comp rx=0 tx=0).stop 2023-09-13T03:12:01.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.129+0000 7f9a2919d700 1 -- 172.21.15.64:0/3511102009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a24100620 msgr2=0x7f9a24072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:01.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.129+0000 7f9a2919d700 1 --2- 172.21.15.64:0/3511102009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a24100620 0x7f9a24072900 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9a1400e9a0 tx=0x7f9a1400ecb0 comp rx=0 tx=0).stop 2023-09-13T03:12:01.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.129+0000 7f9a2919d700 1 -- 172.21.15.64:0/3511102009 shutdown_connections 2023-09-13T03:12:01.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.129+0000 7f9a2919d700 1 --2- 172.21.15.64:0/3511102009 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f9a0c0a06d0 0x7f9a0c0a2b90 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:01.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.129+0000 7f9a2919d700 1 --2- 172.21.15.64:0/3511102009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a24100620 0x7f9a24072900 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:01.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.129+0000 7f9a2919d700 1 --2- 172.21.15.64:0/3511102009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a24106620 0x7f9a240733c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:01.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.129+0000 7f9a2919d700 1 --2- 172.21.15.64:0/3511102009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a24100fe0 0x7f9a24072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:01.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.129+0000 7f9a2919d700 1 -- 172.21.15.64:0/3511102009 >> 172.21.15.64:0/3511102009 conn(0x7f9a240fc0f0 msgr2=0x7f9a240fc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:01.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.130+0000 7f9a2919d700 1 -- 172.21.15.64:0/3511102009 shutdown_connections 2023-09-13T03:12:01.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.130+0000 7f9a2919d700 1 -- 172.21.15.64:0/3511102009 wait complete. 2023-09-13T03:12:01.608 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841564 got 283467841566 for osd.9 2023-09-13T03:12:01.608 DEBUG:teuthology.parallel:result is None 2023-09-13T03:12:01.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.874+0000 7efc897db700 1 -- 172.21.15.64:0/94449217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc84100fe0 msgr2=0x7efc8410d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:01.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.874+0000 7efc897db700 1 --2- 172.21.15.64:0/94449217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc84100fe0 0x7efc8410d4f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00b0a0 tx=0x7efc7c00b3b0 comp rx=0 tx=0).stop 2023-09-13T03:12:01.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.874+0000 7efc897db700 1 -- 172.21.15.64:0/94449217 shutdown_connections 2023-09-13T03:12:01.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.874+0000 7efc897db700 1 --2- 172.21.15.64:0/94449217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc84106620 0x7efc84106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:01.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.874+0000 7efc897db700 1 --2- 172.21.15.64:0/94449217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc84100fe0 0x7efc8410d4f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:01.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.874+0000 7efc897db700 1 --2- 172.21.15.64:0/94449217 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc84100620 0x7efc84100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:01.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.874+0000 7efc897db700 1 -- 172.21.15.64:0/94449217 >> 172.21.15.64:0/94449217 conn(0x7efc840fc0f0 msgr2=0x7efc840fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:01.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.874+0000 7efc897db700 1 -- 172.21.15.64:0/94449217 shutdown_connections 2023-09-13T03:12:01.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.874+0000 7efc897db700 1 -- 172.21.15.64:0/94449217 wait complete. 2023-09-13T03:12:01.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.875+0000 7efc897db700 1 Processor -- start 2023-09-13T03:12:01.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.876+0000 7efc897db700 1 -- start start 2023-09-13T03:12:01.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.876+0000 7efc897db700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc84100620 0x7efc84072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:01.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.876+0000 7efc897db700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc84100fe0 0x7efc84072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:01.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.876+0000 7efc897db700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc84106620 0x7efc84073380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:01.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.876+0000 7efc897db700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc841a9200 con 0x7efc84100fe0 2023-09-13T03:12:01.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.876+0000 7efc897db700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc841a9340 con 0x7efc84100620 2023-09-13T03:12:01.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.876+0000 7efc897db700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc841a9480 con 0x7efc84106620 2023-09-13T03:12:01.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.876+0000 7efc82ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc84100620 0x7efc84072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:01.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.876+0000 7efc827fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc84100fe0 0x7efc84072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:01.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.876+0000 7efc82ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc84100620 0x7efc84072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:41258/0 (socket says 172.21.15.64:41258) 2023-09-13T03:12:01.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.876+0000 7efc82ffd700 1 -- 172.21.15.64:0/1335578832 learned_addr learned my addr 172.21.15.64:0/1335578832 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:01.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.876+0000 7efc837fe700 1 --2- 172.21.15.64:0/1335578832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc84106620 0x7efc84073380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:01.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.877+0000 7efc82ffd700 1 -- 172.21.15.64:0/1335578832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc84106620 msgr2=0x7efc84073380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:01.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.877+0000 7efc82ffd700 1 --2- 172.21.15.64:0/1335578832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc84106620 0x7efc84073380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:01.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.877+0000 7efc82ffd700 1 -- 172.21.15.64:0/1335578832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc84100fe0 msgr2=0x7efc84072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:01.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.877+0000 7efc82ffd700 1 --2- 172.21.15.64:0/1335578832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc84100fe0 0x7efc84072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:01.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.877+0000 7efc82ffd700 1 -- 172.21.15.64:0/1335578832 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc7c009d00 con 0x7efc84100620 2023-09-13T03:12:01.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.877+0000 7efc827fc700 1 --2- 172.21.15.64:0/1335578832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc84100fe0 0x7efc84072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:01.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.877+0000 7efc82ffd700 1 --2- 172.21.15.64:0/1335578832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc84100620 0x7efc84072900 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efc7400e9a0 tx=0x7efc7400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:01.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.877+0000 7efc6bfff700 1 -- 172.21.15.64:0/1335578832 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc74009790 con 0x7efc84100620 2023-09-13T03:12:01.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.877+0000 7efc897db700 1 -- 172.21.15.64:0/1335578832 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc841a9760 con 0x7efc84100620 2023-09-13T03:12:01.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.877+0000 7efc6bfff700 1 -- 172.21.15.64:0/1335578832 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc74010430 con 0x7efc84100620 2023-09-13T03:12:01.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.877+0000 7efc6bfff700 1 -- 172.21.15.64:0/1335578832 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc74018680 con 0x7efc84100620 2023-09-13T03:12:01.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.877+0000 7efc897db700 1 -- 172.21.15.64:0/1335578832 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc841a9cb0 con 0x7efc84100620 2023-09-13T03:12:01.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.878+0000 7efc897db700 1 -- 172.21.15.64:0/1335578832 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc8404e300 con 0x7efc84100620 2023-09-13T03:12:01.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.880+0000 7efc6bfff700 1 -- 172.21.15.64:0/1335578832 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7efc740105a0 con 0x7efc84100620 2023-09-13T03:12:01.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.882+0000 7efc6bfff700 1 --2- 172.21.15.64:0/1335578832 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7efc6c0a06d0 0x7efc6c0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:01.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.882+0000 7efc6bfff700 1 -- 172.21.15.64:0/1335578832 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7efc74014070 con 0x7efc84100620 2023-09-13T03:12:01.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.882+0000 7efc827fc700 1 --2- 172.21.15.64:0/1335578832 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7efc6c0a06d0 0x7efc6c0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:01.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.883+0000 7efc827fc700 1 --2- 172.21.15.64:0/1335578832 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7efc6c0a06d0 0x7efc6c0a2b90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efc840740f0 tx=0x7efc78009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:01.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:01.884+0000 7efc6bfff700 1 -- 172.21.15.64:0/1335578832 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7efc74095ae0 con 0x7efc84100620 2023-09-13T03:12:01.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:01 smithi170 ceph-mon[134328]: pgmap v304: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2023-09-13T03:12:01.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:01 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/3511102009' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-09-13T03:12:01.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:01 smithi064 ceph-mon[97675]: pgmap v304: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2023-09-13T03:12:01.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:01 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/3511102009' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-09-13T03:12:01.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:01 smithi142 ceph-mon[134082]: pgmap v304: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2023-09-13T03:12:01.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:01 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/3511102009' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-09-13T03:12:02.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:02.091+0000 7efc897db700 1 -- 172.21.15.64:0/1335578832 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7efc84073e30 con 0x7efc84100620 2023-09-13T03:12:02.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:02.092+0000 7efc6bfff700 1 -- 172.21.15.64:0/1335578832 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7efc74099100 con 0x7efc84100620 2023-09-13T03:12:02.093 INFO:teuthology.orchestra.run.smithi064.stdout:55834574914 2023-09-13T03:12:02.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:02.094+0000 7efc897db700 1 -- 172.21.15.64:0/1335578832 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7efc6c0a06d0 msgr2=0x7efc6c0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:02.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:02.094+0000 7efc897db700 1 --2- 172.21.15.64:0/1335578832 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7efc6c0a06d0 0x7efc6c0a2b90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efc840740f0 tx=0x7efc78009260 comp rx=0 tx=0).stop 2023-09-13T03:12:02.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:02.094+0000 7efc897db700 1 -- 172.21.15.64:0/1335578832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc84100620 msgr2=0x7efc84072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:02.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:02.094+0000 7efc897db700 1 --2- 172.21.15.64:0/1335578832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc84100620 0x7efc84072900 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efc7400e9a0 tx=0x7efc7400ecb0 comp rx=0 tx=0).stop 2023-09-13T03:12:02.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:02.094+0000 7efc897db700 1 -- 172.21.15.64:0/1335578832 shutdown_connections 2023-09-13T03:12:02.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:02.094+0000 7efc897db700 1 --2- 172.21.15.64:0/1335578832 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7efc6c0a06d0 0x7efc6c0a2b90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:02.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:02.094+0000 7efc897db700 1 --2- 172.21.15.64:0/1335578832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc84100620 0x7efc84072900 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:02.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:02.094+0000 7efc897db700 1 --2- 172.21.15.64:0/1335578832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc84106620 0x7efc84073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:02.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:02.094+0000 7efc897db700 1 --2- 172.21.15.64:0/1335578832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc84100fe0 0x7efc84072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:02.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:02.094+0000 7efc897db700 1 -- 172.21.15.64:0/1335578832 >> 172.21.15.64:0/1335578832 conn(0x7efc840fc0f0 msgr2=0x7efc840fc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:02.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:02.094+0000 7efc897db700 1 -- 172.21.15.64:0/1335578832 shutdown_connections 2023-09-13T03:12:02.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:02.094+0000 7efc897db700 1 -- 172.21.15.64:0/1335578832 wait complete. 2023-09-13T03:12:02.580 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574912 got 55834574914 for osd.1 2023-09-13T03:12:02.581 DEBUG:teuthology.parallel:result is None 2023-09-13T03:12:02.581 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-09-13T03:12:02.581 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph pg dump --format=json 2023-09-13T03:12:02.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:02 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/1335578832' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-09-13T03:12:02.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:02 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/1335578832' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-09-13T03:12:02.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:02 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/1335578832' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-09-13T03:12:03.315 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:12:03.812 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:03 smithi064 ceph-mon[97675]: pgmap v305: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2023-09-13T03:12:03.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:03 smithi170 ceph-mon[134328]: pgmap v305: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2023-09-13T03:12:03.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:03 smithi142 ceph-mon[134082]: pgmap v305: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2023-09-13T03:12:04.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.685+0000 7fe7a58e7700 1 -- 172.21.15.64:0/661393128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7a00729d0 msgr2=0x7fe7a0072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:04.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.685+0000 7fe7a58e7700 1 --2- 172.21.15.64:0/661393128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7a00729d0 0x7fe7a0072db0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe790009990 tx=0x7fe790009ca0 comp rx=0 tx=0).stop 2023-09-13T03:12:04.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.685+0000 7fe7a58e7700 1 -- 172.21.15.64:0/661393128 shutdown_connections 2023-09-13T03:12:04.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.685+0000 7fe7a58e7700 1 --2- 172.21.15.64:0/661393128 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7a010cec0 0x7fe7a010f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:04.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.685+0000 7fe7a58e7700 1 --2- 172.21.15.64:0/661393128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7a00729d0 0x7fe7a0072db0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:04.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.685+0000 7fe7a58e7700 1 --2- 172.21.15.64:0/661393128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7a00732f0 0x7fe7a010c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:04.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.685+0000 7fe7a58e7700 1 -- 172.21.15.64:0/661393128 >> 172.21.15.64:0/661393128 conn(0x7fe7a00fc0f0 msgr2=0x7fe7a00fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:04.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.685+0000 7fe7a58e7700 1 -- 172.21.15.64:0/661393128 shutdown_connections 2023-09-13T03:12:04.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.685+0000 7fe7a58e7700 1 -- 172.21.15.64:0/661393128 wait complete. 2023-09-13T03:12:04.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.686+0000 7fe7a58e7700 1 Processor -- start 2023-09-13T03:12:04.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.686+0000 7fe7a58e7700 1 -- start start 2023-09-13T03:12:04.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.687+0000 7fe7a58e7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7a00729d0 0x7fe7a0198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:04.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.687+0000 7fe7a58e7700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7a00732f0 0x7fe7a0198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:04.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.687+0000 7fe7a58e7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7a010cec0 0x7fe7a019d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:04.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.687+0000 7fe7a58e7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a019d910 con 0x7fe7a00729d0 2023-09-13T03:12:04.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.687+0000 7fe7a58e7700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a019da80 con 0x7fe7a00732f0 2023-09-13T03:12:04.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.687+0000 7fe7a58e7700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a019dc40 con 0x7fe7a010cec0 2023-09-13T03:12:04.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.687+0000 7fe79effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7a00729d0 0x7fe7a0198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:04.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.687+0000 7fe79effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7a00729d0 0x7fe7a0198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54054/0 (socket says 172.21.15.64:54054) 2023-09-13T03:12:04.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.687+0000 7fe79effd700 1 -- 172.21.15.64:0/624453033 learned_addr learned my addr 172.21.15.64:0/624453033 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:04.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.687+0000 7fe79f7fe700 1 --2- 172.21.15.64:0/624453033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7a010cec0 0x7fe7a019d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:04.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.687+0000 7fe79e7fc700 1 --2- 172.21.15.64:0/624453033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7a00732f0 0x7fe7a0198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:04.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.687+0000 7fe79effd700 1 -- 172.21.15.64:0/624453033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7a010cec0 msgr2=0x7fe7a019d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:04.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.687+0000 7fe79effd700 1 --2- 172.21.15.64:0/624453033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7a010cec0 0x7fe7a019d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:04.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.687+0000 7fe79effd700 1 -- 172.21.15.64:0/624453033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7a00732f0 msgr2=0x7fe7a0198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:04.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.687+0000 7fe79effd700 1 --2- 172.21.15.64:0/624453033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7a00732f0 0x7fe7a0198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:04.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.687+0000 7fe79effd700 1 -- 172.21.15.64:0/624453033 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7900095f0 con 0x7fe7a00729d0 2023-09-13T03:12:04.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.688+0000 7fe79effd700 1 --2- 172.21.15.64:0/624453033 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7a00729d0 0x7fe7a0198960 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe790000c00 tx=0x7fe79000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:04.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.688+0000 7fe7a48e5700 1 -- 172.21.15.64:0/624453033 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79001d070 con 0x7fe7a00729d0 2023-09-13T03:12:04.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.688+0000 7fe7a48e5700 1 -- 172.21.15.64:0/624453033 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe79000be40 con 0x7fe7a00729d0 2023-09-13T03:12:04.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.688+0000 7fe7a58e7700 1 -- 172.21.15.64:0/624453033 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7a019dec0 con 0x7fe7a00729d0 2023-09-13T03:12:04.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.688+0000 7fe7a48e5700 1 -- 172.21.15.64:0/624453033 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7900173f0 con 0x7fe7a00729d0 2023-09-13T03:12:04.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.688+0000 7fe7a58e7700 1 -- 172.21.15.64:0/624453033 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7a01a57c0 con 0x7fe7a00729d0 2023-09-13T03:12:04.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.689+0000 7fe7a58e7700 1 -- 172.21.15.64:0/624453033 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7a004e300 con 0x7fe7a00729d0 2023-09-13T03:12:04.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.691+0000 7fe7a48e5700 1 -- 172.21.15.64:0/624453033 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fe790022470 con 0x7fe7a00729d0 2023-09-13T03:12:04.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.693+0000 7fe7a48e5700 1 --2- 172.21.15.64:0/624453033 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe7880a0750 0x7fe7880a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:04.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.693+0000 7fe7a48e5700 1 -- 172.21.15.64:0/624453033 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fe7900ccf70 con 0x7fe7a00729d0 2023-09-13T03:12:04.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.693+0000 7fe79e7fc700 1 --2- 172.21.15.64:0/624453033 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe7880a0750 0x7fe7880a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:04.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.693+0000 7fe79e7fc700 1 --2- 172.21.15.64:0/624453033 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe7880a0750 0x7fe7880a2c10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0199f80 tx=0x7fe794009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:04.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.697+0000 7fe7a48e5700 1 -- 172.21.15.64:0/624453033 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe79005ce10 con 0x7fe7a00729d0 2023-09-13T03:12:04.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.899+0000 7fe7a58e7700 1 -- 172.21.15.64:0/624453033 --> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe7a0112940 con 0x7fe7880a0750 2023-09-13T03:12:04.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.911+0000 7fe7a48e5700 1 -- 172.21.15.64:0/624453033 <== mgr.14150 v2:172.21.15.64:6800/873679308 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340410 (secure 0 0 0) 0x7fe7a0112940 con 0x7fe7880a0750 2023-09-13T03:12:04.917 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:12:04.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.916+0000 7fe7a58e7700 1 -- 172.21.15.64:0/624453033 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe7880a0750 msgr2=0x7fe7880a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:04.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.916+0000 7fe7a58e7700 1 --2- 172.21.15.64:0/624453033 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe7880a0750 0x7fe7880a2c10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0199f80 tx=0x7fe794009260 comp rx=0 tx=0).stop 2023-09-13T03:12:04.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.916+0000 7fe7a58e7700 1 -- 172.21.15.64:0/624453033 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7a00729d0 msgr2=0x7fe7a0198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:04.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.916+0000 7fe7a58e7700 1 --2- 172.21.15.64:0/624453033 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7a00729d0 0x7fe7a0198960 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe790000c00 tx=0x7fe79000f690 comp rx=0 tx=0).stop 2023-09-13T03:12:04.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.916+0000 7fe7a58e7700 1 -- 172.21.15.64:0/624453033 shutdown_connections 2023-09-13T03:12:04.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.916+0000 7fe7a58e7700 1 --2- 172.21.15.64:0/624453033 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe7880a0750 0x7fe7880a2c10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:04.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.916+0000 7fe7a58e7700 1 --2- 172.21.15.64:0/624453033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7a010cec0 0x7fe7a019d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:04.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.916+0000 7fe7a58e7700 1 --2- 172.21.15.64:0/624453033 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7a00729d0 0x7fe7a0198960 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:04.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.916+0000 7fe7a58e7700 1 --2- 172.21.15.64:0/624453033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7a00732f0 0x7fe7a0198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:04.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.916+0000 7fe7a58e7700 1 -- 172.21.15.64:0/624453033 >> 172.21.15.64:0/624453033 conn(0x7fe7a00fc0f0 msgr2=0x7fe7a010f070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:04.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.917+0000 7fe7a58e7700 1 -- 172.21.15.64:0/624453033 shutdown_connections 2023-09-13T03:12:04.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:04.917+0000 7fe7a58e7700 1 -- 172.21.15.64:0/624453033 wait complete. 2023-09-13T03:12:04.925 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-13T03:12:05.390 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2023-09-13T03:12:04.466734+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655660,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":616,"num_read_kb":430,"num_write":416,"num_write_kb":3067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180804,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":342,"ondisk_log_size":342,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337236,"kb_used_data":12996,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604588,"statfs":{"total":1151940427776,"available":1151595098112,"internally_reserved":0,"allocated":13307904,"data_stored":7282439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19130,"internal_metadata":331855174},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":135,"num_read_kb":70,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002920"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.265869+0000","last_change":"2023-09-13T03:11:10.949449+0000","last_active":"2023-09-13T03:11:23.265869+0000","last_peered":"2023-09-13T03:11:23.265869+0000","last_clean":"2023-09-13T03:11:23.265869+0000","last_became_active":"2023-09-13T03:11:10.948912+0000","last_became_peered":"2023-09-13T03:11:10.948912+0000","last_unstale":"2023-09-13T03:11:23.265869+0000","last_undegraded":"2023-09-13T03:11:23.265869+0000","last_fullsized":"2023-09-13T03:11:23.265869+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:34:27.381352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.266683+0000","last_change":"2023-09-13T03:11:10.956817+0000","last_active":"2023-09-13T03:11:23.266683+0000","last_peered":"2023-09-13T03:11:23.266683+0000","last_clean":"2023-09-13T03:11:23.266683+0000","last_became_active":"2023-09-13T03:11:10.955359+0000","last_became_peered":"2023-09-13T03:11:10.955359+0000","last_unstale":"2023-09-13T03:11:23.266683+0000","last_undegraded":"2023-09-13T03:11:23.266683+0000","last_fullsized":"2023-09-13T03:11:23.266683+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:06:01.055910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.266531+0000","last_change":"2023-09-13T03:11:10.959915+0000","last_active":"2023-09-13T03:11:23.266531+0000","last_peered":"2023-09-13T03:11:23.266531+0000","last_clean":"2023-09-13T03:11:23.266531+0000","last_became_active":"2023-09-13T03:11:10.959611+0000","last_became_peered":"2023-09-13T03:11:10.959611+0000","last_unstale":"2023-09-13T03:11:23.266531+0000","last_undegraded":"2023-09-13T03:11:23.266531+0000","last_fullsized":"2023-09-13T03:11:23.266531+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:31:40.075569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:11:24.269796+0000","last_change":"2023-09-13T03:11:24.269022+0000","last_active":"2023-09-13T03:11:24.269796+0000","last_peered":"2023-09-13T03:11:24.269796+0000","last_clean":"2023-09-13T03:11:24.269796+0000","last_became_active":"2023-09-13T03:11:24.268019+0000","last_became_peered":"2023-09-13T03:11:24.268019+0000","last_unstale":"2023-09-13T03:11:24.269796+0000","last_undegraded":"2023-09-13T03:11:24.269796+0000","last_fullsized":"2023-09-13T03:11:24.269796+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:58:06.518437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.264628+0000","last_change":"2023-09-13T03:11:10.951554+0000","last_active":"2023-09-13T03:11:23.264628+0000","last_peered":"2023-09-13T03:11:23.264628+0000","last_clean":"2023-09-13T03:11:23.264628+0000","last_became_active":"2023-09-13T03:11:10.951252+0000","last_became_peered":"2023-09-13T03:11:10.951252+0000","last_unstale":"2023-09-13T03:11:23.264628+0000","last_undegraded":"2023-09-13T03:11:23.264628+0000","last_fullsized":"2023-09-13T03:11:23.264628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:38:42.890256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.270747+0000","last_change":"2023-09-13T03:11:10.953087+0000","last_active":"2023-09-13T03:11:23.270747+0000","last_peered":"2023-09-13T03:11:23.270747+0000","last_clean":"2023-09-13T03:11:23.270747+0000","last_became_active":"2023-09-13T03:11:10.952773+0000","last_became_peered":"2023-09-13T03:11:10.952773+0000","last_unstale":"2023-09-13T03:11:23.270747+0000","last_undegraded":"2023-09-13T03:11:23.270747+0000","last_fullsized":"2023-09-13T03:11:23.270747+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:14:55.203347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.262021+0000","last_change":"2023-09-13T03:11:10.956058+0000","last_active":"2023-09-13T03:11:23.262021+0000","last_peered":"2023-09-13T03:11:23.262021+0000","last_clean":"2023-09-13T03:11:23.262021+0000","last_became_active":"2023-09-13T03:11:10.955783+0000","last_became_peered":"2023-09-13T03:11:10.955783+0000","last_unstale":"2023-09-13T03:11:23.262021+0000","last_undegraded":"2023-09-13T03:11:23.262021+0000","last_fullsized":"2023-09-13T03:11:23.262021+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:31:06.810338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.266523+0000","last_change":"2023-09-13T03:11:10.950102+0000","last_active":"2023-09-13T03:11:23.266523+0000","last_peered":"2023-09-13T03:11:23.266523+0000","last_clean":"2023-09-13T03:11:23.266523+0000","last_became_active":"2023-09-13T03:11:10.948877+0000","last_became_peered":"2023-09-13T03:11:10.948877+0000","last_unstale":"2023-09-13T03:11:23.266523+0000","last_undegraded":"2023-09-13T03:11:23.266523+0000","last_fullsized":"2023-09-13T03:11:23.266523+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:32:09.168793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.270038+0000","last_change":"2023-09-13T03:11:10.945244+0000","last_active":"2023-09-13T03:11:23.270038+0000","last_peered":"2023-09-13T03:11:23.270038+0000","last_clean":"2023-09-13T03:11:23.270038+0000","last_became_active":"2023-09-13T03:11:10.944951+0000","last_became_peered":"2023-09-13T03:11:10.944951+0000","last_unstale":"2023-09-13T03:11:23.270038+0000","last_undegraded":"2023-09-13T03:11:23.270038+0000","last_fullsized":"2023-09-13T03:11:23.270038+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:42:35.420767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.271052+0000","last_change":"2023-09-13T03:11:10.952312+0000","last_active":"2023-09-13T03:11:23.271052+0000","last_peered":"2023-09-13T03:11:23.271052+0000","last_clean":"2023-09-13T03:11:23.271052+0000","last_became_active":"2023-09-13T03:11:10.952077+0000","last_became_peered":"2023-09-13T03:11:10.952077+0000","last_unstale":"2023-09-13T03:11:23.271052+0000","last_undegraded":"2023-09-13T03:11:23.271052+0000","last_fullsized":"2023-09-13T03:11:23.271052+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:57:10.888184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.262424+0000","last_change":"2023-09-13T03:11:10.959259+0000","last_active":"2023-09-13T03:11:23.262424+0000","last_peered":"2023-09-13T03:11:23.262424+0000","last_clean":"2023-09-13T03:11:23.262424+0000","last_became_active":"2023-09-13T03:11:10.958922+0000","last_became_peered":"2023-09-13T03:11:10.958922+0000","last_unstale":"2023-09-13T03:11:23.262424+0000","last_undegraded":"2023-09-13T03:11:23.262424+0000","last_fullsized":"2023-09-13T03:11:23.262424+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:08:54.680718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.265477+0000","last_change":"2023-09-13T03:11:10.950946+0000","last_active":"2023-09-13T03:11:23.265477+0000","last_peered":"2023-09-13T03:11:23.265477+0000","last_clean":"2023-09-13T03:11:23.265477+0000","last_became_active":"2023-09-13T03:11:10.950633+0000","last_became_peered":"2023-09-13T03:11:10.950633+0000","last_unstale":"2023-09-13T03:11:23.265477+0000","last_undegraded":"2023-09-13T03:11:23.265477+0000","last_fullsized":"2023-09-13T03:11:23.265477+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:37:25.796555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.261964+0000","last_change":"2023-09-13T03:11:10.957712+0000","last_active":"2023-09-13T03:11:23.261964+0000","last_peered":"2023-09-13T03:11:23.261964+0000","last_clean":"2023-09-13T03:11:23.261964+0000","last_became_active":"2023-09-13T03:11:10.957379+0000","last_became_peered":"2023-09-13T03:11:10.957379+0000","last_unstale":"2023-09-13T03:11:23.261964+0000","last_undegraded":"2023-09-13T03:11:23.261964+0000","last_fullsized":"2023-09-13T03:11:23.261964+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:31:17.084185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.266012+0000","last_change":"2023-09-13T03:11:10.952951+0000","last_active":"2023-09-13T03:11:23.266012+0000","last_peered":"2023-09-13T03:11:23.266012+0000","last_clean":"2023-09-13T03:11:23.266012+0000","last_became_active":"2023-09-13T03:11:10.952629+0000","last_became_peered":"2023-09-13T03:11:10.952629+0000","last_unstale":"2023-09-13T03:11:23.266012+0000","last_undegraded":"2023-09-13T03:11:23.266012+0000","last_fullsized":"2023-09-13T03:11:23.266012+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:20:40.802584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.265840+0000","last_change":"2023-09-13T03:11:10.950799+0000","last_active":"2023-09-13T03:11:23.265840+0000","last_peered":"2023-09-13T03:11:23.265840+0000","last_clean":"2023-09-13T03:11:23.265840+0000","last_became_active":"2023-09-13T03:11:10.950500+0000","last_became_peered":"2023-09-13T03:11:10.950500+0000","last_unstale":"2023-09-13T03:11:23.265840+0000","last_undegraded":"2023-09-13T03:11:23.265840+0000","last_fullsized":"2023-09-13T03:11:23.265840+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:24:45.770696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.269219+0000","last_change":"2023-09-13T03:11:10.955141+0000","last_active":"2023-09-13T03:11:23.269219+0000","last_peered":"2023-09-13T03:11:23.269219+0000","last_clean":"2023-09-13T03:11:23.269219+0000","last_became_active":"2023-09-13T03:11:10.954840+0000","last_became_peered":"2023-09-13T03:11:10.954840+0000","last_unstale":"2023-09-13T03:11:23.269219+0000","last_undegraded":"2023-09-13T03:11:23.269219+0000","last_fullsized":"2023-09-13T03:11:23.269219+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:56:08.318784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.265976+0000","last_change":"2023-09-13T03:11:10.949199+0000","last_active":"2023-09-13T03:11:23.265976+0000","last_peered":"2023-09-13T03:11:23.265976+0000","last_clean":"2023-09-13T03:11:23.265976+0000","last_became_active":"2023-09-13T03:11:10.948506+0000","last_became_peered":"2023-09-13T03:11:10.948506+0000","last_unstale":"2023-09-13T03:11:23.265976+0000","last_undegraded":"2023-09-13T03:11:23.265976+0000","last_fullsized":"2023-09-13T03:11:23.265976+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:04:07.714033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.261637+0000","last_change":"2023-09-13T03:11:10.953207+0000","last_active":"2023-09-13T03:11:23.261637+0000","last_peered":"2023-09-13T03:11:23.261637+0000","last_clean":"2023-09-13T03:11:23.261637+0000","last_became_active":"2023-09-13T03:11:10.951566+0000","last_became_peered":"2023-09-13T03:11:10.951566+0000","last_unstale":"2023-09-13T03:11:23.261637+0000","last_undegraded":"2023-09-13T03:11:23.261637+0000","last_fullsized":"2023-09-13T03:11:23.261637+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:23:06.020880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.268154+0000","last_change":"2023-09-13T03:11:10.951695+0000","last_active":"2023-09-13T03:11:23.268154+0000","last_peered":"2023-09-13T03:11:23.268154+0000","last_clean":"2023-09-13T03:11:23.268154+0000","last_became_active":"2023-09-13T03:11:10.951382+0000","last_became_peered":"2023-09-13T03:11:10.951382+0000","last_unstale":"2023-09-13T03:11:23.268154+0000","last_undegraded":"2023-09-13T03:11:23.268154+0000","last_fullsized":"2023-09-13T03:11:23.268154+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:29:05.200980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'13","reported_seq":73,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:00.306164+0000","last_change":"2023-09-13T03:11:09.956193+0000","last_active":"2023-09-13T03:12:00.306164+0000","last_peered":"2023-09-13T03:12:00.306164+0000","last_clean":"2023-09-13T03:12:00.306164+0000","last_became_active":"2023-09-13T03:11:09.955778+0000","last_became_peered":"2023-09-13T03:11:09.955778+0000","last_unstale":"2023-09-13T03:12:00.306164+0000","last_undegraded":"2023-09-13T03:12:00.306164+0000","last_fullsized":"2023-09-13T03:12:00.306164+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:15:57.425730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":45,"num_read_kb":45,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.266371+0000","last_change":"2023-09-13T03:11:10.944437+0000","last_active":"2023-09-13T03:11:23.266371+0000","last_peered":"2023-09-13T03:11:23.266371+0000","last_clean":"2023-09-13T03:11:23.266371+0000","last_became_active":"2023-09-13T03:11:10.943961+0000","last_became_peered":"2023-09-13T03:11:10.943961+0000","last_unstale":"2023-09-13T03:11:23.266371+0000","last_undegraded":"2023-09-13T03:11:23.266371+0000","last_fullsized":"2023-09-13T03:11:23.266371+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:39:43.928334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.266435+0000","last_change":"2023-09-13T03:11:09.952017+0000","last_active":"2023-09-13T03:11:23.266435+0000","last_peered":"2023-09-13T03:11:23.266435+0000","last_clean":"2023-09-13T03:11:23.266435+0000","last_became_active":"2023-09-13T03:11:09.951700+0000","last_became_peered":"2023-09-13T03:11:09.951700+0000","last_unstale":"2023-09-13T03:11:23.266435+0000","last_undegraded":"2023-09-13T03:11:23.266435+0000","last_fullsized":"2023-09-13T03:11:23.266435+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:28:50.687271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.267073+0000","last_change":"2023-09-13T03:11:10.960198+0000","last_active":"2023-09-13T03:11:23.267073+0000","last_peered":"2023-09-13T03:11:23.267073+0000","last_clean":"2023-09-13T03:11:23.267073+0000","last_became_active":"2023-09-13T03:11:10.959839+0000","last_became_peered":"2023-09-13T03:11:10.959839+0000","last_unstale":"2023-09-13T03:11:23.267073+0000","last_undegraded":"2023-09-13T03:11:23.267073+0000","last_fullsized":"2023-09-13T03:11:23.267073+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:37:16.868093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.264967+0000","last_change":"2023-09-13T03:11:09.954684+0000","last_active":"2023-09-13T03:11:23.264967+0000","last_peered":"2023-09-13T03:11:23.264967+0000","last_clean":"2023-09-13T03:11:23.264967+0000","last_became_active":"2023-09-13T03:11:09.954392+0000","last_became_peered":"2023-09-13T03:11:09.954392+0000","last_unstale":"2023-09-13T03:11:23.264967+0000","last_undegraded":"2023-09-13T03:11:23.264967+0000","last_fullsized":"2023-09-13T03:11:23.264967+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:29:20.333063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.269329+0000","last_change":"2023-09-13T03:11:10.944364+0000","last_active":"2023-09-13T03:11:23.269329+0000","last_peered":"2023-09-13T03:11:23.269329+0000","last_clean":"2023-09-13T03:11:23.269329+0000","last_became_active":"2023-09-13T03:11:10.943971+0000","last_became_peered":"2023-09-13T03:11:10.943971+0000","last_unstale":"2023-09-13T03:11:23.269329+0000","last_undegraded":"2023-09-13T03:11:23.269329+0000","last_fullsized":"2023-09-13T03:11:23.269329+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:55:07.831397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.265196+0000","last_change":"2023-09-13T03:11:09.952751+0000","last_active":"2023-09-13T03:11:23.265196+0000","last_peered":"2023-09-13T03:11:23.265196+0000","last_clean":"2023-09-13T03:11:23.265196+0000","last_became_active":"2023-09-13T03:11:09.952447+0000","last_became_peered":"2023-09-13T03:11:09.952447+0000","last_unstale":"2023-09-13T03:11:23.265196+0000","last_undegraded":"2023-09-13T03:11:23.265196+0000","last_fullsized":"2023-09-13T03:11:23.265196+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:55:09.971742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:11:24.460373+0000","last_change":"2023-09-13T03:11:24.460373+0000","last_active":"2023-09-13T03:11:24.460373+0000","last_peered":"2023-09-13T03:11:24.460373+0000","last_clean":"2023-09-13T03:11:24.460373+0000","last_became_active":"2023-09-13T03:11:24.459994+0000","last_became_peered":"2023-09-13T03:11:24.459994+0000","last_unstale":"2023-09-13T03:11:24.460373+0000","last_undegraded":"2023-09-13T03:11:24.460373+0000","last_fullsized":"2023-09-13T03:11:24.460373+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:15:58.998213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":68,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:01.309474+0000","last_change":"2023-09-13T03:11:09.947207+0000","last_active":"2023-09-13T03:12:01.309474+0000","last_peered":"2023-09-13T03:12:01.309474+0000","last_clean":"2023-09-13T03:12:01.309474+0000","last_became_active":"2023-09-13T03:11:09.946787+0000","last_became_peered":"2023-09-13T03:11:09.946787+0000","last_unstale":"2023-09-13T03:12:01.309474+0000","last_undegraded":"2023-09-13T03:12:01.309474+0000","last_fullsized":"2023-09-13T03:12:01.309474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:28:47.100723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":47,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.264541+0000","last_change":"2023-09-13T03:11:10.948427+0000","last_active":"2023-09-13T03:11:23.264541+0000","last_peered":"2023-09-13T03:11:23.264541+0000","last_clean":"2023-09-13T03:11:23.264541+0000","last_became_active":"2023-09-13T03:11:10.948095+0000","last_became_peered":"2023-09-13T03:11:10.948095+0000","last_unstale":"2023-09-13T03:11:23.264541+0000","last_undegraded":"2023-09-13T03:11:23.264541+0000","last_fullsized":"2023-09-13T03:11:23.264541+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:55:07.532772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.270733+0000","last_change":"2023-09-13T03:11:09.952335+0000","last_active":"2023-09-13T03:11:23.270733+0000","last_peered":"2023-09-13T03:11:23.270733+0000","last_clean":"2023-09-13T03:11:23.270733+0000","last_became_active":"2023-09-13T03:11:09.951974+0000","last_became_peered":"2023-09-13T03:11:09.951974+0000","last_unstale":"2023-09-13T03:11:23.270733+0000","last_undegraded":"2023-09-13T03:11:23.270733+0000","last_fullsized":"2023-09-13T03:11:23.270733+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:15:50.681751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:11:24.351777+0000","last_change":"2023-09-13T03:11:24.351298+0000","last_active":"2023-09-13T03:11:24.351777+0000","last_peered":"2023-09-13T03:11:24.351777+0000","last_clean":"2023-09-13T03:11:24.351777+0000","last_became_active":"2023-09-13T03:11:24.350758+0000","last_became_peered":"2023-09-13T03:11:24.350758+0000","last_unstale":"2023-09-13T03:11:24.351777+0000","last_undegraded":"2023-09-13T03:11:24.351777+0000","last_fullsized":"2023-09-13T03:11:24.351777+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:40:20.262641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.261290+0000","last_change":"2023-09-13T03:11:09.957644+0000","last_active":"2023-09-13T03:11:23.261290+0000","last_peered":"2023-09-13T03:11:23.261290+0000","last_clean":"2023-09-13T03:11:23.261290+0000","last_became_active":"2023-09-13T03:11:09.957289+0000","last_became_peered":"2023-09-13T03:11:09.957289+0000","last_unstale":"2023-09-13T03:11:23.261290+0000","last_undegraded":"2023-09-13T03:11:23.261290+0000","last_fullsized":"2023-09-13T03:11:23.261290+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:40:48.615322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.266602+0000","last_change":"2023-09-13T03:11:10.947561+0000","last_active":"2023-09-13T03:11:23.266602+0000","last_peered":"2023-09-13T03:11:23.266602+0000","last_clean":"2023-09-13T03:11:23.266602+0000","last_became_active":"2023-09-13T03:11:10.947238+0000","last_became_peered":"2023-09-13T03:11:10.947238+0000","last_unstale":"2023-09-13T03:11:23.266602+0000","last_undegraded":"2023-09-13T03:11:23.266602+0000","last_fullsized":"2023-09-13T03:11:23.266602+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:54:31.360657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.267100+0000","last_change":"2023-09-13T03:11:09.947806+0000","last_active":"2023-09-13T03:11:23.267100+0000","last_peered":"2023-09-13T03:11:23.267100+0000","last_clean":"2023-09-13T03:11:23.267100+0000","last_became_active":"2023-09-13T03:11:09.947428+0000","last_became_peered":"2023-09-13T03:11:09.947428+0000","last_unstale":"2023-09-13T03:11:23.267100+0000","last_undegraded":"2023-09-13T03:11:23.267100+0000","last_fullsized":"2023-09-13T03:11:23.267100+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:25:00.421873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.261727+0000","last_change":"2023-09-13T03:11:10.951637+0000","last_active":"2023-09-13T03:11:23.261727+0000","last_peered":"2023-09-13T03:11:23.261727+0000","last_clean":"2023-09-13T03:11:23.261727+0000","last_became_active":"2023-09-13T03:11:10.951249+0000","last_became_peered":"2023-09-13T03:11:10.951249+0000","last_unstale":"2023-09-13T03:11:23.261727+0000","last_undegraded":"2023-09-13T03:11:23.261727+0000","last_fullsized":"2023-09-13T03:11:23.261727+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:24:27.715167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":109,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:02.310067+0000","last_change":"2023-09-13T03:11:09.954343+0000","last_active":"2023-09-13T03:12:02.310067+0000","last_peered":"2023-09-13T03:12:02.310067+0000","last_clean":"2023-09-13T03:12:02.310067+0000","last_became_active":"2023-09-13T03:11:09.954063+0000","last_became_peered":"2023-09-13T03:11:09.954063+0000","last_unstale":"2023-09-13T03:12:02.310067+0000","last_undegraded":"2023-09-13T03:12:02.310067+0000","last_fullsized":"2023-09-13T03:12:02.310067+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:34:50.057539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":90,"num_read_kb":90,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.270013+0000","last_change":"2023-09-13T03:11:10.956994+0000","last_active":"2023-09-13T03:11:23.270013+0000","last_peered":"2023-09-13T03:11:23.270013+0000","last_clean":"2023-09-13T03:11:23.270013+0000","last_became_active":"2023-09-13T03:11:10.956336+0000","last_became_peered":"2023-09-13T03:11:10.956336+0000","last_unstale":"2023-09-13T03:11:23.270013+0000","last_undegraded":"2023-09-13T03:11:23.270013+0000","last_fullsized":"2023-09-13T03:11:23.270013+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:11:54.186143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.262358+0000","last_change":"2023-09-13T03:11:09.945775+0000","last_active":"2023-09-13T03:11:23.262358+0000","last_peered":"2023-09-13T03:11:23.262358+0000","last_clean":"2023-09-13T03:11:23.262358+0000","last_became_active":"2023-09-13T03:11:09.945406+0000","last_became_peered":"2023-09-13T03:11:09.945406+0000","last_unstale":"2023-09-13T03:11:23.262358+0000","last_undegraded":"2023-09-13T03:11:23.262358+0000","last_fullsized":"2023-09-13T03:11:23.262358+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:29:26.677351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.266969+0000","last_change":"2023-09-13T03:11:10.951941+0000","last_active":"2023-09-13T03:11:23.266969+0000","last_peered":"2023-09-13T03:11:23.266969+0000","last_clean":"2023-09-13T03:11:23.266969+0000","last_became_active":"2023-09-13T03:11:10.951331+0000","last_became_peered":"2023-09-13T03:11:10.951331+0000","last_unstale":"2023-09-13T03:11:23.266969+0000","last_undegraded":"2023-09-13T03:11:23.266969+0000","last_fullsized":"2023-09-13T03:11:23.266969+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:15:53.741657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.268129+0000","last_change":"2023-09-13T03:11:09.959503+0000","last_active":"2023-09-13T03:11:23.268129+0000","last_peered":"2023-09-13T03:11:23.268129+0000","last_clean":"2023-09-13T03:11:23.268129+0000","last_became_active":"2023-09-13T03:11:09.959166+0000","last_became_peered":"2023-09-13T03:11:09.959166+0000","last_unstale":"2023-09-13T03:11:23.268129+0000","last_undegraded":"2023-09-13T03:11:23.268129+0000","last_fullsized":"2023-09-13T03:11:23.268129+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:12:36.179520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.269760+0000","last_change":"2023-09-13T03:11:10.941575+0000","last_active":"2023-09-13T03:11:23.269760+0000","last_peered":"2023-09-13T03:11:23.269760+0000","last_clean":"2023-09-13T03:11:23.269760+0000","last_became_active":"2023-09-13T03:11:10.941285+0000","last_became_peered":"2023-09-13T03:11:10.941285+0000","last_unstale":"2023-09-13T03:11:23.269760+0000","last_undegraded":"2023-09-13T03:11:23.269760+0000","last_fullsized":"2023-09-13T03:11:23.269760+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:03:09.377471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'29","reported_seq":141,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:02.313879+0000","last_change":"2023-09-13T03:11:09.948041+0000","last_active":"2023-09-13T03:12:02.313879+0000","last_peered":"2023-09-13T03:12:02.313879+0000","last_clean":"2023-09-13T03:12:02.313879+0000","last_became_active":"2023-09-13T03:11:09.947785+0000","last_became_peered":"2023-09-13T03:11:09.947785+0000","last_unstale":"2023-09-13T03:12:02.313879+0000","last_undegraded":"2023-09-13T03:12:02.313879+0000","last_fullsized":"2023-09-13T03:12:02.313879+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:46:47.348307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57844,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":97,"num_read_kb":105,"num_write":31,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.264708+0000","last_change":"2023-09-13T03:11:10.950709+0000","last_active":"2023-09-13T03:11:23.264708+0000","last_peered":"2023-09-13T03:11:23.264708+0000","last_clean":"2023-09-13T03:11:23.264708+0000","last_became_active":"2023-09-13T03:11:10.950395+0000","last_became_peered":"2023-09-13T03:11:10.950395+0000","last_unstale":"2023-09-13T03:11:23.264708+0000","last_undegraded":"2023-09-13T03:11:23.264708+0000","last_fullsized":"2023-09-13T03:11:23.264708+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:50:08.628777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:11:24.281233+0000","last_change":"2023-09-13T03:11:24.281233+0000","last_active":"2023-09-13T03:11:24.281233+0000","last_peered":"2023-09-13T03:11:24.281233+0000","last_clean":"2023-09-13T03:11:24.281233+0000","last_became_active":"2023-09-13T03:11:24.269814+0000","last_became_peered":"2023-09-13T03:11:24.269814+0000","last_unstale":"2023-09-13T03:11:24.281233+0000","last_undegraded":"2023-09-13T03:11:24.281233+0000","last_fullsized":"2023-09-13T03:11:24.281233+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:09:00.743406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.261538+0000","last_change":"2023-09-13T03:11:10.955763+0000","last_active":"2023-09-13T03:11:23.261538+0000","last_peered":"2023-09-13T03:11:23.261538+0000","last_clean":"2023-09-13T03:11:23.261538+0000","last_became_active":"2023-09-13T03:11:10.955455+0000","last_became_peered":"2023-09-13T03:11:10.955455+0000","last_unstale":"2023-09-13T03:11:23.261538+0000","last_undegraded":"2023-09-13T03:11:23.261538+0000","last_fullsized":"2023-09-13T03:11:23.261538+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:43:37.127490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.267485+0000","last_change":"2023-09-13T03:11:09.947739+0000","last_active":"2023-09-13T03:11:23.267485+0000","last_peered":"2023-09-13T03:11:23.267485+0000","last_clean":"2023-09-13T03:11:23.267485+0000","last_became_active":"2023-09-13T03:11:09.947348+0000","last_became_peered":"2023-09-13T03:11:09.947348+0000","last_unstale":"2023-09-13T03:11:23.267485+0000","last_undegraded":"2023-09-13T03:11:23.267485+0000","last_fullsized":"2023-09-13T03:11:23.267485+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:47:58.408877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.262430+0000","last_change":"2023-09-13T03:11:10.958140+0000","last_active":"2023-09-13T03:11:23.262430+0000","last_peered":"2023-09-13T03:11:23.262430+0000","last_clean":"2023-09-13T03:11:23.262430+0000","last_became_active":"2023-09-13T03:11:10.957798+0000","last_became_peered":"2023-09-13T03:11:10.957798+0000","last_unstale":"2023-09-13T03:11:23.262430+0000","last_undegraded":"2023-09-13T03:11:23.262430+0000","last_fullsized":"2023-09-13T03:11:23.262430+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:44:40.208559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'6","reported_seq":67,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:00.306888+0000","last_change":"2023-09-13T03:11:09.945554+0000","last_active":"2023-09-13T03:12:00.306888+0000","last_peered":"2023-09-13T03:12:00.306888+0000","last_clean":"2023-09-13T03:12:00.306888+0000","last_became_active":"2023-09-13T03:11:09.945214+0000","last_became_peered":"2023-09-13T03:11:09.945214+0000","last_unstale":"2023-09-13T03:12:00.306888+0000","last_undegraded":"2023-09-13T03:12:00.306888+0000","last_fullsized":"2023-09-13T03:12:00.306888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:46:12.338617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":47,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.261918+0000","last_change":"2023-09-13T03:11:10.952589+0000","last_active":"2023-09-13T03:11:23.261918+0000","last_peered":"2023-09-13T03:11:23.261918+0000","last_clean":"2023-09-13T03:11:23.261918+0000","last_became_active":"2023-09-13T03:11:10.952302+0000","last_became_peered":"2023-09-13T03:11:10.952302+0000","last_unstale":"2023-09-13T03:11:23.261918+0000","last_undegraded":"2023-09-13T03:11:23.261918+0000","last_fullsized":"2023-09-13T03:11:23.261918+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:13:46.972293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.264367+0000","last_change":"2023-09-13T03:11:09.959509+0000","last_active":"2023-09-13T03:11:23.264367+0000","last_peered":"2023-09-13T03:11:23.264367+0000","last_clean":"2023-09-13T03:11:23.264367+0000","last_became_active":"2023-09-13T03:11:09.959150+0000","last_became_peered":"2023-09-13T03:11:09.959150+0000","last_unstale":"2023-09-13T03:11:23.264367+0000","last_undegraded":"2023-09-13T03:11:23.264367+0000","last_fullsized":"2023-09-13T03:11:23.264367+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:33:30.648283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.267783+0000","last_change":"2023-09-13T03:11:10.956610+0000","last_active":"2023-09-13T03:11:23.267783+0000","last_peered":"2023-09-13T03:11:23.267783+0000","last_clean":"2023-09-13T03:11:23.267783+0000","last_became_active":"2023-09-13T03:11:10.955726+0000","last_became_peered":"2023-09-13T03:11:10.955726+0000","last_unstale":"2023-09-13T03:11:23.267783+0000","last_undegraded":"2023-09-13T03:11:23.267783+0000","last_fullsized":"2023-09-13T03:11:23.267783+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:55:45.667752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.264782+0000","last_change":"2023-09-13T03:11:09.949951+0000","last_active":"2023-09-13T03:11:23.264782+0000","last_peered":"2023-09-13T03:11:23.264782+0000","last_clean":"2023-09-13T03:11:23.264782+0000","last_became_active":"2023-09-13T03:11:09.949106+0000","last_became_peered":"2023-09-13T03:11:09.949106+0000","last_unstale":"2023-09-13T03:11:23.264782+0000","last_undegraded":"2023-09-13T03:11:23.264782+0000","last_fullsized":"2023-09-13T03:11:23.264782+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:30:30.092007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.265653+0000","last_change":"2023-09-13T03:11:10.942255+0000","last_active":"2023-09-13T03:11:23.265653+0000","last_peered":"2023-09-13T03:11:23.265653+0000","last_clean":"2023-09-13T03:11:23.265653+0000","last_became_active":"2023-09-13T03:11:10.941953+0000","last_became_peered":"2023-09-13T03:11:10.941953+0000","last_unstale":"2023-09-13T03:11:23.265653+0000","last_undegraded":"2023-09-13T03:11:23.265653+0000","last_fullsized":"2023-09-13T03:11:23.265653+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:38:55.260227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.262253+0000","last_change":"2023-09-13T03:11:09.956042+0000","last_active":"2023-09-13T03:11:23.262253+0000","last_peered":"2023-09-13T03:11:23.262253+0000","last_clean":"2023-09-13T03:11:23.262253+0000","last_became_active":"2023-09-13T03:11:09.955669+0000","last_became_peered":"2023-09-13T03:11:09.955669+0000","last_unstale":"2023-09-13T03:11:23.262253+0000","last_undegraded":"2023-09-13T03:11:23.262253+0000","last_fullsized":"2023-09-13T03:11:23.262253+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:16:25.227447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.270951+0000","last_change":"2023-09-13T03:11:10.952421+0000","last_active":"2023-09-13T03:11:23.270951+0000","last_peered":"2023-09-13T03:11:23.270951+0000","last_clean":"2023-09-13T03:11:23.270951+0000","last_became_active":"2023-09-13T03:11:10.951664+0000","last_became_peered":"2023-09-13T03:11:10.951664+0000","last_unstale":"2023-09-13T03:11:23.270951+0000","last_undegraded":"2023-09-13T03:11:23.270951+0000","last_fullsized":"2023-09-13T03:11:23.270951+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:46:52.664524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'5","reported_seq":63,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:11:58.303555+0000","last_change":"2023-09-13T03:11:09.952286+0000","last_active":"2023-09-13T03:11:58.303555+0000","last_peered":"2023-09-13T03:11:58.303555+0000","last_clean":"2023-09-13T03:11:58.303555+0000","last_became_active":"2023-09-13T03:11:09.951999+0000","last_became_peered":"2023-09-13T03:11:09.951999+0000","last_unstale":"2023-09-13T03:11:58.303555+0000","last_undegraded":"2023-09-13T03:11:58.303555+0000","last_fullsized":"2023-09-13T03:11:58.303555+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:54:46.910087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.271135+0000","last_change":"2023-09-13T03:11:10.950866+0000","last_active":"2023-09-13T03:11:23.271135+0000","last_peered":"2023-09-13T03:11:23.271135+0000","last_clean":"2023-09-13T03:11:23.271135+0000","last_became_active":"2023-09-13T03:11:10.950562+0000","last_became_peered":"2023-09-13T03:11:10.950562+0000","last_unstale":"2023-09-13T03:11:23.271135+0000","last_undegraded":"2023-09-13T03:11:23.271135+0000","last_fullsized":"2023-09-13T03:11:23.271135+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:55:07.333207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.267318+0000","last_change":"2023-09-13T03:11:10.950736+0000","last_active":"2023-09-13T03:11:23.267318+0000","last_peered":"2023-09-13T03:11:23.267318+0000","last_clean":"2023-09-13T03:11:23.267318+0000","last_became_active":"2023-09-13T03:11:10.950164+0000","last_became_peered":"2023-09-13T03:11:10.950164+0000","last_unstale":"2023-09-13T03:11:23.267318+0000","last_undegraded":"2023-09-13T03:11:23.267318+0000","last_fullsized":"2023-09-13T03:11:23.267318+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:22:44.453642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":67,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:02.309191+0000","last_change":"2023-09-13T03:11:09.954070+0000","last_active":"2023-09-13T03:12:02.309191+0000","last_peered":"2023-09-13T03:12:02.309191+0000","last_clean":"2023-09-13T03:12:02.309191+0000","last_became_active":"2023-09-13T03:11:09.953735+0000","last_became_peered":"2023-09-13T03:11:09.953735+0000","last_unstale":"2023-09-13T03:12:02.309191+0000","last_undegraded":"2023-09-13T03:12:02.309191+0000","last_fullsized":"2023-09-13T03:12:02.309191+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:36:59.891449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":47,"num_read_kb":47,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.267022+0000","last_change":"2023-09-13T03:11:10.948509+0000","last_active":"2023-09-13T03:11:23.267022+0000","last_peered":"2023-09-13T03:11:23.267022+0000","last_clean":"2023-09-13T03:11:23.267022+0000","last_became_active":"2023-09-13T03:11:10.948149+0000","last_became_peered":"2023-09-13T03:11:10.948149+0000","last_unstale":"2023-09-13T03:11:23.267022+0000","last_undegraded":"2023-09-13T03:11:23.267022+0000","last_fullsized":"2023-09-13T03:11:23.267022+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:45:54.197368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.265055+0000","last_change":"2023-09-13T03:11:09.954476+0000","last_active":"2023-09-13T03:11:23.265055+0000","last_peered":"2023-09-13T03:11:23.265055+0000","last_clean":"2023-09-13T03:11:23.265055+0000","last_became_active":"2023-09-13T03:11:09.954094+0000","last_became_peered":"2023-09-13T03:11:09.954094+0000","last_unstale":"2023-09-13T03:11:23.265055+0000","last_undegraded":"2023-09-13T03:11:23.265055+0000","last_fullsized":"2023-09-13T03:11:23.265055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:08:23.815751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":320,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.261991+0000","last_change":"2023-09-13T03:10:05.689061+0000","last_active":"2023-09-13T03:11:23.261991+0000","last_peered":"2023-09-13T03:11:23.261991+0000","last_clean":"2023-09-13T03:11:23.261991+0000","last_became_active":"2023-09-13T03:10:05.688557+0000","last_became_peered":"2023-09-13T03:10:05.688557+0000","last_unstale":"2023-09-13T03:11:23.261991+0000","last_undegraded":"2023-09-13T03:11:23.261991+0000","last_fullsized":"2023-09-13T03:11:23.261991+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:06:59.216900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:06:59.216900+0000","last_clean_scrub_stamp":"2023-09-13T03:06:59.216900+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:10:03.279019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.261185+0000","last_change":"2023-09-13T03:11:10.952523+0000","last_active":"2023-09-13T03:11:23.261185+0000","last_peered":"2023-09-13T03:11:23.261185+0000","last_clean":"2023-09-13T03:11:23.261185+0000","last_became_active":"2023-09-13T03:11:10.952193+0000","last_became_peered":"2023-09-13T03:11:10.952193+0000","last_unstale":"2023-09-13T03:11:23.261185+0000","last_undegraded":"2023-09-13T03:11:23.261185+0000","last_fullsized":"2023-09-13T03:11:23.261185+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:31:01.811841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.262105+0000","last_change":"2023-09-13T03:11:09.950414+0000","last_active":"2023-09-13T03:11:23.262105+0000","last_peered":"2023-09-13T03:11:23.262105+0000","last_clean":"2023-09-13T03:11:23.262105+0000","last_became_active":"2023-09-13T03:11:09.950057+0000","last_became_peered":"2023-09-13T03:11:09.950057+0000","last_unstale":"2023-09-13T03:11:23.262105+0000","last_undegraded":"2023-09-13T03:11:23.262105+0000","last_fullsized":"2023-09-13T03:11:23.262105+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:57:01.317880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.264838+0000","last_change":"2023-09-13T03:11:09.955535+0000","last_active":"2023-09-13T03:11:23.264838+0000","last_peered":"2023-09-13T03:11:23.264838+0000","last_clean":"2023-09-13T03:11:23.264838+0000","last_became_active":"2023-09-13T03:11:09.955212+0000","last_became_peered":"2023-09-13T03:11:09.955212+0000","last_unstale":"2023-09-13T03:11:23.264838+0000","last_undegraded":"2023-09-13T03:11:23.264838+0000","last_fullsized":"2023-09-13T03:11:23.264838+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:15:04.365337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.262140+0000","last_change":"2023-09-13T03:11:10.954255+0000","last_active":"2023-09-13T03:11:23.262140+0000","last_peered":"2023-09-13T03:11:23.262140+0000","last_clean":"2023-09-13T03:11:23.262140+0000","last_became_active":"2023-09-13T03:11:10.953866+0000","last_became_peered":"2023-09-13T03:11:10.953866+0000","last_unstale":"2023-09-13T03:11:23.262140+0000","last_undegraded":"2023-09-13T03:11:23.262140+0000","last_fullsized":"2023-09-13T03:11:23.262140+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:25:12.281218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'10","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:11:24.286392+0000","last_change":"2023-09-13T03:11:24.285884+0000","last_active":"2023-09-13T03:11:24.286392+0000","last_peered":"2023-09-13T03:11:24.286392+0000","last_clean":"2023-09-13T03:11:24.286392+0000","last_became_active":"2023-09-13T03:11:24.269933+0000","last_became_peered":"2023-09-13T03:11:24.269933+0000","last_unstale":"2023-09-13T03:11:24.286392+0000","last_undegraded":"2023-09-13T03:11:24.286392+0000","last_fullsized":"2023-09-13T03:11:24.286392+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:30:13.957854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.265672+0000","last_change":"2023-09-13T03:11:10.949037+0000","last_active":"2023-09-13T03:11:23.265672+0000","last_peered":"2023-09-13T03:11:23.265672+0000","last_clean":"2023-09-13T03:11:23.265672+0000","last_became_active":"2023-09-13T03:11:10.948641+0000","last_became_peered":"2023-09-13T03:11:10.948641+0000","last_unstale":"2023-09-13T03:11:23.265672+0000","last_undegraded":"2023-09-13T03:11:23.265672+0000","last_fullsized":"2023-09-13T03:11:23.265672+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:00:57.563300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.265201+0000","last_change":"2023-09-13T03:11:09.946435+0000","last_active":"2023-09-13T03:11:23.265201+0000","last_peered":"2023-09-13T03:11:23.265201+0000","last_clean":"2023-09-13T03:11:23.265201+0000","last_became_active":"2023-09-13T03:11:09.946176+0000","last_became_peered":"2023-09-13T03:11:09.946176+0000","last_unstale":"2023-09-13T03:11:23.265201+0000","last_undegraded":"2023-09-13T03:11:23.265201+0000","last_fullsized":"2023-09-13T03:11:23.265201+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:34:16.055188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.266757+0000","last_change":"2023-09-13T03:11:10.957035+0000","last_active":"2023-09-13T03:11:23.266757+0000","last_peered":"2023-09-13T03:11:23.266757+0000","last_clean":"2023-09-13T03:11:23.266757+0000","last_became_active":"2023-09-13T03:11:10.954656+0000","last_became_peered":"2023-09-13T03:11:10.954656+0000","last_unstale":"2023-09-13T03:11:23.266757+0000","last_undegraded":"2023-09-13T03:11:23.266757+0000","last_fullsized":"2023-09-13T03:11:23.266757+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:54:51.142734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.269239+0000","last_change":"2023-09-13T03:11:09.959358+0000","last_active":"2023-09-13T03:11:23.269239+0000","last_peered":"2023-09-13T03:11:23.269239+0000","last_clean":"2023-09-13T03:11:23.269239+0000","last_became_active":"2023-09-13T03:11:09.959129+0000","last_became_peered":"2023-09-13T03:11:09.959129+0000","last_unstale":"2023-09-13T03:11:23.269239+0000","last_undegraded":"2023-09-13T03:11:23.269239+0000","last_fullsized":"2023-09-13T03:11:23.269239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:29:26.930910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.264403+0000","last_change":"2023-09-13T03:11:10.952588+0000","last_active":"2023-09-13T03:11:23.264403+0000","last_peered":"2023-09-13T03:11:23.264403+0000","last_clean":"2023-09-13T03:11:23.264403+0000","last_became_active":"2023-09-13T03:11:10.952304+0000","last_became_peered":"2023-09-13T03:11:10.952304+0000","last_unstale":"2023-09-13T03:11:23.264403+0000","last_undegraded":"2023-09-13T03:11:23.264403+0000","last_fullsized":"2023-09-13T03:11:23.264403+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:05:49.824053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.266834+0000","last_change":"2023-09-13T03:11:09.956947+0000","last_active":"2023-09-13T03:11:23.266834+0000","last_peered":"2023-09-13T03:11:23.266834+0000","last_clean":"2023-09-13T03:11:23.266834+0000","last_became_active":"2023-09-13T03:11:09.956639+0000","last_became_peered":"2023-09-13T03:11:09.956639+0000","last_unstale":"2023-09-13T03:11:23.266834+0000","last_undegraded":"2023-09-13T03:11:23.266834+0000","last_fullsized":"2023-09-13T03:11:23.266834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:21:37.759130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.265325+0000","last_change":"2023-09-13T03:11:10.947314+0000","last_active":"2023-09-13T03:11:23.265325+0000","last_peered":"2023-09-13T03:11:23.265325+0000","last_clean":"2023-09-13T03:11:23.265325+0000","last_became_active":"2023-09-13T03:11:10.946964+0000","last_became_peered":"2023-09-13T03:11:10.946964+0000","last_unstale":"2023-09-13T03:11:23.265325+0000","last_undegraded":"2023-09-13T03:11:23.265325+0000","last_fullsized":"2023-09-13T03:11:23.265325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:55:08.542303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.268605+0000","last_change":"2023-09-13T03:11:09.952540+0000","last_active":"2023-09-13T03:11:23.268605+0000","last_peered":"2023-09-13T03:11:23.268605+0000","last_clean":"2023-09-13T03:11:23.268605+0000","last_became_active":"2023-09-13T03:11:09.952208+0000","last_became_peered":"2023-09-13T03:11:09.952208+0000","last_unstale":"2023-09-13T03:11:23.268605+0000","last_undegraded":"2023-09-13T03:11:23.268605+0000","last_fullsized":"2023-09-13T03:11:23.268605+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:11:41.200401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.267200+0000","last_change":"2023-09-13T03:11:10.951305+0000","last_active":"2023-09-13T03:11:23.267200+0000","last_peered":"2023-09-13T03:11:23.267200+0000","last_clean":"2023-09-13T03:11:23.267200+0000","last_became_active":"2023-09-13T03:11:10.950982+0000","last_became_peered":"2023-09-13T03:11:10.950982+0000","last_unstale":"2023-09-13T03:11:23.267200+0000","last_undegraded":"2023-09-13T03:11:23.267200+0000","last_fullsized":"2023-09-13T03:11:23.267200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:55:04.497698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:11:24.355955+0000","last_change":"2023-09-13T03:11:24.355955+0000","last_active":"2023-09-13T03:11:24.355955+0000","last_peered":"2023-09-13T03:11:24.355955+0000","last_clean":"2023-09-13T03:11:24.355955+0000","last_became_active":"2023-09-13T03:11:24.351069+0000","last_became_peered":"2023-09-13T03:11:24.351069+0000","last_unstale":"2023-09-13T03:11:24.355955+0000","last_undegraded":"2023-09-13T03:11:24.355955+0000","last_fullsized":"2023-09-13T03:11:24.355955+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:57:55.311708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.262165+0000","last_change":"2023-09-13T03:11:10.959487+0000","last_active":"2023-09-13T03:11:23.262165+0000","last_peered":"2023-09-13T03:11:23.262165+0000","last_clean":"2023-09-13T03:11:23.262165+0000","last_became_active":"2023-09-13T03:11:10.959157+0000","last_became_peered":"2023-09-13T03:11:10.959157+0000","last_unstale":"2023-09-13T03:11:23.262165+0000","last_undegraded":"2023-09-13T03:11:23.262165+0000","last_fullsized":"2023-09-13T03:11:23.262165+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:37:50.852197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'5","reported_seq":68,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:02.310736+0000","last_change":"2023-09-13T03:11:09.951208+0000","last_active":"2023-09-13T03:12:02.310736+0000","last_peered":"2023-09-13T03:12:02.310736+0000","last_clean":"2023-09-13T03:12:02.310736+0000","last_became_active":"2023-09-13T03:11:09.950869+0000","last_became_peered":"2023-09-13T03:11:09.950869+0000","last_unstale":"2023-09-13T03:12:02.310736+0000","last_undegraded":"2023-09-13T03:12:02.310736+0000","last_fullsized":"2023-09-13T03:12:02.310736+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:22:53.697803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":48,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.262465+0000","last_change":"2023-09-13T03:11:10.959129+0000","last_active":"2023-09-13T03:11:23.262465+0000","last_peered":"2023-09-13T03:11:23.262465+0000","last_clean":"2023-09-13T03:11:23.262465+0000","last_became_active":"2023-09-13T03:11:10.958820+0000","last_became_peered":"2023-09-13T03:11:10.958820+0000","last_unstale":"2023-09-13T03:11:23.262465+0000","last_undegraded":"2023-09-13T03:11:23.262465+0000","last_fullsized":"2023-09-13T03:11:23.262465+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:20:30.607854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.267998+0000","last_change":"2023-09-13T03:11:09.946855+0000","last_active":"2023-09-13T03:11:23.267998+0000","last_peered":"2023-09-13T03:11:23.267998+0000","last_clean":"2023-09-13T03:11:23.267998+0000","last_became_active":"2023-09-13T03:11:09.946156+0000","last_became_peered":"2023-09-13T03:11:09.946156+0000","last_unstale":"2023-09-13T03:11:23.267998+0000","last_undegraded":"2023-09-13T03:11:23.267998+0000","last_fullsized":"2023-09-13T03:11:23.267998+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:42:19.387280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.262313+0000","last_change":"2023-09-13T03:11:10.951335+0000","last_active":"2023-09-13T03:11:23.262313+0000","last_peered":"2023-09-13T03:11:23.262313+0000","last_clean":"2023-09-13T03:11:23.262313+0000","last_became_active":"2023-09-13T03:11:10.951034+0000","last_became_peered":"2023-09-13T03:11:10.951034+0000","last_unstale":"2023-09-13T03:11:23.262313+0000","last_undegraded":"2023-09-13T03:11:23.262313+0000","last_fullsized":"2023-09-13T03:11:23.262313+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:57:24.373196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.263077+0000","last_change":"2023-09-13T03:11:09.950937+0000","last_active":"2023-09-13T03:11:23.263077+0000","last_peered":"2023-09-13T03:11:23.263077+0000","last_clean":"2023-09-13T03:11:23.263077+0000","last_became_active":"2023-09-13T03:11:09.950574+0000","last_became_peered":"2023-09-13T03:11:09.950574+0000","last_unstale":"2023-09-13T03:11:23.263077+0000","last_undegraded":"2023-09-13T03:11:23.263077+0000","last_fullsized":"2023-09-13T03:11:23.263077+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:10:23.215080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.262509+0000","last_change":"2023-09-13T03:11:10.950517+0000","last_active":"2023-09-13T03:11:23.262509+0000","last_peered":"2023-09-13T03:11:23.262509+0000","last_clean":"2023-09-13T03:11:23.262509+0000","last_became_active":"2023-09-13T03:11:10.950028+0000","last_became_peered":"2023-09-13T03:11:10.950028+0000","last_unstale":"2023-09-13T03:11:23.262509+0000","last_undegraded":"2023-09-13T03:11:23.262509+0000","last_fullsized":"2023-09-13T03:11:23.262509+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:06:40.008527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.265417+0000","last_change":"2023-09-13T03:11:10.942192+0000","last_active":"2023-09-13T03:11:23.265417+0000","last_peered":"2023-09-13T03:11:23.265417+0000","last_clean":"2023-09-13T03:11:23.265417+0000","last_became_active":"2023-09-13T03:11:10.941893+0000","last_became_peered":"2023-09-13T03:11:10.941893+0000","last_unstale":"2023-09-13T03:11:23.265417+0000","last_undegraded":"2023-09-13T03:11:23.265417+0000","last_fullsized":"2023-09-13T03:11:23.265417+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:26:22.064843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.268237+0000","last_change":"2023-09-13T03:11:10.947186+0000","last_active":"2023-09-13T03:11:23.268237+0000","last_peered":"2023-09-13T03:11:23.268237+0000","last_clean":"2023-09-13T03:11:23.268237+0000","last_became_active":"2023-09-13T03:11:10.946902+0000","last_became_peered":"2023-09-13T03:11:10.946902+0000","last_unstale":"2023-09-13T03:11:23.268237+0000","last_undegraded":"2023-09-13T03:11:23.268237+0000","last_fullsized":"2023-09-13T03:11:23.268237+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:26:01.325597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.268602+0000","last_change":"2023-09-13T03:11:10.946504+0000","last_active":"2023-09-13T03:11:23.268602+0000","last_peered":"2023-09-13T03:11:23.268602+0000","last_clean":"2023-09-13T03:11:23.268602+0000","last_became_active":"2023-09-13T03:11:10.946196+0000","last_became_peered":"2023-09-13T03:11:10.946196+0000","last_unstale":"2023-09-13T03:11:23.268602+0000","last_undegraded":"2023-09-13T03:11:23.268602+0000","last_fullsized":"2023-09-13T03:11:23.268602+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:37:43.578685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.266297+0000","last_change":"2023-09-13T03:11:10.959637+0000","last_active":"2023-09-13T03:11:23.266297+0000","last_peered":"2023-09-13T03:11:23.266297+0000","last_clean":"2023-09-13T03:11:23.266297+0000","last_became_active":"2023-09-13T03:11:10.959324+0000","last_became_peered":"2023-09-13T03:11:10.959324+0000","last_unstale":"2023-09-13T03:11:23.266297+0000","last_undegraded":"2023-09-13T03:11:23.266297+0000","last_fullsized":"2023-09-13T03:11:23.266297+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:48:19.245785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:11:24.576318+0000","last_change":"2023-09-13T03:11:24.576318+0000","last_active":"2023-09-13T03:11:24.576318+0000","last_peered":"2023-09-13T03:11:24.576318+0000","last_clean":"2023-09-13T03:11:24.576318+0000","last_became_active":"2023-09-13T03:11:24.575775+0000","last_became_peered":"2023-09-13T03:11:24.575775+0000","last_unstale":"2023-09-13T03:11:24.576318+0000","last_undegraded":"2023-09-13T03:11:24.576318+0000","last_fullsized":"2023-09-13T03:11:24.576318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:16:00.534554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.265575+0000","last_change":"2023-09-13T03:11:10.945076+0000","last_active":"2023-09-13T03:11:23.265575+0000","last_peered":"2023-09-13T03:11:23.265575+0000","last_clean":"2023-09-13T03:11:23.265575+0000","last_became_active":"2023-09-13T03:11:10.944776+0000","last_became_peered":"2023-09-13T03:11:10.944776+0000","last_unstale":"2023-09-13T03:11:23.265575+0000","last_undegraded":"2023-09-13T03:11:23.265575+0000","last_fullsized":"2023-09-13T03:11:23.265575+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:22:03.624488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.267991+0000","last_change":"2023-09-13T03:11:10.942663+0000","last_active":"2023-09-13T03:11:23.267991+0000","last_peered":"2023-09-13T03:11:23.267991+0000","last_clean":"2023-09-13T03:11:23.267991+0000","last_became_active":"2023-09-13T03:11:10.942203+0000","last_became_peered":"2023-09-13T03:11:10.942203+0000","last_unstale":"2023-09-13T03:11:23.267991+0000","last_undegraded":"2023-09-13T03:11:23.267991+0000","last_fullsized":"2023-09-13T03:11:23.267991+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:46:48.091425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.265584+0000","last_change":"2023-09-13T03:11:10.940089+0000","last_active":"2023-09-13T03:11:23.265584+0000","last_peered":"2023-09-13T03:11:23.265584+0000","last_clean":"2023-09-13T03:11:23.265584+0000","last_became_active":"2023-09-13T03:11:10.939622+0000","last_became_peered":"2023-09-13T03:11:10.939622+0000","last_unstale":"2023-09-13T03:11:23.265584+0000","last_undegraded":"2023-09-13T03:11:23.265584+0000","last_fullsized":"2023-09-13T03:11:23.265584+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:43:55.024944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.267205+0000","last_change":"2023-09-13T03:11:10.956896+0000","last_active":"2023-09-13T03:11:23.267205+0000","last_peered":"2023-09-13T03:11:23.267205+0000","last_clean":"2023-09-13T03:11:23.267205+0000","last_became_active":"2023-09-13T03:11:10.956624+0000","last_became_peered":"2023-09-13T03:11:10.956624+0000","last_unstale":"2023-09-13T03:11:23.267205+0000","last_undegraded":"2023-09-13T03:11:23.267205+0000","last_fullsized":"2023-09-13T03:11:23.267205+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:29:41.305999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.266747+0000","last_change":"2023-09-13T03:11:10.939240+0000","last_active":"2023-09-13T03:11:23.266747+0000","last_peered":"2023-09-13T03:11:23.266747+0000","last_clean":"2023-09-13T03:11:23.266747+0000","last_became_active":"2023-09-13T03:11:10.938869+0000","last_became_peered":"2023-09-13T03:11:10.938869+0000","last_unstale":"2023-09-13T03:11:23.266747+0000","last_undegraded":"2023-09-13T03:11:23.266747+0000","last_fullsized":"2023-09-13T03:11:23.266747+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:06:44.931165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.269653+0000","last_change":"2023-09-13T03:11:10.956704+0000","last_active":"2023-09-13T03:11:23.269653+0000","last_peered":"2023-09-13T03:11:23.269653+0000","last_clean":"2023-09-13T03:11:23.269653+0000","last_became_active":"2023-09-13T03:11:10.956413+0000","last_became_peered":"2023-09-13T03:11:10.956413+0000","last_unstale":"2023-09-13T03:11:23.269653+0000","last_undegraded":"2023-09-13T03:11:23.269653+0000","last_fullsized":"2023-09-13T03:11:23.269653+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:50:04.626111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.261884+0000","last_change":"2023-09-13T03:11:10.952707+0000","last_active":"2023-09-13T03:11:23.261884+0000","last_peered":"2023-09-13T03:11:23.261884+0000","last_clean":"2023-09-13T03:11:23.261884+0000","last_became_active":"2023-09-13T03:11:10.952423+0000","last_became_peered":"2023-09-13T03:11:10.952423+0000","last_unstale":"2023-09-13T03:11:23.261884+0000","last_undegraded":"2023-09-13T03:11:23.261884+0000","last_fullsized":"2023-09-13T03:11:23.261884+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:35:09.869172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.265372+0000","last_change":"2023-09-13T03:11:10.941096+0000","last_active":"2023-09-13T03:11:23.265372+0000","last_peered":"2023-09-13T03:11:23.265372+0000","last_clean":"2023-09-13T03:11:23.265372+0000","last_became_active":"2023-09-13T03:11:10.940761+0000","last_became_peered":"2023-09-13T03:11:10.940761+0000","last_unstale":"2023-09-13T03:11:23.265372+0000","last_undegraded":"2023-09-13T03:11:23.265372+0000","last_fullsized":"2023-09-13T03:11:23.265372+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:34:27.389650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65046,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":480,"num_read_kb":311,"num_write":233,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":68,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":195138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":203,"ondisk_log_size":203,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449111,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":444288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:11:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.56999999999999995}]},{"osd":1,"last update":"Wed Sep 13 03:11:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.504}]},{"osd":2,"last update":"Wed Sep 13 03:11:26 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.441}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":4,"last update":"Wed Sep 13 03:11:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.64700000000000002}]},{"osd":5,"last update":"Wed Sep 13 03:11:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.70699999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":7,"last update":"Wed Sep 13 03:11:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.74199999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":9,"last update":"Wed Sep 13 03:11:26 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.83299999999999996}]},{"osd":10,"last update":"Wed Sep 13 03:11:26 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.72799999999999998}]}]},{"osd":10,"up_from":71,"seq":304942678042,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1093489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":1,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.496}]},{"osd":2,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.52300000000000002}]},{"osd":3,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.746}]},{"osd":4,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.54900000000000004}]},{"osd":5,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.57299999999999995}]},{"osd":6,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":0.86099999999999999}]},{"osd":7,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.47599999999999998}]},{"osd":8,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.77100000000000002}]},{"osd":9,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.94799999999999995}]},{"osd":11,"last update":"Wed Sep 13 03:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.80500000000000005}]}]},{"osd":9,"up_from":66,"seq":283467841567,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":950272,"data_stored":444150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.60199999999999998,"15min":0.61399999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.58699999999999997,"15min":0.59499999999999997},"min":{"1min":0.34999999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.60399999999999998}]},{"osd":1,"last update":"Wed Sep 13 03:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.65800000000000003,"15min":0.66300000000000003},"min":{"1min":0.441,"5min":0.379,"15min":0.379},"max":{"1min":0.89000000000000001,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.73999999999999999,"15min":0.75800000000000001},"min":{"1min":0.48399999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.80000000000000004,"5min":1.107,"15min":1.107},"last":0.68200000000000005}]},{"osd":2,"last update":"Wed Sep 13 03:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.72599999999999998,"15min":0.73599999999999999},"min":{"1min":0.41999999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.96299999999999997,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.76900000000000002,"15min":0.78700000000000003},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.97399999999999998,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.84199999999999997}]},{"osd":3,"last update":"Wed Sep 13 03:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.70199999999999996,"15min":0.71899999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.91000000000000003,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.75,"15min":0.77600000000000002},"min":{"1min":0.39600000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.002,"5min":1.238,"15min":1.238},"last":0.63200000000000001}]},{"osd":4,"last update":"Wed Sep 13 03:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.82999999999999996,"15min":0.85299999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.86499999999999999,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.79800000000000004,"15min":0.81399999999999995},"min":{"1min":0.56999999999999995,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.81100000000000005,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.876}]},{"osd":5,"last update":"Wed Sep 13 03:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.84199999999999997,"15min":0.86499999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0289999999999999,"5min":1.486,"15min":1.486},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.746,"5min":1.0109999999999999,"15min":1.0549999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.0860000000000001,"5min":4.2590000000000003,"15min":4.2590000000000003},"last":0.78300000000000003}]},{"osd":6,"last update":"Wed Sep 13 03:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.67600000000000005}]},{"osd":7,"last update":"Wed Sep 13 03:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":1.056,"15min":1.1000000000000001},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.0660000000000001,"5min":4.367,"15min":4.367},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.84699999999999998,"5min":1.083,"15min":1.1220000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.175,"5min":4.3520000000000003,"15min":4.3520000000000003},"last":0.75}]},{"osd":8,"last update":"Wed Sep 13 03:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.97199999999999998,"15min":0.998},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.113,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.85599999999999998,"5min":1.103,"15min":1.1439999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1000000000000001,"5min":4.298,"15min":4.298},"last":0.95799999999999996}]},{"osd":10,"last update":"Wed Sep 13 03:11:05 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]}]},{"osd":8,"up_from":58,"seq":249108103203,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":444197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40300000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.71999999999999997,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58399999999999996,"15min":0.58199999999999996},"min":{"1min":0.435,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.89900000000000002,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.51100000000000001}]},{"osd":1,"last update":"Wed Sep 13 03:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.45300000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58699999999999997,"15min":0.58799999999999997},"min":{"1min":0.45500000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.71899999999999997,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.69399999999999995}]},{"osd":2,"last update":"Wed Sep 13 03:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.46300000000000002,"15min":0.46500000000000002},"min":{"1min":0.27900000000000003,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.745,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.47399999999999998,"15min":0.47799999999999998},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.65400000000000003,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.47899999999999998}]},{"osd":3,"last update":"Wed Sep 13 03:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.65300000000000002,"15min":0.66000000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.82299999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.63600000000000001}]},{"osd":4,"last update":"Wed Sep 13 03:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60899999999999999,"15min":0.60699999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.77200000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.64300000000000002,"15min":0.64700000000000002},"min":{"1min":0.40100000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.879,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.71999999999999997}]},{"osd":5,"last update":"Wed Sep 13 03:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.55100000000000005,"15min":0.55800000000000005},"min":{"1min":0.28999999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.83199999999999996,"5min":0.995,"15min":0.995},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.60599999999999998,"15min":0.61399999999999999},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.83999999999999997,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.77400000000000002}]},{"osd":6,"last update":"Wed Sep 13 03:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.73599999999999999,"15min":0.747},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.85999999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.75700000000000001,"15min":0.76300000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.86899999999999999,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.876}]},{"osd":7,"last update":"Wed Sep 13 03:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.69499999999999995,"15min":0.70099999999999996},"min":{"1min":0.46700000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.84099999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.755},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71599999999999997,"15min":0.71499999999999997},"min":{"1min":0.53500000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.89000000000000001,"5min":1.054,"15min":1.054},"last":0.83299999999999996}]},{"osd":9,"last update":"Wed Sep 13 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.78800000000000003,"15min":0.78100000000000003},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.0329999999999999,"5min":1.21,"15min":1.21},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.77100000000000002,"15min":0.76300000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0289999999999999,"5min":1.081,"15min":1.081},"last":0.86799999999999999}]},{"osd":10,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.82299999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]}]},{"osd":7,"up_from":51,"seq":219043332135,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28528,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716624,"statfs":{"total":95995035648,"available":95965822976,"internally_reserved":0,"allocated":1540096,"data_stored":1037168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:10:55 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.59699999999999998,"15min":0.58599999999999997},"min":{"1min":0.498,"5min":0.438,"15min":0.438},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.378},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60099999999999998,"15min":0.59099999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.41099999999999998}]},{"osd":1,"last update":"Wed Sep 13 03:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.42999999999999999,"15min":0.40799999999999997},"min":{"1min":0.376,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.625,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.44700000000000001,"15min":0.42099999999999999},"min":{"1min":0.39100000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.60499999999999998}]},{"osd":2,"last update":"Wed Sep 13 03:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.60799999999999998,"15min":0.59399999999999997},"min":{"1min":0.437,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86899999999999999,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.59899999999999998,"15min":0.58099999999999996},"min":{"1min":0.496,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.67400000000000004}]},{"osd":3,"last update":"Wed Sep 13 03:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.68799999999999994,"15min":0.63100000000000001},"min":{"1min":0.46899999999999997,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.81899999999999995,"5min":6.2720000000000002,"15min":6.2720000000000002},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.64700000000000002,"15min":0.625},"min":{"1min":0.48299999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88900000000000001,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.84399999999999997}]},{"osd":4,"last update":"Wed Sep 13 03:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.53900000000000003,"15min":0.51700000000000002},"min":{"1min":0.46800000000000003,"5min":0.314,"15min":0.314},"max":{"1min":0.82399999999999995,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55500000000000005,"15min":0.52700000000000002},"min":{"1min":0.39900000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.88300000000000001,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.57599999999999996}]},{"osd":5,"last update":"Wed Sep 13 03:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.66200000000000003,"15min":0.64300000000000002},"min":{"1min":0.52500000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.65300000000000002,"15min":0.625},"min":{"1min":0.56699999999999995,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.95599999999999996,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.71999999999999997}]},{"osd":6,"last update":"Wed Sep 13 03:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.72999999999999998,"15min":0.70199999999999996},"min":{"1min":0.57499999999999996,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.98999999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.75800000000000001,"15min":0.72799999999999998},"min":{"1min":0.622,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0129999999999999,"5min":1.212,"15min":1.212},"last":0.86899999999999999}]},{"osd":8,"last update":"Wed Sep 13 03:11:20 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.66400000000000003,"15min":0.65000000000000002},"min":{"1min":0.53900000000000003,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.70599999999999996,"15min":0.69199999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.81100000000000005}]},{"osd":9,"last update":"Wed Sep 13 03:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.81899999999999995,"15min":0.81200000000000006},"min":{"1min":0.68799999999999994,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.04,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.81200000000000006,"15min":0.80800000000000005},"min":{"1min":0.66300000000000003,"5min":0.628,"15min":0.628},"max":{"1min":1.069,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.86099999999999999}]},{"osd":10,"last update":"Wed Sep 13 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.753}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93600000000000005}]}]},{"osd":6,"up_from":46,"seq":197568495661,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":443873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:11:38 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.496,"15min":0.496},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.66200000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.434},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.52500000000000002,"15min":0.52700000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.66500000000000004,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.38900000000000001}]},{"osd":1,"last update":"Wed Sep 13 03:11:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.64000000000000001,"15min":0.63},"min":{"1min":0.43099999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.89700000000000002,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.627,"15min":0.61599999999999999},"min":{"1min":0.45400000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.878,"5min":1.069,"15min":1.069},"last":0.60699999999999998}]},{"osd":2,"last update":"Wed Sep 13 03:11:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.68000000000000005,"15min":0.67100000000000004},"min":{"1min":0.54900000000000004,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.82299999999999995,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.51000000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88100000000000001,"5min":1.149,"15min":1.149},"last":0.67300000000000004}]},{"osd":3,"last update":"Wed Sep 13 03:11:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.628,"15min":0.63900000000000001},"min":{"1min":0.38500000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.71999999999999997,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.65000000000000002,"15min":0.65200000000000002},"min":{"1min":0.47799999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93200000000000005,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.49199999999999999}]},{"osd":4,"last update":"Wed Sep 13 03:11:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.73099999999999998,"15min":0.72899999999999998},"min":{"1min":0.49299999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.95699999999999996,"5min":1.161,"15min":1.161},"last":0.745},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.754,"15min":0.76200000000000001},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.01,"5min":1.54,"15min":1.54},"last":0.70699999999999996}]},{"osd":5,"last update":"Wed Sep 13 03:11:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.78600000000000003,"15min":0.79900000000000004},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0169999999999999,"5min":1.115,"15min":1.115},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.77700000000000002,"15min":0.77900000000000003},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.88700000000000001,"5min":1.21,"15min":1.21},"last":0.71799999999999997}]},{"osd":7,"last update":"Wed Sep 13 03:12:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77500000000000002,"15min":0.77000000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.1899999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.78600000000000003,"15min":0.75800000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":0.83399999999999996}]},{"osd":8,"last update":"Wed Sep 13 03:11:20 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.877,"15min":0.88700000000000001},"min":{"1min":0.73299999999999998,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.0720000000000001,"5min":1.131,"15min":1.131},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.83199999999999996,"15min":0.83999999999999997},"min":{"1min":0.64000000000000001,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":0.95299999999999996,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.80900000000000005}]},{"osd":9,"last update":"Wed Sep 13 03:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.871,"15min":0.88600000000000001},"min":{"1min":0.63900000000000001,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":0.97099999999999997,"5min":1.319,"15min":1.319},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.92200000000000004,"15min":0.93799999999999994},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0329999999999999,"5min":1.24,"15min":1.24},"last":0.878}]},{"osd":10,"last update":"Wed Sep 13 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.91000000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":502351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.57099999999999995,"15min":0.53900000000000003},"min":{"1min":0.38700000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.88900000000000001,"5min":2.9260000000000002,"15min":2.9260000000000002},"last":0.44},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.54700000000000004,"15min":0.53300000000000003},"min":{"1min":0.34100000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.503}]},{"osd":2,"last update":"Wed Sep 13 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.54400000000000004,"15min":0.52600000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.54900000000000004,"15min":0.52900000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.68700000000000006}]},{"osd":3,"last update":"Wed Sep 13 03:11:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59999999999999998,"15min":0.58599999999999997},"min":{"1min":0.36799999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84499999999999997,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.629,"15min":0.60499999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.66800000000000004}]},{"osd":4,"last update":"Wed Sep 13 03:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.44700000000000001,"15min":0.39700000000000002},"min":{"1min":0.36699999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.69799999999999995,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.48499999999999999,"15min":0.44},"min":{"1min":0.35499999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.41999999999999998}]},{"osd":5,"last update":"Wed Sep 13 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.57199999999999995,"15min":0.53400000000000003},"min":{"1min":0.44900000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.754},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.57799999999999996,"15min":0.54500000000000004},"min":{"1min":0.46500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.80000000000000004}]},{"osd":6,"last update":"Wed Sep 13 03:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.66900000000000004,"15min":0.65500000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.69899999999999995,"15min":0.67300000000000004},"min":{"1min":0.46800000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.90900000000000003,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":0.59899999999999998}]},{"osd":7,"last update":"Wed Sep 13 03:12:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.59899999999999998,"15min":0.57099999999999995},"min":{"1min":0.45100000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.61599999999999999,"15min":0.60399999999999998},"min":{"1min":0.45900000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84799999999999998,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.70099999999999996}]},{"osd":8,"last update":"Wed Sep 13 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.66700000000000004,"15min":0.66400000000000003},"min":{"1min":0.499,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.72499999999999998,"15min":0.71999999999999997},"min":{"1min":0.55200000000000005,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.90400000000000003}]},{"osd":9,"last update":"Wed Sep 13 03:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.77900000000000003,"15min":0.77500000000000002},"min":{"1min":0.57499999999999996,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.82699999999999996,"15min":0.82599999999999996},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.995,"5min":1.024,"15min":1.024},"last":0.88100000000000001}]},{"osd":10,"last update":"Wed Sep 13 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.84299999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92800000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":954368,"data_stored":445206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 13 03:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.61399999999999999,"15min":0.56000000000000005},"min":{"1min":0.38,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.97299999999999998,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.64700000000000002,"15min":0.57799999999999996},"min":{"1min":0.39500000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":6.0039999999999996,"5min":6.0039999999999996,"15min":6.0039999999999996},"last":0.55000000000000004}]},{"osd":2,"last update":"Wed Sep 13 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.64800000000000002,"15min":0.60799999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.621,"15min":0.59099999999999997},"min":{"1min":0.5,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.86599999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.69099999999999995}]},{"osd":3,"last update":"Wed Sep 13 03:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.46800000000000003,"15min":0.45800000000000002},"min":{"1min":0.309,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.437},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.56200000000000006,"15min":0.54700000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.47799999999999998}]},{"osd":4,"last update":"Wed Sep 13 03:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.74199999999999999,"15min":0.71499999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":5.9370000000000003,"5min":5.9370000000000003,"15min":5.9370000000000003},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69299999999999995,"15min":0.68300000000000005},"min":{"1min":0.45300000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0409999999999999,"5min":1.665,"15min":1.665},"last":0.82299999999999995}]},{"osd":5,"last update":"Wed Sep 13 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.70299999999999996,"15min":0.69299999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.90300000000000002,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68600000000000005,"15min":0.67400000000000004},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.88400000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.78800000000000003}]},{"osd":6,"last update":"Wed Sep 13 03:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.72999999999999998,"15min":0.68600000000000005},"min":{"1min":0.442,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":6.1130000000000004,"5min":6.1130000000000004,"15min":6.1130000000000004},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.75800000000000001,"15min":0.74399999999999999},"min":{"1min":0.48699999999999999,"5min":0.434,"15min":0.434},"max":{"1min":2.7229999999999999,"5min":2.7229999999999999,"15min":2.7229999999999999},"last":0.73399999999999999}]},{"osd":7,"last update":"Wed Sep 13 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.74299999999999999,"15min":0.73899999999999999},"min":{"1min":0.56299999999999994,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.626},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.80600000000000005,"15min":0.79400000000000004},"min":{"1min":0.74399999999999999,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1379999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.77900000000000003}]},{"osd":8,"last update":"Wed Sep 13 03:11:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.81000000000000005,"15min":0.81699999999999995},"min":{"1min":0.58299999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":0.97999999999999998,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.77300000000000002,"15min":0.77000000000000002},"min":{"1min":0.60899999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.80700000000000005}]},{"osd":9,"last update":"Wed Sep 13 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.88500000000000001,"15min":0.89500000000000002},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.88300000000000001,"15min":0.89300000000000002},"min":{"1min":0.59399999999999997,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.88100000000000001}]},{"osd":10,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.753},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.93700000000000006}]},{"osd":11,"last update":"Wed Sep 13 03:11:27 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.96999999999999997}]}]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":503166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.50800000000000001,"15min":0.47599999999999998},"min":{"1min":0.35399999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.68000000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.628},{"interface":"front","average":{"1min":0.502,"5min":0.50800000000000001,"15min":0.48299999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73099999999999998,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.53500000000000003}]},{"osd":1,"last update":"Wed Sep 13 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53100000000000003,"15min":0.51400000000000001},"min":{"1min":0.38400000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.69899999999999995,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.52900000000000003,"15min":0.52500000000000002},"min":{"1min":0.40400000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.60799999999999998}]},{"osd":3,"last update":"Wed Sep 13 03:11:37 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.626,"15min":0.56999999999999995},"min":{"1min":0.432,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":2.879,"5min":2.879,"15min":2.879},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.60999999999999999,"15min":0.56899999999999995},"min":{"1min":0.39100000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":2.8969999999999998,"5min":2.8969999999999998,"15min":2.8969999999999998},"last":0.58999999999999997}]},{"osd":4,"last update":"Wed Sep 13 03:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.56000000000000005,"15min":0.55200000000000005},"min":{"1min":0.375,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73999999999999999,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56899999999999995,"15min":0.55900000000000005},"min":{"1min":0.42599999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.752,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.73499999999999999}]},{"osd":5,"last update":"Wed Sep 13 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.442,"15min":0.42399999999999999},"min":{"1min":0.33300000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70499999999999996,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.46100000000000002,"15min":0.442},"min":{"1min":0.29699999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.46100000000000002}]},{"osd":6,"last update":"Wed Sep 13 03:11:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.64100000000000001,"15min":0.622},"min":{"1min":0.40899999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.65000000000000002,"15min":0.621},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.64600000000000002}]},{"osd":7,"last update":"Wed Sep 13 03:10:58 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.63700000000000001,"15min":0.63600000000000001},"min":{"1min":0.47499999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.84699999999999998,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.66100000000000003,"15min":0.65900000000000003},"min":{"1min":0.51700000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.83399999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.497}]},{"osd":8,"last update":"Wed Sep 13 03:11:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57599999999999996,"15min":0.57999999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.67800000000000005,"15min":0.68600000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.66300000000000003}]},{"osd":9,"last update":"Wed Sep 13 03:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.77700000000000002,"15min":0.75700000000000001},"min":{"1min":0.46600000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.6459999999999999,"5min":2.6459999999999999,"15min":2.6459999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.74099999999999999,"15min":0.73599999999999999},"min":{"1min":0.504,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.70299999999999996}]},{"osd":10,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.747},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.77700000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]}]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":443704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:11:30 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.42699999999999999,"15min":0.40300000000000002},"min":{"1min":0.33800000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.59199999999999997,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.46200000000000002,"15min":0.436},"min":{"1min":0.378,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.752,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.42099999999999999}]},{"osd":1,"last update":"Wed Sep 13 03:11:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.63100000000000001,"15min":0.59599999999999997},"min":{"1min":0.47299999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0229999999999999,"5min":1.117,"15min":1.117},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.64000000000000001,"15min":0.59999999999999998},"min":{"1min":0.57499999999999996,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.86599999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.71199999999999997}]},{"osd":2,"last update":"Wed Sep 13 03:11:30 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.67900000000000005,"15min":0.63800000000000001},"min":{"1min":0.58099999999999996,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.89500000000000002,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.68700000000000006,"15min":0.64700000000000002},"min":{"1min":0.55000000000000004,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.052,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.44}]},{"osd":4,"last update":"Wed Sep 13 03:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.68300000000000005,"15min":0.63600000000000001},"min":{"1min":0.49399999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0900000000000001,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.67500000000000004,"15min":0.627},"min":{"1min":0.52000000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0389999999999999,"5min":1.177,"15min":1.177},"last":0.65500000000000003}]},{"osd":5,"last update":"Wed Sep 13 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.69499999999999995,"15min":0.68000000000000005},"min":{"1min":0.53500000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1,"5min":1.012,"15min":1.012},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.71999999999999997,"15min":0.70199999999999996},"min":{"1min":0.53500000000000003,"5min":0.498,"15min":0.498},"max":{"1min":0.95999999999999996,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.61599999999999999}]},{"osd":6,"last update":"Wed Sep 13 03:11:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.64000000000000001,"15min":0.623},"min":{"1min":0.42599999999999999,"5min":0.374,"15min":0.374},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.76800000000000002,"15min":0.749},"min":{"1min":0.53200000000000003,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.93700000000000006,"5min":1.599,"15min":1.599},"last":0.75600000000000001}]},{"osd":7,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.86199999999999999,"15min":0.85399999999999998},"min":{"1min":0.63200000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.4890000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.83999999999999997,"15min":0.82999999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.512,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.82699999999999996}]},{"osd":8,"last update":"Wed Sep 13 03:10:17 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.82199999999999995}]},{"osd":9,"last update":"Wed Sep 13 03:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.91500000000000004,"15min":0.92400000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.135,"5min":1.538,"15min":1.538},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.88800000000000001,"15min":0.90000000000000002},"min":{"1min":0.621,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.198,"5min":1.677,"15min":1.677},"last":0.59399999999999997}]},{"osd":10,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.708,"5min":1.708,"15min":1.708},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":0.79900000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]}]},{"osd":4,"up_from":33,"seq":141733920822,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":444118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.54500000000000004,"15min":0.51500000000000001},"min":{"1min":0.36699999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.55900000000000005,"15min":0.52900000000000003},"min":{"1min":0.35599999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.63800000000000001}]},{"osd":1,"last update":"Wed Sep 13 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.39700000000000002,"15min":0.36199999999999999},"min":{"1min":0.308,"5min":0.245,"15min":0.245},"max":{"1min":0.66700000000000004,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.443,"5min":0.40799999999999997,"15min":0.372},"min":{"1min":0.32500000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.68200000000000005,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.50800000000000001}]},{"osd":2,"last update":"Wed Sep 13 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58099999999999996,"15min":0.56299999999999994},"min":{"1min":0.44,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.81999999999999995,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.58999999999999997,"15min":0.54000000000000004},"min":{"1min":0.38600000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.84999999999999998}]},{"osd":3,"last update":"Wed Sep 13 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63100000000000001,"15min":0.56299999999999994},"min":{"1min":0.46400000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83899999999999997,"5min":4.2919999999999998,"15min":4.2919999999999998},"last":0.752},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.59999999999999998,"15min":0.56999999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81999999999999995,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.66200000000000003}]},{"osd":5,"last update":"Wed Sep 13 03:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59499999999999997,"15min":0.57599999999999996},"min":{"1min":0.42799999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.85199999999999998,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.57999999999999996,"15min":0.56000000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.81299999999999994,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.72899999999999998}]},{"osd":6,"last update":"Wed Sep 13 03:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.753,"15min":0.71499999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":6.1859999999999999,"5min":6.1859999999999999,"15min":6.1859999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.71199999999999997,"15min":0.71399999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.85999999999999999,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.81200000000000006}]},{"osd":7,"last update":"Wed Sep 13 03:11:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.628,"15min":0.60999999999999999},"min":{"1min":0.34200000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.93300000000000005,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63600000000000001,"15min":0.622},"min":{"1min":0.39700000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89700000000000002,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.64600000000000002}]},{"osd":8,"last update":"Wed Sep 13 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71799999999999997,"15min":0.72099999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.95099999999999996}]},{"osd":9,"last update":"Wed Sep 13 03:11:44 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":0.90600000000000003,"15min":0.88},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":6.1529999999999996,"5min":6.1529999999999996,"15min":6.1529999999999996},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.75,"5min":0.81899999999999995,"15min":0.83099999999999996},"min":{"1min":0.58099999999999996,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.96999999999999997,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.83599999999999997}]},{"osd":10,"last update":"Wed Sep 13 03:11:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.90900000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]}]},{"osd":5,"up_from":38,"seq":163208757299,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1556480,"data_stored":1036729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.59699999999999998,"15min":0.61299999999999999},"min":{"1min":0.39000000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78000000000000003,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.65800000000000003,"15min":0.63600000000000001},"min":{"1min":0.375,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":6.0179999999999998,"5min":6.0179999999999998,"15min":6.0179999999999998},"last":0.47199999999999998}]},{"osd":1,"last update":"Wed Sep 13 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.54000000000000004,"15min":0.51800000000000002},"min":{"1min":0.372,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.53300000000000003,"15min":0.51000000000000001},"min":{"1min":0.45200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.84299999999999997}]},{"osd":2,"last update":"Wed Sep 13 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.40100000000000002,"15min":0.38400000000000001},"min":{"1min":0.29099999999999998,"5min":0.193,"15min":0.193},"max":{"1min":0.71299999999999997,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.434,"5min":0.40600000000000003,"15min":0.38700000000000001},"min":{"1min":0.27000000000000002,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.76800000000000002,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.436}]},{"osd":3,"last update":"Wed Sep 13 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.40400000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.69099999999999995,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60099999999999998,"15min":0.58599999999999997},"min":{"1min":0.38300000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.85299999999999998,"5min":1.48,"15min":1.48},"last":0.64900000000000002}]},{"osd":4,"last update":"Wed Sep 13 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.59899999999999998,"15min":0.55700000000000005},"min":{"1min":0.40899999999999997,"5min":0.376,"15min":0.376},"max":{"1min":3.3740000000000001,"5min":3.3740000000000001,"15min":3.3740000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56699999999999995,"15min":0.54900000000000004},"min":{"1min":0.39600000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79400000000000004,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.78800000000000003}]},{"osd":6,"last update":"Wed Sep 13 03:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.68400000000000005,"15min":0.64800000000000002},"min":{"1min":0.52800000000000002,"5min":0.371,"15min":0.371},"max":{"1min":4.8250000000000002,"5min":4.8250000000000002,"15min":4.8250000000000002},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.70899999999999996,"15min":0.66800000000000004},"min":{"1min":0.51000000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":4.8049999999999997,"5min":4.8049999999999997,"15min":4.8049999999999997},"last":0.63200000000000001}]},{"osd":7,"last update":"Wed Sep 13 03:12:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.63400000000000001,"15min":0.63300000000000001},"min":{"1min":0.47899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.86399999999999999,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.66600000000000004,"15min":0.65900000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85499999999999998,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.76200000000000001}]},{"osd":8,"last update":"Wed Sep 13 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.84299999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61599999999999999,"15min":0.61699999999999999},"min":{"1min":0.38800000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.91100000000000003,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.745}]},{"osd":9,"last update":"Wed Sep 13 03:11:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.0640000000000001,"15min":1.0429999999999999},"min":{"1min":0.55900000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":8.3070000000000004,"5min":8.3070000000000004,"15min":8.3070000000000004},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.78500000000000003,"15min":0.76700000000000002},"min":{"1min":0.60299999999999998,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":3.016,"5min":3.016,"15min":3.016},"last":0.73099999999999998}]},{"osd":10,"last update":"Wed Sep 13 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.91800000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":59596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":60138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":59970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-13T03:12:05.398 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph pg dump --format=json 2023-09-13T03:12:05.461 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:05 smithi064 ceph-mon[97675]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-13T03:12:05.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:05 smithi142 ceph-mon[134082]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-13T03:12:05.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:05 smithi170 ceph-mon[134328]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-13T03:12:06.135 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:12:06.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:06 smithi142 ceph-mon[134082]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T03:12:06.537 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:06 smithi064 ceph-mon[97675]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T03:12:06.659 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:06 smithi170 ceph-mon[134328]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T03:12:07.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:07 smithi142 ceph-mon[134082]: pgmap v307: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-13T03:12:07.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.524+0000 7f46301ce700 1 -- 172.21.15.64:0/117469425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4628073350 msgr2=0x7f462810c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:07.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.524+0000 7f46301ce700 1 --2- 172.21.15.64:0/117469425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4628073350 0x7f462810c920 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4618009990 tx=0x7f4618009ca0 comp rx=0 tx=0).stop 2023-09-13T03:12:07.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.525+0000 7f46301ce700 1 -- 172.21.15.64:0/117469425 shutdown_connections 2023-09-13T03:12:07.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.525+0000 7f46301ce700 1 --2- 172.21.15.64:0/117469425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4628072a30 0x7f4628072e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:07.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.525+0000 7f46301ce700 1 --2- 172.21.15.64:0/117469425 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f462810ce60 0x7f462810f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:07.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.525+0000 7f46301ce700 1 --2- 172.21.15.64:0/117469425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4628073350 0x7f462810c920 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:07.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.525+0000 7f46301ce700 1 -- 172.21.15.64:0/117469425 >> 172.21.15.64:0/117469425 conn(0x7f46280fc0f0 msgr2=0x7f46280fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:07.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.525+0000 7f46301ce700 1 -- 172.21.15.64:0/117469425 shutdown_connections 2023-09-13T03:12:07.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.525+0000 7f46301ce700 1 -- 172.21.15.64:0/117469425 wait complete. 2023-09-13T03:12:07.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.526+0000 7f46301ce700 1 Processor -- start 2023-09-13T03:12:07.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.526+0000 7f46301ce700 1 -- start start 2023-09-13T03:12:07.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.526+0000 7f46301ce700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4628072a30 0x7f462819e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:07.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.526+0000 7f46301ce700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4628073350 0x7f462819ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:07.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.526+0000 7f46301ce700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f462810ce60 0x7f4628198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:07.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.526+0000 7f46301ce700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4628198db0 con 0x7f4628072a30 2023-09-13T03:12:07.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.527+0000 7f46301ce700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4628198f20 con 0x7f4628073350 2023-09-13T03:12:07.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.527+0000 7f46301ce700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46281990e0 con 0x7f462810ce60 2023-09-13T03:12:07.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.527+0000 7f462df6a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4628072a30 0x7f462819e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:07.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.527+0000 7f462df6a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4628072a30 0x7f462819e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37714/0 (socket says 172.21.15.64:37714) 2023-09-13T03:12:07.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.527+0000 7f462df6a700 1 -- 172.21.15.64:0/363219099 learned_addr learned my addr 172.21.15.64:0/363219099 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:07.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.527+0000 7f462d769700 1 --2- 172.21.15.64:0/363219099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4628073350 0x7f462819ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:07.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.527+0000 7f462e76b700 1 --2- 172.21.15.64:0/363219099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f462810ce60 0x7f4628198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:07.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.527+0000 7f462df6a700 1 -- 172.21.15.64:0/363219099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f462810ce60 msgr2=0x7f4628198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:07.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.527+0000 7f462df6a700 1 --2- 172.21.15.64:0/363219099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f462810ce60 0x7f4628198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:07.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.527+0000 7f462df6a700 1 -- 172.21.15.64:0/363219099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4628073350 msgr2=0x7f462819ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:07.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.527+0000 7f462df6a700 1 --2- 172.21.15.64:0/363219099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4628073350 0x7f462819ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:07.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.527+0000 7f462df6a700 1 -- 172.21.15.64:0/363219099 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46180095f0 con 0x7f4628072a30 2023-09-13T03:12:07.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.528+0000 7f462df6a700 1 --2- 172.21.15.64:0/363219099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4628072a30 0x7f462819e6c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f462400e9a0 tx=0x7f462400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:07.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.528+0000 7f461effd700 1 -- 172.21.15.64:0/363219099 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4624009790 con 0x7f4628072a30 2023-09-13T03:12:07.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.528+0000 7f461effd700 1 -- 172.21.15.64:0/363219099 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4624010430 con 0x7f4628072a30 2023-09-13T03:12:07.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.528+0000 7f46301ce700 1 -- 172.21.15.64:0/363219099 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46281993c0 con 0x7f4628072a30 2023-09-13T03:12:07.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.528+0000 7f461effd700 1 -- 172.21.15.64:0/363219099 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4624018680 con 0x7f4628072a30 2023-09-13T03:12:07.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.528+0000 7f46301ce700 1 -- 172.21.15.64:0/363219099 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46281a56b0 con 0x7f4628072a30 2023-09-13T03:12:07.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.529+0000 7f46301ce700 1 -- 172.21.15.64:0/363219099 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f462804e300 con 0x7f4628072a30 2023-09-13T03:12:07.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.531+0000 7f461effd700 1 -- 172.21.15.64:0/363219099 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f46240105a0 con 0x7f4628072a30 2023-09-13T03:12:07.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.532+0000 7f461effd700 1 --2- 172.21.15.64:0/363219099 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f46140a0750 0x7f46140a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:07.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.533+0000 7f461effd700 1 -- 172.21.15.64:0/363219099 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f4624014070 con 0x7f4628072a30 2023-09-13T03:12:07.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.533+0000 7f462d769700 1 --2- 172.21.15.64:0/363219099 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f46140a0750 0x7f46140a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:07.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.533+0000 7f462d769700 1 --2- 172.21.15.64:0/363219099 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f46140a0750 0x7f46140a2c10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f461800b2e0 tx=0x7f4618005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:07.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.537+0000 7f461effd700 1 -- 172.21.15.64:0/363219099 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f4624051e10 con 0x7f4628072a30 2023-09-13T03:12:07.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:07 smithi170 ceph-mon[134328]: pgmap v307: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-13T03:12:07.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:07 smithi064 ceph-mon[97675]: pgmap v307: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-13T03:12:07.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.741+0000 7f46301ce700 1 -- 172.21.15.64:0/363219099 --> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f46281128f0 con 0x7f46140a0750 2023-09-13T03:12:07.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.751+0000 7f461effd700 1 -- 172.21.15.64:0/363219099 <== mgr.14150 v2:172.21.15.64:6800/873679308 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340355 (secure 0 0 0) 0x7f46281128f0 con 0x7f46140a0750 2023-09-13T03:12:07.756 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:12:07.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.756+0000 7f46301ce700 1 -- 172.21.15.64:0/363219099 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f46140a0750 msgr2=0x7f46140a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:07.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.756+0000 7f46301ce700 1 --2- 172.21.15.64:0/363219099 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f46140a0750 0x7f46140a2c10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f461800b2e0 tx=0x7f4618005960 comp rx=0 tx=0).stop 2023-09-13T03:12:07.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.756+0000 7f46301ce700 1 -- 172.21.15.64:0/363219099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4628072a30 msgr2=0x7f462819e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:07.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.756+0000 7f46301ce700 1 --2- 172.21.15.64:0/363219099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4628072a30 0x7f462819e6c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f462400e9a0 tx=0x7f462400ecb0 comp rx=0 tx=0).stop 2023-09-13T03:12:07.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.756+0000 7f46301ce700 1 -- 172.21.15.64:0/363219099 shutdown_connections 2023-09-13T03:12:07.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.756+0000 7f46301ce700 1 --2- 172.21.15.64:0/363219099 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f46140a0750 0x7f46140a2c10 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:07.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.756+0000 7f46301ce700 1 --2- 172.21.15.64:0/363219099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f462810ce60 0x7f4628198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:07.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.756+0000 7f46301ce700 1 --2- 172.21.15.64:0/363219099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4628072a30 0x7f462819e6c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:07.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.756+0000 7f46301ce700 1 --2- 172.21.15.64:0/363219099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4628073350 0x7f462819ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:07.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.756+0000 7f46301ce700 1 -- 172.21.15.64:0/363219099 >> 172.21.15.64:0/363219099 conn(0x7f46280fc0f0 msgr2=0x7f462810efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:07.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.756+0000 7f46301ce700 1 -- 172.21.15.64:0/363219099 shutdown_connections 2023-09-13T03:12:07.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:07.757+0000 7f46301ce700 1 -- 172.21.15.64:0/363219099 wait complete. 2023-09-13T03:12:07.762 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-13T03:12:08.220 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2023-09-13T03:12:06.467229+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655660,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":621,"num_read_kb":430,"num_write":416,"num_write_kb":3067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180804,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":342,"ondisk_log_size":342,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337236,"kb_used_data":12996,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604588,"statfs":{"total":1151940427776,"available":1151595098112,"internally_reserved":0,"allocated":13307904,"data_stored":7282439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19130,"internal_metadata":331855174},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002819"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.265869+0000","last_change":"2023-09-13T03:11:10.949449+0000","last_active":"2023-09-13T03:11:23.265869+0000","last_peered":"2023-09-13T03:11:23.265869+0000","last_clean":"2023-09-13T03:11:23.265869+0000","last_became_active":"2023-09-13T03:11:10.948912+0000","last_became_peered":"2023-09-13T03:11:10.948912+0000","last_unstale":"2023-09-13T03:11:23.265869+0000","last_undegraded":"2023-09-13T03:11:23.265869+0000","last_fullsized":"2023-09-13T03:11:23.265869+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:34:27.381352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.266683+0000","last_change":"2023-09-13T03:11:10.956817+0000","last_active":"2023-09-13T03:11:23.266683+0000","last_peered":"2023-09-13T03:11:23.266683+0000","last_clean":"2023-09-13T03:11:23.266683+0000","last_became_active":"2023-09-13T03:11:10.955359+0000","last_became_peered":"2023-09-13T03:11:10.955359+0000","last_unstale":"2023-09-13T03:11:23.266683+0000","last_undegraded":"2023-09-13T03:11:23.266683+0000","last_fullsized":"2023-09-13T03:11:23.266683+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:06:01.055910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.266531+0000","last_change":"2023-09-13T03:11:10.959915+0000","last_active":"2023-09-13T03:11:23.266531+0000","last_peered":"2023-09-13T03:11:23.266531+0000","last_clean":"2023-09-13T03:11:23.266531+0000","last_became_active":"2023-09-13T03:11:10.959611+0000","last_became_peered":"2023-09-13T03:11:10.959611+0000","last_unstale":"2023-09-13T03:11:23.266531+0000","last_undegraded":"2023-09-13T03:11:23.266531+0000","last_fullsized":"2023-09-13T03:11:23.266531+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:31:40.075569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:11:24.269796+0000","last_change":"2023-09-13T03:11:24.269022+0000","last_active":"2023-09-13T03:11:24.269796+0000","last_peered":"2023-09-13T03:11:24.269796+0000","last_clean":"2023-09-13T03:11:24.269796+0000","last_became_active":"2023-09-13T03:11:24.268019+0000","last_became_peered":"2023-09-13T03:11:24.268019+0000","last_unstale":"2023-09-13T03:11:24.269796+0000","last_undegraded":"2023-09-13T03:11:24.269796+0000","last_fullsized":"2023-09-13T03:11:24.269796+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:58:06.518437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.264628+0000","last_change":"2023-09-13T03:11:10.951554+0000","last_active":"2023-09-13T03:11:23.264628+0000","last_peered":"2023-09-13T03:11:23.264628+0000","last_clean":"2023-09-13T03:11:23.264628+0000","last_became_active":"2023-09-13T03:11:10.951252+0000","last_became_peered":"2023-09-13T03:11:10.951252+0000","last_unstale":"2023-09-13T03:11:23.264628+0000","last_undegraded":"2023-09-13T03:11:23.264628+0000","last_fullsized":"2023-09-13T03:11:23.264628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:38:42.890256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.270747+0000","last_change":"2023-09-13T03:11:10.953087+0000","last_active":"2023-09-13T03:11:23.270747+0000","last_peered":"2023-09-13T03:11:23.270747+0000","last_clean":"2023-09-13T03:11:23.270747+0000","last_became_active":"2023-09-13T03:11:10.952773+0000","last_became_peered":"2023-09-13T03:11:10.952773+0000","last_unstale":"2023-09-13T03:11:23.270747+0000","last_undegraded":"2023-09-13T03:11:23.270747+0000","last_fullsized":"2023-09-13T03:11:23.270747+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:14:55.203347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.262021+0000","last_change":"2023-09-13T03:11:10.956058+0000","last_active":"2023-09-13T03:11:23.262021+0000","last_peered":"2023-09-13T03:11:23.262021+0000","last_clean":"2023-09-13T03:11:23.262021+0000","last_became_active":"2023-09-13T03:11:10.955783+0000","last_became_peered":"2023-09-13T03:11:10.955783+0000","last_unstale":"2023-09-13T03:11:23.262021+0000","last_undegraded":"2023-09-13T03:11:23.262021+0000","last_fullsized":"2023-09-13T03:11:23.262021+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:31:06.810338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.266523+0000","last_change":"2023-09-13T03:11:10.950102+0000","last_active":"2023-09-13T03:11:23.266523+0000","last_peered":"2023-09-13T03:11:23.266523+0000","last_clean":"2023-09-13T03:11:23.266523+0000","last_became_active":"2023-09-13T03:11:10.948877+0000","last_became_peered":"2023-09-13T03:11:10.948877+0000","last_unstale":"2023-09-13T03:11:23.266523+0000","last_undegraded":"2023-09-13T03:11:23.266523+0000","last_fullsized":"2023-09-13T03:11:23.266523+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:32:09.168793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.270038+0000","last_change":"2023-09-13T03:11:10.945244+0000","last_active":"2023-09-13T03:11:23.270038+0000","last_peered":"2023-09-13T03:11:23.270038+0000","last_clean":"2023-09-13T03:11:23.270038+0000","last_became_active":"2023-09-13T03:11:10.944951+0000","last_became_peered":"2023-09-13T03:11:10.944951+0000","last_unstale":"2023-09-13T03:11:23.270038+0000","last_undegraded":"2023-09-13T03:11:23.270038+0000","last_fullsized":"2023-09-13T03:11:23.270038+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:42:35.420767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.271052+0000","last_change":"2023-09-13T03:11:10.952312+0000","last_active":"2023-09-13T03:11:23.271052+0000","last_peered":"2023-09-13T03:11:23.271052+0000","last_clean":"2023-09-13T03:11:23.271052+0000","last_became_active":"2023-09-13T03:11:10.952077+0000","last_became_peered":"2023-09-13T03:11:10.952077+0000","last_unstale":"2023-09-13T03:11:23.271052+0000","last_undegraded":"2023-09-13T03:11:23.271052+0000","last_fullsized":"2023-09-13T03:11:23.271052+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:57:10.888184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.262424+0000","last_change":"2023-09-13T03:11:10.959259+0000","last_active":"2023-09-13T03:11:23.262424+0000","last_peered":"2023-09-13T03:11:23.262424+0000","last_clean":"2023-09-13T03:11:23.262424+0000","last_became_active":"2023-09-13T03:11:10.958922+0000","last_became_peered":"2023-09-13T03:11:10.958922+0000","last_unstale":"2023-09-13T03:11:23.262424+0000","last_undegraded":"2023-09-13T03:11:23.262424+0000","last_fullsized":"2023-09-13T03:11:23.262424+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:08:54.680718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.265477+0000","last_change":"2023-09-13T03:11:10.950946+0000","last_active":"2023-09-13T03:11:23.265477+0000","last_peered":"2023-09-13T03:11:23.265477+0000","last_clean":"2023-09-13T03:11:23.265477+0000","last_became_active":"2023-09-13T03:11:10.950633+0000","last_became_peered":"2023-09-13T03:11:10.950633+0000","last_unstale":"2023-09-13T03:11:23.265477+0000","last_undegraded":"2023-09-13T03:11:23.265477+0000","last_fullsized":"2023-09-13T03:11:23.265477+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:37:25.796555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.261964+0000","last_change":"2023-09-13T03:11:10.957712+0000","last_active":"2023-09-13T03:11:23.261964+0000","last_peered":"2023-09-13T03:11:23.261964+0000","last_clean":"2023-09-13T03:11:23.261964+0000","last_became_active":"2023-09-13T03:11:10.957379+0000","last_became_peered":"2023-09-13T03:11:10.957379+0000","last_unstale":"2023-09-13T03:11:23.261964+0000","last_undegraded":"2023-09-13T03:11:23.261964+0000","last_fullsized":"2023-09-13T03:11:23.261964+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:31:17.084185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.266012+0000","last_change":"2023-09-13T03:11:10.952951+0000","last_active":"2023-09-13T03:11:23.266012+0000","last_peered":"2023-09-13T03:11:23.266012+0000","last_clean":"2023-09-13T03:11:23.266012+0000","last_became_active":"2023-09-13T03:11:10.952629+0000","last_became_peered":"2023-09-13T03:11:10.952629+0000","last_unstale":"2023-09-13T03:11:23.266012+0000","last_undegraded":"2023-09-13T03:11:23.266012+0000","last_fullsized":"2023-09-13T03:11:23.266012+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:20:40.802584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.265840+0000","last_change":"2023-09-13T03:11:10.950799+0000","last_active":"2023-09-13T03:11:23.265840+0000","last_peered":"2023-09-13T03:11:23.265840+0000","last_clean":"2023-09-13T03:11:23.265840+0000","last_became_active":"2023-09-13T03:11:10.950500+0000","last_became_peered":"2023-09-13T03:11:10.950500+0000","last_unstale":"2023-09-13T03:11:23.265840+0000","last_undegraded":"2023-09-13T03:11:23.265840+0000","last_fullsized":"2023-09-13T03:11:23.265840+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:24:45.770696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.269219+0000","last_change":"2023-09-13T03:11:10.955141+0000","last_active":"2023-09-13T03:11:23.269219+0000","last_peered":"2023-09-13T03:11:23.269219+0000","last_clean":"2023-09-13T03:11:23.269219+0000","last_became_active":"2023-09-13T03:11:10.954840+0000","last_became_peered":"2023-09-13T03:11:10.954840+0000","last_unstale":"2023-09-13T03:11:23.269219+0000","last_undegraded":"2023-09-13T03:11:23.269219+0000","last_fullsized":"2023-09-13T03:11:23.269219+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:56:08.318784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.265976+0000","last_change":"2023-09-13T03:11:10.949199+0000","last_active":"2023-09-13T03:11:23.265976+0000","last_peered":"2023-09-13T03:11:23.265976+0000","last_clean":"2023-09-13T03:11:23.265976+0000","last_became_active":"2023-09-13T03:11:10.948506+0000","last_became_peered":"2023-09-13T03:11:10.948506+0000","last_unstale":"2023-09-13T03:11:23.265976+0000","last_undegraded":"2023-09-13T03:11:23.265976+0000","last_fullsized":"2023-09-13T03:11:23.265976+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:04:07.714033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.261637+0000","last_change":"2023-09-13T03:11:10.953207+0000","last_active":"2023-09-13T03:11:23.261637+0000","last_peered":"2023-09-13T03:11:23.261637+0000","last_clean":"2023-09-13T03:11:23.261637+0000","last_became_active":"2023-09-13T03:11:10.951566+0000","last_became_peered":"2023-09-13T03:11:10.951566+0000","last_unstale":"2023-09-13T03:11:23.261637+0000","last_undegraded":"2023-09-13T03:11:23.261637+0000","last_fullsized":"2023-09-13T03:11:23.261637+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:23:06.020880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.268154+0000","last_change":"2023-09-13T03:11:10.951695+0000","last_active":"2023-09-13T03:11:23.268154+0000","last_peered":"2023-09-13T03:11:23.268154+0000","last_clean":"2023-09-13T03:11:23.268154+0000","last_became_active":"2023-09-13T03:11:10.951382+0000","last_became_peered":"2023-09-13T03:11:10.951382+0000","last_unstale":"2023-09-13T03:11:23.268154+0000","last_undegraded":"2023-09-13T03:11:23.268154+0000","last_fullsized":"2023-09-13T03:11:23.268154+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:29:05.200980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'13","reported_seq":73,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:00.306164+0000","last_change":"2023-09-13T03:11:09.956193+0000","last_active":"2023-09-13T03:12:00.306164+0000","last_peered":"2023-09-13T03:12:00.306164+0000","last_clean":"2023-09-13T03:12:00.306164+0000","last_became_active":"2023-09-13T03:11:09.955778+0000","last_became_peered":"2023-09-13T03:11:09.955778+0000","last_unstale":"2023-09-13T03:12:00.306164+0000","last_undegraded":"2023-09-13T03:12:00.306164+0000","last_fullsized":"2023-09-13T03:12:00.306164+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:15:57.425730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":45,"num_read_kb":45,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.266371+0000","last_change":"2023-09-13T03:11:10.944437+0000","last_active":"2023-09-13T03:11:23.266371+0000","last_peered":"2023-09-13T03:11:23.266371+0000","last_clean":"2023-09-13T03:11:23.266371+0000","last_became_active":"2023-09-13T03:11:10.943961+0000","last_became_peered":"2023-09-13T03:11:10.943961+0000","last_unstale":"2023-09-13T03:11:23.266371+0000","last_undegraded":"2023-09-13T03:11:23.266371+0000","last_fullsized":"2023-09-13T03:11:23.266371+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:39:43.928334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.266435+0000","last_change":"2023-09-13T03:11:09.952017+0000","last_active":"2023-09-13T03:11:23.266435+0000","last_peered":"2023-09-13T03:11:23.266435+0000","last_clean":"2023-09-13T03:11:23.266435+0000","last_became_active":"2023-09-13T03:11:09.951700+0000","last_became_peered":"2023-09-13T03:11:09.951700+0000","last_unstale":"2023-09-13T03:11:23.266435+0000","last_undegraded":"2023-09-13T03:11:23.266435+0000","last_fullsized":"2023-09-13T03:11:23.266435+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:28:50.687271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.267073+0000","last_change":"2023-09-13T03:11:10.960198+0000","last_active":"2023-09-13T03:11:23.267073+0000","last_peered":"2023-09-13T03:11:23.267073+0000","last_clean":"2023-09-13T03:11:23.267073+0000","last_became_active":"2023-09-13T03:11:10.959839+0000","last_became_peered":"2023-09-13T03:11:10.959839+0000","last_unstale":"2023-09-13T03:11:23.267073+0000","last_undegraded":"2023-09-13T03:11:23.267073+0000","last_fullsized":"2023-09-13T03:11:23.267073+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:37:16.868093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.264967+0000","last_change":"2023-09-13T03:11:09.954684+0000","last_active":"2023-09-13T03:11:23.264967+0000","last_peered":"2023-09-13T03:11:23.264967+0000","last_clean":"2023-09-13T03:11:23.264967+0000","last_became_active":"2023-09-13T03:11:09.954392+0000","last_became_peered":"2023-09-13T03:11:09.954392+0000","last_unstale":"2023-09-13T03:11:23.264967+0000","last_undegraded":"2023-09-13T03:11:23.264967+0000","last_fullsized":"2023-09-13T03:11:23.264967+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:29:20.333063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.269329+0000","last_change":"2023-09-13T03:11:10.944364+0000","last_active":"2023-09-13T03:11:23.269329+0000","last_peered":"2023-09-13T03:11:23.269329+0000","last_clean":"2023-09-13T03:11:23.269329+0000","last_became_active":"2023-09-13T03:11:10.943971+0000","last_became_peered":"2023-09-13T03:11:10.943971+0000","last_unstale":"2023-09-13T03:11:23.269329+0000","last_undegraded":"2023-09-13T03:11:23.269329+0000","last_fullsized":"2023-09-13T03:11:23.269329+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:55:07.831397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.265196+0000","last_change":"2023-09-13T03:11:09.952751+0000","last_active":"2023-09-13T03:11:23.265196+0000","last_peered":"2023-09-13T03:11:23.265196+0000","last_clean":"2023-09-13T03:11:23.265196+0000","last_became_active":"2023-09-13T03:11:09.952447+0000","last_became_peered":"2023-09-13T03:11:09.952447+0000","last_unstale":"2023-09-13T03:11:23.265196+0000","last_undegraded":"2023-09-13T03:11:23.265196+0000","last_fullsized":"2023-09-13T03:11:23.265196+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:55:09.971742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:11:24.460373+0000","last_change":"2023-09-13T03:11:24.460373+0000","last_active":"2023-09-13T03:11:24.460373+0000","last_peered":"2023-09-13T03:11:24.460373+0000","last_clean":"2023-09-13T03:11:24.460373+0000","last_became_active":"2023-09-13T03:11:24.459994+0000","last_became_peered":"2023-09-13T03:11:24.459994+0000","last_unstale":"2023-09-13T03:11:24.460373+0000","last_undegraded":"2023-09-13T03:11:24.460373+0000","last_fullsized":"2023-09-13T03:11:24.460373+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:15:58.998213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":68,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:01.309474+0000","last_change":"2023-09-13T03:11:09.947207+0000","last_active":"2023-09-13T03:12:01.309474+0000","last_peered":"2023-09-13T03:12:01.309474+0000","last_clean":"2023-09-13T03:12:01.309474+0000","last_became_active":"2023-09-13T03:11:09.946787+0000","last_became_peered":"2023-09-13T03:11:09.946787+0000","last_unstale":"2023-09-13T03:12:01.309474+0000","last_undegraded":"2023-09-13T03:12:01.309474+0000","last_fullsized":"2023-09-13T03:12:01.309474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:28:47.100723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":47,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.264541+0000","last_change":"2023-09-13T03:11:10.948427+0000","last_active":"2023-09-13T03:11:23.264541+0000","last_peered":"2023-09-13T03:11:23.264541+0000","last_clean":"2023-09-13T03:11:23.264541+0000","last_became_active":"2023-09-13T03:11:10.948095+0000","last_became_peered":"2023-09-13T03:11:10.948095+0000","last_unstale":"2023-09-13T03:11:23.264541+0000","last_undegraded":"2023-09-13T03:11:23.264541+0000","last_fullsized":"2023-09-13T03:11:23.264541+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:55:07.532772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.270733+0000","last_change":"2023-09-13T03:11:09.952335+0000","last_active":"2023-09-13T03:11:23.270733+0000","last_peered":"2023-09-13T03:11:23.270733+0000","last_clean":"2023-09-13T03:11:23.270733+0000","last_became_active":"2023-09-13T03:11:09.951974+0000","last_became_peered":"2023-09-13T03:11:09.951974+0000","last_unstale":"2023-09-13T03:11:23.270733+0000","last_undegraded":"2023-09-13T03:11:23.270733+0000","last_fullsized":"2023-09-13T03:11:23.270733+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:15:50.681751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:11:24.351777+0000","last_change":"2023-09-13T03:11:24.351298+0000","last_active":"2023-09-13T03:11:24.351777+0000","last_peered":"2023-09-13T03:11:24.351777+0000","last_clean":"2023-09-13T03:11:24.351777+0000","last_became_active":"2023-09-13T03:11:24.350758+0000","last_became_peered":"2023-09-13T03:11:24.350758+0000","last_unstale":"2023-09-13T03:11:24.351777+0000","last_undegraded":"2023-09-13T03:11:24.351777+0000","last_fullsized":"2023-09-13T03:11:24.351777+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:40:20.262641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.261290+0000","last_change":"2023-09-13T03:11:09.957644+0000","last_active":"2023-09-13T03:11:23.261290+0000","last_peered":"2023-09-13T03:11:23.261290+0000","last_clean":"2023-09-13T03:11:23.261290+0000","last_became_active":"2023-09-13T03:11:09.957289+0000","last_became_peered":"2023-09-13T03:11:09.957289+0000","last_unstale":"2023-09-13T03:11:23.261290+0000","last_undegraded":"2023-09-13T03:11:23.261290+0000","last_fullsized":"2023-09-13T03:11:23.261290+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:40:48.615322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.266602+0000","last_change":"2023-09-13T03:11:10.947561+0000","last_active":"2023-09-13T03:11:23.266602+0000","last_peered":"2023-09-13T03:11:23.266602+0000","last_clean":"2023-09-13T03:11:23.266602+0000","last_became_active":"2023-09-13T03:11:10.947238+0000","last_became_peered":"2023-09-13T03:11:10.947238+0000","last_unstale":"2023-09-13T03:11:23.266602+0000","last_undegraded":"2023-09-13T03:11:23.266602+0000","last_fullsized":"2023-09-13T03:11:23.266602+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:54:31.360657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.267100+0000","last_change":"2023-09-13T03:11:09.947806+0000","last_active":"2023-09-13T03:11:23.267100+0000","last_peered":"2023-09-13T03:11:23.267100+0000","last_clean":"2023-09-13T03:11:23.267100+0000","last_became_active":"2023-09-13T03:11:09.947428+0000","last_became_peered":"2023-09-13T03:11:09.947428+0000","last_unstale":"2023-09-13T03:11:23.267100+0000","last_undegraded":"2023-09-13T03:11:23.267100+0000","last_fullsized":"2023-09-13T03:11:23.267100+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:25:00.421873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.261727+0000","last_change":"2023-09-13T03:11:10.951637+0000","last_active":"2023-09-13T03:11:23.261727+0000","last_peered":"2023-09-13T03:11:23.261727+0000","last_clean":"2023-09-13T03:11:23.261727+0000","last_became_active":"2023-09-13T03:11:10.951249+0000","last_became_peered":"2023-09-13T03:11:10.951249+0000","last_unstale":"2023-09-13T03:11:23.261727+0000","last_undegraded":"2023-09-13T03:11:23.261727+0000","last_fullsized":"2023-09-13T03:11:23.261727+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:24:27.715167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":109,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:02.310067+0000","last_change":"2023-09-13T03:11:09.954343+0000","last_active":"2023-09-13T03:12:02.310067+0000","last_peered":"2023-09-13T03:12:02.310067+0000","last_clean":"2023-09-13T03:12:02.310067+0000","last_became_active":"2023-09-13T03:11:09.954063+0000","last_became_peered":"2023-09-13T03:11:09.954063+0000","last_unstale":"2023-09-13T03:12:02.310067+0000","last_undegraded":"2023-09-13T03:12:02.310067+0000","last_fullsized":"2023-09-13T03:12:02.310067+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:34:50.057539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":90,"num_read_kb":90,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.270013+0000","last_change":"2023-09-13T03:11:10.956994+0000","last_active":"2023-09-13T03:11:23.270013+0000","last_peered":"2023-09-13T03:11:23.270013+0000","last_clean":"2023-09-13T03:11:23.270013+0000","last_became_active":"2023-09-13T03:11:10.956336+0000","last_became_peered":"2023-09-13T03:11:10.956336+0000","last_unstale":"2023-09-13T03:11:23.270013+0000","last_undegraded":"2023-09-13T03:11:23.270013+0000","last_fullsized":"2023-09-13T03:11:23.270013+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:11:54.186143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.262358+0000","last_change":"2023-09-13T03:11:09.945775+0000","last_active":"2023-09-13T03:11:23.262358+0000","last_peered":"2023-09-13T03:11:23.262358+0000","last_clean":"2023-09-13T03:11:23.262358+0000","last_became_active":"2023-09-13T03:11:09.945406+0000","last_became_peered":"2023-09-13T03:11:09.945406+0000","last_unstale":"2023-09-13T03:11:23.262358+0000","last_undegraded":"2023-09-13T03:11:23.262358+0000","last_fullsized":"2023-09-13T03:11:23.262358+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:29:26.677351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.266969+0000","last_change":"2023-09-13T03:11:10.951941+0000","last_active":"2023-09-13T03:11:23.266969+0000","last_peered":"2023-09-13T03:11:23.266969+0000","last_clean":"2023-09-13T03:11:23.266969+0000","last_became_active":"2023-09-13T03:11:10.951331+0000","last_became_peered":"2023-09-13T03:11:10.951331+0000","last_unstale":"2023-09-13T03:11:23.266969+0000","last_undegraded":"2023-09-13T03:11:23.266969+0000","last_fullsized":"2023-09-13T03:11:23.266969+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:15:53.741657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.268129+0000","last_change":"2023-09-13T03:11:09.959503+0000","last_active":"2023-09-13T03:11:23.268129+0000","last_peered":"2023-09-13T03:11:23.268129+0000","last_clean":"2023-09-13T03:11:23.268129+0000","last_became_active":"2023-09-13T03:11:09.959166+0000","last_became_peered":"2023-09-13T03:11:09.959166+0000","last_unstale":"2023-09-13T03:11:23.268129+0000","last_undegraded":"2023-09-13T03:11:23.268129+0000","last_fullsized":"2023-09-13T03:11:23.268129+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:12:36.179520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.269760+0000","last_change":"2023-09-13T03:11:10.941575+0000","last_active":"2023-09-13T03:11:23.269760+0000","last_peered":"2023-09-13T03:11:23.269760+0000","last_clean":"2023-09-13T03:11:23.269760+0000","last_became_active":"2023-09-13T03:11:10.941285+0000","last_became_peered":"2023-09-13T03:11:10.941285+0000","last_unstale":"2023-09-13T03:11:23.269760+0000","last_undegraded":"2023-09-13T03:11:23.269760+0000","last_fullsized":"2023-09-13T03:11:23.269760+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:03:09.377471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'29","reported_seq":141,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:02.313879+0000","last_change":"2023-09-13T03:11:09.948041+0000","last_active":"2023-09-13T03:12:02.313879+0000","last_peered":"2023-09-13T03:12:02.313879+0000","last_clean":"2023-09-13T03:12:02.313879+0000","last_became_active":"2023-09-13T03:11:09.947785+0000","last_became_peered":"2023-09-13T03:11:09.947785+0000","last_unstale":"2023-09-13T03:12:02.313879+0000","last_undegraded":"2023-09-13T03:12:02.313879+0000","last_fullsized":"2023-09-13T03:12:02.313879+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:46:47.348307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57844,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":97,"num_read_kb":105,"num_write":31,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.264708+0000","last_change":"2023-09-13T03:11:10.950709+0000","last_active":"2023-09-13T03:11:23.264708+0000","last_peered":"2023-09-13T03:11:23.264708+0000","last_clean":"2023-09-13T03:11:23.264708+0000","last_became_active":"2023-09-13T03:11:10.950395+0000","last_became_peered":"2023-09-13T03:11:10.950395+0000","last_unstale":"2023-09-13T03:11:23.264708+0000","last_undegraded":"2023-09-13T03:11:23.264708+0000","last_fullsized":"2023-09-13T03:11:23.264708+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:50:08.628777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:11:24.281233+0000","last_change":"2023-09-13T03:11:24.281233+0000","last_active":"2023-09-13T03:11:24.281233+0000","last_peered":"2023-09-13T03:11:24.281233+0000","last_clean":"2023-09-13T03:11:24.281233+0000","last_became_active":"2023-09-13T03:11:24.269814+0000","last_became_peered":"2023-09-13T03:11:24.269814+0000","last_unstale":"2023-09-13T03:11:24.281233+0000","last_undegraded":"2023-09-13T03:11:24.281233+0000","last_fullsized":"2023-09-13T03:11:24.281233+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:09:00.743406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.261538+0000","last_change":"2023-09-13T03:11:10.955763+0000","last_active":"2023-09-13T03:11:23.261538+0000","last_peered":"2023-09-13T03:11:23.261538+0000","last_clean":"2023-09-13T03:11:23.261538+0000","last_became_active":"2023-09-13T03:11:10.955455+0000","last_became_peered":"2023-09-13T03:11:10.955455+0000","last_unstale":"2023-09-13T03:11:23.261538+0000","last_undegraded":"2023-09-13T03:11:23.261538+0000","last_fullsized":"2023-09-13T03:11:23.261538+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:43:37.127490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.267485+0000","last_change":"2023-09-13T03:11:09.947739+0000","last_active":"2023-09-13T03:11:23.267485+0000","last_peered":"2023-09-13T03:11:23.267485+0000","last_clean":"2023-09-13T03:11:23.267485+0000","last_became_active":"2023-09-13T03:11:09.947348+0000","last_became_peered":"2023-09-13T03:11:09.947348+0000","last_unstale":"2023-09-13T03:11:23.267485+0000","last_undegraded":"2023-09-13T03:11:23.267485+0000","last_fullsized":"2023-09-13T03:11:23.267485+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:47:58.408877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.262430+0000","last_change":"2023-09-13T03:11:10.958140+0000","last_active":"2023-09-13T03:11:23.262430+0000","last_peered":"2023-09-13T03:11:23.262430+0000","last_clean":"2023-09-13T03:11:23.262430+0000","last_became_active":"2023-09-13T03:11:10.957798+0000","last_became_peered":"2023-09-13T03:11:10.957798+0000","last_unstale":"2023-09-13T03:11:23.262430+0000","last_undegraded":"2023-09-13T03:11:23.262430+0000","last_fullsized":"2023-09-13T03:11:23.262430+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:44:40.208559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'6","reported_seq":67,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:00.306888+0000","last_change":"2023-09-13T03:11:09.945554+0000","last_active":"2023-09-13T03:12:00.306888+0000","last_peered":"2023-09-13T03:12:00.306888+0000","last_clean":"2023-09-13T03:12:00.306888+0000","last_became_active":"2023-09-13T03:11:09.945214+0000","last_became_peered":"2023-09-13T03:11:09.945214+0000","last_unstale":"2023-09-13T03:12:00.306888+0000","last_undegraded":"2023-09-13T03:12:00.306888+0000","last_fullsized":"2023-09-13T03:12:00.306888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:46:12.338617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":47,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.261918+0000","last_change":"2023-09-13T03:11:10.952589+0000","last_active":"2023-09-13T03:11:23.261918+0000","last_peered":"2023-09-13T03:11:23.261918+0000","last_clean":"2023-09-13T03:11:23.261918+0000","last_became_active":"2023-09-13T03:11:10.952302+0000","last_became_peered":"2023-09-13T03:11:10.952302+0000","last_unstale":"2023-09-13T03:11:23.261918+0000","last_undegraded":"2023-09-13T03:11:23.261918+0000","last_fullsized":"2023-09-13T03:11:23.261918+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:13:46.972293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.264367+0000","last_change":"2023-09-13T03:11:09.959509+0000","last_active":"2023-09-13T03:11:23.264367+0000","last_peered":"2023-09-13T03:11:23.264367+0000","last_clean":"2023-09-13T03:11:23.264367+0000","last_became_active":"2023-09-13T03:11:09.959150+0000","last_became_peered":"2023-09-13T03:11:09.959150+0000","last_unstale":"2023-09-13T03:11:23.264367+0000","last_undegraded":"2023-09-13T03:11:23.264367+0000","last_fullsized":"2023-09-13T03:11:23.264367+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:33:30.648283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.267783+0000","last_change":"2023-09-13T03:11:10.956610+0000","last_active":"2023-09-13T03:11:23.267783+0000","last_peered":"2023-09-13T03:11:23.267783+0000","last_clean":"2023-09-13T03:11:23.267783+0000","last_became_active":"2023-09-13T03:11:10.955726+0000","last_became_peered":"2023-09-13T03:11:10.955726+0000","last_unstale":"2023-09-13T03:11:23.267783+0000","last_undegraded":"2023-09-13T03:11:23.267783+0000","last_fullsized":"2023-09-13T03:11:23.267783+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:55:45.667752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.264782+0000","last_change":"2023-09-13T03:11:09.949951+0000","last_active":"2023-09-13T03:11:23.264782+0000","last_peered":"2023-09-13T03:11:23.264782+0000","last_clean":"2023-09-13T03:11:23.264782+0000","last_became_active":"2023-09-13T03:11:09.949106+0000","last_became_peered":"2023-09-13T03:11:09.949106+0000","last_unstale":"2023-09-13T03:11:23.264782+0000","last_undegraded":"2023-09-13T03:11:23.264782+0000","last_fullsized":"2023-09-13T03:11:23.264782+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:30:30.092007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.265653+0000","last_change":"2023-09-13T03:11:10.942255+0000","last_active":"2023-09-13T03:11:23.265653+0000","last_peered":"2023-09-13T03:11:23.265653+0000","last_clean":"2023-09-13T03:11:23.265653+0000","last_became_active":"2023-09-13T03:11:10.941953+0000","last_became_peered":"2023-09-13T03:11:10.941953+0000","last_unstale":"2023-09-13T03:11:23.265653+0000","last_undegraded":"2023-09-13T03:11:23.265653+0000","last_fullsized":"2023-09-13T03:11:23.265653+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:38:55.260227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.262253+0000","last_change":"2023-09-13T03:11:09.956042+0000","last_active":"2023-09-13T03:11:23.262253+0000","last_peered":"2023-09-13T03:11:23.262253+0000","last_clean":"2023-09-13T03:11:23.262253+0000","last_became_active":"2023-09-13T03:11:09.955669+0000","last_became_peered":"2023-09-13T03:11:09.955669+0000","last_unstale":"2023-09-13T03:11:23.262253+0000","last_undegraded":"2023-09-13T03:11:23.262253+0000","last_fullsized":"2023-09-13T03:11:23.262253+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:16:25.227447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.270951+0000","last_change":"2023-09-13T03:11:10.952421+0000","last_active":"2023-09-13T03:11:23.270951+0000","last_peered":"2023-09-13T03:11:23.270951+0000","last_clean":"2023-09-13T03:11:23.270951+0000","last_became_active":"2023-09-13T03:11:10.951664+0000","last_became_peered":"2023-09-13T03:11:10.951664+0000","last_unstale":"2023-09-13T03:11:23.270951+0000","last_undegraded":"2023-09-13T03:11:23.270951+0000","last_fullsized":"2023-09-13T03:11:23.270951+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:46:52.664524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'5","reported_seq":68,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:03.310798+0000","last_change":"2023-09-13T03:11:09.952286+0000","last_active":"2023-09-13T03:12:03.310798+0000","last_peered":"2023-09-13T03:12:03.310798+0000","last_clean":"2023-09-13T03:12:03.310798+0000","last_became_active":"2023-09-13T03:11:09.951999+0000","last_became_peered":"2023-09-13T03:11:09.951999+0000","last_unstale":"2023-09-13T03:12:03.310798+0000","last_undegraded":"2023-09-13T03:12:03.310798+0000","last_fullsized":"2023-09-13T03:12:03.310798+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:54:46.910087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":48,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.271135+0000","last_change":"2023-09-13T03:11:10.950866+0000","last_active":"2023-09-13T03:11:23.271135+0000","last_peered":"2023-09-13T03:11:23.271135+0000","last_clean":"2023-09-13T03:11:23.271135+0000","last_became_active":"2023-09-13T03:11:10.950562+0000","last_became_peered":"2023-09-13T03:11:10.950562+0000","last_unstale":"2023-09-13T03:11:23.271135+0000","last_undegraded":"2023-09-13T03:11:23.271135+0000","last_fullsized":"2023-09-13T03:11:23.271135+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:55:07.333207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.267318+0000","last_change":"2023-09-13T03:11:10.950736+0000","last_active":"2023-09-13T03:11:23.267318+0000","last_peered":"2023-09-13T03:11:23.267318+0000","last_clean":"2023-09-13T03:11:23.267318+0000","last_became_active":"2023-09-13T03:11:10.950164+0000","last_became_peered":"2023-09-13T03:11:10.950164+0000","last_unstale":"2023-09-13T03:11:23.267318+0000","last_undegraded":"2023-09-13T03:11:23.267318+0000","last_fullsized":"2023-09-13T03:11:23.267318+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:22:44.453642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":67,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:02.309191+0000","last_change":"2023-09-13T03:11:09.954070+0000","last_active":"2023-09-13T03:12:02.309191+0000","last_peered":"2023-09-13T03:12:02.309191+0000","last_clean":"2023-09-13T03:12:02.309191+0000","last_became_active":"2023-09-13T03:11:09.953735+0000","last_became_peered":"2023-09-13T03:11:09.953735+0000","last_unstale":"2023-09-13T03:12:02.309191+0000","last_undegraded":"2023-09-13T03:12:02.309191+0000","last_fullsized":"2023-09-13T03:12:02.309191+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:36:59.891449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":47,"num_read_kb":47,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.267022+0000","last_change":"2023-09-13T03:11:10.948509+0000","last_active":"2023-09-13T03:11:23.267022+0000","last_peered":"2023-09-13T03:11:23.267022+0000","last_clean":"2023-09-13T03:11:23.267022+0000","last_became_active":"2023-09-13T03:11:10.948149+0000","last_became_peered":"2023-09-13T03:11:10.948149+0000","last_unstale":"2023-09-13T03:11:23.267022+0000","last_undegraded":"2023-09-13T03:11:23.267022+0000","last_fullsized":"2023-09-13T03:11:23.267022+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:45:54.197368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.265055+0000","last_change":"2023-09-13T03:11:09.954476+0000","last_active":"2023-09-13T03:11:23.265055+0000","last_peered":"2023-09-13T03:11:23.265055+0000","last_clean":"2023-09-13T03:11:23.265055+0000","last_became_active":"2023-09-13T03:11:09.954094+0000","last_became_peered":"2023-09-13T03:11:09.954094+0000","last_unstale":"2023-09-13T03:11:23.265055+0000","last_undegraded":"2023-09-13T03:11:23.265055+0000","last_fullsized":"2023-09-13T03:11:23.265055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:08:23.815751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":320,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.261991+0000","last_change":"2023-09-13T03:10:05.689061+0000","last_active":"2023-09-13T03:11:23.261991+0000","last_peered":"2023-09-13T03:11:23.261991+0000","last_clean":"2023-09-13T03:11:23.261991+0000","last_became_active":"2023-09-13T03:10:05.688557+0000","last_became_peered":"2023-09-13T03:10:05.688557+0000","last_unstale":"2023-09-13T03:11:23.261991+0000","last_undegraded":"2023-09-13T03:11:23.261991+0000","last_fullsized":"2023-09-13T03:11:23.261991+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:06:59.216900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:06:59.216900+0000","last_clean_scrub_stamp":"2023-09-13T03:06:59.216900+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:10:03.279019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.261185+0000","last_change":"2023-09-13T03:11:10.952523+0000","last_active":"2023-09-13T03:11:23.261185+0000","last_peered":"2023-09-13T03:11:23.261185+0000","last_clean":"2023-09-13T03:11:23.261185+0000","last_became_active":"2023-09-13T03:11:10.952193+0000","last_became_peered":"2023-09-13T03:11:10.952193+0000","last_unstale":"2023-09-13T03:11:23.261185+0000","last_undegraded":"2023-09-13T03:11:23.261185+0000","last_fullsized":"2023-09-13T03:11:23.261185+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:31:01.811841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.262105+0000","last_change":"2023-09-13T03:11:09.950414+0000","last_active":"2023-09-13T03:11:23.262105+0000","last_peered":"2023-09-13T03:11:23.262105+0000","last_clean":"2023-09-13T03:11:23.262105+0000","last_became_active":"2023-09-13T03:11:09.950057+0000","last_became_peered":"2023-09-13T03:11:09.950057+0000","last_unstale":"2023-09-13T03:11:23.262105+0000","last_undegraded":"2023-09-13T03:11:23.262105+0000","last_fullsized":"2023-09-13T03:11:23.262105+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:57:01.317880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.264838+0000","last_change":"2023-09-13T03:11:09.955535+0000","last_active":"2023-09-13T03:11:23.264838+0000","last_peered":"2023-09-13T03:11:23.264838+0000","last_clean":"2023-09-13T03:11:23.264838+0000","last_became_active":"2023-09-13T03:11:09.955212+0000","last_became_peered":"2023-09-13T03:11:09.955212+0000","last_unstale":"2023-09-13T03:11:23.264838+0000","last_undegraded":"2023-09-13T03:11:23.264838+0000","last_fullsized":"2023-09-13T03:11:23.264838+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:15:04.365337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.262140+0000","last_change":"2023-09-13T03:11:10.954255+0000","last_active":"2023-09-13T03:11:23.262140+0000","last_peered":"2023-09-13T03:11:23.262140+0000","last_clean":"2023-09-13T03:11:23.262140+0000","last_became_active":"2023-09-13T03:11:10.953866+0000","last_became_peered":"2023-09-13T03:11:10.953866+0000","last_unstale":"2023-09-13T03:11:23.262140+0000","last_undegraded":"2023-09-13T03:11:23.262140+0000","last_fullsized":"2023-09-13T03:11:23.262140+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:25:12.281218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'10","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:11:24.286392+0000","last_change":"2023-09-13T03:11:24.285884+0000","last_active":"2023-09-13T03:11:24.286392+0000","last_peered":"2023-09-13T03:11:24.286392+0000","last_clean":"2023-09-13T03:11:24.286392+0000","last_became_active":"2023-09-13T03:11:24.269933+0000","last_became_peered":"2023-09-13T03:11:24.269933+0000","last_unstale":"2023-09-13T03:11:24.286392+0000","last_undegraded":"2023-09-13T03:11:24.286392+0000","last_fullsized":"2023-09-13T03:11:24.286392+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:30:13.957854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.265672+0000","last_change":"2023-09-13T03:11:10.949037+0000","last_active":"2023-09-13T03:11:23.265672+0000","last_peered":"2023-09-13T03:11:23.265672+0000","last_clean":"2023-09-13T03:11:23.265672+0000","last_became_active":"2023-09-13T03:11:10.948641+0000","last_became_peered":"2023-09-13T03:11:10.948641+0000","last_unstale":"2023-09-13T03:11:23.265672+0000","last_undegraded":"2023-09-13T03:11:23.265672+0000","last_fullsized":"2023-09-13T03:11:23.265672+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:00:57.563300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.265201+0000","last_change":"2023-09-13T03:11:09.946435+0000","last_active":"2023-09-13T03:11:23.265201+0000","last_peered":"2023-09-13T03:11:23.265201+0000","last_clean":"2023-09-13T03:11:23.265201+0000","last_became_active":"2023-09-13T03:11:09.946176+0000","last_became_peered":"2023-09-13T03:11:09.946176+0000","last_unstale":"2023-09-13T03:11:23.265201+0000","last_undegraded":"2023-09-13T03:11:23.265201+0000","last_fullsized":"2023-09-13T03:11:23.265201+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:34:16.055188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.266757+0000","last_change":"2023-09-13T03:11:10.957035+0000","last_active":"2023-09-13T03:11:23.266757+0000","last_peered":"2023-09-13T03:11:23.266757+0000","last_clean":"2023-09-13T03:11:23.266757+0000","last_became_active":"2023-09-13T03:11:10.954656+0000","last_became_peered":"2023-09-13T03:11:10.954656+0000","last_unstale":"2023-09-13T03:11:23.266757+0000","last_undegraded":"2023-09-13T03:11:23.266757+0000","last_fullsized":"2023-09-13T03:11:23.266757+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:54:51.142734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.269239+0000","last_change":"2023-09-13T03:11:09.959358+0000","last_active":"2023-09-13T03:11:23.269239+0000","last_peered":"2023-09-13T03:11:23.269239+0000","last_clean":"2023-09-13T03:11:23.269239+0000","last_became_active":"2023-09-13T03:11:09.959129+0000","last_became_peered":"2023-09-13T03:11:09.959129+0000","last_unstale":"2023-09-13T03:11:23.269239+0000","last_undegraded":"2023-09-13T03:11:23.269239+0000","last_fullsized":"2023-09-13T03:11:23.269239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:29:26.930910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.264403+0000","last_change":"2023-09-13T03:11:10.952588+0000","last_active":"2023-09-13T03:11:23.264403+0000","last_peered":"2023-09-13T03:11:23.264403+0000","last_clean":"2023-09-13T03:11:23.264403+0000","last_became_active":"2023-09-13T03:11:10.952304+0000","last_became_peered":"2023-09-13T03:11:10.952304+0000","last_unstale":"2023-09-13T03:11:23.264403+0000","last_undegraded":"2023-09-13T03:11:23.264403+0000","last_fullsized":"2023-09-13T03:11:23.264403+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:05:49.824053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.266834+0000","last_change":"2023-09-13T03:11:09.956947+0000","last_active":"2023-09-13T03:11:23.266834+0000","last_peered":"2023-09-13T03:11:23.266834+0000","last_clean":"2023-09-13T03:11:23.266834+0000","last_became_active":"2023-09-13T03:11:09.956639+0000","last_became_peered":"2023-09-13T03:11:09.956639+0000","last_unstale":"2023-09-13T03:11:23.266834+0000","last_undegraded":"2023-09-13T03:11:23.266834+0000","last_fullsized":"2023-09-13T03:11:23.266834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:21:37.759130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.265325+0000","last_change":"2023-09-13T03:11:10.947314+0000","last_active":"2023-09-13T03:11:23.265325+0000","last_peered":"2023-09-13T03:11:23.265325+0000","last_clean":"2023-09-13T03:11:23.265325+0000","last_became_active":"2023-09-13T03:11:10.946964+0000","last_became_peered":"2023-09-13T03:11:10.946964+0000","last_unstale":"2023-09-13T03:11:23.265325+0000","last_undegraded":"2023-09-13T03:11:23.265325+0000","last_fullsized":"2023-09-13T03:11:23.265325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:55:08.542303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.268605+0000","last_change":"2023-09-13T03:11:09.952540+0000","last_active":"2023-09-13T03:11:23.268605+0000","last_peered":"2023-09-13T03:11:23.268605+0000","last_clean":"2023-09-13T03:11:23.268605+0000","last_became_active":"2023-09-13T03:11:09.952208+0000","last_became_peered":"2023-09-13T03:11:09.952208+0000","last_unstale":"2023-09-13T03:11:23.268605+0000","last_undegraded":"2023-09-13T03:11:23.268605+0000","last_fullsized":"2023-09-13T03:11:23.268605+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:11:41.200401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.267200+0000","last_change":"2023-09-13T03:11:10.951305+0000","last_active":"2023-09-13T03:11:23.267200+0000","last_peered":"2023-09-13T03:11:23.267200+0000","last_clean":"2023-09-13T03:11:23.267200+0000","last_became_active":"2023-09-13T03:11:10.950982+0000","last_became_peered":"2023-09-13T03:11:10.950982+0000","last_unstale":"2023-09-13T03:11:23.267200+0000","last_undegraded":"2023-09-13T03:11:23.267200+0000","last_fullsized":"2023-09-13T03:11:23.267200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:55:04.497698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:11:24.355955+0000","last_change":"2023-09-13T03:11:24.355955+0000","last_active":"2023-09-13T03:11:24.355955+0000","last_peered":"2023-09-13T03:11:24.355955+0000","last_clean":"2023-09-13T03:11:24.355955+0000","last_became_active":"2023-09-13T03:11:24.351069+0000","last_became_peered":"2023-09-13T03:11:24.351069+0000","last_unstale":"2023-09-13T03:11:24.355955+0000","last_undegraded":"2023-09-13T03:11:24.355955+0000","last_fullsized":"2023-09-13T03:11:24.355955+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:57:55.311708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.262165+0000","last_change":"2023-09-13T03:11:10.959487+0000","last_active":"2023-09-13T03:11:23.262165+0000","last_peered":"2023-09-13T03:11:23.262165+0000","last_clean":"2023-09-13T03:11:23.262165+0000","last_became_active":"2023-09-13T03:11:10.959157+0000","last_became_peered":"2023-09-13T03:11:10.959157+0000","last_unstale":"2023-09-13T03:11:23.262165+0000","last_undegraded":"2023-09-13T03:11:23.262165+0000","last_fullsized":"2023-09-13T03:11:23.262165+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:37:50.852197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'5","reported_seq":68,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:02.310736+0000","last_change":"2023-09-13T03:11:09.951208+0000","last_active":"2023-09-13T03:12:02.310736+0000","last_peered":"2023-09-13T03:12:02.310736+0000","last_clean":"2023-09-13T03:12:02.310736+0000","last_became_active":"2023-09-13T03:11:09.950869+0000","last_became_peered":"2023-09-13T03:11:09.950869+0000","last_unstale":"2023-09-13T03:12:02.310736+0000","last_undegraded":"2023-09-13T03:12:02.310736+0000","last_fullsized":"2023-09-13T03:12:02.310736+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:22:53.697803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":48,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.262465+0000","last_change":"2023-09-13T03:11:10.959129+0000","last_active":"2023-09-13T03:11:23.262465+0000","last_peered":"2023-09-13T03:11:23.262465+0000","last_clean":"2023-09-13T03:11:23.262465+0000","last_became_active":"2023-09-13T03:11:10.958820+0000","last_became_peered":"2023-09-13T03:11:10.958820+0000","last_unstale":"2023-09-13T03:11:23.262465+0000","last_undegraded":"2023-09-13T03:11:23.262465+0000","last_fullsized":"2023-09-13T03:11:23.262465+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:20:30.607854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.267998+0000","last_change":"2023-09-13T03:11:09.946855+0000","last_active":"2023-09-13T03:11:23.267998+0000","last_peered":"2023-09-13T03:11:23.267998+0000","last_clean":"2023-09-13T03:11:23.267998+0000","last_became_active":"2023-09-13T03:11:09.946156+0000","last_became_peered":"2023-09-13T03:11:09.946156+0000","last_unstale":"2023-09-13T03:11:23.267998+0000","last_undegraded":"2023-09-13T03:11:23.267998+0000","last_fullsized":"2023-09-13T03:11:23.267998+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:42:19.387280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.262313+0000","last_change":"2023-09-13T03:11:10.951335+0000","last_active":"2023-09-13T03:11:23.262313+0000","last_peered":"2023-09-13T03:11:23.262313+0000","last_clean":"2023-09-13T03:11:23.262313+0000","last_became_active":"2023-09-13T03:11:10.951034+0000","last_became_peered":"2023-09-13T03:11:10.951034+0000","last_unstale":"2023-09-13T03:11:23.262313+0000","last_undegraded":"2023-09-13T03:11:23.262313+0000","last_fullsized":"2023-09-13T03:11:23.262313+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:57:24.373196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.263077+0000","last_change":"2023-09-13T03:11:09.950937+0000","last_active":"2023-09-13T03:11:23.263077+0000","last_peered":"2023-09-13T03:11:23.263077+0000","last_clean":"2023-09-13T03:11:23.263077+0000","last_became_active":"2023-09-13T03:11:09.950574+0000","last_became_peered":"2023-09-13T03:11:09.950574+0000","last_unstale":"2023-09-13T03:11:23.263077+0000","last_undegraded":"2023-09-13T03:11:23.263077+0000","last_fullsized":"2023-09-13T03:11:23.263077+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:10:23.215080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.262509+0000","last_change":"2023-09-13T03:11:10.950517+0000","last_active":"2023-09-13T03:11:23.262509+0000","last_peered":"2023-09-13T03:11:23.262509+0000","last_clean":"2023-09-13T03:11:23.262509+0000","last_became_active":"2023-09-13T03:11:10.950028+0000","last_became_peered":"2023-09-13T03:11:10.950028+0000","last_unstale":"2023-09-13T03:11:23.262509+0000","last_undegraded":"2023-09-13T03:11:23.262509+0000","last_fullsized":"2023-09-13T03:11:23.262509+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:06:40.008527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.265417+0000","last_change":"2023-09-13T03:11:10.942192+0000","last_active":"2023-09-13T03:11:23.265417+0000","last_peered":"2023-09-13T03:11:23.265417+0000","last_clean":"2023-09-13T03:11:23.265417+0000","last_became_active":"2023-09-13T03:11:10.941893+0000","last_became_peered":"2023-09-13T03:11:10.941893+0000","last_unstale":"2023-09-13T03:11:23.265417+0000","last_undegraded":"2023-09-13T03:11:23.265417+0000","last_fullsized":"2023-09-13T03:11:23.265417+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:26:22.064843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.268237+0000","last_change":"2023-09-13T03:11:10.947186+0000","last_active":"2023-09-13T03:11:23.268237+0000","last_peered":"2023-09-13T03:11:23.268237+0000","last_clean":"2023-09-13T03:11:23.268237+0000","last_became_active":"2023-09-13T03:11:10.946902+0000","last_became_peered":"2023-09-13T03:11:10.946902+0000","last_unstale":"2023-09-13T03:11:23.268237+0000","last_undegraded":"2023-09-13T03:11:23.268237+0000","last_fullsized":"2023-09-13T03:11:23.268237+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:26:01.325597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.268602+0000","last_change":"2023-09-13T03:11:10.946504+0000","last_active":"2023-09-13T03:11:23.268602+0000","last_peered":"2023-09-13T03:11:23.268602+0000","last_clean":"2023-09-13T03:11:23.268602+0000","last_became_active":"2023-09-13T03:11:10.946196+0000","last_became_peered":"2023-09-13T03:11:10.946196+0000","last_unstale":"2023-09-13T03:11:23.268602+0000","last_undegraded":"2023-09-13T03:11:23.268602+0000","last_fullsized":"2023-09-13T03:11:23.268602+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:37:43.578685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.266297+0000","last_change":"2023-09-13T03:11:10.959637+0000","last_active":"2023-09-13T03:11:23.266297+0000","last_peered":"2023-09-13T03:11:23.266297+0000","last_clean":"2023-09-13T03:11:23.266297+0000","last_became_active":"2023-09-13T03:11:10.959324+0000","last_became_peered":"2023-09-13T03:11:10.959324+0000","last_unstale":"2023-09-13T03:11:23.266297+0000","last_undegraded":"2023-09-13T03:11:23.266297+0000","last_fullsized":"2023-09-13T03:11:23.266297+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:48:19.245785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:11:24.576318+0000","last_change":"2023-09-13T03:11:24.576318+0000","last_active":"2023-09-13T03:11:24.576318+0000","last_peered":"2023-09-13T03:11:24.576318+0000","last_clean":"2023-09-13T03:11:24.576318+0000","last_became_active":"2023-09-13T03:11:24.575775+0000","last_became_peered":"2023-09-13T03:11:24.575775+0000","last_unstale":"2023-09-13T03:11:24.576318+0000","last_undegraded":"2023-09-13T03:11:24.576318+0000","last_fullsized":"2023-09-13T03:11:24.576318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:16:00.534554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.265575+0000","last_change":"2023-09-13T03:11:10.945076+0000","last_active":"2023-09-13T03:11:23.265575+0000","last_peered":"2023-09-13T03:11:23.265575+0000","last_clean":"2023-09-13T03:11:23.265575+0000","last_became_active":"2023-09-13T03:11:10.944776+0000","last_became_peered":"2023-09-13T03:11:10.944776+0000","last_unstale":"2023-09-13T03:11:23.265575+0000","last_undegraded":"2023-09-13T03:11:23.265575+0000","last_fullsized":"2023-09-13T03:11:23.265575+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:22:03.624488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.267991+0000","last_change":"2023-09-13T03:11:10.942663+0000","last_active":"2023-09-13T03:11:23.267991+0000","last_peered":"2023-09-13T03:11:23.267991+0000","last_clean":"2023-09-13T03:11:23.267991+0000","last_became_active":"2023-09-13T03:11:10.942203+0000","last_became_peered":"2023-09-13T03:11:10.942203+0000","last_unstale":"2023-09-13T03:11:23.267991+0000","last_undegraded":"2023-09-13T03:11:23.267991+0000","last_fullsized":"2023-09-13T03:11:23.267991+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:46:48.091425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.265584+0000","last_change":"2023-09-13T03:11:10.940089+0000","last_active":"2023-09-13T03:11:23.265584+0000","last_peered":"2023-09-13T03:11:23.265584+0000","last_clean":"2023-09-13T03:11:23.265584+0000","last_became_active":"2023-09-13T03:11:10.939622+0000","last_became_peered":"2023-09-13T03:11:10.939622+0000","last_unstale":"2023-09-13T03:11:23.265584+0000","last_undegraded":"2023-09-13T03:11:23.265584+0000","last_fullsized":"2023-09-13T03:11:23.265584+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:43:55.024944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.267205+0000","last_change":"2023-09-13T03:11:10.956896+0000","last_active":"2023-09-13T03:11:23.267205+0000","last_peered":"2023-09-13T03:11:23.267205+0000","last_clean":"2023-09-13T03:11:23.267205+0000","last_became_active":"2023-09-13T03:11:10.956624+0000","last_became_peered":"2023-09-13T03:11:10.956624+0000","last_unstale":"2023-09-13T03:11:23.267205+0000","last_undegraded":"2023-09-13T03:11:23.267205+0000","last_fullsized":"2023-09-13T03:11:23.267205+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:29:41.305999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.266747+0000","last_change":"2023-09-13T03:11:10.939240+0000","last_active":"2023-09-13T03:11:23.266747+0000","last_peered":"2023-09-13T03:11:23.266747+0000","last_clean":"2023-09-13T03:11:23.266747+0000","last_became_active":"2023-09-13T03:11:10.938869+0000","last_became_peered":"2023-09-13T03:11:10.938869+0000","last_unstale":"2023-09-13T03:11:23.266747+0000","last_undegraded":"2023-09-13T03:11:23.266747+0000","last_fullsized":"2023-09-13T03:11:23.266747+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:06:44.931165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.269653+0000","last_change":"2023-09-13T03:11:10.956704+0000","last_active":"2023-09-13T03:11:23.269653+0000","last_peered":"2023-09-13T03:11:23.269653+0000","last_clean":"2023-09-13T03:11:23.269653+0000","last_became_active":"2023-09-13T03:11:10.956413+0000","last_became_peered":"2023-09-13T03:11:10.956413+0000","last_unstale":"2023-09-13T03:11:23.269653+0000","last_undegraded":"2023-09-13T03:11:23.269653+0000","last_fullsized":"2023-09-13T03:11:23.269653+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:50:04.626111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.261884+0000","last_change":"2023-09-13T03:11:10.952707+0000","last_active":"2023-09-13T03:11:23.261884+0000","last_peered":"2023-09-13T03:11:23.261884+0000","last_clean":"2023-09-13T03:11:23.261884+0000","last_became_active":"2023-09-13T03:11:10.952423+0000","last_became_peered":"2023-09-13T03:11:10.952423+0000","last_unstale":"2023-09-13T03:11:23.261884+0000","last_undegraded":"2023-09-13T03:11:23.261884+0000","last_fullsized":"2023-09-13T03:11:23.261884+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:35:09.869172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.265372+0000","last_change":"2023-09-13T03:11:10.941096+0000","last_active":"2023-09-13T03:11:23.265372+0000","last_peered":"2023-09-13T03:11:23.265372+0000","last_clean":"2023-09-13T03:11:23.265372+0000","last_became_active":"2023-09-13T03:11:10.940761+0000","last_became_peered":"2023-09-13T03:11:10.940761+0000","last_unstale":"2023-09-13T03:11:23.265372+0000","last_undegraded":"2023-09-13T03:11:23.265372+0000","last_fullsized":"2023-09-13T03:11:23.265372+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:34:27.389650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65046,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":485,"num_read_kb":311,"num_write":233,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":68,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":195138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":203,"ondisk_log_size":203,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449111,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":444288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:11:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.56999999999999995}]},{"osd":1,"last update":"Wed Sep 13 03:11:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.504}]},{"osd":2,"last update":"Wed Sep 13 03:11:26 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.441}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":4,"last update":"Wed Sep 13 03:11:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.64700000000000002}]},{"osd":5,"last update":"Wed Sep 13 03:11:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.70699999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":7,"last update":"Wed Sep 13 03:11:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.74199999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":9,"last update":"Wed Sep 13 03:11:26 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.83299999999999996}]},{"osd":10,"last update":"Wed Sep 13 03:11:26 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.72799999999999998}]}]},{"osd":10,"up_from":71,"seq":304942678042,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1093489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":1,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.496}]},{"osd":2,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.52300000000000002}]},{"osd":3,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.746}]},{"osd":4,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.54900000000000004}]},{"osd":5,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.57299999999999995}]},{"osd":6,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":0.86099999999999999}]},{"osd":7,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.47599999999999998}]},{"osd":8,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.77100000000000002}]},{"osd":9,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.94799999999999995}]},{"osd":11,"last update":"Wed Sep 13 03:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.80500000000000005}]}]},{"osd":9,"up_from":66,"seq":283467841567,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":950272,"data_stored":444150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.60199999999999998,"15min":0.61399999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.58699999999999997,"15min":0.59499999999999997},"min":{"1min":0.34999999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.60399999999999998}]},{"osd":1,"last update":"Wed Sep 13 03:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.65800000000000003,"15min":0.66300000000000003},"min":{"1min":0.441,"5min":0.379,"15min":0.379},"max":{"1min":0.89000000000000001,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.73999999999999999,"15min":0.75800000000000001},"min":{"1min":0.48399999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.80000000000000004,"5min":1.107,"15min":1.107},"last":0.68200000000000005}]},{"osd":2,"last update":"Wed Sep 13 03:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.72599999999999998,"15min":0.73599999999999999},"min":{"1min":0.41999999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.96299999999999997,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.76900000000000002,"15min":0.78700000000000003},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.97399999999999998,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.84199999999999997}]},{"osd":3,"last update":"Wed Sep 13 03:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.70199999999999996,"15min":0.71899999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.91000000000000003,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.75,"15min":0.77600000000000002},"min":{"1min":0.39600000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.002,"5min":1.238,"15min":1.238},"last":0.63200000000000001}]},{"osd":4,"last update":"Wed Sep 13 03:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.82999999999999996,"15min":0.85299999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.86499999999999999,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.79800000000000004,"15min":0.81399999999999995},"min":{"1min":0.56999999999999995,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.81100000000000005,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.876}]},{"osd":5,"last update":"Wed Sep 13 03:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.84199999999999997,"15min":0.86499999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0289999999999999,"5min":1.486,"15min":1.486},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.746,"5min":1.0109999999999999,"15min":1.0549999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.0860000000000001,"5min":4.2590000000000003,"15min":4.2590000000000003},"last":0.78300000000000003}]},{"osd":6,"last update":"Wed Sep 13 03:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.67600000000000005}]},{"osd":7,"last update":"Wed Sep 13 03:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":1.056,"15min":1.1000000000000001},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.0660000000000001,"5min":4.367,"15min":4.367},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.84699999999999998,"5min":1.083,"15min":1.1220000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.175,"5min":4.3520000000000003,"15min":4.3520000000000003},"last":0.75}]},{"osd":8,"last update":"Wed Sep 13 03:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.97199999999999998,"15min":0.998},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.113,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.85599999999999998,"5min":1.103,"15min":1.1439999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1000000000000001,"5min":4.298,"15min":4.298},"last":0.95799999999999996}]},{"osd":10,"last update":"Wed Sep 13 03:11:05 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]}]},{"osd":8,"up_from":58,"seq":249108103204,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":444197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40300000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.71999999999999997,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58399999999999996,"15min":0.58199999999999996},"min":{"1min":0.435,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.89900000000000002,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.63300000000000001}]},{"osd":1,"last update":"Wed Sep 13 03:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.45300000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58699999999999997,"15min":0.58799999999999997},"min":{"1min":0.45500000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.71899999999999997,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.67800000000000005}]},{"osd":2,"last update":"Wed Sep 13 03:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.46300000000000002,"15min":0.46500000000000002},"min":{"1min":0.27900000000000003,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.745,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.47399999999999998,"15min":0.47799999999999998},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.65400000000000003,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.40100000000000002}]},{"osd":3,"last update":"Wed Sep 13 03:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.65300000000000002,"15min":0.66000000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.82299999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.66000000000000003}]},{"osd":4,"last update":"Wed Sep 13 03:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60899999999999999,"15min":0.60699999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.77200000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.64300000000000002,"15min":0.64700000000000002},"min":{"1min":0.40100000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.879,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.71199999999999997}]},{"osd":5,"last update":"Wed Sep 13 03:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.55100000000000005,"15min":0.55800000000000005},"min":{"1min":0.28999999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.83199999999999996,"5min":0.995,"15min":0.995},"last":0.435},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.60599999999999998,"15min":0.61399999999999999},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.83999999999999997,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.59399999999999997}]},{"osd":6,"last update":"Wed Sep 13 03:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.73599999999999999,"15min":0.747},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.85999999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.621},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.75700000000000001,"15min":0.76300000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.86899999999999999,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.68999999999999995}]},{"osd":7,"last update":"Wed Sep 13 03:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.69499999999999995,"15min":0.70099999999999996},"min":{"1min":0.46700000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.84099999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71599999999999997,"15min":0.71499999999999997},"min":{"1min":0.53500000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.89000000000000001,"5min":1.054,"15min":1.054},"last":0.77400000000000002}]},{"osd":9,"last update":"Wed Sep 13 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.78800000000000003,"15min":0.78100000000000003},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.0329999999999999,"5min":1.21,"15min":1.21},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.77100000000000002,"15min":0.76300000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0289999999999999,"5min":1.081,"15min":1.081},"last":0.75900000000000001}]},{"osd":10,"last update":"Wed Sep 13 03:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.73999999999999999,"15min":0.71399999999999997},"min":{"1min":0.55700000000000005,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.72799999999999998,"15min":0.69899999999999995},"min":{"1min":0.54400000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":2.004,"5min":2.004,"15min":2.004},"last":0.82999999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]}]},{"osd":7,"up_from":51,"seq":219043332136,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28528,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716624,"statfs":{"total":95995035648,"available":95965822976,"internally_reserved":0,"allocated":1540096,"data_stored":1037168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:10:55 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.59699999999999998,"15min":0.58599999999999997},"min":{"1min":0.498,"5min":0.438,"15min":0.438},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.378},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60099999999999998,"15min":0.59099999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.41099999999999998}]},{"osd":1,"last update":"Wed Sep 13 03:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.42999999999999999,"15min":0.40799999999999997},"min":{"1min":0.376,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.625,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.44700000000000001,"15min":0.42099999999999999},"min":{"1min":0.39100000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.39300000000000002}]},{"osd":2,"last update":"Wed Sep 13 03:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.60799999999999998,"15min":0.59399999999999997},"min":{"1min":0.437,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86899999999999999,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.59899999999999998,"15min":0.58099999999999996},"min":{"1min":0.496,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.54100000000000004}]},{"osd":3,"last update":"Wed Sep 13 03:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.68799999999999994,"15min":0.63100000000000001},"min":{"1min":0.46899999999999997,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.81899999999999995,"5min":6.2720000000000002,"15min":6.2720000000000002},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.64700000000000002,"15min":0.625},"min":{"1min":0.48299999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88900000000000001,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.61399999999999999}]},{"osd":4,"last update":"Wed Sep 13 03:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.53900000000000003,"15min":0.51700000000000002},"min":{"1min":0.46800000000000003,"5min":0.314,"15min":0.314},"max":{"1min":0.82399999999999995,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.505},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55500000000000005,"15min":0.52700000000000002},"min":{"1min":0.39900000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.88300000000000001,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.44900000000000001}]},{"osd":5,"last update":"Wed Sep 13 03:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.66200000000000003,"15min":0.64300000000000002},"min":{"1min":0.52500000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.65300000000000002,"15min":0.625},"min":{"1min":0.56699999999999995,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.95599999999999996,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.66600000000000004}]},{"osd":6,"last update":"Wed Sep 13 03:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.72999999999999998,"15min":0.70199999999999996},"min":{"1min":0.57499999999999996,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.98999999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.75800000000000001,"15min":0.72799999999999998},"min":{"1min":0.622,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0129999999999999,"5min":1.212,"15min":1.212},"last":0.74199999999999999}]},{"osd":8,"last update":"Wed Sep 13 03:11:20 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.66400000000000003,"15min":0.65000000000000002},"min":{"1min":0.53900000000000003,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.70599999999999996,"15min":0.69199999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.58499999999999996}]},{"osd":9,"last update":"Wed Sep 13 03:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.81899999999999995,"15min":0.81200000000000006},"min":{"1min":0.68799999999999994,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.04,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.81200000000000006,"15min":0.80800000000000005},"min":{"1min":0.66300000000000003,"5min":0.628,"15min":0.628},"max":{"1min":1.069,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.70599999999999996}]},{"osd":10,"last update":"Wed Sep 13 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.75700000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]}]},{"osd":6,"up_from":46,"seq":197568495661,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":443873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:11:38 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.496,"15min":0.496},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.66200000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.434},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.52500000000000002,"15min":0.52700000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.66500000000000004,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.38900000000000001}]},{"osd":1,"last update":"Wed Sep 13 03:11:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.64000000000000001,"15min":0.63},"min":{"1min":0.43099999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.89700000000000002,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.627,"15min":0.61599999999999999},"min":{"1min":0.45400000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.878,"5min":1.069,"15min":1.069},"last":0.60699999999999998}]},{"osd":2,"last update":"Wed Sep 13 03:11:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.68000000000000005,"15min":0.67100000000000004},"min":{"1min":0.54900000000000004,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.82299999999999995,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.51000000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88100000000000001,"5min":1.149,"15min":1.149},"last":0.67300000000000004}]},{"osd":3,"last update":"Wed Sep 13 03:11:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.628,"15min":0.63900000000000001},"min":{"1min":0.38500000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.71999999999999997,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.65000000000000002,"15min":0.65200000000000002},"min":{"1min":0.47799999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93200000000000005,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.49199999999999999}]},{"osd":4,"last update":"Wed Sep 13 03:11:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.73099999999999998,"15min":0.72899999999999998},"min":{"1min":0.49299999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.95699999999999996,"5min":1.161,"15min":1.161},"last":0.745},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.754,"15min":0.76200000000000001},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.01,"5min":1.54,"15min":1.54},"last":0.70699999999999996}]},{"osd":5,"last update":"Wed Sep 13 03:11:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.78600000000000003,"15min":0.79900000000000004},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0169999999999999,"5min":1.115,"15min":1.115},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.77700000000000002,"15min":0.77900000000000003},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.88700000000000001,"5min":1.21,"15min":1.21},"last":0.71799999999999997}]},{"osd":7,"last update":"Wed Sep 13 03:12:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77500000000000002,"15min":0.77000000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.1899999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.78600000000000003,"15min":0.75800000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":0.83399999999999996}]},{"osd":8,"last update":"Wed Sep 13 03:11:20 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.877,"15min":0.88700000000000001},"min":{"1min":0.73299999999999998,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.0720000000000001,"5min":1.131,"15min":1.131},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.83199999999999996,"15min":0.83999999999999997},"min":{"1min":0.64000000000000001,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":0.95299999999999996,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.80900000000000005}]},{"osd":9,"last update":"Wed Sep 13 03:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.871,"15min":0.88600000000000001},"min":{"1min":0.63900000000000001,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":0.97099999999999997,"5min":1.319,"15min":1.319},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.92200000000000004,"15min":0.93799999999999994},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0329999999999999,"5min":1.24,"15min":1.24},"last":0.878}]},{"osd":10,"last update":"Wed Sep 13 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.91000000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":502351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.57099999999999995,"15min":0.53900000000000003},"min":{"1min":0.38700000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.88900000000000001,"5min":2.9260000000000002,"15min":2.9260000000000002},"last":0.44},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.54700000000000004,"15min":0.53300000000000003},"min":{"1min":0.34100000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.503}]},{"osd":2,"last update":"Wed Sep 13 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.54400000000000004,"15min":0.52600000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.54900000000000004,"15min":0.52900000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.68700000000000006}]},{"osd":3,"last update":"Wed Sep 13 03:11:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59999999999999998,"15min":0.58599999999999997},"min":{"1min":0.36799999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84499999999999997,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.629,"15min":0.60499999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.66800000000000004}]},{"osd":4,"last update":"Wed Sep 13 03:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.44700000000000001,"15min":0.39700000000000002},"min":{"1min":0.36699999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.69799999999999995,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.48499999999999999,"15min":0.44},"min":{"1min":0.35499999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.41999999999999998}]},{"osd":5,"last update":"Wed Sep 13 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.57199999999999995,"15min":0.53400000000000003},"min":{"1min":0.44900000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.754},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.57799999999999996,"15min":0.54500000000000004},"min":{"1min":0.46500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.80000000000000004}]},{"osd":6,"last update":"Wed Sep 13 03:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.66900000000000004,"15min":0.65500000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.69899999999999995,"15min":0.67300000000000004},"min":{"1min":0.46800000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.90900000000000003,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":0.59899999999999998}]},{"osd":7,"last update":"Wed Sep 13 03:12:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.59899999999999998,"15min":0.57099999999999995},"min":{"1min":0.45100000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.61599999999999999,"15min":0.60399999999999998},"min":{"1min":0.45900000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84799999999999998,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.70099999999999996}]},{"osd":8,"last update":"Wed Sep 13 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.66700000000000004,"15min":0.66400000000000003},"min":{"1min":0.499,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.72499999999999998,"15min":0.71999999999999997},"min":{"1min":0.55200000000000005,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.90400000000000003}]},{"osd":9,"last update":"Wed Sep 13 03:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.77900000000000003,"15min":0.77500000000000002},"min":{"1min":0.57499999999999996,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.82699999999999996,"15min":0.82599999999999996},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.995,"5min":1.024,"15min":1.024},"last":0.88100000000000001}]},{"osd":10,"last update":"Wed Sep 13 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.84299999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92800000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":954368,"data_stored":445206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 13 03:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.61399999999999999,"15min":0.56000000000000005},"min":{"1min":0.38,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.97299999999999998,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.64700000000000002,"15min":0.57799999999999996},"min":{"1min":0.39500000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":6.0039999999999996,"5min":6.0039999999999996,"15min":6.0039999999999996},"last":0.55000000000000004}]},{"osd":2,"last update":"Wed Sep 13 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.64800000000000002,"15min":0.60799999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.621,"15min":0.59099999999999997},"min":{"1min":0.5,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.86599999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.69099999999999995}]},{"osd":3,"last update":"Wed Sep 13 03:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.46800000000000003,"15min":0.45800000000000002},"min":{"1min":0.309,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.437},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.56200000000000006,"15min":0.54700000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.47799999999999998}]},{"osd":4,"last update":"Wed Sep 13 03:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.74199999999999999,"15min":0.71499999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":5.9370000000000003,"5min":5.9370000000000003,"15min":5.9370000000000003},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69299999999999995,"15min":0.68300000000000005},"min":{"1min":0.45300000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0409999999999999,"5min":1.665,"15min":1.665},"last":0.82299999999999995}]},{"osd":5,"last update":"Wed Sep 13 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.70299999999999996,"15min":0.69299999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.90300000000000002,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68600000000000005,"15min":0.67400000000000004},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.88400000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.78800000000000003}]},{"osd":6,"last update":"Wed Sep 13 03:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.72999999999999998,"15min":0.68600000000000005},"min":{"1min":0.442,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":6.1130000000000004,"5min":6.1130000000000004,"15min":6.1130000000000004},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.75800000000000001,"15min":0.74399999999999999},"min":{"1min":0.48699999999999999,"5min":0.434,"15min":0.434},"max":{"1min":2.7229999999999999,"5min":2.7229999999999999,"15min":2.7229999999999999},"last":0.73399999999999999}]},{"osd":7,"last update":"Wed Sep 13 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.74299999999999999,"15min":0.73899999999999999},"min":{"1min":0.56299999999999994,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.626},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.80600000000000005,"15min":0.79400000000000004},"min":{"1min":0.74399999999999999,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1379999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.77900000000000003}]},{"osd":8,"last update":"Wed Sep 13 03:11:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.81000000000000005,"15min":0.81699999999999995},"min":{"1min":0.58299999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":0.97999999999999998,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.77300000000000002,"15min":0.77000000000000002},"min":{"1min":0.60899999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.80700000000000005}]},{"osd":9,"last update":"Wed Sep 13 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.88500000000000001,"15min":0.89500000000000002},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.88300000000000001,"15min":0.89300000000000002},"min":{"1min":0.59399999999999997,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.88100000000000001}]},{"osd":10,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.753},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.93700000000000006}]},{"osd":11,"last update":"Wed Sep 13 03:11:27 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.96999999999999997}]}]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":503166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.50800000000000001,"15min":0.47599999999999998},"min":{"1min":0.35399999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.68000000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.628},{"interface":"front","average":{"1min":0.502,"5min":0.50800000000000001,"15min":0.48299999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73099999999999998,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.53500000000000003}]},{"osd":1,"last update":"Wed Sep 13 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53100000000000003,"15min":0.51400000000000001},"min":{"1min":0.38400000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.69899999999999995,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.52900000000000003,"15min":0.52500000000000002},"min":{"1min":0.40400000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.60799999999999998}]},{"osd":3,"last update":"Wed Sep 13 03:11:37 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.626,"15min":0.56999999999999995},"min":{"1min":0.432,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":2.879,"5min":2.879,"15min":2.879},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.60999999999999999,"15min":0.56899999999999995},"min":{"1min":0.39100000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":2.8969999999999998,"5min":2.8969999999999998,"15min":2.8969999999999998},"last":0.58999999999999997}]},{"osd":4,"last update":"Wed Sep 13 03:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.56000000000000005,"15min":0.55200000000000005},"min":{"1min":0.375,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73999999999999999,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56899999999999995,"15min":0.55900000000000005},"min":{"1min":0.42599999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.752,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.73499999999999999}]},{"osd":5,"last update":"Wed Sep 13 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.442,"15min":0.42399999999999999},"min":{"1min":0.33300000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70499999999999996,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.46100000000000002,"15min":0.442},"min":{"1min":0.29699999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.46100000000000002}]},{"osd":6,"last update":"Wed Sep 13 03:11:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.64100000000000001,"15min":0.622},"min":{"1min":0.40899999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.65000000000000002,"15min":0.621},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.64600000000000002}]},{"osd":7,"last update":"Wed Sep 13 03:10:58 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.63700000000000001,"15min":0.63600000000000001},"min":{"1min":0.47499999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.84699999999999998,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.66100000000000003,"15min":0.65900000000000003},"min":{"1min":0.51700000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.83399999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.497}]},{"osd":8,"last update":"Wed Sep 13 03:11:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57599999999999996,"15min":0.57999999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.67800000000000005,"15min":0.68600000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.66300000000000003}]},{"osd":9,"last update":"Wed Sep 13 03:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.77700000000000002,"15min":0.75700000000000001},"min":{"1min":0.46600000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.6459999999999999,"5min":2.6459999999999999,"15min":2.6459999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.74099999999999999,"15min":0.73599999999999999},"min":{"1min":0.504,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.70299999999999996}]},{"osd":10,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.747},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.77700000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]}]},{"osd":3,"up_from":25,"seq":107374182459,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":443704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:11:30 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.42699999999999999,"15min":0.40300000000000002},"min":{"1min":0.33800000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.59199999999999997,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.46200000000000002,"15min":0.436},"min":{"1min":0.378,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.752,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.52700000000000002}]},{"osd":1,"last update":"Wed Sep 13 03:11:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.63100000000000001,"15min":0.59599999999999997},"min":{"1min":0.47299999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0229999999999999,"5min":1.117,"15min":1.117},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.64000000000000001,"15min":0.59999999999999998},"min":{"1min":0.57499999999999996,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.86599999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.65600000000000003}]},{"osd":2,"last update":"Wed Sep 13 03:11:30 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.67900000000000005,"15min":0.63800000000000001},"min":{"1min":0.58099999999999996,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.89500000000000002,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.68700000000000006,"15min":0.64700000000000002},"min":{"1min":0.55000000000000004,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.052,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.67900000000000005}]},{"osd":4,"last update":"Wed Sep 13 03:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.68300000000000005,"15min":0.63600000000000001},"min":{"1min":0.49399999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0900000000000001,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.67500000000000004,"15min":0.627},"min":{"1min":0.52000000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0389999999999999,"5min":1.177,"15min":1.177},"last":0.73099999999999998}]},{"osd":5,"last update":"Wed Sep 13 03:11:14 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.69499999999999995,"15min":0.68000000000000005},"min":{"1min":0.53500000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1,"5min":1.012,"15min":1.012},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.71999999999999997,"15min":0.70199999999999996},"min":{"1min":0.53500000000000003,"5min":0.498,"15min":0.498},"max":{"1min":0.95999999999999996,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.84099999999999997}]},{"osd":6,"last update":"Wed Sep 13 03:11:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.64000000000000001,"15min":0.623},"min":{"1min":0.42599999999999999,"5min":0.374,"15min":0.374},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.76800000000000002,"15min":0.749},"min":{"1min":0.53200000000000003,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.93700000000000006,"5min":1.599,"15min":1.599},"last":0.69999999999999996}]},{"osd":7,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.86199999999999999,"15min":0.85399999999999998},"min":{"1min":0.63200000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.4890000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.83999999999999997,"15min":0.82999999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.512,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.71499999999999997}]},{"osd":8,"last update":"Wed Sep 13 03:10:17 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.82199999999999995}]},{"osd":9,"last update":"Wed Sep 13 03:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.91500000000000004,"15min":0.92400000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.135,"5min":1.538,"15min":1.538},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.88800000000000001,"15min":0.90000000000000002},"min":{"1min":0.621,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.198,"5min":1.677,"15min":1.677},"last":0.751}]},{"osd":10,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.708,"5min":1.708,"15min":1.708},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":0.93600000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]}]},{"osd":4,"up_from":33,"seq":141733920822,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":444118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.54500000000000004,"15min":0.51500000000000001},"min":{"1min":0.36699999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.55900000000000005,"15min":0.52900000000000003},"min":{"1min":0.35599999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.63800000000000001}]},{"osd":1,"last update":"Wed Sep 13 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.39700000000000002,"15min":0.36199999999999999},"min":{"1min":0.308,"5min":0.245,"15min":0.245},"max":{"1min":0.66700000000000004,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.443,"5min":0.40799999999999997,"15min":0.372},"min":{"1min":0.32500000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.68200000000000005,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.50800000000000001}]},{"osd":2,"last update":"Wed Sep 13 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58099999999999996,"15min":0.56299999999999994},"min":{"1min":0.44,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.81999999999999995,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.58999999999999997,"15min":0.54000000000000004},"min":{"1min":0.38600000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.84999999999999998}]},{"osd":3,"last update":"Wed Sep 13 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63100000000000001,"15min":0.56299999999999994},"min":{"1min":0.46400000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83899999999999997,"5min":4.2919999999999998,"15min":4.2919999999999998},"last":0.752},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.59999999999999998,"15min":0.56999999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81999999999999995,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.66200000000000003}]},{"osd":5,"last update":"Wed Sep 13 03:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59499999999999997,"15min":0.57599999999999996},"min":{"1min":0.42799999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.85199999999999998,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.57999999999999996,"15min":0.56000000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.81299999999999994,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.72899999999999998}]},{"osd":6,"last update":"Wed Sep 13 03:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.753,"15min":0.71499999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":6.1859999999999999,"5min":6.1859999999999999,"15min":6.1859999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.71199999999999997,"15min":0.71399999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.85999999999999999,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.81200000000000006}]},{"osd":7,"last update":"Wed Sep 13 03:11:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.628,"15min":0.60999999999999999},"min":{"1min":0.34200000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.93300000000000005,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63600000000000001,"15min":0.622},"min":{"1min":0.39700000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89700000000000002,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.64600000000000002}]},{"osd":8,"last update":"Wed Sep 13 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71799999999999997,"15min":0.72099999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.95099999999999996}]},{"osd":9,"last update":"Wed Sep 13 03:11:44 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":0.90600000000000003,"15min":0.88},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":6.1529999999999996,"5min":6.1529999999999996,"15min":6.1529999999999996},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.75,"5min":0.81899999999999995,"15min":0.83099999999999996},"min":{"1min":0.58099999999999996,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.96999999999999997,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.83599999999999997}]},{"osd":10,"last update":"Wed Sep 13 03:11:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.90900000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]}]},{"osd":5,"up_from":38,"seq":163208757299,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1556480,"data_stored":1036729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.59699999999999998,"15min":0.61299999999999999},"min":{"1min":0.39000000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78000000000000003,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.65800000000000003,"15min":0.63600000000000001},"min":{"1min":0.375,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":6.0179999999999998,"5min":6.0179999999999998,"15min":6.0179999999999998},"last":0.47199999999999998}]},{"osd":1,"last update":"Wed Sep 13 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.54000000000000004,"15min":0.51800000000000002},"min":{"1min":0.372,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.53300000000000003,"15min":0.51000000000000001},"min":{"1min":0.45200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.84299999999999997}]},{"osd":2,"last update":"Wed Sep 13 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.40100000000000002,"15min":0.38400000000000001},"min":{"1min":0.29099999999999998,"5min":0.193,"15min":0.193},"max":{"1min":0.71299999999999997,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.434,"5min":0.40600000000000003,"15min":0.38700000000000001},"min":{"1min":0.27000000000000002,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.76800000000000002,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.436}]},{"osd":3,"last update":"Wed Sep 13 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.40400000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.69099999999999995,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60099999999999998,"15min":0.58599999999999997},"min":{"1min":0.38300000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.85299999999999998,"5min":1.48,"15min":1.48},"last":0.64900000000000002}]},{"osd":4,"last update":"Wed Sep 13 03:11:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.59899999999999998,"15min":0.55700000000000005},"min":{"1min":0.40899999999999997,"5min":0.376,"15min":0.376},"max":{"1min":3.3740000000000001,"5min":3.3740000000000001,"15min":3.3740000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56699999999999995,"15min":0.54900000000000004},"min":{"1min":0.39600000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79400000000000004,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.78800000000000003}]},{"osd":6,"last update":"Wed Sep 13 03:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.68400000000000005,"15min":0.64800000000000002},"min":{"1min":0.52800000000000002,"5min":0.371,"15min":0.371},"max":{"1min":4.8250000000000002,"5min":4.8250000000000002,"15min":4.8250000000000002},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.70899999999999996,"15min":0.66800000000000004},"min":{"1min":0.51000000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":4.8049999999999997,"5min":4.8049999999999997,"15min":4.8049999999999997},"last":0.63200000000000001}]},{"osd":7,"last update":"Wed Sep 13 03:12:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.63400000000000001,"15min":0.63300000000000001},"min":{"1min":0.47899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.86399999999999999,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.66600000000000004,"15min":0.65900000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85499999999999998,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.76200000000000001}]},{"osd":8,"last update":"Wed Sep 13 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.84299999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61599999999999999,"15min":0.61699999999999999},"min":{"1min":0.38800000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.91100000000000003,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.745}]},{"osd":9,"last update":"Wed Sep 13 03:11:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.0640000000000001,"15min":1.0429999999999999},"min":{"1min":0.55900000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":8.3070000000000004,"5min":8.3070000000000004,"15min":8.3070000000000004},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.78500000000000003,"15min":0.76700000000000002},"min":{"1min":0.60299999999999998,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":3.016,"5min":3.016,"15min":3.016},"last":0.73099999999999998}]},{"osd":10,"last update":"Wed Sep 13 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.91800000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":59596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":60138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":59970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-13T03:12:08.228 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-09-13T03:12:08.229 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-09-13T03:12:08.229 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-09-13T03:12:08.229 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph health --format=json 2023-09-13T03:12:08.968 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:12:09.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:09 smithi170 ceph-mon[134328]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T03:12:09.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:09 smithi170 ceph-mon[134328]: pgmap v308: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-13T03:12:09.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:09 smithi064 ceph-mon[97675]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T03:12:09.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:09 smithi064 ceph-mon[97675]: pgmap v308: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-13T03:12:09.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:09 smithi142 ceph-mon[134082]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T03:12:09.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:09 smithi142 ceph-mon[134082]: pgmap v308: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-13T03:12:10.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.348+0000 7fcb137ff700 1 -- 172.21.15.64:0/3333658217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb0c100620 msgr2=0x7fcb0c100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:10.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.348+0000 7fcb137ff700 1 --2- 172.21.15.64:0/3333658217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb0c100620 0x7fcb0c100aa0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcafc009990 tx=0x7fcafc009ca0 comp rx=0 tx=0).stop 2023-09-13T03:12:10.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.348+0000 7fcb137ff700 1 -- 172.21.15.64:0/3333658217 shutdown_connections 2023-09-13T03:12:10.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.348+0000 7fcb137ff700 1 --2- 172.21.15.64:0/3333658217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb0c100fe0 0x7fcb0c10d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:10.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.348+0000 7fcb137ff700 1 --2- 172.21.15.64:0/3333658217 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb0c106620 0x7fcb0c106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:10.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.348+0000 7fcb137ff700 1 --2- 172.21.15.64:0/3333658217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb0c100620 0x7fcb0c100aa0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:10.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.348+0000 7fcb137ff700 1 -- 172.21.15.64:0/3333658217 >> 172.21.15.64:0/3333658217 conn(0x7fcb0c0fc0f0 msgr2=0x7fcb0c0fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:10.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.348+0000 7fcb137ff700 1 -- 172.21.15.64:0/3333658217 shutdown_connections 2023-09-13T03:12:10.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.348+0000 7fcb137ff700 1 -- 172.21.15.64:0/3333658217 wait complete. 2023-09-13T03:12:10.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.349+0000 7fcb137ff700 1 Processor -- start 2023-09-13T03:12:10.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.349+0000 7fcb137ff700 1 -- start start 2023-09-13T03:12:10.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.350+0000 7fcb137ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb0c100620 0x7fcb0c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:10.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.350+0000 7fcb137ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb0c100fe0 0x7fcb0c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:10.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.350+0000 7fcb137ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb0c106620 0x7fcb0c0733c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:10.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.350+0000 7fcb137ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c1a4be0 con 0x7fcb0c100620 2023-09-13T03:12:10.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.350+0000 7fcb137ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c1a4d50 con 0x7fcb0c100fe0 2023-09-13T03:12:10.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.350+0000 7fcb137ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c1a4f10 con 0x7fcb0c106620 2023-09-13T03:12:10.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.350+0000 7fcb1159b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb0c100620 0x7fcb0c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:10.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.350+0000 7fcb10d9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb0c100fe0 0x7fcb0c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:10.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.350+0000 7fcb10d9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb0c100fe0 0x7fcb0c072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:59782/0 (socket says 172.21.15.64:59782) 2023-09-13T03:12:10.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.350+0000 7fcb10d9a700 1 -- 172.21.15.64:0/3887643684 learned_addr learned my addr 172.21.15.64:0/3887643684 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:10.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.350+0000 7fcb11d9c700 1 --2- 172.21.15.64:0/3887643684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb0c106620 0x7fcb0c0733c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:10.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.350+0000 7fcb10d9a700 1 -- 172.21.15.64:0/3887643684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb0c106620 msgr2=0x7fcb0c0733c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:10.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.350+0000 7fcb10d9a700 1 --2- 172.21.15.64:0/3887643684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb0c106620 0x7fcb0c0733c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:10.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.351+0000 7fcb10d9a700 1 -- 172.21.15.64:0/3887643684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb0c100620 msgr2=0x7fcb0c072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:10.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.351+0000 7fcb10d9a700 1 --2- 172.21.15.64:0/3887643684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb0c100620 0x7fcb0c072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:10.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.351+0000 7fcb10d9a700 1 -- 172.21.15.64:0/3887643684 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcafc0095f0 con 0x7fcb0c100fe0 2023-09-13T03:12:10.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.351+0000 7fcb1159b700 1 --2- 172.21.15.64:0/3887643684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb0c100620 0x7fcb0c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:10.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.351+0000 7fcb10d9a700 1 --2- 172.21.15.64:0/3887643684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb0c100fe0 0x7fcb0c072e40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcafc005910 tx=0x7fcafc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:10.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.351+0000 7fcb027fc700 1 -- 172.21.15.64:0/3887643684 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc01d070 con 0x7fcb0c100fe0 2023-09-13T03:12:10.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.351+0000 7fcb137ff700 1 -- 172.21.15.64:0/3887643684 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb0c1a5190 con 0x7fcb0c100fe0 2023-09-13T03:12:10.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.351+0000 7fcb137ff700 1 -- 172.21.15.64:0/3887643684 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb0c1a5600 con 0x7fcb0c100fe0 2023-09-13T03:12:10.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.351+0000 7fcb027fc700 1 -- 172.21.15.64:0/3887643684 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcafc022470 con 0x7fcb0c100fe0 2023-09-13T03:12:10.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.351+0000 7fcb027fc700 1 -- 172.21.15.64:0/3887643684 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc022e40 con 0x7fcb0c100fe0 2023-09-13T03:12:10.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.352+0000 7fcb137ff700 1 -- 172.21.15.64:0/3887643684 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaf0005320 con 0x7fcb0c100fe0 2023-09-13T03:12:10.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.355+0000 7fcb027fc700 1 -- 172.21.15.64:0/3887643684 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fcafc0225e0 con 0x7fcb0c100fe0 2023-09-13T03:12:10.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.356+0000 7fcb027fc700 1 --2- 172.21.15.64:0/3887643684 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fcaf80a06d0 0x7fcaf80a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:10.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.356+0000 7fcb027fc700 1 -- 172.21.15.64:0/3887643684 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fcafc0cd190 con 0x7fcb0c100fe0 2023-09-13T03:12:10.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.356+0000 7fcb1159b700 1 --2- 172.21.15.64:0/3887643684 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fcaf80a06d0 0x7fcaf80a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:10.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.357+0000 7fcb1159b700 1 --2- 172.21.15.64:0/3887643684 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fcaf80a06d0 0x7fcaf80a2b90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009fd0 tx=0x7fcb08009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:10.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.358+0000 7fcb027fc700 1 -- 172.21.15.64:0/3887643684 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fcafc096bc0 con 0x7fcb0c100fe0 2023-09-13T03:12:10.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.613+0000 7fcb137ff700 1 -- 172.21.15.64:0/3887643684 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fcaf0005190 con 0x7fcb0c100fe0 2023-09-13T03:12:10.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.613+0000 7fcb027fc700 1 -- 172.21.15.64:0/3887643684 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fcafc09a1e0 con 0x7fcb0c100fe0 2023-09-13T03:12:10.615 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:12:10.615 INFO:teuthology.orchestra.run.smithi064.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-09-13T03:12:10.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.616+0000 7fcb137ff700 1 -- 172.21.15.64:0/3887643684 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fcaf80a06d0 msgr2=0x7fcaf80a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:10.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.616+0000 7fcb137ff700 1 --2- 172.21.15.64:0/3887643684 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fcaf80a06d0 0x7fcaf80a2b90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009fd0 tx=0x7fcb08009260 comp rx=0 tx=0).stop 2023-09-13T03:12:10.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.616+0000 7fcb137ff700 1 -- 172.21.15.64:0/3887643684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb0c100fe0 msgr2=0x7fcb0c072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:10.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.616+0000 7fcb137ff700 1 --2- 172.21.15.64:0/3887643684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb0c100fe0 0x7fcb0c072e40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcafc005910 tx=0x7fcafc004730 comp rx=0 tx=0).stop 2023-09-13T03:12:10.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.616+0000 7fcb137ff700 1 -- 172.21.15.64:0/3887643684 shutdown_connections 2023-09-13T03:12:10.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.616+0000 7fcb137ff700 1 --2- 172.21.15.64:0/3887643684 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fcaf80a06d0 0x7fcaf80a2b90 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:10.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.616+0000 7fcb137ff700 1 --2- 172.21.15.64:0/3887643684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb0c106620 0x7fcb0c0733c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:10.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.616+0000 7fcb137ff700 1 --2- 172.21.15.64:0/3887643684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb0c100620 0x7fcb0c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:10.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.616+0000 7fcb137ff700 1 --2- 172.21.15.64:0/3887643684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb0c100fe0 0x7fcb0c072e40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:10.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.616+0000 7fcb137ff700 1 -- 172.21.15.64:0/3887643684 >> 172.21.15.64:0/3887643684 conn(0x7fcb0c0fc0f0 msgr2=0x7fcb0c0fc730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:10.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.616+0000 7fcb137ff700 1 -- 172.21.15.64:0/3887643684 shutdown_connections 2023-09-13T03:12:10.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:10.616+0000 7fcb137ff700 1 -- 172.21.15.64:0/3887643684 wait complete. 2023-09-13T03:12:11.143 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-09-13T03:12:11.143 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-13T03:12:11.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.351+0000 7f63739ff700 1 -- 172.21.15.64:0/1935598070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f636c1021a0 msgr2=0x7f636c108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:11.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.351+0000 7f63739ff700 1 --2- 172.21.15.64:0/1935598070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f636c1021a0 0x7f636c108a50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f636400b0d0 tx=0x7f636400b3e0 comp rx=0 tx=0).stop 2023-09-13T03:12:11.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.351+0000 7f63739ff700 1 -- 172.21.15.64:0/1935598070 shutdown_connections 2023-09-13T03:12:11.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.351+0000 7f63739ff700 1 --2- 172.21.15.64:0/1935598070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f636c1021a0 0x7f636c108a50 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:11.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.351+0000 7f63739ff700 1 --2- 172.21.15.64:0/1935598070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f636c100610 0x7f636c100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:11.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.351+0000 7f63739ff700 1 --2- 172.21.15.64:0/1935598070 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f636c101800 0x7f636c101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:11.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.351+0000 7f63739ff700 1 -- 172.21.15.64:0/1935598070 >> 172.21.15.64:0/1935598070 conn(0x7f636c0fbb40 msgr2=0x7f636c0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:11.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.351+0000 7f63739ff700 1 -- 172.21.15.64:0/1935598070 shutdown_connections 2023-09-13T03:12:11.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.351+0000 7f63739ff700 1 -- 172.21.15.64:0/1935598070 wait complete. 2023-09-13T03:12:11.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.353+0000 7f63739ff700 1 Processor -- start 2023-09-13T03:12:11.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.353+0000 7f63739ff700 1 -- start start 2023-09-13T03:12:11.356 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.354+0000 7f63739ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f636c100610 0x7f636c0743b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:11.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.354+0000 7f63739ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f636c101800 0x7f636c0729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:11.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.354+0000 7f63739ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f636c1021a0 0x7f636c072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:11.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.354+0000 7f63739ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636c0736d0 con 0x7f636c1021a0 2023-09-13T03:12:11.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.354+0000 7f63739ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636c073840 con 0x7f636c101800 2023-09-13T03:12:11.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.354+0000 7f63739ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636c1a0420 con 0x7f636c100610 2023-09-13T03:12:11.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.354+0000 7f637179b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f636c100610 0x7f636c0743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:11.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.354+0000 7f6371f9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f636c1021a0 0x7f636c072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:11.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.354+0000 7f637179b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f636c100610 0x7f636c0743b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:45654/0 (socket says 172.21.15.64:45654) 2023-09-13T03:12:11.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.354+0000 7f637179b700 1 -- 172.21.15.64:0/1742089931 learned_addr learned my addr 172.21.15.64:0/1742089931 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:11.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.354+0000 7f6370f9a700 1 --2- 172.21.15.64:0/1742089931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f636c101800 0x7f636c0729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:11.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.354+0000 7f637179b700 1 -- 172.21.15.64:0/1742089931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f636c101800 msgr2=0x7f636c0729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:11.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.354+0000 7f637179b700 1 --2- 172.21.15.64:0/1742089931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f636c101800 0x7f636c0729f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:11.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.354+0000 7f637179b700 1 -- 172.21.15.64:0/1742089931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f636c1021a0 msgr2=0x7f636c072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:11.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.354+0000 7f637179b700 1 --2- 172.21.15.64:0/1742089931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f636c1021a0 0x7f636c072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:11.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.354+0000 7f637179b700 1 -- 172.21.15.64:0/1742089931 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6364009d00 con 0x7f636c100610 2023-09-13T03:12:11.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.355+0000 7f637179b700 1 --2- 172.21.15.64:0/1742089931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f636c100610 0x7f636c0743b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f636800e970 tx=0x7f636800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:11.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.355+0000 7f63627fc700 1 -- 172.21.15.64:0/1742089931 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6368009790 con 0x7f636c100610 2023-09-13T03:12:11.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.355+0000 7f63627fc700 1 -- 172.21.15.64:0/1742089931 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f636800c7e0 con 0x7f636c100610 2023-09-13T03:12:11.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.355+0000 7f63739ff700 1 -- 172.21.15.64:0/1742089931 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f636c1a0700 con 0x7f636c100610 2023-09-13T03:12:11.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.355+0000 7f63627fc700 1 -- 172.21.15.64:0/1742089931 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6368003c70 con 0x7f636c100610 2023-09-13T03:12:11.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.355+0000 7f63739ff700 1 -- 172.21.15.64:0/1742089931 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f636c1a0c10 con 0x7f636c100610 2023-09-13T03:12:11.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.356+0000 7f63739ff700 1 -- 172.21.15.64:0/1742089931 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f636c10be80 con 0x7f636c100610 2023-09-13T03:12:11.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.358+0000 7f63627fc700 1 -- 172.21.15.64:0/1742089931 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f636800cc50 con 0x7f636c100610 2023-09-13T03:12:11.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.359+0000 7f63627fc700 1 --2- 172.21.15.64:0/1742089931 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f63580a0710 0x7f63580a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:11.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.360+0000 7f63627fc700 1 -- 172.21.15.64:0/1742089931 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f6368014070 con 0x7f636c100610 2023-09-13T03:12:11.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.360+0000 7f6370f9a700 1 --2- 172.21.15.64:0/1742089931 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f63580a0710 0x7f63580a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:11.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.360+0000 7f6370f9a700 1 --2- 172.21.15.64:0/1742089931 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f63580a0710 0x7f63580a2bd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f635c009c60 tx=0x7f635c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:11.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.361+0000 7f63627fc700 1 -- 172.21.15.64:0/1742089931 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6368095a40 con 0x7f636c100610 2023-09-13T03:12:11.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.574+0000 7f63739ff700 1 -- 172.21.15.64:0/1742089931 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f636c10c090 con 0x7f636c100610 2023-09-13T03:12:11.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.577+0000 7f63627fc700 1 -- 172.21.15.64:0/1742089931 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22473 (secure 0 0 0) 0x7f6368003dd0 con 0x7f636c100610 2023-09-13T03:12:11.578 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:12:11.578 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":85,"fsid":"f8dbced4-51e1-11ee-9ab7-7b867c8bd7da","created":"2023-09-13T03:03:24.167202+0000","modified":"2023-09-13T03:11:24.258871+0000","last_up_change":"2023-09-13T03:10:22.503529+0000","last_in_change":"2023-09-13T03:10:05.678720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T03:06:58.987479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-13T03:11:08.589202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-13T03:11:09.924659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3997c114-241b-48c4-8feb-653c99209470","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6803","nonce":3940892135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6805","nonce":3940892135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6809","nonce":3940892135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6807","nonce":3940892135}]},"public_addr":"172.21.15.64:6803/3940892135","cluster_addr":"172.21.15.64:6805/3940892135","heartbeat_back_addr":"172.21.15.64:6809/3940892135","heartbeat_front_addr":"172.21.15.64:6807/3940892135","state":["exists","up"]},{"osd":1,"uuid":"94cfae85-322e-4ed2-a6e6-e40349bb14bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6801","nonce":4188149333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6803","nonce":4188149333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6807","nonce":4188149333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6805","nonce":4188149333}]},"public_addr":"172.21.15.142:6801/4188149333","cluster_addr":"172.21.15.142:6803/4188149333","heartbeat_back_addr":"172.21.15.142:6807/4188149333","heartbeat_front_addr":"172.21.15.142:6805/4188149333","state":["exists","up"]},{"osd":2,"uuid":"b3aae17e-96db-4008-8c27-f733c46b30ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6801","nonce":2406669338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6803","nonce":2406669338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6807","nonce":2406669338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6805","nonce":2406669338}]},"public_addr":"172.21.15.170:6801/2406669338","cluster_addr":"172.21.15.170:6803/2406669338","heartbeat_back_addr":"172.21.15.170:6807/2406669338","heartbeat_front_addr":"172.21.15.170:6805/2406669338","state":["exists","up"]},{"osd":3,"uuid":"7fa8d8cb-2ab7-48a7-8b49-5e2903d6779f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6811","nonce":3558318707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6813","nonce":3558318707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6817","nonce":3558318707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6815","nonce":3558318707}]},"public_addr":"172.21.15.64:6811/3558318707","cluster_addr":"172.21.15.64:6813/3558318707","heartbeat_back_addr":"172.21.15.64:6817/3558318707","heartbeat_front_addr":"172.21.15.64:6815/3558318707","state":["exists","up"]},{"osd":4,"uuid":"8c7a9c9f-f919-48d3-9db2-214a25bb5603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6809","nonce":1811142576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6811","nonce":1811142576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6815","nonce":1811142576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6813","nonce":1811142576}]},"public_addr":"172.21.15.142:6809/1811142576","cluster_addr":"172.21.15.142:6811/1811142576","heartbeat_back_addr":"172.21.15.142:6815/1811142576","heartbeat_front_addr":"172.21.15.142:6813/1811142576","state":["exists","up"]},{"osd":5,"uuid":"0c6101c6-2065-4022-a751-d3f216d7f478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6809","nonce":2903841453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6811","nonce":2903841453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6815","nonce":2903841453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6813","nonce":2903841453}]},"public_addr":"172.21.15.170:6809/2903841453","cluster_addr":"172.21.15.170:6811/2903841453","heartbeat_back_addr":"172.21.15.170:6815/2903841453","heartbeat_front_addr":"172.21.15.170:6813/2903841453","state":["exists","up"]},{"osd":6,"uuid":"5cf3e52e-d155-41b1-9194-db047d26a953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6819","nonce":1436584490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6821","nonce":1436584490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6825","nonce":1436584490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6823","nonce":1436584490}]},"public_addr":"172.21.15.64:6819/1436584490","cluster_addr":"172.21.15.64:6821/1436584490","heartbeat_back_addr":"172.21.15.64:6825/1436584490","heartbeat_front_addr":"172.21.15.64:6823/1436584490","state":["exists","up"]},{"osd":7,"uuid":"e97323e3-badd-41a7-af9e-e78a81a64046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6817","nonce":3726793534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6819","nonce":3726793534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6823","nonce":3726793534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6821","nonce":3726793534}]},"public_addr":"172.21.15.142:6817/3726793534","cluster_addr":"172.21.15.142:6819/3726793534","heartbeat_back_addr":"172.21.15.142:6823/3726793534","heartbeat_front_addr":"172.21.15.142:6821/3726793534","state":["exists","up"]},{"osd":8,"uuid":"83d2835c-ed26-493d-9222-c2894ec0e071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6817","nonce":4272513873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6819","nonce":4272513873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6823","nonce":4272513873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6821","nonce":4272513873}]},"public_addr":"172.21.15.170:6817/4272513873","cluster_addr":"172.21.15.170:6819/4272513873","heartbeat_back_addr":"172.21.15.170:6823/4272513873","heartbeat_front_addr":"172.21.15.170:6821/4272513873","state":["exists","up"]},{"osd":9,"uuid":"cb936bdd-1bec-4236-aa2e-a24adcac11e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6827","nonce":2641175211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6829","nonce":2641175211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6833","nonce":2641175211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6831","nonce":2641175211}]},"public_addr":"172.21.15.64:6827/2641175211","cluster_addr":"172.21.15.64:6829/2641175211","heartbeat_back_addr":"172.21.15.64:6833/2641175211","heartbeat_front_addr":"172.21.15.64:6831/2641175211","state":["exists","up"]},{"osd":10,"uuid":"85512a1c-6171-43d4-9c15-c588a8b71be0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6825","nonce":936499217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6827","nonce":936499217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6831","nonce":936499217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6829","nonce":936499217}]},"public_addr":"172.21.15.142:6825/936499217","cluster_addr":"172.21.15.142:6827/936499217","heartbeat_back_addr":"172.21.15.142:6831/936499217","heartbeat_front_addr":"172.21.15.142:6829/936499217","state":["exists","up"]},{"osd":11,"uuid":"9b8417dc-5d6c-4917-94a9-b032c5bc33d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6825","nonce":3605017060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6827","nonce":3605017060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6831","nonce":3605017060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6829","nonce":3605017060}]},"public_addr":"172.21.15.170:6825/3605017060","cluster_addr":"172.21.15.170:6827/3605017060","heartbeat_back_addr":"172.21.15.170:6831/3605017060","heartbeat_front_addr":"172.21.15.170:6829/3605017060","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:14.229044+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:36.147777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:56.655298+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:07:23.107741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:07:44.663770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:05.577432+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:31.168031+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:52.791055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:09:13.658546+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:09:39.150360+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:10:00.601735+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:10:21.407555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6800/2955227347":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/1373223118":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/271194613":"2023-09-14T03:04:22.355817+0000","172.21.15.64:6801/3998091347":"2023-09-14T03:03:56.442541+0000","172.21.15.64:6800/3998091347":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/920818572":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/735789005":"2023-09-14T03:03:56.442541+0000","172.21.15.64:6801/2955227347":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/1975923178":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/1500151178":"2023-09-14T03:04:22.355817+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T03:12:11.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.579+0000 7f63739ff700 1 -- 172.21.15.64:0/1742089931 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f63580a0710 msgr2=0x7f63580a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:11.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.579+0000 7f63739ff700 1 --2- 172.21.15.64:0/1742089931 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f63580a0710 0x7f63580a2bd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f635c009c60 tx=0x7f635c009260 comp rx=0 tx=0).stop 2023-09-13T03:12:11.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.579+0000 7f63739ff700 1 -- 172.21.15.64:0/1742089931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f636c100610 msgr2=0x7f636c0743b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:11.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.579+0000 7f63739ff700 1 --2- 172.21.15.64:0/1742089931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f636c100610 0x7f636c0743b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f636800e970 tx=0x7f636800ec80 comp rx=0 tx=0).stop 2023-09-13T03:12:11.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.579+0000 7f63739ff700 1 -- 172.21.15.64:0/1742089931 shutdown_connections 2023-09-13T03:12:11.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.579+0000 7f63739ff700 1 --2- 172.21.15.64:0/1742089931 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f63580a0710 0x7f63580a2bd0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:11.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.579+0000 7f63739ff700 1 --2- 172.21.15.64:0/1742089931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f636c1021a0 0x7f636c072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:11.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.579+0000 7f63739ff700 1 --2- 172.21.15.64:0/1742089931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f636c100610 0x7f636c0743b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:11.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.579+0000 7f63739ff700 1 --2- 172.21.15.64:0/1742089931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f636c101800 0x7f636c0729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:11.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.579+0000 7f63739ff700 1 -- 172.21.15.64:0/1742089931 >> 172.21.15.64:0/1742089931 conn(0x7f636c0fbb40 msgr2=0x7f636c0fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:11.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.579+0000 7f63739ff700 1 -- 172.21.15.64:0/1742089931 shutdown_connections 2023-09-13T03:12:11.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.579+0000 7f63739ff700 1 -- 172.21.15.64:0/1742089931 wait complete. 2023-09-13T03:12:11.599 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-13T03:06:58.987479+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-13T03:11:08.589202+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-13T03:11:09.924659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-13T03:12:11.600 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-13T03:12:11.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.806+0000 7f1918fff700 1 -- 172.21.15.64:0/3277226174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1914100620 msgr2=0x7f1914100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:11.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.806+0000 7f1918fff700 1 --2- 172.21.15.64:0/3277226174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1914100620 0x7f1914100a40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f19040099c0 tx=0x7f1904009cd0 comp rx=0 tx=0).stop 2023-09-13T03:12:11.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.806+0000 7f1918fff700 1 -- 172.21.15.64:0/3277226174 shutdown_connections 2023-09-13T03:12:11.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.806+0000 7f1918fff700 1 --2- 172.21.15.64:0/3277226174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1914100620 0x7f1914100a40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:11.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.806+0000 7f1918fff700 1 --2- 172.21.15.64:0/3277226174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19141021b0 0x7f1914108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:11.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.806+0000 7f1918fff700 1 --2- 172.21.15.64:0/3277226174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1914101810 0x7f1914101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:11.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.806+0000 7f1918fff700 1 -- 172.21.15.64:0/3277226174 >> 172.21.15.64:0/3277226174 conn(0x7f19140fbbb0 msgr2=0x7f19140fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:11.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.806+0000 7f1918fff700 1 -- 172.21.15.64:0/3277226174 shutdown_connections 2023-09-13T03:12:11.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.807+0000 7f1918fff700 1 -- 172.21.15.64:0/3277226174 wait complete. 2023-09-13T03:12:11.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.808+0000 7f1918fff700 1 Processor -- start 2023-09-13T03:12:11.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.808+0000 7f1918fff700 1 -- start start 2023-09-13T03:12:11.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.808+0000 7f1918fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1914100620 0x7f1914198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:11.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.808+0000 7f1918fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1914101810 0x7f1914198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:11.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.808+0000 7f1918fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19141021b0 0x7f191419fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:11.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.808+0000 7f1918fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19141a0090 con 0x7f1914101810 2023-09-13T03:12:11.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.808+0000 7f1918fff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19141a01d0 con 0x7f1914100620 2023-09-13T03:12:11.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.808+0000 7f1918fff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19141a0360 con 0x7f19141021b0 2023-09-13T03:12:11.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.808+0000 7f1911d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1914101810 0x7f1914198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:11.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.808+0000 7f1911d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1914101810 0x7f1914198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37762/0 (socket says 172.21.15.64:37762) 2023-09-13T03:12:11.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.808+0000 7f1911d9b700 1 -- 172.21.15.64:0/1240459780 learned_addr learned my addr 172.21.15.64:0/1240459780 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:11.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.808+0000 7f191259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1914100620 0x7f1914198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:11.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.809+0000 7f1912d9d700 1 --2- 172.21.15.64:0/1240459780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19141021b0 0x7f191419fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:11.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.809+0000 7f191259c700 1 -- 172.21.15.64:0/1240459780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19141021b0 msgr2=0x7f191419fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:11.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.809+0000 7f191259c700 1 --2- 172.21.15.64:0/1240459780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19141021b0 0x7f191419fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:11.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.809+0000 7f191259c700 1 -- 172.21.15.64:0/1240459780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1914101810 msgr2=0x7f1914198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:11.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.809+0000 7f191259c700 1 --2- 172.21.15.64:0/1240459780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1914101810 0x7f1914198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:11.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.809+0000 7f191259c700 1 -- 172.21.15.64:0/1240459780 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f190c009c40 con 0x7f1914100620 2023-09-13T03:12:11.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.809+0000 7f1911d9b700 1 --2- 172.21.15.64:0/1240459780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1914101810 0x7f1914198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:11.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.809+0000 7f191259c700 1 --2- 172.21.15.64:0/1240459780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1914100620 0x7f1914198510 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f19040098e0 tx=0x7f1904009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:11.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.809+0000 7f19037fe700 1 -- 172.21.15.64:0/1240459780 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190401d070 con 0x7f1914100620 2023-09-13T03:12:11.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.809+0000 7f19037fe700 1 -- 172.21.15.64:0/1240459780 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1904005020 con 0x7f1914100620 2023-09-13T03:12:11.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.809+0000 7f19037fe700 1 -- 172.21.15.64:0/1240459780 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1904004990 con 0x7f1914100620 2023-09-13T03:12:11.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.809+0000 7f1918fff700 1 -- 172.21.15.64:0/1240459780 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19040095f0 con 0x7f1914100620 2023-09-13T03:12:11.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.810+0000 7f1918fff700 1 -- 172.21.15.64:0/1240459780 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19141a07c0 con 0x7f1914100620 2023-09-13T03:12:11.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.810+0000 7f1918fff700 1 -- 172.21.15.64:0/1240459780 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f191410be50 con 0x7f1914100620 2023-09-13T03:12:11.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.812+0000 7f19037fe700 1 -- 172.21.15.64:0/1240459780 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f1904005190 con 0x7f1914100620 2023-09-13T03:12:11.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.814+0000 7f19037fe700 1 --2- 172.21.15.64:0/1240459780 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f18fc0a06d0 0x7f18fc0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:11.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.814+0000 7f19037fe700 1 -- 172.21.15.64:0/1240459780 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f19040ccac0 con 0x7f1914100620 2023-09-13T03:12:11.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.814+0000 7f1911d9b700 1 --2- 172.21.15.64:0/1240459780 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f18fc0a06d0 0x7f18fc0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:11.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.814+0000 7f1911d9b700 1 --2- 172.21.15.64:0/1240459780 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f18fc0a06d0 0x7f18fc0a2b90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1908011550 tx=0x7f1908009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:11.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:11.816+0000 7f19037fe700 1 -- 172.21.15.64:0/1240459780 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f19040964f0 con 0x7f1914100620 2023-09-13T03:12:11.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:11 smithi170 ceph-mon[134328]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-13T03:12:11.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:11 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/3887643684' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-09-13T03:12:11.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:11 smithi064 ceph-mon[97675]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-13T03:12:11.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:11 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/3887643684' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-09-13T03:12:11.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:11 smithi142 ceph-mon[134082]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-13T03:12:11.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:11 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/3887643684' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-09-13T03:12:12.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.026+0000 7f1918fff700 1 -- 172.21.15.64:0/1240459780 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f19141a0a10 con 0x7f1914100620 2023-09-13T03:12:12.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.026+0000 7f19037fe700 1 -- 172.21.15.64:0/1240459780 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1904016e80 con 0x7f1914100620 2023-09-13T03:12:12.027 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-09-13T03:12:12.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.029+0000 7f1918fff700 1 -- 172.21.15.64:0/1240459780 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f18fc0a06d0 msgr2=0x7f18fc0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:12.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.029+0000 7f1918fff700 1 --2- 172.21.15.64:0/1240459780 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f18fc0a06d0 0x7f18fc0a2b90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1908011550 tx=0x7f1908009260 comp rx=0 tx=0).stop 2023-09-13T03:12:12.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.029+0000 7f1918fff700 1 -- 172.21.15.64:0/1240459780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1914100620 msgr2=0x7f1914198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:12.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.029+0000 7f1918fff700 1 --2- 172.21.15.64:0/1240459780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1914100620 0x7f1914198510 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f19040098e0 tx=0x7f1904009580 comp rx=0 tx=0).stop 2023-09-13T03:12:12.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.029+0000 7f1918fff700 1 -- 172.21.15.64:0/1240459780 shutdown_connections 2023-09-13T03:12:12.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.029+0000 7f1918fff700 1 --2- 172.21.15.64:0/1240459780 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f18fc0a06d0 0x7f18fc0a2b90 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:12.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.029+0000 7f1918fff700 1 --2- 172.21.15.64:0/1240459780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1914100620 0x7f1914198510 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:12.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.029+0000 7f1918fff700 1 --2- 172.21.15.64:0/1240459780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19141021b0 0x7f191419fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:12.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.029+0000 7f1918fff700 1 --2- 172.21.15.64:0/1240459780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1914101810 0x7f1914198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:12.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.029+0000 7f1918fff700 1 -- 172.21.15.64:0/1240459780 >> 172.21.15.64:0/1240459780 conn(0x7f19140fbbb0 msgr2=0x7f19140fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:12.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.029+0000 7f1918fff700 1 -- 172.21.15.64:0/1240459780 shutdown_connections 2023-09-13T03:12:12.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.029+0000 7f1918fff700 1 -- 172.21.15.64:0/1240459780 wait complete. 2023-09-13T03:12:12.048 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-13T03:12:12.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.255+0000 7efd015ff700 1 -- 172.21.15.64:0/536837942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efcfc1021b0 msgr2=0x7efcfc108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:12.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.255+0000 7efd015ff700 1 --2- 172.21.15.64:0/536837942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efcfc1021b0 0x7efcfc108a60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efcf400b0a0 tx=0x7efcf400b3b0 comp rx=0 tx=0).stop 2023-09-13T03:12:12.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.255+0000 7efd015ff700 1 -- 172.21.15.64:0/536837942 shutdown_connections 2023-09-13T03:12:12.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.255+0000 7efd015ff700 1 --2- 172.21.15.64:0/536837942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efcfc1021b0 0x7efcfc108a60 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:12.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.255+0000 7efd015ff700 1 --2- 172.21.15.64:0/536837942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcfc100620 0x7efcfc100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:12.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.255+0000 7efd015ff700 1 --2- 172.21.15.64:0/536837942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcfc101810 0x7efcfc101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:12.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.255+0000 7efd015ff700 1 -- 172.21.15.64:0/536837942 >> 172.21.15.64:0/536837942 conn(0x7efcfc0fbbb0 msgr2=0x7efcfc0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:12.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.255+0000 7efd015ff700 1 -- 172.21.15.64:0/536837942 shutdown_connections 2023-09-13T03:12:12.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.255+0000 7efd015ff700 1 -- 172.21.15.64:0/536837942 wait complete. 2023-09-13T03:12:12.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.256+0000 7efd015ff700 1 Processor -- start 2023-09-13T03:12:12.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.256+0000 7efd015ff700 1 -- start start 2023-09-13T03:12:12.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.256+0000 7efd015ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcfc100620 0x7efcfc1963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:12.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.257+0000 7efd015ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcfc101810 0x7efcfc1968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:12.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.257+0000 7efd015ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efcfc1021b0 0x7efcfc19d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:12.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.257+0000 7efd015ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc19df30 con 0x7efcfc100620 2023-09-13T03:12:12.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.257+0000 7efd015ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc19e070 con 0x7efcfc101810 2023-09-13T03:12:12.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.257+0000 7efd015ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc19e200 con 0x7efcfc1021b0 2023-09-13T03:12:12.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.257+0000 7efcfaffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcfc100620 0x7efcfc1963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:12.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.257+0000 7efcfaffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcfc100620 0x7efcfc1963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37780/0 (socket says 172.21.15.64:37780) 2023-09-13T03:12:12.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.257+0000 7efcfaffd700 1 -- 172.21.15.64:0/2810109863 learned_addr learned my addr 172.21.15.64:0/2810109863 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:12.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.257+0000 7efcfb7fe700 1 --2- 172.21.15.64:0/2810109863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efcfc1021b0 0x7efcfc19d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:12.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.257+0000 7efcfa7fc700 1 --2- 172.21.15.64:0/2810109863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcfc101810 0x7efcfc1968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:12.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.257+0000 7efcfb7fe700 1 -- 172.21.15.64:0/2810109863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcfc101810 msgr2=0x7efcfc1968f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:12.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.257+0000 7efcfb7fe700 1 --2- 172.21.15.64:0/2810109863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcfc101810 0x7efcfc1968f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:12.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.257+0000 7efcfb7fe700 1 -- 172.21.15.64:0/2810109863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcfc100620 msgr2=0x7efcfc1963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:12.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.257+0000 7efcfb7fe700 1 --2- 172.21.15.64:0/2810109863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcfc100620 0x7efcfc1963b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:12.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.257+0000 7efcfb7fe700 1 -- 172.21.15.64:0/2810109863 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf4009d00 con 0x7efcfc1021b0 2023-09-13T03:12:12.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.258+0000 7efcfb7fe700 1 --2- 172.21.15.64:0/2810109863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efcfc1021b0 0x7efcfc19d960 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efcf400b070 tx=0x7efcf40079b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:12.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.258+0000 7efce3fff700 1 -- 172.21.15.64:0/2810109863 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf400e040 con 0x7efcfc1021b0 2023-09-13T03:12:12.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.258+0000 7efce3fff700 1 -- 172.21.15.64:0/2810109863 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcf400be40 con 0x7efcfc1021b0 2023-09-13T03:12:12.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.258+0000 7efd015ff700 1 -- 172.21.15.64:0/2810109863 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcfc19e480 con 0x7efcfc1021b0 2023-09-13T03:12:12.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.258+0000 7efce3fff700 1 -- 172.21.15.64:0/2810109863 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf40052a0 con 0x7efcfc1021b0 2023-09-13T03:12:12.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.258+0000 7efd015ff700 1 -- 172.21.15.64:0/2810109863 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcfc19e8b0 con 0x7efcfc1021b0 2023-09-13T03:12:12.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.259+0000 7efd015ff700 1 -- 172.21.15.64:0/2810109863 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcfc1074b0 con 0x7efcfc1021b0 2023-09-13T03:12:12.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.261+0000 7efce3fff700 1 -- 172.21.15.64:0/2810109863 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7efcf401d470 con 0x7efcfc1021b0 2023-09-13T03:12:12.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.263+0000 7efce3fff700 1 --2- 172.21.15.64:0/2810109863 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7efce40a06d0 0x7efce40a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:12.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.263+0000 7efce3fff700 1 -- 172.21.15.64:0/2810109863 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7efcf40cd7e0 con 0x7efcfc1021b0 2023-09-13T03:12:12.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.263+0000 7efcfaffd700 1 --2- 172.21.15.64:0/2810109863 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7efce40a06d0 0x7efce40a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:12.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.263+0000 7efcfaffd700 1 --2- 172.21.15.64:0/2810109863 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7efce40a06d0 0x7efce40a2b90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7efcec005950 tx=0x7efcec009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:12.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.265+0000 7efce3fff700 1 -- 172.21.15.64:0/2810109863 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7efcf4097330 con 0x7efcfc1021b0 2023-09-13T03:12:12.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.478+0000 7efd015ff700 1 -- 172.21.15.64:0/2810109863 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efcfc10c090 con 0x7efcfc1021b0 2023-09-13T03:12:12.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.478+0000 7efce3fff700 1 -- 172.21.15.64:0/2810109863 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7efcf409a950 con 0x7efcfc1021b0 2023-09-13T03:12:12.479 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-09-13T03:12:12.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.481+0000 7efd015ff700 1 -- 172.21.15.64:0/2810109863 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7efce40a06d0 msgr2=0x7efce40a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:12.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.481+0000 7efd015ff700 1 --2- 172.21.15.64:0/2810109863 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7efce40a06d0 0x7efce40a2b90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7efcec005950 tx=0x7efcec009260 comp rx=0 tx=0).stop 2023-09-13T03:12:12.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.481+0000 7efd015ff700 1 -- 172.21.15.64:0/2810109863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efcfc1021b0 msgr2=0x7efcfc19d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:12.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.481+0000 7efd015ff700 1 --2- 172.21.15.64:0/2810109863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efcfc1021b0 0x7efcfc19d960 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efcf400b070 tx=0x7efcf40079b0 comp rx=0 tx=0).stop 2023-09-13T03:12:12.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.482+0000 7efd015ff700 1 -- 172.21.15.64:0/2810109863 shutdown_connections 2023-09-13T03:12:12.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.482+0000 7efd015ff700 1 --2- 172.21.15.64:0/2810109863 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7efce40a06d0 0x7efce40a2b90 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:12.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.482+0000 7efd015ff700 1 --2- 172.21.15.64:0/2810109863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efcfc1021b0 0x7efcfc19d960 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:12.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.482+0000 7efd015ff700 1 --2- 172.21.15.64:0/2810109863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcfc100620 0x7efcfc1963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:12.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.482+0000 7efd015ff700 1 --2- 172.21.15.64:0/2810109863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcfc101810 0x7efcfc1968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:12.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.482+0000 7efd015ff700 1 -- 172.21.15.64:0/2810109863 >> 172.21.15.64:0/2810109863 conn(0x7efcfc0fbbb0 msgr2=0x7efcfc0fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:12.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.482+0000 7efd015ff700 1 -- 172.21.15.64:0/2810109863 shutdown_connections 2023-09-13T03:12:12.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.482+0000 7efd015ff700 1 -- 172.21.15.64:0/2810109863 wait complete. 2023-09-13T03:12:12.502 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-13T03:12:12.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.718+0000 7f810539d700 1 -- 172.21.15.64:0/2681158992 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8100100f80 msgr2=0x7f8100103370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:12.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.718+0000 7f810539d700 1 --2- 172.21.15.64:0/2681158992 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8100100f80 0x7f8100103370 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f80f0009990 tx=0x7f80f0009ca0 comp rx=0 tx=0).stop 2023-09-13T03:12:12.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.718+0000 7f810539d700 1 -- 172.21.15.64:0/2681158992 shutdown_connections 2023-09-13T03:12:12.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.718+0000 7f810539d700 1 --2- 172.21.15.64:0/2681158992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81001061e0 0x7f81001085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:12.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.718+0000 7f810539d700 1 --2- 172.21.15.64:0/2681158992 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8100100f80 0x7f8100103370 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:12.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.718+0000 7f810539d700 1 --2- 172.21.15.64:0/2681158992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81001038b0 0x7f8100105ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:12.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.718+0000 7f810539d700 1 -- 172.21.15.64:0/2681158992 >> 172.21.15.64:0/2681158992 conn(0x7f81000fa8b0 msgr2=0x7f81000fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:12.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.718+0000 7f810539d700 1 -- 172.21.15.64:0/2681158992 shutdown_connections 2023-09-13T03:12:12.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.718+0000 7f810539d700 1 -- 172.21.15.64:0/2681158992 wait complete. 2023-09-13T03:12:12.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.720+0000 7f810539d700 1 Processor -- start 2023-09-13T03:12:12.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.720+0000 7f810539d700 1 -- start start 2023-09-13T03:12:12.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.721+0000 7f810539d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8100100f80 0x7f81001008f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:12.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.721+0000 7f810539d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81001038b0 0x7f81000fef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:12.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.721+0000 7f810539d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81001061e0 0x7f81000ff470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:12.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.721+0000 7f810539d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81000ffc10 con 0x7f8100100f80 2023-09-13T03:12:12.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.721+0000 7f810539d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81000ffd80 con 0x7f81001061e0 2023-09-13T03:12:12.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.721+0000 7f810539d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81001a0100 con 0x7f81001038b0 2023-09-13T03:12:12.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.721+0000 7f80ff7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81001038b0 0x7f81000fef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:12.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.721+0000 7f80fffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8100100f80 0x7f81001008f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:12.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.721+0000 7f80ff7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81001038b0 0x7f81000fef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:45712/0 (socket says 172.21.15.64:45712) 2023-09-13T03:12:12.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.721+0000 7f80ff7fe700 1 -- 172.21.15.64:0/2942475027 learned_addr learned my addr 172.21.15.64:0/2942475027 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:12.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.721+0000 7f8104b9c700 1 --2- 172.21.15.64:0/2942475027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81001061e0 0x7f81000ff470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:12.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.721+0000 7f80ff7fe700 1 -- 172.21.15.64:0/2942475027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81001061e0 msgr2=0x7f81000ff470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:12.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.721+0000 7f80ff7fe700 1 --2- 172.21.15.64:0/2942475027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81001061e0 0x7f81000ff470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:12.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.721+0000 7f80ff7fe700 1 -- 172.21.15.64:0/2942475027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8100100f80 msgr2=0x7f81001008f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:12.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.721+0000 7f80ff7fe700 1 --2- 172.21.15.64:0/2942475027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8100100f80 0x7f81001008f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:12.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.721+0000 7f80ff7fe700 1 -- 172.21.15.64:0/2942475027 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f00095f0 con 0x7f81001038b0 2023-09-13T03:12:12.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.722+0000 7f80ff7fe700 1 --2- 172.21.15.64:0/2942475027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81001038b0 0x7f81000fef30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f80f400e9a0 tx=0x7f80f400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:12.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.722+0000 7f80fd7fa700 1 -- 172.21.15.64:0/2942475027 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f4009790 con 0x7f81001038b0 2023-09-13T03:12:12.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.722+0000 7f80fd7fa700 1 -- 172.21.15.64:0/2942475027 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80f4010430 con 0x7f81001038b0 2023-09-13T03:12:12.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.722+0000 7f80fd7fa700 1 -- 172.21.15.64:0/2942475027 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f4018680 con 0x7f81001038b0 2023-09-13T03:12:12.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.722+0000 7f810539d700 1 -- 172.21.15.64:0/2942475027 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81001a03e0 con 0x7f81001038b0 2023-09-13T03:12:12.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.722+0000 7f810539d700 1 -- 172.21.15.64:0/2942475027 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81001a08a0 con 0x7f81001038b0 2023-09-13T03:12:12.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.723+0000 7f810539d700 1 -- 172.21.15.64:0/2942475027 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81000fc430 con 0x7f81001038b0 2023-09-13T03:12:12.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.725+0000 7f80fd7fa700 1 -- 172.21.15.64:0/2942475027 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f80f40105a0 con 0x7f81001038b0 2023-09-13T03:12:12.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.726+0000 7f80fd7fa700 1 --2- 172.21.15.64:0/2942475027 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f80e80a0800 0x7f80e80a2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:12.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.726+0000 7f80fd7fa700 1 -- 172.21.15.64:0/2942475027 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f80f4014070 con 0x7f81001038b0 2023-09-13T03:12:12.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.727+0000 7f80fffff700 1 --2- 172.21.15.64:0/2942475027 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f80e80a0800 0x7f80e80a2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:12.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.727+0000 7f80fffff700 1 --2- 172.21.15.64:0/2942475027 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f80e80a0800 0x7f80e80a2cc0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f80f000b2e0 tx=0x7f80f0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:12.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.730+0000 7f80fd7fa700 1 -- 172.21.15.64:0/2942475027 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f80f4051e10 con 0x7f81001038b0 2023-09-13T03:12:12.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:12 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/1742089931' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:12:12.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:12 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/1240459780' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T03:12:12.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:12 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/2810109863' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T03:12:12.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:12 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:12:12.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:12 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/1742089931' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:12:12.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:12 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/1240459780' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T03:12:12.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:12 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2810109863' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T03:12:12.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:12 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:12:12.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.945+0000 7f810539d700 1 -- 172.21.15.64:0/2942475027 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f810010be60 con 0x7f81001038b0 2023-09-13T03:12:12.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.945+0000 7f80fd7fa700 1 -- 172.21.15.64:0/2942475027 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f80f4095850 con 0x7f81001038b0 2023-09-13T03:12:12.946 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-09-13T03:12:12.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.948+0000 7f810539d700 1 -- 172.21.15.64:0/2942475027 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f80e80a0800 msgr2=0x7f80e80a2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:12.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.948+0000 7f810539d700 1 --2- 172.21.15.64:0/2942475027 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f80e80a0800 0x7f80e80a2cc0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f80f000b2e0 tx=0x7f80f0005fb0 comp rx=0 tx=0).stop 2023-09-13T03:12:12.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.948+0000 7f810539d700 1 -- 172.21.15.64:0/2942475027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81001038b0 msgr2=0x7f81000fef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:12.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.948+0000 7f810539d700 1 --2- 172.21.15.64:0/2942475027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81001038b0 0x7f81000fef30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f80f400e9a0 tx=0x7f80f400ecb0 comp rx=0 tx=0).stop 2023-09-13T03:12:12.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.948+0000 7f810539d700 1 -- 172.21.15.64:0/2942475027 shutdown_connections 2023-09-13T03:12:12.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.948+0000 7f810539d700 1 --2- 172.21.15.64:0/2942475027 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f80e80a0800 0x7f80e80a2cc0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:12.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.948+0000 7f810539d700 1 --2- 172.21.15.64:0/2942475027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81001061e0 0x7f81000ff470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:12.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.948+0000 7f810539d700 1 --2- 172.21.15.64:0/2942475027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8100100f80 0x7f81001008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:12.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.948+0000 7f810539d700 1 --2- 172.21.15.64:0/2942475027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81001038b0 0x7f81000fef30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:12.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.948+0000 7f810539d700 1 -- 172.21.15.64:0/2942475027 >> 172.21.15.64:0/2942475027 conn(0x7f81000fa8b0 msgr2=0x7f8100101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:12.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.948+0000 7f810539d700 1 -- 172.21.15.64:0/2942475027 shutdown_connections 2023-09-13T03:12:12.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:12.948+0000 7f810539d700 1 -- 172.21.15.64:0/2942475027 wait complete. 2023-09-13T03:12:12.967 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T03:12:12.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:12 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/1742089931' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:12:12.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:12 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/1240459780' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T03:12:12.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:12 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/2810109863' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T03:12:12.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:12 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:12:13.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.176+0000 7fe8f23ff700 1 -- 172.21.15.64:0/3642473528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8ec074640 msgr2=0x7fe8ec072a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:13.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.176+0000 7fe8f23ff700 1 --2- 172.21.15.64:0/3642473528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8ec074640 0x7fe8ec072a90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc009990 tx=0x7fe8dc009ca0 comp rx=0 tx=0).stop 2023-09-13T03:12:13.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.177+0000 7fe8f23ff700 1 -- 172.21.15.64:0/3642473528 shutdown_connections 2023-09-13T03:12:13.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.177+0000 7fe8f23ff700 1 --2- 172.21.15.64:0/3642473528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8ec10cbd0 0x7fe8ec10efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:13.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.177+0000 7fe8f23ff700 1 --2- 172.21.15.64:0/3642473528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8ec074640 0x7fe8ec072a90 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:13.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.177+0000 7fe8f23ff700 1 --2- 172.21.15.64:0/3642473528 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8ec073060 0x7fe8ec0734e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:13.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.177+0000 7fe8f23ff700 1 -- 172.21.15.64:0/3642473528 >> 172.21.15.64:0/3642473528 conn(0x7fe8ec0fbb30 msgr2=0x7fe8ec0fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:13.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.177+0000 7fe8f23ff700 1 -- 172.21.15.64:0/3642473528 shutdown_connections 2023-09-13T03:12:13.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.177+0000 7fe8f23ff700 1 -- 172.21.15.64:0/3642473528 wait complete. 2023-09-13T03:12:13.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.178+0000 7fe8f23ff700 1 Processor -- start 2023-09-13T03:12:13.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.178+0000 7fe8f23ff700 1 -- start start 2023-09-13T03:12:13.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.178+0000 7fe8f23ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8ec073060 0x7fe8ec19c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:13.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.178+0000 7fe8f23ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8ec074640 0x7fe8ec19ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:13.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.178+0000 7fe8f23ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8ec10cbd0 0x7fe8ec1a3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:13.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.178+0000 7fe8f23ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec1a4480 con 0x7fe8ec073060 2023-09-13T03:12:13.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.178+0000 7fe8f23ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec1a45c0 con 0x7fe8ec10cbd0 2023-09-13T03:12:13.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.178+0000 7fe8f23ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec1a4750 con 0x7fe8ec074640 2023-09-13T03:12:13.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.178+0000 7fe8ebfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8ec073060 0x7fe8ec19c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:13.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.179+0000 7fe8ebfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8ec073060 0x7fe8ec19c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37832/0 (socket says 172.21.15.64:37832) 2023-09-13T03:12:13.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.179+0000 7fe8eb7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8ec074640 0x7fe8ec19ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:13.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.179+0000 7fe8ebfff700 1 -- 172.21.15.64:0/3482432989 learned_addr learned my addr 172.21.15.64:0/3482432989 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:13.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.179+0000 7fe8f099c700 1 --2- 172.21.15.64:0/3482432989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8ec10cbd0 0x7fe8ec1a3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:13.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.179+0000 7fe8ebfff700 1 -- 172.21.15.64:0/3482432989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8ec074640 msgr2=0x7fe8ec19ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:13.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.179+0000 7fe8ebfff700 1 --2- 172.21.15.64:0/3482432989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8ec074640 0x7fe8ec19ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:13.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.179+0000 7fe8ebfff700 1 -- 172.21.15.64:0/3482432989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8ec10cbd0 msgr2=0x7fe8ec1a3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:13.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.179+0000 7fe8ebfff700 1 --2- 172.21.15.64:0/3482432989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8ec10cbd0 0x7fe8ec1a3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:13.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.179+0000 7fe8ebfff700 1 -- 172.21.15.64:0/3482432989 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e0009520 con 0x7fe8ec073060 2023-09-13T03:12:13.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.179+0000 7fe8eb7fe700 1 --2- 172.21.15.64:0/3482432989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8ec074640 0x7fe8ec19ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:13.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.179+0000 7fe8ebfff700 1 --2- 172.21.15.64:0/3482432989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8ec073060 0x7fe8ec19c900 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc00b680 tx=0x7fe8dc005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:13.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.179+0000 7fe8e97fa700 1 -- 172.21.15.64:0/3482432989 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc01d070 con 0x7fe8ec073060 2023-09-13T03:12:13.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.180+0000 7fe8e97fa700 1 -- 172.21.15.64:0/3482432989 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8dc022470 con 0x7fe8ec073060 2023-09-13T03:12:13.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.180+0000 7fe8f23ff700 1 -- 172.21.15.64:0/3482432989 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8dc0095f0 con 0x7fe8ec073060 2023-09-13T03:12:13.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.180+0000 7fe8e97fa700 1 -- 172.21.15.64:0/3482432989 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc017650 con 0x7fe8ec073060 2023-09-13T03:12:13.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.180+0000 7fe8f23ff700 1 -- 172.21.15.64:0/3482432989 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8ec1a4c60 con 0x7fe8ec073060 2023-09-13T03:12:13.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.181+0000 7fe8f23ff700 1 -- 172.21.15.64:0/3482432989 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8ec102720 con 0x7fe8ec073060 2023-09-13T03:12:13.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.183+0000 7fe8e97fa700 1 -- 172.21.15.64:0/3482432989 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fe8dc0225e0 con 0x7fe8ec073060 2023-09-13T03:12:13.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.185+0000 7fe8e97fa700 1 --2- 172.21.15.64:0/3482432989 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe8d40a0710 0x7fe8d40a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:13.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.185+0000 7fe8e97fa700 1 -- 172.21.15.64:0/3482432989 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fe8dc0cd8f0 con 0x7fe8ec073060 2023-09-13T03:12:13.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.186+0000 7fe8eb7fe700 1 --2- 172.21.15.64:0/3482432989 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe8d40a0710 0x7fe8d40a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:13.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.186+0000 7fe8eb7fe700 1 --2- 172.21.15.64:0/3482432989 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe8d40a0710 0x7fe8d40a2bd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009fd0 tx=0x7fe8e0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:13.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.187+0000 7fe8e97fa700 1 -- 172.21.15.64:0/3482432989 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe8dc097320 con 0x7fe8ec073060 2023-09-13T03:12:13.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.461+0000 7fe8f23ff700 1 -- 172.21.15.64:0/3482432989 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8ec112690 con 0x7fe8ec073060 2023-09-13T03:12:13.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.462+0000 7fe8e97fa700 1 -- 172.21.15.64:0/3482432989 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11366 (secure 0 0 0) 0x7fe8dc09a940 con 0x7fe8ec073060 2023-09-13T03:12:13.463 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:12:13.463 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2234182985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2234182985},{"type":"v1","addr":"172.21.15.64:6835","nonce":2234182985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4053841873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4053841873},{"type":"v1","addr":"172.21.15.64:6837","nonce":4053841873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-13T03:11:10.490260+0000","modified":"2023-09-13T03:11:18.230740+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24686,"mds_1":24649,"mds_2":24635,"mds_3":24634,"mds_4":24622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6839/99523374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":99523374},{"type":"v1","addr":"172.21.15.64:6839","nonce":99523374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14649":{"gid":14649,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6841/1866582875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":1866582875},{"type":"v1","addr":"172.21.15.64:6841","nonce":1866582875}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/101906999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":101906999},{"type":"v1","addr":"172.21.15.142:6833","nonce":101906999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24574":{"gid":24574,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/1897366227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1897366227},{"type":"v1","addr":"172.21.15.170:6833","nonce":1897366227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24611":{"gid":24611,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2167779918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2167779918},{"type":"v1","addr":"172.21.15.142:6835","nonce":2167779918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24622":{"gid":24622,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6835/1493371121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1493371121},{"type":"v1","addr":"172.21.15.170:6835","nonce":1493371121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24634":{"gid":24634,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.170:6837/1726587275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":1726587275},{"type":"v1","addr":"172.21.15.170:6837","nonce":1726587275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24635":{"gid":24635,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.142:6837/741173268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":741173268},{"type":"v1","addr":"172.21.15.142:6837","nonce":741173268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24649":{"gid":24649,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.170:6839/2189703596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2189703596},{"type":"v1","addr":"172.21.15.170:6839","nonce":2189703596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24686":{"gid":24686,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/3645998616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645998616},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645998616}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-13T03:12:13.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.464+0000 7fe8f23ff700 1 -- 172.21.15.64:0/3482432989 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe8d40a0710 msgr2=0x7fe8d40a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:13.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.464+0000 7fe8f23ff700 1 --2- 172.21.15.64:0/3482432989 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe8d40a0710 0x7fe8d40a2bd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009fd0 tx=0x7fe8e0009260 comp rx=0 tx=0).stop 2023-09-13T03:12:13.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.464+0000 7fe8f23ff700 1 -- 172.21.15.64:0/3482432989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8ec073060 msgr2=0x7fe8ec19c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:13.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.464+0000 7fe8f23ff700 1 --2- 172.21.15.64:0/3482432989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8ec073060 0x7fe8ec19c900 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc00b680 tx=0x7fe8dc005740 comp rx=0 tx=0).stop 2023-09-13T03:12:13.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.464+0000 7fe8f23ff700 1 -- 172.21.15.64:0/3482432989 shutdown_connections 2023-09-13T03:12:13.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.464+0000 7fe8f23ff700 1 --2- 172.21.15.64:0/3482432989 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe8d40a0710 0x7fe8d40a2bd0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:13.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.464+0000 7fe8f23ff700 1 --2- 172.21.15.64:0/3482432989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8ec10cbd0 0x7fe8ec1a3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:13.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.464+0000 7fe8f23ff700 1 --2- 172.21.15.64:0/3482432989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8ec073060 0x7fe8ec19c900 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:13.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.464+0000 7fe8f23ff700 1 --2- 172.21.15.64:0/3482432989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8ec074640 0x7fe8ec19ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:13.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.464+0000 7fe8f23ff700 1 -- 172.21.15.64:0/3482432989 >> 172.21.15.64:0/3482432989 conn(0x7fe8ec0fbb30 msgr2=0x7fe8ec0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:13.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.464+0000 7fe8f23ff700 1 -- 172.21.15.64:0/3482432989 shutdown_connections 2023-09-13T03:12:13.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.465+0000 7fe8f23ff700 1 -- 172.21.15.64:0/3482432989 wait complete. 2023-09-13T03:12:13.469 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 26 2023-09-13T03:12:13.484 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-13T03:12:13.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.704+0000 7f696dbff700 1 -- 172.21.15.64:0/3522219048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69681010c0 msgr2=0x7f69681034b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:13.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.704+0000 7f696dbff700 1 --2- 172.21.15.64:0/3522219048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69681010c0 0x7f69681034b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6958009990 tx=0x7f6958009ca0 comp rx=0 tx=0).stop 2023-09-13T03:12:13.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.704+0000 7f696dbff700 1 -- 172.21.15.64:0/3522219048 shutdown_connections 2023-09-13T03:12:13.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.704+0000 7f696dbff700 1 --2- 172.21.15.64:0/3522219048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69681010c0 0x7f69681034b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:13.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.704+0000 7f696dbff700 1 --2- 172.21.15.64:0/3522219048 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6968106320 0x7f6968108730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:13.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.704+0000 7f696dbff700 1 --2- 172.21.15.64:0/3522219048 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69681039f0 0x7f6968105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:13.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.704+0000 7f696dbff700 1 -- 172.21.15.64:0/3522219048 >> 172.21.15.64:0/3522219048 conn(0x7f69680faa30 msgr2=0x7f69680fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:13.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.704+0000 7f696dbff700 1 -- 172.21.15.64:0/3522219048 shutdown_connections 2023-09-13T03:12:13.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.704+0000 7f696dbff700 1 -- 172.21.15.64:0/3522219048 wait complete. 2023-09-13T03:12:13.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.705+0000 7f696dbff700 1 Processor -- start 2023-09-13T03:12:13.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.705+0000 7f696dbff700 1 -- start start 2023-09-13T03:12:13.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.706+0000 7f696dbff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69681010c0 0x7f69681984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:13.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.706+0000 7f696dbff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69681039f0 0x7f6968198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:13.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.706+0000 7f696dbff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6968106320 0x7f696819fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:13.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.706+0000 7f696dbff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69681a0050 con 0x7f6968106320 2023-09-13T03:12:13.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.706+0000 7f696dbff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69681a0190 con 0x7f69681039f0 2023-09-13T03:12:13.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.706+0000 7f696dbff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69681a0320 con 0x7f69681010c0 2023-09-13T03:12:13.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.706+0000 7f69677fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69681010c0 0x7f69681984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:13.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.706+0000 7f695ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69681039f0 0x7f6968198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:13.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.706+0000 7f69677fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69681010c0 0x7f69681984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:45748/0 (socket says 172.21.15.64:45748) 2023-09-13T03:12:13.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.706+0000 7f695ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69681039f0 0x7f6968198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:59868/0 (socket says 172.21.15.64:59868) 2023-09-13T03:12:13.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.706+0000 7f69677fe700 1 -- 172.21.15.64:0/2778553202 learned_addr learned my addr 172.21.15.64:0/2778553202 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:13.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.706+0000 7f6967fff700 1 --2- 172.21.15.64:0/2778553202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6968106320 0x7f696819fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:13.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.706+0000 7f69677fe700 1 -- 172.21.15.64:0/2778553202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69681039f0 msgr2=0x7f6968198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:13.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.706+0000 7f69677fe700 1 --2- 172.21.15.64:0/2778553202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69681039f0 0x7f6968198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:13.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.706+0000 7f69677fe700 1 -- 172.21.15.64:0/2778553202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6968106320 msgr2=0x7f696819fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:13.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.706+0000 7f69677fe700 1 --2- 172.21.15.64:0/2778553202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6968106320 0x7f696819fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:13.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.706+0000 7f69677fe700 1 -- 172.21.15.64:0/2778553202 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69580095f0 con 0x7f69681010c0 2023-09-13T03:12:13.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.706+0000 7f695ffff700 1 --2- 172.21.15.64:0/2778553202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69681039f0 0x7f6968198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:13.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.707+0000 7f69677fe700 1 --2- 172.21.15.64:0/2778553202 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69681010c0 0x7f69681984d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f695800b680 tx=0x7f6958005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:13.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.707+0000 7f69657fa700 1 -- 172.21.15.64:0/2778553202 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695801d070 con 0x7f69681010c0 2023-09-13T03:12:13.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.707+0000 7f696dbff700 1 -- 172.21.15.64:0/2778553202 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69681a05a0 con 0x7f69681010c0 2023-09-13T03:12:13.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.707+0000 7f69657fa700 1 -- 172.21.15.64:0/2778553202 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6958022470 con 0x7f69681010c0 2023-09-13T03:12:13.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.707+0000 7f69657fa700 1 -- 172.21.15.64:0/2778553202 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6958017650 con 0x7f69681010c0 2023-09-13T03:12:13.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.707+0000 7f696dbff700 1 -- 172.21.15.64:0/2778553202 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69681a09c0 con 0x7f69681010c0 2023-09-13T03:12:13.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.708+0000 7f696dbff700 1 -- 172.21.15.64:0/2778553202 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f696810be30 con 0x7f69681010c0 2023-09-13T03:12:13.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.710+0000 7f69657fa700 1 -- 172.21.15.64:0/2778553202 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f69580225e0 con 0x7f69681010c0 2023-09-13T03:12:13.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.711+0000 7f69657fa700 1 --2- 172.21.15.64:0/2778553202 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f69540a0710 0x7f69540a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:13.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.711+0000 7f69657fa700 1 -- 172.21.15.64:0/2778553202 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f69580cdb00 con 0x7f69681010c0 2023-09-13T03:12:13.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.712+0000 7f695ffff700 1 --2- 172.21.15.64:0/2778553202 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f69540a0710 0x7f69540a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:13.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.712+0000 7f695ffff700 1 --2- 172.21.15.64:0/2778553202 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f69540a0710 0x7f69540a2bd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6950011490 tx=0x7f6950009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:13.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.714+0000 7f69657fa700 1 -- 172.21.15.64:0/2778553202 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6958097530 con 0x7f69681010c0 2023-09-13T03:12:13.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:13 smithi170 ceph-mon[134328]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-13T03:12:13.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:13 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/2942475027' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T03:12:13.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:13 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/3482432989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:12:13.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:13 smithi064 ceph-mon[97675]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-13T03:12:13.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:13 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2942475027' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T03:12:13.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:13 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/3482432989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:12:13.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.934+0000 7f696dbff700 1 -- 172.21.15.64:0/2778553202 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f696810c040 con 0x7f69681010c0 2023-09-13T03:12:13.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.936+0000 7f69657fa700 1 -- 172.21.15.64:0/2778553202 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22473 (secure 0 0 0) 0x7f695809ab50 con 0x7f69681010c0 2023-09-13T03:12:13.937 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:12:13.937 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":85,"fsid":"f8dbced4-51e1-11ee-9ab7-7b867c8bd7da","created":"2023-09-13T03:03:24.167202+0000","modified":"2023-09-13T03:11:24.258871+0000","last_up_change":"2023-09-13T03:10:22.503529+0000","last_in_change":"2023-09-13T03:10:05.678720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T03:06:58.987479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-13T03:11:08.589202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-13T03:11:09.924659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3997c114-241b-48c4-8feb-653c99209470","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6803","nonce":3940892135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6805","nonce":3940892135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6809","nonce":3940892135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6807","nonce":3940892135}]},"public_addr":"172.21.15.64:6803/3940892135","cluster_addr":"172.21.15.64:6805/3940892135","heartbeat_back_addr":"172.21.15.64:6809/3940892135","heartbeat_front_addr":"172.21.15.64:6807/3940892135","state":["exists","up"]},{"osd":1,"uuid":"94cfae85-322e-4ed2-a6e6-e40349bb14bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6801","nonce":4188149333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6803","nonce":4188149333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6807","nonce":4188149333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6805","nonce":4188149333}]},"public_addr":"172.21.15.142:6801/4188149333","cluster_addr":"172.21.15.142:6803/4188149333","heartbeat_back_addr":"172.21.15.142:6807/4188149333","heartbeat_front_addr":"172.21.15.142:6805/4188149333","state":["exists","up"]},{"osd":2,"uuid":"b3aae17e-96db-4008-8c27-f733c46b30ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6801","nonce":2406669338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6803","nonce":2406669338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6807","nonce":2406669338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6805","nonce":2406669338}]},"public_addr":"172.21.15.170:6801/2406669338","cluster_addr":"172.21.15.170:6803/2406669338","heartbeat_back_addr":"172.21.15.170:6807/2406669338","heartbeat_front_addr":"172.21.15.170:6805/2406669338","state":["exists","up"]},{"osd":3,"uuid":"7fa8d8cb-2ab7-48a7-8b49-5e2903d6779f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6811","nonce":3558318707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6813","nonce":3558318707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6817","nonce":3558318707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6815","nonce":3558318707}]},"public_addr":"172.21.15.64:6811/3558318707","cluster_addr":"172.21.15.64:6813/3558318707","heartbeat_back_addr":"172.21.15.64:6817/3558318707","heartbeat_front_addr":"172.21.15.64:6815/3558318707","state":["exists","up"]},{"osd":4,"uuid":"8c7a9c9f-f919-48d3-9db2-214a25bb5603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6809","nonce":1811142576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6811","nonce":1811142576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6815","nonce":1811142576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6813","nonce":1811142576}]},"public_addr":"172.21.15.142:6809/1811142576","cluster_addr":"172.21.15.142:6811/1811142576","heartbeat_back_addr":"172.21.15.142:6815/1811142576","heartbeat_front_addr":"172.21.15.142:6813/1811142576","state":["exists","up"]},{"osd":5,"uuid":"0c6101c6-2065-4022-a751-d3f216d7f478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6809","nonce":2903841453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6811","nonce":2903841453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6815","nonce":2903841453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6813","nonce":2903841453}]},"public_addr":"172.21.15.170:6809/2903841453","cluster_addr":"172.21.15.170:6811/2903841453","heartbeat_back_addr":"172.21.15.170:6815/2903841453","heartbeat_front_addr":"172.21.15.170:6813/2903841453","state":["exists","up"]},{"osd":6,"uuid":"5cf3e52e-d155-41b1-9194-db047d26a953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6819","nonce":1436584490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6821","nonce":1436584490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6825","nonce":1436584490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6823","nonce":1436584490}]},"public_addr":"172.21.15.64:6819/1436584490","cluster_addr":"172.21.15.64:6821/1436584490","heartbeat_back_addr":"172.21.15.64:6825/1436584490","heartbeat_front_addr":"172.21.15.64:6823/1436584490","state":["exists","up"]},{"osd":7,"uuid":"e97323e3-badd-41a7-af9e-e78a81a64046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6817","nonce":3726793534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6819","nonce":3726793534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6823","nonce":3726793534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6821","nonce":3726793534}]},"public_addr":"172.21.15.142:6817/3726793534","cluster_addr":"172.21.15.142:6819/3726793534","heartbeat_back_addr":"172.21.15.142:6823/3726793534","heartbeat_front_addr":"172.21.15.142:6821/3726793534","state":["exists","up"]},{"osd":8,"uuid":"83d2835c-ed26-493d-9222-c2894ec0e071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6817","nonce":4272513873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6819","nonce":4272513873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6823","nonce":4272513873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6821","nonce":4272513873}]},"public_addr":"172.21.15.170:6817/4272513873","cluster_addr":"172.21.15.170:6819/4272513873","heartbeat_back_addr":"172.21.15.170:6823/4272513873","heartbeat_front_addr":"172.21.15.170:6821/4272513873","state":["exists","up"]},{"osd":9,"uuid":"cb936bdd-1bec-4236-aa2e-a24adcac11e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6827","nonce":2641175211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6829","nonce":2641175211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6833","nonce":2641175211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6831","nonce":2641175211}]},"public_addr":"172.21.15.64:6827/2641175211","cluster_addr":"172.21.15.64:6829/2641175211","heartbeat_back_addr":"172.21.15.64:6833/2641175211","heartbeat_front_addr":"172.21.15.64:6831/2641175211","state":["exists","up"]},{"osd":10,"uuid":"85512a1c-6171-43d4-9c15-c588a8b71be0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6825","nonce":936499217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6827","nonce":936499217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6831","nonce":936499217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6829","nonce":936499217}]},"public_addr":"172.21.15.142:6825/936499217","cluster_addr":"172.21.15.142:6827/936499217","heartbeat_back_addr":"172.21.15.142:6831/936499217","heartbeat_front_addr":"172.21.15.142:6829/936499217","state":["exists","up"]},{"osd":11,"uuid":"9b8417dc-5d6c-4917-94a9-b032c5bc33d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6825","nonce":3605017060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6827","nonce":3605017060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6831","nonce":3605017060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6829","nonce":3605017060}]},"public_addr":"172.21.15.170:6825/3605017060","cluster_addr":"172.21.15.170:6827/3605017060","heartbeat_back_addr":"172.21.15.170:6831/3605017060","heartbeat_front_addr":"172.21.15.170:6829/3605017060","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:14.229044+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:36.147777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:56.655298+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:07:23.107741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:07:44.663770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:05.577432+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:31.168031+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:52.791055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:09:13.658546+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:09:39.150360+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:10:00.601735+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:10:21.407555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6800/2955227347":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/1373223118":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/271194613":"2023-09-14T03:04:22.355817+0000","172.21.15.64:6801/3998091347":"2023-09-14T03:03:56.442541+0000","172.21.15.64:6800/3998091347":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/920818572":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/735789005":"2023-09-14T03:03:56.442541+0000","172.21.15.64:6801/2955227347":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/1975923178":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/1500151178":"2023-09-14T03:04:22.355817+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T03:12:13.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.939+0000 7f696dbff700 1 -- 172.21.15.64:0/2778553202 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f69540a0710 msgr2=0x7f69540a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:13.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.939+0000 7f696dbff700 1 --2- 172.21.15.64:0/2778553202 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f69540a0710 0x7f69540a2bd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6950011490 tx=0x7f6950009260 comp rx=0 tx=0).stop 2023-09-13T03:12:13.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.939+0000 7f696dbff700 1 -- 172.21.15.64:0/2778553202 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69681010c0 msgr2=0x7f69681984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:13.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.939+0000 7f696dbff700 1 --2- 172.21.15.64:0/2778553202 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69681010c0 0x7f69681984d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f695800b680 tx=0x7f6958005740 comp rx=0 tx=0).stop 2023-09-13T03:12:13.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.939+0000 7f696dbff700 1 -- 172.21.15.64:0/2778553202 shutdown_connections 2023-09-13T03:12:13.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.939+0000 7f696dbff700 1 --2- 172.21.15.64:0/2778553202 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f69540a0710 0x7f69540a2bd0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:13.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.939+0000 7f696dbff700 1 --2- 172.21.15.64:0/2778553202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6968106320 0x7f696819fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:13.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.939+0000 7f696dbff700 1 --2- 172.21.15.64:0/2778553202 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69681010c0 0x7f69681984d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:13.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.939+0000 7f696dbff700 1 --2- 172.21.15.64:0/2778553202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69681039f0 0x7f6968198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:13.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.939+0000 7f696dbff700 1 -- 172.21.15.64:0/2778553202 >> 172.21.15.64:0/2778553202 conn(0x7f69680faa30 msgr2=0x7f69681025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:13.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.939+0000 7f696dbff700 1 -- 172.21.15.64:0/2778553202 shutdown_connections 2023-09-13T03:12:13.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:13.939+0000 7f696dbff700 1 -- 172.21.15.64:0/2778553202 wait complete. 2023-09-13T03:12:13.960 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-13T03:06:58.987479+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-13T03:11:08.589202+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-13T03:11:09.924659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-13T03:12:13.961 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-13T03:12:13.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:13 smithi142 ceph-mon[134082]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-13T03:12:13.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:13 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/2942475027' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T03:12:13.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:13 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/3482432989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:12:14.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.183+0000 7f8db7bff700 1 -- 172.21.15.64:0/3668117140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8db00fee60 msgr2=0x7f8db0101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:14.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.183+0000 7f8db7bff700 1 --2- 172.21.15.64:0/3668117140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8db00fee60 0x7f8db0101290 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8dac009990 tx=0x7f8dac009ca0 comp rx=0 tx=0).stop 2023-09-13T03:12:14.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.184+0000 7f8db7bff700 1 -- 172.21.15.64:0/3668117140 shutdown_connections 2023-09-13T03:12:14.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.184+0000 7f8db7bff700 1 --2- 172.21.15.64:0/3668117140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8db01041a0 0x7f8db0104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:14.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.184+0000 7f8db7bff700 1 --2- 172.21.15.64:0/3668117140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8db00fee60 0x7f8db0101290 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:14.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.184+0000 7f8db7bff700 1 --2- 172.21.15.64:0/3668117140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8db01017d0 0x7f8db0103c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:14.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.184+0000 7f8db7bff700 1 -- 172.21.15.64:0/3668117140 >> 172.21.15.64:0/3668117140 conn(0x7f8db00faa60 msgr2=0x7f8db00fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:14.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.184+0000 7f8db7bff700 1 -- 172.21.15.64:0/3668117140 shutdown_connections 2023-09-13T03:12:14.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.184+0000 7f8db7bff700 1 -- 172.21.15.64:0/3668117140 wait complete. 2023-09-13T03:12:14.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.185+0000 7f8db7bff700 1 Processor -- start 2023-09-13T03:12:14.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.186+0000 7f8db7bff700 1 -- start start 2023-09-13T03:12:14.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.186+0000 7f8db7bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8db00fee60 0x7f8db019c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:14.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.186+0000 7f8db7bff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8db01017d0 0x7f8db019ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:14.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.186+0000 7f8db7bff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8db01041a0 0x7f8db01a3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:14.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.186+0000 7f8db7bff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db01a44b0 con 0x7f8db01041a0 2023-09-13T03:12:14.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.186+0000 7f8db7bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db01a45f0 con 0x7f8db00fee60 2023-09-13T03:12:14.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.186+0000 7f8db7bff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db01a4780 con 0x7f8db01017d0 2023-09-13T03:12:14.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.186+0000 7f8db619c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8db01041a0 0x7f8db01a3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:14.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.186+0000 7f8db599b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8db00fee60 0x7f8db019c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:14.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.186+0000 7f8db519a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8db01017d0 0x7f8db019ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:14.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.186+0000 7f8db599b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8db00fee60 0x7f8db019c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:59890/0 (socket says 172.21.15.64:59890) 2023-09-13T03:12:14.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.186+0000 7f8db519a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8db01017d0 0x7f8db019ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:45770/0 (socket says 172.21.15.64:45770) 2023-09-13T03:12:14.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.186+0000 7f8db599b700 1 -- 172.21.15.64:0/2473918003 learned_addr learned my addr 172.21.15.64:0/2473918003 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:14.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.187+0000 7f8db619c700 1 -- 172.21.15.64:0/2473918003 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8db01017d0 msgr2=0x7f8db019ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:14.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.187+0000 7f8db619c700 1 --2- 172.21.15.64:0/2473918003 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8db01017d0 0x7f8db019ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:14.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.187+0000 7f8db619c700 1 -- 172.21.15.64:0/2473918003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8db00fee60 msgr2=0x7f8db019c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:14.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.187+0000 7f8db619c700 1 --2- 172.21.15.64:0/2473918003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8db00fee60 0x7f8db019c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:14.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.187+0000 7f8db619c700 1 -- 172.21.15.64:0/2473918003 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dac0095f0 con 0x7f8db01041a0 2023-09-13T03:12:14.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.187+0000 7f8db519a700 1 --2- 172.21.15.64:0/2473918003 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8db01017d0 0x7f8db019ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:14.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.187+0000 7f8db619c700 1 --2- 172.21.15.64:0/2473918003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8db01041a0 0x7f8db01a3ee0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8da800efc0 tx=0x7f8da800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:14.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.187+0000 7f8da6ffd700 1 -- 172.21.15.64:0/2473918003 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da8018070 con 0x7f8db01041a0 2023-09-13T03:12:14.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.187+0000 7f8da6ffd700 1 -- 172.21.15.64:0/2473918003 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8da800f040 con 0x7f8db01041a0 2023-09-13T03:12:14.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.187+0000 7f8db7bff700 1 -- 172.21.15.64:0/2473918003 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8db01a4a00 con 0x7f8db01041a0 2023-09-13T03:12:14.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.187+0000 7f8da6ffd700 1 -- 172.21.15.64:0/2473918003 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da800cd40 con 0x7f8db01041a0 2023-09-13T03:12:14.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.187+0000 7f8db7bff700 1 -- 172.21.15.64:0/2473918003 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8db01a4f10 con 0x7f8db01041a0 2023-09-13T03:12:14.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.189+0000 7f8db7bff700 1 -- 172.21.15.64:0/2473918003 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8db00fc5e0 con 0x7f8db01041a0 2023-09-13T03:12:14.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.190+0000 7f8da6ffd700 1 -- 172.21.15.64:0/2473918003 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f8da8003a00 con 0x7f8db01041a0 2023-09-13T03:12:14.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.193+0000 7f8da6ffd700 1 --2- 172.21.15.64:0/2473918003 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f8d9c0a06d0 0x7f8d9c0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:14.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.193+0000 7f8da6ffd700 1 -- 172.21.15.64:0/2473918003 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f8da80cb8c0 con 0x7f8db01041a0 2023-09-13T03:12:14.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.193+0000 7f8db599b700 1 --2- 172.21.15.64:0/2473918003 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f8d9c0a06d0 0x7f8d9c0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:14.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.194+0000 7f8db599b700 1 --2- 172.21.15.64:0/2473918003 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f8d9c0a06d0 0x7f8d9c0a2b90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8dac006010 tx=0x7f8dac00b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:14.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.196+0000 7f8da6ffd700 1 -- 172.21.15.64:0/2473918003 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f8da80952f0 con 0x7f8db01041a0 2023-09-13T03:12:14.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.416+0000 7f8db7bff700 1 -- 172.21.15.64:0/2473918003 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8db0112640 con 0x7f8db01041a0 2023-09-13T03:12:14.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.417+0000 7f8da6ffd700 1 -- 172.21.15.64:0/2473918003 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8da80b8020 con 0x7f8db01041a0 2023-09-13T03:12:14.418 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-09-13T03:12:14.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.419+0000 7f8db7bff700 1 -- 172.21.15.64:0/2473918003 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f8d9c0a06d0 msgr2=0x7f8d9c0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:14.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.419+0000 7f8db7bff700 1 --2- 172.21.15.64:0/2473918003 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f8d9c0a06d0 0x7f8d9c0a2b90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8dac006010 tx=0x7f8dac00b260 comp rx=0 tx=0).stop 2023-09-13T03:12:14.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.419+0000 7f8db7bff700 1 -- 172.21.15.64:0/2473918003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8db01041a0 msgr2=0x7f8db01a3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:14.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.419+0000 7f8db7bff700 1 --2- 172.21.15.64:0/2473918003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8db01041a0 0x7f8db01a3ee0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8da800efc0 tx=0x7f8da800c3a0 comp rx=0 tx=0).stop 2023-09-13T03:12:14.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.419+0000 7f8db7bff700 1 -- 172.21.15.64:0/2473918003 shutdown_connections 2023-09-13T03:12:14.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.419+0000 7f8db7bff700 1 --2- 172.21.15.64:0/2473918003 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f8d9c0a06d0 0x7f8d9c0a2b90 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:14.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.419+0000 7f8db7bff700 1 --2- 172.21.15.64:0/2473918003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8db00fee60 0x7f8db019c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:14.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.419+0000 7f8db7bff700 1 --2- 172.21.15.64:0/2473918003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8db01041a0 0x7f8db01a3ee0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:14.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.419+0000 7f8db7bff700 1 --2- 172.21.15.64:0/2473918003 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8db01017d0 0x7f8db019ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:14.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.419+0000 7f8db7bff700 1 -- 172.21.15.64:0/2473918003 >> 172.21.15.64:0/2473918003 conn(0x7f8db00faa60 msgr2=0x7f8db01024e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:14.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.419+0000 7f8db7bff700 1 -- 172.21.15.64:0/2473918003 shutdown_connections 2023-09-13T03:12:14.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.419+0000 7f8db7bff700 1 -- 172.21.15.64:0/2473918003 wait complete. 2023-09-13T03:12:14.438 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-13T03:12:14.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.651+0000 7ff3439ff700 1 -- 172.21.15.64:0/1161936548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff33c1005f0 msgr2=0x7ff33c100a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:14.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.651+0000 7ff3439ff700 1 --2- 172.21.15.64:0/1161936548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff33c1005f0 0x7ff33c100a10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff3380099c0 tx=0x7ff338009cd0 comp rx=0 tx=0).stop 2023-09-13T03:12:14.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.651+0000 7ff3439ff700 1 -- 172.21.15.64:0/1161936548 shutdown_connections 2023-09-13T03:12:14.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.651+0000 7ff3439ff700 1 --2- 172.21.15.64:0/1161936548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff33c1005f0 0x7ff33c100a10 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:14.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.651+0000 7ff3439ff700 1 --2- 172.21.15.64:0/1161936548 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff33c102180 0x7ff33c108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:14.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.651+0000 7ff3439ff700 1 --2- 172.21.15.64:0/1161936548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff33c1017e0 0x7ff33c101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:14.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.651+0000 7ff3439ff700 1 -- 172.21.15.64:0/1161936548 >> 172.21.15.64:0/1161936548 conn(0x7ff33c0fbb40 msgr2=0x7ff33c0fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:14.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.651+0000 7ff3439ff700 1 -- 172.21.15.64:0/1161936548 shutdown_connections 2023-09-13T03:12:14.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.651+0000 7ff3439ff700 1 -- 172.21.15.64:0/1161936548 wait complete. 2023-09-13T03:12:14.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.653+0000 7ff3439ff700 1 Processor -- start 2023-09-13T03:12:14.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.653+0000 7ff3439ff700 1 -- start start 2023-09-13T03:12:14.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.653+0000 7ff3439ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff33c1005f0 0x7ff33c196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:14.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.654+0000 7ff3439ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff33c1017e0 0x7ff33c1968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:14.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.654+0000 7ff3439ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff33c102180 0x7ff33c19d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:14.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.654+0000 7ff3439ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c19df00 con 0x7ff33c102180 2023-09-13T03:12:14.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.654+0000 7ff3439ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c19e040 con 0x7ff33c1005f0 2023-09-13T03:12:14.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.654+0000 7ff3439ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c19e1d0 con 0x7ff33c1017e0 2023-09-13T03:12:14.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.654+0000 7ff34179b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff33c1005f0 0x7ff33c196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:14.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.654+0000 7ff34179b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff33c1005f0 0x7ff33c196380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:59896/0 (socket says 172.21.15.64:59896) 2023-09-13T03:12:14.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.654+0000 7ff34179b700 1 -- 172.21.15.64:0/1718455658 learned_addr learned my addr 172.21.15.64:0/1718455658 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:14.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.654+0000 7ff340f9a700 1 --2- 172.21.15.64:0/1718455658 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff33c1017e0 0x7ff33c1968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:14.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.654+0000 7ff341f9c700 1 --2- 172.21.15.64:0/1718455658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff33c102180 0x7ff33c19d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:14.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.654+0000 7ff34179b700 1 -- 172.21.15.64:0/1718455658 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff33c1017e0 msgr2=0x7ff33c1968c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:14.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.654+0000 7ff34179b700 1 --2- 172.21.15.64:0/1718455658 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff33c1017e0 0x7ff33c1968c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:14.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.654+0000 7ff34179b700 1 -- 172.21.15.64:0/1718455658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff33c102180 msgr2=0x7ff33c19d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:14.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.654+0000 7ff34179b700 1 --2- 172.21.15.64:0/1718455658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff33c102180 0x7ff33c19d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:14.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.654+0000 7ff34179b700 1 -- 172.21.15.64:0/1718455658 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3380095f0 con 0x7ff33c1005f0 2023-09-13T03:12:14.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.655+0000 7ff34179b700 1 --2- 172.21.15.64:0/1718455658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff33c1005f0 0x7ff33c196380 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff3380098e0 tx=0x7ff33800f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:14.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.656+0000 7ff3327fc700 1 -- 172.21.15.64:0/1718455658 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33801d070 con 0x7ff33c1005f0 2023-09-13T03:12:14.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.656+0000 7ff3327fc700 1 -- 172.21.15.64:0/1718455658 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff338005050 con 0x7ff33c1005f0 2023-09-13T03:12:14.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.656+0000 7ff3327fc700 1 -- 172.21.15.64:0/1718455658 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff338004b40 con 0x7ff33c1005f0 2023-09-13T03:12:14.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.656+0000 7ff3439ff700 1 -- 172.21.15.64:0/1718455658 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff33c19e450 con 0x7ff33c1005f0 2023-09-13T03:12:14.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.656+0000 7ff3439ff700 1 -- 172.21.15.64:0/1718455658 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff33c19e820 con 0x7ff33c1005f0 2023-09-13T03:12:14.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.657+0000 7ff3439ff700 1 -- 172.21.15.64:0/1718455658 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff33c10be60 con 0x7ff33c1005f0 2023-09-13T03:12:14.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.659+0000 7ff3327fc700 1 -- 172.21.15.64:0/1718455658 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7ff338004ce0 con 0x7ff33c1005f0 2023-09-13T03:12:14.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.660+0000 7ff3327fc700 1 --2- 172.21.15.64:0/1718455658 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ff3280a06d0 0x7ff3280a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:14.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.661+0000 7ff3327fc700 1 -- 172.21.15.64:0/1718455658 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff3380cd070 con 0x7ff33c1005f0 2023-09-13T03:12:14.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.661+0000 7ff340f9a700 1 --2- 172.21.15.64:0/1718455658 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ff3280a06d0 0x7ff3280a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:14.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.661+0000 7ff340f9a700 1 --2- 172.21.15.64:0/1718455658 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ff3280a06d0 0x7ff3280a2b90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff32c009c60 tx=0x7ff32c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:14.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.662+0000 7ff3327fc700 1 -- 172.21.15.64:0/1718455658 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ff338096b00 con 0x7ff33c1005f0 2023-09-13T03:12:14.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.895+0000 7ff3439ff700 1 -- 172.21.15.64:0/1718455658 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff33c1075b0 con 0x7ff33c1005f0 2023-09-13T03:12:14.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.895+0000 7ff3327fc700 1 -- 172.21.15.64:0/1718455658 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff338026020 con 0x7ff33c1005f0 2023-09-13T03:12:14.896 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-09-13T03:12:14.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.898+0000 7ff3439ff700 1 -- 172.21.15.64:0/1718455658 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ff3280a06d0 msgr2=0x7ff3280a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:14.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.898+0000 7ff3439ff700 1 --2- 172.21.15.64:0/1718455658 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ff3280a06d0 0x7ff3280a2b90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff32c009c60 tx=0x7ff32c009260 comp rx=0 tx=0).stop 2023-09-13T03:12:14.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.898+0000 7ff3439ff700 1 -- 172.21.15.64:0/1718455658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff33c1005f0 msgr2=0x7ff33c196380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:14.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.898+0000 7ff3439ff700 1 --2- 172.21.15.64:0/1718455658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff33c1005f0 0x7ff33c196380 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff3380098e0 tx=0x7ff33800f690 comp rx=0 tx=0).stop 2023-09-13T03:12:14.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.898+0000 7ff3439ff700 1 -- 172.21.15.64:0/1718455658 shutdown_connections 2023-09-13T03:12:14.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.898+0000 7ff3439ff700 1 --2- 172.21.15.64:0/1718455658 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ff3280a06d0 0x7ff3280a2b90 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:14.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.898+0000 7ff3439ff700 1 --2- 172.21.15.64:0/1718455658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff33c1005f0 0x7ff33c196380 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:14.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.898+0000 7ff3439ff700 1 --2- 172.21.15.64:0/1718455658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff33c102180 0x7ff33c19d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:14.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.898+0000 7ff3439ff700 1 --2- 172.21.15.64:0/1718455658 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff33c1017e0 0x7ff33c1968c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:14.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.898+0000 7ff3439ff700 1 -- 172.21.15.64:0/1718455658 >> 172.21.15.64:0/1718455658 conn(0x7ff33c0fbb40 msgr2=0x7ff33c0fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:14.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.898+0000 7ff3439ff700 1 -- 172.21.15.64:0/1718455658 shutdown_connections 2023-09-13T03:12:14.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:14.898+0000 7ff3439ff700 1 -- 172.21.15.64:0/1718455658 wait complete. 2023-09-13T03:12:14.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:14 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/2778553202' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:12:14.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:14 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/2473918003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T03:12:14.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:14 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2778553202' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:12:14.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:14 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2473918003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T03:12:14.920 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-13T03:12:14.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:14 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/2778553202' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:12:14.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:14 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/2473918003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T03:12:15.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.140+0000 7f6f6bdff700 1 -- 172.21.15.64:0/2690580496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f64101850 msgr2=0x7f6f64101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:15.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.140+0000 7f6f6bdff700 1 --2- 172.21.15.64:0/2690580496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f64101850 0x7f6f64101cb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6f540099c0 tx=0x7f6f54009cd0 comp rx=0 tx=0).stop 2023-09-13T03:12:15.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.140+0000 7f6f6bdff700 1 -- 172.21.15.64:0/2690580496 shutdown_connections 2023-09-13T03:12:15.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.140+0000 7f6f6bdff700 1 --2- 172.21.15.64:0/2690580496 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f641021f0 0x7f6f64108aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:15.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.140+0000 7f6f6bdff700 1 --2- 172.21.15.64:0/2690580496 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f64100660 0x7f6f64100a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:15.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.140+0000 7f6f6bdff700 1 --2- 172.21.15.64:0/2690580496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f64101850 0x7f6f64101cb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:15.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.140+0000 7f6f6bdff700 1 -- 172.21.15.64:0/2690580496 >> 172.21.15.64:0/2690580496 conn(0x7f6f640fbbd0 msgr2=0x7f6f640fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:15.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.140+0000 7f6f6bdff700 1 -- 172.21.15.64:0/2690580496 shutdown_connections 2023-09-13T03:12:15.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.140+0000 7f6f6bdff700 1 -- 172.21.15.64:0/2690580496 wait complete. 2023-09-13T03:12:15.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.141+0000 7f6f6bdff700 1 Processor -- start 2023-09-13T03:12:15.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.141+0000 7f6f6bdff700 1 -- start start 2023-09-13T03:12:15.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.141+0000 7f6f6bdff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f64100660 0x7f6f641963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:15.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.141+0000 7f6f6bdff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f64101850 0x7f6f64196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:15.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.141+0000 7f6f6bdff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f641021f0 0x7f6f6419d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:15.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.142+0000 7f6f6bdff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6419df40 con 0x7f6f64101850 2023-09-13T03:12:15.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.142+0000 7f6f6bdff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6419e080 con 0x7f6f641021f0 2023-09-13T03:12:15.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.142+0000 7f6f6bdff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6419e210 con 0x7f6f64100660 2023-09-13T03:12:15.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.142+0000 7f6f6a39c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f641021f0 0x7f6f6419d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:15.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.142+0000 7f6f6939a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f64101850 0x7f6f64196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:15.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.142+0000 7f6f6a39c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f641021f0 0x7f6f6419d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:59926/0 (socket says 172.21.15.64:59926) 2023-09-13T03:12:15.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.142+0000 7f6f6a39c700 1 -- 172.21.15.64:0/1927313882 learned_addr learned my addr 172.21.15.64:0/1927313882 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:15.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.142+0000 7f6f69b9b700 1 --2- 172.21.15.64:0/1927313882 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f64100660 0x7f6f641963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:15.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.142+0000 7f6f6a39c700 1 -- 172.21.15.64:0/1927313882 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f64100660 msgr2=0x7f6f641963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:15.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.142+0000 7f6f6a39c700 1 --2- 172.21.15.64:0/1927313882 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f64100660 0x7f6f641963c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:15.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.142+0000 7f6f6a39c700 1 -- 172.21.15.64:0/1927313882 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f64101850 msgr2=0x7f6f64196900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:15.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.142+0000 7f6f6a39c700 1 --2- 172.21.15.64:0/1927313882 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f64101850 0x7f6f64196900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:15.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.142+0000 7f6f6a39c700 1 -- 172.21.15.64:0/1927313882 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f540095f0 con 0x7f6f641021f0 2023-09-13T03:12:15.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.142+0000 7f6f6a39c700 1 --2- 172.21.15.64:0/1927313882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f641021f0 0x7f6f6419d970 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c00ef30 tx=0x7f6f5c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:15.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.143+0000 7f6f5affd700 1 -- 172.21.15.64:0/1927313882 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c018070 con 0x7f6f641021f0 2023-09-13T03:12:15.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.143+0000 7f6f5affd700 1 -- 172.21.15.64:0/1927313882 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f5c00f040 con 0x7f6f641021f0 2023-09-13T03:12:15.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.143+0000 7f6f6bdff700 1 -- 172.21.15.64:0/1927313882 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f64072830 con 0x7f6f641021f0 2023-09-13T03:12:15.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.143+0000 7f6f5affd700 1 -- 172.21.15.64:0/1927313882 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c00cd00 con 0x7f6f641021f0 2023-09-13T03:12:15.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.143+0000 7f6f6bdff700 1 -- 172.21.15.64:0/1927313882 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f64072d90 con 0x7f6f641021f0 2023-09-13T03:12:15.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.144+0000 7f6f6bdff700 1 -- 172.21.15.64:0/1927313882 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f64107cf0 con 0x7f6f641021f0 2023-09-13T03:12:15.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.146+0000 7f6f5affd700 1 -- 172.21.15.64:0/1927313882 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f6f5c003a00 con 0x7f6f641021f0 2023-09-13T03:12:15.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.148+0000 7f6f5affd700 1 --2- 172.21.15.64:0/1927313882 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6f500a0750 0x7f6f500a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:15.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.148+0000 7f6f5affd700 1 -- 172.21.15.64:0/1927313882 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f6f5c0cb740 con 0x7f6f641021f0 2023-09-13T03:12:15.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.148+0000 7f6f69b9b700 1 --2- 172.21.15.64:0/1927313882 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6f500a0750 0x7f6f500a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:15.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.148+0000 7f6f69b9b700 1 --2- 172.21.15.64:0/1927313882 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6f500a0750 0x7f6f500a2c10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6f641016b0 tx=0x7f6f60009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:15.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.152+0000 7f6f5affd700 1 -- 172.21.15.64:0/1927313882 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6f5c095170 con 0x7f6f641021f0 2023-09-13T03:12:15.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.378+0000 7f6f6bdff700 1 -- 172.21.15.64:0/1927313882 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6f6410c000 con 0x7f6f641021f0 2023-09-13T03:12:15.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.378+0000 7f6f5affd700 1 -- 172.21.15.64:0/1927313882 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6f5c098790 con 0x7f6f641021f0 2023-09-13T03:12:15.379 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-09-13T03:12:15.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.381+0000 7f6f6bdff700 1 -- 172.21.15.64:0/1927313882 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6f500a0750 msgr2=0x7f6f500a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:15.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.381+0000 7f6f6bdff700 1 --2- 172.21.15.64:0/1927313882 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6f500a0750 0x7f6f500a2c10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6f641016b0 tx=0x7f6f60009260 comp rx=0 tx=0).stop 2023-09-13T03:12:15.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.381+0000 7f6f6bdff700 1 -- 172.21.15.64:0/1927313882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f641021f0 msgr2=0x7f6f6419d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:15.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.381+0000 7f6f6bdff700 1 --2- 172.21.15.64:0/1927313882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f641021f0 0x7f6f6419d970 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c00ef30 tx=0x7f6f5c00c3a0 comp rx=0 tx=0).stop 2023-09-13T03:12:15.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.381+0000 7f6f6bdff700 1 -- 172.21.15.64:0/1927313882 shutdown_connections 2023-09-13T03:12:15.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.381+0000 7f6f6bdff700 1 --2- 172.21.15.64:0/1927313882 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6f500a0750 0x7f6f500a2c10 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:15.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.381+0000 7f6f6bdff700 1 --2- 172.21.15.64:0/1927313882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f641021f0 0x7f6f6419d970 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:15.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.381+0000 7f6f6bdff700 1 --2- 172.21.15.64:0/1927313882 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f64100660 0x7f6f641963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:15.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.381+0000 7f6f6bdff700 1 --2- 172.21.15.64:0/1927313882 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f64101850 0x7f6f64196900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:15.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.381+0000 7f6f6bdff700 1 -- 172.21.15.64:0/1927313882 >> 172.21.15.64:0/1927313882 conn(0x7f6f640fbbd0 msgr2=0x7f6f64104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:15.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.381+0000 7f6f6bdff700 1 -- 172.21.15.64:0/1927313882 shutdown_connections 2023-09-13T03:12:15.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.381+0000 7f6f6bdff700 1 -- 172.21.15.64:0/1927313882 wait complete. 2023-09-13T03:12:15.401 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T03:12:15.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.614+0000 7f7bdb1ff700 1 -- 172.21.15.64:0/2553536153 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7bd4073060 msgr2=0x7f7bd40734e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:15.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.614+0000 7f7bdb1ff700 1 --2- 172.21.15.64:0/2553536153 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7bd4073060 0x7f7bd40734e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0009990 tx=0x7f7bc0009ca0 comp rx=0 tx=0).stop 2023-09-13T03:12:15.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.614+0000 7f7bdb1ff700 1 -- 172.21.15.64:0/2553536153 shutdown_connections 2023-09-13T03:12:15.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.614+0000 7f7bdb1ff700 1 --2- 172.21.15.64:0/2553536153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bd410cbe0 0x7f7bd410efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:15.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.614+0000 7f7bdb1ff700 1 --2- 172.21.15.64:0/2553536153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bd4074640 0x7f7bd4072a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:15.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.614+0000 7f7bdb1ff700 1 --2- 172.21.15.64:0/2553536153 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7bd4073060 0x7f7bd40734e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:15.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.614+0000 7f7bdb1ff700 1 -- 172.21.15.64:0/2553536153 >> 172.21.15.64:0/2553536153 conn(0x7f7bd40fbb40 msgr2=0x7f7bd40fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:15.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.614+0000 7f7bdb1ff700 1 -- 172.21.15.64:0/2553536153 shutdown_connections 2023-09-13T03:12:15.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.614+0000 7f7bdb1ff700 1 -- 172.21.15.64:0/2553536153 wait complete. 2023-09-13T03:12:15.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.615+0000 7f7bdb1ff700 1 Processor -- start 2023-09-13T03:12:15.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.615+0000 7f7bdb1ff700 1 -- start start 2023-09-13T03:12:15.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.616+0000 7f7bdb1ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7bd4073060 0x7f7bd419c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:15.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.616+0000 7f7bdb1ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bd4074640 0x7f7bd419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:15.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.616+0000 7f7bdb1ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bd410cbe0 0x7f7bd41a3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:15.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.616+0000 7f7bdb1ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd41a4480 con 0x7f7bd4074640 2023-09-13T03:12:15.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.616+0000 7f7bdb1ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd41a45c0 con 0x7f7bd410cbe0 2023-09-13T03:12:15.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.616+0000 7f7bdb1ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd41a4750 con 0x7f7bd4073060 2023-09-13T03:12:15.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.616+0000 7f7bd3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bd4074640 0x7f7bd419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:15.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.616+0000 7f7bd8f9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7bd4073060 0x7f7bd419c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:15.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.616+0000 7f7bd8f9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7bd4073060 0x7f7bd419c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:45840/0 (socket says 172.21.15.64:45840) 2023-09-13T03:12:15.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.616+0000 7f7bd8f9b700 1 -- 172.21.15.64:0/2390192303 learned_addr learned my addr 172.21.15.64:0/2390192303 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:15.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.616+0000 7f7bd979c700 1 --2- 172.21.15.64:0/2390192303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bd410cbe0 0x7f7bd41a3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:15.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.617+0000 7f7bd8f9b700 1 -- 172.21.15.64:0/2390192303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bd410cbe0 msgr2=0x7f7bd41a3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:15.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.617+0000 7f7bd8f9b700 1 --2- 172.21.15.64:0/2390192303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bd410cbe0 0x7f7bd41a3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:15.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.617+0000 7f7bd8f9b700 1 -- 172.21.15.64:0/2390192303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bd4074640 msgr2=0x7f7bd419ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:15.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.617+0000 7f7bd8f9b700 1 --2- 172.21.15.64:0/2390192303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bd4074640 0x7f7bd419ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:15.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.617+0000 7f7bd8f9b700 1 -- 172.21.15.64:0/2390192303 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc00095f0 con 0x7f7bd4073060 2023-09-13T03:12:15.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.617+0000 7f7bd8f9b700 1 --2- 172.21.15.64:0/2390192303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7bd4073060 0x7f7bd419c900 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800e9a0 tx=0x7f7bc800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:15.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.617+0000 7f7bd1ffb700 1 -- 172.21.15.64:0/2390192303 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8009790 con 0x7f7bd4073060 2023-09-13T03:12:15.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.617+0000 7f7bd1ffb700 1 -- 172.21.15.64:0/2390192303 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bc8010430 con 0x7f7bd4073060 2023-09-13T03:12:15.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.617+0000 7f7bd1ffb700 1 -- 172.21.15.64:0/2390192303 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8018680 con 0x7f7bd4073060 2023-09-13T03:12:15.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.617+0000 7f7bdb1ff700 1 -- 172.21.15.64:0/2390192303 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bd41a4a30 con 0x7f7bd4073060 2023-09-13T03:12:15.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.617+0000 7f7bdb1ff700 1 -- 172.21.15.64:0/2390192303 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bd41a4f40 con 0x7f7bd4073060 2023-09-13T03:12:15.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.618+0000 7f7bdb1ff700 1 -- 172.21.15.64:0/2390192303 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bd4102730 con 0x7f7bd4073060 2023-09-13T03:12:15.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.621+0000 7f7bd1ffb700 1 -- 172.21.15.64:0/2390192303 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f7bc80105a0 con 0x7f7bd4073060 2023-09-13T03:12:15.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.623+0000 7f7bd1ffb700 1 --2- 172.21.15.64:0/2390192303 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f7bc40a0790 0x7f7bc40a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:15.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.623+0000 7f7bd1ffb700 1 -- 172.21.15.64:0/2390192303 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f7bc8014070 con 0x7f7bd4073060 2023-09-13T03:12:15.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.624+0000 7f7bd3fff700 1 --2- 172.21.15.64:0/2390192303 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f7bc40a0790 0x7f7bc40a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:15.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.624+0000 7f7bd3fff700 1 --2- 172.21.15.64:0/2390192303 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f7bc40a0790 0x7f7bc40a2c50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7bc000b2e0 tx=0x7f7bc0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:15.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.627+0000 7f7bd1ffb700 1 -- 172.21.15.64:0/2390192303 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f7bc8095b70 con 0x7f7bd4073060 2023-09-13T03:12:15.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:15 smithi064 ceph-mon[97675]: pgmap v311: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-13T03:12:15.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:15 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/1718455658' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T03:12:15.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:15 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:12:15.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:15 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:12:15.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:15 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:12:15.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:15 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:12:15.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:15 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/1927313882' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T03:12:15.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.896+0000 7f7bdb1ff700 1 -- 172.21.15.64:0/2390192303 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7bd4112450 con 0x7f7bd4073060 2023-09-13T03:12:15.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.897+0000 7f7bd1ffb700 1 -- 172.21.15.64:0/2390192303 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11366 (secure 0 0 0) 0x7f7bc8099190 con 0x7f7bd4073060 2023-09-13T03:12:15.898 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:12:15.898 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2234182985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2234182985},{"type":"v1","addr":"172.21.15.64:6835","nonce":2234182985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4053841873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4053841873},{"type":"v1","addr":"172.21.15.64:6837","nonce":4053841873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-13T03:11:10.490260+0000","modified":"2023-09-13T03:11:18.230740+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24686,"mds_1":24649,"mds_2":24635,"mds_3":24634,"mds_4":24622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6839/99523374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":99523374},{"type":"v1","addr":"172.21.15.64:6839","nonce":99523374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14649":{"gid":14649,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6841/1866582875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":1866582875},{"type":"v1","addr":"172.21.15.64:6841","nonce":1866582875}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/101906999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":101906999},{"type":"v1","addr":"172.21.15.142:6833","nonce":101906999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24574":{"gid":24574,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/1897366227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1897366227},{"type":"v1","addr":"172.21.15.170:6833","nonce":1897366227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24611":{"gid":24611,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2167779918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2167779918},{"type":"v1","addr":"172.21.15.142:6835","nonce":2167779918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24622":{"gid":24622,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6835/1493371121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1493371121},{"type":"v1","addr":"172.21.15.170:6835","nonce":1493371121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24634":{"gid":24634,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.170:6837/1726587275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":1726587275},{"type":"v1","addr":"172.21.15.170:6837","nonce":1726587275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24635":{"gid":24635,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.142:6837/741173268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":741173268},{"type":"v1","addr":"172.21.15.142:6837","nonce":741173268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24649":{"gid":24649,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.170:6839/2189703596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2189703596},{"type":"v1","addr":"172.21.15.170:6839","nonce":2189703596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24686":{"gid":24686,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/3645998616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645998616},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645998616}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-13T03:12:15.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.900+0000 7f7bdb1ff700 1 -- 172.21.15.64:0/2390192303 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f7bc40a0790 msgr2=0x7f7bc40a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:15.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.900+0000 7f7bdb1ff700 1 --2- 172.21.15.64:0/2390192303 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f7bc40a0790 0x7f7bc40a2c50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7bc000b2e0 tx=0x7f7bc0005fb0 comp rx=0 tx=0).stop 2023-09-13T03:12:15.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.900+0000 7f7bdb1ff700 1 -- 172.21.15.64:0/2390192303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7bd4073060 msgr2=0x7f7bd419c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:15.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.900+0000 7f7bdb1ff700 1 --2- 172.21.15.64:0/2390192303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7bd4073060 0x7f7bd419c900 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800e9a0 tx=0x7f7bc800ecb0 comp rx=0 tx=0).stop 2023-09-13T03:12:15.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.900+0000 7f7bdb1ff700 1 -- 172.21.15.64:0/2390192303 shutdown_connections 2023-09-13T03:12:15.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.900+0000 7f7bdb1ff700 1 --2- 172.21.15.64:0/2390192303 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f7bc40a0790 0x7f7bc40a2c50 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:15.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.900+0000 7f7bdb1ff700 1 --2- 172.21.15.64:0/2390192303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bd410cbe0 0x7f7bd41a3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:15.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.900+0000 7f7bdb1ff700 1 --2- 172.21.15.64:0/2390192303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7bd4073060 0x7f7bd419c900 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:15.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.900+0000 7f7bdb1ff700 1 --2- 172.21.15.64:0/2390192303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bd4074640 0x7f7bd419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:15.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.900+0000 7f7bdb1ff700 1 -- 172.21.15.64:0/2390192303 >> 172.21.15.64:0/2390192303 conn(0x7f7bd40fbb40 msgr2=0x7f7bd40fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:15.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.900+0000 7f7bdb1ff700 1 -- 172.21.15.64:0/2390192303 shutdown_connections 2023-09-13T03:12:15.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:15.900+0000 7f7bdb1ff700 1 -- 172.21.15.64:0/2390192303 wait complete. 2023-09-13T03:12:15.903 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 26 2023-09-13T03:12:15.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:15 smithi170 ceph-mon[134328]: pgmap v311: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-13T03:12:15.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:15 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/1718455658' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T03:12:15.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:15 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:12:15.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:15 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:12:15.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:15 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:12:15.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:15 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:12:15.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:15 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/1927313882' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T03:12:15.919 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-13T03:12:15.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:15 smithi142 ceph-mon[134082]: pgmap v311: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-13T03:12:15.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:15 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/1718455658' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T03:12:15.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:15 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:12:15.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:15 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:12:15.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:15 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:12:15.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:15 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:12:15.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:15 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/1927313882' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T03:12:16.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.129+0000 7f1d7bdff700 1 -- 172.21.15.64:0/1843700180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d74074640 msgr2=0x7f1d74072a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:16.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.129+0000 7f1d7bdff700 1 --2- 172.21.15.64:0/1843700180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d74074640 0x7f1d74072a90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1d70009990 tx=0x7f1d70009ca0 comp rx=0 tx=0).stop 2023-09-13T03:12:16.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.129+0000 7f1d7bdff700 1 -- 172.21.15.64:0/1843700180 shutdown_connections 2023-09-13T03:12:16.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.129+0000 7f1d7bdff700 1 --2- 172.21.15.64:0/1843700180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d7410cbe0 0x7f1d7410efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:16.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.129+0000 7f1d7bdff700 1 --2- 172.21.15.64:0/1843700180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d74074640 0x7f1d74072a90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:16.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.129+0000 7f1d7bdff700 1 --2- 172.21.15.64:0/1843700180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d74073060 0x7f1d740734e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:16.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.129+0000 7f1d7bdff700 1 -- 172.21.15.64:0/1843700180 >> 172.21.15.64:0/1843700180 conn(0x7f1d740fbb40 msgr2=0x7f1d740fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:16.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.129+0000 7f1d7bdff700 1 -- 172.21.15.64:0/1843700180 shutdown_connections 2023-09-13T03:12:16.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.129+0000 7f1d7bdff700 1 -- 172.21.15.64:0/1843700180 wait complete. 2023-09-13T03:12:16.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.130+0000 7f1d7bdff700 1 Processor -- start 2023-09-13T03:12:16.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.131+0000 7f1d7bdff700 1 -- start start 2023-09-13T03:12:16.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.131+0000 7f1d7bdff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d74073060 0x7f1d7419c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:16.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.131+0000 7f1d7bdff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d74074640 0x7f1d7419ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:16.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.131+0000 7f1d7bdff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d7410cbe0 0x7f1d741a3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:16.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.131+0000 7f1d7bdff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d741a44b0 con 0x7f1d74073060 2023-09-13T03:12:16.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.131+0000 7f1d7bdff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d741a45f0 con 0x7f1d7410cbe0 2023-09-13T03:12:16.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.131+0000 7f1d7bdff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d741a4780 con 0x7f1d74074640 2023-09-13T03:12:16.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.131+0000 7f1d79b9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d74073060 0x7f1d7419c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:16.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.131+0000 7f1d79b9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d74073060 0x7f1d7419c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57754/0 (socket says 172.21.15.64:57754) 2023-09-13T03:12:16.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.131+0000 7f1d79b9b700 1 -- 172.21.15.64:0/1684347523 learned_addr learned my addr 172.21.15.64:0/1684347523 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:16.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.132+0000 7f1d7939a700 1 --2- 172.21.15.64:0/1684347523 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d74074640 0x7f1d7419ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:16.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.132+0000 7f1d7a39c700 1 --2- 172.21.15.64:0/1684347523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d7410cbe0 0x7f1d741a3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:16.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.132+0000 7f1d79b9b700 1 -- 172.21.15.64:0/1684347523 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d74074640 msgr2=0x7f1d7419ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:16.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.132+0000 7f1d79b9b700 1 --2- 172.21.15.64:0/1684347523 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d74074640 0x7f1d7419ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:16.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.132+0000 7f1d79b9b700 1 -- 172.21.15.64:0/1684347523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d7410cbe0 msgr2=0x7f1d741a3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:16.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.132+0000 7f1d79b9b700 1 --2- 172.21.15.64:0/1684347523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d7410cbe0 0x7f1d741a3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:16.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.132+0000 7f1d79b9b700 1 -- 172.21.15.64:0/1684347523 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d700095f0 con 0x7f1d74073060 2023-09-13T03:12:16.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.132+0000 7f1d79b9b700 1 --2- 172.21.15.64:0/1684347523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d74073060 0x7f1d7419c930 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1d70004730 tx=0x7f1d70004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:16.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.132+0000 7f1d6affd700 1 -- 172.21.15.64:0/1684347523 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7001d070 con 0x7f1d74073060 2023-09-13T03:12:16.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.132+0000 7f1d6affd700 1 -- 172.21.15.64:0/1684347523 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d70004c90 con 0x7f1d74073060 2023-09-13T03:12:16.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.132+0000 7f1d7bdff700 1 -- 172.21.15.64:0/1684347523 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d741a4a00 con 0x7f1d74073060 2023-09-13T03:12:16.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.133+0000 7f1d6affd700 1 -- 172.21.15.64:0/1684347523 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d700054b0 con 0x7f1d74073060 2023-09-13T03:12:16.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.133+0000 7f1d7bdff700 1 -- 172.21.15.64:0/1684347523 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d741a4dc0 con 0x7f1d74073060 2023-09-13T03:12:16.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.134+0000 7f1d7bdff700 1 -- 172.21.15.64:0/1684347523 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d74102730 con 0x7f1d74073060 2023-09-13T03:12:16.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.137+0000 7f1d6affd700 1 -- 172.21.15.64:0/1684347523 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f1d7000fbc0 con 0x7f1d74073060 2023-09-13T03:12:16.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.139+0000 7f1d6affd700 1 --2- 172.21.15.64:0/1684347523 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f1d600a0710 0x7f1d600a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:16.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.139+0000 7f1d6affd700 1 -- 172.21.15.64:0/1684347523 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f1d700cd400 con 0x7f1d74073060 2023-09-13T03:12:16.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.139+0000 7f1d7939a700 1 --2- 172.21.15.64:0/1684347523 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f1d600a0710 0x7f1d600a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:16.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.140+0000 7f1d7939a700 1 --2- 172.21.15.64:0/1684347523 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f1d600a0710 0x7f1d600a2bd0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1d74074360 tx=0x7f1d64009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:16.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.141+0000 7f1d6affd700 1 -- 172.21.15.64:0/1684347523 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1d70096e90 con 0x7f1d74073060 2023-09-13T03:12:16.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.357+0000 7f1d7bdff700 1 -- 172.21.15.64:0/1684347523 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d741126c0 con 0x7f1d74073060 2023-09-13T03:12:16.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.358+0000 7f1d6affd700 1 -- 172.21.15.64:0/1684347523 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22473 (secure 0 0 0) 0x7f1d7009a4b0 con 0x7f1d74073060 2023-09-13T03:12:16.359 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:12:16.360 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":85,"fsid":"f8dbced4-51e1-11ee-9ab7-7b867c8bd7da","created":"2023-09-13T03:03:24.167202+0000","modified":"2023-09-13T03:11:24.258871+0000","last_up_change":"2023-09-13T03:10:22.503529+0000","last_in_change":"2023-09-13T03:10:05.678720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T03:06:58.987479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-13T03:11:08.589202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-13T03:11:09.924659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3997c114-241b-48c4-8feb-653c99209470","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6803","nonce":3940892135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6805","nonce":3940892135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6809","nonce":3940892135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6807","nonce":3940892135}]},"public_addr":"172.21.15.64:6803/3940892135","cluster_addr":"172.21.15.64:6805/3940892135","heartbeat_back_addr":"172.21.15.64:6809/3940892135","heartbeat_front_addr":"172.21.15.64:6807/3940892135","state":["exists","up"]},{"osd":1,"uuid":"94cfae85-322e-4ed2-a6e6-e40349bb14bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6801","nonce":4188149333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6803","nonce":4188149333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6807","nonce":4188149333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6805","nonce":4188149333}]},"public_addr":"172.21.15.142:6801/4188149333","cluster_addr":"172.21.15.142:6803/4188149333","heartbeat_back_addr":"172.21.15.142:6807/4188149333","heartbeat_front_addr":"172.21.15.142:6805/4188149333","state":["exists","up"]},{"osd":2,"uuid":"b3aae17e-96db-4008-8c27-f733c46b30ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6801","nonce":2406669338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6803","nonce":2406669338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6807","nonce":2406669338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6805","nonce":2406669338}]},"public_addr":"172.21.15.170:6801/2406669338","cluster_addr":"172.21.15.170:6803/2406669338","heartbeat_back_addr":"172.21.15.170:6807/2406669338","heartbeat_front_addr":"172.21.15.170:6805/2406669338","state":["exists","up"]},{"osd":3,"uuid":"7fa8d8cb-2ab7-48a7-8b49-5e2903d6779f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6811","nonce":3558318707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6813","nonce":3558318707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6817","nonce":3558318707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6815","nonce":3558318707}]},"public_addr":"172.21.15.64:6811/3558318707","cluster_addr":"172.21.15.64:6813/3558318707","heartbeat_back_addr":"172.21.15.64:6817/3558318707","heartbeat_front_addr":"172.21.15.64:6815/3558318707","state":["exists","up"]},{"osd":4,"uuid":"8c7a9c9f-f919-48d3-9db2-214a25bb5603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6809","nonce":1811142576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6811","nonce":1811142576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6815","nonce":1811142576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6813","nonce":1811142576}]},"public_addr":"172.21.15.142:6809/1811142576","cluster_addr":"172.21.15.142:6811/1811142576","heartbeat_back_addr":"172.21.15.142:6815/1811142576","heartbeat_front_addr":"172.21.15.142:6813/1811142576","state":["exists","up"]},{"osd":5,"uuid":"0c6101c6-2065-4022-a751-d3f216d7f478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6809","nonce":2903841453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6811","nonce":2903841453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6815","nonce":2903841453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6813","nonce":2903841453}]},"public_addr":"172.21.15.170:6809/2903841453","cluster_addr":"172.21.15.170:6811/2903841453","heartbeat_back_addr":"172.21.15.170:6815/2903841453","heartbeat_front_addr":"172.21.15.170:6813/2903841453","state":["exists","up"]},{"osd":6,"uuid":"5cf3e52e-d155-41b1-9194-db047d26a953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6819","nonce":1436584490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6821","nonce":1436584490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6825","nonce":1436584490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6823","nonce":1436584490}]},"public_addr":"172.21.15.64:6819/1436584490","cluster_addr":"172.21.15.64:6821/1436584490","heartbeat_back_addr":"172.21.15.64:6825/1436584490","heartbeat_front_addr":"172.21.15.64:6823/1436584490","state":["exists","up"]},{"osd":7,"uuid":"e97323e3-badd-41a7-af9e-e78a81a64046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6817","nonce":3726793534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6819","nonce":3726793534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6823","nonce":3726793534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6821","nonce":3726793534}]},"public_addr":"172.21.15.142:6817/3726793534","cluster_addr":"172.21.15.142:6819/3726793534","heartbeat_back_addr":"172.21.15.142:6823/3726793534","heartbeat_front_addr":"172.21.15.142:6821/3726793534","state":["exists","up"]},{"osd":8,"uuid":"83d2835c-ed26-493d-9222-c2894ec0e071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6817","nonce":4272513873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6819","nonce":4272513873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6823","nonce":4272513873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6821","nonce":4272513873}]},"public_addr":"172.21.15.170:6817/4272513873","cluster_addr":"172.21.15.170:6819/4272513873","heartbeat_back_addr":"172.21.15.170:6823/4272513873","heartbeat_front_addr":"172.21.15.170:6821/4272513873","state":["exists","up"]},{"osd":9,"uuid":"cb936bdd-1bec-4236-aa2e-a24adcac11e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6827","nonce":2641175211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6829","nonce":2641175211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6833","nonce":2641175211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6831","nonce":2641175211}]},"public_addr":"172.21.15.64:6827/2641175211","cluster_addr":"172.21.15.64:6829/2641175211","heartbeat_back_addr":"172.21.15.64:6833/2641175211","heartbeat_front_addr":"172.21.15.64:6831/2641175211","state":["exists","up"]},{"osd":10,"uuid":"85512a1c-6171-43d4-9c15-c588a8b71be0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6825","nonce":936499217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6827","nonce":936499217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6831","nonce":936499217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6829","nonce":936499217}]},"public_addr":"172.21.15.142:6825/936499217","cluster_addr":"172.21.15.142:6827/936499217","heartbeat_back_addr":"172.21.15.142:6831/936499217","heartbeat_front_addr":"172.21.15.142:6829/936499217","state":["exists","up"]},{"osd":11,"uuid":"9b8417dc-5d6c-4917-94a9-b032c5bc33d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6825","nonce":3605017060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6827","nonce":3605017060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6831","nonce":3605017060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6829","nonce":3605017060}]},"public_addr":"172.21.15.170:6825/3605017060","cluster_addr":"172.21.15.170:6827/3605017060","heartbeat_back_addr":"172.21.15.170:6831/3605017060","heartbeat_front_addr":"172.21.15.170:6829/3605017060","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:14.229044+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:36.147777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:56.655298+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:07:23.107741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:07:44.663770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:05.577432+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:31.168031+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:52.791055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:09:13.658546+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:09:39.150360+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:10:00.601735+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:10:21.407555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6800/2955227347":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/1373223118":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/271194613":"2023-09-14T03:04:22.355817+0000","172.21.15.64:6801/3998091347":"2023-09-14T03:03:56.442541+0000","172.21.15.64:6800/3998091347":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/920818572":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/735789005":"2023-09-14T03:03:56.442541+0000","172.21.15.64:6801/2955227347":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/1975923178":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/1500151178":"2023-09-14T03:04:22.355817+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T03:12:16.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.361+0000 7f1d7bdff700 1 -- 172.21.15.64:0/1684347523 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f1d600a0710 msgr2=0x7f1d600a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:16.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.361+0000 7f1d7bdff700 1 --2- 172.21.15.64:0/1684347523 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f1d600a0710 0x7f1d600a2bd0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1d74074360 tx=0x7f1d64009260 comp rx=0 tx=0).stop 2023-09-13T03:12:16.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.361+0000 7f1d7bdff700 1 -- 172.21.15.64:0/1684347523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d74073060 msgr2=0x7f1d7419c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:16.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.361+0000 7f1d7bdff700 1 --2- 172.21.15.64:0/1684347523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d74073060 0x7f1d7419c930 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1d70004730 tx=0x7f1d70004760 comp rx=0 tx=0).stop 2023-09-13T03:12:16.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.361+0000 7f1d7bdff700 1 -- 172.21.15.64:0/1684347523 shutdown_connections 2023-09-13T03:12:16.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.361+0000 7f1d7bdff700 1 --2- 172.21.15.64:0/1684347523 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f1d600a0710 0x7f1d600a2bd0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:16.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.361+0000 7f1d7bdff700 1 --2- 172.21.15.64:0/1684347523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d7410cbe0 0x7f1d741a3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:16.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.361+0000 7f1d7bdff700 1 --2- 172.21.15.64:0/1684347523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d74073060 0x7f1d7419c930 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:16.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.361+0000 7f1d7bdff700 1 --2- 172.21.15.64:0/1684347523 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d74074640 0x7f1d7419ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:16.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.361+0000 7f1d7bdff700 1 -- 172.21.15.64:0/1684347523 >> 172.21.15.64:0/1684347523 conn(0x7f1d740fbb40 msgr2=0x7f1d740fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:16.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.361+0000 7f1d7bdff700 1 -- 172.21.15.64:0/1684347523 shutdown_connections 2023-09-13T03:12:16.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.361+0000 7f1d7bdff700 1 -- 172.21.15.64:0/1684347523 wait complete. 2023-09-13T03:12:16.387 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T03:12:16.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.595+0000 7fa09c5ff700 1 -- 172.21.15.64:0/3431944316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa09410cbd0 msgr2=0x7fa09410efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:16.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.595+0000 7fa09c5ff700 1 --2- 172.21.15.64:0/3431944316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa09410cbd0 0x7fa09410efc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa08c00b0a0 tx=0x7fa08c00b3b0 comp rx=0 tx=0).stop 2023-09-13T03:12:16.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.595+0000 7fa09c5ff700 1 -- 172.21.15.64:0/3431944316 shutdown_connections 2023-09-13T03:12:16.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.595+0000 7fa09c5ff700 1 --2- 172.21.15.64:0/3431944316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa094074640 0x7fa094072a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:16.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.595+0000 7fa09c5ff700 1 --2- 172.21.15.64:0/3431944316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa09410cbd0 0x7fa09410efc0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:16.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.595+0000 7fa09c5ff700 1 --2- 172.21.15.64:0/3431944316 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa094073060 0x7fa0940734e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:16.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.595+0000 7fa09c5ff700 1 -- 172.21.15.64:0/3431944316 >> 172.21.15.64:0/3431944316 conn(0x7fa0940fbb30 msgr2=0x7fa0940fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:16.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.595+0000 7fa09c5ff700 1 -- 172.21.15.64:0/3431944316 shutdown_connections 2023-09-13T03:12:16.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.595+0000 7fa09c5ff700 1 -- 172.21.15.64:0/3431944316 wait complete. 2023-09-13T03:12:16.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.596+0000 7fa09c5ff700 1 Processor -- start 2023-09-13T03:12:16.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.596+0000 7fa09c5ff700 1 -- start start 2023-09-13T03:12:16.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.596+0000 7fa09c5ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa094073060 0x7fa09419ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:16.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.596+0000 7fa09c5ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa094074640 0x7fa09419cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:16.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.596+0000 7fa09c5ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa09410cbd0 0x7fa0941a3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:16.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.596+0000 7fa09c5ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0941a4590 con 0x7fa094073060 2023-09-13T03:12:16.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.597+0000 7fa09c5ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0941a46d0 con 0x7fa094074640 2023-09-13T03:12:16.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.597+0000 7fa09c5ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0941a4860 con 0x7fa09410cbd0 2023-09-13T03:12:16.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.597+0000 7fa09a39b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa094073060 0x7fa09419ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:16.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.597+0000 7fa09a39b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa094073060 0x7fa09419ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57772/0 (socket says 172.21.15.64:57772) 2023-09-13T03:12:16.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.597+0000 7fa09a39b700 1 -- 172.21.15.64:0/160144837 learned_addr learned my addr 172.21.15.64:0/160144837 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:16.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.597+0000 7fa09ab9c700 1 --2- 172.21.15.64:0/160144837 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa09410cbd0 0x7fa0941a3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:16.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.597+0000 7fa099b9a700 1 --2- 172.21.15.64:0/160144837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa094074640 0x7fa09419cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:16.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.597+0000 7fa09a39b700 1 -- 172.21.15.64:0/160144837 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa09410cbd0 msgr2=0x7fa0941a3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:16.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.597+0000 7fa09a39b700 1 --2- 172.21.15.64:0/160144837 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa09410cbd0 0x7fa0941a3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:16.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.597+0000 7fa09a39b700 1 -- 172.21.15.64:0/160144837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa094074640 msgr2=0x7fa09419cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:16.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.597+0000 7fa09a39b700 1 --2- 172.21.15.64:0/160144837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa094074640 0x7fa09419cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:16.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.597+0000 7fa09a39b700 1 -- 172.21.15.64:0/160144837 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa08c009d00 con 0x7fa094073060 2023-09-13T03:12:16.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.597+0000 7fa09a39b700 1 --2- 172.21.15.64:0/160144837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa094073060 0x7fa09419ca10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa09000e9a0 tx=0x7fa09000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:16.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.598+0000 7fa08b7fe700 1 -- 172.21.15.64:0/160144837 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa090009790 con 0x7fa094073060 2023-09-13T03:12:16.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.598+0000 7fa09c5ff700 1 -- 172.21.15.64:0/160144837 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0941a4b40 con 0x7fa094073060 2023-09-13T03:12:16.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.598+0000 7fa08b7fe700 1 -- 172.21.15.64:0/160144837 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa090010430 con 0x7fa094073060 2023-09-13T03:12:16.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.598+0000 7fa08b7fe700 1 -- 172.21.15.64:0/160144837 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa090018680 con 0x7fa094073060 2023-09-13T03:12:16.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.598+0000 7fa09c5ff700 1 -- 172.21.15.64:0/160144837 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0941a5050 con 0x7fa094073060 2023-09-13T03:12:16.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.599+0000 7fa09c5ff700 1 -- 172.21.15.64:0/160144837 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa094102720 con 0x7fa094073060 2023-09-13T03:12:16.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.601+0000 7fa08b7fe700 1 -- 172.21.15.64:0/160144837 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fa0900105a0 con 0x7fa094073060 2023-09-13T03:12:16.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.603+0000 7fa08b7fe700 1 --2- 172.21.15.64:0/160144837 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fa0800a0710 0x7fa0800a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:16.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.603+0000 7fa08b7fe700 1 -- 172.21.15.64:0/160144837 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa090014070 con 0x7fa094073060 2023-09-13T03:12:16.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.603+0000 7fa099b9a700 1 --2- 172.21.15.64:0/160144837 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fa0800a0710 0x7fa0800a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:16.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.604+0000 7fa099b9a700 1 --2- 172.21.15.64:0/160144837 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fa0800a0710 0x7fa0800a2bd0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa094074360 tx=0x7fa084009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:16.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.611+0000 7fa08b7fe700 1 -- 172.21.15.64:0/160144837 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fa090051e10 con 0x7fa094073060 2023-09-13T03:12:16.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:16 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2390192303' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:12:16.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:16 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:12:16.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:16 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:12:16.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:16 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/1684347523' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:12:16.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.869+0000 7fa09c5ff700 1 -- 172.21.15.64:0/160144837 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa09402ce00 con 0x7fa094073060 2023-09-13T03:12:16.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.870+0000 7fa08b7fe700 1 -- 172.21.15.64:0/160144837 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11366 (secure 0 0 0) 0x7fa090095800 con 0x7fa094073060 2023-09-13T03:12:16.871 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:12:16.871 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2234182985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2234182985},{"type":"v1","addr":"172.21.15.64:6835","nonce":2234182985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4053841873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4053841873},{"type":"v1","addr":"172.21.15.64:6837","nonce":4053841873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-13T03:11:10.490260+0000","modified":"2023-09-13T03:11:18.230740+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24686,"mds_1":24649,"mds_2":24635,"mds_3":24634,"mds_4":24622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6839/99523374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":99523374},{"type":"v1","addr":"172.21.15.64:6839","nonce":99523374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14649":{"gid":14649,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6841/1866582875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":1866582875},{"type":"v1","addr":"172.21.15.64:6841","nonce":1866582875}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/101906999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":101906999},{"type":"v1","addr":"172.21.15.142:6833","nonce":101906999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24574":{"gid":24574,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/1897366227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1897366227},{"type":"v1","addr":"172.21.15.170:6833","nonce":1897366227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24611":{"gid":24611,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2167779918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2167779918},{"type":"v1","addr":"172.21.15.142:6835","nonce":2167779918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24622":{"gid":24622,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6835/1493371121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1493371121},{"type":"v1","addr":"172.21.15.170:6835","nonce":1493371121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24634":{"gid":24634,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.170:6837/1726587275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":1726587275},{"type":"v1","addr":"172.21.15.170:6837","nonce":1726587275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24635":{"gid":24635,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.142:6837/741173268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":741173268},{"type":"v1","addr":"172.21.15.142:6837","nonce":741173268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24649":{"gid":24649,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.170:6839/2189703596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2189703596},{"type":"v1","addr":"172.21.15.170:6839","nonce":2189703596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24686":{"gid":24686,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/3645998616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645998616},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645998616}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-13T03:12:16.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.873+0000 7fa09c5ff700 1 -- 172.21.15.64:0/160144837 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fa0800a0710 msgr2=0x7fa0800a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:16.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.873+0000 7fa09c5ff700 1 --2- 172.21.15.64:0/160144837 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fa0800a0710 0x7fa0800a2bd0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa094074360 tx=0x7fa084009260 comp rx=0 tx=0).stop 2023-09-13T03:12:16.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.873+0000 7fa09c5ff700 1 -- 172.21.15.64:0/160144837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa094073060 msgr2=0x7fa09419ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:16.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.873+0000 7fa09c5ff700 1 --2- 172.21.15.64:0/160144837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa094073060 0x7fa09419ca10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa09000e9a0 tx=0x7fa09000ecb0 comp rx=0 tx=0).stop 2023-09-13T03:12:16.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.873+0000 7fa09c5ff700 1 -- 172.21.15.64:0/160144837 shutdown_connections 2023-09-13T03:12:16.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.873+0000 7fa09c5ff700 1 --2- 172.21.15.64:0/160144837 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fa0800a0710 0x7fa0800a2bd0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:16.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.873+0000 7fa09c5ff700 1 --2- 172.21.15.64:0/160144837 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa09410cbd0 0x7fa0941a3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:16.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.873+0000 7fa09c5ff700 1 --2- 172.21.15.64:0/160144837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa094073060 0x7fa09419ca10 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:16.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.873+0000 7fa09c5ff700 1 --2- 172.21.15.64:0/160144837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa094074640 0x7fa09419cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:16.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.873+0000 7fa09c5ff700 1 -- 172.21.15.64:0/160144837 >> 172.21.15.64:0/160144837 conn(0x7fa0940fbb30 msgr2=0x7fa0940fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:16.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.873+0000 7fa09c5ff700 1 -- 172.21.15.64:0/160144837 shutdown_connections 2023-09-13T03:12:16.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:16.873+0000 7fa09c5ff700 1 -- 172.21.15.64:0/160144837 wait complete. 2023-09-13T03:12:16.876 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 26 2023-09-13T03:12:16.894 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-09-13T03:11:10.490260+0000', 'modified': '2023-09-13T03:11:18.230740+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24686, 'mds_1': 24649, 'mds_2': 24635, 'mds_3': 24634, 'mds_4': 24622}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14637': {'gid': 14637, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.64:6839/99523374', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6838', 'nonce': 99523374}, {'type': 'v1', 'addr': '172.21.15.64:6839', 'nonce': 99523374}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_14649': {'gid': 14649, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.64:6841/1866582875', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6840', 'nonce': 1866582875}, {'type': 'v1', 'addr': '172.21.15.64:6841', 'nonce': 1866582875}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24569': {'gid': 24569, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.142:6833/101906999', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6832', 'nonce': 101906999}, {'type': 'v1', 'addr': '172.21.15.142:6833', 'nonce': 101906999}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24574': {'gid': 24574, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.170:6833/1897366227', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6832', 'nonce': 1897366227}, {'type': 'v1', 'addr': '172.21.15.170:6833', 'nonce': 1897366227}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24611': {'gid': 24611, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.142:6835/2167779918', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6834', 'nonce': 2167779918}, {'type': 'v1', 'addr': '172.21.15.142:6835', 'nonce': 2167779918}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24622': {'gid': 24622, 'name': 'f', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.170:6835/1493371121', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6834', 'nonce': 1493371121}, {'type': 'v1', 'addr': '172.21.15.170:6835', 'nonce': 1493371121}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24634': {'gid': 24634, 'name': 'c', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.170:6837/1726587275', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6836', 'nonce': 1726587275}, {'type': 'v1', 'addr': '172.21.15.170:6837', 'nonce': 1726587275}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24635': {'gid': 24635, 'name': 'h', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.142:6837/741173268', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6836', 'nonce': 741173268}, {'type': 'v1', 'addr': '172.21.15.142:6837', 'nonce': 741173268}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24649': {'gid': 24649, 'name': 'i', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.170:6839/2189703596', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6838', 'nonce': 2189703596}, {'type': 'v1', 'addr': '172.21.15.170:6839', 'nonce': 2189703596}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24686': {'gid': 24686, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.142:6839/3645998616', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6838', 'nonce': 3645998616}, {'type': 'v1', 'addr': '172.21.15.142:6839', 'nonce': 3645998616}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-13T03:12:16.894 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-09-13T03:12:16.894 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2023-09-13T03:12:16.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:16 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/2390192303' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:12:16.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:16 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:12:16.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:16 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:12:16.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:16 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/1684347523' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:12:16.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:16 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/2390192303' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:12:16.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:16 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:12:16.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:16 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:12:16.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:16 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/1684347523' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:12:17.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.100+0000 7fa9cd3ff700 1 -- 172.21.15.64:0/2834727225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9c8106390 msgr2=0x7fa9c8108780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:17.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.100+0000 7fa9cd3ff700 1 --2- 172.21.15.64:0/2834727225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9c8106390 0x7fa9c8108780 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa9c000b0a0 tx=0x7fa9c000b3b0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.101+0000 7fa9cd3ff700 1 -- 172.21.15.64:0/2834727225 shutdown_connections 2023-09-13T03:12:17.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.101+0000 7fa9cd3ff700 1 --2- 172.21.15.64:0/2834727225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9c8106390 0x7fa9c8108780 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.101+0000 7fa9cd3ff700 1 --2- 172.21.15.64:0/2834727225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9c8101130 0x7fa9c8103520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.101+0000 7fa9cd3ff700 1 --2- 172.21.15.64:0/2834727225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9c8103a60 0x7fa9c8105e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.101+0000 7fa9cd3ff700 1 -- 172.21.15.64:0/2834727225 >> 172.21.15.64:0/2834727225 conn(0x7fa9c80faa60 msgr2=0x7fa9c80fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:17.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.101+0000 7fa9cd3ff700 1 -- 172.21.15.64:0/2834727225 shutdown_connections 2023-09-13T03:12:17.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.101+0000 7fa9cd3ff700 1 -- 172.21.15.64:0/2834727225 wait complete. 2023-09-13T03:12:17.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.102+0000 7fa9cd3ff700 1 Processor -- start 2023-09-13T03:12:17.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.102+0000 7fa9cd3ff700 1 -- start start 2023-09-13T03:12:17.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.102+0000 7fa9cd3ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9c8101130 0x7fa9c8100aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.102+0000 7fa9cd3ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9c8103a60 0x7fa9c80ff0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.102+0000 7fa9cd3ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9c8106390 0x7fa9c80ff620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.102+0000 7fa9cd3ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c80ffdc0 con 0x7fa9c8103a60 2023-09-13T03:12:17.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.102+0000 7fa9cd3ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c80fff30 con 0x7fa9c8101130 2023-09-13T03:12:17.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.102+0000 7fa9cd3ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c81a0430 con 0x7fa9c8106390 2023-09-13T03:12:17.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.102+0000 7fa9c67fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9c8103a60 0x7fa9c80ff0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.102+0000 7fa9c67fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9c8103a60 0x7fa9c80ff0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57792/0 (socket says 172.21.15.64:57792) 2023-09-13T03:12:17.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.102+0000 7fa9c67fc700 1 -- 172.21.15.64:0/1867234685 learned_addr learned my addr 172.21.15.64:0/1867234685 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:17.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.102+0000 7fa9c6ffd700 1 --2- 172.21.15.64:0/1867234685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9c8101130 0x7fa9c8100aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.103+0000 7fa9c77fe700 1 --2- 172.21.15.64:0/1867234685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9c8106390 0x7fa9c80ff620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.103+0000 7fa9c67fc700 1 -- 172.21.15.64:0/1867234685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9c8106390 msgr2=0x7fa9c80ff620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:17.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.103+0000 7fa9c67fc700 1 --2- 172.21.15.64:0/1867234685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9c8106390 0x7fa9c80ff620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.103+0000 7fa9c67fc700 1 -- 172.21.15.64:0/1867234685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9c8101130 msgr2=0x7fa9c8100aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:17.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.103+0000 7fa9c67fc700 1 --2- 172.21.15.64:0/1867234685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9c8101130 0x7fa9c8100aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.103+0000 7fa9c67fc700 1 -- 172.21.15.64:0/1867234685 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c0009d00 con 0x7fa9c8103a60 2023-09-13T03:12:17.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.103+0000 7fa9c6ffd700 1 --2- 172.21.15.64:0/1867234685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9c8101130 0x7fa9c8100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:17.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.103+0000 7fa9c67fc700 1 --2- 172.21.15.64:0/1867234685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9c8103a60 0x7fa9c80ff0e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc00e9a0 tx=0x7fa9bc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:17.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.103+0000 7fa9affff700 1 -- 172.21.15.64:0/1867234685 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc009790 con 0x7fa9c8103a60 2023-09-13T03:12:17.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.103+0000 7fa9cd3ff700 1 -- 172.21.15.64:0/1867234685 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9c81a0710 con 0x7fa9c8103a60 2023-09-13T03:12:17.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.103+0000 7fa9affff700 1 -- 172.21.15.64:0/1867234685 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9bc010430 con 0x7fa9c8103a60 2023-09-13T03:12:17.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.103+0000 7fa9affff700 1 -- 172.21.15.64:0/1867234685 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc018680 con 0x7fa9c8103a60 2023-09-13T03:12:17.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.104+0000 7fa9cd3ff700 1 -- 172.21.15.64:0/1867234685 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9c81a0c20 con 0x7fa9c8103a60 2023-09-13T03:12:17.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.107+0000 7fa9affff700 1 -- 172.21.15.64:0/1867234685 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fa9bc0105a0 con 0x7fa9c8103a60 2023-09-13T03:12:17.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.108+0000 7fa9affff700 1 --2- 172.21.15.64:0/1867234685 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fa9b00a06d0 0x7fa9b00a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.108+0000 7fa9e0c79b80 1 Processor -- start 2023-09-13T03:12:17.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.108+0000 7fa9affff700 1 -- 172.21.15.64:0/1867234685 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa9bc014070 con 0x7fa9c8103a60 2023-09-13T03:12:17.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.109+0000 7fa9c77fe700 1 --2- 172.21.15.64:0/1867234685 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fa9b00a06d0 0x7fa9b00a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.109+0000 7fa9e0c79b80 1 -- start start 2023-09-13T03:12:17.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.109+0000 7fa9e0c79b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a08d488bb0 0x55a08d488fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.109+0000 7fa9e0c79b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a08d489550 0x55a08d4877b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.109+0000 7fa9e0c79b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a08d487db0 0x55a08d488260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.109+0000 7fa9e0c79b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a08d3e06e0 con 0x55a08d487db0 2023-09-13T03:12:17.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.109+0000 7fa9e0c79b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a08d3de000 con 0x55a08d489550 2023-09-13T03:12:17.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.109+0000 7fa9e0c79b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a08d3d98f0 con 0x55a08d488bb0 2023-09-13T03:12:17.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.109+0000 7fa9c77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a08d487db0 0x55a08d488260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.109+0000 7fa9c77fe700 1 --2- 172.21.15.64:0/1867234685 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fa9b00a06d0 0x7fa9b00a2b90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8101100 tx=0x7fa9c000b720 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:17.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.109+0000 7fa9c77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a08d487db0 0x55a08d488260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57806/0 (socket says 172.21.15.64:57806) 2023-09-13T03:12:17.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.109+0000 7fa9c77fe700 1 -- 172.21.15.64:0/1929801360 learned_addr learned my addr 172.21.15.64:0/1929801360 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:17.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.109+0000 7fa9c6ffd700 1 --2- 172.21.15.64:0/1929801360 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a08d488bb0 0x55a08d488fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.109+0000 7fa9c67fc700 1 --2- 172.21.15.64:0/1929801360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a08d489550 0x55a08d4877b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.110+0000 7fa9c77fe700 1 -- 172.21.15.64:0/1929801360 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a08d488bb0 msgr2=0x55a08d488fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:17.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.110+0000 7fa9c77fe700 1 --2- 172.21.15.64:0/1929801360 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a08d488bb0 0x55a08d488fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.110+0000 7fa9c77fe700 1 -- 172.21.15.64:0/1929801360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a08d489550 msgr2=0x55a08d4877b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:17.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.110+0000 7fa9c77fe700 1 --2- 172.21.15.64:0/1929801360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a08d489550 0x55a08d4877b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.110+0000 7fa9c77fe700 1 -- 172.21.15.64:0/1929801360 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a08d205d90 con 0x55a08d487db0 2023-09-13T03:12:17.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.110+0000 7fa9c77fe700 1 --2- 172.21.15.64:0/1929801360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a08d487db0 0x55a08d488260 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0015000 tx=0x7fa9c001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d73e5cd4a18b0033 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:17.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.110+0000 7fa9ad7fa700 1 -- 172.21.15.64:0/1929801360 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c000e040 con 0x55a08d487db0 2023-09-13T03:12:17.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.110+0000 7fa9ad7fa700 1 -- 172.21.15.64:0/1929801360 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9c0003e90 con 0x55a08d487db0 2023-09-13T03:12:17.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.110+0000 7fa9ad7fa700 1 -- 172.21.15.64:0/1929801360 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c001c7a0 con 0x55a08d487db0 2023-09-13T03:12:17.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.111+0000 7fa9e0c79b80 1 -- 172.21.15.64:0/1929801360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a08d487db0 msgr2=0x55a08d488260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:17.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.111+0000 7fa9e0c79b80 1 --2- 172.21.15.64:0/1929801360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a08d487db0 0x55a08d488260 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0015000 tx=0x7fa9c001c000 comp rx=0 tx=0).stop 2023-09-13T03:12:17.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.111+0000 7fa9e0c79b80 1 -- 172.21.15.64:0/1929801360 shutdown_connections 2023-09-13T03:12:17.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.111+0000 7fa9e0c79b80 1 --2- 172.21.15.64:0/1929801360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a08d487db0 0x55a08d488260 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.111+0000 7fa9e0c79b80 1 --2- 172.21.15.64:0/1929801360 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a08d488bb0 0x55a08d488fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.111+0000 7fa9e0c79b80 1 --2- 172.21.15.64:0/1929801360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a08d489550 0x55a08d4877b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.111+0000 7fa9e0c79b80 1 -- 172.21.15.64:0/1929801360 >> 172.21.15.64:0/1929801360 conn(0x55a08d3fc040 msgr2=0x55a08d4826d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:17.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.111+0000 7fa9e0c79b80 1 -- 172.21.15.64:0/1929801360 shutdown_connections 2023-09-13T03:12:17.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.111+0000 7fa9e0c79b80 1 -- 172.21.15.64:0/1929801360 wait complete. 2023-09-13T03:12:17.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.112+0000 7fa9e0c79b80 1 -- start start 2023-09-13T03:12:17.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.112+0000 7fa9e0c79b80 1 Processor -- start 2023-09-13T03:12:17.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.113+0000 7fa9e0c79b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a08d487db0 0x55a08d4914c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.113+0000 7fa9e0c79b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a08d488bb0 0x55a08d491a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.113+0000 7fa9e0c79b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a08d489550 0x55a08d498a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.113+0000 7fa9e0c79b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a08d377b20 con 0x55a08d487db0 2023-09-13T03:12:17.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.113+0000 7fa9e0c79b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a08d377820 con 0x55a08d488bb0 2023-09-13T03:12:17.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.113+0000 7fa9e0c79b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a08d25b8d0 con 0x55a08d489550 2023-09-13T03:12:17.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.113+0000 7fa9e0c79b80 10 client.0 _ll_register_callbacks cb 0x55a08d2c1a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:12:17.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.113+0000 7fa9c6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a08d487db0 0x55a08d4914c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.113+0000 7fa9c6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a08d487db0 0x55a08d4914c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57810/0 (socket says 172.21.15.64:57810) 2023-09-13T03:12:17.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.113+0000 7fa9c6ffd700 1 -- 172.21.15.64:0/1612390167 learned_addr learned my addr 172.21.15.64:0/1612390167 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:17.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.113+0000 7fa9c67fc700 1 --2- 172.21.15.64:0/1612390167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a08d488bb0 0x55a08d491a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.113+0000 7fa9c77fe700 1 --2- 172.21.15.64:0/1612390167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a08d489550 0x55a08d498a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.114+0000 7fa9c6ffd700 1 -- 172.21.15.64:0/1612390167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a08d489550 msgr2=0x55a08d498a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:17.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.114+0000 7fa9c6ffd700 1 --2- 172.21.15.64:0/1612390167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a08d489550 0x55a08d498a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.114+0000 7fa9c6ffd700 1 -- 172.21.15.64:0/1612390167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a08d488bb0 msgr2=0x55a08d491a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:17.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.114+0000 7fa9c6ffd700 1 --2- 172.21.15.64:0/1612390167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a08d488bb0 0x55a08d491a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.114+0000 7fa9c6ffd700 1 -- 172.21.15.64:0/1612390167 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9c8109350 con 0x55a08d487db0 2023-09-13T03:12:17.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.114+0000 7fa9c67fc700 1 --2- 172.21.15.64:0/1612390167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a08d488bb0 0x55a08d491a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:17.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.114+0000 7fa9c6ffd700 1 --2- 172.21.15.64:0/1612390167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a08d487db0 0x55a08d4914c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8012d20 tx=0x7fa9b800f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a6781e1f963843f server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:17.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.114+0000 7fa9aaff5700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:12:17.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.114+0000 7fa9aaff5700 1 -- 172.21.15.64:0/1612390167 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8017070 con 0x55a08d487db0 2023-09-13T03:12:17.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.114+0000 7fa9e0c79b80 1 -- 172.21.15.64:0/1612390167 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a08d3314a0 con 0x55a08d487db0 2023-09-13T03:12:17.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.114+0000 7fa9aaff5700 1 -- 172.21.15.64:0/1612390167 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa9b800fb80 con 0x55a08d487db0 2023-09-13T03:12:17.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.114+0000 7fa9aaff5700 1 -- 172.21.15.64:0/1612390167 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9b80122e0 con 0x55a08d487db0 2023-09-13T03:12:17.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.114+0000 7fa9e0c79b80 10 client.15063 fetch_fsmap learned FSMap version 26 2023-09-13T03:12:17.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.114+0000 7fa9e0c79b80 1 -- 172.21.15.64:0/1612390167 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55a08d1eec10 con 0x55a08d487db0 2023-09-13T03:12:17.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.115+0000 7fa9aaff5700 1 -- 172.21.15.64:0/1612390167 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a08d3314a0 con 0x55a08d487db0 2023-09-13T03:12:17.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.115+0000 7fa9aaff5700 1 -- 172.21.15.64:0/1612390167 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa9b8025050 con 0x55a08d487db0 2023-09-13T03:12:17.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.115+0000 7fa9e0c79b80 10 client.15063 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:12:17.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.115+0000 7fa9e0c79b80 10 client.15063 resolve_mds: resolved name 'f' to daemon mds.f 2023-09-13T03:12:17.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.115+0000 7fa9e0c79b80 20 client.15063 populate_metadata read hostname 'smithi064' 2023-09-13T03:12:17.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.115+0000 7fa9e0c79b80 1 --2- 172.21.15.64:0/1612390167 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x55a08d49ccb0 0x55a08d49bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.115+0000 7fa9e0c79b80 4 client.15063 mds_command: new command op to 24622 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:12:17.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.115+0000 7fa9e0c79b80 1 -- 172.21.15.64:0/1612390167 --> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a08d3ee850 con 0x55a08d49ccb0 2023-09-13T03:12:17.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.115+0000 7fa9c67fc700 1 --2- 172.21.15.64:0/1612390167 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x55a08d49ccb0 0x55a08d49bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.116+0000 7fa9c67fc700 1 --2- 172.21.15.64:0/1612390167 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x55a08d49ccb0 0x55a08d49bbf0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e65ca662fff0800c server_cookie=ad41f5acd5cb49eb in_seq=0 out_seq=0 2023-09-13T03:12:17.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.116+0000 7fa9aaff5700 10 client.15063 ms_handle_connect on v2:172.21.15.170:6834/1493371121 2023-09-13T03:12:17.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.116+0000 7fa9aaff5700 1 -- 172.21.15.64:0/1612390167 <== mds.4 v2:172.21.15.170:6834/1493371121 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a08d3ee850 con 0x55a08d49ccb0 2023-09-13T03:12:17.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.116+0000 7fa9aaff5700 10 client.15063 handle_command_reply: tid=0 2023-09-13T03:12:17.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.117+0000 7fa9e0c79b80 1 client.15063 shutdown 2023-09-13T03:12:17.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.117+0000 7fa9e0c79b80 1 -- 172.21.15.64:0/1612390167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a08d487db0 msgr2=0x55a08d4914c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:17.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.117+0000 7fa9e0c79b80 1 --2- 172.21.15.64:0/1612390167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a08d487db0 0x55a08d4914c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8012d20 tx=0x7fa9b800f6a0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.117+0000 7fa9e0c79b80 1 -- 172.21.15.64:0/1612390167 shutdown_connections 2023-09-13T03:12:17.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.117+0000 7fa9e0c79b80 1 --2- 172.21.15.64:0/1612390167 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x55a08d49ccb0 0x55a08d49bbf0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.117+0000 7fa9e0c79b80 1 --2- 172.21.15.64:0/1612390167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a08d489550 0x55a08d498a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.117+0000 7fa9e0c79b80 1 --2- 172.21.15.64:0/1612390167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a08d487db0 0x55a08d4914c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.117+0000 7fa9e0c79b80 1 --2- 172.21.15.64:0/1612390167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a08d488bb0 0x55a08d491a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.117+0000 7fa9e0c79b80 1 -- 172.21.15.64:0/1612390167 >> 172.21.15.64:0/1612390167 conn(0x55a08d3fc040 msgr2=0x55a08d481d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:17.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.117+0000 7fa9aaff5700 0 client.15063 ms_handle_reset on v2:172.21.15.170:6834/1493371121 2023-09-13T03:12:17.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.117+0000 7fa9e0c79b80 1 -- 172.21.15.64:0/1612390167 shutdown_connections 2023-09-13T03:12:17.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.117+0000 7fa9e0c79b80 1 -- 172.21.15.64:0/1612390167 wait complete. 2023-09-13T03:12:17.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.117+0000 7fa9e0c79b80 20 client.15063 trim_cache size 0 max 16384 2023-09-13T03:12:17.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.136+0000 7fa9e0c79b80 1 Processor -- start 2023-09-13T03:12:17.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.136+0000 7fa9e0c79b80 1 -- start start 2023-09-13T03:12:17.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.136+0000 7fa9e0c79b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a08d487db0 0x55a08d48f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.136+0000 7fa9e0c79b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a08d488bb0 0x55a08d48f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.136+0000 7fa9e0c79b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a08d489550 0x55a08d498a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.136+0000 7fa9e0c79b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a08d3814d0 con 0x55a08d487db0 2023-09-13T03:12:17.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.136+0000 7fa9e0c79b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a08d4a5f50 con 0x55a08d489550 2023-09-13T03:12:17.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.136+0000 7fa9e0c79b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a08d267de0 con 0x55a08d488bb0 2023-09-13T03:12:17.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.136+0000 7fa9c6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a08d487db0 0x55a08d48f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.137+0000 7fa9c6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a08d487db0 0x55a08d48f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57814/0 (socket says 172.21.15.64:57814) 2023-09-13T03:12:17.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.137+0000 7fa9c6ffd700 1 -- 172.21.15.64:0/228720966 learned_addr learned my addr 172.21.15.64:0/228720966 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:17.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.137+0000 7fa9c77fe700 1 --2- 172.21.15.64:0/228720966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a08d489550 0x55a08d498a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.137+0000 7fa9c67fc700 1 --2- 172.21.15.64:0/228720966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a08d488bb0 0x55a08d48f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.137+0000 7fa9c6ffd700 1 -- 172.21.15.64:0/228720966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a08d488bb0 msgr2=0x55a08d48f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:17.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.137+0000 7fa9c6ffd700 1 --2- 172.21.15.64:0/228720966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a08d488bb0 0x55a08d48f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.137+0000 7fa9c6ffd700 1 -- 172.21.15.64:0/228720966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a08d489550 msgr2=0x55a08d498a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:17.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.137+0000 7fa9c6ffd700 1 --2- 172.21.15.64:0/228720966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a08d489550 0x55a08d498a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.137+0000 7fa9c6ffd700 1 -- 172.21.15.64:0/228720966 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a08d22c200 con 0x55a08d487db0 2023-09-13T03:12:17.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.137+0000 7fa9c77fe700 1 --2- 172.21.15.64:0/228720966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a08d489550 0x55a08d498a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:17.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.137+0000 7fa9c6ffd700 1 --2- 172.21.15.64:0/228720966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a08d487db0 0x55a08d48f330 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8005a80 tx=0x7fa9b800fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=911c428be83b40dc server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:17.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.137+0000 7fa9adffb700 1 -- 172.21.15.64:0/228720966 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b802b070 con 0x55a08d487db0 2023-09-13T03:12:17.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.137+0000 7fa9adffb700 1 -- 172.21.15.64:0/228720966 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9b8005040 con 0x55a08d487db0 2023-09-13T03:12:17.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.137+0000 7fa9adffb700 1 -- 172.21.15.64:0/228720966 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8017040 con 0x55a08d487db0 2023-09-13T03:12:17.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.138+0000 7fa9e0c79b80 1 -- 172.21.15.64:0/228720966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a08d487db0 msgr2=0x55a08d48f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:17.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.138+0000 7fa9e0c79b80 1 --2- 172.21.15.64:0/228720966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a08d487db0 0x55a08d48f330 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8005a80 tx=0x7fa9b800fa70 comp rx=0 tx=0).stop 2023-09-13T03:12:17.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.138+0000 7fa9e0c79b80 1 -- 172.21.15.64:0/228720966 shutdown_connections 2023-09-13T03:12:17.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.138+0000 7fa9e0c79b80 1 --2- 172.21.15.64:0/228720966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a08d489550 0x55a08d498a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.138+0000 7fa9e0c79b80 1 --2- 172.21.15.64:0/228720966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a08d487db0 0x55a08d48f330 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.138+0000 7fa9e0c79b80 1 --2- 172.21.15.64:0/228720966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a08d488bb0 0x55a08d48f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.138+0000 7fa9e0c79b80 1 -- 172.21.15.64:0/228720966 >> 172.21.15.64:0/228720966 conn(0x55a08d3fc040 msgr2=0x55a08d49bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:17.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.138+0000 7fa9e0c79b80 1 -- 172.21.15.64:0/228720966 shutdown_connections 2023-09-13T03:12:17.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.138+0000 7fa9e0c79b80 1 -- 172.21.15.64:0/228720966 wait complete. 2023-09-13T03:12:17.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.138+0000 7fa9e0c79b80 1 -- start start 2023-09-13T03:12:17.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.139+0000 7fa9e0c79b80 1 Processor -- start 2023-09-13T03:12:17.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.139+0000 7fa9e0c79b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a08d487db0 0x55a08d4938b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.139+0000 7fa9e0c79b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a08d488bb0 0x55a08d493df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.139+0000 7fa9e0c79b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a08d489550 0x55a08d4f0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.139+0000 7fa9e0c79b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a08d494500 con 0x55a08d489550 2023-09-13T03:12:17.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.139+0000 7fa9e0c79b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a08d4f1240 con 0x55a08d487db0 2023-09-13T03:12:17.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.139+0000 7fa9e0c79b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a08d4f1400 con 0x55a08d488bb0 2023-09-13T03:12:17.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.139+0000 7fa9e0c79b80 10 client.0 _ll_register_callbacks cb 0x7fa9b80127f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:12:17.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.139+0000 7fa9c77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a08d489550 0x55a08d4f0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.139+0000 7fa9c67fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a08d488bb0 0x55a08d493df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.139+0000 7fa9c67fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a08d488bb0 0x55a08d493df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:53992/0 (socket says 172.21.15.64:53992) 2023-09-13T03:12:17.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.139+0000 7fa9c67fc700 1 -- 172.21.15.64:0/3642086312 learned_addr learned my addr 172.21.15.64:0/3642086312 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:17.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.139+0000 7fa9c77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a08d489550 0x55a08d4f0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57822/0 (socket says 172.21.15.64:57822) 2023-09-13T03:12:17.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.140+0000 7fa9c6ffd700 1 --2- 172.21.15.64:0/3642086312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a08d487db0 0x55a08d4938b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.140+0000 7fa9c67fc700 1 -- 172.21.15.64:0/3642086312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a08d487db0 msgr2=0x55a08d4938b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:17.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.140+0000 7fa9c67fc700 1 --2- 172.21.15.64:0/3642086312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a08d487db0 0x55a08d4938b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.140+0000 7fa9c67fc700 1 -- 172.21.15.64:0/3642086312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a08d489550 msgr2=0x55a08d4f0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:17.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.140+0000 7fa9c67fc700 1 --2- 172.21.15.64:0/3642086312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a08d489550 0x55a08d4f0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.140+0000 7fa9c67fc700 1 -- 172.21.15.64:0/3642086312 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a08d205d90 con 0x55a08d488bb0 2023-09-13T03:12:17.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.140+0000 7fa9c67fc700 1 --2- 172.21.15.64:0/3642086312 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a08d488bb0 0x55a08d493df0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc0994f0 tx=0x7fa9bc0cb040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b97a25959e8d6ace server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:17.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.140+0000 7fa9aaff5700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:12:17.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.140+0000 7fa9aaff5700 1 -- 172.21.15.64:0/3642086312 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc09d390 con 0x55a08d488bb0 2023-09-13T03:12:17.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.140+0000 7fa9e0c79b80 1 -- 172.21.15.64:0/3642086312 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa9c0003e90 con 0x55a08d488bb0 2023-09-13T03:12:17.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.140+0000 7fa9aaff5700 1 -- 172.21.15.64:0/3642086312 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa9bc0a15d0 con 0x55a08d488bb0 2023-09-13T03:12:17.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.140+0000 7fa9aaff5700 1 -- 172.21.15.64:0/3642086312 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9bc010880 con 0x55a08d488bb0 2023-09-13T03:12:17.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.140+0000 7fa9e0c79b80 10 client.25096 fetch_fsmap learned FSMap version 26 2023-09-13T03:12:17.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.140+0000 7fa9e0c79b80 1 -- 172.21.15.64:0/3642086312 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55a08d1eec10 con 0x55a08d488bb0 2023-09-13T03:12:17.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.141+0000 7fa9aaff5700 1 -- 172.21.15.64:0/3642086312 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9c0003e90 con 0x55a08d488bb0 2023-09-13T03:12:17.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.141+0000 7fa9aaff5700 1 -- 172.21.15.64:0/3642086312 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa9bc098a50 con 0x55a08d488bb0 2023-09-13T03:12:17.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.141+0000 7fa9e0c79b80 10 client.25096 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:12:17.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.141+0000 7fa9e0c79b80 10 client.25096 resolve_mds: resolved name 'f' to daemon mds.f 2023-09-13T03:12:17.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.141+0000 7fa9e0c79b80 20 client.25096 populate_metadata read hostname 'smithi064' 2023-09-13T03:12:17.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.141+0000 7fa9e0c79b80 1 --2- 172.21.15.64:0/3642086312 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x55a08d4f6710 0x55a08d4f2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.141+0000 7fa9e0c79b80 4 client.25096 mds_command: new command op to 24622 tid=0[{"prefix": "status"}] 2023-09-13T03:12:17.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.141+0000 7fa9e0c79b80 1 -- 172.21.15.64:0/3642086312 --> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a08d3d28d0 con 0x55a08d4f6710 2023-09-13T03:12:17.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.141+0000 7fa9c6ffd700 1 --2- 172.21.15.64:0/3642086312 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x55a08d4f6710 0x55a08d4f2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.142+0000 7fa9c6ffd700 1 --2- 172.21.15.64:0/3642086312 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x55a08d4f6710 0x55a08d4f2b30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cfe6c1e4adef5f4b server_cookie=726d458efe3005f4 in_seq=0 out_seq=0 2023-09-13T03:12:17.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.142+0000 7fa9aaff5700 10 client.25096 ms_handle_connect on v2:172.21.15.170:6834/1493371121 2023-09-13T03:12:17.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.142+0000 7fa9aaff5700 1 -- 172.21.15.64:0/3642086312 <== mds.4 v2:172.21.15.170:6834/1493371121 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55a08d3d28d0 con 0x55a08d4f6710 2023-09-13T03:12:17.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.142+0000 7fa9aaff5700 10 client.25096 handle_command_reply: tid=0 2023-09-13T03:12:17.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.142+0000 7fa9e0c79b80 1 client.25096 shutdown 2023-09-13T03:12:17.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.142+0000 7fa9e0c79b80 1 -- 172.21.15.64:0/3642086312 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a08d488bb0 msgr2=0x55a08d493df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:17.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.142+0000 7fa9e0c79b80 1 --2- 172.21.15.64:0/3642086312 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a08d488bb0 0x55a08d493df0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc0994f0 tx=0x7fa9bc0cb040 comp rx=0 tx=0).stop 2023-09-13T03:12:17.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.142+0000 7fa9e0c79b80 1 -- 172.21.15.64:0/3642086312 shutdown_connections 2023-09-13T03:12:17.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.142+0000 7fa9e0c79b80 1 --2- 172.21.15.64:0/3642086312 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x55a08d4f6710 0x55a08d4f2b30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.142+0000 7fa9e0c79b80 1 --2- 172.21.15.64:0/3642086312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a08d487db0 0x55a08d4938b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.142+0000 7fa9aaff5700 0 client.25096 ms_handle_reset on v2:172.21.15.170:6834/1493371121 2023-09-13T03:12:17.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.142+0000 7fa9e0c79b80 1 --2- 172.21.15.64:0/3642086312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a08d489550 0x55a08d4f0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.142+0000 7fa9e0c79b80 1 --2- 172.21.15.64:0/3642086312 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a08d488bb0 0x55a08d493df0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.142+0000 7fa9e0c79b80 1 -- 172.21.15.64:0/3642086312 >> 172.21.15.64:0/3642086312 conn(0x55a08d3fc040 msgr2=0x55a08d48be60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:17.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.142+0000 7fa9e0c79b80 1 -- 172.21.15.64:0/3642086312 shutdown_connections 2023-09-13T03:12:17.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.142+0000 7fa9e0c79b80 1 -- 172.21.15.64:0/3642086312 wait complete. 2023-09-13T03:12:17.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.142+0000 7fa9e0c79b80 20 client.25096 trim_cache size 0 max 16384 2023-09-13T03:12:17.159 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-13T03:12:17.159 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "f8dbced4-51e1-11ee-9ab7-7b867c8bd7da", 2023-09-13T03:12:17.159 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 4, 2023-09-13T03:12:17.159 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24622, 2023-09-13T03:12:17.159 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2023-09-13T03:12:17.160 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2023-09-13T03:12:17.160 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2023-09-13T03:12:17.160 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 59.918519142000001, 2023-09-13T03:12:17.160 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 26, 2023-09-13T03:12:17.160 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 85, 2023-09-13T03:12:17.161 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 83, 2023-09-13T03:12:17.161 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 88.641628589000007 2023-09-13T03:12:17.161 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-13T03:12:17.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.143+0000 7fa9cd3ff700 1 -- 172.21.15.64:0/1867234685 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fa9b00a06d0 msgr2=0x7fa9b00a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:17.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.143+0000 7fa9cd3ff700 1 --2- 172.21.15.64:0/1867234685 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fa9b00a06d0 0x7fa9b00a2b90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8101100 tx=0x7fa9c000b720 comp rx=0 tx=0).stop 2023-09-13T03:12:17.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.143+0000 7fa9cd3ff700 1 -- 172.21.15.64:0/1867234685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9c8103a60 msgr2=0x7fa9c80ff0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:17.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.143+0000 7fa9cd3ff700 1 --2- 172.21.15.64:0/1867234685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9c8103a60 0x7fa9c80ff0e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc00e9a0 tx=0x7fa9bc00ecb0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.143+0000 7fa9cd3ff700 1 -- 172.21.15.64:0/1867234685 shutdown_connections 2023-09-13T03:12:17.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.143+0000 7fa9cd3ff700 1 --2- 172.21.15.64:0/1867234685 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fa9b00a06d0 0x7fa9b00a2b90 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.143+0000 7fa9cd3ff700 1 --2- 172.21.15.64:0/1867234685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9c8101130 0x7fa9c8100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.143+0000 7fa9cd3ff700 1 --2- 172.21.15.64:0/1867234685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9c8106390 0x7fa9c80ff620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.143+0000 7fa9cd3ff700 1 --2- 172.21.15.64:0/1867234685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9c8103a60 0x7fa9c80ff0e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.143+0000 7fa9cd3ff700 1 -- 172.21.15.64:0/1867234685 >> 172.21.15.64:0/1867234685 conn(0x7fa9c80faa60 msgr2=0x7fa9c8101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:17.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.143+0000 7fa9cd3ff700 1 -- 172.21.15.64:0/1867234685 shutdown_connections 2023-09-13T03:12:17.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.143+0000 7fa9cd3ff700 1 -- 172.21.15.64:0/1867234685 wait complete. 2023-09-13T03:12:17.165 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-09-13T03:12:17.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.368+0000 7f2713fff700 1 -- 172.21.15.64:0/1506120267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f270c1005f0 msgr2=0x7f270c100a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:17.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.368+0000 7f2713fff700 1 --2- 172.21.15.64:0/1506120267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f270c1005f0 0x7f270c100a10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f27080099c0 tx=0x7f2708009cd0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.368+0000 7f2713fff700 1 -- 172.21.15.64:0/1506120267 shutdown_connections 2023-09-13T03:12:17.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.368+0000 7f2713fff700 1 --2- 172.21.15.64:0/1506120267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f270c102180 0x7f270c108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.368+0000 7f2713fff700 1 --2- 172.21.15.64:0/1506120267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f270c1005f0 0x7f270c100a10 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.368+0000 7f2713fff700 1 --2- 172.21.15.64:0/1506120267 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f270c1017e0 0x7f270c101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.368+0000 7f2713fff700 1 -- 172.21.15.64:0/1506120267 >> 172.21.15.64:0/1506120267 conn(0x7f270c0fbb40 msgr2=0x7f270c0fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:17.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.368+0000 7f2713fff700 1 -- 172.21.15.64:0/1506120267 shutdown_connections 2023-09-13T03:12:17.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.369+0000 7f2713fff700 1 -- 172.21.15.64:0/1506120267 wait complete. 2023-09-13T03:12:17.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.369+0000 7f2713fff700 1 Processor -- start 2023-09-13T03:12:17.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.370+0000 7f2713fff700 1 -- start start 2023-09-13T03:12:17.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.370+0000 7f2713fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f270c1005f0 0x7f270c196330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.370+0000 7f2713fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f270c1017e0 0x7f270c196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.370+0000 7f2713fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f270c102180 0x7f270c19d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.370+0000 7f2713fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f270c19deb0 con 0x7f270c1005f0 2023-09-13T03:12:17.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.370+0000 7f2713fff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f270c19dff0 con 0x7f270c102180 2023-09-13T03:12:17.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.370+0000 7f2713fff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f270c19e180 con 0x7f270c1017e0 2023-09-13T03:12:17.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.370+0000 7f2711d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f270c1005f0 0x7f270c196330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.370+0000 7f2711d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f270c1005f0 0x7f270c196330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57848/0 (socket says 172.21.15.64:57848) 2023-09-13T03:12:17.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.370+0000 7f2711d9b700 1 -- 172.21.15.64:0/166156260 learned_addr learned my addr 172.21.15.64:0/166156260 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:17.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.370+0000 7f271259c700 1 --2- 172.21.15.64:0/166156260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f270c102180 0x7f270c19d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.370+0000 7f271159a700 1 --2- 172.21.15.64:0/166156260 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f270c1017e0 0x7f270c196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.371+0000 7f2711d9b700 1 -- 172.21.15.64:0/166156260 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f270c1017e0 msgr2=0x7f270c196870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:17.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.371+0000 7f2711d9b700 1 --2- 172.21.15.64:0/166156260 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f270c1017e0 0x7f270c196870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.371+0000 7f2711d9b700 1 -- 172.21.15.64:0/166156260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f270c102180 msgr2=0x7f270c19d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:17.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.371+0000 7f2711d9b700 1 --2- 172.21.15.64:0/166156260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f270c102180 0x7f270c19d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.371+0000 7f2711d9b700 1 -- 172.21.15.64:0/166156260 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27080095f0 con 0x7f270c1005f0 2023-09-13T03:12:17.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.371+0000 7f2711d9b700 1 --2- 172.21.15.64:0/166156260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f270c1005f0 0x7f270c196330 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2708009fd0 tx=0x7f270800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:17.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.371+0000 7f2702ffd700 1 -- 172.21.15.64:0/166156260 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270801d070 con 0x7f270c1005f0 2023-09-13T03:12:17.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.371+0000 7f2702ffd700 1 -- 172.21.15.64:0/166156260 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2708005030 con 0x7f270c1005f0 2023-09-13T03:12:17.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.371+0000 7f2713fff700 1 -- 172.21.15.64:0/166156260 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f270c19e400 con 0x7f270c1005f0 2023-09-13T03:12:17.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.371+0000 7f2702ffd700 1 -- 172.21.15.64:0/166156260 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2708004aa0 con 0x7f270c1005f0 2023-09-13T03:12:17.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.371+0000 7f2713fff700 1 -- 172.21.15.64:0/166156260 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f270c19e880 con 0x7f270c1005f0 2023-09-13T03:12:17.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.374+0000 7f2702ffd700 1 -- 172.21.15.64:0/166156260 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f2708005410 con 0x7f270c1005f0 2023-09-13T03:12:17.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.376+0000 7f27278d2b80 1 Processor -- start 2023-09-13T03:12:17.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.376+0000 7f2702ffd700 1 --2- 172.21.15.64:0/166156260 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f26f80a0750 0x7f26f80a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.376+0000 7f27278d2b80 1 -- start start 2023-09-13T03:12:17.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.376+0000 7f2702ffd700 1 -- 172.21.15.64:0/166156260 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f27080ccab0 con 0x7f270c1005f0 2023-09-13T03:12:17.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.376+0000 7f271259c700 1 --2- 172.21.15.64:0/166156260 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f26f80a0750 0x7f26f80a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.376+0000 7f27278d2b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560016013bb0 0x560016013fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.376+0000 7f27278d2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560016014550 0x5600160127b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.376+0000 7f27278d2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560016012db0 0x560016013260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.376+0000 7f27278d2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560015f6b6e0 con 0x560016014550 2023-09-13T03:12:17.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.376+0000 7f27278d2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560015f69000 con 0x560016012db0 2023-09-13T03:12:17.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.376+0000 7f27278d2b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560015f648f0 con 0x560016013bb0 2023-09-13T03:12:17.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.377+0000 7f271159a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560016014550 0x5600160127b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.377+0000 7f271159a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560016014550 0x5600160127b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57862/0 (socket says 172.21.15.64:57862) 2023-09-13T03:12:17.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.377+0000 7f271159a700 1 -- 172.21.15.64:0/626643038 learned_addr learned my addr 172.21.15.64:0/626643038 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:17.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.377+0000 7f271259c700 1 --2- 172.21.15.64:0/166156260 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f26f80a0750 0x7f26f80a2c10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2704003e90 tx=0x7f2704009130 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:17.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.377+0000 7f2711d9b700 1 --2- 172.21.15.64:0/626643038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560016013bb0 0x560016013fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.377+0000 7f271259c700 1 --2- 172.21.15.64:0/626643038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560016012db0 0x560016013260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.377+0000 7f271159a700 1 -- 172.21.15.64:0/626643038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560016013bb0 msgr2=0x560016013fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:17.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.377+0000 7f271159a700 1 --2- 172.21.15.64:0/626643038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560016013bb0 0x560016013fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.377+0000 7f271159a700 1 -- 172.21.15.64:0/626643038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560016012db0 msgr2=0x560016013260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:17.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.377+0000 7f271159a700 1 --2- 172.21.15.64:0/626643038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560016012db0 0x560016013260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.377+0000 7f271159a700 1 -- 172.21.15.64:0/626643038 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560015d90d90 con 0x560016014550 2023-09-13T03:12:17.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.377+0000 7f271259c700 1 --2- 172.21.15.64:0/626643038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560016012db0 0x560016013260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:12:17.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.377+0000 7f271159a700 1 --2- 172.21.15.64:0/626643038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560016014550 0x5600160127b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f26fc009d80 tx=0x7f26fc011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cdfbdc8d6cc1cf7 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:17.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.378+0000 7f26f7fff700 1 -- 172.21.15.64:0/626643038 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26fc004230 con 0x560016014550 2023-09-13T03:12:17.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.378+0000 7f26f7fff700 1 -- 172.21.15.64:0/626643038 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26fc004390 con 0x560016014550 2023-09-13T03:12:17.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.378+0000 7f26f7fff700 1 -- 172.21.15.64:0/626643038 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26fc019860 con 0x560016014550 2023-09-13T03:12:17.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.378+0000 7f27278d2b80 1 -- 172.21.15.64:0/626643038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560016014550 msgr2=0x5600160127b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:17.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.378+0000 7f27278d2b80 1 --2- 172.21.15.64:0/626643038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560016014550 0x5600160127b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f26fc009d80 tx=0x7f26fc011720 comp rx=0 tx=0).stop 2023-09-13T03:12:17.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.378+0000 7f27278d2b80 1 -- 172.21.15.64:0/626643038 shutdown_connections 2023-09-13T03:12:17.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.378+0000 7f27278d2b80 1 --2- 172.21.15.64:0/626643038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560016012db0 0x560016013260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.378+0000 7f27278d2b80 1 --2- 172.21.15.64:0/626643038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560016013bb0 0x560016013fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.378+0000 7f27278d2b80 1 --2- 172.21.15.64:0/626643038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560016014550 0x5600160127b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.378+0000 7f27278d2b80 1 -- 172.21.15.64:0/626643038 >> 172.21.15.64:0/626643038 conn(0x560015f87040 msgr2=0x56001600d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:17.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.378+0000 7f27278d2b80 1 -- 172.21.15.64:0/626643038 shutdown_connections 2023-09-13T03:12:17.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.379+0000 7f27278d2b80 1 -- 172.21.15.64:0/626643038 wait complete. 2023-09-13T03:12:17.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.379+0000 7f27278d2b80 1 -- start start 2023-09-13T03:12:17.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.379+0000 7f27278d2b80 1 Processor -- start 2023-09-13T03:12:17.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.380+0000 7f27278d2b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560016012db0 0x560016020920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.380+0000 7f27278d2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560016013bb0 0x560016020e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.380+0000 7f27278d2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560016014550 0x560016027ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.380+0000 7f27278d2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560015f02b20 con 0x560016013bb0 2023-09-13T03:12:17.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.380+0000 7f27278d2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560015f02820 con 0x560016014550 2023-09-13T03:12:17.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.380+0000 7f27278d2b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560015de68d0 con 0x560016012db0 2023-09-13T03:12:17.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.380+0000 7f27278d2b80 10 client.0 _ll_register_callbacks cb 0x560015e4ca30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:12:17.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.380+0000 7f2711d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560016012db0 0x560016020920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.380+0000 7f271159a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560016013bb0 0x560016020e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.380+0000 7f2711d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560016012db0 0x560016020920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:54018/0 (socket says 172.21.15.64:54018) 2023-09-13T03:12:17.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.380+0000 7f2711d9b700 1 -- 172.21.15.64:0/2256554218 learned_addr learned my addr 172.21.15.64:0/2256554218 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:17.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.380+0000 7f271259c700 1 --2- 172.21.15.64:0/2256554218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560016014550 0x560016027ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.380+0000 7f271159a700 1 -- 172.21.15.64:0/2256554218 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560016012db0 msgr2=0x560016020920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:17.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.380+0000 7f271159a700 1 --2- 172.21.15.64:0/2256554218 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560016012db0 0x560016020920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.380+0000 7f271159a700 1 -- 172.21.15.64:0/2256554218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560016014550 msgr2=0x560016027ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:17.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.380+0000 7f271159a700 1 --2- 172.21.15.64:0/2256554218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560016014550 0x560016027ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.380+0000 7f271159a700 1 -- 172.21.15.64:0/2256554218 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26fc0099e0 con 0x560016013bb0 2023-09-13T03:12:17.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.380+0000 7f2711d9b700 1 --2- 172.21.15.64:0/2256554218 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560016012db0 0x560016020920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:17.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.381+0000 7f271159a700 1 --2- 172.21.15.64:0/2256554218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560016013bb0 0x560016020e60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f26fc01b040 tx=0x7f26fc0038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11830f5720bc47c3 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:17.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.381+0000 7f26f57fa700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:12:17.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.381+0000 7f26f57fa700 1 -- 172.21.15.64:0/2256554218 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26fc005240 con 0x560016013bb0 2023-09-13T03:12:17.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.381+0000 7f27278d2b80 1 -- 172.21.15.64:0/2256554218 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560015ebc4a0 con 0x560016013bb0 2023-09-13T03:12:17.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.381+0000 7f26f57fa700 1 -- 172.21.15.64:0/2256554218 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f26fc004500 con 0x560016013bb0 2023-09-13T03:12:17.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.381+0000 7f26f57fa700 1 -- 172.21.15.64:0/2256554218 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26fc005ce0 con 0x560016013bb0 2023-09-13T03:12:17.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.381+0000 7f27278d2b80 10 client.15081 fetch_fsmap learned FSMap version 26 2023-09-13T03:12:17.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.381+0000 7f27278d2b80 1 -- 172.21.15.64:0/2256554218 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x560015d79c10 con 0x560016013bb0 2023-09-13T03:12:17.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.382+0000 7f26f57fa700 1 -- 172.21.15.64:0/2256554218 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560015ebc4a0 con 0x560016013bb0 2023-09-13T03:12:17.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.382+0000 7f26f57fa700 1 -- 172.21.15.64:0/2256554218 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f26fc021040 con 0x560016013bb0 2023-09-13T03:12:17.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.382+0000 7f27278d2b80 10 client.15081 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:12:17.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.382+0000 7f27278d2b80 10 client.15081 resolve_mds: resolved name 'c' to daemon mds.c 2023-09-13T03:12:17.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.382+0000 7f27278d2b80 20 client.15081 populate_metadata read hostname 'smithi064' 2023-09-13T03:12:17.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.382+0000 7f27278d2b80 1 --2- 172.21.15.64:0/2256554218 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x56001602bf40 0x56001602af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.382+0000 7f27278d2b80 4 client.15081 mds_command: new command op to 24634 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:12:17.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.382+0000 7f27278d2b80 1 -- 172.21.15.64:0/2256554218 --> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560015f79850 con 0x56001602bf40 2023-09-13T03:12:17.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.382+0000 7f2711d9b700 1 --2- 172.21.15.64:0/2256554218 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x56001602bf40 0x56001602af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.383+0000 7f2711d9b700 1 --2- 172.21.15.64:0/2256554218 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x56001602bf40 0x56001602af50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b604b55006272105 server_cookie=4e1374cb86653c02 in_seq=0 out_seq=0 2023-09-13T03:12:17.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.383+0000 7f26f57fa700 10 client.15081 ms_handle_connect on v2:172.21.15.170:6836/1726587275 2023-09-13T03:12:17.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.384+0000 7f26f57fa700 1 -- 172.21.15.64:0/2256554218 <== mds.3 v2:172.21.15.170:6836/1726587275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560015f79850 con 0x56001602bf40 2023-09-13T03:12:17.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.384+0000 7f26f57fa700 10 client.15081 handle_command_reply: tid=0 2023-09-13T03:12:17.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.384+0000 7f27278d2b80 1 client.15081 shutdown 2023-09-13T03:12:17.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.384+0000 7f27278d2b80 1 -- 172.21.15.64:0/2256554218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560016013bb0 msgr2=0x560016020e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:17.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.384+0000 7f27278d2b80 1 --2- 172.21.15.64:0/2256554218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560016013bb0 0x560016020e60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f26fc01b040 tx=0x7f26fc0038b0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.384+0000 7f27278d2b80 1 -- 172.21.15.64:0/2256554218 shutdown_connections 2023-09-13T03:12:17.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.384+0000 7f27278d2b80 1 --2- 172.21.15.64:0/2256554218 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x56001602bf40 0x56001602af50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.384+0000 7f27278d2b80 1 --2- 172.21.15.64:0/2256554218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560016014550 0x560016027ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.384+0000 7f27278d2b80 1 --2- 172.21.15.64:0/2256554218 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560016012db0 0x560016020920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.384+0000 7f26f57fa700 0 client.15081 ms_handle_reset on v2:172.21.15.170:6836/1726587275 2023-09-13T03:12:17.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.384+0000 7f27278d2b80 1 --2- 172.21.15.64:0/2256554218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560016013bb0 0x560016020e60 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.384+0000 7f27278d2b80 1 -- 172.21.15.64:0/2256554218 >> 172.21.15.64:0/2256554218 conn(0x560015f87040 msgr2=0x56001600cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:17.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.384+0000 7f27278d2b80 1 -- 172.21.15.64:0/2256554218 shutdown_connections 2023-09-13T03:12:17.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.384+0000 7f27278d2b80 1 -- 172.21.15.64:0/2256554218 wait complete. 2023-09-13T03:12:17.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.384+0000 7f27278d2b80 20 client.15081 trim_cache size 0 max 16384 2023-09-13T03:12:17.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.402+0000 7f27278d2b80 1 Processor -- start 2023-09-13T03:12:17.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.403+0000 7f27278d2b80 1 -- start start 2023-09-13T03:12:17.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.403+0000 7f27278d2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560016012db0 0x5600160126c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.403+0000 7f27278d2b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560016013bb0 0x560016027e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.403+0000 7f27278d2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560016014550 0x56001601f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.403+0000 7f27278d2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560016021b10 con 0x560016014550 2023-09-13T03:12:17.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.403+0000 7f27278d2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560015df2de0 con 0x560016012db0 2023-09-13T03:12:17.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.403+0000 7f27278d2b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560015f2df70 con 0x560016013bb0 2023-09-13T03:12:17.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.403+0000 7f271259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560016014550 0x56001601f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.403+0000 7f271259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560016014550 0x56001601f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57880/0 (socket says 172.21.15.64:57880) 2023-09-13T03:12:17.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.403+0000 7f271259c700 1 -- 172.21.15.64:0/3550250027 learned_addr learned my addr 172.21.15.64:0/3550250027 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:17.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.403+0000 7f271159a700 1 --2- 172.21.15.64:0/3550250027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560016013bb0 0x560016027e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.403+0000 7f2711d9b700 1 --2- 172.21.15.64:0/3550250027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560016012db0 0x5600160126c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.403+0000 7f271159a700 1 -- 172.21.15.64:0/3550250027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560016012db0 msgr2=0x5600160126c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:17.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.404+0000 7f271159a700 1 --2- 172.21.15.64:0/3550250027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560016012db0 0x5600160126c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.404+0000 7f271159a700 1 -- 172.21.15.64:0/3550250027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560016014550 msgr2=0x56001601f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:17.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.404+0000 7f271159a700 1 --2- 172.21.15.64:0/3550250027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560016014550 0x56001601f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.404+0000 7f271159a700 1 -- 172.21.15.64:0/3550250027 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560015db7200 con 0x560016013bb0 2023-09-13T03:12:17.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.404+0000 7f271259c700 1 --2- 172.21.15.64:0/3550250027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560016014550 0x56001601f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:17.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.404+0000 7f271159a700 1 --2- 172.21.15.64:0/3550250027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560016013bb0 0x560016027e60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f26fc000be0 tx=0x7f26fc011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e972d2c13d1e9ee server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:17.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.404+0000 7f2700ff9700 1 -- 172.21.15.64:0/3550250027 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26fc019ac0 con 0x560016013bb0 2023-09-13T03:12:17.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.404+0000 7f2700ff9700 1 -- 172.21.15.64:0/3550250027 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26fc019c20 con 0x560016013bb0 2023-09-13T03:12:17.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.404+0000 7f2700ff9700 1 -- 172.21.15.64:0/3550250027 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26fc004760 con 0x560016013bb0 2023-09-13T03:12:17.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.405+0000 7f27278d2b80 1 -- 172.21.15.64:0/3550250027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560016013bb0 msgr2=0x560016027e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:17.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.405+0000 7f27278d2b80 1 --2- 172.21.15.64:0/3550250027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560016013bb0 0x560016027e60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f26fc000be0 tx=0x7f26fc011720 comp rx=0 tx=0).stop 2023-09-13T03:12:17.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.405+0000 7f27278d2b80 1 -- 172.21.15.64:0/3550250027 shutdown_connections 2023-09-13T03:12:17.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.405+0000 7f27278d2b80 1 --2- 172.21.15.64:0/3550250027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560016012db0 0x5600160126c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.405+0000 7f27278d2b80 1 --2- 172.21.15.64:0/3550250027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560016014550 0x56001601f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.405+0000 7f27278d2b80 1 --2- 172.21.15.64:0/3550250027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560016013bb0 0x560016027e60 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.405+0000 7f27278d2b80 1 -- 172.21.15.64:0/3550250027 >> 172.21.15.64:0/3550250027 conn(0x560015f87040 msgr2=0x56001602af50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:17.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.405+0000 7f27278d2b80 1 -- 172.21.15.64:0/3550250027 shutdown_connections 2023-09-13T03:12:17.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.405+0000 7f27278d2b80 1 -- 172.21.15.64:0/3550250027 wait complete. 2023-09-13T03:12:17.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.405+0000 7f27278d2b80 1 -- start start 2023-09-13T03:12:17.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.405+0000 7f27278d2b80 1 Processor -- start 2023-09-13T03:12:17.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.406+0000 7f27278d2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560016012db0 0x5600160230d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.406+0000 7f27278d2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560016013bb0 0x560016023610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.406+0000 7f27278d2b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560016014550 0x56001607ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.406+0000 7f27278d2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560016027ef0 con 0x560016012db0 2023-09-13T03:12:17.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.406+0000 7f27278d2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560016028060 con 0x560016013bb0 2023-09-13T03:12:17.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.406+0000 7f27278d2b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560016028220 con 0x560016014550 2023-09-13T03:12:17.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.406+0000 7f27278d2b80 10 client.0 _ll_register_callbacks cb 0x7f27080964e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:12:17.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.406+0000 7f2711d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560016012db0 0x5600160230d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.406+0000 7f271259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560016014550 0x56001607ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.406+0000 7f271159a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560016013bb0 0x560016023610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.406+0000 7f2711d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560016012db0 0x5600160230d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57894/0 (socket says 172.21.15.64:57894) 2023-09-13T03:12:17.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.406+0000 7f2711d9b700 1 -- 172.21.15.64:0/4178543411 learned_addr learned my addr 172.21.15.64:0/4178543411 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:17.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.406+0000 7f271259c700 1 -- 172.21.15.64:0/4178543411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560016013bb0 msgr2=0x560016023610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:17.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.406+0000 7f271259c700 1 --2- 172.21.15.64:0/4178543411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560016013bb0 0x560016023610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.406+0000 7f271259c700 1 -- 172.21.15.64:0/4178543411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560016012db0 msgr2=0x5600160230d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:17.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.406+0000 7f271259c700 1 --2- 172.21.15.64:0/4178543411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560016012db0 0x5600160230d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.406+0000 7f271259c700 1 -- 172.21.15.64:0/4178543411 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2704004620 con 0x560016014550 2023-09-13T03:12:17.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.406+0000 7f271159a700 1 --2- 172.21.15.64:0/4178543411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560016013bb0 0x560016023610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:17.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.406+0000 7f2711d9b700 1 --2- 172.21.15.64:0/4178543411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560016012db0 0x5600160230d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:17.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.407+0000 7f271259c700 1 --2- 172.21.15.64:0/4178543411 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560016014550 0x56001607ff90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x560016029c80 tx=0x7f2704004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bb85bc4da42c138 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:17.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.407+0000 7f26f57fa700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:12:17.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.407+0000 7f26f57fa700 1 -- 172.21.15.64:0/4178543411 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2704015ce0 con 0x560016014550 2023-09-13T03:12:17.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.407+0000 7f27278d2b80 1 -- 172.21.15.64:0/4178543411 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f26fc004390 con 0x560016014550 2023-09-13T03:12:17.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.407+0000 7f26f57fa700 1 -- 172.21.15.64:0/4178543411 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f2704015e40 con 0x560016014550 2023-09-13T03:12:17.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.407+0000 7f26f57fa700 1 -- 172.21.15.64:0/4178543411 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2704007e60 con 0x560016014550 2023-09-13T03:12:17.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.407+0000 7f27278d2b80 10 client.25114 fetch_fsmap learned FSMap version 26 2023-09-13T03:12:17.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.407+0000 7f27278d2b80 1 -- 172.21.15.64:0/4178543411 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x560015d79c10 con 0x560016014550 2023-09-13T03:12:17.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.408+0000 7f26f57fa700 1 -- 172.21.15.64:0/4178543411 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26fc004390 con 0x560016014550 2023-09-13T03:12:17.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.408+0000 7f26f57fa700 1 -- 172.21.15.64:0/4178543411 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f270402c5d0 con 0x560016014550 2023-09-13T03:12:17.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.408+0000 7f27278d2b80 10 client.25114 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:12:17.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.408+0000 7f27278d2b80 10 client.25114 resolve_mds: resolved name 'c' to daemon mds.c 2023-09-13T03:12:17.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.408+0000 7f27278d2b80 20 client.25114 populate_metadata read hostname 'smithi064' 2023-09-13T03:12:17.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.408+0000 7f27278d2b80 1 --2- 172.21.15.64:0/4178543411 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x560016085690 0x560016081ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.408+0000 7f27278d2b80 4 client.25114 mds_command: new command op to 24634 tid=0[{"prefix": "status"}] 2023-09-13T03:12:17.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.408+0000 7f27278d2b80 1 -- 172.21.15.64:0/4178543411 --> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560015f5d8d0 con 0x560016085690 2023-09-13T03:12:17.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.408+0000 7f2711d9b700 1 --2- 172.21.15.64:0/4178543411 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x560016085690 0x560016081ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.409+0000 7f2711d9b700 1 --2- 172.21.15.64:0/4178543411 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x560016085690 0x560016081ad0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d2e6daf018c78232 server_cookie=cadd3504d6164306 in_seq=0 out_seq=0 2023-09-13T03:12:17.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.409+0000 7f26f57fa700 10 client.25114 ms_handle_connect on v2:172.21.15.170:6836/1726587275 2023-09-13T03:12:17.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.409+0000 7f26f57fa700 1 -- 172.21.15.64:0/4178543411 <== mds.3 v2:172.21.15.170:6836/1726587275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x560015f5d8d0 con 0x560016085690 2023-09-13T03:12:17.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.409+0000 7f26f57fa700 10 client.25114 handle_command_reply: tid=0 2023-09-13T03:12:17.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.409+0000 7f27278d2b80 1 client.25114 shutdown 2023-09-13T03:12:17.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.409+0000 7f27278d2b80 1 -- 172.21.15.64:0/4178543411 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560016014550 msgr2=0x56001607ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:17.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.409+0000 7f27278d2b80 1 --2- 172.21.15.64:0/4178543411 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560016014550 0x56001607ff90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x560016029c80 tx=0x7f2704004b40 comp rx=0 tx=0).stop 2023-09-13T03:12:17.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.409+0000 7f27278d2b80 1 -- 172.21.15.64:0/4178543411 shutdown_connections 2023-09-13T03:12:17.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.409+0000 7f27278d2b80 1 --2- 172.21.15.64:0/4178543411 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x560016085690 0x560016081ad0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.409+0000 7f27278d2b80 1 --2- 172.21.15.64:0/4178543411 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560016014550 0x56001607ff90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.409+0000 7f27278d2b80 1 --2- 172.21.15.64:0/4178543411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560016012db0 0x5600160230d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.409+0000 7f27278d2b80 1 --2- 172.21.15.64:0/4178543411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560016013bb0 0x560016023610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.409+0000 7f26f57fa700 0 client.25114 ms_handle_reset on v2:172.21.15.170:6836/1726587275 2023-09-13T03:12:17.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.409+0000 7f27278d2b80 1 -- 172.21.15.64:0/4178543411 >> 172.21.15.64:0/4178543411 conn(0x560015f87040 msgr2=0x56001601f690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:17.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.409+0000 7f27278d2b80 1 -- 172.21.15.64:0/4178543411 shutdown_connections 2023-09-13T03:12:17.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.409+0000 7f27278d2b80 1 -- 172.21.15.64:0/4178543411 wait complete. 2023-09-13T03:12:17.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.409+0000 7f27278d2b80 20 client.25114 trim_cache size 0 max 16384 2023-09-13T03:12:17.427 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-13T03:12:17.427 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "f8dbced4-51e1-11ee-9ab7-7b867c8bd7da", 2023-09-13T03:12:17.427 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 3, 2023-09-13T03:12:17.428 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24634, 2023-09-13T03:12:17.428 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2023-09-13T03:12:17.428 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2023-09-13T03:12:17.428 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2023-09-13T03:12:17.428 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 61.199525459, 2023-09-13T03:12:17.429 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 26, 2023-09-13T03:12:17.429 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 85, 2023-09-13T03:12:17.429 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 83, 2023-09-13T03:12:17.429 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 86.659858365000005 2023-09-13T03:12:17.430 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-13T03:12:17.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.410+0000 7f2713fff700 1 -- 172.21.15.64:0/166156260 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f26f80a0750 msgr2=0x7f26f80a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:17.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.410+0000 7f2713fff700 1 --2- 172.21.15.64:0/166156260 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f26f80a0750 0x7f26f80a2c10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2704003e90 tx=0x7f2704009130 comp rx=0 tx=0).stop 2023-09-13T03:12:17.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.410+0000 7f2713fff700 1 -- 172.21.15.64:0/166156260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f270c1005f0 msgr2=0x7f270c196330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:17.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.410+0000 7f2713fff700 1 --2- 172.21.15.64:0/166156260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f270c1005f0 0x7f270c196330 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2708009fd0 tx=0x7f270800f690 comp rx=0 tx=0).stop 2023-09-13T03:12:17.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.410+0000 7f2713fff700 1 -- 172.21.15.64:0/166156260 shutdown_connections 2023-09-13T03:12:17.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.410+0000 7f2713fff700 1 --2- 172.21.15.64:0/166156260 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f26f80a0750 0x7f26f80a2c10 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.410+0000 7f2713fff700 1 --2- 172.21.15.64:0/166156260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f270c102180 0x7f270c19d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.410+0000 7f2713fff700 1 --2- 172.21.15.64:0/166156260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f270c1005f0 0x7f270c196330 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.410+0000 7f2713fff700 1 --2- 172.21.15.64:0/166156260 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f270c1017e0 0x7f270c196870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.410+0000 7f2713fff700 1 -- 172.21.15.64:0/166156260 >> 172.21.15.64:0/166156260 conn(0x7f270c0fbb40 msgr2=0x7f270c0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:17.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.410+0000 7f2713fff700 1 -- 172.21.15.64:0/166156260 shutdown_connections 2023-09-13T03:12:17.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.410+0000 7f2713fff700 1 -- 172.21.15.64:0/166156260 wait complete. 2023-09-13T03:12:17.434 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-09-13T03:12:17.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.638+0000 7ffa463ff700 1 -- 172.21.15.64:0/847184385 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa401005f0 msgr2=0x7ffa40100a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:17.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.638+0000 7ffa463ff700 1 --2- 172.21.15.64:0/847184385 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa401005f0 0x7ffa40100a10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ffa300099c0 tx=0x7ffa30009cd0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.639+0000 7ffa463ff700 1 -- 172.21.15.64:0/847184385 shutdown_connections 2023-09-13T03:12:17.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.639+0000 7ffa463ff700 1 --2- 172.21.15.64:0/847184385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa40102180 0x7ffa40108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.639+0000 7ffa463ff700 1 --2- 172.21.15.64:0/847184385 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa401005f0 0x7ffa40100a10 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.639+0000 7ffa463ff700 1 --2- 172.21.15.64:0/847184385 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa401017e0 0x7ffa40101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.639+0000 7ffa463ff700 1 -- 172.21.15.64:0/847184385 >> 172.21.15.64:0/847184385 conn(0x7ffa400fbb40 msgr2=0x7ffa400fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:17.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.639+0000 7ffa463ff700 1 -- 172.21.15.64:0/847184385 shutdown_connections 2023-09-13T03:12:17.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.639+0000 7ffa463ff700 1 -- 172.21.15.64:0/847184385 wait complete. 2023-09-13T03:12:17.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.640+0000 7ffa463ff700 1 Processor -- start 2023-09-13T03:12:17.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.640+0000 7ffa463ff700 1 -- start start 2023-09-13T03:12:17.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.640+0000 7ffa463ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa401005f0 0x7ffa40196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.640+0000 7ffa463ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa401017e0 0x7ffa401968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.640+0000 7ffa463ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa40102180 0x7ffa4019d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.640+0000 7ffa463ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4019def0 con 0x7ffa40102180 2023-09-13T03:12:17.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.640+0000 7ffa463ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4019e030 con 0x7ffa401017e0 2023-09-13T03:12:17.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.640+0000 7ffa463ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4019e1c0 con 0x7ffa401005f0 2023-09-13T03:12:17.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.640+0000 7ffa4499c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa40102180 0x7ffa4019d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.640+0000 7ffa4499c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa40102180 0x7ffa4019d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57912/0 (socket says 172.21.15.64:57912) 2023-09-13T03:12:17.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.640+0000 7ffa4499c700 1 -- 172.21.15.64:0/780947221 learned_addr learned my addr 172.21.15.64:0/780947221 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:17.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.641+0000 7ffa3f7fe700 1 --2- 172.21.15.64:0/780947221 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa401017e0 0x7ffa401968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.641+0000 7ffa3ffff700 1 --2- 172.21.15.64:0/780947221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa401005f0 0x7ffa40196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.641+0000 7ffa4499c700 1 -- 172.21.15.64:0/780947221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa401005f0 msgr2=0x7ffa40196370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:17.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.641+0000 7ffa4499c700 1 --2- 172.21.15.64:0/780947221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa401005f0 0x7ffa40196370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.641+0000 7ffa4499c700 1 -- 172.21.15.64:0/780947221 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa401017e0 msgr2=0x7ffa401968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:17.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.641+0000 7ffa4499c700 1 --2- 172.21.15.64:0/780947221 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa401017e0 0x7ffa401968b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.641+0000 7ffa4499c700 1 -- 172.21.15.64:0/780947221 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa300095f0 con 0x7ffa40102180 2023-09-13T03:12:17.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.641+0000 7ffa4499c700 1 --2- 172.21.15.64:0/780947221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa40102180 0x7ffa4019d920 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800ef30 tx=0x7ffa3800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:17.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.641+0000 7ffa3d7fa700 1 -- 172.21.15.64:0/780947221 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa38018070 con 0x7ffa40102180 2023-09-13T03:12:17.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.641+0000 7ffa463ff700 1 -- 172.21.15.64:0/780947221 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa4019e440 con 0x7ffa40102180 2023-09-13T03:12:17.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.641+0000 7ffa3d7fa700 1 -- 172.21.15.64:0/780947221 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa3800f040 con 0x7ffa40102180 2023-09-13T03:12:17.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.641+0000 7ffa3d7fa700 1 -- 172.21.15.64:0/780947221 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3800cd00 con 0x7ffa40102180 2023-09-13T03:12:17.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.641+0000 7ffa463ff700 1 -- 172.21.15.64:0/780947221 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa4019e900 con 0x7ffa40102180 2023-09-13T03:12:17.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.644+0000 7ffa3d7fa700 1 -- 172.21.15.64:0/780947221 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7ffa38003a00 con 0x7ffa40102180 2023-09-13T03:12:17.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.646+0000 7ffa59c8ab80 1 Processor -- start 2023-09-13T03:12:17.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.646+0000 7ffa3d7fa700 1 --2- 172.21.15.64:0/780947221 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ffa280a06d0 0x7ffa280a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.646+0000 7ffa59c8ab80 1 -- start start 2023-09-13T03:12:17.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.646+0000 7ffa3d7fa700 1 -- 172.21.15.64:0/780947221 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ffa380cb8c0 con 0x7ffa40102180 2023-09-13T03:12:17.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.646+0000 7ffa3f7fe700 1 --2- 172.21.15.64:0/780947221 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ffa280a06d0 0x7ffa280a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.646+0000 7ffa59c8ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56040049c020 0x56040049c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.646+0000 7ffa59c8ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56040049c980 0x56040049ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.646+0000 7ffa59c8ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040049b1e0 0x56040049b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.646+0000 7ffa59c8ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604003ef6e0 con 0x56040049b1e0 2023-09-13T03:12:17.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.646+0000 7ffa59c8ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604003ed000 con 0x56040049c980 2023-09-13T03:12:17.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.646+0000 7ffa59c8ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604003e88f0 con 0x56040049c020 2023-09-13T03:12:17.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.646+0000 7ffa4499c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040049b1e0 0x56040049b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.646+0000 7ffa3ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56040049c020 0x56040049c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.647+0000 7ffa4499c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040049b1e0 0x56040049b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57914/0 (socket says 172.21.15.64:57914) 2023-09-13T03:12:17.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.647+0000 7ffa3ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56040049c020 0x56040049c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:54062/0 (socket says 172.21.15.64:54062) 2023-09-13T03:12:17.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.647+0000 7ffa4499c700 1 -- 172.21.15.64:0/1844983935 learned_addr learned my addr 172.21.15.64:0/1844983935 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:17.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.647+0000 7ffa3f7fe700 1 --2- 172.21.15.64:0/1844983935 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56040049c980 0x56040049ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.647+0000 7ffa3ffff700 1 -- 172.21.15.64:0/1844983935 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56040049c980 msgr2=0x56040049ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:17.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.647+0000 7ffa3ffff700 1 --2- 172.21.15.64:0/1844983935 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56040049c980 0x56040049ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.647+0000 7ffa3f7fe700 1 --2- 172.21.15.64:0/780947221 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ffa280a06d0 0x7ffa280a2b90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ffa34009c70 tx=0x7ffa34011720 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:17.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.647+0000 7ffa3ffff700 1 -- 172.21.15.64:0/1844983935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040049b1e0 msgr2=0x56040049b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:17.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.647+0000 7ffa3ffff700 1 --2- 172.21.15.64:0/1844983935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040049b1e0 0x56040049b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.647+0000 7ffa3ffff700 1 -- 172.21.15.64:0/1844983935 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560400214d90 con 0x56040049c020 2023-09-13T03:12:17.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.647+0000 7ffa4499c700 1 --2- 172.21.15.64:0/1844983935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040049b1e0 0x56040049b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:17.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.647+0000 7ffa3ffff700 1 --2- 172.21.15.64:0/1844983935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56040049c020 0x56040049c440 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ffa300058e0 tx=0x7ffa30005ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=430f109897da20f7 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:17.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.647+0000 7ffa267fc700 1 -- 172.21.15.64:0/1844983935 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3000ba10 con 0x56040049c020 2023-09-13T03:12:17.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.648+0000 7ffa267fc700 1 -- 172.21.15.64:0/1844983935 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa3001e040 con 0x56040049c020 2023-09-13T03:12:17.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.648+0000 7ffa267fc700 1 -- 172.21.15.64:0/1844983935 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa30003aa0 con 0x56040049c020 2023-09-13T03:12:17.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.648+0000 7ffa59c8ab80 1 -- 172.21.15.64:0/1844983935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56040049c020 msgr2=0x56040049c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:17.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.648+0000 7ffa59c8ab80 1 --2- 172.21.15.64:0/1844983935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56040049c020 0x56040049c440 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ffa300058e0 tx=0x7ffa30005ec0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.648+0000 7ffa59c8ab80 1 -- 172.21.15.64:0/1844983935 shutdown_connections 2023-09-13T03:12:17.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.648+0000 7ffa59c8ab80 1 --2- 172.21.15.64:0/1844983935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040049b1e0 0x56040049b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.648+0000 7ffa59c8ab80 1 --2- 172.21.15.64:0/1844983935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56040049c020 0x56040049c440 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.648+0000 7ffa59c8ab80 1 --2- 172.21.15.64:0/1844983935 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56040049c980 0x56040049ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.648+0000 7ffa59c8ab80 1 -- 172.21.15.64:0/1844983935 >> 172.21.15.64:0/1844983935 conn(0x56040040b040 msgr2=0x5604004916d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:17.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.648+0000 7ffa59c8ab80 1 -- 172.21.15.64:0/1844983935 shutdown_connections 2023-09-13T03:12:17.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.648+0000 7ffa59c8ab80 1 -- 172.21.15.64:0/1844983935 wait complete. 2023-09-13T03:12:17.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.649+0000 7ffa59c8ab80 1 -- start start 2023-09-13T03:12:17.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.649+0000 7ffa59c8ab80 1 Processor -- start 2023-09-13T03:12:17.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.650+0000 7ffa59c8ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56040049b1e0 0x5604004a49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.650+0000 7ffa59c8ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56040049c020 0x5604004a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.650+0000 7ffa59c8ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040049c980 0x5604004abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.650+0000 7ffa59c8ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560400386b20 con 0x56040049c980 2023-09-13T03:12:17.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.650+0000 7ffa59c8ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560400386820 con 0x56040049c020 2023-09-13T03:12:17.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.650+0000 7ffa59c8ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56040026a8d0 con 0x56040049b1e0 2023-09-13T03:12:17.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.650+0000 7ffa59c8ab80 10 client.0 _ll_register_callbacks cb 0x5604002d0a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:12:17.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.650+0000 7ffa3f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56040049c020 0x5604004a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.650+0000 7ffa4499c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040049c980 0x5604004abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.650+0000 7ffa3ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56040049b1e0 0x5604004a49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.650+0000 7ffa3f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56040049c020 0x5604004a4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:45956/0 (socket says 172.21.15.64:45956) 2023-09-13T03:12:17.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.650+0000 7ffa3ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56040049b1e0 0x5604004a49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:54068/0 (socket says 172.21.15.64:54068) 2023-09-13T03:12:17.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.650+0000 7ffa3f7fe700 1 -- 172.21.15.64:0/2874786323 learned_addr learned my addr 172.21.15.64:0/2874786323 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:17.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.650+0000 7ffa4499c700 1 -- 172.21.15.64:0/2874786323 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56040049b1e0 msgr2=0x5604004a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:17.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.650+0000 7ffa4499c700 1 --2- 172.21.15.64:0/2874786323 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56040049b1e0 0x5604004a49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.650+0000 7ffa4499c700 1 -- 172.21.15.64:0/2874786323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56040049c020 msgr2=0x5604004a4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:17.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.650+0000 7ffa4499c700 1 --2- 172.21.15.64:0/2874786323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56040049c020 0x5604004a4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.650+0000 7ffa4499c700 1 -- 172.21.15.64:0/2874786323 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa4019e900 con 0x56040049c980 2023-09-13T03:12:17.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.650+0000 7ffa3f7fe700 1 --2- 172.21.15.64:0/2874786323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56040049c020 0x5604004a4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:17.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.650+0000 7ffa3ffff700 1 --2- 172.21.15.64:0/2874786323 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56040049b1e0 0x5604004a49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:17.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.651+0000 7ffa4499c700 1 --2- 172.21.15.64:0/2874786323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040049c980 0x5604004abf20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ffa38009c10 tx=0x7ffa38098990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7993a0c42ff2cb41 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:17.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.651+0000 7ffa1f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:12:17.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.651+0000 7ffa1f7fe700 1 -- 172.21.15.64:0/2874786323 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa380b0070 con 0x56040049c980 2023-09-13T03:12:17.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.651+0000 7ffa1f7fe700 1 -- 172.21.15.64:0/2874786323 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ffa380996b0 con 0x56040049c980 2023-09-13T03:12:17.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.651+0000 7ffa59c8ab80 1 -- 172.21.15.64:0/2874786323 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5604003404a0 con 0x56040049c980 2023-09-13T03:12:17.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.651+0000 7ffa1f7fe700 1 -- 172.21.15.64:0/2874786323 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa3800ce60 con 0x56040049c980 2023-09-13T03:12:17.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.651+0000 7ffa59c8ab80 10 client.15099 fetch_fsmap learned FSMap version 26 2023-09-13T03:12:17.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.651+0000 7ffa59c8ab80 1 -- 172.21.15.64:0/2874786323 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7ffa40109380 con 0x56040049c980 2023-09-13T03:12:17.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.651+0000 7ffa1f7fe700 1 -- 172.21.15.64:0/2874786323 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5604003404a0 con 0x56040049c980 2023-09-13T03:12:17.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.651+0000 7ffa1f7fe700 1 -- 172.21.15.64:0/2874786323 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ffa380984a0 con 0x56040049c980 2023-09-13T03:12:17.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.652+0000 7ffa59c8ab80 10 client.15099 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:12:17.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.652+0000 7ffa59c8ab80 10 client.15099 resolve_mds: resolved name 'h' to daemon mds.h 2023-09-13T03:12:17.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.652+0000 7ffa59c8ab80 20 client.15099 populate_metadata read hostname 'smithi064' 2023-09-13T03:12:17.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.652+0000 7ffa59c8ab80 1 --2- 172.21.15.64:0/2874786323 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x5604004afff0 0x5604004aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.652+0000 7ffa59c8ab80 4 client.15099 mds_command: new command op to 24635 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:12:17.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.652+0000 7ffa59c8ab80 1 -- 172.21.15.64:0/2874786323 --> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5604003fd850 con 0x5604004afff0 2023-09-13T03:12:17.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.652+0000 7ffa3ffff700 1 --2- 172.21.15.64:0/2874786323 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x5604004afff0 0x5604004aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.653+0000 7ffa3ffff700 1 --2- 172.21.15.64:0/2874786323 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x5604004afff0 0x5604004aefc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d0366ca28955f489 server_cookie=cd2eb385b560980b in_seq=0 out_seq=0 2023-09-13T03:12:17.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.653+0000 7ffa1f7fe700 10 client.15099 ms_handle_connect on v2:172.21.15.142:6836/741173268 2023-09-13T03:12:17.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.654+0000 7ffa1f7fe700 1 -- 172.21.15.64:0/2874786323 <== mds.2 v2:172.21.15.142:6836/741173268 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5604003fd850 con 0x5604004afff0 2023-09-13T03:12:17.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.654+0000 7ffa1f7fe700 10 client.15099 handle_command_reply: tid=0 2023-09-13T03:12:17.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.654+0000 7ffa59c8ab80 1 client.15099 shutdown 2023-09-13T03:12:17.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.654+0000 7ffa59c8ab80 1 -- 172.21.15.64:0/2874786323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040049c980 msgr2=0x5604004abf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:17.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.654+0000 7ffa59c8ab80 1 --2- 172.21.15.64:0/2874786323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040049c980 0x5604004abf20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ffa38009c10 tx=0x7ffa38098990 comp rx=0 tx=0).stop 2023-09-13T03:12:17.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.654+0000 7ffa59c8ab80 1 -- 172.21.15.64:0/2874786323 shutdown_connections 2023-09-13T03:12:17.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.654+0000 7ffa59c8ab80 1 --2- 172.21.15.64:0/2874786323 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x5604004afff0 0x5604004aefc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.654+0000 7ffa59c8ab80 1 --2- 172.21.15.64:0/2874786323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040049c980 0x5604004abf20 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.654+0000 7ffa59c8ab80 1 --2- 172.21.15.64:0/2874786323 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56040049b1e0 0x5604004a49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.654+0000 7ffa1f7fe700 0 client.15099 ms_handle_reset on v2:172.21.15.142:6836/741173268 2023-09-13T03:12:17.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.654+0000 7ffa59c8ab80 1 --2- 172.21.15.64:0/2874786323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56040049c020 0x5604004a4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.654+0000 7ffa59c8ab80 1 -- 172.21.15.64:0/2874786323 >> 172.21.15.64:0/2874786323 conn(0x56040040b040 msgr2=0x560400490d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:17.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.654+0000 7ffa59c8ab80 1 -- 172.21.15.64:0/2874786323 shutdown_connections 2023-09-13T03:12:17.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.654+0000 7ffa59c8ab80 1 -- 172.21.15.64:0/2874786323 wait complete. 2023-09-13T03:12:17.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.654+0000 7ffa59c8ab80 20 client.15099 trim_cache size 0 max 16384 2023-09-13T03:12:17.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.673+0000 7ffa59c8ab80 1 Processor -- start 2023-09-13T03:12:17.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.673+0000 7ffa59c8ab80 1 -- start start 2023-09-13T03:12:17.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.673+0000 7ffa59c8ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56040049b1e0 0x5604004a3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.673+0000 7ffa59c8ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040049c020 0x5604004abee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.673+0000 7ffa59c8ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56040049c980 0x5604004aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.673+0000 7ffa59c8ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604003fd850 con 0x56040049c020 2023-09-13T03:12:17.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.673+0000 7ffa59c8ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604003404a0 con 0x56040049b1e0 2023-09-13T03:12:17.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.673+0000 7ffa59c8ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604004a5b90 con 0x56040049c980 2023-09-13T03:12:17.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.673+0000 7ffa3f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040049c020 0x5604004abee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.673+0000 7ffa3ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56040049b1e0 0x5604004a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.673+0000 7ffa3f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040049c020 0x5604004abee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57936/0 (socket says 172.21.15.64:57936) 2023-09-13T03:12:17.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.673+0000 7ffa3ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56040049b1e0 0x5604004a3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:45958/0 (socket says 172.21.15.64:45958) 2023-09-13T03:12:17.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.673+0000 7ffa3f7fe700 1 -- 172.21.15.64:0/1691030177 learned_addr learned my addr 172.21.15.64:0/1691030177 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:17.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.673+0000 7ffa4499c700 1 --2- 172.21.15.64:0/1691030177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56040049c980 0x5604004aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.674+0000 7ffa3ffff700 1 -- 172.21.15.64:0/1691030177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56040049c980 msgr2=0x5604004aefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:17.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.674+0000 7ffa3ffff700 1 --2- 172.21.15.64:0/1691030177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56040049c980 0x5604004aefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.674+0000 7ffa3ffff700 1 -- 172.21.15.64:0/1691030177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040049c020 msgr2=0x5604004abee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:17.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.674+0000 7ffa3ffff700 1 --2- 172.21.15.64:0/1691030177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040049c020 0x5604004abee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.674+0000 7ffa3ffff700 1 -- 172.21.15.64:0/1691030177 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56040049b890 con 0x56040049b1e0 2023-09-13T03:12:17.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.674+0000 7ffa3f7fe700 1 --2- 172.21.15.64:0/1691030177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040049c020 0x5604004abee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:17.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.674+0000 7ffa3ffff700 1 --2- 172.21.15.64:0/1691030177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56040049b1e0 0x5604004a3640 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ffa30005f50 tx=0x7ffa3001a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f47cb2197b3254ca server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:17.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.674+0000 7ffa26ffd700 1 -- 172.21.15.64:0/1691030177 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa300052c0 con 0x56040049b1e0 2023-09-13T03:12:17.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.674+0000 7ffa26ffd700 1 -- 172.21.15.64:0/1691030177 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa3000bc40 con 0x56040049b1e0 2023-09-13T03:12:17.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.674+0000 7ffa26ffd700 1 -- 172.21.15.64:0/1691030177 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa30010690 con 0x56040049b1e0 2023-09-13T03:12:17.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.675+0000 7ffa59c8ab80 1 -- 172.21.15.64:0/1691030177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56040049b1e0 msgr2=0x5604004a3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:17.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.675+0000 7ffa59c8ab80 1 --2- 172.21.15.64:0/1691030177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56040049b1e0 0x5604004a3640 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ffa30005f50 tx=0x7ffa3001a040 comp rx=0 tx=0).stop 2023-09-13T03:12:17.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.675+0000 7ffa59c8ab80 1 -- 172.21.15.64:0/1691030177 shutdown_connections 2023-09-13T03:12:17.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.675+0000 7ffa59c8ab80 1 --2- 172.21.15.64:0/1691030177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56040049b1e0 0x5604004a3640 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.675+0000 7ffa59c8ab80 1 --2- 172.21.15.64:0/1691030177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56040049c980 0x5604004aefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.675+0000 7ffa59c8ab80 1 --2- 172.21.15.64:0/1691030177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040049c020 0x5604004abee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.675+0000 7ffa59c8ab80 1 -- 172.21.15.64:0/1691030177 >> 172.21.15.64:0/1691030177 conn(0x56040040b040 msgr2=0x5604004916a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:17.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.675+0000 7ffa59c8ab80 1 -- 172.21.15.64:0/1691030177 shutdown_connections 2023-09-13T03:12:17.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.675+0000 7ffa59c8ab80 1 -- 172.21.15.64:0/1691030177 wait complete. 2023-09-13T03:12:17.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.675+0000 7ffa59c8ab80 1 -- start start 2023-09-13T03:12:17.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.675+0000 7ffa59c8ab80 1 Processor -- start 2023-09-13T03:12:17.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.676+0000 7ffa59c8ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56040049b1e0 0x5604004a66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.676+0000 7ffa59c8ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56040049c020 0x5604004abbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.676+0000 7ffa59c8ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040049c980 0x5604004a6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.676+0000 7ffa59c8ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604004a7170 con 0x56040049c980 2023-09-13T03:12:17.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.676+0000 7ffa59c8ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604004a72e0 con 0x56040049c020 2023-09-13T03:12:17.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.676+0000 7ffa59c8ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560400504040 con 0x56040049b1e0 2023-09-13T03:12:17.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.676+0000 7ffa59c8ab80 10 client.0 _ll_register_callbacks cb 0x7ffa38098af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:12:17.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.676+0000 7ffa4499c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040049c980 0x5604004a6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.676+0000 7ffa3f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56040049c020 0x5604004abbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.676+0000 7ffa3f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56040049c020 0x5604004abbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:45968/0 (socket says 172.21.15.64:45968) 2023-09-13T03:12:17.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.676+0000 7ffa3f7fe700 1 -- 172.21.15.64:0/857982503 learned_addr learned my addr 172.21.15.64:0/857982503 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:17.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.676+0000 7ffa3ffff700 1 --2- 172.21.15.64:0/857982503 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56040049b1e0 0x5604004a66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.676+0000 7ffa4499c700 1 -- 172.21.15.64:0/857982503 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56040049b1e0 msgr2=0x5604004a66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:17.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.676+0000 7ffa4499c700 1 --2- 172.21.15.64:0/857982503 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56040049b1e0 0x5604004a66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.676+0000 7ffa4499c700 1 -- 172.21.15.64:0/857982503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56040049c020 msgr2=0x5604004abbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:17.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.676+0000 7ffa4499c700 1 --2- 172.21.15.64:0/857982503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56040049c020 0x5604004abbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.676+0000 7ffa4499c700 1 -- 172.21.15.64:0/857982503 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa3400f840 con 0x56040049c980 2023-09-13T03:12:17.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.677+0000 7ffa4499c700 1 --2- 172.21.15.64:0/857982503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040049c980 0x5604004a6c30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800ef00 tx=0x7ffa380986d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75a2c83ecd87e01f server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:17.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.677+0000 7ffa1f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:12:17.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.677+0000 7ffa1f7fe700 1 -- 172.21.15.64:0/857982503 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa380b0070 con 0x56040049c980 2023-09-13T03:12:17.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.677+0000 7ffa59c8ab80 1 -- 172.21.15.64:0/857982503 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffa3001e040 con 0x56040049c980 2023-09-13T03:12:17.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.677+0000 7ffa1f7fe700 1 -- 172.21.15.64:0/857982503 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ffa380996b0 con 0x56040049c980 2023-09-13T03:12:17.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.677+0000 7ffa1f7fe700 1 -- 172.21.15.64:0/857982503 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa380952f0 con 0x56040049c980 2023-09-13T03:12:17.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.677+0000 7ffa59c8ab80 10 client.15105 fetch_fsmap learned FSMap version 26 2023-09-13T03:12:17.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.677+0000 7ffa59c8ab80 1 -- 172.21.15.64:0/857982503 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x560400214d90 con 0x56040049c980 2023-09-13T03:12:17.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.677+0000 7ffa1f7fe700 1 -- 172.21.15.64:0/857982503 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa3001e040 con 0x56040049c980 2023-09-13T03:12:17.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.678+0000 7ffa1f7fe700 1 -- 172.21.15.64:0/857982503 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ffa38094da0 con 0x56040049c980 2023-09-13T03:12:17.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.678+0000 7ffa59c8ab80 10 client.15105 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:12:17.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.678+0000 7ffa59c8ab80 10 client.15105 resolve_mds: resolved name 'h' to daemon mds.h 2023-09-13T03:12:17.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.678+0000 7ffa59c8ab80 20 client.15105 populate_metadata read hostname 'smithi064' 2023-09-13T03:12:17.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.678+0000 7ffa59c8ab80 1 --2- 172.21.15.64:0/857982503 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x560400508f60 0x5604005053e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.678+0000 7ffa59c8ab80 4 client.15105 mds_command: new command op to 24635 tid=0[{"prefix": "status"}] 2023-09-13T03:12:17.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.678+0000 7ffa59c8ab80 1 -- 172.21.15.64:0/857982503 --> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5604003e18d0 con 0x560400508f60 2023-09-13T03:12:17.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.678+0000 7ffa3ffff700 1 --2- 172.21.15.64:0/857982503 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x560400508f60 0x5604005053e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.679+0000 7ffa3ffff700 1 --2- 172.21.15.64:0/857982503 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x560400508f60 0x5604005053e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b75bece48ace30d2 server_cookie=704232e3532edba8 in_seq=0 out_seq=0 2023-09-13T03:12:17.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.679+0000 7ffa1f7fe700 10 client.15105 ms_handle_connect on v2:172.21.15.142:6836/741173268 2023-09-13T03:12:17.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.679+0000 7ffa1f7fe700 1 -- 172.21.15.64:0/857982503 <== mds.2 v2:172.21.15.142:6836/741173268 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x5604003e18d0 con 0x560400508f60 2023-09-13T03:12:17.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.679+0000 7ffa1f7fe700 10 client.15105 handle_command_reply: tid=0 2023-09-13T03:12:17.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.679+0000 7ffa59c8ab80 1 client.15105 shutdown 2023-09-13T03:12:17.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.679+0000 7ffa59c8ab80 1 -- 172.21.15.64:0/857982503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040049c980 msgr2=0x5604004a6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:17.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.679+0000 7ffa59c8ab80 1 --2- 172.21.15.64:0/857982503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040049c980 0x5604004a6c30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800ef00 tx=0x7ffa380986d0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.679+0000 7ffa59c8ab80 1 -- 172.21.15.64:0/857982503 shutdown_connections 2023-09-13T03:12:17.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.679+0000 7ffa59c8ab80 1 --2- 172.21.15.64:0/857982503 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x560400508f60 0x5604005053e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.679+0000 7ffa59c8ab80 1 --2- 172.21.15.64:0/857982503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040049c980 0x5604004a6c30 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.679+0000 7ffa1f7fe700 0 client.15105 ms_handle_reset on v2:172.21.15.142:6836/741173268 2023-09-13T03:12:17.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.679+0000 7ffa59c8ab80 1 --2- 172.21.15.64:0/857982503 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56040049b1e0 0x5604004a66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.679+0000 7ffa59c8ab80 1 --2- 172.21.15.64:0/857982503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56040049c020 0x5604004abbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.679+0000 7ffa59c8ab80 1 -- 172.21.15.64:0/857982503 >> 172.21.15.64:0/857982503 conn(0x56040040b040 msgr2=0x5604004a2810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:17.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.679+0000 7ffa59c8ab80 1 -- 172.21.15.64:0/857982503 shutdown_connections 2023-09-13T03:12:17.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.679+0000 7ffa59c8ab80 1 -- 172.21.15.64:0/857982503 wait complete. 2023-09-13T03:12:17.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.679+0000 7ffa59c8ab80 20 client.15105 trim_cache size 0 max 16384 2023-09-13T03:12:17.697 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-13T03:12:17.697 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "f8dbced4-51e1-11ee-9ab7-7b867c8bd7da", 2023-09-13T03:12:17.697 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 2, 2023-09-13T03:12:17.698 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24635, 2023-09-13T03:12:17.698 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2023-09-13T03:12:17.698 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2023-09-13T03:12:17.698 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2023-09-13T03:12:17.698 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 62.478884383, 2023-09-13T03:12:17.699 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 26, 2023-09-13T03:12:17.699 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 85, 2023-09-13T03:12:17.699 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 83, 2023-09-13T03:12:17.699 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 84.719104670999997 2023-09-13T03:12:17.700 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-13T03:12:17.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.680+0000 7ffa463ff700 1 -- 172.21.15.64:0/780947221 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ffa280a06d0 msgr2=0x7ffa280a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:17.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.680+0000 7ffa463ff700 1 --2- 172.21.15.64:0/780947221 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ffa280a06d0 0x7ffa280a2b90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ffa34009c70 tx=0x7ffa34011720 comp rx=0 tx=0).stop 2023-09-13T03:12:17.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.680+0000 7ffa463ff700 1 -- 172.21.15.64:0/780947221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa40102180 msgr2=0x7ffa4019d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:17.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.680+0000 7ffa463ff700 1 --2- 172.21.15.64:0/780947221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa40102180 0x7ffa4019d920 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800ef30 tx=0x7ffa3800c3a0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.680+0000 7ffa463ff700 1 -- 172.21.15.64:0/780947221 shutdown_connections 2023-09-13T03:12:17.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.680+0000 7ffa463ff700 1 --2- 172.21.15.64:0/780947221 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ffa280a06d0 0x7ffa280a2b90 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.680+0000 7ffa463ff700 1 --2- 172.21.15.64:0/780947221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa40102180 0x7ffa4019d920 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.680+0000 7ffa463ff700 1 --2- 172.21.15.64:0/780947221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa401005f0 0x7ffa40196370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.680+0000 7ffa463ff700 1 --2- 172.21.15.64:0/780947221 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa401017e0 0x7ffa401968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.680+0000 7ffa463ff700 1 -- 172.21.15.64:0/780947221 >> 172.21.15.64:0/780947221 conn(0x7ffa400fbb40 msgr2=0x7ffa400fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:17.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.680+0000 7ffa463ff700 1 -- 172.21.15.64:0/780947221 shutdown_connections 2023-09-13T03:12:17.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.680+0000 7ffa463ff700 1 -- 172.21.15.64:0/780947221 wait complete. 2023-09-13T03:12:17.703 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:17 smithi064 ceph-mon[97675]: pgmap v312: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-13T03:12:17.704 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:17 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:12:17.704 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:17 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:12:17.704 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:17 smithi064 ceph-mon[97675]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:12:17.704 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:17 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/160144837' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:12:17.705 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-09-13T03:12:17.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.907+0000 7f47ce5ff700 1 -- 172.21.15.64:0/2930873751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47c81017e0 msgr2=0x7f47c8101c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:17.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.907+0000 7f47ce5ff700 1 --2- 172.21.15.64:0/2930873751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47c81017e0 0x7f47c8101c40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f47bc0099c0 tx=0x7f47bc009cd0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.907+0000 7f47ce5ff700 1 -- 172.21.15.64:0/2930873751 shutdown_connections 2023-09-13T03:12:17.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.907+0000 7f47ce5ff700 1 --2- 172.21.15.64:0/2930873751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47c8102180 0x7f47c8108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.907+0000 7f47ce5ff700 1 --2- 172.21.15.64:0/2930873751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47c81005f0 0x7f47c8100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.907+0000 7f47ce5ff700 1 --2- 172.21.15.64:0/2930873751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47c81017e0 0x7f47c8101c40 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.907+0000 7f47ce5ff700 1 -- 172.21.15.64:0/2930873751 >> 172.21.15.64:0/2930873751 conn(0x7f47c80fbb40 msgr2=0x7f47c80fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:17.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.908+0000 7f47ce5ff700 1 -- 172.21.15.64:0/2930873751 shutdown_connections 2023-09-13T03:12:17.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.908+0000 7f47ce5ff700 1 -- 172.21.15.64:0/2930873751 wait complete. 2023-09-13T03:12:17.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:17 smithi170 ceph-mon[134328]: pgmap v312: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-13T03:12:17.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:17 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:12:17.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:17 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:12:17.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:17 smithi170 ceph-mon[134328]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:12:17.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:17 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/160144837' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:12:17.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.909+0000 7f47ce5ff700 1 Processor -- start 2023-09-13T03:12:17.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.909+0000 7f47ce5ff700 1 -- start start 2023-09-13T03:12:17.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.910+0000 7f47ce5ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47c81005f0 0x7f47c80743b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.910+0000 7f47ce5ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47c81017e0 0x7f47c80729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.910+0000 7f47ce5ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47c8102180 0x7f47c8072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.910+0000 7f47ce5ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c80736d0 con 0x7f47c81017e0 2023-09-13T03:12:17.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.910+0000 7f47ce5ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c8073840 con 0x7f47c81005f0 2023-09-13T03:12:17.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.910+0000 7f47ce5ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c81a03a0 con 0x7f47c8102180 2023-09-13T03:12:17.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.910+0000 7f47c77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47c81017e0 0x7f47c80729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.910+0000 7f47c7fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47c81005f0 0x7f47c80743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.910+0000 7f47c77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47c81017e0 0x7f47c80729f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57970/0 (socket says 172.21.15.64:57970) 2023-09-13T03:12:17.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.910+0000 7f47c77fe700 1 -- 172.21.15.64:0/2850802827 learned_addr learned my addr 172.21.15.64:0/2850802827 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:17.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.910+0000 7f47ccb9c700 1 --2- 172.21.15.64:0/2850802827 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47c8102180 0x7f47c8072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.910+0000 7f47c77fe700 1 -- 172.21.15.64:0/2850802827 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47c8102180 msgr2=0x7f47c8072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:17.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.910+0000 7f47c77fe700 1 --2- 172.21.15.64:0/2850802827 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47c8102180 0x7f47c8072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.910+0000 7f47c77fe700 1 -- 172.21.15.64:0/2850802827 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47c81005f0 msgr2=0x7f47c80743b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:17.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.910+0000 7f47c77fe700 1 --2- 172.21.15.64:0/2850802827 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47c81005f0 0x7f47c80743b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.910+0000 7f47c77fe700 1 -- 172.21.15.64:0/2850802827 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c0009c40 con 0x7f47c81017e0 2023-09-13T03:12:17.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.911+0000 7f47c77fe700 1 --2- 172.21.15.64:0/2850802827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47c81017e0 0x7f47c80729f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f47bc009990 tx=0x7f47bc00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:17.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.911+0000 7f47c57fa700 1 -- 172.21.15.64:0/2850802827 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc01d070 con 0x7f47c81017e0 2023-09-13T03:12:17.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.911+0000 7f47ce5ff700 1 -- 172.21.15.64:0/2850802827 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47bc0095f0 con 0x7f47c81017e0 2023-09-13T03:12:17.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.911+0000 7f47c57fa700 1 -- 172.21.15.64:0/2850802827 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47bc0054f0 con 0x7f47c81017e0 2023-09-13T03:12:17.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.911+0000 7f47c57fa700 1 -- 172.21.15.64:0/2850802827 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc004a40 con 0x7f47c81017e0 2023-09-13T03:12:17.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.911+0000 7f47ce5ff700 1 -- 172.21.15.64:0/2850802827 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c81a0910 con 0x7f47c81017e0 2023-09-13T03:12:17.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.914+0000 7f47c57fa700 1 -- 172.21.15.64:0/2850802827 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f47bc004c00 con 0x7f47c81017e0 2023-09-13T03:12:17.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.916+0000 7f47e1edcb80 1 Processor -- start 2023-09-13T03:12:17.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.916+0000 7f47e1edcb80 1 -- start start 2023-09-13T03:12:17.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.917+0000 7f47c57fa700 1 --2- 172.21.15.64:0/2850802827 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f47b00a06d0 0x7f47b00a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.917+0000 7f47e1edcb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558334e1fbb0 0x558334e1ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.917+0000 7f47e1edcb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558334e20550 0x558334e1e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.917+0000 7f47e1edcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558334e1edb0 0x558334e1f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.917+0000 7f47e1edcb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x558334d776e0 con 0x558334e20550 2023-09-13T03:12:17.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.917+0000 7f47e1edcb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558334d75000 con 0x558334e1fbb0 2023-09-13T03:12:17.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.917+0000 7f47e1edcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558334d708f0 con 0x558334e1edb0 2023-09-13T03:12:17.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.917+0000 7f47c57fa700 1 -- 172.21.15.64:0/2850802827 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f47bc0ccaa0 con 0x7f47c81017e0 2023-09-13T03:12:17.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.917+0000 7f47ccb9c700 1 --2- 172.21.15.64:0/2850802827 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f47b00a06d0 0x7f47b00a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.917+0000 7f47c7fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558334e1fbb0 0x558334e1ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.917+0000 7f47c77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558334e20550 0x558334e1e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.917+0000 7f47c7fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558334e1fbb0 0x558334e1ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:45994/0 (socket says 172.21.15.64:45994) 2023-09-13T03:12:17.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.917+0000 7f47c7fff700 1 -- 172.21.15.64:0/2668152464 learned_addr learned my addr 172.21.15.64:0/2668152464 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:17.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.917+0000 7f47ccb9c700 1 --2- 172.21.15.64:0/2668152464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558334e1edb0 0x558334e1f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.918+0000 7f47c77fe700 1 -- 172.21.15.64:0/2668152464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558334e1edb0 msgr2=0x558334e1f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:17.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.918+0000 7f47c77fe700 1 --2- 172.21.15.64:0/2668152464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558334e1edb0 0x558334e1f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.918+0000 7f47c77fe700 1 -- 172.21.15.64:0/2668152464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558334e1fbb0 msgr2=0x558334e1ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:17.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.918+0000 7f47c77fe700 1 --2- 172.21.15.64:0/2668152464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558334e1fbb0 0x558334e1ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.918+0000 7f47c77fe700 1 -- 172.21.15.64:0/2668152464 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558334b9cd90 con 0x558334e20550 2023-09-13T03:12:17.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.918+0000 7f47c7fff700 1 --2- 172.21.15.64:0/2668152464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558334e1fbb0 0x558334e1ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:17.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.918+0000 7f47ccb9c700 1 --2- 172.21.15.64:0/2850802827 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f47b00a06d0 0x7f47b00a2b90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f47c8074a10 tx=0x7f47c000fde0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:17.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.918+0000 7f47c77fe700 1 --2- 172.21.15.64:0/2668152464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558334e20550 0x558334e1e7b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f47bc0ca420 tx=0x7f47bc096600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f2db84f48894d4c server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:17.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.918+0000 7f47ae7fc700 1 -- 172.21.15.64:0/2668152464 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc02aa00 con 0x558334e20550 2023-09-13T03:12:17.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.918+0000 7f47ae7fc700 1 -- 172.21.15.64:0/2668152464 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47bc02a5f0 con 0x558334e20550 2023-09-13T03:12:17.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.918+0000 7f47ae7fc700 1 -- 172.21.15.64:0/2668152464 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc09a190 con 0x558334e20550 2023-09-13T03:12:17.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.919+0000 7f47e1edcb80 1 -- 172.21.15.64:0/2668152464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558334e20550 msgr2=0x558334e1e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:17.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.919+0000 7f47e1edcb80 1 --2- 172.21.15.64:0/2668152464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558334e20550 0x558334e1e7b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f47bc0ca420 tx=0x7f47bc096600 comp rx=0 tx=0).stop 2023-09-13T03:12:17.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.919+0000 7f47e1edcb80 1 -- 172.21.15.64:0/2668152464 shutdown_connections 2023-09-13T03:12:17.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.919+0000 7f47e1edcb80 1 --2- 172.21.15.64:0/2668152464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558334e1fbb0 0x558334e1ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.919+0000 7f47e1edcb80 1 --2- 172.21.15.64:0/2668152464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558334e1edb0 0x558334e1f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.919+0000 7f47e1edcb80 1 --2- 172.21.15.64:0/2668152464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558334e20550 0x558334e1e7b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.919+0000 7f47e1edcb80 1 -- 172.21.15.64:0/2668152464 >> 172.21.15.64:0/2668152464 conn(0x558334d93040 msgr2=0x558334e196d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:17.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.919+0000 7f47e1edcb80 1 -- 172.21.15.64:0/2668152464 shutdown_connections 2023-09-13T03:12:17.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.919+0000 7f47e1edcb80 1 -- 172.21.15.64:0/2668152464 wait complete. 2023-09-13T03:12:17.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.919+0000 7f47e1edcb80 1 -- start start 2023-09-13T03:12:17.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.920+0000 7f47e1edcb80 1 Processor -- start 2023-09-13T03:12:17.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.920+0000 7f47e1edcb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558334e1edb0 0x558334e281e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.920+0000 7f47e1edcb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558334e1fbb0 0x558334e28720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.920+0000 7f47e1edcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558334e20550 0x558334e2f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.920+0000 7f47e1edcb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x558334d0eb20 con 0x558334e1edb0 2023-09-13T03:12:17.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.920+0000 7f47e1edcb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558334d0e820 con 0x558334e1fbb0 2023-09-13T03:12:17.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.920+0000 7f47e1edcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558334bf28d0 con 0x558334e20550 2023-09-13T03:12:17.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.920+0000 7f47e1edcb80 10 client.0 _ll_register_callbacks cb 0x558334c58a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:12:17.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.920+0000 7f47c7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558334e1edb0 0x558334e281e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.920+0000 7f47c7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558334e1edb0 0x558334e281e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57980/0 (socket says 172.21.15.64:57980) 2023-09-13T03:12:17.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.920+0000 7f47c7fff700 1 -- 172.21.15.64:0/2012094555 learned_addr learned my addr 172.21.15.64:0/2012094555 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:17.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.921+0000 7f47c77fe700 1 --2- 172.21.15.64:0/2012094555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558334e1fbb0 0x558334e28720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.921+0000 7f47ccb9c700 1 --2- 172.21.15.64:0/2012094555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558334e20550 0x558334e2f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.921+0000 7f47c7fff700 1 -- 172.21.15.64:0/2012094555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558334e20550 msgr2=0x558334e2f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:17.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.921+0000 7f47c7fff700 1 --2- 172.21.15.64:0/2012094555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558334e20550 0x558334e2f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.921+0000 7f47c7fff700 1 -- 172.21.15.64:0/2012094555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558334e1fbb0 msgr2=0x558334e28720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:17.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.921+0000 7f47c7fff700 1 --2- 172.21.15.64:0/2012094555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558334e1fbb0 0x558334e28720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.921+0000 7f47c7fff700 1 -- 172.21.15.64:0/2012094555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b8009960 con 0x558334e1edb0 2023-09-13T03:12:17.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.921+0000 7f47c77fe700 1 --2- 172.21.15.64:0/2012094555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558334e1fbb0 0x558334e28720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:12:17.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.921+0000 7f47c7fff700 1 --2- 172.21.15.64:0/2012094555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558334e1edb0 0x558334e281e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f47b8012960 tx=0x7f47b8012d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39cac8d9796619f3 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:17.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.921+0000 7f47a77fe700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:12:17.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.921+0000 7f47a77fe700 1 -- 172.21.15.64:0/2012094555 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b8012ea0 con 0x558334e1edb0 2023-09-13T03:12:17.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.921+0000 7f47e1edcb80 1 -- 172.21.15.64:0/2012094555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558334cc84a0 con 0x558334e1edb0 2023-09-13T03:12:17.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.921+0000 7f47a77fe700 1 -- 172.21.15.64:0/2012094555 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f47b800f830 con 0x558334e1edb0 2023-09-13T03:12:17.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.922+0000 7f47a77fe700 1 -- 172.21.15.64:0/2012094555 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47b8017070 con 0x558334e1edb0 2023-09-13T03:12:17.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.922+0000 7f47e1edcb80 10 client.15117 fetch_fsmap learned FSMap version 26 2023-09-13T03:12:17.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.922+0000 7f47e1edcb80 1 -- 172.21.15.64:0/2012094555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f47c81a0910 con 0x558334e1edb0 2023-09-13T03:12:17.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.922+0000 7f47a77fe700 1 -- 172.21.15.64:0/2012094555 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558334cc84a0 con 0x558334e1edb0 2023-09-13T03:12:17.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.922+0000 7f47a77fe700 1 -- 172.21.15.64:0/2012094555 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f47b8025050 con 0x558334e1edb0 2023-09-13T03:12:17.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.922+0000 7f47e1edcb80 10 client.15117 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:12:17.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.922+0000 7f47e1edcb80 10 client.15117 resolve_mds: resolved name 'i' to daemon mds.i 2023-09-13T03:12:17.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.922+0000 7f47e1edcb80 20 client.15117 populate_metadata read hostname 'smithi064' 2023-09-13T03:12:17.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.922+0000 7f47e1edcb80 1 --2- 172.21.15.64:0/2012094555 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x558334e33940 0x558334e328c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.922+0000 7f47e1edcb80 4 client.15117 mds_command: new command op to 24649 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:12:17.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.922+0000 7f47e1edcb80 1 -- 172.21.15.64:0/2012094555 --> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558334d85850 con 0x558334e33940 2023-09-13T03:12:17.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.923+0000 7f47c77fe700 1 --2- 172.21.15.64:0/2012094555 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x558334e33940 0x558334e328c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.923+0000 7f47c77fe700 1 --2- 172.21.15.64:0/2012094555 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x558334e33940 0x558334e328c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a3ae67970f6eebbc server_cookie=fa0b6b7d9f525f0b in_seq=0 out_seq=0 2023-09-13T03:12:17.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.923+0000 7f47a77fe700 10 client.15117 ms_handle_connect on v2:172.21.15.170:6838/2189703596 2023-09-13T03:12:17.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.924+0000 7f47a77fe700 1 -- 172.21.15.64:0/2012094555 <== mds.1 v2:172.21.15.170:6838/2189703596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558334d85850 con 0x558334e33940 2023-09-13T03:12:17.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.924+0000 7f47a77fe700 10 client.15117 handle_command_reply: tid=0 2023-09-13T03:12:17.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.924+0000 7f47e1edcb80 1 client.15117 shutdown 2023-09-13T03:12:17.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.924+0000 7f47e1edcb80 1 -- 172.21.15.64:0/2012094555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558334e1edb0 msgr2=0x558334e281e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:17.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.924+0000 7f47e1edcb80 1 --2- 172.21.15.64:0/2012094555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558334e1edb0 0x558334e281e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f47b8012960 tx=0x7f47b8012d20 comp rx=0 tx=0).stop 2023-09-13T03:12:17.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.924+0000 7f47e1edcb80 1 -- 172.21.15.64:0/2012094555 shutdown_connections 2023-09-13T03:12:17.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.924+0000 7f47e1edcb80 1 --2- 172.21.15.64:0/2012094555 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x558334e33940 0x558334e328c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.924+0000 7f47e1edcb80 1 --2- 172.21.15.64:0/2012094555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558334e20550 0x558334e2f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.924+0000 7f47e1edcb80 1 --2- 172.21.15.64:0/2012094555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558334e1edb0 0x558334e281e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.924+0000 7f47a77fe700 0 client.15117 ms_handle_reset on v2:172.21.15.170:6838/2189703596 2023-09-13T03:12:17.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.924+0000 7f47e1edcb80 1 --2- 172.21.15.64:0/2012094555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558334e1fbb0 0x558334e28720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.924+0000 7f47e1edcb80 1 -- 172.21.15.64:0/2012094555 >> 172.21.15.64:0/2012094555 conn(0x558334d93040 msgr2=0x558334e18d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:17.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.924+0000 7f47e1edcb80 1 -- 172.21.15.64:0/2012094555 shutdown_connections 2023-09-13T03:12:17.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.924+0000 7f47e1edcb80 1 -- 172.21.15.64:0/2012094555 wait complete. 2023-09-13T03:12:17.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.925+0000 7f47e1edcb80 20 client.15117 trim_cache size 0 max 16384 2023-09-13T03:12:17.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.943+0000 7f47e1edcb80 1 Processor -- start 2023-09-13T03:12:17.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.943+0000 7f47e1edcb80 1 -- start start 2023-09-13T03:12:17.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.943+0000 7f47e1edcb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558334e1edb0 0x558334e328c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.943+0000 7f47e1edcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558334e1fbb0 0x558334e2f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.943+0000 7f47e1edcb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558334e20550 0x558334e26310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.943+0000 7f47e1edcb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x558334bfede0 con 0x558334e20550 2023-09-13T03:12:17.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.943+0000 7f47e1edcb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558334d39f70 con 0x558334e1edb0 2023-09-13T03:12:17.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.943+0000 7f47e1edcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558334d2ced0 con 0x558334e1fbb0 2023-09-13T03:12:17.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.943+0000 7f47ccb9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558334e20550 0x558334e26310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.943+0000 7f47ccb9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558334e20550 0x558334e26310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57982/0 (socket says 172.21.15.64:57982) 2023-09-13T03:12:17.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.944+0000 7f47ccb9c700 1 -- 172.21.15.64:0/2113156277 learned_addr learned my addr 172.21.15.64:0/2113156277 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:17.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.944+0000 7f47c7fff700 1 --2- 172.21.15.64:0/2113156277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558334e1edb0 0x558334e328c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.944+0000 7f47c77fe700 1 --2- 172.21.15.64:0/2113156277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558334e1fbb0 0x558334e2f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.944+0000 7f47c77fe700 1 -- 172.21.15.64:0/2113156277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558334e1edb0 msgr2=0x558334e328c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:17.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.944+0000 7f47c77fe700 1 --2- 172.21.15.64:0/2113156277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558334e1edb0 0x558334e328c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.944+0000 7f47c77fe700 1 -- 172.21.15.64:0/2113156277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558334e20550 msgr2=0x558334e26310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:17.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.944+0000 7f47c77fe700 1 --2- 172.21.15.64:0/2113156277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558334e20550 0x558334e26310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.944+0000 7f47c77fe700 1 -- 172.21.15.64:0/2113156277 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558334b85c10 con 0x558334e1fbb0 2023-09-13T03:12:17.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.944+0000 7f47ccb9c700 1 --2- 172.21.15.64:0/2113156277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558334e20550 0x558334e26310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:17.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.944+0000 7f47c77fe700 1 --2- 172.21.15.64:0/2113156277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558334e1fbb0 0x558334e2f720 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00b580 tx=0x7f47bc005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37744f320c2636a2 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:17.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.944+0000 7f47aeffd700 1 -- 172.21.15.64:0/2113156277 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc0056e0 con 0x558334e1fbb0 2023-09-13T03:12:17.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.944+0000 7f47aeffd700 1 -- 172.21.15.64:0/2113156277 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47bc0a00b0 con 0x558334e1fbb0 2023-09-13T03:12:17.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.944+0000 7f47aeffd700 1 -- 172.21.15.64:0/2113156277 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc00fb80 con 0x558334e1fbb0 2023-09-13T03:12:17.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.945+0000 7f47e1edcb80 1 -- 172.21.15.64:0/2113156277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558334e1fbb0 msgr2=0x558334e2f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:17.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.945+0000 7f47e1edcb80 1 --2- 172.21.15.64:0/2113156277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558334e1fbb0 0x558334e2f720 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00b580 tx=0x7f47bc005c00 comp rx=0 tx=0).stop 2023-09-13T03:12:17.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.945+0000 7f47e1edcb80 1 -- 172.21.15.64:0/2113156277 shutdown_connections 2023-09-13T03:12:17.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.945+0000 7f47e1edcb80 1 --2- 172.21.15.64:0/2113156277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558334e1edb0 0x558334e328c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.945+0000 7f47e1edcb80 1 --2- 172.21.15.64:0/2113156277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558334e20550 0x558334e26310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.945+0000 7f47e1edcb80 1 --2- 172.21.15.64:0/2113156277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558334e1fbb0 0x558334e2f720 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.945+0000 7f47e1edcb80 1 -- 172.21.15.64:0/2113156277 >> 172.21.15.64:0/2113156277 conn(0x558334d93040 msgr2=0x558334e26fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:17.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.945+0000 7f47e1edcb80 1 -- 172.21.15.64:0/2113156277 shutdown_connections 2023-09-13T03:12:17.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.945+0000 7f47e1edcb80 1 -- 172.21.15.64:0/2113156277 wait complete. 2023-09-13T03:12:17.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.945+0000 7f47e1edcb80 1 -- start start 2023-09-13T03:12:17.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.946+0000 7f47e1edcb80 1 Processor -- start 2023-09-13T03:12:17.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.946+0000 7f47e1edcb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558334e1edb0 0x558334e2a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.946+0000 7f47e1edcb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558334e1fbb0 0x558334e2a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.946+0000 7f47e1edcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558334e20550 0x558334e87990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.946+0000 7f47e1edcb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x558334e2aed0 con 0x558334e1edb0 2023-09-13T03:12:17.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.946+0000 7f47e1edcb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558334e2b040 con 0x558334e1fbb0 2023-09-13T03:12:17.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.946+0000 7f47e1edcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558334e87ed0 con 0x558334e20550 2023-09-13T03:12:17.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.946+0000 7f47e1edcb80 10 client.0 _ll_register_callbacks cb 0x7f47b8012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:12:17.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.946+0000 7f47c7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558334e1edb0 0x558334e2a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.946+0000 7f47c77fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558334e1fbb0 0x558334e2a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.947+0000 7f47c7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558334e1edb0 0x558334e2a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57992/0 (socket says 172.21.15.64:57992) 2023-09-13T03:12:17.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.947+0000 7f47c7fff700 1 -- 172.21.15.64:0/2714904495 learned_addr learned my addr 172.21.15.64:0/2714904495 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:17.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.947+0000 7f47ccb9c700 1 --2- 172.21.15.64:0/2714904495 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558334e20550 0x558334e87990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.947+0000 7f47c77fe700 1 -- 172.21.15.64:0/2714904495 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558334e20550 msgr2=0x558334e87990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:17.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.947+0000 7f47c77fe700 1 --2- 172.21.15.64:0/2714904495 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558334e20550 0x558334e87990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.947+0000 7f47c77fe700 1 -- 172.21.15.64:0/2714904495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558334e1edb0 msgr2=0x558334e2a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:17.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.947+0000 7f47c77fe700 1 --2- 172.21.15.64:0/2714904495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558334e1edb0 0x558334e2a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.947+0000 7f47c77fe700 1 -- 172.21.15.64:0/2714904495 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c0009030 con 0x558334e1fbb0 2023-09-13T03:12:17.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.947+0000 7f47c7fff700 1 --2- 172.21.15.64:0/2714904495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558334e1edb0 0x558334e2a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:17.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.947+0000 7f47c77fe700 1 --2- 172.21.15.64:0/2714904495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558334e1fbb0 0x558334e2a990 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f47bc0b2000 tx=0x7f47bc099b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64b9ced832900200 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:17.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.947+0000 7f47a77fe700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:12:17.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.947+0000 7f47a77fe700 1 -- 172.21.15.64:0/2714904495 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc0056e0 con 0x558334e1fbb0 2023-09-13T03:12:17.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.947+0000 7f47a77fe700 1 -- 172.21.15.64:0/2714904495 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f47bc09dfc0 con 0x558334e1fbb0 2023-09-13T03:12:17.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.947+0000 7f47e1edcb80 1 -- 172.21.15.64:0/2714904495 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f47bc02a5f0 con 0x558334e1fbb0 2023-09-13T03:12:17.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.948+0000 7f47a77fe700 1 -- 172.21.15.64:0/2714904495 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47bc0a2170 con 0x558334e1fbb0 2023-09-13T03:12:17.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.948+0000 7f47e1edcb80 10 client.25145 fetch_fsmap learned FSMap version 26 2023-09-13T03:12:17.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.948+0000 7f47e1edcb80 1 -- 172.21.15.64:0/2714904495 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x558334b9cd90 con 0x558334e1fbb0 2023-09-13T03:12:17.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.948+0000 7f47a77fe700 1 -- 172.21.15.64:0/2714904495 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47bc02a5f0 con 0x558334e1fbb0 2023-09-13T03:12:17.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.948+0000 7f47a77fe700 1 -- 172.21.15.64:0/2714904495 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f47bc099680 con 0x558334e1fbb0 2023-09-13T03:12:17.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.948+0000 7f47e1edcb80 10 client.25145 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:12:17.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.948+0000 7f47e1edcb80 10 client.25145 resolve_mds: resolved name 'i' to daemon mds.i 2023-09-13T03:12:17.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.948+0000 7f47e1edcb80 20 client.25145 populate_metadata read hostname 'smithi064' 2023-09-13T03:12:17.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.948+0000 7f47e1edcb80 1 --2- 172.21.15.64:0/2714904495 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x558334e8cc40 0x558334e89210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:17.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.948+0000 7f47e1edcb80 4 client.25145 mds_command: new command op to 24649 tid=0[{"prefix": "status"}] 2023-09-13T03:12:17.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.948+0000 7f47e1edcb80 1 -- 172.21.15.64:0/2714904495 --> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558334d698d0 con 0x558334e8cc40 2023-09-13T03:12:17.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.949+0000 7f47c7fff700 1 --2- 172.21.15.64:0/2714904495 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x558334e8cc40 0x558334e89210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:17.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.949+0000 7f47c7fff700 1 --2- 172.21.15.64:0/2714904495 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x558334e8cc40 0x558334e89210 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d09c39ebbecfe8b1 server_cookie=6b0a493c34749562 in_seq=0 out_seq=0 2023-09-13T03:12:17.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.949+0000 7f47a77fe700 10 client.25145 ms_handle_connect on v2:172.21.15.170:6838/2189703596 2023-09-13T03:12:17.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.949+0000 7f47a77fe700 1 -- 172.21.15.64:0/2714904495 <== mds.1 v2:172.21.15.170:6838/2189703596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x558334d698d0 con 0x558334e8cc40 2023-09-13T03:12:17.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.949+0000 7f47a77fe700 10 client.25145 handle_command_reply: tid=0 2023-09-13T03:12:17.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.949+0000 7f47e1edcb80 1 client.25145 shutdown 2023-09-13T03:12:17.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.950+0000 7f47e1edcb80 1 -- 172.21.15.64:0/2714904495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558334e1fbb0 msgr2=0x558334e2a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:17.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.950+0000 7f47e1edcb80 1 --2- 172.21.15.64:0/2714904495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558334e1fbb0 0x558334e2a990 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f47bc0b2000 tx=0x7f47bc099b30 comp rx=0 tx=0).stop 2023-09-13T03:12:17.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.950+0000 7f47e1edcb80 1 -- 172.21.15.64:0/2714904495 shutdown_connections 2023-09-13T03:12:17.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.950+0000 7f47e1edcb80 1 --2- 172.21.15.64:0/2714904495 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x558334e8cc40 0x558334e89210 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.950+0000 7f47e1edcb80 1 --2- 172.21.15.64:0/2714904495 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558334e20550 0x558334e87990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.950+0000 7f47e1edcb80 1 --2- 172.21.15.64:0/2714904495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558334e1edb0 0x558334e2a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.950+0000 7f47e1edcb80 1 --2- 172.21.15.64:0/2714904495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558334e1fbb0 0x558334e2a990 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.950+0000 7f47a77fe700 0 client.25145 ms_handle_reset on v2:172.21.15.170:6838/2189703596 2023-09-13T03:12:17.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.950+0000 7f47e1edcb80 1 -- 172.21.15.64:0/2714904495 >> 172.21.15.64:0/2714904495 conn(0x558334d93040 msgr2=0x558334e328c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:17.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.950+0000 7f47e1edcb80 1 -- 172.21.15.64:0/2714904495 shutdown_connections 2023-09-13T03:12:17.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.950+0000 7f47e1edcb80 1 -- 172.21.15.64:0/2714904495 wait complete. 2023-09-13T03:12:17.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.950+0000 7f47e1edcb80 20 client.25145 trim_cache size 0 max 16384 2023-09-13T03:12:17.965 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-13T03:12:17.965 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "f8dbced4-51e1-11ee-9ab7-7b867c8bd7da", 2023-09-13T03:12:17.966 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 1, 2023-09-13T03:12:17.966 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24649, 2023-09-13T03:12:17.966 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2023-09-13T03:12:17.966 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2023-09-13T03:12:17.966 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2023-09-13T03:12:17.966 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 63.762091931, 2023-09-13T03:12:17.967 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 26, 2023-09-13T03:12:17.967 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 85, 2023-09-13T03:12:17.967 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 83, 2023-09-13T03:12:17.967 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 82.625533044999997 2023-09-13T03:12:17.967 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-13T03:12:17.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.950+0000 7f47ce5ff700 1 -- 172.21.15.64:0/2850802827 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f47b00a06d0 msgr2=0x7f47b00a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:17.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.950+0000 7f47ce5ff700 1 --2- 172.21.15.64:0/2850802827 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f47b00a06d0 0x7f47b00a2b90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f47c8074a10 tx=0x7f47c000fde0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.950+0000 7f47ce5ff700 1 -- 172.21.15.64:0/2850802827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47c81017e0 msgr2=0x7f47c80729f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:17.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.950+0000 7f47ce5ff700 1 --2- 172.21.15.64:0/2850802827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47c81017e0 0x7f47c80729f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f47bc009990 tx=0x7f47bc00f690 comp rx=0 tx=0).stop 2023-09-13T03:12:17.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.951+0000 7f47ce5ff700 1 -- 172.21.15.64:0/2850802827 shutdown_connections 2023-09-13T03:12:17.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.951+0000 7f47ce5ff700 1 --2- 172.21.15.64:0/2850802827 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f47b00a06d0 0x7f47b00a2b90 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.951+0000 7f47ce5ff700 1 --2- 172.21.15.64:0/2850802827 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47c81005f0 0x7f47c80743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.951+0000 7f47ce5ff700 1 --2- 172.21.15.64:0/2850802827 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47c8102180 0x7f47c8072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.951+0000 7f47ce5ff700 1 --2- 172.21.15.64:0/2850802827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47c81017e0 0x7f47c80729f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:17.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.951+0000 7f47ce5ff700 1 -- 172.21.15.64:0/2850802827 >> 172.21.15.64:0/2850802827 conn(0x7f47c80fbb40 msgr2=0x7f47c80fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:17.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.951+0000 7f47ce5ff700 1 -- 172.21.15.64:0/2850802827 shutdown_connections 2023-09-13T03:12:17.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:17.951+0000 7f47ce5ff700 1 -- 172.21.15.64:0/2850802827 wait complete. 2023-09-13T03:12:17.972 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-09-13T03:12:17.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:17 smithi142 ceph-mon[134082]: pgmap v312: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-13T03:12:17.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:17 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:12:17.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:17 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:12:17.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:17 smithi142 ceph-mon[134082]: from='mgr.14150 172.21.15.64:0/2842349472' entity='mgr.x' 2023-09-13T03:12:17.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:17 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/160144837' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:12:18.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.176+0000 7fab285ff700 1 -- 172.21.15.64:0/183328197 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab201017e0 msgr2=0x7fab20101c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:18.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.176+0000 7fab285ff700 1 --2- 172.21.15.64:0/183328197 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab201017e0 0x7fab20101c40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fab100099c0 tx=0x7fab10009cd0 comp rx=0 tx=0).stop 2023-09-13T03:12:18.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.176+0000 7fab285ff700 1 -- 172.21.15.64:0/183328197 shutdown_connections 2023-09-13T03:12:18.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.176+0000 7fab285ff700 1 --2- 172.21.15.64:0/183328197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab201005f0 0x7fab20100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:18.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.176+0000 7fab285ff700 1 --2- 172.21.15.64:0/183328197 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab20102180 0x7fab20108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:18.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.176+0000 7fab285ff700 1 --2- 172.21.15.64:0/183328197 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab201017e0 0x7fab20101c40 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:18.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.176+0000 7fab285ff700 1 -- 172.21.15.64:0/183328197 >> 172.21.15.64:0/183328197 conn(0x7fab200fbb40 msgr2=0x7fab200fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:18.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.176+0000 7fab285ff700 1 -- 172.21.15.64:0/183328197 shutdown_connections 2023-09-13T03:12:18.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.176+0000 7fab285ff700 1 -- 172.21.15.64:0/183328197 wait complete. 2023-09-13T03:12:18.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.177+0000 7fab285ff700 1 Processor -- start 2023-09-13T03:12:18.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.177+0000 7fab285ff700 1 -- start start 2023-09-13T03:12:18.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.177+0000 7fab285ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab201005f0 0x7fab20196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:18.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.178+0000 7fab285ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab201017e0 0x7fab20196850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:18.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.178+0000 7fab285ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab20102180 0x7fab2019d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:18.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.178+0000 7fab285ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2019de90 con 0x7fab201017e0 2023-09-13T03:12:18.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.178+0000 7fab285ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2019dfd0 con 0x7fab20102180 2023-09-13T03:12:18.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.178+0000 7fab285ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2019e160 con 0x7fab201005f0 2023-09-13T03:12:18.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.178+0000 7fab25b9a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab201017e0 0x7fab20196850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:18.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.178+0000 7fab2639b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab201005f0 0x7fab20196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:18.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.178+0000 7fab2639b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab201005f0 0x7fab20196310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:54160/0 (socket says 172.21.15.64:54160) 2023-09-13T03:12:18.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.178+0000 7fab2639b700 1 -- 172.21.15.64:0/1681791120 learned_addr learned my addr 172.21.15.64:0/1681791120 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:18.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.178+0000 7fab25b9a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab201017e0 0x7fab20196850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58006/0 (socket says 172.21.15.64:58006) 2023-09-13T03:12:18.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.178+0000 7fab26b9c700 1 --2- 172.21.15.64:0/1681791120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab20102180 0x7fab2019d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:18.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.178+0000 7fab2639b700 1 -- 172.21.15.64:0/1681791120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab20102180 msgr2=0x7fab2019d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:18.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.178+0000 7fab2639b700 1 --2- 172.21.15.64:0/1681791120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab20102180 0x7fab2019d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:18.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.178+0000 7fab2639b700 1 -- 172.21.15.64:0/1681791120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab201017e0 msgr2=0x7fab20196850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:18.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.178+0000 7fab2639b700 1 --2- 172.21.15.64:0/1681791120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab201017e0 0x7fab20196850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:18.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.178+0000 7fab2639b700 1 -- 172.21.15.64:0/1681791120 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab1c009520 con 0x7fab201005f0 2023-09-13T03:12:18.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.178+0000 7fab25b9a700 1 --2- 172.21.15.64:0/1681791120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab201017e0 0x7fab20196850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:18.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.179+0000 7fab2639b700 1 --2- 172.21.15.64:0/1681791120 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab201005f0 0x7fab20196310 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00ea50 tx=0x7fab1c00ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:18.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.179+0000 7fab177fe700 1 -- 172.21.15.64:0/1681791120 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c00cbd0 con 0x7fab201005f0 2023-09-13T03:12:18.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.179+0000 7fab285ff700 1 -- 172.21.15.64:0/1681791120 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab100095f0 con 0x7fab201005f0 2023-09-13T03:12:18.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.179+0000 7fab177fe700 1 -- 172.21.15.64:0/1681791120 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab1c00cd30 con 0x7fab201005f0 2023-09-13T03:12:18.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.179+0000 7fab177fe700 1 -- 172.21.15.64:0/1681791120 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c018680 con 0x7fab201005f0 2023-09-13T03:12:18.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.179+0000 7fab285ff700 1 -- 172.21.15.64:0/1681791120 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab2019e7c0 con 0x7fab201005f0 2023-09-13T03:12:18.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.182+0000 7fab177fe700 1 -- 172.21.15.64:0/1681791120 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fab1c010430 con 0x7fab201005f0 2023-09-13T03:12:18.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.183+0000 7fab3be36b80 1 Processor -- start 2023-09-13T03:12:18.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.183+0000 7fab3be36b80 1 -- start start 2023-09-13T03:12:18.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.184+0000 7fab3be36b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b779fff0 0x5647b77a0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:18.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.184+0000 7fab3be36b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b77a0950 0x5647b779ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:18.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.184+0000 7fab3be36b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647b779f1e0 0x5647b779f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:18.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.184+0000 7fab3be36b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647b76f36e0 con 0x5647b779fff0 2023-09-13T03:12:18.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.184+0000 7fab3be36b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647b76f1000 con 0x5647b77a0950 2023-09-13T03:12:18.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.184+0000 7fab3be36b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647b76ec8f0 con 0x5647b779f1e0 2023-09-13T03:12:18.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.184+0000 7fab177fe700 1 --2- 172.21.15.64:0/1681791120 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fab0c0a06d0 0x7fab0c0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:18.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.184+0000 7fab2639b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b77a0950 0x5647b779ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:18.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.184+0000 7fab25b9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647b779f1e0 0x5647b779f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:18.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.184+0000 7fab26b9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b779fff0 0x5647b77a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:18.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.184+0000 7fab25b9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647b779f1e0 0x5647b779f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:54174/0 (socket says 172.21.15.64:54174) 2023-09-13T03:12:18.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.184+0000 7fab25b9a700 1 -- 172.21.15.64:0/2681749343 learned_addr learned my addr 172.21.15.64:0/2681749343 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:18.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.184+0000 7fab177fe700 1 -- 172.21.15.64:0/1681791120 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fab1c014070 con 0x7fab201005f0 2023-09-13T03:12:18.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.184+0000 7fab26b9c700 1 --2- 172.21.15.64:0/1681791120 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fab0c0a06d0 0x7fab0c0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:18.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.185+0000 7fab2639b700 1 -- 172.21.15.64:0/2681749343 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647b779f1e0 msgr2=0x5647b779f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:18.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.185+0000 7fab2639b700 1 --2- 172.21.15.64:0/2681749343 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647b779f1e0 0x5647b779f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:18.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.185+0000 7fab2639b700 1 -- 172.21.15.64:0/2681749343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b779fff0 msgr2=0x5647b77a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:18.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.185+0000 7fab2639b700 1 --2- 172.21.15.64:0/2681749343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b779fff0 0x5647b77a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:18.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.185+0000 7fab2639b700 1 -- 172.21.15.64:0/2681749343 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647b7518d90 con 0x5647b77a0950 2023-09-13T03:12:18.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.185+0000 7fab25b9a700 1 --2- 172.21.15.64:0/2681749343 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647b779f1e0 0x5647b779f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:18.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.185+0000 7fab2639b700 1 --2- 172.21.15.64:0/2681749343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b77a0950 0x5647b779ec00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0c9bc0 tx=0x7fab1c06b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66766242b2822011 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:18.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.185+0000 7fab26b9c700 1 --2- 172.21.15.64:0/1681791120 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fab0c0a06d0 0x7fab0c0a2b90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fab18009ec0 tx=0x7fab1800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:18.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.185+0000 7fab14bfe700 1 -- 172.21.15.64:0/2681749343 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c065630 con 0x5647b77a0950 2023-09-13T03:12:18.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.185+0000 7fab14bfe700 1 -- 172.21.15.64:0/2681749343 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab1c099fa0 con 0x5647b77a0950 2023-09-13T03:12:18.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.185+0000 7fab14bfe700 1 -- 172.21.15.64:0/2681749343 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c0ce0d0 con 0x5647b77a0950 2023-09-13T03:12:18.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.186+0000 7fab3be36b80 1 -- 172.21.15.64:0/2681749343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b77a0950 msgr2=0x5647b779ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:18.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.186+0000 7fab3be36b80 1 --2- 172.21.15.64:0/2681749343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b77a0950 0x5647b779ec00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0c9bc0 tx=0x7fab1c06b960 comp rx=0 tx=0).stop 2023-09-13T03:12:18.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.186+0000 7fab3be36b80 1 -- 172.21.15.64:0/2681749343 shutdown_connections 2023-09-13T03:12:18.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.186+0000 7fab3be36b80 1 --2- 172.21.15.64:0/2681749343 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647b779f1e0 0x5647b779f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:18.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.186+0000 7fab3be36b80 1 --2- 172.21.15.64:0/2681749343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b779fff0 0x5647b77a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:18.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.186+0000 7fab3be36b80 1 --2- 172.21.15.64:0/2681749343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b77a0950 0x5647b779ec00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:18.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.186+0000 7fab3be36b80 1 -- 172.21.15.64:0/2681749343 >> 172.21.15.64:0/2681749343 conn(0x5647b770f040 msgr2=0x5647b7795570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:18.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.186+0000 7fab3be36b80 1 -- 172.21.15.64:0/2681749343 shutdown_connections 2023-09-13T03:12:18.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.186+0000 7fab3be36b80 1 -- 172.21.15.64:0/2681749343 wait complete. 2023-09-13T03:12:18.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.186+0000 7fab3be36b80 1 -- start start 2023-09-13T03:12:18.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.187+0000 7fab3be36b80 1 Processor -- start 2023-09-13T03:12:18.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.187+0000 7fab3be36b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b779f1e0 0x5647b77a8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:18.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.187+0000 7fab3be36b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647b779fff0 0x5647b77a8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:18.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.187+0000 7fab3be36b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b77a0950 0x5647b77b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:18.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.187+0000 7fab3be36b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647b768ab20 con 0x5647b77a0950 2023-09-13T03:12:18.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.187+0000 7fab3be36b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647b768a820 con 0x5647b779f1e0 2023-09-13T03:12:18.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.187+0000 7fab3be36b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647b756e8d0 con 0x5647b779fff0 2023-09-13T03:12:18.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.187+0000 7fab3be36b80 10 client.0 _ll_register_callbacks cb 0x5647b75d4a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:12:18.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.188+0000 7fab26b9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b77a0950 0x5647b77b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:18.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.188+0000 7fab2639b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b779f1e0 0x5647b77a8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:18.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.188+0000 7fab25b9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647b779fff0 0x5647b77a8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:18.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.188+0000 7fab2639b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b779f1e0 0x5647b77a8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:46084/0 (socket says 172.21.15.64:46084) 2023-09-13T03:12:18.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.188+0000 7fab2639b700 1 -- 172.21.15.64:0/1830994021 learned_addr learned my addr 172.21.15.64:0/1830994021 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:18.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.188+0000 7fab26b9c700 1 -- 172.21.15.64:0/1830994021 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647b779fff0 msgr2=0x5647b77a8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:18.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.188+0000 7fab26b9c700 1 --2- 172.21.15.64:0/1830994021 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647b779fff0 0x5647b77a8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:18.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.188+0000 7fab26b9c700 1 -- 172.21.15.64:0/1830994021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b779f1e0 msgr2=0x5647b77a8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:18.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.188+0000 7fab26b9c700 1 --2- 172.21.15.64:0/1830994021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b779f1e0 0x5647b77a8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:18.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.188+0000 7fab26b9c700 1 -- 172.21.15.64:0/1830994021 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab20109380 con 0x5647b77a0950 2023-09-13T03:12:18.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.188+0000 7fab25b9a700 1 --2- 172.21.15.64:0/1830994021 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647b779fff0 0x5647b77a8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:18.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.188+0000 7fab2639b700 1 --2- 172.21.15.64:0/1830994021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b779f1e0 0x5647b77a8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:18.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.188+0000 7fab26b9c700 1 --2- 172.21.15.64:0/1830994021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b77a0950 0x5647b77b0000 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fab18003d50 tx=0x7fab18004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e3ea46b2c72a6e server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:18.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.188+0000 7fab09ffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:12:18.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.189+0000 7fab09ffb700 1 -- 172.21.15.64:0/1830994021 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab18021070 con 0x5647b77a0950 2023-09-13T03:12:18.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.189+0000 7fab3be36b80 1 -- 172.21.15.64:0/1830994021 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5647b76444a0 con 0x5647b77a0950 2023-09-13T03:12:18.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.189+0000 7fab09ffb700 1 -- 172.21.15.64:0/1830994021 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fab18018db0 con 0x5647b77a0950 2023-09-13T03:12:18.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.189+0000 7fab09ffb700 1 -- 172.21.15.64:0/1830994021 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab1800e030 con 0x5647b77a0950 2023-09-13T03:12:18.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.189+0000 7fab3be36b80 10 client.15135 fetch_fsmap learned FSMap version 26 2023-09-13T03:12:18.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.189+0000 7fab3be36b80 1 -- 172.21.15.64:0/1830994021 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7fab2019e7c0 con 0x5647b77a0950 2023-09-13T03:12:18.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.189+0000 7fab09ffb700 1 -- 172.21.15.64:0/1830994021 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5647b76444a0 con 0x5647b77a0950 2023-09-13T03:12:18.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.189+0000 7fab09ffb700 1 -- 172.21.15.64:0/1830994021 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fab18031480 con 0x5647b77a0950 2023-09-13T03:12:18.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.189+0000 7fab3be36b80 10 client.15135 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:12:18.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.189+0000 7fab3be36b80 10 client.15135 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-13T03:12:18.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.190+0000 7fab3be36b80 20 client.15135 populate_metadata read hostname 'smithi064' 2023-09-13T03:12:18.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.190+0000 7fab3be36b80 1 --2- 172.21.15.64:0/1830994021 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5647b77b4150 0x5647b77b3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:18.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.190+0000 7fab3be36b80 4 client.15135 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:12:18.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.190+0000 7fab3be36b80 1 -- 172.21.15.64:0/1830994021 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5647b7701850 con 0x5647b77b4150 2023-09-13T03:12:18.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.190+0000 7fab2639b700 1 --2- 172.21.15.64:0/1830994021 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5647b77b4150 0x5647b77b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:18.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.190+0000 7fab2639b700 1 --2- 172.21.15.64:0/1830994021 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5647b77b4150 0x5647b77b3100 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c38ea15283cb742 server_cookie=ba49d21d9fdee2f3 in_seq=0 out_seq=0 2023-09-13T03:12:18.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.190+0000 7fab09ffb700 10 client.15135 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:12:18.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.191+0000 7fab09ffb700 1 -- 172.21.15.64:0/1830994021 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5647b7701850 con 0x5647b77b4150 2023-09-13T03:12:18.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.191+0000 7fab09ffb700 10 client.15135 handle_command_reply: tid=0 2023-09-13T03:12:18.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.191+0000 7fab3be36b80 1 client.15135 shutdown 2023-09-13T03:12:18.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.192+0000 7fab3be36b80 1 -- 172.21.15.64:0/1830994021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b77a0950 msgr2=0x5647b77b0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:18.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.192+0000 7fab3be36b80 1 --2- 172.21.15.64:0/1830994021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b77a0950 0x5647b77b0000 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fab18003d50 tx=0x7fab18004ad0 comp rx=0 tx=0).stop 2023-09-13T03:12:18.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.192+0000 7fab3be36b80 1 -- 172.21.15.64:0/1830994021 shutdown_connections 2023-09-13T03:12:18.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.192+0000 7fab3be36b80 1 --2- 172.21.15.64:0/1830994021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b779f1e0 0x5647b77a8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:18.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.192+0000 7fab3be36b80 1 --2- 172.21.15.64:0/1830994021 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5647b77b4150 0x5647b77b3100 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:18.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.192+0000 7fab3be36b80 1 --2- 172.21.15.64:0/1830994021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b77a0950 0x5647b77b0000 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:18.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.192+0000 7fab3be36b80 1 --2- 172.21.15.64:0/1830994021 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647b779fff0 0x5647b77a8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:18.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.192+0000 7fab09ffb700 0 client.15135 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:12:18.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.192+0000 7fab3be36b80 1 -- 172.21.15.64:0/1830994021 >> 172.21.15.64:0/1830994021 conn(0x5647b770f040 msgr2=0x5647b779dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:18.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.192+0000 7fab3be36b80 1 -- 172.21.15.64:0/1830994021 shutdown_connections 2023-09-13T03:12:18.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.192+0000 7fab3be36b80 1 -- 172.21.15.64:0/1830994021 wait complete. 2023-09-13T03:12:18.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.192+0000 7fab3be36b80 20 client.15135 trim_cache size 0 max 16384 2023-09-13T03:12:18.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.210+0000 7fab3be36b80 1 Processor -- start 2023-09-13T03:12:18.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.210+0000 7fab3be36b80 1 -- start start 2023-09-13T03:12:18.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.211+0000 7fab3be36b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647b779f1e0 0x5647b779ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:18.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.211+0000 7fab3be36b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b779fff0 0x5647b77aff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:18.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.211+0000 7fab3be36b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b77a0950 0x5647b77b3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:18.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.211+0000 7fab3be36b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647b757ade0 con 0x5647b779fff0 2023-09-13T03:12:18.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.211+0000 7fab3be36b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647b76b5f70 con 0x5647b77a0950 2023-09-13T03:12:18.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.211+0000 7fab3be36b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647b76a8ed0 con 0x5647b779f1e0 2023-09-13T03:12:18.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.211+0000 7fab25b9a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b779fff0 0x5647b77aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:18.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.211+0000 7fab25b9a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b779fff0 0x5647b77aff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58034/0 (socket says 172.21.15.64:58034) 2023-09-13T03:12:18.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.211+0000 7fab25b9a700 1 -- 172.21.15.64:0/3524562920 learned_addr learned my addr 172.21.15.64:0/3524562920 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:18.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.211+0000 7fab2639b700 1 --2- 172.21.15.64:0/3524562920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647b779f1e0 0x5647b779ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:18.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.211+0000 7fab26b9c700 1 --2- 172.21.15.64:0/3524562920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b77a0950 0x5647b77b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:18.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.211+0000 7fab25b9a700 1 -- 172.21.15.64:0/3524562920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647b779f1e0 msgr2=0x5647b779ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:18.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.211+0000 7fab25b9a700 1 --2- 172.21.15.64:0/3524562920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647b779f1e0 0x5647b779ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:18.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.211+0000 7fab25b9a700 1 -- 172.21.15.64:0/3524562920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b77a0950 msgr2=0x5647b77b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:18.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.211+0000 7fab25b9a700 1 --2- 172.21.15.64:0/3524562920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b77a0950 0x5647b77b3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:18.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.211+0000 7fab25b9a700 1 -- 172.21.15.64:0/3524562920 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647b7501c10 con 0x5647b779fff0 2023-09-13T03:12:18.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.212+0000 7fab25b9a700 1 --2- 172.21.15.64:0/3524562920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b779fff0 0x5647b77aff20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fab100099c0 tx=0x7fab10005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f70784e042ffc5 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:18.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.212+0000 7fab153ff700 1 -- 172.21.15.64:0/3524562920 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1000fa50 con 0x5647b779fff0 2023-09-13T03:12:18.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.212+0000 7fab153ff700 1 -- 172.21.15.64:0/3524562920 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab1001e040 con 0x5647b779fff0 2023-09-13T03:12:18.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.212+0000 7fab153ff700 1 -- 172.21.15.64:0/3524562920 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab100053a0 con 0x5647b779fff0 2023-09-13T03:12:18.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.212+0000 7fab3be36b80 1 -- 172.21.15.64:0/3524562920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b779fff0 msgr2=0x5647b77aff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:18.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.212+0000 7fab3be36b80 1 --2- 172.21.15.64:0/3524562920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b779fff0 0x5647b77aff20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fab100099c0 tx=0x7fab10005fb0 comp rx=0 tx=0).stop 2023-09-13T03:12:18.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.213+0000 7fab3be36b80 1 -- 172.21.15.64:0/3524562920 shutdown_connections 2023-09-13T03:12:18.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.213+0000 7fab3be36b80 1 --2- 172.21.15.64:0/3524562920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b77a0950 0x5647b77b3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:18.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.213+0000 7fab3be36b80 1 --2- 172.21.15.64:0/3524562920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647b779f1e0 0x5647b779ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:18.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.213+0000 7fab3be36b80 1 --2- 172.21.15.64:0/3524562920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b779fff0 0x5647b77aff20 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:18.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.213+0000 7fab3be36b80 1 -- 172.21.15.64:0/3524562920 >> 172.21.15.64:0/3524562920 conn(0x5647b770f040 msgr2=0x5647b779f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:18.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.213+0000 7fab3be36b80 1 -- 172.21.15.64:0/3524562920 shutdown_connections 2023-09-13T03:12:18.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.213+0000 7fab3be36b80 1 -- 172.21.15.64:0/3524562920 wait complete. 2023-09-13T03:12:18.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.213+0000 7fab3be36b80 1 -- start start 2023-09-13T03:12:18.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.213+0000 7fab3be36b80 1 Processor -- start 2023-09-13T03:12:18.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.214+0000 7fab3be36b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b779f1e0 0x5647b77af970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:18.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.214+0000 7fab3be36b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b779fff0 0x5647b77afeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:18.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.214+0000 7fab3be36b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647b77a0950 0x5647b77ac500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:18.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.214+0000 7fab3be36b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647b77acba0 con 0x5647b779f1e0 2023-09-13T03:12:18.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.214+0000 7fab3be36b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647b77acd10 con 0x5647b779fff0 2023-09-13T03:12:18.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.214+0000 7fab3be36b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647b77aced0 con 0x5647b77a0950 2023-09-13T03:12:18.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.214+0000 7fab3be36b80 10 client.0 _ll_register_callbacks cb 0x7fab18008d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:12:18.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.214+0000 7fab2639b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b779f1e0 0x5647b77af970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:18.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.214+0000 7fab25b9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b779fff0 0x5647b77afeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:18.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.214+0000 7fab25b9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b779fff0 0x5647b77afeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:46102/0 (socket says 172.21.15.64:46102) 2023-09-13T03:12:18.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.214+0000 7fab26b9c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647b77a0950 0x5647b77ac500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:18.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.214+0000 7fab25b9a700 1 -- 172.21.15.64:0/3642139264 learned_addr learned my addr 172.21.15.64:0/3642139264 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:18.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.214+0000 7fab2639b700 1 -- 172.21.15.64:0/3642139264 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647b77a0950 msgr2=0x5647b77ac500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:18.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.214+0000 7fab2639b700 1 --2- 172.21.15.64:0/3642139264 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647b77a0950 0x5647b77ac500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:18.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.214+0000 7fab2639b700 1 -- 172.21.15.64:0/3642139264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b779fff0 msgr2=0x5647b77afeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:18.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.214+0000 7fab2639b700 1 --2- 172.21.15.64:0/3642139264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b779fff0 0x5647b77afeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:18.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.214+0000 7fab2639b700 1 -- 172.21.15.64:0/3642139264 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab1000f460 con 0x5647b779f1e0 2023-09-13T03:12:18.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.215+0000 7fab2639b700 1 --2- 172.21.15.64:0/3642139264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b779f1e0 0x5647b77af970 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fab1c096310 tx=0x7fab1c09a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcfb4a88f7cb19c3 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:18.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.215+0000 7fab09ffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:12:18.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.215+0000 7fab09ffb700 1 -- 172.21.15.64:0/3642139264 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c0b5070 con 0x5647b779f1e0 2023-09-13T03:12:18.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.215+0000 7fab09ffb700 1 -- 172.21.15.64:0/3642139264 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fab1c0998a0 con 0x5647b779f1e0 2023-09-13T03:12:18.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.215+0000 7fab3be36b80 1 -- 172.21.15.64:0/3642139264 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab1c099fa0 con 0x5647b779f1e0 2023-09-13T03:12:18.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.215+0000 7fab09ffb700 1 -- 172.21.15.64:0/3642139264 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab1c0b1ea0 con 0x5647b779f1e0 2023-09-13T03:12:18.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.215+0000 7fab3be36b80 10 client.15141 fetch_fsmap learned FSMap version 26 2023-09-13T03:12:18.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.215+0000 7fab3be36b80 1 -- 172.21.15.64:0/3642139264 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5647b753f200 con 0x5647b779f1e0 2023-09-13T03:12:18.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.215+0000 7fab09ffb700 1 -- 172.21.15.64:0/3642139264 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab1c099fa0 con 0x5647b779f1e0 2023-09-13T03:12:18.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.215+0000 7fab09ffb700 1 -- 172.21.15.64:0/3642139264 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fab1c0a1480 con 0x5647b779f1e0 2023-09-13T03:12:18.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.215+0000 7fab3be36b80 10 client.15141 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:12:18.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.215+0000 7fab3be36b80 10 client.15141 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-13T03:12:18.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.215+0000 7fab3be36b80 20 client.15141 populate_metadata read hostname 'smithi064' 2023-09-13T03:12:18.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.215+0000 7fab3be36b80 1 --2- 172.21.15.64:0/3642139264 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5647b780ce30 0x5647b78092b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:18.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.215+0000 7fab3be36b80 4 client.15141 mds_command: new command op to 24686 tid=0[{"prefix": "status"}] 2023-09-13T03:12:18.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.215+0000 7fab3be36b80 1 -- 172.21.15.64:0/3642139264 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5647b76e58d0 con 0x5647b780ce30 2023-09-13T03:12:18.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.216+0000 7fab25b9a700 1 --2- 172.21.15.64:0/3642139264 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5647b780ce30 0x5647b78092b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:18.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.216+0000 7fab25b9a700 1 --2- 172.21.15.64:0/3642139264 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5647b780ce30 0x5647b78092b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7b1285f60e1a5c8 server_cookie=82749637de9423e6 in_seq=0 out_seq=0 2023-09-13T03:12:18.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.216+0000 7fab09ffb700 10 client.15141 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:12:18.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.217+0000 7fab09ffb700 1 -- 172.21.15.64:0/3642139264 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5647b76e58d0 con 0x5647b780ce30 2023-09-13T03:12:18.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.217+0000 7fab09ffb700 10 client.15141 handle_command_reply: tid=0 2023-09-13T03:12:18.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.217+0000 7fab3be36b80 1 client.15141 shutdown 2023-09-13T03:12:18.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.217+0000 7fab3be36b80 1 -- 172.21.15.64:0/3642139264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b779f1e0 msgr2=0x5647b77af970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:18.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.217+0000 7fab3be36b80 1 --2- 172.21.15.64:0/3642139264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b779f1e0 0x5647b77af970 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fab1c096310 tx=0x7fab1c09a110 comp rx=0 tx=0).stop 2023-09-13T03:12:18.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.217+0000 7fab3be36b80 1 -- 172.21.15.64:0/3642139264 shutdown_connections 2023-09-13T03:12:18.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.217+0000 7fab3be36b80 1 --2- 172.21.15.64:0/3642139264 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5647b780ce30 0x5647b78092b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:18.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.217+0000 7fab3be36b80 1 --2- 172.21.15.64:0/3642139264 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647b77a0950 0x5647b77ac500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:18.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.217+0000 7fab3be36b80 1 --2- 172.21.15.64:0/3642139264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647b779f1e0 0x5647b77af970 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:18.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.217+0000 7fab09ffb700 0 client.15141 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:12:18.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.217+0000 7fab3be36b80 1 --2- 172.21.15.64:0/3642139264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647b779fff0 0x5647b77afeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:18.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.217+0000 7fab3be36b80 1 -- 172.21.15.64:0/3642139264 >> 172.21.15.64:0/3642139264 conn(0x5647b770f040 msgr2=0x5647b77b3100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:18.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.217+0000 7fab3be36b80 1 -- 172.21.15.64:0/3642139264 shutdown_connections 2023-09-13T03:12:18.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.217+0000 7fab3be36b80 1 -- 172.21.15.64:0/3642139264 wait complete. 2023-09-13T03:12:18.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.217+0000 7fab3be36b80 20 client.15141 trim_cache size 0 max 16384 2023-09-13T03:12:18.235 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-13T03:12:18.236 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "f8dbced4-51e1-11ee-9ab7-7b867c8bd7da", 2023-09-13T03:12:18.236 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 0, 2023-09-13T03:12:18.236 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24686, 2023-09-13T03:12:18.236 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2023-09-13T03:12:18.236 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2023-09-13T03:12:18.237 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2023-09-13T03:12:18.237 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 67.255489518999994, 2023-09-13T03:12:18.237 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 26, 2023-09-13T03:12:18.237 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 85, 2023-09-13T03:12:18.237 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 83, 2023-09-13T03:12:18.238 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 74.201058348000004 2023-09-13T03:12:18.238 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-13T03:12:18.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.218+0000 7fab285ff700 1 -- 172.21.15.64:0/1681791120 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fab0c0a06d0 msgr2=0x7fab0c0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:18.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.218+0000 7fab285ff700 1 --2- 172.21.15.64:0/1681791120 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fab0c0a06d0 0x7fab0c0a2b90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fab18009ec0 tx=0x7fab1800b040 comp rx=0 tx=0).stop 2023-09-13T03:12:18.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.218+0000 7fab285ff700 1 -- 172.21.15.64:0/1681791120 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab201005f0 msgr2=0x7fab20196310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:18.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.218+0000 7fab285ff700 1 --2- 172.21.15.64:0/1681791120 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab201005f0 0x7fab20196310 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00ea50 tx=0x7fab1c00ed60 comp rx=0 tx=0).stop 2023-09-13T03:12:18.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.218+0000 7fab285ff700 1 -- 172.21.15.64:0/1681791120 shutdown_connections 2023-09-13T03:12:18.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.218+0000 7fab285ff700 1 --2- 172.21.15.64:0/1681791120 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fab0c0a06d0 0x7fab0c0a2b90 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:18.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.218+0000 7fab285ff700 1 --2- 172.21.15.64:0/1681791120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab20102180 0x7fab2019d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:18.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.218+0000 7fab285ff700 1 --2- 172.21.15.64:0/1681791120 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab201005f0 0x7fab20196310 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:18.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.218+0000 7fab285ff700 1 --2- 172.21.15.64:0/1681791120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab201017e0 0x7fab20196850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:18.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.218+0000 7fab285ff700 1 -- 172.21.15.64:0/1681791120 >> 172.21.15.64:0/1681791120 conn(0x7fab200fbb40 msgr2=0x7fab200fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:18.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.218+0000 7fab285ff700 1 -- 172.21.15.64:0/1681791120 shutdown_connections 2023-09-13T03:12:18.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:18.218+0000 7fab285ff700 1 -- 172.21.15.64:0/1681791120 wait complete. 2023-09-13T03:12:18.242 INFO:tasks.cephadm:Setup complete, yielding 2023-09-13T03:12:18.243 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-09-13T03:12:18.252 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi064.front.sepia.ceph.com 2023-09-13T03:12:18.253 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- bash -c 'ceph orch status' 2023-09-13T03:12:18.981 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:12:19.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:19 smithi170 ceph-mon[134328]: pgmap v313: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-13T03:12:19.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:19 smithi064 ceph-mon[97675]: pgmap v313: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-13T03:12:19.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:19 smithi142 ceph-mon[134082]: pgmap v313: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-13T03:12:20.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.360+0000 7fe78a539700 1 -- 172.21.15.64:0/4114952860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe784100690 msgr2=0x7fe784100ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:20.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.360+0000 7fe78a539700 1 --2- 172.21.15.64:0/4114952860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe784100690 0x7fe784100ab0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe7740099c0 tx=0x7fe774009cd0 comp rx=0 tx=0).stop 2023-09-13T03:12:20.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.360+0000 7fe78a539700 1 -- 172.21.15.64:0/4114952860 shutdown_connections 2023-09-13T03:12:20.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.360+0000 7fe78a539700 1 --2- 172.21.15.64:0/4114952860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe784102230 0x7fe784108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:20.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.360+0000 7fe78a539700 1 --2- 172.21.15.64:0/4114952860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe784100690 0x7fe784100ab0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:20.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.360+0000 7fe78a539700 1 --2- 172.21.15.64:0/4114952860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe784101890 0x7fe784101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:20.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.360+0000 7fe78a539700 1 -- 172.21.15.64:0/4114952860 >> 172.21.15.64:0/4114952860 conn(0x7fe7840fbbf0 msgr2=0x7fe7840fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:20.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.360+0000 7fe78a539700 1 -- 172.21.15.64:0/4114952860 shutdown_connections 2023-09-13T03:12:20.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.361+0000 7fe78a539700 1 -- 172.21.15.64:0/4114952860 wait complete. 2023-09-13T03:12:20.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.361+0000 7fe78a539700 1 Processor -- start 2023-09-13T03:12:20.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.362+0000 7fe78a539700 1 -- start start 2023-09-13T03:12:20.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.362+0000 7fe78a539700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe784100690 0x7fe784198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:20.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.362+0000 7fe78a539700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe784101890 0x7fe784198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:20.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.362+0000 7fe78a539700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe784102230 0x7fe78419faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:20.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.362+0000 7fe78a539700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7841a00c0 con 0x7fe784100690 2023-09-13T03:12:20.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.362+0000 7fe78a539700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7841a0200 con 0x7fe784101890 2023-09-13T03:12:20.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.362+0000 7fe78a539700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7841a0390 con 0x7fe784102230 2023-09-13T03:12:20.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.362+0000 7fe783fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe784100690 0x7fe784198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:20.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.362+0000 7fe783fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe784100690 0x7fe784198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58072/0 (socket says 172.21.15.64:58072) 2023-09-13T03:12:20.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.362+0000 7fe783fff700 1 -- 172.21.15.64:0/1187195273 learned_addr learned my addr 172.21.15.64:0/1187195273 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:20.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.362+0000 7fe7837fe700 1 --2- 172.21.15.64:0/1187195273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe784101890 0x7fe784198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:20.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.363+0000 7fe788ad6700 1 --2- 172.21.15.64:0/1187195273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe784102230 0x7fe78419faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:20.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.363+0000 7fe783fff700 1 -- 172.21.15.64:0/1187195273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe784102230 msgr2=0x7fe78419faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:20.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.363+0000 7fe783fff700 1 --2- 172.21.15.64:0/1187195273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe784102230 0x7fe78419faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:20.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.363+0000 7fe783fff700 1 -- 172.21.15.64:0/1187195273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe784101890 msgr2=0x7fe784198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:20.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.363+0000 7fe783fff700 1 --2- 172.21.15.64:0/1187195273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe784101890 0x7fe784198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:20.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.363+0000 7fe783fff700 1 -- 172.21.15.64:0/1187195273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7740095f0 con 0x7fe784100690 2023-09-13T03:12:20.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.363+0000 7fe7837fe700 1 --2- 172.21.15.64:0/1187195273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe784101890 0x7fe784198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:20.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.363+0000 7fe783fff700 1 --2- 172.21.15.64:0/1187195273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe784100690 0x7fe784198530 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe77400f690 tx=0x7fe77400f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:20.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.363+0000 7fe7817fa700 1 -- 172.21.15.64:0/1187195273 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77401d070 con 0x7fe784100690 2023-09-13T03:12:20.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.363+0000 7fe7817fa700 1 -- 172.21.15.64:0/1187195273 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe774005050 con 0x7fe784100690 2023-09-13T03:12:20.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.363+0000 7fe78a539700 1 -- 172.21.15.64:0/1187195273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7841a0610 con 0x7fe784100690 2023-09-13T03:12:20.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.364+0000 7fe7817fa700 1 -- 172.21.15.64:0/1187195273 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe774004b10 con 0x7fe784100690 2023-09-13T03:12:20.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.364+0000 7fe78a539700 1 -- 172.21.15.64:0/1187195273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7841a0a20 con 0x7fe784100690 2023-09-13T03:12:20.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.365+0000 7fe78a539700 1 -- 172.21.15.64:0/1187195273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe784192c90 con 0x7fe784100690 2023-09-13T03:12:20.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.367+0000 7fe7817fa700 1 -- 172.21.15.64:0/1187195273 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fe7740051c0 con 0x7fe784100690 2023-09-13T03:12:20.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.368+0000 7fe7817fa700 1 --2- 172.21.15.64:0/1187195273 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe76c0a06d0 0x7fe76c0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:20.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.368+0000 7fe7817fa700 1 -- 172.21.15.64:0/1187195273 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fe7740ccc10 con 0x7fe784100690 2023-09-13T03:12:20.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.368+0000 7fe7837fe700 1 --2- 172.21.15.64:0/1187195273 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe76c0a06d0 0x7fe76c0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:20.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.369+0000 7fe7837fe700 1 --2- 172.21.15.64:0/1187195273 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe76c0a06d0 0x7fe76c0a2b90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe778010420 tx=0x7fe7780073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:20.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.372+0000 7fe7817fa700 1 -- 172.21.15.64:0/1187195273 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe774096760 con 0x7fe784100690 2023-09-13T03:12:20.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.599+0000 7fe78a539700 1 -- 172.21.15.64:0/1187195273 --> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe78402ced0 con 0x7fe76c0a06d0 2023-09-13T03:12:20.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.601+0000 7fe7817fa700 1 -- 172.21.15.64:0/1187195273 <== mgr.14150 v2:172.21.15.64:6800/873679308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fe78402ced0 con 0x7fe76c0a06d0 2023-09-13T03:12:20.601 INFO:teuthology.orchestra.run.smithi064.stdout:Backend: cephadm 2023-09-13T03:12:20.601 INFO:teuthology.orchestra.run.smithi064.stdout:Available: Yes 2023-09-13T03:12:20.602 INFO:teuthology.orchestra.run.smithi064.stdout:Paused: No 2023-09-13T03:12:20.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.603+0000 7fe78a539700 1 -- 172.21.15.64:0/1187195273 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe76c0a06d0 msgr2=0x7fe76c0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:20.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.603+0000 7fe78a539700 1 --2- 172.21.15.64:0/1187195273 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe76c0a06d0 0x7fe76c0a2b90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe778010420 tx=0x7fe7780073d0 comp rx=0 tx=0).stop 2023-09-13T03:12:20.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.603+0000 7fe78a539700 1 -- 172.21.15.64:0/1187195273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe784100690 msgr2=0x7fe784198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:20.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.603+0000 7fe78a539700 1 --2- 172.21.15.64:0/1187195273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe784100690 0x7fe784198530 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe77400f690 tx=0x7fe77400f6c0 comp rx=0 tx=0).stop 2023-09-13T03:12:20.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.603+0000 7fe78a539700 1 -- 172.21.15.64:0/1187195273 shutdown_connections 2023-09-13T03:12:20.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.603+0000 7fe78a539700 1 --2- 172.21.15.64:0/1187195273 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe76c0a06d0 0x7fe76c0a2b90 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:20.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.603+0000 7fe78a539700 1 --2- 172.21.15.64:0/1187195273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe784102230 0x7fe78419faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:20.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.603+0000 7fe78a539700 1 --2- 172.21.15.64:0/1187195273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe784100690 0x7fe784198530 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:20.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.603+0000 7fe78a539700 1 --2- 172.21.15.64:0/1187195273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe784101890 0x7fe784198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:20.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.603+0000 7fe78a539700 1 -- 172.21.15.64:0/1187195273 >> 172.21.15.64:0/1187195273 conn(0x7fe7840fbbf0 msgr2=0x7fe784107210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:20.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.603+0000 7fe78a539700 1 -- 172.21.15.64:0/1187195273 shutdown_connections 2023-09-13T03:12:20.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:20.603+0000 7fe78a539700 1 -- 172.21.15.64:0/1187195273 wait complete. 2023-09-13T03:12:21.061 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- bash -c 'ceph orch ps' 2023-09-13T03:12:21.793 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:12:21.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:21 smithi170 ceph-mon[134328]: pgmap v314: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-13T03:12:21.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:21 smithi170 ceph-mon[134328]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:12:21.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:21 smithi064 ceph-mon[97675]: pgmap v314: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-13T03:12:21.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:21 smithi064 ceph-mon[97675]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:12:21.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:21 smithi142 ceph-mon[134082]: pgmap v314: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-13T03:12:21.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:21 smithi142 ceph-mon[134082]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:12:23.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.292+0000 7fc89d0fe700 1 -- 172.21.15.64:0/1522683213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc898101890 msgr2=0x7fc898101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:23.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.292+0000 7fc89d0fe700 1 --2- 172.21.15.64:0/1522683213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc898101890 0x7fc898101cf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc88c0099c0 tx=0x7fc88c009cd0 comp rx=0 tx=0).stop 2023-09-13T03:12:23.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.292+0000 7fc89d0fe700 1 -- 172.21.15.64:0/1522683213 shutdown_connections 2023-09-13T03:12:23.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.292+0000 7fc89d0fe700 1 --2- 172.21.15.64:0/1522683213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc898100690 0x7fc898100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:23.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.292+0000 7fc89d0fe700 1 --2- 172.21.15.64:0/1522683213 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc898102230 0x7fc898108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:23.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.292+0000 7fc89d0fe700 1 --2- 172.21.15.64:0/1522683213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc898101890 0x7fc898101cf0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:23.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.292+0000 7fc89d0fe700 1 -- 172.21.15.64:0/1522683213 >> 172.21.15.64:0/1522683213 conn(0x7fc8980fbbf0 msgr2=0x7fc8980fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:23.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.292+0000 7fc89d0fe700 1 -- 172.21.15.64:0/1522683213 shutdown_connections 2023-09-13T03:12:23.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.292+0000 7fc89d0fe700 1 -- 172.21.15.64:0/1522683213 wait complete. 2023-09-13T03:12:23.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.293+0000 7fc89d0fe700 1 Processor -- start 2023-09-13T03:12:23.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.293+0000 7fc89d0fe700 1 -- start start 2023-09-13T03:12:23.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.293+0000 7fc89d0fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc898100690 0x7fc8981963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:23.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.293+0000 7fc89d0fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc898101890 0x7fc898196930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:23.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.294+0000 7fc89d0fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc898102230 0x7fc89819d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:23.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.294+0000 7fc89d0fe700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89819df80 con 0x7fc898101890 2023-09-13T03:12:23.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.294+0000 7fc89d0fe700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89819e0c0 con 0x7fc898102230 2023-09-13T03:12:23.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.294+0000 7fc89d0fe700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89819e250 con 0x7fc898100690 2023-09-13T03:12:23.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.294+0000 7fc89659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc898101890 0x7fc898196930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:23.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.294+0000 7fc89659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc898101890 0x7fc898196930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58082/0 (socket says 172.21.15.64:58082) 2023-09-13T03:12:23.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.294+0000 7fc89659c700 1 -- 172.21.15.64:0/962448191 learned_addr learned my addr 172.21.15.64:0/962448191 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:23.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.294+0000 7fc896d9d700 1 --2- 172.21.15.64:0/962448191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc898100690 0x7fc8981963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:23.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.294+0000 7fc89759e700 1 --2- 172.21.15.64:0/962448191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc898102230 0x7fc89819d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:23.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.294+0000 7fc896d9d700 1 -- 172.21.15.64:0/962448191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc898102230 msgr2=0x7fc89819d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:23.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.294+0000 7fc896d9d700 1 --2- 172.21.15.64:0/962448191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc898102230 0x7fc89819d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:23.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.294+0000 7fc896d9d700 1 -- 172.21.15.64:0/962448191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc898101890 msgr2=0x7fc898196930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:23.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.294+0000 7fc896d9d700 1 --2- 172.21.15.64:0/962448191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc898101890 0x7fc898196930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:23.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.294+0000 7fc896d9d700 1 -- 172.21.15.64:0/962448191 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc88c0095f0 con 0x7fc898100690 2023-09-13T03:12:23.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.295+0000 7fc896d9d700 1 --2- 172.21.15.64:0/962448191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc898100690 0x7fc8981963f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc88800e9f0 tx=0x7fc88800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:23.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.295+0000 7fc87ffff700 1 -- 172.21.15.64:0/962448191 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc888004220 con 0x7fc898100690 2023-09-13T03:12:23.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.295+0000 7fc87ffff700 1 -- 172.21.15.64:0/962448191 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc888004380 con 0x7fc898100690 2023-09-13T03:12:23.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.295+0000 7fc89d0fe700 1 -- 172.21.15.64:0/962448191 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc89819e530 con 0x7fc898100690 2023-09-13T03:12:23.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.295+0000 7fc87ffff700 1 -- 172.21.15.64:0/962448191 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc888003bb0 con 0x7fc898100690 2023-09-13T03:12:23.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.295+0000 7fc89d0fe700 1 -- 172.21.15.64:0/962448191 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc89819ea30 con 0x7fc898100690 2023-09-13T03:12:23.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.296+0000 7fc89d0fe700 1 -- 172.21.15.64:0/962448191 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc898107d30 con 0x7fc898100690 2023-09-13T03:12:23.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.298+0000 7fc87ffff700 1 -- 172.21.15.64:0/962448191 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fc888003d10 con 0x7fc898100690 2023-09-13T03:12:23.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.300+0000 7fc87ffff700 1 --2- 172.21.15.64:0/962448191 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc8800a0640 0x7fc8800a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:23.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.300+0000 7fc87ffff700 1 -- 172.21.15.64:0/962448191 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fc888014070 con 0x7fc898100690 2023-09-13T03:12:23.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.300+0000 7fc89659c700 1 --2- 172.21.15.64:0/962448191 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc8800a0640 0x7fc8800a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:23.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.300+0000 7fc89659c700 1 --2- 172.21.15.64:0/962448191 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc8800a0640 0x7fc8800a2b00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc88c000c00 tx=0x7fc88c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:23.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.302+0000 7fc87ffff700 1 -- 172.21.15.64:0/962448191 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fc888095a50 con 0x7fc898100690 2023-09-13T03:12:23.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.522+0000 7fc89d0fe700 1 -- 172.21.15.64:0/962448191 --> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc89802d010 con 0x7fc8800a0640 2023-09-13T03:12:23.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.535+0000 7fc87ffff700 1 -- 172.21.15.64:0/962448191 <== mgr.14150 v2:172.21.15.64:6800/873679308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7fc89802d010 con 0x7fc8800a0640 2023-09-13T03:12:23.536 INFO:teuthology.orchestra.run.smithi064.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-09-13T03:12:23.536 INFO:teuthology.orchestra.run.smithi064.stdout:mds.a smithi064 running (82s) - 81s 16.8M - 18.0.0-6088-g2110e007 bf15e540d611 210febf3b329 2023-09-13T03:12:23.536 INFO:teuthology.orchestra.run.smithi064.stdout:mds.b smithi142 running (79s) - 79s 30.2M - 18.0.0-6088-g2110e007 bf15e540d611 79a5d73f522c 2023-09-13T03:12:23.537 INFO:teuthology.orchestra.run.smithi064.stdout:mds.c smithi170 running (93s) - 92s 21.2M - 18.0.0-6088-g2110e007 bf15e540d611 ca0afca0c00b 2023-09-13T03:12:23.537 INFO:teuthology.orchestra.run.smithi064.stdout:mds.d smithi064 running (99s) - 99s 15.8M - 18.0.0-6088-g2110e007 bf15e540d611 6490b80c61b3 2023-09-13T03:12:23.537 INFO:teuthology.orchestra.run.smithi064.stdout:mds.e smithi142 running (97s) - 97s 18.5M - 18.0.0-6088-g2110e007 bf15e540d611 cae6724bca99 2023-09-13T03:12:23.537 INFO:teuthology.orchestra.run.smithi064.stdout:mds.f smithi170 running (95s) - 94s 20.5M - 18.0.0-6088-g2110e007 bf15e540d611 3f2e6a45d76a 2023-09-13T03:12:23.537 INFO:teuthology.orchestra.run.smithi064.stdout:mds.g smithi064 running (85s) - 85s 16.8M - 18.0.0-6088-g2110e007 bf15e540d611 6fdaab75e036 2023-09-13T03:12:23.538 INFO:teuthology.orchestra.run.smithi064.stdout:mds.h smithi142 running (90s) - 90s 18.3M - 18.0.0-6088-g2110e007 bf15e540d611 fc1fa908a50f 2023-09-13T03:12:23.538 INFO:teuthology.orchestra.run.smithi064.stdout:mds.i smithi170 running (88s) - 88s 18.6M - 18.0.0-6088-g2110e007 bf15e540d611 94b86140e34b 2023-09-13T03:12:23.538 INFO:teuthology.orchestra.run.smithi064.stdout:mds.j smithi064 running (102s) - 102s 13.8M - 18.0.0-6088-g2110e007 bf15e540d611 619defdce61d 2023-09-13T03:12:23.538 INFO:teuthology.orchestra.run.smithi064.stdout:mds.k smithi142 running (105s) - 105s 17.3M - 18.0.0-6088-g2110e007 bf15e540d611 b9ea33840375 2023-09-13T03:12:23.538 INFO:teuthology.orchestra.run.smithi064.stdout:mds.l smithi170 running (107s) - 107s 17.2M - 18.0.0-6088-g2110e007 bf15e540d611 be07ceacb357 2023-09-13T03:12:23.539 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.x smithi064 *:9283,8765 running (8m) - 8m 526M - 18.0.0-6088-g2110e007 bf15e540d611 b19027cebd1d 2023-09-13T03:12:23.539 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.y smithi142 *:8443,8765 running (6m) - 6m 452M - 18.0.0-6088-g2110e007 bf15e540d611 255a7d29b941 2023-09-13T03:12:23.539 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.z smithi170 *:8443,8765 running (6m) - 6m 452M - 18.0.0-6088-g2110e007 bf15e540d611 a56ff4b27965 2023-09-13T03:12:23.539 INFO:teuthology.orchestra.run.smithi064.stdout:mon.a smithi064 running (8m) - 8m 95.5M 2048M 18.0.0-6088-g2110e007 bf15e540d611 27338a39719e 2023-09-13T03:12:23.539 INFO:teuthology.orchestra.run.smithi064.stdout:mon.b smithi142 running (7m) - 7m 65.0M 2048M 18.0.0-6088-g2110e007 bf15e540d611 920fce172171 2023-09-13T03:12:23.540 INFO:teuthology.orchestra.run.smithi064.stdout:mon.c smithi170 running (7m) - 7m 66.4M 2048M 18.0.0-6088-g2110e007 bf15e540d611 ab82ebb517af 2023-09-13T03:12:23.540 INFO:teuthology.orchestra.run.smithi064.stdout:osd.0 smithi064 running (6m) - 6m 62.9M 4316M 18.0.0-6088-g2110e007 bf15e540d611 7d73a59811c2 2023-09-13T03:12:23.540 INFO:teuthology.orchestra.run.smithi064.stdout:osd.1 smithi142 running (5m) - 5m 67.5M 4292M 18.0.0-6088-g2110e007 bf15e540d611 8672482510c3 2023-09-13T03:12:23.540 INFO:teuthology.orchestra.run.smithi064.stdout:osd.2 smithi170 running (5m) - 5m 70.2M 4292M 18.0.0-6088-g2110e007 bf15e540d611 8684a022c07f 2023-09-13T03:12:23.540 INFO:teuthology.orchestra.run.smithi064.stdout:osd.3 smithi064 running (5m) - 5m 60.4M 4316M 18.0.0-6088-g2110e007 bf15e540d611 4e8b1895eabd 2023-09-13T03:12:23.541 INFO:teuthology.orchestra.run.smithi064.stdout:osd.4 smithi142 running (4m) - 4m 60.5M 4292M 18.0.0-6088-g2110e007 bf15e540d611 01c5cbdf139e 2023-09-13T03:12:23.541 INFO:teuthology.orchestra.run.smithi064.stdout:osd.5 smithi170 running (4m) - 4m 65.5M 4292M 18.0.0-6088-g2110e007 bf15e540d611 0bf8078ef764 2023-09-13T03:12:23.541 INFO:teuthology.orchestra.run.smithi064.stdout:osd.6 smithi064 running (3m) - 3m 57.3M 4316M 18.0.0-6088-g2110e007 bf15e540d611 16bace500e84 2023-09-13T03:12:23.541 INFO:teuthology.orchestra.run.smithi064.stdout:osd.7 smithi142 running (3m) - 3m 61.2M 4292M 18.0.0-6088-g2110e007 bf15e540d611 f0770015ffb3 2023-09-13T03:12:23.541 INFO:teuthology.orchestra.run.smithi064.stdout:osd.8 smithi170 running (3m) - 3m 58.9M 4292M 18.0.0-6088-g2110e007 bf15e540d611 c80824e0a63e 2023-09-13T03:12:23.542 INFO:teuthology.orchestra.run.smithi064.stdout:osd.9 smithi064 running (2m) - 2m 56.1M 4316M 18.0.0-6088-g2110e007 bf15e540d611 47fc58c009b4 2023-09-13T03:12:23.542 INFO:teuthology.orchestra.run.smithi064.stdout:osd.10 smithi142 running (2m) - 2m 60.8M 4292M 18.0.0-6088-g2110e007 bf15e540d611 bb86175afcb7 2023-09-13T03:12:23.542 INFO:teuthology.orchestra.run.smithi064.stdout:osd.11 smithi170 running (2m) - 2m 54.2M 4292M 18.0.0-6088-g2110e007 bf15e540d611 a2985ff6a0e6 2023-09-13T03:12:23.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.537+0000 7fc89d0fe700 1 -- 172.21.15.64:0/962448191 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc8800a0640 msgr2=0x7fc8800a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:23.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.537+0000 7fc89d0fe700 1 --2- 172.21.15.64:0/962448191 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc8800a0640 0x7fc8800a2b00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc88c000c00 tx=0x7fc88c005fb0 comp rx=0 tx=0).stop 2023-09-13T03:12:23.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.537+0000 7fc89d0fe700 1 -- 172.21.15.64:0/962448191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc898100690 msgr2=0x7fc8981963f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:23.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.537+0000 7fc89d0fe700 1 --2- 172.21.15.64:0/962448191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc898100690 0x7fc8981963f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc88800e9f0 tx=0x7fc88800ed00 comp rx=0 tx=0).stop 2023-09-13T03:12:23.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.537+0000 7fc89d0fe700 1 -- 172.21.15.64:0/962448191 shutdown_connections 2023-09-13T03:12:23.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.537+0000 7fc89d0fe700 1 --2- 172.21.15.64:0/962448191 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc8800a0640 0x7fc8800a2b00 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:23.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.537+0000 7fc89d0fe700 1 --2- 172.21.15.64:0/962448191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc898102230 0x7fc89819d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:23.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.537+0000 7fc89d0fe700 1 --2- 172.21.15.64:0/962448191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc898100690 0x7fc8981963f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:23.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.537+0000 7fc89d0fe700 1 --2- 172.21.15.64:0/962448191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc898101890 0x7fc898196930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:23.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.537+0000 7fc89d0fe700 1 -- 172.21.15.64:0/962448191 >> 172.21.15.64:0/962448191 conn(0x7fc8980fbbf0 msgr2=0x7fc898104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:23.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.537+0000 7fc89d0fe700 1 -- 172.21.15.64:0/962448191 shutdown_connections 2023-09-13T03:12:23.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:23.537+0000 7fc89d0fe700 1 -- 172.21.15.64:0/962448191 wait complete. 2023-09-13T03:12:23.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:23 smithi170 ceph-mon[134328]: pgmap v315: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-13T03:12:23.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:23 smithi170 ceph-mon[134328]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-09-13T03:12:23.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:23 smithi064 ceph-mon[97675]: pgmap v315: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-13T03:12:23.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:23 smithi064 ceph-mon[97675]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-09-13T03:12:23.974 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- bash -c 'ceph orch ls' 2023-09-13T03:12:23.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:23 smithi142 ceph-mon[134082]: pgmap v315: 97 pgs: 97 active+clean; 640 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-13T03:12:23.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:23 smithi142 ceph-mon[134082]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-09-13T03:12:24.708 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:12:24.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:24 smithi170 ceph-mon[134328]: from='client.25180 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:12:24.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:24 smithi064 ceph-mon[97675]: from='client.25180 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:12:24.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:24 smithi142 ceph-mon[134082]: from='client.25180 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:12:25.866 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:25 smithi064 ceph-mon[97675]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 11 op/s 2023-09-13T03:12:25.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:25 smithi170 ceph-mon[134328]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 11 op/s 2023-09-13T03:12:25.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:25 smithi142 ceph-mon[134082]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 11 op/s 2023-09-13T03:12:26.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.108+0000 7f02d1fde700 1 -- 172.21.15.64:0/1409625876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02cc101890 msgr2=0x7f02cc101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:26.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.108+0000 7f02d1fde700 1 --2- 172.21.15.64:0/1409625876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02cc101890 0x7f02cc101cf0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f02c00099c0 tx=0x7f02c0009cd0 comp rx=0 tx=0).stop 2023-09-13T03:12:26.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.108+0000 7f02d1fde700 1 -- 172.21.15.64:0/1409625876 shutdown_connections 2023-09-13T03:12:26.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.108+0000 7f02d1fde700 1 --2- 172.21.15.64:0/1409625876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02cc100690 0x7f02cc100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:26.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.108+0000 7f02d1fde700 1 --2- 172.21.15.64:0/1409625876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02cc102230 0x7f02cc108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:26.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.108+0000 7f02d1fde700 1 --2- 172.21.15.64:0/1409625876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02cc101890 0x7f02cc101cf0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:26.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.108+0000 7f02d1fde700 1 -- 172.21.15.64:0/1409625876 >> 172.21.15.64:0/1409625876 conn(0x7f02cc0fbbf0 msgr2=0x7f02cc0fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:26.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.108+0000 7f02d1fde700 1 -- 172.21.15.64:0/1409625876 shutdown_connections 2023-09-13T03:12:26.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.108+0000 7f02d1fde700 1 -- 172.21.15.64:0/1409625876 wait complete. 2023-09-13T03:12:26.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.109+0000 7f02d1fde700 1 Processor -- start 2023-09-13T03:12:26.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.110+0000 7f02d1fde700 1 -- start start 2023-09-13T03:12:26.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.110+0000 7f02d1fde700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02cc100690 0x7f02cc196330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:26.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.110+0000 7f02d1fde700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02cc101890 0x7f02cc196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:26.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.110+0000 7f02d1fde700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02cc102230 0x7f02cc19d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:26.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.110+0000 7f02d1fde700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc19dec0 con 0x7f02cc101890 2023-09-13T03:12:26.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.110+0000 7f02d1fde700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc19e000 con 0x7f02cc100690 2023-09-13T03:12:26.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.110+0000 7f02d1fde700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc19e190 con 0x7f02cc102230 2023-09-13T03:12:26.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.110+0000 7f02cb7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02cc100690 0x7f02cc196330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:26.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.110+0000 7f02cb7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02cc100690 0x7f02cc196330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:41904/0 (socket says 172.21.15.64:41904) 2023-09-13T03:12:26.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.110+0000 7f02cb7fe700 1 -- 172.21.15.64:0/3502463034 learned_addr learned my addr 172.21.15.64:0/3502463034 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:26.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.110+0000 7f02caffd700 1 --2- 172.21.15.64:0/3502463034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02cc101890 0x7f02cc196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:26.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.110+0000 7f02cbfff700 1 --2- 172.21.15.64:0/3502463034 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02cc102230 0x7f02cc19d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:26.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.111+0000 7f02cb7fe700 1 -- 172.21.15.64:0/3502463034 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02cc102230 msgr2=0x7f02cc19d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:26.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.111+0000 7f02cb7fe700 1 --2- 172.21.15.64:0/3502463034 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02cc102230 0x7f02cc19d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:26.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.111+0000 7f02cb7fe700 1 -- 172.21.15.64:0/3502463034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02cc101890 msgr2=0x7f02cc196870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:26.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.111+0000 7f02cb7fe700 1 --2- 172.21.15.64:0/3502463034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02cc101890 0x7f02cc196870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:26.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.111+0000 7f02cb7fe700 1 -- 172.21.15.64:0/3502463034 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c00095f0 con 0x7f02cc100690 2023-09-13T03:12:26.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.111+0000 7f02cb7fe700 1 --2- 172.21.15.64:0/3502463034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02cc100690 0x7f02cc196330 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f02bc00e970 tx=0x7f02bc00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:26.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.111+0000 7f02c8ff9700 1 -- 172.21.15.64:0/3502463034 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02bc009790 con 0x7f02cc100690 2023-09-13T03:12:26.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.111+0000 7f02c8ff9700 1 -- 172.21.15.64:0/3502463034 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02bc00c7e0 con 0x7f02cc100690 2023-09-13T03:12:26.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.111+0000 7f02d1fde700 1 -- 172.21.15.64:0/3502463034 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02cc19e470 con 0x7f02cc100690 2023-09-13T03:12:26.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.111+0000 7f02c8ff9700 1 -- 172.21.15.64:0/3502463034 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02bc003c70 con 0x7f02cc100690 2023-09-13T03:12:26.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.111+0000 7f02d1fde700 1 -- 172.21.15.64:0/3502463034 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02cc19e970 con 0x7f02cc100690 2023-09-13T03:12:26.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.112+0000 7f02d1fde700 1 -- 172.21.15.64:0/3502463034 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02cc10be90 con 0x7f02cc100690 2023-09-13T03:12:26.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.115+0000 7f02c8ff9700 1 -- 172.21.15.64:0/3502463034 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f02bc00c950 con 0x7f02cc100690 2023-09-13T03:12:26.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.116+0000 7f02c8ff9700 1 --2- 172.21.15.64:0/3502463034 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f02b40a0640 0x7f02b40a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:26.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.116+0000 7f02c8ff9700 1 -- 172.21.15.64:0/3502463034 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f02bc014070 con 0x7f02cc100690 2023-09-13T03:12:26.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.116+0000 7f02caffd700 1 --2- 172.21.15.64:0/3502463034 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f02b40a0640 0x7f02b40a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:26.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.117+0000 7f02caffd700 1 --2- 172.21.15.64:0/3502463034 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f02b40a0640 0x7f02b40a2b00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f02c000b580 tx=0x7f02c0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:26.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.118+0000 7f02c8ff9700 1 -- 172.21.15.64:0/3502463034 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f02bc095b20 con 0x7f02cc100690 2023-09-13T03:12:26.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.336+0000 7f02d1fde700 1 -- 172.21.15.64:0/3502463034 --> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f02cc060a40 con 0x7f02b40a0640 2023-09-13T03:12:26.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.341+0000 7f02c8ff9700 1 -- 172.21.15.64:0/3502463034 <== mgr.14150 v2:172.21.15.64:6800/873679308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+985 (secure 0 0 0) 0x7f02cc060a40 con 0x7f02b40a0640 2023-09-13T03:12:26.341 INFO:teuthology.orchestra.run.smithi064.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-09-13T03:12:26.341 INFO:teuthology.orchestra.run.smithi064.stdout:mds.all 12/12 - 112s smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l;count:12 2023-09-13T03:12:26.341 INFO:teuthology.orchestra.run.smithi064.stdout:mgr 3/3 - 6m smithi064=x;smithi142=y;smithi170=z;count:3 2023-09-13T03:12:26.342 INFO:teuthology.orchestra.run.smithi064.stdout:mon 3/3 - 7m smithi064:172.21.15.64=a;smithi142:172.21.15.142=b;smithi170:172.21.15.170=c;count:3 2023-09-13T03:12:26.342 INFO:teuthology.orchestra.run.smithi064.stdout:osd 12 - - 2023-09-13T03:12:26.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.344+0000 7f02d1fde700 1 -- 172.21.15.64:0/3502463034 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f02b40a0640 msgr2=0x7f02b40a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:26.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.344+0000 7f02d1fde700 1 --2- 172.21.15.64:0/3502463034 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f02b40a0640 0x7f02b40a2b00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f02c000b580 tx=0x7f02c0005fb0 comp rx=0 tx=0).stop 2023-09-13T03:12:26.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.344+0000 7f02d1fde700 1 -- 172.21.15.64:0/3502463034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02cc100690 msgr2=0x7f02cc196330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:26.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.344+0000 7f02d1fde700 1 --2- 172.21.15.64:0/3502463034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02cc100690 0x7f02cc196330 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f02bc00e970 tx=0x7f02bc00ec80 comp rx=0 tx=0).stop 2023-09-13T03:12:26.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.344+0000 7f02d1fde700 1 -- 172.21.15.64:0/3502463034 shutdown_connections 2023-09-13T03:12:26.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.344+0000 7f02d1fde700 1 --2- 172.21.15.64:0/3502463034 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f02b40a0640 0x7f02b40a2b00 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:26.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.344+0000 7f02d1fde700 1 --2- 172.21.15.64:0/3502463034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02cc100690 0x7f02cc196330 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:26.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.344+0000 7f02d1fde700 1 --2- 172.21.15.64:0/3502463034 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02cc102230 0x7f02cc19d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:26.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.344+0000 7f02d1fde700 1 --2- 172.21.15.64:0/3502463034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02cc101890 0x7f02cc196870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:26.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.344+0000 7f02d1fde700 1 -- 172.21.15.64:0/3502463034 >> 172.21.15.64:0/3502463034 conn(0x7f02cc0fbbf0 msgr2=0x7f02cc0fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:26.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.344+0000 7f02d1fde700 1 -- 172.21.15.64:0/3502463034 shutdown_connections 2023-09-13T03:12:26.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:26.344+0000 7f02d1fde700 1 -- 172.21.15.64:0/3502463034 wait complete. 2023-09-13T03:12:26.788 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- bash -c 'ceph orch host ls' 2023-09-13T03:12:27.518 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:12:27.828 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:27 smithi064 ceph-mon[97675]: from='client.25181 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:12:27.829 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:27 smithi064 ceph-mon[97675]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2023-09-13T03:12:27.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:27 smithi170 ceph-mon[134328]: from='client.25181 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:12:27.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:27 smithi170 ceph-mon[134328]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2023-09-13T03:12:27.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:27 smithi142 ceph-mon[134082]: from='client.25181 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:12:27.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:27 smithi142 ceph-mon[134082]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2023-09-13T03:12:28.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:28.809+0000 7f36f41f2700 1 -- 172.21.15.64:0/3706573451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36ec100690 msgr2=0x7f36ec100ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:28.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:28.809+0000 7f36f41f2700 1 --2- 172.21.15.64:0/3706573451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36ec100690 0x7f36ec100ab0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f36e80099c0 tx=0x7f36e8009cd0 comp rx=0 tx=0).stop 2023-09-13T03:12:28.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:28.810+0000 7f36f41f2700 1 -- 172.21.15.64:0/3706573451 shutdown_connections 2023-09-13T03:12:28.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:28.810+0000 7f36f41f2700 1 --2- 172.21.15.64:0/3706573451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36ec102230 0x7f36ec108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:28.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:28.810+0000 7f36f41f2700 1 --2- 172.21.15.64:0/3706573451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36ec100690 0x7f36ec100ab0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:28.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:28.810+0000 7f36f41f2700 1 --2- 172.21.15.64:0/3706573451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36ec101890 0x7f36ec101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:28.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:28.810+0000 7f36f41f2700 1 -- 172.21.15.64:0/3706573451 >> 172.21.15.64:0/3706573451 conn(0x7f36ec0fbbf0 msgr2=0x7f36ec0fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:28.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:28.810+0000 7f36f41f2700 1 -- 172.21.15.64:0/3706573451 shutdown_connections 2023-09-13T03:12:28.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:28.810+0000 7f36f41f2700 1 -- 172.21.15.64:0/3706573451 wait complete. 2023-09-13T03:12:28.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:28.811+0000 7f36f41f2700 1 Processor -- start 2023-09-13T03:12:28.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:28.811+0000 7f36f41f2700 1 -- start start 2023-09-13T03:12:28.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:28.811+0000 7f36f41f2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36ec100690 0x7f36ec1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:28.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:28.811+0000 7f36f41f2700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36ec101890 0x7f36ec198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:28.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:28.811+0000 7f36f41f2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36ec102230 0x7f36ec19fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:28.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:28.811+0000 7f36f41f2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec1a0170 con 0x7f36ec100690 2023-09-13T03:12:28.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:28.811+0000 7f36f41f2700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec1a02b0 con 0x7f36ec101890 2023-09-13T03:12:28.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:28.811+0000 7f36f41f2700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec1a0440 con 0x7f36ec102230 2023-09-13T03:12:28.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:28.812+0000 7f36f1f8e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36ec100690 0x7f36ec1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:28.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:28.812+0000 7f36f1f8e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36ec100690 0x7f36ec1985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56080/0 (socket says 172.21.15.64:56080) 2023-09-13T03:12:28.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:28.812+0000 7f36f1f8e700 1 -- 172.21.15.64:0/1417317556 learned_addr learned my addr 172.21.15.64:0/1417317556 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:28.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:28.812+0000 7f36f178d700 1 --2- 172.21.15.64:0/1417317556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36ec101890 0x7f36ec198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:28.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:28.812+0000 7f36f278f700 1 --2- 172.21.15.64:0/1417317556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36ec102230 0x7f36ec19fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:28.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:28.812+0000 7f36f1f8e700 1 -- 172.21.15.64:0/1417317556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36ec102230 msgr2=0x7f36ec19fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:28.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:28.812+0000 7f36f1f8e700 1 --2- 172.21.15.64:0/1417317556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36ec102230 0x7f36ec19fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:28.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:28.812+0000 7f36f1f8e700 1 -- 172.21.15.64:0/1417317556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36ec101890 msgr2=0x7f36ec198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:28.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:28.812+0000 7f36f1f8e700 1 --2- 172.21.15.64:0/1417317556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36ec101890 0x7f36ec198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:28.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:28.812+0000 7f36f1f8e700 1 -- 172.21.15.64:0/1417317556 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e80095f0 con 0x7f36ec100690 2023-09-13T03:12:28.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:28.812+0000 7f36f1f8e700 1 --2- 172.21.15.64:0/1417317556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36ec100690 0x7f36ec1985e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f36e80098e0 tx=0x7f36e800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:28.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:28.813+0000 7f36e2ffd700 1 -- 172.21.15.64:0/1417317556 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e801d070 con 0x7f36ec100690 2023-09-13T03:12:28.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:28.813+0000 7f36f41f2700 1 -- 172.21.15.64:0/1417317556 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36ec1a06c0 con 0x7f36ec100690 2023-09-13T03:12:28.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:28.813+0000 7f36e2ffd700 1 -- 172.21.15.64:0/1417317556 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36e8005050 con 0x7f36ec100690 2023-09-13T03:12:28.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:28.813+0000 7f36e2ffd700 1 -- 172.21.15.64:0/1417317556 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e8004ae0 con 0x7f36ec100690 2023-09-13T03:12:28.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:28.813+0000 7f36f41f2700 1 -- 172.21.15.64:0/1417317556 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36ec1a0ad0 con 0x7f36ec100690 2023-09-13T03:12:28.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:28.814+0000 7f36f41f2700 1 -- 172.21.15.64:0/1417317556 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36ec10bee0 con 0x7f36ec100690 2023-09-13T03:12:28.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:28.816+0000 7f36e2ffd700 1 -- 172.21.15.64:0/1417317556 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f36e80051c0 con 0x7f36ec100690 2023-09-13T03:12:28.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:28.817+0000 7f36e2ffd700 1 --2- 172.21.15.64:0/1417317556 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f36d80a0640 0x7f36d80a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:28.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:28.817+0000 7f36e2ffd700 1 -- 172.21.15.64:0/1417317556 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f36e80cc9e0 con 0x7f36ec100690 2023-09-13T03:12:28.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:28.817+0000 7f36f178d700 1 --2- 172.21.15.64:0/1417317556 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f36d80a0640 0x7f36d80a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:28.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:28.818+0000 7f36f178d700 1 --2- 172.21.15.64:0/1417317556 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f36d80a0640 0x7f36d80a2b00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f36dc009c60 tx=0x7f36dc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:28.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:28.822+0000 7f36e2ffd700 1 -- 172.21.15.64:0/1417317556 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f36e8096570 con 0x7f36ec100690 2023-09-13T03:12:29.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:29.038+0000 7f36f41f2700 1 -- 172.21.15.64:0/1417317556 --> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f36ec108280 con 0x7f36d80a0640 2023-09-13T03:12:29.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:29.039+0000 7f36e2ffd700 1 -- 172.21.15.64:0/1417317556 <== mgr.14150 v2:172.21.15.64:6800/873679308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f36ec108280 con 0x7f36d80a0640 2023-09-13T03:12:29.040 INFO:teuthology.orchestra.run.smithi064.stdout:HOST ADDR LABELS STATUS 2023-09-13T03:12:29.040 INFO:teuthology.orchestra.run.smithi064.stdout:smithi064 172.21.15.64 2023-09-13T03:12:29.041 INFO:teuthology.orchestra.run.smithi064.stdout:smithi142 172.21.15.142 2023-09-13T03:12:29.041 INFO:teuthology.orchestra.run.smithi064.stdout:smithi170 172.21.15.170 2023-09-13T03:12:29.041 INFO:teuthology.orchestra.run.smithi064.stdout:3 hosts in cluster 2023-09-13T03:12:29.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:29.041+0000 7f36f41f2700 1 -- 172.21.15.64:0/1417317556 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f36d80a0640 msgr2=0x7f36d80a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:29.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:29.041+0000 7f36f41f2700 1 --2- 172.21.15.64:0/1417317556 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f36d80a0640 0x7f36d80a2b00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f36dc009c60 tx=0x7f36dc009260 comp rx=0 tx=0).stop 2023-09-13T03:12:29.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:29.041+0000 7f36f41f2700 1 -- 172.21.15.64:0/1417317556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36ec100690 msgr2=0x7f36ec1985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:29.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:29.041+0000 7f36f41f2700 1 --2- 172.21.15.64:0/1417317556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36ec100690 0x7f36ec1985e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f36e80098e0 tx=0x7f36e800f690 comp rx=0 tx=0).stop 2023-09-13T03:12:29.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:29.042+0000 7f36f41f2700 1 -- 172.21.15.64:0/1417317556 shutdown_connections 2023-09-13T03:12:29.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:29.042+0000 7f36f41f2700 1 --2- 172.21.15.64:0/1417317556 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f36d80a0640 0x7f36d80a2b00 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:29.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:29.042+0000 7f36f41f2700 1 --2- 172.21.15.64:0/1417317556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36ec102230 0x7f36ec19fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:29.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:29.042+0000 7f36f41f2700 1 --2- 172.21.15.64:0/1417317556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36ec100690 0x7f36ec1985e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:29.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:29.042+0000 7f36f41f2700 1 --2- 172.21.15.64:0/1417317556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36ec101890 0x7f36ec198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:29.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:29.042+0000 7f36f41f2700 1 -- 172.21.15.64:0/1417317556 >> 172.21.15.64:0/1417317556 conn(0x7f36ec0fbbf0 msgr2=0x7f36ec0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:29.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:29.042+0000 7f36f41f2700 1 -- 172.21.15.64:0/1417317556 shutdown_connections 2023-09-13T03:12:29.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:29.042+0000 7f36f41f2700 1 -- 172.21.15.64:0/1417317556 wait complete. 2023-09-13T03:12:29.500 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- bash -c 'ceph orch device ls' 2023-09-13T03:12:29.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:29 smithi170 ceph-mon[134328]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2023-09-13T03:12:29.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:29 smithi064 ceph-mon[97675]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2023-09-13T03:12:29.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:29 smithi142 ceph-mon[134082]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2023-09-13T03:12:30.236 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:12:30.755 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:30 smithi064 ceph-mon[97675]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:12:30.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:30 smithi170 ceph-mon[134328]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:12:30.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:30 smithi142 ceph-mon[134082]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:12:31.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.602+0000 7f7589d4e700 1 -- 172.21.15.64:0/2768444151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75841004f0 msgr2=0x7f7584100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:31.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.602+0000 7f7589d4e700 1 --2- 172.21.15.64:0/2768444151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75841004f0 0x7f7584100910 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f75780099c0 tx=0x7f7578009cd0 comp rx=0 tx=0).stop 2023-09-13T03:12:31.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.603+0000 7f7589d4e700 1 -- 172.21.15.64:0/2768444151 shutdown_connections 2023-09-13T03:12:31.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.603+0000 7f7589d4e700 1 --2- 172.21.15.64:0/2768444151 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7584102090 0x7f7584108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:31.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.603+0000 7f7589d4e700 1 --2- 172.21.15.64:0/2768444151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75841004f0 0x7f7584100910 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:31.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.603+0000 7f7589d4e700 1 --2- 172.21.15.64:0/2768444151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75841016f0 0x7f7584101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:31.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.603+0000 7f7589d4e700 1 -- 172.21.15.64:0/2768444151 >> 172.21.15.64:0/2768444151 conn(0x7f75840fbab0 msgr2=0x7f75840fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:31.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.603+0000 7f7589d4e700 1 -- 172.21.15.64:0/2768444151 shutdown_connections 2023-09-13T03:12:31.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.603+0000 7f7589d4e700 1 -- 172.21.15.64:0/2768444151 wait complete. 2023-09-13T03:12:31.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.604+0000 7f7589d4e700 1 Processor -- start 2023-09-13T03:12:31.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.604+0000 7f7589d4e700 1 -- start start 2023-09-13T03:12:31.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.604+0000 7f7589d4e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75841004f0 0x7f7584198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:31.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.604+0000 7f7589d4e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75841016f0 0x7f7584198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:31.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.604+0000 7f7589d4e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7584102090 0x7f758419f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:31.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.604+0000 7f7589d4e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758419ff90 con 0x7f75841004f0 2023-09-13T03:12:31.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.604+0000 7f7589d4e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75841a00d0 con 0x7f75841016f0 2023-09-13T03:12:31.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.604+0000 7f7589d4e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75841a0260 con 0x7f7584102090 2023-09-13T03:12:31.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.604+0000 7f7588d4c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75841004f0 0x7f7584198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:31.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.605+0000 7f7588d4c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75841004f0 0x7f7584198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56100/0 (socket says 172.21.15.64:56100) 2023-09-13T03:12:31.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.605+0000 7f7588d4c700 1 -- 172.21.15.64:0/2542834487 learned_addr learned my addr 172.21.15.64:0/2542834487 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:31.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.605+0000 7f7583fff700 1 --2- 172.21.15.64:0/2542834487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75841016f0 0x7f7584198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:31.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.605+0000 7f758954d700 1 --2- 172.21.15.64:0/2542834487 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7584102090 0x7f758419f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:31.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.605+0000 7f7588d4c700 1 -- 172.21.15.64:0/2542834487 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7584102090 msgr2=0x7f758419f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:31.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.605+0000 7f7588d4c700 1 --2- 172.21.15.64:0/2542834487 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7584102090 0x7f758419f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:31.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.605+0000 7f7588d4c700 1 -- 172.21.15.64:0/2542834487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75841016f0 msgr2=0x7f7584198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:31.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.605+0000 7f7588d4c700 1 --2- 172.21.15.64:0/2542834487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75841016f0 0x7f7584198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:31.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.605+0000 7f7588d4c700 1 -- 172.21.15.64:0/2542834487 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757c009c40 con 0x7f75841004f0 2023-09-13T03:12:31.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.605+0000 7f7588d4c700 1 --2- 172.21.15.64:0/2542834487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75841004f0 0x7f7584198400 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f75780098e0 tx=0x7f757800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:31.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.605+0000 7f7581ffb700 1 -- 172.21.15.64:0/2542834487 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757801d070 con 0x7f75841004f0 2023-09-13T03:12:31.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.606+0000 7f7581ffb700 1 -- 172.21.15.64:0/2542834487 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7578005030 con 0x7f75841004f0 2023-09-13T03:12:31.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.606+0000 7f7589d4e700 1 -- 172.21.15.64:0/2542834487 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75780095f0 con 0x7f75841004f0 2023-09-13T03:12:31.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.606+0000 7f7581ffb700 1 -- 172.21.15.64:0/2542834487 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7578004b10 con 0x7f75841004f0 2023-09-13T03:12:31.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.606+0000 7f7589d4e700 1 -- 172.21.15.64:0/2542834487 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75841a0760 con 0x7f75841004f0 2023-09-13T03:12:31.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.607+0000 7f7589d4e700 1 -- 172.21.15.64:0/2542834487 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f758410bd50 con 0x7f75841004f0 2023-09-13T03:12:31.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.608+0000 7f7581ffb700 1 -- 172.21.15.64:0/2542834487 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f75780051a0 con 0x7f75841004f0 2023-09-13T03:12:31.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.610+0000 7f7581ffb700 1 --2- 172.21.15.64:0/2542834487 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f75740a0640 0x7f75740a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:31.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.610+0000 7f7581ffb700 1 -- 172.21.15.64:0/2542834487 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f75780cc8a0 con 0x7f75841004f0 2023-09-13T03:12:31.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.610+0000 7f7583fff700 1 --2- 172.21.15.64:0/2542834487 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f75740a0640 0x7f75740a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:31.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.611+0000 7f7583fff700 1 --2- 172.21.15.64:0/2542834487 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f75740a0640 0x7f75740a2b00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7570007c60 tx=0x7f75700073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:31.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.614+0000 7f7581ffb700 1 -- 172.21.15.64:0/2542834487 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f757805be10 con 0x7f75841004f0 2023-09-13T03:12:31.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:31 smithi064 ceph-mon[97675]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2023-09-13T03:12:31.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.831+0000 7f7589d4e700 1 -- 172.21.15.64:0/2542834487 --> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f75841080e0 con 0x7f75740a0640 2023-09-13T03:12:31.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.833+0000 7f7581ffb700 1 -- 172.21.15.64:0/2542834487 <== mgr.14150 v2:172.21.15.64:6800/873679308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f75841080e0 con 0x7f75740a0640 2023-09-13T03:12:31.834 INFO:teuthology.orchestra.run.smithi064.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-09-13T03:12:31.834 INFO:teuthology.orchestra.run.smithi064.stdout:smithi064 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT534400C2400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-09-13T03:12:31.834 INFO:teuthology.orchestra.run.smithi064.stdout:smithi142 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6303009Y400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-09-13T03:12:31.834 INFO:teuthology.orchestra.run.smithi064.stdout:smithi170 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7476002R480DGN 447G No 2m ago Has a FileSystem, LVM detected 2023-09-13T03:12:31.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.835+0000 7f7589d4e700 1 -- 172.21.15.64:0/2542834487 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f75740a0640 msgr2=0x7f75740a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:31.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.835+0000 7f7589d4e700 1 --2- 172.21.15.64:0/2542834487 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f75740a0640 0x7f75740a2b00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7570007c60 tx=0x7f75700073d0 comp rx=0 tx=0).stop 2023-09-13T03:12:31.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.835+0000 7f7589d4e700 1 -- 172.21.15.64:0/2542834487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75841004f0 msgr2=0x7f7584198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:31.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.835+0000 7f7589d4e700 1 --2- 172.21.15.64:0/2542834487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75841004f0 0x7f7584198400 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f75780098e0 tx=0x7f757800f690 comp rx=0 tx=0).stop 2023-09-13T03:12:31.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.835+0000 7f7589d4e700 1 -- 172.21.15.64:0/2542834487 shutdown_connections 2023-09-13T03:12:31.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.835+0000 7f7589d4e700 1 --2- 172.21.15.64:0/2542834487 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f75740a0640 0x7f75740a2b00 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:31.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.835+0000 7f7589d4e700 1 --2- 172.21.15.64:0/2542834487 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7584102090 0x7f758419f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:31.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.835+0000 7f7589d4e700 1 --2- 172.21.15.64:0/2542834487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75841004f0 0x7f7584198400 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:31.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.835+0000 7f7589d4e700 1 --2- 172.21.15.64:0/2542834487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75841016f0 0x7f7584198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:31.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.835+0000 7f7589d4e700 1 -- 172.21.15.64:0/2542834487 >> 172.21.15.64:0/2542834487 conn(0x7f75840fbab0 msgr2=0x7f75840fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:31.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.835+0000 7f7589d4e700 1 -- 172.21.15.64:0/2542834487 shutdown_connections 2023-09-13T03:12:31.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:31.835+0000 7f7589d4e700 1 -- 172.21.15.64:0/2542834487 wait complete. 2023-09-13T03:12:31.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:31 smithi170 ceph-mon[134328]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2023-09-13T03:12:31.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:31 smithi142 ceph-mon[134082]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2023-09-13T03:12:32.311 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-09-13T03:12:32.322 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi064.front.sepia.ceph.com 2023-09-13T03:12:32.322 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- bash -c 'ceph fs dump' 2023-09-13T03:12:33.055 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:12:33.872 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:33 smithi064 ceph-mon[97675]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:12:33.873 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:33 smithi064 ceph-mon[97675]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2023-09-13T03:12:33.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:33 smithi170 ceph-mon[134328]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:12:33.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:33 smithi170 ceph-mon[134328]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2023-09-13T03:12:33.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:33 smithi142 ceph-mon[134082]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:12:33.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:33 smithi142 ceph-mon[134082]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2023-09-13T03:12:34.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.433+0000 7f1a6b125700 1 -- 172.21.15.64:0/2707413622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a64101890 msgr2=0x7f1a64101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:34.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.433+0000 7f1a6b125700 1 --2- 172.21.15.64:0/2707413622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a64101890 0x7f1a64101cf0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1a500099c0 tx=0x7f1a50009cd0 comp rx=0 tx=0).stop 2023-09-13T03:12:34.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.433+0000 7f1a6b125700 1 -- 172.21.15.64:0/2707413622 shutdown_connections 2023-09-13T03:12:34.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.433+0000 7f1a6b125700 1 --2- 172.21.15.64:0/2707413622 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a64102230 0x7f1a64108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:34.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.433+0000 7f1a6b125700 1 --2- 172.21.15.64:0/2707413622 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a64100690 0x7f1a64100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:34.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.433+0000 7f1a6b125700 1 --2- 172.21.15.64:0/2707413622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a64101890 0x7f1a64101cf0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:34.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.433+0000 7f1a6b125700 1 -- 172.21.15.64:0/2707413622 >> 172.21.15.64:0/2707413622 conn(0x7f1a640fbbf0 msgr2=0x7f1a640fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:34.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.433+0000 7f1a6b125700 1 -- 172.21.15.64:0/2707413622 shutdown_connections 2023-09-13T03:12:34.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.434+0000 7f1a6b125700 1 -- 172.21.15.64:0/2707413622 wait complete. 2023-09-13T03:12:34.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.434+0000 7f1a6b125700 1 Processor -- start 2023-09-13T03:12:34.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.435+0000 7f1a6b125700 1 -- start start 2023-09-13T03:12:34.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.435+0000 7f1a6b125700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a64100690 0x7f1a641985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:34.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.435+0000 7f1a6b125700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a64101890 0x7f1a64198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:34.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.435+0000 7f1a6b125700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a64102230 0x7f1a6419fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:34.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.435+0000 7f1a6b125700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a641a0140 con 0x7f1a64100690 2023-09-13T03:12:34.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.435+0000 7f1a6b125700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a641a0280 con 0x7f1a64101890 2023-09-13T03:12:34.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.435+0000 7f1a6b125700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a641a0410 con 0x7f1a64102230 2023-09-13T03:12:34.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.435+0000 7f1a68ec1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a64100690 0x7f1a641985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:34.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.435+0000 7f1a68ec1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a64100690 0x7f1a641985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56116/0 (socket says 172.21.15.64:56116) 2023-09-13T03:12:34.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.435+0000 7f1a68ec1700 1 -- 172.21.15.64:0/3935160859 learned_addr learned my addr 172.21.15.64:0/3935160859 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:34.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.435+0000 7f1a696c2700 1 --2- 172.21.15.64:0/3935160859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a64102230 0x7f1a6419fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:34.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.435+0000 7f1a63fff700 1 --2- 172.21.15.64:0/3935160859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a64101890 0x7f1a64198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:34.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.436+0000 7f1a68ec1700 1 -- 172.21.15.64:0/3935160859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a64102230 msgr2=0x7f1a6419fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:34.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.436+0000 7f1a68ec1700 1 --2- 172.21.15.64:0/3935160859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a64102230 0x7f1a6419fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:34.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.436+0000 7f1a68ec1700 1 -- 172.21.15.64:0/3935160859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a64101890 msgr2=0x7f1a64198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:34.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.436+0000 7f1a68ec1700 1 --2- 172.21.15.64:0/3935160859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a64101890 0x7f1a64198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:34.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.436+0000 7f1a68ec1700 1 -- 172.21.15.64:0/3935160859 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a500095f0 con 0x7f1a64100690 2023-09-13T03:12:34.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.436+0000 7f1a68ec1700 1 --2- 172.21.15.64:0/3935160859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a64100690 0x7f1a641985b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800e980 tx=0x7f1a5800ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:34.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.436+0000 7f1a61ffb700 1 -- 172.21.15.64:0/3935160859 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a58009770 con 0x7f1a64100690 2023-09-13T03:12:34.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.436+0000 7f1a6b125700 1 -- 172.21.15.64:0/3935160859 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a641a06f0 con 0x7f1a64100690 2023-09-13T03:12:34.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.436+0000 7f1a61ffb700 1 -- 172.21.15.64:0/3935160859 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a5800c810 con 0x7f1a64100690 2023-09-13T03:12:34.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.436+0000 7f1a61ffb700 1 -- 172.21.15.64:0/3935160859 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a58003c20 con 0x7f1a64100690 2023-09-13T03:12:34.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.436+0000 7f1a6b125700 1 -- 172.21.15.64:0/3935160859 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a641a0ba0 con 0x7f1a64100690 2023-09-13T03:12:34.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.437+0000 7f1a6b125700 1 -- 172.21.15.64:0/3935160859 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a64192cf0 con 0x7f1a64100690 2023-09-13T03:12:34.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.439+0000 7f1a61ffb700 1 -- 172.21.15.64:0/3935160859 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f1a5800c980 con 0x7f1a64100690 2023-09-13T03:12:34.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.441+0000 7f1a61ffb700 1 --2- 172.21.15.64:0/3935160859 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f1a540a0640 0x7f1a540a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:34.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.441+0000 7f1a61ffb700 1 -- 172.21.15.64:0/3935160859 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f1a58014070 con 0x7f1a64100690 2023-09-13T03:12:34.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.441+0000 7f1a63fff700 1 --2- 172.21.15.64:0/3935160859 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f1a540a0640 0x7f1a540a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:34.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.442+0000 7f1a63fff700 1 --2- 172.21.15.64:0/3935160859 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f1a540a0640 0x7f1a540a2b00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000b580 tx=0x7f1a50005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:34.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.445+0000 7f1a61ffb700 1 -- 172.21.15.64:0/3935160859 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1a580958f0 con 0x7f1a64100690 2023-09-13T03:12:34.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.700+0000 7f1a6b125700 1 -- 172.21.15.64:0/3935160859 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1a6410c120 con 0x7f1a64100690 2023-09-13T03:12:34.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.701+0000 7f1a61ffb700 1 -- 172.21.15.64:0/3935160859 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+2969 (secure 0 0 0) 0x7f1a58098f10 con 0x7f1a64100690 2023-09-13T03:12:34.702 INFO:teuthology.orchestra.run.smithi064.stdout:e26 2023-09-13T03:12:34.702 INFO:teuthology.orchestra.run.smithi064.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-09-13T03:12:34.702 INFO:teuthology.orchestra.run.smithi064.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-13T03:12:34.702 INFO:teuthology.orchestra.run.smithi064.stdout:legacy client fscid: 1 2023-09-13T03:12:34.703 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:12:34.703 INFO:teuthology.orchestra.run.smithi064.stdout:Filesystem 'cephfs' (1) 2023-09-13T03:12:34.703 INFO:teuthology.orchestra.run.smithi064.stdout:fs_name cephfs 2023-09-13T03:12:34.703 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 26 2023-09-13T03:12:34.703 INFO:teuthology.orchestra.run.smithi064.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-09-13T03:12:34.704 INFO:teuthology.orchestra.run.smithi064.stdout:created 2023-09-13T03:11:10.490260+0000 2023-09-13T03:12:34.704 INFO:teuthology.orchestra.run.smithi064.stdout:modified 2023-09-13T03:11:18.230740+0000 2023-09-13T03:12:34.704 INFO:teuthology.orchestra.run.smithi064.stdout:tableserver 0 2023-09-13T03:12:34.704 INFO:teuthology.orchestra.run.smithi064.stdout:root 0 2023-09-13T03:12:34.704 INFO:teuthology.orchestra.run.smithi064.stdout:session_timeout 300 2023-09-13T03:12:34.705 INFO:teuthology.orchestra.run.smithi064.stdout:session_autoclose 300 2023-09-13T03:12:34.705 INFO:teuthology.orchestra.run.smithi064.stdout:max_file_size 1099511627776 2023-09-13T03:12:34.705 INFO:teuthology.orchestra.run.smithi064.stdout:max_xattr_size 65536 2023-09-13T03:12:34.705 INFO:teuthology.orchestra.run.smithi064.stdout:required_client_features {} 2023-09-13T03:12:34.705 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure 0 2023-09-13T03:12:34.706 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure_osd_epoch 0 2023-09-13T03:12:34.706 INFO:teuthology.orchestra.run.smithi064.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-13T03:12:34.706 INFO:teuthology.orchestra.run.smithi064.stdout:max_mds 5 2023-09-13T03:12:34.706 INFO:teuthology.orchestra.run.smithi064.stdout:in 0,1,2,3,4 2023-09-13T03:12:34.706 INFO:teuthology.orchestra.run.smithi064.stdout:up {0=24686,1=24649,2=24635,3=24634,4=24622} 2023-09-13T03:12:34.707 INFO:teuthology.orchestra.run.smithi064.stdout:failed 2023-09-13T03:12:34.707 INFO:teuthology.orchestra.run.smithi064.stdout:damaged 2023-09-13T03:12:34.707 INFO:teuthology.orchestra.run.smithi064.stdout:stopped 2023-09-13T03:12:34.707 INFO:teuthology.orchestra.run.smithi064.stdout:data_pools [3] 2023-09-13T03:12:34.707 INFO:teuthology.orchestra.run.smithi064.stdout:metadata_pool 2 2023-09-13T03:12:34.708 INFO:teuthology.orchestra.run.smithi064.stdout:inline_data disabled 2023-09-13T03:12:34.708 INFO:teuthology.orchestra.run.smithi064.stdout:balancer 2023-09-13T03:12:34.708 INFO:teuthology.orchestra.run.smithi064.stdout:bal_rank_mask -1 2023-09-13T03:12:34.708 INFO:teuthology.orchestra.run.smithi064.stdout:standby_count_wanted 0 2023-09-13T03:12:34.708 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.b{0:24686} state up:active seq 3 addr [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] compat {c=[1],r=[1],i=[fff]}] 2023-09-13T03:12:34.709 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.e{0:24611} state up:standby-replay seq 1 addr [v2:172.21.15.142:6834/2167779918,v1:172.21.15.142:6835/2167779918] compat {c=[1],r=[1],i=[fff]}] 2023-09-13T03:12:34.709 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.i{1:24649} state up:active seq 6 addr [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] compat {c=[1],r=[1],i=[fff]}] 2023-09-13T03:12:34.709 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.l{1:24574} state up:standby-replay seq 1 addr [v2:172.21.15.170:6832/1897366227,v1:172.21.15.170:6833/1897366227] compat {c=[1],r=[1],i=[fff]}] 2023-09-13T03:12:34.709 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.h{2:24635} state up:active seq 7 addr [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] compat {c=[1],r=[1],i=[fff]}] 2023-09-13T03:12:34.709 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.k{2:24569} state up:standby-replay seq 1 addr [v2:172.21.15.142:6832/101906999,v1:172.21.15.142:6833/101906999] compat {c=[1],r=[1],i=[fff]}] 2023-09-13T03:12:34.710 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.c{3:24634} state up:active seq 8 addr [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] compat {c=[1],r=[1],i=[fff]}] 2023-09-13T03:12:34.710 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.a{3:14649} state up:standby-replay seq 1 addr [v2:172.21.15.64:6840/1866582875,v1:172.21.15.64:6841/1866582875] compat {c=[1],r=[1],i=[fff]}] 2023-09-13T03:12:34.710 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.f{4:24622} state up:active seq 9 addr [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] compat {c=[1],r=[1],i=[fff]}] 2023-09-13T03:12:34.710 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.g{4:14637} state up:standby-replay seq 1 addr [v2:172.21.15.64:6838/99523374,v1:172.21.15.64:6839/99523374] compat {c=[1],r=[1],i=[fff]}] 2023-09-13T03:12:34.710 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:12:34.711 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:12:34.711 INFO:teuthology.orchestra.run.smithi064.stdout:Standby daemons: 2023-09-13T03:12:34.711 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:12:34.711 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.j{-1:14571} state up:standby seq 1 addr [v2:172.21.15.64:6834/2234182985,v1:172.21.15.64:6835/2234182985] compat {c=[1],r=[1],i=[fff]}] 2023-09-13T03:12:34.711 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.d{-1:14580} state up:standby seq 1 addr [v2:172.21.15.64:6836/4053841873,v1:172.21.15.64:6837/4053841873] compat {c=[1],r=[1],i=[fff]}] 2023-09-13T03:12:34.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.703+0000 7f1a6b125700 1 -- 172.21.15.64:0/3935160859 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f1a540a0640 msgr2=0x7f1a540a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:34.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.703+0000 7f1a6b125700 1 --2- 172.21.15.64:0/3935160859 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f1a540a0640 0x7f1a540a2b00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000b580 tx=0x7f1a50005fb0 comp rx=0 tx=0).stop 2023-09-13T03:12:34.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.703+0000 7f1a6b125700 1 -- 172.21.15.64:0/3935160859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a64100690 msgr2=0x7f1a641985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:34.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.703+0000 7f1a6b125700 1 --2- 172.21.15.64:0/3935160859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a64100690 0x7f1a641985b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800e980 tx=0x7f1a5800ec90 comp rx=0 tx=0).stop 2023-09-13T03:12:34.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.703+0000 7f1a6b125700 1 -- 172.21.15.64:0/3935160859 shutdown_connections 2023-09-13T03:12:34.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.703+0000 7f1a6b125700 1 --2- 172.21.15.64:0/3935160859 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f1a540a0640 0x7f1a540a2b00 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:34.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.703+0000 7f1a6b125700 1 --2- 172.21.15.64:0/3935160859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a64102230 0x7f1a6419fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:34.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.703+0000 7f1a6b125700 1 --2- 172.21.15.64:0/3935160859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a64100690 0x7f1a641985b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:34.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.703+0000 7f1a6b125700 1 --2- 172.21.15.64:0/3935160859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a64101890 0x7f1a64198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:34.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.703+0000 7f1a6b125700 1 -- 172.21.15.64:0/3935160859 >> 172.21.15.64:0/3935160859 conn(0x7f1a640fbbf0 msgr2=0x7f1a641071d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:34.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.703+0000 7f1a6b125700 1 -- 172.21.15.64:0/3935160859 shutdown_connections 2023-09-13T03:12:34.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:34.703+0000 7f1a6b125700 1 -- 172.21.15.64:0/3935160859 wait complete. 2023-09-13T03:12:34.715 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 26 2023-09-13T03:12:35.133 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- bash -c 'ceph osd dump' 2023-09-13T03:12:35.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:35 smithi064 ceph-mon[97675]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2023-09-13T03:12:35.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:35 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/3935160859' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-09-13T03:12:35.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:35 smithi142 ceph-mon[134082]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2023-09-13T03:12:35.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:35 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/3935160859' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-09-13T03:12:35.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:35 smithi170 ceph-mon[134328]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2023-09-13T03:12:35.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:35 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/3935160859' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-09-13T03:12:35.769 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:12:37.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.140+0000 7f0d59e65700 1 -- 172.21.15.64:0/3825386371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d54100690 msgr2=0x7f0d54100ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:37.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.140+0000 7f0d59e65700 1 --2- 172.21.15.64:0/3825386371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d54100690 0x7f0d54100ab0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0d440099c0 tx=0x7f0d44009cd0 comp rx=0 tx=0).stop 2023-09-13T03:12:37.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.140+0000 7f0d59e65700 1 -- 172.21.15.64:0/3825386371 shutdown_connections 2023-09-13T03:12:37.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.140+0000 7f0d59e65700 1 --2- 172.21.15.64:0/3825386371 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d54102230 0x7f0d54108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:37.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.140+0000 7f0d59e65700 1 --2- 172.21.15.64:0/3825386371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d54100690 0x7f0d54100ab0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:37.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.140+0000 7f0d59e65700 1 --2- 172.21.15.64:0/3825386371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d54101890 0x7f0d54101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:37.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.140+0000 7f0d59e65700 1 -- 172.21.15.64:0/3825386371 >> 172.21.15.64:0/3825386371 conn(0x7f0d540fbbf0 msgr2=0x7f0d540fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:37.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.140+0000 7f0d59e65700 1 -- 172.21.15.64:0/3825386371 shutdown_connections 2023-09-13T03:12:37.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.141+0000 7f0d59e65700 1 -- 172.21.15.64:0/3825386371 wait complete. 2023-09-13T03:12:37.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.141+0000 7f0d59e65700 1 Processor -- start 2023-09-13T03:12:37.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.142+0000 7f0d59e65700 1 -- start start 2023-09-13T03:12:37.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.142+0000 7f0d59e65700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d54100690 0x7f0d540743a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:37.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.142+0000 7f0d59e65700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d54101890 0x7f0d540729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:37.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.142+0000 7f0d59e65700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d54102230 0x7f0d54072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:37.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.142+0000 7f0d59e65700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d540736b0 con 0x7f0d54101890 2023-09-13T03:12:37.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.142+0000 7f0d59e65700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d54073820 con 0x7f0d54102230 2023-09-13T03:12:37.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.142+0000 7f0d59e65700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d541a03f0 con 0x7f0d54100690 2023-09-13T03:12:37.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.142+0000 7f0d52ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d54101890 0x7f0d540729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:37.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.142+0000 7f0d52ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d54101890 0x7f0d540729f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53992/0 (socket says 172.21.15.64:53992) 2023-09-13T03:12:37.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.142+0000 7f0d52ffd700 1 -- 172.21.15.64:0/671475286 learned_addr learned my addr 172.21.15.64:0/671475286 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:37.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.142+0000 7f0d537fe700 1 --2- 172.21.15.64:0/671475286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d54100690 0x7f0d540743a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:37.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.143+0000 7f0d53fff700 1 --2- 172.21.15.64:0/671475286 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d54102230 0x7f0d54072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:37.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.143+0000 7f0d52ffd700 1 -- 172.21.15.64:0/671475286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d54100690 msgr2=0x7f0d540743a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:37.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.143+0000 7f0d52ffd700 1 --2- 172.21.15.64:0/671475286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d54100690 0x7f0d540743a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:37.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.143+0000 7f0d52ffd700 1 -- 172.21.15.64:0/671475286 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d54102230 msgr2=0x7f0d54072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:37.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.143+0000 7f0d52ffd700 1 --2- 172.21.15.64:0/671475286 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d54102230 0x7f0d54072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:37.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.143+0000 7f0d52ffd700 1 -- 172.21.15.64:0/671475286 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d440095f0 con 0x7f0d54101890 2023-09-13T03:12:37.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.143+0000 7f0d52ffd700 1 --2- 172.21.15.64:0/671475286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d54101890 0x7f0d540729f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0d4800e9a0 tx=0x7f0d4800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:37.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.143+0000 7f0d50ff9700 1 -- 172.21.15.64:0/671475286 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d48009790 con 0x7f0d54101890 2023-09-13T03:12:37.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.143+0000 7f0d50ff9700 1 -- 172.21.15.64:0/671475286 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d48010430 con 0x7f0d54101890 2023-09-13T03:12:37.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.143+0000 7f0d59e65700 1 -- 172.21.15.64:0/671475286 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d541a06d0 con 0x7f0d54101890 2023-09-13T03:12:37.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.143+0000 7f0d50ff9700 1 -- 172.21.15.64:0/671475286 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d48018680 con 0x7f0d54101890 2023-09-13T03:12:37.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.144+0000 7f0d59e65700 1 -- 172.21.15.64:0/671475286 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d541a0c20 con 0x7f0d54101890 2023-09-13T03:12:37.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.145+0000 7f0d59e65700 1 -- 172.21.15.64:0/671475286 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d54107c30 con 0x7f0d54101890 2023-09-13T03:12:37.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.147+0000 7f0d50ff9700 1 -- 172.21.15.64:0/671475286 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f0d480105a0 con 0x7f0d54101890 2023-09-13T03:12:37.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.148+0000 7f0d50ff9700 1 --2- 172.21.15.64:0/671475286 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f0d3c0a06c0 0x7f0d3c0a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:37.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.148+0000 7f0d50ff9700 1 -- 172.21.15.64:0/671475286 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f0d48014070 con 0x7f0d54101890 2023-09-13T03:12:37.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.148+0000 7f0d537fe700 1 --2- 172.21.15.64:0/671475286 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f0d3c0a06c0 0x7f0d3c0a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:37.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.149+0000 7f0d537fe700 1 --2- 172.21.15.64:0/671475286 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f0d3c0a06c0 0x7f0d3c0a2b80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0d44009990 tx=0x7f0d44005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:37.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.152+0000 7f0d50ff9700 1 -- 172.21.15.64:0/671475286 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0d48051e10 con 0x7f0d54101890 2023-09-13T03:12:37.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.361+0000 7f0d59e65700 1 -- 172.21.15.64:0/671475286 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f0d5410bf10 con 0x7f0d54101890 2023-09-13T03:12:37.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.362+0000 7f0d50ff9700 1 -- 172.21.15.64:0/671475286 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v85) v1 ==== 56+0+5264 (secure 0 0 0) 0x7f0d48095820 con 0x7f0d54101890 2023-09-13T03:12:37.363 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 85 2023-09-13T03:12:37.363 INFO:teuthology.orchestra.run.smithi064.stdout:fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da 2023-09-13T03:12:37.364 INFO:teuthology.orchestra.run.smithi064.stdout:created 2023-09-13T03:03:24.167202+0000 2023-09-13T03:12:37.364 INFO:teuthology.orchestra.run.smithi064.stdout:modified 2023-09-13T03:11:24.258871+0000 2023-09-13T03:12:37.364 INFO:teuthology.orchestra.run.smithi064.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-09-13T03:12:37.364 INFO:teuthology.orchestra.run.smithi064.stdout:crush_version 26 2023-09-13T03:12:37.364 INFO:teuthology.orchestra.run.smithi064.stdout:full_ratio 0.95 2023-09-13T03:12:37.365 INFO:teuthology.orchestra.run.smithi064.stdout:backfillfull_ratio 0.9 2023-09-13T03:12:37.365 INFO:teuthology.orchestra.run.smithi064.stdout:nearfull_ratio 0.85 2023-09-13T03:12:37.365 INFO:teuthology.orchestra.run.smithi064.stdout:require_min_compat_client luminous 2023-09-13T03:12:37.365 INFO:teuthology.orchestra.run.smithi064.stdout:min_compat_client luminous 2023-09-13T03:12:37.365 INFO:teuthology.orchestra.run.smithi064.stdout:require_osd_release reef 2023-09-13T03:12:37.366 INFO:teuthology.orchestra.run.smithi064.stdout:stretch_mode_enabled false 2023-09-13T03:12:37.366 INFO:teuthology.orchestra.run.smithi064.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-09-13T03:12:37.366 INFO:teuthology.orchestra.run.smithi064.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2023-09-13T03:12:37.366 INFO:teuthology.orchestra.run.smithi064.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 83 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2023-09-13T03:12:37.366 INFO:teuthology.orchestra.run.smithi064.stdout:max_osd 12 2023-09-13T03:12:37.367 INFO:teuthology.orchestra.run.smithi064.stdout:osd.0 up in weight 1 up_from 8 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6802/3940892135,v1:172.21.15.64:6803/3940892135] [v2:172.21.15.64:6804/3940892135,v1:172.21.15.64:6805/3940892135] exists,up 3997c114-241b-48c4-8feb-653c99209470 2023-09-13T03:12:37.367 INFO:teuthology.orchestra.run.smithi064.stdout:osd.1 up in weight 1 up_from 13 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6800/4188149333,v1:172.21.15.142:6801/4188149333] [v2:172.21.15.142:6802/4188149333,v1:172.21.15.142:6803/4188149333] exists,up 94cfae85-322e-4ed2-a6e6-e40349bb14bf 2023-09-13T03:12:37.367 INFO:teuthology.orchestra.run.smithi064.stdout:osd.2 up in weight 1 up_from 18 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6800/2406669338,v1:172.21.15.170:6801/2406669338] [v2:172.21.15.170:6802/2406669338,v1:172.21.15.170:6803/2406669338] exists,up b3aae17e-96db-4008-8c27-f733c46b30ee 2023-09-13T03:12:37.367 INFO:teuthology.orchestra.run.smithi064.stdout:osd.3 up in weight 1 up_from 25 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6810/3558318707,v1:172.21.15.64:6811/3558318707] [v2:172.21.15.64:6812/3558318707,v1:172.21.15.64:6813/3558318707] exists,up 7fa8d8cb-2ab7-48a7-8b49-5e2903d6779f 2023-09-13T03:12:37.367 INFO:teuthology.orchestra.run.smithi064.stdout:osd.4 up in weight 1 up_from 33 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6808/1811142576,v1:172.21.15.142:6809/1811142576] [v2:172.21.15.142:6810/1811142576,v1:172.21.15.142:6811/1811142576] exists,up 8c7a9c9f-f919-48d3-9db2-214a25bb5603 2023-09-13T03:12:37.368 INFO:teuthology.orchestra.run.smithi064.stdout:osd.5 up in weight 1 up_from 38 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6808/2903841453,v1:172.21.15.170:6809/2903841453] [v2:172.21.15.170:6810/2903841453,v1:172.21.15.170:6811/2903841453] exists,up 0c6101c6-2065-4022-a751-d3f216d7f478 2023-09-13T03:12:37.368 INFO:teuthology.orchestra.run.smithi064.stdout:osd.6 up in weight 1 up_from 46 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6818/1436584490,v1:172.21.15.64:6819/1436584490] [v2:172.21.15.64:6820/1436584490,v1:172.21.15.64:6821/1436584490] exists,up 5cf3e52e-d155-41b1-9194-db047d26a953 2023-09-13T03:12:37.368 INFO:teuthology.orchestra.run.smithi064.stdout:osd.7 up in weight 1 up_from 51 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6816/3726793534,v1:172.21.15.142:6817/3726793534] [v2:172.21.15.142:6818/3726793534,v1:172.21.15.142:6819/3726793534] exists,up e97323e3-badd-41a7-af9e-e78a81a64046 2023-09-13T03:12:37.368 INFO:teuthology.orchestra.run.smithi064.stdout:osd.8 up in weight 1 up_from 58 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6816/4272513873,v1:172.21.15.170:6817/4272513873] [v2:172.21.15.170:6818/4272513873,v1:172.21.15.170:6819/4272513873] exists,up 83d2835c-ed26-493d-9222-c2894ec0e071 2023-09-13T03:12:37.368 INFO:teuthology.orchestra.run.smithi064.stdout:osd.9 up in weight 1 up_from 66 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6826/2641175211,v1:172.21.15.64:6827/2641175211] [v2:172.21.15.64:6828/2641175211,v1:172.21.15.64:6829/2641175211] exists,up cb936bdd-1bec-4236-aa2e-a24adcac11e3 2023-09-13T03:12:37.369 INFO:teuthology.orchestra.run.smithi064.stdout:osd.10 up in weight 1 up_from 71 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6824/936499217,v1:172.21.15.142:6825/936499217] [v2:172.21.15.142:6826/936499217,v1:172.21.15.142:6827/936499217] exists,up 85512a1c-6171-43d4-9c15-c588a8b71be0 2023-09-13T03:12:37.369 INFO:teuthology.orchestra.run.smithi064.stdout:osd.11 up in weight 1 up_from 78 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6824/3605017060,v1:172.21.15.170:6825/3605017060] [v2:172.21.15.170:6826/3605017060,v1:172.21.15.170:6827/3605017060] exists,up 9b8417dc-5d6c-4917-94a9-b032c5bc33d6 2023-09-13T03:12:37.369 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 2.6 [8,11] 2023-09-13T03:12:37.369 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 2.b [2,4] 2023-09-13T03:12:37.369 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 2.13 [2,11] 2023-09-13T03:12:37.370 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 3.1b [5,9] 2023-09-13T03:12:37.370 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 3.1d [4,8] 2023-09-13T03:12:37.370 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 3.24 [4,9] 2023-09-13T03:12:37.370 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 3.3c [4,9] 2023-09-13T03:12:37.370 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:6800/2955227347 expires 2023-09-14T03:04:22.355817+0000 2023-09-13T03:12:37.371 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:0/1373223118 expires 2023-09-14T03:04:22.355817+0000 2023-09-13T03:12:37.371 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:0/271194613 expires 2023-09-14T03:04:22.355817+0000 2023-09-13T03:12:37.371 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:6801/3998091347 expires 2023-09-14T03:03:56.442541+0000 2023-09-13T03:12:37.371 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:6800/3998091347 expires 2023-09-14T03:03:56.442541+0000 2023-09-13T03:12:37.371 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:0/920818572 expires 2023-09-14T03:03:56.442541+0000 2023-09-13T03:12:37.371 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:0/735789005 expires 2023-09-14T03:03:56.442541+0000 2023-09-13T03:12:37.372 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:6801/2955227347 expires 2023-09-14T03:04:22.355817+0000 2023-09-13T03:12:37.372 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:0/1975923178 expires 2023-09-14T03:03:56.442541+0000 2023-09-13T03:12:37.372 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:0/1500151178 expires 2023-09-14T03:04:22.355817+0000 2023-09-13T03:12:37.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.365+0000 7f0d59e65700 1 -- 172.21.15.64:0/671475286 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f0d3c0a06c0 msgr2=0x7f0d3c0a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:37.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.365+0000 7f0d59e65700 1 --2- 172.21.15.64:0/671475286 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f0d3c0a06c0 0x7f0d3c0a2b80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0d44009990 tx=0x7f0d44005fb0 comp rx=0 tx=0).stop 2023-09-13T03:12:37.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.365+0000 7f0d59e65700 1 -- 172.21.15.64:0/671475286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d54101890 msgr2=0x7f0d540729f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:37.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.365+0000 7f0d59e65700 1 --2- 172.21.15.64:0/671475286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d54101890 0x7f0d540729f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0d4800e9a0 tx=0x7f0d4800ecb0 comp rx=0 tx=0).stop 2023-09-13T03:12:37.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.365+0000 7f0d59e65700 1 -- 172.21.15.64:0/671475286 shutdown_connections 2023-09-13T03:12:37.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.365+0000 7f0d59e65700 1 --2- 172.21.15.64:0/671475286 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f0d3c0a06c0 0x7f0d3c0a2b80 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:37.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.365+0000 7f0d59e65700 1 --2- 172.21.15.64:0/671475286 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d54102230 0x7f0d54072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:37.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.365+0000 7f0d59e65700 1 --2- 172.21.15.64:0/671475286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d54100690 0x7f0d540743a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:37.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.365+0000 7f0d59e65700 1 --2- 172.21.15.64:0/671475286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d54101890 0x7f0d540729f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:37.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.365+0000 7f0d59e65700 1 -- 172.21.15.64:0/671475286 >> 172.21.15.64:0/671475286 conn(0x7f0d540fbbf0 msgr2=0x7f0d54104d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:37.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.365+0000 7f0d59e65700 1 -- 172.21.15.64:0/671475286 shutdown_connections 2023-09-13T03:12:37.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:37.365+0000 7f0d59e65700 1 -- 172.21.15.64:0/671475286 wait complete. 2023-09-13T03:12:37.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:37 smithi064 ceph-mon[97675]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 170 B/s wr, 9 op/s 2023-09-13T03:12:37.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:37 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/671475286' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-09-13T03:12:37.843 INFO:teuthology.run_tasks:Running task fs.ready... 2023-09-13T03:12:37.855 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-13T03:12:37.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:37 smithi170 ceph-mon[134328]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 170 B/s wr, 9 op/s 2023-09-13T03:12:37.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:37 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/671475286' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-09-13T03:12:37.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:37 smithi142 ceph-mon[134082]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 170 B/s wr, 9 op/s 2023-09-13T03:12:37.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:37 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/671475286' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-09-13T03:12:38.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.064+0000 7f67fd9ff700 1 -- 172.21.15.64:0/3834432763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67f81021b0 msgr2=0x7f67f8108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:38.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.064+0000 7f67fd9ff700 1 --2- 172.21.15.64:0/3834432763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67f81021b0 0x7f67f8108a60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f67f000b0d0 tx=0x7f67f000b3e0 comp rx=0 tx=0).stop 2023-09-13T03:12:38.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.064+0000 7f67fd9ff700 1 -- 172.21.15.64:0/3834432763 shutdown_connections 2023-09-13T03:12:38.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.064+0000 7f67fd9ff700 1 --2- 172.21.15.64:0/3834432763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67f8100620 0x7f67f8100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:38.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.064+0000 7f67fd9ff700 1 --2- 172.21.15.64:0/3834432763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67f81021b0 0x7f67f8108a60 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:38.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.064+0000 7f67fd9ff700 1 --2- 172.21.15.64:0/3834432763 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67f8101810 0x7f67f8101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:38.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.064+0000 7f67fd9ff700 1 -- 172.21.15.64:0/3834432763 >> 172.21.15.64:0/3834432763 conn(0x7f67f80fbb70 msgr2=0x7f67f80fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:38.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.064+0000 7f67fd9ff700 1 -- 172.21.15.64:0/3834432763 shutdown_connections 2023-09-13T03:12:38.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.064+0000 7f67fd9ff700 1 -- 172.21.15.64:0/3834432763 wait complete. 2023-09-13T03:12:38.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.066+0000 7f67fd9ff700 1 Processor -- start 2023-09-13T03:12:38.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.066+0000 7f67fd9ff700 1 -- start start 2023-09-13T03:12:38.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.066+0000 7f67fd9ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67f8100620 0x7f67f8196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:38.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.066+0000 7f67fd9ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67f8101810 0x7f67f8196970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:38.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.066+0000 7f67fd9ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67f81021b0 0x7f67f819d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:38.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.066+0000 7f67fd9ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f819dfb0 con 0x7f67f8100620 2023-09-13T03:12:38.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.066+0000 7f67fd9ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f819e0f0 con 0x7f67f8101810 2023-09-13T03:12:38.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.066+0000 7f67fd9ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f819e280 con 0x7f67f81021b0 2023-09-13T03:12:38.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.066+0000 7f67f6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67f8100620 0x7f67f8196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:38.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.066+0000 7f67f6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67f8100620 0x7f67f8196430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54006/0 (socket says 172.21.15.64:54006) 2023-09-13T03:12:38.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.066+0000 7f67f6ffd700 1 -- 172.21.15.64:0/3569000146 learned_addr learned my addr 172.21.15.64:0/3569000146 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:38.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.066+0000 7f67f77fe700 1 --2- 172.21.15.64:0/3569000146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67f81021b0 0x7f67f819d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:38.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.066+0000 7f67f67fc700 1 --2- 172.21.15.64:0/3569000146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67f8101810 0x7f67f8196970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:38.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.067+0000 7f67f6ffd700 1 -- 172.21.15.64:0/3569000146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67f81021b0 msgr2=0x7f67f819d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:38.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.067+0000 7f67f6ffd700 1 --2- 172.21.15.64:0/3569000146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67f81021b0 0x7f67f819d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:38.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.067+0000 7f67f6ffd700 1 -- 172.21.15.64:0/3569000146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67f8101810 msgr2=0x7f67f8196970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:38.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.067+0000 7f67f6ffd700 1 --2- 172.21.15.64:0/3569000146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67f8101810 0x7f67f8196970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:38.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.067+0000 7f67f6ffd700 1 -- 172.21.15.64:0/3569000146 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f0009d00 con 0x7f67f8100620 2023-09-13T03:12:38.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.067+0000 7f67f6ffd700 1 --2- 172.21.15.64:0/3569000146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67f8100620 0x7f67f8196430 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f67e800d980 tx=0x7f67e800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:38.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.067+0000 7f67fc9fd700 1 -- 172.21.15.64:0/3569000146 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e800f890 con 0x7f67f8100620 2023-09-13T03:12:38.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.067+0000 7f67fc9fd700 1 -- 172.21.15.64:0/3569000146 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67e8010460 con 0x7f67f8100620 2023-09-13T03:12:38.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.067+0000 7f67fc9fd700 1 -- 172.21.15.64:0/3569000146 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e80195c0 con 0x7f67f8100620 2023-09-13T03:12:38.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.067+0000 7f67fd9ff700 1 -- 172.21.15.64:0/3569000146 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67f819e560 con 0x7f67f8100620 2023-09-13T03:12:38.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.068+0000 7f67fd9ff700 1 -- 172.21.15.64:0/3569000146 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67f819ea20 con 0x7f67f8100620 2023-09-13T03:12:38.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.068+0000 7f67fd9ff700 1 -- 172.21.15.64:0/3569000146 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67f8107b10 con 0x7f67f8100620 2023-09-13T03:12:38.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.070+0000 7f67fc9fd700 1 -- 172.21.15.64:0/3569000146 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f67e80105d0 con 0x7f67f8100620 2023-09-13T03:12:38.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.072+0000 7f67fc9fd700 1 --2- 172.21.15.64:0/3569000146 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f67e00a0710 0x7f67e00a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:38.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.072+0000 7f67fc9fd700 1 -- 172.21.15.64:0/3569000146 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f67e80cb840 con 0x7f67f8100620 2023-09-13T03:12:38.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.072+0000 7f67f67fc700 1 --2- 172.21.15.64:0/3569000146 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f67e00a0710 0x7f67e00a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:38.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.073+0000 7f67f67fc700 1 --2- 172.21.15.64:0/3569000146 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f67e00a0710 0x7f67e00a2bd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f67ec009be0 tx=0x7f67ec009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:38.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.076+0000 7f67fc9fd700 1 -- 172.21.15.64:0/3569000146 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f67e8095340 con 0x7f67f8100620 2023-09-13T03:12:38.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.288+0000 7f67fd9ff700 1 -- 172.21.15.64:0/3569000146 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67f810be90 con 0x7f67f8100620 2023-09-13T03:12:38.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.290+0000 7f67fc9fd700 1 -- 172.21.15.64:0/3569000146 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22473 (secure 0 0 0) 0x7f67e8098960 con 0x7f67f8100620 2023-09-13T03:12:38.291 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:12:38.291 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":85,"fsid":"f8dbced4-51e1-11ee-9ab7-7b867c8bd7da","created":"2023-09-13T03:03:24.167202+0000","modified":"2023-09-13T03:11:24.258871+0000","last_up_change":"2023-09-13T03:10:22.503529+0000","last_in_change":"2023-09-13T03:10:05.678720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T03:06:58.987479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-13T03:11:08.589202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-13T03:11:09.924659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3997c114-241b-48c4-8feb-653c99209470","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6803","nonce":3940892135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6805","nonce":3940892135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6809","nonce":3940892135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6807","nonce":3940892135}]},"public_addr":"172.21.15.64:6803/3940892135","cluster_addr":"172.21.15.64:6805/3940892135","heartbeat_back_addr":"172.21.15.64:6809/3940892135","heartbeat_front_addr":"172.21.15.64:6807/3940892135","state":["exists","up"]},{"osd":1,"uuid":"94cfae85-322e-4ed2-a6e6-e40349bb14bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6801","nonce":4188149333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6803","nonce":4188149333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6807","nonce":4188149333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6805","nonce":4188149333}]},"public_addr":"172.21.15.142:6801/4188149333","cluster_addr":"172.21.15.142:6803/4188149333","heartbeat_back_addr":"172.21.15.142:6807/4188149333","heartbeat_front_addr":"172.21.15.142:6805/4188149333","state":["exists","up"]},{"osd":2,"uuid":"b3aae17e-96db-4008-8c27-f733c46b30ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6801","nonce":2406669338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6803","nonce":2406669338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6807","nonce":2406669338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6805","nonce":2406669338}]},"public_addr":"172.21.15.170:6801/2406669338","cluster_addr":"172.21.15.170:6803/2406669338","heartbeat_back_addr":"172.21.15.170:6807/2406669338","heartbeat_front_addr":"172.21.15.170:6805/2406669338","state":["exists","up"]},{"osd":3,"uuid":"7fa8d8cb-2ab7-48a7-8b49-5e2903d6779f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6811","nonce":3558318707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6813","nonce":3558318707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6817","nonce":3558318707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6815","nonce":3558318707}]},"public_addr":"172.21.15.64:6811/3558318707","cluster_addr":"172.21.15.64:6813/3558318707","heartbeat_back_addr":"172.21.15.64:6817/3558318707","heartbeat_front_addr":"172.21.15.64:6815/3558318707","state":["exists","up"]},{"osd":4,"uuid":"8c7a9c9f-f919-48d3-9db2-214a25bb5603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6809","nonce":1811142576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6811","nonce":1811142576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6815","nonce":1811142576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6813","nonce":1811142576}]},"public_addr":"172.21.15.142:6809/1811142576","cluster_addr":"172.21.15.142:6811/1811142576","heartbeat_back_addr":"172.21.15.142:6815/1811142576","heartbeat_front_addr":"172.21.15.142:6813/1811142576","state":["exists","up"]},{"osd":5,"uuid":"0c6101c6-2065-4022-a751-d3f216d7f478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6809","nonce":2903841453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6811","nonce":2903841453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6815","nonce":2903841453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6813","nonce":2903841453}]},"public_addr":"172.21.15.170:6809/2903841453","cluster_addr":"172.21.15.170:6811/2903841453","heartbeat_back_addr":"172.21.15.170:6815/2903841453","heartbeat_front_addr":"172.21.15.170:6813/2903841453","state":["exists","up"]},{"osd":6,"uuid":"5cf3e52e-d155-41b1-9194-db047d26a953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6819","nonce":1436584490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6821","nonce":1436584490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6825","nonce":1436584490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6823","nonce":1436584490}]},"public_addr":"172.21.15.64:6819/1436584490","cluster_addr":"172.21.15.64:6821/1436584490","heartbeat_back_addr":"172.21.15.64:6825/1436584490","heartbeat_front_addr":"172.21.15.64:6823/1436584490","state":["exists","up"]},{"osd":7,"uuid":"e97323e3-badd-41a7-af9e-e78a81a64046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6817","nonce":3726793534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6819","nonce":3726793534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6823","nonce":3726793534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6821","nonce":3726793534}]},"public_addr":"172.21.15.142:6817/3726793534","cluster_addr":"172.21.15.142:6819/3726793534","heartbeat_back_addr":"172.21.15.142:6823/3726793534","heartbeat_front_addr":"172.21.15.142:6821/3726793534","state":["exists","up"]},{"osd":8,"uuid":"83d2835c-ed26-493d-9222-c2894ec0e071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6817","nonce":4272513873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6819","nonce":4272513873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6823","nonce":4272513873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6821","nonce":4272513873}]},"public_addr":"172.21.15.170:6817/4272513873","cluster_addr":"172.21.15.170:6819/4272513873","heartbeat_back_addr":"172.21.15.170:6823/4272513873","heartbeat_front_addr":"172.21.15.170:6821/4272513873","state":["exists","up"]},{"osd":9,"uuid":"cb936bdd-1bec-4236-aa2e-a24adcac11e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6827","nonce":2641175211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6829","nonce":2641175211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6833","nonce":2641175211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6831","nonce":2641175211}]},"public_addr":"172.21.15.64:6827/2641175211","cluster_addr":"172.21.15.64:6829/2641175211","heartbeat_back_addr":"172.21.15.64:6833/2641175211","heartbeat_front_addr":"172.21.15.64:6831/2641175211","state":["exists","up"]},{"osd":10,"uuid":"85512a1c-6171-43d4-9c15-c588a8b71be0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6825","nonce":936499217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6827","nonce":936499217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6831","nonce":936499217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6829","nonce":936499217}]},"public_addr":"172.21.15.142:6825/936499217","cluster_addr":"172.21.15.142:6827/936499217","heartbeat_back_addr":"172.21.15.142:6831/936499217","heartbeat_front_addr":"172.21.15.142:6829/936499217","state":["exists","up"]},{"osd":11,"uuid":"9b8417dc-5d6c-4917-94a9-b032c5bc33d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6825","nonce":3605017060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6827","nonce":3605017060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6831","nonce":3605017060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6829","nonce":3605017060}]},"public_addr":"172.21.15.170:6825/3605017060","cluster_addr":"172.21.15.170:6827/3605017060","heartbeat_back_addr":"172.21.15.170:6831/3605017060","heartbeat_front_addr":"172.21.15.170:6829/3605017060","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:14.229044+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:36.147777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:56.655298+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:07:23.107741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:07:44.663770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:05.577432+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:31.168031+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:52.791055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:09:13.658546+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:09:39.150360+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:10:00.601735+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:10:21.407555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6800/2955227347":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/1373223118":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/271194613":"2023-09-14T03:04:22.355817+0000","172.21.15.64:6801/3998091347":"2023-09-14T03:03:56.442541+0000","172.21.15.64:6800/3998091347":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/920818572":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/735789005":"2023-09-14T03:03:56.442541+0000","172.21.15.64:6801/2955227347":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/1975923178":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/1500151178":"2023-09-14T03:04:22.355817+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T03:12:38.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.292+0000 7f67fd9ff700 1 -- 172.21.15.64:0/3569000146 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f67e00a0710 msgr2=0x7f67e00a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:38.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.292+0000 7f67fd9ff700 1 --2- 172.21.15.64:0/3569000146 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f67e00a0710 0x7f67e00a2bd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f67ec009be0 tx=0x7f67ec009260 comp rx=0 tx=0).stop 2023-09-13T03:12:38.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.292+0000 7f67fd9ff700 1 -- 172.21.15.64:0/3569000146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67f8100620 msgr2=0x7f67f8196430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:38.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.292+0000 7f67fd9ff700 1 --2- 172.21.15.64:0/3569000146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67f8100620 0x7f67f8196430 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f67e800d980 tx=0x7f67e800dc90 comp rx=0 tx=0).stop 2023-09-13T03:12:38.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.292+0000 7f67fd9ff700 1 -- 172.21.15.64:0/3569000146 shutdown_connections 2023-09-13T03:12:38.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.292+0000 7f67fd9ff700 1 --2- 172.21.15.64:0/3569000146 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f67e00a0710 0x7f67e00a2bd0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:38.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.292+0000 7f67fd9ff700 1 --2- 172.21.15.64:0/3569000146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67f81021b0 0x7f67f819d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:38.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.292+0000 7f67fd9ff700 1 --2- 172.21.15.64:0/3569000146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67f8100620 0x7f67f8196430 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:38.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.292+0000 7f67fd9ff700 1 --2- 172.21.15.64:0/3569000146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67f8101810 0x7f67f8196970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:38.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.292+0000 7f67fd9ff700 1 -- 172.21.15.64:0/3569000146 >> 172.21.15.64:0/3569000146 conn(0x7f67f80fbb70 msgr2=0x7f67f8104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:38.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.293+0000 7f67fd9ff700 1 -- 172.21.15.64:0/3569000146 shutdown_connections 2023-09-13T03:12:38.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.293+0000 7f67fd9ff700 1 -- 172.21.15.64:0/3569000146 wait complete. 2023-09-13T03:12:38.312 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-13T03:06:58.987479+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-13T03:11:08.589202+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-13T03:11:09.924659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-13T03:12:38.313 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-13T03:12:38.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.521+0000 7f0376fff700 1 -- 172.21.15.64:0/150984095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0370100610 msgr2=0x7f0370100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:38.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.521+0000 7f0376fff700 1 --2- 172.21.15.64:0/150984095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0370100610 0x7f0370100a30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f03640099c0 tx=0x7f0364009cd0 comp rx=0 tx=0).stop 2023-09-13T03:12:38.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.521+0000 7f0376fff700 1 -- 172.21.15.64:0/150984095 shutdown_connections 2023-09-13T03:12:38.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.521+0000 7f0376fff700 1 --2- 172.21.15.64:0/150984095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03701021a0 0x7f0370108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:38.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.521+0000 7f0376fff700 1 --2- 172.21.15.64:0/150984095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0370100610 0x7f0370100a30 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:38.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.521+0000 7f0376fff700 1 --2- 172.21.15.64:0/150984095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0370101800 0x7f0370101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:38.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.521+0000 7f0376fff700 1 -- 172.21.15.64:0/150984095 >> 172.21.15.64:0/150984095 conn(0x7f03700fbb80 msgr2=0x7f03700fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:38.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.521+0000 7f0376fff700 1 -- 172.21.15.64:0/150984095 shutdown_connections 2023-09-13T03:12:38.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.521+0000 7f0376fff700 1 -- 172.21.15.64:0/150984095 wait complete. 2023-09-13T03:12:38.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.522+0000 7f0376fff700 1 Processor -- start 2023-09-13T03:12:38.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.522+0000 7f0376fff700 1 -- start start 2023-09-13T03:12:38.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.522+0000 7f0376fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0370100610 0x7f03700743b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:38.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.522+0000 7f0376fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0370101800 0x7f03700729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:38.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.522+0000 7f0376fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03701021a0 0x7f0370072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:38.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.522+0000 7f0376fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03700736d0 con 0x7f03701021a0 2023-09-13T03:12:38.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.522+0000 7f0376fff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0370073840 con 0x7f0370101800 2023-09-13T03:12:38.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.522+0000 7f0376fff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03701a03c0 con 0x7f0370100610 2023-09-13T03:12:38.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.523+0000 7f037559c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03701021a0 0x7f0370072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:38.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.523+0000 7f037559c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03701021a0 0x7f0370072f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54028/0 (socket says 172.21.15.64:54028) 2023-09-13T03:12:38.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.523+0000 7f037559c700 1 -- 172.21.15.64:0/510220343 learned_addr learned my addr 172.21.15.64:0/510220343 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:38.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.523+0000 7f0374d9b700 1 --2- 172.21.15.64:0/510220343 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0370100610 0x7f03700743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:38.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.523+0000 7f036ffff700 1 --2- 172.21.15.64:0/510220343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0370101800 0x7f03700729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:38.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.523+0000 7f037559c700 1 -- 172.21.15.64:0/510220343 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0370100610 msgr2=0x7f03700743b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:38.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.523+0000 7f037559c700 1 --2- 172.21.15.64:0/510220343 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0370100610 0x7f03700743b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:38.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.523+0000 7f037559c700 1 -- 172.21.15.64:0/510220343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0370101800 msgr2=0x7f03700729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:38.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.523+0000 7f037559c700 1 --2- 172.21.15.64:0/510220343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0370101800 0x7f03700729f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:38.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.523+0000 7f037559c700 1 -- 172.21.15.64:0/510220343 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0368009c40 con 0x7f03701021a0 2023-09-13T03:12:38.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.523+0000 7f036ffff700 1 --2- 172.21.15.64:0/510220343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0370101800 0x7f03700729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:12:38.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.523+0000 7f037559c700 1 --2- 172.21.15.64:0/510220343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03701021a0 0x7f0370072f30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f036800c370 tx=0x7f036800c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:38.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.523+0000 7f036dffb700 1 -- 172.21.15.64:0/510220343 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0368017070 con 0x7f03701021a0 2023-09-13T03:12:38.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.524+0000 7f036dffb700 1 -- 172.21.15.64:0/510220343 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f036800f040 con 0x7f03701021a0 2023-09-13T03:12:38.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.524+0000 7f0376fff700 1 -- 172.21.15.64:0/510220343 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03640095f0 con 0x7f03701021a0 2023-09-13T03:12:38.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.524+0000 7f036dffb700 1 -- 172.21.15.64:0/510220343 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036800e050 con 0x7f03701021a0 2023-09-13T03:12:38.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.524+0000 7f0376fff700 1 -- 172.21.15.64:0/510220343 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03701a09c0 con 0x7f03701021a0 2023-09-13T03:12:38.533 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.525+0000 7f0376fff700 1 -- 172.21.15.64:0/510220343 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0370107590 con 0x7f03701021a0 2023-09-13T03:12:38.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.527+0000 7f036dffb700 1 -- 172.21.15.64:0/510220343 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f0368004520 con 0x7f03701021a0 2023-09-13T03:12:38.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.528+0000 7f036dffb700 1 --2- 172.21.15.64:0/510220343 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f03600a0640 0x7f03600a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:38.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.528+0000 7f036dffb700 1 -- 172.21.15.64:0/510220343 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f03680cb550 con 0x7f03701021a0 2023-09-13T03:12:38.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.528+0000 7f0374d9b700 1 --2- 172.21.15.64:0/510220343 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f03600a0640 0x7f03600a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:38.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.529+0000 7f0374d9b700 1 --2- 172.21.15.64:0/510220343 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f03600a0640 0x7f03600a2b00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0364009990 tx=0x7f0364005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:38.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.532+0000 7f036dffb700 1 -- 172.21.15.64:0/510220343 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0368095050 con 0x7f03701021a0 2023-09-13T03:12:38.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:38 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/3569000146' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:12:38.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.743+0000 7f0376fff700 1 -- 172.21.15.64:0/510220343 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f037002cfa0 con 0x7f03701021a0 2023-09-13T03:12:38.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.743+0000 7f036dffb700 1 -- 172.21.15.64:0/510220343 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0368098670 con 0x7f03701021a0 2023-09-13T03:12:38.744 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-09-13T03:12:38.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.746+0000 7f0376fff700 1 -- 172.21.15.64:0/510220343 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f03600a0640 msgr2=0x7f03600a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:38.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.746+0000 7f0376fff700 1 --2- 172.21.15.64:0/510220343 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f03600a0640 0x7f03600a2b00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0364009990 tx=0x7f0364005fb0 comp rx=0 tx=0).stop 2023-09-13T03:12:38.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.746+0000 7f0376fff700 1 -- 172.21.15.64:0/510220343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03701021a0 msgr2=0x7f0370072f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:38.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.746+0000 7f0376fff700 1 --2- 172.21.15.64:0/510220343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03701021a0 0x7f0370072f30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f036800c370 tx=0x7f036800c680 comp rx=0 tx=0).stop 2023-09-13T03:12:38.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.746+0000 7f0376fff700 1 -- 172.21.15.64:0/510220343 shutdown_connections 2023-09-13T03:12:38.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.746+0000 7f0376fff700 1 --2- 172.21.15.64:0/510220343 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f03600a0640 0x7f03600a2b00 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:38.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.746+0000 7f0376fff700 1 --2- 172.21.15.64:0/510220343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03701021a0 0x7f0370072f30 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:38.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.746+0000 7f0376fff700 1 --2- 172.21.15.64:0/510220343 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0370100610 0x7f03700743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:38.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.746+0000 7f0376fff700 1 --2- 172.21.15.64:0/510220343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0370101800 0x7f03700729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:38.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.746+0000 7f0376fff700 1 -- 172.21.15.64:0/510220343 >> 172.21.15.64:0/510220343 conn(0x7f03700fbb80 msgr2=0x7f03700fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:38.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.746+0000 7f0376fff700 1 -- 172.21.15.64:0/510220343 shutdown_connections 2023-09-13T03:12:38.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.746+0000 7f0376fff700 1 -- 172.21.15.64:0/510220343 wait complete. 2023-09-13T03:12:38.765 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-13T03:12:38.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:38 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/3569000146' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:12:38.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.974+0000 7f11bf3ff700 1 -- 172.21.15.64:0/1622053590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11b8100610 msgr2=0x7f11b8100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:38.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.974+0000 7f11bf3ff700 1 --2- 172.21.15.64:0/1622053590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11b8100610 0x7f11b8100a30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f11b400b0d0 tx=0x7f11b400b3e0 comp rx=0 tx=0).stop 2023-09-13T03:12:38.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.975+0000 7f11bf3ff700 1 -- 172.21.15.64:0/1622053590 shutdown_connections 2023-09-13T03:12:38.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.975+0000 7f11bf3ff700 1 --2- 172.21.15.64:0/1622053590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11b81021a0 0x7f11b8108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:38.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.975+0000 7f11bf3ff700 1 --2- 172.21.15.64:0/1622053590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11b8100610 0x7f11b8100a30 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:38.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.975+0000 7f11bf3ff700 1 --2- 172.21.15.64:0/1622053590 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11b8101800 0x7f11b8101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:38.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.975+0000 7f11bf3ff700 1 -- 172.21.15.64:0/1622053590 >> 172.21.15.64:0/1622053590 conn(0x7f11b80fbb80 msgr2=0x7f11b80fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:38.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.975+0000 7f11bf3ff700 1 -- 172.21.15.64:0/1622053590 shutdown_connections 2023-09-13T03:12:38.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.975+0000 7f11bf3ff700 1 -- 172.21.15.64:0/1622053590 wait complete. 2023-09-13T03:12:38.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.976+0000 7f11bf3ff700 1 Processor -- start 2023-09-13T03:12:38.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.976+0000 7f11bf3ff700 1 -- start start 2023-09-13T03:12:38.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.976+0000 7f11bf3ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11b8100610 0x7f11b81964a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:38.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.976+0000 7f11bf3ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11b8101800 0x7f11b81969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:38.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.976+0000 7f11bf3ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11b81021a0 0x7f11b819da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:38.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.976+0000 7f11bf3ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b819e020 con 0x7f11b8101800 2023-09-13T03:12:38.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.976+0000 7f11bf3ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b819e160 con 0x7f11b81021a0 2023-09-13T03:12:38.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.976+0000 7f11bf3ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b819e2f0 con 0x7f11b8100610 2023-09-13T03:12:38.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.976+0000 7f11bc99a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11b8101800 0x7f11b81969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:38.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.976+0000 7f11bc99a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11b8101800 0x7f11b81969e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54044/0 (socket says 172.21.15.64:54044) 2023-09-13T03:12:38.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.976+0000 7f11bc99a700 1 -- 172.21.15.64:0/1956541900 learned_addr learned my addr 172.21.15.64:0/1956541900 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:38.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.976+0000 7f11bd99c700 1 --2- 172.21.15.64:0/1956541900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11b81021a0 0x7f11b819da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:38.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.976+0000 7f11bd19b700 1 --2- 172.21.15.64:0/1956541900 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11b8100610 0x7f11b81964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:38.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.977+0000 7f11bc99a700 1 -- 172.21.15.64:0/1956541900 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11b8100610 msgr2=0x7f11b81964a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:38.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.977+0000 7f11bc99a700 1 --2- 172.21.15.64:0/1956541900 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11b8100610 0x7f11b81964a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:38.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.977+0000 7f11bc99a700 1 -- 172.21.15.64:0/1956541900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11b81021a0 msgr2=0x7f11b819da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:38.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.977+0000 7f11bc99a700 1 --2- 172.21.15.64:0/1956541900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11b81021a0 0x7f11b819da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:38.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.977+0000 7f11bc99a700 1 -- 172.21.15.64:0/1956541900 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b4009d00 con 0x7f11b8101800 2023-09-13T03:12:38.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.977+0000 7f11bc99a700 1 --2- 172.21.15.64:0/1956541900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11b8101800 0x7f11b81969e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f11a800d980 tx=0x7f11a800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:38.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.977+0000 7f11ae7fc700 1 -- 172.21.15.64:0/1956541900 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a8004220 con 0x7f11b8101800 2023-09-13T03:12:38.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.977+0000 7f11ae7fc700 1 -- 172.21.15.64:0/1956541900 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11a8007640 con 0x7f11b8101800 2023-09-13T03:12:38.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.977+0000 7f11bf3ff700 1 -- 172.21.15.64:0/1956541900 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11b819e5d0 con 0x7f11b8101800 2023-09-13T03:12:38.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.977+0000 7f11ae7fc700 1 -- 172.21.15.64:0/1956541900 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a8005690 con 0x7f11b8101800 2023-09-13T03:12:38.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.977+0000 7f11bf3ff700 1 -- 172.21.15.64:0/1956541900 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11b819eae0 con 0x7f11b8101800 2023-09-13T03:12:38.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.978+0000 7f11bf3ff700 1 -- 172.21.15.64:0/1956541900 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11b810be80 con 0x7f11b8101800 2023-09-13T03:12:38.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.980+0000 7f11ae7fc700 1 -- 172.21.15.64:0/1956541900 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f11a8004750 con 0x7f11b8101800 2023-09-13T03:12:38.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.982+0000 7f11ae7fc700 1 --2- 172.21.15.64:0/1956541900 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f11a40a0640 0x7f11a40a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:38.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.982+0000 7f11ae7fc700 1 -- 172.21.15.64:0/1956541900 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f11a80cb9f0 con 0x7f11b8101800 2023-09-13T03:12:38.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.982+0000 7f11bd19b700 1 --2- 172.21.15.64:0/1956541900 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f11a40a0640 0x7f11a40a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:38.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.983+0000 7f11bd19b700 1 --2- 172.21.15.64:0/1956541900 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f11a40a0640 0x7f11a40a2b00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f11b4009fd0 tx=0x7f11b400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:38.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:38.986+0000 7f11ae7fc700 1 -- 172.21.15.64:0/1956541900 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f11a80954f0 con 0x7f11b8101800 2023-09-13T03:12:38.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:38 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/3569000146' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:12:39.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.195+0000 7f11bf3ff700 1 -- 172.21.15.64:0/1956541900 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f11b8107560 con 0x7f11b8101800 2023-09-13T03:12:39.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.195+0000 7f11ae7fc700 1 -- 172.21.15.64:0/1956541900 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f11a80160a0 con 0x7f11b8101800 2023-09-13T03:12:39.198 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-09-13T03:12:39.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.198+0000 7f11bf3ff700 1 -- 172.21.15.64:0/1956541900 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f11a40a0640 msgr2=0x7f11a40a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:39.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.198+0000 7f11bf3ff700 1 --2- 172.21.15.64:0/1956541900 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f11a40a0640 0x7f11a40a2b00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f11b4009fd0 tx=0x7f11b400b040 comp rx=0 tx=0).stop 2023-09-13T03:12:39.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.198+0000 7f11bf3ff700 1 -- 172.21.15.64:0/1956541900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11b8101800 msgr2=0x7f11b81969e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:39.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.198+0000 7f11bf3ff700 1 --2- 172.21.15.64:0/1956541900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11b8101800 0x7f11b81969e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f11a800d980 tx=0x7f11a800dc90 comp rx=0 tx=0).stop 2023-09-13T03:12:39.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.198+0000 7f11bf3ff700 1 -- 172.21.15.64:0/1956541900 shutdown_connections 2023-09-13T03:12:39.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.198+0000 7f11bf3ff700 1 --2- 172.21.15.64:0/1956541900 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f11a40a0640 0x7f11a40a2b00 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:39.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.198+0000 7f11bf3ff700 1 --2- 172.21.15.64:0/1956541900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11b81021a0 0x7f11b819da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:39.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.198+0000 7f11bf3ff700 1 --2- 172.21.15.64:0/1956541900 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11b8100610 0x7f11b81964a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:39.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.198+0000 7f11bf3ff700 1 --2- 172.21.15.64:0/1956541900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11b8101800 0x7f11b81969e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:39.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.198+0000 7f11bf3ff700 1 -- 172.21.15.64:0/1956541900 >> 172.21.15.64:0/1956541900 conn(0x7f11b80fbb80 msgr2=0x7f11b80fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:39.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.198+0000 7f11bf3ff700 1 -- 172.21.15.64:0/1956541900 shutdown_connections 2023-09-13T03:12:39.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.198+0000 7f11bf3ff700 1 -- 172.21.15.64:0/1956541900 wait complete. 2023-09-13T03:12:39.218 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-13T03:12:39.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.424+0000 7ff8ab9ff700 1 -- 172.21.15.64:0/3345058277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8a4073060 msgr2=0x7ff8a40734e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:39.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.424+0000 7ff8ab9ff700 1 --2- 172.21.15.64:0/3345058277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8a4073060 0x7ff8a40734e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff8940098f0 tx=0x7ff894009c00 comp rx=0 tx=0).stop 2023-09-13T03:12:39.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.424+0000 7ff8ab9ff700 1 -- 172.21.15.64:0/3345058277 shutdown_connections 2023-09-13T03:12:39.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.424+0000 7ff8ab9ff700 1 --2- 172.21.15.64:0/3345058277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8a4074640 0x7ff8a4072a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:39.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.424+0000 7ff8ab9ff700 1 --2- 172.21.15.64:0/3345058277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8a41084d0 0x7ff8a410a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:39.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.424+0000 7ff8ab9ff700 1 --2- 172.21.15.64:0/3345058277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8a4073060 0x7ff8a40734e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:39.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.424+0000 7ff8ab9ff700 1 -- 172.21.15.64:0/3345058277 >> 172.21.15.64:0/3345058277 conn(0x7ff8a40fbb80 msgr2=0x7ff8a40fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:39.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.424+0000 7ff8ab9ff700 1 -- 172.21.15.64:0/3345058277 shutdown_connections 2023-09-13T03:12:39.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.424+0000 7ff8ab9ff700 1 -- 172.21.15.64:0/3345058277 wait complete. 2023-09-13T03:12:39.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.425+0000 7ff8ab9ff700 1 Processor -- start 2023-09-13T03:12:39.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.426+0000 7ff8ab9ff700 1 -- start start 2023-09-13T03:12:39.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.426+0000 7ff8ab9ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8a4073060 0x7ff8a41985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:39.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.426+0000 7ff8ab9ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8a4074640 0x7ff8a4198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:39.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.426+0000 7ff8ab9ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8a41084d0 0x7ff8a419fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:39.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.426+0000 7ff8ab9ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a41a0150 con 0x7ff8a4074640 2023-09-13T03:12:39.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.426+0000 7ff8ab9ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a41a0290 con 0x7ff8a41084d0 2023-09-13T03:12:39.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.426+0000 7ff8ab9ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a41a0420 con 0x7ff8a4073060 2023-09-13T03:12:39.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.426+0000 7ff8a979b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8a4073060 0x7ff8a41985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:39.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.426+0000 7ff8a8f9a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8a4074640 0x7ff8a4198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:39.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.426+0000 7ff8a979b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8a4073060 0x7ff8a41985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:50602/0 (socket says 172.21.15.64:50602) 2023-09-13T03:12:39.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.426+0000 7ff8a979b700 1 -- 172.21.15.64:0/3527321489 learned_addr learned my addr 172.21.15.64:0/3527321489 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:39.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.426+0000 7ff8a9f9c700 1 --2- 172.21.15.64:0/3527321489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8a41084d0 0x7ff8a419fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:39.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.427+0000 7ff8a979b700 1 -- 172.21.15.64:0/3527321489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8a41084d0 msgr2=0x7ff8a419fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:39.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.427+0000 7ff8a979b700 1 --2- 172.21.15.64:0/3527321489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8a41084d0 0x7ff8a419fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:39.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.427+0000 7ff8a979b700 1 -- 172.21.15.64:0/3527321489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8a4074640 msgr2=0x7ff8a4198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:39.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.427+0000 7ff8a979b700 1 --2- 172.21.15.64:0/3527321489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8a4074640 0x7ff8a4198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:39.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.427+0000 7ff8a979b700 1 -- 172.21.15.64:0/3527321489 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff89c009d00 con 0x7ff8a4073060 2023-09-13T03:12:39.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.427+0000 7ff8a8f9a700 1 --2- 172.21.15.64:0/3527321489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8a4074640 0x7ff8a4198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:39.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.427+0000 7ff8a979b700 1 --2- 172.21.15.64:0/3527321489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8a4073060 0x7ff8a41985d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff8a000e9a0 tx=0x7ff8a000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:39.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.427+0000 7ff89a7fc700 1 -- 172.21.15.64:0/3527321489 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a0009790 con 0x7ff8a4073060 2023-09-13T03:12:39.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.427+0000 7ff8ab9ff700 1 -- 172.21.15.64:0/3527321489 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff894009520 con 0x7ff8a4073060 2023-09-13T03:12:39.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.427+0000 7ff89a7fc700 1 -- 172.21.15.64:0/3527321489 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8a0010430 con 0x7ff8a4073060 2023-09-13T03:12:39.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.427+0000 7ff89a7fc700 1 -- 172.21.15.64:0/3527321489 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a0018680 con 0x7ff8a4073060 2023-09-13T03:12:39.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.427+0000 7ff8ab9ff700 1 -- 172.21.15.64:0/3527321489 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8a41a0a80 con 0x7ff8a4073060 2023-09-13T03:12:39.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.428+0000 7ff8ab9ff700 1 -- 172.21.15.64:0/3527321489 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8a4106b80 con 0x7ff8a4073060 2023-09-13T03:12:39.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.430+0000 7ff89a7fc700 1 -- 172.21.15.64:0/3527321489 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7ff8a00105a0 con 0x7ff8a4073060 2023-09-13T03:12:39.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.432+0000 7ff89a7fc700 1 --2- 172.21.15.64:0/3527321489 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ff8900a0640 0x7ff8900a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:39.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.432+0000 7ff89a7fc700 1 -- 172.21.15.64:0/3527321489 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff8a0014070 con 0x7ff8a4073060 2023-09-13T03:12:39.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.432+0000 7ff8a8f9a700 1 --2- 172.21.15.64:0/3527321489 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ff8900a0640 0x7ff8900a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:39.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.432+0000 7ff8a8f9a700 1 --2- 172.21.15.64:0/3527321489 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ff8900a0640 0x7ff8900a2b00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff894009fd0 tx=0x7ff894019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:39.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.434+0000 7ff89a7fc700 1 -- 172.21.15.64:0/3527321489 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ff8a0095b20 con 0x7ff8a4073060 2023-09-13T03:12:39.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.645+0000 7ff8ab9ff700 1 -- 172.21.15.64:0/3527321489 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff8a410e1e0 con 0x7ff8a4073060 2023-09-13T03:12:39.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.646+0000 7ff89a7fc700 1 -- 172.21.15.64:0/3527321489 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff8a0099140 con 0x7ff8a4073060 2023-09-13T03:12:39.647 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-09-13T03:12:39.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.648+0000 7ff8ab9ff700 1 -- 172.21.15.64:0/3527321489 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ff8900a0640 msgr2=0x7ff8900a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:39.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.648+0000 7ff8ab9ff700 1 --2- 172.21.15.64:0/3527321489 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ff8900a0640 0x7ff8900a2b00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff894009fd0 tx=0x7ff894019040 comp rx=0 tx=0).stop 2023-09-13T03:12:39.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.648+0000 7ff8ab9ff700 1 -- 172.21.15.64:0/3527321489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8a4073060 msgr2=0x7ff8a41985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:39.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.648+0000 7ff8ab9ff700 1 --2- 172.21.15.64:0/3527321489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8a4073060 0x7ff8a41985d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff8a000e9a0 tx=0x7ff8a000ecb0 comp rx=0 tx=0).stop 2023-09-13T03:12:39.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.648+0000 7ff8ab9ff700 1 -- 172.21.15.64:0/3527321489 shutdown_connections 2023-09-13T03:12:39.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.648+0000 7ff8ab9ff700 1 --2- 172.21.15.64:0/3527321489 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ff8900a0640 0x7ff8900a2b00 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:39.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.648+0000 7ff8ab9ff700 1 --2- 172.21.15.64:0/3527321489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8a41084d0 0x7ff8a419fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:39.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.648+0000 7ff8ab9ff700 1 --2- 172.21.15.64:0/3527321489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8a4073060 0x7ff8a41985d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:39.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.648+0000 7ff8ab9ff700 1 --2- 172.21.15.64:0/3527321489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8a4074640 0x7ff8a4198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:39.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.648+0000 7ff8ab9ff700 1 -- 172.21.15.64:0/3527321489 >> 172.21.15.64:0/3527321489 conn(0x7ff8a40fbb80 msgr2=0x7ff8a40fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:39.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.649+0000 7ff8ab9ff700 1 -- 172.21.15.64:0/3527321489 shutdown_connections 2023-09-13T03:12:39.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.649+0000 7ff8ab9ff700 1 -- 172.21.15.64:0/3527321489 wait complete. 2023-09-13T03:12:39.668 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T03:12:39.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.871+0000 7f85297ff700 1 -- 172.21.15.64:0/256681179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8524100610 msgr2=0x7f8524100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:39.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.871+0000 7f85297ff700 1 --2- 172.21.15.64:0/256681179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8524100610 0x7f8524100a30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f85140099c0 tx=0x7f8514009cd0 comp rx=0 tx=0).stop 2023-09-13T03:12:39.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.871+0000 7f85297ff700 1 -- 172.21.15.64:0/256681179 shutdown_connections 2023-09-13T03:12:39.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.871+0000 7f85297ff700 1 --2- 172.21.15.64:0/256681179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8524100610 0x7f8524100a30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:39.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.871+0000 7f85297ff700 1 --2- 172.21.15.64:0/256681179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85241021a0 0x7f8524108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:39.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.871+0000 7f85297ff700 1 --2- 172.21.15.64:0/256681179 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8524101800 0x7f8524101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:39.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.871+0000 7f85297ff700 1 -- 172.21.15.64:0/256681179 >> 172.21.15.64:0/256681179 conn(0x7f85240fbb40 msgr2=0x7f85240fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:39.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.871+0000 7f85297ff700 1 -- 172.21.15.64:0/256681179 shutdown_connections 2023-09-13T03:12:39.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.871+0000 7f85297ff700 1 -- 172.21.15.64:0/256681179 wait complete. 2023-09-13T03:12:39.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.872+0000 7f85297ff700 1 Processor -- start 2023-09-13T03:12:39.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.872+0000 7f85297ff700 1 -- start start 2023-09-13T03:12:39.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.873+0000 7f85297ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8524100610 0x7f85241984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:39.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.873+0000 7f85297ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8524101800 0x7f85241989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:39.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.873+0000 7f85297ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85241021a0 0x7f852419fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:39.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.873+0000 7f85297ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85241a0030 con 0x7f85241021a0 2023-09-13T03:12:39.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.873+0000 7f85297ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85241a0170 con 0x7f8524100610 2023-09-13T03:12:39.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.873+0000 7f85297ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85241a0300 con 0x7f8524101800 2023-09-13T03:12:39.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.873+0000 7f85237fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85241021a0 0x7f852419fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:39.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.873+0000 7f8522ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8524100610 0x7f85241984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:39.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.873+0000 7f85237fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85241021a0 0x7f852419fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54074/0 (socket says 172.21.15.64:54074) 2023-09-13T03:12:39.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.873+0000 7f85237fe700 1 -- 172.21.15.64:0/818666910 learned_addr learned my addr 172.21.15.64:0/818666910 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:39.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.873+0000 7f85227fc700 1 --2- 172.21.15.64:0/818666910 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8524101800 0x7f85241989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:39.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.873+0000 7f85237fe700 1 -- 172.21.15.64:0/818666910 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8524101800 msgr2=0x7f85241989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:39.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.873+0000 7f85237fe700 1 --2- 172.21.15.64:0/818666910 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8524101800 0x7f85241989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:39.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.873+0000 7f85237fe700 1 -- 172.21.15.64:0/818666910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8524100610 msgr2=0x7f85241984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:39.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.873+0000 7f85237fe700 1 --2- 172.21.15.64:0/818666910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8524100610 0x7f85241984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:39.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.873+0000 7f85237fe700 1 -- 172.21.15.64:0/818666910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f851c009c40 con 0x7f85241021a0 2023-09-13T03:12:39.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.874+0000 7f85237fe700 1 --2- 172.21.15.64:0/818666910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85241021a0 0x7f852419fa60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f851c00c370 tx=0x7f851c00c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:39.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.874+0000 7f850bfff700 1 -- 172.21.15.64:0/818666910 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c017070 con 0x7f85241021a0 2023-09-13T03:12:39.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.874+0000 7f850bfff700 1 -- 172.21.15.64:0/818666910 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f851c00f040 con 0x7f85241021a0 2023-09-13T03:12:39.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.874+0000 7f85297ff700 1 -- 172.21.15.64:0/818666910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85140095f0 con 0x7f85241021a0 2023-09-13T03:12:39.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.874+0000 7f850bfff700 1 -- 172.21.15.64:0/818666910 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c00e050 con 0x7f85241021a0 2023-09-13T03:12:39.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.874+0000 7f85297ff700 1 -- 172.21.15.64:0/818666910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85241a0900 con 0x7f85241021a0 2023-09-13T03:12:39.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.875+0000 7f85297ff700 1 -- 172.21.15.64:0/818666910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8524192c30 con 0x7f85241021a0 2023-09-13T03:12:39.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.877+0000 7f850bfff700 1 -- 172.21.15.64:0/818666910 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f851c00cb60 con 0x7f85241021a0 2023-09-13T03:12:39.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.878+0000 7f850bfff700 1 --2- 172.21.15.64:0/818666910 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f850c0a0690 0x7f850c0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:39.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.879+0000 7f850bfff700 1 -- 172.21.15.64:0/818666910 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f851c0cb530 con 0x7f85241021a0 2023-09-13T03:12:39.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.879+0000 7f8522ffd700 1 --2- 172.21.15.64:0/818666910 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f850c0a0690 0x7f850c0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:39.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.879+0000 7f8522ffd700 1 --2- 172.21.15.64:0/818666910 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f850c0a0690 0x7f850c0a2b50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8514009fd0 tx=0x7f8514005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:39.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:39.883+0000 7f850bfff700 1 -- 172.21.15.64:0/818666910 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f851c095030 con 0x7f85241021a0 2023-09-13T03:12:39.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:39 smithi170 ceph-mon[134328]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 170 B/s wr, 11 op/s 2023-09-13T03:12:39.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:39 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/510220343' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T03:12:39.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:39 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/1956541900' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T03:12:39.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:39 smithi064 ceph-mon[97675]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 170 B/s wr, 11 op/s 2023-09-13T03:12:39.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:39 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/510220343' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T03:12:39.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:39 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/1956541900' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T03:12:39.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:39 smithi142 ceph-mon[134082]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 170 B/s wr, 11 op/s 2023-09-13T03:12:39.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:39 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/510220343' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T03:12:39.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:39 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/1956541900' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T03:12:40.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.140+0000 7f85297ff700 1 -- 172.21.15.64:0/818666910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f852410c060 con 0x7f85241021a0 2023-09-13T03:12:40.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.141+0000 7f850bfff700 1 -- 172.21.15.64:0/818666910 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11366 (secure 0 0 0) 0x7f851c098650 con 0x7f85241021a0 2023-09-13T03:12:40.142 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:12:40.142 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2234182985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2234182985},{"type":"v1","addr":"172.21.15.64:6835","nonce":2234182985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4053841873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4053841873},{"type":"v1","addr":"172.21.15.64:6837","nonce":4053841873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-13T03:11:10.490260+0000","modified":"2023-09-13T03:11:18.230740+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24686,"mds_1":24649,"mds_2":24635,"mds_3":24634,"mds_4":24622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6839/99523374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":99523374},{"type":"v1","addr":"172.21.15.64:6839","nonce":99523374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14649":{"gid":14649,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6841/1866582875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":1866582875},{"type":"v1","addr":"172.21.15.64:6841","nonce":1866582875}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/101906999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":101906999},{"type":"v1","addr":"172.21.15.142:6833","nonce":101906999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24574":{"gid":24574,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/1897366227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1897366227},{"type":"v1","addr":"172.21.15.170:6833","nonce":1897366227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24611":{"gid":24611,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2167779918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2167779918},{"type":"v1","addr":"172.21.15.142:6835","nonce":2167779918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24622":{"gid":24622,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6835/1493371121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1493371121},{"type":"v1","addr":"172.21.15.170:6835","nonce":1493371121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24634":{"gid":24634,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.170:6837/1726587275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":1726587275},{"type":"v1","addr":"172.21.15.170:6837","nonce":1726587275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24635":{"gid":24635,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.142:6837/741173268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":741173268},{"type":"v1","addr":"172.21.15.142:6837","nonce":741173268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24649":{"gid":24649,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.170:6839/2189703596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2189703596},{"type":"v1","addr":"172.21.15.170:6839","nonce":2189703596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24686":{"gid":24686,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/3645998616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645998616},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645998616}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-13T03:12:40.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.143+0000 7f85297ff700 1 -- 172.21.15.64:0/818666910 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f850c0a0690 msgr2=0x7f850c0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:40.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.143+0000 7f85297ff700 1 --2- 172.21.15.64:0/818666910 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f850c0a0690 0x7f850c0a2b50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8514009fd0 tx=0x7f8514005fb0 comp rx=0 tx=0).stop 2023-09-13T03:12:40.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.143+0000 7f85297ff700 1 -- 172.21.15.64:0/818666910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85241021a0 msgr2=0x7f852419fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:40.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.143+0000 7f85297ff700 1 --2- 172.21.15.64:0/818666910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85241021a0 0x7f852419fa60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f851c00c370 tx=0x7f851c00c680 comp rx=0 tx=0).stop 2023-09-13T03:12:40.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.143+0000 7f85297ff700 1 -- 172.21.15.64:0/818666910 shutdown_connections 2023-09-13T03:12:40.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.143+0000 7f85297ff700 1 --2- 172.21.15.64:0/818666910 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f850c0a0690 0x7f850c0a2b50 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:40.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.143+0000 7f85297ff700 1 --2- 172.21.15.64:0/818666910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8524100610 0x7f85241984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:40.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.143+0000 7f85297ff700 1 --2- 172.21.15.64:0/818666910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85241021a0 0x7f852419fa60 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:40.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.143+0000 7f85297ff700 1 --2- 172.21.15.64:0/818666910 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8524101800 0x7f85241989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:40.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.143+0000 7f85297ff700 1 -- 172.21.15.64:0/818666910 >> 172.21.15.64:0/818666910 conn(0x7f85240fbb40 msgr2=0x7f85241070e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:40.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.143+0000 7f85297ff700 1 -- 172.21.15.64:0/818666910 shutdown_connections 2023-09-13T03:12:40.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.143+0000 7f85297ff700 1 -- 172.21.15.64:0/818666910 wait complete. 2023-09-13T03:12:40.147 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 26 2023-09-13T03:12:40.163 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-13T03:12:40.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.365+0000 7f5871bff700 1 -- 172.21.15.64:0/531325929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f586c100610 msgr2=0x7f586c100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:40.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.365+0000 7f5871bff700 1 --2- 172.21.15.64:0/531325929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f586c100610 0x7f586c100a30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f585c009920 tx=0x7f585c009c30 comp rx=0 tx=0).stop 2023-09-13T03:12:40.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.365+0000 7f5871bff700 1 -- 172.21.15.64:0/531325929 shutdown_connections 2023-09-13T03:12:40.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.365+0000 7f5871bff700 1 --2- 172.21.15.64:0/531325929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f586c1021a0 0x7f586c108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:40.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.365+0000 7f5871bff700 1 --2- 172.21.15.64:0/531325929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f586c100610 0x7f586c100a30 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:40.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.365+0000 7f5871bff700 1 --2- 172.21.15.64:0/531325929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f586c101800 0x7f586c101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:40.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.365+0000 7f5871bff700 1 -- 172.21.15.64:0/531325929 >> 172.21.15.64:0/531325929 conn(0x7f586c0fbb40 msgr2=0x7f586c0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:40.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.365+0000 7f5871bff700 1 -- 172.21.15.64:0/531325929 shutdown_connections 2023-09-13T03:12:40.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.365+0000 7f5871bff700 1 -- 172.21.15.64:0/531325929 wait complete. 2023-09-13T03:12:40.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.366+0000 7f5871bff700 1 Processor -- start 2023-09-13T03:12:40.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.367+0000 7f5871bff700 1 -- start start 2023-09-13T03:12:40.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.367+0000 7f5871bff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f586c100610 0x7f586c198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:40.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.367+0000 7f5871bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f586c101800 0x7f586c198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:40.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.367+0000 7f5871bff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f586c1021a0 0x7f586c19fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:40.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.367+0000 7f5871bff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586c1a0100 con 0x7f586c1021a0 2023-09-13T03:12:40.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.367+0000 7f5871bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586c1a0240 con 0x7f586c101800 2023-09-13T03:12:40.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.367+0000 7f5871bff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586c1a03d0 con 0x7f586c100610 2023-09-13T03:12:40.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.367+0000 7f586bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f586c1021a0 0x7f586c19fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:40.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.367+0000 7f586affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f586c101800 0x7f586c198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:40.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.367+0000 7f586affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f586c101800 0x7f586c198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:53062/0 (socket says 172.21.15.64:53062) 2023-09-13T03:12:40.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.367+0000 7f586affd700 1 -- 172.21.15.64:0/3185540942 learned_addr learned my addr 172.21.15.64:0/3185540942 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:40.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.367+0000 7f586b7fe700 1 --2- 172.21.15.64:0/3185540942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f586c100610 0x7f586c198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:40.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.368+0000 7f586affd700 1 -- 172.21.15.64:0/3185540942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f586c100610 msgr2=0x7f586c198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:40.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.368+0000 7f586affd700 1 --2- 172.21.15.64:0/3185540942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f586c100610 0x7f586c198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:40.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.368+0000 7f586affd700 1 -- 172.21.15.64:0/3185540942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f586c1021a0 msgr2=0x7f586c19fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:40.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.368+0000 7f586affd700 1 --2- 172.21.15.64:0/3185540942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f586c1021a0 0x7f586c19fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:40.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.368+0000 7f586affd700 1 -- 172.21.15.64:0/3185540942 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58600095f0 con 0x7f586c101800 2023-09-13T03:12:40.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.368+0000 7f586bfff700 1 --2- 172.21.15.64:0/3185540942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f586c1021a0 0x7f586c19fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:40.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.368+0000 7f586affd700 1 --2- 172.21.15.64:0/3185540942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f586c101800 0x7f586c198ac0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f586000ec10 tx=0x7f586000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:40.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.368+0000 7f5868ff9700 1 -- 172.21.15.64:0/3185540942 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586000cc90 con 0x7f586c101800 2023-09-13T03:12:40.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.368+0000 7f5868ff9700 1 -- 172.21.15.64:0/3185540942 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58600044e0 con 0x7f586c101800 2023-09-13T03:12:40.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.368+0000 7f5871bff700 1 -- 172.21.15.64:0/3185540942 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f585c009520 con 0x7f586c101800 2023-09-13T03:12:40.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.368+0000 7f5868ff9700 1 -- 172.21.15.64:0/3185540942 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5860010430 con 0x7f586c101800 2023-09-13T03:12:40.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.368+0000 7f5871bff700 1 -- 172.21.15.64:0/3185540942 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f586c1a0a30 con 0x7f586c101800 2023-09-13T03:12:40.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.369+0000 7f5871bff700 1 -- 172.21.15.64:0/3185540942 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f586c192c70 con 0x7f586c101800 2023-09-13T03:12:40.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.371+0000 7f5868ff9700 1 -- 172.21.15.64:0/3185540942 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f5860003fb0 con 0x7f586c101800 2023-09-13T03:12:40.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.373+0000 7f5868ff9700 1 --2- 172.21.15.64:0/3185540942 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f58540a0690 0x7f58540a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:40.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.373+0000 7f5868ff9700 1 -- 172.21.15.64:0/3185540942 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f5860014070 con 0x7f586c101800 2023-09-13T03:12:40.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.373+0000 7f586b7fe700 1 --2- 172.21.15.64:0/3185540942 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f58540a0690 0x7f58540a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:40.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.374+0000 7f586b7fe700 1 --2- 172.21.15.64:0/3185540942 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f58540a0690 0x7f58540a2b50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f585c0098f0 tx=0x7f585c019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:40.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.375+0000 7f5868ff9700 1 -- 172.21.15.64:0/3185540942 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5860095e70 con 0x7f586c101800 2023-09-13T03:12:40.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.589+0000 7f5871bff700 1 -- 172.21.15.64:0/3185540942 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f586c10be80 con 0x7f586c101800 2023-09-13T03:12:40.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.591+0000 7f5868ff9700 1 -- 172.21.15.64:0/3185540942 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22473 (secure 0 0 0) 0x7f5860017e90 con 0x7f586c101800 2023-09-13T03:12:40.592 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:12:40.592 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":85,"fsid":"f8dbced4-51e1-11ee-9ab7-7b867c8bd7da","created":"2023-09-13T03:03:24.167202+0000","modified":"2023-09-13T03:11:24.258871+0000","last_up_change":"2023-09-13T03:10:22.503529+0000","last_in_change":"2023-09-13T03:10:05.678720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T03:06:58.987479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-13T03:11:08.589202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-13T03:11:09.924659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3997c114-241b-48c4-8feb-653c99209470","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6803","nonce":3940892135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6805","nonce":3940892135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6809","nonce":3940892135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6807","nonce":3940892135}]},"public_addr":"172.21.15.64:6803/3940892135","cluster_addr":"172.21.15.64:6805/3940892135","heartbeat_back_addr":"172.21.15.64:6809/3940892135","heartbeat_front_addr":"172.21.15.64:6807/3940892135","state":["exists","up"]},{"osd":1,"uuid":"94cfae85-322e-4ed2-a6e6-e40349bb14bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6801","nonce":4188149333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6803","nonce":4188149333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6807","nonce":4188149333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6805","nonce":4188149333}]},"public_addr":"172.21.15.142:6801/4188149333","cluster_addr":"172.21.15.142:6803/4188149333","heartbeat_back_addr":"172.21.15.142:6807/4188149333","heartbeat_front_addr":"172.21.15.142:6805/4188149333","state":["exists","up"]},{"osd":2,"uuid":"b3aae17e-96db-4008-8c27-f733c46b30ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6801","nonce":2406669338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6803","nonce":2406669338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6807","nonce":2406669338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6805","nonce":2406669338}]},"public_addr":"172.21.15.170:6801/2406669338","cluster_addr":"172.21.15.170:6803/2406669338","heartbeat_back_addr":"172.21.15.170:6807/2406669338","heartbeat_front_addr":"172.21.15.170:6805/2406669338","state":["exists","up"]},{"osd":3,"uuid":"7fa8d8cb-2ab7-48a7-8b49-5e2903d6779f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6811","nonce":3558318707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6813","nonce":3558318707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6817","nonce":3558318707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6815","nonce":3558318707}]},"public_addr":"172.21.15.64:6811/3558318707","cluster_addr":"172.21.15.64:6813/3558318707","heartbeat_back_addr":"172.21.15.64:6817/3558318707","heartbeat_front_addr":"172.21.15.64:6815/3558318707","state":["exists","up"]},{"osd":4,"uuid":"8c7a9c9f-f919-48d3-9db2-214a25bb5603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6809","nonce":1811142576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6811","nonce":1811142576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6815","nonce":1811142576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6813","nonce":1811142576}]},"public_addr":"172.21.15.142:6809/1811142576","cluster_addr":"172.21.15.142:6811/1811142576","heartbeat_back_addr":"172.21.15.142:6815/1811142576","heartbeat_front_addr":"172.21.15.142:6813/1811142576","state":["exists","up"]},{"osd":5,"uuid":"0c6101c6-2065-4022-a751-d3f216d7f478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6809","nonce":2903841453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6811","nonce":2903841453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6815","nonce":2903841453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6813","nonce":2903841453}]},"public_addr":"172.21.15.170:6809/2903841453","cluster_addr":"172.21.15.170:6811/2903841453","heartbeat_back_addr":"172.21.15.170:6815/2903841453","heartbeat_front_addr":"172.21.15.170:6813/2903841453","state":["exists","up"]},{"osd":6,"uuid":"5cf3e52e-d155-41b1-9194-db047d26a953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6819","nonce":1436584490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6821","nonce":1436584490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6825","nonce":1436584490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6823","nonce":1436584490}]},"public_addr":"172.21.15.64:6819/1436584490","cluster_addr":"172.21.15.64:6821/1436584490","heartbeat_back_addr":"172.21.15.64:6825/1436584490","heartbeat_front_addr":"172.21.15.64:6823/1436584490","state":["exists","up"]},{"osd":7,"uuid":"e97323e3-badd-41a7-af9e-e78a81a64046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6817","nonce":3726793534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6819","nonce":3726793534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6823","nonce":3726793534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6821","nonce":3726793534}]},"public_addr":"172.21.15.142:6817/3726793534","cluster_addr":"172.21.15.142:6819/3726793534","heartbeat_back_addr":"172.21.15.142:6823/3726793534","heartbeat_front_addr":"172.21.15.142:6821/3726793534","state":["exists","up"]},{"osd":8,"uuid":"83d2835c-ed26-493d-9222-c2894ec0e071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6817","nonce":4272513873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6819","nonce":4272513873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6823","nonce":4272513873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6821","nonce":4272513873}]},"public_addr":"172.21.15.170:6817/4272513873","cluster_addr":"172.21.15.170:6819/4272513873","heartbeat_back_addr":"172.21.15.170:6823/4272513873","heartbeat_front_addr":"172.21.15.170:6821/4272513873","state":["exists","up"]},{"osd":9,"uuid":"cb936bdd-1bec-4236-aa2e-a24adcac11e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6827","nonce":2641175211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6829","nonce":2641175211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6833","nonce":2641175211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6831","nonce":2641175211}]},"public_addr":"172.21.15.64:6827/2641175211","cluster_addr":"172.21.15.64:6829/2641175211","heartbeat_back_addr":"172.21.15.64:6833/2641175211","heartbeat_front_addr":"172.21.15.64:6831/2641175211","state":["exists","up"]},{"osd":10,"uuid":"85512a1c-6171-43d4-9c15-c588a8b71be0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6825","nonce":936499217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6827","nonce":936499217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6831","nonce":936499217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6829","nonce":936499217}]},"public_addr":"172.21.15.142:6825/936499217","cluster_addr":"172.21.15.142:6827/936499217","heartbeat_back_addr":"172.21.15.142:6831/936499217","heartbeat_front_addr":"172.21.15.142:6829/936499217","state":["exists","up"]},{"osd":11,"uuid":"9b8417dc-5d6c-4917-94a9-b032c5bc33d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6825","nonce":3605017060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6827","nonce":3605017060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6831","nonce":3605017060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6829","nonce":3605017060}]},"public_addr":"172.21.15.170:6825/3605017060","cluster_addr":"172.21.15.170:6827/3605017060","heartbeat_back_addr":"172.21.15.170:6831/3605017060","heartbeat_front_addr":"172.21.15.170:6829/3605017060","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:14.229044+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:36.147777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:56.655298+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:07:23.107741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:07:44.663770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:05.577432+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:31.168031+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:52.791055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:09:13.658546+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:09:39.150360+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:10:00.601735+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:10:21.407555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6800/2955227347":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/1373223118":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/271194613":"2023-09-14T03:04:22.355817+0000","172.21.15.64:6801/3998091347":"2023-09-14T03:03:56.442541+0000","172.21.15.64:6800/3998091347":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/920818572":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/735789005":"2023-09-14T03:03:56.442541+0000","172.21.15.64:6801/2955227347":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/1975923178":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/1500151178":"2023-09-14T03:04:22.355817+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T03:12:40.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.594+0000 7f5871bff700 1 -- 172.21.15.64:0/3185540942 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f58540a0690 msgr2=0x7f58540a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:40.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.594+0000 7f5871bff700 1 --2- 172.21.15.64:0/3185540942 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f58540a0690 0x7f58540a2b50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f585c0098f0 tx=0x7f585c019040 comp rx=0 tx=0).stop 2023-09-13T03:12:40.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.594+0000 7f5871bff700 1 -- 172.21.15.64:0/3185540942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f586c101800 msgr2=0x7f586c198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:40.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.594+0000 7f5871bff700 1 --2- 172.21.15.64:0/3185540942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f586c101800 0x7f586c198ac0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f586000ec10 tx=0x7f586000ef20 comp rx=0 tx=0).stop 2023-09-13T03:12:40.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.594+0000 7f5871bff700 1 -- 172.21.15.64:0/3185540942 shutdown_connections 2023-09-13T03:12:40.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.594+0000 7f5871bff700 1 --2- 172.21.15.64:0/3185540942 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f58540a0690 0x7f58540a2b50 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:40.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.594+0000 7f5871bff700 1 --2- 172.21.15.64:0/3185540942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f586c1021a0 0x7f586c19fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:40.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.594+0000 7f5871bff700 1 --2- 172.21.15.64:0/3185540942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f586c100610 0x7f586c198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:40.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.594+0000 7f5871bff700 1 --2- 172.21.15.64:0/3185540942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f586c101800 0x7f586c198ac0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:40.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.594+0000 7f5871bff700 1 -- 172.21.15.64:0/3185540942 >> 172.21.15.64:0/3185540942 conn(0x7f586c0fbb40 msgr2=0x7f586c107190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:40.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.594+0000 7f5871bff700 1 -- 172.21.15.64:0/3185540942 shutdown_connections 2023-09-13T03:12:40.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.594+0000 7f5871bff700 1 -- 172.21.15.64:0/3185540942 wait complete. 2023-09-13T03:12:40.614 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-13T03:06:58.987479+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-13T03:11:08.589202+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-13T03:11:09.924659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-13T03:12:40.615 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-13T03:12:40.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.820+0000 7f9ffffff700 1 -- 172.21.15.64:0/3571677369 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ff80fee50 msgr2=0x7f9ff80ff270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:40.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.820+0000 7f9ffffff700 1 --2- 172.21.15.64:0/3571677369 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ff80fee50 0x7f9ff80ff270 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9ff4009990 tx=0x7f9ff4009ca0 comp rx=0 tx=0).stop 2023-09-13T03:12:40.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.820+0000 7f9ffffff700 1 -- 172.21.15.64:0/3571677369 shutdown_connections 2023-09-13T03:12:40.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.820+0000 7f9ffffff700 1 --2- 172.21.15.64:0/3571677369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ff8100170 0x7f9ff81005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:40.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.820+0000 7f9ffffff700 1 --2- 172.21.15.64:0/3571677369 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ff80fee50 0x7f9ff80ff270 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:40.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.820+0000 7f9ffffff700 1 --2- 172.21.15.64:0/3571677369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ff80ff7b0 0x7f9ff80ffc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:40.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.820+0000 7f9ffffff700 1 -- 172.21.15.64:0/3571677369 >> 172.21.15.64:0/3571677369 conn(0x7f9ff80faa10 msgr2=0x7f9ff80fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:40.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.820+0000 7f9ffffff700 1 -- 172.21.15.64:0/3571677369 shutdown_connections 2023-09-13T03:12:40.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.820+0000 7f9ffffff700 1 -- 172.21.15.64:0/3571677369 wait complete. 2023-09-13T03:12:40.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.821+0000 7f9ffffff700 1 Processor -- start 2023-09-13T03:12:40.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.821+0000 7f9ffffff700 1 -- start start 2023-09-13T03:12:40.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.822+0000 7f9ffffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ff80fee50 0x7f9ff8198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:40.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.822+0000 7f9ffffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ff80ff7b0 0x7f9ff8198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:40.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.822+0000 7f9ffffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ff8100170 0x7f9ff819fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:40.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.822+0000 7f9ffffff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff81a0080 con 0x7f9ff80fee50 2023-09-13T03:12:40.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.822+0000 7f9ffffff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff81a01c0 con 0x7f9ff8100170 2023-09-13T03:12:40.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.822+0000 7f9ffffff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff81a0350 con 0x7f9ff80ff7b0 2023-09-13T03:12:40.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.822+0000 7f9ffdd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ff80fee50 0x7f9ff8198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:40.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.822+0000 7f9ffd59a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ff80ff7b0 0x7f9ff8198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:40.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.822+0000 7f9ffdd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ff80fee50 0x7f9ff8198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54114/0 (socket says 172.21.15.64:54114) 2023-09-13T03:12:40.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.822+0000 7f9ffdd9b700 1 -- 172.21.15.64:0/1387965185 learned_addr learned my addr 172.21.15.64:0/1387965185 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:40.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.822+0000 7f9ffe59c700 1 --2- 172.21.15.64:0/1387965185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ff8100170 0x7f9ff819fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:40.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.822+0000 7f9ffd59a700 1 -- 172.21.15.64:0/1387965185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ff8100170 msgr2=0x7f9ff819fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:40.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.822+0000 7f9ffd59a700 1 --2- 172.21.15.64:0/1387965185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ff8100170 0x7f9ff819fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:40.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.822+0000 7f9ffd59a700 1 -- 172.21.15.64:0/1387965185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ff80fee50 msgr2=0x7f9ff8198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:40.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.822+0000 7f9ffd59a700 1 --2- 172.21.15.64:0/1387965185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ff80fee50 0x7f9ff8198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:40.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.822+0000 7f9ffd59a700 1 -- 172.21.15.64:0/1387965185 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff40095f0 con 0x7f9ff80ff7b0 2023-09-13T03:12:40.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.822+0000 7f9ffdd9b700 1 --2- 172.21.15.64:0/1387965185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ff80fee50 0x7f9ff8198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:40.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.823+0000 7f9ffd59a700 1 --2- 172.21.15.64:0/1387965185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ff80ff7b0 0x7f9ff8198a40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9fe800e9f0 tx=0x7f9fe800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:40.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.823+0000 7f9feeffd700 1 -- 172.21.15.64:0/1387965185 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe800c9c0 con 0x7f9ff80ff7b0 2023-09-13T03:12:40.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.823+0000 7f9ffffff700 1 -- 172.21.15.64:0/1387965185 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ff81a0630 con 0x7f9ff80ff7b0 2023-09-13T03:12:40.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.823+0000 7f9feeffd700 1 -- 172.21.15.64:0/1387965185 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fe8010430 con 0x7f9ff80ff7b0 2023-09-13T03:12:40.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.823+0000 7f9feeffd700 1 -- 172.21.15.64:0/1387965185 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe8018680 con 0x7f9ff80ff7b0 2023-09-13T03:12:40.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.823+0000 7f9ffffff700 1 -- 172.21.15.64:0/1387965185 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ff81a0b90 con 0x7f9ff80ff7b0 2023-09-13T03:12:40.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.824+0000 7f9ffffff700 1 -- 172.21.15.64:0/1387965185 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ff8109c30 con 0x7f9ff80ff7b0 2023-09-13T03:12:40.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.826+0000 7f9feeffd700 1 -- 172.21.15.64:0/1387965185 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f9fe8010860 con 0x7f9ff80ff7b0 2023-09-13T03:12:40.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.827+0000 7f9feeffd700 1 --2- 172.21.15.64:0/1387965185 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f9fe40a0640 0x7f9fe40a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:40.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.827+0000 7f9feeffd700 1 -- 172.21.15.64:0/1387965185 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f9fe8014070 con 0x7f9ff80ff7b0 2023-09-13T03:12:40.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.828+0000 7f9ffdd9b700 1 --2- 172.21.15.64:0/1387965185 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f9fe40a0640 0x7f9fe40a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:40.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.828+0000 7f9ffdd9b700 1 --2- 172.21.15.64:0/1387965185 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f9fe40a0640 0x7f9fe40a2b00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9ff400b2e0 tx=0x7f9ff4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:40.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:40.829+0000 7f9feeffd700 1 -- 172.21.15.64:0/1387965185 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f9fe8095a30 con 0x7f9ff80ff7b0 2023-09-13T03:12:40.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:40 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/3527321489' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T03:12:40.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:40 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/818666910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:12:40.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:40 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/3527321489' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T03:12:40.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:40 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/818666910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:12:40.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:40 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/3527321489' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T03:12:40.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:40 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/818666910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:12:41.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.039+0000 7f9ffffff700 1 -- 172.21.15.64:0/1387965185 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9ff810e170 con 0x7f9ff80ff7b0 2023-09-13T03:12:41.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.040+0000 7f9feeffd700 1 -- 172.21.15.64:0/1387965185 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9fe8099050 con 0x7f9ff80ff7b0 2023-09-13T03:12:41.041 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-09-13T03:12:41.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.043+0000 7f9ffffff700 1 -- 172.21.15.64:0/1387965185 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f9fe40a0640 msgr2=0x7f9fe40a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:41.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.043+0000 7f9ffffff700 1 --2- 172.21.15.64:0/1387965185 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f9fe40a0640 0x7f9fe40a2b00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9ff400b2e0 tx=0x7f9ff4005fb0 comp rx=0 tx=0).stop 2023-09-13T03:12:41.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.043+0000 7f9ffffff700 1 -- 172.21.15.64:0/1387965185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ff80ff7b0 msgr2=0x7f9ff8198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:41.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.043+0000 7f9ffffff700 1 --2- 172.21.15.64:0/1387965185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ff80ff7b0 0x7f9ff8198a40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9fe800e9f0 tx=0x7f9fe800ed00 comp rx=0 tx=0).stop 2023-09-13T03:12:41.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.043+0000 7f9ffffff700 1 -- 172.21.15.64:0/1387965185 shutdown_connections 2023-09-13T03:12:41.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.043+0000 7f9ffffff700 1 --2- 172.21.15.64:0/1387965185 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f9fe40a0640 0x7f9fe40a2b00 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:41.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.043+0000 7f9ffffff700 1 --2- 172.21.15.64:0/1387965185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ff8100170 0x7f9ff819fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:41.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.043+0000 7f9ffffff700 1 --2- 172.21.15.64:0/1387965185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ff80fee50 0x7f9ff8198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:41.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.043+0000 7f9ffffff700 1 --2- 172.21.15.64:0/1387965185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ff80ff7b0 0x7f9ff8198a40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:41.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.043+0000 7f9ffffff700 1 -- 172.21.15.64:0/1387965185 >> 172.21.15.64:0/1387965185 conn(0x7f9ff80faa10 msgr2=0x7f9ff80fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:41.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.043+0000 7f9ffffff700 1 -- 172.21.15.64:0/1387965185 shutdown_connections 2023-09-13T03:12:41.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.043+0000 7f9ffffff700 1 -- 172.21.15.64:0/1387965185 wait complete. 2023-09-13T03:12:41.063 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-13T03:12:41.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.269+0000 7f62e47ff700 1 -- 172.21.15.64:0/952303220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62dc1005f0 msgr2=0x7f62dc100a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:41.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.269+0000 7f62e47ff700 1 --2- 172.21.15.64:0/952303220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62dc1005f0 0x7f62dc100a10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f62d8009990 tx=0x7f62d8009ca0 comp rx=0 tx=0).stop 2023-09-13T03:12:41.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.269+0000 7f62e47ff700 1 -- 172.21.15.64:0/952303220 shutdown_connections 2023-09-13T03:12:41.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.269+0000 7f62e47ff700 1 --2- 172.21.15.64:0/952303220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62dc102180 0x7f62dc108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:41.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.269+0000 7f62e47ff700 1 --2- 172.21.15.64:0/952303220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62dc1005f0 0x7f62dc100a10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:41.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.269+0000 7f62e47ff700 1 --2- 172.21.15.64:0/952303220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62dc1017e0 0x7f62dc101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:41.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.269+0000 7f62e47ff700 1 -- 172.21.15.64:0/952303220 >> 172.21.15.64:0/952303220 conn(0x7f62dc0fbb40 msgr2=0x7f62dc0fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:41.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.269+0000 7f62e47ff700 1 -- 172.21.15.64:0/952303220 shutdown_connections 2023-09-13T03:12:41.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.269+0000 7f62e47ff700 1 -- 172.21.15.64:0/952303220 wait complete. 2023-09-13T03:12:41.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.270+0000 7f62e47ff700 1 Processor -- start 2023-09-13T03:12:41.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.270+0000 7f62e47ff700 1 -- start start 2023-09-13T03:12:41.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.270+0000 7f62e47ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62dc1005f0 0x7f62dc106370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:41.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.270+0000 7f62e47ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62dc1017e0 0x7f62dc1068b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:41.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.270+0000 7f62e47ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62dc102180 0x7f62dc102f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:41.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.270+0000 7f62e47ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc1036a0 con 0x7f62dc102180 2023-09-13T03:12:41.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.270+0000 7f62e47ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc103810 con 0x7f62dc1017e0 2023-09-13T03:12:41.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.270+0000 7f62e47ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc1039d0 con 0x7f62dc1005f0 2023-09-13T03:12:41.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.271+0000 7f62e2d9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62dc102180 0x7f62dc102f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:41.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.271+0000 7f62e1d9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62dc1017e0 0x7f62dc1068b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:41.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.271+0000 7f62e2d9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62dc102180 0x7f62dc102f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54130/0 (socket says 172.21.15.64:54130) 2023-09-13T03:12:41.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.271+0000 7f62e1d9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62dc1017e0 0x7f62dc1068b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:53092/0 (socket says 172.21.15.64:53092) 2023-09-13T03:12:41.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.271+0000 7f62e2d9c700 1 -- 172.21.15.64:0/3859013298 learned_addr learned my addr 172.21.15.64:0/3859013298 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:41.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.271+0000 7f62e259b700 1 --2- 172.21.15.64:0/3859013298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62dc1005f0 0x7f62dc106370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:41.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.271+0000 7f62e2d9c700 1 -- 172.21.15.64:0/3859013298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62dc1005f0 msgr2=0x7f62dc106370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:41.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.271+0000 7f62e2d9c700 1 --2- 172.21.15.64:0/3859013298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62dc1005f0 0x7f62dc106370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:41.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.271+0000 7f62e2d9c700 1 -- 172.21.15.64:0/3859013298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62dc1017e0 msgr2=0x7f62dc1068b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:41.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.271+0000 7f62e2d9c700 1 --2- 172.21.15.64:0/3859013298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62dc1017e0 0x7f62dc1068b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:41.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.271+0000 7f62e2d9c700 1 -- 172.21.15.64:0/3859013298 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d80095f0 con 0x7f62dc102180 2023-09-13T03:12:41.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.271+0000 7f62e1d9a700 1 --2- 172.21.15.64:0/3859013298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62dc1017e0 0x7f62dc1068b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:41.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.271+0000 7f62e2d9c700 1 --2- 172.21.15.64:0/3859013298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62dc102180 0x7f62dc102f00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f62d400ef30 tx=0x7f62d400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:41.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.272+0000 7f62d37fe700 1 -- 172.21.15.64:0/3859013298 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d4018070 con 0x7f62dc102180 2023-09-13T03:12:41.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.272+0000 7f62e47ff700 1 -- 172.21.15.64:0/3859013298 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62dc1a4a70 con 0x7f62dc102180 2023-09-13T03:12:41.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.272+0000 7f62d37fe700 1 -- 172.21.15.64:0/3859013298 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62d400f040 con 0x7f62dc102180 2023-09-13T03:12:41.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.272+0000 7f62d37fe700 1 -- 172.21.15.64:0/3859013298 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d400cd00 con 0x7f62dc102180 2023-09-13T03:12:41.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.272+0000 7f62e47ff700 1 -- 172.21.15.64:0/3859013298 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62dc1a4fa0 con 0x7f62dc102180 2023-09-13T03:12:41.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.273+0000 7f62e47ff700 1 -- 172.21.15.64:0/3859013298 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62dc192c50 con 0x7f62dc102180 2023-09-13T03:12:41.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.275+0000 7f62d37fe700 1 -- 172.21.15.64:0/3859013298 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f62d4003a00 con 0x7f62dc102180 2023-09-13T03:12:41.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.276+0000 7f62d37fe700 1 --2- 172.21.15.64:0/3859013298 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f62c80a0640 0x7f62c80a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:41.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.276+0000 7f62d37fe700 1 -- 172.21.15.64:0/3859013298 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f62d40cb6c0 con 0x7f62dc102180 2023-09-13T03:12:41.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.276+0000 7f62e259b700 1 --2- 172.21.15.64:0/3859013298 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f62c80a0640 0x7f62c80a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:41.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.277+0000 7f62e259b700 1 --2- 172.21.15.64:0/3859013298 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f62c80a0640 0x7f62c80a2b00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f62d800b2e0 tx=0x7f62d8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:41.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.280+0000 7f62d37fe700 1 -- 172.21.15.64:0/3859013298 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f62d4095250 con 0x7f62dc102180 2023-09-13T03:12:41.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.491+0000 7f62e47ff700 1 -- 172.21.15.64:0/3859013298 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f62dc02cfa0 con 0x7f62dc102180 2023-09-13T03:12:41.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.492+0000 7f62d37fe700 1 -- 172.21.15.64:0/3859013298 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f62d4098870 con 0x7f62dc102180 2023-09-13T03:12:41.493 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-09-13T03:12:41.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.494+0000 7f62e47ff700 1 -- 172.21.15.64:0/3859013298 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f62c80a0640 msgr2=0x7f62c80a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:41.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.494+0000 7f62e47ff700 1 --2- 172.21.15.64:0/3859013298 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f62c80a0640 0x7f62c80a2b00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f62d800b2e0 tx=0x7f62d8005fb0 comp rx=0 tx=0).stop 2023-09-13T03:12:41.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.494+0000 7f62e47ff700 1 -- 172.21.15.64:0/3859013298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62dc102180 msgr2=0x7f62dc102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:41.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.494+0000 7f62e47ff700 1 --2- 172.21.15.64:0/3859013298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62dc102180 0x7f62dc102f00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f62d400ef30 tx=0x7f62d400c3a0 comp rx=0 tx=0).stop 2023-09-13T03:12:41.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.495+0000 7f62e47ff700 1 -- 172.21.15.64:0/3859013298 shutdown_connections 2023-09-13T03:12:41.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.495+0000 7f62e47ff700 1 --2- 172.21.15.64:0/3859013298 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f62c80a0640 0x7f62c80a2b00 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:41.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.495+0000 7f62e47ff700 1 --2- 172.21.15.64:0/3859013298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62dc102180 0x7f62dc102f00 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:41.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.495+0000 7f62e47ff700 1 --2- 172.21.15.64:0/3859013298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62dc1005f0 0x7f62dc106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:41.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.495+0000 7f62e47ff700 1 --2- 172.21.15.64:0/3859013298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62dc1017e0 0x7f62dc1068b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:41.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.495+0000 7f62e47ff700 1 -- 172.21.15.64:0/3859013298 >> 172.21.15.64:0/3859013298 conn(0x7f62dc0fbb40 msgr2=0x7f62dc107140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:41.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.495+0000 7f62e47ff700 1 -- 172.21.15.64:0/3859013298 shutdown_connections 2023-09-13T03:12:41.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.495+0000 7f62e47ff700 1 -- 172.21.15.64:0/3859013298 wait complete. 2023-09-13T03:12:41.514 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-13T03:12:41.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.722+0000 7f78341ff700 1 -- 172.21.15.64:0/3094907894 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f782c1017e0 msgr2=0x7f782c101c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:41.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.722+0000 7f78341ff700 1 --2- 172.21.15.64:0/3094907894 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f782c1017e0 0x7f782c101c40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f781c0098f0 tx=0x7f781c009c00 comp rx=0 tx=0).stop 2023-09-13T03:12:41.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.722+0000 7f78341ff700 1 -- 172.21.15.64:0/3094907894 shutdown_connections 2023-09-13T03:12:41.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.722+0000 7f78341ff700 1 --2- 172.21.15.64:0/3094907894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f782c1005f0 0x7f782c100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:41.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.722+0000 7f78341ff700 1 --2- 172.21.15.64:0/3094907894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f782c102180 0x7f782c108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:41.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.722+0000 7f78341ff700 1 --2- 172.21.15.64:0/3094907894 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f782c1017e0 0x7f782c101c40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:41.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.722+0000 7f78341ff700 1 -- 172.21.15.64:0/3094907894 >> 172.21.15.64:0/3094907894 conn(0x7f782c0fbb40 msgr2=0x7f782c0fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:41.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.722+0000 7f78341ff700 1 -- 172.21.15.64:0/3094907894 shutdown_connections 2023-09-13T03:12:41.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.722+0000 7f78341ff700 1 -- 172.21.15.64:0/3094907894 wait complete. 2023-09-13T03:12:41.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.723+0000 7f78341ff700 1 Processor -- start 2023-09-13T03:12:41.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.723+0000 7f78341ff700 1 -- start start 2023-09-13T03:12:41.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.724+0000 7f78341ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f782c1005f0 0x7f782c1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:41.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.724+0000 7f78341ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f782c1017e0 0x7f782c198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:41.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.724+0000 7f78341ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f782c102180 0x7f782c19fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:41.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.724+0000 7f78341ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782c1a0140 con 0x7f782c1017e0 2023-09-13T03:12:41.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.724+0000 7f78341ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782c1a0280 con 0x7f782c102180 2023-09-13T03:12:41.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.724+0000 7f78341ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782c1a0410 con 0x7f782c1005f0 2023-09-13T03:12:41.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.724+0000 7f783179a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f782c1017e0 0x7f782c198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:41.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.724+0000 7f783179a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f782c1017e0 0x7f782c198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54158/0 (socket says 172.21.15.64:54158) 2023-09-13T03:12:41.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.724+0000 7f783179a700 1 -- 172.21.15.64:0/2310138174 learned_addr learned my addr 172.21.15.64:0/2310138174 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:41.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.724+0000 7f7831f9b700 1 --2- 172.21.15.64:0/2310138174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f782c1005f0 0x7f782c1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:41.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.724+0000 7f783279c700 1 --2- 172.21.15.64:0/2310138174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f782c102180 0x7f782c19fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:41.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.724+0000 7f783179a700 1 -- 172.21.15.64:0/2310138174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f782c1005f0 msgr2=0x7f782c1985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:41.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.724+0000 7f783179a700 1 --2- 172.21.15.64:0/2310138174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f782c1005f0 0x7f782c1985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:41.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.724+0000 7f783179a700 1 -- 172.21.15.64:0/2310138174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f782c102180 msgr2=0x7f782c19fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:41.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.724+0000 7f783179a700 1 --2- 172.21.15.64:0/2310138174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f782c102180 0x7f782c19fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:41.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.724+0000 7f783179a700 1 -- 172.21.15.64:0/2310138174 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f781c009520 con 0x7f782c1017e0 2023-09-13T03:12:41.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.725+0000 7f783179a700 1 --2- 172.21.15.64:0/2310138174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f782c1017e0 0x7f782c198b00 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f781c009fd0 tx=0x7f781c00b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:41.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.725+0000 7f7822ffd700 1 -- 172.21.15.64:0/2310138174 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781c01c070 con 0x7f782c1017e0 2023-09-13T03:12:41.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.725+0000 7f7822ffd700 1 -- 172.21.15.64:0/2310138174 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f781c00bc10 con 0x7f782c1017e0 2023-09-13T03:12:41.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.725+0000 7f78341ff700 1 -- 172.21.15.64:0/2310138174 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f782c1a0690 con 0x7f782c1017e0 2023-09-13T03:12:41.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.725+0000 7f7822ffd700 1 -- 172.21.15.64:0/2310138174 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781c004a20 con 0x7f782c1017e0 2023-09-13T03:12:41.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.725+0000 7f78341ff700 1 -- 172.21.15.64:0/2310138174 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f782c1a0ab0 con 0x7f782c1017e0 2023-09-13T03:12:41.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.726+0000 7f78341ff700 1 -- 172.21.15.64:0/2310138174 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f782c192c40 con 0x7f782c1017e0 2023-09-13T03:12:41.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.728+0000 7f7822ffd700 1 -- 172.21.15.64:0/2310138174 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f781c020070 con 0x7f782c1017e0 2023-09-13T03:12:41.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.729+0000 7f7822ffd700 1 --2- 172.21.15.64:0/2310138174 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f78180a0640 0x7f78180a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:41.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.729+0000 7f7822ffd700 1 -- 172.21.15.64:0/2310138174 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f781c0cc810 con 0x7f782c1017e0 2023-09-13T03:12:41.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.729+0000 7f7831f9b700 1 --2- 172.21.15.64:0/2310138174 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f78180a0640 0x7f78180a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:41.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.730+0000 7f7831f9b700 1 --2- 172.21.15.64:0/2310138174 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f78180a0640 0x7f78180a2b00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f782c101640 tx=0x7f7828009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:41.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.733+0000 7f7822ffd700 1 -- 172.21.15.64:0/2310138174 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f781c05ce10 con 0x7f782c1017e0 2023-09-13T03:12:41.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:41 smithi170 ceph-mon[134328]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-13T03:12:41.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:41 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/3185540942' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:12:41.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:41 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/1387965185' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T03:12:41.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:41 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/3859013298' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T03:12:41.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:41 smithi064 ceph-mon[97675]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-13T03:12:41.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:41 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/3185540942' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:12:41.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:41 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/1387965185' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T03:12:41.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:41 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/3859013298' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T03:12:41.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.945+0000 7f78341ff700 1 -- 172.21.15.64:0/2310138174 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f782c0008d0 con 0x7f782c1017e0 2023-09-13T03:12:41.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.945+0000 7f7822ffd700 1 -- 172.21.15.64:0/2310138174 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f781c0963a0 con 0x7f782c1017e0 2023-09-13T03:12:41.946 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-09-13T03:12:41.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.948+0000 7f78341ff700 1 -- 172.21.15.64:0/2310138174 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f78180a0640 msgr2=0x7f78180a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:41.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.948+0000 7f78341ff700 1 --2- 172.21.15.64:0/2310138174 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f78180a0640 0x7f78180a2b00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f782c101640 tx=0x7f7828009260 comp rx=0 tx=0).stop 2023-09-13T03:12:41.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.948+0000 7f78341ff700 1 -- 172.21.15.64:0/2310138174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f782c1017e0 msgr2=0x7f782c198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:41.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.948+0000 7f78341ff700 1 --2- 172.21.15.64:0/2310138174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f782c1017e0 0x7f782c198b00 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f781c009fd0 tx=0x7f781c00b560 comp rx=0 tx=0).stop 2023-09-13T03:12:41.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.948+0000 7f78341ff700 1 -- 172.21.15.64:0/2310138174 shutdown_connections 2023-09-13T03:12:41.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.948+0000 7f78341ff700 1 --2- 172.21.15.64:0/2310138174 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f78180a0640 0x7f78180a2b00 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:41.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.948+0000 7f78341ff700 1 --2- 172.21.15.64:0/2310138174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f782c102180 0x7f782c19fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:41.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.948+0000 7f78341ff700 1 --2- 172.21.15.64:0/2310138174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f782c1005f0 0x7f782c1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:41.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.948+0000 7f78341ff700 1 --2- 172.21.15.64:0/2310138174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f782c1017e0 0x7f782c198b00 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:41.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.948+0000 7f78341ff700 1 -- 172.21.15.64:0/2310138174 >> 172.21.15.64:0/2310138174 conn(0x7f782c0fbb40 msgr2=0x7f782c1071c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:41.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.948+0000 7f78341ff700 1 -- 172.21.15.64:0/2310138174 shutdown_connections 2023-09-13T03:12:41.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:41.948+0000 7f78341ff700 1 -- 172.21.15.64:0/2310138174 wait complete. 2023-09-13T03:12:41.968 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T03:12:41.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:41 smithi142 ceph-mon[134082]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-13T03:12:41.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:41 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/3185540942' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:12:41.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:41 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/1387965185' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T03:12:41.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:41 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/3859013298' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T03:12:42.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.170+0000 7f2fa47ff700 1 -- 172.21.15.64:0/4174735534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f9c1017f0 msgr2=0x7f2f9c101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:42.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.170+0000 7f2fa47ff700 1 --2- 172.21.15.64:0/4174735534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f9c1017f0 0x7f2f9c101c50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c0099c0 tx=0x7f2f8c009cd0 comp rx=0 tx=0).stop 2023-09-13T03:12:42.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.170+0000 7f2fa47ff700 1 -- 172.21.15.64:0/4174735534 shutdown_connections 2023-09-13T03:12:42.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.170+0000 7f2fa47ff700 1 --2- 172.21.15.64:0/4174735534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f9c100600 0x7f2f9c100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:42.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.170+0000 7f2fa47ff700 1 --2- 172.21.15.64:0/4174735534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f9c102190 0x7f2f9c108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:42.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.170+0000 7f2fa47ff700 1 --2- 172.21.15.64:0/4174735534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f9c1017f0 0x7f2f9c101c50 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:42.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.170+0000 7f2fa47ff700 1 -- 172.21.15.64:0/4174735534 >> 172.21.15.64:0/4174735534 conn(0x7f2f9c0fbb30 msgr2=0x7f2f9c0fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:42.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.170+0000 7f2fa47ff700 1 -- 172.21.15.64:0/4174735534 shutdown_connections 2023-09-13T03:12:42.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.170+0000 7f2fa47ff700 1 -- 172.21.15.64:0/4174735534 wait complete. 2023-09-13T03:12:42.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.171+0000 7f2fa47ff700 1 Processor -- start 2023-09-13T03:12:42.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.172+0000 7f2fa47ff700 1 -- start start 2023-09-13T03:12:42.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.172+0000 7f2fa47ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f9c100600 0x7f2f9c1984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:42.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.172+0000 7f2fa47ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f9c1017f0 0x7f2f9c198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:42.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.172+0000 7f2fa47ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f9c102190 0x7f2f9c19fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:42.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.172+0000 7f2fa47ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9c1a0040 con 0x7f2f9c102190 2023-09-13T03:12:42.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.172+0000 7f2fa47ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9c1a0180 con 0x7f2f9c1017f0 2023-09-13T03:12:42.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.172+0000 7f2fa47ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9c1a0310 con 0x7f2f9c100600 2023-09-13T03:12:42.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.172+0000 7f2fa2d9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f9c102190 0x7f2f9c19fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:42.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.172+0000 7f2fa259b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f9c100600 0x7f2f9c1984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:42.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.172+0000 7f2fa2d9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f9c102190 0x7f2f9c19fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54190/0 (socket says 172.21.15.64:54190) 2023-09-13T03:12:42.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.172+0000 7f2fa259b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f9c100600 0x7f2f9c1984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:50698/0 (socket says 172.21.15.64:50698) 2023-09-13T03:12:42.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.172+0000 7f2fa2d9c700 1 -- 172.21.15.64:0/665854313 learned_addr learned my addr 172.21.15.64:0/665854313 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:42.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.172+0000 7f2fa1d9a700 1 --2- 172.21.15.64:0/665854313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f9c1017f0 0x7f2f9c198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:42.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.173+0000 7f2fa2d9c700 1 -- 172.21.15.64:0/665854313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f9c100600 msgr2=0x7f2f9c1984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:42.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.173+0000 7f2fa2d9c700 1 --2- 172.21.15.64:0/665854313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f9c100600 0x7f2f9c1984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:42.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.173+0000 7f2fa2d9c700 1 -- 172.21.15.64:0/665854313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f9c1017f0 msgr2=0x7f2f9c198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:42.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.173+0000 7f2fa2d9c700 1 --2- 172.21.15.64:0/665854313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f9c1017f0 0x7f2f9c198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:42.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.173+0000 7f2fa2d9c700 1 -- 172.21.15.64:0/665854313 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f98009520 con 0x7f2f9c102190 2023-09-13T03:12:42.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.173+0000 7f2fa2d9c700 1 --2- 172.21.15.64:0/665854313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f9c102190 0x7f2f9c19fa70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2f9400ef30 tx=0x7f2f9400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:42.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.173+0000 7f2f937fe700 1 -- 172.21.15.64:0/665854313 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f94018070 con 0x7f2f9c102190 2023-09-13T03:12:42.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.173+0000 7f2f937fe700 1 -- 172.21.15.64:0/665854313 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f9400f040 con 0x7f2f9c102190 2023-09-13T03:12:42.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.173+0000 7f2f937fe700 1 -- 172.21.15.64:0/665854313 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9400cd00 con 0x7f2f9c102190 2023-09-13T03:12:42.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.173+0000 7f2fa47ff700 1 -- 172.21.15.64:0/665854313 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f8c0095f0 con 0x7f2f9c102190 2023-09-13T03:12:42.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.173+0000 7f2fa47ff700 1 -- 172.21.15.64:0/665854313 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f9c1a08c0 con 0x7f2f9c102190 2023-09-13T03:12:42.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.174+0000 7f2fa47ff700 1 -- 172.21.15.64:0/665854313 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f9c10be60 con 0x7f2f9c102190 2023-09-13T03:12:42.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.176+0000 7f2f937fe700 1 -- 172.21.15.64:0/665854313 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f2f94003a00 con 0x7f2f9c102190 2023-09-13T03:12:42.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.178+0000 7f2f937fe700 1 --2- 172.21.15.64:0/665854313 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f2f880a0710 0x7f2f880a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:42.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.178+0000 7f2f937fe700 1 -- 172.21.15.64:0/665854313 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f2f940cbff0 con 0x7f2f9c102190 2023-09-13T03:12:42.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.178+0000 7f2fa259b700 1 --2- 172.21.15.64:0/665854313 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f2f880a0710 0x7f2f880a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:42.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.179+0000 7f2fa259b700 1 --2- 172.21.15.64:0/665854313 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f2f880a0710 0x7f2f880a2bd0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c101650 tx=0x7f2f98009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:42.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.180+0000 7f2f937fe700 1 -- 172.21.15.64:0/665854313 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f2f94095bc0 con 0x7f2f9c102190 2023-09-13T03:12:42.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.438+0000 7f2fa47ff700 1 -- 172.21.15.64:0/665854313 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f9c10c0a0 con 0x7f2f9c102190 2023-09-13T03:12:42.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.439+0000 7f2f937fe700 1 -- 172.21.15.64:0/665854313 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11366 (secure 0 0 0) 0x7f2f940991e0 con 0x7f2f9c102190 2023-09-13T03:12:42.440 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:12:42.440 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2234182985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2234182985},{"type":"v1","addr":"172.21.15.64:6835","nonce":2234182985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4053841873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4053841873},{"type":"v1","addr":"172.21.15.64:6837","nonce":4053841873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-13T03:11:10.490260+0000","modified":"2023-09-13T03:11:18.230740+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24686,"mds_1":24649,"mds_2":24635,"mds_3":24634,"mds_4":24622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6839/99523374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":99523374},{"type":"v1","addr":"172.21.15.64:6839","nonce":99523374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14649":{"gid":14649,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6841/1866582875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":1866582875},{"type":"v1","addr":"172.21.15.64:6841","nonce":1866582875}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/101906999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":101906999},{"type":"v1","addr":"172.21.15.142:6833","nonce":101906999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24574":{"gid":24574,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/1897366227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1897366227},{"type":"v1","addr":"172.21.15.170:6833","nonce":1897366227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24611":{"gid":24611,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2167779918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2167779918},{"type":"v1","addr":"172.21.15.142:6835","nonce":2167779918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24622":{"gid":24622,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6835/1493371121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1493371121},{"type":"v1","addr":"172.21.15.170:6835","nonce":1493371121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24634":{"gid":24634,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.170:6837/1726587275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":1726587275},{"type":"v1","addr":"172.21.15.170:6837","nonce":1726587275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24635":{"gid":24635,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.142:6837/741173268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":741173268},{"type":"v1","addr":"172.21.15.142:6837","nonce":741173268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24649":{"gid":24649,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.170:6839/2189703596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2189703596},{"type":"v1","addr":"172.21.15.170:6839","nonce":2189703596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24686":{"gid":24686,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/3645998616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645998616},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645998616}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-13T03:12:42.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.442+0000 7f2fa47ff700 1 -- 172.21.15.64:0/665854313 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f2f880a0710 msgr2=0x7f2f880a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:42.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.442+0000 7f2fa47ff700 1 --2- 172.21.15.64:0/665854313 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f2f880a0710 0x7f2f880a2bd0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c101650 tx=0x7f2f98009260 comp rx=0 tx=0).stop 2023-09-13T03:12:42.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.442+0000 7f2fa47ff700 1 -- 172.21.15.64:0/665854313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f9c102190 msgr2=0x7f2f9c19fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:42.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.442+0000 7f2fa47ff700 1 --2- 172.21.15.64:0/665854313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f9c102190 0x7f2f9c19fa70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2f9400ef30 tx=0x7f2f9400c3a0 comp rx=0 tx=0).stop 2023-09-13T03:12:42.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.442+0000 7f2fa47ff700 1 -- 172.21.15.64:0/665854313 shutdown_connections 2023-09-13T03:12:42.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.442+0000 7f2fa47ff700 1 --2- 172.21.15.64:0/665854313 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f2f880a0710 0x7f2f880a2bd0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:42.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.442+0000 7f2fa47ff700 1 --2- 172.21.15.64:0/665854313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f9c102190 0x7f2f9c19fa70 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:42.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.442+0000 7f2fa47ff700 1 --2- 172.21.15.64:0/665854313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f9c100600 0x7f2f9c1984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:42.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.442+0000 7f2fa47ff700 1 --2- 172.21.15.64:0/665854313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f9c1017f0 0x7f2f9c198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:42.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.442+0000 7f2fa47ff700 1 -- 172.21.15.64:0/665854313 >> 172.21.15.64:0/665854313 conn(0x7f2f9c0fbb30 msgr2=0x7f2f9c107080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:42.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.442+0000 7f2fa47ff700 1 -- 172.21.15.64:0/665854313 shutdown_connections 2023-09-13T03:12:42.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.442+0000 7f2fa47ff700 1 -- 172.21.15.64:0/665854313 wait complete. 2023-09-13T03:12:42.445 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 26 2023-09-13T03:12:42.462 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-13T03:12:42.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.665+0000 7f5c86fff700 1 -- 172.21.15.64:0/4116681057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c80102190 msgr2=0x7f5c80108a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:42.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.665+0000 7f5c86fff700 1 --2- 172.21.15.64:0/4116681057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c80102190 0x7f5c80108a40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5c7800b0d0 tx=0x7f5c7800b3e0 comp rx=0 tx=0).stop 2023-09-13T03:12:42.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.665+0000 7f5c86fff700 1 -- 172.21.15.64:0/4116681057 shutdown_connections 2023-09-13T03:12:42.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.665+0000 7f5c86fff700 1 --2- 172.21.15.64:0/4116681057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c80100600 0x7f5c80100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:42.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.665+0000 7f5c86fff700 1 --2- 172.21.15.64:0/4116681057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c80102190 0x7f5c80108a40 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:42.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.665+0000 7f5c86fff700 1 --2- 172.21.15.64:0/4116681057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c801017f0 0x7f5c80101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:42.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.665+0000 7f5c86fff700 1 -- 172.21.15.64:0/4116681057 >> 172.21.15.64:0/4116681057 conn(0x7f5c800fbb30 msgr2=0x7f5c800fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:42.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.665+0000 7f5c86fff700 1 -- 172.21.15.64:0/4116681057 shutdown_connections 2023-09-13T03:12:42.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.666+0000 7f5c86fff700 1 -- 172.21.15.64:0/4116681057 wait complete. 2023-09-13T03:12:42.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.667+0000 7f5c86fff700 1 Processor -- start 2023-09-13T03:12:42.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.667+0000 7f5c86fff700 1 -- start start 2023-09-13T03:12:42.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.668+0000 7f5c86fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c80100600 0x7f5c800743b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:42.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.668+0000 7f5c86fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c801017f0 0x7f5c800729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:42.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.668+0000 7f5c86fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c80102190 0x7f5c80072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:42.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.668+0000 7f5c86fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c800736d0 con 0x7f5c80100600 2023-09-13T03:12:42.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.668+0000 7f5c86fff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c80073840 con 0x7f5c801017f0 2023-09-13T03:12:42.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.668+0000 7f5c86fff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c801a0410 con 0x7f5c80102190 2023-09-13T03:12:42.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.668+0000 7f5c84d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c80100600 0x7f5c800743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:42.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.668+0000 7f5c84d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c80100600 0x7f5c800743b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54220/0 (socket says 172.21.15.64:54220) 2023-09-13T03:12:42.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.668+0000 7f5c84d9b700 1 -- 172.21.15.64:0/246091133 learned_addr learned my addr 172.21.15.64:0/246091133 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:42.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.668+0000 7f5c8559c700 1 --2- 172.21.15.64:0/246091133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c80102190 0x7f5c80072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:42.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.668+0000 7f5c7ffff700 1 --2- 172.21.15.64:0/246091133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c801017f0 0x7f5c800729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:42.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.668+0000 7f5c84d9b700 1 -- 172.21.15.64:0/246091133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c80102190 msgr2=0x7f5c80072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:42.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.668+0000 7f5c84d9b700 1 --2- 172.21.15.64:0/246091133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c80102190 0x7f5c80072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:42.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.668+0000 7f5c84d9b700 1 -- 172.21.15.64:0/246091133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c801017f0 msgr2=0x7f5c800729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:42.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.668+0000 7f5c84d9b700 1 --2- 172.21.15.64:0/246091133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c801017f0 0x7f5c800729f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:42.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.668+0000 7f5c84d9b700 1 -- 172.21.15.64:0/246091133 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c78009d00 con 0x7f5c80100600 2023-09-13T03:12:42.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.668+0000 7f5c7ffff700 1 --2- 172.21.15.64:0/246091133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c801017f0 0x7f5c800729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:12:42.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.669+0000 7f5c84d9b700 1 --2- 172.21.15.64:0/246091133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c80100600 0x7f5c800743b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5c7400e970 tx=0x7f5c7400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:42.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.669+0000 7f5c7dffb700 1 -- 172.21.15.64:0/246091133 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c74009790 con 0x7f5c80100600 2023-09-13T03:12:42.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.669+0000 7f5c7dffb700 1 -- 172.21.15.64:0/246091133 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c7400eec0 con 0x7f5c80100600 2023-09-13T03:12:42.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.669+0000 7f5c86fff700 1 -- 172.21.15.64:0/246091133 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c801a06f0 con 0x7f5c80100600 2023-09-13T03:12:42.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.669+0000 7f5c7dffb700 1 -- 172.21.15.64:0/246091133 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c74003cf0 con 0x7f5c80100600 2023-09-13T03:12:42.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.669+0000 7f5c86fff700 1 -- 172.21.15.64:0/246091133 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c801a0c50 con 0x7f5c80100600 2023-09-13T03:12:42.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.670+0000 7f5c86fff700 1 -- 172.21.15.64:0/246091133 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c8010be70 con 0x7f5c80100600 2023-09-13T03:12:42.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.672+0000 7f5c7dffb700 1 -- 172.21.15.64:0/246091133 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f5c7400ccd0 con 0x7f5c80100600 2023-09-13T03:12:42.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.673+0000 7f5c7dffb700 1 --2- 172.21.15.64:0/246091133 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f5c700a0710 0x7f5c700a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:42.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.673+0000 7f5c7dffb700 1 -- 172.21.15.64:0/246091133 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f5c74014070 con 0x7f5c80100600 2023-09-13T03:12:42.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.674+0000 7f5c7ffff700 1 --2- 172.21.15.64:0/246091133 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f5c700a0710 0x7f5c700a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:42.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.674+0000 7f5c7ffff700 1 --2- 172.21.15.64:0/246091133 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f5c700a0710 0x7f5c700a2bd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c009cb0 tx=0x7f5c6c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:42.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.677+0000 7f5c7dffb700 1 -- 172.21.15.64:0/246091133 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5c740958b0 con 0x7f5c80100600 2023-09-13T03:12:42.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.889+0000 7f5c86fff700 1 -- 172.21.15.64:0/246091133 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c8010c080 con 0x7f5c80100600 2023-09-13T03:12:42.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.890+0000 7f5c7dffb700 1 -- 172.21.15.64:0/246091133 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22473 (secure 0 0 0) 0x7f5c74098ed0 con 0x7f5c80100600 2023-09-13T03:12:42.891 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:12:42.892 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":85,"fsid":"f8dbced4-51e1-11ee-9ab7-7b867c8bd7da","created":"2023-09-13T03:03:24.167202+0000","modified":"2023-09-13T03:11:24.258871+0000","last_up_change":"2023-09-13T03:10:22.503529+0000","last_in_change":"2023-09-13T03:10:05.678720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T03:06:58.987479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-13T03:11:08.589202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-13T03:11:09.924659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3997c114-241b-48c4-8feb-653c99209470","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6803","nonce":3940892135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6805","nonce":3940892135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6809","nonce":3940892135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6807","nonce":3940892135}]},"public_addr":"172.21.15.64:6803/3940892135","cluster_addr":"172.21.15.64:6805/3940892135","heartbeat_back_addr":"172.21.15.64:6809/3940892135","heartbeat_front_addr":"172.21.15.64:6807/3940892135","state":["exists","up"]},{"osd":1,"uuid":"94cfae85-322e-4ed2-a6e6-e40349bb14bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6801","nonce":4188149333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6803","nonce":4188149333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6807","nonce":4188149333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6805","nonce":4188149333}]},"public_addr":"172.21.15.142:6801/4188149333","cluster_addr":"172.21.15.142:6803/4188149333","heartbeat_back_addr":"172.21.15.142:6807/4188149333","heartbeat_front_addr":"172.21.15.142:6805/4188149333","state":["exists","up"]},{"osd":2,"uuid":"b3aae17e-96db-4008-8c27-f733c46b30ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6801","nonce":2406669338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6803","nonce":2406669338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6807","nonce":2406669338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6805","nonce":2406669338}]},"public_addr":"172.21.15.170:6801/2406669338","cluster_addr":"172.21.15.170:6803/2406669338","heartbeat_back_addr":"172.21.15.170:6807/2406669338","heartbeat_front_addr":"172.21.15.170:6805/2406669338","state":["exists","up"]},{"osd":3,"uuid":"7fa8d8cb-2ab7-48a7-8b49-5e2903d6779f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6811","nonce":3558318707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6813","nonce":3558318707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6817","nonce":3558318707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6815","nonce":3558318707}]},"public_addr":"172.21.15.64:6811/3558318707","cluster_addr":"172.21.15.64:6813/3558318707","heartbeat_back_addr":"172.21.15.64:6817/3558318707","heartbeat_front_addr":"172.21.15.64:6815/3558318707","state":["exists","up"]},{"osd":4,"uuid":"8c7a9c9f-f919-48d3-9db2-214a25bb5603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6809","nonce":1811142576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6811","nonce":1811142576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6815","nonce":1811142576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6813","nonce":1811142576}]},"public_addr":"172.21.15.142:6809/1811142576","cluster_addr":"172.21.15.142:6811/1811142576","heartbeat_back_addr":"172.21.15.142:6815/1811142576","heartbeat_front_addr":"172.21.15.142:6813/1811142576","state":["exists","up"]},{"osd":5,"uuid":"0c6101c6-2065-4022-a751-d3f216d7f478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6809","nonce":2903841453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6811","nonce":2903841453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6815","nonce":2903841453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6813","nonce":2903841453}]},"public_addr":"172.21.15.170:6809/2903841453","cluster_addr":"172.21.15.170:6811/2903841453","heartbeat_back_addr":"172.21.15.170:6815/2903841453","heartbeat_front_addr":"172.21.15.170:6813/2903841453","state":["exists","up"]},{"osd":6,"uuid":"5cf3e52e-d155-41b1-9194-db047d26a953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6819","nonce":1436584490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6821","nonce":1436584490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6825","nonce":1436584490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6823","nonce":1436584490}]},"public_addr":"172.21.15.64:6819/1436584490","cluster_addr":"172.21.15.64:6821/1436584490","heartbeat_back_addr":"172.21.15.64:6825/1436584490","heartbeat_front_addr":"172.21.15.64:6823/1436584490","state":["exists","up"]},{"osd":7,"uuid":"e97323e3-badd-41a7-af9e-e78a81a64046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6817","nonce":3726793534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6819","nonce":3726793534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6823","nonce":3726793534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6821","nonce":3726793534}]},"public_addr":"172.21.15.142:6817/3726793534","cluster_addr":"172.21.15.142:6819/3726793534","heartbeat_back_addr":"172.21.15.142:6823/3726793534","heartbeat_front_addr":"172.21.15.142:6821/3726793534","state":["exists","up"]},{"osd":8,"uuid":"83d2835c-ed26-493d-9222-c2894ec0e071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6817","nonce":4272513873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6819","nonce":4272513873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6823","nonce":4272513873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6821","nonce":4272513873}]},"public_addr":"172.21.15.170:6817/4272513873","cluster_addr":"172.21.15.170:6819/4272513873","heartbeat_back_addr":"172.21.15.170:6823/4272513873","heartbeat_front_addr":"172.21.15.170:6821/4272513873","state":["exists","up"]},{"osd":9,"uuid":"cb936bdd-1bec-4236-aa2e-a24adcac11e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6827","nonce":2641175211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6829","nonce":2641175211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6833","nonce":2641175211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6831","nonce":2641175211}]},"public_addr":"172.21.15.64:6827/2641175211","cluster_addr":"172.21.15.64:6829/2641175211","heartbeat_back_addr":"172.21.15.64:6833/2641175211","heartbeat_front_addr":"172.21.15.64:6831/2641175211","state":["exists","up"]},{"osd":10,"uuid":"85512a1c-6171-43d4-9c15-c588a8b71be0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6825","nonce":936499217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6827","nonce":936499217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6831","nonce":936499217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6829","nonce":936499217}]},"public_addr":"172.21.15.142:6825/936499217","cluster_addr":"172.21.15.142:6827/936499217","heartbeat_back_addr":"172.21.15.142:6831/936499217","heartbeat_front_addr":"172.21.15.142:6829/936499217","state":["exists","up"]},{"osd":11,"uuid":"9b8417dc-5d6c-4917-94a9-b032c5bc33d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6825","nonce":3605017060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6827","nonce":3605017060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6831","nonce":3605017060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6829","nonce":3605017060}]},"public_addr":"172.21.15.170:6825/3605017060","cluster_addr":"172.21.15.170:6827/3605017060","heartbeat_back_addr":"172.21.15.170:6831/3605017060","heartbeat_front_addr":"172.21.15.170:6829/3605017060","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:14.229044+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:36.147777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:56.655298+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:07:23.107741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:07:44.663770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:05.577432+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:31.168031+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:52.791055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:09:13.658546+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:09:39.150360+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:10:00.601735+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:10:21.407555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6800/2955227347":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/1373223118":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/271194613":"2023-09-14T03:04:22.355817+0000","172.21.15.64:6801/3998091347":"2023-09-14T03:03:56.442541+0000","172.21.15.64:6800/3998091347":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/920818572":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/735789005":"2023-09-14T03:03:56.442541+0000","172.21.15.64:6801/2955227347":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/1975923178":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/1500151178":"2023-09-14T03:04:22.355817+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T03:12:42.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.893+0000 7f5c86fff700 1 -- 172.21.15.64:0/246091133 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f5c700a0710 msgr2=0x7f5c700a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:42.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.893+0000 7f5c86fff700 1 --2- 172.21.15.64:0/246091133 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f5c700a0710 0x7f5c700a2bd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c009cb0 tx=0x7f5c6c009260 comp rx=0 tx=0).stop 2023-09-13T03:12:42.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.893+0000 7f5c86fff700 1 -- 172.21.15.64:0/246091133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c80100600 msgr2=0x7f5c800743b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:42.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.893+0000 7f5c86fff700 1 --2- 172.21.15.64:0/246091133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c80100600 0x7f5c800743b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5c7400e970 tx=0x7f5c7400ec80 comp rx=0 tx=0).stop 2023-09-13T03:12:42.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.893+0000 7f5c86fff700 1 -- 172.21.15.64:0/246091133 shutdown_connections 2023-09-13T03:12:42.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.893+0000 7f5c86fff700 1 --2- 172.21.15.64:0/246091133 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f5c700a0710 0x7f5c700a2bd0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:42.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.893+0000 7f5c86fff700 1 --2- 172.21.15.64:0/246091133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c80102190 0x7f5c80072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:42.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.893+0000 7f5c86fff700 1 --2- 172.21.15.64:0/246091133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c80100600 0x7f5c800743b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:42.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.893+0000 7f5c86fff700 1 --2- 172.21.15.64:0/246091133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c801017f0 0x7f5c800729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:42.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.893+0000 7f5c86fff700 1 -- 172.21.15.64:0/246091133 >> 172.21.15.64:0/246091133 conn(0x7f5c800fbb30 msgr2=0x7f5c800fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:42.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.893+0000 7f5c86fff700 1 -- 172.21.15.64:0/246091133 shutdown_connections 2023-09-13T03:12:42.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:42.893+0000 7f5c86fff700 1 -- 172.21.15.64:0/246091133 wait complete. 2023-09-13T03:12:42.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:42 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/2310138174' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T03:12:42.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:42 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/665854313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:12:42.910 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:42 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2310138174' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T03:12:42.911 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:42 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/665854313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:12:42.913 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-09-13T03:11:10.490260+0000', 'modified': '2023-09-13T03:11:18.230740+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24686, 'mds_1': 24649, 'mds_2': 24635, 'mds_3': 24634, 'mds_4': 24622}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14637': {'gid': 14637, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.64:6839/99523374', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6838', 'nonce': 99523374}, {'type': 'v1', 'addr': '172.21.15.64:6839', 'nonce': 99523374}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_14649': {'gid': 14649, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.64:6841/1866582875', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6840', 'nonce': 1866582875}, {'type': 'v1', 'addr': '172.21.15.64:6841', 'nonce': 1866582875}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24569': {'gid': 24569, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.142:6833/101906999', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6832', 'nonce': 101906999}, {'type': 'v1', 'addr': '172.21.15.142:6833', 'nonce': 101906999}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24574': {'gid': 24574, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.170:6833/1897366227', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6832', 'nonce': 1897366227}, {'type': 'v1', 'addr': '172.21.15.170:6833', 'nonce': 1897366227}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24611': {'gid': 24611, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.142:6835/2167779918', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6834', 'nonce': 2167779918}, {'type': 'v1', 'addr': '172.21.15.142:6835', 'nonce': 2167779918}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24622': {'gid': 24622, 'name': 'f', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.170:6835/1493371121', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6834', 'nonce': 1493371121}, {'type': 'v1', 'addr': '172.21.15.170:6835', 'nonce': 1493371121}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24634': {'gid': 24634, 'name': 'c', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.170:6837/1726587275', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6836', 'nonce': 1726587275}, {'type': 'v1', 'addr': '172.21.15.170:6837', 'nonce': 1726587275}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24635': {'gid': 24635, 'name': 'h', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.142:6837/741173268', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6836', 'nonce': 741173268}, {'type': 'v1', 'addr': '172.21.15.142:6837', 'nonce': 741173268}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24649': {'gid': 24649, 'name': 'i', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.170:6839/2189703596', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6838', 'nonce': 2189703596}, {'type': 'v1', 'addr': '172.21.15.170:6839', 'nonce': 2189703596}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24686': {'gid': 24686, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.142:6839/3645998616', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6838', 'nonce': 3645998616}, {'type': 'v1', 'addr': '172.21.15.142:6839', 'nonce': 3645998616}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-13T03:12:42.913 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-09-13T03:12:42.914 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2023-09-13T03:12:42.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:42 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/2310138174' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T03:12:42.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:42 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/665854313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:12:43.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.118+0000 7fc434bff700 1 -- 172.21.15.64:0/984095174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4301005f0 msgr2=0x7fc430100a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:43.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.118+0000 7fc434bff700 1 --2- 172.21.15.64:0/984095174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4301005f0 0x7fc430100a10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc4200099c0 tx=0x7fc420009cd0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.118+0000 7fc434bff700 1 -- 172.21.15.64:0/984095174 shutdown_connections 2023-09-13T03:12:43.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.118+0000 7fc434bff700 1 --2- 172.21.15.64:0/984095174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc430102180 0x7fc430108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.118+0000 7fc434bff700 1 --2- 172.21.15.64:0/984095174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4301005f0 0x7fc430100a10 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.118+0000 7fc434bff700 1 --2- 172.21.15.64:0/984095174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4301017e0 0x7fc430101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.118+0000 7fc434bff700 1 -- 172.21.15.64:0/984095174 >> 172.21.15.64:0/984095174 conn(0x7fc4300fbb40 msgr2=0x7fc4300fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:43.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.118+0000 7fc434bff700 1 -- 172.21.15.64:0/984095174 shutdown_connections 2023-09-13T03:12:43.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.118+0000 7fc434bff700 1 -- 172.21.15.64:0/984095174 wait complete. 2023-09-13T03:12:43.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.120+0000 7fc434bff700 1 Processor -- start 2023-09-13T03:12:43.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.120+0000 7fc434bff700 1 -- start start 2023-09-13T03:12:43.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.120+0000 7fc434bff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4301005f0 0x7fc430196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.120+0000 7fc434bff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4301017e0 0x7fc4301968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.120+0000 7fc434bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc430102180 0x7fc43019d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.120+0000 7fc434bff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43019dee0 con 0x7fc4301005f0 2023-09-13T03:12:43.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.120+0000 7fc434bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43019e020 con 0x7fc430102180 2023-09-13T03:12:43.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.121+0000 7fc434bff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43019e1b0 con 0x7fc4301017e0 2023-09-13T03:12:43.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.121+0000 7fc42e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4301005f0 0x7fc430196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.121+0000 7fc42e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4301005f0 0x7fc430196360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54246/0 (socket says 172.21.15.64:54246) 2023-09-13T03:12:43.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.121+0000 7fc42e59c700 1 -- 172.21.15.64:0/1522971227 learned_addr learned my addr 172.21.15.64:0/1522971227 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:43.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.121+0000 7fc42dd9b700 1 --2- 172.21.15.64:0/1522971227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4301017e0 0x7fc4301968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.121+0000 7fc42ed9d700 1 --2- 172.21.15.64:0/1522971227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc430102180 0x7fc43019d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.121+0000 7fc42e59c700 1 -- 172.21.15.64:0/1522971227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4301017e0 msgr2=0x7fc4301968a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:43.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.121+0000 7fc42e59c700 1 --2- 172.21.15.64:0/1522971227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4301017e0 0x7fc4301968a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.121+0000 7fc42e59c700 1 -- 172.21.15.64:0/1522971227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc430102180 msgr2=0x7fc43019d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:43.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.121+0000 7fc42e59c700 1 --2- 172.21.15.64:0/1522971227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc430102180 0x7fc43019d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.121+0000 7fc42e59c700 1 -- 172.21.15.64:0/1522971227 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4200095f0 con 0x7fc4301005f0 2023-09-13T03:12:43.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.121+0000 7fc42e59c700 1 --2- 172.21.15.64:0/1522971227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4301005f0 0x7fc430196360 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc420009990 tx=0x7fc42000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:43.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.122+0000 7fc41f7fe700 1 -- 172.21.15.64:0/1522971227 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42001d070 con 0x7fc4301005f0 2023-09-13T03:12:43.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.122+0000 7fc41f7fe700 1 -- 172.21.15.64:0/1522971227 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4200054f0 con 0x7fc4301005f0 2023-09-13T03:12:43.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.122+0000 7fc434bff700 1 -- 172.21.15.64:0/1522971227 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc43019e430 con 0x7fc4301005f0 2023-09-13T03:12:43.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.122+0000 7fc41f7fe700 1 -- 172.21.15.64:0/1522971227 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc420004c70 con 0x7fc4301005f0 2023-09-13T03:12:43.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.122+0000 7fc434bff700 1 -- 172.21.15.64:0/1522971227 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc43019e8b0 con 0x7fc4301005f0 2023-09-13T03:12:43.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.125+0000 7fc41f7fe700 1 -- 172.21.15.64:0/1522971227 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fc42000fc30 con 0x7fc4301005f0 2023-09-13T03:12:43.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.126+0000 7fc41f7fe700 1 --2- 172.21.15.64:0/1522971227 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc4180a06c0 0x7fc4180a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.126+0000 7fc4485c8b80 1 Processor -- start 2023-09-13T03:12:43.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.126+0000 7fc41f7fe700 1 -- 172.21.15.64:0/1522971227 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fc4200cca80 con 0x7fc4301005f0 2023-09-13T03:12:43.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.126+0000 7fc4485c8b80 1 -- start start 2023-09-13T03:12:43.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.126+0000 7fc42dd9b700 1 --2- 172.21.15.64:0/1522971227 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc4180a06c0 0x7fc4180a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.127+0000 7fc4485c8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c35b4bb0 0x55b3c35b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.127+0000 7fc4485c8b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3c35b5550 0x55b3c35b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.127+0000 7fc4485c8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3c35b3db0 0x55b3c35b4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.127+0000 7fc4485c8b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c350c6e0 con 0x55b3c35b5550 2023-09-13T03:12:43.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.127+0000 7fc4485c8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c350a000 con 0x55b3c35b4bb0 2023-09-13T03:12:43.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.127+0000 7fc4485c8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c35058f0 con 0x55b3c35b3db0 2023-09-13T03:12:43.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.127+0000 7fc42e59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c35b4bb0 0x55b3c35b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.127+0000 7fc42ed9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3c35b3db0 0x55b3c35b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.127+0000 7fc42e59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c35b4bb0 0x55b3c35b4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:53180/0 (socket says 172.21.15.64:53180) 2023-09-13T03:12:43.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.127+0000 7fc42dd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3c35b5550 0x55b3c35b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.127+0000 7fc42e59c700 1 -- 172.21.15.64:0/739626248 learned_addr learned my addr 172.21.15.64:0/739626248 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:43.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.127+0000 7fc42e59c700 1 -- 172.21.15.64:0/739626248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3c35b3db0 msgr2=0x55b3c35b4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:43.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.127+0000 7fc42e59c700 1 --2- 172.21.15.64:0/739626248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3c35b3db0 0x55b3c35b4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.127+0000 7fc42e59c700 1 -- 172.21.15.64:0/739626248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3c35b5550 msgr2=0x55b3c35b37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:43.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.127+0000 7fc42e59c700 1 --2- 172.21.15.64:0/739626248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3c35b5550 0x55b3c35b37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.127+0000 7fc42dd9b700 1 --2- 172.21.15.64:0/1522971227 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc4180a06c0 0x7fc4180a2b80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc424009c60 tx=0x7fc4240116c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:43.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.127+0000 7fc42e59c700 1 -- 172.21.15.64:0/739626248 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3c3331d90 con 0x55b3c35b4bb0 2023-09-13T03:12:43.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.127+0000 7fc42ed9d700 1 --2- 172.21.15.64:0/739626248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3c35b3db0 0x55b3c35b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:43.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.128+0000 7fc42e59c700 1 --2- 172.21.15.64:0/739626248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c35b4bb0 0x55b3c35b4fd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc420096050 tx=0x7fc420099cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=107deb4433ca2039 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:43.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.128+0000 7fc41cff9700 1 -- 172.21.15.64:0/739626248 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42009a2d0 con 0x55b3c35b4bb0 2023-09-13T03:12:43.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.128+0000 7fc41cff9700 1 -- 172.21.15.64:0/739626248 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc42009e100 con 0x55b3c35b4bb0 2023-09-13T03:12:43.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.128+0000 7fc41cff9700 1 -- 172.21.15.64:0/739626248 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4200997c0 con 0x55b3c35b4bb0 2023-09-13T03:12:43.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.129+0000 7fc4485c8b80 1 -- 172.21.15.64:0/739626248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c35b4bb0 msgr2=0x55b3c35b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:43.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.129+0000 7fc4485c8b80 1 --2- 172.21.15.64:0/739626248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c35b4bb0 0x55b3c35b4fd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc420096050 tx=0x7fc420099cb0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.129+0000 7fc4485c8b80 1 -- 172.21.15.64:0/739626248 shutdown_connections 2023-09-13T03:12:43.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.129+0000 7fc4485c8b80 1 --2- 172.21.15.64:0/739626248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c35b4bb0 0x55b3c35b4fd0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.129+0000 7fc4485c8b80 1 --2- 172.21.15.64:0/739626248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3c35b3db0 0x55b3c35b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.129+0000 7fc4485c8b80 1 --2- 172.21.15.64:0/739626248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3c35b5550 0x55b3c35b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.129+0000 7fc4485c8b80 1 -- 172.21.15.64:0/739626248 >> 172.21.15.64:0/739626248 conn(0x55b3c3528040 msgr2=0x55b3c35ae6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:43.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.129+0000 7fc4485c8b80 1 -- 172.21.15.64:0/739626248 shutdown_connections 2023-09-13T03:12:43.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.129+0000 7fc4485c8b80 1 -- 172.21.15.64:0/739626248 wait complete. 2023-09-13T03:12:43.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.129+0000 7fc4485c8b80 1 -- start start 2023-09-13T03:12:43.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.130+0000 7fc4485c8b80 1 Processor -- start 2023-09-13T03:12:43.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.130+0000 7fc4485c8b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3c35b3db0 0x55b3c35c15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.130+0000 7fc4485c8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c35b4bb0 0x55b3c35c1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.130+0000 7fc4485c8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3c35b5550 0x55b3c35c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.130+0000 7fc4485c8b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c34a3b20 con 0x55b3c35b3db0 2023-09-13T03:12:43.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.130+0000 7fc4485c8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c34a3820 con 0x55b3c35b4bb0 2023-09-13T03:12:43.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.130+0000 7fc4485c8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c33878d0 con 0x55b3c35b5550 2023-09-13T03:12:43.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.130+0000 7fc4485c8b80 10 client.0 _ll_register_callbacks cb 0x55b3c33eda30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:12:43.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.130+0000 7fc42e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3c35b3db0 0x55b3c35c15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.130+0000 7fc42e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3c35b3db0 0x55b3c35c15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54266/0 (socket says 172.21.15.64:54266) 2023-09-13T03:12:43.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.130+0000 7fc42e59c700 1 -- 172.21.15.64:0/1960079004 learned_addr learned my addr 172.21.15.64:0/1960079004 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:43.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.131+0000 7fc42dd9b700 1 --2- 172.21.15.64:0/1960079004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c35b4bb0 0x55b3c35c1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.131+0000 7fc42ed9d700 1 --2- 172.21.15.64:0/1960079004 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3c35b5550 0x55b3c35c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.131+0000 7fc42e59c700 1 -- 172.21.15.64:0/1960079004 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3c35b5550 msgr2=0x55b3c35c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:43.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.131+0000 7fc42e59c700 1 --2- 172.21.15.64:0/1960079004 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3c35b5550 0x55b3c35c8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.131+0000 7fc42e59c700 1 -- 172.21.15.64:0/1960079004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c35b4bb0 msgr2=0x55b3c35c1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:43.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.131+0000 7fc42e59c700 1 --2- 172.21.15.64:0/1960079004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c35b4bb0 0x55b3c35c1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.131+0000 7fc42e59c700 1 -- 172.21.15.64:0/1960079004 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc428009940 con 0x55b3c35b3db0 2023-09-13T03:12:43.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.131+0000 7fc42dd9b700 1 --2- 172.21.15.64:0/1960079004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c35b4bb0 0x55b3c35c1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:43.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.131+0000 7fc42e59c700 1 --2- 172.21.15.64:0/1960079004 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3c35b3db0 0x55b3c35c15f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc420096d40 tx=0x7fc4200a22b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ef900d65f8fe0d server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:43.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.131+0000 7fc415ffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:12:43.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.131+0000 7fc415ffb700 1 -- 172.21.15.64:0/1960079004 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4200999c0 con 0x55b3c35b3db0 2023-09-13T03:12:43.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.131+0000 7fc415ffb700 1 -- 172.21.15.64:0/1960079004 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fc42009ee40 con 0x55b3c35b3db0 2023-09-13T03:12:43.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.131+0000 7fc4485c8b80 1 -- 172.21.15.64:0/1960079004 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b3c345d4a0 con 0x55b3c35b3db0 2023-09-13T03:12:43.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.132+0000 7fc415ffb700 1 -- 172.21.15.64:0/1960079004 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc420098670 con 0x55b3c35b3db0 2023-09-13T03:12:43.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.132+0000 7fc4485c8b80 10 client.15261 fetch_fsmap learned FSMap version 26 2023-09-13T03:12:43.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.132+0000 7fc4485c8b80 1 -- 172.21.15.64:0/1960079004 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7fc43019e8b0 con 0x55b3c35b3db0 2023-09-13T03:12:43.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.132+0000 7fc415ffb700 1 -- 172.21.15.64:0/1960079004 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b3c345d4a0 con 0x55b3c35b3db0 2023-09-13T03:12:43.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.132+0000 7fc415ffb700 1 -- 172.21.15.64:0/1960079004 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc4200960c0 con 0x55b3c35b3db0 2023-09-13T03:12:43.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.132+0000 7fc4485c8b80 10 client.15261 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:12:43.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.132+0000 7fc4485c8b80 10 client.15261 resolve_mds: resolved name 'f' to daemon mds.f 2023-09-13T03:12:43.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.132+0000 7fc4485c8b80 20 client.15261 populate_metadata read hostname 'smithi064' 2023-09-13T03:12:43.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.132+0000 7fc4485c8b80 1 --2- 172.21.15.64:0/1960079004 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x55b3c35cc790 0x55b3c35d1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.132+0000 7fc4485c8b80 4 client.15261 mds_command: new command op to 24622 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:12:43.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.132+0000 7fc4485c8b80 1 -- 172.21.15.64:0/1960079004 --> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b3c351a850 con 0x55b3c35cc790 2023-09-13T03:12:43.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.133+0000 7fc42dd9b700 1 --2- 172.21.15.64:0/1960079004 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x55b3c35cc790 0x55b3c35d1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.133+0000 7fc415ffb700 10 client.15261 ms_handle_connect on v2:172.21.15.170:6834/1493371121 2023-09-13T03:12:43.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.133+0000 7fc42dd9b700 1 --2- 172.21.15.64:0/1960079004 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x55b3c35cc790 0x55b3c35d1730 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7d8196c1979ca310 server_cookie=ad66140d5bcac8f in_seq=0 out_seq=0 2023-09-13T03:12:43.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.134+0000 7fc415ffb700 1 -- 172.21.15.64:0/1960079004 <== mds.4 v2:172.21.15.170:6834/1493371121 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b3c351a850 con 0x55b3c35cc790 2023-09-13T03:12:43.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.134+0000 7fc415ffb700 10 client.15261 handle_command_reply: tid=0 2023-09-13T03:12:43.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.134+0000 7fc4485c8b80 1 client.15261 shutdown 2023-09-13T03:12:43.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.134+0000 7fc4485c8b80 1 -- 172.21.15.64:0/1960079004 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3c35b3db0 msgr2=0x55b3c35c15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:43.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.134+0000 7fc4485c8b80 1 --2- 172.21.15.64:0/1960079004 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3c35b3db0 0x55b3c35c15f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc420096d40 tx=0x7fc4200a22b0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.134+0000 7fc4485c8b80 1 -- 172.21.15.64:0/1960079004 shutdown_connections 2023-09-13T03:12:43.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.134+0000 7fc4485c8b80 1 --2- 172.21.15.64:0/1960079004 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x55b3c35cc790 0x55b3c35d1730 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.134+0000 7fc4485c8b80 1 --2- 172.21.15.64:0/1960079004 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3c35b5550 0x55b3c35c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.134+0000 7fc4485c8b80 1 --2- 172.21.15.64:0/1960079004 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3c35b3db0 0x55b3c35c15f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.134+0000 7fc4485c8b80 1 --2- 172.21.15.64:0/1960079004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c35b4bb0 0x55b3c35c1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.134+0000 7fc415ffb700 0 client.15261 ms_handle_reset on v2:172.21.15.170:6834/1493371121 2023-09-13T03:12:43.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.134+0000 7fc4485c8b80 1 -- 172.21.15.64:0/1960079004 >> 172.21.15.64:0/1960079004 conn(0x55b3c3528040 msgr2=0x55b3c35add70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:43.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.134+0000 7fc4485c8b80 1 -- 172.21.15.64:0/1960079004 shutdown_connections 2023-09-13T03:12:43.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.134+0000 7fc4485c8b80 1 -- 172.21.15.64:0/1960079004 wait complete. 2023-09-13T03:12:43.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.134+0000 7fc4485c8b80 20 client.15261 trim_cache size 0 max 16384 2023-09-13T03:12:43.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.153+0000 7fc4485c8b80 1 Processor -- start 2023-09-13T03:12:43.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.153+0000 7fc4485c8b80 1 -- start start 2023-09-13T03:12:43.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.153+0000 7fc4485c8b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3c3528040 0x55b3c35b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.153+0000 7fc4485c8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3c35b3db0 0x55b3c35c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.153+0000 7fc4485c8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c35b4bb0 0x55b3c35bb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.153+0000 7fc4485c8b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c35cbe60 con 0x55b3c3528040 2023-09-13T03:12:43.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.153+0000 7fc4485c8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c35c24f0 con 0x55b3c35b4bb0 2023-09-13T03:12:43.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.153+0000 7fc4485c8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c351a850 con 0x55b3c35b3db0 2023-09-13T03:12:43.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.153+0000 7fc42e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3c3528040 0x55b3c35b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.153+0000 7fc42e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3c3528040 0x55b3c35b36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54280/0 (socket says 172.21.15.64:54280) 2023-09-13T03:12:43.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.153+0000 7fc42e59c700 1 -- 172.21.15.64:0/23751158 learned_addr learned my addr 172.21.15.64:0/23751158 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:43.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.153+0000 7fc42ed9d700 1 --2- 172.21.15.64:0/23751158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c35b4bb0 0x55b3c35bb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.153+0000 7fc42dd9b700 1 --2- 172.21.15.64:0/23751158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3c35b3db0 0x55b3c35c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.154+0000 7fc42e59c700 1 -- 172.21.15.64:0/23751158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3c35b3db0 msgr2=0x55b3c35c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:43.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.154+0000 7fc42e59c700 1 --2- 172.21.15.64:0/23751158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3c35b3db0 0x55b3c35c8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.154+0000 7fc42e59c700 1 -- 172.21.15.64:0/23751158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c35b4bb0 msgr2=0x55b3c35bb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:43.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.154+0000 7fc42e59c700 1 --2- 172.21.15.64:0/23751158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c35b4bb0 0x55b3c35bb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.154+0000 7fc42e59c700 1 -- 172.21.15.64:0/23751158 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3c331ac10 con 0x55b3c3528040 2023-09-13T03:12:43.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.154+0000 7fc42dd9b700 1 --2- 172.21.15.64:0/23751158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3c35b3db0 0x55b3c35c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:12:43.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.154+0000 7fc42ed9d700 1 --2- 172.21.15.64:0/23751158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c35b4bb0 0x55b3c35bb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:43.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.154+0000 7fc42e59c700 1 --2- 172.21.15.64:0/23751158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3c3528040 0x55b3c35b36c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc42000bf40 tx=0x7fc420009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3b21ac875341327 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:43.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.154+0000 7fc41d7fa700 1 -- 172.21.15.64:0/23751158 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4200995b0 con 0x55b3c3528040 2023-09-13T03:12:43.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.154+0000 7fc41d7fa700 1 -- 172.21.15.64:0/23751158 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc42009a2d0 con 0x55b3c3528040 2023-09-13T03:12:43.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.154+0000 7fc41d7fa700 1 -- 172.21.15.64:0/23751158 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4200a1e20 con 0x55b3c3528040 2023-09-13T03:12:43.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.155+0000 7fc4485c8b80 1 -- 172.21.15.64:0/23751158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3c3528040 msgr2=0x55b3c35b36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:43.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.155+0000 7fc4485c8b80 1 --2- 172.21.15.64:0/23751158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3c3528040 0x55b3c35b36c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc42000bf40 tx=0x7fc420009310 comp rx=0 tx=0).stop 2023-09-13T03:12:43.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.155+0000 7fc4485c8b80 1 -- 172.21.15.64:0/23751158 shutdown_connections 2023-09-13T03:12:43.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.155+0000 7fc4485c8b80 1 --2- 172.21.15.64:0/23751158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c35b4bb0 0x55b3c35bb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.155+0000 7fc4485c8b80 1 --2- 172.21.15.64:0/23751158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3c3528040 0x55b3c35b36c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.155+0000 7fc4485c8b80 1 --2- 172.21.15.64:0/23751158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3c35b3db0 0x55b3c35c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.155+0000 7fc4485c8b80 1 -- 172.21.15.64:0/23751158 >> 172.21.15.64:0/23751158 conn(0x55b3c35aea00 msgr2=0x55b3c35c93a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:43.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.155+0000 7fc4485c8b80 1 -- 172.21.15.64:0/23751158 shutdown_connections 2023-09-13T03:12:43.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.155+0000 7fc4485c8b80 1 -- 172.21.15.64:0/23751158 wait complete. 2023-09-13T03:12:43.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.155+0000 7fc4485c8b80 1 -- start start 2023-09-13T03:12:43.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.155+0000 7fc4485c8b80 1 Processor -- start 2023-09-13T03:12:43.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.156+0000 7fc4485c8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c3528040 0x55b3c35c3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.156+0000 7fc4485c8b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3c35b3db0 0x55b3c35c4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.156+0000 7fc4485c8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3c35b4bb0 0x55b3c3620d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.156+0000 7fc4485c8b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c35c8d30 con 0x55b3c35b3db0 2023-09-13T03:12:43.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.156+0000 7fc4485c8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c35c8ea0 con 0x55b3c3528040 2023-09-13T03:12:43.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.156+0000 7fc4485c8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c3621260 con 0x55b3c35b4bb0 2023-09-13T03:12:43.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.156+0000 7fc4485c8b80 10 client.0 _ll_register_callbacks cb 0x7fc428009190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:12:43.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.156+0000 7fc42dd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3c35b3db0 0x55b3c35c4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.156+0000 7fc42dd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3c35b3db0 0x55b3c35c4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54294/0 (socket says 172.21.15.64:54294) 2023-09-13T03:12:43.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.156+0000 7fc42dd9b700 1 -- 172.21.15.64:0/3372034557 learned_addr learned my addr 172.21.15.64:0/3372034557 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:43.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.156+0000 7fc42e59c700 1 --2- 172.21.15.64:0/3372034557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c3528040 0x55b3c35c3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.156+0000 7fc42ed9d700 1 --2- 172.21.15.64:0/3372034557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3c35b4bb0 0x55b3c3620d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.156+0000 7fc42dd9b700 1 -- 172.21.15.64:0/3372034557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3c35b4bb0 msgr2=0x55b3c3620d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:43.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.157+0000 7fc42dd9b700 1 --2- 172.21.15.64:0/3372034557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3c35b4bb0 0x55b3c3620d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.157+0000 7fc42dd9b700 1 -- 172.21.15.64:0/3372034557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c3528040 msgr2=0x55b3c35c3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:43.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.157+0000 7fc42dd9b700 1 --2- 172.21.15.64:0/3372034557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c3528040 0x55b3c35c3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.157+0000 7fc42dd9b700 1 -- 172.21.15.64:0/3372034557 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc428009c60 con 0x55b3c35b3db0 2023-09-13T03:12:43.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.157+0000 7fc42dd9b700 1 --2- 172.21.15.64:0/3372034557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3c35b3db0 0x55b3c35c4240 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc424005d90 tx=0x7fc4240047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9b730108c8fb5ab server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:43.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.157+0000 7fc415ffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:12:43.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.157+0000 7fc415ffb700 1 -- 172.21.15.64:0/3372034557 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc424026040 con 0x55b3c35b3db0 2023-09-13T03:12:43.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.157+0000 7fc4485c8b80 1 -- 172.21.15.64:0/3372034557 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc42009e100 con 0x55b3c35b3db0 2023-09-13T03:12:43.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.157+0000 7fc415ffb700 1 -- 172.21.15.64:0/3372034557 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fc424004220 con 0x55b3c35b3db0 2023-09-13T03:12:43.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.157+0000 7fc415ffb700 1 -- 172.21.15.64:0/3372034557 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc42402a5c0 con 0x55b3c35b3db0 2023-09-13T03:12:43.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.157+0000 7fc4485c8b80 10 client.15267 fetch_fsmap learned FSMap version 26 2023-09-13T03:12:43.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.157+0000 7fc4485c8b80 1 -- 172.21.15.64:0/3372034557 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7fc43019e8b0 con 0x55b3c35b3db0 2023-09-13T03:12:43.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.158+0000 7fc415ffb700 1 -- 172.21.15.64:0/3372034557 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc42009e100 con 0x55b3c35b3db0 2023-09-13T03:12:43.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.158+0000 7fc415ffb700 1 -- 172.21.15.64:0/3372034557 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc424029030 con 0x55b3c35b3db0 2023-09-13T03:12:43.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.158+0000 7fc4485c8b80 10 client.15267 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:12:43.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.158+0000 7fc4485c8b80 10 client.15267 resolve_mds: resolved name 'f' to daemon mds.f 2023-09-13T03:12:43.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.158+0000 7fc4485c8b80 20 client.15267 populate_metadata read hostname 'smithi064' 2023-09-13T03:12:43.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.158+0000 7fc4485c8b80 1 --2- 172.21.15.64:0/3372034557 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x55b3c35b5550 0x55b3c36229c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.158+0000 7fc4485c8b80 4 client.15267 mds_command: new command op to 24622 tid=0[{"prefix": "status"}] 2023-09-13T03:12:43.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.158+0000 7fc4485c8b80 1 -- 172.21.15.64:0/3372034557 --> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b3c34fe8d0 con 0x55b3c35b5550 2023-09-13T03:12:43.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.158+0000 7fc42e59c700 1 --2- 172.21.15.64:0/3372034557 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x55b3c35b5550 0x55b3c36229c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.158+0000 7fc42e59c700 1 --2- 172.21.15.64:0/3372034557 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x55b3c35b5550 0x55b3c36229c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=28a7845479242aed server_cookie=d713489189b8dba9 in_seq=0 out_seq=0 2023-09-13T03:12:43.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.158+0000 7fc415ffb700 10 client.15267 ms_handle_connect on v2:172.21.15.170:6834/1493371121 2023-09-13T03:12:43.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.159+0000 7fc415ffb700 1 -- 172.21.15.64:0/3372034557 <== mds.4 v2:172.21.15.170:6834/1493371121 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55b3c34fe8d0 con 0x55b3c35b5550 2023-09-13T03:12:43.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.159+0000 7fc415ffb700 10 client.15267 handle_command_reply: tid=0 2023-09-13T03:12:43.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.159+0000 7fc4485c8b80 1 client.15267 shutdown 2023-09-13T03:12:43.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.159+0000 7fc4485c8b80 1 -- 172.21.15.64:0/3372034557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3c35b3db0 msgr2=0x55b3c35c4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:43.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.159+0000 7fc4485c8b80 1 --2- 172.21.15.64:0/3372034557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3c35b3db0 0x55b3c35c4240 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc424005d90 tx=0x7fc4240047c0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.159+0000 7fc4485c8b80 1 -- 172.21.15.64:0/3372034557 shutdown_connections 2023-09-13T03:12:43.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.159+0000 7fc4485c8b80 1 --2- 172.21.15.64:0/3372034557 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x55b3c35b5550 0x55b3c36229c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.159+0000 7fc4485c8b80 1 --2- 172.21.15.64:0/3372034557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c3528040 0x55b3c35c3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.159+0000 7fc4485c8b80 1 --2- 172.21.15.64:0/3372034557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3c35b4bb0 0x55b3c3620d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.159+0000 7fc4485c8b80 1 --2- 172.21.15.64:0/3372034557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b3c35b3db0 0x55b3c35c4240 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.159+0000 7fc4485c8b80 1 -- 172.21.15.64:0/3372034557 >> 172.21.15.64:0/3372034557 conn(0x55b3c35aea00 msgr2=0x55b3c35bb360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:43.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.159+0000 7fc415ffb700 0 client.15267 ms_handle_reset on v2:172.21.15.170:6834/1493371121 2023-09-13T03:12:43.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.159+0000 7fc4485c8b80 1 -- 172.21.15.64:0/3372034557 shutdown_connections 2023-09-13T03:12:43.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.159+0000 7fc4485c8b80 1 -- 172.21.15.64:0/3372034557 wait complete. 2023-09-13T03:12:43.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.159+0000 7fc4485c8b80 20 client.15267 trim_cache size 0 max 16384 2023-09-13T03:12:43.175 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-13T03:12:43.175 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "f8dbced4-51e1-11ee-9ab7-7b867c8bd7da", 2023-09-13T03:12:43.175 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 4, 2023-09-13T03:12:43.175 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24622, 2023-09-13T03:12:43.175 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2023-09-13T03:12:43.176 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2023-09-13T03:12:43.176 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2023-09-13T03:12:43.176 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 85.935349711000001, 2023-09-13T03:12:43.176 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 26, 2023-09-13T03:12:43.176 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 85, 2023-09-13T03:12:43.177 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 83, 2023-09-13T03:12:43.177 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 114.65844713200001 2023-09-13T03:12:43.177 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-13T03:12:43.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.160+0000 7fc434bff700 1 -- 172.21.15.64:0/1522971227 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc4180a06c0 msgr2=0x7fc4180a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:43.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.160+0000 7fc434bff700 1 --2- 172.21.15.64:0/1522971227 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc4180a06c0 0x7fc4180a2b80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc424009c60 tx=0x7fc4240116c0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.160+0000 7fc434bff700 1 -- 172.21.15.64:0/1522971227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4301005f0 msgr2=0x7fc430196360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:43.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.160+0000 7fc434bff700 1 --2- 172.21.15.64:0/1522971227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4301005f0 0x7fc430196360 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc420009990 tx=0x7fc42000f740 comp rx=0 tx=0).stop 2023-09-13T03:12:43.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.160+0000 7fc434bff700 1 -- 172.21.15.64:0/1522971227 shutdown_connections 2023-09-13T03:12:43.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.160+0000 7fc434bff700 1 --2- 172.21.15.64:0/1522971227 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc4180a06c0 0x7fc4180a2b80 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.160+0000 7fc434bff700 1 --2- 172.21.15.64:0/1522971227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc430102180 0x7fc43019d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.160+0000 7fc434bff700 1 --2- 172.21.15.64:0/1522971227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4301005f0 0x7fc430196360 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.160+0000 7fc434bff700 1 --2- 172.21.15.64:0/1522971227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4301017e0 0x7fc4301968a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.160+0000 7fc434bff700 1 -- 172.21.15.64:0/1522971227 >> 172.21.15.64:0/1522971227 conn(0x7fc4300fbb40 msgr2=0x7fc4300fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:43.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.160+0000 7fc434bff700 1 -- 172.21.15.64:0/1522971227 shutdown_connections 2023-09-13T03:12:43.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.160+0000 7fc434bff700 1 -- 172.21.15.64:0/1522971227 wait complete. 2023-09-13T03:12:43.181 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-09-13T03:12:43.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.384+0000 7f2bff9ff700 1 -- 172.21.15.64:0/2097652388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bf8102180 msgr2=0x7f2bf8108a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:43.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.384+0000 7f2bff9ff700 1 --2- 172.21.15.64:0/2097652388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bf8102180 0x7f2bf8108a30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2bf000b0d0 tx=0x7f2bf000b3e0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.384+0000 7f2bff9ff700 1 -- 172.21.15.64:0/2097652388 shutdown_connections 2023-09-13T03:12:43.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.384+0000 7f2bff9ff700 1 --2- 172.21.15.64:0/2097652388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bf8102180 0x7f2bf8108a30 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.384+0000 7f2bff9ff700 1 --2- 172.21.15.64:0/2097652388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2bf81005f0 0x7f2bf8100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.384+0000 7f2bff9ff700 1 --2- 172.21.15.64:0/2097652388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bf81017e0 0x7f2bf8101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.384+0000 7f2bff9ff700 1 -- 172.21.15.64:0/2097652388 >> 172.21.15.64:0/2097652388 conn(0x7f2bf80fbb40 msgr2=0x7f2bf80fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:43.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.384+0000 7f2bff9ff700 1 -- 172.21.15.64:0/2097652388 shutdown_connections 2023-09-13T03:12:43.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.384+0000 7f2bff9ff700 1 -- 172.21.15.64:0/2097652388 wait complete. 2023-09-13T03:12:43.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.385+0000 7f2bff9ff700 1 Processor -- start 2023-09-13T03:12:43.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.385+0000 7f2bff9ff700 1 -- start start 2023-09-13T03:12:43.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.386+0000 7f2bff9ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bf81005f0 0x7f2bf8196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.386+0000 7f2bff9ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bf81017e0 0x7f2bf81968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.386+0000 7f2bff9ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2bf8102180 0x7f2bf819d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.386+0000 7f2bff9ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf819df10 con 0x7f2bf81017e0 2023-09-13T03:12:43.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.386+0000 7f2bff9ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf819e050 con 0x7f2bf81005f0 2023-09-13T03:12:43.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.386+0000 7f2bff9ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf819e1e0 con 0x7f2bf8102180 2023-09-13T03:12:43.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.386+0000 7f2bfcf9a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bf81017e0 0x7f2bf81968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.386+0000 7f2bfcf9a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bf81017e0 0x7f2bf81968d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54318/0 (socket says 172.21.15.64:54318) 2023-09-13T03:12:43.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.386+0000 7f2bfcf9a700 1 -- 172.21.15.64:0/2311613772 learned_addr learned my addr 172.21.15.64:0/2311613772 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:43.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.386+0000 7f2bfd79b700 1 --2- 172.21.15.64:0/2311613772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bf81005f0 0x7f2bf8196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.386+0000 7f2bfdf9c700 1 --2- 172.21.15.64:0/2311613772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2bf8102180 0x7f2bf819d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.386+0000 7f2bfcf9a700 1 -- 172.21.15.64:0/2311613772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2bf8102180 msgr2=0x7f2bf819d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:43.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.386+0000 7f2bfcf9a700 1 --2- 172.21.15.64:0/2311613772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2bf8102180 0x7f2bf819d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.386+0000 7f2bfcf9a700 1 -- 172.21.15.64:0/2311613772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bf81005f0 msgr2=0x7f2bf8196390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:43.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.386+0000 7f2bfcf9a700 1 --2- 172.21.15.64:0/2311613772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bf81005f0 0x7f2bf8196390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.386+0000 7f2bfcf9a700 1 -- 172.21.15.64:0/2311613772 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf0009d00 con 0x7f2bf81017e0 2023-09-13T03:12:43.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.387+0000 7f2bfcf9a700 1 --2- 172.21.15.64:0/2311613772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bf81017e0 0x7f2bf81968d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2be800e9a0 tx=0x7f2be800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:43.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.387+0000 7f2bee7fc700 1 -- 172.21.15.64:0/2311613772 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be8009790 con 0x7f2bf81017e0 2023-09-13T03:12:43.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.387+0000 7f2bee7fc700 1 -- 172.21.15.64:0/2311613772 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2be8010430 con 0x7f2bf81017e0 2023-09-13T03:12:43.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.387+0000 7f2bff9ff700 1 -- 172.21.15.64:0/2311613772 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bf819e4c0 con 0x7f2bf81017e0 2023-09-13T03:12:43.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.387+0000 7f2bee7fc700 1 -- 172.21.15.64:0/2311613772 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be8018680 con 0x7f2bf81017e0 2023-09-13T03:12:43.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.387+0000 7f2bff9ff700 1 -- 172.21.15.64:0/2311613772 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bf819e9d0 con 0x7f2bf81017e0 2023-09-13T03:12:43.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.392+0000 7f2bee7fc700 1 -- 172.21.15.64:0/2311613772 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f2be80105a0 con 0x7f2bf81017e0 2023-09-13T03:12:43.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.393+0000 7f2c1332db80 1 Processor -- start 2023-09-13T03:12:43.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.393+0000 7f2c1332db80 1 -- start start 2023-09-13T03:12:43.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.393+0000 7f2c1332db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f3a7f4020 0x562f3a7f4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.393+0000 7f2c1332db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f3a7f49a0 0x562f3a7f2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.393+0000 7f2c1332db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f3a7f3190 0x562f3a7f3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.393+0000 7f2c1332db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f3a7476e0 con 0x562f3a7f49a0 2023-09-13T03:12:43.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.393+0000 7f2c1332db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f3a745000 con 0x562f3a7f3190 2023-09-13T03:12:43.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.393+0000 7f2c1332db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f3a7408f0 con 0x562f3a7f4020 2023-09-13T03:12:43.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.393+0000 7f2bee7fc700 1 --2- 172.21.15.64:0/2311613772 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f2be40a0640 0x7f2be40a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.394+0000 7f2bee7fc700 1 -- 172.21.15.64:0/2311613772 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f2be8014070 con 0x7f2bf81017e0 2023-09-13T03:12:43.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.394+0000 7f2bfdf9c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f3a7f4020 0x562f3a7f4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.394+0000 7f2bfcf9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f3a7f3190 0x562f3a7f3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.394+0000 7f2bfd79b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f3a7f49a0 0x562f3a7f2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.394+0000 7f2bfcf9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f3a7f3190 0x562f3a7f3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:53236/0 (socket says 172.21.15.64:53236) 2023-09-13T03:12:43.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.394+0000 7f2bfdf9c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f3a7f4020 0x562f3a7f4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:50790/0 (socket says 172.21.15.64:50790) 2023-09-13T03:12:43.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.394+0000 7f2bfcf9a700 1 -- 172.21.15.64:0/1349404153 learned_addr learned my addr 172.21.15.64:0/1349404153 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:43.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.394+0000 7f2bfdf9c700 1 --2- 172.21.15.64:0/2311613772 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f2be40a0640 0x7f2be40a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.394+0000 7f2bfcf9a700 1 -- 172.21.15.64:0/1349404153 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f3a7f4020 msgr2=0x562f3a7f4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:43.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.394+0000 7f2bfcf9a700 1 --2- 172.21.15.64:0/1349404153 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f3a7f4020 0x562f3a7f4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.394+0000 7f2bfcf9a700 1 -- 172.21.15.64:0/1349404153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f3a7f49a0 msgr2=0x562f3a7f2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:43.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.394+0000 7f2bfcf9a700 1 --2- 172.21.15.64:0/1349404153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f3a7f49a0 0x562f3a7f2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.394+0000 7f2bfcf9a700 1 -- 172.21.15.64:0/1349404153 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f3a56cd90 con 0x562f3a7f3190 2023-09-13T03:12:43.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.394+0000 7f2bfdf9c700 1 --2- 172.21.15.64:0/1349404153 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f3a7f4020 0x562f3a7f4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:12:43.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.394+0000 7f2bfcf9a700 1 --2- 172.21.15.64:0/1349404153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f3a7f3190 0x562f3a7f3610 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2be80a1070 tx=0x7f2be8099300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a97e24868b6f9d server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:43.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.395+0000 7f2be37fe700 1 -- 172.21.15.64:0/1349404153 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be8099c30 con 0x562f3a7f3190 2023-09-13T03:12:43.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.395+0000 7f2be37fe700 1 -- 172.21.15.64:0/1349404153 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2be8095050 con 0x562f3a7f3190 2023-09-13T03:12:43.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.395+0000 7f2bfdf9c700 1 --2- 172.21.15.64:0/2311613772 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f2be40a0640 0x7f2be40a2b00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0006110 tx=0x7f2bf000b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:43.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.395+0000 7f2be37fe700 1 -- 172.21.15.64:0/1349404153 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be8010880 con 0x562f3a7f3190 2023-09-13T03:12:43.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.395+0000 7f2c1332db80 1 -- 172.21.15.64:0/1349404153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f3a7f3190 msgr2=0x562f3a7f3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:43.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.395+0000 7f2c1332db80 1 --2- 172.21.15.64:0/1349404153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f3a7f3190 0x562f3a7f3610 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2be80a1070 tx=0x7f2be8099300 comp rx=0 tx=0).stop 2023-09-13T03:12:43.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.395+0000 7f2c1332db80 1 -- 172.21.15.64:0/1349404153 shutdown_connections 2023-09-13T03:12:43.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.395+0000 7f2c1332db80 1 --2- 172.21.15.64:0/1349404153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f3a7f3190 0x562f3a7f3610 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.395+0000 7f2c1332db80 1 --2- 172.21.15.64:0/1349404153 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f3a7f4020 0x562f3a7f4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.395+0000 7f2c1332db80 1 --2- 172.21.15.64:0/1349404153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f3a7f49a0 0x562f3a7f2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.395+0000 7f2c1332db80 1 -- 172.21.15.64:0/1349404153 >> 172.21.15.64:0/1349404153 conn(0x562f3a763040 msgr2=0x562f3a7e96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:43.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.395+0000 7f2c1332db80 1 -- 172.21.15.64:0/1349404153 shutdown_connections 2023-09-13T03:12:43.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.396+0000 7f2c1332db80 1 -- 172.21.15.64:0/1349404153 wait complete. 2023-09-13T03:12:43.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.396+0000 7f2c1332db80 1 -- start start 2023-09-13T03:12:43.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.396+0000 7f2c1332db80 1 Processor -- start 2023-09-13T03:12:43.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.397+0000 7f2c1332db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f3a7f3190 0x562f3a7fc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.397+0000 7f2c1332db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f3a7f4020 0x562f3a7fcf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.397+0000 7f2c1332db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f3a7f49a0 0x562f3a803f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.397+0000 7f2c1332db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f3a6deb20 con 0x562f3a7f3190 2023-09-13T03:12:43.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.397+0000 7f2c1332db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f3a6de820 con 0x562f3a7f4020 2023-09-13T03:12:43.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.397+0000 7f2c1332db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f3a5c28d0 con 0x562f3a7f49a0 2023-09-13T03:12:43.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.397+0000 7f2c1332db80 10 client.0 _ll_register_callbacks cb 0x562f3a628a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:12:43.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.397+0000 7f2bfd79b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f3a7f3190 0x562f3a7fc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.397+0000 7f2bfd79b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f3a7f3190 0x562f3a7fc9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54334/0 (socket says 172.21.15.64:54334) 2023-09-13T03:12:43.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.397+0000 7f2bfd79b700 1 -- 172.21.15.64:0/1094451421 learned_addr learned my addr 172.21.15.64:0/1094451421 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:43.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.397+0000 7f2bfcf9a700 1 --2- 172.21.15.64:0/1094451421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f3a7f4020 0x562f3a7fcf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.397+0000 7f2bfdf9c700 1 --2- 172.21.15.64:0/1094451421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f3a7f49a0 0x562f3a803f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.397+0000 7f2bfd79b700 1 -- 172.21.15.64:0/1094451421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f3a7f49a0 msgr2=0x562f3a803f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:43.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.397+0000 7f2bfd79b700 1 --2- 172.21.15.64:0/1094451421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f3a7f49a0 0x562f3a803f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.397+0000 7f2bfd79b700 1 -- 172.21.15.64:0/1094451421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f3a7f4020 msgr2=0x562f3a7fcf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:43.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.397+0000 7f2bfd79b700 1 --2- 172.21.15.64:0/1094451421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f3a7f4020 0x562f3a7fcf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.397+0000 7f2bfd79b700 1 -- 172.21.15.64:0/1094451421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bf819e9d0 con 0x562f3a7f3190 2023-09-13T03:12:43.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.398+0000 7f2bfd79b700 1 --2- 172.21.15.64:0/1094451421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f3a7f3190 0x562f3a7fc9d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2bf40126d0 tx=0x7f2bf40129e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86aa93e336fe3e4b server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:43.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.398+0000 7f2be0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:12:43.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.398+0000 7f2be0ff9700 1 -- 172.21.15.64:0/1094451421 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf4017070 con 0x562f3a7f3190 2023-09-13T03:12:43.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.398+0000 7f2be0ff9700 1 -- 172.21.15.64:0/1094451421 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f2bf4004050 con 0x562f3a7f3190 2023-09-13T03:12:43.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.398+0000 7f2c1332db80 1 -- 172.21.15.64:0/1094451421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562f3a6984a0 con 0x562f3a7f3190 2023-09-13T03:12:43.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.398+0000 7f2be0ff9700 1 -- 172.21.15.64:0/1094451421 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2bf4004210 con 0x562f3a7f3190 2023-09-13T03:12:43.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.398+0000 7f2c1332db80 10 client.15279 fetch_fsmap learned FSMap version 26 2023-09-13T03:12:43.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.398+0000 7f2c1332db80 1 -- 172.21.15.64:0/1094451421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x562f3a555c10 con 0x562f3a7f3190 2023-09-13T03:12:43.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.398+0000 7f2be0ff9700 1 -- 172.21.15.64:0/1094451421 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562f3a6984a0 con 0x562f3a7f3190 2023-09-13T03:12:43.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.399+0000 7f2be0ff9700 1 -- 172.21.15.64:0/1094451421 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2bf4010450 con 0x562f3a7f3190 2023-09-13T03:12:43.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.399+0000 7f2c1332db80 10 client.15279 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:12:43.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.399+0000 7f2c1332db80 10 client.15279 resolve_mds: resolved name 'c' to daemon mds.c 2023-09-13T03:12:43.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.399+0000 7f2c1332db80 20 client.15279 populate_metadata read hostname 'smithi064' 2023-09-13T03:12:43.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.399+0000 7f2c1332db80 1 --2- 172.21.15.64:0/1094451421 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x562f3a8080f0 0x562f3a8070b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.399+0000 7f2c1332db80 4 client.15279 mds_command: new command op to 24634 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:12:43.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.399+0000 7f2c1332db80 1 -- 172.21.15.64:0/1094451421 --> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562f3a755850 con 0x562f3a8080f0 2023-09-13T03:12:43.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.399+0000 7f2bfcf9a700 1 --2- 172.21.15.64:0/1094451421 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x562f3a8080f0 0x562f3a8070b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.400+0000 7f2bfcf9a700 1 --2- 172.21.15.64:0/1094451421 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x562f3a8080f0 0x562f3a8070b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=64d1a6f26d0a0942 server_cookie=d893af6ead79f1bb in_seq=0 out_seq=0 2023-09-13T03:12:43.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.400+0000 7f2be0ff9700 10 client.15279 ms_handle_connect on v2:172.21.15.170:6836/1726587275 2023-09-13T03:12:43.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.400+0000 7f2be0ff9700 1 -- 172.21.15.64:0/1094451421 <== mds.3 v2:172.21.15.170:6836/1726587275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562f3a755850 con 0x562f3a8080f0 2023-09-13T03:12:43.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.401+0000 7f2be0ff9700 10 client.15279 handle_command_reply: tid=0 2023-09-13T03:12:43.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.401+0000 7f2c1332db80 1 client.15279 shutdown 2023-09-13T03:12:43.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.401+0000 7f2c1332db80 1 -- 172.21.15.64:0/1094451421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f3a7f3190 msgr2=0x562f3a7fc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:43.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.401+0000 7f2c1332db80 1 --2- 172.21.15.64:0/1094451421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f3a7f3190 0x562f3a7fc9d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2bf40126d0 tx=0x7f2bf40129e0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.401+0000 7f2c1332db80 1 -- 172.21.15.64:0/1094451421 shutdown_connections 2023-09-13T03:12:43.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.401+0000 7f2c1332db80 1 --2- 172.21.15.64:0/1094451421 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x562f3a8080f0 0x562f3a8070b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.401+0000 7f2c1332db80 1 --2- 172.21.15.64:0/1094451421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f3a7f49a0 0x562f3a803f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.401+0000 7f2be0ff9700 0 client.15279 ms_handle_reset on v2:172.21.15.170:6836/1726587275 2023-09-13T03:12:43.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.401+0000 7f2c1332db80 1 --2- 172.21.15.64:0/1094451421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f3a7f3190 0x562f3a7fc9d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.401+0000 7f2c1332db80 1 --2- 172.21.15.64:0/1094451421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f3a7f4020 0x562f3a7fcf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.401+0000 7f2c1332db80 1 -- 172.21.15.64:0/1094451421 >> 172.21.15.64:0/1094451421 conn(0x562f3a763040 msgr2=0x562f3a7e8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:43.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.401+0000 7f2c1332db80 1 -- 172.21.15.64:0/1094451421 shutdown_connections 2023-09-13T03:12:43.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.401+0000 7f2c1332db80 1 -- 172.21.15.64:0/1094451421 wait complete. 2023-09-13T03:12:43.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.401+0000 7f2c1332db80 20 client.15279 trim_cache size 0 max 16384 2023-09-13T03:12:43.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.419+0000 7f2c1332db80 1 Processor -- start 2023-09-13T03:12:43.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.419+0000 7f2c1332db80 1 -- start start 2023-09-13T03:12:43.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.419+0000 7f2c1332db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f3a7f3190 0x562f3a7fb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.419+0000 7f2c1332db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f3a7f4020 0x562f3a803f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.419+0000 7f2c1332db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f3a7f49a0 0x562f3a7fa7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.419+0000 7f2c1332db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f3a5cede0 con 0x562f3a7f3190 2023-09-13T03:12:43.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.419+0000 7f2c1332db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f3a709f70 con 0x562f3a7f49a0 2023-09-13T03:12:43.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.419+0000 7f2c1332db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f3a6fced0 con 0x562f3a7f4020 2023-09-13T03:12:43.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.420+0000 7f2bfd79b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f3a7f3190 0x562f3a7fb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.420+0000 7f2bfd79b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f3a7f3190 0x562f3a7fb670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54346/0 (socket says 172.21.15.64:54346) 2023-09-13T03:12:43.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.420+0000 7f2bfd79b700 1 -- 172.21.15.64:0/3253922482 learned_addr learned my addr 172.21.15.64:0/3253922482 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:43.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.420+0000 7f2bfdf9c700 1 --2- 172.21.15.64:0/3253922482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f3a7f49a0 0x562f3a7fa7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.420+0000 7f2bfcf9a700 1 --2- 172.21.15.64:0/3253922482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f3a7f4020 0x562f3a803f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.420+0000 7f2bfd79b700 1 -- 172.21.15.64:0/3253922482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f3a7f4020 msgr2=0x562f3a803f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:43.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.420+0000 7f2bfd79b700 1 --2- 172.21.15.64:0/3253922482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f3a7f4020 0x562f3a803f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.420+0000 7f2bfd79b700 1 -- 172.21.15.64:0/3253922482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f3a7f49a0 msgr2=0x562f3a7fa7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:43.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.420+0000 7f2bfd79b700 1 --2- 172.21.15.64:0/3253922482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f3a7f49a0 0x562f3a7fa7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.420+0000 7f2bfd79b700 1 -- 172.21.15.64:0/3253922482 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f3a7f2da0 con 0x562f3a7f3190 2023-09-13T03:12:43.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.420+0000 7f2bfd79b700 1 --2- 172.21.15.64:0/3253922482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f3a7f3190 0x562f3a7fb670 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4012420 tx=0x7f2bf4012e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82195b9615348c5a server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:43.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.421+0000 7f2be3fff700 1 -- 172.21.15.64:0/3253922482 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf4017040 con 0x562f3a7f3190 2023-09-13T03:12:43.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.421+0000 7f2be3fff700 1 -- 172.21.15.64:0/3253922482 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2bf4010a20 con 0x562f3a7f3190 2023-09-13T03:12:43.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.421+0000 7f2be3fff700 1 -- 172.21.15.64:0/3253922482 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf40038c0 con 0x562f3a7f3190 2023-09-13T03:12:43.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.421+0000 7f2c1332db80 1 -- 172.21.15.64:0/3253922482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f3a7f3190 msgr2=0x562f3a7fb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:43.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.421+0000 7f2c1332db80 1 --2- 172.21.15.64:0/3253922482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f3a7f3190 0x562f3a7fb670 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4012420 tx=0x7f2bf4012e60 comp rx=0 tx=0).stop 2023-09-13T03:12:43.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.421+0000 7f2c1332db80 1 -- 172.21.15.64:0/3253922482 shutdown_connections 2023-09-13T03:12:43.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.421+0000 7f2c1332db80 1 --2- 172.21.15.64:0/3253922482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f3a7f49a0 0x562f3a7fa7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.421+0000 7f2c1332db80 1 --2- 172.21.15.64:0/3253922482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f3a7f3190 0x562f3a7fb670 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.421+0000 7f2c1332db80 1 --2- 172.21.15.64:0/3253922482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f3a7f4020 0x562f3a803f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.421+0000 7f2c1332db80 1 -- 172.21.15.64:0/3253922482 >> 172.21.15.64:0/3253922482 conn(0x562f3a763040 msgr2=0x562f3a8070b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:43.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.421+0000 7f2c1332db80 1 -- 172.21.15.64:0/3253922482 shutdown_connections 2023-09-13T03:12:43.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.421+0000 7f2c1332db80 1 -- 172.21.15.64:0/3253922482 wait complete. 2023-09-13T03:12:43.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.422+0000 7f2c1332db80 1 -- start start 2023-09-13T03:12:43.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.422+0000 7f2c1332db80 1 Processor -- start 2023-09-13T03:12:43.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.422+0000 7f2c1332db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f3a7f3190 0x562f3a7ff0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.422+0000 7f2c1332db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f3a7f4020 0x562f3a7ff5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.422+0000 7f2c1332db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f3a7f49a0 0x562f3a802600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.422+0000 7f2c1332db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f3a802b40 con 0x562f3a7f49a0 2023-09-13T03:12:43.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.422+0000 7f2c1332db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f3a802cb0 con 0x562f3a7f3190 2023-09-13T03:12:43.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.422+0000 7f2c1332db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f3a802e70 con 0x562f3a7f4020 2023-09-13T03:12:43.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.422+0000 7f2c1332db80 10 client.0 _ll_register_callbacks cb 0x562f3a628a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:12:43.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.423+0000 7f2bfd79b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f3a7f3190 0x562f3a7ff0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.423+0000 7f2bfd79b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f3a7f3190 0x562f3a7ff0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:53256/0 (socket says 172.21.15.64:53256) 2023-09-13T03:12:43.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.423+0000 7f2bfdf9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f3a7f49a0 0x562f3a802600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.423+0000 7f2bfd79b700 1 -- 172.21.15.64:0/782606683 learned_addr learned my addr 172.21.15.64:0/782606683 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:43.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.423+0000 7f2bfcf9a700 1 --2- 172.21.15.64:0/782606683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f3a7f4020 0x562f3a7ff5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.423+0000 7f2bfd79b700 1 -- 172.21.15.64:0/782606683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f3a7f4020 msgr2=0x562f3a7ff5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:43.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.423+0000 7f2bfd79b700 1 --2- 172.21.15.64:0/782606683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f3a7f4020 0x562f3a7ff5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.423+0000 7f2bfd79b700 1 -- 172.21.15.64:0/782606683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f3a7f49a0 msgr2=0x562f3a802600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:43.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.423+0000 7f2bfd79b700 1 --2- 172.21.15.64:0/782606683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f3a7f49a0 0x562f3a802600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.423+0000 7f2bfd79b700 1 -- 172.21.15.64:0/782606683 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562f3a555c10 con 0x562f3a7f3190 2023-09-13T03:12:43.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.423+0000 7f2bfdf9c700 1 --2- 172.21.15.64:0/782606683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f3a7f49a0 0x562f3a802600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:43.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.423+0000 7f2bfd79b700 1 --2- 172.21.15.64:0/782606683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f3a7f3190 0x562f3a7ff0a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2bf40107b0 tx=0x7f2bf4005300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=497f3b2eda117a93 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:43.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.423+0000 7f2be0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:12:43.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.424+0000 7f2be0ff9700 1 -- 172.21.15.64:0/782606683 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf4017070 con 0x562f3a7f3190 2023-09-13T03:12:43.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.424+0000 7f2c1332db80 1 -- 172.21.15.64:0/782606683 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2be8095050 con 0x562f3a7f3190 2023-09-13T03:12:43.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.424+0000 7f2be0ff9700 1 -- 172.21.15.64:0/782606683 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f2bf4003880 con 0x562f3a7f3190 2023-09-13T03:12:43.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.424+0000 7f2be0ff9700 1 -- 172.21.15.64:0/782606683 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2bf4009340 con 0x562f3a7f3190 2023-09-13T03:12:43.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.424+0000 7f2c1332db80 10 client.25307 fetch_fsmap learned FSMap version 26 2023-09-13T03:12:43.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.424+0000 7f2c1332db80 1 -- 172.21.15.64:0/782606683 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x562f3a593200 con 0x562f3a7f3190 2023-09-13T03:12:43.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.424+0000 7f2be0ff9700 1 -- 172.21.15.64:0/782606683 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2be8095050 con 0x562f3a7f3190 2023-09-13T03:12:43.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.425+0000 7f2be0ff9700 1 -- 172.21.15.64:0/782606683 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2bf4027ac0 con 0x562f3a7f3190 2023-09-13T03:12:43.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.425+0000 7f2c1332db80 10 client.25307 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:12:43.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.425+0000 7f2c1332db80 10 client.25307 resolve_mds: resolved name 'c' to daemon mds.c 2023-09-13T03:12:43.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.425+0000 7f2c1332db80 20 client.25307 populate_metadata read hostname 'smithi064' 2023-09-13T03:12:43.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.425+0000 7f2c1332db80 1 --2- 172.21.15.64:0/782606683 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x562f3a861a50 0x562f3a85e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.425+0000 7f2c1332db80 4 client.25307 mds_command: new command op to 24634 tid=0[{"prefix": "status"}] 2023-09-13T03:12:43.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.425+0000 7f2c1332db80 1 -- 172.21.15.64:0/782606683 --> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562f3a7398d0 con 0x562f3a861a50 2023-09-13T03:12:43.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.425+0000 7f2bfcf9a700 1 --2- 172.21.15.64:0/782606683 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x562f3a861a50 0x562f3a85e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.425+0000 7f2bfcf9a700 1 --2- 172.21.15.64:0/782606683 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x562f3a861a50 0x562f3a85e150 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=57c8541162eea233 server_cookie=9ea33481df0e21d8 in_seq=0 out_seq=0 2023-09-13T03:12:43.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.425+0000 7f2be0ff9700 10 client.25307 ms_handle_connect on v2:172.21.15.170:6836/1726587275 2023-09-13T03:12:43.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.426+0000 7f2be0ff9700 1 -- 172.21.15.64:0/782606683 <== mds.3 v2:172.21.15.170:6836/1726587275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x562f3a7398d0 con 0x562f3a861a50 2023-09-13T03:12:43.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.426+0000 7f2be0ff9700 10 client.25307 handle_command_reply: tid=0 2023-09-13T03:12:43.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.426+0000 7f2c1332db80 1 client.25307 shutdown 2023-09-13T03:12:43.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.426+0000 7f2c1332db80 1 -- 172.21.15.64:0/782606683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f3a7f3190 msgr2=0x562f3a7ff0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:43.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.426+0000 7f2c1332db80 1 --2- 172.21.15.64:0/782606683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f3a7f3190 0x562f3a7ff0a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2bf40107b0 tx=0x7f2bf4005300 comp rx=0 tx=0).stop 2023-09-13T03:12:43.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.426+0000 7f2c1332db80 1 -- 172.21.15.64:0/782606683 shutdown_connections 2023-09-13T03:12:43.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.426+0000 7f2c1332db80 1 --2- 172.21.15.64:0/782606683 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x562f3a861a50 0x562f3a85e150 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.426+0000 7f2c1332db80 1 --2- 172.21.15.64:0/782606683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f3a7f3190 0x562f3a7ff0a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.426+0000 7f2c1332db80 1 --2- 172.21.15.64:0/782606683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f3a7f49a0 0x562f3a802600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.426+0000 7f2c1332db80 1 --2- 172.21.15.64:0/782606683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f3a7f4020 0x562f3a7ff5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.426+0000 7f2be0ff9700 0 client.25307 ms_handle_reset on v2:172.21.15.170:6836/1726587275 2023-09-13T03:12:43.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.426+0000 7f2c1332db80 1 -- 172.21.15.64:0/782606683 >> 172.21.15.64:0/782606683 conn(0x562f3a763040 msgr2=0x562f3a7fa7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:43.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.426+0000 7f2c1332db80 1 -- 172.21.15.64:0/782606683 shutdown_connections 2023-09-13T03:12:43.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.426+0000 7f2c1332db80 1 -- 172.21.15.64:0/782606683 wait complete. 2023-09-13T03:12:43.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.426+0000 7f2c1332db80 20 client.25307 trim_cache size 0 max 16384 2023-09-13T03:12:43.442 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-13T03:12:43.443 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "f8dbced4-51e1-11ee-9ab7-7b867c8bd7da", 2023-09-13T03:12:43.443 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 3, 2023-09-13T03:12:43.443 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24634, 2023-09-13T03:12:43.443 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2023-09-13T03:12:43.444 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2023-09-13T03:12:43.444 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2023-09-13T03:12:43.444 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 87.216413454999994, 2023-09-13T03:12:43.444 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 26, 2023-09-13T03:12:43.444 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 85, 2023-09-13T03:12:43.445 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 83, 2023-09-13T03:12:43.445 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 112.67674682099999 2023-09-13T03:12:43.445 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-13T03:12:43.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.427+0000 7f2bff9ff700 1 -- 172.21.15.64:0/2311613772 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f2be40a0640 msgr2=0x7f2be40a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:43.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.427+0000 7f2bff9ff700 1 --2- 172.21.15.64:0/2311613772 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f2be40a0640 0x7f2be40a2b00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0006110 tx=0x7f2bf000b040 comp rx=0 tx=0).stop 2023-09-13T03:12:43.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.427+0000 7f2bff9ff700 1 -- 172.21.15.64:0/2311613772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bf81017e0 msgr2=0x7f2bf81968d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:43.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.427+0000 7f2bff9ff700 1 --2- 172.21.15.64:0/2311613772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bf81017e0 0x7f2bf81968d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2be800e9a0 tx=0x7f2be800ecb0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.427+0000 7f2bff9ff700 1 -- 172.21.15.64:0/2311613772 shutdown_connections 2023-09-13T03:12:43.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.427+0000 7f2bff9ff700 1 --2- 172.21.15.64:0/2311613772 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f2be40a0640 0x7f2be40a2b00 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.427+0000 7f2bff9ff700 1 --2- 172.21.15.64:0/2311613772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bf81005f0 0x7f2bf8196390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.427+0000 7f2bff9ff700 1 --2- 172.21.15.64:0/2311613772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2bf8102180 0x7f2bf819d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.427+0000 7f2bff9ff700 1 --2- 172.21.15.64:0/2311613772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bf81017e0 0x7f2bf81968d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.427+0000 7f2bff9ff700 1 -- 172.21.15.64:0/2311613772 >> 172.21.15.64:0/2311613772 conn(0x7f2bf80fbb40 msgr2=0x7f2bf80fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:43.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.427+0000 7f2bff9ff700 1 -- 172.21.15.64:0/2311613772 shutdown_connections 2023-09-13T03:12:43.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.427+0000 7f2bff9ff700 1 -- 172.21.15.64:0/2311613772 wait complete. 2023-09-13T03:12:43.449 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-09-13T03:12:43.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.653+0000 7fd2e1fff700 1 -- 172.21.15.64:0/4209002383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2dc074640 msgr2=0x7fd2dc072a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:43.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.653+0000 7fd2e1fff700 1 --2- 172.21.15.64:0/4209002383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2dc074640 0x7fd2dc072a90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc009990 tx=0x7fd2cc009ca0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.653+0000 7fd2e1fff700 1 -- 172.21.15.64:0/4209002383 shutdown_connections 2023-09-13T03:12:43.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.653+0000 7fd2e1fff700 1 --2- 172.21.15.64:0/4209002383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2dc1084d0 0x7fd2dc10a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.653+0000 7fd2e1fff700 1 --2- 172.21.15.64:0/4209002383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2dc074640 0x7fd2dc072a90 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.653+0000 7fd2e1fff700 1 --2- 172.21.15.64:0/4209002383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2dc073060 0x7fd2dc0734e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.653+0000 7fd2e1fff700 1 -- 172.21.15.64:0/4209002383 >> 172.21.15.64:0/4209002383 conn(0x7fd2dc0fbb80 msgr2=0x7fd2dc0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:43.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.653+0000 7fd2e1fff700 1 -- 172.21.15.64:0/4209002383 shutdown_connections 2023-09-13T03:12:43.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.653+0000 7fd2e1fff700 1 -- 172.21.15.64:0/4209002383 wait complete. 2023-09-13T03:12:43.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.654+0000 7fd2e1fff700 1 Processor -- start 2023-09-13T03:12:43.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.655+0000 7fd2e1fff700 1 -- start start 2023-09-13T03:12:43.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.655+0000 7fd2e1fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2dc073060 0x7fd2dc198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.655+0000 7fd2e1fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2dc074640 0x7fd2dc198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.655+0000 7fd2e1fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2dc1084d0 0x7fd2dc19faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.655+0000 7fd2e1fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2dc1a00c0 con 0x7fd2dc074640 2023-09-13T03:12:43.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.655+0000 7fd2e1fff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2dc1a0200 con 0x7fd2dc1084d0 2023-09-13T03:12:43.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.655+0000 7fd2e1fff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2dc1a0390 con 0x7fd2dc073060 2023-09-13T03:12:43.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.655+0000 7fd2daffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2dc074640 0x7fd2dc198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.656+0000 7fd2daffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2dc074640 0x7fd2dc198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54386/0 (socket says 172.21.15.64:54386) 2023-09-13T03:12:43.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.656+0000 7fd2daffd700 1 -- 172.21.15.64:0/3514770362 learned_addr learned my addr 172.21.15.64:0/3514770362 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:43.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.656+0000 7fd2db7fe700 1 --2- 172.21.15.64:0/3514770362 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2dc073060 0x7fd2dc198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.656+0000 7fd2dbfff700 1 --2- 172.21.15.64:0/3514770362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2dc1084d0 0x7fd2dc19faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.656+0000 7fd2daffd700 1 -- 172.21.15.64:0/3514770362 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2dc073060 msgr2=0x7fd2dc198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:43.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.656+0000 7fd2daffd700 1 --2- 172.21.15.64:0/3514770362 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2dc073060 0x7fd2dc198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.656+0000 7fd2daffd700 1 -- 172.21.15.64:0/3514770362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2dc1084d0 msgr2=0x7fd2dc19faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:43.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.656+0000 7fd2daffd700 1 --2- 172.21.15.64:0/3514770362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2dc1084d0 0x7fd2dc19faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.656+0000 7fd2daffd700 1 -- 172.21.15.64:0/3514770362 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2cc0095f0 con 0x7fd2dc074640 2023-09-13T03:12:43.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.656+0000 7fd2daffd700 1 --2- 172.21.15.64:0/3514770362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2dc074640 0x7fd2dc198a80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd2d000e970 tx=0x7fd2d000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:43.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.656+0000 7fd2d8ff9700 1 -- 172.21.15.64:0/3514770362 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d0009790 con 0x7fd2dc074640 2023-09-13T03:12:43.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.656+0000 7fd2d8ff9700 1 -- 172.21.15.64:0/3514770362 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2d000c7e0 con 0x7fd2dc074640 2023-09-13T03:12:43.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.656+0000 7fd2e1fff700 1 -- 172.21.15.64:0/3514770362 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2dc1a0670 con 0x7fd2dc074640 2023-09-13T03:12:43.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.656+0000 7fd2d8ff9700 1 -- 172.21.15.64:0/3514770362 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d0003c70 con 0x7fd2dc074640 2023-09-13T03:12:43.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.657+0000 7fd2e1fff700 1 -- 172.21.15.64:0/3514770362 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2dc1a0b80 con 0x7fd2dc074640 2023-09-13T03:12:43.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.660+0000 7fd2d8ff9700 1 -- 172.21.15.64:0/3514770362 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fd2d000cc50 con 0x7fd2dc074640 2023-09-13T03:12:43.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.662+0000 7fd2f5946b80 1 Processor -- start 2023-09-13T03:12:43.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.662+0000 7fd2f5946b80 1 -- start start 2023-09-13T03:12:43.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.662+0000 7fd2d8ff9700 1 --2- 172.21.15.64:0/3514770362 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd2c40a0640 0x7fd2c40a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.662+0000 7fd2f5946b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e59e13bb0 0x556e59e13fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.662+0000 7fd2f5946b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e59e14550 0x556e59e127b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.662+0000 7fd2f5946b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e59e12db0 0x556e59e13260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.662+0000 7fd2f5946b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e59d6b6e0 con 0x556e59e13bb0 2023-09-13T03:12:43.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.662+0000 7fd2f5946b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e59d69000 con 0x556e59e12db0 2023-09-13T03:12:43.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.662+0000 7fd2f5946b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e59d648f0 con 0x556e59e14550 2023-09-13T03:12:43.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.663+0000 7fd2d8ff9700 1 -- 172.21.15.64:0/3514770362 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fd2d0014070 con 0x7fd2dc074640 2023-09-13T03:12:43.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.663+0000 7fd2db7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e59e13bb0 0x556e59e13fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.663+0000 7fd2db7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e59e13bb0 0x556e59e13fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54390/0 (socket says 172.21.15.64:54390) 2023-09-13T03:12:43.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.663+0000 7fd2db7fe700 1 -- 172.21.15.64:0/243903642 learned_addr learned my addr 172.21.15.64:0/243903642 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:43.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.663+0000 7fd2dbfff700 1 --2- 172.21.15.64:0/3514770362 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd2c40a0640 0x7fd2c40a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.663+0000 7fd2daffd700 1 --2- 172.21.15.64:0/243903642 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e59e14550 0x556e59e127b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.663+0000 7fd2dbfff700 1 --2- 172.21.15.64:0/243903642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e59e12db0 0x556e59e13260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.663+0000 7fd2db7fe700 1 -- 172.21.15.64:0/243903642 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e59e14550 msgr2=0x556e59e127b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:43.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.663+0000 7fd2db7fe700 1 --2- 172.21.15.64:0/243903642 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e59e14550 0x556e59e127b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.663+0000 7fd2db7fe700 1 -- 172.21.15.64:0/243903642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e59e12db0 msgr2=0x556e59e13260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:43.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.663+0000 7fd2db7fe700 1 --2- 172.21.15.64:0/243903642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e59e12db0 0x556e59e13260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.663+0000 7fd2db7fe700 1 -- 172.21.15.64:0/243903642 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e59b90d90 con 0x556e59e13bb0 2023-09-13T03:12:43.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.664+0000 7fd2db7fe700 1 --2- 172.21.15.64:0/243903642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e59e13bb0 0x556e59e13fd0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc015000 tx=0x7fd2cc005ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ceb94f3895946d server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:43.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.664+0000 7fd2dbfff700 1 --2- 172.21.15.64:0/3514770362 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd2c40a0640 0x7fd2c40a2b00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4009ec0 tx=0x7fd2d4009920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:43.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.664+0000 7fd2c1ffb700 1 -- 172.21.15.64:0/243903642 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc01d070 con 0x556e59e13bb0 2023-09-13T03:12:43.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.664+0000 7fd2c1ffb700 1 -- 172.21.15.64:0/243903642 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2cc01e040 con 0x556e59e13bb0 2023-09-13T03:12:43.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.664+0000 7fd2c1ffb700 1 -- 172.21.15.64:0/243903642 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc0048b0 con 0x556e59e13bb0 2023-09-13T03:12:43.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.664+0000 7fd2f5946b80 1 -- 172.21.15.64:0/243903642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e59e13bb0 msgr2=0x556e59e13fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:43.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.664+0000 7fd2f5946b80 1 --2- 172.21.15.64:0/243903642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e59e13bb0 0x556e59e13fd0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc015000 tx=0x7fd2cc005ea0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.665+0000 7fd2f5946b80 1 -- 172.21.15.64:0/243903642 shutdown_connections 2023-09-13T03:12:43.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.665+0000 7fd2f5946b80 1 --2- 172.21.15.64:0/243903642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e59e12db0 0x556e59e13260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.665+0000 7fd2f5946b80 1 --2- 172.21.15.64:0/243903642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e59e13bb0 0x556e59e13fd0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.665+0000 7fd2f5946b80 1 --2- 172.21.15.64:0/243903642 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e59e14550 0x556e59e127b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.665+0000 7fd2f5946b80 1 -- 172.21.15.64:0/243903642 >> 172.21.15.64:0/243903642 conn(0x556e59d87040 msgr2=0x556e59e0d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:43.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.665+0000 7fd2f5946b80 1 -- 172.21.15.64:0/243903642 shutdown_connections 2023-09-13T03:12:43.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.665+0000 7fd2f5946b80 1 -- 172.21.15.64:0/243903642 wait complete. 2023-09-13T03:12:43.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.665+0000 7fd2f5946b80 1 -- start start 2023-09-13T03:12:43.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.665+0000 7fd2f5946b80 1 Processor -- start 2023-09-13T03:12:43.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.666+0000 7fd2f5946b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e59e12db0 0x556e59e1c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.666+0000 7fd2f5946b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e59e13bb0 0x556e59e1c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.666+0000 7fd2f5946b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e59e14550 0x556e59e23a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.666+0000 7fd2f5946b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e59d02b20 con 0x556e59e14550 2023-09-13T03:12:43.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.666+0000 7fd2f5946b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e59d02820 con 0x556e59e12db0 2023-09-13T03:12:43.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.666+0000 7fd2f5946b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e59be68d0 con 0x556e59e13bb0 2023-09-13T03:12:43.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.666+0000 7fd2f5946b80 10 client.0 _ll_register_callbacks cb 0x556e59c4ca30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:12:43.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.666+0000 7fd2db7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e59e12db0 0x556e59e1c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.666+0000 7fd2db7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e59e12db0 0x556e59e1c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:53284/0 (socket says 172.21.15.64:53284) 2023-09-13T03:12:43.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.666+0000 7fd2db7fe700 1 -- 172.21.15.64:0/424136915 learned_addr learned my addr 172.21.15.64:0/424136915 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:43.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.666+0000 7fd2dbfff700 1 --2- 172.21.15.64:0/424136915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e59e14550 0x556e59e23a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.666+0000 7fd2daffd700 1 --2- 172.21.15.64:0/424136915 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e59e13bb0 0x556e59e1c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.667+0000 7fd2db7fe700 1 -- 172.21.15.64:0/424136915 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e59e13bb0 msgr2=0x556e59e1c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:43.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.667+0000 7fd2db7fe700 1 --2- 172.21.15.64:0/424136915 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e59e13bb0 0x556e59e1c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.667+0000 7fd2db7fe700 1 -- 172.21.15.64:0/424136915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e59e14550 msgr2=0x556e59e23a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:43.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.667+0000 7fd2db7fe700 1 --2- 172.21.15.64:0/424136915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e59e14550 0x556e59e23a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.667+0000 7fd2db7fe700 1 -- 172.21.15.64:0/424136915 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2dc10b580 con 0x556e59e12db0 2023-09-13T03:12:43.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.667+0000 7fd2db7fe700 1 --2- 172.21.15.64:0/424136915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e59e12db0 0x556e59e1c490 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc00b2e0 tx=0x7fd2cc022bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de9720372d2f2b5b server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:43.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.667+0000 7fd2baffd700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:12:43.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.667+0000 7fd2baffd700 1 -- 172.21.15.64:0/424136915 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc01d030 con 0x556e59e12db0 2023-09-13T03:12:43.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.667+0000 7fd2f5946b80 1 -- 172.21.15.64:0/424136915 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556e59cbc4a0 con 0x556e59e12db0 2023-09-13T03:12:43.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.667+0000 7fd2baffd700 1 -- 172.21.15.64:0/424136915 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fd2cc00f460 con 0x556e59e12db0 2023-09-13T03:12:43.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.668+0000 7fd2baffd700 1 -- 172.21.15.64:0/424136915 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2cc005bc0 con 0x556e59e12db0 2023-09-13T03:12:43.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.668+0000 7fd2f5946b80 10 client.25319 fetch_fsmap learned FSMap version 26 2023-09-13T03:12:43.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.668+0000 7fd2f5946b80 1 -- 172.21.15.64:0/424136915 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x556e59b79c10 con 0x556e59e12db0 2023-09-13T03:12:43.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.668+0000 7fd2baffd700 1 -- 172.21.15.64:0/424136915 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556e59cbc4a0 con 0x556e59e12db0 2023-09-13T03:12:43.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.668+0000 7fd2baffd700 1 -- 172.21.15.64:0/424136915 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd2cc02c790 con 0x556e59e12db0 2023-09-13T03:12:43.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.668+0000 7fd2f5946b80 10 client.25319 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:12:43.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.668+0000 7fd2f5946b80 10 client.25319 resolve_mds: resolved name 'h' to daemon mds.h 2023-09-13T03:12:43.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.668+0000 7fd2f5946b80 20 client.25319 populate_metadata read hostname 'smithi064' 2023-09-13T03:12:43.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.668+0000 7fd2f5946b80 1 --2- 172.21.15.64:0/424136915 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x556e59e27670 0x556e59e26940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.668+0000 7fd2f5946b80 4 client.25319 mds_command: new command op to 24635 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:12:43.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.668+0000 7fd2f5946b80 1 -- 172.21.15.64:0/424136915 --> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556e59d79850 con 0x556e59e27670 2023-09-13T03:12:43.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.669+0000 7fd2daffd700 1 --2- 172.21.15.64:0/424136915 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x556e59e27670 0x556e59e26940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.669+0000 7fd2daffd700 1 --2- 172.21.15.64:0/424136915 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x556e59e27670 0x556e59e26940 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=68c9caaa8d0b8884 server_cookie=f4bf8ac1c3632f44 in_seq=0 out_seq=0 2023-09-13T03:12:43.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.669+0000 7fd2baffd700 10 client.25319 ms_handle_connect on v2:172.21.15.142:6836/741173268 2023-09-13T03:12:43.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.670+0000 7fd2baffd700 1 -- 172.21.15.64:0/424136915 <== mds.2 v2:172.21.15.142:6836/741173268 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556e59d79850 con 0x556e59e27670 2023-09-13T03:12:43.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.670+0000 7fd2baffd700 10 client.25319 handle_command_reply: tid=0 2023-09-13T03:12:43.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.670+0000 7fd2f5946b80 1 client.25319 shutdown 2023-09-13T03:12:43.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.670+0000 7fd2f5946b80 1 -- 172.21.15.64:0/424136915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e59e12db0 msgr2=0x556e59e1c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:43.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.670+0000 7fd2f5946b80 1 --2- 172.21.15.64:0/424136915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e59e12db0 0x556e59e1c490 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc00b2e0 tx=0x7fd2cc022bb0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.670+0000 7fd2f5946b80 1 -- 172.21.15.64:0/424136915 shutdown_connections 2023-09-13T03:12:43.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.670+0000 7fd2f5946b80 1 --2- 172.21.15.64:0/424136915 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x556e59e27670 0x556e59e26940 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.670+0000 7fd2f5946b80 1 --2- 172.21.15.64:0/424136915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e59e12db0 0x556e59e1c490 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.670+0000 7fd2f5946b80 1 --2- 172.21.15.64:0/424136915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e59e14550 0x556e59e23a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.670+0000 7fd2f5946b80 1 --2- 172.21.15.64:0/424136915 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e59e13bb0 0x556e59e1c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.670+0000 7fd2baffd700 0 client.25319 ms_handle_reset on v2:172.21.15.142:6836/741173268 2023-09-13T03:12:43.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.670+0000 7fd2f5946b80 1 -- 172.21.15.64:0/424136915 >> 172.21.15.64:0/424136915 conn(0x556e59d87040 msgr2=0x556e59e0cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:43.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.670+0000 7fd2f5946b80 1 -- 172.21.15.64:0/424136915 shutdown_connections 2023-09-13T03:12:43.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.670+0000 7fd2f5946b80 1 -- 172.21.15.64:0/424136915 wait complete. 2023-09-13T03:12:43.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.670+0000 7fd2f5946b80 20 client.25319 trim_cache size 0 max 16384 2023-09-13T03:12:43.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.688+0000 7fd2f5946b80 1 Processor -- start 2023-09-13T03:12:43.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.688+0000 7fd2f5946b80 1 -- start start 2023-09-13T03:12:43.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.689+0000 7fd2f5946b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e59e12db0 0x556e59e126c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.689+0000 7fd2f5946b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e59e13bb0 0x556e59e239d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.689+0000 7fd2f5946b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e59e14550 0x556e59e0cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.689+0000 7fd2f5946b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e59e28a60 con 0x556e59e14550 2023-09-13T03:12:43.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.689+0000 7fd2f5946b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e59d0c4d0 con 0x556e59e12db0 2023-09-13T03:12:43.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.689+0000 7fd2f5946b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e59d2df70 con 0x556e59e13bb0 2023-09-13T03:12:43.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.689+0000 7fd2db7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e59e12db0 0x556e59e126c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.689+0000 7fd2db7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e59e12db0 0x556e59e126c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:53296/0 (socket says 172.21.15.64:53296) 2023-09-13T03:12:43.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.689+0000 7fd2db7fe700 1 -- 172.21.15.64:0/2199335010 learned_addr learned my addr 172.21.15.64:0/2199335010 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:43.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.689+0000 7fd2dbfff700 1 --2- 172.21.15.64:0/2199335010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e59e14550 0x556e59e0cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.689+0000 7fd2daffd700 1 --2- 172.21.15.64:0/2199335010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e59e13bb0 0x556e59e239d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.689+0000 7fd2db7fe700 1 -- 172.21.15.64:0/2199335010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e59e13bb0 msgr2=0x556e59e239d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:43.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.689+0000 7fd2db7fe700 1 --2- 172.21.15.64:0/2199335010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e59e13bb0 0x556e59e239d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.689+0000 7fd2db7fe700 1 -- 172.21.15.64:0/2199335010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e59e14550 msgr2=0x556e59e0cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:43.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.689+0000 7fd2db7fe700 1 --2- 172.21.15.64:0/2199335010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e59e14550 0x556e59e0cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.689+0000 7fd2db7fe700 1 -- 172.21.15.64:0/2199335010 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e59bb7200 con 0x556e59e12db0 2023-09-13T03:12:43.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.690+0000 7fd2db7fe700 1 --2- 172.21.15.64:0/2199335010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e59e12db0 0x556e59e126c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc017410 tx=0x7fd2cc005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27acaedb5565484c server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:43.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.690+0000 7fd2c27fc700 1 -- 172.21.15.64:0/2199335010 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc01d040 con 0x556e59e12db0 2023-09-13T03:12:43.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.690+0000 7fd2c27fc700 1 -- 172.21.15.64:0/2199335010 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2cc02ce90 con 0x556e59e12db0 2023-09-13T03:12:43.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.690+0000 7fd2c27fc700 1 -- 172.21.15.64:0/2199335010 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc02c660 con 0x556e59e12db0 2023-09-13T03:12:43.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.691+0000 7fd2f5946b80 1 -- 172.21.15.64:0/2199335010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e59e12db0 msgr2=0x556e59e126c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:43.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.691+0000 7fd2f5946b80 1 --2- 172.21.15.64:0/2199335010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e59e12db0 0x556e59e126c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc017410 tx=0x7fd2cc005fb0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.691+0000 7fd2f5946b80 1 -- 172.21.15.64:0/2199335010 shutdown_connections 2023-09-13T03:12:43.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.691+0000 7fd2f5946b80 1 --2- 172.21.15.64:0/2199335010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e59e12db0 0x556e59e126c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.691+0000 7fd2f5946b80 1 --2- 172.21.15.64:0/2199335010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e59e14550 0x556e59e0cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.691+0000 7fd2f5946b80 1 --2- 172.21.15.64:0/2199335010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e59e13bb0 0x556e59e239d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.691+0000 7fd2f5946b80 1 -- 172.21.15.64:0/2199335010 >> 172.21.15.64:0/2199335010 conn(0x556e59d87040 msgr2=0x556e59e13190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:43.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.691+0000 7fd2f5946b80 1 -- 172.21.15.64:0/2199335010 shutdown_connections 2023-09-13T03:12:43.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.691+0000 7fd2f5946b80 1 -- 172.21.15.64:0/2199335010 wait complete. 2023-09-13T03:12:43.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.691+0000 7fd2f5946b80 1 -- start start 2023-09-13T03:12:43.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.691+0000 7fd2f5946b80 1 Processor -- start 2023-09-13T03:12:43.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.692+0000 7fd2f5946b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e59e12db0 0x556e59e20c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.692+0000 7fd2f5946b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e59e13bb0 0x556e59e231a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.692+0000 7fd2f5946b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e59e14550 0x556e59e237d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.692+0000 7fd2f5946b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e59e7b6c0 con 0x556e59e14550 2023-09-13T03:12:43.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.692+0000 7fd2f5946b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e59e7b800 con 0x556e59e12db0 2023-09-13T03:12:43.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.692+0000 7fd2f5946b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e59e7b940 con 0x556e59e13bb0 2023-09-13T03:12:43.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.692+0000 7fd2f5946b80 10 client.0 _ll_register_callbacks cb 0x7fd2d001ff60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:12:43.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.692+0000 7fd2dbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e59e14550 0x556e59e237d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.692+0000 7fd2dbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e59e14550 0x556e59e237d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54410/0 (socket says 172.21.15.64:54410) 2023-09-13T03:12:43.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.692+0000 7fd2dbfff700 1 -- 172.21.15.64:0/574220193 learned_addr learned my addr 172.21.15.64:0/574220193 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:43.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.693+0000 7fd2db7fe700 1 --2- 172.21.15.64:0/574220193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e59e12db0 0x556e59e20c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.693+0000 7fd2daffd700 1 --2- 172.21.15.64:0/574220193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e59e13bb0 0x556e59e231a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.693+0000 7fd2dbfff700 1 -- 172.21.15.64:0/574220193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e59e13bb0 msgr2=0x556e59e231a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:43.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.693+0000 7fd2dbfff700 1 --2- 172.21.15.64:0/574220193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e59e13bb0 0x556e59e231a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.693+0000 7fd2dbfff700 1 -- 172.21.15.64:0/574220193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e59e12db0 msgr2=0x556e59e20c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:43.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.693+0000 7fd2dbfff700 1 --2- 172.21.15.64:0/574220193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e59e12db0 0x556e59e20c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.693+0000 7fd2dbfff700 1 -- 172.21.15.64:0/574220193 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556e59b79c10 con 0x556e59e14550 2023-09-13T03:12:43.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.693+0000 7fd2dbfff700 1 --2- 172.21.15.64:0/574220193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e59e14550 0x556e59e237d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4018cd0 tx=0x7fd2d4004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfcbc9cf39305668 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:43.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.693+0000 7fd2baffd700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:12:43.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.693+0000 7fd2baffd700 1 -- 172.21.15.64:0/574220193 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d400e070 con 0x556e59e14550 2023-09-13T03:12:43.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.693+0000 7fd2f5946b80 1 -- 172.21.15.64:0/574220193 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2cc01e040 con 0x556e59e14550 2023-09-13T03:12:43.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.694+0000 7fd2baffd700 1 -- 172.21.15.64:0/574220193 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fd2d4003a00 con 0x556e59e14550 2023-09-13T03:12:43.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.694+0000 7fd2baffd700 1 -- 172.21.15.64:0/574220193 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2d4003bc0 con 0x556e59e14550 2023-09-13T03:12:43.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.694+0000 7fd2f5946b80 10 client.15303 fetch_fsmap learned FSMap version 26 2023-09-13T03:12:43.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.694+0000 7fd2f5946b80 1 -- 172.21.15.64:0/574220193 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x556e59b8f260 con 0x556e59e14550 2023-09-13T03:12:43.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.694+0000 7fd2baffd700 1 -- 172.21.15.64:0/574220193 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2cc01e040 con 0x556e59e14550 2023-09-13T03:12:43.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.694+0000 7fd2baffd700 1 -- 172.21.15.64:0/574220193 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd2d4004020 con 0x556e59e14550 2023-09-13T03:12:43.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.694+0000 7fd2f5946b80 10 client.15303 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:12:43.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.694+0000 7fd2f5946b80 10 client.15303 resolve_mds: resolved name 'h' to daemon mds.h 2023-09-13T03:12:43.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.694+0000 7fd2f5946b80 20 client.15303 populate_metadata read hostname 'smithi064' 2023-09-13T03:12:43.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.694+0000 7fd2f5946b80 1 --2- 172.21.15.64:0/574220193 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x556e59e80d40 0x556e59e7d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.694+0000 7fd2f5946b80 4 client.15303 mds_command: new command op to 24635 tid=0[{"prefix": "status"}] 2023-09-13T03:12:43.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.694+0000 7fd2f5946b80 1 -- 172.21.15.64:0/574220193 --> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556e59d5d8d0 con 0x556e59e80d40 2023-09-13T03:12:43.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.695+0000 7fd2db7fe700 1 --2- 172.21.15.64:0/574220193 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x556e59e80d40 0x556e59e7d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.695+0000 7fd2db7fe700 1 --2- 172.21.15.64:0/574220193 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x556e59e80d40 0x556e59e7d180 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7da7da04b7175d95 server_cookie=e526098433e10957 in_seq=0 out_seq=0 2023-09-13T03:12:43.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.695+0000 7fd2baffd700 10 client.15303 ms_handle_connect on v2:172.21.15.142:6836/741173268 2023-09-13T03:12:43.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.695+0000 7fd2baffd700 1 -- 172.21.15.64:0/574220193 <== mds.2 v2:172.21.15.142:6836/741173268 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x556e59d5d8d0 con 0x556e59e80d40 2023-09-13T03:12:43.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.695+0000 7fd2baffd700 10 client.15303 handle_command_reply: tid=0 2023-09-13T03:12:43.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.695+0000 7fd2f5946b80 1 client.15303 shutdown 2023-09-13T03:12:43.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.696+0000 7fd2f5946b80 1 -- 172.21.15.64:0/574220193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e59e14550 msgr2=0x556e59e237d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:43.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.696+0000 7fd2f5946b80 1 --2- 172.21.15.64:0/574220193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e59e14550 0x556e59e237d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4018cd0 tx=0x7fd2d4004b40 comp rx=0 tx=0).stop 2023-09-13T03:12:43.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.696+0000 7fd2f5946b80 1 -- 172.21.15.64:0/574220193 shutdown_connections 2023-09-13T03:12:43.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.696+0000 7fd2f5946b80 1 --2- 172.21.15.64:0/574220193 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x556e59e80d40 0x556e59e7d180 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.696+0000 7fd2f5946b80 1 --2- 172.21.15.64:0/574220193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e59e12db0 0x556e59e20c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.696+0000 7fd2f5946b80 1 --2- 172.21.15.64:0/574220193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e59e14550 0x556e59e237d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.696+0000 7fd2f5946b80 1 --2- 172.21.15.64:0/574220193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e59e13bb0 0x556e59e231a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.696+0000 7fd2f5946b80 1 -- 172.21.15.64:0/574220193 >> 172.21.15.64:0/574220193 conn(0x556e59d87040 msgr2=0x556e59e0cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:43.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.696+0000 7fd2baffd700 0 client.15303 ms_handle_reset on v2:172.21.15.142:6836/741173268 2023-09-13T03:12:43.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.696+0000 7fd2f5946b80 1 -- 172.21.15.64:0/574220193 shutdown_connections 2023-09-13T03:12:43.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.696+0000 7fd2f5946b80 1 -- 172.21.15.64:0/574220193 wait complete. 2023-09-13T03:12:43.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.696+0000 7fd2f5946b80 20 client.15303 trim_cache size 0 max 16384 2023-09-13T03:12:43.712 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-13T03:12:43.712 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "f8dbced4-51e1-11ee-9ab7-7b867c8bd7da", 2023-09-13T03:12:43.712 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 2, 2023-09-13T03:12:43.712 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24635, 2023-09-13T03:12:43.712 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2023-09-13T03:12:43.713 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2023-09-13T03:12:43.713 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2023-09-13T03:12:43.713 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 88.495516933999994, 2023-09-13T03:12:43.713 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 26, 2023-09-13T03:12:43.713 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 85, 2023-09-13T03:12:43.714 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 83, 2023-09-13T03:12:43.714 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 110.7357386 2023-09-13T03:12:43.714 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-13T03:12:43.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.696+0000 7fd2e1fff700 1 -- 172.21.15.64:0/3514770362 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd2c40a0640 msgr2=0x7fd2c40a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:43.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.696+0000 7fd2e1fff700 1 --2- 172.21.15.64:0/3514770362 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd2c40a0640 0x7fd2c40a2b00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4009ec0 tx=0x7fd2d4009920 comp rx=0 tx=0).stop 2023-09-13T03:12:43.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.696+0000 7fd2e1fff700 1 -- 172.21.15.64:0/3514770362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2dc074640 msgr2=0x7fd2dc198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:43.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.696+0000 7fd2e1fff700 1 --2- 172.21.15.64:0/3514770362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2dc074640 0x7fd2dc198a80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd2d000e970 tx=0x7fd2d000ec80 comp rx=0 tx=0).stop 2023-09-13T03:12:43.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.697+0000 7fd2e1fff700 1 -- 172.21.15.64:0/3514770362 shutdown_connections 2023-09-13T03:12:43.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.697+0000 7fd2e1fff700 1 --2- 172.21.15.64:0/3514770362 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd2c40a0640 0x7fd2c40a2b00 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.697+0000 7fd2e1fff700 1 --2- 172.21.15.64:0/3514770362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2dc1084d0 0x7fd2dc19faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.697+0000 7fd2e1fff700 1 --2- 172.21.15.64:0/3514770362 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2dc073060 0x7fd2dc198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.697+0000 7fd2e1fff700 1 --2- 172.21.15.64:0/3514770362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2dc074640 0x7fd2dc198a80 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.697+0000 7fd2e1fff700 1 -- 172.21.15.64:0/3514770362 >> 172.21.15.64:0/3514770362 conn(0x7fd2dc0fbb80 msgr2=0x7fd2dc0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:43.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.697+0000 7fd2e1fff700 1 -- 172.21.15.64:0/3514770362 shutdown_connections 2023-09-13T03:12:43.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.697+0000 7fd2e1fff700 1 -- 172.21.15.64:0/3514770362 wait complete. 2023-09-13T03:12:43.717 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:43 smithi064 ceph-mon[97675]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-13T03:12:43.717 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:43 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/246091133' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:12:43.718 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-09-13T03:12:43.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:43 smithi170 ceph-mon[134328]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-13T03:12:43.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:43 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/246091133' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:12:43.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.923+0000 7ff51bfff700 1 -- 172.21.15.64:0/2379537097 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff514101800 msgr2=0x7ff514101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:43.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.923+0000 7ff51bfff700 1 --2- 172.21.15.64:0/2379537097 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff514101800 0x7ff514101c60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff5040099c0 tx=0x7ff504009cd0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.923+0000 7ff51bfff700 1 -- 172.21.15.64:0/2379537097 shutdown_connections 2023-09-13T03:12:43.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.923+0000 7ff51bfff700 1 --2- 172.21.15.64:0/2379537097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5141021a0 0x7ff514108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.923+0000 7ff51bfff700 1 --2- 172.21.15.64:0/2379537097 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff514100610 0x7ff514100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.923+0000 7ff51bfff700 1 --2- 172.21.15.64:0/2379537097 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff514101800 0x7ff514101c60 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.923+0000 7ff51bfff700 1 -- 172.21.15.64:0/2379537097 >> 172.21.15.64:0/2379537097 conn(0x7ff5140fbb80 msgr2=0x7ff5140fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:43.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.923+0000 7ff51bfff700 1 -- 172.21.15.64:0/2379537097 shutdown_connections 2023-09-13T03:12:43.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.923+0000 7ff51bfff700 1 -- 172.21.15.64:0/2379537097 wait complete. 2023-09-13T03:12:43.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.924+0000 7ff51bfff700 1 Processor -- start 2023-09-13T03:12:43.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.924+0000 7ff51bfff700 1 -- start start 2023-09-13T03:12:43.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.924+0000 7ff51bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff514100610 0x7ff5141984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.924+0000 7ff51bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff514101800 0x7ff514198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.924+0000 7ff51bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5141021a0 0x7ff51419fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.924+0000 7ff51bfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5141a0040 con 0x7ff514100610 2023-09-13T03:12:43.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.924+0000 7ff51bfff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5141a0180 con 0x7ff514101800 2023-09-13T03:12:43.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.924+0000 7ff51bfff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5141a0310 con 0x7ff5141021a0 2023-09-13T03:12:43.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.925+0000 7ff519d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff514100610 0x7ff5141984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.925+0000 7ff519d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff514100610 0x7ff5141984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54432/0 (socket says 172.21.15.64:54432) 2023-09-13T03:12:43.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.925+0000 7ff519d9b700 1 -- 172.21.15.64:0/3407848400 learned_addr learned my addr 172.21.15.64:0/3407848400 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:43.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.925+0000 7ff51959a700 1 --2- 172.21.15.64:0/3407848400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff514101800 0x7ff514198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.925+0000 7ff51a59c700 1 --2- 172.21.15.64:0/3407848400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5141021a0 0x7ff51419fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.925+0000 7ff519d9b700 1 -- 172.21.15.64:0/3407848400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5141021a0 msgr2=0x7ff51419fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:43.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.925+0000 7ff519d9b700 1 --2- 172.21.15.64:0/3407848400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5141021a0 0x7ff51419fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.925+0000 7ff519d9b700 1 -- 172.21.15.64:0/3407848400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff514101800 msgr2=0x7ff514198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:43.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.925+0000 7ff519d9b700 1 --2- 172.21.15.64:0/3407848400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff514101800 0x7ff514198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.925+0000 7ff519d9b700 1 -- 172.21.15.64:0/3407848400 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5040095f0 con 0x7ff514100610 2023-09-13T03:12:43.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.925+0000 7ff519d9b700 1 --2- 172.21.15.64:0/3407848400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff514100610 0x7ff5141984c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff51000e970 tx=0x7ff51000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:43.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.925+0000 7ff50affd700 1 -- 172.21.15.64:0/3407848400 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff510009790 con 0x7ff514100610 2023-09-13T03:12:43.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.926+0000 7ff50affd700 1 -- 172.21.15.64:0/3407848400 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff51000eec0 con 0x7ff514100610 2023-09-13T03:12:43.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.926+0000 7ff51bfff700 1 -- 172.21.15.64:0/3407848400 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5141a05f0 con 0x7ff514100610 2023-09-13T03:12:43.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.926+0000 7ff50affd700 1 -- 172.21.15.64:0/3407848400 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff510003cf0 con 0x7ff514100610 2023-09-13T03:12:43.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.926+0000 7ff51bfff700 1 -- 172.21.15.64:0/3407848400 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5141a0b00 con 0x7ff514100610 2023-09-13T03:12:43.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.929+0000 7ff50affd700 1 -- 172.21.15.64:0/3407848400 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7ff51000ccd0 con 0x7ff514100610 2023-09-13T03:12:43.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.930+0000 7ff50affd700 1 --2- 172.21.15.64:0/3407848400 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ff5000a0640 0x7ff5000a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.930+0000 7ff52f831b80 1 Processor -- start 2023-09-13T03:12:43.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.930+0000 7ff50affd700 1 -- 172.21.15.64:0/3407848400 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff510014070 con 0x7ff514100610 2023-09-13T03:12:43.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.930+0000 7ff52f831b80 1 -- start start 2023-09-13T03:12:43.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.931+0000 7ff51959a700 1 --2- 172.21.15.64:0/3407848400 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ff5000a0640 0x7ff5000a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.931+0000 7ff52f831b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daaaa76bb0 0x55daaaa76fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.931+0000 7ff52f831b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55daaaa77550 0x55daaaa757b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.931+0000 7ff52f831b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55daaaa75db0 0x55daaaa76260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.931+0000 7ff52f831b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daaa9ce6e0 con 0x55daaaa76bb0 2023-09-13T03:12:43.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.931+0000 7ff52f831b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daaa9cc000 con 0x55daaaa77550 2023-09-13T03:12:43.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.931+0000 7ff52f831b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daaa9c78f0 con 0x55daaaa75db0 2023-09-13T03:12:43.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.931+0000 7ff519d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daaaa76bb0 0x55daaaa76fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.931+0000 7ff519d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daaaa76bb0 0x55daaaa76fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54438/0 (socket says 172.21.15.64:54438) 2023-09-13T03:12:43.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.931+0000 7ff519d9b700 1 -- 172.21.15.64:0/2807525172 learned_addr learned my addr 172.21.15.64:0/2807525172 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:43.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.931+0000 7ff51959a700 1 --2- 172.21.15.64:0/2807525172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55daaaa77550 0x55daaaa757b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.931+0000 7ff51a59c700 1 --2- 172.21.15.64:0/2807525172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55daaaa75db0 0x55daaaa76260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.931+0000 7ff519d9b700 1 -- 172.21.15.64:0/2807525172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55daaaa75db0 msgr2=0x55daaaa76260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:43.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.931+0000 7ff519d9b700 1 --2- 172.21.15.64:0/2807525172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55daaaa75db0 0x55daaaa76260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.931+0000 7ff51959a700 1 --2- 172.21.15.64:0/3407848400 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ff5000a0640 0x7ff5000a2b00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff50400b580 tx=0x7ff504005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:43.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.932+0000 7ff519d9b700 1 -- 172.21.15.64:0/2807525172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55daaaa77550 msgr2=0x55daaaa757b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:43.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.932+0000 7ff519d9b700 1 --2- 172.21.15.64:0/2807525172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55daaaa77550 0x55daaaa757b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.932+0000 7ff519d9b700 1 -- 172.21.15.64:0/2807525172 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55daaa7f3d90 con 0x55daaaa76bb0 2023-09-13T03:12:43.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.932+0000 7ff51959a700 1 --2- 172.21.15.64:0/2807525172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55daaaa77550 0x55daaaa757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:12:43.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.932+0000 7ff519d9b700 1 --2- 172.21.15.64:0/2807525172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daaaa76bb0 0x55daaaa76fd0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff510096070 tx=0x7ff510098f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d41c6393572cad33 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:43.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.932+0000 7ff4fffff700 1 -- 172.21.15.64:0/2807525172 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51009d430 con 0x55daaaa76bb0 2023-09-13T03:12:43.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.932+0000 7ff4fffff700 1 -- 172.21.15.64:0/2807525172 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5100a1670 con 0x55daaaa76bb0 2023-09-13T03:12:43.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.932+0000 7ff4fffff700 1 -- 172.21.15.64:0/2807525172 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff510084320 con 0x55daaaa76bb0 2023-09-13T03:12:43.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.933+0000 7ff52f831b80 1 -- 172.21.15.64:0/2807525172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daaaa76bb0 msgr2=0x55daaaa76fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:43.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.933+0000 7ff52f831b80 1 --2- 172.21.15.64:0/2807525172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daaaa76bb0 0x55daaaa76fd0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff510096070 tx=0x7ff510098f60 comp rx=0 tx=0).stop 2023-09-13T03:12:43.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.933+0000 7ff52f831b80 1 -- 172.21.15.64:0/2807525172 shutdown_connections 2023-09-13T03:12:43.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.933+0000 7ff52f831b80 1 --2- 172.21.15.64:0/2807525172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55daaaa75db0 0x55daaaa76260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.933+0000 7ff52f831b80 1 --2- 172.21.15.64:0/2807525172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daaaa76bb0 0x55daaaa76fd0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.933+0000 7ff52f831b80 1 --2- 172.21.15.64:0/2807525172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55daaaa77550 0x55daaaa757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.933+0000 7ff52f831b80 1 -- 172.21.15.64:0/2807525172 >> 172.21.15.64:0/2807525172 conn(0x55daaa9ea040 msgr2=0x55daaaa706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:43.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.933+0000 7ff52f831b80 1 -- 172.21.15.64:0/2807525172 shutdown_connections 2023-09-13T03:12:43.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.933+0000 7ff52f831b80 1 -- 172.21.15.64:0/2807525172 wait complete. 2023-09-13T03:12:43.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.933+0000 7ff52f831b80 1 -- start start 2023-09-13T03:12:43.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.934+0000 7ff52f831b80 1 Processor -- start 2023-09-13T03:12:43.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.935+0000 7ff52f831b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55daaaa75db0 0x55daaaa7f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.935+0000 7ff52f831b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55daaaa76bb0 0x55daaaa7fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.935+0000 7ff52f831b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daaaa77550 0x55daaaa86b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.935+0000 7ff52f831b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daaa965b20 con 0x55daaaa77550 2023-09-13T03:12:43.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.935+0000 7ff52f831b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daaa965820 con 0x55daaaa75db0 2023-09-13T03:12:43.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.935+0000 7ff52f831b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daaa8498d0 con 0x55daaaa76bb0 2023-09-13T03:12:43.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.935+0000 7ff52f831b80 10 client.0 _ll_register_callbacks cb 0x55daaa8afa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:12:43.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.935+0000 7ff51a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daaaa77550 0x55daaaa86b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.935+0000 7ff51959a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55daaaa76bb0 0x55daaaa7fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.935+0000 7ff51959a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55daaaa76bb0 0x55daaaa7fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:50922/0 (socket says 172.21.15.64:50922) 2023-09-13T03:12:43.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.935+0000 7ff51959a700 1 -- 172.21.15.64:0/1687988187 learned_addr learned my addr 172.21.15.64:0/1687988187 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:43.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.935+0000 7ff519d9b700 1 --2- 172.21.15.64:0/1687988187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55daaaa75db0 0x55daaaa7f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.935+0000 7ff51a59c700 1 -- 172.21.15.64:0/1687988187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55daaaa76bb0 msgr2=0x55daaaa7fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:43.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.935+0000 7ff51a59c700 1 --2- 172.21.15.64:0/1687988187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55daaaa76bb0 0x55daaaa7fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.935+0000 7ff51a59c700 1 -- 172.21.15.64:0/1687988187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55daaaa75db0 msgr2=0x55daaaa7f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:43.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.935+0000 7ff51a59c700 1 --2- 172.21.15.64:0/1687988187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55daaaa75db0 0x55daaaa7f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.935+0000 7ff51a59c700 1 -- 172.21.15.64:0/1687988187 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5141a0b00 con 0x55daaaa77550 2023-09-13T03:12:43.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.935+0000 7ff519d9b700 1 --2- 172.21.15.64:0/1687988187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55daaaa75db0 0x55daaaa7f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:43.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.936+0000 7ff51a59c700 1 --2- 172.21.15.64:0/1687988187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daaaa77550 0x55daaaa86b40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff50c007960 tx=0x7ff50c007c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47534161b8871a71 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:43.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.936+0000 7ff4fd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:12:43.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.936+0000 7ff4fd7fa700 1 -- 172.21.15.64:0/1687988187 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50c00e070 con 0x55daaaa77550 2023-09-13T03:12:43.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.936+0000 7ff52f831b80 1 -- 172.21.15.64:0/1687988187 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55daaa91f4a0 con 0x55daaaa77550 2023-09-13T03:12:43.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.936+0000 7ff4fd7fa700 1 -- 172.21.15.64:0/1687988187 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff50c012900 con 0x55daaaa77550 2023-09-13T03:12:43.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.936+0000 7ff4fd7fa700 1 -- 172.21.15.64:0/1687988187 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff50c012ac0 con 0x55daaaa77550 2023-09-13T03:12:43.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.936+0000 7ff52f831b80 10 client.15315 fetch_fsmap learned FSMap version 26 2023-09-13T03:12:43.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.936+0000 7ff52f831b80 1 -- 172.21.15.64:0/1687988187 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55daaa7dcc10 con 0x55daaaa77550 2023-09-13T03:12:43.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.936+0000 7ff4fd7fa700 1 -- 172.21.15.64:0/1687988187 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55daaa91f4a0 con 0x55daaaa77550 2023-09-13T03:12:43.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.936+0000 7ff4fd7fa700 1 -- 172.21.15.64:0/1687988187 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff50c014040 con 0x55daaaa77550 2023-09-13T03:12:43.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.937+0000 7ff52f831b80 10 client.15315 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:12:43.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.937+0000 7ff52f831b80 10 client.15315 resolve_mds: resolved name 'i' to daemon mds.i 2023-09-13T03:12:43.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.937+0000 7ff52f831b80 20 client.15315 populate_metadata read hostname 'smithi064' 2023-09-13T03:12:43.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.937+0000 7ff52f831b80 1 --2- 172.21.15.64:0/1687988187 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x55daaaa8acc0 0x55daaaa89c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.937+0000 7ff52f831b80 4 client.15315 mds_command: new command op to 24649 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:12:43.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.937+0000 7ff52f831b80 1 -- 172.21.15.64:0/1687988187 --> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55daaa9dc850 con 0x55daaaa8acc0 2023-09-13T03:12:43.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.937+0000 7ff519d9b700 1 --2- 172.21.15.64:0/1687988187 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x55daaaa8acc0 0x55daaaa89c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.937+0000 7ff519d9b700 1 --2- 172.21.15.64:0/1687988187 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x55daaaa8acc0 0x55daaaa89c50 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=15023dcd38465d1b server_cookie=17ce429d6db73ac in_seq=0 out_seq=0 2023-09-13T03:12:43.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.937+0000 7ff4fd7fa700 10 client.15315 ms_handle_connect on v2:172.21.15.170:6838/2189703596 2023-09-13T03:12:43.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.938+0000 7ff4fd7fa700 1 -- 172.21.15.64:0/1687988187 <== mds.1 v2:172.21.15.170:6838/2189703596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55daaa9dc850 con 0x55daaaa8acc0 2023-09-13T03:12:43.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.938+0000 7ff4fd7fa700 10 client.15315 handle_command_reply: tid=0 2023-09-13T03:12:43.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.938+0000 7ff52f831b80 1 client.15315 shutdown 2023-09-13T03:12:43.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.939+0000 7ff52f831b80 1 -- 172.21.15.64:0/1687988187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daaaa77550 msgr2=0x55daaaa86b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:43.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.939+0000 7ff52f831b80 1 --2- 172.21.15.64:0/1687988187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daaaa77550 0x55daaaa86b40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff50c007960 tx=0x7ff50c007c70 comp rx=0 tx=0).stop 2023-09-13T03:12:43.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.939+0000 7ff52f831b80 1 -- 172.21.15.64:0/1687988187 shutdown_connections 2023-09-13T03:12:43.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.939+0000 7ff52f831b80 1 --2- 172.21.15.64:0/1687988187 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x55daaaa8acc0 0x55daaaa89c50 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.939+0000 7ff52f831b80 1 --2- 172.21.15.64:0/1687988187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55daaaa75db0 0x55daaaa7f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.939+0000 7ff52f831b80 1 --2- 172.21.15.64:0/1687988187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daaaa77550 0x55daaaa86b40 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.939+0000 7ff4fd7fa700 0 client.15315 ms_handle_reset on v2:172.21.15.170:6838/2189703596 2023-09-13T03:12:43.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.939+0000 7ff52f831b80 1 --2- 172.21.15.64:0/1687988187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55daaaa76bb0 0x55daaaa7fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.939+0000 7ff52f831b80 1 -- 172.21.15.64:0/1687988187 >> 172.21.15.64:0/1687988187 conn(0x55daaa9ea040 msgr2=0x55daaaa6fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:43.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.939+0000 7ff52f831b80 1 -- 172.21.15.64:0/1687988187 shutdown_connections 2023-09-13T03:12:43.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.939+0000 7ff52f831b80 1 -- 172.21.15.64:0/1687988187 wait complete. 2023-09-13T03:12:43.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.939+0000 7ff52f831b80 20 client.15315 trim_cache size 0 max 16384 2023-09-13T03:12:43.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.957+0000 7ff52f831b80 1 Processor -- start 2023-09-13T03:12:43.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.957+0000 7ff52f831b80 1 -- start start 2023-09-13T03:12:43.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.958+0000 7ff52f831b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55daaaa75db0 0x55daaaa756c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.958+0000 7ff52f831b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daaaa76bb0 0x55daaaa76190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.958+0000 7ff52f831b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55daaaa77550 0x55daaaa86a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.958+0000 7ff52f831b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daaa855de0 con 0x55daaaa76bb0 2023-09-13T03:12:43.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.958+0000 7ff52f831b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daaa990f70 con 0x55daaaa75db0 2023-09-13T03:12:43.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.958+0000 7ff52f831b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daaa983ed0 con 0x55daaaa77550 2023-09-13T03:12:43.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.958+0000 7ff51959a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daaaa76bb0 0x55daaaa76190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.958+0000 7ff51959a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daaaa76bb0 0x55daaaa76190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54454/0 (socket says 172.21.15.64:54454) 2023-09-13T03:12:43.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.958+0000 7ff51959a700 1 -- 172.21.15.64:0/1232793887 learned_addr learned my addr 172.21.15.64:0/1232793887 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:43.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.958+0000 7ff51a59c700 1 --2- 172.21.15.64:0/1232793887 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55daaaa77550 0x55daaaa86a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.958+0000 7ff519d9b700 1 --2- 172.21.15.64:0/1232793887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55daaaa75db0 0x55daaaa756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.958+0000 7ff51959a700 1 -- 172.21.15.64:0/1232793887 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55daaaa77550 msgr2=0x55daaaa86a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:43.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.958+0000 7ff51959a700 1 --2- 172.21.15.64:0/1232793887 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55daaaa77550 0x55daaaa86a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.958+0000 7ff51959a700 1 -- 172.21.15.64:0/1232793887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55daaaa75db0 msgr2=0x55daaaa756c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:43.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.958+0000 7ff51959a700 1 --2- 172.21.15.64:0/1232793887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55daaaa75db0 0x55daaaa756c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.958+0000 7ff51959a700 1 -- 172.21.15.64:0/1232793887 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55daaa81a200 con 0x55daaaa76bb0 2023-09-13T03:12:43.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.958+0000 7ff51a59c700 1 --2- 172.21.15.64:0/1232793887 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55daaaa77550 0x55daaaa86a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:43.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.959+0000 7ff51959a700 1 --2- 172.21.15.64:0/1232793887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daaaa76bb0 0x55daaaa76190 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff5040058e0 tx=0x7ff50401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8e16d9aac427894 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:43.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.959+0000 7ff508ff9700 1 -- 172.21.15.64:0/1232793887 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff504003e60 con 0x55daaaa76bb0 2023-09-13T03:12:43.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.959+0000 7ff508ff9700 1 -- 172.21.15.64:0/1232793887 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff50402c050 con 0x55daaaa76bb0 2023-09-13T03:12:43.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.959+0000 7ff508ff9700 1 -- 172.21.15.64:0/1232793887 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50400fac0 con 0x55daaaa76bb0 2023-09-13T03:12:43.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.959+0000 7ff52f831b80 1 -- 172.21.15.64:0/1232793887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daaaa76bb0 msgr2=0x55daaaa76190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:43.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.959+0000 7ff52f831b80 1 --2- 172.21.15.64:0/1232793887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daaaa76bb0 0x55daaaa76190 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff5040058e0 tx=0x7ff50401f000 comp rx=0 tx=0).stop 2023-09-13T03:12:43.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.959+0000 7ff52f831b80 1 -- 172.21.15.64:0/1232793887 shutdown_connections 2023-09-13T03:12:43.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.960+0000 7ff52f831b80 1 --2- 172.21.15.64:0/1232793887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55daaaa75db0 0x55daaaa756c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.960+0000 7ff52f831b80 1 --2- 172.21.15.64:0/1232793887 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55daaaa77550 0x55daaaa86a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.960+0000 7ff52f831b80 1 --2- 172.21.15.64:0/1232793887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daaaa76bb0 0x55daaaa76190 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.960+0000 7ff52f831b80 1 -- 172.21.15.64:0/1232793887 >> 172.21.15.64:0/1232793887 conn(0x55daaa9ea040 msgr2=0x55daaaa7e280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:43.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.960+0000 7ff52f831b80 1 -- 172.21.15.64:0/1232793887 shutdown_connections 2023-09-13T03:12:43.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.960+0000 7ff52f831b80 1 -- 172.21.15.64:0/1232793887 wait complete. 2023-09-13T03:12:43.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.960+0000 7ff52f831b80 1 -- start start 2023-09-13T03:12:43.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.960+0000 7ff52f831b80 1 Processor -- start 2023-09-13T03:12:43.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.961+0000 7ff52f831b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daaaa75db0 0x55daaaa81cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.961+0000 7ff52f831b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55daaaa76bb0 0x55daaaa82210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.961+0000 7ff52f831b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55daaaa77550 0x55daaaaded10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.961+0000 7ff52f831b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daaaa82860 con 0x55daaaa75db0 2023-09-13T03:12:43.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.961+0000 7ff52f831b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daaaadf250 con 0x55daaaa77550 2023-09-13T03:12:43.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.961+0000 7ff52f831b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daaaadf390 con 0x55daaaa76bb0 2023-09-13T03:12:43.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.961+0000 7ff52f831b80 10 client.0 _ll_register_callbacks cb 0x7ff50c0094d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:12:43.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.961+0000 7ff519d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daaaa75db0 0x55daaaa81cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.961+0000 7ff519d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daaaa75db0 0x55daaaa81cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54458/0 (socket says 172.21.15.64:54458) 2023-09-13T03:12:43.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.961+0000 7ff519d9b700 1 -- 172.21.15.64:0/3453643067 learned_addr learned my addr 172.21.15.64:0/3453643067 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:43.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.961+0000 7ff51a59c700 1 --2- 172.21.15.64:0/3453643067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55daaaa77550 0x55daaaaded10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.961+0000 7ff51959a700 1 --2- 172.21.15.64:0/3453643067 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55daaaa76bb0 0x55daaaa82210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.961+0000 7ff519d9b700 1 -- 172.21.15.64:0/3453643067 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55daaaa76bb0 msgr2=0x55daaaa82210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:43.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.961+0000 7ff519d9b700 1 --2- 172.21.15.64:0/3453643067 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55daaaa76bb0 0x55daaaa82210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.961+0000 7ff519d9b700 1 -- 172.21.15.64:0/3453643067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55daaaa77550 msgr2=0x55daaaaded10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:43.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.961+0000 7ff519d9b700 1 --2- 172.21.15.64:0/3453643067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55daaaa77550 0x55daaaaded10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.961+0000 7ff519d9b700 1 -- 172.21.15.64:0/3453643067 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55daaa7dcc10 con 0x55daaaa75db0 2023-09-13T03:12:43.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.961+0000 7ff51a59c700 1 --2- 172.21.15.64:0/3453643067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55daaaa77550 0x55daaaaded10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:12:43.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.962+0000 7ff519d9b700 1 --2- 172.21.15.64:0/3453643067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daaaa75db0 0x55daaaa81cd0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff510005950 tx=0x7ff5100092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c993665eb26514 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:43.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.962+0000 7ff4fd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:12:43.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.962+0000 7ff4fd7fa700 1 -- 172.21.15.64:0/3453643067 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51009d1f0 con 0x55daaaa75db0 2023-09-13T03:12:43.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.962+0000 7ff52f831b80 1 -- 172.21.15.64:0/3453643067 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff5100a1670 con 0x55daaaa75db0 2023-09-13T03:12:43.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.962+0000 7ff4fd7fa700 1 -- 172.21.15.64:0/3453643067 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff51001fde0 con 0x55daaaa75db0 2023-09-13T03:12:43.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.962+0000 7ff4fd7fa700 1 -- 172.21.15.64:0/3453643067 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5100840b0 con 0x55daaaa75db0 2023-09-13T03:12:43.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.962+0000 7ff52f831b80 10 client.15321 fetch_fsmap learned FSMap version 26 2023-09-13T03:12:43.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.962+0000 7ff52f831b80 1 -- 172.21.15.64:0/3453643067 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7ff5141093a0 con 0x55daaaa75db0 2023-09-13T03:12:43.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.962+0000 7ff4fd7fa700 1 -- 172.21.15.64:0/3453643067 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5100a1670 con 0x55daaaa75db0 2023-09-13T03:12:43.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.962+0000 7ff4fd7fa700 1 -- 172.21.15.64:0/3453643067 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff5100a11e0 con 0x55daaaa75db0 2023-09-13T03:12:43.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.962+0000 7ff52f831b80 10 client.15321 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:12:43.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.962+0000 7ff52f831b80 10 client.15321 resolve_mds: resolved name 'i' to daemon mds.i 2023-09-13T03:12:43.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.962+0000 7ff52f831b80 20 client.15321 populate_metadata read hostname 'smithi064' 2023-09-13T03:12:43.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.962+0000 7ff52f831b80 1 --2- 172.21.15.64:0/3453643067 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x55daaaae41e0 0x55daaaae0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:43.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.962+0000 7ff52f831b80 4 client.15321 mds_command: new command op to 24649 tid=0[{"prefix": "status"}] 2023-09-13T03:12:43.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.962+0000 7ff52f831b80 1 -- 172.21.15.64:0/3453643067 --> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55daaa9c08d0 con 0x55daaaae41e0 2023-09-13T03:12:43.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.963+0000 7ff51959a700 1 --2- 172.21.15.64:0/3453643067 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x55daaaae41e0 0x55daaaae0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:43.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.963+0000 7ff51959a700 1 --2- 172.21.15.64:0/3453643067 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x55daaaae41e0 0x55daaaae0680 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7427f8048779be31 server_cookie=474f6d47c1e27ce9 in_seq=0 out_seq=0 2023-09-13T03:12:43.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.963+0000 7ff4fd7fa700 10 client.15321 ms_handle_connect on v2:172.21.15.170:6838/2189703596 2023-09-13T03:12:43.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.964+0000 7ff4fd7fa700 1 -- 172.21.15.64:0/3453643067 <== mds.1 v2:172.21.15.170:6838/2189703596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55daaa9c08d0 con 0x55daaaae41e0 2023-09-13T03:12:43.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.964+0000 7ff4fd7fa700 10 client.15321 handle_command_reply: tid=0 2023-09-13T03:12:43.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.964+0000 7ff52f831b80 1 client.15321 shutdown 2023-09-13T03:12:43.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.964+0000 7ff52f831b80 1 -- 172.21.15.64:0/3453643067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daaaa75db0 msgr2=0x55daaaa81cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:43.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.964+0000 7ff52f831b80 1 --2- 172.21.15.64:0/3453643067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daaaa75db0 0x55daaaa81cd0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff510005950 tx=0x7ff5100092d0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.964+0000 7ff52f831b80 1 -- 172.21.15.64:0/3453643067 shutdown_connections 2023-09-13T03:12:43.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.964+0000 7ff52f831b80 1 --2- 172.21.15.64:0/3453643067 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x55daaaae41e0 0x55daaaae0680 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.964+0000 7ff52f831b80 1 --2- 172.21.15.64:0/3453643067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55daaaa77550 0x55daaaaded10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.964+0000 7ff52f831b80 1 --2- 172.21.15.64:0/3453643067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55daaaa75db0 0x55daaaa81cd0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.964+0000 7ff4fd7fa700 0 client.15321 ms_handle_reset on v2:172.21.15.170:6838/2189703596 2023-09-13T03:12:43.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.964+0000 7ff52f831b80 1 --2- 172.21.15.64:0/3453643067 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55daaaa76bb0 0x55daaaa82210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.964+0000 7ff52f831b80 1 -- 172.21.15.64:0/3453643067 >> 172.21.15.64:0/3453643067 conn(0x55daaa9ea040 msgr2=0x55daaaa76190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:43.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.964+0000 7ff52f831b80 1 -- 172.21.15.64:0/3453643067 shutdown_connections 2023-09-13T03:12:43.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.964+0000 7ff52f831b80 1 -- 172.21.15.64:0/3453643067 wait complete. 2023-09-13T03:12:43.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.964+0000 7ff52f831b80 20 client.15321 trim_cache size 0 max 16384 2023-09-13T03:12:43.980 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-13T03:12:43.980 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "f8dbced4-51e1-11ee-9ab7-7b867c8bd7da", 2023-09-13T03:12:43.980 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 1, 2023-09-13T03:12:43.981 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24649, 2023-09-13T03:12:43.981 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2023-09-13T03:12:43.981 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2023-09-13T03:12:43.981 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2023-09-13T03:12:43.981 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 89.776228009999997, 2023-09-13T03:12:43.982 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 26, 2023-09-13T03:12:43.982 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 85, 2023-09-13T03:12:43.982 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 83, 2023-09-13T03:12:43.982 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 108.639669758 2023-09-13T03:12:43.983 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-13T03:12:43.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.964+0000 7ff51bfff700 1 -- 172.21.15.64:0/3407848400 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ff5000a0640 msgr2=0x7ff5000a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:43.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.965+0000 7ff51bfff700 1 --2- 172.21.15.64:0/3407848400 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ff5000a0640 0x7ff5000a2b00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff50400b580 tx=0x7ff504005fb0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.965+0000 7ff51bfff700 1 -- 172.21.15.64:0/3407848400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff514100610 msgr2=0x7ff5141984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:43.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.965+0000 7ff51bfff700 1 --2- 172.21.15.64:0/3407848400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff514100610 0x7ff5141984c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff51000e970 tx=0x7ff51000ec80 comp rx=0 tx=0).stop 2023-09-13T03:12:43.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.965+0000 7ff51bfff700 1 -- 172.21.15.64:0/3407848400 shutdown_connections 2023-09-13T03:12:43.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.965+0000 7ff51bfff700 1 --2- 172.21.15.64:0/3407848400 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7ff5000a0640 0x7ff5000a2b00 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.965+0000 7ff51bfff700 1 --2- 172.21.15.64:0/3407848400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5141021a0 0x7ff51419fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.965+0000 7ff51bfff700 1 --2- 172.21.15.64:0/3407848400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff514100610 0x7ff5141984c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.965+0000 7ff51bfff700 1 --2- 172.21.15.64:0/3407848400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff514101800 0x7ff514198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:43.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.965+0000 7ff51bfff700 1 -- 172.21.15.64:0/3407848400 >> 172.21.15.64:0/3407848400 conn(0x7ff5140fbb80 msgr2=0x7ff5141070f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:43.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.965+0000 7ff51bfff700 1 -- 172.21.15.64:0/3407848400 shutdown_connections 2023-09-13T03:12:43.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:43.965+0000 7ff51bfff700 1 -- 172.21.15.64:0/3407848400 wait complete. 2023-09-13T03:12:43.987 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-09-13T03:12:43.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:43 smithi142 ceph-mon[134082]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-13T03:12:43.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:43 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/246091133' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:12:44.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.191+0000 7f3c29dff700 1 -- 172.21.15.64:0/3606572524 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c241005f0 msgr2=0x7f3c24100a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:44.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.191+0000 7f3c29dff700 1 --2- 172.21.15.64:0/3606572524 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c241005f0 0x7f3c24100a10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3c14009920 tx=0x7f3c14009c30 comp rx=0 tx=0).stop 2023-09-13T03:12:44.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.191+0000 7f3c29dff700 1 -- 172.21.15.64:0/3606572524 shutdown_connections 2023-09-13T03:12:44.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.191+0000 7f3c29dff700 1 --2- 172.21.15.64:0/3606572524 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c241005f0 0x7f3c24100a10 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:44.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.191+0000 7f3c29dff700 1 --2- 172.21.15.64:0/3606572524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c24102180 0x7f3c24108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:44.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.191+0000 7f3c29dff700 1 --2- 172.21.15.64:0/3606572524 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c241017e0 0x7f3c24101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:44.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.191+0000 7f3c29dff700 1 -- 172.21.15.64:0/3606572524 >> 172.21.15.64:0/3606572524 conn(0x7f3c240fbb40 msgr2=0x7f3c240fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:44.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.191+0000 7f3c29dff700 1 -- 172.21.15.64:0/3606572524 shutdown_connections 2023-09-13T03:12:44.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.191+0000 7f3c29dff700 1 -- 172.21.15.64:0/3606572524 wait complete. 2023-09-13T03:12:44.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.192+0000 7f3c29dff700 1 Processor -- start 2023-09-13T03:12:44.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.192+0000 7f3c29dff700 1 -- start start 2023-09-13T03:12:44.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.192+0000 7f3c29dff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c241005f0 0x7f3c241985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:44.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.193+0000 7f3c29dff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c241017e0 0x7f3c24198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:44.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.193+0000 7f3c29dff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c24102180 0x7f3c2419fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:44.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.193+0000 7f3c29dff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c241a0150 con 0x7f3c24102180 2023-09-13T03:12:44.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.193+0000 7f3c29dff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c241a0290 con 0x7f3c241017e0 2023-09-13T03:12:44.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.193+0000 7f3c29dff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c241a0420 con 0x7f3c241005f0 2023-09-13T03:12:44.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.193+0000 7f3c237fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c241005f0 0x7f3c241985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:44.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.193+0000 7f3c237fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c241005f0 0x7f3c241985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:50956/0 (socket says 172.21.15.64:50956) 2023-09-13T03:12:44.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.193+0000 7f3c237fe700 1 -- 172.21.15.64:0/2162613583 learned_addr learned my addr 172.21.15.64:0/2162613583 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:44.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.193+0000 7f3c23fff700 1 --2- 172.21.15.64:0/2162613583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c24102180 0x7f3c2419fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:44.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.193+0000 7f3c22ffd700 1 --2- 172.21.15.64:0/2162613583 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c241017e0 0x7f3c24198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:44.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.193+0000 7f3c237fe700 1 -- 172.21.15.64:0/2162613583 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c241017e0 msgr2=0x7f3c24198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:44.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.193+0000 7f3c237fe700 1 --2- 172.21.15.64:0/2162613583 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c241017e0 0x7f3c24198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:44.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.193+0000 7f3c237fe700 1 -- 172.21.15.64:0/2162613583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c24102180 msgr2=0x7f3c2419fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:44.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.193+0000 7f3c237fe700 1 --2- 172.21.15.64:0/2162613583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c24102180 0x7f3c2419fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:44.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.193+0000 7f3c237fe700 1 -- 172.21.15.64:0/2162613583 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c14009520 con 0x7f3c241005f0 2023-09-13T03:12:44.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.193+0000 7f3c23fff700 1 --2- 172.21.15.64:0/2162613583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c24102180 0x7f3c2419fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:44.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.194+0000 7f3c237fe700 1 --2- 172.21.15.64:0/2162613583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c241005f0 0x7f3c241985d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3c14009230 tx=0x7f3c1400b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:44.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.194+0000 7f3c20ff9700 1 -- 172.21.15.64:0/2162613583 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1401c070 con 0x7f3c241005f0 2023-09-13T03:12:44.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.194+0000 7f3c29dff700 1 -- 172.21.15.64:0/2162613583 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c241a06a0 con 0x7f3c241005f0 2023-09-13T03:12:44.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.194+0000 7f3c20ff9700 1 -- 172.21.15.64:0/2162613583 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c1400f7d0 con 0x7f3c241005f0 2023-09-13T03:12:44.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.194+0000 7f3c20ff9700 1 -- 172.21.15.64:0/2162613583 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c14004930 con 0x7f3c241005f0 2023-09-13T03:12:44.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.194+0000 7f3c29dff700 1 -- 172.21.15.64:0/2162613583 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c241a0ac0 con 0x7f3c241005f0 2023-09-13T03:12:44.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.197+0000 7f3c20ff9700 1 -- 172.21.15.64:0/2162613583 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f3c14020070 con 0x7f3c241005f0 2023-09-13T03:12:44.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.198+0000 7f3c3d69ab80 1 Processor -- start 2023-09-13T03:12:44.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.199+0000 7f3c20ff9700 1 --2- 172.21.15.64:0/2162613583 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f3c0c0a0640 0x7f3c0c0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:44.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.199+0000 7f3c3d69ab80 1 -- start start 2023-09-13T03:12:44.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.199+0000 7f3c20ff9700 1 -- 172.21.15.64:0/2162613583 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f3c140cc890 con 0x7f3c241005f0 2023-09-13T03:12:44.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.199+0000 7f3c23fff700 1 --2- 172.21.15.64:0/2162613583 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f3c0c0a0640 0x7f3c0c0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:44.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.199+0000 7f3c3d69ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56285ab81bb0 0x56285ab81fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:44.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.199+0000 7f3c3d69ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56285ab82550 0x56285ab807b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:44.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.199+0000 7f3c3d69ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56285ab80db0 0x56285ab81260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:44.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.199+0000 7f3c3d69ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56285aad96e0 con 0x56285ab80db0 2023-09-13T03:12:44.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.199+0000 7f3c3d69ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56285aad7000 con 0x56285ab82550 2023-09-13T03:12:44.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.199+0000 7f3c3d69ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56285aad28f0 con 0x56285ab81bb0 2023-09-13T03:12:44.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.200+0000 7f3c237fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56285ab81bb0 0x56285ab81fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:44.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.200+0000 7f3c22ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56285ab82550 0x56285ab807b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:44.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.200+0000 7f3c23fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56285ab80db0 0x56285ab81260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:44.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.200+0000 7f3c237fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56285ab81bb0 0x56285ab81fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:50968/0 (socket says 172.21.15.64:50968) 2023-09-13T03:12:44.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.200+0000 7f3c22ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56285ab82550 0x56285ab807b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:53382/0 (socket says 172.21.15.64:53382) 2023-09-13T03:12:44.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.200+0000 7f3c237fe700 1 -- 172.21.15.64:0/3578557494 learned_addr learned my addr 172.21.15.64:0/3578557494 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:44.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.200+0000 7f3c23fff700 1 --2- 172.21.15.64:0/2162613583 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f3c0c0a0640 0x7f3c0c0a2b00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c003e30 tx=0x7f3c1c00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:44.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.200+0000 7f3c237fe700 1 -- 172.21.15.64:0/3578557494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56285ab82550 msgr2=0x56285ab807b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:44.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.200+0000 7f3c237fe700 1 --2- 172.21.15.64:0/3578557494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56285ab82550 0x56285ab807b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:44.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.200+0000 7f3c237fe700 1 -- 172.21.15.64:0/3578557494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56285ab80db0 msgr2=0x56285ab81260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:44.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.200+0000 7f3c237fe700 1 --2- 172.21.15.64:0/3578557494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56285ab80db0 0x56285ab81260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:44.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.200+0000 7f3c237fe700 1 -- 172.21.15.64:0/3578557494 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56285a8fed90 con 0x56285ab81bb0 2023-09-13T03:12:44.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.200+0000 7f3c22ffd700 1 --2- 172.21.15.64:0/3578557494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56285ab82550 0x56285ab807b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:44.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.200+0000 7f3c23fff700 1 --2- 172.21.15.64:0/3578557494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56285ab80db0 0x56285ab81260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:44.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.200+0000 7f3c237fe700 1 --2- 172.21.15.64:0/3578557494 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56285ab81bb0 0x56285ab81fd0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3c140965e0 tx=0x7f3c1400f940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c6304ef60276118 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:44.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.200+0000 7f3c09ffb700 1 -- 172.21.15.64:0/3578557494 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1409df10 con 0x56285ab81bb0 2023-09-13T03:12:44.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.201+0000 7f3c09ffb700 1 -- 172.21.15.64:0/3578557494 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c1409d030 con 0x56285ab81bb0 2023-09-13T03:12:44.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.201+0000 7f3c09ffb700 1 -- 172.21.15.64:0/3578557494 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c140995d0 con 0x56285ab81bb0 2023-09-13T03:12:44.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.201+0000 7f3c3d69ab80 1 -- 172.21.15.64:0/3578557494 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56285ab81bb0 msgr2=0x56285ab81fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:44.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.201+0000 7f3c3d69ab80 1 --2- 172.21.15.64:0/3578557494 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56285ab81bb0 0x56285ab81fd0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3c140965e0 tx=0x7f3c1400f940 comp rx=0 tx=0).stop 2023-09-13T03:12:44.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.201+0000 7f3c3d69ab80 1 -- 172.21.15.64:0/3578557494 shutdown_connections 2023-09-13T03:12:44.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.201+0000 7f3c3d69ab80 1 --2- 172.21.15.64:0/3578557494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56285ab80db0 0x56285ab81260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:44.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.201+0000 7f3c3d69ab80 1 --2- 172.21.15.64:0/3578557494 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56285ab81bb0 0x56285ab81fd0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:44.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.201+0000 7f3c3d69ab80 1 --2- 172.21.15.64:0/3578557494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56285ab82550 0x56285ab807b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:44.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.201+0000 7f3c3d69ab80 1 -- 172.21.15.64:0/3578557494 >> 172.21.15.64:0/3578557494 conn(0x56285aaf5040 msgr2=0x56285ab7b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:44.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.201+0000 7f3c3d69ab80 1 -- 172.21.15.64:0/3578557494 shutdown_connections 2023-09-13T03:12:44.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.201+0000 7f3c3d69ab80 1 -- 172.21.15.64:0/3578557494 wait complete. 2023-09-13T03:12:44.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.202+0000 7f3c3d69ab80 1 -- start start 2023-09-13T03:12:44.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.202+0000 7f3c3d69ab80 1 Processor -- start 2023-09-13T03:12:44.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.203+0000 7f3c3d69ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56285ab80db0 0x56285ab8a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:44.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.203+0000 7f3c3d69ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56285ab81bb0 0x56285ab8a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:44.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.203+0000 7f3c3d69ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56285ab82550 0x56285ab91740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:44.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.203+0000 7f3c3d69ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56285aa70b20 con 0x56285ab81bb0 2023-09-13T03:12:44.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.203+0000 7f3c3d69ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56285aa70820 con 0x56285ab80db0 2023-09-13T03:12:44.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.203+0000 7f3c3d69ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56285a9548d0 con 0x56285ab82550 2023-09-13T03:12:44.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.203+0000 7f3c3d69ab80 10 client.0 _ll_register_callbacks cb 0x56285a9baa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:12:44.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.203+0000 7f3c22ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56285ab81bb0 0x56285ab8a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:44.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.203+0000 7f3c22ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56285ab81bb0 0x56285ab8a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54490/0 (socket says 172.21.15.64:54490) 2023-09-13T03:12:44.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.203+0000 7f3c22ffd700 1 -- 172.21.15.64:0/4108653446 learned_addr learned my addr 172.21.15.64:0/4108653446 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:44.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.204+0000 7f3c237fe700 1 --2- 172.21.15.64:0/4108653446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56285ab80db0 0x56285ab8a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:44.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.204+0000 7f3c23fff700 1 --2- 172.21.15.64:0/4108653446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56285ab82550 0x56285ab91740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:44.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.204+0000 7f3c22ffd700 1 -- 172.21.15.64:0/4108653446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56285ab82550 msgr2=0x56285ab91740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:44.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.204+0000 7f3c22ffd700 1 --2- 172.21.15.64:0/4108653446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56285ab82550 0x56285ab91740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:44.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.204+0000 7f3c22ffd700 1 -- 172.21.15.64:0/4108653446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56285ab80db0 msgr2=0x56285ab8a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:44.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.204+0000 7f3c22ffd700 1 --2- 172.21.15.64:0/4108653446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56285ab80db0 0x56285ab8a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:44.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.204+0000 7f3c22ffd700 1 -- 172.21.15.64:0/4108653446 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c1c004800 con 0x56285ab81bb0 2023-09-13T03:12:44.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.204+0000 7f3c237fe700 1 --2- 172.21.15.64:0/4108653446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56285ab80db0 0x56285ab8a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:12:44.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.204+0000 7f3c22ffd700 1 --2- 172.21.15.64:0/4108653446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56285ab81bb0 0x56285ab8a700 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3c18012980 tx=0x7f3c18012d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=240df5f190e45497 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:44.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.204+0000 7f3c02ffd700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:12:44.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.204+0000 7f3c02ffd700 1 -- 172.21.15.64:0/4108653446 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c18017070 con 0x56285ab81bb0 2023-09-13T03:12:44.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.204+0000 7f3c3d69ab80 1 -- 172.21.15.64:0/4108653446 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56285aa2a4a0 con 0x56285ab81bb0 2023-09-13T03:12:44.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.204+0000 7f3c02ffd700 1 -- 172.21.15.64:0/4108653446 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f3c18004030 con 0x56285ab81bb0 2023-09-13T03:12:44.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.205+0000 7f3c02ffd700 1 -- 172.21.15.64:0/4108653446 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c180041f0 con 0x56285ab81bb0 2023-09-13T03:12:44.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.205+0000 7f3c3d69ab80 10 client.15333 fetch_fsmap learned FSMap version 26 2023-09-13T03:12:44.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.205+0000 7f3c3d69ab80 1 -- 172.21.15.64:0/4108653446 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f3c180098e0 con 0x56285ab81bb0 2023-09-13T03:12:44.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.205+0000 7f3c02ffd700 1 -- 172.21.15.64:0/4108653446 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56285aa2a4a0 con 0x56285ab81bb0 2023-09-13T03:12:44.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.205+0000 7f3c02ffd700 1 -- 172.21.15.64:0/4108653446 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3c18003830 con 0x56285ab81bb0 2023-09-13T03:12:44.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.205+0000 7f3c3d69ab80 10 client.15333 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:12:44.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.205+0000 7f3c3d69ab80 10 client.15333 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-13T03:12:44.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.205+0000 7f3c3d69ab80 20 client.15333 populate_metadata read hostname 'smithi064' 2023-09-13T03:12:44.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.205+0000 7f3c3d69ab80 1 --2- 172.21.15.64:0/4108653446 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56285ab95920 0x56285ab948f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:44.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.205+0000 7f3c3d69ab80 4 client.15333 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:12:44.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.205+0000 7f3c3d69ab80 1 -- 172.21.15.64:0/4108653446 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56285aae7850 con 0x56285ab95920 2023-09-13T03:12:44.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.205+0000 7f3c237fe700 1 --2- 172.21.15.64:0/4108653446 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56285ab95920 0x56285ab948f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:44.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.206+0000 7f3c237fe700 1 --2- 172.21.15.64:0/4108653446 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56285ab95920 0x56285ab948f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5b2c8b68267366a server_cookie=9b0e9f86cdde9c9e in_seq=0 out_seq=0 2023-09-13T03:12:44.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.206+0000 7f3c02ffd700 10 client.15333 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:12:44.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.207+0000 7f3c02ffd700 1 -- 172.21.15.64:0/4108653446 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56285aae7850 con 0x56285ab95920 2023-09-13T03:12:44.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.207+0000 7f3c02ffd700 10 client.15333 handle_command_reply: tid=0 2023-09-13T03:12:44.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.207+0000 7f3c3d69ab80 1 client.15333 shutdown 2023-09-13T03:12:44.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.207+0000 7f3c3d69ab80 1 -- 172.21.15.64:0/4108653446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56285ab81bb0 msgr2=0x56285ab8a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:44.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.207+0000 7f3c3d69ab80 1 --2- 172.21.15.64:0/4108653446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56285ab81bb0 0x56285ab8a700 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3c18012980 tx=0x7f3c18012d40 comp rx=0 tx=0).stop 2023-09-13T03:12:44.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.207+0000 7f3c3d69ab80 1 -- 172.21.15.64:0/4108653446 shutdown_connections 2023-09-13T03:12:44.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.207+0000 7f3c3d69ab80 1 --2- 172.21.15.64:0/4108653446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56285ab80db0 0x56285ab8a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:44.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.207+0000 7f3c3d69ab80 1 --2- 172.21.15.64:0/4108653446 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56285ab95920 0x56285ab948f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:44.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.207+0000 7f3c3d69ab80 1 --2- 172.21.15.64:0/4108653446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56285ab82550 0x56285ab91740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:44.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.207+0000 7f3c3d69ab80 1 --2- 172.21.15.64:0/4108653446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56285ab81bb0 0x56285ab8a700 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:44.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.207+0000 7f3c02ffd700 0 client.15333 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:12:44.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.207+0000 7f3c3d69ab80 1 -- 172.21.15.64:0/4108653446 >> 172.21.15.64:0/4108653446 conn(0x56285aaf5040 msgr2=0x56285ab7ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:44.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.207+0000 7f3c3d69ab80 1 -- 172.21.15.64:0/4108653446 shutdown_connections 2023-09-13T03:12:44.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.207+0000 7f3c3d69ab80 1 -- 172.21.15.64:0/4108653446 wait complete. 2023-09-13T03:12:44.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.208+0000 7f3c3d69ab80 20 client.15333 trim_cache size 0 max 16384 2023-09-13T03:12:44.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.226+0000 7f3c3d69ab80 1 Processor -- start 2023-09-13T03:12:44.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.226+0000 7f3c3d69ab80 1 -- start start 2023-09-13T03:12:44.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.226+0000 7f3c3d69ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56285ab80db0 0x56285ab806c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:44.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.226+0000 7f3c3d69ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56285ab81bb0 0x56285ab91700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:44.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.226+0000 7f3c3d69ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56285ab82550 0x56285ab948f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:44.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.226+0000 7f3c3d69ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56285ab75d20 con 0x56285ab80db0 2023-09-13T03:12:44.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.226+0000 7f3c3d69ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56285a960de0 con 0x56285ab81bb0 2023-09-13T03:12:44.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.226+0000 7f3c3d69ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56285aa9bf70 con 0x56285ab82550 2023-09-13T03:12:44.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.227+0000 7f3c237fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56285ab80db0 0x56285ab806c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:44.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.227+0000 7f3c23fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56285ab82550 0x56285ab948f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:44.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.227+0000 7f3c237fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56285ab80db0 0x56285ab806c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54492/0 (socket says 172.21.15.64:54492) 2023-09-13T03:12:44.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.227+0000 7f3c237fe700 1 -- 172.21.15.64:0/1671472946 learned_addr learned my addr 172.21.15.64:0/1671472946 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:44.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.227+0000 7f3c22ffd700 1 --2- 172.21.15.64:0/1671472946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56285ab81bb0 0x56285ab91700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:44.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.227+0000 7f3c237fe700 1 -- 172.21.15.64:0/1671472946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56285ab82550 msgr2=0x56285ab948f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:44.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.227+0000 7f3c237fe700 1 --2- 172.21.15.64:0/1671472946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56285ab82550 0x56285ab948f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:44.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.227+0000 7f3c237fe700 1 -- 172.21.15.64:0/1671472946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56285ab81bb0 msgr2=0x56285ab91700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:44.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.227+0000 7f3c237fe700 1 --2- 172.21.15.64:0/1671472946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56285ab81bb0 0x56285ab91700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:44.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.227+0000 7f3c237fe700 1 -- 172.21.15.64:0/1671472946 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c241a0ac0 con 0x56285ab80db0 2023-09-13T03:12:44.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.227+0000 7f3c237fe700 1 --2- 172.21.15.64:0/1671472946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56285ab80db0 0x56285ab806c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3c140965e0 tx=0x7f3c14009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f698a6db659c6ad server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:44.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.227+0000 7f3c0a7fc700 1 -- 172.21.15.64:0/1671472946 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1409df90 con 0x56285ab80db0 2023-09-13T03:12:44.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.228+0000 7f3c0a7fc700 1 -- 172.21.15.64:0/1671472946 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c14098480 con 0x56285ab80db0 2023-09-13T03:12:44.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.228+0000 7f3c0a7fc700 1 -- 172.21.15.64:0/1671472946 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c140995d0 con 0x56285ab80db0 2023-09-13T03:12:44.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.228+0000 7f3c3d69ab80 1 -- 172.21.15.64:0/1671472946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56285ab80db0 msgr2=0x56285ab806c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:44.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.228+0000 7f3c3d69ab80 1 --2- 172.21.15.64:0/1671472946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56285ab80db0 0x56285ab806c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3c140965e0 tx=0x7f3c14009c30 comp rx=0 tx=0).stop 2023-09-13T03:12:44.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.228+0000 7f3c3d69ab80 1 -- 172.21.15.64:0/1671472946 shutdown_connections 2023-09-13T03:12:44.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.228+0000 7f3c3d69ab80 1 --2- 172.21.15.64:0/1671472946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56285ab82550 0x56285ab948f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:44.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.228+0000 7f3c3d69ab80 1 --2- 172.21.15.64:0/1671472946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56285ab80db0 0x56285ab806c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:44.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.228+0000 7f3c3d69ab80 1 --2- 172.21.15.64:0/1671472946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56285ab81bb0 0x56285ab91700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:44.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.228+0000 7f3c3d69ab80 1 -- 172.21.15.64:0/1671472946 >> 172.21.15.64:0/1671472946 conn(0x56285aaf5040 msgr2=0x56285ab7b640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:44.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.228+0000 7f3c3d69ab80 1 -- 172.21.15.64:0/1671472946 shutdown_connections 2023-09-13T03:12:44.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.228+0000 7f3c3d69ab80 1 -- 172.21.15.64:0/1671472946 wait complete. 2023-09-13T03:12:44.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.229+0000 7f3c3d69ab80 1 -- start start 2023-09-13T03:12:44.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.229+0000 7f3c3d69ab80 1 Processor -- start 2023-09-13T03:12:44.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.229+0000 7f3c3d69ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56285ab80db0 0x56285ab8c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:44.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.229+0000 7f3c3d69ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56285ab81bb0 0x56285ab8ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:44.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.229+0000 7f3c3d69ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56285ab82550 0x56285ab8fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:44.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.229+0000 7f3c3d69ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56285ab903d0 con 0x56285ab80db0 2023-09-13T03:12:44.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.229+0000 7f3c3d69ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56285ab90540 con 0x56285ab82550 2023-09-13T03:12:44.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.229+0000 7f3c3d69ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56285ab90700 con 0x56285ab81bb0 2023-09-13T03:12:44.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.229+0000 7f3c3d69ab80 10 client.0 _ll_register_callbacks cb 0x7f3c18012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:12:44.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.230+0000 7f3c237fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56285ab80db0 0x56285ab8c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:44.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.230+0000 7f3c237fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56285ab80db0 0x56285ab8c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54494/0 (socket says 172.21.15.64:54494) 2023-09-13T03:12:44.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.230+0000 7f3c237fe700 1 -- 172.21.15.64:0/1317210893 learned_addr learned my addr 172.21.15.64:0/1317210893 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:44.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.230+0000 7f3c23fff700 1 --2- 172.21.15.64:0/1317210893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56285ab82550 0x56285ab8fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:44.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.230+0000 7f3c22ffd700 1 --2- 172.21.15.64:0/1317210893 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56285ab81bb0 0x56285ab8ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:44.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.230+0000 7f3c237fe700 1 -- 172.21.15.64:0/1317210893 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56285ab81bb0 msgr2=0x56285ab8ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:44.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.230+0000 7f3c237fe700 1 --2- 172.21.15.64:0/1317210893 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56285ab81bb0 0x56285ab8ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:44.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.230+0000 7f3c237fe700 1 -- 172.21.15.64:0/1317210893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56285ab82550 msgr2=0x56285ab8fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:44.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.230+0000 7f3c237fe700 1 --2- 172.21.15.64:0/1317210893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56285ab82550 0x56285ab8fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:44.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.230+0000 7f3c237fe700 1 -- 172.21.15.64:0/1317210893 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56285a8fed90 con 0x56285ab80db0 2023-09-13T03:12:44.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.230+0000 7f3c23fff700 1 --2- 172.21.15.64:0/1317210893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56285ab82550 0x56285ab8fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:12:44.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.230+0000 7f3c237fe700 1 --2- 172.21.15.64:0/1317210893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56285ab80db0 0x56285ab8c930 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3c1400a010 tx=0x7f3c14005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15c9528c5171a41 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:44.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.230+0000 7f3c02ffd700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:12:44.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.230+0000 7f3c02ffd700 1 -- 172.21.15.64:0/1317210893 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1409df90 con 0x56285ab80db0 2023-09-13T03:12:44.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.230+0000 7f3c02ffd700 1 -- 172.21.15.64:0/1317210893 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f3c14098220 con 0x56285ab80db0 2023-09-13T03:12:44.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.230+0000 7f3c3d69ab80 1 -- 172.21.15.64:0/1317210893 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3c1409d030 con 0x56285ab80db0 2023-09-13T03:12:44.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.231+0000 7f3c02ffd700 1 -- 172.21.15.64:0/1317210893 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c14099eb0 con 0x56285ab80db0 2023-09-13T03:12:44.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.231+0000 7f3c3d69ab80 10 client.15339 fetch_fsmap learned FSMap version 26 2023-09-13T03:12:44.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.231+0000 7f3c3d69ab80 1 -- 172.21.15.64:0/1317210893 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56285a8e7c10 con 0x56285ab80db0 2023-09-13T03:12:44.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.231+0000 7f3c02ffd700 1 -- 172.21.15.64:0/1317210893 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c1409d030 con 0x56285ab80db0 2023-09-13T03:12:44.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.231+0000 7f3c02ffd700 1 -- 172.21.15.64:0/1317210893 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3c140a1c30 con 0x56285ab80db0 2023-09-13T03:12:44.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.231+0000 7f3c3d69ab80 10 client.15339 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:12:44.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.231+0000 7f3c3d69ab80 10 client.15339 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-13T03:12:44.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.231+0000 7f3c3d69ab80 20 client.15339 populate_metadata read hostname 'smithi064' 2023-09-13T03:12:44.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.231+0000 7f3c3d69ab80 1 --2- 172.21.15.64:0/1317210893 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56285abef150 0x56285abeb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:44.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.231+0000 7f3c3d69ab80 4 client.15339 mds_command: new command op to 24686 tid=0[{"prefix": "status"}] 2023-09-13T03:12:44.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.231+0000 7f3c3d69ab80 1 -- 172.21.15.64:0/1317210893 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56285aacb8d0 con 0x56285abef150 2023-09-13T03:12:44.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.231+0000 7f3c22ffd700 1 --2- 172.21.15.64:0/1317210893 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56285abef150 0x56285abeb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:44.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.232+0000 7f3c02ffd700 10 client.15339 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:12:44.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.232+0000 7f3c22ffd700 1 --2- 172.21.15.64:0/1317210893 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56285abef150 0x56285abeb980 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=447c4505633ecf4a server_cookie=b3c142545ad12f80 in_seq=0 out_seq=0 2023-09-13T03:12:44.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.232+0000 7f3c02ffd700 1 -- 172.21.15.64:0/1317210893 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x56285aacb8d0 con 0x56285abef150 2023-09-13T03:12:44.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.232+0000 7f3c02ffd700 10 client.15339 handle_command_reply: tid=0 2023-09-13T03:12:44.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.232+0000 7f3c3d69ab80 1 client.15339 shutdown 2023-09-13T03:12:44.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.232+0000 7f3c3d69ab80 1 -- 172.21.15.64:0/1317210893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56285ab80db0 msgr2=0x56285ab8c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:44.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.232+0000 7f3c3d69ab80 1 --2- 172.21.15.64:0/1317210893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56285ab80db0 0x56285ab8c930 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3c1400a010 tx=0x7f3c14005e20 comp rx=0 tx=0).stop 2023-09-13T03:12:44.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.232+0000 7f3c3d69ab80 1 -- 172.21.15.64:0/1317210893 shutdown_connections 2023-09-13T03:12:44.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.232+0000 7f3c3d69ab80 1 --2- 172.21.15.64:0/1317210893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56285ab82550 0x56285ab8fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:44.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.233+0000 7f3c3d69ab80 1 --2- 172.21.15.64:0/1317210893 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56285abef150 0x56285abeb980 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:44.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.233+0000 7f3c3d69ab80 1 --2- 172.21.15.64:0/1317210893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56285ab80db0 0x56285ab8c930 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:44.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.233+0000 7f3c3d69ab80 1 --2- 172.21.15.64:0/1317210893 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56285ab81bb0 0x56285ab8ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:44.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.233+0000 7f3c02ffd700 0 client.15339 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:12:44.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.233+0000 7f3c3d69ab80 1 -- 172.21.15.64:0/1317210893 >> 172.21.15.64:0/1317210893 conn(0x56285aaf5040 msgr2=0x56285ab948f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:44.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.233+0000 7f3c3d69ab80 1 -- 172.21.15.64:0/1317210893 shutdown_connections 2023-09-13T03:12:44.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.233+0000 7f3c3d69ab80 1 -- 172.21.15.64:0/1317210893 wait complete. 2023-09-13T03:12:44.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.233+0000 7f3c3d69ab80 20 client.15339 trim_cache size 0 max 16384 2023-09-13T03:12:44.249 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-13T03:12:44.250 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "f8dbced4-51e1-11ee-9ab7-7b867c8bd7da", 2023-09-13T03:12:44.250 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 0, 2023-09-13T03:12:44.250 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24686, 2023-09-13T03:12:44.250 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2023-09-13T03:12:44.251 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2023-09-13T03:12:44.251 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2023-09-13T03:12:44.251 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 93.271158729999996, 2023-09-13T03:12:44.251 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 26, 2023-09-13T03:12:44.251 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 85, 2023-09-13T03:12:44.251 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 83, 2023-09-13T03:12:44.252 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 100.216717319 2023-09-13T03:12:44.252 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-13T03:12:44.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.233+0000 7f3c29dff700 1 -- 172.21.15.64:0/2162613583 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f3c0c0a0640 msgr2=0x7f3c0c0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:44.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.233+0000 7f3c29dff700 1 --2- 172.21.15.64:0/2162613583 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f3c0c0a0640 0x7f3c0c0a2b00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c003e30 tx=0x7f3c1c00b040 comp rx=0 tx=0).stop 2023-09-13T03:12:44.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.233+0000 7f3c29dff700 1 -- 172.21.15.64:0/2162613583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c241005f0 msgr2=0x7f3c241985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:44.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.233+0000 7f3c29dff700 1 --2- 172.21.15.64:0/2162613583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c241005f0 0x7f3c241985d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3c14009230 tx=0x7f3c1400b790 comp rx=0 tx=0).stop 2023-09-13T03:12:44.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.233+0000 7f3c29dff700 1 -- 172.21.15.64:0/2162613583 shutdown_connections 2023-09-13T03:12:44.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.233+0000 7f3c29dff700 1 --2- 172.21.15.64:0/2162613583 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f3c0c0a0640 0x7f3c0c0a2b00 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:44.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.233+0000 7f3c29dff700 1 --2- 172.21.15.64:0/2162613583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c24102180 0x7f3c2419fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:44.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.233+0000 7f3c29dff700 1 --2- 172.21.15.64:0/2162613583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c241005f0 0x7f3c241985d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:44.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.233+0000 7f3c29dff700 1 --2- 172.21.15.64:0/2162613583 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c241017e0 0x7f3c24198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:44.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.233+0000 7f3c29dff700 1 -- 172.21.15.64:0/2162613583 >> 172.21.15.64:0/2162613583 conn(0x7f3c240fbb40 msgr2=0x7f3c240fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:44.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.234+0000 7f3c29dff700 1 -- 172.21.15.64:0/2162613583 shutdown_connections 2023-09-13T03:12:44.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:44.234+0000 7f3c29dff700 1 -- 172.21.15.64:0/2162613583 wait complete. 2023-09-13T03:12:44.256 INFO:teuthology.run_tasks:Running task kclient... 2023-09-13T03:12:44.282 INFO:tasks.kclient:Mounting kernel clients... 2023-09-13T03:12:44.282 INFO:tasks.kclient:config is {'client.0': {}} 2023-09-13T03:12:44.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T03:12:44.283 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2023-09-13T03:12:44.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T03:12:44.329 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link delete ceph-brx 2023-09-13T03:12:44.442 INFO:teuthology.orchestra.run.smithi064.stderr:Cannot find device "ceph-brx" 2023-09-13T03:12:44.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-13T03:12:44.445 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2023-09-13T03:12:44.445 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-09-13T03:12:44.446 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-09-13T03:12:44.446 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph fs ls 2023-09-13T03:12:45.179 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:12:45.483 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:45 smithi064 ceph-mon[97675]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-13T03:12:45.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:45 smithi142 ceph-mon[134082]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-13T03:12:45.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:45 smithi170 ceph-mon[134328]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-13T03:12:46.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.537+0000 7f00f4ebe700 1 -- 172.21.15.64:0/1442565597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00f0106620 msgr2=0x7f00f0106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:46.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.537+0000 7f00f4ebe700 1 --2- 172.21.15.64:0/1442565597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00f0106620 0x7f00f0106a00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f00e0009990 tx=0x7f00e0009ca0 comp rx=0 tx=0).stop 2023-09-13T03:12:46.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.537+0000 7f00f4ebe700 1 -- 172.21.15.64:0/1442565597 shutdown_connections 2023-09-13T03:12:46.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.537+0000 7f00f4ebe700 1 --2- 172.21.15.64:0/1442565597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00f0100fe0 0x7f00f010d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:46.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.537+0000 7f00f4ebe700 1 --2- 172.21.15.64:0/1442565597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00f0106620 0x7f00f0106a00 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:46.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.537+0000 7f00f4ebe700 1 --2- 172.21.15.64:0/1442565597 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00f0100620 0x7f00f0100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:46.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.537+0000 7f00f4ebe700 1 -- 172.21.15.64:0/1442565597 >> 172.21.15.64:0/1442565597 conn(0x7f00f00fc0f0 msgr2=0x7f00f00fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:46.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.537+0000 7f00f4ebe700 1 -- 172.21.15.64:0/1442565597 shutdown_connections 2023-09-13T03:12:46.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.537+0000 7f00f4ebe700 1 -- 172.21.15.64:0/1442565597 wait complete. 2023-09-13T03:12:46.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.538+0000 7f00f4ebe700 1 Processor -- start 2023-09-13T03:12:46.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.538+0000 7f00f4ebe700 1 -- start start 2023-09-13T03:12:46.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.539+0000 7f00f4ebe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00f0100620 0x7f00f0198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:46.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.539+0000 7f00f4ebe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00f0100fe0 0x7f00f0198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:46.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.539+0000 7f00f4ebe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00f0106620 0x7f00f019d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:46.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.539+0000 7f00f4ebe700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f019d900 con 0x7f00f0100fe0 2023-09-13T03:12:46.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.539+0000 7f00f4ebe700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f019da70 con 0x7f00f0106620 2023-09-13T03:12:46.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.539+0000 7f00f4ebe700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f019dc30 con 0x7f00f0100620 2023-09-13T03:12:46.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.539+0000 7f00ee59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00f0100620 0x7f00f0198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:46.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.539+0000 7f00ee59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00f0100620 0x7f00f0198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:35764/0 (socket says 172.21.15.64:35764) 2023-09-13T03:12:46.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.539+0000 7f00ee59c700 1 -- 172.21.15.64:0/900693732 learned_addr learned my addr 172.21.15.64:0/900693732 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:46.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.539+0000 7f00edd9b700 1 --2- 172.21.15.64:0/900693732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00f0100fe0 0x7f00f0198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:46.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.539+0000 7f00eed9d700 1 --2- 172.21.15.64:0/900693732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00f0106620 0x7f00f019d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:46.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.539+0000 7f00ee59c700 1 -- 172.21.15.64:0/900693732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00f0106620 msgr2=0x7f00f019d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:46.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.539+0000 7f00ee59c700 1 --2- 172.21.15.64:0/900693732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00f0106620 0x7f00f019d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:46.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.539+0000 7f00ee59c700 1 -- 172.21.15.64:0/900693732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00f0100fe0 msgr2=0x7f00f0198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:46.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.539+0000 7f00ee59c700 1 --2- 172.21.15.64:0/900693732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00f0100fe0 0x7f00f0198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:46.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.540+0000 7f00ee59c700 1 -- 172.21.15.64:0/900693732 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e00095f0 con 0x7f00f0100620 2023-09-13T03:12:46.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.540+0000 7f00edd9b700 1 --2- 172.21.15.64:0/900693732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00f0100fe0 0x7f00f0198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:46.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.540+0000 7f00ee59c700 1 --2- 172.21.15.64:0/900693732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00f0100620 0x7f00f0198950 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f00e0000c00 tx=0x7f00e00047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:46.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.540+0000 7f00df7fe700 1 -- 172.21.15.64:0/900693732 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e001d070 con 0x7f00f0100620 2023-09-13T03:12:46.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.540+0000 7f00df7fe700 1 -- 172.21.15.64:0/900693732 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00e000fb80 con 0x7f00f0100620 2023-09-13T03:12:46.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.540+0000 7f00f4ebe700 1 -- 172.21.15.64:0/900693732 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00f019deb0 con 0x7f00f0100620 2023-09-13T03:12:46.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.540+0000 7f00df7fe700 1 -- 172.21.15.64:0/900693732 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e00173f0 con 0x7f00f0100620 2023-09-13T03:12:46.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.540+0000 7f00f4ebe700 1 -- 172.21.15.64:0/900693732 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00f019e180 con 0x7f00f0100620 2023-09-13T03:12:46.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.541+0000 7f00f4ebe700 1 -- 172.21.15.64:0/900693732 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00f004e300 con 0x7f00f0100620 2023-09-13T03:12:46.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.543+0000 7f00df7fe700 1 -- 172.21.15.64:0/900693732 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f00e000fcf0 con 0x7f00f0100620 2023-09-13T03:12:46.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.544+0000 7f00df7fe700 1 --2- 172.21.15.64:0/900693732 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f00d80a0640 0x7f00d80a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:46.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.545+0000 7f00df7fe700 1 -- 172.21.15.64:0/900693732 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f00e00cd2e0 con 0x7f00f0100620 2023-09-13T03:12:46.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.545+0000 7f00edd9b700 1 --2- 172.21.15.64:0/900693732 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f00d80a0640 0x7f00d80a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:46.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.545+0000 7f00edd9b700 1 --2- 172.21.15.64:0/900693732 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f00d80a0640 0x7f00d80a2b00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f00f0199f70 tx=0x7f00e4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:46.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.547+0000 7f00df7fe700 1 -- 172.21.15.64:0/900693732 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f00e0096e70 con 0x7f00f0100620 2023-09-13T03:12:46.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.803+0000 7f00f4ebe700 1 -- 172.21.15.64:0/900693732 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f00f0199cb0 con 0x7f00f0100620 2023-09-13T03:12:46.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.804+0000 7f00df7fe700 1 -- 172.21.15.64:0/900693732 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) v1 ==== 53+0+73 (secure 0 0 0) 0x7f00e009a490 con 0x7f00f0100620 2023-09-13T03:12:46.805 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-09-13T03:12:46.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.806+0000 7f00f4ebe700 1 -- 172.21.15.64:0/900693732 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f00d80a0640 msgr2=0x7f00d80a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:46.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.806+0000 7f00f4ebe700 1 --2- 172.21.15.64:0/900693732 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f00d80a0640 0x7f00d80a2b00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f00f0199f70 tx=0x7f00e4009260 comp rx=0 tx=0).stop 2023-09-13T03:12:46.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.806+0000 7f00f4ebe700 1 -- 172.21.15.64:0/900693732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00f0100620 msgr2=0x7f00f0198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:46.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.806+0000 7f00f4ebe700 1 --2- 172.21.15.64:0/900693732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00f0100620 0x7f00f0198950 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f00e0000c00 tx=0x7f00e00047b0 comp rx=0 tx=0).stop 2023-09-13T03:12:46.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.806+0000 7f00f4ebe700 1 -- 172.21.15.64:0/900693732 shutdown_connections 2023-09-13T03:12:46.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.806+0000 7f00f4ebe700 1 --2- 172.21.15.64:0/900693732 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f00d80a0640 0x7f00d80a2b00 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:46.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.806+0000 7f00f4ebe700 1 --2- 172.21.15.64:0/900693732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00f0106620 0x7f00f019d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:46.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.806+0000 7f00f4ebe700 1 --2- 172.21.15.64:0/900693732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00f0100620 0x7f00f0198950 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:46.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.806+0000 7f00f4ebe700 1 --2- 172.21.15.64:0/900693732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00f0100fe0 0x7f00f0198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:46.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.806+0000 7f00f4ebe700 1 -- 172.21.15.64:0/900693732 >> 172.21.15.64:0/900693732 conn(0x7f00f00fc0f0 msgr2=0x7f00f010bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:46.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.807+0000 7f00f4ebe700 1 -- 172.21.15.64:0/900693732 shutdown_connections 2023-09-13T03:12:46.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:46.807+0000 7f00f4ebe700 1 -- 172.21.15.64:0/900693732 wait complete. 2023-09-13T03:12:47.258 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-13T03:12:47.259 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-13T03:12:47.259 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2023-09-13T03:12:47.259 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-09-13T03:12:47.260 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-09-13T03:12:47.260 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-13T03:12:47.260 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-13T03:12:47.260 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-13T03:12:47.261 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-13T03:12:47.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T03:12:47.261 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2023-09-13T03:12:47.306 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-13T03:12:47.306 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-13T03:12:47.306 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2023-09-13T03:12:47.306 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-13T03:12:47.307 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2023-09-13T03:12:47.307 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-13T03:12:47.307 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-13T03:12:47.307 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2023-09-13T03:12:47.307 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f0 2023-09-13T03:12:47.308 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-13T03:12:47.308 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2023-09-13T03:12:47.308 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f1 2023-09-13T03:12:47.308 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-13T03:12:47.308 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2023-09-13T03:12:47.309 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-13T03:12:47.309 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2023-09-13T03:12:47.309 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-09-13T03:12:47.309 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 41668sec preferred_lft 41668sec 2023-09-13T03:12:47.309 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::7de2:f971:923f:f167/64 scope link noprefixroute 2023-09-13T03:12:47.310 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-09-13T03:12:47.311 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-09-13T03:12:47.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T03:12:47.312 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-09-13T03:12:47.312 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-09-13T03:12:47.312 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link add name ceph-brx type bridge 2023-09-13T03:12:47.313 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip addr flush dev ceph-brx 2023-09-13T03:12:47.313 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set ceph-brx up 2023-09-13T03:12:47.313 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-09-13T03:12:47.313 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-09-13T03:12:47.748 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:47 smithi064 ceph-mon[97675]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-13T03:12:47.749 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:47 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/900693732' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-09-13T03:12:47.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T03:12:47.851 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-13T03:12:47.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:47 smithi170 ceph-mon[134328]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-13T03:12:47.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:47 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/900693732' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-09-13T03:12:47.925 INFO:teuthology.orchestra.run.smithi064.stdout:1 2023-09-13T03:12:47.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T03:12:47.929 DEBUG:teuthology.orchestra.run.smithi064:> route 2023-09-13T03:12:47.991 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2023-09-13T03:12:47.992 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-13T03:12:47.992 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-13T03:12:47.992 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-09-13T03:12:47.992 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-13T03:12:47.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T03:12:47.994 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-09-13T03:12:47.994 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-09-13T03:12:47.994 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-13T03:12:47.995 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-13T03:12:47.995 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-13T03:12:47.995 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-09-13T03:12:47.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:47 smithi142 ceph-mon[134082]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-13T03:12:47.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:47 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/900693732' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-09-13T03:12:48.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T03:12:48.689 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2023-09-13T03:12:48.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T03:12:48.735 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list-id 2023-09-13T03:12:48.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T03:12:48.818 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-09-13T03:12:48.818 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-09-13T03:12:48.818 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-13T03:12:48.819 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-09-13T03:12:48.819 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-09-13T03:12:49.006 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-09-13T03:12:49.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T03:12:49.007 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-09-13T03:12:49.007 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-09-13T03:12:49.007 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-09-13T03:12:49.008 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-09-13T03:12:49.008 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-09-13T03:12:49.008 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-09-13T03:12:49.009 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-09-13T03:12:49.009 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-09-13T03:12:49.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T03:12:49.299 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-09-13T03:12:49.300 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-09-13T03:12:49.300 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.0 up 2023-09-13T03:12:49.300 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set dev brx.0 master ceph-brx 2023-09-13T03:12:49.300 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-09-13T03:12:49.477 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-13T03:12:49.478 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-09-13T03:12:49.522 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-09-13T03:12:49.522 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-13T03:12:49.522 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-09-13T03:12:49.606 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2023-09-13T03:12:49.606 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2023-09-13T03:12:49.607 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed 2023-09-13T03:12:49.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T03:12:49.607 DEBUG:teuthology.orchestra.run.smithi064:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync 2023-09-13T03:12:49.761 INFO:teuthology.orchestra.run.smithi064.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync 2023-09-13T03:12:49.761 INFO:teuthology.orchestra.run.smithi064.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync". 2023-09-13T03:12:49.762 INFO:teuthology.orchestra.run.smithi064.stdout:invalid new device string format 2023-09-13T03:12:49.852 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:49 smithi064 ceph-mon[97675]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-13T03:12:49.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:49 smithi170 ceph-mon[134328]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-13T03:12:49.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:49 smithi142 ceph-mon[134082]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-13T03:12:50.208 INFO:teuthology.orchestra.run.smithi064.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync 2023-09-13T03:12:50.209 INFO:teuthology.orchestra.run.smithi064.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync". 2023-09-13T03:12:50.209 INFO:teuthology.orchestra.run.smithi064.stdout:invalid new device string format 2023-09-13T03:12:50.209 INFO:teuthology.orchestra.run.smithi064.stdout:mount.ceph: resolved to: "172.21.15.64:6789,172.21.15.142:6789,172.21.15.170:6789" 2023-09-13T03:12:50.209 INFO:teuthology.orchestra.run.smithi064.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.64:6789,172.21.15.142:6789,172.21.15.170:6789:/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed 2023-09-13T03:12:50.210 INFO:teuthology.orchestra.run.smithi064.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync,key=0,fsid=f8dbced4-51e1-11ee-9ab7-7b867c8bd7da" will pass to kernel 2023-09-13T03:12:50.214 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-09-13T03:12:50.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T03:12:50.215 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-09-13T03:12:50.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T03:12:50.293 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-09-13T03:12:50.387 INFO:teuthology.orchestra.run.smithi064.stdout:client25375 2023-09-13T03:12:50.388 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-13T03:12:50.388 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-13T03:12:50.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T03:12:50.433 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-09-13T03:12:50.560 INFO:teuthology.orchestra.run.smithi064.stdout:client25375 2023-09-13T03:12:50.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T03:12:50.561 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-09-13T03:12:50.647 INFO:teuthology.orchestra.run.smithi064.stdout:client25375 2023-09-13T03:12:50.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T03:12:50.647 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da.client25375/status') 2023-09-13T03:12:50.757 INFO:teuthology.orchestra.run.smithi064.stdout:instance: client.25375 (3)172.21.15.64:0/1618608350 2023-09-13T03:12:50.757 INFO:teuthology.orchestra.run.smithi064.stdout:blocklisted: false 2023-09-13T03:12:50.758 INFO:teuthology.orchestra.run.smithi064.stderr:0+1 records in 2023-09-13T03:12:50.758 INFO:teuthology.orchestra.run.smithi064.stderr:0+1 records out 2023-09-13T03:12:50.758 INFO:teuthology.orchestra.run.smithi064.stderr:71 bytes copied, 5.5545e-05 s, 1.3 MB/s 2023-09-13T03:12:50.760 INFO:teuthology.run_tasks:Running task check-counter... 2023-09-13T03:12:50.776 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-09-13T03:12:50.846 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-13T03:12:51.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.050+0000 7f0c249ff700 1 -- 172.21.15.64:0/1138479177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c20101800 msgr2=0x7f0c20101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:51.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.050+0000 7f0c249ff700 1 --2- 172.21.15.64:0/1138479177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c20101800 0x7f0c20101c60 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0c08009920 tx=0x7f0c08009c30 comp rx=0 tx=0).stop 2023-09-13T03:12:51.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.050+0000 7f0c249ff700 1 -- 172.21.15.64:0/1138479177 shutdown_connections 2023-09-13T03:12:51.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.050+0000 7f0c249ff700 1 --2- 172.21.15.64:0/1138479177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c20100610 0x7f0c20100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:51.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.050+0000 7f0c249ff700 1 --2- 172.21.15.64:0/1138479177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c201021a0 0x7f0c20108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:51.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.050+0000 7f0c249ff700 1 --2- 172.21.15.64:0/1138479177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c20101800 0x7f0c20101c60 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:51.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.050+0000 7f0c249ff700 1 -- 172.21.15.64:0/1138479177 >> 172.21.15.64:0/1138479177 conn(0x7f0c200fbb40 msgr2=0x7f0c200fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:51.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.051+0000 7f0c249ff700 1 -- 172.21.15.64:0/1138479177 shutdown_connections 2023-09-13T03:12:51.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.051+0000 7f0c249ff700 1 -- 172.21.15.64:0/1138479177 wait complete. 2023-09-13T03:12:51.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.052+0000 7f0c249ff700 1 Processor -- start 2023-09-13T03:12:51.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.052+0000 7f0c249ff700 1 -- start start 2023-09-13T03:12:51.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.052+0000 7f0c249ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c20100610 0x7f0c20198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:51.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.052+0000 7f0c249ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c20101800 0x7f0c20198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:51.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.052+0000 7f0c249ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c201021a0 0x7f0c2019fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:51.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.052+0000 7f0c249ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c201a00f0 con 0x7f0c20101800 2023-09-13T03:12:51.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.052+0000 7f0c249ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c201a0230 con 0x7f0c201021a0 2023-09-13T03:12:51.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.052+0000 7f0c249ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c201a03c0 con 0x7f0c20100610 2023-09-13T03:12:51.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.052+0000 7f0c17fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c20101800 0x7f0c20198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:51.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.052+0000 7f0c17fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c20101800 0x7f0c20198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40932/0 (socket says 172.21.15.64:40932) 2023-09-13T03:12:51.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.052+0000 7f0c17fff700 1 -- 172.21.15.64:0/3422693681 learned_addr learned my addr 172.21.15.64:0/3422693681 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:51.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.052+0000 7f0c1e59c700 1 --2- 172.21.15.64:0/3422693681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c20100610 0x7f0c20198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:51.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.053+0000 7f0c1ed9d700 1 --2- 172.21.15.64:0/3422693681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c201021a0 0x7f0c2019fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:51.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.053+0000 7f0c17fff700 1 -- 172.21.15.64:0/3422693681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c20100610 msgr2=0x7f0c20198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:51.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.053+0000 7f0c17fff700 1 --2- 172.21.15.64:0/3422693681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c20100610 0x7f0c20198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:51.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.053+0000 7f0c17fff700 1 -- 172.21.15.64:0/3422693681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c201021a0 msgr2=0x7f0c2019fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:51.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.053+0000 7f0c17fff700 1 --2- 172.21.15.64:0/3422693681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c201021a0 0x7f0c2019fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:51.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.053+0000 7f0c17fff700 1 -- 172.21.15.64:0/3422693681 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c18009c40 con 0x7f0c20101800 2023-09-13T03:12:51.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.053+0000 7f0c1e59c700 1 --2- 172.21.15.64:0/3422693681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c20100610 0x7f0c20198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:12:51.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.053+0000 7f0c1ed9d700 1 --2- 172.21.15.64:0/3422693681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c201021a0 0x7f0c2019fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:12:51.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.053+0000 7f0c17fff700 1 --2- 172.21.15.64:0/3422693681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c20101800 0x7f0c20198ab0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0c08009230 tx=0x7f0c0800b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:51.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.053+0000 7f0c177fe700 1 -- 172.21.15.64:0/3422693681 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0801c070 con 0x7f0c20101800 2023-09-13T03:12:51.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.053+0000 7f0c177fe700 1 -- 172.21.15.64:0/3422693681 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c08005040 con 0x7f0c20101800 2023-09-13T03:12:51.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.053+0000 7f0c177fe700 1 -- 172.21.15.64:0/3422693681 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c08003ba0 con 0x7f0c20101800 2023-09-13T03:12:51.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.053+0000 7f0c249ff700 1 -- 172.21.15.64:0/3422693681 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c08009520 con 0x7f0c20101800 2023-09-13T03:12:51.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.053+0000 7f0c249ff700 1 -- 172.21.15.64:0/3422693681 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c201a0880 con 0x7f0c20101800 2023-09-13T03:12:51.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.054+0000 7f0c249ff700 1 -- 172.21.15.64:0/3422693681 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c20192c60 con 0x7f0c20101800 2023-09-13T03:12:51.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.056+0000 7f0c177fe700 1 -- 172.21.15.64:0/3422693681 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f0c08020030 con 0x7f0c20101800 2023-09-13T03:12:51.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.058+0000 7f0c177fe700 1 --2- 172.21.15.64:0/3422693681 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f0c0c0a0710 0x7f0c0c0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:51.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.058+0000 7f0c177fe700 1 -- 172.21.15.64:0/3422693681 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f0c080ccc10 con 0x7f0c20101800 2023-09-13T03:12:51.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.058+0000 7f0c1e59c700 1 --2- 172.21.15.64:0/3422693681 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f0c0c0a0710 0x7f0c0c0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:51.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.059+0000 7f0c1e59c700 1 --2- 172.21.15.64:0/3422693681 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f0c0c0a0710 0x7f0c0c0a2bd0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0c20101660 tx=0x7f0c10009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:51.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.062+0000 7f0c177fe700 1 -- 172.21.15.64:0/3422693681 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0c080967a0 con 0x7f0c20101800 2023-09-13T03:12:51.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.273+0000 7f0c249ff700 1 -- 172.21.15.64:0/3422693681 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0c2010be70 con 0x7f0c20101800 2023-09-13T03:12:51.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.274+0000 7f0c177fe700 1 -- 172.21.15.64:0/3422693681 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22473 (secure 0 0 0) 0x7f0c08003d00 con 0x7f0c20101800 2023-09-13T03:12:51.276 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:12:51.277 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":85,"fsid":"f8dbced4-51e1-11ee-9ab7-7b867c8bd7da","created":"2023-09-13T03:03:24.167202+0000","modified":"2023-09-13T03:11:24.258871+0000","last_up_change":"2023-09-13T03:10:22.503529+0000","last_in_change":"2023-09-13T03:10:05.678720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T03:06:58.987479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-13T03:11:08.589202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-13T03:11:09.924659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3997c114-241b-48c4-8feb-653c99209470","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6803","nonce":3940892135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6805","nonce":3940892135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6809","nonce":3940892135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6807","nonce":3940892135}]},"public_addr":"172.21.15.64:6803/3940892135","cluster_addr":"172.21.15.64:6805/3940892135","heartbeat_back_addr":"172.21.15.64:6809/3940892135","heartbeat_front_addr":"172.21.15.64:6807/3940892135","state":["exists","up"]},{"osd":1,"uuid":"94cfae85-322e-4ed2-a6e6-e40349bb14bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6801","nonce":4188149333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6803","nonce":4188149333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6807","nonce":4188149333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6805","nonce":4188149333}]},"public_addr":"172.21.15.142:6801/4188149333","cluster_addr":"172.21.15.142:6803/4188149333","heartbeat_back_addr":"172.21.15.142:6807/4188149333","heartbeat_front_addr":"172.21.15.142:6805/4188149333","state":["exists","up"]},{"osd":2,"uuid":"b3aae17e-96db-4008-8c27-f733c46b30ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6801","nonce":2406669338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6803","nonce":2406669338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6807","nonce":2406669338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6805","nonce":2406669338}]},"public_addr":"172.21.15.170:6801/2406669338","cluster_addr":"172.21.15.170:6803/2406669338","heartbeat_back_addr":"172.21.15.170:6807/2406669338","heartbeat_front_addr":"172.21.15.170:6805/2406669338","state":["exists","up"]},{"osd":3,"uuid":"7fa8d8cb-2ab7-48a7-8b49-5e2903d6779f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6811","nonce":3558318707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6813","nonce":3558318707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6817","nonce":3558318707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6815","nonce":3558318707}]},"public_addr":"172.21.15.64:6811/3558318707","cluster_addr":"172.21.15.64:6813/3558318707","heartbeat_back_addr":"172.21.15.64:6817/3558318707","heartbeat_front_addr":"172.21.15.64:6815/3558318707","state":["exists","up"]},{"osd":4,"uuid":"8c7a9c9f-f919-48d3-9db2-214a25bb5603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6809","nonce":1811142576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6811","nonce":1811142576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6815","nonce":1811142576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6813","nonce":1811142576}]},"public_addr":"172.21.15.142:6809/1811142576","cluster_addr":"172.21.15.142:6811/1811142576","heartbeat_back_addr":"172.21.15.142:6815/1811142576","heartbeat_front_addr":"172.21.15.142:6813/1811142576","state":["exists","up"]},{"osd":5,"uuid":"0c6101c6-2065-4022-a751-d3f216d7f478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6809","nonce":2903841453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6811","nonce":2903841453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6815","nonce":2903841453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6813","nonce":2903841453}]},"public_addr":"172.21.15.170:6809/2903841453","cluster_addr":"172.21.15.170:6811/2903841453","heartbeat_back_addr":"172.21.15.170:6815/2903841453","heartbeat_front_addr":"172.21.15.170:6813/2903841453","state":["exists","up"]},{"osd":6,"uuid":"5cf3e52e-d155-41b1-9194-db047d26a953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6819","nonce":1436584490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6821","nonce":1436584490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6825","nonce":1436584490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6823","nonce":1436584490}]},"public_addr":"172.21.15.64:6819/1436584490","cluster_addr":"172.21.15.64:6821/1436584490","heartbeat_back_addr":"172.21.15.64:6825/1436584490","heartbeat_front_addr":"172.21.15.64:6823/1436584490","state":["exists","up"]},{"osd":7,"uuid":"e97323e3-badd-41a7-af9e-e78a81a64046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6817","nonce":3726793534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6819","nonce":3726793534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6823","nonce":3726793534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6821","nonce":3726793534}]},"public_addr":"172.21.15.142:6817/3726793534","cluster_addr":"172.21.15.142:6819/3726793534","heartbeat_back_addr":"172.21.15.142:6823/3726793534","heartbeat_front_addr":"172.21.15.142:6821/3726793534","state":["exists","up"]},{"osd":8,"uuid":"83d2835c-ed26-493d-9222-c2894ec0e071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6817","nonce":4272513873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6819","nonce":4272513873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6823","nonce":4272513873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6821","nonce":4272513873}]},"public_addr":"172.21.15.170:6817/4272513873","cluster_addr":"172.21.15.170:6819/4272513873","heartbeat_back_addr":"172.21.15.170:6823/4272513873","heartbeat_front_addr":"172.21.15.170:6821/4272513873","state":["exists","up"]},{"osd":9,"uuid":"cb936bdd-1bec-4236-aa2e-a24adcac11e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6827","nonce":2641175211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6829","nonce":2641175211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6833","nonce":2641175211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6831","nonce":2641175211}]},"public_addr":"172.21.15.64:6827/2641175211","cluster_addr":"172.21.15.64:6829/2641175211","heartbeat_back_addr":"172.21.15.64:6833/2641175211","heartbeat_front_addr":"172.21.15.64:6831/2641175211","state":["exists","up"]},{"osd":10,"uuid":"85512a1c-6171-43d4-9c15-c588a8b71be0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6825","nonce":936499217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6827","nonce":936499217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6831","nonce":936499217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6829","nonce":936499217}]},"public_addr":"172.21.15.142:6825/936499217","cluster_addr":"172.21.15.142:6827/936499217","heartbeat_back_addr":"172.21.15.142:6831/936499217","heartbeat_front_addr":"172.21.15.142:6829/936499217","state":["exists","up"]},{"osd":11,"uuid":"9b8417dc-5d6c-4917-94a9-b032c5bc33d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6825","nonce":3605017060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6827","nonce":3605017060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6831","nonce":3605017060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6829","nonce":3605017060}]},"public_addr":"172.21.15.170:6825/3605017060","cluster_addr":"172.21.15.170:6827/3605017060","heartbeat_back_addr":"172.21.15.170:6831/3605017060","heartbeat_front_addr":"172.21.15.170:6829/3605017060","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:14.229044+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:36.147777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:56.655298+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:07:23.107741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:07:44.663770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:05.577432+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:31.168031+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:52.791055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:09:13.658546+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:09:39.150360+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:10:00.601735+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:10:21.407555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6800/2955227347":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/1373223118":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/271194613":"2023-09-14T03:04:22.355817+0000","172.21.15.64:6801/3998091347":"2023-09-14T03:03:56.442541+0000","172.21.15.64:6800/3998091347":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/920818572":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/735789005":"2023-09-14T03:03:56.442541+0000","172.21.15.64:6801/2955227347":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/1975923178":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/1500151178":"2023-09-14T03:04:22.355817+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T03:12:51.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.277+0000 7f0c249ff700 1 -- 172.21.15.64:0/3422693681 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f0c0c0a0710 msgr2=0x7f0c0c0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:51.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.277+0000 7f0c249ff700 1 --2- 172.21.15.64:0/3422693681 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f0c0c0a0710 0x7f0c0c0a2bd0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0c20101660 tx=0x7f0c10009310 comp rx=0 tx=0).stop 2023-09-13T03:12:51.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.277+0000 7f0c249ff700 1 -- 172.21.15.64:0/3422693681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c20101800 msgr2=0x7f0c20198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:51.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.277+0000 7f0c249ff700 1 --2- 172.21.15.64:0/3422693681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c20101800 0x7f0c20198ab0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0c08009230 tx=0x7f0c0800b560 comp rx=0 tx=0).stop 2023-09-13T03:12:51.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.277+0000 7f0c249ff700 1 -- 172.21.15.64:0/3422693681 shutdown_connections 2023-09-13T03:12:51.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.277+0000 7f0c249ff700 1 --2- 172.21.15.64:0/3422693681 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f0c0c0a0710 0x7f0c0c0a2bd0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:51.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.277+0000 7f0c249ff700 1 --2- 172.21.15.64:0/3422693681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c201021a0 0x7f0c2019fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:51.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.277+0000 7f0c249ff700 1 --2- 172.21.15.64:0/3422693681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c20100610 0x7f0c20198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:51.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.277+0000 7f0c249ff700 1 --2- 172.21.15.64:0/3422693681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c20101800 0x7f0c20198ab0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:51.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.277+0000 7f0c249ff700 1 -- 172.21.15.64:0/3422693681 >> 172.21.15.64:0/3422693681 conn(0x7f0c200fbb40 msgr2=0x7f0c20107160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:51.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.277+0000 7f0c249ff700 1 -- 172.21.15.64:0/3422693681 shutdown_connections 2023-09-13T03:12:51.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.277+0000 7f0c249ff700 1 -- 172.21.15.64:0/3422693681 wait complete. 2023-09-13T03:12:51.298 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-13T03:06:58.987479+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-13T03:11:08.589202+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-13T03:11:09.924659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-13T03:12:51.300 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-13T03:12:51.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.507+0000 7f86e1bff700 1 -- 172.21.15.64:0/1951500809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86dc074640 msgr2=0x7f86dc072a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:51.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.507+0000 7f86e1bff700 1 --2- 172.21.15.64:0/1951500809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86dc074640 0x7f86dc072a90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f86cc009990 tx=0x7f86cc009ca0 comp rx=0 tx=0).stop 2023-09-13T03:12:51.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.507+0000 7f86e1bff700 1 -- 172.21.15.64:0/1951500809 shutdown_connections 2023-09-13T03:12:51.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.507+0000 7f86e1bff700 1 --2- 172.21.15.64:0/1951500809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86dc1084d0 0x7f86dc10a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:51.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.507+0000 7f86e1bff700 1 --2- 172.21.15.64:0/1951500809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86dc074640 0x7f86dc072a90 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:51.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.507+0000 7f86e1bff700 1 --2- 172.21.15.64:0/1951500809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86dc073060 0x7f86dc0734e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:51.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.507+0000 7f86e1bff700 1 -- 172.21.15.64:0/1951500809 >> 172.21.15.64:0/1951500809 conn(0x7f86dc0fbb80 msgr2=0x7f86dc0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:51.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.507+0000 7f86e1bff700 1 -- 172.21.15.64:0/1951500809 shutdown_connections 2023-09-13T03:12:51.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.507+0000 7f86e1bff700 1 -- 172.21.15.64:0/1951500809 wait complete. 2023-09-13T03:12:51.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.508+0000 7f86e1bff700 1 Processor -- start 2023-09-13T03:12:51.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.508+0000 7f86e1bff700 1 -- start start 2023-09-13T03:12:51.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.509+0000 7f86e1bff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86dc073060 0x7f86dc198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:51.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.509+0000 7f86e1bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86dc074640 0x7f86dc198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:51.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.509+0000 7f86e1bff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86dc1084d0 0x7f86dc19fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:51.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.509+0000 7f86e1bff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc1a0090 con 0x7f86dc1084d0 2023-09-13T03:12:51.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.509+0000 7f86e1bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc1a01d0 con 0x7f86dc074640 2023-09-13T03:12:51.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.509+0000 7f86e1bff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc1a0360 con 0x7f86dc073060 2023-09-13T03:12:51.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.509+0000 7f86dbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86dc1084d0 0x7f86dc19fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:51.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.509+0000 7f86dbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86dc1084d0 0x7f86dc19fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40948/0 (socket says 172.21.15.64:40948) 2023-09-13T03:12:51.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.509+0000 7f86dbfff700 1 -- 172.21.15.64:0/2751993926 learned_addr learned my addr 172.21.15.64:0/2751993926 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:51.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.509+0000 7f86daffd700 1 --2- 172.21.15.64:0/2751993926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86dc074640 0x7f86dc198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:51.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.509+0000 7f86db7fe700 1 --2- 172.21.15.64:0/2751993926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86dc073060 0x7f86dc198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:51.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.509+0000 7f86dbfff700 1 -- 172.21.15.64:0/2751993926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86dc073060 msgr2=0x7f86dc198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:51.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.509+0000 7f86dbfff700 1 --2- 172.21.15.64:0/2751993926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86dc073060 0x7f86dc198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:51.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.509+0000 7f86dbfff700 1 -- 172.21.15.64:0/2751993926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86dc074640 msgr2=0x7f86dc198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:51.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.509+0000 7f86dbfff700 1 --2- 172.21.15.64:0/2751993926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86dc074640 0x7f86dc198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:51.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.509+0000 7f86dbfff700 1 -- 172.21.15.64:0/2751993926 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86cc0095f0 con 0x7f86dc1084d0 2023-09-13T03:12:51.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.510+0000 7f86dbfff700 1 --2- 172.21.15.64:0/2751993926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86dc1084d0 0x7f86dc19fac0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f86d400ef30 tx=0x7f86d400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:51.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.510+0000 7f86d8ff9700 1 -- 172.21.15.64:0/2751993926 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d4018070 con 0x7f86dc1084d0 2023-09-13T03:12:51.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.510+0000 7f86e1bff700 1 -- 172.21.15.64:0/2751993926 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86dc1a05e0 con 0x7f86dc1084d0 2023-09-13T03:12:51.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.510+0000 7f86d8ff9700 1 -- 172.21.15.64:0/2751993926 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86d400f040 con 0x7f86dc1084d0 2023-09-13T03:12:51.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.510+0000 7f86d8ff9700 1 -- 172.21.15.64:0/2751993926 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d400cd00 con 0x7f86dc1084d0 2023-09-13T03:12:51.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.510+0000 7f86e1bff700 1 -- 172.21.15.64:0/2751993926 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86dc1a0b40 con 0x7f86dc1084d0 2023-09-13T03:12:51.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.511+0000 7f86e1bff700 1 -- 172.21.15.64:0/2751993926 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86dc106b80 con 0x7f86dc1084d0 2023-09-13T03:12:51.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.513+0000 7f86d8ff9700 1 -- 172.21.15.64:0/2751993926 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f86d4003a00 con 0x7f86dc1084d0 2023-09-13T03:12:51.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.514+0000 7f86d8ff9700 1 --2- 172.21.15.64:0/2751993926 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f86c40a0640 0x7f86c40a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:51.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.515+0000 7f86d8ff9700 1 -- 172.21.15.64:0/2751993926 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f86d40cb6c0 con 0x7f86dc1084d0 2023-09-13T03:12:51.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.515+0000 7f86db7fe700 1 --2- 172.21.15.64:0/2751993926 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f86c40a0640 0x7f86c40a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:51.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.515+0000 7f86db7fe700 1 --2- 172.21.15.64:0/2751993926 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f86c40a0640 0x7f86c40a2b00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f86cc00b2e0 tx=0x7f86cc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:51.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.518+0000 7f86d8ff9700 1 -- 172.21.15.64:0/2751993926 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f86d4095250 con 0x7f86dc1084d0 2023-09-13T03:12:51.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:51 smithi064 ceph-mon[97675]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-13T03:12:51.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:51 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/3422693681' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:12:51.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.730+0000 7f86e1bff700 1 -- 172.21.15.64:0/2751993926 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f86dc02ce50 con 0x7f86dc1084d0 2023-09-13T03:12:51.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.730+0000 7f86d8ff9700 1 -- 172.21.15.64:0/2751993926 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f86d4098870 con 0x7f86dc1084d0 2023-09-13T03:12:51.731 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-09-13T03:12:51.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.733+0000 7f86e1bff700 1 -- 172.21.15.64:0/2751993926 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f86c40a0640 msgr2=0x7f86c40a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:51.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.733+0000 7f86e1bff700 1 --2- 172.21.15.64:0/2751993926 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f86c40a0640 0x7f86c40a2b00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f86cc00b2e0 tx=0x7f86cc005fb0 comp rx=0 tx=0).stop 2023-09-13T03:12:51.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.733+0000 7f86e1bff700 1 -- 172.21.15.64:0/2751993926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86dc1084d0 msgr2=0x7f86dc19fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:51.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.733+0000 7f86e1bff700 1 --2- 172.21.15.64:0/2751993926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86dc1084d0 0x7f86dc19fac0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f86d400ef30 tx=0x7f86d400c3a0 comp rx=0 tx=0).stop 2023-09-13T03:12:51.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.733+0000 7f86e1bff700 1 -- 172.21.15.64:0/2751993926 shutdown_connections 2023-09-13T03:12:51.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.733+0000 7f86e1bff700 1 --2- 172.21.15.64:0/2751993926 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f86c40a0640 0x7f86c40a2b00 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:51.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.733+0000 7f86e1bff700 1 --2- 172.21.15.64:0/2751993926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86dc1084d0 0x7f86dc19fac0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:51.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.733+0000 7f86e1bff700 1 --2- 172.21.15.64:0/2751993926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86dc073060 0x7f86dc198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:51.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.733+0000 7f86e1bff700 1 --2- 172.21.15.64:0/2751993926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86dc074640 0x7f86dc198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:51.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.733+0000 7f86e1bff700 1 -- 172.21.15.64:0/2751993926 >> 172.21.15.64:0/2751993926 conn(0x7f86dc0fbb80 msgr2=0x7f86dc0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:51.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.733+0000 7f86e1bff700 1 -- 172.21.15.64:0/2751993926 shutdown_connections 2023-09-13T03:12:51.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.733+0000 7f86e1bff700 1 -- 172.21.15.64:0/2751993926 wait complete. 2023-09-13T03:12:51.752 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-13T03:12:51.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:51 smithi170 ceph-mon[134328]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-13T03:12:51.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:51 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/3422693681' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:12:51.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.960+0000 7fe5511ff700 1 -- 172.21.15.64:0/2412126519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe54c1005f0 msgr2=0x7fe54c100a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:51.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.960+0000 7fe5511ff700 1 --2- 172.21.15.64:0/2412126519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe54c1005f0 0x7fe54c100a10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe53c0099c0 tx=0x7fe53c009cd0 comp rx=0 tx=0).stop 2023-09-13T03:12:51.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.960+0000 7fe5511ff700 1 -- 172.21.15.64:0/2412126519 shutdown_connections 2023-09-13T03:12:51.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.960+0000 7fe5511ff700 1 --2- 172.21.15.64:0/2412126519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe54c102180 0x7fe54c108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:51.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.960+0000 7fe5511ff700 1 --2- 172.21.15.64:0/2412126519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe54c1005f0 0x7fe54c100a10 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:51.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.960+0000 7fe5511ff700 1 --2- 172.21.15.64:0/2412126519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe54c1017e0 0x7fe54c101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:51.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.960+0000 7fe5511ff700 1 -- 172.21.15.64:0/2412126519 >> 172.21.15.64:0/2412126519 conn(0x7fe54c0fbb40 msgr2=0x7fe54c0fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:51.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.960+0000 7fe5511ff700 1 -- 172.21.15.64:0/2412126519 shutdown_connections 2023-09-13T03:12:51.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.960+0000 7fe5511ff700 1 -- 172.21.15.64:0/2412126519 wait complete. 2023-09-13T03:12:51.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.961+0000 7fe5511ff700 1 Processor -- start 2023-09-13T03:12:51.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.961+0000 7fe5511ff700 1 -- start start 2023-09-13T03:12:51.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.962+0000 7fe5511ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe54c1005f0 0x7fe54c196350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:51.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.962+0000 7fe5511ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe54c1017e0 0x7fe54c196890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:51.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.962+0000 7fe5511ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe54c102180 0x7fe54c19d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:51.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.962+0000 7fe5511ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54c19ded0 con 0x7fe54c102180 2023-09-13T03:12:51.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.962+0000 7fe5511ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54c19e010 con 0x7fe54c1017e0 2023-09-13T03:12:51.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.962+0000 7fe5511ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54c19e1a0 con 0x7fe54c1005f0 2023-09-13T03:12:51.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.962+0000 7fe54ad9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe54c1005f0 0x7fe54c196350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:51.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.962+0000 7fe54b59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe54c102180 0x7fe54c19d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:51.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.962+0000 7fe54ad9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe54c1005f0 0x7fe54c196350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:35820/0 (socket says 172.21.15.64:35820) 2023-09-13T03:12:51.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.962+0000 7fe54ad9d700 1 -- 172.21.15.64:0/2737916428 learned_addr learned my addr 172.21.15.64:0/2737916428 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:51.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.962+0000 7fe54a59c700 1 --2- 172.21.15.64:0/2737916428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe54c1017e0 0x7fe54c196890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:51.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.962+0000 7fe54b59e700 1 -- 172.21.15.64:0/2737916428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe54c1005f0 msgr2=0x7fe54c196350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:51.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.962+0000 7fe54b59e700 1 --2- 172.21.15.64:0/2737916428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe54c1005f0 0x7fe54c196350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:51.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.962+0000 7fe54b59e700 1 -- 172.21.15.64:0/2737916428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe54c1017e0 msgr2=0x7fe54c196890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:51.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.962+0000 7fe54b59e700 1 --2- 172.21.15.64:0/2737916428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe54c1017e0 0x7fe54c196890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:51.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.962+0000 7fe54b59e700 1 -- 172.21.15.64:0/2737916428 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe540009520 con 0x7fe54c102180 2023-09-13T03:12:51.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.963+0000 7fe54b59e700 1 --2- 172.21.15.64:0/2737916428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe54c102180 0x7fe54c19d900 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe54400efc0 tx=0x7fe54400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:51.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.963+0000 7fe533fff700 1 -- 172.21.15.64:0/2737916428 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe544018070 con 0x7fe54c102180 2023-09-13T03:12:51.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.963+0000 7fe5511ff700 1 -- 172.21.15.64:0/2737916428 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe53c0095f0 con 0x7fe54c102180 2023-09-13T03:12:51.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.963+0000 7fe533fff700 1 -- 172.21.15.64:0/2737916428 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe54400f040 con 0x7fe54c102180 2023-09-13T03:12:51.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.963+0000 7fe533fff700 1 -- 172.21.15.64:0/2737916428 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54400cd40 con 0x7fe54c102180 2023-09-13T03:12:51.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.963+0000 7fe5511ff700 1 -- 172.21.15.64:0/2737916428 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe54c19e7a0 con 0x7fe54c102180 2023-09-13T03:12:51.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.964+0000 7fe5511ff700 1 -- 172.21.15.64:0/2737916428 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe54c10be60 con 0x7fe54c102180 2023-09-13T03:12:51.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.966+0000 7fe533fff700 1 -- 172.21.15.64:0/2737916428 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fe544003a00 con 0x7fe54c102180 2023-09-13T03:12:51.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.967+0000 7fe533fff700 1 --2- 172.21.15.64:0/2737916428 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe5340a0640 0x7fe5340a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:51.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.967+0000 7fe533fff700 1 -- 172.21.15.64:0/2737916428 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fe5440cb730 con 0x7fe54c102180 2023-09-13T03:12:51.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.968+0000 7fe54ad9d700 1 --2- 172.21.15.64:0/2737916428 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe5340a0640 0x7fe5340a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:51.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.968+0000 7fe54ad9d700 1 --2- 172.21.15.64:0/2737916428 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe5340a0640 0x7fe5340a2b00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe53c000c00 tx=0x7fe53c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:51.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:51.972+0000 7fe533fff700 1 -- 172.21.15.64:0/2737916428 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe5440952c0 con 0x7fe54c102180 2023-09-13T03:12:51.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:51 smithi142 ceph-mon[134082]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-13T03:12:51.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:51 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/3422693681' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:12:52.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.183+0000 7fe5511ff700 1 -- 172.21.15.64:0/2737916428 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe54c107570 con 0x7fe54c102180 2023-09-13T03:12:52.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.183+0000 7fe533fff700 1 -- 172.21.15.64:0/2737916428 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe5440988e0 con 0x7fe54c102180 2023-09-13T03:12:52.185 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-09-13T03:12:52.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.186+0000 7fe5511ff700 1 -- 172.21.15.64:0/2737916428 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe5340a0640 msgr2=0x7fe5340a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:52.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.186+0000 7fe5511ff700 1 --2- 172.21.15.64:0/2737916428 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe5340a0640 0x7fe5340a2b00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe53c000c00 tx=0x7fe53c005fb0 comp rx=0 tx=0).stop 2023-09-13T03:12:52.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.186+0000 7fe5511ff700 1 -- 172.21.15.64:0/2737916428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe54c102180 msgr2=0x7fe54c19d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:52.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.186+0000 7fe5511ff700 1 --2- 172.21.15.64:0/2737916428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe54c102180 0x7fe54c19d900 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe54400efc0 tx=0x7fe54400c3a0 comp rx=0 tx=0).stop 2023-09-13T03:12:52.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.186+0000 7fe5511ff700 1 -- 172.21.15.64:0/2737916428 shutdown_connections 2023-09-13T03:12:52.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.186+0000 7fe5511ff700 1 --2- 172.21.15.64:0/2737916428 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe5340a0640 0x7fe5340a2b00 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:52.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.186+0000 7fe5511ff700 1 --2- 172.21.15.64:0/2737916428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe54c102180 0x7fe54c19d900 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:52.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.186+0000 7fe5511ff700 1 --2- 172.21.15.64:0/2737916428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe54c1005f0 0x7fe54c196350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:52.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.186+0000 7fe5511ff700 1 --2- 172.21.15.64:0/2737916428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe54c1017e0 0x7fe54c196890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:52.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.186+0000 7fe5511ff700 1 -- 172.21.15.64:0/2737916428 >> 172.21.15.64:0/2737916428 conn(0x7fe54c0fbb40 msgr2=0x7fe54c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:52.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.186+0000 7fe5511ff700 1 -- 172.21.15.64:0/2737916428 shutdown_connections 2023-09-13T03:12:52.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.186+0000 7fe5511ff700 1 -- 172.21.15.64:0/2737916428 wait complete. 2023-09-13T03:12:52.205 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-13T03:12:52.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.413+0000 7fde117ff700 1 -- 172.21.15.64:0/3648482220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde0c101110 msgr2=0x7fde0c103500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:52.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.413+0000 7fde117ff700 1 --2- 172.21.15.64:0/3648482220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde0c101110 0x7fde0c103500 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fddfc009990 tx=0x7fddfc009ca0 comp rx=0 tx=0).stop 2023-09-13T03:12:52.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.413+0000 7fde117ff700 1 -- 172.21.15.64:0/3648482220 shutdown_connections 2023-09-13T03:12:52.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.413+0000 7fde117ff700 1 --2- 172.21.15.64:0/3648482220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde0c106370 0x7fde0c108760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:52.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.413+0000 7fde117ff700 1 --2- 172.21.15.64:0/3648482220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde0c101110 0x7fde0c103500 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:52.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.413+0000 7fde117ff700 1 --2- 172.21.15.64:0/3648482220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde0c103a40 0x7fde0c105e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:52.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.413+0000 7fde117ff700 1 -- 172.21.15.64:0/3648482220 >> 172.21.15.64:0/3648482220 conn(0x7fde0c0faa60 msgr2=0x7fde0c0fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:52.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.413+0000 7fde117ff700 1 -- 172.21.15.64:0/3648482220 shutdown_connections 2023-09-13T03:12:52.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.413+0000 7fde117ff700 1 -- 172.21.15.64:0/3648482220 wait complete. 2023-09-13T03:12:52.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.414+0000 7fde117ff700 1 Processor -- start 2023-09-13T03:12:52.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.414+0000 7fde117ff700 1 -- start start 2023-09-13T03:12:52.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.414+0000 7fde117ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde0c101110 0x7fde0c198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:52.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.414+0000 7fde117ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde0c103a40 0x7fde0c198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:52.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.414+0000 7fde117ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde0c106370 0x7fde0c19fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:52.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.414+0000 7fde117ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c1a00a0 con 0x7fde0c101110 2023-09-13T03:12:52.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.414+0000 7fde117ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c1a01e0 con 0x7fde0c103a40 2023-09-13T03:12:52.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.414+0000 7fde117ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c1a0370 con 0x7fde0c106370 2023-09-13T03:12:52.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.414+0000 7fde0affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde0c101110 0x7fde0c198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:52.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.415+0000 7fde0affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde0c101110 0x7fde0c198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40982/0 (socket says 172.21.15.64:40982) 2023-09-13T03:12:52.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.415+0000 7fde0affd700 1 -- 172.21.15.64:0/593274469 learned_addr learned my addr 172.21.15.64:0/593274469 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:52.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.415+0000 7fde03fff700 1 --2- 172.21.15.64:0/593274469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde0c103a40 0x7fde0c198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:52.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.415+0000 7fde0b7fe700 1 --2- 172.21.15.64:0/593274469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde0c106370 0x7fde0c19fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:52.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.415+0000 7fde0affd700 1 -- 172.21.15.64:0/593274469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde0c106370 msgr2=0x7fde0c19fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:52.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.415+0000 7fde0affd700 1 --2- 172.21.15.64:0/593274469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde0c106370 0x7fde0c19fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:52.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.415+0000 7fde0affd700 1 -- 172.21.15.64:0/593274469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde0c103a40 msgr2=0x7fde0c198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:52.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.415+0000 7fde0affd700 1 --2- 172.21.15.64:0/593274469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde0c103a40 0x7fde0c198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:52.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.415+0000 7fde0affd700 1 -- 172.21.15.64:0/593274469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddfc0095f0 con 0x7fde0c101110 2023-09-13T03:12:52.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.415+0000 7fde0affd700 1 --2- 172.21.15.64:0/593274469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde0c101110 0x7fde0c198520 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fddfc004740 tx=0x7fddfc009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:52.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.416+0000 7fde08ff9700 1 -- 172.21.15.64:0/593274469 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc01d070 con 0x7fde0c101110 2023-09-13T03:12:52.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.416+0000 7fde08ff9700 1 -- 172.21.15.64:0/593274469 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fddfc004830 con 0x7fde0c101110 2023-09-13T03:12:52.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.416+0000 7fde117ff700 1 -- 172.21.15.64:0/593274469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde0c1a05f0 con 0x7fde0c101110 2023-09-13T03:12:52.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.416+0000 7fde08ff9700 1 -- 172.21.15.64:0/593274469 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc0053c0 con 0x7fde0c101110 2023-09-13T03:12:52.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.416+0000 7fde117ff700 1 -- 172.21.15.64:0/593274469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde0c1a0a10 con 0x7fde0c101110 2023-09-13T03:12:52.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.417+0000 7fde117ff700 1 -- 172.21.15.64:0/593274469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde0c107ef0 con 0x7fde0c101110 2023-09-13T03:12:52.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.419+0000 7fde08ff9700 1 -- 172.21.15.64:0/593274469 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fddfc005560 con 0x7fde0c101110 2023-09-13T03:12:52.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.420+0000 7fde08ff9700 1 --2- 172.21.15.64:0/593274469 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fddf80a06c0 0x7fddf80a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:52.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.420+0000 7fde08ff9700 1 -- 172.21.15.64:0/593274469 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fddfc0cd170 con 0x7fde0c101110 2023-09-13T03:12:52.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.420+0000 7fde03fff700 1 --2- 172.21.15.64:0/593274469 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fddf80a06c0 0x7fddf80a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:52.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.421+0000 7fde03fff700 1 --2- 172.21.15.64:0/593274469 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fddf80a06c0 0x7fddf80a2b80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fddf4009c60 tx=0x7fddf4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:52.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.422+0000 7fde08ff9700 1 -- 172.21.15.64:0/593274469 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fddfc096d00 con 0x7fde0c101110 2023-09-13T03:12:52.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.635+0000 7fde117ff700 1 -- 172.21.15.64:0/593274469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fde0c10bfd0 con 0x7fde0c101110 2023-09-13T03:12:52.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.636+0000 7fde08ff9700 1 -- 172.21.15.64:0/593274469 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7fddfc09a320 con 0x7fde0c101110 2023-09-13T03:12:52.637 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-09-13T03:12:52.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.638+0000 7fde117ff700 1 -- 172.21.15.64:0/593274469 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fddf80a06c0 msgr2=0x7fddf80a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:52.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.638+0000 7fde117ff700 1 --2- 172.21.15.64:0/593274469 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fddf80a06c0 0x7fddf80a2b80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fddf4009c60 tx=0x7fddf4009260 comp rx=0 tx=0).stop 2023-09-13T03:12:52.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.638+0000 7fde117ff700 1 -- 172.21.15.64:0/593274469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde0c101110 msgr2=0x7fde0c198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:52.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.638+0000 7fde117ff700 1 --2- 172.21.15.64:0/593274469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde0c101110 0x7fde0c198520 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fddfc004740 tx=0x7fddfc009580 comp rx=0 tx=0).stop 2023-09-13T03:12:52.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.638+0000 7fde117ff700 1 -- 172.21.15.64:0/593274469 shutdown_connections 2023-09-13T03:12:52.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.638+0000 7fde117ff700 1 --2- 172.21.15.64:0/593274469 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fddf80a06c0 0x7fddf80a2b80 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:52.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.638+0000 7fde117ff700 1 --2- 172.21.15.64:0/593274469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde0c106370 0x7fde0c19fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:52.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.638+0000 7fde117ff700 1 --2- 172.21.15.64:0/593274469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde0c101110 0x7fde0c198520 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:52.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.638+0000 7fde117ff700 1 --2- 172.21.15.64:0/593274469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde0c103a40 0x7fde0c198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:52.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.638+0000 7fde117ff700 1 -- 172.21.15.64:0/593274469 >> 172.21.15.64:0/593274469 conn(0x7fde0c0faa60 msgr2=0x7fde0c0ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:52.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.638+0000 7fde117ff700 1 -- 172.21.15.64:0/593274469 shutdown_connections 2023-09-13T03:12:52.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.638+0000 7fde117ff700 1 -- 172.21.15.64:0/593274469 wait complete. 2023-09-13T03:12:52.658 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-13T03:12:52.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.860+0000 7f13f71ff700 1 -- 172.21.15.64:0/2804204386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13f0100600 msgr2=0x7f13f0100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:52.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.860+0000 7f13f71ff700 1 --2- 172.21.15.64:0/2804204386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13f0100600 0x7f13f0100a20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f13e40099c0 tx=0x7f13e4009cd0 comp rx=0 tx=0).stop 2023-09-13T03:12:52.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.860+0000 7f13f71ff700 1 -- 172.21.15.64:0/2804204386 shutdown_connections 2023-09-13T03:12:52.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.860+0000 7f13f71ff700 1 --2- 172.21.15.64:0/2804204386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13f0102190 0x7f13f0108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:52.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.860+0000 7f13f71ff700 1 --2- 172.21.15.64:0/2804204386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13f0100600 0x7f13f0100a20 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:52.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.860+0000 7f13f71ff700 1 --2- 172.21.15.64:0/2804204386 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f13f01017f0 0x7f13f0101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:52.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.860+0000 7f13f71ff700 1 -- 172.21.15.64:0/2804204386 >> 172.21.15.64:0/2804204386 conn(0x7f13f00fbb30 msgr2=0x7f13f00fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:52.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.860+0000 7f13f71ff700 1 -- 172.21.15.64:0/2804204386 shutdown_connections 2023-09-13T03:12:52.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.860+0000 7f13f71ff700 1 -- 172.21.15.64:0/2804204386 wait complete. 2023-09-13T03:12:52.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.861+0000 7f13f71ff700 1 Processor -- start 2023-09-13T03:12:52.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.861+0000 7f13f71ff700 1 -- start start 2023-09-13T03:12:52.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.862+0000 7f13f71ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13f0100600 0x7f13f01984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:52.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.862+0000 7f13f71ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f13f01017f0 0x7f13f0198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:52.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.862+0000 7f13f71ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13f0102190 0x7f13f019faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:52.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.862+0000 7f13f71ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f01a0070 con 0x7f13f0100600 2023-09-13T03:12:52.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.862+0000 7f13f71ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f01a01b0 con 0x7f13f0102190 2023-09-13T03:12:52.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.862+0000 7f13f71ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f01a0340 con 0x7f13f01017f0 2023-09-13T03:12:52.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.862+0000 7f13f4f9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13f0100600 0x7f13f01984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:52.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.862+0000 7f13effff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f13f01017f0 0x7f13f0198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:52.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.862+0000 7f13f4f9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13f0100600 0x7f13f01984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40998/0 (socket says 172.21.15.64:40998) 2023-09-13T03:12:52.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.862+0000 7f13f4f9b700 1 -- 172.21.15.64:0/2208047205 learned_addr learned my addr 172.21.15.64:0/2208047205 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:52.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.862+0000 7f13f579c700 1 --2- 172.21.15.64:0/2208047205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13f0102190 0x7f13f019faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:52.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.862+0000 7f13f4f9b700 1 -- 172.21.15.64:0/2208047205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f13f01017f0 msgr2=0x7f13f0198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:52.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.862+0000 7f13f4f9b700 1 --2- 172.21.15.64:0/2208047205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f13f01017f0 0x7f13f0198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:52.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.862+0000 7f13f4f9b700 1 -- 172.21.15.64:0/2208047205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13f0102190 msgr2=0x7f13f019faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:52.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.862+0000 7f13f4f9b700 1 --2- 172.21.15.64:0/2208047205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13f0102190 0x7f13f019faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:52.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.862+0000 7f13f4f9b700 1 -- 172.21.15.64:0/2208047205 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e40095f0 con 0x7f13f0100600 2023-09-13T03:12:52.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.863+0000 7f13f4f9b700 1 --2- 172.21.15.64:0/2208047205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13f0100600 0x7f13f01984f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f13e40058e0 tx=0x7f13e400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:52.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.863+0000 7f13edffb700 1 -- 172.21.15.64:0/2208047205 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e401d070 con 0x7f13f0100600 2023-09-13T03:12:52.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.863+0000 7f13edffb700 1 -- 172.21.15.64:0/2208047205 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13e4005050 con 0x7f13f0100600 2023-09-13T03:12:52.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.863+0000 7f13edffb700 1 -- 172.21.15.64:0/2208047205 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e4004b30 con 0x7f13f0100600 2023-09-13T03:12:52.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.863+0000 7f13f71ff700 1 -- 172.21.15.64:0/2208047205 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13f01a05c0 con 0x7f13f0100600 2023-09-13T03:12:52.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.863+0000 7f13f71ff700 1 -- 172.21.15.64:0/2208047205 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13f01a0990 con 0x7f13f0100600 2023-09-13T03:12:52.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.864+0000 7f13f71ff700 1 -- 172.21.15.64:0/2208047205 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13f010be30 con 0x7f13f0100600 2023-09-13T03:12:52.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.866+0000 7f13edffb700 1 -- 172.21.15.64:0/2208047205 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f13e40051c0 con 0x7f13f0100600 2023-09-13T03:12:52.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.867+0000 7f13edffb700 1 --2- 172.21.15.64:0/2208047205 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f13e00a0710 0x7f13e00a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:52.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.868+0000 7f13edffb700 1 -- 172.21.15.64:0/2208047205 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f13e40ccd40 con 0x7f13f0100600 2023-09-13T03:12:52.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.868+0000 7f13effff700 1 --2- 172.21.15.64:0/2208047205 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f13e00a0710 0x7f13e00a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:52.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.868+0000 7f13effff700 1 --2- 172.21.15.64:0/2208047205 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f13e00a0710 0x7f13e00a2bd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f13dc009c60 tx=0x7f13dc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:52.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:52.869+0000 7f13edffb700 1 -- 172.21.15.64:0/2208047205 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f13e40968d0 con 0x7f13f0100600 2023-09-13T03:12:52.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:52 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/2751993926' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T03:12:52.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:52 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/2737916428' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T03:12:52.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:52 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2751993926' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T03:12:52.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:52 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2737916428' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T03:12:52.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:52 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/2751993926' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T03:12:52.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:52 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/2737916428' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T03:12:53.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.081+0000 7f13f71ff700 1 -- 172.21.15.64:0/2208047205 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f13f010c040 con 0x7f13f0100600 2023-09-13T03:12:53.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.083+0000 7f13edffb700 1 -- 172.21.15.64:0/2208047205 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22473 (secure 0 0 0) 0x7f13e4099ef0 con 0x7f13f0100600 2023-09-13T03:12:53.084 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:12:53.084 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":85,"fsid":"f8dbced4-51e1-11ee-9ab7-7b867c8bd7da","created":"2023-09-13T03:03:24.167202+0000","modified":"2023-09-13T03:11:24.258871+0000","last_up_change":"2023-09-13T03:10:22.503529+0000","last_in_change":"2023-09-13T03:10:05.678720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T03:06:58.987479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-13T03:11:08.589202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-13T03:11:09.924659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3997c114-241b-48c4-8feb-653c99209470","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6803","nonce":3940892135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6805","nonce":3940892135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6809","nonce":3940892135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6807","nonce":3940892135}]},"public_addr":"172.21.15.64:6803/3940892135","cluster_addr":"172.21.15.64:6805/3940892135","heartbeat_back_addr":"172.21.15.64:6809/3940892135","heartbeat_front_addr":"172.21.15.64:6807/3940892135","state":["exists","up"]},{"osd":1,"uuid":"94cfae85-322e-4ed2-a6e6-e40349bb14bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6801","nonce":4188149333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6803","nonce":4188149333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6807","nonce":4188149333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6805","nonce":4188149333}]},"public_addr":"172.21.15.142:6801/4188149333","cluster_addr":"172.21.15.142:6803/4188149333","heartbeat_back_addr":"172.21.15.142:6807/4188149333","heartbeat_front_addr":"172.21.15.142:6805/4188149333","state":["exists","up"]},{"osd":2,"uuid":"b3aae17e-96db-4008-8c27-f733c46b30ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6801","nonce":2406669338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6803","nonce":2406669338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6807","nonce":2406669338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6805","nonce":2406669338}]},"public_addr":"172.21.15.170:6801/2406669338","cluster_addr":"172.21.15.170:6803/2406669338","heartbeat_back_addr":"172.21.15.170:6807/2406669338","heartbeat_front_addr":"172.21.15.170:6805/2406669338","state":["exists","up"]},{"osd":3,"uuid":"7fa8d8cb-2ab7-48a7-8b49-5e2903d6779f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6811","nonce":3558318707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6813","nonce":3558318707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6817","nonce":3558318707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6815","nonce":3558318707}]},"public_addr":"172.21.15.64:6811/3558318707","cluster_addr":"172.21.15.64:6813/3558318707","heartbeat_back_addr":"172.21.15.64:6817/3558318707","heartbeat_front_addr":"172.21.15.64:6815/3558318707","state":["exists","up"]},{"osd":4,"uuid":"8c7a9c9f-f919-48d3-9db2-214a25bb5603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6809","nonce":1811142576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6811","nonce":1811142576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6815","nonce":1811142576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6813","nonce":1811142576}]},"public_addr":"172.21.15.142:6809/1811142576","cluster_addr":"172.21.15.142:6811/1811142576","heartbeat_back_addr":"172.21.15.142:6815/1811142576","heartbeat_front_addr":"172.21.15.142:6813/1811142576","state":["exists","up"]},{"osd":5,"uuid":"0c6101c6-2065-4022-a751-d3f216d7f478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6809","nonce":2903841453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6811","nonce":2903841453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6815","nonce":2903841453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6813","nonce":2903841453}]},"public_addr":"172.21.15.170:6809/2903841453","cluster_addr":"172.21.15.170:6811/2903841453","heartbeat_back_addr":"172.21.15.170:6815/2903841453","heartbeat_front_addr":"172.21.15.170:6813/2903841453","state":["exists","up"]},{"osd":6,"uuid":"5cf3e52e-d155-41b1-9194-db047d26a953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6819","nonce":1436584490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6821","nonce":1436584490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6825","nonce":1436584490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6823","nonce":1436584490}]},"public_addr":"172.21.15.64:6819/1436584490","cluster_addr":"172.21.15.64:6821/1436584490","heartbeat_back_addr":"172.21.15.64:6825/1436584490","heartbeat_front_addr":"172.21.15.64:6823/1436584490","state":["exists","up"]},{"osd":7,"uuid":"e97323e3-badd-41a7-af9e-e78a81a64046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6817","nonce":3726793534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6819","nonce":3726793534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6823","nonce":3726793534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6821","nonce":3726793534}]},"public_addr":"172.21.15.142:6817/3726793534","cluster_addr":"172.21.15.142:6819/3726793534","heartbeat_back_addr":"172.21.15.142:6823/3726793534","heartbeat_front_addr":"172.21.15.142:6821/3726793534","state":["exists","up"]},{"osd":8,"uuid":"83d2835c-ed26-493d-9222-c2894ec0e071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6817","nonce":4272513873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6819","nonce":4272513873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6823","nonce":4272513873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6821","nonce":4272513873}]},"public_addr":"172.21.15.170:6817/4272513873","cluster_addr":"172.21.15.170:6819/4272513873","heartbeat_back_addr":"172.21.15.170:6823/4272513873","heartbeat_front_addr":"172.21.15.170:6821/4272513873","state":["exists","up"]},{"osd":9,"uuid":"cb936bdd-1bec-4236-aa2e-a24adcac11e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6827","nonce":2641175211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6829","nonce":2641175211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6833","nonce":2641175211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6831","nonce":2641175211}]},"public_addr":"172.21.15.64:6827/2641175211","cluster_addr":"172.21.15.64:6829/2641175211","heartbeat_back_addr":"172.21.15.64:6833/2641175211","heartbeat_front_addr":"172.21.15.64:6831/2641175211","state":["exists","up"]},{"osd":10,"uuid":"85512a1c-6171-43d4-9c15-c588a8b71be0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6825","nonce":936499217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6827","nonce":936499217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6831","nonce":936499217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6829","nonce":936499217}]},"public_addr":"172.21.15.142:6825/936499217","cluster_addr":"172.21.15.142:6827/936499217","heartbeat_back_addr":"172.21.15.142:6831/936499217","heartbeat_front_addr":"172.21.15.142:6829/936499217","state":["exists","up"]},{"osd":11,"uuid":"9b8417dc-5d6c-4917-94a9-b032c5bc33d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6825","nonce":3605017060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6827","nonce":3605017060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6831","nonce":3605017060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6829","nonce":3605017060}]},"public_addr":"172.21.15.170:6825/3605017060","cluster_addr":"172.21.15.170:6827/3605017060","heartbeat_back_addr":"172.21.15.170:6831/3605017060","heartbeat_front_addr":"172.21.15.170:6829/3605017060","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:14.229044+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:36.147777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:56.655298+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:07:23.107741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:07:44.663770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:05.577432+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:31.168031+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:52.791055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:09:13.658546+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:09:39.150360+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:10:00.601735+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:10:21.407555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6800/2955227347":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/1373223118":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/271194613":"2023-09-14T03:04:22.355817+0000","172.21.15.64:6801/3998091347":"2023-09-14T03:03:56.442541+0000","172.21.15.64:6800/3998091347":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/920818572":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/735789005":"2023-09-14T03:03:56.442541+0000","172.21.15.64:6801/2955227347":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/1975923178":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/1500151178":"2023-09-14T03:04:22.355817+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T03:12:53.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.085+0000 7f13f71ff700 1 -- 172.21.15.64:0/2208047205 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f13e00a0710 msgr2=0x7f13e00a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:53.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.085+0000 7f13f71ff700 1 --2- 172.21.15.64:0/2208047205 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f13e00a0710 0x7f13e00a2bd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f13dc009c60 tx=0x7f13dc009260 comp rx=0 tx=0).stop 2023-09-13T03:12:53.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.085+0000 7f13f71ff700 1 -- 172.21.15.64:0/2208047205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13f0100600 msgr2=0x7f13f01984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:53.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.085+0000 7f13f71ff700 1 --2- 172.21.15.64:0/2208047205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13f0100600 0x7f13f01984f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f13e40058e0 tx=0x7f13e400f690 comp rx=0 tx=0).stop 2023-09-13T03:12:53.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.085+0000 7f13f71ff700 1 -- 172.21.15.64:0/2208047205 shutdown_connections 2023-09-13T03:12:53.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.085+0000 7f13f71ff700 1 --2- 172.21.15.64:0/2208047205 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f13e00a0710 0x7f13e00a2bd0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:53.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.085+0000 7f13f71ff700 1 --2- 172.21.15.64:0/2208047205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13f0102190 0x7f13f019faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:53.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.085+0000 7f13f71ff700 1 --2- 172.21.15.64:0/2208047205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13f0100600 0x7f13f01984f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:53.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.085+0000 7f13f71ff700 1 --2- 172.21.15.64:0/2208047205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f13f01017f0 0x7f13f0198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:53.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.085+0000 7f13f71ff700 1 -- 172.21.15.64:0/2208047205 >> 172.21.15.64:0/2208047205 conn(0x7f13f00fbb30 msgr2=0x7f13f00fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:53.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.085+0000 7f13f71ff700 1 -- 172.21.15.64:0/2208047205 shutdown_connections 2023-09-13T03:12:53.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.085+0000 7f13f71ff700 1 -- 172.21.15.64:0/2208047205 wait complete. 2023-09-13T03:12:53.105 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-13T03:06:58.987479+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-13T03:11:08.589202+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-13T03:11:09.924659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-13T03:12:53.106 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-13T03:12:53.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.312+0000 7fe158bff700 1 -- 172.21.15.64:0/3754604730 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1541005f0 msgr2=0x7fe154100a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:53.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.312+0000 7fe158bff700 1 --2- 172.21.15.64:0/3754604730 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1541005f0 0x7fe154100a10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe1440098f0 tx=0x7fe144009c00 comp rx=0 tx=0).stop 2023-09-13T03:12:53.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.312+0000 7fe158bff700 1 -- 172.21.15.64:0/3754604730 shutdown_connections 2023-09-13T03:12:53.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.312+0000 7fe158bff700 1 --2- 172.21.15.64:0/3754604730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe154102180 0x7fe154108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:53.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.312+0000 7fe158bff700 1 --2- 172.21.15.64:0/3754604730 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1541005f0 0x7fe154100a10 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:53.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.312+0000 7fe158bff700 1 --2- 172.21.15.64:0/3754604730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1541017e0 0x7fe154101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:53.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.312+0000 7fe158bff700 1 -- 172.21.15.64:0/3754604730 >> 172.21.15.64:0/3754604730 conn(0x7fe1540fbb40 msgr2=0x7fe1540fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:53.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.312+0000 7fe158bff700 1 -- 172.21.15.64:0/3754604730 shutdown_connections 2023-09-13T03:12:53.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.312+0000 7fe158bff700 1 -- 172.21.15.64:0/3754604730 wait complete. 2023-09-13T03:12:53.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.313+0000 7fe158bff700 1 Processor -- start 2023-09-13T03:12:53.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.313+0000 7fe158bff700 1 -- start start 2023-09-13T03:12:53.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.313+0000 7fe158bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1541005f0 0x7fe1541963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:53.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.313+0000 7fe158bff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1541017e0 0x7fe154196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:53.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.314+0000 7fe158bff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe154102180 0x7fe15419d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:53.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.314+0000 7fe158bff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15419df40 con 0x7fe1541017e0 2023-09-13T03:12:53.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.314+0000 7fe158bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15419e080 con 0x7fe1541005f0 2023-09-13T03:12:53.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.314+0000 7fe158bff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15419e210 con 0x7fe154102180 2023-09-13T03:12:53.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.314+0000 7fe151d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1541017e0 0x7fe154196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:53.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.314+0000 7fe151d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1541017e0 0x7fe154196900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41028/0 (socket says 172.21.15.64:41028) 2023-09-13T03:12:53.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.314+0000 7fe151d9b700 1 -- 172.21.15.64:0/2101230724 learned_addr learned my addr 172.21.15.64:0/2101230724 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:53.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.314+0000 7fe15259c700 1 --2- 172.21.15.64:0/2101230724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1541005f0 0x7fe1541963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:53.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.314+0000 7fe152d9d700 1 --2- 172.21.15.64:0/2101230724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe154102180 0x7fe15419d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:53.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.314+0000 7fe151d9b700 1 -- 172.21.15.64:0/2101230724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe154102180 msgr2=0x7fe15419d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:53.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.314+0000 7fe151d9b700 1 --2- 172.21.15.64:0/2101230724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe154102180 0x7fe15419d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:53.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.314+0000 7fe151d9b700 1 -- 172.21.15.64:0/2101230724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1541005f0 msgr2=0x7fe1541963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:53.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.314+0000 7fe151d9b700 1 --2- 172.21.15.64:0/2101230724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1541005f0 0x7fe1541963c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:53.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.314+0000 7fe151d9b700 1 -- 172.21.15.64:0/2101230724 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1480095f0 con 0x7fe1541017e0 2023-09-13T03:12:53.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.315+0000 7fe151d9b700 1 --2- 172.21.15.64:0/2101230724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1541017e0 0x7fe154196900 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe14800ec10 tx=0x7fe14800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:53.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.315+0000 7fe1437fe700 1 -- 172.21.15.64:0/2101230724 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14800cc90 con 0x7fe1541017e0 2023-09-13T03:12:53.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.315+0000 7fe1437fe700 1 -- 172.21.15.64:0/2101230724 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1480044e0 con 0x7fe1541017e0 2023-09-13T03:12:53.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.315+0000 7fe158bff700 1 -- 172.21.15.64:0/2101230724 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe144009520 con 0x7fe1541017e0 2023-09-13T03:12:53.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.315+0000 7fe1437fe700 1 -- 172.21.15.64:0/2101230724 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe148010430 con 0x7fe1541017e0 2023-09-13T03:12:53.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.315+0000 7fe158bff700 1 -- 172.21.15.64:0/2101230724 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe15419e870 con 0x7fe1541017e0 2023-09-13T03:12:53.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.316+0000 7fe158bff700 1 -- 172.21.15.64:0/2101230724 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe154107cb0 con 0x7fe1541017e0 2023-09-13T03:12:53.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.318+0000 7fe1437fe700 1 -- 172.21.15.64:0/2101230724 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fe148003fb0 con 0x7fe1541017e0 2023-09-13T03:12:53.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.320+0000 7fe1437fe700 1 --2- 172.21.15.64:0/2101230724 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe13c0a0640 0x7fe13c0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:53.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.320+0000 7fe1437fe700 1 -- 172.21.15.64:0/2101230724 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fe148014070 con 0x7fe1541017e0 2023-09-13T03:12:53.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.320+0000 7fe15259c700 1 --2- 172.21.15.64:0/2101230724 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe13c0a0640 0x7fe13c0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:53.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.321+0000 7fe15259c700 1 --2- 172.21.15.64:0/2101230724 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe13c0a0640 0x7fe13c0a2b00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe144009fd0 tx=0x7fe144019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:53.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.324+0000 7fe1437fe700 1 -- 172.21.15.64:0/2101230724 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe148095c40 con 0x7fe1541017e0 2023-09-13T03:12:53.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.537+0000 7fe158bff700 1 -- 172.21.15.64:0/2101230724 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe15402cf10 con 0x7fe1541017e0 2023-09-13T03:12:53.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.538+0000 7fe1437fe700 1 -- 172.21.15.64:0/2101230724 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe148099260 con 0x7fe1541017e0 2023-09-13T03:12:53.539 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-09-13T03:12:53.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.540+0000 7fe158bff700 1 -- 172.21.15.64:0/2101230724 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe13c0a0640 msgr2=0x7fe13c0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:53.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.540+0000 7fe158bff700 1 --2- 172.21.15.64:0/2101230724 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe13c0a0640 0x7fe13c0a2b00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe144009fd0 tx=0x7fe144019040 comp rx=0 tx=0).stop 2023-09-13T03:12:53.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.540+0000 7fe158bff700 1 -- 172.21.15.64:0/2101230724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1541017e0 msgr2=0x7fe154196900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:53.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.540+0000 7fe158bff700 1 --2- 172.21.15.64:0/2101230724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1541017e0 0x7fe154196900 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe14800ec10 tx=0x7fe14800ef20 comp rx=0 tx=0).stop 2023-09-13T03:12:53.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.540+0000 7fe158bff700 1 -- 172.21.15.64:0/2101230724 shutdown_connections 2023-09-13T03:12:53.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.540+0000 7fe158bff700 1 --2- 172.21.15.64:0/2101230724 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe13c0a0640 0x7fe13c0a2b00 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:53.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.540+0000 7fe158bff700 1 --2- 172.21.15.64:0/2101230724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1541005f0 0x7fe1541963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:53.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.540+0000 7fe158bff700 1 --2- 172.21.15.64:0/2101230724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe154102180 0x7fe15419d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:53.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.540+0000 7fe158bff700 1 --2- 172.21.15.64:0/2101230724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1541017e0 0x7fe154196900 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:53.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.540+0000 7fe158bff700 1 -- 172.21.15.64:0/2101230724 >> 172.21.15.64:0/2101230724 conn(0x7fe1540fbb40 msgr2=0x7fe154104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:53.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.540+0000 7fe158bff700 1 -- 172.21.15.64:0/2101230724 shutdown_connections 2023-09-13T03:12:53.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.540+0000 7fe158bff700 1 -- 172.21.15.64:0/2101230724 wait complete. 2023-09-13T03:12:53.560 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-13T03:12:53.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.766+0000 7f419b5ff700 1 -- 172.21.15.64:0/2755896074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41941017e0 msgr2=0x7f4194101c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:53.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.766+0000 7f419b5ff700 1 --2- 172.21.15.64:0/2755896074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41941017e0 0x7f4194101c40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f41840099c0 tx=0x7f4184009cd0 comp rx=0 tx=0).stop 2023-09-13T03:12:53.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.766+0000 7f419b5ff700 1 -- 172.21.15.64:0/2755896074 shutdown_connections 2023-09-13T03:12:53.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.766+0000 7f419b5ff700 1 --2- 172.21.15.64:0/2755896074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41941005f0 0x7f4194100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:53.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.766+0000 7f419b5ff700 1 --2- 172.21.15.64:0/2755896074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4194102180 0x7f4194108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:53.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.766+0000 7f419b5ff700 1 --2- 172.21.15.64:0/2755896074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41941017e0 0x7f4194101c40 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:53.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.766+0000 7f419b5ff700 1 -- 172.21.15.64:0/2755896074 >> 172.21.15.64:0/2755896074 conn(0x7f41940fbb40 msgr2=0x7f41940fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:53.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.766+0000 7f419b5ff700 1 -- 172.21.15.64:0/2755896074 shutdown_connections 2023-09-13T03:12:53.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.766+0000 7f419b5ff700 1 -- 172.21.15.64:0/2755896074 wait complete. 2023-09-13T03:12:53.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.767+0000 7f419b5ff700 1 Processor -- start 2023-09-13T03:12:53.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.767+0000 7f419b5ff700 1 -- start start 2023-09-13T03:12:53.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.767+0000 7f419b5ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41941005f0 0x7f4194196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:53.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.767+0000 7f419b5ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41941017e0 0x7f4194196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:53.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.768+0000 7f419b5ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4194102180 0x7f419419d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:53.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.768+0000 7f419b5ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419419dec0 con 0x7f41941005f0 2023-09-13T03:12:53.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.768+0000 7f419b5ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419419e000 con 0x7f41941017e0 2023-09-13T03:12:53.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.768+0000 7f419b5ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419419e190 con 0x7f4194102180 2023-09-13T03:12:53.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.768+0000 7f419939b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41941005f0 0x7f4194196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:53.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.768+0000 7f419939b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41941005f0 0x7f4194196340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41050/0 (socket says 172.21.15.64:41050) 2023-09-13T03:12:53.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.768+0000 7f419939b700 1 -- 172.21.15.64:0/1329590005 learned_addr learned my addr 172.21.15.64:0/1329590005 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:53.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.768+0000 7f4199b9c700 1 --2- 172.21.15.64:0/1329590005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4194102180 0x7f419419d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:53.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.768+0000 7f4198b9a700 1 --2- 172.21.15.64:0/1329590005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41941017e0 0x7f4194196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:53.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.768+0000 7f419939b700 1 -- 172.21.15.64:0/1329590005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4194102180 msgr2=0x7f419419d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:53.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.768+0000 7f419939b700 1 --2- 172.21.15.64:0/1329590005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4194102180 0x7f419419d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:53.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.768+0000 7f419939b700 1 -- 172.21.15.64:0/1329590005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41941017e0 msgr2=0x7f4194196880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:53.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.768+0000 7f419939b700 1 --2- 172.21.15.64:0/1329590005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41941017e0 0x7f4194196880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:53.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.768+0000 7f419939b700 1 -- 172.21.15.64:0/1329590005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41840095f0 con 0x7f41941005f0 2023-09-13T03:12:53.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.768+0000 7f4198b9a700 1 --2- 172.21.15.64:0/1329590005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41941017e0 0x7f4194196880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:12:53.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.768+0000 7f419939b700 1 --2- 172.21.15.64:0/1329590005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41941005f0 0x7f4194196340 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f419000e970 tx=0x7f419000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:53.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.769+0000 7f418a7fc700 1 -- 172.21.15.64:0/1329590005 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4190009790 con 0x7f41941005f0 2023-09-13T03:12:53.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.769+0000 7f419b5ff700 1 -- 172.21.15.64:0/1329590005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f419419e470 con 0x7f41941005f0 2023-09-13T03:12:53.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.769+0000 7f418a7fc700 1 -- 172.21.15.64:0/1329590005 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f419000eec0 con 0x7f41941005f0 2023-09-13T03:12:53.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.769+0000 7f418a7fc700 1 -- 172.21.15.64:0/1329590005 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4190003cf0 con 0x7f41941005f0 2023-09-13T03:12:53.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.769+0000 7f419b5ff700 1 -- 172.21.15.64:0/1329590005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f419419e980 con 0x7f41941005f0 2023-09-13T03:12:53.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.770+0000 7f419b5ff700 1 -- 172.21.15.64:0/1329590005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4194107570 con 0x7f41941005f0 2023-09-13T03:12:53.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.772+0000 7f418a7fc700 1 -- 172.21.15.64:0/1329590005 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f419000ccd0 con 0x7f41941005f0 2023-09-13T03:12:53.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.773+0000 7f418a7fc700 1 --2- 172.21.15.64:0/1329590005 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f41800a0640 0x7f41800a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:53.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.774+0000 7f418a7fc700 1 -- 172.21.15.64:0/1329590005 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f4190014070 con 0x7f41941005f0 2023-09-13T03:12:53.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.774+0000 7f4198b9a700 1 --2- 172.21.15.64:0/1329590005 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f41800a0640 0x7f41800a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:53.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.774+0000 7f4198b9a700 1 --2- 172.21.15.64:0/1329590005 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f41800a0640 0x7f41800a2b00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f418400b580 tx=0x7f418401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:53.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.777+0000 7f418a7fc700 1 -- 172.21.15.64:0/1329590005 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f4190095940 con 0x7f41941005f0 2023-09-13T03:12:53.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:53 smithi170 ceph-mon[134328]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-13T03:12:53.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:53 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/593274469' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T03:12:53.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:53 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/2208047205' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:12:53.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:53 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/2101230724' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T03:12:53.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:53 smithi064 ceph-mon[97675]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-13T03:12:53.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:53 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/593274469' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T03:12:53.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:53 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2208047205' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:12:53.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:53 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2101230724' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T03:12:53.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.989+0000 7f419b5ff700 1 -- 172.21.15.64:0/1329590005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f419402cf30 con 0x7f41941005f0 2023-09-13T03:12:53.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.990+0000 7f418a7fc700 1 -- 172.21.15.64:0/1329590005 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4190098f60 con 0x7f41941005f0 2023-09-13T03:12:53.991 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-09-13T03:12:53.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.992+0000 7f419b5ff700 1 -- 172.21.15.64:0/1329590005 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f41800a0640 msgr2=0x7f41800a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:53.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.992+0000 7f419b5ff700 1 --2- 172.21.15.64:0/1329590005 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f41800a0640 0x7f41800a2b00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f418400b580 tx=0x7f418401a040 comp rx=0 tx=0).stop 2023-09-13T03:12:53.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.992+0000 7f419b5ff700 1 -- 172.21.15.64:0/1329590005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41941005f0 msgr2=0x7f4194196340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:53.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.992+0000 7f419b5ff700 1 --2- 172.21.15.64:0/1329590005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41941005f0 0x7f4194196340 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f419000e970 tx=0x7f419000ec80 comp rx=0 tx=0).stop 2023-09-13T03:12:53.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.993+0000 7f419b5ff700 1 -- 172.21.15.64:0/1329590005 shutdown_connections 2023-09-13T03:12:53.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.993+0000 7f419b5ff700 1 --2- 172.21.15.64:0/1329590005 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f41800a0640 0x7f41800a2b00 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:53.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.993+0000 7f419b5ff700 1 --2- 172.21.15.64:0/1329590005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4194102180 0x7f419419d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:53.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.993+0000 7f419b5ff700 1 --2- 172.21.15.64:0/1329590005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41941005f0 0x7f4194196340 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:53.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.993+0000 7f419b5ff700 1 --2- 172.21.15.64:0/1329590005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41941017e0 0x7f4194196880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:53.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.993+0000 7f419b5ff700 1 -- 172.21.15.64:0/1329590005 >> 172.21.15.64:0/1329590005 conn(0x7f41940fbb40 msgr2=0x7f41940fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:53.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.993+0000 7f419b5ff700 1 -- 172.21.15.64:0/1329590005 shutdown_connections 2023-09-13T03:12:53.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:53.993+0000 7f419b5ff700 1 -- 172.21.15.64:0/1329590005 wait complete. 2023-09-13T03:12:53.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:53 smithi142 ceph-mon[134082]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-13T03:12:53.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:53 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/593274469' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T03:12:53.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:53 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/2208047205' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:12:53.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:53 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/2101230724' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T03:12:54.012 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-13T03:12:54.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.220+0000 7f9d299ff700 1 -- 172.21.15.64:0/974094571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d241062d0 msgr2=0x7f9d24108740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:54.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.220+0000 7f9d299ff700 1 --2- 172.21.15.64:0/974094571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d241062d0 0x7f9d24108740 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c00b0a0 tx=0x7f9d1c00b3b0 comp rx=0 tx=0).stop 2023-09-13T03:12:54.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.220+0000 7f9d299ff700 1 -- 172.21.15.64:0/974094571 shutdown_connections 2023-09-13T03:12:54.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.220+0000 7f9d299ff700 1 --2- 172.21.15.64:0/974094571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d241062d0 0x7f9d24108740 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:54.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.220+0000 7f9d299ff700 1 --2- 172.21.15.64:0/974094571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d24101070 0x7f9d24103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:54.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.220+0000 7f9d299ff700 1 --2- 172.21.15.64:0/974094571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d241039a0 0x7f9d24105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:54.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.220+0000 7f9d299ff700 1 -- 172.21.15.64:0/974094571 >> 172.21.15.64:0/974094571 conn(0x7f9d240faa60 msgr2=0x7f9d240fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:54.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.220+0000 7f9d299ff700 1 -- 172.21.15.64:0/974094571 shutdown_connections 2023-09-13T03:12:54.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.220+0000 7f9d299ff700 1 -- 172.21.15.64:0/974094571 wait complete. 2023-09-13T03:12:54.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.221+0000 7f9d299ff700 1 Processor -- start 2023-09-13T03:12:54.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.221+0000 7f9d299ff700 1 -- start start 2023-09-13T03:12:54.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.221+0000 7f9d299ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d24101070 0x7f9d24198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:54.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.221+0000 7f9d299ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d241039a0 0x7f9d24198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:54.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.222+0000 7f9d299ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d241062d0 0x7f9d2419fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:54.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.222+0000 7f9d299ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d241a0180 con 0x7f9d24101070 2023-09-13T03:12:54.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.222+0000 7f9d299ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d241a02c0 con 0x7f9d241039a0 2023-09-13T03:12:54.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.222+0000 7f9d299ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d241a0450 con 0x7f9d241062d0 2023-09-13T03:12:54.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.222+0000 7f9d22ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d24101070 0x7f9d24198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:54.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.222+0000 7f9d227fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d241039a0 0x7f9d24198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:54.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.222+0000 7f9d22ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d24101070 0x7f9d24198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41068/0 (socket says 172.21.15.64:41068) 2023-09-13T03:12:54.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.222+0000 7f9d22ffd700 1 -- 172.21.15.64:0/1364201309 learned_addr learned my addr 172.21.15.64:0/1364201309 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:54.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.222+0000 7f9d237fe700 1 --2- 172.21.15.64:0/1364201309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d241062d0 0x7f9d2419fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:54.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.222+0000 7f9d22ffd700 1 -- 172.21.15.64:0/1364201309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d241062d0 msgr2=0x7f9d2419fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:54.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.222+0000 7f9d22ffd700 1 --2- 172.21.15.64:0/1364201309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d241062d0 0x7f9d2419fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:54.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.222+0000 7f9d22ffd700 1 -- 172.21.15.64:0/1364201309 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d241039a0 msgr2=0x7f9d24198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:54.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.222+0000 7f9d22ffd700 1 --2- 172.21.15.64:0/1364201309 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d241039a0 0x7f9d24198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:54.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.222+0000 7f9d22ffd700 1 -- 172.21.15.64:0/1364201309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d14009520 con 0x7f9d24101070 2023-09-13T03:12:54.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.222+0000 7f9d237fe700 1 --2- 172.21.15.64:0/1364201309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d241062d0 0x7f9d2419fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:12:54.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.222+0000 7f9d22ffd700 1 --2- 172.21.15.64:0/1364201309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d24101070 0x7f9d24198600 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400eb30 tx=0x7f9d1400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:54.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.223+0000 7f9d289fd700 1 -- 172.21.15.64:0/1364201309 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d14009880 con 0x7f9d24101070 2023-09-13T03:12:54.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.223+0000 7f9d289fd700 1 -- 172.21.15.64:0/1364201309 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d140044e0 con 0x7f9d24101070 2023-09-13T03:12:54.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.223+0000 7f9d289fd700 1 -- 172.21.15.64:0/1364201309 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d14003e90 con 0x7f9d24101070 2023-09-13T03:12:54.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.223+0000 7f9d299ff700 1 -- 172.21.15.64:0/1364201309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d18009520 con 0x7f9d24101070 2023-09-13T03:12:54.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.223+0000 7f9d299ff700 1 -- 172.21.15.64:0/1364201309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d1c009d00 con 0x7f9d24101070 2023-09-13T03:12:54.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.224+0000 7f9d299ff700 1 -- 172.21.15.64:0/1364201309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d24107e50 con 0x7f9d24101070 2023-09-13T03:12:54.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.226+0000 7f9d289fd700 1 -- 172.21.15.64:0/1364201309 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f9d14004000 con 0x7f9d24101070 2023-09-13T03:12:54.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.228+0000 7f9d289fd700 1 --2- 172.21.15.64:0/1364201309 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f9d0c0a06c0 0x7f9d0c0a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:54.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.228+0000 7f9d289fd700 1 -- 172.21.15.64:0/1364201309 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f9d14014070 con 0x7f9d24101070 2023-09-13T03:12:54.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.228+0000 7f9d227fc700 1 --2- 172.21.15.64:0/1364201309 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f9d0c0a06c0 0x7f9d0c0a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:54.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.229+0000 7f9d227fc700 1 --2- 172.21.15.64:0/1364201309 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f9d0c0a06c0 0x7f9d0c0a2b80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9d18009df0 tx=0x7f9d18009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:54.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.233+0000 7f9d289fd700 1 -- 172.21.15.64:0/1364201309 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f9d14095a90 con 0x7f9d24101070 2023-09-13T03:12:54.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.442+0000 7f9d299ff700 1 -- 172.21.15.64:0/1364201309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9d2410be40 con 0x7f9d24101070 2023-09-13T03:12:54.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.442+0000 7f9d289fd700 1 -- 172.21.15.64:0/1364201309 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9d140990b0 con 0x7f9d24101070 2023-09-13T03:12:54.443 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-09-13T03:12:54.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.445+0000 7f9d299ff700 1 -- 172.21.15.64:0/1364201309 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f9d0c0a06c0 msgr2=0x7f9d0c0a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:54.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.445+0000 7f9d299ff700 1 --2- 172.21.15.64:0/1364201309 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f9d0c0a06c0 0x7f9d0c0a2b80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9d18009df0 tx=0x7f9d18009260 comp rx=0 tx=0).stop 2023-09-13T03:12:54.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.445+0000 7f9d299ff700 1 -- 172.21.15.64:0/1364201309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d24101070 msgr2=0x7f9d24198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:54.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.445+0000 7f9d299ff700 1 --2- 172.21.15.64:0/1364201309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d24101070 0x7f9d24198600 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400eb30 tx=0x7f9d1400ee40 comp rx=0 tx=0).stop 2023-09-13T03:12:54.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.445+0000 7f9d299ff700 1 -- 172.21.15.64:0/1364201309 shutdown_connections 2023-09-13T03:12:54.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.445+0000 7f9d299ff700 1 --2- 172.21.15.64:0/1364201309 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f9d0c0a06c0 0x7f9d0c0a2b80 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:54.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.445+0000 7f9d299ff700 1 --2- 172.21.15.64:0/1364201309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d241062d0 0x7f9d2419fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:54.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.445+0000 7f9d299ff700 1 --2- 172.21.15.64:0/1364201309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d24101070 0x7f9d24198600 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:54.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.445+0000 7f9d299ff700 1 --2- 172.21.15.64:0/1364201309 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d241039a0 0x7f9d24198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:54.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.445+0000 7f9d299ff700 1 -- 172.21.15.64:0/1364201309 >> 172.21.15.64:0/1364201309 conn(0x7f9d240faa60 msgr2=0x7f9d24101ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:54.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.445+0000 7f9d299ff700 1 -- 172.21.15.64:0/1364201309 shutdown_connections 2023-09-13T03:12:54.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.445+0000 7f9d299ff700 1 -- 172.21.15.64:0/1364201309 wait complete. 2023-09-13T03:12:54.464 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-09-13T03:12:54.465 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T03:12:54.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.669+0000 7fd31d5ff700 1 -- 172.21.15.64:0/1941081128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd318100610 msgr2=0x7fd318100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:54.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.669+0000 7fd31d5ff700 1 --2- 172.21.15.64:0/1941081128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd318100610 0x7fd318100a30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd308009920 tx=0x7fd308009c30 comp rx=0 tx=0).stop 2023-09-13T03:12:54.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.669+0000 7fd31d5ff700 1 -- 172.21.15.64:0/1941081128 shutdown_connections 2023-09-13T03:12:54.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.669+0000 7fd31d5ff700 1 --2- 172.21.15.64:0/1941081128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd318100610 0x7fd318100a30 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:54.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.669+0000 7fd31d5ff700 1 --2- 172.21.15.64:0/1941081128 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3181021a0 0x7fd318108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:54.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.669+0000 7fd31d5ff700 1 --2- 172.21.15.64:0/1941081128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd318101800 0x7fd318101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:54.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.669+0000 7fd31d5ff700 1 -- 172.21.15.64:0/1941081128 >> 172.21.15.64:0/1941081128 conn(0x7fd3180fbb40 msgr2=0x7fd3180fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:54.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.669+0000 7fd31d5ff700 1 -- 172.21.15.64:0/1941081128 shutdown_connections 2023-09-13T03:12:54.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.669+0000 7fd31d5ff700 1 -- 172.21.15.64:0/1941081128 wait complete. 2023-09-13T03:12:54.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.670+0000 7fd31d5ff700 1 Processor -- start 2023-09-13T03:12:54.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.670+0000 7fd31d5ff700 1 -- start start 2023-09-13T03:12:54.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.671+0000 7fd31d5ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd318100610 0x7fd318198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:54.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.671+0000 7fd31d5ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd318101800 0x7fd318198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:54.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.671+0000 7fd31d5ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3181021a0 0x7fd31819fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:54.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.671+0000 7fd31d5ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3181a00f0 con 0x7fd318101800 2023-09-13T03:12:54.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.671+0000 7fd31d5ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3181a0230 con 0x7fd3181021a0 2023-09-13T03:12:54.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.671+0000 7fd31d5ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3181a03c0 con 0x7fd318100610 2023-09-13T03:12:54.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.671+0000 7fd3167fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd318101800 0x7fd318198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:54.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.671+0000 7fd3167fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd318101800 0x7fd318198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41086/0 (socket says 172.21.15.64:41086) 2023-09-13T03:12:54.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.671+0000 7fd3167fc700 1 -- 172.21.15.64:0/1278840691 learned_addr learned my addr 172.21.15.64:0/1278840691 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:54.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.671+0000 7fd316ffd700 1 --2- 172.21.15.64:0/1278840691 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd318100610 0x7fd318198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:54.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.671+0000 7fd3177fe700 1 --2- 172.21.15.64:0/1278840691 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3181021a0 0x7fd31819fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:54.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.671+0000 7fd3167fc700 1 -- 172.21.15.64:0/1278840691 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd318100610 msgr2=0x7fd318198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:54.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.671+0000 7fd3167fc700 1 --2- 172.21.15.64:0/1278840691 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd318100610 0x7fd318198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:54.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.671+0000 7fd3167fc700 1 -- 172.21.15.64:0/1278840691 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3181021a0 msgr2=0x7fd31819fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:54.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.671+0000 7fd3167fc700 1 --2- 172.21.15.64:0/1278840691 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3181021a0 0x7fd31819fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:54.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.671+0000 7fd3167fc700 1 -- 172.21.15.64:0/1278840691 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd308009520 con 0x7fd318101800 2023-09-13T03:12:54.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.672+0000 7fd3167fc700 1 --2- 172.21.15.64:0/1278840691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd318101800 0x7fd318198ab0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd30c00e9c0 tx=0x7fd30c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:54.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.672+0000 7fd2fffff700 1 -- 172.21.15.64:0/1278840691 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30c004a30 con 0x7fd318101800 2023-09-13T03:12:54.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.672+0000 7fd31d5ff700 1 -- 172.21.15.64:0/1278840691 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3181a06a0 con 0x7fd318101800 2023-09-13T03:12:54.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.672+0000 7fd2fffff700 1 -- 172.21.15.64:0/1278840691 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd30c00cc80 con 0x7fd318101800 2023-09-13T03:12:54.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.672+0000 7fd2fffff700 1 -- 172.21.15.64:0/1278840691 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30c0054e0 con 0x7fd318101800 2023-09-13T03:12:54.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.672+0000 7fd31d5ff700 1 -- 172.21.15.64:0/1278840691 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3181a0bb0 con 0x7fd318101800 2023-09-13T03:12:54.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.673+0000 7fd31d5ff700 1 -- 172.21.15.64:0/1278840691 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd318192c70 con 0x7fd318101800 2023-09-13T03:12:54.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.675+0000 7fd2fffff700 1 -- 172.21.15.64:0/1278840691 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fd30c004b90 con 0x7fd318101800 2023-09-13T03:12:54.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.677+0000 7fd2fffff700 1 --2- 172.21.15.64:0/1278840691 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd3000a0690 0x7fd3000a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:54.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.677+0000 7fd2fffff700 1 -- 172.21.15.64:0/1278840691 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fd30c014070 con 0x7fd318101800 2023-09-13T03:12:54.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.677+0000 7fd316ffd700 1 --2- 172.21.15.64:0/1278840691 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd3000a0690 0x7fd3000a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:54.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.677+0000 7fd316ffd700 1 --2- 172.21.15.64:0/1278840691 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd3000a0690 0x7fd3000a2b50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd3080098f0 tx=0x7fd308019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:54.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.680+0000 7fd2fffff700 1 -- 172.21.15.64:0/1278840691 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fd30c095790 con 0x7fd318101800 2023-09-13T03:12:54.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:54 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/1329590005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T03:12:54.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:54 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/1364201309' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T03:12:54.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:54 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/1329590005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T03:12:54.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:54 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/1364201309' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T03:12:54.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.938+0000 7fd31d5ff700 1 -- 172.21.15.64:0/1278840691 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd31810be80 con 0x7fd318101800 2023-09-13T03:12:54.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.939+0000 7fd2fffff700 1 -- 172.21.15.64:0/1278840691 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11366 (secure 0 0 0) 0x7fd30c098db0 con 0x7fd318101800 2023-09-13T03:12:54.941 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:12:54.942 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2234182985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2234182985},{"type":"v1","addr":"172.21.15.64:6835","nonce":2234182985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4053841873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4053841873},{"type":"v1","addr":"172.21.15.64:6837","nonce":4053841873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-13T03:11:10.490260+0000","modified":"2023-09-13T03:11:18.230740+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24686,"mds_1":24649,"mds_2":24635,"mds_3":24634,"mds_4":24622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6839/99523374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":99523374},{"type":"v1","addr":"172.21.15.64:6839","nonce":99523374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14649":{"gid":14649,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6841/1866582875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":1866582875},{"type":"v1","addr":"172.21.15.64:6841","nonce":1866582875}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/101906999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":101906999},{"type":"v1","addr":"172.21.15.142:6833","nonce":101906999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24574":{"gid":24574,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/1897366227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1897366227},{"type":"v1","addr":"172.21.15.170:6833","nonce":1897366227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24611":{"gid":24611,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2167779918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2167779918},{"type":"v1","addr":"172.21.15.142:6835","nonce":2167779918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24622":{"gid":24622,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6835/1493371121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1493371121},{"type":"v1","addr":"172.21.15.170:6835","nonce":1493371121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24634":{"gid":24634,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.170:6837/1726587275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":1726587275},{"type":"v1","addr":"172.21.15.170:6837","nonce":1726587275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24635":{"gid":24635,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.142:6837/741173268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":741173268},{"type":"v1","addr":"172.21.15.142:6837","nonce":741173268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24649":{"gid":24649,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.170:6839/2189703596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2189703596},{"type":"v1","addr":"172.21.15.170:6839","nonce":2189703596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24686":{"gid":24686,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/3645998616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645998616},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645998616}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-13T03:12:54.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.942+0000 7fd31d5ff700 1 -- 172.21.15.64:0/1278840691 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd3000a0690 msgr2=0x7fd3000a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:54.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.942+0000 7fd31d5ff700 1 --2- 172.21.15.64:0/1278840691 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd3000a0690 0x7fd3000a2b50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd3080098f0 tx=0x7fd308019040 comp rx=0 tx=0).stop 2023-09-13T03:12:54.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.942+0000 7fd31d5ff700 1 -- 172.21.15.64:0/1278840691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd318101800 msgr2=0x7fd318198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:54.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.942+0000 7fd31d5ff700 1 --2- 172.21.15.64:0/1278840691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd318101800 0x7fd318198ab0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd30c00e9c0 tx=0x7fd30c00ecd0 comp rx=0 tx=0).stop 2023-09-13T03:12:54.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.942+0000 7fd31d5ff700 1 -- 172.21.15.64:0/1278840691 shutdown_connections 2023-09-13T03:12:54.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.942+0000 7fd31d5ff700 1 --2- 172.21.15.64:0/1278840691 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fd3000a0690 0x7fd3000a2b50 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:54.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.942+0000 7fd31d5ff700 1 --2- 172.21.15.64:0/1278840691 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3181021a0 0x7fd31819fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:54.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.942+0000 7fd31d5ff700 1 --2- 172.21.15.64:0/1278840691 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd318100610 0x7fd318198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:54.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.942+0000 7fd31d5ff700 1 --2- 172.21.15.64:0/1278840691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd318101800 0x7fd318198ab0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:54.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.942+0000 7fd31d5ff700 1 -- 172.21.15.64:0/1278840691 >> 172.21.15.64:0/1278840691 conn(0x7fd3180fbb40 msgr2=0x7fd318107120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:54.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.942+0000 7fd31d5ff700 1 -- 172.21.15.64:0/1278840691 shutdown_connections 2023-09-13T03:12:54.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:54.942+0000 7fd31d5ff700 1 -- 172.21.15.64:0/1278840691 wait complete. 2023-09-13T03:12:54.945 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 26 2023-09-13T03:12:54.963 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-09-13T03:12:54.964 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-09-13T03:12:54.964 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-13T03:12:54.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:54 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/1329590005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T03:12:54.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:54 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/1364201309' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T03:12:55.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.168+0000 7f6f90dff700 1 -- 172.21.15.64:0/991312817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f8c101800 msgr2=0x7f6f8c101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:55.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.168+0000 7f6f90dff700 1 --2- 172.21.15.64:0/991312817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f8c101800 0x7f6f8c101c60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6f740099c0 tx=0x7f6f74009cd0 comp rx=0 tx=0).stop 2023-09-13T03:12:55.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.168+0000 7f6f90dff700 1 -- 172.21.15.64:0/991312817 shutdown_connections 2023-09-13T03:12:55.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.168+0000 7f6f90dff700 1 --2- 172.21.15.64:0/991312817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f8c100610 0x7f6f8c100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:55.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.168+0000 7f6f90dff700 1 --2- 172.21.15.64:0/991312817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f8c1021a0 0x7f6f8c108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:55.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.168+0000 7f6f90dff700 1 --2- 172.21.15.64:0/991312817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f8c101800 0x7f6f8c101c60 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:55.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.168+0000 7f6f90dff700 1 -- 172.21.15.64:0/991312817 >> 172.21.15.64:0/991312817 conn(0x7f6f8c0fbb40 msgr2=0x7f6f8c0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:55.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.168+0000 7f6f90dff700 1 -- 172.21.15.64:0/991312817 shutdown_connections 2023-09-13T03:12:55.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.168+0000 7f6f90dff700 1 -- 172.21.15.64:0/991312817 wait complete. 2023-09-13T03:12:55.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.169+0000 7f6f90dff700 1 Processor -- start 2023-09-13T03:12:55.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.169+0000 7f6f90dff700 1 -- start start 2023-09-13T03:12:55.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.169+0000 7f6f90dff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f8c100610 0x7f6f8c198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:55.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.169+0000 7f6f90dff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f8c101800 0x7f6f8c198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:55.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.170+0000 7f6f90dff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f8c1021a0 0x7f6f8c19fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:55.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.170+0000 7f6f90dff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8c1a00b0 con 0x7f6f8c1021a0 2023-09-13T03:12:55.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.170+0000 7f6f90dff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8c1a01f0 con 0x7f6f8c101800 2023-09-13T03:12:55.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.170+0000 7f6f90dff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8c1a0380 con 0x7f6f8c100610 2023-09-13T03:12:55.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.170+0000 7f6f8ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f8c1021a0 0x7f6f8c19fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:55.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.170+0000 7f6f8ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f8c1021a0 0x7f6f8c19fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41096/0 (socket says 172.21.15.64:41096) 2023-09-13T03:12:55.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.170+0000 7f6f8ad9d700 1 -- 172.21.15.64:0/421151087 learned_addr learned my addr 172.21.15.64:0/421151087 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:55.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.170+0000 7f6f83fff700 1 --2- 172.21.15.64:0/421151087 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f8c101800 0x7f6f8c198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:55.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.170+0000 7f6f8a59c700 1 --2- 172.21.15.64:0/421151087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f8c100610 0x7f6f8c198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:55.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.170+0000 7f6f8ad9d700 1 -- 172.21.15.64:0/421151087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f8c100610 msgr2=0x7f6f8c198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:55.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.170+0000 7f6f8ad9d700 1 --2- 172.21.15.64:0/421151087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f8c100610 0x7f6f8c198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:55.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.170+0000 7f6f8ad9d700 1 -- 172.21.15.64:0/421151087 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f8c101800 msgr2=0x7f6f8c198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:55.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.170+0000 7f6f8ad9d700 1 --2- 172.21.15.64:0/421151087 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f8c101800 0x7f6f8c198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:55.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.170+0000 7f6f8ad9d700 1 -- 172.21.15.64:0/421151087 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f740095f0 con 0x7f6f8c1021a0 2023-09-13T03:12:55.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.171+0000 7f6f8ad9d700 1 --2- 172.21.15.64:0/421151087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f8c1021a0 0x7f6f8c19fae0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6f8400ef30 tx=0x7f6f8400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:55.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.171+0000 7f6f837fe700 1 -- 172.21.15.64:0/421151087 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f84018070 con 0x7f6f8c1021a0 2023-09-13T03:12:55.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.171+0000 7f6f90dff700 1 -- 172.21.15.64:0/421151087 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f8c1a0600 con 0x7f6f8c1021a0 2023-09-13T03:12:55.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.171+0000 7f6f837fe700 1 -- 172.21.15.64:0/421151087 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f8400f040 con 0x7f6f8c1021a0 2023-09-13T03:12:55.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.171+0000 7f6f837fe700 1 -- 172.21.15.64:0/421151087 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8400cd00 con 0x7f6f8c1021a0 2023-09-13T03:12:55.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.171+0000 7f6f90dff700 1 -- 172.21.15.64:0/421151087 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f8c1a0b10 con 0x7f6f8c1021a0 2023-09-13T03:12:55.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.172+0000 7f6f90dff700 1 -- 172.21.15.64:0/421151087 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f8c192c60 con 0x7f6f8c1021a0 2023-09-13T03:12:55.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.174+0000 7f6f837fe700 1 -- 172.21.15.64:0/421151087 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f6f84003a00 con 0x7f6f8c1021a0 2023-09-13T03:12:55.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.175+0000 7f6f837fe700 1 --2- 172.21.15.64:0/421151087 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6f780a0690 0x7f6f780a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:55.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.175+0000 7f6f837fe700 1 -- 172.21.15.64:0/421151087 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f6f840cb650 con 0x7f6f8c1021a0 2023-09-13T03:12:55.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.175+0000 7f6f8a59c700 1 --2- 172.21.15.64:0/421151087 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6f780a0690 0x7f6f780a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:55.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.176+0000 7f6f8a59c700 1 --2- 172.21.15.64:0/421151087 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6f780a0690 0x7f6f780a2b50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c101660 tx=0x7f6f7c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:55.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.179+0000 7f6f837fe700 1 -- 172.21.15.64:0/421151087 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6f840951e0 con 0x7f6f8c1021a0 2023-09-13T03:12:55.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.387+0000 7f6f90dff700 1 -- 172.21.15.64:0/421151087 --> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6f8c02cff0 con 0x7f6f780a0690 2023-09-13T03:12:55.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.399+0000 7f6f837fe700 1 -- 172.21.15.64:0/421151087 <== mgr.14150 v2:172.21.15.64:6800/873679308 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345051 (secure 0 0 0) 0x7f6f8c02cff0 con 0x7f6f780a0690 2023-09-13T03:12:55.407 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:12:55.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.403+0000 7f6f90dff700 1 -- 172.21.15.64:0/421151087 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6f780a0690 msgr2=0x7f6f780a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:55.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.403+0000 7f6f90dff700 1 --2- 172.21.15.64:0/421151087 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6f780a0690 0x7f6f780a2b50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c101660 tx=0x7f6f7c009260 comp rx=0 tx=0).stop 2023-09-13T03:12:55.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.403+0000 7f6f90dff700 1 -- 172.21.15.64:0/421151087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f8c1021a0 msgr2=0x7f6f8c19fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:55.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.403+0000 7f6f90dff700 1 --2- 172.21.15.64:0/421151087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f8c1021a0 0x7f6f8c19fae0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6f8400ef30 tx=0x7f6f8400c3a0 comp rx=0 tx=0).stop 2023-09-13T03:12:55.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.404+0000 7f6f90dff700 1 -- 172.21.15.64:0/421151087 shutdown_connections 2023-09-13T03:12:55.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.404+0000 7f6f90dff700 1 --2- 172.21.15.64:0/421151087 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f6f780a0690 0x7f6f780a2b50 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:55.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.404+0000 7f6f90dff700 1 --2- 172.21.15.64:0/421151087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f8c1021a0 0x7f6f8c19fae0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:55.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.404+0000 7f6f90dff700 1 --2- 172.21.15.64:0/421151087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f8c100610 0x7f6f8c198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:55.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.404+0000 7f6f90dff700 1 --2- 172.21.15.64:0/421151087 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f8c101800 0x7f6f8c198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:55.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.404+0000 7f6f90dff700 1 -- 172.21.15.64:0/421151087 >> 172.21.15.64:0/421151087 conn(0x7f6f8c0fbb40 msgr2=0x7f6f8c1070c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:55.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.404+0000 7f6f90dff700 1 -- 172.21.15.64:0/421151087 shutdown_connections 2023-09-13T03:12:55.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.404+0000 7f6f90dff700 1 -- 172.21.15.64:0/421151087 wait complete. 2023-09-13T03:12:55.412 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-13T03:12:55.423 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2023-09-13T03:12:54.478794+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656507,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1120,"num_read_kb":684,"num_write":428,"num_write_kb":3077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180804,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":853,"ondisk_log_size":853,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337200,"kb_used_data":13008,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604624,"statfs":{"total":1151940427776,"available":1151595134976,"internally_reserved":0,"allocated":13320192,"data_stored":7285612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19130,"internal_metadata":331855174},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":316,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":136,"num_read_kb":71,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002563"},"pg_stats":[{"pgid":"3.3f","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.480907+0000","last_change":"2023-09-13T03:11:10.949449+0000","last_active":"2023-09-13T03:12:22.480907+0000","last_peered":"2023-09-13T03:12:22.480907+0000","last_clean":"2023-09-13T03:12:22.480907+0000","last_became_active":"2023-09-13T03:11:10.948912+0000","last_became_peered":"2023-09-13T03:11:10.948912+0000","last_unstale":"2023-09-13T03:12:22.480907+0000","last_undegraded":"2023-09-13T03:12:22.480907+0000","last_fullsized":"2023-09-13T03:12:22.480907+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:34:27.381352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.479553+0000","last_change":"2023-09-13T03:11:10.956817+0000","last_active":"2023-09-13T03:12:22.479553+0000","last_peered":"2023-09-13T03:12:22.479553+0000","last_clean":"2023-09-13T03:12:22.479553+0000","last_became_active":"2023-09-13T03:11:10.955359+0000","last_became_peered":"2023-09-13T03:11:10.955359+0000","last_unstale":"2023-09-13T03:12:22.479553+0000","last_undegraded":"2023-09-13T03:12:22.479553+0000","last_fullsized":"2023-09-13T03:12:22.479553+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:06:01.055910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.483472+0000","last_change":"2023-09-13T03:11:10.959915+0000","last_active":"2023-09-13T03:12:22.483472+0000","last_peered":"2023-09-13T03:12:22.483472+0000","last_clean":"2023-09-13T03:12:22.483472+0000","last_became_active":"2023-09-13T03:11:10.959611+0000","last_became_peered":"2023-09-13T03:11:10.959611+0000","last_unstale":"2023-09-13T03:12:22.483472+0000","last_undegraded":"2023-09-13T03:12:22.483472+0000","last_fullsized":"2023-09-13T03:12:22.483472+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:31:40.075569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.474409+0000","last_change":"2023-09-13T03:11:24.269022+0000","last_active":"2023-09-13T03:12:22.474409+0000","last_peered":"2023-09-13T03:12:22.474409+0000","last_clean":"2023-09-13T03:12:22.474409+0000","last_became_active":"2023-09-13T03:11:24.268019+0000","last_became_peered":"2023-09-13T03:11:24.268019+0000","last_unstale":"2023-09-13T03:12:22.474409+0000","last_undegraded":"2023-09-13T03:12:22.474409+0000","last_fullsized":"2023-09-13T03:12:22.474409+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:58:06.518437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.469930+0000","last_change":"2023-09-13T03:11:10.951554+0000","last_active":"2023-09-13T03:12:22.469930+0000","last_peered":"2023-09-13T03:12:22.469930+0000","last_clean":"2023-09-13T03:12:22.469930+0000","last_became_active":"2023-09-13T03:11:10.951252+0000","last_became_peered":"2023-09-13T03:11:10.951252+0000","last_unstale":"2023-09-13T03:12:22.469930+0000","last_undegraded":"2023-09-13T03:12:22.469930+0000","last_fullsized":"2023-09-13T03:12:22.469930+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:38:42.890256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.487106+0000","last_change":"2023-09-13T03:11:10.953087+0000","last_active":"2023-09-13T03:12:22.487106+0000","last_peered":"2023-09-13T03:12:22.487106+0000","last_clean":"2023-09-13T03:12:22.487106+0000","last_became_active":"2023-09-13T03:11:10.952773+0000","last_became_peered":"2023-09-13T03:11:10.952773+0000","last_unstale":"2023-09-13T03:12:22.487106+0000","last_undegraded":"2023-09-13T03:12:22.487106+0000","last_fullsized":"2023-09-13T03:12:22.487106+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:14:55.203347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.481903+0000","last_change":"2023-09-13T03:11:10.956058+0000","last_active":"2023-09-13T03:12:22.481903+0000","last_peered":"2023-09-13T03:12:22.481903+0000","last_clean":"2023-09-13T03:12:22.481903+0000","last_became_active":"2023-09-13T03:11:10.955783+0000","last_became_peered":"2023-09-13T03:11:10.955783+0000","last_unstale":"2023-09-13T03:12:22.481903+0000","last_undegraded":"2023-09-13T03:12:22.481903+0000","last_fullsized":"2023-09-13T03:12:22.481903+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:31:06.810338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.482123+0000","last_change":"2023-09-13T03:11:10.950102+0000","last_active":"2023-09-13T03:12:22.482123+0000","last_peered":"2023-09-13T03:12:22.482123+0000","last_clean":"2023-09-13T03:12:22.482123+0000","last_became_active":"2023-09-13T03:11:10.948877+0000","last_became_peered":"2023-09-13T03:11:10.948877+0000","last_unstale":"2023-09-13T03:12:22.482123+0000","last_undegraded":"2023-09-13T03:12:22.482123+0000","last_fullsized":"2023-09-13T03:12:22.482123+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:32:09.168793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.480791+0000","last_change":"2023-09-13T03:11:10.945244+0000","last_active":"2023-09-13T03:12:22.480791+0000","last_peered":"2023-09-13T03:12:22.480791+0000","last_clean":"2023-09-13T03:12:22.480791+0000","last_became_active":"2023-09-13T03:11:10.944951+0000","last_became_peered":"2023-09-13T03:11:10.944951+0000","last_unstale":"2023-09-13T03:12:22.480791+0000","last_undegraded":"2023-09-13T03:12:22.480791+0000","last_fullsized":"2023-09-13T03:12:22.480791+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:42:35.420767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.487770+0000","last_change":"2023-09-13T03:11:10.952312+0000","last_active":"2023-09-13T03:12:22.487770+0000","last_peered":"2023-09-13T03:12:22.487770+0000","last_clean":"2023-09-13T03:12:22.487770+0000","last_became_active":"2023-09-13T03:11:10.952077+0000","last_became_peered":"2023-09-13T03:11:10.952077+0000","last_unstale":"2023-09-13T03:12:22.487770+0000","last_undegraded":"2023-09-13T03:12:22.487770+0000","last_fullsized":"2023-09-13T03:12:22.487770+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:57:10.888184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.478509+0000","last_change":"2023-09-13T03:11:10.959259+0000","last_active":"2023-09-13T03:12:22.478509+0000","last_peered":"2023-09-13T03:12:22.478509+0000","last_clean":"2023-09-13T03:12:22.478509+0000","last_became_active":"2023-09-13T03:11:10.958922+0000","last_became_peered":"2023-09-13T03:11:10.958922+0000","last_unstale":"2023-09-13T03:12:22.478509+0000","last_undegraded":"2023-09-13T03:12:22.478509+0000","last_fullsized":"2023-09-13T03:12:22.478509+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:08:54.680718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.478268+0000","last_change":"2023-09-13T03:11:10.950946+0000","last_active":"2023-09-13T03:12:22.478268+0000","last_peered":"2023-09-13T03:12:22.478268+0000","last_clean":"2023-09-13T03:12:22.478268+0000","last_became_active":"2023-09-13T03:11:10.950633+0000","last_became_peered":"2023-09-13T03:11:10.950633+0000","last_unstale":"2023-09-13T03:12:22.478268+0000","last_undegraded":"2023-09-13T03:12:22.478268+0000","last_fullsized":"2023-09-13T03:12:22.478268+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:37:25.796555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.480994+0000","last_change":"2023-09-13T03:11:10.957712+0000","last_active":"2023-09-13T03:12:22.480994+0000","last_peered":"2023-09-13T03:12:22.480994+0000","last_clean":"2023-09-13T03:12:22.480994+0000","last_became_active":"2023-09-13T03:11:10.957379+0000","last_became_peered":"2023-09-13T03:11:10.957379+0000","last_unstale":"2023-09-13T03:12:22.480994+0000","last_undegraded":"2023-09-13T03:12:22.480994+0000","last_fullsized":"2023-09-13T03:12:22.480994+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:31:17.084185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.480310+0000","last_change":"2023-09-13T03:11:10.952951+0000","last_active":"2023-09-13T03:12:22.480310+0000","last_peered":"2023-09-13T03:12:22.480310+0000","last_clean":"2023-09-13T03:12:22.480310+0000","last_became_active":"2023-09-13T03:11:10.952629+0000","last_became_peered":"2023-09-13T03:11:10.952629+0000","last_unstale":"2023-09-13T03:12:22.480310+0000","last_undegraded":"2023-09-13T03:12:22.480310+0000","last_fullsized":"2023-09-13T03:12:22.480310+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:20:40.802584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.486815+0000","last_change":"2023-09-13T03:11:10.950799+0000","last_active":"2023-09-13T03:12:22.486815+0000","last_peered":"2023-09-13T03:12:22.486815+0000","last_clean":"2023-09-13T03:12:22.486815+0000","last_became_active":"2023-09-13T03:11:10.950500+0000","last_became_peered":"2023-09-13T03:11:10.950500+0000","last_unstale":"2023-09-13T03:12:22.486815+0000","last_undegraded":"2023-09-13T03:12:22.486815+0000","last_fullsized":"2023-09-13T03:12:22.486815+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:24:45.770696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.473581+0000","last_change":"2023-09-13T03:11:10.955141+0000","last_active":"2023-09-13T03:12:22.473581+0000","last_peered":"2023-09-13T03:12:22.473581+0000","last_clean":"2023-09-13T03:12:22.473581+0000","last_became_active":"2023-09-13T03:11:10.954840+0000","last_became_peered":"2023-09-13T03:11:10.954840+0000","last_unstale":"2023-09-13T03:12:22.473581+0000","last_undegraded":"2023-09-13T03:12:22.473581+0000","last_fullsized":"2023-09-13T03:12:22.473581+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:56:08.318784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.483342+0000","last_change":"2023-09-13T03:11:10.949199+0000","last_active":"2023-09-13T03:12:22.483342+0000","last_peered":"2023-09-13T03:12:22.483342+0000","last_clean":"2023-09-13T03:12:22.483342+0000","last_became_active":"2023-09-13T03:11:10.948506+0000","last_became_peered":"2023-09-13T03:11:10.948506+0000","last_unstale":"2023-09-13T03:12:22.483342+0000","last_undegraded":"2023-09-13T03:12:22.483342+0000","last_fullsized":"2023-09-13T03:12:22.483342+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:04:07.714033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.481246+0000","last_change":"2023-09-13T03:11:10.953207+0000","last_active":"2023-09-13T03:12:22.481246+0000","last_peered":"2023-09-13T03:12:22.481246+0000","last_clean":"2023-09-13T03:12:22.481246+0000","last_became_active":"2023-09-13T03:11:10.951566+0000","last_became_peered":"2023-09-13T03:11:10.951566+0000","last_unstale":"2023-09-13T03:12:22.481246+0000","last_undegraded":"2023-09-13T03:12:22.481246+0000","last_fullsized":"2023-09-13T03:12:22.481246+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:23:06.020880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.473777+0000","last_change":"2023-09-13T03:11:10.951695+0000","last_active":"2023-09-13T03:12:22.473777+0000","last_peered":"2023-09-13T03:12:22.473777+0000","last_clean":"2023-09-13T03:12:22.473777+0000","last_became_active":"2023-09-13T03:11:10.951382+0000","last_became_peered":"2023-09-13T03:11:10.951382+0000","last_unstale":"2023-09-13T03:12:22.473777+0000","last_undegraded":"2023-09-13T03:12:22.473777+0000","last_fullsized":"2023-09-13T03:12:22.473777+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:29:05.200980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'14","reported_seq":124,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:50.375945+0000","last_change":"2023-09-13T03:11:09.956193+0000","last_active":"2023-09-13T03:12:50.375945+0000","last_peered":"2023-09-13T03:12:50.375945+0000","last_clean":"2023-09-13T03:12:50.375945+0000","last_became_active":"2023-09-13T03:11:09.955778+0000","last_became_peered":"2023-09-13T03:11:09.955778+0000","last_unstale":"2023-09-13T03:12:50.375945+0000","last_undegraded":"2023-09-13T03:12:50.375945+0000","last_fullsized":"2023-09-13T03:12:50.375945+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:15:57.425730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":95,"num_read_kb":95,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.475680+0000","last_change":"2023-09-13T03:11:10.944437+0000","last_active":"2023-09-13T03:12:22.475680+0000","last_peered":"2023-09-13T03:12:22.475680+0000","last_clean":"2023-09-13T03:12:22.475680+0000","last_became_active":"2023-09-13T03:11:10.943961+0000","last_became_peered":"2023-09-13T03:11:10.943961+0000","last_unstale":"2023-09-13T03:12:22.475680+0000","last_undegraded":"2023-09-13T03:12:22.475680+0000","last_fullsized":"2023-09-13T03:12:22.475680+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:39:43.928334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.266435+0000","last_change":"2023-09-13T03:11:09.952017+0000","last_active":"2023-09-13T03:11:23.266435+0000","last_peered":"2023-09-13T03:11:23.266435+0000","last_clean":"2023-09-13T03:11:23.266435+0000","last_became_active":"2023-09-13T03:11:09.951700+0000","last_became_peered":"2023-09-13T03:11:09.951700+0000","last_unstale":"2023-09-13T03:11:23.266435+0000","last_undegraded":"2023-09-13T03:11:23.266435+0000","last_fullsized":"2023-09-13T03:11:23.266435+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:28:50.687271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.480661+0000","last_change":"2023-09-13T03:11:10.960198+0000","last_active":"2023-09-13T03:12:22.480661+0000","last_peered":"2023-09-13T03:12:22.480661+0000","last_clean":"2023-09-13T03:12:22.480661+0000","last_became_active":"2023-09-13T03:11:10.959839+0000","last_became_peered":"2023-09-13T03:11:10.959839+0000","last_unstale":"2023-09-13T03:12:22.480661+0000","last_undegraded":"2023-09-13T03:12:22.480661+0000","last_fullsized":"2023-09-13T03:12:22.480661+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:37:16.868093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.264967+0000","last_change":"2023-09-13T03:11:09.954684+0000","last_active":"2023-09-13T03:11:23.264967+0000","last_peered":"2023-09-13T03:11:23.264967+0000","last_clean":"2023-09-13T03:11:23.264967+0000","last_became_active":"2023-09-13T03:11:09.954392+0000","last_became_peered":"2023-09-13T03:11:09.954392+0000","last_unstale":"2023-09-13T03:11:23.264967+0000","last_undegraded":"2023-09-13T03:11:23.264967+0000","last_fullsized":"2023-09-13T03:11:23.264967+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:29:20.333063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.475916+0000","last_change":"2023-09-13T03:11:10.944364+0000","last_active":"2023-09-13T03:12:22.475916+0000","last_peered":"2023-09-13T03:12:22.475916+0000","last_clean":"2023-09-13T03:12:22.475916+0000","last_became_active":"2023-09-13T03:11:10.943971+0000","last_became_peered":"2023-09-13T03:11:10.943971+0000","last_unstale":"2023-09-13T03:12:22.475916+0000","last_undegraded":"2023-09-13T03:12:22.475916+0000","last_fullsized":"2023-09-13T03:12:22.475916+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:55:07.831397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.265196+0000","last_change":"2023-09-13T03:11:09.952751+0000","last_active":"2023-09-13T03:11:23.265196+0000","last_peered":"2023-09-13T03:11:23.265196+0000","last_clean":"2023-09-13T03:11:23.265196+0000","last_became_active":"2023-09-13T03:11:09.952447+0000","last_became_peered":"2023-09-13T03:11:09.952447+0000","last_unstale":"2023-09-13T03:11:23.265196+0000","last_undegraded":"2023-09-13T03:11:23.265196+0000","last_fullsized":"2023-09-13T03:11:23.265196+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:55:09.971742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.486746+0000","last_change":"2023-09-13T03:11:24.460373+0000","last_active":"2023-09-13T03:12:22.486746+0000","last_peered":"2023-09-13T03:12:22.486746+0000","last_clean":"2023-09-13T03:12:22.486746+0000","last_became_active":"2023-09-13T03:11:24.459994+0000","last_became_peered":"2023-09-13T03:11:24.459994+0000","last_unstale":"2023-09-13T03:12:22.486746+0000","last_undegraded":"2023-09-13T03:12:22.486746+0000","last_fullsized":"2023-09-13T03:12:22.486746+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:15:58.998213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'7","reported_seq":120,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:51.379310+0000","last_change":"2023-09-13T03:11:09.947207+0000","last_active":"2023-09-13T03:12:51.379310+0000","last_peered":"2023-09-13T03:12:51.379310+0000","last_clean":"2023-09-13T03:12:51.379310+0000","last_became_active":"2023-09-13T03:11:09.946787+0000","last_became_peered":"2023-09-13T03:11:09.946787+0000","last_unstale":"2023-09-13T03:12:51.379310+0000","last_undegraded":"2023-09-13T03:12:51.379310+0000","last_fullsized":"2023-09-13T03:12:51.379310+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:28:47.100723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1153,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.464131+0000","last_change":"2023-09-13T03:11:10.948427+0000","last_active":"2023-09-13T03:12:22.464131+0000","last_peered":"2023-09-13T03:12:22.464131+0000","last_clean":"2023-09-13T03:12:22.464131+0000","last_became_active":"2023-09-13T03:11:10.948095+0000","last_became_peered":"2023-09-13T03:11:10.948095+0000","last_unstale":"2023-09-13T03:12:22.464131+0000","last_undegraded":"2023-09-13T03:12:22.464131+0000","last_fullsized":"2023-09-13T03:12:22.464131+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:55:07.532772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.270733+0000","last_change":"2023-09-13T03:11:09.952335+0000","last_active":"2023-09-13T03:11:23.270733+0000","last_peered":"2023-09-13T03:11:23.270733+0000","last_clean":"2023-09-13T03:11:23.270733+0000","last_became_active":"2023-09-13T03:11:09.951974+0000","last_became_peered":"2023-09-13T03:11:09.951974+0000","last_unstale":"2023-09-13T03:11:23.270733+0000","last_undegraded":"2023-09-13T03:11:23.270733+0000","last_fullsized":"2023-09-13T03:11:23.270733+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:15:50.681751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.479139+0000","last_change":"2023-09-13T03:11:24.351298+0000","last_active":"2023-09-13T03:12:22.479139+0000","last_peered":"2023-09-13T03:12:22.479139+0000","last_clean":"2023-09-13T03:12:22.479139+0000","last_became_active":"2023-09-13T03:11:24.350758+0000","last_became_peered":"2023-09-13T03:11:24.350758+0000","last_unstale":"2023-09-13T03:12:22.479139+0000","last_undegraded":"2023-09-13T03:12:22.479139+0000","last_fullsized":"2023-09-13T03:12:22.479139+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:40:20.262641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.261290+0000","last_change":"2023-09-13T03:11:09.957644+0000","last_active":"2023-09-13T03:11:23.261290+0000","last_peered":"2023-09-13T03:11:23.261290+0000","last_clean":"2023-09-13T03:11:23.261290+0000","last_became_active":"2023-09-13T03:11:09.957289+0000","last_became_peered":"2023-09-13T03:11:09.957289+0000","last_unstale":"2023-09-13T03:11:23.261290+0000","last_undegraded":"2023-09-13T03:11:23.261290+0000","last_fullsized":"2023-09-13T03:11:23.261290+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:40:48.615322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.482645+0000","last_change":"2023-09-13T03:11:10.947561+0000","last_active":"2023-09-13T03:12:22.482645+0000","last_peered":"2023-09-13T03:12:22.482645+0000","last_clean":"2023-09-13T03:12:22.482645+0000","last_became_active":"2023-09-13T03:11:10.947238+0000","last_became_peered":"2023-09-13T03:11:10.947238+0000","last_unstale":"2023-09-13T03:12:22.482645+0000","last_undegraded":"2023-09-13T03:12:22.482645+0000","last_fullsized":"2023-09-13T03:12:22.482645+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:54:31.360657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.267100+0000","last_change":"2023-09-13T03:11:09.947806+0000","last_active":"2023-09-13T03:11:23.267100+0000","last_peered":"2023-09-13T03:11:23.267100+0000","last_clean":"2023-09-13T03:11:23.267100+0000","last_became_active":"2023-09-13T03:11:09.947428+0000","last_became_peered":"2023-09-13T03:11:09.947428+0000","last_unstale":"2023-09-13T03:11:23.267100+0000","last_undegraded":"2023-09-13T03:11:23.267100+0000","last_fullsized":"2023-09-13T03:11:23.267100+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:25:00.421873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.480287+0000","last_change":"2023-09-13T03:11:10.951637+0000","last_active":"2023-09-13T03:12:22.480287+0000","last_peered":"2023-09-13T03:12:22.480287+0000","last_clean":"2023-09-13T03:12:22.480287+0000","last_became_active":"2023-09-13T03:11:10.951249+0000","last_became_peered":"2023-09-13T03:11:10.951249+0000","last_unstale":"2023-09-13T03:12:22.480287+0000","last_undegraded":"2023-09-13T03:12:22.480287+0000","last_fullsized":"2023-09-13T03:12:22.480287+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:24:27.715167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'5","reported_seq":210,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:52.379887+0000","last_change":"2023-09-13T03:11:09.954343+0000","last_active":"2023-09-13T03:12:52.379887+0000","last_peered":"2023-09-13T03:12:52.379887+0000","last_clean":"2023-09-13T03:12:52.379887+0000","last_became_active":"2023-09-13T03:11:09.954063+0000","last_became_peered":"2023-09-13T03:11:09.954063+0000","last_unstale":"2023-09-13T03:12:52.379887+0000","last_undegraded":"2023-09-13T03:12:52.379887+0000","last_fullsized":"2023-09-13T03:12:52.379887+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:34:50.057539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":190,"num_read_kb":190,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.475149+0000","last_change":"2023-09-13T03:11:10.956994+0000","last_active":"2023-09-13T03:12:22.475149+0000","last_peered":"2023-09-13T03:12:22.475149+0000","last_clean":"2023-09-13T03:12:22.475149+0000","last_became_active":"2023-09-13T03:11:10.956336+0000","last_became_peered":"2023-09-13T03:11:10.956336+0000","last_unstale":"2023-09-13T03:12:22.475149+0000","last_undegraded":"2023-09-13T03:12:22.475149+0000","last_fullsized":"2023-09-13T03:12:22.475149+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:11:54.186143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.262358+0000","last_change":"2023-09-13T03:11:09.945775+0000","last_active":"2023-09-13T03:11:23.262358+0000","last_peered":"2023-09-13T03:11:23.262358+0000","last_clean":"2023-09-13T03:11:23.262358+0000","last_became_active":"2023-09-13T03:11:09.945406+0000","last_became_peered":"2023-09-13T03:11:09.945406+0000","last_unstale":"2023-09-13T03:11:23.262358+0000","last_undegraded":"2023-09-13T03:11:23.262358+0000","last_fullsized":"2023-09-13T03:11:23.262358+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:29:26.677351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.487156+0000","last_change":"2023-09-13T03:11:10.951941+0000","last_active":"2023-09-13T03:12:22.487156+0000","last_peered":"2023-09-13T03:12:22.487156+0000","last_clean":"2023-09-13T03:12:22.487156+0000","last_became_active":"2023-09-13T03:11:10.951331+0000","last_became_peered":"2023-09-13T03:11:10.951331+0000","last_unstale":"2023-09-13T03:12:22.487156+0000","last_undegraded":"2023-09-13T03:12:22.487156+0000","last_fullsized":"2023-09-13T03:12:22.487156+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:15:53.741657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.268129+0000","last_change":"2023-09-13T03:11:09.959503+0000","last_active":"2023-09-13T03:11:23.268129+0000","last_peered":"2023-09-13T03:11:23.268129+0000","last_clean":"2023-09-13T03:11:23.268129+0000","last_became_active":"2023-09-13T03:11:09.959166+0000","last_became_peered":"2023-09-13T03:11:09.959166+0000","last_unstale":"2023-09-13T03:11:23.268129+0000","last_undegraded":"2023-09-13T03:11:23.268129+0000","last_fullsized":"2023-09-13T03:11:23.268129+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:12:36.179520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.483877+0000","last_change":"2023-09-13T03:11:10.941575+0000","last_active":"2023-09-13T03:12:22.483877+0000","last_peered":"2023-09-13T03:12:22.483877+0000","last_clean":"2023-09-13T03:12:22.483877+0000","last_became_active":"2023-09-13T03:11:10.941285+0000","last_became_peered":"2023-09-13T03:11:10.941285+0000","last_unstale":"2023-09-13T03:12:22.483877+0000","last_undegraded":"2023-09-13T03:12:22.483877+0000","last_fullsized":"2023-09-13T03:12:22.483877+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:03:09.377471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'34","reported_seq":248,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:52.379878+0000","last_change":"2023-09-13T03:11:09.948041+0000","last_active":"2023-09-13T03:12:52.379878+0000","last_peered":"2023-09-13T03:12:52.379878+0000","last_clean":"2023-09-13T03:12:52.379878+0000","last_became_active":"2023-09-13T03:11:09.947785+0000","last_became_peered":"2023-09-13T03:11:09.947785+0000","last_unstale":"2023-09-13T03:12:52.379878+0000","last_undegraded":"2023-09-13T03:12:52.379878+0000","last_fullsized":"2023-09-13T03:12:52.379878+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:46:47.348307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58409,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":199,"num_read_kb":157,"num_write":36,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.460271+0000","last_change":"2023-09-13T03:11:10.950709+0000","last_active":"2023-09-13T03:12:22.460271+0000","last_peered":"2023-09-13T03:12:22.460271+0000","last_clean":"2023-09-13T03:12:22.460271+0000","last_became_active":"2023-09-13T03:11:10.950395+0000","last_became_peered":"2023-09-13T03:11:10.950395+0000","last_unstale":"2023-09-13T03:12:22.460271+0000","last_undegraded":"2023-09-13T03:12:22.460271+0000","last_fullsized":"2023-09-13T03:12:22.460271+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:50:08.628777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:11:24.281233+0000","last_change":"2023-09-13T03:11:24.281233+0000","last_active":"2023-09-13T03:11:24.281233+0000","last_peered":"2023-09-13T03:11:24.281233+0000","last_clean":"2023-09-13T03:11:24.281233+0000","last_became_active":"2023-09-13T03:11:24.269814+0000","last_became_peered":"2023-09-13T03:11:24.269814+0000","last_unstale":"2023-09-13T03:11:24.281233+0000","last_undegraded":"2023-09-13T03:11:24.281233+0000","last_fullsized":"2023-09-13T03:11:24.281233+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:09:00.743406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.482202+0000","last_change":"2023-09-13T03:11:10.955763+0000","last_active":"2023-09-13T03:12:22.482202+0000","last_peered":"2023-09-13T03:12:22.482202+0000","last_clean":"2023-09-13T03:12:22.482202+0000","last_became_active":"2023-09-13T03:11:10.955455+0000","last_became_peered":"2023-09-13T03:11:10.955455+0000","last_unstale":"2023-09-13T03:12:22.482202+0000","last_undegraded":"2023-09-13T03:12:22.482202+0000","last_fullsized":"2023-09-13T03:12:22.482202+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:43:37.127490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.267485+0000","last_change":"2023-09-13T03:11:09.947739+0000","last_active":"2023-09-13T03:11:23.267485+0000","last_peered":"2023-09-13T03:11:23.267485+0000","last_clean":"2023-09-13T03:11:23.267485+0000","last_became_active":"2023-09-13T03:11:09.947348+0000","last_became_peered":"2023-09-13T03:11:09.947348+0000","last_unstale":"2023-09-13T03:11:23.267485+0000","last_undegraded":"2023-09-13T03:11:23.267485+0000","last_fullsized":"2023-09-13T03:11:23.267485+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:47:58.408877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.473579+0000","last_change":"2023-09-13T03:11:10.958140+0000","last_active":"2023-09-13T03:12:22.473579+0000","last_peered":"2023-09-13T03:12:22.473579+0000","last_clean":"2023-09-13T03:12:22.473579+0000","last_became_active":"2023-09-13T03:11:10.957798+0000","last_became_peered":"2023-09-13T03:11:10.957798+0000","last_unstale":"2023-09-13T03:12:22.473579+0000","last_undegraded":"2023-09-13T03:12:22.473579+0000","last_fullsized":"2023-09-13T03:12:22.473579+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:44:40.208559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'7","reported_seq":119,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:50.376971+0000","last_change":"2023-09-13T03:11:09.945554+0000","last_active":"2023-09-13T03:12:50.376971+0000","last_peered":"2023-09-13T03:12:50.376971+0000","last_clean":"2023-09-13T03:12:50.376971+0000","last_became_active":"2023-09-13T03:11:09.945214+0000","last_became_peered":"2023-09-13T03:11:09.945214+0000","last_unstale":"2023-09-13T03:12:50.376971+0000","last_undegraded":"2023-09-13T03:12:50.376971+0000","last_fullsized":"2023-09-13T03:12:50.376971+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:46:12.338617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.470149+0000","last_change":"2023-09-13T03:11:10.952589+0000","last_active":"2023-09-13T03:12:22.470149+0000","last_peered":"2023-09-13T03:12:22.470149+0000","last_clean":"2023-09-13T03:12:22.470149+0000","last_became_active":"2023-09-13T03:11:10.952302+0000","last_became_peered":"2023-09-13T03:11:10.952302+0000","last_unstale":"2023-09-13T03:12:22.470149+0000","last_undegraded":"2023-09-13T03:12:22.470149+0000","last_fullsized":"2023-09-13T03:12:22.470149+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:13:46.972293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.264367+0000","last_change":"2023-09-13T03:11:09.959509+0000","last_active":"2023-09-13T03:11:23.264367+0000","last_peered":"2023-09-13T03:11:23.264367+0000","last_clean":"2023-09-13T03:11:23.264367+0000","last_became_active":"2023-09-13T03:11:09.959150+0000","last_became_peered":"2023-09-13T03:11:09.959150+0000","last_unstale":"2023-09-13T03:11:23.264367+0000","last_undegraded":"2023-09-13T03:11:23.264367+0000","last_fullsized":"2023-09-13T03:11:23.264367+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:33:30.648283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.476760+0000","last_change":"2023-09-13T03:11:10.956610+0000","last_active":"2023-09-13T03:12:22.476760+0000","last_peered":"2023-09-13T03:12:22.476760+0000","last_clean":"2023-09-13T03:12:22.476760+0000","last_became_active":"2023-09-13T03:11:10.955726+0000","last_became_peered":"2023-09-13T03:11:10.955726+0000","last_unstale":"2023-09-13T03:12:22.476760+0000","last_undegraded":"2023-09-13T03:12:22.476760+0000","last_fullsized":"2023-09-13T03:12:22.476760+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:55:45.667752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.264782+0000","last_change":"2023-09-13T03:11:09.949951+0000","last_active":"2023-09-13T03:11:23.264782+0000","last_peered":"2023-09-13T03:11:23.264782+0000","last_clean":"2023-09-13T03:11:23.264782+0000","last_became_active":"2023-09-13T03:11:09.949106+0000","last_became_peered":"2023-09-13T03:11:09.949106+0000","last_unstale":"2023-09-13T03:11:23.264782+0000","last_undegraded":"2023-09-13T03:11:23.264782+0000","last_fullsized":"2023-09-13T03:11:23.264782+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:30:30.092007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.471749+0000","last_change":"2023-09-13T03:11:10.942255+0000","last_active":"2023-09-13T03:12:22.471749+0000","last_peered":"2023-09-13T03:12:22.471749+0000","last_clean":"2023-09-13T03:12:22.471749+0000","last_became_active":"2023-09-13T03:11:10.941953+0000","last_became_peered":"2023-09-13T03:11:10.941953+0000","last_unstale":"2023-09-13T03:12:22.471749+0000","last_undegraded":"2023-09-13T03:12:22.471749+0000","last_fullsized":"2023-09-13T03:12:22.471749+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:38:55.260227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.262253+0000","last_change":"2023-09-13T03:11:09.956042+0000","last_active":"2023-09-13T03:11:23.262253+0000","last_peered":"2023-09-13T03:11:23.262253+0000","last_clean":"2023-09-13T03:11:23.262253+0000","last_became_active":"2023-09-13T03:11:09.955669+0000","last_became_peered":"2023-09-13T03:11:09.955669+0000","last_unstale":"2023-09-13T03:11:23.262253+0000","last_undegraded":"2023-09-13T03:11:23.262253+0000","last_fullsized":"2023-09-13T03:11:23.262253+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:16:25.227447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.482214+0000","last_change":"2023-09-13T03:11:10.952421+0000","last_active":"2023-09-13T03:12:22.482214+0000","last_peered":"2023-09-13T03:12:22.482214+0000","last_clean":"2023-09-13T03:12:22.482214+0000","last_became_active":"2023-09-13T03:11:10.951664+0000","last_became_peered":"2023-09-13T03:11:10.951664+0000","last_unstale":"2023-09-13T03:12:22.482214+0000","last_undegraded":"2023-09-13T03:12:22.482214+0000","last_fullsized":"2023-09-13T03:12:22.482214+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:46:52.664524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'5","reported_seq":113,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:48.373905+0000","last_change":"2023-09-13T03:11:09.952286+0000","last_active":"2023-09-13T03:12:48.373905+0000","last_peered":"2023-09-13T03:12:48.373905+0000","last_clean":"2023-09-13T03:12:48.373905+0000","last_became_active":"2023-09-13T03:11:09.951999+0000","last_became_peered":"2023-09-13T03:11:09.951999+0000","last_unstale":"2023-09-13T03:12:48.373905+0000","last_undegraded":"2023-09-13T03:12:48.373905+0000","last_fullsized":"2023-09-13T03:12:48.373905+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:54:46.910087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":93,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.485589+0000","last_change":"2023-09-13T03:11:10.950866+0000","last_active":"2023-09-13T03:12:22.485589+0000","last_peered":"2023-09-13T03:12:22.485589+0000","last_clean":"2023-09-13T03:12:22.485589+0000","last_became_active":"2023-09-13T03:11:10.950562+0000","last_became_peered":"2023-09-13T03:11:10.950562+0000","last_unstale":"2023-09-13T03:12:22.485589+0000","last_undegraded":"2023-09-13T03:12:22.485589+0000","last_fullsized":"2023-09-13T03:12:22.485589+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:55:07.333207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.476331+0000","last_change":"2023-09-13T03:11:10.950736+0000","last_active":"2023-09-13T03:12:22.476331+0000","last_peered":"2023-09-13T03:12:22.476331+0000","last_clean":"2023-09-13T03:12:22.476331+0000","last_became_active":"2023-09-13T03:11:10.950164+0000","last_became_peered":"2023-09-13T03:11:10.950164+0000","last_unstale":"2023-09-13T03:12:22.476331+0000","last_undegraded":"2023-09-13T03:12:22.476331+0000","last_fullsized":"2023-09-13T03:12:22.476331+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:22:44.453642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'6","reported_seq":118,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:52.379483+0000","last_change":"2023-09-13T03:11:09.954070+0000","last_active":"2023-09-13T03:12:52.379483+0000","last_peered":"2023-09-13T03:12:52.379483+0000","last_clean":"2023-09-13T03:12:52.379483+0000","last_became_active":"2023-09-13T03:11:09.953735+0000","last_became_peered":"2023-09-13T03:11:09.953735+0000","last_unstale":"2023-09-13T03:12:52.379483+0000","last_undegraded":"2023-09-13T03:12:52.379483+0000","last_fullsized":"2023-09-13T03:12:52.379483+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:36:59.891449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.486035+0000","last_change":"2023-09-13T03:11:10.948509+0000","last_active":"2023-09-13T03:12:22.486035+0000","last_peered":"2023-09-13T03:12:22.486035+0000","last_clean":"2023-09-13T03:12:22.486035+0000","last_became_active":"2023-09-13T03:11:10.948149+0000","last_became_peered":"2023-09-13T03:11:10.948149+0000","last_unstale":"2023-09-13T03:12:22.486035+0000","last_undegraded":"2023-09-13T03:12:22.486035+0000","last_fullsized":"2023-09-13T03:12:22.486035+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:45:54.197368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.265055+0000","last_change":"2023-09-13T03:11:09.954476+0000","last_active":"2023-09-13T03:11:23.265055+0000","last_peered":"2023-09-13T03:11:23.265055+0000","last_clean":"2023-09-13T03:11:23.265055+0000","last_became_active":"2023-09-13T03:11:09.954094+0000","last_became_peered":"2023-09-13T03:11:09.954094+0000","last_unstale":"2023-09-13T03:11:23.265055+0000","last_undegraded":"2023-09-13T03:11:23.265055+0000","last_fullsized":"2023-09-13T03:11:23.265055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:08:23.815751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":320,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.261991+0000","last_change":"2023-09-13T03:10:05.689061+0000","last_active":"2023-09-13T03:11:23.261991+0000","last_peered":"2023-09-13T03:11:23.261991+0000","last_clean":"2023-09-13T03:11:23.261991+0000","last_became_active":"2023-09-13T03:10:05.688557+0000","last_became_peered":"2023-09-13T03:10:05.688557+0000","last_unstale":"2023-09-13T03:11:23.261991+0000","last_undegraded":"2023-09-13T03:11:23.261991+0000","last_fullsized":"2023-09-13T03:11:23.261991+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:06:59.216900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:06:59.216900+0000","last_clean_scrub_stamp":"2023-09-13T03:06:59.216900+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:10:03.279019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.468705+0000","last_change":"2023-09-13T03:11:10.952523+0000","last_active":"2023-09-13T03:12:22.468705+0000","last_peered":"2023-09-13T03:12:22.468705+0000","last_clean":"2023-09-13T03:12:22.468705+0000","last_became_active":"2023-09-13T03:11:10.952193+0000","last_became_peered":"2023-09-13T03:11:10.952193+0000","last_unstale":"2023-09-13T03:12:22.468705+0000","last_undegraded":"2023-09-13T03:12:22.468705+0000","last_fullsized":"2023-09-13T03:12:22.468705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:31:01.811841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.262105+0000","last_change":"2023-09-13T03:11:09.950414+0000","last_active":"2023-09-13T03:11:23.262105+0000","last_peered":"2023-09-13T03:11:23.262105+0000","last_clean":"2023-09-13T03:11:23.262105+0000","last_became_active":"2023-09-13T03:11:09.950057+0000","last_became_peered":"2023-09-13T03:11:09.950057+0000","last_unstale":"2023-09-13T03:11:23.262105+0000","last_undegraded":"2023-09-13T03:11:23.262105+0000","last_fullsized":"2023-09-13T03:11:23.262105+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:57:01.317880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.264838+0000","last_change":"2023-09-13T03:11:09.955535+0000","last_active":"2023-09-13T03:11:23.264838+0000","last_peered":"2023-09-13T03:11:23.264838+0000","last_clean":"2023-09-13T03:11:23.264838+0000","last_became_active":"2023-09-13T03:11:09.955212+0000","last_became_peered":"2023-09-13T03:11:09.955212+0000","last_unstale":"2023-09-13T03:11:23.264838+0000","last_undegraded":"2023-09-13T03:11:23.264838+0000","last_fullsized":"2023-09-13T03:11:23.264838+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:15:04.365337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.476395+0000","last_change":"2023-09-13T03:11:10.954255+0000","last_active":"2023-09-13T03:12:22.476395+0000","last_peered":"2023-09-13T03:12:22.476395+0000","last_clean":"2023-09-13T03:12:22.476395+0000","last_became_active":"2023-09-13T03:11:10.953866+0000","last_became_peered":"2023-09-13T03:11:10.953866+0000","last_unstale":"2023-09-13T03:12:22.476395+0000","last_undegraded":"2023-09-13T03:12:22.476395+0000","last_fullsized":"2023-09-13T03:12:22.476395+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:25:12.281218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'11","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:11:29.021219+0000","last_change":"2023-09-13T03:11:24.285884+0000","last_active":"2023-09-13T03:11:29.021219+0000","last_peered":"2023-09-13T03:11:29.021219+0000","last_clean":"2023-09-13T03:11:29.021219+0000","last_became_active":"2023-09-13T03:11:24.269933+0000","last_became_peered":"2023-09-13T03:11:24.269933+0000","last_unstale":"2023-09-13T03:11:29.021219+0000","last_undegraded":"2023-09-13T03:11:29.021219+0000","last_fullsized":"2023-09-13T03:11:29.021219+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:30:13.957854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.480142+0000","last_change":"2023-09-13T03:11:10.949037+0000","last_active":"2023-09-13T03:12:22.480142+0000","last_peered":"2023-09-13T03:12:22.480142+0000","last_clean":"2023-09-13T03:12:22.480142+0000","last_became_active":"2023-09-13T03:11:10.948641+0000","last_became_peered":"2023-09-13T03:11:10.948641+0000","last_unstale":"2023-09-13T03:12:22.480142+0000","last_undegraded":"2023-09-13T03:12:22.480142+0000","last_fullsized":"2023-09-13T03:12:22.480142+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:00:57.563300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.265201+0000","last_change":"2023-09-13T03:11:09.946435+0000","last_active":"2023-09-13T03:11:23.265201+0000","last_peered":"2023-09-13T03:11:23.265201+0000","last_clean":"2023-09-13T03:11:23.265201+0000","last_became_active":"2023-09-13T03:11:09.946176+0000","last_became_peered":"2023-09-13T03:11:09.946176+0000","last_unstale":"2023-09-13T03:11:23.265201+0000","last_undegraded":"2023-09-13T03:11:23.265201+0000","last_fullsized":"2023-09-13T03:11:23.265201+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:34:16.055188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.487824+0000","last_change":"2023-09-13T03:11:10.957035+0000","last_active":"2023-09-13T03:12:22.487824+0000","last_peered":"2023-09-13T03:12:22.487824+0000","last_clean":"2023-09-13T03:12:22.487824+0000","last_became_active":"2023-09-13T03:11:10.954656+0000","last_became_peered":"2023-09-13T03:11:10.954656+0000","last_unstale":"2023-09-13T03:12:22.487824+0000","last_undegraded":"2023-09-13T03:12:22.487824+0000","last_fullsized":"2023-09-13T03:12:22.487824+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:54:51.142734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.269239+0000","last_change":"2023-09-13T03:11:09.959358+0000","last_active":"2023-09-13T03:11:23.269239+0000","last_peered":"2023-09-13T03:11:23.269239+0000","last_clean":"2023-09-13T03:11:23.269239+0000","last_became_active":"2023-09-13T03:11:09.959129+0000","last_became_peered":"2023-09-13T03:11:09.959129+0000","last_unstale":"2023-09-13T03:11:23.269239+0000","last_undegraded":"2023-09-13T03:11:23.269239+0000","last_fullsized":"2023-09-13T03:11:23.269239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:29:26.930910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.444904+0000","last_change":"2023-09-13T03:11:10.952588+0000","last_active":"2023-09-13T03:12:22.444904+0000","last_peered":"2023-09-13T03:12:22.444904+0000","last_clean":"2023-09-13T03:12:22.444904+0000","last_became_active":"2023-09-13T03:11:10.952304+0000","last_became_peered":"2023-09-13T03:11:10.952304+0000","last_unstale":"2023-09-13T03:12:22.444904+0000","last_undegraded":"2023-09-13T03:12:22.444904+0000","last_fullsized":"2023-09-13T03:12:22.444904+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:05:49.824053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.266834+0000","last_change":"2023-09-13T03:11:09.956947+0000","last_active":"2023-09-13T03:11:23.266834+0000","last_peered":"2023-09-13T03:11:23.266834+0000","last_clean":"2023-09-13T03:11:23.266834+0000","last_became_active":"2023-09-13T03:11:09.956639+0000","last_became_peered":"2023-09-13T03:11:09.956639+0000","last_unstale":"2023-09-13T03:11:23.266834+0000","last_undegraded":"2023-09-13T03:11:23.266834+0000","last_fullsized":"2023-09-13T03:11:23.266834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:21:37.759130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.480248+0000","last_change":"2023-09-13T03:11:10.947314+0000","last_active":"2023-09-13T03:12:22.480248+0000","last_peered":"2023-09-13T03:12:22.480248+0000","last_clean":"2023-09-13T03:12:22.480248+0000","last_became_active":"2023-09-13T03:11:10.946964+0000","last_became_peered":"2023-09-13T03:11:10.946964+0000","last_unstale":"2023-09-13T03:12:22.480248+0000","last_undegraded":"2023-09-13T03:12:22.480248+0000","last_fullsized":"2023-09-13T03:12:22.480248+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:55:08.542303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.268605+0000","last_change":"2023-09-13T03:11:09.952540+0000","last_active":"2023-09-13T03:11:23.268605+0000","last_peered":"2023-09-13T03:11:23.268605+0000","last_clean":"2023-09-13T03:11:23.268605+0000","last_became_active":"2023-09-13T03:11:09.952208+0000","last_became_peered":"2023-09-13T03:11:09.952208+0000","last_unstale":"2023-09-13T03:11:23.268605+0000","last_undegraded":"2023-09-13T03:11:23.268605+0000","last_fullsized":"2023-09-13T03:11:23.268605+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:11:41.200401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.482930+0000","last_change":"2023-09-13T03:11:10.951305+0000","last_active":"2023-09-13T03:12:22.482930+0000","last_peered":"2023-09-13T03:12:22.482930+0000","last_clean":"2023-09-13T03:12:22.482930+0000","last_became_active":"2023-09-13T03:11:10.950982+0000","last_became_peered":"2023-09-13T03:11:10.950982+0000","last_unstale":"2023-09-13T03:12:22.482930+0000","last_undegraded":"2023-09-13T03:12:22.482930+0000","last_fullsized":"2023-09-13T03:12:22.482930+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:55:04.497698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:11:24.355955+0000","last_change":"2023-09-13T03:11:24.355955+0000","last_active":"2023-09-13T03:11:24.355955+0000","last_peered":"2023-09-13T03:11:24.355955+0000","last_clean":"2023-09-13T03:11:24.355955+0000","last_became_active":"2023-09-13T03:11:24.351069+0000","last_became_peered":"2023-09-13T03:11:24.351069+0000","last_unstale":"2023-09-13T03:11:24.355955+0000","last_undegraded":"2023-09-13T03:11:24.355955+0000","last_fullsized":"2023-09-13T03:11:24.355955+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:57:55.311708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.479112+0000","last_change":"2023-09-13T03:11:10.959487+0000","last_active":"2023-09-13T03:12:22.479112+0000","last_peered":"2023-09-13T03:12:22.479112+0000","last_clean":"2023-09-13T03:12:22.479112+0000","last_became_active":"2023-09-13T03:11:10.959157+0000","last_became_peered":"2023-09-13T03:11:10.959157+0000","last_unstale":"2023-09-13T03:12:22.479112+0000","last_undegraded":"2023-09-13T03:12:22.479112+0000","last_fullsized":"2023-09-13T03:12:22.479112+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:37:50.852197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'5","reported_seq":118,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:52.380460+0000","last_change":"2023-09-13T03:11:09.951208+0000","last_active":"2023-09-13T03:12:52.380460+0000","last_peered":"2023-09-13T03:12:52.380460+0000","last_clean":"2023-09-13T03:12:52.380460+0000","last_became_active":"2023-09-13T03:11:09.950869+0000","last_became_peered":"2023-09-13T03:11:09.950869+0000","last_unstale":"2023-09-13T03:12:52.380460+0000","last_undegraded":"2023-09-13T03:12:52.380460+0000","last_fullsized":"2023-09-13T03:12:52.380460+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:22:53.697803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.482981+0000","last_change":"2023-09-13T03:11:10.959129+0000","last_active":"2023-09-13T03:12:22.482981+0000","last_peered":"2023-09-13T03:12:22.482981+0000","last_clean":"2023-09-13T03:12:22.482981+0000","last_became_active":"2023-09-13T03:11:10.958820+0000","last_became_peered":"2023-09-13T03:11:10.958820+0000","last_unstale":"2023-09-13T03:12:22.482981+0000","last_undegraded":"2023-09-13T03:12:22.482981+0000","last_fullsized":"2023-09-13T03:12:22.482981+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:20:30.607854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.267998+0000","last_change":"2023-09-13T03:11:09.946855+0000","last_active":"2023-09-13T03:11:23.267998+0000","last_peered":"2023-09-13T03:11:23.267998+0000","last_clean":"2023-09-13T03:11:23.267998+0000","last_became_active":"2023-09-13T03:11:09.946156+0000","last_became_peered":"2023-09-13T03:11:09.946156+0000","last_unstale":"2023-09-13T03:11:23.267998+0000","last_undegraded":"2023-09-13T03:11:23.267998+0000","last_fullsized":"2023-09-13T03:11:23.267998+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:42:19.387280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.479096+0000","last_change":"2023-09-13T03:11:10.951335+0000","last_active":"2023-09-13T03:12:22.479096+0000","last_peered":"2023-09-13T03:12:22.479096+0000","last_clean":"2023-09-13T03:12:22.479096+0000","last_became_active":"2023-09-13T03:11:10.951034+0000","last_became_peered":"2023-09-13T03:11:10.951034+0000","last_unstale":"2023-09-13T03:12:22.479096+0000","last_undegraded":"2023-09-13T03:12:22.479096+0000","last_fullsized":"2023-09-13T03:12:22.479096+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:57:24.373196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.263077+0000","last_change":"2023-09-13T03:11:09.950937+0000","last_active":"2023-09-13T03:11:23.263077+0000","last_peered":"2023-09-13T03:11:23.263077+0000","last_clean":"2023-09-13T03:11:23.263077+0000","last_became_active":"2023-09-13T03:11:09.950574+0000","last_became_peered":"2023-09-13T03:11:09.950574+0000","last_unstale":"2023-09-13T03:11:23.263077+0000","last_undegraded":"2023-09-13T03:11:23.263077+0000","last_fullsized":"2023-09-13T03:11:23.263077+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:10:23.215080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.477392+0000","last_change":"2023-09-13T03:11:10.950517+0000","last_active":"2023-09-13T03:12:22.477392+0000","last_peered":"2023-09-13T03:12:22.477392+0000","last_clean":"2023-09-13T03:12:22.477392+0000","last_became_active":"2023-09-13T03:11:10.950028+0000","last_became_peered":"2023-09-13T03:11:10.950028+0000","last_unstale":"2023-09-13T03:12:22.477392+0000","last_undegraded":"2023-09-13T03:12:22.477392+0000","last_fullsized":"2023-09-13T03:12:22.477392+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:06:40.008527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.473384+0000","last_change":"2023-09-13T03:11:10.942192+0000","last_active":"2023-09-13T03:12:22.473384+0000","last_peered":"2023-09-13T03:12:22.473384+0000","last_clean":"2023-09-13T03:12:22.473384+0000","last_became_active":"2023-09-13T03:11:10.941893+0000","last_became_peered":"2023-09-13T03:11:10.941893+0000","last_unstale":"2023-09-13T03:12:22.473384+0000","last_undegraded":"2023-09-13T03:12:22.473384+0000","last_fullsized":"2023-09-13T03:12:22.473384+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:26:22.064843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.470613+0000","last_change":"2023-09-13T03:11:10.947186+0000","last_active":"2023-09-13T03:12:22.470613+0000","last_peered":"2023-09-13T03:12:22.470613+0000","last_clean":"2023-09-13T03:12:22.470613+0000","last_became_active":"2023-09-13T03:11:10.946902+0000","last_became_peered":"2023-09-13T03:11:10.946902+0000","last_unstale":"2023-09-13T03:12:22.470613+0000","last_undegraded":"2023-09-13T03:12:22.470613+0000","last_fullsized":"2023-09-13T03:12:22.470613+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:26:01.325597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.476365+0000","last_change":"2023-09-13T03:11:10.946504+0000","last_active":"2023-09-13T03:12:22.476365+0000","last_peered":"2023-09-13T03:12:22.476365+0000","last_clean":"2023-09-13T03:12:22.476365+0000","last_became_active":"2023-09-13T03:11:10.946196+0000","last_became_peered":"2023-09-13T03:11:10.946196+0000","last_unstale":"2023-09-13T03:12:22.476365+0000","last_undegraded":"2023-09-13T03:12:22.476365+0000","last_fullsized":"2023-09-13T03:12:22.476365+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:37:43.578685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.487305+0000","last_change":"2023-09-13T03:11:10.959637+0000","last_active":"2023-09-13T03:12:22.487305+0000","last_peered":"2023-09-13T03:12:22.487305+0000","last_clean":"2023-09-13T03:12:22.487305+0000","last_became_active":"2023-09-13T03:11:10.959324+0000","last_became_peered":"2023-09-13T03:11:10.959324+0000","last_unstale":"2023-09-13T03:12:22.487305+0000","last_undegraded":"2023-09-13T03:12:22.487305+0000","last_fullsized":"2023-09-13T03:12:22.487305+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:48:19.245785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.476762+0000","last_change":"2023-09-13T03:11:24.576318+0000","last_active":"2023-09-13T03:12:22.476762+0000","last_peered":"2023-09-13T03:12:22.476762+0000","last_clean":"2023-09-13T03:12:22.476762+0000","last_became_active":"2023-09-13T03:11:24.575775+0000","last_became_peered":"2023-09-13T03:11:24.575775+0000","last_unstale":"2023-09-13T03:12:22.476762+0000","last_undegraded":"2023-09-13T03:12:22.476762+0000","last_fullsized":"2023-09-13T03:12:22.476762+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:16:00.534554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.472987+0000","last_change":"2023-09-13T03:11:10.945076+0000","last_active":"2023-09-13T03:12:22.472987+0000","last_peered":"2023-09-13T03:12:22.472987+0000","last_clean":"2023-09-13T03:12:22.472987+0000","last_became_active":"2023-09-13T03:11:10.944776+0000","last_became_peered":"2023-09-13T03:11:10.944776+0000","last_unstale":"2023-09-13T03:12:22.472987+0000","last_undegraded":"2023-09-13T03:12:22.472987+0000","last_fullsized":"2023-09-13T03:12:22.472987+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:22:03.624488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.475887+0000","last_change":"2023-09-13T03:11:10.942663+0000","last_active":"2023-09-13T03:12:22.475887+0000","last_peered":"2023-09-13T03:12:22.475887+0000","last_clean":"2023-09-13T03:12:22.475887+0000","last_became_active":"2023-09-13T03:11:10.942203+0000","last_became_peered":"2023-09-13T03:11:10.942203+0000","last_unstale":"2023-09-13T03:12:22.475887+0000","last_undegraded":"2023-09-13T03:12:22.475887+0000","last_fullsized":"2023-09-13T03:12:22.475887+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:46:48.091425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.469990+0000","last_change":"2023-09-13T03:11:10.940089+0000","last_active":"2023-09-13T03:12:22.469990+0000","last_peered":"2023-09-13T03:12:22.469990+0000","last_clean":"2023-09-13T03:12:22.469990+0000","last_became_active":"2023-09-13T03:11:10.939622+0000","last_became_peered":"2023-09-13T03:11:10.939622+0000","last_unstale":"2023-09-13T03:12:22.469990+0000","last_undegraded":"2023-09-13T03:12:22.469990+0000","last_fullsized":"2023-09-13T03:12:22.469990+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:43:55.024944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.477959+0000","last_change":"2023-09-13T03:11:10.956896+0000","last_active":"2023-09-13T03:12:22.477959+0000","last_peered":"2023-09-13T03:12:22.477959+0000","last_clean":"2023-09-13T03:12:22.477959+0000","last_became_active":"2023-09-13T03:11:10.956624+0000","last_became_peered":"2023-09-13T03:11:10.956624+0000","last_unstale":"2023-09-13T03:12:22.477959+0000","last_undegraded":"2023-09-13T03:12:22.477959+0000","last_fullsized":"2023-09-13T03:12:22.477959+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:29:41.305999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.479538+0000","last_change":"2023-09-13T03:11:10.939240+0000","last_active":"2023-09-13T03:12:22.479538+0000","last_peered":"2023-09-13T03:12:22.479538+0000","last_clean":"2023-09-13T03:12:22.479538+0000","last_became_active":"2023-09-13T03:11:10.938869+0000","last_became_peered":"2023-09-13T03:11:10.938869+0000","last_unstale":"2023-09-13T03:12:22.479538+0000","last_undegraded":"2023-09-13T03:12:22.479538+0000","last_fullsized":"2023-09-13T03:12:22.479538+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:06:44.931165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.483365+0000","last_change":"2023-09-13T03:11:10.956704+0000","last_active":"2023-09-13T03:12:22.483365+0000","last_peered":"2023-09-13T03:12:22.483365+0000","last_clean":"2023-09-13T03:12:22.483365+0000","last_became_active":"2023-09-13T03:11:10.956413+0000","last_became_peered":"2023-09-13T03:11:10.956413+0000","last_unstale":"2023-09-13T03:12:22.483365+0000","last_undegraded":"2023-09-13T03:12:22.483365+0000","last_fullsized":"2023-09-13T03:12:22.483365+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:50:04.626111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.475930+0000","last_change":"2023-09-13T03:11:10.952707+0000","last_active":"2023-09-13T03:12:22.475930+0000","last_peered":"2023-09-13T03:12:22.475930+0000","last_clean":"2023-09-13T03:12:22.475930+0000","last_became_active":"2023-09-13T03:11:10.952423+0000","last_became_peered":"2023-09-13T03:11:10.952423+0000","last_unstale":"2023-09-13T03:12:22.475930+0000","last_undegraded":"2023-09-13T03:12:22.475930+0000","last_fullsized":"2023-09-13T03:12:22.475930+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:35:09.869172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.474718+0000","last_change":"2023-09-13T03:11:10.941096+0000","last_active":"2023-09-13T03:12:22.474718+0000","last_peered":"2023-09-13T03:12:22.474718+0000","last_clean":"2023-09-13T03:12:22.474718+0000","last_became_active":"2023-09-13T03:11:10.940761+0000","last_became_peered":"2023-09-13T03:11:10.940761+0000","last_unstale":"2023-09-13T03:12:22.474718+0000","last_undegraded":"2023-09-13T03:12:22.474718+0000","last_fullsized":"2023-09-13T03:12:22.474718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:34:27.389650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65893,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":984,"num_read_kb":565,"num_write":245,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":68,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":573440,"data_stored":206951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449121,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":444288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:12:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.59999999999999998,"15min":0.60599999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.86099999999999999,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.58099999999999996,"15min":0.59199999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.79300000000000004,"5min":1.018,"15min":1.018},"last":0.44500000000000001}]},{"osd":1,"last update":"Wed Sep 13 03:12:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.60199999999999998,"15min":0.58399999999999996},"min":{"1min":0.55300000000000005,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.57499999999999996,"15min":0.56499999999999995},"min":{"1min":0.48299999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.68700000000000006}]},{"osd":2,"last update":"Wed Sep 13 03:12:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.45400000000000001,"15min":0.439},"min":{"1min":0.35799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.504,"15min":0.502},"min":{"1min":0.35799999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.77000000000000002,"5min":0.879,"15min":0.879},"last":0.60399999999999998}]},{"osd":3,"last update":"Wed Sep 13 03:12:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":6.415,"5min":6.415,"15min":6.415},"last":0.54000000000000004}]},{"osd":4,"last update":"Wed Sep 13 03:12:29 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.63300000000000001,"15min":0.61399999999999999},"min":{"1min":0.55100000000000005,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.61299999999999999,"15min":0.60499999999999998},"min":{"1min":0.56000000000000005,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.629}]},{"osd":5,"last update":"Wed Sep 13 03:12:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.55400000000000005,"15min":0.55000000000000004},"min":{"1min":0.36899999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79700000000000004,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.65900000000000003,"15min":0.64500000000000002},"min":{"1min":0.46300000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.70799999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":7,"last update":"Wed Sep 13 03:12:29 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.71999999999999997,"15min":0.70099999999999996},"min":{"1min":0.58099999999999996,"5min":0.497,"15min":0.497},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.71699999999999997,"15min":0.71499999999999997},"min":{"1min":0.497,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.74199999999999999}]},{"osd":8,"last update":"Wed Sep 13 03:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.75}]},{"osd":9,"last update":"Wed Sep 13 03:12:29 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.80500000000000005,"15min":0.77700000000000002},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":4.3049999999999997,"5min":4.3049999999999997,"15min":4.3049999999999997},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.82899999999999996,"15min":0.81799999999999995},"min":{"1min":0.68799999999999994,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.88800000000000001}]},{"osd":10,"last update":"Wed Sep 13 03:12:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.72399999999999998,"15min":0.71799999999999997},"min":{"1min":0.58399999999999996,"5min":0.499,"15min":0.499},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.71799999999999997,"15min":0.70599999999999996},"min":{"1min":0.64300000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.72899999999999998}]}]},{"osd":10,"up_from":71,"seq":304942678052,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1626112,"data_stored":1094195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.61399999999999999}]},{"osd":1,"last update":"Wed Sep 13 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.497,"15min":0.502},"min":{"1min":0.33800000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.71999999999999997,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.498,"15min":0.501},"min":{"1min":0.30099999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.82599999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.55700000000000005}]},{"osd":2,"last update":"Wed Sep 13 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.66500000000000004,"15min":0.67700000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81000000000000005,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.66700000000000004,"15min":0.67800000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.80400000000000005,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.67700000000000005}]},{"osd":3,"last update":"Wed Sep 13 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.66200000000000003,"15min":0.67200000000000004},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.81299999999999994,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.68500000000000005,"15min":0.69399999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.84599999999999997,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.60099999999999998}]},{"osd":4,"last update":"Wed Sep 13 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.61499999999999999,"15min":0.61899999999999999},"min":{"1min":0.376,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78300000000000003,"5min":1.03,"15min":1.03},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.39000000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80400000000000005,"5min":1.173,"15min":1.173},"last":0.52500000000000002}]},{"osd":5,"last update":"Wed Sep 13 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.78400000000000003,"15min":0.80400000000000005},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.92300000000000004,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.79600000000000004,"15min":0.81599999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.88200000000000001,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.73699999999999999}]},{"osd":6,"last update":"Wed Sep 13 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.82799999999999996,"15min":0.84799999999999998},"min":{"1min":0.53800000000000003,"5min":0.498,"15min":0.498},"max":{"1min":0.94899999999999995,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.63},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.82999999999999996,"15min":0.83999999999999997},"min":{"1min":0.56000000000000005,"5min":0.499,"15min":0.499},"max":{"1min":1.0429999999999999,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":0.70899999999999996}]},{"osd":7,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.47599999999999998}]},{"osd":8,"last update":"Wed Sep 13 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.84199999999999997,"15min":0.85999999999999999},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.88100000000000001,"5min":1.653,"15min":1.653},"last":0.752},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.879,"15min":0.90500000000000003},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.91600000000000004,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.72399999999999998}]},{"osd":9,"last update":"Wed Sep 13 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.88200000000000001,"15min":0.89700000000000002},"min":{"1min":0.61499999999999999,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.016,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.90300000000000002,"15min":0.92300000000000004},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.96899999999999997,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.65100000000000002}]},{"osd":11,"last update":"Wed Sep 13 03:12:24 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.81599999999999995,"15min":0.81200000000000006},"min":{"1min":0.70099999999999996,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":0.98899999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81999999999999995,"15min":0.82099999999999995},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.004,"5min":1.008,"15min":1.008},"last":0.81299999999999994}]}]},{"osd":9,"up_from":66,"seq":283467841577,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":444150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.57699999999999996,"15min":0.60599999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.56899999999999995,"15min":0.58899999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72499999999999998,"5min":0.875,"15min":0.875},"last":0.437}]},{"osd":1,"last update":"Wed Sep 13 03:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.64700000000000002,"15min":0.65900000000000003},"min":{"1min":0.44400000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.84999999999999998,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.70999999999999996,"15min":0.748},"min":{"1min":0.46700000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.82299999999999995,"5min":1.107,"15min":1.107},"last":0.56299999999999994}]},{"osd":2,"last update":"Wed Sep 13 03:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.71099999999999997,"15min":0.73099999999999998},"min":{"1min":0.45600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.123,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.752,"15min":0.78100000000000003},"min":{"1min":0.56200000000000006,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.62}]},{"osd":3,"last update":"Wed Sep 13 03:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.67000000000000004,"15min":0.70799999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.85099999999999998,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.71399999999999997,"15min":0.76400000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.98299999999999998,"5min":1.238,"15min":1.238},"last":0.49299999999999999}]},{"osd":4,"last update":"Wed Sep 13 03:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.79600000000000004,"15min":0.84199999999999997},"min":{"1min":0.442,"5min":0.438,"15min":0.438},"max":{"1min":0.93500000000000005,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.77000000000000002,"15min":0.80400000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.876,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.67000000000000004}]},{"osd":5,"last update":"Wed Sep 13 03:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.81799999999999995,"15min":0.85699999999999998},"min":{"1min":0.52000000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.95399999999999996,"5min":1.486,"15min":1.486},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.94799999999999995,"15min":1.034},"min":{"1min":0.59599999999999997,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.0800000000000001,"5min":4.2590000000000003,"15min":4.2590000000000003},"last":0.82499999999999996}]},{"osd":6,"last update":"Wed Sep 13 03:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.67600000000000005}]},{"osd":7,"last update":"Wed Sep 13 03:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.98699999999999999,"15min":1.077},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.252,"5min":4.367,"15min":4.367},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.752,"5min":1.0049999999999999,"15min":1.0960000000000001},"min":{"1min":0.60099999999999998,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":0.97199999999999998,"5min":4.3520000000000003,"15min":4.3520000000000003},"last":0.59599999999999997}]},{"osd":8,"last update":"Wed Sep 13 03:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.92600000000000005,"15min":0.98199999999999998},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.1830000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.82999999999999996,"5min":1.036,"15min":1.1220000000000001},"min":{"1min":0.57199999999999995,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.2,"5min":4.298,"15min":4.298},"last":0.85899999999999999}]},{"osd":10,"last update":"Wed Sep 13 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.95899999999999996,"15min":0.97899999999999998},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0900000000000001,"5min":1.321,"15min":1.321},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.93400000000000005,"15min":0.94299999999999995},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.2929999999999999,"5min":1.306,"15min":1.306},"last":0.80800000000000005}]},{"osd":11,"last update":"Wed Sep 13 03:12:23 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.83799999999999997}]}]},{"osd":8,"up_from":58,"seq":249108103213,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":444197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57899999999999996,"15min":0.58499999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.57799999999999996,"15min":0.57999999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.86099999999999999,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.57499999999999996}]},{"osd":1,"last update":"Wed Sep 13 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.61299999999999999,"15min":0.59899999999999998},"min":{"1min":0.48899999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.59899999999999998,"15min":0.59199999999999997},"min":{"1min":0.45400000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.81999999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.71399999999999997}]},{"osd":2,"last update":"Wed Sep 13 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.47599999999999998,"15min":0.46999999999999997},"min":{"1min":0.39600000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.48599999999999999,"15min":0.48199999999999998},"min":{"1min":0.34799999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.72799999999999998,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.52400000000000002}]},{"osd":3,"last update":"Wed Sep 13 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.65000000000000002,"15min":0.64300000000000002},"min":{"1min":0.46999999999999997,"5min":0.44,"15min":0.44},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.504},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.65000000000000002,"15min":0.65900000000000003},"min":{"1min":0.45000000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.87,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.54800000000000004}]},{"osd":4,"last update":"Wed Sep 13 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.63800000000000001,"15min":0.61699999999999999},"min":{"1min":0.48399999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.75,"5min":0.66300000000000003,"15min":0.65400000000000003},"min":{"1min":0.43099999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.83499999999999996}]},{"osd":5,"last update":"Wed Sep 13 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.57599999999999996,"15min":0.56599999999999995},"min":{"1min":0.40000000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.96999999999999997,"5min":0.995,"15min":0.995},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.61699999999999999,"15min":0.61799999999999999},"min":{"1min":0.38300000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.46899999999999997}]},{"osd":6,"last update":"Wed Sep 13 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.73599999999999999,"15min":0.747},"min":{"1min":0.51800000000000002,"5min":0.44,"15min":0.44},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.54600000000000004,"5min":0.501,"15min":0.501},"max":{"1min":1.103,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.70199999999999996}]},{"osd":7,"last update":"Wed Sep 13 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.71099999999999997,"15min":0.70599999999999996},"min":{"1min":0.52000000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91500000000000004,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.749,"15min":0.72599999999999998},"min":{"1min":0.64900000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.88}]},{"osd":9,"last update":"Wed Sep 13 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.78800000000000003,"15min":0.78100000000000003},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.0329999999999999,"5min":1.21,"15min":1.21},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.77100000000000002,"15min":0.76300000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0289999999999999,"5min":1.081,"15min":1.081},"last":0.68700000000000006}]},{"osd":10,"last update":"Wed Sep 13 03:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.73999999999999999,"15min":0.71399999999999997},"min":{"1min":0.55700000000000005,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.72799999999999998,"15min":0.69899999999999995},"min":{"1min":0.54400000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":2.004,"5min":2.004,"15min":2.004},"last":0.72499999999999998}]},{"osd":11,"last update":"Wed Sep 13 03:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.73999999999999999}]}]},{"osd":7,"up_from":51,"seq":219043332145,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28520,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716632,"statfs":{"total":95995035648,"available":95965831168,"internally_reserved":0,"allocated":1540096,"data_stored":1037450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:10:55 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.59699999999999998,"15min":0.58599999999999997},"min":{"1min":0.498,"5min":0.438,"15min":0.438},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.378},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60099999999999998,"15min":0.59099999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.41099999999999998}]},{"osd":1,"last update":"Wed Sep 13 03:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.42999999999999999,"15min":0.40799999999999997},"min":{"1min":0.376,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.625,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.44700000000000001,"15min":0.42099999999999999},"min":{"1min":0.39100000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.46999999999999997}]},{"osd":2,"last update":"Wed Sep 13 03:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.60799999999999998,"15min":0.59399999999999997},"min":{"1min":0.437,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86899999999999999,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.59899999999999998,"15min":0.58099999999999996},"min":{"1min":0.496,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.55700000000000005}]},{"osd":3,"last update":"Wed Sep 13 03:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.68799999999999994,"15min":0.63100000000000001},"min":{"1min":0.46899999999999997,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.81899999999999995,"5min":6.2720000000000002,"15min":6.2720000000000002},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.64700000000000002,"15min":0.625},"min":{"1min":0.48299999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88900000000000001,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.57699999999999996}]},{"osd":4,"last update":"Wed Sep 13 03:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.53900000000000003,"15min":0.51700000000000002},"min":{"1min":0.46800000000000003,"5min":0.314,"15min":0.314},"max":{"1min":0.82399999999999995,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55500000000000005,"15min":0.52700000000000002},"min":{"1min":0.39900000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.88300000000000001,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.72099999999999997}]},{"osd":5,"last update":"Wed Sep 13 03:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.66200000000000003,"15min":0.64300000000000002},"min":{"1min":0.52500000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.65300000000000002,"15min":0.625},"min":{"1min":0.56699999999999995,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.95599999999999996,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.58999999999999997}]},{"osd":6,"last update":"Wed Sep 13 03:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.72999999999999998,"15min":0.70199999999999996},"min":{"1min":0.57499999999999996,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.98999999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.625},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.75800000000000001,"15min":0.72799999999999998},"min":{"1min":0.622,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0129999999999999,"5min":1.212,"15min":1.212},"last":0.78000000000000003}]},{"osd":8,"last update":"Wed Sep 13 03:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.68700000000000006,"15min":0.65800000000000003},"min":{"1min":0.53100000000000003,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.72699999999999998,"15min":0.69899999999999995},"min":{"1min":0.51300000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.93999999999999995,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.75600000000000001}]},{"osd":9,"last update":"Wed Sep 13 03:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82199999999999995,"15min":0.81299999999999994},"min":{"1min":0.60299999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.81000000000000005,"15min":0.80800000000000005},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":0.95799999999999996,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.74399999999999999}]},{"osd":10,"last update":"Wed Sep 13 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82699999999999996,"15min":0.82799999999999996},"min":{"1min":0.60599999999999998,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.0209999999999999,"5min":1.083,"15min":1.083},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.64400000000000002,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.999,"5min":1.169,"15min":1.169},"last":0.70099999999999996}]},{"osd":11,"last update":"Wed Sep 13 03:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.76600000000000001}]}]},{"osd":6,"up_from":46,"seq":197568495671,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":444189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.50800000000000001,"15min":0.5},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.53700000000000003,"15min":0.53100000000000003},"min":{"1min":0.37,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.39500000000000002}]},{"osd":1,"last update":"Wed Sep 13 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.65400000000000003,"15min":0.63500000000000001},"min":{"1min":0.42299999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.63700000000000001,"15min":0.61899999999999999},"min":{"1min":0.47199999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.60899999999999999}]},{"osd":2,"last update":"Wed Sep 13 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.68999999999999995,"15min":0.67400000000000004},"min":{"1min":0.502,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.748},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72099999999999997,"15min":0.71699999999999997},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.879,"5min":1.879,"15min":1.879},"last":0.56299999999999994}]},{"osd":3,"last update":"Wed Sep 13 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.63300000000000001,"15min":0.64100000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.64700000000000002,"15min":0.65100000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.88500000000000001}]},{"osd":4,"last update":"Wed Sep 13 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.73499999999999999,"15min":0.73099999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.745,"15min":0.75900000000000001},"min":{"1min":0.51000000000000001,"5min":0.495,"15min":0.495},"max":{"1min":1.1659999999999999,"5min":1.54,"15min":1.54},"last":0.67400000000000004}]},{"osd":5,"last update":"Wed Sep 13 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.77700000000000002,"15min":0.79600000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.77800000000000002,"15min":0.78000000000000003},"min":{"1min":0.57599999999999996,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1339999999999999,"5min":1.21,"15min":1.21},"last":0.79500000000000004}]},{"osd":7,"last update":"Wed Sep 13 03:12:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77500000000000002,"15min":0.77000000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.1899999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.78600000000000003,"15min":0.75800000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":0.69399999999999995}]},{"osd":8,"last update":"Wed Sep 13 03:12:23 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.88,"15min":0.88800000000000001},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.82999999999999996,"15min":0.83999999999999997},"min":{"1min":0.60099999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.86199999999999999}]},{"osd":9,"last update":"Wed Sep 13 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.85799999999999998,"15min":0.88200000000000001},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.246,"5min":1.319,"15min":1.319},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.92100000000000004,"15min":0.93799999999999994},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.024,"5min":2.024,"15min":2.024},"last":0.92800000000000005}]},{"osd":10,"last update":"Wed Sep 13 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.88500000000000001,"15min":0.88400000000000001},"min":{"1min":0.60699999999999998,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.226,"5min":1.262,"15min":1.262},"last":0.874},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.95199999999999996,"15min":0.94599999999999995},"min":{"1min":0.73099999999999998,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.958,"5min":1.958,"15min":1.958},"last":0.89200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":503232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.56899999999999995,"15min":0.53900000000000003},"min":{"1min":0.39300000000000002,"5min":0.35499999999999998,"15min":0.34999999999999998},"max":{"1min":0.78600000000000003,"5min":2.9260000000000002,"15min":2.9260000000000002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.56399999999999995,"15min":0.53900000000000003},"min":{"1min":0.42399999999999999,"5min":0.34100000000000003,"15min":0.32300000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.51500000000000001}]},{"osd":2,"last update":"Wed Sep 13 03:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.58399999999999996,"15min":0.53900000000000003},"min":{"1min":0.47899999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.58699999999999997,"15min":0.54200000000000004},"min":{"1min":0.57999999999999996,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.65800000000000003}]},{"osd":3,"last update":"Wed Sep 13 03:12:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60499999999999998,"15min":0.58699999999999997},"min":{"1min":0.47599999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.65700000000000003,"15min":0.61399999999999999},"min":{"1min":0.53700000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.90300000000000002}]},{"osd":4,"last update":"Wed Sep 13 03:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.47299999999999998,"15min":0.40600000000000003},"min":{"1min":0.41399999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.83299999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.51100000000000001,"15min":0.44900000000000001},"min":{"1min":0.41299999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.77100000000000002,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.70999999999999996}]},{"osd":5,"last update":"Wed Sep 13 03:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.622,"15min":0.55100000000000005},"min":{"1min":0.625,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.626,"15min":0.56100000000000005},"min":{"1min":0.621,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.752}]},{"osd":6,"last update":"Wed Sep 13 03:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.68600000000000005,"15min":0.66000000000000003},"min":{"1min":0.53800000000000003,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.70399999999999996,"15min":0.67500000000000004},"min":{"1min":0.51900000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.94699999999999995,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":0.72999999999999998}]},{"osd":7,"last update":"Wed Sep 13 03:12:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.59899999999999998,"15min":0.57099999999999995},"min":{"1min":0.45100000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.61599999999999999,"15min":0.60399999999999998},"min":{"1min":0.45900000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84799999999999998,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.79500000000000004}]},{"osd":8,"last update":"Wed Sep 13 03:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.68999999999999995,"15min":0.67200000000000004},"min":{"1min":0.63800000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.96399999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.751,"15min":0.72899999999999998},"min":{"1min":0.629,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":1.0049999999999999}]},{"osd":9,"last update":"Wed Sep 13 03:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.79300000000000004,"15min":0.78000000000000003},"min":{"1min":0.70299999999999996,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.83899999999999997,"15min":0.82999999999999996},"min":{"1min":0.67200000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":1.0369999999999999}]},{"osd":10,"last update":"Wed Sep 13 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.79200000000000004,"15min":0.78400000000000003},"min":{"1min":0.51600000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":1.073},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.78300000000000003,"15min":0.77600000000000002},"min":{"1min":0.57399999999999995,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.88100000000000001}]},{"osd":11,"last update":"Wed Sep 13 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":1.105},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.94799999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":445206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 13 03:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.61399999999999999,"15min":0.56000000000000005},"min":{"1min":0.38,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.97299999999999998,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.64700000000000002,"15min":0.57799999999999996},"min":{"1min":0.39500000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":6.0039999999999996,"5min":6.0039999999999996,"15min":6.0039999999999996},"last":0.59199999999999997}]},{"osd":2,"last update":"Wed Sep 13 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.73199999999999998,"15min":0.63600000000000001},"min":{"1min":0.47899999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":6.9640000000000004,"5min":6.9640000000000004,"15min":6.9640000000000004},"last":0.63300000000000001},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.70899999999999996,"15min":0.621},"min":{"1min":0.504,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":6.9459999999999997,"5min":6.9459999999999997,"15min":6.9459999999999997},"last":0.50700000000000001}]},{"osd":3,"last update":"Wed Sep 13 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":8.1940000000000008,"5min":8.1940000000000008,"15min":8.1940000000000008},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":0.83299999999999996}]},{"osd":4,"last update":"Wed Sep 13 03:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.74199999999999999,"15min":0.71499999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":5.9370000000000003,"5min":5.9370000000000003,"15min":5.9370000000000003},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69299999999999995,"15min":0.68300000000000005},"min":{"1min":0.45300000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0409999999999999,"5min":1.665,"15min":1.665},"last":0.56799999999999995}]},{"osd":5,"last update":"Wed Sep 13 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71099999999999997,"15min":0.69599999999999995},"min":{"1min":0.47899999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93200000000000005,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.70799999999999996,"15min":0.68200000000000005},"min":{"1min":0.59099999999999997,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.02,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.753}]},{"osd":6,"last update":"Wed Sep 13 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.71499999999999997,"15min":0.68100000000000005},"min":{"1min":0.42299999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.83899999999999997,"5min":6.1130000000000004,"15min":6.1130000000000004},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.752,"15min":0.74199999999999999},"min":{"1min":0.51200000000000001,"5min":0.434,"15min":0.434},"max":{"1min":1.008,"5min":2.7229999999999999,"15min":2.7229999999999999},"last":0.70899999999999996}]},{"osd":7,"last update":"Wed Sep 13 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.74299999999999999,"15min":0.73899999999999999},"min":{"1min":0.56299999999999994,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.626},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.80600000000000005,"15min":0.79400000000000004},"min":{"1min":0.74399999999999999,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1379999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.77900000000000003}]},{"osd":8,"last update":"Wed Sep 13 03:12:24 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.81399999999999995,"15min":0.81799999999999995},"min":{"1min":0.67200000000000004,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.2330000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.3680000000000001,"5min":0.89300000000000002,"15min":0.81000000000000005},"min":{"1min":0.57299999999999995,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":6.9489999999999998,"5min":6.9489999999999998,"15min":6.9489999999999998},"last":0.88200000000000001}]},{"osd":9,"last update":"Wed Sep 13 03:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.86499999999999999,"15min":0.88900000000000001},"min":{"1min":0.59099999999999997,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.2549999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.86499999999999999,"15min":0.88700000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.85199999999999998}]},{"osd":10,"last update":"Wed Sep 13 03:12:09 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":0.92400000000000004,"15min":0.89000000000000001},"min":{"1min":0.68200000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":6.0830000000000002,"5min":6.0830000000000002,"15min":6.0830000000000002},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.165,"5min":1.0009999999999999,"15min":0.97299999999999998},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":6.9669999999999996,"5min":6.9669999999999996,"15min":6.9669999999999996},"last":0.70199999999999996}]},{"osd":11,"last update":"Wed Sep 13 03:12:31 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.88700000000000001,"15min":0.88500000000000001},"min":{"1min":0.70199999999999996,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.94299999999999995,"15min":0.95099999999999996},"min":{"1min":0.71499999999999997,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.2210000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.76800000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":503872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.55400000000000005,"15min":0.49099999999999999},"min":{"1min":0.41599999999999998,"5min":0.307,"15min":0.307},"max":{"1min":2.4390000000000001,"5min":2.4390000000000001,"15min":2.4390000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.57999999999999996,"15min":0.50700000000000001},"min":{"1min":0.46000000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":3.0979999999999999,"5min":3.0979999999999999,"15min":3.0979999999999999},"last":0.79700000000000004}]},{"osd":1,"last update":"Wed Sep 13 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.56299999999999994,"15min":0.52500000000000002},"min":{"1min":0.443,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.55200000000000005,"15min":0.53200000000000003},"min":{"1min":0.37,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.497}]},{"osd":3,"last update":"Wed Sep 13 03:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.65600000000000003,"15min":0.57999999999999996},"min":{"1min":0.52800000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90400000000000003,"5min":2.879,"15min":2.879},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.65600000000000003,"15min":0.58399999999999996},"min":{"1min":0.44600000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":2.782,"5min":2.8969999999999998,"15min":2.8969999999999998},"last":0.60599999999999998}]},{"osd":4,"last update":"Wed Sep 13 03:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.624},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.59099999999999997}]},{"osd":5,"last update":"Wed Sep 13 03:12:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.46999999999999997,"15min":0.433},"min":{"1min":0.36299999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.502,"15min":0.45600000000000002},"min":{"1min":0.36399999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.95699999999999996,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.69999999999999996}]},{"osd":6,"last update":"Wed Sep 13 03:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.67100000000000004,"15min":0.63200000000000001},"min":{"1min":0.51400000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.95099999999999996,"5min":1.048,"15min":1.048},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.67600000000000005,"15min":0.63},"min":{"1min":0.39700000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90400000000000003,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.74199999999999999}]},{"osd":7,"last update":"Wed Sep 13 03:10:58 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.63700000000000001,"15min":0.63600000000000001},"min":{"1min":0.47499999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.84699999999999998,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.66100000000000003,"15min":0.65900000000000003},"min":{"1min":0.51700000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.83399999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.497}]},{"osd":8,"last update":"Wed Sep 13 03:12:22 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.59899999999999998,"15min":0.58799999999999997},"min":{"1min":0.41499999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.69899999999999995,"15min":0.69299999999999995},"min":{"1min":0.59599999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.78200000000000003}]},{"osd":9,"last update":"Wed Sep 13 03:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.78200000000000003,"15min":0.75800000000000001},"min":{"1min":0.56000000000000005,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.98899999999999999,"5min":2.6459999999999999,"15min":2.6459999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.753,"15min":0.73999999999999999},"min":{"1min":0.59099999999999997,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.002,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.84399999999999997}]},{"osd":10,"last update":"Wed Sep 13 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.68500000000000005,"15min":0.66800000000000004},"min":{"1min":0.60299999999999998,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.73399999999999999,"15min":0.71499999999999997},"min":{"1min":0.58899999999999997,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.75800000000000001}]},{"osd":11,"last update":"Wed Sep 13 03:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.81000000000000005}]}]},{"osd":3,"up_from":25,"seq":107374182468,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":443704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:12:31 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.44600000000000001,"15min":0.40999999999999998},"min":{"1min":0.38500000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.437,"5min":0.46500000000000002,"15min":0.437},"min":{"1min":0.36199999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.57099999999999995,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.36299999999999999}]},{"osd":1,"last update":"Wed Sep 13 03:12:31 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.63700000000000001,"15min":0.59799999999999998},"min":{"1min":0.44,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80500000000000005,"5min":1.117,"15min":1.117},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.66100000000000003,"15min":0.60699999999999998},"min":{"1min":0.42199999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94899999999999995,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.57899999999999996}]},{"osd":2,"last update":"Wed Sep 13 03:12:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.69299999999999995,"15min":0.64300000000000002},"min":{"1min":0.55000000000000004,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.88800000000000001,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.69299999999999995,"15min":0.64900000000000002},"min":{"1min":0.44,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.85499999999999998,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.59999999999999998}]},{"osd":4,"last update":"Wed Sep 13 03:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.68300000000000005,"15min":0.63600000000000001},"min":{"1min":0.49399999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0900000000000001,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.67500000000000004,"15min":0.627},"min":{"1min":0.52000000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0389999999999999,"5min":1.177,"15min":1.177},"last":0.54500000000000004}]},{"osd":5,"last update":"Wed Sep 13 03:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.71399999999999997,"15min":0.68600000000000005},"min":{"1min":0.49299999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.96599999999999997,"5min":1.012,"15min":1.012},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72599999999999998,"15min":0.70399999999999996},"min":{"1min":0.53300000000000003,"5min":0.498,"15min":0.498},"max":{"1min":1.0069999999999999,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.76000000000000001}]},{"osd":6,"last update":"Wed Sep 13 03:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.64400000000000002,"15min":0.624},"min":{"1min":0.53700000000000003,"5min":0.374,"15min":0.374},"max":{"1min":0.72699999999999998,"5min":1.117,"15min":1.117},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.77800000000000002,"15min":0.752},"min":{"1min":0.628,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0149999999999999,"5min":1.599,"15min":1.599},"last":0.85599999999999998}]},{"osd":7,"last update":"Wed Sep 13 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.86099999999999999,"15min":0.85399999999999998},"min":{"1min":0.55300000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0529999999999999,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.83299999999999996,"15min":0.82799999999999996},"min":{"1min":0.56699999999999995,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0269999999999999,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.70099999999999996}]},{"osd":8,"last update":"Wed Sep 13 03:10:17 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.82199999999999995}]},{"osd":9,"last update":"Wed Sep 13 03:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.88700000000000001,"15min":0.91500000000000004},"min":{"1min":0.52700000000000002,"5min":0.502,"15min":0.502},"max":{"1min":0.91700000000000004,"5min":1.538,"15min":1.538},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.84099999999999997,"15min":0.88500000000000001},"min":{"1min":0.57499999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.90100000000000002,"5min":1.677,"15min":1.677},"last":0.79300000000000004}]},{"osd":10,"last update":"Wed Sep 13 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.93000000000000005,"15min":0.93899999999999995},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.034,"5min":1.708,"15min":1.708},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.95399999999999996,"15min":0.96399999999999997},"min":{"1min":0.69899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.1180000000000001,"5min":1.635,"15min":1.635},"last":0.86299999999999999}]},{"osd":11,"last update":"Wed Sep 13 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.747}]}]},{"osd":4,"up_from":33,"seq":141733920832,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":444259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.54500000000000004,"15min":0.51500000000000001},"min":{"1min":0.36699999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.55900000000000005,"15min":0.52900000000000003},"min":{"1min":0.35599999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.63800000000000001}]},{"osd":1,"last update":"Wed Sep 13 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.39700000000000002,"15min":0.36199999999999999},"min":{"1min":0.308,"5min":0.245,"15min":0.245},"max":{"1min":0.66700000000000004,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.379},{"interface":"front","average":{"1min":0.443,"5min":0.40799999999999997,"15min":0.372},"min":{"1min":0.32500000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.68200000000000005,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.436}]},{"osd":2,"last update":"Wed Sep 13 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58099999999999996,"15min":0.56299999999999994},"min":{"1min":0.44,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.81999999999999995,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.58999999999999997,"15min":0.54000000000000004},"min":{"1min":0.38600000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.59699999999999998}]},{"osd":3,"last update":"Wed Sep 13 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63100000000000001,"15min":0.56299999999999994},"min":{"1min":0.46400000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83899999999999997,"5min":4.2919999999999998,"15min":4.2919999999999998},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.59999999999999998,"15min":0.56999999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81999999999999995,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.57099999999999995}]},{"osd":5,"last update":"Wed Sep 13 03:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.61699999999999999,"15min":0.58299999999999996},"min":{"1min":0.44900000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.874,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.60699999999999998,"15min":0.56899999999999995},"min":{"1min":0.42399999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.46800000000000003}]},{"osd":6,"last update":"Wed Sep 13 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.745,"15min":0.71299999999999997},"min":{"1min":0.45100000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.86799999999999999,"5min":6.1859999999999999,"15min":6.1859999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.70599999999999996,"15min":0.71199999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.84399999999999997,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.66300000000000003}]},{"osd":7,"last update":"Wed Sep 13 03:11:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.628,"15min":0.60999999999999999},"min":{"1min":0.34200000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.93300000000000005,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63600000000000001,"15min":0.622},"min":{"1min":0.39700000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89700000000000002,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.67800000000000005}]},{"osd":8,"last update":"Wed Sep 13 03:12:20 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.72399999999999998,"15min":0.72299999999999998},"min":{"1min":0.51300000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0029999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.76100000000000001,"15min":0.755},"min":{"1min":0.629,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.98499999999999999,"5min":1.01,"15min":1.01},"last":0.74299999999999999}]},{"osd":9,"last update":"Wed Sep 13 03:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.90000000000000002,"15min":0.878},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0569999999999999,"5min":6.1529999999999996,"15min":6.1529999999999996},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.81299999999999994,"15min":0.82899999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.96099999999999997,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.71799999999999997}]},{"osd":10,"last update":"Wed Sep 13 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.81200000000000006,"15min":0.82099999999999995},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.016,"5min":1.113,"15min":1.113},"last":0.80000000000000004}]},{"osd":11,"last update":"Wed Sep 13 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.75800000000000001}]}]},{"osd":5,"up_from":38,"seq":163208757309,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28536,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716616,"statfs":{"total":95995035648,"available":95965814784,"internally_reserved":0,"allocated":1556480,"data_stored":1036870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:12:14 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":0.69999999999999996,"15min":0.64700000000000002},"min":{"1min":0.505,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":9.3100000000000005,"5min":9.3100000000000005,"15min":9.3100000000000005},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.67600000000000005,"15min":0.64200000000000002},"min":{"1min":0.42399999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":2.9169999999999998,"5min":6.0179999999999998,"15min":6.0179999999999998},"last":0.64600000000000002}]},{"osd":1,"last update":"Wed Sep 13 03:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.56699999999999995,"15min":0.52700000000000002},"min":{"1min":0.42999999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76200000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.56399999999999995,"15min":0.52000000000000002},"min":{"1min":0.49299999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.48999999999999999}]},{"osd":2,"last update":"Wed Sep 13 03:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.41999999999999998,"15min":0.39000000000000001},"min":{"1min":0.34799999999999998,"5min":0.193,"15min":0.193},"max":{"1min":0.76000000000000001,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.42099999999999999,"15min":0.39200000000000002},"min":{"1min":0.35799999999999998,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.64900000000000002,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.36699999999999999}]},{"osd":3,"last update":"Wed Sep 13 03:12:14 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":0.68400000000000005,"15min":0.60199999999999998},"min":{"1min":0.48499999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":9.4369999999999994,"5min":9.4369999999999994,"15min":9.4369999999999994},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61199999999999999,"15min":0.58999999999999997},"min":{"1min":0.46100000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.96299999999999997,"5min":1.48,"15min":1.48},"last":0.58899999999999997}]},{"osd":4,"last update":"Wed Sep 13 03:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.626,"15min":0.56599999999999995},"min":{"1min":0.48299999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.88400000000000001,"5min":3.3740000000000001,"15min":3.3740000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.59299999999999997,"15min":0.55700000000000005},"min":{"1min":0.44600000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79100000000000004,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.46500000000000002}]},{"osd":6,"last update":"Wed Sep 13 03:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.68600000000000005,"15min":0.64900000000000002},"min":{"1min":0.498,"5min":0.371,"15min":0.371},"max":{"1min":0.88300000000000001,"5min":4.8250000000000002,"15min":4.8250000000000002},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72099999999999997,"15min":0.67200000000000004},"min":{"1min":0.44600000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.90800000000000003,"5min":4.8049999999999997,"15min":4.8049999999999997},"last":0.71199999999999997}]},{"osd":7,"last update":"Wed Sep 13 03:12:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.63400000000000001,"15min":0.63300000000000001},"min":{"1min":0.47899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.86399999999999999,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.66600000000000004,"15min":0.65900000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85499999999999998,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.66600000000000004}]},{"osd":8,"last update":"Wed Sep 13 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57099999999999995,"15min":0.57599999999999996},"min":{"1min":0.36799999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83299999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61899999999999999,"15min":0.61799999999999999},"min":{"1min":0.48299999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.80800000000000005,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.68200000000000005}]},{"osd":9,"last update":"Wed Sep 13 03:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":1.0069999999999999,"15min":1.024},"min":{"1min":0.623,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.91500000000000004,"5min":8.3070000000000004,"15min":8.3070000000000004},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79300000000000004,"15min":0.76900000000000002},"min":{"1min":0.63400000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.97499999999999998,"5min":3.016,"15min":3.016},"last":0.79300000000000004}]},{"osd":10,"last update":"Wed Sep 13 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.68600000000000005,"15min":0.67600000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.876,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.74099999999999999,"15min":0.73299999999999998},"min":{"1min":0.57499999999999996,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.755}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":65164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-13T03:12:55.431 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-13T03:12:55.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.668+0000 7f8af93ff700 1 -- 172.21.15.64:0/262985102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8af4100590 msgr2=0x7f8af41009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:55.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.668+0000 7f8af93ff700 1 --2- 172.21.15.64:0/262985102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8af4100590 0x7f8af41009b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8ae40099c0 tx=0x7f8ae4009cd0 comp rx=0 tx=0).stop 2023-09-13T03:12:55.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.668+0000 7f8af93ff700 1 -- 172.21.15.64:0/262985102 shutdown_connections 2023-09-13T03:12:55.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.668+0000 7f8af93ff700 1 --2- 172.21.15.64:0/262985102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8af4100590 0x7f8af41009b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:55.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.668+0000 7f8af93ff700 1 --2- 172.21.15.64:0/262985102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8af4102120 0x7f8af41089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:55.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.668+0000 7f8af93ff700 1 --2- 172.21.15.64:0/262985102 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8af4101780 0x7f8af4101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:55.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.668+0000 7f8af93ff700 1 -- 172.21.15.64:0/262985102 >> 172.21.15.64:0/262985102 conn(0x7f8af40fbae0 msgr2=0x7f8af40fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:55.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.668+0000 7f8af93ff700 1 -- 172.21.15.64:0/262985102 shutdown_connections 2023-09-13T03:12:55.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.668+0000 7f8af93ff700 1 -- 172.21.15.64:0/262985102 wait complete. 2023-09-13T03:12:55.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.669+0000 7f8af93ff700 1 Processor -- start 2023-09-13T03:12:55.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.669+0000 7f8af93ff700 1 -- start start 2023-09-13T03:12:55.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.670+0000 7f8af93ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8af4100590 0x7f8af4196210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:55.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.670+0000 7f8af93ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8af4101780 0x7f8af4196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:55.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.670+0000 7f8af93ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8af4102120 0x7f8af419d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:55.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.670+0000 7f8af93ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af419dd90 con 0x7f8af4100590 2023-09-13T03:12:55.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.670+0000 7f8af93ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af419ded0 con 0x7f8af4102120 2023-09-13T03:12:55.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.670+0000 7f8af93ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af419e060 con 0x7f8af4101780 2023-09-13T03:12:55.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.670+0000 7f8af2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8af4100590 0x7f8af4196210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:55.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.670+0000 7f8af2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8af4100590 0x7f8af4196210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41108/0 (socket says 172.21.15.64:41108) 2023-09-13T03:12:55.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.670+0000 7f8af27fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8af4101780 0x7f8af4196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:55.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.670+0000 7f8af2ffd700 1 -- 172.21.15.64:0/4141971495 learned_addr learned my addr 172.21.15.64:0/4141971495 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:55.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.670+0000 7f8af37fe700 1 --2- 172.21.15.64:0/4141971495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8af4102120 0x7f8af419d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:55.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.670+0000 7f8af2ffd700 1 -- 172.21.15.64:0/4141971495 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8af4101780 msgr2=0x7f8af4196750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:55.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.670+0000 7f8af2ffd700 1 --2- 172.21.15.64:0/4141971495 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8af4101780 0x7f8af4196750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:55.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.670+0000 7f8af2ffd700 1 -- 172.21.15.64:0/4141971495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8af4102120 msgr2=0x7f8af419d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:55.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.670+0000 7f8af2ffd700 1 --2- 172.21.15.64:0/4141971495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8af4102120 0x7f8af419d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:55.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.670+0000 7f8af2ffd700 1 -- 172.21.15.64:0/4141971495 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae40095f0 con 0x7f8af4100590 2023-09-13T03:12:55.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.670+0000 7f8af37fe700 1 --2- 172.21.15.64:0/4141971495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8af4102120 0x7f8af419d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:12:55.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.671+0000 7f8af2ffd700 1 --2- 172.21.15.64:0/4141971495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8af4100590 0x7f8af4196210 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4009fd0 tx=0x7f8ae4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:55.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.671+0000 7f8adbfff700 1 -- 172.21.15.64:0/4141971495 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae401d070 con 0x7f8af4100590 2023-09-13T03:12:55.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.671+0000 7f8adbfff700 1 -- 172.21.15.64:0/4141971495 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ae4005030 con 0x7f8af4100590 2023-09-13T03:12:55.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.671+0000 7f8af93ff700 1 -- 172.21.15.64:0/4141971495 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8af419e2e0 con 0x7f8af4100590 2023-09-13T03:12:55.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.671+0000 7f8adbfff700 1 -- 172.21.15.64:0/4141971495 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4004b30 con 0x7f8af4100590 2023-09-13T03:12:55.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.671+0000 7f8af93ff700 1 -- 172.21.15.64:0/4141971495 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af419e760 con 0x7f8af4100590 2023-09-13T03:12:55.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.672+0000 7f8af93ff700 1 -- 172.21.15.64:0/4141971495 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8af410bdb0 con 0x7f8af4100590 2023-09-13T03:12:55.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.674+0000 7f8adbfff700 1 -- 172.21.15.64:0/4141971495 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f8ae40051a0 con 0x7f8af4100590 2023-09-13T03:12:55.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.675+0000 7f8adbfff700 1 --2- 172.21.15.64:0/4141971495 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f8adc0a0710 0x7f8adc0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:55.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.676+0000 7f8adbfff700 1 -- 172.21.15.64:0/4141971495 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f8ae40cccc0 con 0x7f8af4100590 2023-09-13T03:12:55.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.676+0000 7f8af27fc700 1 --2- 172.21.15.64:0/4141971495 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f8adc0a0710 0x7f8adc0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:55.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.676+0000 7f8af27fc700 1 --2- 172.21.15.64:0/4141971495 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f8adc0a0710 0x7f8adc0a2bd0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8009c60 tx=0x7f8ae8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:55.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.677+0000 7f8adbfff700 1 -- 172.21.15.64:0/4141971495 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f8ae4096850 con 0x7f8af4100590 2023-09-13T03:12:55.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.884+0000 7f8af93ff700 1 -- 172.21.15.64:0/4141971495 --> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8af419e980 con 0x7f8adc0a0710 2023-09-13T03:12:55.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.895+0000 7f8adbfff700 1 -- 172.21.15.64:0/4141971495 <== mgr.14150 v2:172.21.15.64:6800/873679308 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345051 (secure 0 0 0) 0x7f8af419e980 con 0x7f8adc0a0710 2023-09-13T03:12:55.900 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:12:55.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.900+0000 7f8af93ff700 1 -- 172.21.15.64:0/4141971495 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f8adc0a0710 msgr2=0x7f8adc0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:55.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.900+0000 7f8af93ff700 1 --2- 172.21.15.64:0/4141971495 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f8adc0a0710 0x7f8adc0a2bd0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8009c60 tx=0x7f8ae8009260 comp rx=0 tx=0).stop 2023-09-13T03:12:55.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.900+0000 7f8af93ff700 1 -- 172.21.15.64:0/4141971495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8af4100590 msgr2=0x7f8af4196210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:55.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.900+0000 7f8af93ff700 1 --2- 172.21.15.64:0/4141971495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8af4100590 0x7f8af4196210 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4009fd0 tx=0x7f8ae4009580 comp rx=0 tx=0).stop 2023-09-13T03:12:55.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.900+0000 7f8af93ff700 1 -- 172.21.15.64:0/4141971495 shutdown_connections 2023-09-13T03:12:55.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.900+0000 7f8af93ff700 1 --2- 172.21.15.64:0/4141971495 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f8adc0a0710 0x7f8adc0a2bd0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:55.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.900+0000 7f8af93ff700 1 --2- 172.21.15.64:0/4141971495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8af4102120 0x7f8af419d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:55.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.900+0000 7f8af93ff700 1 --2- 172.21.15.64:0/4141971495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8af4100590 0x7f8af4196210 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:55.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.900+0000 7f8af93ff700 1 --2- 172.21.15.64:0/4141971495 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8af4101780 0x7f8af4196750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:55.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.900+0000 7f8af93ff700 1 -- 172.21.15.64:0/4141971495 >> 172.21.15.64:0/4141971495 conn(0x7f8af40fbae0 msgr2=0x7f8af40fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:55.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.900+0000 7f8af93ff700 1 -- 172.21.15.64:0/4141971495 shutdown_connections 2023-09-13T03:12:55.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:55.900+0000 7f8af93ff700 1 -- 172.21.15.64:0/4141971495 wait complete. 2023-09-13T03:12:55.905 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-13T03:12:55.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:55 smithi170 ceph-mon[134328]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2023-09-13T03:12:55.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:55 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/1278840691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:12:55.917 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:55 smithi064 ceph-mon[97675]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2023-09-13T03:12:55.917 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:55 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/1278840691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:12:55.919 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2023-09-13T03:12:54.478794+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656507,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1120,"num_read_kb":684,"num_write":428,"num_write_kb":3077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180804,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":853,"ondisk_log_size":853,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337200,"kb_used_data":13008,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604624,"statfs":{"total":1151940427776,"available":1151595134976,"internally_reserved":0,"allocated":13320192,"data_stored":7285612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19130,"internal_metadata":331855174},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":316,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":136,"num_read_kb":71,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002563"},"pg_stats":[{"pgid":"3.3f","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.480907+0000","last_change":"2023-09-13T03:11:10.949449+0000","last_active":"2023-09-13T03:12:22.480907+0000","last_peered":"2023-09-13T03:12:22.480907+0000","last_clean":"2023-09-13T03:12:22.480907+0000","last_became_active":"2023-09-13T03:11:10.948912+0000","last_became_peered":"2023-09-13T03:11:10.948912+0000","last_unstale":"2023-09-13T03:12:22.480907+0000","last_undegraded":"2023-09-13T03:12:22.480907+0000","last_fullsized":"2023-09-13T03:12:22.480907+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:34:27.381352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.479553+0000","last_change":"2023-09-13T03:11:10.956817+0000","last_active":"2023-09-13T03:12:22.479553+0000","last_peered":"2023-09-13T03:12:22.479553+0000","last_clean":"2023-09-13T03:12:22.479553+0000","last_became_active":"2023-09-13T03:11:10.955359+0000","last_became_peered":"2023-09-13T03:11:10.955359+0000","last_unstale":"2023-09-13T03:12:22.479553+0000","last_undegraded":"2023-09-13T03:12:22.479553+0000","last_fullsized":"2023-09-13T03:12:22.479553+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:06:01.055910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.483472+0000","last_change":"2023-09-13T03:11:10.959915+0000","last_active":"2023-09-13T03:12:22.483472+0000","last_peered":"2023-09-13T03:12:22.483472+0000","last_clean":"2023-09-13T03:12:22.483472+0000","last_became_active":"2023-09-13T03:11:10.959611+0000","last_became_peered":"2023-09-13T03:11:10.959611+0000","last_unstale":"2023-09-13T03:12:22.483472+0000","last_undegraded":"2023-09-13T03:12:22.483472+0000","last_fullsized":"2023-09-13T03:12:22.483472+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:31:40.075569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.474409+0000","last_change":"2023-09-13T03:11:24.269022+0000","last_active":"2023-09-13T03:12:22.474409+0000","last_peered":"2023-09-13T03:12:22.474409+0000","last_clean":"2023-09-13T03:12:22.474409+0000","last_became_active":"2023-09-13T03:11:24.268019+0000","last_became_peered":"2023-09-13T03:11:24.268019+0000","last_unstale":"2023-09-13T03:12:22.474409+0000","last_undegraded":"2023-09-13T03:12:22.474409+0000","last_fullsized":"2023-09-13T03:12:22.474409+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:58:06.518437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.469930+0000","last_change":"2023-09-13T03:11:10.951554+0000","last_active":"2023-09-13T03:12:22.469930+0000","last_peered":"2023-09-13T03:12:22.469930+0000","last_clean":"2023-09-13T03:12:22.469930+0000","last_became_active":"2023-09-13T03:11:10.951252+0000","last_became_peered":"2023-09-13T03:11:10.951252+0000","last_unstale":"2023-09-13T03:12:22.469930+0000","last_undegraded":"2023-09-13T03:12:22.469930+0000","last_fullsized":"2023-09-13T03:12:22.469930+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:38:42.890256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.487106+0000","last_change":"2023-09-13T03:11:10.953087+0000","last_active":"2023-09-13T03:12:22.487106+0000","last_peered":"2023-09-13T03:12:22.487106+0000","last_clean":"2023-09-13T03:12:22.487106+0000","last_became_active":"2023-09-13T03:11:10.952773+0000","last_became_peered":"2023-09-13T03:11:10.952773+0000","last_unstale":"2023-09-13T03:12:22.487106+0000","last_undegraded":"2023-09-13T03:12:22.487106+0000","last_fullsized":"2023-09-13T03:12:22.487106+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:14:55.203347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.481903+0000","last_change":"2023-09-13T03:11:10.956058+0000","last_active":"2023-09-13T03:12:22.481903+0000","last_peered":"2023-09-13T03:12:22.481903+0000","last_clean":"2023-09-13T03:12:22.481903+0000","last_became_active":"2023-09-13T03:11:10.955783+0000","last_became_peered":"2023-09-13T03:11:10.955783+0000","last_unstale":"2023-09-13T03:12:22.481903+0000","last_undegraded":"2023-09-13T03:12:22.481903+0000","last_fullsized":"2023-09-13T03:12:22.481903+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:31:06.810338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.482123+0000","last_change":"2023-09-13T03:11:10.950102+0000","last_active":"2023-09-13T03:12:22.482123+0000","last_peered":"2023-09-13T03:12:22.482123+0000","last_clean":"2023-09-13T03:12:22.482123+0000","last_became_active":"2023-09-13T03:11:10.948877+0000","last_became_peered":"2023-09-13T03:11:10.948877+0000","last_unstale":"2023-09-13T03:12:22.482123+0000","last_undegraded":"2023-09-13T03:12:22.482123+0000","last_fullsized":"2023-09-13T03:12:22.482123+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:32:09.168793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.480791+0000","last_change":"2023-09-13T03:11:10.945244+0000","last_active":"2023-09-13T03:12:22.480791+0000","last_peered":"2023-09-13T03:12:22.480791+0000","last_clean":"2023-09-13T03:12:22.480791+0000","last_became_active":"2023-09-13T03:11:10.944951+0000","last_became_peered":"2023-09-13T03:11:10.944951+0000","last_unstale":"2023-09-13T03:12:22.480791+0000","last_undegraded":"2023-09-13T03:12:22.480791+0000","last_fullsized":"2023-09-13T03:12:22.480791+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:42:35.420767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.487770+0000","last_change":"2023-09-13T03:11:10.952312+0000","last_active":"2023-09-13T03:12:22.487770+0000","last_peered":"2023-09-13T03:12:22.487770+0000","last_clean":"2023-09-13T03:12:22.487770+0000","last_became_active":"2023-09-13T03:11:10.952077+0000","last_became_peered":"2023-09-13T03:11:10.952077+0000","last_unstale":"2023-09-13T03:12:22.487770+0000","last_undegraded":"2023-09-13T03:12:22.487770+0000","last_fullsized":"2023-09-13T03:12:22.487770+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:57:10.888184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.478509+0000","last_change":"2023-09-13T03:11:10.959259+0000","last_active":"2023-09-13T03:12:22.478509+0000","last_peered":"2023-09-13T03:12:22.478509+0000","last_clean":"2023-09-13T03:12:22.478509+0000","last_became_active":"2023-09-13T03:11:10.958922+0000","last_became_peered":"2023-09-13T03:11:10.958922+0000","last_unstale":"2023-09-13T03:12:22.478509+0000","last_undegraded":"2023-09-13T03:12:22.478509+0000","last_fullsized":"2023-09-13T03:12:22.478509+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:08:54.680718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.478268+0000","last_change":"2023-09-13T03:11:10.950946+0000","last_active":"2023-09-13T03:12:22.478268+0000","last_peered":"2023-09-13T03:12:22.478268+0000","last_clean":"2023-09-13T03:12:22.478268+0000","last_became_active":"2023-09-13T03:11:10.950633+0000","last_became_peered":"2023-09-13T03:11:10.950633+0000","last_unstale":"2023-09-13T03:12:22.478268+0000","last_undegraded":"2023-09-13T03:12:22.478268+0000","last_fullsized":"2023-09-13T03:12:22.478268+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:37:25.796555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.480994+0000","last_change":"2023-09-13T03:11:10.957712+0000","last_active":"2023-09-13T03:12:22.480994+0000","last_peered":"2023-09-13T03:12:22.480994+0000","last_clean":"2023-09-13T03:12:22.480994+0000","last_became_active":"2023-09-13T03:11:10.957379+0000","last_became_peered":"2023-09-13T03:11:10.957379+0000","last_unstale":"2023-09-13T03:12:22.480994+0000","last_undegraded":"2023-09-13T03:12:22.480994+0000","last_fullsized":"2023-09-13T03:12:22.480994+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:31:17.084185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.480310+0000","last_change":"2023-09-13T03:11:10.952951+0000","last_active":"2023-09-13T03:12:22.480310+0000","last_peered":"2023-09-13T03:12:22.480310+0000","last_clean":"2023-09-13T03:12:22.480310+0000","last_became_active":"2023-09-13T03:11:10.952629+0000","last_became_peered":"2023-09-13T03:11:10.952629+0000","last_unstale":"2023-09-13T03:12:22.480310+0000","last_undegraded":"2023-09-13T03:12:22.480310+0000","last_fullsized":"2023-09-13T03:12:22.480310+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:20:40.802584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.486815+0000","last_change":"2023-09-13T03:11:10.950799+0000","last_active":"2023-09-13T03:12:22.486815+0000","last_peered":"2023-09-13T03:12:22.486815+0000","last_clean":"2023-09-13T03:12:22.486815+0000","last_became_active":"2023-09-13T03:11:10.950500+0000","last_became_peered":"2023-09-13T03:11:10.950500+0000","last_unstale":"2023-09-13T03:12:22.486815+0000","last_undegraded":"2023-09-13T03:12:22.486815+0000","last_fullsized":"2023-09-13T03:12:22.486815+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:24:45.770696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.473581+0000","last_change":"2023-09-13T03:11:10.955141+0000","last_active":"2023-09-13T03:12:22.473581+0000","last_peered":"2023-09-13T03:12:22.473581+0000","last_clean":"2023-09-13T03:12:22.473581+0000","last_became_active":"2023-09-13T03:11:10.954840+0000","last_became_peered":"2023-09-13T03:11:10.954840+0000","last_unstale":"2023-09-13T03:12:22.473581+0000","last_undegraded":"2023-09-13T03:12:22.473581+0000","last_fullsized":"2023-09-13T03:12:22.473581+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:56:08.318784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.483342+0000","last_change":"2023-09-13T03:11:10.949199+0000","last_active":"2023-09-13T03:12:22.483342+0000","last_peered":"2023-09-13T03:12:22.483342+0000","last_clean":"2023-09-13T03:12:22.483342+0000","last_became_active":"2023-09-13T03:11:10.948506+0000","last_became_peered":"2023-09-13T03:11:10.948506+0000","last_unstale":"2023-09-13T03:12:22.483342+0000","last_undegraded":"2023-09-13T03:12:22.483342+0000","last_fullsized":"2023-09-13T03:12:22.483342+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:04:07.714033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.481246+0000","last_change":"2023-09-13T03:11:10.953207+0000","last_active":"2023-09-13T03:12:22.481246+0000","last_peered":"2023-09-13T03:12:22.481246+0000","last_clean":"2023-09-13T03:12:22.481246+0000","last_became_active":"2023-09-13T03:11:10.951566+0000","last_became_peered":"2023-09-13T03:11:10.951566+0000","last_unstale":"2023-09-13T03:12:22.481246+0000","last_undegraded":"2023-09-13T03:12:22.481246+0000","last_fullsized":"2023-09-13T03:12:22.481246+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:23:06.020880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.473777+0000","last_change":"2023-09-13T03:11:10.951695+0000","last_active":"2023-09-13T03:12:22.473777+0000","last_peered":"2023-09-13T03:12:22.473777+0000","last_clean":"2023-09-13T03:12:22.473777+0000","last_became_active":"2023-09-13T03:11:10.951382+0000","last_became_peered":"2023-09-13T03:11:10.951382+0000","last_unstale":"2023-09-13T03:12:22.473777+0000","last_undegraded":"2023-09-13T03:12:22.473777+0000","last_fullsized":"2023-09-13T03:12:22.473777+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:29:05.200980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'14","reported_seq":124,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:50.375945+0000","last_change":"2023-09-13T03:11:09.956193+0000","last_active":"2023-09-13T03:12:50.375945+0000","last_peered":"2023-09-13T03:12:50.375945+0000","last_clean":"2023-09-13T03:12:50.375945+0000","last_became_active":"2023-09-13T03:11:09.955778+0000","last_became_peered":"2023-09-13T03:11:09.955778+0000","last_unstale":"2023-09-13T03:12:50.375945+0000","last_undegraded":"2023-09-13T03:12:50.375945+0000","last_fullsized":"2023-09-13T03:12:50.375945+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:15:57.425730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":95,"num_read_kb":95,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.475680+0000","last_change":"2023-09-13T03:11:10.944437+0000","last_active":"2023-09-13T03:12:22.475680+0000","last_peered":"2023-09-13T03:12:22.475680+0000","last_clean":"2023-09-13T03:12:22.475680+0000","last_became_active":"2023-09-13T03:11:10.943961+0000","last_became_peered":"2023-09-13T03:11:10.943961+0000","last_unstale":"2023-09-13T03:12:22.475680+0000","last_undegraded":"2023-09-13T03:12:22.475680+0000","last_fullsized":"2023-09-13T03:12:22.475680+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:39:43.928334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.266435+0000","last_change":"2023-09-13T03:11:09.952017+0000","last_active":"2023-09-13T03:11:23.266435+0000","last_peered":"2023-09-13T03:11:23.266435+0000","last_clean":"2023-09-13T03:11:23.266435+0000","last_became_active":"2023-09-13T03:11:09.951700+0000","last_became_peered":"2023-09-13T03:11:09.951700+0000","last_unstale":"2023-09-13T03:11:23.266435+0000","last_undegraded":"2023-09-13T03:11:23.266435+0000","last_fullsized":"2023-09-13T03:11:23.266435+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:28:50.687271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.480661+0000","last_change":"2023-09-13T03:11:10.960198+0000","last_active":"2023-09-13T03:12:22.480661+0000","last_peered":"2023-09-13T03:12:22.480661+0000","last_clean":"2023-09-13T03:12:22.480661+0000","last_became_active":"2023-09-13T03:11:10.959839+0000","last_became_peered":"2023-09-13T03:11:10.959839+0000","last_unstale":"2023-09-13T03:12:22.480661+0000","last_undegraded":"2023-09-13T03:12:22.480661+0000","last_fullsized":"2023-09-13T03:12:22.480661+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:37:16.868093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.264967+0000","last_change":"2023-09-13T03:11:09.954684+0000","last_active":"2023-09-13T03:11:23.264967+0000","last_peered":"2023-09-13T03:11:23.264967+0000","last_clean":"2023-09-13T03:11:23.264967+0000","last_became_active":"2023-09-13T03:11:09.954392+0000","last_became_peered":"2023-09-13T03:11:09.954392+0000","last_unstale":"2023-09-13T03:11:23.264967+0000","last_undegraded":"2023-09-13T03:11:23.264967+0000","last_fullsized":"2023-09-13T03:11:23.264967+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:29:20.333063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.475916+0000","last_change":"2023-09-13T03:11:10.944364+0000","last_active":"2023-09-13T03:12:22.475916+0000","last_peered":"2023-09-13T03:12:22.475916+0000","last_clean":"2023-09-13T03:12:22.475916+0000","last_became_active":"2023-09-13T03:11:10.943971+0000","last_became_peered":"2023-09-13T03:11:10.943971+0000","last_unstale":"2023-09-13T03:12:22.475916+0000","last_undegraded":"2023-09-13T03:12:22.475916+0000","last_fullsized":"2023-09-13T03:12:22.475916+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:55:07.831397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.265196+0000","last_change":"2023-09-13T03:11:09.952751+0000","last_active":"2023-09-13T03:11:23.265196+0000","last_peered":"2023-09-13T03:11:23.265196+0000","last_clean":"2023-09-13T03:11:23.265196+0000","last_became_active":"2023-09-13T03:11:09.952447+0000","last_became_peered":"2023-09-13T03:11:09.952447+0000","last_unstale":"2023-09-13T03:11:23.265196+0000","last_undegraded":"2023-09-13T03:11:23.265196+0000","last_fullsized":"2023-09-13T03:11:23.265196+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:55:09.971742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.486746+0000","last_change":"2023-09-13T03:11:24.460373+0000","last_active":"2023-09-13T03:12:22.486746+0000","last_peered":"2023-09-13T03:12:22.486746+0000","last_clean":"2023-09-13T03:12:22.486746+0000","last_became_active":"2023-09-13T03:11:24.459994+0000","last_became_peered":"2023-09-13T03:11:24.459994+0000","last_unstale":"2023-09-13T03:12:22.486746+0000","last_undegraded":"2023-09-13T03:12:22.486746+0000","last_fullsized":"2023-09-13T03:12:22.486746+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:15:58.998213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'7","reported_seq":120,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:51.379310+0000","last_change":"2023-09-13T03:11:09.947207+0000","last_active":"2023-09-13T03:12:51.379310+0000","last_peered":"2023-09-13T03:12:51.379310+0000","last_clean":"2023-09-13T03:12:51.379310+0000","last_became_active":"2023-09-13T03:11:09.946787+0000","last_became_peered":"2023-09-13T03:11:09.946787+0000","last_unstale":"2023-09-13T03:12:51.379310+0000","last_undegraded":"2023-09-13T03:12:51.379310+0000","last_fullsized":"2023-09-13T03:12:51.379310+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:28:47.100723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1153,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.464131+0000","last_change":"2023-09-13T03:11:10.948427+0000","last_active":"2023-09-13T03:12:22.464131+0000","last_peered":"2023-09-13T03:12:22.464131+0000","last_clean":"2023-09-13T03:12:22.464131+0000","last_became_active":"2023-09-13T03:11:10.948095+0000","last_became_peered":"2023-09-13T03:11:10.948095+0000","last_unstale":"2023-09-13T03:12:22.464131+0000","last_undegraded":"2023-09-13T03:12:22.464131+0000","last_fullsized":"2023-09-13T03:12:22.464131+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:55:07.532772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.270733+0000","last_change":"2023-09-13T03:11:09.952335+0000","last_active":"2023-09-13T03:11:23.270733+0000","last_peered":"2023-09-13T03:11:23.270733+0000","last_clean":"2023-09-13T03:11:23.270733+0000","last_became_active":"2023-09-13T03:11:09.951974+0000","last_became_peered":"2023-09-13T03:11:09.951974+0000","last_unstale":"2023-09-13T03:11:23.270733+0000","last_undegraded":"2023-09-13T03:11:23.270733+0000","last_fullsized":"2023-09-13T03:11:23.270733+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:15:50.681751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.479139+0000","last_change":"2023-09-13T03:11:24.351298+0000","last_active":"2023-09-13T03:12:22.479139+0000","last_peered":"2023-09-13T03:12:22.479139+0000","last_clean":"2023-09-13T03:12:22.479139+0000","last_became_active":"2023-09-13T03:11:24.350758+0000","last_became_peered":"2023-09-13T03:11:24.350758+0000","last_unstale":"2023-09-13T03:12:22.479139+0000","last_undegraded":"2023-09-13T03:12:22.479139+0000","last_fullsized":"2023-09-13T03:12:22.479139+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:40:20.262641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.261290+0000","last_change":"2023-09-13T03:11:09.957644+0000","last_active":"2023-09-13T03:11:23.261290+0000","last_peered":"2023-09-13T03:11:23.261290+0000","last_clean":"2023-09-13T03:11:23.261290+0000","last_became_active":"2023-09-13T03:11:09.957289+0000","last_became_peered":"2023-09-13T03:11:09.957289+0000","last_unstale":"2023-09-13T03:11:23.261290+0000","last_undegraded":"2023-09-13T03:11:23.261290+0000","last_fullsized":"2023-09-13T03:11:23.261290+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:40:48.615322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.482645+0000","last_change":"2023-09-13T03:11:10.947561+0000","last_active":"2023-09-13T03:12:22.482645+0000","last_peered":"2023-09-13T03:12:22.482645+0000","last_clean":"2023-09-13T03:12:22.482645+0000","last_became_active":"2023-09-13T03:11:10.947238+0000","last_became_peered":"2023-09-13T03:11:10.947238+0000","last_unstale":"2023-09-13T03:12:22.482645+0000","last_undegraded":"2023-09-13T03:12:22.482645+0000","last_fullsized":"2023-09-13T03:12:22.482645+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:54:31.360657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.267100+0000","last_change":"2023-09-13T03:11:09.947806+0000","last_active":"2023-09-13T03:11:23.267100+0000","last_peered":"2023-09-13T03:11:23.267100+0000","last_clean":"2023-09-13T03:11:23.267100+0000","last_became_active":"2023-09-13T03:11:09.947428+0000","last_became_peered":"2023-09-13T03:11:09.947428+0000","last_unstale":"2023-09-13T03:11:23.267100+0000","last_undegraded":"2023-09-13T03:11:23.267100+0000","last_fullsized":"2023-09-13T03:11:23.267100+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:25:00.421873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.480287+0000","last_change":"2023-09-13T03:11:10.951637+0000","last_active":"2023-09-13T03:12:22.480287+0000","last_peered":"2023-09-13T03:12:22.480287+0000","last_clean":"2023-09-13T03:12:22.480287+0000","last_became_active":"2023-09-13T03:11:10.951249+0000","last_became_peered":"2023-09-13T03:11:10.951249+0000","last_unstale":"2023-09-13T03:12:22.480287+0000","last_undegraded":"2023-09-13T03:12:22.480287+0000","last_fullsized":"2023-09-13T03:12:22.480287+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:24:27.715167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'5","reported_seq":210,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:52.379887+0000","last_change":"2023-09-13T03:11:09.954343+0000","last_active":"2023-09-13T03:12:52.379887+0000","last_peered":"2023-09-13T03:12:52.379887+0000","last_clean":"2023-09-13T03:12:52.379887+0000","last_became_active":"2023-09-13T03:11:09.954063+0000","last_became_peered":"2023-09-13T03:11:09.954063+0000","last_unstale":"2023-09-13T03:12:52.379887+0000","last_undegraded":"2023-09-13T03:12:52.379887+0000","last_fullsized":"2023-09-13T03:12:52.379887+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:34:50.057539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":190,"num_read_kb":190,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.475149+0000","last_change":"2023-09-13T03:11:10.956994+0000","last_active":"2023-09-13T03:12:22.475149+0000","last_peered":"2023-09-13T03:12:22.475149+0000","last_clean":"2023-09-13T03:12:22.475149+0000","last_became_active":"2023-09-13T03:11:10.956336+0000","last_became_peered":"2023-09-13T03:11:10.956336+0000","last_unstale":"2023-09-13T03:12:22.475149+0000","last_undegraded":"2023-09-13T03:12:22.475149+0000","last_fullsized":"2023-09-13T03:12:22.475149+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:11:54.186143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.262358+0000","last_change":"2023-09-13T03:11:09.945775+0000","last_active":"2023-09-13T03:11:23.262358+0000","last_peered":"2023-09-13T03:11:23.262358+0000","last_clean":"2023-09-13T03:11:23.262358+0000","last_became_active":"2023-09-13T03:11:09.945406+0000","last_became_peered":"2023-09-13T03:11:09.945406+0000","last_unstale":"2023-09-13T03:11:23.262358+0000","last_undegraded":"2023-09-13T03:11:23.262358+0000","last_fullsized":"2023-09-13T03:11:23.262358+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:29:26.677351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.487156+0000","last_change":"2023-09-13T03:11:10.951941+0000","last_active":"2023-09-13T03:12:22.487156+0000","last_peered":"2023-09-13T03:12:22.487156+0000","last_clean":"2023-09-13T03:12:22.487156+0000","last_became_active":"2023-09-13T03:11:10.951331+0000","last_became_peered":"2023-09-13T03:11:10.951331+0000","last_unstale":"2023-09-13T03:12:22.487156+0000","last_undegraded":"2023-09-13T03:12:22.487156+0000","last_fullsized":"2023-09-13T03:12:22.487156+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:15:53.741657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.268129+0000","last_change":"2023-09-13T03:11:09.959503+0000","last_active":"2023-09-13T03:11:23.268129+0000","last_peered":"2023-09-13T03:11:23.268129+0000","last_clean":"2023-09-13T03:11:23.268129+0000","last_became_active":"2023-09-13T03:11:09.959166+0000","last_became_peered":"2023-09-13T03:11:09.959166+0000","last_unstale":"2023-09-13T03:11:23.268129+0000","last_undegraded":"2023-09-13T03:11:23.268129+0000","last_fullsized":"2023-09-13T03:11:23.268129+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:12:36.179520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.483877+0000","last_change":"2023-09-13T03:11:10.941575+0000","last_active":"2023-09-13T03:12:22.483877+0000","last_peered":"2023-09-13T03:12:22.483877+0000","last_clean":"2023-09-13T03:12:22.483877+0000","last_became_active":"2023-09-13T03:11:10.941285+0000","last_became_peered":"2023-09-13T03:11:10.941285+0000","last_unstale":"2023-09-13T03:12:22.483877+0000","last_undegraded":"2023-09-13T03:12:22.483877+0000","last_fullsized":"2023-09-13T03:12:22.483877+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:03:09.377471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'34","reported_seq":248,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:52.379878+0000","last_change":"2023-09-13T03:11:09.948041+0000","last_active":"2023-09-13T03:12:52.379878+0000","last_peered":"2023-09-13T03:12:52.379878+0000","last_clean":"2023-09-13T03:12:52.379878+0000","last_became_active":"2023-09-13T03:11:09.947785+0000","last_became_peered":"2023-09-13T03:11:09.947785+0000","last_unstale":"2023-09-13T03:12:52.379878+0000","last_undegraded":"2023-09-13T03:12:52.379878+0000","last_fullsized":"2023-09-13T03:12:52.379878+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:46:47.348307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58409,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":199,"num_read_kb":157,"num_write":36,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.460271+0000","last_change":"2023-09-13T03:11:10.950709+0000","last_active":"2023-09-13T03:12:22.460271+0000","last_peered":"2023-09-13T03:12:22.460271+0000","last_clean":"2023-09-13T03:12:22.460271+0000","last_became_active":"2023-09-13T03:11:10.950395+0000","last_became_peered":"2023-09-13T03:11:10.950395+0000","last_unstale":"2023-09-13T03:12:22.460271+0000","last_undegraded":"2023-09-13T03:12:22.460271+0000","last_fullsized":"2023-09-13T03:12:22.460271+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:50:08.628777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:11:24.281233+0000","last_change":"2023-09-13T03:11:24.281233+0000","last_active":"2023-09-13T03:11:24.281233+0000","last_peered":"2023-09-13T03:11:24.281233+0000","last_clean":"2023-09-13T03:11:24.281233+0000","last_became_active":"2023-09-13T03:11:24.269814+0000","last_became_peered":"2023-09-13T03:11:24.269814+0000","last_unstale":"2023-09-13T03:11:24.281233+0000","last_undegraded":"2023-09-13T03:11:24.281233+0000","last_fullsized":"2023-09-13T03:11:24.281233+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:09:00.743406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.482202+0000","last_change":"2023-09-13T03:11:10.955763+0000","last_active":"2023-09-13T03:12:22.482202+0000","last_peered":"2023-09-13T03:12:22.482202+0000","last_clean":"2023-09-13T03:12:22.482202+0000","last_became_active":"2023-09-13T03:11:10.955455+0000","last_became_peered":"2023-09-13T03:11:10.955455+0000","last_unstale":"2023-09-13T03:12:22.482202+0000","last_undegraded":"2023-09-13T03:12:22.482202+0000","last_fullsized":"2023-09-13T03:12:22.482202+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:43:37.127490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.267485+0000","last_change":"2023-09-13T03:11:09.947739+0000","last_active":"2023-09-13T03:11:23.267485+0000","last_peered":"2023-09-13T03:11:23.267485+0000","last_clean":"2023-09-13T03:11:23.267485+0000","last_became_active":"2023-09-13T03:11:09.947348+0000","last_became_peered":"2023-09-13T03:11:09.947348+0000","last_unstale":"2023-09-13T03:11:23.267485+0000","last_undegraded":"2023-09-13T03:11:23.267485+0000","last_fullsized":"2023-09-13T03:11:23.267485+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:47:58.408877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.473579+0000","last_change":"2023-09-13T03:11:10.958140+0000","last_active":"2023-09-13T03:12:22.473579+0000","last_peered":"2023-09-13T03:12:22.473579+0000","last_clean":"2023-09-13T03:12:22.473579+0000","last_became_active":"2023-09-13T03:11:10.957798+0000","last_became_peered":"2023-09-13T03:11:10.957798+0000","last_unstale":"2023-09-13T03:12:22.473579+0000","last_undegraded":"2023-09-13T03:12:22.473579+0000","last_fullsized":"2023-09-13T03:12:22.473579+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:44:40.208559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'7","reported_seq":119,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:50.376971+0000","last_change":"2023-09-13T03:11:09.945554+0000","last_active":"2023-09-13T03:12:50.376971+0000","last_peered":"2023-09-13T03:12:50.376971+0000","last_clean":"2023-09-13T03:12:50.376971+0000","last_became_active":"2023-09-13T03:11:09.945214+0000","last_became_peered":"2023-09-13T03:11:09.945214+0000","last_unstale":"2023-09-13T03:12:50.376971+0000","last_undegraded":"2023-09-13T03:12:50.376971+0000","last_fullsized":"2023-09-13T03:12:50.376971+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:46:12.338617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.470149+0000","last_change":"2023-09-13T03:11:10.952589+0000","last_active":"2023-09-13T03:12:22.470149+0000","last_peered":"2023-09-13T03:12:22.470149+0000","last_clean":"2023-09-13T03:12:22.470149+0000","last_became_active":"2023-09-13T03:11:10.952302+0000","last_became_peered":"2023-09-13T03:11:10.952302+0000","last_unstale":"2023-09-13T03:12:22.470149+0000","last_undegraded":"2023-09-13T03:12:22.470149+0000","last_fullsized":"2023-09-13T03:12:22.470149+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:13:46.972293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.264367+0000","last_change":"2023-09-13T03:11:09.959509+0000","last_active":"2023-09-13T03:11:23.264367+0000","last_peered":"2023-09-13T03:11:23.264367+0000","last_clean":"2023-09-13T03:11:23.264367+0000","last_became_active":"2023-09-13T03:11:09.959150+0000","last_became_peered":"2023-09-13T03:11:09.959150+0000","last_unstale":"2023-09-13T03:11:23.264367+0000","last_undegraded":"2023-09-13T03:11:23.264367+0000","last_fullsized":"2023-09-13T03:11:23.264367+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:33:30.648283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.476760+0000","last_change":"2023-09-13T03:11:10.956610+0000","last_active":"2023-09-13T03:12:22.476760+0000","last_peered":"2023-09-13T03:12:22.476760+0000","last_clean":"2023-09-13T03:12:22.476760+0000","last_became_active":"2023-09-13T03:11:10.955726+0000","last_became_peered":"2023-09-13T03:11:10.955726+0000","last_unstale":"2023-09-13T03:12:22.476760+0000","last_undegraded":"2023-09-13T03:12:22.476760+0000","last_fullsized":"2023-09-13T03:12:22.476760+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:55:45.667752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.264782+0000","last_change":"2023-09-13T03:11:09.949951+0000","last_active":"2023-09-13T03:11:23.264782+0000","last_peered":"2023-09-13T03:11:23.264782+0000","last_clean":"2023-09-13T03:11:23.264782+0000","last_became_active":"2023-09-13T03:11:09.949106+0000","last_became_peered":"2023-09-13T03:11:09.949106+0000","last_unstale":"2023-09-13T03:11:23.264782+0000","last_undegraded":"2023-09-13T03:11:23.264782+0000","last_fullsized":"2023-09-13T03:11:23.264782+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:30:30.092007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.471749+0000","last_change":"2023-09-13T03:11:10.942255+0000","last_active":"2023-09-13T03:12:22.471749+0000","last_peered":"2023-09-13T03:12:22.471749+0000","last_clean":"2023-09-13T03:12:22.471749+0000","last_became_active":"2023-09-13T03:11:10.941953+0000","last_became_peered":"2023-09-13T03:11:10.941953+0000","last_unstale":"2023-09-13T03:12:22.471749+0000","last_undegraded":"2023-09-13T03:12:22.471749+0000","last_fullsized":"2023-09-13T03:12:22.471749+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:38:55.260227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.262253+0000","last_change":"2023-09-13T03:11:09.956042+0000","last_active":"2023-09-13T03:11:23.262253+0000","last_peered":"2023-09-13T03:11:23.262253+0000","last_clean":"2023-09-13T03:11:23.262253+0000","last_became_active":"2023-09-13T03:11:09.955669+0000","last_became_peered":"2023-09-13T03:11:09.955669+0000","last_unstale":"2023-09-13T03:11:23.262253+0000","last_undegraded":"2023-09-13T03:11:23.262253+0000","last_fullsized":"2023-09-13T03:11:23.262253+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:16:25.227447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.482214+0000","last_change":"2023-09-13T03:11:10.952421+0000","last_active":"2023-09-13T03:12:22.482214+0000","last_peered":"2023-09-13T03:12:22.482214+0000","last_clean":"2023-09-13T03:12:22.482214+0000","last_became_active":"2023-09-13T03:11:10.951664+0000","last_became_peered":"2023-09-13T03:11:10.951664+0000","last_unstale":"2023-09-13T03:12:22.482214+0000","last_undegraded":"2023-09-13T03:12:22.482214+0000","last_fullsized":"2023-09-13T03:12:22.482214+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:46:52.664524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'5","reported_seq":113,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:48.373905+0000","last_change":"2023-09-13T03:11:09.952286+0000","last_active":"2023-09-13T03:12:48.373905+0000","last_peered":"2023-09-13T03:12:48.373905+0000","last_clean":"2023-09-13T03:12:48.373905+0000","last_became_active":"2023-09-13T03:11:09.951999+0000","last_became_peered":"2023-09-13T03:11:09.951999+0000","last_unstale":"2023-09-13T03:12:48.373905+0000","last_undegraded":"2023-09-13T03:12:48.373905+0000","last_fullsized":"2023-09-13T03:12:48.373905+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:54:46.910087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":93,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.485589+0000","last_change":"2023-09-13T03:11:10.950866+0000","last_active":"2023-09-13T03:12:22.485589+0000","last_peered":"2023-09-13T03:12:22.485589+0000","last_clean":"2023-09-13T03:12:22.485589+0000","last_became_active":"2023-09-13T03:11:10.950562+0000","last_became_peered":"2023-09-13T03:11:10.950562+0000","last_unstale":"2023-09-13T03:12:22.485589+0000","last_undegraded":"2023-09-13T03:12:22.485589+0000","last_fullsized":"2023-09-13T03:12:22.485589+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:55:07.333207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.476331+0000","last_change":"2023-09-13T03:11:10.950736+0000","last_active":"2023-09-13T03:12:22.476331+0000","last_peered":"2023-09-13T03:12:22.476331+0000","last_clean":"2023-09-13T03:12:22.476331+0000","last_became_active":"2023-09-13T03:11:10.950164+0000","last_became_peered":"2023-09-13T03:11:10.950164+0000","last_unstale":"2023-09-13T03:12:22.476331+0000","last_undegraded":"2023-09-13T03:12:22.476331+0000","last_fullsized":"2023-09-13T03:12:22.476331+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:22:44.453642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'6","reported_seq":118,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:52.379483+0000","last_change":"2023-09-13T03:11:09.954070+0000","last_active":"2023-09-13T03:12:52.379483+0000","last_peered":"2023-09-13T03:12:52.379483+0000","last_clean":"2023-09-13T03:12:52.379483+0000","last_became_active":"2023-09-13T03:11:09.953735+0000","last_became_peered":"2023-09-13T03:11:09.953735+0000","last_unstale":"2023-09-13T03:12:52.379483+0000","last_undegraded":"2023-09-13T03:12:52.379483+0000","last_fullsized":"2023-09-13T03:12:52.379483+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:36:59.891449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.486035+0000","last_change":"2023-09-13T03:11:10.948509+0000","last_active":"2023-09-13T03:12:22.486035+0000","last_peered":"2023-09-13T03:12:22.486035+0000","last_clean":"2023-09-13T03:12:22.486035+0000","last_became_active":"2023-09-13T03:11:10.948149+0000","last_became_peered":"2023-09-13T03:11:10.948149+0000","last_unstale":"2023-09-13T03:12:22.486035+0000","last_undegraded":"2023-09-13T03:12:22.486035+0000","last_fullsized":"2023-09-13T03:12:22.486035+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:45:54.197368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.265055+0000","last_change":"2023-09-13T03:11:09.954476+0000","last_active":"2023-09-13T03:11:23.265055+0000","last_peered":"2023-09-13T03:11:23.265055+0000","last_clean":"2023-09-13T03:11:23.265055+0000","last_became_active":"2023-09-13T03:11:09.954094+0000","last_became_peered":"2023-09-13T03:11:09.954094+0000","last_unstale":"2023-09-13T03:11:23.265055+0000","last_undegraded":"2023-09-13T03:11:23.265055+0000","last_fullsized":"2023-09-13T03:11:23.265055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:08:23.815751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":320,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.261991+0000","last_change":"2023-09-13T03:10:05.689061+0000","last_active":"2023-09-13T03:11:23.261991+0000","last_peered":"2023-09-13T03:11:23.261991+0000","last_clean":"2023-09-13T03:11:23.261991+0000","last_became_active":"2023-09-13T03:10:05.688557+0000","last_became_peered":"2023-09-13T03:10:05.688557+0000","last_unstale":"2023-09-13T03:11:23.261991+0000","last_undegraded":"2023-09-13T03:11:23.261991+0000","last_fullsized":"2023-09-13T03:11:23.261991+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:06:59.216900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:06:59.216900+0000","last_clean_scrub_stamp":"2023-09-13T03:06:59.216900+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:10:03.279019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.468705+0000","last_change":"2023-09-13T03:11:10.952523+0000","last_active":"2023-09-13T03:12:22.468705+0000","last_peered":"2023-09-13T03:12:22.468705+0000","last_clean":"2023-09-13T03:12:22.468705+0000","last_became_active":"2023-09-13T03:11:10.952193+0000","last_became_peered":"2023-09-13T03:11:10.952193+0000","last_unstale":"2023-09-13T03:12:22.468705+0000","last_undegraded":"2023-09-13T03:12:22.468705+0000","last_fullsized":"2023-09-13T03:12:22.468705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:31:01.811841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.262105+0000","last_change":"2023-09-13T03:11:09.950414+0000","last_active":"2023-09-13T03:11:23.262105+0000","last_peered":"2023-09-13T03:11:23.262105+0000","last_clean":"2023-09-13T03:11:23.262105+0000","last_became_active":"2023-09-13T03:11:09.950057+0000","last_became_peered":"2023-09-13T03:11:09.950057+0000","last_unstale":"2023-09-13T03:11:23.262105+0000","last_undegraded":"2023-09-13T03:11:23.262105+0000","last_fullsized":"2023-09-13T03:11:23.262105+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:57:01.317880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.264838+0000","last_change":"2023-09-13T03:11:09.955535+0000","last_active":"2023-09-13T03:11:23.264838+0000","last_peered":"2023-09-13T03:11:23.264838+0000","last_clean":"2023-09-13T03:11:23.264838+0000","last_became_active":"2023-09-13T03:11:09.955212+0000","last_became_peered":"2023-09-13T03:11:09.955212+0000","last_unstale":"2023-09-13T03:11:23.264838+0000","last_undegraded":"2023-09-13T03:11:23.264838+0000","last_fullsized":"2023-09-13T03:11:23.264838+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:15:04.365337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.476395+0000","last_change":"2023-09-13T03:11:10.954255+0000","last_active":"2023-09-13T03:12:22.476395+0000","last_peered":"2023-09-13T03:12:22.476395+0000","last_clean":"2023-09-13T03:12:22.476395+0000","last_became_active":"2023-09-13T03:11:10.953866+0000","last_became_peered":"2023-09-13T03:11:10.953866+0000","last_unstale":"2023-09-13T03:12:22.476395+0000","last_undegraded":"2023-09-13T03:12:22.476395+0000","last_fullsized":"2023-09-13T03:12:22.476395+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:25:12.281218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'11","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:11:29.021219+0000","last_change":"2023-09-13T03:11:24.285884+0000","last_active":"2023-09-13T03:11:29.021219+0000","last_peered":"2023-09-13T03:11:29.021219+0000","last_clean":"2023-09-13T03:11:29.021219+0000","last_became_active":"2023-09-13T03:11:24.269933+0000","last_became_peered":"2023-09-13T03:11:24.269933+0000","last_unstale":"2023-09-13T03:11:29.021219+0000","last_undegraded":"2023-09-13T03:11:29.021219+0000","last_fullsized":"2023-09-13T03:11:29.021219+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:30:13.957854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.480142+0000","last_change":"2023-09-13T03:11:10.949037+0000","last_active":"2023-09-13T03:12:22.480142+0000","last_peered":"2023-09-13T03:12:22.480142+0000","last_clean":"2023-09-13T03:12:22.480142+0000","last_became_active":"2023-09-13T03:11:10.948641+0000","last_became_peered":"2023-09-13T03:11:10.948641+0000","last_unstale":"2023-09-13T03:12:22.480142+0000","last_undegraded":"2023-09-13T03:12:22.480142+0000","last_fullsized":"2023-09-13T03:12:22.480142+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:00:57.563300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.265201+0000","last_change":"2023-09-13T03:11:09.946435+0000","last_active":"2023-09-13T03:11:23.265201+0000","last_peered":"2023-09-13T03:11:23.265201+0000","last_clean":"2023-09-13T03:11:23.265201+0000","last_became_active":"2023-09-13T03:11:09.946176+0000","last_became_peered":"2023-09-13T03:11:09.946176+0000","last_unstale":"2023-09-13T03:11:23.265201+0000","last_undegraded":"2023-09-13T03:11:23.265201+0000","last_fullsized":"2023-09-13T03:11:23.265201+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:34:16.055188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.487824+0000","last_change":"2023-09-13T03:11:10.957035+0000","last_active":"2023-09-13T03:12:22.487824+0000","last_peered":"2023-09-13T03:12:22.487824+0000","last_clean":"2023-09-13T03:12:22.487824+0000","last_became_active":"2023-09-13T03:11:10.954656+0000","last_became_peered":"2023-09-13T03:11:10.954656+0000","last_unstale":"2023-09-13T03:12:22.487824+0000","last_undegraded":"2023-09-13T03:12:22.487824+0000","last_fullsized":"2023-09-13T03:12:22.487824+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:54:51.142734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.269239+0000","last_change":"2023-09-13T03:11:09.959358+0000","last_active":"2023-09-13T03:11:23.269239+0000","last_peered":"2023-09-13T03:11:23.269239+0000","last_clean":"2023-09-13T03:11:23.269239+0000","last_became_active":"2023-09-13T03:11:09.959129+0000","last_became_peered":"2023-09-13T03:11:09.959129+0000","last_unstale":"2023-09-13T03:11:23.269239+0000","last_undegraded":"2023-09-13T03:11:23.269239+0000","last_fullsized":"2023-09-13T03:11:23.269239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:29:26.930910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.444904+0000","last_change":"2023-09-13T03:11:10.952588+0000","last_active":"2023-09-13T03:12:22.444904+0000","last_peered":"2023-09-13T03:12:22.444904+0000","last_clean":"2023-09-13T03:12:22.444904+0000","last_became_active":"2023-09-13T03:11:10.952304+0000","last_became_peered":"2023-09-13T03:11:10.952304+0000","last_unstale":"2023-09-13T03:12:22.444904+0000","last_undegraded":"2023-09-13T03:12:22.444904+0000","last_fullsized":"2023-09-13T03:12:22.444904+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:05:49.824053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.266834+0000","last_change":"2023-09-13T03:11:09.956947+0000","last_active":"2023-09-13T03:11:23.266834+0000","last_peered":"2023-09-13T03:11:23.266834+0000","last_clean":"2023-09-13T03:11:23.266834+0000","last_became_active":"2023-09-13T03:11:09.956639+0000","last_became_peered":"2023-09-13T03:11:09.956639+0000","last_unstale":"2023-09-13T03:11:23.266834+0000","last_undegraded":"2023-09-13T03:11:23.266834+0000","last_fullsized":"2023-09-13T03:11:23.266834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:21:37.759130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.480248+0000","last_change":"2023-09-13T03:11:10.947314+0000","last_active":"2023-09-13T03:12:22.480248+0000","last_peered":"2023-09-13T03:12:22.480248+0000","last_clean":"2023-09-13T03:12:22.480248+0000","last_became_active":"2023-09-13T03:11:10.946964+0000","last_became_peered":"2023-09-13T03:11:10.946964+0000","last_unstale":"2023-09-13T03:12:22.480248+0000","last_undegraded":"2023-09-13T03:12:22.480248+0000","last_fullsized":"2023-09-13T03:12:22.480248+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:55:08.542303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.268605+0000","last_change":"2023-09-13T03:11:09.952540+0000","last_active":"2023-09-13T03:11:23.268605+0000","last_peered":"2023-09-13T03:11:23.268605+0000","last_clean":"2023-09-13T03:11:23.268605+0000","last_became_active":"2023-09-13T03:11:09.952208+0000","last_became_peered":"2023-09-13T03:11:09.952208+0000","last_unstale":"2023-09-13T03:11:23.268605+0000","last_undegraded":"2023-09-13T03:11:23.268605+0000","last_fullsized":"2023-09-13T03:11:23.268605+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:11:41.200401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.482930+0000","last_change":"2023-09-13T03:11:10.951305+0000","last_active":"2023-09-13T03:12:22.482930+0000","last_peered":"2023-09-13T03:12:22.482930+0000","last_clean":"2023-09-13T03:12:22.482930+0000","last_became_active":"2023-09-13T03:11:10.950982+0000","last_became_peered":"2023-09-13T03:11:10.950982+0000","last_unstale":"2023-09-13T03:12:22.482930+0000","last_undegraded":"2023-09-13T03:12:22.482930+0000","last_fullsized":"2023-09-13T03:12:22.482930+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:55:04.497698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:11:24.355955+0000","last_change":"2023-09-13T03:11:24.355955+0000","last_active":"2023-09-13T03:11:24.355955+0000","last_peered":"2023-09-13T03:11:24.355955+0000","last_clean":"2023-09-13T03:11:24.355955+0000","last_became_active":"2023-09-13T03:11:24.351069+0000","last_became_peered":"2023-09-13T03:11:24.351069+0000","last_unstale":"2023-09-13T03:11:24.355955+0000","last_undegraded":"2023-09-13T03:11:24.355955+0000","last_fullsized":"2023-09-13T03:11:24.355955+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:57:55.311708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.479112+0000","last_change":"2023-09-13T03:11:10.959487+0000","last_active":"2023-09-13T03:12:22.479112+0000","last_peered":"2023-09-13T03:12:22.479112+0000","last_clean":"2023-09-13T03:12:22.479112+0000","last_became_active":"2023-09-13T03:11:10.959157+0000","last_became_peered":"2023-09-13T03:11:10.959157+0000","last_unstale":"2023-09-13T03:12:22.479112+0000","last_undegraded":"2023-09-13T03:12:22.479112+0000","last_fullsized":"2023-09-13T03:12:22.479112+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:37:50.852197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'5","reported_seq":118,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:52.380460+0000","last_change":"2023-09-13T03:11:09.951208+0000","last_active":"2023-09-13T03:12:52.380460+0000","last_peered":"2023-09-13T03:12:52.380460+0000","last_clean":"2023-09-13T03:12:52.380460+0000","last_became_active":"2023-09-13T03:11:09.950869+0000","last_became_peered":"2023-09-13T03:11:09.950869+0000","last_unstale":"2023-09-13T03:12:52.380460+0000","last_undegraded":"2023-09-13T03:12:52.380460+0000","last_fullsized":"2023-09-13T03:12:52.380460+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:22:53.697803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.482981+0000","last_change":"2023-09-13T03:11:10.959129+0000","last_active":"2023-09-13T03:12:22.482981+0000","last_peered":"2023-09-13T03:12:22.482981+0000","last_clean":"2023-09-13T03:12:22.482981+0000","last_became_active":"2023-09-13T03:11:10.958820+0000","last_became_peered":"2023-09-13T03:11:10.958820+0000","last_unstale":"2023-09-13T03:12:22.482981+0000","last_undegraded":"2023-09-13T03:12:22.482981+0000","last_fullsized":"2023-09-13T03:12:22.482981+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:20:30.607854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.267998+0000","last_change":"2023-09-13T03:11:09.946855+0000","last_active":"2023-09-13T03:11:23.267998+0000","last_peered":"2023-09-13T03:11:23.267998+0000","last_clean":"2023-09-13T03:11:23.267998+0000","last_became_active":"2023-09-13T03:11:09.946156+0000","last_became_peered":"2023-09-13T03:11:09.946156+0000","last_unstale":"2023-09-13T03:11:23.267998+0000","last_undegraded":"2023-09-13T03:11:23.267998+0000","last_fullsized":"2023-09-13T03:11:23.267998+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:42:19.387280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.479096+0000","last_change":"2023-09-13T03:11:10.951335+0000","last_active":"2023-09-13T03:12:22.479096+0000","last_peered":"2023-09-13T03:12:22.479096+0000","last_clean":"2023-09-13T03:12:22.479096+0000","last_became_active":"2023-09-13T03:11:10.951034+0000","last_became_peered":"2023-09-13T03:11:10.951034+0000","last_unstale":"2023-09-13T03:12:22.479096+0000","last_undegraded":"2023-09-13T03:12:22.479096+0000","last_fullsized":"2023-09-13T03:12:22.479096+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:57:24.373196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.263077+0000","last_change":"2023-09-13T03:11:09.950937+0000","last_active":"2023-09-13T03:11:23.263077+0000","last_peered":"2023-09-13T03:11:23.263077+0000","last_clean":"2023-09-13T03:11:23.263077+0000","last_became_active":"2023-09-13T03:11:09.950574+0000","last_became_peered":"2023-09-13T03:11:09.950574+0000","last_unstale":"2023-09-13T03:11:23.263077+0000","last_undegraded":"2023-09-13T03:11:23.263077+0000","last_fullsized":"2023-09-13T03:11:23.263077+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:10:23.215080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.477392+0000","last_change":"2023-09-13T03:11:10.950517+0000","last_active":"2023-09-13T03:12:22.477392+0000","last_peered":"2023-09-13T03:12:22.477392+0000","last_clean":"2023-09-13T03:12:22.477392+0000","last_became_active":"2023-09-13T03:11:10.950028+0000","last_became_peered":"2023-09-13T03:11:10.950028+0000","last_unstale":"2023-09-13T03:12:22.477392+0000","last_undegraded":"2023-09-13T03:12:22.477392+0000","last_fullsized":"2023-09-13T03:12:22.477392+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:06:40.008527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.473384+0000","last_change":"2023-09-13T03:11:10.942192+0000","last_active":"2023-09-13T03:12:22.473384+0000","last_peered":"2023-09-13T03:12:22.473384+0000","last_clean":"2023-09-13T03:12:22.473384+0000","last_became_active":"2023-09-13T03:11:10.941893+0000","last_became_peered":"2023-09-13T03:11:10.941893+0000","last_unstale":"2023-09-13T03:12:22.473384+0000","last_undegraded":"2023-09-13T03:12:22.473384+0000","last_fullsized":"2023-09-13T03:12:22.473384+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:26:22.064843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.470613+0000","last_change":"2023-09-13T03:11:10.947186+0000","last_active":"2023-09-13T03:12:22.470613+0000","last_peered":"2023-09-13T03:12:22.470613+0000","last_clean":"2023-09-13T03:12:22.470613+0000","last_became_active":"2023-09-13T03:11:10.946902+0000","last_became_peered":"2023-09-13T03:11:10.946902+0000","last_unstale":"2023-09-13T03:12:22.470613+0000","last_undegraded":"2023-09-13T03:12:22.470613+0000","last_fullsized":"2023-09-13T03:12:22.470613+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:26:01.325597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.476365+0000","last_change":"2023-09-13T03:11:10.946504+0000","last_active":"2023-09-13T03:12:22.476365+0000","last_peered":"2023-09-13T03:12:22.476365+0000","last_clean":"2023-09-13T03:12:22.476365+0000","last_became_active":"2023-09-13T03:11:10.946196+0000","last_became_peered":"2023-09-13T03:11:10.946196+0000","last_unstale":"2023-09-13T03:12:22.476365+0000","last_undegraded":"2023-09-13T03:12:22.476365+0000","last_fullsized":"2023-09-13T03:12:22.476365+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:37:43.578685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.487305+0000","last_change":"2023-09-13T03:11:10.959637+0000","last_active":"2023-09-13T03:12:22.487305+0000","last_peered":"2023-09-13T03:12:22.487305+0000","last_clean":"2023-09-13T03:12:22.487305+0000","last_became_active":"2023-09-13T03:11:10.959324+0000","last_became_peered":"2023-09-13T03:11:10.959324+0000","last_unstale":"2023-09-13T03:12:22.487305+0000","last_undegraded":"2023-09-13T03:12:22.487305+0000","last_fullsized":"2023-09-13T03:12:22.487305+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:48:19.245785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.476762+0000","last_change":"2023-09-13T03:11:24.576318+0000","last_active":"2023-09-13T03:12:22.476762+0000","last_peered":"2023-09-13T03:12:22.476762+0000","last_clean":"2023-09-13T03:12:22.476762+0000","last_became_active":"2023-09-13T03:11:24.575775+0000","last_became_peered":"2023-09-13T03:11:24.575775+0000","last_unstale":"2023-09-13T03:12:22.476762+0000","last_undegraded":"2023-09-13T03:12:22.476762+0000","last_fullsized":"2023-09-13T03:12:22.476762+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:16:00.534554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.472987+0000","last_change":"2023-09-13T03:11:10.945076+0000","last_active":"2023-09-13T03:12:22.472987+0000","last_peered":"2023-09-13T03:12:22.472987+0000","last_clean":"2023-09-13T03:12:22.472987+0000","last_became_active":"2023-09-13T03:11:10.944776+0000","last_became_peered":"2023-09-13T03:11:10.944776+0000","last_unstale":"2023-09-13T03:12:22.472987+0000","last_undegraded":"2023-09-13T03:12:22.472987+0000","last_fullsized":"2023-09-13T03:12:22.472987+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:22:03.624488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.475887+0000","last_change":"2023-09-13T03:11:10.942663+0000","last_active":"2023-09-13T03:12:22.475887+0000","last_peered":"2023-09-13T03:12:22.475887+0000","last_clean":"2023-09-13T03:12:22.475887+0000","last_became_active":"2023-09-13T03:11:10.942203+0000","last_became_peered":"2023-09-13T03:11:10.942203+0000","last_unstale":"2023-09-13T03:12:22.475887+0000","last_undegraded":"2023-09-13T03:12:22.475887+0000","last_fullsized":"2023-09-13T03:12:22.475887+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:46:48.091425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.469990+0000","last_change":"2023-09-13T03:11:10.940089+0000","last_active":"2023-09-13T03:12:22.469990+0000","last_peered":"2023-09-13T03:12:22.469990+0000","last_clean":"2023-09-13T03:12:22.469990+0000","last_became_active":"2023-09-13T03:11:10.939622+0000","last_became_peered":"2023-09-13T03:11:10.939622+0000","last_unstale":"2023-09-13T03:12:22.469990+0000","last_undegraded":"2023-09-13T03:12:22.469990+0000","last_fullsized":"2023-09-13T03:12:22.469990+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:43:55.024944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.477959+0000","last_change":"2023-09-13T03:11:10.956896+0000","last_active":"2023-09-13T03:12:22.477959+0000","last_peered":"2023-09-13T03:12:22.477959+0000","last_clean":"2023-09-13T03:12:22.477959+0000","last_became_active":"2023-09-13T03:11:10.956624+0000","last_became_peered":"2023-09-13T03:11:10.956624+0000","last_unstale":"2023-09-13T03:12:22.477959+0000","last_undegraded":"2023-09-13T03:12:22.477959+0000","last_fullsized":"2023-09-13T03:12:22.477959+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:29:41.305999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.479538+0000","last_change":"2023-09-13T03:11:10.939240+0000","last_active":"2023-09-13T03:12:22.479538+0000","last_peered":"2023-09-13T03:12:22.479538+0000","last_clean":"2023-09-13T03:12:22.479538+0000","last_became_active":"2023-09-13T03:11:10.938869+0000","last_became_peered":"2023-09-13T03:11:10.938869+0000","last_unstale":"2023-09-13T03:12:22.479538+0000","last_undegraded":"2023-09-13T03:12:22.479538+0000","last_fullsized":"2023-09-13T03:12:22.479538+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:06:44.931165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.483365+0000","last_change":"2023-09-13T03:11:10.956704+0000","last_active":"2023-09-13T03:12:22.483365+0000","last_peered":"2023-09-13T03:12:22.483365+0000","last_clean":"2023-09-13T03:12:22.483365+0000","last_became_active":"2023-09-13T03:11:10.956413+0000","last_became_peered":"2023-09-13T03:11:10.956413+0000","last_unstale":"2023-09-13T03:12:22.483365+0000","last_undegraded":"2023-09-13T03:12:22.483365+0000","last_fullsized":"2023-09-13T03:12:22.483365+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:50:04.626111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.475930+0000","last_change":"2023-09-13T03:11:10.952707+0000","last_active":"2023-09-13T03:12:22.475930+0000","last_peered":"2023-09-13T03:12:22.475930+0000","last_clean":"2023-09-13T03:12:22.475930+0000","last_became_active":"2023-09-13T03:11:10.952423+0000","last_became_peered":"2023-09-13T03:11:10.952423+0000","last_unstale":"2023-09-13T03:12:22.475930+0000","last_undegraded":"2023-09-13T03:12:22.475930+0000","last_fullsized":"2023-09-13T03:12:22.475930+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:35:09.869172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.474718+0000","last_change":"2023-09-13T03:11:10.941096+0000","last_active":"2023-09-13T03:12:22.474718+0000","last_peered":"2023-09-13T03:12:22.474718+0000","last_clean":"2023-09-13T03:12:22.474718+0000","last_became_active":"2023-09-13T03:11:10.940761+0000","last_became_peered":"2023-09-13T03:11:10.940761+0000","last_unstale":"2023-09-13T03:12:22.474718+0000","last_undegraded":"2023-09-13T03:12:22.474718+0000","last_fullsized":"2023-09-13T03:12:22.474718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:34:27.389650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65893,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":984,"num_read_kb":565,"num_write":245,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":68,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":573440,"data_stored":206951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449121,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":444288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:12:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.59999999999999998,"15min":0.60599999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.86099999999999999,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.58099999999999996,"15min":0.59199999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.79300000000000004,"5min":1.018,"15min":1.018},"last":0.44500000000000001}]},{"osd":1,"last update":"Wed Sep 13 03:12:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.60199999999999998,"15min":0.58399999999999996},"min":{"1min":0.55300000000000005,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.57499999999999996,"15min":0.56499999999999995},"min":{"1min":0.48299999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.68700000000000006}]},{"osd":2,"last update":"Wed Sep 13 03:12:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.45400000000000001,"15min":0.439},"min":{"1min":0.35799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.504,"15min":0.502},"min":{"1min":0.35799999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.77000000000000002,"5min":0.879,"15min":0.879},"last":0.60399999999999998}]},{"osd":3,"last update":"Wed Sep 13 03:12:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":6.415,"5min":6.415,"15min":6.415},"last":0.54000000000000004}]},{"osd":4,"last update":"Wed Sep 13 03:12:29 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.63300000000000001,"15min":0.61399999999999999},"min":{"1min":0.55100000000000005,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.61299999999999999,"15min":0.60499999999999998},"min":{"1min":0.56000000000000005,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.629}]},{"osd":5,"last update":"Wed Sep 13 03:12:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.55400000000000005,"15min":0.55000000000000004},"min":{"1min":0.36899999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79700000000000004,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.65900000000000003,"15min":0.64500000000000002},"min":{"1min":0.46300000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.70799999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":7,"last update":"Wed Sep 13 03:12:29 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.71999999999999997,"15min":0.70099999999999996},"min":{"1min":0.58099999999999996,"5min":0.497,"15min":0.497},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.71699999999999997,"15min":0.71499999999999997},"min":{"1min":0.497,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.74199999999999999}]},{"osd":8,"last update":"Wed Sep 13 03:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.75}]},{"osd":9,"last update":"Wed Sep 13 03:12:29 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.80500000000000005,"15min":0.77700000000000002},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":4.3049999999999997,"5min":4.3049999999999997,"15min":4.3049999999999997},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.82899999999999996,"15min":0.81799999999999995},"min":{"1min":0.68799999999999994,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.88800000000000001}]},{"osd":10,"last update":"Wed Sep 13 03:12:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.72399999999999998,"15min":0.71799999999999997},"min":{"1min":0.58399999999999996,"5min":0.499,"15min":0.499},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.71799999999999997,"15min":0.70599999999999996},"min":{"1min":0.64300000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.72899999999999998}]}]},{"osd":10,"up_from":71,"seq":304942678052,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1626112,"data_stored":1094195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.61399999999999999}]},{"osd":1,"last update":"Wed Sep 13 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.497,"15min":0.502},"min":{"1min":0.33800000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.71999999999999997,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.498,"15min":0.501},"min":{"1min":0.30099999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.82599999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.55700000000000005}]},{"osd":2,"last update":"Wed Sep 13 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.66500000000000004,"15min":0.67700000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81000000000000005,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.66700000000000004,"15min":0.67800000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.80400000000000005,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.67700000000000005}]},{"osd":3,"last update":"Wed Sep 13 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.66200000000000003,"15min":0.67200000000000004},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.81299999999999994,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.68500000000000005,"15min":0.69399999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.84599999999999997,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.60099999999999998}]},{"osd":4,"last update":"Wed Sep 13 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.61499999999999999,"15min":0.61899999999999999},"min":{"1min":0.376,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78300000000000003,"5min":1.03,"15min":1.03},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.39000000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80400000000000005,"5min":1.173,"15min":1.173},"last":0.52500000000000002}]},{"osd":5,"last update":"Wed Sep 13 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.78400000000000003,"15min":0.80400000000000005},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.92300000000000004,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.79600000000000004,"15min":0.81599999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.88200000000000001,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.73699999999999999}]},{"osd":6,"last update":"Wed Sep 13 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.82799999999999996,"15min":0.84799999999999998},"min":{"1min":0.53800000000000003,"5min":0.498,"15min":0.498},"max":{"1min":0.94899999999999995,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.63},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.82999999999999996,"15min":0.83999999999999997},"min":{"1min":0.56000000000000005,"5min":0.499,"15min":0.499},"max":{"1min":1.0429999999999999,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":0.70899999999999996}]},{"osd":7,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.47599999999999998}]},{"osd":8,"last update":"Wed Sep 13 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.84199999999999997,"15min":0.85999999999999999},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.88100000000000001,"5min":1.653,"15min":1.653},"last":0.752},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.879,"15min":0.90500000000000003},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.91600000000000004,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.72399999999999998}]},{"osd":9,"last update":"Wed Sep 13 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.88200000000000001,"15min":0.89700000000000002},"min":{"1min":0.61499999999999999,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.016,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.90300000000000002,"15min":0.92300000000000004},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.96899999999999997,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.65100000000000002}]},{"osd":11,"last update":"Wed Sep 13 03:12:24 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.81599999999999995,"15min":0.81200000000000006},"min":{"1min":0.70099999999999996,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":0.98899999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81999999999999995,"15min":0.82099999999999995},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.004,"5min":1.008,"15min":1.008},"last":0.81299999999999994}]}]},{"osd":9,"up_from":66,"seq":283467841577,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":444150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.57699999999999996,"15min":0.60599999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.56899999999999995,"15min":0.58899999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72499999999999998,"5min":0.875,"15min":0.875},"last":0.437}]},{"osd":1,"last update":"Wed Sep 13 03:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.64700000000000002,"15min":0.65900000000000003},"min":{"1min":0.44400000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.84999999999999998,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.70999999999999996,"15min":0.748},"min":{"1min":0.46700000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.82299999999999995,"5min":1.107,"15min":1.107},"last":0.56299999999999994}]},{"osd":2,"last update":"Wed Sep 13 03:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.71099999999999997,"15min":0.73099999999999998},"min":{"1min":0.45600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.123,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.752,"15min":0.78100000000000003},"min":{"1min":0.56200000000000006,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.62}]},{"osd":3,"last update":"Wed Sep 13 03:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.67000000000000004,"15min":0.70799999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.85099999999999998,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.71399999999999997,"15min":0.76400000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.98299999999999998,"5min":1.238,"15min":1.238},"last":0.49299999999999999}]},{"osd":4,"last update":"Wed Sep 13 03:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.79600000000000004,"15min":0.84199999999999997},"min":{"1min":0.442,"5min":0.438,"15min":0.438},"max":{"1min":0.93500000000000005,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.77000000000000002,"15min":0.80400000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.876,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.67000000000000004}]},{"osd":5,"last update":"Wed Sep 13 03:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.81799999999999995,"15min":0.85699999999999998},"min":{"1min":0.52000000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.95399999999999996,"5min":1.486,"15min":1.486},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.94799999999999995,"15min":1.034},"min":{"1min":0.59599999999999997,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.0800000000000001,"5min":4.2590000000000003,"15min":4.2590000000000003},"last":0.82499999999999996}]},{"osd":6,"last update":"Wed Sep 13 03:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.67600000000000005}]},{"osd":7,"last update":"Wed Sep 13 03:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.98699999999999999,"15min":1.077},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.252,"5min":4.367,"15min":4.367},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.752,"5min":1.0049999999999999,"15min":1.0960000000000001},"min":{"1min":0.60099999999999998,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":0.97199999999999998,"5min":4.3520000000000003,"15min":4.3520000000000003},"last":0.59599999999999997}]},{"osd":8,"last update":"Wed Sep 13 03:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.92600000000000005,"15min":0.98199999999999998},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.1830000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.82999999999999996,"5min":1.036,"15min":1.1220000000000001},"min":{"1min":0.57199999999999995,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.2,"5min":4.298,"15min":4.298},"last":0.85899999999999999}]},{"osd":10,"last update":"Wed Sep 13 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.95899999999999996,"15min":0.97899999999999998},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0900000000000001,"5min":1.321,"15min":1.321},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.93400000000000005,"15min":0.94299999999999995},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.2929999999999999,"5min":1.306,"15min":1.306},"last":0.80800000000000005}]},{"osd":11,"last update":"Wed Sep 13 03:12:23 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.83799999999999997}]}]},{"osd":8,"up_from":58,"seq":249108103213,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":444197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57899999999999996,"15min":0.58499999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.57799999999999996,"15min":0.57999999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.86099999999999999,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.57499999999999996}]},{"osd":1,"last update":"Wed Sep 13 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.61299999999999999,"15min":0.59899999999999998},"min":{"1min":0.48899999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.59899999999999998,"15min":0.59199999999999997},"min":{"1min":0.45400000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.81999999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.71399999999999997}]},{"osd":2,"last update":"Wed Sep 13 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.47599999999999998,"15min":0.46999999999999997},"min":{"1min":0.39600000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.48599999999999999,"15min":0.48199999999999998},"min":{"1min":0.34799999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.72799999999999998,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.52400000000000002}]},{"osd":3,"last update":"Wed Sep 13 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.65000000000000002,"15min":0.64300000000000002},"min":{"1min":0.46999999999999997,"5min":0.44,"15min":0.44},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.504},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.65000000000000002,"15min":0.65900000000000003},"min":{"1min":0.45000000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.87,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.54800000000000004}]},{"osd":4,"last update":"Wed Sep 13 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.63800000000000001,"15min":0.61699999999999999},"min":{"1min":0.48399999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.75,"5min":0.66300000000000003,"15min":0.65400000000000003},"min":{"1min":0.43099999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.83499999999999996}]},{"osd":5,"last update":"Wed Sep 13 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.57599999999999996,"15min":0.56599999999999995},"min":{"1min":0.40000000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.96999999999999997,"5min":0.995,"15min":0.995},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.61699999999999999,"15min":0.61799999999999999},"min":{"1min":0.38300000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.46899999999999997}]},{"osd":6,"last update":"Wed Sep 13 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.73599999999999999,"15min":0.747},"min":{"1min":0.51800000000000002,"5min":0.44,"15min":0.44},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.54600000000000004,"5min":0.501,"15min":0.501},"max":{"1min":1.103,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.70199999999999996}]},{"osd":7,"last update":"Wed Sep 13 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.71099999999999997,"15min":0.70599999999999996},"min":{"1min":0.52000000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91500000000000004,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.749,"15min":0.72599999999999998},"min":{"1min":0.64900000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.88}]},{"osd":9,"last update":"Wed Sep 13 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.78800000000000003,"15min":0.78100000000000003},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.0329999999999999,"5min":1.21,"15min":1.21},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.77100000000000002,"15min":0.76300000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0289999999999999,"5min":1.081,"15min":1.081},"last":0.68700000000000006}]},{"osd":10,"last update":"Wed Sep 13 03:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.73999999999999999,"15min":0.71399999999999997},"min":{"1min":0.55700000000000005,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.72799999999999998,"15min":0.69899999999999995},"min":{"1min":0.54400000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":2.004,"5min":2.004,"15min":2.004},"last":0.72499999999999998}]},{"osd":11,"last update":"Wed Sep 13 03:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.73999999999999999}]}]},{"osd":7,"up_from":51,"seq":219043332145,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28520,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716632,"statfs":{"total":95995035648,"available":95965831168,"internally_reserved":0,"allocated":1540096,"data_stored":1037450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:10:55 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.59699999999999998,"15min":0.58599999999999997},"min":{"1min":0.498,"5min":0.438,"15min":0.438},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.378},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60099999999999998,"15min":0.59099999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.41099999999999998}]},{"osd":1,"last update":"Wed Sep 13 03:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.42999999999999999,"15min":0.40799999999999997},"min":{"1min":0.376,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.625,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.44700000000000001,"15min":0.42099999999999999},"min":{"1min":0.39100000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.46999999999999997}]},{"osd":2,"last update":"Wed Sep 13 03:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.60799999999999998,"15min":0.59399999999999997},"min":{"1min":0.437,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86899999999999999,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.59899999999999998,"15min":0.58099999999999996},"min":{"1min":0.496,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.55700000000000005}]},{"osd":3,"last update":"Wed Sep 13 03:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.68799999999999994,"15min":0.63100000000000001},"min":{"1min":0.46899999999999997,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.81899999999999995,"5min":6.2720000000000002,"15min":6.2720000000000002},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.64700000000000002,"15min":0.625},"min":{"1min":0.48299999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88900000000000001,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.57699999999999996}]},{"osd":4,"last update":"Wed Sep 13 03:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.53900000000000003,"15min":0.51700000000000002},"min":{"1min":0.46800000000000003,"5min":0.314,"15min":0.314},"max":{"1min":0.82399999999999995,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55500000000000005,"15min":0.52700000000000002},"min":{"1min":0.39900000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.88300000000000001,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.72099999999999997}]},{"osd":5,"last update":"Wed Sep 13 03:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.66200000000000003,"15min":0.64300000000000002},"min":{"1min":0.52500000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.65300000000000002,"15min":0.625},"min":{"1min":0.56699999999999995,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.95599999999999996,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.58999999999999997}]},{"osd":6,"last update":"Wed Sep 13 03:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.72999999999999998,"15min":0.70199999999999996},"min":{"1min":0.57499999999999996,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.98999999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.625},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.75800000000000001,"15min":0.72799999999999998},"min":{"1min":0.622,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0129999999999999,"5min":1.212,"15min":1.212},"last":0.78000000000000003}]},{"osd":8,"last update":"Wed Sep 13 03:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.68700000000000006,"15min":0.65800000000000003},"min":{"1min":0.53100000000000003,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.72699999999999998,"15min":0.69899999999999995},"min":{"1min":0.51300000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.93999999999999995,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.75600000000000001}]},{"osd":9,"last update":"Wed Sep 13 03:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82199999999999995,"15min":0.81299999999999994},"min":{"1min":0.60299999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.81000000000000005,"15min":0.80800000000000005},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":0.95799999999999996,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.74399999999999999}]},{"osd":10,"last update":"Wed Sep 13 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82699999999999996,"15min":0.82799999999999996},"min":{"1min":0.60599999999999998,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.0209999999999999,"5min":1.083,"15min":1.083},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.64400000000000002,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.999,"5min":1.169,"15min":1.169},"last":0.70099999999999996}]},{"osd":11,"last update":"Wed Sep 13 03:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.76600000000000001}]}]},{"osd":6,"up_from":46,"seq":197568495671,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":444189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.50800000000000001,"15min":0.5},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.53700000000000003,"15min":0.53100000000000003},"min":{"1min":0.37,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.39500000000000002}]},{"osd":1,"last update":"Wed Sep 13 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.65400000000000003,"15min":0.63500000000000001},"min":{"1min":0.42299999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.63700000000000001,"15min":0.61899999999999999},"min":{"1min":0.47199999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.60899999999999999}]},{"osd":2,"last update":"Wed Sep 13 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.68999999999999995,"15min":0.67400000000000004},"min":{"1min":0.502,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.748},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72099999999999997,"15min":0.71699999999999997},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.879,"5min":1.879,"15min":1.879},"last":0.56299999999999994}]},{"osd":3,"last update":"Wed Sep 13 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.63300000000000001,"15min":0.64100000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.64700000000000002,"15min":0.65100000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.88500000000000001}]},{"osd":4,"last update":"Wed Sep 13 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.73499999999999999,"15min":0.73099999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.745,"15min":0.75900000000000001},"min":{"1min":0.51000000000000001,"5min":0.495,"15min":0.495},"max":{"1min":1.1659999999999999,"5min":1.54,"15min":1.54},"last":0.67400000000000004}]},{"osd":5,"last update":"Wed Sep 13 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.77700000000000002,"15min":0.79600000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.77800000000000002,"15min":0.78000000000000003},"min":{"1min":0.57599999999999996,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1339999999999999,"5min":1.21,"15min":1.21},"last":0.79500000000000004}]},{"osd":7,"last update":"Wed Sep 13 03:12:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77500000000000002,"15min":0.77000000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.1899999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.78600000000000003,"15min":0.75800000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":0.69399999999999995}]},{"osd":8,"last update":"Wed Sep 13 03:12:23 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.88,"15min":0.88800000000000001},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.82999999999999996,"15min":0.83999999999999997},"min":{"1min":0.60099999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.86199999999999999}]},{"osd":9,"last update":"Wed Sep 13 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.85799999999999998,"15min":0.88200000000000001},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.246,"5min":1.319,"15min":1.319},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.92100000000000004,"15min":0.93799999999999994},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.024,"5min":2.024,"15min":2.024},"last":0.92800000000000005}]},{"osd":10,"last update":"Wed Sep 13 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.88500000000000001,"15min":0.88400000000000001},"min":{"1min":0.60699999999999998,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.226,"5min":1.262,"15min":1.262},"last":0.874},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.95199999999999996,"15min":0.94599999999999995},"min":{"1min":0.73099999999999998,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.958,"5min":1.958,"15min":1.958},"last":0.89200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":503232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.56899999999999995,"15min":0.53900000000000003},"min":{"1min":0.39300000000000002,"5min":0.35499999999999998,"15min":0.34999999999999998},"max":{"1min":0.78600000000000003,"5min":2.9260000000000002,"15min":2.9260000000000002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.56399999999999995,"15min":0.53900000000000003},"min":{"1min":0.42399999999999999,"5min":0.34100000000000003,"15min":0.32300000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.51500000000000001}]},{"osd":2,"last update":"Wed Sep 13 03:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.58399999999999996,"15min":0.53900000000000003},"min":{"1min":0.47899999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.58699999999999997,"15min":0.54200000000000004},"min":{"1min":0.57999999999999996,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.65800000000000003}]},{"osd":3,"last update":"Wed Sep 13 03:12:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60499999999999998,"15min":0.58699999999999997},"min":{"1min":0.47599999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.65700000000000003,"15min":0.61399999999999999},"min":{"1min":0.53700000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.90300000000000002}]},{"osd":4,"last update":"Wed Sep 13 03:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.47299999999999998,"15min":0.40600000000000003},"min":{"1min":0.41399999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.83299999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.51100000000000001,"15min":0.44900000000000001},"min":{"1min":0.41299999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.77100000000000002,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.70999999999999996}]},{"osd":5,"last update":"Wed Sep 13 03:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.622,"15min":0.55100000000000005},"min":{"1min":0.625,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.626,"15min":0.56100000000000005},"min":{"1min":0.621,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.752}]},{"osd":6,"last update":"Wed Sep 13 03:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.68600000000000005,"15min":0.66000000000000003},"min":{"1min":0.53800000000000003,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.70399999999999996,"15min":0.67500000000000004},"min":{"1min":0.51900000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.94699999999999995,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":0.72999999999999998}]},{"osd":7,"last update":"Wed Sep 13 03:12:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.59899999999999998,"15min":0.57099999999999995},"min":{"1min":0.45100000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.61599999999999999,"15min":0.60399999999999998},"min":{"1min":0.45900000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84799999999999998,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.79500000000000004}]},{"osd":8,"last update":"Wed Sep 13 03:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.68999999999999995,"15min":0.67200000000000004},"min":{"1min":0.63800000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.96399999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.751,"15min":0.72899999999999998},"min":{"1min":0.629,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":1.0049999999999999}]},{"osd":9,"last update":"Wed Sep 13 03:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.79300000000000004,"15min":0.78000000000000003},"min":{"1min":0.70299999999999996,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.83899999999999997,"15min":0.82999999999999996},"min":{"1min":0.67200000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":1.0369999999999999}]},{"osd":10,"last update":"Wed Sep 13 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.79200000000000004,"15min":0.78400000000000003},"min":{"1min":0.51600000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":1.073},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.78300000000000003,"15min":0.77600000000000002},"min":{"1min":0.57399999999999995,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.88100000000000001}]},{"osd":11,"last update":"Wed Sep 13 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":1.105},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.94799999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":445206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 13 03:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.61399999999999999,"15min":0.56000000000000005},"min":{"1min":0.38,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.97299999999999998,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.64700000000000002,"15min":0.57799999999999996},"min":{"1min":0.39500000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":6.0039999999999996,"5min":6.0039999999999996,"15min":6.0039999999999996},"last":0.59199999999999997}]},{"osd":2,"last update":"Wed Sep 13 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.73199999999999998,"15min":0.63600000000000001},"min":{"1min":0.47899999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":6.9640000000000004,"5min":6.9640000000000004,"15min":6.9640000000000004},"last":0.63300000000000001},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.70899999999999996,"15min":0.621},"min":{"1min":0.504,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":6.9459999999999997,"5min":6.9459999999999997,"15min":6.9459999999999997},"last":0.50700000000000001}]},{"osd":3,"last update":"Wed Sep 13 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":8.1940000000000008,"5min":8.1940000000000008,"15min":8.1940000000000008},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":0.83299999999999996}]},{"osd":4,"last update":"Wed Sep 13 03:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.74199999999999999,"15min":0.71499999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":5.9370000000000003,"5min":5.9370000000000003,"15min":5.9370000000000003},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69299999999999995,"15min":0.68300000000000005},"min":{"1min":0.45300000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0409999999999999,"5min":1.665,"15min":1.665},"last":0.56799999999999995}]},{"osd":5,"last update":"Wed Sep 13 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71099999999999997,"15min":0.69599999999999995},"min":{"1min":0.47899999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93200000000000005,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.70799999999999996,"15min":0.68200000000000005},"min":{"1min":0.59099999999999997,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.02,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.753}]},{"osd":6,"last update":"Wed Sep 13 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.71499999999999997,"15min":0.68100000000000005},"min":{"1min":0.42299999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.83899999999999997,"5min":6.1130000000000004,"15min":6.1130000000000004},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.752,"15min":0.74199999999999999},"min":{"1min":0.51200000000000001,"5min":0.434,"15min":0.434},"max":{"1min":1.008,"5min":2.7229999999999999,"15min":2.7229999999999999},"last":0.70899999999999996}]},{"osd":7,"last update":"Wed Sep 13 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.74299999999999999,"15min":0.73899999999999999},"min":{"1min":0.56299999999999994,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.626},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.80600000000000005,"15min":0.79400000000000004},"min":{"1min":0.74399999999999999,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1379999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.77900000000000003}]},{"osd":8,"last update":"Wed Sep 13 03:12:24 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.81399999999999995,"15min":0.81799999999999995},"min":{"1min":0.67200000000000004,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.2330000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.3680000000000001,"5min":0.89300000000000002,"15min":0.81000000000000005},"min":{"1min":0.57299999999999995,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":6.9489999999999998,"5min":6.9489999999999998,"15min":6.9489999999999998},"last":0.88200000000000001}]},{"osd":9,"last update":"Wed Sep 13 03:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.86499999999999999,"15min":0.88900000000000001},"min":{"1min":0.59099999999999997,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.2549999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.86499999999999999,"15min":0.88700000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.85199999999999998}]},{"osd":10,"last update":"Wed Sep 13 03:12:09 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":0.92400000000000004,"15min":0.89000000000000001},"min":{"1min":0.68200000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":6.0830000000000002,"5min":6.0830000000000002,"15min":6.0830000000000002},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.165,"5min":1.0009999999999999,"15min":0.97299999999999998},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":6.9669999999999996,"5min":6.9669999999999996,"15min":6.9669999999999996},"last":0.70199999999999996}]},{"osd":11,"last update":"Wed Sep 13 03:12:31 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.88700000000000001,"15min":0.88500000000000001},"min":{"1min":0.70199999999999996,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.94299999999999995,"15min":0.95099999999999996},"min":{"1min":0.71499999999999997,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.2210000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.76800000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":503872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.55400000000000005,"15min":0.49099999999999999},"min":{"1min":0.41599999999999998,"5min":0.307,"15min":0.307},"max":{"1min":2.4390000000000001,"5min":2.4390000000000001,"15min":2.4390000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.57999999999999996,"15min":0.50700000000000001},"min":{"1min":0.46000000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":3.0979999999999999,"5min":3.0979999999999999,"15min":3.0979999999999999},"last":0.79700000000000004}]},{"osd":1,"last update":"Wed Sep 13 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.56299999999999994,"15min":0.52500000000000002},"min":{"1min":0.443,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.55200000000000005,"15min":0.53200000000000003},"min":{"1min":0.37,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.497}]},{"osd":3,"last update":"Wed Sep 13 03:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.65600000000000003,"15min":0.57999999999999996},"min":{"1min":0.52800000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90400000000000003,"5min":2.879,"15min":2.879},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.65600000000000003,"15min":0.58399999999999996},"min":{"1min":0.44600000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":2.782,"5min":2.8969999999999998,"15min":2.8969999999999998},"last":0.60599999999999998}]},{"osd":4,"last update":"Wed Sep 13 03:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.624},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.59099999999999997}]},{"osd":5,"last update":"Wed Sep 13 03:12:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.46999999999999997,"15min":0.433},"min":{"1min":0.36299999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.502,"15min":0.45600000000000002},"min":{"1min":0.36399999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.95699999999999996,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.69999999999999996}]},{"osd":6,"last update":"Wed Sep 13 03:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.67100000000000004,"15min":0.63200000000000001},"min":{"1min":0.51400000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.95099999999999996,"5min":1.048,"15min":1.048},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.67600000000000005,"15min":0.63},"min":{"1min":0.39700000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90400000000000003,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.74199999999999999}]},{"osd":7,"last update":"Wed Sep 13 03:10:58 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.63700000000000001,"15min":0.63600000000000001},"min":{"1min":0.47499999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.84699999999999998,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.66100000000000003,"15min":0.65900000000000003},"min":{"1min":0.51700000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.83399999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.497}]},{"osd":8,"last update":"Wed Sep 13 03:12:22 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.59899999999999998,"15min":0.58799999999999997},"min":{"1min":0.41499999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.69899999999999995,"15min":0.69299999999999995},"min":{"1min":0.59599999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.78200000000000003}]},{"osd":9,"last update":"Wed Sep 13 03:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.78200000000000003,"15min":0.75800000000000001},"min":{"1min":0.56000000000000005,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.98899999999999999,"5min":2.6459999999999999,"15min":2.6459999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.753,"15min":0.73999999999999999},"min":{"1min":0.59099999999999997,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.002,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.84399999999999997}]},{"osd":10,"last update":"Wed Sep 13 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.68500000000000005,"15min":0.66800000000000004},"min":{"1min":0.60299999999999998,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.73399999999999999,"15min":0.71499999999999997},"min":{"1min":0.58899999999999997,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.75800000000000001}]},{"osd":11,"last update":"Wed Sep 13 03:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.81000000000000005}]}]},{"osd":3,"up_from":25,"seq":107374182468,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":443704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:12:31 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.44600000000000001,"15min":0.40999999999999998},"min":{"1min":0.38500000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.437,"5min":0.46500000000000002,"15min":0.437},"min":{"1min":0.36199999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.57099999999999995,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.36299999999999999}]},{"osd":1,"last update":"Wed Sep 13 03:12:31 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.63700000000000001,"15min":0.59799999999999998},"min":{"1min":0.44,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80500000000000005,"5min":1.117,"15min":1.117},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.66100000000000003,"15min":0.60699999999999998},"min":{"1min":0.42199999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94899999999999995,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.57899999999999996}]},{"osd":2,"last update":"Wed Sep 13 03:12:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.69299999999999995,"15min":0.64300000000000002},"min":{"1min":0.55000000000000004,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.88800000000000001,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.69299999999999995,"15min":0.64900000000000002},"min":{"1min":0.44,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.85499999999999998,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.59999999999999998}]},{"osd":4,"last update":"Wed Sep 13 03:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.68300000000000005,"15min":0.63600000000000001},"min":{"1min":0.49399999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0900000000000001,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.67500000000000004,"15min":0.627},"min":{"1min":0.52000000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0389999999999999,"5min":1.177,"15min":1.177},"last":0.54500000000000004}]},{"osd":5,"last update":"Wed Sep 13 03:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.71399999999999997,"15min":0.68600000000000005},"min":{"1min":0.49299999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.96599999999999997,"5min":1.012,"15min":1.012},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72599999999999998,"15min":0.70399999999999996},"min":{"1min":0.53300000000000003,"5min":0.498,"15min":0.498},"max":{"1min":1.0069999999999999,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.76000000000000001}]},{"osd":6,"last update":"Wed Sep 13 03:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.64400000000000002,"15min":0.624},"min":{"1min":0.53700000000000003,"5min":0.374,"15min":0.374},"max":{"1min":0.72699999999999998,"5min":1.117,"15min":1.117},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.77800000000000002,"15min":0.752},"min":{"1min":0.628,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0149999999999999,"5min":1.599,"15min":1.599},"last":0.85599999999999998}]},{"osd":7,"last update":"Wed Sep 13 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.86099999999999999,"15min":0.85399999999999998},"min":{"1min":0.55300000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0529999999999999,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.83299999999999996,"15min":0.82799999999999996},"min":{"1min":0.56699999999999995,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0269999999999999,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.70099999999999996}]},{"osd":8,"last update":"Wed Sep 13 03:10:17 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.82199999999999995}]},{"osd":9,"last update":"Wed Sep 13 03:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.88700000000000001,"15min":0.91500000000000004},"min":{"1min":0.52700000000000002,"5min":0.502,"15min":0.502},"max":{"1min":0.91700000000000004,"5min":1.538,"15min":1.538},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.84099999999999997,"15min":0.88500000000000001},"min":{"1min":0.57499999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.90100000000000002,"5min":1.677,"15min":1.677},"last":0.79300000000000004}]},{"osd":10,"last update":"Wed Sep 13 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.93000000000000005,"15min":0.93899999999999995},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.034,"5min":1.708,"15min":1.708},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.95399999999999996,"15min":0.96399999999999997},"min":{"1min":0.69899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.1180000000000001,"5min":1.635,"15min":1.635},"last":0.86299999999999999}]},{"osd":11,"last update":"Wed Sep 13 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.747}]}]},{"osd":4,"up_from":33,"seq":141733920832,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":444259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.54500000000000004,"15min":0.51500000000000001},"min":{"1min":0.36699999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.55900000000000005,"15min":0.52900000000000003},"min":{"1min":0.35599999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.63800000000000001}]},{"osd":1,"last update":"Wed Sep 13 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.39700000000000002,"15min":0.36199999999999999},"min":{"1min":0.308,"5min":0.245,"15min":0.245},"max":{"1min":0.66700000000000004,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.379},{"interface":"front","average":{"1min":0.443,"5min":0.40799999999999997,"15min":0.372},"min":{"1min":0.32500000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.68200000000000005,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.436}]},{"osd":2,"last update":"Wed Sep 13 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58099999999999996,"15min":0.56299999999999994},"min":{"1min":0.44,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.81999999999999995,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.58999999999999997,"15min":0.54000000000000004},"min":{"1min":0.38600000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.59699999999999998}]},{"osd":3,"last update":"Wed Sep 13 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63100000000000001,"15min":0.56299999999999994},"min":{"1min":0.46400000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83899999999999997,"5min":4.2919999999999998,"15min":4.2919999999999998},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.59999999999999998,"15min":0.56999999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81999999999999995,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.57099999999999995}]},{"osd":5,"last update":"Wed Sep 13 03:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.61699999999999999,"15min":0.58299999999999996},"min":{"1min":0.44900000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.874,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.60699999999999998,"15min":0.56899999999999995},"min":{"1min":0.42399999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.46800000000000003}]},{"osd":6,"last update":"Wed Sep 13 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.745,"15min":0.71299999999999997},"min":{"1min":0.45100000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.86799999999999999,"5min":6.1859999999999999,"15min":6.1859999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.70599999999999996,"15min":0.71199999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.84399999999999997,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.66300000000000003}]},{"osd":7,"last update":"Wed Sep 13 03:11:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.628,"15min":0.60999999999999999},"min":{"1min":0.34200000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.93300000000000005,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63600000000000001,"15min":0.622},"min":{"1min":0.39700000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89700000000000002,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.67800000000000005}]},{"osd":8,"last update":"Wed Sep 13 03:12:20 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.72399999999999998,"15min":0.72299999999999998},"min":{"1min":0.51300000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0029999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.76100000000000001,"15min":0.755},"min":{"1min":0.629,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.98499999999999999,"5min":1.01,"15min":1.01},"last":0.74299999999999999}]},{"osd":9,"last update":"Wed Sep 13 03:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.90000000000000002,"15min":0.878},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0569999999999999,"5min":6.1529999999999996,"15min":6.1529999999999996},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.81299999999999994,"15min":0.82899999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.96099999999999997,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.71799999999999997}]},{"osd":10,"last update":"Wed Sep 13 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.81200000000000006,"15min":0.82099999999999995},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.016,"5min":1.113,"15min":1.113},"last":0.80000000000000004}]},{"osd":11,"last update":"Wed Sep 13 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.75800000000000001}]}]},{"osd":5,"up_from":38,"seq":163208757309,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28536,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716616,"statfs":{"total":95995035648,"available":95965814784,"internally_reserved":0,"allocated":1556480,"data_stored":1036870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:12:14 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":0.69999999999999996,"15min":0.64700000000000002},"min":{"1min":0.505,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":9.3100000000000005,"5min":9.3100000000000005,"15min":9.3100000000000005},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.67600000000000005,"15min":0.64200000000000002},"min":{"1min":0.42399999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":2.9169999999999998,"5min":6.0179999999999998,"15min":6.0179999999999998},"last":0.64600000000000002}]},{"osd":1,"last update":"Wed Sep 13 03:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.56699999999999995,"15min":0.52700000000000002},"min":{"1min":0.42999999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76200000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.56399999999999995,"15min":0.52000000000000002},"min":{"1min":0.49299999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.48999999999999999}]},{"osd":2,"last update":"Wed Sep 13 03:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.41999999999999998,"15min":0.39000000000000001},"min":{"1min":0.34799999999999998,"5min":0.193,"15min":0.193},"max":{"1min":0.76000000000000001,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.42099999999999999,"15min":0.39200000000000002},"min":{"1min":0.35799999999999998,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.64900000000000002,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.36699999999999999}]},{"osd":3,"last update":"Wed Sep 13 03:12:14 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":0.68400000000000005,"15min":0.60199999999999998},"min":{"1min":0.48499999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":9.4369999999999994,"5min":9.4369999999999994,"15min":9.4369999999999994},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61199999999999999,"15min":0.58999999999999997},"min":{"1min":0.46100000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.96299999999999997,"5min":1.48,"15min":1.48},"last":0.58899999999999997}]},{"osd":4,"last update":"Wed Sep 13 03:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.626,"15min":0.56599999999999995},"min":{"1min":0.48299999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.88400000000000001,"5min":3.3740000000000001,"15min":3.3740000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.59299999999999997,"15min":0.55700000000000005},"min":{"1min":0.44600000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79100000000000004,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.46500000000000002}]},{"osd":6,"last update":"Wed Sep 13 03:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.68600000000000005,"15min":0.64900000000000002},"min":{"1min":0.498,"5min":0.371,"15min":0.371},"max":{"1min":0.88300000000000001,"5min":4.8250000000000002,"15min":4.8250000000000002},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72099999999999997,"15min":0.67200000000000004},"min":{"1min":0.44600000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.90800000000000003,"5min":4.8049999999999997,"15min":4.8049999999999997},"last":0.71199999999999997}]},{"osd":7,"last update":"Wed Sep 13 03:12:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.63400000000000001,"15min":0.63300000000000001},"min":{"1min":0.47899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.86399999999999999,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.66600000000000004,"15min":0.65900000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85499999999999998,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.66600000000000004}]},{"osd":8,"last update":"Wed Sep 13 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57099999999999995,"15min":0.57599999999999996},"min":{"1min":0.36799999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83299999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61899999999999999,"15min":0.61799999999999999},"min":{"1min":0.48299999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.80800000000000005,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.68200000000000005}]},{"osd":9,"last update":"Wed Sep 13 03:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":1.0069999999999999,"15min":1.024},"min":{"1min":0.623,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.91500000000000004,"5min":8.3070000000000004,"15min":8.3070000000000004},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79300000000000004,"15min":0.76900000000000002},"min":{"1min":0.63400000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.97499999999999998,"5min":3.016,"15min":3.016},"last":0.79300000000000004}]},{"osd":10,"last update":"Wed Sep 13 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.68600000000000005,"15min":0.67600000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.876,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.74099999999999999,"15min":0.73299999999999998},"min":{"1min":0.57499999999999996,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.755}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":65164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-13T03:12:55.926 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-09-13T03:12:55.927 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-13T03:12:55.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:55 smithi142 ceph-mon[134082]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2023-09-13T03:12:55.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:55 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/1278840691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:12:56.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.162+0000 7f1a279ff700 1 -- 172.21.15.64:0/3941409353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a20100600 msgr2=0x7f1a20100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:56.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.162+0000 7f1a279ff700 1 --2- 172.21.15.64:0/3941409353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a20100600 0x7f1a20100a20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c0098f0 tx=0x7f1a1c009c00 comp rx=0 tx=0).stop 2023-09-13T03:12:56.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.162+0000 7f1a279ff700 1 -- 172.21.15.64:0/3941409353 shutdown_connections 2023-09-13T03:12:56.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.162+0000 7f1a279ff700 1 --2- 172.21.15.64:0/3941409353 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a20102190 0x7f1a20108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:56.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.162+0000 7f1a279ff700 1 --2- 172.21.15.64:0/3941409353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a20100600 0x7f1a20100a20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:56.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.162+0000 7f1a279ff700 1 --2- 172.21.15.64:0/3941409353 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a201017f0 0x7f1a20101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:56.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.162+0000 7f1a279ff700 1 -- 172.21.15.64:0/3941409353 >> 172.21.15.64:0/3941409353 conn(0x7f1a200fbb30 msgr2=0x7f1a200fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:56.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.162+0000 7f1a279ff700 1 -- 172.21.15.64:0/3941409353 shutdown_connections 2023-09-13T03:12:56.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.162+0000 7f1a279ff700 1 -- 172.21.15.64:0/3941409353 wait complete. 2023-09-13T03:12:56.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.163+0000 7f1a279ff700 1 Processor -- start 2023-09-13T03:12:56.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.163+0000 7f1a279ff700 1 -- start start 2023-09-13T03:12:56.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.164+0000 7f1a279ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a20100600 0x7f1a200743b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:56.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.164+0000 7f1a279ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a201017f0 0x7f1a200729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:56.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.164+0000 7f1a279ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a20102190 0x7f1a20072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:56.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.164+0000 7f1a279ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a200736d0 con 0x7f1a20100600 2023-09-13T03:12:56.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.164+0000 7f1a279ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a20073840 con 0x7f1a20102190 2023-09-13T03:12:56.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.164+0000 7f1a279ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a201a0370 con 0x7f1a201017f0 2023-09-13T03:12:56.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.164+0000 7f1a2579b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a20100600 0x7f1a200743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:56.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.164+0000 7f1a25f9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a20102190 0x7f1a20072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:56.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.164+0000 7f1a24f9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a201017f0 0x7f1a200729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:56.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.164+0000 7f1a2579b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a20100600 0x7f1a200743b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56072/0 (socket says 172.21.15.64:56072) 2023-09-13T03:12:56.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.164+0000 7f1a2579b700 1 -- 172.21.15.64:0/1641587848 learned_addr learned my addr 172.21.15.64:0/1641587848 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:56.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.164+0000 7f1a25f9c700 1 -- 172.21.15.64:0/1641587848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a201017f0 msgr2=0x7f1a200729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:56.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.164+0000 7f1a25f9c700 1 --2- 172.21.15.64:0/1641587848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a201017f0 0x7f1a200729f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:56.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.164+0000 7f1a25f9c700 1 -- 172.21.15.64:0/1641587848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a20100600 msgr2=0x7f1a200743b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:56.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.164+0000 7f1a25f9c700 1 --2- 172.21.15.64:0/1641587848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a20100600 0x7f1a200743b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:56.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.164+0000 7f1a25f9c700 1 -- 172.21.15.64:0/1641587848 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a10009520 con 0x7f1a20102190 2023-09-13T03:12:56.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.164+0000 7f1a2579b700 1 --2- 172.21.15.64:0/1641587848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a20100600 0x7f1a200743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:56.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.165+0000 7f1a25f9c700 1 --2- 172.21.15.64:0/1641587848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a20102190 0x7f1a20072f30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1a1800efc0 tx=0x7f1a1800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:56.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.165+0000 7f1a167fc700 1 -- 172.21.15.64:0/1641587848 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a18018070 con 0x7f1a20102190 2023-09-13T03:12:56.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.165+0000 7f1a167fc700 1 -- 172.21.15.64:0/1641587848 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a1800f040 con 0x7f1a20102190 2023-09-13T03:12:56.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.165+0000 7f1a279ff700 1 -- 172.21.15.64:0/1641587848 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a1c009520 con 0x7f1a20102190 2023-09-13T03:12:56.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.165+0000 7f1a167fc700 1 -- 172.21.15.64:0/1641587848 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1800ccb0 con 0x7f1a20102190 2023-09-13T03:12:56.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.165+0000 7f1a279ff700 1 -- 172.21.15.64:0/1641587848 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a201a0970 con 0x7f1a20102190 2023-09-13T03:12:56.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.166+0000 7f1a279ff700 1 -- 172.21.15.64:0/1641587848 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a2010be30 con 0x7f1a20102190 2023-09-13T03:12:56.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.168+0000 7f1a167fc700 1 -- 172.21.15.64:0/1641587848 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f1a18003a00 con 0x7f1a20102190 2023-09-13T03:12:56.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.170+0000 7f1a167fc700 1 --2- 172.21.15.64:0/1641587848 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f1a0c0a0690 0x7f1a0c0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:56.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.170+0000 7f1a167fc700 1 -- 172.21.15.64:0/1641587848 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f1a180cbfc0 con 0x7f1a20102190 2023-09-13T03:12:56.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.170+0000 7f1a2579b700 1 --2- 172.21.15.64:0/1641587848 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f1a0c0a0690 0x7f1a0c0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:56.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.171+0000 7f1a2579b700 1 --2- 172.21.15.64:0/1641587848 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f1a0c0a0690 0x7f1a0c0a2b50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c00c010 tx=0x7f1a1c019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:56.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.172+0000 7f1a167fc700 1 -- 172.21.15.64:0/1641587848 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1a18095b50 con 0x7f1a20102190 2023-09-13T03:12:56.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.377+0000 7f1a279ff700 1 -- 172.21.15.64:0/1641587848 --> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1a20060a40 con 0x7f1a0c0a0690 2023-09-13T03:12:56.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.388+0000 7f1a167fc700 1 -- 172.21.15.64:0/1641587848 <== mgr.14150 v2:172.21.15.64:6800/873679308 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345051 (secure 0 0 0) 0x7f1a20060a40 con 0x7f1a0c0a0690 2023-09-13T03:12:56.393 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:12:56.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.393+0000 7f1a279ff700 1 -- 172.21.15.64:0/1641587848 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f1a0c0a0690 msgr2=0x7f1a0c0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:56.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.393+0000 7f1a279ff700 1 --2- 172.21.15.64:0/1641587848 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f1a0c0a0690 0x7f1a0c0a2b50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c00c010 tx=0x7f1a1c019040 comp rx=0 tx=0).stop 2023-09-13T03:12:56.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.393+0000 7f1a279ff700 1 -- 172.21.15.64:0/1641587848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a20102190 msgr2=0x7f1a20072f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:56.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.393+0000 7f1a279ff700 1 --2- 172.21.15.64:0/1641587848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a20102190 0x7f1a20072f30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1a1800efc0 tx=0x7f1a1800c3a0 comp rx=0 tx=0).stop 2023-09-13T03:12:56.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.393+0000 7f1a279ff700 1 -- 172.21.15.64:0/1641587848 shutdown_connections 2023-09-13T03:12:56.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.393+0000 7f1a279ff700 1 --2- 172.21.15.64:0/1641587848 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f1a0c0a0690 0x7f1a0c0a2b50 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:56.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.393+0000 7f1a279ff700 1 --2- 172.21.15.64:0/1641587848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a20102190 0x7f1a20072f30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:56.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.393+0000 7f1a279ff700 1 --2- 172.21.15.64:0/1641587848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a20100600 0x7f1a200743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:56.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.393+0000 7f1a279ff700 1 --2- 172.21.15.64:0/1641587848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a201017f0 0x7f1a200729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:56.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.393+0000 7f1a279ff700 1 -- 172.21.15.64:0/1641587848 >> 172.21.15.64:0/1641587848 conn(0x7f1a200fbb30 msgr2=0x7f1a200fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:56.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.393+0000 7f1a279ff700 1 -- 172.21.15.64:0/1641587848 shutdown_connections 2023-09-13T03:12:56.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.393+0000 7f1a279ff700 1 -- 172.21.15.64:0/1641587848 wait complete. 2023-09-13T03:12:56.398 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-09-13T03:12:56.412 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2023-09-13T03:12:54.478794+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656507,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1120,"num_read_kb":684,"num_write":428,"num_write_kb":3077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180804,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":853,"ondisk_log_size":853,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337200,"kb_used_data":13008,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604624,"statfs":{"total":1151940427776,"available":1151595134976,"internally_reserved":0,"allocated":13320192,"data_stored":7285612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19130,"internal_metadata":331855174},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":316,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":136,"num_read_kb":71,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002563"},"pg_stats":[{"pgid":"3.3f","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.480907+0000","last_change":"2023-09-13T03:11:10.949449+0000","last_active":"2023-09-13T03:12:22.480907+0000","last_peered":"2023-09-13T03:12:22.480907+0000","last_clean":"2023-09-13T03:12:22.480907+0000","last_became_active":"2023-09-13T03:11:10.948912+0000","last_became_peered":"2023-09-13T03:11:10.948912+0000","last_unstale":"2023-09-13T03:12:22.480907+0000","last_undegraded":"2023-09-13T03:12:22.480907+0000","last_fullsized":"2023-09-13T03:12:22.480907+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:34:27.381352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.479553+0000","last_change":"2023-09-13T03:11:10.956817+0000","last_active":"2023-09-13T03:12:22.479553+0000","last_peered":"2023-09-13T03:12:22.479553+0000","last_clean":"2023-09-13T03:12:22.479553+0000","last_became_active":"2023-09-13T03:11:10.955359+0000","last_became_peered":"2023-09-13T03:11:10.955359+0000","last_unstale":"2023-09-13T03:12:22.479553+0000","last_undegraded":"2023-09-13T03:12:22.479553+0000","last_fullsized":"2023-09-13T03:12:22.479553+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:06:01.055910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.483472+0000","last_change":"2023-09-13T03:11:10.959915+0000","last_active":"2023-09-13T03:12:22.483472+0000","last_peered":"2023-09-13T03:12:22.483472+0000","last_clean":"2023-09-13T03:12:22.483472+0000","last_became_active":"2023-09-13T03:11:10.959611+0000","last_became_peered":"2023-09-13T03:11:10.959611+0000","last_unstale":"2023-09-13T03:12:22.483472+0000","last_undegraded":"2023-09-13T03:12:22.483472+0000","last_fullsized":"2023-09-13T03:12:22.483472+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:31:40.075569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.474409+0000","last_change":"2023-09-13T03:11:24.269022+0000","last_active":"2023-09-13T03:12:22.474409+0000","last_peered":"2023-09-13T03:12:22.474409+0000","last_clean":"2023-09-13T03:12:22.474409+0000","last_became_active":"2023-09-13T03:11:24.268019+0000","last_became_peered":"2023-09-13T03:11:24.268019+0000","last_unstale":"2023-09-13T03:12:22.474409+0000","last_undegraded":"2023-09-13T03:12:22.474409+0000","last_fullsized":"2023-09-13T03:12:22.474409+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:58:06.518437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.469930+0000","last_change":"2023-09-13T03:11:10.951554+0000","last_active":"2023-09-13T03:12:22.469930+0000","last_peered":"2023-09-13T03:12:22.469930+0000","last_clean":"2023-09-13T03:12:22.469930+0000","last_became_active":"2023-09-13T03:11:10.951252+0000","last_became_peered":"2023-09-13T03:11:10.951252+0000","last_unstale":"2023-09-13T03:12:22.469930+0000","last_undegraded":"2023-09-13T03:12:22.469930+0000","last_fullsized":"2023-09-13T03:12:22.469930+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:38:42.890256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.487106+0000","last_change":"2023-09-13T03:11:10.953087+0000","last_active":"2023-09-13T03:12:22.487106+0000","last_peered":"2023-09-13T03:12:22.487106+0000","last_clean":"2023-09-13T03:12:22.487106+0000","last_became_active":"2023-09-13T03:11:10.952773+0000","last_became_peered":"2023-09-13T03:11:10.952773+0000","last_unstale":"2023-09-13T03:12:22.487106+0000","last_undegraded":"2023-09-13T03:12:22.487106+0000","last_fullsized":"2023-09-13T03:12:22.487106+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:14:55.203347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.481903+0000","last_change":"2023-09-13T03:11:10.956058+0000","last_active":"2023-09-13T03:12:22.481903+0000","last_peered":"2023-09-13T03:12:22.481903+0000","last_clean":"2023-09-13T03:12:22.481903+0000","last_became_active":"2023-09-13T03:11:10.955783+0000","last_became_peered":"2023-09-13T03:11:10.955783+0000","last_unstale":"2023-09-13T03:12:22.481903+0000","last_undegraded":"2023-09-13T03:12:22.481903+0000","last_fullsized":"2023-09-13T03:12:22.481903+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:31:06.810338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.482123+0000","last_change":"2023-09-13T03:11:10.950102+0000","last_active":"2023-09-13T03:12:22.482123+0000","last_peered":"2023-09-13T03:12:22.482123+0000","last_clean":"2023-09-13T03:12:22.482123+0000","last_became_active":"2023-09-13T03:11:10.948877+0000","last_became_peered":"2023-09-13T03:11:10.948877+0000","last_unstale":"2023-09-13T03:12:22.482123+0000","last_undegraded":"2023-09-13T03:12:22.482123+0000","last_fullsized":"2023-09-13T03:12:22.482123+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:32:09.168793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.480791+0000","last_change":"2023-09-13T03:11:10.945244+0000","last_active":"2023-09-13T03:12:22.480791+0000","last_peered":"2023-09-13T03:12:22.480791+0000","last_clean":"2023-09-13T03:12:22.480791+0000","last_became_active":"2023-09-13T03:11:10.944951+0000","last_became_peered":"2023-09-13T03:11:10.944951+0000","last_unstale":"2023-09-13T03:12:22.480791+0000","last_undegraded":"2023-09-13T03:12:22.480791+0000","last_fullsized":"2023-09-13T03:12:22.480791+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:42:35.420767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.487770+0000","last_change":"2023-09-13T03:11:10.952312+0000","last_active":"2023-09-13T03:12:22.487770+0000","last_peered":"2023-09-13T03:12:22.487770+0000","last_clean":"2023-09-13T03:12:22.487770+0000","last_became_active":"2023-09-13T03:11:10.952077+0000","last_became_peered":"2023-09-13T03:11:10.952077+0000","last_unstale":"2023-09-13T03:12:22.487770+0000","last_undegraded":"2023-09-13T03:12:22.487770+0000","last_fullsized":"2023-09-13T03:12:22.487770+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:57:10.888184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.478509+0000","last_change":"2023-09-13T03:11:10.959259+0000","last_active":"2023-09-13T03:12:22.478509+0000","last_peered":"2023-09-13T03:12:22.478509+0000","last_clean":"2023-09-13T03:12:22.478509+0000","last_became_active":"2023-09-13T03:11:10.958922+0000","last_became_peered":"2023-09-13T03:11:10.958922+0000","last_unstale":"2023-09-13T03:12:22.478509+0000","last_undegraded":"2023-09-13T03:12:22.478509+0000","last_fullsized":"2023-09-13T03:12:22.478509+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:08:54.680718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.478268+0000","last_change":"2023-09-13T03:11:10.950946+0000","last_active":"2023-09-13T03:12:22.478268+0000","last_peered":"2023-09-13T03:12:22.478268+0000","last_clean":"2023-09-13T03:12:22.478268+0000","last_became_active":"2023-09-13T03:11:10.950633+0000","last_became_peered":"2023-09-13T03:11:10.950633+0000","last_unstale":"2023-09-13T03:12:22.478268+0000","last_undegraded":"2023-09-13T03:12:22.478268+0000","last_fullsized":"2023-09-13T03:12:22.478268+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:37:25.796555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.480994+0000","last_change":"2023-09-13T03:11:10.957712+0000","last_active":"2023-09-13T03:12:22.480994+0000","last_peered":"2023-09-13T03:12:22.480994+0000","last_clean":"2023-09-13T03:12:22.480994+0000","last_became_active":"2023-09-13T03:11:10.957379+0000","last_became_peered":"2023-09-13T03:11:10.957379+0000","last_unstale":"2023-09-13T03:12:22.480994+0000","last_undegraded":"2023-09-13T03:12:22.480994+0000","last_fullsized":"2023-09-13T03:12:22.480994+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:31:17.084185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.480310+0000","last_change":"2023-09-13T03:11:10.952951+0000","last_active":"2023-09-13T03:12:22.480310+0000","last_peered":"2023-09-13T03:12:22.480310+0000","last_clean":"2023-09-13T03:12:22.480310+0000","last_became_active":"2023-09-13T03:11:10.952629+0000","last_became_peered":"2023-09-13T03:11:10.952629+0000","last_unstale":"2023-09-13T03:12:22.480310+0000","last_undegraded":"2023-09-13T03:12:22.480310+0000","last_fullsized":"2023-09-13T03:12:22.480310+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:20:40.802584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.486815+0000","last_change":"2023-09-13T03:11:10.950799+0000","last_active":"2023-09-13T03:12:22.486815+0000","last_peered":"2023-09-13T03:12:22.486815+0000","last_clean":"2023-09-13T03:12:22.486815+0000","last_became_active":"2023-09-13T03:11:10.950500+0000","last_became_peered":"2023-09-13T03:11:10.950500+0000","last_unstale":"2023-09-13T03:12:22.486815+0000","last_undegraded":"2023-09-13T03:12:22.486815+0000","last_fullsized":"2023-09-13T03:12:22.486815+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:24:45.770696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.473581+0000","last_change":"2023-09-13T03:11:10.955141+0000","last_active":"2023-09-13T03:12:22.473581+0000","last_peered":"2023-09-13T03:12:22.473581+0000","last_clean":"2023-09-13T03:12:22.473581+0000","last_became_active":"2023-09-13T03:11:10.954840+0000","last_became_peered":"2023-09-13T03:11:10.954840+0000","last_unstale":"2023-09-13T03:12:22.473581+0000","last_undegraded":"2023-09-13T03:12:22.473581+0000","last_fullsized":"2023-09-13T03:12:22.473581+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:56:08.318784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.483342+0000","last_change":"2023-09-13T03:11:10.949199+0000","last_active":"2023-09-13T03:12:22.483342+0000","last_peered":"2023-09-13T03:12:22.483342+0000","last_clean":"2023-09-13T03:12:22.483342+0000","last_became_active":"2023-09-13T03:11:10.948506+0000","last_became_peered":"2023-09-13T03:11:10.948506+0000","last_unstale":"2023-09-13T03:12:22.483342+0000","last_undegraded":"2023-09-13T03:12:22.483342+0000","last_fullsized":"2023-09-13T03:12:22.483342+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:04:07.714033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.481246+0000","last_change":"2023-09-13T03:11:10.953207+0000","last_active":"2023-09-13T03:12:22.481246+0000","last_peered":"2023-09-13T03:12:22.481246+0000","last_clean":"2023-09-13T03:12:22.481246+0000","last_became_active":"2023-09-13T03:11:10.951566+0000","last_became_peered":"2023-09-13T03:11:10.951566+0000","last_unstale":"2023-09-13T03:12:22.481246+0000","last_undegraded":"2023-09-13T03:12:22.481246+0000","last_fullsized":"2023-09-13T03:12:22.481246+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:23:06.020880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.473777+0000","last_change":"2023-09-13T03:11:10.951695+0000","last_active":"2023-09-13T03:12:22.473777+0000","last_peered":"2023-09-13T03:12:22.473777+0000","last_clean":"2023-09-13T03:12:22.473777+0000","last_became_active":"2023-09-13T03:11:10.951382+0000","last_became_peered":"2023-09-13T03:11:10.951382+0000","last_unstale":"2023-09-13T03:12:22.473777+0000","last_undegraded":"2023-09-13T03:12:22.473777+0000","last_fullsized":"2023-09-13T03:12:22.473777+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:29:05.200980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'14","reported_seq":124,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:50.375945+0000","last_change":"2023-09-13T03:11:09.956193+0000","last_active":"2023-09-13T03:12:50.375945+0000","last_peered":"2023-09-13T03:12:50.375945+0000","last_clean":"2023-09-13T03:12:50.375945+0000","last_became_active":"2023-09-13T03:11:09.955778+0000","last_became_peered":"2023-09-13T03:11:09.955778+0000","last_unstale":"2023-09-13T03:12:50.375945+0000","last_undegraded":"2023-09-13T03:12:50.375945+0000","last_fullsized":"2023-09-13T03:12:50.375945+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:15:57.425730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":95,"num_read_kb":95,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.475680+0000","last_change":"2023-09-13T03:11:10.944437+0000","last_active":"2023-09-13T03:12:22.475680+0000","last_peered":"2023-09-13T03:12:22.475680+0000","last_clean":"2023-09-13T03:12:22.475680+0000","last_became_active":"2023-09-13T03:11:10.943961+0000","last_became_peered":"2023-09-13T03:11:10.943961+0000","last_unstale":"2023-09-13T03:12:22.475680+0000","last_undegraded":"2023-09-13T03:12:22.475680+0000","last_fullsized":"2023-09-13T03:12:22.475680+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:39:43.928334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.266435+0000","last_change":"2023-09-13T03:11:09.952017+0000","last_active":"2023-09-13T03:11:23.266435+0000","last_peered":"2023-09-13T03:11:23.266435+0000","last_clean":"2023-09-13T03:11:23.266435+0000","last_became_active":"2023-09-13T03:11:09.951700+0000","last_became_peered":"2023-09-13T03:11:09.951700+0000","last_unstale":"2023-09-13T03:11:23.266435+0000","last_undegraded":"2023-09-13T03:11:23.266435+0000","last_fullsized":"2023-09-13T03:11:23.266435+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:28:50.687271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.480661+0000","last_change":"2023-09-13T03:11:10.960198+0000","last_active":"2023-09-13T03:12:22.480661+0000","last_peered":"2023-09-13T03:12:22.480661+0000","last_clean":"2023-09-13T03:12:22.480661+0000","last_became_active":"2023-09-13T03:11:10.959839+0000","last_became_peered":"2023-09-13T03:11:10.959839+0000","last_unstale":"2023-09-13T03:12:22.480661+0000","last_undegraded":"2023-09-13T03:12:22.480661+0000","last_fullsized":"2023-09-13T03:12:22.480661+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:37:16.868093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.264967+0000","last_change":"2023-09-13T03:11:09.954684+0000","last_active":"2023-09-13T03:11:23.264967+0000","last_peered":"2023-09-13T03:11:23.264967+0000","last_clean":"2023-09-13T03:11:23.264967+0000","last_became_active":"2023-09-13T03:11:09.954392+0000","last_became_peered":"2023-09-13T03:11:09.954392+0000","last_unstale":"2023-09-13T03:11:23.264967+0000","last_undegraded":"2023-09-13T03:11:23.264967+0000","last_fullsized":"2023-09-13T03:11:23.264967+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:29:20.333063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.475916+0000","last_change":"2023-09-13T03:11:10.944364+0000","last_active":"2023-09-13T03:12:22.475916+0000","last_peered":"2023-09-13T03:12:22.475916+0000","last_clean":"2023-09-13T03:12:22.475916+0000","last_became_active":"2023-09-13T03:11:10.943971+0000","last_became_peered":"2023-09-13T03:11:10.943971+0000","last_unstale":"2023-09-13T03:12:22.475916+0000","last_undegraded":"2023-09-13T03:12:22.475916+0000","last_fullsized":"2023-09-13T03:12:22.475916+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:55:07.831397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.265196+0000","last_change":"2023-09-13T03:11:09.952751+0000","last_active":"2023-09-13T03:11:23.265196+0000","last_peered":"2023-09-13T03:11:23.265196+0000","last_clean":"2023-09-13T03:11:23.265196+0000","last_became_active":"2023-09-13T03:11:09.952447+0000","last_became_peered":"2023-09-13T03:11:09.952447+0000","last_unstale":"2023-09-13T03:11:23.265196+0000","last_undegraded":"2023-09-13T03:11:23.265196+0000","last_fullsized":"2023-09-13T03:11:23.265196+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:55:09.971742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.486746+0000","last_change":"2023-09-13T03:11:24.460373+0000","last_active":"2023-09-13T03:12:22.486746+0000","last_peered":"2023-09-13T03:12:22.486746+0000","last_clean":"2023-09-13T03:12:22.486746+0000","last_became_active":"2023-09-13T03:11:24.459994+0000","last_became_peered":"2023-09-13T03:11:24.459994+0000","last_unstale":"2023-09-13T03:12:22.486746+0000","last_undegraded":"2023-09-13T03:12:22.486746+0000","last_fullsized":"2023-09-13T03:12:22.486746+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:15:58.998213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'7","reported_seq":120,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:51.379310+0000","last_change":"2023-09-13T03:11:09.947207+0000","last_active":"2023-09-13T03:12:51.379310+0000","last_peered":"2023-09-13T03:12:51.379310+0000","last_clean":"2023-09-13T03:12:51.379310+0000","last_became_active":"2023-09-13T03:11:09.946787+0000","last_became_peered":"2023-09-13T03:11:09.946787+0000","last_unstale":"2023-09-13T03:12:51.379310+0000","last_undegraded":"2023-09-13T03:12:51.379310+0000","last_fullsized":"2023-09-13T03:12:51.379310+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:28:47.100723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1153,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.464131+0000","last_change":"2023-09-13T03:11:10.948427+0000","last_active":"2023-09-13T03:12:22.464131+0000","last_peered":"2023-09-13T03:12:22.464131+0000","last_clean":"2023-09-13T03:12:22.464131+0000","last_became_active":"2023-09-13T03:11:10.948095+0000","last_became_peered":"2023-09-13T03:11:10.948095+0000","last_unstale":"2023-09-13T03:12:22.464131+0000","last_undegraded":"2023-09-13T03:12:22.464131+0000","last_fullsized":"2023-09-13T03:12:22.464131+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:55:07.532772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.270733+0000","last_change":"2023-09-13T03:11:09.952335+0000","last_active":"2023-09-13T03:11:23.270733+0000","last_peered":"2023-09-13T03:11:23.270733+0000","last_clean":"2023-09-13T03:11:23.270733+0000","last_became_active":"2023-09-13T03:11:09.951974+0000","last_became_peered":"2023-09-13T03:11:09.951974+0000","last_unstale":"2023-09-13T03:11:23.270733+0000","last_undegraded":"2023-09-13T03:11:23.270733+0000","last_fullsized":"2023-09-13T03:11:23.270733+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:15:50.681751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.479139+0000","last_change":"2023-09-13T03:11:24.351298+0000","last_active":"2023-09-13T03:12:22.479139+0000","last_peered":"2023-09-13T03:12:22.479139+0000","last_clean":"2023-09-13T03:12:22.479139+0000","last_became_active":"2023-09-13T03:11:24.350758+0000","last_became_peered":"2023-09-13T03:11:24.350758+0000","last_unstale":"2023-09-13T03:12:22.479139+0000","last_undegraded":"2023-09-13T03:12:22.479139+0000","last_fullsized":"2023-09-13T03:12:22.479139+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:40:20.262641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.261290+0000","last_change":"2023-09-13T03:11:09.957644+0000","last_active":"2023-09-13T03:11:23.261290+0000","last_peered":"2023-09-13T03:11:23.261290+0000","last_clean":"2023-09-13T03:11:23.261290+0000","last_became_active":"2023-09-13T03:11:09.957289+0000","last_became_peered":"2023-09-13T03:11:09.957289+0000","last_unstale":"2023-09-13T03:11:23.261290+0000","last_undegraded":"2023-09-13T03:11:23.261290+0000","last_fullsized":"2023-09-13T03:11:23.261290+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:40:48.615322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.482645+0000","last_change":"2023-09-13T03:11:10.947561+0000","last_active":"2023-09-13T03:12:22.482645+0000","last_peered":"2023-09-13T03:12:22.482645+0000","last_clean":"2023-09-13T03:12:22.482645+0000","last_became_active":"2023-09-13T03:11:10.947238+0000","last_became_peered":"2023-09-13T03:11:10.947238+0000","last_unstale":"2023-09-13T03:12:22.482645+0000","last_undegraded":"2023-09-13T03:12:22.482645+0000","last_fullsized":"2023-09-13T03:12:22.482645+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:54:31.360657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.267100+0000","last_change":"2023-09-13T03:11:09.947806+0000","last_active":"2023-09-13T03:11:23.267100+0000","last_peered":"2023-09-13T03:11:23.267100+0000","last_clean":"2023-09-13T03:11:23.267100+0000","last_became_active":"2023-09-13T03:11:09.947428+0000","last_became_peered":"2023-09-13T03:11:09.947428+0000","last_unstale":"2023-09-13T03:11:23.267100+0000","last_undegraded":"2023-09-13T03:11:23.267100+0000","last_fullsized":"2023-09-13T03:11:23.267100+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:25:00.421873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.480287+0000","last_change":"2023-09-13T03:11:10.951637+0000","last_active":"2023-09-13T03:12:22.480287+0000","last_peered":"2023-09-13T03:12:22.480287+0000","last_clean":"2023-09-13T03:12:22.480287+0000","last_became_active":"2023-09-13T03:11:10.951249+0000","last_became_peered":"2023-09-13T03:11:10.951249+0000","last_unstale":"2023-09-13T03:12:22.480287+0000","last_undegraded":"2023-09-13T03:12:22.480287+0000","last_fullsized":"2023-09-13T03:12:22.480287+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:24:27.715167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'5","reported_seq":210,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:52.379887+0000","last_change":"2023-09-13T03:11:09.954343+0000","last_active":"2023-09-13T03:12:52.379887+0000","last_peered":"2023-09-13T03:12:52.379887+0000","last_clean":"2023-09-13T03:12:52.379887+0000","last_became_active":"2023-09-13T03:11:09.954063+0000","last_became_peered":"2023-09-13T03:11:09.954063+0000","last_unstale":"2023-09-13T03:12:52.379887+0000","last_undegraded":"2023-09-13T03:12:52.379887+0000","last_fullsized":"2023-09-13T03:12:52.379887+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:34:50.057539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":190,"num_read_kb":190,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.475149+0000","last_change":"2023-09-13T03:11:10.956994+0000","last_active":"2023-09-13T03:12:22.475149+0000","last_peered":"2023-09-13T03:12:22.475149+0000","last_clean":"2023-09-13T03:12:22.475149+0000","last_became_active":"2023-09-13T03:11:10.956336+0000","last_became_peered":"2023-09-13T03:11:10.956336+0000","last_unstale":"2023-09-13T03:12:22.475149+0000","last_undegraded":"2023-09-13T03:12:22.475149+0000","last_fullsized":"2023-09-13T03:12:22.475149+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:11:54.186143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.262358+0000","last_change":"2023-09-13T03:11:09.945775+0000","last_active":"2023-09-13T03:11:23.262358+0000","last_peered":"2023-09-13T03:11:23.262358+0000","last_clean":"2023-09-13T03:11:23.262358+0000","last_became_active":"2023-09-13T03:11:09.945406+0000","last_became_peered":"2023-09-13T03:11:09.945406+0000","last_unstale":"2023-09-13T03:11:23.262358+0000","last_undegraded":"2023-09-13T03:11:23.262358+0000","last_fullsized":"2023-09-13T03:11:23.262358+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:29:26.677351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.487156+0000","last_change":"2023-09-13T03:11:10.951941+0000","last_active":"2023-09-13T03:12:22.487156+0000","last_peered":"2023-09-13T03:12:22.487156+0000","last_clean":"2023-09-13T03:12:22.487156+0000","last_became_active":"2023-09-13T03:11:10.951331+0000","last_became_peered":"2023-09-13T03:11:10.951331+0000","last_unstale":"2023-09-13T03:12:22.487156+0000","last_undegraded":"2023-09-13T03:12:22.487156+0000","last_fullsized":"2023-09-13T03:12:22.487156+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:15:53.741657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.268129+0000","last_change":"2023-09-13T03:11:09.959503+0000","last_active":"2023-09-13T03:11:23.268129+0000","last_peered":"2023-09-13T03:11:23.268129+0000","last_clean":"2023-09-13T03:11:23.268129+0000","last_became_active":"2023-09-13T03:11:09.959166+0000","last_became_peered":"2023-09-13T03:11:09.959166+0000","last_unstale":"2023-09-13T03:11:23.268129+0000","last_undegraded":"2023-09-13T03:11:23.268129+0000","last_fullsized":"2023-09-13T03:11:23.268129+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:12:36.179520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.483877+0000","last_change":"2023-09-13T03:11:10.941575+0000","last_active":"2023-09-13T03:12:22.483877+0000","last_peered":"2023-09-13T03:12:22.483877+0000","last_clean":"2023-09-13T03:12:22.483877+0000","last_became_active":"2023-09-13T03:11:10.941285+0000","last_became_peered":"2023-09-13T03:11:10.941285+0000","last_unstale":"2023-09-13T03:12:22.483877+0000","last_undegraded":"2023-09-13T03:12:22.483877+0000","last_fullsized":"2023-09-13T03:12:22.483877+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:03:09.377471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'34","reported_seq":248,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:52.379878+0000","last_change":"2023-09-13T03:11:09.948041+0000","last_active":"2023-09-13T03:12:52.379878+0000","last_peered":"2023-09-13T03:12:52.379878+0000","last_clean":"2023-09-13T03:12:52.379878+0000","last_became_active":"2023-09-13T03:11:09.947785+0000","last_became_peered":"2023-09-13T03:11:09.947785+0000","last_unstale":"2023-09-13T03:12:52.379878+0000","last_undegraded":"2023-09-13T03:12:52.379878+0000","last_fullsized":"2023-09-13T03:12:52.379878+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:46:47.348307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58409,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":199,"num_read_kb":157,"num_write":36,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.460271+0000","last_change":"2023-09-13T03:11:10.950709+0000","last_active":"2023-09-13T03:12:22.460271+0000","last_peered":"2023-09-13T03:12:22.460271+0000","last_clean":"2023-09-13T03:12:22.460271+0000","last_became_active":"2023-09-13T03:11:10.950395+0000","last_became_peered":"2023-09-13T03:11:10.950395+0000","last_unstale":"2023-09-13T03:12:22.460271+0000","last_undegraded":"2023-09-13T03:12:22.460271+0000","last_fullsized":"2023-09-13T03:12:22.460271+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:50:08.628777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:11:24.281233+0000","last_change":"2023-09-13T03:11:24.281233+0000","last_active":"2023-09-13T03:11:24.281233+0000","last_peered":"2023-09-13T03:11:24.281233+0000","last_clean":"2023-09-13T03:11:24.281233+0000","last_became_active":"2023-09-13T03:11:24.269814+0000","last_became_peered":"2023-09-13T03:11:24.269814+0000","last_unstale":"2023-09-13T03:11:24.281233+0000","last_undegraded":"2023-09-13T03:11:24.281233+0000","last_fullsized":"2023-09-13T03:11:24.281233+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T10:09:00.743406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.482202+0000","last_change":"2023-09-13T03:11:10.955763+0000","last_active":"2023-09-13T03:12:22.482202+0000","last_peered":"2023-09-13T03:12:22.482202+0000","last_clean":"2023-09-13T03:12:22.482202+0000","last_became_active":"2023-09-13T03:11:10.955455+0000","last_became_peered":"2023-09-13T03:11:10.955455+0000","last_unstale":"2023-09-13T03:12:22.482202+0000","last_undegraded":"2023-09-13T03:12:22.482202+0000","last_fullsized":"2023-09-13T03:12:22.482202+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:43:37.127490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.267485+0000","last_change":"2023-09-13T03:11:09.947739+0000","last_active":"2023-09-13T03:11:23.267485+0000","last_peered":"2023-09-13T03:11:23.267485+0000","last_clean":"2023-09-13T03:11:23.267485+0000","last_became_active":"2023-09-13T03:11:09.947348+0000","last_became_peered":"2023-09-13T03:11:09.947348+0000","last_unstale":"2023-09-13T03:11:23.267485+0000","last_undegraded":"2023-09-13T03:11:23.267485+0000","last_fullsized":"2023-09-13T03:11:23.267485+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:47:58.408877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.473579+0000","last_change":"2023-09-13T03:11:10.958140+0000","last_active":"2023-09-13T03:12:22.473579+0000","last_peered":"2023-09-13T03:12:22.473579+0000","last_clean":"2023-09-13T03:12:22.473579+0000","last_became_active":"2023-09-13T03:11:10.957798+0000","last_became_peered":"2023-09-13T03:11:10.957798+0000","last_unstale":"2023-09-13T03:12:22.473579+0000","last_undegraded":"2023-09-13T03:12:22.473579+0000","last_fullsized":"2023-09-13T03:12:22.473579+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:44:40.208559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'7","reported_seq":119,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:50.376971+0000","last_change":"2023-09-13T03:11:09.945554+0000","last_active":"2023-09-13T03:12:50.376971+0000","last_peered":"2023-09-13T03:12:50.376971+0000","last_clean":"2023-09-13T03:12:50.376971+0000","last_became_active":"2023-09-13T03:11:09.945214+0000","last_became_peered":"2023-09-13T03:11:09.945214+0000","last_unstale":"2023-09-13T03:12:50.376971+0000","last_undegraded":"2023-09-13T03:12:50.376971+0000","last_fullsized":"2023-09-13T03:12:50.376971+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:46:12.338617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.470149+0000","last_change":"2023-09-13T03:11:10.952589+0000","last_active":"2023-09-13T03:12:22.470149+0000","last_peered":"2023-09-13T03:12:22.470149+0000","last_clean":"2023-09-13T03:12:22.470149+0000","last_became_active":"2023-09-13T03:11:10.952302+0000","last_became_peered":"2023-09-13T03:11:10.952302+0000","last_unstale":"2023-09-13T03:12:22.470149+0000","last_undegraded":"2023-09-13T03:12:22.470149+0000","last_fullsized":"2023-09-13T03:12:22.470149+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:13:46.972293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.264367+0000","last_change":"2023-09-13T03:11:09.959509+0000","last_active":"2023-09-13T03:11:23.264367+0000","last_peered":"2023-09-13T03:11:23.264367+0000","last_clean":"2023-09-13T03:11:23.264367+0000","last_became_active":"2023-09-13T03:11:09.959150+0000","last_became_peered":"2023-09-13T03:11:09.959150+0000","last_unstale":"2023-09-13T03:11:23.264367+0000","last_undegraded":"2023-09-13T03:11:23.264367+0000","last_fullsized":"2023-09-13T03:11:23.264367+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:33:30.648283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.476760+0000","last_change":"2023-09-13T03:11:10.956610+0000","last_active":"2023-09-13T03:12:22.476760+0000","last_peered":"2023-09-13T03:12:22.476760+0000","last_clean":"2023-09-13T03:12:22.476760+0000","last_became_active":"2023-09-13T03:11:10.955726+0000","last_became_peered":"2023-09-13T03:11:10.955726+0000","last_unstale":"2023-09-13T03:12:22.476760+0000","last_undegraded":"2023-09-13T03:12:22.476760+0000","last_fullsized":"2023-09-13T03:12:22.476760+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:55:45.667752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.264782+0000","last_change":"2023-09-13T03:11:09.949951+0000","last_active":"2023-09-13T03:11:23.264782+0000","last_peered":"2023-09-13T03:11:23.264782+0000","last_clean":"2023-09-13T03:11:23.264782+0000","last_became_active":"2023-09-13T03:11:09.949106+0000","last_became_peered":"2023-09-13T03:11:09.949106+0000","last_unstale":"2023-09-13T03:11:23.264782+0000","last_undegraded":"2023-09-13T03:11:23.264782+0000","last_fullsized":"2023-09-13T03:11:23.264782+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:30:30.092007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.471749+0000","last_change":"2023-09-13T03:11:10.942255+0000","last_active":"2023-09-13T03:12:22.471749+0000","last_peered":"2023-09-13T03:12:22.471749+0000","last_clean":"2023-09-13T03:12:22.471749+0000","last_became_active":"2023-09-13T03:11:10.941953+0000","last_became_peered":"2023-09-13T03:11:10.941953+0000","last_unstale":"2023-09-13T03:12:22.471749+0000","last_undegraded":"2023-09-13T03:12:22.471749+0000","last_fullsized":"2023-09-13T03:12:22.471749+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:38:55.260227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.262253+0000","last_change":"2023-09-13T03:11:09.956042+0000","last_active":"2023-09-13T03:11:23.262253+0000","last_peered":"2023-09-13T03:11:23.262253+0000","last_clean":"2023-09-13T03:11:23.262253+0000","last_became_active":"2023-09-13T03:11:09.955669+0000","last_became_peered":"2023-09-13T03:11:09.955669+0000","last_unstale":"2023-09-13T03:11:23.262253+0000","last_undegraded":"2023-09-13T03:11:23.262253+0000","last_fullsized":"2023-09-13T03:11:23.262253+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:16:25.227447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.482214+0000","last_change":"2023-09-13T03:11:10.952421+0000","last_active":"2023-09-13T03:12:22.482214+0000","last_peered":"2023-09-13T03:12:22.482214+0000","last_clean":"2023-09-13T03:12:22.482214+0000","last_became_active":"2023-09-13T03:11:10.951664+0000","last_became_peered":"2023-09-13T03:11:10.951664+0000","last_unstale":"2023-09-13T03:12:22.482214+0000","last_undegraded":"2023-09-13T03:12:22.482214+0000","last_fullsized":"2023-09-13T03:12:22.482214+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:46:52.664524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'5","reported_seq":113,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:48.373905+0000","last_change":"2023-09-13T03:11:09.952286+0000","last_active":"2023-09-13T03:12:48.373905+0000","last_peered":"2023-09-13T03:12:48.373905+0000","last_clean":"2023-09-13T03:12:48.373905+0000","last_became_active":"2023-09-13T03:11:09.951999+0000","last_became_peered":"2023-09-13T03:11:09.951999+0000","last_unstale":"2023-09-13T03:12:48.373905+0000","last_undegraded":"2023-09-13T03:12:48.373905+0000","last_fullsized":"2023-09-13T03:12:48.373905+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:54:46.910087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":93,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.485589+0000","last_change":"2023-09-13T03:11:10.950866+0000","last_active":"2023-09-13T03:12:22.485589+0000","last_peered":"2023-09-13T03:12:22.485589+0000","last_clean":"2023-09-13T03:12:22.485589+0000","last_became_active":"2023-09-13T03:11:10.950562+0000","last_became_peered":"2023-09-13T03:11:10.950562+0000","last_unstale":"2023-09-13T03:12:22.485589+0000","last_undegraded":"2023-09-13T03:12:22.485589+0000","last_fullsized":"2023-09-13T03:12:22.485589+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:55:07.333207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.476331+0000","last_change":"2023-09-13T03:11:10.950736+0000","last_active":"2023-09-13T03:12:22.476331+0000","last_peered":"2023-09-13T03:12:22.476331+0000","last_clean":"2023-09-13T03:12:22.476331+0000","last_became_active":"2023-09-13T03:11:10.950164+0000","last_became_peered":"2023-09-13T03:11:10.950164+0000","last_unstale":"2023-09-13T03:12:22.476331+0000","last_undegraded":"2023-09-13T03:12:22.476331+0000","last_fullsized":"2023-09-13T03:12:22.476331+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:22:44.453642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'6","reported_seq":118,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:52.379483+0000","last_change":"2023-09-13T03:11:09.954070+0000","last_active":"2023-09-13T03:12:52.379483+0000","last_peered":"2023-09-13T03:12:52.379483+0000","last_clean":"2023-09-13T03:12:52.379483+0000","last_became_active":"2023-09-13T03:11:09.953735+0000","last_became_peered":"2023-09-13T03:11:09.953735+0000","last_unstale":"2023-09-13T03:12:52.379483+0000","last_undegraded":"2023-09-13T03:12:52.379483+0000","last_fullsized":"2023-09-13T03:12:52.379483+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:36:59.891449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.486035+0000","last_change":"2023-09-13T03:11:10.948509+0000","last_active":"2023-09-13T03:12:22.486035+0000","last_peered":"2023-09-13T03:12:22.486035+0000","last_clean":"2023-09-13T03:12:22.486035+0000","last_became_active":"2023-09-13T03:11:10.948149+0000","last_became_peered":"2023-09-13T03:11:10.948149+0000","last_unstale":"2023-09-13T03:12:22.486035+0000","last_undegraded":"2023-09-13T03:12:22.486035+0000","last_fullsized":"2023-09-13T03:12:22.486035+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:45:54.197368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.265055+0000","last_change":"2023-09-13T03:11:09.954476+0000","last_active":"2023-09-13T03:11:23.265055+0000","last_peered":"2023-09-13T03:11:23.265055+0000","last_clean":"2023-09-13T03:11:23.265055+0000","last_became_active":"2023-09-13T03:11:09.954094+0000","last_became_peered":"2023-09-13T03:11:09.954094+0000","last_unstale":"2023-09-13T03:11:23.265055+0000","last_undegraded":"2023-09-13T03:11:23.265055+0000","last_fullsized":"2023-09-13T03:11:23.265055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:08:23.815751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":320,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.261991+0000","last_change":"2023-09-13T03:10:05.689061+0000","last_active":"2023-09-13T03:11:23.261991+0000","last_peered":"2023-09-13T03:11:23.261991+0000","last_clean":"2023-09-13T03:11:23.261991+0000","last_became_active":"2023-09-13T03:10:05.688557+0000","last_became_peered":"2023-09-13T03:10:05.688557+0000","last_unstale":"2023-09-13T03:11:23.261991+0000","last_undegraded":"2023-09-13T03:11:23.261991+0000","last_fullsized":"2023-09-13T03:11:23.261991+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:06:59.216900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:06:59.216900+0000","last_clean_scrub_stamp":"2023-09-13T03:06:59.216900+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:10:03.279019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.468705+0000","last_change":"2023-09-13T03:11:10.952523+0000","last_active":"2023-09-13T03:12:22.468705+0000","last_peered":"2023-09-13T03:12:22.468705+0000","last_clean":"2023-09-13T03:12:22.468705+0000","last_became_active":"2023-09-13T03:11:10.952193+0000","last_became_peered":"2023-09-13T03:11:10.952193+0000","last_unstale":"2023-09-13T03:12:22.468705+0000","last_undegraded":"2023-09-13T03:12:22.468705+0000","last_fullsized":"2023-09-13T03:12:22.468705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:31:01.811841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.262105+0000","last_change":"2023-09-13T03:11:09.950414+0000","last_active":"2023-09-13T03:11:23.262105+0000","last_peered":"2023-09-13T03:11:23.262105+0000","last_clean":"2023-09-13T03:11:23.262105+0000","last_became_active":"2023-09-13T03:11:09.950057+0000","last_became_peered":"2023-09-13T03:11:09.950057+0000","last_unstale":"2023-09-13T03:11:23.262105+0000","last_undegraded":"2023-09-13T03:11:23.262105+0000","last_fullsized":"2023-09-13T03:11:23.262105+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:57:01.317880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.264838+0000","last_change":"2023-09-13T03:11:09.955535+0000","last_active":"2023-09-13T03:11:23.264838+0000","last_peered":"2023-09-13T03:11:23.264838+0000","last_clean":"2023-09-13T03:11:23.264838+0000","last_became_active":"2023-09-13T03:11:09.955212+0000","last_became_peered":"2023-09-13T03:11:09.955212+0000","last_unstale":"2023-09-13T03:11:23.264838+0000","last_undegraded":"2023-09-13T03:11:23.264838+0000","last_fullsized":"2023-09-13T03:11:23.264838+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:15:04.365337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.476395+0000","last_change":"2023-09-13T03:11:10.954255+0000","last_active":"2023-09-13T03:12:22.476395+0000","last_peered":"2023-09-13T03:12:22.476395+0000","last_clean":"2023-09-13T03:12:22.476395+0000","last_became_active":"2023-09-13T03:11:10.953866+0000","last_became_peered":"2023-09-13T03:11:10.953866+0000","last_unstale":"2023-09-13T03:12:22.476395+0000","last_undegraded":"2023-09-13T03:12:22.476395+0000","last_fullsized":"2023-09-13T03:12:22.476395+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:25:12.281218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'11","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:11:29.021219+0000","last_change":"2023-09-13T03:11:24.285884+0000","last_active":"2023-09-13T03:11:29.021219+0000","last_peered":"2023-09-13T03:11:29.021219+0000","last_clean":"2023-09-13T03:11:29.021219+0000","last_became_active":"2023-09-13T03:11:24.269933+0000","last_became_peered":"2023-09-13T03:11:24.269933+0000","last_unstale":"2023-09-13T03:11:29.021219+0000","last_undegraded":"2023-09-13T03:11:29.021219+0000","last_fullsized":"2023-09-13T03:11:29.021219+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:30:13.957854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.480142+0000","last_change":"2023-09-13T03:11:10.949037+0000","last_active":"2023-09-13T03:12:22.480142+0000","last_peered":"2023-09-13T03:12:22.480142+0000","last_clean":"2023-09-13T03:12:22.480142+0000","last_became_active":"2023-09-13T03:11:10.948641+0000","last_became_peered":"2023-09-13T03:11:10.948641+0000","last_unstale":"2023-09-13T03:12:22.480142+0000","last_undegraded":"2023-09-13T03:12:22.480142+0000","last_fullsized":"2023-09-13T03:12:22.480142+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:00:57.563300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.265201+0000","last_change":"2023-09-13T03:11:09.946435+0000","last_active":"2023-09-13T03:11:23.265201+0000","last_peered":"2023-09-13T03:11:23.265201+0000","last_clean":"2023-09-13T03:11:23.265201+0000","last_became_active":"2023-09-13T03:11:09.946176+0000","last_became_peered":"2023-09-13T03:11:09.946176+0000","last_unstale":"2023-09-13T03:11:23.265201+0000","last_undegraded":"2023-09-13T03:11:23.265201+0000","last_fullsized":"2023-09-13T03:11:23.265201+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T13:34:16.055188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.487824+0000","last_change":"2023-09-13T03:11:10.957035+0000","last_active":"2023-09-13T03:12:22.487824+0000","last_peered":"2023-09-13T03:12:22.487824+0000","last_clean":"2023-09-13T03:12:22.487824+0000","last_became_active":"2023-09-13T03:11:10.954656+0000","last_became_peered":"2023-09-13T03:11:10.954656+0000","last_unstale":"2023-09-13T03:12:22.487824+0000","last_undegraded":"2023-09-13T03:12:22.487824+0000","last_fullsized":"2023-09-13T03:12:22.487824+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:54:51.142734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.269239+0000","last_change":"2023-09-13T03:11:09.959358+0000","last_active":"2023-09-13T03:11:23.269239+0000","last_peered":"2023-09-13T03:11:23.269239+0000","last_clean":"2023-09-13T03:11:23.269239+0000","last_became_active":"2023-09-13T03:11:09.959129+0000","last_became_peered":"2023-09-13T03:11:09.959129+0000","last_unstale":"2023-09-13T03:11:23.269239+0000","last_undegraded":"2023-09-13T03:11:23.269239+0000","last_fullsized":"2023-09-13T03:11:23.269239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:29:26.930910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.444904+0000","last_change":"2023-09-13T03:11:10.952588+0000","last_active":"2023-09-13T03:12:22.444904+0000","last_peered":"2023-09-13T03:12:22.444904+0000","last_clean":"2023-09-13T03:12:22.444904+0000","last_became_active":"2023-09-13T03:11:10.952304+0000","last_became_peered":"2023-09-13T03:11:10.952304+0000","last_unstale":"2023-09-13T03:12:22.444904+0000","last_undegraded":"2023-09-13T03:12:22.444904+0000","last_fullsized":"2023-09-13T03:12:22.444904+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:05:49.824053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.266834+0000","last_change":"2023-09-13T03:11:09.956947+0000","last_active":"2023-09-13T03:11:23.266834+0000","last_peered":"2023-09-13T03:11:23.266834+0000","last_clean":"2023-09-13T03:11:23.266834+0000","last_became_active":"2023-09-13T03:11:09.956639+0000","last_became_peered":"2023-09-13T03:11:09.956639+0000","last_unstale":"2023-09-13T03:11:23.266834+0000","last_undegraded":"2023-09-13T03:11:23.266834+0000","last_fullsized":"2023-09-13T03:11:23.266834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:21:37.759130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.480248+0000","last_change":"2023-09-13T03:11:10.947314+0000","last_active":"2023-09-13T03:12:22.480248+0000","last_peered":"2023-09-13T03:12:22.480248+0000","last_clean":"2023-09-13T03:12:22.480248+0000","last_became_active":"2023-09-13T03:11:10.946964+0000","last_became_peered":"2023-09-13T03:11:10.946964+0000","last_unstale":"2023-09-13T03:12:22.480248+0000","last_undegraded":"2023-09-13T03:12:22.480248+0000","last_fullsized":"2023-09-13T03:12:22.480248+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:55:08.542303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.268605+0000","last_change":"2023-09-13T03:11:09.952540+0000","last_active":"2023-09-13T03:11:23.268605+0000","last_peered":"2023-09-13T03:11:23.268605+0000","last_clean":"2023-09-13T03:11:23.268605+0000","last_became_active":"2023-09-13T03:11:09.952208+0000","last_became_peered":"2023-09-13T03:11:09.952208+0000","last_unstale":"2023-09-13T03:11:23.268605+0000","last_undegraded":"2023-09-13T03:11:23.268605+0000","last_fullsized":"2023-09-13T03:11:23.268605+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:11:41.200401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.482930+0000","last_change":"2023-09-13T03:11:10.951305+0000","last_active":"2023-09-13T03:12:22.482930+0000","last_peered":"2023-09-13T03:12:22.482930+0000","last_clean":"2023-09-13T03:12:22.482930+0000","last_became_active":"2023-09-13T03:11:10.950982+0000","last_became_peered":"2023-09-13T03:11:10.950982+0000","last_unstale":"2023-09-13T03:12:22.482930+0000","last_undegraded":"2023-09-13T03:12:22.482930+0000","last_fullsized":"2023-09-13T03:12:22.482930+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:55:04.497698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:11:24.355955+0000","last_change":"2023-09-13T03:11:24.355955+0000","last_active":"2023-09-13T03:11:24.355955+0000","last_peered":"2023-09-13T03:11:24.355955+0000","last_clean":"2023-09-13T03:11:24.355955+0000","last_became_active":"2023-09-13T03:11:24.351069+0000","last_became_peered":"2023-09-13T03:11:24.351069+0000","last_unstale":"2023-09-13T03:11:24.355955+0000","last_undegraded":"2023-09-13T03:11:24.355955+0000","last_fullsized":"2023-09-13T03:11:24.355955+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:57:55.311708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.479112+0000","last_change":"2023-09-13T03:11:10.959487+0000","last_active":"2023-09-13T03:12:22.479112+0000","last_peered":"2023-09-13T03:12:22.479112+0000","last_clean":"2023-09-13T03:12:22.479112+0000","last_became_active":"2023-09-13T03:11:10.959157+0000","last_became_peered":"2023-09-13T03:11:10.959157+0000","last_unstale":"2023-09-13T03:12:22.479112+0000","last_undegraded":"2023-09-13T03:12:22.479112+0000","last_fullsized":"2023-09-13T03:12:22.479112+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:37:50.852197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'5","reported_seq":118,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:52.380460+0000","last_change":"2023-09-13T03:11:09.951208+0000","last_active":"2023-09-13T03:12:52.380460+0000","last_peered":"2023-09-13T03:12:52.380460+0000","last_clean":"2023-09-13T03:12:52.380460+0000","last_became_active":"2023-09-13T03:11:09.950869+0000","last_became_peered":"2023-09-13T03:11:09.950869+0000","last_unstale":"2023-09-13T03:12:52.380460+0000","last_undegraded":"2023-09-13T03:12:52.380460+0000","last_fullsized":"2023-09-13T03:12:52.380460+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T14:22:53.697803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.482981+0000","last_change":"2023-09-13T03:11:10.959129+0000","last_active":"2023-09-13T03:12:22.482981+0000","last_peered":"2023-09-13T03:12:22.482981+0000","last_clean":"2023-09-13T03:12:22.482981+0000","last_became_active":"2023-09-13T03:11:10.958820+0000","last_became_peered":"2023-09-13T03:11:10.958820+0000","last_unstale":"2023-09-13T03:12:22.482981+0000","last_undegraded":"2023-09-13T03:12:22.482981+0000","last_fullsized":"2023-09-13T03:12:22.482981+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:20:30.607854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.267998+0000","last_change":"2023-09-13T03:11:09.946855+0000","last_active":"2023-09-13T03:11:23.267998+0000","last_peered":"2023-09-13T03:11:23.267998+0000","last_clean":"2023-09-13T03:11:23.267998+0000","last_became_active":"2023-09-13T03:11:09.946156+0000","last_became_peered":"2023-09-13T03:11:09.946156+0000","last_unstale":"2023-09-13T03:11:23.267998+0000","last_undegraded":"2023-09-13T03:11:23.267998+0000","last_fullsized":"2023-09-13T03:11:23.267998+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:42:19.387280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.479096+0000","last_change":"2023-09-13T03:11:10.951335+0000","last_active":"2023-09-13T03:12:22.479096+0000","last_peered":"2023-09-13T03:12:22.479096+0000","last_clean":"2023-09-13T03:12:22.479096+0000","last_became_active":"2023-09-13T03:11:10.951034+0000","last_became_peered":"2023-09-13T03:11:10.951034+0000","last_unstale":"2023-09-13T03:12:22.479096+0000","last_undegraded":"2023-09-13T03:12:22.479096+0000","last_fullsized":"2023-09-13T03:12:22.479096+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T05:57:24.373196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-13T03:11:23.263077+0000","last_change":"2023-09-13T03:11:09.950937+0000","last_active":"2023-09-13T03:11:23.263077+0000","last_peered":"2023-09-13T03:11:23.263077+0000","last_clean":"2023-09-13T03:11:23.263077+0000","last_became_active":"2023-09-13T03:11:09.950574+0000","last_became_peered":"2023-09-13T03:11:09.950574+0000","last_unstale":"2023-09-13T03:11:23.263077+0000","last_undegraded":"2023-09-13T03:11:23.263077+0000","last_fullsized":"2023-09-13T03:11:23.263077+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:08.899260+0000","last_clean_scrub_stamp":"2023-09-13T03:11:08.899260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:10:23.215080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.477392+0000","last_change":"2023-09-13T03:11:10.950517+0000","last_active":"2023-09-13T03:12:22.477392+0000","last_peered":"2023-09-13T03:12:22.477392+0000","last_clean":"2023-09-13T03:12:22.477392+0000","last_became_active":"2023-09-13T03:11:10.950028+0000","last_became_peered":"2023-09-13T03:11:10.950028+0000","last_unstale":"2023-09-13T03:12:22.477392+0000","last_undegraded":"2023-09-13T03:12:22.477392+0000","last_fullsized":"2023-09-13T03:12:22.477392+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:06:40.008527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.473384+0000","last_change":"2023-09-13T03:11:10.942192+0000","last_active":"2023-09-13T03:12:22.473384+0000","last_peered":"2023-09-13T03:12:22.473384+0000","last_clean":"2023-09-13T03:12:22.473384+0000","last_became_active":"2023-09-13T03:11:10.941893+0000","last_became_peered":"2023-09-13T03:11:10.941893+0000","last_unstale":"2023-09-13T03:12:22.473384+0000","last_undegraded":"2023-09-13T03:12:22.473384+0000","last_fullsized":"2023-09-13T03:12:22.473384+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:26:22.064843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.470613+0000","last_change":"2023-09-13T03:11:10.947186+0000","last_active":"2023-09-13T03:12:22.470613+0000","last_peered":"2023-09-13T03:12:22.470613+0000","last_clean":"2023-09-13T03:12:22.470613+0000","last_became_active":"2023-09-13T03:11:10.946902+0000","last_became_peered":"2023-09-13T03:11:10.946902+0000","last_unstale":"2023-09-13T03:12:22.470613+0000","last_undegraded":"2023-09-13T03:12:22.470613+0000","last_fullsized":"2023-09-13T03:12:22.470613+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:26:01.325597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.476365+0000","last_change":"2023-09-13T03:11:10.946504+0000","last_active":"2023-09-13T03:12:22.476365+0000","last_peered":"2023-09-13T03:12:22.476365+0000","last_clean":"2023-09-13T03:12:22.476365+0000","last_became_active":"2023-09-13T03:11:10.946196+0000","last_became_peered":"2023-09-13T03:11:10.946196+0000","last_unstale":"2023-09-13T03:12:22.476365+0000","last_undegraded":"2023-09-13T03:12:22.476365+0000","last_fullsized":"2023-09-13T03:12:22.476365+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T11:37:43.578685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.487305+0000","last_change":"2023-09-13T03:11:10.959637+0000","last_active":"2023-09-13T03:12:22.487305+0000","last_peered":"2023-09-13T03:12:22.487305+0000","last_clean":"2023-09-13T03:12:22.487305+0000","last_became_active":"2023-09-13T03:11:10.959324+0000","last_became_peered":"2023-09-13T03:11:10.959324+0000","last_unstale":"2023-09-13T03:12:22.487305+0000","last_undegraded":"2023-09-13T03:12:22.487305+0000","last_fullsized":"2023-09-13T03:12:22.487305+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:48:19.245785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.476762+0000","last_change":"2023-09-13T03:11:24.576318+0000","last_active":"2023-09-13T03:12:22.476762+0000","last_peered":"2023-09-13T03:12:22.476762+0000","last_clean":"2023-09-13T03:12:22.476762+0000","last_became_active":"2023-09-13T03:11:24.575775+0000","last_became_peered":"2023-09-13T03:11:24.575775+0000","last_unstale":"2023-09-13T03:12:22.476762+0000","last_undegraded":"2023-09-13T03:12:22.476762+0000","last_fullsized":"2023-09-13T03:12:22.476762+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T06:16:00.534554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.472987+0000","last_change":"2023-09-13T03:11:10.945076+0000","last_active":"2023-09-13T03:12:22.472987+0000","last_peered":"2023-09-13T03:12:22.472987+0000","last_clean":"2023-09-13T03:12:22.472987+0000","last_became_active":"2023-09-13T03:11:10.944776+0000","last_became_peered":"2023-09-13T03:11:10.944776+0000","last_unstale":"2023-09-13T03:12:22.472987+0000","last_undegraded":"2023-09-13T03:12:22.472987+0000","last_fullsized":"2023-09-13T03:12:22.472987+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T12:22:03.624488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.475887+0000","last_change":"2023-09-13T03:11:10.942663+0000","last_active":"2023-09-13T03:12:22.475887+0000","last_peered":"2023-09-13T03:12:22.475887+0000","last_clean":"2023-09-13T03:12:22.475887+0000","last_became_active":"2023-09-13T03:11:10.942203+0000","last_became_peered":"2023-09-13T03:11:10.942203+0000","last_unstale":"2023-09-13T03:12:22.475887+0000","last_undegraded":"2023-09-13T03:12:22.475887+0000","last_fullsized":"2023-09-13T03:12:22.475887+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T04:46:48.091425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.469990+0000","last_change":"2023-09-13T03:11:10.940089+0000","last_active":"2023-09-13T03:12:22.469990+0000","last_peered":"2023-09-13T03:12:22.469990+0000","last_clean":"2023-09-13T03:12:22.469990+0000","last_became_active":"2023-09-13T03:11:10.939622+0000","last_became_peered":"2023-09-13T03:11:10.939622+0000","last_unstale":"2023-09-13T03:12:22.469990+0000","last_undegraded":"2023-09-13T03:12:22.469990+0000","last_fullsized":"2023-09-13T03:12:22.469990+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:43:55.024944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.477959+0000","last_change":"2023-09-13T03:11:10.956896+0000","last_active":"2023-09-13T03:12:22.477959+0000","last_peered":"2023-09-13T03:12:22.477959+0000","last_clean":"2023-09-13T03:12:22.477959+0000","last_became_active":"2023-09-13T03:11:10.956624+0000","last_became_peered":"2023-09-13T03:11:10.956624+0000","last_unstale":"2023-09-13T03:12:22.477959+0000","last_undegraded":"2023-09-13T03:12:22.477959+0000","last_fullsized":"2023-09-13T03:12:22.477959+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T08:29:41.305999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.479538+0000","last_change":"2023-09-13T03:11:10.939240+0000","last_active":"2023-09-13T03:12:22.479538+0000","last_peered":"2023-09-13T03:12:22.479538+0000","last_clean":"2023-09-13T03:12:22.479538+0000","last_became_active":"2023-09-13T03:11:10.938869+0000","last_became_peered":"2023-09-13T03:11:10.938869+0000","last_unstale":"2023-09-13T03:12:22.479538+0000","last_undegraded":"2023-09-13T03:12:22.479538+0000","last_fullsized":"2023-09-13T03:12:22.479538+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T15:06:44.931165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.483365+0000","last_change":"2023-09-13T03:11:10.956704+0000","last_active":"2023-09-13T03:12:22.483365+0000","last_peered":"2023-09-13T03:12:22.483365+0000","last_clean":"2023-09-13T03:12:22.483365+0000","last_became_active":"2023-09-13T03:11:10.956413+0000","last_became_peered":"2023-09-13T03:11:10.956413+0000","last_unstale":"2023-09-13T03:12:22.483365+0000","last_undegraded":"2023-09-13T03:12:22.483365+0000","last_fullsized":"2023-09-13T03:12:22.483365+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T07:50:04.626111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.475930+0000","last_change":"2023-09-13T03:11:10.952707+0000","last_active":"2023-09-13T03:12:22.475930+0000","last_peered":"2023-09-13T03:12:22.475930+0000","last_clean":"2023-09-13T03:12:22.475930+0000","last_became_active":"2023-09-13T03:11:10.952423+0000","last_became_peered":"2023-09-13T03:11:10.952423+0000","last_unstale":"2023-09-13T03:12:22.475930+0000","last_undegraded":"2023-09-13T03:12:22.475930+0000","last_fullsized":"2023-09-13T03:12:22.475930+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T03:35:09.869172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-13T03:12:22.474718+0000","last_change":"2023-09-13T03:11:10.941096+0000","last_active":"2023-09-13T03:12:22.474718+0000","last_peered":"2023-09-13T03:12:22.474718+0000","last_clean":"2023-09-13T03:12:22.474718+0000","last_became_active":"2023-09-13T03:11:10.940761+0000","last_became_peered":"2023-09-13T03:11:10.940761+0000","last_unstale":"2023-09-13T03:12:22.474718+0000","last_undegraded":"2023-09-13T03:12:22.474718+0000","last_fullsized":"2023-09-13T03:12:22.474718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-13T03:11:09.925262+0000","last_clean_scrub_stamp":"2023-09-13T03:11:09.925262+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-14T09:34:27.389650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65893,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":984,"num_read_kb":565,"num_write":245,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":68,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":573440,"data_stored":206951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449121,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":444288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:12:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.59999999999999998,"15min":0.60599999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.86099999999999999,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.58099999999999996,"15min":0.59199999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.79300000000000004,"5min":1.018,"15min":1.018},"last":0.44500000000000001}]},{"osd":1,"last update":"Wed Sep 13 03:12:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.60199999999999998,"15min":0.58399999999999996},"min":{"1min":0.55300000000000005,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.57499999999999996,"15min":0.56499999999999995},"min":{"1min":0.48299999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.68700000000000006}]},{"osd":2,"last update":"Wed Sep 13 03:12:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.45400000000000001,"15min":0.439},"min":{"1min":0.35799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.504,"15min":0.502},"min":{"1min":0.35799999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.77000000000000002,"5min":0.879,"15min":0.879},"last":0.60399999999999998}]},{"osd":3,"last update":"Wed Sep 13 03:12:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":6.415,"5min":6.415,"15min":6.415},"last":0.54000000000000004}]},{"osd":4,"last update":"Wed Sep 13 03:12:29 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.63300000000000001,"15min":0.61399999999999999},"min":{"1min":0.55100000000000005,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.61299999999999999,"15min":0.60499999999999998},"min":{"1min":0.56000000000000005,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.629}]},{"osd":5,"last update":"Wed Sep 13 03:12:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.55400000000000005,"15min":0.55000000000000004},"min":{"1min":0.36899999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79700000000000004,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.65900000000000003,"15min":0.64500000000000002},"min":{"1min":0.46300000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.70799999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":7,"last update":"Wed Sep 13 03:12:29 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.71999999999999997,"15min":0.70099999999999996},"min":{"1min":0.58099999999999996,"5min":0.497,"15min":0.497},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.71699999999999997,"15min":0.71499999999999997},"min":{"1min":0.497,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.74199999999999999}]},{"osd":8,"last update":"Wed Sep 13 03:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.75}]},{"osd":9,"last update":"Wed Sep 13 03:12:29 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.80500000000000005,"15min":0.77700000000000002},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":4.3049999999999997,"5min":4.3049999999999997,"15min":4.3049999999999997},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.82899999999999996,"15min":0.81799999999999995},"min":{"1min":0.68799999999999994,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.88800000000000001}]},{"osd":10,"last update":"Wed Sep 13 03:12:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.72399999999999998,"15min":0.71799999999999997},"min":{"1min":0.58399999999999996,"5min":0.499,"15min":0.499},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.71799999999999997,"15min":0.70599999999999996},"min":{"1min":0.64300000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.72899999999999998}]}]},{"osd":10,"up_from":71,"seq":304942678052,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1626112,"data_stored":1094195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.61399999999999999}]},{"osd":1,"last update":"Wed Sep 13 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.497,"15min":0.502},"min":{"1min":0.33800000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.71999999999999997,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.498,"15min":0.501},"min":{"1min":0.30099999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.82599999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.55700000000000005}]},{"osd":2,"last update":"Wed Sep 13 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.66500000000000004,"15min":0.67700000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81000000000000005,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.66700000000000004,"15min":0.67800000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.80400000000000005,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.67700000000000005}]},{"osd":3,"last update":"Wed Sep 13 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.66200000000000003,"15min":0.67200000000000004},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.81299999999999994,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.68500000000000005,"15min":0.69399999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.84599999999999997,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.60099999999999998}]},{"osd":4,"last update":"Wed Sep 13 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.61499999999999999,"15min":0.61899999999999999},"min":{"1min":0.376,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78300000000000003,"5min":1.03,"15min":1.03},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.39000000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80400000000000005,"5min":1.173,"15min":1.173},"last":0.52500000000000002}]},{"osd":5,"last update":"Wed Sep 13 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.78400000000000003,"15min":0.80400000000000005},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.92300000000000004,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.79600000000000004,"15min":0.81599999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.88200000000000001,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.73699999999999999}]},{"osd":6,"last update":"Wed Sep 13 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.82799999999999996,"15min":0.84799999999999998},"min":{"1min":0.53800000000000003,"5min":0.498,"15min":0.498},"max":{"1min":0.94899999999999995,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.63},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.82999999999999996,"15min":0.83999999999999997},"min":{"1min":0.56000000000000005,"5min":0.499,"15min":0.499},"max":{"1min":1.0429999999999999,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":0.70899999999999996}]},{"osd":7,"last update":"Wed Sep 13 03:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.47599999999999998}]},{"osd":8,"last update":"Wed Sep 13 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.84199999999999997,"15min":0.85999999999999999},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.88100000000000001,"5min":1.653,"15min":1.653},"last":0.752},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.879,"15min":0.90500000000000003},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.91600000000000004,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.72399999999999998}]},{"osd":9,"last update":"Wed Sep 13 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.88200000000000001,"15min":0.89700000000000002},"min":{"1min":0.61499999999999999,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.016,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.90300000000000002,"15min":0.92300000000000004},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.96899999999999997,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.65100000000000002}]},{"osd":11,"last update":"Wed Sep 13 03:12:24 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.81599999999999995,"15min":0.81200000000000006},"min":{"1min":0.70099999999999996,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":0.98899999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81999999999999995,"15min":0.82099999999999995},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.004,"5min":1.008,"15min":1.008},"last":0.81299999999999994}]}]},{"osd":9,"up_from":66,"seq":283467841577,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":444150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.57699999999999996,"15min":0.60599999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.56899999999999995,"15min":0.58899999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72499999999999998,"5min":0.875,"15min":0.875},"last":0.437}]},{"osd":1,"last update":"Wed Sep 13 03:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.64700000000000002,"15min":0.65900000000000003},"min":{"1min":0.44400000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.84999999999999998,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.70999999999999996,"15min":0.748},"min":{"1min":0.46700000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.82299999999999995,"5min":1.107,"15min":1.107},"last":0.56299999999999994}]},{"osd":2,"last update":"Wed Sep 13 03:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.71099999999999997,"15min":0.73099999999999998},"min":{"1min":0.45600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.123,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.752,"15min":0.78100000000000003},"min":{"1min":0.56200000000000006,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.62}]},{"osd":3,"last update":"Wed Sep 13 03:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.67000000000000004,"15min":0.70799999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.85099999999999998,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.71399999999999997,"15min":0.76400000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.98299999999999998,"5min":1.238,"15min":1.238},"last":0.49299999999999999}]},{"osd":4,"last update":"Wed Sep 13 03:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.79600000000000004,"15min":0.84199999999999997},"min":{"1min":0.442,"5min":0.438,"15min":0.438},"max":{"1min":0.93500000000000005,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.77000000000000002,"15min":0.80400000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.876,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.67000000000000004}]},{"osd":5,"last update":"Wed Sep 13 03:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.81799999999999995,"15min":0.85699999999999998},"min":{"1min":0.52000000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.95399999999999996,"5min":1.486,"15min":1.486},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.94799999999999995,"15min":1.034},"min":{"1min":0.59599999999999997,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.0800000000000001,"5min":4.2590000000000003,"15min":4.2590000000000003},"last":0.82499999999999996}]},{"osd":6,"last update":"Wed Sep 13 03:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.67600000000000005}]},{"osd":7,"last update":"Wed Sep 13 03:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.98699999999999999,"15min":1.077},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.252,"5min":4.367,"15min":4.367},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.752,"5min":1.0049999999999999,"15min":1.0960000000000001},"min":{"1min":0.60099999999999998,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":0.97199999999999998,"5min":4.3520000000000003,"15min":4.3520000000000003},"last":0.59599999999999997}]},{"osd":8,"last update":"Wed Sep 13 03:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.92600000000000005,"15min":0.98199999999999998},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.1830000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.82999999999999996,"5min":1.036,"15min":1.1220000000000001},"min":{"1min":0.57199999999999995,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.2,"5min":4.298,"15min":4.298},"last":0.85899999999999999}]},{"osd":10,"last update":"Wed Sep 13 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.95899999999999996,"15min":0.97899999999999998},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0900000000000001,"5min":1.321,"15min":1.321},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.93400000000000005,"15min":0.94299999999999995},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.2929999999999999,"5min":1.306,"15min":1.306},"last":0.80800000000000005}]},{"osd":11,"last update":"Wed Sep 13 03:12:23 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.83799999999999997}]}]},{"osd":8,"up_from":58,"seq":249108103213,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":444197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57899999999999996,"15min":0.58499999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.57799999999999996,"15min":0.57999999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.86099999999999999,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.57499999999999996}]},{"osd":1,"last update":"Wed Sep 13 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.61299999999999999,"15min":0.59899999999999998},"min":{"1min":0.48899999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.59899999999999998,"15min":0.59199999999999997},"min":{"1min":0.45400000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.81999999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.71399999999999997}]},{"osd":2,"last update":"Wed Sep 13 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.47599999999999998,"15min":0.46999999999999997},"min":{"1min":0.39600000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.48599999999999999,"15min":0.48199999999999998},"min":{"1min":0.34799999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.72799999999999998,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.52400000000000002}]},{"osd":3,"last update":"Wed Sep 13 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.65000000000000002,"15min":0.64300000000000002},"min":{"1min":0.46999999999999997,"5min":0.44,"15min":0.44},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.504},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.65000000000000002,"15min":0.65900000000000003},"min":{"1min":0.45000000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.87,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.54800000000000004}]},{"osd":4,"last update":"Wed Sep 13 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.63800000000000001,"15min":0.61699999999999999},"min":{"1min":0.48399999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.75,"5min":0.66300000000000003,"15min":0.65400000000000003},"min":{"1min":0.43099999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.83499999999999996}]},{"osd":5,"last update":"Wed Sep 13 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.57599999999999996,"15min":0.56599999999999995},"min":{"1min":0.40000000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.96999999999999997,"5min":0.995,"15min":0.995},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.61699999999999999,"15min":0.61799999999999999},"min":{"1min":0.38300000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.46899999999999997}]},{"osd":6,"last update":"Wed Sep 13 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.73599999999999999,"15min":0.747},"min":{"1min":0.51800000000000002,"5min":0.44,"15min":0.44},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.54600000000000004,"5min":0.501,"15min":0.501},"max":{"1min":1.103,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.70199999999999996}]},{"osd":7,"last update":"Wed Sep 13 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.71099999999999997,"15min":0.70599999999999996},"min":{"1min":0.52000000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91500000000000004,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.749,"15min":0.72599999999999998},"min":{"1min":0.64900000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.88}]},{"osd":9,"last update":"Wed Sep 13 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.78800000000000003,"15min":0.78100000000000003},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.0329999999999999,"5min":1.21,"15min":1.21},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.77100000000000002,"15min":0.76300000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0289999999999999,"5min":1.081,"15min":1.081},"last":0.68700000000000006}]},{"osd":10,"last update":"Wed Sep 13 03:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.73999999999999999,"15min":0.71399999999999997},"min":{"1min":0.55700000000000005,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.72799999999999998,"15min":0.69899999999999995},"min":{"1min":0.54400000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":2.004,"5min":2.004,"15min":2.004},"last":0.72499999999999998}]},{"osd":11,"last update":"Wed Sep 13 03:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.73999999999999999}]}]},{"osd":7,"up_from":51,"seq":219043332145,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28520,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716632,"statfs":{"total":95995035648,"available":95965831168,"internally_reserved":0,"allocated":1540096,"data_stored":1037450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:10:55 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.59699999999999998,"15min":0.58599999999999997},"min":{"1min":0.498,"5min":0.438,"15min":0.438},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.378},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60099999999999998,"15min":0.59099999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.41099999999999998}]},{"osd":1,"last update":"Wed Sep 13 03:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.42999999999999999,"15min":0.40799999999999997},"min":{"1min":0.376,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.625,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.44700000000000001,"15min":0.42099999999999999},"min":{"1min":0.39100000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.46999999999999997}]},{"osd":2,"last update":"Wed Sep 13 03:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.60799999999999998,"15min":0.59399999999999997},"min":{"1min":0.437,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86899999999999999,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.59899999999999998,"15min":0.58099999999999996},"min":{"1min":0.496,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.55700000000000005}]},{"osd":3,"last update":"Wed Sep 13 03:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.68799999999999994,"15min":0.63100000000000001},"min":{"1min":0.46899999999999997,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.81899999999999995,"5min":6.2720000000000002,"15min":6.2720000000000002},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.64700000000000002,"15min":0.625},"min":{"1min":0.48299999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88900000000000001,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.57699999999999996}]},{"osd":4,"last update":"Wed Sep 13 03:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.53900000000000003,"15min":0.51700000000000002},"min":{"1min":0.46800000000000003,"5min":0.314,"15min":0.314},"max":{"1min":0.82399999999999995,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55500000000000005,"15min":0.52700000000000002},"min":{"1min":0.39900000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.88300000000000001,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.72099999999999997}]},{"osd":5,"last update":"Wed Sep 13 03:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.66200000000000003,"15min":0.64300000000000002},"min":{"1min":0.52500000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.65300000000000002,"15min":0.625},"min":{"1min":0.56699999999999995,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.95599999999999996,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.58999999999999997}]},{"osd":6,"last update":"Wed Sep 13 03:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.72999999999999998,"15min":0.70199999999999996},"min":{"1min":0.57499999999999996,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.98999999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.625},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.75800000000000001,"15min":0.72799999999999998},"min":{"1min":0.622,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0129999999999999,"5min":1.212,"15min":1.212},"last":0.78000000000000003}]},{"osd":8,"last update":"Wed Sep 13 03:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.68700000000000006,"15min":0.65800000000000003},"min":{"1min":0.53100000000000003,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.72699999999999998,"15min":0.69899999999999995},"min":{"1min":0.51300000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.93999999999999995,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.75600000000000001}]},{"osd":9,"last update":"Wed Sep 13 03:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82199999999999995,"15min":0.81299999999999994},"min":{"1min":0.60299999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.81000000000000005,"15min":0.80800000000000005},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":0.95799999999999996,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.74399999999999999}]},{"osd":10,"last update":"Wed Sep 13 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82699999999999996,"15min":0.82799999999999996},"min":{"1min":0.60599999999999998,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.0209999999999999,"5min":1.083,"15min":1.083},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.64400000000000002,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.999,"5min":1.169,"15min":1.169},"last":0.70099999999999996}]},{"osd":11,"last update":"Wed Sep 13 03:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.76600000000000001}]}]},{"osd":6,"up_from":46,"seq":197568495671,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":444189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.50800000000000001,"15min":0.5},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.53700000000000003,"15min":0.53100000000000003},"min":{"1min":0.37,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.39500000000000002}]},{"osd":1,"last update":"Wed Sep 13 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.65400000000000003,"15min":0.63500000000000001},"min":{"1min":0.42299999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.63700000000000001,"15min":0.61899999999999999},"min":{"1min":0.47199999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.60899999999999999}]},{"osd":2,"last update":"Wed Sep 13 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.68999999999999995,"15min":0.67400000000000004},"min":{"1min":0.502,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.748},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72099999999999997,"15min":0.71699999999999997},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.879,"5min":1.879,"15min":1.879},"last":0.56299999999999994}]},{"osd":3,"last update":"Wed Sep 13 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.63300000000000001,"15min":0.64100000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.64700000000000002,"15min":0.65100000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.88500000000000001}]},{"osd":4,"last update":"Wed Sep 13 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.73499999999999999,"15min":0.73099999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.745,"15min":0.75900000000000001},"min":{"1min":0.51000000000000001,"5min":0.495,"15min":0.495},"max":{"1min":1.1659999999999999,"5min":1.54,"15min":1.54},"last":0.67400000000000004}]},{"osd":5,"last update":"Wed Sep 13 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.77700000000000002,"15min":0.79600000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.77800000000000002,"15min":0.78000000000000003},"min":{"1min":0.57599999999999996,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1339999999999999,"5min":1.21,"15min":1.21},"last":0.79500000000000004}]},{"osd":7,"last update":"Wed Sep 13 03:12:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77500000000000002,"15min":0.77000000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.1899999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.78600000000000003,"15min":0.75800000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":0.69399999999999995}]},{"osd":8,"last update":"Wed Sep 13 03:12:23 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.88,"15min":0.88800000000000001},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.82999999999999996,"15min":0.83999999999999997},"min":{"1min":0.60099999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.86199999999999999}]},{"osd":9,"last update":"Wed Sep 13 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.85799999999999998,"15min":0.88200000000000001},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.246,"5min":1.319,"15min":1.319},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.92100000000000004,"15min":0.93799999999999994},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.024,"5min":2.024,"15min":2.024},"last":0.92800000000000005}]},{"osd":10,"last update":"Wed Sep 13 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.88500000000000001,"15min":0.88400000000000001},"min":{"1min":0.60699999999999998,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.226,"5min":1.262,"15min":1.262},"last":0.874},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.95199999999999996,"15min":0.94599999999999995},"min":{"1min":0.73099999999999998,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.958,"5min":1.958,"15min":1.958},"last":0.89200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":503232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.56899999999999995,"15min":0.53900000000000003},"min":{"1min":0.39300000000000002,"5min":0.35499999999999998,"15min":0.34999999999999998},"max":{"1min":0.78600000000000003,"5min":2.9260000000000002,"15min":2.9260000000000002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.56399999999999995,"15min":0.53900000000000003},"min":{"1min":0.42399999999999999,"5min":0.34100000000000003,"15min":0.32300000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.51500000000000001}]},{"osd":2,"last update":"Wed Sep 13 03:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.58399999999999996,"15min":0.53900000000000003},"min":{"1min":0.47899999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.58699999999999997,"15min":0.54200000000000004},"min":{"1min":0.57999999999999996,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.65800000000000003}]},{"osd":3,"last update":"Wed Sep 13 03:12:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60499999999999998,"15min":0.58699999999999997},"min":{"1min":0.47599999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.65700000000000003,"15min":0.61399999999999999},"min":{"1min":0.53700000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.90300000000000002}]},{"osd":4,"last update":"Wed Sep 13 03:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.47299999999999998,"15min":0.40600000000000003},"min":{"1min":0.41399999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.83299999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.51100000000000001,"15min":0.44900000000000001},"min":{"1min":0.41299999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.77100000000000002,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.70999999999999996}]},{"osd":5,"last update":"Wed Sep 13 03:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.622,"15min":0.55100000000000005},"min":{"1min":0.625,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.626,"15min":0.56100000000000005},"min":{"1min":0.621,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.752}]},{"osd":6,"last update":"Wed Sep 13 03:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.68600000000000005,"15min":0.66000000000000003},"min":{"1min":0.53800000000000003,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.70399999999999996,"15min":0.67500000000000004},"min":{"1min":0.51900000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.94699999999999995,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":0.72999999999999998}]},{"osd":7,"last update":"Wed Sep 13 03:12:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.59899999999999998,"15min":0.57099999999999995},"min":{"1min":0.45100000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.61599999999999999,"15min":0.60399999999999998},"min":{"1min":0.45900000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84799999999999998,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.79500000000000004}]},{"osd":8,"last update":"Wed Sep 13 03:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.68999999999999995,"15min":0.67200000000000004},"min":{"1min":0.63800000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.96399999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.751,"15min":0.72899999999999998},"min":{"1min":0.629,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":1.0049999999999999}]},{"osd":9,"last update":"Wed Sep 13 03:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.79300000000000004,"15min":0.78000000000000003},"min":{"1min":0.70299999999999996,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.83899999999999997,"15min":0.82999999999999996},"min":{"1min":0.67200000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":1.0369999999999999}]},{"osd":10,"last update":"Wed Sep 13 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.79200000000000004,"15min":0.78400000000000003},"min":{"1min":0.51600000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":1.073},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.78300000000000003,"15min":0.77600000000000002},"min":{"1min":0.57399999999999995,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.88100000000000001}]},{"osd":11,"last update":"Wed Sep 13 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":1.105},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.94799999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":445206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 13 03:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.61399999999999999,"15min":0.56000000000000005},"min":{"1min":0.38,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.97299999999999998,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.64700000000000002,"15min":0.57799999999999996},"min":{"1min":0.39500000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":6.0039999999999996,"5min":6.0039999999999996,"15min":6.0039999999999996},"last":0.59199999999999997}]},{"osd":2,"last update":"Wed Sep 13 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.73199999999999998,"15min":0.63600000000000001},"min":{"1min":0.47899999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":6.9640000000000004,"5min":6.9640000000000004,"15min":6.9640000000000004},"last":0.63300000000000001},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.70899999999999996,"15min":0.621},"min":{"1min":0.504,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":6.9459999999999997,"5min":6.9459999999999997,"15min":6.9459999999999997},"last":0.50700000000000001}]},{"osd":3,"last update":"Wed Sep 13 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":8.1940000000000008,"5min":8.1940000000000008,"15min":8.1940000000000008},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":0.83299999999999996}]},{"osd":4,"last update":"Wed Sep 13 03:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.74199999999999999,"15min":0.71499999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":5.9370000000000003,"5min":5.9370000000000003,"15min":5.9370000000000003},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69299999999999995,"15min":0.68300000000000005},"min":{"1min":0.45300000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0409999999999999,"5min":1.665,"15min":1.665},"last":0.56799999999999995}]},{"osd":5,"last update":"Wed Sep 13 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71099999999999997,"15min":0.69599999999999995},"min":{"1min":0.47899999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93200000000000005,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.70799999999999996,"15min":0.68200000000000005},"min":{"1min":0.59099999999999997,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.02,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.753}]},{"osd":6,"last update":"Wed Sep 13 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.71499999999999997,"15min":0.68100000000000005},"min":{"1min":0.42299999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.83899999999999997,"5min":6.1130000000000004,"15min":6.1130000000000004},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.752,"15min":0.74199999999999999},"min":{"1min":0.51200000000000001,"5min":0.434,"15min":0.434},"max":{"1min":1.008,"5min":2.7229999999999999,"15min":2.7229999999999999},"last":0.70899999999999996}]},{"osd":7,"last update":"Wed Sep 13 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.74299999999999999,"15min":0.73899999999999999},"min":{"1min":0.56299999999999994,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.626},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.80600000000000005,"15min":0.79400000000000004},"min":{"1min":0.74399999999999999,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1379999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.77900000000000003}]},{"osd":8,"last update":"Wed Sep 13 03:12:24 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.81399999999999995,"15min":0.81799999999999995},"min":{"1min":0.67200000000000004,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.2330000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.3680000000000001,"5min":0.89300000000000002,"15min":0.81000000000000005},"min":{"1min":0.57299999999999995,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":6.9489999999999998,"5min":6.9489999999999998,"15min":6.9489999999999998},"last":0.88200000000000001}]},{"osd":9,"last update":"Wed Sep 13 03:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.86499999999999999,"15min":0.88900000000000001},"min":{"1min":0.59099999999999997,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.2549999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.86499999999999999,"15min":0.88700000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.85199999999999998}]},{"osd":10,"last update":"Wed Sep 13 03:12:09 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":0.92400000000000004,"15min":0.89000000000000001},"min":{"1min":0.68200000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":6.0830000000000002,"5min":6.0830000000000002,"15min":6.0830000000000002},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.165,"5min":1.0009999999999999,"15min":0.97299999999999998},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":6.9669999999999996,"5min":6.9669999999999996,"15min":6.9669999999999996},"last":0.70199999999999996}]},{"osd":11,"last update":"Wed Sep 13 03:12:31 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.88700000000000001,"15min":0.88500000000000001},"min":{"1min":0.70199999999999996,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.94299999999999995,"15min":0.95099999999999996},"min":{"1min":0.71499999999999997,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.2210000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.76800000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":503872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.55400000000000005,"15min":0.49099999999999999},"min":{"1min":0.41599999999999998,"5min":0.307,"15min":0.307},"max":{"1min":2.4390000000000001,"5min":2.4390000000000001,"15min":2.4390000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.57999999999999996,"15min":0.50700000000000001},"min":{"1min":0.46000000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":3.0979999999999999,"5min":3.0979999999999999,"15min":3.0979999999999999},"last":0.79700000000000004}]},{"osd":1,"last update":"Wed Sep 13 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.56299999999999994,"15min":0.52500000000000002},"min":{"1min":0.443,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.55200000000000005,"15min":0.53200000000000003},"min":{"1min":0.37,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.497}]},{"osd":3,"last update":"Wed Sep 13 03:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.65600000000000003,"15min":0.57999999999999996},"min":{"1min":0.52800000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90400000000000003,"5min":2.879,"15min":2.879},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.65600000000000003,"15min":0.58399999999999996},"min":{"1min":0.44600000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":2.782,"5min":2.8969999999999998,"15min":2.8969999999999998},"last":0.60599999999999998}]},{"osd":4,"last update":"Wed Sep 13 03:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.624},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.59099999999999997}]},{"osd":5,"last update":"Wed Sep 13 03:12:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.46999999999999997,"15min":0.433},"min":{"1min":0.36299999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.502,"15min":0.45600000000000002},"min":{"1min":0.36399999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.95699999999999996,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.69999999999999996}]},{"osd":6,"last update":"Wed Sep 13 03:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.67100000000000004,"15min":0.63200000000000001},"min":{"1min":0.51400000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.95099999999999996,"5min":1.048,"15min":1.048},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.67600000000000005,"15min":0.63},"min":{"1min":0.39700000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90400000000000003,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.74199999999999999}]},{"osd":7,"last update":"Wed Sep 13 03:10:58 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.63700000000000001,"15min":0.63600000000000001},"min":{"1min":0.47499999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.84699999999999998,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.66100000000000003,"15min":0.65900000000000003},"min":{"1min":0.51700000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.83399999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.497}]},{"osd":8,"last update":"Wed Sep 13 03:12:22 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.59899999999999998,"15min":0.58799999999999997},"min":{"1min":0.41499999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.69899999999999995,"15min":0.69299999999999995},"min":{"1min":0.59599999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.78200000000000003}]},{"osd":9,"last update":"Wed Sep 13 03:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.78200000000000003,"15min":0.75800000000000001},"min":{"1min":0.56000000000000005,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.98899999999999999,"5min":2.6459999999999999,"15min":2.6459999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.753,"15min":0.73999999999999999},"min":{"1min":0.59099999999999997,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.002,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.84399999999999997}]},{"osd":10,"last update":"Wed Sep 13 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.68500000000000005,"15min":0.66800000000000004},"min":{"1min":0.60299999999999998,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.73399999999999999,"15min":0.71499999999999997},"min":{"1min":0.58899999999999997,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.75800000000000001}]},{"osd":11,"last update":"Wed Sep 13 03:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.81000000000000005}]}]},{"osd":3,"up_from":25,"seq":107374182468,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":443704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:12:31 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.44600000000000001,"15min":0.40999999999999998},"min":{"1min":0.38500000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.437,"5min":0.46500000000000002,"15min":0.437},"min":{"1min":0.36199999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.57099999999999995,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.36299999999999999}]},{"osd":1,"last update":"Wed Sep 13 03:12:31 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.63700000000000001,"15min":0.59799999999999998},"min":{"1min":0.44,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80500000000000005,"5min":1.117,"15min":1.117},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.66100000000000003,"15min":0.60699999999999998},"min":{"1min":0.42199999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94899999999999995,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.57899999999999996}]},{"osd":2,"last update":"Wed Sep 13 03:12:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.69299999999999995,"15min":0.64300000000000002},"min":{"1min":0.55000000000000004,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.88800000000000001,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.69299999999999995,"15min":0.64900000000000002},"min":{"1min":0.44,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.85499999999999998,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.59999999999999998}]},{"osd":4,"last update":"Wed Sep 13 03:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.68300000000000005,"15min":0.63600000000000001},"min":{"1min":0.49399999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0900000000000001,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.67500000000000004,"15min":0.627},"min":{"1min":0.52000000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0389999999999999,"5min":1.177,"15min":1.177},"last":0.54500000000000004}]},{"osd":5,"last update":"Wed Sep 13 03:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.71399999999999997,"15min":0.68600000000000005},"min":{"1min":0.49299999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.96599999999999997,"5min":1.012,"15min":1.012},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72599999999999998,"15min":0.70399999999999996},"min":{"1min":0.53300000000000003,"5min":0.498,"15min":0.498},"max":{"1min":1.0069999999999999,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.76000000000000001}]},{"osd":6,"last update":"Wed Sep 13 03:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.64400000000000002,"15min":0.624},"min":{"1min":0.53700000000000003,"5min":0.374,"15min":0.374},"max":{"1min":0.72699999999999998,"5min":1.117,"15min":1.117},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.77800000000000002,"15min":0.752},"min":{"1min":0.628,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0149999999999999,"5min":1.599,"15min":1.599},"last":0.85599999999999998}]},{"osd":7,"last update":"Wed Sep 13 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.86099999999999999,"15min":0.85399999999999998},"min":{"1min":0.55300000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0529999999999999,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.83299999999999996,"15min":0.82799999999999996},"min":{"1min":0.56699999999999995,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0269999999999999,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.70099999999999996}]},{"osd":8,"last update":"Wed Sep 13 03:10:17 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.82199999999999995}]},{"osd":9,"last update":"Wed Sep 13 03:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.88700000000000001,"15min":0.91500000000000004},"min":{"1min":0.52700000000000002,"5min":0.502,"15min":0.502},"max":{"1min":0.91700000000000004,"5min":1.538,"15min":1.538},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.84099999999999997,"15min":0.88500000000000001},"min":{"1min":0.57499999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.90100000000000002,"5min":1.677,"15min":1.677},"last":0.79300000000000004}]},{"osd":10,"last update":"Wed Sep 13 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.93000000000000005,"15min":0.93899999999999995},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.034,"5min":1.708,"15min":1.708},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.95399999999999996,"15min":0.96399999999999997},"min":{"1min":0.69899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.1180000000000001,"5min":1.635,"15min":1.635},"last":0.86299999999999999}]},{"osd":11,"last update":"Wed Sep 13 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.747}]}]},{"osd":4,"up_from":33,"seq":141733920832,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":444259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.54500000000000004,"15min":0.51500000000000001},"min":{"1min":0.36699999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.55900000000000005,"15min":0.52900000000000003},"min":{"1min":0.35599999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.63800000000000001}]},{"osd":1,"last update":"Wed Sep 13 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.39700000000000002,"15min":0.36199999999999999},"min":{"1min":0.308,"5min":0.245,"15min":0.245},"max":{"1min":0.66700000000000004,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.379},{"interface":"front","average":{"1min":0.443,"5min":0.40799999999999997,"15min":0.372},"min":{"1min":0.32500000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.68200000000000005,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.436}]},{"osd":2,"last update":"Wed Sep 13 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58099999999999996,"15min":0.56299999999999994},"min":{"1min":0.44,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.81999999999999995,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.58999999999999997,"15min":0.54000000000000004},"min":{"1min":0.38600000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.59699999999999998}]},{"osd":3,"last update":"Wed Sep 13 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63100000000000001,"15min":0.56299999999999994},"min":{"1min":0.46400000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83899999999999997,"5min":4.2919999999999998,"15min":4.2919999999999998},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.59999999999999998,"15min":0.56999999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81999999999999995,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.57099999999999995}]},{"osd":5,"last update":"Wed Sep 13 03:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.61699999999999999,"15min":0.58299999999999996},"min":{"1min":0.44900000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.874,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.60699999999999998,"15min":0.56899999999999995},"min":{"1min":0.42399999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.46800000000000003}]},{"osd":6,"last update":"Wed Sep 13 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.745,"15min":0.71299999999999997},"min":{"1min":0.45100000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.86799999999999999,"5min":6.1859999999999999,"15min":6.1859999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.70599999999999996,"15min":0.71199999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.84399999999999997,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.66300000000000003}]},{"osd":7,"last update":"Wed Sep 13 03:11:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.628,"15min":0.60999999999999999},"min":{"1min":0.34200000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.93300000000000005,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63600000000000001,"15min":0.622},"min":{"1min":0.39700000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89700000000000002,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.67800000000000005}]},{"osd":8,"last update":"Wed Sep 13 03:12:20 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.72399999999999998,"15min":0.72299999999999998},"min":{"1min":0.51300000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0029999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.76100000000000001,"15min":0.755},"min":{"1min":0.629,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.98499999999999999,"5min":1.01,"15min":1.01},"last":0.74299999999999999}]},{"osd":9,"last update":"Wed Sep 13 03:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.90000000000000002,"15min":0.878},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0569999999999999,"5min":6.1529999999999996,"15min":6.1529999999999996},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.81299999999999994,"15min":0.82899999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.96099999999999997,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.71799999999999997}]},{"osd":10,"last update":"Wed Sep 13 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.81200000000000006,"15min":0.82099999999999995},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.016,"5min":1.113,"15min":1.113},"last":0.80000000000000004}]},{"osd":11,"last update":"Wed Sep 13 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.75800000000000001}]}]},{"osd":5,"up_from":38,"seq":163208757309,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28536,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716616,"statfs":{"total":95995035648,"available":95965814784,"internally_reserved":0,"allocated":1556480,"data_stored":1036870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 13 03:12:14 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":0.69999999999999996,"15min":0.64700000000000002},"min":{"1min":0.505,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":9.3100000000000005,"5min":9.3100000000000005,"15min":9.3100000000000005},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.67600000000000005,"15min":0.64200000000000002},"min":{"1min":0.42399999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":2.9169999999999998,"5min":6.0179999999999998,"15min":6.0179999999999998},"last":0.64600000000000002}]},{"osd":1,"last update":"Wed Sep 13 03:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.56699999999999995,"15min":0.52700000000000002},"min":{"1min":0.42999999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76200000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.56399999999999995,"15min":0.52000000000000002},"min":{"1min":0.49299999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.48999999999999999}]},{"osd":2,"last update":"Wed Sep 13 03:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.41999999999999998,"15min":0.39000000000000001},"min":{"1min":0.34799999999999998,"5min":0.193,"15min":0.193},"max":{"1min":0.76000000000000001,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.42099999999999999,"15min":0.39200000000000002},"min":{"1min":0.35799999999999998,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.64900000000000002,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.36699999999999999}]},{"osd":3,"last update":"Wed Sep 13 03:12:14 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":0.68400000000000005,"15min":0.60199999999999998},"min":{"1min":0.48499999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":9.4369999999999994,"5min":9.4369999999999994,"15min":9.4369999999999994},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61199999999999999,"15min":0.58999999999999997},"min":{"1min":0.46100000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.96299999999999997,"5min":1.48,"15min":1.48},"last":0.58899999999999997}]},{"osd":4,"last update":"Wed Sep 13 03:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.626,"15min":0.56599999999999995},"min":{"1min":0.48299999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.88400000000000001,"5min":3.3740000000000001,"15min":3.3740000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.59299999999999997,"15min":0.55700000000000005},"min":{"1min":0.44600000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79100000000000004,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.46500000000000002}]},{"osd":6,"last update":"Wed Sep 13 03:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.68600000000000005,"15min":0.64900000000000002},"min":{"1min":0.498,"5min":0.371,"15min":0.371},"max":{"1min":0.88300000000000001,"5min":4.8250000000000002,"15min":4.8250000000000002},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72099999999999997,"15min":0.67200000000000004},"min":{"1min":0.44600000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.90800000000000003,"5min":4.8049999999999997,"15min":4.8049999999999997},"last":0.71199999999999997}]},{"osd":7,"last update":"Wed Sep 13 03:12:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.63400000000000001,"15min":0.63300000000000001},"min":{"1min":0.47899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.86399999999999999,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.66600000000000004,"15min":0.65900000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85499999999999998,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.66600000000000004}]},{"osd":8,"last update":"Wed Sep 13 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57099999999999995,"15min":0.57599999999999996},"min":{"1min":0.36799999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83299999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61899999999999999,"15min":0.61799999999999999},"min":{"1min":0.48299999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.80800000000000005,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.68200000000000005}]},{"osd":9,"last update":"Wed Sep 13 03:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":1.0069999999999999,"15min":1.024},"min":{"1min":0.623,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.91500000000000004,"5min":8.3070000000000004,"15min":8.3070000000000004},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79300000000000004,"15min":0.76900000000000002},"min":{"1min":0.63400000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.97499999999999998,"5min":3.016,"15min":3.016},"last":0.79300000000000004}]},{"osd":10,"last update":"Wed Sep 13 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.68600000000000005,"15min":0.67600000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.876,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.74099999999999999,"15min":0.73299999999999998},"min":{"1min":0.57499999999999996,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.755}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":65164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-13T03:12:56.420 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-13T03:12:56.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.656+0000 7f07d73ff700 1 -- 172.21.15.64:0/1614594073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07d0100610 msgr2=0x7f07d0100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:56.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.656+0000 7f07d73ff700 1 --2- 172.21.15.64:0/1614594073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07d0100610 0x7f07d0100a30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f07cc0099c0 tx=0x7f07cc009cd0 comp rx=0 tx=0).stop 2023-09-13T03:12:56.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.656+0000 7f07d73ff700 1 -- 172.21.15.64:0/1614594073 shutdown_connections 2023-09-13T03:12:56.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.656+0000 7f07d73ff700 1 --2- 172.21.15.64:0/1614594073 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07d01021a0 0x7f07d0108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:56.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.656+0000 7f07d73ff700 1 --2- 172.21.15.64:0/1614594073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07d0100610 0x7f07d0100a30 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:56.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.656+0000 7f07d73ff700 1 --2- 172.21.15.64:0/1614594073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07d0101800 0x7f07d0101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:56.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.656+0000 7f07d73ff700 1 -- 172.21.15.64:0/1614594073 >> 172.21.15.64:0/1614594073 conn(0x7f07d00fbb40 msgr2=0x7f07d00fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:56.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.656+0000 7f07d73ff700 1 -- 172.21.15.64:0/1614594073 shutdown_connections 2023-09-13T03:12:56.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.656+0000 7f07d73ff700 1 -- 172.21.15.64:0/1614594073 wait complete. 2023-09-13T03:12:56.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.657+0000 7f07d73ff700 1 Processor -- start 2023-09-13T03:12:56.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.657+0000 7f07d73ff700 1 -- start start 2023-09-13T03:12:56.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.658+0000 7f07d73ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07d0100610 0x7f07d01984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:56.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.658+0000 7f07d73ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07d0101800 0x7f07d0198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:56.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.658+0000 7f07d73ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07d01021a0 0x7f07d019faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:56.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.658+0000 7f07d73ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d01a0070 con 0x7f07d0100610 2023-09-13T03:12:56.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.658+0000 7f07d73ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d01a01b0 con 0x7f07d0101800 2023-09-13T03:12:56.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.658+0000 7f07d73ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d01a0340 con 0x7f07d01021a0 2023-09-13T03:12:56.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.658+0000 7f07d519b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07d0100610 0x7f07d01984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:56.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.658+0000 7f07d519b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07d0100610 0x7f07d01984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56088/0 (socket says 172.21.15.64:56088) 2023-09-13T03:12:56.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.658+0000 7f07d519b700 1 -- 172.21.15.64:0/3170845587 learned_addr learned my addr 172.21.15.64:0/3170845587 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:56.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.658+0000 7f07d499a700 1 --2- 172.21.15.64:0/3170845587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07d0101800 0x7f07d0198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:56.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.658+0000 7f07d599c700 1 --2- 172.21.15.64:0/3170845587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07d01021a0 0x7f07d019faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:56.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.658+0000 7f07d519b700 1 -- 172.21.15.64:0/3170845587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07d01021a0 msgr2=0x7f07d019faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:56.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.658+0000 7f07d519b700 1 --2- 172.21.15.64:0/3170845587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07d01021a0 0x7f07d019faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:56.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.658+0000 7f07d519b700 1 -- 172.21.15.64:0/3170845587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07d0101800 msgr2=0x7f07d0198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:56.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.658+0000 7f07d519b700 1 --2- 172.21.15.64:0/3170845587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07d0101800 0x7f07d0198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:56.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.658+0000 7f07d519b700 1 -- 172.21.15.64:0/3170845587 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07cc0095f0 con 0x7f07d0100610 2023-09-13T03:12:56.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.658+0000 7f07d519b700 1 --2- 172.21.15.64:0/3170845587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07d0100610 0x7f07d01984f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f07cc0058e0 tx=0x7f07cc00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:56.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.659+0000 7f07c67fc700 1 -- 172.21.15.64:0/3170845587 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07cc01d070 con 0x7f07d0100610 2023-09-13T03:12:56.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.659+0000 7f07d73ff700 1 -- 172.21.15.64:0/3170845587 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07d01a05c0 con 0x7f07d0100610 2023-09-13T03:12:56.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.659+0000 7f07c67fc700 1 -- 172.21.15.64:0/3170845587 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07cc005510 con 0x7f07d0100610 2023-09-13T03:12:56.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.659+0000 7f07c67fc700 1 -- 172.21.15.64:0/3170845587 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07cc004b10 con 0x7f07d0100610 2023-09-13T03:12:56.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.659+0000 7f07d73ff700 1 -- 172.21.15.64:0/3170845587 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07d01a09e0 con 0x7f07d0100610 2023-09-13T03:12:56.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.660+0000 7f07d73ff700 1 -- 172.21.15.64:0/3170845587 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07d010be40 con 0x7f07d0100610 2023-09-13T03:12:56.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.662+0000 7f07c67fc700 1 -- 172.21.15.64:0/3170845587 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f07cc00fbb0 con 0x7f07d0100610 2023-09-13T03:12:56.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.663+0000 7f07c67fc700 1 --2- 172.21.15.64:0/3170845587 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f07bc0a0690 0x7f07bc0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:56.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.663+0000 7f07c67fc700 1 -- 172.21.15.64:0/3170845587 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f07cc0ccaa0 con 0x7f07d0100610 2023-09-13T03:12:56.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.663+0000 7f07d499a700 1 --2- 172.21.15.64:0/3170845587 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f07bc0a0690 0x7f07bc0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:56.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.664+0000 7f07d499a700 1 --2- 172.21.15.64:0/3170845587 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f07bc0a0690 0x7f07bc0a2b50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f07c0009c60 tx=0x7f07c0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:56.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.667+0000 7f07c67fc700 1 -- 172.21.15.64:0/3170845587 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f07cc096750 con 0x7f07d0100610 2023-09-13T03:12:56.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:56 smithi064 ceph-mon[97675]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T03:12:56.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.882+0000 7f07d73ff700 1 -- 172.21.15.64:0/3170845587 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f07d010c080 con 0x7f07d0100610 2023-09-13T03:12:56.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.884+0000 7f07c67fc700 1 -- 172.21.15.64:0/3170845587 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22473 (secure 0 0 0) 0x7f07cc099d70 con 0x7f07d0100610 2023-09-13T03:12:56.885 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:12:56.885 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":85,"fsid":"f8dbced4-51e1-11ee-9ab7-7b867c8bd7da","created":"2023-09-13T03:03:24.167202+0000","modified":"2023-09-13T03:11:24.258871+0000","last_up_change":"2023-09-13T03:10:22.503529+0000","last_in_change":"2023-09-13T03:10:05.678720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T03:06:58.987479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-13T03:11:08.589202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-13T03:11:09.924659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3997c114-241b-48c4-8feb-653c99209470","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6803","nonce":3940892135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6805","nonce":3940892135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6809","nonce":3940892135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6807","nonce":3940892135}]},"public_addr":"172.21.15.64:6803/3940892135","cluster_addr":"172.21.15.64:6805/3940892135","heartbeat_back_addr":"172.21.15.64:6809/3940892135","heartbeat_front_addr":"172.21.15.64:6807/3940892135","state":["exists","up"]},{"osd":1,"uuid":"94cfae85-322e-4ed2-a6e6-e40349bb14bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6801","nonce":4188149333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6803","nonce":4188149333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6807","nonce":4188149333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6805","nonce":4188149333}]},"public_addr":"172.21.15.142:6801/4188149333","cluster_addr":"172.21.15.142:6803/4188149333","heartbeat_back_addr":"172.21.15.142:6807/4188149333","heartbeat_front_addr":"172.21.15.142:6805/4188149333","state":["exists","up"]},{"osd":2,"uuid":"b3aae17e-96db-4008-8c27-f733c46b30ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6801","nonce":2406669338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6803","nonce":2406669338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6807","nonce":2406669338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6805","nonce":2406669338}]},"public_addr":"172.21.15.170:6801/2406669338","cluster_addr":"172.21.15.170:6803/2406669338","heartbeat_back_addr":"172.21.15.170:6807/2406669338","heartbeat_front_addr":"172.21.15.170:6805/2406669338","state":["exists","up"]},{"osd":3,"uuid":"7fa8d8cb-2ab7-48a7-8b49-5e2903d6779f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6811","nonce":3558318707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6813","nonce":3558318707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6817","nonce":3558318707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6815","nonce":3558318707}]},"public_addr":"172.21.15.64:6811/3558318707","cluster_addr":"172.21.15.64:6813/3558318707","heartbeat_back_addr":"172.21.15.64:6817/3558318707","heartbeat_front_addr":"172.21.15.64:6815/3558318707","state":["exists","up"]},{"osd":4,"uuid":"8c7a9c9f-f919-48d3-9db2-214a25bb5603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6809","nonce":1811142576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6811","nonce":1811142576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6815","nonce":1811142576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6813","nonce":1811142576}]},"public_addr":"172.21.15.142:6809/1811142576","cluster_addr":"172.21.15.142:6811/1811142576","heartbeat_back_addr":"172.21.15.142:6815/1811142576","heartbeat_front_addr":"172.21.15.142:6813/1811142576","state":["exists","up"]},{"osd":5,"uuid":"0c6101c6-2065-4022-a751-d3f216d7f478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6809","nonce":2903841453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6811","nonce":2903841453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6815","nonce":2903841453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6813","nonce":2903841453}]},"public_addr":"172.21.15.170:6809/2903841453","cluster_addr":"172.21.15.170:6811/2903841453","heartbeat_back_addr":"172.21.15.170:6815/2903841453","heartbeat_front_addr":"172.21.15.170:6813/2903841453","state":["exists","up"]},{"osd":6,"uuid":"5cf3e52e-d155-41b1-9194-db047d26a953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6819","nonce":1436584490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6821","nonce":1436584490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6825","nonce":1436584490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6823","nonce":1436584490}]},"public_addr":"172.21.15.64:6819/1436584490","cluster_addr":"172.21.15.64:6821/1436584490","heartbeat_back_addr":"172.21.15.64:6825/1436584490","heartbeat_front_addr":"172.21.15.64:6823/1436584490","state":["exists","up"]},{"osd":7,"uuid":"e97323e3-badd-41a7-af9e-e78a81a64046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6817","nonce":3726793534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6819","nonce":3726793534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6823","nonce":3726793534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6821","nonce":3726793534}]},"public_addr":"172.21.15.142:6817/3726793534","cluster_addr":"172.21.15.142:6819/3726793534","heartbeat_back_addr":"172.21.15.142:6823/3726793534","heartbeat_front_addr":"172.21.15.142:6821/3726793534","state":["exists","up"]},{"osd":8,"uuid":"83d2835c-ed26-493d-9222-c2894ec0e071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6817","nonce":4272513873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6819","nonce":4272513873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6823","nonce":4272513873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6821","nonce":4272513873}]},"public_addr":"172.21.15.170:6817/4272513873","cluster_addr":"172.21.15.170:6819/4272513873","heartbeat_back_addr":"172.21.15.170:6823/4272513873","heartbeat_front_addr":"172.21.15.170:6821/4272513873","state":["exists","up"]},{"osd":9,"uuid":"cb936bdd-1bec-4236-aa2e-a24adcac11e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6827","nonce":2641175211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6829","nonce":2641175211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6833","nonce":2641175211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6831","nonce":2641175211}]},"public_addr":"172.21.15.64:6827/2641175211","cluster_addr":"172.21.15.64:6829/2641175211","heartbeat_back_addr":"172.21.15.64:6833/2641175211","heartbeat_front_addr":"172.21.15.64:6831/2641175211","state":["exists","up"]},{"osd":10,"uuid":"85512a1c-6171-43d4-9c15-c588a8b71be0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6825","nonce":936499217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6827","nonce":936499217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6831","nonce":936499217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6829","nonce":936499217}]},"public_addr":"172.21.15.142:6825/936499217","cluster_addr":"172.21.15.142:6827/936499217","heartbeat_back_addr":"172.21.15.142:6831/936499217","heartbeat_front_addr":"172.21.15.142:6829/936499217","state":["exists","up"]},{"osd":11,"uuid":"9b8417dc-5d6c-4917-94a9-b032c5bc33d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6825","nonce":3605017060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6827","nonce":3605017060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6831","nonce":3605017060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6829","nonce":3605017060}]},"public_addr":"172.21.15.170:6825/3605017060","cluster_addr":"172.21.15.170:6827/3605017060","heartbeat_back_addr":"172.21.15.170:6831/3605017060","heartbeat_front_addr":"172.21.15.170:6829/3605017060","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:14.229044+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:36.147777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:56.655298+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:07:23.107741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:07:44.663770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:05.577432+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:31.168031+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:52.791055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:09:13.658546+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:09:39.150360+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:10:00.601735+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:10:21.407555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6800/2955227347":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/1373223118":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/271194613":"2023-09-14T03:04:22.355817+0000","172.21.15.64:6801/3998091347":"2023-09-14T03:03:56.442541+0000","172.21.15.64:6800/3998091347":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/920818572":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/735789005":"2023-09-14T03:03:56.442541+0000","172.21.15.64:6801/2955227347":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/1975923178":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/1500151178":"2023-09-14T03:04:22.355817+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T03:12:56.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.886+0000 7f07d73ff700 1 -- 172.21.15.64:0/3170845587 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f07bc0a0690 msgr2=0x7f07bc0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:56.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.886+0000 7f07d73ff700 1 --2- 172.21.15.64:0/3170845587 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f07bc0a0690 0x7f07bc0a2b50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f07c0009c60 tx=0x7f07c0009260 comp rx=0 tx=0).stop 2023-09-13T03:12:56.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.886+0000 7f07d73ff700 1 -- 172.21.15.64:0/3170845587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07d0100610 msgr2=0x7f07d01984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:56.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.886+0000 7f07d73ff700 1 --2- 172.21.15.64:0/3170845587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07d0100610 0x7f07d01984f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f07cc0058e0 tx=0x7f07cc00f690 comp rx=0 tx=0).stop 2023-09-13T03:12:56.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.887+0000 7f07d73ff700 1 -- 172.21.15.64:0/3170845587 shutdown_connections 2023-09-13T03:12:56.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.887+0000 7f07d73ff700 1 --2- 172.21.15.64:0/3170845587 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f07bc0a0690 0x7f07bc0a2b50 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:56.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.887+0000 7f07d73ff700 1 --2- 172.21.15.64:0/3170845587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07d01021a0 0x7f07d019faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:56.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.887+0000 7f07d73ff700 1 --2- 172.21.15.64:0/3170845587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07d0100610 0x7f07d01984f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:56.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.887+0000 7f07d73ff700 1 --2- 172.21.15.64:0/3170845587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07d0101800 0x7f07d0198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:56.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.887+0000 7f07d73ff700 1 -- 172.21.15.64:0/3170845587 >> 172.21.15.64:0/3170845587 conn(0x7f07d00fbb40 msgr2=0x7f07d00fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:56.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.887+0000 7f07d73ff700 1 -- 172.21.15.64:0/3170845587 shutdown_connections 2023-09-13T03:12:56.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:56.887+0000 7f07d73ff700 1 -- 172.21.15.64:0/3170845587 wait complete. 2023-09-13T03:12:56.906 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-13T03:06:58.987479+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-13T03:11:08.589202+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-13T03:11:09.924659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-13T03:12:56.907 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-13T03:12:56.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:56 smithi170 ceph-mon[134328]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T03:12:56.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:56 smithi142 ceph-mon[134082]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T03:12:57.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.113+0000 7fbe03dff700 1 -- 172.21.15.64:0/3868864034 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbdfc074640 msgr2=0x7fbdfc072a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:57.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.113+0000 7fbe03dff700 1 --2- 172.21.15.64:0/3868864034 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbdfc074640 0x7fbdfc072a90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbdf80098f0 tx=0x7fbdf8009c00 comp rx=0 tx=0).stop 2023-09-13T03:12:57.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.113+0000 7fbe03dff700 1 -- 172.21.15.64:0/3868864034 shutdown_connections 2023-09-13T03:12:57.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.113+0000 7fbe03dff700 1 --2- 172.21.15.64:0/3868864034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdfc1084b0 0x7fbdfc10a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:57.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.113+0000 7fbe03dff700 1 --2- 172.21.15.64:0/3868864034 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbdfc074640 0x7fbdfc072a90 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:57.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.113+0000 7fbe03dff700 1 --2- 172.21.15.64:0/3868864034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbdfc073060 0x7fbdfc0734e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:57.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.113+0000 7fbe03dff700 1 -- 172.21.15.64:0/3868864034 >> 172.21.15.64:0/3868864034 conn(0x7fbdfc0fbb80 msgr2=0x7fbdfc0fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:57.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.114+0000 7fbe03dff700 1 -- 172.21.15.64:0/3868864034 shutdown_connections 2023-09-13T03:12:57.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.114+0000 7fbe03dff700 1 -- 172.21.15.64:0/3868864034 wait complete. 2023-09-13T03:12:57.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.115+0000 7fbe03dff700 1 Processor -- start 2023-09-13T03:12:57.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.115+0000 7fbe03dff700 1 -- start start 2023-09-13T03:12:57.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.115+0000 7fbe03dff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbdfc073060 0x7fbdfc198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:57.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.115+0000 7fbe03dff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbdfc074640 0x7fbdfc198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:57.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.115+0000 7fbe03dff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdfc1084b0 0x7fbdfc19fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:57.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.115+0000 7fbe03dff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdfc1a0100 con 0x7fbdfc1084b0 2023-09-13T03:12:57.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.115+0000 7fbe03dff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdfc1a0240 con 0x7fbdfc074640 2023-09-13T03:12:57.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.115+0000 7fbe03dff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdfc1a03d0 con 0x7fbdfc073060 2023-09-13T03:12:57.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.115+0000 7fbe0239c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdfc1084b0 0x7fbdfc19fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:57.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.115+0000 7fbe01b9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbdfc073060 0x7fbdfc198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:57.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.115+0000 7fbe0139a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbdfc074640 0x7fbdfc198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:57.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.115+0000 7fbe0239c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdfc1084b0 0x7fbdfc19fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56104/0 (socket says 172.21.15.64:56104) 2023-09-13T03:12:57.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.115+0000 7fbe0239c700 1 -- 172.21.15.64:0/3543581527 learned_addr learned my addr 172.21.15.64:0/3543581527 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:57.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.116+0000 7fbe0139a700 1 -- 172.21.15.64:0/3543581527 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbdfc073060 msgr2=0x7fbdfc198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:57.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.116+0000 7fbe0139a700 1 --2- 172.21.15.64:0/3543581527 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbdfc073060 0x7fbdfc198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:57.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.116+0000 7fbe0139a700 1 -- 172.21.15.64:0/3543581527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdfc1084b0 msgr2=0x7fbdfc19fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:57.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.116+0000 7fbe0139a700 1 --2- 172.21.15.64:0/3543581527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdfc1084b0 0x7fbdfc19fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:57.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.116+0000 7fbe0139a700 1 -- 172.21.15.64:0/3543581527 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf8009520 con 0x7fbdfc074640 2023-09-13T03:12:57.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.116+0000 7fbe0139a700 1 --2- 172.21.15.64:0/3543581527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbdfc074640 0x7fbdfc198ac0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbdec005fd0 tx=0x7fbdec00eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:57.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.116+0000 7fbdf2ffd700 1 -- 172.21.15.64:0/3543581527 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdec004220 con 0x7fbdfc074640 2023-09-13T03:12:57.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.117+0000 7fbe03dff700 1 -- 172.21.15.64:0/3543581527 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdfc1a06b0 con 0x7fbdfc074640 2023-09-13T03:12:57.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.117+0000 7fbdf2ffd700 1 -- 172.21.15.64:0/3543581527 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbdec004380 con 0x7fbdfc074640 2023-09-13T03:12:57.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.117+0000 7fbdf2ffd700 1 -- 172.21.15.64:0/3543581527 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdec003be0 con 0x7fbdfc074640 2023-09-13T03:12:57.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.117+0000 7fbe03dff700 1 -- 172.21.15.64:0/3543581527 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdfc1a0c10 con 0x7fbdfc074640 2023-09-13T03:12:57.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.118+0000 7fbe03dff700 1 -- 172.21.15.64:0/3543581527 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdfc106b60 con 0x7fbdfc074640 2023-09-13T03:12:57.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.120+0000 7fbdf2ffd700 1 -- 172.21.15.64:0/3543581527 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fbdec003d40 con 0x7fbdfc074640 2023-09-13T03:12:57.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.121+0000 7fbdf2ffd700 1 --2- 172.21.15.64:0/3543581527 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fbde80a0640 0x7fbde80a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:57.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.121+0000 7fbdf2ffd700 1 -- 172.21.15.64:0/3543581527 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fbdec014070 con 0x7fbdfc074640 2023-09-13T03:12:57.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.122+0000 7fbe01b9b700 1 --2- 172.21.15.64:0/3543581527 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fbde80a0640 0x7fbde80a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:57.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.122+0000 7fbe01b9b700 1 --2- 172.21.15.64:0/3543581527 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fbde80a0640 0x7fbde80a2b00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8005e90 tx=0x7fbdf8019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:57.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.123+0000 7fbdf2ffd700 1 -- 172.21.15.64:0/3543581527 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fbdec0959f0 con 0x7fbdfc074640 2023-09-13T03:12:57.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.335+0000 7fbe03dff700 1 -- 172.21.15.64:0/3543581527 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbdfc10e030 con 0x7fbdfc074640 2023-09-13T03:12:57.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.337+0000 7fbdf2ffd700 1 -- 172.21.15.64:0/3543581527 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbdec099010 con 0x7fbdfc074640 2023-09-13T03:12:57.337 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-09-13T03:12:57.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.339+0000 7fbe03dff700 1 -- 172.21.15.64:0/3543581527 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fbde80a0640 msgr2=0x7fbde80a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:57.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.339+0000 7fbe03dff700 1 --2- 172.21.15.64:0/3543581527 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fbde80a0640 0x7fbde80a2b00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8005e90 tx=0x7fbdf8019040 comp rx=0 tx=0).stop 2023-09-13T03:12:57.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.339+0000 7fbe03dff700 1 -- 172.21.15.64:0/3543581527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbdfc074640 msgr2=0x7fbdfc198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:57.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.339+0000 7fbe03dff700 1 --2- 172.21.15.64:0/3543581527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbdfc074640 0x7fbdfc198ac0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbdec005fd0 tx=0x7fbdec00eca0 comp rx=0 tx=0).stop 2023-09-13T03:12:57.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.339+0000 7fbe03dff700 1 -- 172.21.15.64:0/3543581527 shutdown_connections 2023-09-13T03:12:57.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.339+0000 7fbe03dff700 1 --2- 172.21.15.64:0/3543581527 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fbde80a0640 0x7fbde80a2b00 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:57.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.339+0000 7fbe03dff700 1 --2- 172.21.15.64:0/3543581527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdfc1084b0 0x7fbdfc19fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:57.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.339+0000 7fbe03dff700 1 --2- 172.21.15.64:0/3543581527 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbdfc073060 0x7fbdfc198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:57.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.339+0000 7fbe03dff700 1 --2- 172.21.15.64:0/3543581527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbdfc074640 0x7fbdfc198ac0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:57.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.339+0000 7fbe03dff700 1 -- 172.21.15.64:0/3543581527 >> 172.21.15.64:0/3543581527 conn(0x7fbdfc0fbb80 msgr2=0x7fbdfc0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:57.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.339+0000 7fbe03dff700 1 -- 172.21.15.64:0/3543581527 shutdown_connections 2023-09-13T03:12:57.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.339+0000 7fbe03dff700 1 -- 172.21.15.64:0/3543581527 wait complete. 2023-09-13T03:12:57.358 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-13T03:12:57.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.565+0000 7f7b741ff700 1 -- 172.21.15.64:0/1795966942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b6c100610 msgr2=0x7f7b6c100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:57.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.565+0000 7f7b741ff700 1 --2- 172.21.15.64:0/1795966942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b6c100610 0x7f7b6c100a30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7b680099c0 tx=0x7f7b68009cd0 comp rx=0 tx=0).stop 2023-09-13T03:12:57.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.566+0000 7f7b741ff700 1 -- 172.21.15.64:0/1795966942 shutdown_connections 2023-09-13T03:12:57.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.566+0000 7f7b741ff700 1 --2- 172.21.15.64:0/1795966942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b6c1021a0 0x7f7b6c108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:57.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.566+0000 7f7b741ff700 1 --2- 172.21.15.64:0/1795966942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b6c100610 0x7f7b6c100a30 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:57.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.566+0000 7f7b741ff700 1 --2- 172.21.15.64:0/1795966942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b6c101800 0x7f7b6c101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:57.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.566+0000 7f7b741ff700 1 -- 172.21.15.64:0/1795966942 >> 172.21.15.64:0/1795966942 conn(0x7f7b6c0fbb80 msgr2=0x7f7b6c0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:57.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.566+0000 7f7b741ff700 1 -- 172.21.15.64:0/1795966942 shutdown_connections 2023-09-13T03:12:57.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.566+0000 7f7b741ff700 1 -- 172.21.15.64:0/1795966942 wait complete. 2023-09-13T03:12:57.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.567+0000 7f7b741ff700 1 Processor -- start 2023-09-13T03:12:57.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.567+0000 7f7b741ff700 1 -- start start 2023-09-13T03:12:57.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.567+0000 7f7b741ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b6c100610 0x7f7b6c198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:57.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.567+0000 7f7b741ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b6c101800 0x7f7b6c198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:57.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.567+0000 7f7b741ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b6c1021a0 0x7f7b6c19fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:57.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.567+0000 7f7b741ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6c1a00e0 con 0x7f7b6c101800 2023-09-13T03:12:57.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.567+0000 7f7b741ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6c1a0220 con 0x7f7b6c100610 2023-09-13T03:12:57.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.567+0000 7f7b741ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6c1a03b0 con 0x7f7b6c1021a0 2023-09-13T03:12:57.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.567+0000 7f7b7179a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b6c101800 0x7f7b6c198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:57.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.567+0000 7f7b7179a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b6c101800 0x7f7b6c198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56130/0 (socket says 172.21.15.64:56130) 2023-09-13T03:12:57.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.567+0000 7f7b7179a700 1 -- 172.21.15.64:0/704174105 learned_addr learned my addr 172.21.15.64:0/704174105 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:57.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.567+0000 7f7b71f9b700 1 --2- 172.21.15.64:0/704174105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b6c100610 0x7f7b6c198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:57.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.568+0000 7f7b7279c700 1 --2- 172.21.15.64:0/704174105 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b6c1021a0 0x7f7b6c19fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:57.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.568+0000 7f7b7179a700 1 -- 172.21.15.64:0/704174105 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b6c1021a0 msgr2=0x7f7b6c19fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:57.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.568+0000 7f7b7179a700 1 --2- 172.21.15.64:0/704174105 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b6c1021a0 0x7f7b6c19fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:57.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.568+0000 7f7b7179a700 1 -- 172.21.15.64:0/704174105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b6c100610 msgr2=0x7f7b6c198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:57.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.568+0000 7f7b7179a700 1 --2- 172.21.15.64:0/704174105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b6c100610 0x7f7b6c198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:57.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.568+0000 7f7b7179a700 1 -- 172.21.15.64:0/704174105 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b680095f0 con 0x7f7b6c101800 2023-09-13T03:12:57.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.568+0000 7f7b7179a700 1 --2- 172.21.15.64:0/704174105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b6c101800 0x7f7b6c198aa0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00e9a0 tx=0x7f7b5c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:57.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.568+0000 7f7b62ffd700 1 -- 172.21.15.64:0/704174105 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c009790 con 0x7f7b6c101800 2023-09-13T03:12:57.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.568+0000 7f7b741ff700 1 -- 172.21.15.64:0/704174105 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b6c1a0690 con 0x7f7b6c101800 2023-09-13T03:12:57.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.568+0000 7f7b62ffd700 1 -- 172.21.15.64:0/704174105 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b5c010430 con 0x7f7b6c101800 2023-09-13T03:12:57.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.568+0000 7f7b62ffd700 1 -- 172.21.15.64:0/704174105 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c018680 con 0x7f7b6c101800 2023-09-13T03:12:57.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.569+0000 7f7b741ff700 1 -- 172.21.15.64:0/704174105 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b6c1a0ba0 con 0x7f7b6c101800 2023-09-13T03:12:57.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.570+0000 7f7b741ff700 1 -- 172.21.15.64:0/704174105 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b6c192c70 con 0x7f7b6c101800 2023-09-13T03:12:57.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.572+0000 7f7b62ffd700 1 -- 172.21.15.64:0/704174105 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f7b5c0105a0 con 0x7f7b6c101800 2023-09-13T03:12:57.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.573+0000 7f7b62ffd700 1 --2- 172.21.15.64:0/704174105 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f7b580a0640 0x7f7b580a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:57.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.573+0000 7f7b62ffd700 1 -- 172.21.15.64:0/704174105 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f7b5c014070 con 0x7f7b6c101800 2023-09-13T03:12:57.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.573+0000 7f7b71f9b700 1 --2- 172.21.15.64:0/704174105 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f7b580a0640 0x7f7b580a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:57.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.574+0000 7f7b71f9b700 1 --2- 172.21.15.64:0/704174105 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f7b580a0640 0x7f7b580a2b00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7b68009990 tx=0x7f7b68005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:57.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.577+0000 7f7b62ffd700 1 -- 172.21.15.64:0/704174105 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f7b5c051e10 con 0x7f7b6c101800 2023-09-13T03:12:57.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:57 smithi064 ceph-mon[97675]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T03:12:57.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:57 smithi064 ceph-mon[97675]: from='client.25439 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T03:12:57.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:57 smithi064 ceph-mon[97675]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2023-09-13T03:12:57.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:57 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/3170845587' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:12:57.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:57 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/3543581527' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T03:12:57.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.791+0000 7f7b741ff700 1 -- 172.21.15.64:0/704174105 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7b6c02cf10 con 0x7f7b6c101800 2023-09-13T03:12:57.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.792+0000 7f7b62ffd700 1 -- 172.21.15.64:0/704174105 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7b5c095870 con 0x7f7b6c101800 2023-09-13T03:12:57.793 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-09-13T03:12:57.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.794+0000 7f7b741ff700 1 -- 172.21.15.64:0/704174105 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f7b580a0640 msgr2=0x7f7b580a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:57.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.794+0000 7f7b741ff700 1 --2- 172.21.15.64:0/704174105 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f7b580a0640 0x7f7b580a2b00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7b68009990 tx=0x7f7b68005fb0 comp rx=0 tx=0).stop 2023-09-13T03:12:57.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.794+0000 7f7b741ff700 1 -- 172.21.15.64:0/704174105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b6c101800 msgr2=0x7f7b6c198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:57.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.794+0000 7f7b741ff700 1 --2- 172.21.15.64:0/704174105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b6c101800 0x7f7b6c198aa0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00e9a0 tx=0x7f7b5c00ecb0 comp rx=0 tx=0).stop 2023-09-13T03:12:57.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.795+0000 7f7b741ff700 1 -- 172.21.15.64:0/704174105 shutdown_connections 2023-09-13T03:12:57.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.795+0000 7f7b741ff700 1 --2- 172.21.15.64:0/704174105 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f7b580a0640 0x7f7b580a2b00 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:57.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.795+0000 7f7b741ff700 1 --2- 172.21.15.64:0/704174105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b6c100610 0x7f7b6c198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:57.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.795+0000 7f7b741ff700 1 --2- 172.21.15.64:0/704174105 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b6c1021a0 0x7f7b6c19fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:57.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.795+0000 7f7b741ff700 1 --2- 172.21.15.64:0/704174105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b6c101800 0x7f7b6c198aa0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:57.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.795+0000 7f7b741ff700 1 -- 172.21.15.64:0/704174105 >> 172.21.15.64:0/704174105 conn(0x7f7b6c0fbb80 msgr2=0x7f7b6c1070e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:57.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.795+0000 7f7b741ff700 1 -- 172.21.15.64:0/704174105 shutdown_connections 2023-09-13T03:12:57.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:57.795+0000 7f7b741ff700 1 -- 172.21.15.64:0/704174105 wait complete. 2023-09-13T03:12:57.814 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-13T03:12:57.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:57 smithi170 ceph-mon[134328]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T03:12:57.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:57 smithi170 ceph-mon[134328]: from='client.25439 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T03:12:57.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:57 smithi170 ceph-mon[134328]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2023-09-13T03:12:57.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:57 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/3170845587' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:12:57.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:57 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/3543581527' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T03:12:57.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:57 smithi142 ceph-mon[134082]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T03:12:57.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:57 smithi142 ceph-mon[134082]: from='client.25439 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-13T03:12:57.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:57 smithi142 ceph-mon[134082]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2023-09-13T03:12:57.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:57 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/3170845587' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:12:57.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:57 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/3543581527' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T03:12:58.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.022+0000 7f222edff700 1 -- 172.21.15.64:0/3634557300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2228101070 msgr2=0x7f2228103460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:58.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.022+0000 7f222edff700 1 --2- 172.21.15.64:0/3634557300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2228101070 0x7f2228103460 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f221c009990 tx=0x7f221c009ca0 comp rx=0 tx=0).stop 2023-09-13T03:12:58.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.022+0000 7f222edff700 1 -- 172.21.15.64:0/3634557300 shutdown_connections 2023-09-13T03:12:58.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.022+0000 7f222edff700 1 --2- 172.21.15.64:0/3634557300 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22281062d0 0x7f2228108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:58.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.022+0000 7f222edff700 1 --2- 172.21.15.64:0/3634557300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2228101070 0x7f2228103460 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:58.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.022+0000 7f222edff700 1 --2- 172.21.15.64:0/3634557300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22281039a0 0x7f2228105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:58.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.022+0000 7f222edff700 1 -- 172.21.15.64:0/3634557300 >> 172.21.15.64:0/3634557300 conn(0x7f22280faa60 msgr2=0x7f22280fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:58.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.022+0000 7f222edff700 1 -- 172.21.15.64:0/3634557300 shutdown_connections 2023-09-13T03:12:58.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.022+0000 7f222edff700 1 -- 172.21.15.64:0/3634557300 wait complete. 2023-09-13T03:12:58.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.023+0000 7f222edff700 1 Processor -- start 2023-09-13T03:12:58.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.023+0000 7f222edff700 1 -- start start 2023-09-13T03:12:58.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.023+0000 7f222edff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2228101070 0x7f2228196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:58.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.023+0000 7f222edff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22281039a0 0x7f2228196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:58.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.023+0000 7f222edff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22281062d0 0x7f222819d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:58.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.023+0000 7f222edff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222819dec0 con 0x7f2228101070 2023-09-13T03:12:58.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.024+0000 7f222edff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222819e000 con 0x7f22281039a0 2023-09-13T03:12:58.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.024+0000 7f222edff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222819e190 con 0x7f22281062d0 2023-09-13T03:12:58.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.024+0000 7f222cb9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2228101070 0x7f2228196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:58.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.024+0000 7f222cb9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2228101070 0x7f2228196340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56152/0 (socket says 172.21.15.64:56152) 2023-09-13T03:12:58.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.024+0000 7f222cb9b700 1 -- 172.21.15.64:0/1461123726 learned_addr learned my addr 172.21.15.64:0/1461123726 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:58.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.024+0000 7f222d39c700 1 --2- 172.21.15.64:0/1461123726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22281062d0 0x7f222819d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:58.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.024+0000 7f2227fff700 1 --2- 172.21.15.64:0/1461123726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22281039a0 0x7f2228196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:58.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.024+0000 7f222cb9b700 1 -- 172.21.15.64:0/1461123726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22281062d0 msgr2=0x7f222819d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:58.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.024+0000 7f222cb9b700 1 --2- 172.21.15.64:0/1461123726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22281062d0 0x7f222819d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:58.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.024+0000 7f222cb9b700 1 -- 172.21.15.64:0/1461123726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22281039a0 msgr2=0x7f2228196880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:58.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.024+0000 7f222cb9b700 1 --2- 172.21.15.64:0/1461123726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22281039a0 0x7f2228196880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:58.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.024+0000 7f222cb9b700 1 -- 172.21.15.64:0/1461123726 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f221c0095f0 con 0x7f2228101070 2023-09-13T03:12:58.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.024+0000 7f222cb9b700 1 --2- 172.21.15.64:0/1461123726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2228101070 0x7f2228196340 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f221c004730 tx=0x7f221c004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:58.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.025+0000 7f2225ffb700 1 -- 172.21.15.64:0/1461123726 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c01d070 con 0x7f2228101070 2023-09-13T03:12:58.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.025+0000 7f2225ffb700 1 -- 172.21.15.64:0/1461123726 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f221c004c90 con 0x7f2228101070 2023-09-13T03:12:58.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.025+0000 7f222edff700 1 -- 172.21.15.64:0/1461123726 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f222819e410 con 0x7f2228101070 2023-09-13T03:12:58.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.025+0000 7f2225ffb700 1 -- 172.21.15.64:0/1461123726 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c0054b0 con 0x7f2228101070 2023-09-13T03:12:58.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.025+0000 7f222edff700 1 -- 172.21.15.64:0/1461123726 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f222819e7d0 con 0x7f2228101070 2023-09-13T03:12:58.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.026+0000 7f222edff700 1 -- 172.21.15.64:0/1461123726 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22280fc5e0 con 0x7f2228101070 2023-09-13T03:12:58.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.028+0000 7f2225ffb700 1 -- 172.21.15.64:0/1461123726 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f221c00fbc0 con 0x7f2228101070 2023-09-13T03:12:58.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.029+0000 7f2225ffb700 1 --2- 172.21.15.64:0/1461123726 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f22180a0640 0x7f22180a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:58.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.029+0000 7f2225ffb700 1 -- 172.21.15.64:0/1461123726 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f221c0cce40 con 0x7f2228101070 2023-09-13T03:12:58.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.029+0000 7f2227fff700 1 --2- 172.21.15.64:0/1461123726 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f22180a0640 0x7f22180a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:58.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.030+0000 7f2227fff700 1 --2- 172.21.15.64:0/1461123726 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f22180a0640 0x7f22180a2b00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2214007c60 tx=0x7f22140073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:58.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.033+0000 7f2225ffb700 1 -- 172.21.15.64:0/1461123726 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f221c05ae10 con 0x7f2228101070 2023-09-13T03:12:58.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.242+0000 7f222edff700 1 -- 172.21.15.64:0/1461123726 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f222802cf10 con 0x7f2228101070 2023-09-13T03:12:58.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.242+0000 7f2225ffb700 1 -- 172.21.15.64:0/1461123726 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f221c0969d0 con 0x7f2228101070 2023-09-13T03:12:58.243 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-09-13T03:12:58.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.245+0000 7f222edff700 1 -- 172.21.15.64:0/1461123726 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f22180a0640 msgr2=0x7f22180a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:58.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.245+0000 7f222edff700 1 --2- 172.21.15.64:0/1461123726 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f22180a0640 0x7f22180a2b00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2214007c60 tx=0x7f22140073d0 comp rx=0 tx=0).stop 2023-09-13T03:12:58.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.245+0000 7f222edff700 1 -- 172.21.15.64:0/1461123726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2228101070 msgr2=0x7f2228196340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:58.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.245+0000 7f222edff700 1 --2- 172.21.15.64:0/1461123726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2228101070 0x7f2228196340 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f221c004730 tx=0x7f221c004760 comp rx=0 tx=0).stop 2023-09-13T03:12:58.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.245+0000 7f222edff700 1 -- 172.21.15.64:0/1461123726 shutdown_connections 2023-09-13T03:12:58.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.245+0000 7f222edff700 1 --2- 172.21.15.64:0/1461123726 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f22180a0640 0x7f22180a2b00 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:58.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.245+0000 7f222edff700 1 --2- 172.21.15.64:0/1461123726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22281062d0 0x7f222819d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:58.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.245+0000 7f222edff700 1 --2- 172.21.15.64:0/1461123726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2228101070 0x7f2228196340 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:58.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.245+0000 7f222edff700 1 --2- 172.21.15.64:0/1461123726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22281039a0 0x7f2228196880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:58.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.245+0000 7f222edff700 1 -- 172.21.15.64:0/1461123726 >> 172.21.15.64:0/1461123726 conn(0x7f22280faa60 msgr2=0x7f2228101ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:58.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.245+0000 7f222edff700 1 -- 172.21.15.64:0/1461123726 shutdown_connections 2023-09-13T03:12:58.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.245+0000 7f222edff700 1 -- 172.21.15.64:0/1461123726 wait complete. 2023-09-13T03:12:58.264 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T03:12:58.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.468+0000 7f5f50bff700 1 -- 172.21.15.64:0/2990212789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f4c100610 msgr2=0x7f5f4c100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:58.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.468+0000 7f5f50bff700 1 --2- 172.21.15.64:0/2990212789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f4c100610 0x7f5f4c100a30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c0099c0 tx=0x7f5f3c009cd0 comp rx=0 tx=0).stop 2023-09-13T03:12:58.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.468+0000 7f5f50bff700 1 -- 172.21.15.64:0/2990212789 shutdown_connections 2023-09-13T03:12:58.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.468+0000 7f5f50bff700 1 --2- 172.21.15.64:0/2990212789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f4c1021a0 0x7f5f4c108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:58.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.468+0000 7f5f50bff700 1 --2- 172.21.15.64:0/2990212789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f4c100610 0x7f5f4c100a30 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:58.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.468+0000 7f5f50bff700 1 --2- 172.21.15.64:0/2990212789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f4c101800 0x7f5f4c101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:58.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.468+0000 7f5f50bff700 1 -- 172.21.15.64:0/2990212789 >> 172.21.15.64:0/2990212789 conn(0x7f5f4c0fbb40 msgr2=0x7f5f4c0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:58.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.468+0000 7f5f50bff700 1 -- 172.21.15.64:0/2990212789 shutdown_connections 2023-09-13T03:12:58.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.468+0000 7f5f50bff700 1 -- 172.21.15.64:0/2990212789 wait complete. 2023-09-13T03:12:58.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.469+0000 7f5f50bff700 1 Processor -- start 2023-09-13T03:12:58.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.469+0000 7f5f50bff700 1 -- start start 2023-09-13T03:12:58.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.469+0000 7f5f50bff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f4c100610 0x7f5f4c196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:58.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.469+0000 7f5f50bff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f4c101800 0x7f5f4c196850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:58.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.469+0000 7f5f50bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f4c1021a0 0x7f5f4c19d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:58.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.469+0000 7f5f50bff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c19de90 con 0x7f5f4c100610 2023-09-13T03:12:58.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.470+0000 7f5f50bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c19dfd0 con 0x7f5f4c1021a0 2023-09-13T03:12:58.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.470+0000 7f5f50bff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c19e160 con 0x7f5f4c101800 2023-09-13T03:12:58.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.470+0000 7f5f4a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f4c100610 0x7f5f4c196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:58.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.470+0000 7f5f4a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f4c100610 0x7f5f4c196310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56182/0 (socket says 172.21.15.64:56182) 2023-09-13T03:12:58.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.470+0000 7f5f4a59c700 1 -- 172.21.15.64:0/757725150 learned_addr learned my addr 172.21.15.64:0/757725150 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:58.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.470+0000 7f5f49d9b700 1 --2- 172.21.15.64:0/757725150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f4c101800 0x7f5f4c196850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:58.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.470+0000 7f5f4ad9d700 1 --2- 172.21.15.64:0/757725150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f4c1021a0 0x7f5f4c19d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:58.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.470+0000 7f5f4a59c700 1 -- 172.21.15.64:0/757725150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f4c101800 msgr2=0x7f5f4c196850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:58.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.470+0000 7f5f4a59c700 1 --2- 172.21.15.64:0/757725150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f4c101800 0x7f5f4c196850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:58.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.470+0000 7f5f4a59c700 1 -- 172.21.15.64:0/757725150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f4c1021a0 msgr2=0x7f5f4c19d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:58.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.470+0000 7f5f4a59c700 1 --2- 172.21.15.64:0/757725150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f4c1021a0 0x7f5f4c19d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:58.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.470+0000 7f5f4a59c700 1 -- 172.21.15.64:0/757725150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f3c0095f0 con 0x7f5f4c100610 2023-09-13T03:12:58.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.470+0000 7f5f4a59c700 1 --2- 172.21.15.64:0/757725150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f4c100610 0x7f5f4c196310 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c009990 tx=0x7f5f3c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:58.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.471+0000 7f5f3b7fe700 1 -- 172.21.15.64:0/757725150 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c01d070 con 0x7f5f4c100610 2023-09-13T03:12:58.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.471+0000 7f5f3b7fe700 1 -- 172.21.15.64:0/757725150 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f3c0054f0 con 0x7f5f4c100610 2023-09-13T03:12:58.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.471+0000 7f5f50bff700 1 -- 172.21.15.64:0/757725150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f4c19e3e0 con 0x7f5f4c100610 2023-09-13T03:12:58.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.471+0000 7f5f3b7fe700 1 -- 172.21.15.64:0/757725150 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c004bd0 con 0x7f5f4c100610 2023-09-13T03:12:58.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.471+0000 7f5f50bff700 1 -- 172.21.15.64:0/757725150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f4c19e860 con 0x7f5f4c100610 2023-09-13T03:12:58.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.472+0000 7f5f50bff700 1 -- 172.21.15.64:0/757725150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f4c107ba0 con 0x7f5f4c100610 2023-09-13T03:12:58.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.474+0000 7f5f3b7fe700 1 -- 172.21.15.64:0/757725150 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f5f3c00fb80 con 0x7f5f4c100610 2023-09-13T03:12:58.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.477+0000 7f5f3b7fe700 1 --2- 172.21.15.64:0/757725150 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f5f340a0710 0x7f5f340a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:58.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.477+0000 7f5f3b7fe700 1 -- 172.21.15.64:0/757725150 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f5f3c0ccdd0 con 0x7f5f4c100610 2023-09-13T03:12:58.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.477+0000 7f5f49d9b700 1 --2- 172.21.15.64:0/757725150 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f5f340a0710 0x7f5f340a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:58.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.477+0000 7f5f49d9b700 1 --2- 172.21.15.64:0/757725150 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f5f340a0710 0x7f5f340a2bd0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5f40007c60 tx=0x7f5f400073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:58.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.479+0000 7f5f3b7fe700 1 -- 172.21.15.64:0/757725150 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5f3c096960 con 0x7f5f4c100610 2023-09-13T03:12:58.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.741+0000 7f5f50bff700 1 -- 172.21.15.64:0/757725150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f4c10c0d0 con 0x7f5f4c100610 2023-09-13T03:12:58.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.742+0000 7f5f3b7fe700 1 -- 172.21.15.64:0/757725150 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11366 (secure 0 0 0) 0x7f5f3c026090 con 0x7f5f4c100610 2023-09-13T03:12:58.743 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:12:58.743 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2234182985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2234182985},{"type":"v1","addr":"172.21.15.64:6835","nonce":2234182985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4053841873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4053841873},{"type":"v1","addr":"172.21.15.64:6837","nonce":4053841873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-13T03:11:10.490260+0000","modified":"2023-09-13T03:11:18.230740+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24686,"mds_1":24649,"mds_2":24635,"mds_3":24634,"mds_4":24622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6839/99523374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":99523374},{"type":"v1","addr":"172.21.15.64:6839","nonce":99523374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14649":{"gid":14649,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6841/1866582875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":1866582875},{"type":"v1","addr":"172.21.15.64:6841","nonce":1866582875}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/101906999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":101906999},{"type":"v1","addr":"172.21.15.142:6833","nonce":101906999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24574":{"gid":24574,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/1897366227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1897366227},{"type":"v1","addr":"172.21.15.170:6833","nonce":1897366227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24611":{"gid":24611,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2167779918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2167779918},{"type":"v1","addr":"172.21.15.142:6835","nonce":2167779918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24622":{"gid":24622,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6835/1493371121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1493371121},{"type":"v1","addr":"172.21.15.170:6835","nonce":1493371121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24634":{"gid":24634,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.170:6837/1726587275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":1726587275},{"type":"v1","addr":"172.21.15.170:6837","nonce":1726587275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24635":{"gid":24635,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.142:6837/741173268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":741173268},{"type":"v1","addr":"172.21.15.142:6837","nonce":741173268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24649":{"gid":24649,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.170:6839/2189703596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2189703596},{"type":"v1","addr":"172.21.15.170:6839","nonce":2189703596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24686":{"gid":24686,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/3645998616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645998616},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645998616}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-13T03:12:58.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.745+0000 7f5f50bff700 1 -- 172.21.15.64:0/757725150 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f5f340a0710 msgr2=0x7f5f340a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:58.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.745+0000 7f5f50bff700 1 --2- 172.21.15.64:0/757725150 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f5f340a0710 0x7f5f340a2bd0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5f40007c60 tx=0x7f5f400073d0 comp rx=0 tx=0).stop 2023-09-13T03:12:58.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.745+0000 7f5f50bff700 1 -- 172.21.15.64:0/757725150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f4c100610 msgr2=0x7f5f4c196310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:58.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.745+0000 7f5f50bff700 1 --2- 172.21.15.64:0/757725150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f4c100610 0x7f5f4c196310 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c009990 tx=0x7f5f3c00f690 comp rx=0 tx=0).stop 2023-09-13T03:12:58.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.745+0000 7f5f50bff700 1 -- 172.21.15.64:0/757725150 shutdown_connections 2023-09-13T03:12:58.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.745+0000 7f5f50bff700 1 --2- 172.21.15.64:0/757725150 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f5f340a0710 0x7f5f340a2bd0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:58.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.745+0000 7f5f50bff700 1 --2- 172.21.15.64:0/757725150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f4c1021a0 0x7f5f4c19d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:58.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.745+0000 7f5f50bff700 1 --2- 172.21.15.64:0/757725150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f4c100610 0x7f5f4c196310 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:58.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.745+0000 7f5f50bff700 1 --2- 172.21.15.64:0/757725150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f4c101800 0x7f5f4c196850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:58.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.745+0000 7f5f50bff700 1 -- 172.21.15.64:0/757725150 >> 172.21.15.64:0/757725150 conn(0x7f5f4c0fbb40 msgr2=0x7f5f4c104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:58.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.745+0000 7f5f50bff700 1 -- 172.21.15.64:0/757725150 shutdown_connections 2023-09-13T03:12:58.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.745+0000 7f5f50bff700 1 -- 172.21.15.64:0/757725150 wait complete. 2023-09-13T03:12:58.748 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 26 2023-09-13T03:12:58.764 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-13T03:12:58.825 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:58 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/704174105' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T03:12:58.826 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:58 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/1461123726' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T03:12:58.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:58 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/704174105' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T03:12:58.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:58 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/1461123726' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T03:12:58.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.967+0000 7f37b19ff700 1 -- 172.21.15.64:0/3968287975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37ac102190 msgr2=0x7f37ac108a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:58.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.967+0000 7f37b19ff700 1 --2- 172.21.15.64:0/3968287975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37ac102190 0x7f37ac108a40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f37a400b0d0 tx=0x7f37a400b3e0 comp rx=0 tx=0).stop 2023-09-13T03:12:58.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.968+0000 7f37b19ff700 1 -- 172.21.15.64:0/3968287975 shutdown_connections 2023-09-13T03:12:58.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.968+0000 7f37b19ff700 1 --2- 172.21.15.64:0/3968287975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37ac100600 0x7f37ac100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:58.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.968+0000 7f37b19ff700 1 --2- 172.21.15.64:0/3968287975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37ac102190 0x7f37ac108a40 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:58.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.968+0000 7f37b19ff700 1 --2- 172.21.15.64:0/3968287975 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37ac1017f0 0x7f37ac101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:58.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.968+0000 7f37b19ff700 1 -- 172.21.15.64:0/3968287975 >> 172.21.15.64:0/3968287975 conn(0x7f37ac0fbb30 msgr2=0x7f37ac0fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:58.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.968+0000 7f37b19ff700 1 -- 172.21.15.64:0/3968287975 shutdown_connections 2023-09-13T03:12:58.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.968+0000 7f37b19ff700 1 -- 172.21.15.64:0/3968287975 wait complete. 2023-09-13T03:12:58.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.969+0000 7f37b19ff700 1 Processor -- start 2023-09-13T03:12:58.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.969+0000 7f37b19ff700 1 -- start start 2023-09-13T03:12:58.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.970+0000 7f37b19ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37ac100600 0x7f37ac196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:58.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.970+0000 7f37b19ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37ac1017f0 0x7f37ac1968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:58.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.970+0000 7f37b19ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37ac102190 0x7f37ac19d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:58.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.970+0000 7f37b19ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac19dee0 con 0x7f37ac1017f0 2023-09-13T03:12:58.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.970+0000 7f37b19ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac19e020 con 0x7f37ac102190 2023-09-13T03:12:58.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.970+0000 7f37b19ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac19e1b0 con 0x7f37ac100600 2023-09-13T03:12:58.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.970+0000 7f37aa7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37ac1017f0 0x7f37ac1968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:58.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.970+0000 7f37aa7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37ac1017f0 0x7f37ac1968a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56194/0 (socket says 172.21.15.64:56194) 2023-09-13T03:12:58.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.970+0000 7f37aa7fc700 1 -- 172.21.15.64:0/3534189000 learned_addr learned my addr 172.21.15.64:0/3534189000 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:58.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.970+0000 7f37aaffd700 1 --2- 172.21.15.64:0/3534189000 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37ac100600 0x7f37ac196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:58.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.970+0000 7f37ab7fe700 1 --2- 172.21.15.64:0/3534189000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37ac102190 0x7f37ac19d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:58.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.971+0000 7f37aa7fc700 1 -- 172.21.15.64:0/3534189000 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37ac100600 msgr2=0x7f37ac196360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:58.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.971+0000 7f37aa7fc700 1 --2- 172.21.15.64:0/3534189000 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37ac100600 0x7f37ac196360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:58.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.971+0000 7f37aa7fc700 1 -- 172.21.15.64:0/3534189000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37ac102190 msgr2=0x7f37ac19d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:58.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.971+0000 7f37aa7fc700 1 --2- 172.21.15.64:0/3534189000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37ac102190 0x7f37ac19d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:58.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.971+0000 7f37aa7fc700 1 -- 172.21.15.64:0/3534189000 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a4009d00 con 0x7f37ac1017f0 2023-09-13T03:12:58.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.971+0000 7f37aa7fc700 1 --2- 172.21.15.64:0/3534189000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37ac1017f0 0x7f37ac1968a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f37a000e9a0 tx=0x7f37a000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:58.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.971+0000 7f37b09fd700 1 -- 172.21.15.64:0/3534189000 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a0009790 con 0x7f37ac1017f0 2023-09-13T03:12:58.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.971+0000 7f37b19ff700 1 -- 172.21.15.64:0/3534189000 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37ac19e490 con 0x7f37ac1017f0 2023-09-13T03:12:58.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.971+0000 7f37b09fd700 1 -- 172.21.15.64:0/3534189000 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37a0010430 con 0x7f37ac1017f0 2023-09-13T03:12:58.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.971+0000 7f37b09fd700 1 -- 172.21.15.64:0/3534189000 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a0018680 con 0x7f37ac1017f0 2023-09-13T03:12:58.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.971+0000 7f37b19ff700 1 -- 172.21.15.64:0/3534189000 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37ac19e9a0 con 0x7f37ac1017f0 2023-09-13T03:12:58.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.972+0000 7f37b19ff700 1 -- 172.21.15.64:0/3534189000 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37ac107af0 con 0x7f37ac1017f0 2023-09-13T03:12:58.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.974+0000 7f37b09fd700 1 -- 172.21.15.64:0/3534189000 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f37a00105a0 con 0x7f37ac1017f0 2023-09-13T03:12:58.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.976+0000 7f37b09fd700 1 --2- 172.21.15.64:0/3534189000 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f37940a0690 0x7f37940a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:58.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.976+0000 7f37b09fd700 1 -- 172.21.15.64:0/3534189000 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f37a0014070 con 0x7f37ac1017f0 2023-09-13T03:12:58.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.976+0000 7f37aaffd700 1 --2- 172.21.15.64:0/3534189000 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f37940a0690 0x7f37940a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:58.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.977+0000 7f37aaffd700 1 --2- 172.21.15.64:0/3534189000 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f37940a0690 0x7f37940a2b50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f37ac101650 tx=0x7f379c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:58.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:58.980+0000 7f37b09fd700 1 -- 172.21.15.64:0/3534189000 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f37a0051e10 con 0x7f37ac1017f0 2023-09-13T03:12:58.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:58 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/704174105' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T03:12:58.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:58 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/1461123726' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T03:12:59.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.191+0000 7f37b19ff700 1 -- 172.21.15.64:0/3534189000 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f37ac10be20 con 0x7f37ac1017f0 2023-09-13T03:12:59.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.193+0000 7f37b09fd700 1 -- 172.21.15.64:0/3534189000 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22473 (secure 0 0 0) 0x7f37a0095870 con 0x7f37ac1017f0 2023-09-13T03:12:59.194 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:12:59.194 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":85,"fsid":"f8dbced4-51e1-11ee-9ab7-7b867c8bd7da","created":"2023-09-13T03:03:24.167202+0000","modified":"2023-09-13T03:11:24.258871+0000","last_up_change":"2023-09-13T03:10:22.503529+0000","last_in_change":"2023-09-13T03:10:05.678720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T03:06:58.987479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-13T03:11:08.589202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-13T03:11:09.924659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3997c114-241b-48c4-8feb-653c99209470","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6803","nonce":3940892135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6805","nonce":3940892135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6809","nonce":3940892135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6807","nonce":3940892135}]},"public_addr":"172.21.15.64:6803/3940892135","cluster_addr":"172.21.15.64:6805/3940892135","heartbeat_back_addr":"172.21.15.64:6809/3940892135","heartbeat_front_addr":"172.21.15.64:6807/3940892135","state":["exists","up"]},{"osd":1,"uuid":"94cfae85-322e-4ed2-a6e6-e40349bb14bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6801","nonce":4188149333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6803","nonce":4188149333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6807","nonce":4188149333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6805","nonce":4188149333}]},"public_addr":"172.21.15.142:6801/4188149333","cluster_addr":"172.21.15.142:6803/4188149333","heartbeat_back_addr":"172.21.15.142:6807/4188149333","heartbeat_front_addr":"172.21.15.142:6805/4188149333","state":["exists","up"]},{"osd":2,"uuid":"b3aae17e-96db-4008-8c27-f733c46b30ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6801","nonce":2406669338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6803","nonce":2406669338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6807","nonce":2406669338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6805","nonce":2406669338}]},"public_addr":"172.21.15.170:6801/2406669338","cluster_addr":"172.21.15.170:6803/2406669338","heartbeat_back_addr":"172.21.15.170:6807/2406669338","heartbeat_front_addr":"172.21.15.170:6805/2406669338","state":["exists","up"]},{"osd":3,"uuid":"7fa8d8cb-2ab7-48a7-8b49-5e2903d6779f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6811","nonce":3558318707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6813","nonce":3558318707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6817","nonce":3558318707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6815","nonce":3558318707}]},"public_addr":"172.21.15.64:6811/3558318707","cluster_addr":"172.21.15.64:6813/3558318707","heartbeat_back_addr":"172.21.15.64:6817/3558318707","heartbeat_front_addr":"172.21.15.64:6815/3558318707","state":["exists","up"]},{"osd":4,"uuid":"8c7a9c9f-f919-48d3-9db2-214a25bb5603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6809","nonce":1811142576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6811","nonce":1811142576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6815","nonce":1811142576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6813","nonce":1811142576}]},"public_addr":"172.21.15.142:6809/1811142576","cluster_addr":"172.21.15.142:6811/1811142576","heartbeat_back_addr":"172.21.15.142:6815/1811142576","heartbeat_front_addr":"172.21.15.142:6813/1811142576","state":["exists","up"]},{"osd":5,"uuid":"0c6101c6-2065-4022-a751-d3f216d7f478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6809","nonce":2903841453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6811","nonce":2903841453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6815","nonce":2903841453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6813","nonce":2903841453}]},"public_addr":"172.21.15.170:6809/2903841453","cluster_addr":"172.21.15.170:6811/2903841453","heartbeat_back_addr":"172.21.15.170:6815/2903841453","heartbeat_front_addr":"172.21.15.170:6813/2903841453","state":["exists","up"]},{"osd":6,"uuid":"5cf3e52e-d155-41b1-9194-db047d26a953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6819","nonce":1436584490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6821","nonce":1436584490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6825","nonce":1436584490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6823","nonce":1436584490}]},"public_addr":"172.21.15.64:6819/1436584490","cluster_addr":"172.21.15.64:6821/1436584490","heartbeat_back_addr":"172.21.15.64:6825/1436584490","heartbeat_front_addr":"172.21.15.64:6823/1436584490","state":["exists","up"]},{"osd":7,"uuid":"e97323e3-badd-41a7-af9e-e78a81a64046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6817","nonce":3726793534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6819","nonce":3726793534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6823","nonce":3726793534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6821","nonce":3726793534}]},"public_addr":"172.21.15.142:6817/3726793534","cluster_addr":"172.21.15.142:6819/3726793534","heartbeat_back_addr":"172.21.15.142:6823/3726793534","heartbeat_front_addr":"172.21.15.142:6821/3726793534","state":["exists","up"]},{"osd":8,"uuid":"83d2835c-ed26-493d-9222-c2894ec0e071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6817","nonce":4272513873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6819","nonce":4272513873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6823","nonce":4272513873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6821","nonce":4272513873}]},"public_addr":"172.21.15.170:6817/4272513873","cluster_addr":"172.21.15.170:6819/4272513873","heartbeat_back_addr":"172.21.15.170:6823/4272513873","heartbeat_front_addr":"172.21.15.170:6821/4272513873","state":["exists","up"]},{"osd":9,"uuid":"cb936bdd-1bec-4236-aa2e-a24adcac11e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6827","nonce":2641175211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6829","nonce":2641175211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6833","nonce":2641175211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6831","nonce":2641175211}]},"public_addr":"172.21.15.64:6827/2641175211","cluster_addr":"172.21.15.64:6829/2641175211","heartbeat_back_addr":"172.21.15.64:6833/2641175211","heartbeat_front_addr":"172.21.15.64:6831/2641175211","state":["exists","up"]},{"osd":10,"uuid":"85512a1c-6171-43d4-9c15-c588a8b71be0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6825","nonce":936499217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6827","nonce":936499217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6831","nonce":936499217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6829","nonce":936499217}]},"public_addr":"172.21.15.142:6825/936499217","cluster_addr":"172.21.15.142:6827/936499217","heartbeat_back_addr":"172.21.15.142:6831/936499217","heartbeat_front_addr":"172.21.15.142:6829/936499217","state":["exists","up"]},{"osd":11,"uuid":"9b8417dc-5d6c-4917-94a9-b032c5bc33d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6825","nonce":3605017060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6827","nonce":3605017060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6831","nonce":3605017060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6829","nonce":3605017060}]},"public_addr":"172.21.15.170:6825/3605017060","cluster_addr":"172.21.15.170:6827/3605017060","heartbeat_back_addr":"172.21.15.170:6831/3605017060","heartbeat_front_addr":"172.21.15.170:6829/3605017060","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:14.229044+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:36.147777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:56.655298+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:07:23.107741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:07:44.663770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:05.577432+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:31.168031+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:52.791055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:09:13.658546+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:09:39.150360+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:10:00.601735+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:10:21.407555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6800/2955227347":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/1373223118":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/271194613":"2023-09-14T03:04:22.355817+0000","172.21.15.64:6801/3998091347":"2023-09-14T03:03:56.442541+0000","172.21.15.64:6800/3998091347":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/920818572":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/735789005":"2023-09-14T03:03:56.442541+0000","172.21.15.64:6801/2955227347":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/1975923178":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/1500151178":"2023-09-14T03:04:22.355817+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T03:12:59.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.195+0000 7f37b19ff700 1 -- 172.21.15.64:0/3534189000 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f37940a0690 msgr2=0x7f37940a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:59.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.195+0000 7f37b19ff700 1 --2- 172.21.15.64:0/3534189000 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f37940a0690 0x7f37940a2b50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f37ac101650 tx=0x7f379c00b410 comp rx=0 tx=0).stop 2023-09-13T03:12:59.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.195+0000 7f37b19ff700 1 -- 172.21.15.64:0/3534189000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37ac1017f0 msgr2=0x7f37ac1968a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:59.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.195+0000 7f37b19ff700 1 --2- 172.21.15.64:0/3534189000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37ac1017f0 0x7f37ac1968a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f37a000e9a0 tx=0x7f37a000ecb0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.196+0000 7f37b19ff700 1 -- 172.21.15.64:0/3534189000 shutdown_connections 2023-09-13T03:12:59.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.196+0000 7f37b19ff700 1 --2- 172.21.15.64:0/3534189000 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f37940a0690 0x7f37940a2b50 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.196+0000 7f37b19ff700 1 --2- 172.21.15.64:0/3534189000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37ac102190 0x7f37ac19d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.196+0000 7f37b19ff700 1 --2- 172.21.15.64:0/3534189000 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37ac100600 0x7f37ac196360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.196+0000 7f37b19ff700 1 --2- 172.21.15.64:0/3534189000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37ac1017f0 0x7f37ac1968a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.196+0000 7f37b19ff700 1 -- 172.21.15.64:0/3534189000 >> 172.21.15.64:0/3534189000 conn(0x7f37ac0fbb30 msgr2=0x7f37ac104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:59.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.196+0000 7f37b19ff700 1 -- 172.21.15.64:0/3534189000 shutdown_connections 2023-09-13T03:12:59.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.196+0000 7f37b19ff700 1 -- 172.21.15.64:0/3534189000 wait complete. 2023-09-13T03:12:59.216 DEBUG:tasks.fwd_scrub:Yielding 2023-09-13T03:12:59.216 INFO:teuthology.run_tasks:Running task exec... 2023-09-13T03:12:59.227 INFO:teuthology.task.exec:Executing custom commands... 2023-09-13T03:12:59.227 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi064.front.sepia.ceph.com 2023-09-13T03:12:59.227 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2023-09-13T03:12:59.228 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-09-13T03:12:59.228 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-09-13T03:12:59.228 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-09-13T03:12:59.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.430+0000 7fb8f77ff700 1 -- 172.21.15.64:0/49145789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8f01084f0 msgr2=0x7fb8f010a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:59.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.430+0000 7fb8f77ff700 1 --2- 172.21.15.64:0/49145789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8f01084f0 0x7fb8f010a980 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8009f90 tx=0x7fb8e800b330 comp rx=0 tx=0).stop 2023-09-13T03:12:59.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.430+0000 7fb8f77ff700 1 -- 172.21.15.64:0/49145789 shutdown_connections 2023-09-13T03:12:59.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.430+0000 7fb8f77ff700 1 --2- 172.21.15.64:0/49145789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8f01084f0 0x7fb8f010a980 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.430+0000 7fb8f77ff700 1 --2- 172.21.15.64:0/49145789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8f0074640 0x7fb8f0072a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.430+0000 7fb8f77ff700 1 --2- 172.21.15.64:0/49145789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8f0073060 0x7fb8f00734e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.430+0000 7fb8f77ff700 1 -- 172.21.15.64:0/49145789 >> 172.21.15.64:0/49145789 conn(0x7fb8f00fbb80 msgr2=0x7fb8f00fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:59.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.430+0000 7fb8f77ff700 1 -- 172.21.15.64:0/49145789 shutdown_connections 2023-09-13T03:12:59.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.430+0000 7fb8f77ff700 1 -- 172.21.15.64:0/49145789 wait complete. 2023-09-13T03:12:59.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.431+0000 7fb8f77ff700 1 Processor -- start 2023-09-13T03:12:59.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.431+0000 7fb8f77ff700 1 -- start start 2023-09-13T03:12:59.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.432+0000 7fb8f77ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8f0073060 0x7fb8f01986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:59.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.432+0000 7fb8f77ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8f0074640 0x7fb8f0198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:59.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.432+0000 7fb8f77ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8f01084f0 0x7fb8f0199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:59.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.432+0000 7fb8f77ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f019a0f0 con 0x7fb8f0073060 2023-09-13T03:12:59.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.432+0000 7fb8f77ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f019a260 con 0x7fb8f01084f0 2023-09-13T03:12:59.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.432+0000 7fb8f77ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f019a420 con 0x7fb8f0074640 2023-09-13T03:12:59.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.432+0000 7fb8f559b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8f0073060 0x7fb8f01986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:59.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.432+0000 7fb8f559b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8f0073060 0x7fb8f01986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56202/0 (socket says 172.21.15.64:56202) 2023-09-13T03:12:59.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.432+0000 7fb8f559b700 1 -- 172.21.15.64:0/377572512 learned_addr learned my addr 172.21.15.64:0/377572512 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:59.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.432+0000 7fb8f5d9c700 1 --2- 172.21.15.64:0/377572512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8f01084f0 0x7fb8f0199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:59.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.432+0000 7fb8f4d9a700 1 --2- 172.21.15.64:0/377572512 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8f0074640 0x7fb8f0198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:59.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.432+0000 7fb8f559b700 1 -- 172.21.15.64:0/377572512 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8f0074640 msgr2=0x7fb8f0198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:59.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.432+0000 7fb8f559b700 1 --2- 172.21.15.64:0/377572512 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8f0074640 0x7fb8f0198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.432+0000 7fb8f559b700 1 -- 172.21.15.64:0/377572512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8f01084f0 msgr2=0x7fb8f0199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:59.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.432+0000 7fb8f559b700 1 --2- 172.21.15.64:0/377572512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8f01084f0 0x7fb8f0199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.432+0000 7fb8f559b700 1 -- 172.21.15.64:0/377572512 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e00095f0 con 0x7fb8f0073060 2023-09-13T03:12:59.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.432+0000 7fb8f5d9c700 1 --2- 172.21.15.64:0/377572512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8f01084f0 0x7fb8f0199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:59.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.433+0000 7fb8f559b700 1 --2- 172.21.15.64:0/377572512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8f0073060 0x7fb8f01986d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00e9a0 tx=0x7fb8ec00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:59.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.433+0000 7fb8e67fc700 1 -- 172.21.15.64:0/377572512 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec009790 con 0x7fb8f0073060 2023-09-13T03:12:59.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.433+0000 7fb8e67fc700 1 -- 172.21.15.64:0/377572512 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb8ec010430 con 0x7fb8f0073060 2023-09-13T03:12:59.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.433+0000 7fb8e67fc700 1 -- 172.21.15.64:0/377572512 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec018680 con 0x7fb8f0073060 2023-09-13T03:12:59.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.433+0000 7fb8f77ff700 1 -- 172.21.15.64:0/377572512 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8e8009c40 con 0x7fb8f0073060 2023-09-13T03:12:59.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.433+0000 7fb8f77ff700 1 -- 172.21.15.64:0/377572512 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8f01a0ce0 con 0x7fb8f0073060 2023-09-13T03:12:59.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.434+0000 7fb8f77ff700 1 -- 172.21.15.64:0/377572512 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8f0106ba0 con 0x7fb8f0073060 2023-09-13T03:12:59.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.436+0000 7fb8e67fc700 1 -- 172.21.15.64:0/377572512 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fb8ec0105a0 con 0x7fb8f0073060 2023-09-13T03:12:59.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.437+0000 7fb8e67fc700 1 --2- 172.21.15.64:0/377572512 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fb8dc0a0640 0x7fb8dc0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:59.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.438+0000 7fb8e67fc700 1 -- 172.21.15.64:0/377572512 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb8ec014070 con 0x7fb8f0073060 2023-09-13T03:12:59.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.438+0000 7fb8f4d9a700 1 --2- 172.21.15.64:0/377572512 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fb8dc0a0640 0x7fb8dc0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:59.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.438+0000 7fb8f4d9a700 1 --2- 172.21.15.64:0/377572512 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fb8dc0a0640 0x7fb8dc0a2b00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0074360 tx=0x7fb8e0009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:59.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.443+0000 7fb8e67fc700 1 -- 172.21.15.64:0/377572512 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb8ec051e10 con 0x7fb8f0073060 2023-09-13T03:12:59.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.444+0000 7f73d3fff700 1 -- 172.21.15.64:0/165296907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73cc074640 msgr2=0x7f73cc072a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:59.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.444+0000 7f73d3fff700 1 --2- 172.21.15.64:0/165296907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73cc074640 0x7f73cc072a90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f73c8009990 tx=0x7f73c8009ca0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.444+0000 7f73d3fff700 1 -- 172.21.15.64:0/165296907 shutdown_connections 2023-09-13T03:12:59.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.444+0000 7f73d3fff700 1 --2- 172.21.15.64:0/165296907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73cc1084d0 0x7f73cc10a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.444+0000 7f73d3fff700 1 --2- 172.21.15.64:0/165296907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73cc074640 0x7f73cc072a90 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.444+0000 7f73d3fff700 1 --2- 172.21.15.64:0/165296907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73cc073060 0x7f73cc0734e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.444+0000 7f73d3fff700 1 -- 172.21.15.64:0/165296907 >> 172.21.15.64:0/165296907 conn(0x7f73cc0fbb80 msgr2=0x7f73cc0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:59.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.444+0000 7f73d3fff700 1 -- 172.21.15.64:0/165296907 shutdown_connections 2023-09-13T03:12:59.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.444+0000 7f73d3fff700 1 -- 172.21.15.64:0/165296907 wait complete. 2023-09-13T03:12:59.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.445+0000 7f73d3fff700 1 Processor -- start 2023-09-13T03:12:59.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.445+0000 7f73d3fff700 1 -- start start 2023-09-13T03:12:59.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.445+0000 7f73d3fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73cc073060 0x7f73cc198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:59.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.445+0000 7f73d3fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73cc074640 0x7f73cc198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:59.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.445+0000 7f73d3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73cc1084d0 0x7f73cc19fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:59.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.445+0000 7f73d3fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73cc1a00b0 con 0x7f73cc1084d0 2023-09-13T03:12:59.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.445+0000 7f73d3fff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73cc1a01f0 con 0x7f73cc073060 2023-09-13T03:12:59.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.445+0000 7f73d3fff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73cc1a0380 con 0x7f73cc074640 2023-09-13T03:12:59.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.446+0000 7f73d259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73cc1084d0 0x7f73cc19fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:59.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.446+0000 7f73d259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73cc1084d0 0x7f73cc19fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56232/0 (socket says 172.21.15.64:56232) 2023-09-13T03:12:59.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.446+0000 7f73d259c700 1 -- 172.21.15.64:0/1967035157 learned_addr learned my addr 172.21.15.64:0/1967035157 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:59.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.446+0000 7f73d159a700 1 --2- 172.21.15.64:0/1967035157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73cc074640 0x7f73cc198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:59.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.446+0000 7f73d1d9b700 1 --2- 172.21.15.64:0/1967035157 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73cc073060 0x7f73cc198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:59.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.446+0000 7f73d259c700 1 -- 172.21.15.64:0/1967035157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73cc074640 msgr2=0x7f73cc198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:59.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.446+0000 7f73d259c700 1 --2- 172.21.15.64:0/1967035157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73cc074640 0x7f73cc198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.446+0000 7f73d259c700 1 -- 172.21.15.64:0/1967035157 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73cc073060 msgr2=0x7f73cc198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:59.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.446+0000 7f73d259c700 1 --2- 172.21.15.64:0/1967035157 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73cc073060 0x7f73cc198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.446+0000 7f73d259c700 1 -- 172.21.15.64:0/1967035157 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c80095f0 con 0x7f73cc1084d0 2023-09-13T03:12:59.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.446+0000 7f73d1d9b700 1 --2- 172.21.15.64:0/1967035157 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73cc073060 0x7f73cc198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:12:59.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.447+0000 7f73d259c700 1 --2- 172.21.15.64:0/1967035157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73cc1084d0 0x7f73cc19fae0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f73c400ef30 tx=0x7f73c400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:59.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.447+0000 7f73c2ffd700 1 -- 172.21.15.64:0/1967035157 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c4018070 con 0x7f73cc1084d0 2023-09-13T03:12:59.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.447+0000 7f73c2ffd700 1 -- 172.21.15.64:0/1967035157 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73c400f040 con 0x7f73cc1084d0 2023-09-13T03:12:59.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.447+0000 7f73d3fff700 1 -- 172.21.15.64:0/1967035157 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73cc1a0600 con 0x7f73cc1084d0 2023-09-13T03:12:59.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.447+0000 7f73c2ffd700 1 -- 172.21.15.64:0/1967035157 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c400cdb0 con 0x7f73cc1084d0 2023-09-13T03:12:59.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.447+0000 7f73d3fff700 1 -- 172.21.15.64:0/1967035157 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73cc1a0b60 con 0x7f73cc1084d0 2023-09-13T03:12:59.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.451+0000 7f73c2ffd700 1 -- 172.21.15.64:0/1967035157 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f73c4003a00 con 0x7f73cc1084d0 2023-09-13T03:12:59.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.452+0000 7f73c2ffd700 1 --2- 172.21.15.64:0/1967035157 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f73b80a06c0 0x7f73b80a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:59.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.452+0000 7f73c2ffd700 1 -- 172.21.15.64:0/1967035157 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f73c40cb720 con 0x7f73cc1084d0 2023-09-13T03:12:59.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.453+0000 7f73d1d9b700 1 --2- 172.21.15.64:0/1967035157 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f73b80a06c0 0x7f73b80a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:59.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.453+0000 7f73d1d9b700 1 --2- 172.21.15.64:0/1967035157 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f73b80a06c0 0x7f73b80a2b80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f73c800b2e0 tx=0x7f73c8005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:59.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.454+0000 7f73e7873b80 1 Processor -- start 2023-09-13T03:12:59.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.454+0000 7f73e7873b80 1 -- start start 2023-09-13T03:12:59.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.455+0000 7f73e7873b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7fe65bc80 0x55e7fe65c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:59.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.455+0000 7f73e7873b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7fe65c620 0x55e7fe65a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:59.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.455+0000 7f73e7873b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7fe65ae80 0x55e7fe65b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:59.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.455+0000 7f73e7873b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7fe5b37b0 con 0x55e7fe65ae80 2023-09-13T03:12:59.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.455+0000 7f73e7873b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7fe5b10d0 con 0x55e7fe65c620 2023-09-13T03:12:59.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.455+0000 7f73e7873b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7fe5ac9c0 con 0x55e7fe65bc80 2023-09-13T03:12:59.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.455+0000 7f73d1d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7fe65bc80 0x55e7fe65c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:59.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.455+0000 7f73d1d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7fe65bc80 0x55e7fe65c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:46774/0 (socket says 172.21.15.64:46774) 2023-09-13T03:12:59.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.455+0000 7f73d259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7fe65ae80 0x55e7fe65b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:59.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.455+0000 7f73d1d9b700 1 -- 172.21.15.64:0/3324809310 learned_addr learned my addr 172.21.15.64:0/3324809310 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:59.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.455+0000 7f73d159a700 1 --2- 172.21.15.64:0/3324809310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7fe65c620 0x55e7fe65a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:59.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.455+0000 7f73d259c700 1 -- 172.21.15.64:0/3324809310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7fe65bc80 msgr2=0x55e7fe65c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:59.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.455+0000 7f73d259c700 1 --2- 172.21.15.64:0/3324809310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7fe65bc80 0x55e7fe65c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.455+0000 7f73d259c700 1 -- 172.21.15.64:0/3324809310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7fe65c620 msgr2=0x55e7fe65a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:59.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.455+0000 7f73d259c700 1 --2- 172.21.15.64:0/3324809310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7fe65c620 0x55e7fe65a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.455+0000 7f73d259c700 1 -- 172.21.15.64:0/3324809310 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7fe3d8e60 con 0x55e7fe65ae80 2023-09-13T03:12:59.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.456+0000 7f73d259c700 1 --2- 172.21.15.64:0/3324809310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7fe65ae80 0x55e7fe65b330 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f73c40c91f0 tx=0x7f73c4098dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31860afb685e954c server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:59.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.456+0000 7f73b7fff700 1 -- 172.21.15.64:0/3324809310 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c40b8040 con 0x55e7fe65ae80 2023-09-13T03:12:59.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.456+0000 7f73b7fff700 1 -- 172.21.15.64:0/3324809310 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73c4004ad0 con 0x55e7fe65ae80 2023-09-13T03:12:59.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.456+0000 7f73b7fff700 1 -- 172.21.15.64:0/3324809310 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c40a0f50 con 0x55e7fe65ae80 2023-09-13T03:12:59.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.457+0000 7f73e7873b80 1 -- 172.21.15.64:0/3324809310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7fe65ae80 msgr2=0x55e7fe65b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:59.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.457+0000 7f73e7873b80 1 --2- 172.21.15.64:0/3324809310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7fe65ae80 0x55e7fe65b330 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f73c40c91f0 tx=0x7f73c4098dc0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.457+0000 7f73e7873b80 1 -- 172.21.15.64:0/3324809310 shutdown_connections 2023-09-13T03:12:59.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.457+0000 7f73e7873b80 1 --2- 172.21.15.64:0/3324809310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7fe65ae80 0x55e7fe65b330 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.457+0000 7f73e7873b80 1 --2- 172.21.15.64:0/3324809310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7fe65bc80 0x55e7fe65c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.457+0000 7f73e7873b80 1 --2- 172.21.15.64:0/3324809310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7fe65c620 0x55e7fe65a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.457+0000 7f73e7873b80 1 -- 172.21.15.64:0/3324809310 >> 172.21.15.64:0/3324809310 conn(0x55e7fe5cf110 msgr2=0x55e7fe6557a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:59.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.457+0000 7f73e7873b80 1 -- 172.21.15.64:0/3324809310 shutdown_connections 2023-09-13T03:12:59.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.457+0000 7f73e7873b80 1 -- 172.21.15.64:0/3324809310 wait complete. 2023-09-13T03:12:59.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.458+0000 7f73e7873b80 1 -- start start 2023-09-13T03:12:59.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.458+0000 7f73e7873b80 1 Processor -- start 2023-09-13T03:12:59.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.458+0000 7f73e7873b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7fe65ae80 0x55e7fe664610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:59.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.458+0000 7f73e7873b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7fe65bc80 0x55e7fe664b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:59.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.458+0000 7f73e7873b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7fe65c620 0x55e7fe66bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:59.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.458+0000 7f73e7873b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7fe54abf0 con 0x55e7fe65c620 2023-09-13T03:12:59.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.458+0000 7f73e7873b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7fe54a8f0 con 0x55e7fe65ae80 2023-09-13T03:12:59.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.458+0000 7f73e7873b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7fe42e9a0 con 0x55e7fe65bc80 2023-09-13T03:12:59.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.459+0000 7f73e7873b80 10 client.0 _ll_register_callbacks cb 0x55e7fe494b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:12:59.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.459+0000 7f73d259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7fe65c620 0x55e7fe66bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:59.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.459+0000 7f73d259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7fe65c620 0x55e7fe66bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56262/0 (socket says 172.21.15.64:56262) 2023-09-13T03:12:59.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.459+0000 7f73d259c700 1 -- 172.21.15.64:0/2928186056 learned_addr learned my addr 172.21.15.64:0/2928186056 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:59.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.459+0000 7f73d1d9b700 1 --2- 172.21.15.64:0/2928186056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7fe65ae80 0x55e7fe664610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:59.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.459+0000 7f73d159a700 1 --2- 172.21.15.64:0/2928186056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7fe65bc80 0x55e7fe664b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:59.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.459+0000 7f73d259c700 1 -- 172.21.15.64:0/2928186056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7fe65bc80 msgr2=0x55e7fe664b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:59.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.459+0000 7f73d259c700 1 --2- 172.21.15.64:0/2928186056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7fe65bc80 0x55e7fe664b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.459+0000 7f73d259c700 1 -- 172.21.15.64:0/2928186056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7fe65ae80 msgr2=0x55e7fe664610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:59.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.459+0000 7f73d259c700 1 --2- 172.21.15.64:0/2928186056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7fe65ae80 0x55e7fe664610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.459+0000 7f73d259c700 1 -- 172.21.15.64:0/2928186056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73cc1a0b60 con 0x55e7fe65c620 2023-09-13T03:12:59.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.459+0000 7f73d159a700 1 --2- 172.21.15.64:0/2928186056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7fe65bc80 0x55e7fe664b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:12:59.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.459+0000 7f73d259c700 1 --2- 172.21.15.64:0/2928186056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7fe65c620 0x55e7fe66bb90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f73c400c7a0 tx=0x7f73c4004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=611c02b0677a82ba server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:59.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.459+0000 7f73b57fa700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:12:59.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.460+0000 7f73b57fa700 1 -- 172.21.15.64:0/2928186056 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c401b3a0 con 0x55e7fe65c620 2023-09-13T03:12:59.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.460+0000 7f73e7873b80 1 -- 172.21.15.64:0/2928186056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e7fe504570 con 0x55e7fe65c620 2023-09-13T03:12:59.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.460+0000 7f73b57fa700 1 -- 172.21.15.64:0/2928186056 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f73c4099670 con 0x55e7fe65c620 2023-09-13T03:12:59.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.460+0000 7f73b57fa700 1 -- 172.21.15.64:0/2928186056 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73c40a0f50 con 0x55e7fe65c620 2023-09-13T03:12:59.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.460+0000 7f73e7873b80 10 client.15471 fetch_fsmap learned FSMap version 26 2023-09-13T03:12:59.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.460+0000 7f73e7873b80 1 -- 172.21.15.64:0/2928186056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55e7fe3c1ce0 con 0x55e7fe65c620 2023-09-13T03:12:59.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.460+0000 7f73b57fa700 1 -- 172.21.15.64:0/2928186056 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e7fe504570 con 0x55e7fe65c620 2023-09-13T03:12:59.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.460+0000 7f73b57fa700 1 -- 172.21.15.64:0/2928186056 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f73c40a0be0 con 0x55e7fe65c620 2023-09-13T03:12:59.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.460+0000 7f73e7873b80 10 client.15471 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:12:59.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.461+0000 7f73e7873b80 10 client.15471 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T03:12:59.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.461+0000 7f73e7873b80 20 client.15471 populate_metadata read hostname 'smithi064' 2023-09-13T03:12:59.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.461+0000 7f73e7873b80 1 --2- 172.21.15.64:0/2928186056 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55e7fe66ff10 0x55e7fe672330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:59.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.461+0000 7f73e7873b80 4 client.15471 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:12:59.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.461+0000 7f73e7873b80 1 -- 172.21.15.64:0/2928186056 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e7fe5c1920 con 0x55e7fe66ff10 2023-09-13T03:12:59.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.461+0000 7f73d1d9b700 1 --2- 172.21.15.64:0/2928186056 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55e7fe66ff10 0x55e7fe672330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:59.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.462+0000 7f73b57fa700 10 client.15471 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:12:59.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.462+0000 7f73d1d9b700 1 --2- 172.21.15.64:0/2928186056 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55e7fe66ff10 0x55e7fe672330 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc410d3f7498c545 server_cookie=5c7b15ce37be5ea7 in_seq=0 out_seq=0 2023-09-13T03:12:59.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.462+0000 7f73b57fa700 1 -- 172.21.15.64:0/2928186056 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e7fe5c1920 con 0x55e7fe66ff10 2023-09-13T03:12:59.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.462+0000 7f73b57fa700 10 client.15471 handle_command_reply: tid=0 2023-09-13T03:12:59.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.463+0000 7f73e7873b80 1 client.15471 shutdown 2023-09-13T03:12:59.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.463+0000 7f73e7873b80 1 -- 172.21.15.64:0/2928186056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7fe65c620 msgr2=0x55e7fe66bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:59.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.463+0000 7f73e7873b80 1 --2- 172.21.15.64:0/2928186056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7fe65c620 0x55e7fe66bb90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f73c400c7a0 tx=0x7f73c4004cc0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.463+0000 7f73e7873b80 1 -- 172.21.15.64:0/2928186056 shutdown_connections 2023-09-13T03:12:59.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.463+0000 7f73e7873b80 1 --2- 172.21.15.64:0/2928186056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7fe65ae80 0x55e7fe664610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.463+0000 7f73e7873b80 1 --2- 172.21.15.64:0/2928186056 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55e7fe66ff10 0x55e7fe672330 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.463+0000 7f73e7873b80 1 --2- 172.21.15.64:0/2928186056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7fe65c620 0x55e7fe66bb90 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.463+0000 7f73e7873b80 1 --2- 172.21.15.64:0/2928186056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7fe65bc80 0x55e7fe664b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.463+0000 7f73b57fa700 0 client.15471 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:12:59.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.463+0000 7f73e7873b80 1 -- 172.21.15.64:0/2928186056 >> 172.21.15.64:0/2928186056 conn(0x55e7fe5cf110 msgr2=0x55e7fe654e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:59.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.463+0000 7f73e7873b80 1 -- 172.21.15.64:0/2928186056 shutdown_connections 2023-09-13T03:12:59.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.463+0000 7f73e7873b80 1 -- 172.21.15.64:0/2928186056 wait complete. 2023-09-13T03:12:59.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.463+0000 7f73e7873b80 20 client.15471 trim_cache size 0 max 16384 2023-09-13T03:12:59.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.499+0000 7f73e7873b80 1 Processor -- start 2023-09-13T03:12:59.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.500+0000 7f73e7873b80 1 -- start start 2023-09-13T03:12:59.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.500+0000 7f73e7873b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7fe65ae80 0x55e7fe659780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:59.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.500+0000 7f73e7873b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7fe65bc80 0x55e7fe66bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:59.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.500+0000 7f73e7873b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7fe65c620 0x55e7fe659cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:59.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.500+0000 7f73e7873b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7fe664180 con 0x55e7fe65ae80 2023-09-13T03:12:59.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.500+0000 7f73e7873b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7fe656a30 con 0x55e7fe65c620 2023-09-13T03:12:59.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.500+0000 7f73e7873b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7fe665790 con 0x55e7fe65bc80 2023-09-13T03:12:59.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.500+0000 7f73d1d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7fe65ae80 0x55e7fe659780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:59.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.500+0000 7f73d1d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7fe65ae80 0x55e7fe659780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56272/0 (socket says 172.21.15.64:56272) 2023-09-13T03:12:59.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.500+0000 7f73d159a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7fe65bc80 0x55e7fe66bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:59.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.500+0000 7f73d1d9b700 1 -- 172.21.15.64:0/2331423625 learned_addr learned my addr 172.21.15.64:0/2331423625 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:59.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.500+0000 7f73d259c700 1 --2- 172.21.15.64:0/2331423625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7fe65c620 0x55e7fe659cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:59.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.501+0000 7f73d1d9b700 1 -- 172.21.15.64:0/2331423625 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7fe65bc80 msgr2=0x55e7fe66bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:59.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.501+0000 7f73d1d9b700 1 --2- 172.21.15.64:0/2331423625 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7fe65bc80 0x55e7fe66bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.501+0000 7f73d1d9b700 1 -- 172.21.15.64:0/2331423625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7fe65c620 msgr2=0x55e7fe659cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:59.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.501+0000 7f73d1d9b700 1 --2- 172.21.15.64:0/2331423625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7fe65c620 0x55e7fe659cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.501+0000 7f73d1d9b700 1 -- 172.21.15.64:0/2331423625 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7fe3ff2d0 con 0x55e7fe65ae80 2023-09-13T03:12:59.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.501+0000 7f73d1d9b700 1 --2- 172.21.15.64:0/2331423625 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7fe65ae80 0x55e7fe659780 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f73c8015000 tx=0x7f73c801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f3b8ed5015b7cb server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:59.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.501+0000 7f73c0ff9700 1 -- 172.21.15.64:0/2331423625 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c8039070 con 0x55e7fe65ae80 2023-09-13T03:12:59.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.501+0000 7f73c0ff9700 1 -- 172.21.15.64:0/2331423625 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73c802c040 con 0x55e7fe65ae80 2023-09-13T03:12:59.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.501+0000 7f73c0ff9700 1 -- 172.21.15.64:0/2331423625 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c802b040 con 0x55e7fe65ae80 2023-09-13T03:12:59.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.502+0000 7f73e7873b80 1 -- 172.21.15.64:0/2331423625 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7fe65ae80 msgr2=0x55e7fe659780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:59.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.502+0000 7f73e7873b80 1 --2- 172.21.15.64:0/2331423625 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7fe65ae80 0x55e7fe659780 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f73c8015000 tx=0x7f73c801f000 comp rx=0 tx=0).stop 2023-09-13T03:12:59.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.502+0000 7f73e7873b80 1 -- 172.21.15.64:0/2331423625 shutdown_connections 2023-09-13T03:12:59.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.502+0000 7f73e7873b80 1 --2- 172.21.15.64:0/2331423625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7fe65c620 0x55e7fe659cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.502+0000 7f73e7873b80 1 --2- 172.21.15.64:0/2331423625 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7fe65ae80 0x55e7fe659780 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.502+0000 7f73e7873b80 1 --2- 172.21.15.64:0/2331423625 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7fe65bc80 0x55e7fe66bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.502+0000 7f73e7873b80 1 -- 172.21.15.64:0/2331423625 >> 172.21.15.64:0/2331423625 conn(0x55e7fe5cf110 msgr2=0x55e7fe65ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:59.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.503+0000 7f73e7873b80 1 -- 172.21.15.64:0/2331423625 shutdown_connections 2023-09-13T03:12:59.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.503+0000 7f73e7873b80 1 -- 172.21.15.64:0/2331423625 wait complete. 2023-09-13T03:12:59.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.503+0000 7f73e7873b80 1 -- start start 2023-09-13T03:12:59.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.503+0000 7f73e7873b80 1 Processor -- start 2023-09-13T03:12:59.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.504+0000 7f73e7873b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7fe65ae80 0x55e7fe668de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:59.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.504+0000 7f73e7873b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7fe65bc80 0x55e7fe669320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:59.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.504+0000 7f73e7873b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7fe65c620 0x55e7fe66a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:59.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.504+0000 7f73e7873b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7fe66a870 con 0x55e7fe65bc80 2023-09-13T03:12:59.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.504+0000 7f73e7873b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7fe66a9e0 con 0x55e7fe65c620 2023-09-13T03:12:59.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.504+0000 7f73e7873b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7fe66aba0 con 0x55e7fe65ae80 2023-09-13T03:12:59.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.504+0000 7f73e7873b80 10 client.0 _ll_register_callbacks cb 0x7f73bc009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:12:59.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.504+0000 7f73d1d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7fe65ae80 0x55e7fe668de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:59.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.504+0000 7f73d1d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7fe65ae80 0x55e7fe668de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:46806/0 (socket says 172.21.15.64:46806) 2023-09-13T03:12:59.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.504+0000 7f73d1d9b700 1 -- 172.21.15.64:0/1734319383 learned_addr learned my addr 172.21.15.64:0/1734319383 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:59.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.504+0000 7f73d159a700 1 --2- 172.21.15.64:0/1734319383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7fe65bc80 0x55e7fe669320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:59.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.504+0000 7f73d259c700 1 --2- 172.21.15.64:0/1734319383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7fe65c620 0x55e7fe66a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:59.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.504+0000 7f73d1d9b700 1 -- 172.21.15.64:0/1734319383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7fe65c620 msgr2=0x55e7fe66a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:59.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.504+0000 7f73d1d9b700 1 --2- 172.21.15.64:0/1734319383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7fe65c620 0x55e7fe66a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.504+0000 7f73d1d9b700 1 -- 172.21.15.64:0/1734319383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7fe65bc80 msgr2=0x55e7fe669320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:59.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.504+0000 7f73d1d9b700 1 --2- 172.21.15.64:0/1734319383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7fe65bc80 0x55e7fe669320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.504+0000 7f73d1d9b700 1 -- 172.21.15.64:0/1734319383 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73cc10b580 con 0x55e7fe65ae80 2023-09-13T03:12:59.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.505+0000 7f73d1d9b700 1 --2- 172.21.15.64:0/1734319383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7fe65ae80 0x55e7fe668de0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f73c8035510 tx=0x7f73c80356b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca4aa99c2e633b15 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:59.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.505+0000 7f73b57fa700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:12:59.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.505+0000 7f73b57fa700 1 -- 172.21.15.64:0/1734319383 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c802b070 con 0x55e7fe65ae80 2023-09-13T03:12:59.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.505+0000 7f73e7873b80 1 -- 172.21.15.64:0/1734319383 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f73c4004ad0 con 0x55e7fe65ae80 2023-09-13T03:12:59.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.505+0000 7f73b57fa700 1 -- 172.21.15.64:0/1734319383 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f73c8035230 con 0x55e7fe65ae80 2023-09-13T03:12:59.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.505+0000 7f73b57fa700 1 -- 172.21.15.64:0/1734319383 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73c8039030 con 0x55e7fe65ae80 2023-09-13T03:12:59.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.505+0000 7f73e7873b80 10 client.25507 fetch_fsmap learned FSMap version 26 2023-09-13T03:12:59.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.505+0000 7f73e7873b80 1 -- 172.21.15.64:0/1734319383 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55e7fe3d7330 con 0x55e7fe65ae80 2023-09-13T03:12:59.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.506+0000 7f73b57fa700 1 -- 172.21.15.64:0/1734319383 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73c4004ad0 con 0x55e7fe65ae80 2023-09-13T03:12:59.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.506+0000 7f73b57fa700 1 -- 172.21.15.64:0/1734319383 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f73c800f5f0 con 0x55e7fe65ae80 2023-09-13T03:12:59.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.506+0000 7f73e7873b80 10 client.25507 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:12:59.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.506+0000 7f73e7873b80 10 client.25507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T03:12:59.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.506+0000 7f73e7873b80 20 client.25507 populate_metadata read hostname 'smithi064' 2023-09-13T03:12:59.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.506+0000 7f73e7873b80 1 --2- 172.21.15.64:0/1734319383 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55e7fe6c5e70 0x55e7fe6c8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:59.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.506+0000 7f73e7873b80 4 client.25507 mds_command: new command op to 24686 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-09-13T03:12:59.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.506+0000 7f73e7873b80 1 -- 172.21.15.64:0/1734319383 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e7fe5a59a0 con 0x55e7fe6c5e70 2023-09-13T03:12:59.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.506+0000 7f73d159a700 1 --2- 172.21.15.64:0/1734319383 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55e7fe6c5e70 0x55e7fe6c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:59.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.507+0000 7f73d159a700 1 --2- 172.21.15.64:0/1734319383 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55e7fe6c5e70 0x55e7fe6c8260 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=743c5fd17e7da688 server_cookie=24ea18f376707a0d in_seq=0 out_seq=0 2023-09-13T03:12:59.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.507+0000 7f73b57fa700 10 client.25507 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:12:59.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.508+0000 7f73b57fa700 1 -- 172.21.15.64:0/1734319383 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e7fe5a59a0 con 0x55e7fe6c5e70 2023-09-13T03:12:59.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.508+0000 7f73b57fa700 10 client.25507 handle_command_reply: tid=0 2023-09-13T03:12:59.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.508+0000 7f73e7873b80 1 client.25507 shutdown 2023-09-13T03:12:59.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.508+0000 7f73e7873b80 1 -- 172.21.15.64:0/1734319383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7fe65ae80 msgr2=0x55e7fe668de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:59.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.508+0000 7f73e7873b80 1 --2- 172.21.15.64:0/1734319383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7fe65ae80 0x55e7fe668de0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f73c8035510 tx=0x7f73c80356b0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.508+0000 7f73e7873b80 1 -- 172.21.15.64:0/1734319383 shutdown_connections 2023-09-13T03:12:59.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.508+0000 7f73e7873b80 1 --2- 172.21.15.64:0/1734319383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7fe65c620 0x55e7fe66a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.508+0000 7f73e7873b80 1 --2- 172.21.15.64:0/1734319383 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55e7fe6c5e70 0x55e7fe6c8260 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.508+0000 7f73e7873b80 1 --2- 172.21.15.64:0/1734319383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7fe65ae80 0x55e7fe668de0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.508+0000 7f73e7873b80 1 --2- 172.21.15.64:0/1734319383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7fe65bc80 0x55e7fe669320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.508+0000 7f73b57fa700 0 client.25507 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:12:59.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.508+0000 7f73e7873b80 1 -- 172.21.15.64:0/1734319383 >> 172.21.15.64:0/1734319383 conn(0x55e7fe5cf110 msgr2=0x55e7fe654e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:59.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.508+0000 7f73e7873b80 1 -- 172.21.15.64:0/1734319383 shutdown_connections 2023-09-13T03:12:59.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.509+0000 7f73e7873b80 1 -- 172.21.15.64:0/1734319383 wait complete. 2023-09-13T03:12:59.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.509+0000 7f73e7873b80 20 client.25507 trim_cache size 0 max 16384 2023-09-13T03:12:59.521 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-13T03:12:59.522 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2023-09-13T03:12:59.522 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "d0ff213d-a6dd-485a-b264-900af2aca808", 2023-09-13T03:12:59.522 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2023-09-13T03:12:59.522 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-13T03:12:59.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.509+0000 7f73d3fff700 1 -- 172.21.15.64:0/1967035157 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f73b80a06c0 msgr2=0x7f73b80a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:59.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.509+0000 7f73d3fff700 1 --2- 172.21.15.64:0/1967035157 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f73b80a06c0 0x7f73b80a2b80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f73c800b2e0 tx=0x7f73c8005fd0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.509+0000 7f73d3fff700 1 -- 172.21.15.64:0/1967035157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73cc1084d0 msgr2=0x7f73cc19fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:59.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.509+0000 7f73d3fff700 1 --2- 172.21.15.64:0/1967035157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73cc1084d0 0x7f73cc19fae0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f73c400ef30 tx=0x7f73c400c3a0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.509+0000 7f73d3fff700 1 -- 172.21.15.64:0/1967035157 shutdown_connections 2023-09-13T03:12:59.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.509+0000 7f73d3fff700 1 --2- 172.21.15.64:0/1967035157 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7f73b80a06c0 0x7f73b80a2b80 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.509+0000 7f73d3fff700 1 --2- 172.21.15.64:0/1967035157 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73cc073060 0x7f73cc198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.509+0000 7f73d3fff700 1 --2- 172.21.15.64:0/1967035157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73cc1084d0 0x7f73cc19fae0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.509+0000 7f73d3fff700 1 --2- 172.21.15.64:0/1967035157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73cc074640 0x7f73cc198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.509+0000 7f73d3fff700 1 -- 172.21.15.64:0/1967035157 >> 172.21.15.64:0/1967035157 conn(0x7f73cc0fbb80 msgr2=0x7f73cc0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:59.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.510+0000 7f73d3fff700 1 -- 172.21.15.64:0/1967035157 shutdown_connections 2023-09-13T03:12:59.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.510+0000 7f73d3fff700 1 -- 172.21.15.64:0/1967035157 wait complete. 2023-09-13T03:12:59.528 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-13T03:12:59.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.733+0000 7fc3b8dff700 1 -- 172.21.15.64:0/286080413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3b4073060 msgr2=0x7fc3b40734e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:59.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.733+0000 7fc3b8dff700 1 --2- 172.21.15.64:0/286080413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3b4073060 0x7fc3b40734e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8009990 tx=0x7fc3a8009ca0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.734+0000 7fc3b8dff700 1 -- 172.21.15.64:0/286080413 shutdown_connections 2023-09-13T03:12:59.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.734+0000 7fc3b8dff700 1 --2- 172.21.15.64:0/286080413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3b4074640 0x7fc3b4072a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.734+0000 7fc3b8dff700 1 --2- 172.21.15.64:0/286080413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3b410cbe0 0x7fc3b410efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.734+0000 7fc3b8dff700 1 --2- 172.21.15.64:0/286080413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3b4073060 0x7fc3b40734e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.734+0000 7fc3b8dff700 1 -- 172.21.15.64:0/286080413 >> 172.21.15.64:0/286080413 conn(0x7fc3b40fbb80 msgr2=0x7fc3b40fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:59.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.734+0000 7fc3b8dff700 1 -- 172.21.15.64:0/286080413 shutdown_connections 2023-09-13T03:12:59.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.734+0000 7fc3b8dff700 1 -- 172.21.15.64:0/286080413 wait complete. 2023-09-13T03:12:59.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.735+0000 7fc3b8dff700 1 Processor -- start 2023-09-13T03:12:59.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.735+0000 7fc3b8dff700 1 -- start start 2023-09-13T03:12:59.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.735+0000 7fc3b8dff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3b4073060 0x7fc3b419c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:59.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.735+0000 7fc3b8dff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3b4074640 0x7fc3b419ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:59.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.735+0000 7fc3b8dff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3b410cbe0 0x7fc3b41a3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:59.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.735+0000 7fc3b8dff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b41a44a0 con 0x7fc3b4074640 2023-09-13T03:12:59.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.735+0000 7fc3b8dff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b41a45e0 con 0x7fc3b410cbe0 2023-09-13T03:12:59.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.735+0000 7fc3b8dff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b41a4770 con 0x7fc3b4073060 2023-09-13T03:12:59.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.735+0000 7fc3b1d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3b4074640 0x7fc3b419ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:59.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.736+0000 7fc3b1d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3b4074640 0x7fc3b419ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56306/0 (socket says 172.21.15.64:56306) 2023-09-13T03:12:59.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.736+0000 7fc3b1d9b700 1 -- 172.21.15.64:0/1109899629 learned_addr learned my addr 172.21.15.64:0/1109899629 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:59.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.736+0000 7fc3b259c700 1 --2- 172.21.15.64:0/1109899629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3b4073060 0x7fc3b419c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:59.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.736+0000 7fc3b2d9d700 1 --2- 172.21.15.64:0/1109899629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3b410cbe0 0x7fc3b41a3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:59.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.736+0000 7fc3b1d9b700 1 -- 172.21.15.64:0/1109899629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3b4073060 msgr2=0x7fc3b419c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:59.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.736+0000 7fc3b1d9b700 1 --2- 172.21.15.64:0/1109899629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3b4073060 0x7fc3b419c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.736+0000 7fc3b1d9b700 1 -- 172.21.15.64:0/1109899629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3b410cbe0 msgr2=0x7fc3b41a3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:59.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.736+0000 7fc3b1d9b700 1 --2- 172.21.15.64:0/1109899629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3b410cbe0 0x7fc3b41a3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.736+0000 7fc3b1d9b700 1 -- 172.21.15.64:0/1109899629 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a4009520 con 0x7fc3b4074640 2023-09-13T03:12:59.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.736+0000 7fc3b1d9b700 1 --2- 172.21.15.64:0/1109899629 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3b4074640 0x7fc3b419ce60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8004710 tx=0x7fc3a8009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:59.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.737+0000 7fc3a37fe700 1 -- 172.21.15.64:0/1109899629 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a801d070 con 0x7fc3b4074640 2023-09-13T03:12:59.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.737+0000 7fc3b8dff700 1 -- 172.21.15.64:0/1109899629 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3a80095f0 con 0x7fc3b4074640 2023-09-13T03:12:59.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.737+0000 7fc3a37fe700 1 -- 172.21.15.64:0/1109899629 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3a8021410 con 0x7fc3b4074640 2023-09-13T03:12:59.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.737+0000 7fc3a37fe700 1 -- 172.21.15.64:0/1109899629 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8017650 con 0x7fc3b4074640 2023-09-13T03:12:59.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.737+0000 7fc3b8dff700 1 -- 172.21.15.64:0/1109899629 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3b41a4ce0 con 0x7fc3b4074640 2023-09-13T03:12:59.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.740+0000 7fc3a37fe700 1 -- 172.21.15.64:0/1109899629 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fc3a8021580 con 0x7fc3b4074640 2023-09-13T03:12:59.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.741+0000 7fc3cc6e5b80 1 Processor -- start 2023-09-13T03:12:59.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.741+0000 7fc3cc6e5b80 1 -- start start 2023-09-13T03:12:59.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.741+0000 7fc3a37fe700 1 --2- 172.21.15.64:0/1109899629 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc39c0a0640 0x7fc39c0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:59.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.741+0000 7fc3cc6e5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f3953c00 0x5624f3954020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:59.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.742+0000 7fc3cc6e5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f39545a0 0x5624f3952800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:59.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.742+0000 7fc3cc6e5b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624f3952e00 0x5624f39532b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:59.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.742+0000 7fc3cc6e5b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f38ab730 con 0x5624f3952e00 2023-09-13T03:12:59.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.742+0000 7fc3cc6e5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f38a9050 con 0x5624f39545a0 2023-09-13T03:12:59.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.742+0000 7fc3cc6e5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f38a4940 con 0x5624f3953c00 2023-09-13T03:12:59.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.742+0000 7fc3a37fe700 1 -- 172.21.15.64:0/1109899629 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fc3a80ccee0 con 0x7fc3b4074640 2023-09-13T03:12:59.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.742+0000 7fc3b2d9d700 1 --2- 172.21.15.64:0/1109899629 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc39c0a0640 0x7fc39c0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:59.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.742+0000 7fc3b1d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f39545a0 0x5624f3952800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:59.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.742+0000 7fc3b1d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f39545a0 0x5624f3952800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:47752/0 (socket says 172.21.15.64:47752) 2023-09-13T03:12:59.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.742+0000 7fc3b1d9b700 1 -- 172.21.15.64:0/182812121 learned_addr learned my addr 172.21.15.64:0/182812121 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:59.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.742+0000 7fc3b2d9d700 1 --2- 172.21.15.64:0/182812121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624f3952e00 0x5624f39532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:59.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.742+0000 7fc3b259c700 1 --2- 172.21.15.64:0/182812121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f3953c00 0x5624f3954020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:59.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.742+0000 7fc3b1d9b700 1 -- 172.21.15.64:0/182812121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f3953c00 msgr2=0x5624f3954020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:59.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.742+0000 7fc3b1d9b700 1 --2- 172.21.15.64:0/182812121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f3953c00 0x5624f3954020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.742+0000 7fc3b1d9b700 1 -- 172.21.15.64:0/182812121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624f3952e00 msgr2=0x5624f39532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:59.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.742+0000 7fc3b1d9b700 1 --2- 172.21.15.64:0/182812121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624f3952e00 0x5624f39532b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.742+0000 7fc3b1d9b700 1 -- 172.21.15.64:0/182812121 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624f36d0de0 con 0x5624f39545a0 2023-09-13T03:12:59.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.743+0000 7fc3b2d9d700 1 --2- 172.21.15.64:0/1109899629 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc39c0a0640 0x7fc39c0a2b00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac009ec0 tx=0x7fc3ac0092f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:59.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.743+0000 7fc3b1d9b700 1 --2- 172.21.15.64:0/182812121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f39545a0 0x5624f3952800 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8096c30 tx=0x7fc3a809a580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0594a9c8b005868 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:59.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.743+0000 7fc3a0ff9700 1 -- 172.21.15.64:0/182812121 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8077e50 con 0x5624f39545a0 2023-09-13T03:12:59.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.743+0000 7fc3a0ff9700 1 -- 172.21.15.64:0/182812121 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3a8077c70 con 0x5624f39545a0 2023-09-13T03:12:59.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.743+0000 7fc3a0ff9700 1 -- 172.21.15.64:0/182812121 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a80a2710 con 0x5624f39545a0 2023-09-13T03:12:59.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.744+0000 7fc3cc6e5b80 1 -- 172.21.15.64:0/182812121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f39545a0 msgr2=0x5624f3952800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:59.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.744+0000 7fc3cc6e5b80 1 --2- 172.21.15.64:0/182812121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f39545a0 0x5624f3952800 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8096c30 tx=0x7fc3a809a580 comp rx=0 tx=0).stop 2023-09-13T03:12:59.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.744+0000 7fc3cc6e5b80 1 -- 172.21.15.64:0/182812121 shutdown_connections 2023-09-13T03:12:59.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.744+0000 7fc3cc6e5b80 1 --2- 172.21.15.64:0/182812121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624f3952e00 0x5624f39532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.744+0000 7fc3cc6e5b80 1 --2- 172.21.15.64:0/182812121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f3953c00 0x5624f3954020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.744+0000 7fc3cc6e5b80 1 --2- 172.21.15.64:0/182812121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f39545a0 0x5624f3952800 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.744+0000 7fc3cc6e5b80 1 -- 172.21.15.64:0/182812121 >> 172.21.15.64:0/182812121 conn(0x5624f38c7090 msgr2=0x5624f394d720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:59.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.744+0000 7fc3cc6e5b80 1 -- 172.21.15.64:0/182812121 shutdown_connections 2023-09-13T03:12:59.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.744+0000 7fc3cc6e5b80 1 -- 172.21.15.64:0/182812121 wait complete. 2023-09-13T03:12:59.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.744+0000 7fc3cc6e5b80 1 -- start start 2023-09-13T03:12:59.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.745+0000 7fc3cc6e5b80 1 Processor -- start 2023-09-13T03:12:59.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.745+0000 7fc3cc6e5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f3952e00 0x5624f395c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:59.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.745+0000 7fc3cc6e5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f3953c00 0x5624f395c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:59.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.745+0000 7fc3cc6e5b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624f39545a0 0x5624f39637b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:59.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.745+0000 7fc3cc6e5b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f3842b70 con 0x5624f39545a0 2023-09-13T03:12:59.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.745+0000 7fc3cc6e5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f3842870 con 0x5624f3953c00 2023-09-13T03:12:59.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.745+0000 7fc3cc6e5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f3726920 con 0x5624f3952e00 2023-09-13T03:12:59.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.745+0000 7fc3cc6e5b80 10 client.0 _ll_register_callbacks cb 0x5624f378ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:12:59.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.746+0000 7fc3b1d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f3953c00 0x5624f395c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:59.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.746+0000 7fc3b2d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624f39545a0 0x5624f39637b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:59.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.746+0000 7fc3b1d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f3953c00 0x5624f395c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:47756/0 (socket says 172.21.15.64:47756) 2023-09-13T03:12:59.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.746+0000 7fc3b1d9b700 1 -- 172.21.15.64:0/2170865218 learned_addr learned my addr 172.21.15.64:0/2170865218 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:59.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.746+0000 7fc3b259c700 1 --2- 172.21.15.64:0/2170865218 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f3952e00 0x5624f395c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:59.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.746+0000 7fc3b2d9d700 1 -- 172.21.15.64:0/2170865218 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f3952e00 msgr2=0x5624f395c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:59.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.746+0000 7fc3b2d9d700 1 --2- 172.21.15.64:0/2170865218 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f3952e00 0x5624f395c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.746+0000 7fc3b2d9d700 1 -- 172.21.15.64:0/2170865218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f3953c00 msgr2=0x5624f395c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:59.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.746+0000 7fc3b2d9d700 1 --2- 172.21.15.64:0/2170865218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f3953c00 0x5624f395c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.746+0000 7fc3b2d9d700 1 -- 172.21.15.64:0/2170865218 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3b41a4ce0 con 0x5624f39545a0 2023-09-13T03:12:59.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.746+0000 7fc3b2d9d700 1 --2- 172.21.15.64:0/2170865218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624f39545a0 0x5624f39637b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac009fd0 tx=0x7fc3ac00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61a4b10480322534 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:59.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.746+0000 7fc399ffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:12:59.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.747+0000 7fc399ffb700 1 -- 172.21.15.64:0/2170865218 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac00e070 con 0x5624f39545a0 2023-09-13T03:12:59.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.747+0000 7fc3cc6e5b80 1 -- 172.21.15.64:0/2170865218 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5624f37fc4f0 con 0x5624f39545a0 2023-09-13T03:12:59.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.747+0000 7fc399ffb700 1 -- 172.21.15.64:0/2170865218 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fc3ac021040 con 0x5624f39545a0 2023-09-13T03:12:59.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.747+0000 7fc399ffb700 1 -- 172.21.15.64:0/2170865218 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3ac008570 con 0x5624f39545a0 2023-09-13T03:12:59.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.747+0000 7fc3cc6e5b80 10 client.15489 fetch_fsmap learned FSMap version 26 2023-09-13T03:12:59.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.747+0000 7fc3cc6e5b80 1 -- 172.21.15.64:0/2170865218 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5624f36b9c60 con 0x5624f39545a0 2023-09-13T03:12:59.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.747+0000 7fc399ffb700 1 -- 172.21.15.64:0/2170865218 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5624f37fc4f0 con 0x5624f39545a0 2023-09-13T03:12:59.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.747+0000 7fc399ffb700 1 -- 172.21.15.64:0/2170865218 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc3ac027c80 con 0x5624f39545a0 2023-09-13T03:12:59.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.747+0000 7fc3cc6e5b80 10 client.15489 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:12:59.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.747+0000 7fc3cc6e5b80 10 client.15489 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T03:12:59.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.748+0000 7fc3cc6e5b80 20 client.15489 populate_metadata read hostname 'smithi064' 2023-09-13T03:12:59.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.748+0000 7fc3cc6e5b80 1 --2- 172.21.15.64:0/2170865218 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5624f39680a0 0x5624f396a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:59.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.748+0000 7fc3cc6e5b80 4 client.15489 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:12:59.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.748+0000 7fc3cc6e5b80 1 -- 172.21.15.64:0/2170865218 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5624f38b98a0 con 0x5624f39680a0 2023-09-13T03:12:59.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.748+0000 7fc3b259c700 1 --2- 172.21.15.64:0/2170865218 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5624f39680a0 0x5624f396a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:59.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.748+0000 7fc3b259c700 1 --2- 172.21.15.64:0/2170865218 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5624f39680a0 0x5624f396a4c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3454f45499c3fab server_cookie=a32ea76f8b2d294e in_seq=0 out_seq=0 2023-09-13T03:12:59.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.748+0000 7fc399ffb700 10 client.15489 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:12:59.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.749+0000 7fc399ffb700 1 -- 172.21.15.64:0/2170865218 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5624f38b98a0 con 0x5624f39680a0 2023-09-13T03:12:59.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.749+0000 7fc399ffb700 10 client.15489 handle_command_reply: tid=0 2023-09-13T03:12:59.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.749+0000 7fc3cc6e5b80 1 client.15489 shutdown 2023-09-13T03:12:59.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.749+0000 7fc3cc6e5b80 1 -- 172.21.15.64:0/2170865218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624f39545a0 msgr2=0x5624f39637b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:59.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.749+0000 7fc3cc6e5b80 1 --2- 172.21.15.64:0/2170865218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624f39545a0 0x5624f39637b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac009fd0 tx=0x7fc3ac00b040 comp rx=0 tx=0).stop 2023-09-13T03:12:59.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.750+0000 7fc3cc6e5b80 1 -- 172.21.15.64:0/2170865218 shutdown_connections 2023-09-13T03:12:59.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.750+0000 7fc3cc6e5b80 1 --2- 172.21.15.64:0/2170865218 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5624f39680a0 0x5624f396a4c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.750+0000 7fc3cc6e5b80 1 --2- 172.21.15.64:0/2170865218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624f39545a0 0x5624f39637b0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.750+0000 7fc3cc6e5b80 1 --2- 172.21.15.64:0/2170865218 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f3952e00 0x5624f395c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.750+0000 7fc3cc6e5b80 1 --2- 172.21.15.64:0/2170865218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f3953c00 0x5624f395c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.750+0000 7fc399ffb700 0 client.15489 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:12:59.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.750+0000 7fc3cc6e5b80 1 -- 172.21.15.64:0/2170865218 >> 172.21.15.64:0/2170865218 conn(0x5624f38c7090 msgr2=0x5624f394cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:59.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.750+0000 7fc3cc6e5b80 1 -- 172.21.15.64:0/2170865218 shutdown_connections 2023-09-13T03:12:59.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.750+0000 7fc3cc6e5b80 1 -- 172.21.15.64:0/2170865218 wait complete. 2023-09-13T03:12:59.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.750+0000 7fc3cc6e5b80 20 client.15489 trim_cache size 0 max 16384 2023-09-13T03:12:59.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.753+0000 7fb8f77ff700 1 -- 172.21.15.64:0/377572512 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fb8f010e2d0 con 0x7fb8f0073060 2023-09-13T03:12:59.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.767+0000 7fc3cc6e5b80 1 Processor -- start 2023-09-13T03:12:59.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.767+0000 7fc3cc6e5b80 1 -- start start 2023-09-13T03:12:59.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.768+0000 7fc3cc6e5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f3952e00 0x5624f3952710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:59.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.768+0000 7fc3cc6e5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f3953c00 0x5624f39636d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:59.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.768+0000 7fc3cc6e5b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624f39545a0 0x5624f395a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:59.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.768+0000 7fc3cc6e5b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f395b6e0 con 0x5624f39545a0 2023-09-13T03:12:59.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.768+0000 7fc3cc6e5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f394e670 con 0x5624f3953c00 2023-09-13T03:12:59.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.768+0000 7fc3cc6e5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f3964750 con 0x5624f3952e00 2023-09-13T03:12:59.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.768+0000 7fc3b2d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624f39545a0 0x5624f395a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:59.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.768+0000 7fc3b259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f3952e00 0x5624f3952710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:59.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.768+0000 7fc3b259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f3952e00 0x5624f3952710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:46850/0 (socket says 172.21.15.64:46850) 2023-09-13T03:12:59.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.768+0000 7fc3b259c700 1 -- 172.21.15.64:0/2576215431 learned_addr learned my addr 172.21.15.64:0/2576215431 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:59.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.768+0000 7fc3b1d9b700 1 --2- 172.21.15.64:0/2576215431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f3953c00 0x5624f39636d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:59.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.768+0000 7fc3b259c700 1 -- 172.21.15.64:0/2576215431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f3953c00 msgr2=0x5624f39636d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:59.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.768+0000 7fc3b259c700 1 --2- 172.21.15.64:0/2576215431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f3953c00 0x5624f39636d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.768+0000 7fc3b259c700 1 -- 172.21.15.64:0/2576215431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624f39545a0 msgr2=0x5624f395a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:59.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.768+0000 7fc3b259c700 1 --2- 172.21.15.64:0/2576215431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624f39545a0 0x5624f395a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.768+0000 7fc3b259c700 1 -- 172.21.15.64:0/2576215431 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624f36f7250 con 0x5624f3952e00 2023-09-13T03:12:59.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.768+0000 7fc3b1d9b700 1 --2- 172.21.15.64:0/2576215431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f3953c00 0x5624f39636d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:12:59.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.768+0000 7fc3b2d9d700 1 --2- 172.21.15.64:0/2576215431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624f39545a0 0x5624f395a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:12:59.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.769+0000 7fc3b259c700 1 --2- 172.21.15.64:0/2576215431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f3952e00 0x5624f3952710 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x5624f396add0 tx=0x7fc3a4009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0a44b6702173956 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:59.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.769+0000 7fc3a17fa700 1 -- 172.21.15.64:0/2576215431 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a4016030 con 0x5624f3952e00 2023-09-13T03:12:59.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.769+0000 7fc3a17fa700 1 -- 172.21.15.64:0/2576215431 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3a4017040 con 0x5624f3952e00 2023-09-13T03:12:59.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.769+0000 7fc3a17fa700 1 -- 172.21.15.64:0/2576215431 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a40053a0 con 0x5624f3952e00 2023-09-13T03:12:59.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.770+0000 7fc3cc6e5b80 1 -- 172.21.15.64:0/2576215431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f3952e00 msgr2=0x5624f3952710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:59.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.770+0000 7fc3cc6e5b80 1 --2- 172.21.15.64:0/2576215431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f3952e00 0x5624f3952710 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x5624f396add0 tx=0x7fc3a4009260 comp rx=0 tx=0).stop 2023-09-13T03:12:59.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.770+0000 7fc3cc6e5b80 1 -- 172.21.15.64:0/2576215431 shutdown_connections 2023-09-13T03:12:59.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.770+0000 7fc3cc6e5b80 1 --2- 172.21.15.64:0/2576215431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624f39545a0 0x5624f395a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.770+0000 7fc3cc6e5b80 1 --2- 172.21.15.64:0/2576215431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f3952e00 0x5624f3952710 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.770+0000 7fc3cc6e5b80 1 --2- 172.21.15.64:0/2576215431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f3953c00 0x5624f39636d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.770+0000 7fc3cc6e5b80 1 -- 172.21.15.64:0/2576215431 >> 172.21.15.64:0/2576215431 conn(0x5624f38c7090 msgr2=0x5624f3964040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:59.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.770+0000 7fc3cc6e5b80 1 -- 172.21.15.64:0/2576215431 shutdown_connections 2023-09-13T03:12:59.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.770+0000 7fc3cc6e5b80 1 -- 172.21.15.64:0/2576215431 wait complete. 2023-09-13T03:12:59.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.770+0000 7fc3cc6e5b80 1 -- start start 2023-09-13T03:12:59.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.770+0000 7fc3cc6e5b80 1 Processor -- start 2023-09-13T03:12:59.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.771+0000 7fc3cc6e5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f3952e00 0x5624f3960690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:59.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.771+0000 7fc3cc6e5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f3953c00 0x5624f3960bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:59.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.771+0000 7fc3cc6e5b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624f39545a0 0x5624f39bbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:59.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.771+0000 7fc3cc6e5b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f39612b0 con 0x5624f39545a0 2023-09-13T03:12:59.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.771+0000 7fc3cc6e5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f39bc400 con 0x5624f3953c00 2023-09-13T03:12:59.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.771+0000 7fc3cc6e5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f39bc540 con 0x5624f3952e00 2023-09-13T03:12:59.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.771+0000 7fc3cc6e5b80 10 client.0 _ll_register_callbacks cb 0x7fc3ac006040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:12:59.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.771+0000 7fc3b259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f3952e00 0x5624f3960690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:59.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.771+0000 7fc3b1d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f3953c00 0x5624f3960bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:59.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.771+0000 7fc3b259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f3952e00 0x5624f3960690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:46860/0 (socket says 172.21.15.64:46860) 2023-09-13T03:12:59.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.771+0000 7fc3b259c700 1 -- 172.21.15.64:0/2638222686 learned_addr learned my addr 172.21.15.64:0/2638222686 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:12:59.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.771+0000 7fc3b2d9d700 1 --2- 172.21.15.64:0/2638222686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624f39545a0 0x5624f39bbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:59.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.771+0000 7fc3b259c700 1 -- 172.21.15.64:0/2638222686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f3953c00 msgr2=0x5624f3960bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:59.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.771+0000 7fc3b259c700 1 --2- 172.21.15.64:0/2638222686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f3953c00 0x5624f3960bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.771+0000 7fc3b259c700 1 -- 172.21.15.64:0/2638222686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624f39545a0 msgr2=0x5624f39bbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:12:59.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.771+0000 7fc3b259c700 1 --2- 172.21.15.64:0/2638222686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624f39545a0 0x5624f39bbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.772+0000 7fc3b259c700 1 -- 172.21.15.64:0/2638222686 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5624f36b9c60 con 0x5624f3952e00 2023-09-13T03:12:59.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.772+0000 7fc3b259c700 1 --2- 172.21.15.64:0/2638222686 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f3952e00 0x5624f3960690 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc3a401f700 tx=0x7fc3a4009bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7592f9cc32519b76 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:12:59.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.772+0000 7fc399ffb700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:12:59.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.772+0000 7fc399ffb700 1 -- 172.21.15.64:0/2638222686 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a4016050 con 0x5624f3952e00 2023-09-13T03:12:59.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.772+0000 7fc3cc6e5b80 1 -- 172.21.15.64:0/2638222686 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc3a8077c70 con 0x5624f3952e00 2023-09-13T03:12:59.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.772+0000 7fc399ffb700 1 -- 172.21.15.64:0/2638222686 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fc3a4005120 con 0x5624f3952e00 2023-09-13T03:12:59.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.772+0000 7fc399ffb700 1 -- 172.21.15.64:0/2638222686 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3a4005300 con 0x5624f3952e00 2023-09-13T03:12:59.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.772+0000 7fc3cc6e5b80 10 client.25525 fetch_fsmap learned FSMap version 26 2023-09-13T03:12:59.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.772+0000 7fc3cc6e5b80 1 -- 172.21.15.64:0/2638222686 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7fc3a4009eb0 con 0x5624f3952e00 2023-09-13T03:12:59.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.773+0000 7fc399ffb700 1 -- 172.21.15.64:0/2638222686 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3a8077c70 con 0x5624f3952e00 2023-09-13T03:12:59.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.773+0000 7fc399ffb700 1 -- 172.21.15.64:0/2638222686 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc3a4003840 con 0x5624f3952e00 2023-09-13T03:12:59.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.773+0000 7fc3cc6e5b80 10 client.25525 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:12:59.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.773+0000 7fc3cc6e5b80 10 client.25525 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T03:12:59.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.773+0000 7fc3cc6e5b80 20 client.25525 populate_metadata read hostname 'smithi064' 2023-09-13T03:12:59.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.773+0000 7fc3cc6e5b80 1 --2- 172.21.15.64:0/2638222686 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5624f39bdf10 0x5624f39c0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:12:59.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.773+0000 7fc3cc6e5b80 4 client.25525 mds_command: new command op to 24686 tid=0[{"prefix": "scrub status"}] 2023-09-13T03:12:59.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.773+0000 7fc3cc6e5b80 1 -- 172.21.15.64:0/2638222686 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5624f389d920 con 0x5624f39bdf10 2023-09-13T03:12:59.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.773+0000 7fc3b1d9b700 1 --2- 172.21.15.64:0/2638222686 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5624f39bdf10 0x5624f39c0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:12:59.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.774+0000 7fc399ffb700 10 client.25525 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:12:59.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.774+0000 7fc3b1d9b700 1 --2- 172.21.15.64:0/2638222686 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5624f39bdf10 0x5624f39c0330 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a58225c17105573 server_cookie=497043e92d352e47 in_seq=0 out_seq=0 2023-09-13T03:12:59.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.774+0000 7fc399ffb700 1 -- 172.21.15.64:0/2638222686 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5624f389d920 con 0x5624f39bdf10 2023-09-13T03:12:59.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.774+0000 7fc399ffb700 10 client.25525 handle_command_reply: tid=0 2023-09-13T03:12:59.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.774+0000 7fc3cc6e5b80 1 client.25525 shutdown 2023-09-13T03:12:59.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.774+0000 7fc3cc6e5b80 1 -- 172.21.15.64:0/2638222686 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f3952e00 msgr2=0x5624f3960690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:59.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.774+0000 7fc3cc6e5b80 1 --2- 172.21.15.64:0/2638222686 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f3952e00 0x5624f3960690 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc3a401f700 tx=0x7fc3a4009bb0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.774+0000 7fc3cc6e5b80 1 -- 172.21.15.64:0/2638222686 shutdown_connections 2023-09-13T03:12:59.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.774+0000 7fc3cc6e5b80 1 --2- 172.21.15.64:0/2638222686 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5624f39bdf10 0x5624f39c0330 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.774+0000 7fc3cc6e5b80 1 --2- 172.21.15.64:0/2638222686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5624f39545a0 0x5624f39bbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.774+0000 7fc3cc6e5b80 1 --2- 172.21.15.64:0/2638222686 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f3952e00 0x5624f3960690 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.774+0000 7fc3cc6e5b80 1 --2- 172.21.15.64:0/2638222686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f3953c00 0x5624f3960bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.774+0000 7fc399ffb700 0 client.25525 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:12:59.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.774+0000 7fc3cc6e5b80 1 -- 172.21.15.64:0/2638222686 >> 172.21.15.64:0/2638222686 conn(0x5624f38c7090 msgr2=0x5624f395a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:59.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.774+0000 7fc3cc6e5b80 1 -- 172.21.15.64:0/2638222686 shutdown_connections 2023-09-13T03:12:59.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.774+0000 7fc3cc6e5b80 1 -- 172.21.15.64:0/2638222686 wait complete. 2023-09-13T03:12:59.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.774+0000 7fc3cc6e5b80 20 client.25525 trim_cache size 0 max 16384 2023-09-13T03:12:59.790 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-13T03:12:59.790 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-13T03:12:59.790 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2023-09-13T03:12:59.790 INFO:teuthology.orchestra.run.smithi064.stdout: "d0ff213d-a6dd-485a-b264-900af2aca808": { 2023-09-13T03:12:59.791 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2023-09-13T03:12:59.791 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "d0ff213d-a6dd-485a-b264-900af2aca808", 2023-09-13T03:12:59.791 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2023-09-13T03:12:59.791 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:12:59.791 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:12:59.792 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-13T03:12:59.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.775+0000 7fc3b8dff700 1 -- 172.21.15.64:0/1109899629 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc39c0a0640 msgr2=0x7fc39c0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:59.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.775+0000 7fc3b8dff700 1 --2- 172.21.15.64:0/1109899629 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc39c0a0640 0x7fc39c0a2b00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac009ec0 tx=0x7fc3ac0092f0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.775+0000 7fc3b8dff700 1 -- 172.21.15.64:0/1109899629 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3b4074640 msgr2=0x7fc3b419ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:12:59.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.775+0000 7fc3b8dff700 1 --2- 172.21.15.64:0/1109899629 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3b4074640 0x7fc3b419ce60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8004710 tx=0x7fc3a8009580 comp rx=0 tx=0).stop 2023-09-13T03:12:59.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.775+0000 7fc3b8dff700 1 -- 172.21.15.64:0/1109899629 shutdown_connections 2023-09-13T03:12:59.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.775+0000 7fc3b8dff700 1 --2- 172.21.15.64:0/1109899629 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fc39c0a0640 0x7fc39c0a2b00 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.775+0000 7fc3b8dff700 1 --2- 172.21.15.64:0/1109899629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3b410cbe0 0x7fc3b41a3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.775+0000 7fc3b8dff700 1 --2- 172.21.15.64:0/1109899629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3b4073060 0x7fc3b419c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.775+0000 7fc3b8dff700 1 --2- 172.21.15.64:0/1109899629 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3b4074640 0x7fc3b419ce60 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:12:59.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.775+0000 7fc3b8dff700 1 -- 172.21.15.64:0/1109899629 >> 172.21.15.64:0/1109899629 conn(0x7fc3b40fbb80 msgr2=0x7fc3b40fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:12:59.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.775+0000 7fc3b8dff700 1 -- 172.21.15.64:0/1109899629 shutdown_connections 2023-09-13T03:12:59.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:12:59.775+0000 7fc3b8dff700 1 -- 172.21.15.64:0/1109899629 wait complete. 2023-09-13T03:12:59.795 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:59 smithi064 ceph-mon[97675]: pgmap v333: 97 pgs: 97 active+clean; 646 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 12 op/s 2023-09-13T03:12:59.795 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:59 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/757725150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:12:59.795 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:12:59 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/3534189000' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:12:59.796 INFO:tasks.cephfs.filesystem:scrub status for tag:d0ff213d-a6dd-485a-b264-900af2aca808 - {'path': '/', 'tag': 'd0ff213d-a6dd-485a-b264-900af2aca808', 'options': 'recursive,force'} 2023-09-13T03:12:59.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:59 smithi170 ceph-mon[134328]: pgmap v333: 97 pgs: 97 active+clean; 646 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 12 op/s 2023-09-13T03:12:59.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:59 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/757725150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:12:59.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:12:59 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/3534189000' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:12:59.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:59 smithi142 ceph-mon[134082]: pgmap v333: 97 pgs: 97 active+clean; 646 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 12 op/s 2023-09-13T03:12:59.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:59 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/757725150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:12:59.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:12:59 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/3534189000' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:13:00.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.573+0000 7fb8e67fc700 1 -- 172.21.15.64:0/377572512 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7fb8ec095870 con 0x7fb8f0073060 2023-09-13T03:13:00.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.576+0000 7fb8f77ff700 1 -- 172.21.15.64:0/377572512 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fb8dc0a0640 msgr2=0x7fb8dc0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:00.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.576+0000 7fb8f77ff700 1 --2- 172.21.15.64:0/377572512 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fb8dc0a0640 0x7fb8dc0a2b00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0074360 tx=0x7fb8e0009310 comp rx=0 tx=0).stop 2023-09-13T03:13:00.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.576+0000 7fb8f77ff700 1 -- 172.21.15.64:0/377572512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8f0073060 msgr2=0x7fb8f01986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:00.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.576+0000 7fb8f77ff700 1 --2- 172.21.15.64:0/377572512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8f0073060 0x7fb8f01986d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00e9a0 tx=0x7fb8ec00ecb0 comp rx=0 tx=0).stop 2023-09-13T03:13:00.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.576+0000 7fb8f77ff700 1 -- 172.21.15.64:0/377572512 shutdown_connections 2023-09-13T03:13:00.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.576+0000 7fb8f77ff700 1 --2- 172.21.15.64:0/377572512 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fb8dc0a0640 0x7fb8dc0a2b00 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:00.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.576+0000 7fb8f77ff700 1 --2- 172.21.15.64:0/377572512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8f01084f0 0x7fb8f0199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:00.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.576+0000 7fb8f77ff700 1 --2- 172.21.15.64:0/377572512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8f0073060 0x7fb8f01986d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:00.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.576+0000 7fb8f77ff700 1 --2- 172.21.15.64:0/377572512 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8f0074640 0x7fb8f0198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:00.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.576+0000 7fb8f77ff700 1 -- 172.21.15.64:0/377572512 >> 172.21.15.64:0/377572512 conn(0x7fb8f00fbb80 msgr2=0x7fb8f00fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:00.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.580+0000 7fb8f77ff700 1 -- 172.21.15.64:0/377572512 shutdown_connections 2023-09-13T03:13:00.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.581+0000 7fb8f77ff700 1 -- 172.21.15.64:0/377572512 wait complete. 2023-09-13T03:13:00.615 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2023-09-13T03:13:00.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.825+0000 7fe59f1ff700 1 -- 172.21.15.64:0/2160349257 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe598074640 msgr2=0x7fe598072a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:00.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.825+0000 7fe59f1ff700 1 --2- 172.21.15.64:0/2160349257 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe598074640 0x7fe598072a90 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe58c009990 tx=0x7fe58c009ca0 comp rx=0 tx=0).stop 2023-09-13T03:13:00.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.825+0000 7fe59f1ff700 1 -- 172.21.15.64:0/2160349257 shutdown_connections 2023-09-13T03:13:00.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.825+0000 7fe59f1ff700 1 --2- 172.21.15.64:0/2160349257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe59810cc00 0x7fe59810eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:00.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.825+0000 7fe59f1ff700 1 --2- 172.21.15.64:0/2160349257 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe598074640 0x7fe598072a90 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:00.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.825+0000 7fe59f1ff700 1 --2- 172.21.15.64:0/2160349257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe598073060 0x7fe5980734e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:00.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.825+0000 7fe59f1ff700 1 -- 172.21.15.64:0/2160349257 >> 172.21.15.64:0/2160349257 conn(0x7fe5980fbb80 msgr2=0x7fe5980fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:00.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.825+0000 7fe59f1ff700 1 -- 172.21.15.64:0/2160349257 shutdown_connections 2023-09-13T03:13:00.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.825+0000 7fe59f1ff700 1 -- 172.21.15.64:0/2160349257 wait complete. 2023-09-13T03:13:00.829 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:00 smithi142 ceph-mon[134082]: scrub queued for path: / 2023-09-13T03:13:00.829 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:00 smithi142 ceph-mon[134082]: scrub summary: idle+waiting paths [/] 2023-09-13T03:13:00.829 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:00 smithi142 ceph-mon[134082]: scrub summary: active paths [/] 2023-09-13T03:13:00.829 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:00 smithi142 ceph-mon[134082]: scrub summary: idle+waiting paths [/] 2023-09-13T03:13:00.829 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:00 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/377572512' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-09-13T03:13:00.830 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:13:00 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: ignoring --setuser ceph since I am not root 2023-09-13T03:13:00.830 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:13:00 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: ignoring --setgroup ceph since I am not root 2023-09-13T03:13:00.830 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:13:00 smithi142 ceph-mgr[135593]: -- 172.21.15.142:0/4022556286 <== mon.2 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556e0c7b31e0 con 0x556e0c788800 2023-09-13T03:13:00.831 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:13:00 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:13:00.747+0000 7f329c069200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-13T03:13:00.831 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:13:00 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:13:00.826+0000 7f329c069200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-13T03:13:00.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.826+0000 7fe59f1ff700 1 Processor -- start 2023-09-13T03:13:00.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.826+0000 7fe59f1ff700 1 -- start start 2023-09-13T03:13:00.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.827+0000 7fe59f1ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe598073060 0x7fe59819ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:00.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.827+0000 7fe59f1ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe598074640 0x7fe59819cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:00.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.827+0000 7fe59f1ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe59810cc00 0x7fe59819de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:00.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.827+0000 7fe59f1ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59819e470 con 0x7fe59810cc00 2023-09-13T03:13:00.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.827+0000 7fe59f1ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59819e5e0 con 0x7fe598074640 2023-09-13T03:13:00.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.827+0000 7fe59f1ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59819e7a0 con 0x7fe598073060 2023-09-13T03:13:00.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.827+0000 7fe59cf9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe598073060 0x7fe59819ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:00.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.827+0000 7fe59cf9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe598073060 0x7fe59819ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:46876/0 (socket says 172.21.15.64:46876) 2023-09-13T03:13:00.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.827+0000 7fe59cf9b700 1 -- 172.21.15.64:0/2314186572 learned_addr learned my addr 172.21.15.64:0/2314186572 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:00.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.827+0000 7fe597fff700 1 --2- 172.21.15.64:0/2314186572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe598074640 0x7fe59819cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:00.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.827+0000 7fe59d79c700 1 --2- 172.21.15.64:0/2314186572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe59810cc00 0x7fe59819de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:00.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.827+0000 7fe59cf9b700 1 -- 172.21.15.64:0/2314186572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe598074640 msgr2=0x7fe59819cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:00.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.827+0000 7fe59cf9b700 1 --2- 172.21.15.64:0/2314186572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe598074640 0x7fe59819cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:00.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.827+0000 7fe59cf9b700 1 -- 172.21.15.64:0/2314186572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe59810cc00 msgr2=0x7fe59819de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:00.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.827+0000 7fe59cf9b700 1 --2- 172.21.15.64:0/2314186572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe59810cc00 0x7fe59819de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:00.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.827+0000 7fe59cf9b700 1 -- 172.21.15.64:0/2314186572 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe58c0095f0 con 0x7fe598073060 2023-09-13T03:13:00.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.828+0000 7fe59cf9b700 1 --2- 172.21.15.64:0/2314186572 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe598073060 0x7fe59819ca50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe58c00b680 tx=0x7fe58c005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:00.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.828+0000 7fe595ffb700 1 -- 172.21.15.64:0/2314186572 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58c01d070 con 0x7fe598073060 2023-09-13T03:13:00.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.828+0000 7fe59f1ff700 1 -- 172.21.15.64:0/2314186572 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe59819ea20 con 0x7fe598073060 2023-09-13T03:13:00.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.828+0000 7fe595ffb700 1 -- 172.21.15.64:0/2314186572 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe58c022470 con 0x7fe598073060 2023-09-13T03:13:00.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.828+0000 7fe595ffb700 1 -- 172.21.15.64:0/2314186572 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58c017650 con 0x7fe598073060 2023-09-13T03:13:00.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.828+0000 7fe59f1ff700 1 -- 172.21.15.64:0/2314186572 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5981a5240 con 0x7fe598073060 2023-09-13T03:13:00.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.829+0000 7fe59f1ff700 1 -- 172.21.15.64:0/2314186572 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe598102750 con 0x7fe598073060 2023-09-13T03:13:00.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.831+0000 7fe595ffb700 1 -- 172.21.15.64:0/2314186572 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 19) v1 ==== 135276+0+0 (secure 0 0 0) 0x7fe58c0225e0 con 0x7fe598073060 2023-09-13T03:13:00.838 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:13:00 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: ignoring --setuser ceph since I am not root 2023-09-13T03:13:00.839 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:13:00 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: ignoring --setgroup ceph since I am not root 2023-09-13T03:13:00.839 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:13:00 smithi170 ceph-mgr[136489]: -- 172.21.15.170:0/1165907280 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cb25fcf1e0 con 0x55cb25fa4c00 2023-09-13T03:13:00.839 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:13:00 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:13:00.756+0000 7fb78eba7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-13T03:13:00.839 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:00 smithi170 ceph-mon[134328]: scrub queued for path: / 2023-09-13T03:13:00.840 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:00 smithi170 ceph-mon[134328]: scrub summary: idle+waiting paths [/] 2023-09-13T03:13:00.840 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:00 smithi170 ceph-mon[134328]: scrub summary: active paths [/] 2023-09-13T03:13:00.840 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:00 smithi170 ceph-mon[134328]: scrub summary: idle+waiting paths [/] 2023-09-13T03:13:00.840 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:00 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/377572512' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-09-13T03:13:00.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.833+0000 7fe595ffb700 1 --2- 172.21.15.64:0/2314186572 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe5880a0710 0x7fe5880a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:00.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.833+0000 7fe595ffb700 1 -- 172.21.15.64:0/2314186572 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fe58c0cd8b0 con 0x7fe598073060 2023-09-13T03:13:00.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.833+0000 7fe597fff700 1 -- 172.21.15.64:0/2314186572 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe5880a0710 msgr2=0x7fe5880a2bd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/873679308 2023-09-13T03:13:00.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.833+0000 7fe597fff700 1 --2- 172.21.15.64:0/2314186572 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe5880a0710 0x7fe5880a2bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-13T03:13:00.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:00.839+0000 7fe595ffb700 1 -- 172.21.15.64:0/2314186572 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe58c0974c0 con 0x7fe598073060 2023-09-13T03:13:00.873 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:13:00 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: ignoring --setuser ceph since I am not root 2023-09-13T03:13:00.874 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:13:00 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: ignoring --setgroup ceph since I am not root 2023-09-13T03:13:00.874 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:13:00 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:13:00.872+0000 7fc9dd082200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-13T03:13:00.874 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:00 smithi064 ceph-mon[97675]: scrub queued for path: / 2023-09-13T03:13:00.875 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:00 smithi064 ceph-mon[97675]: scrub summary: idle+waiting paths [/] 2023-09-13T03:13:00.875 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:00 smithi064 ceph-mon[97675]: scrub summary: active paths [/] 2023-09-13T03:13:00.875 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:00 smithi064 ceph-mon[97675]: scrub summary: idle+waiting paths [/] 2023-09-13T03:13:00.875 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:00 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/377572512' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-09-13T03:13:01.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.034+0000 7fe597fff700 1 -- 172.21.15.64:0/2314186572 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe5880a0710 msgr2=0x7fe5880a2bd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/873679308 2023-09-13T03:13:01.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.034+0000 7fe597fff700 1 --2- 172.21.15.64:0/2314186572 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe5880a0710 0x7fe5880a2bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-13T03:13:01.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.054+0000 7fe59f1ff700 1 -- 172.21.15.64:0/2314186572 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fe59804e3e0 con 0x7fe598073060 2023-09-13T03:13:01.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.059+0000 7fe595ffb700 1 -- 172.21.15.64:0/2314186572 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v44)=0 v44) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe58c0270d0 con 0x7fe598073060 2023-09-13T03:13:01.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.062+0000 7fe59f1ff700 1 -- 172.21.15.64:0/2314186572 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe5880a0710 msgr2=0x7fe5880a2bd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-13T03:13:01.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.062+0000 7fe59f1ff700 1 --2- 172.21.15.64:0/2314186572 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe5880a0710 0x7fe5880a2bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:01.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.062+0000 7fe59f1ff700 1 -- 172.21.15.64:0/2314186572 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe598073060 msgr2=0x7fe59819ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:01.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.062+0000 7fe59f1ff700 1 --2- 172.21.15.64:0/2314186572 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe598073060 0x7fe59819ca50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe58c00b680 tx=0x7fe58c005740 comp rx=0 tx=0).stop 2023-09-13T03:13:01.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.062+0000 7fe59f1ff700 1 -- 172.21.15.64:0/2314186572 shutdown_connections 2023-09-13T03:13:01.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.062+0000 7fe59f1ff700 1 --2- 172.21.15.64:0/2314186572 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fe5880a0710 0x7fe5880a2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:01.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.062+0000 7fe59f1ff700 1 --2- 172.21.15.64:0/2314186572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe59810cc00 0x7fe59819de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:01.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.062+0000 7fe59f1ff700 1 --2- 172.21.15.64:0/2314186572 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe598073060 0x7fe59819ca50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:01.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.062+0000 7fe59f1ff700 1 --2- 172.21.15.64:0/2314186572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe598074640 0x7fe59819cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:01.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.062+0000 7fe59f1ff700 1 -- 172.21.15.64:0/2314186572 >> 172.21.15.64:0/2314186572 conn(0x7fe5980fbb80 msgr2=0x7fe5980fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:01.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.062+0000 7fe59f1ff700 1 -- 172.21.15.64:0/2314186572 shutdown_connections 2023-09-13T03:13:01.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.062+0000 7fe59f1ff700 1 -- 172.21.15.64:0/2314186572 wait complete. 2023-09-13T03:13:01.082 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2023-09-13T03:13:01.143 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:13:00 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:13:00.982+0000 7fc9dd082200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-13T03:13:01.143 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:13:01 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:13:01.106+0000 7fc9dd082200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-13T03:13:01.160 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:13:00 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:13:00.838+0000 7fb78eba7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-13T03:13:01.160 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:13:00 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:13:00.928+0000 7fb78eba7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-13T03:13:01.160 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:13:01 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:13:01.020+0000 7fb78eba7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-13T03:13:01.180 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:13:00 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:13:00.917+0000 7f329c069200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-13T03:13:01.180 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:13:01 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:13:01.009+0000 7f329c069200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-13T03:13:01.180 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:13:01 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:13:01.179+0000 7f329c069200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-13T03:13:01.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.285+0000 7fdeb3dff700 1 -- 172.21.15.64:0/3471263271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdeac074640 msgr2=0x7fdeac072a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:01.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.285+0000 7fdeb3dff700 1 --2- 172.21.15.64:0/3471263271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdeac074640 0x7fdeac072a90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fdea8009990 tx=0x7fdea8009ca0 comp rx=0 tx=0).stop 2023-09-13T03:13:01.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.285+0000 7fdeb3dff700 1 -- 172.21.15.64:0/3471263271 shutdown_connections 2023-09-13T03:13:01.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.285+0000 7fdeb3dff700 1 --2- 172.21.15.64:0/3471263271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdeac1084f0 0x7fdeac10a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:01.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.285+0000 7fdeb3dff700 1 --2- 172.21.15.64:0/3471263271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdeac074640 0x7fdeac072a90 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:01.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.285+0000 7fdeb3dff700 1 --2- 172.21.15.64:0/3471263271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdeac073060 0x7fdeac0734e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:01.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.285+0000 7fdeb3dff700 1 -- 172.21.15.64:0/3471263271 >> 172.21.15.64:0/3471263271 conn(0x7fdeac0fbb80 msgr2=0x7fdeac0fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:01.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.285+0000 7fdeb3dff700 1 -- 172.21.15.64:0/3471263271 shutdown_connections 2023-09-13T03:13:01.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.286+0000 7fdeb3dff700 1 -- 172.21.15.64:0/3471263271 wait complete. 2023-09-13T03:13:01.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.286+0000 7fdeb3dff700 1 Processor -- start 2023-09-13T03:13:01.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.287+0000 7fdeb3dff700 1 -- start start 2023-09-13T03:13:01.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.287+0000 7fdeb3dff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdeac073060 0x7fdeac198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:01.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.287+0000 7fdeb3dff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdeac074640 0x7fdeac198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:01.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.287+0000 7fdeb3dff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdeac1084f0 0x7fdeac199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:01.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.287+0000 7fdeb3dff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac19a060 con 0x7fdeac074640 2023-09-13T03:13:01.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.287+0000 7fdeb3dff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac19a1d0 con 0x7fdeac073060 2023-09-13T03:13:01.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.287+0000 7fdeb3dff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac19a390 con 0x7fdeac1084f0 2023-09-13T03:13:01.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.287+0000 7fdeb1b9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdeac073060 0x7fdeac198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:01.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.287+0000 7fdeb1b9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdeac073060 0x7fdeac198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:47816/0 (socket says 172.21.15.64:47816) 2023-09-13T03:13:01.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.287+0000 7fdeb1b9b700 1 -- 172.21.15.64:0/52391472 learned_addr learned my addr 172.21.15.64:0/52391472 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:01.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.287+0000 7fdeb239c700 1 --2- 172.21.15.64:0/52391472 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdeac1084f0 0x7fdeac199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:01.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.287+0000 7fdeb139a700 1 --2- 172.21.15.64:0/52391472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdeac074640 0x7fdeac198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:01.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.288+0000 7fdeb1b9b700 1 -- 172.21.15.64:0/52391472 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdeac1084f0 msgr2=0x7fdeac199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:01.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.288+0000 7fdeb1b9b700 1 --2- 172.21.15.64:0/52391472 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdeac1084f0 0x7fdeac199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:01.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.288+0000 7fdeb1b9b700 1 -- 172.21.15.64:0/52391472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdeac074640 msgr2=0x7fdeac198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:01.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.288+0000 7fdeb1b9b700 1 --2- 172.21.15.64:0/52391472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdeac074640 0x7fdeac198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:01.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.288+0000 7fdeb1b9b700 1 -- 172.21.15.64:0/52391472 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea80095f0 con 0x7fdeac073060 2023-09-13T03:13:01.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.288+0000 7fdeb139a700 1 --2- 172.21.15.64:0/52391472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdeac074640 0x7fdeac198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:13:01.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.288+0000 7fdeb1b9b700 1 --2- 172.21.15.64:0/52391472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdeac073060 0x7fdeac198640 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdea8004740 tx=0x7fdea8009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:01.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.288+0000 7fdea2ffd700 1 -- 172.21.15.64:0/52391472 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea801d070 con 0x7fdeac073060 2023-09-13T03:13:01.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.288+0000 7fdeb3dff700 1 -- 172.21.15.64:0/52391472 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeac19a610 con 0x7fdeac073060 2023-09-13T03:13:01.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.288+0000 7fdea2ffd700 1 -- 172.21.15.64:0/52391472 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdea8004830 con 0x7fdeac073060 2023-09-13T03:13:01.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.288+0000 7fdea2ffd700 1 -- 172.21.15.64:0/52391472 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea8005450 con 0x7fdeac073060 2023-09-13T03:13:01.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.288+0000 7fdeb3dff700 1 -- 172.21.15.64:0/52391472 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeac1a0e30 con 0x7fdeac073060 2023-09-13T03:13:01.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.289+0000 7fdeb3dff700 1 -- 172.21.15.64:0/52391472 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdeac106ba0 con 0x7fdeac073060 2023-09-13T03:13:01.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.291+0000 7fdea2ffd700 1 -- 172.21.15.64:0/52391472 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 135276+0+0 (secure 0 0 0) 0x7fdea8051020 con 0x7fdeac073060 2023-09-13T03:13:01.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.292+0000 7fdea2ffd700 1 --2- 172.21.15.64:0/52391472 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fde980a0690 0x7fde980a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:01.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.293+0000 7fdea2ffd700 1 -- 172.21.15.64:0/52391472 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fdea80cd140 con 0x7fdeac073060 2023-09-13T03:13:01.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.293+0000 7fdeb139a700 1 -- 172.21.15.64:0/52391472 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fde980a0690 msgr2=0x7fde980a2b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/873679308 2023-09-13T03:13:01.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.293+0000 7fdeb139a700 1 --2- 172.21.15.64:0/52391472 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fde980a0690 0x7fde980a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-13T03:13:01.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.294+0000 7fdea2ffd700 1 -- 172.21.15.64:0/52391472 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fdea8096cd0 con 0x7fdeac073060 2023-09-13T03:13:01.418 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:13:01 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:13:01.235+0000 7fc9dd082200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-13T03:13:01.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.494+0000 7fdeb139a700 1 -- 172.21.15.64:0/52391472 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fde980a0690 msgr2=0x7fde980a2b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/873679308 2023-09-13T03:13:01.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.494+0000 7fdeb139a700 1 --2- 172.21.15.64:0/52391472 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fde980a0690 0x7fde980a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-13T03:13:01.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.508+0000 7fdeb3dff700 1 -- 172.21.15.64:0/52391472 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fdeac10e070 con 0x7fdeac073060 2023-09-13T03:13:01.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.516+0000 7fdea2ffd700 1 -- 172.21.15.64:0/52391472 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v45)=0 v45) v1 ==== 133+0+0 (secure 0 0 0) 0x7fdea809a2f0 con 0x7fdeac073060 2023-09-13T03:13:01.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.519+0000 7fdeb3dff700 1 -- 172.21.15.64:0/52391472 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fde980a0690 msgr2=0x7fde980a2b50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-13T03:13:01.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.519+0000 7fdeb3dff700 1 --2- 172.21.15.64:0/52391472 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fde980a0690 0x7fde980a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:01.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.519+0000 7fdeb3dff700 1 -- 172.21.15.64:0/52391472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdeac073060 msgr2=0x7fdeac198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:01.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.519+0000 7fdeb3dff700 1 --2- 172.21.15.64:0/52391472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdeac073060 0x7fdeac198640 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdea8004740 tx=0x7fdea8009580 comp rx=0 tx=0).stop 2023-09-13T03:13:01.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.519+0000 7fdeb3dff700 1 -- 172.21.15.64:0/52391472 shutdown_connections 2023-09-13T03:13:01.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.519+0000 7fdeb3dff700 1 --2- 172.21.15.64:0/52391472 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fde980a0690 0x7fde980a2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:01.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.519+0000 7fdeb3dff700 1 --2- 172.21.15.64:0/52391472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdeac073060 0x7fdeac198640 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:01.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.519+0000 7fdeb3dff700 1 --2- 172.21.15.64:0/52391472 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdeac1084f0 0x7fdeac199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:01.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.519+0000 7fdeb3dff700 1 --2- 172.21.15.64:0/52391472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdeac074640 0x7fdeac198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:01.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.519+0000 7fdeb3dff700 1 -- 172.21.15.64:0/52391472 >> 172.21.15.64:0/52391472 conn(0x7fdeac0fbb80 msgr2=0x7fdeac0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:01.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.519+0000 7fdeb3dff700 1 -- 172.21.15.64:0/52391472 shutdown_connections 2023-09-13T03:13:01.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.519+0000 7fdeb3dff700 1 -- 172.21.15.64:0/52391472 wait complete. 2023-09-13T03:13:01.539 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1M' 2023-09-13T03:13:01.555 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:13:01 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:13:01.191+0000 7fb78eba7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-13T03:13:01.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.745+0000 7fa39ffff700 1 -- 172.21.15.64:0/772840950 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa398074640 msgr2=0x7fa398072a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:01.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.745+0000 7fa39ffff700 1 --2- 172.21.15.64:0/772840950 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa398074640 0x7fa398072a90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa394009990 tx=0x7fa394009ca0 comp rx=0 tx=0).stop 2023-09-13T03:13:01.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.745+0000 7fa39ffff700 1 -- 172.21.15.64:0/772840950 shutdown_connections 2023-09-13T03:13:01.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.745+0000 7fa39ffff700 1 --2- 172.21.15.64:0/772840950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa39810cbe0 0x7fa39810efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:01.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.745+0000 7fa39ffff700 1 --2- 172.21.15.64:0/772840950 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa398074640 0x7fa398072a90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:01.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.745+0000 7fa39ffff700 1 --2- 172.21.15.64:0/772840950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa398073060 0x7fa3980734e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:01.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.745+0000 7fa39ffff700 1 -- 172.21.15.64:0/772840950 >> 172.21.15.64:0/772840950 conn(0x7fa3980fbb40 msgr2=0x7fa3980fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:01.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.745+0000 7fa39ffff700 1 -- 172.21.15.64:0/772840950 shutdown_connections 2023-09-13T03:13:01.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.746+0000 7fa39ffff700 1 -- 172.21.15.64:0/772840950 wait complete. 2023-09-13T03:13:01.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.746+0000 7fa39ffff700 1 Processor -- start 2023-09-13T03:13:01.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.747+0000 7fa39ffff700 1 -- start start 2023-09-13T03:13:01.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.747+0000 7fa39ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa398073060 0x7fa39819ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:01.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.747+0000 7fa39ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa398074640 0x7fa39819cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:01.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.747+0000 7fa39ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa39810cbe0 0x7fa39819de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:01.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.747+0000 7fa39ffff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39819e470 con 0x7fa398074640 2023-09-13T03:13:01.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.747+0000 7fa39ffff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39819e5e0 con 0x7fa39810cbe0 2023-09-13T03:13:01.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.747+0000 7fa39ffff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39819e7a0 con 0x7fa398073060 2023-09-13T03:13:01.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.747+0000 7fa39d59a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa398074640 0x7fa39819cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:01.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.747+0000 7fa39dd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa398073060 0x7fa39819ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:01.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.747+0000 7fa39d59a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa398074640 0x7fa39819cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56394/0 (socket says 172.21.15.64:56394) 2023-09-13T03:13:01.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.747+0000 7fa39d59a700 1 -- 172.21.15.64:0/1490123833 learned_addr learned my addr 172.21.15.64:0/1490123833 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:01.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.747+0000 7fa39e59c700 1 --2- 172.21.15.64:0/1490123833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa39810cbe0 0x7fa39819de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:01.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.747+0000 7fa39dd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa398073060 0x7fa39819ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:46914/0 (socket says 172.21.15.64:46914) 2023-09-13T03:13:01.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.748+0000 7fa39d59a700 1 -- 172.21.15.64:0/1490123833 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa398073060 msgr2=0x7fa39819ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:01.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.748+0000 7fa39d59a700 1 --2- 172.21.15.64:0/1490123833 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa398073060 0x7fa39819ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:01.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.748+0000 7fa39d59a700 1 -- 172.21.15.64:0/1490123833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa39810cbe0 msgr2=0x7fa39819de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:01.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.748+0000 7fa39d59a700 1 --2- 172.21.15.64:0/1490123833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa39810cbe0 0x7fa39819de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:01.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.748+0000 7fa39d59a700 1 -- 172.21.15.64:0/1490123833 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3940095f0 con 0x7fa398074640 2023-09-13T03:13:01.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.748+0000 7fa39e59c700 1 --2- 172.21.15.64:0/1490123833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa39810cbe0 0x7fa39819de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:13:01.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.748+0000 7fa39dd9b700 1 --2- 172.21.15.64:0/1490123833 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa398073060 0x7fa39819ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:13:01.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.748+0000 7fa39d59a700 1 --2- 172.21.15.64:0/1490123833 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa398074640 0x7fa39819cf90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa38800e970 tx=0x7fa38800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:01.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.748+0000 7fa38effd700 1 -- 172.21.15.64:0/1490123833 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa388009790 con 0x7fa398074640 2023-09-13T03:13:01.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.748+0000 7fa38effd700 1 -- 172.21.15.64:0/1490123833 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa38800c7e0 con 0x7fa398074640 2023-09-13T03:13:01.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.748+0000 7fa39ffff700 1 -- 172.21.15.64:0/1490123833 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa39819ea80 con 0x7fa398074640 2023-09-13T03:13:01.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.748+0000 7fa38effd700 1 -- 172.21.15.64:0/1490123833 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa388003c70 con 0x7fa398074640 2023-09-13T03:13:01.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.748+0000 7fa39ffff700 1 -- 172.21.15.64:0/1490123833 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3981a5240 con 0x7fa398074640 2023-09-13T03:13:01.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.750+0000 7fa39ffff700 1 -- 172.21.15.64:0/1490123833 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa398102730 con 0x7fa398074640 2023-09-13T03:13:01.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.752+0000 7fa38effd700 1 -- 172.21.15.64:0/1490123833 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) v1 ==== 135276+0+0 (secure 0 0 0) 0x7fa38800cc50 con 0x7fa398074640 2023-09-13T03:13:01.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.753+0000 7fa38effd700 1 --2- 172.21.15.64:0/1490123833 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fa3840a0690 0x7fa3840a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:01.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.753+0000 7fa39dd9b700 1 -- 172.21.15.64:0/1490123833 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fa3840a0690 msgr2=0x7fa3840a2b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/873679308 2023-09-13T03:13:01.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.753+0000 7fa38effd700 1 -- 172.21.15.64:0/1490123833 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa388014070 con 0x7fa398074640 2023-09-13T03:13:01.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.753+0000 7fa39dd9b700 1 --2- 172.21.15.64:0/1490123833 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fa3840a0690 0x7fa3840a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-13T03:13:01.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.757+0000 7fa38effd700 1 -- 172.21.15.64:0/1490123833 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fa3880958f0 con 0x7fa398074640 2023-09-13T03:13:01.861 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:01 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/377572512' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-09-13T03:13:01.861 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:01 smithi142 ceph-mon[134082]: mgrmap e19: x(active, since 8m), standbys: z, y 2023-09-13T03:13:01.861 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:01 smithi142 ceph-mon[134082]: from='client.? ' entity='client.admin' 2023-09-13T03:13:01.861 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:01 smithi142 ceph-mon[134082]: from='client.? ' entity='client.admin' 2023-09-13T03:13:01.862 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:13:01 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:13:01.544+0000 7f329c069200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-13T03:13:01.862 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:13:01 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:13:01.676+0000 7f329c069200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-13T03:13:01.862 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:13:01 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:13:01.766+0000 7f329c069200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-13T03:13:01.862 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:13:01 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:13:01.860+0000 7f329c069200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-13T03:13:01.867 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:01 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/377572512' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-09-13T03:13:01.867 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:01 smithi170 ceph-mon[134328]: mgrmap e19: x(active, since 8m), standbys: z, y 2023-09-13T03:13:01.868 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:01 smithi170 ceph-mon[134328]: from='client.? ' entity='client.admin' 2023-09-13T03:13:01.868 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:01 smithi170 ceph-mon[134328]: from='client.? ' entity='client.admin' 2023-09-13T03:13:01.868 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:13:01 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:13:01.555+0000 7fb78eba7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-13T03:13:01.868 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:13:01 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:13:01.686+0000 7fb78eba7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-13T03:13:01.868 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:13:01 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:13:01.774+0000 7fb78eba7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-13T03:13:01.869 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:13:01 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:13:01.867+0000 7fb78eba7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-13T03:13:01.918 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:13:01 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:13:01.472+0000 7fc9dd082200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-13T03:13:01.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:01 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/377572512' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-09-13T03:13:01.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:01 smithi064 ceph-mon[97675]: mgrmap e19: x(active, since 8m), standbys: z, y 2023-09-13T03:13:01.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:01 smithi064 ceph-mon[97675]: from='client.? ' entity='client.admin' 2023-09-13T03:13:01.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:01 smithi064 ceph-mon[97675]: from='client.? ' entity='client.admin' 2023-09-13T03:13:01.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.954+0000 7fa39dd9b700 1 -- 172.21.15.64:0/1490123833 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fa3840a0690 msgr2=0x7fa3840a2b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/873679308 2023-09-13T03:13:01.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:01.954+0000 7fa39dd9b700 1 --2- 172.21.15.64:0/1490123833 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fa3840a0690 0x7fa3840a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-13T03:13:02.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:02.001+0000 7fa39ffff700 1 -- 172.21.15.64:0/1490123833 --> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fa39802cdc0 con 0x7fa3840a0690 2023-09-13T03:13:02.254 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:13:01 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:13:01.962+0000 7fc9dd082200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-13T03:13:02.254 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:13:02 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:13:02.132+0000 7fc9dd082200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-13T03:13:02.254 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:13:02 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:13:02.253+0000 7fc9dd082200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-13T03:13:02.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:02.355+0000 7fa39dd9b700 1 -- 172.21.15.64:0/1490123833 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fa3840a0690 msgr2=0x7fa3840a2b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/873679308 2023-09-13T03:13:02.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:02.355+0000 7fa39dd9b700 1 --2- 172.21.15.64:0/1490123833 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fa3840a0690 0x7fa3840a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-09-13T03:13:02.659 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:13:02 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:13:02.289+0000 7fb78eba7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-13T03:13:02.668 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:13:02 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:13:02.382+0000 7fc9dd082200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-13T03:13:02.745 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:13:02 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:13:02.283+0000 7f329c069200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-13T03:13:03.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:03.156+0000 7fa39dd9b700 1 -- 172.21.15.64:0/1490123833 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fa3840a0690 msgr2=0x7fa3840a2b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/873679308 2023-09-13T03:13:03.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:03.156+0000 7fa39dd9b700 1 --2- 172.21.15.64:0/1490123833 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fa3840a0690 0x7fa3840a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-09-13T03:13:03.245 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:13:02 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:13:02.950+0000 7f329c069200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-13T03:13:03.245 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:13:03 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:13:03.034+0000 7f329c069200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-13T03:13:03.410 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:13:02 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:13:02.943+0000 7fb78eba7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-13T03:13:03.410 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:13:03 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:13:03.027+0000 7fb78eba7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-13T03:13:03.418 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:13:02 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:13:02.939+0000 7fc9dd082200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-13T03:13:04.060 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:13:03 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:13:03.796+0000 7fb78eba7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-13T03:13:04.060 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:13:03 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:13:03.887+0000 7fb78eba7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-13T03:13:04.070 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:13:03 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:13:03.806+0000 7f329c069200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-13T03:13:04.070 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:13:03 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:13:03.897+0000 7f329c069200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-13T03:13:04.168 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:13:03 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:13:03.821+0000 7fc9dd082200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-13T03:13:04.168 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:13:03 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:13:03.932+0000 7fc9dd082200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-13T03:13:04.339 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:13:04 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:13:04.060+0000 7fb78eba7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-13T03:13:04.349 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:13:04 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:13:04.069+0000 7f329c069200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-13T03:13:04.659 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:13:04 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:13:04.339+0000 7fb78eba7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-13T03:13:04.660 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:13:04 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:13:04.569+0000 7fb78eba7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-13T03:13:04.726 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:13:04 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:13:04.348+0000 7f329c069200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-13T03:13:04.726 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:13:04 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:13:04.578+0000 7f329c069200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-13T03:13:04.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:04.758+0000 7fa39dd9b700 1 -- 172.21.15.64:0/1490123833 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fa3840a0690 msgr2=0x7fa3840a2b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/873679308 2023-09-13T03:13:04.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:04.758+0000 7fa39dd9b700 1 --2- 172.21.15.64:0/1490123833 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fa3840a0690 0x7fa3840a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-09-13T03:13:04.977 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:13:04 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:13:04.715+0000 7fb78eba7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-13T03:13:04.986 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:13:04 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:13:04.725+0000 7f329c069200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-13T03:13:05.245 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:13:04 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:13:04.986+0000 7f329c069200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-13T03:13:05.288 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:13:04 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:13:04.928+0000 7fc9dd082200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-13T03:13:05.288 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:13:05 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:13:05.052+0000 7fc9dd082200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-13T03:13:05.410 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:13:04 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:13:04.976+0000 7fb78eba7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-13T03:13:05.666 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:13:05 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:13:05.287+0000 7fc9dd082200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-13T03:13:05.919 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:13:05 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:13:05.666+0000 7fc9dd082200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-13T03:13:05.995 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:13:05 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:13:05.690+0000 7f329c069200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-13T03:13:05.995 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:13:05 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:13:05.820+0000 7f329c069200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-13T03:13:06.001 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:13:05 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:13:05.679+0000 7fb78eba7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-13T03:13:06.002 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:13:05 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:13:05.809+0000 7fb78eba7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-13T03:13:06.331 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:13:06 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:13:06.001+0000 7fb78eba7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-13T03:13:06.331 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:13:06 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:13:06.113+0000 7fb78eba7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-13T03:13:06.340 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:13:06 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:13:06.011+0000 7f329c069200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-13T03:13:06.340 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:13:06 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:13:06.124+0000 7f329c069200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-13T03:13:06.418 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:13:05 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:13:05.949+0000 7fc9dd082200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-13T03:13:06.418 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:13:06 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:13:06.152+0000 7fc9dd082200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-13T03:13:06.660 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:13:06 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:13:06.330+0000 7fb78eba7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-13T03:13:06.745 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:13:06 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:13:06.339+0000 7f329c069200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-13T03:13:06.918 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:13:06 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:13:06.510+0000 7fc9dd082200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-13T03:13:07.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:07.270+0000 7fa38effd700 1 -- 172.21.15.64:0/1490123833 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 20) v1 ==== 135276+0+0 (secure 0 0 0) 0x7fa38800c950 con 0x7fa398074640 2023-09-13T03:13:07.495 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:13:07 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y[135569]: 2023-09-13T03:13:07.208+0000 7f329c069200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-13T03:13:07.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:07 smithi142 ceph-mon[134082]: Standby manager daemon z restarted 2023-09-13T03:13:07.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:07 smithi142 ceph-mon[134082]: Standby manager daemon z started 2023-09-13T03:13:07.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:07 smithi142 ceph-mon[134082]: from='mgr.? 172.21.15.170:0/854444234' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-13T03:13:07.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:07 smithi142 ceph-mon[134082]: from='mgr.? 172.21.15.170:0/854444234' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-13T03:13:07.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:07 smithi142 ceph-mon[134082]: from='mgr.? 172.21.15.170:0/854444234' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-13T03:13:07.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:07 smithi142 ceph-mon[134082]: from='mgr.? 172.21.15.170:0/854444234' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-13T03:13:07.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:07 smithi142 ceph-mon[134082]: Standby manager daemon y restarted 2023-09-13T03:13:07.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:07 smithi142 ceph-mon[134082]: Standby manager daemon y started 2023-09-13T03:13:07.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:07 smithi142 ceph-mon[134082]: from='mgr.? 172.21.15.142:0/470358378' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-13T03:13:07.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:07 smithi142 ceph-mon[134082]: from='mgr.? 172.21.15.142:0/470358378' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-13T03:13:07.498 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:07 smithi142 ceph-mon[134082]: from='mgr.? 172.21.15.142:0/470358378' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-13T03:13:07.498 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:07 smithi142 ceph-mon[134082]: from='mgr.? 172.21.15.142:0/470358378' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-13T03:13:07.660 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:13:07 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z[136403]: 2023-09-13T03:13:07.195+0000 7fb78eba7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-13T03:13:07.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:07 smithi170 ceph-mon[134328]: Standby manager daemon z restarted 2023-09-13T03:13:07.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:07 smithi170 ceph-mon[134328]: Standby manager daemon z started 2023-09-13T03:13:07.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:07 smithi170 ceph-mon[134328]: from='mgr.? 172.21.15.170:0/854444234' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-13T03:13:07.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:07 smithi170 ceph-mon[134328]: from='mgr.? 172.21.15.170:0/854444234' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-13T03:13:07.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:07 smithi170 ceph-mon[134328]: from='mgr.? 172.21.15.170:0/854444234' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-13T03:13:07.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:07 smithi170 ceph-mon[134328]: from='mgr.? 172.21.15.170:0/854444234' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-13T03:13:07.662 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:07 smithi170 ceph-mon[134328]: Standby manager daemon y restarted 2023-09-13T03:13:07.662 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:07 smithi170 ceph-mon[134328]: Standby manager daemon y started 2023-09-13T03:13:07.662 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:07 smithi170 ceph-mon[134328]: from='mgr.? 172.21.15.142:0/470358378' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-13T03:13:07.662 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:07 smithi170 ceph-mon[134328]: from='mgr.? 172.21.15.142:0/470358378' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-13T03:13:07.662 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:07 smithi170 ceph-mon[134328]: from='mgr.? 172.21.15.142:0/470358378' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-13T03:13:07.663 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:07 smithi170 ceph-mon[134328]: from='mgr.? 172.21.15.142:0/470358378' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-13T03:13:07.667 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:13:07 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:13:07.484+0000 7fc9dd082200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-13T03:13:07.668 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:13:07 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:13:07.667+0000 7fc9dd082200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-13T03:13:07.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:07 smithi064 ceph-mon[97675]: Standby manager daemon z restarted 2023-09-13T03:13:07.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:07 smithi064 ceph-mon[97675]: Standby manager daemon z started 2023-09-13T03:13:07.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:07 smithi064 ceph-mon[97675]: from='mgr.? 172.21.15.170:0/854444234' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-13T03:13:07.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:07 smithi064 ceph-mon[97675]: from='mgr.? 172.21.15.170:0/854444234' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-13T03:13:07.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:07 smithi064 ceph-mon[97675]: from='mgr.? 172.21.15.170:0/854444234' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-13T03:13:07.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:07 smithi064 ceph-mon[97675]: from='mgr.? 172.21.15.170:0/854444234' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-13T03:13:07.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:07 smithi064 ceph-mon[97675]: Standby manager daemon y restarted 2023-09-13T03:13:07.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:07 smithi064 ceph-mon[97675]: Standby manager daemon y started 2023-09-13T03:13:07.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:07 smithi064 ceph-mon[97675]: from='mgr.? 172.21.15.142:0/470358378' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-13T03:13:07.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:07 smithi064 ceph-mon[97675]: from='mgr.? 172.21.15.142:0/470358378' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-13T03:13:07.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:07 smithi064 ceph-mon[97675]: from='mgr.? 172.21.15.142:0/470358378' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-13T03:13:07.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:07 smithi064 ceph-mon[97675]: from='mgr.? 172.21.15.142:0/470358378' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-13T03:13:07.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:07.961+0000 7fa39dd9b700 1 -- 172.21.15.64:0/1490123833 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fa3840a0690 msgr2=0x7fa3840a2b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/873679308 2023-09-13T03:13:07.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:07.961+0000 7fa39dd9b700 1 --2- 172.21.15.64:0/1490123833 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fa3840a0690 0x7fa3840a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-09-13T03:13:08.274 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:13:07 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:13:07.928+0000 7fc9dd082200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-13T03:13:08.274 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:13:08 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:13:08.083+0000 7fc9dd082200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-13T03:13:08.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:08 smithi170 ceph-mon[134328]: mgrmap e20: x(active, since 8m), standbys: y, z 2023-09-13T03:13:08.668 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:13:08 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:13:08.372+0000 7fc9dd082200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-13T03:13:08.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:08 smithi064 ceph-mon[97675]: mgrmap e20: x(active, since 8m), standbys: y, z 2023-09-13T03:13:08.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:08 smithi142 ceph-mon[134082]: mgrmap e20: x(active, since 8m), standbys: y, z 2023-09-13T03:13:09.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:09.505+0000 7fa38effd700 1 -- 172.21.15.64:0/1490123833 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mgrmap(e 21) v1 ==== 134944+0+0 (secure 0 0 0) 0x7fa3880c9910 con 0x7fa398074640 2023-09-13T03:13:09.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:09.505+0000 7fa38effd700 1 -- 172.21.15.64:0/1490123833 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fa3840a0690 msgr2=0x7fa3840a2b50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-13T03:13:09.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:09.505+0000 7fa38effd700 1 --2- 172.21.15.64:0/1490123833 >> [v2:172.21.15.64:6800/873679308,v1:172.21.15.64:6801/873679308] conn(0x7fa3840a0690 0x7fa3840a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:09.866 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:13:09 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:13:09.490+0000 7fc9dd082200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-13T03:13:09.866 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:13:09 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:13:09.704+0000 7fc94027d700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-13T03:13:09.866 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:13:09 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:13:09.704+0000 7fc94027d700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-13T03:13:09.866 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:13:09 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:13:09.704+0000 7fc94027d700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-13T03:13:09.867 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:13:09 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:13:09.704+0000 7fc94027d700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-13T03:13:09.867 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:13:09 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:13:09.704+0000 7fc94027d700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-13T03:13:09.867 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:13:09 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:13:09.705+0000 7fc93d277700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-13T03:13:09.867 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:13:09 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:13:09.705+0000 7fc93d277700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-13T03:13:09.867 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:13:09 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:13:09.705+0000 7fc93d277700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-13T03:13:09.868 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:13:09 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:13:09.705+0000 7fc93d277700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-13T03:13:09.868 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:13:09 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:13:09.705+0000 7fc93d277700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-13T03:13:10.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:10 smithi142 ceph-mon[134082]: Active manager daemon x restarted 2023-09-13T03:13:10.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:10 smithi142 ceph-mon[134082]: Activating manager daemon x 2023-09-13T03:13:10.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:10 smithi142 ceph-mon[134082]: osdmap e86: 12 total, 12 up, 12 in 2023-09-13T03:13:10.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:10 smithi142 ceph-mon[134082]: mgrmap e21: x(active, starting, since 0.0139249s), standbys: y, z 2023-09-13T03:13:10.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:10 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-13T03:13:10.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:10 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T03:13:10.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:10 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T03:13:10.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:10 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-13T03:13:10.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:10 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-13T03:13:10.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:10 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-13T03:13:10.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:10 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-13T03:13:10.498 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:10 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-13T03:13:10.498 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:10 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-13T03:13:10.498 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:10 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-13T03:13:10.498 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:10 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-13T03:13:10.498 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:10 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-13T03:13:10.499 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:10 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-13T03:13:10.499 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:10 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-13T03:13:10.499 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:10 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-13T03:13:10.499 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:10 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-13T03:13:10.499 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:10 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-13T03:13:10.500 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:10 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-13T03:13:10.500 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:10 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-13T03:13:10.500 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:10 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-13T03:13:10.500 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:10 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-13T03:13:10.500 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:10 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-13T03:13:10.500 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:10 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-13T03:13:10.501 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:10 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-13T03:13:10.501 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:10 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-13T03:13:10.501 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:10 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-13T03:13:10.501 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:10 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-13T03:13:10.501 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:10 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-13T03:13:10.502 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:10 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-13T03:13:10.502 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:10 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-13T03:13:10.502 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:10 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-13T03:13:10.502 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:10 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-13T03:13:10.502 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:10 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-13T03:13:10.503 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:10 smithi142 ceph-mon[134082]: Manager daemon x is now available 2023-09-13T03:13:10.503 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:10 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:13:10.503 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:10 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:13:10.503 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:10 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-13T03:13:10.503 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:10 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-13T03:13:10.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.511+0000 7fa38effd700 1 -- 172.21.15.64:0/1490123833 <== mon.0 v2:172.21.15.64:3300/0 9 ==== mgrmap(e 22) v1 ==== 135071+0+0 (secure 0 0 0) 0x7fa388095cc0 con 0x7fa398074640 2023-09-13T03:13:10.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.512+0000 7fa38effd700 1 --2- 172.21.15.64:0/1490123833 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fa38401b6c0 0x7fa38401baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:10.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.512+0000 7fa38effd700 1 -- 172.21.15.64:0/1490123833 --> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fa39802cdc0 con 0x7fa38401b6c0 2023-09-13T03:13:10.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.516+0000 7fa39dd9b700 1 --2- 172.21.15.64:0/1490123833 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fa38401b6c0 0x7fa38401baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:10.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.520+0000 7fa39dd9b700 1 --2- 172.21.15.64:0/1490123833 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fa38401b6c0 0x7fa38401baa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa39400b2e0 tx=0x7fa394005fb0 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:10.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.582+0000 7fa38effd700 1 -- 172.21.15.64:0/1490123833 <== mgr.15507 v2:172.21.15.64:6800/3860981465 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fa39802cdc0 con 0x7fa38401b6c0 2023-09-13T03:13:10.583 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule set for path / 2023-09-13T03:13:10.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.585+0000 7fa39ffff700 1 -- 172.21.15.64:0/1490123833 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fa38401b6c0 msgr2=0x7fa38401baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:10.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.585+0000 7fa39ffff700 1 --2- 172.21.15.64:0/1490123833 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fa38401b6c0 0x7fa38401baa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa39400b2e0 tx=0x7fa394005fb0 comp rx=0 tx=0).stop 2023-09-13T03:13:10.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.585+0000 7fa39ffff700 1 -- 172.21.15.64:0/1490123833 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa398074640 msgr2=0x7fa39819cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:10.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.585+0000 7fa39ffff700 1 --2- 172.21.15.64:0/1490123833 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa398074640 0x7fa39819cf90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa38800e970 tx=0x7fa38800ec80 comp rx=0 tx=0).stop 2023-09-13T03:13:10.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.585+0000 7fa39e59c700 1 -- 172.21.15.64:0/1490123833 reap_dead start 2023-09-13T03:13:10.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.585+0000 7fa39ffff700 1 -- 172.21.15.64:0/1490123833 shutdown_connections 2023-09-13T03:13:10.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.585+0000 7fa39ffff700 1 -- 172.21.15.64:0/1490123833 >> 172.21.15.64:0/1490123833 conn(0x7fa3980fbb40 msgr2=0x7fa3980fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:10.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.585+0000 7fa39ffff700 1 -- 172.21.15.64:0/1490123833 shutdown_connections 2023-09-13T03:13:10.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.585+0000 7fa39ffff700 1 -- 172.21.15.64:0/1490123833 wait complete. 2023-09-13T03:13:10.603 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:10 smithi064 ceph-mon[97675]: Active manager daemon x restarted 2023-09-13T03:13:10.603 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:10 smithi064 ceph-mon[97675]: Activating manager daemon x 2023-09-13T03:13:10.603 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:10 smithi064 ceph-mon[97675]: osdmap e86: 12 total, 12 up, 12 in 2023-09-13T03:13:10.603 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:10 smithi064 ceph-mon[97675]: mgrmap e21: x(active, starting, since 0.0139249s), standbys: y, z 2023-09-13T03:13:10.604 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:10 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-13T03:13:10.604 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:10 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T03:13:10.604 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:10 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T03:13:10.604 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:10 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-13T03:13:10.604 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:10 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-13T03:13:10.605 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:10 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-13T03:13:10.605 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:10 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-13T03:13:10.605 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:10 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-13T03:13:10.605 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:10 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-13T03:13:10.605 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:10 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-13T03:13:10.606 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:10 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-13T03:13:10.606 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:10 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-13T03:13:10.606 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:10 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-13T03:13:10.606 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:10 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-13T03:13:10.606 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:10 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-13T03:13:10.607 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:10 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-13T03:13:10.607 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:10 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-13T03:13:10.607 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:10 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-13T03:13:10.607 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:10 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-13T03:13:10.608 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:10 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-13T03:13:10.608 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:10 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-13T03:13:10.608 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:10 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-13T03:13:10.608 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:10 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-13T03:13:10.608 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:10 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-13T03:13:10.608 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:10 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-13T03:13:10.609 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:10 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-13T03:13:10.609 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:10 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-13T03:13:10.609 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:10 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-13T03:13:10.609 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:10 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-13T03:13:10.609 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:10 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-13T03:13:10.610 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:10 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-13T03:13:10.610 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:10 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-13T03:13:10.610 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:10 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-13T03:13:10.610 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:10 smithi064 ceph-mon[97675]: Manager daemon x is now available 2023-09-13T03:13:10.610 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:10 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:13:10.611 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:10 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:13:10.611 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:10 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-13T03:13:10.611 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:10 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-13T03:13:10.612 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6M3h' 2023-09-13T03:13:10.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:10 smithi170 ceph-mon[134328]: Active manager daemon x restarted 2023-09-13T03:13:10.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:10 smithi170 ceph-mon[134328]: Activating manager daemon x 2023-09-13T03:13:10.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:10 smithi170 ceph-mon[134328]: osdmap e86: 12 total, 12 up, 12 in 2023-09-13T03:13:10.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:10 smithi170 ceph-mon[134328]: mgrmap e21: x(active, starting, since 0.0139249s), standbys: y, z 2023-09-13T03:13:10.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:10 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-13T03:13:10.662 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:10 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-13T03:13:10.662 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:10 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-13T03:13:10.662 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:10 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-13T03:13:10.663 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:10 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-13T03:13:10.663 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:10 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-13T03:13:10.663 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:10 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-13T03:13:10.664 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:10 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-13T03:13:10.664 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:10 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-13T03:13:10.664 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:10 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-13T03:13:10.664 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:10 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-13T03:13:10.665 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:10 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-13T03:13:10.665 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:10 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-13T03:13:10.665 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:10 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-13T03:13:10.665 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:10 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-13T03:13:10.665 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:10 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-13T03:13:10.666 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:10 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-13T03:13:10.666 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:10 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-13T03:13:10.666 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:10 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-13T03:13:10.666 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:10 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-13T03:13:10.667 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:10 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-13T03:13:10.667 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:10 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-13T03:13:10.667 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:10 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-13T03:13:10.667 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:10 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-13T03:13:10.668 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:10 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-13T03:13:10.668 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:10 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-13T03:13:10.668 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:10 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-13T03:13:10.668 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:10 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-13T03:13:10.669 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:10 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-13T03:13:10.669 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:10 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-13T03:13:10.669 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:10 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-13T03:13:10.669 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:10 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-13T03:13:10.669 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:10 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-13T03:13:10.670 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:10 smithi170 ceph-mon[134328]: Manager daemon x is now available 2023-09-13T03:13:10.670 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:10 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:13:10.670 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:10 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:13:10.670 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:10 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-13T03:13:10.671 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:10 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-13T03:13:10.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.849+0000 7f92da1ff700 1 -- 172.21.15.64:0/3585026664 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92d41006f0 msgr2=0x7f92d4100b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:10.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.849+0000 7f92da1ff700 1 --2- 172.21.15.64:0/3585026664 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92d41006f0 0x7f92d4100b50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f92c8009990 tx=0x7f92c8009ca0 comp rx=0 tx=0).stop 2023-09-13T03:13:10.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.849+0000 7f92da1ff700 1 -- 172.21.15.64:0/3585026664 shutdown_connections 2023-09-13T03:13:10.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.849+0000 7f92da1ff700 1 --2- 172.21.15.64:0/3585026664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92d4101090 0x7f92d4107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:10.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.849+0000 7f92da1ff700 1 --2- 172.21.15.64:0/3585026664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92d40ff500 0x7f92d40ff920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:10.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.849+0000 7f92da1ff700 1 --2- 172.21.15.64:0/3585026664 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92d41006f0 0x7f92d4100b50 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:10.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.849+0000 7f92da1ff700 1 -- 172.21.15.64:0/3585026664 >> 172.21.15.64:0/3585026664 conn(0x7f92d40faa50 msgr2=0x7f92d40fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:10.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.849+0000 7f92da1ff700 1 -- 172.21.15.64:0/3585026664 shutdown_connections 2023-09-13T03:13:10.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.849+0000 7f92da1ff700 1 -- 172.21.15.64:0/3585026664 wait complete. 2023-09-13T03:13:10.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.850+0000 7f92da1ff700 1 Processor -- start 2023-09-13T03:13:10.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.850+0000 7f92da1ff700 1 -- start start 2023-09-13T03:13:10.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.851+0000 7f92da1ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92d40ff500 0x7f92d41985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:10.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.851+0000 7f92da1ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92d41006f0 0x7f92d4198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:10.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.851+0000 7f92da1ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92d4101090 0x7f92d41999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:10.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.851+0000 7f92da1ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d419a010 con 0x7f92d40ff500 2023-09-13T03:13:10.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.851+0000 7f92da1ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d419a180 con 0x7f92d41006f0 2023-09-13T03:13:10.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.851+0000 7f92da1ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d419a340 con 0x7f92d4101090 2023-09-13T03:13:10.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.851+0000 7f92d37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92d40ff500 0x7f92d41985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:10.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.851+0000 7f92d3fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92d4101090 0x7f92d41999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:10.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.851+0000 7f92d37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92d40ff500 0x7f92d41985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44794/0 (socket says 172.21.15.64:44794) 2023-09-13T03:13:10.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.851+0000 7f92d37fe700 1 -- 172.21.15.64:0/39287118 learned_addr learned my addr 172.21.15.64:0/39287118 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:10.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.851+0000 7f92d2ffd700 1 --2- 172.21.15.64:0/39287118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92d41006f0 0x7f92d4198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:10.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.852+0000 7f92d3fff700 1 -- 172.21.15.64:0/39287118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92d41006f0 msgr2=0x7f92d4198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:10.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.852+0000 7f92d3fff700 1 --2- 172.21.15.64:0/39287118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92d41006f0 0x7f92d4198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:10.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.852+0000 7f92d3fff700 1 -- 172.21.15.64:0/39287118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92d40ff500 msgr2=0x7f92d41985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:10.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.852+0000 7f92d3fff700 1 --2- 172.21.15.64:0/39287118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92d40ff500 0x7f92d41985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:10.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.852+0000 7f92d3fff700 1 -- 172.21.15.64:0/39287118 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c4009520 con 0x7f92d4101090 2023-09-13T03:13:10.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.852+0000 7f92d37fe700 1 --2- 172.21.15.64:0/39287118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92d40ff500 0x7f92d41985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:13:10.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.852+0000 7f92d3fff700 1 --2- 172.21.15.64:0/39287118 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92d4101090 0x7f92d41999b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f92cc00efc0 tx=0x7f92cc00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:10.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.852+0000 7f92d0ff9700 1 -- 172.21.15.64:0/39287118 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92cc018070 con 0x7f92d4101090 2023-09-13T03:13:10.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.852+0000 7f92d0ff9700 1 -- 172.21.15.64:0/39287118 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92cc00f040 con 0x7f92d4101090 2023-09-13T03:13:10.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.852+0000 7f92d0ff9700 1 -- 172.21.15.64:0/39287118 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92cc00cd40 con 0x7f92d4101090 2023-09-13T03:13:10.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.852+0000 7f92da1ff700 1 -- 172.21.15.64:0/39287118 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92c80095f0 con 0x7f92d4101090 2023-09-13T03:13:10.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.852+0000 7f92da1ff700 1 -- 172.21.15.64:0/39287118 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92d41a0c00 con 0x7f92d4101090 2023-09-13T03:13:10.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.854+0000 7f92da1ff700 1 -- 172.21.15.64:0/39287118 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92d404e3e0 con 0x7f92d4101090 2023-09-13T03:13:10.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.856+0000 7f92d0ff9700 1 -- 172.21.15.64:0/39287118 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) v1 ==== 135071+0+0 (secure 0 0 0) 0x7f92cc003a00 con 0x7f92d4101090 2023-09-13T03:13:10.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.857+0000 7f92d0ff9700 1 --2- 172.21.15.64:0/39287118 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f92bc0a04c0 0x7f92bc0a2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:10.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.857+0000 7f92d0ff9700 1 -- 172.21.15.64:0/39287118 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f92cc0cbcc0 con 0x7f92d4101090 2023-09-13T03:13:10.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.858+0000 7f92d37fe700 1 --2- 172.21.15.64:0/39287118 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f92bc0a04c0 0x7f92bc0a2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:10.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.859+0000 7f92d37fe700 1 --2- 172.21.15.64:0/39287118 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f92bc0a04c0 0x7f92bc0a2980 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f92c4009fd0 tx=0x7f92c4009260 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:10.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:10.859+0000 7f92d0ff9700 1 -- 172.21.15.64:0/39287118 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f92cc095a60 con 0x7f92d4101090 2023-09-13T03:13:11.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.111+0000 7f92da1ff700 1 -- 172.21.15.64:0/39287118 --> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}) v1 -- 0x7f92d4068160 con 0x7f92bc0a04c0 2023-09-13T03:13:11.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.168+0000 7f92d0ff9700 1 -- 172.21.15.64:0/39287118 <== mgr.15507 v2:172.21.15.64:6800/3860981465 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f92d4068160 con 0x7f92bc0a04c0 2023-09-13T03:13:11.169 INFO:teuthology.orchestra.run.smithi064.stdout:Retention added to path / 2023-09-13T03:13:11.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.172+0000 7f92da1ff700 1 -- 172.21.15.64:0/39287118 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f92bc0a04c0 msgr2=0x7f92bc0a2980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:11.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.172+0000 7f92da1ff700 1 --2- 172.21.15.64:0/39287118 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f92bc0a04c0 0x7f92bc0a2980 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f92c4009fd0 tx=0x7f92c4009260 comp rx=0 tx=0).stop 2023-09-13T03:13:11.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.172+0000 7f92da1ff700 1 -- 172.21.15.64:0/39287118 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92d4101090 msgr2=0x7f92d41999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:11.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.172+0000 7f92da1ff700 1 --2- 172.21.15.64:0/39287118 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92d4101090 0x7f92d41999b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f92cc00efc0 tx=0x7f92cc00c3a0 comp rx=0 tx=0).stop 2023-09-13T03:13:11.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.172+0000 7f92da1ff700 1 -- 172.21.15.64:0/39287118 shutdown_connections 2023-09-13T03:13:11.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.172+0000 7f92da1ff700 1 --2- 172.21.15.64:0/39287118 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f92bc0a04c0 0x7f92bc0a2980 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:11.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.172+0000 7f92da1ff700 1 --2- 172.21.15.64:0/39287118 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92d4101090 0x7f92d41999b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:11.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.172+0000 7f92da1ff700 1 --2- 172.21.15.64:0/39287118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92d40ff500 0x7f92d41985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:11.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.172+0000 7f92da1ff700 1 --2- 172.21.15.64:0/39287118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92d41006f0 0x7f92d4198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:11.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.172+0000 7f92da1ff700 1 -- 172.21.15.64:0/39287118 >> 172.21.15.64:0/39287118 conn(0x7f92d40faa50 msgr2=0x7f92d4067a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:11.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.172+0000 7f92da1ff700 1 -- 172.21.15.64:0/39287118 shutdown_connections 2023-09-13T03:13:11.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.172+0000 7f92da1ff700 1 -- 172.21.15.64:0/39287118 wait complete. 2023-09-13T03:13:11.200 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-09-13T03:13:11.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.414+0000 7f21d33ff700 1 -- 172.21.15.64:0/3921619448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21cc073060 msgr2=0x7f21cc0734e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:11.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.414+0000 7f21d33ff700 1 --2- 172.21.15.64:0/3921619448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21cc073060 0x7f21cc0734e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f21bc009990 tx=0x7f21bc009ca0 comp rx=0 tx=0).stop 2023-09-13T03:13:11.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.414+0000 7f21d33ff700 1 -- 172.21.15.64:0/3921619448 shutdown_connections 2023-09-13T03:13:11.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.414+0000 7f21d33ff700 1 --2- 172.21.15.64:0/3921619448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21cc10cc00 0x7f21cc10eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:11.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.414+0000 7f21d33ff700 1 --2- 172.21.15.64:0/3921619448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21cc074640 0x7f21cc072a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:11.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.414+0000 7f21d33ff700 1 --2- 172.21.15.64:0/3921619448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21cc073060 0x7f21cc0734e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:11.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.414+0000 7f21d33ff700 1 -- 172.21.15.64:0/3921619448 >> 172.21.15.64:0/3921619448 conn(0x7f21cc0fbb80 msgr2=0x7f21cc0fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:11.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.414+0000 7f21d33ff700 1 -- 172.21.15.64:0/3921619448 shutdown_connections 2023-09-13T03:13:11.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.414+0000 7f21d33ff700 1 -- 172.21.15.64:0/3921619448 wait complete. 2023-09-13T03:13:11.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.415+0000 7f21d33ff700 1 Processor -- start 2023-09-13T03:13:11.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.415+0000 7f21d33ff700 1 -- start start 2023-09-13T03:13:11.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.416+0000 7f21d33ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21cc073060 0x7f21cc19ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:11.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.416+0000 7f21d33ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21cc074640 0x7f21cc19cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:11.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.416+0000 7f21d33ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21cc10cc00 0x7f21cc19de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:11.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.416+0000 7f21d33ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21cc19e490 con 0x7f21cc073060 2023-09-13T03:13:11.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.416+0000 7f21d33ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21cc19e600 con 0x7f21cc10cc00 2023-09-13T03:13:11.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.416+0000 7f21d33ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21cc19e7c0 con 0x7f21cc074640 2023-09-13T03:13:11.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.416+0000 7f21d099a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21cc074640 0x7f21cc19cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:11.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.416+0000 7f21d119b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21cc073060 0x7f21cc19ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:11.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.416+0000 7f21d099a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21cc074640 0x7f21cc19cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:34096/0 (socket says 172.21.15.64:34096) 2023-09-13T03:13:11.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.416+0000 7f21d099a700 1 -- 172.21.15.64:0/3194394330 learned_addr learned my addr 172.21.15.64:0/3194394330 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:11.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.416+0000 7f21d199c700 1 --2- 172.21.15.64:0/3194394330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21cc10cc00 0x7f21cc19de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:11.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.416+0000 7f21d099a700 1 -- 172.21.15.64:0/3194394330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21cc10cc00 msgr2=0x7f21cc19de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:11.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.416+0000 7f21d099a700 1 --2- 172.21.15.64:0/3194394330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21cc10cc00 0x7f21cc19de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:11.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.416+0000 7f21d099a700 1 -- 172.21.15.64:0/3194394330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21cc073060 msgr2=0x7f21cc19ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:11.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.417+0000 7f21d099a700 1 --2- 172.21.15.64:0/3194394330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21cc073060 0x7f21cc19ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:11.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.417+0000 7f21d099a700 1 -- 172.21.15.64:0/3194394330 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21bc0095f0 con 0x7f21cc074640 2023-09-13T03:13:11.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.417+0000 7f21d199c700 1 --2- 172.21.15.64:0/3194394330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21cc10cc00 0x7f21cc19de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:13:11.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.417+0000 7f21d119b700 1 --2- 172.21.15.64:0/3194394330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21cc073060 0x7f21cc19ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:13:11.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.417+0000 7f21d099a700 1 --2- 172.21.15.64:0/3194394330 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21cc074640 0x7f21cc19cfb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f21bc005720 tx=0x7f21bc009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:11.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.417+0000 7f21c27fc700 1 -- 172.21.15.64:0/3194394330 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21bc01d070 con 0x7f21cc074640 2023-09-13T03:13:11.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.417+0000 7f21c27fc700 1 -- 172.21.15.64:0/3194394330 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f21bc021410 con 0x7f21cc074640 2023-09-13T03:13:11.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.417+0000 7f21c27fc700 1 -- 172.21.15.64:0/3194394330 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21bc017650 con 0x7f21cc074640 2023-09-13T03:13:11.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.417+0000 7f21d33ff700 1 -- 172.21.15.64:0/3194394330 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21cc19ea40 con 0x7f21cc074640 2023-09-13T03:13:11.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.417+0000 7f21d33ff700 1 -- 172.21.15.64:0/3194394330 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21cc1a5260 con 0x7f21cc074640 2023-09-13T03:13:11.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.419+0000 7f21d33ff700 1 -- 172.21.15.64:0/3194394330 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21cc102750 con 0x7f21cc074640 2023-09-13T03:13:11.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.420+0000 7f21c27fc700 1 -- 172.21.15.64:0/3194394330 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) v1 ==== 135071+0+0 (secure 0 0 0) 0x7f21bc021580 con 0x7f21cc074640 2023-09-13T03:13:11.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.422+0000 7f21c27fc700 1 --2- 172.21.15.64:0/3194394330 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f21b80a04c0 0x7f21b80a2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:11.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.422+0000 7f21c27fc700 1 -- 172.21.15.64:0/3194394330 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f21bc0cc9b0 con 0x7f21cc074640 2023-09-13T03:13:11.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.422+0000 7f21d119b700 1 --2- 172.21.15.64:0/3194394330 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f21b80a04c0 0x7f21b80a2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:11.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.423+0000 7f21d119b700 1 --2- 172.21.15.64:0/3194394330 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f21b80a04c0 0x7f21b80a2980 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f21c8011490 tx=0x7f21c8009260 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:11.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.426+0000 7f21c27fc700 1 -- 172.21.15.64:0/3194394330 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f21bc096870 con 0x7f21cc074640 2023-09-13T03:13:11.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.519+0000 7f21c27fc700 1 -- 172.21.15.64:0/3194394330 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mgrmap(e 23) v1 ==== 135379+0+0 (secure 0 0 0) 0x7f21bc096c40 con 0x7f21cc074640 2023-09-13T03:13:11.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.682+0000 7f21d33ff700 1 -- 172.21.15.64:0/3194394330 --> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f21cc060a40 con 0x7f21b80a04c0 2023-09-13T03:13:11.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.690+0000 7f21c27fc700 1 -- 172.21.15.64:0/3194394330 <== mgr.15507 v2:172.21.15.64:6800/3860981465 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+283 (secure 0 0 0) 0x7f21cc060a40 con 0x7f21b80a04c0 2023-09-13T03:13:11.691 INFO:teuthology.orchestra.run.smithi064.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1M", "retention": {"h": 3, "M": 6}, "start": "2023-09-13T00:00:00", "created": "2023-09-13T03:13:10", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2023-09-13T03:13:11.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.693+0000 7f21d33ff700 1 -- 172.21.15.64:0/3194394330 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f21b80a04c0 msgr2=0x7f21b80a2980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:11.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.693+0000 7f21d33ff700 1 --2- 172.21.15.64:0/3194394330 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f21b80a04c0 0x7f21b80a2980 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f21c8011490 tx=0x7f21c8009260 comp rx=0 tx=0).stop 2023-09-13T03:13:11.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.693+0000 7f21d33ff700 1 -- 172.21.15.64:0/3194394330 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21cc074640 msgr2=0x7f21cc19cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:11.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.693+0000 7f21d33ff700 1 --2- 172.21.15.64:0/3194394330 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21cc074640 0x7f21cc19cfb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f21bc005720 tx=0x7f21bc009580 comp rx=0 tx=0).stop 2023-09-13T03:13:11.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.693+0000 7f21d33ff700 1 -- 172.21.15.64:0/3194394330 shutdown_connections 2023-09-13T03:13:11.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.693+0000 7f21d33ff700 1 --2- 172.21.15.64:0/3194394330 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f21b80a04c0 0x7f21b80a2980 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:11.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.693+0000 7f21d33ff700 1 --2- 172.21.15.64:0/3194394330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21cc10cc00 0x7f21cc19de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:11.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.693+0000 7f21d33ff700 1 --2- 172.21.15.64:0/3194394330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21cc073060 0x7f21cc19ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:11.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.693+0000 7f21d33ff700 1 --2- 172.21.15.64:0/3194394330 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21cc074640 0x7f21cc19cfb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:11.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.693+0000 7f21d33ff700 1 -- 172.21.15.64:0/3194394330 >> 172.21.15.64:0/3194394330 conn(0x7f21cc0fbb80 msgr2=0x7f21cc0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:11.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.693+0000 7f21d33ff700 1 -- 172.21.15.64:0/3194394330 shutdown_connections 2023-09-13T03:13:11.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.693+0000 7f21d33ff700 1 -- 172.21.15.64:0/3194394330 wait complete. 2023-09-13T03:13:11.712 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-09-13T03:13:11.777 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:11 smithi064 ceph-mon[97675]: [13/Sep/2023:03:13:09] ENGINE Bus STARTING 2023-09-13T03:13:11.778 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:11 smithi064 ceph-mon[97675]: [13/Sep/2023:03:13:10] ENGINE Serving on http://172.21.15.64:8765 2023-09-13T03:13:11.778 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:11 smithi064 ceph-mon[97675]: [13/Sep/2023:03:13:10] ENGINE Serving on https://172.21.15.64:7150 2023-09-13T03:13:11.778 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:11 smithi064 ceph-mon[97675]: [13/Sep/2023:03:13:10] ENGINE Bus STARTED 2023-09-13T03:13:11.778 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:11 smithi064 ceph-mon[97675]: mgrmap e22: x(active, since 1.02004s), standbys: y, z 2023-09-13T03:13:11.779 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:11 smithi064 ceph-mon[97675]: from='client.15525 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:13:11.779 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:11 smithi064 ceph-mon[97675]: pgmap v3: 97 pgs: 97 active+clean; 1.0 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:13:11.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:11 smithi170 ceph-mon[134328]: [13/Sep/2023:03:13:09] ENGINE Bus STARTING 2023-09-13T03:13:11.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:11 smithi170 ceph-mon[134328]: [13/Sep/2023:03:13:10] ENGINE Serving on http://172.21.15.64:8765 2023-09-13T03:13:11.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:11 smithi170 ceph-mon[134328]: [13/Sep/2023:03:13:10] ENGINE Serving on https://172.21.15.64:7150 2023-09-13T03:13:11.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:11 smithi170 ceph-mon[134328]: [13/Sep/2023:03:13:10] ENGINE Bus STARTED 2023-09-13T03:13:11.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:11 smithi170 ceph-mon[134328]: mgrmap e22: x(active, since 1.02004s), standbys: y, z 2023-09-13T03:13:11.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:11 smithi170 ceph-mon[134328]: from='client.15525 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:13:11.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:11 smithi170 ceph-mon[134328]: pgmap v3: 97 pgs: 97 active+clean; 1.0 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:13:11.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.931+0000 7fb4b39ff700 1 -- 172.21.15.64:0/3402817420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4ac101080 msgr2=0x7fb4ac103470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:11.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.931+0000 7fb4b39ff700 1 --2- 172.21.15.64:0/3402817420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4ac101080 0x7fb4ac103470 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8009990 tx=0x7fb4a8009ca0 comp rx=0 tx=0).stop 2023-09-13T03:13:11.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.931+0000 7fb4b39ff700 1 -- 172.21.15.64:0/3402817420 shutdown_connections 2023-09-13T03:13:11.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.931+0000 7fb4b39ff700 1 --2- 172.21.15.64:0/3402817420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4ac1062e0 0x7fb4ac108750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:11.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.931+0000 7fb4b39ff700 1 --2- 172.21.15.64:0/3402817420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4ac101080 0x7fb4ac103470 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:11.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.931+0000 7fb4b39ff700 1 --2- 172.21.15.64:0/3402817420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4ac1039b0 0x7fb4ac105da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:11.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.931+0000 7fb4b39ff700 1 -- 172.21.15.64:0/3402817420 >> 172.21.15.64:0/3402817420 conn(0x7fb4ac0faa50 msgr2=0x7fb4ac0fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:11.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.932+0000 7fb4b39ff700 1 -- 172.21.15.64:0/3402817420 shutdown_connections 2023-09-13T03:13:11.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.932+0000 7fb4b39ff700 1 -- 172.21.15.64:0/3402817420 wait complete. 2023-09-13T03:13:11.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.932+0000 7fb4b39ff700 1 Processor -- start 2023-09-13T03:13:11.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.933+0000 7fb4b39ff700 1 -- start start 2023-09-13T03:13:11.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.933+0000 7fb4b39ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4ac101080 0x7fb4ac1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:11.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.933+0000 7fb4b39ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4ac1039b0 0x7fb4ac198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:11.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.933+0000 7fb4b39ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4ac1062e0 0x7fb4ac1999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:11.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.933+0000 7fb4b39ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ac19a000 con 0x7fb4ac101080 2023-09-13T03:13:11.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.933+0000 7fb4b39ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ac19a170 con 0x7fb4ac1039b0 2023-09-13T03:13:11.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.933+0000 7fb4b39ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ac19a330 con 0x7fb4ac1062e0 2023-09-13T03:13:11.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.933+0000 7fb4b179b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4ac101080 0x7fb4ac1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:11.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.933+0000 7fb4b179b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4ac101080 0x7fb4ac1985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44856/0 (socket says 172.21.15.64:44856) 2023-09-13T03:13:11.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.933+0000 7fb4b179b700 1 -- 172.21.15.64:0/1547765454 learned_addr learned my addr 172.21.15.64:0/1547765454 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:11.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.933+0000 7fb4b0f9a700 1 --2- 172.21.15.64:0/1547765454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4ac1039b0 0x7fb4ac198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:11.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.934+0000 7fb4b1f9c700 1 --2- 172.21.15.64:0/1547765454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4ac1062e0 0x7fb4ac1999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:11.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.934+0000 7fb4b0f9a700 1 -- 172.21.15.64:0/1547765454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4ac1062e0 msgr2=0x7fb4ac1999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:11.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.934+0000 7fb4b0f9a700 1 --2- 172.21.15.64:0/1547765454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4ac1062e0 0x7fb4ac1999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:11.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.934+0000 7fb4b0f9a700 1 -- 172.21.15.64:0/1547765454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4ac101080 msgr2=0x7fb4ac1985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:11.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.934+0000 7fb4b0f9a700 1 --2- 172.21.15.64:0/1547765454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4ac101080 0x7fb4ac1985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:11.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.934+0000 7fb4b0f9a700 1 -- 172.21.15.64:0/1547765454 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a80095f0 con 0x7fb4ac1039b0 2023-09-13T03:13:11.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.934+0000 7fb4b179b700 1 --2- 172.21.15.64:0/1547765454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4ac101080 0x7fb4ac1985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:13:11.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.934+0000 7fb4b0f9a700 1 --2- 172.21.15.64:0/1547765454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4ac1039b0 0x7fb4ac198b20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb49c00e9f0 tx=0x7fb49c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:11.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.934+0000 7fb4a27fc700 1 -- 172.21.15.64:0/1547765454 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49c00c9c0 con 0x7fb4ac1039b0 2023-09-13T03:13:11.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.934+0000 7fb4b39ff700 1 -- 172.21.15.64:0/1547765454 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4ac19a610 con 0x7fb4ac1039b0 2023-09-13T03:13:11.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.934+0000 7fb4a27fc700 1 -- 172.21.15.64:0/1547765454 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb49c010430 con 0x7fb4ac1039b0 2023-09-13T03:13:11.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.935+0000 7fb4b39ff700 1 -- 172.21.15.64:0/1547765454 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4ac1a0dd0 con 0x7fb4ac1039b0 2023-09-13T03:13:11.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.935+0000 7fb4a27fc700 1 -- 172.21.15.64:0/1547765454 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49c018620 con 0x7fb4ac1039b0 2023-09-13T03:13:11.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.936+0000 7fb4b39ff700 1 -- 172.21.15.64:0/1547765454 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4ac107ea0 con 0x7fb4ac1039b0 2023-09-13T03:13:11.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.938+0000 7fb4a27fc700 1 -- 172.21.15.64:0/1547765454 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 23) v1 ==== 135379+0+0 (secure 0 0 0) 0x7fb49c010860 con 0x7fb4ac1039b0 2023-09-13T03:13:11.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.939+0000 7fb4a27fc700 1 --2- 172.21.15.64:0/1547765454 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fb4980a07b0 0x7fb4980a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:11.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.940+0000 7fb4a27fc700 1 -- 172.21.15.64:0/1547765454 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fb49c014070 con 0x7fb4ac1039b0 2023-09-13T03:13:11.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.940+0000 7fb4b179b700 1 --2- 172.21.15.64:0/1547765454 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fb4980a07b0 0x7fb4980a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:11.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.940+0000 7fb4b179b700 1 --2- 172.21.15.64:0/1547765454 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fb4980a07b0 0x7fb4980a2c70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb4a800b2e0 tx=0x7fb4a8005fb0 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:11.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:11.941+0000 7fb4a27fc700 1 -- 172.21.15.64:0/1547765454 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb49c095f00 con 0x7fb4ac1039b0 2023-09-13T03:13:11.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:11 smithi142 ceph-mon[134082]: [13/Sep/2023:03:13:09] ENGINE Bus STARTING 2023-09-13T03:13:11.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:11 smithi142 ceph-mon[134082]: [13/Sep/2023:03:13:10] ENGINE Serving on http://172.21.15.64:8765 2023-09-13T03:13:11.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:11 smithi142 ceph-mon[134082]: [13/Sep/2023:03:13:10] ENGINE Serving on https://172.21.15.64:7150 2023-09-13T03:13:11.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:11 smithi142 ceph-mon[134082]: [13/Sep/2023:03:13:10] ENGINE Bus STARTED 2023-09-13T03:13:11.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:11 smithi142 ceph-mon[134082]: mgrmap e22: x(active, since 1.02004s), standbys: y, z 2023-09-13T03:13:11.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:11 smithi142 ceph-mon[134082]: from='client.15525 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:13:11.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:11 smithi142 ceph-mon[134082]: pgmap v3: 97 pgs: 97 active+clean; 1.0 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:13:12.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:12.222+0000 7fb4b39ff700 1 -- 172.21.15.64:0/1547765454 --> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fb4ac060a40 con 0x7fb4980a07b0 2023-09-13T03:13:12.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:12.230+0000 7fb4a27fc700 1 -- 172.21.15.64:0/1547765454 <== mgr.15507 v2:172.21.15.64:6800/3860981465 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7fb4ac060a40 con 0x7fb4980a07b0 2023-09-13T03:13:12.231 INFO:teuthology.orchestra.run.smithi064.stdout:/ 1M 6M3h 2023-09-13T03:13:12.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:12.232+0000 7fb4b39ff700 1 -- 172.21.15.64:0/1547765454 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fb4980a07b0 msgr2=0x7fb4980a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:12.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:12.232+0000 7fb4b39ff700 1 --2- 172.21.15.64:0/1547765454 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fb4980a07b0 0x7fb4980a2c70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb4a800b2e0 tx=0x7fb4a8005fb0 comp rx=0 tx=0).stop 2023-09-13T03:13:12.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:12.232+0000 7fb4b39ff700 1 -- 172.21.15.64:0/1547765454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4ac1039b0 msgr2=0x7fb4ac198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:12.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:12.232+0000 7fb4b39ff700 1 --2- 172.21.15.64:0/1547765454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4ac1039b0 0x7fb4ac198b20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb49c00e9f0 tx=0x7fb49c00ed00 comp rx=0 tx=0).stop 2023-09-13T03:13:12.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:12.232+0000 7fb4b39ff700 1 -- 172.21.15.64:0/1547765454 shutdown_connections 2023-09-13T03:13:12.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:12.232+0000 7fb4b39ff700 1 --2- 172.21.15.64:0/1547765454 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fb4980a07b0 0x7fb4980a2c70 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:12.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:12.232+0000 7fb4b39ff700 1 --2- 172.21.15.64:0/1547765454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4ac1062e0 0x7fb4ac1999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:12.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:12.232+0000 7fb4b39ff700 1 --2- 172.21.15.64:0/1547765454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4ac101080 0x7fb4ac1985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:12.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:12.232+0000 7fb4b39ff700 1 --2- 172.21.15.64:0/1547765454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4ac1039b0 0x7fb4ac198b20 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:12.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:12.232+0000 7fb4b39ff700 1 -- 172.21.15.64:0/1547765454 >> 172.21.15.64:0/1547765454 conn(0x7fb4ac0faa50 msgr2=0x7fb4ac0ff720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:12.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:12.232+0000 7fb4b39ff700 1 -- 172.21.15.64:0/1547765454 shutdown_connections 2023-09-13T03:13:12.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:12.232+0000 7fb4b39ff700 1 -- 172.21.15.64:0/1547765454 wait complete. 2023-09-13T03:13:12.251 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2023-09-13T03:13:12.326 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2023-09-13T03:13:12.339 INFO:teuthology.run_tasks:Running task background_exec... 2023-09-13T03:13:12.403 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi064.front.sepia.ceph.com 2023-09-13T03:13:12.404 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2023-09-13T03:13:12.409 INFO:teuthology.run_tasks:Running task workunit... 2023-09-13T03:13:12.434 INFO:tasks.workunit:Pulling workunits from ref 3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f 2023-09-13T03:13:12.434 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-09-13T03:13:12.435 INFO:tasks.workunit:timeout=3h 2023-09-13T03:13:12.435 INFO:tasks.workunit:cleanup=True 2023-09-13T03:13:12.436 DEBUG:teuthology.orchestra.run.smithi064:> stat -- /home/ubuntu/cephtest/mnt.0 2023-09-13T03:13:12.483 INFO:teuthology.orchestra.run.smithi064.stdout: File: /home/ubuntu/cephtest/mnt.0 2023-09-13T03:13:12.483 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2023-09-13T03:13:12.483 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 79h/121d Inode: 1099511628283 Links: 2 2023-09-13T03:13:12.483 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-13T03:13:12.484 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:cephfs_t:s0 2023-09-13T03:13:12.484 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2023-09-13 03:11:19.263366872 +0000 2023-09-13T03:13:12.484 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2023-09-13 03:11:19.263366872 +0000 2023-09-13T03:13:12.484 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2023-09-13 03:12:50.283584680 +0000 2023-09-13T03:13:12.484 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2023-09-13 03:11:19.263366872 +0000 2023-09-13T03:13:12.485 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2023-09-13T03:13:12.485 DEBUG:teuthology.orchestra.run.smithi064:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2023-09-13T03:13:12.612 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/batrick/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f 2023-09-13T03:13:12.660 INFO:tasks.workunit.client.0.smithi064.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-09-13T03:13:12.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:12 smithi170 ceph-mon[134328]: from='client.25600 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:13:12.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:12 smithi170 ceph-mon[134328]: pgmap v4: 97 pgs: 97 active+clean; 1.0 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:13:12.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:12 smithi170 ceph-mon[134328]: mgrmap e23: x(active, since 2s), standbys: y, z 2023-09-13T03:13:12.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:12 smithi170 ceph-mon[134328]: from='client.25606 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:13:12.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:12 smithi064 ceph-mon[97675]: from='client.25600 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:13:12.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:12 smithi064 ceph-mon[97675]: pgmap v4: 97 pgs: 97 active+clean; 1.0 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:13:12.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:12 smithi064 ceph-mon[97675]: mgrmap e23: x(active, since 2s), standbys: y, z 2023-09-13T03:13:12.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:12 smithi064 ceph-mon[97675]: from='client.25606 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:13:12.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:12 smithi142 ceph-mon[134082]: from='client.25600 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:13:12.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:12 smithi142 ceph-mon[134082]: pgmap v4: 97 pgs: 97 active+clean; 1.0 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:13:12.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:12 smithi142 ceph-mon[134082]: mgrmap e23: x(active, since 2s), standbys: y, z 2023-09-13T03:13:12.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:12 smithi142 ceph-mon[134082]: from='client.25606 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:13:13.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:13 smithi170 ceph-mon[134328]: from='client.25610 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:13:13.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:13 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:13.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:13 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:13.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:13 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:13.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:13 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:13.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:13 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:13.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:13 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:13.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:13 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:13:13.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:13 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:13:13.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:13 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:13:13.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:13 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:13:13.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:13 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:13.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:13 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:13.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:13 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:13.913 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:13 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:13.913 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:13 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:13:13.913 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:13 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:13:13.913 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:13 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:13:13.913 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:13 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:13:13.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:13 smithi064 ceph-mon[97675]: from='client.25610 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:13:13.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:13 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:13.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:13 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:13.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:13 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:13.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:13 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:13.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:13 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:13.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:13 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:13.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:13 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:13:13.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:13 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:13:13.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:13 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:13:13.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:13 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:13:13.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:13 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:13.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:13 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:13.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:13 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:13.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:13 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:13.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:13 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:13:13.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:13 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:13:13.922 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:13 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:13:13.922 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:13 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:13:13.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:13 smithi142 ceph-mon[134082]: from='client.25610 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:13:13.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:13 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:13.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:13 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:13.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:13 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:13.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:13 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:13.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:13 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:13.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:13 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:13.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:13 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:13:13.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:13 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:13:13.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:13 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:13:13.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:13 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:13:13.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:13 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:13.998 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:13 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:13.998 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:13 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:13.998 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:13 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:13.998 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:13 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:13:13.999 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:13 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:13:13.999 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:13 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:13:13.999 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:13 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:13:14.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:14 smithi170 ceph-mon[134328]: pgmap v5: 97 pgs: 97 active+clean; 1.0 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:13:14.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:14 smithi170 ceph-mon[134328]: Adjusting osd_memory_target on smithi142 to 196.2M 2023-09-13T03:13:14.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:14 smithi170 ceph-mon[134328]: Unable to set osd_memory_target on smithi142 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-09-13T03:13:14.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:14 smithi170 ceph-mon[134328]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-13T03:13:14.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:14 smithi170 ceph-mon[134328]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-13T03:13:14.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:14 smithi170 ceph-mon[134328]: mgrmap e24: x(active, since 4s), standbys: y, z 2023-09-13T03:13:14.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:14 smithi170 ceph-mon[134328]: scrub summary: idle 2023-09-13T03:13:14.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:14 smithi064 ceph-mon[97675]: pgmap v5: 97 pgs: 97 active+clean; 1.0 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:13:14.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:14 smithi064 ceph-mon[97675]: Adjusting osd_memory_target on smithi142 to 196.2M 2023-09-13T03:13:14.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:14 smithi064 ceph-mon[97675]: Unable to set osd_memory_target on smithi142 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-09-13T03:13:14.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:14 smithi064 ceph-mon[97675]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-13T03:13:14.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:14 smithi064 ceph-mon[97675]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-13T03:13:14.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:14 smithi064 ceph-mon[97675]: mgrmap e24: x(active, since 4s), standbys: y, z 2023-09-13T03:13:14.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:14 smithi064 ceph-mon[97675]: scrub summary: idle 2023-09-13T03:13:14.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:14 smithi142 ceph-mon[134082]: pgmap v5: 97 pgs: 97 active+clean; 1.0 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:13:14.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:14 smithi142 ceph-mon[134082]: Adjusting osd_memory_target on smithi142 to 196.2M 2023-09-13T03:13:14.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:14 smithi142 ceph-mon[134082]: Unable to set osd_memory_target on smithi142 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-09-13T03:13:14.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:14 smithi142 ceph-mon[134082]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-13T03:13:14.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:14 smithi142 ceph-mon[134082]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-13T03:13:14.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:14 smithi142 ceph-mon[134082]: mgrmap e24: x(active, since 4s), standbys: y, z 2023-09-13T03:13:14.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:14 smithi142 ceph-mon[134082]: scrub summary: idle 2023-09-13T03:13:16.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:16 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:16.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:16 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:16.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:16 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:13:16.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:16 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:13:16.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:16 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:13:16.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:16 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:13:16.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:16 smithi170 ceph-mon[134328]: Adjusting osd_memory_target on smithi064 to 220.9M 2023-09-13T03:13:16.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:16 smithi170 ceph-mon[134328]: Unable to set osd_memory_target on smithi064 to 231682662: error parsing value: Value '231682662' is below minimum 939524096 2023-09-13T03:13:16.412 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:16 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:13:16.412 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:16 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:13:16.412 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:16 smithi170 ceph-mon[134328]: Updating smithi064:/etc/ceph/ceph.conf 2023-09-13T03:13:16.412 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:16 smithi170 ceph-mon[134328]: Updating smithi142:/etc/ceph/ceph.conf 2023-09-13T03:13:16.413 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:16 smithi170 ceph-mon[134328]: Updating smithi170:/etc/ceph/ceph.conf 2023-09-13T03:13:16.413 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:16 smithi170 ceph-mon[134328]: Updating smithi170:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-13T03:13:16.413 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:16 smithi170 ceph-mon[134328]: Updating smithi142:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-13T03:13:16.413 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:16 smithi170 ceph-mon[134328]: pgmap v6: 97 pgs: 97 active+clean; 1.0 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:13:16.413 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:16 smithi170 ceph-mon[134328]: Updating smithi064:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-13T03:13:16.414 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:16 smithi170 ceph-mon[134328]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2023-09-13T03:13:16.414 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:16 smithi170 ceph-mon[134328]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2023-09-13T03:13:16.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:16 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:16.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:16 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:16.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:16 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:13:16.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:16 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:13:16.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:16 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:13:16.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:16 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:13:16.419 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:16 smithi064 ceph-mon[97675]: Adjusting osd_memory_target on smithi064 to 220.9M 2023-09-13T03:13:16.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:16 smithi064 ceph-mon[97675]: Unable to set osd_memory_target on smithi064 to 231682662: error parsing value: Value '231682662' is below minimum 939524096 2023-09-13T03:13:16.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:16 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:13:16.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:16 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:13:16.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:16 smithi064 ceph-mon[97675]: Updating smithi064:/etc/ceph/ceph.conf 2023-09-13T03:13:16.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:16 smithi064 ceph-mon[97675]: Updating smithi142:/etc/ceph/ceph.conf 2023-09-13T03:13:16.421 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:16 smithi064 ceph-mon[97675]: Updating smithi170:/etc/ceph/ceph.conf 2023-09-13T03:13:16.421 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:16 smithi064 ceph-mon[97675]: Updating smithi170:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-13T03:13:16.421 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:16 smithi064 ceph-mon[97675]: Updating smithi142:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-13T03:13:16.421 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:16 smithi064 ceph-mon[97675]: pgmap v6: 97 pgs: 97 active+clean; 1.0 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:13:16.421 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:16 smithi064 ceph-mon[97675]: Updating smithi064:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-13T03:13:16.421 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:16 smithi064 ceph-mon[97675]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2023-09-13T03:13:16.422 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:16 smithi064 ceph-mon[97675]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2023-09-13T03:13:16.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:16 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:16.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:16 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:16.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:16 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:13:16.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:16 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:13:16.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:16 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:13:16.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:16 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:13:16.497 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:16 smithi142 ceph-mon[134082]: Adjusting osd_memory_target on smithi064 to 220.9M 2023-09-13T03:13:16.498 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:16 smithi142 ceph-mon[134082]: Unable to set osd_memory_target on smithi064 to 231682662: error parsing value: Value '231682662' is below minimum 939524096 2023-09-13T03:13:16.498 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:16 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:13:16.498 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:16 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:13:16.498 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:16 smithi142 ceph-mon[134082]: Updating smithi064:/etc/ceph/ceph.conf 2023-09-13T03:13:16.499 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:16 smithi142 ceph-mon[134082]: Updating smithi142:/etc/ceph/ceph.conf 2023-09-13T03:13:16.499 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:16 smithi142 ceph-mon[134082]: Updating smithi170:/etc/ceph/ceph.conf 2023-09-13T03:13:16.499 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:16 smithi142 ceph-mon[134082]: Updating smithi170:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-13T03:13:16.499 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:16 smithi142 ceph-mon[134082]: Updating smithi142:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-13T03:13:16.499 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:16 smithi142 ceph-mon[134082]: pgmap v6: 97 pgs: 97 active+clean; 1.0 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-13T03:13:16.500 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:16 smithi142 ceph-mon[134082]: Updating smithi064:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-13T03:13:16.500 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:16 smithi142 ceph-mon[134082]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2023-09-13T03:13:16.500 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:16 smithi142 ceph-mon[134082]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2023-09-13T03:13:17.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:17 smithi142 ceph-mon[134082]: Updating smithi170:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-13T03:13:17.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:17 smithi142 ceph-mon[134082]: Updating smithi142:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-13T03:13:17.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:17 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:17.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:17 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:17.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:17 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:17.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:17 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:17.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:17 smithi142 ceph-mon[134082]: Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2023-09-13T03:13:17.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:17 smithi170 ceph-mon[134328]: Updating smithi170:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-13T03:13:17.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:17 smithi170 ceph-mon[134328]: Updating smithi142:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-13T03:13:17.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:17 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:17.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:17 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:17.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:17 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:17.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:17 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:17.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:17 smithi170 ceph-mon[134328]: Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2023-09-13T03:13:17.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:17 smithi064 ceph-mon[97675]: Updating smithi170:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-13T03:13:17.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:17 smithi064 ceph-mon[97675]: Updating smithi142:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-13T03:13:17.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:17 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:17.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:17 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:17.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:17 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:17.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:17 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:17.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:17 smithi064 ceph-mon[97675]: Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2023-09-13T03:13:18.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:18 smithi170 ceph-mon[134328]: Updating smithi064:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-13T03:13:18.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:18 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:18.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:18 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:18.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:18 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:18.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:18 smithi170 ceph-mon[134328]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 132 KiB/s wr, 71 op/s 2023-09-13T03:13:18.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:18 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:13:18.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:18 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:13:18.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:18 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:13:18.662 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:18 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:18.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:18 smithi064 ceph-mon[97675]: Updating smithi064:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-13T03:13:18.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:18 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:18.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:18 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:18.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:18 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:18.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:18 smithi064 ceph-mon[97675]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 132 KiB/s wr, 71 op/s 2023-09-13T03:13:18.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:18 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:13:18.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:18 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:13:18.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:18 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:13:18.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:18 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:18.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:18 smithi142 ceph-mon[134082]: Updating smithi064:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-13T03:13:18.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:18 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:18.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:18 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:18.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:18 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:18.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:18 smithi142 ceph-mon[134082]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 132 KiB/s wr, 71 op/s 2023-09-13T03:13:18.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:18 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:13:18.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:18 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:13:18.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:18 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:13:18.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:18 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:13:20.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:20 smithi142 ceph-mon[134082]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 103 KiB/s wr, 55 op/s 2023-09-13T03:13:20.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:20 smithi170 ceph-mon[134328]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 103 KiB/s wr, 55 op/s 2023-09-13T03:13:20.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:20 smithi064 ceph-mon[97675]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 103 KiB/s wr, 55 op/s 2023-09-13T03:13:22.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:22 smithi170 ceph-mon[134328]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 84 KiB/s wr, 50 op/s 2023-09-13T03:13:22.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:22 smithi064 ceph-mon[97675]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 84 KiB/s wr, 50 op/s 2023-09-13T03:13:22.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:22 smithi142 ceph-mon[134082]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 84 KiB/s wr, 50 op/s 2023-09-13T03:13:24.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:24 smithi170 ceph-mon[134328]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 45 op/s 2023-09-13T03:13:24.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:24 smithi064 ceph-mon[97675]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 45 op/s 2023-09-13T03:13:24.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:24 smithi142 ceph-mon[134082]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 45 op/s 2023-09-13T03:13:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.674+0000 7f74371ff700 1 -- 172.21.15.64:0/4094081934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74300ffe70 msgr2=0x7f743010c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.674+0000 7f74371ff700 1 --2- 172.21.15.64:0/4094081934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74300ffe70 0x7f743010c310 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f742800b0a0 tx=0x7f742800b3b0 comp rx=0 tx=0).stop 2023-09-13T03:13:25.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.674+0000 7f74371ff700 1 -- 172.21.15.64:0/4094081934 shutdown_connections 2023-09-13T03:13:25.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.674+0000 7f74371ff700 1 --2- 172.21.15.64:0/4094081934 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7430104420 0x7f7430104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:25.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.674+0000 7f74371ff700 1 --2- 172.21.15.64:0/4094081934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74300ffe70 0x7f743010c310 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:25.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.674+0000 7f74371ff700 1 --2- 172.21.15.64:0/4094081934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74300ff4b0 0x7f74300ff930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.674+0000 7f74371ff700 1 -- 172.21.15.64:0/4094081934 >> 172.21.15.64:0/4094081934 conn(0x7f7430076600 msgr2=0x7f7430076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.674+0000 7f74371ff700 1 -- 172.21.15.64:0/4094081934 shutdown_connections 2023-09-13T03:13:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.674+0000 7f74371ff700 1 -- 172.21.15.64:0/4094081934 wait complete. 2023-09-13T03:13:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.676+0000 7f74371ff700 1 Processor -- start 2023-09-13T03:13:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.676+0000 7f74371ff700 1 -- start start 2023-09-13T03:13:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.676+0000 7f74371ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74300ff4b0 0x7f743019ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.677+0000 7f74371ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74300ffe70 0x7f743019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.677+0000 7f74371ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7430104420 0x7f74301988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.677+0000 7f74371ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7430198f20 con 0x7f74300ff4b0 2023-09-13T03:13:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.677+0000 7f74371ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7430199090 con 0x7f7430104420 2023-09-13T03:13:25.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.677+0000 7f74371ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7430199250 con 0x7f74300ffe70 2023-09-13T03:13:25.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.677+0000 7f7434f9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74300ff4b0 0x7f743019ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:25.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.677+0000 7f7434f9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74300ff4b0 0x7f743019ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43746/0 (socket says 172.21.15.64:43746) 2023-09-13T03:13:25.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.677+0000 7f7434f9b700 1 -- 172.21.15.64:0/1167581509 learned_addr learned my addr 172.21.15.64:0/1167581509 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:25.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.677+0000 7f742ffff700 1 --2- 172.21.15.64:0/1167581509 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74300ffe70 0x7f743019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:25.682 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.677+0000 7f743579c700 1 --2- 172.21.15.64:0/1167581509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7430104420 0x7f74301988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:25.682 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.677+0000 7f7434f9b700 1 -- 172.21.15.64:0/1167581509 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74300ffe70 msgr2=0x7f743019efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:25.682 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.677+0000 7f7434f9b700 1 --2- 172.21.15.64:0/1167581509 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74300ffe70 0x7f743019efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:25.682 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.677+0000 7f7434f9b700 1 -- 172.21.15.64:0/1167581509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7430104420 msgr2=0x7f74301988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:25.682 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.677+0000 7f7434f9b700 1 --2- 172.21.15.64:0/1167581509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7430104420 0x7f74301988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:25.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.677+0000 7f7434f9b700 1 -- 172.21.15.64:0/1167581509 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7428009d00 con 0x7f74300ff4b0 2023-09-13T03:13:25.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.678+0000 7f7434f9b700 1 --2- 172.21.15.64:0/1167581509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74300ff4b0 0x7f743019ea60 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f742400e9a0 tx=0x7f742400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:25.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.678+0000 7f742dffb700 1 -- 172.21.15.64:0/1167581509 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7424009790 con 0x7f74300ff4b0 2023-09-13T03:13:25.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.678+0000 7f742dffb700 1 -- 172.21.15.64:0/1167581509 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7424010430 con 0x7f74300ff4b0 2023-09-13T03:13:25.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.678+0000 7f74371ff700 1 -- 172.21.15.64:0/1167581509 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7430199530 con 0x7f74300ff4b0 2023-09-13T03:13:25.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.678+0000 7f742dffb700 1 -- 172.21.15.64:0/1167581509 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7424018680 con 0x7f74300ff4b0 2023-09-13T03:13:25.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.678+0000 7f74371ff700 1 -- 172.21.15.64:0/1167581509 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74301a5a50 con 0x7f74300ff4b0 2023-09-13T03:13:25.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.683+0000 7f742dffb700 1 -- 172.21.15.64:0/1167581509 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 24) v1 ==== 135435+0+0 (secure 0 0 0) 0x7f74240105a0 con 0x7f74300ff4b0 2023-09-13T03:13:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.684+0000 7f744aa58b80 1 Processor -- start 2023-09-13T03:13:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.684+0000 7f744aa58b80 1 -- start start 2023-09-13T03:13:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.685+0000 7f742dffb700 1 --2- 172.21.15.64:0/1167581509 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f74200a08c0 0x7f74200a2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.685+0000 7f744aa58b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e59faf01a0 0x55e59faf0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.685+0000 7f744aa58b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e59faea300 0x55e59faea780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.685+0000 7f744aa58b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e59faeacc0 0x55e59faf6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:25.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.685+0000 7f744aa58b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59fa436d0 con 0x55e59faeacc0 2023-09-13T03:13:25.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.685+0000 7f744aa58b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59fa40ff0 con 0x55e59faf01a0 2023-09-13T03:13:25.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.685+0000 7f744aa58b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59fa3c8e0 con 0x55e59faea300 2023-09-13T03:13:25.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.685+0000 7f742dffb700 1 -- 172.21.15.64:0/1167581509 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f7424014070 con 0x7f74300ff4b0 2023-09-13T03:13:25.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.685+0000 7f7434f9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e59faf01a0 0x55e59faf0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.685+0000 7f743579c700 1 --2- 172.21.15.64:0/1167581509 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f74200a08c0 0x7f74200a2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.685+0000 7f742ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e59faea300 0x55e59faea780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.685+0000 7f7434f9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e59faf01a0 0x55e59faf0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:33358/0 (socket says 172.21.15.64:33358) 2023-09-13T03:13:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.685+0000 7f7434f9b700 1 -- 172.21.15.64:0/1203680681 learned_addr learned my addr 172.21.15.64:0/1203680681 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.685+0000 7f743579c700 1 --2- 172.21.15.64:0/1203680681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e59faeacc0 0x55e59faf6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.686+0000 7f742ffff700 1 -- 172.21.15.64:0/1203680681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e59faf01a0 msgr2=0x55e59faf0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.686+0000 7f742ffff700 1 --2- 172.21.15.64:0/1203680681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e59faf01a0 0x55e59faf0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.686+0000 7f742ffff700 1 -- 172.21.15.64:0/1203680681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e59faeacc0 msgr2=0x55e59faf6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.686+0000 7f742ffff700 1 --2- 172.21.15.64:0/1203680681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e59faeacc0 0x55e59faf6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.686+0000 7f742ffff700 1 -- 172.21.15.64:0/1203680681 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e59f868d80 con 0x55e59faea300 2023-09-13T03:13:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.686+0000 7f743579c700 1 --2- 172.21.15.64:0/1203680681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e59faeacc0 0x55e59faf6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:13:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.686+0000 7f742ffff700 1 --2- 172.21.15.64:0/1203680681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e59faea300 0x55e59faea780 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f741c009c80 tx=0x7f741c009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1224f7ec298dc27b server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.686+0000 7f743579c700 1 --2- 172.21.15.64:0/1167581509 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f74200a08c0 0x7f74200a2d80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f74280060b0 tx=0x7f7428006000 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.686+0000 7f741affd700 1 -- 172.21.15.64:0/1203680681 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c011cb0 con 0x55e59faea300 2023-09-13T03:13:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.686+0000 7f741affd700 1 -- 172.21.15.64:0/1203680681 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f741c011e10 con 0x55e59faea300 2023-09-13T03:13:25.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.686+0000 7f741affd700 1 -- 172.21.15.64:0/1203680681 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c005280 con 0x55e59faea300 2023-09-13T03:13:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.687+0000 7f744aa58b80 1 -- 172.21.15.64:0/1203680681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e59faea300 msgr2=0x55e59faea780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.687+0000 7f744aa58b80 1 --2- 172.21.15.64:0/1203680681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e59faea300 0x55e59faea780 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f741c009c80 tx=0x7f741c009f90 comp rx=0 tx=0).stop 2023-09-13T03:13:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.688+0000 7f744aa58b80 1 -- 172.21.15.64:0/1203680681 shutdown_connections 2023-09-13T03:13:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.688+0000 7f744aa58b80 1 --2- 172.21.15.64:0/1203680681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e59faf01a0 0x55e59faf0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.688+0000 7f744aa58b80 1 --2- 172.21.15.64:0/1203680681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e59faeacc0 0x55e59faf6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.688+0000 7f744aa58b80 1 --2- 172.21.15.64:0/1203680681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e59faea300 0x55e59faea780 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.688+0000 7f744aa58b80 1 -- 172.21.15.64:0/1203680681 >> 172.21.15.64:0/1203680681 conn(0x55e59fa5f030 msgr2=0x55e59fae5560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.688+0000 7f744aa58b80 1 -- 172.21.15.64:0/1203680681 shutdown_connections 2023-09-13T03:13:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.688+0000 7f744aa58b80 1 -- 172.21.15.64:0/1203680681 wait complete. 2023-09-13T03:13:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.688+0000 7f744aa58b80 1 -- start start 2023-09-13T03:13:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.689+0000 7f744aa58b80 1 Processor -- start 2023-09-13T03:13:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.690+0000 7f744aa58b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e59faea300 0x55e59faf6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.690+0000 7f744aa58b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e59faeacc0 0x55e59faf4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.690+0000 7f744aa58b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e59faf01a0 0x55e59faf45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.690+0000 7f744aa58b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59f9dab10 con 0x55e59faeacc0 2023-09-13T03:13:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.690+0000 7f744aa58b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59f9da810 con 0x55e59faea300 2023-09-13T03:13:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.690+0000 7f744aa58b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59f8be8c0 con 0x55e59faf01a0 2023-09-13T03:13:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.690+0000 7f744aa58b80 10 client.0 _ll_register_callbacks cb 0x55e59f924a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:13:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.690+0000 7f7434f9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e59faea300 0x55e59faf6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.690+0000 7f743579c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e59faf01a0 0x55e59faf45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.690+0000 7f7434f9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e59faea300 0x55e59faf6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:33360/0 (socket says 172.21.15.64:33360) 2023-09-13T03:13:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.690+0000 7f742ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e59faeacc0 0x55e59faf4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:25.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.690+0000 7f7434f9b700 1 -- 172.21.15.64:0/447680555 learned_addr learned my addr 172.21.15.64:0/447680555 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:25.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.690+0000 7f7434f9b700 1 -- 172.21.15.64:0/447680555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e59faf01a0 msgr2=0x55e59faf45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:25.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.690+0000 7f7434f9b700 1 --2- 172.21.15.64:0/447680555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e59faf01a0 0x55e59faf45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:25.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.690+0000 7f7434f9b700 1 -- 172.21.15.64:0/447680555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e59faeacc0 msgr2=0x55e59faf4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:25.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.690+0000 7f7434f9b700 1 --2- 172.21.15.64:0/447680555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e59faeacc0 0x55e59faf4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:25.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.690+0000 7f7434f9b700 1 -- 172.21.15.64:0/447680555 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f741c0098e0 con 0x55e59faea300 2023-09-13T03:13:25.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.691+0000 7f7434f9b700 1 --2- 172.21.15.64:0/447680555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e59faea300 0x55e59faf6ce0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7424099520 tx=0x7f74240c9e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6cea153d44c0b1c server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:25.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.691+0000 7f740ffff700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:13:25.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.691+0000 7f740ffff700 1 -- 172.21.15.64:0/447680555 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7424099c90 con 0x55e59faea300 2023-09-13T03:13:25.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.691+0000 7f740ffff700 1 -- 172.21.15.64:0/447680555 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f742400b8e0 con 0x55e59faea300 2023-09-13T03:13:25.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.691+0000 7f744aa58b80 1 -- 172.21.15.64:0/447680555 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e59f994490 con 0x55e59faea300 2023-09-13T03:13:25.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.691+0000 7f740ffff700 1 -- 172.21.15.64:0/447680555 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7424099590 con 0x55e59faea300 2023-09-13T03:13:25.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.691+0000 7f744aa58b80 10 client.25622 fetch_fsmap learned FSMap version 26 2023-09-13T03:13:25.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.691+0000 7f744aa58b80 1 -- 172.21.15.64:0/447680555 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55e59f851c00 con 0x55e59faea300 2023-09-13T03:13:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.692+0000 7f740ffff700 1 -- 172.21.15.64:0/447680555 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e59f994490 con 0x55e59faea300 2023-09-13T03:13:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.692+0000 7f740ffff700 1 -- 172.21.15.64:0/447680555 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f74240840b0 con 0x55e59faea300 2023-09-13T03:13:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.692+0000 7f744aa58b80 10 client.25622 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:13:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.692+0000 7f744aa58b80 10 client.25622 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:13:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.692+0000 7f744aa58b80 20 client.25622 populate_metadata read hostname 'smithi064' 2023-09-13T03:13:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.692+0000 7f744aa58b80 1 --2- 172.21.15.64:0/447680555 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55e59fb04380 0x55e59fb06770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.692+0000 7f744aa58b80 4 client.25622 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:13:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.692+0000 7f744aa58b80 1 -- 172.21.15.64:0/447680555 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e59fa51840 con 0x55e59fb04380 2023-09-13T03:13:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.692+0000 7f742ffff700 1 --2- 172.21.15.64:0/447680555 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55e59fb04380 0x55e59fb06770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.693+0000 7f742ffff700 1 --2- 172.21.15.64:0/447680555 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55e59fb04380 0x55e59fb06770 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cf5e29532086c43 server_cookie=6b91e6e2c1545b8e in_seq=0 out_seq=0 2023-09-13T03:13:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.693+0000 7f740ffff700 10 client.25622 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:13:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.694+0000 7f740ffff700 1 -- 172.21.15.64:0/447680555 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e59fa51840 con 0x55e59fb04380 2023-09-13T03:13:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.694+0000 7f740ffff700 10 client.25622 handle_command_reply: tid=0 2023-09-13T03:13:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.694+0000 7f744aa58b80 1 client.25622 shutdown 2023-09-13T03:13:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.694+0000 7f744aa58b80 1 -- 172.21.15.64:0/447680555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e59faea300 msgr2=0x55e59faf6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.694+0000 7f744aa58b80 1 --2- 172.21.15.64:0/447680555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e59faea300 0x55e59faf6ce0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7424099520 tx=0x7f74240c9e70 comp rx=0 tx=0).stop 2023-09-13T03:13:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.694+0000 7f744aa58b80 1 -- 172.21.15.64:0/447680555 shutdown_connections 2023-09-13T03:13:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.694+0000 7f744aa58b80 1 --2- 172.21.15.64:0/447680555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e59faea300 0x55e59faf6ce0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.694+0000 7f744aa58b80 1 --2- 172.21.15.64:0/447680555 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55e59fb04380 0x55e59fb06770 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.694+0000 7f744aa58b80 1 --2- 172.21.15.64:0/447680555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e59faf01a0 0x55e59faf45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.694+0000 7f740ffff700 0 client.25622 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:13:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.694+0000 7f744aa58b80 1 --2- 172.21.15.64:0/447680555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e59faeacc0 0x55e59faf4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.694+0000 7f744aa58b80 1 -- 172.21.15.64:0/447680555 >> 172.21.15.64:0/447680555 conn(0x55e59fa5f030 msgr2=0x55e59fae47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.694+0000 7f744aa58b80 1 -- 172.21.15.64:0/447680555 shutdown_connections 2023-09-13T03:13:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.694+0000 7f744aa58b80 1 -- 172.21.15.64:0/447680555 wait complete. 2023-09-13T03:13:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.695+0000 7f744aa58b80 20 client.25622 trim_cache size 0 max 16384 2023-09-13T03:13:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.712+0000 7f744aa58b80 1 Processor -- start 2023-09-13T03:13:25.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.712+0000 7f744aa58b80 1 -- start start 2023-09-13T03:13:25.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.712+0000 7f744aa58b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e59faea300 0x55e59fae9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:25.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.712+0000 7f744aa58b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e59faeacc0 0x55e59faecb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:25.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.712+0000 7f744aa58b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e59faf01a0 0x55e59faf6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:25.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.712+0000 7f744aa58b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59fadf8a0 con 0x55e59faeacc0 2023-09-13T03:13:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.712+0000 7f744aa58b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59fb019f0 con 0x55e59faf01a0 2023-09-13T03:13:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.712+0000 7f744aa58b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59fae6d10 con 0x55e59faea300 2023-09-13T03:13:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.713+0000 7f742ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e59faeacc0 0x55e59faecb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.713+0000 7f743579c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e59faf01a0 0x55e59faf6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.713+0000 7f743579c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e59faf01a0 0x55e59faf6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:33370/0 (socket says 172.21.15.64:33370) 2023-09-13T03:13:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.713+0000 7f742ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e59faeacc0 0x55e59faecb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43768/0 (socket says 172.21.15.64:43768) 2023-09-13T03:13:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.713+0000 7f743579c700 1 -- 172.21.15.64:0/435561895 learned_addr learned my addr 172.21.15.64:0/435561895 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.713+0000 7f7434f9b700 1 --2- 172.21.15.64:0/435561895 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e59faea300 0x55e59fae9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.713+0000 7f742ffff700 1 -- 172.21.15.64:0/435561895 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e59faea300 msgr2=0x55e59fae9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.713+0000 7f742ffff700 1 --2- 172.21.15.64:0/435561895 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e59faea300 0x55e59fae9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:25.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.713+0000 7f742ffff700 1 -- 172.21.15.64:0/435561895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e59faf01a0 msgr2=0x55e59faf6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:25.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.713+0000 7f742ffff700 1 --2- 172.21.15.64:0/435561895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e59faf01a0 0x55e59faf6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:25.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.713+0000 7f742ffff700 1 -- 172.21.15.64:0/435561895 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e59f88f1f0 con 0x55e59faeacc0 2023-09-13T03:13:25.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.713+0000 7f743579c700 1 --2- 172.21.15.64:0/435561895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e59faf01a0 0x55e59faf6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:13:25.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.714+0000 7f742ffff700 1 --2- 172.21.15.64:0/435561895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e59faeacc0 0x55e59faecb60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f741c004d10 tx=0x7f741c004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=710382e4f700c15d server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.714+0000 7f741b7fe700 1 -- 172.21.15.64:0/435561895 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c029070 con 0x55e59faeacc0 2023-09-13T03:13:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.714+0000 7f741b7fe700 1 -- 172.21.15.64:0/435561895 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f741c004b00 con 0x55e59faeacc0 2023-09-13T03:13:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.714+0000 7f741b7fe700 1 -- 172.21.15.64:0/435561895 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c003680 con 0x55e59faeacc0 2023-09-13T03:13:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.715+0000 7f744aa58b80 1 -- 172.21.15.64:0/435561895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e59faeacc0 msgr2=0x55e59faecb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.715+0000 7f744aa58b80 1 --2- 172.21.15.64:0/435561895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e59faeacc0 0x55e59faecb60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f741c004d10 tx=0x7f741c004a20 comp rx=0 tx=0).stop 2023-09-13T03:13:25.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.715+0000 7f744aa58b80 1 -- 172.21.15.64:0/435561895 shutdown_connections 2023-09-13T03:13:25.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.715+0000 7f744aa58b80 1 --2- 172.21.15.64:0/435561895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e59faf01a0 0x55e59faf6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:25.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.715+0000 7f744aa58b80 1 --2- 172.21.15.64:0/435561895 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e59faea300 0x55e59fae9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:25.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.715+0000 7f744aa58b80 1 --2- 172.21.15.64:0/435561895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e59faeacc0 0x55e59faecb60 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:25.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.715+0000 7f744aa58b80 1 -- 172.21.15.64:0/435561895 >> 172.21.15.64:0/435561895 conn(0x55e59fa5f030 msgr2=0x55e59fae92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:25.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.715+0000 7f744aa58b80 1 -- 172.21.15.64:0/435561895 shutdown_connections 2023-09-13T03:13:25.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.715+0000 7f744aa58b80 1 -- 172.21.15.64:0/435561895 wait complete. 2023-09-13T03:13:25.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.716+0000 7f744aa58b80 1 -- start start 2023-09-13T03:13:25.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.716+0000 7f744aa58b80 1 Processor -- start 2023-09-13T03:13:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.717+0000 7f744aa58b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e59faea300 0x55e59fafb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.717+0000 7f744aa58b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e59faeacc0 0x55e59faf9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.717+0000 7f744aa58b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e59faf01a0 0x55e59fafa000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.717+0000 7f744aa58b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59fafa6d0 con 0x55e59faf01a0 2023-09-13T03:13:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.717+0000 7f744aa58b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59fb581a0 con 0x55e59faea300 2023-09-13T03:13:25.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.717+0000 7f744aa58b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59fb582e0 con 0x55e59faeacc0 2023-09-13T03:13:25.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.717+0000 7f744aa58b80 10 client.0 _ll_register_callbacks cb 0x55e59f924a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:13:25.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.717+0000 7f743579c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e59faf01a0 0x55e59fafa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:25.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.717+0000 7f7434f9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e59faea300 0x55e59fafb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:25.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.717+0000 7f742ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e59faeacc0 0x55e59faf9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:25.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.717+0000 7f7434f9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e59faea300 0x55e59fafb670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:33376/0 (socket says 172.21.15.64:33376) 2023-09-13T03:13:25.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.717+0000 7f7434f9b700 1 -- 172.21.15.64:0/921012195 learned_addr learned my addr 172.21.15.64:0/921012195 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:25.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.718+0000 7f7434f9b700 1 -- 172.21.15.64:0/921012195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e59faeacc0 msgr2=0x55e59faf9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:25.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.718+0000 7f7434f9b700 1 --2- 172.21.15.64:0/921012195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e59faeacc0 0x55e59faf9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:25.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.718+0000 7f7434f9b700 1 -- 172.21.15.64:0/921012195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e59faf01a0 msgr2=0x55e59fafa000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:25.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.718+0000 7f7434f9b700 1 --2- 172.21.15.64:0/921012195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e59faf01a0 0x55e59fafa000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:25.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.718+0000 7f7434f9b700 1 -- 172.21.15.64:0/921012195 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7430068920 con 0x55e59faea300 2023-09-13T03:13:25.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.718+0000 7f7434f9b700 1 --2- 172.21.15.64:0/921012195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e59faea300 0x55e59fafb670 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f742400c2d0 tx=0x7f7424010c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c71981441a88865f server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:25.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.718+0000 7f740ffff700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:13:25.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.718+0000 7f740ffff700 1 -- 172.21.15.64:0/921012195 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74240990d0 con 0x55e59faea300 2023-09-13T03:13:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.718+0000 7f740ffff700 1 -- 172.21.15.64:0/921012195 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f7424099c90 con 0x55e59faea300 2023-09-13T03:13:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.718+0000 7f744aa58b80 1 -- 172.21.15.64:0/921012195 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f741c011e10 con 0x55e59faea300 2023-09-13T03:13:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.718+0000 7f740ffff700 1 -- 172.21.15.64:0/921012195 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7424099590 con 0x55e59faea300 2023-09-13T03:13:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.718+0000 7f744aa58b80 10 client.25628 fetch_fsmap learned FSMap version 26 2023-09-13T03:13:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.718+0000 7f744aa58b80 1 -- 172.21.15.64:0/921012195 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f741c01d040 con 0x55e59faea300 2023-09-13T03:13:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.719+0000 7f740ffff700 1 -- 172.21.15.64:0/921012195 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f741c011e10 con 0x55e59faea300 2023-09-13T03:13:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.719+0000 7f740ffff700 1 -- 172.21.15.64:0/921012195 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7424098a80 con 0x55e59faea300 2023-09-13T03:13:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.719+0000 7f744aa58b80 10 client.25628 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:13:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.719+0000 7f744aa58b80 10 client.25628 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:13:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.719+0000 7f744aa58b80 20 client.25628 populate_metadata read hostname 'smithi064' 2023-09-13T03:13:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.719+0000 7f744aa58b80 1 --2- 172.21.15.64:0/921012195 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55e59fb596c0 0x55e59fb5bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.719+0000 7f744aa58b80 4 client.25628 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:13:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.719+0000 7f744aa58b80 1 -- 172.21.15.64:0/921012195 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e59fa358c0 con 0x55e59fb596c0 2023-09-13T03:13:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.719+0000 7f742ffff700 1 --2- 172.21.15.64:0/921012195 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55e59fb596c0 0x55e59fb5bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.720+0000 7f742ffff700 1 --2- 172.21.15.64:0/921012195 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55e59fb596c0 0x55e59fb5bae0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c46339cd0b45f631 server_cookie=6b493e02fde92c29 in_seq=0 out_seq=0 2023-09-13T03:13:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.720+0000 7f740ffff700 10 client.25628 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:13:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.732+0000 7f740ffff700 1 -- 172.21.15.64:0/921012195 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e59fa358c0 con 0x55e59fb596c0 2023-09-13T03:13:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.732+0000 7f740ffff700 10 client.25628 handle_command_reply: tid=0 2023-09-13T03:13:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.732+0000 7f744aa58b80 1 client.25628 shutdown 2023-09-13T03:13:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.732+0000 7f744aa58b80 1 -- 172.21.15.64:0/921012195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e59faea300 msgr2=0x55e59fafb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.732+0000 7f744aa58b80 1 --2- 172.21.15.64:0/921012195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e59faea300 0x55e59fafb670 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f742400c2d0 tx=0x7f7424010c80 comp rx=0 tx=0).stop 2023-09-13T03:13:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.732+0000 7f744aa58b80 1 -- 172.21.15.64:0/921012195 shutdown_connections 2023-09-13T03:13:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.732+0000 7f744aa58b80 1 --2- 172.21.15.64:0/921012195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e59faea300 0x55e59fafb670 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.732+0000 7f744aa58b80 1 --2- 172.21.15.64:0/921012195 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55e59fb596c0 0x55e59fb5bae0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.732+0000 7f744aa58b80 1 --2- 172.21.15.64:0/921012195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e59faf01a0 0x55e59fafa000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.732+0000 7f744aa58b80 1 --2- 172.21.15.64:0/921012195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e59faeacc0 0x55e59faf9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.732+0000 7f740ffff700 0 client.25628 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:13:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.732+0000 7f744aa58b80 1 -- 172.21.15.64:0/921012195 >> 172.21.15.64:0/921012195 conn(0x55e59fa5f030 msgr2=0x55e59fae3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.732+0000 7f744aa58b80 1 -- 172.21.15.64:0/921012195 shutdown_connections 2023-09-13T03:13:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.732+0000 7f744aa58b80 1 -- 172.21.15.64:0/921012195 wait complete. 2023-09-13T03:13:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.732+0000 7f744aa58b80 20 client.25628 trim_cache size 0 max 16384 2023-09-13T03:13:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:13:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:13:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:13:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:13:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.733+0000 7f74371ff700 1 -- 172.21.15.64:0/1167581509 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f74200a08c0 msgr2=0x7f74200a2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.733+0000 7f74371ff700 1 --2- 172.21.15.64:0/1167581509 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f74200a08c0 0x7f74200a2d80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f74280060b0 tx=0x7f7428006000 comp rx=0 tx=0).stop 2023-09-13T03:13:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.733+0000 7f74371ff700 1 -- 172.21.15.64:0/1167581509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74300ff4b0 msgr2=0x7f743019ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.733+0000 7f74371ff700 1 --2- 172.21.15.64:0/1167581509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74300ff4b0 0x7f743019ea60 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f742400e9a0 tx=0x7f742400ecb0 comp rx=0 tx=0).stop 2023-09-13T03:13:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.733+0000 7f74371ff700 1 -- 172.21.15.64:0/1167581509 shutdown_connections 2023-09-13T03:13:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.733+0000 7f74371ff700 1 --2- 172.21.15.64:0/1167581509 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f74200a08c0 0x7f74200a2d80 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.733+0000 7f74371ff700 1 --2- 172.21.15.64:0/1167581509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7430104420 0x7f74301988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.733+0000 7f74371ff700 1 --2- 172.21.15.64:0/1167581509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74300ff4b0 0x7f743019ea60 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.733+0000 7f74371ff700 1 --2- 172.21.15.64:0/1167581509 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74300ffe70 0x7f743019efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.733+0000 7f74371ff700 1 -- 172.21.15.64:0/1167581509 >> 172.21.15.64:0/1167581509 conn(0x7f7430076600 msgr2=0x7f74300fd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.734+0000 7f74371ff700 1 -- 172.21.15.64:0/1167581509 shutdown_connections 2023-09-13T03:13:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:25.734+0000 7f74371ff700 1 -- 172.21.15.64:0/1167581509 wait complete. 2023-09-13T03:13:26.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:26 smithi170 ceph-mon[134328]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 45 op/s 2023-09-13T03:13:26.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:26 smithi064 ceph-mon[97675]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 45 op/s 2023-09-13T03:13:26.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:26 smithi142 ceph-mon[134082]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 45 op/s 2023-09-13T03:13:28.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:28 smithi170 ceph-mon[134328]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 50 op/s 2023-09-13T03:13:28.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:28 smithi064 ceph-mon[97675]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 50 op/s 2023-09-13T03:13:28.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:28 smithi142 ceph-mon[134082]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 50 op/s 2023-09-13T03:13:29.797 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-13T03:13:30.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.029+0000 7f744a3ff700 1 -- 172.21.15.64:0/3166941856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7444103140 msgr2=0x7f744410f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:30.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.029+0000 7f744a3ff700 1 --2- 172.21.15.64:0/3166941856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7444103140 0x7f744410f650 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f743c00b0a0 tx=0x7f743c00b3b0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.029+0000 7f744a3ff700 1 -- 172.21.15.64:0/3166941856 shutdown_connections 2023-09-13T03:13:30.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.029+0000 7f744a3ff700 1 --2- 172.21.15.64:0/3166941856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7444103140 0x7f744410f650 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.029+0000 7f744a3ff700 1 --2- 172.21.15.64:0/3166941856 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7444108790 0x7f7444108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.029+0000 7f744a3ff700 1 --2- 172.21.15.64:0/3166941856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7444102780 0x7f7444102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.029+0000 7f744a3ff700 1 -- 172.21.15.64:0/3166941856 >> 172.21.15.64:0/3166941856 conn(0x7f74440fe270 msgr2=0x7f7444100690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:30.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.030+0000 7f744a3ff700 1 -- 172.21.15.64:0/3166941856 shutdown_connections 2023-09-13T03:13:30.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.030+0000 7f744a3ff700 1 -- 172.21.15.64:0/3166941856 wait complete. 2023-09-13T03:13:30.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.031+0000 7f744a3ff700 1 Processor -- start 2023-09-13T03:13:30.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.031+0000 7f744a3ff700 1 -- start start 2023-09-13T03:13:30.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.032+0000 7f744a3ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7444102780 0x7f744419e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:30.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.032+0000 7f744a3ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7444103140 0x7f744419eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:30.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.032+0000 7f744a3ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7444108790 0x7f7444198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:30.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.032+0000 7f744a3ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7444198cc0 con 0x7f7444102780 2023-09-13T03:13:30.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.032+0000 7f744a3ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7444198e30 con 0x7f7444103140 2023-09-13T03:13:30.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.032+0000 7f744a3ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7444198ff0 con 0x7f7444108790 2023-09-13T03:13:30.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.032+0000 7f7443fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7444102780 0x7f744419e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:30.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.032+0000 7f7443fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7444102780 0x7f744419e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38830/0 (socket says 172.21.15.64:38830) 2023-09-13T03:13:30.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.032+0000 7f7443fff700 1 -- 172.21.15.64:0/4170809653 learned_addr learned my addr 172.21.15.64:0/4170809653 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:30.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.032+0000 7f744899c700 1 --2- 172.21.15.64:0/4170809653 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7444108790 0x7f7444198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:30.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.032+0000 7f74437fe700 1 --2- 172.21.15.64:0/4170809653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7444103140 0x7f744419eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:30.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.032+0000 7f7443fff700 1 -- 172.21.15.64:0/4170809653 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7444108790 msgr2=0x7f7444198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:30.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.032+0000 7f7443fff700 1 --2- 172.21.15.64:0/4170809653 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7444108790 0x7f7444198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.032+0000 7f7443fff700 1 -- 172.21.15.64:0/4170809653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7444103140 msgr2=0x7f744419eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:30.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.032+0000 7f7443fff700 1 --2- 172.21.15.64:0/4170809653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7444103140 0x7f744419eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.032+0000 7f7443fff700 1 -- 172.21.15.64:0/4170809653 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f743c009d00 con 0x7f7444102780 2023-09-13T03:13:30.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.032+0000 7f74437fe700 1 --2- 172.21.15.64:0/4170809653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7444103140 0x7f744419eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:13:30.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.033+0000 7f7443fff700 1 --2- 172.21.15.64:0/4170809653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7444102780 0x7f744419e5e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f743400e9a0 tx=0x7f743400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:30.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.033+0000 7f74417fa700 1 -- 172.21.15.64:0/4170809653 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7434009790 con 0x7f7444102780 2023-09-13T03:13:30.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.033+0000 7f744a3ff700 1 -- 172.21.15.64:0/4170809653 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74441992d0 con 0x7f7444102780 2023-09-13T03:13:30.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.033+0000 7f74417fa700 1 -- 172.21.15.64:0/4170809653 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f743400cbd0 con 0x7f7444102780 2023-09-13T03:13:30.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.033+0000 7f74417fa700 1 -- 172.21.15.64:0/4170809653 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7434018680 con 0x7f7444102780 2023-09-13T03:13:30.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.033+0000 7f744a3ff700 1 -- 172.21.15.64:0/4170809653 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74441a55d0 con 0x7f7444102780 2023-09-13T03:13:30.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.037+0000 7f74417fa700 1 -- 172.21.15.64:0/4170809653 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 24) v1 ==== 135435+0+0 (secure 0 0 0) 0x7f743400cd40 con 0x7f7444102780 2023-09-13T03:13:30.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.038+0000 7f745dcedb80 1 Processor -- start 2023-09-13T03:13:30.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.038+0000 7f745dcedb80 1 -- start start 2023-09-13T03:13:30.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.038+0000 7f74417fa700 1 --2- 172.21.15.64:0/4170809653 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f742c0a0840 0x7f742c0a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:30.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.038+0000 7f74417fa700 1 -- 172.21.15.64:0/4170809653 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f7434014070 con 0x7f7444102780 2023-09-13T03:13:30.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.038+0000 7f745dcedb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec8a4bc710 0x55ec8a4bcaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:30.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.038+0000 7f745dcedb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec8a4bd030 0x55ec8a4c6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:30.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.038+0000 7f745dcedb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec8a4c6e40 0x55ec8a4c9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:30.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.038+0000 7f745dcedb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec8a415730 con 0x55ec8a4bc710 2023-09-13T03:13:30.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.039+0000 7f745dcedb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec8a413050 con 0x55ec8a4c6e40 2023-09-13T03:13:30.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.039+0000 7f745dcedb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec8a40e940 con 0x55ec8a4bd030 2023-09-13T03:13:30.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.039+0000 7f744899c700 1 --2- 172.21.15.64:0/4170809653 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f742c0a0840 0x7f742c0a2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:30.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.039+0000 7f74437fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec8a4bd030 0x55ec8a4c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:30.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.039+0000 7f74437fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec8a4bd030 0x55ec8a4c6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:46730/0 (socket says 172.21.15.64:46730) 2023-09-13T03:13:30.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.039+0000 7f74437fe700 1 -- 172.21.15.64:0/2987666797 learned_addr learned my addr 172.21.15.64:0/2987666797 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:30.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.039+0000 7f744899c700 1 --2- 172.21.15.64:0/2987666797 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec8a4c6e40 0x55ec8a4c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:30.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.039+0000 7f7443fff700 1 --2- 172.21.15.64:0/2987666797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec8a4bc710 0x55ec8a4bcaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:30.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.039+0000 7f74437fe700 1 -- 172.21.15.64:0/2987666797 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec8a4c6e40 msgr2=0x55ec8a4c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:30.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.039+0000 7f74437fe700 1 --2- 172.21.15.64:0/2987666797 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec8a4c6e40 0x55ec8a4c9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.039+0000 7f74437fe700 1 -- 172.21.15.64:0/2987666797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec8a4bc710 msgr2=0x55ec8a4bcaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:30.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.039+0000 7f74437fe700 1 --2- 172.21.15.64:0/2987666797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec8a4bc710 0x55ec8a4bcaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.039+0000 7f74437fe700 1 -- 172.21.15.64:0/2987666797 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec8a23ade0 con 0x55ec8a4bd030 2023-09-13T03:13:30.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.039+0000 7f7443fff700 1 --2- 172.21.15.64:0/2987666797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec8a4bc710 0x55ec8a4bcaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T03:13:30.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.040+0000 7f74437fe700 1 --2- 172.21.15.64:0/2987666797 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec8a4bd030 0x55ec8a4c6900 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7438009d80 tx=0x7f74380116c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2f275d4ac6587da server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:30.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.040+0000 7f744899c700 1 --2- 172.21.15.64:0/4170809653 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f742c0a0840 0x7f742c0a2d00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f743c00b780 tx=0x7f743c00bb10 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:30.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.040+0000 7f742a7fc700 1 -- 172.21.15.64:0/2987666797 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7438011e20 con 0x55ec8a4bd030 2023-09-13T03:13:30.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.040+0000 7f742a7fc700 1 -- 172.21.15.64:0/2987666797 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7438012910 con 0x55ec8a4bd030 2023-09-13T03:13:30.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.040+0000 7f742a7fc700 1 -- 172.21.15.64:0/2987666797 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743800f7e0 con 0x55ec8a4bd030 2023-09-13T03:13:30.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.041+0000 7f745dcedb80 1 -- 172.21.15.64:0/2987666797 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec8a4bd030 msgr2=0x55ec8a4c6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:30.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.041+0000 7f745dcedb80 1 --2- 172.21.15.64:0/2987666797 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec8a4bd030 0x55ec8a4c6900 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7438009d80 tx=0x7f74380116c0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.041+0000 7f745dcedb80 1 -- 172.21.15.64:0/2987666797 shutdown_connections 2023-09-13T03:13:30.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.041+0000 7f745dcedb80 1 --2- 172.21.15.64:0/2987666797 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec8a4c6e40 0x55ec8a4c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.041+0000 7f745dcedb80 1 --2- 172.21.15.64:0/2987666797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec8a4bc710 0x55ec8a4bcaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.041+0000 7f745dcedb80 1 --2- 172.21.15.64:0/2987666797 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec8a4bd030 0x55ec8a4c6900 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.041+0000 7f745dcedb80 1 -- 172.21.15.64:0/2987666797 >> 172.21.15.64:0/2987666797 conn(0x55ec8a431090 msgr2=0x55ec8a4b7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:30.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.041+0000 7f745dcedb80 1 -- 172.21.15.64:0/2987666797 shutdown_connections 2023-09-13T03:13:30.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.041+0000 7f745dcedb80 1 -- 172.21.15.64:0/2987666797 wait complete. 2023-09-13T03:13:30.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.042+0000 7f745dcedb80 1 -- start start 2023-09-13T03:13:30.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.042+0000 7f745dcedb80 1 Processor -- start 2023-09-13T03:13:30.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.043+0000 7f745dcedb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec8a4bc710 0x55ec8a4c6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:30.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.043+0000 7f745dcedb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec8a4bd030 0x55ec8a4c39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:30.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.043+0000 7f745dcedb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec8a4c6e40 0x55ec8a4c3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:30.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.043+0000 7f745dcedb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec8a3acb70 con 0x55ec8a4c6e40 2023-09-13T03:13:30.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.043+0000 7f745dcedb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec8a3ac870 con 0x55ec8a4bd030 2023-09-13T03:13:30.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.043+0000 7f745dcedb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec8a290920 con 0x55ec8a4bc710 2023-09-13T03:13:30.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.043+0000 7f745dcedb80 10 client.0 _ll_register_callbacks cb 0x55ec8a2f6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:13:30.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.043+0000 7f744899c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec8a4c6e40 0x55ec8a4c3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:30.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.043+0000 7f744899c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec8a4c6e40 0x55ec8a4c3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38854/0 (socket says 172.21.15.64:38854) 2023-09-13T03:13:30.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.043+0000 7f744899c700 1 -- 172.21.15.64:0/3632090787 learned_addr learned my addr 172.21.15.64:0/3632090787 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:30.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.044+0000 7f74437fe700 1 --2- 172.21.15.64:0/3632090787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec8a4bd030 0x55ec8a4c39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:30.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.044+0000 7f7443fff700 1 --2- 172.21.15.64:0/3632090787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec8a4bc710 0x55ec8a4c6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:30.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.044+0000 7f744899c700 1 -- 172.21.15.64:0/3632090787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec8a4bc710 msgr2=0x55ec8a4c6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:30.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.044+0000 7f744899c700 1 --2- 172.21.15.64:0/3632090787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec8a4bc710 0x55ec8a4c6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.044+0000 7f744899c700 1 -- 172.21.15.64:0/3632090787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec8a4bd030 msgr2=0x55ec8a4c39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:30.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.044+0000 7f744899c700 1 --2- 172.21.15.64:0/3632090787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec8a4bd030 0x55ec8a4c39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.044+0000 7f744899c700 1 -- 172.21.15.64:0/3632090787 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74380099b0 con 0x55ec8a4c6e40 2023-09-13T03:13:30.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.044+0000 7f744899c700 1 --2- 172.21.15.64:0/3632090787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec8a4c6e40 0x55ec8a4c3f30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f743c00b0a0 tx=0x7f743c00b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a40d1be8211db9e server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:30.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.044+0000 7f74237fe700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:13:30.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.044+0000 7f74237fe700 1 -- 172.21.15.64:0/3632090787 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743c031070 con 0x55ec8a4c6e40 2023-09-13T03:13:30.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.045+0000 7f745dcedb80 1 -- 172.21.15.64:0/3632090787 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ec8a3664f0 con 0x55ec8a4c6e40 2023-09-13T03:13:30.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.045+0000 7f74237fe700 1 -- 172.21.15.64:0/3632090787 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f743c009490 con 0x55ec8a4c6e40 2023-09-13T03:13:30.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.045+0000 7f74237fe700 1 -- 172.21.15.64:0/3632090787 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f743c025070 con 0x55ec8a4c6e40 2023-09-13T03:13:30.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.045+0000 7f745dcedb80 10 client.15606 fetch_fsmap learned FSMap version 26 2023-09-13T03:13:30.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.045+0000 7f745dcedb80 1 -- 172.21.15.64:0/3632090787 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55ec8a223c60 con 0x55ec8a4c6e40 2023-09-13T03:13:30.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.045+0000 7f74237fe700 1 -- 172.21.15.64:0/3632090787 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ec8a3664f0 con 0x55ec8a4c6e40 2023-09-13T03:13:30.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.045+0000 7f74237fe700 1 -- 172.21.15.64:0/3632090787 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f743c034030 con 0x55ec8a4c6e40 2023-09-13T03:13:30.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.045+0000 7f745dcedb80 10 client.15606 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:13:30.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.045+0000 7f745dcedb80 10 client.15606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T03:13:30.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.045+0000 7f745dcedb80 20 client.15606 populate_metadata read hostname 'smithi064' 2023-09-13T03:13:30.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.045+0000 7f745dcedb80 1 --2- 172.21.15.64:0/3632090787 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55ec8a4d6790 0x55ec8a4d8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:30.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.045+0000 7f745dcedb80 4 client.15606 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:13:30.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.045+0000 7f745dcedb80 1 -- 172.21.15.64:0/3632090787 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ec8a4238a0 con 0x55ec8a4d6790 2023-09-13T03:13:30.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.046+0000 7f7443fff700 1 --2- 172.21.15.64:0/3632090787 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55ec8a4d6790 0x55ec8a4d8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:30.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.046+0000 7f74237fe700 10 client.15606 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:13:30.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.046+0000 7f7443fff700 1 --2- 172.21.15.64:0/3632090787 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55ec8a4d6790 0x55ec8a4d8bb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c3ba9ae71151f9e server_cookie=5dc0a6c22f07227d in_seq=0 out_seq=0 2023-09-13T03:13:30.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.047+0000 7f74237fe700 1 -- 172.21.15.64:0/3632090787 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ec8a4238a0 con 0x55ec8a4d6790 2023-09-13T03:13:30.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.047+0000 7f74237fe700 10 client.15606 handle_command_reply: tid=0 2023-09-13T03:13:30.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.047+0000 7f745dcedb80 1 client.15606 shutdown 2023-09-13T03:13:30.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.048+0000 7f745dcedb80 1 -- 172.21.15.64:0/3632090787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec8a4c6e40 msgr2=0x55ec8a4c3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:30.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.048+0000 7f745dcedb80 1 --2- 172.21.15.64:0/3632090787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec8a4c6e40 0x55ec8a4c3f30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f743c00b0a0 tx=0x7f743c00b3b0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.048+0000 7f745dcedb80 1 -- 172.21.15.64:0/3632090787 shutdown_connections 2023-09-13T03:13:30.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.048+0000 7f745dcedb80 1 --2- 172.21.15.64:0/3632090787 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55ec8a4d6790 0x55ec8a4d8bb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.048+0000 7f745dcedb80 1 --2- 172.21.15.64:0/3632090787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec8a4c6e40 0x55ec8a4c3f30 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.048+0000 7f745dcedb80 1 --2- 172.21.15.64:0/3632090787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec8a4bc710 0x55ec8a4c6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.048+0000 7f745dcedb80 1 --2- 172.21.15.64:0/3632090787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec8a4bd030 0x55ec8a4c39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.048+0000 7f745dcedb80 1 -- 172.21.15.64:0/3632090787 >> 172.21.15.64:0/3632090787 conn(0x55ec8a431090 msgr2=0x55ec8a4b7780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:30.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.048+0000 7f74237fe700 0 client.15606 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:13:30.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.048+0000 7f745dcedb80 1 -- 172.21.15.64:0/3632090787 shutdown_connections 2023-09-13T03:13:30.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.048+0000 7f745dcedb80 1 -- 172.21.15.64:0/3632090787 wait complete. 2023-09-13T03:13:30.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.048+0000 7f745dcedb80 20 client.15606 trim_cache size 0 max 16384 2023-09-13T03:13:30.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.068+0000 7f745dcedb80 1 Processor -- start 2023-09-13T03:13:30.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.068+0000 7f745dcedb80 1 -- start start 2023-09-13T03:13:30.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.068+0000 7f745dcedb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec8a4bc710 0x55ec8a4cda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:30.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.068+0000 7f745dcedb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec8a4bd030 0x55ec8a4d2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:30.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.068+0000 7f745dcedb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec8a4c6e40 0x55ec8a4beda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:30.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.068+0000 7f745dcedb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec8a4c9750 con 0x55ec8a4bd030 2023-09-13T03:13:30.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.068+0000 7f745dcedb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec8a4cdf70 con 0x55ec8a4bc710 2023-09-13T03:13:30.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.068+0000 7f745dcedb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec8a4c1f70 con 0x55ec8a4c6e40 2023-09-13T03:13:30.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.069+0000 7f744899c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec8a4c6e40 0x55ec8a4beda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:30.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.069+0000 7f74437fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec8a4bd030 0x55ec8a4d2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:30.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.069+0000 7f744899c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec8a4c6e40 0x55ec8a4beda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:46742/0 (socket says 172.21.15.64:46742) 2023-09-13T03:13:30.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.069+0000 7f744899c700 1 -- 172.21.15.64:0/2712404238 learned_addr learned my addr 172.21.15.64:0/2712404238 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:30.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.069+0000 7f7443fff700 1 --2- 172.21.15.64:0/2712404238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec8a4bc710 0x55ec8a4cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:30.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.069+0000 7f744899c700 1 -- 172.21.15.64:0/2712404238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec8a4bc710 msgr2=0x55ec8a4cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:30.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.069+0000 7f744899c700 1 --2- 172.21.15.64:0/2712404238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec8a4bc710 0x55ec8a4cda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.069+0000 7f744899c700 1 -- 172.21.15.64:0/2712404238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec8a4bd030 msgr2=0x55ec8a4d2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:30.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.069+0000 7f744899c700 1 --2- 172.21.15.64:0/2712404238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec8a4bd030 0x55ec8a4d2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.069+0000 7f744899c700 1 -- 172.21.15.64:0/2712404238 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec8a261250 con 0x55ec8a4c6e40 2023-09-13T03:13:30.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.069+0000 7f7443fff700 1 --2- 172.21.15.64:0/2712404238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec8a4bc710 0x55ec8a4cda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:13:30.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.069+0000 7f744899c700 1 --2- 172.21.15.64:0/2712404238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec8a4c6e40 0x55ec8a4beda0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f743c0062a0 tx=0x7f743c004a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e7f3d166d0b9388 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:30.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.070+0000 7f742affd700 1 -- 172.21.15.64:0/2712404238 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743c031070 con 0x55ec8a4c6e40 2023-09-13T03:13:30.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.070+0000 7f742affd700 1 -- 172.21.15.64:0/2712404238 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f743c0094f0 con 0x55ec8a4c6e40 2023-09-13T03:13:30.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.070+0000 7f742affd700 1 -- 172.21.15.64:0/2712404238 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743c00e040 con 0x55ec8a4c6e40 2023-09-13T03:13:30.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.070+0000 7f745dcedb80 1 -- 172.21.15.64:0/2712404238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec8a4c6e40 msgr2=0x55ec8a4beda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:30.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.070+0000 7f745dcedb80 1 --2- 172.21.15.64:0/2712404238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec8a4c6e40 0x55ec8a4beda0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f743c0062a0 tx=0x7f743c004a90 comp rx=0 tx=0).stop 2023-09-13T03:13:30.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.070+0000 7f745dcedb80 1 -- 172.21.15.64:0/2712404238 shutdown_connections 2023-09-13T03:13:30.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.070+0000 7f745dcedb80 1 --2- 172.21.15.64:0/2712404238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec8a4bc710 0x55ec8a4cda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.070+0000 7f745dcedb80 1 --2- 172.21.15.64:0/2712404238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec8a4c6e40 0x55ec8a4beda0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.070+0000 7f745dcedb80 1 --2- 172.21.15.64:0/2712404238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec8a4bd030 0x55ec8a4d2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.070+0000 7f745dcedb80 1 -- 172.21.15.64:0/2712404238 >> 172.21.15.64:0/2712404238 conn(0x55ec8a431090 msgr2=0x55ec8a4d87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:30.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.070+0000 7f745dcedb80 1 -- 172.21.15.64:0/2712404238 shutdown_connections 2023-09-13T03:13:30.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.070+0000 7f745dcedb80 1 -- 172.21.15.64:0/2712404238 wait complete. 2023-09-13T03:13:30.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.071+0000 7f745dcedb80 1 -- start start 2023-09-13T03:13:30.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.071+0000 7f745dcedb80 1 Processor -- start 2023-09-13T03:13:30.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.071+0000 7f745dcedb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec8a4bc710 0x55ec8a51e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:30.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.072+0000 7f745dcedb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec8a4bd030 0x55ec8a51ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:30.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.072+0000 7f745dcedb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec8a4c6e40 0x55ec8a523610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:30.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.072+0000 7f745dcedb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec8a523b50 con 0x55ec8a4c6e40 2023-09-13T03:13:30.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.072+0000 7f745dcedb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec8a523c90 con 0x55ec8a4bc710 2023-09-13T03:13:30.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.072+0000 7f745dcedb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec8a523e50 con 0x55ec8a4bd030 2023-09-13T03:13:30.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.072+0000 7f745dcedb80 10 client.0 _ll_register_callbacks cb 0x7f743c0077f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:13:30.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.072+0000 7f744899c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec8a4c6e40 0x55ec8a523610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:30.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.072+0000 7f7443fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec8a4bc710 0x55ec8a51e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:30.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.072+0000 7f74437fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec8a4bd030 0x55ec8a51ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:30.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.072+0000 7f744899c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec8a4c6e40 0x55ec8a523610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38876/0 (socket says 172.21.15.64:38876) 2023-09-13T03:13:30.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.072+0000 7f7443fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec8a4bc710 0x55ec8a51e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:57782/0 (socket says 172.21.15.64:57782) 2023-09-13T03:13:30.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.072+0000 7f744899c700 1 -- 172.21.15.64:0/1906807281 learned_addr learned my addr 172.21.15.64:0/1906807281 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:30.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.072+0000 7f7443fff700 1 -- 172.21.15.64:0/1906807281 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec8a4bd030 msgr2=0x55ec8a51ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:30.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.072+0000 7f7443fff700 1 --2- 172.21.15.64:0/1906807281 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec8a4bd030 0x55ec8a51ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.072+0000 7f7443fff700 1 -- 172.21.15.64:0/1906807281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec8a4c6e40 msgr2=0x55ec8a523610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:30.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.072+0000 7f7443fff700 1 --2- 172.21.15.64:0/1906807281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec8a4c6e40 0x55ec8a523610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.072+0000 7f7443fff700 1 -- 172.21.15.64:0/1906807281 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec8a223c60 con 0x55ec8a4bc710 2023-09-13T03:13:30.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.073+0000 7f7443fff700 1 --2- 172.21.15.64:0/1906807281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec8a4bc710 0x55ec8a51e740 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f74340c9a00 tx=0x7f74340953f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1d4f90a630426bb server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:30.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.073+0000 7f74237fe700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:13:30.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.073+0000 7f74237fe700 1 -- 172.21.15.64:0/1906807281 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74340996b0 con 0x55ec8a4bc710 2023-09-13T03:13:30.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.073+0000 7f745dcedb80 1 -- 172.21.15.64:0/1906807281 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7438012910 con 0x55ec8a4bc710 2023-09-13T03:13:30.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.073+0000 7f74237fe700 1 -- 172.21.15.64:0/1906807281 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f743409d4e0 con 0x55ec8a4bc710 2023-09-13T03:13:30.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.073+0000 7f74237fe700 1 -- 172.21.15.64:0/1906807281 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74340a1720 con 0x55ec8a4bc710 2023-09-13T03:13:30.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.073+0000 7f745dcedb80 10 client.25646 fetch_fsmap learned FSMap version 26 2023-09-13T03:13:30.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.073+0000 7f745dcedb80 1 -- 172.21.15.64:0/1906807281 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55ec8a2392b0 con 0x55ec8a4bc710 2023-09-13T03:13:30.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.073+0000 7f74237fe700 1 -- 172.21.15.64:0/1906807281 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7438012910 con 0x55ec8a4bc710 2023-09-13T03:13:30.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.074+0000 7f74237fe700 1 -- 172.21.15.64:0/1906807281 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7434098ca0 con 0x55ec8a4bc710 2023-09-13T03:13:30.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.074+0000 7f745dcedb80 10 client.25646 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:13:30.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.074+0000 7f745dcedb80 10 client.25646 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T03:13:30.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.074+0000 7f745dcedb80 20 client.25646 populate_metadata read hostname 'smithi064' 2023-09-13T03:13:30.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.074+0000 7f745dcedb80 1 --2- 172.21.15.64:0/1906807281 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55ec8a52b600 0x55ec8a52da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:30.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.074+0000 7f745dcedb80 4 client.25646 mds_command: new command op to 24686 tid=0[{"prefix": "scrub status"}] 2023-09-13T03:13:30.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.074+0000 7f745dcedb80 1 -- 172.21.15.64:0/1906807281 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ec8a407920 con 0x55ec8a52b600 2023-09-13T03:13:30.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.074+0000 7f74437fe700 1 --2- 172.21.15.64:0/1906807281 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55ec8a52b600 0x55ec8a52da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:30.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.074+0000 7f74437fe700 1 --2- 172.21.15.64:0/1906807281 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55ec8a52b600 0x55ec8a52da20 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae59497496689934 server_cookie=95a7e7a7ae16539e in_seq=0 out_seq=0 2023-09-13T03:13:30.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.074+0000 7f74237fe700 10 client.25646 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:13:30.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.075+0000 7f74237fe700 1 -- 172.21.15.64:0/1906807281 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ec8a407920 con 0x55ec8a52b600 2023-09-13T03:13:30.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.075+0000 7f74237fe700 10 client.25646 handle_command_reply: tid=0 2023-09-13T03:13:30.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.075+0000 7f745dcedb80 1 client.25646 shutdown 2023-09-13T03:13:30.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.075+0000 7f745dcedb80 1 -- 172.21.15.64:0/1906807281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec8a4bc710 msgr2=0x55ec8a51e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:30.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.075+0000 7f745dcedb80 1 --2- 172.21.15.64:0/1906807281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec8a4bc710 0x55ec8a51e740 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f74340c9a00 tx=0x7f74340953f0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.075+0000 7f745dcedb80 1 -- 172.21.15.64:0/1906807281 shutdown_connections 2023-09-13T03:13:30.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.075+0000 7f745dcedb80 1 --2- 172.21.15.64:0/1906807281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec8a4bc710 0x55ec8a51e740 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.075+0000 7f745dcedb80 1 --2- 172.21.15.64:0/1906807281 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55ec8a52b600 0x55ec8a52da20 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.075+0000 7f745dcedb80 1 --2- 172.21.15.64:0/1906807281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec8a4c6e40 0x55ec8a523610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.075+0000 7f745dcedb80 1 --2- 172.21.15.64:0/1906807281 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec8a4bd030 0x55ec8a51ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.075+0000 7f74237fe700 0 client.25646 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:13:30.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.075+0000 7f745dcedb80 1 -- 172.21.15.64:0/1906807281 >> 172.21.15.64:0/1906807281 conn(0x55ec8a431090 msgr2=0x55ec8a4d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:30.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.075+0000 7f745dcedb80 1 -- 172.21.15.64:0/1906807281 shutdown_connections 2023-09-13T03:13:30.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.075+0000 7f745dcedb80 1 -- 172.21.15.64:0/1906807281 wait complete. 2023-09-13T03:13:30.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.075+0000 7f745dcedb80 20 client.25646 trim_cache size 0 max 16384 2023-09-13T03:13:30.093 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-13T03:13:30.093 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2023-09-13T03:13:30.093 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2023-09-13T03:13:30.093 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-13T03:13:30.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.076+0000 7f744a3ff700 1 -- 172.21.15.64:0/4170809653 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f742c0a0840 msgr2=0x7f742c0a2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:30.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.076+0000 7f744a3ff700 1 --2- 172.21.15.64:0/4170809653 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f742c0a0840 0x7f742c0a2d00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f743c00b780 tx=0x7f743c00bb10 comp rx=0 tx=0).stop 2023-09-13T03:13:30.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.076+0000 7f744a3ff700 1 -- 172.21.15.64:0/4170809653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7444102780 msgr2=0x7f744419e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:30.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.076+0000 7f744a3ff700 1 --2- 172.21.15.64:0/4170809653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7444102780 0x7f744419e5e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f743400e9a0 tx=0x7f743400ecb0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.076+0000 7f744a3ff700 1 -- 172.21.15.64:0/4170809653 shutdown_connections 2023-09-13T03:13:30.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.076+0000 7f744a3ff700 1 --2- 172.21.15.64:0/4170809653 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f742c0a0840 0x7f742c0a2d00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.076+0000 7f744a3ff700 1 --2- 172.21.15.64:0/4170809653 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7444108790 0x7f7444198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.076+0000 7f744a3ff700 1 --2- 172.21.15.64:0/4170809653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7444102780 0x7f744419e5e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.076+0000 7f744a3ff700 1 --2- 172.21.15.64:0/4170809653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7444103140 0x7f744419eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.076+0000 7f744a3ff700 1 -- 172.21.15.64:0/4170809653 >> 172.21.15.64:0/4170809653 conn(0x7f74440fe270 msgr2=0x7f7444106290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:30.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.076+0000 7f744a3ff700 1 -- 172.21.15.64:0/4170809653 shutdown_connections 2023-09-13T03:13:30.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.076+0000 7f744a3ff700 1 -- 172.21.15.64:0/4170809653 wait complete. 2023-09-13T03:13:30.097 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-09-13T03:13:30.097 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T03:13:30.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.320+0000 7f208659d700 1 -- 172.21.15.64:0/2578521925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2080068380 msgr2=0x7f2080105010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:30.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.320+0000 7f208659d700 1 --2- 172.21.15.64:0/2578521925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2080068380 0x7f2080105010 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f20700098f0 tx=0x7f2070009c00 comp rx=0 tx=0).stop 2023-09-13T03:13:30.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.320+0000 7f208659d700 1 -- 172.21.15.64:0/2578521925 shutdown_connections 2023-09-13T03:13:30.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.320+0000 7f208659d700 1 --2- 172.21.15.64:0/2578521925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2080105740 0x7f208010d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.320+0000 7f208659d700 1 --2- 172.21.15.64:0/2578521925 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f20800679d0 0x7f2080067db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.320+0000 7f208659d700 1 --2- 172.21.15.64:0/2578521925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2080068380 0x7f2080105010 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.320+0000 7f208659d700 1 -- 172.21.15.64:0/2578521925 >> 172.21.15.64:0/2578521925 conn(0x7f2080074e00 msgr2=0x7f2080075210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:30.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.320+0000 7f208659d700 1 -- 172.21.15.64:0/2578521925 shutdown_connections 2023-09-13T03:13:30.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.321+0000 7f208659d700 1 -- 172.21.15.64:0/2578521925 wait complete. 2023-09-13T03:13:30.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.321+0000 7f208659d700 1 Processor -- start 2023-09-13T03:13:30.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.322+0000 7f208659d700 1 -- start start 2023-09-13T03:13:30.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.322+0000 7f208659d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20800679d0 0x7f2080196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:30.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.322+0000 7f208659d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2080105740 0x7f2080196970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:30.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.322+0000 7f208659d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f208019add0 0x7f208019b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:30.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.322+0000 7f208659d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f208019b850 con 0x7f208019add0 2023-09-13T03:13:30.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.322+0000 7f208659d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f208019b9c0 con 0x7f20800679d0 2023-09-13T03:13:30.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.322+0000 7f208659d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f208019bb80 con 0x7f2080105740 2023-09-13T03:13:30.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.322+0000 7f2085d9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f208019add0 0x7f208019b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:30.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.322+0000 7f208559b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20800679d0 0x7f2080196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:30.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.322+0000 7f2085d9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f208019add0 0x7f208019b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38898/0 (socket says 172.21.15.64:38898) 2023-09-13T03:13:30.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.322+0000 7f2085d9c700 1 -- 172.21.15.64:0/3468022372 learned_addr learned my addr 172.21.15.64:0/3468022372 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:30.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.322+0000 7f2084d9a700 1 --2- 172.21.15.64:0/3468022372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2080105740 0x7f2080196970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:30.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.323+0000 7f2085d9c700 1 -- 172.21.15.64:0/3468022372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2080105740 msgr2=0x7f2080196970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:30.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.323+0000 7f2085d9c700 1 --2- 172.21.15.64:0/3468022372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2080105740 0x7f2080196970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.323+0000 7f2085d9c700 1 -- 172.21.15.64:0/3468022372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20800679d0 msgr2=0x7f2080196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:30.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.323+0000 7f2085d9c700 1 --2- 172.21.15.64:0/3468022372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20800679d0 0x7f2080196430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.323+0000 7f2085d9c700 1 -- 172.21.15.64:0/3468022372 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2070009520 con 0x7f208019add0 2023-09-13T03:13:30.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.323+0000 7f2085d9c700 1 --2- 172.21.15.64:0/3468022372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f208019add0 0x7f208019b250 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f207800ef30 tx=0x7f207800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:30.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.323+0000 7f20767fc700 1 -- 172.21.15.64:0/3468022372 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2078018070 con 0x7f208019add0 2023-09-13T03:13:30.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.323+0000 7f208659d700 1 -- 172.21.15.64:0/3468022372 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20801a3080 con 0x7f208019add0 2023-09-13T03:13:30.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.323+0000 7f20767fc700 1 -- 172.21.15.64:0/3468022372 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f207800f040 con 0x7f208019add0 2023-09-13T03:13:30.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.323+0000 7f20767fc700 1 -- 172.21.15.64:0/3468022372 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207800ccf0 con 0x7f208019add0 2023-09-13T03:13:30.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.324+0000 7f208659d700 1 -- 172.21.15.64:0/3468022372 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20801a3560 con 0x7f208019add0 2023-09-13T03:13:30.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.325+0000 7f208659d700 1 -- 172.21.15.64:0/3468022372 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2080067310 con 0x7f208019add0 2023-09-13T03:13:30.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.326+0000 7f20767fc700 1 -- 172.21.15.64:0/3468022372 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 24) v1 ==== 135435+0+0 (secure 0 0 0) 0x7f2078003a00 con 0x7f208019add0 2023-09-13T03:13:30.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.328+0000 7f20767fc700 1 --2- 172.21.15.64:0/3468022372 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f206c0a0930 0x7f206c0a2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:30.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.328+0000 7f20767fc700 1 -- 172.21.15.64:0/3468022372 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f20780cbbb0 con 0x7f208019add0 2023-09-13T03:13:30.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.328+0000 7f208559b700 1 --2- 172.21.15.64:0/3468022372 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f206c0a0930 0x7f206c0a2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:30.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.329+0000 7f208559b700 1 --2- 172.21.15.64:0/3468022372 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f206c0a0930 0x7f206c0a2df0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f207c005fd0 tx=0x7f207c009310 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:30.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.333+0000 7f20767fc700 1 -- 172.21.15.64:0/3468022372 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f20780952b0 con 0x7f208019add0 2023-09-13T03:13:30.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:30 smithi064 ceph-mon[97675]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-13T03:13:30.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:30 smithi142 ceph-mon[134082]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-13T03:13:30.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.597+0000 7f208659d700 1 -- 172.21.15.64:0/3468022372 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2080197780 con 0x7f208019add0 2023-09-13T03:13:30.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.598+0000 7f20767fc700 1 -- 172.21.15.64:0/3468022372 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11366 (secure 0 0 0) 0x7f20780988d0 con 0x7f208019add0 2023-09-13T03:13:30.598 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:13:30.599 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2234182985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2234182985},{"type":"v1","addr":"172.21.15.64:6835","nonce":2234182985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4053841873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4053841873},{"type":"v1","addr":"172.21.15.64:6837","nonce":4053841873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-13T03:11:10.490260+0000","modified":"2023-09-13T03:11:18.230740+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24686,"mds_1":24649,"mds_2":24635,"mds_3":24634,"mds_4":24622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6839/99523374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":99523374},{"type":"v1","addr":"172.21.15.64:6839","nonce":99523374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14649":{"gid":14649,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6841/1866582875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":1866582875},{"type":"v1","addr":"172.21.15.64:6841","nonce":1866582875}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/101906999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":101906999},{"type":"v1","addr":"172.21.15.142:6833","nonce":101906999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24574":{"gid":24574,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/1897366227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1897366227},{"type":"v1","addr":"172.21.15.170:6833","nonce":1897366227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24611":{"gid":24611,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2167779918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2167779918},{"type":"v1","addr":"172.21.15.142:6835","nonce":2167779918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24622":{"gid":24622,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6835/1493371121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1493371121},{"type":"v1","addr":"172.21.15.170:6835","nonce":1493371121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24634":{"gid":24634,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.170:6837/1726587275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":1726587275},{"type":"v1","addr":"172.21.15.170:6837","nonce":1726587275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24635":{"gid":24635,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.142:6837/741173268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":741173268},{"type":"v1","addr":"172.21.15.142:6837","nonce":741173268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24649":{"gid":24649,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.170:6839/2189703596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2189703596},{"type":"v1","addr":"172.21.15.170:6839","nonce":2189703596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24686":{"gid":24686,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/3645998616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645998616},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645998616}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-13T03:13:30.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.601+0000 7f208659d700 1 -- 172.21.15.64:0/3468022372 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f206c0a0930 msgr2=0x7f206c0a2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:30.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.601+0000 7f208659d700 1 --2- 172.21.15.64:0/3468022372 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f206c0a0930 0x7f206c0a2df0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f207c005fd0 tx=0x7f207c009310 comp rx=0 tx=0).stop 2023-09-13T03:13:30.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.601+0000 7f208659d700 1 -- 172.21.15.64:0/3468022372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f208019add0 msgr2=0x7f208019b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:30.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.601+0000 7f208659d700 1 --2- 172.21.15.64:0/3468022372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f208019add0 0x7f208019b250 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f207800ef30 tx=0x7f207800c3a0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.601+0000 7f208659d700 1 -- 172.21.15.64:0/3468022372 shutdown_connections 2023-09-13T03:13:30.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.601+0000 7f208659d700 1 --2- 172.21.15.64:0/3468022372 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f206c0a0930 0x7f206c0a2df0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.601+0000 7f208659d700 1 --2- 172.21.15.64:0/3468022372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20800679d0 0x7f2080196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.601+0000 7f208659d700 1 --2- 172.21.15.64:0/3468022372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f208019add0 0x7f208019b250 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.601+0000 7f208659d700 1 --2- 172.21.15.64:0/3468022372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2080105740 0x7f2080196970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.601+0000 7f208659d700 1 -- 172.21.15.64:0/3468022372 >> 172.21.15.64:0/3468022372 conn(0x7f2080074e00 msgr2=0x7f20800fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:30.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.601+0000 7f208659d700 1 -- 172.21.15.64:0/3468022372 shutdown_connections 2023-09-13T03:13:30.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.601+0000 7f208659d700 1 -- 172.21.15.64:0/3468022372 wait complete. 2023-09-13T03:13:30.604 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 26 2023-09-13T03:13:30.632 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-09-13T03:13:30.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:30 smithi170 ceph-mon[134328]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-13T03:13:30.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.976+0000 7f6fcb1ff700 1 -- 172.21.15.64:0/4224410791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fc41017b0 msgr2=0x7f6fc4101b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:30.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.976+0000 7f6fcb1ff700 1 --2- 172.21.15.64:0/4224410791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fc41017b0 0x7f6fc4101b90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0009990 tx=0x7f6fc0009ca0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.977+0000 7f6fcb1ff700 1 -- 172.21.15.64:0/4224410791 shutdown_connections 2023-09-13T03:13:30.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.977+0000 7f6fcb1ff700 1 --2- 172.21.15.64:0/4224410791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fc41017b0 0x7f6fc4101b90 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.977+0000 7f6fcb1ff700 1 --2- 172.21.15.64:0/4224410791 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6fc410ab80 0x7f6fc410cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.977+0000 7f6fcb1ff700 1 --2- 172.21.15.64:0/4224410791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fc41020d0 0x7f6fc410a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.977+0000 7f6fcb1ff700 1 -- 172.21.15.64:0/4224410791 >> 172.21.15.64:0/4224410791 conn(0x7f6fc40faf50 msgr2=0x7f6fc40fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:30.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.977+0000 7f6fcb1ff700 1 -- 172.21.15.64:0/4224410791 shutdown_connections 2023-09-13T03:13:30.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.977+0000 7f6fcb1ff700 1 -- 172.21.15.64:0/4224410791 wait complete. 2023-09-13T03:13:30.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.978+0000 7f6fcb1ff700 1 Processor -- start 2023-09-13T03:13:30.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.979+0000 7f6fcb1ff700 1 -- start start 2023-09-13T03:13:30.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.979+0000 7f6fcb1ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fc41017b0 0x7f6fc406cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:30.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.979+0000 7f6fcb1ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fc41020d0 0x7f6fc406d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:30.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.979+0000 7f6fcb1ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6fc410ab80 0x7f6fc40717f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:30.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.979+0000 7f6fcb1ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc4071dc0 con 0x7f6fc41017b0 2023-09-13T03:13:30.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.979+0000 7f6fcb1ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc4071f30 con 0x7f6fc41020d0 2023-09-13T03:13:30.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.979+0000 7f6fcb1ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc40720f0 con 0x7f6fc410ab80 2023-09-13T03:13:30.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.980+0000 7f6fc8f9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fc41017b0 0x7f6fc406cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:30.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.980+0000 7f6fbbfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fc41020d0 0x7f6fc406d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:30.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.980+0000 7f6fbbfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fc41020d0 0x7f6fc406d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:57812/0 (socket says 172.21.15.64:57812) 2023-09-13T03:13:30.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.980+0000 7f6fbbfff700 1 -- 172.21.15.64:0/1563464778 learned_addr learned my addr 172.21.15.64:0/1563464778 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:30.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.980+0000 7f6fc979c700 1 --2- 172.21.15.64:0/1563464778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6fc410ab80 0x7f6fc40717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:30.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.980+0000 7f6fbbfff700 1 -- 172.21.15.64:0/1563464778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6fc410ab80 msgr2=0x7f6fc40717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:30.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.980+0000 7f6fbbfff700 1 --2- 172.21.15.64:0/1563464778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6fc410ab80 0x7f6fc40717f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.980+0000 7f6fbbfff700 1 -- 172.21.15.64:0/1563464778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fc41017b0 msgr2=0x7f6fc406cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:30.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.980+0000 7f6fbbfff700 1 --2- 172.21.15.64:0/1563464778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fc41017b0 0x7f6fc406cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.980+0000 7f6fbbfff700 1 -- 172.21.15.64:0/1563464778 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc00095f0 con 0x7f6fc41020d0 2023-09-13T03:13:30.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.981+0000 7f6fbbfff700 1 --2- 172.21.15.64:0/1563464778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fc41020d0 0x7f6fc406d320 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6fb400d950 tx=0x7f6fb400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:30.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.981+0000 7f6fb9ffb700 1 -- 172.21.15.64:0/1563464778 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb400f890 con 0x7f6fc41020d0 2023-09-13T03:13:30.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.981+0000 7f6fb9ffb700 1 -- 172.21.15.64:0/1563464778 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fb4010460 con 0x7f6fc41020d0 2023-09-13T03:13:30.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.981+0000 7f6fb9ffb700 1 -- 172.21.15.64:0/1563464778 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb400e610 con 0x7f6fc41020d0 2023-09-13T03:13:30.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.981+0000 7f6fcb1ff700 1 -- 172.21.15.64:0/1563464778 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fc40723d0 con 0x7f6fc41020d0 2023-09-13T03:13:30.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.981+0000 7f6fcb1ff700 1 -- 172.21.15.64:0/1563464778 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fc41adf60 con 0x7f6fc41020d0 2023-09-13T03:13:30.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.987+0000 7f6fb9ffb700 1 -- 172.21.15.64:0/1563464778 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) v1 ==== 135435+0+0 (secure 0 0 0) 0x7f6fb40108d0 con 0x7f6fc41020d0 2023-09-13T03:13:30.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.988+0000 7f6fdeb56b80 1 Processor -- start 2023-09-13T03:13:30.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.988+0000 7f6fdeb56b80 1 -- start start 2023-09-13T03:13:30.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.989+0000 7f6fdeb56b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e048f55f10 0x55e048f562f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:30.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.989+0000 7f6fdeb56b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e048f56830 0x55e048f61120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:30.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.989+0000 7f6fdeb56b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e048f61660 0x55e048f63a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:30.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.989+0000 7f6fdeb56b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e048ea7730 con 0x55e048f55f10 2023-09-13T03:13:30.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.989+0000 7f6fdeb56b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e048ea5050 con 0x55e048f61660 2023-09-13T03:13:30.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.989+0000 7f6fdeb56b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e048ea0940 con 0x55e048f56830 2023-09-13T03:13:30.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.989+0000 7f6fc8f9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e048f56830 0x55e048f61120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:30.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.990+0000 7f6fc8f9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e048f56830 0x55e048f61120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:46790/0 (socket says 172.21.15.64:46790) 2023-09-13T03:13:30.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.990+0000 7f6fc8f9b700 1 -- 172.21.15.64:0/2566997171 learned_addr learned my addr 172.21.15.64:0/2566997171 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:30.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.990+0000 7f6fc979c700 1 --2- 172.21.15.64:0/2566997171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e048f55f10 0x55e048f562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:30.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.990+0000 7f6fb9ffb700 1 --2- 172.21.15.64:0/1563464778 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f6fac0a08c0 0x7f6fac0a2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:30.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.990+0000 7f6fc8f9b700 1 -- 172.21.15.64:0/2566997171 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e048f61660 msgr2=0x55e048f63a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:30.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.990+0000 7f6fc8f9b700 1 --2- 172.21.15.64:0/2566997171 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e048f61660 0x55e048f63a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.990+0000 7f6fb9ffb700 1 -- 172.21.15.64:0/1563464778 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f6fb40cbe90 con 0x7f6fc41020d0 2023-09-13T03:13:30.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.990+0000 7f6fc8f9b700 1 -- 172.21.15.64:0/2566997171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e048f55f10 msgr2=0x55e048f562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:30.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.990+0000 7f6fc8f9b700 1 --2- 172.21.15.64:0/2566997171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e048f55f10 0x55e048f562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.990+0000 7f6fc8f9b700 1 -- 172.21.15.64:0/2566997171 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e048cccde0 con 0x55e048f56830 2023-09-13T03:13:30.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.990+0000 7f6fc8f9b700 1 --2- 172.21.15.64:0/2566997171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e048f56830 0x55e048f61120 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0015000 tx=0x7f6fc0009550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb4dac6ee3b37267 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:30.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.991+0000 7f6fc979c700 1 --2- 172.21.15.64:0/1563464778 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f6fac0a08c0 0x7f6fac0a2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:30.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.991+0000 7f6faaffd700 1 -- 172.21.15.64:0/2566997171 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc001d050 con 0x55e048f56830 2023-09-13T03:13:30.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.991+0000 7f6faaffd700 1 -- 172.21.15.64:0/2566997171 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fc001e040 con 0x55e048f56830 2023-09-13T03:13:30.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.991+0000 7f6faaffd700 1 -- 172.21.15.64:0/2566997171 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc00053d0 con 0x55e048f56830 2023-09-13T03:13:30.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.992+0000 7f6fdeb56b80 1 -- 172.21.15.64:0/2566997171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e048f56830 msgr2=0x55e048f61120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:30.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.992+0000 7f6fc979c700 1 --2- 172.21.15.64:0/1563464778 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f6fac0a08c0 0x7f6fac0a2d80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc009790 tx=0x7f6fbc004ad0 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:30.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.992+0000 7f6fdeb56b80 1 --2- 172.21.15.64:0/2566997171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e048f56830 0x55e048f61120 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0015000 tx=0x7f6fc0009550 comp rx=0 tx=0).stop 2023-09-13T03:13:30.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.992+0000 7f6fdeb56b80 1 -- 172.21.15.64:0/2566997171 shutdown_connections 2023-09-13T03:13:30.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.992+0000 7f6fdeb56b80 1 --2- 172.21.15.64:0/2566997171 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e048f61660 0x55e048f63a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.992+0000 7f6fdeb56b80 1 --2- 172.21.15.64:0/2566997171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e048f55f10 0x55e048f562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.992+0000 7f6fdeb56b80 1 --2- 172.21.15.64:0/2566997171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e048f56830 0x55e048f61120 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:30.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.992+0000 7f6fdeb56b80 1 -- 172.21.15.64:0/2566997171 >> 172.21.15.64:0/2566997171 conn(0x55e048ec3090 msgr2=0x55e048f49a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:30.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.992+0000 7f6fdeb56b80 1 -- 172.21.15.64:0/2566997171 shutdown_connections 2023-09-13T03:13:30.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.993+0000 7f6fdeb56b80 1 -- 172.21.15.64:0/2566997171 wait complete. 2023-09-13T03:13:31.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.993+0000 7f6fdeb56b80 1 -- start start 2023-09-13T03:13:31.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.994+0000 7f6fdeb56b80 1 Processor -- start 2023-09-13T03:13:31.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.994+0000 7f6fdeb56b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e048f55f10 0x55e048f5e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.994+0000 7f6fdeb56b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e048f56830 0x55e048f5e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.000 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.994+0000 7f6fdeb56b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e048f61660 0x55e048f5eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.995+0000 7f6fdeb56b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e048e3eb70 con 0x55e048f55f10 2023-09-13T03:13:31.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.995+0000 7f6fdeb56b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e048e3e870 con 0x55e048f61660 2023-09-13T03:13:31.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.995+0000 7f6fdeb56b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e048d22920 con 0x55e048f56830 2023-09-13T03:13:31.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.995+0000 7f6fdeb56b80 10 client.0 _ll_register_callbacks cb 0x55e048d88a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:13:31.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.995+0000 7f6fbbfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e048f56830 0x55e048f5e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:31.001 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.995+0000 7f6fbbfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e048f56830 0x55e048f5e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:46802/0 (socket says 172.21.15.64:46802) 2023-09-13T03:13:31.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.995+0000 7f6fbbfff700 1 -- 172.21.15.64:0/1956659504 learned_addr learned my addr 172.21.15.64:0/1956659504 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:31.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.995+0000 7f6fc979c700 1 --2- 172.21.15.64:0/1956659504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e048f61660 0x55e048f5eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:31.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.995+0000 7f6fbbfff700 1 -- 172.21.15.64:0/1956659504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e048f61660 msgr2=0x55e048f5eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:31.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.995+0000 7f6fbbfff700 1 --2- 172.21.15.64:0/1956659504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e048f61660 0x55e048f5eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.002 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.995+0000 7f6fbbfff700 1 -- 172.21.15.64:0/1956659504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e048f55f10 msgr2=0x55e048f5e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:31.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.995+0000 7f6fbbfff700 1 --2- 172.21.15.64:0/1956659504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e048f55f10 0x55e048f5e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.995+0000 7f6fbbfff700 1 -- 172.21.15.64:0/1956659504 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fc410dcb0 con 0x55e048f56830 2023-09-13T03:13:31.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.996+0000 7f6fbbfff700 1 --2- 172.21.15.64:0/1956659504 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e048f56830 0x55e048f5e9a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4098ec0 tx=0x7f6fb4099940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94686b77d52fdc76 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:31.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.996+0000 7f6fa3fff700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:13:31.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.996+0000 7f6fa3fff700 1 -- 172.21.15.64:0/1956659504 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb40a1230 con 0x55e048f56830 2023-09-13T03:13:31.003 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.996+0000 7f6fdeb56b80 1 -- 172.21.15.64:0/1956659504 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e048df84f0 con 0x55e048f56830 2023-09-13T03:13:31.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.996+0000 7f6fa3fff700 1 -- 172.21.15.64:0/1956659504 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f6fb4099630 con 0x55e048f56830 2023-09-13T03:13:31.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.996+0000 7f6fa3fff700 1 -- 172.21.15.64:0/1956659504 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6fb40986b0 con 0x55e048f56830 2023-09-13T03:13:31.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.996+0000 7f6fdeb56b80 10 client.25672 fetch_fsmap learned FSMap version 26 2023-09-13T03:13:31.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.996+0000 7f6fdeb56b80 1 -- 172.21.15.64:0/1956659504 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55e048cb5c60 con 0x55e048f56830 2023-09-13T03:13:31.004 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.997+0000 7f6fa3fff700 1 -- 172.21.15.64:0/1956659504 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fb40b5020 con 0x55e048f56830 2023-09-13T03:13:31.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.997+0000 7f6fa3fff700 1 -- 172.21.15.64:0/1956659504 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6fb40c9d20 con 0x55e048f56830 2023-09-13T03:13:31.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.997+0000 7f6fdeb56b80 10 client.25672 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:13:31.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.997+0000 7f6fdeb56b80 10 client.25672 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2023-09-13T03:13:31.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.997+0000 7f6fdeb56b80 20 client.25672 populate_metadata read hostname 'smithi064' 2023-09-13T03:13:31.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.997+0000 7f6fdeb56b80 1 --2- 172.21.15.64:0/1956659504 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x55e048f710a0 0x55e048f734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.997+0000 7f6fdeb56b80 4 client.25672 mds_command: new command op to 24622 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:13:31.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.997+0000 7f6fdeb56b80 1 -- 172.21.15.64:0/1956659504 --> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e048eb58a0 con 0x55e048f710a0 2023-09-13T03:13:31.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.997+0000 7f6fc8f9b700 1 --2- 172.21.15.64:0/1956659504 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x55e048f710a0 0x55e048f734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:31.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.998+0000 7f6fc8f9b700 1 --2- 172.21.15.64:0/1956659504 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x55e048f710a0 0x55e048f734c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a87078c7be53ba22 server_cookie=3a363c039463e285 in_seq=0 out_seq=0 2023-09-13T03:13:31.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.998+0000 7f6fa3fff700 10 client.25672 ms_handle_connect on v2:172.21.15.170:6834/1493371121 2023-09-13T03:13:31.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.999+0000 7f6fa3fff700 1 -- 172.21.15.64:0/1956659504 <== mds.4 v2:172.21.15.170:6834/1493371121 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e048eb58a0 con 0x55e048f710a0 2023-09-13T03:13:31.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.999+0000 7f6fa3fff700 10 client.25672 handle_command_reply: tid=0 2023-09-13T03:13:31.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.999+0000 7f6fdeb56b80 1 client.25672 shutdown 2023-09-13T03:13:31.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.999+0000 7f6fdeb56b80 1 -- 172.21.15.64:0/1956659504 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e048f56830 msgr2=0x55e048f5e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:31.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.999+0000 7f6fdeb56b80 1 --2- 172.21.15.64:0/1956659504 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e048f56830 0x55e048f5e9a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4098ec0 tx=0x7f6fb4099940 comp rx=0 tx=0).stop 2023-09-13T03:13:31.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.999+0000 7f6fdeb56b80 1 -- 172.21.15.64:0/1956659504 shutdown_connections 2023-09-13T03:13:31.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.999+0000 7f6fdeb56b80 1 --2- 172.21.15.64:0/1956659504 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x55e048f710a0 0x55e048f734c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:30.999+0000 7f6fdeb56b80 1 --2- 172.21.15.64:0/1956659504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e048f61660 0x55e048f5eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.000+0000 7f6fdeb56b80 1 --2- 172.21.15.64:0/1956659504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e048f55f10 0x55e048f5e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.000+0000 7f6fdeb56b80 1 --2- 172.21.15.64:0/1956659504 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e048f56830 0x55e048f5e9a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.000+0000 7f6fdeb56b80 1 -- 172.21.15.64:0/1956659504 >> 172.21.15.64:0/1956659504 conn(0x55e048ec3090 msgr2=0x55e048f496a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:31.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.000+0000 7f6fa3fff700 0 client.25672 ms_handle_reset on v2:172.21.15.170:6834/1493371121 2023-09-13T03:13:31.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.000+0000 7f6fdeb56b80 1 -- 172.21.15.64:0/1956659504 shutdown_connections 2023-09-13T03:13:31.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.000+0000 7f6fdeb56b80 1 -- 172.21.15.64:0/1956659504 wait complete. 2023-09-13T03:13:31.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.000+0000 7f6fdeb56b80 20 client.25672 trim_cache size 0 max 16384 2023-09-13T03:13:31.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.026+0000 7f6fdeb56b80 1 Processor -- start 2023-09-13T03:13:31.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.026+0000 7f6fdeb56b80 1 -- start start 2023-09-13T03:13:31.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.027+0000 7f6fdeb56b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e048f55f10 0x55e048f6c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.027+0000 7f6fdeb56b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e048f56830 0x55e048f59760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.027+0000 7f6fdeb56b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e048f61660 0x55e048f59ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.027+0000 7f6fdeb56b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e048f57110 con 0x55e048f56830 2023-09-13T03:13:31.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.027+0000 7f6fdeb56b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e048f4ae50 con 0x55e048f55f10 2023-09-13T03:13:31.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.027+0000 7f6fdeb56b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e048f43d70 con 0x55e048f61660 2023-09-13T03:13:31.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.027+0000 7f6fc8f9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e048f55f10 0x55e048f6c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:31.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.027+0000 7f6fc8f9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e048f55f10 0x55e048f6c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:57842/0 (socket says 172.21.15.64:57842) 2023-09-13T03:13:31.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.027+0000 7f6fc8f9b700 1 -- 172.21.15.64:0/1458308775 learned_addr learned my addr 172.21.15.64:0/1458308775 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:31.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.027+0000 7f6fc979c700 1 --2- 172.21.15.64:0/1458308775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e048f61660 0x55e048f59ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:31.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.028+0000 7f6fc8f9b700 1 -- 172.21.15.64:0/1458308775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e048f61660 msgr2=0x55e048f59ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:31.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.028+0000 7f6fc8f9b700 1 --2- 172.21.15.64:0/1458308775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e048f61660 0x55e048f59ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.028+0000 7f6fbbfff700 1 --2- 172.21.15.64:0/1458308775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e048f56830 0x55e048f59760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:31.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.028+0000 7f6fc8f9b700 1 -- 172.21.15.64:0/1458308775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e048f56830 msgr2=0x55e048f59760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:31.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.028+0000 7f6fc8f9b700 1 --2- 172.21.15.64:0/1458308775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e048f56830 0x55e048f59760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.028+0000 7f6fc8f9b700 1 -- 172.21.15.64:0/1458308775 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e048cf3250 con 0x55e048f55f10 2023-09-13T03:13:31.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.028+0000 7f6fbbfff700 1 --2- 172.21.15.64:0/1458308775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e048f56830 0x55e048f59760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T03:13:31.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.028+0000 7f6fc8f9b700 1 --2- 172.21.15.64:0/1458308775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e048f55f10 0x55e048f6c840 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0026040 tx=0x7f6fc0005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f13978e9a3803991 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:31.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.028+0000 7f6fab7fe700 1 -- 172.21.15.64:0/1458308775 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc001d040 con 0x55e048f55f10 2023-09-13T03:13:31.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.028+0000 7f6fab7fe700 1 -- 172.21.15.64:0/1458308775 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fc000b9d0 con 0x55e048f55f10 2023-09-13T03:13:31.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.029+0000 7f6fab7fe700 1 -- 172.21.15.64:0/1458308775 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc0030410 con 0x55e048f55f10 2023-09-13T03:13:31.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.029+0000 7f6fdeb56b80 1 -- 172.21.15.64:0/1458308775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e048f55f10 msgr2=0x55e048f6c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:31.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.029+0000 7f6fdeb56b80 1 --2- 172.21.15.64:0/1458308775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e048f55f10 0x55e048f6c840 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0026040 tx=0x7f6fc0005fd0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.030+0000 7f6fdeb56b80 1 -- 172.21.15.64:0/1458308775 shutdown_connections 2023-09-13T03:13:31.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.030+0000 7f6fdeb56b80 1 --2- 172.21.15.64:0/1458308775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e048f55f10 0x55e048f6c840 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.030+0000 7f6fdeb56b80 1 --2- 172.21.15.64:0/1458308775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e048f61660 0x55e048f59ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.030+0000 7f6fdeb56b80 1 --2- 172.21.15.64:0/1458308775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e048f56830 0x55e048f59760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.030+0000 7f6fdeb56b80 1 -- 172.21.15.64:0/1458308775 >> 172.21.15.64:0/1458308775 conn(0x55e048ec3090 msgr2=0x55e048f6d450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:31.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.030+0000 7f6fdeb56b80 1 -- 172.21.15.64:0/1458308775 shutdown_connections 2023-09-13T03:13:31.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.030+0000 7f6fdeb56b80 1 -- 172.21.15.64:0/1458308775 wait complete. 2023-09-13T03:13:31.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.030+0000 7f6fdeb56b80 1 -- start start 2023-09-13T03:13:31.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.031+0000 7f6fdeb56b80 1 Processor -- start 2023-09-13T03:13:31.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.031+0000 7f6fdeb56b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e048f55f10 0x55e048fbfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.031+0000 7f6fdeb56b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e048f56830 0x55e048fc0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.031+0000 7f6fdeb56b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e048f61660 0x55e048f672a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.031+0000 7f6fdeb56b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e048fc0be0 con 0x55e048f61660 2023-09-13T03:13:31.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.031+0000 7f6fdeb56b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e048fc0d50 con 0x55e048f55f10 2023-09-13T03:13:31.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.031+0000 7f6fdeb56b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e048f67830 con 0x55e048f56830 2023-09-13T03:13:31.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.031+0000 7f6fdeb56b80 10 client.0 _ll_register_callbacks cb 0x7f6fb4098d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:13:31.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.032+0000 7f6fc8f9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e048f55f10 0x55e048fbfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:31.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.032+0000 7f6fc8f9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e048f55f10 0x55e048fbfed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:57846/0 (socket says 172.21.15.64:57846) 2023-09-13T03:13:31.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.032+0000 7f6fc8f9b700 1 -- 172.21.15.64:0/2478349129 learned_addr learned my addr 172.21.15.64:0/2478349129 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:31.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.032+0000 7f6fc979c700 1 --2- 172.21.15.64:0/2478349129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e048f61660 0x55e048f672a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:31.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.033+0000 7f6fc8f9b700 1 -- 172.21.15.64:0/2478349129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e048f56830 msgr2=0x55e048fc0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:31.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.033+0000 7f6fc8f9b700 1 --2- 172.21.15.64:0/2478349129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e048f56830 0x55e048fc0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.033+0000 7f6fc8f9b700 1 -- 172.21.15.64:0/2478349129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e048f61660 msgr2=0x55e048f672a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:31.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.033+0000 7f6fc8f9b700 1 --2- 172.21.15.64:0/2478349129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e048f61660 0x55e048f672a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.033+0000 7f6fc8f9b700 1 -- 172.21.15.64:0/2478349129 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e048cccde0 con 0x55e048f55f10 2023-09-13T03:13:31.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.033+0000 7f6fc8f9b700 1 --2- 172.21.15.64:0/2478349129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e048f55f10 0x55e048fbfed0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0030040 tx=0x7f6fc0009290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7d3a50370a24ddb server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:31.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.033+0000 7f6fa3fff700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:13:31.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.033+0000 7f6fa3fff700 1 -- 172.21.15.64:0/2478349129 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc001d040 con 0x55e048f55f10 2023-09-13T03:13:31.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.034+0000 7f6fdeb56b80 1 -- 172.21.15.64:0/2478349129 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6fc001e040 con 0x55e048f55f10 2023-09-13T03:13:31.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.034+0000 7f6fa3fff700 1 -- 172.21.15.64:0/2478349129 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f6fc00039d0 con 0x55e048f55f10 2023-09-13T03:13:31.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.034+0000 7f6fa3fff700 1 -- 172.21.15.64:0/2478349129 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6fc0003bb0 con 0x55e048f55f10 2023-09-13T03:13:31.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.034+0000 7f6fdeb56b80 10 client.25667 fetch_fsmap learned FSMap version 26 2023-09-13T03:13:31.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.034+0000 7f6fdeb56b80 1 -- 172.21.15.64:0/2478349129 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55e048ccb2b0 con 0x55e048f55f10 2023-09-13T03:13:31.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.035+0000 7f6fa3fff700 1 -- 172.21.15.64:0/2478349129 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fc001e040 con 0x55e048f55f10 2023-09-13T03:13:31.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.035+0000 7f6fa3fff700 1 -- 172.21.15.64:0/2478349129 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6fc0005380 con 0x55e048f55f10 2023-09-13T03:13:31.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.035+0000 7f6fdeb56b80 10 client.25667 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:13:31.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.035+0000 7f6fdeb56b80 10 client.25667 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2023-09-13T03:13:31.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.035+0000 7f6fdeb56b80 20 client.25667 populate_metadata read hostname 'smithi064' 2023-09-13T03:13:31.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.035+0000 7f6fdeb56b80 1 --2- 172.21.15.64:0/2478349129 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x55e048f4fe50 0x55e048f52270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.035+0000 7f6fdeb56b80 4 client.25667 mds_command: new command op to 24622 tid=0[{"prefix": "damage ls"}] 2023-09-13T03:13:31.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.035+0000 7f6fdeb56b80 1 -- 172.21.15.64:0/2478349129 --> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e048e99920 con 0x55e048f4fe50 2023-09-13T03:13:31.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.035+0000 7f6fbbfff700 1 --2- 172.21.15.64:0/2478349129 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x55e048f4fe50 0x55e048f52270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:31.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.036+0000 7f6fbbfff700 1 --2- 172.21.15.64:0/2478349129 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x55e048f4fe50 0x55e048f52270 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a25dbfe4e84dd7b server_cookie=4b7f9af71f308470 in_seq=0 out_seq=0 2023-09-13T03:13:31.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.036+0000 7f6fa3fff700 10 client.25667 ms_handle_connect on v2:172.21.15.170:6834/1493371121 2023-09-13T03:13:31.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.036+0000 7f6fa3fff700 1 -- 172.21.15.64:0/2478349129 <== mds.4 v2:172.21.15.170:6834/1493371121 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e048e99920 con 0x55e048f4fe50 2023-09-13T03:13:31.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.036+0000 7f6fa3fff700 10 client.25667 handle_command_reply: tid=0 2023-09-13T03:13:31.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.036+0000 7f6fdeb56b80 1 client.25667 shutdown 2023-09-13T03:13:31.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.036+0000 7f6fdeb56b80 1 -- 172.21.15.64:0/2478349129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e048f55f10 msgr2=0x55e048fbfed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:31.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.036+0000 7f6fdeb56b80 1 --2- 172.21.15.64:0/2478349129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e048f55f10 0x55e048fbfed0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0030040 tx=0x7f6fc0009290 comp rx=0 tx=0).stop 2023-09-13T03:13:31.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.037+0000 7f6fdeb56b80 1 -- 172.21.15.64:0/2478349129 shutdown_connections 2023-09-13T03:13:31.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.037+0000 7f6fdeb56b80 1 --2- 172.21.15.64:0/2478349129 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x55e048f4fe50 0x55e048f52270 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.037+0000 7f6fdeb56b80 1 --2- 172.21.15.64:0/2478349129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e048f55f10 0x55e048fbfed0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.037+0000 7f6fdeb56b80 1 --2- 172.21.15.64:0/2478349129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e048f61660 0x55e048f672a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.037+0000 7f6fdeb56b80 1 --2- 172.21.15.64:0/2478349129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e048f56830 0x55e048fc0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.037+0000 7f6fdeb56b80 1 -- 172.21.15.64:0/2478349129 >> 172.21.15.64:0/2478349129 conn(0x55e048ec3090 msgr2=0x55e048f6c840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:31.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.037+0000 7f6fa3fff700 0 client.25667 ms_handle_reset on v2:172.21.15.170:6834/1493371121 2023-09-13T03:13:31.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.037+0000 7f6fdeb56b80 1 -- 172.21.15.64:0/2478349129 shutdown_connections 2023-09-13T03:13:31.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.037+0000 7f6fdeb56b80 1 -- 172.21.15.64:0/2478349129 wait complete. 2023-09-13T03:13:31.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.037+0000 7f6fdeb56b80 20 client.25667 trim_cache size 0 max 16384 2023-09-13T03:13:31.048 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2023-09-13T03:13:31.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.038+0000 7f6fcb1ff700 1 -- 172.21.15.64:0/1563464778 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f6fac0a08c0 msgr2=0x7f6fac0a2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:31.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.038+0000 7f6fcb1ff700 1 --2- 172.21.15.64:0/1563464778 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f6fac0a08c0 0x7f6fac0a2d80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc009790 tx=0x7f6fbc004ad0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.038+0000 7f6fcb1ff700 1 -- 172.21.15.64:0/1563464778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fc41020d0 msgr2=0x7f6fc406d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:31.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.038+0000 7f6fcb1ff700 1 --2- 172.21.15.64:0/1563464778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fc41020d0 0x7f6fc406d320 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6fb400d950 tx=0x7f6fb400dc60 comp rx=0 tx=0).stop 2023-09-13T03:13:31.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.038+0000 7f6fcb1ff700 1 -- 172.21.15.64:0/1563464778 shutdown_connections 2023-09-13T03:13:31.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.038+0000 7f6fcb1ff700 1 --2- 172.21.15.64:0/1563464778 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f6fac0a08c0 0x7f6fac0a2d80 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.038+0000 7f6fcb1ff700 1 --2- 172.21.15.64:0/1563464778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6fc410ab80 0x7f6fc40717f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.038+0000 7f6fcb1ff700 1 --2- 172.21.15.64:0/1563464778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fc41017b0 0x7f6fc406cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.038+0000 7f6fcb1ff700 1 --2- 172.21.15.64:0/1563464778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fc41020d0 0x7f6fc406d320 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.038+0000 7f6fcb1ff700 1 -- 172.21.15.64:0/1563464778 >> 172.21.15.64:0/1563464778 conn(0x7f6fc40faf50 msgr2=0x7f6fc410cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:31.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.038+0000 7f6fcb1ff700 1 -- 172.21.15.64:0/1563464778 shutdown_connections 2023-09-13T03:13:31.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.039+0000 7f6fcb1ff700 1 -- 172.21.15.64:0/1563464778 wait complete. 2023-09-13T03:13:31.067 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-09-13T03:13:31.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.386+0000 7f12c99ff700 1 -- 172.21.15.64:0/3065624998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12c40ff440 msgr2=0x7f12c40ff8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:31.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.386+0000 7f12c99ff700 1 --2- 172.21.15.64:0/3065624998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12c40ff440 0x7f12c40ff8c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f12b8009990 tx=0x7f12b8009ca0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.386+0000 7f12c99ff700 1 -- 172.21.15.64:0/3065624998 shutdown_connections 2023-09-13T03:13:31.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.386+0000 7f12c99ff700 1 --2- 172.21.15.64:0/3065624998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12c40ffe00 0x7f12c410c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.386+0000 7f12c99ff700 1 --2- 172.21.15.64:0/3065624998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12c41054e0 0x7f12c41058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.386+0000 7f12c99ff700 1 --2- 172.21.15.64:0/3065624998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12c40ff440 0x7f12c40ff8c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.386+0000 7f12c99ff700 1 -- 172.21.15.64:0/3065624998 >> 172.21.15.64:0/3065624998 conn(0x7f12c40faf50 msgr2=0x7f12c40fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:31.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.386+0000 7f12c99ff700 1 -- 172.21.15.64:0/3065624998 shutdown_connections 2023-09-13T03:13:31.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.387+0000 7f12c99ff700 1 -- 172.21.15.64:0/3065624998 wait complete. 2023-09-13T03:13:31.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.388+0000 7f12c99ff700 1 Processor -- start 2023-09-13T03:13:31.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.388+0000 7f12c99ff700 1 -- start start 2023-09-13T03:13:31.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.389+0000 7f12c99ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12c40ff440 0x7f12c419c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.389+0000 7f12c99ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12c40ffe00 0x7f12c419c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.389+0000 7f12c99ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12c41054e0 0x7f12c41964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.389+0000 7f12c99ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c4196b40 con 0x7f12c40ff440 2023-09-13T03:13:31.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.389+0000 7f12c99ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c4196cb0 con 0x7f12c40ffe00 2023-09-13T03:13:31.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.389+0000 7f12c99ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c4196e70 con 0x7f12c41054e0 2023-09-13T03:13:31.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.389+0000 7f12c37fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12c41054e0 0x7f12c41964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:31.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.389+0000 7f12c37fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12c41054e0 0x7f12c41964e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:46832/0 (socket says 172.21.15.64:46832) 2023-09-13T03:13:31.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.389+0000 7f12c37fe700 1 -- 172.21.15.64:0/2718085479 learned_addr learned my addr 172.21.15.64:0/2718085479 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:31.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.389+0000 7f12c27fc700 1 --2- 172.21.15.64:0/2718085479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12c40ffe00 0x7f12c419c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:31.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.389+0000 7f12c2ffd700 1 --2- 172.21.15.64:0/2718085479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12c40ff440 0x7f12c419c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:31.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.390+0000 7f12c37fe700 1 -- 172.21.15.64:0/2718085479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12c40ffe00 msgr2=0x7f12c419c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:31.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.390+0000 7f12c37fe700 1 --2- 172.21.15.64:0/2718085479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12c40ffe00 0x7f12c419c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.390+0000 7f12c37fe700 1 -- 172.21.15.64:0/2718085479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12c40ff440 msgr2=0x7f12c419c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:31.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.390+0000 7f12c37fe700 1 --2- 172.21.15.64:0/2718085479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12c40ff440 0x7f12c419c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.390+0000 7f12c37fe700 1 -- 172.21.15.64:0/2718085479 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b80095f0 con 0x7f12c41054e0 2023-09-13T03:13:31.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.390+0000 7f12c2ffd700 1 --2- 172.21.15.64:0/2718085479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12c40ff440 0x7f12c419c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:13:31.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.390+0000 7f12c27fc700 1 --2- 172.21.15.64:0/2718085479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12c40ffe00 0x7f12c419c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:13:31.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.390+0000 7f12c37fe700 1 --2- 172.21.15.64:0/2718085479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12c41054e0 0x7f12c41964e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f12bc00ef30 tx=0x7f12bc00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:31.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.390+0000 7f12c89fd700 1 -- 172.21.15.64:0/2718085479 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc018070 con 0x7f12c41054e0 2023-09-13T03:13:31.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.391+0000 7f12c99ff700 1 -- 172.21.15.64:0/2718085479 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12c41970f0 con 0x7f12c41054e0 2023-09-13T03:13:31.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.391+0000 7f12c99ff700 1 -- 172.21.15.64:0/2718085479 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12c410ccb0 con 0x7f12c41054e0 2023-09-13T03:13:31.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.391+0000 7f12c89fd700 1 -- 172.21.15.64:0/2718085479 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12bc00f040 con 0x7f12c41054e0 2023-09-13T03:13:31.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.391+0000 7f12c89fd700 1 -- 172.21.15.64:0/2718085479 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc00cdb0 con 0x7f12c41054e0 2023-09-13T03:13:31.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.396+0000 7f12c89fd700 1 -- 172.21.15.64:0/2718085479 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 24) v1 ==== 135435+0+0 (secure 0 0 0) 0x7f12bc004ad0 con 0x7f12c41054e0 2023-09-13T03:13:31.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.397+0000 7f12dd271b80 1 Processor -- start 2023-09-13T03:13:31.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.397+0000 7f12dd271b80 1 -- start start 2023-09-13T03:13:31.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.398+0000 7f12dd271b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fc410a3710 0x55fc410a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.398+0000 7f12dd271b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fc410a4030 0x55fc410ad900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.398+0000 7f12dd271b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc410ade40 0x55fc410b0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.398+0000 7f12dd271b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc40ffc730 con 0x55fc410a4030 2023-09-13T03:13:31.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.398+0000 7f12dd271b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc40ffa050 con 0x55fc410ade40 2023-09-13T03:13:31.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.398+0000 7f12dd271b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc40ff5940 con 0x55fc410a3710 2023-09-13T03:13:31.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.398+0000 7f12c2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fc410a4030 0x55fc410ad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:31.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.398+0000 7f12c2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fc410a4030 0x55fc410ad900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38980/0 (socket says 172.21.15.64:38980) 2023-09-13T03:13:31.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.398+0000 7f12c2ffd700 1 -- 172.21.15.64:0/2445678237 learned_addr learned my addr 172.21.15.64:0/2445678237 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:31.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.399+0000 7f12c2ffd700 1 -- 172.21.15.64:0/2445678237 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fc410a3710 msgr2=0x55fc410a3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:31.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.399+0000 7f12c2ffd700 1 --2- 172.21.15.64:0/2445678237 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fc410a3710 0x55fc410a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.399+0000 7f12c2ffd700 1 -- 172.21.15.64:0/2445678237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc410ade40 msgr2=0x55fc410b0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:31.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.399+0000 7f12c2ffd700 1 --2- 172.21.15.64:0/2445678237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc410ade40 0x55fc410b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.399+0000 7f12c2ffd700 1 -- 172.21.15.64:0/2445678237 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc40e21de0 con 0x55fc410a4030 2023-09-13T03:13:31.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.399+0000 7f12c2ffd700 1 --2- 172.21.15.64:0/2445678237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fc410a4030 0x55fc410ad900 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f12b4009dc0 tx=0x7f12b40106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a5818e47235e54a server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:31.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.400+0000 7f12c89fd700 1 --2- 172.21.15.64:0/2718085479 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f12ac0a08c0 0x7f12ac0a2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.400+0000 7f12a9ffb700 1 -- 172.21.15.64:0/2445678237 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b4010ec0 con 0x55fc410a4030 2023-09-13T03:13:31.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.400+0000 7f12a9ffb700 1 -- 172.21.15.64:0/2445678237 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12b4015070 con 0x55fc410a4030 2023-09-13T03:13:31.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.400+0000 7f12a9ffb700 1 -- 172.21.15.64:0/2445678237 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b4011800 con 0x55fc410a4030 2023-09-13T03:13:31.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.400+0000 7f12c89fd700 1 -- 172.21.15.64:0/2718085479 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f12bc0cbc10 con 0x7f12c41054e0 2023-09-13T03:13:31.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.400+0000 7f12c27fc700 1 --2- 172.21.15.64:0/2718085479 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f12ac0a08c0 0x7f12ac0a2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:31.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.401+0000 7f12dd271b80 1 -- 172.21.15.64:0/2445678237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fc410a4030 msgr2=0x55fc410ad900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:31.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.401+0000 7f12dd271b80 1 --2- 172.21.15.64:0/2445678237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fc410a4030 0x55fc410ad900 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f12b4009dc0 tx=0x7f12b40106b0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.401+0000 7f12dd271b80 1 -- 172.21.15.64:0/2445678237 shutdown_connections 2023-09-13T03:13:31.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.401+0000 7f12dd271b80 1 --2- 172.21.15.64:0/2445678237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc410ade40 0x55fc410b0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.401+0000 7f12dd271b80 1 --2- 172.21.15.64:0/2445678237 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fc410a3710 0x55fc410a3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.401+0000 7f12dd271b80 1 --2- 172.21.15.64:0/2445678237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fc410a4030 0x55fc410ad900 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.401+0000 7f12dd271b80 1 -- 172.21.15.64:0/2445678237 >> 172.21.15.64:0/2445678237 conn(0x55fc41018090 msgr2=0x55fc4109ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:31.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.402+0000 7f12dd271b80 1 -- 172.21.15.64:0/2445678237 shutdown_connections 2023-09-13T03:13:31.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.402+0000 7f12dd271b80 1 -- 172.21.15.64:0/2445678237 wait complete. 2023-09-13T03:13:31.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.402+0000 7f12c27fc700 1 --2- 172.21.15.64:0/2718085479 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f12ac0a08c0 0x7f12ac0a2d80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f12b8005950 tx=0x7f12b80058e0 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:31.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.402+0000 7f12dd271b80 1 -- start start 2023-09-13T03:13:31.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.403+0000 7f12dd271b80 1 Processor -- start 2023-09-13T03:13:31.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.403+0000 7f12dd271b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fc410a3710 0x55fc410ad780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.404+0000 7f12dd271b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc410a4030 0x55fc410aaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.404+0000 7f12dd271b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fc410ade40 0x55fc410aaf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.404+0000 7f12dd271b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc40f93b70 con 0x55fc410ade40 2023-09-13T03:13:31.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.404+0000 7f12dd271b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc40f93870 con 0x55fc410a4030 2023-09-13T03:13:31.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.404+0000 7f12dd271b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc40e77920 con 0x55fc410a3710 2023-09-13T03:13:31.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.404+0000 7f12dd271b80 10 client.0 _ll_register_callbacks cb 0x55fc40edda80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:13:31.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.404+0000 7f12c2ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fc410a3710 0x55fc410ad780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:31.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.404+0000 7f12c37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fc410ade40 0x55fc410aaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:31.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.404+0000 7f12c2ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fc410a3710 0x55fc410ad780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:46848/0 (socket says 172.21.15.64:46848) 2023-09-13T03:13:31.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.404+0000 7f12c2ffd700 1 -- 172.21.15.64:0/421301507 learned_addr learned my addr 172.21.15.64:0/421301507 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:31.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.404+0000 7f12c27fc700 1 --2- 172.21.15.64:0/421301507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc410a4030 0x55fc410aaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:31.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.405+0000 7f12c27fc700 1 -- 172.21.15.64:0/421301507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fc410a3710 msgr2=0x55fc410ad780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:31.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.405+0000 7f12c27fc700 1 --2- 172.21.15.64:0/421301507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fc410a3710 0x55fc410ad780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.405+0000 7f12c27fc700 1 -- 172.21.15.64:0/421301507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fc410ade40 msgr2=0x55fc410aaf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:31.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.405+0000 7f12c27fc700 1 --2- 172.21.15.64:0/421301507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fc410ade40 0x55fc410aaf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.405+0000 7f12c27fc700 1 -- 172.21.15.64:0/421301507 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12b40099f0 con 0x55fc410a4030 2023-09-13T03:13:31.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.405+0000 7f12c27fc700 1 --2- 172.21.15.64:0/421301507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc410a4030 0x55fc410aaa00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f12b800b2e0 tx=0x7f12b8005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1406edb3d5a4283a server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:31.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.405+0000 7f12a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:13:31.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.405+0000 7f12a2ffd700 1 -- 172.21.15.64:0/421301507 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b802b070 con 0x55fc410a4030 2023-09-13T03:13:31.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.405+0000 7f12a2ffd700 1 -- 172.21.15.64:0/421301507 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f12b8004dd0 con 0x55fc410a4030 2023-09-13T03:13:31.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.405+0000 7f12dd271b80 1 -- 172.21.15.64:0/421301507 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fc40f4d4f0 con 0x55fc410a4030 2023-09-13T03:13:31.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.406+0000 7f12a2ffd700 1 -- 172.21.15.64:0/421301507 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12b800f460 con 0x55fc410a4030 2023-09-13T03:13:31.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.406+0000 7f12dd271b80 10 client.25676 fetch_fsmap learned FSMap version 26 2023-09-13T03:13:31.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.406+0000 7f12dd271b80 1 -- 172.21.15.64:0/421301507 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55fc40e0ac60 con 0x55fc410a4030 2023-09-13T03:13:31.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.406+0000 7f12a2ffd700 1 -- 172.21.15.64:0/421301507 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fc40f4d4f0 con 0x55fc410a4030 2023-09-13T03:13:31.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.406+0000 7f12a2ffd700 1 -- 172.21.15.64:0/421301507 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f12b803a6e0 con 0x55fc410a4030 2023-09-13T03:13:31.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.406+0000 7f12dd271b80 10 client.25676 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:13:31.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.406+0000 7f12dd271b80 10 client.25676 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2023-09-13T03:13:31.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.406+0000 7f12dd271b80 20 client.25676 populate_metadata read hostname 'smithi064' 2023-09-13T03:13:31.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.406+0000 7f12dd271b80 1 --2- 172.21.15.64:0/421301507 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x55fc410bd840 0x55fc410bfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.406+0000 7f12dd271b80 4 client.25676 mds_command: new command op to 24634 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:13:31.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.406+0000 7f12dd271b80 1 -- 172.21.15.64:0/421301507 --> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fc4100a8a0 con 0x55fc410bd840 2023-09-13T03:13:31.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.407+0000 7f12c2ffd700 1 --2- 172.21.15.64:0/421301507 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x55fc410bd840 0x55fc410bfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:31.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.407+0000 7f12c2ffd700 1 --2- 172.21.15.64:0/421301507 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x55fc410bd840 0x55fc410bfc60 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2ff9f015c41b5cdf server_cookie=ed7afa9f13cf94b1 in_seq=0 out_seq=0 2023-09-13T03:13:31.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.407+0000 7f12a2ffd700 10 client.25676 ms_handle_connect on v2:172.21.15.170:6836/1726587275 2023-09-13T03:13:31.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.408+0000 7f12a2ffd700 1 -- 172.21.15.64:0/421301507 <== mds.3 v2:172.21.15.170:6836/1726587275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fc4100a8a0 con 0x55fc410bd840 2023-09-13T03:13:31.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.408+0000 7f12a2ffd700 10 client.25676 handle_command_reply: tid=0 2023-09-13T03:13:31.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.408+0000 7f12dd271b80 1 client.25676 shutdown 2023-09-13T03:13:31.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.409+0000 7f12dd271b80 1 -- 172.21.15.64:0/421301507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc410a4030 msgr2=0x55fc410aaa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:31.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.409+0000 7f12dd271b80 1 --2- 172.21.15.64:0/421301507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc410a4030 0x55fc410aaa00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f12b800b2e0 tx=0x7f12b8005fb0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.409+0000 7f12dd271b80 1 -- 172.21.15.64:0/421301507 shutdown_connections 2023-09-13T03:13:31.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.409+0000 7f12dd271b80 1 --2- 172.21.15.64:0/421301507 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x55fc410bd840 0x55fc410bfc60 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.409+0000 7f12dd271b80 1 --2- 172.21.15.64:0/421301507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fc410ade40 0x55fc410aaf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.409+0000 7f12dd271b80 1 --2- 172.21.15.64:0/421301507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fc410a3710 0x55fc410ad780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.409+0000 7f12dd271b80 1 --2- 172.21.15.64:0/421301507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc410a4030 0x55fc410aaa00 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.409+0000 7f12dd271b80 1 -- 172.21.15.64:0/421301507 >> 172.21.15.64:0/421301507 conn(0x55fc41018090 msgr2=0x55fc4109e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:31.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.409+0000 7f12a2ffd700 0 client.25676 ms_handle_reset on v2:172.21.15.170:6836/1726587275 2023-09-13T03:13:31.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.409+0000 7f12dd271b80 1 -- 172.21.15.64:0/421301507 shutdown_connections 2023-09-13T03:13:31.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.409+0000 7f12dd271b80 1 -- 172.21.15.64:0/421301507 wait complete. 2023-09-13T03:13:31.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.409+0000 7f12dd271b80 20 client.25676 trim_cache size 0 max 16384 2023-09-13T03:13:31.421 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:31 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/3468022372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:13:31.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.423+0000 7f12dd271b80 1 Processor -- start 2023-09-13T03:13:31.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.423+0000 7f12dd271b80 1 -- start start 2023-09-13T03:13:31.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.424+0000 7f12dd271b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc410a3710 0x55fc410a2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.424+0000 7f12dd271b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fc410a4030 0x55fc410ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.424+0000 7f12dd271b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fc410ade40 0x55fc410a7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.424+0000 7f12dd271b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc410a66b0 con 0x55fc410ade40 2023-09-13T03:13:31.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.424+0000 7f12dd271b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc410a47a0 con 0x55fc410a3710 2023-09-13T03:13:31.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.424+0000 7f12dd271b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc410ab560 con 0x55fc410a4030 2023-09-13T03:13:31.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.424+0000 7f12c2ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc410a3710 0x55fc410a2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:31.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.424+0000 7f12c2ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc410a3710 0x55fc410a2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:57898/0 (socket says 172.21.15.64:57898) 2023-09-13T03:13:31.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.424+0000 7f12c2ffd700 1 -- 172.21.15.64:0/343514580 learned_addr learned my addr 172.21.15.64:0/343514580 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:31.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.424+0000 7f12c27fc700 1 --2- 172.21.15.64:0/343514580 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fc410a4030 0x55fc410ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:31.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.424+0000 7f12c37fe700 1 --2- 172.21.15.64:0/343514580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fc410ade40 0x55fc410a7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:31.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.425+0000 7f12c2ffd700 1 -- 172.21.15.64:0/343514580 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fc410a4030 msgr2=0x55fc410ad2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:31.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.425+0000 7f12c2ffd700 1 --2- 172.21.15.64:0/343514580 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fc410a4030 0x55fc410ad2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.425+0000 7f12c2ffd700 1 -- 172.21.15.64:0/343514580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fc410ade40 msgr2=0x55fc410a7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:31.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.425+0000 7f12c2ffd700 1 --2- 172.21.15.64:0/343514580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fc410ade40 0x55fc410a7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.425+0000 7f12c2ffd700 1 -- 172.21.15.64:0/343514580 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc40e48250 con 0x55fc410a3710 2023-09-13T03:13:31.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.425+0000 7f12c27fc700 1 --2- 172.21.15.64:0/343514580 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fc410a4030 0x55fc410ad2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:13:31.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.425+0000 7f12c2ffd700 1 --2- 172.21.15.64:0/343514580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc410a3710 0x55fc410a2e10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f12b401cf80 tx=0x7f12b4005ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8291cc03d98d14fc server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:31.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.425+0000 7f12aa7fc700 1 -- 172.21.15.64:0/343514580 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b4010ea0 con 0x55fc410a3710 2023-09-13T03:13:31.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.425+0000 7f12aa7fc700 1 -- 172.21.15.64:0/343514580 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12b4021040 con 0x55fc410a3710 2023-09-13T03:13:31.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.425+0000 7f12aa7fc700 1 -- 172.21.15.64:0/343514580 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b401ca40 con 0x55fc410a3710 2023-09-13T03:13:31.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.426+0000 7f12dd271b80 1 -- 172.21.15.64:0/343514580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc410a3710 msgr2=0x55fc410a2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:31.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.426+0000 7f12dd271b80 1 --2- 172.21.15.64:0/343514580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc410a3710 0x55fc410a2e10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f12b401cf80 tx=0x7f12b4005ba0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.426+0000 7f12dd271b80 1 -- 172.21.15.64:0/343514580 shutdown_connections 2023-09-13T03:13:31.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.426+0000 7f12dd271b80 1 --2- 172.21.15.64:0/343514580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc410a3710 0x55fc410a2e10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.426+0000 7f12dd271b80 1 --2- 172.21.15.64:0/343514580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fc410ade40 0x55fc410a7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.426+0000 7f12dd271b80 1 --2- 172.21.15.64:0/343514580 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fc410a4030 0x55fc410ad2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.426+0000 7f12dd271b80 1 -- 172.21.15.64:0/343514580 >> 172.21.15.64:0/343514580 conn(0x55fc41018090 msgr2=0x55fc410a2700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:31.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.426+0000 7f12dd271b80 1 -- 172.21.15.64:0/343514580 shutdown_connections 2023-09-13T03:13:31.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.426+0000 7f12dd271b80 1 -- 172.21.15.64:0/343514580 wait complete. 2023-09-13T03:13:31.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.427+0000 7f12dd271b80 1 -- start start 2023-09-13T03:13:31.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.427+0000 7f12dd271b80 1 Processor -- start 2023-09-13T03:13:31.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.428+0000 7f12dd271b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fc410a3710 0x55fc410b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.428+0000 7f12dd271b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc410a4030 0x55fc410b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.428+0000 7f12dd271b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fc410ade40 0x55fc410b3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.428+0000 7f12dd271b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc410b3d10 con 0x55fc410ade40 2023-09-13T03:13:31.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.428+0000 7f12dd271b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc41111680 con 0x55fc410a4030 2023-09-13T03:13:31.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.428+0000 7f12dd271b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc411227d0 con 0x55fc410a3710 2023-09-13T03:13:31.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.428+0000 7f12dd271b80 10 client.0 _ll_register_callbacks cb 0x7f12b8005340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:13:31.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.429+0000 7f12c2ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fc410a3710 0x55fc410b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:31.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.429+0000 7f12c27fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc410a4030 0x55fc410b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:31.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.429+0000 7f12c2ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fc410a3710 0x55fc410b4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:46866/0 (socket says 172.21.15.64:46866) 2023-09-13T03:13:31.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.429+0000 7f12c2ffd700 1 -- 172.21.15.64:0/2312243603 learned_addr learned my addr 172.21.15.64:0/2312243603 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:31.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.429+0000 7f12c37fe700 1 --2- 172.21.15.64:0/2312243603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fc410ade40 0x55fc410b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:31.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.430+0000 7f12c2ffd700 1 -- 172.21.15.64:0/2312243603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc410a4030 msgr2=0x55fc410b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:31.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.430+0000 7f12c2ffd700 1 --2- 172.21.15.64:0/2312243603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc410a4030 0x55fc410b2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.430+0000 7f12c2ffd700 1 -- 172.21.15.64:0/2312243603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fc410ade40 msgr2=0x55fc410b3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:31.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.430+0000 7f12c2ffd700 1 --2- 172.21.15.64:0/2312243603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fc410ade40 0x55fc410b3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.430+0000 7f12c2ffd700 1 -- 172.21.15.64:0/2312243603 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fc40e0ac60 con 0x55fc410a3710 2023-09-13T03:13:31.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.430+0000 7f12c2ffd700 1 --2- 172.21.15.64:0/2312243603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fc410a3710 0x55fc410b4c10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f12b4009dc0 tx=0x7f12b4000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c9154f7607efe10 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:31.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.431+0000 7f12a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:13:31.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.431+0000 7f12a2ffd700 1 -- 172.21.15.64:0/2312243603 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b4027b80 con 0x55fc410a3710 2023-09-13T03:13:31.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.431+0000 7f12a2ffd700 1 -- 172.21.15.64:0/2312243603 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f12b4017070 con 0x55fc410a3710 2023-09-13T03:13:31.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.431+0000 7f12dd271b80 1 -- 172.21.15.64:0/2312243603 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f12b4015070 con 0x55fc410a3710 2023-09-13T03:13:31.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.431+0000 7f12a2ffd700 1 -- 172.21.15.64:0/2312243603 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12b4010ea0 con 0x55fc410a3710 2023-09-13T03:13:31.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.431+0000 7f12dd271b80 10 client.25690 fetch_fsmap learned FSMap version 26 2023-09-13T03:13:31.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.432+0000 7f12dd271b80 1 -- 172.21.15.64:0/2312243603 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55fc40e21de0 con 0x55fc410a3710 2023-09-13T03:13:31.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.432+0000 7f12a2ffd700 1 -- 172.21.15.64:0/2312243603 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12b4015070 con 0x55fc410a3710 2023-09-13T03:13:31.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.432+0000 7f12a2ffd700 1 -- 172.21.15.64:0/2312243603 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f12b403a070 con 0x55fc410a3710 2023-09-13T03:13:31.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.432+0000 7f12dd271b80 10 client.25690 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:13:31.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.432+0000 7f12dd271b80 10 client.25690 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2023-09-13T03:13:31.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.432+0000 7f12dd271b80 20 client.25690 populate_metadata read hostname 'smithi064' 2023-09-13T03:13:31.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.432+0000 7f12dd271b80 1 --2- 172.21.15.64:0/2312243603 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x55fc41123970 0x55fc41125d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.432+0000 7f12dd271b80 4 client.25690 mds_command: new command op to 24634 tid=0[{"prefix": "damage ls"}] 2023-09-13T03:13:31.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.432+0000 7f12dd271b80 1 -- 172.21.15.64:0/2312243603 --> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fc40fee920 con 0x55fc41123970 2023-09-13T03:13:31.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.433+0000 7f12c27fc700 1 --2- 172.21.15.64:0/2312243603 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x55fc41123970 0x55fc41125d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:31.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.433+0000 7f12c27fc700 1 --2- 172.21.15.64:0/2312243603 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x55fc41123970 0x55fc41125d60 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d6648df86dba9fbc server_cookie=bcad7de9d197f25e in_seq=0 out_seq=0 2023-09-13T03:13:31.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.433+0000 7f12a2ffd700 10 client.25690 ms_handle_connect on v2:172.21.15.170:6836/1726587275 2023-09-13T03:13:31.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.433+0000 7f12a2ffd700 1 -- 172.21.15.64:0/2312243603 <== mds.3 v2:172.21.15.170:6836/1726587275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fc40fee920 con 0x55fc41123970 2023-09-13T03:13:31.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.433+0000 7f12a2ffd700 10 client.25690 handle_command_reply: tid=0 2023-09-13T03:13:31.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.433+0000 7f12dd271b80 1 client.25690 shutdown 2023-09-13T03:13:31.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.434+0000 7f12dd271b80 1 -- 172.21.15.64:0/2312243603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fc410a3710 msgr2=0x55fc410b4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:31.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.434+0000 7f12dd271b80 1 --2- 172.21.15.64:0/2312243603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fc410a3710 0x55fc410b4c10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f12b4009dc0 tx=0x7f12b4000bc0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.434+0000 7f12dd271b80 1 -- 172.21.15.64:0/2312243603 shutdown_connections 2023-09-13T03:13:31.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.434+0000 7f12dd271b80 1 --2- 172.21.15.64:0/2312243603 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x55fc41123970 0x55fc41125d60 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.434+0000 7f12dd271b80 1 --2- 172.21.15.64:0/2312243603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fc410ade40 0x55fc410b3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.434+0000 7f12dd271b80 1 --2- 172.21.15.64:0/2312243603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fc410a3710 0x55fc410b4c10 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.434+0000 7f12dd271b80 1 --2- 172.21.15.64:0/2312243603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc410a4030 0x55fc410b2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.434+0000 7f12dd271b80 1 -- 172.21.15.64:0/2312243603 >> 172.21.15.64:0/2312243603 conn(0x55fc41018090 msgr2=0x55fc410a2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:31.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.434+0000 7f12a2ffd700 0 client.25690 ms_handle_reset on v2:172.21.15.170:6836/1726587275 2023-09-13T03:13:31.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.434+0000 7f12dd271b80 1 -- 172.21.15.64:0/2312243603 shutdown_connections 2023-09-13T03:13:31.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.434+0000 7f12dd271b80 1 -- 172.21.15.64:0/2312243603 wait complete. 2023-09-13T03:13:31.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.434+0000 7f12dd271b80 20 client.25690 trim_cache size 0 max 16384 2023-09-13T03:13:31.447 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2023-09-13T03:13:31.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.435+0000 7f12c99ff700 1 -- 172.21.15.64:0/2718085479 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f12ac0a08c0 msgr2=0x7f12ac0a2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:31.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.435+0000 7f12c99ff700 1 --2- 172.21.15.64:0/2718085479 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f12ac0a08c0 0x7f12ac0a2d80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f12b8005950 tx=0x7f12b80058e0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.435+0000 7f12c99ff700 1 -- 172.21.15.64:0/2718085479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12c41054e0 msgr2=0x7f12c41964e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:31.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.435+0000 7f12c99ff700 1 --2- 172.21.15.64:0/2718085479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12c41054e0 0x7f12c41964e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f12bc00ef30 tx=0x7f12bc00c3a0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.435+0000 7f12c99ff700 1 -- 172.21.15.64:0/2718085479 shutdown_connections 2023-09-13T03:13:31.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.435+0000 7f12c99ff700 1 --2- 172.21.15.64:0/2718085479 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f12ac0a08c0 0x7f12ac0a2d80 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.435+0000 7f12c99ff700 1 --2- 172.21.15.64:0/2718085479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12c41054e0 0x7f12c41964e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.435+0000 7f12c99ff700 1 --2- 172.21.15.64:0/2718085479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12c40ff440 0x7f12c419c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.435+0000 7f12c99ff700 1 --2- 172.21.15.64:0/2718085479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12c40ffe00 0x7f12c419c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.435+0000 7f12c99ff700 1 -- 172.21.15.64:0/2718085479 >> 172.21.15.64:0/2718085479 conn(0x7f12c40faf50 msgr2=0x7f12c40fb610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:31.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.435+0000 7f12c99ff700 1 -- 172.21.15.64:0/2718085479 shutdown_connections 2023-09-13T03:13:31.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.435+0000 7f12c99ff700 1 -- 172.21.15.64:0/2718085479 wait complete. 2023-09-13T03:13:31.457 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-09-13T03:13:31.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:31 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/3468022372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:13:31.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:31 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/3468022372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:13:31.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.728+0000 7f6a7efff700 1 -- 172.21.15.64:0/1746693182 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a78106630 msgr2=0x7f6a78106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:31.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.728+0000 7f6a7efff700 1 --2- 172.21.15.64:0/1746693182 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a78106630 0x7f6a78106a10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c009990 tx=0x7f6a6c009ca0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.728+0000 7f6a7efff700 1 -- 172.21.15.64:0/1746693182 shutdown_connections 2023-09-13T03:13:31.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.728+0000 7f6a7efff700 1 --2- 172.21.15.64:0/1746693182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a78100f30 0x7f6a7810d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.728+0000 7f6a7efff700 1 --2- 172.21.15.64:0/1746693182 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a78106630 0x7f6a78106a10 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.728+0000 7f6a7efff700 1 --2- 172.21.15.64:0/1746693182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a78100570 0x7f6a781009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.728+0000 7f6a7efff700 1 -- 172.21.15.64:0/1746693182 >> 172.21.15.64:0/1746693182 conn(0x7f6a780fc080 msgr2=0x7f6a780fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:31.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.728+0000 7f6a7efff700 1 -- 172.21.15.64:0/1746693182 shutdown_connections 2023-09-13T03:13:31.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.728+0000 7f6a7efff700 1 -- 172.21.15.64:0/1746693182 wait complete. 2023-09-13T03:13:31.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.730+0000 7f6a7efff700 1 Processor -- start 2023-09-13T03:13:31.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.730+0000 7f6a7efff700 1 -- start start 2023-09-13T03:13:31.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.731+0000 7f6a7efff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a78100570 0x7f6a7819e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.731+0000 7f6a7efff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a78100f30 0x7f6a7819ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.731+0000 7f6a7efff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a78106630 0x7f6a78198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.731+0000 7f6a7efff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a78198db0 con 0x7f6a78100f30 2023-09-13T03:13:31.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.731+0000 7f6a7efff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a78198f20 con 0x7f6a78100570 2023-09-13T03:13:31.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.731+0000 7f6a7efff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a781990e0 con 0x7f6a78106630 2023-09-13T03:13:31.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.731+0000 7f6a7cd9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a78100570 0x7f6a7819e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:31.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.732+0000 7f6a7d59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a78106630 0x7f6a78198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:31.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.732+0000 7f6a7d59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a78106630 0x7f6a78198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:46890/0 (socket says 172.21.15.64:46890) 2023-09-13T03:13:31.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.732+0000 7f6a7cd9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a78100570 0x7f6a7819e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:57932/0 (socket says 172.21.15.64:57932) 2023-09-13T03:13:31.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.732+0000 7f6a7d59c700 1 -- 172.21.15.64:0/4257393435 learned_addr learned my addr 172.21.15.64:0/4257393435 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:31.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.732+0000 7f6a77fff700 1 --2- 172.21.15.64:0/4257393435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a78100f30 0x7f6a7819ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:31.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.732+0000 7f6a7d59c700 1 -- 172.21.15.64:0/4257393435 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a78100570 msgr2=0x7f6a7819e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:31.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.732+0000 7f6a7d59c700 1 --2- 172.21.15.64:0/4257393435 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a78100570 0x7f6a7819e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.732+0000 7f6a7d59c700 1 -- 172.21.15.64:0/4257393435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a78100f30 msgr2=0x7f6a7819ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:31.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.732+0000 7f6a7d59c700 1 --2- 172.21.15.64:0/4257393435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a78100f30 0x7f6a7819ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.732+0000 7f6a7d59c700 1 -- 172.21.15.64:0/4257393435 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a6c0095f0 con 0x7f6a78106630 2023-09-13T03:13:31.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.732+0000 7f6a7cd9b700 1 --2- 172.21.15.64:0/4257393435 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a78100570 0x7f6a7819e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:13:31.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.732+0000 7f6a7d59c700 1 --2- 172.21.15.64:0/4257393435 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a78106630 0x7f6a78198740 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6a7000ef30 tx=0x7f6a7000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:31.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.733+0000 7f6a75ffb700 1 -- 172.21.15.64:0/4257393435 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a70018070 con 0x7f6a78106630 2023-09-13T03:13:31.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.733+0000 7f6a75ffb700 1 -- 172.21.15.64:0/4257393435 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a7000f040 con 0x7f6a78106630 2023-09-13T03:13:31.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.733+0000 7f6a75ffb700 1 -- 172.21.15.64:0/4257393435 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7000cd00 con 0x7f6a78106630 2023-09-13T03:13:31.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.733+0000 7f6a7efff700 1 -- 172.21.15.64:0/4257393435 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a78199360 con 0x7f6a78106630 2023-09-13T03:13:31.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.733+0000 7f6a7efff700 1 -- 172.21.15.64:0/4257393435 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a781a56c0 con 0x7f6a78106630 2023-09-13T03:13:31.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.739+0000 7f6a75ffb700 1 -- 172.21.15.64:0/4257393435 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 24) v1 ==== 135435+0+0 (secure 0 0 0) 0x7f6a70003a00 con 0x7f6a78106630 2023-09-13T03:13:31.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.740+0000 7f6a9292bb80 1 Processor -- start 2023-09-13T03:13:31.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.740+0000 7f6a9292bb80 1 -- start start 2023-09-13T03:13:31.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.740+0000 7f6a9292bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55608cf8a710 0x55608cf8aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.740+0000 7f6a9292bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55608cf8b030 0x55608cf9d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.740+0000 7f6a9292bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55608cf9d650 0x55608cf9fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.740+0000 7f6a9292bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55608cee3730 con 0x55608cf8b030 2023-09-13T03:13:31.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.740+0000 7f6a9292bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55608cee1050 con 0x55608cf9d650 2023-09-13T03:13:31.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.740+0000 7f6a9292bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55608cedc940 con 0x55608cf8a710 2023-09-13T03:13:31.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.741+0000 7f6a7cd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55608cf8b030 0x55608cf9d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:31.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.741+0000 7f6a7cd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55608cf8b030 0x55608cf9d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39024/0 (socket says 172.21.15.64:39024) 2023-09-13T03:13:31.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.741+0000 7f6a7cd9b700 1 -- 172.21.15.64:0/2970924789 learned_addr learned my addr 172.21.15.64:0/2970924789 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:31.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.741+0000 7f6a77fff700 1 --2- 172.21.15.64:0/2970924789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55608cf8a710 0x55608cf8aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:31.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.741+0000 7f6a77fff700 1 --2- 172.21.15.64:0/2970924789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55608cf9d650 0x55608cf9fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:31.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.742+0000 7f6a7cd9b700 1 -- 172.21.15.64:0/2970924789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55608cf8a710 msgr2=0x55608cf8aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:31.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.742+0000 7f6a7cd9b700 1 --2- 172.21.15.64:0/2970924789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55608cf8a710 0x55608cf8aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.742+0000 7f6a7cd9b700 1 -- 172.21.15.64:0/2970924789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55608cf9d650 msgr2=0x55608cf9fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:31.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.742+0000 7f6a7cd9b700 1 --2- 172.21.15.64:0/2970924789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55608cf9d650 0x55608cf9fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.742+0000 7f6a7cd9b700 1 -- 172.21.15.64:0/2970924789 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55608cd08de0 con 0x55608cf8b030 2023-09-13T03:13:31.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.742+0000 7f6a75ffb700 1 --2- 172.21.15.64:0/4257393435 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f6a680a0940 0x7f6a680a2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.742+0000 7f6a7cd9b700 1 --2- 172.21.15.64:0/2970924789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55608cf8b030 0x55608cf9d110 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c005e80 tx=0x7f6a6c005eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8bcf483f7c1fb06 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:31.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.743+0000 7f6a75ffb700 1 -- 172.21.15.64:0/4257393435 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f6a700cb370 con 0x7f6a78106630 2023-09-13T03:13:31.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.743+0000 7f6a77fff700 1 --2- 172.21.15.64:0/4257393435 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f6a680a0940 0x7f6a680a2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:31.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.743+0000 7f6a62ffd700 1 -- 172.21.15.64:0/2970924789 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c026070 con 0x55608cf8b030 2023-09-13T03:13:31.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.743+0000 7f6a62ffd700 1 -- 172.21.15.64:0/2970924789 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a6c01e040 con 0x55608cf8b030 2023-09-13T03:13:31.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.743+0000 7f6a62ffd700 1 -- 172.21.15.64:0/2970924789 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c01d070 con 0x55608cf8b030 2023-09-13T03:13:31.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.744+0000 7f6a9292bb80 1 -- 172.21.15.64:0/2970924789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55608cf8b030 msgr2=0x55608cf9d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:31.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.744+0000 7f6a9292bb80 1 --2- 172.21.15.64:0/2970924789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55608cf8b030 0x55608cf9d110 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c005e80 tx=0x7f6a6c005eb0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.744+0000 7f6a77fff700 1 --2- 172.21.15.64:0/4257393435 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f6a680a0940 0x7f6a680a2e00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6a78199bb0 tx=0x7f6a64015040 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:31.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.745+0000 7f6a9292bb80 1 -- 172.21.15.64:0/2970924789 shutdown_connections 2023-09-13T03:13:31.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.745+0000 7f6a9292bb80 1 --2- 172.21.15.64:0/2970924789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55608cf9d650 0x55608cf9fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.745+0000 7f6a9292bb80 1 --2- 172.21.15.64:0/2970924789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55608cf8a710 0x55608cf8aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.745+0000 7f6a9292bb80 1 --2- 172.21.15.64:0/2970924789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55608cf8b030 0x55608cf9d110 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.745+0000 7f6a9292bb80 1 -- 172.21.15.64:0/2970924789 >> 172.21.15.64:0/2970924789 conn(0x55608ceff090 msgr2=0x55608cf85bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:31.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.745+0000 7f6a9292bb80 1 -- 172.21.15.64:0/2970924789 shutdown_connections 2023-09-13T03:13:31.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.745+0000 7f6a9292bb80 1 -- 172.21.15.64:0/2970924789 wait complete. 2023-09-13T03:13:31.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.746+0000 7f6a9292bb80 1 -- start start 2023-09-13T03:13:31.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.746+0000 7f6a9292bb80 1 Processor -- start 2023-09-13T03:13:31.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.747+0000 7f6a9292bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55608cf8a710 0x55608cf9d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.747+0000 7f6a9292bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55608cf8b030 0x55608cf9a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.747+0000 7f6a9292bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55608cf9d650 0x55608cf9aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.747+0000 7f6a9292bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55608ce7ab70 con 0x55608cf8a710 2023-09-13T03:13:31.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.747+0000 7f6a9292bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55608ce7a870 con 0x55608cf9d650 2023-09-13T03:13:31.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.747+0000 7f6a9292bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55608cd5e920 con 0x55608cf8b030 2023-09-13T03:13:31.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.747+0000 7f6a9292bb80 10 client.0 _ll_register_callbacks cb 0x55608cdc4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:13:31.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.748+0000 7f6a7d59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55608cf9d650 0x55608cf9aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:31.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.748+0000 7f6a7d59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55608cf9d650 0x55608cf9aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:57952/0 (socket says 172.21.15.64:57952) 2023-09-13T03:13:31.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.748+0000 7f6a77fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55608cf8b030 0x55608cf9a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:31.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.748+0000 7f6a7d59c700 1 -- 172.21.15.64:0/1147662808 learned_addr learned my addr 172.21.15.64:0/1147662808 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:31.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.748+0000 7f6a7cd9b700 1 --2- 172.21.15.64:0/1147662808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55608cf8a710 0x55608cf9d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:31.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.748+0000 7f6a77fff700 1 -- 172.21.15.64:0/1147662808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55608cf9d650 msgr2=0x55608cf9aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:31.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.748+0000 7f6a77fff700 1 --2- 172.21.15.64:0/1147662808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55608cf9d650 0x55608cf9aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.748+0000 7f6a77fff700 1 -- 172.21.15.64:0/1147662808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55608cf8a710 msgr2=0x55608cf9d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:31.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.748+0000 7f6a77fff700 1 --2- 172.21.15.64:0/1147662808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55608cf8a710 0x55608cf9d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.748+0000 7f6a77fff700 1 -- 172.21.15.64:0/1147662808 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a7810dcf0 con 0x55608cf8b030 2023-09-13T03:13:31.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.749+0000 7f6a77fff700 1 --2- 172.21.15.64:0/1147662808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55608cf8b030 0x55608cf9a340 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6a64013f60 tx=0x7f6a64004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d5d6cba0ba6a640 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:31.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.749+0000 7f6a57fff700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:13:31.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.749+0000 7f6a57fff700 1 -- 172.21.15.64:0/1147662808 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a64027070 con 0x55608cf8b030 2023-09-13T03:13:31.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.749+0000 7f6a9292bb80 1 -- 172.21.15.64:0/1147662808 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55608ce344f0 con 0x55608cf8b030 2023-09-13T03:13:31.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.749+0000 7f6a57fff700 1 -- 172.21.15.64:0/1147662808 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f6a64022850 con 0x55608cf8b030 2023-09-13T03:13:31.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.750+0000 7f6a57fff700 1 -- 172.21.15.64:0/1147662808 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a64010ec0 con 0x55608cf8b030 2023-09-13T03:13:31.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.750+0000 7f6a9292bb80 10 client.25702 fetch_fsmap learned FSMap version 26 2023-09-13T03:13:31.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.750+0000 7f6a9292bb80 1 -- 172.21.15.64:0/1147662808 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55608ccf1c60 con 0x55608cf8b030 2023-09-13T03:13:31.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.751+0000 7f6a57fff700 1 -- 172.21.15.64:0/1147662808 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55608ce344f0 con 0x55608cf8b030 2023-09-13T03:13:31.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.751+0000 7f6a57fff700 1 -- 172.21.15.64:0/1147662808 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6a64035070 con 0x55608cf8b030 2023-09-13T03:13:31.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.751+0000 7f6a9292bb80 10 client.25702 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:13:31.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.751+0000 7f6a9292bb80 10 client.25702 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-09-13T03:13:31.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.751+0000 7f6a9292bb80 20 client.25702 populate_metadata read hostname 'smithi064' 2023-09-13T03:13:31.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.751+0000 7f6a9292bb80 1 --2- 172.21.15.64:0/1147662808 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x55608cfad190 0x55608cfaf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.751+0000 7f6a9292bb80 4 client.25702 mds_command: new command op to 24635 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:13:31.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.751+0000 7f6a9292bb80 1 -- 172.21.15.64:0/1147662808 --> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55608cef18a0 con 0x55608cfad190 2023-09-13T03:13:31.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.752+0000 7f6a7cd9b700 1 --2- 172.21.15.64:0/1147662808 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x55608cfad190 0x55608cfaf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:31.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.752+0000 7f6a7cd9b700 1 --2- 172.21.15.64:0/1147662808 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x55608cfad190 0x55608cfaf5b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=740aee90233e64e6 server_cookie=c584c86f0af6182e in_seq=0 out_seq=0 2023-09-13T03:13:31.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.752+0000 7f6a57fff700 10 client.25702 ms_handle_connect on v2:172.21.15.142:6836/741173268 2023-09-13T03:13:31.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.753+0000 7f6a57fff700 1 -- 172.21.15.64:0/1147662808 <== mds.2 v2:172.21.15.142:6836/741173268 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55608cef18a0 con 0x55608cfad190 2023-09-13T03:13:31.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.753+0000 7f6a57fff700 10 client.25702 handle_command_reply: tid=0 2023-09-13T03:13:31.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.753+0000 7f6a9292bb80 1 client.25702 shutdown 2023-09-13T03:13:31.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.754+0000 7f6a9292bb80 1 -- 172.21.15.64:0/1147662808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55608cf8b030 msgr2=0x55608cf9a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:31.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.754+0000 7f6a9292bb80 1 --2- 172.21.15.64:0/1147662808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55608cf8b030 0x55608cf9a340 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6a64013f60 tx=0x7f6a64004750 comp rx=0 tx=0).stop 2023-09-13T03:13:31.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.754+0000 7f6a9292bb80 1 -- 172.21.15.64:0/1147662808 shutdown_connections 2023-09-13T03:13:31.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.754+0000 7f6a9292bb80 1 --2- 172.21.15.64:0/1147662808 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x55608cfad190 0x55608cfaf5b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.754+0000 7f6a9292bb80 1 --2- 172.21.15.64:0/1147662808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55608cf9d650 0x55608cf9aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.754+0000 7f6a9292bb80 1 --2- 172.21.15.64:0/1147662808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55608cf8a710 0x55608cf9d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.754+0000 7f6a9292bb80 1 --2- 172.21.15.64:0/1147662808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55608cf8b030 0x55608cf9a340 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.754+0000 7f6a9292bb80 1 -- 172.21.15.64:0/1147662808 >> 172.21.15.64:0/1147662808 conn(0x55608ceff090 msgr2=0x55608cf85780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:31.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.754+0000 7f6a57fff700 0 client.25702 ms_handle_reset on v2:172.21.15.142:6836/741173268 2023-09-13T03:13:31.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.754+0000 7f6a9292bb80 1 -- 172.21.15.64:0/1147662808 shutdown_connections 2023-09-13T03:13:31.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.754+0000 7f6a9292bb80 1 -- 172.21.15.64:0/1147662808 wait complete. 2023-09-13T03:13:31.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.755+0000 7f6a9292bb80 20 client.25702 trim_cache size 0 max 16384 2023-09-13T03:13:31.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.781+0000 7f6a9292bb80 1 Processor -- start 2023-09-13T03:13:31.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.781+0000 7f6a9292bb80 1 -- start start 2023-09-13T03:13:31.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.781+0000 7f6a9292bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55608cf8a710 0x55608cfa41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.782+0000 7f6a9292bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55608cf8b030 0x55608cf9ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.782+0000 7f6a9292bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55608cf9d650 0x55608cfa8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.782+0000 7f6a9292bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55608cf866e0 con 0x55608cf8b030 2023-09-13T03:13:31.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.782+0000 7f6a9292bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55608cfa4720 con 0x55608cf8a710 2023-09-13T03:13:31.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.782+0000 7f6a9292bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55608cf7d400 con 0x55608cf9d650 2023-09-13T03:13:31.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.782+0000 7f6a7cd9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55608cf8a710 0x55608cfa41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:31.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.782+0000 7f6a7cd9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55608cf8a710 0x55608cfa41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:57966/0 (socket says 172.21.15.64:57966) 2023-09-13T03:13:31.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.782+0000 7f6a7cd9b700 1 -- 172.21.15.64:0/4027720078 learned_addr learned my addr 172.21.15.64:0/4027720078 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:31.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.782+0000 7f6a7d59c700 1 --2- 172.21.15.64:0/4027720078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55608cf9d650 0x55608cfa8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:31.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.782+0000 7f6a77fff700 1 --2- 172.21.15.64:0/4027720078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55608cf8b030 0x55608cf9ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:31.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.782+0000 7f6a7cd9b700 1 -- 172.21.15.64:0/4027720078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55608cf9d650 msgr2=0x55608cfa8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:31.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.782+0000 7f6a7cd9b700 1 --2- 172.21.15.64:0/4027720078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55608cf9d650 0x55608cfa8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.782+0000 7f6a7cd9b700 1 -- 172.21.15.64:0/4027720078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55608cf8b030 msgr2=0x55608cf9ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:31.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.782+0000 7f6a7cd9b700 1 --2- 172.21.15.64:0/4027720078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55608cf8b030 0x55608cf9ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.783+0000 7f6a7cd9b700 1 -- 172.21.15.64:0/4027720078 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55608cd2f250 con 0x55608cf8a710 2023-09-13T03:13:31.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.783+0000 7f6a7cd9b700 1 --2- 172.21.15.64:0/4027720078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55608cf8a710 0x55608cfa41e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c009990 tx=0x7f6a6c00b300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e8abe7bf3963d7a server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:31.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.783+0000 7f6a637fe700 1 -- 172.21.15.64:0/4027720078 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c026040 con 0x55608cf8a710 2023-09-13T03:13:31.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.783+0000 7f6a637fe700 1 -- 172.21.15.64:0/4027720078 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a6c027040 con 0x55608cf8a710 2023-09-13T03:13:31.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.783+0000 7f6a637fe700 1 -- 172.21.15.64:0/4027720078 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c01d040 con 0x55608cf8a710 2023-09-13T03:13:31.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.784+0000 7f6a9292bb80 1 -- 172.21.15.64:0/4027720078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55608cf8a710 msgr2=0x55608cfa41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:31.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.784+0000 7f6a9292bb80 1 --2- 172.21.15.64:0/4027720078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55608cf8a710 0x55608cfa41e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c009990 tx=0x7f6a6c00b300 comp rx=0 tx=0).stop 2023-09-13T03:13:31.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.784+0000 7f6a9292bb80 1 -- 172.21.15.64:0/4027720078 shutdown_connections 2023-09-13T03:13:31.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.784+0000 7f6a9292bb80 1 --2- 172.21.15.64:0/4027720078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55608cf8a710 0x55608cfa41e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.784+0000 7f6a9292bb80 1 --2- 172.21.15.64:0/4027720078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55608cf9d650 0x55608cfa8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.784+0000 7f6a9292bb80 1 --2- 172.21.15.64:0/4027720078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55608cf8b030 0x55608cf9ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.784+0000 7f6a9292bb80 1 -- 172.21.15.64:0/4027720078 >> 172.21.15.64:0/4027720078 conn(0x55608ceff090 msgr2=0x55608cfa9490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:31.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.785+0000 7f6a9292bb80 1 -- 172.21.15.64:0/4027720078 shutdown_connections 2023-09-13T03:13:31.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.785+0000 7f6a9292bb80 1 -- 172.21.15.64:0/4027720078 wait complete. 2023-09-13T03:13:31.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.785+0000 7f6a9292bb80 1 -- start start 2023-09-13T03:13:31.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.785+0000 7f6a9292bb80 1 Processor -- start 2023-09-13T03:13:31.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.786+0000 7f6a9292bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55608cf8a710 0x55608cfa3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.786+0000 7f6a9292bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55608cf8b030 0x55608cfa21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.786+0000 7f6a9292bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55608cf9d650 0x55608cfa2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.786+0000 7f6a9292bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55608cfa47d0 con 0x55608cf8a710 2023-09-13T03:13:31.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.786+0000 7f6a9292bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55608cfa2c70 con 0x55608cf9d650 2023-09-13T03:13:31.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.786+0000 7f6a9292bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55608cfa2de0 con 0x55608cf8b030 2023-09-13T03:13:31.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.786+0000 7f6a9292bb80 10 client.0 _ll_register_callbacks cb 0x7f6a64010890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:13:31.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.787+0000 7f6a7cd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55608cf8a710 0x55608cfa3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:31.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.787+0000 7f6a7cd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55608cf8a710 0x55608cfa3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39040/0 (socket says 172.21.15.64:39040) 2023-09-13T03:13:31.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.787+0000 7f6a7cd9b700 1 -- 172.21.15.64:0/485702980 learned_addr learned my addr 172.21.15.64:0/485702980 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:31.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.787+0000 7f6a7d59c700 1 --2- 172.21.15.64:0/485702980 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55608cf9d650 0x55608cfa2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:31.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.787+0000 7f6a7cd9b700 1 -- 172.21.15.64:0/485702980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55608cf8b030 msgr2=0x55608cfa21f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-13T03:13:31.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.787+0000 7f6a7cd9b700 1 --2- 172.21.15.64:0/485702980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55608cf8b030 0x55608cfa21f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.787+0000 7f6a7cd9b700 1 -- 172.21.15.64:0/485702980 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55608cf9d650 msgr2=0x55608cfa2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:31.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.787+0000 7f6a7cd9b700 1 --2- 172.21.15.64:0/485702980 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55608cf9d650 0x55608cfa2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.787+0000 7f6a7cd9b700 1 -- 172.21.15.64:0/485702980 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55608cd08de0 con 0x55608cf8a710 2023-09-13T03:13:31.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.787+0000 7f6a7d59c700 1 --2- 172.21.15.64:0/485702980 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55608cf9d650 0x55608cfa2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:13:31.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.788+0000 7f6a7cd9b700 1 --2- 172.21.15.64:0/485702980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55608cf8a710 0x55608cfa3f00 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c005f40 tx=0x7f6a6c00bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff8ef8d288abbe45 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:31.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.788+0000 7f6a57fff700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:13:31.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.788+0000 7f6a57fff700 1 -- 172.21.15.64:0/485702980 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c01d040 con 0x55608cf8a710 2023-09-13T03:13:31.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.788+0000 7f6a57fff700 1 -- 172.21.15.64:0/485702980 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f6a6c030350 con 0x55608cf8a710 2023-09-13T03:13:31.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.788+0000 7f6a9292bb80 1 -- 172.21.15.64:0/485702980 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a6c01e040 con 0x55608cf8a710 2023-09-13T03:13:31.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.789+0000 7f6a57fff700 1 -- 172.21.15.64:0/485702980 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a6c026040 con 0x55608cf8a710 2023-09-13T03:13:31.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.789+0000 7f6a9292bb80 10 client.15666 fetch_fsmap learned FSMap version 26 2023-09-13T03:13:31.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.789+0000 7f6a9292bb80 1 -- 172.21.15.64:0/485702980 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55608cd072b0 con 0x55608cf8a710 2023-09-13T03:13:31.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.789+0000 7f6a57fff700 1 -- 172.21.15.64:0/485702980 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a6c01e040 con 0x55608cf8a710 2023-09-13T03:13:31.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.789+0000 7f6a57fff700 1 -- 172.21.15.64:0/485702980 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6a6c005450 con 0x55608cf8a710 2023-09-13T03:13:31.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.789+0000 7f6a9292bb80 10 client.15666 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:13:31.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.789+0000 7f6a9292bb80 10 client.15666 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-09-13T03:13:31.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.789+0000 7f6a9292bb80 20 client.15666 populate_metadata read hostname 'smithi064' 2023-09-13T03:13:31.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.790+0000 7f6a9292bb80 1 --2- 172.21.15.64:0/485702980 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x55608d001ed0 0x55608d0042f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:31.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.790+0000 7f6a9292bb80 4 client.15666 mds_command: new command op to 24635 tid=0[{"prefix": "damage ls"}] 2023-09-13T03:13:31.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.790+0000 7f6a9292bb80 1 -- 172.21.15.64:0/485702980 --> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55608ced5920 con 0x55608d001ed0 2023-09-13T03:13:31.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.790+0000 7f6a77fff700 1 --2- 172.21.15.64:0/485702980 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x55608d001ed0 0x55608d0042f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:31.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.791+0000 7f6a77fff700 1 --2- 172.21.15.64:0/485702980 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x55608d001ed0 0x55608d0042f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b923a637c25f7f49 server_cookie=c466df35e9f83fe9 in_seq=0 out_seq=0 2023-09-13T03:13:31.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.791+0000 7f6a57fff700 10 client.15666 ms_handle_connect on v2:172.21.15.142:6836/741173268 2023-09-13T03:13:31.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.791+0000 7f6a57fff700 1 -- 172.21.15.64:0/485702980 <== mds.2 v2:172.21.15.142:6836/741173268 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55608ced5920 con 0x55608d001ed0 2023-09-13T03:13:31.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.791+0000 7f6a57fff700 10 client.15666 handle_command_reply: tid=0 2023-09-13T03:13:31.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.791+0000 7f6a9292bb80 1 client.15666 shutdown 2023-09-13T03:13:31.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.791+0000 7f6a9292bb80 1 -- 172.21.15.64:0/485702980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55608cf8a710 msgr2=0x55608cfa3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:31.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.791+0000 7f6a9292bb80 1 --2- 172.21.15.64:0/485702980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55608cf8a710 0x55608cfa3f00 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c005f40 tx=0x7f6a6c00bef0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.791+0000 7f6a9292bb80 1 -- 172.21.15.64:0/485702980 shutdown_connections 2023-09-13T03:13:31.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.791+0000 7f6a9292bb80 1 --2- 172.21.15.64:0/485702980 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x55608d001ed0 0x55608d0042f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.791+0000 7f6a9292bb80 1 --2- 172.21.15.64:0/485702980 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55608cf9d650 0x55608cfa2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.791+0000 7f6a9292bb80 1 --2- 172.21.15.64:0/485702980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55608cf8a710 0x55608cfa3f00 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.791+0000 7f6a57fff700 0 client.15666 ms_handle_reset on v2:172.21.15.142:6836/741173268 2023-09-13T03:13:31.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.791+0000 7f6a9292bb80 1 --2- 172.21.15.64:0/485702980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55608cf8b030 0x55608cfa21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.791+0000 7f6a9292bb80 1 -- 172.21.15.64:0/485702980 >> 172.21.15.64:0/485702980 conn(0x55608ceff090 msgr2=0x55608cf9ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:31.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.792+0000 7f6a9292bb80 1 -- 172.21.15.64:0/485702980 shutdown_connections 2023-09-13T03:13:31.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.792+0000 7f6a9292bb80 1 -- 172.21.15.64:0/485702980 wait complete. 2023-09-13T03:13:31.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.792+0000 7f6a9292bb80 20 client.15666 trim_cache size 0 max 16384 2023-09-13T03:13:31.803 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2023-09-13T03:13:31.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.793+0000 7f6a7efff700 1 -- 172.21.15.64:0/4257393435 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f6a680a0940 msgr2=0x7f6a680a2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:31.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.793+0000 7f6a7efff700 1 --2- 172.21.15.64:0/4257393435 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f6a680a0940 0x7f6a680a2e00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6a78199bb0 tx=0x7f6a64015040 comp rx=0 tx=0).stop 2023-09-13T03:13:31.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.793+0000 7f6a7efff700 1 -- 172.21.15.64:0/4257393435 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a78106630 msgr2=0x7f6a78198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:31.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.793+0000 7f6a7efff700 1 --2- 172.21.15.64:0/4257393435 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a78106630 0x7f6a78198740 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6a7000ef30 tx=0x7f6a7000c3a0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.793+0000 7f6a7efff700 1 -- 172.21.15.64:0/4257393435 shutdown_connections 2023-09-13T03:13:31.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.793+0000 7f6a7efff700 1 --2- 172.21.15.64:0/4257393435 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f6a680a0940 0x7f6a680a2e00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.793+0000 7f6a7efff700 1 --2- 172.21.15.64:0/4257393435 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a78100570 0x7f6a7819e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.793+0000 7f6a7efff700 1 --2- 172.21.15.64:0/4257393435 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a78106630 0x7f6a78198740 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.793+0000 7f6a7efff700 1 --2- 172.21.15.64:0/4257393435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a78100f30 0x7f6a7819ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:31.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.793+0000 7f6a7efff700 1 -- 172.21.15.64:0/4257393435 >> 172.21.15.64:0/4257393435 conn(0x7f6a780fc080 msgr2=0x7f6a78072830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:31.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.793+0000 7f6a7efff700 1 -- 172.21.15.64:0/4257393435 shutdown_connections 2023-09-13T03:13:31.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:31.793+0000 7f6a7efff700 1 -- 172.21.15.64:0/4257393435 wait complete. 2023-09-13T03:13:31.824 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-09-13T03:13:32.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.081+0000 7f6cd5b9d700 1 -- 172.21.15.64:0/598218625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cd0068100 msgr2=0x7f6cd010d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:32.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.081+0000 7f6cd5b9d700 1 --2- 172.21.15.64:0/598218625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cd0068100 0x7f6cd010d250 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8009f90 tx=0x7f6cc800b330 comp rx=0 tx=0).stop 2023-09-13T03:13:32.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.081+0000 7f6cd5b9d700 1 -- 172.21.15.64:0/598218625 shutdown_connections 2023-09-13T03:13:32.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.081+0000 7f6cd5b9d700 1 --2- 172.21.15.64:0/598218625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cd0068100 0x7f6cd010d250 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.081+0000 7f6cd5b9d700 1 --2- 172.21.15.64:0/598218625 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cd01053c0 0x7f6cd01057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.081+0000 7f6cd5b9d700 1 --2- 172.21.15.64:0/598218625 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6cd0067760 0x7f6cd0067bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.081+0000 7f6cd5b9d700 1 -- 172.21.15.64:0/598218625 >> 172.21.15.64:0/598218625 conn(0x7f6cd0074dc0 msgr2=0x7f6cd00751d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:32.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.081+0000 7f6cd5b9d700 1 -- 172.21.15.64:0/598218625 shutdown_connections 2023-09-13T03:13:32.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.082+0000 7f6cd5b9d700 1 -- 172.21.15.64:0/598218625 wait complete. 2023-09-13T03:13:32.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.083+0000 7f6cd5b9d700 1 Processor -- start 2023-09-13T03:13:32.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.083+0000 7f6cd5b9d700 1 -- start start 2023-09-13T03:13:32.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.084+0000 7f6cd5b9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cd0067760 0x7f6cd0198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:32.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.084+0000 7f6cd5b9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6cd0068100 0x7f6cd0198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:32.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.084+0000 7f6cd5b9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cd01053c0 0x7f6cd019c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:32.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.084+0000 7f6cd5b9d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd019c600 con 0x7f6cd01053c0 2023-09-13T03:13:32.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.084+0000 7f6cd5b9d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd019c770 con 0x7f6cd0067760 2023-09-13T03:13:32.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.084+0000 7f6cd5b9d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd019c930 con 0x7f6cd0068100 2023-09-13T03:13:32.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.084+0000 7f6cd4b9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cd0067760 0x7f6cd0198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:32.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.084+0000 7f6cd4b9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cd0067760 0x7f6cd0198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:58008/0 (socket says 172.21.15.64:58008) 2023-09-13T03:13:32.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.084+0000 7f6cd4b9b700 1 -- 172.21.15.64:0/232289335 learned_addr learned my addr 172.21.15.64:0/232289335 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:32.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.084+0000 7f6ccffff700 1 --2- 172.21.15.64:0/232289335 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6cd0068100 0x7f6cd0198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:32.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.085+0000 7f6cd539c700 1 --2- 172.21.15.64:0/232289335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cd01053c0 0x7f6cd019c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:32.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.085+0000 7f6cd4b9b700 1 -- 172.21.15.64:0/232289335 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6cd0068100 msgr2=0x7f6cd0198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:32.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.085+0000 7f6cd4b9b700 1 --2- 172.21.15.64:0/232289335 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6cd0068100 0x7f6cd0198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.085+0000 7f6cd4b9b700 1 -- 172.21.15.64:0/232289335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cd01053c0 msgr2=0x7f6cd019c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:32.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.085+0000 7f6cd4b9b700 1 --2- 172.21.15.64:0/232289335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cd01053c0 0x7f6cd019c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.085+0000 7f6cd4b9b700 1 -- 172.21.15.64:0/232289335 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc8009c40 con 0x7f6cd0067760 2023-09-13T03:13:32.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.085+0000 7f6cd4b9b700 1 --2- 172.21.15.64:0/232289335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cd0067760 0x7f6cd0198710 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6cc400d980 tx=0x7f6cc400dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:32.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.085+0000 7f6ccdffb700 1 -- 172.21.15.64:0/232289335 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc400bb60 con 0x7f6cd0067760 2023-09-13T03:13:32.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.086+0000 7f6ccdffb700 1 -- 172.21.15.64:0/232289335 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cc4007620 con 0x7f6cd0067760 2023-09-13T03:13:32.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.086+0000 7f6ccdffb700 1 -- 172.21.15.64:0/232289335 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc400f3c0 con 0x7f6cd0067760 2023-09-13T03:13:32.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.086+0000 7f6cd5b9d700 1 -- 172.21.15.64:0/232289335 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cd019cc10 con 0x7f6cd0067760 2023-09-13T03:13:32.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.086+0000 7f6cd5b9d700 1 -- 172.21.15.64:0/232289335 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cd01a5570 con 0x7f6cd0067760 2023-09-13T03:13:32.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.090+0000 7f6cea778b80 1 Processor -- start 2023-09-13T03:13:32.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.091+0000 7f6cea778b80 1 -- start start 2023-09-13T03:13:32.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.092+0000 7f6cea778b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ec8805860 0x563ec8805c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:32.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.092+0000 7f6cea778b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ec8806180 0x563ec8819300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:32.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.092+0000 7f6cea778b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ec8819840 0x563ec881bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:32.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.092+0000 7f6cea778b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ec875f730 con 0x563ec8819840 2023-09-13T03:13:32.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.092+0000 7f6cea778b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ec875d050 con 0x563ec8806180 2023-09-13T03:13:32.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.092+0000 7f6cea778b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ec8758940 con 0x563ec8805860 2023-09-13T03:13:32.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.092+0000 7f6ccffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ec8819840 0x563ec881bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:32.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.092+0000 7f6cd539c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ec8805860 0x563ec8805c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:32.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.092+0000 7f6cd539c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ec8805860 0x563ec8805c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:46938/0 (socket says 172.21.15.64:46938) 2023-09-13T03:13:32.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.092+0000 7f6cd539c700 1 -- 172.21.15.64:0/2919126230 learned_addr learned my addr 172.21.15.64:0/2919126230 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:32.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.092+0000 7f6ccdffb700 1 -- 172.21.15.64:0/232289335 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) v1 ==== 135435+0+0 (secure 0 0 0) 0x7f6cc4010460 con 0x7f6cd0067760 2023-09-13T03:13:32.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.093+0000 7f6cd539c700 1 -- 172.21.15.64:0/2919126230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ec8806180 msgr2=0x563ec8819300 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-13T03:13:32.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.093+0000 7f6cd539c700 1 --2- 172.21.15.64:0/2919126230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ec8806180 0x563ec8819300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.093+0000 7f6cd539c700 1 -- 172.21.15.64:0/2919126230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ec8819840 msgr2=0x563ec881bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:32.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.093+0000 7f6cd539c700 1 --2- 172.21.15.64:0/2919126230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ec8819840 0x563ec881bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.093+0000 7f6cd539c700 1 -- 172.21.15.64:0/2919126230 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ec8584de0 con 0x563ec8805860 2023-09-13T03:13:32.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.093+0000 7f6cd539c700 1 --2- 172.21.15.64:0/2919126230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ec8805860 0x563ec8805c40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8009340 tx=0x7f6cc8009370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ae4348f6cdef9fe server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:32.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.093+0000 7f6cbaffd700 1 -- 172.21.15.64:0/2919126230 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc800e070 con 0x563ec8805860 2023-09-13T03:13:32.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.093+0000 7f6cbaffd700 1 -- 172.21.15.64:0/2919126230 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cc8007d50 con 0x563ec8805860 2023-09-13T03:13:32.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.093+0000 7f6cbaffd700 1 -- 172.21.15.64:0/2919126230 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc800bdf0 con 0x563ec8805860 2023-09-13T03:13:32.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.094+0000 7f6cea778b80 1 -- 172.21.15.64:0/2919126230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ec8805860 msgr2=0x563ec8805c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:32.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.094+0000 7f6cea778b80 1 --2- 172.21.15.64:0/2919126230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ec8805860 0x563ec8805c40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8009340 tx=0x7f6cc8009370 comp rx=0 tx=0).stop 2023-09-13T03:13:32.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.095+0000 7f6cea778b80 1 -- 172.21.15.64:0/2919126230 shutdown_connections 2023-09-13T03:13:32.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.095+0000 7f6cea778b80 1 --2- 172.21.15.64:0/2919126230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ec8819840 0x563ec881bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.095+0000 7f6cea778b80 1 --2- 172.21.15.64:0/2919126230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ec8805860 0x563ec8805c40 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.095+0000 7f6cea778b80 1 --2- 172.21.15.64:0/2919126230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ec8806180 0x563ec8819300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.095+0000 7f6cea778b80 1 -- 172.21.15.64:0/2919126230 >> 172.21.15.64:0/2919126230 conn(0x563ec877b090 msgr2=0x563ec8801d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:32.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.095+0000 7f6ccdffb700 1 --2- 172.21.15.64:0/232289335 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f6cc00a0960 0x7f6cc00a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:32.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.095+0000 7f6ccdffb700 1 -- 172.21.15.64:0/232289335 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f6cc40cc280 con 0x7f6cd0067760 2023-09-13T03:13:32.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.095+0000 7f6cea778b80 1 -- 172.21.15.64:0/2919126230 shutdown_connections 2023-09-13T03:13:32.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.095+0000 7f6cea778b80 1 -- 172.21.15.64:0/2919126230 wait complete. 2023-09-13T03:13:32.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.096+0000 7f6ccffff700 1 --2- 172.21.15.64:0/232289335 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f6cc00a0960 0x7f6cc00a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:32.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.096+0000 7f6cea778b80 1 -- start start 2023-09-13T03:13:32.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.096+0000 7f6cea778b80 1 Processor -- start 2023-09-13T03:13:32.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.097+0000 7f6ccffff700 1 --2- 172.21.15.64:0/232289335 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f6cc00a0960 0x7f6cc00a2e20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0199d20 tx=0x7f6cbc009310 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:32.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.097+0000 7f6cea778b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ec8805860 0x563ec88165a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:32.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.097+0000 7f6cea778b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ec8806180 0x563ec8816ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:32.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.097+0000 7f6cea778b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ec8819840 0x563ec8817020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:32.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.097+0000 7f6cea778b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ec86f6b70 con 0x563ec8819840 2023-09-13T03:13:32.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.097+0000 7f6cea778b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ec86f6870 con 0x563ec8806180 2023-09-13T03:13:32.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.097+0000 7f6cea778b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ec85da920 con 0x563ec8805860 2023-09-13T03:13:32.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.097+0000 7f6cea778b80 10 client.0 _ll_register_callbacks cb 0x563ec8640a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:13:32.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.098+0000 7f6cd4b9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ec8805860 0x563ec88165a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:32.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.098+0000 7f6cd4b9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ec8805860 0x563ec88165a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:46944/0 (socket says 172.21.15.64:46944) 2023-09-13T03:13:32.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.098+0000 7f6cd4b9b700 1 -- 172.21.15.64:0/601954062 learned_addr learned my addr 172.21.15.64:0/601954062 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:32.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.098+0000 7f6cd539c700 1 --2- 172.21.15.64:0/601954062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ec8819840 0x563ec8817020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:32.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.098+0000 7f6ccffff700 1 --2- 172.21.15.64:0/601954062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ec8806180 0x563ec8816ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:32.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.098+0000 7f6cd4b9b700 1 -- 172.21.15.64:0/601954062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ec8806180 msgr2=0x563ec8816ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:32.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.098+0000 7f6cd4b9b700 1 --2- 172.21.15.64:0/601954062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ec8806180 0x563ec8816ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.098+0000 7f6cd4b9b700 1 -- 172.21.15.64:0/601954062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ec8819840 msgr2=0x563ec8817020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:32.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.098+0000 7f6cd4b9b700 1 --2- 172.21.15.64:0/601954062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ec8819840 0x563ec8817020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.098+0000 7f6cd4b9b700 1 -- 172.21.15.64:0/601954062 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cd010db50 con 0x563ec8805860 2023-09-13T03:13:32.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.099+0000 7f6cd4b9b700 1 --2- 172.21.15.64:0/601954062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ec8805860 0x563ec88165a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4000c00 tx=0x7f6cc4007f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23e6b25aedf485c8 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:32.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.099+0000 7f6caffff700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:13:32.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.099+0000 7f6caffff700 1 -- 172.21.15.64:0/601954062 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc409d3e0 con 0x563ec8805860 2023-09-13T03:13:32.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.099+0000 7f6caffff700 1 -- 172.21.15.64:0/601954062 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f6cc40a1620 con 0x563ec8805860 2023-09-13T03:13:32.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.099+0000 7f6cea778b80 1 -- 172.21.15.64:0/601954062 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563ec86b04f0 con 0x563ec8805860 2023-09-13T03:13:32.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.099+0000 7f6caffff700 1 -- 172.21.15.64:0/601954062 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6cc40840b0 con 0x563ec8805860 2023-09-13T03:13:32.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.099+0000 7f6cea778b80 10 client.25717 fetch_fsmap learned FSMap version 26 2023-09-13T03:13:32.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.099+0000 7f6cea778b80 1 -- 172.21.15.64:0/601954062 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x563ec856dc60 con 0x563ec8805860 2023-09-13T03:13:32.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.100+0000 7f6caffff700 1 -- 172.21.15.64:0/601954062 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563ec86b04f0 con 0x563ec8805860 2023-09-13T03:13:32.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.100+0000 7f6caffff700 1 -- 172.21.15.64:0/601954062 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6cc4098b30 con 0x563ec8805860 2023-09-13T03:13:32.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.100+0000 7f6cea778b80 10 client.25717 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:13:32.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.100+0000 7f6cea778b80 10 client.25717 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-09-13T03:13:32.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.100+0000 7f6cea778b80 20 client.25717 populate_metadata read hostname 'smithi064' 2023-09-13T03:13:32.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.100+0000 7f6cea778b80 1 --2- 172.21.15.64:0/601954062 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x563ec88292a0 0x563ec882b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:32.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.100+0000 7f6cea778b80 4 client.25717 mds_command: new command op to 24649 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:13:32.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.100+0000 7f6cea778b80 1 -- 172.21.15.64:0/601954062 --> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563ec876d8a0 con 0x563ec88292a0 2023-09-13T03:13:32.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.100+0000 7f6ccffff700 1 --2- 172.21.15.64:0/601954062 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x563ec88292a0 0x563ec882b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:32.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.101+0000 7f6ccffff700 1 --2- 172.21.15.64:0/601954062 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x563ec88292a0 0x563ec882b6c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5e21dec48aca9b1a server_cookie=366cef2f79e32d5b in_seq=0 out_seq=0 2023-09-13T03:13:32.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.101+0000 7f6caffff700 10 client.25717 ms_handle_connect on v2:172.21.15.170:6838/2189703596 2023-09-13T03:13:32.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.101+0000 7f6caffff700 1 -- 172.21.15.64:0/601954062 <== mds.1 v2:172.21.15.170:6838/2189703596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563ec876d8a0 con 0x563ec88292a0 2023-09-13T03:13:32.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.101+0000 7f6caffff700 10 client.25717 handle_command_reply: tid=0 2023-09-13T03:13:32.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.102+0000 7f6cea778b80 1 client.25717 shutdown 2023-09-13T03:13:32.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.102+0000 7f6cea778b80 1 -- 172.21.15.64:0/601954062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ec8805860 msgr2=0x563ec88165a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:32.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.102+0000 7f6cea778b80 1 --2- 172.21.15.64:0/601954062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ec8805860 0x563ec88165a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4000c00 tx=0x7f6cc4007f50 comp rx=0 tx=0).stop 2023-09-13T03:13:32.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.102+0000 7f6cea778b80 1 -- 172.21.15.64:0/601954062 shutdown_connections 2023-09-13T03:13:32.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.102+0000 7f6cea778b80 1 --2- 172.21.15.64:0/601954062 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x563ec88292a0 0x563ec882b6c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.102+0000 7f6cea778b80 1 --2- 172.21.15.64:0/601954062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ec8819840 0x563ec8817020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.102+0000 7f6cea778b80 1 --2- 172.21.15.64:0/601954062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ec8805860 0x563ec88165a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.102+0000 7f6cea778b80 1 --2- 172.21.15.64:0/601954062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ec8806180 0x563ec8816ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.102+0000 7f6cea778b80 1 -- 172.21.15.64:0/601954062 >> 172.21.15.64:0/601954062 conn(0x563ec877b090 msgr2=0x563ec88018e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:32.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.102+0000 7f6caffff700 0 client.25717 ms_handle_reset on v2:172.21.15.170:6838/2189703596 2023-09-13T03:13:32.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.102+0000 7f6cea778b80 1 -- 172.21.15.64:0/601954062 shutdown_connections 2023-09-13T03:13:32.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.102+0000 7f6cea778b80 1 -- 172.21.15.64:0/601954062 wait complete. 2023-09-13T03:13:32.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.102+0000 7f6cea778b80 20 client.25717 trim_cache size 0 max 16384 2023-09-13T03:13:32.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.116+0000 7f6cea778b80 1 Processor -- start 2023-09-13T03:13:32.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.116+0000 7f6cea778b80 1 -- start start 2023-09-13T03:13:32.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.116+0000 7f6cea778b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ec8805860 0x563ec8815830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:32.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.116+0000 7f6cea778b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ec8806180 0x563ec8815d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:32.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.116+0000 7f6cea778b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ec8819840 0x563ec8813830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:32.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.116+0000 7f6cea778b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ec88290c0 con 0x563ec8805860 2023-09-13T03:13:32.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.116+0000 7f6cea778b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ec8802a90 con 0x563ec8819840 2023-09-13T03:13:32.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.116+0000 7f6cea778b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ec87fe530 con 0x563ec8806180 2023-09-13T03:13:32.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.117+0000 7f6cd539c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ec8819840 0x563ec8813830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:32.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.117+0000 7f6cd539c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ec8819840 0x563ec8813830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:58030/0 (socket says 172.21.15.64:58030) 2023-09-13T03:13:32.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.117+0000 7f6cd539c700 1 -- 172.21.15.64:0/3120096953 learned_addr learned my addr 172.21.15.64:0/3120096953 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:32.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.117+0000 7f6cd4b9b700 1 --2- 172.21.15.64:0/3120096953 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ec8805860 0x563ec8815830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:32.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.117+0000 7f6cd539c700 1 -- 172.21.15.64:0/3120096953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ec8806180 msgr2=0x563ec8815d70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-13T03:13:32.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.117+0000 7f6cd539c700 1 --2- 172.21.15.64:0/3120096953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ec8806180 0x563ec8815d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.117+0000 7f6cd539c700 1 -- 172.21.15.64:0/3120096953 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ec8805860 msgr2=0x563ec8815830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:32.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.117+0000 7f6cd539c700 1 --2- 172.21.15.64:0/3120096953 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ec8805860 0x563ec8815830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.117+0000 7f6cd539c700 1 -- 172.21.15.64:0/3120096953 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ec88302b0 con 0x563ec8819840 2023-09-13T03:13:32.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.118+0000 7f6cd539c700 1 --2- 172.21.15.64:0/3120096953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ec8819840 0x563ec8813830 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8003ee0 tx=0x7f6cc8003e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a742f34b9bdcaf15 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:32.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.118+0000 7f6cbb7fe700 1 -- 172.21.15.64:0/3120096953 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc800e040 con 0x563ec8819840 2023-09-13T03:13:32.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.118+0000 7f6cbb7fe700 1 -- 172.21.15.64:0/3120096953 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cc8003c00 con 0x563ec8819840 2023-09-13T03:13:32.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.118+0000 7f6cbb7fe700 1 -- 172.21.15.64:0/3120096953 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc801b7a0 con 0x563ec8819840 2023-09-13T03:13:32.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.119+0000 7f6cea778b80 1 -- 172.21.15.64:0/3120096953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ec8819840 msgr2=0x563ec8813830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:32.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.119+0000 7f6cea778b80 1 --2- 172.21.15.64:0/3120096953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ec8819840 0x563ec8813830 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8003ee0 tx=0x7f6cc8003e70 comp rx=0 tx=0).stop 2023-09-13T03:13:32.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.119+0000 7f6cea778b80 1 -- 172.21.15.64:0/3120096953 shutdown_connections 2023-09-13T03:13:32.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.119+0000 7f6cea778b80 1 --2- 172.21.15.64:0/3120096953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ec8819840 0x563ec8813830 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.119+0000 7f6cea778b80 1 --2- 172.21.15.64:0/3120096953 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ec8805860 0x563ec8815830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.119+0000 7f6cea778b80 1 --2- 172.21.15.64:0/3120096953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ec8806180 0x563ec8815d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.119+0000 7f6cea778b80 1 -- 172.21.15.64:0/3120096953 >> 172.21.15.64:0/3120096953 conn(0x563ec877b090 msgr2=0x563ec88080f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:32.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.119+0000 7f6cea778b80 1 -- 172.21.15.64:0/3120096953 shutdown_connections 2023-09-13T03:13:32.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.119+0000 7f6cea778b80 1 -- 172.21.15.64:0/3120096953 wait complete. 2023-09-13T03:13:32.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.119+0000 7f6cea778b80 1 -- start start 2023-09-13T03:13:32.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.119+0000 7f6cea778b80 1 Processor -- start 2023-09-13T03:13:32.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.120+0000 7f6cea778b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ec8805860 0x563ec8820350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:32.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.120+0000 7f6cea778b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ec8806180 0x563ec881e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:32.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.120+0000 7f6cea778b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ec8819840 0x563ec881ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:32.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.120+0000 7f6cea778b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ec881f3c0 con 0x563ec8806180 2023-09-13T03:13:32.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.120+0000 7f6cea778b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ec881f530 con 0x563ec8805860 2023-09-13T03:13:32.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.120+0000 7f6cea778b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ec887d0a0 con 0x563ec8819840 2023-09-13T03:13:32.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.120+0000 7f6cea778b80 10 client.0 _ll_register_callbacks cb 0x7f6cc40991e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:13:32.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.120+0000 7f6cd4b9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ec8805860 0x563ec8820350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:32.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.120+0000 7f6cd4b9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ec8805860 0x563ec8820350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:58042/0 (socket says 172.21.15.64:58042) 2023-09-13T03:13:32.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.120+0000 7f6cd4b9b700 1 -- 172.21.15.64:0/1094432999 learned_addr learned my addr 172.21.15.64:0/1094432999 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:32.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.120+0000 7f6cd539c700 1 --2- 172.21.15.64:0/1094432999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ec8819840 0x563ec881ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:32.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.121+0000 7f6cd4b9b700 1 -- 172.21.15.64:0/1094432999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ec8819840 msgr2=0x563ec881ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:32.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.121+0000 7f6cd4b9b700 1 --2- 172.21.15.64:0/1094432999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ec8819840 0x563ec881ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.121+0000 7f6cd4b9b700 1 -- 172.21.15.64:0/1094432999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ec8806180 msgr2=0x563ec881e790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-13T03:13:32.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.121+0000 7f6cd4b9b700 1 --2- 172.21.15.64:0/1094432999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ec8806180 0x563ec881e790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.121+0000 7f6cd4b9b700 1 -- 172.21.15.64:0/1094432999 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ec8584de0 con 0x563ec8805860 2023-09-13T03:13:32.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.121+0000 7f6cd4b9b700 1 --2- 172.21.15.64:0/1094432999 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ec8805860 0x563ec8820350 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4099540 tx=0x7f6cc4005840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f81ea5a5e7a3a4a1 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:32.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.121+0000 7f6caffff700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:13:32.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.121+0000 7f6caffff700 1 -- 172.21.15.64:0/1094432999 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc407c070 con 0x563ec8805860 2023-09-13T03:13:32.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.121+0000 7f6cea778b80 1 -- 172.21.15.64:0/1094432999 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6cc8007d50 con 0x563ec8805860 2023-09-13T03:13:32.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.121+0000 7f6caffff700 1 -- 172.21.15.64:0/1094432999 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f6cc40a1620 con 0x563ec8805860 2023-09-13T03:13:32.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.122+0000 7f6caffff700 1 -- 172.21.15.64:0/1094432999 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6cc40c9840 con 0x563ec8805860 2023-09-13T03:13:32.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.122+0000 7f6cea778b80 10 client.25715 fetch_fsmap learned FSMap version 26 2023-09-13T03:13:32.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.122+0000 7f6cea778b80 1 -- 172.21.15.64:0/1094432999 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x563ec85ab250 con 0x563ec8805860 2023-09-13T03:13:32.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.122+0000 7f6caffff700 1 -- 172.21.15.64:0/1094432999 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cc40a5080 con 0x563ec8805860 2023-09-13T03:13:32.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.122+0000 7f6caffff700 1 -- 172.21.15.64:0/1094432999 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6cc407f070 con 0x563ec8805860 2023-09-13T03:13:32.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.122+0000 7f6cea778b80 10 client.25715 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:13:32.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.122+0000 7f6cea778b80 10 client.25715 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-09-13T03:13:32.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.122+0000 7f6cea778b80 20 client.25715 populate_metadata read hostname 'smithi064' 2023-09-13T03:13:32.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.122+0000 7f6cea778b80 1 --2- 172.21.15.64:0/1094432999 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x563ec887e270 0x563ec8880660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:32.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.122+0000 7f6cea778b80 4 client.25715 mds_command: new command op to 24649 tid=0[{"prefix": "damage ls"}] 2023-09-13T03:13:32.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.122+0000 7f6cea778b80 1 -- 172.21.15.64:0/1094432999 --> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563ec8751920 con 0x563ec887e270 2023-09-13T03:13:32.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.123+0000 7f6ccffff700 1 --2- 172.21.15.64:0/1094432999 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x563ec887e270 0x563ec8880660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:32.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.123+0000 7f6caffff700 10 client.25715 ms_handle_connect on v2:172.21.15.170:6838/2189703596 2023-09-13T03:13:32.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.123+0000 7f6ccffff700 1 --2- 172.21.15.64:0/1094432999 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x563ec887e270 0x563ec8880660 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=33c5054cc8aaa7a9 server_cookie=e7d6a2b6d9522c32 in_seq=0 out_seq=0 2023-09-13T03:13:32.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.123+0000 7f6caffff700 1 -- 172.21.15.64:0/1094432999 <== mds.1 v2:172.21.15.170:6838/2189703596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563ec8751920 con 0x563ec887e270 2023-09-13T03:13:32.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.123+0000 7f6caffff700 10 client.25715 handle_command_reply: tid=0 2023-09-13T03:13:32.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.123+0000 7f6cea778b80 1 client.25715 shutdown 2023-09-13T03:13:32.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.124+0000 7f6cea778b80 1 -- 172.21.15.64:0/1094432999 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ec8805860 msgr2=0x563ec8820350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:32.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.124+0000 7f6cea778b80 1 --2- 172.21.15.64:0/1094432999 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ec8805860 0x563ec8820350 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4099540 tx=0x7f6cc4005840 comp rx=0 tx=0).stop 2023-09-13T03:13:32.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.124+0000 7f6cea778b80 1 -- 172.21.15.64:0/1094432999 shutdown_connections 2023-09-13T03:13:32.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.124+0000 7f6cea778b80 1 --2- 172.21.15.64:0/1094432999 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x563ec887e270 0x563ec8880660 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.124+0000 7f6cea778b80 1 --2- 172.21.15.64:0/1094432999 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ec8805860 0x563ec8820350 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.124+0000 7f6cea778b80 1 --2- 172.21.15.64:0/1094432999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ec8819840 0x563ec881ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.124+0000 7f6caffff700 0 client.25715 ms_handle_reset on v2:172.21.15.170:6838/2189703596 2023-09-13T03:13:32.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.124+0000 7f6cea778b80 1 --2- 172.21.15.64:0/1094432999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ec8806180 0x563ec881e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.124+0000 7f6cea778b80 1 -- 172.21.15.64:0/1094432999 >> 172.21.15.64:0/1094432999 conn(0x563ec877b090 msgr2=0x563ec8816560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:32.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.124+0000 7f6cea778b80 1 -- 172.21.15.64:0/1094432999 shutdown_connections 2023-09-13T03:13:32.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.124+0000 7f6cea778b80 1 -- 172.21.15.64:0/1094432999 wait complete. 2023-09-13T03:13:32.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.124+0000 7f6cea778b80 20 client.25715 trim_cache size 0 max 16384 2023-09-13T03:13:32.137 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2023-09-13T03:13:32.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.125+0000 7f6cd5b9d700 1 -- 172.21.15.64:0/232289335 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f6cc00a0960 msgr2=0x7f6cc00a2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:32.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.125+0000 7f6cd5b9d700 1 --2- 172.21.15.64:0/232289335 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f6cc00a0960 0x7f6cc00a2e20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0199d20 tx=0x7f6cbc009310 comp rx=0 tx=0).stop 2023-09-13T03:13:32.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.125+0000 7f6cd5b9d700 1 -- 172.21.15.64:0/232289335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cd0067760 msgr2=0x7f6cd0198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:32.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.125+0000 7f6cd5b9d700 1 --2- 172.21.15.64:0/232289335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cd0067760 0x7f6cd0198710 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6cc400d980 tx=0x7f6cc400dd40 comp rx=0 tx=0).stop 2023-09-13T03:13:32.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.125+0000 7f6cd5b9d700 1 -- 172.21.15.64:0/232289335 shutdown_connections 2023-09-13T03:13:32.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.125+0000 7f6cd5b9d700 1 --2- 172.21.15.64:0/232289335 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f6cc00a0960 0x7f6cc00a2e20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.125+0000 7f6cd5b9d700 1 --2- 172.21.15.64:0/232289335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cd0067760 0x7f6cd0198710 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.125+0000 7f6cd5b9d700 1 --2- 172.21.15.64:0/232289335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cd01053c0 0x7f6cd019c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.125+0000 7f6cd5b9d700 1 --2- 172.21.15.64:0/232289335 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6cd0068100 0x7f6cd0198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.125+0000 7f6cd5b9d700 1 -- 172.21.15.64:0/232289335 >> 172.21.15.64:0/232289335 conn(0x7f6cd0074dc0 msgr2=0x7f6cd010b610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:32.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.125+0000 7f6cd5b9d700 1 -- 172.21.15.64:0/232289335 shutdown_connections 2023-09-13T03:13:32.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.125+0000 7f6cd5b9d700 1 -- 172.21.15.64:0/232289335 wait complete. 2023-09-13T03:13:32.146 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-09-13T03:13:32.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.366+0000 7fcca45ff700 1 -- 172.21.15.64:0/4277308120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc9c10ab30 msgr2=0x7fcc9c10cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:32.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.366+0000 7fcca45ff700 1 --2- 172.21.15.64:0/4277308120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc9c10ab30 0x7fcc9c10cff0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fcc94009f90 tx=0x7fcc9400b330 comp rx=0 tx=0).stop 2023-09-13T03:13:32.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.366+0000 7fcca45ff700 1 -- 172.21.15.64:0/4277308120 shutdown_connections 2023-09-13T03:13:32.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.366+0000 7fcca45ff700 1 --2- 172.21.15.64:0/4277308120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc9c10ab30 0x7fcc9c10cff0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.366+0000 7fcca45ff700 1 --2- 172.21.15.64:0/4277308120 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc9c101710 0x7fcc9c101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.366+0000 7fcca45ff700 1 --2- 172.21.15.64:0/4277308120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc9c102030 0x7fcc9c10a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.366+0000 7fcca45ff700 1 -- 172.21.15.64:0/4277308120 >> 172.21.15.64:0/4277308120 conn(0x7fcc9c0faf50 msgr2=0x7fcc9c0fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:32.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.366+0000 7fcca45ff700 1 -- 172.21.15.64:0/4277308120 shutdown_connections 2023-09-13T03:13:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.366+0000 7fcca45ff700 1 -- 172.21.15.64:0/4277308120 wait complete. 2023-09-13T03:13:32.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.367+0000 7fcca45ff700 1 Processor -- start 2023-09-13T03:13:32.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.368+0000 7fcca45ff700 1 -- start start 2023-09-13T03:13:32.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.368+0000 7fcca45ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc9c101710 0x7fcc9c0ff360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:32.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.368+0000 7fcca45ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc9c102030 0x7fcc9c0ff8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:32.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.368+0000 7fcca45ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc9c10ab30 0x7fcc9c1a4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:32.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.368+0000 7fcca45ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9c1000d0 con 0x7fcc9c10ab30 2023-09-13T03:13:32.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.368+0000 7fcca45ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9c1a4ed0 con 0x7fcc9c102030 2023-09-13T03:13:32.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.368+0000 7fcca45ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9c1a5010 con 0x7fcc9c101710 2023-09-13T03:13:32.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.368+0000 7fcca2b9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc9c10ab30 0x7fcc9c1a4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:32.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.369+0000 7fcca1b9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc9c102030 0x7fcc9c0ff8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:32.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.369+0000 7fcca1b9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc9c102030 0x7fcc9c0ff8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:58058/0 (socket says 172.21.15.64:58058) 2023-09-13T03:13:32.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.369+0000 7fcca1b9a700 1 -- 172.21.15.64:0/2400050403 learned_addr learned my addr 172.21.15.64:0/2400050403 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:32.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.369+0000 7fcca1b9a700 1 -- 172.21.15.64:0/2400050403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc9c101710 msgr2=0x7fcc9c0ff360 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-13T03:13:32.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.369+0000 7fcca1b9a700 1 --2- 172.21.15.64:0/2400050403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc9c101710 0x7fcc9c0ff360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.369+0000 7fcca1b9a700 1 -- 172.21.15.64:0/2400050403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc9c10ab30 msgr2=0x7fcc9c1a4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:32.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.369+0000 7fcca1b9a700 1 --2- 172.21.15.64:0/2400050403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc9c10ab30 0x7fcc9c1a4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.369+0000 7fcca1b9a700 1 -- 172.21.15.64:0/2400050403 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc94009c40 con 0x7fcc9c102030 2023-09-13T03:13:32.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.369+0000 7fcca1b9a700 1 --2- 172.21.15.64:0/2400050403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc9c102030 0x7fcc9c0ff8a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00d980 tx=0x7fcc8c00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:32.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.370+0000 7fcc937fe700 1 -- 172.21.15.64:0/2400050403 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c009930 con 0x7fcc9c102030 2023-09-13T03:13:32.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.370+0000 7fcca45ff700 1 -- 172.21.15.64:0/2400050403 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc9c1a5270 con 0x7fcc9c102030 2023-09-13T03:13:32.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.370+0000 7fcc937fe700 1 -- 172.21.15.64:0/2400050403 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc8c010460 con 0x7fcc9c102030 2023-09-13T03:13:32.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.370+0000 7fcc937fe700 1 -- 172.21.15.64:0/2400050403 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c00f620 con 0x7fcc9c102030 2023-09-13T03:13:32.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.370+0000 7fcca45ff700 1 -- 172.21.15.64:0/2400050403 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc9c1a57a0 con 0x7fcc9c102030 2023-09-13T03:13:32.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.375+0000 7fcc937fe700 1 -- 172.21.15.64:0/2400050403 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) v1 ==== 135435+0+0 (secure 0 0 0) 0x7fcc8c0105d0 con 0x7fcc9c102030 2023-09-13T03:13:32.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.377+0000 7fccb7f89b80 1 Processor -- start 2023-09-13T03:13:32.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.377+0000 7fcc937fe700 1 --2- 172.21.15.64:0/2400050403 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fcc880a0940 0x7fcc880a2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:32.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.378+0000 7fcc937fe700 1 -- 172.21.15.64:0/2400050403 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fcc8c0cba30 con 0x7fcc9c102030 2023-09-13T03:13:32.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.378+0000 7fcca2b9c700 1 --2- 172.21.15.64:0/2400050403 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fcc880a0940 0x7fcc880a2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:32.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.378+0000 7fccb7f89b80 1 -- start start 2023-09-13T03:13:32.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.379+0000 7fccb7f89b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91573e710 0x55f91573eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:32.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.379+0000 7fccb7f89b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f91573f030 0x55f915751110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:32.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.379+0000 7fccb7f89b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f915751650 0x55f915753a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:32.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.379+0000 7fccb7f89b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f915697730 con 0x55f915751650 2023-09-13T03:13:32.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.379+0000 7fccb7f89b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f915695050 con 0x55f91573e710 2023-09-13T03:13:32.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.379+0000 7fccb7f89b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f915690940 con 0x55f91573f030 2023-09-13T03:13:32.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.379+0000 7fcca239b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91573e710 0x55f91573eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:32.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.379+0000 7fcca1b9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f91573f030 0x55f915751110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:32.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.379+0000 7fcca239b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91573e710 0x55f91573eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:58066/0 (socket says 172.21.15.64:58066) 2023-09-13T03:13:32.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.379+0000 7fcca239b700 1 -- 172.21.15.64:0/3942129666 learned_addr learned my addr 172.21.15.64:0/3942129666 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:32.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.379+0000 7fcca2b9c700 1 --2- 172.21.15.64:0/3942129666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f915751650 0x55f915753a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:32.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.379+0000 7fcca239b700 1 -- 172.21.15.64:0/3942129666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f91573f030 msgr2=0x55f915751110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:32.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.379+0000 7fcca239b700 1 --2- 172.21.15.64:0/3942129666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f91573f030 0x55f915751110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.380+0000 7fcca239b700 1 -- 172.21.15.64:0/3942129666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f915751650 msgr2=0x55f915753a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:32.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.380+0000 7fcca239b700 1 --2- 172.21.15.64:0/3942129666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f915751650 0x55f915753a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.380+0000 7fcca239b700 1 -- 172.21.15.64:0/3942129666 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9154bcde0 con 0x55f91573e710 2023-09-13T03:13:32.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.380+0000 7fcca1b9a700 1 --2- 172.21.15.64:0/3942129666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f91573f030 0x55f915751110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:13:32.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.380+0000 7fcca239b700 1 --2- 172.21.15.64:0/3942129666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91573e710 0x55f91573eaf0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcc9800aa10 tx=0x7fcc9800ad20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eea71a9f7ef1178a server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:32.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.380+0000 7fcc90bfe700 1 -- 172.21.15.64:0/3942129666 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9800cc10 con 0x55f91573e710 2023-09-13T03:13:32.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.380+0000 7fcc90bfe700 1 -- 172.21.15.64:0/3942129666 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc9800cd70 con 0x55f91573e710 2023-09-13T03:13:32.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.380+0000 7fcc90bfe700 1 -- 172.21.15.64:0/3942129666 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc98018690 con 0x55f91573e710 2023-09-13T03:13:32.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.381+0000 7fccb7f89b80 1 -- 172.21.15.64:0/3942129666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91573e710 msgr2=0x55f91573eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:32.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.381+0000 7fccb7f89b80 1 --2- 172.21.15.64:0/3942129666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91573e710 0x55f91573eaf0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcc9800aa10 tx=0x7fcc9800ad20 comp rx=0 tx=0).stop 2023-09-13T03:13:32.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.381+0000 7fccb7f89b80 1 -- 172.21.15.64:0/3942129666 shutdown_connections 2023-09-13T03:13:32.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.381+0000 7fccb7f89b80 1 --2- 172.21.15.64:0/3942129666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91573e710 0x55f91573eaf0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.381+0000 7fccb7f89b80 1 --2- 172.21.15.64:0/3942129666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f915751650 0x55f915753a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.381+0000 7fccb7f89b80 1 --2- 172.21.15.64:0/3942129666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f91573f030 0x55f915751110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.381+0000 7fccb7f89b80 1 -- 172.21.15.64:0/3942129666 >> 172.21.15.64:0/3942129666 conn(0x55f9156b3090 msgr2=0x55f915739bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:32.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.381+0000 7fccb7f89b80 1 -- 172.21.15.64:0/3942129666 shutdown_connections 2023-09-13T03:13:32.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.381+0000 7fcca2b9c700 1 --2- 172.21.15.64:0/2400050403 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fcc880a0940 0x7fcc880a2e00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c101470 tx=0x7fcc9400bb80 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:32.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.381+0000 7fccb7f89b80 1 -- 172.21.15.64:0/3942129666 wait complete. 2023-09-13T03:13:32.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.382+0000 7fccb7f89b80 1 -- start start 2023-09-13T03:13:32.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.382+0000 7fccb7f89b80 1 Processor -- start 2023-09-13T03:13:32.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.383+0000 7fccb7f89b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91573e710 0x55f915748360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:32.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.383+0000 7fccb7f89b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f91573f030 0x55f9157488a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:32.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.383+0000 7fccb7f89b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f915751650 0x55f915748de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:32.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.383+0000 7fccb7f89b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91562eb70 con 0x55f915751650 2023-09-13T03:13:32.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.383+0000 7fccb7f89b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91562e870 con 0x55f91573e710 2023-09-13T03:13:32.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.383+0000 7fccb7f89b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f915512920 con 0x55f91573f030 2023-09-13T03:13:32.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.383+0000 7fccb7f89b80 10 client.0 _ll_register_callbacks cb 0x55f915578a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:13:32.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.383+0000 7fcca2b9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f915751650 0x55f915748de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:32.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.383+0000 7fcca2b9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f915751650 0x55f915748de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39102/0 (socket says 172.21.15.64:39102) 2023-09-13T03:13:32.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.383+0000 7fcca2b9c700 1 -- 172.21.15.64:0/789599990 learned_addr learned my addr 172.21.15.64:0/789599990 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:32.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.384+0000 7fcca2b9c700 1 -- 172.21.15.64:0/789599990 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f91573f030 msgr2=0x55f9157488a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-13T03:13:32.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.384+0000 7fcca2b9c700 1 --2- 172.21.15.64:0/789599990 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f91573f030 0x55f9157488a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.384+0000 7fcca2b9c700 1 -- 172.21.15.64:0/789599990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91573e710 msgr2=0x55f915748360 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-13T03:13:32.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.384+0000 7fcca2b9c700 1 --2- 172.21.15.64:0/789599990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91573e710 0x55f915748360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.384+0000 7fcca2b9c700 1 -- 172.21.15.64:0/789599990 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc9c1a57a0 con 0x55f915751650 2023-09-13T03:13:32.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.384+0000 7fcca2b9c700 1 --2- 172.21.15.64:0/789599990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f915751650 0x55f915748de0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fcc94007c30 tx=0x7fcc9400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2adcaedf51a3ab server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:32.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.384+0000 7fcc85ffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:13:32.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.385+0000 7fcc85ffb700 1 -- 172.21.15.64:0/789599990 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9400e040 con 0x55f915751650 2023-09-13T03:13:32.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.385+0000 7fccb7f89b80 1 -- 172.21.15.64:0/789599990 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f9155e84f0 con 0x55f915751650 2023-09-13T03:13:32.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.385+0000 7fcc85ffb700 1 -- 172.21.15.64:0/789599990 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fcc94005860 con 0x55f915751650 2023-09-13T03:13:32.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.385+0000 7fcc85ffb700 1 -- 172.21.15.64:0/789599990 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc94009ac0 con 0x55f915751650 2023-09-13T03:13:32.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.385+0000 7fccb7f89b80 10 client.15693 fetch_fsmap learned FSMap version 26 2023-09-13T03:13:32.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.385+0000 7fccb7f89b80 1 -- 172.21.15.64:0/789599990 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7fcc9800a640 con 0x55f915751650 2023-09-13T03:13:32.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.386+0000 7fcc85ffb700 1 -- 172.21.15.64:0/789599990 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc94038020 con 0x55f915751650 2023-09-13T03:13:32.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.386+0000 7fcc85ffb700 1 -- 172.21.15.64:0/789599990 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fcc9402a460 con 0x55f915751650 2023-09-13T03:13:32.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.386+0000 7fccb7f89b80 10 client.15693 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:13:32.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.386+0000 7fccb7f89b80 10 client.15693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T03:13:32.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.386+0000 7fccb7f89b80 20 client.15693 populate_metadata read hostname 'smithi064' 2023-09-13T03:13:32.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.386+0000 7fccb7f89b80 1 --2- 172.21.15.64:0/789599990 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f915758a00 0x55f91575ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:32.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.386+0000 7fccb7f89b80 4 client.15693 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:13:32.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.386+0000 7fccb7f89b80 1 -- 172.21.15.64:0/789599990 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f9156a58a0 con 0x55f915758a00 2023-09-13T03:13:32.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.386+0000 7fcca239b700 1 --2- 172.21.15.64:0/789599990 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f915758a00 0x55f91575ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:32.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.387+0000 7fcca239b700 1 --2- 172.21.15.64:0/789599990 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f915758a00 0x55f91575ae60 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91a1f30b783184c0 server_cookie=87a7c84111fb7c7b in_seq=0 out_seq=0 2023-09-13T03:13:32.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.387+0000 7fcc85ffb700 10 client.15693 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:13:32.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.388+0000 7fcc85ffb700 1 -- 172.21.15.64:0/789599990 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f9156a58a0 con 0x55f915758a00 2023-09-13T03:13:32.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.388+0000 7fcc85ffb700 10 client.15693 handle_command_reply: tid=0 2023-09-13T03:13:32.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.388+0000 7fccb7f89b80 1 client.15693 shutdown 2023-09-13T03:13:32.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.388+0000 7fccb7f89b80 1 -- 172.21.15.64:0/789599990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f915751650 msgr2=0x55f915748de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:32.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.388+0000 7fccb7f89b80 1 --2- 172.21.15.64:0/789599990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f915751650 0x55f915748de0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fcc94007c30 tx=0x7fcc9400b330 comp rx=0 tx=0).stop 2023-09-13T03:13:32.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.388+0000 7fccb7f89b80 1 -- 172.21.15.64:0/789599990 shutdown_connections 2023-09-13T03:13:32.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.388+0000 7fccb7f89b80 1 --2- 172.21.15.64:0/789599990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91573e710 0x55f915748360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.388+0000 7fccb7f89b80 1 --2- 172.21.15.64:0/789599990 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f915758a00 0x55f91575ae60 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.388+0000 7fccb7f89b80 1 --2- 172.21.15.64:0/789599990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f915751650 0x55f915748de0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.388+0000 7fccb7f89b80 1 --2- 172.21.15.64:0/789599990 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f91573f030 0x55f9157488a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.388+0000 7fccb7f89b80 1 -- 172.21.15.64:0/789599990 >> 172.21.15.64:0/789599990 conn(0x55f9156b3090 msgr2=0x55f9157397e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:32.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.388+0000 7fcc85ffb700 0 client.15693 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:13:32.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.389+0000 7fccb7f89b80 1 -- 172.21.15.64:0/789599990 shutdown_connections 2023-09-13T03:13:32.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.389+0000 7fccb7f89b80 1 -- 172.21.15.64:0/789599990 wait complete. 2023-09-13T03:13:32.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.389+0000 7fccb7f89b80 20 client.15693 trim_cache size 0 max 16384 2023-09-13T03:13:32.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.403+0000 7fccb7f89b80 1 Processor -- start 2023-09-13T03:13:32.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.403+0000 7fccb7f89b80 1 -- start start 2023-09-13T03:13:32.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.403+0000 7fccb7f89b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f91573e710 0x55f915750c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:32.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.403+0000 7fccb7f89b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91573f030 0x55f915768680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:32.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.403+0000 7fccb7f89b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f915751650 0x55f915747b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:32.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.403+0000 7fccb7f89b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9157363d0 con 0x55f91573e710 2023-09-13T03:13:32.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.403+0000 7fccb7f89b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91573ae60 con 0x55f91573f030 2023-09-13T03:13:32.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.403+0000 7fccb7f89b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f915743190 con 0x55f915751650 2023-09-13T03:13:32.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.404+0000 7fcca239b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f91573e710 0x55f915750c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:32.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.404+0000 7fcca1b9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91573f030 0x55f915768680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:32.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.404+0000 7fcca1b9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91573f030 0x55f915768680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:58084/0 (socket says 172.21.15.64:58084) 2023-09-13T03:13:32.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.404+0000 7fcca239b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f91573e710 0x55f915750c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39108/0 (socket says 172.21.15.64:39108) 2023-09-13T03:13:32.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.404+0000 7fcca1b9a700 1 -- 172.21.15.64:0/2825042835 learned_addr learned my addr 172.21.15.64:0/2825042835 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:32.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.404+0000 7fcca2b9c700 1 --2- 172.21.15.64:0/2825042835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f915751650 0x55f915747b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:32.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.404+0000 7fcca1b9a700 1 -- 172.21.15.64:0/2825042835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f915751650 msgr2=0x55f915747b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:32.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.404+0000 7fcca1b9a700 1 --2- 172.21.15.64:0/2825042835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f915751650 0x55f915747b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.404+0000 7fcca1b9a700 1 -- 172.21.15.64:0/2825042835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f91573e710 msgr2=0x55f915750c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:32.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.404+0000 7fcca1b9a700 1 --2- 172.21.15.64:0/2825042835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f91573e710 0x55f915750c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.404+0000 7fcca1b9a700 1 -- 172.21.15.64:0/2825042835 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9154a5c60 con 0x55f91573f030 2023-09-13T03:13:32.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.404+0000 7fcca2b9c700 1 --2- 172.21.15.64:0/2825042835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f915751650 0x55f915747b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T03:13:32.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.404+0000 7fcca1b9a700 1 --2- 172.21.15.64:0/2825042835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91573f030 0x55f915768680 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00bfd0 tx=0x7fcc8c00b4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5b0596742bc95fa server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:32.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.405+0000 7fcc913ff700 1 -- 172.21.15.64:0/2825042835 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c0986c0 con 0x55f91573f030 2023-09-13T03:13:32.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.405+0000 7fcc913ff700 1 -- 172.21.15.64:0/2825042835 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc8c099460 con 0x55f91573f030 2023-09-13T03:13:32.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.405+0000 7fcc913ff700 1 -- 172.21.15.64:0/2825042835 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c0984b0 con 0x55f91573f030 2023-09-13T03:13:32.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.406+0000 7fccb7f89b80 1 -- 172.21.15.64:0/2825042835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91573f030 msgr2=0x55f915768680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:32.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.406+0000 7fccb7f89b80 1 --2- 172.21.15.64:0/2825042835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91573f030 0x55f915768680 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00bfd0 tx=0x7fcc8c00b4e0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.406+0000 7fccb7f89b80 1 -- 172.21.15.64:0/2825042835 shutdown_connections 2023-09-13T03:13:32.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.406+0000 7fccb7f89b80 1 --2- 172.21.15.64:0/2825042835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f915751650 0x55f915747b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.406+0000 7fccb7f89b80 1 --2- 172.21.15.64:0/2825042835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f91573e710 0x55f915750c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.406+0000 7fccb7f89b80 1 --2- 172.21.15.64:0/2825042835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91573f030 0x55f915768680 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.406+0000 7fccb7f89b80 1 -- 172.21.15.64:0/2825042835 >> 172.21.15.64:0/2825042835 conn(0x55f9156b3090 msgr2=0x55f91573d700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:32.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.406+0000 7fccb7f89b80 1 -- 172.21.15.64:0/2825042835 shutdown_connections 2023-09-13T03:13:32.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.406+0000 7fccb7f89b80 1 -- 172.21.15.64:0/2825042835 wait complete. 2023-09-13T03:13:32.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.406+0000 7fccb7f89b80 1 -- start start 2023-09-13T03:13:32.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.407+0000 7fccb7f89b80 1 Processor -- start 2023-09-13T03:13:32.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.407+0000 7fccb7f89b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f91573e710 0x55f915764850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:32.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.407+0000 7fccb7f89b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91573f030 0x55f915764d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:32.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.407+0000 7fccb7f89b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f915751650 0x55f9157abfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:32.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.407+0000 7fccb7f89b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9157654a0 con 0x55f91573e710 2023-09-13T03:13:32.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.407+0000 7fccb7f89b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9157ac520 con 0x55f91573f030 2023-09-13T03:13:32.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.407+0000 7fccb7f89b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9157ac660 con 0x55f915751650 2023-09-13T03:13:32.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.407+0000 7fccb7f89b80 10 client.0 _ll_register_callbacks cb 0x7fcc94003d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:13:32.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.407+0000 7fcca239b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f91573e710 0x55f915764850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:32.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.408+0000 7fcca239b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f91573e710 0x55f915764850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39110/0 (socket says 172.21.15.64:39110) 2023-09-13T03:13:32.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.408+0000 7fcca239b700 1 -- 172.21.15.64:0/2193253307 learned_addr learned my addr 172.21.15.64:0/2193253307 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:32.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.408+0000 7fcca1b9a700 1 --2- 172.21.15.64:0/2193253307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91573f030 0x55f915764d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:32.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.408+0000 7fcca239b700 1 -- 172.21.15.64:0/2193253307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f915751650 msgr2=0x55f9157abfe0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-13T03:13:32.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.408+0000 7fcca239b700 1 --2- 172.21.15.64:0/2193253307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f915751650 0x55f9157abfe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.408+0000 7fcca239b700 1 -- 172.21.15.64:0/2193253307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91573f030 msgr2=0x55f915764d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:32.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.408+0000 7fcca239b700 1 --2- 172.21.15.64:0/2193253307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91573f030 0x55f915764d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.408+0000 7fcca239b700 1 -- 172.21.15.64:0/2193253307 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc9c1a5270 con 0x55f91573e710 2023-09-13T03:13:32.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.408+0000 7fcca1b9a700 1 --2- 172.21.15.64:0/2193253307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91573f030 0x55f915764d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:13:32.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.408+0000 7fcca239b700 1 --2- 172.21.15.64:0/2193253307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f91573e710 0x55f915764850 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcc9800aa10 tx=0x7fcc9800cfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e786e044f5abbe server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:32.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.409+0000 7fcc85ffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:13:32.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.409+0000 7fcc85ffb700 1 -- 172.21.15.64:0/2193253307 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9800cbc0 con 0x55f91573e710 2023-09-13T03:13:32.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.409+0000 7fcc85ffb700 1 -- 172.21.15.64:0/2193253307 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fcc98004020 con 0x55f91573e710 2023-09-13T03:13:32.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.409+0000 7fccb7f89b80 1 -- 172.21.15.64:0/2193253307 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcc9800cd70 con 0x55f91573e710 2023-09-13T03:13:32.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.409+0000 7fcc85ffb700 1 -- 172.21.15.64:0/2193253307 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc980041e0 con 0x55f91573e710 2023-09-13T03:13:32.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.409+0000 7fccb7f89b80 10 client.15699 fetch_fsmap learned FSMap version 26 2023-09-13T03:13:32.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.409+0000 7fccb7f89b80 1 -- 172.21.15.64:0/2193253307 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55f9154e3250 con 0x55f91573e710 2023-09-13T03:13:32.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.410+0000 7fcc85ffb700 1 -- 172.21.15.64:0/2193253307 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc9800cd70 con 0x55f91573e710 2023-09-13T03:13:32.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.410+0000 7fcc85ffb700 1 -- 172.21.15.64:0/2193253307 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fcc9804e030 con 0x55f91573e710 2023-09-13T03:13:32.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.410+0000 7fccb7f89b80 10 client.15699 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:13:32.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.410+0000 7fccb7f89b80 10 client.15699 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T03:13:32.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.410+0000 7fccb7f89b80 20 client.15699 populate_metadata read hostname 'smithi064' 2023-09-13T03:13:32.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.410+0000 7fccb7f89b80 1 --2- 172.21.15.64:0/2193253307 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f9157adaf0 0x55f9157afee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:32.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.410+0000 7fccb7f89b80 4 client.15699 mds_command: new command op to 24686 tid=0[{"prefix": "damage ls"}] 2023-09-13T03:13:32.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.410+0000 7fccb7f89b80 1 -- 172.21.15.64:0/2193253307 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f915689920 con 0x55f9157adaf0 2023-09-13T03:13:32.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.410+0000 7fcca1b9a700 1 --2- 172.21.15.64:0/2193253307 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f9157adaf0 0x55f9157afee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:32.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.411+0000 7fcca1b9a700 1 --2- 172.21.15.64:0/2193253307 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f9157adaf0 0x55f9157afee0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a038b5575532a1af server_cookie=d15f3a1940453f20 in_seq=0 out_seq=0 2023-09-13T03:13:32.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.411+0000 7fcc85ffb700 10 client.15699 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:13:32.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.411+0000 7fcc85ffb700 1 -- 172.21.15.64:0/2193253307 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f915689920 con 0x55f9157adaf0 2023-09-13T03:13:32.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.411+0000 7fcc85ffb700 10 client.15699 handle_command_reply: tid=0 2023-09-13T03:13:32.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.411+0000 7fccb7f89b80 1 client.15699 shutdown 2023-09-13T03:13:32.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.411+0000 7fccb7f89b80 1 -- 172.21.15.64:0/2193253307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f91573e710 msgr2=0x55f915764850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:32.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.411+0000 7fccb7f89b80 1 --2- 172.21.15.64:0/2193253307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f91573e710 0x55f915764850 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcc9800aa10 tx=0x7fcc9800cfd0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.411+0000 7fccb7f89b80 1 -- 172.21.15.64:0/2193253307 shutdown_connections 2023-09-13T03:13:32.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.411+0000 7fccb7f89b80 1 --2- 172.21.15.64:0/2193253307 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f9157adaf0 0x55f9157afee0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.411+0000 7fccb7f89b80 1 --2- 172.21.15.64:0/2193253307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f915751650 0x55f9157abfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.411+0000 7fccb7f89b80 1 --2- 172.21.15.64:0/2193253307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f91573e710 0x55f915764850 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.411+0000 7fccb7f89b80 1 --2- 172.21.15.64:0/2193253307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f91573f030 0x55f915764d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.411+0000 7fccb7f89b80 1 -- 172.21.15.64:0/2193253307 >> 172.21.15.64:0/2193253307 conn(0x55f9156b3090 msgr2=0x55f91573df90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:32.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.412+0000 7fcc85ffb700 0 client.15699 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:13:32.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.412+0000 7fccb7f89b80 1 -- 172.21.15.64:0/2193253307 shutdown_connections 2023-09-13T03:13:32.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.412+0000 7fccb7f89b80 1 -- 172.21.15.64:0/2193253307 wait complete. 2023-09-13T03:13:32.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.412+0000 7fccb7f89b80 20 client.15699 trim_cache size 0 max 16384 2023-09-13T03:13:32.426 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2023-09-13T03:13:32.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.413+0000 7fcca45ff700 1 -- 172.21.15.64:0/2400050403 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fcc880a0940 msgr2=0x7fcc880a2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:32.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.413+0000 7fcca45ff700 1 --2- 172.21.15.64:0/2400050403 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fcc880a0940 0x7fcc880a2e00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c101470 tx=0x7fcc9400bb80 comp rx=0 tx=0).stop 2023-09-13T03:13:32.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.413+0000 7fcca45ff700 1 -- 172.21.15.64:0/2400050403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc9c102030 msgr2=0x7fcc9c0ff8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:32.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.413+0000 7fcca45ff700 1 --2- 172.21.15.64:0/2400050403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc9c102030 0x7fcc9c0ff8a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00d980 tx=0x7fcc8c00dc90 comp rx=0 tx=0).stop 2023-09-13T03:13:32.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.413+0000 7fcca45ff700 1 -- 172.21.15.64:0/2400050403 shutdown_connections 2023-09-13T03:13:32.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.413+0000 7fcca45ff700 1 --2- 172.21.15.64:0/2400050403 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fcc880a0940 0x7fcc880a2e00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.413+0000 7fcca45ff700 1 --2- 172.21.15.64:0/2400050403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc9c10ab30 0x7fcc9c1a4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.413+0000 7fcca45ff700 1 --2- 172.21.15.64:0/2400050403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc9c101710 0x7fcc9c0ff360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.413+0000 7fcca45ff700 1 --2- 172.21.15.64:0/2400050403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc9c102030 0x7fcc9c0ff8a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:32.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.413+0000 7fcca45ff700 1 -- 172.21.15.64:0/2400050403 >> 172.21.15.64:0/2400050403 conn(0x7fcc9c0faf50 msgr2=0x7fcc9c10cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:32.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.413+0000 7fcca45ff700 1 -- 172.21.15.64:0/2400050403 shutdown_connections 2023-09-13T03:13:32.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:32.413+0000 7fcca45ff700 1 -- 172.21.15.64:0/2400050403 wait complete. 2023-09-13T03:13:32.429 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:32 smithi064 ceph-mon[97675]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.6 KiB/s wr, 16 op/s 2023-09-13T03:13:32.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:32 smithi170 ceph-mon[134328]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.6 KiB/s wr, 16 op/s 2023-09-13T03:13:32.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:32 smithi142 ceph-mon[134082]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.6 KiB/s wr, 16 op/s 2023-09-13T03:13:33.435 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-09-13T03:13:33.435 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-09-13T03:13:33.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.787+0000 7f8025dff700 1 -- 172.21.15.64:0/3322360400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8020106630 msgr2=0x7f8020106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:33.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.787+0000 7f8025dff700 1 --2- 172.21.15.64:0/3322360400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8020106630 0x7f8020106a10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f80100099c0 tx=0x7f8010009cd0 comp rx=0 tx=0).stop 2023-09-13T03:13:33.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.787+0000 7f8025dff700 1 -- 172.21.15.64:0/3322360400 shutdown_connections 2023-09-13T03:13:33.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.787+0000 7f8025dff700 1 --2- 172.21.15.64:0/3322360400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8020106630 0x7f8020106a10 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:33.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.787+0000 7f8025dff700 1 --2- 172.21.15.64:0/3322360400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8020100f30 0x7f802010d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:33.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.787+0000 7f8025dff700 1 --2- 172.21.15.64:0/3322360400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8020100570 0x7f80201009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:33.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.787+0000 7f8025dff700 1 -- 172.21.15.64:0/3322360400 >> 172.21.15.64:0/3322360400 conn(0x7f80200fc080 msgr2=0x7f80200fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:33.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.787+0000 7f8025dff700 1 -- 172.21.15.64:0/3322360400 shutdown_connections 2023-09-13T03:13:33.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.787+0000 7f8025dff700 1 -- 172.21.15.64:0/3322360400 wait complete. 2023-09-13T03:13:33.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.789+0000 7f8025dff700 1 Processor -- start 2023-09-13T03:13:33.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.789+0000 7f8025dff700 1 -- start start 2023-09-13T03:13:33.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.789+0000 7f8025dff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8020100570 0x7f802019e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:33.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.789+0000 7f8025dff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8020100f30 0x7f802019ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:33.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.789+0000 7f8025dff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8020106630 0x7f80201986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:33.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.789+0000 7f8025dff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8020198d50 con 0x7f8020100f30 2023-09-13T03:13:33.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.789+0000 7f8025dff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8020198ec0 con 0x7f8020106630 2023-09-13T03:13:33.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.789+0000 7f8025dff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8020199080 con 0x7f8020100570 2023-09-13T03:13:33.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.790+0000 7f801effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8020100f30 0x7f802019ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:33.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.790+0000 7f801effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8020100f30 0x7f802019ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39126/0 (socket says 172.21.15.64:39126) 2023-09-13T03:13:33.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.790+0000 7f801effd700 1 -- 172.21.15.64:0/431176529 learned_addr learned my addr 172.21.15.64:0/431176529 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:33.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.790+0000 7f801f7fe700 1 --2- 172.21.15.64:0/431176529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8020100570 0x7f802019e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:33.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.790+0000 7f801ffff700 1 --2- 172.21.15.64:0/431176529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8020106630 0x7f80201986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:33.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.790+0000 7f801effd700 1 -- 172.21.15.64:0/431176529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8020100570 msgr2=0x7f802019e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:33.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.790+0000 7f801effd700 1 --2- 172.21.15.64:0/431176529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8020100570 0x7f802019e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:33.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.790+0000 7f801effd700 1 -- 172.21.15.64:0/431176529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8020106630 msgr2=0x7f80201986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:33.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.790+0000 7f801effd700 1 --2- 172.21.15.64:0/431176529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8020106630 0x7f80201986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:33.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.790+0000 7f801effd700 1 -- 172.21.15.64:0/431176529 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80100095f0 con 0x7f8020100f30 2023-09-13T03:13:33.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.790+0000 7f801ffff700 1 --2- 172.21.15.64:0/431176529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8020106630 0x7f80201986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:13:33.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.791+0000 7f801effd700 1 --2- 172.21.15.64:0/431176529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8020100f30 0x7f802019ebb0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f801400e9a0 tx=0x7f801400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:33.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.791+0000 7f801cff9700 1 -- 172.21.15.64:0/431176529 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8014009790 con 0x7f8020100f30 2023-09-13T03:13:33.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.791+0000 7f8025dff700 1 -- 172.21.15.64:0/431176529 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8020199360 con 0x7f8020100f30 2023-09-13T03:13:33.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.791+0000 7f801cff9700 1 -- 172.21.15.64:0/431176529 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f801400cbd0 con 0x7f8020100f30 2023-09-13T03:13:33.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.791+0000 7f801cff9700 1 -- 172.21.15.64:0/431176529 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8014018680 con 0x7f8020100f30 2023-09-13T03:13:33.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.792+0000 7f8025dff700 1 -- 172.21.15.64:0/431176529 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80201a5660 con 0x7f8020100f30 2023-09-13T03:13:33.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.797+0000 7f801cff9700 1 -- 172.21.15.64:0/431176529 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 24) v1 ==== 135435+0+0 (secure 0 0 0) 0x7f801400cd40 con 0x7f8020100f30 2023-09-13T03:13:33.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.798+0000 7f80397b4b80 1 Processor -- start 2023-09-13T03:13:33.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.799+0000 7f80397b4b80 1 -- start start 2023-09-13T03:13:33.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.799+0000 7f80397b4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ce3299790 0x563ce3299b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:33.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.799+0000 7f80397b4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ce329a0b0 0x563ce32a3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:33.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.799+0000 7f80397b4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ce32a3ec0 0x563ce32a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:33.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.799+0000 7f80397b4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ce31f27b0 con 0x563ce3299790 2023-09-13T03:13:33.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.799+0000 7f80397b4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ce31f00d0 con 0x563ce329a0b0 2023-09-13T03:13:33.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.799+0000 7f80397b4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ce31eb9c0 con 0x563ce32a3ec0 2023-09-13T03:13:33.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.800+0000 7f801ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ce3299790 0x563ce3299b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:33.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.800+0000 7f801ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ce3299790 0x563ce3299b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39140/0 (socket says 172.21.15.64:39140) 2023-09-13T03:13:33.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.800+0000 7f801ffff700 1 -- 172.21.15.64:0/586926079 learned_addr learned my addr 172.21.15.64:0/586926079 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:33.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.800+0000 7f801f7fe700 1 --2- 172.21.15.64:0/586926079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ce329a0b0 0x563ce32a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:33.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.800+0000 7f801f7fe700 1 -- 172.21.15.64:0/586926079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ce32a3ec0 msgr2=0x563ce32a62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:33.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.800+0000 7f801f7fe700 1 --2- 172.21.15.64:0/586926079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ce32a3ec0 0x563ce32a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:33.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.800+0000 7f801f7fe700 1 -- 172.21.15.64:0/586926079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ce3299790 msgr2=0x563ce3299b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:33.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.800+0000 7f801f7fe700 1 --2- 172.21.15.64:0/586926079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ce3299790 0x563ce3299b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:33.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.800+0000 7f801f7fe700 1 -- 172.21.15.64:0/586926079 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ce3017e60 con 0x563ce329a0b0 2023-09-13T03:13:33.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.800+0000 7f801ffff700 1 --2- 172.21.15.64:0/586926079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ce3299790 0x563ce3299b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:13:33.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.801+0000 7f801f7fe700 1 --2- 172.21.15.64:0/586926079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ce329a0b0 0x563ce32a3980 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f801000b580 tx=0x7f8010005c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5d413cf9574b6c6 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:33.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.801+0000 7f801cff9700 1 --2- 172.21.15.64:0/431176529 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f80080a0940 0x7f80080a2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:33.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.801+0000 7f8005ffb700 1 -- 172.21.15.64:0/586926079 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801001d030 con 0x563ce329a0b0 2023-09-13T03:13:33.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.801+0000 7f8005ffb700 1 -- 172.21.15.64:0/586926079 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f801001e040 con 0x563ce329a0b0 2023-09-13T03:13:33.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.801+0000 7f8005ffb700 1 -- 172.21.15.64:0/586926079 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801000ba80 con 0x563ce329a0b0 2023-09-13T03:13:33.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.801+0000 7f801cff9700 1 -- 172.21.15.64:0/431176529 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f8014014070 con 0x7f8020100f30 2023-09-13T03:13:33.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.801+0000 7f801ffff700 1 --2- 172.21.15.64:0/431176529 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f80080a0940 0x7f80080a2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:33.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.802+0000 7f80397b4b80 1 -- 172.21.15.64:0/586926079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ce329a0b0 msgr2=0x563ce32a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:33.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.802+0000 7f80397b4b80 1 --2- 172.21.15.64:0/586926079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ce329a0b0 0x563ce32a3980 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f801000b580 tx=0x7f8010005c40 comp rx=0 tx=0).stop 2023-09-13T03:13:33.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.802+0000 7f801ffff700 1 --2- 172.21.15.64:0/431176529 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f80080a0940 0x7f80080a2e00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8018009fd0 tx=0x7f801800b040 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:33.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.802+0000 7f80397b4b80 1 -- 172.21.15.64:0/586926079 shutdown_connections 2023-09-13T03:13:33.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.802+0000 7f80397b4b80 1 --2- 172.21.15.64:0/586926079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ce32a3ec0 0x563ce32a62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:33.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.802+0000 7f80397b4b80 1 --2- 172.21.15.64:0/586926079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ce3299790 0x563ce3299b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:33.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.802+0000 7f80397b4b80 1 --2- 172.21.15.64:0/586926079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ce329a0b0 0x563ce32a3980 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:33.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.802+0000 7f80397b4b80 1 -- 172.21.15.64:0/586926079 >> 172.21.15.64:0/586926079 conn(0x563ce320e110 msgr2=0x563ce3294c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:33.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.803+0000 7f80397b4b80 1 -- 172.21.15.64:0/586926079 shutdown_connections 2023-09-13T03:13:33.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.803+0000 7f80397b4b80 1 -- 172.21.15.64:0/586926079 wait complete. 2023-09-13T03:13:33.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.804+0000 7f80397b4b80 1 -- start start 2023-09-13T03:13:33.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.804+0000 7f80397b4b80 1 Processor -- start 2023-09-13T03:13:33.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.805+0000 7f80397b4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ce3299790 0x563ce32a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:33.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.805+0000 7f80397b4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ce329a0b0 0x563ce32a1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:33.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.805+0000 7f80397b4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ce32a3ec0 0x563ce32a1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:33.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.805+0000 7f80397b4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ce3189bf0 con 0x563ce32a3ec0 2023-09-13T03:13:33.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.805+0000 7f80397b4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ce31898f0 con 0x563ce3299790 2023-09-13T03:13:33.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.805+0000 7f80397b4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ce306d9a0 con 0x563ce329a0b0 2023-09-13T03:13:33.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.805+0000 7f80397b4b80 10 client.0 _ll_register_callbacks cb 0x563ce30d3b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:13:33.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.805+0000 7f801f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ce3299790 0x563ce32a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:33.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.805+0000 7f801f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ce3299790 0x563ce32a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:58134/0 (socket says 172.21.15.64:58134) 2023-09-13T03:13:33.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.805+0000 7f801f7fe700 1 -- 172.21.15.64:0/3159827432 learned_addr learned my addr 172.21.15.64:0/3159827432 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:33.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.805+0000 7f801ffff700 1 --2- 172.21.15.64:0/3159827432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ce32a3ec0 0x563ce32a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:33.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.805+0000 7f801effd700 1 --2- 172.21.15.64:0/3159827432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ce329a0b0 0x563ce32a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:33.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.806+0000 7f801ffff700 1 -- 172.21.15.64:0/3159827432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ce329a0b0 msgr2=0x563ce32a1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:33.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.806+0000 7f801ffff700 1 --2- 172.21.15.64:0/3159827432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ce329a0b0 0x563ce32a1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:33.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.806+0000 7f801ffff700 1 -- 172.21.15.64:0/3159827432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ce3299790 msgr2=0x563ce32a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:33.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.806+0000 7f801ffff700 1 --2- 172.21.15.64:0/3159827432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ce3299790 0x563ce32a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:33.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.806+0000 7f801ffff700 1 -- 172.21.15.64:0/3159827432 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8018009090 con 0x563ce32a3ec0 2023-09-13T03:13:33.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.806+0000 7f801f7fe700 1 --2- 172.21.15.64:0/3159827432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ce3299790 0x563ce32a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:13:33.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.807+0000 7f801ffff700 1 --2- 172.21.15.64:0/3159827432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ce32a3ec0 0x563ce32a1660 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8018007bf0 tx=0x7f8018004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9302c9c6ec8c61 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:33.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.807+0000 7f7ffeffd700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:13:33.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.807+0000 7f7ffeffd700 1 -- 172.21.15.64:0/3159827432 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801800e050 con 0x563ce32a3ec0 2023-09-13T03:13:33.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.807+0000 7f7ffeffd700 1 -- 172.21.15.64:0/3159827432 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f8018016de0 con 0x563ce32a3ec0 2023-09-13T03:13:33.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.807+0000 7f80397b4b80 1 -- 172.21.15.64:0/3159827432 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563ce3143570 con 0x563ce32a3ec0 2023-09-13T03:13:33.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.808+0000 7f7ffeffd700 1 -- 172.21.15.64:0/3159827432 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8018007de0 con 0x563ce32a3ec0 2023-09-13T03:13:33.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.808+0000 7f80397b4b80 10 client.15711 fetch_fsmap learned FSMap version 26 2023-09-13T03:13:33.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.808+0000 7f80397b4b80 1 -- 172.21.15.64:0/3159827432 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f802010dcf0 con 0x563ce32a3ec0 2023-09-13T03:13:33.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.808+0000 7f7ffeffd700 1 -- 172.21.15.64:0/3159827432 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8018030030 con 0x563ce32a3ec0 2023-09-13T03:13:33.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.809+0000 7f7ffeffd700 1 -- 172.21.15.64:0/3159827432 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8018004bb0 con 0x563ce32a3ec0 2023-09-13T03:13:33.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.809+0000 7f80397b4b80 10 client.15711 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:13:33.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.809+0000 7f80397b4b80 10 client.15711 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T03:13:33.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.809+0000 7f80397b4b80 20 client.15711 populate_metadata read hostname 'smithi064' 2023-09-13T03:13:33.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.809+0000 7f80397b4b80 1 --2- 172.21.15.64:0/3159827432 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x563ce32b3a50 0x563ce32b5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:33.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.809+0000 7f80397b4b80 4 client.15711 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:13:33.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.809+0000 7f80397b4b80 1 -- 172.21.15.64:0/3159827432 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563ce3200920 con 0x563ce32b3a50 2023-09-13T03:13:33.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.810+0000 7f801f7fe700 1 --2- 172.21.15.64:0/3159827432 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x563ce32b3a50 0x563ce32b5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:33.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.811+0000 7f801f7fe700 1 --2- 172.21.15.64:0/3159827432 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x563ce32b3a50 0x563ce32b5e70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e53757bf93fd132 server_cookie=ec61e5226c90a82 in_seq=0 out_seq=0 2023-09-13T03:13:33.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.811+0000 7f7ffeffd700 10 client.15711 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:13:33.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.812+0000 7f7ffeffd700 1 -- 172.21.15.64:0/3159827432 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563ce3200920 con 0x563ce32b3a50 2023-09-13T03:13:33.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.812+0000 7f7ffeffd700 10 client.15711 handle_command_reply: tid=0 2023-09-13T03:13:33.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.812+0000 7f80397b4b80 1 client.15711 shutdown 2023-09-13T03:13:33.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.812+0000 7f80397b4b80 1 -- 172.21.15.64:0/3159827432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ce32a3ec0 msgr2=0x563ce32a1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:33.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.812+0000 7f80397b4b80 1 --2- 172.21.15.64:0/3159827432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ce32a3ec0 0x563ce32a1660 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8018007bf0 tx=0x7f8018004b40 comp rx=0 tx=0).stop 2023-09-13T03:13:33.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.812+0000 7f80397b4b80 1 -- 172.21.15.64:0/3159827432 shutdown_connections 2023-09-13T03:13:33.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.812+0000 7f80397b4b80 1 --2- 172.21.15.64:0/3159827432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ce3299790 0x563ce32a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:33.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.812+0000 7f80397b4b80 1 --2- 172.21.15.64:0/3159827432 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x563ce32b3a50 0x563ce32b5e70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:33.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.812+0000 7f80397b4b80 1 --2- 172.21.15.64:0/3159827432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ce32a3ec0 0x563ce32a1660 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:33.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.812+0000 7f80397b4b80 1 --2- 172.21.15.64:0/3159827432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ce329a0b0 0x563ce32a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:33.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.812+0000 7f80397b4b80 1 -- 172.21.15.64:0/3159827432 >> 172.21.15.64:0/3159827432 conn(0x563ce320e110 msgr2=0x563ce3294860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:33.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.812+0000 7f7ffeffd700 0 client.15711 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:13:33.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.813+0000 7f80397b4b80 1 -- 172.21.15.64:0/3159827432 shutdown_connections 2023-09-13T03:13:33.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.813+0000 7f80397b4b80 1 -- 172.21.15.64:0/3159827432 wait complete. 2023-09-13T03:13:33.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.813+0000 7f80397b4b80 20 client.15711 trim_cache size 0 max 16384 2023-09-13T03:13:33.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.847+0000 7f80397b4b80 1 Processor -- start 2023-09-13T03:13:33.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.847+0000 7f80397b4b80 1 -- start start 2023-09-13T03:13:33.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.848+0000 7f80397b4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ce3299790 0x563ce32a1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:33.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.848+0000 7f80397b4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ce329a0b0 0x563ce329de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:33.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.848+0000 7f80397b4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ce32a3ec0 0x563ce329e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:33.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.848+0000 7f80397b4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ce32b3870 con 0x563ce3299790 2023-09-13T03:13:33.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.848+0000 7f80397b4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ce32ba7d0 con 0x563ce329a0b0 2023-09-13T03:13:33.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.848+0000 7f80397b4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ce32a0150 con 0x563ce32a3ec0 2023-09-13T03:13:33.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.848+0000 7f801ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ce32a3ec0 0x563ce329e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:33.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.848+0000 7f801f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ce3299790 0x563ce32a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:33.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.848+0000 7f801ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ce32a3ec0 0x563ce329e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:47056/0 (socket says 172.21.15.64:47056) 2023-09-13T03:13:33.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.849+0000 7f801ffff700 1 -- 172.21.15.64:0/3227634888 learned_addr learned my addr 172.21.15.64:0/3227634888 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:33.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.849+0000 7f801f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ce3299790 0x563ce32a1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39162/0 (socket says 172.21.15.64:39162) 2023-09-13T03:13:33.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.849+0000 7f801effd700 1 --2- 172.21.15.64:0/3227634888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ce329a0b0 0x563ce329de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:33.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.849+0000 7f801ffff700 1 -- 172.21.15.64:0/3227634888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ce329a0b0 msgr2=0x563ce329de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:33.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.849+0000 7f801ffff700 1 --2- 172.21.15.64:0/3227634888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ce329a0b0 0x563ce329de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:33.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.849+0000 7f801ffff700 1 -- 172.21.15.64:0/3227634888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ce3299790 msgr2=0x563ce32a1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:33.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.849+0000 7f801ffff700 1 --2- 172.21.15.64:0/3227634888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ce3299790 0x563ce32a1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:33.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.849+0000 7f801ffff700 1 -- 172.21.15.64:0/3227634888 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ce32baaa0 con 0x563ce32a3ec0 2023-09-13T03:13:33.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.849+0000 7f801f7fe700 1 --2- 172.21.15.64:0/3227634888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ce3299790 0x563ce32a1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:13:33.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.850+0000 7f801ffff700 1 --2- 172.21.15.64:0/3227634888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ce32a3ec0 0x563ce329e3b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8018004ee0 tx=0x7f8018006160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d2e149a83de6a56 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:33.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.850+0000 7f80067fc700 1 -- 172.21.15.64:0/3227634888 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801800e040 con 0x563ce32a3ec0 2023-09-13T03:13:33.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.850+0000 7f80067fc700 1 -- 172.21.15.64:0/3227634888 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8018030070 con 0x563ce32a3ec0 2023-09-13T03:13:33.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.850+0000 7f80067fc700 1 -- 172.21.15.64:0/3227634888 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8018023a90 con 0x563ce32a3ec0 2023-09-13T03:13:33.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.851+0000 7f80397b4b80 1 -- 172.21.15.64:0/3227634888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ce32a3ec0 msgr2=0x563ce329e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:33.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.851+0000 7f80397b4b80 1 --2- 172.21.15.64:0/3227634888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ce32a3ec0 0x563ce329e3b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8018004ee0 tx=0x7f8018006160 comp rx=0 tx=0).stop 2023-09-13T03:13:33.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.851+0000 7f80397b4b80 1 -- 172.21.15.64:0/3227634888 shutdown_connections 2023-09-13T03:13:33.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.851+0000 7f80397b4b80 1 --2- 172.21.15.64:0/3227634888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ce32a3ec0 0x563ce329e3b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:33.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.851+0000 7f80397b4b80 1 --2- 172.21.15.64:0/3227634888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ce3299790 0x563ce32a1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:33.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.851+0000 7f80397b4b80 1 --2- 172.21.15.64:0/3227634888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ce329a0b0 0x563ce329de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:33.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.851+0000 7f80397b4b80 1 -- 172.21.15.64:0/3227634888 >> 172.21.15.64:0/3227634888 conn(0x563ce320e110 msgr2=0x563ce3299010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:33.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.851+0000 7f80397b4b80 1 -- 172.21.15.64:0/3227634888 shutdown_connections 2023-09-13T03:13:33.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.852+0000 7f80397b4b80 1 -- 172.21.15.64:0/3227634888 wait complete. 2023-09-13T03:13:33.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.852+0000 7f80397b4b80 1 -- start start 2023-09-13T03:13:33.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.852+0000 7f80397b4b80 1 Processor -- start 2023-09-13T03:13:33.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.853+0000 7f80397b4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ce3299790 0x563ce330a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:33.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.853+0000 7f80397b4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ce329a0b0 0x563ce330a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:33.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.853+0000 7f80397b4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ce32a3ec0 0x563ce32a9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:33.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.853+0000 7f80397b4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ce32a9e10 con 0x563ce32a3ec0 2023-09-13T03:13:33.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.853+0000 7f80397b4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ce32a9f80 con 0x563ce329a0b0 2023-09-13T03:13:33.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.853+0000 7f80397b4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ce32aa140 con 0x563ce3299790 2023-09-13T03:13:33.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.853+0000 7f80397b4b80 10 client.0 _ll_register_callbacks cb 0x7f80180167d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:13:33.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.854+0000 7f801f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ce3299790 0x563ce330a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:33.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.854+0000 7f801f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ce3299790 0x563ce330a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:47072/0 (socket says 172.21.15.64:47072) 2023-09-13T03:13:33.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.854+0000 7f801f7fe700 1 -- 172.21.15.64:0/2789572288 learned_addr learned my addr 172.21.15.64:0/2789572288 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:33.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.854+0000 7f801ffff700 1 --2- 172.21.15.64:0/2789572288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ce32a3ec0 0x563ce32a9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:33.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.854+0000 7f801effd700 1 --2- 172.21.15.64:0/2789572288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ce329a0b0 0x563ce330a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:33.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.854+0000 7f801f7fe700 1 -- 172.21.15.64:0/2789572288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ce329a0b0 msgr2=0x563ce330a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:33.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.854+0000 7f801f7fe700 1 --2- 172.21.15.64:0/2789572288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ce329a0b0 0x563ce330a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:33.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.854+0000 7f801f7fe700 1 -- 172.21.15.64:0/2789572288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ce32a3ec0 msgr2=0x563ce32a9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:33.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.854+0000 7f801f7fe700 1 --2- 172.21.15.64:0/2789572288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ce32a3ec0 0x563ce32a9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:33.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.854+0000 7f801f7fe700 1 -- 172.21.15.64:0/2789572288 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ce3017e60 con 0x563ce3299790 2023-09-13T03:13:33.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.854+0000 7f801ffff700 1 --2- 172.21.15.64:0/2789572288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ce32a3ec0 0x563ce32a9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:13:33.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.854+0000 7f801effd700 1 --2- 172.21.15.64:0/2789572288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ce329a0b0 0x563ce330a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:13:33.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.855+0000 7f801f7fe700 1 --2- 172.21.15.64:0/2789572288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ce3299790 0x563ce330a3b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8010003e70 tx=0x7f8010003ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa512c90c5478f29 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:33.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.855+0000 7f7ffeffd700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:13:33.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.855+0000 7f7ffeffd700 1 -- 172.21.15.64:0/2789572288 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8010035070 con 0x563ce3299790 2023-09-13T03:13:33.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.855+0000 7f80397b4b80 1 -- 172.21.15.64:0/2789572288 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f801001e040 con 0x563ce3299790 2023-09-13T03:13:33.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.855+0000 7f7ffeffd700 1 -- 172.21.15.64:0/2789572288 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f80100314f0 con 0x563ce3299790 2023-09-13T03:13:33.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.855+0000 7f7ffeffd700 1 -- 172.21.15.64:0/2789572288 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f801001d040 con 0x563ce3299790 2023-09-13T03:13:33.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.855+0000 7f80397b4b80 10 client.25738 fetch_fsmap learned FSMap version 26 2023-09-13T03:13:33.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.855+0000 7f80397b4b80 1 -- 172.21.15.64:0/2789572288 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f802010dcf0 con 0x563ce3299790 2023-09-13T03:13:33.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.856+0000 7f7ffeffd700 1 -- 172.21.15.64:0/2789572288 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f801001e040 con 0x563ce3299790 2023-09-13T03:13:33.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.856+0000 7f7ffeffd700 1 -- 172.21.15.64:0/2789572288 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8010038070 con 0x563ce3299790 2023-09-13T03:13:33.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.856+0000 7f80397b4b80 10 client.25738 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:13:33.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.856+0000 7f80397b4b80 10 client.25738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T03:13:33.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.856+0000 7f80397b4b80 20 client.25738 populate_metadata read hostname 'smithi064' 2023-09-13T03:13:33.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.856+0000 7f80397b4b80 1 --2- 172.21.15.64:0/2789572288 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x563ce3311dd0 0x563ce33141c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:33.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.856+0000 7f80397b4b80 4 client.25738 mds_command: new command op to 24686 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-09-13T03:13:33.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.856+0000 7f80397b4b80 1 -- 172.21.15.64:0/2789572288 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x563ce31e49a0 con 0x563ce3311dd0 2023-09-13T03:13:33.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.857+0000 7f801effd700 1 --2- 172.21.15.64:0/2789572288 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x563ce3311dd0 0x563ce33141c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:33.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.857+0000 7f801effd700 1 --2- 172.21.15.64:0/2789572288 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x563ce3311dd0 0x563ce33141c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a318f01b6815ed4a server_cookie=4a726bdc92465633 in_seq=0 out_seq=0 2023-09-13T03:13:33.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.857+0000 7f7ffeffd700 10 client.25738 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:13:33.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.858+0000 7f7ffeffd700 1 -- 172.21.15.64:0/2789572288 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x563ce31e49a0 con 0x563ce3311dd0 2023-09-13T03:13:33.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.858+0000 7f7ffeffd700 10 client.25738 handle_command_reply: tid=0 2023-09-13T03:13:33.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.858+0000 7f80397b4b80 1 client.25738 shutdown 2023-09-13T03:13:33.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.859+0000 7f80397b4b80 1 -- 172.21.15.64:0/2789572288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ce3299790 msgr2=0x563ce330a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:33.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.859+0000 7f80397b4b80 1 --2- 172.21.15.64:0/2789572288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ce3299790 0x563ce330a3b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8010003e70 tx=0x7f8010003ea0 comp rx=0 tx=0).stop 2023-09-13T03:13:33.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.859+0000 7f80397b4b80 1 -- 172.21.15.64:0/2789572288 shutdown_connections 2023-09-13T03:13:33.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.859+0000 7f80397b4b80 1 --2- 172.21.15.64:0/2789572288 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x563ce3311dd0 0x563ce33141c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:33.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.859+0000 7f80397b4b80 1 --2- 172.21.15.64:0/2789572288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ce32a3ec0 0x563ce32a9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:33.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.859+0000 7f80397b4b80 1 --2- 172.21.15.64:0/2789572288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ce3299790 0x563ce330a3b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:33.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.859+0000 7f80397b4b80 1 --2- 172.21.15.64:0/2789572288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ce329a0b0 0x563ce330a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:33.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.859+0000 7f80397b4b80 1 -- 172.21.15.64:0/2789572288 >> 172.21.15.64:0/2789572288 conn(0x563ce320e110 msgr2=0x563ce329f140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:33.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.859+0000 7f7ffeffd700 0 client.25738 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:13:33.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.859+0000 7f80397b4b80 1 -- 172.21.15.64:0/2789572288 shutdown_connections 2023-09-13T03:13:33.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.860+0000 7f80397b4b80 1 -- 172.21.15.64:0/2789572288 wait complete. 2023-09-13T03:13:33.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.860+0000 7f80397b4b80 20 client.25738 trim_cache size 0 max 16384 2023-09-13T03:13:33.871 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-13T03:13:33.871 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2023-09-13T03:13:33.872 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "ceb8edfb-3dea-42c5-aa19-34f86a92e2c6", 2023-09-13T03:13:33.872 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2023-09-13T03:13:33.872 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-13T03:13:33.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.861+0000 7f8025dff700 1 -- 172.21.15.64:0/431176529 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f80080a0940 msgr2=0x7f80080a2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:33.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.861+0000 7f8025dff700 1 --2- 172.21.15.64:0/431176529 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f80080a0940 0x7f80080a2e00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8018009fd0 tx=0x7f801800b040 comp rx=0 tx=0).stop 2023-09-13T03:13:33.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.861+0000 7f8025dff700 1 -- 172.21.15.64:0/431176529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8020100f30 msgr2=0x7f802019ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:33.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.861+0000 7f8025dff700 1 --2- 172.21.15.64:0/431176529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8020100f30 0x7f802019ebb0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f801400e9a0 tx=0x7f801400ecb0 comp rx=0 tx=0).stop 2023-09-13T03:13:33.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.861+0000 7f8025dff700 1 -- 172.21.15.64:0/431176529 shutdown_connections 2023-09-13T03:13:33.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.861+0000 7f8025dff700 1 --2- 172.21.15.64:0/431176529 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f80080a0940 0x7f80080a2e00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:33.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.861+0000 7f8025dff700 1 --2- 172.21.15.64:0/431176529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8020106630 0x7f80201986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:33.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.861+0000 7f8025dff700 1 --2- 172.21.15.64:0/431176529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8020100570 0x7f802019e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:33.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.861+0000 7f8025dff700 1 --2- 172.21.15.64:0/431176529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8020100f30 0x7f802019ebb0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:33.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.861+0000 7f8025dff700 1 -- 172.21.15.64:0/431176529 >> 172.21.15.64:0/431176529 conn(0x7f80200fc080 msgr2=0x7f8020072830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:33.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.861+0000 7f8025dff700 1 -- 172.21.15.64:0/431176529 shutdown_connections 2023-09-13T03:13:33.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:33.861+0000 7f8025dff700 1 -- 172.21.15.64:0/431176529 wait complete. 2023-09-13T03:13:33.891 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-13T03:13:34.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.239+0000 7f1319d9d700 1 -- 172.21.15.64:0/1838396439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1314100d90 msgr2=0x7f131410d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:34.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.239+0000 7f1319d9d700 1 --2- 172.21.15.64:0/1838396439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1314100d90 0x7f131410d2a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f130c009f90 tx=0x7f130c00b330 comp rx=0 tx=0).stop 2023-09-13T03:13:34.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.239+0000 7f1319d9d700 1 -- 172.21.15.64:0/1838396439 shutdown_connections 2023-09-13T03:13:34.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.239+0000 7f1319d9d700 1 --2- 172.21.15.64:0/1838396439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1314106470 0x7f1314106850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:34.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.239+0000 7f1319d9d700 1 --2- 172.21.15.64:0/1838396439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1314100d90 0x7f131410d2a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:34.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.239+0000 7f1319d9d700 1 --2- 172.21.15.64:0/1838396439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f13141003d0 0x7f1314100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:34.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.239+0000 7f1319d9d700 1 -- 172.21.15.64:0/1838396439 >> 172.21.15.64:0/1838396439 conn(0x7f13140fbea0 msgr2=0x7f13140fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:34.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.239+0000 7f1319d9d700 1 -- 172.21.15.64:0/1838396439 shutdown_connections 2023-09-13T03:13:34.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.239+0000 7f1319d9d700 1 -- 172.21.15.64:0/1838396439 wait complete. 2023-09-13T03:13:34.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.241+0000 7f1319d9d700 1 Processor -- start 2023-09-13T03:13:34.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.241+0000 7f1319d9d700 1 -- start start 2023-09-13T03:13:34.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.241+0000 7f1319d9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13141003d0 0x7f131406aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:34.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.241+0000 7f1319d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1314100d90 0x7f131406b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:34.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.241+0000 7f1319d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1314106470 0x7f13141aa980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:34.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.241+0000 7f1319d9d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13141aaec0 con 0x7f1314100d90 2023-09-13T03:13:34.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.241+0000 7f1319d9d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13141ab000 con 0x7f13141003d0 2023-09-13T03:13:34.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.241+0000 7f1319d9d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13141ab190 con 0x7f1314106470 2023-09-13T03:13:34.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.242+0000 7f1318d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13141003d0 0x7f131406aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:34.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.242+0000 7f131959c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1314106470 0x7f13141aa980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:34.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.242+0000 7f1318d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13141003d0 0x7f131406aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:58162/0 (socket says 172.21.15.64:58162) 2023-09-13T03:13:34.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.242+0000 7f1318d9b700 1 -- 172.21.15.64:0/1792655233 learned_addr learned my addr 172.21.15.64:0/1792655233 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:34.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.242+0000 7f1313fff700 1 --2- 172.21.15.64:0/1792655233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1314100d90 0x7f131406b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:34.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.243+0000 7f131959c700 1 -- 172.21.15.64:0/1792655233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13141003d0 msgr2=0x7f131406aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:34.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.243+0000 7f131959c700 1 --2- 172.21.15.64:0/1792655233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13141003d0 0x7f131406aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:34.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.243+0000 7f131959c700 1 -- 172.21.15.64:0/1792655233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1314100d90 msgr2=0x7f131406b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:34.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.243+0000 7f131959c700 1 --2- 172.21.15.64:0/1792655233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1314100d90 0x7f131406b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:34.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.243+0000 7f131959c700 1 -- 172.21.15.64:0/1792655233 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f130c009c40 con 0x7f1314106470 2023-09-13T03:13:34.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.243+0000 7f131959c700 1 --2- 172.21.15.64:0/1792655233 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1314106470 0x7f13141aa980 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f130c00b6c0 tx=0x7f130c009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:34.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.243+0000 7f1311ffb700 1 -- 172.21.15.64:0/1792655233 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130c00e040 con 0x7f1314106470 2023-09-13T03:13:34.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.243+0000 7f1311ffb700 1 -- 172.21.15.64:0/1792655233 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f130c01d470 con 0x7f1314106470 2023-09-13T03:13:34.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.243+0000 7f1311ffb700 1 -- 172.21.15.64:0/1792655233 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130c0052a0 con 0x7f1314106470 2023-09-13T03:13:34.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.244+0000 7f1319d9d700 1 -- 172.21.15.64:0/1792655233 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13141ab410 con 0x7f1314106470 2023-09-13T03:13:34.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.244+0000 7f1319d9d700 1 -- 172.21.15.64:0/1792655233 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13141ab840 con 0x7f1314106470 2023-09-13T03:13:34.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.249+0000 7f1311ffb700 1 -- 172.21.15.64:0/1792655233 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 24) v1 ==== 135435+0+0 (secure 0 0 0) 0x7f130c01d8e0 con 0x7f1314106470 2023-09-13T03:13:34.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.250+0000 7f132e980b80 1 Processor -- start 2023-09-13T03:13:34.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.251+0000 7f132e980b80 1 -- start start 2023-09-13T03:13:34.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.251+0000 7f132e980b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4016e5080 0x55f4016e5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:34.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.251+0000 7f132e980b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f4016e59a0 0x55f4016d6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:34.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.251+0000 7f132e980b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4016d6e00 0x55f4016d7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:34.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.251+0000 7f132e980b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f40162f730 con 0x55f4016e59a0 2023-09-13T03:13:34.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.251+0000 7f132e980b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f40162d050 con 0x55f4016d6e00 2023-09-13T03:13:34.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.251+0000 7f132e980b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f401628940 con 0x55f4016e5080 2023-09-13T03:13:34.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.252+0000 7f1313fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4016e5080 0x55f4016e5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:34.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.252+0000 7f1313fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4016e5080 0x55f4016e5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:47096/0 (socket says 172.21.15.64:47096) 2023-09-13T03:13:34.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.252+0000 7f1313fff700 1 -- 172.21.15.64:0/556346964 learned_addr learned my addr 172.21.15.64:0/556346964 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:34.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.252+0000 7f1313fff700 1 --2- 172.21.15.64:0/556346964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4016d6e00 0x55f4016d7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:34.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.252+0000 7f1311ffb700 1 --2- 172.21.15.64:0/1792655233 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f13040a0960 0x7f13040a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:34.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.252+0000 7f1311ffb700 1 -- 172.21.15.64:0/1792655233 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f130c0cd6d0 con 0x7f1314106470 2023-09-13T03:13:34.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.252+0000 7f1313fff700 1 -- 172.21.15.64:0/556346964 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4016e5080 msgr2=0x55f4016e5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:34.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.252+0000 7f1313fff700 1 --2- 172.21.15.64:0/556346964 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4016e5080 0x55f4016e5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:34.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.252+0000 7f1313fff700 1 -- 172.21.15.64:0/556346964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f4016e59a0 msgr2=0x55f4016d6870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-13T03:13:34.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.252+0000 7f1313fff700 1 --2- 172.21.15.64:0/556346964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f4016e59a0 0x55f4016d6870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:34.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.252+0000 7f1313fff700 1 -- 172.21.15.64:0/556346964 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f401454de0 con 0x55f4016d6e00 2023-09-13T03:13:34.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.252+0000 7f131959c700 1 --2- 172.21.15.64:0/1792655233 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f13040a0960 0x7f13040a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:34.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.253+0000 7f1313fff700 1 --2- 172.21.15.64:0/556346964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4016d6e00 0x55f4016d7280 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f13000095f0 tx=0x7f1300009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5364f291434feeb3 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:34.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.253+0000 7f12feffd700 1 -- 172.21.15.64:0/556346964 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1300011e70 con 0x55f4016d6e00 2023-09-13T03:13:34.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.253+0000 7f12feffd700 1 -- 172.21.15.64:0/556346964 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1300011540 con 0x55f4016d6e00 2023-09-13T03:13:34.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.253+0000 7f12feffd700 1 -- 172.21.15.64:0/556346964 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1300005280 con 0x55f4016d6e00 2023-09-13T03:13:34.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.254+0000 7f132e980b80 1 -- 172.21.15.64:0/556346964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4016d6e00 msgr2=0x55f4016d7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:34.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.254+0000 7f132e980b80 1 --2- 172.21.15.64:0/556346964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4016d6e00 0x55f4016d7280 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f13000095f0 tx=0x7f1300009260 comp rx=0 tx=0).stop 2023-09-13T03:13:34.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.254+0000 7f131959c700 1 --2- 172.21.15.64:0/1792655233 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f13040a0960 0x7f13040a2e20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f130c0cadb0 tx=0x7f130c00b330 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:34.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.255+0000 7f132e980b80 1 -- 172.21.15.64:0/556346964 shutdown_connections 2023-09-13T03:13:34.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.255+0000 7f132e980b80 1 --2- 172.21.15.64:0/556346964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4016d6e00 0x55f4016d7280 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:34.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.255+0000 7f132e980b80 1 --2- 172.21.15.64:0/556346964 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4016e5080 0x55f4016e5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:34.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.255+0000 7f132e980b80 1 --2- 172.21.15.64:0/556346964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f4016e59a0 0x55f4016d6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:34.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.255+0000 7f132e980b80 1 -- 172.21.15.64:0/556346964 >> 172.21.15.64:0/556346964 conn(0x55f40164b090 msgr2=0x55f4016d1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:34.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.255+0000 7f132e980b80 1 -- 172.21.15.64:0/556346964 shutdown_connections 2023-09-13T03:13:34.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.255+0000 7f132e980b80 1 -- 172.21.15.64:0/556346964 wait complete. 2023-09-13T03:13:34.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.256+0000 7f132e980b80 1 -- start start 2023-09-13T03:13:34.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.256+0000 7f132e980b80 1 Processor -- start 2023-09-13T03:13:34.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.257+0000 7f132e980b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4016d6e00 0x55f4016e8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:34.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.257+0000 7f132e980b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4016e5080 0x55f4016e9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:34.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.257+0000 7f132e980b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f4016e59a0 0x55f4016e97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:34.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.257+0000 7f132e980b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4015c6b70 con 0x55f4016e59a0 2023-09-13T03:13:34.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.257+0000 7f132e980b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4015c6870 con 0x55f4016e5080 2023-09-13T03:13:34.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.257+0000 7f132e980b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4014aa920 con 0x55f4016d6e00 2023-09-13T03:13:34.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.257+0000 7f132e980b80 10 client.0 _ll_register_callbacks cb 0x55f401510a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:13:34.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.257+0000 7f1313fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4016d6e00 0x55f4016e8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:34.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.257+0000 7f1318d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4016e5080 0x55f4016e9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:34.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.257+0000 7f1318d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4016e5080 0x55f4016e9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:58186/0 (socket says 172.21.15.64:58186) 2023-09-13T03:13:34.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.257+0000 7f1318d9b700 1 -- 172.21.15.64:0/2120410515 learned_addr learned my addr 172.21.15.64:0/2120410515 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:34.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.257+0000 7f1313fff700 1 --2- 172.21.15.64:0/2120410515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f4016e59a0 0x55f4016e97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:34.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.258+0000 7f1318d9b700 1 -- 172.21.15.64:0/2120410515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4016d6e00 msgr2=0x55f4016e8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:34.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.258+0000 7f1318d9b700 1 --2- 172.21.15.64:0/2120410515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4016d6e00 0x55f4016e8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:34.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.258+0000 7f1318d9b700 1 -- 172.21.15.64:0/2120410515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f4016e59a0 msgr2=0x55f4016e97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:34.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.258+0000 7f1318d9b700 1 --2- 172.21.15.64:0/2120410515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f4016e59a0 0x55f4016e97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:34.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.258+0000 7f1318d9b700 1 -- 172.21.15.64:0/2120410515 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1300019040 con 0x55f4016e5080 2023-09-13T03:13:34.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.258+0000 7f1318d9b700 1 --2- 172.21.15.64:0/2120410515 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4016e5080 0x55f4016e9280 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f13080116f0 tx=0x7f1308009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86f25906059e23b3 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:34.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.258+0000 7f12f3fff700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:13:34.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.258+0000 7f12f3fff700 1 -- 172.21.15.64:0/2120410515 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1308004500 con 0x55f4016e5080 2023-09-13T03:13:34.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.259+0000 7f132e980b80 1 -- 172.21.15.64:0/2120410515 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f4015804f0 con 0x55f4016e5080 2023-09-13T03:13:34.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.259+0000 7f12f3fff700 1 -- 172.21.15.64:0/2120410515 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f1308003680 con 0x55f4016e5080 2023-09-13T03:13:34.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.259+0000 7f12f3fff700 1 -- 172.21.15.64:0/2120410515 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1308003840 con 0x55f4016e5080 2023-09-13T03:13:34.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.259+0000 7f132e980b80 10 client.25760 fetch_fsmap learned FSMap version 26 2023-09-13T03:13:34.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.259+0000 7f132e980b80 1 -- 172.21.15.64:0/2120410515 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55f40143dc60 con 0x55f4016e5080 2023-09-13T03:13:34.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.259+0000 7f12f3fff700 1 -- 172.21.15.64:0/2120410515 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f4015804f0 con 0x55f4016e5080 2023-09-13T03:13:34.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.260+0000 7f12f3fff700 1 -- 172.21.15.64:0/2120410515 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f130801aa90 con 0x55f4016e5080 2023-09-13T03:13:34.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.260+0000 7f132e980b80 10 client.25760 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:13:34.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.260+0000 7f132e980b80 10 client.25760 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T03:13:34.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.260+0000 7f132e980b80 20 client.25760 populate_metadata read hostname 'smithi064' 2023-09-13T03:13:34.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.260+0000 7f132e980b80 1 --2- 172.21.15.64:0/2120410515 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f4016f9550 0x55f4016fb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:34.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.260+0000 7f132e980b80 4 client.25760 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:13:34.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.260+0000 7f132e980b80 1 -- 172.21.15.64:0/2120410515 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f40163d8a0 con 0x55f4016f9550 2023-09-13T03:13:34.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.260+0000 7f1313fff700 1 --2- 172.21.15.64:0/2120410515 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f4016f9550 0x55f4016fb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:34.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.261+0000 7f1313fff700 1 --2- 172.21.15.64:0/2120410515 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f4016f9550 0x55f4016fb970 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=126aa919e760564 server_cookie=bfc5129a477f420f in_seq=0 out_seq=0 2023-09-13T03:13:34.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.261+0000 7f12f3fff700 10 client.25760 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:13:34.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.262+0000 7f12f3fff700 1 -- 172.21.15.64:0/2120410515 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f40163d8a0 con 0x55f4016f9550 2023-09-13T03:13:34.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.262+0000 7f12f3fff700 10 client.25760 handle_command_reply: tid=0 2023-09-13T03:13:34.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.262+0000 7f132e980b80 1 client.25760 shutdown 2023-09-13T03:13:34.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.262+0000 7f132e980b80 1 -- 172.21.15.64:0/2120410515 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4016e5080 msgr2=0x55f4016e9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:34.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.262+0000 7f132e980b80 1 --2- 172.21.15.64:0/2120410515 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4016e5080 0x55f4016e9280 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f13080116f0 tx=0x7f1308009f90 comp rx=0 tx=0).stop 2023-09-13T03:13:34.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.263+0000 7f132e980b80 1 -- 172.21.15.64:0/2120410515 shutdown_connections 2023-09-13T03:13:34.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.263+0000 7f132e980b80 1 --2- 172.21.15.64:0/2120410515 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f4016f9550 0x55f4016fb970 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:34.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.263+0000 7f132e980b80 1 --2- 172.21.15.64:0/2120410515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f4016e59a0 0x55f4016e97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:34.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.263+0000 7f132e980b80 1 --2- 172.21.15.64:0/2120410515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4016d6e00 0x55f4016e8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:34.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.263+0000 7f132e980b80 1 --2- 172.21.15.64:0/2120410515 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4016e5080 0x55f4016e9280 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:34.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.263+0000 7f132e980b80 1 -- 172.21.15.64:0/2120410515 >> 172.21.15.64:0/2120410515 conn(0x55f40164b090 msgr2=0x55f4016d17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:34.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.263+0000 7f12f3fff700 0 client.25760 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:13:34.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.263+0000 7f132e980b80 1 -- 172.21.15.64:0/2120410515 shutdown_connections 2023-09-13T03:13:34.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.263+0000 7f132e980b80 1 -- 172.21.15.64:0/2120410515 wait complete. 2023-09-13T03:13:34.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.263+0000 7f132e980b80 20 client.25760 trim_cache size 0 max 16384 2023-09-13T03:13:34.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.297+0000 7f132e980b80 1 Processor -- start 2023-09-13T03:13:34.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.297+0000 7f132e980b80 1 -- start start 2023-09-13T03:13:34.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.297+0000 7f132e980b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4016d6e00 0x55f4016d6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:34.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.297+0000 7f132e980b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f4016e5080 0x55f4016f0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:34.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.297+0000 7f132e980b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4016e59a0 0x55f4016e82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:34.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.297+0000 7f132e980b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4016e5820 con 0x55f4016e5080 2023-09-13T03:13:34.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.297+0000 7f132e980b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4016cbed0 con 0x55f4016e59a0 2023-09-13T03:13:34.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.297+0000 7f132e980b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4016d2a60 con 0x55f4016d6e00 2023-09-13T03:13:34.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.298+0000 7f1318d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f4016e5080 0x55f4016f0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:34.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.298+0000 7f1318d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f4016e5080 0x55f4016f0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39224/0 (socket says 172.21.15.64:39224) 2023-09-13T03:13:34.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.298+0000 7f1318d9b700 1 -- 172.21.15.64:0/3948533952 learned_addr learned my addr 172.21.15.64:0/3948533952 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:34.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.298+0000 7f1318d9b700 1 -- 172.21.15.64:0/3948533952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4016d6e00 msgr2=0x55f4016d6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:34.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.298+0000 7f1318d9b700 1 --2- 172.21.15.64:0/3948533952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4016d6e00 0x55f4016d6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:34.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.298+0000 7f1318d9b700 1 -- 172.21.15.64:0/3948533952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4016e59a0 msgr2=0x55f4016e82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:34.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.298+0000 7f1318d9b700 1 --2- 172.21.15.64:0/3948533952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4016e59a0 0x55f4016e82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:34.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.299+0000 7f1318d9b700 1 -- 172.21.15.64:0/3948533952 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f40147b250 con 0x55f4016e5080 2023-09-13T03:13:34.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.299+0000 7f1318d9b700 1 --2- 172.21.15.64:0/3948533952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f4016e5080 0x55f4016f0390 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f13080116c0 tx=0x7f1308000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aece3e5611aaae3 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:34.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.300+0000 7f12ff7fe700 1 -- 172.21.15.64:0/3948533952 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1308005570 con 0x55f4016e5080 2023-09-13T03:13:34.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.300+0000 7f12ff7fe700 1 -- 172.21.15.64:0/3948533952 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13080056d0 con 0x55f4016e5080 2023-09-13T03:13:34.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.300+0000 7f12ff7fe700 1 -- 172.21.15.64:0/3948533952 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1308003be0 con 0x55f4016e5080 2023-09-13T03:13:34.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.300+0000 7f132e980b80 1 -- 172.21.15.64:0/3948533952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f4016e5080 msgr2=0x55f4016f0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:34.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.300+0000 7f132e980b80 1 --2- 172.21.15.64:0/3948533952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f4016e5080 0x55f4016f0390 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f13080116c0 tx=0x7f1308000bc0 comp rx=0 tx=0).stop 2023-09-13T03:13:34.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.301+0000 7f132e980b80 1 -- 172.21.15.64:0/3948533952 shutdown_connections 2023-09-13T03:13:34.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.301+0000 7f132e980b80 1 --2- 172.21.15.64:0/3948533952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4016e59a0 0x55f4016e82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:34.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.301+0000 7f132e980b80 1 --2- 172.21.15.64:0/3948533952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4016d6e00 0x55f4016d6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:34.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.301+0000 7f132e980b80 1 --2- 172.21.15.64:0/3948533952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f4016e5080 0x55f4016f0390 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:34.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.301+0000 7f132e980b80 1 -- 172.21.15.64:0/3948533952 >> 172.21.15.64:0/3948533952 conn(0x55f40164b090 msgr2=0x55f4016e45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:34.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.301+0000 7f132e980b80 1 -- 172.21.15.64:0/3948533952 shutdown_connections 2023-09-13T03:13:34.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.301+0000 7f132e980b80 1 -- 172.21.15.64:0/3948533952 wait complete. 2023-09-13T03:13:34.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.301+0000 7f132e980b80 1 -- start start 2023-09-13T03:13:34.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.302+0000 7f132e980b80 1 Processor -- start 2023-09-13T03:13:34.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.302+0000 7f132e980b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f4016d6e00 0x55f4016f0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:34.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.303+0000 7f132e980b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4016e5080 0x55f4016f0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:34.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.303+0000 7f132e980b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4016e59a0 0x55f4016ee4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:34.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.303+0000 7f132e980b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4016eebf0 con 0x55f4016d6e00 2023-09-13T03:13:34.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.303+0000 7f132e980b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4016eed60 con 0x55f4016e59a0 2023-09-13T03:13:34.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.303+0000 7f132e980b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4016eef20 con 0x55f4016e5080 2023-09-13T03:13:34.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.303+0000 7f132e980b80 10 client.0 _ll_register_callbacks cb 0x7f1308009d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:13:34.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.303+0000 7f1318d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4016e5080 0x55f4016f0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:34.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.303+0000 7f1318d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4016e5080 0x55f4016f0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:47112/0 (socket says 172.21.15.64:47112) 2023-09-13T03:13:34.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.303+0000 7f1318d9b700 1 -- 172.21.15.64:0/3719810829 learned_addr learned my addr 172.21.15.64:0/3719810829 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:34.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.303+0000 7f1318d9b700 1 -- 172.21.15.64:0/3719810829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4016e59a0 msgr2=0x55f4016ee4e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-13T03:13:34.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.303+0000 7f1318d9b700 1 --2- 172.21.15.64:0/3719810829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4016e59a0 0x55f4016ee4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:34.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.303+0000 7f1318d9b700 1 -- 172.21.15.64:0/3719810829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f4016d6e00 msgr2=0x55f4016f0000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-13T03:13:34.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.303+0000 7f1318d9b700 1 --2- 172.21.15.64:0/3719810829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f4016d6e00 0x55f4016f0000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:34.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.303+0000 7f1318d9b700 1 -- 172.21.15.64:0/3719810829 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f40143dc60 con 0x55f4016e5080 2023-09-13T03:13:34.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.305+0000 7f1318d9b700 1 --2- 172.21.15.64:0/3719810829 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4016e5080 0x55f4016f0540 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1308005950 tx=0x7f13080094b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f507ba6ac8a47168 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:34.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.305+0000 7f12f3fff700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:13:34.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.305+0000 7f12f3fff700 1 -- 172.21.15.64:0/3719810829 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130803b070 con 0x55f4016e5080 2023-09-13T03:13:34.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.305+0000 7f132e980b80 1 -- 172.21.15.64:0/3719810829 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1300011540 con 0x55f4016e5080 2023-09-13T03:13:34.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.305+0000 7f12f3fff700 1 -- 172.21.15.64:0/3719810829 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f1308000e40 con 0x55f4016e5080 2023-09-13T03:13:34.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.306+0000 7f12f3fff700 1 -- 172.21.15.64:0/3719810829 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1308003b50 con 0x55f4016e5080 2023-09-13T03:13:34.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.306+0000 7f132e980b80 10 client.25753 fetch_fsmap learned FSMap version 26 2023-09-13T03:13:34.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.306+0000 7f132e980b80 1 -- 172.21.15.64:0/3719810829 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55f4014532b0 con 0x55f4016e5080 2023-09-13T03:13:34.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.306+0000 7f12f3fff700 1 -- 172.21.15.64:0/3719810829 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1300011540 con 0x55f4016e5080 2023-09-13T03:13:34.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.308+0000 7f12f3fff700 1 -- 172.21.15.64:0/3719810829 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f130803e020 con 0x55f4016e5080 2023-09-13T03:13:34.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.308+0000 7f132e980b80 10 client.25753 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:13:34.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.308+0000 7f132e980b80 10 client.25753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T03:13:34.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.308+0000 7f132e980b80 20 client.25753 populate_metadata read hostname 'smithi064' 2023-09-13T03:13:34.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.308+0000 7f132e980b80 1 --2- 172.21.15.64:0/3719810829 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f40174e230 0x55f401750650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:34.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.308+0000 7f132e980b80 4 client.25753 mds_command: new command op to 24686 tid=0[{"prefix": "scrub status"}] 2023-09-13T03:13:34.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.308+0000 7f132e980b80 1 -- 172.21.15.64:0/3719810829 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f401621920 con 0x55f40174e230 2023-09-13T03:13:34.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.308+0000 7f1313fff700 1 --2- 172.21.15.64:0/3719810829 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f40174e230 0x55f401750650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:34.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.309+0000 7f1313fff700 1 --2- 172.21.15.64:0/3719810829 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f40174e230 0x55f401750650 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=639d54824dbaa200 server_cookie=6c3c308a164028c in_seq=0 out_seq=0 2023-09-13T03:13:34.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.309+0000 7f12f3fff700 10 client.25753 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:13:34.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.309+0000 7f12f3fff700 1 -- 172.21.15.64:0/3719810829 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f401621920 con 0x55f40174e230 2023-09-13T03:13:34.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.309+0000 7f12f3fff700 10 client.25753 handle_command_reply: tid=0 2023-09-13T03:13:34.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.309+0000 7f132e980b80 1 client.25753 shutdown 2023-09-13T03:13:34.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.309+0000 7f132e980b80 1 -- 172.21.15.64:0/3719810829 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4016e5080 msgr2=0x55f4016f0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:34.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.309+0000 7f132e980b80 1 --2- 172.21.15.64:0/3719810829 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4016e5080 0x55f4016f0540 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1308005950 tx=0x7f13080094b0 comp rx=0 tx=0).stop 2023-09-13T03:13:34.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.309+0000 7f132e980b80 1 -- 172.21.15.64:0/3719810829 shutdown_connections 2023-09-13T03:13:34.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.309+0000 7f132e980b80 1 --2- 172.21.15.64:0/3719810829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4016e59a0 0x55f4016ee4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:34.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.309+0000 7f132e980b80 1 --2- 172.21.15.64:0/3719810829 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f40174e230 0x55f401750650 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:34.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.309+0000 7f132e980b80 1 --2- 172.21.15.64:0/3719810829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f4016d6e00 0x55f4016f0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:34.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.309+0000 7f132e980b80 1 --2- 172.21.15.64:0/3719810829 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4016e5080 0x55f4016f0540 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:34.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.309+0000 7f132e980b80 1 -- 172.21.15.64:0/3719810829 >> 172.21.15.64:0/3719810829 conn(0x55f40164b090 msgr2=0x55f4016e92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:34.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.310+0000 7f12f3fff700 0 client.25753 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:13:34.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.310+0000 7f132e980b80 1 -- 172.21.15.64:0/3719810829 shutdown_connections 2023-09-13T03:13:34.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.310+0000 7f132e980b80 1 -- 172.21.15.64:0/3719810829 wait complete. 2023-09-13T03:13:34.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.310+0000 7f132e980b80 20 client.25753 trim_cache size 0 max 16384 2023-09-13T03:13:34.319 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-13T03:13:34.319 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2023-09-13T03:13:34.320 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2023-09-13T03:13:34.320 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-13T03:13:34.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.311+0000 7f1319d9d700 1 -- 172.21.15.64:0/1792655233 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f13040a0960 msgr2=0x7f13040a2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:34.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.311+0000 7f1319d9d700 1 --2- 172.21.15.64:0/1792655233 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f13040a0960 0x7f13040a2e20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f130c0cadb0 tx=0x7f130c00b330 comp rx=0 tx=0).stop 2023-09-13T03:13:34.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.311+0000 7f1319d9d700 1 -- 172.21.15.64:0/1792655233 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1314106470 msgr2=0x7f13141aa980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:34.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.311+0000 7f1319d9d700 1 --2- 172.21.15.64:0/1792655233 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1314106470 0x7f13141aa980 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f130c00b6c0 tx=0x7f130c009500 comp rx=0 tx=0).stop 2023-09-13T03:13:34.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.312+0000 7f1319d9d700 1 -- 172.21.15.64:0/1792655233 shutdown_connections 2023-09-13T03:13:34.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.312+0000 7f1319d9d700 1 --2- 172.21.15.64:0/1792655233 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f13040a0960 0x7f13040a2e20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:34.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.312+0000 7f1319d9d700 1 --2- 172.21.15.64:0/1792655233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13141003d0 0x7f131406aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:34.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.312+0000 7f1319d9d700 1 --2- 172.21.15.64:0/1792655233 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1314106470 0x7f13141aa980 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:34.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.312+0000 7f1319d9d700 1 --2- 172.21.15.64:0/1792655233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1314100d90 0x7f131406b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:34.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.312+0000 7f1319d9d700 1 -- 172.21.15.64:0/1792655233 >> 172.21.15.64:0/1792655233 conn(0x7f13140fbea0 msgr2=0x7f13140fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:34.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.312+0000 7f1319d9d700 1 -- 172.21.15.64:0/1792655233 shutdown_connections 2023-09-13T03:13:34.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.312+0000 7f1319d9d700 1 -- 172.21.15.64:0/1792655233 wait complete. 2023-09-13T03:13:34.340 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-09-13T03:13:34.340 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T03:13:34.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:34 smithi064 ceph-mon[97675]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2023-09-13T03:13:34.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:34 smithi064 ceph-mon[97675]: scrub queued for path: / 2023-09-13T03:13:34.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:34 smithi064 ceph-mon[97675]: scrub summary: idle+waiting paths [/] 2023-09-13T03:13:34.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:34 smithi064 ceph-mon[97675]: scrub summary: active paths [/] 2023-09-13T03:13:34.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:34 smithi064 ceph-mon[97675]: scrub summary: idle+waiting paths [/] 2023-09-13T03:13:34.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:34 smithi064 ceph-mon[97675]: scrub summary: idle 2023-09-13T03:13:34.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.684+0000 7eff439ff700 1 -- 172.21.15.64:0/3200307759 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff3c0729a0 msgr2=0x7eff3c072d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:34.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.684+0000 7eff439ff700 1 --2- 172.21.15.64:0/3200307759 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff3c0729a0 0x7eff3c072d80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7eff38009990 tx=0x7eff38009ca0 comp rx=0 tx=0).stop 2023-09-13T03:13:34.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.684+0000 7eff439ff700 1 -- 172.21.15.64:0/3200307759 shutdown_connections 2023-09-13T03:13:34.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.684+0000 7eff439ff700 1 --2- 172.21.15.64:0/3200307759 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff3c0729a0 0x7eff3c072d80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:34.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.684+0000 7eff439ff700 1 --2- 172.21.15.64:0/3200307759 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff3c111590 0x7eff3c113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:34.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.684+0000 7eff439ff700 1 --2- 172.21.15.64:0/3200307759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff3c0732c0 0x7eff3c111050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:34.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.684+0000 7eff439ff700 1 -- 172.21.15.64:0/3200307759 >> 172.21.15.64:0/3200307759 conn(0x7eff3c0fc0b0 msgr2=0x7eff3c0fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:34.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.684+0000 7eff439ff700 1 -- 172.21.15.64:0/3200307759 shutdown_connections 2023-09-13T03:13:34.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.684+0000 7eff439ff700 1 -- 172.21.15.64:0/3200307759 wait complete. 2023-09-13T03:13:34.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.686+0000 7eff439ff700 1 Processor -- start 2023-09-13T03:13:34.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.686+0000 7eff439ff700 1 -- start start 2023-09-13T03:13:34.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.687+0000 7eff439ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff3c0729a0 0x7eff3c1a2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:34.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.687+0000 7eff439ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff3c0732c0 0x7eff3c1a3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:34.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.687+0000 7eff439ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff3c111590 0x7eff3c19cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:34.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.687+0000 7eff439ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c19d1a0 con 0x7eff3c0732c0 2023-09-13T03:13:34.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.687+0000 7eff439ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c19d310 con 0x7eff3c111590 2023-09-13T03:13:34.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.687+0000 7eff439ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c19d4d0 con 0x7eff3c0729a0 2023-09-13T03:13:34.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.687+0000 7eff4179b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff3c0729a0 0x7eff3c1a2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:34.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.687+0000 7eff4179b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff3c0729a0 0x7eff3c1a2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:47120/0 (socket says 172.21.15.64:47120) 2023-09-13T03:13:34.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.687+0000 7eff4179b700 1 -- 172.21.15.64:0/2256578416 learned_addr learned my addr 172.21.15.64:0/2256578416 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:34.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.687+0000 7eff41f9c700 1 --2- 172.21.15.64:0/2256578416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff3c111590 0x7eff3c19cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:34.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.687+0000 7eff40f9a700 1 --2- 172.21.15.64:0/2256578416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff3c0732c0 0x7eff3c1a3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:34.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.688+0000 7eff4179b700 1 -- 172.21.15.64:0/2256578416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff3c111590 msgr2=0x7eff3c19cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:34.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.688+0000 7eff4179b700 1 --2- 172.21.15.64:0/2256578416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff3c111590 0x7eff3c19cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:34.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.688+0000 7eff4179b700 1 -- 172.21.15.64:0/2256578416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff3c0732c0 msgr2=0x7eff3c1a3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:34.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.688+0000 7eff4179b700 1 --2- 172.21.15.64:0/2256578416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff3c0732c0 0x7eff3c1a3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:34.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.688+0000 7eff4179b700 1 -- 172.21.15.64:0/2256578416 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff380095f0 con 0x7eff3c0729a0 2023-09-13T03:13:34.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.688+0000 7eff40f9a700 1 --2- 172.21.15.64:0/2256578416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff3c0732c0 0x7eff3c1a3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:13:34.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.688+0000 7eff4179b700 1 --2- 172.21.15.64:0/2256578416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff3c0729a0 0x7eff3c1a2ac0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7eff38000c00 tx=0x7eff38005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:34.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.688+0000 7eff327fc700 1 -- 172.21.15.64:0/2256578416 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3801d070 con 0x7eff3c0729a0 2023-09-13T03:13:34.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.689+0000 7eff327fc700 1 -- 172.21.15.64:0/2256578416 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff38022470 con 0x7eff3c0729a0 2023-09-13T03:13:34.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.689+0000 7eff327fc700 1 -- 172.21.15.64:0/2256578416 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff38017650 con 0x7eff3c0729a0 2023-09-13T03:13:34.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.689+0000 7eff439ff700 1 -- 172.21.15.64:0/2256578416 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff3c19d750 con 0x7eff3c0729a0 2023-09-13T03:13:34.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.689+0000 7eff439ff700 1 -- 172.21.15.64:0/2256578416 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff3c19d970 con 0x7eff3c0729a0 2023-09-13T03:13:34.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.690+0000 7eff439ff700 1 -- 172.21.15.64:0/2256578416 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff3c103ab0 con 0x7eff3c0729a0 2023-09-13T03:13:34.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.694+0000 7eff327fc700 1 -- 172.21.15.64:0/2256578416 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 24) v1 ==== 135435+0+0 (secure 0 0 0) 0x7eff38022850 con 0x7eff3c0729a0 2023-09-13T03:13:34.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.697+0000 7eff327fc700 1 --2- 172.21.15.64:0/2256578416 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7eff280a0900 0x7eff280a2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:34.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.697+0000 7eff327fc700 1 -- 172.21.15.64:0/2256578416 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7eff380cdbf0 con 0x7eff3c0729a0 2023-09-13T03:13:34.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.697+0000 7eff327fc700 1 -- 172.21.15.64:0/2256578416 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7eff380f9e90 con 0x7eff3c0729a0 2023-09-13T03:13:34.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.697+0000 7eff40f9a700 1 --2- 172.21.15.64:0/2256578416 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7eff280a0900 0x7eff280a2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:34.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:34.698+0000 7eff40f9a700 1 --2- 172.21.15.64:0/2256578416 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7eff280a0900 0x7eff280a2dc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7eff3c112bc0 tx=0x7eff2c009260 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:34.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:34 smithi170 ceph-mon[134328]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2023-09-13T03:13:34.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:34 smithi170 ceph-mon[134328]: scrub queued for path: / 2023-09-13T03:13:34.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:34 smithi170 ceph-mon[134328]: scrub summary: idle+waiting paths [/] 2023-09-13T03:13:34.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:34 smithi170 ceph-mon[134328]: scrub summary: active paths [/] 2023-09-13T03:13:34.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:34 smithi170 ceph-mon[134328]: scrub summary: idle+waiting paths [/] 2023-09-13T03:13:34.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:34 smithi170 ceph-mon[134328]: scrub summary: idle 2023-09-13T03:13:34.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:34 smithi142 ceph-mon[134082]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2023-09-13T03:13:34.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:34 smithi142 ceph-mon[134082]: scrub queued for path: / 2023-09-13T03:13:34.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:34 smithi142 ceph-mon[134082]: scrub summary: idle+waiting paths [/] 2023-09-13T03:13:34.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:34 smithi142 ceph-mon[134082]: scrub summary: active paths [/] 2023-09-13T03:13:34.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:34 smithi142 ceph-mon[134082]: scrub summary: idle+waiting paths [/] 2023-09-13T03:13:34.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:34 smithi142 ceph-mon[134082]: scrub summary: idle 2023-09-13T03:13:35.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.203+0000 7eff439ff700 1 -- 172.21.15.64:0/2256578416 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff3c0673f0 con 0x7eff3c0729a0 2023-09-13T03:13:35.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.204+0000 7eff327fc700 1 -- 172.21.15.64:0/2256578416 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11366 (secure 0 0 0) 0x7eff380ba070 con 0x7eff3c0729a0 2023-09-13T03:13:35.205 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:13:35.205 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2234182985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2234182985},{"type":"v1","addr":"172.21.15.64:6835","nonce":2234182985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4053841873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4053841873},{"type":"v1","addr":"172.21.15.64:6837","nonce":4053841873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-13T03:11:10.490260+0000","modified":"2023-09-13T03:11:18.230740+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24686,"mds_1":24649,"mds_2":24635,"mds_3":24634,"mds_4":24622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6839/99523374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":99523374},{"type":"v1","addr":"172.21.15.64:6839","nonce":99523374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14649":{"gid":14649,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6841/1866582875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":1866582875},{"type":"v1","addr":"172.21.15.64:6841","nonce":1866582875}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/101906999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":101906999},{"type":"v1","addr":"172.21.15.142:6833","nonce":101906999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24574":{"gid":24574,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/1897366227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1897366227},{"type":"v1","addr":"172.21.15.170:6833","nonce":1897366227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24611":{"gid":24611,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2167779918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2167779918},{"type":"v1","addr":"172.21.15.142:6835","nonce":2167779918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24622":{"gid":24622,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6835/1493371121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1493371121},{"type":"v1","addr":"172.21.15.170:6835","nonce":1493371121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24634":{"gid":24634,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.170:6837/1726587275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":1726587275},{"type":"v1","addr":"172.21.15.170:6837","nonce":1726587275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24635":{"gid":24635,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.142:6837/741173268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":741173268},{"type":"v1","addr":"172.21.15.142:6837","nonce":741173268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24649":{"gid":24649,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.170:6839/2189703596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2189703596},{"type":"v1","addr":"172.21.15.170:6839","nonce":2189703596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24686":{"gid":24686,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/3645998616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645998616},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645998616}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-13T03:13:35.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.208+0000 7eff439ff700 1 -- 172.21.15.64:0/2256578416 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7eff280a0900 msgr2=0x7eff280a2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:35.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.208+0000 7eff439ff700 1 --2- 172.21.15.64:0/2256578416 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7eff280a0900 0x7eff280a2dc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7eff3c112bc0 tx=0x7eff2c009260 comp rx=0 tx=0).stop 2023-09-13T03:13:35.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.208+0000 7eff439ff700 1 -- 172.21.15.64:0/2256578416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff3c0729a0 msgr2=0x7eff3c1a2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:35.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.208+0000 7eff439ff700 1 --2- 172.21.15.64:0/2256578416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff3c0729a0 0x7eff3c1a2ac0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7eff38000c00 tx=0x7eff38005720 comp rx=0 tx=0).stop 2023-09-13T03:13:35.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.208+0000 7eff439ff700 1 -- 172.21.15.64:0/2256578416 shutdown_connections 2023-09-13T03:13:35.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.208+0000 7eff439ff700 1 --2- 172.21.15.64:0/2256578416 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7eff280a0900 0x7eff280a2dc0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.208+0000 7eff439ff700 1 --2- 172.21.15.64:0/2256578416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff3c111590 0x7eff3c19cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.208+0000 7eff439ff700 1 --2- 172.21.15.64:0/2256578416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff3c0729a0 0x7eff3c1a2ac0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.208+0000 7eff439ff700 1 --2- 172.21.15.64:0/2256578416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff3c0732c0 0x7eff3c1a3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.208+0000 7eff439ff700 1 -- 172.21.15.64:0/2256578416 >> 172.21.15.64:0/2256578416 conn(0x7eff3c0fc0b0 msgr2=0x7eff3c0fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:35.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.208+0000 7eff439ff700 1 -- 172.21.15.64:0/2256578416 shutdown_connections 2023-09-13T03:13:35.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.208+0000 7eff439ff700 1 -- 172.21.15.64:0/2256578416 wait complete. 2023-09-13T03:13:35.211 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 26 2023-09-13T03:13:35.238 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-09-13T03:13:35.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.579+0000 7f40109ff700 1 -- 172.21.15.64:0/4272461133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f400c1020f0 msgr2=0x7f400c10a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:35.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.579+0000 7f40109ff700 1 --2- 172.21.15.64:0/4272461133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f400c1020f0 0x7f400c10a5d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f40000098f0 tx=0x7f4000009c00 comp rx=0 tx=0).stop 2023-09-13T03:13:35.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.579+0000 7f40109ff700 1 -- 172.21.15.64:0/4272461133 shutdown_connections 2023-09-13T03:13:35.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.579+0000 7f40109ff700 1 --2- 172.21.15.64:0/4272461133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f400c10aba0 0x7f400c10cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.579+0000 7f40109ff700 1 --2- 172.21.15.64:0/4272461133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f400c1017d0 0x7f400c101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.579+0000 7f40109ff700 1 --2- 172.21.15.64:0/4272461133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f400c1020f0 0x7f400c10a5d0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.579+0000 7f40109ff700 1 -- 172.21.15.64:0/4272461133 >> 172.21.15.64:0/4272461133 conn(0x7f400c0faf50 msgr2=0x7f400c0fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:35.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.579+0000 7f40109ff700 1 -- 172.21.15.64:0/4272461133 shutdown_connections 2023-09-13T03:13:35.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.579+0000 7f40109ff700 1 -- 172.21.15.64:0/4272461133 wait complete. 2023-09-13T03:13:35.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.581+0000 7f40109ff700 1 Processor -- start 2023-09-13T03:13:35.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.581+0000 7f40109ff700 1 -- start start 2023-09-13T03:13:35.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.581+0000 7f40109ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f400c1017d0 0x7f400c0ff420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:35.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.581+0000 7f40109ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f400c1020f0 0x7f400c0ff960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:35.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.581+0000 7f40109ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f400c10aba0 0x7f400c1a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:35.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.581+0000 7f40109ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400c100190 con 0x7f400c10aba0 2023-09-13T03:13:35.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.581+0000 7f40109ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400c1a4e30 con 0x7f400c1017d0 2023-09-13T03:13:35.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.581+0000 7f40109ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400c1a4f70 con 0x7f400c1020f0 2023-09-13T03:13:35.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.582+0000 7f4009d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f400c1020f0 0x7f400c0ff960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:35.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.582+0000 7f400ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f400c10aba0 0x7f400c1a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:35.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.582+0000 7f4009d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f400c1020f0 0x7f400c0ff960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:47144/0 (socket says 172.21.15.64:47144) 2023-09-13T03:13:35.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.582+0000 7f4009d9b700 1 -- 172.21.15.64:0/4029831577 learned_addr learned my addr 172.21.15.64:0/4029831577 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:35.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.582+0000 7f400a59c700 1 --2- 172.21.15.64:0/4029831577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f400c1017d0 0x7f400c0ff420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:35.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.582+0000 7f4009d9b700 1 -- 172.21.15.64:0/4029831577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f400c1017d0 msgr2=0x7f400c0ff420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:35.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.583+0000 7f4009d9b700 1 --2- 172.21.15.64:0/4029831577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f400c1017d0 0x7f400c0ff420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.583+0000 7f4009d9b700 1 -- 172.21.15.64:0/4029831577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f400c10aba0 msgr2=0x7f400c1a48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:35.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.583+0000 7f4009d9b700 1 --2- 172.21.15.64:0/4029831577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f400c10aba0 0x7f400c1a48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.583+0000 7f4009d9b700 1 -- 172.21.15.64:0/4029831577 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4000009520 con 0x7f400c1020f0 2023-09-13T03:13:35.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.583+0000 7f400a59c700 1 --2- 172.21.15.64:0/4029831577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f400c1017d0 0x7f400c0ff420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:13:35.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.583+0000 7f4009d9b700 1 --2- 172.21.15.64:0/4029831577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f400c1020f0 0x7f400c0ff960 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4000009810 tx=0x7f400000fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:35.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.583+0000 7f3ffb7fe700 1 -- 172.21.15.64:0/4029831577 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400001c070 con 0x7f400c1020f0 2023-09-13T03:13:35.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.583+0000 7f3ffb7fe700 1 -- 172.21.15.64:0/4029831577 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f400000bbd0 con 0x7f400c1020f0 2023-09-13T03:13:35.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.583+0000 7f3ffb7fe700 1 -- 172.21.15.64:0/4029831577 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4000004930 con 0x7f400c1020f0 2023-09-13T03:13:35.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.583+0000 7f40109ff700 1 -- 172.21.15.64:0/4029831577 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f400c1a5170 con 0x7f400c1020f0 2023-09-13T03:13:35.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.584+0000 7f40109ff700 1 -- 172.21.15.64:0/4029831577 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f400c1a55a0 con 0x7f400c1020f0 2023-09-13T03:13:35.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.588+0000 7f3ffb7fe700 1 -- 172.21.15.64:0/4029831577 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 24) v1 ==== 135435+0+0 (secure 0 0 0) 0x7f4000020070 con 0x7f400c1020f0 2023-09-13T03:13:35.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.591+0000 7f3ffb7fe700 1 --2- 172.21.15.64:0/4029831577 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f3ff40a0840 0x7f3ff40a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:35.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.591+0000 7f4024373b80 1 Processor -- start 2023-09-13T03:13:35.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.591+0000 7f3ffb7fe700 1 -- 172.21.15.64:0/4029831577 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f40000ccd60 con 0x7f400c1020f0 2023-09-13T03:13:35.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.591+0000 7f4024373b80 1 -- start start 2023-09-13T03:13:35.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.591+0000 7f400a59c700 1 --2- 172.21.15.64:0/4029831577 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f3ff40a0840 0x7f3ff40a2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:35.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.591+0000 7f4024373b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582ca410f10 0x5582ca4112f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:35.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.591+0000 7f4024373b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582ca411830 0x5582ca41c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:35.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.592+0000 7f4024373b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582ca41c660 0x5582ca41ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:35.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.592+0000 7f4024373b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582ca362730 con 0x5582ca41c660 2023-09-13T03:13:35.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.592+0000 7f4024373b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582ca360050 con 0x5582ca410f10 2023-09-13T03:13:35.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.592+0000 7f4024373b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582ca35b940 con 0x5582ca411830 2023-09-13T03:13:35.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.592+0000 7f4009d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582ca411830 0x5582ca41c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:35.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.592+0000 7f4009d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582ca411830 0x5582ca41c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:47160/0 (socket says 172.21.15.64:47160) 2023-09-13T03:13:35.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.592+0000 7f4009d9b700 1 -- 172.21.15.64:0/1172206615 learned_addr learned my addr 172.21.15.64:0/1172206615 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:35.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.592+0000 7f400ad9d700 1 --2- 172.21.15.64:0/1172206615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582ca41c660 0x5582ca41ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:35.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.592+0000 7f4009d9b700 1 -- 172.21.15.64:0/1172206615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582ca410f10 msgr2=0x5582ca4112f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:35.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.592+0000 7f4009d9b700 1 --2- 172.21.15.64:0/1172206615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582ca410f10 0x5582ca4112f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.592+0000 7f4009d9b700 1 -- 172.21.15.64:0/1172206615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582ca41c660 msgr2=0x5582ca41ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:35.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.592+0000 7f400a59c700 1 --2- 172.21.15.64:0/4029831577 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f3ff40a0840 0x7f3ff40a2d00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc005fd0 tx=0x7f3ffc009310 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:35.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.593+0000 7f4009d9b700 1 --2- 172.21.15.64:0/1172206615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582ca41c660 0x5582ca41ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.593+0000 7f4009d9b700 1 -- 172.21.15.64:0/1172206615 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582ca187de0 con 0x5582ca411830 2023-09-13T03:13:35.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.593+0000 7f400ad9d700 1 --2- 172.21.15.64:0/1172206615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582ca41c660 0x5582ca41ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:13:35.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.593+0000 7f4009d9b700 1 --2- 172.21.15.64:0/1172206615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582ca411830 0x5582ca41c120 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4000099e20 tx=0x7f4000009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ea1f31f3a3bd3de server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:35.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.594+0000 7f3ff8ff9700 1 -- 172.21.15.64:0/1172206615 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400009c070 con 0x5582ca411830 2023-09-13T03:13:35.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.594+0000 7f3ff8ff9700 1 -- 172.21.15.64:0/1172206615 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f400009a120 con 0x5582ca411830 2023-09-13T03:13:35.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.594+0000 7f3ff8ff9700 1 -- 172.21.15.64:0/1172206615 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400009a5e0 con 0x5582ca411830 2023-09-13T03:13:35.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.595+0000 7f4024373b80 1 -- 172.21.15.64:0/1172206615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582ca411830 msgr2=0x5582ca41c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:35.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.595+0000 7f4024373b80 1 --2- 172.21.15.64:0/1172206615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582ca411830 0x5582ca41c120 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4000099e20 tx=0x7f4000009260 comp rx=0 tx=0).stop 2023-09-13T03:13:35.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.595+0000 7f4024373b80 1 -- 172.21.15.64:0/1172206615 shutdown_connections 2023-09-13T03:13:35.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.595+0000 7f4024373b80 1 --2- 172.21.15.64:0/1172206615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582ca410f10 0x5582ca4112f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.595+0000 7f4024373b80 1 --2- 172.21.15.64:0/1172206615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582ca41c660 0x5582ca41ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.595+0000 7f4024373b80 1 --2- 172.21.15.64:0/1172206615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582ca411830 0x5582ca41c120 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.595+0000 7f4024373b80 1 -- 172.21.15.64:0/1172206615 >> 172.21.15.64:0/1172206615 conn(0x5582ca37e090 msgr2=0x5582ca404a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:35.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.595+0000 7f4024373b80 1 -- 172.21.15.64:0/1172206615 shutdown_connections 2023-09-13T03:13:35.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.595+0000 7f4024373b80 1 -- 172.21.15.64:0/1172206615 wait complete. 2023-09-13T03:13:35.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.596+0000 7f4024373b80 1 -- start start 2023-09-13T03:13:35.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.596+0000 7f4024373b80 1 Processor -- start 2023-09-13T03:13:35.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.597+0000 7f4024373b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582ca410f10 0x5582ca41ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:35.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.597+0000 7f4024373b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582ca411830 0x5582ca41cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:35.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.597+0000 7f4024373b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582ca41c660 0x5582ca427ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:35.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.597+0000 7f4024373b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582ca2f9b70 con 0x5582ca41c660 2023-09-13T03:13:35.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.597+0000 7f4024373b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582ca2f9870 con 0x5582ca411830 2023-09-13T03:13:35.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.597+0000 7f4024373b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582ca1dd920 con 0x5582ca410f10 2023-09-13T03:13:35.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.597+0000 7f4024373b80 10 client.0 _ll_register_callbacks cb 0x5582ca243a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:13:35.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.597+0000 7f400ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582ca41c660 0x5582ca427ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:35.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.597+0000 7f4009d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582ca411830 0x5582ca41cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:35.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.598+0000 7f400a59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582ca410f10 0x5582ca41ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:35.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.598+0000 7f400ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582ca41c660 0x5582ca427ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39270/0 (socket says 172.21.15.64:39270) 2023-09-13T03:13:35.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.598+0000 7f400ad9d700 1 -- 172.21.15.64:0/2589385050 learned_addr learned my addr 172.21.15.64:0/2589385050 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:35.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.598+0000 7f4009d9b700 1 -- 172.21.15.64:0/2589385050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582ca410f10 msgr2=0x5582ca41ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:35.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.598+0000 7f4009d9b700 1 --2- 172.21.15.64:0/2589385050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582ca410f10 0x5582ca41ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.598+0000 7f4009d9b700 1 -- 172.21.15.64:0/2589385050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582ca41c660 msgr2=0x5582ca427ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:35.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.598+0000 7f4009d9b700 1 --2- 172.21.15.64:0/2589385050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582ca41c660 0x5582ca427ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.598+0000 7f4009d9b700 1 -- 172.21.15.64:0/2589385050 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f400c1a55a0 con 0x5582ca411830 2023-09-13T03:13:35.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.598+0000 7f400a59c700 1 --2- 172.21.15.64:0/2589385050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582ca410f10 0x5582ca41ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:13:35.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.598+0000 7f4009d9b700 1 --2- 172.21.15.64:0/2589385050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582ca411830 0x5582ca41cf80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f400009a0b0 tx=0x7f400000fa70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23b21356715fc42 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:35.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.598+0000 7f3ff1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:13:35.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.599+0000 7f3ff1ffb700 1 -- 172.21.15.64:0/2589385050 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400009c040 con 0x5582ca411830 2023-09-13T03:13:35.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.599+0000 7f3ff1ffb700 1 -- 172.21.15.64:0/2589385050 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f40000a2100 con 0x5582ca411830 2023-09-13T03:13:35.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.599+0000 7f4024373b80 1 -- 172.21.15.64:0/2589385050 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5582ca2b34f0 con 0x5582ca411830 2023-09-13T03:13:35.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.599+0000 7f3ff1ffb700 1 -- 172.21.15.64:0/2589385050 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f400009df50 con 0x5582ca411830 2023-09-13T03:13:35.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.599+0000 7f4024373b80 10 client.25775 fetch_fsmap learned FSMap version 26 2023-09-13T03:13:35.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.599+0000 7f4024373b80 1 -- 172.21.15.64:0/2589385050 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5582ca170c60 con 0x5582ca411830 2023-09-13T03:13:35.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.599+0000 7f3ff1ffb700 1 -- 172.21.15.64:0/2589385050 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5582ca2b34f0 con 0x5582ca411830 2023-09-13T03:13:35.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.600+0000 7f3ff1ffb700 1 -- 172.21.15.64:0/2589385050 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f40000a1d90 con 0x5582ca411830 2023-09-13T03:13:35.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.600+0000 7f4024373b80 10 client.25775 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:13:35.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.600+0000 7f4024373b80 10 client.25775 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2023-09-13T03:13:35.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.600+0000 7f4024373b80 20 client.25775 populate_metadata read hostname 'smithi064' 2023-09-13T03:13:35.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.600+0000 7f4024373b80 1 --2- 172.21.15.64:0/2589385050 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x5582ca42b920 0x5582ca42dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:35.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.600+0000 7f4024373b80 4 client.25775 mds_command: new command op to 24622 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:13:35.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.600+0000 7f4024373b80 1 -- 172.21.15.64:0/2589385050 --> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5582ca3708a0 con 0x5582ca42b920 2023-09-13T03:13:35.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.600+0000 7f400a59c700 1 --2- 172.21.15.64:0/2589385050 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x5582ca42b920 0x5582ca42dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:35.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.601+0000 7f400a59c700 1 --2- 172.21.15.64:0/2589385050 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x5582ca42b920 0x5582ca42dd10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7fce0f7b637f7e3 server_cookie=9c54b3aa886e9aaf in_seq=0 out_seq=0 2023-09-13T03:13:35.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.601+0000 7f3ff1ffb700 10 client.25775 ms_handle_connect on v2:172.21.15.170:6834/1493371121 2023-09-13T03:13:35.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.602+0000 7f3ff1ffb700 1 -- 172.21.15.64:0/2589385050 <== mds.4 v2:172.21.15.170:6834/1493371121 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5582ca3708a0 con 0x5582ca42b920 2023-09-13T03:13:35.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.602+0000 7f3ff1ffb700 10 client.25775 handle_command_reply: tid=0 2023-09-13T03:13:35.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.602+0000 7f4024373b80 1 client.25775 shutdown 2023-09-13T03:13:35.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.602+0000 7f4024373b80 1 -- 172.21.15.64:0/2589385050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582ca411830 msgr2=0x5582ca41cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:35.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.602+0000 7f4024373b80 1 --2- 172.21.15.64:0/2589385050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582ca411830 0x5582ca41cf80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f400009a0b0 tx=0x7f400000fa70 comp rx=0 tx=0).stop 2023-09-13T03:13:35.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.602+0000 7f4024373b80 1 -- 172.21.15.64:0/2589385050 shutdown_connections 2023-09-13T03:13:35.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.602+0000 7f4024373b80 1 --2- 172.21.15.64:0/2589385050 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x5582ca42b920 0x5582ca42dd10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.602+0000 7f4024373b80 1 --2- 172.21.15.64:0/2589385050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582ca41c660 0x5582ca427ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.602+0000 7f4024373b80 1 --2- 172.21.15.64:0/2589385050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582ca410f10 0x5582ca41ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.602+0000 7f4024373b80 1 --2- 172.21.15.64:0/2589385050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582ca411830 0x5582ca41cf80 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.602+0000 7f4024373b80 1 -- 172.21.15.64:0/2589385050 >> 172.21.15.64:0/2589385050 conn(0x5582ca37e090 msgr2=0x5582ca4046a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:35.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.602+0000 7f3ff1ffb700 0 client.25775 ms_handle_reset on v2:172.21.15.170:6834/1493371121 2023-09-13T03:13:35.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.602+0000 7f4024373b80 1 -- 172.21.15.64:0/2589385050 shutdown_connections 2023-09-13T03:13:35.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.602+0000 7f4024373b80 1 -- 172.21.15.64:0/2589385050 wait complete. 2023-09-13T03:13:35.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.603+0000 7f4024373b80 20 client.25775 trim_cache size 0 max 16384 2023-09-13T03:13:35.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.628+0000 7f4024373b80 1 Processor -- start 2023-09-13T03:13:35.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.629+0000 7f4024373b80 1 -- start start 2023-09-13T03:13:35.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.629+0000 7f4024373b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582ca410f10 0x5582ca403dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:35.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.629+0000 7f4024373b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582ca411830 0x5582ca416af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:35.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.629+0000 7f4024373b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582ca41c660 0x5582ca417030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:35.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.629+0000 7f4024373b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582ca412110 con 0x5582ca41c660 2023-09-13T03:13:35.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.629+0000 7f4024373b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582ca4013d0 con 0x5582ca411830 2023-09-13T03:13:35.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.629+0000 7f4024373b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582ca41d2f0 con 0x5582ca410f10 2023-09-13T03:13:35.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.629+0000 7f4009d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582ca411830 0x5582ca416af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:35.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.630+0000 7f4009d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582ca411830 0x5582ca416af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:58262/0 (socket says 172.21.15.64:58262) 2023-09-13T03:13:35.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.630+0000 7f4009d9b700 1 -- 172.21.15.64:0/1413414458 learned_addr learned my addr 172.21.15.64:0/1413414458 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:35.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.630+0000 7f400ad9d700 1 --2- 172.21.15.64:0/1413414458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582ca41c660 0x5582ca417030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:35.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.630+0000 7f4009d9b700 1 -- 172.21.15.64:0/1413414458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582ca410f10 msgr2=0x5582ca403dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:35.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.630+0000 7f400a59c700 1 --2- 172.21.15.64:0/1413414458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582ca410f10 0x5582ca403dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:35.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.630+0000 7f4009d9b700 1 --2- 172.21.15.64:0/1413414458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582ca410f10 0x5582ca403dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.630+0000 7f4009d9b700 1 -- 172.21.15.64:0/1413414458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582ca41c660 msgr2=0x5582ca417030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:35.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.630+0000 7f4009d9b700 1 --2- 172.21.15.64:0/1413414458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582ca41c660 0x5582ca417030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.630+0000 7f4009d9b700 1 -- 172.21.15.64:0/1413414458 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582ca1ae250 con 0x5582ca411830 2023-09-13T03:13:35.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.630+0000 7f4009d9b700 1 --2- 172.21.15.64:0/1413414458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582ca411830 0x5582ca416af0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f40000098c0 tx=0x7f4000009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a350cd778e9c264 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:35.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.631+0000 7f3ff97fa700 1 -- 172.21.15.64:0/1413414458 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400009c070 con 0x5582ca411830 2023-09-13T03:13:35.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.631+0000 7f3ff97fa700 1 -- 172.21.15.64:0/1413414458 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f400009df50 con 0x5582ca411830 2023-09-13T03:13:35.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.631+0000 7f3ff97fa700 1 -- 172.21.15.64:0/1413414458 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4000098010 con 0x5582ca411830 2023-09-13T03:13:35.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.631+0000 7f4024373b80 1 -- 172.21.15.64:0/1413414458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582ca411830 msgr2=0x5582ca416af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:35.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.631+0000 7f4024373b80 1 --2- 172.21.15.64:0/1413414458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582ca411830 0x5582ca416af0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f40000098c0 tx=0x7f4000009c00 comp rx=0 tx=0).stop 2023-09-13T03:13:35.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.632+0000 7f4024373b80 1 -- 172.21.15.64:0/1413414458 shutdown_connections 2023-09-13T03:13:35.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.632+0000 7f4024373b80 1 --2- 172.21.15.64:0/1413414458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582ca41c660 0x5582ca417030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.632+0000 7f4024373b80 1 --2- 172.21.15.64:0/1413414458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582ca410f10 0x5582ca403dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.632+0000 7f4024373b80 1 --2- 172.21.15.64:0/1413414458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582ca411830 0x5582ca416af0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.632+0000 7f4024373b80 1 -- 172.21.15.64:0/1413414458 >> 172.21.15.64:0/1413414458 conn(0x5582ca37e090 msgr2=0x5582ca4160d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:35.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.632+0000 7f4024373b80 1 -- 172.21.15.64:0/1413414458 shutdown_connections 2023-09-13T03:13:35.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.632+0000 7f4024373b80 1 -- 172.21.15.64:0/1413414458 wait complete. 2023-09-13T03:13:35.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.632+0000 7f4024373b80 1 -- start start 2023-09-13T03:13:35.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.632+0000 7f4024373b80 1 Processor -- start 2023-09-13T03:13:35.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.633+0000 7f4024373b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582ca410f10 0x5582ca421720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:35.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.633+0000 7f4024373b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582ca411830 0x5582ca41fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:35.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.633+0000 7f4024373b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582ca41c660 0x5582ca41ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:35.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.633+0000 7f4024373b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582ca4204d0 con 0x5582ca41c660 2023-09-13T03:13:35.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.633+0000 7f4024373b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582ca420640 con 0x5582ca410f10 2023-09-13T03:13:35.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.633+0000 7f4024373b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582ca420800 con 0x5582ca411830 2023-09-13T03:13:35.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.633+0000 7f4024373b80 10 client.0 _ll_register_callbacks cb 0x7f4000099ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:13:35.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.633+0000 7f400ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582ca41c660 0x5582ca41ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:35.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.634+0000 7f400ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582ca41c660 0x5582ca41ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39282/0 (socket says 172.21.15.64:39282) 2023-09-13T03:13:35.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.634+0000 7f400ad9d700 1 -- 172.21.15.64:0/759637443 learned_addr learned my addr 172.21.15.64:0/759637443 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:35.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.634+0000 7f4009d9b700 1 --2- 172.21.15.64:0/759637443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582ca411830 0x5582ca41fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:35.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.634+0000 7f400ad9d700 1 -- 172.21.15.64:0/759637443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582ca411830 msgr2=0x5582ca41fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:35.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.634+0000 7f400ad9d700 1 --2- 172.21.15.64:0/759637443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582ca411830 0x5582ca41fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.634+0000 7f400ad9d700 1 -- 172.21.15.64:0/759637443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582ca410f10 msgr2=0x5582ca421720 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-13T03:13:35.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.634+0000 7f400ad9d700 1 --2- 172.21.15.64:0/759637443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582ca410f10 0x5582ca421720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.634+0000 7f400ad9d700 1 -- 172.21.15.64:0/759637443 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5582ca170c60 con 0x5582ca41c660 2023-09-13T03:13:35.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.635+0000 7f400ad9d700 1 --2- 172.21.15.64:0/759637443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582ca41c660 0x5582ca41ff90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4004007b50 tx=0x7f4004007b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b293f3115428fbed server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:35.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.635+0000 7f3ff1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:13:35.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.635+0000 7f3ff1ffb700 1 -- 172.21.15.64:0/759637443 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400400e040 con 0x5582ca41c660 2023-09-13T03:13:35.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.635+0000 7f3ff1ffb700 1 -- 172.21.15.64:0/759637443 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f4004003c00 con 0x5582ca41c660 2023-09-13T03:13:35.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.635+0000 7f4024373b80 1 -- 172.21.15.64:0/759637443 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f400009a120 con 0x5582ca41c660 2023-09-13T03:13:35.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.635+0000 7f3ff1ffb700 1 -- 172.21.15.64:0/759637443 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4004003dc0 con 0x5582ca41c660 2023-09-13T03:13:35.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.635+0000 7f4024373b80 10 client.15753 fetch_fsmap learned FSMap version 26 2023-09-13T03:13:35.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.635+0000 7f4024373b80 1 -- 172.21.15.64:0/759637443 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5582ca1862b0 con 0x5582ca41c660 2023-09-13T03:13:35.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.636+0000 7f3ff1ffb700 1 -- 172.21.15.64:0/759637443 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f400009a120 con 0x5582ca41c660 2023-09-13T03:13:35.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.636+0000 7f3ff1ffb700 1 -- 172.21.15.64:0/759637443 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f40040045f0 con 0x5582ca41c660 2023-09-13T03:13:35.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.636+0000 7f4024373b80 10 client.15753 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:13:35.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.636+0000 7f4024373b80 10 client.15753 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2023-09-13T03:13:35.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.636+0000 7f4024373b80 20 client.15753 populate_metadata read hostname 'smithi064' 2023-09-13T03:13:35.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.636+0000 7f4024373b80 1 --2- 172.21.15.64:0/759637443 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x5582ca480c10 0x5582ca483030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:35.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.636+0000 7f4024373b80 4 client.15753 mds_command: new command op to 24622 tid=0[{"prefix": "damage ls"}] 2023-09-13T03:13:35.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.636+0000 7f4024373b80 1 -- 172.21.15.64:0/759637443 --> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5582ca354920 con 0x5582ca480c10 2023-09-13T03:13:35.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.637+0000 7f400a59c700 1 --2- 172.21.15.64:0/759637443 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x5582ca480c10 0x5582ca483030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:35.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.637+0000 7f3ff1ffb700 10 client.15753 ms_handle_connect on v2:172.21.15.170:6834/1493371121 2023-09-13T03:13:35.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.637+0000 7f400a59c700 1 --2- 172.21.15.64:0/759637443 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x5582ca480c10 0x5582ca483030 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=83d27140cacb10e7 server_cookie=1407117467f9ff5f in_seq=0 out_seq=0 2023-09-13T03:13:35.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.638+0000 7f3ff1ffb700 1 -- 172.21.15.64:0/759637443 <== mds.4 v2:172.21.15.170:6834/1493371121 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5582ca354920 con 0x5582ca480c10 2023-09-13T03:13:35.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.638+0000 7f3ff1ffb700 10 client.15753 handle_command_reply: tid=0 2023-09-13T03:13:35.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.638+0000 7f4024373b80 1 client.15753 shutdown 2023-09-13T03:13:35.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.638+0000 7f4024373b80 1 -- 172.21.15.64:0/759637443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582ca41c660 msgr2=0x5582ca41ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:35.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.638+0000 7f4024373b80 1 --2- 172.21.15.64:0/759637443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582ca41c660 0x5582ca41ff90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4004007b50 tx=0x7f4004007b80 comp rx=0 tx=0).stop 2023-09-13T03:13:35.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.638+0000 7f4024373b80 1 -- 172.21.15.64:0/759637443 shutdown_connections 2023-09-13T03:13:35.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.638+0000 7f4024373b80 1 --2- 172.21.15.64:0/759637443 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x5582ca480c10 0x5582ca483030 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.638+0000 7f4024373b80 1 --2- 172.21.15.64:0/759637443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582ca410f10 0x5582ca421720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.638+0000 7f4024373b80 1 --2- 172.21.15.64:0/759637443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582ca41c660 0x5582ca41ff90 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.638+0000 7f4024373b80 1 --2- 172.21.15.64:0/759637443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582ca411830 0x5582ca41fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.638+0000 7f4024373b80 1 -- 172.21.15.64:0/759637443 >> 172.21.15.64:0/759637443 conn(0x5582ca37e090 msgr2=0x5582ca416090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:35.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.638+0000 7f3ff1ffb700 0 client.15753 ms_handle_reset on v2:172.21.15.170:6834/1493371121 2023-09-13T03:13:35.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.638+0000 7f4024373b80 1 -- 172.21.15.64:0/759637443 shutdown_connections 2023-09-13T03:13:35.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.638+0000 7f4024373b80 1 -- 172.21.15.64:0/759637443 wait complete. 2023-09-13T03:13:35.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.639+0000 7f4024373b80 20 client.15753 trim_cache size 0 max 16384 2023-09-13T03:13:35.650 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2023-09-13T03:13:35.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.639+0000 7f40109ff700 1 -- 172.21.15.64:0/4029831577 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f3ff40a0840 msgr2=0x7f3ff40a2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:35.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.639+0000 7f40109ff700 1 --2- 172.21.15.64:0/4029831577 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f3ff40a0840 0x7f3ff40a2d00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc005fd0 tx=0x7f3ffc009310 comp rx=0 tx=0).stop 2023-09-13T03:13:35.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.639+0000 7f40109ff700 1 -- 172.21.15.64:0/4029831577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f400c1020f0 msgr2=0x7f400c0ff960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:35.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.639+0000 7f40109ff700 1 --2- 172.21.15.64:0/4029831577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f400c1020f0 0x7f400c0ff960 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4000009810 tx=0x7f400000fa00 comp rx=0 tx=0).stop 2023-09-13T03:13:35.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.639+0000 7f40109ff700 1 -- 172.21.15.64:0/4029831577 shutdown_connections 2023-09-13T03:13:35.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.640+0000 7f40109ff700 1 --2- 172.21.15.64:0/4029831577 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f3ff40a0840 0x7f3ff40a2d00 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.640+0000 7f40109ff700 1 --2- 172.21.15.64:0/4029831577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f400c1017d0 0x7f400c0ff420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.640+0000 7f40109ff700 1 --2- 172.21.15.64:0/4029831577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f400c10aba0 0x7f400c1a48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.640+0000 7f40109ff700 1 --2- 172.21.15.64:0/4029831577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f400c1020f0 0x7f400c0ff960 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.640+0000 7f40109ff700 1 -- 172.21.15.64:0/4029831577 >> 172.21.15.64:0/4029831577 conn(0x7f400c0faf50 msgr2=0x7f400c0fbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:35.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.640+0000 7f40109ff700 1 -- 172.21.15.64:0/4029831577 shutdown_connections 2023-09-13T03:13:35.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.640+0000 7f40109ff700 1 -- 172.21.15.64:0/4029831577 wait complete. 2023-09-13T03:13:35.668 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-09-13T03:13:35.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.890+0000 7f5613fff700 1 -- 172.21.15.64:0/3883602987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f560c1020d0 msgr2=0x7f560c10a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:35.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.890+0000 7f5613fff700 1 --2- 172.21.15.64:0/3883602987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f560c1020d0 0x7f560c10a5b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f55fc0098f0 tx=0x7f55fc009c00 comp rx=0 tx=0).stop 2023-09-13T03:13:35.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.890+0000 7f5613fff700 1 -- 172.21.15.64:0/3883602987 shutdown_connections 2023-09-13T03:13:35.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.890+0000 7f5613fff700 1 --2- 172.21.15.64:0/3883602987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f560c10ab80 0x7f560c10cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.890+0000 7f5613fff700 1 --2- 172.21.15.64:0/3883602987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f560c1017b0 0x7f560c101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.890+0000 7f5613fff700 1 --2- 172.21.15.64:0/3883602987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f560c1020d0 0x7f560c10a5b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.890+0000 7f5613fff700 1 -- 172.21.15.64:0/3883602987 >> 172.21.15.64:0/3883602987 conn(0x7f560c0faf50 msgr2=0x7f560c0fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:35.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.890+0000 7f5613fff700 1 -- 172.21.15.64:0/3883602987 shutdown_connections 2023-09-13T03:13:35.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.890+0000 7f5613fff700 1 -- 172.21.15.64:0/3883602987 wait complete. 2023-09-13T03:13:35.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.891+0000 7f5613fff700 1 Processor -- start 2023-09-13T03:13:35.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.891+0000 7f5613fff700 1 -- start start 2023-09-13T03:13:35.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.892+0000 7f5613fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f560c1017b0 0x7f560c19e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:35.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.892+0000 7f5613fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f560c1020d0 0x7f560c19ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:35.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.892+0000 7f5613fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f560c10ab80 0x7f560c198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:35.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.892+0000 7f5613fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560c198de0 con 0x7f560c1017b0 2023-09-13T03:13:35.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.892+0000 7f5613fff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560c198f50 con 0x7f560c10ab80 2023-09-13T03:13:35.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.892+0000 7f5613fff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560c199110 con 0x7f560c1020d0 2023-09-13T03:13:35.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.892+0000 7f561159a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f560c1020d0 0x7f560c19ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:35.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.893+0000 7f561159a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f560c1020d0 0x7f560c19ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:54256/0 (socket says 172.21.15.64:54256) 2023-09-13T03:13:35.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.893+0000 7f561159a700 1 -- 172.21.15.64:0/3713426427 learned_addr learned my addr 172.21.15.64:0/3713426427 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:35.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.893+0000 7f561259c700 1 --2- 172.21.15.64:0/3713426427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f560c10ab80 0x7f560c198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:35.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.893+0000 7f5611d9b700 1 --2- 172.21.15.64:0/3713426427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f560c1017b0 0x7f560c19e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:35.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.893+0000 7f561159a700 1 -- 172.21.15.64:0/3713426427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f560c10ab80 msgr2=0x7f560c198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:35.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.893+0000 7f561159a700 1 --2- 172.21.15.64:0/3713426427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f560c10ab80 0x7f560c198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.893+0000 7f561159a700 1 -- 172.21.15.64:0/3713426427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f560c1017b0 msgr2=0x7f560c19e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:35.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.893+0000 7f561159a700 1 --2- 172.21.15.64:0/3713426427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f560c1017b0 0x7f560c19e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.893+0000 7f561159a700 1 -- 172.21.15.64:0/3713426427 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55fc009520 con 0x7f560c1020d0 2023-09-13T03:13:35.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.893+0000 7f561159a700 1 --2- 172.21.15.64:0/3713426427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f560c1020d0 0x7f560c19ec40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f55fc009810 tx=0x7f55fc00fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:35.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.894+0000 7f5602ffd700 1 -- 172.21.15.64:0/3713426427 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55fc01c070 con 0x7f560c1020d0 2023-09-13T03:13:35.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.894+0000 7f5602ffd700 1 -- 172.21.15.64:0/3713426427 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55fc00bbd0 con 0x7f560c1020d0 2023-09-13T03:13:35.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.894+0000 7f5613fff700 1 -- 172.21.15.64:0/3713426427 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f560c199390 con 0x7f560c1020d0 2023-09-13T03:13:35.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.894+0000 7f5602ffd700 1 -- 172.21.15.64:0/3713426427 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55fc0049c0 con 0x7f560c1020d0 2023-09-13T03:13:35.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.894+0000 7f5613fff700 1 -- 172.21.15.64:0/3713426427 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f560c1a56f0 con 0x7f560c1020d0 2023-09-13T03:13:35.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.899+0000 7f562796cb80 1 Processor -- start 2023-09-13T03:13:35.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.899+0000 7f5602ffd700 1 -- 172.21.15.64:0/3713426427 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 24) v1 ==== 135435+0+0 (secure 0 0 0) 0x7f55fc020070 con 0x7f560c1020d0 2023-09-13T03:13:35.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.899+0000 7f562796cb80 1 -- start start 2023-09-13T03:13:35.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.900+0000 7f562796cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be85b38f10 0x55be85b392f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:35.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.900+0000 7f562796cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be85b39830 0x55be85b44120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:35.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.900+0000 7f562796cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be85b44660 0x55be85b46a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:35.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.900+0000 7f562796cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be85a8a730 con 0x55be85b38f10 2023-09-13T03:13:35.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.900+0000 7f562796cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be85a88050 con 0x55be85b39830 2023-09-13T03:13:35.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.900+0000 7f562796cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be85a83940 con 0x55be85b44660 2023-09-13T03:13:35.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.900+0000 7f5611d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be85b39830 0x55be85b44120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:35.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.900+0000 7f5611d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be85b39830 0x55be85b44120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:57172/0 (socket says 172.21.15.64:57172) 2023-09-13T03:13:35.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.900+0000 7f5611d9b700 1 -- 172.21.15.64:0/2637082406 learned_addr learned my addr 172.21.15.64:0/2637082406 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:35.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.900+0000 7f561159a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be85b44660 0x55be85b46a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:35.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.901+0000 7f561259c700 1 --2- 172.21.15.64:0/2637082406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be85b38f10 0x55be85b392f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:35.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.901+0000 7f5611d9b700 1 -- 172.21.15.64:0/2637082406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be85b44660 msgr2=0x55be85b46a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:35.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.901+0000 7f5611d9b700 1 --2- 172.21.15.64:0/2637082406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be85b44660 0x55be85b46a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.901+0000 7f5611d9b700 1 -- 172.21.15.64:0/2637082406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be85b38f10 msgr2=0x55be85b392f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:35.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.901+0000 7f5611d9b700 1 --2- 172.21.15.64:0/2637082406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be85b38f10 0x55be85b392f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.901+0000 7f5611d9b700 1 -- 172.21.15.64:0/2637082406 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be858afde0 con 0x55be85b39830 2023-09-13T03:13:35.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.901+0000 7f561159a700 1 --2- 172.21.15.64:0/2637082406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be85b44660 0x55be85b46a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:13:35.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.901+0000 7f5602ffd700 1 --2- 172.21.15.64:0/3713426427 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f55f80a0940 0x7f55f80a2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:35.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.902+0000 7f5602ffd700 1 -- 172.21.15.64:0/3713426427 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f55fc0ccdf0 con 0x7f560c1020d0 2023-09-13T03:13:35.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.902+0000 7f561259c700 1 --2- 172.21.15.64:0/3713426427 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f55f80a0940 0x7f55f80a2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:35.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.902+0000 7f5611d9b700 1 --2- 172.21.15.64:0/2637082406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be85b39830 0x55be85b44120 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5608009d10 tx=0x7f56080116c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbc2b622e980637f server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:35.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.902+0000 7f55f7fff700 1 -- 172.21.15.64:0/2637082406 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56080097e0 con 0x55be85b39830 2023-09-13T03:13:35.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.902+0000 7f55f7fff700 1 -- 172.21.15.64:0/2637082406 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5608009e60 con 0x55be85b39830 2023-09-13T03:13:35.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.902+0000 7f55f7fff700 1 -- 172.21.15.64:0/2637082406 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560800f800 con 0x55be85b39830 2023-09-13T03:13:35.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.903+0000 7f562796cb80 1 -- 172.21.15.64:0/2637082406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be85b39830 msgr2=0x55be85b44120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:35.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.903+0000 7f561259c700 1 --2- 172.21.15.64:0/3713426427 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f55f80a0940 0x7f55f80a2e00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f560c19f400 tx=0x7f560400b040 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:35.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.903+0000 7f562796cb80 1 --2- 172.21.15.64:0/2637082406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be85b39830 0x55be85b44120 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5608009d10 tx=0x7f56080116c0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.904+0000 7f562796cb80 1 -- 172.21.15.64:0/2637082406 shutdown_connections 2023-09-13T03:13:35.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.904+0000 7f562796cb80 1 --2- 172.21.15.64:0/2637082406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be85b44660 0x55be85b46a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.904+0000 7f562796cb80 1 --2- 172.21.15.64:0/2637082406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be85b38f10 0x55be85b392f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.904+0000 7f562796cb80 1 --2- 172.21.15.64:0/2637082406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be85b39830 0x55be85b44120 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.904+0000 7f562796cb80 1 -- 172.21.15.64:0/2637082406 >> 172.21.15.64:0/2637082406 conn(0x55be85aa6090 msgr2=0x55be85b2ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:35.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.904+0000 7f562796cb80 1 -- 172.21.15.64:0/2637082406 shutdown_connections 2023-09-13T03:13:35.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.904+0000 7f562796cb80 1 -- 172.21.15.64:0/2637082406 wait complete. 2023-09-13T03:13:35.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.904+0000 7f562796cb80 1 -- start start 2023-09-13T03:13:35.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.905+0000 7f562796cb80 1 Processor -- start 2023-09-13T03:13:35.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.906+0000 7f562796cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be85b38f10 0x55be85b440c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:35.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.906+0000 7f562796cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be85b39830 0x55be85b41340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:35.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.906+0000 7f562796cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be85b44660 0x55be85b41a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:35.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.906+0000 7f562796cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be85a21b70 con 0x55be85b39830 2023-09-13T03:13:35.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.906+0000 7f562796cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be85a21870 con 0x55be85b44660 2023-09-13T03:13:35.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.906+0000 7f562796cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be85905920 con 0x55be85b38f10 2023-09-13T03:13:35.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.906+0000 7f562796cb80 10 client.0 _ll_register_callbacks cb 0x55be8596ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:13:35.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.906+0000 7f561259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be85b44660 0x55be85b41a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:35.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.906+0000 7f561259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be85b44660 0x55be85b41a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:57178/0 (socket says 172.21.15.64:57178) 2023-09-13T03:13:35.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.906+0000 7f561259c700 1 -- 172.21.15.64:0/3235938947 learned_addr learned my addr 172.21.15.64:0/3235938947 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:35.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.906+0000 7f561159a700 1 --2- 172.21.15.64:0/3235938947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be85b39830 0x55be85b41340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:35.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.906+0000 7f5611d9b700 1 --2- 172.21.15.64:0/3235938947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be85b38f10 0x55be85b440c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:35.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.907+0000 7f561259c700 1 -- 172.21.15.64:0/3235938947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be85b38f10 msgr2=0x55be85b440c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:35.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.907+0000 7f561259c700 1 --2- 172.21.15.64:0/3235938947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be85b38f10 0x55be85b440c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.907+0000 7f561259c700 1 -- 172.21.15.64:0/3235938947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be85b39830 msgr2=0x55be85b41340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:35.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.907+0000 7f561259c700 1 --2- 172.21.15.64:0/3235938947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be85b39830 0x55be85b41340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.907+0000 7f561259c700 1 -- 172.21.15.64:0/3235938947 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f560c1a56f0 con 0x55be85b44660 2023-09-13T03:13:35.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.907+0000 7f561259c700 1 --2- 172.21.15.64:0/3235938947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be85b44660 0x55be85b41a80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f56040169e0 tx=0x7f5604004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80b4f5eb4e28a67e server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:35.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.907+0000 7f55f57fa700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:13:35.914 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:35 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/2256578416' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:13:35.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.907+0000 7f55f57fa700 1 -- 172.21.15.64:0/3235938947 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560400e040 con 0x55be85b44660 2023-09-13T03:13:35.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.907+0000 7f562796cb80 1 -- 172.21.15.64:0/3235938947 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be859db4f0 con 0x55be85b44660 2023-09-13T03:13:35.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.907+0000 7f55f57fa700 1 -- 172.21.15.64:0/3235938947 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f5604015a20 con 0x55be85b44660 2023-09-13T03:13:35.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.908+0000 7f55f57fa700 1 -- 172.21.15.64:0/3235938947 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5604016e50 con 0x55be85b44660 2023-09-13T03:13:35.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.908+0000 7f562796cb80 10 client.25790 fetch_fsmap learned FSMap version 26 2023-09-13T03:13:35.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.908+0000 7f562796cb80 1 -- 172.21.15.64:0/3235938947 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f5608009940 con 0x55be85b44660 2023-09-13T03:13:35.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.908+0000 7f55f57fa700 1 -- 172.21.15.64:0/3235938947 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55be859db4f0 con 0x55be85b44660 2023-09-13T03:13:35.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.908+0000 7f55f57fa700 1 -- 172.21.15.64:0/3235938947 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f56040223a0 con 0x55be85b44660 2023-09-13T03:13:35.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.909+0000 7f562796cb80 10 client.25790 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:13:35.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.909+0000 7f562796cb80 10 client.25790 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2023-09-13T03:13:35.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.909+0000 7f562796cb80 20 client.25790 populate_metadata read hostname 'smithi064' 2023-09-13T03:13:35.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.909+0000 7f562796cb80 1 --2- 172.21.15.64:0/3235938947 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x55be85b54240 0x55be85b56660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:35.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.909+0000 7f562796cb80 4 client.25790 mds_command: new command op to 24634 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:13:35.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.909+0000 7f562796cb80 1 -- 172.21.15.64:0/3235938947 --> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be85a988a0 con 0x55be85b54240 2023-09-13T03:13:35.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.909+0000 7f5611d9b700 1 --2- 172.21.15.64:0/3235938947 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x55be85b54240 0x55be85b56660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:35.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.910+0000 7f55f57fa700 10 client.25790 ms_handle_connect on v2:172.21.15.170:6836/1726587275 2023-09-13T03:13:35.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.910+0000 7f5611d9b700 1 --2- 172.21.15.64:0/3235938947 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x55be85b54240 0x55be85b56660 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5f9e6b4a34b38bb7 server_cookie=d35e83fdf6e08ab9 in_seq=0 out_seq=0 2023-09-13T03:13:35.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.910+0000 7f55f57fa700 1 -- 172.21.15.64:0/3235938947 <== mds.3 v2:172.21.15.170:6836/1726587275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55be85a988a0 con 0x55be85b54240 2023-09-13T03:13:35.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.910+0000 7f55f57fa700 10 client.25790 handle_command_reply: tid=0 2023-09-13T03:13:35.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.911+0000 7f562796cb80 1 client.25790 shutdown 2023-09-13T03:13:35.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.911+0000 7f562796cb80 1 -- 172.21.15.64:0/3235938947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be85b44660 msgr2=0x55be85b41a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:35.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.911+0000 7f562796cb80 1 --2- 172.21.15.64:0/3235938947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be85b44660 0x55be85b41a80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f56040169e0 tx=0x7f5604004b40 comp rx=0 tx=0).stop 2023-09-13T03:13:35.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.911+0000 7f562796cb80 1 -- 172.21.15.64:0/3235938947 shutdown_connections 2023-09-13T03:13:35.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.911+0000 7f562796cb80 1 --2- 172.21.15.64:0/3235938947 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x55be85b54240 0x55be85b56660 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.911+0000 7f562796cb80 1 --2- 172.21.15.64:0/3235938947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be85b44660 0x55be85b41a80 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.911+0000 7f562796cb80 1 --2- 172.21.15.64:0/3235938947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be85b38f10 0x55be85b440c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.911+0000 7f562796cb80 1 --2- 172.21.15.64:0/3235938947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be85b39830 0x55be85b41340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.911+0000 7f562796cb80 1 -- 172.21.15.64:0/3235938947 >> 172.21.15.64:0/3235938947 conn(0x55be85aa6090 msgr2=0x55be85b2c680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:35.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.911+0000 7f55f57fa700 0 client.25790 ms_handle_reset on v2:172.21.15.170:6836/1726587275 2023-09-13T03:13:35.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.911+0000 7f562796cb80 1 -- 172.21.15.64:0/3235938947 shutdown_connections 2023-09-13T03:13:35.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.911+0000 7f562796cb80 1 -- 172.21.15.64:0/3235938947 wait complete. 2023-09-13T03:13:35.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.912+0000 7f562796cb80 20 client.25790 trim_cache size 0 max 16384 2023-09-13T03:13:35.923 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:35 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2256578416' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:13:35.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.938+0000 7f562796cb80 1 Processor -- start 2023-09-13T03:13:35.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.938+0000 7f562796cb80 1 -- start start 2023-09-13T03:13:35.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.939+0000 7f562796cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be85b38f10 0x55be85b43bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:35.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.939+0000 7f562796cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be85b39830 0x55be85b3c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:35.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.939+0000 7f562796cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be85b44660 0x55be85b3cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:35.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.939+0000 7f562796cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be85af6850 con 0x55be85b39830 2023-09-13T03:13:35.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.939+0000 7f562796cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be85b42320 con 0x55be85b38f10 2023-09-13T03:13:35.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.939+0000 7f562796cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be85b3bda0 con 0x55be85b44660 2023-09-13T03:13:35.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.939+0000 7f5611d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be85b38f10 0x55be85b43bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:35.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.939+0000 7f5611d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be85b38f10 0x55be85b43bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:57190/0 (socket says 172.21.15.64:57190) 2023-09-13T03:13:35.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.939+0000 7f5611d9b700 1 -- 172.21.15.64:0/3401888298 learned_addr learned my addr 172.21.15.64:0/3401888298 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:35.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.939+0000 7f561159a700 1 --2- 172.21.15.64:0/3401888298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be85b39830 0x55be85b3c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:35.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.940+0000 7f561259c700 1 --2- 172.21.15.64:0/3401888298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be85b44660 0x55be85b3cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:35.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.940+0000 7f5611d9b700 1 -- 172.21.15.64:0/3401888298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be85b44660 msgr2=0x55be85b3cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:35.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.940+0000 7f5611d9b700 1 --2- 172.21.15.64:0/3401888298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be85b44660 0x55be85b3cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.940+0000 7f5611d9b700 1 -- 172.21.15.64:0/3401888298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be85b39830 msgr2=0x55be85b3c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:35.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.940+0000 7f5611d9b700 1 --2- 172.21.15.64:0/3401888298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be85b39830 0x55be85b3c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.940+0000 7f5611d9b700 1 -- 172.21.15.64:0/3401888298 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be85898c60 con 0x55be85b38f10 2023-09-13T03:13:35.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.940+0000 7f5611d9b700 1 --2- 172.21.15.64:0/3401888298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be85b38f10 0x55be85b43bf0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f560801c040 tx=0x7f5608005920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91ddbf14a2e31a97 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:35.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.941+0000 7f5600ff9700 1 -- 172.21.15.64:0/3401888298 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5608011de0 con 0x55be85b38f10 2023-09-13T03:13:35.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.941+0000 7f5600ff9700 1 -- 172.21.15.64:0/3401888298 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56080280e0 con 0x55be85b38f10 2023-09-13T03:13:35.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.941+0000 7f5600ff9700 1 -- 172.21.15.64:0/3401888298 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5608005570 con 0x55be85b38f10 2023-09-13T03:13:35.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.941+0000 7f562796cb80 1 -- 172.21.15.64:0/3401888298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be85b38f10 msgr2=0x55be85b43bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:35.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.941+0000 7f562796cb80 1 --2- 172.21.15.64:0/3401888298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be85b38f10 0x55be85b43bf0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f560801c040 tx=0x7f5608005920 comp rx=0 tx=0).stop 2023-09-13T03:13:35.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.942+0000 7f562796cb80 1 -- 172.21.15.64:0/3401888298 shutdown_connections 2023-09-13T03:13:35.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.942+0000 7f562796cb80 1 --2- 172.21.15.64:0/3401888298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be85b38f10 0x55be85b43bf0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.942+0000 7f562796cb80 1 --2- 172.21.15.64:0/3401888298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be85b44660 0x55be85b3cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.942+0000 7f562796cb80 1 --2- 172.21.15.64:0/3401888298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be85b39830 0x55be85b3c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.942+0000 7f562796cb80 1 -- 172.21.15.64:0/3401888298 >> 172.21.15.64:0/3401888298 conn(0x55be85aa6090 msgr2=0x55be85b4f910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:35.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.942+0000 7f562796cb80 1 -- 172.21.15.64:0/3401888298 shutdown_connections 2023-09-13T03:13:35.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.942+0000 7f562796cb80 1 -- 172.21.15.64:0/3401888298 wait complete. 2023-09-13T03:13:35.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.942+0000 7f562796cb80 1 -- start start 2023-09-13T03:13:35.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.943+0000 7f562796cb80 1 Processor -- start 2023-09-13T03:13:35.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.943+0000 7f562796cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be85b38f10 0x55be85b4b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:35.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.943+0000 7f562796cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be85b39830 0x55be85b475e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:35.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.943+0000 7f562796cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be85b44660 0x55be85b47b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:35.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.943+0000 7f562796cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be85b48280 con 0x55be85b39830 2023-09-13T03:13:35.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.943+0000 7f562796cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be85b483f0 con 0x55be85b44660 2023-09-13T03:13:35.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.943+0000 7f562796cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be85b2e6f0 con 0x55be85b38f10 2023-09-13T03:13:35.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.943+0000 7f562796cb80 10 client.0 _ll_register_callbacks cb 0x55be8596ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:13:35.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.944+0000 7f561159a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be85b39830 0x55be85b475e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:35.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.944+0000 7f5611d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be85b38f10 0x55be85b4b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:35.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.944+0000 7f5611d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be85b38f10 0x55be85b4b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:54286/0 (socket says 172.21.15.64:54286) 2023-09-13T03:13:35.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.944+0000 7f5611d9b700 1 -- 172.21.15.64:0/2609165246 learned_addr learned my addr 172.21.15.64:0/2609165246 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:35.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.944+0000 7f561259c700 1 --2- 172.21.15.64:0/2609165246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be85b44660 0x55be85b47b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:35.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.944+0000 7f5611d9b700 1 -- 172.21.15.64:0/2609165246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be85b44660 msgr2=0x55be85b47b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:35.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.944+0000 7f5611d9b700 1 --2- 172.21.15.64:0/2609165246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be85b44660 0x55be85b47b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.944+0000 7f5611d9b700 1 -- 172.21.15.64:0/2609165246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be85b39830 msgr2=0x55be85b475e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:35.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.944+0000 7f561159a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be85b39830 0x55be85b475e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59102/0 (socket says 172.21.15.64:59102) 2023-09-13T03:13:35.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.944+0000 7f5611d9b700 1 --2- 172.21.15.64:0/2609165246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be85b39830 0x55be85b475e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.944+0000 7f5611d9b700 1 -- 172.21.15.64:0/2609165246 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be858afde0 con 0x55be85b38f10 2023-09-13T03:13:35.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.944+0000 7f561159a700 1 --2- 172.21.15.64:0/2609165246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be85b39830 0x55be85b475e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T03:13:35.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.945+0000 7f5611d9b700 1 --2- 172.21.15.64:0/2609165246 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be85b38f10 0x55be85b4b1c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5608000c00 tx=0x7f5608011de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44780c41b8fa5c43 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:35.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.945+0000 7f55f57fa700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:13:35.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.945+0000 7f55f57fa700 1 -- 172.21.15.64:0/2609165246 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5608028c90 con 0x55be85b38f10 2023-09-13T03:13:35.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.945+0000 7f562796cb80 1 -- 172.21.15.64:0/2609165246 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5608009e60 con 0x55be85b38f10 2023-09-13T03:13:35.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.945+0000 7f55f57fa700 1 -- 172.21.15.64:0/2609165246 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f5608028df0 con 0x55be85b38f10 2023-09-13T03:13:35.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.945+0000 7f55f57fa700 1 -- 172.21.15.64:0/2609165246 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56080051f0 con 0x55be85b38f10 2023-09-13T03:13:35.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.945+0000 7f562796cb80 10 client.25792 fetch_fsmap learned FSMap version 26 2023-09-13T03:13:35.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.945+0000 7f562796cb80 1 -- 172.21.15.64:0/2609165246 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55be858d6250 con 0x55be85b38f10 2023-09-13T03:13:35.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.946+0000 7f55f57fa700 1 -- 172.21.15.64:0/2609165246 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5608009e60 con 0x55be85b38f10 2023-09-13T03:13:35.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.946+0000 7f55f57fa700 1 -- 172.21.15.64:0/2609165246 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5608012450 con 0x55be85b38f10 2023-09-13T03:13:35.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.946+0000 7f562796cb80 10 client.25792 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:13:35.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.946+0000 7f562796cb80 10 client.25792 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2023-09-13T03:13:35.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.946+0000 7f562796cb80 20 client.25792 populate_metadata read hostname 'smithi064' 2023-09-13T03:13:35.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.946+0000 7f562796cb80 1 --2- 172.21.15.64:0/2609165246 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x55be85b2f930 0x55be85b31d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:35.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.946+0000 7f562796cb80 4 client.25792 mds_command: new command op to 24634 tid=0[{"prefix": "damage ls"}] 2023-09-13T03:13:35.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.946+0000 7f562796cb80 1 -- 172.21.15.64:0/2609165246 --> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55be85a7c920 con 0x55be85b2f930 2023-09-13T03:13:35.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.947+0000 7f561159a700 1 --2- 172.21.15.64:0/2609165246 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x55be85b2f930 0x55be85b31d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:35.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.947+0000 7f561159a700 1 --2- 172.21.15.64:0/2609165246 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x55be85b2f930 0x55be85b31d20 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4c0ab9f8f8458193 server_cookie=9ab1c1bdad8bfaa in_seq=0 out_seq=0 2023-09-13T03:13:35.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.947+0000 7f55f57fa700 10 client.25792 ms_handle_connect on v2:172.21.15.170:6836/1726587275 2023-09-13T03:13:35.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.948+0000 7f55f57fa700 1 -- 172.21.15.64:0/2609165246 <== mds.3 v2:172.21.15.170:6836/1726587275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55be85a7c920 con 0x55be85b2f930 2023-09-13T03:13:35.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.948+0000 7f55f57fa700 10 client.25792 handle_command_reply: tid=0 2023-09-13T03:13:35.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.948+0000 7f562796cb80 1 client.25792 shutdown 2023-09-13T03:13:35.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.948+0000 7f562796cb80 1 -- 172.21.15.64:0/2609165246 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be85b38f10 msgr2=0x55be85b4b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:35.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.948+0000 7f562796cb80 1 --2- 172.21.15.64:0/2609165246 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be85b38f10 0x55be85b4b1c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5608000c00 tx=0x7f5608011de0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.948+0000 7f562796cb80 1 -- 172.21.15.64:0/2609165246 shutdown_connections 2023-09-13T03:13:35.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.948+0000 7f562796cb80 1 --2- 172.21.15.64:0/2609165246 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x55be85b2f930 0x55be85b31d20 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.948+0000 7f562796cb80 1 --2- 172.21.15.64:0/2609165246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be85b44660 0x55be85b47b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.948+0000 7f562796cb80 1 --2- 172.21.15.64:0/2609165246 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55be85b38f10 0x55be85b4b1c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.948+0000 7f562796cb80 1 --2- 172.21.15.64:0/2609165246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be85b39830 0x55be85b475e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.948+0000 7f562796cb80 1 -- 172.21.15.64:0/2609165246 >> 172.21.15.64:0/2609165246 conn(0x55be85aa6090 msgr2=0x55be85b3c620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:35.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.948+0000 7f55f57fa700 0 client.25792 ms_handle_reset on v2:172.21.15.170:6836/1726587275 2023-09-13T03:13:35.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.948+0000 7f562796cb80 1 -- 172.21.15.64:0/2609165246 shutdown_connections 2023-09-13T03:13:35.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.948+0000 7f562796cb80 1 -- 172.21.15.64:0/2609165246 wait complete. 2023-09-13T03:13:35.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.949+0000 7f562796cb80 20 client.25792 trim_cache size 0 max 16384 2023-09-13T03:13:35.962 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2023-09-13T03:13:35.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.949+0000 7f5613fff700 1 -- 172.21.15.64:0/3713426427 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f55f80a0940 msgr2=0x7f55f80a2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:35.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.949+0000 7f5613fff700 1 --2- 172.21.15.64:0/3713426427 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f55f80a0940 0x7f55f80a2e00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f560c19f400 tx=0x7f560400b040 comp rx=0 tx=0).stop 2023-09-13T03:13:35.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.949+0000 7f5613fff700 1 -- 172.21.15.64:0/3713426427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f560c1020d0 msgr2=0x7f560c19ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:35.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.949+0000 7f5613fff700 1 --2- 172.21.15.64:0/3713426427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f560c1020d0 0x7f560c19ec40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f55fc009810 tx=0x7f55fc00fa00 comp rx=0 tx=0).stop 2023-09-13T03:13:35.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.950+0000 7f5613fff700 1 -- 172.21.15.64:0/3713426427 shutdown_connections 2023-09-13T03:13:35.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.950+0000 7f5613fff700 1 --2- 172.21.15.64:0/3713426427 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f55f80a0940 0x7f55f80a2e00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.950+0000 7f5613fff700 1 --2- 172.21.15.64:0/3713426427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f560c10ab80 0x7f560c198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.950+0000 7f5613fff700 1 --2- 172.21.15.64:0/3713426427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f560c1017b0 0x7f560c19e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.950+0000 7f5613fff700 1 --2- 172.21.15.64:0/3713426427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f560c1020d0 0x7f560c19ec40 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:35.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.950+0000 7f5613fff700 1 -- 172.21.15.64:0/3713426427 >> 172.21.15.64:0/3713426427 conn(0x7f560c0faf50 msgr2=0x7f560c10cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:35.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.950+0000 7f5613fff700 1 -- 172.21.15.64:0/3713426427 shutdown_connections 2023-09-13T03:13:35.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:35.950+0000 7f5613fff700 1 -- 172.21.15.64:0/3713426427 wait complete. 2023-09-13T03:13:35.978 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-09-13T03:13:35.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:35 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/2256578416' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:13:36.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.203+0000 7f196759e700 1 -- 172.21.15.64:0/1931106738 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1968068080 msgr2=0x7f196810cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:36.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.203+0000 7f196759e700 1 --2- 172.21.15.64:0/1931106738 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1968068080 0x7f196810cbb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f195c009990 tx=0x7f195c009ca0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.203+0000 7f196759e700 1 -- 172.21.15.64:0/1931106738 shutdown_connections 2023-09-13T03:13:36.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.203+0000 7f196759e700 1 --2- 172.21.15.64:0/1931106738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f196810d0f0 0x7f196810f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.203+0000 7f196759e700 1 --2- 172.21.15.64:0/1931106738 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1968067760 0x7f1968067b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.203+0000 7f196759e700 1 --2- 172.21.15.64:0/1931106738 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1968068080 0x7f196810cbb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.203+0000 7f196759e700 1 -- 172.21.15.64:0/1931106738 >> 172.21.15.64:0/1931106738 conn(0x7f1968074dc0 msgr2=0x7f19680751d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:36.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.203+0000 7f196759e700 1 -- 172.21.15.64:0/1931106738 shutdown_connections 2023-09-13T03:13:36.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.203+0000 7f196759e700 1 -- 172.21.15.64:0/1931106738 wait complete. 2023-09-13T03:13:36.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.204+0000 7f196759e700 1 Processor -- start 2023-09-13T03:13:36.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.204+0000 7f196759e700 1 -- start start 2023-09-13T03:13:36.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.205+0000 7f196759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1968067760 0x7f19681986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.205+0000 7f196759e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1968068080 0x7f1968198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.205+0000 7f196759e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f196810d0f0 0x7f196819bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.205+0000 7f196759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196819c590 con 0x7f1968067760 2023-09-13T03:13:36.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.205+0000 7f196759e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196819c700 con 0x7f1968068080 2023-09-13T03:13:36.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.205+0000 7f196759e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196819c8c0 con 0x7f196810d0f0 2023-09-13T03:13:36.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.205+0000 7f1965d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1968068080 0x7f1968198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.205+0000 7f1965d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1968068080 0x7f1968198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:57218/0 (socket says 172.21.15.64:57218) 2023-09-13T03:13:36.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.205+0000 7f1965d9b700 1 -- 172.21.15.64:0/3538690282 learned_addr learned my addr 172.21.15.64:0/3538690282 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:36.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.206+0000 7f1966d9d700 1 --2- 172.21.15.64:0/3538690282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f196810d0f0 0x7f196819bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.206+0000 7f1965d9b700 1 -- 172.21.15.64:0/3538690282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f196810d0f0 msgr2=0x7f196819bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:36.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.206+0000 7f1965d9b700 1 --2- 172.21.15.64:0/3538690282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f196810d0f0 0x7f196819bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.206+0000 7f1965d9b700 1 -- 172.21.15.64:0/3538690282 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1968067760 msgr2=0x7f19681986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:36.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.206+0000 7f1965d9b700 1 --2- 172.21.15.64:0/3538690282 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1968067760 0x7f19681986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.206+0000 7f1965d9b700 1 -- 172.21.15.64:0/3538690282 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f195c0095f0 con 0x7f1968068080 2023-09-13T03:13:36.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.206+0000 7f1965d9b700 1 --2- 172.21.15.64:0/3538690282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1968068080 0x7f1968198be0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f195c005910 tx=0x7f195c0057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:36.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.207+0000 7f19577fe700 1 -- 172.21.15.64:0/3538690282 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c01d070 con 0x7f1968068080 2023-09-13T03:13:36.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.207+0000 7f19577fe700 1 -- 172.21.15.64:0/3538690282 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f195c00fb80 con 0x7f1968068080 2023-09-13T03:13:36.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.207+0000 7f196759e700 1 -- 172.21.15.64:0/3538690282 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f196819cb40 con 0x7f1968068080 2023-09-13T03:13:36.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.207+0000 7f19577fe700 1 -- 172.21.15.64:0/3538690282 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c017650 con 0x7f1968068080 2023-09-13T03:13:36.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.207+0000 7f196759e700 1 -- 172.21.15.64:0/3538690282 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19681a5500 con 0x7f1968068080 2023-09-13T03:13:36.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.213+0000 7f19577fe700 1 -- 172.21.15.64:0/3538690282 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) v1 ==== 135435+0+0 (secure 0 0 0) 0x7f195c00fcf0 con 0x7f1968068080 2023-09-13T03:13:36.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.213+0000 7f198064fb80 1 Processor -- start 2023-09-13T03:13:36.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.214+0000 7f198064fb80 1 -- start start 2023-09-13T03:13:36.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.214+0000 7f198064fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e0b27c860 0x562e0b27cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.214+0000 7f198064fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e0b27d180 0x562e0b287af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.214+0000 7f198064fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e0b288030 0x562e0b28a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.214+0000 7f198064fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e0b1d6730 con 0x562e0b27c860 2023-09-13T03:13:36.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.214+0000 7f198064fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e0b1d4050 con 0x562e0b288030 2023-09-13T03:13:36.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.214+0000 7f198064fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e0b1cf940 con 0x562e0b27d180 2023-09-13T03:13:36.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.215+0000 7f196659c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e0b27d180 0x562e0b287af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.215+0000 7f1965d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e0b288030 0x562e0b28a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.215+0000 7f196659c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e0b27d180 0x562e0b287af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:54312/0 (socket says 172.21.15.64:54312) 2023-09-13T03:13:36.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.215+0000 7f196659c700 1 -- 172.21.15.64:0/2485187047 learned_addr learned my addr 172.21.15.64:0/2485187047 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:36.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.215+0000 7f1966d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e0b27c860 0x562e0b27cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.216+0000 7f1966d9d700 1 -- 172.21.15.64:0/2485187047 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e0b27d180 msgr2=0x562e0b287af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:36.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.216+0000 7f1966d9d700 1 --2- 172.21.15.64:0/2485187047 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e0b27d180 0x562e0b287af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.216+0000 7f1966d9d700 1 -- 172.21.15.64:0/2485187047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e0b288030 msgr2=0x562e0b28a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:36.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.216+0000 7f1966d9d700 1 --2- 172.21.15.64:0/2485187047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e0b288030 0x562e0b28a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.216+0000 7f1966d9d700 1 -- 172.21.15.64:0/2485187047 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e0affbde0 con 0x562e0b27c860 2023-09-13T03:13:36.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.216+0000 7f1965d9b700 1 --2- 172.21.15.64:0/2485187047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e0b288030 0x562e0b28a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:13:36.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.216+0000 7f19577fe700 1 --2- 172.21.15.64:0/3538690282 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f19500a09e0 0x7f19500a2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.216+0000 7f1966d9d700 1 --2- 172.21.15.64:0/2485187047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e0b27c860 0x562e0b27cc40 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f19600092e0 tx=0x7f1960009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ebd8d77c2306268 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:36.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.216+0000 7f19577fe700 1 -- 172.21.15.64:0/3538690282 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f195c0cd660 con 0x7f1968068080 2023-09-13T03:13:36.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.217+0000 7f196659c700 1 --2- 172.21.15.64:0/3538690282 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f19500a09e0 0x7f19500a2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.217+0000 7f1954ff9700 1 -- 172.21.15.64:0/2485187047 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196000e070 con 0x562e0b27c860 2023-09-13T03:13:36.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.217+0000 7f1954ff9700 1 -- 172.21.15.64:0/2485187047 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f196000f040 con 0x562e0b27c860 2023-09-13T03:13:36.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.217+0000 7f1954ff9700 1 -- 172.21.15.64:0/2485187047 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1960014780 con 0x562e0b27c860 2023-09-13T03:13:36.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.218+0000 7f198064fb80 1 -- 172.21.15.64:0/2485187047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e0b27c860 msgr2=0x562e0b27cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:36.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.218+0000 7f198064fb80 1 --2- 172.21.15.64:0/2485187047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e0b27c860 0x562e0b27cc40 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f19600092e0 tx=0x7f1960009920 comp rx=0 tx=0).stop 2023-09-13T03:13:36.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.218+0000 7f198064fb80 1 -- 172.21.15.64:0/2485187047 shutdown_connections 2023-09-13T03:13:36.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.218+0000 7f198064fb80 1 --2- 172.21.15.64:0/2485187047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e0b288030 0x562e0b28a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.218+0000 7f198064fb80 1 --2- 172.21.15.64:0/2485187047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e0b27c860 0x562e0b27cc40 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.218+0000 7f198064fb80 1 --2- 172.21.15.64:0/2485187047 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e0b27d180 0x562e0b287af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.218+0000 7f198064fb80 1 -- 172.21.15.64:0/2485187047 >> 172.21.15.64:0/2485187047 conn(0x562e0b1f2090 msgr2=0x562e0b278d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:36.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.218+0000 7f198064fb80 1 -- 172.21.15.64:0/2485187047 shutdown_connections 2023-09-13T03:13:36.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.218+0000 7f198064fb80 1 -- 172.21.15.64:0/2485187047 wait complete. 2023-09-13T03:13:36.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.219+0000 7f196659c700 1 --2- 172.21.15.64:0/3538690282 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f19500a09e0 0x7f19500a2ea0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1958005d90 tx=0x7f195800b4a0 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:36.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.219+0000 7f198064fb80 1 -- start start 2023-09-13T03:13:36.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.219+0000 7f198064fb80 1 Processor -- start 2023-09-13T03:13:36.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.220+0000 7f198064fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e0b27c860 0x562e0b287ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.220+0000 7f198064fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e0b27d180 0x562e0b284d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.220+0000 7f198064fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e0b288030 0x562e0b2854a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.220+0000 7f198064fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e0b16db70 con 0x562e0b288030 2023-09-13T03:13:36.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.220+0000 7f198064fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e0b16d870 con 0x562e0b27c860 2023-09-13T03:13:36.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.220+0000 7f198064fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e0b051920 con 0x562e0b27d180 2023-09-13T03:13:36.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.220+0000 7f198064fb80 10 client.0 _ll_register_callbacks cb 0x562e0b0b7a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:13:36.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.220+0000 7f1966d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e0b288030 0x562e0b2854a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.220+0000 7f1966d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e0b288030 0x562e0b2854a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59144/0 (socket says 172.21.15.64:59144) 2023-09-13T03:13:36.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.220+0000 7f1965d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e0b27d180 0x562e0b284d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.220+0000 7f1966d9d700 1 -- 172.21.15.64:0/2783341356 learned_addr learned my addr 172.21.15.64:0/2783341356 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:36.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.220+0000 7f1966d9d700 1 -- 172.21.15.64:0/2783341356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e0b27d180 msgr2=0x562e0b284d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:36.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.221+0000 7f1966d9d700 1 --2- 172.21.15.64:0/2783341356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e0b27d180 0x562e0b284d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.221+0000 7f1966d9d700 1 -- 172.21.15.64:0/2783341356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e0b27c860 msgr2=0x562e0b287ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-13T03:13:36.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.221+0000 7f1966d9d700 1 --2- 172.21.15.64:0/2783341356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e0b27c860 0x562e0b287ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.221+0000 7f1966d9d700 1 -- 172.21.15.64:0/2783341356 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19681a5500 con 0x562e0b288030 2023-09-13T03:13:36.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.221+0000 7f1965d9b700 1 --2- 172.21.15.64:0/2783341356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e0b27d180 0x562e0b284d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:13:36.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.221+0000 7f1966d9d700 1 --2- 172.21.15.64:0/2783341356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e0b288030 0x562e0b2854a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1960005ae0 tx=0x7f1960007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d89d05def8d35d server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:36.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.221+0000 7f194dffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:13:36.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.221+0000 7f194dffb700 1 -- 172.21.15.64:0/2783341356 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196000e040 con 0x562e0b288030 2023-09-13T03:13:36.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.222+0000 7f198064fb80 1 -- 172.21.15.64:0/2783341356 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562e0b1274f0 con 0x562e0b288030 2023-09-13T03:13:36.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.222+0000 7f194dffb700 1 -- 172.21.15.64:0/2783341356 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f1960047070 con 0x562e0b288030 2023-09-13T03:13:36.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.222+0000 7f194dffb700 1 -- 172.21.15.64:0/2783341356 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19600095e0 con 0x562e0b288030 2023-09-13T03:13:36.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.222+0000 7f198064fb80 10 client.15777 fetch_fsmap learned FSMap version 26 2023-09-13T03:13:36.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.222+0000 7f198064fb80 1 -- 172.21.15.64:0/2783341356 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f1960009050 con 0x562e0b288030 2023-09-13T03:13:36.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.222+0000 7f194dffb700 1 -- 172.21.15.64:0/2783341356 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562e0b1274f0 con 0x562e0b288030 2023-09-13T03:13:36.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.222+0000 7f194dffb700 1 -- 172.21.15.64:0/2783341356 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f196004b710 con 0x562e0b288030 2023-09-13T03:13:36.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.223+0000 7f198064fb80 10 client.15777 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:13:36.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.223+0000 7f198064fb80 10 client.15777 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-09-13T03:13:36.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.223+0000 7f198064fb80 20 client.15777 populate_metadata read hostname 'smithi064' 2023-09-13T03:13:36.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.223+0000 7f198064fb80 1 --2- 172.21.15.64:0/2783341356 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x562e0b28e720 0x562e0b290b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.223+0000 7f198064fb80 4 client.15777 mds_command: new command op to 24635 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:13:36.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.223+0000 7f198064fb80 1 -- 172.21.15.64:0/2783341356 --> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e0b1e48a0 con 0x562e0b28e720 2023-09-13T03:13:36.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.223+0000 7f196659c700 1 --2- 172.21.15.64:0/2783341356 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x562e0b28e720 0x562e0b290b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.224+0000 7f194dffb700 10 client.15777 ms_handle_connect on v2:172.21.15.142:6836/741173268 2023-09-13T03:13:36.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.224+0000 7f196659c700 1 --2- 172.21.15.64:0/2783341356 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x562e0b28e720 0x562e0b290b40 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f6ff119eebc34474 server_cookie=7067024ab3f48493 in_seq=0 out_seq=0 2023-09-13T03:13:36.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.225+0000 7f194dffb700 1 -- 172.21.15.64:0/2783341356 <== mds.2 v2:172.21.15.142:6836/741173268 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562e0b1e48a0 con 0x562e0b28e720 2023-09-13T03:13:36.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.225+0000 7f194dffb700 10 client.15777 handle_command_reply: tid=0 2023-09-13T03:13:36.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.225+0000 7f198064fb80 1 client.15777 shutdown 2023-09-13T03:13:36.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.225+0000 7f198064fb80 1 -- 172.21.15.64:0/2783341356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e0b288030 msgr2=0x562e0b2854a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:36.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.225+0000 7f198064fb80 1 --2- 172.21.15.64:0/2783341356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e0b288030 0x562e0b2854a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1960005ae0 tx=0x7f1960007e60 comp rx=0 tx=0).stop 2023-09-13T03:13:36.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.225+0000 7f198064fb80 1 -- 172.21.15.64:0/2783341356 shutdown_connections 2023-09-13T03:13:36.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.225+0000 7f198064fb80 1 --2- 172.21.15.64:0/2783341356 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x562e0b28e720 0x562e0b290b40 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.225+0000 7f198064fb80 1 --2- 172.21.15.64:0/2783341356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e0b27c860 0x562e0b287ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.225+0000 7f198064fb80 1 --2- 172.21.15.64:0/2783341356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e0b288030 0x562e0b2854a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.225+0000 7f198064fb80 1 --2- 172.21.15.64:0/2783341356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e0b27d180 0x562e0b284d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.225+0000 7f198064fb80 1 -- 172.21.15.64:0/2783341356 >> 172.21.15.64:0/2783341356 conn(0x562e0b1f2090 msgr2=0x562e0b278940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:36.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.225+0000 7f194dffb700 0 client.15777 ms_handle_reset on v2:172.21.15.142:6836/741173268 2023-09-13T03:13:36.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.226+0000 7f198064fb80 1 -- 172.21.15.64:0/2783341356 shutdown_connections 2023-09-13T03:13:36.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.226+0000 7f198064fb80 1 -- 172.21.15.64:0/2783341356 wait complete. 2023-09-13T03:13:36.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.226+0000 7f198064fb80 20 client.15777 trim_cache size 0 max 16384 2023-09-13T03:13:36.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.240+0000 7f198064fb80 1 Processor -- start 2023-09-13T03:13:36.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.240+0000 7f198064fb80 1 -- start start 2023-09-13T03:13:36.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.240+0000 7f198064fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e0b27c860 0x562e0b27f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.240+0000 7f198064fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e0b27d180 0x562e0b2a0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.240+0000 7f198064fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e0b288030 0x562e0b284520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.240+0000 7f198064fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e0b275530 con 0x562e0b27c860 2023-09-13T03:13:36.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.240+0000 7f198064fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e0b27da80 con 0x562e0b288030 2023-09-13T03:13:36.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.240+0000 7f198064fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e0b286340 con 0x562e0b27d180 2023-09-13T03:13:36.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.241+0000 7f1966d9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e0b288030 0x562e0b284520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.241+0000 7f1965d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e0b27d180 0x562e0b2a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.241+0000 7f1966d9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e0b288030 0x562e0b284520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:57238/0 (socket says 172.21.15.64:57238) 2023-09-13T03:13:36.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.241+0000 7f1966d9d700 1 -- 172.21.15.64:0/2269287620 learned_addr learned my addr 172.21.15.64:0/2269287620 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:36.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.241+0000 7f1966d9d700 1 -- 172.21.15.64:0/2269287620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e0b27d180 msgr2=0x562e0b2a0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:36.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.241+0000 7f1966d9d700 1 --2- 172.21.15.64:0/2269287620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e0b27d180 0x562e0b2a0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.241+0000 7f1966d9d700 1 -- 172.21.15.64:0/2269287620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e0b27c860 msgr2=0x562e0b27f760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-13T03:13:36.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.241+0000 7f1966d9d700 1 --2- 172.21.15.64:0/2269287620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e0b27c860 0x562e0b27f760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.241+0000 7f1966d9d700 1 -- 172.21.15.64:0/2269287620 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e0b277750 con 0x562e0b288030 2023-09-13T03:13:36.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.241+0000 7f1966d9d700 1 --2- 172.21.15.64:0/2269287620 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e0b288030 0x562e0b284520 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1960007c20 tx=0x7f1960009b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fb76bc5872bdb6c server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:36.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.242+0000 7f19557fa700 1 -- 172.21.15.64:0/2269287620 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1960015c80 con 0x562e0b288030 2023-09-13T03:13:36.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.242+0000 7f19557fa700 1 -- 172.21.15.64:0/2269287620 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1960045070 con 0x562e0b288030 2023-09-13T03:13:36.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.242+0000 7f19557fa700 1 -- 172.21.15.64:0/2269287620 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196004b780 con 0x562e0b288030 2023-09-13T03:13:36.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.242+0000 7f198064fb80 1 -- 172.21.15.64:0/2269287620 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e0b288030 msgr2=0x562e0b284520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:36.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.242+0000 7f198064fb80 1 --2- 172.21.15.64:0/2269287620 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e0b288030 0x562e0b284520 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1960007c20 tx=0x7f1960009b40 comp rx=0 tx=0).stop 2023-09-13T03:13:36.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.243+0000 7f198064fb80 1 -- 172.21.15.64:0/2269287620 shutdown_connections 2023-09-13T03:13:36.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.243+0000 7f198064fb80 1 --2- 172.21.15.64:0/2269287620 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e0b288030 0x562e0b284520 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.243+0000 7f198064fb80 1 --2- 172.21.15.64:0/2269287620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e0b27c860 0x562e0b27f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.243+0000 7f198064fb80 1 --2- 172.21.15.64:0/2269287620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e0b27d180 0x562e0b2a0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.243+0000 7f198064fb80 1 -- 172.21.15.64:0/2269287620 >> 172.21.15.64:0/2269287620 conn(0x562e0b1f2090 msgr2=0x562e0b29fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:36.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.243+0000 7f198064fb80 1 -- 172.21.15.64:0/2269287620 shutdown_connections 2023-09-13T03:13:36.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.243+0000 7f198064fb80 1 -- 172.21.15.64:0/2269287620 wait complete. 2023-09-13T03:13:36.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.243+0000 7f198064fb80 1 -- start start 2023-09-13T03:13:36.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.244+0000 7f198064fb80 1 Processor -- start 2023-09-13T03:13:36.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.244+0000 7f198064fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e0b27c860 0x562e0b297370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.244+0000 7f198064fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e0b27d180 0x562e0b2937b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.244+0000 7f198064fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e0b288030 0x562e0b293cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.244+0000 7f198064fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e0b294400 con 0x562e0b27d180 2023-09-13T03:13:36.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.244+0000 7f198064fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e0b294570 con 0x562e0b27c860 2023-09-13T03:13:36.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.244+0000 7f198064fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e0b2eb4c0 con 0x562e0b288030 2023-09-13T03:13:36.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.244+0000 7f198064fb80 10 client.0 _ll_register_callbacks cb 0x7f195c0cab70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:13:36.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.245+0000 7f1966d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e0b288030 0x562e0b293cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.245+0000 7f1965d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e0b27d180 0x562e0b2937b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.245+0000 7f1966d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e0b288030 0x562e0b293cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:54330/0 (socket says 172.21.15.64:54330) 2023-09-13T03:13:36.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.245+0000 7f196659c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e0b27c860 0x562e0b297370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.245+0000 7f1966d9d700 1 -- 172.21.15.64:0/3666284816 learned_addr learned my addr 172.21.15.64:0/3666284816 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:36.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.245+0000 7f1965d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e0b27d180 0x562e0b2937b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59154/0 (socket says 172.21.15.64:59154) 2023-09-13T03:13:36.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.245+0000 7f1966d9d700 1 -- 172.21.15.64:0/3666284816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e0b27c860 msgr2=0x562e0b297370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:36.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.246+0000 7f1966d9d700 1 --2- 172.21.15.64:0/3666284816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e0b27c860 0x562e0b297370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.246+0000 7f1966d9d700 1 -- 172.21.15.64:0/3666284816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e0b27d180 msgr2=0x562e0b2937b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:36.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.246+0000 7f1966d9d700 1 --2- 172.21.15.64:0/3666284816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e0b27d180 0x562e0b2937b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.246+0000 7f1966d9d700 1 -- 172.21.15.64:0/3666284816 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1960009050 con 0x562e0b288030 2023-09-13T03:13:36.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.246+0000 7f1966d9d700 1 --2- 172.21.15.64:0/3666284816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e0b288030 0x562e0b293cf0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1960000f60 tx=0x7f1960007e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcac0e709522ce63 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:36.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.246+0000 7f194dffb700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:13:36.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.246+0000 7f194dffb700 1 -- 172.21.15.64:0/3666284816 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196000e030 con 0x562e0b288030 2023-09-13T03:13:36.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.246+0000 7f198064fb80 1 -- 172.21.15.64:0/3666284816 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f196000f040 con 0x562e0b288030 2023-09-13T03:13:36.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.246+0000 7f194dffb700 1 -- 172.21.15.64:0/3666284816 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f1960047040 con 0x562e0b288030 2023-09-13T03:13:36.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.247+0000 7f194dffb700 1 -- 172.21.15.64:0/3666284816 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1960003e10 con 0x562e0b288030 2023-09-13T03:13:36.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.247+0000 7f198064fb80 10 client.25810 fetch_fsmap learned FSMap version 26 2023-09-13T03:13:36.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.247+0000 7f198064fb80 1 -- 172.21.15.64:0/3666284816 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x562e0afe4c60 con 0x562e0b288030 2023-09-13T03:13:36.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.247+0000 7f194dffb700 1 -- 172.21.15.64:0/3666284816 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f196000f040 con 0x562e0b288030 2023-09-13T03:13:36.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.248+0000 7f194dffb700 1 -- 172.21.15.64:0/3666284816 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1960003f70 con 0x562e0b288030 2023-09-13T03:13:36.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.248+0000 7f198064fb80 10 client.25810 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:13:36.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.248+0000 7f198064fb80 10 client.25810 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-09-13T03:13:36.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.248+0000 7f198064fb80 20 client.25810 populate_metadata read hostname 'smithi064' 2023-09-13T03:13:36.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.248+0000 7f198064fb80 1 --2- 172.21.15.64:0/3666284816 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x562e0b2ec710 0x562e0b2eeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.248+0000 7f198064fb80 4 client.25810 mds_command: new command op to 24635 tid=0[{"prefix": "damage ls"}] 2023-09-13T03:13:36.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.248+0000 7f198064fb80 1 -- 172.21.15.64:0/3666284816 --> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562e0b1c8920 con 0x562e0b2ec710 2023-09-13T03:13:36.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.248+0000 7f196659c700 1 --2- 172.21.15.64:0/3666284816 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x562e0b2ec710 0x562e0b2eeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.249+0000 7f196659c700 1 --2- 172.21.15.64:0/3666284816 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x562e0b2ec710 0x562e0b2eeb00 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bb86212ad7c164f9 server_cookie=d6d63c06baa85be2 in_seq=0 out_seq=0 2023-09-13T03:13:36.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.249+0000 7f194dffb700 10 client.25810 ms_handle_connect on v2:172.21.15.142:6836/741173268 2023-09-13T03:13:36.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.249+0000 7f194dffb700 1 -- 172.21.15.64:0/3666284816 <== mds.2 v2:172.21.15.142:6836/741173268 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562e0b1c8920 con 0x562e0b2ec710 2023-09-13T03:13:36.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.249+0000 7f194dffb700 10 client.25810 handle_command_reply: tid=0 2023-09-13T03:13:36.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.249+0000 7f198064fb80 1 client.25810 shutdown 2023-09-13T03:13:36.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.249+0000 7f198064fb80 1 -- 172.21.15.64:0/3666284816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e0b288030 msgr2=0x562e0b293cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:36.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.249+0000 7f198064fb80 1 --2- 172.21.15.64:0/3666284816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e0b288030 0x562e0b293cf0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1960000f60 tx=0x7f1960007e60 comp rx=0 tx=0).stop 2023-09-13T03:13:36.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.249+0000 7f198064fb80 1 -- 172.21.15.64:0/3666284816 shutdown_connections 2023-09-13T03:13:36.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.249+0000 7f198064fb80 1 --2- 172.21.15.64:0/3666284816 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x562e0b2ec710 0x562e0b2eeb00 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.249+0000 7f198064fb80 1 --2- 172.21.15.64:0/3666284816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562e0b27c860 0x562e0b297370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.249+0000 7f198064fb80 1 --2- 172.21.15.64:0/3666284816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e0b288030 0x562e0b293cf0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.249+0000 7f198064fb80 1 --2- 172.21.15.64:0/3666284816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562e0b27d180 0x562e0b2937b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.249+0000 7f198064fb80 1 -- 172.21.15.64:0/3666284816 >> 172.21.15.64:0/3666284816 conn(0x562e0b1f2090 msgr2=0x562e0b2a0390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:36.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.249+0000 7f194dffb700 0 client.25810 ms_handle_reset on v2:172.21.15.142:6836/741173268 2023-09-13T03:13:36.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.250+0000 7f198064fb80 1 -- 172.21.15.64:0/3666284816 shutdown_connections 2023-09-13T03:13:36.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.250+0000 7f198064fb80 1 -- 172.21.15.64:0/3666284816 wait complete. 2023-09-13T03:13:36.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.250+0000 7f198064fb80 20 client.25810 trim_cache size 0 max 16384 2023-09-13T03:13:36.262 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2023-09-13T03:13:36.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.251+0000 7f196759e700 1 -- 172.21.15.64:0/3538690282 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f19500a09e0 msgr2=0x7f19500a2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:36.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.251+0000 7f196759e700 1 --2- 172.21.15.64:0/3538690282 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f19500a09e0 0x7f19500a2ea0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1958005d90 tx=0x7f195800b4a0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.251+0000 7f196759e700 1 -- 172.21.15.64:0/3538690282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1968068080 msgr2=0x7f1968198be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:36.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.251+0000 7f196759e700 1 --2- 172.21.15.64:0/3538690282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1968068080 0x7f1968198be0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f195c005910 tx=0x7f195c0057f0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.251+0000 7f196759e700 1 -- 172.21.15.64:0/3538690282 shutdown_connections 2023-09-13T03:13:36.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.251+0000 7f196759e700 1 --2- 172.21.15.64:0/3538690282 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f19500a09e0 0x7f19500a2ea0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.251+0000 7f196759e700 1 --2- 172.21.15.64:0/3538690282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f196810d0f0 0x7f196819bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.251+0000 7f196759e700 1 --2- 172.21.15.64:0/3538690282 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1968067760 0x7f19681986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.251+0000 7f196759e700 1 --2- 172.21.15.64:0/3538690282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1968068080 0x7f1968198be0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.251+0000 7f196759e700 1 -- 172.21.15.64:0/3538690282 >> 172.21.15.64:0/3538690282 conn(0x7f1968074dc0 msgr2=0x7f196810e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:36.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.251+0000 7f196759e700 1 -- 172.21.15.64:0/3538690282 shutdown_connections 2023-09-13T03:13:36.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.251+0000 7f196759e700 1 -- 172.21.15.64:0/3538690282 wait complete. 2023-09-13T03:13:36.277 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-09-13T03:13:36.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.511+0000 7f0af63ff700 1 -- 172.21.15.64:0/3625459455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0af0102090 msgr2=0x7f0af010a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:36.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.511+0000 7f0af63ff700 1 --2- 172.21.15.64:0/3625459455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0af0102090 0x7f0af010a570 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4009990 tx=0x7f0ae4009ca0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.511+0000 7f0af63ff700 1 -- 172.21.15.64:0/3625459455 shutdown_connections 2023-09-13T03:13:36.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.511+0000 7f0af63ff700 1 --2- 172.21.15.64:0/3625459455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0af0101770 0x7f0af0101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.511+0000 7f0af63ff700 1 --2- 172.21.15.64:0/3625459455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0af010ab40 0x7f0af010cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.511+0000 7f0af63ff700 1 --2- 172.21.15.64:0/3625459455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0af0102090 0x7f0af010a570 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.511+0000 7f0af63ff700 1 -- 172.21.15.64:0/3625459455 >> 172.21.15.64:0/3625459455 conn(0x7f0af00faf50 msgr2=0x7f0af00fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:36.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.511+0000 7f0af63ff700 1 -- 172.21.15.64:0/3625459455 shutdown_connections 2023-09-13T03:13:36.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.511+0000 7f0af63ff700 1 -- 172.21.15.64:0/3625459455 wait complete. 2023-09-13T03:13:36.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.513+0000 7f0af63ff700 1 Processor -- start 2023-09-13T03:13:36.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.513+0000 7f0af63ff700 1 -- start start 2023-09-13T03:13:36.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.513+0000 7f0af63ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0af0101770 0x7f0af0071d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.513+0000 7f0af63ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0af0102090 0x7f0af0072290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.513+0000 7f0af63ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0af010ab40 0x7f0af00ff3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.514+0000 7f0af63ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af00ffb90 con 0x7f0af0101770 2023-09-13T03:13:36.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.514+0000 7f0af63ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af00ffd00 con 0x7f0af010ab40 2023-09-13T03:13:36.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.514+0000 7f0af63ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af00ffec0 con 0x7f0af0102090 2023-09-13T03:13:36.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.514+0000 7f0aef7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0af0102090 0x7f0af0072290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.514+0000 7f0af499c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0af010ab40 0x7f0af00ff3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.514+0000 7f0aef7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0af0102090 0x7f0af0072290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:54346/0 (socket says 172.21.15.64:54346) 2023-09-13T03:13:36.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.514+0000 7f0aef7fe700 1 -- 172.21.15.64:0/619862624 learned_addr learned my addr 172.21.15.64:0/619862624 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:36.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.514+0000 7f0aeffff700 1 --2- 172.21.15.64:0/619862624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0af0101770 0x7f0af0071d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.515+0000 7f0aef7fe700 1 -- 172.21.15.64:0/619862624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0af010ab40 msgr2=0x7f0af00ff3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:36.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.515+0000 7f0aef7fe700 1 --2- 172.21.15.64:0/619862624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0af010ab40 0x7f0af00ff3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.515+0000 7f0aef7fe700 1 -- 172.21.15.64:0/619862624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0af0101770 msgr2=0x7f0af0071d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:36.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.515+0000 7f0aef7fe700 1 --2- 172.21.15.64:0/619862624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0af0101770 0x7f0af0071d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.515+0000 7f0aef7fe700 1 -- 172.21.15.64:0/619862624 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae40095f0 con 0x7f0af0102090 2023-09-13T03:13:36.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.515+0000 7f0aeffff700 1 --2- 172.21.15.64:0/619862624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0af0101770 0x7f0af0071d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:13:36.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.515+0000 7f0aef7fe700 1 --2- 172.21.15.64:0/619862624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0af0102090 0x7f0af0072290 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4005910 tx=0x7f0ae400b610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:36.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.515+0000 7f0aed7fa700 1 -- 172.21.15.64:0/619862624 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae401d070 con 0x7f0af0102090 2023-09-13T03:13:36.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.515+0000 7f0af63ff700 1 -- 172.21.15.64:0/619862624 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af0100140 con 0x7f0af0102090 2023-09-13T03:13:36.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.516+0000 7f0af63ff700 1 -- 172.21.15.64:0/619862624 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af01ade10 con 0x7f0af0102090 2023-09-13T03:13:36.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.516+0000 7f0aed7fa700 1 -- 172.21.15.64:0/619862624 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ae400fb80 con 0x7f0af0102090 2023-09-13T03:13:36.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.516+0000 7f0aed7fa700 1 -- 172.21.15.64:0/619862624 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae400e3e0 con 0x7f0af0102090 2023-09-13T03:13:36.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.521+0000 7f0aed7fa700 1 -- 172.21.15.64:0/619862624 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 24) v1 ==== 135435+0+0 (secure 0 0 0) 0x7f0ae400fcf0 con 0x7f0af0102090 2023-09-13T03:13:36.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.522+0000 7f0b09d4db80 1 Processor -- start 2023-09-13T03:13:36.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.522+0000 7f0b09d4db80 1 -- start start 2023-09-13T03:13:36.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.523+0000 7f0b09d4db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556496f50710 0x556496f50af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.523+0000 7f0b09d4db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556496f51030 0x556496f5a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.523+0000 7f0b09d4db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556496f5ae40 0x556496f5d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.523+0000 7f0b09d4db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x556496ea9730 con 0x556496f50710 2023-09-13T03:13:36.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.523+0000 7f0b09d4db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556496ea7050 con 0x556496f51030 2023-09-13T03:13:36.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.523+0000 7f0b09d4db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x556496ea2940 con 0x556496f5ae40 2023-09-13T03:13:36.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.523+0000 7f0aed7fa700 1 --2- 172.21.15.64:0/619862624 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f0ad80a0940 0x7f0ad80a2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.523+0000 7f0aed7fa700 1 -- 172.21.15.64:0/619862624 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f0ae40cd5b0 con 0x7f0af0102090 2023-09-13T03:13:36.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.524+0000 7f0aeffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556496f51030 0x556496f5a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.524+0000 7f0aef7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556496f5ae40 0x556496f5d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.524+0000 7f0aef7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556496f5ae40 0x556496f5d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:54360/0 (socket says 172.21.15.64:54360) 2023-09-13T03:13:36.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.524+0000 7f0aef7fe700 1 -- 172.21.15.64:0/14004935 learned_addr learned my addr 172.21.15.64:0/14004935 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:36.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.524+0000 7f0af499c700 1 --2- 172.21.15.64:0/619862624 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f0ad80a0940 0x7f0ad80a2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.524+0000 7f0af499c700 1 --2- 172.21.15.64:0/14004935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556496f50710 0x556496f50af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.524+0000 7f0aef7fe700 1 -- 172.21.15.64:0/14004935 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556496f51030 msgr2=0x556496f5a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:36.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.524+0000 7f0aef7fe700 1 --2- 172.21.15.64:0/14004935 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556496f51030 0x556496f5a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.524+0000 7f0aef7fe700 1 -- 172.21.15.64:0/14004935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556496f50710 msgr2=0x556496f50af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:36.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.524+0000 7f0aef7fe700 1 --2- 172.21.15.64:0/14004935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556496f50710 0x556496f50af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.524+0000 7f0aef7fe700 1 -- 172.21.15.64:0/14004935 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556496ccede0 con 0x556496f5ae40 2023-09-13T03:13:36.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.525+0000 7f0af499c700 1 --2- 172.21.15.64:0/14004935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556496f50710 0x556496f50af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:13:36.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.525+0000 7f0aeffff700 1 --2- 172.21.15.64:0/14004935 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556496f51030 0x556496f5a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:13:36.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.525+0000 7f0aef7fe700 1 --2- 172.21.15.64:0/14004935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556496f5ae40 0x556496f5d230 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0ae409a670 tx=0x7f0ae4009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=236d06b187f16b72 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:36.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.525+0000 7f0af499c700 1 --2- 172.21.15.64:0/619862624 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f0ad80a0940 0x7f0ad80a2e00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8009710 tx=0x7f0ae80090b0 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:36.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.525+0000 7f0ad67fc700 1 -- 172.21.15.64:0/14004935 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae409b070 con 0x556496f5ae40 2023-09-13T03:13:36.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.525+0000 7f0ad67fc700 1 -- 172.21.15.64:0/14004935 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ae409a970 con 0x556496f5ae40 2023-09-13T03:13:36.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.525+0000 7f0ad67fc700 1 -- 172.21.15.64:0/14004935 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae40a0860 con 0x556496f5ae40 2023-09-13T03:13:36.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.526+0000 7f0b09d4db80 1 -- 172.21.15.64:0/14004935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556496f5ae40 msgr2=0x556496f5d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:36.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.526+0000 7f0b09d4db80 1 --2- 172.21.15.64:0/14004935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556496f5ae40 0x556496f5d230 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0ae409a670 tx=0x7f0ae4009310 comp rx=0 tx=0).stop 2023-09-13T03:13:36.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.526+0000 7f0b09d4db80 1 -- 172.21.15.64:0/14004935 shutdown_connections 2023-09-13T03:13:36.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.526+0000 7f0b09d4db80 1 --2- 172.21.15.64:0/14004935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556496f5ae40 0x556496f5d230 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.526+0000 7f0b09d4db80 1 --2- 172.21.15.64:0/14004935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556496f50710 0x556496f50af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.526+0000 7f0b09d4db80 1 --2- 172.21.15.64:0/14004935 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556496f51030 0x556496f5a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.526+0000 7f0b09d4db80 1 -- 172.21.15.64:0/14004935 >> 172.21.15.64:0/14004935 conn(0x556496ec5090 msgr2=0x556496f4bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:36.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.526+0000 7f0b09d4db80 1 -- 172.21.15.64:0/14004935 shutdown_connections 2023-09-13T03:13:36.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.526+0000 7f0b09d4db80 1 -- 172.21.15.64:0/14004935 wait complete. 2023-09-13T03:13:36.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.527+0000 7f0b09d4db80 1 -- start start 2023-09-13T03:13:36.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.527+0000 7f0b09d4db80 1 Processor -- start 2023-09-13T03:13:36.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.528+0000 7f0b09d4db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556496f50710 0x556496f5b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.528+0000 7f0b09d4db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556496f51030 0x556496f5b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.528+0000 7f0b09d4db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556496f5ae40 0x556496f662a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.528+0000 7f0b09d4db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x556496e40b70 con 0x556496f51030 2023-09-13T03:13:36.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.528+0000 7f0b09d4db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556496e40870 con 0x556496f50710 2023-09-13T03:13:36.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.528+0000 7f0b09d4db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x556496d24920 con 0x556496f5ae40 2023-09-13T03:13:36.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.528+0000 7f0b09d4db80 10 client.0 _ll_register_callbacks cb 0x556496d8aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:13:36.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.528+0000 7f0aeffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556496f50710 0x556496f5b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.528+0000 7f0aef7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556496f51030 0x556496f5b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.528+0000 7f0aeffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556496f50710 0x556496f5b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:57266/0 (socket says 172.21.15.64:57266) 2023-09-13T03:13:36.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.528+0000 7f0aeffff700 1 -- 172.21.15.64:0/4009228019 learned_addr learned my addr 172.21.15.64:0/4009228019 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:36.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.529+0000 7f0af499c700 1 --2- 172.21.15.64:0/4009228019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556496f5ae40 0x556496f662a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.529+0000 7f0aeffff700 1 -- 172.21.15.64:0/4009228019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556496f5ae40 msgr2=0x556496f662a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:36.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.529+0000 7f0aeffff700 1 --2- 172.21.15.64:0/4009228019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556496f5ae40 0x556496f662a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.529+0000 7f0aeffff700 1 -- 172.21.15.64:0/4009228019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556496f51030 msgr2=0x556496f5b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:36.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.529+0000 7f0aeffff700 1 --2- 172.21.15.64:0/4009228019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556496f51030 0x556496f5b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.529+0000 7f0aeffff700 1 -- 172.21.15.64:0/4009228019 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ae00098f0 con 0x556496f50710 2023-09-13T03:13:36.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.529+0000 7f0aef7fe700 1 --2- 172.21.15.64:0/4009228019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556496f51030 0x556496f5b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:13:36.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.529+0000 7f0aeffff700 1 --2- 172.21.15.64:0/4009228019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556496f50710 0x556496f5b220 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0012950 tx=0x7f0ae0012c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e079ea1346091ae2 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:36.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.529+0000 7f0acf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:13:36.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.530+0000 7f0acf7fe700 1 -- 172.21.15.64:0/4009228019 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0017070 con 0x556496f50710 2023-09-13T03:13:36.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.530+0000 7f0acf7fe700 1 -- 172.21.15.64:0/4009228019 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f0ae0011a40 con 0x556496f50710 2023-09-13T03:13:36.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.530+0000 7f0b09d4db80 1 -- 172.21.15.64:0/4009228019 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556496dfa4f0 con 0x556496f50710 2023-09-13T03:13:36.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.530+0000 7f0acf7fe700 1 -- 172.21.15.64:0/4009228019 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ae0011c00 con 0x556496f50710 2023-09-13T03:13:36.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.530+0000 7f0b09d4db80 10 client.25823 fetch_fsmap learned FSMap version 26 2023-09-13T03:13:36.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.530+0000 7f0b09d4db80 1 -- 172.21.15.64:0/4009228019 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f0af01ade10 con 0x556496f50710 2023-09-13T03:13:36.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.530+0000 7f0acf7fe700 1 -- 172.21.15.64:0/4009228019 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556496dfa4f0 con 0x556496f50710 2023-09-13T03:13:36.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.531+0000 7f0acf7fe700 1 -- 172.21.15.64:0/4009228019 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0ae0003ba0 con 0x556496f50710 2023-09-13T03:13:36.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.531+0000 7f0b09d4db80 10 client.25823 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:13:36.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.531+0000 7f0b09d4db80 10 client.25823 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-09-13T03:13:36.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.531+0000 7f0b09d4db80 20 client.25823 populate_metadata read hostname 'smithi064' 2023-09-13T03:13:36.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.531+0000 7f0b09d4db80 1 --2- 172.21.15.64:0/4009228019 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x556496f72da0 0x556496f751c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.531+0000 7f0b09d4db80 4 client.25823 mds_command: new command op to 24649 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:13:36.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.531+0000 7f0b09d4db80 1 -- 172.21.15.64:0/4009228019 --> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556496eb78a0 con 0x556496f72da0 2023-09-13T03:13:36.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.531+0000 7f0aef7fe700 1 --2- 172.21.15.64:0/4009228019 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x556496f72da0 0x556496f751c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.532+0000 7f0acf7fe700 10 client.25823 ms_handle_connect on v2:172.21.15.170:6838/2189703596 2023-09-13T03:13:36.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.532+0000 7f0aef7fe700 1 --2- 172.21.15.64:0/4009228019 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x556496f72da0 0x556496f751c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fad1fb8cf46fb12c server_cookie=3dd7e956dcd1bae2 in_seq=0 out_seq=0 2023-09-13T03:13:36.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.533+0000 7f0acf7fe700 1 -- 172.21.15.64:0/4009228019 <== mds.1 v2:172.21.15.170:6838/2189703596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556496eb78a0 con 0x556496f72da0 2023-09-13T03:13:36.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.533+0000 7f0acf7fe700 10 client.25823 handle_command_reply: tid=0 2023-09-13T03:13:36.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.533+0000 7f0b09d4db80 1 client.25823 shutdown 2023-09-13T03:13:36.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.534+0000 7f0b09d4db80 1 -- 172.21.15.64:0/4009228019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556496f50710 msgr2=0x556496f5b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:36.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.534+0000 7f0b09d4db80 1 --2- 172.21.15.64:0/4009228019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556496f50710 0x556496f5b220 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0012950 tx=0x7f0ae0012c60 comp rx=0 tx=0).stop 2023-09-13T03:13:36.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.534+0000 7f0b09d4db80 1 -- 172.21.15.64:0/4009228019 shutdown_connections 2023-09-13T03:13:36.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.534+0000 7f0b09d4db80 1 --2- 172.21.15.64:0/4009228019 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x556496f72da0 0x556496f751c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.534+0000 7f0b09d4db80 1 --2- 172.21.15.64:0/4009228019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556496f50710 0x556496f5b220 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.534+0000 7f0b09d4db80 1 --2- 172.21.15.64:0/4009228019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556496f5ae40 0x556496f662a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.534+0000 7f0acf7fe700 0 client.25823 ms_handle_reset on v2:172.21.15.170:6838/2189703596 2023-09-13T03:13:36.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.534+0000 7f0b09d4db80 1 --2- 172.21.15.64:0/4009228019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556496f51030 0x556496f5b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.534+0000 7f0b09d4db80 1 -- 172.21.15.64:0/4009228019 >> 172.21.15.64:0/4009228019 conn(0x556496ec5090 msgr2=0x556496f4b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:36.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.534+0000 7f0b09d4db80 1 -- 172.21.15.64:0/4009228019 shutdown_connections 2023-09-13T03:13:36.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.534+0000 7f0b09d4db80 1 -- 172.21.15.64:0/4009228019 wait complete. 2023-09-13T03:13:36.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.534+0000 7f0b09d4db80 20 client.25823 trim_cache size 0 max 16384 2023-09-13T03:13:36.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.548+0000 7f0b09d4db80 1 Processor -- start 2023-09-13T03:13:36.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.548+0000 7f0b09d4db80 1 -- start start 2023-09-13T03:13:36.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.549+0000 7f0b09d4db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556496f50710 0x556496f75190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.549+0000 7f0b09d4db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556496f51030 0x556496f59640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.549+0000 7f0b09d4db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556496f5ae40 0x556496f65970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.549+0000 7f0b09d4db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x556496f52280 con 0x556496f5ae40 2023-09-13T03:13:36.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.549+0000 7f0b09d4db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556496f66f20 con 0x556496f50710 2023-09-13T03:13:36.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.549+0000 7f0b09d4db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x556496f5a500 con 0x556496f51030 2023-09-13T03:13:36.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.549+0000 7f0aef7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556496f51030 0x556496f59640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.549+0000 7f0aef7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556496f51030 0x556496f59640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:54366/0 (socket says 172.21.15.64:54366) 2023-09-13T03:13:36.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.549+0000 7f0aef7fe700 1 -- 172.21.15.64:0/2110087225 learned_addr learned my addr 172.21.15.64:0/2110087225 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:36.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.549+0000 7f0aeffff700 1 --2- 172.21.15.64:0/2110087225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556496f50710 0x556496f75190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.549+0000 7f0af499c700 1 --2- 172.21.15.64:0/2110087225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556496f5ae40 0x556496f65970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.549+0000 7f0aeffff700 1 -- 172.21.15.64:0/2110087225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556496f51030 msgr2=0x556496f59640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:36.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.549+0000 7f0aeffff700 1 --2- 172.21.15.64:0/2110087225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556496f51030 0x556496f59640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.550+0000 7f0aeffff700 1 -- 172.21.15.64:0/2110087225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556496f5ae40 msgr2=0x556496f65970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:36.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.550+0000 7f0aeffff700 1 --2- 172.21.15.64:0/2110087225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556496f5ae40 0x556496f65970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.550+0000 7f0aeffff700 1 -- 172.21.15.64:0/2110087225 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556496cb7c60 con 0x556496f50710 2023-09-13T03:13:36.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.550+0000 7f0aeffff700 1 --2- 172.21.15.64:0/2110087225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556496f50710 0x556496f75190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0000c00 tx=0x7f0ae0009710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc5170b8bc3c60e6 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:36.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.550+0000 7f0ad6ffd700 1 -- 172.21.15.64:0/2110087225 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0017040 con 0x556496f50710 2023-09-13T03:13:36.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.550+0000 7f0ad6ffd700 1 -- 172.21.15.64:0/2110087225 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ae00092d0 con 0x556496f50710 2023-09-13T03:13:36.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.550+0000 7f0ad6ffd700 1 -- 172.21.15.64:0/2110087225 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0005530 con 0x556496f50710 2023-09-13T03:13:36.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.551+0000 7f0b09d4db80 1 -- 172.21.15.64:0/2110087225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556496f50710 msgr2=0x556496f75190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:36.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.551+0000 7f0b09d4db80 1 --2- 172.21.15.64:0/2110087225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556496f50710 0x556496f75190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0000c00 tx=0x7f0ae0009710 comp rx=0 tx=0).stop 2023-09-13T03:13:36.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.551+0000 7f0b09d4db80 1 -- 172.21.15.64:0/2110087225 shutdown_connections 2023-09-13T03:13:36.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.551+0000 7f0b09d4db80 1 --2- 172.21.15.64:0/2110087225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556496f50710 0x556496f75190 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.551+0000 7f0b09d4db80 1 --2- 172.21.15.64:0/2110087225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556496f5ae40 0x556496f65970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.551+0000 7f0b09d4db80 1 --2- 172.21.15.64:0/2110087225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556496f51030 0x556496f59640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.551+0000 7f0b09d4db80 1 -- 172.21.15.64:0/2110087225 >> 172.21.15.64:0/2110087225 conn(0x556496ec5090 msgr2=0x556496f4a320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:36.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.551+0000 7f0b09d4db80 1 -- 172.21.15.64:0/2110087225 shutdown_connections 2023-09-13T03:13:36.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.551+0000 7f0b09d4db80 1 -- 172.21.15.64:0/2110087225 wait complete. 2023-09-13T03:13:36.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.552+0000 7f0b09d4db80 1 -- start start 2023-09-13T03:13:36.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.552+0000 7f0b09d4db80 1 Processor -- start 2023-09-13T03:13:36.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.552+0000 7f0b09d4db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556496f50710 0x556496f5fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.552+0000 7f0b09d4db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556496f51030 0x556496f60180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.552+0000 7f0b09d4db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556496f5ae40 0x556496f5dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.552+0000 7f0b09d4db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x556496f5e6f0 con 0x556496f50710 2023-09-13T03:13:36.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.552+0000 7f0b09d4db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556496f5e860 con 0x556496f51030 2023-09-13T03:13:36.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.552+0000 7f0b09d4db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x556496f5ea20 con 0x556496f5ae40 2023-09-13T03:13:36.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.553+0000 7f0b09d4db80 10 client.0 _ll_register_callbacks cb 0x7f0ae0012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:13:36.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.553+0000 7f0aeffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556496f50710 0x556496f5fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.553+0000 7f0aef7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556496f51030 0x556496f60180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.553+0000 7f0aeffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556496f50710 0x556496f5fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59218/0 (socket says 172.21.15.64:59218) 2023-09-13T03:13:36.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.553+0000 7f0aeffff700 1 -- 172.21.15.64:0/1787972715 learned_addr learned my addr 172.21.15.64:0/1787972715 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:36.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.553+0000 7f0af499c700 1 --2- 172.21.15.64:0/1787972715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556496f5ae40 0x556496f5dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.553+0000 7f0aef7fe700 1 -- 172.21.15.64:0/1787972715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556496f5ae40 msgr2=0x556496f5dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:36.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.553+0000 7f0aef7fe700 1 --2- 172.21.15.64:0/1787972715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556496f5ae40 0x556496f5dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.553+0000 7f0aef7fe700 1 -- 172.21.15.64:0/1787972715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556496f50710 msgr2=0x556496f5fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:36.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.553+0000 7f0aef7fe700 1 --2- 172.21.15.64:0/1787972715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556496f50710 0x556496f5fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.553+0000 7f0aef7fe700 1 -- 172.21.15.64:0/1787972715 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af01ade10 con 0x556496f51030 2023-09-13T03:13:36.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.554+0000 7f0aef7fe700 1 --2- 172.21.15.64:0/1787972715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556496f51030 0x556496f60180 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4096e70 tx=0x7f0ae409f660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df95d9184f6bc06a server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:36.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.554+0000 7f0acf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:13:36.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.554+0000 7f0acf7fe700 1 -- 172.21.15.64:0/1787972715 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae40a2950 con 0x556496f51030 2023-09-13T03:13:36.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.554+0000 7f0b09d4db80 1 -- 172.21.15.64:0/1787972715 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ae409a970 con 0x556496f51030 2023-09-13T03:13:36.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.554+0000 7f0acf7fe700 1 -- 172.21.15.64:0/1787972715 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f0ae409e7a0 con 0x556496f51030 2023-09-13T03:13:36.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.554+0000 7f0acf7fe700 1 -- 172.21.15.64:0/1787972715 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ae409fbf0 con 0x556496f51030 2023-09-13T03:13:36.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.554+0000 7f0b09d4db80 10 client.25829 fetch_fsmap learned FSMap version 26 2023-09-13T03:13:36.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.554+0000 7f0b09d4db80 1 -- 172.21.15.64:0/1787972715 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x556496cf5250 con 0x556496f51030 2023-09-13T03:13:36.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.554+0000 7f0acf7fe700 1 -- 172.21.15.64:0/1787972715 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ae409a970 con 0x556496f51030 2023-09-13T03:13:36.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.555+0000 7f0acf7fe700 1 -- 172.21.15.64:0/1787972715 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0ae40a2550 con 0x556496f51030 2023-09-13T03:13:36.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.555+0000 7f0b09d4db80 10 client.25829 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:13:36.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.555+0000 7f0b09d4db80 10 client.25829 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-09-13T03:13:36.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.555+0000 7f0b09d4db80 20 client.25829 populate_metadata read hostname 'smithi064' 2023-09-13T03:13:36.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.555+0000 7f0b09d4db80 1 --2- 172.21.15.64:0/1787972715 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x556496fc7900 0x556496fc9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.555+0000 7f0b09d4db80 4 client.25829 mds_command: new command op to 24649 tid=0[{"prefix": "damage ls"}] 2023-09-13T03:13:36.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.555+0000 7f0b09d4db80 1 -- 172.21.15.64:0/1787972715 --> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556496e9b920 con 0x556496fc7900 2023-09-13T03:13:36.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.555+0000 7f0aeffff700 1 --2- 172.21.15.64:0/1787972715 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x556496fc7900 0x556496fc9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.556+0000 7f0aeffff700 1 --2- 172.21.15.64:0/1787972715 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x556496fc7900 0x556496fc9d20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=89216502ba59d26d server_cookie=2a3d1ff67d070f61 in_seq=0 out_seq=0 2023-09-13T03:13:36.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.556+0000 7f0acf7fe700 10 client.25829 ms_handle_connect on v2:172.21.15.170:6838/2189703596 2023-09-13T03:13:36.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.556+0000 7f0acf7fe700 1 -- 172.21.15.64:0/1787972715 <== mds.1 v2:172.21.15.170:6838/2189703596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556496e9b920 con 0x556496fc7900 2023-09-13T03:13:36.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.556+0000 7f0acf7fe700 10 client.25829 handle_command_reply: tid=0 2023-09-13T03:13:36.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.556+0000 7f0b09d4db80 1 client.25829 shutdown 2023-09-13T03:13:36.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.556+0000 7f0b09d4db80 1 -- 172.21.15.64:0/1787972715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556496f51030 msgr2=0x556496f60180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:36.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.556+0000 7f0b09d4db80 1 --2- 172.21.15.64:0/1787972715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556496f51030 0x556496f60180 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4096e70 tx=0x7f0ae409f660 comp rx=0 tx=0).stop 2023-09-13T03:13:36.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.556+0000 7f0b09d4db80 1 -- 172.21.15.64:0/1787972715 shutdown_connections 2023-09-13T03:13:36.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.556+0000 7f0b09d4db80 1 --2- 172.21.15.64:0/1787972715 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x556496fc7900 0x556496fc9d20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.556+0000 7f0b09d4db80 1 --2- 172.21.15.64:0/1787972715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556496f5ae40 0x556496f5dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.556+0000 7f0b09d4db80 1 --2- 172.21.15.64:0/1787972715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556496f50710 0x556496f5fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.556+0000 7f0acf7fe700 0 client.25829 ms_handle_reset on v2:172.21.15.170:6838/2189703596 2023-09-13T03:13:36.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.556+0000 7f0b09d4db80 1 --2- 172.21.15.64:0/1787972715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556496f51030 0x556496f60180 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.556+0000 7f0b09d4db80 1 -- 172.21.15.64:0/1787972715 >> 172.21.15.64:0/1787972715 conn(0x556496ec5090 msgr2=0x556496f4f700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:36.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.557+0000 7f0b09d4db80 1 -- 172.21.15.64:0/1787972715 shutdown_connections 2023-09-13T03:13:36.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.557+0000 7f0b09d4db80 1 -- 172.21.15.64:0/1787972715 wait complete. 2023-09-13T03:13:36.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.557+0000 7f0b09d4db80 20 client.25829 trim_cache size 0 max 16384 2023-09-13T03:13:36.570 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2023-09-13T03:13:36.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.557+0000 7f0af63ff700 1 -- 172.21.15.64:0/619862624 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f0ad80a0940 msgr2=0x7f0ad80a2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:36.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.557+0000 7f0af63ff700 1 --2- 172.21.15.64:0/619862624 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f0ad80a0940 0x7f0ad80a2e00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8009710 tx=0x7f0ae80090b0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.557+0000 7f0af63ff700 1 -- 172.21.15.64:0/619862624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0af0102090 msgr2=0x7f0af0072290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:36.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.557+0000 7f0af63ff700 1 --2- 172.21.15.64:0/619862624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0af0102090 0x7f0af0072290 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4005910 tx=0x7f0ae400b610 comp rx=0 tx=0).stop 2023-09-13T03:13:36.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.558+0000 7f0af63ff700 1 -- 172.21.15.64:0/619862624 shutdown_connections 2023-09-13T03:13:36.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.558+0000 7f0af63ff700 1 --2- 172.21.15.64:0/619862624 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f0ad80a0940 0x7f0ad80a2e00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.558+0000 7f0af63ff700 1 --2- 172.21.15.64:0/619862624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0af010ab40 0x7f0af00ff3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.558+0000 7f0af63ff700 1 --2- 172.21.15.64:0/619862624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0af0101770 0x7f0af0071d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.558+0000 7f0af63ff700 1 --2- 172.21.15.64:0/619862624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0af0102090 0x7f0af0072290 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.558+0000 7f0af63ff700 1 -- 172.21.15.64:0/619862624 >> 172.21.15.64:0/619862624 conn(0x7f0af00faf50 msgr2=0x7f0af00fbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:36.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.558+0000 7f0af63ff700 1 -- 172.21.15.64:0/619862624 shutdown_connections 2023-09-13T03:13:36.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.558+0000 7f0af63ff700 1 -- 172.21.15.64:0/619862624 wait complete. 2023-09-13T03:13:36.578 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-09-13T03:13:36.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.800+0000 7f77547ff700 1 -- 172.21.15.64:0/2660873234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f774c102040 msgr2=0x7f774c10a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:36.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.800+0000 7f77547ff700 1 --2- 172.21.15.64:0/2660873234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f774c102040 0x7f774c10a520 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f773c009990 tx=0x7f773c009ca0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.800+0000 7f77547ff700 1 -- 172.21.15.64:0/2660873234 shutdown_connections 2023-09-13T03:13:36.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.800+0000 7f77547ff700 1 --2- 172.21.15.64:0/2660873234 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f774c10ab40 0x7f774c10d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.800+0000 7f77547ff700 1 --2- 172.21.15.64:0/2660873234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f774c101720 0x7f774c101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.800+0000 7f77547ff700 1 --2- 172.21.15.64:0/2660873234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f774c102040 0x7f774c10a520 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.800+0000 7f77547ff700 1 -- 172.21.15.64:0/2660873234 >> 172.21.15.64:0/2660873234 conn(0x7f774c0faf60 msgr2=0x7f774c0fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:36.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.800+0000 7f77547ff700 1 -- 172.21.15.64:0/2660873234 shutdown_connections 2023-09-13T03:13:36.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.801+0000 7f77547ff700 1 -- 172.21.15.64:0/2660873234 wait complete. 2023-09-13T03:13:36.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.802+0000 7f77547ff700 1 Processor -- start 2023-09-13T03:13:36.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.802+0000 7f77547ff700 1 -- start start 2023-09-13T03:13:36.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.803+0000 7f77547ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f774c101720 0x7f774c0ff370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.803+0000 7f77547ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f774c102040 0x7f774c0ff8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.803+0000 7f77547ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f774c10ab40 0x7f774c1a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.803+0000 7f77547ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f774c1000e0 con 0x7f774c101720 2023-09-13T03:13:36.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.803+0000 7f77547ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f774c1a4de0 con 0x7f774c102040 2023-09-13T03:13:36.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.803+0000 7f77547ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f774c1a4f20 con 0x7f774c10ab40 2023-09-13T03:13:36.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.803+0000 7f7751d9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f774c102040 0x7f774c0ff8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.803+0000 7f7751d9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f774c102040 0x7f774c0ff8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:57308/0 (socket says 172.21.15.64:57308) 2023-09-13T03:13:36.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.804+0000 7f7751d9a700 1 -- 172.21.15.64:0/1638697774 learned_addr learned my addr 172.21.15.64:0/1638697774 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:36.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.804+0000 7f775259b700 1 --2- 172.21.15.64:0/1638697774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f774c101720 0x7f774c0ff370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.804+0000 7f7751d9a700 1 -- 172.21.15.64:0/1638697774 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f774c10ab40 msgr2=0x7f774c1a48a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-13T03:13:36.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.804+0000 7f7751d9a700 1 --2- 172.21.15.64:0/1638697774 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f774c10ab40 0x7f774c1a48a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.804+0000 7f7751d9a700 1 -- 172.21.15.64:0/1638697774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f774c101720 msgr2=0x7f774c0ff370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:36.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.804+0000 7f7751d9a700 1 --2- 172.21.15.64:0/1638697774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f774c101720 0x7f774c0ff370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.804+0000 7f7751d9a700 1 -- 172.21.15.64:0/1638697774 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f773c0095f0 con 0x7f774c102040 2023-09-13T03:13:36.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.804+0000 7f775259b700 1 --2- 172.21.15.64:0/1638697774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f774c101720 0x7f774c0ff370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:13:36.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.804+0000 7f7751d9a700 1 --2- 172.21.15.64:0/1638697774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f774c102040 0x7f774c0ff8b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f773c005910 tx=0x7f773c0057d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:36.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.805+0000 7f77437fe700 1 -- 172.21.15.64:0/1638697774 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773c01d070 con 0x7f774c102040 2023-09-13T03:13:36.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.805+0000 7f77437fe700 1 -- 172.21.15.64:0/1638697774 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f773c022470 con 0x7f774c102040 2023-09-13T03:13:36.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.805+0000 7f77437fe700 1 -- 172.21.15.64:0/1638697774 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773c017650 con 0x7f774c102040 2023-09-13T03:13:36.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.805+0000 7f77547ff700 1 -- 172.21.15.64:0/1638697774 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f774c1a5120 con 0x7f774c102040 2023-09-13T03:13:36.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.805+0000 7f77547ff700 1 -- 172.21.15.64:0/1638697774 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f774c1a5550 con 0x7f774c102040 2023-09-13T03:13:36.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.810+0000 7f77437fe700 1 -- 172.21.15.64:0/1638697774 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) v1 ==== 135435+0+0 (secure 0 0 0) 0x7f773c0225e0 con 0x7f774c102040 2023-09-13T03:13:36.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.811+0000 7f77681cab80 1 Processor -- start 2023-09-13T03:13:36.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.811+0000 7f77681cab80 1 -- start start 2023-09-13T03:13:36.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.812+0000 7f77681cab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56112098c710 0x56112098caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.812+0000 7f77681cab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56112098d030 0x561120996900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.812+0000 7f77681cab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561120996e40 0x561120999230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.812+0000 7f77681cab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611208e5730 con 0x561120996e40 2023-09-13T03:13:36.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.812+0000 7f77681cab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611208e3050 con 0x56112098c710 2023-09-13T03:13:36.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.812+0000 7f77681cab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611208de940 con 0x56112098d030 2023-09-13T03:13:36.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.812+0000 7f775259b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56112098d030 0x561120996900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.812+0000 7f775259b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56112098d030 0x561120996900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:54402/0 (socket says 172.21.15.64:54402) 2023-09-13T03:13:36.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.812+0000 7f775259b700 1 -- 172.21.15.64:0/3398983524 learned_addr learned my addr 172.21.15.64:0/3398983524 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:36.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.812+0000 7f7752d9c700 1 --2- 172.21.15.64:0/3398983524 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56112098c710 0x56112098caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.813+0000 7f775259b700 1 -- 172.21.15.64:0/3398983524 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56112098c710 msgr2=0x56112098caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:36.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.813+0000 7f775259b700 1 --2- 172.21.15.64:0/3398983524 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56112098c710 0x56112098caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.813+0000 7f775259b700 1 -- 172.21.15.64:0/3398983524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561120996e40 msgr2=0x561120999230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:36.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.813+0000 7f775259b700 1 --2- 172.21.15.64:0/3398983524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561120996e40 0x561120999230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.813+0000 7f775259b700 1 -- 172.21.15.64:0/3398983524 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56112070ade0 con 0x56112098d030 2023-09-13T03:13:36.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.813+0000 7f77437fe700 1 --2- 172.21.15.64:0/1638697774 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f77380a08c0 0x7f77380a2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.813+0000 7f775259b700 1 --2- 172.21.15.64:0/3398983524 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56112098d030 0x561120996900 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7748009de0 tx=0x7f77480116c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55f612e3c4cd80ae server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:36.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.813+0000 7f77437fe700 1 -- 172.21.15.64:0/1638697774 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f773c0cd4d0 con 0x7f774c102040 2023-09-13T03:13:36.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.813+0000 7f7740ff9700 1 -- 172.21.15.64:0/3398983524 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7748011ed0 con 0x56112098d030 2023-09-13T03:13:36.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.813+0000 7f7740ff9700 1 -- 172.21.15.64:0/3398983524 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7748012910 con 0x56112098d030 2023-09-13T03:13:36.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.813+0000 7f7740ff9700 1 -- 172.21.15.64:0/3398983524 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774800f920 con 0x56112098d030 2023-09-13T03:13:36.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.814+0000 7f7752d9c700 1 --2- 172.21.15.64:0/1638697774 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f77380a08c0 0x7f77380a2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.814+0000 7f77681cab80 1 -- 172.21.15.64:0/3398983524 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56112098d030 msgr2=0x561120996900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:36.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.814+0000 7f77681cab80 1 --2- 172.21.15.64:0/3398983524 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56112098d030 0x561120996900 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7748009de0 tx=0x7f77480116c0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.815+0000 7f7752d9c700 1 --2- 172.21.15.64:0/1638697774 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f77380a08c0 0x7f77380a2d80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f774c101480 tx=0x7f7744009040 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:36.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.815+0000 7f77681cab80 1 -- 172.21.15.64:0/3398983524 shutdown_connections 2023-09-13T03:13:36.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.815+0000 7f77681cab80 1 --2- 172.21.15.64:0/3398983524 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56112098c710 0x56112098caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.815+0000 7f77681cab80 1 --2- 172.21.15.64:0/3398983524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561120996e40 0x561120999230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.815+0000 7f77681cab80 1 --2- 172.21.15.64:0/3398983524 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56112098d030 0x561120996900 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.815+0000 7f77681cab80 1 -- 172.21.15.64:0/3398983524 >> 172.21.15.64:0/3398983524 conn(0x561120901090 msgr2=0x561120987bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:36.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.815+0000 7f77681cab80 1 -- 172.21.15.64:0/3398983524 shutdown_connections 2023-09-13T03:13:36.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.816+0000 7f77681cab80 1 -- 172.21.15.64:0/3398983524 wait complete. 2023-09-13T03:13:36.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.816+0000 7f77681cab80 1 -- start start 2023-09-13T03:13:36.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.816+0000 7f77681cab80 1 Processor -- start 2023-09-13T03:13:36.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.817+0000 7f77681cab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56112098c710 0x561120993be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.817+0000 7f77681cab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56112098d030 0x561120994120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.817+0000 7f77681cab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561120996e40 0x561120994660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.817+0000 7f77681cab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56112087cb70 con 0x56112098c710 2023-09-13T03:13:36.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.817+0000 7f77681cab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56112087c870 con 0x561120996e40 2023-09-13T03:13:36.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.817+0000 7f77681cab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561120760920 con 0x56112098d030 2023-09-13T03:13:36.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.817+0000 7f77681cab80 10 client.0 _ll_register_callbacks cb 0x5611207c6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:13:36.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.817+0000 7f7751d9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56112098d030 0x561120994120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.817+0000 7f7751d9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56112098d030 0x561120994120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:54404/0 (socket says 172.21.15.64:54404) 2023-09-13T03:13:36.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.817+0000 7f7751d9a700 1 -- 172.21.15.64:0/3954389708 learned_addr learned my addr 172.21.15.64:0/3954389708 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:36.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.817+0000 7f775259b700 1 --2- 172.21.15.64:0/3954389708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56112098c710 0x561120993be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.818+0000 7f7751d9a700 1 -- 172.21.15.64:0/3954389708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561120996e40 msgr2=0x561120994660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-13T03:13:36.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.818+0000 7f7751d9a700 1 --2- 172.21.15.64:0/3954389708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561120996e40 0x561120994660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.818+0000 7f7751d9a700 1 -- 172.21.15.64:0/3954389708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56112098c710 msgr2=0x561120993be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:36.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.818+0000 7f7751d9a700 1 --2- 172.21.15.64:0/3954389708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56112098c710 0x561120993be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.818+0000 7f7751d9a700 1 -- 172.21.15.64:0/3954389708 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7748009a10 con 0x56112098d030 2023-09-13T03:13:36.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.818+0000 7f7751d9a700 1 --2- 172.21.15.64:0/3954389708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56112098d030 0x561120994120 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f773c0cac40 tx=0x7f773c0cb140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7dd843050ad0f96c server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:36.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.818+0000 7f7735ffb700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:13:36.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.818+0000 7f7735ffb700 1 -- 172.21.15.64:0/3954389708 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773c09a100 con 0x56112098d030 2023-09-13T03:13:36.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.818+0000 7f77681cab80 1 -- 172.21.15.64:0/3954389708 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5611208364f0 con 0x56112098d030 2023-09-13T03:13:36.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.818+0000 7f7735ffb700 1 -- 172.21.15.64:0/3954389708 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f773c09b110 con 0x56112098d030 2023-09-13T03:13:36.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.819+0000 7f7735ffb700 1 -- 172.21.15.64:0/3954389708 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f773c09a550 con 0x56112098d030 2023-09-13T03:13:36.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.819+0000 7f77681cab80 10 client.25837 fetch_fsmap learned FSMap version 26 2023-09-13T03:13:36.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.819+0000 7f77681cab80 1 -- 172.21.15.64:0/3954389708 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5611206f3c60 con 0x56112098d030 2023-09-13T03:13:36.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.819+0000 7f7735ffb700 1 -- 172.21.15.64:0/3954389708 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5611208364f0 con 0x56112098d030 2023-09-13T03:13:36.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.819+0000 7f7735ffb700 1 -- 172.21.15.64:0/3954389708 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f773c0a2560 con 0x56112098d030 2023-09-13T03:13:36.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.819+0000 7f77681cab80 10 client.25837 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:13:36.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.819+0000 7f77681cab80 10 client.25837 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T03:13:36.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.819+0000 7f77681cab80 20 client.25837 populate_metadata read hostname 'smithi064' 2023-09-13T03:13:36.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.819+0000 7f77681cab80 1 --2- 172.21.15.64:0/3954389708 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5611209a6670 0x5611209a8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.819+0000 7f77681cab80 4 client.25837 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:13:36.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.819+0000 7f77681cab80 1 -- 172.21.15.64:0/3954389708 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5611208f38a0 con 0x5611209a6670 2023-09-13T03:13:36.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.820+0000 7f775259b700 1 --2- 172.21.15.64:0/3954389708 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5611209a6670 0x5611209a8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.820+0000 7f775259b700 1 --2- 172.21.15.64:0/3954389708 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5611209a6670 0x5611209a8a60 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e08d0e4197155be7 server_cookie=5385e5e8a95bcafa in_seq=0 out_seq=0 2023-09-13T03:13:36.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.820+0000 7f7735ffb700 10 client.25837 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:13:36.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.821+0000 7f7735ffb700 1 -- 172.21.15.64:0/3954389708 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5611208f38a0 con 0x5611209a6670 2023-09-13T03:13:36.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.821+0000 7f7735ffb700 10 client.25837 handle_command_reply: tid=0 2023-09-13T03:13:36.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.821+0000 7f77681cab80 1 client.25837 shutdown 2023-09-13T03:13:36.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.821+0000 7f77681cab80 1 -- 172.21.15.64:0/3954389708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56112098d030 msgr2=0x561120994120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:36.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.821+0000 7f77681cab80 1 --2- 172.21.15.64:0/3954389708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56112098d030 0x561120994120 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f773c0cac40 tx=0x7f773c0cb140 comp rx=0 tx=0).stop 2023-09-13T03:13:36.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.822+0000 7f77681cab80 1 -- 172.21.15.64:0/3954389708 shutdown_connections 2023-09-13T03:13:36.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.822+0000 7f77681cab80 1 --2- 172.21.15.64:0/3954389708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561120996e40 0x561120994660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.822+0000 7f77681cab80 1 --2- 172.21.15.64:0/3954389708 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5611209a6670 0x5611209a8a60 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.822+0000 7f77681cab80 1 --2- 172.21.15.64:0/3954389708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56112098c710 0x561120993be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.822+0000 7f77681cab80 1 --2- 172.21.15.64:0/3954389708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56112098d030 0x561120994120 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.822+0000 7f77681cab80 1 -- 172.21.15.64:0/3954389708 >> 172.21.15.64:0/3954389708 conn(0x561120901090 msgr2=0x5611209877e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:36.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.822+0000 7f7735ffb700 0 client.25837 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:13:36.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.822+0000 7f77681cab80 1 -- 172.21.15.64:0/3954389708 shutdown_connections 2023-09-13T03:13:36.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.822+0000 7f77681cab80 1 -- 172.21.15.64:0/3954389708 wait complete. 2023-09-13T03:13:36.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.822+0000 7f77681cab80 20 client.25837 trim_cache size 0 max 16384 2023-09-13T03:13:36.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.848+0000 7f77681cab80 1 Processor -- start 2023-09-13T03:13:36.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.849+0000 7f77681cab80 1 -- start start 2023-09-13T03:13:36.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.849+0000 7f77681cab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56112098c710 0x56112099d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.849+0000 7f77681cab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56112098d030 0x561120990e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.849+0000 7f77681cab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561120996e40 0x5611209913b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.849+0000 7f77681cab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611209843d0 con 0x56112098c710 2023-09-13T03:13:36.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.849+0000 7f77681cab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611209996d0 con 0x56112098d030 2023-09-13T03:13:36.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.849+0000 7f77681cab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56112098ea10 con 0x561120996e40 2023-09-13T03:13:36.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.850+0000 7f775259b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56112098c710 0x56112099d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.850+0000 7f7751d9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56112098d030 0x561120990e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.850+0000 7f7751d9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56112098d030 0x561120990e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:57344/0 (socket says 172.21.15.64:57344) 2023-09-13T03:13:36.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.850+0000 7f7751d9a700 1 -- 172.21.15.64:0/2333036588 learned_addr learned my addr 172.21.15.64:0/2333036588 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:36.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.850+0000 7f7752d9c700 1 --2- 172.21.15.64:0/2333036588 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561120996e40 0x5611209913b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.850+0000 7f775259b700 1 -- 172.21.15.64:0/2333036588 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561120996e40 msgr2=0x5611209913b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:36.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.850+0000 7f775259b700 1 --2- 172.21.15.64:0/2333036588 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561120996e40 0x5611209913b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.850+0000 7f775259b700 1 -- 172.21.15.64:0/2333036588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56112098d030 msgr2=0x561120990e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:36.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.850+0000 7f775259b700 1 --2- 172.21.15.64:0/2333036588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56112098d030 0x561120990e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.850+0000 7f775259b700 1 -- 172.21.15.64:0/2333036588 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561120993750 con 0x56112098c710 2023-09-13T03:13:36.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.851+0000 7f775259b700 1 --2- 172.21.15.64:0/2333036588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56112098c710 0x56112099d9e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f774801c040 tx=0x7f7748011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7307efd0503995a server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:36.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.851+0000 7f77417fa700 1 -- 172.21.15.64:0/2333036588 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77480093a0 con 0x56112098c710 2023-09-13T03:13:36.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.851+0000 7f77417fa700 1 -- 172.21.15.64:0/2333036588 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7748011cd0 con 0x56112098c710 2023-09-13T03:13:36.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.851+0000 7f77417fa700 1 -- 172.21.15.64:0/2333036588 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7748012e60 con 0x56112098c710 2023-09-13T03:13:36.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.852+0000 7f77681cab80 1 -- 172.21.15.64:0/2333036588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56112098c710 msgr2=0x56112099d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:36.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.852+0000 7f77681cab80 1 --2- 172.21.15.64:0/2333036588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56112098c710 0x56112099d9e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f774801c040 tx=0x7f7748011b70 comp rx=0 tx=0).stop 2023-09-13T03:13:36.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.852+0000 7f77681cab80 1 -- 172.21.15.64:0/2333036588 shutdown_connections 2023-09-13T03:13:36.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.852+0000 7f77681cab80 1 --2- 172.21.15.64:0/2333036588 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561120996e40 0x5611209913b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.852+0000 7f77681cab80 1 --2- 172.21.15.64:0/2333036588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56112098c710 0x56112099d9e0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.852+0000 7f77681cab80 1 --2- 172.21.15.64:0/2333036588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56112098d030 0x561120990e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.852+0000 7f77681cab80 1 -- 172.21.15.64:0/2333036588 >> 172.21.15.64:0/2333036588 conn(0x561120901090 msgr2=0x56112098eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:36.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.852+0000 7f77681cab80 1 -- 172.21.15.64:0/2333036588 shutdown_connections 2023-09-13T03:13:36.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.852+0000 7f77681cab80 1 -- 172.21.15.64:0/2333036588 wait complete. 2023-09-13T03:13:36.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.853+0000 7f77681cab80 1 -- start start 2023-09-13T03:13:36.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.853+0000 7f77681cab80 1 Processor -- start 2023-09-13T03:13:36.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.854+0000 7f77681cab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56112098c710 0x56112099d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.854+0000 7f77681cab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56112098d030 0x56112099bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.854+0000 7f77681cab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561120996e40 0x56112099c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.854+0000 7f77681cab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56112099c640 con 0x56112098c710 2023-09-13T03:13:36.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.854+0000 7f77681cab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56112099c7b0 con 0x561120996e40 2023-09-13T03:13:36.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.854+0000 7f77681cab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56112099c970 con 0x56112098d030 2023-09-13T03:13:36.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.854+0000 7f77681cab80 10 client.0 _ll_register_callbacks cb 0x7f773c005c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:13:36.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.855+0000 7f775259b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56112098c710 0x56112099d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.855+0000 7f7751d9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56112098d030 0x56112099bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.855+0000 7f775259b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56112098c710 0x56112099d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59292/0 (socket says 172.21.15.64:59292) 2023-09-13T03:13:36.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.855+0000 7f775259b700 1 -- 172.21.15.64:0/3176780595 learned_addr learned my addr 172.21.15.64:0/3176780595 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:36.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.855+0000 7f7752d9c700 1 --2- 172.21.15.64:0/3176780595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561120996e40 0x56112099c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.855+0000 7f775259b700 1 -- 172.21.15.64:0/3176780595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56112098d030 msgr2=0x56112099bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:36.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.855+0000 7f775259b700 1 --2- 172.21.15.64:0/3176780595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56112098d030 0x56112099bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.855+0000 7f775259b700 1 -- 172.21.15.64:0/3176780595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561120996e40 msgr2=0x56112099c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:36.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.855+0000 7f775259b700 1 --2- 172.21.15.64:0/3176780595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561120996e40 0x56112099c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.855+0000 7f775259b700 1 -- 172.21.15.64:0/3176780595 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56112070ade0 con 0x56112098c710 2023-09-13T03:13:36.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.856+0000 7f775259b700 1 --2- 172.21.15.64:0/3176780595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56112098c710 0x56112099d880 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7748012010 tx=0x7f7748011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffd0f0c9dda112a server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:36.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.856+0000 7f7735ffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:13:36.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.856+0000 7f7735ffb700 1 -- 172.21.15.64:0/3176780595 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7748039070 con 0x56112098c710 2023-09-13T03:13:36.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.856+0000 7f77681cab80 1 -- 172.21.15.64:0/3176780595 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7748012910 con 0x56112098c710 2023-09-13T03:13:36.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.856+0000 7f7735ffb700 1 -- 172.21.15.64:0/3176780595 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f7748012450 con 0x56112098c710 2023-09-13T03:13:36.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.857+0000 7f7735ffb700 1 -- 172.21.15.64:0/3176780595 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7748012610 con 0x56112098c710 2023-09-13T03:13:36.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.857+0000 7f77681cab80 10 client.15813 fetch_fsmap learned FSMap version 26 2023-09-13T03:13:36.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.857+0000 7f77681cab80 1 -- 172.21.15.64:0/3176780595 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x561120731250 con 0x56112098c710 2023-09-13T03:13:36.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.857+0000 7f7735ffb700 1 -- 172.21.15.64:0/3176780595 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7748012910 con 0x56112098c710 2023-09-13T03:13:36.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.857+0000 7f7735ffb700 1 -- 172.21.15.64:0/3176780595 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f774803c030 con 0x56112098c710 2023-09-13T03:13:36.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.857+0000 7f77681cab80 10 client.15813 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:13:36.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.857+0000 7f77681cab80 10 client.15813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T03:13:36.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.857+0000 7f77681cab80 20 client.15813 populate_metadata read hostname 'smithi064' 2023-09-13T03:13:36.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.857+0000 7f77681cab80 1 --2- 172.21.15.64:0/3176780595 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5611209fb460 0x5611209fd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:36.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.857+0000 7f77681cab80 4 client.15813 mds_command: new command op to 24686 tid=0[{"prefix": "damage ls"}] 2023-09-13T03:13:36.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.857+0000 7f77681cab80 1 -- 172.21.15.64:0/3176780595 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5611208d7920 con 0x5611209fb460 2023-09-13T03:13:36.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.858+0000 7f7751d9a700 1 --2- 172.21.15.64:0/3176780595 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5611209fb460 0x5611209fd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:36.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.859+0000 7f7735ffb700 10 client.15813 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:13:36.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.859+0000 7f7751d9a700 1 --2- 172.21.15.64:0/3176780595 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5611209fb460 0x5611209fd850 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b278d9616e55bb3 server_cookie=8cb7386a6ed72a2b in_seq=0 out_seq=0 2023-09-13T03:13:36.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.859+0000 7f7735ffb700 1 -- 172.21.15.64:0/3176780595 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5611208d7920 con 0x5611209fb460 2023-09-13T03:13:36.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.859+0000 7f7735ffb700 10 client.15813 handle_command_reply: tid=0 2023-09-13T03:13:36.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.859+0000 7f77681cab80 1 client.15813 shutdown 2023-09-13T03:13:36.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.859+0000 7f77681cab80 1 -- 172.21.15.64:0/3176780595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56112098c710 msgr2=0x56112099d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:36.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.859+0000 7f77681cab80 1 --2- 172.21.15.64:0/3176780595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56112098c710 0x56112099d880 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7748012010 tx=0x7f7748011ef0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.859+0000 7f77681cab80 1 -- 172.21.15.64:0/3176780595 shutdown_connections 2023-09-13T03:13:36.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.859+0000 7f77681cab80 1 --2- 172.21.15.64:0/3176780595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561120996e40 0x56112099c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.859+0000 7f77681cab80 1 --2- 172.21.15.64:0/3176780595 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5611209fb460 0x5611209fd850 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.859+0000 7f77681cab80 1 --2- 172.21.15.64:0/3176780595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56112098c710 0x56112099d880 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.859+0000 7f77681cab80 1 --2- 172.21.15.64:0/3176780595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56112098d030 0x56112099bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.859+0000 7f77681cab80 1 -- 172.21.15.64:0/3176780595 >> 172.21.15.64:0/3176780595 conn(0x561120901090 msgr2=0x56112098b700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:36.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.859+0000 7f7735ffb700 0 client.15813 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:13:36.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.860+0000 7f77681cab80 1 -- 172.21.15.64:0/3176780595 shutdown_connections 2023-09-13T03:13:36.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.860+0000 7f77681cab80 1 -- 172.21.15.64:0/3176780595 wait complete. 2023-09-13T03:13:36.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.860+0000 7f77681cab80 20 client.15813 trim_cache size 0 max 16384 2023-09-13T03:13:36.870 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2023-09-13T03:13:36.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.861+0000 7f77547ff700 1 -- 172.21.15.64:0/1638697774 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f77380a08c0 msgr2=0x7f77380a2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:36.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.861+0000 7f77547ff700 1 --2- 172.21.15.64:0/1638697774 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f77380a08c0 0x7f77380a2d80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f774c101480 tx=0x7f7744009040 comp rx=0 tx=0).stop 2023-09-13T03:13:36.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.861+0000 7f77547ff700 1 -- 172.21.15.64:0/1638697774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f774c102040 msgr2=0x7f774c0ff8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:36.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.861+0000 7f77547ff700 1 --2- 172.21.15.64:0/1638697774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f774c102040 0x7f774c0ff8b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f773c005910 tx=0x7f773c0057d0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.861+0000 7f77547ff700 1 -- 172.21.15.64:0/1638697774 shutdown_connections 2023-09-13T03:13:36.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.861+0000 7f77547ff700 1 --2- 172.21.15.64:0/1638697774 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f77380a08c0 0x7f77380a2d80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.861+0000 7f77547ff700 1 --2- 172.21.15.64:0/1638697774 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f774c10ab40 0x7f774c1a48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.861+0000 7f77547ff700 1 --2- 172.21.15.64:0/1638697774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f774c101720 0x7f774c0ff370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.861+0000 7f77547ff700 1 --2- 172.21.15.64:0/1638697774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f774c102040 0x7f774c0ff8b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:36.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.861+0000 7f77547ff700 1 -- 172.21.15.64:0/1638697774 >> 172.21.15.64:0/1638697774 conn(0x7f774c0faf60 msgr2=0x7f774c10cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:36.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.861+0000 7f77547ff700 1 -- 172.21.15.64:0/1638697774 shutdown_connections 2023-09-13T03:13:36.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:36.861+0000 7f77547ff700 1 -- 172.21.15.64:0/1638697774 wait complete. 2023-09-13T03:13:36.881 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:36 smithi064 ceph-mon[97675]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2023-09-13T03:13:36.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:36 smithi170 ceph-mon[134328]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2023-09-13T03:13:36.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:36 smithi142 ceph-mon[134082]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2023-09-13T03:13:37.883 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-09-13T03:13:37.884 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-09-13T03:13:38.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.118+0000 7f7d5d9ff700 1 -- 172.21.15.64:0/2631319398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d580ff300 msgr2=0x7f7d580ff6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:38.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.118+0000 7f7d5d9ff700 1 --2- 172.21.15.64:0/2631319398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d580ff300 0x7f7d580ff6e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7d480098f0 tx=0x7f7d48009c00 comp rx=0 tx=0).stop 2023-09-13T03:13:38.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.118+0000 7f7d5d9ff700 1 -- 172.21.15.64:0/2631319398 shutdown_connections 2023-09-13T03:13:38.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.118+0000 7f7d5d9ff700 1 --2- 172.21.15.64:0/2631319398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d5810ceb0 0x7f7d5810f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.118+0000 7f7d5d9ff700 1 --2- 172.21.15.64:0/2631319398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d580ff300 0x7f7d580ff6e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.118+0000 7f7d5d9ff700 1 --2- 172.21.15.64:0/2631319398 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d580ffcb0 0x7f7d5810c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.118+0000 7f7d5d9ff700 1 -- 172.21.15.64:0/2631319398 >> 172.21.15.64:0/2631319398 conn(0x7f7d580faf60 msgr2=0x7f7d580fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:38.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.118+0000 7f7d5d9ff700 1 -- 172.21.15.64:0/2631319398 shutdown_connections 2023-09-13T03:13:38.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.119+0000 7f7d5d9ff700 1 -- 172.21.15.64:0/2631319398 wait complete. 2023-09-13T03:13:38.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.120+0000 7f7d5d9ff700 1 Processor -- start 2023-09-13T03:13:38.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.120+0000 7f7d5d9ff700 1 -- start start 2023-09-13T03:13:38.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.121+0000 7f7d5d9ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d580ff300 0x7f7d5819e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.121+0000 7f7d5d9ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d580ffcb0 0x7f7d5819ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.121+0000 7f7d5d9ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d5810ceb0 0x7f7d58198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.121+0000 7f7d5d9ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d58198e00 con 0x7f7d5810ceb0 2023-09-13T03:13:38.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.121+0000 7f7d5d9ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d58198f70 con 0x7f7d580ff300 2023-09-13T03:13:38.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.121+0000 7f7d5d9ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d58199130 con 0x7f7d580ffcb0 2023-09-13T03:13:38.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.121+0000 7f7d567fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d580ffcb0 0x7f7d5819ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:38.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.121+0000 7f7d567fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d580ffcb0 0x7f7d5819ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:54438/0 (socket says 172.21.15.64:54438) 2023-09-13T03:13:38.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.121+0000 7f7d567fc700 1 -- 172.21.15.64:0/3127953836 learned_addr learned my addr 172.21.15.64:0/3127953836 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:38.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.121+0000 7f7d56ffd700 1 --2- 172.21.15.64:0/3127953836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d580ff300 0x7f7d5819e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:38.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.121+0000 7f7d577fe700 1 --2- 172.21.15.64:0/3127953836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d5810ceb0 0x7f7d58198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:38.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.122+0000 7f7d567fc700 1 -- 172.21.15.64:0/3127953836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d580ff300 msgr2=0x7f7d5819e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:38.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.122+0000 7f7d567fc700 1 --2- 172.21.15.64:0/3127953836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d580ff300 0x7f7d5819e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.122+0000 7f7d567fc700 1 -- 172.21.15.64:0/3127953836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d5810ceb0 msgr2=0x7f7d58198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:38.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.122+0000 7f7d567fc700 1 --2- 172.21.15.64:0/3127953836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d5810ceb0 0x7f7d58198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.122+0000 7f7d567fc700 1 -- 172.21.15.64:0/3127953836 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d48009520 con 0x7f7d580ffcb0 2023-09-13T03:13:38.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.122+0000 7f7d56ffd700 1 --2- 172.21.15.64:0/3127953836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d580ff300 0x7f7d5819e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T03:13:38.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.122+0000 7f7d577fe700 1 --2- 172.21.15.64:0/3127953836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d5810ceb0 0x7f7d58198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T03:13:38.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.122+0000 7f7d567fc700 1 --2- 172.21.15.64:0/3127953836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d580ffcb0 0x7f7d5819ec60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c00e9c0 tx=0x7f7d4c00ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:38.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.122+0000 7f7d5c9fd700 1 -- 172.21.15.64:0/3127953836 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c00cbd0 con 0x7f7d580ffcb0 2023-09-13T03:13:38.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.122+0000 7f7d5c9fd700 1 -- 172.21.15.64:0/3127953836 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d4c00cd30 con 0x7f7d580ffcb0 2023-09-13T03:13:38.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.122+0000 7f7d5c9fd700 1 -- 172.21.15.64:0/3127953836 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c005580 con 0x7f7d580ffcb0 2023-09-13T03:13:38.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.122+0000 7f7d5d9ff700 1 -- 172.21.15.64:0/3127953836 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d58199410 con 0x7f7d580ffcb0 2023-09-13T03:13:38.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.123+0000 7f7d5d9ff700 1 -- 172.21.15.64:0/3127953836 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d581a5710 con 0x7f7d580ffcb0 2023-09-13T03:13:38.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.127+0000 7f7d71249b80 1 Processor -- start 2023-09-13T03:13:38.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.127+0000 7f7d71249b80 1 -- start start 2023-09-13T03:13:38.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.128+0000 7f7d71249b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585c71d5790 0x5585c71d5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.128+0000 7f7d71249b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c71d60b0 0x5585c71df980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.128+0000 7f7d71249b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5585c71dfec0 0x5585c71e22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.128+0000 7f7d71249b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585c712e7b0 con 0x5585c71dfec0 2023-09-13T03:13:38.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.128+0000 7f7d71249b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585c712c0d0 con 0x5585c71d60b0 2023-09-13T03:13:38.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.128+0000 7f7d71249b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585c71279c0 con 0x5585c71d5790 2023-09-13T03:13:38.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.128+0000 7f7d5c9fd700 1 -- 172.21.15.64:0/3127953836 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 24) v1 ==== 135435+0+0 (secure 0 0 0) 0x7f7d4c004750 con 0x7f7d580ffcb0 2023-09-13T03:13:38.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.129+0000 7f7d567fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5585c71dfec0 0x5585c71e22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:38.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.129+0000 7f7d567fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5585c71dfec0 0x5585c71e22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59316/0 (socket says 172.21.15.64:59316) 2023-09-13T03:13:38.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.129+0000 7f7d567fc700 1 -- 172.21.15.64:0/1862565705 learned_addr learned my addr 172.21.15.64:0/1862565705 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:38.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.129+0000 7f7d56ffd700 1 --2- 172.21.15.64:0/1862565705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c71d60b0 0x5585c71df980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:38.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.129+0000 7f7d577fe700 1 --2- 172.21.15.64:0/1862565705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585c71d5790 0x5585c71d5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:38.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.130+0000 7f7d577fe700 1 -- 172.21.15.64:0/1862565705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c71d60b0 msgr2=0x5585c71df980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:38.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.130+0000 7f7d577fe700 1 --2- 172.21.15.64:0/1862565705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c71d60b0 0x5585c71df980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.130+0000 7f7d577fe700 1 -- 172.21.15.64:0/1862565705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5585c71dfec0 msgr2=0x5585c71e22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:38.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.130+0000 7f7d577fe700 1 --2- 172.21.15.64:0/1862565705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5585c71dfec0 0x5585c71e22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.130+0000 7f7d577fe700 1 -- 172.21.15.64:0/1862565705 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585c6f53e60 con 0x5585c71d5790 2023-09-13T03:13:38.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.130+0000 7f7d567fc700 1 --2- 172.21.15.64:0/1862565705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5585c71dfec0 0x5585c71e22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:13:38.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.130+0000 7f7d56ffd700 1 --2- 172.21.15.64:0/1862565705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c71d60b0 0x5585c71df980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:13:38.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.130+0000 7f7d577fe700 1 --2- 172.21.15.64:0/1862565705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585c71d5790 0x5585c71d5b70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7d58068600 tx=0x7f7d50009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=129e0dd842c61ed5 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:38.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.131+0000 7f7d3dffb700 1 -- 172.21.15.64:0/1862565705 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d500153f0 con 0x5585c71d5790 2023-09-13T03:13:38.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.131+0000 7f7d3dffb700 1 -- 172.21.15.64:0/1862565705 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d50011040 con 0x5585c71d5790 2023-09-13T03:13:38.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.131+0000 7f7d3dffb700 1 -- 172.21.15.64:0/1862565705 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d500148f0 con 0x5585c71d5790 2023-09-13T03:13:38.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.131+0000 7f7d5c9fd700 1 --2- 172.21.15.64:0/3127953836 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f7d400a0940 0x7f7d400a2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.131+0000 7f7d5c9fd700 1 -- 172.21.15.64:0/3127953836 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f7d4c014070 con 0x7f7d580ffcb0 2023-09-13T03:13:38.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.132+0000 7f7d56ffd700 1 --2- 172.21.15.64:0/3127953836 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f7d400a0940 0x7f7d400a2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:38.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.132+0000 7f7d71249b80 1 -- 172.21.15.64:0/1862565705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585c71d5790 msgr2=0x5585c71d5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:38.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.132+0000 7f7d71249b80 1 --2- 172.21.15.64:0/1862565705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585c71d5790 0x5585c71d5b70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7d58068600 tx=0x7f7d50009920 comp rx=0 tx=0).stop 2023-09-13T03:13:38.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.132+0000 7f7d71249b80 1 -- 172.21.15.64:0/1862565705 shutdown_connections 2023-09-13T03:13:38.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.132+0000 7f7d71249b80 1 --2- 172.21.15.64:0/1862565705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5585c71dfec0 0x5585c71e22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.132+0000 7f7d71249b80 1 --2- 172.21.15.64:0/1862565705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585c71d5790 0x5585c71d5b70 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.132+0000 7f7d71249b80 1 --2- 172.21.15.64:0/1862565705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c71d60b0 0x5585c71df980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.132+0000 7f7d71249b80 1 -- 172.21.15.64:0/1862565705 >> 172.21.15.64:0/1862565705 conn(0x5585c714a110 msgr2=0x5585c71d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:38.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.132+0000 7f7d71249b80 1 -- 172.21.15.64:0/1862565705 shutdown_connections 2023-09-13T03:13:38.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.132+0000 7f7d71249b80 1 -- 172.21.15.64:0/1862565705 wait complete. 2023-09-13T03:13:38.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.133+0000 7f7d71249b80 1 -- start start 2023-09-13T03:13:38.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.133+0000 7f7d56ffd700 1 --2- 172.21.15.64:0/3127953836 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f7d400a0940 0x7f7d400a2e00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7d48000c00 tx=0x7f7d48019040 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:38.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.133+0000 7f7d71249b80 1 Processor -- start 2023-09-13T03:13:38.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.134+0000 7f7d71249b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5585c71d5790 0x5585c71df890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.134+0000 7f7d71249b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585c71d60b0 0x5585c71dcb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.134+0000 7f7d71249b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c71dfec0 0x5585c71dd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.134+0000 7f7d71249b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585c70c5bf0 con 0x5585c71d5790 2023-09-13T03:13:38.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.134+0000 7f7d71249b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585c70c58f0 con 0x5585c71dfec0 2023-09-13T03:13:38.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.134+0000 7f7d71249b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585c6fa99a0 con 0x5585c71d60b0 2023-09-13T03:13:38.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.134+0000 7f7d71249b80 10 client.0 _ll_register_callbacks cb 0x5585c700fb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:13:38.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.135+0000 7f7d56ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5585c71d5790 0x5585c71df890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:38.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.135+0000 7f7d567fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585c71d60b0 0x5585c71dcb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:38.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.135+0000 7f7d567fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585c71d60b0 0x5585c71dcb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:54464/0 (socket says 172.21.15.64:54464) 2023-09-13T03:13:38.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.135+0000 7f7d567fc700 1 -- 172.21.15.64:0/171564592 learned_addr learned my addr 172.21.15.64:0/171564592 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:38.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.135+0000 7f7d577fe700 1 --2- 172.21.15.64:0/171564592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c71dfec0 0x5585c71dd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:38.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.135+0000 7f7d567fc700 1 -- 172.21.15.64:0/171564592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c71dfec0 msgr2=0x5585c71dd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:38.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.135+0000 7f7d567fc700 1 --2- 172.21.15.64:0/171564592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c71dfec0 0x5585c71dd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.135+0000 7f7d567fc700 1 -- 172.21.15.64:0/171564592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5585c71d5790 msgr2=0x5585c71df890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:38.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.135+0000 7f7d567fc700 1 --2- 172.21.15.64:0/171564592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5585c71d5790 0x5585c71df890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.135+0000 7f7d567fc700 1 -- 172.21.15.64:0/171564592 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d5810fed0 con 0x5585c71d60b0 2023-09-13T03:13:38.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.136+0000 7f7d567fc700 1 --2- 172.21.15.64:0/171564592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585c71d60b0 0x5585c71dcb10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c0a1220 tx=0x7f7d4c009740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95b47a9ca0ebebd7 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:38.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.136+0000 7f7d36ffd700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:13:38.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.136+0000 7f7d36ffd700 1 -- 172.21.15.64:0/171564592 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c09d320 con 0x5585c71d60b0 2023-09-13T03:13:38.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.136+0000 7f7d71249b80 1 -- 172.21.15.64:0/171564592 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5585c707f570 con 0x5585c71d60b0 2023-09-13T03:13:38.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.136+0000 7f7d36ffd700 1 -- 172.21.15.64:0/171564592 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f7d4c099de0 con 0x5585c71d60b0 2023-09-13T03:13:38.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.136+0000 7f7d36ffd700 1 -- 172.21.15.64:0/171564592 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d4c099220 con 0x5585c71d60b0 2023-09-13T03:13:38.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.136+0000 7f7d71249b80 10 client.25855 fetch_fsmap learned FSMap version 26 2023-09-13T03:13:38.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.136+0000 7f7d71249b80 1 -- 172.21.15.64:0/171564592 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f7d581a5710 con 0x5585c71d60b0 2023-09-13T03:13:38.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.137+0000 7f7d36ffd700 1 -- 172.21.15.64:0/171564592 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5585c707f570 con 0x5585c71d60b0 2023-09-13T03:13:38.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.137+0000 7f7d36ffd700 1 -- 172.21.15.64:0/171564592 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7d4c0cc070 con 0x5585c71d60b0 2023-09-13T03:13:38.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.137+0000 7f7d71249b80 10 client.25855 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:13:38.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.137+0000 7f7d71249b80 10 client.25855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T03:13:38.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.137+0000 7f7d71249b80 20 client.25855 populate_metadata read hostname 'smithi064' 2023-09-13T03:13:38.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.137+0000 7f7d71249b80 1 --2- 172.21.15.64:0/171564592 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5585c71ef920 0x5585c71f1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.137+0000 7f7d71249b80 4 client.25855 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:13:38.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.137+0000 7f7d71249b80 1 -- 172.21.15.64:0/171564592 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5585c713c920 con 0x5585c71ef920 2023-09-13T03:13:38.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.138+0000 7f7d56ffd700 1 --2- 172.21.15.64:0/171564592 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5585c71ef920 0x5585c71f1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:38.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.138+0000 7f7d56ffd700 1 --2- 172.21.15.64:0/171564592 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5585c71ef920 0x5585c71f1d40 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20add343e55b40d4 server_cookie=a22e1383f550a20e in_seq=0 out_seq=0 2023-09-13T03:13:38.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.138+0000 7f7d36ffd700 10 client.25855 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:13:38.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.139+0000 7f7d36ffd700 1 -- 172.21.15.64:0/171564592 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5585c713c920 con 0x5585c71ef920 2023-09-13T03:13:38.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.139+0000 7f7d36ffd700 10 client.25855 handle_command_reply: tid=0 2023-09-13T03:13:38.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.139+0000 7f7d71249b80 1 client.25855 shutdown 2023-09-13T03:13:38.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.139+0000 7f7d71249b80 1 -- 172.21.15.64:0/171564592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585c71d60b0 msgr2=0x5585c71dcb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:38.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.139+0000 7f7d71249b80 1 --2- 172.21.15.64:0/171564592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585c71d60b0 0x5585c71dcb10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c0a1220 tx=0x7f7d4c009740 comp rx=0 tx=0).stop 2023-09-13T03:13:38.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.139+0000 7f7d71249b80 1 -- 172.21.15.64:0/171564592 shutdown_connections 2023-09-13T03:13:38.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.139+0000 7f7d71249b80 1 --2- 172.21.15.64:0/171564592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c71dfec0 0x5585c71dd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.139+0000 7f7d71249b80 1 --2- 172.21.15.64:0/171564592 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5585c71ef920 0x5585c71f1d40 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.139+0000 7f7d71249b80 1 --2- 172.21.15.64:0/171564592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5585c71d5790 0x5585c71df890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.139+0000 7f7d71249b80 1 --2- 172.21.15.64:0/171564592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585c71d60b0 0x5585c71dcb10 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.139+0000 7f7d71249b80 1 -- 172.21.15.64:0/171564592 >> 172.21.15.64:0/171564592 conn(0x5585c714a110 msgr2=0x5585c71d0860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:38.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.139+0000 7f7d36ffd700 0 client.25855 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:13:38.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.140+0000 7f7d71249b80 1 -- 172.21.15.64:0/171564592 shutdown_connections 2023-09-13T03:13:38.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.140+0000 7f7d71249b80 1 -- 172.21.15.64:0/171564592 wait complete. 2023-09-13T03:13:38.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.140+0000 7f7d71249b80 20 client.25855 trim_cache size 0 max 16384 2023-09-13T03:13:38.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.175+0000 7f7d71249b80 1 Processor -- start 2023-09-13T03:13:38.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.175+0000 7f7d71249b80 1 -- start start 2023-09-13T03:13:38.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.175+0000 7f7d71249b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5585c71d5790 0x5585c71e6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.175+0000 7f7d71249b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c71d60b0 0x5585c71dc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.175+0000 7f7d71249b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585c71dfec0 0x5585c71dc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.175+0000 7f7d71249b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585c71ddab0 con 0x5585c71d5790 2023-09-13T03:13:38.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.175+0000 7f7d71249b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585c71d1830 con 0x5585c71d60b0 2023-09-13T03:13:38.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.175+0000 7f7d71249b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585c71de140 con 0x5585c71dfec0 2023-09-13T03:13:38.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.176+0000 7f7d567fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c71d60b0 0x5585c71dc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:38.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.176+0000 7f7d56ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5585c71d5790 0x5585c71e6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:38.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.176+0000 7f7d567fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c71d60b0 0x5585c71dc2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:57396/0 (socket says 172.21.15.64:57396) 2023-09-13T03:13:38.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.176+0000 7f7d567fc700 1 -- 172.21.15.64:0/2544378821 learned_addr learned my addr 172.21.15.64:0/2544378821 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:38.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.176+0000 7f7d577fe700 1 --2- 172.21.15.64:0/2544378821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585c71dfec0 0x5585c71dc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:38.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.176+0000 7f7d567fc700 1 -- 172.21.15.64:0/2544378821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585c71dfec0 msgr2=0x5585c71dc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:38.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.176+0000 7f7d567fc700 1 --2- 172.21.15.64:0/2544378821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585c71dfec0 0x5585c71dc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.176+0000 7f7d567fc700 1 -- 172.21.15.64:0/2544378821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5585c71d5790 msgr2=0x5585c71e6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:38.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.176+0000 7f7d567fc700 1 --2- 172.21.15.64:0/2544378821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5585c71d5790 0x5585c71e6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.176+0000 7f7d567fc700 1 -- 172.21.15.64:0/2544378821 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585c71f6990 con 0x5585c71d60b0 2023-09-13T03:13:38.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.177+0000 7f7d567fc700 1 --2- 172.21.15.64:0/2544378821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c71d60b0 0x5585c71dc2d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c099670 tx=0x7f7d4c099570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65f38c2719202918 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:38.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.177+0000 7f7d3e7fc700 1 -- 172.21.15.64:0/2544378821 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c099220 con 0x5585c71d60b0 2023-09-13T03:13:38.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.177+0000 7f7d3e7fc700 1 -- 172.21.15.64:0/2544378821 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d4c099040 con 0x5585c71d60b0 2023-09-13T03:13:38.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.177+0000 7f7d3e7fc700 1 -- 172.21.15.64:0/2544378821 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c09d510 con 0x5585c71d60b0 2023-09-13T03:13:38.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.178+0000 7f7d71249b80 1 -- 172.21.15.64:0/2544378821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c71d60b0 msgr2=0x5585c71dc2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:38.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.178+0000 7f7d71249b80 1 --2- 172.21.15.64:0/2544378821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c71d60b0 0x5585c71dc2d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c099670 tx=0x7f7d4c099570 comp rx=0 tx=0).stop 2023-09-13T03:13:38.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.178+0000 7f7d71249b80 1 -- 172.21.15.64:0/2544378821 shutdown_connections 2023-09-13T03:13:38.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.178+0000 7f7d71249b80 1 --2- 172.21.15.64:0/2544378821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585c71dfec0 0x5585c71dc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.178+0000 7f7d71249b80 1 --2- 172.21.15.64:0/2544378821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5585c71d5790 0x5585c71e6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.178+0000 7f7d71249b80 1 --2- 172.21.15.64:0/2544378821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c71d60b0 0x5585c71dc2d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.178+0000 7f7d71249b80 1 -- 172.21.15.64:0/2544378821 >> 172.21.15.64:0/2544378821 conn(0x5585c714a110 msgr2=0x5585c71d5010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:38.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.178+0000 7f7d71249b80 1 -- 172.21.15.64:0/2544378821 shutdown_connections 2023-09-13T03:13:38.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.178+0000 7f7d71249b80 1 -- 172.21.15.64:0/2544378821 wait complete. 2023-09-13T03:13:38.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.179+0000 7f7d71249b80 1 -- start start 2023-09-13T03:13:38.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.179+0000 7f7d71249b80 1 Processor -- start 2023-09-13T03:13:38.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.180+0000 7f7d71249b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585c71d5790 0x5585c7239710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.180+0000 7f7d71249b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5585c71d60b0 0x5585c7239c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.180+0000 7f7d71249b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c71dfec0 0x5585c723a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.180+0000 7f7d71249b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585c7244910 con 0x5585c71d60b0 2023-09-13T03:13:38.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.180+0000 7f7d71249b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585c7244a80 con 0x5585c71dfec0 2023-09-13T03:13:38.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.180+0000 7f7d71249b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585c7244c40 con 0x5585c71d5790 2023-09-13T03:13:38.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.180+0000 7f7d71249b80 10 client.0 _ll_register_callbacks cb 0x5585c700fb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:13:38.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.180+0000 7f7d577fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c71dfec0 0x5585c723a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:38.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.180+0000 7f7d56ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585c71d5790 0x5585c7239710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:38.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.180+0000 7f7d577fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c71dfec0 0x5585c723a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:57404/0 (socket says 172.21.15.64:57404) 2023-09-13T03:13:38.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.180+0000 7f7d577fe700 1 -- 172.21.15.64:0/3208005451 learned_addr learned my addr 172.21.15.64:0/3208005451 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:38.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.181+0000 7f7d567fc700 1 --2- 172.21.15.64:0/3208005451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5585c71d60b0 0x5585c7239c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:38.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.181+0000 7f7d577fe700 1 -- 172.21.15.64:0/3208005451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585c71d5790 msgr2=0x5585c7239710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:38.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.181+0000 7f7d577fe700 1 --2- 172.21.15.64:0/3208005451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585c71d5790 0x5585c7239710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.181+0000 7f7d577fe700 1 -- 172.21.15.64:0/3208005451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5585c71d60b0 msgr2=0x5585c7239c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:38.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.181+0000 7f7d577fe700 1 --2- 172.21.15.64:0/3208005451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5585c71d60b0 0x5585c7239c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.181+0000 7f7d577fe700 1 -- 172.21.15.64:0/3208005451 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d581a5710 con 0x5585c71dfec0 2023-09-13T03:13:38.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.181+0000 7f7d567fc700 1 --2- 172.21.15.64:0/3208005451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5585c71d60b0 0x5585c7239c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T03:13:38.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.181+0000 7f7d577fe700 1 --2- 172.21.15.64:0/3208005451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c71dfec0 0x5585c723a190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7d500093e0 tx=0x7f7d50022d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0392fc713fd5610 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:38.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.181+0000 7f7d36ffd700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:13:38.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.181+0000 7f7d36ffd700 1 -- 172.21.15.64:0/3208005451 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d50022ea0 con 0x5585c71dfec0 2023-09-13T03:13:38.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.182+0000 7f7d71249b80 1 -- 172.21.15.64:0/3208005451 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d50011040 con 0x5585c71dfec0 2023-09-13T03:13:38.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.182+0000 7f7d36ffd700 1 -- 172.21.15.64:0/3208005451 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f7d50014750 con 0x5585c71dfec0 2023-09-13T03:13:38.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.182+0000 7f7d36ffd700 1 -- 172.21.15.64:0/3208005451 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d50027070 con 0x5585c71dfec0 2023-09-13T03:13:38.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.182+0000 7f7d71249b80 10 client.25859 fetch_fsmap learned FSMap version 26 2023-09-13T03:13:38.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.182+0000 7f7d71249b80 1 -- 172.21.15.64:0/3208005451 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f7d500090f0 con 0x5585c71dfec0 2023-09-13T03:13:38.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.182+0000 7f7d36ffd700 1 -- 172.21.15.64:0/3208005451 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d50038020 con 0x5585c71dfec0 2023-09-13T03:13:38.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.182+0000 7f7d36ffd700 1 -- 172.21.15.64:0/3208005451 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7d50005220 con 0x5585c71dfec0 2023-09-13T03:13:38.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.183+0000 7f7d71249b80 10 client.25859 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:13:38.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.183+0000 7f7d71249b80 10 client.25859 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T03:13:38.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.183+0000 7f7d71249b80 20 client.25859 populate_metadata read hostname 'smithi064' 2023-09-13T03:13:38.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.183+0000 7f7d71249b80 1 --2- 172.21.15.64:0/3208005451 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5585c7246360 0x5585c7248780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.183+0000 7f7d71249b80 4 client.25859 mds_command: new command op to 24686 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-09-13T03:13:38.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.183+0000 7f7d71249b80 1 -- 172.21.15.64:0/3208005451 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5585c71209a0 con 0x5585c7246360 2023-09-13T03:13:38.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.183+0000 7f7d56ffd700 1 --2- 172.21.15.64:0/3208005451 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5585c7246360 0x5585c7248780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:38.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.184+0000 7f7d56ffd700 1 --2- 172.21.15.64:0/3208005451 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5585c7246360 0x5585c7248780 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=467828b9db940100 server_cookie=5cb2bfd2c3d5d1ad in_seq=0 out_seq=0 2023-09-13T03:13:38.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.184+0000 7f7d36ffd700 10 client.25859 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:13:38.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.185+0000 7f7d36ffd700 1 -- 172.21.15.64:0/3208005451 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5585c71209a0 con 0x5585c7246360 2023-09-13T03:13:38.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.185+0000 7f7d36ffd700 10 client.25859 handle_command_reply: tid=0 2023-09-13T03:13:38.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.185+0000 7f7d71249b80 1 client.25859 shutdown 2023-09-13T03:13:38.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.185+0000 7f7d71249b80 1 -- 172.21.15.64:0/3208005451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c71dfec0 msgr2=0x5585c723a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:38.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.185+0000 7f7d71249b80 1 --2- 172.21.15.64:0/3208005451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c71dfec0 0x5585c723a190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7d500093e0 tx=0x7f7d50022d00 comp rx=0 tx=0).stop 2023-09-13T03:13:38.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.185+0000 7f7d71249b80 1 -- 172.21.15.64:0/3208005451 shutdown_connections 2023-09-13T03:13:38.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.185+0000 7f7d71249b80 1 --2- 172.21.15.64:0/3208005451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c71dfec0 0x5585c723a190 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.185+0000 7f7d71249b80 1 --2- 172.21.15.64:0/3208005451 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5585c7246360 0x5585c7248780 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.185+0000 7f7d71249b80 1 --2- 172.21.15.64:0/3208005451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585c71d5790 0x5585c7239710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.185+0000 7f7d71249b80 1 --2- 172.21.15.64:0/3208005451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5585c71d60b0 0x5585c7239c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.185+0000 7f7d36ffd700 0 client.25859 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:13:38.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.185+0000 7f7d71249b80 1 -- 172.21.15.64:0/3208005451 >> 172.21.15.64:0/3208005451 conn(0x5585c714a110 msgr2=0x5585c71cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:38.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.186+0000 7f7d71249b80 1 -- 172.21.15.64:0/3208005451 shutdown_connections 2023-09-13T03:13:38.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.186+0000 7f7d71249b80 1 -- 172.21.15.64:0/3208005451 wait complete. 2023-09-13T03:13:38.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.186+0000 7f7d71249b80 20 client.25859 trim_cache size 0 max 16384 2023-09-13T03:13:38.196 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-13T03:13:38.196 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2023-09-13T03:13:38.196 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "c06fb957-4a86-4cfe-bac9-7fcc14b05a2b", 2023-09-13T03:13:38.196 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2023-09-13T03:13:38.196 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-13T03:13:38.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.187+0000 7f7d5d9ff700 1 -- 172.21.15.64:0/3127953836 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f7d400a0940 msgr2=0x7f7d400a2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:38.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.187+0000 7f7d5d9ff700 1 --2- 172.21.15.64:0/3127953836 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f7d400a0940 0x7f7d400a2e00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7d48000c00 tx=0x7f7d48019040 comp rx=0 tx=0).stop 2023-09-13T03:13:38.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.187+0000 7f7d5d9ff700 1 -- 172.21.15.64:0/3127953836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d580ffcb0 msgr2=0x7f7d5819ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:38.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.187+0000 7f7d5d9ff700 1 --2- 172.21.15.64:0/3127953836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d580ffcb0 0x7f7d5819ec60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c00e9c0 tx=0x7f7d4c00ed80 comp rx=0 tx=0).stop 2023-09-13T03:13:38.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.187+0000 7f7d5d9ff700 1 -- 172.21.15.64:0/3127953836 shutdown_connections 2023-09-13T03:13:38.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.187+0000 7f7d5d9ff700 1 --2- 172.21.15.64:0/3127953836 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f7d400a0940 0x7f7d400a2e00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.187+0000 7f7d5d9ff700 1 --2- 172.21.15.64:0/3127953836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d580ff300 0x7f7d5819e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.187+0000 7f7d5d9ff700 1 --2- 172.21.15.64:0/3127953836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d5810ceb0 0x7f7d58198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.187+0000 7f7d5d9ff700 1 --2- 172.21.15.64:0/3127953836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d580ffcb0 0x7f7d5819ec60 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.187+0000 7f7d5d9ff700 1 -- 172.21.15.64:0/3127953836 >> 172.21.15.64:0/3127953836 conn(0x7f7d580faf60 msgr2=0x7f7d580fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:38.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.188+0000 7f7d5d9ff700 1 -- 172.21.15.64:0/3127953836 shutdown_connections 2023-09-13T03:13:38.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.188+0000 7f7d5d9ff700 1 -- 172.21.15.64:0/3127953836 wait complete. 2023-09-13T03:13:38.207 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-13T03:13:38.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.434+0000 7f2e481ff700 1 -- 172.21.15.64:0/527799104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e40101720 msgr2=0x7f2e40101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:38.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.434+0000 7f2e481ff700 1 --2- 172.21.15.64:0/527799104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e40101720 0x7f2e40101b00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c0098f0 tx=0x7f2e3c009c00 comp rx=0 tx=0).stop 2023-09-13T03:13:38.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.434+0000 7f2e481ff700 1 -- 172.21.15.64:0/527799104 shutdown_connections 2023-09-13T03:13:38.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.434+0000 7f2e481ff700 1 --2- 172.21.15.64:0/527799104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e40101720 0x7f2e40101b00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.434+0000 7f2e481ff700 1 --2- 172.21.15.64:0/527799104 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e4010ab40 0x7f2e4010d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.434+0000 7f2e481ff700 1 --2- 172.21.15.64:0/527799104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e40102040 0x7f2e4010a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.434+0000 7f2e481ff700 1 -- 172.21.15.64:0/527799104 >> 172.21.15.64:0/527799104 conn(0x7f2e400faf60 msgr2=0x7f2e400fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:38.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.434+0000 7f2e481ff700 1 -- 172.21.15.64:0/527799104 shutdown_connections 2023-09-13T03:13:38.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.434+0000 7f2e481ff700 1 -- 172.21.15.64:0/527799104 wait complete. 2023-09-13T03:13:38.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.435+0000 7f2e481ff700 1 Processor -- start 2023-09-13T03:13:38.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.435+0000 7f2e481ff700 1 -- start start 2023-09-13T03:13:38.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.436+0000 7f2e481ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e40101720 0x7f2e4019e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.436+0000 7f2e481ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e40102040 0x7f2e4019ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.436+0000 7f2e481ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e4010ab40 0x7f2e40198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.436+0000 7f2e481ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e40198db0 con 0x7f2e4010ab40 2023-09-13T03:13:38.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.436+0000 7f2e481ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e40198f20 con 0x7f2e40101720 2023-09-13T03:13:38.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.436+0000 7f2e481ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e401990e0 con 0x7f2e40102040 2023-09-13T03:13:38.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.436+0000 7f2e4679c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e4010ab40 0x7f2e40198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:38.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.437+0000 7f2e4579a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e40102040 0x7f2e4019ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:38.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.437+0000 7f2e4579a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e40102040 0x7f2e4019ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:54492/0 (socket says 172.21.15.64:54492) 2023-09-13T03:13:38.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.437+0000 7f2e4579a700 1 -- 172.21.15.64:0/1070602034 learned_addr learned my addr 172.21.15.64:0/1070602034 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:38.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.437+0000 7f2e45f9b700 1 --2- 172.21.15.64:0/1070602034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e40101720 0x7f2e4019e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:38.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.437+0000 7f2e4579a700 1 -- 172.21.15.64:0/1070602034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e40101720 msgr2=0x7f2e4019e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:38.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.437+0000 7f2e4579a700 1 --2- 172.21.15.64:0/1070602034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e40101720 0x7f2e4019e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.437+0000 7f2e4579a700 1 -- 172.21.15.64:0/1070602034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e4010ab40 msgr2=0x7f2e40198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:38.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.437+0000 7f2e4579a700 1 --2- 172.21.15.64:0/1070602034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e4010ab40 0x7f2e40198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.437+0000 7f2e4579a700 1 -- 172.21.15.64:0/1070602034 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e300095f0 con 0x7f2e40102040 2023-09-13T03:13:38.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.437+0000 7f2e4579a700 1 --2- 172.21.15.64:0/1070602034 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e40102040 0x7f2e4019ec10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2e30009fd0 tx=0x7f2e3000ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:38.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.438+0000 7f2e36ffd700 1 -- 172.21.15.64:0/1070602034 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e30009960 con 0x7f2e40102040 2023-09-13T03:13:38.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.438+0000 7f2e481ff700 1 -- 172.21.15.64:0/1070602034 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e3c009520 con 0x7f2e40102040 2023-09-13T03:13:38.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.438+0000 7f2e36ffd700 1 -- 172.21.15.64:0/1070602034 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e300044e0 con 0x7f2e40102040 2023-09-13T03:13:38.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.438+0000 7f2e36ffd700 1 -- 172.21.15.64:0/1070602034 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e30003e90 con 0x7f2e40102040 2023-09-13T03:13:38.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.438+0000 7f2e481ff700 1 -- 172.21.15.64:0/1070602034 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e401995b0 con 0x7f2e40102040 2023-09-13T03:13:38.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.442+0000 7f2e36ffd700 1 -- 172.21.15.64:0/1070602034 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 24) v1 ==== 135435+0+0 (secure 0 0 0) 0x7f2e30004000 con 0x7f2e40102040 2023-09-13T03:13:38.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.444+0000 7f2e36ffd700 1 --2- 172.21.15.64:0/1070602034 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f2e2c0a08c0 0x7f2e2c0a2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.444+0000 7f2e36ffd700 1 -- 172.21.15.64:0/1070602034 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f2e30014070 con 0x7f2e40102040 2023-09-13T03:13:38.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.445+0000 7f2e5bbb7b80 1 Processor -- start 2023-09-13T03:13:38.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.445+0000 7f2e45f9b700 1 --2- 172.21.15.64:0/1070602034 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f2e2c0a08c0 0x7f2e2c0a2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:38.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.445+0000 7f2e5bbb7b80 1 -- start start 2023-09-13T03:13:38.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.445+0000 7f2e5bbb7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fdc5532710 0x55fdc5532af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.445+0000 7f2e5bbb7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdc5533030 0x55fdc5545110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.445+0000 7f2e5bbb7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdc5545650 0x55fdc5547a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.445+0000 7f2e5bbb7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdc548b730 con 0x55fdc5545650 2023-09-13T03:13:38.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.446+0000 7f2e5bbb7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdc5489050 con 0x55fdc5533030 2023-09-13T03:13:38.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.446+0000 7f2e5bbb7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdc5484940 con 0x55fdc5532710 2023-09-13T03:13:38.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.446+0000 7f2e45f9b700 1 --2- 172.21.15.64:0/1070602034 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f2e2c0a08c0 0x7f2e2c0a2d80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c005e90 tx=0x7f2e3c019040 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:38.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.446+0000 7f2e4579a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdc5533030 0x55fdc5545110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:38.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.446+0000 7f2e45f9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fdc5532710 0x55fdc5532af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:38.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.446+0000 7f2e4579a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdc5533030 0x55fdc5545110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:57428/0 (socket says 172.21.15.64:57428) 2023-09-13T03:13:38.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.446+0000 7f2e4579a700 1 -- 172.21.15.64:0/2417721510 learned_addr learned my addr 172.21.15.64:0/2417721510 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:38.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.446+0000 7f2e45f9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fdc5532710 0x55fdc5532af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:54496/0 (socket says 172.21.15.64:54496) 2023-09-13T03:13:38.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.446+0000 7f2e4679c700 1 --2- 172.21.15.64:0/2417721510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdc5545650 0x55fdc5547a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:38.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.447+0000 7f2e4579a700 1 -- 172.21.15.64:0/2417721510 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fdc5532710 msgr2=0x55fdc5532af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:38.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.447+0000 7f2e4579a700 1 --2- 172.21.15.64:0/2417721510 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fdc5532710 0x55fdc5532af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.447+0000 7f2e4579a700 1 -- 172.21.15.64:0/2417721510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdc5545650 msgr2=0x55fdc5547a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:38.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.447+0000 7f2e4579a700 1 --2- 172.21.15.64:0/2417721510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdc5545650 0x55fdc5547a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.447+0000 7f2e4579a700 1 -- 172.21.15.64:0/2417721510 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fdc52b0de0 con 0x55fdc5533030 2023-09-13T03:13:38.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.447+0000 7f2e4579a700 1 --2- 172.21.15.64:0/2417721510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdc5533030 0x55fdc5545110 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2e30099680 tx=0x7f2e300992b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=937f57efccd19e37 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:38.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.447+0000 7f2e2bfff700 1 -- 172.21.15.64:0/2417721510 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e30099df0 con 0x55fdc5533030 2023-09-13T03:13:38.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.447+0000 7f2e2bfff700 1 -- 172.21.15.64:0/2417721510 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e300996f0 con 0x55fdc5533030 2023-09-13T03:13:38.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.447+0000 7f2e2bfff700 1 -- 172.21.15.64:0/2417721510 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3001fe90 con 0x55fdc5533030 2023-09-13T03:13:38.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.448+0000 7f2e5bbb7b80 1 -- 172.21.15.64:0/2417721510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdc5533030 msgr2=0x55fdc5545110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:38.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.448+0000 7f2e5bbb7b80 1 --2- 172.21.15.64:0/2417721510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdc5533030 0x55fdc5545110 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2e30099680 tx=0x7f2e300992b0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.449+0000 7f2e5bbb7b80 1 -- 172.21.15.64:0/2417721510 shutdown_connections 2023-09-13T03:13:38.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.449+0000 7f2e5bbb7b80 1 --2- 172.21.15.64:0/2417721510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdc5545650 0x55fdc5547a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.449+0000 7f2e5bbb7b80 1 --2- 172.21.15.64:0/2417721510 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fdc5532710 0x55fdc5532af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.449+0000 7f2e5bbb7b80 1 --2- 172.21.15.64:0/2417721510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdc5533030 0x55fdc5545110 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.449+0000 7f2e5bbb7b80 1 -- 172.21.15.64:0/2417721510 >> 172.21.15.64:0/2417721510 conn(0x55fdc54a7090 msgr2=0x55fdc552dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:38.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.449+0000 7f2e5bbb7b80 1 -- 172.21.15.64:0/2417721510 shutdown_connections 2023-09-13T03:13:38.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.449+0000 7f2e5bbb7b80 1 -- 172.21.15.64:0/2417721510 wait complete. 2023-09-13T03:13:38.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.449+0000 7f2e5bbb7b80 1 -- start start 2023-09-13T03:13:38.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.450+0000 7f2e5bbb7b80 1 Processor -- start 2023-09-13T03:13:38.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.451+0000 7f2e5bbb7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fdc5532710 0x55fdc5542390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.451+0000 7f2e5bbb7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdc5533030 0x55fdc55428d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.451+0000 7f2e5bbb7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdc5545650 0x55fdc5542e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.451+0000 7f2e5bbb7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdc5422b70 con 0x55fdc5533030 2023-09-13T03:13:38.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.451+0000 7f2e5bbb7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdc5422870 con 0x55fdc5545650 2023-09-13T03:13:38.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.451+0000 7f2e5bbb7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdc5306920 con 0x55fdc5532710 2023-09-13T03:13:38.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.451+0000 7f2e5bbb7b80 10 client.0 _ll_register_callbacks cb 0x55fdc536ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:13:38.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.451+0000 7f2e45f9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fdc5532710 0x55fdc5542390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:38.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.451+0000 7f2e45f9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fdc5532710 0x55fdc5542390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:54510/0 (socket says 172.21.15.64:54510) 2023-09-13T03:13:38.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.451+0000 7f2e45f9b700 1 -- 172.21.15.64:0/2592726633 learned_addr learned my addr 172.21.15.64:0/2592726633 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:38.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.451+0000 7f2e4579a700 1 --2- 172.21.15.64:0/2592726633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdc5533030 0x55fdc55428d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:38.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.451+0000 7f2e4679c700 1 --2- 172.21.15.64:0/2592726633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdc5545650 0x55fdc5542e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:38.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.452+0000 7f2e45f9b700 1 -- 172.21.15.64:0/2592726633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdc5545650 msgr2=0x55fdc5542e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:38.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.452+0000 7f2e45f9b700 1 --2- 172.21.15.64:0/2592726633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdc5545650 0x55fdc5542e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.452+0000 7f2e45f9b700 1 -- 172.21.15.64:0/2592726633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdc5533030 msgr2=0x55fdc55428d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:38.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.452+0000 7f2e45f9b700 1 --2- 172.21.15.64:0/2592726633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdc5533030 0x55fdc55428d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.452+0000 7f2e45f9b700 1 -- 172.21.15.64:0/2592726633 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e3c009520 con 0x55fdc5532710 2023-09-13T03:13:38.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.452+0000 7f2e45f9b700 1 --2- 172.21.15.64:0/2592726633 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fdc5532710 0x55fdc5542390 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c000c00 tx=0x7f2e3c01e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd7ce717f05f889e server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:38.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.452+0000 7f2e297fa700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:13:38.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.453+0000 7f2e297fa700 1 -- 172.21.15.64:0/2592726633 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c02c040 con 0x55fdc5532710 2023-09-13T03:13:38.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.453+0000 7f2e5bbb7b80 1 -- 172.21.15.64:0/2592726633 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fdc53dc4f0 con 0x55fdc5532710 2023-09-13T03:13:38.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.453+0000 7f2e297fa700 1 -- 172.21.15.64:0/2592726633 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f2e3c005450 con 0x55fdc5532710 2023-09-13T03:13:38.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.453+0000 7f2e297fa700 1 -- 172.21.15.64:0/2592726633 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e3c005610 con 0x55fdc5532710 2023-09-13T03:13:38.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.453+0000 7f2e5bbb7b80 10 client.25873 fetch_fsmap learned FSMap version 26 2023-09-13T03:13:38.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.453+0000 7f2e5bbb7b80 1 -- 172.21.15.64:0/2592726633 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55fdc5299c60 con 0x55fdc5532710 2023-09-13T03:13:38.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.453+0000 7f2e297fa700 1 -- 172.21.15.64:0/2592726633 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fdc53dc4f0 con 0x55fdc5532710 2023-09-13T03:13:38.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.454+0000 7f2e297fa700 1 -- 172.21.15.64:0/2592726633 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2e3c03a6b0 con 0x55fdc5532710 2023-09-13T03:13:38.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.454+0000 7f2e5bbb7b80 10 client.25873 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:13:38.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.454+0000 7f2e5bbb7b80 10 client.25873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T03:13:38.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.454+0000 7f2e5bbb7b80 20 client.25873 populate_metadata read hostname 'smithi064' 2023-09-13T03:13:38.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.454+0000 7f2e5bbb7b80 1 --2- 172.21.15.64:0/2592726633 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55fdc5555200 0x55fdc5557620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.454+0000 7f2e5bbb7b80 4 client.25873 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:13:38.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.454+0000 7f2e5bbb7b80 1 -- 172.21.15.64:0/2592726633 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fdc54998a0 con 0x55fdc5555200 2023-09-13T03:13:38.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.454+0000 7f2e4579a700 1 --2- 172.21.15.64:0/2592726633 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55fdc5555200 0x55fdc5557620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:38.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.455+0000 7f2e4579a700 1 --2- 172.21.15.64:0/2592726633 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55fdc5555200 0x55fdc5557620 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e62d28b1a4f021e server_cookie=970ea461a85c10d2 in_seq=0 out_seq=0 2023-09-13T03:13:38.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.455+0000 7f2e297fa700 10 client.25873 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:13:38.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.456+0000 7f2e297fa700 1 -- 172.21.15.64:0/2592726633 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fdc54998a0 con 0x55fdc5555200 2023-09-13T03:13:38.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.456+0000 7f2e297fa700 10 client.25873 handle_command_reply: tid=0 2023-09-13T03:13:38.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.456+0000 7f2e5bbb7b80 1 client.25873 shutdown 2023-09-13T03:13:38.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.456+0000 7f2e5bbb7b80 1 -- 172.21.15.64:0/2592726633 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fdc5532710 msgr2=0x55fdc5542390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:38.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.456+0000 7f2e5bbb7b80 1 --2- 172.21.15.64:0/2592726633 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fdc5532710 0x55fdc5542390 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c000c00 tx=0x7f2e3c01e000 comp rx=0 tx=0).stop 2023-09-13T03:13:38.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.456+0000 7f2e5bbb7b80 1 -- 172.21.15.64:0/2592726633 shutdown_connections 2023-09-13T03:13:38.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.456+0000 7f2e5bbb7b80 1 --2- 172.21.15.64:0/2592726633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdc5545650 0x55fdc5542e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.456+0000 7f2e5bbb7b80 1 --2- 172.21.15.64:0/2592726633 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55fdc5555200 0x55fdc5557620 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.456+0000 7f2e5bbb7b80 1 --2- 172.21.15.64:0/2592726633 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fdc5532710 0x55fdc5542390 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.456+0000 7f2e5bbb7b80 1 --2- 172.21.15.64:0/2592726633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdc5533030 0x55fdc55428d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.456+0000 7f2e5bbb7b80 1 -- 172.21.15.64:0/2592726633 >> 172.21.15.64:0/2592726633 conn(0x55fdc54a7090 msgr2=0x55fdc552d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:38.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.456+0000 7f2e297fa700 0 client.25873 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:13:38.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.456+0000 7f2e5bbb7b80 1 -- 172.21.15.64:0/2592726633 shutdown_connections 2023-09-13T03:13:38.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.457+0000 7f2e5bbb7b80 1 -- 172.21.15.64:0/2592726633 wait complete. 2023-09-13T03:13:38.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.457+0000 7f2e5bbb7b80 20 client.25873 trim_cache size 0 max 16384 2023-09-13T03:13:38.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.491+0000 7f2e5bbb7b80 1 Processor -- start 2023-09-13T03:13:38.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.491+0000 7f2e5bbb7b80 1 -- start start 2023-09-13T03:13:38.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.491+0000 7f2e5bbb7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fdc5532710 0x55fdc554c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.491+0000 7f2e5bbb7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdc5533030 0x55fdc5542c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.491+0000 7f2e5bbb7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdc5545650 0x55fdc553f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.491+0000 7f2e5bbb7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdc5547ee0 con 0x55fdc5533030 2023-09-13T03:13:38.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.491+0000 7f2e5bbb7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdc552c9a0 con 0x55fdc5545650 2023-09-13T03:13:38.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.491+0000 7f2e5bbb7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdc5525400 con 0x55fdc5532710 2023-09-13T03:13:38.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.492+0000 7f2e45f9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fdc5532710 0x55fdc554c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:38.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.492+0000 7f2e45f9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fdc5532710 0x55fdc554c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:54522/0 (socket says 172.21.15.64:54522) 2023-09-13T03:13:38.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.492+0000 7f2e45f9b700 1 -- 172.21.15.64:0/672450486 learned_addr learned my addr 172.21.15.64:0/672450486 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:38.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.492+0000 7f2e4579a700 1 --2- 172.21.15.64:0/672450486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdc5533030 0x55fdc5542c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:38.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.492+0000 7f2e4679c700 1 --2- 172.21.15.64:0/672450486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdc5545650 0x55fdc553f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:38.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.492+0000 7f2e45f9b700 1 -- 172.21.15.64:0/672450486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdc5545650 msgr2=0x55fdc553f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:38.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.492+0000 7f2e45f9b700 1 --2- 172.21.15.64:0/672450486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdc5545650 0x55fdc553f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.492+0000 7f2e45f9b700 1 -- 172.21.15.64:0/672450486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdc5533030 msgr2=0x55fdc5542c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:38.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.492+0000 7f2e45f9b700 1 --2- 172.21.15.64:0/672450486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdc5533030 0x55fdc5542c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.492+0000 7f2e45f9b700 1 -- 172.21.15.64:0/672450486 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fdc52d7250 con 0x55fdc5532710 2023-09-13T03:13:38.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.492+0000 7f2e4579a700 1 --2- 172.21.15.64:0/672450486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdc5533030 0x55fdc5542c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:13:38.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.493+0000 7f2e45f9b700 1 --2- 172.21.15.64:0/672450486 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fdc5532710 0x55fdc554c1f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c005a80 tx=0x7f2e3c005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed906f9d14e16033 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:38.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.493+0000 7f2e34ff9700 1 -- 172.21.15.64:0/672450486 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c02c040 con 0x55fdc5532710 2023-09-13T03:13:38.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.493+0000 7f2e34ff9700 1 -- 172.21.15.64:0/672450486 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e3c01d040 con 0x55fdc5532710 2023-09-13T03:13:38.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.493+0000 7f2e34ff9700 1 -- 172.21.15.64:0/672450486 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c003680 con 0x55fdc5532710 2023-09-13T03:13:38.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.494+0000 7f2e5bbb7b80 1 -- 172.21.15.64:0/672450486 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fdc5532710 msgr2=0x55fdc554c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:38.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.494+0000 7f2e5bbb7b80 1 --2- 172.21.15.64:0/672450486 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fdc5532710 0x55fdc554c1f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c005a80 tx=0x7f2e3c005dc0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.494+0000 7f2e5bbb7b80 1 -- 172.21.15.64:0/672450486 shutdown_connections 2023-09-13T03:13:38.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.494+0000 7f2e5bbb7b80 1 --2- 172.21.15.64:0/672450486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdc5545650 0x55fdc553f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.494+0000 7f2e5bbb7b80 1 --2- 172.21.15.64:0/672450486 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fdc5532710 0x55fdc554c1f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.494+0000 7f2e5bbb7b80 1 --2- 172.21.15.64:0/672450486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdc5533030 0x55fdc5542c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.494+0000 7f2e5bbb7b80 1 -- 172.21.15.64:0/672450486 >> 172.21.15.64:0/672450486 conn(0x55fdc54a7090 msgr2=0x55fdc553d670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:38.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.495+0000 7f2e5bbb7b80 1 -- 172.21.15.64:0/672450486 shutdown_connections 2023-09-13T03:13:38.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.495+0000 7f2e5bbb7b80 1 -- 172.21.15.64:0/672450486 wait complete. 2023-09-13T03:13:38.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.495+0000 7f2e5bbb7b80 1 -- start start 2023-09-13T03:13:38.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.495+0000 7f2e5bbb7b80 1 Processor -- start 2023-09-13T03:13:38.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.496+0000 7f2e5bbb7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdc5532710 0x55fdc554ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.496+0000 7f2e5bbb7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdc5533030 0x55fdc554bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.496+0000 7f2e5bbb7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fdc5545650 0x55fdc5549f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.496+0000 7f2e5bbb7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdc554a630 con 0x55fdc5533030 2023-09-13T03:13:38.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.496+0000 7f2e5bbb7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdc554a7a0 con 0x55fdc5532710 2023-09-13T03:13:38.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.496+0000 7f2e5bbb7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdc554a960 con 0x55fdc5545650 2023-09-13T03:13:38.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.496+0000 7f2e5bbb7b80 10 client.0 _ll_register_callbacks cb 0x7f2e3c00b870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:13:38.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.497+0000 7f2e4579a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdc5533030 0x55fdc554bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:38.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.497+0000 7f2e4579a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdc5533030 0x55fdc554bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59406/0 (socket says 172.21.15.64:59406) 2023-09-13T03:13:38.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.497+0000 7f2e4579a700 1 -- 172.21.15.64:0/4010255453 learned_addr learned my addr 172.21.15.64:0/4010255453 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:38.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.497+0000 7f2e4679c700 1 --2- 172.21.15.64:0/4010255453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fdc5545650 0x55fdc5549f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:38.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.497+0000 7f2e45f9b700 1 --2- 172.21.15.64:0/4010255453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdc5532710 0x55fdc554ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:38.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.497+0000 7f2e45f9b700 1 -- 172.21.15.64:0/4010255453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fdc5545650 msgr2=0x55fdc5549f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:38.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.497+0000 7f2e45f9b700 1 --2- 172.21.15.64:0/4010255453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fdc5545650 0x55fdc5549f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.497+0000 7f2e45f9b700 1 -- 172.21.15.64:0/4010255453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdc5533030 msgr2=0x55fdc554bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:38.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.497+0000 7f2e45f9b700 1 --2- 172.21.15.64:0/4010255453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdc5533030 0x55fdc554bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.497+0000 7f2e45f9b700 1 -- 172.21.15.64:0/4010255453 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fdc5299c60 con 0x55fdc5532710 2023-09-13T03:13:38.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.498+0000 7f2e45f9b700 1 --2- 172.21.15.64:0/4010255453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdc5532710 0x55fdc554ba90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c03aae0 tx=0x7f2e3c00b360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dde1badca5f712e server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:38.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.498+0000 7f2e297fa700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:13:38.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.498+0000 7f2e297fa700 1 -- 172.21.15.64:0/4010255453 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c02c040 con 0x55fdc5532710 2023-09-13T03:13:38.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.498+0000 7f2e5bbb7b80 1 -- 172.21.15.64:0/4010255453 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2e300996f0 con 0x55fdc5532710 2023-09-13T03:13:38.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.498+0000 7f2e297fa700 1 -- 172.21.15.64:0/4010255453 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f2e3c03e070 con 0x55fdc5532710 2023-09-13T03:13:38.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.499+0000 7f2e297fa700 1 -- 172.21.15.64:0/4010255453 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e3c03a420 con 0x55fdc5532710 2023-09-13T03:13:38.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.499+0000 7f2e5bbb7b80 10 client.25877 fetch_fsmap learned FSMap version 26 2023-09-13T03:13:38.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.499+0000 7f2e5bbb7b80 1 -- 172.21.15.64:0/4010255453 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55fdc52af2b0 con 0x55fdc5532710 2023-09-13T03:13:38.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.499+0000 7f2e297fa700 1 -- 172.21.15.64:0/4010255453 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e300996f0 con 0x55fdc5532710 2023-09-13T03:13:38.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.499+0000 7f2e297fa700 1 -- 172.21.15.64:0/4010255453 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2e3c043470 con 0x55fdc5532710 2023-09-13T03:13:38.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.499+0000 7f2e5bbb7b80 10 client.25877 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:13:38.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.499+0000 7f2e5bbb7b80 10 client.25877 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T03:13:38.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.499+0000 7f2e5bbb7b80 20 client.25877 populate_metadata read hostname 'smithi064' 2023-09-13T03:13:38.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.499+0000 7f2e5bbb7b80 1 --2- 172.21.15.64:0/4010255453 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55fdc55a9dd0 0x55fdc55ac1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.499+0000 7f2e5bbb7b80 4 client.25877 mds_command: new command op to 24686 tid=0[{"prefix": "scrub status"}] 2023-09-13T03:13:38.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.499+0000 7f2e5bbb7b80 1 -- 172.21.15.64:0/4010255453 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fdc547d920 con 0x55fdc55a9dd0 2023-09-13T03:13:38.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.500+0000 7f2e4579a700 1 --2- 172.21.15.64:0/4010255453 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55fdc55a9dd0 0x55fdc55ac1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:38.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.500+0000 7f2e297fa700 10 client.25877 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:13:38.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.500+0000 7f2e4579a700 1 --2- 172.21.15.64:0/4010255453 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55fdc55a9dd0 0x55fdc55ac1c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f741812bfe27979 server_cookie=a88fca3c4e0fe36f in_seq=0 out_seq=0 2023-09-13T03:13:38.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.501+0000 7f2e297fa700 1 -- 172.21.15.64:0/4010255453 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55fdc547d920 con 0x55fdc55a9dd0 2023-09-13T03:13:38.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.501+0000 7f2e297fa700 10 client.25877 handle_command_reply: tid=0 2023-09-13T03:13:38.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.501+0000 7f2e5bbb7b80 1 client.25877 shutdown 2023-09-13T03:13:38.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.501+0000 7f2e5bbb7b80 1 -- 172.21.15.64:0/4010255453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdc5532710 msgr2=0x55fdc554ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:38.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.501+0000 7f2e5bbb7b80 1 --2- 172.21.15.64:0/4010255453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdc5532710 0x55fdc554ba90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c03aae0 tx=0x7f2e3c00b360 comp rx=0 tx=0).stop 2023-09-13T03:13:38.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.501+0000 7f2e5bbb7b80 1 -- 172.21.15.64:0/4010255453 shutdown_connections 2023-09-13T03:13:38.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.501+0000 7f2e5bbb7b80 1 --2- 172.21.15.64:0/4010255453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdc5532710 0x55fdc554ba90 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.501+0000 7f2e5bbb7b80 1 --2- 172.21.15.64:0/4010255453 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55fdc55a9dd0 0x55fdc55ac1c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.501+0000 7f2e5bbb7b80 1 --2- 172.21.15.64:0/4010255453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fdc5545650 0x55fdc5549f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.501+0000 7f2e297fa700 0 client.25877 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:13:38.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.501+0000 7f2e5bbb7b80 1 --2- 172.21.15.64:0/4010255453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdc5533030 0x55fdc554bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.501+0000 7f2e5bbb7b80 1 -- 172.21.15.64:0/4010255453 >> 172.21.15.64:0/4010255453 conn(0x55fdc54a7090 msgr2=0x55fdc5531700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:38.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.501+0000 7f2e5bbb7b80 1 -- 172.21.15.64:0/4010255453 shutdown_connections 2023-09-13T03:13:38.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.501+0000 7f2e5bbb7b80 1 -- 172.21.15.64:0/4010255453 wait complete. 2023-09-13T03:13:38.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.502+0000 7f2e5bbb7b80 20 client.25877 trim_cache size 0 max 16384 2023-09-13T03:13:38.513 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-13T03:13:38.513 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-13T03:13:38.513 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2023-09-13T03:13:38.514 INFO:teuthology.orchestra.run.smithi064.stdout: "c06fb957-4a86-4cfe-bac9-7fcc14b05a2b": { 2023-09-13T03:13:38.514 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2023-09-13T03:13:38.514 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "c06fb957-4a86-4cfe-bac9-7fcc14b05a2b", 2023-09-13T03:13:38.514 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2023-09-13T03:13:38.514 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:13:38.515 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:13:38.515 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-13T03:13:38.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.502+0000 7f2e481ff700 1 -- 172.21.15.64:0/1070602034 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f2e2c0a08c0 msgr2=0x7f2e2c0a2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:38.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.502+0000 7f2e481ff700 1 --2- 172.21.15.64:0/1070602034 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f2e2c0a08c0 0x7f2e2c0a2d80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c005e90 tx=0x7f2e3c019040 comp rx=0 tx=0).stop 2023-09-13T03:13:38.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.502+0000 7f2e481ff700 1 -- 172.21.15.64:0/1070602034 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e40102040 msgr2=0x7f2e4019ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:38.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.502+0000 7f2e481ff700 1 --2- 172.21.15.64:0/1070602034 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e40102040 0x7f2e4019ec10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2e30009fd0 tx=0x7f2e3000ee70 comp rx=0 tx=0).stop 2023-09-13T03:13:38.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.503+0000 7f2e481ff700 1 -- 172.21.15.64:0/1070602034 shutdown_connections 2023-09-13T03:13:38.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.503+0000 7f2e481ff700 1 --2- 172.21.15.64:0/1070602034 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f2e2c0a08c0 0x7f2e2c0a2d80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.503+0000 7f2e481ff700 1 --2- 172.21.15.64:0/1070602034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e40101720 0x7f2e4019e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.503+0000 7f2e481ff700 1 --2- 172.21.15.64:0/1070602034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e4010ab40 0x7f2e40198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.503+0000 7f2e481ff700 1 --2- 172.21.15.64:0/1070602034 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e40102040 0x7f2e4019ec10 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.503+0000 7f2e481ff700 1 -- 172.21.15.64:0/1070602034 >> 172.21.15.64:0/1070602034 conn(0x7f2e400faf60 msgr2=0x7f2e4010ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:38.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.503+0000 7f2e481ff700 1 -- 172.21.15.64:0/1070602034 shutdown_connections 2023-09-13T03:13:38.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:13:38.503+0000 7f2e481ff700 1 -- 172.21.15.64:0/1070602034 wait complete. 2023-09-13T03:13:38.532 INFO:tasks.cephfs.filesystem:scrub status for tag:c06fb957-4a86-4cfe-bac9-7fcc14b05a2b - {'path': '/', 'tag': 'c06fb957-4a86-4cfe-bac9-7fcc14b05a2b', 'options': 'recursive,force'} 2023-09-13T03:13:38.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:38 smithi170 ceph-mon[134328]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.6 KiB/s wr, 17 op/s 2023-09-13T03:13:38.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:38 smithi064 ceph-mon[97675]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.6 KiB/s wr, 17 op/s 2023-09-13T03:13:38.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.976+0000 7f51185ff700 1 -- 172.21.15.64:0/176719266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5110100f70 msgr2=0x7f511010d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:38.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.976+0000 7f51185ff700 1 --2- 172.21.15.64:0/176719266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5110100f70 0x7f511010d430 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f510800b0a0 tx=0x7f510800b3b0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.976+0000 7f51185ff700 1 -- 172.21.15.64:0/176719266 shutdown_connections 2023-09-13T03:13:38.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.976+0000 7f51185ff700 1 --2- 172.21.15.64:0/176719266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5110106670 0x7f5110106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.976+0000 7f51185ff700 1 --2- 172.21.15.64:0/176719266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5110100f70 0x7f511010d430 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.976+0000 7f51185ff700 1 --2- 172.21.15.64:0/176719266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51101005b0 0x7f5110100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.976+0000 7f51185ff700 1 -- 172.21.15.64:0/176719266 >> 172.21.15.64:0/176719266 conn(0x7f51100fc060 msgr2=0x7f51100fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:38.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.976+0000 7f51185ff700 1 -- 172.21.15.64:0/176719266 shutdown_connections 2023-09-13T03:13:38.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.976+0000 7f51185ff700 1 -- 172.21.15.64:0/176719266 wait complete. 2023-09-13T03:13:38.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.978+0000 7f51185ff700 1 Processor -- start 2023-09-13T03:13:38.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.978+0000 7f51185ff700 1 -- start start 2023-09-13T03:13:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.978+0000 7f51185ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51101005b0 0x7f511019e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.978+0000 7f51185ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5110100f70 0x7f511019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.979+0000 7f51185ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5110106670 0x7f51101987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.979+0000 7f51185ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5110198e60 con 0x7f51101005b0 2023-09-13T03:13:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.979+0000 7f51185ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5110198fd0 con 0x7f5110100f70 2023-09-13T03:13:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.979+0000 7f51185ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5110199190 con 0x7f5110106670 2023-09-13T03:13:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.979+0000 7f511639b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51101005b0 0x7f511019e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.979+0000 7f511639b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51101005b0 0x7f511019e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59414/0 (socket says 172.21.15.64:59414) 2023-09-13T03:13:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.979+0000 7f511639b700 1 -- 172.21.15.64:0/2412809301 learned_addr learned my addr 172.21.15.64:0/2412809301 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.979+0000 7f5116b9c700 1 --2- 172.21.15.64:0/2412809301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5110106670 0x7f51101987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.979+0000 7f5115b9a700 1 --2- 172.21.15.64:0/2412809301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5110100f70 0x7f511019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.979+0000 7f5116b9c700 1 -- 172.21.15.64:0/2412809301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5110100f70 msgr2=0x7f511019eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.979+0000 7f5116b9c700 1 --2- 172.21.15.64:0/2412809301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5110100f70 0x7f511019eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.979+0000 7f5116b9c700 1 -- 172.21.15.64:0/2412809301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51101005b0 msgr2=0x7f511019e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.979+0000 7f5116b9c700 1 --2- 172.21.15.64:0/2412809301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51101005b0 0x7f511019e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.979+0000 7f5116b9c700 1 -- 172.21.15.64:0/2412809301 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5108009d00 con 0x7f5110106670 2023-09-13T03:13:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.980+0000 7f5116b9c700 1 --2- 172.21.15.64:0/2412809301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5110106670 0x7f51101987f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5108009fd0 tx=0x7f5108004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.980+0000 7f51077fe700 1 -- 172.21.15.64:0/2412809301 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510800e040 con 0x7f5110106670 2023-09-13T03:13:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.980+0000 7f51077fe700 1 -- 172.21.15.64:0/2412809301 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5108004e70 con 0x7f5110106670 2023-09-13T03:13:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.980+0000 7f51185ff700 1 -- 172.21.15.64:0/2412809301 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5110199410 con 0x7f5110106670 2023-09-13T03:13:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.980+0000 7f51077fe700 1 -- 172.21.15.64:0/2412809301 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510801d9a0 con 0x7f5110106670 2023-09-13T03:13:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.981+0000 7f51185ff700 1 -- 172.21.15.64:0/2412809301 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51101a5980 con 0x7f5110106670 2023-09-13T03:13:38.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.987+0000 7f51077fe700 1 -- 172.21.15.64:0/2412809301 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 24) v1 ==== 135435+0+0 (secure 0 0 0) 0x7f51080057a0 con 0x7f5110106670 2023-09-13T03:13:38.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.987+0000 7f512bea3b80 1 Processor -- start 2023-09-13T03:13:38.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.987+0000 7f512bea3b80 1 -- start start 2023-09-13T03:13:38.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.988+0000 7f512bea3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bbc6311a0 0x560bbc631580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.988+0000 7f512bea3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560bbc62b300 0x560bbc62b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.988+0000 7f512bea3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560bbc62bcc0 0x560bbc637f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.988+0000 7f512bea3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bbc5846d0 con 0x560bbc62bcc0 2023-09-13T03:13:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.988+0000 7f512bea3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bbc581ff0 con 0x560bbc62b300 2023-09-13T03:13:38.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.988+0000 7f512bea3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bbc57d8e0 con 0x560bbc6311a0 2023-09-13T03:13:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.988+0000 7f5115b9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bbc6311a0 0x560bbc631580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.988+0000 7f5115b9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bbc6311a0 0x560bbc631580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:54556/0 (socket says 172.21.15.64:54556) 2023-09-13T03:13:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.988+0000 7f5115b9a700 1 -- 172.21.15.64:0/508019906 learned_addr learned my addr 172.21.15.64:0/508019906 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.989+0000 7f5115b9a700 1 --2- 172.21.15.64:0/508019906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560bbc62bcc0 0x560bbc637f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.989+0000 7f511639b700 1 --2- 172.21.15.64:0/508019906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560bbc62b300 0x560bbc62b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.989+0000 7f5115b9a700 1 -- 172.21.15.64:0/508019906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560bbc62b300 msgr2=0x560bbc62b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.989+0000 7f5115b9a700 1 --2- 172.21.15.64:0/508019906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560bbc62b300 0x560bbc62b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.989+0000 7f5115b9a700 1 -- 172.21.15.64:0/508019906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560bbc62bcc0 msgr2=0x560bbc637f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.989+0000 7f5115b9a700 1 --2- 172.21.15.64:0/508019906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560bbc62bcc0 0x560bbc637f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.989+0000 7f5115b9a700 1 -- 172.21.15.64:0/508019906 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560bbc3a9d80 con 0x560bbc6311a0 2023-09-13T03:13:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.989+0000 7f51077fe700 1 --2- 172.21.15.64:0/2412809301 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f50fc0a08c0 0x7f50fc0a2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.989+0000 7f5115b9a700 1 --2- 172.21.15.64:0/508019906 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bbc6311a0 0x560bbc631580 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5100012840 tx=0x7f510000b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e433849853c1d826 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.989+0000 7f51077fe700 1 -- 172.21.15.64:0/2412809301 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f51080cd390 con 0x7f5110106670 2023-09-13T03:13:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.990+0000 7f511639b700 1 --2- 172.21.15.64:0/2412809301 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f50fc0a08c0 0x7f50fc0a2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.990+0000 7f5104bfe700 1 -- 172.21.15.64:0/508019906 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5100012e60 con 0x560bbc6311a0 2023-09-13T03:13:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.990+0000 7f5104bfe700 1 -- 172.21.15.64:0/508019906 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51000044f0 con 0x560bbc6311a0 2023-09-13T03:13:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.990+0000 7f5104bfe700 1 -- 172.21.15.64:0/508019906 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5100005280 con 0x560bbc6311a0 2023-09-13T03:13:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.991+0000 7f512bea3b80 1 -- 172.21.15.64:0/508019906 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bbc6311a0 msgr2=0x560bbc631580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.991+0000 7f511639b700 1 --2- 172.21.15.64:0/2412809301 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f50fc0a08c0 0x7f50fc0a2d80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x560bbc626060 tx=0x7f510c00b500 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.991+0000 7f512bea3b80 1 --2- 172.21.15.64:0/508019906 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bbc6311a0 0x560bbc631580 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5100012840 tx=0x7f510000b860 comp rx=0 tx=0).stop 2023-09-13T03:13:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.991+0000 7f512bea3b80 1 -- 172.21.15.64:0/508019906 shutdown_connections 2023-09-13T03:13:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.991+0000 7f512bea3b80 1 --2- 172.21.15.64:0/508019906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560bbc62bcc0 0x560bbc637f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.991+0000 7f512bea3b80 1 --2- 172.21.15.64:0/508019906 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bbc6311a0 0x560bbc631580 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.991+0000 7f512bea3b80 1 --2- 172.21.15.64:0/508019906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560bbc62b300 0x560bbc62b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.991+0000 7f512bea3b80 1 -- 172.21.15.64:0/508019906 >> 172.21.15.64:0/508019906 conn(0x560bbc5a0030 msgr2=0x560bbc626560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.991+0000 7f512bea3b80 1 -- 172.21.15.64:0/508019906 shutdown_connections 2023-09-13T03:13:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.991+0000 7f512bea3b80 1 -- 172.21.15.64:0/508019906 wait complete. 2023-09-13T03:13:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.992+0000 7f512bea3b80 1 -- start start 2023-09-13T03:13:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.993+0000 7f512bea3b80 1 Processor -- start 2023-09-13T03:13:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.993+0000 7f512bea3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560bbc62b300 0x560bbc637e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.993+0000 7f512bea3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bbc62bcc0 0x560bbc6350a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.993+0000 7f512bea3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560bbc6311a0 0x560bbc6356d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.993+0000 7f512bea3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bbc51bb10 con 0x560bbc6311a0 2023-09-13T03:13:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.993+0000 7f512bea3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bbc51b810 con 0x560bbc62b300 2023-09-13T03:13:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.993+0000 7f512bea3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bbc3ff8c0 con 0x560bbc62bcc0 2023-09-13T03:13:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.993+0000 7f512bea3b80 10 client.0 _ll_register_callbacks cb 0x560bbc465a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:13:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.994+0000 7f5115b9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bbc62bcc0 0x560bbc6350a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.994+0000 7f5115b9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bbc62bcc0 0x560bbc6350a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:54572/0 (socket says 172.21.15.64:54572) 2023-09-13T03:13:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.994+0000 7f5115b9a700 1 -- 172.21.15.64:0/442754659 learned_addr learned my addr 172.21.15.64:0/442754659 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.994+0000 7f511639b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560bbc62b300 0x560bbc637e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.994+0000 7f5116b9c700 1 --2- 172.21.15.64:0/442754659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560bbc6311a0 0x560bbc6356d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.994+0000 7f5115b9a700 1 -- 172.21.15.64:0/442754659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560bbc62b300 msgr2=0x560bbc637e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.994+0000 7f5115b9a700 1 --2- 172.21.15.64:0/442754659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560bbc62b300 0x560bbc637e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.994+0000 7f5115b9a700 1 -- 172.21.15.64:0/442754659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560bbc6311a0 msgr2=0x560bbc6356d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.995+0000 7f5115b9a700 1 --2- 172.21.15.64:0/442754659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560bbc6311a0 0x560bbc6356d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.995+0000 7f5115b9a700 1 -- 172.21.15.64:0/442754659 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f510001a040 con 0x560bbc62bcc0 2023-09-13T03:13:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.995+0000 7f5115b9a700 1 --2- 172.21.15.64:0/442754659 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bbc62bcc0 0x560bbc6350a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5100012810 tx=0x7f5100004980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eed6377e28554b8b server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:39.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.995+0000 7f50f9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:13:39.002 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:38 smithi142 ceph-mon[134082]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.6 KiB/s wr, 17 op/s 2023-09-13T03:13:39.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.995+0000 7f50f9ffb700 1 -- 172.21.15.64:0/442754659 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5100012e30 con 0x560bbc62bcc0 2023-09-13T03:13:39.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.995+0000 7f512bea3b80 1 -- 172.21.15.64:0/442754659 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560bbc4d5490 con 0x560bbc62bcc0 2023-09-13T03:13:39.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.995+0000 7f50f9ffb700 1 -- 172.21.15.64:0/442754659 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f5100005230 con 0x560bbc62bcc0 2023-09-13T03:13:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.996+0000 7f50f9ffb700 1 -- 172.21.15.64:0/442754659 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51000124f0 con 0x560bbc62bcc0 2023-09-13T03:13:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.996+0000 7f512bea3b80 10 client.25891 fetch_fsmap learned FSMap version 26 2023-09-13T03:13:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.996+0000 7f512bea3b80 1 -- 172.21.15.64:0/442754659 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x560bbc392c00 con 0x560bbc62bcc0 2023-09-13T03:13:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.996+0000 7f50f9ffb700 1 -- 172.21.15.64:0/442754659 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560bbc4d5490 con 0x560bbc62bcc0 2023-09-13T03:13:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.997+0000 7f50f9ffb700 1 -- 172.21.15.64:0/442754659 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f510001f920 con 0x560bbc62bcc0 2023-09-13T03:13:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.997+0000 7f512bea3b80 10 client.25891 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:13:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.997+0000 7f512bea3b80 10 client.25891 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:13:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.997+0000 7f512bea3b80 20 client.25891 populate_metadata read hostname 'smithi064' 2023-09-13T03:13:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.997+0000 7f512bea3b80 1 --2- 172.21.15.64:0/442754659 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x560bbc645350 0x560bbc647770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.997+0000 7f512bea3b80 4 client.25891 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:13:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.997+0000 7f512bea3b80 1 -- 172.21.15.64:0/442754659 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560bbc592840 con 0x560bbc645350 2023-09-13T03:13:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.998+0000 7f511639b700 1 --2- 172.21.15.64:0/442754659 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x560bbc645350 0x560bbc647770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.998+0000 7f511639b700 1 --2- 172.21.15.64:0/442754659 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x560bbc645350 0x560bbc647770 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d2bd677dd6cccdb server_cookie=d0e3291c9fb6de7e in_seq=0 out_seq=0 2023-09-13T03:13:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.998+0000 7f50f9ffb700 10 client.25891 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:13:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.999+0000 7f50f9ffb700 1 -- 172.21.15.64:0/442754659 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560bbc592840 con 0x560bbc645350 2023-09-13T03:13:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.999+0000 7f50f9ffb700 10 client.25891 handle_command_reply: tid=0 2023-09-13T03:13:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:38.999+0000 7f512bea3b80 1 client.25891 shutdown 2023-09-13T03:13:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.000+0000 7f512bea3b80 1 -- 172.21.15.64:0/442754659 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bbc62bcc0 msgr2=0x560bbc6350a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.000+0000 7f512bea3b80 1 --2- 172.21.15.64:0/442754659 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bbc62bcc0 0x560bbc6350a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5100012810 tx=0x7f5100004980 comp rx=0 tx=0).stop 2023-09-13T03:13:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.000+0000 7f512bea3b80 1 -- 172.21.15.64:0/442754659 shutdown_connections 2023-09-13T03:13:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.000+0000 7f512bea3b80 1 --2- 172.21.15.64:0/442754659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560bbc62b300 0x560bbc637e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.000+0000 7f512bea3b80 1 --2- 172.21.15.64:0/442754659 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x560bbc645350 0x560bbc647770 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.000+0000 7f512bea3b80 1 --2- 172.21.15.64:0/442754659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560bbc6311a0 0x560bbc6356d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.000+0000 7f512bea3b80 1 --2- 172.21.15.64:0/442754659 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bbc62bcc0 0x560bbc6350a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.000+0000 7f512bea3b80 1 -- 172.21.15.64:0/442754659 >> 172.21.15.64:0/442754659 conn(0x560bbc5a0030 msgr2=0x560bbc6257a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:39.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.000+0000 7f50f9ffb700 0 client.25891 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:13:39.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.000+0000 7f512bea3b80 1 -- 172.21.15.64:0/442754659 shutdown_connections 2023-09-13T03:13:39.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.000+0000 7f512bea3b80 1 -- 172.21.15.64:0/442754659 wait complete. 2023-09-13T03:13:39.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.000+0000 7f512bea3b80 20 client.25891 trim_cache size 0 max 16384 2023-09-13T03:13:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.030+0000 7f512bea3b80 1 Processor -- start 2023-09-13T03:13:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.030+0000 7f512bea3b80 1 -- start start 2023-09-13T03:13:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.030+0000 7f512bea3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bbc62b300 0x560bbc632d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.030+0000 7f512bea3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560bbc62bcc0 0x560bbc626090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.030+0000 7f512bea3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560bbc6311a0 0x560bbc6265d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.031+0000 7f512bea3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bbc62c0a0 con 0x560bbc6311a0 2023-09-13T03:13:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.031+0000 7f512bea3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bbc627320 con 0x560bbc62bcc0 2023-09-13T03:13:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.031+0000 7f512bea3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bbc647dd0 con 0x560bbc62b300 2023-09-13T03:13:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.031+0000 7f5115b9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560bbc62bcc0 0x560bbc626090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.031+0000 7f511639b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bbc62b300 0x560bbc632d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.031+0000 7f5115b9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560bbc62bcc0 0x560bbc626090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:57514/0 (socket says 172.21.15.64:57514) 2023-09-13T03:13:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.031+0000 7f5115b9a700 1 -- 172.21.15.64:0/2696888186 learned_addr learned my addr 172.21.15.64:0/2696888186 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.031+0000 7f5116b9c700 1 --2- 172.21.15.64:0/2696888186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560bbc6311a0 0x560bbc6265d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.032+0000 7f5115b9a700 1 -- 172.21.15.64:0/2696888186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bbc62b300 msgr2=0x560bbc632d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.032+0000 7f5115b9a700 1 --2- 172.21.15.64:0/2696888186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bbc62b300 0x560bbc632d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.032+0000 7f5115b9a700 1 -- 172.21.15.64:0/2696888186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560bbc6311a0 msgr2=0x560bbc6265d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.032+0000 7f5115b9a700 1 --2- 172.21.15.64:0/2696888186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560bbc6311a0 0x560bbc6265d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.032+0000 7f5115b9a700 1 -- 172.21.15.64:0/2696888186 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560bbc3d01f0 con 0x560bbc62bcc0 2023-09-13T03:13:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.032+0000 7f5115b9a700 1 --2- 172.21.15.64:0/2696888186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560bbc62bcc0 0x560bbc626090 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f510001fca0 tx=0x7f5100001e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78437f274d95bb86 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.032+0000 7f51053ff700 1 -- 172.21.15.64:0/2696888186 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510000bbe0 con 0x560bbc62bcc0 2023-09-13T03:13:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.033+0000 7f51053ff700 1 -- 172.21.15.64:0/2696888186 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f510001fe90 con 0x560bbc62bcc0 2023-09-13T03:13:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.033+0000 7f51053ff700 1 -- 172.21.15.64:0/2696888186 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51000054d0 con 0x560bbc62bcc0 2023-09-13T03:13:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.033+0000 7f512bea3b80 1 -- 172.21.15.64:0/2696888186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560bbc62bcc0 msgr2=0x560bbc626090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.034+0000 7f512bea3b80 1 --2- 172.21.15.64:0/2696888186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560bbc62bcc0 0x560bbc626090 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f510001fca0 tx=0x7f5100001e50 comp rx=0 tx=0).stop 2023-09-13T03:13:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.034+0000 7f512bea3b80 1 -- 172.21.15.64:0/2696888186 shutdown_connections 2023-09-13T03:13:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.034+0000 7f512bea3b80 1 --2- 172.21.15.64:0/2696888186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560bbc6311a0 0x560bbc6265d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.034+0000 7f512bea3b80 1 --2- 172.21.15.64:0/2696888186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bbc62b300 0x560bbc632d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.034+0000 7f512bea3b80 1 --2- 172.21.15.64:0/2696888186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560bbc62bcc0 0x560bbc626090 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.034+0000 7f512bea3b80 1 -- 172.21.15.64:0/2696888186 >> 172.21.15.64:0/2696888186 conn(0x560bbc5a0030 msgr2=0x560bbc630090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.034+0000 7f512bea3b80 1 -- 172.21.15.64:0/2696888186 shutdown_connections 2023-09-13T03:13:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.034+0000 7f512bea3b80 1 -- 172.21.15.64:0/2696888186 wait complete. 2023-09-13T03:13:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.035+0000 7f512bea3b80 1 -- start start 2023-09-13T03:13:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.035+0000 7f512bea3b80 1 Processor -- start 2023-09-13T03:13:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.036+0000 7f512bea3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bbc62b300 0x560bbc63c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.036+0000 7f512bea3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560bbc62bcc0 0x560bbc63a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.036+0000 7f512bea3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560bbc6311a0 0x560bbc63aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.036+0000 7f512bea3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bbc63afd0 con 0x560bbc62bcc0 2023-09-13T03:13:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.036+0000 7f512bea3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bbc63b140 con 0x560bbc6311a0 2023-09-13T03:13:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.036+0000 7f512bea3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bbc63b300 con 0x560bbc62b300 2023-09-13T03:13:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.036+0000 7f512bea3b80 10 client.0 _ll_register_callbacks cb 0x7f510000bf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:13:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.036+0000 7f511639b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bbc62b300 0x560bbc63c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.036+0000 7f511639b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bbc62b300 0x560bbc63c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:54590/0 (socket says 172.21.15.64:54590) 2023-09-13T03:13:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.036+0000 7f511639b700 1 -- 172.21.15.64:0/3091716604 learned_addr learned my addr 172.21.15.64:0/3091716604 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.036+0000 7f5116b9c700 1 --2- 172.21.15.64:0/3091716604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560bbc6311a0 0x560bbc63aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.037+0000 7f511639b700 1 -- 172.21.15.64:0/3091716604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560bbc6311a0 msgr2=0x560bbc63aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.037+0000 7f511639b700 1 --2- 172.21.15.64:0/3091716604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560bbc6311a0 0x560bbc63aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.037+0000 7f511639b700 1 -- 172.21.15.64:0/3091716604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560bbc62bcc0 msgr2=0x560bbc63a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.037+0000 7f511639b700 1 --2- 172.21.15.64:0/3091716604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560bbc62bcc0 0x560bbc63a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.037+0000 7f511639b700 1 -- 172.21.15.64:0/3091716604 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560bbc392c00 con 0x560bbc62b300 2023-09-13T03:13:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.037+0000 7f511639b700 1 --2- 172.21.15.64:0/3091716604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bbc62b300 0x560bbc63c220 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x560bbc648040 tx=0x7f510c018070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ff1ffa4d1c7202e server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.037+0000 7f50f9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:13:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.037+0000 7f50f9ffb700 1 -- 172.21.15.64:0/3091716604 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510c00d990 con 0x560bbc62b300 2023-09-13T03:13:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.038+0000 7f50f9ffb700 1 -- 172.21.15.64:0/3091716604 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f510c00daf0 con 0x560bbc62b300 2023-09-13T03:13:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.038+0000 7f512bea3b80 1 -- 172.21.15.64:0/3091716604 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51000044f0 con 0x560bbc62b300 2023-09-13T03:13:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.038+0000 7f50f9ffb700 1 -- 172.21.15.64:0/3091716604 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f510c00dcb0 con 0x560bbc62b300 2023-09-13T03:13:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.038+0000 7f512bea3b80 10 client.25897 fetch_fsmap learned FSMap version 26 2023-09-13T03:13:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.038+0000 7f512bea3b80 1 -- 172.21.15.64:0/3091716604 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x560bbc3a8250 con 0x560bbc62b300 2023-09-13T03:13:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.038+0000 7f50f9ffb700 1 -- 172.21.15.64:0/3091716604 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51000044f0 con 0x560bbc62b300 2023-09-13T03:13:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.038+0000 7f50f9ffb700 1 -- 172.21.15.64:0/3091716604 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f510c02d070 con 0x560bbc62b300 2023-09-13T03:13:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.039+0000 7f512bea3b80 10 client.25897 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:13:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.039+0000 7f512bea3b80 10 client.25897 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:13:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.039+0000 7f512bea3b80 20 client.25897 populate_metadata read hostname 'smithi064' 2023-09-13T03:13:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.039+0000 7f512bea3b80 1 --2- 172.21.15.64:0/3091716604 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x560bbc69a6c0 0x560bbc69cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.039+0000 7f512bea3b80 4 client.25897 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:13:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.039+0000 7f512bea3b80 1 -- 172.21.15.64:0/3091716604 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560bbc5768c0 con 0x560bbc69a6c0 2023-09-13T03:13:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.039+0000 7f5115b9a700 1 --2- 172.21.15.64:0/3091716604 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x560bbc69a6c0 0x560bbc69cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.040+0000 7f5115b9a700 1 --2- 172.21.15.64:0/3091716604 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x560bbc69a6c0 0x560bbc69cb00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=114a2b237383f626 server_cookie=c4f8fd11d97c7617 in_seq=0 out_seq=0 2023-09-13T03:13:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.040+0000 7f50f9ffb700 10 client.25897 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:13:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.044+0000 7f50f9ffb700 1 -- 172.21.15.64:0/3091716604 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560bbc5768c0 con 0x560bbc69a6c0 2023-09-13T03:13:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.044+0000 7f50f9ffb700 10 client.25897 handle_command_reply: tid=0 2023-09-13T03:13:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.044+0000 7f512bea3b80 1 client.25897 shutdown 2023-09-13T03:13:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.044+0000 7f512bea3b80 1 -- 172.21.15.64:0/3091716604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bbc62b300 msgr2=0x560bbc63c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.044+0000 7f512bea3b80 1 --2- 172.21.15.64:0/3091716604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bbc62b300 0x560bbc63c220 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x560bbc648040 tx=0x7f510c018070 comp rx=0 tx=0).stop 2023-09-13T03:13:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.044+0000 7f512bea3b80 1 -- 172.21.15.64:0/3091716604 shutdown_connections 2023-09-13T03:13:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.044+0000 7f512bea3b80 1 --2- 172.21.15.64:0/3091716604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560bbc6311a0 0x560bbc63aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.044+0000 7f512bea3b80 1 --2- 172.21.15.64:0/3091716604 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x560bbc69a6c0 0x560bbc69cb00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.044+0000 7f512bea3b80 1 --2- 172.21.15.64:0/3091716604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bbc62b300 0x560bbc63c220 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.044+0000 7f512bea3b80 1 --2- 172.21.15.64:0/3091716604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560bbc62bcc0 0x560bbc63a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.044+0000 7f50f9ffb700 0 client.25897 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:13:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.044+0000 7f512bea3b80 1 -- 172.21.15.64:0/3091716604 >> 172.21.15.64:0/3091716604 conn(0x560bbc5a0030 msgr2=0x560bbc626090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.045+0000 7f512bea3b80 1 -- 172.21.15.64:0/3091716604 shutdown_connections 2023-09-13T03:13:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.045+0000 7f512bea3b80 1 -- 172.21.15.64:0/3091716604 wait complete. 2023-09-13T03:13:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.045+0000 7f512bea3b80 20 client.25897 trim_cache size 0 max 16384 2023-09-13T03:13:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:13:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:13:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:13:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:13:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.046+0000 7f51185ff700 1 -- 172.21.15.64:0/2412809301 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f50fc0a08c0 msgr2=0x7f50fc0a2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.046+0000 7f51185ff700 1 --2- 172.21.15.64:0/2412809301 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f50fc0a08c0 0x7f50fc0a2d80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x560bbc626060 tx=0x7f510c00b500 comp rx=0 tx=0).stop 2023-09-13T03:13:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.046+0000 7f51185ff700 1 -- 172.21.15.64:0/2412809301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5110106670 msgr2=0x7f51101987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.046+0000 7f51185ff700 1 --2- 172.21.15.64:0/2412809301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5110106670 0x7f51101987f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5108009fd0 tx=0x7f5108004c80 comp rx=0 tx=0).stop 2023-09-13T03:13:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.046+0000 7f51185ff700 1 -- 172.21.15.64:0/2412809301 shutdown_connections 2023-09-13T03:13:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.046+0000 7f51185ff700 1 --2- 172.21.15.64:0/2412809301 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f50fc0a08c0 0x7f50fc0a2d80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.046+0000 7f51185ff700 1 --2- 172.21.15.64:0/2412809301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5110106670 0x7f51101987f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.046+0000 7f51185ff700 1 --2- 172.21.15.64:0/2412809301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51101005b0 0x7f511019e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.046+0000 7f51185ff700 1 --2- 172.21.15.64:0/2412809301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5110100f70 0x7f511019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.046+0000 7f51185ff700 1 -- 172.21.15.64:0/2412809301 >> 172.21.15.64:0/2412809301 conn(0x7f51100fc060 msgr2=0x7f511010b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.047+0000 7f51185ff700 1 -- 172.21.15.64:0/2412809301 shutdown_connections 2023-09-13T03:13:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:39.047+0000 7f51185ff700 1 -- 172.21.15.64:0/2412809301 wait complete. 2023-09-13T03:13:39.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:39 smithi170 ceph-mon[134328]: scrub queued for path: / 2023-09-13T03:13:39.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:39 smithi170 ceph-mon[134328]: scrub summary: idle+waiting paths [/] 2023-09-13T03:13:39.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:39 smithi170 ceph-mon[134328]: scrub summary: active paths [/] 2023-09-13T03:13:39.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:39 smithi170 ceph-mon[134328]: scrub summary: idle+waiting paths [/] 2023-09-13T03:13:39.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:39 smithi170 ceph-mon[134328]: scrub summary: idle 2023-09-13T03:13:39.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:39 smithi064 ceph-mon[97675]: scrub queued for path: / 2023-09-13T03:13:39.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:39 smithi064 ceph-mon[97675]: scrub summary: idle+waiting paths [/] 2023-09-13T03:13:39.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:39 smithi064 ceph-mon[97675]: scrub summary: active paths [/] 2023-09-13T03:13:39.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:39 smithi064 ceph-mon[97675]: scrub summary: idle+waiting paths [/] 2023-09-13T03:13:39.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:39 smithi064 ceph-mon[97675]: scrub summary: idle 2023-09-13T03:13:39.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:39 smithi142 ceph-mon[134082]: scrub queued for path: / 2023-09-13T03:13:39.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:39 smithi142 ceph-mon[134082]: scrub summary: idle+waiting paths [/] 2023-09-13T03:13:39.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:39 smithi142 ceph-mon[134082]: scrub summary: active paths [/] 2023-09-13T03:13:39.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:39 smithi142 ceph-mon[134082]: scrub summary: idle+waiting paths [/] 2023-09-13T03:13:39.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:39 smithi142 ceph-mon[134082]: scrub summary: idle 2023-09-13T03:13:40.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:40 smithi170 ceph-mon[134328]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2023-09-13T03:13:40.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:40 smithi064 ceph-mon[97675]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2023-09-13T03:13:40.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:40 smithi142 ceph-mon[134082]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2023-09-13T03:13:42.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:42 smithi170 ceph-mon[134328]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 1.8 KiB/s wr, 19 op/s 2023-09-13T03:13:42.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:42 smithi064 ceph-mon[97675]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 1.8 KiB/s wr, 19 op/s 2023-09-13T03:13:42.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:42 smithi142 ceph-mon[134082]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 1.8 KiB/s wr, 19 op/s 2023-09-13T03:13:44.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:44 smithi170 ceph-mon[134328]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 10 op/s 2023-09-13T03:13:44.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:44 smithi064 ceph-mon[97675]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 10 op/s 2023-09-13T03:13:44.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:44 smithi142 ceph-mon[134082]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 10 op/s 2023-09-13T03:13:46.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:46 smithi170 ceph-mon[134328]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 255 B/s wr, 11 op/s 2023-09-13T03:13:46.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:46 smithi064 ceph-mon[97675]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 255 B/s wr, 11 op/s 2023-09-13T03:13:46.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:46 smithi142 ceph-mon[134082]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 255 B/s wr, 11 op/s 2023-09-13T03:13:48.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:48 smithi170 ceph-mon[134328]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 255 B/s wr, 15 op/s 2023-09-13T03:13:48.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:48 smithi064 ceph-mon[97675]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 255 B/s wr, 15 op/s 2023-09-13T03:13:48.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:48 smithi142 ceph-mon[134082]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 255 B/s wr, 15 op/s 2023-09-13T03:13:50.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:50 smithi142 ceph-mon[134082]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2023-09-13T03:13:50.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:50 smithi170 ceph-mon[134328]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2023-09-13T03:13:50.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:50 smithi064 ceph-mon[97675]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2023-09-13T03:13:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.212+0000 7fc23afff700 1 -- 172.21.15.64:0/3087989014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2340732c0 msgr2=0x7fc234111050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.212+0000 7fc23afff700 1 --2- 172.21.15.64:0/3087989014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2340732c0 0x7fc234111050 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc220009990 tx=0x7fc220009ca0 comp rx=0 tx=0).stop 2023-09-13T03:13:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.212+0000 7fc23afff700 1 -- 172.21.15.64:0/3087989014 shutdown_connections 2023-09-13T03:13:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.212+0000 7fc23afff700 1 --2- 172.21.15.64:0/3087989014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2340729a0 0x7fc234072d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.212+0000 7fc23afff700 1 --2- 172.21.15.64:0/3087989014 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc234111590 0x7fc234113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.212+0000 7fc23afff700 1 --2- 172.21.15.64:0/3087989014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2340732c0 0x7fc234111050 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.212+0000 7fc23afff700 1 -- 172.21.15.64:0/3087989014 >> 172.21.15.64:0/3087989014 conn(0x7fc2340fc050 msgr2=0x7fc2340fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.212+0000 7fc23afff700 1 -- 172.21.15.64:0/3087989014 shutdown_connections 2023-09-13T03:13:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.212+0000 7fc23afff700 1 -- 172.21.15.64:0/3087989014 wait complete. 2023-09-13T03:13:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.214+0000 7fc23afff700 1 Processor -- start 2023-09-13T03:13:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.214+0000 7fc23afff700 1 -- start start 2023-09-13T03:13:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.215+0000 7fc23afff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2340729a0 0x7fc2341a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.215+0000 7fc23afff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2340732c0 0x7fc2341a31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.215+0000 7fc23afff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc234111590 0x7fc23419cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.215+0000 7fc23afff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23419d140 con 0x7fc2340732c0 2023-09-13T03:13:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.215+0000 7fc23afff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23419d2b0 con 0x7fc234111590 2023-09-13T03:13:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.215+0000 7fc23afff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23419d470 con 0x7fc2340729a0 2023-09-13T03:13:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.215+0000 7fc238d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2340729a0 0x7fc2341a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.215+0000 7fc238d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2340729a0 0x7fc2341a2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:48866/0 (socket says 172.21.15.64:48866) 2023-09-13T03:13:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.215+0000 7fc238d9b700 1 -- 172.21.15.64:0/3768192767 learned_addr learned my addr 172.21.15.64:0/3768192767 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.215+0000 7fc233fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2340732c0 0x7fc2341a31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.215+0000 7fc23959c700 1 --2- 172.21.15.64:0/3768192767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc234111590 0x7fc23419cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.215+0000 7fc238d9b700 1 -- 172.21.15.64:0/3768192767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc234111590 msgr2=0x7fc23419cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.215+0000 7fc238d9b700 1 --2- 172.21.15.64:0/3768192767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc234111590 0x7fc23419cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.215+0000 7fc238d9b700 1 -- 172.21.15.64:0/3768192767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2340732c0 msgr2=0x7fc2341a31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.215+0000 7fc238d9b700 1 --2- 172.21.15.64:0/3768192767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2340732c0 0x7fc2341a31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:52.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.215+0000 7fc238d9b700 1 -- 172.21.15.64:0/3768192767 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2200095f0 con 0x7fc2340729a0 2023-09-13T03:13:52.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.216+0000 7fc238d9b700 1 --2- 172.21.15.64:0/3768192767 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2340729a0 0x7fc2341a2c70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc22800e9a0 tx=0x7fc22800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:52.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.216+0000 7fc231ffb700 1 -- 172.21.15.64:0/3768192767 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc228009790 con 0x7fc2340729a0 2023-09-13T03:13:52.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.216+0000 7fc231ffb700 1 -- 172.21.15.64:0/3768192767 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc228010430 con 0x7fc2340729a0 2023-09-13T03:13:52.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.216+0000 7fc23afff700 1 -- 172.21.15.64:0/3768192767 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc23419d750 con 0x7fc2340729a0 2023-09-13T03:13:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.216+0000 7fc231ffb700 1 -- 172.21.15.64:0/3768192767 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc228018680 con 0x7fc2340729a0 2023-09-13T03:13:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.216+0000 7fc23afff700 1 -- 172.21.15.64:0/3768192767 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2341a9c60 con 0x7fc2340729a0 2023-09-13T03:13:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.221+0000 7fc231ffb700 1 -- 172.21.15.64:0/3768192767 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 24) v1 ==== 135435+0+0 (secure 0 0 0) 0x7fc2280105a0 con 0x7fc2340729a0 2023-09-13T03:13:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.222+0000 7fc24e9a7b80 1 Processor -- start 2023-09-13T03:13:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.222+0000 7fc24e9a7b80 1 -- start start 2023-09-13T03:13:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.223+0000 7fc24e9a7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f7ba8d31a0 0x55f7ba8d3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.223+0000 7fc24e9a7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7ba8cd300 0x55f7ba8cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.223+0000 7fc24e9a7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7ba8cdcc0 0x55f7ba8d9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.223+0000 7fc24e9a7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7ba8266d0 con 0x55f7ba8cd300 2023-09-13T03:13:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.223+0000 7fc24e9a7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7ba823ff0 con 0x55f7ba8cdcc0 2023-09-13T03:13:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.223+0000 7fc24e9a7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7ba81f8e0 con 0x55f7ba8d31a0 2023-09-13T03:13:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.223+0000 7fc231ffb700 1 --2- 172.21.15.64:0/3768192767 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fc2240a0840 0x7fc2240a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.223+0000 7fc238d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7ba8cd300 0x55f7ba8cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:52.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.223+0000 7fc231ffb700 1 -- 172.21.15.64:0/3768192767 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc228014070 con 0x7fc2340729a0 2023-09-13T03:13:52.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.223+0000 7fc23959c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f7ba8d31a0 0x55f7ba8d3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:52.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.223+0000 7fc238d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7ba8cd300 0x55f7ba8cd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48784/0 (socket says 172.21.15.64:48784) 2023-09-13T03:13:52.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.223+0000 7fc238d9b700 1 -- 172.21.15.64:0/3458889665 learned_addr learned my addr 172.21.15.64:0/3458889665 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:52.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.223+0000 7fc23959c700 1 --2- 172.21.15.64:0/3768192767 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fc2240a0840 0x7fc2240a2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.223+0000 7fc233fff700 1 --2- 172.21.15.64:0/3458889665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7ba8cdcc0 0x55f7ba8d9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.224+0000 7fc238d9b700 1 -- 172.21.15.64:0/3458889665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f7ba8d31a0 msgr2=0x55f7ba8d3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.224+0000 7fc238d9b700 1 --2- 172.21.15.64:0/3458889665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f7ba8d31a0 0x55f7ba8d3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.224+0000 7fc238d9b700 1 -- 172.21.15.64:0/3458889665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7ba8cdcc0 msgr2=0x55f7ba8d9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.224+0000 7fc238d9b700 1 --2- 172.21.15.64:0/3458889665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7ba8cdcc0 0x55f7ba8d9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.224+0000 7fc238d9b700 1 -- 172.21.15.64:0/3458889665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7ba64bd80 con 0x55f7ba8cd300 2023-09-13T03:13:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.224+0000 7fc238d9b700 1 --2- 172.21.15.64:0/3458889665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7ba8cd300 0x55f7ba8cd780 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc2280a1170 tx=0x7fc22801cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55749fbc6175003e server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.224+0000 7fc23959c700 1 --2- 172.21.15.64:0/3768192767 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fc2240a0840 0x7fc2240a2d00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc22c009790 tx=0x7fc22c009130 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.224+0000 7fc21effd700 1 -- 172.21.15.64:0/3458889665 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc228098f90 con 0x55f7ba8cd300 2023-09-13T03:13:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.224+0000 7fc21effd700 1 -- 172.21.15.64:0/3458889665 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc228099d30 con 0x55f7ba8cd300 2023-09-13T03:13:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.224+0000 7fc21effd700 1 -- 172.21.15.64:0/3458889665 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc228095d40 con 0x55f7ba8cd300 2023-09-13T03:13:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.225+0000 7fc24e9a7b80 1 -- 172.21.15.64:0/3458889665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7ba8cd300 msgr2=0x55f7ba8cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.225+0000 7fc24e9a7b80 1 --2- 172.21.15.64:0/3458889665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7ba8cd300 0x55f7ba8cd780 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc2280a1170 tx=0x7fc22801cdd0 comp rx=0 tx=0).stop 2023-09-13T03:13:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.225+0000 7fc24e9a7b80 1 -- 172.21.15.64:0/3458889665 shutdown_connections 2023-09-13T03:13:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.225+0000 7fc24e9a7b80 1 --2- 172.21.15.64:0/3458889665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7ba8cdcc0 0x55f7ba8d9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.225+0000 7fc24e9a7b80 1 --2- 172.21.15.64:0/3458889665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f7ba8d31a0 0x55f7ba8d3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.225+0000 7fc24e9a7b80 1 --2- 172.21.15.64:0/3458889665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7ba8cd300 0x55f7ba8cd780 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.225+0000 7fc24e9a7b80 1 -- 172.21.15.64:0/3458889665 >> 172.21.15.64:0/3458889665 conn(0x55f7ba842030 msgr2=0x55f7ba8c8560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.225+0000 7fc24e9a7b80 1 -- 172.21.15.64:0/3458889665 shutdown_connections 2023-09-13T03:13:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.225+0000 7fc24e9a7b80 1 -- 172.21.15.64:0/3458889665 wait complete. 2023-09-13T03:13:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.226+0000 7fc24e9a7b80 1 -- start start 2023-09-13T03:13:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.226+0000 7fc24e9a7b80 1 Processor -- start 2023-09-13T03:13:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.227+0000 7fc24e9a7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7ba8cd300 0x55f7ba8d9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.227+0000 7fc24e9a7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f7ba8cdcc0 0x55f7ba8d7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.227+0000 7fc24e9a7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7ba8d31a0 0x55f7ba8d75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.227+0000 7fc24e9a7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7ba7bdb10 con 0x55f7ba8cd300 2023-09-13T03:13:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.227+0000 7fc24e9a7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7ba7bd810 con 0x55f7ba8d31a0 2023-09-13T03:13:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.227+0000 7fc24e9a7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7ba6a18c0 con 0x55f7ba8cdcc0 2023-09-13T03:13:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.227+0000 7fc24e9a7b80 10 client.0 _ll_register_callbacks cb 0x55f7ba707a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:13:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.227+0000 7fc238d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7ba8cd300 0x55f7ba8d9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.227+0000 7fc238d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7ba8cd300 0x55f7ba8d9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48798/0 (socket says 172.21.15.64:48798) 2023-09-13T03:13:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.227+0000 7fc238d9b700 1 -- 172.21.15.64:0/1438192119 learned_addr learned my addr 172.21.15.64:0/1438192119 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.228+0000 7fc233fff700 1 --2- 172.21.15.64:0/1438192119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f7ba8cdcc0 0x55f7ba8d7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.228+0000 7fc23959c700 1 --2- 172.21.15.64:0/1438192119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7ba8d31a0 0x55f7ba8d75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.228+0000 7fc238d9b700 1 -- 172.21.15.64:0/1438192119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f7ba8cdcc0 msgr2=0x55f7ba8d7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.228+0000 7fc238d9b700 1 --2- 172.21.15.64:0/1438192119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f7ba8cdcc0 0x55f7ba8d7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.228+0000 7fc238d9b700 1 -- 172.21.15.64:0/1438192119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7ba8d31a0 msgr2=0x55f7ba8d75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.228+0000 7fc238d9b700 1 --2- 172.21.15.64:0/1438192119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7ba8d31a0 0x55f7ba8d75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.228+0000 7fc238d9b700 1 -- 172.21.15.64:0/1438192119 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2341a9c60 con 0x55f7ba8cd300 2023-09-13T03:13:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.228+0000 7fc233fff700 1 --2- 172.21.15.64:0/1438192119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f7ba8cdcc0 0x55f7ba8d7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:13:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.228+0000 7fc238d9b700 1 --2- 172.21.15.64:0/1438192119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7ba8cd300 0x55f7ba8d9d70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc228099330 tx=0x7fc228010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba521c47e037d11f server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.228+0000 7fc213fff700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:13:52.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.228+0000 7fc213fff700 1 -- 172.21.15.64:0/1438192119 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc228098f90 con 0x55f7ba8cd300 2023-09-13T03:13:52.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.228+0000 7fc213fff700 1 -- 172.21.15.64:0/1438192119 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc2280cb030 con 0x55f7ba8cd300 2023-09-13T03:13:52.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.228+0000 7fc24e9a7b80 1 -- 172.21.15.64:0/1438192119 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f7ba777490 con 0x55f7ba8cd300 2023-09-13T03:13:52.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.229+0000 7fc213fff700 1 -- 172.21.15.64:0/1438192119 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc228074060 con 0x55f7ba8cd300 2023-09-13T03:13:52.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.229+0000 7fc24e9a7b80 10 client.15870 fetch_fsmap learned FSMap version 26 2023-09-13T03:13:52.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.229+0000 7fc24e9a7b80 1 -- 172.21.15.64:0/1438192119 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55f7ba634c00 con 0x55f7ba8cd300 2023-09-13T03:13:52.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.229+0000 7fc213fff700 1 -- 172.21.15.64:0/1438192119 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f7ba777490 con 0x55f7ba8cd300 2023-09-13T03:13:52.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.229+0000 7fc213fff700 1 -- 172.21.15.64:0/1438192119 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc2280c9c30 con 0x55f7ba8cd300 2023-09-13T03:13:52.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.229+0000 7fc24e9a7b80 10 client.15870 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:13:52.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.229+0000 7fc24e9a7b80 10 client.15870 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:13:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.229+0000 7fc24e9a7b80 20 client.15870 populate_metadata read hostname 'smithi064' 2023-09-13T03:13:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.229+0000 7fc24e9a7b80 1 --2- 172.21.15.64:0/1438192119 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f7ba8e7030 0x55f7ba8e9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.229+0000 7fc24e9a7b80 4 client.15870 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:13:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.229+0000 7fc24e9a7b80 1 -- 172.21.15.64:0/1438192119 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f7ba834840 con 0x55f7ba8e7030 2023-09-13T03:13:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.230+0000 7fc233fff700 1 --2- 172.21.15.64:0/1438192119 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f7ba8e7030 0x55f7ba8e9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.230+0000 7fc213fff700 10 client.15870 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:13:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.230+0000 7fc233fff700 1 --2- 172.21.15.64:0/1438192119 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f7ba8e7030 0x55f7ba8e9420 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47b6a070aee5422d server_cookie=f10489d37ddadaff in_seq=0 out_seq=0 2023-09-13T03:13:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.231+0000 7fc213fff700 1 -- 172.21.15.64:0/1438192119 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f7ba834840 con 0x55f7ba8e7030 2023-09-13T03:13:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.231+0000 7fc213fff700 10 client.15870 handle_command_reply: tid=0 2023-09-13T03:13:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.231+0000 7fc24e9a7b80 1 client.15870 shutdown 2023-09-13T03:13:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.231+0000 7fc24e9a7b80 1 -- 172.21.15.64:0/1438192119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7ba8cd300 msgr2=0x55f7ba8d9d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.231+0000 7fc24e9a7b80 1 --2- 172.21.15.64:0/1438192119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7ba8cd300 0x55f7ba8d9d70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc228099330 tx=0x7fc228010c80 comp rx=0 tx=0).stop 2023-09-13T03:13:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.231+0000 7fc24e9a7b80 1 -- 172.21.15.64:0/1438192119 shutdown_connections 2023-09-13T03:13:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.231+0000 7fc24e9a7b80 1 --2- 172.21.15.64:0/1438192119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7ba8d31a0 0x55f7ba8d75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.231+0000 7fc24e9a7b80 1 --2- 172.21.15.64:0/1438192119 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f7ba8e7030 0x55f7ba8e9420 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.231+0000 7fc24e9a7b80 1 --2- 172.21.15.64:0/1438192119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7ba8cd300 0x55f7ba8d9d70 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.231+0000 7fc24e9a7b80 1 --2- 172.21.15.64:0/1438192119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f7ba8cdcc0 0x55f7ba8d7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.231+0000 7fc24e9a7b80 1 -- 172.21.15.64:0/1438192119 >> 172.21.15.64:0/1438192119 conn(0x55f7ba842030 msgr2=0x55f7ba8c77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.231+0000 7fc213fff700 0 client.15870 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:13:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.232+0000 7fc24e9a7b80 1 -- 172.21.15.64:0/1438192119 shutdown_connections 2023-09-13T03:13:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.232+0000 7fc24e9a7b80 1 -- 172.21.15.64:0/1438192119 wait complete. 2023-09-13T03:13:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.232+0000 7fc24e9a7b80 20 client.15870 trim_cache size 0 max 16384 2023-09-13T03:13:52.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.247+0000 7fc24e9a7b80 1 Processor -- start 2023-09-13T03:13:52.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.247+0000 7fc24e9a7b80 1 -- start start 2023-09-13T03:13:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.248+0000 7fc24e9a7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f7ba8cd300 0x55f7ba8d75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.248+0000 7fc24e9a7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7ba8cdcc0 0x55f7ba8cca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.248+0000 7fc24e9a7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7ba8d31a0 0x55f7ba8d9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.248+0000 7fc24e9a7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7ba8ce550 con 0x55f7ba8cdcc0 2023-09-13T03:13:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.248+0000 7fc24e9a7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7ba8e3c90 con 0x55f7ba8d31a0 2023-09-13T03:13:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.248+0000 7fc24e9a7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7ba8d1d40 con 0x55f7ba8cd300 2023-09-13T03:13:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.248+0000 7fc233fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7ba8cdcc0 0x55f7ba8cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.248+0000 7fc233fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7ba8cdcc0 0x55f7ba8cca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48814/0 (socket says 172.21.15.64:48814) 2023-09-13T03:13:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.248+0000 7fc233fff700 1 -- 172.21.15.64:0/308580569 learned_addr learned my addr 172.21.15.64:0/308580569 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.248+0000 7fc238d9b700 1 --2- 172.21.15.64:0/308580569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f7ba8cd300 0x55f7ba8d75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.248+0000 7fc23959c700 1 --2- 172.21.15.64:0/308580569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7ba8d31a0 0x55f7ba8d9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.249+0000 7fc23959c700 1 -- 172.21.15.64:0/308580569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f7ba8cd300 msgr2=0x55f7ba8d75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.249+0000 7fc23959c700 1 --2- 172.21.15.64:0/308580569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f7ba8cd300 0x55f7ba8d75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.249+0000 7fc23959c700 1 -- 172.21.15.64:0/308580569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7ba8cdcc0 msgr2=0x55f7ba8cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.249+0000 7fc23959c700 1 --2- 172.21.15.64:0/308580569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7ba8cdcc0 0x55f7ba8cca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.249+0000 7fc23959c700 1 -- 172.21.15.64:0/308580569 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7ba6721f0 con 0x55f7ba8d31a0 2023-09-13T03:13:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.249+0000 7fc233fff700 1 --2- 172.21.15.64:0/308580569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7ba8cdcc0 0x55f7ba8cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:13:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.249+0000 7fc23959c700 1 --2- 172.21.15.64:0/308580569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7ba8d31a0 0x55f7ba8d9910 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc22c009bf0 tx=0x7fc22c00b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d62dd81ededbc225 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.249+0000 7fc21f7fe700 1 -- 172.21.15.64:0/308580569 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc22c02b070 con 0x55f7ba8d31a0 2023-09-13T03:13:52.258 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.249+0000 7fc21f7fe700 1 -- 172.21.15.64:0/308580569 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc22c022040 con 0x55f7ba8d31a0 2023-09-13T03:13:52.258 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.249+0000 7fc21f7fe700 1 -- 172.21.15.64:0/308580569 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc22c021050 con 0x55f7ba8d31a0 2023-09-13T03:13:52.258 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.250+0000 7fc24e9a7b80 1 -- 172.21.15.64:0/308580569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7ba8d31a0 msgr2=0x55f7ba8d9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:52.258 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.250+0000 7fc24e9a7b80 1 --2- 172.21.15.64:0/308580569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7ba8d31a0 0x55f7ba8d9910 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc22c009bf0 tx=0x7fc22c00b040 comp rx=0 tx=0).stop 2023-09-13T03:13:52.258 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.250+0000 7fc24e9a7b80 1 -- 172.21.15.64:0/308580569 shutdown_connections 2023-09-13T03:13:52.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.250+0000 7fc24e9a7b80 1 --2- 172.21.15.64:0/308580569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7ba8d31a0 0x55f7ba8d9910 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:52.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.250+0000 7fc24e9a7b80 1 --2- 172.21.15.64:0/308580569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f7ba8cd300 0x55f7ba8d75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:52.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.250+0000 7fc24e9a7b80 1 --2- 172.21.15.64:0/308580569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7ba8cdcc0 0x55f7ba8cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:52.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.250+0000 7fc24e9a7b80 1 -- 172.21.15.64:0/308580569 >> 172.21.15.64:0/308580569 conn(0x55f7ba842030 msgr2=0x55f7ba8cc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:52.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.250+0000 7fc24e9a7b80 1 -- 172.21.15.64:0/308580569 shutdown_connections 2023-09-13T03:13:52.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.250+0000 7fc24e9a7b80 1 -- 172.21.15.64:0/308580569 wait complete. 2023-09-13T03:13:52.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.251+0000 7fc24e9a7b80 1 -- start start 2023-09-13T03:13:52.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.251+0000 7fc24e9a7b80 1 Processor -- start 2023-09-13T03:13:52.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.252+0000 7fc24e9a7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7ba8cd300 0x55f7ba8dbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:52.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.252+0000 7fc24e9a7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f7ba8cdcc0 0x55f7ba8dc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.252+0000 7fc24e9a7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7ba8d31a0 0x55f7ba8dc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.252+0000 7fc24e9a7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7ba939ff0 con 0x55f7ba8cd300 2023-09-13T03:13:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.252+0000 7fc24e9a7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7ba93a160 con 0x55f7ba8d31a0 2023-09-13T03:13:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.252+0000 7fc24e9a7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7ba93a320 con 0x55f7ba8cdcc0 2023-09-13T03:13:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.252+0000 7fc24e9a7b80 10 client.0 _ll_register_callbacks cb 0x7fc22c009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:13:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.252+0000 7fc238d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7ba8cd300 0x55f7ba8dbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.252+0000 7fc23959c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7ba8d31a0 0x55f7ba8dc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.252+0000 7fc238d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7ba8cd300 0x55f7ba8dbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48822/0 (socket says 172.21.15.64:48822) 2023-09-13T03:13:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.252+0000 7fc238d9b700 1 -- 172.21.15.64:0/3760589707 learned_addr learned my addr 172.21.15.64:0/3760589707 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:13:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.252+0000 7fc233fff700 1 --2- 172.21.15.64:0/3760589707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f7ba8cdcc0 0x55f7ba8dc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.252+0000 7fc23959c700 1 -- 172.21.15.64:0/3760589707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f7ba8cdcc0 msgr2=0x55f7ba8dc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.252+0000 7fc23959c700 1 --2- 172.21.15.64:0/3760589707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f7ba8cdcc0 0x55f7ba8dc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.253+0000 7fc23959c700 1 -- 172.21.15.64:0/3760589707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7ba8cd300 msgr2=0x55f7ba8dbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:13:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.253+0000 7fc23959c700 1 --2- 172.21.15.64:0/3760589707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7ba8cd300 0x55f7ba8dbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.253+0000 7fc23959c700 1 -- 172.21.15.64:0/3760589707 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc234114330 con 0x55f7ba8d31a0 2023-09-13T03:13:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.253+0000 7fc233fff700 1 --2- 172.21.15.64:0/3760589707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f7ba8cdcc0 0x55f7ba8dc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:13:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.253+0000 7fc238d9b700 1 --2- 172.21.15.64:0/3760589707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7ba8cd300 0x55f7ba8dbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:13:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.253+0000 7fc23959c700 1 --2- 172.21.15.64:0/3760589707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7ba8d31a0 0x55f7ba8dc8f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc22c007fd0 tx=0x7fc22c007500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd9d039cc83f8422 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:13:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.253+0000 7fc213fff700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:13:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.253+0000 7fc213fff700 1 -- 172.21.15.64:0/3760589707 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc22c02b040 con 0x55f7ba8d31a0 2023-09-13T03:13:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.253+0000 7fc24e9a7b80 1 -- 172.21.15.64:0/3760589707 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc228099d30 con 0x55f7ba8d31a0 2023-09-13T03:13:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.253+0000 7fc213fff700 1 -- 172.21.15.64:0/3760589707 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc22c003a00 con 0x55f7ba8d31a0 2023-09-13T03:13:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.253+0000 7fc213fff700 1 -- 172.21.15.64:0/3760589707 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc22c021030 con 0x55f7ba8d31a0 2023-09-13T03:13:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.254+0000 7fc24e9a7b80 10 client.25913 fetch_fsmap learned FSMap version 26 2023-09-13T03:13:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.254+0000 7fc24e9a7b80 1 -- 172.21.15.64:0/3760589707 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7fc22c011040 con 0x55f7ba8d31a0 2023-09-13T03:13:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.254+0000 7fc213fff700 1 -- 172.21.15.64:0/3760589707 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc228099d30 con 0x55f7ba8d31a0 2023-09-13T03:13:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.254+0000 7fc213fff700 1 -- 172.21.15.64:0/3760589707 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc22c004020 con 0x55f7ba8d31a0 2023-09-13T03:13:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.254+0000 7fc24e9a7b80 10 client.25913 fetch_fsmap finished waiting for FSMap version 26 2023-09-13T03:13:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.254+0000 7fc24e9a7b80 10 client.25913 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:13:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.254+0000 7fc24e9a7b80 20 client.25913 populate_metadata read hostname 'smithi064' 2023-09-13T03:13:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.254+0000 7fc24e9a7b80 1 --2- 172.21.15.64:0/3760589707 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f7ba93bab0 0x55f7ba93ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:13:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.254+0000 7fc24e9a7b80 4 client.25913 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:13:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.254+0000 7fc24e9a7b80 1 -- 172.21.15.64:0/3760589707 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f7ba8188c0 con 0x55f7ba93bab0 2023-09-13T03:13:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.255+0000 7fc238d9b700 1 --2- 172.21.15.64:0/3760589707 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f7ba93bab0 0x55f7ba93ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:13:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.255+0000 7fc238d9b700 1 --2- 172.21.15.64:0/3760589707 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f7ba93bab0 0x55f7ba93ded0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79648392066382f7 server_cookie=79b2e645398de5ab in_seq=0 out_seq=0 2023-09-13T03:13:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.255+0000 7fc213fff700 10 client.25913 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:13:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.263+0000 7fc213fff700 1 -- 172.21.15.64:0/3760589707 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f7ba8188c0 con 0x55f7ba93bab0 2023-09-13T03:13:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.263+0000 7fc213fff700 10 client.25913 handle_command_reply: tid=0 2023-09-13T03:13:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.263+0000 7fc24e9a7b80 1 client.25913 shutdown 2023-09-13T03:13:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.263+0000 7fc24e9a7b80 1 -- 172.21.15.64:0/3760589707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7ba8d31a0 msgr2=0x55f7ba8dc8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.263+0000 7fc24e9a7b80 1 --2- 172.21.15.64:0/3760589707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7ba8d31a0 0x55f7ba8dc8f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc22c007fd0 tx=0x7fc22c007500 comp rx=0 tx=0).stop 2023-09-13T03:13:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.263+0000 7fc24e9a7b80 1 -- 172.21.15.64:0/3760589707 shutdown_connections 2023-09-13T03:13:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.263+0000 7fc24e9a7b80 1 --2- 172.21.15.64:0/3760589707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7ba8d31a0 0x55f7ba8dc8f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.263+0000 7fc24e9a7b80 1 --2- 172.21.15.64:0/3760589707 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f7ba93bab0 0x55f7ba93ded0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.263+0000 7fc24e9a7b80 1 --2- 172.21.15.64:0/3760589707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f7ba8cd300 0x55f7ba8dbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.263+0000 7fc24e9a7b80 1 --2- 172.21.15.64:0/3760589707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f7ba8cdcc0 0x55f7ba8dc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.263+0000 7fc213fff700 0 client.25913 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:13:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.263+0000 7fc24e9a7b80 1 -- 172.21.15.64:0/3760589707 >> 172.21.15.64:0/3760589707 conn(0x55f7ba842030 msgr2=0x55f7ba8ccb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.263+0000 7fc24e9a7b80 1 -- 172.21.15.64:0/3760589707 shutdown_connections 2023-09-13T03:13:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.263+0000 7fc24e9a7b80 1 -- 172.21.15.64:0/3760589707 wait complete. 2023-09-13T03:13:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.263+0000 7fc24e9a7b80 20 client.25913 trim_cache size 0 max 16384 2023-09-13T03:13:52.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:13:52.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:13:52.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:13:52.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:13:52.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.264+0000 7fc23afff700 1 -- 172.21.15.64:0/3768192767 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fc2240a0840 msgr2=0x7fc2240a2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:52.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.264+0000 7fc23afff700 1 --2- 172.21.15.64:0/3768192767 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fc2240a0840 0x7fc2240a2d00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc22c009790 tx=0x7fc22c009130 comp rx=0 tx=0).stop 2023-09-13T03:13:52.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.264+0000 7fc23afff700 1 -- 172.21.15.64:0/3768192767 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2340729a0 msgr2=0x7fc2341a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:13:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.264+0000 7fc23afff700 1 --2- 172.21.15.64:0/3768192767 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2340729a0 0x7fc2341a2c70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc22800e9a0 tx=0x7fc22800ecb0 comp rx=0 tx=0).stop 2023-09-13T03:13:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.264+0000 7fc23afff700 1 -- 172.21.15.64:0/3768192767 shutdown_connections 2023-09-13T03:13:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.264+0000 7fc23afff700 1 --2- 172.21.15.64:0/3768192767 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fc2240a0840 0x7fc2240a2d00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.264+0000 7fc23afff700 1 --2- 172.21.15.64:0/3768192767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc234111590 0x7fc23419cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.264+0000 7fc23afff700 1 --2- 172.21.15.64:0/3768192767 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2340729a0 0x7fc2341a2c70 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.264+0000 7fc23afff700 1 --2- 172.21.15.64:0/3768192767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2340732c0 0x7fc2341a31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:13:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.264+0000 7fc23afff700 1 -- 172.21.15.64:0/3768192767 >> 172.21.15.64:0/3768192767 conn(0x7fc2340fc050 msgr2=0x7fc2340fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:13:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.264+0000 7fc23afff700 1 -- 172.21.15.64:0/3768192767 shutdown_connections 2023-09-13T03:13:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:13:52.264+0000 7fc23afff700 1 -- 172.21.15.64:0/3768192767 wait complete. 2023-09-13T03:13:52.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:52 smithi170 ceph-mon[134328]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2023-09-13T03:13:52.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:52 smithi064 ceph-mon[97675]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2023-09-13T03:13:52.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:52 smithi142 ceph-mon[134082]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2023-09-13T03:13:54.722 INFO:tasks.workunit.client.0.smithi064.stderr:Note: switching to '3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f'. 2023-09-13T03:13:54.722 INFO:tasks.workunit.client.0.smithi064.stderr: 2023-09-13T03:13:54.723 INFO:tasks.workunit.client.0.smithi064.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-09-13T03:13:54.723 INFO:tasks.workunit.client.0.smithi064.stderr:changes and commit them, and you can discard any commits you make in this 2023-09-13T03:13:54.723 INFO:tasks.workunit.client.0.smithi064.stderr:state without impacting any branches by switching back to a branch. 2023-09-13T03:13:54.723 INFO:tasks.workunit.client.0.smithi064.stderr: 2023-09-13T03:13:54.723 INFO:tasks.workunit.client.0.smithi064.stderr:If you want to create a new branch to retain commits you create, you may 2023-09-13T03:13:54.724 INFO:tasks.workunit.client.0.smithi064.stderr:do so (now or later) by using -c with the switch command. Example: 2023-09-13T03:13:54.724 INFO:tasks.workunit.client.0.smithi064.stderr: 2023-09-13T03:13:54.724 INFO:tasks.workunit.client.0.smithi064.stderr: git switch -c 2023-09-13T03:13:54.724 INFO:tasks.workunit.client.0.smithi064.stderr: 2023-09-13T03:13:54.724 INFO:tasks.workunit.client.0.smithi064.stderr:Or undo this operation with: 2023-09-13T03:13:54.725 INFO:tasks.workunit.client.0.smithi064.stderr: 2023-09-13T03:13:54.725 INFO:tasks.workunit.client.0.smithi064.stderr: git switch - 2023-09-13T03:13:54.725 INFO:tasks.workunit.client.0.smithi064.stderr: 2023-09-13T03:13:54.725 INFO:tasks.workunit.client.0.smithi064.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-09-13T03:13:54.725 INFO:tasks.workunit.client.0.smithi064.stderr: 2023-09-13T03:13:54.726 INFO:tasks.workunit.client.0.smithi064.stderr:HEAD is now at 3711e7dcbd7 qa: use exisitng ignorelist override list for fs:mirror[-ha] 2023-09-13T03:13:54.736 DEBUG:teuthology.orchestra.run.smithi064:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-09-13T03:13:54.797 INFO:tasks.workunit.client.0.smithi064.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-09-13T03:13:54.802 INFO:tasks.workunit.client.0.smithi064.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-09-13T03:13:54.802 INFO:tasks.workunit.client.0.smithi064.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-09-13T03:13:54.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:54 smithi170 ceph-mon[134328]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 8 op/s 2023-09-13T03:13:54.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:54 smithi064 ceph-mon[97675]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 8 op/s 2023-09-13T03:13:54.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:54 smithi142 ceph-mon[134082]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 8 op/s 2023-09-13T03:13:55.412 INFO:tasks.workunit.client.0.smithi064.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-09-13T03:13:55.494 INFO:tasks.workunit.client.0.smithi064.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-09-13T03:13:55.560 INFO:tasks.workunit.client.0.smithi064.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-09-13T03:13:55.563 INFO:tasks.workunit.client.0.smithi064.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-09-13T03:13:55.564 INFO:tasks.workunit.client.0.smithi064.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-09-13T03:13:55.627 INFO:tasks.workunit.client.0.smithi064.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-09-13T03:13:55.634 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-13T03:13:55.635 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-09-13T03:13:55.679 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2023-09-13T03:13:55.680 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2023-09-13T03:13:55.680 DEBUG:teuthology.orchestra.run.smithi064:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2023-09-13T03:13:55.777 INFO:tasks.workunit.client.0.smithi064.stderr:+ dbench 1 2023-09-13T03:13:55.781 INFO:tasks.workunit.client.0.smithi064.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2023-09-13T03:13:55.782 INFO:tasks.workunit.client.0.smithi064.stdout: 2023-09-13T03:13:55.782 INFO:tasks.workunit.client.0.smithi064.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2023-09-13T03:13:55.783 INFO:tasks.workunit.client.0.smithi064.stdout:0 of 1 processes prepared for launch 0 sec 2023-09-13T03:13:55.883 INFO:tasks.workunit.client.0.smithi064.stdout:1 of 1 processes prepared for launch 0 sec 2023-09-13T03:13:55.884 INFO:tasks.workunit.client.0.smithi064.stdout:releasing clients 2023-09-13T03:13:56.884 INFO:tasks.workunit.client.0.smithi064.stdout: 1 3352 35.37 MB/sec warmup 1 sec latency 6.085 ms 2023-09-13T03:13:56.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:56 smithi170 ceph-mon[134328]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-09-13T03:13:56.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:56 smithi064 ceph-mon[97675]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-09-13T03:13:56.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:56 smithi142 ceph-mon[134082]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-09-13T03:13:57.884 INFO:tasks.workunit.client.0.smithi064.stdout: 1 6154 24.06 MB/sec warmup 2 sec latency 6.814 ms 2023-09-13T03:13:58.884 INFO:tasks.workunit.client.0.smithi064.stdout: 1 8993 21.53 MB/sec warmup 3 sec latency 6.552 ms 2023-09-13T03:13:58.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:13:58 smithi170 ceph-mon[134328]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2023-09-13T03:13:58.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:13:58 smithi064 ceph-mon[97675]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2023-09-13T03:13:58.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:13:58 smithi142 ceph-mon[134082]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2023-09-13T03:13:59.884 INFO:tasks.workunit.client.0.smithi064.stdout: 1 11551 19.76 MB/sec warmup 4 sec latency 11.646 ms 2023-09-13T03:14:00.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:00 smithi142 ceph-mon[134082]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2023-09-13T03:14:00.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:00 smithi170 ceph-mon[134328]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2023-09-13T03:14:00.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:00 smithi064 ceph-mon[97675]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2023-09-13T03:14:00.668 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:14:00 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:14:00.178+0000 7fc91ddb9700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-13T03:14:00.669 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:14:00 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:14:00.178+0000 7fc91ddb9700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-13T03:14:00.669 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:14:00 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:14:00.178+0000 7fc91ddb9700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-13T03:14:00.669 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:14:00 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:14:00.178+0000 7fc91ddb9700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-13T03:14:00.669 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:14:00 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:14:00.178+0000 7fc91ddb9700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-13T03:14:00.884 INFO:tasks.workunit.client.0.smithi064.stdout: 1 13839 18.32 MB/sec warmup 5 sec latency 21.873 ms 2023-09-13T03:14:01.884 INFO:tasks.workunit.client.0.smithi064.stdout: 1 16089 17.14 MB/sec warmup 6 sec latency 8.242 ms 2023-09-13T03:14:02.884 INFO:tasks.workunit.client.0.smithi064.stdout: 1 18656 16.75 MB/sec warmup 7 sec latency 8.961 ms 2023-09-13T03:14:02.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:02 smithi170 ceph-mon[134328]: pgmap v29: 97 pgs: 97 active+clean; 8.0 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 687 KiB/s wr, 119 op/s 2023-09-13T03:14:02.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:02 smithi170 ceph-mon[134328]: mgrmap e25: x(active, since 52s), standbys: y, z 2023-09-13T03:14:02.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:02 smithi064 ceph-mon[97675]: pgmap v29: 97 pgs: 97 active+clean; 8.0 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 687 KiB/s wr, 119 op/s 2023-09-13T03:14:02.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:02 smithi064 ceph-mon[97675]: mgrmap e25: x(active, since 52s), standbys: y, z 2023-09-13T03:14:02.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:02 smithi142 ceph-mon[134082]: pgmap v29: 97 pgs: 97 active+clean; 8.0 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 687 KiB/s wr, 119 op/s 2023-09-13T03:14:02.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:02 smithi142 ceph-mon[134082]: mgrmap e25: x(active, since 52s), standbys: y, z 2023-09-13T03:14:03.884 INFO:tasks.workunit.client.0.smithi064.stdout: 1 21717 16.96 MB/sec warmup 8 sec latency 14.270 ms 2023-09-13T03:14:04.884 INFO:tasks.workunit.client.0.smithi064.stdout: 1 24675 16.68 MB/sec warmup 9 sec latency 12.854 ms 2023-09-13T03:14:04.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:04 smithi170 ceph-mon[134328]: pgmap v30: 97 pgs: 97 active+clean; 8.0 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 686 KiB/s wr, 114 op/s 2023-09-13T03:14:04.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:04 smithi064 ceph-mon[97675]: pgmap v30: 97 pgs: 97 active+clean; 8.0 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 686 KiB/s wr, 114 op/s 2023-09-13T03:14:04.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:04 smithi142 ceph-mon[134082]: pgmap v30: 97 pgs: 97 active+clean; 8.0 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 686 KiB/s wr, 114 op/s 2023-09-13T03:14:05.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.414+0000 7f54b7fff700 1 -- 172.21.15.64:0/1426881461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f54b010ce20 msgr2=0x7f54b010f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:05.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.414+0000 7f54b7fff700 1 --2- 172.21.15.64:0/1426881461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f54b010ce20 0x7f54b010f250 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f54a8009f90 tx=0x7f54a800b330 comp rx=0 tx=0).stop 2023-09-13T03:14:05.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.414+0000 7f54b7fff700 1 -- 172.21.15.64:0/1426881461 shutdown_connections 2023-09-13T03:14:05.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.414+0000 7f54b7fff700 1 --2- 172.21.15.64:0/1426881461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54b00729d0 0x7f54b0072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:05.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.414+0000 7f54b7fff700 1 --2- 172.21.15.64:0/1426881461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f54b010ce20 0x7f54b010f250 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:05.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.414+0000 7f54b7fff700 1 --2- 172.21.15.64:0/1426881461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54b00732f0 0x7f54b010c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:05.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.414+0000 7f54b7fff700 1 -- 172.21.15.64:0/1426881461 >> 172.21.15.64:0/1426881461 conn(0x7f54b00fc0b0 msgr2=0x7f54b00fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:05.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.414+0000 7f54b7fff700 1 -- 172.21.15.64:0/1426881461 shutdown_connections 2023-09-13T03:14:05.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.414+0000 7f54b7fff700 1 -- 172.21.15.64:0/1426881461 wait complete. 2023-09-13T03:14:05.418 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.415+0000 7f54b7fff700 1 Processor -- start 2023-09-13T03:14:05.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.415+0000 7f54b7fff700 1 -- start start 2023-09-13T03:14:05.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.416+0000 7f54b7fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54b00729d0 0x7f54b019e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:05.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.416+0000 7f54b7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54b00732f0 0x7f54b019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:05.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.416+0000 7f54b7fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f54b010ce20 0x7f54b01987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:05.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.416+0000 7f54b7fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b0198e60 con 0x7f54b00732f0 2023-09-13T03:14:05.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.416+0000 7f54b7fff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b0198fd0 con 0x7f54b00729d0 2023-09-13T03:14:05.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.416+0000 7f54b7fff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b0199190 con 0x7f54b010ce20 2023-09-13T03:14:05.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.416+0000 7f54b5d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54b00729d0 0x7f54b019e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:05.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.416+0000 7f54b5d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54b00729d0 0x7f54b019e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:34044/0 (socket says 172.21.15.64:34044) 2023-09-13T03:14:05.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.416+0000 7f54b5d9b700 1 -- 172.21.15.64:0/3734355763 learned_addr learned my addr 172.21.15.64:0/3734355763 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:05.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.416+0000 7f54b659c700 1 --2- 172.21.15.64:0/3734355763 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f54b010ce20 0x7f54b01987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:05.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.416+0000 7f54b559a700 1 --2- 172.21.15.64:0/3734355763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54b00732f0 0x7f54b019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:05.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.416+0000 7f54b5d9b700 1 -- 172.21.15.64:0/3734355763 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f54b010ce20 msgr2=0x7f54b01987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:05.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.416+0000 7f54b5d9b700 1 --2- 172.21.15.64:0/3734355763 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f54b010ce20 0x7f54b01987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:05.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.416+0000 7f54b5d9b700 1 -- 172.21.15.64:0/3734355763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54b00732f0 msgr2=0x7f54b019eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.416+0000 7f54b5d9b700 1 --2- 172.21.15.64:0/3734355763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54b00732f0 0x7f54b019eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.416+0000 7f54b5d9b700 1 -- 172.21.15.64:0/3734355763 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ac0095f0 con 0x7f54b00729d0 2023-09-13T03:14:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.417+0000 7f54b5d9b700 1 --2- 172.21.15.64:0/3734355763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54b00729d0 0x7f54b019e990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f54ac00ec10 tx=0x7f54ac00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.417+0000 7f54a6ffd700 1 -- 172.21.15.64:0/3734355763 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ac00cc90 con 0x7f54b00729d0 2023-09-13T03:14:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.417+0000 7f54a6ffd700 1 -- 172.21.15.64:0/3734355763 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54ac0044e0 con 0x7f54b00729d0 2023-09-13T03:14:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.417+0000 7f54b7fff700 1 -- 172.21.15.64:0/3734355763 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54a0009520 con 0x7f54b00729d0 2023-09-13T03:14:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.417+0000 7f54a6ffd700 1 -- 172.21.15.64:0/3734355763 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ac010430 con 0x7f54b00729d0 2023-09-13T03:14:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.417+0000 7f54b7fff700 1 -- 172.21.15.64:0/3734355763 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54a8009c40 con 0x7f54b00729d0 2023-09-13T03:14:05.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.420+0000 7f54a6ffd700 1 -- 172.21.15.64:0/3734355763 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 135492+0+0 (secure 0 0 0) 0x7f54ac003fb0 con 0x7f54b00729d0 2023-09-13T03:14:05.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.421+0000 7f54cb8d3b80 1 Processor -- start 2023-09-13T03:14:05.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.421+0000 7f54a6ffd700 1 --2- 172.21.15.64:0/3734355763 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f549c0a0950 0x7f549c0a2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.422+0000 7f54cb8d3b80 1 -- start start 2023-09-13T03:14:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.422+0000 7f54a6ffd700 1 -- 172.21.15.64:0/3734355763 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f54ac014070 con 0x7f54b00729d0 2023-09-13T03:14:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.422+0000 7f54b659c700 1 --2- 172.21.15.64:0/3734355763 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f549c0a0950 0x7f549c0a2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.422+0000 7f54cb8d3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6dd1131a0 0x55b6dd113580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.422+0000 7f54cb8d3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6dd10d300 0x55b6dd10d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.422+0000 7f54cb8d3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6dd10dcc0 0x55b6dd119f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.422+0000 7f54cb8d3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6dd0666d0 con 0x55b6dd10dcc0 2023-09-13T03:14:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.422+0000 7f54cb8d3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6dd063ff0 con 0x55b6dd1131a0 2023-09-13T03:14:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.422+0000 7f54cb8d3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6dd05f8e0 con 0x55b6dd10d300 2023-09-13T03:14:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.422+0000 7f54b5d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6dd1131a0 0x55b6dd113580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.422+0000 7f54b559a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6dd10d300 0x55b6dd10d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.422+0000 7f54b5d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6dd1131a0 0x55b6dd113580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:34046/0 (socket says 172.21.15.64:34046) 2023-09-13T03:14:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.422+0000 7f54b5d9b700 1 -- 172.21.15.64:0/1407870347 learned_addr learned my addr 172.21.15.64:0/1407870347 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.422+0000 7f54b659c700 1 --2- 172.21.15.64:0/1407870347 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6dd10dcc0 0x55b6dd119f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.423+0000 7f54b5d9b700 1 -- 172.21.15.64:0/1407870347 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6dd10d300 msgr2=0x55b6dd10d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.423+0000 7f54b5d9b700 1 --2- 172.21.15.64:0/1407870347 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6dd10d300 0x55b6dd10d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.423+0000 7f54b5d9b700 1 -- 172.21.15.64:0/1407870347 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6dd10dcc0 msgr2=0x55b6dd119f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.423+0000 7f54b5d9b700 1 --2- 172.21.15.64:0/1407870347 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6dd10dcc0 0x55b6dd119f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.423+0000 7f54b5d9b700 1 -- 172.21.15.64:0/1407870347 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6dce8bd80 con 0x55b6dd1131a0 2023-09-13T03:14:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.423+0000 7f54b659c700 1 --2- 172.21.15.64:0/1407870347 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6dd10dcc0 0x55b6dd119f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:14:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.423+0000 7f54b5d9b700 1 --2- 172.21.15.64:0/1407870347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6dd1131a0 0x55b6dd113580 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f54ac0a14d0 tx=0x7f54ac0994d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb5ee764b9db662d server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.423+0000 7f54b659c700 1 --2- 172.21.15.64:0/3734355763 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f549c0a0950 0x7f549c0a2e10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f54a8009c10 tx=0x7f54a8009ba0 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:05.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.423+0000 7f549bfff700 1 -- 172.21.15.64:0/1407870347 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ac0b9070 con 0x55b6dd1131a0 2023-09-13T03:14:05.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.423+0000 7f549bfff700 1 -- 172.21.15.64:0/1407870347 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54ac09a090 con 0x55b6dd1131a0 2023-09-13T03:14:05.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.423+0000 7f549bfff700 1 -- 172.21.15.64:0/1407870347 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ac0b7be0 con 0x55b6dd1131a0 2023-09-13T03:14:05.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.424+0000 7f54cb8d3b80 1 -- 172.21.15.64:0/1407870347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6dd1131a0 msgr2=0x55b6dd113580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:05.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.424+0000 7f54cb8d3b80 1 --2- 172.21.15.64:0/1407870347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6dd1131a0 0x55b6dd113580 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f54ac0a14d0 tx=0x7f54ac0994d0 comp rx=0 tx=0).stop 2023-09-13T03:14:05.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.424+0000 7f54cb8d3b80 1 -- 172.21.15.64:0/1407870347 shutdown_connections 2023-09-13T03:14:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.424+0000 7f54cb8d3b80 1 --2- 172.21.15.64:0/1407870347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6dd1131a0 0x55b6dd113580 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.424+0000 7f54cb8d3b80 1 --2- 172.21.15.64:0/1407870347 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6dd10dcc0 0x55b6dd119f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.424+0000 7f54cb8d3b80 1 --2- 172.21.15.64:0/1407870347 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6dd10d300 0x55b6dd10d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.424+0000 7f54cb8d3b80 1 -- 172.21.15.64:0/1407870347 >> 172.21.15.64:0/1407870347 conn(0x55b6dd082030 msgr2=0x55b6dd108560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.424+0000 7f54cb8d3b80 1 -- 172.21.15.64:0/1407870347 shutdown_connections 2023-09-13T03:14:05.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.424+0000 7f54cb8d3b80 1 -- 172.21.15.64:0/1407870347 wait complete. 2023-09-13T03:14:05.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.424+0000 7f54cb8d3b80 1 -- start start 2023-09-13T03:14:05.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.425+0000 7f54cb8d3b80 1 Processor -- start 2023-09-13T03:14:05.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.425+0000 7f54cb8d3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6dd10d300 0x55b6dd119ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:05.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.425+0000 7f54cb8d3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6dd10dcc0 0x55b6dd1171b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:05.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.425+0000 7f54cb8d3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6dd1131a0 0x55b6dd1178f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.425+0000 7f54cb8d3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6dcffdb10 con 0x55b6dd1131a0 2023-09-13T03:14:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.425+0000 7f54cb8d3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6dcffd810 con 0x55b6dd10dcc0 2023-09-13T03:14:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.425+0000 7f54cb8d3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6dcee18c0 con 0x55b6dd10d300 2023-09-13T03:14:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.425+0000 7f54cb8d3b80 10 client.0 _ll_register_callbacks cb 0x55b6dcf47a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:14:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.425+0000 7f54b5d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6dd10d300 0x55b6dd119ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.425+0000 7f54b559a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6dd10dcc0 0x55b6dd1171b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.425+0000 7f54b559a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6dd10dcc0 0x55b6dd1171b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:34056/0 (socket says 172.21.15.64:34056) 2023-09-13T03:14:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.425+0000 7f54b5d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6dd10d300 0x55b6dd119ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:57404/0 (socket says 172.21.15.64:57404) 2023-09-13T03:14:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.425+0000 7f54b559a700 1 -- 172.21.15.64:0/129452854 learned_addr learned my addr 172.21.15.64:0/129452854 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.425+0000 7f54b659c700 1 --2- 172.21.15.64:0/129452854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6dd1131a0 0x55b6dd1178f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.426+0000 7f54b559a700 1 -- 172.21.15.64:0/129452854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6dd10d300 msgr2=0x55b6dd119ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.426+0000 7f54b559a700 1 --2- 172.21.15.64:0/129452854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6dd10d300 0x55b6dd119ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.426+0000 7f54b559a700 1 -- 172.21.15.64:0/129452854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6dd1131a0 msgr2=0x55b6dd1178f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.426+0000 7f54b559a700 1 --2- 172.21.15.64:0/129452854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6dd1131a0 0x55b6dd1178f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.426+0000 7f54b559a700 1 -- 172.21.15.64:0/129452854 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54a8009c40 con 0x55b6dd10dcc0 2023-09-13T03:14:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.426+0000 7f54b659c700 1 --2- 172.21.15.64:0/129452854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6dd1131a0 0x55b6dd1178f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:14:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.426+0000 7f54b559a700 1 --2- 172.21.15.64:0/129452854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6dd10dcc0 0x55b6dd1171b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f54a0009fd0 tx=0x7f54a0012af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48d58ae4d3ce64db server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.426+0000 7f54997fa700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:14:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.426+0000 7f54997fa700 1 -- 172.21.15.64:0/129452854 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a0017070 con 0x55b6dd10dcc0 2023-09-13T03:14:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.426+0000 7f54cb8d3b80 1 -- 172.21.15.64:0/129452854 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b6dcfb7490 con 0x55b6dd10dcc0 2023-09-13T03:14:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.426+0000 7f54997fa700 1 -- 172.21.15.64:0/129452854 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f54a0004030 con 0x55b6dd10dcc0 2023-09-13T03:14:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.427+0000 7f54997fa700 1 -- 172.21.15.64:0/129452854 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54a00041f0 con 0x55b6dd10dcc0 2023-09-13T03:14:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.427+0000 7f54cb8d3b80 10 client.25931 fetch_fsmap learned FSMap version 27 2023-09-13T03:14:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.427+0000 7f54cb8d3b80 1 -- 172.21.15.64:0/129452854 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b6dce74c00 con 0x55b6dd10dcc0 2023-09-13T03:14:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.427+0000 7f54997fa700 1 -- 172.21.15.64:0/129452854 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b6dcfb7490 con 0x55b6dd10dcc0 2023-09-13T03:14:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.427+0000 7f54997fa700 1 -- 172.21.15.64:0/129452854 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f54a000f560 con 0x55b6dd10dcc0 2023-09-13T03:14:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.427+0000 7f54cb8d3b80 10 client.25931 fetch_fsmap finished waiting for FSMap version 27 2023-09-13T03:14:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.427+0000 7f54cb8d3b80 10 client.25931 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:14:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.427+0000 7f54cb8d3b80 20 client.25931 populate_metadata read hostname 'smithi064' 2023-09-13T03:14:05.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.427+0000 7f54cb8d3b80 1 --2- 172.21.15.64:0/129452854 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55b6dd1274c0 0x55b6dd1298e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:05.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.427+0000 7f54cb8d3b80 4 client.25931 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:14:05.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.427+0000 7f54cb8d3b80 1 -- 172.21.15.64:0/129452854 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b6dd074840 con 0x55b6dd1274c0 2023-09-13T03:14:05.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.427+0000 7f54b5d9b700 1 --2- 172.21.15.64:0/129452854 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55b6dd1274c0 0x55b6dd1298e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.428+0000 7f54b5d9b700 1 --2- 172.21.15.64:0/129452854 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55b6dd1274c0 0x55b6dd1298e0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cbeb4e925b59033 server_cookie=9521ab37d65b8ab4 in_seq=0 out_seq=0 2023-09-13T03:14:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.428+0000 7f54997fa700 10 client.25931 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.428+0000 7f54997fa700 1 -- 172.21.15.64:0/129452854 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b6dd074840 con 0x55b6dd1274c0 2023-09-13T03:14:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.428+0000 7f54997fa700 10 client.25931 handle_command_reply: tid=0 2023-09-13T03:14:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.429+0000 7f54cb8d3b80 1 client.25931 shutdown 2023-09-13T03:14:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.429+0000 7f54cb8d3b80 1 -- 172.21.15.64:0/129452854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6dd10dcc0 msgr2=0x55b6dd1171b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.429+0000 7f54cb8d3b80 1 --2- 172.21.15.64:0/129452854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6dd10dcc0 0x55b6dd1171b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f54a0009fd0 tx=0x7f54a0012af0 comp rx=0 tx=0).stop 2023-09-13T03:14:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.429+0000 7f54cb8d3b80 1 -- 172.21.15.64:0/129452854 shutdown_connections 2023-09-13T03:14:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.429+0000 7f54cb8d3b80 1 --2- 172.21.15.64:0/129452854 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55b6dd1274c0 0x55b6dd1298e0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.429+0000 7f54cb8d3b80 1 --2- 172.21.15.64:0/129452854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6dd1131a0 0x55b6dd1178f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.429+0000 7f54997fa700 0 client.25931 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.429+0000 7f54cb8d3b80 1 --2- 172.21.15.64:0/129452854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6dd10d300 0x55b6dd119ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.429+0000 7f54cb8d3b80 1 --2- 172.21.15.64:0/129452854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6dd10dcc0 0x55b6dd1171b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.429+0000 7f54cb8d3b80 1 -- 172.21.15.64:0/129452854 >> 172.21.15.64:0/129452854 conn(0x55b6dd082030 msgr2=0x55b6dd1077a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.429+0000 7f54cb8d3b80 1 -- 172.21.15.64:0/129452854 shutdown_connections 2023-09-13T03:14:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.429+0000 7f54cb8d3b80 1 -- 172.21.15.64:0/129452854 wait complete. 2023-09-13T03:14:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.429+0000 7f54cb8d3b80 20 client.25931 trim_cache size 0 max 16384 2023-09-13T03:14:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.444+0000 7f54cb8d3b80 1 Processor -- start 2023-09-13T03:14:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.444+0000 7f54cb8d3b80 1 -- start start 2023-09-13T03:14:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.445+0000 7f54cb8d3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6dd10d300 0x55b6dd114df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.445+0000 7f54cb8d3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6dd10dcc0 0x55b6dd11e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.445+0000 7f54cb8d3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6dd1131a0 0x55b6dd10fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.445+0000 7f54cb8d3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6dd109d10 con 0x55b6dd1131a0 2023-09-13T03:14:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.445+0000 7f54cb8d3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6dd10e580 con 0x55b6dd10d300 2023-09-13T03:14:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.445+0000 7f54cb8d3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6dd104ff0 con 0x55b6dd10dcc0 2023-09-13T03:14:05.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.445+0000 7f54b5d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6dd10d300 0x55b6dd114df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:05.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.445+0000 7f54b5d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6dd10d300 0x55b6dd114df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:34070/0 (socket says 172.21.15.64:34070) 2023-09-13T03:14:05.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.445+0000 7f54b5d9b700 1 -- 172.21.15.64:0/1883905991 learned_addr learned my addr 172.21.15.64:0/1883905991 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:05.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.445+0000 7f54b559a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6dd10dcc0 0x55b6dd11e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:05.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.445+0000 7f54b659c700 1 --2- 172.21.15.64:0/1883905991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6dd1131a0 0x55b6dd10fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:05.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.445+0000 7f54b5d9b700 1 -- 172.21.15.64:0/1883905991 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6dd10dcc0 msgr2=0x55b6dd11e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:05.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.445+0000 7f54b5d9b700 1 --2- 172.21.15.64:0/1883905991 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6dd10dcc0 0x55b6dd11e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:05.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.445+0000 7f54b5d9b700 1 -- 172.21.15.64:0/1883905991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6dd1131a0 msgr2=0x55b6dd10fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:05.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.445+0000 7f54b5d9b700 1 --2- 172.21.15.64:0/1883905991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6dd1131a0 0x55b6dd10fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:05.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.445+0000 7f54b5d9b700 1 -- 172.21.15.64:0/1883905991 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6dceb21f0 con 0x55b6dd10d300 2023-09-13T03:14:05.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.445+0000 7f54b559a700 1 --2- 172.21.15.64:0/1883905991 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6dd10dcc0 0x55b6dd11e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.446+0000 7f54b5d9b700 1 --2- 172.21.15.64:0/1883905991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6dd10d300 0x55b6dd114df0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f54ac099690 tx=0x7f54ac003950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad5f269e4e470458 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.446+0000 7f54a4ff9700 1 -- 172.21.15.64:0/1883905991 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ac0a1a00 con 0x55b6dd10d300 2023-09-13T03:14:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.446+0000 7f54a4ff9700 1 -- 172.21.15.64:0/1883905991 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54ac09d7c0 con 0x55b6dd10d300 2023-09-13T03:14:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.446+0000 7f54a4ff9700 1 -- 172.21.15.64:0/1883905991 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ac099bf0 con 0x55b6dd10d300 2023-09-13T03:14:05.455 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.447+0000 7f54cb8d3b80 1 -- 172.21.15.64:0/1883905991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6dd10d300 msgr2=0x55b6dd114df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:05.455 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.447+0000 7f54cb8d3b80 1 --2- 172.21.15.64:0/1883905991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6dd10d300 0x55b6dd114df0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f54ac099690 tx=0x7f54ac003950 comp rx=0 tx=0).stop 2023-09-13T03:14:05.455 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.447+0000 7f54cb8d3b80 1 -- 172.21.15.64:0/1883905991 shutdown_connections 2023-09-13T03:14:05.455 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.447+0000 7f54cb8d3b80 1 --2- 172.21.15.64:0/1883905991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6dd10d300 0x55b6dd114df0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:05.455 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.447+0000 7f54cb8d3b80 1 --2- 172.21.15.64:0/1883905991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6dd1131a0 0x55b6dd10fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:05.455 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.447+0000 7f54cb8d3b80 1 --2- 172.21.15.64:0/1883905991 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6dd10dcc0 0x55b6dd11e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:05.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.447+0000 7f54cb8d3b80 1 -- 172.21.15.64:0/1883905991 >> 172.21.15.64:0/1883905991 conn(0x55b6dd082030 msgr2=0x55b6dd112130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:05.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.447+0000 7f54cb8d3b80 1 -- 172.21.15.64:0/1883905991 shutdown_connections 2023-09-13T03:14:05.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.447+0000 7f54cb8d3b80 1 -- 172.21.15.64:0/1883905991 wait complete. 2023-09-13T03:14:05.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.447+0000 7f54cb8d3b80 1 -- start start 2023-09-13T03:14:05.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.447+0000 7f54cb8d3b80 1 Processor -- start 2023-09-13T03:14:05.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.448+0000 7f54cb8d3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6dd10d300 0x55b6dd106f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:05.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.448+0000 7f54cb8d3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6dd10dcc0 0x55b6dd107450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:05.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.448+0000 7f54cb8d3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6dd1131a0 0x55b6dd107990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:05.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.448+0000 7f54cb8d3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6dd107ed0 con 0x55b6dd1131a0 2023-09-13T03:14:05.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.448+0000 7f54cb8d3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6dd108040 con 0x55b6dd10d300 2023-09-13T03:14:05.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.448+0000 7f54cb8d3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6dd108200 con 0x55b6dd10dcc0 2023-09-13T03:14:05.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.448+0000 7f54cb8d3b80 10 client.0 _ll_register_callbacks cb 0x7f54a0009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:14:05.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.448+0000 7f54b5d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6dd10d300 0x55b6dd106f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:05.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.448+0000 7f54b559a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6dd10dcc0 0x55b6dd107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:05.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.448+0000 7f54b5d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6dd10d300 0x55b6dd106f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:34072/0 (socket says 172.21.15.64:34072) 2023-09-13T03:14:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.448+0000 7f54b5d9b700 1 -- 172.21.15.64:0/3251849593 learned_addr learned my addr 172.21.15.64:0/3251849593 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.448+0000 7f54b659c700 1 --2- 172.21.15.64:0/3251849593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6dd1131a0 0x55b6dd107990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.448+0000 7f54b5d9b700 1 -- 172.21.15.64:0/3251849593 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6dd10dcc0 msgr2=0x55b6dd107450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.448+0000 7f54b5d9b700 1 --2- 172.21.15.64:0/3251849593 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6dd10dcc0 0x55b6dd107450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.448+0000 7f54b5d9b700 1 -- 172.21.15.64:0/3251849593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6dd1131a0 msgr2=0x55b6dd107990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.449+0000 7f54b5d9b700 1 --2- 172.21.15.64:0/3251849593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6dd1131a0 0x55b6dd107990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.449+0000 7f54b5d9b700 1 -- 172.21.15.64:0/3251849593 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b6dce74c00 con 0x55b6dd10d300 2023-09-13T03:14:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.449+0000 7f54b559a700 1 --2- 172.21.15.64:0/3251849593 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6dd10dcc0 0x55b6dd107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.449+0000 7f54b5d9b700 1 --2- 172.21.15.64:0/3251849593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6dd10d300 0x55b6dd106f10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f54ac0039c0 tx=0x7f54ac005810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8d2b36d3ebcb785 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.449+0000 7f54997fa700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:14:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.449+0000 7f54997fa700 1 -- 172.21.15.64:0/3251849593 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ac0b9050 con 0x55b6dd10d300 2023-09-13T03:14:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.449+0000 7f54997fa700 1 -- 172.21.15.64:0/3251849593 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f54ac084040 con 0x55b6dd10d300 2023-09-13T03:14:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.449+0000 7f54cb8d3b80 1 -- 172.21.15.64:0/3251849593 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f54ac09a090 con 0x55b6dd10d300 2023-09-13T03:14:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.449+0000 7f54997fa700 1 -- 172.21.15.64:0/3251849593 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54ac0b7be0 con 0x55b6dd10d300 2023-09-13T03:14:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.449+0000 7f54cb8d3b80 10 client.25937 fetch_fsmap learned FSMap version 27 2023-09-13T03:14:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.449+0000 7f54cb8d3b80 1 -- 172.21.15.64:0/3251849593 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b6dce8bd80 con 0x55b6dd10d300 2023-09-13T03:14:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.449+0000 7f54997fa700 1 -- 172.21.15.64:0/3251849593 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54ac09a090 con 0x55b6dd10d300 2023-09-13T03:14:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.450+0000 7f54997fa700 1 -- 172.21.15.64:0/3251849593 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f54ac0ae030 con 0x55b6dd10d300 2023-09-13T03:14:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.450+0000 7f54cb8d3b80 10 client.25937 fetch_fsmap finished waiting for FSMap version 27 2023-09-13T03:14:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.450+0000 7f54cb8d3b80 10 client.25937 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:14:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.450+0000 7f54cb8d3b80 20 client.25937 populate_metadata read hostname 'smithi064' 2023-09-13T03:14:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.450+0000 7f54cb8d3b80 1 --2- 172.21.15.64:0/3251849593 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55b6dd17bab0 0x55b6dd17dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.450+0000 7f54cb8d3b80 4 client.25937 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:14:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.450+0000 7f54cb8d3b80 1 -- 172.21.15.64:0/3251849593 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b6dd0588c0 con 0x55b6dd17bab0 2023-09-13T03:14:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.450+0000 7f54b559a700 1 --2- 172.21.15.64:0/3251849593 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55b6dd17bab0 0x55b6dd17dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:05.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.450+0000 7f54b559a700 1 --2- 172.21.15.64:0/3251849593 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55b6dd17bab0 0x55b6dd17dea0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bda9056b50a4634f server_cookie=b057cc5ae6f66561 in_seq=0 out_seq=0 2023-09-13T03:14:05.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.450+0000 7f54997fa700 10 client.25937 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.545+0000 7f54997fa700 1 -- 172.21.15.64:0/3251849593 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b6dd0588c0 con 0x55b6dd17bab0 2023-09-13T03:14:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.545+0000 7f54997fa700 10 client.25937 handle_command_reply: tid=0 2023-09-13T03:14:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.545+0000 7f54cb8d3b80 1 client.25937 shutdown 2023-09-13T03:14:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.545+0000 7f54cb8d3b80 1 -- 172.21.15.64:0/3251849593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6dd10d300 msgr2=0x55b6dd106f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.545+0000 7f54cb8d3b80 1 --2- 172.21.15.64:0/3251849593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6dd10d300 0x55b6dd106f10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f54ac0039c0 tx=0x7f54ac005810 comp rx=0 tx=0).stop 2023-09-13T03:14:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.545+0000 7f54cb8d3b80 1 -- 172.21.15.64:0/3251849593 shutdown_connections 2023-09-13T03:14:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.545+0000 7f54cb8d3b80 1 --2- 172.21.15.64:0/3251849593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6dd10d300 0x55b6dd106f10 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.545+0000 7f54cb8d3b80 1 --2- 172.21.15.64:0/3251849593 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55b6dd17bab0 0x55b6dd17dea0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.545+0000 7f54cb8d3b80 1 --2- 172.21.15.64:0/3251849593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6dd1131a0 0x55b6dd107990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.545+0000 7f54cb8d3b80 1 --2- 172.21.15.64:0/3251849593 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6dd10dcc0 0x55b6dd107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.545+0000 7f54997fa700 0 client.25937 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.545+0000 7f54cb8d3b80 1 -- 172.21.15.64:0/3251849593 >> 172.21.15.64:0/3251849593 conn(0x55b6dd082030 msgr2=0x55b6dd111190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.546+0000 7f54cb8d3b80 1 -- 172.21.15.64:0/3251849593 shutdown_connections 2023-09-13T03:14:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.546+0000 7f54cb8d3b80 1 -- 172.21.15.64:0/3251849593 wait complete. 2023-09-13T03:14:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.546+0000 7f54cb8d3b80 20 client.25937 trim_cache size 0 max 16384 2023-09-13T03:14:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:14:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:14:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:14:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:14:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.546+0000 7f54b7fff700 1 -- 172.21.15.64:0/3734355763 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f549c0a0950 msgr2=0x7f549c0a2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.546+0000 7f54b7fff700 1 --2- 172.21.15.64:0/3734355763 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f549c0a0950 0x7f549c0a2e10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f54a8009c10 tx=0x7f54a8009ba0 comp rx=0 tx=0).stop 2023-09-13T03:14:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.546+0000 7f54b7fff700 1 -- 172.21.15.64:0/3734355763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54b00729d0 msgr2=0x7f54b019e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:05.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.546+0000 7f54b7fff700 1 --2- 172.21.15.64:0/3734355763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54b00729d0 0x7f54b019e990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f54ac00ec10 tx=0x7f54ac00ef20 comp rx=0 tx=0).stop 2023-09-13T03:14:05.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.546+0000 7f54b7fff700 1 -- 172.21.15.64:0/3734355763 shutdown_connections 2023-09-13T03:14:05.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.546+0000 7f54b7fff700 1 --2- 172.21.15.64:0/3734355763 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f549c0a0950 0x7f549c0a2e10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:05.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.546+0000 7f54b7fff700 1 --2- 172.21.15.64:0/3734355763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54b00729d0 0x7f54b019e990 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.546+0000 7f54b7fff700 1 --2- 172.21.15.64:0/3734355763 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f54b010ce20 0x7f54b01987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.546+0000 7f54b7fff700 1 --2- 172.21.15.64:0/3734355763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54b00732f0 0x7f54b019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.547+0000 7f54b7fff700 1 -- 172.21.15.64:0/3734355763 >> 172.21.15.64:0/3734355763 conn(0x7f54b00fc0b0 msgr2=0x7f54b0104880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.547+0000 7f54b7fff700 1 -- 172.21.15.64:0/3734355763 shutdown_connections 2023-09-13T03:14:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:05.547+0000 7f54b7fff700 1 -- 172.21.15.64:0/3734355763 wait complete. 2023-09-13T03:14:05.885 INFO:tasks.workunit.client.0.smithi064.stdout: 1 25247 15.48 MB/sec warmup 10 sec latency 308.571 ms 2023-09-13T03:14:05.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:05 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:14:05.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:05 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:14:05.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:05 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:14:06.885 INFO:tasks.workunit.client.0.smithi064.stdout: 1 25247 14.08 MB/sec warmup 11 sec latency 1308.723 ms 2023-09-13T03:14:06.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:06 smithi170 ceph-mon[134328]: pgmap v31: 97 pgs: 97 active+clean; 12 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 1.1 MiB/s wr, 200 op/s 2023-09-13T03:14:06.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:06 smithi064 ceph-mon[97675]: pgmap v31: 97 pgs: 97 active+clean; 12 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 1.1 MiB/s wr, 200 op/s 2023-09-13T03:14:06.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:06 smithi142 ceph-mon[134082]: pgmap v31: 97 pgs: 97 active+clean; 12 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 1.1 MiB/s wr, 200 op/s 2023-09-13T03:14:07.885 INFO:tasks.workunit.client.0.smithi064.stdout: 1 25247 12.90 MB/sec warmup 12 sec latency 2308.795 ms 2023-09-13T03:14:08.533 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-13T03:14:08.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.754+0000 7f6481bff700 1 -- 172.21.15.64:0/3527059582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f647c106650 msgr2=0x7f647c106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:08.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.754+0000 7f6481bff700 1 --2- 172.21.15.64:0/3527059582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f647c106650 0x7f647c106a30 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f646c0099c0 tx=0x7f646c009cd0 comp rx=0 tx=0).stop 2023-09-13T03:14:08.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.754+0000 7f6481bff700 1 -- 172.21.15.64:0/3527059582 shutdown_connections 2023-09-13T03:14:08.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.754+0000 7f6481bff700 1 --2- 172.21.15.64:0/3527059582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f647c100f50 0x7f647c10d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:08.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.754+0000 7f6481bff700 1 --2- 172.21.15.64:0/3527059582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f647c106650 0x7f647c106a30 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:08.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.754+0000 7f6481bff700 1 --2- 172.21.15.64:0/3527059582 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f647c100590 0x7f647c100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:08.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.754+0000 7f6481bff700 1 -- 172.21.15.64:0/3527059582 >> 172.21.15.64:0/3527059582 conn(0x7f647c0fc060 msgr2=0x7f647c0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:08.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.754+0000 7f6481bff700 1 -- 172.21.15.64:0/3527059582 shutdown_connections 2023-09-13T03:14:08.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.754+0000 7f6481bff700 1 -- 172.21.15.64:0/3527059582 wait complete. 2023-09-13T03:14:08.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.755+0000 7f6481bff700 1 Processor -- start 2023-09-13T03:14:08.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.755+0000 7f6481bff700 1 -- start start 2023-09-13T03:14:08.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.756+0000 7f6481bff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f647c100590 0x7f647c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:08.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.756+0000 7f6481bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f647c100f50 0x7f647c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:08.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.756+0000 7f6481bff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f647c106650 0x7f647c073380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:08.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.756+0000 7f6481bff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f647c1a4a30 con 0x7f647c106650 2023-09-13T03:14:08.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.756+0000 7f6481bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f647c1a4ba0 con 0x7f647c100f50 2023-09-13T03:14:08.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.756+0000 7f6481bff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f647c1a4d60 con 0x7f647c100590 2023-09-13T03:14:08.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.756+0000 7f647b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f647c100590 0x7f647c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:08.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.756+0000 7f647b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f647c100590 0x7f647c072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:36760/0 (socket says 172.21.15.64:36760) 2023-09-13T03:14:08.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.756+0000 7f647b7fe700 1 -- 172.21.15.64:0/3369298688 learned_addr learned my addr 172.21.15.64:0/3369298688 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:08.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.756+0000 7f6473fff700 1 --2- 172.21.15.64:0/3369298688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f647c100f50 0x7f647c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:08.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.756+0000 7f647bfff700 1 --2- 172.21.15.64:0/3369298688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f647c106650 0x7f647c073380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:08.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.756+0000 7f647b7fe700 1 -- 172.21.15.64:0/3369298688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f647c100f50 msgr2=0x7f647c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:08.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.757+0000 7f647b7fe700 1 --2- 172.21.15.64:0/3369298688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f647c100f50 0x7f647c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:08.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.757+0000 7f647b7fe700 1 -- 172.21.15.64:0/3369298688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f647c106650 msgr2=0x7f647c073380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:08.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.757+0000 7f647b7fe700 1 --2- 172.21.15.64:0/3369298688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f647c106650 0x7f647c073380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:08.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.757+0000 7f647b7fe700 1 -- 172.21.15.64:0/3369298688 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f646c0095f0 con 0x7f647c100590 2023-09-13T03:14:08.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.757+0000 7f6473fff700 1 --2- 172.21.15.64:0/3369298688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f647c100f50 0x7f647c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:08.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.757+0000 7f647b7fe700 1 --2- 172.21.15.64:0/3369298688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f647c100590 0x7f647c072900 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f646c00b580 tx=0x7f646c00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:08.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.757+0000 7f64797fa700 1 -- 172.21.15.64:0/3369298688 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646c01d070 con 0x7f647c100590 2023-09-13T03:14:08.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.757+0000 7f6481bff700 1 -- 172.21.15.64:0/3369298688 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f647c1a4fe0 con 0x7f647c100590 2023-09-13T03:14:08.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.757+0000 7f64797fa700 1 -- 172.21.15.64:0/3369298688 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f646c005050 con 0x7f647c100590 2023-09-13T03:14:08.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.757+0000 7f64797fa700 1 -- 172.21.15.64:0/3369298688 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646c004a40 con 0x7f647c100590 2023-09-13T03:14:08.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.757+0000 7f6481bff700 1 -- 172.21.15.64:0/3369298688 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f647c1a5400 con 0x7f647c100590 2023-09-13T03:14:08.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.761+0000 7f64797fa700 1 -- 172.21.15.64:0/3369298688 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) v1 ==== 135492+0+0 (secure 0 0 0) 0x7f646c005430 con 0x7f647c100590 2023-09-13T03:14:08.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.762+0000 7f6495554b80 1 Processor -- start 2023-09-13T03:14:08.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.762+0000 7f6495554b80 1 -- start start 2023-09-13T03:14:08.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.762+0000 7f6495554b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56302b607710 0x56302b607af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:08.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.762+0000 7f6495554b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56302b608030 0x56302b611900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:08.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.762+0000 7f6495554b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56302b611e40 0x56302b614230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:08.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.762+0000 7f6495554b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56302b560730 con 0x56302b607710 2023-09-13T03:14:08.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.762+0000 7f6495554b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56302b55e050 con 0x56302b611e40 2023-09-13T03:14:08.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.762+0000 7f6495554b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56302b559940 con 0x56302b608030 2023-09-13T03:14:08.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.762+0000 7f64797fa700 1 --2- 172.21.15.64:0/3369298688 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f64680a08d0 0x7f64680a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:08.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.763+0000 7f647b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56302b608030 0x56302b611900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:08.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.763+0000 7f6473fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56302b611e40 0x56302b614230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:08.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.763+0000 7f647bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56302b607710 0x56302b607af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:08.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.763+0000 7f6473fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56302b611e40 0x56302b614230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:53544/0 (socket says 172.21.15.64:53544) 2023-09-13T03:14:08.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.763+0000 7f6473fff700 1 -- 172.21.15.64:0/521856883 learned_addr learned my addr 172.21.15.64:0/521856883 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:08.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.763+0000 7f64797fa700 1 -- 172.21.15.64:0/3369298688 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f646c0ccfe0 con 0x7f647c100590 2023-09-13T03:14:08.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.763+0000 7f647bfff700 1 --2- 172.21.15.64:0/3369298688 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f64680a08d0 0x7f64680a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:08.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.763+0000 7f6473fff700 1 -- 172.21.15.64:0/521856883 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56302b608030 msgr2=0x56302b611900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:08.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.763+0000 7f6473fff700 1 --2- 172.21.15.64:0/521856883 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56302b608030 0x56302b611900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:08.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.763+0000 7f6473fff700 1 -- 172.21.15.64:0/521856883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56302b607710 msgr2=0x56302b607af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:08.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.763+0000 7f6473fff700 1 --2- 172.21.15.64:0/521856883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56302b607710 0x56302b607af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:08.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.763+0000 7f6473fff700 1 -- 172.21.15.64:0/521856883 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56302b385de0 con 0x56302b611e40 2023-09-13T03:14:08.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.763+0000 7f6473fff700 1 --2- 172.21.15.64:0/521856883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56302b611e40 0x56302b614230 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6464010580 tx=0x7f6464010890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb5e6e0b5b9f7564 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:08.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.763+0000 7f647bfff700 1 --2- 172.21.15.64:0/3369298688 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f64680a08d0 0x7f64680a2d90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6474009ec0 tx=0x7f647400b040 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:08.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.764+0000 7f6471ffb700 1 -- 172.21.15.64:0/521856883 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646400bbf0 con 0x56302b611e40 2023-09-13T03:14:08.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.764+0000 7f6471ffb700 1 -- 172.21.15.64:0/521856883 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6464015070 con 0x56302b611e40 2023-09-13T03:14:08.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.764+0000 7f6471ffb700 1 -- 172.21.15.64:0/521856883 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6464011a30 con 0x56302b611e40 2023-09-13T03:14:08.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.764+0000 7f6495554b80 1 -- 172.21.15.64:0/521856883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56302b611e40 msgr2=0x56302b614230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:08.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.764+0000 7f6495554b80 1 --2- 172.21.15.64:0/521856883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56302b611e40 0x56302b614230 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6464010580 tx=0x7f6464010890 comp rx=0 tx=0).stop 2023-09-13T03:14:08.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.764+0000 7f6495554b80 1 -- 172.21.15.64:0/521856883 shutdown_connections 2023-09-13T03:14:08.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.764+0000 7f6495554b80 1 --2- 172.21.15.64:0/521856883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56302b611e40 0x56302b614230 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:08.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.764+0000 7f6495554b80 1 --2- 172.21.15.64:0/521856883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56302b607710 0x56302b607af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:08.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.764+0000 7f6495554b80 1 --2- 172.21.15.64:0/521856883 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56302b608030 0x56302b611900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:08.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.765+0000 7f6495554b80 1 -- 172.21.15.64:0/521856883 >> 172.21.15.64:0/521856883 conn(0x56302b57c090 msgr2=0x56302b602bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:08.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.765+0000 7f6495554b80 1 -- 172.21.15.64:0/521856883 shutdown_connections 2023-09-13T03:14:08.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.765+0000 7f6495554b80 1 -- 172.21.15.64:0/521856883 wait complete. 2023-09-13T03:14:08.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.765+0000 7f6495554b80 1 -- start start 2023-09-13T03:14:08.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.765+0000 7f6495554b80 1 Processor -- start 2023-09-13T03:14:08.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.766+0000 7f6495554b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56302b607710 0x56302b611840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:08.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.766+0000 7f6495554b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56302b608030 0x56302b60ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:08.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.766+0000 7f6495554b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56302b611e40 0x56302b60f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:08.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.766+0000 7f6495554b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56302b4f7b70 con 0x56302b608030 2023-09-13T03:14:08.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.766+0000 7f6495554b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56302b4f7870 con 0x56302b607710 2023-09-13T03:14:08.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.766+0000 7f6495554b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56302b3db920 con 0x56302b611e40 2023-09-13T03:14:08.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.766+0000 7f6495554b80 10 client.0 _ll_register_callbacks cb 0x56302b441a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:14:08.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.766+0000 7f647b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56302b607710 0x56302b611840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:08.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.766+0000 7f6473fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56302b608030 0x56302b60ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:08.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.766+0000 7f647b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56302b607710 0x56302b611840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:53546/0 (socket says 172.21.15.64:53546) 2023-09-13T03:14:08.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.766+0000 7f647b7fe700 1 -- 172.21.15.64:0/1996458658 learned_addr learned my addr 172.21.15.64:0/1996458658 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:08.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.766+0000 7f647bfff700 1 --2- 172.21.15.64:0/1996458658 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56302b611e40 0x56302b60f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:08.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.767+0000 7f647b7fe700 1 -- 172.21.15.64:0/1996458658 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56302b611e40 msgr2=0x56302b60f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:08.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.767+0000 7f647b7fe700 1 --2- 172.21.15.64:0/1996458658 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56302b611e40 0x56302b60f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:08.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.767+0000 7f647b7fe700 1 -- 172.21.15.64:0/1996458658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56302b608030 msgr2=0x56302b60ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:08.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.767+0000 7f647b7fe700 1 --2- 172.21.15.64:0/1996458658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56302b608030 0x56302b60ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:08.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.767+0000 7f647b7fe700 1 -- 172.21.15.64:0/1996458658 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f646400bd50 con 0x56302b607710 2023-09-13T03:14:08.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.767+0000 7f647b7fe700 1 --2- 172.21.15.64:0/1996458658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56302b607710 0x56302b611840 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f646c096820 tx=0x7f646c069aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22390f42974e1720 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:08.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.767+0000 7f645affd700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:14:08.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.767+0000 7f645affd700 1 -- 172.21.15.64:0/1996458658 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646c06c2f0 con 0x56302b607710 2023-09-13T03:14:08.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.767+0000 7f6495554b80 1 -- 172.21.15.64:0/1996458658 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56302b4b14f0 con 0x56302b607710 2023-09-13T03:14:08.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.767+0000 7f645affd700 1 -- 172.21.15.64:0/1996458658 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f646c09aa20 con 0x56302b607710 2023-09-13T03:14:08.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.767+0000 7f645affd700 1 -- 172.21.15.64:0/1996458658 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f646c09a320 con 0x56302b607710 2023-09-13T03:14:08.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.767+0000 7f6495554b80 10 client.25949 fetch_fsmap learned FSMap version 27 2023-09-13T03:14:08.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.767+0000 7f6495554b80 1 -- 172.21.15.64:0/1996458658 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56302b36ec60 con 0x56302b607710 2023-09-13T03:14:08.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.768+0000 7f645affd700 1 -- 172.21.15.64:0/1996458658 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56302b4b14f0 con 0x56302b607710 2023-09-13T03:14:08.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.768+0000 7f645affd700 1 -- 172.21.15.64:0/1996458658 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f646c099810 con 0x56302b607710 2023-09-13T03:14:08.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.768+0000 7f6495554b80 10 client.25949 fetch_fsmap finished waiting for FSMap version 27 2023-09-13T03:14:08.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.768+0000 7f6495554b80 10 client.25949 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T03:14:08.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.768+0000 7f6495554b80 20 client.25949 populate_metadata read hostname 'smithi064' 2023-09-13T03:14:08.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.768+0000 7f6495554b80 1 --2- 172.21.15.64:0/1996458658 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56302b6214b0 0x56302b6238d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:08.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.768+0000 7f6495554b80 4 client.25949 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:14:08.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.768+0000 7f6495554b80 1 -- 172.21.15.64:0/1996458658 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56302b56e8a0 con 0x56302b6214b0 2023-09-13T03:14:08.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.768+0000 7f6473fff700 1 --2- 172.21.15.64:0/1996458658 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56302b6214b0 0x56302b6238d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:08.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.769+0000 7f6473fff700 1 --2- 172.21.15.64:0/1996458658 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56302b6214b0 0x56302b6238d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f6fba126284fb66 server_cookie=34e3f574f0808c5f in_seq=0 out_seq=0 2023-09-13T03:14:08.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.769+0000 7f645affd700 10 client.25949 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:08.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.769+0000 7f645affd700 1 -- 172.21.15.64:0/1996458658 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56302b56e8a0 con 0x56302b6214b0 2023-09-13T03:14:08.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.769+0000 7f645affd700 10 client.25949 handle_command_reply: tid=0 2023-09-13T03:14:08.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.769+0000 7f6495554b80 1 client.25949 shutdown 2023-09-13T03:14:08.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.770+0000 7f6495554b80 1 -- 172.21.15.64:0/1996458658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56302b607710 msgr2=0x56302b611840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:08.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.770+0000 7f6495554b80 1 --2- 172.21.15.64:0/1996458658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56302b607710 0x56302b611840 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f646c096820 tx=0x7f646c069aa0 comp rx=0 tx=0).stop 2023-09-13T03:14:08.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.770+0000 7f6495554b80 1 -- 172.21.15.64:0/1996458658 shutdown_connections 2023-09-13T03:14:08.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.770+0000 7f6495554b80 1 --2- 172.21.15.64:0/1996458658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56302b607710 0x56302b611840 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:08.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.770+0000 7f6495554b80 1 --2- 172.21.15.64:0/1996458658 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56302b6214b0 0x56302b6238d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:08.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.770+0000 7f6495554b80 1 --2- 172.21.15.64:0/1996458658 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56302b611e40 0x56302b60f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:08.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.770+0000 7f645affd700 0 client.25949 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:08.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.770+0000 7f6495554b80 1 --2- 172.21.15.64:0/1996458658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56302b608030 0x56302b60ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:08.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.770+0000 7f6495554b80 1 -- 172.21.15.64:0/1996458658 >> 172.21.15.64:0/1996458658 conn(0x56302b57c090 msgr2=0x56302b6027e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:08.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.770+0000 7f6495554b80 1 -- 172.21.15.64:0/1996458658 shutdown_connections 2023-09-13T03:14:08.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.770+0000 7f6495554b80 1 -- 172.21.15.64:0/1996458658 wait complete. 2023-09-13T03:14:08.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.770+0000 7f6495554b80 20 client.25949 trim_cache size 0 max 16384 2023-09-13T03:14:08.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.787+0000 7f6495554b80 1 Processor -- start 2023-09-13T03:14:08.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.787+0000 7f6495554b80 1 -- start start 2023-09-13T03:14:08.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.788+0000 7f6495554b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56302b607710 0x56302b601320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:08.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.788+0000 7f6495554b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56302b608030 0x56302b609ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:08.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.788+0000 7f6495554b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56302b611e40 0x56302b61d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:08.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.788+0000 7f6495554b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56302b6036e0 con 0x56302b607710 2023-09-13T03:14:08.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.788+0000 7f6495554b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56302b609640 con 0x56302b611e40 2023-09-13T03:14:08.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.788+0000 7f6495554b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56302b609800 con 0x56302b608030 2023-09-13T03:14:08.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.788+0000 7f647b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56302b607710 0x56302b601320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:08.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.788+0000 7f647b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56302b607710 0x56302b601320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59774/0 (socket says 172.21.15.64:59774) 2023-09-13T03:14:08.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.788+0000 7f647b7fe700 1 -- 172.21.15.64:0/4184756598 learned_addr learned my addr 172.21.15.64:0/4184756598 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:08.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.788+0000 7f6473fff700 1 --2- 172.21.15.64:0/4184756598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56302b608030 0x56302b609ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:08.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.788+0000 7f647bfff700 1 --2- 172.21.15.64:0/4184756598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56302b611e40 0x56302b61d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:08.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.788+0000 7f647b7fe700 1 -- 172.21.15.64:0/4184756598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56302b608030 msgr2=0x56302b609ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:08.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.788+0000 7f647b7fe700 1 --2- 172.21.15.64:0/4184756598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56302b608030 0x56302b609ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:08.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.788+0000 7f647b7fe700 1 -- 172.21.15.64:0/4184756598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56302b611e40 msgr2=0x56302b61d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:08.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.788+0000 7f647b7fe700 1 --2- 172.21.15.64:0/4184756598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56302b611e40 0x56302b61d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:08.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.788+0000 7f647b7fe700 1 -- 172.21.15.64:0/4184756598 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56302b3ac250 con 0x56302b607710 2023-09-13T03:14:08.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.789+0000 7f647b7fe700 1 --2- 172.21.15.64:0/4184756598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56302b607710 0x56302b601320 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f646c009fd0 tx=0x7f646c004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a6b83b7079153b server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:08.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.789+0000 7f64727fc700 1 -- 172.21.15.64:0/4184756598 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646c065fc0 con 0x56302b607710 2023-09-13T03:14:08.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.789+0000 7f64727fc700 1 -- 172.21.15.64:0/4184756598 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f646c09ee90 con 0x56302b607710 2023-09-13T03:14:08.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.789+0000 7f64727fc700 1 -- 172.21.15.64:0/4184756598 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646c09e1e0 con 0x56302b607710 2023-09-13T03:14:08.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.789+0000 7f6495554b80 1 -- 172.21.15.64:0/4184756598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56302b607710 msgr2=0x56302b601320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:08.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.789+0000 7f6495554b80 1 --2- 172.21.15.64:0/4184756598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56302b607710 0x56302b601320 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f646c009fd0 tx=0x7f646c004be0 comp rx=0 tx=0).stop 2023-09-13T03:14:08.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.789+0000 7f6495554b80 1 -- 172.21.15.64:0/4184756598 shutdown_connections 2023-09-13T03:14:08.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.789+0000 7f6495554b80 1 --2- 172.21.15.64:0/4184756598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56302b611e40 0x56302b61d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:08.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.789+0000 7f6495554b80 1 --2- 172.21.15.64:0/4184756598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56302b607710 0x56302b601320 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:08.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.789+0000 7f6495554b80 1 --2- 172.21.15.64:0/4184756598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56302b608030 0x56302b609ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:08.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.789+0000 7f6495554b80 1 -- 172.21.15.64:0/4184756598 >> 172.21.15.64:0/4184756598 conn(0x56302b57c090 msgr2=0x56302b628230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:08.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.790+0000 7f6495554b80 1 -- 172.21.15.64:0/4184756598 shutdown_connections 2023-09-13T03:14:08.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.790+0000 7f6495554b80 1 -- 172.21.15.64:0/4184756598 wait complete. 2023-09-13T03:14:08.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.790+0000 7f6495554b80 1 -- start start 2023-09-13T03:14:08.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.790+0000 7f6495554b80 1 Processor -- start 2023-09-13T03:14:08.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.791+0000 7f6495554b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56302b607710 0x56302b618680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:08.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.791+0000 7f6495554b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56302b608030 0x56302b618bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:08.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.791+0000 7f6495554b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56302b611e40 0x56302b616b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:08.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.791+0000 7f6495554b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56302b6172a0 con 0x56302b608030 2023-09-13T03:14:08.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.791+0000 7f6495554b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56302b617410 con 0x56302b611e40 2023-09-13T03:14:08.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.791+0000 7f6495554b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56302b6175d0 con 0x56302b607710 2023-09-13T03:14:08.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.791+0000 7f6495554b80 10 client.0 _ll_register_callbacks cb 0x7f646c09a1b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:14:08.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.791+0000 7f647b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56302b607710 0x56302b618680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:08.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.791+0000 7f647b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56302b607710 0x56302b618680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:36796/0 (socket says 172.21.15.64:36796) 2023-09-13T03:14:08.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.791+0000 7f647b7fe700 1 -- 172.21.15.64:0/2636943710 learned_addr learned my addr 172.21.15.64:0/2636943710 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:08.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.791+0000 7f6473fff700 1 --2- 172.21.15.64:0/2636943710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56302b608030 0x56302b618bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:08.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.791+0000 7f647bfff700 1 --2- 172.21.15.64:0/2636943710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56302b611e40 0x56302b616b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:08.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.791+0000 7f647b7fe700 1 -- 172.21.15.64:0/2636943710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56302b611e40 msgr2=0x56302b616b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:08.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.791+0000 7f647b7fe700 1 --2- 172.21.15.64:0/2636943710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56302b611e40 0x56302b616b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:08.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.791+0000 7f647b7fe700 1 -- 172.21.15.64:0/2636943710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56302b608030 msgr2=0x56302b618bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:08.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.791+0000 7f647b7fe700 1 --2- 172.21.15.64:0/2636943710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56302b608030 0x56302b618bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:08.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.791+0000 7f647b7fe700 1 -- 172.21.15.64:0/2636943710 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56302b36ec60 con 0x56302b607710 2023-09-13T03:14:08.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.792+0000 7f647b7fe700 1 --2- 172.21.15.64:0/2636943710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56302b607710 0x56302b618680 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f646c096d90 tx=0x7f646c004c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46171b7d95c07945 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:08.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.792+0000 7f645affd700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:14:08.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.792+0000 7f645affd700 1 -- 172.21.15.64:0/2636943710 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646c0a2070 con 0x56302b607710 2023-09-13T03:14:08.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.792+0000 7f6495554b80 1 -- 172.21.15.64:0/2636943710 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6464015070 con 0x56302b607710 2023-09-13T03:14:08.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.792+0000 7f645affd700 1 -- 172.21.15.64:0/2636943710 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f646c09a790 con 0x56302b607710 2023-09-13T03:14:08.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.792+0000 7f645affd700 1 -- 172.21.15.64:0/2636943710 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f646c096660 con 0x56302b607710 2023-09-13T03:14:08.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.792+0000 7f6495554b80 10 client.25957 fetch_fsmap learned FSMap version 27 2023-09-13T03:14:08.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.792+0000 7f6495554b80 1 -- 172.21.15.64:0/2636943710 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56302b3842b0 con 0x56302b607710 2023-09-13T03:14:08.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.792+0000 7f645affd700 1 -- 172.21.15.64:0/2636943710 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6464015070 con 0x56302b607710 2023-09-13T03:14:08.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.793+0000 7f645affd700 1 -- 172.21.15.64:0/2636943710 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 27) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f646c0b8170 con 0x56302b607710 2023-09-13T03:14:08.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.793+0000 7f6495554b80 10 client.25957 fetch_fsmap finished waiting for FSMap version 27 2023-09-13T03:14:08.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.793+0000 7f6495554b80 10 client.25957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T03:14:08.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.793+0000 7f6495554b80 20 client.25957 populate_metadata read hostname 'smithi064' 2023-09-13T03:14:08.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.793+0000 7f6495554b80 1 --2- 172.21.15.64:0/2636943710 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56302b6760f0 0x56302b6784e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:08.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.793+0000 7f6495554b80 4 client.25957 mds_command: new command op to 24686 tid=0[{"prefix": "scrub status"}] 2023-09-13T03:14:08.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.793+0000 7f6495554b80 1 -- 172.21.15.64:0/2636943710 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56302b552920 con 0x56302b6760f0 2023-09-13T03:14:08.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.793+0000 7f6473fff700 1 --2- 172.21.15.64:0/2636943710 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56302b6760f0 0x56302b6784e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:08.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.794+0000 7f6473fff700 1 --2- 172.21.15.64:0/2636943710 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56302b6760f0 0x56302b6784e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=690956c3241a915a server_cookie=1d7736730552f61e in_seq=0 out_seq=0 2023-09-13T03:14:08.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.794+0000 7f645affd700 10 client.25957 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:08.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.794+0000 7f645affd700 1 -- 172.21.15.64:0/2636943710 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56302b552920 con 0x56302b6760f0 2023-09-13T03:14:08.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.794+0000 7f645affd700 10 client.25957 handle_command_reply: tid=0 2023-09-13T03:14:08.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.794+0000 7f6495554b80 1 client.25957 shutdown 2023-09-13T03:14:08.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.794+0000 7f6495554b80 1 -- 172.21.15.64:0/2636943710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56302b607710 msgr2=0x56302b618680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:08.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.794+0000 7f6495554b80 1 --2- 172.21.15.64:0/2636943710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56302b607710 0x56302b618680 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f646c096d90 tx=0x7f646c004c50 comp rx=0 tx=0).stop 2023-09-13T03:14:08.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.794+0000 7f6495554b80 1 -- 172.21.15.64:0/2636943710 shutdown_connections 2023-09-13T03:14:08.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.794+0000 7f6495554b80 1 --2- 172.21.15.64:0/2636943710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56302b611e40 0x56302b616b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:08.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.794+0000 7f6495554b80 1 --2- 172.21.15.64:0/2636943710 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56302b6760f0 0x56302b6784e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:08.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.794+0000 7f6495554b80 1 --2- 172.21.15.64:0/2636943710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56302b607710 0x56302b618680 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:08.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.794+0000 7f6495554b80 1 --2- 172.21.15.64:0/2636943710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56302b608030 0x56302b618bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:08.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.794+0000 7f6495554b80 1 -- 172.21.15.64:0/2636943710 >> 172.21.15.64:0/2636943710 conn(0x56302b57c090 msgr2=0x56302b61d140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:08.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.794+0000 7f645affd700 0 client.25957 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:08.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.794+0000 7f6495554b80 1 -- 172.21.15.64:0/2636943710 shutdown_connections 2023-09-13T03:14:08.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.794+0000 7f6495554b80 1 -- 172.21.15.64:0/2636943710 wait complete. 2023-09-13T03:14:08.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.794+0000 7f6495554b80 20 client.25957 trim_cache size 0 max 16384 2023-09-13T03:14:08.815 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-13T03:14:08.815 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2023-09-13T03:14:08.816 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2023-09-13T03:14:08.816 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-13T03:14:08.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.795+0000 7f6481bff700 1 -- 172.21.15.64:0/3369298688 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f64680a08d0 msgr2=0x7f64680a2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:08.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.795+0000 7f6481bff700 1 --2- 172.21.15.64:0/3369298688 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f64680a08d0 0x7f64680a2d90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6474009ec0 tx=0x7f647400b040 comp rx=0 tx=0).stop 2023-09-13T03:14:08.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.795+0000 7f6481bff700 1 -- 172.21.15.64:0/3369298688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f647c100590 msgr2=0x7f647c072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:08.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.795+0000 7f6481bff700 1 --2- 172.21.15.64:0/3369298688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f647c100590 0x7f647c072900 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f646c00b580 tx=0x7f646c00f690 comp rx=0 tx=0).stop 2023-09-13T03:14:08.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.795+0000 7f6481bff700 1 -- 172.21.15.64:0/3369298688 shutdown_connections 2023-09-13T03:14:08.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.795+0000 7f6481bff700 1 --2- 172.21.15.64:0/3369298688 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f64680a08d0 0x7f64680a2d90 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:08.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.795+0000 7f6481bff700 1 --2- 172.21.15.64:0/3369298688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f647c106650 0x7f647c073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:08.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.795+0000 7f6481bff700 1 --2- 172.21.15.64:0/3369298688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f647c100590 0x7f647c072900 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:08.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.795+0000 7f6481bff700 1 --2- 172.21.15.64:0/3369298688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f647c100f50 0x7f647c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:08.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.795+0000 7f6481bff700 1 -- 172.21.15.64:0/3369298688 >> 172.21.15.64:0/3369298688 conn(0x7f647c0fc060 msgr2=0x7f647c0fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:08.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.795+0000 7f6481bff700 1 -- 172.21.15.64:0/3369298688 shutdown_connections 2023-09-13T03:14:08.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:08.795+0000 7f6481bff700 1 -- 172.21.15.64:0/3369298688 wait complete. 2023-09-13T03:14:08.821 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-09-13T03:14:08.821 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T03:14:08.882 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:08 smithi064 ceph-mon[97675]: pgmap v32: 97 pgs: 97 active+clean; 15 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 1.3 MiB/s wr, 271 op/s 2023-09-13T03:14:08.884 INFO:tasks.workunit.client.0.smithi064.stdout: 1 25247 11.91 MB/sec warmup 13 sec latency 3308.845 ms 2023-09-13T03:14:08.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:08 smithi170 ceph-mon[134328]: pgmap v32: 97 pgs: 97 active+clean; 15 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 1.3 MiB/s wr, 271 op/s 2023-09-13T03:14:08.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:08 smithi142 ceph-mon[134082]: pgmap v32: 97 pgs: 97 active+clean; 15 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 1.3 MiB/s wr, 271 op/s 2023-09-13T03:14:09.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.025+0000 7fa51f1ff700 1 -- 172.21.15.64:0/4004778150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa518106650 msgr2=0x7fa518106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:09.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.025+0000 7fa51f1ff700 1 --2- 172.21.15.64:0/4004778150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa518106650 0x7fa518106a30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa50c0099c0 tx=0x7fa50c009cd0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.026+0000 7fa51f1ff700 1 -- 172.21.15.64:0/4004778150 shutdown_connections 2023-09-13T03:14:09.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.026+0000 7fa51f1ff700 1 --2- 172.21.15.64:0/4004778150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa518106650 0x7fa518106a30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.026+0000 7fa51f1ff700 1 --2- 172.21.15.64:0/4004778150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa518100f50 0x7fa51810d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.026+0000 7fa51f1ff700 1 --2- 172.21.15.64:0/4004778150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa518100590 0x7fa518100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.026+0000 7fa51f1ff700 1 -- 172.21.15.64:0/4004778150 >> 172.21.15.64:0/4004778150 conn(0x7fa5180fc020 msgr2=0x7fa5180fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:09.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.026+0000 7fa51f1ff700 1 -- 172.21.15.64:0/4004778150 shutdown_connections 2023-09-13T03:14:09.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.026+0000 7fa51f1ff700 1 -- 172.21.15.64:0/4004778150 wait complete. 2023-09-13T03:14:09.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.027+0000 7fa51f1ff700 1 Processor -- start 2023-09-13T03:14:09.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.027+0000 7fa51f1ff700 1 -- start start 2023-09-13T03:14:09.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.027+0000 7fa51f1ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa518100590 0x7fa518072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:09.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.027+0000 7fa51f1ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa518100f50 0x7fa518072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:09.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.027+0000 7fa51f1ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa518106650 0x7fa5181a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:09.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.027+0000 7fa51f1ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa518073670 con 0x7fa518100f50 2023-09-13T03:14:09.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.027+0000 7fa51f1ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5181a4dc0 con 0x7fa518106650 2023-09-13T03:14:09.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.027+0000 7fa51f1ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5181a4f00 con 0x7fa518100590 2023-09-13T03:14:09.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.028+0000 7fa517fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa518100f50 0x7fa518072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:09.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.028+0000 7fa51cf9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa518100590 0x7fa518072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:09.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.028+0000 7fa51d79c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa518106650 0x7fa5181a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:09.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.028+0000 7fa517fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa518100f50 0x7fa518072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59802/0 (socket says 172.21.15.64:59802) 2023-09-13T03:14:09.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.028+0000 7fa517fff700 1 -- 172.21.15.64:0/656134701 learned_addr learned my addr 172.21.15.64:0/656134701 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:09.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.028+0000 7fa517fff700 1 -- 172.21.15.64:0/656134701 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa518100590 msgr2=0x7fa518072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:09.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.028+0000 7fa517fff700 1 --2- 172.21.15.64:0/656134701 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa518100590 0x7fa518072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.028+0000 7fa517fff700 1 -- 172.21.15.64:0/656134701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa518106650 msgr2=0x7fa5181a4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:09.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.028+0000 7fa517fff700 1 --2- 172.21.15.64:0/656134701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa518106650 0x7fa5181a4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.028+0000 7fa517fff700 1 -- 172.21.15.64:0/656134701 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa510009c40 con 0x7fa518100f50 2023-09-13T03:14:09.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.028+0000 7fa51cf9b700 1 --2- 172.21.15.64:0/656134701 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa518100590 0x7fa518072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:09.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.028+0000 7fa517fff700 1 --2- 172.21.15.64:0/656134701 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa518100f50 0x7fa518072e40 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa50400e9a0 tx=0x7fa50400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:09.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.028+0000 7fa515ffb700 1 -- 172.21.15.64:0/656134701 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa504009790 con 0x7fa518100f50 2023-09-13T03:14:09.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.029+0000 7fa51f1ff700 1 -- 172.21.15.64:0/656134701 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa50c0095f0 con 0x7fa518100f50 2023-09-13T03:14:09.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.029+0000 7fa515ffb700 1 -- 172.21.15.64:0/656134701 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa50400ce80 con 0x7fa518100f50 2023-09-13T03:14:09.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.029+0000 7fa515ffb700 1 -- 172.21.15.64:0/656134701 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa504018680 con 0x7fa518100f50 2023-09-13T03:14:09.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.029+0000 7fa51f1ff700 1 -- 172.21.15.64:0/656134701 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5181a5500 con 0x7fa518100f50 2023-09-13T03:14:09.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.030+0000 7fa51f1ff700 1 -- 172.21.15.64:0/656134701 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5180673f0 con 0x7fa518100f50 2023-09-13T03:14:09.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.032+0000 7fa515ffb700 1 -- 172.21.15.64:0/656134701 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) v1 ==== 135492+0+0 (secure 0 0 0) 0x7fa504010430 con 0x7fa518100f50 2023-09-13T03:14:09.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.033+0000 7fa515ffb700 1 --2- 172.21.15.64:0/656134701 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fa5080a09a0 0x7fa5080a2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:09.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.034+0000 7fa515ffb700 1 -- 172.21.15.64:0/656134701 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa504014070 con 0x7fa518100f50 2023-09-13T03:14:09.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.034+0000 7fa51cf9b700 1 --2- 172.21.15.64:0/656134701 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fa5080a09a0 0x7fa5080a2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:09.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.034+0000 7fa51cf9b700 1 --2- 172.21.15.64:0/656134701 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fa5080a09a0 0x7fa5080a2e60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa50c009990 tx=0x7fa50c01a040 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:09.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.039+0000 7fa515ffb700 1 -- 172.21.15.64:0/656134701 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fa504051e10 con 0x7fa518100f50 2023-09-13T03:14:09.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.300+0000 7fa51f1ff700 1 -- 172.21.15.64:0/656134701 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa5180008d0 con 0x7fa518100f50 2023-09-13T03:14:09.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.301+0000 7fa515ffb700 1 -- 172.21.15.64:0/656134701 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11367 (secure 0 0 0) 0x7fa5040959e0 con 0x7fa518100f50 2023-09-13T03:14:09.301 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:14:09.302 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2234182985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2234182985},{"type":"v1","addr":"172.21.15.64:6835","nonce":2234182985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4053841873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4053841873},{"type":"v1","addr":"172.21.15.64:6837","nonce":4053841873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-13T03:11:10.490260+0000","modified":"2023-09-13T03:14:04.025734+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24686,"mds_1":24649,"mds_2":24635,"mds_3":24634,"mds_4":24622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6839/99523374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":99523374},{"type":"v1","addr":"172.21.15.64:6839","nonce":99523374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14649":{"gid":14649,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6841/1866582875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":1866582875},{"type":"v1","addr":"172.21.15.64:6841","nonce":1866582875}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/101906999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":101906999},{"type":"v1","addr":"172.21.15.142:6833","nonce":101906999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24574":{"gid":24574,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/1897366227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1897366227},{"type":"v1","addr":"172.21.15.170:6833","nonce":1897366227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24611":{"gid":24611,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2167779918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2167779918},{"type":"v1","addr":"172.21.15.142:6835","nonce":2167779918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24622":{"gid":24622,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6835/1493371121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1493371121},{"type":"v1","addr":"172.21.15.170:6835","nonce":1493371121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24634":{"gid":24634,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.170:6837/1726587275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":1726587275},{"type":"v1","addr":"172.21.15.170:6837","nonce":1726587275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24635":{"gid":24635,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.142:6837/741173268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":741173268},{"type":"v1","addr":"172.21.15.142:6837","nonce":741173268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24649":{"gid":24649,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.170:6839/2189703596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2189703596},{"type":"v1","addr":"172.21.15.170:6839","nonce":2189703596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24686":{"gid":24686,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/3645998616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645998616},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645998616}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-13T03:14:09.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.303+0000 7fa51f1ff700 1 -- 172.21.15.64:0/656134701 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fa5080a09a0 msgr2=0x7fa5080a2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:09.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.303+0000 7fa51f1ff700 1 --2- 172.21.15.64:0/656134701 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fa5080a09a0 0x7fa5080a2e60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa50c009990 tx=0x7fa50c01a040 comp rx=0 tx=0).stop 2023-09-13T03:14:09.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.303+0000 7fa51f1ff700 1 -- 172.21.15.64:0/656134701 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa518100f50 msgr2=0x7fa518072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:09.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.303+0000 7fa51f1ff700 1 --2- 172.21.15.64:0/656134701 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa518100f50 0x7fa518072e40 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa50400e9a0 tx=0x7fa50400ecb0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.303+0000 7fa51f1ff700 1 -- 172.21.15.64:0/656134701 shutdown_connections 2023-09-13T03:14:09.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.303+0000 7fa51f1ff700 1 --2- 172.21.15.64:0/656134701 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fa5080a09a0 0x7fa5080a2e60 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.303+0000 7fa51f1ff700 1 --2- 172.21.15.64:0/656134701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa518106650 0x7fa5181a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.303+0000 7fa51f1ff700 1 --2- 172.21.15.64:0/656134701 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa518100590 0x7fa518072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.303+0000 7fa51f1ff700 1 --2- 172.21.15.64:0/656134701 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa518100f50 0x7fa518072e40 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.303+0000 7fa51f1ff700 1 -- 172.21.15.64:0/656134701 >> 172.21.15.64:0/656134701 conn(0x7fa5180fc020 msgr2=0x7fa5180fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:09.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.303+0000 7fa51f1ff700 1 -- 172.21.15.64:0/656134701 shutdown_connections 2023-09-13T03:14:09.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.303+0000 7fa51f1ff700 1 -- 172.21.15.64:0/656134701 wait complete. 2023-09-13T03:14:09.307 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 27 2023-09-13T03:14:09.323 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-09-13T03:14:09.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.529+0000 7f3dd0dff700 1 -- 172.21.15.64:0/2955306028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3dcc106630 msgr2=0x7f3dcc106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:09.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.529+0000 7f3dd0dff700 1 --2- 172.21.15.64:0/2955306028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3dcc106630 0x7f3dcc106a10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc0099c0 tx=0x7f3dbc009cd0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.529+0000 7f3dd0dff700 1 -- 172.21.15.64:0/2955306028 shutdown_connections 2023-09-13T03:14:09.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.529+0000 7f3dd0dff700 1 --2- 172.21.15.64:0/2955306028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3dcc106630 0x7f3dcc106a10 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.529+0000 7f3dd0dff700 1 --2- 172.21.15.64:0/2955306028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3dcc100f30 0x7f3dcc10d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.529+0000 7f3dd0dff700 1 --2- 172.21.15.64:0/2955306028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3dcc100570 0x7f3dcc1009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.529+0000 7f3dd0dff700 1 -- 172.21.15.64:0/2955306028 >> 172.21.15.64:0/2955306028 conn(0x7f3dcc0fc080 msgr2=0x7f3dcc0fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:09.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.529+0000 7f3dd0dff700 1 -- 172.21.15.64:0/2955306028 shutdown_connections 2023-09-13T03:14:09.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.530+0000 7f3dd0dff700 1 -- 172.21.15.64:0/2955306028 wait complete. 2023-09-13T03:14:09.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.530+0000 7f3dd0dff700 1 Processor -- start 2023-09-13T03:14:09.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.531+0000 7f3dd0dff700 1 -- start start 2023-09-13T03:14:09.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.531+0000 7f3dd0dff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3dcc100570 0x7f3dcc19e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:09.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.531+0000 7f3dd0dff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3dcc100f30 0x7f3dcc19eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:09.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.531+0000 7f3dd0dff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3dcc106630 0x7f3dcc198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:09.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.531+0000 7f3dd0dff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc198cf0 con 0x7f3dcc100f30 2023-09-13T03:14:09.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.531+0000 7f3dd0dff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc198e60 con 0x7f3dcc106630 2023-09-13T03:14:09.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.531+0000 7f3dd0dff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc199020 con 0x7f3dcc100570 2023-09-13T03:14:09.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.531+0000 7f3dc3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3dcc100f30 0x7f3dcc19eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:09.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.531+0000 7f3dc3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3dcc100f30 0x7f3dcc19eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59828/0 (socket says 172.21.15.64:59828) 2023-09-13T03:14:09.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.531+0000 7f3dc3fff700 1 -- 172.21.15.64:0/965170645 learned_addr learned my addr 172.21.15.64:0/965170645 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:09.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.532+0000 7f3dca59c700 1 --2- 172.21.15.64:0/965170645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3dcc100570 0x7f3dcc19e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:09.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.532+0000 7f3dcad9d700 1 --2- 172.21.15.64:0/965170645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3dcc106630 0x7f3dcc198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:09.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.532+0000 7f3dc3fff700 1 -- 172.21.15.64:0/965170645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3dcc100570 msgr2=0x7f3dcc19e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:09.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.532+0000 7f3dc3fff700 1 --2- 172.21.15.64:0/965170645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3dcc100570 0x7f3dcc19e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.532+0000 7f3dc3fff700 1 -- 172.21.15.64:0/965170645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3dcc106630 msgr2=0x7f3dcc198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:09.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.532+0000 7f3dc3fff700 1 --2- 172.21.15.64:0/965170645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3dcc106630 0x7f3dcc198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.532+0000 7f3dc3fff700 1 -- 172.21.15.64:0/965170645 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dbc0095f0 con 0x7f3dcc100f30 2023-09-13T03:14:09.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.532+0000 7f3dc3fff700 1 --2- 172.21.15.64:0/965170645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3dcc100f30 0x7f3dcc19eb50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3db400e9a0 tx=0x7f3db400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:09.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.532+0000 7f3dc37fe700 1 -- 172.21.15.64:0/965170645 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db4009790 con 0x7f3dcc100f30 2023-09-13T03:14:09.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.532+0000 7f3dd0dff700 1 -- 172.21.15.64:0/965170645 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dcc199300 con 0x7f3dcc100f30 2023-09-13T03:14:09.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.533+0000 7f3dc37fe700 1 -- 172.21.15.64:0/965170645 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3db4010430 con 0x7f3dcc100f30 2023-09-13T03:14:09.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.533+0000 7f3dc37fe700 1 -- 172.21.15.64:0/965170645 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db4018680 con 0x7f3dcc100f30 2023-09-13T03:14:09.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.533+0000 7f3dd0dff700 1 -- 172.21.15.64:0/965170645 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dcc1a5600 con 0x7f3dcc100f30 2023-09-13T03:14:09.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.536+0000 7f3dc37fe700 1 -- 172.21.15.64:0/965170645 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) v1 ==== 135492+0+0 (secure 0 0 0) 0x7f3db40105a0 con 0x7f3dcc100f30 2023-09-13T03:14:09.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.537+0000 7f3dc37fe700 1 --2- 172.21.15.64:0/965170645 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f3db80a08d0 0x7f3db80a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:09.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.537+0000 7f3de47d1b80 1 Processor -- start 2023-09-13T03:14:09.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.537+0000 7f3dc37fe700 1 -- 172.21.15.64:0/965170645 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f3db4014070 con 0x7f3dcc100f30 2023-09-13T03:14:09.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.537+0000 7f3de47d1b80 1 -- start start 2023-09-13T03:14:09.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.537+0000 7f3dca59c700 1 --2- 172.21.15.64:0/965170645 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f3db80a08d0 0x7f3db80a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:09.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.538+0000 7f3de47d1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ce16995710 0x55ce16995af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:09.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.538+0000 7f3de47d1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce16996030 0x55ce1699f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:09.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.538+0000 7f3de47d1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce1699fe40 0x55ce169a2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:09.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.538+0000 7f3de47d1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce168ee730 con 0x55ce1699fe40 2023-09-13T03:14:09.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.538+0000 7f3de47d1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce168ec050 con 0x55ce16996030 2023-09-13T03:14:09.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.538+0000 7f3de47d1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce168e7940 con 0x55ce16995710 2023-09-13T03:14:09.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.538+0000 7f3dc3fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce16996030 0x55ce1699f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:09.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.538+0000 7f3dcad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce1699fe40 0x55ce169a2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:09.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.538+0000 7f3dc3fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce16996030 0x55ce1699f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:53596/0 (socket says 172.21.15.64:53596) 2023-09-13T03:14:09.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.538+0000 7f3dc3fff700 1 -- 172.21.15.64:0/1789698797 learned_addr learned my addr 172.21.15.64:0/1789698797 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:09.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.538+0000 7f3dca59c700 1 --2- 172.21.15.64:0/1789698797 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ce16995710 0x55ce16995af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:09.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.538+0000 7f3dca59c700 1 --2- 172.21.15.64:0/965170645 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f3db80a08d0 0x7f3db80a2d90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc009990 tx=0x7f3dbc005fb0 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:09.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.539+0000 7f3dc3fff700 1 -- 172.21.15.64:0/1789698797 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ce16995710 msgr2=0x55ce16995af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:09.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.539+0000 7f3dc3fff700 1 --2- 172.21.15.64:0/1789698797 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ce16995710 0x55ce16995af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.539+0000 7f3dc3fff700 1 -- 172.21.15.64:0/1789698797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce1699fe40 msgr2=0x55ce169a2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:09.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.539+0000 7f3dc3fff700 1 --2- 172.21.15.64:0/1789698797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce1699fe40 0x55ce169a2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.539+0000 7f3dc3fff700 1 -- 172.21.15.64:0/1789698797 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce16713de0 con 0x55ce16996030 2023-09-13T03:14:09.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.539+0000 7f3dcad9d700 1 --2- 172.21.15.64:0/1789698797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce1699fe40 0x55ce169a2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:09.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.539+0000 7f3dc3fff700 1 --2- 172.21.15.64:0/1789698797 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce16996030 0x55ce1699f900 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3db4099600 tx=0x7f3db4098fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=680ff309bdfbd03b server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:09.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.539+0000 7f3dc0ff9700 1 -- 172.21.15.64:0/1789698797 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db4099670 con 0x55ce16996030 2023-09-13T03:14:09.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.539+0000 7f3dc0ff9700 1 -- 172.21.15.64:0/1789698797 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3db409d4a0 con 0x55ce16996030 2023-09-13T03:14:09.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.539+0000 7f3dc0ff9700 1 -- 172.21.15.64:0/1789698797 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db4099b30 con 0x55ce16996030 2023-09-13T03:14:09.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.540+0000 7f3de47d1b80 1 -- 172.21.15.64:0/1789698797 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce16996030 msgr2=0x55ce1699f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:09.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.540+0000 7f3de47d1b80 1 --2- 172.21.15.64:0/1789698797 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce16996030 0x55ce1699f900 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3db4099600 tx=0x7f3db4098fd0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.540+0000 7f3de47d1b80 1 -- 172.21.15.64:0/1789698797 shutdown_connections 2023-09-13T03:14:09.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.540+0000 7f3de47d1b80 1 --2- 172.21.15.64:0/1789698797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce1699fe40 0x55ce169a2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.540+0000 7f3de47d1b80 1 --2- 172.21.15.64:0/1789698797 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ce16995710 0x55ce16995af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.540+0000 7f3de47d1b80 1 --2- 172.21.15.64:0/1789698797 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce16996030 0x55ce1699f900 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.540+0000 7f3de47d1b80 1 -- 172.21.15.64:0/1789698797 >> 172.21.15.64:0/1789698797 conn(0x55ce1690a090 msgr2=0x55ce16990bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:09.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.540+0000 7f3de47d1b80 1 -- 172.21.15.64:0/1789698797 shutdown_connections 2023-09-13T03:14:09.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.540+0000 7f3de47d1b80 1 -- 172.21.15.64:0/1789698797 wait complete. 2023-09-13T03:14:09.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.541+0000 7f3de47d1b80 1 -- start start 2023-09-13T03:14:09.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.541+0000 7f3de47d1b80 1 Processor -- start 2023-09-13T03:14:09.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.542+0000 7f3de47d1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce16995710 0x55ce1699f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:09.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.542+0000 7f3de47d1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce16996030 0x55ce1699ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:09.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.542+0000 7f3de47d1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ce1699fe40 0x55ce1699d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:09.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.542+0000 7f3de47d1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce16885b70 con 0x55ce16995710 2023-09-13T03:14:09.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.542+0000 7f3de47d1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce16885870 con 0x55ce16996030 2023-09-13T03:14:09.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.542+0000 7f3de47d1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce16769920 con 0x55ce1699fe40 2023-09-13T03:14:09.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.542+0000 7f3de47d1b80 10 client.0 _ll_register_callbacks cb 0x55ce167cfa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:14:09.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.542+0000 7f3dca59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce16995710 0x55ce1699f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:09.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.542+0000 7f3dca59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce16995710 0x55ce1699f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59840/0 (socket says 172.21.15.64:59840) 2023-09-13T03:14:09.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.542+0000 7f3dca59c700 1 -- 172.21.15.64:0/1825384962 learned_addr learned my addr 172.21.15.64:0/1825384962 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:09.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.542+0000 7f3dc3fff700 1 --2- 172.21.15.64:0/1825384962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce16996030 0x55ce1699ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:09.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.542+0000 7f3dcad9d700 1 --2- 172.21.15.64:0/1825384962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ce1699fe40 0x55ce1699d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:09.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.542+0000 7f3dc3fff700 1 -- 172.21.15.64:0/1825384962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ce1699fe40 msgr2=0x55ce1699d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:09.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.542+0000 7f3dc3fff700 1 --2- 172.21.15.64:0/1825384962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ce1699fe40 0x55ce1699d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.542+0000 7f3dc3fff700 1 -- 172.21.15.64:0/1825384962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce16995710 msgr2=0x55ce1699f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:09.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.542+0000 7f3dc3fff700 1 --2- 172.21.15.64:0/1825384962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce16995710 0x55ce1699f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.542+0000 7f3dc3fff700 1 -- 172.21.15.64:0/1825384962 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc4011230 con 0x55ce16996030 2023-09-13T03:14:09.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.543+0000 7f3dc3fff700 1 --2- 172.21.15.64:0/1825384962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce16996030 0x55ce1699ca60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3db4010b20 tx=0x7f3db4010ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa7f29207ad157fa server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:09.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.543+0000 7f3db1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:14:09.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.543+0000 7f3db1ffb700 1 -- 172.21.15.64:0/1825384962 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db40a16e0 con 0x55ce16996030 2023-09-13T03:14:09.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.543+0000 7f3de47d1b80 1 -- 172.21.15.64:0/1825384962 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ce1683f4f0 con 0x55ce16996030 2023-09-13T03:14:09.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.543+0000 7f3db1ffb700 1 -- 172.21.15.64:0/1825384962 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f3db4099670 con 0x55ce16996030 2023-09-13T03:14:09.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.543+0000 7f3db1ffb700 1 -- 172.21.15.64:0/1825384962 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3db40652b0 con 0x55ce16996030 2023-09-13T03:14:09.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.543+0000 7f3de47d1b80 10 client.25973 fetch_fsmap learned FSMap version 27 2023-09-13T03:14:09.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.543+0000 7f3de47d1b80 1 -- 172.21.15.64:0/1825384962 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f3dcc1a5600 con 0x55ce16996030 2023-09-13T03:14:09.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.543+0000 7f3db1ffb700 1 -- 172.21.15.64:0/1825384962 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ce1683f4f0 con 0x55ce16996030 2023-09-13T03:14:09.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.544+0000 7f3db1ffb700 1 -- 172.21.15.64:0/1825384962 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f3db40a1250 con 0x55ce16996030 2023-09-13T03:14:09.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.544+0000 7f3de47d1b80 10 client.25973 fetch_fsmap finished waiting for FSMap version 27 2023-09-13T03:14:09.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.544+0000 7f3de47d1b80 10 client.25973 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2023-09-13T03:14:09.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.544+0000 7f3de47d1b80 20 client.25973 populate_metadata read hostname 'smithi064' 2023-09-13T03:14:09.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.544+0000 7f3de47d1b80 1 --2- 172.21.15.64:0/1825384962 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x55ce169af520 0x55ce169b1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:09.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.544+0000 7f3de47d1b80 4 client.25973 mds_command: new command op to 24622 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:14:09.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.544+0000 7f3de47d1b80 1 -- 172.21.15.64:0/1825384962 --> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ce168fc8a0 con 0x55ce169af520 2023-09-13T03:14:09.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.544+0000 7f3dca59c700 1 --2- 172.21.15.64:0/1825384962 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x55ce169af520 0x55ce169b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:09.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.545+0000 7f3db1ffb700 10 client.25973 ms_handle_connect on v2:172.21.15.170:6834/1493371121 2023-09-13T03:14:09.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.545+0000 7f3dca59c700 1 --2- 172.21.15.64:0/1825384962 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x55ce169af520 0x55ce169b1910 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=56abc226f20173c6 server_cookie=a37d7265f3cff860 in_seq=0 out_seq=0 2023-09-13T03:14:09.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.545+0000 7f3db1ffb700 1 -- 172.21.15.64:0/1825384962 <== mds.4 v2:172.21.15.170:6834/1493371121 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ce168fc8a0 con 0x55ce169af520 2023-09-13T03:14:09.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.545+0000 7f3db1ffb700 10 client.25973 handle_command_reply: tid=0 2023-09-13T03:14:09.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.546+0000 7f3de47d1b80 1 client.25973 shutdown 2023-09-13T03:14:09.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.546+0000 7f3de47d1b80 1 -- 172.21.15.64:0/1825384962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce16996030 msgr2=0x55ce1699ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:09.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.546+0000 7f3de47d1b80 1 --2- 172.21.15.64:0/1825384962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce16996030 0x55ce1699ca60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3db4010b20 tx=0x7f3db4010ab0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.546+0000 7f3de47d1b80 1 -- 172.21.15.64:0/1825384962 shutdown_connections 2023-09-13T03:14:09.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.546+0000 7f3de47d1b80 1 --2- 172.21.15.64:0/1825384962 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x55ce169af520 0x55ce169b1910 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.546+0000 7f3de47d1b80 1 --2- 172.21.15.64:0/1825384962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ce1699fe40 0x55ce1699d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.546+0000 7f3de47d1b80 1 --2- 172.21.15.64:0/1825384962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce16995710 0x55ce1699f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.546+0000 7f3db1ffb700 0 client.25973 ms_handle_reset on v2:172.21.15.170:6834/1493371121 2023-09-13T03:14:09.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.546+0000 7f3de47d1b80 1 --2- 172.21.15.64:0/1825384962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce16996030 0x55ce1699ca60 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.546+0000 7f3de47d1b80 1 -- 172.21.15.64:0/1825384962 >> 172.21.15.64:0/1825384962 conn(0x55ce1690a090 msgr2=0x55ce16990780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:09.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.546+0000 7f3de47d1b80 1 -- 172.21.15.64:0/1825384962 shutdown_connections 2023-09-13T03:14:09.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.546+0000 7f3de47d1b80 1 -- 172.21.15.64:0/1825384962 wait complete. 2023-09-13T03:14:09.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.546+0000 7f3de47d1b80 20 client.25973 trim_cache size 0 max 16384 2023-09-13T03:14:09.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.560+0000 7f3de47d1b80 1 Processor -- start 2023-09-13T03:14:09.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.560+0000 7f3de47d1b80 1 -- start start 2023-09-13T03:14:09.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.560+0000 7f3de47d1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce16995710 0x55ce1699ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:09.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.560+0000 7f3de47d1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ce16996030 0x55ce1699f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:09.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.560+0000 7f3de47d1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1699fe40 0x55ce16999cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:09.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.560+0000 7f3de47d1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce1698d3d0 con 0x55ce16995710 2023-09-13T03:14:09.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.560+0000 7f3de47d1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce16997280 con 0x55ce1699fe40 2023-09-13T03:14:09.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.560+0000 7f3de47d1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce1699d670 con 0x55ce16996030 2023-09-13T03:14:09.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.560+0000 7f3dca59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce16995710 0x55ce1699ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:09.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.560+0000 7f3dcad9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1699fe40 0x55ce16999cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:09.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.560+0000 7f3dcad9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1699fe40 0x55ce16999cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:53608/0 (socket says 172.21.15.64:53608) 2023-09-13T03:14:09.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.560+0000 7f3dca59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce16995710 0x55ce1699ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59854/0 (socket says 172.21.15.64:59854) 2023-09-13T03:14:09.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.560+0000 7f3dcad9d700 1 -- 172.21.15.64:0/4052076401 learned_addr learned my addr 172.21.15.64:0/4052076401 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:09.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.560+0000 7f3dc3fff700 1 --2- 172.21.15.64:0/4052076401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ce16996030 0x55ce1699f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:09.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.561+0000 7f3dca59c700 1 -- 172.21.15.64:0/4052076401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ce16996030 msgr2=0x55ce1699f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:09.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.561+0000 7f3dca59c700 1 --2- 172.21.15.64:0/4052076401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ce16996030 0x55ce1699f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.561+0000 7f3dca59c700 1 -- 172.21.15.64:0/4052076401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1699fe40 msgr2=0x55ce16999cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:09.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.561+0000 7f3dca59c700 1 --2- 172.21.15.64:0/4052076401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1699fe40 0x55ce16999cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.561+0000 7f3dca59c700 1 -- 172.21.15.64:0/4052076401 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce166fcc60 con 0x55ce16995710 2023-09-13T03:14:09.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.561+0000 7f3dcad9d700 1 --2- 172.21.15.64:0/4052076401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1699fe40 0x55ce16999cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:09.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.561+0000 7f3dca59c700 1 --2- 172.21.15.64:0/4052076401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce16995710 0x55ce1699ca10 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x55ce169b21e0 tx=0x7f3dbc01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1576d0b181237f7a server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:09.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.561+0000 7f3dc17fa700 1 -- 172.21.15.64:0/4052076401 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc029030 con 0x55ce16995710 2023-09-13T03:14:09.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.561+0000 7f3dc17fa700 1 -- 172.21.15.64:0/4052076401 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3dbc02c040 con 0x55ce16995710 2023-09-13T03:14:09.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.561+0000 7f3dc17fa700 1 -- 172.21.15.64:0/4052076401 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc005040 con 0x55ce16995710 2023-09-13T03:14:09.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.562+0000 7f3de47d1b80 1 -- 172.21.15.64:0/4052076401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce16995710 msgr2=0x55ce1699ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:09.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.562+0000 7f3de47d1b80 1 --2- 172.21.15.64:0/4052076401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce16995710 0x55ce1699ca10 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x55ce169b21e0 tx=0x7f3dbc01f000 comp rx=0 tx=0).stop 2023-09-13T03:14:09.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.562+0000 7f3de47d1b80 1 -- 172.21.15.64:0/4052076401 shutdown_connections 2023-09-13T03:14:09.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.562+0000 7f3de47d1b80 1 --2- 172.21.15.64:0/4052076401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1699fe40 0x55ce16999cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.562+0000 7f3de47d1b80 1 --2- 172.21.15.64:0/4052076401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce16995710 0x55ce1699ca10 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.562+0000 7f3de47d1b80 1 --2- 172.21.15.64:0/4052076401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ce16996030 0x55ce1699f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.562+0000 7f3de47d1b80 1 -- 172.21.15.64:0/4052076401 >> 172.21.15.64:0/4052076401 conn(0x55ce1690a090 msgr2=0x55ce1699c300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:09.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.562+0000 7f3de47d1b80 1 -- 172.21.15.64:0/4052076401 shutdown_connections 2023-09-13T03:14:09.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.562+0000 7f3de47d1b80 1 -- 172.21.15.64:0/4052076401 wait complete. 2023-09-13T03:14:09.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.562+0000 7f3de47d1b80 1 -- start start 2023-09-13T03:14:09.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.563+0000 7f3de47d1b80 1 Processor -- start 2023-09-13T03:14:09.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.563+0000 7f3de47d1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ce16995710 0x55ce169a4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:09.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.563+0000 7f3de47d1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce16996030 0x55ce169a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:09.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.564+0000 7f3de47d1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1699fe40 0x55ce169a59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:09.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.564+0000 7f3de47d1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce169a6fb0 con 0x55ce16996030 2023-09-13T03:14:09.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.564+0000 7f3de47d1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce16a032f0 con 0x55ce1699fe40 2023-09-13T03:14:09.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.564+0000 7f3de47d1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce16a03430 con 0x55ce16995710 2023-09-13T03:14:09.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.564+0000 7f3de47d1b80 10 client.0 _ll_register_callbacks cb 0x7f3dc4009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:14:09.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.564+0000 7f3dc3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce16996030 0x55ce169a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:09.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.564+0000 7f3dc3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce16996030 0x55ce169a54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59864/0 (socket says 172.21.15.64:59864) 2023-09-13T03:14:09.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.564+0000 7f3dc3fff700 1 -- 172.21.15.64:0/3155151260 learned_addr learned my addr 172.21.15.64:0/3155151260 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:09.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.564+0000 7f3dcad9d700 1 --2- 172.21.15.64:0/3155151260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1699fe40 0x55ce169a59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:09.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.564+0000 7f3dca59c700 1 --2- 172.21.15.64:0/3155151260 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ce16995710 0x55ce169a4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:09.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.564+0000 7f3dc3fff700 1 -- 172.21.15.64:0/3155151260 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ce16995710 msgr2=0x55ce169a4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:09.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.564+0000 7f3dc3fff700 1 --2- 172.21.15.64:0/3155151260 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ce16995710 0x55ce169a4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.564+0000 7f3dc3fff700 1 -- 172.21.15.64:0/3155151260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1699fe40 msgr2=0x55ce169a59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:09.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.564+0000 7f3dc3fff700 1 --2- 172.21.15.64:0/3155151260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1699fe40 0x55ce169a59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.564+0000 7f3dc3fff700 1 -- 172.21.15.64:0/3155151260 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc40092d0 con 0x55ce16996030 2023-09-13T03:14:09.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.564+0000 7f3dc3fff700 1 --2- 172.21.15.64:0/3155151260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce16996030 0x55ce169a54a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3db40960e0 tx=0x7f3db4099c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b43e4cbbfb4f500 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:09.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.564+0000 7f3db1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:14:09.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.565+0000 7f3db1ffb700 1 -- 172.21.15.64:0/3155151260 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db4099670 con 0x55ce16996030 2023-09-13T03:14:09.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.565+0000 7f3de47d1b80 1 -- 172.21.15.64:0/3155151260 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3db409d4a0 con 0x55ce16996030 2023-09-13T03:14:09.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.565+0000 7f3db1ffb700 1 -- 172.21.15.64:0/3155151260 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f3db40a16e0 con 0x55ce16996030 2023-09-13T03:14:09.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.565+0000 7f3db1ffb700 1 -- 172.21.15.64:0/3155151260 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3db4097a10 con 0x55ce16996030 2023-09-13T03:14:09.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.565+0000 7f3de47d1b80 10 client.15942 fetch_fsmap learned FSMap version 27 2023-09-13T03:14:09.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.565+0000 7f3de47d1b80 1 -- 172.21.15.64:0/3155151260 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f3dcc10dcf0 con 0x55ce16996030 2023-09-13T03:14:09.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.565+0000 7f3db1ffb700 1 -- 172.21.15.64:0/3155151260 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3db409d4a0 con 0x55ce16996030 2023-09-13T03:14:09.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.565+0000 7f3db1ffb700 1 -- 172.21.15.64:0/3155151260 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f3db40652b0 con 0x55ce16996030 2023-09-13T03:14:09.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.565+0000 7f3de47d1b80 10 client.15942 fetch_fsmap finished waiting for FSMap version 27 2023-09-13T03:14:09.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.565+0000 7f3de47d1b80 10 client.15942 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2023-09-13T03:14:09.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.565+0000 7f3de47d1b80 20 client.15942 populate_metadata read hostname 'smithi064' 2023-09-13T03:14:09.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.565+0000 7f3de47d1b80 1 --2- 172.21.15.64:0/3155151260 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x55ce16a046d0 0x55ce16a06af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:09.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.565+0000 7f3de47d1b80 4 client.15942 mds_command: new command op to 24622 tid=0[{"prefix": "damage ls"}] 2023-09-13T03:14:09.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.565+0000 7f3de47d1b80 1 -- 172.21.15.64:0/3155151260 --> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ce168e0920 con 0x55ce16a046d0 2023-09-13T03:14:09.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.566+0000 7f3dca59c700 1 --2- 172.21.15.64:0/3155151260 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x55ce16a046d0 0x55ce16a06af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:09.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.566+0000 7f3dca59c700 1 --2- 172.21.15.64:0/3155151260 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x55ce16a046d0 0x55ce16a06af0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3058f11b7d1944f server_cookie=e383e3c2c1b0193b in_seq=0 out_seq=0 2023-09-13T03:14:09.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.566+0000 7f3db1ffb700 10 client.15942 ms_handle_connect on v2:172.21.15.170:6834/1493371121 2023-09-13T03:14:09.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.567+0000 7f3db1ffb700 1 -- 172.21.15.64:0/3155151260 <== mds.4 v2:172.21.15.170:6834/1493371121 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ce168e0920 con 0x55ce16a046d0 2023-09-13T03:14:09.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.567+0000 7f3db1ffb700 10 client.15942 handle_command_reply: tid=0 2023-09-13T03:14:09.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.567+0000 7f3de47d1b80 1 client.15942 shutdown 2023-09-13T03:14:09.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.567+0000 7f3de47d1b80 1 -- 172.21.15.64:0/3155151260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce16996030 msgr2=0x55ce169a54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:09.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.567+0000 7f3de47d1b80 1 --2- 172.21.15.64:0/3155151260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce16996030 0x55ce169a54a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3db40960e0 tx=0x7f3db4099c70 comp rx=0 tx=0).stop 2023-09-13T03:14:09.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.567+0000 7f3de47d1b80 1 -- 172.21.15.64:0/3155151260 shutdown_connections 2023-09-13T03:14:09.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.567+0000 7f3de47d1b80 1 --2- 172.21.15.64:0/3155151260 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x55ce16a046d0 0x55ce16a06af0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.567+0000 7f3de47d1b80 1 --2- 172.21.15.64:0/3155151260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1699fe40 0x55ce169a59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.567+0000 7f3de47d1b80 1 --2- 172.21.15.64:0/3155151260 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ce16995710 0x55ce169a4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.567+0000 7f3db1ffb700 0 client.15942 ms_handle_reset on v2:172.21.15.170:6834/1493371121 2023-09-13T03:14:09.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.567+0000 7f3de47d1b80 1 --2- 172.21.15.64:0/3155151260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce16996030 0x55ce169a54a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.567+0000 7f3de47d1b80 1 -- 172.21.15.64:0/3155151260 >> 172.21.15.64:0/3155151260 conn(0x55ce1690a090 msgr2=0x55ce16994700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:09.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.567+0000 7f3de47d1b80 1 -- 172.21.15.64:0/3155151260 shutdown_connections 2023-09-13T03:14:09.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.567+0000 7f3de47d1b80 1 -- 172.21.15.64:0/3155151260 wait complete. 2023-09-13T03:14:09.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.568+0000 7f3de47d1b80 20 client.15942 trim_cache size 0 max 16384 2023-09-13T03:14:09.592 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2023-09-13T03:14:09.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.568+0000 7f3dd0dff700 1 -- 172.21.15.64:0/965170645 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f3db80a08d0 msgr2=0x7f3db80a2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:09.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.568+0000 7f3dd0dff700 1 --2- 172.21.15.64:0/965170645 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f3db80a08d0 0x7f3db80a2d90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc009990 tx=0x7f3dbc005fb0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.568+0000 7f3dd0dff700 1 -- 172.21.15.64:0/965170645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3dcc100f30 msgr2=0x7f3dcc19eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:09.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.568+0000 7f3dd0dff700 1 --2- 172.21.15.64:0/965170645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3dcc100f30 0x7f3dcc19eb50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3db400e9a0 tx=0x7f3db400ecb0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.568+0000 7f3dd0dff700 1 -- 172.21.15.64:0/965170645 shutdown_connections 2023-09-13T03:14:09.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.568+0000 7f3dd0dff700 1 --2- 172.21.15.64:0/965170645 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f3db80a08d0 0x7f3db80a2d90 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.568+0000 7f3dd0dff700 1 --2- 172.21.15.64:0/965170645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3dcc106630 0x7f3dcc198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.568+0000 7f3dd0dff700 1 --2- 172.21.15.64:0/965170645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3dcc100570 0x7f3dcc19e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.568+0000 7f3dd0dff700 1 --2- 172.21.15.64:0/965170645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3dcc100f30 0x7f3dcc19eb50 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.568+0000 7f3dd0dff700 1 -- 172.21.15.64:0/965170645 >> 172.21.15.64:0/965170645 conn(0x7f3dcc0fc080 msgr2=0x7f3dcc0fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:09.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.568+0000 7f3dd0dff700 1 -- 172.21.15.64:0/965170645 shutdown_connections 2023-09-13T03:14:09.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.568+0000 7f3dd0dff700 1 -- 172.21.15.64:0/965170645 wait complete. 2023-09-13T03:14:09.596 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-09-13T03:14:09.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.806+0000 7f87c63ff700 1 -- 172.21.15.64:0/2637322317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87c0100f50 msgr2=0x7f87c010d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:09.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.806+0000 7f87c63ff700 1 --2- 172.21.15.64:0/2637322317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87c0100f50 0x7f87c010d410 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f87b800b0d0 tx=0x7f87b800b3e0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.806+0000 7f87c63ff700 1 -- 172.21.15.64:0/2637322317 shutdown_connections 2023-09-13T03:14:09.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.806+0000 7f87c63ff700 1 --2- 172.21.15.64:0/2637322317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87c0106650 0x7f87c0106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.806+0000 7f87c63ff700 1 --2- 172.21.15.64:0/2637322317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87c0100f50 0x7f87c010d410 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.806+0000 7f87c63ff700 1 --2- 172.21.15.64:0/2637322317 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87c0100590 0x7f87c0100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.806+0000 7f87c63ff700 1 -- 172.21.15.64:0/2637322317 >> 172.21.15.64:0/2637322317 conn(0x7f87c00fc060 msgr2=0x7f87c00fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:09.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.806+0000 7f87c63ff700 1 -- 172.21.15.64:0/2637322317 shutdown_connections 2023-09-13T03:14:09.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.806+0000 7f87c63ff700 1 -- 172.21.15.64:0/2637322317 wait complete. 2023-09-13T03:14:09.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.807+0000 7f87c63ff700 1 Processor -- start 2023-09-13T03:14:09.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.807+0000 7f87c63ff700 1 -- start start 2023-09-13T03:14:09.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.808+0000 7f87c63ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87c0100590 0x7f87c019e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:09.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.808+0000 7f87c63ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87c0100f50 0x7f87c019ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:09.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.808+0000 7f87c63ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87c0106650 0x7f87c0198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:09.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.808+0000 7f87c63ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c0198da0 con 0x7f87c0100f50 2023-09-13T03:14:09.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.808+0000 7f87c63ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c0198f10 con 0x7f87c0106650 2023-09-13T03:14:09.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.808+0000 7f87c63ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c01990d0 con 0x7f87c0100590 2023-09-13T03:14:09.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.808+0000 7f87bffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87c0100590 0x7f87c019e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:09.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.808+0000 7f87bf7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87c0100f50 0x7f87c019ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:09.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.808+0000 7f87bffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87c0100590 0x7f87c019e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:36882/0 (socket says 172.21.15.64:36882) 2023-09-13T03:14:09.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.808+0000 7f87bffff700 1 -- 172.21.15.64:0/3870024512 learned_addr learned my addr 172.21.15.64:0/3870024512 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:09.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.808+0000 7f87c499c700 1 --2- 172.21.15.64:0/3870024512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87c0106650 0x7f87c0198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:09.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.808+0000 7f87bf7fe700 1 -- 172.21.15.64:0/3870024512 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87c0100590 msgr2=0x7f87c019e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:09.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.808+0000 7f87bf7fe700 1 --2- 172.21.15.64:0/3870024512 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87c0100590 0x7f87c019e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.808+0000 7f87bf7fe700 1 -- 172.21.15.64:0/3870024512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87c0106650 msgr2=0x7f87c0198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:09.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.808+0000 7f87bf7fe700 1 --2- 172.21.15.64:0/3870024512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87c0106650 0x7f87c0198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.808+0000 7f87bf7fe700 1 -- 172.21.15.64:0/3870024512 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b8009d00 con 0x7f87c0100f50 2023-09-13T03:14:09.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.809+0000 7f87bf7fe700 1 --2- 172.21.15.64:0/3870024512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87c0100f50 0x7f87c019ec00 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f87b400d950 tx=0x7f87b400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:09.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.809+0000 7f87bd7fa700 1 -- 172.21.15.64:0/3870024512 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b400f890 con 0x7f87c0100f50 2023-09-13T03:14:09.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.809+0000 7f87bd7fa700 1 -- 172.21.15.64:0/3870024512 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87b4010460 con 0x7f87c0100f50 2023-09-13T03:14:09.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.809+0000 7f87c63ff700 1 -- 172.21.15.64:0/3870024512 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87c01993b0 con 0x7f87c0100f50 2023-09-13T03:14:09.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.809+0000 7f87bd7fa700 1 -- 172.21.15.64:0/3870024512 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b400e610 con 0x7f87c0100f50 2023-09-13T03:14:09.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.809+0000 7f87c63ff700 1 -- 172.21.15.64:0/3870024512 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87c01a56b0 con 0x7f87c0100f50 2023-09-13T03:14:09.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.814+0000 7f87bd7fa700 1 -- 172.21.15.64:0/3870024512 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) v1 ==== 135492+0+0 (secure 0 0 0) 0x7f87b40105d0 con 0x7f87c0100f50 2023-09-13T03:14:09.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.815+0000 7f87d9c88b80 1 Processor -- start 2023-09-13T03:14:09.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.815+0000 7f87d9c88b80 1 -- start start 2023-09-13T03:14:09.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.815+0000 7f87bd7fa700 1 --2- 172.21.15.64:0/3870024512 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f87a80a0950 0x7f87a80a2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:09.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.815+0000 7f87d9c88b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556408159710 0x556408159af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:09.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.815+0000 7f87d9c88b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55640815a030 0x556408163900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:09.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.815+0000 7f87d9c88b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556408163e40 0x556408166230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:09.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.815+0000 7f87d9c88b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564080b2730 con 0x556408159710 2023-09-13T03:14:09.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.815+0000 7f87d9c88b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564080b0050 con 0x556408163e40 2023-09-13T03:14:09.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.815+0000 7f87d9c88b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564080ab940 con 0x55640815a030 2023-09-13T03:14:09.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.815+0000 7f87bd7fa700 1 -- 172.21.15.64:0/3870024512 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f87b40cbe90 con 0x7f87c0100f50 2023-09-13T03:14:09.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.816+0000 7f87bffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556408159710 0x556408159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:09.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.816+0000 7f87bf7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55640815a030 0x556408163900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:09.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.816+0000 7f87bf7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55640815a030 0x556408163900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:36886/0 (socket says 172.21.15.64:36886) 2023-09-13T03:14:09.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.816+0000 7f87c499c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556408163e40 0x556408166230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:09.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.816+0000 7f87bf7fe700 1 -- 172.21.15.64:0/1352321603 learned_addr learned my addr 172.21.15.64:0/1352321603 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:09.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.816+0000 7f87c499c700 1 --2- 172.21.15.64:0/3870024512 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f87a80a0950 0x7f87a80a2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:09.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.816+0000 7f87bf7fe700 1 -- 172.21.15.64:0/1352321603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556408163e40 msgr2=0x556408166230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:09.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.816+0000 7f87bf7fe700 1 --2- 172.21.15.64:0/1352321603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556408163e40 0x556408166230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.816+0000 7f87bf7fe700 1 -- 172.21.15.64:0/1352321603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556408159710 msgr2=0x556408159af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:09.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.816+0000 7f87bf7fe700 1 --2- 172.21.15.64:0/1352321603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556408159710 0x556408159af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.816+0000 7f87bf7fe700 1 -- 172.21.15.64:0/1352321603 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556407ed7de0 con 0x55640815a030 2023-09-13T03:14:09.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.816+0000 7f87bf7fe700 1 --2- 172.21.15.64:0/1352321603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55640815a030 0x556408163900 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f87b40955e0 tx=0x7f87b4098ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b48cac3eacd08bf4 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:09.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.816+0000 7f87c499c700 1 --2- 172.21.15.64:0/3870024512 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f87a80a0950 0x7f87a80a2e10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f87b8009cd0 tx=0x7f87b800b040 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:09.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.816+0000 7f87a67fc700 1 -- 172.21.15.64:0/1352321603 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b40990e0 con 0x55640815a030 2023-09-13T03:14:09.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.817+0000 7f87a67fc700 1 -- 172.21.15.64:0/1352321603 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87b409cf10 con 0x55640815a030 2023-09-13T03:14:09.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.817+0000 7f87a67fc700 1 -- 172.21.15.64:0/1352321603 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b40a0cc0 con 0x55640815a030 2023-09-13T03:14:09.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.817+0000 7f87d9c88b80 1 -- 172.21.15.64:0/1352321603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55640815a030 msgr2=0x556408163900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:09.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.817+0000 7f87d9c88b80 1 --2- 172.21.15.64:0/1352321603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55640815a030 0x556408163900 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f87b40955e0 tx=0x7f87b4098ac0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.817+0000 7f87d9c88b80 1 -- 172.21.15.64:0/1352321603 shutdown_connections 2023-09-13T03:14:09.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.817+0000 7f87d9c88b80 1 --2- 172.21.15.64:0/1352321603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556408163e40 0x556408166230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.817+0000 7f87d9c88b80 1 --2- 172.21.15.64:0/1352321603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556408159710 0x556408159af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.817+0000 7f87d9c88b80 1 --2- 172.21.15.64:0/1352321603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55640815a030 0x556408163900 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.817+0000 7f87d9c88b80 1 -- 172.21.15.64:0/1352321603 >> 172.21.15.64:0/1352321603 conn(0x5564080ce090 msgr2=0x556408154bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:09.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.817+0000 7f87d9c88b80 1 -- 172.21.15.64:0/1352321603 shutdown_connections 2023-09-13T03:14:09.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.817+0000 7f87d9c88b80 1 -- 172.21.15.64:0/1352321603 wait complete. 2023-09-13T03:14:09.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.818+0000 7f87d9c88b80 1 -- start start 2023-09-13T03:14:09.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.818+0000 7f87d9c88b80 1 Processor -- start 2023-09-13T03:14:09.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.819+0000 7f87d9c88b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556408159710 0x556408163820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:09.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.819+0000 7f87d9c88b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55640815a030 0x556408160a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:09.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.819+0000 7f87d9c88b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556408163e40 0x556408161020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:09.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.819+0000 7f87d9c88b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x556408049b70 con 0x556408159710 2023-09-13T03:14:09.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.819+0000 7f87d9c88b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556408049870 con 0x55640815a030 2023-09-13T03:14:09.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.819+0000 7f87d9c88b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x556407f2d920 con 0x556408163e40 2023-09-13T03:14:09.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.819+0000 7f87d9c88b80 10 client.0 _ll_register_callbacks cb 0x556407f93a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:14:09.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.819+0000 7f87bffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556408159710 0x556408163820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:09.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.819+0000 7f87bffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556408159710 0x556408163820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59896/0 (socket says 172.21.15.64:59896) 2023-09-13T03:14:09.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.819+0000 7f87bffff700 1 -- 172.21.15.64:0/1601316098 learned_addr learned my addr 172.21.15.64:0/1601316098 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:09.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.819+0000 7f87bf7fe700 1 --2- 172.21.15.64:0/1601316098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55640815a030 0x556408160a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:09.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.819+0000 7f87c499c700 1 --2- 172.21.15.64:0/1601316098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556408163e40 0x556408161020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:09.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.819+0000 7f87bffff700 1 -- 172.21.15.64:0/1601316098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556408163e40 msgr2=0x556408161020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:09.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.819+0000 7f87bffff700 1 --2- 172.21.15.64:0/1601316098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556408163e40 0x556408161020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.819+0000 7f87bffff700 1 -- 172.21.15.64:0/1601316098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55640815a030 msgr2=0x556408160a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:09.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.819+0000 7f87bffff700 1 --2- 172.21.15.64:0/1601316098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55640815a030 0x556408160a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.819+0000 7f87bffff700 1 -- 172.21.15.64:0/1601316098 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87c01a56b0 con 0x556408159710 2023-09-13T03:14:09.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.819+0000 7f87bffff700 1 --2- 172.21.15.64:0/1601316098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556408159710 0x556408163820 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f87b0012840 tx=0x7f87b0012b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da711d8506db49f server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:09.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.820+0000 7f879f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:14:09.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.820+0000 7f879f7fe700 1 -- 172.21.15.64:0/1601316098 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b0017070 con 0x556408159710 2023-09-13T03:14:09.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.820+0000 7f879f7fe700 1 -- 172.21.15.64:0/1601316098 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f87b0004020 con 0x556408159710 2023-09-13T03:14:09.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.820+0000 7f87d9c88b80 1 -- 172.21.15.64:0/1601316098 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5564080034f0 con 0x556408159710 2023-09-13T03:14:09.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.820+0000 7f879f7fe700 1 -- 172.21.15.64:0/1601316098 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87b00122e0 con 0x556408159710 2023-09-13T03:14:09.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.820+0000 7f87d9c88b80 10 client.15954 fetch_fsmap learned FSMap version 28 2023-09-13T03:14:09.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.820+0000 7f87d9c88b80 1 -- 172.21.15.64:0/1601316098 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x556407ec0c60 con 0x556408159710 2023-09-13T03:14:09.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.820+0000 7f879f7fe700 1 -- 172.21.15.64:0/1601316098 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5564080034f0 con 0x556408159710 2023-09-13T03:14:09.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.820+0000 7f879f7fe700 1 -- 172.21.15.64:0/1601316098 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f87b000f540 con 0x556408159710 2023-09-13T03:14:09.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.821+0000 7f87d9c88b80 10 client.15954 fetch_fsmap finished waiting for FSMap version 28 2023-09-13T03:14:09.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.821+0000 7f87d9c88b80 10 client.15954 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2023-09-13T03:14:09.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.821+0000 7f87d9c88b80 20 client.15954 populate_metadata read hostname 'smithi064' 2023-09-13T03:14:09.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.821+0000 7f87d9c88b80 1 --2- 172.21.15.64:0/1601316098 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x5564081738a0 0x556408175cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:09.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.821+0000 7f87d9c88b80 4 client.15954 mds_command: new command op to 24634 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:14:09.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.821+0000 7f87d9c88b80 1 -- 172.21.15.64:0/1601316098 --> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5564080c08a0 con 0x5564081738a0 2023-09-13T03:14:09.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.821+0000 7f87bf7fe700 1 --2- 172.21.15.64:0/1601316098 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x5564081738a0 0x556408175cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:09.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.822+0000 7f87bf7fe700 1 --2- 172.21.15.64:0/1601316098 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x5564081738a0 0x556408175cc0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=faea5d3e757bd05 server_cookie=9ad6cd1ead109255 in_seq=0 out_seq=0 2023-09-13T03:14:09.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.822+0000 7f879f7fe700 10 client.15954 ms_handle_connect on v2:172.21.15.170:6836/1726587275 2023-09-13T03:14:09.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.822+0000 7f879f7fe700 1 -- 172.21.15.64:0/1601316098 <== mds.3 v2:172.21.15.170:6836/1726587275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5564080c08a0 con 0x5564081738a0 2023-09-13T03:14:09.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.822+0000 7f879f7fe700 10 client.15954 handle_command_reply: tid=0 2023-09-13T03:14:09.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.822+0000 7f87d9c88b80 1 client.15954 shutdown 2023-09-13T03:14:09.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.823+0000 7f87d9c88b80 1 -- 172.21.15.64:0/1601316098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556408159710 msgr2=0x556408163820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:09.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.823+0000 7f87d9c88b80 1 --2- 172.21.15.64:0/1601316098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556408159710 0x556408163820 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f87b0012840 tx=0x7f87b0012b50 comp rx=0 tx=0).stop 2023-09-13T03:14:09.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.823+0000 7f87d9c88b80 1 -- 172.21.15.64:0/1601316098 shutdown_connections 2023-09-13T03:14:09.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.823+0000 7f87d9c88b80 1 --2- 172.21.15.64:0/1601316098 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x5564081738a0 0x556408175cc0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.823+0000 7f87d9c88b80 1 --2- 172.21.15.64:0/1601316098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556408163e40 0x556408161020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.823+0000 7f87d9c88b80 1 --2- 172.21.15.64:0/1601316098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556408159710 0x556408163820 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.823+0000 7f879f7fe700 0 client.15954 ms_handle_reset on v2:172.21.15.170:6836/1726587275 2023-09-13T03:14:09.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.823+0000 7f87d9c88b80 1 --2- 172.21.15.64:0/1601316098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55640815a030 0x556408160a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.823+0000 7f87d9c88b80 1 -- 172.21.15.64:0/1601316098 >> 172.21.15.64:0/1601316098 conn(0x5564080ce090 msgr2=0x5564081547e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:09.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.823+0000 7f87d9c88b80 1 -- 172.21.15.64:0/1601316098 shutdown_connections 2023-09-13T03:14:09.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.823+0000 7f87d9c88b80 1 -- 172.21.15.64:0/1601316098 wait complete. 2023-09-13T03:14:09.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.823+0000 7f87d9c88b80 20 client.15954 trim_cache size 0 max 16384 2023-09-13T03:14:09.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.836+0000 7f87d9c88b80 1 Processor -- start 2023-09-13T03:14:09.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.836+0000 7f87d9c88b80 1 -- start start 2023-09-13T03:14:09.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.837+0000 7f87d9c88b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556408159710 0x556408166230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:09.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.837+0000 7f87d9c88b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55640815a030 0x55640816fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:09.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.837+0000 7f87d9c88b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556408163e40 0x556408163350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:09.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.837+0000 7f87d9c88b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55640815a410 con 0x556408159710 2023-09-13T03:14:09.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.837+0000 7f87d9c88b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55640815b280 con 0x556408163e40 2023-09-13T03:14:09.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.837+0000 7f87d9c88b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55640815c5b0 con 0x55640815a030 2023-09-13T03:14:09.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.837+0000 7f87c499c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556408163e40 0x556408163350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:09.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.837+0000 7f87c499c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556408163e40 0x556408163350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:53680/0 (socket says 172.21.15.64:53680) 2023-09-13T03:14:09.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.837+0000 7f87c499c700 1 -- 172.21.15.64:0/1881509526 learned_addr learned my addr 172.21.15.64:0/1881509526 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:09.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.837+0000 7f87bf7fe700 1 --2- 172.21.15.64:0/1881509526 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55640815a030 0x55640816fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:09.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.837+0000 7f87bffff700 1 --2- 172.21.15.64:0/1881509526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556408159710 0x556408166230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:09.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.837+0000 7f87c499c700 1 -- 172.21.15.64:0/1881509526 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55640815a030 msgr2=0x55640816fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:09.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.838+0000 7f87c499c700 1 --2- 172.21.15.64:0/1881509526 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55640815a030 0x55640816fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.838+0000 7f87c499c700 1 -- 172.21.15.64:0/1881509526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556408159710 msgr2=0x556408166230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:09.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.838+0000 7f87c499c700 1 --2- 172.21.15.64:0/1881509526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556408159710 0x556408166230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.838+0000 7f87c499c700 1 -- 172.21.15.64:0/1881509526 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556407efe250 con 0x556408163e40 2023-09-13T03:14:09.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.838+0000 7f87bf7fe700 1 --2- 172.21.15.64:0/1881509526 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55640815a030 0x55640816fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:09.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.838+0000 7f87c499c700 1 --2- 172.21.15.64:0/1881509526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556408163e40 0x556408163350 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f87b800b7f0 tx=0x7f87b8013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7734b6e86ed79b3a server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:09.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.838+0000 7f87a6ffd700 1 -- 172.21.15.64:0/1881509526 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b800e040 con 0x556408163e40 2023-09-13T03:14:09.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.838+0000 7f87a6ffd700 1 -- 172.21.15.64:0/1881509526 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87b8029040 con 0x556408163e40 2023-09-13T03:14:09.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.838+0000 7f87a6ffd700 1 -- 172.21.15.64:0/1881509526 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b801e460 con 0x556408163e40 2023-09-13T03:14:09.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.839+0000 7f87d9c88b80 1 -- 172.21.15.64:0/1881509526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556408163e40 msgr2=0x556408163350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:09.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.839+0000 7f87d9c88b80 1 --2- 172.21.15.64:0/1881509526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556408163e40 0x556408163350 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f87b800b7f0 tx=0x7f87b8013000 comp rx=0 tx=0).stop 2023-09-13T03:14:09.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.839+0000 7f87d9c88b80 1 -- 172.21.15.64:0/1881509526 shutdown_connections 2023-09-13T03:14:09.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.839+0000 7f87d9c88b80 1 --2- 172.21.15.64:0/1881509526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556408163e40 0x556408163350 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.839+0000 7f87d9c88b80 1 --2- 172.21.15.64:0/1881509526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556408159710 0x556408166230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.839+0000 7f87d9c88b80 1 --2- 172.21.15.64:0/1881509526 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55640815a030 0x55640816fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.839+0000 7f87d9c88b80 1 -- 172.21.15.64:0/1881509526 >> 172.21.15.64:0/1881509526 conn(0x5564080ce090 msgr2=0x556408153320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:09.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.839+0000 7f87d9c88b80 1 -- 172.21.15.64:0/1881509526 shutdown_connections 2023-09-13T03:14:09.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.839+0000 7f87d9c88b80 1 -- 172.21.15.64:0/1881509526 wait complete. 2023-09-13T03:14:09.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.839+0000 7f87d9c88b80 1 -- start start 2023-09-13T03:14:09.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.839+0000 7f87d9c88b80 1 Processor -- start 2023-09-13T03:14:09.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.840+0000 7f87d9c88b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556408159710 0x55640816a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:09.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.840+0000 7f87d9c88b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55640815a030 0x556408168a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:09.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.840+0000 7f87d9c88b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556408163e40 0x556408168fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:09.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.840+0000 7f87d9c88b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x556408169510 con 0x556408163e40 2023-09-13T03:14:09.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.840+0000 7f87d9c88b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556408169680 con 0x55640815a030 2023-09-13T03:14:09.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.840+0000 7f87d9c88b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x556408169840 con 0x556408159710 2023-09-13T03:14:09.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.840+0000 7f87d9c88b80 10 client.0 _ll_register_callbacks cb 0x7f87b0009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:14:09.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.840+0000 7f87c499c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556408163e40 0x556408168fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:09.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.840+0000 7f87bf7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55640815a030 0x556408168a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:09.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.840+0000 7f87bf7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55640815a030 0x556408168a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:53686/0 (socket says 172.21.15.64:53686) 2023-09-13T03:14:09.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.840+0000 7f87bf7fe700 1 -- 172.21.15.64:0/2876421815 learned_addr learned my addr 172.21.15.64:0/2876421815 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:09.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.840+0000 7f87bffff700 1 --2- 172.21.15.64:0/2876421815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556408159710 0x55640816a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:09.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.841+0000 7f87bf7fe700 1 -- 172.21.15.64:0/2876421815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556408159710 msgr2=0x55640816a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:09.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.841+0000 7f87bf7fe700 1 --2- 172.21.15.64:0/2876421815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556408159710 0x55640816a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.841+0000 7f87bf7fe700 1 -- 172.21.15.64:0/2876421815 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556408163e40 msgr2=0x556408168fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:09.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.841+0000 7f87bf7fe700 1 --2- 172.21.15.64:0/2876421815 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556408163e40 0x556408168fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.841+0000 7f87bf7fe700 1 -- 172.21.15.64:0/2876421815 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b409c040 con 0x55640815a030 2023-09-13T03:14:09.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.841+0000 7f87bf7fe700 1 --2- 172.21.15.64:0/2876421815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55640815a030 0x556408168a90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f87b4005950 tx=0x7f87b4098fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34ef425c3c1f3dd6 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:09.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.841+0000 7f879f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:14:09.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.841+0000 7f879f7fe700 1 -- 172.21.15.64:0/2876421815 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b4099160 con 0x55640815a030 2023-09-13T03:14:09.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.841+0000 7f87d9c88b80 1 -- 172.21.15.64:0/2876421815 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f87b409cf10 con 0x55640815a030 2023-09-13T03:14:09.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.841+0000 7f879f7fe700 1 -- 172.21.15.64:0/2876421815 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f87b40985d0 con 0x55640815a030 2023-09-13T03:14:09.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.841+0000 7f879f7fe700 1 -- 172.21.15.64:0/2876421815 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87b4098790 con 0x55640815a030 2023-09-13T03:14:09.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.841+0000 7f87d9c88b80 10 client.25997 fetch_fsmap learned FSMap version 28 2023-09-13T03:14:09.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.841+0000 7f87d9c88b80 1 -- 172.21.15.64:0/2876421815 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f87c010dd10 con 0x55640815a030 2023-09-13T03:14:09.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.842+0000 7f879f7fe700 1 -- 172.21.15.64:0/2876421815 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87b40b5080 con 0x55640815a030 2023-09-13T03:14:09.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.842+0000 7f879f7fe700 1 -- 172.21.15.64:0/2876421815 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f87b40c9d20 con 0x55640815a030 2023-09-13T03:14:09.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.842+0000 7f87d9c88b80 10 client.25997 fetch_fsmap finished waiting for FSMap version 28 2023-09-13T03:14:09.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.842+0000 7f87d9c88b80 10 client.25997 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2023-09-13T03:14:09.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.842+0000 7f87d9c88b80 20 client.25997 populate_metadata read hostname 'smithi064' 2023-09-13T03:14:09.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.842+0000 7f87d9c88b80 1 --2- 172.21.15.64:0/2876421815 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x5564081c8740 0x5564081cab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:09.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.842+0000 7f87d9c88b80 4 client.25997 mds_command: new command op to 24634 tid=0[{"prefix": "damage ls"}] 2023-09-13T03:14:09.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.842+0000 7f87d9c88b80 1 -- 172.21.15.64:0/2876421815 --> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5564080a4920 con 0x5564081c8740 2023-09-13T03:14:09.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.842+0000 7f87bffff700 1 --2- 172.21.15.64:0/2876421815 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x5564081c8740 0x5564081cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:09.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.843+0000 7f879f7fe700 10 client.25997 ms_handle_connect on v2:172.21.15.170:6836/1726587275 2023-09-13T03:14:09.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.843+0000 7f87bffff700 1 --2- 172.21.15.64:0/2876421815 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x5564081c8740 0x5564081cab60 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cfa19429b38975f6 server_cookie=9a8fda44ca48d108 in_seq=0 out_seq=0 2023-09-13T03:14:09.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.843+0000 7f879f7fe700 1 -- 172.21.15.64:0/2876421815 <== mds.3 v2:172.21.15.170:6836/1726587275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5564080a4920 con 0x5564081c8740 2023-09-13T03:14:09.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.843+0000 7f879f7fe700 10 client.25997 handle_command_reply: tid=0 2023-09-13T03:14:09.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.843+0000 7f87d9c88b80 1 client.25997 shutdown 2023-09-13T03:14:09.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.843+0000 7f87d9c88b80 1 -- 172.21.15.64:0/2876421815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55640815a030 msgr2=0x556408168a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:09.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.843+0000 7f87d9c88b80 1 --2- 172.21.15.64:0/2876421815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55640815a030 0x556408168a90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f87b4005950 tx=0x7f87b4098fe0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.843+0000 7f87d9c88b80 1 -- 172.21.15.64:0/2876421815 shutdown_connections 2023-09-13T03:14:09.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.843+0000 7f87d9c88b80 1 --2- 172.21.15.64:0/2876421815 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x5564081c8740 0x5564081cab60 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.843+0000 7f87d9c88b80 1 --2- 172.21.15.64:0/2876421815 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556408163e40 0x556408168fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.843+0000 7f87d9c88b80 1 --2- 172.21.15.64:0/2876421815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556408159710 0x55640816a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.843+0000 7f87d9c88b80 1 --2- 172.21.15.64:0/2876421815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55640815a030 0x556408168a90 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.843+0000 7f879f7fe700 0 client.25997 ms_handle_reset on v2:172.21.15.170:6836/1726587275 2023-09-13T03:14:09.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.843+0000 7f87d9c88b80 1 -- 172.21.15.64:0/2876421815 >> 172.21.15.64:0/2876421815 conn(0x5564080ce090 msgr2=0x556408160260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:09.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.843+0000 7f87d9c88b80 1 -- 172.21.15.64:0/2876421815 shutdown_connections 2023-09-13T03:14:09.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.843+0000 7f87d9c88b80 1 -- 172.21.15.64:0/2876421815 wait complete. 2023-09-13T03:14:09.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.843+0000 7f87d9c88b80 20 client.25997 trim_cache size 0 max 16384 2023-09-13T03:14:09.865 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2023-09-13T03:14:09.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.844+0000 7f87c63ff700 1 -- 172.21.15.64:0/3870024512 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f87a80a0950 msgr2=0x7f87a80a2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:09.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.844+0000 7f87c63ff700 1 --2- 172.21.15.64:0/3870024512 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f87a80a0950 0x7f87a80a2e10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f87b8009cd0 tx=0x7f87b800b040 comp rx=0 tx=0).stop 2023-09-13T03:14:09.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.844+0000 7f87c63ff700 1 -- 172.21.15.64:0/3870024512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87c0100f50 msgr2=0x7f87c019ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:09.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.844+0000 7f87c63ff700 1 --2- 172.21.15.64:0/3870024512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87c0100f50 0x7f87c019ec00 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f87b400d950 tx=0x7f87b400dc60 comp rx=0 tx=0).stop 2023-09-13T03:14:09.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.844+0000 7f87c63ff700 1 -- 172.21.15.64:0/3870024512 shutdown_connections 2023-09-13T03:14:09.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.844+0000 7f87c63ff700 1 --2- 172.21.15.64:0/3870024512 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f87a80a0950 0x7f87a80a2e10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.844+0000 7f87c63ff700 1 --2- 172.21.15.64:0/3870024512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87c0106650 0x7f87c0198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.844+0000 7f87c63ff700 1 --2- 172.21.15.64:0/3870024512 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87c0100590 0x7f87c019e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.844+0000 7f87c63ff700 1 --2- 172.21.15.64:0/3870024512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87c0100f50 0x7f87c019ec00 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:09.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.844+0000 7f87c63ff700 1 -- 172.21.15.64:0/3870024512 >> 172.21.15.64:0/3870024512 conn(0x7f87c00fc060 msgr2=0x7f87c010b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:09.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.844+0000 7f87c63ff700 1 -- 172.21.15.64:0/3870024512 shutdown_connections 2023-09-13T03:14:09.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:09.844+0000 7f87c63ff700 1 -- 172.21.15.64:0/3870024512 wait complete. 2023-09-13T03:14:09.869 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-09-13T03:14:09.884 INFO:tasks.workunit.client.0.smithi064.stdout: 1 25247 11.06 MB/sec warmup 14 sec latency 4308.878 ms 2023-09-13T03:14:10.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.074+0000 7f90ac3ff700 1 -- 172.21.15.64:0/3241728423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90a4100570 msgr2=0x7f90a41009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:10.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.074+0000 7f90ac3ff700 1 --2- 172.21.15.64:0/3241728423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90a4100570 0x7f90a41009f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f90940099c0 tx=0x7f9094009cd0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.075+0000 7f90ac3ff700 1 -- 172.21.15.64:0/3241728423 shutdown_connections 2023-09-13T03:14:10.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.075+0000 7f90ac3ff700 1 --2- 172.21.15.64:0/3241728423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90a4106630 0x7f90a4106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.075+0000 7f90ac3ff700 1 --2- 172.21.15.64:0/3241728423 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90a4100f30 0x7f90a410d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.075+0000 7f90ac3ff700 1 --2- 172.21.15.64:0/3241728423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90a4100570 0x7f90a41009f0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.075+0000 7f90ac3ff700 1 -- 172.21.15.64:0/3241728423 >> 172.21.15.64:0/3241728423 conn(0x7f90a40fc080 msgr2=0x7f90a40fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:10.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.075+0000 7f90ac3ff700 1 -- 172.21.15.64:0/3241728423 shutdown_connections 2023-09-13T03:14:10.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.075+0000 7f90ac3ff700 1 -- 172.21.15.64:0/3241728423 wait complete. 2023-09-13T03:14:10.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.076+0000 7f90ac3ff700 1 Processor -- start 2023-09-13T03:14:10.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.076+0000 7f90ac3ff700 1 -- start start 2023-09-13T03:14:10.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.076+0000 7f90ac3ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90a4100570 0x7f90a419e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.076+0000 7f90ac3ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90a4100f30 0x7f90a419eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.076+0000 7f90ac3ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90a4106630 0x7f90a4198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.076+0000 7f90ac3ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a4198ca0 con 0x7f90a4106630 2023-09-13T03:14:10.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.076+0000 7f90ac3ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a4198e10 con 0x7f90a4100f30 2023-09-13T03:14:10.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.076+0000 7f90ac3ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a4198fd0 con 0x7f90a4100570 2023-09-13T03:14:10.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.077+0000 7f90aa99c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90a4106630 0x7f90a4198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.077+0000 7f90aa19b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90a4100570 0x7f90a419e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.077+0000 7f90a999a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90a4100f30 0x7f90a419eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.077+0000 7f90aa99c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90a4106630 0x7f90a4198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59934/0 (socket says 172.21.15.64:59934) 2023-09-13T03:14:10.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.077+0000 7f90aa99c700 1 -- 172.21.15.64:0/516415503 learned_addr learned my addr 172.21.15.64:0/516415503 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:10.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.077+0000 7f90aa99c700 1 -- 172.21.15.64:0/516415503 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90a4100570 msgr2=0x7f90a419e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:10.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.077+0000 7f90aa99c700 1 --2- 172.21.15.64:0/516415503 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90a4100570 0x7f90a419e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.077+0000 7f90aa99c700 1 -- 172.21.15.64:0/516415503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90a4100f30 msgr2=0x7f90a419eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:10.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.077+0000 7f90aa99c700 1 --2- 172.21.15.64:0/516415503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90a4100f30 0x7f90a419eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.077+0000 7f90aa99c700 1 -- 172.21.15.64:0/516415503 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a0009520 con 0x7f90a4106630 2023-09-13T03:14:10.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.077+0000 7f90aa99c700 1 --2- 172.21.15.64:0/516415503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90a4106630 0x7f90a4198630 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f909c00ef30 tx=0x7f909c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:10.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.078+0000 7f909b7fe700 1 -- 172.21.15.64:0/516415503 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909c018070 con 0x7f90a4106630 2023-09-13T03:14:10.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.078+0000 7f909b7fe700 1 -- 172.21.15.64:0/516415503 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f909c00f040 con 0x7f90a4106630 2023-09-13T03:14:10.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.078+0000 7f90ac3ff700 1 -- 172.21.15.64:0/516415503 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90940095f0 con 0x7f90a4106630 2023-09-13T03:14:10.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.078+0000 7f909b7fe700 1 -- 172.21.15.64:0/516415503 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909c00cd00 con 0x7f90a4106630 2023-09-13T03:14:10.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.078+0000 7f90ac3ff700 1 -- 172.21.15.64:0/516415503 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90a41a53d0 con 0x7f90a4106630 2023-09-13T03:14:10.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.083+0000 7f909b7fe700 1 -- 172.21.15.64:0/516415503 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) v1 ==== 135492+0+0 (secure 0 0 0) 0x7f909c003a00 con 0x7f90a4106630 2023-09-13T03:14:10.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.084+0000 7f90bfcf1b80 1 Processor -- start 2023-09-13T03:14:10.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.084+0000 7f90bfcf1b80 1 -- start start 2023-09-13T03:14:10.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.084+0000 7f90bfcf1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581c2856710 0x5581c2856af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.084+0000 7f90bfcf1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c2857030 0x5581c2860900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.084+0000 7f90bfcf1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581c2860e40 0x5581c2863230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.084+0000 7f90bfcf1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581c27af730 con 0x5581c2857030 2023-09-13T03:14:10.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.084+0000 7f90bfcf1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581c27ad050 con 0x5581c2860e40 2023-09-13T03:14:10.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.084+0000 7f90bfcf1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581c27a8940 con 0x5581c2856710 2023-09-13T03:14:10.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.084+0000 7f909b7fe700 1 --2- 172.21.15.64:0/516415503 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f90900a0950 0x7f90900a2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.085+0000 7f90aa19b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c2857030 0x5581c2860900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.085+0000 7f909b7fe700 1 -- 172.21.15.64:0/516415503 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f909c0cbd20 con 0x7f90a4106630 2023-09-13T03:14:10.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.085+0000 7f90a999a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581c2856710 0x5581c2856af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.085+0000 7f90aa19b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c2857030 0x5581c2860900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59948/0 (socket says 172.21.15.64:59948) 2023-09-13T03:14:10.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.085+0000 7f90aa19b700 1 -- 172.21.15.64:0/1136497453 learned_addr learned my addr 172.21.15.64:0/1136497453 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:10.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.085+0000 7f90a999a700 1 --2- 172.21.15.64:0/1136497453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581c2860e40 0x5581c2863230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.085+0000 7f90aa99c700 1 --2- 172.21.15.64:0/516415503 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f90900a0950 0x7f90900a2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.085+0000 7f90aa19b700 1 -- 172.21.15.64:0/1136497453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581c2856710 msgr2=0x5581c2856af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:10.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.085+0000 7f90aa19b700 1 --2- 172.21.15.64:0/1136497453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581c2856710 0x5581c2856af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.085+0000 7f90aa19b700 1 -- 172.21.15.64:0/1136497453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581c2860e40 msgr2=0x5581c2863230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:10.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.085+0000 7f90aa19b700 1 --2- 172.21.15.64:0/1136497453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581c2860e40 0x5581c2863230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.085+0000 7f90aa19b700 1 -- 172.21.15.64:0/1136497453 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581c25d4de0 con 0x5581c2857030 2023-09-13T03:14:10.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.085+0000 7f90aa99c700 1 --2- 172.21.15.64:0/516415503 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f90900a0950 0x7f90900a2e10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f909c0c9340 tx=0x7f909c004cc0 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:10.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.086+0000 7f90aa19b700 1 --2- 172.21.15.64:0/1136497453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c2857030 0x5581c2860900 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f90a0009f10 tx=0x7f90a0011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d14c364881fe6a server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:10.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.086+0000 7f9098bfe700 1 -- 172.21.15.64:0/1136497453 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a0019630 con 0x5581c2857030 2023-09-13T03:14:10.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.086+0000 7f9098bfe700 1 -- 172.21.15.64:0/1136497453 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90a0019c50 con 0x5581c2857030 2023-09-13T03:14:10.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.086+0000 7f9098bfe700 1 -- 172.21.15.64:0/1136497453 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a000ebd0 con 0x5581c2857030 2023-09-13T03:14:10.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.086+0000 7f90bfcf1b80 1 -- 172.21.15.64:0/1136497453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c2857030 msgr2=0x5581c2860900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:10.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.086+0000 7f90bfcf1b80 1 --2- 172.21.15.64:0/1136497453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c2857030 0x5581c2860900 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f90a0009f10 tx=0x7f90a0011720 comp rx=0 tx=0).stop 2023-09-13T03:14:10.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.087+0000 7f90bfcf1b80 1 -- 172.21.15.64:0/1136497453 shutdown_connections 2023-09-13T03:14:10.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.087+0000 7f90bfcf1b80 1 --2- 172.21.15.64:0/1136497453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581c2860e40 0x5581c2863230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.087+0000 7f90bfcf1b80 1 --2- 172.21.15.64:0/1136497453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581c2856710 0x5581c2856af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.087+0000 7f90bfcf1b80 1 --2- 172.21.15.64:0/1136497453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c2857030 0x5581c2860900 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.087+0000 7f90bfcf1b80 1 -- 172.21.15.64:0/1136497453 >> 172.21.15.64:0/1136497453 conn(0x5581c27cb090 msgr2=0x5581c2851bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:10.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.087+0000 7f90bfcf1b80 1 -- 172.21.15.64:0/1136497453 shutdown_connections 2023-09-13T03:14:10.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.087+0000 7f90bfcf1b80 1 -- 172.21.15.64:0/1136497453 wait complete. 2023-09-13T03:14:10.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.087+0000 7f90bfcf1b80 1 -- start start 2023-09-13T03:14:10.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.087+0000 7f90bfcf1b80 1 Processor -- start 2023-09-13T03:14:10.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.088+0000 7f90bfcf1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581c2856710 0x5581c28607f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.088+0000 7f90bfcf1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581c2857030 0x5581c285dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.088+0000 7f90bfcf1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c2860e40 0x5581c285e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.088+0000 7f90bfcf1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581c2746b70 con 0x5581c2860e40 2023-09-13T03:14:10.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.088+0000 7f90bfcf1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581c2746870 con 0x5581c2856710 2023-09-13T03:14:10.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.088+0000 7f90bfcf1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581c262a920 con 0x5581c2857030 2023-09-13T03:14:10.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.088+0000 7f90bfcf1b80 10 client.0 _ll_register_callbacks cb 0x5581c2690a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:14:10.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.088+0000 7f90a999a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581c2856710 0x5581c28607f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.088+0000 7f90a999a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c2860e40 0x5581c285e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.088+0000 7f90a999a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c2860e40 0x5581c285e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59950/0 (socket says 172.21.15.64:59950) 2023-09-13T03:14:10.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.088+0000 7f90a999a700 1 -- 172.21.15.64:0/3991447070 learned_addr learned my addr 172.21.15.64:0/3991447070 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:10.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.088+0000 7f90aa19b700 1 --2- 172.21.15.64:0/3991447070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581c2857030 0x5581c285dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.089+0000 7f90a999a700 1 -- 172.21.15.64:0/3991447070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581c2857030 msgr2=0x5581c285dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:10.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.089+0000 7f90a999a700 1 --2- 172.21.15.64:0/3991447070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581c2857030 0x5581c285dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.089+0000 7f90a999a700 1 -- 172.21.15.64:0/3991447070 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581c2856710 msgr2=0x5581c28607f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:10.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.089+0000 7f90a999a700 1 --2- 172.21.15.64:0/3991447070 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581c2856710 0x5581c28607f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.089+0000 7f90a999a700 1 -- 172.21.15.64:0/3991447070 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90a0009b70 con 0x5581c2860e40 2023-09-13T03:14:10.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.089+0000 7f90a999a700 1 --2- 172.21.15.64:0/3991447070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c2860e40 0x5581c285e110 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f90940099c0 tx=0x7f9094005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=518e890f0357a597 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:10.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.089+0000 7f908dffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:14:10.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.089+0000 7f908dffb700 1 -- 172.21.15.64:0/3991447070 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909400b790 con 0x5581c2860e40 2023-09-13T03:14:10.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.089+0000 7f90bfcf1b80 1 -- 172.21.15.64:0/3991447070 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5581c27004f0 con 0x5581c2860e40 2023-09-13T03:14:10.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.089+0000 7f908dffb700 1 -- 172.21.15.64:0/3991447070 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f909400b8f0 con 0x5581c2860e40 2023-09-13T03:14:10.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.090+0000 7f908dffb700 1 -- 172.21.15.64:0/3991447070 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f909400bab0 con 0x5581c2860e40 2023-09-13T03:14:10.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.090+0000 7f90bfcf1b80 10 client.15972 fetch_fsmap learned FSMap version 28 2023-09-13T03:14:10.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.090+0000 7f90bfcf1b80 1 -- 172.21.15.64:0/3991447070 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5581c25bdc60 con 0x5581c2860e40 2023-09-13T03:14:10.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.090+0000 7f908dffb700 1 -- 172.21.15.64:0/3991447070 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5581c27004f0 con 0x5581c2860e40 2023-09-13T03:14:10.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.090+0000 7f908dffb700 1 -- 172.21.15.64:0/3991447070 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f909400fa20 con 0x5581c2860e40 2023-09-13T03:14:10.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.090+0000 7f90bfcf1b80 10 client.15972 fetch_fsmap finished waiting for FSMap version 28 2023-09-13T03:14:10.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.090+0000 7f90bfcf1b80 10 client.15972 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-09-13T03:14:10.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.090+0000 7f90bfcf1b80 20 client.15972 populate_metadata read hostname 'smithi064' 2023-09-13T03:14:10.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.090+0000 7f90bfcf1b80 1 --2- 172.21.15.64:0/3991447070 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x5581c28708d0 0x5581c2872cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.090+0000 7f90bfcf1b80 4 client.15972 mds_command: new command op to 24635 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:14:10.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.090+0000 7f90bfcf1b80 1 -- 172.21.15.64:0/3991447070 --> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5581c27bd8a0 con 0x5581c28708d0 2023-09-13T03:14:10.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.091+0000 7f90aa19b700 1 --2- 172.21.15.64:0/3991447070 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x5581c28708d0 0x5581c2872cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.091+0000 7f90aa19b700 1 --2- 172.21.15.64:0/3991447070 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x5581c28708d0 0x5581c2872cf0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bc9f38a38004d86e server_cookie=9f3c21f0ccc2918b in_seq=0 out_seq=0 2023-09-13T03:14:10.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.091+0000 7f908dffb700 10 client.15972 ms_handle_connect on v2:172.21.15.142:6836/741173268 2023-09-13T03:14:10.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.092+0000 7f908dffb700 1 -- 172.21.15.64:0/3991447070 <== mds.2 v2:172.21.15.142:6836/741173268 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5581c27bd8a0 con 0x5581c28708d0 2023-09-13T03:14:10.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.092+0000 7f908dffb700 10 client.15972 handle_command_reply: tid=0 2023-09-13T03:14:10.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.092+0000 7f90bfcf1b80 1 client.15972 shutdown 2023-09-13T03:14:10.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.092+0000 7f90bfcf1b80 1 -- 172.21.15.64:0/3991447070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c2860e40 msgr2=0x5581c285e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:10.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.092+0000 7f90bfcf1b80 1 --2- 172.21.15.64:0/3991447070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c2860e40 0x5581c285e110 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f90940099c0 tx=0x7f9094005fb0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.092+0000 7f90bfcf1b80 1 -- 172.21.15.64:0/3991447070 shutdown_connections 2023-09-13T03:14:10.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.092+0000 7f90bfcf1b80 1 --2- 172.21.15.64:0/3991447070 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x5581c28708d0 0x5581c2872cf0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.092+0000 7f90bfcf1b80 1 --2- 172.21.15.64:0/3991447070 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581c2856710 0x5581c28607f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.092+0000 7f90bfcf1b80 1 --2- 172.21.15.64:0/3991447070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c2860e40 0x5581c285e110 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.092+0000 7f90bfcf1b80 1 --2- 172.21.15.64:0/3991447070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581c2857030 0x5581c285dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.092+0000 7f908dffb700 0 client.15972 ms_handle_reset on v2:172.21.15.142:6836/741173268 2023-09-13T03:14:10.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.092+0000 7f90bfcf1b80 1 -- 172.21.15.64:0/3991447070 >> 172.21.15.64:0/3991447070 conn(0x5581c27cb090 msgr2=0x5581c28517e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:10.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.092+0000 7f90bfcf1b80 1 -- 172.21.15.64:0/3991447070 shutdown_connections 2023-09-13T03:14:10.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.092+0000 7f90bfcf1b80 1 -- 172.21.15.64:0/3991447070 wait complete. 2023-09-13T03:14:10.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.093+0000 7f90bfcf1b80 20 client.15972 trim_cache size 0 max 16384 2023-09-13T03:14:10.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.106+0000 7f90bfcf1b80 1 Processor -- start 2023-09-13T03:14:10.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.106+0000 7f90bfcf1b80 1 -- start start 2023-09-13T03:14:10.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.107+0000 7f90bfcf1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581c2856710 0x5581c28679f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.107+0000 7f90bfcf1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581c2857030 0x5581c286cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.107+0000 7f90bfcf1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c2860e40 0x5581c2858e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.107+0000 7f90bfcf1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581c27004f0 con 0x5581c2860e40 2023-09-13T03:14:10.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.107+0000 7f90bfcf1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581c28527a0 con 0x5581c2856710 2023-09-13T03:14:10.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.107+0000 7f90bfcf1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581c2857790 con 0x5581c2857030 2023-09-13T03:14:10.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.107+0000 7f90a999a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581c2856710 0x5581c28679f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.107+0000 7f90a999a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c2860e40 0x5581c2858e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.107+0000 7f90aa19b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581c2857030 0x5581c286cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.107+0000 7f90aa19b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581c2857030 0x5581c286cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:36966/0 (socket says 172.21.15.64:36966) 2023-09-13T03:14:10.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.107+0000 7f90a999a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c2860e40 0x5581c2858e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59952/0 (socket says 172.21.15.64:59952) 2023-09-13T03:14:10.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.107+0000 7f90aa19b700 1 -- 172.21.15.64:0/86141585 learned_addr learned my addr 172.21.15.64:0/86141585 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:10.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.107+0000 7f90aa19b700 1 -- 172.21.15.64:0/86141585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581c2856710 msgr2=0x5581c28679f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:10.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.107+0000 7f90aa19b700 1 --2- 172.21.15.64:0/86141585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581c2856710 0x5581c28679f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.108+0000 7f90aa19b700 1 -- 172.21.15.64:0/86141585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c2860e40 msgr2=0x5581c2858e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:10.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.108+0000 7f90aa19b700 1 --2- 172.21.15.64:0/86141585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c2860e40 0x5581c2858e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.108+0000 7f90aa19b700 1 -- 172.21.15.64:0/86141585 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581c25fb250 con 0x5581c2857030 2023-09-13T03:14:10.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.108+0000 7f90a999a700 1 --2- 172.21.15.64:0/86141585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c2860e40 0x5581c2858e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:10.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.108+0000 7f90aa19b700 1 --2- 172.21.15.64:0/86141585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581c2857030 0x5581c286cc60 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f90a0009f10 tx=0x7f90a0009fc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41dcd41b0a606af6 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:10.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.108+0000 7f90993ff700 1 -- 172.21.15.64:0/86141585 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a000e3b0 con 0x5581c2857030 2023-09-13T03:14:10.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.108+0000 7f90993ff700 1 -- 172.21.15.64:0/86141585 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90a0005040 con 0x5581c2857030 2023-09-13T03:14:10.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.108+0000 7f90993ff700 1 -- 172.21.15.64:0/86141585 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a000e9e0 con 0x5581c2857030 2023-09-13T03:14:10.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.109+0000 7f90bfcf1b80 1 -- 172.21.15.64:0/86141585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581c2857030 msgr2=0x5581c286cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:10.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.109+0000 7f90bfcf1b80 1 --2- 172.21.15.64:0/86141585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581c2857030 0x5581c286cc60 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f90a0009f10 tx=0x7f90a0009fc0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.109+0000 7f90bfcf1b80 1 -- 172.21.15.64:0/86141585 shutdown_connections 2023-09-13T03:14:10.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.109+0000 7f90bfcf1b80 1 --2- 172.21.15.64:0/86141585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581c2856710 0x5581c28679f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.109+0000 7f90bfcf1b80 1 --2- 172.21.15.64:0/86141585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c2860e40 0x5581c2858e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.109+0000 7f90bfcf1b80 1 --2- 172.21.15.64:0/86141585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581c2857030 0x5581c286cc60 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.109+0000 7f90bfcf1b80 1 -- 172.21.15.64:0/86141585 >> 172.21.15.64:0/86141585 conn(0x5581c27cb090 msgr2=0x5581c2850320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:10.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.109+0000 7f90bfcf1b80 1 -- 172.21.15.64:0/86141585 shutdown_connections 2023-09-13T03:14:10.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.109+0000 7f90bfcf1b80 1 -- 172.21.15.64:0/86141585 wait complete. 2023-09-13T03:14:10.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.109+0000 7f90bfcf1b80 1 -- start start 2023-09-13T03:14:10.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.110+0000 7f90bfcf1b80 1 Processor -- start 2023-09-13T03:14:10.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.110+0000 7f90bfcf1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c2856710 0x5581c2867790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.110+0000 7f90bfcf1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581c2857030 0x5581c2865ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.110+0000 7f90bfcf1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581c2860e40 0x5581c2866000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.110+0000 7f90bfcf1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581c2866710 con 0x5581c2856710 2023-09-13T03:14:10.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.110+0000 7f90bfcf1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581c2866880 con 0x5581c2860e40 2023-09-13T03:14:10.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.110+0000 7f90bfcf1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581c28c4730 con 0x5581c2857030 2023-09-13T03:14:10.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.110+0000 7f90bfcf1b80 10 client.0 _ll_register_callbacks cb 0x7f9094005350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:14:10.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.111+0000 7f90a999a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c2856710 0x5581c2867790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.111+0000 7f90a999a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c2856710 0x5581c2867790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59968/0 (socket says 172.21.15.64:59968) 2023-09-13T03:14:10.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.111+0000 7f90a999a700 1 -- 172.21.15.64:0/1294759484 learned_addr learned my addr 172.21.15.64:0/1294759484 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:10.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.111+0000 7f90aa19b700 1 --2- 172.21.15.64:0/1294759484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581c2857030 0x5581c2865ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.111+0000 7f90a999a700 1 --2- 172.21.15.64:0/1294759484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581c2860e40 0x5581c2866000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.111+0000 7f90a999a700 1 -- 172.21.15.64:0/1294759484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581c2857030 msgr2=0x5581c2865ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:10.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.111+0000 7f90a999a700 1 --2- 172.21.15.64:0/1294759484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581c2857030 0x5581c2865ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.111+0000 7f90a999a700 1 -- 172.21.15.64:0/1294759484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581c2860e40 msgr2=0x5581c2866000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:10.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.111+0000 7f90a999a700 1 --2- 172.21.15.64:0/1294759484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581c2860e40 0x5581c2866000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.111+0000 7f90a999a700 1 -- 172.21.15.64:0/1294759484 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5581c25bdc60 con 0x5581c2856710 2023-09-13T03:14:10.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.111+0000 7f90aa19b700 1 --2- 172.21.15.64:0/1294759484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581c2857030 0x5581c2865ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:10.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.111+0000 7f90a999a700 1 --2- 172.21.15.64:0/1294759484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c2856710 0x5581c2867790 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9094005010 tx=0x7f9094004810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ecb07f34bc34eb8 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:10.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.111+0000 7f908dffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:14:10.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.112+0000 7f908dffb700 1 -- 172.21.15.64:0/1294759484 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909400f820 con 0x5581c2856710 2023-09-13T03:14:10.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.112+0000 7f90bfcf1b80 1 -- 172.21.15.64:0/1294759484 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f90a0019c50 con 0x5581c2856710 2023-09-13T03:14:10.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.112+0000 7f908dffb700 1 -- 172.21.15.64:0/1294759484 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f9094000e40 con 0x5581c2856710 2023-09-13T03:14:10.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.112+0000 7f908dffb700 1 -- 172.21.15.64:0/1294759484 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f909400f980 con 0x5581c2856710 2023-09-13T03:14:10.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.112+0000 7f90bfcf1b80 10 client.15978 fetch_fsmap learned FSMap version 28 2023-09-13T03:14:10.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.112+0000 7f90bfcf1b80 1 -- 172.21.15.64:0/1294759484 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5581c25d4de0 con 0x5581c2856710 2023-09-13T03:14:10.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.112+0000 7f908dffb700 1 -- 172.21.15.64:0/1294759484 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90a0019c50 con 0x5581c2856710 2023-09-13T03:14:10.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.112+0000 7f908dffb700 1 -- 172.21.15.64:0/1294759484 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f909403f040 con 0x5581c2856710 2023-09-13T03:14:10.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.112+0000 7f90bfcf1b80 10 client.15978 fetch_fsmap finished waiting for FSMap version 28 2023-09-13T03:14:10.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.112+0000 7f90bfcf1b80 10 client.15978 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-09-13T03:14:10.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.112+0000 7f90bfcf1b80 20 client.15978 populate_metadata read hostname 'smithi064' 2023-09-13T03:14:10.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.112+0000 7f90bfcf1b80 1 --2- 172.21.15.64:0/1294759484 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x5581c28c59c0 0x5581c28c7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.112+0000 7f90bfcf1b80 4 client.15978 mds_command: new command op to 24635 tid=0[{"prefix": "damage ls"}] 2023-09-13T03:14:10.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.112+0000 7f90bfcf1b80 1 -- 172.21.15.64:0/1294759484 --> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5581c27a1920 con 0x5581c28c59c0 2023-09-13T03:14:10.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.113+0000 7f90aa19b700 1 --2- 172.21.15.64:0/1294759484 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x5581c28c59c0 0x5581c28c7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.113+0000 7f90aa19b700 1 --2- 172.21.15.64:0/1294759484 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x5581c28c59c0 0x5581c28c7de0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8536b27cff27e352 server_cookie=ee226b7872f6876 in_seq=0 out_seq=0 2023-09-13T03:14:10.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.113+0000 7f908dffb700 10 client.15978 ms_handle_connect on v2:172.21.15.142:6836/741173268 2023-09-13T03:14:10.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.113+0000 7f908dffb700 1 -- 172.21.15.64:0/1294759484 <== mds.2 v2:172.21.15.142:6836/741173268 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5581c27a1920 con 0x5581c28c59c0 2023-09-13T03:14:10.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.114+0000 7f908dffb700 10 client.15978 handle_command_reply: tid=0 2023-09-13T03:14:10.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.114+0000 7f90bfcf1b80 1 client.15978 shutdown 2023-09-13T03:14:10.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.114+0000 7f90bfcf1b80 1 -- 172.21.15.64:0/1294759484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c2856710 msgr2=0x5581c2867790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:10.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.114+0000 7f90bfcf1b80 1 --2- 172.21.15.64:0/1294759484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c2856710 0x5581c2867790 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9094005010 tx=0x7f9094004810 comp rx=0 tx=0).stop 2023-09-13T03:14:10.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.114+0000 7f90bfcf1b80 1 -- 172.21.15.64:0/1294759484 shutdown_connections 2023-09-13T03:14:10.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.114+0000 7f90bfcf1b80 1 --2- 172.21.15.64:0/1294759484 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x5581c28c59c0 0x5581c28c7de0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.114+0000 7f90bfcf1b80 1 --2- 172.21.15.64:0/1294759484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581c2860e40 0x5581c2866000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.114+0000 7f90bfcf1b80 1 --2- 172.21.15.64:0/1294759484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581c2856710 0x5581c2867790 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.114+0000 7f90bfcf1b80 1 --2- 172.21.15.64:0/1294759484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581c2857030 0x5581c2865ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.114+0000 7f90bfcf1b80 1 -- 172.21.15.64:0/1294759484 >> 172.21.15.64:0/1294759484 conn(0x5581c27cb090 msgr2=0x5581c286cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:10.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.114+0000 7f908dffb700 0 client.15978 ms_handle_reset on v2:172.21.15.142:6836/741173268 2023-09-13T03:14:10.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.114+0000 7f90bfcf1b80 1 -- 172.21.15.64:0/1294759484 shutdown_connections 2023-09-13T03:14:10.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.114+0000 7f90bfcf1b80 1 -- 172.21.15.64:0/1294759484 wait complete. 2023-09-13T03:14:10.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.114+0000 7f90bfcf1b80 20 client.15978 trim_cache size 0 max 16384 2023-09-13T03:14:10.134 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2023-09-13T03:14:10.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.115+0000 7f90ac3ff700 1 -- 172.21.15.64:0/516415503 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f90900a0950 msgr2=0x7f90900a2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:10.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.115+0000 7f90ac3ff700 1 --2- 172.21.15.64:0/516415503 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f90900a0950 0x7f90900a2e10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f909c0c9340 tx=0x7f909c004cc0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.115+0000 7f90ac3ff700 1 -- 172.21.15.64:0/516415503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90a4106630 msgr2=0x7f90a4198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:10.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.115+0000 7f90ac3ff700 1 --2- 172.21.15.64:0/516415503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90a4106630 0x7f90a4198630 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f909c00ef30 tx=0x7f909c00c3a0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.115+0000 7f90ac3ff700 1 -- 172.21.15.64:0/516415503 shutdown_connections 2023-09-13T03:14:10.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.115+0000 7f90ac3ff700 1 --2- 172.21.15.64:0/516415503 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f90900a0950 0x7f90900a2e10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.115+0000 7f90ac3ff700 1 --2- 172.21.15.64:0/516415503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90a4106630 0x7f90a4198630 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.115+0000 7f90ac3ff700 1 --2- 172.21.15.64:0/516415503 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90a4100570 0x7f90a419e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.115+0000 7f90ac3ff700 1 --2- 172.21.15.64:0/516415503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90a4100f30 0x7f90a419eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.115+0000 7f90ac3ff700 1 -- 172.21.15.64:0/516415503 >> 172.21.15.64:0/516415503 conn(0x7f90a40fc080 msgr2=0x7f90a40fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:10.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.115+0000 7f90ac3ff700 1 -- 172.21.15.64:0/516415503 shutdown_connections 2023-09-13T03:14:10.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.115+0000 7f90ac3ff700 1 -- 172.21.15.64:0/516415503 wait complete. 2023-09-13T03:14:10.139 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-09-13T03:14:10.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.383+0000 7fd0f97ff700 1 -- 172.21.15.64:0/534251540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0f4072a30 msgr2=0x7fd0f4072e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:10.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.383+0000 7fd0f97ff700 1 --2- 172.21.15.64:0/534251540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0f4072a30 0x7fd0f4072e10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4009990 tx=0x7fd0e4009ca0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.384+0000 7fd0f97ff700 1 -- 172.21.15.64:0/534251540 shutdown_connections 2023-09-13T03:14:10.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.384+0000 7fd0f97ff700 1 --2- 172.21.15.64:0/534251540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0f410cd90 0x7fd0f410f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.384+0000 7fd0f97ff700 1 --2- 172.21.15.64:0/534251540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0f4072a30 0x7fd0f4072e10 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.384+0000 7fd0f97ff700 1 --2- 172.21.15.64:0/534251540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0f4073350 0x7fd0f410c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.384+0000 7fd0f97ff700 1 -- 172.21.15.64:0/534251540 >> 172.21.15.64:0/534251540 conn(0x7fd0f40fc080 msgr2=0x7fd0f40fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:10.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.384+0000 7fd0f97ff700 1 -- 172.21.15.64:0/534251540 shutdown_connections 2023-09-13T03:14:10.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.384+0000 7fd0f97ff700 1 -- 172.21.15.64:0/534251540 wait complete. 2023-09-13T03:14:10.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.385+0000 7fd0f97ff700 1 Processor -- start 2023-09-13T03:14:10.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.385+0000 7fd0f97ff700 1 -- start start 2023-09-13T03:14:10.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.385+0000 7fd0f97ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0f4072a30 0x7fd0f419e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.385+0000 7fd0f97ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0f4073350 0x7fd0f419ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.385+0000 7fd0f97ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0f410cd90 0x7fd0f4198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.385+0000 7fd0f97ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f4198d80 con 0x7fd0f4073350 2023-09-13T03:14:10.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.385+0000 7fd0f97ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f4198ef0 con 0x7fd0f4072a30 2023-09-13T03:14:10.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.385+0000 7fd0f97ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f41990b0 con 0x7fd0f410cd90 2023-09-13T03:14:10.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.386+0000 7fd0f27fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0f4073350 0x7fd0f419ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.386+0000 7fd0f2ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0f4072a30 0x7fd0f419e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.386+0000 7fd0f2ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0f4072a30 0x7fd0f419e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:53760/0 (socket says 172.21.15.64:53760) 2023-09-13T03:14:10.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.386+0000 7fd0f2ffd700 1 -- 172.21.15.64:0/937200892 learned_addr learned my addr 172.21.15.64:0/937200892 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:10.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.386+0000 7fd0f37fe700 1 --2- 172.21.15.64:0/937200892 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0f410cd90 0x7fd0f4198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.386+0000 7fd0f2ffd700 1 -- 172.21.15.64:0/937200892 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0f410cd90 msgr2=0x7fd0f4198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:10.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.386+0000 7fd0f2ffd700 1 --2- 172.21.15.64:0/937200892 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0f410cd90 0x7fd0f4198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.386+0000 7fd0f2ffd700 1 -- 172.21.15.64:0/937200892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0f4073350 msgr2=0x7fd0f419ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:10.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.386+0000 7fd0f2ffd700 1 --2- 172.21.15.64:0/937200892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0f4073350 0x7fd0f419ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.386+0000 7fd0f2ffd700 1 -- 172.21.15.64:0/937200892 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e8009520 con 0x7fd0f4072a30 2023-09-13T03:14:10.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.386+0000 7fd0f2ffd700 1 --2- 172.21.15.64:0/937200892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0f4072a30 0x7fd0f419e6a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4005910 tx=0x7fd0e4004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:10.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.387+0000 7fd0dbfff700 1 -- 172.21.15.64:0/937200892 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e401d070 con 0x7fd0f4072a30 2023-09-13T03:14:10.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.387+0000 7fd0dbfff700 1 -- 172.21.15.64:0/937200892 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0e4022470 con 0x7fd0f4072a30 2023-09-13T03:14:10.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.387+0000 7fd0dbfff700 1 -- 172.21.15.64:0/937200892 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4017650 con 0x7fd0f4072a30 2023-09-13T03:14:10.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.387+0000 7fd0f97ff700 1 -- 172.21.15.64:0/937200892 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0e40095f0 con 0x7fd0f4072a30 2023-09-13T03:14:10.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.387+0000 7fd0f97ff700 1 -- 172.21.15.64:0/937200892 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f41a54b0 con 0x7fd0f4072a30 2023-09-13T03:14:10.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.390+0000 7fd0dbfff700 1 -- 172.21.15.64:0/937200892 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 135492+0+0 (secure 0 0 0) 0x7fd0e40225e0 con 0x7fd0f4072a30 2023-09-13T03:14:10.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.391+0000 7fd10d1e4b80 1 Processor -- start 2023-09-13T03:14:10.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.392+0000 7fd10d1e4b80 1 -- start start 2023-09-13T03:14:10.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.392+0000 7fd10d1e4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580976d4710 0x5580976d4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.392+0000 7fd10d1e4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580976d5030 0x5580976de900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.392+0000 7fd10d1e4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580976dee40 0x5580976e1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.392+0000 7fd10d1e4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55809762d730 con 0x5580976dee40 2023-09-13T03:14:10.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.392+0000 7fd10d1e4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55809762b050 con 0x5580976d5030 2023-09-13T03:14:10.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.392+0000 7fd10d1e4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558097626940 con 0x5580976d4710 2023-09-13T03:14:10.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.392+0000 7fd0f27fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580976dee40 0x5580976e1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.393+0000 7fd0f37fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580976d4710 0x5580976d4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.393+0000 7fd0f37fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580976d4710 0x5580976d4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:37000/0 (socket says 172.21.15.64:37000) 2023-09-13T03:14:10.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.393+0000 7fd0f27fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580976dee40 0x5580976e1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59984/0 (socket says 172.21.15.64:59984) 2023-09-13T03:14:10.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.393+0000 7fd0f37fe700 1 -- 172.21.15.64:0/3530394944 learned_addr learned my addr 172.21.15.64:0/3530394944 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:10.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.393+0000 7fd0f2ffd700 1 --2- 172.21.15.64:0/3530394944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580976d5030 0x5580976de900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.393+0000 7fd0dbfff700 1 --2- 172.21.15.64:0/937200892 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fd0dc0a08d0 0x7fd0dc0a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.393+0000 7fd0f27fc700 1 -- 172.21.15.64:0/3530394944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580976d4710 msgr2=0x5580976d4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:10.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.393+0000 7fd0f27fc700 1 --2- 172.21.15.64:0/3530394944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580976d4710 0x5580976d4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.393+0000 7fd0f27fc700 1 -- 172.21.15.64:0/3530394944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580976d5030 msgr2=0x5580976de900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:10.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.393+0000 7fd0f27fc700 1 --2- 172.21.15.64:0/3530394944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580976d5030 0x5580976de900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.393+0000 7fd0dbfff700 1 -- 172.21.15.64:0/937200892 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd0e40cd490 con 0x7fd0f4072a30 2023-09-13T03:14:10.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.393+0000 7fd0f27fc700 1 -- 172.21.15.64:0/3530394944 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558097452de0 con 0x5580976dee40 2023-09-13T03:14:10.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.393+0000 7fd0f37fe700 1 --2- 172.21.15.64:0/937200892 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fd0dc0a08d0 0x7fd0dc0a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.393+0000 7fd0f27fc700 1 --2- 172.21.15.64:0/3530394944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580976dee40 0x5580976e1230 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8009e10 tx=0x7fd0e8011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=330e57af192da6 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:10.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.394+0000 7fd0d97fa700 1 -- 172.21.15.64:0/3530394944 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e8019630 con 0x5580976dee40 2023-09-13T03:14:10.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.394+0000 7fd0d97fa700 1 -- 172.21.15.64:0/3530394944 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0e8019c50 con 0x5580976dee40 2023-09-13T03:14:10.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.394+0000 7fd0d97fa700 1 -- 172.21.15.64:0/3530394944 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e800ed70 con 0x5580976dee40 2023-09-13T03:14:10.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.394+0000 7fd10d1e4b80 1 -- 172.21.15.64:0/3530394944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580976dee40 msgr2=0x5580976e1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:10.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.394+0000 7fd0f37fe700 1 --2- 172.21.15.64:0/937200892 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fd0dc0a08d0 0x7fd0dc0a2d90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec009550 tx=0x7fd0ec009040 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:10.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.394+0000 7fd10d1e4b80 1 --2- 172.21.15.64:0/3530394944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580976dee40 0x5580976e1230 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8009e10 tx=0x7fd0e8011720 comp rx=0 tx=0).stop 2023-09-13T03:14:10.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.394+0000 7fd10d1e4b80 1 -- 172.21.15.64:0/3530394944 shutdown_connections 2023-09-13T03:14:10.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.394+0000 7fd10d1e4b80 1 --2- 172.21.15.64:0/3530394944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580976dee40 0x5580976e1230 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.394+0000 7fd10d1e4b80 1 --2- 172.21.15.64:0/3530394944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580976d4710 0x5580976d4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.394+0000 7fd10d1e4b80 1 --2- 172.21.15.64:0/3530394944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580976d5030 0x5580976de900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.394+0000 7fd10d1e4b80 1 -- 172.21.15.64:0/3530394944 >> 172.21.15.64:0/3530394944 conn(0x558097649090 msgr2=0x5580976cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:10.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.395+0000 7fd10d1e4b80 1 -- 172.21.15.64:0/3530394944 shutdown_connections 2023-09-13T03:14:10.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.395+0000 7fd10d1e4b80 1 -- 172.21.15.64:0/3530394944 wait complete. 2023-09-13T03:14:10.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.395+0000 7fd10d1e4b80 1 -- start start 2023-09-13T03:14:10.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.395+0000 7fd10d1e4b80 1 Processor -- start 2023-09-13T03:14:10.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.396+0000 7fd10d1e4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580976d4710 0x5580976de860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.396+0000 7fd10d1e4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580976d5030 0x5580976dbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.396+0000 7fd10d1e4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580976dee40 0x5580976dc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.396+0000 7fd10d1e4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580975c4b70 con 0x5580976d4710 2023-09-13T03:14:10.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.396+0000 7fd10d1e4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580975c4870 con 0x5580976d5030 2023-09-13T03:14:10.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.396+0000 7fd10d1e4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580974a8920 con 0x5580976dee40 2023-09-13T03:14:10.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.396+0000 7fd10d1e4b80 10 client.0 _ll_register_callbacks cb 0x55809750ea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:14:10.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.396+0000 7fd0f2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580976d4710 0x5580976de860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.396+0000 7fd0f37fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580976dee40 0x5580976dc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.396+0000 7fd0f37fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580976dee40 0x5580976dc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:37008/0 (socket says 172.21.15.64:37008) 2023-09-13T03:14:10.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.396+0000 7fd0f37fe700 1 -- 172.21.15.64:0/245704341 learned_addr learned my addr 172.21.15.64:0/245704341 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:10.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.396+0000 7fd0f27fc700 1 --2- 172.21.15.64:0/245704341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580976d5030 0x5580976dbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.396+0000 7fd0f2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580976d4710 0x5580976de860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59990/0 (socket says 172.21.15.64:59990) 2023-09-13T03:14:10.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.397+0000 7fd0f37fe700 1 -- 172.21.15.64:0/245704341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580976d5030 msgr2=0x5580976dbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:10.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.397+0000 7fd0f37fe700 1 --2- 172.21.15.64:0/245704341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580976d5030 0x5580976dbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.397+0000 7fd0f37fe700 1 -- 172.21.15.64:0/245704341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580976d4710 msgr2=0x5580976de860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:10.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.397+0000 7fd0f37fe700 1 --2- 172.21.15.64:0/245704341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580976d4710 0x5580976de860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.397+0000 7fd0f37fe700 1 -- 172.21.15.64:0/245704341 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0e8009a70 con 0x5580976dee40 2023-09-13T03:14:10.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.397+0000 7fd0f2ffd700 1 --2- 172.21.15.64:0/245704341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580976d4710 0x5580976de860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:10.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.397+0000 7fd0f37fe700 1 --2- 172.21.15.64:0/245704341 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580976dee40 0x5580976dc110 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec017ee0 tx=0x7fd0ec004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be1ca4831c79a8b6 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:10.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.397+0000 7fd0d6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:14:10.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.397+0000 7fd0d6ff5700 1 -- 172.21.15.64:0/245704341 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec00e040 con 0x5580976dee40 2023-09-13T03:14:10.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.397+0000 7fd10d1e4b80 1 -- 172.21.15.64:0/245704341 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55809757e4f0 con 0x5580976dee40 2023-09-13T03:14:10.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.397+0000 7fd0d6ff5700 1 -- 172.21.15.64:0/245704341 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd0ec003a00 con 0x5580976dee40 2023-09-13T03:14:10.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.398+0000 7fd0d6ff5700 1 -- 172.21.15.64:0/245704341 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0ec003c00 con 0x5580976dee40 2023-09-13T03:14:10.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.398+0000 7fd10d1e4b80 10 client.26029 fetch_fsmap learned FSMap version 28 2023-09-13T03:14:10.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.398+0000 7fd10d1e4b80 1 -- 172.21.15.64:0/245704341 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55809743bc60 con 0x5580976dee40 2023-09-13T03:14:10.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.398+0000 7fd0d6ff5700 1 -- 172.21.15.64:0/245704341 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0ec031020 con 0x5580976dee40 2023-09-13T03:14:10.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.398+0000 7fd0d6ff5700 1 -- 172.21.15.64:0/245704341 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd0ec02d460 con 0x5580976dee40 2023-09-13T03:14:10.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.398+0000 7fd10d1e4b80 10 client.26029 fetch_fsmap finished waiting for FSMap version 28 2023-09-13T03:14:10.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.398+0000 7fd10d1e4b80 10 client.26029 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-09-13T03:14:10.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.398+0000 7fd10d1e4b80 20 client.26029 populate_metadata read hostname 'smithi064' 2023-09-13T03:14:10.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.398+0000 7fd10d1e4b80 1 --2- 172.21.15.64:0/245704341 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x5580976ee910 0x5580976f0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.398+0000 7fd10d1e4b80 4 client.26029 mds_command: new command op to 24649 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:14:10.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.398+0000 7fd10d1e4b80 1 -- 172.21.15.64:0/245704341 --> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55809763b8a0 con 0x5580976ee910 2023-09-13T03:14:10.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.398+0000 7fd0f2ffd700 1 --2- 172.21.15.64:0/245704341 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x5580976ee910 0x5580976f0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.399+0000 7fd0f2ffd700 1 --2- 172.21.15.64:0/245704341 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x5580976ee910 0x5580976f0d30 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f62af8b225b2c818 server_cookie=427429eda709f1ea in_seq=0 out_seq=0 2023-09-13T03:14:10.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.399+0000 7fd0d6ff5700 10 client.26029 ms_handle_connect on v2:172.21.15.170:6838/2189703596 2023-09-13T03:14:10.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.400+0000 7fd0d6ff5700 1 -- 172.21.15.64:0/245704341 <== mds.1 v2:172.21.15.170:6838/2189703596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55809763b8a0 con 0x5580976ee910 2023-09-13T03:14:10.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.400+0000 7fd0d6ff5700 10 client.26029 handle_command_reply: tid=0 2023-09-13T03:14:10.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.400+0000 7fd10d1e4b80 1 client.26029 shutdown 2023-09-13T03:14:10.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.400+0000 7fd10d1e4b80 1 -- 172.21.15.64:0/245704341 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580976dee40 msgr2=0x5580976dc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:10.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.400+0000 7fd10d1e4b80 1 --2- 172.21.15.64:0/245704341 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580976dee40 0x5580976dc110 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec017ee0 tx=0x7fd0ec004ad0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.400+0000 7fd10d1e4b80 1 -- 172.21.15.64:0/245704341 shutdown_connections 2023-09-13T03:14:10.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.400+0000 7fd10d1e4b80 1 --2- 172.21.15.64:0/245704341 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x5580976ee910 0x5580976f0d30 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.400+0000 7fd10d1e4b80 1 --2- 172.21.15.64:0/245704341 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580976dee40 0x5580976dc110 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.400+0000 7fd10d1e4b80 1 --2- 172.21.15.64:0/245704341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580976d4710 0x5580976de860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.400+0000 7fd10d1e4b80 1 --2- 172.21.15.64:0/245704341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580976d5030 0x5580976dbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.400+0000 7fd0d6ff5700 0 client.26029 ms_handle_reset on v2:172.21.15.170:6838/2189703596 2023-09-13T03:14:10.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.400+0000 7fd10d1e4b80 1 -- 172.21.15.64:0/245704341 >> 172.21.15.64:0/245704341 conn(0x558097649090 msgr2=0x5580976cf7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:10.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.400+0000 7fd10d1e4b80 1 -- 172.21.15.64:0/245704341 shutdown_connections 2023-09-13T03:14:10.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.400+0000 7fd10d1e4b80 1 -- 172.21.15.64:0/245704341 wait complete. 2023-09-13T03:14:10.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.401+0000 7fd10d1e4b80 20 client.26029 trim_cache size 0 max 16384 2023-09-13T03:14:10.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.414+0000 7fd10d1e4b80 1 Processor -- start 2023-09-13T03:14:10.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.414+0000 7fd10d1e4b80 1 -- start start 2023-09-13T03:14:10.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.414+0000 7fd10d1e4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580976d4710 0x5580976f0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.414+0000 7fd10d1e4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580976d5030 0x5580976e59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.414+0000 7fd10d1e4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580976dee40 0x5580976de390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.414+0000 7fd10d1e4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580976e5f30 con 0x5580976d5030 2023-09-13T03:14:10.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.414+0000 7fd10d1e4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580976eb2e0 con 0x5580976d4710 2023-09-13T03:14:10.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.414+0000 7fd10d1e4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580976dc710 con 0x5580976dee40 2023-09-13T03:14:10.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.415+0000 7fd0f27fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580976d5030 0x5580976e59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.415+0000 7fd0f2ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580976d4710 0x5580976f0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.415+0000 7fd0f27fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580976d5030 0x5580976e59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59996/0 (socket says 172.21.15.64:59996) 2023-09-13T03:14:10.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.415+0000 7fd0f27fc700 1 -- 172.21.15.64:0/3176573589 learned_addr learned my addr 172.21.15.64:0/3176573589 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:10.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.415+0000 7fd0f37fe700 1 --2- 172.21.15.64:0/3176573589 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580976dee40 0x5580976de390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.415+0000 7fd0f2ffd700 1 -- 172.21.15.64:0/3176573589 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580976dee40 msgr2=0x5580976de390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:10.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.415+0000 7fd0f2ffd700 1 --2- 172.21.15.64:0/3176573589 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580976dee40 0x5580976de390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.415+0000 7fd0f2ffd700 1 -- 172.21.15.64:0/3176573589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580976d5030 msgr2=0x5580976e59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:10.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.415+0000 7fd0f2ffd700 1 --2- 172.21.15.64:0/3176573589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580976d5030 0x5580976e59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.415+0000 7fd0f2ffd700 1 -- 172.21.15.64:0/3176573589 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558097479250 con 0x5580976d4710 2023-09-13T03:14:10.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.415+0000 7fd0f37fe700 1 --2- 172.21.15.64:0/3176573589 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580976dee40 0x5580976de390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:10.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.415+0000 7fd0f2ffd700 1 --2- 172.21.15.64:0/3176573589 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580976d4710 0x5580976f0d00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd0e40caa50 tx=0x7fd0e401a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=164eacd4b52f4d1b server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:10.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.415+0000 7fd0d9ffb700 1 -- 172.21.15.64:0/3176573589 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e409a540 con 0x5580976d4710 2023-09-13T03:14:10.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.415+0000 7fd0d9ffb700 1 -- 172.21.15.64:0/3176573589 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0e409a2b0 con 0x5580976d4710 2023-09-13T03:14:10.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.416+0000 7fd0d9ffb700 1 -- 172.21.15.64:0/3176573589 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e409e5a0 con 0x5580976d4710 2023-09-13T03:14:10.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.416+0000 7fd10d1e4b80 1 -- 172.21.15.64:0/3176573589 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580976d4710 msgr2=0x5580976f0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:10.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.416+0000 7fd10d1e4b80 1 --2- 172.21.15.64:0/3176573589 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580976d4710 0x5580976f0d00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd0e40caa50 tx=0x7fd0e401a040 comp rx=0 tx=0).stop 2023-09-13T03:14:10.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.416+0000 7fd10d1e4b80 1 -- 172.21.15.64:0/3176573589 shutdown_connections 2023-09-13T03:14:10.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.416+0000 7fd10d1e4b80 1 --2- 172.21.15.64:0/3176573589 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580976d4710 0x5580976f0d00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.416+0000 7fd10d1e4b80 1 --2- 172.21.15.64:0/3176573589 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580976dee40 0x5580976de390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.416+0000 7fd10d1e4b80 1 --2- 172.21.15.64:0/3176573589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580976d5030 0x5580976e59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.416+0000 7fd10d1e4b80 1 -- 172.21.15.64:0/3176573589 >> 172.21.15.64:0/3176573589 conn(0x558097649090 msgr2=0x5580976ea140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:10.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.416+0000 7fd10d1e4b80 1 -- 172.21.15.64:0/3176573589 shutdown_connections 2023-09-13T03:14:10.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.416+0000 7fd10d1e4b80 1 -- 172.21.15.64:0/3176573589 wait complete. 2023-09-13T03:14:10.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.417+0000 7fd10d1e4b80 1 -- start start 2023-09-13T03:14:10.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.417+0000 7fd10d1e4b80 1 Processor -- start 2023-09-13T03:14:10.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.417+0000 7fd10d1e4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580976d4710 0x5580976e5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.417+0000 7fd10d1e4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580976d5030 0x5580976e3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.417+0000 7fd10d1e4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580976dee40 0x5580976e3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.417+0000 7fd10d1e4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580976e4480 con 0x5580976d4710 2023-09-13T03:14:10.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.417+0000 7fd10d1e4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580976e45f0 con 0x5580976dee40 2023-09-13T03:14:10.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.417+0000 7fd10d1e4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580976e47b0 con 0x5580976d5030 2023-09-13T03:14:10.429 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:10 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/656134701' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:14:10.429 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:10 smithi064 ceph-mon[97675]: pgmap v33: 97 pgs: 97 active+clean; 15 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 1.3 MiB/s wr, 268 op/s 2023-09-13T03:14:10.429 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:10 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:14:10.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.417+0000 7fd10d1e4b80 10 client.0 _ll_register_callbacks cb 0x7fd0ec017880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:14:10.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.418+0000 7fd0f2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580976d4710 0x5580976e5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.418+0000 7fd0f2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580976d4710 0x5580976e5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60010/0 (socket says 172.21.15.64:60010) 2023-09-13T03:14:10.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.418+0000 7fd0f2ffd700 1 -- 172.21.15.64:0/1988058956 learned_addr learned my addr 172.21.15.64:0/1988058956 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:10.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.418+0000 7fd0f37fe700 1 --2- 172.21.15.64:0/1988058956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580976dee40 0x5580976e3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.418+0000 7fd0f27fc700 1 --2- 172.21.15.64:0/1988058956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580976d5030 0x5580976e3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.418+0000 7fd0f2ffd700 1 -- 172.21.15.64:0/1988058956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580976d5030 msgr2=0x5580976e3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:10.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.418+0000 7fd0f2ffd700 1 --2- 172.21.15.64:0/1988058956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580976d5030 0x5580976e3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.418+0000 7fd0f2ffd700 1 -- 172.21.15.64:0/1988058956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580976dee40 msgr2=0x5580976e3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:10.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.418+0000 7fd0f2ffd700 1 --2- 172.21.15.64:0/1988058956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580976dee40 0x5580976e3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.418+0000 7fd0f2ffd700 1 -- 172.21.15.64:0/1988058956 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55809743bc60 con 0x5580976d4710 2023-09-13T03:14:10.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.418+0000 7fd0f37fe700 1 --2- 172.21.15.64:0/1988058956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580976dee40 0x5580976e3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:10.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.418+0000 7fd0f2ffd700 1 --2- 172.21.15.64:0/1988058956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580976d4710 0x5580976e5780 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd0e40971e0 tx=0x7fd0e4009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8883d64cecfdccb server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:10.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.418+0000 7fd0d6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:14:10.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.419+0000 7fd0d6ff5700 1 -- 172.21.15.64:0/1988058956 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e409ae70 con 0x5580976d4710 2023-09-13T03:14:10.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.419+0000 7fd0d6ff5700 1 -- 172.21.15.64:0/1988058956 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd0e40a2750 con 0x5580976d4710 2023-09-13T03:14:10.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.419+0000 7fd10d1e4b80 1 -- 172.21.15.64:0/1988058956 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd0e8019c50 con 0x5580976d4710 2023-09-13T03:14:10.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.419+0000 7fd0d6ff5700 1 -- 172.21.15.64:0/1988058956 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0e409e5a0 con 0x5580976d4710 2023-09-13T03:14:10.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.419+0000 7fd10d1e4b80 10 client.15996 fetch_fsmap learned FSMap version 28 2023-09-13T03:14:10.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.419+0000 7fd10d1e4b80 1 -- 172.21.15.64:0/1988058956 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7fd0f41a54b0 con 0x5580976d4710 2023-09-13T03:14:10.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.419+0000 7fd0d6ff5700 1 -- 172.21.15.64:0/1988058956 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0e8019c50 con 0x5580976d4710 2023-09-13T03:14:10.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.419+0000 7fd0d6ff5700 1 -- 172.21.15.64:0/1988058956 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd0e4099c60 con 0x5580976d4710 2023-09-13T03:14:10.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.419+0000 7fd10d1e4b80 10 client.15996 fetch_fsmap finished waiting for FSMap version 28 2023-09-13T03:14:10.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.419+0000 7fd10d1e4b80 10 client.15996 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-09-13T03:14:10.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.419+0000 7fd10d1e4b80 20 client.15996 populate_metadata read hostname 'smithi064' 2023-09-13T03:14:10.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.419+0000 7fd10d1e4b80 1 --2- 172.21.15.64:0/1988058956 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x558097743750 0x558097745b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.419+0000 7fd10d1e4b80 4 client.15996 mds_command: new command op to 24649 tid=0[{"prefix": "damage ls"}] 2023-09-13T03:14:10.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.419+0000 7fd10d1e4b80 1 -- 172.21.15.64:0/1988058956 --> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55809761f920 con 0x558097743750 2023-09-13T03:14:10.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.420+0000 7fd0f27fc700 1 --2- 172.21.15.64:0/1988058956 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x558097743750 0x558097745b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.420+0000 7fd0d6ff5700 10 client.15996 ms_handle_connect on v2:172.21.15.170:6838/2189703596 2023-09-13T03:14:10.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.420+0000 7fd0f27fc700 1 --2- 172.21.15.64:0/1988058956 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x558097743750 0x558097745b40 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ad0edc48a10950c server_cookie=dd0f71414d6eb4ad in_seq=0 out_seq=0 2023-09-13T03:14:10.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.420+0000 7fd0d6ff5700 1 -- 172.21.15.64:0/1988058956 <== mds.1 v2:172.21.15.170:6838/2189703596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55809761f920 con 0x558097743750 2023-09-13T03:14:10.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.420+0000 7fd0d6ff5700 10 client.15996 handle_command_reply: tid=0 2023-09-13T03:14:10.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.420+0000 7fd10d1e4b80 1 client.15996 shutdown 2023-09-13T03:14:10.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.421+0000 7fd10d1e4b80 1 -- 172.21.15.64:0/1988058956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580976d4710 msgr2=0x5580976e5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:10.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.421+0000 7fd10d1e4b80 1 --2- 172.21.15.64:0/1988058956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580976d4710 0x5580976e5780 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd0e40971e0 tx=0x7fd0e4009ca0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.421+0000 7fd10d1e4b80 1 -- 172.21.15.64:0/1988058956 shutdown_connections 2023-09-13T03:14:10.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.421+0000 7fd10d1e4b80 1 --2- 172.21.15.64:0/1988058956 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x558097743750 0x558097745b40 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.421+0000 7fd10d1e4b80 1 --2- 172.21.15.64:0/1988058956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580976dee40 0x5580976e3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.421+0000 7fd10d1e4b80 1 --2- 172.21.15.64:0/1988058956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580976d4710 0x5580976e5780 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.421+0000 7fd10d1e4b80 1 --2- 172.21.15.64:0/1988058956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5580976d5030 0x5580976e3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.421+0000 7fd0d6ff5700 0 client.15996 ms_handle_reset on v2:172.21.15.170:6838/2189703596 2023-09-13T03:14:10.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.421+0000 7fd10d1e4b80 1 -- 172.21.15.64:0/1988058956 >> 172.21.15.64:0/1988058956 conn(0x558097649090 msgr2=0x5580976d6e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:10.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.421+0000 7fd10d1e4b80 1 -- 172.21.15.64:0/1988058956 shutdown_connections 2023-09-13T03:14:10.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.421+0000 7fd10d1e4b80 1 -- 172.21.15.64:0/1988058956 wait complete. 2023-09-13T03:14:10.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.421+0000 7fd10d1e4b80 20 client.15996 trim_cache size 0 max 16384 2023-09-13T03:14:10.440 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2023-09-13T03:14:10.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.422+0000 7fd0f97ff700 1 -- 172.21.15.64:0/937200892 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fd0dc0a08d0 msgr2=0x7fd0dc0a2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:10.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.422+0000 7fd0f97ff700 1 --2- 172.21.15.64:0/937200892 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fd0dc0a08d0 0x7fd0dc0a2d90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec009550 tx=0x7fd0ec009040 comp rx=0 tx=0).stop 2023-09-13T03:14:10.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.422+0000 7fd0f97ff700 1 -- 172.21.15.64:0/937200892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0f4072a30 msgr2=0x7fd0f419e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:10.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.422+0000 7fd0f97ff700 1 --2- 172.21.15.64:0/937200892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0f4072a30 0x7fd0f419e6a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4005910 tx=0x7fd0e4004710 comp rx=0 tx=0).stop 2023-09-13T03:14:10.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.422+0000 7fd0f97ff700 1 -- 172.21.15.64:0/937200892 shutdown_connections 2023-09-13T03:14:10.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.422+0000 7fd0f97ff700 1 --2- 172.21.15.64:0/937200892 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fd0dc0a08d0 0x7fd0dc0a2d90 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.422+0000 7fd0f97ff700 1 --2- 172.21.15.64:0/937200892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0f4072a30 0x7fd0f419e6a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.422+0000 7fd0f97ff700 1 --2- 172.21.15.64:0/937200892 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0f410cd90 0x7fd0f4198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.422+0000 7fd0f97ff700 1 --2- 172.21.15.64:0/937200892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0f4073350 0x7fd0f419ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.422+0000 7fd0f97ff700 1 -- 172.21.15.64:0/937200892 >> 172.21.15.64:0/937200892 conn(0x7fd0f40fc080 msgr2=0x7fd0f410efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:10.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.422+0000 7fd0f97ff700 1 -- 172.21.15.64:0/937200892 shutdown_connections 2023-09-13T03:14:10.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.422+0000 7fd0f97ff700 1 -- 172.21.15.64:0/937200892 wait complete. 2023-09-13T03:14:10.444 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-09-13T03:14:10.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.651+0000 7f36ff59e700 1 -- 172.21.15.64:0/671307162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3700072a30 msgr2=0x7f3700072e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:10.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.651+0000 7f36ff59e700 1 --2- 172.21.15.64:0/671307162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3700072a30 0x7f3700072e10 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f36f0009990 tx=0x7f36f0009ca0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.651+0000 7f36ff59e700 1 -- 172.21.15.64:0/671307162 shutdown_connections 2023-09-13T03:14:10.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.651+0000 7f36ff59e700 1 --2- 172.21.15.64:0/671307162 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f370010cc30 0x7f370010f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.651+0000 7f36ff59e700 1 --2- 172.21.15.64:0/671307162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3700072a30 0x7f3700072e10 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.651+0000 7f36ff59e700 1 --2- 172.21.15.64:0/671307162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3700073350 0x7f370010c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.651+0000 7f36ff59e700 1 -- 172.21.15.64:0/671307162 >> 172.21.15.64:0/671307162 conn(0x7f37000fbec0 msgr2=0x7f37000fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:10.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.651+0000 7f36ff59e700 1 -- 172.21.15.64:0/671307162 shutdown_connections 2023-09-13T03:14:10.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.651+0000 7f36ff59e700 1 -- 172.21.15.64:0/671307162 wait complete. 2023-09-13T03:14:10.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.652+0000 7f36ff59e700 1 Processor -- start 2023-09-13T03:14:10.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.653+0000 7f36ff59e700 1 -- start start 2023-09-13T03:14:10.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.653+0000 7f36ff59e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3700072a30 0x7f3700198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.653+0000 7f36ff59e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3700073350 0x7f3700198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.653+0000 7f36ff59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f370010cc30 0x7f370019c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.653+0000 7f36ff59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370019c600 con 0x7f370010cc30 2023-09-13T03:14:10.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.653+0000 7f36ff59e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370019c770 con 0x7f3700072a30 2023-09-13T03:14:10.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.653+0000 7f36ff59e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370019c930 con 0x7f3700073350 2023-09-13T03:14:10.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.653+0000 7f36fe59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3700072a30 0x7f3700198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.653+0000 7f36fed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f370010cc30 0x7f370019c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.653+0000 7f36fe59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3700072a30 0x7f3700198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:53802/0 (socket says 172.21.15.64:53802) 2023-09-13T03:14:10.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.653+0000 7f36fe59c700 1 -- 172.21.15.64:0/629865408 learned_addr learned my addr 172.21.15.64:0/629865408 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:10.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.653+0000 7f36fdd9b700 1 --2- 172.21.15.64:0/629865408 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3700073350 0x7f3700198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.654+0000 7f36fe59c700 1 -- 172.21.15.64:0/629865408 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3700073350 msgr2=0x7f3700198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:10.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.654+0000 7f36fe59c700 1 --2- 172.21.15.64:0/629865408 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3700073350 0x7f3700198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.654+0000 7f36fe59c700 1 -- 172.21.15.64:0/629865408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f370010cc30 msgr2=0x7f370019c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:10.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.654+0000 7f36fe59c700 1 --2- 172.21.15.64:0/629865408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f370010cc30 0x7f370019c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.654+0000 7f36fe59c700 1 -- 172.21.15.64:0/629865408 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36f00095f0 con 0x7f3700072a30 2023-09-13T03:14:10.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.654+0000 7f36fed9d700 1 --2- 172.21.15.64:0/629865408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f370010cc30 0x7f370019c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:10.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.654+0000 7f36fe59c700 1 --2- 172.21.15.64:0/629865408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3700072a30 0x7f3700198710 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f36f0000c00 tx=0x7f36f0005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:10.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.654+0000 7f36ef7fe700 1 -- 172.21.15.64:0/629865408 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f001d070 con 0x7f3700072a30 2023-09-13T03:14:10.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.654+0000 7f36ff59e700 1 -- 172.21.15.64:0/629865408 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f370019cbb0 con 0x7f3700072a30 2023-09-13T03:14:10.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.654+0000 7f36ef7fe700 1 -- 172.21.15.64:0/629865408 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36f0022470 con 0x7f3700072a30 2023-09-13T03:14:10.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.654+0000 7f36ef7fe700 1 -- 172.21.15.64:0/629865408 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f0017650 con 0x7f3700072a30 2023-09-13T03:14:10.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.654+0000 7f36ff59e700 1 -- 172.21.15.64:0/629865408 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37001a5570 con 0x7f3700072a30 2023-09-13T03:14:10.662 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:10 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/656134701' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:14:10.662 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:10 smithi170 ceph-mon[134328]: pgmap v33: 97 pgs: 97 active+clean; 15 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 1.3 MiB/s wr, 268 op/s 2023-09-13T03:14:10.663 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:10 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:14:10.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.658+0000 7f36ef7fe700 1 -- 172.21.15.64:0/629865408 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 135492+0+0 (secure 0 0 0) 0x7f36f00225e0 con 0x7f3700072a30 2023-09-13T03:14:10.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.659+0000 7f3718688b80 1 Processor -- start 2023-09-13T03:14:10.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.659+0000 7f36ef7fe700 1 --2- 172.21.15.64:0/629865408 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f36e80a0970 0x7f36e80a2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.659+0000 7f3718688b80 1 -- start start 2023-09-13T03:14:10.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.659+0000 7f36ef7fe700 1 -- 172.21.15.64:0/629865408 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f36f00cd540 con 0x7f3700072a30 2023-09-13T03:14:10.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.659+0000 7f36fed9d700 1 --2- 172.21.15.64:0/629865408 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f36e80a0970 0x7f36e80a2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.660+0000 7f3718688b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c040e6860 0x555c040e6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.660+0000 7f3718688b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c040e7180 0x555c040f1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.660+0000 7f3718688b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c040f2030 0x555c040f4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.660+0000 7f3718688b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c04040730 con 0x555c040f2030 2023-09-13T03:14:10.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.660+0000 7f3718688b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c0403e050 con 0x555c040e7180 2023-09-13T03:14:10.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.660+0000 7f3718688b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c04039940 con 0x555c040e6860 2023-09-13T03:14:10.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.660+0000 7f36fe59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c040e6860 0x555c040e6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.660+0000 7f36fe59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c040e6860 0x555c040e6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:37050/0 (socket says 172.21.15.64:37050) 2023-09-13T03:14:10.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.660+0000 7f36fe59c700 1 -- 172.21.15.64:0/1929474483 learned_addr learned my addr 172.21.15.64:0/1929474483 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:10.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.660+0000 7f36fdd9b700 1 --2- 172.21.15.64:0/1929474483 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c040e7180 0x555c040f1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.660+0000 7f36fed9d700 1 --2- 172.21.15.64:0/1929474483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c040f2030 0x555c040f4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.660+0000 7f36fe59c700 1 -- 172.21.15.64:0/1929474483 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c040e7180 msgr2=0x555c040f1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:10.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.660+0000 7f36fe59c700 1 --2- 172.21.15.64:0/1929474483 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c040e7180 0x555c040f1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.660+0000 7f36fe59c700 1 -- 172.21.15.64:0/1929474483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c040f2030 msgr2=0x555c040f4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:10.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.660+0000 7f36fe59c700 1 --2- 172.21.15.64:0/1929474483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c040f2030 0x555c040f4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.660+0000 7f36fe59c700 1 -- 172.21.15.64:0/1929474483 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c03e65de0 con 0x555c040e6860 2023-09-13T03:14:10.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.660+0000 7f36fed9d700 1 --2- 172.21.15.64:0/1929474483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c040f2030 0x555c040f4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T03:14:10.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.661+0000 7f36fed9d700 1 --2- 172.21.15.64:0/629865408 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f36e80a0970 0x7f36e80a2e30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f36f8003e80 tx=0x7f36f8009920 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:10.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.661+0000 7f36fe59c700 1 --2- 172.21.15.64:0/1929474483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c040e6860 0x555c040e6c40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f36f0096df0 tx=0x7f36f009a500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4892df7822d3659 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:10.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.661+0000 7f36ecff9700 1 -- 172.21.15.64:0/1929474483 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f009a8f0 con 0x555c040e6860 2023-09-13T03:14:10.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.661+0000 7f36ecff9700 1 -- 172.21.15.64:0/1929474483 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36f009e720 con 0x555c040e6860 2023-09-13T03:14:10.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.661+0000 7f36ecff9700 1 -- 172.21.15.64:0/1929474483 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f009a040 con 0x555c040e6860 2023-09-13T03:14:10.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.662+0000 7f3718688b80 1 -- 172.21.15.64:0/1929474483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c040e6860 msgr2=0x555c040e6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:10.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.662+0000 7f3718688b80 1 --2- 172.21.15.64:0/1929474483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c040e6860 0x555c040e6c40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f36f0096df0 tx=0x7f36f009a500 comp rx=0 tx=0).stop 2023-09-13T03:14:10.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.662+0000 7f3718688b80 1 -- 172.21.15.64:0/1929474483 shutdown_connections 2023-09-13T03:14:10.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.662+0000 7f3718688b80 1 --2- 172.21.15.64:0/1929474483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c040f2030 0x555c040f4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.662+0000 7f3718688b80 1 --2- 172.21.15.64:0/1929474483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c040e6860 0x555c040e6c40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.662+0000 7f3718688b80 1 --2- 172.21.15.64:0/1929474483 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c040e7180 0x555c040f1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.662+0000 7f3718688b80 1 -- 172.21.15.64:0/1929474483 >> 172.21.15.64:0/1929474483 conn(0x555c0405c090 msgr2=0x555c040e2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:10.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.662+0000 7f3718688b80 1 -- 172.21.15.64:0/1929474483 shutdown_connections 2023-09-13T03:14:10.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.662+0000 7f3718688b80 1 -- 172.21.15.64:0/1929474483 wait complete. 2023-09-13T03:14:10.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.662+0000 7f3718688b80 1 -- start start 2023-09-13T03:14:10.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.663+0000 7f3718688b80 1 Processor -- start 2023-09-13T03:14:10.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.664+0000 7f3718688b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c040e6860 0x555c040f2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.664+0000 7f3718688b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c040e7180 0x555c040f2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.664+0000 7f3718688b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c040f2030 0x555c040fd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.664+0000 7f3718688b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c03fd7b70 con 0x555c040f2030 2023-09-13T03:14:10.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.664+0000 7f3718688b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c03fd7870 con 0x555c040e6860 2023-09-13T03:14:10.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.664+0000 7f3718688b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c03ebb920 con 0x555c040e7180 2023-09-13T03:14:10.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.664+0000 7f3718688b80 10 client.0 _ll_register_callbacks cb 0x555c03f21a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:14:10.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.664+0000 7f36fed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c040f2030 0x555c040fd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.664+0000 7f36fdd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c040e7180 0x555c040f2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.664+0000 7f36fe59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c040e6860 0x555c040f2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.664+0000 7f36fdd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c040e7180 0x555c040f2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:37052/0 (socket says 172.21.15.64:37052) 2023-09-13T03:14:10.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.664+0000 7f36fdd9b700 1 -- 172.21.15.64:0/2421275348 learned_addr learned my addr 172.21.15.64:0/2421275348 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:10.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.664+0000 7f36fdd9b700 1 -- 172.21.15.64:0/2421275348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c040e6860 msgr2=0x555c040f2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:10.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.664+0000 7f36fdd9b700 1 --2- 172.21.15.64:0/2421275348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c040e6860 0x555c040f2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.664+0000 7f36fdd9b700 1 -- 172.21.15.64:0/2421275348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c040f2030 msgr2=0x555c040fd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:10.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.664+0000 7f36fdd9b700 1 --2- 172.21.15.64:0/2421275348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c040f2030 0x555c040fd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.664+0000 7f36fdd9b700 1 -- 172.21.15.64:0/2421275348 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37001a5570 con 0x555c040e7180 2023-09-13T03:14:10.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.665+0000 7f36fdd9b700 1 --2- 172.21.15.64:0/2421275348 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c040e7180 0x555c040f2950 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f36f4012840 tx=0x7f36f4012b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ade4948f3c1adde5 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:10.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.665+0000 7f36e5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:14:10.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.665+0000 7f36e5ffb700 1 -- 172.21.15.64:0/2421275348 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f4017070 con 0x555c040e7180 2023-09-13T03:14:10.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.665+0000 7f3718688b80 1 -- 172.21.15.64:0/2421275348 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c03f914f0 con 0x555c040e7180 2023-09-13T03:14:10.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.665+0000 7f36e5ffb700 1 -- 172.21.15.64:0/2421275348 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f36f4004830 con 0x555c040e7180 2023-09-13T03:14:10.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.665+0000 7f36e5ffb700 1 -- 172.21.15.64:0/2421275348 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36f4004a30 con 0x555c040e7180 2023-09-13T03:14:10.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.665+0000 7f3718688b80 10 client.26047 fetch_fsmap learned FSMap version 28 2023-09-13T03:14:10.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.665+0000 7f3718688b80 1 -- 172.21.15.64:0/2421275348 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555c03e4ec60 con 0x555c040e7180 2023-09-13T03:14:10.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.666+0000 7f36e5ffb700 1 -- 172.21.15.64:0/2421275348 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555c03f914f0 con 0x555c040e7180 2023-09-13T03:14:10.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.666+0000 7f36e5ffb700 1 -- 172.21.15.64:0/2421275348 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f36f400f4e0 con 0x555c040e7180 2023-09-13T03:14:10.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.666+0000 7f3718688b80 10 client.26047 fetch_fsmap finished waiting for FSMap version 28 2023-09-13T03:14:10.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.666+0000 7f3718688b80 10 client.26047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T03:14:10.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.666+0000 7f3718688b80 20 client.26047 populate_metadata read hostname 'smithi064' 2023-09-13T03:14:10.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.666+0000 7f3718688b80 1 --2- 172.21.15.64:0/2421275348 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x555c04101770 0x555c04103b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.666+0000 7f3718688b80 4 client.26047 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:14:10.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.666+0000 7f3718688b80 1 -- 172.21.15.64:0/2421275348 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555c0404e8a0 con 0x555c04101770 2023-09-13T03:14:10.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.666+0000 7f36fe59c700 1 --2- 172.21.15.64:0/2421275348 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x555c04101770 0x555c04103b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.667+0000 7f36e5ffb700 10 client.26047 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:10.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.667+0000 7f36fe59c700 1 --2- 172.21.15.64:0/2421275348 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x555c04101770 0x555c04103b90 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=111f59fc7e1c8bb4 server_cookie=98447df110412265 in_seq=0 out_seq=0 2023-09-13T03:14:10.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.668+0000 7f36e5ffb700 1 -- 172.21.15.64:0/2421275348 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555c0404e8a0 con 0x555c04101770 2023-09-13T03:14:10.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.668+0000 7f36e5ffb700 10 client.26047 handle_command_reply: tid=0 2023-09-13T03:14:10.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.668+0000 7f3718688b80 1 client.26047 shutdown 2023-09-13T03:14:10.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.668+0000 7f3718688b80 1 -- 172.21.15.64:0/2421275348 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c040e7180 msgr2=0x555c040f2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:10.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.668+0000 7f3718688b80 1 --2- 172.21.15.64:0/2421275348 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c040e7180 0x555c040f2950 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f36f4012840 tx=0x7f36f4012b50 comp rx=0 tx=0).stop 2023-09-13T03:14:10.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.668+0000 7f3718688b80 1 -- 172.21.15.64:0/2421275348 shutdown_connections 2023-09-13T03:14:10.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.668+0000 7f3718688b80 1 --2- 172.21.15.64:0/2421275348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c040e6860 0x555c040f2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.668+0000 7f3718688b80 1 --2- 172.21.15.64:0/2421275348 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x555c04101770 0x555c04103b90 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.668+0000 7f3718688b80 1 --2- 172.21.15.64:0/2421275348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c040f2030 0x555c040fd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.668+0000 7f3718688b80 1 --2- 172.21.15.64:0/2421275348 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c040e7180 0x555c040f2950 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.668+0000 7f36e5ffb700 0 client.26047 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:10.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.668+0000 7f3718688b80 1 -- 172.21.15.64:0/2421275348 >> 172.21.15.64:0/2421275348 conn(0x555c0405c090 msgr2=0x555c040e28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:10.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.668+0000 7f3718688b80 1 -- 172.21.15.64:0/2421275348 shutdown_connections 2023-09-13T03:14:10.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.668+0000 7f3718688b80 1 -- 172.21.15.64:0/2421275348 wait complete. 2023-09-13T03:14:10.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.668+0000 7f3718688b80 20 client.26047 trim_cache size 0 max 16384 2023-09-13T03:14:10.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.682+0000 7f3718688b80 1 Processor -- start 2023-09-13T03:14:10.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.682+0000 7f3718688b80 1 -- start start 2023-09-13T03:14:10.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.682+0000 7f3718688b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c040e6860 0x555c040f7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.682+0000 7f3718688b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c040e7180 0x555c040ec500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.682+0000 7f3718688b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c040f2030 0x555c040eca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.682+0000 7f3718688b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c040f2cc0 con 0x555c040f2030 2023-09-13T03:14:10.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.682+0000 7f3718688b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c040e8320 con 0x555c040e6860 2023-09-13T03:14:10.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.682+0000 7f3718688b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c040f7990 con 0x555c040e7180 2023-09-13T03:14:10.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.683+0000 7f36fed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c040f2030 0x555c040eca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.683+0000 7f36fed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c040f2030 0x555c040eca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60050/0 (socket says 172.21.15.64:60050) 2023-09-13T03:14:10.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.683+0000 7f36fed9d700 1 -- 172.21.15.64:0/390015748 learned_addr learned my addr 172.21.15.64:0/390015748 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:10.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.683+0000 7f36fe59c700 1 --2- 172.21.15.64:0/390015748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c040e6860 0x555c040f7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.683+0000 7f36fdd9b700 1 --2- 172.21.15.64:0/390015748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c040e7180 0x555c040ec500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.683+0000 7f36fed9d700 1 -- 172.21.15.64:0/390015748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c040e7180 msgr2=0x555c040ec500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:10.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.683+0000 7f36fed9d700 1 --2- 172.21.15.64:0/390015748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c040e7180 0x555c040ec500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.683+0000 7f36fed9d700 1 -- 172.21.15.64:0/390015748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c040e6860 msgr2=0x555c040f7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:10.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.683+0000 7f36fed9d700 1 --2- 172.21.15.64:0/390015748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c040e6860 0x555c040f7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.683+0000 7f36fed9d700 1 -- 172.21.15.64:0/390015748 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c040f1b20 con 0x555c040f2030 2023-09-13T03:14:10.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.683+0000 7f36fe59c700 1 --2- 172.21.15.64:0/390015748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c040e6860 0x555c040f7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:10.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.683+0000 7f36fed9d700 1 --2- 172.21.15.64:0/390015748 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c040f2030 0x555c040eca40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f36f800e6b0 tx=0x7f36f800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d927aa7991b6e2db server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:10.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.684+0000 7f36ed7fa700 1 -- 172.21.15.64:0/390015748 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f8025670 con 0x555c040f2030 2023-09-13T03:14:10.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.684+0000 7f36ed7fa700 1 -- 172.21.15.64:0/390015748 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36f800f040 con 0x555c040f2030 2023-09-13T03:14:10.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.684+0000 7f36ed7fa700 1 -- 172.21.15.64:0/390015748 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f8019a90 con 0x555c040f2030 2023-09-13T03:14:10.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.684+0000 7f3718688b80 1 -- 172.21.15.64:0/390015748 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c040f2030 msgr2=0x555c040eca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:10.694 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.684+0000 7f3718688b80 1 --2- 172.21.15.64:0/390015748 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c040f2030 0x555c040eca40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f36f800e6b0 tx=0x7f36f800b040 comp rx=0 tx=0).stop 2023-09-13T03:14:10.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.684+0000 7f3718688b80 1 -- 172.21.15.64:0/390015748 shutdown_connections 2023-09-13T03:14:10.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.684+0000 7f3718688b80 1 --2- 172.21.15.64:0/390015748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c040e6860 0x555c040f7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.684+0000 7f3718688b80 1 --2- 172.21.15.64:0/390015748 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c040f2030 0x555c040eca40 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.684+0000 7f3718688b80 1 --2- 172.21.15.64:0/390015748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c040e7180 0x555c040ec500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.695 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.684+0000 7f3718688b80 1 -- 172.21.15.64:0/390015748 >> 172.21.15.64:0/390015748 conn(0x555c0405c090 msgr2=0x555c040fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:10.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.684+0000 7f3718688b80 1 -- 172.21.15.64:0/390015748 shutdown_connections 2023-09-13T03:14:10.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.684+0000 7f3718688b80 1 -- 172.21.15.64:0/390015748 wait complete. 2023-09-13T03:14:10.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.685+0000 7f3718688b80 1 -- start start 2023-09-13T03:14:10.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.685+0000 7f3718688b80 1 Processor -- start 2023-09-13T03:14:10.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.686+0000 7f3718688b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c040e6860 0x555c040f7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.696 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.686+0000 7f3718688b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c040e7180 0x555c040f5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.686+0000 7f3718688b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c040f2030 0x555c040f5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.686+0000 7f3718688b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c040f6160 con 0x555c040e7180 2023-09-13T03:14:10.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.686+0000 7f3718688b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c040f62d0 con 0x555c040e6860 2023-09-13T03:14:10.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.686+0000 7f3718688b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c04155570 con 0x555c040f2030 2023-09-13T03:14:10.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.686+0000 7f3718688b80 10 client.0 _ll_register_callbacks cb 0x7f36f4009e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:14:10.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.686+0000 7f36fdd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c040e7180 0x555c040f5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.686+0000 7f36fe59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c040e6860 0x555c040f7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.686+0000 7f36fdd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c040e7180 0x555c040f5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60056/0 (socket says 172.21.15.64:60056) 2023-09-13T03:14:10.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.686+0000 7f36fdd9b700 1 -- 172.21.15.64:0/896501942 learned_addr learned my addr 172.21.15.64:0/896501942 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:10.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.686+0000 7f36fed9d700 1 --2- 172.21.15.64:0/896501942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c040f2030 0x555c040f5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.687+0000 7f36fe59c700 1 -- 172.21.15.64:0/896501942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c040f2030 msgr2=0x555c040f5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:10.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.687+0000 7f36fe59c700 1 --2- 172.21.15.64:0/896501942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c040f2030 0x555c040f5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.687+0000 7f36fe59c700 1 -- 172.21.15.64:0/896501942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c040e7180 msgr2=0x555c040f5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:10.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.687+0000 7f36fe59c700 1 --2- 172.21.15.64:0/896501942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c040e7180 0x555c040f5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.687+0000 7f36fe59c700 1 -- 172.21.15.64:0/896501942 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555c03e65de0 con 0x555c040e6860 2023-09-13T03:14:10.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.687+0000 7f36fdd9b700 1 --2- 172.21.15.64:0/896501942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c040e7180 0x555c040f5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:10.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.687+0000 7f36fe59c700 1 --2- 172.21.15.64:0/896501942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c040e6860 0x555c040f7260 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f36f00098e0 tx=0x7f36f0009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b182987f23cbd4fb server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:10.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.687+0000 7f36e5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:14:10.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.687+0000 7f36e5ffb700 1 -- 172.21.15.64:0/896501942 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f009a8f0 con 0x555c040e6860 2023-09-13T03:14:10.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.687+0000 7f36e5ffb700 1 -- 172.21.15.64:0/896501942 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f36f00a28d0 con 0x555c040e6860 2023-09-13T03:14:10.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.687+0000 7f3718688b80 1 -- 172.21.15.64:0/896501942 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f36f009e720 con 0x555c040e6860 2023-09-13T03:14:10.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.687+0000 7f36e5ffb700 1 -- 172.21.15.64:0/896501942 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36f0099de0 con 0x555c040e6860 2023-09-13T03:14:10.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.687+0000 7f3718688b80 10 client.26051 fetch_fsmap learned FSMap version 28 2023-09-13T03:14:10.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.687+0000 7f3718688b80 1 -- 172.21.15.64:0/896501942 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f36f8004900 con 0x555c040e6860 2023-09-13T03:14:10.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.688+0000 7f36e5ffb700 1 -- 172.21.15.64:0/896501942 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36f009e720 con 0x555c040e6860 2023-09-13T03:14:10.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.688+0000 7f36e5ffb700 1 -- 172.21.15.64:0/896501942 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f36f0097000 con 0x555c040e6860 2023-09-13T03:14:10.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.688+0000 7f3718688b80 10 client.26051 fetch_fsmap finished waiting for FSMap version 28 2023-09-13T03:14:10.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.688+0000 7f3718688b80 10 client.26051 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T03:14:10.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.688+0000 7f3718688b80 20 client.26051 populate_metadata read hostname 'smithi064' 2023-09-13T03:14:10.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.688+0000 7f3718688b80 1 --2- 172.21.15.64:0/896501942 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x555c04156740 0x555c04158b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:10.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.688+0000 7f3718688b80 4 client.26051 mds_command: new command op to 24686 tid=0[{"prefix": "damage ls"}] 2023-09-13T03:14:10.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.688+0000 7f3718688b80 1 -- 172.21.15.64:0/896501942 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555c04032920 con 0x555c04156740 2023-09-13T03:14:10.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.688+0000 7f36fdd9b700 1 --2- 172.21.15.64:0/896501942 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x555c04156740 0x555c04158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:10.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.689+0000 7f36e5ffb700 10 client.26051 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:10.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.689+0000 7f36fdd9b700 1 --2- 172.21.15.64:0/896501942 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x555c04156740 0x555c04158b30 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2241b2bf05760ea6 server_cookie=3846842c5a06ffb in_seq=0 out_seq=0 2023-09-13T03:14:10.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.689+0000 7f36e5ffb700 1 -- 172.21.15.64:0/896501942 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555c04032920 con 0x555c04156740 2023-09-13T03:14:10.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.689+0000 7f36e5ffb700 10 client.26051 handle_command_reply: tid=0 2023-09-13T03:14:10.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.689+0000 7f3718688b80 1 client.26051 shutdown 2023-09-13T03:14:10.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.689+0000 7f3718688b80 1 -- 172.21.15.64:0/896501942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c040e6860 msgr2=0x555c040f7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:10.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.689+0000 7f3718688b80 1 --2- 172.21.15.64:0/896501942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c040e6860 0x555c040f7260 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f36f00098e0 tx=0x7f36f0009ca0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.690+0000 7f3718688b80 1 -- 172.21.15.64:0/896501942 shutdown_connections 2023-09-13T03:14:10.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.690+0000 7f3718688b80 1 --2- 172.21.15.64:0/896501942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c040e6860 0x555c040f7260 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.690+0000 7f3718688b80 1 --2- 172.21.15.64:0/896501942 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x555c04156740 0x555c04158b30 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.690+0000 7f3718688b80 1 --2- 172.21.15.64:0/896501942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c040f2030 0x555c040f5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.690+0000 7f36e5ffb700 0 client.26051 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:10.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.690+0000 7f3718688b80 1 --2- 172.21.15.64:0/896501942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c040e7180 0x555c040f5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.690+0000 7f3718688b80 1 -- 172.21.15.64:0/896501942 >> 172.21.15.64:0/896501942 conn(0x555c0405c090 msgr2=0x555c040ef9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:10.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.690+0000 7f3718688b80 1 -- 172.21.15.64:0/896501942 shutdown_connections 2023-09-13T03:14:10.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.690+0000 7f3718688b80 1 -- 172.21.15.64:0/896501942 wait complete. 2023-09-13T03:14:10.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.690+0000 7f3718688b80 20 client.26051 trim_cache size 0 max 16384 2023-09-13T03:14:10.707 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2023-09-13T03:14:10.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.690+0000 7f36ff59e700 1 -- 172.21.15.64:0/629865408 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f36e80a0970 msgr2=0x7f36e80a2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:10.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.690+0000 7f36ff59e700 1 --2- 172.21.15.64:0/629865408 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f36e80a0970 0x7f36e80a2e30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f36f8003e80 tx=0x7f36f8009920 comp rx=0 tx=0).stop 2023-09-13T03:14:10.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.690+0000 7f36ff59e700 1 -- 172.21.15.64:0/629865408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3700072a30 msgr2=0x7f3700198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:10.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.690+0000 7f36ff59e700 1 --2- 172.21.15.64:0/629865408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3700072a30 0x7f3700198710 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f36f0000c00 tx=0x7f36f0005740 comp rx=0 tx=0).stop 2023-09-13T03:14:10.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.690+0000 7f36ff59e700 1 -- 172.21.15.64:0/629865408 shutdown_connections 2023-09-13T03:14:10.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.690+0000 7f36ff59e700 1 --2- 172.21.15.64:0/629865408 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f36e80a0970 0x7f36e80a2e30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.690+0000 7f36ff59e700 1 --2- 172.21.15.64:0/629865408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3700072a30 0x7f3700198710 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.690+0000 7f36ff59e700 1 --2- 172.21.15.64:0/629865408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f370010cc30 0x7f370019c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.690+0000 7f36ff59e700 1 --2- 172.21.15.64:0/629865408 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3700073350 0x7f3700198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:10.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.690+0000 7f36ff59e700 1 -- 172.21.15.64:0/629865408 >> 172.21.15.64:0/629865408 conn(0x7f37000fbec0 msgr2=0x7f370010ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:10.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.691+0000 7f36ff59e700 1 -- 172.21.15.64:0/629865408 shutdown_connections 2023-09-13T03:14:10.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:10.691+0000 7f36ff59e700 1 -- 172.21.15.64:0/629865408 wait complete. 2023-09-13T03:14:10.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:10 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/656134701' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:14:10.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:10 smithi142 ceph-mon[134082]: pgmap v33: 97 pgs: 97 active+clean; 15 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 1.3 MiB/s wr, 268 op/s 2023-09-13T03:14:10.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:10 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:14:10.885 INFO:tasks.workunit.client.0.smithi064.stdout: 1 25259 10.32 MB/sec warmup 15 sec latency 4781.305 ms 2023-09-13T03:14:11.711 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-09-13T03:14:11.712 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-09-13T03:14:11.885 INFO:tasks.workunit.client.0.smithi064.stdout: 1 25259 9.68 MB/sec warmup 16 sec latency 1509.506 ms 2023-09-13T03:14:11.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.933+0000 7f8c18dff700 1 -- 172.21.15.64:0/2587747803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c14100570 msgr2=0x7f8c141009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:11.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.933+0000 7f8c18dff700 1 --2- 172.21.15.64:0/2587747803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c14100570 0x7f8c141009f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8c080099c0 tx=0x7f8c08009cd0 comp rx=0 tx=0).stop 2023-09-13T03:14:11.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.933+0000 7f8c18dff700 1 -- 172.21.15.64:0/2587747803 shutdown_connections 2023-09-13T03:14:11.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.933+0000 7f8c18dff700 1 --2- 172.21.15.64:0/2587747803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c14106630 0x7f8c14106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:11.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.933+0000 7f8c18dff700 1 --2- 172.21.15.64:0/2587747803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c14100f30 0x7f8c1410d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:11.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.933+0000 7f8c18dff700 1 --2- 172.21.15.64:0/2587747803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c14100570 0x7f8c141009f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:11.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.933+0000 7f8c18dff700 1 -- 172.21.15.64:0/2587747803 >> 172.21.15.64:0/2587747803 conn(0x7f8c140fc080 msgr2=0x7f8c140fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:11.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.933+0000 7f8c18dff700 1 -- 172.21.15.64:0/2587747803 shutdown_connections 2023-09-13T03:14:11.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.933+0000 7f8c18dff700 1 -- 172.21.15.64:0/2587747803 wait complete. 2023-09-13T03:14:11.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.934+0000 7f8c18dff700 1 Processor -- start 2023-09-13T03:14:11.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.934+0000 7f8c18dff700 1 -- start start 2023-09-13T03:14:11.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.934+0000 7f8c18dff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c14100570 0x7f8c14196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:11.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.934+0000 7f8c18dff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c14100f30 0x7f8c14196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:11.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.934+0000 7f8c18dff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c14106630 0x7f8c1419afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:11.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.934+0000 7f8c18dff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1419b620 con 0x7f8c14100570 2023-09-13T03:14:11.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.934+0000 7f8c18dff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1419b790 con 0x7f8c14100f30 2023-09-13T03:14:11.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.934+0000 7f8c18dff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1419b950 con 0x7f8c14106630 2023-09-13T03:14:11.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.935+0000 7f8c1259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c14100570 0x7f8c14196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:11.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.935+0000 7f8c1259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c14100570 0x7f8c14196640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60088/0 (socket says 172.21.15.64:60088) 2023-09-13T03:14:11.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.935+0000 7f8c1259c700 1 -- 172.21.15.64:0/1957801663 learned_addr learned my addr 172.21.15.64:0/1957801663 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:11.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.935+0000 7f8c12d9d700 1 --2- 172.21.15.64:0/1957801663 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c14106630 0x7f8c1419afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:11.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.935+0000 7f8c11d9b700 1 --2- 172.21.15.64:0/1957801663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c14100f30 0x7f8c14196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:11.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.935+0000 7f8c1259c700 1 -- 172.21.15.64:0/1957801663 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c14106630 msgr2=0x7f8c1419afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:11.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.935+0000 7f8c1259c700 1 --2- 172.21.15.64:0/1957801663 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c14106630 0x7f8c1419afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:11.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.935+0000 7f8c1259c700 1 -- 172.21.15.64:0/1957801663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c14100f30 msgr2=0x7f8c14196b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:11.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.935+0000 7f8c1259c700 1 --2- 172.21.15.64:0/1957801663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c14100f30 0x7f8c14196b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:11.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.935+0000 7f8c1259c700 1 -- 172.21.15.64:0/1957801663 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c080095f0 con 0x7f8c14100570 2023-09-13T03:14:11.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.935+0000 7f8c1259c700 1 --2- 172.21.15.64:0/1957801663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c14100570 0x7f8c14196640 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8c0400e9a0 tx=0x7f8c0400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:11.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.936+0000 7f8c037fe700 1 -- 172.21.15.64:0/1957801663 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c04009790 con 0x7f8c14100570 2023-09-13T03:14:11.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.936+0000 7f8c18dff700 1 -- 172.21.15.64:0/1957801663 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c1419bc30 con 0x7f8c14100570 2023-09-13T03:14:11.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.936+0000 7f8c037fe700 1 -- 172.21.15.64:0/1957801663 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c04010430 con 0x7f8c14100570 2023-09-13T03:14:11.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.936+0000 7f8c037fe700 1 -- 172.21.15.64:0/1957801663 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c04018680 con 0x7f8c14100570 2023-09-13T03:14:11.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.936+0000 7f8c18dff700 1 -- 172.21.15.64:0/1957801663 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c141a3470 con 0x7f8c14100570 2023-09-13T03:14:11.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.939+0000 7f8c037fe700 1 -- 172.21.15.64:0/1957801663 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) v1 ==== 135492+0+0 (secure 0 0 0) 0x7f8c040105a0 con 0x7f8c14100570 2023-09-13T03:14:11.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.940+0000 7f8c2c78db80 1 Processor -- start 2023-09-13T03:14:11.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.940+0000 7f8c037fe700 1 --2- 172.21.15.64:0/1957801663 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f8bfc0a0950 0x7f8bfc0a2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:11.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.940+0000 7f8c2c78db80 1 -- start start 2023-09-13T03:14:11.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.940+0000 7f8c037fe700 1 -- 172.21.15.64:0/1957801663 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f8c04014070 con 0x7f8c14100570 2023-09-13T03:14:11.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.941+0000 7f8c2c78db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b73bc38790 0x55b73bc38b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:11.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.941+0000 7f8c2c78db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b73bc390b0 0x55b73bc42980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:11.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.941+0000 7f8c2c78db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b73bc42ec0 0x55b73bc452b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:11.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.941+0000 7f8c12d9d700 1 --2- 172.21.15.64:0/1957801663 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f8bfc0a0950 0x7f8bfc0a2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:11.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.941+0000 7f8c2c78db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b73bb917b0 con 0x55b73bc42ec0 2023-09-13T03:14:11.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.941+0000 7f8c2c78db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b73bb8f0d0 con 0x55b73bc38790 2023-09-13T03:14:11.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.941+0000 7f8c2c78db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b73bb8a9c0 con 0x55b73bc390b0 2023-09-13T03:14:11.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.941+0000 7f8c1259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b73bc38790 0x55b73bc38b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:11.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.941+0000 7f8c11d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b73bc390b0 0x55b73bc42980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:11.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.941+0000 7f8c11d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b73bc390b0 0x55b73bc42980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:37110/0 (socket says 172.21.15.64:37110) 2023-09-13T03:14:11.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.941+0000 7f8c11d9b700 1 -- 172.21.15.64:0/524780871 learned_addr learned my addr 172.21.15.64:0/524780871 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:11.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.941+0000 7f8c1259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b73bc38790 0x55b73bc38b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:53842/0 (socket says 172.21.15.64:53842) 2023-09-13T03:14:11.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.941+0000 7f8c12d9d700 1 --2- 172.21.15.64:0/524780871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b73bc42ec0 0x55b73bc452b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:11.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.941+0000 7f8c11d9b700 1 -- 172.21.15.64:0/524780871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b73bc38790 msgr2=0x55b73bc38b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:11.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.941+0000 7f8c11d9b700 1 --2- 172.21.15.64:0/524780871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b73bc38790 0x55b73bc38b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:11.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.941+0000 7f8c11d9b700 1 -- 172.21.15.64:0/524780871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b73bc42ec0 msgr2=0x55b73bc452b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:11.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.941+0000 7f8c11d9b700 1 --2- 172.21.15.64:0/524780871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b73bc42ec0 0x55b73bc452b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:11.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.941+0000 7f8c11d9b700 1 -- 172.21.15.64:0/524780871 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b73b9b6e60 con 0x55b73bc390b0 2023-09-13T03:14:11.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.941+0000 7f8c1259c700 1 --2- 172.21.15.64:0/524780871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b73bc38790 0x55b73bc38b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:11.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.942+0000 7f8c12d9d700 1 --2- 172.21.15.64:0/1957801663 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f8bfc0a0950 0x7f8bfc0a2e10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c009c90 tx=0x7f8c0c004320 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:11.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.942+0000 7f8c11d9b700 1 --2- 172.21.15.64:0/524780871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b73bc390b0 0x55b73bc42980 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8c08009fd0 tx=0x7f8c08005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3bdd6c959cc9c6e server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:11.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.942+0000 7f8c00ff9700 1 -- 172.21.15.64:0/524780871 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0800b990 con 0x55b73bc390b0 2023-09-13T03:14:11.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.942+0000 7f8c00ff9700 1 -- 172.21.15.64:0/524780871 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c0801e040 con 0x55b73bc390b0 2023-09-13T03:14:11.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.942+0000 7f8c00ff9700 1 -- 172.21.15.64:0/524780871 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0800fc80 con 0x55b73bc390b0 2023-09-13T03:14:11.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.943+0000 7f8c2c78db80 1 -- 172.21.15.64:0/524780871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b73bc390b0 msgr2=0x55b73bc42980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:11.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.943+0000 7f8c2c78db80 1 --2- 172.21.15.64:0/524780871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b73bc390b0 0x55b73bc42980 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8c08009fd0 tx=0x7f8c08005f00 comp rx=0 tx=0).stop 2023-09-13T03:14:11.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.943+0000 7f8c2c78db80 1 -- 172.21.15.64:0/524780871 shutdown_connections 2023-09-13T03:14:11.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.943+0000 7f8c2c78db80 1 --2- 172.21.15.64:0/524780871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b73bc38790 0x55b73bc38b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:11.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.943+0000 7f8c2c78db80 1 --2- 172.21.15.64:0/524780871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b73bc42ec0 0x55b73bc452b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:11.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.943+0000 7f8c2c78db80 1 --2- 172.21.15.64:0/524780871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b73bc390b0 0x55b73bc42980 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:11.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.943+0000 7f8c2c78db80 1 -- 172.21.15.64:0/524780871 >> 172.21.15.64:0/524780871 conn(0x55b73bbad110 msgr2=0x55b73bc33c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:11.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.943+0000 7f8c2c78db80 1 -- 172.21.15.64:0/524780871 shutdown_connections 2023-09-13T03:14:11.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.943+0000 7f8c2c78db80 1 -- 172.21.15.64:0/524780871 wait complete. 2023-09-13T03:14:11.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.944+0000 7f8c2c78db80 1 -- start start 2023-09-13T03:14:11.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.944+0000 7f8c2c78db80 1 Processor -- start 2023-09-13T03:14:11.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.944+0000 7f8c2c78db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b73bc38790 0x55b73bc42840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:11.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.944+0000 7f8c2c78db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b73bc390b0 0x55b73bc3fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:11.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.944+0000 7f8c2c78db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b73bc42ec0 0x55b73bc40060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:11.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.944+0000 7f8c2c78db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b73bb28bf0 con 0x55b73bc390b0 2023-09-13T03:14:11.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.944+0000 7f8c2c78db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b73bb288f0 con 0x55b73bc38790 2023-09-13T03:14:11.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.944+0000 7f8c2c78db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b73ba0c9a0 con 0x55b73bc42ec0 2023-09-13T03:14:11.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.944+0000 7f8c2c78db80 10 client.0 _ll_register_callbacks cb 0x55b73ba72b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:14:11.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.945+0000 7f8c11d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b73bc390b0 0x55b73bc3fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:11.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.945+0000 7f8c12d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b73bc42ec0 0x55b73bc40060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:11.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.945+0000 7f8c12d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b73bc42ec0 0x55b73bc40060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:37116/0 (socket says 172.21.15.64:37116) 2023-09-13T03:14:11.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.945+0000 7f8c11d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b73bc390b0 0x55b73bc3fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60104/0 (socket says 172.21.15.64:60104) 2023-09-13T03:14:11.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.945+0000 7f8c12d9d700 1 -- 172.21.15.64:0/3918100919 learned_addr learned my addr 172.21.15.64:0/3918100919 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:11.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.945+0000 7f8c1259c700 1 --2- 172.21.15.64:0/3918100919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b73bc38790 0x55b73bc42840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:11.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.945+0000 7f8c12d9d700 1 -- 172.21.15.64:0/3918100919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b73bc38790 msgr2=0x55b73bc42840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:11.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.945+0000 7f8c12d9d700 1 --2- 172.21.15.64:0/3918100919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b73bc38790 0x55b73bc42840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:11.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.945+0000 7f8c12d9d700 1 -- 172.21.15.64:0/3918100919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b73bc390b0 msgr2=0x55b73bc3fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:11.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.945+0000 7f8c12d9d700 1 --2- 172.21.15.64:0/3918100919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b73bc390b0 0x55b73bc3fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:11.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.945+0000 7f8c12d9d700 1 -- 172.21.15.64:0/3918100919 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c141a3470 con 0x55b73bc42ec0 2023-09-13T03:14:11.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.945+0000 7f8c11d9b700 1 --2- 172.21.15.64:0/3918100919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b73bc390b0 0x55b73bc3fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:11.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.945+0000 7f8c12d9d700 1 --2- 172.21.15.64:0/3918100919 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b73bc42ec0 0x55b73bc40060 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c01b700 tx=0x7f8c0c004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfd55c1841d08e42 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:11.958 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.945+0000 7f8bf9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:14:11.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.946+0000 7f8bf9ffb700 1 -- 172.21.15.64:0/3918100919 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c01bea0 con 0x55b73bc42ec0 2023-09-13T03:14:11.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.946+0000 7f8c2c78db80 1 -- 172.21.15.64:0/3918100919 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b73bae2570 con 0x55b73bc42ec0 2023-09-13T03:14:11.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.946+0000 7f8bf9ffb700 1 -- 172.21.15.64:0/3918100919 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f8c0c007830 con 0x55b73bc42ec0 2023-09-13T03:14:11.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.946+0000 7f8bf9ffb700 1 -- 172.21.15.64:0/3918100919 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c0c0079f0 con 0x55b73bc42ec0 2023-09-13T03:14:11.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.946+0000 7f8c2c78db80 10 client.26065 fetch_fsmap learned FSMap version 28 2023-09-13T03:14:11.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.946+0000 7f8c2c78db80 1 -- 172.21.15.64:0/3918100919 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f8c1410dcf0 con 0x55b73bc42ec0 2023-09-13T03:14:11.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.946+0000 7f8bf9ffb700 1 -- 172.21.15.64:0/3918100919 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b73bae2570 con 0x55b73bc42ec0 2023-09-13T03:14:11.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.946+0000 7f8bf9ffb700 1 -- 172.21.15.64:0/3918100919 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8c0c024bc0 con 0x55b73bc42ec0 2023-09-13T03:14:11.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.947+0000 7f8c2c78db80 10 client.26065 fetch_fsmap finished waiting for FSMap version 28 2023-09-13T03:14:11.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.947+0000 7f8c2c78db80 10 client.26065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T03:14:11.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.947+0000 7f8c2c78db80 20 client.26065 populate_metadata read hostname 'smithi064' 2023-09-13T03:14:11.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.947+0000 7f8c2c78db80 1 --2- 172.21.15.64:0/3918100919 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55b73bc529a0 0x55b73bc54dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:11.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.947+0000 7f8c2c78db80 4 client.26065 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:14:11.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.947+0000 7f8c2c78db80 1 -- 172.21.15.64:0/3918100919 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b73bb9f920 con 0x55b73bc529a0 2023-09-13T03:14:11.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.947+0000 7f8c1259c700 1 --2- 172.21.15.64:0/3918100919 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55b73bc529a0 0x55b73bc54dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:11.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.947+0000 7f8bf9ffb700 10 client.26065 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:11.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.947+0000 7f8c1259c700 1 --2- 172.21.15.64:0/3918100919 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55b73bc529a0 0x55b73bc54dc0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fea7b82b6fe2c72 server_cookie=dbd6bf036fd57380 in_seq=0 out_seq=0 2023-09-13T03:14:11.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.948+0000 7f8bf9ffb700 1 -- 172.21.15.64:0/3918100919 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b73bb9f920 con 0x55b73bc529a0 2023-09-13T03:14:11.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.948+0000 7f8bf9ffb700 10 client.26065 handle_command_reply: tid=0 2023-09-13T03:14:11.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.948+0000 7f8c2c78db80 1 client.26065 shutdown 2023-09-13T03:14:11.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.949+0000 7f8c2c78db80 1 -- 172.21.15.64:0/3918100919 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b73bc42ec0 msgr2=0x55b73bc40060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:11.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.949+0000 7f8c2c78db80 1 --2- 172.21.15.64:0/3918100919 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b73bc42ec0 0x55b73bc40060 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c01b700 tx=0x7f8c0c004b40 comp rx=0 tx=0).stop 2023-09-13T03:14:11.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.949+0000 7f8c2c78db80 1 -- 172.21.15.64:0/3918100919 shutdown_connections 2023-09-13T03:14:11.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.949+0000 7f8c2c78db80 1 --2- 172.21.15.64:0/3918100919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b73bc38790 0x55b73bc42840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:11.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.949+0000 7f8c2c78db80 1 --2- 172.21.15.64:0/3918100919 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55b73bc529a0 0x55b73bc54dc0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:11.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.949+0000 7f8c2c78db80 1 --2- 172.21.15.64:0/3918100919 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b73bc42ec0 0x55b73bc40060 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:11.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.949+0000 7f8c2c78db80 1 --2- 172.21.15.64:0/3918100919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b73bc390b0 0x55b73bc3fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:11.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.949+0000 7f8bf9ffb700 0 client.26065 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:11.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.949+0000 7f8c2c78db80 1 -- 172.21.15.64:0/3918100919 >> 172.21.15.64:0/3918100919 conn(0x55b73bbad110 msgr2=0x55b73bc33860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:11.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.949+0000 7f8c2c78db80 1 -- 172.21.15.64:0/3918100919 shutdown_connections 2023-09-13T03:14:11.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.949+0000 7f8c2c78db80 1 -- 172.21.15.64:0/3918100919 wait complete. 2023-09-13T03:14:11.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.949+0000 7f8c2c78db80 20 client.26065 trim_cache size 0 max 16384 2023-09-13T03:14:11.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.967+0000 7f8c2c78db80 1 Processor -- start 2023-09-13T03:14:11.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.967+0000 7f8c2c78db80 1 -- start start 2023-09-13T03:14:11.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.967+0000 7f8c2c78db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b73bc38790 0x55b73bc42370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:11.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.967+0000 7f8c2c78db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b73bc390b0 0x55b73bc3cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:11.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.967+0000 7f8c2c78db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b73bc42ec0 0x55b73bc3d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:11.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.967+0000 7f8c2c78db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b73bc3e0e0 con 0x55b73bc42ec0 2023-09-13T03:14:11.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.967+0000 7f8c2c78db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b73bc32a20 con 0x55b73bc38790 2023-09-13T03:14:11.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.967+0000 7f8c2c78db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b73bc40660 con 0x55b73bc390b0 2023-09-13T03:14:11.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.967+0000 7f8c12d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b73bc42ec0 0x55b73bc3d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:11.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.967+0000 7f8c12d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b73bc42ec0 0x55b73bc3d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60112/0 (socket says 172.21.15.64:60112) 2023-09-13T03:14:11.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.967+0000 7f8c11d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b73bc390b0 0x55b73bc3cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:11.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.967+0000 7f8c12d9d700 1 -- 172.21.15.64:0/2108812288 learned_addr learned my addr 172.21.15.64:0/2108812288 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:11.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.968+0000 7f8c1259c700 1 --2- 172.21.15.64:0/2108812288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b73bc38790 0x55b73bc42370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:11.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.968+0000 7f8c12d9d700 1 -- 172.21.15.64:0/2108812288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b73bc390b0 msgr2=0x55b73bc3cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:11.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.968+0000 7f8c12d9d700 1 --2- 172.21.15.64:0/2108812288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b73bc390b0 0x55b73bc3cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:11.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.968+0000 7f8c12d9d700 1 -- 172.21.15.64:0/2108812288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b73bc38790 msgr2=0x55b73bc42370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:11.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.968+0000 7f8c12d9d700 1 --2- 172.21.15.64:0/2108812288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b73bc38790 0x55b73bc42370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:11.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.968+0000 7f8c12d9d700 1 -- 172.21.15.64:0/2108812288 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b73b99fce0 con 0x55b73bc42ec0 2023-09-13T03:14:11.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.968+0000 7f8c12d9d700 1 --2- 172.21.15.64:0/2108812288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b73bc42ec0 0x55b73bc3d2c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c024f50 tx=0x7f8c0c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c9fa76cf8da781 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:11.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.968+0000 7f8c017fa700 1 -- 172.21.15.64:0/2108812288 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c0196c0 con 0x55b73bc42ec0 2023-09-13T03:14:11.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.968+0000 7f8c017fa700 1 -- 172.21.15.64:0/2108812288 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c0c00f040 con 0x55b73bc42ec0 2023-09-13T03:14:11.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.968+0000 7f8c017fa700 1 -- 172.21.15.64:0/2108812288 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c007da0 con 0x55b73bc42ec0 2023-09-13T03:14:11.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.969+0000 7f8c2c78db80 1 -- 172.21.15.64:0/2108812288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b73bc42ec0 msgr2=0x55b73bc3d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:11.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.969+0000 7f8c2c78db80 1 --2- 172.21.15.64:0/2108812288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b73bc42ec0 0x55b73bc3d2c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c024f50 tx=0x7f8c0c004ad0 comp rx=0 tx=0).stop 2023-09-13T03:14:11.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.969+0000 7f8c2c78db80 1 -- 172.21.15.64:0/2108812288 shutdown_connections 2023-09-13T03:14:11.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.969+0000 7f8c2c78db80 1 --2- 172.21.15.64:0/2108812288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b73bc38790 0x55b73bc42370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:11.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.969+0000 7f8c2c78db80 1 --2- 172.21.15.64:0/2108812288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b73bc42ec0 0x55b73bc3d2c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:11.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.969+0000 7f8c2c78db80 1 --2- 172.21.15.64:0/2108812288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b73bc390b0 0x55b73bc3cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:11.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.969+0000 7f8c2c78db80 1 -- 172.21.15.64:0/2108812288 >> 172.21.15.64:0/2108812288 conn(0x55b73bbad110 msgr2=0x55b73bc3fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:11.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.969+0000 7f8c2c78db80 1 -- 172.21.15.64:0/2108812288 shutdown_connections 2023-09-13T03:14:11.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.969+0000 7f8c2c78db80 1 -- 172.21.15.64:0/2108812288 wait complete. 2023-09-13T03:14:11.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.970+0000 7f8c2c78db80 1 -- start start 2023-09-13T03:14:11.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.970+0000 7f8c2c78db80 1 Processor -- start 2023-09-13T03:14:11.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.970+0000 7f8c2c78db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b73bc38790 0x55b73bc48940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:11.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.970+0000 7f8c2c78db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b73bc390b0 0x55b73bc48e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:11.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.970+0000 7f8c2c78db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b73bc42ec0 0x55b73bca0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:11.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.970+0000 7f8c2c78db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b73bca0e40 con 0x55b73bc38790 2023-09-13T03:14:11.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.970+0000 7f8c2c78db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b73bca0f80 con 0x55b73bc42ec0 2023-09-13T03:14:11.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.970+0000 7f8c2c78db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b73bca1140 con 0x55b73bc390b0 2023-09-13T03:14:11.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.970+0000 7f8c2c78db80 10 client.0 _ll_register_callbacks cb 0x55b73ba72b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:14:11.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.971+0000 7f8c1259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b73bc38790 0x55b73bc48940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:11.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.971+0000 7f8c11d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b73bc390b0 0x55b73bc48e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:11.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.971+0000 7f8c11d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b73bc390b0 0x55b73bc48e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:37132/0 (socket says 172.21.15.64:37132) 2023-09-13T03:14:11.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.971+0000 7f8c11d9b700 1 -- 172.21.15.64:0/3794672285 learned_addr learned my addr 172.21.15.64:0/3794672285 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:11.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.971+0000 7f8c12d9d700 1 --2- 172.21.15.64:0/3794672285 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b73bc42ec0 0x55b73bca0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:11.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.971+0000 7f8c1259c700 1 -- 172.21.15.64:0/3794672285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b73bc390b0 msgr2=0x55b73bc48e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:11.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.971+0000 7f8c1259c700 1 --2- 172.21.15.64:0/3794672285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b73bc390b0 0x55b73bc48e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:11.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.971+0000 7f8c1259c700 1 -- 172.21.15.64:0/3794672285 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b73bc42ec0 msgr2=0x55b73bca0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:11.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.971+0000 7f8c1259c700 1 --2- 172.21.15.64:0/3794672285 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b73bc42ec0 0x55b73bca0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:11.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.971+0000 7f8c1259c700 1 -- 172.21.15.64:0/3794672285 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c1410dcf0 con 0x55b73bc38790 2023-09-13T03:14:11.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.971+0000 7f8c1259c700 1 --2- 172.21.15.64:0/3794672285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b73bc38790 0x55b73bc48940 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8c040c9990 tx=0x7f8c04084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=358e2e473ccc8a3 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:11.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.971+0000 7f8bf9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:14:11.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.971+0000 7f8bf9ffb700 1 -- 172.21.15.64:0/3794672285 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c040a1690 con 0x55b73bc38790 2023-09-13T03:14:11.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.971+0000 7f8bf9ffb700 1 -- 172.21.15.64:0/3794672285 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f8c0409d120 con 0x55b73bc38790 2023-09-13T03:14:11.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.971+0000 7f8c2c78db80 1 -- 172.21.15.64:0/3794672285 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c0801e040 con 0x55b73bc38790 2023-09-13T03:14:11.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.972+0000 7f8bf9ffb700 1 -- 172.21.15.64:0/3794672285 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c04099880 con 0x55b73bc38790 2023-09-13T03:14:11.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.972+0000 7f8c2c78db80 10 client.16029 fetch_fsmap learned FSMap version 28 2023-09-13T03:14:11.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.972+0000 7f8c2c78db80 1 -- 172.21.15.64:0/3794672285 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b73b9dd2d0 con 0x55b73bc38790 2023-09-13T03:14:11.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.972+0000 7f8bf9ffb700 1 -- 172.21.15.64:0/3794672285 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c0801e040 con 0x55b73bc38790 2023-09-13T03:14:11.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.972+0000 7f8bf9ffb700 1 -- 172.21.15.64:0/3794672285 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8c040a1200 con 0x55b73bc38790 2023-09-13T03:14:11.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.972+0000 7f8c2c78db80 10 client.16029 fetch_fsmap finished waiting for FSMap version 28 2023-09-13T03:14:11.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.972+0000 7f8c2c78db80 10 client.16029 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T03:14:11.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.972+0000 7f8c2c78db80 20 client.16029 populate_metadata read hostname 'smithi064' 2023-09-13T03:14:11.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.972+0000 7f8c2c78db80 1 --2- 172.21.15.64:0/3794672285 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55b73bca89c0 0x55b73bcaade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:11.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.972+0000 7f8c2c78db80 4 client.16029 mds_command: new command op to 24686 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-09-13T03:14:11.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.972+0000 7f8c2c78db80 1 -- 172.21.15.64:0/3794672285 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b73bb839a0 con 0x55b73bca89c0 2023-09-13T03:14:11.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.972+0000 7f8c11d9b700 1 --2- 172.21.15.64:0/3794672285 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55b73bca89c0 0x55b73bcaade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:11.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.973+0000 7f8c11d9b700 1 --2- 172.21.15.64:0/3794672285 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55b73bca89c0 0x55b73bcaade0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4ca4c67c79614b3 server_cookie=c691a957efc6579c in_seq=0 out_seq=0 2023-09-13T03:14:11.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.973+0000 7f8bf9ffb700 10 client.16029 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:11.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.974+0000 7f8bf9ffb700 1 -- 172.21.15.64:0/3794672285 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b73bb839a0 con 0x55b73bca89c0 2023-09-13T03:14:11.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.974+0000 7f8bf9ffb700 10 client.16029 handle_command_reply: tid=0 2023-09-13T03:14:11.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.974+0000 7f8c2c78db80 1 client.16029 shutdown 2023-09-13T03:14:11.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.974+0000 7f8c2c78db80 1 -- 172.21.15.64:0/3794672285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b73bc38790 msgr2=0x55b73bc48940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:11.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.974+0000 7f8c2c78db80 1 --2- 172.21.15.64:0/3794672285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b73bc38790 0x55b73bc48940 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8c040c9990 tx=0x7f8c04084040 comp rx=0 tx=0).stop 2023-09-13T03:14:11.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.974+0000 7f8c2c78db80 1 -- 172.21.15.64:0/3794672285 shutdown_connections 2023-09-13T03:14:11.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.974+0000 7f8c2c78db80 1 --2- 172.21.15.64:0/3794672285 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b73bc42ec0 0x55b73bca0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:11.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.974+0000 7f8c2c78db80 1 --2- 172.21.15.64:0/3794672285 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55b73bca89c0 0x55b73bcaade0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:11.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.974+0000 7f8c2c78db80 1 --2- 172.21.15.64:0/3794672285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b73bc38790 0x55b73bc48940 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:11.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.974+0000 7f8c2c78db80 1 --2- 172.21.15.64:0/3794672285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b73bc390b0 0x55b73bc48e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:11.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.974+0000 7f8bf9ffb700 0 client.16029 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:11.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.974+0000 7f8c2c78db80 1 -- 172.21.15.64:0/3794672285 >> 172.21.15.64:0/3794672285 conn(0x55b73bbad110 msgr2=0x55b73bc3e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:11.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.974+0000 7f8c2c78db80 1 -- 172.21.15.64:0/3794672285 shutdown_connections 2023-09-13T03:14:11.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.974+0000 7f8c2c78db80 1 -- 172.21.15.64:0/3794672285 wait complete. 2023-09-13T03:14:11.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.974+0000 7f8c2c78db80 20 client.16029 trim_cache size 0 max 16384 2023-09-13T03:14:11.990 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-13T03:14:11.990 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2023-09-13T03:14:11.991 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "b25c3e6c-9b8f-45ea-bd90-20b180279b6b", 2023-09-13T03:14:11.991 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2023-09-13T03:14:11.991 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-13T03:14:11.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.975+0000 7f8c18dff700 1 -- 172.21.15.64:0/1957801663 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f8bfc0a0950 msgr2=0x7f8bfc0a2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:11.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.975+0000 7f8c18dff700 1 --2- 172.21.15.64:0/1957801663 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f8bfc0a0950 0x7f8bfc0a2e10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c009c90 tx=0x7f8c0c004320 comp rx=0 tx=0).stop 2023-09-13T03:14:11.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.975+0000 7f8c18dff700 1 -- 172.21.15.64:0/1957801663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c14100570 msgr2=0x7f8c14196640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:11.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.975+0000 7f8c18dff700 1 --2- 172.21.15.64:0/1957801663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c14100570 0x7f8c14196640 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8c0400e9a0 tx=0x7f8c0400ecb0 comp rx=0 tx=0).stop 2023-09-13T03:14:11.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.975+0000 7f8c18dff700 1 -- 172.21.15.64:0/1957801663 shutdown_connections 2023-09-13T03:14:11.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.975+0000 7f8c18dff700 1 --2- 172.21.15.64:0/1957801663 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f8bfc0a0950 0x7f8bfc0a2e10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:11.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.975+0000 7f8c18dff700 1 --2- 172.21.15.64:0/1957801663 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c14106630 0x7f8c1419afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:11.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.975+0000 7f8c18dff700 1 --2- 172.21.15.64:0/1957801663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c14100570 0x7f8c14196640 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:11.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.975+0000 7f8c18dff700 1 --2- 172.21.15.64:0/1957801663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c14100f30 0x7f8c14196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:11.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.975+0000 7f8c18dff700 1 -- 172.21.15.64:0/1957801663 >> 172.21.15.64:0/1957801663 conn(0x7f8c140fc080 msgr2=0x7f8c140fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:11.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.975+0000 7f8c18dff700 1 -- 172.21.15.64:0/1957801663 shutdown_connections 2023-09-13T03:14:11.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:11.975+0000 7f8c18dff700 1 -- 172.21.15.64:0/1957801663 wait complete. 2023-09-13T03:14:11.995 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-13T03:14:12.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.208+0000 7fdc91bff700 1 -- 172.21.15.64:0/3410023662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc8c100590 msgr2=0x7fdc8c100a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:12.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.208+0000 7fdc91bff700 1 --2- 172.21.15.64:0/3410023662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc8c100590 0x7fdc8c100a10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdc80009920 tx=0x7fdc80009c30 comp rx=0 tx=0).stop 2023-09-13T03:14:12.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.208+0000 7fdc91bff700 1 -- 172.21.15.64:0/3410023662 shutdown_connections 2023-09-13T03:14:12.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.208+0000 7fdc91bff700 1 --2- 172.21.15.64:0/3410023662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc8c100f50 0x7fdc8c10d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:12.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.208+0000 7fdc91bff700 1 --2- 172.21.15.64:0/3410023662 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc8c106650 0x7fdc8c106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:12.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.208+0000 7fdc91bff700 1 --2- 172.21.15.64:0/3410023662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc8c100590 0x7fdc8c100a10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:12.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.208+0000 7fdc91bff700 1 -- 172.21.15.64:0/3410023662 >> 172.21.15.64:0/3410023662 conn(0x7fdc8c0fc060 msgr2=0x7fdc8c0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:12.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.208+0000 7fdc91bff700 1 -- 172.21.15.64:0/3410023662 shutdown_connections 2023-09-13T03:14:12.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.208+0000 7fdc91bff700 1 -- 172.21.15.64:0/3410023662 wait complete. 2023-09-13T03:14:12.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.209+0000 7fdc91bff700 1 Processor -- start 2023-09-13T03:14:12.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.209+0000 7fdc91bff700 1 -- start start 2023-09-13T03:14:12.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.210+0000 7fdc91bff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc8c100590 0x7fdc8c19e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:12.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.210+0000 7fdc91bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc8c100f50 0x7fdc8c19ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:12.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.210+0000 7fdc91bff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc8c106650 0x7fdc8c198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:12.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.210+0000 7fdc91bff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c198db0 con 0x7fdc8c106650 2023-09-13T03:14:12.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.210+0000 7fdc91bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c198f20 con 0x7fdc8c100f50 2023-09-13T03:14:12.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.210+0000 7fdc91bff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c1990e0 con 0x7fdc8c100590 2023-09-13T03:14:12.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.210+0000 7fdc8b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc8c100590 0x7fdc8c19e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:12.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.210+0000 7fdc8bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc8c106650 0x7fdc8c198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:12.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.210+0000 7fdc8b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc8c100590 0x7fdc8c19e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:37154/0 (socket says 172.21.15.64:37154) 2023-09-13T03:14:12.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.210+0000 7fdc8b7fe700 1 -- 172.21.15.64:0/1087783240 learned_addr learned my addr 172.21.15.64:0/1087783240 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:12.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.210+0000 7fdc8affd700 1 --2- 172.21.15.64:0/1087783240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc8c100f50 0x7fdc8c19ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:12.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.211+0000 7fdc8bfff700 1 -- 172.21.15.64:0/1087783240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc8c100590 msgr2=0x7fdc8c19e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:12.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.211+0000 7fdc8bfff700 1 --2- 172.21.15.64:0/1087783240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc8c100590 0x7fdc8c19e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:12.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.211+0000 7fdc8bfff700 1 -- 172.21.15.64:0/1087783240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc8c100f50 msgr2=0x7fdc8c19ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:12.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.211+0000 7fdc8bfff700 1 --2- 172.21.15.64:0/1087783240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc8c100f50 0x7fdc8c19ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:12.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.211+0000 7fdc8bfff700 1 -- 172.21.15.64:0/1087783240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc84009c40 con 0x7fdc8c106650 2023-09-13T03:14:12.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.211+0000 7fdc8affd700 1 --2- 172.21.15.64:0/1087783240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc8c100f50 0x7fdc8c19ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:12.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.211+0000 7fdc8bfff700 1 --2- 172.21.15.64:0/1087783240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc8c106650 0x7fdc8c198740 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fdc8400efb0 tx=0x7fdc8400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:12.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.211+0000 7fdc88ff9700 1 -- 172.21.15.64:0/1087783240 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc84003a00 con 0x7fdc8c106650 2023-09-13T03:14:12.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.211+0000 7fdc91bff700 1 -- 172.21.15.64:0/1087783240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc7c0095f0 con 0x7fdc8c106650 2023-09-13T03:14:12.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.211+0000 7fdc88ff9700 1 -- 172.21.15.64:0/1087783240 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc8400f040 con 0x7fdc8c106650 2023-09-13T03:14:12.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.211+0000 7fdc88ff9700 1 -- 172.21.15.64:0/1087783240 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc84004550 con 0x7fdc8c106650 2023-09-13T03:14:12.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.211+0000 7fdc91bff700 1 -- 172.21.15.64:0/1087783240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc80009520 con 0x7fdc8c106650 2023-09-13T03:14:12.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.214+0000 7fdc88ff9700 1 -- 172.21.15.64:0/1087783240 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) v1 ==== 135492+0+0 (secure 0 0 0) 0x7fdc84003bf0 con 0x7fdc8c106650 2023-09-13T03:14:12.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.216+0000 7fdca547eb80 1 Processor -- start 2023-09-13T03:14:12.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.216+0000 7fdc88ff9700 1 --2- 172.21.15.64:0/1087783240 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fdc740a0950 0x7fdc740a2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:12.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.216+0000 7fdca547eb80 1 -- start start 2023-09-13T03:14:12.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.216+0000 7fdc88ff9700 1 -- 172.21.15.64:0/1087783240 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fdc840cbe10 con 0x7fdc8c106650 2023-09-13T03:14:12.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.216+0000 7fdc8affd700 1 --2- 172.21.15.64:0/1087783240 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fdc740a0950 0x7fdc740a2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:12.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.217+0000 7fdca547eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55993c9bc710 0x55993c9bcaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:12.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.217+0000 7fdca547eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55993c9bd030 0x55993c9c6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:12.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.217+0000 7fdca547eb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993c9c6e40 0x55993c9c9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:12.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.217+0000 7fdca547eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55993c915730 con 0x55993c9bd030 2023-09-13T03:14:12.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.217+0000 7fdca547eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55993c913050 con 0x55993c9bc710 2023-09-13T03:14:12.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.217+0000 7fdca547eb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55993c90e940 con 0x55993c9c6e40 2023-09-13T03:14:12.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.217+0000 7fdc8b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55993c9bc710 0x55993c9bcaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:12.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.217+0000 7fdc8bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993c9c6e40 0x55993c9c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:12.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.217+0000 7fdc8b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55993c9bc710 0x55993c9bcaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:53906/0 (socket says 172.21.15.64:53906) 2023-09-13T03:14:12.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.217+0000 7fdc8b7fe700 1 -- 172.21.15.64:0/3240934972 learned_addr learned my addr 172.21.15.64:0/3240934972 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:12.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.217+0000 7fdc8affd700 1 --2- 172.21.15.64:0/3240934972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55993c9bd030 0x55993c9c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:12.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.217+0000 7fdc8bfff700 1 -- 172.21.15.64:0/3240934972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55993c9bc710 msgr2=0x55993c9bcaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:12.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.217+0000 7fdc8bfff700 1 --2- 172.21.15.64:0/3240934972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55993c9bc710 0x55993c9bcaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:12.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.217+0000 7fdc8bfff700 1 -- 172.21.15.64:0/3240934972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55993c9bd030 msgr2=0x55993c9c6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:12.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.217+0000 7fdc8bfff700 1 --2- 172.21.15.64:0/3240934972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55993c9bd030 0x55993c9c6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:12.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.217+0000 7fdc8bfff700 1 -- 172.21.15.64:0/3240934972 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55993c73ade0 con 0x55993c9c6e40 2023-09-13T03:14:12.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.217+0000 7fdc8affd700 1 --2- 172.21.15.64:0/1087783240 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fdc740a0950 0x7fdc740a2e10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdc8000c010 tx=0x7fdc8000f4c0 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:12.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.218+0000 7fdc8bfff700 1 --2- 172.21.15.64:0/3240934972 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993c9c6e40 0x55993c9c9230 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdc840c93d0 tx=0x7fdc84098ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d756e07db41af3f server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:12.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.218+0000 7fdc71ffb700 1 -- 172.21.15.64:0/3240934972 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc840997f0 con 0x55993c9c6e40 2023-09-13T03:14:12.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.218+0000 7fdc71ffb700 1 -- 172.21.15.64:0/3240934972 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc840990f0 con 0x55993c9c6e40 2023-09-13T03:14:12.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.218+0000 7fdc71ffb700 1 -- 172.21.15.64:0/3240934972 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc840046b0 con 0x55993c9c6e40 2023-09-13T03:14:12.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.218+0000 7fdca547eb80 1 -- 172.21.15.64:0/3240934972 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993c9c6e40 msgr2=0x55993c9c9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:12.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.218+0000 7fdca547eb80 1 --2- 172.21.15.64:0/3240934972 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993c9c6e40 0x55993c9c9230 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdc840c93d0 tx=0x7fdc84098ec0 comp rx=0 tx=0).stop 2023-09-13T03:14:12.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.219+0000 7fdca547eb80 1 -- 172.21.15.64:0/3240934972 shutdown_connections 2023-09-13T03:14:12.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.219+0000 7fdca547eb80 1 --2- 172.21.15.64:0/3240934972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55993c9bc710 0x55993c9bcaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:12.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.219+0000 7fdca547eb80 1 --2- 172.21.15.64:0/3240934972 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993c9c6e40 0x55993c9c9230 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:12.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.219+0000 7fdca547eb80 1 --2- 172.21.15.64:0/3240934972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55993c9bd030 0x55993c9c6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:12.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.219+0000 7fdca547eb80 1 -- 172.21.15.64:0/3240934972 >> 172.21.15.64:0/3240934972 conn(0x55993c931090 msgr2=0x55993c9b7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:12.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.219+0000 7fdca547eb80 1 -- 172.21.15.64:0/3240934972 shutdown_connections 2023-09-13T03:14:12.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.219+0000 7fdca547eb80 1 -- 172.21.15.64:0/3240934972 wait complete. 2023-09-13T03:14:12.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.219+0000 7fdca547eb80 1 -- start start 2023-09-13T03:14:12.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.220+0000 7fdca547eb80 1 Processor -- start 2023-09-13T03:14:12.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.221+0000 7fdca547eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55993c9bc710 0x55993c9c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:12.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.221+0000 7fdca547eb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993c9bd030 0x55993c9c3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:12.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.221+0000 7fdca547eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55993c9c6e40 0x55993c9c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:12.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.221+0000 7fdca547eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55993c8acb70 con 0x55993c9bc710 2023-09-13T03:14:12.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.221+0000 7fdca547eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55993c8ac870 con 0x55993c9c6e40 2023-09-13T03:14:12.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.221+0000 7fdca547eb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55993c790920 con 0x55993c9bd030 2023-09-13T03:14:12.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.221+0000 7fdca547eb80 10 client.0 _ll_register_callbacks cb 0x55993c7f6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:14:12.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.221+0000 7fdc8b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55993c9bc710 0x55993c9c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:12.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.221+0000 7fdc8bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55993c9c6e40 0x55993c9c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:12.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.221+0000 7fdc8bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55993c9c6e40 0x55993c9c42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:53912/0 (socket says 172.21.15.64:53912) 2023-09-13T03:14:12.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.221+0000 7fdc8bfff700 1 -- 172.21.15.64:0/3048465668 learned_addr learned my addr 172.21.15.64:0/3048465668 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:12.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.221+0000 7fdc8affd700 1 --2- 172.21.15.64:0/3048465668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993c9bd030 0x55993c9c3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:12.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.221+0000 7fdc8bfff700 1 -- 172.21.15.64:0/3048465668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993c9bd030 msgr2=0x55993c9c3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:12.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.221+0000 7fdc8bfff700 1 --2- 172.21.15.64:0/3048465668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993c9bd030 0x55993c9c3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:12.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.221+0000 7fdc8bfff700 1 -- 172.21.15.64:0/3048465668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55993c9bc710 msgr2=0x55993c9c68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:12.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.221+0000 7fdc8bfff700 1 --2- 172.21.15.64:0/3048465668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55993c9bc710 0x55993c9c68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:12.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.221+0000 7fdc8bfff700 1 -- 172.21.15.64:0/3048465668 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc80009520 con 0x55993c9c6e40 2023-09-13T03:14:12.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.222+0000 7fdc8bfff700 1 --2- 172.21.15.64:0/3048465668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55993c9c6e40 0x55993c9c42a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdc8400c6f0 tx=0x7fdc84009f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=192a5136edcafa5d server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:12.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.222+0000 7fdc6affd700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:14:12.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.222+0000 7fdc6affd700 1 -- 172.21.15.64:0/3048465668 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc840048b0 con 0x55993c9c6e40 2023-09-13T03:14:12.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.222+0000 7fdca547eb80 1 -- 172.21.15.64:0/3048465668 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55993c8664f0 con 0x55993c9c6e40 2023-09-13T03:14:12.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.222+0000 7fdc6affd700 1 -- 172.21.15.64:0/3048465668 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fdc8409cf20 con 0x55993c9c6e40 2023-09-13T03:14:12.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.222+0000 7fdc6affd700 1 -- 172.21.15.64:0/3048465668 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc840b70c0 con 0x55993c9c6e40 2023-09-13T03:14:12.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.222+0000 7fdca547eb80 10 client.26081 fetch_fsmap learned FSMap version 28 2023-09-13T03:14:12.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.222+0000 7fdca547eb80 1 -- 172.21.15.64:0/3048465668 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55993c723c60 con 0x55993c9c6e40 2023-09-13T03:14:12.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.222+0000 7fdc6affd700 1 -- 172.21.15.64:0/3048465668 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55993c8664f0 con 0x55993c9c6e40 2023-09-13T03:14:12.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.222+0000 7fdc6affd700 1 -- 172.21.15.64:0/3048465668 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdc840985e0 con 0x55993c9c6e40 2023-09-13T03:14:12.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.222+0000 7fdca547eb80 10 client.26081 fetch_fsmap finished waiting for FSMap version 28 2023-09-13T03:14:12.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.222+0000 7fdca547eb80 10 client.26081 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T03:14:12.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.223+0000 7fdca547eb80 20 client.26081 populate_metadata read hostname 'smithi064' 2023-09-13T03:14:12.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.223+0000 7fdca547eb80 1 --2- 172.21.15.64:0/3048465668 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55993c9d6740 0x55993c9d8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:12.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.223+0000 7fdca547eb80 4 client.26081 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:14:12.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.223+0000 7fdca547eb80 1 -- 172.21.15.64:0/3048465668 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55993c9238a0 con 0x55993c9d6740 2023-09-13T03:14:12.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.223+0000 7fdc8b7fe700 1 --2- 172.21.15.64:0/3048465668 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55993c9d6740 0x55993c9d8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:12.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.223+0000 7fdc8b7fe700 1 --2- 172.21.15.64:0/3048465668 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55993c9d6740 0x55993c9d8b60 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89dca149c05194b3 server_cookie=f6c83b38cc3318c0 in_seq=0 out_seq=0 2023-09-13T03:14:12.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.223+0000 7fdc6affd700 10 client.26081 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:12.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.224+0000 7fdc6affd700 1 -- 172.21.15.64:0/3048465668 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55993c9238a0 con 0x55993c9d6740 2023-09-13T03:14:12.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.224+0000 7fdc6affd700 10 client.26081 handle_command_reply: tid=0 2023-09-13T03:14:12.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.224+0000 7fdca547eb80 1 client.26081 shutdown 2023-09-13T03:14:12.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.224+0000 7fdca547eb80 1 -- 172.21.15.64:0/3048465668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55993c9c6e40 msgr2=0x55993c9c42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:12.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.224+0000 7fdca547eb80 1 --2- 172.21.15.64:0/3048465668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55993c9c6e40 0x55993c9c42a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdc8400c6f0 tx=0x7fdc84009f30 comp rx=0 tx=0).stop 2023-09-13T03:14:12.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.224+0000 7fdca547eb80 1 -- 172.21.15.64:0/3048465668 shutdown_connections 2023-09-13T03:14:12.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.224+0000 7fdca547eb80 1 --2- 172.21.15.64:0/3048465668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55993c9c6e40 0x55993c9c42a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:12.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.224+0000 7fdca547eb80 1 --2- 172.21.15.64:0/3048465668 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55993c9d6740 0x55993c9d8b60 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:12.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.224+0000 7fdca547eb80 1 --2- 172.21.15.64:0/3048465668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55993c9bc710 0x55993c9c68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:12.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.224+0000 7fdca547eb80 1 --2- 172.21.15.64:0/3048465668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993c9bd030 0x55993c9c3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:12.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.224+0000 7fdc6affd700 0 client.26081 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:12.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.224+0000 7fdca547eb80 1 -- 172.21.15.64:0/3048465668 >> 172.21.15.64:0/3048465668 conn(0x55993c931090 msgr2=0x55993c9b7780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:12.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.224+0000 7fdca547eb80 1 -- 172.21.15.64:0/3048465668 shutdown_connections 2023-09-13T03:14:12.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.224+0000 7fdca547eb80 1 -- 172.21.15.64:0/3048465668 wait complete. 2023-09-13T03:14:12.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.225+0000 7fdca547eb80 20 client.26081 trim_cache size 0 max 16384 2023-09-13T03:14:12.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.243+0000 7fdca547eb80 1 Processor -- start 2023-09-13T03:14:12.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.243+0000 7fdca547eb80 1 -- start start 2023-09-13T03:14:12.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.243+0000 7fdca547eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55993c9bc710 0x55993c9bee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:12.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.243+0000 7fdca547eb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993c9bd030 0x55993c9bf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:12.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.243+0000 7fdca547eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55993c9c6e40 0x55993c9c0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:12.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.243+0000 7fdca547eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55993c9be210 con 0x55993c9c6e40 2023-09-13T03:14:12.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.243+0000 7fdca547eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55993c9b8860 con 0x55993c9bc710 2023-09-13T03:14:12.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.243+0000 7fdca547eb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55993c9bd9c0 con 0x55993c9bd030 2023-09-13T03:14:12.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.243+0000 7fdc8b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55993c9bc710 0x55993c9bee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:12.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.243+0000 7fdc8b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55993c9bc710 0x55993c9bee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:53924/0 (socket says 172.21.15.64:53924) 2023-09-13T03:14:12.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.243+0000 7fdc8b7fe700 1 -- 172.21.15.64:0/1477256850 learned_addr learned my addr 172.21.15.64:0/1477256850 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:12.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.243+0000 7fdc8affd700 1 --2- 172.21.15.64:0/1477256850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993c9bd030 0x55993c9bf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:12.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.244+0000 7fdc8bfff700 1 --2- 172.21.15.64:0/1477256850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55993c9c6e40 0x55993c9c0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:12.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.244+0000 7fdc8b7fe700 1 -- 172.21.15.64:0/1477256850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993c9bd030 msgr2=0x55993c9bf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:12.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.244+0000 7fdc8b7fe700 1 --2- 172.21.15.64:0/1477256850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993c9bd030 0x55993c9bf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:12.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.244+0000 7fdc8b7fe700 1 -- 172.21.15.64:0/1477256850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55993c9c6e40 msgr2=0x55993c9c0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:12.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.244+0000 7fdc8b7fe700 1 --2- 172.21.15.64:0/1477256850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55993c9c6e40 0x55993c9c0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:12.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.244+0000 7fdc8b7fe700 1 -- 172.21.15.64:0/1477256850 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55993c761250 con 0x55993c9bc710 2023-09-13T03:14:12.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.244+0000 7fdc8b7fe700 1 --2- 172.21.15.64:0/1477256850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55993c9bc710 0x55993c9bee40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c005d10 tx=0x7fdc7c009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d00bab3b9c025797 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:12.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.244+0000 7fdc727fc700 1 -- 172.21.15.64:0/1477256850 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c017040 con 0x55993c9bc710 2023-09-13T03:14:12.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.244+0000 7fdc727fc700 1 -- 172.21.15.64:0/1477256850 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc7c018040 con 0x55993c9bc710 2023-09-13T03:14:12.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.244+0000 7fdc727fc700 1 -- 172.21.15.64:0/1477256850 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c005280 con 0x55993c9bc710 2023-09-13T03:14:12.251 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:14:11 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-h[170323]: 2023-09-13T03:14:11.981+0000 7f5cc35fa700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021f [2,head] /volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT auth{0=1} v20 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25375=0-4194304@1} caps={25375=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25375 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55845e2a4580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000021f:[<0x10000000215/ROSE.CPT v20>,<0x10000000009/COREL v4731>,<0x10000000008/~dmtmp v83>,<0x10000000007/client0 v35>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v26>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v40>,<0x10000000001/sv_1 v61>,<0x10000000000/_nogroup v47>,<0x1/volumes v44>]//[]","memoryvalue":"(3)0x1000000021f:[<0x10000000215/ROSE.CPT v20>,<0x10000000009/COREL v4731>,<0x10000000008/~dmtmp v85>,<0x10000000007/client0 v38>,<0x10000000006/clients v28>,<0x10000000005/tmp v28>,<0x100000001fb/client.0 v28>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v43>,<0x10000000001/sv_1 v63>,<0x10000000000/_nogroup v37>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T03:14:12.251 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:14:11 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-h[170323]: 2023-09-13T03:14:11.981+0000 7f5cc35fa700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021b [2,head] /volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT auth{0=1} v12 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25375=0-4194304@1} caps={25375=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25375 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55845e2bdb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000021b:[<0x10000000215/TEAPOT2.CPT v12>,<0x10000000009/COREL v4731>,<0x10000000008/~dmtmp v83>,<0x10000000007/client0 v35>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v26>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v40>,<0x10000000001/sv_1 v61>,<0x10000000000/_nogroup v47>,<0x1/volumes v44>]//[]","memoryvalue":"(3)0x1000000021b:[<0x10000000215/TEAPOT2.CPT v12>,<0x10000000009/COREL v4731>,<0x10000000008/~dmtmp v85>,<0x10000000007/client0 v38>,<0x10000000006/clients v28>,<0x10000000005/tmp v28>,<0x100000001fb/client.0 v28>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v43>,<0x10000000001/sv_1 v63>,<0x10000000000/_nogroup v37>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T03:14:12.252 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:14:11 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-h[170323]: 2023-09-13T03:14:11.982+0000 7f5cc35fa700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000220 [2,head] /volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT auth{0=1} v22 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25375=0-4194304@1} caps={25375=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25375 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55845e2be680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000220:[<0x10000000215/CORELDRW.CDT v22>,<0x10000000009/COREL v4731>,<0x10000000008/~dmtmp v83>,<0x10000000007/client0 v35>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v26>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v40>,<0x10000000001/sv_1 v61>,<0x10000000000/_nogroup v47>,<0x1/volumes v44>]//[]","memoryvalue":"(3)0x10000000220:[<0x10000000215/CORELDRW.CDT v22>,<0x10000000009/COREL v4731>,<0x10000000008/~dmtmp v85>,<0x10000000007/client0 v38>,<0x10000000006/clients v28>,<0x10000000005/tmp v28>,<0x100000001fb/client.0 v28>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v43>,<0x10000000001/sv_1 v63>,<0x10000000000/_nogroup v37>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T03:14:12.252 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:14:11 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-h[170323]: 2023-09-13T03:14:11.982+0000 7f5cc35fa700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000580 [3,head] /volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPH2.BAK auth v428 ap=1 s=0 n(v0 rc2023-09-13T03:14:05.013161+0000 1=1+0)/n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25375=0-4194304@2} caps={25375=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25375 | importingcaps=0 caps=1 dirtyrstat=1 replicated=0 authpin=1 scrubqueue=0 0x55845e2bec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000580:[<0x10000000215/GRAPH2.BAK v428>,<0x10000000009/COREL v4731>,<0x10000000008/~dmtmp v83>,<0x10000000007/client0 v35>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v26>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v40>,<0x10000000001/sv_1 v61>,<0x10000000000/_nogroup v47>,<0x1/volumes v44>]//[]","memoryvalue":"(3)0x10000000580:[<0x10000000215/GRAPH2.BAK v428>,<0x10000000009/COREL v4731>,<0x10000000008/~dmtmp v85>,<0x10000000007/client0 v38>,<0x10000000006/clients v28>,<0x10000000005/tmp v28>,<0x100000001fb/client.0 v28>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v43>,<0x10000000001/sv_1 v63>,<0x10000000000/_nogroup v37>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T03:14:12.252 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:14:11 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-h[170323]: 2023-09-13T03:14:11.982+0000 7f5cc35fa700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021a [2,head] /volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT auth{0=1} v10 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25375=0-4194304@1} caps={25375=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25375 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55845e2be100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000021a:[<0x10000000215/MAILBOX2.CPT v10>,<0x10000000009/COREL v4731>,<0x10000000008/~dmtmp v83>,<0x10000000007/client0 v35>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v26>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v40>,<0x10000000001/sv_1 v61>,<0x10000000000/_nogroup v47>,<0x1/volumes v44>]//[]","memoryvalue":"(3)0x1000000021a:[<0x10000000215/MAILBOX2.CPT v10>,<0x10000000009/COREL v4731>,<0x10000000008/~dmtmp v85>,<0x10000000007/client0 v38>,<0x10000000006/clients v28>,<0x10000000005/tmp v28>,<0x100000001fb/client.0 v28>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v43>,<0x10000000001/sv_1 v63>,<0x10000000000/_nogroup v37>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T03:14:12.252 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:14:11 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-h[170323]: 2023-09-13T03:14:11.982+0000 7f5cc35fa700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021e [2,head] /volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR auth{0=1} v18 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25375=0-4194304@1} caps={25375=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25375 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55845e2bf180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000021e:[<0x10000000215/GRAPHIC2.CDR v18>,<0x10000000009/COREL v4731>,<0x10000000008/~dmtmp v83>,<0x10000000007/client0 v35>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v26>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v40>,<0x10000000001/sv_1 v61>,<0x10000000000/_nogroup v47>,<0x1/volumes v44>]//[]","memoryvalue":"(3)0x1000000021e:[<0x10000000215/GRAPHIC2.CDR v18>,<0x10000000009/COREL v4731>,<0x10000000008/~dmtmp v85>,<0x10000000007/client0 v38>,<0x10000000006/clients v28>,<0x10000000005/tmp v28>,<0x100000001fb/client.0 v28>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v43>,<0x10000000001/sv_1 v63>,<0x10000000000/_nogroup v37>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T03:14:12.253 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:14:11 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-h[170323]: 2023-09-13T03:14:11.983+0000 7f5cc35fa700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000218 [2,head] /volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL auth{0=1} v6 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25375=0-4194304@1} caps={25375=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25375 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55845e2a5080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000218:[<0x10000000215/ARTISTIC.ACL v6>,<0x10000000009/COREL v4731>,<0x10000000008/~dmtmp v83>,<0x10000000007/client0 v35>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v26>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v40>,<0x10000000001/sv_1 v61>,<0x10000000000/_nogroup v47>,<0x1/volumes v44>]//[]","memoryvalue":"(3)0x10000000218:[<0x10000000215/ARTISTIC.ACL v6>,<0x10000000009/COREL v4731>,<0x10000000008/~dmtmp v85>,<0x10000000007/client0 v38>,<0x10000000006/clients v28>,<0x10000000005/tmp v28>,<0x100000001fb/client.0 v28>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v43>,<0x10000000001/sv_1 v63>,<0x10000000000/_nogroup v37>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T03:14:12.253 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:14:11 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-h[170323]: 2023-09-13T03:14:11.983+0000 7f5cc35fa700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000057d [3,head] /volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPH1.CDR auth{0=1} v406 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25375=0-4194304@2} caps={25375=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25375 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55845e2a5600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000057d:[<0x10000000215/GRAPH1.CDR v406>,<0x10000000009/COREL v4731>,<0x10000000008/~dmtmp v83>,<0x10000000007/client0 v35>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v26>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v40>,<0x10000000001/sv_1 v61>,<0x10000000000/_nogroup v47>,<0x1/volumes v44>]//[]","memoryvalue":"(3)0x1000000057d:[<0x10000000215/GRAPH1.CDR v406>,<0x10000000009/COREL v4731>,<0x10000000008/~dmtmp v85>,<0x10000000007/client0 v38>,<0x10000000006/clients v28>,<0x10000000005/tmp v28>,<0x100000001fb/client.0 v28>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v43>,<0x10000000001/sv_1 v63>,<0x10000000000/_nogroup v37>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T03:14:12.253 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:14:11 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-h[170323]: 2023-09-13T03:14:11.983+0000 7f5cc35fa700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021c [2,head] /volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT auth{0=1} v14 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25375=0-4194304@1} caps={25375=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25375 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55845e2a5b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000021c:[<0x10000000215/SIGN2.CPT v14>,<0x10000000009/COREL v4731>,<0x10000000008/~dmtmp v83>,<0x10000000007/client0 v35>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v26>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v40>,<0x10000000001/sv_1 v61>,<0x10000000000/_nogroup v47>,<0x1/volumes v44>]//[]","memoryvalue":"(3)0x1000000021c:[<0x10000000215/SIGN2.CPT v14>,<0x10000000009/COREL v4731>,<0x10000000008/~dmtmp v85>,<0x10000000007/client0 v38>,<0x10000000006/clients v28>,<0x10000000005/tmp v28>,<0x100000001fb/client.0 v28>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v43>,<0x10000000001/sv_1 v63>,<0x10000000000/_nogroup v37>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T03:14:12.253 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:14:11 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-h[170323]: 2023-09-13T03:14:11.983+0000 7f5cc35fa700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000217 [2,head] /volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG auth{0=1} v4 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25375=0-4194304@1} caps={25375=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25375 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55845e2a6100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000217:[<0x10000000215/CDRBARS.CFG v4>,<0x10000000009/COREL v4731>,<0x10000000008/~dmtmp v83>,<0x10000000007/client0 v35>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v26>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v40>,<0x10000000001/sv_1 v61>,<0x10000000000/_nogroup v47>,<0x1/volumes v44>]//[]","memoryvalue":"(3)0x10000000217:[<0x10000000215/CDRBARS.CFG v4>,<0x10000000009/COREL v4731>,<0x10000000008/~dmtmp v85>,<0x10000000007/client0 v38>,<0x10000000006/clients v28>,<0x10000000005/tmp v28>,<0x100000001fb/client.0 v28>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v43>,<0x10000000001/sv_1 v63>,<0x10000000000/_nogroup v37>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T03:14:12.254 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:14:11 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-h[170323]: 2023-09-13T03:14:11.984+0000 7f5cc35fa700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000057c [3,head] /volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPH1.BAK auth v408 ap=1 s=0 n(v0 rc2023-09-13T03:14:04.973162+0000 1=1+0)/n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25375=0-4194304@2} caps={25375=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25375 | importingcaps=0 caps=1 dirtyrstat=1 replicated=0 authpin=1 scrubqueue=0 0x55845e2bf700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000057c:[<0x10000000215/GRAPH1.BAK v408>,<0x10000000009/COREL v4731>,<0x10000000008/~dmtmp v83>,<0x10000000007/client0 v35>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v26>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v40>,<0x10000000001/sv_1 v61>,<0x10000000000/_nogroup v47>,<0x1/volumes v44>]//[]","memoryvalue":"(3)0x1000000057c:[<0x10000000215/GRAPH1.BAK v408>,<0x10000000009/COREL v4731>,<0x10000000008/~dmtmp v85>,<0x10000000007/client0 v38>,<0x10000000006/clients v28>,<0x10000000005/tmp v28>,<0x100000001fb/client.0 v28>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v43>,<0x10000000001/sv_1 v63>,<0x10000000000/_nogroup v37>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T03:14:12.254 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:14:11 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-h[170323]: 2023-09-13T03:14:11.984+0000 7f5cc35fa700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000581 [3,head] /volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPH2.CDR auth{0=1} v426 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25375=0-4194304@2} caps={25375=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25375 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55845e2a6680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000581:[<0x10000000215/GRAPH2.CDR v426>,<0x10000000009/COREL v4731>,<0x10000000008/~dmtmp v83>,<0x10000000007/client0 v35>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v26>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v40>,<0x10000000001/sv_1 v61>,<0x10000000000/_nogroup v47>,<0x1/volumes v44>]//[]","memoryvalue":"(3)0x10000000581:[<0x10000000215/GRAPH2.CDR v426>,<0x10000000009/COREL v4731>,<0x10000000008/~dmtmp v85>,<0x10000000007/client0 v38>,<0x10000000006/clients v28>,<0x10000000005/tmp v28>,<0x100000001fb/client.0 v28>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v43>,<0x10000000001/sv_1 v63>,<0x10000000000/_nogroup v37>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T03:14:12.254 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:14:11 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-h[170323]: 2023-09-13T03:14:11.984+0000 7f5cc35fa700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020d [2,head] /volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25375=0-4194304@1} caps={25375=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25375 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55845e316000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000020d:[<0x1000000020c/PCMAGCD.XLS v2>,<0x10000000009/EXCEL v4571>,<0x10000000008/~dmtmp v83>,<0x10000000007/client0 v35>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v26>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v40>,<0x10000000001/sv_1 v61>,<0x10000000000/_nogroup v47>,<0x1/volumes v44>]//[]","memoryvalue":"(3)0x1000000020d:[<0x1000000020c/PCMAGCD.XLS v2>,<0x10000000009/EXCEL v4571>,<0x10000000008/~dmtmp v85>,<0x10000000007/client0 v38>,<0x10000000006/clients v28>,<0x10000000005/tmp v28>,<0x100000001fb/client.0 v28>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v43>,<0x10000000001/sv_1 v63>,<0x10000000000/_nogroup v37>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T03:14:12.254 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:14:11 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-h[170323]: 2023-09-13T03:14:11.984+0000 7f5cc35fa700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021d [2,head] /volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR auth{0=1} v16 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25375=0-4194304@1} caps={25375=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25375 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55845e2a6c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000021d:[<0x10000000215/TRAIN.CDR v16>,<0x10000000009/COREL v4731>,<0x10000000008/~dmtmp v83>,<0x10000000007/client0 v35>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v26>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v40>,<0x10000000001/sv_1 v61>,<0x10000000000/_nogroup v47>,<0x1/volumes v44>]//[]","memoryvalue":"(3)0x1000000021d:[<0x10000000215/TRAIN.CDR v16>,<0x10000000009/COREL v4731>,<0x10000000008/~dmtmp v85>,<0x10000000007/client0 v38>,<0x10000000006/clients v28>,<0x10000000005/tmp v28>,<0x100000001fb/client.0 v28>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v43>,<0x10000000001/sv_1 v63>,<0x10000000000/_nogroup v37>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T03:14:12.255 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:14:11 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-h[170323]: 2023-09-13T03:14:11.985+0000 7f5cc35fa700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000055f [3,head] /volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES1.XLS auth{0=1} v656 ap=1 s=0 n(v0 rc2023-09-13T03:14:04.693168+0000 1=1+0)/n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25375=0-4194304@2} caps={25375=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25375 | importingcaps=0 caps=1 dirtyrstat=1 replicated=1 authpin=1 scrubqueue=0 0x55845e316580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000055f:[<0x1000000020c/SALES1.XLS v656>,<0x10000000009/EXCEL v4571>,<0x10000000008/~dmtmp v83>,<0x10000000007/client0 v35>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v26>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v40>,<0x10000000001/sv_1 v61>,<0x10000000000/_nogroup v47>,<0x1/volumes v44>]//[]","memoryvalue":"(3)0x1000000055f:[<0x1000000020c/SALES1.XLS v656>,<0x10000000009/EXCEL v4571>,<0x10000000008/~dmtmp v85>,<0x10000000007/client0 v38>,<0x10000000006/clients v28>,<0x10000000005/tmp v28>,<0x100000001fb/client.0 v28>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v43>,<0x10000000001/sv_1 v63>,<0x10000000000/_nogroup v37>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T03:14:12.255 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:14:11 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-h[170323]: 2023-09-13T03:14:11.985+0000 7f5cc35fa700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000219 [2,head] /volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR auth{0=1} v8 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25375=0-4194304@1} caps={25375=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25375 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55845e2a7180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000219:[<0x10000000215/GRAPHIC1.CDR v8>,<0x10000000009/COREL v4731>,<0x10000000008/~dmtmp v83>,<0x10000000007/client0 v35>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v26>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v40>,<0x10000000001/sv_1 v61>,<0x10000000000/_nogroup v47>,<0x1/volumes v44>]//[]","memoryvalue":"(3)0x10000000219:[<0x10000000215/GRAPHIC1.CDR v8>,<0x10000000009/COREL v4731>,<0x10000000008/~dmtmp v85>,<0x10000000007/client0 v38>,<0x10000000006/clients v28>,<0x10000000005/tmp v28>,<0x100000001fb/client.0 v28>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v43>,<0x10000000001/sv_1 v63>,<0x10000000000/_nogroup v37>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T03:14:12.255 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:14:11 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-h[170323]: 2023-09-13T03:14:11.985+0000 7f5cc35fa700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000216 [2,head] /volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG auth{0=1} v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25375=0-4194304@1} caps={25375=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25375 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55845e2a7700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000216:[<0x10000000215/CDRROLS.CFG v2>,<0x10000000009/COREL v4731>,<0x10000000008/~dmtmp v83>,<0x10000000007/client0 v35>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v26>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v40>,<0x10000000001/sv_1 v61>,<0x10000000000/_nogroup v47>,<0x1/volumes v44>]//[]","memoryvalue":"(3)0x10000000216:[<0x10000000215/CDRROLS.CFG v2>,<0x10000000009/COREL v4731>,<0x10000000008/~dmtmp v85>,<0x10000000007/client0 v38>,<0x10000000006/clients v28>,<0x10000000005/tmp v28>,<0x100000001fb/client.0 v28>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v43>,<0x10000000001/sv_1 v63>,<0x10000000000/_nogroup v37>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T03:14:12.255 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:14:11 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-h[170323]: 2023-09-13T03:14:11.986+0000 7f5cc35fa700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020e [2,head] /volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS auth{0=1} v4 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25375=0-4194304@1} caps={25375=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25375 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55845e316b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000020e:[<0x1000000020c/SALES.XLS v4>,<0x10000000009/EXCEL v4571>,<0x10000000008/~dmtmp v83>,<0x10000000007/client0 v35>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v26>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v40>,<0x10000000001/sv_1 v61>,<0x10000000000/_nogroup v47>,<0x1/volumes v44>]//[]","memoryvalue":"(3)0x1000000020e:[<0x1000000020c/SALES.XLS v4>,<0x10000000009/EXCEL v4571>,<0x10000000008/~dmtmp v85>,<0x10000000007/client0 v38>,<0x10000000006/clients v28>,<0x10000000005/tmp v28>,<0x100000001fb/client.0 v28>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v43>,<0x10000000001/sv_1 v63>,<0x10000000000/_nogroup v37>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T03:14:12.255 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:14:11 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-h[170323]: 2023-09-13T03:14:11.986+0000 7f5cc35fa700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000559 [3,head] /volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/RESULTS2.PRN auth{0=1} v624 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25375=0-4194304@2} caps={25375=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25375 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55845e317080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000559:[<0x1000000020c/RESULTS2.PRN v624>,<0x10000000009/EXCEL v4571>,<0x10000000008/~dmtmp v83>,<0x10000000007/client0 v35>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v26>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v40>,<0x10000000001/sv_1 v61>,<0x10000000000/_nogroup v47>,<0x1/volumes v44>]//[]","memoryvalue":"(3)0x10000000559:[<0x1000000020c/RESULTS2.PRN v624>,<0x10000000009/EXCEL v4571>,<0x10000000008/~dmtmp v85>,<0x10000000007/client0 v38>,<0x10000000006/clients v28>,<0x10000000005/tmp v28>,<0x100000001fb/client.0 v28>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v43>,<0x10000000001/sv_1 v63>,<0x10000000000/_nogroup v37>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T03:14:12.256 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:14:11 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-h[170323]: 2023-09-13T03:14:11.986+0000 7f5cc35fa700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000556 [3,head] /volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/RESULTS1.PRN auth v608 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25375=0-4194304@2} caps={25375=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25375 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55845e317600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000556:[<0x1000000020c/RESULTS1.PRN v608>,<0x10000000009/EXCEL v4571>,<0x10000000008/~dmtmp v83>,<0x10000000007/client0 v35>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v26>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v40>,<0x10000000001/sv_1 v61>,<0x10000000000/_nogroup v47>,<0x1/volumes v44>]//[]","memoryvalue":"(3)0x10000000556:[<0x1000000020c/RESULTS1.PRN v608>,<0x10000000009/EXCEL v4571>,<0x10000000008/~dmtmp v85>,<0x10000000007/client0 v38>,<0x10000000006/clients v28>,<0x10000000005/tmp v28>,<0x100000001fb/client.0 v28>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v43>,<0x10000000001/sv_1 v63>,<0x10000000000/_nogroup v37>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T03:14:12.256 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:14:11 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-h[170323]: 2023-09-13T03:14:11.987+0000 7f5cc35fa700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000055c [3,head] /volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/RESULTS.XLS auth{0=1} v638 ap=1 s=88384 n(v0 rc2023-09-13T03:14:04.658168+0000 b88384 1=1+0)/n(v0 rc2023-09-13T03:14:04.646168+0000 b88384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25375=0-4194304@2} caps={25375=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25375 | importingcaps=0 caps=1 dirtyrstat=1 replicated=1 authpin=1 scrubqueue=0 0x55845e317b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000055c:[<0x1000000020c/RESULTS.XLS v638>,<0x10000000009/EXCEL v4571>,<0x10000000008/~dmtmp v83>,<0x10000000007/client0 v35>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v26>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v40>,<0x10000000001/sv_1 v61>,<0x10000000000/_nogroup v47>,<0x1/volumes v44>]//[]","memoryvalue":"(3)0x1000000055c:[<0x1000000020c/RESULTS.XLS v638>,<0x10000000009/EXCEL v4571>,<0x10000000008/~dmtmp v85>,<0x10000000007/client0 v38>,<0x10000000006/clients v28>,<0x10000000005/tmp v28>,<0x100000001fb/client.0 v28>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v43>,<0x10000000001/sv_1 v63>,<0x10000000000/_nogroup v37>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T03:14:12.256 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:14:11 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-h[170323]: 2023-09-13T03:14:11.987+0000 7f5cc35fa700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000055b [3,head] /volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/RESULTS3.PRN auth{0=1} v630 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25375=0-4194304@2} caps={25375=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25375 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55845e318100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000055b:[<0x1000000020c/RESULTS3.PRN v630>,<0x10000000009/EXCEL v4571>,<0x10000000008/~dmtmp v83>,<0x10000000007/client0 v35>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v26>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v40>,<0x10000000001/sv_1 v61>,<0x10000000000/_nogroup v47>,<0x1/volumes v44>]//[]","memoryvalue":"(3)0x1000000055b:[<0x1000000020c/RESULTS3.PRN v630>,<0x10000000009/EXCEL v4571>,<0x10000000008/~dmtmp v85>,<0x10000000007/client0 v38>,<0x10000000006/clients v28>,<0x10000000005/tmp v28>,<0x100000001fb/client.0 v28>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v43>,<0x10000000001/sv_1 v63>,<0x10000000000/_nogroup v37>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T03:14:12.256 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:14:11 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-h[170323]: 2023-09-13T03:14:11.987+0000 7f5cc35fa700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000561 [3,head] /volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES1.PRN auth{0=1} v664 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25375=0-4194304@2} caps={25375=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25375 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55845e318680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000561:[<0x1000000020c/SALES1.PRN v664>,<0x10000000009/EXCEL v4571>,<0x10000000008/~dmtmp v83>,<0x10000000007/client0 v35>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v26>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v40>,<0x10000000001/sv_1 v61>,<0x10000000000/_nogroup v47>,<0x1/volumes v44>]//[]","memoryvalue":"(3)0x10000000561:[<0x1000000020c/SALES1.PRN v664>,<0x10000000009/EXCEL v4571>,<0x10000000008/~dmtmp v85>,<0x10000000007/client0 v38>,<0x10000000006/clients v28>,<0x10000000005/tmp v28>,<0x100000001fb/client.0 v28>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v43>,<0x10000000001/sv_1 v63>,<0x10000000000/_nogroup v37>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T03:14:12.256 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:14:11 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-h[170323]: 2023-09-13T03:14:11.987+0000 7f5cc35fa700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000056f [3,head] /volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1B.PRN auth v614 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25375=0-4194304@2} caps={25375=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25375 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55845e2a4b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000056f:[<0x10000000221/NEWS1_1B.PRN v614>,<0x10000000009/WORDPRO v4649>,<0x10000000008/~dmtmp v83>,<0x10000000007/client0 v35>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v26>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v40>,<0x10000000001/sv_1 v61>,<0x10000000000/_nogroup v47>,<0x1/volumes v44>]//[]","memoryvalue":"(3)0x1000000056f:[<0x10000000221/NEWS1_1B.PRN v614>,<0x10000000009/WORDPRO v4649>,<0x10000000008/~dmtmp v85>,<0x10000000007/client0 v38>,<0x10000000006/clients v28>,<0x10000000005/tmp v28>,<0x100000001fb/client.0 v28>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v43>,<0x10000000001/sv_1 v63>,<0x10000000000/_nogroup v37>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T03:14:12.257 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:14:11 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-h[170323]: 2023-09-13T03:14:11.988+0000 7f5cc35fa700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000568 [3,head] /volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS1.LWP auth v578 ap=1 s=12543 n(v0 rc2023-09-13T03:14:04.778166+0000 b12543 1=1+0)/n(v0 rc2023-09-13T03:14:04.768166+0000 b12543 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25375=0-4194304@2} caps={25375=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25375 | importingcaps=0 caps=1 dirtyrstat=1 replicated=0 authpin=1 scrubqueue=0 0x55845e319180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000568:[<0x10000000221/BENCHS1.LWP v578>,<0x10000000009/WORDPRO v4649>,<0x10000000008/~dmtmp v83>,<0x10000000007/client0 v35>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v26>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v40>,<0x10000000001/sv_1 v61>,<0x10000000000/_nogroup v47>,<0x1/volumes v44>]//[]","memoryvalue":"(3)0x10000000568:[<0x10000000221/BENCHS1.LWP v578>,<0x10000000009/WORDPRO v4649>,<0x10000000008/~dmtmp v85>,<0x10000000007/client0 v38>,<0x10000000006/clients v28>,<0x10000000005/tmp v28>,<0x100000001fb/client.0 v28>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v43>,<0x10000000001/sv_1 v63>,<0x10000000000/_nogroup v37>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T03:14:12.257 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:14:11 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-h[170323]: 2023-09-13T03:14:11.988+0000 7f5cc35fa700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000222 [2,head] /volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS.LWP auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25375=0-4194304@1} caps={25375=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25375 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55845e344000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000222:[<0x10000000221/BENCHS.LWP v2>,<0x10000000009/WORDPRO v4649>,<0x10000000008/~dmtmp v83>,<0x10000000007/client0 v35>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v26>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v40>,<0x10000000001/sv_1 v61>,<0x10000000000/_nogroup v47>,<0x1/volumes v44>]//[]","memoryvalue":"(3)0x10000000222:[<0x10000000221/BENCHS.LWP v2>,<0x10000000009/WORDPRO v4649>,<0x10000000008/~dmtmp v85>,<0x10000000007/client0 v38>,<0x10000000006/clients v28>,<0x10000000005/tmp v28>,<0x100000001fb/client.0 v28>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v43>,<0x10000000001/sv_1 v63>,<0x10000000000/_nogroup v37>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T03:14:12.257 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:14:11 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-h[170323]: 2023-09-13T03:14:11.988+0000 7f5cc35fa700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000565 [3,head] /volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS1A.PRN auth v564 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25375=0-4194304@2} caps={25375=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25375 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55845e319700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000565:[<0x10000000221/BENCHS1A.PRN v564>,<0x10000000009/WORDPRO v4649>,<0x10000000008/~dmtmp v83>,<0x10000000007/client0 v35>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v26>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v40>,<0x10000000001/sv_1 v61>,<0x10000000000/_nogroup v47>,<0x1/volumes v44>]//[]","memoryvalue":"(3)0x10000000565:[<0x10000000221/BENCHS1A.PRN v564>,<0x10000000009/WORDPRO v4649>,<0x10000000008/~dmtmp v85>,<0x10000000007/client0 v38>,<0x10000000006/clients v28>,<0x10000000005/tmp v28>,<0x100000001fb/client.0 v28>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v43>,<0x10000000001/sv_1 v63>,<0x10000000000/_nogroup v37>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T03:14:12.257 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:14:11 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-h[170323]: 2023-09-13T03:14:11.988+0000 7f5cc35fa700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000223 [2,head] /volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/RESULTS.XLS auth v4 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25375=0-4194304@1} caps={25375=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25375 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55845e344580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000223:[<0x10000000221/RESULTS.XLS v4>,<0x10000000009/WORDPRO v4649>,<0x10000000008/~dmtmp v83>,<0x10000000007/client0 v35>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v26>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v40>,<0x10000000001/sv_1 v61>,<0x10000000000/_nogroup v47>,<0x1/volumes v44>]//[]","memoryvalue":"(3)0x10000000223:[<0x10000000221/RESULTS.XLS v4>,<0x10000000009/WORDPRO v4649>,<0x10000000008/~dmtmp v85>,<0x10000000007/client0 v38>,<0x10000000006/clients v28>,<0x10000000005/tmp v28>,<0x100000001fb/client.0 v28>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v43>,<0x10000000001/sv_1 v63>,<0x10000000000/_nogroup v37>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T03:14:12.257 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:14:11 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-h[170323]: 2023-09-13T03:14:11.988+0000 7f5cc35fa700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000567 [3,head] /volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS1.PRN auth v570 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25375=0-4194304@2} caps={25375=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25375 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55845e344b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000567:[<0x10000000221/BENCHS1.PRN v570>,<0x10000000009/WORDPRO v4649>,<0x10000000008/~dmtmp v83>,<0x10000000007/client0 v35>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v26>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v40>,<0x10000000001/sv_1 v61>,<0x10000000000/_nogroup v47>,<0x1/volumes v44>]//[]","memoryvalue":"(3)0x10000000567:[<0x10000000221/BENCHS1.PRN v570>,<0x10000000009/WORDPRO v4649>,<0x10000000008/~dmtmp v85>,<0x10000000007/client0 v38>,<0x10000000006/clients v28>,<0x10000000005/tmp v28>,<0x100000001fb/client.0 v28>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v43>,<0x10000000001/sv_1 v63>,<0x10000000000/_nogroup v37>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T03:14:12.258 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:14:11 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-h[170323]: 2023-09-13T03:14:11.988+0000 7f5cc35fa700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000570 [3,head] /volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1A.LWP auth v622 ap=1 s=18630 n(v0 rc2023-09-13T03:14:04.855164+0000 b18630 1=1+0)/n(v0 rc2023-09-13T03:14:04.844165+0000 b18630 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25375=0-4194304@2} caps={25375=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25375 | importingcaps=0 caps=1 dirtyrstat=1 replicated=0 authpin=1 scrubqueue=0 0x55845e345080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000570:[<0x10000000221/NEWS1_1A.LWP v622>,<0x10000000009/WORDPRO v4649>,<0x10000000008/~dmtmp v83>,<0x10000000007/client0 v35>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v26>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v40>,<0x10000000001/sv_1 v61>,<0x10000000000/_nogroup v47>,<0x1/volumes v44>]//[]","memoryvalue":"(3)0x10000000570:[<0x10000000221/NEWS1_1A.LWP v622>,<0x10000000009/WORDPRO v4649>,<0x10000000008/~dmtmp v85>,<0x10000000007/client0 v38>,<0x10000000006/clients v28>,<0x10000000005/tmp v28>,<0x100000001fb/client.0 v28>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v43>,<0x10000000001/sv_1 v63>,<0x10000000000/_nogroup v37>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T03:14:12.258 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:14:11 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-h[170323]: 2023-09-13T03:14:11.989+0000 7f5cc35fa700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000056d [3,head] /volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1A.PRN auth v608 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25375=0-4194304@2} caps={25375=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25375 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55845e345600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000056d:[<0x10000000221/NEWS1_1A.PRN v608>,<0x10000000009/WORDPRO v4649>,<0x10000000008/~dmtmp v83>,<0x10000000007/client0 v35>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v26>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v40>,<0x10000000001/sv_1 v61>,<0x10000000000/_nogroup v47>,<0x1/volumes v44>]//[]","memoryvalue":"(3)0x1000000056d:[<0x10000000221/NEWS1_1A.PRN v608>,<0x10000000009/WORDPRO v4649>,<0x10000000008/~dmtmp v85>,<0x10000000007/client0 v38>,<0x10000000006/clients v28>,<0x10000000005/tmp v28>,<0x100000001fb/client.0 v28>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v43>,<0x10000000001/sv_1 v63>,<0x10000000000/_nogroup v37>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T03:14:12.258 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:14:11 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-h[170323]: 2023-09-13T03:14:11.989+0000 7f5cc35fa700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000224 [2,head] /volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1.LWP auth v6 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25375=0-4194304@1} caps={25375=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=25375 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55845e345b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000224:[<0x10000000221/NEWS1_1.LWP v6>,<0x10000000009/WORDPRO v4649>,<0x10000000008/~dmtmp v83>,<0x10000000007/client0 v35>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v26>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v40>,<0x10000000001/sv_1 v61>,<0x10000000000/_nogroup v47>,<0x1/volumes v44>]//[]","memoryvalue":"(3)0x10000000224:[<0x10000000221/NEWS1_1.LWP v6>,<0x10000000009/WORDPRO v4649>,<0x10000000008/~dmtmp v85>,<0x10000000007/client0 v38>,<0x10000000006/clients v28>,<0x10000000005/tmp v28>,<0x100000001fb/client.0 v28>,<0x100000001fa/54b26f1a-07a6-4d22-a4b6-2c425efa64ed v43>,<0x10000000001/sv_1 v63>,<0x10000000000/_nogroup v37>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-13T03:14:12.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.245+0000 7fdca547eb80 1 -- 172.21.15.64:0/1477256850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55993c9bc710 msgr2=0x55993c9bee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:12.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.245+0000 7fdca547eb80 1 --2- 172.21.15.64:0/1477256850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55993c9bc710 0x55993c9bee40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c005d10 tx=0x7fdc7c009310 comp rx=0 tx=0).stop 2023-09-13T03:14:12.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.245+0000 7fdca547eb80 1 -- 172.21.15.64:0/1477256850 shutdown_connections 2023-09-13T03:14:12.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.245+0000 7fdca547eb80 1 --2- 172.21.15.64:0/1477256850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55993c9bc710 0x55993c9bee40 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:12.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.245+0000 7fdca547eb80 1 --2- 172.21.15.64:0/1477256850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55993c9c6e40 0x55993c9c0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:12.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.245+0000 7fdca547eb80 1 --2- 172.21.15.64:0/1477256850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993c9bd030 0x55993c9bf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:12.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.245+0000 7fdca547eb80 1 -- 172.21.15.64:0/1477256850 >> 172.21.15.64:0/1477256850 conn(0x55993c931090 msgr2=0x55993c9d2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:12.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.245+0000 7fdca547eb80 1 -- 172.21.15.64:0/1477256850 shutdown_connections 2023-09-13T03:14:12.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.245+0000 7fdca547eb80 1 -- 172.21.15.64:0/1477256850 wait complete. 2023-09-13T03:14:12.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.246+0000 7fdca547eb80 1 -- start start 2023-09-13T03:14:12.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.246+0000 7fdca547eb80 1 Processor -- start 2023-09-13T03:14:12.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.246+0000 7fdca547eb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993c9bc710 0x55993c9cc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:12.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.246+0000 7fdca547eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55993c9bd030 0x55993c9cc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:12.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.246+0000 7fdca547eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55993c9c6e40 0x55993c9ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:12.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.246+0000 7fdca547eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55993ca2a540 con 0x55993c9c6e40 2023-09-13T03:14:12.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.246+0000 7fdca547eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55993ca2a680 con 0x55993c9bd030 2023-09-13T03:14:12.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.246+0000 7fdca547eb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55993ca2a840 con 0x55993c9bc710 2023-09-13T03:14:12.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.247+0000 7fdca547eb80 10 client.0 _ll_register_callbacks cb 0x7fdc84095500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:14:12.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.247+0000 7fdc8affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55993c9bd030 0x55993c9cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:12.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.247+0000 7fdc8bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55993c9c6e40 0x55993c9ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:12.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.247+0000 7fdc8affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55993c9bd030 0x55993c9cc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:53932/0 (socket says 172.21.15.64:53932) 2023-09-13T03:14:12.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.247+0000 7fdc8affd700 1 -- 172.21.15.64:0/3425302743 learned_addr learned my addr 172.21.15.64:0/3425302743 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:12.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.247+0000 7fdc8b7fe700 1 --2- 172.21.15.64:0/3425302743 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993c9bc710 0x55993c9cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:12.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.247+0000 7fdc8affd700 1 -- 172.21.15.64:0/3425302743 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993c9bc710 msgr2=0x55993c9cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:12.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.247+0000 7fdc8affd700 1 --2- 172.21.15.64:0/3425302743 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993c9bc710 0x55993c9cc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:12.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.247+0000 7fdc8affd700 1 -- 172.21.15.64:0/3425302743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55993c9c6e40 msgr2=0x55993c9ccb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:12.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.247+0000 7fdc8affd700 1 --2- 172.21.15.64:0/3425302743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55993c9c6e40 0x55993c9ccb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:12.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.247+0000 7fdc8affd700 1 -- 172.21.15.64:0/3425302743 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc7c012040 con 0x55993c9bd030 2023-09-13T03:14:12.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.247+0000 7fdc8bfff700 1 --2- 172.21.15.64:0/3425302743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55993c9c6e40 0x55993c9ccb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:12.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.248+0000 7fdc8affd700 1 --2- 172.21.15.64:0/3425302743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55993c9bd030 0x55993c9cc640 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdc80016000 tx=0x7fdc80009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fe0667c2b4fc840 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:12.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.248+0000 7fdc6affd700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:14:12.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.248+0000 7fdc6affd700 1 -- 172.21.15.64:0/3425302743 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc80005040 con 0x55993c9bd030 2023-09-13T03:14:12.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.248+0000 7fdca547eb80 1 -- 172.21.15.64:0/3425302743 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc840990f0 con 0x55993c9bd030 2023-09-13T03:14:12.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.248+0000 7fdc6affd700 1 -- 172.21.15.64:0/3425302743 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fdc800051a0 con 0x55993c9bd030 2023-09-13T03:14:12.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.248+0000 7fdc6affd700 1 -- 172.21.15.64:0/3425302743 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc80005360 con 0x55993c9bd030 2023-09-13T03:14:12.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.248+0000 7fdca547eb80 10 client.26087 fetch_fsmap learned FSMap version 28 2023-09-13T03:14:12.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.248+0000 7fdca547eb80 1 -- 172.21.15.64:0/3425302743 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55993c7392b0 con 0x55993c9bd030 2023-09-13T03:14:12.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.248+0000 7fdc6affd700 1 -- 172.21.15.64:0/3425302743 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc840990f0 con 0x55993c9bd030 2023-09-13T03:14:12.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.249+0000 7fdc6affd700 1 -- 172.21.15.64:0/3425302743 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdc80030430 con 0x55993c9bd030 2023-09-13T03:14:12.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.249+0000 7fdca547eb80 10 client.26087 fetch_fsmap finished waiting for FSMap version 28 2023-09-13T03:14:12.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.249+0000 7fdca547eb80 10 client.26087 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T03:14:12.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.249+0000 7fdca547eb80 20 client.26087 populate_metadata read hostname 'smithi064' 2023-09-13T03:14:12.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.249+0000 7fdca547eb80 1 --2- 172.21.15.64:0/3425302743 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55993ca2bf90 0x55993ca2e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:12.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.249+0000 7fdca547eb80 4 client.26087 mds_command: new command op to 24686 tid=0[{"prefix": "scrub status"}] 2023-09-13T03:14:12.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.249+0000 7fdca547eb80 1 -- 172.21.15.64:0/3425302743 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55993c907920 con 0x55993ca2bf90 2023-09-13T03:14:12.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.249+0000 7fdc8b7fe700 1 --2- 172.21.15.64:0/3425302743 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55993ca2bf90 0x55993ca2e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:12.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.249+0000 7fdc6affd700 10 client.26087 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:12.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.249+0000 7fdc8b7fe700 1 --2- 172.21.15.64:0/3425302743 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55993ca2bf90 0x55993ca2e3b0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=780368fbefb71031 server_cookie=a8787d25dae11756 in_seq=0 out_seq=0 2023-09-13T03:14:12.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.250+0000 7fdc6affd700 1 -- 172.21.15.64:0/3425302743 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55993c907920 con 0x55993ca2bf90 2023-09-13T03:14:12.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.250+0000 7fdc6affd700 10 client.26087 handle_command_reply: tid=0 2023-09-13T03:14:12.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.250+0000 7fdca547eb80 1 client.26087 shutdown 2023-09-13T03:14:12.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.250+0000 7fdca547eb80 1 -- 172.21.15.64:0/3425302743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55993c9bd030 msgr2=0x55993c9cc640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:12.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.250+0000 7fdca547eb80 1 --2- 172.21.15.64:0/3425302743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55993c9bd030 0x55993c9cc640 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdc80016000 tx=0x7fdc80009c30 comp rx=0 tx=0).stop 2023-09-13T03:14:12.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.250+0000 7fdca547eb80 1 -- 172.21.15.64:0/3425302743 shutdown_connections 2023-09-13T03:14:12.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.250+0000 7fdca547eb80 1 --2- 172.21.15.64:0/3425302743 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55993ca2bf90 0x55993ca2e3b0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:12.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.250+0000 7fdca547eb80 1 --2- 172.21.15.64:0/3425302743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55993c9c6e40 0x55993c9ccb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:12.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.250+0000 7fdc6affd700 0 client.26087 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:12.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.250+0000 7fdca547eb80 1 --2- 172.21.15.64:0/3425302743 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993c9bc710 0x55993c9cc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:12.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.250+0000 7fdca547eb80 1 --2- 172.21.15.64:0/3425302743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55993c9bd030 0x55993c9cc640 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:12.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.250+0000 7fdca547eb80 1 -- 172.21.15.64:0/3425302743 >> 172.21.15.64:0/3425302743 conn(0x55993c931090 msgr2=0x55993c9bee40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:12.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.250+0000 7fdca547eb80 1 -- 172.21.15.64:0/3425302743 shutdown_connections 2023-09-13T03:14:12.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.250+0000 7fdca547eb80 1 -- 172.21.15.64:0/3425302743 wait complete. 2023-09-13T03:14:12.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.250+0000 7fdca547eb80 20 client.26087 trim_cache size 0 max 16384 2023-09-13T03:14:12.274 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-13T03:14:12.274 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-13T03:14:12.274 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2023-09-13T03:14:12.275 INFO:teuthology.orchestra.run.smithi064.stdout: "b25c3e6c-9b8f-45ea-bd90-20b180279b6b": { 2023-09-13T03:14:12.275 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2023-09-13T03:14:12.275 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "b25c3e6c-9b8f-45ea-bd90-20b180279b6b", 2023-09-13T03:14:12.275 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2023-09-13T03:14:12.275 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:14:12.276 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:14:12.276 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-13T03:14:12.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.251+0000 7fdc91bff700 1 -- 172.21.15.64:0/1087783240 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fdc740a0950 msgr2=0x7fdc740a2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:12.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.251+0000 7fdc91bff700 1 --2- 172.21.15.64:0/1087783240 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fdc740a0950 0x7fdc740a2e10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdc8000c010 tx=0x7fdc8000f4c0 comp rx=0 tx=0).stop 2023-09-13T03:14:12.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.251+0000 7fdc91bff700 1 -- 172.21.15.64:0/1087783240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc8c106650 msgr2=0x7fdc8c198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:12.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.251+0000 7fdc91bff700 1 --2- 172.21.15.64:0/1087783240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc8c106650 0x7fdc8c198740 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fdc8400efb0 tx=0x7fdc8400c3a0 comp rx=0 tx=0).stop 2023-09-13T03:14:12.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.251+0000 7fdc91bff700 1 -- 172.21.15.64:0/1087783240 shutdown_connections 2023-09-13T03:14:12.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.251+0000 7fdc91bff700 1 --2- 172.21.15.64:0/1087783240 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fdc740a0950 0x7fdc740a2e10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:12.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.251+0000 7fdc91bff700 1 --2- 172.21.15.64:0/1087783240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc8c106650 0x7fdc8c198740 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:12.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.251+0000 7fdc91bff700 1 --2- 172.21.15.64:0/1087783240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc8c100590 0x7fdc8c19e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:12.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.251+0000 7fdc91bff700 1 --2- 172.21.15.64:0/1087783240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc8c100f50 0x7fdc8c19ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:12.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.251+0000 7fdc91bff700 1 -- 172.21.15.64:0/1087783240 >> 172.21.15.64:0/1087783240 conn(0x7fdc8c0fc060 msgr2=0x7fdc8c0fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:12.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.251+0000 7fdc91bff700 1 -- 172.21.15.64:0/1087783240 shutdown_connections 2023-09-13T03:14:12.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:12.251+0000 7fdc91bff700 1 -- 172.21.15.64:0/1087783240 wait complete. 2023-09-13T03:14:12.280 INFO:tasks.cephfs.filesystem:scrub status for tag:b25c3e6c-9b8f-45ea-bd90-20b180279b6b - {'path': '/', 'tag': 'b25c3e6c-9b8f-45ea-bd90-20b180279b6b', 'options': 'recursive,force'} 2023-09-13T03:14:12.885 INFO:tasks.workunit.client.0.smithi064.stdout: 1 25597 9.12 MB/sec warmup 17 sec latency 1630.815 ms 2023-09-13T03:14:12.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:12 smithi170 ceph-mon[134328]: pgmap v34: 97 pgs: 97 active+clean; 11 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 1.4 MiB/s wr, 285 op/s 2023-09-13T03:14:12.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:12 smithi170 ceph-mon[134328]: scrub queued for path: / 2023-09-13T03:14:12.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:12 smithi170 ceph-mon[134328]: scrub summary: idle+waiting paths [/] 2023-09-13T03:14:12.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:12 smithi170 ceph-mon[134328]: scrub summary: active paths [/] 2023-09-13T03:14:12.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:12 smithi170 ceph-mon[134328]: scrub summary: active paths [/] 2023-09-13T03:14:12.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:12 smithi170 ceph-mon[134328]: Scrub error on inode 0x1000000021f (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:12 smithi170 ceph-mon[134328]: Scrub error on inode 0x1000000021b (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:12 smithi170 ceph-mon[134328]: Scrub error on inode 0x10000000220 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:12 smithi170 ceph-mon[134328]: scrub summary: active paths [/] 2023-09-13T03:14:12.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:12 smithi170 ceph-mon[134328]: Scrub error on inode 0x10000000580 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPH2.BAK) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:12 smithi170 ceph-mon[134328]: Scrub error on inode 0x1000000021a (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.913 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:12 smithi170 ceph-mon[134328]: Scrub error on inode 0x1000000021e (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.913 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:12 smithi170 ceph-mon[134328]: Scrub error on inode 0x10000000218 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.913 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:12 smithi170 ceph-mon[134328]: Scrub error on inode 0x1000000057d (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPH1.CDR) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.913 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:12 smithi170 ceph-mon[134328]: Scrub error on inode 0x1000000021c (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.913 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:12 smithi170 ceph-mon[134328]: scrub summary: idle+waiting paths [/] 2023-09-13T03:14:12.914 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:12 smithi170 ceph-mon[134328]: Scrub error on inode 0x10000000217 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.914 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:12 smithi170 ceph-mon[134328]: Scrub error on inode 0x1000000057c (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPH1.BAK) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.914 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:12 smithi170 ceph-mon[134328]: Scrub error on inode 0x10000000581 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPH2.CDR) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.914 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:12 smithi170 ceph-mon[134328]: Scrub error on inode 0x1000000020d (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.914 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:12 smithi170 ceph-mon[134328]: Scrub error on inode 0x1000000021d (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.915 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:12 smithi170 ceph-mon[134328]: Scrub error on inode 0x1000000055f (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES1.XLS) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.915 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:12 smithi170 ceph-mon[134328]: Scrub error on inode 0x10000000219 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.915 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:12 smithi170 ceph-mon[134328]: Scrub error on inode 0x10000000216 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.915 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:12 smithi170 ceph-mon[134328]: Scrub error on inode 0x1000000020e (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.915 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:12 smithi170 ceph-mon[134328]: Scrub error on inode 0x10000000559 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/RESULTS2.PRN) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.916 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:12 smithi170 ceph-mon[134328]: Scrub error on inode 0x10000000556 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/RESULTS1.PRN) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.916 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:12 smithi170 ceph-mon[134328]: Scrub error on inode 0x1000000055c (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/RESULTS.XLS) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.916 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:12 smithi170 ceph-mon[134328]: Scrub error on inode 0x1000000055b (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/RESULTS3.PRN) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.916 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:12 smithi170 ceph-mon[134328]: Scrub error on inode 0x10000000561 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES1.PRN) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.916 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:12 smithi170 ceph-mon[134328]: Scrub error on inode 0x1000000056f (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1B.PRN) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.917 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:12 smithi170 ceph-mon[134328]: Scrub error on inode 0x10000000568 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS1.LWP) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.917 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:12 smithi170 ceph-mon[134328]: scrub summary: active paths [/] 2023-09-13T03:14:12.917 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:12 smithi170 ceph-mon[134328]: scrub summary: idle+waiting paths [/] 2023-09-13T03:14:12.917 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:12 smithi170 ceph-mon[134328]: scrub summary: idle+waiting paths [/] 2023-09-13T03:14:12.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:12 smithi064 ceph-mon[97675]: pgmap v34: 97 pgs: 97 active+clean; 11 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 1.4 MiB/s wr, 285 op/s 2023-09-13T03:14:12.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:12 smithi064 ceph-mon[97675]: scrub queued for path: / 2023-09-13T03:14:12.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:12 smithi064 ceph-mon[97675]: scrub summary: idle+waiting paths [/] 2023-09-13T03:14:12.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:12 smithi064 ceph-mon[97675]: scrub summary: active paths [/] 2023-09-13T03:14:12.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:12 smithi064 ceph-mon[97675]: scrub summary: active paths [/] 2023-09-13T03:14:12.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:12 smithi064 ceph-mon[97675]: Scrub error on inode 0x1000000021f (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:12 smithi064 ceph-mon[97675]: Scrub error on inode 0x1000000021b (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:12 smithi064 ceph-mon[97675]: Scrub error on inode 0x10000000220 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:12 smithi064 ceph-mon[97675]: scrub summary: active paths [/] 2023-09-13T03:14:12.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:12 smithi064 ceph-mon[97675]: Scrub error on inode 0x10000000580 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPH2.BAK) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:12 smithi064 ceph-mon[97675]: Scrub error on inode 0x1000000021a (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:12 smithi064 ceph-mon[97675]: Scrub error on inode 0x1000000021e (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:12 smithi064 ceph-mon[97675]: Scrub error on inode 0x10000000218 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:12 smithi064 ceph-mon[97675]: Scrub error on inode 0x1000000057d (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPH1.CDR) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:12 smithi064 ceph-mon[97675]: Scrub error on inode 0x1000000021c (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.922 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:12 smithi064 ceph-mon[97675]: scrub summary: idle+waiting paths [/] 2023-09-13T03:14:12.922 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:12 smithi064 ceph-mon[97675]: Scrub error on inode 0x10000000217 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.922 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:12 smithi064 ceph-mon[97675]: Scrub error on inode 0x1000000057c (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPH1.BAK) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.922 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:12 smithi064 ceph-mon[97675]: Scrub error on inode 0x10000000581 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPH2.CDR) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.922 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:12 smithi064 ceph-mon[97675]: Scrub error on inode 0x1000000020d (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.923 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:12 smithi064 ceph-mon[97675]: Scrub error on inode 0x1000000021d (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.923 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:12 smithi064 ceph-mon[97675]: Scrub error on inode 0x1000000055f (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES1.XLS) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.923 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:12 smithi064 ceph-mon[97675]: Scrub error on inode 0x10000000219 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.923 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:12 smithi064 ceph-mon[97675]: Scrub error on inode 0x10000000216 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.923 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:12 smithi064 ceph-mon[97675]: Scrub error on inode 0x1000000020e (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.924 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:12 smithi064 ceph-mon[97675]: Scrub error on inode 0x10000000559 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/RESULTS2.PRN) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.924 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:12 smithi064 ceph-mon[97675]: Scrub error on inode 0x10000000556 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/RESULTS1.PRN) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.924 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:12 smithi064 ceph-mon[97675]: Scrub error on inode 0x1000000055c (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/RESULTS.XLS) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.924 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:12 smithi064 ceph-mon[97675]: Scrub error on inode 0x1000000055b (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/RESULTS3.PRN) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.924 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:12 smithi064 ceph-mon[97675]: Scrub error on inode 0x10000000561 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES1.PRN) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.925 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:12 smithi064 ceph-mon[97675]: Scrub error on inode 0x1000000056f (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1B.PRN) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.925 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:12 smithi064 ceph-mon[97675]: Scrub error on inode 0x10000000568 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS1.LWP) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.925 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:12 smithi064 ceph-mon[97675]: scrub summary: active paths [/] 2023-09-13T03:14:12.925 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:12 smithi064 ceph-mon[97675]: scrub summary: idle+waiting paths [/] 2023-09-13T03:14:12.925 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:12 smithi064 ceph-mon[97675]: scrub summary: idle+waiting paths [/] 2023-09-13T03:14:12.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:12 smithi142 ceph-mon[134082]: pgmap v34: 97 pgs: 97 active+clean; 11 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 1.4 MiB/s wr, 285 op/s 2023-09-13T03:14:12.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:12 smithi142 ceph-mon[134082]: scrub queued for path: / 2023-09-13T03:14:12.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:12 smithi142 ceph-mon[134082]: scrub summary: idle+waiting paths [/] 2023-09-13T03:14:12.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:12 smithi142 ceph-mon[134082]: scrub summary: active paths [/] 2023-09-13T03:14:12.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:12 smithi142 ceph-mon[134082]: scrub summary: active paths [/] 2023-09-13T03:14:12.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:12 smithi142 ceph-mon[134082]: Scrub error on inode 0x1000000021f (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:12 smithi142 ceph-mon[134082]: Scrub error on inode 0x1000000021b (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:12 smithi142 ceph-mon[134082]: Scrub error on inode 0x10000000220 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:12 smithi142 ceph-mon[134082]: scrub summary: active paths [/] 2023-09-13T03:14:12.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:12 smithi142 ceph-mon[134082]: Scrub error on inode 0x10000000580 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPH2.BAK) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.998 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:12 smithi142 ceph-mon[134082]: Scrub error on inode 0x1000000021a (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.998 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:12 smithi142 ceph-mon[134082]: Scrub error on inode 0x1000000021e (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.998 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:12 smithi142 ceph-mon[134082]: Scrub error on inode 0x10000000218 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.998 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:12 smithi142 ceph-mon[134082]: Scrub error on inode 0x1000000057d (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPH1.CDR) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.998 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:12 smithi142 ceph-mon[134082]: Scrub error on inode 0x1000000021c (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.999 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:12 smithi142 ceph-mon[134082]: scrub summary: idle+waiting paths [/] 2023-09-13T03:14:12.999 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:12 smithi142 ceph-mon[134082]: Scrub error on inode 0x10000000217 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.999 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:12 smithi142 ceph-mon[134082]: Scrub error on inode 0x1000000057c (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPH1.BAK) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.999 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:12 smithi142 ceph-mon[134082]: Scrub error on inode 0x10000000581 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPH2.CDR) see mds.h log and `damage ls` output for details 2023-09-13T03:14:12.999 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:12 smithi142 ceph-mon[134082]: Scrub error on inode 0x1000000020d (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS) see mds.h log and `damage ls` output for details 2023-09-13T03:14:13.000 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:12 smithi142 ceph-mon[134082]: Scrub error on inode 0x1000000021d (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR) see mds.h log and `damage ls` output for details 2023-09-13T03:14:13.000 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:12 smithi142 ceph-mon[134082]: Scrub error on inode 0x1000000055f (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES1.XLS) see mds.h log and `damage ls` output for details 2023-09-13T03:14:13.000 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:12 smithi142 ceph-mon[134082]: Scrub error on inode 0x10000000219 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR) see mds.h log and `damage ls` output for details 2023-09-13T03:14:13.000 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:12 smithi142 ceph-mon[134082]: Scrub error on inode 0x10000000216 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG) see mds.h log and `damage ls` output for details 2023-09-13T03:14:13.000 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:12 smithi142 ceph-mon[134082]: Scrub error on inode 0x1000000020e (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS) see mds.h log and `damage ls` output for details 2023-09-13T03:14:13.001 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:12 smithi142 ceph-mon[134082]: Scrub error on inode 0x10000000559 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/RESULTS2.PRN) see mds.h log and `damage ls` output for details 2023-09-13T03:14:13.001 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:12 smithi142 ceph-mon[134082]: Scrub error on inode 0x10000000556 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/RESULTS1.PRN) see mds.h log and `damage ls` output for details 2023-09-13T03:14:13.001 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:12 smithi142 ceph-mon[134082]: Scrub error on inode 0x1000000055c (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/RESULTS.XLS) see mds.h log and `damage ls` output for details 2023-09-13T03:14:13.001 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:12 smithi142 ceph-mon[134082]: Scrub error on inode 0x1000000055b (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/RESULTS3.PRN) see mds.h log and `damage ls` output for details 2023-09-13T03:14:13.002 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:12 smithi142 ceph-mon[134082]: Scrub error on inode 0x10000000561 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES1.PRN) see mds.h log and `damage ls` output for details 2023-09-13T03:14:13.002 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:12 smithi142 ceph-mon[134082]: Scrub error on inode 0x1000000056f (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1B.PRN) see mds.h log and `damage ls` output for details 2023-09-13T03:14:13.002 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:12 smithi142 ceph-mon[134082]: Scrub error on inode 0x10000000568 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS1.LWP) see mds.h log and `damage ls` output for details 2023-09-13T03:14:13.002 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:12 smithi142 ceph-mon[134082]: scrub summary: active paths [/] 2023-09-13T03:14:13.003 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:12 smithi142 ceph-mon[134082]: scrub summary: idle+waiting paths [/] 2023-09-13T03:14:13.003 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:12 smithi142 ceph-mon[134082]: scrub summary: idle+waiting paths [/] 2023-09-13T03:14:13.885 INFO:tasks.workunit.client.0.smithi064.stdout: 1 25597 8.62 MB/sec warmup 18 sec latency 1530.829 ms 2023-09-13T03:14:13.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:13 smithi170 ceph-mon[134328]: Scrub error on inode 0x10000000222 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS.LWP) see mds.h log and `damage ls` output for details 2023-09-13T03:14:13.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:13 smithi170 ceph-mon[134328]: Scrub error on inode 0x10000000565 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS1A.PRN) see mds.h log and `damage ls` output for details 2023-09-13T03:14:13.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:13 smithi170 ceph-mon[134328]: Scrub error on inode 0x10000000223 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/RESULTS.XLS) see mds.h log and `damage ls` output for details 2023-09-13T03:14:13.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:13 smithi170 ceph-mon[134328]: Scrub error on inode 0x10000000567 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS1.PRN) see mds.h log and `damage ls` output for details 2023-09-13T03:14:13.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:13 smithi170 ceph-mon[134328]: Scrub error on inode 0x10000000570 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1A.LWP) see mds.h log and `damage ls` output for details 2023-09-13T03:14:13.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:13 smithi170 ceph-mon[134328]: Scrub error on inode 0x1000000056d (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1A.PRN) see mds.h log and `damage ls` output for details 2023-09-13T03:14:13.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:13 smithi170 ceph-mon[134328]: Scrub error on inode 0x10000000224 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1.LWP) see mds.h log and `damage ls` output for details 2023-09-13T03:14:13.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:13 smithi170 ceph-mon[134328]: scrub summary: idle+waiting paths [/] 2023-09-13T03:14:13.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:13 smithi064 ceph-mon[97675]: Scrub error on inode 0x10000000222 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS.LWP) see mds.h log and `damage ls` output for details 2023-09-13T03:14:13.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:13 smithi064 ceph-mon[97675]: Scrub error on inode 0x10000000565 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS1A.PRN) see mds.h log and `damage ls` output for details 2023-09-13T03:14:13.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:13 smithi064 ceph-mon[97675]: Scrub error on inode 0x10000000223 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/RESULTS.XLS) see mds.h log and `damage ls` output for details 2023-09-13T03:14:13.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:13 smithi064 ceph-mon[97675]: Scrub error on inode 0x10000000567 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS1.PRN) see mds.h log and `damage ls` output for details 2023-09-13T03:14:13.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:13 smithi064 ceph-mon[97675]: Scrub error on inode 0x10000000570 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1A.LWP) see mds.h log and `damage ls` output for details 2023-09-13T03:14:13.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:13 smithi064 ceph-mon[97675]: Scrub error on inode 0x1000000056d (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1A.PRN) see mds.h log and `damage ls` output for details 2023-09-13T03:14:13.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:13 smithi064 ceph-mon[97675]: Scrub error on inode 0x10000000224 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1.LWP) see mds.h log and `damage ls` output for details 2023-09-13T03:14:13.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:13 smithi064 ceph-mon[97675]: scrub summary: idle+waiting paths [/] 2023-09-13T03:14:13.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:13 smithi142 ceph-mon[134082]: Scrub error on inode 0x10000000222 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS.LWP) see mds.h log and `damage ls` output for details 2023-09-13T03:14:13.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:13 smithi142 ceph-mon[134082]: Scrub error on inode 0x10000000565 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS1A.PRN) see mds.h log and `damage ls` output for details 2023-09-13T03:14:13.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:13 smithi142 ceph-mon[134082]: Scrub error on inode 0x10000000223 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/RESULTS.XLS) see mds.h log and `damage ls` output for details 2023-09-13T03:14:13.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:13 smithi142 ceph-mon[134082]: Scrub error on inode 0x10000000567 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS1.PRN) see mds.h log and `damage ls` output for details 2023-09-13T03:14:13.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:13 smithi142 ceph-mon[134082]: Scrub error on inode 0x10000000570 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1A.LWP) see mds.h log and `damage ls` output for details 2023-09-13T03:14:13.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:13 smithi142 ceph-mon[134082]: Scrub error on inode 0x1000000056d (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1A.PRN) see mds.h log and `damage ls` output for details 2023-09-13T03:14:13.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:13 smithi142 ceph-mon[134082]: Scrub error on inode 0x10000000224 (/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1.LWP) see mds.h log and `damage ls` output for details 2023-09-13T03:14:13.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:13 smithi142 ceph-mon[134082]: scrub summary: idle+waiting paths [/] 2023-09-13T03:14:14.885 INFO:tasks.workunit.client.0.smithi064.stdout: 1 25597 8.16 MB/sec warmup 19 sec latency 2530.986 ms 2023-09-13T03:14:14.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:14 smithi170 ceph-mon[134328]: pgmap v35: 97 pgs: 97 active+clean; 11 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 755 KiB/s wr, 176 op/s 2023-09-13T03:14:14.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:14 smithi064 ceph-mon[97675]: pgmap v35: 97 pgs: 97 active+clean; 11 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 755 KiB/s wr, 176 op/s 2023-09-13T03:14:14.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:14 smithi142 ceph-mon[134082]: pgmap v35: 97 pgs: 97 active+clean; 11 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 755 KiB/s wr, 176 op/s 2023-09-13T03:14:15.885 INFO:tasks.workunit.client.0.smithi064.stdout: 1 25788 7.78 MB/sec warmup 20 sec latency 2988.065 ms 2023-09-13T03:14:15.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:15 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:14:15.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:15 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mon-a[97651]: 2023-09-13T03:14:15.596+0000 7f9fd6939700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-09-13T03:14:15.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:15 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:14:15.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:15 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:14:16.886 INFO:tasks.workunit.client.0.smithi064.stdout: 1 25788 7.41 MB/sec warmup 21 sec latency 1419.497 ms 2023-09-13T03:14:16.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:16 smithi170 ceph-mon[134328]: pgmap v36: 97 pgs: 97 active+clean; 11 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 803 KiB/s wr, 197 op/s 2023-09-13T03:14:16.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:16 smithi170 ceph-mon[134328]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-09-13T03:14:16.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:16 smithi170 ceph-mon[134328]: mds.? [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] up:active 2023-09-13T03:14:16.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:16 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:14:16.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:16 smithi064 ceph-mon[97675]: pgmap v36: 97 pgs: 97 active+clean; 11 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 803 KiB/s wr, 197 op/s 2023-09-13T03:14:16.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:16 smithi064 ceph-mon[97675]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-09-13T03:14:16.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:16 smithi064 ceph-mon[97675]: mds.? [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] up:active 2023-09-13T03:14:16.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:16 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:14:16.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:16 smithi142 ceph-mon[134082]: pgmap v36: 97 pgs: 97 active+clean; 11 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 803 KiB/s wr, 197 op/s 2023-09-13T03:14:16.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:16 smithi142 ceph-mon[134082]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-09-13T03:14:16.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:16 smithi142 ceph-mon[134082]: mds.? [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] up:active 2023-09-13T03:14:16.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:16 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:14:17.886 INFO:tasks.workunit.client.0.smithi064.stdout: 1 25788 7.07 MB/sec warmup 22 sec latency 2419.625 ms 2023-09-13T03:14:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.701+0000 7f02d35ff700 1 -- 172.21.15.64:0/2010369108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02cc1005a0 msgr2=0x7f02cc100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.701+0000 7f02d35ff700 1 --2- 172.21.15.64:0/2010369108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02cc1005a0 0x7f02cc100a20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f02bc0099c0 tx=0x7f02bc009cd0 comp rx=0 tx=0).stop 2023-09-13T03:14:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.701+0000 7f02d35ff700 1 -- 172.21.15.64:0/2010369108 shutdown_connections 2023-09-13T03:14:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.701+0000 7f02d35ff700 1 --2- 172.21.15.64:0/2010369108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02cc106660 0x7f02cc106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.701+0000 7f02d35ff700 1 --2- 172.21.15.64:0/2010369108 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02cc100f60 0x7f02cc10d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.701+0000 7f02d35ff700 1 --2- 172.21.15.64:0/2010369108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02cc1005a0 0x7f02cc100a20 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.701+0000 7f02d35ff700 1 -- 172.21.15.64:0/2010369108 >> 172.21.15.64:0/2010369108 conn(0x7f02cc0fc050 msgr2=0x7f02cc0fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.701+0000 7f02d35ff700 1 -- 172.21.15.64:0/2010369108 shutdown_connections 2023-09-13T03:14:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.701+0000 7f02d35ff700 1 -- 172.21.15.64:0/2010369108 wait complete. 2023-09-13T03:14:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.702+0000 7f02d35ff700 1 Processor -- start 2023-09-13T03:14:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.702+0000 7f02d35ff700 1 -- start start 2023-09-13T03:14:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.703+0000 7f02d35ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02cc1005a0 0x7f02cc19e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.703+0000 7f02d35ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02cc100f60 0x7f02cc19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.703+0000 7f02d35ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02cc106660 0x7f02cc198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.703+0000 7f02d35ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc198cc0 con 0x7f02cc106660 2023-09-13T03:14:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.703+0000 7f02d35ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc198e30 con 0x7f02cc1005a0 2023-09-13T03:14:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.703+0000 7f02d35ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc198ff0 con 0x7f02cc100f60 2023-09-13T03:14:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.703+0000 7f02d139b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02cc1005a0 0x7f02cc19e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.703+0000 7f02d139b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02cc1005a0 0x7f02cc19e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:37070/0 (socket says 172.21.15.64:37070) 2023-09-13T03:14:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.703+0000 7f02d139b700 1 -- 172.21.15.64:0/3171176616 learned_addr learned my addr 172.21.15.64:0/3171176616 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.703+0000 7f02d1b9c700 1 --2- 172.21.15.64:0/3171176616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02cc106660 0x7f02cc198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.703+0000 7f02d0b9a700 1 --2- 172.21.15.64:0/3171176616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02cc100f60 0x7f02cc19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.704+0000 7f02d139b700 1 -- 172.21.15.64:0/3171176616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02cc100f60 msgr2=0x7f02cc19ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.704+0000 7f02d139b700 1 --2- 172.21.15.64:0/3171176616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02cc100f60 0x7f02cc19ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.704+0000 7f02d139b700 1 -- 172.21.15.64:0/3171176616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02cc106660 msgr2=0x7f02cc198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.704+0000 7f02d139b700 1 --2- 172.21.15.64:0/3171176616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02cc106660 0x7f02cc198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.704+0000 7f02d139b700 1 -- 172.21.15.64:0/3171176616 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02bc0095f0 con 0x7f02cc1005a0 2023-09-13T03:14:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.704+0000 7f02d1b9c700 1 --2- 172.21.15.64:0/3171176616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02cc106660 0x7f02cc198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.704+0000 7f02d139b700 1 --2- 172.21.15.64:0/3171176616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02cc1005a0 0x7f02cc19e7f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f02c800e9a0 tx=0x7f02c800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.704+0000 7f02c27fc700 1 -- 172.21.15.64:0/3171176616 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c8009790 con 0x7f02cc1005a0 2023-09-13T03:14:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.704+0000 7f02c27fc700 1 -- 172.21.15.64:0/3171176616 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02c8010430 con 0x7f02cc1005a0 2023-09-13T03:14:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.704+0000 7f02d35ff700 1 -- 172.21.15.64:0/3171176616 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02cc1992d0 con 0x7f02cc1005a0 2023-09-13T03:14:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.704+0000 7f02c27fc700 1 -- 172.21.15.64:0/3171176616 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c8018680 con 0x7f02cc1005a0 2023-09-13T03:14:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.705+0000 7f02d35ff700 1 -- 172.21.15.64:0/3171176616 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02cc1a57e0 con 0x7f02cc1005a0 2023-09-13T03:14:18.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.708+0000 7f02c27fc700 1 -- 172.21.15.64:0/3171176616 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 135492+0+0 (secure 0 0 0) 0x7f02c80105a0 con 0x7f02cc1005a0 2023-09-13T03:14:18.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.709+0000 7f02e6fe6b80 1 Processor -- start 2023-09-13T03:14:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.709+0000 7f02e6fe6b80 1 -- start start 2023-09-13T03:14:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.710+0000 7f02e6fe6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cea80c01a0 0x55cea80c0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.710+0000 7f02e6fe6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cea80ba300 0x55cea80ba780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.710+0000 7f02e6fe6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cea80bacc0 0x55cea80c6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.710+0000 7f02e6fe6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea80136d0 con 0x55cea80bacc0 2023-09-13T03:14:18.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.710+0000 7f02e6fe6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea8010ff0 con 0x55cea80c01a0 2023-09-13T03:14:18.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.710+0000 7f02e6fe6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea800c8e0 con 0x55cea80ba300 2023-09-13T03:14:18.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.710+0000 7f02d139b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cea80ba300 0x55cea80ba780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:18.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.710+0000 7f02d0b9a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cea80bacc0 0x55cea80c6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:18.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.710+0000 7f02d1b9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cea80c01a0 0x55cea80c0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:18.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.710+0000 7f02d0b9a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cea80bacc0 0x55cea80c6f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38384/0 (socket says 172.21.15.64:38384) 2023-09-13T03:14:18.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.710+0000 7f02d0b9a700 1 -- 172.21.15.64:0/2931339542 learned_addr learned my addr 172.21.15.64:0/2931339542 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:18.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.710+0000 7f02d139b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cea80ba300 0x55cea80ba780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:35508/0 (socket says 172.21.15.64:35508) 2023-09-13T03:14:18.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.710+0000 7f02c27fc700 1 --2- 172.21.15.64:0/3171176616 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f02b80a9160 0x7f02b80ab620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.711+0000 7f02d0b9a700 1 -- 172.21.15.64:0/2931339542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cea80ba300 msgr2=0x55cea80ba780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.711+0000 7f02d0b9a700 1 --2- 172.21.15.64:0/2931339542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cea80ba300 0x55cea80ba780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.711+0000 7f02d0b9a700 1 -- 172.21.15.64:0/2931339542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cea80c01a0 msgr2=0x55cea80c0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.711+0000 7f02d0b9a700 1 --2- 172.21.15.64:0/2931339542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cea80c01a0 0x55cea80c0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.711+0000 7f02c27fc700 1 -- 172.21.15.64:0/3171176616 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f02c8014070 con 0x7f02cc1005a0 2023-09-13T03:14:18.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.711+0000 7f02d0b9a700 1 -- 172.21.15.64:0/2931339542 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cea7e38d80 con 0x55cea80bacc0 2023-09-13T03:14:18.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.711+0000 7f02d1b9c700 1 --2- 172.21.15.64:0/3171176616 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f02b80a9160 0x7f02b80ab620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:18.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.711+0000 7f02d0b9a700 1 --2- 172.21.15.64:0/2931339542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cea80bacc0 0x55cea80c6f30 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f02bc009fd0 tx=0x7f02bc005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=167b0e53f7e778fe server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:18.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.711+0000 7f02b77fe700 1 -- 172.21.15.64:0/2931339542 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02bc00b990 con 0x55cea80bacc0 2023-09-13T03:14:18.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.711+0000 7f02b77fe700 1 -- 172.21.15.64:0/2931339542 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02bc01e040 con 0x55cea80bacc0 2023-09-13T03:14:18.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.711+0000 7f02b77fe700 1 -- 172.21.15.64:0/2931339542 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02bc00fe20 con 0x55cea80bacc0 2023-09-13T03:14:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.712+0000 7f02e6fe6b80 1 -- 172.21.15.64:0/2931339542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cea80bacc0 msgr2=0x55cea80c6f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.712+0000 7f02d1b9c700 1 --2- 172.21.15.64:0/3171176616 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f02b80a9160 0x7f02b80ab620 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f02cc19f4f0 tx=0x7f02c4004ad0 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.712+0000 7f02e6fe6b80 1 --2- 172.21.15.64:0/2931339542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cea80bacc0 0x55cea80c6f30 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f02bc009fd0 tx=0x7f02bc005f00 comp rx=0 tx=0).stop 2023-09-13T03:14:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.712+0000 7f02e6fe6b80 1 -- 172.21.15.64:0/2931339542 shutdown_connections 2023-09-13T03:14:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.712+0000 7f02e6fe6b80 1 --2- 172.21.15.64:0/2931339542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cea80c01a0 0x55cea80c0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.712+0000 7f02e6fe6b80 1 --2- 172.21.15.64:0/2931339542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cea80bacc0 0x55cea80c6f30 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.712+0000 7f02e6fe6b80 1 --2- 172.21.15.64:0/2931339542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cea80ba300 0x55cea80ba780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.712+0000 7f02e6fe6b80 1 -- 172.21.15.64:0/2931339542 >> 172.21.15.64:0/2931339542 conn(0x55cea802f030 msgr2=0x55cea80b5560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.712+0000 7f02e6fe6b80 1 -- 172.21.15.64:0/2931339542 shutdown_connections 2023-09-13T03:14:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.712+0000 7f02e6fe6b80 1 -- 172.21.15.64:0/2931339542 wait complete. 2023-09-13T03:14:18.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.713+0000 7f02e6fe6b80 1 -- start start 2023-09-13T03:14:18.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.713+0000 7f02e6fe6b80 1 Processor -- start 2023-09-13T03:14:18.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.714+0000 7f02e6fe6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cea80ba300 0x55cea80c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:18.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.714+0000 7f02e6fe6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cea80bacc0 0x55cea80c40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:18.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.714+0000 7f02e6fe6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cea80c01a0 0x55cea80c4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.714+0000 7f02e6fe6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea7faab10 con 0x55cea80bacc0 2023-09-13T03:14:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.714+0000 7f02e6fe6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea7faa810 con 0x55cea80ba300 2023-09-13T03:14:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.714+0000 7f02e6fe6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea7e8e8c0 con 0x55cea80c01a0 2023-09-13T03:14:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.714+0000 7f02e6fe6b80 10 client.0 _ll_register_callbacks cb 0x55cea7ef4a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:14:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.714+0000 7f02d0b9a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cea80bacc0 0x55cea80c40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.714+0000 7f02d139b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cea80ba300 0x55cea80c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.714+0000 7f02d139b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cea80ba300 0x55cea80c6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:37082/0 (socket says 172.21.15.64:37082) 2023-09-13T03:14:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.714+0000 7f02d139b700 1 -- 172.21.15.64:0/1780554217 learned_addr learned my addr 172.21.15.64:0/1780554217 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.714+0000 7f02d1b9c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cea80c01a0 0x55cea80c4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.714+0000 7f02d0b9a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cea80bacc0 0x55cea80c40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38400/0 (socket says 172.21.15.64:38400) 2023-09-13T03:14:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.714+0000 7f02d139b700 1 -- 172.21.15.64:0/1780554217 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cea80c01a0 msgr2=0x55cea80c4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.714+0000 7f02d139b700 1 --2- 172.21.15.64:0/1780554217 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cea80c01a0 0x55cea80c4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.714+0000 7f02d139b700 1 -- 172.21.15.64:0/1780554217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cea80bacc0 msgr2=0x55cea80c40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.714+0000 7f02d139b700 1 --2- 172.21.15.64:0/1780554217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cea80bacc0 0x55cea80c40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:18.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.714+0000 7f02d139b700 1 -- 172.21.15.64:0/1780554217 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02cc10dd20 con 0x55cea80ba300 2023-09-13T03:14:18.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.715+0000 7f02d139b700 1 --2- 172.21.15.64:0/1780554217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cea80ba300 0x55cea80c6dc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f02c80995a0 tx=0x7f02c8099b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7357a073ae713a71 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:18.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.715+0000 7f02b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:14:18.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.715+0000 7f02b4ff9700 1 -- 172.21.15.64:0/1780554217 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c8098f70 con 0x55cea80ba300 2023-09-13T03:14:18.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.715+0000 7f02e6fe6b80 1 -- 172.21.15.64:0/1780554217 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cea7f64490 con 0x55cea80ba300 2023-09-13T03:14:18.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.715+0000 7f02b4ff9700 1 -- 172.21.15.64:0/1780554217 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f02c8099d10 con 0x55cea80ba300 2023-09-13T03:14:18.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.715+0000 7f02b4ff9700 1 -- 172.21.15.64:0/1780554217 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02c8099610 con 0x55cea80ba300 2023-09-13T03:14:18.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.715+0000 7f02e6fe6b80 10 client.26099 fetch_fsmap learned FSMap version 30 2023-09-13T03:14:18.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.715+0000 7f02e6fe6b80 1 -- 172.21.15.64:0/1780554217 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55cea7e21c00 con 0x55cea80ba300 2023-09-13T03:14:18.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.715+0000 7f02b4ff9700 1 -- 172.21.15.64:0/1780554217 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cea7f64490 con 0x55cea80ba300 2023-09-13T03:14:18.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.715+0000 7f02b4ff9700 1 -- 172.21.15.64:0/1780554217 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 30) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f02c80a1280 con 0x55cea80ba300 2023-09-13T03:14:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.715+0000 7f02e6fe6b80 10 client.26099 fetch_fsmap finished waiting for FSMap version 30 2023-09-13T03:14:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.715+0000 7f02e6fe6b80 10 client.26099 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:14:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.716+0000 7f02e6fe6b80 20 client.26099 populate_metadata read hostname 'smithi064' 2023-09-13T03:14:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.716+0000 7f02e6fe6b80 1 --2- 172.21.15.64:0/1780554217 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55cea80d4380 0x55cea80d67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.716+0000 7f02e6fe6b80 4 client.26099 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:14:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.716+0000 7f02e6fe6b80 1 -- 172.21.15.64:0/1780554217 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cea8021840 con 0x55cea80d4380 2023-09-13T03:14:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.716+0000 7f02d0b9a700 1 --2- 172.21.15.64:0/1780554217 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55cea80d4380 0x55cea80d67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.716+0000 7f02d0b9a700 1 --2- 172.21.15.64:0/1780554217 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55cea80d4380 0x55cea80d67a0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b33e4652adff5065 server_cookie=6dd31695c394c09c in_seq=0 out_seq=0 2023-09-13T03:14:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.716+0000 7f02b4ff9700 10 client.26099 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.717+0000 7f02b4ff9700 1 -- 172.21.15.64:0/1780554217 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cea8021840 con 0x55cea80d4380 2023-09-13T03:14:18.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.717+0000 7f02b4ff9700 10 client.26099 handle_command_reply: tid=0 2023-09-13T03:14:18.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.717+0000 7f02e6fe6b80 1 client.26099 shutdown 2023-09-13T03:14:18.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.717+0000 7f02e6fe6b80 1 -- 172.21.15.64:0/1780554217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cea80ba300 msgr2=0x55cea80c6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:18.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.717+0000 7f02e6fe6b80 1 --2- 172.21.15.64:0/1780554217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cea80ba300 0x55cea80c6dc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f02c80995a0 tx=0x7f02c8099b50 comp rx=0 tx=0).stop 2023-09-13T03:14:18.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.717+0000 7f02e6fe6b80 1 -- 172.21.15.64:0/1780554217 shutdown_connections 2023-09-13T03:14:18.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.717+0000 7f02e6fe6b80 1 --2- 172.21.15.64:0/1780554217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cea80ba300 0x55cea80c6dc0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:18.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.717+0000 7f02e6fe6b80 1 --2- 172.21.15.64:0/1780554217 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55cea80d4380 0x55cea80d67a0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:18.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.718+0000 7f02e6fe6b80 1 --2- 172.21.15.64:0/1780554217 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cea80c01a0 0x55cea80c4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:18.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.718+0000 7f02b4ff9700 0 client.26099 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:18.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.718+0000 7f02e6fe6b80 1 --2- 172.21.15.64:0/1780554217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cea80bacc0 0x55cea80c40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.718+0000 7f02e6fe6b80 1 -- 172.21.15.64:0/1780554217 >> 172.21.15.64:0/1780554217 conn(0x55cea802f030 msgr2=0x55cea80b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.718+0000 7f02e6fe6b80 1 -- 172.21.15.64:0/1780554217 shutdown_connections 2023-09-13T03:14:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.718+0000 7f02e6fe6b80 1 -- 172.21.15.64:0/1780554217 wait complete. 2023-09-13T03:14:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.718+0000 7f02e6fe6b80 20 client.26099 trim_cache size 0 max 16384 2023-09-13T03:14:18.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.750+0000 7f02e6fe6b80 1 Processor -- start 2023-09-13T03:14:18.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.750+0000 7f02e6fe6b80 1 -- start start 2023-09-13T03:14:18.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.750+0000 7f02e6fe6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cea80ba300 0x55cea80c6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:18.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.750+0000 7f02e6fe6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cea80bacc0 0x55cea80bf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.750+0000 7f02e6fe6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cea80c01a0 0x55cea80bf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.750+0000 7f02e6fe6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea80d1990 con 0x55cea80bacc0 2023-09-13T03:14:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.750+0000 7f02e6fe6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea80c2610 con 0x55cea80ba300 2023-09-13T03:14:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.750+0000 7f02e6fe6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea80bed60 con 0x55cea80c01a0 2023-09-13T03:14:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.751+0000 7f02d139b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cea80ba300 0x55cea80c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.751+0000 7f02d139b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cea80ba300 0x55cea80c6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:37098/0 (socket says 172.21.15.64:37098) 2023-09-13T03:14:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.751+0000 7f02d139b700 1 -- 172.21.15.64:0/2814920417 learned_addr learned my addr 172.21.15.64:0/2814920417 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.751+0000 7f02d0b9a700 1 --2- 172.21.15.64:0/2814920417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cea80bacc0 0x55cea80bf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.751+0000 7f02d1b9c700 1 --2- 172.21.15.64:0/2814920417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cea80c01a0 0x55cea80bf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.751+0000 7f02d139b700 1 -- 172.21.15.64:0/2814920417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cea80c01a0 msgr2=0x55cea80bf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.751+0000 7f02d139b700 1 --2- 172.21.15.64:0/2814920417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cea80c01a0 0x55cea80bf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:18.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.751+0000 7f02d139b700 1 -- 172.21.15.64:0/2814920417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cea80bacc0 msgr2=0x55cea80bf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:18.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.751+0000 7f02d139b700 1 --2- 172.21.15.64:0/2814920417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cea80bacc0 0x55cea80bf170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:18.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.751+0000 7f02d139b700 1 -- 172.21.15.64:0/2814920417 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cea7e5f1f0 con 0x55cea80ba300 2023-09-13T03:14:18.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.751+0000 7f02d139b700 1 --2- 172.21.15.64:0/2814920417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cea80ba300 0x55cea80c6960 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f02c8097960 tx=0x7f02c8098de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f1f308934db284b server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:18.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.751+0000 7f02b7fff700 1 -- 172.21.15.64:0/2814920417 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c8098f70 con 0x55cea80ba300 2023-09-13T03:14:18.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.751+0000 7f02b7fff700 1 -- 172.21.15.64:0/2814920417 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02c80c98c0 con 0x55cea80ba300 2023-09-13T03:14:18.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.751+0000 7f02b7fff700 1 -- 172.21.15.64:0/2814920417 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c80a1710 con 0x55cea80ba300 2023-09-13T03:14:18.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.752+0000 7f02e6fe6b80 1 -- 172.21.15.64:0/2814920417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cea80ba300 msgr2=0x55cea80c6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:18.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.752+0000 7f02e6fe6b80 1 --2- 172.21.15.64:0/2814920417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cea80ba300 0x55cea80c6960 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f02c8097960 tx=0x7f02c8098de0 comp rx=0 tx=0).stop 2023-09-13T03:14:18.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.753+0000 7f02e6fe6b80 1 -- 172.21.15.64:0/2814920417 shutdown_connections 2023-09-13T03:14:18.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.753+0000 7f02e6fe6b80 1 --2- 172.21.15.64:0/2814920417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cea80ba300 0x55cea80c6960 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:18.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.753+0000 7f02e6fe6b80 1 --2- 172.21.15.64:0/2814920417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cea80c01a0 0x55cea80bf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:18.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.753+0000 7f02e6fe6b80 1 --2- 172.21.15.64:0/2814920417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cea80bacc0 0x55cea80bf170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:18.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.753+0000 7f02e6fe6b80 1 -- 172.21.15.64:0/2814920417 >> 172.21.15.64:0/2814920417 conn(0x55cea802f030 msgr2=0x55cea80d0630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:18.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.753+0000 7f02e6fe6b80 1 -- 172.21.15.64:0/2814920417 shutdown_connections 2023-09-13T03:14:18.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.753+0000 7f02e6fe6b80 1 -- 172.21.15.64:0/2814920417 wait complete. 2023-09-13T03:14:18.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.753+0000 7f02e6fe6b80 1 -- start start 2023-09-13T03:14:18.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.754+0000 7f02e6fe6b80 1 Processor -- start 2023-09-13T03:14:18.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.754+0000 7f02e6fe6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cea80ba300 0x55cea80c7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:18.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.754+0000 7f02e6fe6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cea80bacc0 0x55cea80c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:18.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.754+0000 7f02e6fe6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cea80c01a0 0x55cea81271a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:18.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.754+0000 7f02e6fe6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea80c8200 con 0x55cea80ba300 2023-09-13T03:14:18.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.754+0000 7f02e6fe6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea81276e0 con 0x55cea80c01a0 2023-09-13T03:14:18.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.754+0000 7f02e6fe6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea8127870 con 0x55cea80bacc0 2023-09-13T03:14:18.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.754+0000 7f02e6fe6b80 10 client.0 _ll_register_callbacks cb 0x55cea7ef4a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:14:18.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.755+0000 7f02d139b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cea80ba300 0x55cea80c7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:18.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.755+0000 7f02d139b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cea80ba300 0x55cea80c7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38414/0 (socket says 172.21.15.64:38414) 2023-09-13T03:14:18.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.755+0000 7f02d139b700 1 -- 172.21.15.64:0/277995295 learned_addr learned my addr 172.21.15.64:0/277995295 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:18.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.755+0000 7f02d1b9c700 1 --2- 172.21.15.64:0/277995295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cea80c01a0 0x55cea81271a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:18.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.755+0000 7f02d0b9a700 1 --2- 172.21.15.64:0/277995295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cea80bacc0 0x55cea80c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:18.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.755+0000 7f02d139b700 1 -- 172.21.15.64:0/277995295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cea80bacc0 msgr2=0x55cea80c7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:18.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.755+0000 7f02d139b700 1 --2- 172.21.15.64:0/277995295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cea80bacc0 0x55cea80c7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:18.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.755+0000 7f02d139b700 1 -- 172.21.15.64:0/277995295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cea80c01a0 msgr2=0x55cea81271a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:18.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.755+0000 7f02d139b700 1 --2- 172.21.15.64:0/277995295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cea80c01a0 0x55cea81271a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:18.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.755+0000 7f02d139b700 1 -- 172.21.15.64:0/277995295 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cea7e21c00 con 0x55cea80ba300 2023-09-13T03:14:18.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.755+0000 7f02d139b700 1 --2- 172.21.15.64:0/277995295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cea80ba300 0x55cea80c7590 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f02c8010ad0 tx=0x7f02c8099b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c51128acc0b2ed server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.755+0000 7f02b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:14:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.755+0000 7f02b4ff9700 1 -- 172.21.15.64:0/277995295 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c8098f70 con 0x55cea80ba300 2023-09-13T03:14:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.756+0000 7f02e6fe6b80 1 -- 172.21.15.64:0/277995295 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f02bc01e040 con 0x55cea80ba300 2023-09-13T03:14:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.756+0000 7f02b4ff9700 1 -- 172.21.15.64:0/277995295 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f02c8099610 con 0x55cea80ba300 2023-09-13T03:14:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.756+0000 7f02b4ff9700 1 -- 172.21.15.64:0/277995295 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02c8099d10 con 0x55cea80ba300 2023-09-13T03:14:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.756+0000 7f02e6fe6b80 10 client.16065 fetch_fsmap learned FSMap version 30 2023-09-13T03:14:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.756+0000 7f02e6fe6b80 1 -- 172.21.15.64:0/277995295 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55cea7e37250 con 0x55cea80ba300 2023-09-13T03:14:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.756+0000 7f02b4ff9700 1 -- 172.21.15.64:0/277995295 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02bc01e040 con 0x55cea80ba300 2023-09-13T03:14:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.756+0000 7f02b4ff9700 1 -- 172.21.15.64:0/277995295 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 30) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f02c8095400 con 0x55cea80ba300 2023-09-13T03:14:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.756+0000 7f02e6fe6b80 10 client.16065 fetch_fsmap finished waiting for FSMap version 30 2023-09-13T03:14:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.756+0000 7f02e6fe6b80 10 client.16065 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:14:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.756+0000 7f02e6fe6b80 20 client.16065 populate_metadata read hostname 'smithi064' 2023-09-13T03:14:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.756+0000 7f02e6fe6b80 1 --2- 172.21.15.64:0/277995295 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55cea8128bc0 0x55cea812afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:18.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.756+0000 7f02e6fe6b80 4 client.16065 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:14:18.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.756+0000 7f02e6fe6b80 1 -- 172.21.15.64:0/277995295 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cea80058c0 con 0x55cea8128bc0 2023-09-13T03:14:18.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.757+0000 7f02d0b9a700 1 --2- 172.21.15.64:0/277995295 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55cea8128bc0 0x55cea812afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:18.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.757+0000 7f02d0b9a700 1 --2- 172.21.15.64:0/277995295 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55cea8128bc0 0x55cea812afb0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45699aaa1aad00f0 server_cookie=e4c02fdb780ee4ba in_seq=0 out_seq=0 2023-09-13T03:14:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.757+0000 7f02b4ff9700 10 client.16065 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.789+0000 7f02b4ff9700 1 -- 172.21.15.64:0/277995295 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cea80058c0 con 0x55cea8128bc0 2023-09-13T03:14:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.789+0000 7f02b4ff9700 10 client.16065 handle_command_reply: tid=0 2023-09-13T03:14:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.789+0000 7f02e6fe6b80 1 client.16065 shutdown 2023-09-13T03:14:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.790+0000 7f02e6fe6b80 1 -- 172.21.15.64:0/277995295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cea80ba300 msgr2=0x55cea80c7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.790+0000 7f02e6fe6b80 1 --2- 172.21.15.64:0/277995295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cea80ba300 0x55cea80c7590 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f02c8010ad0 tx=0x7f02c8099b50 comp rx=0 tx=0).stop 2023-09-13T03:14:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.790+0000 7f02e6fe6b80 1 -- 172.21.15.64:0/277995295 shutdown_connections 2023-09-13T03:14:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.790+0000 7f02e6fe6b80 1 --2- 172.21.15.64:0/277995295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cea80c01a0 0x55cea81271a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.790+0000 7f02e6fe6b80 1 --2- 172.21.15.64:0/277995295 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55cea8128bc0 0x55cea812afb0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.790+0000 7f02e6fe6b80 1 --2- 172.21.15.64:0/277995295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cea80ba300 0x55cea80c7590 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:18.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.790+0000 7f02e6fe6b80 1 --2- 172.21.15.64:0/277995295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cea80bacc0 0x55cea80c7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:18.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.790+0000 7f02b4ff9700 0 client.16065 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:18.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.790+0000 7f02e6fe6b80 1 -- 172.21.15.64:0/277995295 >> 172.21.15.64:0/277995295 conn(0x55cea802f030 msgr2=0x55cea80be1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:18.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.790+0000 7f02e6fe6b80 1 -- 172.21.15.64:0/277995295 shutdown_connections 2023-09-13T03:14:18.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.790+0000 7f02e6fe6b80 1 -- 172.21.15.64:0/277995295 wait complete. 2023-09-13T03:14:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.790+0000 7f02e6fe6b80 20 client.16065 trim_cache size 0 max 16384 2023-09-13T03:14:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:14:18.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:14:18.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:14:18.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:14:18.796 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.791+0000 7f02d35ff700 1 -- 172.21.15.64:0/3171176616 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f02b80a9160 msgr2=0x7f02b80ab620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:18.796 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.791+0000 7f02d35ff700 1 --2- 172.21.15.64:0/3171176616 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f02b80a9160 0x7f02b80ab620 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f02cc19f4f0 tx=0x7f02c4004ad0 comp rx=0 tx=0).stop 2023-09-13T03:14:18.796 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.791+0000 7f02d35ff700 1 -- 172.21.15.64:0/3171176616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02cc1005a0 msgr2=0x7f02cc19e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:18.796 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.791+0000 7f02d35ff700 1 --2- 172.21.15.64:0/3171176616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02cc1005a0 0x7f02cc19e7f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f02c800e9a0 tx=0x7f02c800ecb0 comp rx=0 tx=0).stop 2023-09-13T03:14:18.796 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.791+0000 7f02d35ff700 1 -- 172.21.15.64:0/3171176616 shutdown_connections 2023-09-13T03:14:18.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.791+0000 7f02d35ff700 1 --2- 172.21.15.64:0/3171176616 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f02b80a9160 0x7f02b80ab620 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:18.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.791+0000 7f02d35ff700 1 --2- 172.21.15.64:0/3171176616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02cc1005a0 0x7f02cc19e7f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:18.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.791+0000 7f02d35ff700 1 --2- 172.21.15.64:0/3171176616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02cc106660 0x7f02cc198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:18.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.791+0000 7f02d35ff700 1 --2- 172.21.15.64:0/3171176616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02cc100f60 0x7f02cc19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:18.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.791+0000 7f02d35ff700 1 -- 172.21.15.64:0/3171176616 >> 172.21.15.64:0/3171176616 conn(0x7f02cc0fc050 msgr2=0x7f02cc10b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:18.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.792+0000 7f02d35ff700 1 -- 172.21.15.64:0/3171176616 shutdown_connections 2023-09-13T03:14:18.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:18.792+0000 7f02d35ff700 1 -- 172.21.15.64:0/3171176616 wait complete. 2023-09-13T03:14:18.886 INFO:tasks.workunit.client.0.smithi064.stdout: 1 25788 6.76 MB/sec warmup 23 sec latency 3419.713 ms 2023-09-13T03:14:18.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:18 smithi170 ceph-mon[134328]: pgmap v37: 97 pgs: 97 active+clean; 12 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 432 KiB/s wr, 118 op/s 2023-09-13T03:14:18.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:18 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:14:18.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:18 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:14:18.969 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:18 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:14:18.970 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:18 smithi064 ceph-mon[97675]: pgmap v37: 97 pgs: 97 active+clean; 12 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 432 KiB/s wr, 118 op/s 2023-09-13T03:14:18.970 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:18 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:14:18.970 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:18 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:14:18.970 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:18 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:14:18.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:18 smithi142 ceph-mon[134082]: pgmap v37: 97 pgs: 97 active+clean; 12 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 432 KiB/s wr, 118 op/s 2023-09-13T03:14:18.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:18 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:14:18.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:18 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:14:18.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:18 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:14:19.886 INFO:tasks.workunit.client.0.smithi064.stdout: 1 25788 6.48 MB/sec warmup 24 sec latency 4419.818 ms 2023-09-13T03:14:19.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:19 smithi170 ceph-mon[134328]: scrub summary: idle 2023-09-13T03:14:19.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:19 smithi170 ceph-mon[134328]: scrub summary: idle 2023-09-13T03:14:19.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:19 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:14:19.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:19 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:14:19.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:19 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:14:19.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:19 smithi064 ceph-mon[97675]: scrub summary: idle 2023-09-13T03:14:19.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:19 smithi064 ceph-mon[97675]: scrub summary: idle 2023-09-13T03:14:19.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:19 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:14:19.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:19 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:14:19.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:19 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:14:19.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:19 smithi142 ceph-mon[134082]: scrub summary: idle 2023-09-13T03:14:19.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:19 smithi142 ceph-mon[134082]: scrub summary: idle 2023-09-13T03:14:19.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:19 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:14:19.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:19 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:14:19.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:19 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:14:20.886 INFO:tasks.workunit.client.0.smithi064.stdout: 1 25853 6.23 MB/sec warmup 25 sec latency 4877.148 ms 2023-09-13T03:14:20.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:20 smithi170 ceph-mon[134328]: scrub summary: idle 2023-09-13T03:14:20.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:20 smithi170 ceph-mon[134328]: pgmap v38: 97 pgs: 97 active+clean; 12 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 169 KiB/s wr, 45 op/s 2023-09-13T03:14:20.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:20 smithi064 ceph-mon[97675]: scrub summary: idle 2023-09-13T03:14:20.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:20 smithi064 ceph-mon[97675]: pgmap v38: 97 pgs: 97 active+clean; 12 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 169 KiB/s wr, 45 op/s 2023-09-13T03:14:20.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:20 smithi142 ceph-mon[134082]: scrub summary: idle 2023-09-13T03:14:20.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:20 smithi142 ceph-mon[134082]: pgmap v38: 97 pgs: 97 active+clean; 12 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 169 KiB/s wr, 45 op/s 2023-09-13T03:14:21.886 INFO:tasks.workunit.client.0.smithi064.stdout: 1 25853 5.99 MB/sec warmup 26 sec latency 1510.392 ms 2023-09-13T03:14:22.886 INFO:tasks.workunit.client.0.smithi064.stdout: 1 25853 5.77 MB/sec warmup 27 sec latency 2510.539 ms 2023-09-13T03:14:22.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:22 smithi170 ceph-mon[134328]: pgmap v39: 97 pgs: 97 active+clean; 12 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 216 KiB/s wr, 61 op/s 2023-09-13T03:14:22.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:22 smithi064 ceph-mon[97675]: pgmap v39: 97 pgs: 97 active+clean; 12 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 216 KiB/s wr, 61 op/s 2023-09-13T03:14:22.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:22 smithi142 ceph-mon[134082]: pgmap v39: 97 pgs: 97 active+clean; 12 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 216 KiB/s wr, 61 op/s 2023-09-13T03:14:23.886 INFO:tasks.workunit.client.0.smithi064.stdout: 1 25853 5.56 MB/sec warmup 28 sec latency 3510.634 ms 2023-09-13T03:14:24.887 INFO:tasks.workunit.client.0.smithi064.stdout: 1 25853 5.37 MB/sec warmup 29 sec latency 4510.754 ms 2023-09-13T03:14:24.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:24 smithi170 ceph-mon[134328]: pgmap v40: 97 pgs: 97 active+clean; 12 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 131 KiB/s wr, 44 op/s 2023-09-13T03:14:24.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:24 smithi064 ceph-mon[97675]: pgmap v40: 97 pgs: 97 active+clean; 12 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 131 KiB/s wr, 44 op/s 2023-09-13T03:14:24.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:24 smithi142 ceph-mon[134082]: pgmap v40: 97 pgs: 97 active+clean; 12 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 131 KiB/s wr, 44 op/s 2023-09-13T03:14:25.886 INFO:tasks.workunit.client.0.smithi064.stdout: 1 25854 5.19 MB/sec warmup 30 sec latency 4976.875 ms 2023-09-13T03:14:26.887 INFO:tasks.workunit.client.0.smithi064.stdout: 1 25854 5.02 MB/sec warmup 31 sec latency 1534.012 ms 2023-09-13T03:14:26.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:26 smithi170 ceph-mon[134328]: pgmap v41: 97 pgs: 97 active+clean; 25 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 1.8 MiB/s wr, 72 op/s 2023-09-13T03:14:26.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:26 smithi064 ceph-mon[97675]: pgmap v41: 97 pgs: 97 active+clean; 25 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 1.8 MiB/s wr, 72 op/s 2023-09-13T03:14:26.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:26 smithi142 ceph-mon[134082]: pgmap v41: 97 pgs: 97 active+clean; 25 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 1.8 MiB/s wr, 72 op/s 2023-09-13T03:14:27.887 INFO:tasks.workunit.client.0.smithi064.stdout: 1 25854 4.87 MB/sec warmup 32 sec latency 2534.098 ms 2023-09-13T03:14:28.887 INFO:tasks.workunit.client.0.smithi064.stdout: 1 25854 4.72 MB/sec warmup 33 sec latency 3534.238 ms 2023-09-13T03:14:28.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:28 smithi170 ceph-mon[134328]: pgmap v42: 97 pgs: 97 active+clean; 29 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 2.4 MiB/s wr, 63 op/s 2023-09-13T03:14:28.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:28 smithi064 ceph-mon[97675]: pgmap v42: 97 pgs: 97 active+clean; 29 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 2.4 MiB/s wr, 63 op/s 2023-09-13T03:14:28.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:28 smithi142 ceph-mon[134082]: pgmap v42: 97 pgs: 97 active+clean; 29 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 2.4 MiB/s wr, 63 op/s 2023-09-13T03:14:29.887 INFO:tasks.workunit.client.0.smithi064.stdout: 1 25854 4.58 MB/sec warmup 34 sec latency 4534.389 ms 2023-09-13T03:14:29.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:29 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:14:29.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:29 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:14:29.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:29 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:14:30.887 INFO:tasks.workunit.client.0.smithi064.stdout: 1 26058 4.48 MB/sec warmup 35 sec latency 4989.960 ms 2023-09-13T03:14:30.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:30 smithi170 ceph-mon[134328]: pgmap v43: 97 pgs: 97 active+clean; 29 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 2.4 MiB/s wr, 56 op/s 2023-09-13T03:14:30.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:30 smithi064 ceph-mon[97675]: pgmap v43: 97 pgs: 97 active+clean; 29 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 2.4 MiB/s wr, 56 op/s 2023-09-13T03:14:30.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:30 smithi142 ceph-mon[134082]: pgmap v43: 97 pgs: 97 active+clean; 29 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 2.4 MiB/s wr, 56 op/s 2023-09-13T03:14:31.887 INFO:tasks.workunit.client.0.smithi064.stdout: 1 26058 4.36 MB/sec warmup 36 sec latency 1030.966 ms 2023-09-13T03:14:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.951+0000 7fd1cefff700 1 -- 172.21.15.64:0/4216138255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1c8106660 msgr2=0x7fd1c8106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.951+0000 7fd1cefff700 1 --2- 172.21.15.64:0/4216138255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1c8106660 0x7fd1c8106a40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc009920 tx=0x7fd1bc009c30 comp rx=0 tx=0).stop 2023-09-13T03:14:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.951+0000 7fd1cefff700 1 -- 172.21.15.64:0/4216138255 shutdown_connections 2023-09-13T03:14:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.951+0000 7fd1cefff700 1 --2- 172.21.15.64:0/4216138255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1c8100f60 0x7fd1c810d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.951+0000 7fd1cefff700 1 --2- 172.21.15.64:0/4216138255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1c8106660 0x7fd1c8106a40 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.951+0000 7fd1cefff700 1 --2- 172.21.15.64:0/4216138255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1c81005a0 0x7fd1c8100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.951+0000 7fd1cefff700 1 -- 172.21.15.64:0/4216138255 >> 172.21.15.64:0/4216138255 conn(0x7fd1c80fc0b0 msgr2=0x7fd1c80fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.951+0000 7fd1cefff700 1 -- 172.21.15.64:0/4216138255 shutdown_connections 2023-09-13T03:14:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.951+0000 7fd1cefff700 1 -- 172.21.15.64:0/4216138255 wait complete. 2023-09-13T03:14:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.952+0000 7fd1cefff700 1 Processor -- start 2023-09-13T03:14:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.953+0000 7fd1cefff700 1 -- start start 2023-09-13T03:14:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.953+0000 7fd1cefff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1c81005a0 0x7fd1c819c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.953+0000 7fd1cefff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1c8100f60 0x7fd1c819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.953+0000 7fd1cefff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1c8106660 0x7fd1c8196520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.953+0000 7fd1cefff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c8196b90 con 0x7fd1c8106660 2023-09-13T03:14:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.953+0000 7fd1cefff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c8196d00 con 0x7fd1c8100f60 2023-09-13T03:14:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.953+0000 7fd1cefff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c8196ec0 con 0x7fd1c81005a0 2023-09-13T03:14:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.953+0000 7fd1cd59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1c8106660 0x7fd1c8196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.953+0000 7fd1ccd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1c81005a0 0x7fd1c819c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.953+0000 7fd1c7fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1c8100f60 0x7fd1c819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.953+0000 7fd1ccd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1c81005a0 0x7fd1c819c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:52808/0 (socket says 172.21.15.64:52808) 2023-09-13T03:14:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.953+0000 7fd1ccd9b700 1 -- 172.21.15.64:0/689909611 learned_addr learned my addr 172.21.15.64:0/689909611 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.954+0000 7fd1cd59c700 1 -- 172.21.15.64:0/689909611 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1c81005a0 msgr2=0x7fd1c819c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.954+0000 7fd1cd59c700 1 --2- 172.21.15.64:0/689909611 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1c81005a0 0x7fd1c819c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.954+0000 7fd1cd59c700 1 -- 172.21.15.64:0/689909611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1c8100f60 msgr2=0x7fd1c819cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.954+0000 7fd1cd59c700 1 --2- 172.21.15.64:0/689909611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1c8100f60 0x7fd1c819cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.954+0000 7fd1cd59c700 1 -- 172.21.15.64:0/689909611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b40095f0 con 0x7fd1c8106660 2023-09-13T03:14:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.954+0000 7fd1cd59c700 1 --2- 172.21.15.64:0/689909611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1c8106660 0x7fd1c8196520 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd1c000efc0 tx=0x7fd1c000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.954+0000 7fd1c5ffb700 1 -- 172.21.15.64:0/689909611 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c0018070 con 0x7fd1c8106660 2023-09-13T03:14:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.954+0000 7fd1c5ffb700 1 -- 172.21.15.64:0/689909611 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1c000f040 con 0x7fd1c8106660 2023-09-13T03:14:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.954+0000 7fd1cefff700 1 -- 172.21.15.64:0/689909611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1bc009520 con 0x7fd1c8106660 2023-09-13T03:14:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.954+0000 7fd1c5ffb700 1 -- 172.21.15.64:0/689909611 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c000cd40 con 0x7fd1c8106660 2023-09-13T03:14:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.954+0000 7fd1cefff700 1 -- 172.21.15.64:0/689909611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1c81a34d0 con 0x7fd1c8106660 2023-09-13T03:14:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.958+0000 7fd1c5ffb700 1 -- 172.21.15.64:0/689909611 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) v1 ==== 135492+0+0 (secure 0 0 0) 0x7fd1c0003a00 con 0x7fd1c8106660 2023-09-13T03:14:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.959+0000 7fd1e2921b80 1 Processor -- start 2023-09-13T03:14:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.960+0000 7fd1e2921b80 1 -- start start 2023-09-13T03:14:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.960+0000 7fd1e2921b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2aa30c1a0 0x55e2aa30c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.960+0000 7fd1e2921b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e2aa306300 0x55e2aa306780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.960+0000 7fd1e2921b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2aa306cc0 0x55e2aa312f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.960+0000 7fd1e2921b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2aa25f6d0 con 0x55e2aa30c1a0 2023-09-13T03:14:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.960+0000 7fd1e2921b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2aa25cff0 con 0x55e2aa306300 2023-09-13T03:14:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.960+0000 7fd1c5ffb700 1 --2- 172.21.15.64:0/689909611 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fd1b80a0950 0x7fd1b80a2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.960+0000 7fd1e2921b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2aa2588e0 con 0x55e2aa306cc0 2023-09-13T03:14:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.961+0000 7fd1c5ffb700 1 -- 172.21.15.64:0/689909611 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd1c00cbd70 con 0x7fd1c8106660 2023-09-13T03:14:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.961+0000 7fd1c7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2aa30c1a0 0x55e2aa30c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.961+0000 7fd1cd59c700 1 --2- 172.21.15.64:0/689909611 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fd1b80a0950 0x7fd1b80a2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.961+0000 7fd1c7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2aa30c1a0 0x55e2aa30c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50288/0 (socket says 172.21.15.64:50288) 2023-09-13T03:14:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.961+0000 7fd1ccd9b700 1 --2- 172.21.15.64:0/493911139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e2aa306300 0x55e2aa306780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.961+0000 7fd1c7fff700 1 -- 172.21.15.64:0/493911139 learned_addr learned my addr 172.21.15.64:0/493911139 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.961+0000 7fd1c7fff700 1 --2- 172.21.15.64:0/493911139 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2aa306cc0 0x55e2aa312f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.961+0000 7fd1ccd9b700 1 -- 172.21.15.64:0/493911139 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2aa306cc0 msgr2=0x55e2aa312f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.961+0000 7fd1ccd9b700 1 --2- 172.21.15.64:0/493911139 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2aa306cc0 0x55e2aa312f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.961+0000 7fd1ccd9b700 1 -- 172.21.15.64:0/493911139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2aa30c1a0 msgr2=0x55e2aa30c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.961+0000 7fd1ccd9b700 1 --2- 172.21.15.64:0/493911139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2aa30c1a0 0x55e2aa30c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.961+0000 7fd1ccd9b700 1 -- 172.21.15.64:0/493911139 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2aa084d80 con 0x55e2aa306300 2023-09-13T03:14:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.961+0000 7fd1c7fff700 1 --2- 172.21.15.64:0/493911139 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2aa306cc0 0x55e2aa312f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T03:14:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.961+0000 7fd1cd59c700 1 --2- 172.21.15.64:0/689909611 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fd1b80a0950 0x7fd1b80a2e10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd1c00c9390 tx=0x7fd1c0000f40 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.962+0000 7fd1ccd9b700 1 --2- 172.21.15.64:0/493911139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e2aa306300 0x55e2aa306780 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc005e90 tx=0x7fd1bc005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=859ce897ea635889 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.962+0000 7fd1b2ffd700 1 -- 172.21.15.64:0/493911139 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc00f720 con 0x55e2aa306300 2023-09-13T03:14:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.962+0000 7fd1b2ffd700 1 -- 172.21.15.64:0/493911139 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1bc01d040 con 0x55e2aa306300 2023-09-13T03:14:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.962+0000 7fd1b2ffd700 1 -- 172.21.15.64:0/493911139 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc005470 con 0x55e2aa306300 2023-09-13T03:14:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.963+0000 7fd1e2921b80 1 -- 172.21.15.64:0/493911139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e2aa306300 msgr2=0x55e2aa306780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.963+0000 7fd1e2921b80 1 --2- 172.21.15.64:0/493911139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e2aa306300 0x55e2aa306780 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc005e90 tx=0x7fd1bc005dc0 comp rx=0 tx=0).stop 2023-09-13T03:14:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.963+0000 7fd1e2921b80 1 -- 172.21.15.64:0/493911139 shutdown_connections 2023-09-13T03:14:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.963+0000 7fd1e2921b80 1 --2- 172.21.15.64:0/493911139 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2aa306cc0 0x55e2aa312f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.963+0000 7fd1e2921b80 1 --2- 172.21.15.64:0/493911139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2aa30c1a0 0x55e2aa30c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.963+0000 7fd1e2921b80 1 --2- 172.21.15.64:0/493911139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e2aa306300 0x55e2aa306780 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.963+0000 7fd1e2921b80 1 -- 172.21.15.64:0/493911139 >> 172.21.15.64:0/493911139 conn(0x55e2aa27b030 msgr2=0x55e2aa301560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.963+0000 7fd1e2921b80 1 -- 172.21.15.64:0/493911139 shutdown_connections 2023-09-13T03:14:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.963+0000 7fd1e2921b80 1 -- 172.21.15.64:0/493911139 wait complete. 2023-09-13T03:14:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.963+0000 7fd1e2921b80 1 -- start start 2023-09-13T03:14:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.964+0000 7fd1e2921b80 1 Processor -- start 2023-09-13T03:14:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.964+0000 7fd1e2921b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2aa306300 0x55e2aa312db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.964+0000 7fd1e2921b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2aa306cc0 0x55e2aa310030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.964+0000 7fd1e2921b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e2aa30c1a0 0x55e2aa3105f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.964+0000 7fd1e2921b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2aa1f6b10 con 0x55e2aa306300 2023-09-13T03:14:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.964+0000 7fd1e2921b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2aa1f6810 con 0x55e2aa30c1a0 2023-09-13T03:14:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.964+0000 7fd1e2921b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2aa0da8c0 con 0x55e2aa306cc0 2023-09-13T03:14:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.964+0000 7fd1e2921b80 10 client.0 _ll_register_callbacks cb 0x55e2aa140a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:14:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.964+0000 7fd1c7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2aa306300 0x55e2aa312db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.964+0000 7fd1c7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2aa306300 0x55e2aa312db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50300/0 (socket says 172.21.15.64:50300) 2023-09-13T03:14:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.964+0000 7fd1c7fff700 1 -- 172.21.15.64:0/3359105777 learned_addr learned my addr 172.21.15.64:0/3359105777 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.965+0000 7fd1ccd9b700 1 --2- 172.21.15.64:0/3359105777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2aa306cc0 0x55e2aa310030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.965+0000 7fd1c7fff700 1 --2- 172.21.15.64:0/3359105777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e2aa30c1a0 0x55e2aa3105f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.965+0000 7fd1ccd9b700 1 -- 172.21.15.64:0/3359105777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e2aa30c1a0 msgr2=0x55e2aa3105f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.965+0000 7fd1ccd9b700 1 --2- 172.21.15.64:0/3359105777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e2aa30c1a0 0x55e2aa3105f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.965+0000 7fd1ccd9b700 1 -- 172.21.15.64:0/3359105777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2aa306300 msgr2=0x55e2aa312db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.965+0000 7fd1ccd9b700 1 --2- 172.21.15.64:0/3359105777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2aa306300 0x55e2aa312db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.965+0000 7fd1ccd9b700 1 -- 172.21.15.64:0/3359105777 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1c810dd20 con 0x55e2aa306cc0 2023-09-13T03:14:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.965+0000 7fd1c7fff700 1 --2- 172.21.15.64:0/3359105777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2aa306300 0x55e2aa312db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.965+0000 7fd1ccd9b700 1 --2- 172.21.15.64:0/3359105777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2aa306cc0 0x55e2aa310030 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc00c010 tx=0x7fd1bc00b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93909f14a937dc55 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.965+0000 7fd1a7fff700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:14:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.966+0000 7fd1a7fff700 1 -- 172.21.15.64:0/3359105777 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc005040 con 0x55e2aa306cc0 2023-09-13T03:14:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.966+0000 7fd1e2921b80 1 -- 172.21.15.64:0/3359105777 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e2aa1b0490 con 0x55e2aa306cc0 2023-09-13T03:14:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.966+0000 7fd1a7fff700 1 -- 172.21.15.64:0/3359105777 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd1bc0051a0 con 0x55e2aa306cc0 2023-09-13T03:14:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.966+0000 7fd1a7fff700 1 -- 172.21.15.64:0/3359105777 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1bc005360 con 0x55e2aa306cc0 2023-09-13T03:14:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.966+0000 7fd1e2921b80 10 client.26116 fetch_fsmap learned FSMap version 31 2023-09-13T03:14:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.966+0000 7fd1e2921b80 1 -- 172.21.15.64:0/3359105777 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55e2aa06dc00 con 0x55e2aa306cc0 2023-09-13T03:14:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.966+0000 7fd1a7fff700 1 -- 172.21.15.64:0/3359105777 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e2aa1b0490 con 0x55e2aa306cc0 2023-09-13T03:14:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.967+0000 7fd1a7fff700 1 -- 172.21.15.64:0/3359105777 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 31) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fd1bc009ca0 con 0x55e2aa306cc0 2023-09-13T03:14:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.967+0000 7fd1e2921b80 10 client.26116 fetch_fsmap finished waiting for FSMap version 31 2023-09-13T03:14:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.967+0000 7fd1e2921b80 10 client.26116 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:14:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.967+0000 7fd1e2921b80 20 client.26116 populate_metadata read hostname 'smithi064' 2023-09-13T03:14:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.967+0000 7fd1e2921b80 1 --2- 172.21.15.64:0/3359105777 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55e2aa31fe70 0x55e2aa322290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.967+0000 7fd1e2921b80 4 client.26116 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:14:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.967+0000 7fd1e2921b80 1 -- 172.21.15.64:0/3359105777 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e2aa26d840 con 0x55e2aa31fe70 2023-09-13T03:14:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.967+0000 7fd1c7fff700 1 --2- 172.21.15.64:0/3359105777 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55e2aa31fe70 0x55e2aa322290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.968+0000 7fd1c7fff700 1 --2- 172.21.15.64:0/3359105777 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55e2aa31fe70 0x55e2aa322290 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e4f254d62f823b4 server_cookie=4dc3254ad67f7f0b in_seq=0 out_seq=0 2023-09-13T03:14:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.968+0000 7fd1a7fff700 10 client.26116 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.968+0000 7fd1a7fff700 1 -- 172.21.15.64:0/3359105777 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e2aa26d840 con 0x55e2aa31fe70 2023-09-13T03:14:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.968+0000 7fd1a7fff700 10 client.26116 handle_command_reply: tid=0 2023-09-13T03:14:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.968+0000 7fd1e2921b80 1 client.26116 shutdown 2023-09-13T03:14:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.969+0000 7fd1e2921b80 1 -- 172.21.15.64:0/3359105777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2aa306cc0 msgr2=0x55e2aa310030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.969+0000 7fd1e2921b80 1 --2- 172.21.15.64:0/3359105777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2aa306cc0 0x55e2aa310030 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc00c010 tx=0x7fd1bc00b810 comp rx=0 tx=0).stop 2023-09-13T03:14:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.969+0000 7fd1e2921b80 1 -- 172.21.15.64:0/3359105777 shutdown_connections 2023-09-13T03:14:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.969+0000 7fd1e2921b80 1 --2- 172.21.15.64:0/3359105777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e2aa30c1a0 0x55e2aa3105f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.969+0000 7fd1e2921b80 1 --2- 172.21.15.64:0/3359105777 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55e2aa31fe70 0x55e2aa322290 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.969+0000 7fd1e2921b80 1 --2- 172.21.15.64:0/3359105777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2aa306300 0x55e2aa312db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.969+0000 7fd1e2921b80 1 --2- 172.21.15.64:0/3359105777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2aa306cc0 0x55e2aa310030 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.969+0000 7fd1a7fff700 0 client.26116 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.969+0000 7fd1e2921b80 1 -- 172.21.15.64:0/3359105777 >> 172.21.15.64:0/3359105777 conn(0x55e2aa27b030 msgr2=0x55e2aa3007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.969+0000 7fd1e2921b80 1 -- 172.21.15.64:0/3359105777 shutdown_connections 2023-09-13T03:14:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.969+0000 7fd1e2921b80 1 -- 172.21.15.64:0/3359105777 wait complete. 2023-09-13T03:14:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.969+0000 7fd1e2921b80 20 client.26116 trim_cache size 0 max 16384 2023-09-13T03:14:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.985+0000 7fd1e2921b80 1 Processor -- start 2023-09-13T03:14:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.985+0000 7fd1e2921b80 1 -- start start 2023-09-13T03:14:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.985+0000 7fd1e2921b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e2aa306300 0x55e2aa312950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.985+0000 7fd1e2921b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2aa306cc0 0x55e2aa30b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.985+0000 7fd1e2921b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2aa30c1a0 0x55e2aa30b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.985+0000 7fd1e2921b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2aa31cce0 con 0x55e2aa30c1a0 2023-09-13T03:14:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.985+0000 7fd1e2921b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2aa3228f0 con 0x55e2aa306300 2023-09-13T03:14:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.985+0000 7fd1e2921b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2aa30e5b0 con 0x55e2aa306cc0 2023-09-13T03:14:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.985+0000 7fd1c7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2aa30c1a0 0x55e2aa30b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.985+0000 7fd1c7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2aa30c1a0 0x55e2aa30b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50312/0 (socket says 172.21.15.64:50312) 2023-09-13T03:14:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.985+0000 7fd1c7fff700 1 -- 172.21.15.64:0/4069183327 learned_addr learned my addr 172.21.15.64:0/4069183327 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.985+0000 7fd1ccd9b700 1 --2- 172.21.15.64:0/4069183327 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2aa306cc0 0x55e2aa30b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.986+0000 7fd1c7fff700 1 --2- 172.21.15.64:0/4069183327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e2aa306300 0x55e2aa312950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.986+0000 7fd1c7fff700 1 -- 172.21.15.64:0/4069183327 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2aa306cc0 msgr2=0x55e2aa30b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.986+0000 7fd1c7fff700 1 --2- 172.21.15.64:0/4069183327 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2aa306cc0 0x55e2aa30b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.986+0000 7fd1c7fff700 1 -- 172.21.15.64:0/4069183327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e2aa306300 msgr2=0x55e2aa312950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.986+0000 7fd1c7fff700 1 --2- 172.21.15.64:0/4069183327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e2aa306300 0x55e2aa312950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.986+0000 7fd1c7fff700 1 -- 172.21.15.64:0/4069183327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2aa0ab1f0 con 0x55e2aa30c1a0 2023-09-13T03:14:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.986+0000 7fd1ccd9b700 1 --2- 172.21.15.64:0/4069183327 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2aa306cc0 0x55e2aa30b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.986+0000 7fd1c7fff700 1 --2- 172.21.15.64:0/4069183327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2aa30c1a0 0x55e2aa30b6b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x55e2aa322b30 tx=0x7fd1b4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e88d57438c966af server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.986+0000 7fd1b37fe700 1 -- 172.21.15.64:0/4069183327 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4012e90 con 0x55e2aa30c1a0 2023-09-13T03:14:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.987+0000 7fd1b37fe700 1 -- 172.21.15.64:0/4069183327 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1b4029030 con 0x55e2aa30c1a0 2023-09-13T03:14:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.987+0000 7fd1b37fe700 1 -- 172.21.15.64:0/4069183327 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4004c50 con 0x55e2aa30c1a0 2023-09-13T03:14:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.987+0000 7fd1e2921b80 1 -- 172.21.15.64:0/4069183327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2aa30c1a0 msgr2=0x55e2aa30b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.987+0000 7fd1e2921b80 1 --2- 172.21.15.64:0/4069183327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2aa30c1a0 0x55e2aa30b6b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x55e2aa322b30 tx=0x7fd1b4009310 comp rx=0 tx=0).stop 2023-09-13T03:14:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.987+0000 7fd1e2921b80 1 -- 172.21.15.64:0/4069183327 shutdown_connections 2023-09-13T03:14:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.987+0000 7fd1e2921b80 1 --2- 172.21.15.64:0/4069183327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e2aa306300 0x55e2aa312950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.987+0000 7fd1e2921b80 1 --2- 172.21.15.64:0/4069183327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2aa30c1a0 0x55e2aa30b6b0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.987+0000 7fd1e2921b80 1 --2- 172.21.15.64:0/4069183327 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2aa306cc0 0x55e2aa30b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.987+0000 7fd1e2921b80 1 -- 172.21.15.64:0/4069183327 >> 172.21.15.64:0/4069183327 conn(0x55e2aa27b030 msgr2=0x55e2aa31c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.987+0000 7fd1e2921b80 1 -- 172.21.15.64:0/4069183327 shutdown_connections 2023-09-13T03:14:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.987+0000 7fd1e2921b80 1 -- 172.21.15.64:0/4069183327 wait complete. 2023-09-13T03:14:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.988+0000 7fd1e2921b80 1 -- start start 2023-09-13T03:14:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.988+0000 7fd1e2921b80 1 Processor -- start 2023-09-13T03:14:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.989+0000 7fd1e2921b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2aa306300 0x55e2aa3135f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.989+0000 7fd1e2921b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e2aa306cc0 0x55e2aa313b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.989+0000 7fd1e2921b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2aa30c1a0 0x55e2aa372c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.989+0000 7fd1e2921b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2aa314270 con 0x55e2aa306300 2023-09-13T03:14:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.989+0000 7fd1e2921b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2aa373190 con 0x55e2aa306cc0 2023-09-13T03:14:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.989+0000 7fd1e2921b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2aa373320 con 0x55e2aa30c1a0 2023-09-13T03:14:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.989+0000 7fd1e2921b80 10 client.0 _ll_register_callbacks cb 0x7fd1b4009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:14:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.989+0000 7fd1c7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2aa306300 0x55e2aa3135f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.989+0000 7fd1ccd9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e2aa306cc0 0x55e2aa313b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.989+0000 7fd1ccd9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e2aa306cc0 0x55e2aa313b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:40244/0 (socket says 172.21.15.64:40244) 2023-09-13T03:14:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.989+0000 7fd1ccd9b700 1 -- 172.21.15.64:0/1816923568 learned_addr learned my addr 172.21.15.64:0/1816923568 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.989+0000 7fd1c7fff700 1 --2- 172.21.15.64:0/1816923568 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2aa30c1a0 0x55e2aa372c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.989+0000 7fd1ccd9b700 1 -- 172.21.15.64:0/1816923568 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2aa30c1a0 msgr2=0x55e2aa372c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.989+0000 7fd1ccd9b700 1 --2- 172.21.15.64:0/1816923568 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2aa30c1a0 0x55e2aa372c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.989+0000 7fd1ccd9b700 1 -- 172.21.15.64:0/1816923568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2aa306300 msgr2=0x55e2aa3135f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.989+0000 7fd1ccd9b700 1 --2- 172.21.15.64:0/1816923568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2aa306300 0x55e2aa3135f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.990+0000 7fd1ccd9b700 1 -- 172.21.15.64:0/1816923568 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e2aa06dc00 con 0x55e2aa306cc0 2023-09-13T03:14:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.990+0000 7fd1ccd9b700 1 --2- 172.21.15.64:0/1816923568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e2aa306cc0 0x55e2aa313b30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc004fd0 tx=0x7fd1bc0038b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38fc1fec7d37fd9d server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.990+0000 7fd1a7fff700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:14:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.990+0000 7fd1a7fff700 1 -- 172.21.15.64:0/1816923568 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc041070 con 0x55e2aa306cc0 2023-09-13T03:14:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.990+0000 7fd1e2921b80 1 -- 172.21.15.64:0/1816923568 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd1bc01d040 con 0x55e2aa306cc0 2023-09-13T03:14:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.990+0000 7fd1a7fff700 1 -- 172.21.15.64:0/1816923568 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd1bc000e40 con 0x55e2aa306cc0 2023-09-13T03:14:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.990+0000 7fd1a7fff700 1 -- 172.21.15.64:0/1816923568 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1bc00b2e0 con 0x55e2aa306cc0 2023-09-13T03:14:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.990+0000 7fd1e2921b80 10 client.26123 fetch_fsmap learned FSMap version 31 2023-09-13T03:14:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.990+0000 7fd1e2921b80 1 -- 172.21.15.64:0/1816923568 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fd1b4023490 con 0x55e2aa306cc0 2023-09-13T03:14:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.991+0000 7fd1a7fff700 1 -- 172.21.15.64:0/1816923568 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1bc04a020 con 0x55e2aa306cc0 2023-09-13T03:14:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.991+0000 7fd1a7fff700 1 -- 172.21.15.64:0/1816923568 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fd1bc044030 con 0x55e2aa306cc0 2023-09-13T03:14:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.991+0000 7fd1e2921b80 10 client.26123 fetch_fsmap finished waiting for FSMap version 31 2023-09-13T03:14:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.991+0000 7fd1e2921b80 10 client.26123 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:14:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.991+0000 7fd1e2921b80 20 client.26123 populate_metadata read hostname 'smithi064' 2023-09-13T03:14:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.991+0000 7fd1e2921b80 1 --2- 172.21.15.64:0/1816923568 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55e2aa3747a0 0x55e2aa376bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.991+0000 7fd1e2921b80 4 client.26123 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:14:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.991+0000 7fd1e2921b80 1 -- 172.21.15.64:0/1816923568 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e2aa2518c0 con 0x55e2aa3747a0 2023-09-13T03:14:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.991+0000 7fd1c7fff700 1 --2- 172.21.15.64:0/1816923568 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55e2aa3747a0 0x55e2aa376bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.992+0000 7fd1a7fff700 10 client.26123 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:31.992+0000 7fd1c7fff700 1 --2- 172.21.15.64:0/1816923568 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55e2aa3747a0 0x55e2aa376bc0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5aa36384cf293704 server_cookie=bdeddc159e63b05f in_seq=0 out_seq=0 2023-09-13T03:14:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:32.017+0000 7fd1a7fff700 1 -- 172.21.15.64:0/1816923568 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e2aa2518c0 con 0x55e2aa3747a0 2023-09-13T03:14:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:32.017+0000 7fd1a7fff700 10 client.26123 handle_command_reply: tid=0 2023-09-13T03:14:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:32.017+0000 7fd1e2921b80 1 client.26123 shutdown 2023-09-13T03:14:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:32.017+0000 7fd1e2921b80 1 -- 172.21.15.64:0/1816923568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e2aa306cc0 msgr2=0x55e2aa313b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:32.017+0000 7fd1e2921b80 1 --2- 172.21.15.64:0/1816923568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e2aa306cc0 0x55e2aa313b30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc004fd0 tx=0x7fd1bc0038b0 comp rx=0 tx=0).stop 2023-09-13T03:14:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:32.017+0000 7fd1e2921b80 1 -- 172.21.15.64:0/1816923568 shutdown_connections 2023-09-13T03:14:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:32.017+0000 7fd1e2921b80 1 --2- 172.21.15.64:0/1816923568 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55e2aa3747a0 0x55e2aa376bc0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:32.017+0000 7fd1e2921b80 1 --2- 172.21.15.64:0/1816923568 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2aa30c1a0 0x55e2aa372c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:32.017+0000 7fd1a7fff700 0 client.26123 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:32.017+0000 7fd1e2921b80 1 --2- 172.21.15.64:0/1816923568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2aa306300 0x55e2aa3135f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:32.017+0000 7fd1e2921b80 1 --2- 172.21.15.64:0/1816923568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e2aa306cc0 0x55e2aa313b30 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:32.017+0000 7fd1e2921b80 1 -- 172.21.15.64:0/1816923568 >> 172.21.15.64:0/1816923568 conn(0x55e2aa27b030 msgr2=0x55e2aa30b170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:32.018+0000 7fd1e2921b80 1 -- 172.21.15.64:0/1816923568 shutdown_connections 2023-09-13T03:14:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:32.018+0000 7fd1e2921b80 1 -- 172.21.15.64:0/1816923568 wait complete. 2023-09-13T03:14:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:32.018+0000 7fd1e2921b80 20 client.26123 trim_cache size 0 max 16384 2023-09-13T03:14:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:14:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:14:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:14:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:14:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:32.018+0000 7fd1cefff700 1 -- 172.21.15.64:0/689909611 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fd1b80a0950 msgr2=0x7fd1b80a2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:32.018+0000 7fd1cefff700 1 --2- 172.21.15.64:0/689909611 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fd1b80a0950 0x7fd1b80a2e10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd1c00c9390 tx=0x7fd1c0000f40 comp rx=0 tx=0).stop 2023-09-13T03:14:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:32.018+0000 7fd1cefff700 1 -- 172.21.15.64:0/689909611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1c8106660 msgr2=0x7fd1c8196520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:32.018+0000 7fd1cefff700 1 --2- 172.21.15.64:0/689909611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1c8106660 0x7fd1c8196520 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd1c000efc0 tx=0x7fd1c000c3a0 comp rx=0 tx=0).stop 2023-09-13T03:14:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:32.019+0000 7fd1cefff700 1 -- 172.21.15.64:0/689909611 shutdown_connections 2023-09-13T03:14:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:32.019+0000 7fd1cefff700 1 --2- 172.21.15.64:0/689909611 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fd1b80a0950 0x7fd1b80a2e10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:32.019+0000 7fd1cefff700 1 --2- 172.21.15.64:0/689909611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1c8106660 0x7fd1c8196520 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:32.019+0000 7fd1cefff700 1 --2- 172.21.15.64:0/689909611 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1c81005a0 0x7fd1c819c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:32.019+0000 7fd1cefff700 1 --2- 172.21.15.64:0/689909611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1c8100f60 0x7fd1c819cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:32.019+0000 7fd1cefff700 1 -- 172.21.15.64:0/689909611 >> 172.21.15.64:0/689909611 conn(0x7fd1c80fc0b0 msgr2=0x7fd1c80fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:32.019+0000 7fd1cefff700 1 -- 172.21.15.64:0/689909611 shutdown_connections 2023-09-13T03:14:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:32.019+0000 7fd1cefff700 1 -- 172.21.15.64:0/689909611 wait complete. 2023-09-13T03:14:32.887 INFO:tasks.workunit.client.0.smithi064.stdout: 1 26058 4.24 MB/sec warmup 37 sec latency 2031.082 ms 2023-09-13T03:14:32.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:32 smithi170 ceph-mon[134328]: pgmap v44: 97 pgs: 97 active+clean; 29 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 3.4 MiB/s wr, 98 op/s 2023-09-13T03:14:32.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:32 smithi064 ceph-mon[97675]: pgmap v44: 97 pgs: 97 active+clean; 29 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 3.4 MiB/s wr, 98 op/s 2023-09-13T03:14:32.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:32 smithi142 ceph-mon[134082]: pgmap v44: 97 pgs: 97 active+clean; 29 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 3.4 MiB/s wr, 98 op/s 2023-09-13T03:14:33.888 INFO:tasks.workunit.client.0.smithi064.stdout: 1 26058 4.13 MB/sec warmup 38 sec latency 3031.228 ms 2023-09-13T03:14:34.888 INFO:tasks.workunit.client.0.smithi064.stdout: 1 26058 4.02 MB/sec warmup 39 sec latency 4031.392 ms 2023-09-13T03:14:34.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:34 smithi170 ceph-mon[134328]: pgmap v45: 97 pgs: 97 active+clean; 29 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 3.4 MiB/s wr, 81 op/s 2023-09-13T03:14:34.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:34 smithi064 ceph-mon[97675]: pgmap v45: 97 pgs: 97 active+clean; 29 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 3.4 MiB/s wr, 81 op/s 2023-09-13T03:14:34.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:34 smithi142 ceph-mon[134082]: pgmap v45: 97 pgs: 97 active+clean; 29 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 3.4 MiB/s wr, 81 op/s 2023-09-13T03:14:35.888 INFO:tasks.workunit.client.0.smithi064.stdout: 1 26326 4.00 MB/sec warmup 40 sec latency 4914.011 ms 2023-09-13T03:14:36.888 INFO:tasks.workunit.client.0.smithi064.stdout: 1 27276 3.95 MB/sec warmup 41 sec latency 714.872 ms 2023-09-13T03:14:36.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:36 smithi170 ceph-mon[134328]: pgmap v46: 97 pgs: 97 active+clean; 25 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 3.4 MiB/s wr, 95 op/s 2023-09-13T03:14:36.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:36 smithi064 ceph-mon[97675]: pgmap v46: 97 pgs: 97 active+clean; 25 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 3.4 MiB/s wr, 95 op/s 2023-09-13T03:14:36.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:36 smithi142 ceph-mon[134082]: pgmap v46: 97 pgs: 97 active+clean; 25 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 3.4 MiB/s wr, 95 op/s 2023-09-13T03:14:37.889 INFO:tasks.workunit.client.0.smithi064.stdout: 1 27276 3.85 MB/sec warmup 42 sec latency 1715.013 ms 2023-09-13T03:14:38.889 INFO:tasks.workunit.client.0.smithi064.stdout: 1 27304 3.77 MB/sec warmup 43 sec latency 2345.226 ms 2023-09-13T03:14:38.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:38 smithi170 ceph-mon[134328]: pgmap v47: 97 pgs: 97 active+clean; 25 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 1.7 MiB/s wr, 67 op/s 2023-09-13T03:14:38.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:38 smithi064 ceph-mon[97675]: pgmap v47: 97 pgs: 97 active+clean; 25 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 1.7 MiB/s wr, 67 op/s 2023-09-13T03:14:38.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:38 smithi142 ceph-mon[134082]: pgmap v47: 97 pgs: 97 active+clean; 25 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 1.7 MiB/s wr, 67 op/s 2023-09-13T03:14:39.888 INFO:tasks.workunit.client.0.smithi064.stdout: 1 27329 3.68 MB/sec warmup 44 sec latency 1162.762 ms 2023-09-13T03:14:39.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:39 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:14:39.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:39 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:14:39.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:39 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:14:40.888 INFO:tasks.workunit.client.0.smithi064.stdout: 1 27329 3.60 MB/sec warmup 45 sec latency 1143.269 ms 2023-09-13T03:14:40.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:40 smithi170 ceph-mon[134328]: pgmap v48: 97 pgs: 97 active+clean; 25 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 1.1 MiB/s wr, 55 op/s 2023-09-13T03:14:40.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:40 smithi064 ceph-mon[97675]: pgmap v48: 97 pgs: 97 active+clean; 25 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 1.1 MiB/s wr, 55 op/s 2023-09-13T03:14:40.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:40 smithi142 ceph-mon[134082]: pgmap v48: 97 pgs: 97 active+clean; 25 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 1.1 MiB/s wr, 55 op/s 2023-09-13T03:14:41.888 INFO:tasks.workunit.client.0.smithi064.stdout: 1 27329 3.52 MB/sec warmup 46 sec latency 2143.348 ms 2023-09-13T03:14:42.281 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-13T03:14:42.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.499+0000 7ffb049ff700 1 -- 172.21.15.64:0/3263932479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb00106630 msgr2=0x7ffb00106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:42.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.499+0000 7ffb049ff700 1 --2- 172.21.15.64:0/3263932479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb00106630 0x7ffb00106a10 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ffaf00099c0 tx=0x7ffaf0009cd0 comp rx=0 tx=0).stop 2023-09-13T03:14:42.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.499+0000 7ffb049ff700 1 -- 172.21.15.64:0/3263932479 shutdown_connections 2023-09-13T03:14:42.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.499+0000 7ffb049ff700 1 --2- 172.21.15.64:0/3263932479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb00100f30 0x7ffb0010d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:42.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.499+0000 7ffb049ff700 1 --2- 172.21.15.64:0/3263932479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb00106630 0x7ffb00106a10 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:42.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.499+0000 7ffb049ff700 1 --2- 172.21.15.64:0/3263932479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb00100570 0x7ffb001009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:42.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.499+0000 7ffb049ff700 1 -- 172.21.15.64:0/3263932479 >> 172.21.15.64:0/3263932479 conn(0x7ffb000fc080 msgr2=0x7ffb000fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:42.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.500+0000 7ffb049ff700 1 -- 172.21.15.64:0/3263932479 shutdown_connections 2023-09-13T03:14:42.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.500+0000 7ffb049ff700 1 -- 172.21.15.64:0/3263932479 wait complete. 2023-09-13T03:14:42.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.501+0000 7ffb049ff700 1 Processor -- start 2023-09-13T03:14:42.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.501+0000 7ffb049ff700 1 -- start start 2023-09-13T03:14:42.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.501+0000 7ffb049ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb00100570 0x7ffb0019e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:42.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.501+0000 7ffb049ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb00100f30 0x7ffb0019eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:42.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.501+0000 7ffb049ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb00106630 0x7ffb00198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:42.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.501+0000 7ffb049ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb00198ce0 con 0x7ffb00106630 2023-09-13T03:14:42.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.501+0000 7ffb049ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb00198e50 con 0x7ffb00100f30 2023-09-13T03:14:42.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.501+0000 7ffb049ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb00199010 con 0x7ffb00100570 2023-09-13T03:14:42.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.502+0000 7ffafed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb00106630 0x7ffb00198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:42.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.502+0000 7ffafed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb00106630 0x7ffb00198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37304/0 (socket says 172.21.15.64:37304) 2023-09-13T03:14:42.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.502+0000 7ffafed9d700 1 -- 172.21.15.64:0/1946517577 learned_addr learned my addr 172.21.15.64:0/1946517577 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:42.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.502+0000 7ffafe59c700 1 --2- 172.21.15.64:0/1946517577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb00100570 0x7ffb0019e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:42.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.502+0000 7ffaf7fff700 1 --2- 172.21.15.64:0/1946517577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb00100f30 0x7ffb0019eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:42.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.502+0000 7ffafed9d700 1 -- 172.21.15.64:0/1946517577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb00100570 msgr2=0x7ffb0019e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:42.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.502+0000 7ffafed9d700 1 --2- 172.21.15.64:0/1946517577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb00100570 0x7ffb0019e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:42.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.502+0000 7ffafed9d700 1 -- 172.21.15.64:0/1946517577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb00100f30 msgr2=0x7ffb0019eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:42.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.502+0000 7ffafed9d700 1 --2- 172.21.15.64:0/1946517577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb00100f30 0x7ffb0019eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:42.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.502+0000 7ffafed9d700 1 -- 172.21.15.64:0/1946517577 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf00095f0 con 0x7ffb00106630 2023-09-13T03:14:42.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.502+0000 7ffafed9d700 1 --2- 172.21.15.64:0/1946517577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb00106630 0x7ffb00198670 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ffaf800ef30 tx=0x7ffaf800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:42.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.502+0000 7ffaf77fe700 1 -- 172.21.15.64:0/1946517577 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8018070 con 0x7ffb00106630 2023-09-13T03:14:42.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.503+0000 7ffb049ff700 1 -- 172.21.15.64:0/1946517577 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb00199290 con 0x7ffb00106630 2023-09-13T03:14:42.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.503+0000 7ffaf77fe700 1 -- 172.21.15.64:0/1946517577 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffaf800f040 con 0x7ffb00106630 2023-09-13T03:14:42.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.503+0000 7ffaf77fe700 1 -- 172.21.15.64:0/1946517577 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf800cd00 con 0x7ffb00106630 2023-09-13T03:14:42.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.503+0000 7ffb049ff700 1 -- 172.21.15.64:0/1946517577 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb001a55f0 con 0x7ffb00106630 2023-09-13T03:14:42.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.506+0000 7ffaf77fe700 1 -- 172.21.15.64:0/1946517577 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7ffaf8003a00 con 0x7ffb00106630 2023-09-13T03:14:42.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.507+0000 7ffaf77fe700 1 --2- 172.21.15.64:0/1946517577 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7ffaec0a07b0 0x7ffaec0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:42.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.507+0000 7ffb183b9b80 1 Processor -- start 2023-09-13T03:14:42.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.507+0000 7ffaf77fe700 1 -- 172.21.15.64:0/1946517577 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7ffaf80cba40 con 0x7ffb00106630 2023-09-13T03:14:42.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.508+0000 7ffb183b9b80 1 -- start start 2023-09-13T03:14:42.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.508+0000 7ffafe59c700 1 --2- 172.21.15.64:0/1946517577 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7ffaec0a07b0 0x7ffaec0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:42.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.508+0000 7ffb183b9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c8174b710 0x561c8174baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:42.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.508+0000 7ffb183b9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c8174c030 0x561c81755900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:42.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.508+0000 7ffb183b9b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c81755e40 0x561c81758230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:42.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.508+0000 7ffb183b9b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c816a4730 con 0x561c81755e40 2023-09-13T03:14:42.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.508+0000 7ffb183b9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c816a2050 con 0x561c8174b710 2023-09-13T03:14:42.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.508+0000 7ffb183b9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c8169d940 con 0x561c8174c030 2023-09-13T03:14:42.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.508+0000 7ffafe59c700 1 --2- 172.21.15.64:0/1946517577 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7ffaec0a07b0 0x7ffaec0a2c70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0009fd0 tx=0x7ffaf0005fb0 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:42.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.508+0000 7ffaf7fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c8174c030 0x561c81755900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:42.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.508+0000 7ffaf7fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c8174c030 0x561c81755900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:47336/0 (socket says 172.21.15.64:47336) 2023-09-13T03:14:42.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.508+0000 7ffaf7fff700 1 -- 172.21.15.64:0/2001507574 learned_addr learned my addr 172.21.15.64:0/2001507574 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:42.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.509+0000 7ffafe59c700 1 --2- 172.21.15.64:0/2001507574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c8174b710 0x561c8174baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:42.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.509+0000 7ffafed9d700 1 --2- 172.21.15.64:0/2001507574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c81755e40 0x561c81758230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:42.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.509+0000 7ffaf7fff700 1 -- 172.21.15.64:0/2001507574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c8174b710 msgr2=0x561c8174baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:42.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.509+0000 7ffaf7fff700 1 --2- 172.21.15.64:0/2001507574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c8174b710 0x561c8174baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:42.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.509+0000 7ffaf7fff700 1 -- 172.21.15.64:0/2001507574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c81755e40 msgr2=0x561c81758230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:42.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.509+0000 7ffaf7fff700 1 --2- 172.21.15.64:0/2001507574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c81755e40 0x561c81758230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:42.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.509+0000 7ffaf7fff700 1 -- 172.21.15.64:0/2001507574 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c814c9de0 con 0x561c8174c030 2023-09-13T03:14:42.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.509+0000 7ffaf7fff700 1 --2- 172.21.15.64:0/2001507574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c8174c030 0x561c81755900 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ffae800bd30 tx=0x7ffae80106b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee2730cae54fc21a server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:42.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.509+0000 7ffaf4ff9700 1 -- 172.21.15.64:0/2001507574 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae8010e10 con 0x561c8174c030 2023-09-13T03:14:42.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.509+0000 7ffaf4ff9700 1 -- 172.21.15.64:0/2001507574 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffae8015070 con 0x561c8174c030 2023-09-13T03:14:42.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.509+0000 7ffaf4ff9700 1 -- 172.21.15.64:0/2001507574 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae8003df0 con 0x561c8174c030 2023-09-13T03:14:42.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.510+0000 7ffb183b9b80 1 -- 172.21.15.64:0/2001507574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c8174c030 msgr2=0x561c81755900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:42.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.510+0000 7ffb183b9b80 1 --2- 172.21.15.64:0/2001507574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c8174c030 0x561c81755900 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ffae800bd30 tx=0x7ffae80106b0 comp rx=0 tx=0).stop 2023-09-13T03:14:42.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.510+0000 7ffb183b9b80 1 -- 172.21.15.64:0/2001507574 shutdown_connections 2023-09-13T03:14:42.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.510+0000 7ffb183b9b80 1 --2- 172.21.15.64:0/2001507574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c8174b710 0x561c8174baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:42.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.510+0000 7ffb183b9b80 1 --2- 172.21.15.64:0/2001507574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c81755e40 0x561c81758230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:42.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.510+0000 7ffb183b9b80 1 --2- 172.21.15.64:0/2001507574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c8174c030 0x561c81755900 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:42.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.510+0000 7ffb183b9b80 1 -- 172.21.15.64:0/2001507574 >> 172.21.15.64:0/2001507574 conn(0x561c816c0090 msgr2=0x561c81746bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:42.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.510+0000 7ffb183b9b80 1 -- 172.21.15.64:0/2001507574 shutdown_connections 2023-09-13T03:14:42.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.510+0000 7ffb183b9b80 1 -- 172.21.15.64:0/2001507574 wait complete. 2023-09-13T03:14:42.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.511+0000 7ffb183b9b80 1 -- start start 2023-09-13T03:14:42.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.511+0000 7ffb183b9b80 1 Processor -- start 2023-09-13T03:14:42.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.512+0000 7ffb183b9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c8174b710 0x561c81755840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:42.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.512+0000 7ffb183b9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c8174c030 0x561c81752ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:42.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.512+0000 7ffb183b9b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c81755e40 0x561c817530d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:42.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.512+0000 7ffb183b9b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c8163bb70 con 0x561c81755e40 2023-09-13T03:14:42.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.512+0000 7ffb183b9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c8163b870 con 0x561c8174b710 2023-09-13T03:14:42.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.512+0000 7ffb183b9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c8151f920 con 0x561c8174c030 2023-09-13T03:14:42.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.512+0000 7ffb183b9b80 10 client.0 _ll_register_callbacks cb 0x561c81585a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:14:42.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.512+0000 7ffafed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c81755e40 0x561c817530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:42.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.512+0000 7ffafed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c81755e40 0x561c817530d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37310/0 (socket says 172.21.15.64:37310) 2023-09-13T03:14:42.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.512+0000 7ffafed9d700 1 -- 172.21.15.64:0/4282059797 learned_addr learned my addr 172.21.15.64:0/4282059797 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:42.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.512+0000 7ffafe59c700 1 --2- 172.21.15.64:0/4282059797 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c8174b710 0x561c81755840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:42.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.513+0000 7ffaf7fff700 1 --2- 172.21.15.64:0/4282059797 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c8174c030 0x561c81752ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:42.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.513+0000 7ffafed9d700 1 -- 172.21.15.64:0/4282059797 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c8174c030 msgr2=0x561c81752ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:42.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.513+0000 7ffafed9d700 1 --2- 172.21.15.64:0/4282059797 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c8174c030 0x561c81752ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:42.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.513+0000 7ffafed9d700 1 -- 172.21.15.64:0/4282059797 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c8174b710 msgr2=0x561c81755840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:42.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.513+0000 7ffafed9d700 1 --2- 172.21.15.64:0/4282059797 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c8174b710 0x561c81755840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:42.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.513+0000 7ffafed9d700 1 -- 172.21.15.64:0/4282059797 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffae800b960 con 0x561c81755e40 2023-09-13T03:14:42.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.513+0000 7ffafed9d700 1 --2- 172.21.15.64:0/4282059797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c81755e40 0x561c817530d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8095470 tx=0x7ffaf8098950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=532b4582af153b21 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:42.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.513+0000 7ffae5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:14:42.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.513+0000 7ffae5ffb700 1 -- 172.21.15.64:0/4282059797 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf80b8040 con 0x561c81755e40 2023-09-13T03:14:42.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.513+0000 7ffae5ffb700 1 -- 172.21.15.64:0/4282059797 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7ffaf8077e40 con 0x561c81755e40 2023-09-13T03:14:42.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.513+0000 7ffb183b9b80 1 -- 172.21.15.64:0/4282059797 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561c815f54f0 con 0x561c81755e40 2023-09-13T03:14:42.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.513+0000 7ffae5ffb700 1 -- 172.21.15.64:0/4282059797 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffaf801b320 con 0x561c81755e40 2023-09-13T03:14:42.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.513+0000 7ffb183b9b80 10 client.16095 fetch_fsmap learned FSMap version 32 2023-09-13T03:14:42.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.513+0000 7ffb183b9b80 1 -- 172.21.15.64:0/4282059797 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x561c814b2c60 con 0x561c81755e40 2023-09-13T03:14:42.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.514+0000 7ffae5ffb700 1 -- 172.21.15.64:0/4282059797 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561c815f54f0 con 0x561c81755e40 2023-09-13T03:14:42.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.514+0000 7ffae5ffb700 1 -- 172.21.15.64:0/4282059797 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 32) v1 ==== 7906+0+0 (secure 0 0 0) 0x7ffaf80b10c0 con 0x561c81755e40 2023-09-13T03:14:42.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.514+0000 7ffb183b9b80 10 client.16095 fetch_fsmap finished waiting for FSMap version 32 2023-09-13T03:14:42.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.514+0000 7ffb183b9b80 10 client.16095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T03:14:42.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.514+0000 7ffb183b9b80 20 client.16095 populate_metadata read hostname 'smithi064' 2023-09-13T03:14:42.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.514+0000 7ffb183b9b80 1 --2- 172.21.15.64:0/4282059797 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x561c81765940 0x561c81767d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:42.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.514+0000 7ffb183b9b80 4 client.16095 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:14:42.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.514+0000 7ffb183b9b80 1 -- 172.21.15.64:0/4282059797 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561c816b28a0 con 0x561c81765940 2023-09-13T03:14:42.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.514+0000 7ffafe59c700 1 --2- 172.21.15.64:0/4282059797 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x561c81765940 0x561c81767d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:42.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.515+0000 7ffafe59c700 1 --2- 172.21.15.64:0/4282059797 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x561c81765940 0x561c81767d60 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c2b3c1b6442230e server_cookie=f499f9a17f439a38 in_seq=0 out_seq=0 2023-09-13T03:14:42.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.515+0000 7ffae5ffb700 10 client.16095 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:42.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.516+0000 7ffae5ffb700 1 -- 172.21.15.64:0/4282059797 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561c816b28a0 con 0x561c81765940 2023-09-13T03:14:42.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.516+0000 7ffae5ffb700 10 client.16095 handle_command_reply: tid=0 2023-09-13T03:14:42.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.516+0000 7ffb183b9b80 1 client.16095 shutdown 2023-09-13T03:14:42.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.516+0000 7ffb183b9b80 1 -- 172.21.15.64:0/4282059797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c81755e40 msgr2=0x561c817530d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:42.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.516+0000 7ffb183b9b80 1 --2- 172.21.15.64:0/4282059797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c81755e40 0x561c817530d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8095470 tx=0x7ffaf8098950 comp rx=0 tx=0).stop 2023-09-13T03:14:42.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.516+0000 7ffb183b9b80 1 -- 172.21.15.64:0/4282059797 shutdown_connections 2023-09-13T03:14:42.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.516+0000 7ffb183b9b80 1 --2- 172.21.15.64:0/4282059797 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c8174b710 0x561c81755840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:42.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.516+0000 7ffb183b9b80 1 --2- 172.21.15.64:0/4282059797 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x561c81765940 0x561c81767d60 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:42.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.516+0000 7ffb183b9b80 1 --2- 172.21.15.64:0/4282059797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c81755e40 0x561c817530d0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:42.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.516+0000 7ffb183b9b80 1 --2- 172.21.15.64:0/4282059797 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c8174c030 0x561c81752ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:42.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.516+0000 7ffae5ffb700 0 client.16095 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:42.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.516+0000 7ffb183b9b80 1 -- 172.21.15.64:0/4282059797 >> 172.21.15.64:0/4282059797 conn(0x561c816c0090 msgr2=0x561c817467e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:42.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.516+0000 7ffb183b9b80 1 -- 172.21.15.64:0/4282059797 shutdown_connections 2023-09-13T03:14:42.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.516+0000 7ffb183b9b80 1 -- 172.21.15.64:0/4282059797 wait complete. 2023-09-13T03:14:42.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.517+0000 7ffb183b9b80 20 client.16095 trim_cache size 0 max 16384 2023-09-13T03:14:42.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.534+0000 7ffb183b9b80 1 Processor -- start 2023-09-13T03:14:42.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.534+0000 7ffb183b9b80 1 -- start start 2023-09-13T03:14:42.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.535+0000 7ffb183b9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c8174b710 0x561c8175c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:42.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.535+0000 7ffb183b9b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c8174c030 0x561c81755370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:42.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.535+0000 7ffb183b9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c81755e40 0x561c8174de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:42.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.535+0000 7ffb183b9b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c81753e10 con 0x561c8174c030 2023-09-13T03:14:42.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.535+0000 7ffb183b9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c8174d280 con 0x561c81755e40 2023-09-13T03:14:42.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.535+0000 7ffb183b9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c8175cf30 con 0x561c8174b710 2023-09-13T03:14:42.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.535+0000 7ffaf7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c8174c030 0x561c81755370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:42.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.535+0000 7ffaf7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c8174c030 0x561c81755370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37316/0 (socket says 172.21.15.64:37316) 2023-09-13T03:14:42.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.535+0000 7ffaf7fff700 1 -- 172.21.15.64:0/1177169557 learned_addr learned my addr 172.21.15.64:0/1177169557 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:42.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.535+0000 7ffafe59c700 1 --2- 172.21.15.64:0/1177169557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c8174b710 0x561c8175c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:42.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.535+0000 7ffafed9d700 1 --2- 172.21.15.64:0/1177169557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c81755e40 0x561c8174de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:42.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.535+0000 7ffaf7fff700 1 -- 172.21.15.64:0/1177169557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c8174b710 msgr2=0x561c8175c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:42.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.535+0000 7ffaf7fff700 1 --2- 172.21.15.64:0/1177169557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c8174b710 0x561c8175c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:42.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.535+0000 7ffaf7fff700 1 -- 172.21.15.64:0/1177169557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c81755e40 msgr2=0x561c8174de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:42.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.535+0000 7ffaf7fff700 1 --2- 172.21.15.64:0/1177169557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c81755e40 0x561c8174de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:42.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.535+0000 7ffaf7fff700 1 -- 172.21.15.64:0/1177169557 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c814f0250 con 0x561c8174c030 2023-09-13T03:14:42.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.536+0000 7ffaf7fff700 1 --2- 172.21.15.64:0/1177169557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c8174c030 0x561c81755370 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ffae801a000 tx=0x7ffae800b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e85933f8cfafd2 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:42.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.536+0000 7ffaf57fa700 1 -- 172.21.15.64:0/1177169557 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae8010e10 con 0x561c8174c030 2023-09-13T03:14:42.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.536+0000 7ffaf57fa700 1 -- 172.21.15.64:0/1177169557 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffae8021040 con 0x561c8174c030 2023-09-13T03:14:42.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.536+0000 7ffaf57fa700 1 -- 172.21.15.64:0/1177169557 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae8003b50 con 0x561c8174c030 2023-09-13T03:14:42.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.536+0000 7ffb183b9b80 1 -- 172.21.15.64:0/1177169557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c8174c030 msgr2=0x561c81755370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:42.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.536+0000 7ffb183b9b80 1 --2- 172.21.15.64:0/1177169557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c8174c030 0x561c81755370 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ffae801a000 tx=0x7ffae800b710 comp rx=0 tx=0).stop 2023-09-13T03:14:42.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.537+0000 7ffb183b9b80 1 -- 172.21.15.64:0/1177169557 shutdown_connections 2023-09-13T03:14:42.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.537+0000 7ffb183b9b80 1 --2- 172.21.15.64:0/1177169557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c81755e40 0x561c8174de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:42.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.537+0000 7ffb183b9b80 1 --2- 172.21.15.64:0/1177169557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c8174b710 0x561c8175c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:42.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.537+0000 7ffb183b9b80 1 --2- 172.21.15.64:0/1177169557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c8174c030 0x561c81755370 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:42.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.537+0000 7ffb183b9b80 1 -- 172.21.15.64:0/1177169557 >> 172.21.15.64:0/1177169557 conn(0x561c816c0090 msgr2=0x561c81761140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:42.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.537+0000 7ffb183b9b80 1 -- 172.21.15.64:0/1177169557 shutdown_connections 2023-09-13T03:14:42.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.537+0000 7ffb183b9b80 1 -- 172.21.15.64:0/1177169557 wait complete. 2023-09-13T03:14:42.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.537+0000 7ffb183b9b80 1 -- start start 2023-09-13T03:14:42.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.537+0000 7ffb183b9b80 1 Processor -- start 2023-09-13T03:14:42.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.538+0000 7ffb183b9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c8174b710 0x561c8175b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:42.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.538+0000 7ffb183b9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c8174c030 0x561c8175b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:42.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.538+0000 7ffb183b9b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c81755e40 0x561c8175ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:42.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.538+0000 7ffb183b9b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c817b9870 con 0x561c81755e40 2023-09-13T03:14:42.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.538+0000 7ffb183b9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c817b99b0 con 0x561c8174b710 2023-09-13T03:14:42.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.538+0000 7ffb183b9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c817b9b70 con 0x561c8174c030 2023-09-13T03:14:42.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.538+0000 7ffb183b9b80 10 client.0 _ll_register_callbacks cb 0x7ffaf8098e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:14:42.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.538+0000 7ffafed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c81755e40 0x561c8175ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:42.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.538+0000 7ffafed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c81755e40 0x561c8175ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37324/0 (socket says 172.21.15.64:37324) 2023-09-13T03:14:42.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.538+0000 7ffafed9d700 1 -- 172.21.15.64:0/179338486 learned_addr learned my addr 172.21.15.64:0/179338486 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:42.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.538+0000 7ffafe59c700 1 --2- 172.21.15.64:0/179338486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c8174b710 0x561c8175b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:42.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.539+0000 7ffaf7fff700 1 --2- 172.21.15.64:0/179338486 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c8174c030 0x561c8175b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:42.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.539+0000 7ffafed9d700 1 -- 172.21.15.64:0/179338486 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c8174c030 msgr2=0x561c8175b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:42.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.539+0000 7ffafed9d700 1 --2- 172.21.15.64:0/179338486 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c8174c030 0x561c8175b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:42.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.539+0000 7ffafed9d700 1 -- 172.21.15.64:0/179338486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c8174b710 msgr2=0x561c8175b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:42.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.539+0000 7ffafed9d700 1 --2- 172.21.15.64:0/179338486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c8174b710 0x561c8175b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:42.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.539+0000 7ffafed9d700 1 -- 172.21.15.64:0/179338486 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561c814c9de0 con 0x561c81755e40 2023-09-13T03:14:42.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.539+0000 7ffafe59c700 1 --2- 172.21.15.64:0/179338486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c8174b710 0x561c8175b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:42.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.539+0000 7ffafed9d700 1 --2- 172.21.15.64:0/179338486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c81755e40 0x561c8175ba90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8098460 tx=0x7ffaf8094c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c031da6113907a server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:42.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.539+0000 7ffae5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:14:42.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.539+0000 7ffae5ffb700 1 -- 172.21.15.64:0/179338486 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf80b8040 con 0x561c81755e40 2023-09-13T03:14:42.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.539+0000 7ffb183b9b80 1 -- 172.21.15.64:0/179338486 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffae8015070 con 0x561c81755e40 2023-09-13T03:14:42.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.539+0000 7ffae5ffb700 1 -- 172.21.15.64:0/179338486 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7ffaf8077e40 con 0x561c81755e40 2023-09-13T03:14:42.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.540+0000 7ffae5ffb700 1 -- 172.21.15.64:0/179338486 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffaf80b1260 con 0x561c81755e40 2023-09-13T03:14:42.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.540+0000 7ffb183b9b80 10 client.16101 fetch_fsmap learned FSMap version 32 2023-09-13T03:14:42.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.540+0000 7ffb183b9b80 1 -- 172.21.15.64:0/179338486 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x561c814c82b0 con 0x561c81755e40 2023-09-13T03:14:42.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.540+0000 7ffae5ffb700 1 -- 172.21.15.64:0/179338486 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffae8015070 con 0x561c81755e40 2023-09-13T03:14:42.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.540+0000 7ffae5ffb700 1 -- 172.21.15.64:0/179338486 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 32) v1 ==== 7906+0+0 (secure 0 0 0) 0x7ffaf807dbd0 con 0x561c81755e40 2023-09-13T03:14:42.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.540+0000 7ffb183b9b80 10 client.16101 fetch_fsmap finished waiting for FSMap version 32 2023-09-13T03:14:42.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.540+0000 7ffb183b9b80 10 client.16101 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T03:14:42.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.540+0000 7ffb183b9b80 20 client.16101 populate_metadata read hostname 'smithi064' 2023-09-13T03:14:42.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.540+0000 7ffb183b9b80 1 --2- 172.21.15.64:0/179338486 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x561c817bafc0 0x561c817bd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:42.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.540+0000 7ffb183b9b80 4 client.16101 mds_command: new command op to 24686 tid=0[{"prefix": "scrub status"}] 2023-09-13T03:14:42.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.540+0000 7ffb183b9b80 1 -- 172.21.15.64:0/179338486 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561c81696920 con 0x561c817bafc0 2023-09-13T03:14:42.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.540+0000 7ffafe59c700 1 --2- 172.21.15.64:0/179338486 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x561c817bafc0 0x561c817bd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:42.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.541+0000 7ffafe59c700 1 --2- 172.21.15.64:0/179338486 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x561c817bafc0 0x561c817bd3e0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f686db84456ebc4 server_cookie=d9744c375ea1de16 in_seq=0 out_seq=0 2023-09-13T03:14:42.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.541+0000 7ffae5ffb700 10 client.16101 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:42.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.541+0000 7ffae5ffb700 1 -- 172.21.15.64:0/179338486 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561c81696920 con 0x561c817bafc0 2023-09-13T03:14:42.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.541+0000 7ffae5ffb700 10 client.16101 handle_command_reply: tid=0 2023-09-13T03:14:42.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.541+0000 7ffb183b9b80 1 client.16101 shutdown 2023-09-13T03:14:42.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.541+0000 7ffb183b9b80 1 -- 172.21.15.64:0/179338486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c81755e40 msgr2=0x561c8175ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:42.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.542+0000 7ffb183b9b80 1 --2- 172.21.15.64:0/179338486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c81755e40 0x561c8175ba90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8098460 tx=0x7ffaf8094c00 comp rx=0 tx=0).stop 2023-09-13T03:14:42.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.542+0000 7ffb183b9b80 1 -- 172.21.15.64:0/179338486 shutdown_connections 2023-09-13T03:14:42.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.542+0000 7ffb183b9b80 1 --2- 172.21.15.64:0/179338486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c8174b710 0x561c8175b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:42.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.542+0000 7ffb183b9b80 1 --2- 172.21.15.64:0/179338486 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x561c817bafc0 0x561c817bd3e0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:42.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.542+0000 7ffb183b9b80 1 --2- 172.21.15.64:0/179338486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c81755e40 0x561c8175ba90 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:42.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.542+0000 7ffb183b9b80 1 --2- 172.21.15.64:0/179338486 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c8174c030 0x561c8175b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:42.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.542+0000 7ffb183b9b80 1 -- 172.21.15.64:0/179338486 >> 172.21.15.64:0/179338486 conn(0x561c816c0090 msgr2=0x561c817522f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:42.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.542+0000 7ffae5ffb700 0 client.16101 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:42.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.542+0000 7ffb183b9b80 1 -- 172.21.15.64:0/179338486 shutdown_connections 2023-09-13T03:14:42.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.542+0000 7ffb183b9b80 1 -- 172.21.15.64:0/179338486 wait complete. 2023-09-13T03:14:42.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.542+0000 7ffb183b9b80 20 client.16101 trim_cache size 0 max 16384 2023-09-13T03:14:42.556 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-13T03:14:42.556 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2023-09-13T03:14:42.557 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {} 2023-09-13T03:14:42.557 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-13T03:14:42.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.542+0000 7ffb049ff700 1 -- 172.21.15.64:0/1946517577 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7ffaec0a07b0 msgr2=0x7ffaec0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:42.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.542+0000 7ffb049ff700 1 --2- 172.21.15.64:0/1946517577 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7ffaec0a07b0 0x7ffaec0a2c70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0009fd0 tx=0x7ffaf0005fb0 comp rx=0 tx=0).stop 2023-09-13T03:14:42.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.542+0000 7ffb049ff700 1 -- 172.21.15.64:0/1946517577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb00106630 msgr2=0x7ffb00198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:42.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.542+0000 7ffb049ff700 1 --2- 172.21.15.64:0/1946517577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb00106630 0x7ffb00198670 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ffaf800ef30 tx=0x7ffaf800c3a0 comp rx=0 tx=0).stop 2023-09-13T03:14:42.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.543+0000 7ffb049ff700 1 -- 172.21.15.64:0/1946517577 shutdown_connections 2023-09-13T03:14:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.543+0000 7ffb049ff700 1 --2- 172.21.15.64:0/1946517577 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7ffaec0a07b0 0x7ffaec0a2c70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.543+0000 7ffb049ff700 1 --2- 172.21.15.64:0/1946517577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb00106630 0x7ffb00198670 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.543+0000 7ffb049ff700 1 --2- 172.21.15.64:0/1946517577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb00100570 0x7ffb0019e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.543+0000 7ffb049ff700 1 --2- 172.21.15.64:0/1946517577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb00100f30 0x7ffb0019eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.543+0000 7ffb049ff700 1 -- 172.21.15.64:0/1946517577 >> 172.21.15.64:0/1946517577 conn(0x7ffb000fc080 msgr2=0x7ffb00072830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:42.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.543+0000 7ffb049ff700 1 -- 172.21.15.64:0/1946517577 shutdown_connections 2023-09-13T03:14:42.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.543+0000 7ffb049ff700 1 -- 172.21.15.64:0/1946517577 wait complete. 2023-09-13T03:14:42.561 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-09-13T03:14:42.561 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T03:14:42.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.768+0000 7fe4817ff700 1 -- 172.21.15.64:0/1070456442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe47c106650 msgr2=0x7fe47c106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:42.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.768+0000 7fe4817ff700 1 --2- 172.21.15.64:0/1070456442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe47c106650 0x7fe47c106a30 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe46c0099c0 tx=0x7fe46c009cd0 comp rx=0 tx=0).stop 2023-09-13T03:14:42.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.768+0000 7fe4817ff700 1 -- 172.21.15.64:0/1070456442 shutdown_connections 2023-09-13T03:14:42.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.768+0000 7fe4817ff700 1 --2- 172.21.15.64:0/1070456442 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe47c100f50 0x7fe47c10d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:42.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.768+0000 7fe4817ff700 1 --2- 172.21.15.64:0/1070456442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe47c106650 0x7fe47c106a30 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:42.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.768+0000 7fe4817ff700 1 --2- 172.21.15.64:0/1070456442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe47c100590 0x7fe47c100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:42.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.768+0000 7fe4817ff700 1 -- 172.21.15.64:0/1070456442 >> 172.21.15.64:0/1070456442 conn(0x7fe47c0fc020 msgr2=0x7fe47c0fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:42.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.768+0000 7fe4817ff700 1 -- 172.21.15.64:0/1070456442 shutdown_connections 2023-09-13T03:14:42.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.768+0000 7fe4817ff700 1 -- 172.21.15.64:0/1070456442 wait complete. 2023-09-13T03:14:42.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.769+0000 7fe4817ff700 1 Processor -- start 2023-09-13T03:14:42.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.769+0000 7fe4817ff700 1 -- start start 2023-09-13T03:14:42.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.770+0000 7fe4817ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe47c100590 0x7fe47c19e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:42.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.770+0000 7fe4817ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe47c100f50 0x7fe47c19eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:42.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.770+0000 7fe4817ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe47c106650 0x7fe47c1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:42.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.770+0000 7fe4817ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47c198d40 con 0x7fe47c106650 2023-09-13T03:14:42.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.770+0000 7fe4817ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47c198eb0 con 0x7fe47c100590 2023-09-13T03:14:42.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.770+0000 7fe4817ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47c199070 con 0x7fe47c100f50 2023-09-13T03:14:42.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.770+0000 7fe47b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe47c106650 0x7fe47c1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:42.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.770+0000 7fe47b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe47c106650 0x7fe47c1986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37350/0 (socket says 172.21.15.64:37350) 2023-09-13T03:14:42.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.770+0000 7fe47b7fe700 1 -- 172.21.15.64:0/2570764870 learned_addr learned my addr 172.21.15.64:0/2570764870 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:42.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.770+0000 7fe47affd700 1 --2- 172.21.15.64:0/2570764870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe47c100590 0x7fe47c19e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:42.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.770+0000 7fe473fff700 1 --2- 172.21.15.64:0/2570764870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe47c100f50 0x7fe47c19eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:42.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.770+0000 7fe47b7fe700 1 -- 172.21.15.64:0/2570764870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe47c100f50 msgr2=0x7fe47c19eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:42.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.770+0000 7fe47b7fe700 1 --2- 172.21.15.64:0/2570764870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe47c100f50 0x7fe47c19eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:42.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.770+0000 7fe47b7fe700 1 -- 172.21.15.64:0/2570764870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe47c100590 msgr2=0x7fe47c19e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:42.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.770+0000 7fe47b7fe700 1 --2- 172.21.15.64:0/2570764870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe47c100590 0x7fe47c19e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:42.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.770+0000 7fe47b7fe700 1 -- 172.21.15.64:0/2570764870 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe46c0095f0 con 0x7fe47c106650 2023-09-13T03:14:42.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.770+0000 7fe47affd700 1 --2- 172.21.15.64:0/2570764870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe47c100590 0x7fe47c19e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:42.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.771+0000 7fe47b7fe700 1 --2- 172.21.15.64:0/2570764870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe47c106650 0x7fe47c1986d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe474007ee0 tx=0x7fe47400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:42.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.771+0000 7fe478ff9700 1 -- 172.21.15.64:0/2570764870 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47400dd20 con 0x7fe47c106650 2023-09-13T03:14:42.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.771+0000 7fe478ff9700 1 -- 172.21.15.64:0/2570764870 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe47400f040 con 0x7fe47c106650 2023-09-13T03:14:42.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.771+0000 7fe4817ff700 1 -- 172.21.15.64:0/2570764870 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe47c199350 con 0x7fe47c106650 2023-09-13T03:14:42.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.771+0000 7fe478ff9700 1 -- 172.21.15.64:0/2570764870 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4740048a0 con 0x7fe47c106650 2023-09-13T03:14:42.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.771+0000 7fe4817ff700 1 -- 172.21.15.64:0/2570764870 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe47c1a55b0 con 0x7fe47c106650 2023-09-13T03:14:42.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.772+0000 7fe4817ff700 1 -- 172.21.15.64:0/2570764870 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe47c0673f0 con 0x7fe47c106650 2023-09-13T03:14:42.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.776+0000 7fe478ff9700 1 -- 172.21.15.64:0/2570764870 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fe474003a00 con 0x7fe47c106650 2023-09-13T03:14:42.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.777+0000 7fe478ff9700 1 --2- 172.21.15.64:0/2570764870 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fe4680a07f0 0x7fe4680a2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:42.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.777+0000 7fe478ff9700 1 -- 172.21.15.64:0/2570764870 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe4740cbfb0 con 0x7fe47c106650 2023-09-13T03:14:42.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.778+0000 7fe47affd700 1 --2- 172.21.15.64:0/2570764870 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fe4680a07f0 0x7fe4680a2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:42.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.778+0000 7fe47affd700 1 --2- 172.21.15.64:0/2570764870 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fe4680a07f0 0x7fe4680a2cb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe46c009990 tx=0x7fe46c005fb0 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:42.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:42.780+0000 7fe478ff9700 1 -- 172.21.15.64:0/2570764870 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe474095790 con 0x7fe47c106650 2023-09-13T03:14:42.888 INFO:tasks.workunit.client.0.smithi064.stdout: 1 27329 3.45 MB/sec warmup 47 sec latency 3143.416 ms 2023-09-13T03:14:42.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:42 smithi064 ceph-mon[97675]: pgmap v49: 97 pgs: 97 active+clean; 25 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 1.1 MiB/s wr, 72 op/s 2023-09-13T03:14:42.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:42 smithi064 ceph-mon[97675]: mgrmap e26: x(active, since 92s), standbys: y, z 2023-09-13T03:14:42.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:42 smithi142 ceph-mon[134082]: pgmap v49: 97 pgs: 97 active+clean; 25 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 1.1 MiB/s wr, 72 op/s 2023-09-13T03:14:42.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:42 smithi142 ceph-mon[134082]: mgrmap e26: x(active, since 92s), standbys: y, z 2023-09-13T03:14:43.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.036+0000 7fe4817ff700 1 -- 172.21.15.64:0/2570764870 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe47c199be0 con 0x7fe47c106650 2023-09-13T03:14:43.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.037+0000 7fe478ff9700 1 -- 172.21.15.64:0/2570764870 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+11377 (secure 0 0 0) 0x7fe474098db0 con 0x7fe47c106650 2023-09-13T03:14:43.037 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:14:43.038 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2234182985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2234182985},{"type":"v1","addr":"172.21.15.64:6835","nonce":2234182985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4053841873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4053841873},{"type":"v1","addr":"172.21.15.64:6837","nonce":4053841873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-13T03:11:10.490260+0000","modified":"2023-09-13T03:14:37.968369+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24686,"mds_1":24649,"mds_2":24635,"mds_3":24634,"mds_4":24622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6839/99523374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":99523374},{"type":"v1","addr":"172.21.15.64:6839","nonce":99523374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14649":{"gid":14649,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6841/1866582875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":1866582875},{"type":"v1","addr":"172.21.15.64:6841","nonce":1866582875}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/101906999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":101906999},{"type":"v1","addr":"172.21.15.142:6833","nonce":101906999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24574":{"gid":24574,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/1897366227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1897366227},{"type":"v1","addr":"172.21.15.170:6833","nonce":1897366227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24611":{"gid":24611,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2167779918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2167779918},{"type":"v1","addr":"172.21.15.142:6835","nonce":2167779918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24622":{"gid":24622,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6835/1493371121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1493371121},{"type":"v1","addr":"172.21.15.170:6835","nonce":1493371121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24634":{"gid":24634,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.170:6837/1726587275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":1726587275},{"type":"v1","addr":"172.21.15.170:6837","nonce":1726587275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24635":{"gid":24635,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":52,"addr":"172.21.15.142:6837/741173268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":741173268},{"type":"v1","addr":"172.21.15.142:6837","nonce":741173268}]},"join_fscid":-1,"export_targets":[0,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24649":{"gid":24649,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.170:6839/2189703596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2189703596},{"type":"v1","addr":"172.21.15.170:6839","nonce":2189703596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24686":{"gid":24686,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/3645998616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645998616},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645998616}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-13T03:14:43.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.039+0000 7fe4817ff700 1 -- 172.21.15.64:0/2570764870 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fe4680a07f0 msgr2=0x7fe4680a2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:43.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.039+0000 7fe4817ff700 1 --2- 172.21.15.64:0/2570764870 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fe4680a07f0 0x7fe4680a2cb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe46c009990 tx=0x7fe46c005fb0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.039+0000 7fe4817ff700 1 -- 172.21.15.64:0/2570764870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe47c106650 msgr2=0x7fe47c1986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:43.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.039+0000 7fe4817ff700 1 --2- 172.21.15.64:0/2570764870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe47c106650 0x7fe47c1986d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe474007ee0 tx=0x7fe47400d3b0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.039+0000 7fe4817ff700 1 -- 172.21.15.64:0/2570764870 shutdown_connections 2023-09-13T03:14:43.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.039+0000 7fe4817ff700 1 --2- 172.21.15.64:0/2570764870 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fe4680a07f0 0x7fe4680a2cb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.039+0000 7fe4817ff700 1 --2- 172.21.15.64:0/2570764870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe47c100590 0x7fe47c19e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.039+0000 7fe4817ff700 1 --2- 172.21.15.64:0/2570764870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe47c106650 0x7fe47c1986d0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.039+0000 7fe4817ff700 1 --2- 172.21.15.64:0/2570764870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe47c100f50 0x7fe47c19eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.039+0000 7fe4817ff700 1 -- 172.21.15.64:0/2570764870 >> 172.21.15.64:0/2570764870 conn(0x7fe47c0fc020 msgr2=0x7fe47c0fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:43.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.040+0000 7fe4817ff700 1 -- 172.21.15.64:0/2570764870 shutdown_connections 2023-09-13T03:14:43.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.040+0000 7fe4817ff700 1 -- 172.21.15.64:0/2570764870 wait complete. 2023-09-13T03:14:43.043 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 32 2023-09-13T03:14:43.059 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-09-13T03:14:43.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:42 smithi170 ceph-mon[134328]: pgmap v49: 97 pgs: 97 active+clean; 25 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 1.1 MiB/s wr, 72 op/s 2023-09-13T03:14:43.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:42 smithi170 ceph-mon[134328]: mgrmap e26: x(active, since 92s), standbys: y, z 2023-09-13T03:14:43.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.265+0000 7f5e54bff700 1 -- 172.21.15.64:0/3444412194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e50100570 msgr2=0x7f5e501009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:43.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.265+0000 7f5e54bff700 1 --2- 172.21.15.64:0/3444412194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e50100570 0x7f5e501009f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5e440099c0 tx=0x7f5e44009cd0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.265+0000 7f5e54bff700 1 -- 172.21.15.64:0/3444412194 shutdown_connections 2023-09-13T03:14:43.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.265+0000 7f5e54bff700 1 --2- 172.21.15.64:0/3444412194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e50106630 0x7f5e50106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.265+0000 7f5e54bff700 1 --2- 172.21.15.64:0/3444412194 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e50100f30 0x7f5e5010d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.265+0000 7f5e54bff700 1 --2- 172.21.15.64:0/3444412194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e50100570 0x7f5e501009f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.265+0000 7f5e54bff700 1 -- 172.21.15.64:0/3444412194 >> 172.21.15.64:0/3444412194 conn(0x7f5e500fc080 msgr2=0x7f5e500fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:43.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.265+0000 7f5e54bff700 1 -- 172.21.15.64:0/3444412194 shutdown_connections 2023-09-13T03:14:43.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.265+0000 7f5e54bff700 1 -- 172.21.15.64:0/3444412194 wait complete. 2023-09-13T03:14:43.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.266+0000 7f5e54bff700 1 Processor -- start 2023-09-13T03:14:43.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.266+0000 7f5e54bff700 1 -- start start 2023-09-13T03:14:43.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.266+0000 7f5e54bff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e50100570 0x7f5e50196610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.266+0000 7f5e54bff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e50100f30 0x7f5e50196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.266+0000 7f5e54bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e50106630 0x7f5e5019afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.266+0000 7f5e54bff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5019b5f0 con 0x7f5e50100f30 2023-09-13T03:14:43.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.266+0000 7f5e54bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5019b760 con 0x7f5e50106630 2023-09-13T03:14:43.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.266+0000 7f5e54bff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5019b920 con 0x7f5e50100570 2023-09-13T03:14:43.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.267+0000 7f5e4dd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e50100f30 0x7f5e50196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.267+0000 7f5e4dd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e50100f30 0x7f5e50196b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37366/0 (socket says 172.21.15.64:37366) 2023-09-13T03:14:43.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.267+0000 7f5e4dd9b700 1 -- 172.21.15.64:0/3286049713 learned_addr learned my addr 172.21.15.64:0/3286049713 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:43.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.267+0000 7f5e4ed9d700 1 --2- 172.21.15.64:0/3286049713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e50106630 0x7f5e5019afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.267+0000 7f5e4e59c700 1 --2- 172.21.15.64:0/3286049713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e50100570 0x7f5e50196610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.267+0000 7f5e4dd9b700 1 -- 172.21.15.64:0/3286049713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e50100570 msgr2=0x7f5e50196610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:43.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.267+0000 7f5e4dd9b700 1 --2- 172.21.15.64:0/3286049713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e50100570 0x7f5e50196610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.267+0000 7f5e4dd9b700 1 -- 172.21.15.64:0/3286049713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e50106630 msgr2=0x7f5e5019afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:43.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.267+0000 7f5e4dd9b700 1 --2- 172.21.15.64:0/3286049713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e50106630 0x7f5e5019afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.267+0000 7f5e4dd9b700 1 -- 172.21.15.64:0/3286049713 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e440095f0 con 0x7f5e50100f30 2023-09-13T03:14:43.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.267+0000 7f5e4dd9b700 1 --2- 172.21.15.64:0/3286049713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e50100f30 0x7f5e50196b50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5e4400b580 tx=0x7f5e4400bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:43.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.268+0000 7f5e3f7fe700 1 -- 172.21.15.64:0/3286049713 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4401d070 con 0x7f5e50100f30 2023-09-13T03:14:43.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.268+0000 7f5e3f7fe700 1 -- 172.21.15.64:0/3286049713 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e44005050 con 0x7f5e50100f30 2023-09-13T03:14:43.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.268+0000 7f5e54bff700 1 -- 172.21.15.64:0/3286049713 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e5019bba0 con 0x7f5e50100f30 2023-09-13T03:14:43.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.268+0000 7f5e3f7fe700 1 -- 172.21.15.64:0/3286049713 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e44004a40 con 0x7f5e50100f30 2023-09-13T03:14:43.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.268+0000 7f5e54bff700 1 -- 172.21.15.64:0/3286049713 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e501a3440 con 0x7f5e50100f30 2023-09-13T03:14:43.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.273+0000 7f5e3f7fe700 1 -- 172.21.15.64:0/3286049713 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f5e440051c0 con 0x7f5e50100f30 2023-09-13T03:14:43.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.273+0000 7f5e684c1b80 1 Processor -- start 2023-09-13T03:14:43.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.273+0000 7f5e684c1b80 1 -- start start 2023-09-13T03:14:43.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.274+0000 7f5e684c1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d99e54e710 0x55d99e54eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.274+0000 7f5e684c1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d99e54f030 0x55d99e558900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.274+0000 7f5e684c1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d99e558e40 0x55d99e55b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.274+0000 7f5e684c1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d99e4a7730 con 0x55d99e558e40 2023-09-13T03:14:43.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.274+0000 7f5e684c1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d99e4a5050 con 0x55d99e54e710 2023-09-13T03:14:43.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.274+0000 7f5e684c1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d99e4a0940 con 0x55d99e54f030 2023-09-13T03:14:43.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.274+0000 7f5e4ed9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d99e54e710 0x55d99e54eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.274+0000 7f5e4dd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d99e558e40 0x55d99e55b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.274+0000 7f5e4ed9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d99e54e710 0x55d99e54eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:36302/0 (socket says 172.21.15.64:36302) 2023-09-13T03:14:43.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.274+0000 7f5e4ed9d700 1 -- 172.21.15.64:0/4127701964 learned_addr learned my addr 172.21.15.64:0/4127701964 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:43.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.274+0000 7f5e3f7fe700 1 --2- 172.21.15.64:0/3286049713 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f5e380a0830 0x7f5e380a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.274+0000 7f5e4e59c700 1 --2- 172.21.15.64:0/4127701964 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d99e54f030 0x55d99e558900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.275+0000 7f5e3f7fe700 1 -- 172.21.15.64:0/3286049713 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f5e440ccc40 con 0x7f5e50100f30 2023-09-13T03:14:43.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.275+0000 7f5e4ed9d700 1 --2- 172.21.15.64:0/3286049713 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f5e380a0830 0x7f5e380a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.275+0000 7f5e4dd9b700 1 -- 172.21.15.64:0/4127701964 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d99e54f030 msgr2=0x55d99e558900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:43.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.275+0000 7f5e4dd9b700 1 --2- 172.21.15.64:0/4127701964 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d99e54f030 0x55d99e558900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.275+0000 7f5e4dd9b700 1 -- 172.21.15.64:0/4127701964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d99e54e710 msgr2=0x55d99e54eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:43.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.275+0000 7f5e4dd9b700 1 --2- 172.21.15.64:0/4127701964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d99e54e710 0x55d99e54eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.275+0000 7f5e4dd9b700 1 -- 172.21.15.64:0/4127701964 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d99e2ccde0 con 0x55d99e558e40 2023-09-13T03:14:43.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.275+0000 7f5e4dd9b700 1 --2- 172.21.15.64:0/4127701964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d99e558e40 0x55d99e55b230 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5e440ca3e0 tx=0x7f5e44009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=522c3fe7e57f8bf3 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:43.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.275+0000 7f5e3cff9700 1 -- 172.21.15.64:0/4127701964 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e440a00f0 con 0x55d99e558e40 2023-09-13T03:14:43.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.275+0000 7f5e4ed9d700 1 --2- 172.21.15.64:0/3286049713 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f5e380a0830 0x7f5e380a2cf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5e4800fb80 tx=0x7f5e48004b40 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:43.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.275+0000 7f5e3cff9700 1 -- 172.21.15.64:0/4127701964 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e4409a8d0 con 0x55d99e558e40 2023-09-13T03:14:43.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.275+0000 7f5e3cff9700 1 -- 172.21.15.64:0/4127701964 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4409a690 con 0x55d99e558e40 2023-09-13T03:14:43.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.276+0000 7f5e684c1b80 1 -- 172.21.15.64:0/4127701964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d99e558e40 msgr2=0x55d99e55b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:43.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.276+0000 7f5e684c1b80 1 --2- 172.21.15.64:0/4127701964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d99e558e40 0x55d99e55b230 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5e440ca3e0 tx=0x7f5e44009310 comp rx=0 tx=0).stop 2023-09-13T03:14:43.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.276+0000 7f5e684c1b80 1 -- 172.21.15.64:0/4127701964 shutdown_connections 2023-09-13T03:14:43.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.276+0000 7f5e684c1b80 1 --2- 172.21.15.64:0/4127701964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d99e54e710 0x55d99e54eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.276+0000 7f5e684c1b80 1 --2- 172.21.15.64:0/4127701964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d99e558e40 0x55d99e55b230 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.276+0000 7f5e684c1b80 1 --2- 172.21.15.64:0/4127701964 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d99e54f030 0x55d99e558900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.276+0000 7f5e684c1b80 1 -- 172.21.15.64:0/4127701964 >> 172.21.15.64:0/4127701964 conn(0x55d99e4c3090 msgr2=0x55d99e549bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:43.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.276+0000 7f5e684c1b80 1 -- 172.21.15.64:0/4127701964 shutdown_connections 2023-09-13T03:14:43.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.276+0000 7f5e684c1b80 1 -- 172.21.15.64:0/4127701964 wait complete. 2023-09-13T03:14:43.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.277+0000 7f5e684c1b80 1 -- start start 2023-09-13T03:14:43.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.277+0000 7f5e684c1b80 1 Processor -- start 2023-09-13T03:14:43.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.278+0000 7f5e684c1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d99e54e710 0x55d99e559220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.278+0000 7f5e684c1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d99e54f030 0x55d99e559760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.278+0000 7f5e684c1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d99e558e40 0x55d99e559ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.278+0000 7f5e684c1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d99e43eb70 con 0x55d99e54e710 2023-09-13T03:14:43.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.278+0000 7f5e684c1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d99e43e870 con 0x55d99e54f030 2023-09-13T03:14:43.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.278+0000 7f5e684c1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d99e322920 con 0x55d99e558e40 2023-09-13T03:14:43.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.278+0000 7f5e684c1b80 10 client.0 _ll_register_callbacks cb 0x55d99e388a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:14:43.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.278+0000 7f5e4e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d99e54e710 0x55d99e559220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.278+0000 7f5e4e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d99e54e710 0x55d99e559220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37378/0 (socket says 172.21.15.64:37378) 2023-09-13T03:14:43.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.278+0000 7f5e4dd9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d99e54f030 0x55d99e559760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.278+0000 7f5e4e59c700 1 -- 172.21.15.64:0/737888889 learned_addr learned my addr 172.21.15.64:0/737888889 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:43.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.278+0000 7f5e4ed9d700 1 --2- 172.21.15.64:0/737888889 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d99e558e40 0x55d99e559ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.278+0000 7f5e4e59c700 1 -- 172.21.15.64:0/737888889 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d99e558e40 msgr2=0x55d99e559ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:43.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.278+0000 7f5e4e59c700 1 --2- 172.21.15.64:0/737888889 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d99e558e40 0x55d99e559ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.278+0000 7f5e4e59c700 1 -- 172.21.15.64:0/737888889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d99e54f030 msgr2=0x55d99e559760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:43.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.278+0000 7f5e4e59c700 1 --2- 172.21.15.64:0/737888889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d99e54f030 0x55d99e559760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.278+0000 7f5e4e59c700 1 -- 172.21.15.64:0/737888889 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e501a3440 con 0x55d99e54e710 2023-09-13T03:14:43.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.279+0000 7f5e4e59c700 1 --2- 172.21.15.64:0/737888889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d99e54e710 0x55d99e559220 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5e40012810 tx=0x7f5e40012b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b00f87cbd6fd66c1 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:43.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.279+0000 7f5e35ffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:14:43.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.279+0000 7f5e35ffb700 1 -- 172.21.15.64:0/737888889 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e40017070 con 0x55d99e54e710 2023-09-13T03:14:43.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.279+0000 7f5e684c1b80 1 -- 172.21.15.64:0/737888889 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d99e3f84f0 con 0x55d99e54e710 2023-09-13T03:14:43.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.279+0000 7f5e35ffb700 1 -- 172.21.15.64:0/737888889 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f5e4000f4e0 con 0x55d99e54e710 2023-09-13T03:14:43.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.279+0000 7f5e35ffb700 1 -- 172.21.15.64:0/737888889 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e4000f6a0 con 0x55d99e54e710 2023-09-13T03:14:43.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.279+0000 7f5e684c1b80 10 client.16119 fetch_fsmap learned FSMap version 32 2023-09-13T03:14:43.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.279+0000 7f5e684c1b80 1 -- 172.21.15.64:0/737888889 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55d99e2b5c60 con 0x55d99e54e710 2023-09-13T03:14:43.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.279+0000 7f5e35ffb700 1 -- 172.21.15.64:0/737888889 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d99e3f84f0 con 0x55d99e54e710 2023-09-13T03:14:43.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.280+0000 7f5e35ffb700 1 -- 172.21.15.64:0/737888889 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 32) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f5e40010870 con 0x55d99e54e710 2023-09-13T03:14:43.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.280+0000 7f5e684c1b80 10 client.16119 fetch_fsmap finished waiting for FSMap version 32 2023-09-13T03:14:43.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.280+0000 7f5e684c1b80 10 client.16119 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2023-09-13T03:14:43.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.280+0000 7f5e684c1b80 20 client.16119 populate_metadata read hostname 'smithi064' 2023-09-13T03:14:43.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.280+0000 7f5e684c1b80 1 --2- 172.21.15.64:0/737888889 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x55d99e5685d0 0x55d99e56a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.280+0000 7f5e684c1b80 4 client.16119 mds_command: new command op to 24622 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:14:43.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.280+0000 7f5e684c1b80 1 -- 172.21.15.64:0/737888889 --> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d99e4b58a0 con 0x55d99e5685d0 2023-09-13T03:14:43.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.280+0000 7f5e4dd9b700 1 --2- 172.21.15.64:0/737888889 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x55d99e5685d0 0x55d99e56a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.281+0000 7f5e4dd9b700 1 --2- 172.21.15.64:0/737888889 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x55d99e5685d0 0x55d99e56a9f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=964168ad677b7052 server_cookie=77b7d9262f453bad in_seq=0 out_seq=0 2023-09-13T03:14:43.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.281+0000 7f5e35ffb700 10 client.16119 ms_handle_connect on v2:172.21.15.170:6834/1493371121 2023-09-13T03:14:43.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.281+0000 7f5e35ffb700 1 -- 172.21.15.64:0/737888889 <== mds.4 v2:172.21.15.170:6834/1493371121 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d99e4b58a0 con 0x55d99e5685d0 2023-09-13T03:14:43.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.281+0000 7f5e35ffb700 10 client.16119 handle_command_reply: tid=0 2023-09-13T03:14:43.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.281+0000 7f5e684c1b80 1 client.16119 shutdown 2023-09-13T03:14:43.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.282+0000 7f5e684c1b80 1 -- 172.21.15.64:0/737888889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d99e54e710 msgr2=0x55d99e559220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:43.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.282+0000 7f5e684c1b80 1 --2- 172.21.15.64:0/737888889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d99e54e710 0x55d99e559220 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5e40012810 tx=0x7f5e40012b20 comp rx=0 tx=0).stop 2023-09-13T03:14:43.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.282+0000 7f5e684c1b80 1 -- 172.21.15.64:0/737888889 shutdown_connections 2023-09-13T03:14:43.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.282+0000 7f5e684c1b80 1 --2- 172.21.15.64:0/737888889 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x55d99e5685d0 0x55d99e56a9f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.282+0000 7f5e684c1b80 1 --2- 172.21.15.64:0/737888889 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d99e558e40 0x55d99e559ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.282+0000 7f5e684c1b80 1 --2- 172.21.15.64:0/737888889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d99e54e710 0x55d99e559220 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.282+0000 7f5e35ffb700 0 client.16119 ms_handle_reset on v2:172.21.15.170:6834/1493371121 2023-09-13T03:14:43.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.282+0000 7f5e684c1b80 1 --2- 172.21.15.64:0/737888889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d99e54f030 0x55d99e559760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.282+0000 7f5e684c1b80 1 -- 172.21.15.64:0/737888889 >> 172.21.15.64:0/737888889 conn(0x55d99e4c3090 msgr2=0x55d99e549780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:43.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.282+0000 7f5e684c1b80 1 -- 172.21.15.64:0/737888889 shutdown_connections 2023-09-13T03:14:43.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.282+0000 7f5e684c1b80 1 -- 172.21.15.64:0/737888889 wait complete. 2023-09-13T03:14:43.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.282+0000 7f5e684c1b80 20 client.16119 trim_cache size 0 max 16384 2023-09-13T03:14:43.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.295+0000 7f5e684c1b80 1 Processor -- start 2023-09-13T03:14:43.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.295+0000 7f5e684c1b80 1 -- start start 2023-09-13T03:14:43.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.295+0000 7f5e684c1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d99e54e710 0x55d99e55dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.296+0000 7f5e684c1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d99e54f030 0x55d99e553240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.296+0000 7f5e684c1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d99e558e40 0x55d99e553780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.296+0000 7f5e684c1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d99e550210 con 0x55d99e54f030 2023-09-13T03:14:43.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.296+0000 7f5e684c1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d99e552d80 con 0x55d99e558e40 2023-09-13T03:14:43.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.296+0000 7f5e684c1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d99e5584e0 con 0x55d99e54e710 2023-09-13T03:14:43.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.296+0000 7f5e4dd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d99e54f030 0x55d99e553240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.296+0000 7f5e4dd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d99e54f030 0x55d99e553240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37386/0 (socket says 172.21.15.64:37386) 2023-09-13T03:14:43.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.296+0000 7f5e4dd9b700 1 -- 172.21.15.64:0/2675396548 learned_addr learned my addr 172.21.15.64:0/2675396548 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:43.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.296+0000 7f5e4e59c700 1 --2- 172.21.15.64:0/2675396548 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d99e54e710 0x55d99e55dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.296+0000 7f5e4ed9d700 1 --2- 172.21.15.64:0/2675396548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d99e558e40 0x55d99e553780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.296+0000 7f5e4dd9b700 1 -- 172.21.15.64:0/2675396548 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d99e54e710 msgr2=0x55d99e55dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:43.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.296+0000 7f5e4dd9b700 1 --2- 172.21.15.64:0/2675396548 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d99e54e710 0x55d99e55dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.296+0000 7f5e4dd9b700 1 -- 172.21.15.64:0/2675396548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d99e558e40 msgr2=0x55d99e553780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:43.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.296+0000 7f5e4dd9b700 1 --2- 172.21.15.64:0/2675396548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d99e558e40 0x55d99e553780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.296+0000 7f5e4dd9b700 1 -- 172.21.15.64:0/2675396548 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d99e2f3250 con 0x55d99e54f030 2023-09-13T03:14:43.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.296+0000 7f5e4e59c700 1 --2- 172.21.15.64:0/2675396548 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d99e54e710 0x55d99e55dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:43.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.296+0000 7f5e4ed9d700 1 --2- 172.21.15.64:0/2675396548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d99e558e40 0x55d99e553780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:14:43.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.296+0000 7f5e4dd9b700 1 --2- 172.21.15.64:0/2675396548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d99e54f030 0x55d99e553240 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5e440966d0 tx=0x7f5e4409a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c74cdf0b725eb8bf server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:43.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.297+0000 7f5e3d7fa700 1 -- 172.21.15.64:0/2675396548 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e440b1070 con 0x55d99e54f030 2023-09-13T03:14:43.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.297+0000 7f5e3d7fa700 1 -- 172.21.15.64:0/2675396548 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e44096510 con 0x55d99e54f030 2023-09-13T03:14:43.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.297+0000 7f5e3d7fa700 1 -- 172.21.15.64:0/2675396548 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e440996c0 con 0x55d99e54f030 2023-09-13T03:14:43.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.297+0000 7f5e684c1b80 1 -- 172.21.15.64:0/2675396548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d99e54f030 msgr2=0x55d99e553240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:43.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.297+0000 7f5e684c1b80 1 --2- 172.21.15.64:0/2675396548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d99e54f030 0x55d99e553240 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5e440966d0 tx=0x7f5e4409a0d0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.297+0000 7f5e684c1b80 1 -- 172.21.15.64:0/2675396548 shutdown_connections 2023-09-13T03:14:43.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.297+0000 7f5e684c1b80 1 --2- 172.21.15.64:0/2675396548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d99e558e40 0x55d99e553780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.297+0000 7f5e684c1b80 1 --2- 172.21.15.64:0/2675396548 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d99e54e710 0x55d99e55dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.297+0000 7f5e684c1b80 1 --2- 172.21.15.64:0/2675396548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d99e54f030 0x55d99e553240 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.297+0000 7f5e684c1b80 1 -- 172.21.15.64:0/2675396548 >> 172.21.15.64:0/2675396548 conn(0x55d99e4c3090 msgr2=0x55d99e559220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:43.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.297+0000 7f5e684c1b80 1 -- 172.21.15.64:0/2675396548 shutdown_connections 2023-09-13T03:14:43.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.297+0000 7f5e684c1b80 1 -- 172.21.15.64:0/2675396548 wait complete. 2023-09-13T03:14:43.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.298+0000 7f5e684c1b80 1 -- start start 2023-09-13T03:14:43.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.298+0000 7f5e684c1b80 1 Processor -- start 2023-09-13T03:14:43.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.298+0000 7f5e684c1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d99e54e710 0x55d99e55d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.298+0000 7f5e684c1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d99e54f030 0x55d99e55d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.298+0000 7f5e684c1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d99e558e40 0x55d99e5598b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.298+0000 7f5e684c1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d99e559f90 con 0x55d99e558e40 2023-09-13T03:14:43.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.298+0000 7f5e684c1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d99e55a100 con 0x55d99e54e710 2023-09-13T03:14:43.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.298+0000 7f5e684c1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d99e55a2c0 con 0x55d99e54f030 2023-09-13T03:14:43.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.298+0000 7f5e684c1b80 10 client.0 _ll_register_callbacks cb 0x7f5e40009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:14:43.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.299+0000 7f5e4ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d99e558e40 0x55d99e5598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.299+0000 7f5e4ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d99e558e40 0x55d99e5598b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37398/0 (socket says 172.21.15.64:37398) 2023-09-13T03:14:43.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.299+0000 7f5e4ed9d700 1 -- 172.21.15.64:0/2828106789 learned_addr learned my addr 172.21.15.64:0/2828106789 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:43.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.299+0000 7f5e4e59c700 1 --2- 172.21.15.64:0/2828106789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d99e54e710 0x55d99e55d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.299+0000 7f5e4dd9b700 1 --2- 172.21.15.64:0/2828106789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d99e54f030 0x55d99e55d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.299+0000 7f5e4ed9d700 1 -- 172.21.15.64:0/2828106789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d99e54f030 msgr2=0x55d99e55d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:43.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.299+0000 7f5e4ed9d700 1 --2- 172.21.15.64:0/2828106789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d99e54f030 0x55d99e55d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.299+0000 7f5e4ed9d700 1 -- 172.21.15.64:0/2828106789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d99e54e710 msgr2=0x55d99e55d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:43.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.299+0000 7f5e4ed9d700 1 --2- 172.21.15.64:0/2828106789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d99e54e710 0x55d99e55d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.299+0000 7f5e4ed9d700 1 -- 172.21.15.64:0/2828106789 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d99e2b5c60 con 0x55d99e558e40 2023-09-13T03:14:43.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.299+0000 7f5e4e59c700 1 --2- 172.21.15.64:0/2828106789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d99e54e710 0x55d99e55d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:43.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.300+0000 7f5e4ed9d700 1 --2- 172.21.15.64:0/2828106789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d99e558e40 0x55d99e5598b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5e480189a0 tx=0x7f5e48004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b6b08eeb2844675 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:43.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.300+0000 7f5e35ffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:14:43.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.300+0000 7f5e35ffb700 1 -- 172.21.15.64:0/2828106789 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4800e040 con 0x55d99e558e40 2023-09-13T03:14:43.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.300+0000 7f5e684c1b80 1 -- 172.21.15.64:0/2828106789 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5e4409a8d0 con 0x55d99e558e40 2023-09-13T03:14:43.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.300+0000 7f5e35ffb700 1 -- 172.21.15.64:0/2828106789 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f5e48003a00 con 0x55d99e558e40 2023-09-13T03:14:43.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.300+0000 7f5e35ffb700 1 -- 172.21.15.64:0/2828106789 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e48003bc0 con 0x55d99e558e40 2023-09-13T03:14:43.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.300+0000 7f5e684c1b80 10 client.16125 fetch_fsmap learned FSMap version 32 2023-09-13T03:14:43.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.300+0000 7f5e684c1b80 1 -- 172.21.15.64:0/2828106789 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f5e440d0040 con 0x55d99e558e40 2023-09-13T03:14:43.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.300+0000 7f5e35ffb700 1 -- 172.21.15.64:0/2828106789 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e4409a8d0 con 0x55d99e558e40 2023-09-13T03:14:43.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.300+0000 7f5e35ffb700 1 -- 172.21.15.64:0/2828106789 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 32) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f5e48004300 con 0x55d99e558e40 2023-09-13T03:14:43.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.300+0000 7f5e684c1b80 10 client.16125 fetch_fsmap finished waiting for FSMap version 32 2023-09-13T03:14:43.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.300+0000 7f5e684c1b80 10 client.16125 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2023-09-13T03:14:43.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.300+0000 7f5e684c1b80 20 client.16125 populate_metadata read hostname 'smithi064' 2023-09-13T03:14:43.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.300+0000 7f5e684c1b80 1 --2- 172.21.15.64:0/2828106789 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x55d99e5bd610 0x55d99e5bfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.300+0000 7f5e684c1b80 4 client.16125 mds_command: new command op to 24622 tid=0[{"prefix": "damage ls"}] 2023-09-13T03:14:43.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.300+0000 7f5e684c1b80 1 -- 172.21.15.64:0/2828106789 --> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d99e499920 con 0x55d99e5bd610 2023-09-13T03:14:43.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.301+0000 7f5e4e59c700 1 --2- 172.21.15.64:0/2828106789 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x55d99e5bd610 0x55d99e5bfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.301+0000 7f5e4e59c700 1 --2- 172.21.15.64:0/2828106789 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x55d99e5bd610 0x55d99e5bfa30 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7008a92ad113a298 server_cookie=37834d213c6d2c6d in_seq=0 out_seq=0 2023-09-13T03:14:43.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.301+0000 7f5e35ffb700 10 client.16125 ms_handle_connect on v2:172.21.15.170:6834/1493371121 2023-09-13T03:14:43.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.302+0000 7f5e35ffb700 1 -- 172.21.15.64:0/2828106789 <== mds.4 v2:172.21.15.170:6834/1493371121 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d99e499920 con 0x55d99e5bd610 2023-09-13T03:14:43.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.302+0000 7f5e35ffb700 10 client.16125 handle_command_reply: tid=0 2023-09-13T03:14:43.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.302+0000 7f5e684c1b80 1 client.16125 shutdown 2023-09-13T03:14:43.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.302+0000 7f5e684c1b80 1 -- 172.21.15.64:0/2828106789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d99e558e40 msgr2=0x55d99e5598b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:43.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.302+0000 7f5e684c1b80 1 --2- 172.21.15.64:0/2828106789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d99e558e40 0x55d99e5598b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5e480189a0 tx=0x7f5e48004ad0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.302+0000 7f5e684c1b80 1 -- 172.21.15.64:0/2828106789 shutdown_connections 2023-09-13T03:14:43.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.302+0000 7f5e684c1b80 1 --2- 172.21.15.64:0/2828106789 >> [v2:172.21.15.170:6834/1493371121,v1:172.21.15.170:6835/1493371121] conn(0x55d99e5bd610 0x55d99e5bfa30 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.302+0000 7f5e684c1b80 1 --2- 172.21.15.64:0/2828106789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d99e54e710 0x55d99e55d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.302+0000 7f5e684c1b80 1 --2- 172.21.15.64:0/2828106789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d99e558e40 0x55d99e5598b0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.302+0000 7f5e684c1b80 1 --2- 172.21.15.64:0/2828106789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d99e54f030 0x55d99e55d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.302+0000 7f5e684c1b80 1 -- 172.21.15.64:0/2828106789 >> 172.21.15.64:0/2828106789 conn(0x55d99e4c3090 msgr2=0x55d99e556680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:43.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.302+0000 7f5e35ffb700 0 client.16125 ms_handle_reset on v2:172.21.15.170:6834/1493371121 2023-09-13T03:14:43.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.302+0000 7f5e684c1b80 1 -- 172.21.15.64:0/2828106789 shutdown_connections 2023-09-13T03:14:43.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.302+0000 7f5e684c1b80 1 -- 172.21.15.64:0/2828106789 wait complete. 2023-09-13T03:14:43.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.302+0000 7f5e684c1b80 20 client.16125 trim_cache size 0 max 16384 2023-09-13T03:14:43.322 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2023-09-13T03:14:43.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.303+0000 7f5e54bff700 1 -- 172.21.15.64:0/3286049713 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f5e380a0830 msgr2=0x7f5e380a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:43.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.303+0000 7f5e54bff700 1 --2- 172.21.15.64:0/3286049713 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f5e380a0830 0x7f5e380a2cf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5e4800fb80 tx=0x7f5e48004b40 comp rx=0 tx=0).stop 2023-09-13T03:14:43.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.303+0000 7f5e54bff700 1 -- 172.21.15.64:0/3286049713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e50100f30 msgr2=0x7f5e50196b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:43.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.303+0000 7f5e54bff700 1 --2- 172.21.15.64:0/3286049713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e50100f30 0x7f5e50196b50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5e4400b580 tx=0x7f5e4400bfd0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.303+0000 7f5e54bff700 1 -- 172.21.15.64:0/3286049713 shutdown_connections 2023-09-13T03:14:43.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.303+0000 7f5e54bff700 1 --2- 172.21.15.64:0/3286049713 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f5e380a0830 0x7f5e380a2cf0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.303+0000 7f5e54bff700 1 --2- 172.21.15.64:0/3286049713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e50106630 0x7f5e5019afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.303+0000 7f5e54bff700 1 --2- 172.21.15.64:0/3286049713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e50100570 0x7f5e50196610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.303+0000 7f5e54bff700 1 --2- 172.21.15.64:0/3286049713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e50100f30 0x7f5e50196b50 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.303+0000 7f5e54bff700 1 -- 172.21.15.64:0/3286049713 >> 172.21.15.64:0/3286049713 conn(0x7f5e500fc080 msgr2=0x7f5e500fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:43.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.303+0000 7f5e54bff700 1 -- 172.21.15.64:0/3286049713 shutdown_connections 2023-09-13T03:14:43.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.303+0000 7f5e54bff700 1 -- 172.21.15.64:0/3286049713 wait complete. 2023-09-13T03:14:43.326 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-09-13T03:14:43.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.533+0000 7fa4695ff700 1 -- 172.21.15.64:0/3480242032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa46410cf10 msgr2=0x7fa46410f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:43.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.533+0000 7fa4695ff700 1 --2- 172.21.15.64:0/3480242032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa46410cf10 0x7fa46410f300 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa45c009f90 tx=0x7fa45c00b330 comp rx=0 tx=0).stop 2023-09-13T03:14:43.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.533+0000 7fa4695ff700 1 -- 172.21.15.64:0/3480242032 shutdown_connections 2023-09-13T03:14:43.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.533+0000 7fa4695ff700 1 --2- 172.21.15.64:0/3480242032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa46410cf10 0x7fa46410f300 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.533+0000 7fa4695ff700 1 --2- 172.21.15.64:0/3480242032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4640ff360 0x7fa4640ff740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.533+0000 7fa4695ff700 1 --2- 172.21.15.64:0/3480242032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4640ffd10 0x7fa46410c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.533+0000 7fa4695ff700 1 -- 172.21.15.64:0/3480242032 >> 172.21.15.64:0/3480242032 conn(0x7fa4640fafa0 msgr2=0x7fa4640fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:43.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.533+0000 7fa4695ff700 1 -- 172.21.15.64:0/3480242032 shutdown_connections 2023-09-13T03:14:43.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.534+0000 7fa4695ff700 1 -- 172.21.15.64:0/3480242032 wait complete. 2023-09-13T03:14:43.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.535+0000 7fa4695ff700 1 Processor -- start 2023-09-13T03:14:43.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.535+0000 7fa4695ff700 1 -- start start 2023-09-13T03:14:43.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.535+0000 7fa4695ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4640ff360 0x7fa46419e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.535+0000 7fa4695ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4640ffd10 0x7fa46419ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.535+0000 7fa4695ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa46410cf10 0x7fa464198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.536+0000 7fa4695ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa464198e90 con 0x7fa4640ff360 2023-09-13T03:14:43.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.536+0000 7fa4695ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa464199000 con 0x7fa46410cf10 2023-09-13T03:14:43.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.536+0000 7fa4695ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4641991c0 con 0x7fa4640ffd10 2023-09-13T03:14:43.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.536+0000 7fa462ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4640ff360 0x7fa46419e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.536+0000 7fa462ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4640ff360 0x7fa46419e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37418/0 (socket says 172.21.15.64:37418) 2023-09-13T03:14:43.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.536+0000 7fa45bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4640ffd10 0x7fa46419ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.536+0000 7fa462ffd700 1 -- 172.21.15.64:0/3520243623 learned_addr learned my addr 172.21.15.64:0/3520243623 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:43.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.536+0000 7fa4637fe700 1 --2- 172.21.15.64:0/3520243623 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa46410cf10 0x7fa464198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.536+0000 7fa462ffd700 1 -- 172.21.15.64:0/3520243623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4640ffd10 msgr2=0x7fa46419ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:43.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.536+0000 7fa462ffd700 1 --2- 172.21.15.64:0/3520243623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4640ffd10 0x7fa46419ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.536+0000 7fa462ffd700 1 -- 172.21.15.64:0/3520243623 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa46410cf10 msgr2=0x7fa464198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:43.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.536+0000 7fa462ffd700 1 --2- 172.21.15.64:0/3520243623 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa46410cf10 0x7fa464198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.536+0000 7fa462ffd700 1 -- 172.21.15.64:0/3520243623 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa454009520 con 0x7fa4640ff360 2023-09-13T03:14:43.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.536+0000 7fa462ffd700 1 --2- 172.21.15.64:0/3520243623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4640ff360 0x7fa46419e7b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa45400eb30 tx=0x7fa45400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:43.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.537+0000 7fa460ff9700 1 -- 172.21.15.64:0/3520243623 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa454009880 con 0x7fa4640ff360 2023-09-13T03:14:43.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.537+0000 7fa460ff9700 1 -- 172.21.15.64:0/3520243623 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4540044e0 con 0x7fa4640ff360 2023-09-13T03:14:43.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.537+0000 7fa460ff9700 1 -- 172.21.15.64:0/3520243623 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa454003d80 con 0x7fa4640ff360 2023-09-13T03:14:43.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.537+0000 7fa4695ff700 1 -- 172.21.15.64:0/3520243623 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa44c0095f0 con 0x7fa4640ff360 2023-09-13T03:14:43.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.537+0000 7fa4695ff700 1 -- 172.21.15.64:0/3520243623 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa45c009c40 con 0x7fa4640ff360 2023-09-13T03:14:43.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.540+0000 7fa460ff9700 1 -- 172.21.15.64:0/3520243623 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fa454003f20 con 0x7fa4640ff360 2023-09-13T03:14:43.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.542+0000 7fa47ce4ab80 1 Processor -- start 2023-09-13T03:14:43.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.542+0000 7fa47ce4ab80 1 -- start start 2023-09-13T03:14:43.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.543+0000 7fa460ff9700 1 --2- 172.21.15.64:0/3520243623 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fa4500a0830 0x7fa4500a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.543+0000 7fa47ce4ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc53700710 0x55bc53700af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.543+0000 7fa47ce4ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc53701030 0x55bc5370a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.543+0000 7fa47ce4ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc5370ae40 0x55bc5370d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.543+0000 7fa47ce4ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc53659730 con 0x55bc53700710 2023-09-13T03:14:43.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.543+0000 7fa47ce4ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc53657050 con 0x55bc53701030 2023-09-13T03:14:43.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.543+0000 7fa47ce4ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc53652940 con 0x55bc5370ae40 2023-09-13T03:14:43.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.543+0000 7fa460ff9700 1 -- 172.21.15.64:0/3520243623 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa454014070 con 0x7fa4640ff360 2023-09-13T03:14:43.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.543+0000 7fa45bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc53701030 0x55bc5370a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.543+0000 7fa45bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc53701030 0x55bc5370a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:36374/0 (socket says 172.21.15.64:36374) 2023-09-13T03:14:43.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.543+0000 7fa45bfff700 1 -- 172.21.15.64:0/3895392823 learned_addr learned my addr 172.21.15.64:0/3895392823 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:43.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.543+0000 7fa4637fe700 1 --2- 172.21.15.64:0/3520243623 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fa4500a0830 0x7fa4500a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.543+0000 7fa462ffd700 1 --2- 172.21.15.64:0/3895392823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc53700710 0x55bc53700af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.543+0000 7fa4637fe700 1 --2- 172.21.15.64:0/3895392823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc5370ae40 0x55bc5370d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.544+0000 7fa4637fe700 1 -- 172.21.15.64:0/3895392823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc53701030 msgr2=0x55bc5370a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:43.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.544+0000 7fa4637fe700 1 --2- 172.21.15.64:0/3895392823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc53701030 0x55bc5370a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.544+0000 7fa4637fe700 1 -- 172.21.15.64:0/3895392823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc53700710 msgr2=0x55bc53700af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:43.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.544+0000 7fa4637fe700 1 --2- 172.21.15.64:0/3895392823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc53700710 0x55bc53700af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.544+0000 7fa4637fe700 1 -- 172.21.15.64:0/3895392823 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc5347ede0 con 0x55bc5370ae40 2023-09-13T03:14:43.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.544+0000 7fa4637fe700 1 --2- 172.21.15.64:0/3895392823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc5370ae40 0x55bc5370d230 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa45c00b6c0 tx=0x7fa45c008c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c57f8fc65d753c4e server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:43.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.544+0000 7fa4637fe700 1 --2- 172.21.15.64:0/3520243623 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fa4500a0830 0x7fa4500a2cf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa4640697d0 tx=0x7fa45c019000 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:43.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.544+0000 7fa4597fa700 1 -- 172.21.15.64:0/3895392823 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45c026070 con 0x55bc5370ae40 2023-09-13T03:14:43.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.545+0000 7fa4597fa700 1 -- 172.21.15.64:0/3895392823 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa45c00b9f0 con 0x55bc5370ae40 2023-09-13T03:14:43.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.545+0000 7fa4597fa700 1 -- 172.21.15.64:0/3895392823 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45c00e040 con 0x55bc5370ae40 2023-09-13T03:14:43.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.545+0000 7fa47ce4ab80 1 -- 172.21.15.64:0/3895392823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc5370ae40 msgr2=0x55bc5370d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:43.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.545+0000 7fa47ce4ab80 1 --2- 172.21.15.64:0/3895392823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc5370ae40 0x55bc5370d230 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa45c00b6c0 tx=0x7fa45c008c20 comp rx=0 tx=0).stop 2023-09-13T03:14:43.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.545+0000 7fa47ce4ab80 1 -- 172.21.15.64:0/3895392823 shutdown_connections 2023-09-13T03:14:43.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.545+0000 7fa47ce4ab80 1 --2- 172.21.15.64:0/3895392823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc5370ae40 0x55bc5370d230 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.545+0000 7fa47ce4ab80 1 --2- 172.21.15.64:0/3895392823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc53700710 0x55bc53700af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.545+0000 7fa47ce4ab80 1 --2- 172.21.15.64:0/3895392823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc53701030 0x55bc5370a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.545+0000 7fa47ce4ab80 1 -- 172.21.15.64:0/3895392823 >> 172.21.15.64:0/3895392823 conn(0x55bc53675090 msgr2=0x55bc536fbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:43.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.546+0000 7fa47ce4ab80 1 -- 172.21.15.64:0/3895392823 shutdown_connections 2023-09-13T03:14:43.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.546+0000 7fa47ce4ab80 1 -- 172.21.15.64:0/3895392823 wait complete. 2023-09-13T03:14:43.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.546+0000 7fa47ce4ab80 1 -- start start 2023-09-13T03:14:43.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.546+0000 7fa47ce4ab80 1 Processor -- start 2023-09-13T03:14:43.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.547+0000 7fa47ce4ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc53700710 0x55bc5370a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.547+0000 7fa47ce4ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc53701030 0x55bc53707b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.547+0000 7fa47ce4ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc5370ae40 0x55bc53708240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.547+0000 7fa47ce4ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc535f0b70 con 0x55bc53700710 2023-09-13T03:14:43.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.547+0000 7fa47ce4ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc535f0870 con 0x55bc53701030 2023-09-13T03:14:43.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.547+0000 7fa47ce4ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc534d4920 con 0x55bc5370ae40 2023-09-13T03:14:43.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.547+0000 7fa47ce4ab80 10 client.0 _ll_register_callbacks cb 0x55bc5353aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:14:43.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.547+0000 7fa4637fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc5370ae40 0x55bc53708240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.547+0000 7fa45bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc53701030 0x55bc53707b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.547+0000 7fa45bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc53701030 0x55bc53707b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:36380/0 (socket says 172.21.15.64:36380) 2023-09-13T03:14:43.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.547+0000 7fa4637fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc5370ae40 0x55bc53708240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:47482/0 (socket says 172.21.15.64:47482) 2023-09-13T03:14:43.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.547+0000 7fa45bfff700 1 -- 172.21.15.64:0/1787391604 learned_addr learned my addr 172.21.15.64:0/1787391604 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:43.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.547+0000 7fa462ffd700 1 --2- 172.21.15.64:0/1787391604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc53700710 0x55bc5370a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.547+0000 7fa45bfff700 1 -- 172.21.15.64:0/1787391604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc5370ae40 msgr2=0x55bc53708240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:43.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.547+0000 7fa45bfff700 1 --2- 172.21.15.64:0/1787391604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc5370ae40 0x55bc53708240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.547+0000 7fa45bfff700 1 -- 172.21.15.64:0/1787391604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc53700710 msgr2=0x55bc5370a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:43.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.547+0000 7fa45bfff700 1 --2- 172.21.15.64:0/1787391604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc53700710 0x55bc5370a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.547+0000 7fa45bfff700 1 -- 172.21.15.64:0/1787391604 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa46410ff80 con 0x55bc53701030 2023-09-13T03:14:43.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.548+0000 7fa462ffd700 1 --2- 172.21.15.64:0/1787391604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc53700710 0x55bc5370a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:14:43.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.548+0000 7fa4637fe700 1 --2- 172.21.15.64:0/1787391604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc5370ae40 0x55bc53708240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:43.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.548+0000 7fa45bfff700 1 --2- 172.21.15.64:0/1787391604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc53701030 0x55bc53707b00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa44c012950 tx=0x7fa44c012c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56b6dad8a2dc3dd server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:43.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.548+0000 7fa44a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:14:43.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.548+0000 7fa44a7fc700 1 -- 172.21.15.64:0/1787391604 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c017070 con 0x55bc53701030 2023-09-13T03:14:43.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.548+0000 7fa47ce4ab80 1 -- 172.21.15.64:0/1787391604 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bc535aa4f0 con 0x55bc53701030 2023-09-13T03:14:43.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.548+0000 7fa44a7fc700 1 -- 172.21.15.64:0/1787391604 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa44c012e40 con 0x55bc53701030 2023-09-13T03:14:43.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.548+0000 7fa44a7fc700 1 -- 172.21.15.64:0/1787391604 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa44c00f830 con 0x55bc53701030 2023-09-13T03:14:43.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.548+0000 7fa47ce4ab80 10 client.26177 fetch_fsmap learned FSMap version 32 2023-09-13T03:14:43.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.548+0000 7fa47ce4ab80 1 -- 172.21.15.64:0/1787391604 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55bc53467c60 con 0x55bc53701030 2023-09-13T03:14:43.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.548+0000 7fa44a7fc700 1 -- 172.21.15.64:0/1787391604 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bc535aa4f0 con 0x55bc53701030 2023-09-13T03:14:43.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.549+0000 7fa44a7fc700 1 -- 172.21.15.64:0/1787391604 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 32) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fa44c010960 con 0x55bc53701030 2023-09-13T03:14:43.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.549+0000 7fa47ce4ab80 10 client.26177 fetch_fsmap finished waiting for FSMap version 32 2023-09-13T03:14:43.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.549+0000 7fa47ce4ab80 10 client.26177 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2023-09-13T03:14:43.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.549+0000 7fa47ce4ab80 20 client.26177 populate_metadata read hostname 'smithi064' 2023-09-13T03:14:43.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.549+0000 7fa47ce4ab80 1 --2- 172.21.15.64:0/1787391604 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x55bc5371a980 0x55bc5371cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.549+0000 7fa47ce4ab80 4 client.26177 mds_command: new command op to 24634 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:14:43.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.549+0000 7fa47ce4ab80 1 -- 172.21.15.64:0/1787391604 --> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc536678a0 con 0x55bc5371a980 2023-09-13T03:14:43.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.549+0000 7fa462ffd700 1 --2- 172.21.15.64:0/1787391604 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x55bc5371a980 0x55bc5371cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.550+0000 7fa462ffd700 1 --2- 172.21.15.64:0/1787391604 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x55bc5371a980 0x55bc5371cda0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=880489c8a3d3c485 server_cookie=d935eb86611f57ac in_seq=0 out_seq=0 2023-09-13T03:14:43.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.550+0000 7fa44a7fc700 10 client.26177 ms_handle_connect on v2:172.21.15.170:6836/1726587275 2023-09-13T03:14:43.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.550+0000 7fa44a7fc700 1 -- 172.21.15.64:0/1787391604 <== mds.3 v2:172.21.15.170:6836/1726587275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bc536678a0 con 0x55bc5371a980 2023-09-13T03:14:43.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.550+0000 7fa44a7fc700 10 client.26177 handle_command_reply: tid=0 2023-09-13T03:14:43.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.551+0000 7fa47ce4ab80 1 client.26177 shutdown 2023-09-13T03:14:43.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.551+0000 7fa47ce4ab80 1 -- 172.21.15.64:0/1787391604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc53701030 msgr2=0x55bc53707b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:43.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.551+0000 7fa47ce4ab80 1 --2- 172.21.15.64:0/1787391604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc53701030 0x55bc53707b00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa44c012950 tx=0x7fa44c012c60 comp rx=0 tx=0).stop 2023-09-13T03:14:43.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.551+0000 7fa47ce4ab80 1 -- 172.21.15.64:0/1787391604 shutdown_connections 2023-09-13T03:14:43.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.551+0000 7fa47ce4ab80 1 --2- 172.21.15.64:0/1787391604 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x55bc5371a980 0x55bc5371cda0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.551+0000 7fa47ce4ab80 1 --2- 172.21.15.64:0/1787391604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc5370ae40 0x55bc53708240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.551+0000 7fa47ce4ab80 1 --2- 172.21.15.64:0/1787391604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc53700710 0x55bc5370a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.551+0000 7fa47ce4ab80 1 --2- 172.21.15.64:0/1787391604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc53701030 0x55bc53707b00 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.551+0000 7fa47ce4ab80 1 -- 172.21.15.64:0/1787391604 >> 172.21.15.64:0/1787391604 conn(0x55bc53675090 msgr2=0x55bc536fb780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:43.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.551+0000 7fa44a7fc700 0 client.26177 ms_handle_reset on v2:172.21.15.170:6836/1726587275 2023-09-13T03:14:43.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.551+0000 7fa47ce4ab80 1 -- 172.21.15.64:0/1787391604 shutdown_connections 2023-09-13T03:14:43.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.551+0000 7fa47ce4ab80 1 -- 172.21.15.64:0/1787391604 wait complete. 2023-09-13T03:14:43.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.551+0000 7fa47ce4ab80 20 client.26177 trim_cache size 0 max 16384 2023-09-13T03:14:43.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.567+0000 7fa47ce4ab80 1 Processor -- start 2023-09-13T03:14:43.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.567+0000 7fa47ce4ab80 1 -- start start 2023-09-13T03:14:43.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.567+0000 7fa47ce4ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc53700710 0x55bc53716ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.567+0000 7fa47ce4ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc53701030 0x55bc5370a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.567+0000 7fa47ce4ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc5370ae40 0x55bc53702dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.567+0000 7fa47ce4ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc537171e0 con 0x55bc53701030 2023-09-13T03:14:43.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.567+0000 7fa47ce4ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc53717350 con 0x55bc53700710 2023-09-13T03:14:43.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.567+0000 7fa47ce4ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc536fd0b0 con 0x55bc5370ae40 2023-09-13T03:14:43.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.567+0000 7fa462ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc53700710 0x55bc53716ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.567+0000 7fa4637fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc5370ae40 0x55bc53702dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.567+0000 7fa45bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc53701030 0x55bc5370a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.567+0000 7fa462ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc53700710 0x55bc53716ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:36386/0 (socket says 172.21.15.64:36386) 2023-09-13T03:14:43.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.567+0000 7fa4637fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc5370ae40 0x55bc53702dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:47496/0 (socket says 172.21.15.64:47496) 2023-09-13T03:14:43.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.567+0000 7fa462ffd700 1 -- 172.21.15.64:0/2782033523 learned_addr learned my addr 172.21.15.64:0/2782033523 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:43.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.568+0000 7fa462ffd700 1 -- 172.21.15.64:0/2782033523 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc5370ae40 msgr2=0x55bc53702dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:43.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.568+0000 7fa462ffd700 1 --2- 172.21.15.64:0/2782033523 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc5370ae40 0x55bc53702dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.568+0000 7fa462ffd700 1 -- 172.21.15.64:0/2782033523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc53701030 msgr2=0x55bc5370a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:43.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.568+0000 7fa462ffd700 1 --2- 172.21.15.64:0/2782033523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc53701030 0x55bc5370a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.568+0000 7fa462ffd700 1 -- 172.21.15.64:0/2782033523 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc534a5250 con 0x55bc53700710 2023-09-13T03:14:43.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.568+0000 7fa462ffd700 1 --2- 172.21.15.64:0/2782033523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc53700710 0x55bc53716ca0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa4540963d0 tx=0x7fa454086040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c973a630e97c7c4b server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:43.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.568+0000 7fa459ffb700 1 -- 172.21.15.64:0/2782033523 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45400cbd0 con 0x55bc53700710 2023-09-13T03:14:43.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.568+0000 7fa459ffb700 1 -- 172.21.15.64:0/2782033523 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa45409a060 con 0x55bc53700710 2023-09-13T03:14:43.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.568+0000 7fa459ffb700 1 -- 172.21.15.64:0/2782033523 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45401fde0 con 0x55bc53700710 2023-09-13T03:14:43.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.569+0000 7fa47ce4ab80 1 -- 172.21.15.64:0/2782033523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc53700710 msgr2=0x55bc53716ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:43.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.569+0000 7fa47ce4ab80 1 --2- 172.21.15.64:0/2782033523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc53700710 0x55bc53716ca0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa4540963d0 tx=0x7fa454086040 comp rx=0 tx=0).stop 2023-09-13T03:14:43.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.569+0000 7fa47ce4ab80 1 -- 172.21.15.64:0/2782033523 shutdown_connections 2023-09-13T03:14:43.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.569+0000 7fa47ce4ab80 1 --2- 172.21.15.64:0/2782033523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc53700710 0x55bc53716ca0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.569+0000 7fa47ce4ab80 1 --2- 172.21.15.64:0/2782033523 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc5370ae40 0x55bc53702dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.569+0000 7fa47ce4ab80 1 --2- 172.21.15.64:0/2782033523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc53701030 0x55bc5370a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.569+0000 7fa47ce4ab80 1 -- 172.21.15.64:0/2782033523 >> 172.21.15.64:0/2782033523 conn(0x55bc53675090 msgr2=0x55bc53721720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:43.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.569+0000 7fa47ce4ab80 1 -- 172.21.15.64:0/2782033523 shutdown_connections 2023-09-13T03:14:43.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.569+0000 7fa47ce4ab80 1 -- 172.21.15.64:0/2782033523 wait complete. 2023-09-13T03:14:43.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.570+0000 7fa47ce4ab80 1 -- start start 2023-09-13T03:14:43.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.570+0000 7fa47ce4ab80 1 Processor -- start 2023-09-13T03:14:43.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.570+0000 7fa47ce4ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc53700710 0x55bc53711430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.570+0000 7fa47ce4ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc53701030 0x55bc53711970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.570+0000 7fa47ce4ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc5370ae40 0x55bc5370f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.570+0000 7fa47ce4ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5370fe70 con 0x55bc5370ae40 2023-09-13T03:14:43.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.570+0000 7fa47ce4ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5370ffe0 con 0x55bc53701030 2023-09-13T03:14:43.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.570+0000 7fa47ce4ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc537101a0 con 0x55bc53700710 2023-09-13T03:14:43.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.570+0000 7fa47ce4ab80 10 client.0 _ll_register_callbacks cb 0x7fa44c012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:14:43.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.571+0000 7fa45bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc53701030 0x55bc53711970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.571+0000 7fa45bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc53701030 0x55bc53711970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:36394/0 (socket says 172.21.15.64:36394) 2023-09-13T03:14:43.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.571+0000 7fa462ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc53700710 0x55bc53711430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.571+0000 7fa45bfff700 1 -- 172.21.15.64:0/2381735820 learned_addr learned my addr 172.21.15.64:0/2381735820 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:43.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.571+0000 7fa4637fe700 1 --2- 172.21.15.64:0/2381735820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc5370ae40 0x55bc5370f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.571+0000 7fa45bfff700 1 -- 172.21.15.64:0/2381735820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc53700710 msgr2=0x55bc53711430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:43.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.571+0000 7fa45bfff700 1 --2- 172.21.15.64:0/2381735820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc53700710 0x55bc53711430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.571+0000 7fa45bfff700 1 -- 172.21.15.64:0/2381735820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc5370ae40 msgr2=0x55bc5370f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:43.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.571+0000 7fa45bfff700 1 --2- 172.21.15.64:0/2381735820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc5370ae40 0x55bc5370f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.571+0000 7fa45bfff700 1 -- 172.21.15.64:0/2381735820 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa45c009c40 con 0x55bc53701030 2023-09-13T03:14:43.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.571+0000 7fa45bfff700 1 --2- 172.21.15.64:0/2381735820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc53701030 0x55bc53711970 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x55bc537090b0 tx=0x7fa44c00ff40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f25df53aa3a99d27 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:43.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.571+0000 7fa44a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:14:43.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.572+0000 7fa44a7fc700 1 -- 172.21.15.64:0/2381735820 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c017030 con 0x55bc53701030 2023-09-13T03:14:43.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.572+0000 7fa44a7fc700 1 -- 172.21.15.64:0/2381735820 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa44c000e40 con 0x55bc53701030 2023-09-13T03:14:43.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.572+0000 7fa47ce4ab80 1 -- 172.21.15.64:0/2381735820 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa45c00b9f0 con 0x55bc53701030 2023-09-13T03:14:43.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.572+0000 7fa44a7fc700 1 -- 172.21.15.64:0/2381735820 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa44c010630 con 0x55bc53701030 2023-09-13T03:14:43.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.572+0000 7fa47ce4ab80 10 client.26183 fetch_fsmap learned FSMap version 32 2023-09-13T03:14:43.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.572+0000 7fa47ce4ab80 1 -- 172.21.15.64:0/2381735820 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55bc5347d2b0 con 0x55bc53701030 2023-09-13T03:14:43.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.572+0000 7fa44a7fc700 1 -- 172.21.15.64:0/2381735820 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa45c00b9f0 con 0x55bc53701030 2023-09-13T03:14:43.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.573+0000 7fa44a7fc700 1 -- 172.21.15.64:0/2381735820 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 32) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fa44c004cb0 con 0x55bc53701030 2023-09-13T03:14:43.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.573+0000 7fa47ce4ab80 10 client.26183 fetch_fsmap finished waiting for FSMap version 32 2023-09-13T03:14:43.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.573+0000 7fa47ce4ab80 10 client.26183 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2023-09-13T03:14:43.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.573+0000 7fa47ce4ab80 20 client.26183 populate_metadata read hostname 'smithi064' 2023-09-13T03:14:43.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.573+0000 7fa47ce4ab80 1 --2- 172.21.15.64:0/2381735820 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x55bc5376f9c0 0x55bc53771de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.573+0000 7fa47ce4ab80 4 client.26183 mds_command: new command op to 24634 tid=0[{"prefix": "damage ls"}] 2023-09-13T03:14:43.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.573+0000 7fa47ce4ab80 1 -- 172.21.15.64:0/2381735820 --> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bc5364b920 con 0x55bc5376f9c0 2023-09-13T03:14:43.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.573+0000 7fa462ffd700 1 --2- 172.21.15.64:0/2381735820 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x55bc5376f9c0 0x55bc53771de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.573+0000 7fa462ffd700 1 --2- 172.21.15.64:0/2381735820 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x55bc5376f9c0 0x55bc53771de0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1743ec4afb8d0832 server_cookie=edcf0449ad0c9fd0 in_seq=0 out_seq=0 2023-09-13T03:14:43.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.573+0000 7fa44a7fc700 10 client.26183 ms_handle_connect on v2:172.21.15.170:6836/1726587275 2023-09-13T03:14:43.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.574+0000 7fa44a7fc700 1 -- 172.21.15.64:0/2381735820 <== mds.3 v2:172.21.15.170:6836/1726587275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bc5364b920 con 0x55bc5376f9c0 2023-09-13T03:14:43.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.574+0000 7fa44a7fc700 10 client.26183 handle_command_reply: tid=0 2023-09-13T03:14:43.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.574+0000 7fa47ce4ab80 1 client.26183 shutdown 2023-09-13T03:14:43.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.574+0000 7fa47ce4ab80 1 -- 172.21.15.64:0/2381735820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc53701030 msgr2=0x55bc53711970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:43.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.574+0000 7fa47ce4ab80 1 --2- 172.21.15.64:0/2381735820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc53701030 0x55bc53711970 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x55bc537090b0 tx=0x7fa44c00ff40 comp rx=0 tx=0).stop 2023-09-13T03:14:43.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.574+0000 7fa47ce4ab80 1 -- 172.21.15.64:0/2381735820 shutdown_connections 2023-09-13T03:14:43.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.574+0000 7fa47ce4ab80 1 --2- 172.21.15.64:0/2381735820 >> [v2:172.21.15.170:6836/1726587275,v1:172.21.15.170:6837/1726587275] conn(0x55bc5376f9c0 0x55bc53771de0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.574+0000 7fa47ce4ab80 1 --2- 172.21.15.64:0/2381735820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc5370ae40 0x55bc5370f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.574+0000 7fa47ce4ab80 1 --2- 172.21.15.64:0/2381735820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bc53700710 0x55bc53711430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.574+0000 7fa44a7fc700 0 client.26183 ms_handle_reset on v2:172.21.15.170:6836/1726587275 2023-09-13T03:14:43.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.574+0000 7fa47ce4ab80 1 --2- 172.21.15.64:0/2381735820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc53701030 0x55bc53711970 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.574+0000 7fa47ce4ab80 1 -- 172.21.15.64:0/2381735820 >> 172.21.15.64:0/2381735820 conn(0x55bc53675090 msgr2=0x55bc53702dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:43.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.574+0000 7fa47ce4ab80 1 -- 172.21.15.64:0/2381735820 shutdown_connections 2023-09-13T03:14:43.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.574+0000 7fa47ce4ab80 1 -- 172.21.15.64:0/2381735820 wait complete. 2023-09-13T03:14:43.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.574+0000 7fa47ce4ab80 20 client.26183 trim_cache size 0 max 16384 2023-09-13T03:14:43.589 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2023-09-13T03:14:43.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.575+0000 7fa4695ff700 1 -- 172.21.15.64:0/3520243623 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fa4500a0830 msgr2=0x7fa4500a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:43.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.575+0000 7fa4695ff700 1 --2- 172.21.15.64:0/3520243623 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fa4500a0830 0x7fa4500a2cf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa4640697d0 tx=0x7fa45c019000 comp rx=0 tx=0).stop 2023-09-13T03:14:43.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.575+0000 7fa4695ff700 1 -- 172.21.15.64:0/3520243623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4640ff360 msgr2=0x7fa46419e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:43.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.575+0000 7fa4695ff700 1 --2- 172.21.15.64:0/3520243623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4640ff360 0x7fa46419e7b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa45400eb30 tx=0x7fa45400ee40 comp rx=0 tx=0).stop 2023-09-13T03:14:43.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.575+0000 7fa4695ff700 1 -- 172.21.15.64:0/3520243623 shutdown_connections 2023-09-13T03:14:43.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.575+0000 7fa4695ff700 1 --2- 172.21.15.64:0/3520243623 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fa4500a0830 0x7fa4500a2cf0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.575+0000 7fa4695ff700 1 --2- 172.21.15.64:0/3520243623 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa46410cf10 0x7fa464198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.575+0000 7fa4695ff700 1 --2- 172.21.15.64:0/3520243623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4640ff360 0x7fa46419e7b0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.575+0000 7fa4695ff700 1 --2- 172.21.15.64:0/3520243623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4640ffd10 0x7fa46419ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.575+0000 7fa4695ff700 1 -- 172.21.15.64:0/3520243623 >> 172.21.15.64:0/3520243623 conn(0x7fa4640fafa0 msgr2=0x7fa46410d960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:43.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.575+0000 7fa4695ff700 1 -- 172.21.15.64:0/3520243623 shutdown_connections 2023-09-13T03:14:43.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.575+0000 7fa4695ff700 1 -- 172.21.15.64:0/3520243623 wait complete. 2023-09-13T03:14:43.595 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-09-13T03:14:43.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.814+0000 7f1d457ff700 1 -- 172.21.15.64:0/3669743387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d40100570 msgr2=0x7f1d401009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:43.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.814+0000 7f1d457ff700 1 --2- 172.21.15.64:0/3669743387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d40100570 0x7f1d401009f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1d34009990 tx=0x7f1d34009ca0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.815+0000 7f1d457ff700 1 -- 172.21.15.64:0/3669743387 shutdown_connections 2023-09-13T03:14:43.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.815+0000 7f1d457ff700 1 --2- 172.21.15.64:0/3669743387 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d40100f30 0x7f1d4010d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.815+0000 7f1d457ff700 1 --2- 172.21.15.64:0/3669743387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d40106630 0x7f1d40106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.815+0000 7f1d457ff700 1 --2- 172.21.15.64:0/3669743387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d40100570 0x7f1d401009f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.815+0000 7f1d457ff700 1 -- 172.21.15.64:0/3669743387 >> 172.21.15.64:0/3669743387 conn(0x7f1d400fc080 msgr2=0x7f1d400fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:43.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.815+0000 7f1d457ff700 1 -- 172.21.15.64:0/3669743387 shutdown_connections 2023-09-13T03:14:43.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.815+0000 7f1d457ff700 1 -- 172.21.15.64:0/3669743387 wait complete. 2023-09-13T03:14:43.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.816+0000 7f1d457ff700 1 Processor -- start 2023-09-13T03:14:43.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.816+0000 7f1d457ff700 1 -- start start 2023-09-13T03:14:43.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.816+0000 7f1d457ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d40100570 0x7f1d40196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.816+0000 7f1d457ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d40100f30 0x7f1d40196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.816+0000 7f1d457ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d40106630 0x7f1d4019afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.816+0000 7f1d457ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4019b610 con 0x7f1d40100f30 2023-09-13T03:14:43.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.816+0000 7f1d457ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4019b780 con 0x7f1d40106630 2023-09-13T03:14:43.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.816+0000 7f1d457ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4019b940 con 0x7f1d40100570 2023-09-13T03:14:43.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.816+0000 7f1d3e7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d40100f30 0x7f1d40196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.816+0000 7f1d3f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d40106630 0x7f1d4019afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.817+0000 7f1d3e7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d40100f30 0x7f1d40196b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37476/0 (socket says 172.21.15.64:37476) 2023-09-13T03:14:43.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.817+0000 7f1d3e7fc700 1 -- 172.21.15.64:0/2759100353 learned_addr learned my addr 172.21.15.64:0/2759100353 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:43.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.817+0000 7f1d3effd700 1 --2- 172.21.15.64:0/2759100353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d40100570 0x7f1d40196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.817+0000 7f1d3f7fe700 1 -- 172.21.15.64:0/2759100353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d40100570 msgr2=0x7f1d40196630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:43.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.817+0000 7f1d3f7fe700 1 --2- 172.21.15.64:0/2759100353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d40100570 0x7f1d40196630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.817+0000 7f1d3f7fe700 1 -- 172.21.15.64:0/2759100353 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d40100f30 msgr2=0x7f1d40196b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:43.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.817+0000 7f1d3f7fe700 1 --2- 172.21.15.64:0/2759100353 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d40100f30 0x7f1d40196b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.817+0000 7f1d3f7fe700 1 -- 172.21.15.64:0/2759100353 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d340095f0 con 0x7f1d40106630 2023-09-13T03:14:43.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.817+0000 7f1d3f7fe700 1 --2- 172.21.15.64:0/2759100353 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d40106630 0x7f1d4019afd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1d3800efc0 tx=0x7f1d3800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:43.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.817+0000 7f1d27fff700 1 -- 172.21.15.64:0/2759100353 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d38018070 con 0x7f1d40106630 2023-09-13T03:14:43.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.817+0000 7f1d457ff700 1 -- 172.21.15.64:0/2759100353 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d4019bbc0 con 0x7f1d40106630 2023-09-13T03:14:43.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.817+0000 7f1d27fff700 1 -- 172.21.15.64:0/2759100353 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d3800f040 con 0x7f1d40106630 2023-09-13T03:14:43.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.817+0000 7f1d27fff700 1 -- 172.21.15.64:0/2759100353 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3800cd40 con 0x7f1d40106630 2023-09-13T03:14:43.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.818+0000 7f1d457ff700 1 -- 172.21.15.64:0/2759100353 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d401a3460 con 0x7f1d40106630 2023-09-13T03:14:43.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.823+0000 7f1d27fff700 1 -- 172.21.15.64:0/2759100353 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f1d38003a00 con 0x7f1d40106630 2023-09-13T03:14:43.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.823+0000 7f1d590e2b80 1 Processor -- start 2023-09-13T03:14:43.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.824+0000 7f1d590e2b80 1 -- start start 2023-09-13T03:14:43.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.824+0000 7f1d590e2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55967becb710 0x55967becbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.824+0000 7f1d590e2b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55967becc030 0x55967bed9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.824+0000 7f1d590e2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55967beda250 0x55967bedc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.824+0000 7f1d590e2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967be24730 con 0x55967becb710 2023-09-13T03:14:43.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.824+0000 7f1d590e2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967be22050 con 0x55967beda250 2023-09-13T03:14:43.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.824+0000 7f1d590e2b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967be1d940 con 0x55967becc030 2023-09-13T03:14:43.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.824+0000 7f1d3effd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55967becc030 0x55967bed9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.824+0000 7f1d3e7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55967becb710 0x55967becbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.824+0000 7f1d3effd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55967becc030 0x55967bed9d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:47538/0 (socket says 172.21.15.64:47538) 2023-09-13T03:14:43.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.824+0000 7f1d3effd700 1 -- 172.21.15.64:0/2077743926 learned_addr learned my addr 172.21.15.64:0/2077743926 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:43.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.824+0000 7f1d27fff700 1 --2- 172.21.15.64:0/2759100353 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f1d280a07b0 0x7f1d280a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.824+0000 7f1d3e7fc700 1 --2- 172.21.15.64:0/2077743926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55967beda250 0x55967bedc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.825+0000 7f1d3effd700 1 -- 172.21.15.64:0/2077743926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55967beda250 msgr2=0x55967bedc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:43.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.825+0000 7f1d3effd700 1 --2- 172.21.15.64:0/2077743926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55967beda250 0x55967bedc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.825+0000 7f1d27fff700 1 -- 172.21.15.64:0/2759100353 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f1d380cbb20 con 0x7f1d40106630 2023-09-13T03:14:43.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.825+0000 7f1d3effd700 1 -- 172.21.15.64:0/2077743926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55967becb710 msgr2=0x55967becbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:43.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.825+0000 7f1d3effd700 1 --2- 172.21.15.64:0/2077743926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55967becb710 0x55967becbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.825+0000 7f1d3effd700 1 -- 172.21.15.64:0/2077743926 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55967bc49de0 con 0x55967becc030 2023-09-13T03:14:43.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.825+0000 7f1d3e7fc700 1 --2- 172.21.15.64:0/2077743926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55967becb710 0x55967becbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:14:43.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.825+0000 7f1d3f7fe700 1 --2- 172.21.15.64:0/2759100353 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f1d280a07b0 0x7f1d280a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.825+0000 7f1d3effd700 1 --2- 172.21.15.64:0/2077743926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55967becc030 0x55967bed9d10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1d30009d80 tx=0x7f1d30011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f52d5f92170923d4 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:43.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.825+0000 7f1d257fa700 1 -- 172.21.15.64:0/2077743926 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d30004230 con 0x55967becc030 2023-09-13T03:14:43.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.825+0000 7f1d257fa700 1 -- 172.21.15.64:0/2077743926 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d30004390 con 0x55967becc030 2023-09-13T03:14:43.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.825+0000 7f1d3f7fe700 1 --2- 172.21.15.64:0/2759100353 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f1d280a07b0 0x7f1d280a2c70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1d3800c6f0 tx=0x7f1d38004dc0 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:43.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.825+0000 7f1d257fa700 1 -- 172.21.15.64:0/2077743926 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d30019860 con 0x55967becc030 2023-09-13T03:14:43.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.826+0000 7f1d590e2b80 1 -- 172.21.15.64:0/2077743926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55967becc030 msgr2=0x55967bed9d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:43.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.826+0000 7f1d590e2b80 1 --2- 172.21.15.64:0/2077743926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55967becc030 0x55967bed9d10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1d30009d80 tx=0x7f1d30011720 comp rx=0 tx=0).stop 2023-09-13T03:14:43.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.826+0000 7f1d590e2b80 1 -- 172.21.15.64:0/2077743926 shutdown_connections 2023-09-13T03:14:43.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.826+0000 7f1d590e2b80 1 --2- 172.21.15.64:0/2077743926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55967beda250 0x55967bedc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.826+0000 7f1d590e2b80 1 --2- 172.21.15.64:0/2077743926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55967becb710 0x55967becbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.826+0000 7f1d590e2b80 1 --2- 172.21.15.64:0/2077743926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55967becc030 0x55967bed9d10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.826+0000 7f1d590e2b80 1 -- 172.21.15.64:0/2077743926 >> 172.21.15.64:0/2077743926 conn(0x55967be40090 msgr2=0x55967bec6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:43.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.826+0000 7f1d590e2b80 1 -- 172.21.15.64:0/2077743926 shutdown_connections 2023-09-13T03:14:43.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.826+0000 7f1d590e2b80 1 -- 172.21.15.64:0/2077743926 wait complete. 2023-09-13T03:14:43.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.827+0000 7f1d590e2b80 1 -- start start 2023-09-13T03:14:43.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.827+0000 7f1d590e2b80 1 Processor -- start 2023-09-13T03:14:43.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.828+0000 7f1d590e2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55967becb710 0x55967bed9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.828+0000 7f1d590e2b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55967becc030 0x55967bed6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.828+0000 7f1d590e2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55967beda250 0x55967bed7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.828+0000 7f1d590e2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967bdbbb70 con 0x55967becb710 2023-09-13T03:14:43.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.828+0000 7f1d590e2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967bdbb870 con 0x55967beda250 2023-09-13T03:14:43.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.828+0000 7f1d590e2b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967bc9f920 con 0x55967becc030 2023-09-13T03:14:43.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.828+0000 7f1d590e2b80 10 client.0 _ll_register_callbacks cb 0x55967bd05a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:14:43.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.828+0000 7f1d3e7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55967becb710 0x55967bed9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.828+0000 7f1d3effd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55967becc030 0x55967bed6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.828+0000 7f1d3effd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55967becc030 0x55967bed6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:47554/0 (socket says 172.21.15.64:47554) 2023-09-13T03:14:43.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.828+0000 7f1d3effd700 1 -- 172.21.15.64:0/2957487231 learned_addr learned my addr 172.21.15.64:0/2957487231 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:43.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.828+0000 7f1d3e7fc700 1 --2- 172.21.15.64:0/2957487231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55967beda250 0x55967bed7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.828+0000 7f1d3effd700 1 -- 172.21.15.64:0/2957487231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55967beda250 msgr2=0x55967bed7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:43.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.828+0000 7f1d3effd700 1 --2- 172.21.15.64:0/2957487231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55967beda250 0x55967bed7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.828+0000 7f1d3effd700 1 -- 172.21.15.64:0/2957487231 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55967becb710 msgr2=0x55967bed9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:43.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.828+0000 7f1d3effd700 1 --2- 172.21.15.64:0/2957487231 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55967becb710 0x55967bed9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.828+0000 7f1d3effd700 1 -- 172.21.15.64:0/2957487231 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d300099e0 con 0x55967becc030 2023-09-13T03:14:43.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.829+0000 7f1d3effd700 1 --2- 172.21.15.64:0/2957487231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55967becc030 0x55967bed6e30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1d300038b0 tx=0x7f1d300038e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dc77d1250ada00 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:43.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.829+0000 7f1d22ff5700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:14:43.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.829+0000 7f1d22ff5700 1 -- 172.21.15.64:0/2957487231 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d30005240 con 0x55967becc030 2023-09-13T03:14:43.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.829+0000 7f1d22ff5700 1 -- 172.21.15.64:0/2957487231 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f1d30004500 con 0x55967becc030 2023-09-13T03:14:43.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.829+0000 7f1d590e2b80 1 -- 172.21.15.64:0/2957487231 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55967bd754f0 con 0x55967becc030 2023-09-13T03:14:43.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.829+0000 7f1d22ff5700 1 -- 172.21.15.64:0/2957487231 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d30005ce0 con 0x55967becc030 2023-09-13T03:14:43.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.829+0000 7f1d590e2b80 10 client.26194 fetch_fsmap learned FSMap version 32 2023-09-13T03:14:43.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.829+0000 7f1d590e2b80 1 -- 172.21.15.64:0/2957487231 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55967bc32c60 con 0x55967becc030 2023-09-13T03:14:43.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.830+0000 7f1d22ff5700 1 -- 172.21.15.64:0/2957487231 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55967bd754f0 con 0x55967becc030 2023-09-13T03:14:43.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.830+0000 7f1d22ff5700 1 -- 172.21.15.64:0/2957487231 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 32) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f1d30021040 con 0x55967becc030 2023-09-13T03:14:43.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.830+0000 7f1d590e2b80 10 client.26194 fetch_fsmap finished waiting for FSMap version 32 2023-09-13T03:14:43.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.830+0000 7f1d590e2b80 10 client.26194 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-09-13T03:14:43.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.830+0000 7f1d590e2b80 20 client.26194 populate_metadata read hostname 'smithi064' 2023-09-13T03:14:43.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.830+0000 7f1d590e2b80 1 --2- 172.21.15.64:0/2957487231 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x55967bee9c40 0x55967beec060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.830+0000 7f1d590e2b80 4 client.26194 mds_command: new command op to 24635 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:14:43.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.830+0000 7f1d590e2b80 1 -- 172.21.15.64:0/2957487231 --> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55967be328a0 con 0x55967bee9c40 2023-09-13T03:14:43.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.830+0000 7f1d3e7fc700 1 --2- 172.21.15.64:0/2957487231 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x55967bee9c40 0x55967beec060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.831+0000 7f1d3e7fc700 1 --2- 172.21.15.64:0/2957487231 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x55967bee9c40 0x55967beec060 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fef9629f5191f848 server_cookie=5ad9eb99468f0aa9 in_seq=0 out_seq=0 2023-09-13T03:14:43.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.831+0000 7f1d22ff5700 10 client.26194 ms_handle_connect on v2:172.21.15.142:6836/741173268 2023-09-13T03:14:43.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.831+0000 7f1d22ff5700 1 -- 172.21.15.64:0/2957487231 <== mds.2 v2:172.21.15.142:6836/741173268 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55967be328a0 con 0x55967bee9c40 2023-09-13T03:14:43.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.831+0000 7f1d22ff5700 10 client.26194 handle_command_reply: tid=0 2023-09-13T03:14:43.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.831+0000 7f1d590e2b80 1 client.26194 shutdown 2023-09-13T03:14:43.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.831+0000 7f1d590e2b80 1 -- 172.21.15.64:0/2957487231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55967becc030 msgr2=0x55967bed6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:43.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.831+0000 7f1d590e2b80 1 --2- 172.21.15.64:0/2957487231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55967becc030 0x55967bed6e30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1d300038b0 tx=0x7f1d300038e0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.832+0000 7f1d590e2b80 1 -- 172.21.15.64:0/2957487231 shutdown_connections 2023-09-13T03:14:43.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.832+0000 7f1d590e2b80 1 --2- 172.21.15.64:0/2957487231 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x55967bee9c40 0x55967beec060 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.832+0000 7f1d590e2b80 1 --2- 172.21.15.64:0/2957487231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55967beda250 0x55967bed7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.832+0000 7f1d590e2b80 1 --2- 172.21.15.64:0/2957487231 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55967becb710 0x55967bed9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.832+0000 7f1d590e2b80 1 --2- 172.21.15.64:0/2957487231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55967becc030 0x55967bed6e30 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.832+0000 7f1d22ff5700 0 client.26194 ms_handle_reset on v2:172.21.15.142:6836/741173268 2023-09-13T03:14:43.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.832+0000 7f1d590e2b80 1 -- 172.21.15.64:0/2957487231 >> 172.21.15.64:0/2957487231 conn(0x55967be40090 msgr2=0x55967bec67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:43.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.832+0000 7f1d590e2b80 1 -- 172.21.15.64:0/2957487231 shutdown_connections 2023-09-13T03:14:43.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.832+0000 7f1d590e2b80 1 -- 172.21.15.64:0/2957487231 wait complete. 2023-09-13T03:14:43.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.832+0000 7f1d590e2b80 20 client.26194 trim_cache size 0 max 16384 2023-09-13T03:14:43.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.845+0000 7f1d590e2b80 1 Processor -- start 2023-09-13T03:14:43.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.846+0000 7f1d590e2b80 1 -- start start 2023-09-13T03:14:43.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.846+0000 7f1d590e2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55967becb710 0x55967bed6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.846+0000 7f1d590e2b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55967becc030 0x55967bed66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.846+0000 7f1d590e2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55967beda250 0x55967bee6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.846+0000 7f1d590e2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967bed7a20 con 0x55967becb710 2023-09-13T03:14:43.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.846+0000 7f1d590e2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967bec76e0 con 0x55967beda250 2023-09-13T03:14:43.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.846+0000 7f1d590e2b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967bee6550 con 0x55967becc030 2023-09-13T03:14:43.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.846+0000 7f1d3e7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55967becb710 0x55967bed6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.846+0000 7f1d3e7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55967becb710 0x55967bed6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37498/0 (socket says 172.21.15.64:37498) 2023-09-13T03:14:43.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.846+0000 7f1d3effd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55967becc030 0x55967bed66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.846+0000 7f1d3e7fc700 1 -- 172.21.15.64:0/1671202719 learned_addr learned my addr 172.21.15.64:0/1671202719 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:43.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.846+0000 7f1d3e7fc700 1 --2- 172.21.15.64:0/1671202719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55967beda250 0x55967bee6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.847+0000 7f1d3e7fc700 1 -- 172.21.15.64:0/1671202719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55967becc030 msgr2=0x55967bed66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:43.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.847+0000 7f1d3e7fc700 1 --2- 172.21.15.64:0/1671202719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55967becc030 0x55967bed66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.847+0000 7f1d3e7fc700 1 -- 172.21.15.64:0/1671202719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55967beda250 msgr2=0x55967bee6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:43.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.847+0000 7f1d3e7fc700 1 --2- 172.21.15.64:0/1671202719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55967beda250 0x55967bee6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.847+0000 7f1d3e7fc700 1 -- 172.21.15.64:0/1671202719 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55967bc70250 con 0x55967becb710 2023-09-13T03:14:43.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.847+0000 7f1d3effd700 1 --2- 172.21.15.64:0/1671202719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55967becc030 0x55967bed66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:43.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.847+0000 7f1d3e7fc700 1 --2- 172.21.15.64:0/1671202719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55967becb710 0x55967bed6df0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1d3400b6f0 tx=0x7f1d3400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5898790d220647e1 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:43.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.847+0000 7f1d25ffb700 1 -- 172.21.15.64:0/1671202719 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3401d040 con 0x55967becb710 2023-09-13T03:14:43.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.847+0000 7f1d25ffb700 1 -- 172.21.15.64:0/1671202719 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d340056a0 con 0x55967becb710 2023-09-13T03:14:43.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.847+0000 7f1d25ffb700 1 -- 172.21.15.64:0/1671202719 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d34017a60 con 0x55967becb710 2023-09-13T03:14:43.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.848+0000 7f1d590e2b80 1 -- 172.21.15.64:0/1671202719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55967becb710 msgr2=0x55967bed6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:43.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.848+0000 7f1d590e2b80 1 --2- 172.21.15.64:0/1671202719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55967becb710 0x55967bed6df0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1d3400b6f0 tx=0x7f1d3400b8f0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.848+0000 7f1d590e2b80 1 -- 172.21.15.64:0/1671202719 shutdown_connections 2023-09-13T03:14:43.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.848+0000 7f1d590e2b80 1 --2- 172.21.15.64:0/1671202719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55967beda250 0x55967bee6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.848+0000 7f1d590e2b80 1 --2- 172.21.15.64:0/1671202719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55967becb710 0x55967bed6df0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.848+0000 7f1d590e2b80 1 --2- 172.21.15.64:0/1671202719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55967becc030 0x55967bed66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.848+0000 7f1d590e2b80 1 -- 172.21.15.64:0/1671202719 >> 172.21.15.64:0/1671202719 conn(0x55967be40090 msgr2=0x55967bef09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:43.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.848+0000 7f1d590e2b80 1 -- 172.21.15.64:0/1671202719 shutdown_connections 2023-09-13T03:14:43.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.848+0000 7f1d590e2b80 1 -- 172.21.15.64:0/1671202719 wait complete. 2023-09-13T03:14:43.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.848+0000 7f1d590e2b80 1 -- start start 2023-09-13T03:14:43.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.849+0000 7f1d590e2b80 1 Processor -- start 2023-09-13T03:14:43.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.849+0000 7f1d590e2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55967becb710 0x55967bee07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.849+0000 7f1d590e2b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55967becc030 0x55967bee0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.849+0000 7f1d590e2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55967beda250 0x55967bedec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.849+0000 7f1d590e2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967bedf1e0 con 0x55967becb710 2023-09-13T03:14:43.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.849+0000 7f1d590e2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967bedf320 con 0x55967beda250 2023-09-13T03:14:43.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.849+0000 7f1d590e2b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967bedf4e0 con 0x55967becc030 2023-09-13T03:14:43.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.849+0000 7f1d590e2b80 10 client.0 _ll_register_callbacks cb 0x7f1d30009c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:14:43.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.849+0000 7f1d3e7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55967becb710 0x55967bee07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.850+0000 7f1d3e7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55967becb710 0x55967bee07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37514/0 (socket says 172.21.15.64:37514) 2023-09-13T03:14:43.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.850+0000 7f1d3e7fc700 1 -- 172.21.15.64:0/2454064165 learned_addr learned my addr 172.21.15.64:0/2454064165 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:43.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.850+0000 7f1d3effd700 1 --2- 172.21.15.64:0/2454064165 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55967becc030 0x55967bee0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.850+0000 7f1d3e7fc700 1 --2- 172.21.15.64:0/2454064165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55967beda250 0x55967bedec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.850+0000 7f1d3effd700 1 -- 172.21.15.64:0/2454064165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55967beda250 msgr2=0x55967bedec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:43.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.850+0000 7f1d3effd700 1 --2- 172.21.15.64:0/2454064165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55967beda250 0x55967bedec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.850+0000 7f1d3effd700 1 -- 172.21.15.64:0/2454064165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55967becb710 msgr2=0x55967bee07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:43.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.850+0000 7f1d3effd700 1 --2- 172.21.15.64:0/2454064165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55967becb710 0x55967bee07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.850+0000 7f1d3effd700 1 -- 172.21.15.64:0/2454064165 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55967bc32c60 con 0x55967becc030 2023-09-13T03:14:43.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.850+0000 7f1d3effd700 1 --2- 172.21.15.64:0/2454064165 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55967becc030 0x55967bee0d00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1d30000c00 tx=0x7f1d30005fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d880e26889205a46 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:43.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.850+0000 7f1d22ff5700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:14:43.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.850+0000 7f1d22ff5700 1 -- 172.21.15.64:0/2454064165 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d30009cb0 con 0x55967becc030 2023-09-13T03:14:43.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.850+0000 7f1d590e2b80 1 -- 172.21.15.64:0/2454064165 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1d30004390 con 0x55967becc030 2023-09-13T03:14:43.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.850+0000 7f1d22ff5700 1 -- 172.21.15.64:0/2454064165 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f1d30000e40 con 0x55967becc030 2023-09-13T03:14:43.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.851+0000 7f1d22ff5700 1 -- 172.21.15.64:0/2454064165 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d300234b0 con 0x55967becc030 2023-09-13T03:14:43.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.851+0000 7f1d590e2b80 10 client.26200 fetch_fsmap learned FSMap version 32 2023-09-13T03:14:43.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.851+0000 7f1d590e2b80 1 -- 172.21.15.64:0/2454064165 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f1d34020040 con 0x55967becc030 2023-09-13T03:14:43.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.851+0000 7f1d22ff5700 1 -- 172.21.15.64:0/2454064165 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d30004390 con 0x55967becc030 2023-09-13T03:14:43.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.851+0000 7f1d22ff5700 1 -- 172.21.15.64:0/2454064165 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 32) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f1d30011840 con 0x55967becc030 2023-09-13T03:14:43.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.851+0000 7f1d590e2b80 10 client.26200 fetch_fsmap finished waiting for FSMap version 32 2023-09-13T03:14:43.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.851+0000 7f1d590e2b80 10 client.26200 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-09-13T03:14:43.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.851+0000 7f1d590e2b80 20 client.26200 populate_metadata read hostname 'smithi064' 2023-09-13T03:14:43.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.851+0000 7f1d590e2b80 1 --2- 172.21.15.64:0/2454064165 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x55967bf3e8b0 0x55967bf40cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:43.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.851+0000 7f1d590e2b80 4 client.26200 mds_command: new command op to 24635 tid=0[{"prefix": "damage ls"}] 2023-09-13T03:14:43.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.851+0000 7f1d590e2b80 1 -- 172.21.15.64:0/2454064165 --> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55967be16920 con 0x55967bf3e8b0 2023-09-13T03:14:43.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.851+0000 7f1d3e7fc700 1 --2- 172.21.15.64:0/2454064165 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x55967bf3e8b0 0x55967bf40cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:43.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.852+0000 7f1d3e7fc700 1 --2- 172.21.15.64:0/2454064165 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x55967bf3e8b0 0x55967bf40cd0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7f8c21e713dd0e7f server_cookie=add37fbcf9836320 in_seq=0 out_seq=0 2023-09-13T03:14:43.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.852+0000 7f1d22ff5700 10 client.26200 ms_handle_connect on v2:172.21.15.142:6836/741173268 2023-09-13T03:14:43.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.852+0000 7f1d22ff5700 1 -- 172.21.15.64:0/2454064165 <== mds.2 v2:172.21.15.142:6836/741173268 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7591 (crc 0 0 0) 0x55967be16920 con 0x55967bf3e8b0 2023-09-13T03:14:43.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.852+0000 7f1d22ff5700 10 client.26200 handle_command_reply: tid=0 2023-09-13T03:14:43.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.852+0000 7f1d590e2b80 1 client.26200 shutdown 2023-09-13T03:14:43.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.852+0000 7f1d590e2b80 1 -- 172.21.15.64:0/2454064165 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55967becc030 msgr2=0x55967bee0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:43.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.852+0000 7f1d590e2b80 1 --2- 172.21.15.64:0/2454064165 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55967becc030 0x55967bee0d00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1d30000c00 tx=0x7f1d30005fa0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.852+0000 7f1d590e2b80 1 -- 172.21.15.64:0/2454064165 shutdown_connections 2023-09-13T03:14:43.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.852+0000 7f1d590e2b80 1 --2- 172.21.15.64:0/2454064165 >> [v2:172.21.15.142:6836/741173268,v1:172.21.15.142:6837/741173268] conn(0x55967bf3e8b0 0x55967bf40cd0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.853+0000 7f1d590e2b80 1 --2- 172.21.15.64:0/2454064165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55967beda250 0x55967bedec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.853+0000 7f1d590e2b80 1 --2- 172.21.15.64:0/2454064165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55967becb710 0x55967bee07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.853+0000 7f1d22ff5700 0 client.26200 ms_handle_reset on v2:172.21.15.142:6836/741173268 2023-09-13T03:14:43.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.853+0000 7f1d590e2b80 1 --2- 172.21.15.64:0/2454064165 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55967becc030 0x55967bee0d00 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.853+0000 7f1d590e2b80 1 -- 172.21.15.64:0/2454064165 >> 172.21.15.64:0/2454064165 conn(0x55967be40090 msgr2=0x55967bed6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:43.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.853+0000 7f1d590e2b80 1 -- 172.21.15.64:0/2454064165 shutdown_connections 2023-09-13T03:14:43.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.853+0000 7f1d590e2b80 1 -- 172.21.15.64:0/2454064165 wait complete. 2023-09-13T03:14:43.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.853+0000 7f1d590e2b80 20 client.26200 trim_cache size 0 max 16384 2023-09-13T03:14:43.886 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2023-09-13T03:14:43.887 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:14:43.887 INFO:teuthology.orchestra.run.smithi064.stdout: "damage_type": "backtrace", 2023-09-13T03:14:43.887 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 10013249, 2023-09-13T03:14:43.887 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 1099511629151, 2023-09-13T03:14:43.887 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES1.XLS" 2023-09-13T03:14:43.888 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:14:43.888 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:14:43.888 INFO:teuthology.orchestra.run.smithi064.stdout: "damage_type": "backtrace", 2023-09-13T03:14:43.888 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 40339080, 2023-09-13T03:14:43.889 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 1099511628311, 2023-09-13T03:14:43.889 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG" 2023-09-13T03:14:43.889 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:14:43.889 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:14:43.890 INFO:teuthology.orchestra.run.smithi064.stdout: "damage_type": "backtrace", 2023-09-13T03:14:43.890 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 640923080, 2023-09-13T03:14:43.890 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 1099511628323, 2023-09-13T03:14:43.890 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/RESULTS.XLS" 2023-09-13T03:14:43.891 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:14:43.891 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:14:43.891 INFO:teuthology.orchestra.run.smithi064.stdout: "damage_type": "backtrace", 2023-09-13T03:14:43.891 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 675711619, 2023-09-13T03:14:43.892 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 1099511629165, 2023-09-13T03:14:43.892 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1A.PRN" 2023-09-13T03:14:43.892 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:14:43.893 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:14:43.893 INFO:teuthology.orchestra.run.smithi064.stdout: "damage_type": "backtrace", 2023-09-13T03:14:43.893 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 742061124, 2023-09-13T03:14:43.894 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 1099511628319, 2023-09-13T03:14:43.894 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT" 2023-09-13T03:14:43.894 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:14:43.895 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:14:43.895 INFO:teuthology.orchestra.run.smithi064.stdout: "damage_type": "backtrace", 2023-09-13T03:14:43.895 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 813913987, 2023-09-13T03:14:43.895 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 1099511629181, 2023-09-13T03:14:43.895 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPH1.CDR" 2023-09-13T03:14:43.896 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:14:43.896 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:14:43.896 INFO:teuthology.orchestra.run.smithi064.stdout: "damage_type": "backtrace", 2023-09-13T03:14:43.896 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 859347665, 2023-09-13T03:14:43.896 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 1099511628318, 2023-09-13T03:14:43.897 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR" 2023-09-13T03:14:43.897 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:14:43.897 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:14:43.897 INFO:teuthology.orchestra.run.smithi064.stdout: "damage_type": "backtrace", 2023-09-13T03:14:43.897 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 954864380, 2023-09-13T03:14:43.898 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 1099511629159, 2023-09-13T03:14:43.898 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS1.PRN" 2023-09-13T03:14:43.898 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:14:43.898 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:14:43.898 INFO:teuthology.orchestra.run.smithi064.stdout: "damage_type": "backtrace", 2023-09-13T03:14:43.899 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 1118582386, 2023-09-13T03:14:43.899 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 1099511628322, 2023-09-13T03:14:43.899 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS.LWP" 2023-09-13T03:14:43.899 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:14:43.900 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:14:43.900 INFO:teuthology.orchestra.run.smithi064.stdout: "damage_type": "backtrace", 2023-09-13T03:14:43.900 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 1403237912, 2023-09-13T03:14:43.900 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 1099511629168, 2023-09-13T03:14:43.901 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1A.LWP" 2023-09-13T03:14:43.901 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:14:43.901 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:14:43.901 INFO:teuthology.orchestra.run.smithi064.stdout: "damage_type": "backtrace", 2023-09-13T03:14:43.902 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 1765920269, 2023-09-13T03:14:43.902 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 1099511629180, 2023-09-13T03:14:43.902 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPH1.BAK" 2023-09-13T03:14:43.902 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:14:43.902 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:14:43.903 INFO:teuthology.orchestra.run.smithi064.stdout: "damage_type": "backtrace", 2023-09-13T03:14:43.903 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 1914177757, 2023-09-13T03:14:43.903 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 1099511628314, 2023-09-13T03:14:43.903 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT" 2023-09-13T03:14:43.903 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:14:43.904 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:14:43.904 INFO:teuthology.orchestra.run.smithi064.stdout: "damage_type": "backtrace", 2023-09-13T03:14:43.904 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 2084220324, 2023-09-13T03:14:43.904 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 1099511628316, 2023-09-13T03:14:43.904 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT" 2023-09-13T03:14:43.905 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:14:43.905 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:14:43.905 INFO:teuthology.orchestra.run.smithi064.stdout: "damage_type": "backtrace", 2023-09-13T03:14:43.905 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 2120875134, 2023-09-13T03:14:43.905 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 1099511629145, 2023-09-13T03:14:43.906 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/RESULTS2.PRN" 2023-09-13T03:14:43.906 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:14:43.906 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:14:43.906 INFO:teuthology.orchestra.run.smithi064.stdout: "damage_type": "backtrace", 2023-09-13T03:14:43.906 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 2237427117, 2023-09-13T03:14:43.907 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 1099511629157, 2023-09-13T03:14:43.907 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS1A.PRN" 2023-09-13T03:14:43.907 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:14:43.907 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:14:43.907 INFO:teuthology.orchestra.run.smithi064.stdout: "damage_type": "backtrace", 2023-09-13T03:14:43.908 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 2253223607, 2023-09-13T03:14:43.908 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 1099511629167, 2023-09-13T03:14:43.908 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1B.PRN" 2023-09-13T03:14:43.908 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:14:43.908 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:14:43.909 INFO:teuthology.orchestra.run.smithi064.stdout: "damage_type": "backtrace", 2023-09-13T03:14:43.909 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 2253847657, 2023-09-13T03:14:43.909 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 1099511628315, 2023-09-13T03:14:43.909 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT" 2023-09-13T03:14:43.910 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:14:43.910 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:14:43.910 INFO:teuthology.orchestra.run.smithi064.stdout: "damage_type": "backtrace", 2023-09-13T03:14:43.910 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 2303175674, 2023-09-13T03:14:43.911 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 1099511628310, 2023-09-13T03:14:43.911 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG" 2023-09-13T03:14:43.911 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:14:43.911 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:14:43.911 INFO:teuthology.orchestra.run.smithi064.stdout: "damage_type": "backtrace", 2023-09-13T03:14:43.912 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 2374204997, 2023-09-13T03:14:43.912 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 1099511628301, 2023-09-13T03:14:43.912 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS" 2023-09-13T03:14:43.912 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:14:43.912 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:14:43.913 INFO:teuthology.orchestra.run.smithi064.stdout: "damage_type": "backtrace", 2023-09-13T03:14:43.913 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 2418456771, 2023-09-13T03:14:43.913 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 1099511628302, 2023-09-13T03:14:43.913 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS" 2023-09-13T03:14:43.913 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:14:43.914 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:14:43.914 INFO:teuthology.orchestra.run.smithi064.stdout: "damage_type": "backtrace", 2023-09-13T03:14:43.914 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 2438912945, 2023-09-13T03:14:43.914 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 1099511629184, 2023-09-13T03:14:43.915 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPH2.BAK" 2023-09-13T03:14:43.915 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:14:43.915 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:14:43.915 INFO:teuthology.orchestra.run.smithi064.stdout: "damage_type": "backtrace", 2023-09-13T03:14:43.916 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 2505916318, 2023-09-13T03:14:43.916 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 1099511628317, 2023-09-13T03:14:43.916 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR" 2023-09-13T03:14:43.916 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:14:43.917 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:14:43.917 INFO:teuthology.orchestra.run.smithi064.stdout: "damage_type": "backtrace", 2023-09-13T03:14:43.917 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 2643194524, 2023-09-13T03:14:43.918 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 1099511629147, 2023-09-13T03:14:43.918 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/RESULTS3.PRN" 2023-09-13T03:14:43.918 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:14:43.918 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:14:43.919 INFO:teuthology.orchestra.run.smithi064.stdout: "damage_type": "backtrace", 2023-09-13T03:14:43.919 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 2644910402, 2023-09-13T03:14:43.919 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 1099511628312, 2023-09-13T03:14:43.919 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL" 2023-09-13T03:14:43.920 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:14:43.920 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:14:43.920 INFO:teuthology.orchestra.run.smithi064.stdout: "damage_type": "backtrace", 2023-09-13T03:14:43.920 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 3397672852, 2023-09-13T03:14:43.921 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 1099511629153, 2023-09-13T03:14:43.921 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES1.PRN" 2023-09-13T03:14:43.921 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:14:43.921 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:14:43.922 INFO:teuthology.orchestra.run.smithi064.stdout: "damage_type": "backtrace", 2023-09-13T03:14:43.922 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 3693290583, 2023-09-13T03:14:43.922 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 1099511629160, 2023-09-13T03:14:43.922 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS1.LWP" 2023-09-13T03:14:43.923 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:14:43.923 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:14:43.923 INFO:teuthology.orchestra.run.smithi064.stdout: "damage_type": "backtrace", 2023-09-13T03:14:43.923 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 3773396255, 2023-09-13T03:14:43.924 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 1099511628313, 2023-09-13T03:14:43.924 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR" 2023-09-13T03:14:43.924 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:14:43.924 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:14:43.924 INFO:teuthology.orchestra.run.smithi064.stdout: "damage_type": "backtrace", 2023-09-13T03:14:43.925 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 3869321397, 2023-09-13T03:14:43.925 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 1099511629185, 2023-09-13T03:14:43.925 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPH2.CDR" 2023-09-13T03:14:43.925 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:14:43.925 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:14:43.926 INFO:teuthology.orchestra.run.smithi064.stdout: "damage_type": "backtrace", 2023-09-13T03:14:43.926 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 3998708473, 2023-09-13T03:14:43.926 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 1099511629148, 2023-09-13T03:14:43.926 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/RESULTS.XLS" 2023-09-13T03:14:43.927 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:14:43.927 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:14:43.927 INFO:teuthology.orchestra.run.smithi064.stdout: "damage_type": "backtrace", 2023-09-13T03:14:43.927 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 4000931194, 2023-09-13T03:14:43.928 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 1099511629142, 2023-09-13T03:14:43.928 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/RESULTS1.PRN" 2023-09-13T03:14:43.928 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:14:43.928 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:14:43.928 INFO:teuthology.orchestra.run.smithi064.stdout: "damage_type": "backtrace", 2023-09-13T03:14:43.929 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 4029781391, 2023-09-13T03:14:43.929 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 1099511628320, 2023-09-13T03:14:43.929 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT" 2023-09-13T03:14:43.929 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:14:43.930 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:14:43.930 INFO:teuthology.orchestra.run.smithi064.stdout: "damage_type": "backtrace", 2023-09-13T03:14:43.930 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 4117661708, 2023-09-13T03:14:43.931 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 1099511628324, 2023-09-13T03:14:43.931 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1.LWP" 2023-09-13T03:14:43.931 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:14:43.932 INFO:teuthology.orchestra.run.smithi064.stdout:] 2023-09-13T03:14:43.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.853+0000 7f1d457ff700 1 -- 172.21.15.64:0/2759100353 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f1d280a07b0 msgr2=0x7f1d280a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:43.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.853+0000 7f1d457ff700 1 --2- 172.21.15.64:0/2759100353 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f1d280a07b0 0x7f1d280a2c70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1d3800c6f0 tx=0x7f1d38004dc0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.853+0000 7f1d457ff700 1 -- 172.21.15.64:0/2759100353 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d40106630 msgr2=0x7f1d4019afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:43.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.853+0000 7f1d457ff700 1 --2- 172.21.15.64:0/2759100353 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d40106630 0x7f1d4019afd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1d3800efc0 tx=0x7f1d3800c3a0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.853+0000 7f1d457ff700 1 -- 172.21.15.64:0/2759100353 shutdown_connections 2023-09-13T03:14:43.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.853+0000 7f1d457ff700 1 --2- 172.21.15.64:0/2759100353 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f1d280a07b0 0x7f1d280a2c70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.853+0000 7f1d457ff700 1 --2- 172.21.15.64:0/2759100353 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d40106630 0x7f1d4019afd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.853+0000 7f1d457ff700 1 --2- 172.21.15.64:0/2759100353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d40100570 0x7f1d40196630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.853+0000 7f1d457ff700 1 --2- 172.21.15.64:0/2759100353 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d40100f30 0x7f1d40196b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:43.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.854+0000 7f1d457ff700 1 -- 172.21.15.64:0/2759100353 >> 172.21.15.64:0/2759100353 conn(0x7f1d400fc080 msgr2=0x7f1d400fd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:43.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.854+0000 7f1d457ff700 1 -- 172.21.15.64:0/2759100353 shutdown_connections 2023-09-13T03:14:43.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:43.854+0000 7f1d457ff700 1 -- 172.21.15.64:0/2759100353 wait complete. 2023-09-13T03:14:43.936 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:43 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2570764870' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:14:43.937 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-09-13T03:14:43.938 INFO:tasks.workunit.client.0.smithi064.stdout: 1 28145 3.53 MB/sec warmup 48 sec latency 3774.511 ms 2023-09-13T03:14:43.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:43 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/2570764870' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:14:44.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.147+0000 7f0e339ff700 1 -- 172.21.15.64:0/287585649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e2c106650 msgr2=0x7f0e2c106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:44.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.147+0000 7f0e339ff700 1 --2- 172.21.15.64:0/287585649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e2c106650 0x7f0e2c106a30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0e2800b0a0 tx=0x7f0e2800b3b0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.147+0000 7f0e339ff700 1 -- 172.21.15.64:0/287585649 shutdown_connections 2023-09-13T03:14:44.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.147+0000 7f0e339ff700 1 --2- 172.21.15.64:0/287585649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e2c100f50 0x7f0e2c10d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.147+0000 7f0e339ff700 1 --2- 172.21.15.64:0/287585649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e2c106650 0x7f0e2c106a30 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.147+0000 7f0e339ff700 1 --2- 172.21.15.64:0/287585649 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e2c100590 0x7f0e2c100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.147+0000 7f0e339ff700 1 -- 172.21.15.64:0/287585649 >> 172.21.15.64:0/287585649 conn(0x7f0e2c0fc060 msgr2=0x7f0e2c0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:44.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.147+0000 7f0e339ff700 1 -- 172.21.15.64:0/287585649 shutdown_connections 2023-09-13T03:14:44.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.147+0000 7f0e339ff700 1 -- 172.21.15.64:0/287585649 wait complete. 2023-09-13T03:14:44.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.148+0000 7f0e339ff700 1 Processor -- start 2023-09-13T03:14:44.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.148+0000 7f0e339ff700 1 -- start start 2023-09-13T03:14:44.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.149+0000 7f0e339ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e2c100590 0x7f0e2c19e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:44.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.149+0000 7f0e339ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e2c100f50 0x7f0e2c19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:44.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.149+0000 7f0e339ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e2c106650 0x7f0e2c1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:44.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.149+0000 7f0e339ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c198f20 con 0x7f0e2c106650 2023-09-13T03:14:44.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.149+0000 7f0e339ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c199090 con 0x7f0e2c100f50 2023-09-13T03:14:44.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.149+0000 7f0e339ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c199250 con 0x7f0e2c100590 2023-09-13T03:14:44.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.149+0000 7f0e3179b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e2c100590 0x7f0e2c19e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:44.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.149+0000 7f0e30f9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e2c100f50 0x7f0e2c19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:44.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.149+0000 7f0e3179b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e2c100590 0x7f0e2c19e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:47574/0 (socket says 172.21.15.64:47574) 2023-09-13T03:14:44.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.149+0000 7f0e3179b700 1 -- 172.21.15.64:0/2610711801 learned_addr learned my addr 172.21.15.64:0/2610711801 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:44.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.149+0000 7f0e31f9c700 1 --2- 172.21.15.64:0/2610711801 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e2c106650 0x7f0e2c1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:44.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.149+0000 7f0e3179b700 1 -- 172.21.15.64:0/2610711801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e2c100f50 msgr2=0x7f0e2c19ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:44.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.149+0000 7f0e3179b700 1 --2- 172.21.15.64:0/2610711801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e2c100f50 0x7f0e2c19ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.149+0000 7f0e3179b700 1 -- 172.21.15.64:0/2610711801 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e2c106650 msgr2=0x7f0e2c1988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:44.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.149+0000 7f0e3179b700 1 --2- 172.21.15.64:0/2610711801 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e2c106650 0x7f0e2c1988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.149+0000 7f0e3179b700 1 -- 172.21.15.64:0/2610711801 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e1c009520 con 0x7f0e2c100590 2023-09-13T03:14:44.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.149+0000 7f0e30f9a700 1 --2- 172.21.15.64:0/2610711801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e2c100f50 0x7f0e2c19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:44.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.150+0000 7f0e3179b700 1 --2- 172.21.15.64:0/2610711801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e2c100590 0x7f0e2c19e840 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0e280062a0 tx=0x7f0e28007870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:44.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.150+0000 7f0e227fc700 1 -- 172.21.15.64:0/2610711801 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2800e040 con 0x7f0e2c100590 2023-09-13T03:14:44.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.150+0000 7f0e227fc700 1 -- 172.21.15.64:0/2610711801 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e2800be20 con 0x7f0e2c100590 2023-09-13T03:14:44.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.150+0000 7f0e227fc700 1 -- 172.21.15.64:0/2610711801 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2801b6c0 con 0x7f0e2c100590 2023-09-13T03:14:44.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.150+0000 7f0e339ff700 1 -- 172.21.15.64:0/2610711801 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e28009d00 con 0x7f0e2c100590 2023-09-13T03:14:44.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.150+0000 7f0e339ff700 1 -- 172.21.15.64:0/2610711801 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e2c1a5650 con 0x7f0e2c100590 2023-09-13T03:14:44.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.154+0000 7f0e227fc700 1 -- 172.21.15.64:0/2610711801 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f0e2801d470 con 0x7f0e2c100590 2023-09-13T03:14:44.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.155+0000 7f0e472f2b80 1 Processor -- start 2023-09-13T03:14:44.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.155+0000 7f0e472f2b80 1 -- start start 2023-09-13T03:14:44.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.155+0000 7f0e227fc700 1 --2- 172.21.15.64:0/2610711801 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f0e180a0830 0x7f0e180a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:44.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.155+0000 7f0e227fc700 1 -- 172.21.15.64:0/2610711801 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f0e280ccf10 con 0x7f0e2c100590 2023-09-13T03:14:44.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.155+0000 7f0e31f9c700 1 --2- 172.21.15.64:0/2610711801 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f0e180a0830 0x7f0e180a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:44.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.155+0000 7f0e472f2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3a22ef710 0x55e3a22efaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:44.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.155+0000 7f0e472f2b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e3a22f0030 0x55e3a22f9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:44.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.155+0000 7f0e472f2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3a22f9e40 0x55e3a22fc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:44.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.155+0000 7f0e472f2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3a2248730 con 0x55e3a22ef710 2023-09-13T03:14:44.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.155+0000 7f0e472f2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3a2246050 con 0x55e3a22f9e40 2023-09-13T03:14:44.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.155+0000 7f0e472f2b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3a2241940 con 0x55e3a22f0030 2023-09-13T03:14:44.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.156+0000 7f0e3179b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3a22ef710 0x55e3a22efaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:44.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.156+0000 7f0e31f9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3a22f9e40 0x55e3a22fc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:44.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.156+0000 7f0e3179b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3a22ef710 0x55e3a22efaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37534/0 (socket says 172.21.15.64:37534) 2023-09-13T03:14:44.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.156+0000 7f0e3179b700 1 -- 172.21.15.64:0/2705833688 learned_addr learned my addr 172.21.15.64:0/2705833688 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:44.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.156+0000 7f0e30f9a700 1 --2- 172.21.15.64:0/2705833688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e3a22f0030 0x55e3a22f9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:44.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.156+0000 7f0e3179b700 1 -- 172.21.15.64:0/2705833688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e3a22f0030 msgr2=0x55e3a22f9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:44.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.156+0000 7f0e3179b700 1 --2- 172.21.15.64:0/2705833688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e3a22f0030 0x55e3a22f9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.156+0000 7f0e3179b700 1 -- 172.21.15.64:0/2705833688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3a22f9e40 msgr2=0x55e3a22fc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:44.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.156+0000 7f0e3179b700 1 --2- 172.21.15.64:0/2705833688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3a22f9e40 0x55e3a22fc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.156+0000 7f0e3179b700 1 -- 172.21.15.64:0/2705833688 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3a206dde0 con 0x55e3a22ef710 2023-09-13T03:14:44.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.156+0000 7f0e31f9c700 1 --2- 172.21.15.64:0/2705833688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3a22f9e40 0x55e3a22fc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:14:44.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.156+0000 7f0e31f9c700 1 --2- 172.21.15.64:0/2610711801 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f0e180a0830 0x7f0e180a2cf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c19f540 tx=0x7f0e24011820 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:44.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.156+0000 7f0e3179b700 1 --2- 172.21.15.64:0/2705833688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3a22ef710 0x55e3a22efaf0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f0e28007b00 tx=0x7f0e280968b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb4310be12f771e8 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:44.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.157+0000 7f0e177fe700 1 -- 172.21.15.64:0/2705833688 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2801aea0 con 0x55e3a22ef710 2023-09-13T03:14:44.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.157+0000 7f0e177fe700 1 -- 172.21.15.64:0/2705833688 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e280850d0 con 0x55e3a22ef710 2023-09-13T03:14:44.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.157+0000 7f0e177fe700 1 -- 172.21.15.64:0/2705833688 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e280a2420 con 0x55e3a22ef710 2023-09-13T03:14:44.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.157+0000 7f0e472f2b80 1 -- 172.21.15.64:0/2705833688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3a22ef710 msgr2=0x55e3a22efaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:44.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.157+0000 7f0e472f2b80 1 --2- 172.21.15.64:0/2705833688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3a22ef710 0x55e3a22efaf0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f0e28007b00 tx=0x7f0e280968b0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.157+0000 7f0e472f2b80 1 -- 172.21.15.64:0/2705833688 shutdown_connections 2023-09-13T03:14:44.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.157+0000 7f0e472f2b80 1 --2- 172.21.15.64:0/2705833688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3a22f9e40 0x55e3a22fc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.157+0000 7f0e472f2b80 1 --2- 172.21.15.64:0/2705833688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3a22ef710 0x55e3a22efaf0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.157+0000 7f0e472f2b80 1 --2- 172.21.15.64:0/2705833688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e3a22f0030 0x55e3a22f9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.157+0000 7f0e472f2b80 1 -- 172.21.15.64:0/2705833688 >> 172.21.15.64:0/2705833688 conn(0x55e3a2264090 msgr2=0x55e3a22eabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:44.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.158+0000 7f0e472f2b80 1 -- 172.21.15.64:0/2705833688 shutdown_connections 2023-09-13T03:14:44.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.158+0000 7f0e472f2b80 1 -- 172.21.15.64:0/2705833688 wait complete. 2023-09-13T03:14:44.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.158+0000 7f0e472f2b80 1 -- start start 2023-09-13T03:14:44.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.158+0000 7f0e472f2b80 1 Processor -- start 2023-09-13T03:14:44.167 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:43 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/2570764870' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:14:44.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.159+0000 7f0e472f2b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e3a22ef710 0x55e3a22f98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:44.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.159+0000 7f0e472f2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3a22f0030 0x55e3a22f6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:44.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.159+0000 7f0e472f2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3a22f9e40 0x55e3a22f7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:44.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.159+0000 7f0e472f2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3a21dfb70 con 0x55e3a22f9e40 2023-09-13T03:14:44.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.159+0000 7f0e472f2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3a21df870 con 0x55e3a22f0030 2023-09-13T03:14:44.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.159+0000 7f0e472f2b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3a20c3920 con 0x55e3a22ef710 2023-09-13T03:14:44.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.159+0000 7f0e472f2b80 10 client.0 _ll_register_callbacks cb 0x55e3a2129a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:14:44.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.159+0000 7f0e30f9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3a22f0030 0x55e3a22f6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:44.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.159+0000 7f0e3179b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e3a22ef710 0x55e3a22f98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:44.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.159+0000 7f0e30f9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3a22f0030 0x55e3a22f6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:36502/0 (socket says 172.21.15.64:36502) 2023-09-13T03:14:44.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.159+0000 7f0e30f9a700 1 -- 172.21.15.64:0/678081865 learned_addr learned my addr 172.21.15.64:0/678081865 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:44.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.160+0000 7f0e31f9c700 1 --2- 172.21.15.64:0/678081865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3a22f9e40 0x55e3a22f7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:44.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.160+0000 7f0e3179b700 1 -- 172.21.15.64:0/678081865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3a22f0030 msgr2=0x55e3a22f6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:44.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.160+0000 7f0e3179b700 1 --2- 172.21.15.64:0/678081865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3a22f0030 0x55e3a22f6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.160+0000 7f0e3179b700 1 -- 172.21.15.64:0/678081865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3a22f9e40 msgr2=0x55e3a22f7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:44.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.160+0000 7f0e3179b700 1 --2- 172.21.15.64:0/678081865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3a22f9e40 0x55e3a22f7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.160+0000 7f0e3179b700 1 -- 172.21.15.64:0/678081865 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e2c1a5650 con 0x55e3a22ef710 2023-09-13T03:14:44.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.160+0000 7f0e31f9c700 1 --2- 172.21.15.64:0/678081865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3a22f9e40 0x55e3a22f7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:14:44.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.160+0000 7f0e30f9a700 1 --2- 172.21.15.64:0/678081865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3a22f0030 0x55e3a22f6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:44.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.160+0000 7f0e3179b700 1 --2- 172.21.15.64:0/678081865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e3a22ef710 0x55e3a22f98b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0e28096940 tx=0x7f0e2809a950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4dbeb4910f87bb42 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:44.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.160+0000 7f0e14ff9700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:14:44.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.160+0000 7f0e14ff9700 1 -- 172.21.15.64:0/678081865 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2801aea0 con 0x55e3a22ef710 2023-09-13T03:14:44.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.160+0000 7f0e14ff9700 1 -- 172.21.15.64:0/678081865 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f0e2809ab40 con 0x55e3a22ef710 2023-09-13T03:14:44.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.160+0000 7f0e472f2b80 1 -- 172.21.15.64:0/678081865 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e3a21994f0 con 0x55e3a22ef710 2023-09-13T03:14:44.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.160+0000 7f0e14ff9700 1 -- 172.21.15.64:0/678081865 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e280a2420 con 0x55e3a22ef710 2023-09-13T03:14:44.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.160+0000 7f0e472f2b80 10 client.26212 fetch_fsmap learned FSMap version 32 2023-09-13T03:14:44.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.160+0000 7f0e472f2b80 1 -- 172.21.15.64:0/678081865 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55e3a2056c60 con 0x55e3a22ef710 2023-09-13T03:14:44.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.161+0000 7f0e14ff9700 1 -- 172.21.15.64:0/678081865 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e3a21994f0 con 0x55e3a22ef710 2023-09-13T03:14:44.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.161+0000 7f0e14ff9700 1 -- 172.21.15.64:0/678081865 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 32) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f0e280a1f90 con 0x55e3a22ef710 2023-09-13T03:14:44.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.161+0000 7f0e472f2b80 10 client.26212 fetch_fsmap finished waiting for FSMap version 32 2023-09-13T03:14:44.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.161+0000 7f0e472f2b80 10 client.26212 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-09-13T03:14:44.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.161+0000 7f0e472f2b80 20 client.26212 populate_metadata read hostname 'smithi064' 2023-09-13T03:14:44.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.161+0000 7f0e472f2b80 1 --2- 172.21.15.64:0/678081865 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x55e3a23092e0 0x55e3a230b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:44.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.161+0000 7f0e472f2b80 4 client.26212 mds_command: new command op to 24649 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:14:44.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.161+0000 7f0e472f2b80 1 -- 172.21.15.64:0/678081865 --> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e3a22568a0 con 0x55e3a23092e0 2023-09-13T03:14:44.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.161+0000 7f0e30f9a700 1 --2- 172.21.15.64:0/678081865 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x55e3a23092e0 0x55e3a230b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:44.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.162+0000 7f0e30f9a700 1 --2- 172.21.15.64:0/678081865 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x55e3a23092e0 0x55e3a230b6d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6cb67e72cf771e00 server_cookie=52a0ae4d9ba56641 in_seq=0 out_seq=0 2023-09-13T03:14:44.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.162+0000 7f0e14ff9700 10 client.26212 ms_handle_connect on v2:172.21.15.170:6838/2189703596 2023-09-13T03:14:44.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.163+0000 7f0e14ff9700 1 -- 172.21.15.64:0/678081865 <== mds.1 v2:172.21.15.170:6838/2189703596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e3a22568a0 con 0x55e3a23092e0 2023-09-13T03:14:44.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.163+0000 7f0e14ff9700 10 client.26212 handle_command_reply: tid=0 2023-09-13T03:14:44.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.163+0000 7f0e472f2b80 1 client.26212 shutdown 2023-09-13T03:14:44.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.163+0000 7f0e472f2b80 1 -- 172.21.15.64:0/678081865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e3a22ef710 msgr2=0x55e3a22f98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:44.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.163+0000 7f0e472f2b80 1 --2- 172.21.15.64:0/678081865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e3a22ef710 0x55e3a22f98b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0e28096940 tx=0x7f0e2809a950 comp rx=0 tx=0).stop 2023-09-13T03:14:44.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.163+0000 7f0e472f2b80 1 -- 172.21.15.64:0/678081865 shutdown_connections 2023-09-13T03:14:44.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.163+0000 7f0e472f2b80 1 --2- 172.21.15.64:0/678081865 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x55e3a23092e0 0x55e3a230b6d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.163+0000 7f0e472f2b80 1 --2- 172.21.15.64:0/678081865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3a22f9e40 0x55e3a22f7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.163+0000 7f0e472f2b80 1 --2- 172.21.15.64:0/678081865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e3a22ef710 0x55e3a22f98b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.163+0000 7f0e472f2b80 1 --2- 172.21.15.64:0/678081865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3a22f0030 0x55e3a22f6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.163+0000 7f0e472f2b80 1 -- 172.21.15.64:0/678081865 >> 172.21.15.64:0/678081865 conn(0x55e3a2264090 msgr2=0x55e3a22ea7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:44.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.163+0000 7f0e14ff9700 0 client.26212 ms_handle_reset on v2:172.21.15.170:6838/2189703596 2023-09-13T03:14:44.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.164+0000 7f0e472f2b80 1 -- 172.21.15.64:0/678081865 shutdown_connections 2023-09-13T03:14:44.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.164+0000 7f0e472f2b80 1 -- 172.21.15.64:0/678081865 wait complete. 2023-09-13T03:14:44.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.164+0000 7f0e472f2b80 20 client.26212 trim_cache size 0 max 16384 2023-09-13T03:14:44.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.177+0000 7f0e472f2b80 1 Processor -- start 2023-09-13T03:14:44.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.177+0000 7f0e472f2b80 1 -- start start 2023-09-13T03:14:44.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.178+0000 7f0e472f2b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e3a22ef710 0x55e3a22f7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:44.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.178+0000 7f0e472f2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3a22f0030 0x55e3a2300830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:44.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.178+0000 7f0e472f2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3a22f9e40 0x55e3a2304f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:44.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.178+0000 7f0e472f2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3a22ea4d0 con 0x55e3a22f9e40 2023-09-13T03:14:44.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.178+0000 7f0e472f2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3a230bd30 con 0x55e3a22f0030 2023-09-13T03:14:44.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.178+0000 7f0e472f2b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3a22f50b0 con 0x55e3a22ef710 2023-09-13T03:14:44.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.178+0000 7f0e30f9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3a22f0030 0x55e3a2300830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:44.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.178+0000 7f0e3179b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e3a22ef710 0x55e3a22f7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:44.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.178+0000 7f0e30f9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3a22f0030 0x55e3a2300830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:36512/0 (socket says 172.21.15.64:36512) 2023-09-13T03:14:44.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.178+0000 7f0e3179b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e3a22ef710 0x55e3a22f7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:47590/0 (socket says 172.21.15.64:47590) 2023-09-13T03:14:44.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.178+0000 7f0e30f9a700 1 -- 172.21.15.64:0/2695231222 learned_addr learned my addr 172.21.15.64:0/2695231222 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:44.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.178+0000 7f0e31f9c700 1 --2- 172.21.15.64:0/2695231222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3a22f9e40 0x55e3a2304f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:44.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.178+0000 7f0e30f9a700 1 -- 172.21.15.64:0/2695231222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e3a22ef710 msgr2=0x55e3a22f7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:44.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.178+0000 7f0e30f9a700 1 --2- 172.21.15.64:0/2695231222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e3a22ef710 0x55e3a22f7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.178+0000 7f0e30f9a700 1 -- 172.21.15.64:0/2695231222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3a22f9e40 msgr2=0x55e3a2304f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:44.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.178+0000 7f0e30f9a700 1 --2- 172.21.15.64:0/2695231222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3a22f9e40 0x55e3a2304f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.178+0000 7f0e30f9a700 1 -- 172.21.15.64:0/2695231222 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3a2094250 con 0x55e3a22f0030 2023-09-13T03:14:44.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.179+0000 7f0e30f9a700 1 --2- 172.21.15.64:0/2695231222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3a22f0030 0x55e3a2300830 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x55e3a22f9e10 tx=0x7f0e1c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ed198d0e9cae9be server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:44.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.179+0000 7f0e17fff700 1 -- 172.21.15.64:0/2695231222 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c017040 con 0x55e3a22f0030 2023-09-13T03:14:44.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.179+0000 7f0e17fff700 1 -- 172.21.15.64:0/2695231222 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e1c025070 con 0x55e3a22f0030 2023-09-13T03:14:44.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.179+0000 7f0e17fff700 1 -- 172.21.15.64:0/2695231222 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c005020 con 0x55e3a22f0030 2023-09-13T03:14:44.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.180+0000 7f0e472f2b80 1 -- 172.21.15.64:0/2695231222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3a22f0030 msgr2=0x55e3a2300830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:44.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.180+0000 7f0e472f2b80 1 --2- 172.21.15.64:0/2695231222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3a22f0030 0x55e3a2300830 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x55e3a22f9e10 tx=0x7f0e1c009260 comp rx=0 tx=0).stop 2023-09-13T03:14:44.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.180+0000 7f0e472f2b80 1 -- 172.21.15.64:0/2695231222 shutdown_connections 2023-09-13T03:14:44.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.180+0000 7f0e472f2b80 1 --2- 172.21.15.64:0/2695231222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3a22f9e40 0x55e3a2304f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.180+0000 7f0e472f2b80 1 --2- 172.21.15.64:0/2695231222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e3a22ef710 0x55e3a22f7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.180+0000 7f0e472f2b80 1 --2- 172.21.15.64:0/2695231222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3a22f0030 0x55e3a2300830 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.180+0000 7f0e472f2b80 1 -- 172.21.15.64:0/2695231222 >> 172.21.15.64:0/2695231222 conn(0x55e3a2264090 msgr2=0x55e3a22e9dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:44.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.180+0000 7f0e472f2b80 1 -- 172.21.15.64:0/2695231222 shutdown_connections 2023-09-13T03:14:44.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.180+0000 7f0e472f2b80 1 -- 172.21.15.64:0/2695231222 wait complete. 2023-09-13T03:14:44.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.180+0000 7f0e472f2b80 1 -- start start 2023-09-13T03:14:44.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.180+0000 7f0e472f2b80 1 Processor -- start 2023-09-13T03:14:44.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.181+0000 7f0e472f2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3a22ef710 0x55e3a23563a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:44.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.181+0000 7f0e472f2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3a22f0030 0x55e3a23568e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:44.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.181+0000 7f0e472f2b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e3a22f9e40 0x55e3a22fe810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:44.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.181+0000 7f0e472f2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3a22fede0 con 0x55e3a22f0030 2023-09-13T03:14:44.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.181+0000 7f0e472f2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3a22fef50 con 0x55e3a22ef710 2023-09-13T03:14:44.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.181+0000 7f0e472f2b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3a22ff110 con 0x55e3a22f9e40 2023-09-13T03:14:44.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.181+0000 7f0e472f2b80 10 client.0 _ll_register_callbacks cb 0x7f0e2809e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:14:44.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.181+0000 7f0e3179b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3a22ef710 0x55e3a23563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:44.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.181+0000 7f0e3179b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3a22ef710 0x55e3a23563a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:36516/0 (socket says 172.21.15.64:36516) 2023-09-13T03:14:44.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.181+0000 7f0e3179b700 1 -- 172.21.15.64:0/1240740008 learned_addr learned my addr 172.21.15.64:0/1240740008 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:44.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.181+0000 7f0e30f9a700 1 --2- 172.21.15.64:0/1240740008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3a22f0030 0x55e3a23568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:44.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.181+0000 7f0e31f9c700 1 --2- 172.21.15.64:0/1240740008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e3a22f9e40 0x55e3a22fe810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:44.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.182+0000 7f0e3179b700 1 -- 172.21.15.64:0/1240740008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e3a22f9e40 msgr2=0x55e3a22fe810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:44.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.182+0000 7f0e3179b700 1 --2- 172.21.15.64:0/1240740008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e3a22f9e40 0x55e3a22fe810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.182+0000 7f0e3179b700 1 -- 172.21.15.64:0/1240740008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3a22f0030 msgr2=0x55e3a23568e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:44.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.182+0000 7f0e3179b700 1 --2- 172.21.15.64:0/1240740008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3a22f0030 0x55e3a23568e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.182+0000 7f0e3179b700 1 -- 172.21.15.64:0/1240740008 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e1c012a00 con 0x55e3a22ef710 2023-09-13T03:14:44.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.182+0000 7f0e31f9c700 1 --2- 172.21.15.64:0/1240740008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e3a22f9e40 0x55e3a22fe810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:14:44.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.182+0000 7f0e3179b700 1 --2- 172.21.15.64:0/1240740008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3a22ef710 0x55e3a23563a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0e28096e60 tx=0x7f0e280ca730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9e539d6e8df7f34 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:44.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.182+0000 7f0e14ff9700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:14:44.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.182+0000 7f0e14ff9700 1 -- 172.21.15.64:0/1240740008 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2801aea0 con 0x55e3a22ef710 2023-09-13T03:14:44.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.182+0000 7f0e14ff9700 1 -- 172.21.15.64:0/1240740008 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f0e2809ab40 con 0x55e3a22ef710 2023-09-13T03:14:44.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.182+0000 7f0e472f2b80 1 -- 172.21.15.64:0/1240740008 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0e280850d0 con 0x55e3a22ef710 2023-09-13T03:14:44.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.182+0000 7f0e14ff9700 1 -- 172.21.15.64:0/1240740008 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e2801d910 con 0x55e3a22ef710 2023-09-13T03:14:44.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.182+0000 7f0e472f2b80 10 client.26219 fetch_fsmap learned FSMap version 32 2023-09-13T03:14:44.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.182+0000 7f0e472f2b80 1 -- 172.21.15.64:0/1240740008 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55e3a206c2b0 con 0x55e3a22ef710 2023-09-13T03:14:44.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.183+0000 7f0e14ff9700 1 -- 172.21.15.64:0/1240740008 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e280850d0 con 0x55e3a22ef710 2023-09-13T03:14:44.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.183+0000 7f0e14ff9700 1 -- 172.21.15.64:0/1240740008 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 32) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f0e280b81c0 con 0x55e3a22ef710 2023-09-13T03:14:44.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.183+0000 7f0e472f2b80 10 client.26219 fetch_fsmap finished waiting for FSMap version 32 2023-09-13T03:14:44.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.183+0000 7f0e472f2b80 10 client.26219 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-09-13T03:14:44.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.183+0000 7f0e472f2b80 20 client.26219 populate_metadata read hostname 'smithi064' 2023-09-13T03:14:44.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.183+0000 7f0e472f2b80 1 --2- 172.21.15.64:0/1240740008 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x55e3a235dcf0 0x55e3a23600e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:44.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.183+0000 7f0e472f2b80 4 client.26219 mds_command: new command op to 24649 tid=0[{"prefix": "damage ls"}] 2023-09-13T03:14:44.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.183+0000 7f0e472f2b80 1 -- 172.21.15.64:0/1240740008 --> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e3a223a920 con 0x55e3a235dcf0 2023-09-13T03:14:44.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.183+0000 7f0e30f9a700 1 --2- 172.21.15.64:0/1240740008 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x55e3a235dcf0 0x55e3a23600e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:44.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.183+0000 7f0e30f9a700 1 --2- 172.21.15.64:0/1240740008 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x55e3a235dcf0 0x55e3a23600e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e29d2439eaf99fb4 server_cookie=e894bf65d8f4048b in_seq=0 out_seq=0 2023-09-13T03:14:44.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.183+0000 7f0e14ff9700 10 client.26219 ms_handle_connect on v2:172.21.15.170:6838/2189703596 2023-09-13T03:14:44.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.184+0000 7f0e14ff9700 1 -- 172.21.15.64:0/1240740008 <== mds.1 v2:172.21.15.170:6838/2189703596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e3a223a920 con 0x55e3a235dcf0 2023-09-13T03:14:44.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.184+0000 7f0e14ff9700 10 client.26219 handle_command_reply: tid=0 2023-09-13T03:14:44.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.184+0000 7f0e472f2b80 1 client.26219 shutdown 2023-09-13T03:14:44.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.184+0000 7f0e472f2b80 1 -- 172.21.15.64:0/1240740008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3a22ef710 msgr2=0x55e3a23563a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:44.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.184+0000 7f0e472f2b80 1 --2- 172.21.15.64:0/1240740008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3a22ef710 0x55e3a23563a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0e28096e60 tx=0x7f0e280ca730 comp rx=0 tx=0).stop 2023-09-13T03:14:44.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.184+0000 7f0e472f2b80 1 -- 172.21.15.64:0/1240740008 shutdown_connections 2023-09-13T03:14:44.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.184+0000 7f0e472f2b80 1 --2- 172.21.15.64:0/1240740008 >> [v2:172.21.15.170:6838/2189703596,v1:172.21.15.170:6839/2189703596] conn(0x55e3a235dcf0 0x55e3a23600e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.184+0000 7f0e472f2b80 1 --2- 172.21.15.64:0/1240740008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3a22ef710 0x55e3a23563a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.184+0000 7f0e472f2b80 1 --2- 172.21.15.64:0/1240740008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e3a22f9e40 0x55e3a22fe810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.184+0000 7f0e472f2b80 1 --2- 172.21.15.64:0/1240740008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3a22f0030 0x55e3a23568e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.184+0000 7f0e472f2b80 1 -- 172.21.15.64:0/1240740008 >> 172.21.15.64:0/1240740008 conn(0x55e3a2264090 msgr2=0x55e3a2304f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:44.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.184+0000 7f0e14ff9700 0 client.26219 ms_handle_reset on v2:172.21.15.170:6838/2189703596 2023-09-13T03:14:44.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.184+0000 7f0e472f2b80 1 -- 172.21.15.64:0/1240740008 shutdown_connections 2023-09-13T03:14:44.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.184+0000 7f0e472f2b80 1 -- 172.21.15.64:0/1240740008 wait complete. 2023-09-13T03:14:44.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.185+0000 7f0e472f2b80 20 client.26219 trim_cache size 0 max 16384 2023-09-13T03:14:44.203 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2023-09-13T03:14:44.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.185+0000 7f0e339ff700 1 -- 172.21.15.64:0/2610711801 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f0e180a0830 msgr2=0x7f0e180a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:44.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.185+0000 7f0e339ff700 1 --2- 172.21.15.64:0/2610711801 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f0e180a0830 0x7f0e180a2cf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c19f540 tx=0x7f0e24011820 comp rx=0 tx=0).stop 2023-09-13T03:14:44.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.185+0000 7f0e339ff700 1 -- 172.21.15.64:0/2610711801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e2c100590 msgr2=0x7f0e2c19e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:44.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.185+0000 7f0e339ff700 1 --2- 172.21.15.64:0/2610711801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e2c100590 0x7f0e2c19e840 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0e280062a0 tx=0x7f0e28007870 comp rx=0 tx=0).stop 2023-09-13T03:14:44.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.185+0000 7f0e339ff700 1 -- 172.21.15.64:0/2610711801 shutdown_connections 2023-09-13T03:14:44.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.185+0000 7f0e339ff700 1 --2- 172.21.15.64:0/2610711801 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f0e180a0830 0x7f0e180a2cf0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.185+0000 7f0e339ff700 1 --2- 172.21.15.64:0/2610711801 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e2c106650 0x7f0e2c1988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.185+0000 7f0e339ff700 1 --2- 172.21.15.64:0/2610711801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e2c100590 0x7f0e2c19e840 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.185+0000 7f0e339ff700 1 --2- 172.21.15.64:0/2610711801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e2c100f50 0x7f0e2c19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.185+0000 7f0e339ff700 1 -- 172.21.15.64:0/2610711801 >> 172.21.15.64:0/2610711801 conn(0x7f0e2c0fc060 msgr2=0x7f0e2c072830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:44.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.185+0000 7f0e339ff700 1 -- 172.21.15.64:0/2610711801 shutdown_connections 2023-09-13T03:14:44.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.185+0000 7f0e339ff700 1 -- 172.21.15.64:0/2610711801 wait complete. 2023-09-13T03:14:44.207 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-09-13T03:14:44.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.414+0000 7fd3475ff700 1 -- 172.21.15.64:0/1795445811 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd340102090 msgr2=0x7fd34010a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:44.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.414+0000 7fd3475ff700 1 --2- 172.21.15.64:0/1795445811 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd340102090 0x7fd34010a570 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd330009990 tx=0x7fd330009ca0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.414+0000 7fd3475ff700 1 -- 172.21.15.64:0/1795445811 shutdown_connections 2023-09-13T03:14:44.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.414+0000 7fd3475ff700 1 --2- 172.21.15.64:0/1795445811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd34010ab40 0x7fd34010cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.414+0000 7fd3475ff700 1 --2- 172.21.15.64:0/1795445811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd340101770 0x7fd340101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.414+0000 7fd3475ff700 1 --2- 172.21.15.64:0/1795445811 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd340102090 0x7fd34010a570 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.414+0000 7fd3475ff700 1 -- 172.21.15.64:0/1795445811 >> 172.21.15.64:0/1795445811 conn(0x7fd3400faf50 msgr2=0x7fd3400fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:44.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.414+0000 7fd3475ff700 1 -- 172.21.15.64:0/1795445811 shutdown_connections 2023-09-13T03:14:44.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.414+0000 7fd3475ff700 1 -- 172.21.15.64:0/1795445811 wait complete. 2023-09-13T03:14:44.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.415+0000 7fd3475ff700 1 Processor -- start 2023-09-13T03:14:44.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.415+0000 7fd3475ff700 1 -- start start 2023-09-13T03:14:44.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.416+0000 7fd3475ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd340101770 0x7fd34019e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:44.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.416+0000 7fd3475ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd340102090 0x7fd34019ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:44.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.416+0000 7fd3475ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd34010ab40 0x7fd340198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:44.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.416+0000 7fd3475ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd340198dd0 con 0x7fd340101770 2023-09-13T03:14:44.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.416+0000 7fd3475ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd340198f40 con 0x7fd340102090 2023-09-13T03:14:44.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.416+0000 7fd3475ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd340199100 con 0x7fd34010ab40 2023-09-13T03:14:44.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.416+0000 7fd34539b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd340101770 0x7fd34019e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:44.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.416+0000 7fd344b9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd340102090 0x7fd34019ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:44.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.416+0000 7fd34539b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd340101770 0x7fd34019e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37560/0 (socket says 172.21.15.64:37560) 2023-09-13T03:14:44.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.416+0000 7fd344b9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd340102090 0x7fd34019ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:36528/0 (socket says 172.21.15.64:36528) 2023-09-13T03:14:44.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.416+0000 7fd34539b700 1 -- 172.21.15.64:0/4183995030 learned_addr learned my addr 172.21.15.64:0/4183995030 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:44.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.416+0000 7fd345b9c700 1 --2- 172.21.15.64:0/4183995030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd34010ab40 0x7fd340198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:44.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.417+0000 7fd34539b700 1 -- 172.21.15.64:0/4183995030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd34010ab40 msgr2=0x7fd340198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:44.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.417+0000 7fd34539b700 1 --2- 172.21.15.64:0/4183995030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd34010ab40 0x7fd340198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.417+0000 7fd34539b700 1 -- 172.21.15.64:0/4183995030 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd340102090 msgr2=0x7fd34019ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:44.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.417+0000 7fd34539b700 1 --2- 172.21.15.64:0/4183995030 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd340102090 0x7fd34019ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.417+0000 7fd34539b700 1 -- 172.21.15.64:0/4183995030 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3300095f0 con 0x7fd340101770 2023-09-13T03:14:44.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.417+0000 7fd344b9a700 1 --2- 172.21.15.64:0/4183995030 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd340102090 0x7fd34019ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:44.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.417+0000 7fd34539b700 1 --2- 172.21.15.64:0/4183995030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd340101770 0x7fd34019e6a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd33c00e9a0 tx=0x7fd33c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:44.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.417+0000 7fd3367fc700 1 -- 172.21.15.64:0/4183995030 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33c009790 con 0x7fd340101770 2023-09-13T03:14:44.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.417+0000 7fd3475ff700 1 -- 172.21.15.64:0/4183995030 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3401993e0 con 0x7fd340101770 2023-09-13T03:14:44.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.417+0000 7fd3367fc700 1 -- 172.21.15.64:0/4183995030 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd33c010430 con 0x7fd340101770 2023-09-13T03:14:44.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.417+0000 7fd3367fc700 1 -- 172.21.15.64:0/4183995030 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33c018680 con 0x7fd340101770 2023-09-13T03:14:44.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.417+0000 7fd3475ff700 1 -- 172.21.15.64:0/4183995030 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3401a5690 con 0x7fd340101770 2023-09-13T03:14:44.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.420+0000 7fd3367fc700 1 -- 172.21.15.64:0/4183995030 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fd33c0105a0 con 0x7fd340101770 2023-09-13T03:14:44.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.422+0000 7fd3367fc700 1 --2- 172.21.15.64:0/4183995030 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fd32c0a07b0 0x7fd32c0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:44.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.422+0000 7fd35aee8b80 1 Processor -- start 2023-09-13T03:14:44.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.422+0000 7fd3367fc700 1 -- 172.21.15.64:0/4183995030 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd33c014070 con 0x7fd340101770 2023-09-13T03:14:44.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.422+0000 7fd344b9a700 1 --2- 172.21.15.64:0/4183995030 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fd32c0a07b0 0x7fd32c0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:44.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.422+0000 7fd35aee8b80 1 -- start start 2023-09-13T03:14:44.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.422+0000 7fd35aee8b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560360248710 0x560360248af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:44.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.423+0000 7fd35aee8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560360249030 0x560360252900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:44.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.423+0000 7fd35aee8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560360252e40 0x560360255230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:44.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.423+0000 7fd35aee8b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603601a1730 con 0x560360248710 2023-09-13T03:14:44.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.423+0000 7fd35aee8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56036019f050 con 0x560360252e40 2023-09-13T03:14:44.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.423+0000 7fd35aee8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56036019a940 con 0x560360249030 2023-09-13T03:14:44.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.423+0000 7fd34539b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560360248710 0x560360248af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:44.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.423+0000 7fd345b9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560360252e40 0x560360255230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:44.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.423+0000 7fd34539b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560360248710 0x560360248af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37570/0 (socket says 172.21.15.64:37570) 2023-09-13T03:14:44.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.423+0000 7fd34539b700 1 -- 172.21.15.64:0/3290099378 learned_addr learned my addr 172.21.15.64:0/3290099378 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:44.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.423+0000 7fd344b9a700 1 --2- 172.21.15.64:0/3290099378 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560360249030 0x560360252900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:44.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.423+0000 7fd345b9c700 1 -- 172.21.15.64:0/3290099378 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560360249030 msgr2=0x560360252900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:44.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.423+0000 7fd345b9c700 1 --2- 172.21.15.64:0/3290099378 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560360249030 0x560360252900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.423+0000 7fd345b9c700 1 -- 172.21.15.64:0/3290099378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560360248710 msgr2=0x560360248af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:44.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.423+0000 7fd345b9c700 1 --2- 172.21.15.64:0/3290099378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560360248710 0x560360248af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.423+0000 7fd345b9c700 1 -- 172.21.15.64:0/3290099378 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56035ffc6de0 con 0x560360252e40 2023-09-13T03:14:44.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.423+0000 7fd344b9a700 1 --2- 172.21.15.64:0/4183995030 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fd32c0a07b0 0x7fd32c0a2c70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd340199b80 tx=0x7fd330005fb0 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:44.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.423+0000 7fd345b9c700 1 --2- 172.21.15.64:0/3290099378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560360252e40 0x560360255230 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd338011550 tx=0x7fd338009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=763aade533f4765c server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:44.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.424+0000 7fd32b7fe700 1 -- 172.21.15.64:0/3290099378 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33800e040 con 0x560360252e40 2023-09-13T03:14:44.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.424+0000 7fd32b7fe700 1 -- 172.21.15.64:0/3290099378 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd33800f040 con 0x560360252e40 2023-09-13T03:14:44.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.424+0000 7fd32b7fe700 1 -- 172.21.15.64:0/3290099378 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd338004430 con 0x560360252e40 2023-09-13T03:14:44.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.424+0000 7fd35aee8b80 1 -- 172.21.15.64:0/3290099378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560360252e40 msgr2=0x560360255230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:44.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.424+0000 7fd35aee8b80 1 --2- 172.21.15.64:0/3290099378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560360252e40 0x560360255230 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd338011550 tx=0x7fd338009920 comp rx=0 tx=0).stop 2023-09-13T03:14:44.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.424+0000 7fd35aee8b80 1 -- 172.21.15.64:0/3290099378 shutdown_connections 2023-09-13T03:14:44.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.424+0000 7fd35aee8b80 1 --2- 172.21.15.64:0/3290099378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560360252e40 0x560360255230 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.424+0000 7fd35aee8b80 1 --2- 172.21.15.64:0/3290099378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560360248710 0x560360248af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.424+0000 7fd35aee8b80 1 --2- 172.21.15.64:0/3290099378 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560360249030 0x560360252900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.425+0000 7fd35aee8b80 1 -- 172.21.15.64:0/3290099378 >> 172.21.15.64:0/3290099378 conn(0x5603601bd090 msgr2=0x560360243bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:44.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.425+0000 7fd35aee8b80 1 -- 172.21.15.64:0/3290099378 shutdown_connections 2023-09-13T03:14:44.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.425+0000 7fd35aee8b80 1 -- 172.21.15.64:0/3290099378 wait complete. 2023-09-13T03:14:44.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.425+0000 7fd35aee8b80 1 -- start start 2023-09-13T03:14:44.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.425+0000 7fd35aee8b80 1 Processor -- start 2023-09-13T03:14:44.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.426+0000 7fd35aee8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560360248710 0x560360252770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:44.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.426+0000 7fd35aee8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560360249030 0x56036024f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:44.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.426+0000 7fd35aee8b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560360252e40 0x56036024ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:44.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.426+0000 7fd35aee8b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560360138b70 con 0x560360252e40 2023-09-13T03:14:44.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.426+0000 7fd35aee8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560360138870 con 0x560360249030 2023-09-13T03:14:44.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.426+0000 7fd35aee8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56036001c920 con 0x560360248710 2023-09-13T03:14:44.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.426+0000 7fd35aee8b80 10 client.0 _ll_register_callbacks cb 0x560360082a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:14:44.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.426+0000 7fd344b9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560360249030 0x56036024f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:44.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.426+0000 7fd34539b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560360248710 0x560360252770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:44.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.426+0000 7fd344b9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560360249030 0x56036024f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:36538/0 (socket says 172.21.15.64:36538) 2023-09-13T03:14:44.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.426+0000 7fd344b9a700 1 -- 172.21.15.64:0/72448550 learned_addr learned my addr 172.21.15.64:0/72448550 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:44.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.426+0000 7fd345b9c700 1 --2- 172.21.15.64:0/72448550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560360252e40 0x56036024ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:44.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.426+0000 7fd344b9a700 1 -- 172.21.15.64:0/72448550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560360248710 msgr2=0x560360252770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:44.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.426+0000 7fd344b9a700 1 --2- 172.21.15.64:0/72448550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560360248710 0x560360252770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.426+0000 7fd344b9a700 1 -- 172.21.15.64:0/72448550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560360252e40 msgr2=0x56036024ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:44.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.426+0000 7fd344b9a700 1 --2- 172.21.15.64:0/72448550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560360252e40 0x56036024ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.427+0000 7fd344b9a700 1 -- 172.21.15.64:0/72448550 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd338011230 con 0x560360249030 2023-09-13T03:14:44.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.427+0000 7fd34539b700 1 --2- 172.21.15.64:0/72448550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560360248710 0x560360252770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:44.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.427+0000 7fd344b9a700 1 --2- 172.21.15.64:0/72448550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560360249030 0x56036024f9f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd3300098e0 tx=0x7fd33001a050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b0429bd509221c5 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:44.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.427+0000 7fd328ff9700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:14:44.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.427+0000 7fd328ff9700 1 -- 172.21.15.64:0/72448550 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd330004a30 con 0x560360249030 2023-09-13T03:14:44.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.427+0000 7fd35aee8b80 1 -- 172.21.15.64:0/72448550 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5603600f24f0 con 0x560360249030 2023-09-13T03:14:44.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.427+0000 7fd328ff9700 1 -- 172.21.15.64:0/72448550 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd330004b90 con 0x560360249030 2023-09-13T03:14:44.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.427+0000 7fd328ff9700 1 -- 172.21.15.64:0/72448550 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd330004d50 con 0x560360249030 2023-09-13T03:14:44.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.427+0000 7fd35aee8b80 10 client.26231 fetch_fsmap learned FSMap version 32 2023-09-13T03:14:44.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.427+0000 7fd35aee8b80 1 -- 172.21.15.64:0/72448550 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56035ffafc60 con 0x560360249030 2023-09-13T03:14:44.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.428+0000 7fd328ff9700 1 -- 172.21.15.64:0/72448550 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5603600f24f0 con 0x560360249030 2023-09-13T03:14:44.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.428+0000 7fd328ff9700 1 -- 172.21.15.64:0/72448550 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 32) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fd330024ba0 con 0x560360249030 2023-09-13T03:14:44.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.428+0000 7fd35aee8b80 10 client.26231 fetch_fsmap finished waiting for FSMap version 32 2023-09-13T03:14:44.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.428+0000 7fd35aee8b80 10 client.26231 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T03:14:44.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.428+0000 7fd35aee8b80 20 client.26231 populate_metadata read hostname 'smithi064' 2023-09-13T03:14:44.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.428+0000 7fd35aee8b80 1 --2- 172.21.15.64:0/72448550 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x560360262a00 0x560360264e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:44.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.428+0000 7fd35aee8b80 4 client.26231 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:14:44.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.428+0000 7fd35aee8b80 1 -- 172.21.15.64:0/72448550 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5603601af8a0 con 0x560360262a00 2023-09-13T03:14:44.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.428+0000 7fd34539b700 1 --2- 172.21.15.64:0/72448550 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x560360262a00 0x560360264e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:44.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.429+0000 7fd34539b700 1 --2- 172.21.15.64:0/72448550 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x560360262a00 0x560360264e20 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de7470d5636952e3 server_cookie=ca5c085a96e419b8 in_seq=0 out_seq=0 2023-09-13T03:14:44.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.429+0000 7fd328ff9700 10 client.26231 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:44.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.429+0000 7fd328ff9700 1 -- 172.21.15.64:0/72448550 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5603601af8a0 con 0x560360262a00 2023-09-13T03:14:44.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.429+0000 7fd328ff9700 10 client.26231 handle_command_reply: tid=0 2023-09-13T03:14:44.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.429+0000 7fd35aee8b80 1 client.26231 shutdown 2023-09-13T03:14:44.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.429+0000 7fd35aee8b80 1 -- 172.21.15.64:0/72448550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560360249030 msgr2=0x56036024f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:44.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.429+0000 7fd35aee8b80 1 --2- 172.21.15.64:0/72448550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560360249030 0x56036024f9f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd3300098e0 tx=0x7fd33001a050 comp rx=0 tx=0).stop 2023-09-13T03:14:44.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.430+0000 7fd35aee8b80 1 -- 172.21.15.64:0/72448550 shutdown_connections 2023-09-13T03:14:44.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.430+0000 7fd35aee8b80 1 --2- 172.21.15.64:0/72448550 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x560360262a00 0x560360264e20 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.430+0000 7fd35aee8b80 1 --2- 172.21.15.64:0/72448550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560360252e40 0x56036024ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.430+0000 7fd35aee8b80 1 --2- 172.21.15.64:0/72448550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560360248710 0x560360252770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.430+0000 7fd35aee8b80 1 --2- 172.21.15.64:0/72448550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560360249030 0x56036024f9f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.430+0000 7fd328ff9700 0 client.26231 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:44.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.430+0000 7fd35aee8b80 1 -- 172.21.15.64:0/72448550 >> 172.21.15.64:0/72448550 conn(0x5603601bd090 msgr2=0x560360243780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:44.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.430+0000 7fd35aee8b80 1 -- 172.21.15.64:0/72448550 shutdown_connections 2023-09-13T03:14:44.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.430+0000 7fd35aee8b80 1 -- 172.21.15.64:0/72448550 wait complete. 2023-09-13T03:14:44.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.430+0000 7fd35aee8b80 20 client.26231 trim_cache size 0 max 16384 2023-09-13T03:14:44.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.444+0000 7fd35aee8b80 1 Processor -- start 2023-09-13T03:14:44.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.444+0000 7fd35aee8b80 1 -- start start 2023-09-13T03:14:44.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.444+0000 7fd35aee8b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560360248710 0x560360259a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:44.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.444+0000 7fd35aee8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560360249030 0x56036025e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:44.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.444+0000 7fd35aee8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560360252e40 0x56036024ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:44.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.444+0000 7fd35aee8b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603602403d0 con 0x560360248710 2023-09-13T03:14:44.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.444+0000 7fd35aee8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560360250fd0 con 0x560360249030 2023-09-13T03:14:44.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.444+0000 7fd35aee8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560360259f70 con 0x560360252e40 2023-09-13T03:14:44.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.444+0000 7fd345b9c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560360252e40 0x56036024ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:44.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.444+0000 7fd34539b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560360248710 0x560360259a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:44.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.444+0000 7fd344b9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560360249030 0x56036025e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:44.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.445+0000 7fd345b9c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560360252e40 0x56036024ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:47662/0 (socket says 172.21.15.64:47662) 2023-09-13T03:14:44.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.445+0000 7fd345b9c700 1 -- 172.21.15.64:0/1377426059 learned_addr learned my addr 172.21.15.64:0/1377426059 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:44.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.445+0000 7fd344b9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560360249030 0x56036025e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:36548/0 (socket says 172.21.15.64:36548) 2023-09-13T03:14:44.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.445+0000 7fd345b9c700 1 -- 172.21.15.64:0/1377426059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560360249030 msgr2=0x56036025e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:44.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.445+0000 7fd345b9c700 1 --2- 172.21.15.64:0/1377426059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560360249030 0x56036025e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.445+0000 7fd345b9c700 1 -- 172.21.15.64:0/1377426059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560360248710 msgr2=0x560360259a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:44.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.445+0000 7fd345b9c700 1 --2- 172.21.15.64:0/1377426059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560360248710 0x560360259a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.445+0000 7fd345b9c700 1 -- 172.21.15.64:0/1377426059 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56035ffed250 con 0x560360252e40 2023-09-13T03:14:44.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.445+0000 7fd344b9a700 1 --2- 172.21.15.64:0/1377426059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560360249030 0x56036025e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:44.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.445+0000 7fd345b9c700 1 --2- 172.21.15.64:0/1377426059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560360252e40 0x56036024ad00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd338009690 tx=0x7fd338011e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cad12797a7bb58d2 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:44.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.445+0000 7fd32bfff700 1 -- 172.21.15.64:0/1377426059 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33800e040 con 0x560360252e40 2023-09-13T03:14:44.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.445+0000 7fd32bfff700 1 -- 172.21.15.64:0/1377426059 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3380093e0 con 0x560360252e40 2023-09-13T03:14:44.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.445+0000 7fd32bfff700 1 -- 172.21.15.64:0/1377426059 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd338005320 con 0x560360252e40 2023-09-13T03:14:44.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.446+0000 7fd35aee8b80 1 -- 172.21.15.64:0/1377426059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560360252e40 msgr2=0x56036024ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:44.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.446+0000 7fd35aee8b80 1 --2- 172.21.15.64:0/1377426059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560360252e40 0x56036024ad00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd338009690 tx=0x7fd338011e50 comp rx=0 tx=0).stop 2023-09-13T03:14:44.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.446+0000 7fd35aee8b80 1 -- 172.21.15.64:0/1377426059 shutdown_connections 2023-09-13T03:14:44.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.446+0000 7fd35aee8b80 1 --2- 172.21.15.64:0/1377426059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560360252e40 0x56036024ad00 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.446+0000 7fd35aee8b80 1 --2- 172.21.15.64:0/1377426059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560360248710 0x560360259a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.446+0000 7fd35aee8b80 1 --2- 172.21.15.64:0/1377426059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560360249030 0x56036025e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.446+0000 7fd35aee8b80 1 -- 172.21.15.64:0/1377426059 >> 172.21.15.64:0/1377426059 conn(0x5603601bd090 msgr2=0x56036025eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:44.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.446+0000 7fd35aee8b80 1 -- 172.21.15.64:0/1377426059 shutdown_connections 2023-09-13T03:14:44.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.446+0000 7fd35aee8b80 1 -- 172.21.15.64:0/1377426059 wait complete. 2023-09-13T03:14:44.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.446+0000 7fd35aee8b80 1 -- start start 2023-09-13T03:14:44.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.447+0000 7fd35aee8b80 1 Processor -- start 2023-09-13T03:14:44.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.447+0000 7fd35aee8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560360248710 0x560360259220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:44.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.447+0000 7fd35aee8b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560360249030 0x560360259760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:44.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.447+0000 7fd35aee8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560360252e40 0x560360257650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:44.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.447+0000 7fd35aee8b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560360257c20 con 0x560360249030 2023-09-13T03:14:44.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.447+0000 7fd35aee8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560360257d90 con 0x560360248710 2023-09-13T03:14:44.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.447+0000 7fd35aee8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560360257f50 con 0x560360252e40 2023-09-13T03:14:44.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.447+0000 7fd35aee8b80 10 client.0 _ll_register_callbacks cb 0x7fd33000f770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:14:44.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.447+0000 7fd34539b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560360248710 0x560360259220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:44.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.447+0000 7fd344b9a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560360249030 0x560360259760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:44.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.448+0000 7fd345b9c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560360252e40 0x560360257650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:44.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.448+0000 7fd344b9a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560360249030 0x560360259760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37592/0 (socket says 172.21.15.64:37592) 2023-09-13T03:14:44.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.448+0000 7fd345b9c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560360252e40 0x560360257650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:47674/0 (socket says 172.21.15.64:47674) 2023-09-13T03:14:44.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.448+0000 7fd34539b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560360248710 0x560360259220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:36550/0 (socket says 172.21.15.64:36550) 2023-09-13T03:14:44.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.448+0000 7fd344b9a700 1 -- 172.21.15.64:0/854819773 learned_addr learned my addr 172.21.15.64:0/854819773 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:44.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.448+0000 7fd345b9c700 1 -- 172.21.15.64:0/854819773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560360248710 msgr2=0x560360259220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:44.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.448+0000 7fd345b9c700 1 --2- 172.21.15.64:0/854819773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560360248710 0x560360259220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.448+0000 7fd345b9c700 1 -- 172.21.15.64:0/854819773 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560360249030 msgr2=0x560360259760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:44.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.448+0000 7fd345b9c700 1 --2- 172.21.15.64:0/854819773 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560360249030 0x560360259760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.448+0000 7fd345b9c700 1 -- 172.21.15.64:0/854819773 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56035ffafc60 con 0x560360252e40 2023-09-13T03:14:44.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.448+0000 7fd344b9a700 1 --2- 172.21.15.64:0/854819773 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560360249030 0x560360259760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:44.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.448+0000 7fd34539b700 1 --2- 172.21.15.64:0/854819773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560360248710 0x560360259220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:44.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.448+0000 7fd345b9c700 1 --2- 172.21.15.64:0/854819773 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560360252e40 0x560360257650 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd338009960 tx=0x7fd338004d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4735d463213f5d42 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:44.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.448+0000 7fd328ff9700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:14:44.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.448+0000 7fd328ff9700 1 -- 172.21.15.64:0/854819773 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33800e040 con 0x560360252e40 2023-09-13T03:14:44.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.448+0000 7fd328ff9700 1 -- 172.21.15.64:0/854819773 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd3380077f0 con 0x560360252e40 2023-09-13T03:14:44.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.448+0000 7fd35aee8b80 1 -- 172.21.15.64:0/854819773 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd33800f040 con 0x560360252e40 2023-09-13T03:14:44.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.449+0000 7fd328ff9700 1 -- 172.21.15.64:0/854819773 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3380079f0 con 0x560360252e40 2023-09-13T03:14:44.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.449+0000 7fd35aee8b80 10 client.26236 fetch_fsmap learned FSMap version 32 2023-09-13T03:14:44.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.449+0000 7fd35aee8b80 1 -- 172.21.15.64:0/854819773 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56035ffc6de0 con 0x560360252e40 2023-09-13T03:14:44.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.449+0000 7fd328ff9700 1 -- 172.21.15.64:0/854819773 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd33800f040 con 0x560360252e40 2023-09-13T03:14:44.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.449+0000 7fd328ff9700 1 -- 172.21.15.64:0/854819773 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 32) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fd33801b020 con 0x560360252e40 2023-09-13T03:14:44.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.449+0000 7fd35aee8b80 10 client.26236 fetch_fsmap finished waiting for FSMap version 32 2023-09-13T03:14:44.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.449+0000 7fd35aee8b80 10 client.26236 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-13T03:14:44.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.449+0000 7fd35aee8b80 20 client.26236 populate_metadata read hostname 'smithi064' 2023-09-13T03:14:44.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.449+0000 7fd35aee8b80 1 --2- 172.21.15.64:0/854819773 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5603602b7560 0x5603602b9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:44.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.449+0000 7fd35aee8b80 4 client.26236 mds_command: new command op to 24686 tid=0[{"prefix": "damage ls"}] 2023-09-13T03:14:44.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.449+0000 7fd35aee8b80 1 -- 172.21.15.64:0/854819773 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560360193920 con 0x5603602b7560 2023-09-13T03:14:44.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.450+0000 7fd34539b700 1 --2- 172.21.15.64:0/854819773 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5603602b7560 0x5603602b9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:44.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.450+0000 7fd34539b700 1 --2- 172.21.15.64:0/854819773 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5603602b7560 0x5603602b9950 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36b605b375893bce server_cookie=c2cf2a0cd269e56d in_seq=0 out_seq=0 2023-09-13T03:14:44.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.450+0000 7fd328ff9700 10 client.26236 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:44.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.451+0000 7fd328ff9700 1 -- 172.21.15.64:0/854819773 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560360193920 con 0x5603602b7560 2023-09-13T03:14:44.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.451+0000 7fd328ff9700 10 client.26236 handle_command_reply: tid=0 2023-09-13T03:14:44.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.451+0000 7fd35aee8b80 1 client.26236 shutdown 2023-09-13T03:14:44.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.451+0000 7fd35aee8b80 1 -- 172.21.15.64:0/854819773 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560360252e40 msgr2=0x560360257650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:44.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.451+0000 7fd35aee8b80 1 --2- 172.21.15.64:0/854819773 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560360252e40 0x560360257650 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd338009960 tx=0x7fd338004d60 comp rx=0 tx=0).stop 2023-09-13T03:14:44.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.451+0000 7fd35aee8b80 1 -- 172.21.15.64:0/854819773 shutdown_connections 2023-09-13T03:14:44.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.451+0000 7fd35aee8b80 1 --2- 172.21.15.64:0/854819773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560360248710 0x560360259220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.451+0000 7fd35aee8b80 1 --2- 172.21.15.64:0/854819773 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5603602b7560 0x5603602b9950 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.451+0000 7fd35aee8b80 1 --2- 172.21.15.64:0/854819773 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560360252e40 0x560360257650 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.451+0000 7fd328ff9700 0 client.26236 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:44.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.451+0000 7fd35aee8b80 1 --2- 172.21.15.64:0/854819773 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560360249030 0x560360259760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.451+0000 7fd35aee8b80 1 -- 172.21.15.64:0/854819773 >> 172.21.15.64:0/854819773 conn(0x5603601bd090 msgr2=0x56036025e180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:44.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.451+0000 7fd35aee8b80 1 -- 172.21.15.64:0/854819773 shutdown_connections 2023-09-13T03:14:44.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.451+0000 7fd35aee8b80 1 -- 172.21.15.64:0/854819773 wait complete. 2023-09-13T03:14:44.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.451+0000 7fd35aee8b80 20 client.26236 trim_cache size 0 max 16384 2023-09-13T03:14:44.473 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2023-09-13T03:14:44.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.452+0000 7fd3475ff700 1 -- 172.21.15.64:0/4183995030 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fd32c0a07b0 msgr2=0x7fd32c0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:44.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.452+0000 7fd3475ff700 1 --2- 172.21.15.64:0/4183995030 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fd32c0a07b0 0x7fd32c0a2c70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd340199b80 tx=0x7fd330005fb0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.452+0000 7fd3475ff700 1 -- 172.21.15.64:0/4183995030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd340101770 msgr2=0x7fd34019e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:44.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.452+0000 7fd3475ff700 1 --2- 172.21.15.64:0/4183995030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd340101770 0x7fd34019e6a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd33c00e9a0 tx=0x7fd33c00ecb0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.452+0000 7fd3475ff700 1 -- 172.21.15.64:0/4183995030 shutdown_connections 2023-09-13T03:14:44.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.452+0000 7fd3475ff700 1 --2- 172.21.15.64:0/4183995030 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fd32c0a07b0 0x7fd32c0a2c70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.452+0000 7fd3475ff700 1 --2- 172.21.15.64:0/4183995030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd34010ab40 0x7fd340198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.452+0000 7fd3475ff700 1 --2- 172.21.15.64:0/4183995030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd340101770 0x7fd34019e6a0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.452+0000 7fd3475ff700 1 --2- 172.21.15.64:0/4183995030 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd340102090 0x7fd34019ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:44.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.452+0000 7fd3475ff700 1 -- 172.21.15.64:0/4183995030 >> 172.21.15.64:0/4183995030 conn(0x7fd3400faf50 msgr2=0x7fd34010cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:44.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.452+0000 7fd3475ff700 1 -- 172.21.15.64:0/4183995030 shutdown_connections 2023-09-13T03:14:44.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:14:44.452+0000 7fd3475ff700 1 -- 172.21.15.64:0/4183995030 wait complete. 2023-09-13T03:14:44.477 ERROR:tasks.fwd_scrub:rank 2 damaged: [{'damage_type': 'backtrace', 'id': 10013249, 'ino': 1099511629151, 'path': '/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES1.XLS'}, {'damage_type': 'backtrace', 'id': 40339080, 'ino': 1099511628311, 'path': '/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG'}, {'damage_type': 'backtrace', 'id': 640923080, 'ino': 1099511628323, 'path': '/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/RESULTS.XLS'}, {'damage_type': 'backtrace', 'id': 675711619, 'ino': 1099511629165, 'path': '/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1A.PRN'}, {'damage_type': 'backtrace', 'id': 742061124, 'ino': 1099511628319, 'path': '/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT'}, {'damage_type': 'backtrace', 'id': 813913987, 'ino': 1099511629181, 'path': '/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPH1.CDR'}, {'damage_type': 'backtrace', 'id': 859347665, 'ino': 1099511628318, 'path': '/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR'}, {'damage_type': 'backtrace', 'id': 954864380, 'ino': 1099511629159, 'path': '/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS1.PRN'}, {'damage_type': 'backtrace', 'id': 1118582386, 'ino': 1099511628322, 'path': '/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS.LWP'}, {'damage_type': 'backtrace', 'id': 1403237912, 'ino': 1099511629168, 'path': '/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1A.LWP'}, {'damage_type': 'backtrace', 'id': 1765920269, 'ino': 1099511629180, 'path': '/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPH1.BAK'}, {'damage_type': 'backtrace', 'id': 1914177757, 'ino': 1099511628314, 'path': '/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT'}, {'damage_type': 'backtrace', 'id': 2084220324, 'ino': 1099511628316, 'path': '/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT'}, {'damage_type': 'backtrace', 'id': 2120875134, 'ino': 1099511629145, 'path': '/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/RESULTS2.PRN'}, {'damage_type': 'backtrace', 'id': 2237427117, 'ino': 1099511629157, 'path': '/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS1A.PRN'}, {'damage_type': 'backtrace', 'id': 2253223607, 'ino': 1099511629167, 'path': '/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1B.PRN'}, {'damage_type': 'backtrace', 'id': 2253847657, 'ino': 1099511628315, 'path': '/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT'}, {'damage_type': 'backtrace', 'id': 2303175674, 'ino': 1099511628310, 'path': '/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG'}, {'damage_type': 'backtrace', 'id': 2374204997, 'ino': 1099511628301, 'path': '/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS'}, {'damage_type': 'backtrace', 'id': 2418456771, 'ino': 1099511628302, 'path': '/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS'}, {'damage_type': 'backtrace', 'id': 2438912945, 'ino': 1099511629184, 'path': '/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPH2.BAK'}, {'damage_type': 'backtrace', 'id': 2505916318, 'ino': 1099511628317, 'path': '/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR'}, {'damage_type': 'backtrace', 'id': 2643194524, 'ino': 1099511629147, 'path': '/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/RESULTS3.PRN'}, {'damage_type': 'backtrace', 'id': 2644910402, 'ino': 1099511628312, 'path': '/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL'}, {'damage_type': 'backtrace', 'id': 3397672852, 'ino': 1099511629153, 'path': '/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES1.PRN'}, {'damage_type': 'backtrace', 'id': 3693290583, 'ino': 1099511629160, 'path': '/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS1.LWP'}, {'damage_type': 'backtrace', 'id': 3773396255, 'ino': 1099511628313, 'path': '/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR'}, {'damage_type': 'backtrace', 'id': 3869321397, 'ino': 1099511629185, 'path': '/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPH2.CDR'}, {'damage_type': 'backtrace', 'id': 3998708473, 'ino': 1099511629148, 'path': '/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/RESULTS.XLS'}, {'damage_type': 'backtrace', 'id': 4000931194, 'ino': 1099511629142, 'path': '/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/EXCEL/RESULTS1.PRN'}, {'damage_type': 'backtrace', 'id': 4029781391, 'ino': 1099511628320, 'path': '/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT'}, {'damage_type': 'backtrace', 'id': 4117661708, 'ino': 1099511628324, 'path': '/volumes/_nogroup/sv_1/54b26f1a-07a6-4d22-a4b6-2c425efa64ed/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1.LWP'}] 2023-09-13T03:14:44.478 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/github.com_batrick_ceph_3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/github.com_batrick_ceph_3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/github.com_batrick_ceph_3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f/qa/tasks/fwd_scrub.py", line 79, in _scrub self._check_damage() File "/home/teuthworker/src/github.com_batrick_ceph_3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f/qa/tasks/fwd_scrub.py", line 90, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2023-09-13T03:14:44.889 INFO:tasks.workunit.client.0.smithi064.stdout: 1 29207 3.56 MB/sec warmup 49 sec latency 98.862 ms 2023-09-13T03:14:44.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:44 smithi064 ceph-mon[97675]: pgmap v50: 97 pgs: 97 active+clean; 25 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 74 KiB/s wr, 31 op/s 2023-09-13T03:14:44.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:44 smithi142 ceph-mon[134082]: pgmap v50: 97 pgs: 97 active+clean; 25 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 74 KiB/s wr, 31 op/s 2023-09-13T03:14:45.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:44 smithi170 ceph-mon[134328]: pgmap v50: 97 pgs: 97 active+clean; 25 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 74 KiB/s wr, 31 op/s 2023-09-13T03:14:45.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.170+0000 7fb5ad5ff700 1 -- 172.21.15.64:0/2082213978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5a8106660 msgr2=0x7fb5a8106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.170+0000 7fb5ad5ff700 1 --2- 172.21.15.64:0/2082213978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5a8106660 0x7fb5a8106a40 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb5980099c0 tx=0x7fb598009cd0 comp rx=0 tx=0).stop 2023-09-13T03:14:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.170+0000 7fb5ad5ff700 1 -- 172.21.15.64:0/2082213978 shutdown_connections 2023-09-13T03:14:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.170+0000 7fb5ad5ff700 1 --2- 172.21.15.64:0/2082213978 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5a8100f60 0x7fb5a810d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.170+0000 7fb5ad5ff700 1 --2- 172.21.15.64:0/2082213978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5a8106660 0x7fb5a8106a40 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:45.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.170+0000 7fb5ad5ff700 1 --2- 172.21.15.64:0/2082213978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5a81005a0 0x7fb5a8100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:45.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.170+0000 7fb5ad5ff700 1 -- 172.21.15.64:0/2082213978 >> 172.21.15.64:0/2082213978 conn(0x7fb5a80fc050 msgr2=0x7fb5a80fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:45.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.170+0000 7fb5ad5ff700 1 -- 172.21.15.64:0/2082213978 shutdown_connections 2023-09-13T03:14:45.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.170+0000 7fb5ad5ff700 1 -- 172.21.15.64:0/2082213978 wait complete. 2023-09-13T03:14:45.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.171+0000 7fb5ad5ff700 1 Processor -- start 2023-09-13T03:14:45.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.171+0000 7fb5ad5ff700 1 -- start start 2023-09-13T03:14:45.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.171+0000 7fb5ad5ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5a81005a0 0x7fb5a819c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:45.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.171+0000 7fb5ad5ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5a8100f60 0x7fb5a819cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.172+0000 7fb5ad5ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5a8106660 0x7fb5a8196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.172+0000 7fb5ad5ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a8196ad0 con 0x7fb5a81005a0 2023-09-13T03:14:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.172+0000 7fb5ad5ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a8196c40 con 0x7fb5a8106660 2023-09-13T03:14:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.172+0000 7fb5ad5ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a8196e00 con 0x7fb5a8100f60 2023-09-13T03:14:45.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.172+0000 7fb5a6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5a81005a0 0x7fb5a819c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:45.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.172+0000 7fb5a6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5a81005a0 0x7fb5a819c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37614/0 (socket says 172.21.15.64:37614) 2023-09-13T03:14:45.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.172+0000 7fb5a6ffd700 1 -- 172.21.15.64:0/2559929141 learned_addr learned my addr 172.21.15.64:0/2559929141 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:45.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.172+0000 7fb59ffff700 1 --2- 172.21.15.64:0/2559929141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5a8100f60 0x7fb5a819cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:45.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.172+0000 7fb5a77fe700 1 --2- 172.21.15.64:0/2559929141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5a8106660 0x7fb5a8196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.172+0000 7fb5a6ffd700 1 -- 172.21.15.64:0/2559929141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5a8100f60 msgr2=0x7fb5a819cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.172+0000 7fb5a6ffd700 1 --2- 172.21.15.64:0/2559929141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5a8100f60 0x7fb5a819cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.172+0000 7fb5a6ffd700 1 -- 172.21.15.64:0/2559929141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5a8106660 msgr2=0x7fb5a8196460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.172+0000 7fb5a6ffd700 1 --2- 172.21.15.64:0/2559929141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5a8106660 0x7fb5a8196460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.172+0000 7fb5a6ffd700 1 -- 172.21.15.64:0/2559929141 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5980095f0 con 0x7fb5a81005a0 2023-09-13T03:14:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.172+0000 7fb5a6ffd700 1 --2- 172.21.15.64:0/2559929141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5a81005a0 0x7fb5a819c600 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb598000c00 tx=0x7fb59800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.173+0000 7fb5a4ff9700 1 -- 172.21.15.64:0/2559929141 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59801d070 con 0x7fb5a81005a0 2023-09-13T03:14:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.173+0000 7fb5a4ff9700 1 -- 172.21.15.64:0/2559929141 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb598005050 con 0x7fb5a81005a0 2023-09-13T03:14:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.173+0000 7fb5ad5ff700 1 -- 172.21.15.64:0/2559929141 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5a8197080 con 0x7fb5a81005a0 2023-09-13T03:14:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.173+0000 7fb5a4ff9700 1 -- 172.21.15.64:0/2559929141 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb598004ab0 con 0x7fb5a81005a0 2023-09-13T03:14:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.173+0000 7fb5ad5ff700 1 -- 172.21.15.64:0/2559929141 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5a81a35f0 con 0x7fb5a81005a0 2023-09-13T03:14:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.176+0000 7fb5a4ff9700 1 -- 172.21.15.64:0/2559929141 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fb5980051c0 con 0x7fb5a81005a0 2023-09-13T03:14:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.177+0000 7fb5c0ee5b80 1 Processor -- start 2023-09-13T03:14:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.177+0000 7fb5c0ee5b80 1 -- start start 2023-09-13T03:14:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.178+0000 7fb5c0ee5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643f6e15b0 0x55643f6e1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.178+0000 7fb5c0ee5b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55643f6db710 0x55643f6dbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.178+0000 7fb5c0ee5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55643f6dc0d0 0x55643f6e8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.178+0000 7fb5c0ee5b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643f6306d0 con 0x55643f6db710 2023-09-13T03:14:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.178+0000 7fb5c0ee5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643f62dff0 con 0x55643f6dc0d0 2023-09-13T03:14:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.178+0000 7fb5c0ee5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643f6298e0 con 0x55643f6e15b0 2023-09-13T03:14:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.178+0000 7fb5a4ff9700 1 --2- 172.21.15.64:0/2559929141 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fb5940a07b0 0x7fb5940a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.178+0000 7fb5a4ff9700 1 -- 172.21.15.64:0/2559929141 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fb5980ccca0 con 0x7fb5a81005a0 2023-09-13T03:14:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.178+0000 7fb5a6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55643f6db710 0x55643f6dbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:45.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.178+0000 7fb5a77fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643f6e15b0 0x55643f6e1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:45.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.178+0000 7fb59ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55643f6dc0d0 0x55643f6e8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:45.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.178+0000 7fb5a77fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643f6e15b0 0x55643f6e1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:47700/0 (socket says 172.21.15.64:47700) 2023-09-13T03:14:45.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.178+0000 7fb5a77fe700 1 -- 172.21.15.64:0/4053182258 learned_addr learned my addr 172.21.15.64:0/4053182258 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.178+0000 7fb5a77fe700 1 --2- 172.21.15.64:0/2559929141 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fb5940a07b0 0x7fb5940a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.179+0000 7fb59ffff700 1 -- 172.21.15.64:0/4053182258 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643f6e15b0 msgr2=0x55643f6e1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.179+0000 7fb59ffff700 1 --2- 172.21.15.64:0/4053182258 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643f6e15b0 0x55643f6e1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.179+0000 7fb59ffff700 1 -- 172.21.15.64:0/4053182258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55643f6db710 msgr2=0x55643f6dbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.179+0000 7fb59ffff700 1 --2- 172.21.15.64:0/4053182258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55643f6db710 0x55643f6dbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.179+0000 7fb59ffff700 1 -- 172.21.15.64:0/4053182258 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55643f455d80 con 0x55643f6dc0d0 2023-09-13T03:14:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.179+0000 7fb5a6ffd700 1 --2- 172.21.15.64:0/4053182258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55643f6db710 0x55643f6dbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.179+0000 7fb59ffff700 1 --2- 172.21.15.64:0/4053182258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55643f6dc0d0 0x55643f6e8340 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb590009c80 tx=0x7fb590009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=676892b1e74d2fc5 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.179+0000 7fb5a77fe700 1 --2- 172.21.15.64:0/2559929141 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fb5940a07b0 0x7fb5940a2c70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0008d10 tx=0x7fb5a000b040 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.179+0000 7fb59d7fa700 1 -- 172.21.15.64:0/4053182258 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb590011cb0 con 0x55643f6dc0d0 2023-09-13T03:14:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.179+0000 7fb59d7fa700 1 -- 172.21.15.64:0/4053182258 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb590011e10 con 0x55643f6dc0d0 2023-09-13T03:14:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.179+0000 7fb59d7fa700 1 -- 172.21.15.64:0/4053182258 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb590005280 con 0x55643f6dc0d0 2023-09-13T03:14:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.180+0000 7fb5c0ee5b80 1 -- 172.21.15.64:0/4053182258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55643f6dc0d0 msgr2=0x55643f6e8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.180+0000 7fb5c0ee5b80 1 --2- 172.21.15.64:0/4053182258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55643f6dc0d0 0x55643f6e8340 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb590009c80 tx=0x7fb590009f90 comp rx=0 tx=0).stop 2023-09-13T03:14:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.180+0000 7fb5c0ee5b80 1 -- 172.21.15.64:0/4053182258 shutdown_connections 2023-09-13T03:14:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.180+0000 7fb5c0ee5b80 1 --2- 172.21.15.64:0/4053182258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55643f6dc0d0 0x55643f6e8340 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.180+0000 7fb5c0ee5b80 1 --2- 172.21.15.64:0/4053182258 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643f6e15b0 0x55643f6e1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.180+0000 7fb5c0ee5b80 1 --2- 172.21.15.64:0/4053182258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55643f6db710 0x55643f6dbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.180+0000 7fb5c0ee5b80 1 -- 172.21.15.64:0/4053182258 >> 172.21.15.64:0/4053182258 conn(0x55643f64c030 msgr2=0x55643f6d2560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.180+0000 7fb5c0ee5b80 1 -- 172.21.15.64:0/4053182258 shutdown_connections 2023-09-13T03:14:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.180+0000 7fb5c0ee5b80 1 -- 172.21.15.64:0/4053182258 wait complete. 2023-09-13T03:14:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.181+0000 7fb5c0ee5b80 1 -- start start 2023-09-13T03:14:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.181+0000 7fb5c0ee5b80 1 Processor -- start 2023-09-13T03:14:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.181+0000 7fb5c0ee5b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55643f6db710 0x55643f6e8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.181+0000 7fb5c0ee5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55643f6dc0d0 0x55643f6e5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.181+0000 7fb5c0ee5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643f6e15b0 0x55643f6e5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.181+0000 7fb5c0ee5b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643f5c7b10 con 0x55643f6db710 2023-09-13T03:14:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.181+0000 7fb5c0ee5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643f5c7810 con 0x55643f6dc0d0 2023-09-13T03:14:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.181+0000 7fb5c0ee5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643f4ab8c0 con 0x55643f6e15b0 2023-09-13T03:14:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.182+0000 7fb5c0ee5b80 10 client.0 _ll_register_callbacks cb 0x55643f511a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:14:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.182+0000 7fb5a6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55643f6db710 0x55643f6e8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:45.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.182+0000 7fb5a6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55643f6db710 0x55643f6e8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37626/0 (socket says 172.21.15.64:37626) 2023-09-13T03:14:45.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.182+0000 7fb5a6ffd700 1 -- 172.21.15.64:0/3612164213 learned_addr learned my addr 172.21.15.64:0/3612164213 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:45.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.182+0000 7fb59ffff700 1 --2- 172.21.15.64:0/3612164213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55643f6dc0d0 0x55643f6e5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:45.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.182+0000 7fb5a77fe700 1 --2- 172.21.15.64:0/3612164213 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643f6e15b0 0x55643f6e5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:45.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.182+0000 7fb5a6ffd700 1 -- 172.21.15.64:0/3612164213 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643f6e15b0 msgr2=0x55643f6e5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:45.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.182+0000 7fb5a6ffd700 1 --2- 172.21.15.64:0/3612164213 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643f6e15b0 0x55643f6e5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:45.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.182+0000 7fb5a6ffd700 1 -- 172.21.15.64:0/3612164213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55643f6dc0d0 msgr2=0x55643f6e5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:45.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.182+0000 7fb5a6ffd700 1 --2- 172.21.15.64:0/3612164213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55643f6dc0d0 0x55643f6e5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:45.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.182+0000 7fb5a6ffd700 1 -- 172.21.15.64:0/3612164213 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5a81a35f0 con 0x55643f6db710 2023-09-13T03:14:45.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.182+0000 7fb5a6ffd700 1 --2- 172.21.15.64:0/3612164213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55643f6db710 0x55643f6e8120 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb598096c40 tx=0x7fb598005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f5d61f8da211ecd server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:45.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.182+0000 7fb58e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:14:45.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.183+0000 7fb58e7fc700 1 -- 172.21.15.64:0/3612164213 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59809a1d0 con 0x55643f6db710 2023-09-13T03:14:45.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.183+0000 7fb5c0ee5b80 1 -- 172.21.15.64:0/3612164213 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55643f581490 con 0x55643f6db710 2023-09-13T03:14:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.183+0000 7fb58e7fc700 1 -- 172.21.15.64:0/3612164213 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fb59809e000 con 0x55643f6db710 2023-09-13T03:14:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.183+0000 7fb58e7fc700 1 -- 172.21.15.64:0/3612164213 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5980b8170 con 0x55643f6db710 2023-09-13T03:14:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.183+0000 7fb5c0ee5b80 10 client.16209 fetch_fsmap learned FSMap version 32 2023-09-13T03:14:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.183+0000 7fb5c0ee5b80 1 -- 172.21.15.64:0/3612164213 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7fb5900098e0 con 0x55643f6db710 2023-09-13T03:14:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.183+0000 7fb58e7fc700 1 -- 172.21.15.64:0/3612164213 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb598064080 con 0x55643f6db710 2023-09-13T03:14:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.183+0000 7fb58e7fc700 1 -- 172.21.15.64:0/3612164213 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 32) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fb5980996c0 con 0x55643f6db710 2023-09-13T03:14:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.183+0000 7fb5c0ee5b80 10 client.16209 fetch_fsmap finished waiting for FSMap version 32 2023-09-13T03:14:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.183+0000 7fb5c0ee5b80 10 client.16209 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:14:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.183+0000 7fb5c0ee5b80 20 client.16209 populate_metadata read hostname 'smithi064' 2023-09-13T03:14:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.183+0000 7fb5c0ee5b80 1 --2- 172.21.15.64:0/3612164213 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55643f6f5430 0x55643f6f7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.183+0000 7fb5c0ee5b80 4 client.16209 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:14:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.183+0000 7fb5c0ee5b80 1 -- 172.21.15.64:0/3612164213 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55643f63e840 con 0x55643f6f5430 2023-09-13T03:14:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.184+0000 7fb59ffff700 1 --2- 172.21.15.64:0/3612164213 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55643f6f5430 0x55643f6f7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.184+0000 7fb59ffff700 1 --2- 172.21.15.64:0/3612164213 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55643f6f5430 0x55643f6f7850 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b7fbbc15000a84b server_cookie=c793d59c570df04c in_seq=0 out_seq=0 2023-09-13T03:14:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.184+0000 7fb58e7fc700 10 client.16209 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.185+0000 7fb58e7fc700 1 -- 172.21.15.64:0/3612164213 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55643f63e840 con 0x55643f6f5430 2023-09-13T03:14:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.185+0000 7fb58e7fc700 10 client.16209 handle_command_reply: tid=0 2023-09-13T03:14:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.185+0000 7fb5c0ee5b80 1 client.16209 shutdown 2023-09-13T03:14:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.185+0000 7fb5c0ee5b80 1 -- 172.21.15.64:0/3612164213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55643f6db710 msgr2=0x55643f6e8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.185+0000 7fb5c0ee5b80 1 --2- 172.21.15.64:0/3612164213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55643f6db710 0x55643f6e8120 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb598096c40 tx=0x7fb598005780 comp rx=0 tx=0).stop 2023-09-13T03:14:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.185+0000 7fb5c0ee5b80 1 -- 172.21.15.64:0/3612164213 shutdown_connections 2023-09-13T03:14:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.185+0000 7fb5c0ee5b80 1 --2- 172.21.15.64:0/3612164213 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55643f6f5430 0x55643f6f7850 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.185+0000 7fb5c0ee5b80 1 --2- 172.21.15.64:0/3612164213 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643f6e15b0 0x55643f6e5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.185+0000 7fb5c0ee5b80 1 --2- 172.21.15.64:0/3612164213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55643f6db710 0x55643f6e8120 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.185+0000 7fb5c0ee5b80 1 --2- 172.21.15.64:0/3612164213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55643f6dc0d0 0x55643f6e5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.185+0000 7fb5c0ee5b80 1 -- 172.21.15.64:0/3612164213 >> 172.21.15.64:0/3612164213 conn(0x55643f64c030 msgr2=0x55643f6d17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:45.207 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.185+0000 7fb58e7fc700 0 client.16209 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:45.207 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.185+0000 7fb5c0ee5b80 1 -- 172.21.15.64:0/3612164213 shutdown_connections 2023-09-13T03:14:45.207 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.185+0000 7fb5c0ee5b80 1 -- 172.21.15.64:0/3612164213 wait complete. 2023-09-13T03:14:45.207 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.186+0000 7fb5c0ee5b80 20 client.16209 trim_cache size 0 max 16384 2023-09-13T03:14:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.201+0000 7fb5c0ee5b80 1 Processor -- start 2023-09-13T03:14:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.201+0000 7fb5c0ee5b80 1 -- start start 2023-09-13T03:14:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.201+0000 7fb5c0ee5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643f6db710 0x55643f6e0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.201+0000 7fb5c0ee5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55643f6dc0d0 0x55643f6e4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.201+0000 7fb5c0ee5b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55643f6e15b0 0x55643f6e7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.201+0000 7fb5c0ee5b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643f6dc990 con 0x55643f6e15b0 2023-09-13T03:14:45.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.201+0000 7fb5c0ee5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643f6e1210 con 0x55643f6dc0d0 2023-09-13T03:14:45.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.201+0000 7fb5c0ee5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643f6e39c0 con 0x55643f6db710 2023-09-13T03:14:45.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.201+0000 7fb5a77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55643f6e15b0 0x55643f6e7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:45.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.202+0000 7fb5a77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55643f6e15b0 0x55643f6e7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37632/0 (socket says 172.21.15.64:37632) 2023-09-13T03:14:45.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.202+0000 7fb5a77fe700 1 -- 172.21.15.64:0/517137001 learned_addr learned my addr 172.21.15.64:0/517137001 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:45.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.202+0000 7fb5a6ffd700 1 --2- 172.21.15.64:0/517137001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643f6db710 0x55643f6e0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:45.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.202+0000 7fb59ffff700 1 --2- 172.21.15.64:0/517137001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55643f6dc0d0 0x55643f6e4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:45.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.202+0000 7fb5a77fe700 1 -- 172.21.15.64:0/517137001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643f6db710 msgr2=0x55643f6e0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:45.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.202+0000 7fb5a77fe700 1 --2- 172.21.15.64:0/517137001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643f6db710 0x55643f6e0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.202+0000 7fb5a77fe700 1 -- 172.21.15.64:0/517137001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55643f6dc0d0 msgr2=0x55643f6e4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.202+0000 7fb5a77fe700 1 --2- 172.21.15.64:0/517137001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55643f6dc0d0 0x55643f6e4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.202+0000 7fb5a77fe700 1 -- 172.21.15.64:0/517137001 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55643f43ec00 con 0x55643f6e15b0 2023-09-13T03:14:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.202+0000 7fb5a6ffd700 1 --2- 172.21.15.64:0/517137001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643f6db710 0x55643f6e0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.202+0000 7fb5a77fe700 1 --2- 172.21.15.64:0/517137001 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55643f6e15b0 0x55643f6e7c50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb5a001abe0 tx=0x7fb5a0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b044b9b11ee60229 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.202+0000 7fb59dffb700 1 -- 172.21.15.64:0/517137001 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a0021070 con 0x55643f6e15b0 2023-09-13T03:14:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.203+0000 7fb59dffb700 1 -- 172.21.15.64:0/517137001 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5a0022040 con 0x55643f6e15b0 2023-09-13T03:14:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.203+0000 7fb59dffb700 1 -- 172.21.15.64:0/517137001 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a000e070 con 0x55643f6e15b0 2023-09-13T03:14:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.203+0000 7fb5c0ee5b80 1 -- 172.21.15.64:0/517137001 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55643f6e15b0 msgr2=0x55643f6e7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.203+0000 7fb5c0ee5b80 1 --2- 172.21.15.64:0/517137001 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55643f6e15b0 0x55643f6e7c50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb5a001abe0 tx=0x7fb5a0009920 comp rx=0 tx=0).stop 2023-09-13T03:14:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.203+0000 7fb5c0ee5b80 1 -- 172.21.15.64:0/517137001 shutdown_connections 2023-09-13T03:14:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.203+0000 7fb5c0ee5b80 1 --2- 172.21.15.64:0/517137001 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55643f6e15b0 0x55643f6e7c50 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.203+0000 7fb5c0ee5b80 1 --2- 172.21.15.64:0/517137001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643f6db710 0x55643f6e0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.203+0000 7fb5c0ee5b80 1 --2- 172.21.15.64:0/517137001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55643f6dc0d0 0x55643f6e4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.203+0000 7fb5c0ee5b80 1 -- 172.21.15.64:0/517137001 >> 172.21.15.64:0/517137001 conn(0x55643f64c030 msgr2=0x55643f6daf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.203+0000 7fb5c0ee5b80 1 -- 172.21.15.64:0/517137001 shutdown_connections 2023-09-13T03:14:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.203+0000 7fb5c0ee5b80 1 -- 172.21.15.64:0/517137001 wait complete. 2023-09-13T03:14:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.204+0000 7fb5c0ee5b80 1 -- start start 2023-09-13T03:14:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.204+0000 7fb5c0ee5b80 1 Processor -- start 2023-09-13T03:14:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.204+0000 7fb5c0ee5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55643f6db710 0x55643f6ec790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.204+0000 7fb5c0ee5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643f6dc0d0 0x55643f6eaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.204+0000 7fb5c0ee5b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55643f6e15b0 0x55643f6eafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.204+0000 7fb5c0ee5b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643f6eb500 con 0x55643f6e15b0 2023-09-13T03:14:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.204+0000 7fb5c0ee5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643f6eb670 con 0x55643f6db710 2023-09-13T03:14:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.204+0000 7fb5c0ee5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643f6eb830 con 0x55643f6dc0d0 2023-09-13T03:14:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.204+0000 7fb5c0ee5b80 10 client.0 _ll_register_callbacks cb 0x7fb59809a050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:14:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.205+0000 7fb5a77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55643f6e15b0 0x55643f6eafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.205+0000 7fb5a6ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55643f6db710 0x55643f6ec790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.205+0000 7fb59ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643f6dc0d0 0x55643f6eaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.205+0000 7fb5a77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55643f6e15b0 0x55643f6eafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37642/0 (socket says 172.21.15.64:37642) 2023-09-13T03:14:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.205+0000 7fb5a77fe700 1 -- 172.21.15.64:0/1510808440 learned_addr learned my addr 172.21.15.64:0/1510808440 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.205+0000 7fb5a6ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55643f6db710 0x55643f6ec790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:36622/0 (socket says 172.21.15.64:36622) 2023-09-13T03:14:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.205+0000 7fb59ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643f6dc0d0 0x55643f6eaa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:47728/0 (socket says 172.21.15.64:47728) 2023-09-13T03:14:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.205+0000 7fb5a6ffd700 1 -- 172.21.15.64:0/1510808440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643f6dc0d0 msgr2=0x55643f6eaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.205+0000 7fb5a6ffd700 1 --2- 172.21.15.64:0/1510808440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643f6dc0d0 0x55643f6eaa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.205+0000 7fb5a6ffd700 1 -- 172.21.15.64:0/1510808440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55643f6e15b0 msgr2=0x55643f6eafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.205+0000 7fb5a6ffd700 1 --2- 172.21.15.64:0/1510808440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55643f6e15b0 0x55643f6eafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.205+0000 7fb5a6ffd700 1 -- 172.21.15.64:0/1510808440 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5900098e0 con 0x55643f6db710 2023-09-13T03:14:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.205+0000 7fb59ffff700 1 --2- 172.21.15.64:0/1510808440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643f6dc0d0 0x55643f6eaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.205+0000 7fb5a77fe700 1 --2- 172.21.15.64:0/1510808440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55643f6e15b0 0x55643f6eafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.205+0000 7fb5a6ffd700 1 --2- 172.21.15.64:0/1510808440 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55643f6db710 0x55643f6ec790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb5980a1c80 tx=0x7fb59809a7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c09047767e49a1c4 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.205+0000 7fb58e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:14:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.205+0000 7fb58e7fc700 1 -- 172.21.15.64:0/1510808440 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5980b8170 con 0x55643f6db710 2023-09-13T03:14:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.205+0000 7fb5c0ee5b80 1 -- 172.21.15.64:0/1510808440 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb590011e10 con 0x55643f6db710 2023-09-13T03:14:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.206+0000 7fb58e7fc700 1 -- 172.21.15.64:0/1510808440 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fb59809e000 con 0x55643f6db710 2023-09-13T03:14:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.206+0000 7fb58e7fc700 1 -- 172.21.15.64:0/1510808440 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb59809a1d0 con 0x55643f6db710 2023-09-13T03:14:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.206+0000 7fb5c0ee5b80 10 client.26255 fetch_fsmap learned FSMap version 32 2023-09-13T03:14:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.206+0000 7fb5c0ee5b80 1 -- 172.21.15.64:0/1510808440 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55643f47c1f0 con 0x55643f6db710 2023-09-13T03:14:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.206+0000 7fb58e7fc700 1 -- 172.21.15.64:0/1510808440 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5980a6080 con 0x55643f6db710 2023-09-13T03:14:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.206+0000 7fb58e7fc700 1 -- 172.21.15.64:0/1510808440 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 32) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fb5980a1d20 con 0x55643f6db710 2023-09-13T03:14:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.206+0000 7fb5c0ee5b80 10 client.26255 fetch_fsmap finished waiting for FSMap version 32 2023-09-13T03:14:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.206+0000 7fb5c0ee5b80 10 client.26255 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:14:45.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.206+0000 7fb5c0ee5b80 20 client.26255 populate_metadata read hostname 'smithi064' 2023-09-13T03:14:45.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.206+0000 7fb5c0ee5b80 1 --2- 172.21.15.64:0/1510808440 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55643f74a3f0 0x55643f74c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:45.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.206+0000 7fb5c0ee5b80 4 client.26255 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:14:45.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.206+0000 7fb5c0ee5b80 1 -- 172.21.15.64:0/1510808440 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55643f6228c0 con 0x55643f74a3f0 2023-09-13T03:14:45.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.206+0000 7fb59ffff700 1 --2- 172.21.15.64:0/1510808440 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55643f74a3f0 0x55643f74c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:45.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.207+0000 7fb59ffff700 1 --2- 172.21.15.64:0/1510808440 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55643f74a3f0 0x55643f74c810 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a8a6bc9412d0ab0 server_cookie=acbc8489e653ac44 in_seq=0 out_seq=0 2023-09-13T03:14:45.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.207+0000 7fb58e7fc700 10 client.26255 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:45.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.230+0000 7fb58e7fc700 1 -- 172.21.15.64:0/1510808440 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55643f6228c0 con 0x55643f74a3f0 2023-09-13T03:14:45.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.230+0000 7fb58e7fc700 10 client.26255 handle_command_reply: tid=0 2023-09-13T03:14:45.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.230+0000 7fb5c0ee5b80 1 client.26255 shutdown 2023-09-13T03:14:45.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.230+0000 7fb5c0ee5b80 1 -- 172.21.15.64:0/1510808440 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55643f6db710 msgr2=0x55643f6ec790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:45.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.230+0000 7fb5c0ee5b80 1 --2- 172.21.15.64:0/1510808440 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55643f6db710 0x55643f6ec790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb5980a1c80 tx=0x7fb59809a7f0 comp rx=0 tx=0).stop 2023-09-13T03:14:45.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.231+0000 7fb5c0ee5b80 1 -- 172.21.15.64:0/1510808440 shutdown_connections 2023-09-13T03:14:45.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.231+0000 7fb5c0ee5b80 1 --2- 172.21.15.64:0/1510808440 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55643f6db710 0x55643f6ec790 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:45.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.231+0000 7fb5c0ee5b80 1 --2- 172.21.15.64:0/1510808440 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55643f74a3f0 0x55643f74c810 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:45.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.231+0000 7fb5c0ee5b80 1 --2- 172.21.15.64:0/1510808440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55643f6e15b0 0x55643f6eafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:45.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.231+0000 7fb5c0ee5b80 1 --2- 172.21.15.64:0/1510808440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55643f6dc0d0 0x55643f6eaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:45.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.231+0000 7fb58e7fc700 0 client.26255 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:45.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.231+0000 7fb5c0ee5b80 1 -- 172.21.15.64:0/1510808440 >> 172.21.15.64:0/1510808440 conn(0x55643f64c030 msgr2=0x55643f6f1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:45.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.231+0000 7fb5c0ee5b80 1 -- 172.21.15.64:0/1510808440 shutdown_connections 2023-09-13T03:14:45.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.231+0000 7fb5c0ee5b80 1 -- 172.21.15.64:0/1510808440 wait complete. 2023-09-13T03:14:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.231+0000 7fb5c0ee5b80 20 client.26255 trim_cache size 0 max 16384 2023-09-13T03:14:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:14:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:14:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:14:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:14:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.232+0000 7fb5ad5ff700 1 -- 172.21.15.64:0/2559929141 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fb5940a07b0 msgr2=0x7fb5940a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.232+0000 7fb5ad5ff700 1 --2- 172.21.15.64:0/2559929141 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fb5940a07b0 0x7fb5940a2c70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0008d10 tx=0x7fb5a000b040 comp rx=0 tx=0).stop 2023-09-13T03:14:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.232+0000 7fb5ad5ff700 1 -- 172.21.15.64:0/2559929141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5a81005a0 msgr2=0x7fb5a819c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.232+0000 7fb5ad5ff700 1 --2- 172.21.15.64:0/2559929141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5a81005a0 0x7fb5a819c600 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb598000c00 tx=0x7fb59800bfd0 comp rx=0 tx=0).stop 2023-09-13T03:14:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.232+0000 7fb5ad5ff700 1 -- 172.21.15.64:0/2559929141 shutdown_connections 2023-09-13T03:14:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.232+0000 7fb5ad5ff700 1 --2- 172.21.15.64:0/2559929141 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fb5940a07b0 0x7fb5940a2c70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.232+0000 7fb5ad5ff700 1 --2- 172.21.15.64:0/2559929141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5a8106660 0x7fb5a8196460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.232+0000 7fb5ad5ff700 1 --2- 172.21.15.64:0/2559929141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5a81005a0 0x7fb5a819c600 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.232+0000 7fb5ad5ff700 1 --2- 172.21.15.64:0/2559929141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5a8100f60 0x7fb5a819cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.232+0000 7fb5ad5ff700 1 -- 172.21.15.64:0/2559929141 >> 172.21.15.64:0/2559929141 conn(0x7fb5a80fc050 msgr2=0x7fb5a8104230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.232+0000 7fb5ad5ff700 1 -- 172.21.15.64:0/2559929141 shutdown_connections 2023-09-13T03:14:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:45.232+0000 7fb5ad5ff700 1 -- 172.21.15.64:0/2559929141 wait complete. 2023-09-13T03:14:45.889 INFO:tasks.workunit.client.0.smithi064.stdout: 1 29936 3.58 MB/sec warmup 50 sec latency 549.115 ms 2023-09-13T03:14:46.889 INFO:tasks.workunit.client.0.smithi064.stdout: 1 32230 3.78 MB/sec warmup 51 sec latency 15.392 ms 2023-09-13T03:14:46.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:46 smithi142 ceph-mon[134082]: pgmap v51: 97 pgs: 97 active+clean; 27 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 231 KiB/s wr, 71 op/s 2023-09-13T03:14:47.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:46 smithi170 ceph-mon[134328]: pgmap v51: 97 pgs: 97 active+clean; 27 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 231 KiB/s wr, 71 op/s 2023-09-13T03:14:47.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:46 smithi064 ceph-mon[97675]: pgmap v51: 97 pgs: 97 active+clean; 27 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 231 KiB/s wr, 71 op/s 2023-09-13T03:14:47.889 INFO:tasks.workunit.client.0.smithi064.stdout: 1 32755 3.72 MB/sec warmup 52 sec latency 444.494 ms 2023-09-13T03:14:48.889 INFO:tasks.workunit.client.0.smithi064.stdout: 1 32755 3.65 MB/sec warmup 53 sec latency 1444.615 ms 2023-09-13T03:14:48.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:48 smithi142 ceph-mon[134082]: pgmap v52: 97 pgs: 97 active+clean; 27 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 222 KiB/s wr, 58 op/s 2023-09-13T03:14:49.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:48 smithi170 ceph-mon[134328]: pgmap v52: 97 pgs: 97 active+clean; 27 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 222 KiB/s wr, 58 op/s 2023-09-13T03:14:49.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:48 smithi064 ceph-mon[97675]: pgmap v52: 97 pgs: 97 active+clean; 27 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 222 KiB/s wr, 58 op/s 2023-09-13T03:14:49.889 INFO:tasks.workunit.client.0.smithi064.stdout: 1 32755 3.58 MB/sec warmup 54 sec latency 2444.759 ms 2023-09-13T03:14:50.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:50 smithi170 ceph-mon[134328]: pgmap v53: 97 pgs: 97 active+clean; 27 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 222 KiB/s wr, 58 op/s 2023-09-13T03:14:50.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:50 smithi064 ceph-mon[97675]: pgmap v53: 97 pgs: 97 active+clean; 27 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 222 KiB/s wr, 58 op/s 2023-09-13T03:14:50.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:50 smithi142 ceph-mon[134082]: pgmap v53: 97 pgs: 97 active+clean; 27 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 222 KiB/s wr, 58 op/s 2023-09-13T03:14:50.889 INFO:tasks.workunit.client.0.smithi064.stdout: 1 33483 3.60 MB/sec warmup 55 sec latency 2897.934 ms 2023-09-13T03:14:51.889 INFO:tasks.workunit.client.0.smithi064.stdout: 1 35743 3.78 MB/sec warmup 56 sec latency 11.367 ms 2023-09-13T03:14:52.889 INFO:tasks.workunit.client.0.smithi064.stdout: 1 36174 3.72 MB/sec warmup 57 sec latency 404.216 ms 2023-09-13T03:14:52.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:52 smithi170 ceph-mon[134328]: pgmap v54: 97 pgs: 97 active+clean; 29 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 425 KiB/s wr, 98 op/s 2023-09-13T03:14:52.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:52 smithi064 ceph-mon[97675]: pgmap v54: 97 pgs: 97 active+clean; 29 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 425 KiB/s wr, 98 op/s 2023-09-13T03:14:52.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:52 smithi142 ceph-mon[134082]: pgmap v54: 97 pgs: 97 active+clean; 29 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 425 KiB/s wr, 98 op/s 2023-09-13T03:14:53.890 INFO:tasks.workunit.client.0.smithi064.stdout: 1 36174 3.66 MB/sec warmup 58 sec latency 1404.369 ms 2023-09-13T03:14:54.890 INFO:tasks.workunit.client.0.smithi064.stdout: 1 36174 3.60 MB/sec warmup 59 sec latency 2404.516 ms 2023-09-13T03:14:54.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:54 smithi170 ceph-mon[134328]: pgmap v55: 97 pgs: 97 active+clean; 29 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 369 KiB/s wr, 81 op/s 2023-09-13T03:14:54.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:54 smithi064 ceph-mon[97675]: pgmap v55: 97 pgs: 97 active+clean; 29 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 369 KiB/s wr, 81 op/s 2023-09-13T03:14:54.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:54 smithi142 ceph-mon[134082]: pgmap v55: 97 pgs: 97 active+clean; 29 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 369 KiB/s wr, 81 op/s 2023-09-13T03:14:55.890 INFO:tasks.workunit.client.0.smithi064.stdout: 1 36692 3.57 MB/sec warmup 60 sec latency 2858.479 ms 2023-09-13T03:14:56.890 INFO:tasks.workunit.client.0.smithi064.stdout: 1 36692 3.51 MB/sec warmup 61 sec latency 1055.361 ms 2023-09-13T03:14:56.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:56 smithi170 ceph-mon[134328]: pgmap v56: 97 pgs: 97 active+clean; 30 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 586 KiB/s wr, 122 op/s 2023-09-13T03:14:56.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:56 smithi064 ceph-mon[97675]: pgmap v56: 97 pgs: 97 active+clean; 30 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 586 KiB/s wr, 122 op/s 2023-09-13T03:14:56.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:56 smithi142 ceph-mon[134082]: pgmap v56: 97 pgs: 97 active+clean; 30 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 586 KiB/s wr, 122 op/s 2023-09-13T03:14:57.890 INFO:tasks.workunit.client.0.smithi064.stdout: 1 36692 3.46 MB/sec warmup 62 sec latency 2055.519 ms 2023-09-13T03:14:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.392+0000 7fd79a9ff700 1 -- 172.21.15.64:0/3380198432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7941005a0 msgr2=0x7fd794100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.392+0000 7fd79a9ff700 1 --2- 172.21.15.64:0/3380198432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7941005a0 0x7fd794100a20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd7880099c0 tx=0x7fd788009cd0 comp rx=0 tx=0).stop 2023-09-13T03:14:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.392+0000 7fd79a9ff700 1 -- 172.21.15.64:0/3380198432 shutdown_connections 2023-09-13T03:14:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.392+0000 7fd79a9ff700 1 --2- 172.21.15.64:0/3380198432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd794100f60 0x7fd79410d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.392+0000 7fd79a9ff700 1 --2- 172.21.15.64:0/3380198432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd794106660 0x7fd794106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.392+0000 7fd79a9ff700 1 --2- 172.21.15.64:0/3380198432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7941005a0 0x7fd794100a20 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.392+0000 7fd79a9ff700 1 -- 172.21.15.64:0/3380198432 >> 172.21.15.64:0/3380198432 conn(0x7fd7940fc050 msgr2=0x7fd7940fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:58.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.392+0000 7fd79a9ff700 1 -- 172.21.15.64:0/3380198432 shutdown_connections 2023-09-13T03:14:58.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.392+0000 7fd79a9ff700 1 -- 172.21.15.64:0/3380198432 wait complete. 2023-09-13T03:14:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.394+0000 7fd79a9ff700 1 Processor -- start 2023-09-13T03:14:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.394+0000 7fd79a9ff700 1 -- start start 2023-09-13T03:14:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.395+0000 7fd79a9ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7941005a0 0x7fd79419c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.395+0000 7fd79a9ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd794100f60 0x7fd79419ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.395+0000 7fd79a9ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd794106660 0x7fd7941963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.395+0000 7fd79a9ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd794196a20 con 0x7fd794100f60 2023-09-13T03:14:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.395+0000 7fd79a9ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd794196b90 con 0x7fd794106660 2023-09-13T03:14:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.395+0000 7fd79a9ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd794196d50 con 0x7fd7941005a0 2023-09-13T03:14:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.395+0000 7fd7937fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd794100f60 0x7fd79419ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.395+0000 7fd793fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7941005a0 0x7fd79419c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.395+0000 7fd7937fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd794100f60 0x7fd79419ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36462/0 (socket says 172.21.15.64:36462) 2023-09-13T03:14:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.395+0000 7fd7937fe700 1 -- 172.21.15.64:0/3829597859 learned_addr learned my addr 172.21.15.64:0/3829597859 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.395+0000 7fd798f9c700 1 --2- 172.21.15.64:0/3829597859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd794106660 0x7fd7941963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.395+0000 7fd7937fe700 1 -- 172.21.15.64:0/3829597859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7941005a0 msgr2=0x7fd79419c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.395+0000 7fd7937fe700 1 --2- 172.21.15.64:0/3829597859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7941005a0 0x7fd79419c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.395+0000 7fd7937fe700 1 -- 172.21.15.64:0/3829597859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd794106660 msgr2=0x7fd7941963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:58.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.395+0000 7fd7937fe700 1 --2- 172.21.15.64:0/3829597859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd794106660 0x7fd7941963b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:58.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.395+0000 7fd7937fe700 1 -- 172.21.15.64:0/3829597859 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd784009520 con 0x7fd794100f60 2023-09-13T03:14:58.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.396+0000 7fd7937fe700 1 --2- 172.21.15.64:0/3829597859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd794100f60 0x7fd79419ca90 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd78800b580 tx=0x7fd78800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:58.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.396+0000 7fd7917fa700 1 -- 172.21.15.64:0/3829597859 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78801d070 con 0x7fd794100f60 2023-09-13T03:14:58.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.396+0000 7fd79a9ff700 1 -- 172.21.15.64:0/3829597859 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7880095f0 con 0x7fd794100f60 2023-09-13T03:14:58.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.396+0000 7fd7917fa700 1 -- 172.21.15.64:0/3829597859 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd788022470 con 0x7fd794100f60 2023-09-13T03:14:58.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.396+0000 7fd7917fa700 1 -- 172.21.15.64:0/3829597859 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd788017650 con 0x7fd794100f60 2023-09-13T03:14:58.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.396+0000 7fd79a9ff700 1 -- 172.21.15.64:0/3829597859 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd794072830 con 0x7fd794100f60 2023-09-13T03:14:58.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.399+0000 7fd7917fa700 1 -- 172.21.15.64:0/3829597859 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fd7880225e0 con 0x7fd794100f60 2023-09-13T03:14:58.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.401+0000 7fd7917fa700 1 --2- 172.21.15.64:0/3829597859 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fd77c0a07b0 0x7fd77c0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:58.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.401+0000 7fd7ae2d7b80 1 Processor -- start 2023-09-13T03:14:58.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.401+0000 7fd7917fa700 1 -- 172.21.15.64:0/3829597859 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd7880cce70 con 0x7fd794100f60 2023-09-13T03:14:58.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.401+0000 7fd793fff700 1 --2- 172.21.15.64:0/3829597859 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fd77c0a07b0 0x7fd77c0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:58.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.401+0000 7fd7ae2d7b80 1 -- start start 2023-09-13T03:14:58.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.401+0000 7fd7ae2d7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579d0f251a0 0x5579d0f25580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:58.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.401+0000 7fd7ae2d7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579d0f1f300 0x5579d0f1f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:58.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.402+0000 7fd7ae2d7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5579d0f1fcc0 0x5579d0f2bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:58.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.402+0000 7fd7ae2d7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579d0e786d0 con 0x5579d0f251a0 2023-09-13T03:14:58.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.402+0000 7fd7ae2d7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579d0e75ff0 con 0x5579d0f1fcc0 2023-09-13T03:14:58.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.402+0000 7fd7ae2d7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579d0e718e0 con 0x5579d0f1f300 2023-09-13T03:14:58.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.402+0000 7fd793fff700 1 --2- 172.21.15.64:0/3829597859 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fd77c0a07b0 0x7fd77c0a2c70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd784009df0 tx=0x7fd784012040 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:58.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.402+0000 7fd793fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579d0f251a0 0x5579d0f25580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:58.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.402+0000 7fd793fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579d0f251a0 0x5579d0f25580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36472/0 (socket says 172.21.15.64:36472) 2023-09-13T03:14:58.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.402+0000 7fd793fff700 1 -- 172.21.15.64:0/1955593439 learned_addr learned my addr 172.21.15.64:0/1955593439 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:58.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.402+0000 7fd798f9c700 1 --2- 172.21.15.64:0/1955593439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5579d0f1fcc0 0x5579d0f2bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:58.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.402+0000 7fd7937fe700 1 --2- 172.21.15.64:0/1955593439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579d0f1f300 0x5579d0f1f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:58.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.402+0000 7fd793fff700 1 -- 172.21.15.64:0/1955593439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579d0f1f300 msgr2=0x5579d0f1f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:58.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.402+0000 7fd793fff700 1 --2- 172.21.15.64:0/1955593439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579d0f1f300 0x5579d0f1f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:58.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.402+0000 7fd793fff700 1 -- 172.21.15.64:0/1955593439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5579d0f1fcc0 msgr2=0x5579d0f2bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:58.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.402+0000 7fd793fff700 1 --2- 172.21.15.64:0/1955593439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5579d0f1fcc0 0x5579d0f2bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:58.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.402+0000 7fd793fff700 1 -- 172.21.15.64:0/1955593439 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579d0c9dd80 con 0x5579d0f251a0 2023-09-13T03:14:58.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.402+0000 7fd7937fe700 1 --2- 172.21.15.64:0/1955593439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579d0f1f300 0x5579d0f1f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.403+0000 7fd793fff700 1 --2- 172.21.15.64:0/1955593439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579d0f251a0 0x5579d0f25580 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd78401c740 tx=0x7fd784018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e29c41e076e4a6fa server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.403+0000 7fd77a7fc700 1 -- 172.21.15.64:0/1955593439 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78402a070 con 0x5579d0f251a0 2023-09-13T03:14:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.403+0000 7fd77a7fc700 1 -- 172.21.15.64:0/1955593439 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd784016040 con 0x5579d0f251a0 2023-09-13T03:14:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.403+0000 7fd77a7fc700 1 -- 172.21.15.64:0/1955593439 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7840263f0 con 0x5579d0f251a0 2023-09-13T03:14:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.404+0000 7fd7ae2d7b80 1 -- 172.21.15.64:0/1955593439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579d0f251a0 msgr2=0x5579d0f25580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.404+0000 7fd7ae2d7b80 1 --2- 172.21.15.64:0/1955593439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579d0f251a0 0x5579d0f25580 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd78401c740 tx=0x7fd784018000 comp rx=0 tx=0).stop 2023-09-13T03:14:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.404+0000 7fd7ae2d7b80 1 -- 172.21.15.64:0/1955593439 shutdown_connections 2023-09-13T03:14:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.404+0000 7fd7ae2d7b80 1 --2- 172.21.15.64:0/1955593439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5579d0f1fcc0 0x5579d0f2bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.404+0000 7fd7ae2d7b80 1 --2- 172.21.15.64:0/1955593439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579d0f251a0 0x5579d0f25580 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.404+0000 7fd7ae2d7b80 1 --2- 172.21.15.64:0/1955593439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579d0f1f300 0x5579d0f1f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.404+0000 7fd7ae2d7b80 1 -- 172.21.15.64:0/1955593439 >> 172.21.15.64:0/1955593439 conn(0x5579d0e94030 msgr2=0x5579d0f1a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.404+0000 7fd7ae2d7b80 1 -- 172.21.15.64:0/1955593439 shutdown_connections 2023-09-13T03:14:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.404+0000 7fd7ae2d7b80 1 -- 172.21.15.64:0/1955593439 wait complete. 2023-09-13T03:14:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.404+0000 7fd7ae2d7b80 1 -- start start 2023-09-13T03:14:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.405+0000 7fd7ae2d7b80 1 Processor -- start 2023-09-13T03:14:58.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.406+0000 7fd7ae2d7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5579d0f1f300 0x5579d0f2bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:58.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.406+0000 7fd7ae2d7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579d0f1fcc0 0x5579d0f29130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:58.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.406+0000 7fd7ae2d7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579d0f251a0 0x5579d0f296f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.406+0000 7fd7ae2d7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579d0e0fb10 con 0x5579d0f251a0 2023-09-13T03:14:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.406+0000 7fd7ae2d7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579d0e0f810 con 0x5579d0f1f300 2023-09-13T03:14:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.406+0000 7fd7ae2d7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579d0cf38c0 con 0x5579d0f1fcc0 2023-09-13T03:14:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.406+0000 7fd7ae2d7b80 10 client.0 _ll_register_callbacks cb 0x5579d0d59a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:14:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.406+0000 7fd798f9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579d0f251a0 0x5579d0f296f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.406+0000 7fd793fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5579d0f1f300 0x5579d0f2bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.406+0000 7fd798f9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579d0f251a0 0x5579d0f296f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36482/0 (socket says 172.21.15.64:36482) 2023-09-13T03:14:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.406+0000 7fd798f9c700 1 -- 172.21.15.64:0/3867167169 learned_addr learned my addr 172.21.15.64:0/3867167169 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.406+0000 7fd7937fe700 1 --2- 172.21.15.64:0/3867167169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579d0f1fcc0 0x5579d0f29130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.406+0000 7fd798f9c700 1 -- 172.21.15.64:0/3867167169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579d0f1fcc0 msgr2=0x5579d0f29130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:58.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.406+0000 7fd798f9c700 1 --2- 172.21.15.64:0/3867167169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579d0f1fcc0 0x5579d0f29130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:58.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.407+0000 7fd798f9c700 1 -- 172.21.15.64:0/3867167169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5579d0f1f300 msgr2=0x5579d0f2bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:58.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.407+0000 7fd798f9c700 1 --2- 172.21.15.64:0/3867167169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5579d0f1f300 0x5579d0f2bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:58.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.407+0000 7fd798f9c700 1 -- 172.21.15.64:0/3867167169 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd794072830 con 0x5579d0f251a0 2023-09-13T03:14:58.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.407+0000 7fd793fff700 1 --2- 172.21.15.64:0/3867167169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5579d0f1f300 0x5579d0f2bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:58.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.407+0000 7fd798f9c700 1 --2- 172.21.15.64:0/3867167169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579d0f251a0 0x5579d0f296f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd78c011ef0 tx=0x7fd78c014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c2a16fc2df004a server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:58.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.407+0000 7fd7737fe700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:14:58.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.407+0000 7fd7737fe700 1 -- 172.21.15.64:0/3867167169 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78c00e040 con 0x5579d0f251a0 2023-09-13T03:14:58.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.407+0000 7fd7ae2d7b80 1 -- 172.21.15.64:0/3867167169 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5579d0dc9490 con 0x5579d0f251a0 2023-09-13T03:14:58.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.407+0000 7fd7737fe700 1 -- 172.21.15.64:0/3867167169 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd78c0093a0 con 0x5579d0f251a0 2023-09-13T03:14:58.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.407+0000 7fd7737fe700 1 -- 172.21.15.64:0/3867167169 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd78c014ce0 con 0x5579d0f251a0 2023-09-13T03:14:58.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.407+0000 7fd7ae2d7b80 10 client.16227 fetch_fsmap learned FSMap version 32 2023-09-13T03:14:58.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.407+0000 7fd7ae2d7b80 1 -- 172.21.15.64:0/3867167169 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7fd78401c3f0 con 0x5579d0f251a0 2023-09-13T03:14:58.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.408+0000 7fd7737fe700 1 -- 172.21.15.64:0/3867167169 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5579d0dc9490 con 0x5579d0f251a0 2023-09-13T03:14:58.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.408+0000 7fd7737fe700 1 -- 172.21.15.64:0/3867167169 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 32) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fd78c01c020 con 0x5579d0f251a0 2023-09-13T03:14:58.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.408+0000 7fd7ae2d7b80 10 client.16227 fetch_fsmap finished waiting for FSMap version 32 2023-09-13T03:14:58.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.408+0000 7fd7ae2d7b80 10 client.16227 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:14:58.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.408+0000 7fd7ae2d7b80 20 client.16227 populate_metadata read hostname 'smithi064' 2023-09-13T03:14:58.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.408+0000 7fd7ae2d7b80 1 --2- 172.21.15.64:0/3867167169 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5579d0f39480 0x5579d0f3b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:58.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.408+0000 7fd7ae2d7b80 4 client.16227 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:14:58.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.408+0000 7fd7ae2d7b80 1 -- 172.21.15.64:0/3867167169 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5579d0e86840 con 0x5579d0f39480 2023-09-13T03:14:58.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.409+0000 7fd793fff700 1 --2- 172.21.15.64:0/3867167169 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5579d0f39480 0x5579d0f3b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:58.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.409+0000 7fd793fff700 1 --2- 172.21.15.64:0/3867167169 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5579d0f39480 0x5579d0f3b8a0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65eef1b73cc434b3 server_cookie=58e3068e5dfc3277 in_seq=0 out_seq=0 2023-09-13T03:14:58.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.409+0000 7fd7737fe700 10 client.16227 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:58.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.410+0000 7fd7737fe700 1 -- 172.21.15.64:0/3867167169 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5579d0e86840 con 0x5579d0f39480 2023-09-13T03:14:58.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.410+0000 7fd7737fe700 10 client.16227 handle_command_reply: tid=0 2023-09-13T03:14:58.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.410+0000 7fd7ae2d7b80 1 client.16227 shutdown 2023-09-13T03:14:58.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.410+0000 7fd7ae2d7b80 1 -- 172.21.15.64:0/3867167169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579d0f251a0 msgr2=0x5579d0f296f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:58.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.410+0000 7fd7ae2d7b80 1 --2- 172.21.15.64:0/3867167169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579d0f251a0 0x5579d0f296f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd78c011ef0 tx=0x7fd78c014600 comp rx=0 tx=0).stop 2023-09-13T03:14:58.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.410+0000 7fd7ae2d7b80 1 -- 172.21.15.64:0/3867167169 shutdown_connections 2023-09-13T03:14:58.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.410+0000 7fd7ae2d7b80 1 --2- 172.21.15.64:0/3867167169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5579d0f1f300 0x5579d0f2bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:58.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.410+0000 7fd7ae2d7b80 1 --2- 172.21.15.64:0/3867167169 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5579d0f39480 0x5579d0f3b8a0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:58.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.410+0000 7fd7ae2d7b80 1 --2- 172.21.15.64:0/3867167169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579d0f251a0 0x5579d0f296f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:58.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.410+0000 7fd7ae2d7b80 1 --2- 172.21.15.64:0/3867167169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579d0f1fcc0 0x5579d0f29130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:58.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.410+0000 7fd7737fe700 0 client.16227 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:58.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.410+0000 7fd7ae2d7b80 1 -- 172.21.15.64:0/3867167169 >> 172.21.15.64:0/3867167169 conn(0x5579d0e94030 msgr2=0x5579d0f197a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:58.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.410+0000 7fd7ae2d7b80 1 -- 172.21.15.64:0/3867167169 shutdown_connections 2023-09-13T03:14:58.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.410+0000 7fd7ae2d7b80 1 -- 172.21.15.64:0/3867167169 wait complete. 2023-09-13T03:14:58.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.411+0000 7fd7ae2d7b80 20 client.16227 trim_cache size 0 max 16384 2023-09-13T03:14:58.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.426+0000 7fd7ae2d7b80 1 Processor -- start 2023-09-13T03:14:58.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.426+0000 7fd7ae2d7b80 1 -- start start 2023-09-13T03:14:58.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.426+0000 7fd7ae2d7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579d0f1f300 0x5579d0f30530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:58.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.426+0000 7fd7ae2d7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5579d0f1fcc0 0x5579d0f19f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:58.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.426+0000 7fd7ae2d7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579d0f251a0 0x5579d0f1a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:58.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.426+0000 7fd7ae2d7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579d0f24b60 con 0x5579d0f251a0 2023-09-13T03:14:58.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.426+0000 7fd7ae2d7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579d0f30a70 con 0x5579d0f1fcc0 2023-09-13T03:14:58.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.426+0000 7fd7ae2d7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579d0f29cd0 con 0x5579d0f1f300 2023-09-13T03:14:58.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.427+0000 7fd798f9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579d0f251a0 0x5579d0f1a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:58.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.427+0000 7fd798f9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579d0f251a0 0x5579d0f1a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36490/0 (socket says 172.21.15.64:36490) 2023-09-13T03:14:58.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.427+0000 7fd798f9c700 1 -- 172.21.15.64:0/2583613681 learned_addr learned my addr 172.21.15.64:0/2583613681 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:58.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.427+0000 7fd7937fe700 1 --2- 172.21.15.64:0/2583613681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5579d0f1fcc0 0x5579d0f19f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:58.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.427+0000 7fd793fff700 1 --2- 172.21.15.64:0/2583613681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579d0f1f300 0x5579d0f30530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:58.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.427+0000 7fd798f9c700 1 -- 172.21.15.64:0/2583613681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579d0f1f300 msgr2=0x5579d0f30530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:58.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.427+0000 7fd798f9c700 1 --2- 172.21.15.64:0/2583613681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579d0f1f300 0x5579d0f30530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:58.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.427+0000 7fd798f9c700 1 -- 172.21.15.64:0/2583613681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5579d0f1fcc0 msgr2=0x5579d0f19f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:58.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.427+0000 7fd798f9c700 1 --2- 172.21.15.64:0/2583613681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5579d0f1fcc0 0x5579d0f19f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:58.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.427+0000 7fd798f9c700 1 -- 172.21.15.64:0/2583613681 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579d0c86c00 con 0x5579d0f251a0 2023-09-13T03:14:58.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.427+0000 7fd7937fe700 1 --2- 172.21.15.64:0/2583613681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5579d0f1fcc0 0x5579d0f19f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:58.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.427+0000 7fd793fff700 1 --2- 172.21.15.64:0/2583613681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579d0f1f300 0x5579d0f30530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:14:58.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.427+0000 7fd798f9c700 1 --2- 172.21.15.64:0/2583613681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579d0f251a0 0x5579d0f1a460 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd78c004670 tx=0x7fd78c006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be2303c4e124bd6 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:58.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.427+0000 7fd77affd700 1 -- 172.21.15.64:0/2583613681 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78c00e040 con 0x5579d0f251a0 2023-09-13T03:14:58.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.428+0000 7fd77affd700 1 -- 172.21.15.64:0/2583613681 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd78c00f040 con 0x5579d0f251a0 2023-09-13T03:14:58.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.428+0000 7fd77affd700 1 -- 172.21.15.64:0/2583613681 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78c007920 con 0x5579d0f251a0 2023-09-13T03:14:58.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.428+0000 7fd7ae2d7b80 1 -- 172.21.15.64:0/2583613681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579d0f251a0 msgr2=0x5579d0f1a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:58.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.428+0000 7fd7ae2d7b80 1 --2- 172.21.15.64:0/2583613681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579d0f251a0 0x5579d0f1a460 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd78c004670 tx=0x7fd78c006000 comp rx=0 tx=0).stop 2023-09-13T03:14:58.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.428+0000 7fd7ae2d7b80 1 -- 172.21.15.64:0/2583613681 shutdown_connections 2023-09-13T03:14:58.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.428+0000 7fd7ae2d7b80 1 --2- 172.21.15.64:0/2583613681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579d0f251a0 0x5579d0f1a460 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:58.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.428+0000 7fd7ae2d7b80 1 --2- 172.21.15.64:0/2583613681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579d0f1f300 0x5579d0f30530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:58.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.428+0000 7fd7ae2d7b80 1 --2- 172.21.15.64:0/2583613681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5579d0f1fcc0 0x5579d0f19f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:58.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.428+0000 7fd7ae2d7b80 1 -- 172.21.15.64:0/2583613681 >> 172.21.15.64:0/2583613681 conn(0x5579d0e94030 msgr2=0x5579d0f23330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:58.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.428+0000 7fd7ae2d7b80 1 -- 172.21.15.64:0/2583613681 shutdown_connections 2023-09-13T03:14:58.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.428+0000 7fd7ae2d7b80 1 -- 172.21.15.64:0/2583613681 wait complete. 2023-09-13T03:14:58.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.429+0000 7fd7ae2d7b80 1 -- start start 2023-09-13T03:14:58.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.429+0000 7fd7ae2d7b80 1 Processor -- start 2023-09-13T03:14:58.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.429+0000 7fd7ae2d7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579d0f1f300 0x5579d0f857c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:58.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.429+0000 7fd7ae2d7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5579d0f1fcc0 0x5579d0f85d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:58.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.429+0000 7fd7ae2d7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579d0f251a0 0x5579d0f81220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:58.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.429+0000 7fd7ae2d7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579d0f81870 con 0x5579d0f1f300 2023-09-13T03:14:58.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.429+0000 7fd7ae2d7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579d0f819e0 con 0x5579d0f1fcc0 2023-09-13T03:14:58.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.429+0000 7fd7ae2d7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579d0f81ba0 con 0x5579d0f251a0 2023-09-13T03:14:58.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.429+0000 7fd7ae2d7b80 10 client.0 _ll_register_callbacks cb 0x7fd78c011820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:14:58.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.430+0000 7fd793fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579d0f1f300 0x5579d0f857c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:58.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.430+0000 7fd793fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579d0f1f300 0x5579d0f857c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36496/0 (socket says 172.21.15.64:36496) 2023-09-13T03:14:58.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.430+0000 7fd793fff700 1 -- 172.21.15.64:0/3029060399 learned_addr learned my addr 172.21.15.64:0/3029060399 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:14:58.445 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.430+0000 7fd798f9c700 1 --2- 172.21.15.64:0/3029060399 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579d0f251a0 0x5579d0f81220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:58.445 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.430+0000 7fd7937fe700 1 --2- 172.21.15.64:0/3029060399 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5579d0f1fcc0 0x5579d0f85d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:58.445 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.430+0000 7fd793fff700 1 -- 172.21.15.64:0/3029060399 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579d0f251a0 msgr2=0x5579d0f81220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:58.445 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.430+0000 7fd793fff700 1 --2- 172.21.15.64:0/3029060399 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579d0f251a0 0x5579d0f81220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:58.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.430+0000 7fd793fff700 1 -- 172.21.15.64:0/3029060399 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5579d0f1fcc0 msgr2=0x5579d0f85d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:14:58.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.430+0000 7fd793fff700 1 --2- 172.21.15.64:0/3029060399 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5579d0f1fcc0 0x5579d0f85d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:58.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.430+0000 7fd793fff700 1 -- 172.21.15.64:0/3029060399 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7880095f0 con 0x5579d0f1f300 2023-09-13T03:14:58.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.430+0000 7fd798f9c700 1 --2- 172.21.15.64:0/3029060399 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579d0f251a0 0x5579d0f81220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:58.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.430+0000 7fd7937fe700 1 --2- 172.21.15.64:0/3029060399 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5579d0f1fcc0 0x5579d0f85d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:14:58.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.430+0000 7fd793fff700 1 --2- 172.21.15.64:0/3029060399 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579d0f1f300 0x5579d0f857c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd78401cd30 tx=0x7fd784009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=672f8a372779cafc server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:14:58.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.430+0000 7fd7737fe700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:14:58.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.431+0000 7fd7737fe700 1 -- 172.21.15.64:0/3029060399 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd784036040 con 0x5579d0f1f300 2023-09-13T03:14:58.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.431+0000 7fd7737fe700 1 -- 172.21.15.64:0/3029060399 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd784005350 con 0x5579d0f1f300 2023-09-13T03:14:58.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.431+0000 7fd7ae2d7b80 1 -- 172.21.15.64:0/3029060399 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd784016040 con 0x5579d0f1f300 2023-09-13T03:14:58.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.431+0000 7fd7737fe700 1 -- 172.21.15.64:0/3029060399 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd78402a040 con 0x5579d0f1f300 2023-09-13T03:14:58.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.431+0000 7fd7ae2d7b80 10 client.16233 fetch_fsmap learned FSMap version 32 2023-09-13T03:14:58.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.431+0000 7fd7ae2d7b80 1 -- 172.21.15.64:0/3029060399 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7fd78401c3f0 con 0x5579d0f1f300 2023-09-13T03:14:58.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.431+0000 7fd7737fe700 1 -- 172.21.15.64:0/3029060399 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd784016040 con 0x5579d0f1f300 2023-09-13T03:14:58.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.431+0000 7fd7737fe700 1 -- 172.21.15.64:0/3029060399 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 32) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fd78403f030 con 0x5579d0f1f300 2023-09-13T03:14:58.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.431+0000 7fd7ae2d7b80 10 client.16233 fetch_fsmap finished waiting for FSMap version 32 2023-09-13T03:14:58.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.431+0000 7fd7ae2d7b80 10 client.16233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:14:58.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.431+0000 7fd7ae2d7b80 20 client.16233 populate_metadata read hostname 'smithi064' 2023-09-13T03:14:58.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.431+0000 7fd7ae2d7b80 1 --2- 172.21.15.64:0/3029060399 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5579d0f8e780 0x5579d0f90ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:14:58.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.432+0000 7fd7ae2d7b80 4 client.16233 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:14:58.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.432+0000 7fd7ae2d7b80 1 -- 172.21.15.64:0/3029060399 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5579d0e6a8c0 con 0x5579d0f8e780 2023-09-13T03:14:58.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.432+0000 7fd7937fe700 1 --2- 172.21.15.64:0/3029060399 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5579d0f8e780 0x5579d0f90ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:14:58.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.432+0000 7fd7937fe700 1 --2- 172.21.15.64:0/3029060399 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5579d0f8e780 0x5579d0f90ba0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fb8039cd50ac492 server_cookie=c54ef7fdf1be044d in_seq=0 out_seq=0 2023-09-13T03:14:58.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.432+0000 7fd7737fe700 10 client.16233 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.459+0000 7fd7737fe700 1 -- 172.21.15.64:0/3029060399 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5579d0e6a8c0 con 0x5579d0f8e780 2023-09-13T03:14:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.459+0000 7fd7737fe700 10 client.16233 handle_command_reply: tid=0 2023-09-13T03:14:58.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.459+0000 7fd7ae2d7b80 1 client.16233 shutdown 2023-09-13T03:14:58.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.460+0000 7fd7ae2d7b80 1 -- 172.21.15.64:0/3029060399 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579d0f1f300 msgr2=0x5579d0f857c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:58.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.460+0000 7fd7ae2d7b80 1 --2- 172.21.15.64:0/3029060399 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579d0f1f300 0x5579d0f857c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd78401cd30 tx=0x7fd784009260 comp rx=0 tx=0).stop 2023-09-13T03:14:58.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.460+0000 7fd7ae2d7b80 1 -- 172.21.15.64:0/3029060399 shutdown_connections 2023-09-13T03:14:58.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.460+0000 7fd7ae2d7b80 1 --2- 172.21.15.64:0/3029060399 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5579d0f8e780 0x5579d0f90ba0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:58.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.460+0000 7fd7ae2d7b80 1 --2- 172.21.15.64:0/3029060399 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579d0f251a0 0x5579d0f81220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:58.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.460+0000 7fd7ae2d7b80 1 --2- 172.21.15.64:0/3029060399 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579d0f1f300 0x5579d0f857c0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:58.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.460+0000 7fd7737fe700 0 client.16233 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:14:58.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.460+0000 7fd7ae2d7b80 1 --2- 172.21.15.64:0/3029060399 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5579d0f1fcc0 0x5579d0f85d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:58.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.460+0000 7fd7ae2d7b80 1 -- 172.21.15.64:0/3029060399 >> 172.21.15.64:0/3029060399 conn(0x5579d0e94030 msgr2=0x5579d0f1e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:58.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.460+0000 7fd7ae2d7b80 1 -- 172.21.15.64:0/3029060399 shutdown_connections 2023-09-13T03:14:58.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.460+0000 7fd7ae2d7b80 1 -- 172.21.15.64:0/3029060399 wait complete. 2023-09-13T03:14:58.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.460+0000 7fd7ae2d7b80 20 client.16233 trim_cache size 0 max 16384 2023-09-13T03:14:58.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:14:58.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:14:58.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:14:58.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:14:58.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.461+0000 7fd79a9ff700 1 -- 172.21.15.64:0/3829597859 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fd77c0a07b0 msgr2=0x7fd77c0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:58.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.461+0000 7fd79a9ff700 1 --2- 172.21.15.64:0/3829597859 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fd77c0a07b0 0x7fd77c0a2c70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd784009df0 tx=0x7fd784012040 comp rx=0 tx=0).stop 2023-09-13T03:14:58.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.461+0000 7fd79a9ff700 1 -- 172.21.15.64:0/3829597859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd794100f60 msgr2=0x7fd79419ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:14:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.461+0000 7fd79a9ff700 1 --2- 172.21.15.64:0/3829597859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd794100f60 0x7fd79419ca90 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd78800b580 tx=0x7fd78800bfd0 comp rx=0 tx=0).stop 2023-09-13T03:14:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.461+0000 7fd79a9ff700 1 -- 172.21.15.64:0/3829597859 shutdown_connections 2023-09-13T03:14:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.461+0000 7fd79a9ff700 1 --2- 172.21.15.64:0/3829597859 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fd77c0a07b0 0x7fd77c0a2c70 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.461+0000 7fd79a9ff700 1 --2- 172.21.15.64:0/3829597859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd794106660 0x7fd7941963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.461+0000 7fd79a9ff700 1 --2- 172.21.15.64:0/3829597859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7941005a0 0x7fd79419c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.461+0000 7fd79a9ff700 1 --2- 172.21.15.64:0/3829597859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd794100f60 0x7fd79419ca90 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:14:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.461+0000 7fd79a9ff700 1 -- 172.21.15.64:0/3829597859 >> 172.21.15.64:0/3829597859 conn(0x7fd7940fc050 msgr2=0x7fd7940fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:14:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.461+0000 7fd79a9ff700 1 -- 172.21.15.64:0/3829597859 shutdown_connections 2023-09-13T03:14:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:14:58.461+0000 7fd79a9ff700 1 -- 172.21.15.64:0/3829597859 wait complete. 2023-09-13T03:14:58.890 INFO:tasks.workunit.client.0.smithi064.stdout: 1 36692 3.40 MB/sec warmup 63 sec latency 3055.623 ms 2023-09-13T03:14:58.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:14:58 smithi170 ceph-mon[134328]: pgmap v57: 97 pgs: 97 active+clean; 30 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 429 KiB/s wr, 82 op/s 2023-09-13T03:14:58.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:14:58 smithi064 ceph-mon[97675]: pgmap v57: 97 pgs: 97 active+clean; 30 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 429 KiB/s wr, 82 op/s 2023-09-13T03:14:58.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:14:58 smithi142 ceph-mon[134082]: pgmap v57: 97 pgs: 97 active+clean; 30 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 429 KiB/s wr, 82 op/s 2023-09-13T03:14:59.890 INFO:tasks.workunit.client.0.smithi064.stdout: 1 36692 3.35 MB/sec warmup 64 sec latency 4055.712 ms 2023-09-13T03:15:00.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:15:00 smithi170 ceph-mon[134328]: pgmap v58: 97 pgs: 97 active+clean; 30 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 420 KiB/s wr, 81 op/s 2023-09-13T03:15:00.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:15:00 smithi064 ceph-mon[97675]: pgmap v58: 97 pgs: 97 active+clean; 30 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 420 KiB/s wr, 81 op/s 2023-09-13T03:15:00.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:15:00 smithi142 ceph-mon[134082]: pgmap v58: 97 pgs: 97 active+clean; 30 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 420 KiB/s wr, 81 op/s 2023-09-13T03:15:00.890 INFO:tasks.workunit.client.0.smithi064.stdout: 1 37796 3.37 MB/sec warmup 65 sec latency 4444.298 ms 2023-09-13T03:15:01.890 INFO:tasks.workunit.client.0.smithi064.stdout: 1 39355 3.50 MB/sec warmup 66 sec latency 23.328 ms 2023-09-13T03:15:02.891 INFO:tasks.workunit.client.0.smithi064.stdout: 1 39846 3.46 MB/sec warmup 67 sec latency 489.398 ms 2023-09-13T03:15:02.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:15:02 smithi170 ceph-mon[134328]: pgmap v59: 97 pgs: 97 active+clean; 31 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 498 KiB/s wr, 119 op/s 2023-09-13T03:15:02.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:15:02 smithi064 ceph-mon[97675]: pgmap v59: 97 pgs: 97 active+clean; 31 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 498 KiB/s wr, 119 op/s 2023-09-13T03:15:02.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:15:02 smithi142 ceph-mon[134082]: pgmap v59: 97 pgs: 97 active+clean; 31 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 498 KiB/s wr, 119 op/s 2023-09-13T03:15:03.891 INFO:tasks.workunit.client.0.smithi064.stdout: 1 39846 3.41 MB/sec warmup 68 sec latency 1489.909 ms 2023-09-13T03:15:04.891 INFO:tasks.workunit.client.0.smithi064.stdout: 1 39846 3.36 MB/sec warmup 69 sec latency 2490.041 ms 2023-09-13T03:15:04.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:15:04 smithi170 ceph-mon[134328]: pgmap v60: 97 pgs: 97 active+clean; 31 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 295 KiB/s wr, 78 op/s 2023-09-13T03:15:04.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:15:04 smithi064 ceph-mon[97675]: pgmap v60: 97 pgs: 97 active+clean; 31 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 295 KiB/s wr, 78 op/s 2023-09-13T03:15:04.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:15:04 smithi142 ceph-mon[134082]: pgmap v60: 97 pgs: 97 active+clean; 31 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 295 KiB/s wr, 78 op/s 2023-09-13T03:15:05.891 INFO:tasks.workunit.client.0.smithi064.stdout: 1 40293 3.33 MB/sec warmup 70 sec latency 2941.625 ms 2023-09-13T03:15:06.891 INFO:tasks.workunit.client.0.smithi064.stdout: 1 42802 3.52 MB/sec warmup 71 sec latency 23.164 ms 2023-09-13T03:15:06.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:15:06 smithi170 ceph-mon[134328]: pgmap v61: 97 pgs: 97 active+clean; 34 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 667 KiB/s wr, 137 op/s 2023-09-13T03:15:06.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:15:06 smithi064 ceph-mon[97675]: pgmap v61: 97 pgs: 97 active+clean; 34 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 667 KiB/s wr, 137 op/s 2023-09-13T03:15:06.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:15:06 smithi142 ceph-mon[134082]: pgmap v61: 97 pgs: 97 active+clean; 34 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 667 KiB/s wr, 137 op/s 2023-09-13T03:15:07.892 INFO:tasks.workunit.client.0.smithi064.stdout: 1 43264 3.48 MB/sec warmup 72 sec latency 395.296 ms 2023-09-13T03:15:08.892 INFO:tasks.workunit.client.0.smithi064.stdout: 1 43264 3.43 MB/sec warmup 73 sec latency 1395.446 ms 2023-09-13T03:15:08.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:15:08 smithi170 ceph-mon[134328]: pgmap v62: 97 pgs: 97 active+clean; 34 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 450 KiB/s wr, 96 op/s 2023-09-13T03:15:08.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:15:08 smithi064 ceph-mon[97675]: pgmap v62: 97 pgs: 97 active+clean; 34 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 450 KiB/s wr, 96 op/s 2023-09-13T03:15:08.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:15:08 smithi142 ceph-mon[134082]: pgmap v62: 97 pgs: 97 active+clean; 34 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 450 KiB/s wr, 96 op/s 2023-09-13T03:15:09.892 INFO:tasks.workunit.client.0.smithi064.stdout: 1 43264 3.39 MB/sec warmup 74 sec latency 2395.545 ms 2023-09-13T03:15:10.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:15:10 smithi170 ceph-mon[134328]: pgmap v63: 97 pgs: 97 active+clean; 34 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 450 KiB/s wr, 96 op/s 2023-09-13T03:15:10.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:15:10 smithi064 ceph-mon[97675]: pgmap v63: 97 pgs: 97 active+clean; 34 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 450 KiB/s wr, 96 op/s 2023-09-13T03:15:10.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:15:10 smithi142 ceph-mon[134082]: pgmap v63: 97 pgs: 97 active+clean; 34 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 450 KiB/s wr, 96 op/s 2023-09-13T03:15:10.892 INFO:tasks.workunit.client.0.smithi064.stdout: 1 43320 3.34 MB/sec warmup 75 sec latency 2850.331 ms 2023-09-13T03:15:11.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.621+0000 7effe91ff700 1 -- 172.21.15.64:0/2556675994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effe4106660 msgr2=0x7effe4106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:15:11.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.621+0000 7effe91ff700 1 --2- 172.21.15.64:0/2556675994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effe4106660 0x7effe4106a40 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7effd40099c0 tx=0x7effd4009cd0 comp rx=0 tx=0).stop 2023-09-13T03:15:11.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.621+0000 7effe91ff700 1 -- 172.21.15.64:0/2556675994 shutdown_connections 2023-09-13T03:15:11.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.621+0000 7effe91ff700 1 --2- 172.21.15.64:0/2556675994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effe4100f60 0x7effe410d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:11.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.621+0000 7effe91ff700 1 --2- 172.21.15.64:0/2556675994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effe4106660 0x7effe4106a40 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:11.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.621+0000 7effe91ff700 1 --2- 172.21.15.64:0/2556675994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effe41005a0 0x7effe4100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:11.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.621+0000 7effe91ff700 1 -- 172.21.15.64:0/2556675994 >> 172.21.15.64:0/2556675994 conn(0x7effe40fc050 msgr2=0x7effe40fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:15:11.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.621+0000 7effe91ff700 1 -- 172.21.15.64:0/2556675994 shutdown_connections 2023-09-13T03:15:11.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.621+0000 7effe91ff700 1 -- 172.21.15.64:0/2556675994 wait complete. 2023-09-13T03:15:11.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.622+0000 7effe91ff700 1 Processor -- start 2023-09-13T03:15:11.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.623+0000 7effe91ff700 1 -- start start 2023-09-13T03:15:11.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.623+0000 7effe91ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effe41005a0 0x7effe419e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:11.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.623+0000 7effe91ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effe4100f60 0x7effe419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:11.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.623+0000 7effe91ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effe4106660 0x7effe4198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:11.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.623+0000 7effe91ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe4198cb0 con 0x7effe41005a0 2023-09-13T03:15:11.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.623+0000 7effe91ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe4198e20 con 0x7effe4106660 2023-09-13T03:15:11.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.623+0000 7effe91ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe4198fe0 con 0x7effe4100f60 2023-09-13T03:15:11.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.623+0000 7effe2d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effe41005a0 0x7effe419e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:11.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.623+0000 7effe2d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effe41005a0 0x7effe419e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54826/0 (socket says 172.21.15.64:54826) 2023-09-13T03:15:11.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.623+0000 7effe2d9d700 1 -- 172.21.15.64:0/1992566050 learned_addr learned my addr 172.21.15.64:0/1992566050 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:15:11.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.624+0000 7effe359e700 1 --2- 172.21.15.64:0/1992566050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effe4106660 0x7effe4198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:11.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.624+0000 7effe259c700 1 --2- 172.21.15.64:0/1992566050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effe4100f60 0x7effe419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:11.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.624+0000 7effe2d9d700 1 -- 172.21.15.64:0/1992566050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effe4100f60 msgr2=0x7effe419ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:15:11.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.624+0000 7effe2d9d700 1 --2- 172.21.15.64:0/1992566050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effe4100f60 0x7effe419ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:11.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.624+0000 7effe2d9d700 1 -- 172.21.15.64:0/1992566050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effe4106660 msgr2=0x7effe4198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:15:11.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.624+0000 7effe2d9d700 1 --2- 172.21.15.64:0/1992566050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effe4106660 0x7effe4198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:11.631 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.624+0000 7effe2d9d700 1 -- 172.21.15.64:0/1992566050 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd40095f0 con 0x7effe41005a0 2023-09-13T03:15:11.631 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.624+0000 7effe2d9d700 1 --2- 172.21.15.64:0/1992566050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effe41005a0 0x7effe419e7e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7effd400b580 tx=0x7effd400bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:15:11.631 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.624+0000 7effcbfff700 1 -- 172.21.15.64:0/1992566050 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd401d070 con 0x7effe41005a0 2023-09-13T03:15:11.631 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.625+0000 7effe91ff700 1 -- 172.21.15.64:0/1992566050 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effe4199260 con 0x7effe41005a0 2023-09-13T03:15:11.631 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.625+0000 7effcbfff700 1 -- 172.21.15.64:0/1992566050 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effd4005050 con 0x7effe41005a0 2023-09-13T03:15:11.631 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.625+0000 7effcbfff700 1 -- 172.21.15.64:0/1992566050 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd4004a40 con 0x7effe41005a0 2023-09-13T03:15:11.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.625+0000 7effe91ff700 1 -- 172.21.15.64:0/1992566050 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effe41a57d0 con 0x7effe41005a0 2023-09-13T03:15:11.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.628+0000 7effcbfff700 1 -- 172.21.15.64:0/1992566050 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7effd40051c0 con 0x7effe41005a0 2023-09-13T03:15:11.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.630+0000 7efffcb78b80 1 Processor -- start 2023-09-13T03:15:11.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.630+0000 7efffcb78b80 1 -- start start 2023-09-13T03:15:11.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.630+0000 7efffcb78b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56247c3201a0 0x56247c320580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:11.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.631+0000 7efffcb78b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56247c31a300 0x56247c31a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:11.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.631+0000 7efffcb78b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56247c31acc0 0x56247c326f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:11.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.631+0000 7efffcb78b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247c2736d0 con 0x56247c3201a0 2023-09-13T03:15:11.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.631+0000 7efffcb78b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247c270ff0 con 0x56247c31a300 2023-09-13T03:15:11.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.631+0000 7efffcb78b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247c26c8e0 con 0x56247c31acc0 2023-09-13T03:15:11.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.631+0000 7effcbfff700 1 --2- 172.21.15.64:0/1992566050 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7effcc0a07b0 0x7effcc0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:11.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.631+0000 7effcbfff700 1 -- 172.21.15.64:0/1992566050 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7effd40ccce0 con 0x7effe41005a0 2023-09-13T03:15:11.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.631+0000 7effe359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56247c3201a0 0x56247c320580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:11.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.631+0000 7effe359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56247c3201a0 0x56247c320580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54840/0 (socket says 172.21.15.64:54840) 2023-09-13T03:15:11.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.631+0000 7effe359e700 1 -- 172.21.15.64:0/1387109603 learned_addr learned my addr 172.21.15.64:0/1387109603 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:15:11.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.631+0000 7effe259c700 1 --2- 172.21.15.64:0/1387109603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56247c31acc0 0x56247c326f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:11.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.631+0000 7effe2d9d700 1 --2- 172.21.15.64:0/1387109603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56247c31a300 0x56247c31a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:11.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.631+0000 7effe359e700 1 --2- 172.21.15.64:0/1992566050 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7effcc0a07b0 0x7effcc0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:11.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.631+0000 7effe259c700 1 -- 172.21.15.64:0/1387109603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56247c31a300 msgr2=0x56247c31a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:15:11.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.631+0000 7effe259c700 1 --2- 172.21.15.64:0/1387109603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56247c31a300 0x56247c31a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:11.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.631+0000 7effe259c700 1 -- 172.21.15.64:0/1387109603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56247c3201a0 msgr2=0x56247c320580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:15:11.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.631+0000 7effe259c700 1 --2- 172.21.15.64:0/1387109603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56247c3201a0 0x56247c320580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:11.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.631+0000 7effe259c700 1 -- 172.21.15.64:0/1387109603 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56247c098d80 con 0x56247c31acc0 2023-09-13T03:15:11.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.632+0000 7effe2d9d700 1 --2- 172.21.15.64:0/1387109603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56247c31a300 0x56247c31a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:15:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.632+0000 7effe359e700 1 --2- 172.21.15.64:0/1387109603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56247c3201a0 0x56247c320580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:15:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.632+0000 7effe259c700 1 --2- 172.21.15.64:0/1387109603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56247c31acc0 0x56247c326f30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7effd8009c80 tx=0x7effd8009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=144b256551dc164f server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:15:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.632+0000 7effe359e700 1 --2- 172.21.15.64:0/1992566050 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7effcc0a07b0 0x7effcc0a2c70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7effdc009fd0 tx=0x7effdc00b040 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:15:11.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.632+0000 7effc97fa700 1 -- 172.21.15.64:0/1387109603 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd8011cb0 con 0x56247c31acc0 2023-09-13T03:15:11.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.632+0000 7effc97fa700 1 -- 172.21.15.64:0/1387109603 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effd8011e10 con 0x56247c31acc0 2023-09-13T03:15:11.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.632+0000 7effc97fa700 1 -- 172.21.15.64:0/1387109603 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd8005280 con 0x56247c31acc0 2023-09-13T03:15:11.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.633+0000 7efffcb78b80 1 -- 172.21.15.64:0/1387109603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56247c31acc0 msgr2=0x56247c326f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:15:11.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.633+0000 7efffcb78b80 1 --2- 172.21.15.64:0/1387109603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56247c31acc0 0x56247c326f30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7effd8009c80 tx=0x7effd8009f90 comp rx=0 tx=0).stop 2023-09-13T03:15:11.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.633+0000 7efffcb78b80 1 -- 172.21.15.64:0/1387109603 shutdown_connections 2023-09-13T03:15:11.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.633+0000 7efffcb78b80 1 --2- 172.21.15.64:0/1387109603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56247c31acc0 0x56247c326f30 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:11.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.633+0000 7efffcb78b80 1 --2- 172.21.15.64:0/1387109603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56247c3201a0 0x56247c320580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:11.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.633+0000 7efffcb78b80 1 --2- 172.21.15.64:0/1387109603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56247c31a300 0x56247c31a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:11.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.633+0000 7efffcb78b80 1 -- 172.21.15.64:0/1387109603 >> 172.21.15.64:0/1387109603 conn(0x56247c28f030 msgr2=0x56247c315560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:15:11.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.633+0000 7efffcb78b80 1 -- 172.21.15.64:0/1387109603 shutdown_connections 2023-09-13T03:15:11.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.633+0000 7efffcb78b80 1 -- 172.21.15.64:0/1387109603 wait complete. 2023-09-13T03:15:11.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.634+0000 7efffcb78b80 1 -- start start 2023-09-13T03:15:11.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.634+0000 7efffcb78b80 1 Processor -- start 2023-09-13T03:15:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.635+0000 7efffcb78b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56247c31a300 0x56247c326c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.635+0000 7efffcb78b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56247c31acc0 0x56247c324000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:11.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.635+0000 7efffcb78b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56247c3201a0 0x56247c324540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:11.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.635+0000 7efffcb78b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247c20ab10 con 0x56247c3201a0 2023-09-13T03:15:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.635+0000 7efffcb78b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247c20a810 con 0x56247c31acc0 2023-09-13T03:15:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.635+0000 7efffcb78b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247c0ee8c0 con 0x56247c31a300 2023-09-13T03:15:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.635+0000 7efffcb78b80 10 client.0 _ll_register_callbacks cb 0x56247c154a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:15:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.635+0000 7effe2d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56247c31a300 0x56247c326c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.635+0000 7effe2d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56247c31a300 0x56247c326c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:56362/0 (socket says 172.21.15.64:56362) 2023-09-13T03:15:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.635+0000 7effe2d9d700 1 -- 172.21.15.64:0/441098243 learned_addr learned my addr 172.21.15.64:0/441098243 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:15:11.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.635+0000 7effe359e700 1 --2- 172.21.15.64:0/441098243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56247c3201a0 0x56247c324540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:11.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.635+0000 7effe259c700 1 --2- 172.21.15.64:0/441098243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56247c31acc0 0x56247c324000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:11.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.635+0000 7effe2d9d700 1 -- 172.21.15.64:0/441098243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56247c31acc0 msgr2=0x56247c324000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:15:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.635+0000 7effe2d9d700 1 --2- 172.21.15.64:0/441098243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56247c31acc0 0x56247c324000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.635+0000 7effe2d9d700 1 -- 172.21.15.64:0/441098243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56247c3201a0 msgr2=0x56247c324540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:15:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.635+0000 7effe2d9d700 1 --2- 172.21.15.64:0/441098243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56247c3201a0 0x56247c324540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.636+0000 7effe2d9d700 1 -- 172.21.15.64:0/441098243 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effe41a57d0 con 0x56247c31a300 2023-09-13T03:15:11.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.636+0000 7effe2d9d700 1 --2- 172.21.15.64:0/441098243 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56247c31a300 0x56247c326c70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7effd4096c80 tx=0x7effd4005780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=980e2f933675260 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:15:11.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.636+0000 7effc6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:15:11.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.636+0000 7effc6ff5700 1 -- 172.21.15.64:0/441098243 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd409a670 con 0x56247c31a300 2023-09-13T03:15:11.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.636+0000 7efffcb78b80 1 -- 172.21.15.64:0/441098243 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56247c1c4490 con 0x56247c31a300 2023-09-13T03:15:11.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.636+0000 7effc6ff5700 1 -- 172.21.15.64:0/441098243 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7effd40a21f0 con 0x56247c31a300 2023-09-13T03:15:11.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.636+0000 7effc6ff5700 1 -- 172.21.15.64:0/441098243 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7effd4099700 con 0x56247c31a300 2023-09-13T03:15:11.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.636+0000 7efffcb78b80 10 client.26284 fetch_fsmap learned FSMap version 33 2023-09-13T03:15:11.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.636+0000 7efffcb78b80 1 -- 172.21.15.64:0/441098243 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7effdc008b30 con 0x56247c31a300 2023-09-13T03:15:11.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.636+0000 7effc6ff5700 1 -- 172.21.15.64:0/441098243 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56247c1c4490 con 0x56247c31a300 2023-09-13T03:15:11.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.637+0000 7effc6ff5700 1 -- 172.21.15.64:0/441098243 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 33) v1 ==== 7910+0+0 (secure 0 0 0) 0x7effd4004ba0 con 0x56247c31a300 2023-09-13T03:15:11.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.637+0000 7efffcb78b80 10 client.26284 fetch_fsmap finished waiting for FSMap version 33 2023-09-13T03:15:11.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.637+0000 7efffcb78b80 10 client.26284 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:15:11.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.637+0000 7efffcb78b80 20 client.26284 populate_metadata read hostname 'smithi064' 2023-09-13T03:15:11.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.637+0000 7efffcb78b80 1 --2- 172.21.15.64:0/441098243 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56247c333fd0 0x56247c3363f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:11.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.637+0000 7efffcb78b80 4 client.26284 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:15:11.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.637+0000 7efffcb78b80 1 -- 172.21.15.64:0/441098243 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56247c281840 con 0x56247c333fd0 2023-09-13T03:15:11.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.637+0000 7effe259c700 1 --2- 172.21.15.64:0/441098243 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56247c333fd0 0x56247c3363f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:11.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.638+0000 7effe259c700 1 --2- 172.21.15.64:0/441098243 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56247c333fd0 0x56247c3363f0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65639a10c6a2aa83 server_cookie=d614612a4fa041de in_seq=0 out_seq=0 2023-09-13T03:15:11.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.638+0000 7effc6ff5700 10 client.26284 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:15:11.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.639+0000 7effc6ff5700 1 -- 172.21.15.64:0/441098243 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56247c281840 con 0x56247c333fd0 2023-09-13T03:15:11.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.639+0000 7effc6ff5700 10 client.26284 handle_command_reply: tid=0 2023-09-13T03:15:11.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.639+0000 7efffcb78b80 1 client.26284 shutdown 2023-09-13T03:15:11.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.639+0000 7efffcb78b80 1 -- 172.21.15.64:0/441098243 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56247c31a300 msgr2=0x56247c326c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:15:11.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.639+0000 7efffcb78b80 1 --2- 172.21.15.64:0/441098243 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56247c31a300 0x56247c326c70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7effd4096c80 tx=0x7effd4005780 comp rx=0 tx=0).stop 2023-09-13T03:15:11.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.639+0000 7efffcb78b80 1 -- 172.21.15.64:0/441098243 shutdown_connections 2023-09-13T03:15:11.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.639+0000 7efffcb78b80 1 --2- 172.21.15.64:0/441098243 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56247c333fd0 0x56247c3363f0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:11.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.639+0000 7efffcb78b80 1 --2- 172.21.15.64:0/441098243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56247c3201a0 0x56247c324540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:11.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.639+0000 7efffcb78b80 1 --2- 172.21.15.64:0/441098243 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56247c31a300 0x56247c326c70 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:11.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.639+0000 7effc6ff5700 0 client.26284 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:15:11.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.639+0000 7efffcb78b80 1 --2- 172.21.15.64:0/441098243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56247c31acc0 0x56247c324000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:11.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.639+0000 7efffcb78b80 1 -- 172.21.15.64:0/441098243 >> 172.21.15.64:0/441098243 conn(0x56247c28f030 msgr2=0x56247c3147a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:15:11.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.639+0000 7efffcb78b80 1 -- 172.21.15.64:0/441098243 shutdown_connections 2023-09-13T03:15:11.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.639+0000 7efffcb78b80 1 -- 172.21.15.64:0/441098243 wait complete. 2023-09-13T03:15:11.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.639+0000 7efffcb78b80 20 client.26284 trim_cache size 0 max 16384 2023-09-13T03:15:11.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.655+0000 7efffcb78b80 1 Processor -- start 2023-09-13T03:15:11.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.655+0000 7efffcb78b80 1 -- start start 2023-09-13T03:15:11.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.655+0000 7efffcb78b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56247c31a300 0x56247c31f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:11.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.655+0000 7efffcb78b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56247c31acc0 0x56247c3267a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:11.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.655+0000 7efffcb78b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56247c3201a0 0x56247c31cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:11.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.655+0000 7efffcb78b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247c30f8e0 con 0x56247c3201a0 2023-09-13T03:15:11.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.655+0000 7efffcb78b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247c316d90 con 0x56247c31acc0 2023-09-13T03:15:11.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.655+0000 7efffcb78b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247c330c90 con 0x56247c31a300 2023-09-13T03:15:11.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.656+0000 7effe359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56247c3201a0 0x56247c31cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:11.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.656+0000 7effe259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56247c31acc0 0x56247c3267a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:11.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.656+0000 7effe2d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56247c31a300 0x56247c31f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:11.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.656+0000 7effe259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56247c31acc0 0x56247c3267a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:44488/0 (socket says 172.21.15.64:44488) 2023-09-13T03:15:11.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.656+0000 7effe259c700 1 -- 172.21.15.64:0/1048071079 learned_addr learned my addr 172.21.15.64:0/1048071079 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:15:11.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.656+0000 7effe359e700 1 --2- 172.21.15.64:0/1048071079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56247c3201a0 0x56247c31cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54850/0 (socket says 172.21.15.64:54850) 2023-09-13T03:15:11.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.656+0000 7effe2d9d700 1 -- 172.21.15.64:0/1048071079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56247c31acc0 msgr2=0x56247c3267a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:15:11.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.656+0000 7effe2d9d700 1 --2- 172.21.15.64:0/1048071079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56247c31acc0 0x56247c3267a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:11.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.656+0000 7effe2d9d700 1 -- 172.21.15.64:0/1048071079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56247c3201a0 msgr2=0x56247c31cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:15:11.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.656+0000 7effe2d9d700 1 --2- 172.21.15.64:0/1048071079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56247c3201a0 0x56247c31cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:11.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.656+0000 7effe2d9d700 1 -- 172.21.15.64:0/1048071079 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd80098e0 con 0x56247c31a300 2023-09-13T03:15:11.673 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.656+0000 7effe259c700 1 --2- 172.21.15.64:0/1048071079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56247c31acc0 0x56247c3267a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:15:11.673 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.656+0000 7effe2d9d700 1 --2- 172.21.15.64:0/1048071079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56247c31a300 0x56247c31f760 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7effd40036c0 tx=0x7effd40094b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=167f12e007646d7a server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:15:11.673 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.656+0000 7effc9ffb700 1 -- 172.21.15.64:0/1048071079 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd40a1d60 con 0x56247c31a300 2023-09-13T03:15:11.674 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.656+0000 7effc9ffb700 1 -- 172.21.15.64:0/1048071079 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effd40a21f0 con 0x56247c31a300 2023-09-13T03:15:11.674 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.656+0000 7effc9ffb700 1 -- 172.21.15.64:0/1048071079 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd409a470 con 0x56247c31a300 2023-09-13T03:15:11.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.657+0000 7efffcb78b80 1 -- 172.21.15.64:0/1048071079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56247c31a300 msgr2=0x56247c31f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:15:11.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.657+0000 7efffcb78b80 1 --2- 172.21.15.64:0/1048071079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56247c31a300 0x56247c31f760 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7effd40036c0 tx=0x7effd40094b0 comp rx=0 tx=0).stop 2023-09-13T03:15:11.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.657+0000 7efffcb78b80 1 -- 172.21.15.64:0/1048071079 shutdown_connections 2023-09-13T03:15:11.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.657+0000 7efffcb78b80 1 --2- 172.21.15.64:0/1048071079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56247c3201a0 0x56247c31cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:11.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.657+0000 7efffcb78b80 1 --2- 172.21.15.64:0/1048071079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56247c31a300 0x56247c31f760 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:11.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.657+0000 7efffcb78b80 1 --2- 172.21.15.64:0/1048071079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56247c31acc0 0x56247c3267a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:11.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.657+0000 7efffcb78b80 1 -- 172.21.15.64:0/1048071079 >> 172.21.15.64:0/1048071079 conn(0x56247c28f030 msgr2=0x56247c319b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:15:11.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.657+0000 7efffcb78b80 1 -- 172.21.15.64:0/1048071079 shutdown_connections 2023-09-13T03:15:11.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.657+0000 7efffcb78b80 1 -- 172.21.15.64:0/1048071079 wait complete. 2023-09-13T03:15:11.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.658+0000 7efffcb78b80 1 -- start start 2023-09-13T03:15:11.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.658+0000 7efffcb78b80 1 Processor -- start 2023-09-13T03:15:11.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.659+0000 7efffcb78b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56247c31a300 0x56247c32b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:11.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.659+0000 7efffcb78b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56247c31acc0 0x56247c329940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:11.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.659+0000 7efffcb78b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56247c3201a0 0x56247c329f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:11.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.659+0000 7efffcb78b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247c32a610 con 0x56247c31a300 2023-09-13T03:15:11.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.659+0000 7efffcb78b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247c32a780 con 0x56247c31acc0 2023-09-13T03:15:11.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.659+0000 7efffcb78b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247c387e10 con 0x56247c3201a0 2023-09-13T03:15:11.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.659+0000 7efffcb78b80 10 client.0 _ll_register_callbacks cb 0x7effd4099b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:15:11.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.659+0000 7effe2d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56247c31a300 0x56247c32b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:11.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.659+0000 7effe2d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56247c31a300 0x56247c32b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54862/0 (socket says 172.21.15.64:54862) 2023-09-13T03:15:11.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.659+0000 7effe2d9d700 1 -- 172.21.15.64:0/3298370646 learned_addr learned my addr 172.21.15.64:0/3298370646 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:15:11.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.659+0000 7effe359e700 1 --2- 172.21.15.64:0/3298370646 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56247c3201a0 0x56247c329f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.659+0000 7effe259c700 1 --2- 172.21.15.64:0/3298370646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56247c31acc0 0x56247c329940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.659+0000 7effe2d9d700 1 -- 172.21.15.64:0/3298370646 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56247c3201a0 msgr2=0x56247c329f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:15:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.659+0000 7effe2d9d700 1 --2- 172.21.15.64:0/3298370646 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56247c3201a0 0x56247c329f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.659+0000 7effe2d9d700 1 -- 172.21.15.64:0/3298370646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56247c31acc0 msgr2=0x56247c329940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:15:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.659+0000 7effe2d9d700 1 --2- 172.21.15.64:0/3298370646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56247c31acc0 0x56247c329940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:11.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.659+0000 7effe2d9d700 1 -- 172.21.15.64:0/3298370646 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effd801d040 con 0x56247c31a300 2023-09-13T03:15:11.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.660+0000 7effe2d9d700 1 --2- 172.21.15.64:0/3298370646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56247c31a300 0x56247c32b610 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7effd4009fd0 tx=0x7effd4005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d89eceafe1c1310 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:15:11.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.660+0000 7effc6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:15:11.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.660+0000 7effc6ff5700 1 -- 172.21.15.64:0/3298370646 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd40a1d60 con 0x56247c31a300 2023-09-13T03:15:11.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.660+0000 7efffcb78b80 1 -- 172.21.15.64:0/3298370646 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7effd8011e10 con 0x56247c31a300 2023-09-13T03:15:11.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.660+0000 7effc6ff5700 1 -- 172.21.15.64:0/3298370646 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7effd40601f0 con 0x56247c31a300 2023-09-13T03:15:11.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.660+0000 7effc6ff5700 1 -- 172.21.15.64:0/3298370646 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7effd409a670 con 0x56247c31a300 2023-09-13T03:15:11.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.660+0000 7efffcb78b80 10 client.16251 fetch_fsmap learned FSMap version 33 2023-09-13T03:15:11.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.660+0000 7efffcb78b80 1 -- 172.21.15.64:0/3298370646 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7effdc008b30 con 0x56247c31a300 2023-09-13T03:15:11.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.660+0000 7effc6ff5700 1 -- 172.21.15.64:0/3298370646 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effd8011e10 con 0x56247c31a300 2023-09-13T03:15:11.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.660+0000 7effc6ff5700 1 -- 172.21.15.64:0/3298370646 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 33) v1 ==== 7910+0+0 (secure 0 0 0) 0x7effd4099170 con 0x56247c31a300 2023-09-13T03:15:11.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.661+0000 7efffcb78b80 10 client.16251 fetch_fsmap finished waiting for FSMap version 33 2023-09-13T03:15:11.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.661+0000 7efffcb78b80 10 client.16251 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:15:11.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.661+0000 7efffcb78b80 20 client.16251 populate_metadata read hostname 'smithi064' 2023-09-13T03:15:11.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.661+0000 7efffcb78b80 1 --2- 172.21.15.64:0/3298370646 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56247c3891e0 0x56247c38b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:11.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.661+0000 7efffcb78b80 4 client.16251 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:15:11.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.661+0000 7efffcb78b80 1 -- 172.21.15.64:0/3298370646 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56247c2658c0 con 0x56247c3891e0 2023-09-13T03:15:11.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.661+0000 7effe259c700 1 --2- 172.21.15.64:0/3298370646 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56247c3891e0 0x56247c38b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:11.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.661+0000 7effe259c700 1 --2- 172.21.15.64:0/3298370646 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56247c3891e0 0x56247c38b600 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79435c01058edfeb server_cookie=f1f7c8f13d7774a5 in_seq=0 out_seq=0 2023-09-13T03:15:11.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.661+0000 7effc6ff5700 10 client.16251 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:15:11.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.689+0000 7effc6ff5700 1 -- 172.21.15.64:0/3298370646 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56247c2658c0 con 0x56247c3891e0 2023-09-13T03:15:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.689+0000 7effc6ff5700 10 client.16251 handle_command_reply: tid=0 2023-09-13T03:15:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.689+0000 7efffcb78b80 1 client.16251 shutdown 2023-09-13T03:15:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.689+0000 7efffcb78b80 1 -- 172.21.15.64:0/3298370646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56247c31a300 msgr2=0x56247c32b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:15:11.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.689+0000 7efffcb78b80 1 --2- 172.21.15.64:0/3298370646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56247c31a300 0x56247c32b610 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7effd4009fd0 tx=0x7effd4005780 comp rx=0 tx=0).stop 2023-09-13T03:15:11.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.690+0000 7efffcb78b80 1 -- 172.21.15.64:0/3298370646 shutdown_connections 2023-09-13T03:15:11.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.690+0000 7efffcb78b80 1 --2- 172.21.15.64:0/3298370646 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56247c3891e0 0x56247c38b600 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.690+0000 7efffcb78b80 1 --2- 172.21.15.64:0/3298370646 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56247c3201a0 0x56247c329f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.690+0000 7efffcb78b80 1 --2- 172.21.15.64:0/3298370646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56247c31a300 0x56247c32b610 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.690+0000 7efffcb78b80 1 --2- 172.21.15.64:0/3298370646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56247c31acc0 0x56247c329940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.690+0000 7effc6ff5700 0 client.16251 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:15:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.690+0000 7efffcb78b80 1 -- 172.21.15.64:0/3298370646 >> 172.21.15.64:0/3298370646 conn(0x56247c28f030 msgr2=0x56247c32fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:15:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.690+0000 7efffcb78b80 1 -- 172.21.15.64:0/3298370646 shutdown_connections 2023-09-13T03:15:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.690+0000 7efffcb78b80 1 -- 172.21.15.64:0/3298370646 wait complete. 2023-09-13T03:15:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.690+0000 7efffcb78b80 20 client.16251 trim_cache size 0 max 16384 2023-09-13T03:15:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:15:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:15:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:15:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:15:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.691+0000 7effe91ff700 1 -- 172.21.15.64:0/1992566050 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7effcc0a07b0 msgr2=0x7effcc0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:15:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.691+0000 7effe91ff700 1 --2- 172.21.15.64:0/1992566050 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7effcc0a07b0 0x7effcc0a2c70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7effdc009fd0 tx=0x7effdc00b040 comp rx=0 tx=0).stop 2023-09-13T03:15:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.691+0000 7effe91ff700 1 -- 172.21.15.64:0/1992566050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effe41005a0 msgr2=0x7effe419e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:15:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.691+0000 7effe91ff700 1 --2- 172.21.15.64:0/1992566050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effe41005a0 0x7effe419e7e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7effd400b580 tx=0x7effd400bfd0 comp rx=0 tx=0).stop 2023-09-13T03:15:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.691+0000 7effe91ff700 1 -- 172.21.15.64:0/1992566050 shutdown_connections 2023-09-13T03:15:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.691+0000 7effe91ff700 1 --2- 172.21.15.64:0/1992566050 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7effcc0a07b0 0x7effcc0a2c70 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.691+0000 7effe91ff700 1 --2- 172.21.15.64:0/1992566050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effe4106660 0x7effe4198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.691+0000 7effe91ff700 1 --2- 172.21.15.64:0/1992566050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effe41005a0 0x7effe419e7e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.691+0000 7effe91ff700 1 --2- 172.21.15.64:0/1992566050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effe4100f60 0x7effe419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.691+0000 7effe91ff700 1 -- 172.21.15.64:0/1992566050 >> 172.21.15.64:0/1992566050 conn(0x7effe40fc050 msgr2=0x7effe40fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:15:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.691+0000 7effe91ff700 1 -- 172.21.15.64:0/1992566050 shutdown_connections 2023-09-13T03:15:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:11.691+0000 7effe91ff700 1 -- 172.21.15.64:0/1992566050 wait complete. 2023-09-13T03:15:11.892 INFO:tasks.workunit.client.0.smithi064.stdout: 1 43320 3.30 MB/sec warmup 76 sec latency 1502.971 ms 2023-09-13T03:15:12.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:15:12 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:15:12.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:15:12 smithi170 ceph-mon[134328]: pgmap v64: 97 pgs: 97 active+clean; 36 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 649 KiB/s wr, 139 op/s 2023-09-13T03:15:12.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:15:12 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:15:12.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:15:12 smithi064 ceph-mon[97675]: pgmap v64: 97 pgs: 97 active+clean; 36 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 649 KiB/s wr, 139 op/s 2023-09-13T03:15:12.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:15:12 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:15:12.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:15:12 smithi142 ceph-mon[134082]: pgmap v64: 97 pgs: 97 active+clean; 36 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 649 KiB/s wr, 139 op/s 2023-09-13T03:15:12.892 INFO:tasks.workunit.client.0.smithi064.stdout: 1 43320 3.26 MB/sec warmup 77 sec latency 2503.136 ms 2023-09-13T03:15:13.892 INFO:tasks.workunit.client.0.smithi064.stdout: 1 43320 3.21 MB/sec warmup 78 sec latency 3503.261 ms 2023-09-13T03:15:14.892 INFO:tasks.workunit.client.0.smithi064.stdout: 1 43320 3.17 MB/sec warmup 79 sec latency 4503.341 ms 2023-09-13T03:15:14.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:15:14 smithi170 ceph-mon[134328]: pgmap v65: 97 pgs: 97 active+clean; 36 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 571 KiB/s wr, 101 op/s 2023-09-13T03:15:14.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:15:14 smithi064 ceph-mon[97675]: pgmap v65: 97 pgs: 97 active+clean; 36 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 571 KiB/s wr, 101 op/s 2023-09-13T03:15:14.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:15:14 smithi142 ceph-mon[134082]: pgmap v65: 97 pgs: 97 active+clean; 36 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 571 KiB/s wr, 101 op/s 2023-09-13T03:15:15.893 INFO:tasks.workunit.client.0.smithi064.stdout: 1 43388 3.14 MB/sec warmup 80 sec latency 4965.012 ms 2023-09-13T03:15:16.893 INFO:tasks.workunit.client.0.smithi064.stdout: 1 43511 3.10 MB/sec warmup 81 sec latency 1193.600 ms 2023-09-13T03:15:16.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:15:16 smithi170 ceph-mon[134328]: pgmap v66: 97 pgs: 97 active+clean; 32 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 622 KiB/s wr, 123 op/s 2023-09-13T03:15:16.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:15:16 smithi064 ceph-mon[97675]: pgmap v66: 97 pgs: 97 active+clean; 32 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 622 KiB/s wr, 123 op/s 2023-09-13T03:15:16.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:15:16 smithi142 ceph-mon[134082]: pgmap v66: 97 pgs: 97 active+clean; 32 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 622 KiB/s wr, 123 op/s 2023-09-13T03:15:17.893 INFO:tasks.workunit.client.0.smithi064.stdout: 1 43511 3.06 MB/sec warmup 82 sec latency 1216.832 ms 2023-09-13T03:15:18.893 INFO:tasks.workunit.client.0.smithi064.stdout: 1 43577 3.03 MB/sec warmup 83 sec latency 1842.998 ms 2023-09-13T03:15:18.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:15:18 smithi170 ceph-mon[134328]: pgmap v67: 97 pgs: 97 active+clean; 32 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 250 KiB/s wr, 64 op/s 2023-09-13T03:15:18.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:15:18 smithi064 ceph-mon[97675]: pgmap v67: 97 pgs: 97 active+clean; 32 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 250 KiB/s wr, 64 op/s 2023-09-13T03:15:18.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:15:18 smithi142 ceph-mon[134082]: pgmap v67: 97 pgs: 97 active+clean; 32 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 250 KiB/s wr, 64 op/s 2023-09-13T03:15:19.893 INFO:tasks.workunit.client.0.smithi064.stdout: 1 43577 2.99 MB/sec warmup 84 sec latency 1327.619 ms 2023-09-13T03:15:20.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:15:20 smithi170 ceph-mon[134328]: pgmap v68: 97 pgs: 97 active+clean; 32 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 250 KiB/s wr, 64 op/s 2023-09-13T03:15:20.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:15:20 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:15:20.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:15:20 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:15:20.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:15:20 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:15:20.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:15:20 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:15:20.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:15:20 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:15:20.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:15:20 smithi064 ceph-mon[97675]: pgmap v68: 97 pgs: 97 active+clean; 32 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 250 KiB/s wr, 64 op/s 2023-09-13T03:15:20.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:15:20 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:15:20.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:15:20 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:15:20.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:15:20 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:15:20.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:15:20 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:15:20.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:15:20 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:15:20.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:15:20 smithi142 ceph-mon[134082]: pgmap v68: 97 pgs: 97 active+clean; 32 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 250 KiB/s wr, 64 op/s 2023-09-13T03:15:20.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:15:20 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:15:20.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:15:20 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:15:20.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:15:20 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:15:20.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:15:20 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:15:20.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:15:20 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:15:20.893 INFO:tasks.workunit.client.0.smithi064.stdout: 1 43577 2.96 MB/sec warmup 85 sec latency 2327.663 ms 2023-09-13T03:15:21.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:15:21 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:15:21.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:15:21 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:15:21.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:15:21 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:15:21.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:15:21 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:15:21.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:15:21 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:15:21.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:15:21 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:15:21.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:15:21 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:15:21.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:15:21 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:15:21.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:15:21 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:15:21.893 INFO:tasks.workunit.client.0.smithi064.stdout: 1 43577 2.92 MB/sec warmup 86 sec latency 3327.783 ms 2023-09-13T03:15:22.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:15:22 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:15:22.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:15:22 smithi170 ceph-mon[134328]: pgmap v69: 97 pgs: 97 active+clean; 33 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2023-09-13T03:15:22.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:15:22 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:15:22.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:15:22 smithi064 ceph-mon[97675]: pgmap v69: 97 pgs: 97 active+clean; 33 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2023-09-13T03:15:22.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:15:22 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:15:22.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:15:22 smithi142 ceph-mon[134082]: pgmap v69: 97 pgs: 97 active+clean; 33 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2023-09-13T03:15:22.893 INFO:tasks.workunit.client.0.smithi064.stdout: 1 43577 2.89 MB/sec warmup 87 sec latency 4327.924 ms 2023-09-13T03:15:23.893 INFO:tasks.workunit.client.0.smithi064.stdout: 1 44458 2.91 MB/sec warmup 88 sec latency 4955.850 ms 2023-09-13T03:15:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.844+0000 7f376b7ff700 1 -- 172.21.15.64:0/1119775750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3764100f70 msgr2=0x7f376410d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:15:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.844+0000 7f376b7ff700 1 --2- 172.21.15.64:0/1119775750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3764100f70 0x7f376410d430 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f375c00b0d0 tx=0x7f375c00b3e0 comp rx=0 tx=0).stop 2023-09-13T03:15:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.844+0000 7f376b7ff700 1 -- 172.21.15.64:0/1119775750 shutdown_connections 2023-09-13T03:15:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.844+0000 7f376b7ff700 1 --2- 172.21.15.64:0/1119775750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3764100f70 0x7f376410d430 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.844+0000 7f376b7ff700 1 --2- 172.21.15.64:0/1119775750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3764106670 0x7f3764106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.844+0000 7f376b7ff700 1 --2- 172.21.15.64:0/1119775750 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37641005b0 0x7f3764100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.844+0000 7f376b7ff700 1 -- 172.21.15.64:0/1119775750 >> 172.21.15.64:0/1119775750 conn(0x7f37640fc060 msgr2=0x7f37640fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:15:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.844+0000 7f376b7ff700 1 -- 172.21.15.64:0/1119775750 shutdown_connections 2023-09-13T03:15:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.845+0000 7f376b7ff700 1 -- 172.21.15.64:0/1119775750 wait complete. 2023-09-13T03:15:24.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.846+0000 7f376b7ff700 1 Processor -- start 2023-09-13T03:15:24.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.846+0000 7f376b7ff700 1 -- start start 2023-09-13T03:15:24.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.846+0000 7f376b7ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37641005b0 0x7f376419e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:24.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.846+0000 7f376b7ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3764100f70 0x7f376419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:24.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.846+0000 7f376b7ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3764106670 0x7f37641987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:24.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.846+0000 7f376b7ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3764198e40 con 0x7f3764100f70 2023-09-13T03:15:24.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.846+0000 7f376b7ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3764198fb0 con 0x7f37641005b0 2023-09-13T03:15:24.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.846+0000 7f376b7ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3764199170 con 0x7f3764106670 2023-09-13T03:15:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.846+0000 7f376959b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37641005b0 0x7f376419e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.846+0000 7f376959b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37641005b0 0x7f376419e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:53272/0 (socket says 172.21.15.64:53272) 2023-09-13T03:15:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.846+0000 7f376959b700 1 -- 172.21.15.64:0/1164077434 learned_addr learned my addr 172.21.15.64:0/1164077434 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:15:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.846+0000 7f3769d9c700 1 --2- 172.21.15.64:0/1164077434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3764106670 0x7f37641987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.847+0000 7f3768d9a700 1 --2- 172.21.15.64:0/1164077434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3764100f70 0x7f376419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.847+0000 7f376959b700 1 -- 172.21.15.64:0/1164077434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3764106670 msgr2=0x7f37641987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:15:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.847+0000 7f376959b700 1 --2- 172.21.15.64:0/1164077434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3764106670 0x7f37641987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.847+0000 7f376959b700 1 -- 172.21.15.64:0/1164077434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3764100f70 msgr2=0x7f376419eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:15:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.847+0000 7f376959b700 1 --2- 172.21.15.64:0/1164077434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3764100f70 0x7f376419eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.847+0000 7f376959b700 1 -- 172.21.15.64:0/1164077434 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3760009520 con 0x7f37641005b0 2023-09-13T03:15:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.847+0000 7f3769d9c700 1 --2- 172.21.15.64:0/1164077434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3764106670 0x7f37641987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:15:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.847+0000 7f376959b700 1 --2- 172.21.15.64:0/1164077434 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37641005b0 0x7f376419e970 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f376000eb30 tx=0x7f376000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:15:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.847+0000 7f375a7fc700 1 -- 172.21.15.64:0/1164077434 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3760009880 con 0x7f37641005b0 2023-09-13T03:15:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.848+0000 7f375a7fc700 1 -- 172.21.15.64:0/1164077434 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37600044e0 con 0x7f37641005b0 2023-09-13T03:15:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.848+0000 7f376b7ff700 1 -- 172.21.15.64:0/1164077434 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f375c009d00 con 0x7f37641005b0 2023-09-13T03:15:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.848+0000 7f375a7fc700 1 -- 172.21.15.64:0/1164077434 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3760003d80 con 0x7f37641005b0 2023-09-13T03:15:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.848+0000 7f376b7ff700 1 -- 172.21.15.64:0/1164077434 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37641a5780 con 0x7f37641005b0 2023-09-13T03:15:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.851+0000 7f375a7fc700 1 -- 172.21.15.64:0/1164077434 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f3760003f20 con 0x7f37641005b0 2023-09-13T03:15:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.852+0000 7f377f1bcb80 1 Processor -- start 2023-09-13T03:15:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.853+0000 7f377f1bcb80 1 -- start start 2023-09-13T03:15:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.853+0000 7f377f1bcb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ba36991a0 0x564ba3699580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.853+0000 7f377f1bcb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ba3693300 0x564ba3693780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.853+0000 7f377f1bcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ba3693cc0 0x564ba369ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.853+0000 7f377f1bcb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ba35ec6d0 con 0x564ba36991a0 2023-09-13T03:15:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.853+0000 7f377f1bcb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ba35e9ff0 con 0x564ba3693300 2023-09-13T03:15:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.853+0000 7f377f1bcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ba35e58e0 con 0x564ba3693cc0 2023-09-13T03:15:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.853+0000 7f375a7fc700 1 --2- 172.21.15.64:0/1164077434 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f37500a07b0 0x7f37500a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.853+0000 7f376959b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ba3693300 0x564ba3693780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.853+0000 7f3768d9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ba3693cc0 0x564ba369ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.853+0000 7f376959b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ba3693300 0x564ba3693780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:53278/0 (socket says 172.21.15.64:53278) 2023-09-13T03:15:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.853+0000 7f376959b700 1 -- 172.21.15.64:0/587506994 learned_addr learned my addr 172.21.15.64:0/587506994 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:15:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.853+0000 7f3769d9c700 1 --2- 172.21.15.64:0/587506994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ba36991a0 0x564ba3699580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.854+0000 7f375a7fc700 1 -- 172.21.15.64:0/1164077434 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f3760014070 con 0x7f37641005b0 2023-09-13T03:15:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.854+0000 7f3769d9c700 1 --2- 172.21.15.64:0/1164077434 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f37500a07b0 0x7f37500a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.854+0000 7f376959b700 1 -- 172.21.15.64:0/587506994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ba3693cc0 msgr2=0x564ba369ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:15:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.854+0000 7f376959b700 1 --2- 172.21.15.64:0/587506994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ba3693cc0 0x564ba369ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.854+0000 7f376959b700 1 -- 172.21.15.64:0/587506994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ba36991a0 msgr2=0x564ba3699580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:15:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.854+0000 7f376959b700 1 --2- 172.21.15.64:0/587506994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ba36991a0 0x564ba3699580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.854+0000 7f376959b700 1 -- 172.21.15.64:0/587506994 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ba3411d80 con 0x564ba3693300 2023-09-13T03:15:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.854+0000 7f3769d9c700 1 --2- 172.21.15.64:0/587506994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ba36991a0 0x564ba3699580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:15:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.854+0000 7f3768d9a700 1 --2- 172.21.15.64:0/587506994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ba3693cc0 0x564ba369ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:15:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.854+0000 7f376959b700 1 --2- 172.21.15.64:0/587506994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ba3693300 0x564ba3693780 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3760095690 tx=0x7f3760003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df1ca603273f3cdc server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:15:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.854+0000 7f374f7fe700 1 -- 172.21.15.64:0/587506994 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3760099910 con 0x564ba3693300 2023-09-13T03:15:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.854+0000 7f374f7fe700 1 -- 172.21.15.64:0/587506994 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f376009d740 con 0x564ba3693300 2023-09-13T03:15:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.854+0000 7f374f7fe700 1 -- 172.21.15.64:0/587506994 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3760099b70 con 0x564ba3693300 2023-09-13T03:15:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.855+0000 7f377f1bcb80 1 -- 172.21.15.64:0/587506994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ba3693300 msgr2=0x564ba3693780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:15:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.855+0000 7f3769d9c700 1 --2- 172.21.15.64:0/1164077434 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f37500a07b0 0x7f37500a2c70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f375c009cd0 tx=0x7f375c00b040 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:15:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.855+0000 7f377f1bcb80 1 --2- 172.21.15.64:0/587506994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ba3693300 0x564ba3693780 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3760095690 tx=0x7f3760003820 comp rx=0 tx=0).stop 2023-09-13T03:15:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.855+0000 7f377f1bcb80 1 -- 172.21.15.64:0/587506994 shutdown_connections 2023-09-13T03:15:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.855+0000 7f377f1bcb80 1 --2- 172.21.15.64:0/587506994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ba3693cc0 0x564ba369ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.855+0000 7f377f1bcb80 1 --2- 172.21.15.64:0/587506994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ba36991a0 0x564ba3699580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.855+0000 7f377f1bcb80 1 --2- 172.21.15.64:0/587506994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ba3693300 0x564ba3693780 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.855+0000 7f377f1bcb80 1 -- 172.21.15.64:0/587506994 >> 172.21.15.64:0/587506994 conn(0x564ba3608030 msgr2=0x564ba368e560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:15:24.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.855+0000 7f377f1bcb80 1 -- 172.21.15.64:0/587506994 shutdown_connections 2023-09-13T03:15:24.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.855+0000 7f377f1bcb80 1 -- 172.21.15.64:0/587506994 wait complete. 2023-09-13T03:15:24.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.856+0000 7f377f1bcb80 1 -- start start 2023-09-13T03:15:24.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.856+0000 7f377f1bcb80 1 Processor -- start 2023-09-13T03:15:24.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.856+0000 7f377f1bcb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ba3693300 0x564ba369fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:24.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.856+0000 7f377f1bcb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ba3693cc0 0x564ba369d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:24.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.856+0000 7f377f1bcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ba36991a0 0x564ba369d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:24.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.857+0000 7f377f1bcb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ba3583b10 con 0x564ba3693cc0 2023-09-13T03:15:24.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.857+0000 7f377f1bcb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ba3583810 con 0x564ba3693300 2023-09-13T03:15:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.857+0000 7f377f1bcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ba34678c0 con 0x564ba36991a0 2023-09-13T03:15:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.857+0000 7f377f1bcb80 10 client.0 _ll_register_callbacks cb 0x564ba34cda20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:15:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.857+0000 7f376959b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ba3693300 0x564ba369fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.857+0000 7f3769d9c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ba36991a0 0x564ba369d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.857+0000 7f376959b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ba3693300 0x564ba369fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:53282/0 (socket says 172.21.15.64:53282) 2023-09-13T03:15:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.857+0000 7f376959b700 1 -- 172.21.15.64:0/853478242 learned_addr learned my addr 172.21.15.64:0/853478242 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:15:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.857+0000 7f3768d9a700 1 --2- 172.21.15.64:0/853478242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ba3693cc0 0x564ba369d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.857+0000 7f376959b700 1 -- 172.21.15.64:0/853478242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ba36991a0 msgr2=0x564ba369d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:15:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.857+0000 7f376959b700 1 --2- 172.21.15.64:0/853478242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ba36991a0 0x564ba369d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.857+0000 7f376959b700 1 -- 172.21.15.64:0/853478242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ba3693cc0 msgr2=0x564ba369d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:15:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.857+0000 7f376959b700 1 --2- 172.21.15.64:0/853478242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ba3693cc0 0x564ba369d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.857+0000 7f376959b700 1 -- 172.21.15.64:0/853478242 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37540098e0 con 0x564ba3693300 2023-09-13T03:15:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.857+0000 7f376959b700 1 --2- 172.21.15.64:0/853478242 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ba3693300 0x564ba369fdc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3760099610 tx=0x7f376000cc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9854032505e73f7d server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:15:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.857+0000 7f374cff9700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:15:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.858+0000 7f374cff9700 1 -- 172.21.15.64:0/853478242 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37600ca040 con 0x564ba3693300 2023-09-13T03:15:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.858+0000 7f374cff9700 1 -- 172.21.15.64:0/853478242 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f3760099910 con 0x564ba3693300 2023-09-13T03:15:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.858+0000 7f377f1bcb80 1 -- 172.21.15.64:0/853478242 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564ba353d490 con 0x564ba3693300 2023-09-13T03:15:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.858+0000 7f374cff9700 1 -- 172.21.15.64:0/853478242 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f376000cdd0 con 0x564ba3693300 2023-09-13T03:15:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.858+0000 7f377f1bcb80 10 client.26303 fetch_fsmap learned FSMap version 34 2023-09-13T03:15:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.858+0000 7f377f1bcb80 1 -- 172.21.15.64:0/853478242 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7f37641a5780 con 0x564ba3693300 2023-09-13T03:15:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.858+0000 7f374cff9700 1 -- 172.21.15.64:0/853478242 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564ba353d490 con 0x564ba3693300 2023-09-13T03:15:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.858+0000 7f374cff9700 1 -- 172.21.15.64:0/853478242 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 34) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f37600a14f0 con 0x564ba3693300 2023-09-13T03:15:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.858+0000 7f377f1bcb80 10 client.26303 fetch_fsmap finished waiting for FSMap version 34 2023-09-13T03:15:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.858+0000 7f377f1bcb80 10 client.26303 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:15:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.858+0000 7f377f1bcb80 20 client.26303 populate_metadata read hostname 'smithi064' 2023-09-13T03:15:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.858+0000 7f377f1bcb80 1 --2- 172.21.15.64:0/853478242 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x564ba36acfa0 0x564ba36af3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.858+0000 7f377f1bcb80 4 client.26303 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:15:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.858+0000 7f377f1bcb80 1 -- 172.21.15.64:0/853478242 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564ba35fa840 con 0x564ba36acfa0 2023-09-13T03:15:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.859+0000 7f3768d9a700 1 --2- 172.21.15.64:0/853478242 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x564ba36acfa0 0x564ba36af3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.859+0000 7f374cff9700 10 client.26303 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:15:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.859+0000 7f3768d9a700 1 --2- 172.21.15.64:0/853478242 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x564ba36acfa0 0x564ba36af3c0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=526a04fa1332b966 server_cookie=14ad3cf2a0440c75 in_seq=0 out_seq=0 2023-09-13T03:15:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.860+0000 7f374cff9700 1 -- 172.21.15.64:0/853478242 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564ba35fa840 con 0x564ba36acfa0 2023-09-13T03:15:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.860+0000 7f374cff9700 10 client.26303 handle_command_reply: tid=0 2023-09-13T03:15:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.860+0000 7f377f1bcb80 1 client.26303 shutdown 2023-09-13T03:15:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.860+0000 7f377f1bcb80 1 -- 172.21.15.64:0/853478242 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ba3693300 msgr2=0x564ba369fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:15:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.860+0000 7f377f1bcb80 1 --2- 172.21.15.64:0/853478242 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ba3693300 0x564ba369fdc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3760099610 tx=0x7f376000cc90 comp rx=0 tx=0).stop 2023-09-13T03:15:24.877 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.860+0000 7f377f1bcb80 1 -- 172.21.15.64:0/853478242 shutdown_connections 2023-09-13T03:15:24.877 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.860+0000 7f377f1bcb80 1 --2- 172.21.15.64:0/853478242 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ba3693300 0x564ba369fdc0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:24.877 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.860+0000 7f377f1bcb80 1 --2- 172.21.15.64:0/853478242 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x564ba36acfa0 0x564ba36af3c0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:24.877 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.860+0000 7f377f1bcb80 1 --2- 172.21.15.64:0/853478242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ba36991a0 0x564ba369d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:24.877 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.860+0000 7f377f1bcb80 1 --2- 172.21.15.64:0/853478242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ba3693cc0 0x564ba369d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:24.878 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.860+0000 7f377f1bcb80 1 -- 172.21.15.64:0/853478242 >> 172.21.15.64:0/853478242 conn(0x564ba3608030 msgr2=0x564ba368d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:15:24.878 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.860+0000 7f374cff9700 0 client.26303 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:15:24.878 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.860+0000 7f377f1bcb80 1 -- 172.21.15.64:0/853478242 shutdown_connections 2023-09-13T03:15:24.878 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.860+0000 7f377f1bcb80 1 -- 172.21.15.64:0/853478242 wait complete. 2023-09-13T03:15:24.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.861+0000 7f377f1bcb80 20 client.26303 trim_cache size 0 max 16384 2023-09-13T03:15:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.876+0000 7f377f1bcb80 1 Processor -- start 2023-09-13T03:15:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.877+0000 7f377f1bcb80 1 -- start start 2023-09-13T03:15:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.877+0000 7f377f1bcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ba3693300 0x564ba36a4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.877+0000 7f377f1bcb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ba3693cc0 0x564ba369f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.877+0000 7f377f1bcb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ba36991a0 0x564ba3697230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.877+0000 7f377f1bcb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ba368fe50 con 0x564ba36991a0 2023-09-13T03:15:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.877+0000 7f377f1bcb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ba36a4b10 con 0x564ba3693cc0 2023-09-13T03:15:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.877+0000 7f377f1bcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ba3698e30 con 0x564ba3693300 2023-09-13T03:15:24.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.877+0000 7f376959b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ba3693300 0x564ba36a4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:24.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.877+0000 7f376959b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ba3693300 0x564ba36a4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:38970/0 (socket says 172.21.15.64:38970) 2023-09-13T03:15:24.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.877+0000 7f376959b700 1 -- 172.21.15.64:0/3267718269 learned_addr learned my addr 172.21.15.64:0/3267718269 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:15:24.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.877+0000 7f3768d9a700 1 --2- 172.21.15.64:0/3267718269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ba3693cc0 0x564ba369f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:24.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.877+0000 7f3769d9c700 1 --2- 172.21.15.64:0/3267718269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ba36991a0 0x564ba3697230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:24.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.878+0000 7f3768d9a700 1 -- 172.21.15.64:0/3267718269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ba3693300 msgr2=0x564ba36a4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:15:24.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.878+0000 7f3768d9a700 1 --2- 172.21.15.64:0/3267718269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ba3693300 0x564ba36a4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:24.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.878+0000 7f3768d9a700 1 -- 172.21.15.64:0/3267718269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ba36991a0 msgr2=0x564ba3697230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:15:24.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.878+0000 7f3768d9a700 1 --2- 172.21.15.64:0/3267718269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ba36991a0 0x564ba3697230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.878+0000 7f3768d9a700 1 -- 172.21.15.64:0/3267718269 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ba33fac00 con 0x564ba3693cc0 2023-09-13T03:15:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.878+0000 7f376959b700 1 --2- 172.21.15.64:0/3267718269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ba3693300 0x564ba36a4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:15:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.878+0000 7f3769d9c700 1 --2- 172.21.15.64:0/3267718269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ba36991a0 0x564ba3697230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:15:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.878+0000 7f3768d9a700 1 --2- 172.21.15.64:0/3267718269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ba3693cc0 0x564ba369f8f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x564ba36afc70 tx=0x7f3754009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94bc8178f4c0bc16 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:15:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.878+0000 7f374ffff700 1 -- 172.21.15.64:0/3267718269 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3754017070 con 0x564ba3693cc0 2023-09-13T03:15:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.878+0000 7f374ffff700 1 -- 172.21.15.64:0/3267718269 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3754018040 con 0x564ba3693cc0 2023-09-13T03:15:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.878+0000 7f374ffff700 1 -- 172.21.15.64:0/3267718269 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3754010430 con 0x564ba3693cc0 2023-09-13T03:15:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.879+0000 7f377f1bcb80 1 -- 172.21.15.64:0/3267718269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ba3693cc0 msgr2=0x564ba369f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:15:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.879+0000 7f377f1bcb80 1 --2- 172.21.15.64:0/3267718269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ba3693cc0 0x564ba369f8f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x564ba36afc70 tx=0x7f3754009260 comp rx=0 tx=0).stop 2023-09-13T03:15:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.879+0000 7f377f1bcb80 1 -- 172.21.15.64:0/3267718269 shutdown_connections 2023-09-13T03:15:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.879+0000 7f377f1bcb80 1 --2- 172.21.15.64:0/3267718269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ba36991a0 0x564ba3697230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.879+0000 7f377f1bcb80 1 --2- 172.21.15.64:0/3267718269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ba3693300 0x564ba36a4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.879+0000 7f377f1bcb80 1 --2- 172.21.15.64:0/3267718269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ba3693cc0 0x564ba369f8f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.879+0000 7f377f1bcb80 1 -- 172.21.15.64:0/3267718269 >> 172.21.15.64:0/3267718269 conn(0x564ba3608030 msgr2=0x564ba36981b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:15:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.879+0000 7f377f1bcb80 1 -- 172.21.15.64:0/3267718269 shutdown_connections 2023-09-13T03:15:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.879+0000 7f377f1bcb80 1 -- 172.21.15.64:0/3267718269 wait complete. 2023-09-13T03:15:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.880+0000 7f377f1bcb80 1 -- start start 2023-09-13T03:15:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.880+0000 7f377f1bcb80 1 Processor -- start 2023-09-13T03:15:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.880+0000 7f377f1bcb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ba3693300 0x564ba36f4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:24.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.880+0000 7f377f1bcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ba3693cc0 0x564ba36f5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:24.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.880+0000 7f377f1bcb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ba36991a0 0x564ba36ffde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:24.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.880+0000 7f377f1bcb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ba3700320 con 0x564ba3693300 2023-09-13T03:15:24.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.880+0000 7f377f1bcb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ba3700460 con 0x564ba36991a0 2023-09-13T03:15:24.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.880+0000 7f377f1bcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ba37005a0 con 0x564ba3693cc0 2023-09-13T03:15:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.880+0000 7f377f1bcb80 10 client.0 _ll_register_callbacks cb 0x7f3754009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:15:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.881+0000 7f376959b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ba3693300 0x564ba36f4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.881+0000 7f3769d9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ba36991a0 0x564ba36ffde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.881+0000 7f3768d9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ba3693cc0 0x564ba36f5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.881+0000 7f3769d9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ba36991a0 0x564ba36ffde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:53292/0 (socket says 172.21.15.64:53292) 2023-09-13T03:15:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.881+0000 7f3769d9c700 1 -- 172.21.15.64:0/4250150638 learned_addr learned my addr 172.21.15.64:0/4250150638 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:15:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.881+0000 7f3768d9a700 1 -- 172.21.15.64:0/4250150638 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ba36991a0 msgr2=0x564ba36ffde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:15:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.881+0000 7f3768d9a700 1 --2- 172.21.15.64:0/4250150638 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ba36991a0 0x564ba36ffde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.881+0000 7f3768d9a700 1 -- 172.21.15.64:0/4250150638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ba3693300 msgr2=0x564ba36f4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:15:24.894 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.881+0000 7f3768d9a700 1 --2- 172.21.15.64:0/4250150638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ba3693300 0x564ba36f4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:24.894 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.881+0000 7f3768d9a700 1 -- 172.21.15.64:0/4250150638 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37641a5780 con 0x564ba3693cc0 2023-09-13T03:15:24.894 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.881+0000 7f376959b700 1 --2- 172.21.15.64:0/4250150638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ba3693300 0x564ba36f4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:15:24.894 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.881+0000 7f3768d9a700 1 --2- 172.21.15.64:0/4250150638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ba3693cc0 0x564ba36f5490 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x564ba36aaad0 tx=0x7f375401f610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c10561a08c86b060 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:15:24.894 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.881+0000 7f374cff9700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:15:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.882+0000 7f374cff9700 1 -- 172.21.15.64:0/4250150638 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3754017040 con 0x564ba3693cc0 2023-09-13T03:15:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.882+0000 7f377f1bcb80 1 -- 172.21.15.64:0/4250150638 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f376009d740 con 0x564ba3693cc0 2023-09-13T03:15:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.882+0000 7f374cff9700 1 -- 172.21.15.64:0/4250150638 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f3754003ae0 con 0x564ba3693cc0 2023-09-13T03:15:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.882+0000 7f374cff9700 1 -- 172.21.15.64:0/4250150638 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3754003ca0 con 0x564ba3693cc0 2023-09-13T03:15:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.882+0000 7f377f1bcb80 10 client.26308 fetch_fsmap learned FSMap version 34 2023-09-13T03:15:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.882+0000 7f377f1bcb80 1 -- 172.21.15.64:0/4250150638 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7f375c028040 con 0x564ba3693cc0 2023-09-13T03:15:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.882+0000 7f374cff9700 1 -- 172.21.15.64:0/4250150638 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f376009d740 con 0x564ba3693cc0 2023-09-13T03:15:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.882+0000 7f374cff9700 1 -- 172.21.15.64:0/4250150638 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 34) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f3754026070 con 0x564ba3693cc0 2023-09-13T03:15:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.882+0000 7f377f1bcb80 10 client.26308 fetch_fsmap finished waiting for FSMap version 34 2023-09-13T03:15:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.882+0000 7f377f1bcb80 10 client.26308 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:15:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.882+0000 7f377f1bcb80 20 client.26308 populate_metadata read hostname 'smithi064' 2023-09-13T03:15:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.882+0000 7f377f1bcb80 1 --2- 172.21.15.64:0/4250150638 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x564ba3701d60 0x564ba3704180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.883+0000 7f377f1bcb80 4 client.26308 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:15:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.883+0000 7f377f1bcb80 1 -- 172.21.15.64:0/4250150638 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564ba35de8c0 con 0x564ba3701d60 2023-09-13T03:15:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.883+0000 7f376959b700 1 --2- 172.21.15.64:0/4250150638 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x564ba3701d60 0x564ba3704180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:24.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.883+0000 7f376959b700 1 --2- 172.21.15.64:0/4250150638 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x564ba3701d60 0x564ba3704180 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc7c9cef60574df9 server_cookie=ad20874530a227b8 in_seq=0 out_seq=0 2023-09-13T03:15:24.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.883+0000 7f374cff9700 10 client.26308 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:15:24.898 INFO:tasks.workunit.client.0.smithi064.stdout: 1 46508 3.03 MB/sec warmup 89 sec latency 22.749 ms 2023-09-13T03:15:24.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.906+0000 7f374cff9700 1 -- 172.21.15.64:0/4250150638 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564ba35de8c0 con 0x564ba3701d60 2023-09-13T03:15:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.906+0000 7f374cff9700 10 client.26308 handle_command_reply: tid=0 2023-09-13T03:15:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.906+0000 7f377f1bcb80 1 client.26308 shutdown 2023-09-13T03:15:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.906+0000 7f377f1bcb80 1 -- 172.21.15.64:0/4250150638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ba3693cc0 msgr2=0x564ba36f5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:15:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.906+0000 7f377f1bcb80 1 --2- 172.21.15.64:0/4250150638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ba3693cc0 0x564ba36f5490 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x564ba36aaad0 tx=0x7f375401f610 comp rx=0 tx=0).stop 2023-09-13T03:15:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.907+0000 7f377f1bcb80 1 -- 172.21.15.64:0/4250150638 shutdown_connections 2023-09-13T03:15:24.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.907+0000 7f377f1bcb80 1 --2- 172.21.15.64:0/4250150638 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ba36991a0 0x564ba36ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:24.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.907+0000 7f377f1bcb80 1 --2- 172.21.15.64:0/4250150638 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x564ba3701d60 0x564ba3704180 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:24.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.907+0000 7f377f1bcb80 1 --2- 172.21.15.64:0/4250150638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ba3693300 0x564ba36f4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:24.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.907+0000 7f377f1bcb80 1 --2- 172.21.15.64:0/4250150638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ba3693cc0 0x564ba36f5490 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:24.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.907+0000 7f374cff9700 0 client.26308 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:15:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.907+0000 7f377f1bcb80 1 -- 172.21.15.64:0/4250150638 >> 172.21.15.64:0/4250150638 conn(0x564ba3608030 msgr2=0x564ba36981b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:15:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.907+0000 7f377f1bcb80 1 -- 172.21.15.64:0/4250150638 shutdown_connections 2023-09-13T03:15:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.907+0000 7f377f1bcb80 1 -- 172.21.15.64:0/4250150638 wait complete. 2023-09-13T03:15:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.907+0000 7f377f1bcb80 20 client.26308 trim_cache size 0 max 16384 2023-09-13T03:15:24.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:15:24.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:15:24.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:15:24.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:15:24.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.908+0000 7f376b7ff700 1 -- 172.21.15.64:0/1164077434 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f37500a07b0 msgr2=0x7f37500a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:15:24.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.908+0000 7f376b7ff700 1 --2- 172.21.15.64:0/1164077434 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f37500a07b0 0x7f37500a2c70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f375c009cd0 tx=0x7f375c00b040 comp rx=0 tx=0).stop 2023-09-13T03:15:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.908+0000 7f376b7ff700 1 -- 172.21.15.64:0/1164077434 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37641005b0 msgr2=0x7f376419e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:15:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.908+0000 7f376b7ff700 1 --2- 172.21.15.64:0/1164077434 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37641005b0 0x7f376419e970 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f376000eb30 tx=0x7f376000ee40 comp rx=0 tx=0).stop 2023-09-13T03:15:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.908+0000 7f376b7ff700 1 -- 172.21.15.64:0/1164077434 shutdown_connections 2023-09-13T03:15:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.908+0000 7f376b7ff700 1 --2- 172.21.15.64:0/1164077434 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f37500a07b0 0x7f37500a2c70 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.908+0000 7f376b7ff700 1 --2- 172.21.15.64:0/1164077434 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37641005b0 0x7f376419e970 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.908+0000 7f376b7ff700 1 --2- 172.21.15.64:0/1164077434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3764106670 0x7f37641987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.908+0000 7f376b7ff700 1 --2- 172.21.15.64:0/1164077434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3764100f70 0x7f376419eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.908+0000 7f376b7ff700 1 -- 172.21.15.64:0/1164077434 >> 172.21.15.64:0/1164077434 conn(0x7f37640fc060 msgr2=0x7f376410b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:15:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.908+0000 7f376b7ff700 1 -- 172.21.15.64:0/1164077434 shutdown_connections 2023-09-13T03:15:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:24.908+0000 7f376b7ff700 1 -- 172.21.15.64:0/1164077434 wait complete. 2023-09-13T03:15:24.915 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:15:24 smithi170 ceph-mon[134328]: pgmap v70: 97 pgs: 97 active+clean; 33 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 1.3 MiB/s wr, 71 op/s 2023-09-13T03:15:24.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:15:24 smithi064 ceph-mon[97675]: pgmap v70: 97 pgs: 97 active+clean; 33 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 1.3 MiB/s wr, 71 op/s 2023-09-13T03:15:24.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:15:24 smithi142 ceph-mon[134082]: pgmap v70: 97 pgs: 97 active+clean; 33 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 1.3 MiB/s wr, 71 op/s 2023-09-13T03:15:25.895 INFO:tasks.workunit.client.0.smithi064.stdout: 1 46814 3.00 MB/sec warmup 90 sec latency 624.494 ms 2023-09-13T03:15:26.895 INFO:tasks.workunit.client.0.smithi064.stdout: 1 46814 2.96 MB/sec warmup 91 sec latency 1624.610 ms 2023-09-13T03:15:26.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:15:26 smithi170 ceph-mon[134328]: pgmap v71: 97 pgs: 97 active+clean; 32 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 1.5 MiB/s wr, 119 op/s 2023-09-13T03:15:26.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:15:26 smithi064 ceph-mon[97675]: pgmap v71: 97 pgs: 97 active+clean; 32 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 1.5 MiB/s wr, 119 op/s 2023-09-13T03:15:26.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:15:26 smithi142 ceph-mon[134082]: pgmap v71: 97 pgs: 97 active+clean; 32 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 1.5 MiB/s wr, 119 op/s 2023-09-13T03:15:27.895 INFO:tasks.workunit.client.0.smithi064.stdout: 1 46814 2.93 MB/sec warmup 92 sec latency 2624.726 ms 2023-09-13T03:15:28.895 INFO:tasks.workunit.client.0.smithi064.stdout: 1 46938 2.90 MB/sec warmup 93 sec latency 3251.138 ms 2023-09-13T03:15:28.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:15:28 smithi170 ceph-mon[134328]: pgmap v72: 97 pgs: 97 active+clean; 32 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 1.5 MiB/s wr, 97 op/s 2023-09-13T03:15:28.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:15:28 smithi064 ceph-mon[97675]: pgmap v72: 97 pgs: 97 active+clean; 32 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 1.5 MiB/s wr, 97 op/s 2023-09-13T03:15:28.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:15:28 smithi142 ceph-mon[134082]: pgmap v72: 97 pgs: 97 active+clean; 32 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 1.5 MiB/s wr, 97 op/s 2023-09-13T03:15:29.895 INFO:tasks.workunit.client.0.smithi064.stdout: 1 46938 2.87 MB/sec warmup 94 sec latency 1269.307 ms 2023-09-13T03:15:30.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:15:30 smithi170 ceph-mon[134328]: pgmap v73: 97 pgs: 97 active+clean; 32 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 1.5 MiB/s wr, 97 op/s 2023-09-13T03:15:30.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:15:30 smithi064 ceph-mon[97675]: pgmap v73: 97 pgs: 97 active+clean; 32 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 1.5 MiB/s wr, 97 op/s 2023-09-13T03:15:30.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:15:30 smithi142 ceph-mon[134082]: pgmap v73: 97 pgs: 97 active+clean; 32 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 1.5 MiB/s wr, 97 op/s 2023-09-13T03:15:30.895 INFO:tasks.workunit.client.0.smithi064.stdout: 1 46938 2.84 MB/sec warmup 95 sec latency 2269.433 ms 2023-09-13T03:15:31.895 INFO:tasks.workunit.client.0.smithi064.stdout: 1 46938 2.81 MB/sec warmup 96 sec latency 3269.572 ms 2023-09-13T03:15:32.895 INFO:tasks.workunit.client.0.smithi064.stdout: 1 46938 2.78 MB/sec warmup 97 sec latency 4269.701 ms 2023-09-13T03:15:32.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:15:32 smithi170 ceph-mon[134328]: pgmap v74: 97 pgs: 97 active+clean; 32 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 1.5 MiB/s wr, 108 op/s 2023-09-13T03:15:32.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:15:32 smithi064 ceph-mon[97675]: pgmap v74: 97 pgs: 97 active+clean; 32 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 1.5 MiB/s wr, 108 op/s 2023-09-13T03:15:32.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:15:32 smithi142 ceph-mon[134082]: pgmap v74: 97 pgs: 97 active+clean; 32 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 1.5 MiB/s wr, 108 op/s 2023-09-13T03:15:33.895 INFO:tasks.workunit.client.0.smithi064.stdout: 1 47706 2.80 MB/sec warmup 98 sec latency 4897.251 ms 2023-09-13T03:15:34.895 INFO:tasks.workunit.client.0.smithi064.stdout: 1 49979 2.91 MB/sec warmup 99 sec latency 12.244 ms 2023-09-13T03:15:34.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:15:34 smithi170 ceph-mon[134328]: pgmap v75: 97 pgs: 97 active+clean; 32 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 223 KiB/s wr, 58 op/s 2023-09-13T03:15:34.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:15:34 smithi064 ceph-mon[97675]: pgmap v75: 97 pgs: 97 active+clean; 32 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 223 KiB/s wr, 58 op/s 2023-09-13T03:15:34.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:15:34 smithi142 ceph-mon[134082]: pgmap v75: 97 pgs: 97 active+clean; 32 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 223 KiB/s wr, 58 op/s 2023-09-13T03:15:35.896 INFO:tasks.workunit.client.0.smithi064.stdout: 1 50355 2.89 MB/sec warmup 100 sec latency 567.329 ms 2023-09-13T03:15:36.896 INFO:tasks.workunit.client.0.smithi064.stdout: 1 50355 2.86 MB/sec warmup 101 sec latency 1567.443 ms 2023-09-13T03:15:36.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:15:36 smithi170 ceph-mon[134328]: pgmap v76: 97 pgs: 97 active+clean; 34 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 423 KiB/s wr, 96 op/s 2023-09-13T03:15:36.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:15:36 smithi064 ceph-mon[97675]: pgmap v76: 97 pgs: 97 active+clean; 34 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 423 KiB/s wr, 96 op/s 2023-09-13T03:15:36.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:15:36 smithi142 ceph-mon[134082]: pgmap v76: 97 pgs: 97 active+clean; 34 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 423 KiB/s wr, 96 op/s 2023-09-13T03:15:37.896 INFO:tasks.workunit.client.0.smithi064.stdout: 1 50355 2.83 MB/sec warmup 102 sec latency 2567.571 ms 2023-09-13T03:15:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.069+0000 7f43effff700 1 -- 172.21.15.64:0/1267735986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43e8072a30 msgr2=0x7f43e8072e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:15:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.069+0000 7f43effff700 1 --2- 172.21.15.64:0/1267735986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43e8072a30 0x7f43e8072e10 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f43e4009990 tx=0x7f43e4009ca0 comp rx=0 tx=0).stop 2023-09-13T03:15:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.069+0000 7f43effff700 1 -- 172.21.15.64:0/1267735986 shutdown_connections 2023-09-13T03:15:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.069+0000 7f43effff700 1 --2- 172.21.15.64:0/1267735986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43e810cdc0 0x7f43e810f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.069+0000 7f43effff700 1 --2- 172.21.15.64:0/1267735986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43e8072a30 0x7f43e8072e10 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.069+0000 7f43effff700 1 --2- 172.21.15.64:0/1267735986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43e8073350 0x7f43e810c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.069+0000 7f43effff700 1 -- 172.21.15.64:0/1267735986 >> 172.21.15.64:0/1267735986 conn(0x7f43e80fc050 msgr2=0x7f43e80fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:15:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.069+0000 7f43effff700 1 -- 172.21.15.64:0/1267735986 shutdown_connections 2023-09-13T03:15:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.070+0000 7f43effff700 1 -- 172.21.15.64:0/1267735986 wait complete. 2023-09-13T03:15:38.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.070+0000 7f43effff700 1 Processor -- start 2023-09-13T03:15:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.071+0000 7f43effff700 1 -- start start 2023-09-13T03:15:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.071+0000 7f43effff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43e8072a30 0x7f43e819e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.071+0000 7f43effff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43e8073350 0x7f43e819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.071+0000 7f43effff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43e810cdc0 0x7f43e81987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:38.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.071+0000 7f43effff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e8198e50 con 0x7f43e8072a30 2023-09-13T03:15:38.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.071+0000 7f43effff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e8198fc0 con 0x7f43e8073350 2023-09-13T03:15:38.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.071+0000 7f43effff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e8199180 con 0x7f43e810cdc0 2023-09-13T03:15:38.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.071+0000 7f43edd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43e8072a30 0x7f43e819e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:38.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.071+0000 7f43ed59a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43e8073350 0x7f43e819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.071+0000 7f43edd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43e8072a30 0x7f43e819e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39262/0 (socket says 172.21.15.64:39262) 2023-09-13T03:15:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.071+0000 7f43edd9b700 1 -- 172.21.15.64:0/1248318550 learned_addr learned my addr 172.21.15.64:0/1248318550 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:15:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.071+0000 7f43ee59c700 1 --2- 172.21.15.64:0/1248318550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43e810cdc0 0x7f43e81987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.072+0000 7f43edd9b700 1 -- 172.21.15.64:0/1248318550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43e810cdc0 msgr2=0x7f43e81987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:15:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.072+0000 7f43edd9b700 1 --2- 172.21.15.64:0/1248318550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43e810cdc0 0x7f43e81987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:38.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.072+0000 7f43edd9b700 1 -- 172.21.15.64:0/1248318550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43e8073350 msgr2=0x7f43e819eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:15:38.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.072+0000 7f43edd9b700 1 --2- 172.21.15.64:0/1248318550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43e8073350 0x7f43e819eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:38.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.072+0000 7f43edd9b700 1 -- 172.21.15.64:0/1248318550 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e40095f0 con 0x7f43e8072a30 2023-09-13T03:15:38.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.072+0000 7f43ed59a700 1 --2- 172.21.15.64:0/1248318550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43e8073350 0x7f43e819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:15:38.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.072+0000 7f43edd9b700 1 --2- 172.21.15.64:0/1248318550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43e8072a30 0x7f43e819e980 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f43e4000c00 tx=0x7f43e400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:15:38.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.072+0000 7f43deffd700 1 -- 172.21.15.64:0/1248318550 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e401d070 con 0x7f43e8072a30 2023-09-13T03:15:38.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.072+0000 7f43deffd700 1 -- 172.21.15.64:0/1248318550 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43e400be40 con 0x7f43e8072a30 2023-09-13T03:15:38.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.072+0000 7f43effff700 1 -- 172.21.15.64:0/1248318550 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43e8199400 con 0x7f43e8072a30 2023-09-13T03:15:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.072+0000 7f43deffd700 1 -- 172.21.15.64:0/1248318550 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e40173f0 con 0x7f43e8072a30 2023-09-13T03:15:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.073+0000 7f43effff700 1 -- 172.21.15.64:0/1248318550 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e81a5970 con 0x7f43e8072a30 2023-09-13T03:15:38.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.076+0000 7f43deffd700 1 -- 172.21.15.64:0/1248318550 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f43e4022470 con 0x7f43e8072a30 2023-09-13T03:15:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.077+0000 7f43deffd700 1 --2- 172.21.15.64:0/1248318550 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f43d40a07b0 0x7f43d40a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.077+0000 7f43deffd700 1 -- 172.21.15.64:0/1248318550 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f43e40cd250 con 0x7f43e8072a30 2023-09-13T03:15:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.077+0000 7f4403984b80 1 Processor -- start 2023-09-13T03:15:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.077+0000 7f43ed59a700 1 --2- 172.21.15.64:0/1248318550 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f43d40a07b0 0x7f43d40a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:38.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.078+0000 7f4403984b80 1 -- start start 2023-09-13T03:15:38.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.078+0000 7f4403984b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619f0bcf1a0 0x5619f0bcf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:38.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.078+0000 7f43ed59a700 1 --2- 172.21.15.64:0/1248318550 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f43d40a07b0 0x7f43d40a2c70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f43e810e370 tx=0x7f43d8009260 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:15:38.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.078+0000 7f4403984b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619f0bc9300 0x5619f0bc9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:38.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.078+0000 7f4403984b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5619f0bc9cc0 0x5619f0bd5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:38.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.078+0000 7f4403984b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f0b226d0 con 0x5619f0bc9cc0 2023-09-13T03:15:38.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.078+0000 7f4403984b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f0b1fff0 con 0x5619f0bc9300 2023-09-13T03:15:38.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.078+0000 7f4403984b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f0b1b8e0 con 0x5619f0bcf1a0 2023-09-13T03:15:38.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.078+0000 7f43edd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619f0bcf1a0 0x5619f0bcf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:38.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.078+0000 7f43ee59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5619f0bc9cc0 0x5619f0bd5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.078+0000 7f43edd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619f0bcf1a0 0x5619f0bcf580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:57820/0 (socket says 172.21.15.64:57820) 2023-09-13T03:15:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.078+0000 7f43edd9b700 1 -- 172.21.15.64:0/2246756983 learned_addr learned my addr 172.21.15.64:0/2246756983 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:15:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.078+0000 7f43ed59a700 1 --2- 172.21.15.64:0/2246756983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619f0bc9300 0x5619f0bc9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.079+0000 7f43edd9b700 1 -- 172.21.15.64:0/2246756983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619f0bc9300 msgr2=0x5619f0bc9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:15:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.079+0000 7f43edd9b700 1 --2- 172.21.15.64:0/2246756983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619f0bc9300 0x5619f0bc9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.079+0000 7f43edd9b700 1 -- 172.21.15.64:0/2246756983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5619f0bc9cc0 msgr2=0x5619f0bd5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:15:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.079+0000 7f43edd9b700 1 --2- 172.21.15.64:0/2246756983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5619f0bc9cc0 0x5619f0bd5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.079+0000 7f43edd9b700 1 -- 172.21.15.64:0/2246756983 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619f0947d80 con 0x5619f0bcf1a0 2023-09-13T03:15:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.079+0000 7f43ed59a700 1 --2- 172.21.15.64:0/2246756983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619f0bc9300 0x5619f0bc9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:15:38.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.079+0000 7f43edd9b700 1 --2- 172.21.15.64:0/2246756983 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619f0bcf1a0 0x5619f0bcf580 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f43e400b2e0 tx=0x7f43e4096bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fba4973076f559c7 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:15:38.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.079+0000 7f43d3fff700 1 -- 172.21.15.64:0/2246756983 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e409ae80 con 0x5619f0bcf1a0 2023-09-13T03:15:38.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.079+0000 7f43d3fff700 1 -- 172.21.15.64:0/2246756983 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43e409a780 con 0x5619f0bcf1a0 2023-09-13T03:15:38.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.079+0000 7f43d3fff700 1 -- 172.21.15.64:0/2246756983 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e409a9e0 con 0x5619f0bcf1a0 2023-09-13T03:15:38.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.080+0000 7f4403984b80 1 -- 172.21.15.64:0/2246756983 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619f0bcf1a0 msgr2=0x5619f0bcf580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:15:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.080+0000 7f4403984b80 1 --2- 172.21.15.64:0/2246756983 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619f0bcf1a0 0x5619f0bcf580 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f43e400b2e0 tx=0x7f43e4096bd0 comp rx=0 tx=0).stop 2023-09-13T03:15:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.080+0000 7f4403984b80 1 -- 172.21.15.64:0/2246756983 shutdown_connections 2023-09-13T03:15:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.080+0000 7f4403984b80 1 --2- 172.21.15.64:0/2246756983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5619f0bc9cc0 0x5619f0bd5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.080+0000 7f4403984b80 1 --2- 172.21.15.64:0/2246756983 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619f0bcf1a0 0x5619f0bcf580 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.080+0000 7f4403984b80 1 --2- 172.21.15.64:0/2246756983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619f0bc9300 0x5619f0bc9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:38.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.080+0000 7f4403984b80 1 -- 172.21.15.64:0/2246756983 >> 172.21.15.64:0/2246756983 conn(0x5619f0b3e030 msgr2=0x5619f0bc4560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:15:38.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.080+0000 7f4403984b80 1 -- 172.21.15.64:0/2246756983 shutdown_connections 2023-09-13T03:15:38.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.080+0000 7f4403984b80 1 -- 172.21.15.64:0/2246756983 wait complete. 2023-09-13T03:15:38.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.081+0000 7f4403984b80 1 -- start start 2023-09-13T03:15:38.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.081+0000 7f4403984b80 1 Processor -- start 2023-09-13T03:15:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.082+0000 7f4403984b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619f0bc9300 0x5619f0bd8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.082+0000 7f4403984b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619f0bc9cc0 0x5619f0bd9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.082+0000 7f4403984b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5619f0bcf1a0 0x5619f0bd2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.082+0000 7f4403984b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f0ab9b10 con 0x5619f0bcf1a0 2023-09-13T03:15:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.082+0000 7f4403984b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f0ab9810 con 0x5619f0bc9cc0 2023-09-13T03:15:38.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.082+0000 7f4403984b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f099d8c0 con 0x5619f0bc9300 2023-09-13T03:15:38.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.082+0000 7f4403984b80 10 client.0 _ll_register_callbacks cb 0x5619f0a03a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:15:38.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.082+0000 7f43edd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619f0bc9300 0x5619f0bd8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:38.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.082+0000 7f43ee59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5619f0bcf1a0 0x5619f0bd2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:38.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.082+0000 7f43edd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619f0bc9300 0x5619f0bd8b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:57832/0 (socket says 172.21.15.64:57832) 2023-09-13T03:15:38.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.082+0000 7f43edd9b700 1 -- 172.21.15.64:0/3221276720 learned_addr learned my addr 172.21.15.64:0/3221276720 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:15:38.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.082+0000 7f43ed59a700 1 --2- 172.21.15.64:0/3221276720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619f0bc9cc0 0x5619f0bd9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:38.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.083+0000 7f43edd9b700 1 -- 172.21.15.64:0/3221276720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619f0bc9cc0 msgr2=0x5619f0bd9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:15:38.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.083+0000 7f43edd9b700 1 --2- 172.21.15.64:0/3221276720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619f0bc9cc0 0x5619f0bd9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:38.096 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.083+0000 7f43edd9b700 1 -- 172.21.15.64:0/3221276720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5619f0bcf1a0 msgr2=0x5619f0bd2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:15:38.096 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.083+0000 7f43edd9b700 1 --2- 172.21.15.64:0/3221276720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5619f0bcf1a0 0x5619f0bd2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:38.096 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.083+0000 7f43edd9b700 1 -- 172.21.15.64:0/3221276720 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e81a5970 con 0x5619f0bc9300 2023-09-13T03:15:38.096 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.083+0000 7f43edd9b700 1 --2- 172.21.15.64:0/3221276720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619f0bc9300 0x5619f0bd8b20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f43e4096c80 tx=0x7f43e4099f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bb799179669f76f server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:15:38.097 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.083+0000 7f43d17fa700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:15:38.097 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.083+0000 7f43d17fa700 1 -- 172.21.15.64:0/3221276720 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e409ae80 con 0x5619f0bc9300 2023-09-13T03:15:38.097 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.083+0000 7f43d17fa700 1 -- 172.21.15.64:0/3221276720 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f43e40b6070 con 0x5619f0bc9300 2023-09-13T03:15:38.097 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.083+0000 7f4403984b80 1 -- 172.21.15.64:0/3221276720 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5619f0a73490 con 0x5619f0bc9300 2023-09-13T03:15:38.097 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.083+0000 7f43d17fa700 1 -- 172.21.15.64:0/3221276720 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43e409e5b0 con 0x5619f0bc9300 2023-09-13T03:15:38.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.083+0000 7f4403984b80 10 client.26320 fetch_fsmap learned FSMap version 34 2023-09-13T03:15:38.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.083+0000 7f4403984b80 1 -- 172.21.15.64:0/3221276720 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5619f0930c00 con 0x5619f0bc9300 2023-09-13T03:15:38.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.084+0000 7f43d17fa700 1 -- 172.21.15.64:0/3221276720 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5619f0a73490 con 0x5619f0bc9300 2023-09-13T03:15:38.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.084+0000 7f43d17fa700 1 -- 172.21.15.64:0/3221276720 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 34) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f43e4022720 con 0x5619f0bc9300 2023-09-13T03:15:38.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.084+0000 7f4403984b80 10 client.26320 fetch_fsmap finished waiting for FSMap version 34 2023-09-13T03:15:38.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.084+0000 7f4403984b80 10 client.26320 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:15:38.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.084+0000 7f4403984b80 20 client.26320 populate_metadata read hostname 'smithi064' 2023-09-13T03:15:38.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.084+0000 7f4403984b80 1 --2- 172.21.15.64:0/3221276720 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5619f0be2c60 0x5619f0be5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:38.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.084+0000 7f4403984b80 4 client.26320 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:15:38.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.084+0000 7f4403984b80 1 -- 172.21.15.64:0/3221276720 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5619f0b30840 con 0x5619f0be2c60 2023-09-13T03:15:38.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.084+0000 7f43ed59a700 1 --2- 172.21.15.64:0/3221276720 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5619f0be2c60 0x5619f0be5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:38.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.085+0000 7f43ed59a700 1 --2- 172.21.15.64:0/3221276720 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5619f0be2c60 0x5619f0be5080 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a10a8763f055bfa3 server_cookie=73778ca00d8644ef in_seq=0 out_seq=0 2023-09-13T03:15:38.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.085+0000 7f43d17fa700 10 client.26320 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:15:38.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.085+0000 7f43d17fa700 1 -- 172.21.15.64:0/3221276720 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5619f0b30840 con 0x5619f0be2c60 2023-09-13T03:15:38.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.085+0000 7f43d17fa700 10 client.26320 handle_command_reply: tid=0 2023-09-13T03:15:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.086+0000 7f4403984b80 1 client.26320 shutdown 2023-09-13T03:15:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.086+0000 7f4403984b80 1 -- 172.21.15.64:0/3221276720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619f0bc9300 msgr2=0x5619f0bd8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:15:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.086+0000 7f4403984b80 1 --2- 172.21.15.64:0/3221276720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619f0bc9300 0x5619f0bd8b20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f43e4096c80 tx=0x7f43e4099f20 comp rx=0 tx=0).stop 2023-09-13T03:15:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.086+0000 7f4403984b80 1 -- 172.21.15.64:0/3221276720 shutdown_connections 2023-09-13T03:15:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.086+0000 7f4403984b80 1 --2- 172.21.15.64:0/3221276720 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5619f0be2c60 0x5619f0be5080 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.086+0000 7f4403984b80 1 --2- 172.21.15.64:0/3221276720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5619f0bcf1a0 0x5619f0bd2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.086+0000 7f4403984b80 1 --2- 172.21.15.64:0/3221276720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619f0bc9300 0x5619f0bd8b20 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.086+0000 7f4403984b80 1 --2- 172.21.15.64:0/3221276720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619f0bc9cc0 0x5619f0bd9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.086+0000 7f43d17fa700 0 client.26320 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:15:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.086+0000 7f4403984b80 1 -- 172.21.15.64:0/3221276720 >> 172.21.15.64:0/3221276720 conn(0x5619f0b3e030 msgr2=0x5619f0bc37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:15:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.086+0000 7f4403984b80 1 -- 172.21.15.64:0/3221276720 shutdown_connections 2023-09-13T03:15:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.086+0000 7f4403984b80 1 -- 172.21.15.64:0/3221276720 wait complete. 2023-09-13T03:15:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.086+0000 7f4403984b80 20 client.26320 trim_cache size 0 max 16384 2023-09-13T03:15:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.102+0000 7f4403984b80 1 Processor -- start 2023-09-13T03:15:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.102+0000 7f4403984b80 1 -- start start 2023-09-13T03:15:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.102+0000 7f4403984b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619f0bc9300 0x5619f0bc4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.102+0000 7f4403984b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619f0bc9cc0 0x5619f0bce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.102+0000 7f4403984b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5619f0bcf1a0 0x5619f0bd2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.102+0000 7f4403984b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f0bc50d0 con 0x5619f0bcf1a0 2023-09-13T03:15:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.102+0000 7f4403984b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f0bc0fc0 con 0x5619f0bc9cc0 2023-09-13T03:15:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.102+0000 7f4403984b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f0bbbe00 con 0x5619f0bc9300 2023-09-13T03:15:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.102+0000 7f43ee59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5619f0bcf1a0 0x5619f0bd2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.102+0000 7f43ee59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5619f0bcf1a0 0x5619f0bd2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39286/0 (socket says 172.21.15.64:39286) 2023-09-13T03:15:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.102+0000 7f43ee59c700 1 -- 172.21.15.64:0/3351964293 learned_addr learned my addr 172.21.15.64:0/3351964293 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:15:38.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.102+0000 7f43edd9b700 1 --2- 172.21.15.64:0/3351964293 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619f0bc9300 0x5619f0bc4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:38.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.102+0000 7f43ed59a700 1 --2- 172.21.15.64:0/3351964293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619f0bc9cc0 0x5619f0bce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:38.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.103+0000 7f43edd9b700 1 -- 172.21.15.64:0/3351964293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619f0bc9cc0 msgr2=0x5619f0bce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:15:38.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.103+0000 7f43edd9b700 1 --2- 172.21.15.64:0/3351964293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619f0bc9cc0 0x5619f0bce880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:38.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.103+0000 7f43edd9b700 1 -- 172.21.15.64:0/3351964293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5619f0bcf1a0 msgr2=0x5619f0bd2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:15:38.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.103+0000 7f43edd9b700 1 --2- 172.21.15.64:0/3351964293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5619f0bcf1a0 0x5619f0bd2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:38.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.103+0000 7f43edd9b700 1 -- 172.21.15.64:0/3351964293 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619f096e1f0 con 0x5619f0bc9300 2023-09-13T03:15:38.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.103+0000 7f43ee59c700 1 --2- 172.21.15.64:0/3351964293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5619f0bcf1a0 0x5619f0bd2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:15:38.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.103+0000 7f43edd9b700 1 --2- 172.21.15.64:0/3351964293 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619f0bc9300 0x5619f0bc4710 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f43e409a710 tx=0x7f43e4096bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4df3f8f2e0eba5cd server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:15:38.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.103+0000 7f43dcff9700 1 -- 172.21.15.64:0/3351964293 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e409ae80 con 0x5619f0bc9300 2023-09-13T03:15:38.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.103+0000 7f43dcff9700 1 -- 172.21.15.64:0/3351964293 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43e40cd040 con 0x5619f0bc9300 2023-09-13T03:15:38.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.103+0000 7f43dcff9700 1 -- 172.21.15.64:0/3351964293 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e40a2760 con 0x5619f0bc9300 2023-09-13T03:15:38.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.104+0000 7f4403984b80 1 -- 172.21.15.64:0/3351964293 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619f0bc9300 msgr2=0x5619f0bc4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:15:38.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.104+0000 7f4403984b80 1 --2- 172.21.15.64:0/3351964293 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619f0bc9300 0x5619f0bc4710 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f43e409a710 tx=0x7f43e4096bd0 comp rx=0 tx=0).stop 2023-09-13T03:15:38.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.104+0000 7f4403984b80 1 -- 172.21.15.64:0/3351964293 shutdown_connections 2023-09-13T03:15:38.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.104+0000 7f4403984b80 1 --2- 172.21.15.64:0/3351964293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5619f0bcf1a0 0x5619f0bd2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:38.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.104+0000 7f4403984b80 1 --2- 172.21.15.64:0/3351964293 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619f0bc9300 0x5619f0bc4710 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:38.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.104+0000 7f4403984b80 1 --2- 172.21.15.64:0/3351964293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619f0bc9cc0 0x5619f0bce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:38.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.104+0000 7f4403984b80 1 -- 172.21.15.64:0/3351964293 >> 172.21.15.64:0/3351964293 conn(0x5619f0b3e030 msgr2=0x5619f0bd2650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:15:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.104+0000 7f4403984b80 1 -- 172.21.15.64:0/3351964293 shutdown_connections 2023-09-13T03:15:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.104+0000 7f4403984b80 1 -- 172.21.15.64:0/3351964293 wait complete. 2023-09-13T03:15:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.104+0000 7f4403984b80 1 -- start start 2023-09-13T03:15:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.105+0000 7f4403984b80 1 Processor -- start 2023-09-13T03:15:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.105+0000 7f4403984b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619f0bc9300 0x5619f0c301e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.105+0000 7f4403984b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5619f0bc9cc0 0x5619f0c30720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.105+0000 7f4403984b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619f0bcf1a0 0x5619f0c2a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.105+0000 7f4403984b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f0c2afc0 con 0x5619f0bc9cc0 2023-09-13T03:15:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.105+0000 7f4403984b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f0c2b130 con 0x5619f0bcf1a0 2023-09-13T03:15:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.105+0000 7f4403984b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f0c2b2f0 con 0x5619f0bc9300 2023-09-13T03:15:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.105+0000 7f4403984b80 10 client.0 _ll_register_callbacks cb 0x7f43e0009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:15:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.105+0000 7f43ed59a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5619f0bc9cc0 0x5619f0c30720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.105+0000 7f43ed59a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5619f0bc9cc0 0x5619f0c30720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39288/0 (socket says 172.21.15.64:39288) 2023-09-13T03:15:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.105+0000 7f43ed59a700 1 -- 172.21.15.64:0/632082599 learned_addr learned my addr 172.21.15.64:0/632082599 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:15:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.106+0000 7f43edd9b700 1 --2- 172.21.15.64:0/632082599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619f0bc9300 0x5619f0c301e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.106+0000 7f43ee59c700 1 --2- 172.21.15.64:0/632082599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619f0bcf1a0 0x5619f0c2a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.106+0000 7f43ed59a700 1 -- 172.21.15.64:0/632082599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619f0bc9300 msgr2=0x5619f0c301e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:15:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.106+0000 7f43ed59a700 1 --2- 172.21.15.64:0/632082599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619f0bc9300 0x5619f0c301e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.106+0000 7f43ed59a700 1 -- 172.21.15.64:0/632082599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619f0bcf1a0 msgr2=0x5619f0c2a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:15:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.106+0000 7f43ed59a700 1 --2- 172.21.15.64:0/632082599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619f0bcf1a0 0x5619f0c2a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.106+0000 7f43ed59a700 1 -- 172.21.15.64:0/632082599 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e0009c30 con 0x5619f0bc9cc0 2023-09-13T03:15:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.106+0000 7f43ee59c700 1 --2- 172.21.15.64:0/632082599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619f0bcf1a0 0x5619f0c2a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:15:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.106+0000 7f43ed59a700 1 --2- 172.21.15.64:0/632082599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5619f0bc9cc0 0x5619f0c30720 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f43d80094f0 tx=0x7f43d80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd4f1456d0aa43 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:15:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.106+0000 7f43d17fa700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:15:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.106+0000 7f43d17fa700 1 -- 172.21.15.64:0/632082599 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d8024040 con 0x5619f0bc9cc0 2023-09-13T03:15:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.106+0000 7f4403984b80 1 -- 172.21.15.64:0/632082599 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f43e409a780 con 0x5619f0bc9cc0 2023-09-13T03:15:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.106+0000 7f43d17fa700 1 -- 172.21.15.64:0/632082599 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f43d8003680 con 0x5619f0bc9cc0 2023-09-13T03:15:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.107+0000 7f43d17fa700 1 -- 172.21.15.64:0/632082599 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43d8003840 con 0x5619f0bc9cc0 2023-09-13T03:15:38.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.107+0000 7f4403984b80 10 client.16287 fetch_fsmap learned FSMap version 34 2023-09-13T03:15:38.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.107+0000 7f4403984b80 1 -- 172.21.15.64:0/632082599 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5619f0930c00 con 0x5619f0bc9cc0 2023-09-13T03:15:38.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.107+0000 7f43d17fa700 1 -- 172.21.15.64:0/632082599 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43e409a780 con 0x5619f0bc9cc0 2023-09-13T03:15:38.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.107+0000 7f43d17fa700 1 -- 172.21.15.64:0/632082599 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 34) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f43d803cb60 con 0x5619f0bc9cc0 2023-09-13T03:15:38.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.107+0000 7f4403984b80 10 client.16287 fetch_fsmap finished waiting for FSMap version 34 2023-09-13T03:15:38.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.107+0000 7f4403984b80 10 client.16287 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:15:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.107+0000 7f4403984b80 20 client.16287 populate_metadata read hostname 'smithi064' 2023-09-13T03:15:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.107+0000 7f4403984b80 1 --2- 172.21.15.64:0/632082599 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5619f0c381f0 0x5619f0c3a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.107+0000 7f4403984b80 4 client.16287 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:15:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.107+0000 7f4403984b80 1 -- 172.21.15.64:0/632082599 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5619f0b148c0 con 0x5619f0c381f0 2023-09-13T03:15:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.108+0000 7f43edd9b700 1 --2- 172.21.15.64:0/632082599 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5619f0c381f0 0x5619f0c3a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.108+0000 7f43edd9b700 1 --2- 172.21.15.64:0/632082599 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5619f0c381f0 0x5619f0c3a610 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b6aac860320dbb2 server_cookie=423c3e0df92c6496 in_seq=0 out_seq=0 2023-09-13T03:15:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.108+0000 7f43d17fa700 10 client.16287 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:15:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.142+0000 7f43d17fa700 1 -- 172.21.15.64:0/632082599 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5619f0b148c0 con 0x5619f0c381f0 2023-09-13T03:15:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.142+0000 7f43d17fa700 10 client.16287 handle_command_reply: tid=0 2023-09-13T03:15:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.142+0000 7f4403984b80 1 client.16287 shutdown 2023-09-13T03:15:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.142+0000 7f4403984b80 1 -- 172.21.15.64:0/632082599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5619f0bc9cc0 msgr2=0x5619f0c30720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:15:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.142+0000 7f4403984b80 1 --2- 172.21.15.64:0/632082599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5619f0bc9cc0 0x5619f0c30720 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f43d80094f0 tx=0x7f43d80047c0 comp rx=0 tx=0).stop 2023-09-13T03:15:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.142+0000 7f4403984b80 1 -- 172.21.15.64:0/632082599 shutdown_connections 2023-09-13T03:15:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.142+0000 7f4403984b80 1 --2- 172.21.15.64:0/632082599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619f0bcf1a0 0x5619f0c2a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.142+0000 7f4403984b80 1 --2- 172.21.15.64:0/632082599 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5619f0c381f0 0x5619f0c3a610 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.142+0000 7f4403984b80 1 --2- 172.21.15.64:0/632082599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619f0bc9300 0x5619f0c301e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.142+0000 7f4403984b80 1 --2- 172.21.15.64:0/632082599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5619f0bc9cc0 0x5619f0c30720 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.142+0000 7f43d17fa700 0 client.16287 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:15:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.142+0000 7f4403984b80 1 -- 172.21.15.64:0/632082599 >> 172.21.15.64:0/632082599 conn(0x5619f0b3e030 msgr2=0x5619f0bc8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:15:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.142+0000 7f4403984b80 1 -- 172.21.15.64:0/632082599 shutdown_connections 2023-09-13T03:15:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.142+0000 7f4403984b80 1 -- 172.21.15.64:0/632082599 wait complete. 2023-09-13T03:15:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.142+0000 7f4403984b80 20 client.16287 trim_cache size 0 max 16384 2023-09-13T03:15:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:15:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:15:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:15:38.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:15:38.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.143+0000 7f43effff700 1 -- 172.21.15.64:0/1248318550 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f43d40a07b0 msgr2=0x7f43d40a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:15:38.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.143+0000 7f43effff700 1 --2- 172.21.15.64:0/1248318550 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f43d40a07b0 0x7f43d40a2c70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f43e810e370 tx=0x7f43d8009260 comp rx=0 tx=0).stop 2023-09-13T03:15:38.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.143+0000 7f43effff700 1 -- 172.21.15.64:0/1248318550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43e8072a30 msgr2=0x7f43e819e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:15:38.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.143+0000 7f43effff700 1 --2- 172.21.15.64:0/1248318550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43e8072a30 0x7f43e819e980 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f43e4000c00 tx=0x7f43e400f690 comp rx=0 tx=0).stop 2023-09-13T03:15:38.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.143+0000 7f43effff700 1 -- 172.21.15.64:0/1248318550 shutdown_connections 2023-09-13T03:15:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.143+0000 7f43effff700 1 --2- 172.21.15.64:0/1248318550 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f43d40a07b0 0x7f43d40a2c70 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.143+0000 7f43effff700 1 --2- 172.21.15.64:0/1248318550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43e810cdc0 0x7f43e81987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.143+0000 7f43effff700 1 --2- 172.21.15.64:0/1248318550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43e8072a30 0x7f43e819e980 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.143+0000 7f43effff700 1 --2- 172.21.15.64:0/1248318550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43e8073350 0x7f43e819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.143+0000 7f43effff700 1 -- 172.21.15.64:0/1248318550 >> 172.21.15.64:0/1248318550 conn(0x7f43e80fc050 msgr2=0x7f43e810efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:15:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.143+0000 7f43effff700 1 -- 172.21.15.64:0/1248318550 shutdown_connections 2023-09-13T03:15:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:38.143+0000 7f43effff700 1 -- 172.21.15.64:0/1248318550 wait complete. 2023-09-13T03:15:38.896 INFO:tasks.workunit.client.0.smithi064.stdout: 1 50411 2.80 MB/sec warmup 103 sec latency 3196.940 ms 2023-09-13T03:15:38.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:15:38 smithi170 ceph-mon[134328]: pgmap v77: 97 pgs: 97 active+clean; 34 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 240 KiB/s wr, 49 op/s 2023-09-13T03:15:38.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:15:38 smithi064 ceph-mon[97675]: pgmap v77: 97 pgs: 97 active+clean; 34 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 240 KiB/s wr, 49 op/s 2023-09-13T03:15:38.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:15:38 smithi142 ceph-mon[134082]: pgmap v77: 97 pgs: 97 active+clean; 34 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 240 KiB/s wr, 49 op/s 2023-09-13T03:15:39.896 INFO:tasks.workunit.client.0.smithi064.stdout: 1 50411 2.78 MB/sec warmup 104 sec latency 1326.698 ms 2023-09-13T03:15:40.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:15:40 smithi170 ceph-mon[134328]: pgmap v78: 97 pgs: 97 active+clean; 34 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 240 KiB/s wr, 49 op/s 2023-09-13T03:15:40.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:15:40 smithi064 ceph-mon[97675]: pgmap v78: 97 pgs: 97 active+clean; 34 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 240 KiB/s wr, 49 op/s 2023-09-13T03:15:40.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:15:40 smithi142 ceph-mon[134082]: pgmap v78: 97 pgs: 97 active+clean; 34 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 240 KiB/s wr, 49 op/s 2023-09-13T03:15:40.896 INFO:tasks.workunit.client.0.smithi064.stdout: 1 50411 2.75 MB/sec warmup 105 sec latency 2326.854 ms 2023-09-13T03:15:41.896 INFO:tasks.workunit.client.0.smithi064.stdout: 1 50411 2.73 MB/sec warmup 106 sec latency 3327.001 ms 2023-09-13T03:15:42.896 INFO:tasks.workunit.client.0.smithi064.stdout: 1 52139 2.76 MB/sec warmup 107 sec latency 3575.862 ms 2023-09-13T03:15:42.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:15:42 smithi170 ceph-mon[134328]: pgmap v79: 97 pgs: 97 active+clean; 35 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 292 KiB/s wr, 68 op/s 2023-09-13T03:15:42.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:15:42 smithi064 ceph-mon[97675]: pgmap v79: 97 pgs: 97 active+clean; 35 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 292 KiB/s wr, 68 op/s 2023-09-13T03:15:42.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:15:42 smithi142 ceph-mon[134082]: pgmap v79: 97 pgs: 97 active+clean; 35 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 292 KiB/s wr, 68 op/s 2023-09-13T03:15:43.896 INFO:tasks.workunit.client.0.smithi064.stdout: 1 53706 2.85 MB/sec warmup 108 sec latency 12.142 ms 2023-09-13T03:15:44.897 INFO:tasks.workunit.client.0.smithi064.stdout: 1 53903 2.82 MB/sec warmup 109 sec latency 843.785 ms 2023-09-13T03:15:44.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:15:44 smithi170 ceph-mon[134328]: pgmap v80: 97 pgs: 97 active+clean; 35 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 251 KiB/s wr, 56 op/s 2023-09-13T03:15:44.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:15:44 smithi064 ceph-mon[97675]: pgmap v80: 97 pgs: 97 active+clean; 35 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 251 KiB/s wr, 56 op/s 2023-09-13T03:15:44.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:15:44 smithi142 ceph-mon[134082]: pgmap v80: 97 pgs: 97 active+clean; 35 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 251 KiB/s wr, 56 op/s 2023-09-13T03:15:45.897 INFO:tasks.workunit.client.0.smithi064.stdout: 1 53903 2.80 MB/sec warmup 110 sec latency 1843.911 ms 2023-09-13T03:15:46.897 INFO:tasks.workunit.client.0.smithi064.stdout: 1 53903 2.77 MB/sec warmup 111 sec latency 2843.991 ms 2023-09-13T03:15:46.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:15:46 smithi170 ceph-mon[134328]: pgmap v81: 97 pgs: 97 active+clean; 36 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 1.0 MiB/s wr, 121 op/s 2023-09-13T03:15:46.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:15:46 smithi064 ceph-mon[97675]: pgmap v81: 97 pgs: 97 active+clean; 36 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 1.0 MiB/s wr, 121 op/s 2023-09-13T03:15:46.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:15:46 smithi142 ceph-mon[134082]: pgmap v81: 97 pgs: 97 active+clean; 36 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 1.0 MiB/s wr, 121 op/s 2023-09-13T03:15:47.897 INFO:tasks.workunit.client.0.smithi064.stdout: 1 53903 2.75 MB/sec warmup 112 sec latency 3844.101 ms 2023-09-13T03:15:48.897 INFO:tasks.workunit.client.0.smithi064.stdout: 1 53959 2.73 MB/sec warmup 113 sec latency 4472.994 ms 2023-09-13T03:15:48.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:15:48 smithi170 ceph-mon[134328]: pgmap v82: 97 pgs: 97 active+clean; 36 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 842 KiB/s wr, 83 op/s 2023-09-13T03:15:48.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:15:48 smithi064 ceph-mon[97675]: pgmap v82: 97 pgs: 97 active+clean; 36 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 842 KiB/s wr, 83 op/s 2023-09-13T03:15:48.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:15:48 smithi142 ceph-mon[134082]: pgmap v82: 97 pgs: 97 active+clean; 36 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 842 KiB/s wr, 83 op/s 2023-09-13T03:15:49.897 INFO:tasks.workunit.client.0.smithi064.stdout: 1 53959 2.70 MB/sec warmup 114 sec latency 1327.605 ms 2023-09-13T03:15:50.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:15:50 smithi170 ceph-mon[134328]: pgmap v83: 97 pgs: 97 active+clean; 36 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 842 KiB/s wr, 83 op/s 2023-09-13T03:15:50.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:15:50 smithi064 ceph-mon[97675]: pgmap v83: 97 pgs: 97 active+clean; 36 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 842 KiB/s wr, 83 op/s 2023-09-13T03:15:50.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:15:50 smithi142 ceph-mon[134082]: pgmap v83: 97 pgs: 97 active+clean; 36 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 842 KiB/s wr, 83 op/s 2023-09-13T03:15:50.897 INFO:tasks.workunit.client.0.smithi064.stdout: 1 53959 2.68 MB/sec warmup 115 sec latency 2327.704 ms 2023-09-13T03:15:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.303+0000 7f383f1ff700 1 -- 172.21.15.64:0/2229569032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38381005b0 msgr2=0x7f3838100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:15:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.303+0000 7f383f1ff700 1 --2- 172.21.15.64:0/2229569032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38381005b0 0x7f3838100a30 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f38240099c0 tx=0x7f3824009cd0 comp rx=0 tx=0).stop 2023-09-13T03:15:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.304+0000 7f383f1ff700 1 -- 172.21.15.64:0/2229569032 shutdown_connections 2023-09-13T03:15:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.304+0000 7f383f1ff700 1 --2- 172.21.15.64:0/2229569032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3838106670 0x7f3838106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.304+0000 7f383f1ff700 1 --2- 172.21.15.64:0/2229569032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3838100f70 0x7f383810d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.304+0000 7f383f1ff700 1 --2- 172.21.15.64:0/2229569032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38381005b0 0x7f3838100a30 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.304+0000 7f383f1ff700 1 -- 172.21.15.64:0/2229569032 >> 172.21.15.64:0/2229569032 conn(0x7f38380fc060 msgr2=0x7f38380fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:15:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.304+0000 7f383f1ff700 1 -- 172.21.15.64:0/2229569032 shutdown_connections 2023-09-13T03:15:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.304+0000 7f383f1ff700 1 -- 172.21.15.64:0/2229569032 wait complete. 2023-09-13T03:15:51.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.305+0000 7f383f1ff700 1 Processor -- start 2023-09-13T03:15:51.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.305+0000 7f383f1ff700 1 -- start start 2023-09-13T03:15:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.305+0000 7f383f1ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38381005b0 0x7f3838072ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.305+0000 7f383f1ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3838100f70 0x7f3838073020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.305+0000 7f383f1ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3838106670 0x7f38381a4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.305+0000 7f383f1ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38381a4d70 con 0x7f3838100f70 2023-09-13T03:15:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.305+0000 7f383f1ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38381a4eb0 con 0x7f3838106670 2023-09-13T03:15:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.305+0000 7f383f1ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38381a5020 con 0x7f38381005b0 2023-09-13T03:15:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.306+0000 7f383cf9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38381005b0 0x7f3838072ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.306+0000 7f3837fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3838100f70 0x7f3838073020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.306+0000 7f383cf9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38381005b0 0x7f3838072ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:40638/0 (socket says 172.21.15.64:40638) 2023-09-13T03:15:51.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.306+0000 7f383cf9b700 1 -- 172.21.15.64:0/1006675489 learned_addr learned my addr 172.21.15.64:0/1006675489 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:15:51.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.306+0000 7f383d79c700 1 --2- 172.21.15.64:0/1006675489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3838106670 0x7f38381a4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:51.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.306+0000 7f3837fff700 1 -- 172.21.15.64:0/1006675489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38381005b0 msgr2=0x7f3838072ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:15:51.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.306+0000 7f3837fff700 1 --2- 172.21.15.64:0/1006675489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38381005b0 0x7f3838072ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:51.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.306+0000 7f3837fff700 1 -- 172.21.15.64:0/1006675489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3838106670 msgr2=0x7f38381a4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:15:51.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.306+0000 7f3837fff700 1 --2- 172.21.15.64:0/1006675489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3838106670 0x7f38381a4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:51.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.306+0000 7f3837fff700 1 -- 172.21.15.64:0/1006675489 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f382c009520 con 0x7f3838100f70 2023-09-13T03:15:51.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.306+0000 7f383cf9b700 1 --2- 172.21.15.64:0/1006675489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38381005b0 0x7f3838072ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:15:51.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.306+0000 7f3837fff700 1 --2- 172.21.15.64:0/1006675489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3838100f70 0x7f3838073020 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f38240058e0 tx=0x7f382400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:15:51.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.307+0000 7f3835ffb700 1 -- 172.21.15.64:0/1006675489 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382401d070 con 0x7f3838100f70 2023-09-13T03:15:51.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.307+0000 7f383f1ff700 1 -- 172.21.15.64:0/1006675489 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38240095f0 con 0x7f3838100f70 2023-09-13T03:15:51.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.307+0000 7f3835ffb700 1 -- 172.21.15.64:0/1006675489 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3824005050 con 0x7f3838100f70 2023-09-13T03:15:51.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.307+0000 7f3835ffb700 1 -- 172.21.15.64:0/1006675489 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3824004a00 con 0x7f3838100f70 2023-09-13T03:15:51.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.307+0000 7f383f1ff700 1 -- 172.21.15.64:0/1006675489 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38381a5530 con 0x7f3838100f70 2023-09-13T03:15:51.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.310+0000 7f3835ffb700 1 -- 172.21.15.64:0/1006675489 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f3824005400 con 0x7f3838100f70 2023-09-13T03:15:51.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.311+0000 7f3835ffb700 1 --2- 172.21.15.64:0/1006675489 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f38280a07b0 0x7f38280a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:51.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.311+0000 7f3852a31b80 1 Processor -- start 2023-09-13T03:15:51.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.311+0000 7f3835ffb700 1 -- 172.21.15.64:0/1006675489 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f38240cccb0 con 0x7f3838100f70 2023-09-13T03:15:51.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.311+0000 7f3852a31b80 1 -- start start 2023-09-13T03:15:51.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.312+0000 7f383cf9b700 1 --2- 172.21.15.64:0/1006675489 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f38280a07b0 0x7f38280a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:51.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.312+0000 7f3852a31b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559c68131a0 0x5559c6813580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:51.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.312+0000 7f3852a31b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559c680d300 0x5559c680d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:51.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.312+0000 7f3852a31b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5559c680dcc0 0x5559c6819f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:51.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.312+0000 7f3852a31b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559c67666d0 con 0x5559c680dcc0 2023-09-13T03:15:51.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.312+0000 7f3852a31b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559c6763ff0 con 0x5559c680d300 2023-09-13T03:15:51.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.312+0000 7f3852a31b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559c675f8e0 con 0x5559c68131a0 2023-09-13T03:15:51.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.312+0000 7f383cf9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559c68131a0 0x5559c6813580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:51.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.312+0000 7f383d79c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5559c680dcc0 0x5559c6819f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.312+0000 7f3837fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559c680d300 0x5559c680d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.312+0000 7f383cf9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559c68131a0 0x5559c6813580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:40654/0 (socket says 172.21.15.64:40654) 2023-09-13T03:15:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.312+0000 7f3837fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559c680d300 0x5559c680d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:57512/0 (socket says 172.21.15.64:57512) 2023-09-13T03:15:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.312+0000 7f383cf9b700 1 -- 172.21.15.64:0/7515619 learned_addr learned my addr 172.21.15.64:0/7515619 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:15:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.312+0000 7f383cf9b700 1 --2- 172.21.15.64:0/1006675489 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f38280a07b0 0x7f38280a2c70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f382c011700 tx=0x7f382c009260 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:15:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.313+0000 7f3837fff700 1 -- 172.21.15.64:0/7515619 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559c68131a0 msgr2=0x5559c6813580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:15:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.313+0000 7f3837fff700 1 --2- 172.21.15.64:0/7515619 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559c68131a0 0x5559c6813580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.313+0000 7f3837fff700 1 -- 172.21.15.64:0/7515619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5559c680dcc0 msgr2=0x5559c6819f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:15:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.313+0000 7f3837fff700 1 --2- 172.21.15.64:0/7515619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5559c680dcc0 0x5559c6819f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:51.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.313+0000 7f3837fff700 1 -- 172.21.15.64:0/7515619 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559c658bd80 con 0x5559c680d300 2023-09-13T03:15:51.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.313+0000 7f3837fff700 1 --2- 172.21.15.64:0/7515619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559c680d300 0x5559c680d780 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f38240ca3f0 tx=0x7f382409a8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62217d5180d09af6 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:15:51.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.313+0000 7f3822ffd700 1 -- 172.21.15.64:0/7515619 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382409e010 con 0x5559c680d300 2023-09-13T03:15:51.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.313+0000 7f3822ffd700 1 -- 172.21.15.64:0/7515619 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38240a21c0 con 0x5559c680d300 2023-09-13T03:15:51.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.313+0000 7f3822ffd700 1 -- 172.21.15.64:0/7515619 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38240996d0 con 0x5559c680d300 2023-09-13T03:15:51.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.314+0000 7f3852a31b80 1 -- 172.21.15.64:0/7515619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559c680d300 msgr2=0x5559c680d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:15:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.314+0000 7f3852a31b80 1 --2- 172.21.15.64:0/7515619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559c680d300 0x5559c680d780 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f38240ca3f0 tx=0x7f382409a8e0 comp rx=0 tx=0).stop 2023-09-13T03:15:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.314+0000 7f3852a31b80 1 -- 172.21.15.64:0/7515619 shutdown_connections 2023-09-13T03:15:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.314+0000 7f3852a31b80 1 --2- 172.21.15.64:0/7515619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5559c680dcc0 0x5559c6819f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.314+0000 7f3852a31b80 1 --2- 172.21.15.64:0/7515619 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559c68131a0 0x5559c6813580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.314+0000 7f3852a31b80 1 --2- 172.21.15.64:0/7515619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559c680d300 0x5559c680d780 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.314+0000 7f3852a31b80 1 -- 172.21.15.64:0/7515619 >> 172.21.15.64:0/7515619 conn(0x5559c6782030 msgr2=0x5559c6808560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:15:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.314+0000 7f3852a31b80 1 -- 172.21.15.64:0/7515619 shutdown_connections 2023-09-13T03:15:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.314+0000 7f3852a31b80 1 -- 172.21.15.64:0/7515619 wait complete. 2023-09-13T03:15:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.315+0000 7f3852a31b80 1 -- start start 2023-09-13T03:15:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.315+0000 7f3852a31b80 1 Processor -- start 2023-09-13T03:15:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.315+0000 7f3852a31b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559c680d300 0x5559c681cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.315+0000 7f3852a31b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559c680dcc0 0x5559c681d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.315+0000 7f3852a31b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5559c68131a0 0x5559c6816dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.315+0000 7f3852a31b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559c66fdb10 con 0x5559c68131a0 2023-09-13T03:15:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.315+0000 7f3852a31b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559c66fd810 con 0x5559c680d300 2023-09-13T03:15:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.315+0000 7f3852a31b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559c65e18c0 con 0x5559c680dcc0 2023-09-13T03:15:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.315+0000 7f3852a31b80 10 client.0 _ll_register_callbacks cb 0x5559c6647a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:15:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.316+0000 7f383cf9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559c680d300 0x5559c681cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.316+0000 7f383cf9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559c680d300 0x5559c681cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:57516/0 (socket says 172.21.15.64:57516) 2023-09-13T03:15:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.316+0000 7f383cf9b700 1 -- 172.21.15.64:0/2062410640 learned_addr learned my addr 172.21.15.64:0/2062410640 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:15:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.316+0000 7f383d79c700 1 --2- 172.21.15.64:0/2062410640 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5559c68131a0 0x5559c6816dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.316+0000 7f3837fff700 1 --2- 172.21.15.64:0/2062410640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559c680dcc0 0x5559c681d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.316+0000 7f383cf9b700 1 -- 172.21.15.64:0/2062410640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559c680dcc0 msgr2=0x5559c681d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:15:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.316+0000 7f383cf9b700 1 --2- 172.21.15.64:0/2062410640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559c680dcc0 0x5559c681d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.316+0000 7f383cf9b700 1 -- 172.21.15.64:0/2062410640 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5559c68131a0 msgr2=0x5559c6816dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:15:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.316+0000 7f383cf9b700 1 --2- 172.21.15.64:0/2062410640 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5559c68131a0 0x5559c6816dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.316+0000 7f383cf9b700 1 -- 172.21.15.64:0/2062410640 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38381a5530 con 0x5559c680d300 2023-09-13T03:15:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.316+0000 7f3837fff700 1 --2- 172.21.15.64:0/2062410640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559c680dcc0 0x5559c681d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:15:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.316+0000 7f383cf9b700 1 --2- 172.21.15.64:0/2062410640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559c680d300 0x5559c681cb30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f382c00fcf0 tx=0x7f382c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32b37da8013001de server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:15:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.316+0000 7f3817fff700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:15:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.317+0000 7f3817fff700 1 -- 172.21.15.64:0/2062410640 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c022070 con 0x5559c680d300 2023-09-13T03:15:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.317+0000 7f3852a31b80 1 -- 172.21.15.64:0/2062410640 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5559c66b7490 con 0x5559c680d300 2023-09-13T03:15:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.317+0000 7f3817fff700 1 -- 172.21.15.64:0/2062410640 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f382c01b430 con 0x5559c680d300 2023-09-13T03:15:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.317+0000 7f3817fff700 1 -- 172.21.15.64:0/2062410640 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f382c01b5f0 con 0x5559c680d300 2023-09-13T03:15:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.317+0000 7f3852a31b80 10 client.26339 fetch_fsmap learned FSMap version 34 2023-09-13T03:15:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.317+0000 7f3852a31b80 1 -- 172.21.15.64:0/2062410640 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5559c6574c00 con 0x5559c680d300 2023-09-13T03:15:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.317+0000 7f3817fff700 1 -- 172.21.15.64:0/2062410640 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5559c66b7490 con 0x5559c680d300 2023-09-13T03:15:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.317+0000 7f3817fff700 1 -- 172.21.15.64:0/2062410640 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 34) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f382c026830 con 0x5559c680d300 2023-09-13T03:15:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.317+0000 7f3852a31b80 10 client.26339 fetch_fsmap finished waiting for FSMap version 34 2023-09-13T03:15:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.317+0000 7f3852a31b80 10 client.26339 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:15:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.318+0000 7f3852a31b80 20 client.26339 populate_metadata read hostname 'smithi064' 2023-09-13T03:15:51.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.318+0000 7f3852a31b80 1 --2- 172.21.15.64:0/2062410640 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5559c6827100 0x5559c6829520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:51.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.318+0000 7f3852a31b80 4 client.26339 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:15:51.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.318+0000 7f3852a31b80 1 -- 172.21.15.64:0/2062410640 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5559c6774840 con 0x5559c6827100 2023-09-13T03:15:51.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.318+0000 7f3837fff700 1 --2- 172.21.15.64:0/2062410640 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5559c6827100 0x5559c6829520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:51.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.318+0000 7f3817fff700 10 client.26339 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:15:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.318+0000 7f3837fff700 1 --2- 172.21.15.64:0/2062410640 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5559c6827100 0x5559c6829520 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc99a97ba628823 server_cookie=ee3f95a6e7f97dcb in_seq=0 out_seq=0 2023-09-13T03:15:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.319+0000 7f3817fff700 1 -- 172.21.15.64:0/2062410640 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5559c6774840 con 0x5559c6827100 2023-09-13T03:15:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.319+0000 7f3817fff700 10 client.26339 handle_command_reply: tid=0 2023-09-13T03:15:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.319+0000 7f3852a31b80 1 client.26339 shutdown 2023-09-13T03:15:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.319+0000 7f3852a31b80 1 -- 172.21.15.64:0/2062410640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559c680d300 msgr2=0x5559c681cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:15:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.319+0000 7f3852a31b80 1 --2- 172.21.15.64:0/2062410640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559c680d300 0x5559c681cb30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f382c00fcf0 tx=0x7f382c0047c0 comp rx=0 tx=0).stop 2023-09-13T03:15:51.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.320+0000 7f3852a31b80 1 -- 172.21.15.64:0/2062410640 shutdown_connections 2023-09-13T03:15:51.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.320+0000 7f3852a31b80 1 --2- 172.21.15.64:0/2062410640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559c680d300 0x5559c681cb30 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:51.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.320+0000 7f3852a31b80 1 --2- 172.21.15.64:0/2062410640 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5559c6827100 0x5559c6829520 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:51.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.320+0000 7f3852a31b80 1 --2- 172.21.15.64:0/2062410640 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5559c68131a0 0x5559c6816dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:51.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.320+0000 7f3852a31b80 1 --2- 172.21.15.64:0/2062410640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559c680dcc0 0x5559c681d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.320+0000 7f3852a31b80 1 -- 172.21.15.64:0/2062410640 >> 172.21.15.64:0/2062410640 conn(0x5559c6782030 msgr2=0x5559c68077a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:15:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.320+0000 7f3817fff700 0 client.26339 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:15:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.320+0000 7f3852a31b80 1 -- 172.21.15.64:0/2062410640 shutdown_connections 2023-09-13T03:15:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.320+0000 7f3852a31b80 1 -- 172.21.15.64:0/2062410640 wait complete. 2023-09-13T03:15:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.320+0000 7f3852a31b80 20 client.26339 trim_cache size 0 max 16384 2023-09-13T03:15:51.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.335+0000 7f3852a31b80 1 Processor -- start 2023-09-13T03:15:51.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.335+0000 7f3852a31b80 1 -- start start 2023-09-13T03:15:51.339 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.336+0000 7f3852a31b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559c680d300 0x5559c681c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.336+0000 7f3852a31b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5559c680dcc0 0x5559c681cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.336+0000 7f3852a31b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559c68131a0 0x5559c68165a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.336+0000 7f3852a31b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559c6811c20 con 0x5559c680dcc0 2023-09-13T03:15:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.336+0000 7f3852a31b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559c680d6e0 con 0x5559c680d300 2023-09-13T03:15:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.336+0000 7f3852a31b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559c6811940 con 0x5559c68131a0 2023-09-13T03:15:51.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.336+0000 7f383cf9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559c680d300 0x5559c681c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:51.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.336+0000 7f383cf9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559c680d300 0x5559c681c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:57528/0 (socket says 172.21.15.64:57528) 2023-09-13T03:15:51.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.336+0000 7f383cf9b700 1 -- 172.21.15.64:0/1901941318 learned_addr learned my addr 172.21.15.64:0/1901941318 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:15:51.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.336+0000 7f3837fff700 1 --2- 172.21.15.64:0/1901941318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5559c680dcc0 0x5559c681cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:51.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.336+0000 7f383d79c700 1 --2- 172.21.15.64:0/1901941318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559c68131a0 0x5559c68165a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:51.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.336+0000 7f383cf9b700 1 -- 172.21.15.64:0/1901941318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559c68131a0 msgr2=0x5559c68165a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:15:51.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.336+0000 7f383cf9b700 1 --2- 172.21.15.64:0/1901941318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559c68131a0 0x5559c68165a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:51.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.336+0000 7f383cf9b700 1 -- 172.21.15.64:0/1901941318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5559c680dcc0 msgr2=0x5559c681cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:15:51.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.336+0000 7f383cf9b700 1 --2- 172.21.15.64:0/1901941318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5559c680dcc0 0x5559c681cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:51.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.336+0000 7f383cf9b700 1 -- 172.21.15.64:0/1901941318 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559c65b21f0 con 0x5559c680d300 2023-09-13T03:15:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.337+0000 7f383cf9b700 1 --2- 172.21.15.64:0/1901941318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559c680d300 0x5559c681c700 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f382c00a010 tx=0x7f382c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae4c10924e7a81d3 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:15:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.337+0000 7f38237fe700 1 -- 172.21.15.64:0/1901941318 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c022070 con 0x5559c680d300 2023-09-13T03:15:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.337+0000 7f38237fe700 1 -- 172.21.15.64:0/1901941318 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f382c00f530 con 0x5559c680d300 2023-09-13T03:15:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.337+0000 7f38237fe700 1 -- 172.21.15.64:0/1901941318 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c01aad0 con 0x5559c680d300 2023-09-13T03:15:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.337+0000 7f3852a31b80 1 -- 172.21.15.64:0/1901941318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559c680d300 msgr2=0x5559c681c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:15:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.337+0000 7f3852a31b80 1 --2- 172.21.15.64:0/1901941318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559c680d300 0x5559c681c700 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f382c00a010 tx=0x7f382c004750 comp rx=0 tx=0).stop 2023-09-13T03:15:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.337+0000 7f3852a31b80 1 -- 172.21.15.64:0/1901941318 shutdown_connections 2023-09-13T03:15:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.337+0000 7f3852a31b80 1 --2- 172.21.15.64:0/1901941318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559c680d300 0x5559c681c700 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.337+0000 7f3852a31b80 1 --2- 172.21.15.64:0/1901941318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559c68131a0 0x5559c68165a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.338+0000 7f3852a31b80 1 --2- 172.21.15.64:0/1901941318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5559c680dcc0 0x5559c681cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.338+0000 7f3852a31b80 1 -- 172.21.15.64:0/1901941318 >> 172.21.15.64:0/1901941318 conn(0x5559c6782030 msgr2=0x5559c6808090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:15:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.338+0000 7f3852a31b80 1 -- 172.21.15.64:0/1901941318 shutdown_connections 2023-09-13T03:15:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.338+0000 7f3852a31b80 1 -- 172.21.15.64:0/1901941318 wait complete. 2023-09-13T03:15:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.338+0000 7f3852a31b80 1 -- start start 2023-09-13T03:15:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.338+0000 7f3852a31b80 1 Processor -- start 2023-09-13T03:15:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.339+0000 7f3852a31b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559c680d300 0x5559c686f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.339+0000 7f3852a31b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559c680dcc0 0x5559c686f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.339+0000 7f3852a31b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5559c68131a0 0x5559c6879f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.339+0000 7f3852a31b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559c687a4a0 con 0x5559c68131a0 2023-09-13T03:15:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.339+0000 7f3852a31b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559c687a5e0 con 0x5559c680dcc0 2023-09-13T03:15:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.339+0000 7f3852a31b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559c687a720 con 0x5559c680d300 2023-09-13T03:15:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.339+0000 7f3852a31b80 10 client.0 _ll_register_callbacks cb 0x5559c6647a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:15:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.339+0000 7f383d79c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5559c68131a0 0x5559c6879f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:51.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.339+0000 7f383d79c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5559c68131a0 0x5559c6879f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45570/0 (socket says 172.21.15.64:45570) 2023-09-13T03:15:51.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.339+0000 7f383cf9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559c680d300 0x5559c686f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:51.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.339+0000 7f383d79c700 1 -- 172.21.15.64:0/250439940 learned_addr learned my addr 172.21.15.64:0/250439940 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:15:51.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.339+0000 7f3837fff700 1 --2- 172.21.15.64:0/250439940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559c680dcc0 0x5559c686f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:51.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.340+0000 7f3837fff700 1 -- 172.21.15.64:0/250439940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559c680d300 msgr2=0x5559c686f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:15:51.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.340+0000 7f3837fff700 1 --2- 172.21.15.64:0/250439940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559c680d300 0x5559c686f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:51.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.340+0000 7f3837fff700 1 -- 172.21.15.64:0/250439940 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5559c68131a0 msgr2=0x5559c6879f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:15:51.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.340+0000 7f3837fff700 1 --2- 172.21.15.64:0/250439940 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5559c68131a0 0x5559c6879f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:51.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.340+0000 7f3837fff700 1 -- 172.21.15.64:0/250439940 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5559c6574c00 con 0x5559c680dcc0 2023-09-13T03:15:51.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.340+0000 7f383d79c700 1 --2- 172.21.15.64:0/250439940 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5559c68131a0 0x5559c6879f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:15:51.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.340+0000 7f383cf9b700 1 --2- 172.21.15.64:0/250439940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559c680d300 0x5559c686f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:15:51.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.340+0000 7f3837fff700 1 --2- 172.21.15.64:0/250439940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559c680dcc0 0x5559c686f610 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x5559c680e290 tx=0x7f3824003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be26d03053c3cfc2 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:15:51.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.340+0000 7f3817fff700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:15:51.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.340+0000 7f3817fff700 1 -- 172.21.15.64:0/250439940 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382409e090 con 0x5559c680dcc0 2023-09-13T03:15:51.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.340+0000 7f3852a31b80 1 -- 172.21.15.64:0/250439940 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f38240a21c0 con 0x5559c680dcc0 2023-09-13T03:15:51.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.340+0000 7f3817fff700 1 -- 172.21.15.64:0/250439940 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f38240996d0 con 0x5559c680dcc0 2023-09-13T03:15:51.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.341+0000 7f3817fff700 1 -- 172.21.15.64:0/250439940 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3824099890 con 0x5559c680dcc0 2023-09-13T03:15:51.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.341+0000 7f3852a31b80 10 client.26345 fetch_fsmap learned FSMap version 34 2023-09-13T03:15:51.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.341+0000 7f3852a31b80 1 -- 172.21.15.64:0/250439940 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5559c658a250 con 0x5559c680dcc0 2023-09-13T03:15:51.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.341+0000 7f3817fff700 1 -- 172.21.15.64:0/250439940 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38240a21c0 con 0x5559c680dcc0 2023-09-13T03:15:51.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.341+0000 7f3817fff700 1 -- 172.21.15.64:0/250439940 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 34) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f38240b2250 con 0x5559c680dcc0 2023-09-13T03:15:51.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.341+0000 7f3852a31b80 10 client.26345 fetch_fsmap finished waiting for FSMap version 34 2023-09-13T03:15:51.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.341+0000 7f3852a31b80 10 client.26345 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:15:51.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.341+0000 7f3852a31b80 20 client.26345 populate_metadata read hostname 'smithi064' 2023-09-13T03:15:51.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.341+0000 7f3852a31b80 1 --2- 172.21.15.64:0/250439940 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5559c687baa0 0x5559c687dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:15:51.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.341+0000 7f3852a31b80 4 client.26345 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:15:51.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.341+0000 7f3852a31b80 1 -- 172.21.15.64:0/250439940 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5559c67588c0 con 0x5559c687baa0 2023-09-13T03:15:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.341+0000 7f383cf9b700 1 --2- 172.21.15.64:0/250439940 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5559c687baa0 0x5559c687dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:15:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.342+0000 7f3817fff700 10 client.26345 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:15:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.342+0000 7f383cf9b700 1 --2- 172.21.15.64:0/250439940 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5559c687baa0 0x5559c687dec0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b13e17aa8211d9c8 server_cookie=e73b227e99c27363 in_seq=0 out_seq=0 2023-09-13T03:15:51.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.368+0000 7f3817fff700 1 -- 172.21.15.64:0/250439940 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5559c67588c0 con 0x5559c687baa0 2023-09-13T03:15:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.368+0000 7f3817fff700 10 client.26345 handle_command_reply: tid=0 2023-09-13T03:15:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.368+0000 7f3852a31b80 1 client.26345 shutdown 2023-09-13T03:15:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.368+0000 7f3852a31b80 1 -- 172.21.15.64:0/250439940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559c680dcc0 msgr2=0x5559c686f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:15:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.368+0000 7f3852a31b80 1 --2- 172.21.15.64:0/250439940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559c680dcc0 0x5559c686f610 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x5559c680e290 tx=0x7f3824003680 comp rx=0 tx=0).stop 2023-09-13T03:15:51.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.368+0000 7f3852a31b80 1 -- 172.21.15.64:0/250439940 shutdown_connections 2023-09-13T03:15:51.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.368+0000 7f3852a31b80 1 --2- 172.21.15.64:0/250439940 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5559c687baa0 0x5559c687dec0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:51.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.368+0000 7f3852a31b80 1 --2- 172.21.15.64:0/250439940 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5559c68131a0 0x5559c6879f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:51.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.368+0000 7f3852a31b80 1 --2- 172.21.15.64:0/250439940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559c680d300 0x5559c686f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:51.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.368+0000 7f3817fff700 0 client.26345 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:15:51.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.368+0000 7f3852a31b80 1 --2- 172.21.15.64:0/250439940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559c680dcc0 0x5559c686f610 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:51.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.368+0000 7f3852a31b80 1 -- 172.21.15.64:0/250439940 >> 172.21.15.64:0/250439940 conn(0x5559c6782030 msgr2=0x5559c6812030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:15:51.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.368+0000 7f3852a31b80 1 -- 172.21.15.64:0/250439940 shutdown_connections 2023-09-13T03:15:51.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.368+0000 7f3852a31b80 1 -- 172.21.15.64:0/250439940 wait complete. 2023-09-13T03:15:51.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.368+0000 7f3852a31b80 20 client.26345 trim_cache size 0 max 16384 2023-09-13T03:15:51.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:15:51.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:15:51.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:15:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:15:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.369+0000 7f383f1ff700 1 -- 172.21.15.64:0/1006675489 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f38280a07b0 msgr2=0x7f38280a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:15:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.369+0000 7f383f1ff700 1 --2- 172.21.15.64:0/1006675489 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f38280a07b0 0x7f38280a2c70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f382c011700 tx=0x7f382c009260 comp rx=0 tx=0).stop 2023-09-13T03:15:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.369+0000 7f383f1ff700 1 -- 172.21.15.64:0/1006675489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3838100f70 msgr2=0x7f3838073020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:15:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.369+0000 7f383f1ff700 1 --2- 172.21.15.64:0/1006675489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3838100f70 0x7f3838073020 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f38240058e0 tx=0x7f382400f690 comp rx=0 tx=0).stop 2023-09-13T03:15:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.369+0000 7f383f1ff700 1 -- 172.21.15.64:0/1006675489 shutdown_connections 2023-09-13T03:15:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.369+0000 7f383f1ff700 1 --2- 172.21.15.64:0/1006675489 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f38280a07b0 0x7f38280a2c70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.369+0000 7f383f1ff700 1 --2- 172.21.15.64:0/1006675489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3838106670 0x7f38381a4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.369+0000 7f383f1ff700 1 --2- 172.21.15.64:0/1006675489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38381005b0 0x7f3838072ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.369+0000 7f383f1ff700 1 --2- 172.21.15.64:0/1006675489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3838100f70 0x7f3838073020 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:15:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.369+0000 7f383f1ff700 1 -- 172.21.15.64:0/1006675489 >> 172.21.15.64:0/1006675489 conn(0x7f38380fc060 msgr2=0x7f38380fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:15:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.369+0000 7f383f1ff700 1 -- 172.21.15.64:0/1006675489 shutdown_connections 2023-09-13T03:15:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:15:51.369+0000 7f383f1ff700 1 -- 172.21.15.64:0/1006675489 wait complete. 2023-09-13T03:15:51.897 INFO:tasks.workunit.client.0.smithi064.stdout: 1 53959 2.66 MB/sec warmup 116 sec latency 3327.776 ms 2023-09-13T03:15:52.897 INFO:tasks.workunit.client.0.smithi064.stdout: 1 53959 2.63 MB/sec warmup 117 sec latency 4327.921 ms 2023-09-13T03:15:52.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:15:52 smithi170 ceph-mon[134328]: pgmap v84: 97 pgs: 97 active+clean; 36 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 885 KiB/s wr, 94 op/s 2023-09-13T03:15:52.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:15:52 smithi064 ceph-mon[97675]: pgmap v84: 97 pgs: 97 active+clean; 36 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 885 KiB/s wr, 94 op/s 2023-09-13T03:15:52.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:15:52 smithi142 ceph-mon[134082]: pgmap v84: 97 pgs: 97 active+clean; 36 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 885 KiB/s wr, 94 op/s 2023-09-13T03:15:53.898 INFO:tasks.workunit.client.0.smithi064.stdout: 1 54027 2.61 MB/sec warmup 118 sec latency 4952.254 ms 2023-09-13T03:15:54.898 INFO:tasks.workunit.client.0.smithi064.stdout: 1 54027 2.59 MB/sec warmup 119 sec latency 1323.564 ms 2023-09-13T03:15:54.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:15:54 smithi170 ceph-mon[134328]: pgmap v85: 97 pgs: 97 active+clean; 36 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 833 KiB/s wr, 75 op/s 2023-09-13T03:15:54.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:15:54 smithi064 ceph-mon[97675]: pgmap v85: 97 pgs: 97 active+clean; 36 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 833 KiB/s wr, 75 op/s 2023-09-13T03:15:54.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:15:54 smithi142 ceph-mon[134082]: pgmap v85: 97 pgs: 97 active+clean; 36 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 833 KiB/s wr, 75 op/s 2023-09-13T03:15:56.898 INFO:tasks.workunit.client.0.smithi064.stdout: 1 54027 0.00 MB/sec execute 1 sec latency 3323.803 ms 2023-09-13T03:15:56.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:15:56 smithi170 ceph-mon[134328]: pgmap v86: 97 pgs: 97 active+clean; 33 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 865 KiB/s wr, 93 op/s 2023-09-13T03:15:56.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:15:56 smithi064 ceph-mon[97675]: pgmap v86: 97 pgs: 97 active+clean; 33 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 865 KiB/s wr, 93 op/s 2023-09-13T03:15:56.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:15:56 smithi142 ceph-mon[134082]: pgmap v86: 97 pgs: 97 active+clean; 33 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 865 KiB/s wr, 93 op/s 2023-09-13T03:15:57.898 INFO:tasks.workunit.client.0.smithi064.stdout: 1 54027 0.00 MB/sec execute 2 sec latency 4323.963 ms 2023-09-13T03:15:58.898 INFO:tasks.workunit.client.0.smithi064.stdout: 1 54708 1.56 MB/sec execute 3 sec latency 4946.984 ms 2023-09-13T03:15:58.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:15:58 smithi170 ceph-mon[134328]: pgmap v87: 97 pgs: 97 active+clean; 33 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 75 KiB/s wr, 29 op/s 2023-09-13T03:15:58.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:15:58 smithi064 ceph-mon[97675]: pgmap v87: 97 pgs: 97 active+clean; 33 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 75 KiB/s wr, 29 op/s 2023-09-13T03:15:58.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:15:58 smithi142 ceph-mon[134082]: pgmap v87: 97 pgs: 97 active+clean; 33 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 75 KiB/s wr, 29 op/s 2023-09-13T03:15:59.898 INFO:tasks.workunit.client.0.smithi064.stdout: 1 56984 4.60 MB/sec execute 4 sec latency 10.418 ms 2023-09-13T03:16:00.659 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:16:00 smithi170 ceph-mon[134328]: pgmap v88: 97 pgs: 97 active+clean; 33 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 75 KiB/s wr, 29 op/s 2023-09-13T03:16:00.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:16:00 smithi064 ceph-mon[97675]: pgmap v88: 97 pgs: 97 active+clean; 33 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 75 KiB/s wr, 29 op/s 2023-09-13T03:16:00.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:16:00 smithi142 ceph-mon[134082]: pgmap v88: 97 pgs: 97 active+clean; 33 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 75 KiB/s wr, 29 op/s 2023-09-13T03:16:00.898 INFO:tasks.workunit.client.0.smithi064.stdout: 1 57447 3.76 MB/sec execute 5 sec latency 421.105 ms 2023-09-13T03:16:01.898 INFO:tasks.workunit.client.0.smithi064.stdout: 1 57447 3.14 MB/sec execute 6 sec latency 1421.182 ms 2023-09-13T03:16:02.899 INFO:tasks.workunit.client.0.smithi064.stdout: 1 57447 2.69 MB/sec execute 7 sec latency 2421.251 ms 2023-09-13T03:16:02.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:16:02 smithi170 ceph-mon[134328]: pgmap v89: 97 pgs: 97 active+clean; 35 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 291 KiB/s wr, 80 op/s 2023-09-13T03:16:02.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:16:02 smithi064 ceph-mon[97675]: pgmap v89: 97 pgs: 97 active+clean; 35 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 291 KiB/s wr, 80 op/s 2023-09-13T03:16:02.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:16:02 smithi142 ceph-mon[134082]: pgmap v89: 97 pgs: 97 active+clean; 35 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 291 KiB/s wr, 80 op/s 2023-09-13T03:16:03.899 INFO:tasks.workunit.client.0.smithi064.stdout: 1 57503 2.37 MB/sec execute 8 sec latency 3050.814 ms 2023-09-13T03:16:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.531+0000 7fd7ab7ff700 1 -- 172.21.15.64:0/3197146557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7a410ce20 msgr2=0x7fd7a410f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.531+0000 7fd7ab7ff700 1 --2- 172.21.15.64:0/3197146557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7a410ce20 0x7fd7a410f250 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd79c00b0a0 tx=0x7fd79c00b3b0 comp rx=0 tx=0).stop 2023-09-13T03:16:04.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.531+0000 7fd7ab7ff700 1 -- 172.21.15.64:0/3197146557 shutdown_connections 2023-09-13T03:16:04.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.531+0000 7fd7ab7ff700 1 --2- 172.21.15.64:0/3197146557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7a410ce20 0x7fd7a410f250 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:04.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.531+0000 7fd7ab7ff700 1 --2- 172.21.15.64:0/3197146557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7a40729d0 0x7fd7a4072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:04.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.531+0000 7fd7ab7ff700 1 --2- 172.21.15.64:0/3197146557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7a40732f0 0x7fd7a410c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.531+0000 7fd7ab7ff700 1 -- 172.21.15.64:0/3197146557 >> 172.21.15.64:0/3197146557 conn(0x7fd7a40fc050 msgr2=0x7fd7a40fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:16:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.531+0000 7fd7ab7ff700 1 -- 172.21.15.64:0/3197146557 shutdown_connections 2023-09-13T03:16:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.531+0000 7fd7ab7ff700 1 -- 172.21.15.64:0/3197146557 wait complete. 2023-09-13T03:16:04.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.532+0000 7fd7ab7ff700 1 Processor -- start 2023-09-13T03:16:04.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.533+0000 7fd7ab7ff700 1 -- start start 2023-09-13T03:16:04.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.533+0000 7fd7ab7ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7a40729d0 0x7fd7a419e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:04.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.533+0000 7fd7ab7ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7a40732f0 0x7fd7a419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:04.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.533+0000 7fd7ab7ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7a410ce20 0x7fd7a4198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:04.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.533+0000 7fd7ab7ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a4198eb0 con 0x7fd7a410ce20 2023-09-13T03:16:04.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.533+0000 7fd7ab7ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a4199020 con 0x7fd7a40729d0 2023-09-13T03:16:04.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.533+0000 7fd7ab7ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a41991e0 con 0x7fd7a40732f0 2023-09-13T03:16:04.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.533+0000 7fd7a959b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7a40729d0 0x7fd7a419e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:04.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.533+0000 7fd7a9d9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7a410ce20 0x7fd7a4198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:04.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.533+0000 7fd7a959b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7a40729d0 0x7fd7a419e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:45700/0 (socket says 172.21.15.64:45700) 2023-09-13T03:16:04.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.533+0000 7fd7a8d9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7a40732f0 0x7fd7a419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:04.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.533+0000 7fd7a9d9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7a410ce20 0x7fd7a4198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44442/0 (socket says 172.21.15.64:44442) 2023-09-13T03:16:04.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.533+0000 7fd7a959b700 1 -- 172.21.15.64:0/2709874383 learned_addr learned my addr 172.21.15.64:0/2709874383 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:16:04.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.534+0000 7fd7a9d9c700 1 -- 172.21.15.64:0/2709874383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7a40732f0 msgr2=0x7fd7a419ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:04.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.534+0000 7fd7a9d9c700 1 --2- 172.21.15.64:0/2709874383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7a40732f0 0x7fd7a419ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:04.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.534+0000 7fd7a9d9c700 1 -- 172.21.15.64:0/2709874383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7a40729d0 msgr2=0x7fd7a419e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:04.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.534+0000 7fd7a9d9c700 1 --2- 172.21.15.64:0/2709874383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7a40729d0 0x7fd7a419e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:04.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.534+0000 7fd7a9d9c700 1 -- 172.21.15.64:0/2709874383 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd79c009d00 con 0x7fd7a410ce20 2023-09-13T03:16:04.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.534+0000 7fd7a8d9a700 1 --2- 172.21.15.64:0/2709874383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7a40732f0 0x7fd7a419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:16:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.534+0000 7fd7a959b700 1 --2- 172.21.15.64:0/2709874383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7a40729d0 0x7fd7a419e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:16:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.534+0000 7fd7a9d9c700 1 --2- 172.21.15.64:0/2709874383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7a410ce20 0x7fd7a4198840 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd79c009fd0 tx=0x7fd79c007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:16:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.534+0000 7fd79a7fc700 1 -- 172.21.15.64:0/2709874383 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79c00e040 con 0x7fd7a410ce20 2023-09-13T03:16:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.534+0000 7fd7ab7ff700 1 -- 172.21.15.64:0/2709874383 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7a4199460 con 0x7fd7a410ce20 2023-09-13T03:16:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.534+0000 7fd79a7fc700 1 -- 172.21.15.64:0/2709874383 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd79c01d470 con 0x7fd7a410ce20 2023-09-13T03:16:04.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.534+0000 7fd79a7fc700 1 -- 172.21.15.64:0/2709874383 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79c0052a0 con 0x7fd7a410ce20 2023-09-13T03:16:04.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.535+0000 7fd7ab7ff700 1 -- 172.21.15.64:0/2709874383 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7a41a59d0 con 0x7fd7a410ce20 2023-09-13T03:16:04.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.538+0000 7fd79a7fc700 1 -- 172.21.15.64:0/2709874383 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fd79c01d5e0 con 0x7fd7a410ce20 2023-09-13T03:16:04.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.540+0000 7fd7bf212b80 1 Processor -- start 2023-09-13T03:16:04.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.540+0000 7fd7bf212b80 1 -- start start 2023-09-13T03:16:04.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.540+0000 7fd79a7fc700 1 --2- 172.21.15.64:0/2709874383 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fd7900a0830 0x7fd7900a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:04.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.540+0000 7fd7bf212b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b5a4271a0 0x558b5a427580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:04.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.540+0000 7fd7bf212b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b5a421300 0x558b5a421780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:04.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.540+0000 7fd7bf212b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b5a421cc0 0x558b5a42df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:04.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.540+0000 7fd7bf212b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b5a37a6d0 con 0x558b5a421cc0 2023-09-13T03:16:04.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.540+0000 7fd79a7fc700 1 -- 172.21.15.64:0/2709874383 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd79c0cd340 con 0x7fd7a410ce20 2023-09-13T03:16:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.540+0000 7fd7bf212b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b5a377ff0 con 0x558b5a4271a0 2023-09-13T03:16:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.540+0000 7fd7bf212b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b5a3738e0 con 0x558b5a421300 2023-09-13T03:16:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.541+0000 7fd7a8d9a700 1 --2- 172.21.15.64:0/2709874383 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fd7900a0830 0x7fd7900a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.541+0000 7fd7a959b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b5a4271a0 0x558b5a427580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.541+0000 7fd7a9d9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b5a421cc0 0x558b5a42df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.541+0000 7fd7a959b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b5a4271a0 0x558b5a427580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:45716/0 (socket says 172.21.15.64:45716) 2023-09-13T03:16:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.541+0000 7fd7a959b700 1 -- 172.21.15.64:0/1754302291 learned_addr learned my addr 172.21.15.64:0/1754302291 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:16:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.541+0000 7fd7a8d9a700 1 --2- 172.21.15.64:0/1754302291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b5a421300 0x558b5a421780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.541+0000 7fd7a959b700 1 -- 172.21.15.64:0/1754302291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b5a421300 msgr2=0x558b5a421780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:16:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.541+0000 7fd7a959b700 1 --2- 172.21.15.64:0/1754302291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b5a421300 0x558b5a421780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.541+0000 7fd7a959b700 1 -- 172.21.15.64:0/1754302291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b5a421cc0 msgr2=0x558b5a42df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:16:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.541+0000 7fd7a959b700 1 --2- 172.21.15.64:0/1754302291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b5a421cc0 0x558b5a42df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.541+0000 7fd7a959b700 1 -- 172.21.15.64:0/1754302291 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b5a19fd80 con 0x558b5a4271a0 2023-09-13T03:16:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.541+0000 7fd7a8d9a700 1 --2- 172.21.15.64:0/2709874383 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fd7900a0830 0x7fd7900a2cf0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd794009fd0 tx=0x7fd794013c30 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:16:04.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.542+0000 7fd7a959b700 1 --2- 172.21.15.64:0/1754302291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b5a4271a0 0x558b5a427580 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd7a00117e0 tx=0x7fd7a0011af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=833caea87ced085e server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:16:04.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.542+0000 7fd78f7fe700 1 -- 172.21.15.64:0/1754302291 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a0019410 con 0x558b5a4271a0 2023-09-13T03:16:04.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.542+0000 7fd78f7fe700 1 -- 172.21.15.64:0/1754302291 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7a0019a30 con 0x558b5a4271a0 2023-09-13T03:16:04.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.542+0000 7fd78f7fe700 1 -- 172.21.15.64:0/1754302291 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a00189b0 con 0x558b5a4271a0 2023-09-13T03:16:04.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.542+0000 7fd7bf212b80 1 -- 172.21.15.64:0/1754302291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b5a4271a0 msgr2=0x558b5a427580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.542+0000 7fd7bf212b80 1 --2- 172.21.15.64:0/1754302291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b5a4271a0 0x558b5a427580 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd7a00117e0 tx=0x7fd7a0011af0 comp rx=0 tx=0).stop 2023-09-13T03:16:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.543+0000 7fd7bf212b80 1 -- 172.21.15.64:0/1754302291 shutdown_connections 2023-09-13T03:16:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.543+0000 7fd7bf212b80 1 --2- 172.21.15.64:0/1754302291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b5a4271a0 0x558b5a427580 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.543+0000 7fd7bf212b80 1 --2- 172.21.15.64:0/1754302291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b5a421cc0 0x558b5a42df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.543+0000 7fd7bf212b80 1 --2- 172.21.15.64:0/1754302291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b5a421300 0x558b5a421780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:04.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.543+0000 7fd7bf212b80 1 -- 172.21.15.64:0/1754302291 >> 172.21.15.64:0/1754302291 conn(0x558b5a396030 msgr2=0x558b5a41c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:16:04.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.543+0000 7fd7bf212b80 1 -- 172.21.15.64:0/1754302291 shutdown_connections 2023-09-13T03:16:04.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.543+0000 7fd7bf212b80 1 -- 172.21.15.64:0/1754302291 wait complete. 2023-09-13T03:16:04.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.543+0000 7fd7bf212b80 1 -- start start 2023-09-13T03:16:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.543+0000 7fd7bf212b80 1 Processor -- start 2023-09-13T03:16:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.544+0000 7fd7bf212b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b5a421300 0x558b5a42dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.544+0000 7fd7bf212b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b5a421cc0 0x558b5a42b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.544+0000 7fd7bf212b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b5a4271a0 0x558b5a42b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.544+0000 7fd7bf212b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b5a311b10 con 0x558b5a421cc0 2023-09-13T03:16:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.544+0000 7fd7bf212b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b5a311810 con 0x558b5a4271a0 2023-09-13T03:16:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.544+0000 7fd7bf212b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b5a1f58c0 con 0x558b5a421300 2023-09-13T03:16:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.544+0000 7fd7bf212b80 10 client.0 _ll_register_callbacks cb 0x558b5a25ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:16:04.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.544+0000 7fd7a8d9a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b5a421cc0 0x558b5a42b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:04.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.544+0000 7fd7a8d9a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b5a421cc0 0x558b5a42b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44448/0 (socket says 172.21.15.64:44448) 2023-09-13T03:16:04.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.544+0000 7fd7a8d9a700 1 -- 172.21.15.64:0/365932966 learned_addr learned my addr 172.21.15.64:0/365932966 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:16:04.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.544+0000 7fd7a959b700 1 --2- 172.21.15.64:0/365932966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b5a421300 0x558b5a42dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:04.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.544+0000 7fd7a9d9c700 1 --2- 172.21.15.64:0/365932966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b5a4271a0 0x558b5a42b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:04.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.545+0000 7fd7a8d9a700 1 -- 172.21.15.64:0/365932966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b5a421300 msgr2=0x558b5a42dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:16:04.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.545+0000 7fd7a8d9a700 1 --2- 172.21.15.64:0/365932966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b5a421300 0x558b5a42dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:04.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.545+0000 7fd7a8d9a700 1 -- 172.21.15.64:0/365932966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b5a4271a0 msgr2=0x558b5a42b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:16:04.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.545+0000 7fd7a8d9a700 1 --2- 172.21.15.64:0/365932966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b5a4271a0 0x558b5a42b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:04.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.545+0000 7fd7a8d9a700 1 -- 172.21.15.64:0/365932966 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7a0011440 con 0x558b5a421cc0 2023-09-13T03:16:04.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.545+0000 7fd7a8d9a700 1 --2- 172.21.15.64:0/365932966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b5a421cc0 0x558b5a42b040 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd794012240 tx=0x7fd7940047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be72b76dc1092057 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:16:04.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.545+0000 7fd78cff9700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:16:04.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.545+0000 7fd78cff9700 1 -- 172.21.15.64:0/365932966 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd794003890 con 0x558b5a421cc0 2023-09-13T03:16:04.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.545+0000 7fd7bf212b80 1 -- 172.21.15.64:0/365932966 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558b5a2cb490 con 0x558b5a421cc0 2023-09-13T03:16:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.545+0000 7fd78cff9700 1 -- 172.21.15.64:0/365932966 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd7940039f0 con 0x558b5a421cc0 2023-09-13T03:16:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.545+0000 7fd78cff9700 1 -- 172.21.15.64:0/365932966 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd794003bb0 con 0x558b5a421cc0 2023-09-13T03:16:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.546+0000 7fd7bf212b80 10 client.16317 fetch_fsmap learned FSMap version 34 2023-09-13T03:16:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.546+0000 7fd7bf212b80 1 -- 172.21.15.64:0/365932966 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x558b5a188c00 con 0x558b5a421cc0 2023-09-13T03:16:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.546+0000 7fd78cff9700 1 -- 172.21.15.64:0/365932966 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558b5a2cb490 con 0x558b5a421cc0 2023-09-13T03:16:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.546+0000 7fd78cff9700 1 -- 172.21.15.64:0/365932966 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 34) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fd794020cf0 con 0x558b5a421cc0 2023-09-13T03:16:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.546+0000 7fd7bf212b80 10 client.16317 fetch_fsmap finished waiting for FSMap version 34 2023-09-13T03:16:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.546+0000 7fd7bf212b80 10 client.16317 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:16:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.546+0000 7fd7bf212b80 20 client.16317 populate_metadata read hostname 'smithi064' 2023-09-13T03:16:04.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.546+0000 7fd7bf212b80 1 --2- 172.21.15.64:0/365932966 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x558b5a43b470 0x558b5a43d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:04.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.546+0000 7fd7bf212b80 4 client.16317 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:16:04.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.546+0000 7fd7bf212b80 1 -- 172.21.15.64:0/365932966 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558b5a388840 con 0x558b5a43b470 2023-09-13T03:16:04.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.546+0000 7fd7a959b700 1 --2- 172.21.15.64:0/365932966 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x558b5a43b470 0x558b5a43d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:04.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.547+0000 7fd7a959b700 1 --2- 172.21.15.64:0/365932966 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x558b5a43b470 0x558b5a43d890 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8eb3bb6d391b95d server_cookie=57651a907fd2ff83 in_seq=0 out_seq=0 2023-09-13T03:16:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.547+0000 7fd78cff9700 10 client.16317 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:16:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.548+0000 7fd78cff9700 1 -- 172.21.15.64:0/365932966 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558b5a388840 con 0x558b5a43b470 2023-09-13T03:16:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.548+0000 7fd78cff9700 10 client.16317 handle_command_reply: tid=0 2023-09-13T03:16:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.548+0000 7fd7bf212b80 1 client.16317 shutdown 2023-09-13T03:16:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.548+0000 7fd7bf212b80 1 -- 172.21.15.64:0/365932966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b5a421cc0 msgr2=0x558b5a42b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.548+0000 7fd7bf212b80 1 --2- 172.21.15.64:0/365932966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b5a421cc0 0x558b5a42b040 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd794012240 tx=0x7fd7940047c0 comp rx=0 tx=0).stop 2023-09-13T03:16:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.548+0000 7fd7bf212b80 1 -- 172.21.15.64:0/365932966 shutdown_connections 2023-09-13T03:16:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.548+0000 7fd7bf212b80 1 --2- 172.21.15.64:0/365932966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b5a4271a0 0x558b5a42b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.548+0000 7fd7bf212b80 1 --2- 172.21.15.64:0/365932966 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x558b5a43b470 0x558b5a43d890 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.548+0000 7fd7bf212b80 1 --2- 172.21.15.64:0/365932966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b5a421300 0x558b5a42dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.548+0000 7fd78cff9700 0 client.16317 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:16:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.548+0000 7fd7bf212b80 1 --2- 172.21.15.64:0/365932966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b5a421cc0 0x558b5a42b040 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.548+0000 7fd7bf212b80 1 -- 172.21.15.64:0/365932966 >> 172.21.15.64:0/365932966 conn(0x558b5a396030 msgr2=0x558b5a41b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:16:04.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.549+0000 7fd7bf212b80 1 -- 172.21.15.64:0/365932966 shutdown_connections 2023-09-13T03:16:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.549+0000 7fd7bf212b80 1 -- 172.21.15.64:0/365932966 wait complete. 2023-09-13T03:16:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.549+0000 7fd7bf212b80 20 client.16317 trim_cache size 0 max 16384 2023-09-13T03:16:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.564+0000 7fd7bf212b80 1 Processor -- start 2023-09-13T03:16:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.564+0000 7fd7bf212b80 1 -- start start 2023-09-13T03:16:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.564+0000 7fd7bf212b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b5a421300 0x558b5a432530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.564+0000 7fd7bf212b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b5a421cc0 0x558b5a41bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.564+0000 7fd7bf212b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b5a4271a0 0x558b5a42c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.564+0000 7fd7bf212b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b5a413f50 con 0x558b5a421cc0 2023-09-13T03:16:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.564+0000 7fd7bf212b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b5a425a80 con 0x558b5a4271a0 2023-09-13T03:16:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.564+0000 7fd7bf212b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b5a422580 con 0x558b5a421300 2023-09-13T03:16:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.565+0000 7fd7a8d9a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b5a421cc0 0x558b5a41bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.565+0000 7fd7a8d9a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b5a421cc0 0x558b5a41bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44460/0 (socket says 172.21.15.64:44460) 2023-09-13T03:16:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.565+0000 7fd7a9d9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b5a4271a0 0x558b5a42c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.565+0000 7fd7a8d9a700 1 -- 172.21.15.64:0/4076326692 learned_addr learned my addr 172.21.15.64:0/4076326692 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:16:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.565+0000 7fd7a959b700 1 --2- 172.21.15.64:0/4076326692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b5a421300 0x558b5a432530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.565+0000 7fd7a8d9a700 1 -- 172.21.15.64:0/4076326692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b5a421300 msgr2=0x558b5a432530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:16:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.565+0000 7fd7a8d9a700 1 --2- 172.21.15.64:0/4076326692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b5a421300 0x558b5a432530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.565+0000 7fd7a8d9a700 1 -- 172.21.15.64:0/4076326692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b5a4271a0 msgr2=0x558b5a42c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:16:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.565+0000 7fd7a8d9a700 1 --2- 172.21.15.64:0/4076326692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b5a4271a0 0x558b5a42c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.565+0000 7fd7a8d9a700 1 -- 172.21.15.64:0/4076326692 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b5a1c61f0 con 0x558b5a421cc0 2023-09-13T03:16:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.565+0000 7fd7a8d9a700 1 --2- 172.21.15.64:0/4076326692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b5a421cc0 0x558b5a41bf20 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd794012140 tx=0x7fd794009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2835b01598a09342 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:16:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.566+0000 7fd78ffff700 1 -- 172.21.15.64:0/4076326692 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7940134c0 con 0x558b5a421cc0 2023-09-13T03:16:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.566+0000 7fd78ffff700 1 -- 172.21.15.64:0/4076326692 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd79400fe40 con 0x558b5a421cc0 2023-09-13T03:16:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.566+0000 7fd78ffff700 1 -- 172.21.15.64:0/4076326692 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd794003b30 con 0x558b5a421cc0 2023-09-13T03:16:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.566+0000 7fd7bf212b80 1 -- 172.21.15.64:0/4076326692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b5a421cc0 msgr2=0x558b5a41bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.566+0000 7fd7bf212b80 1 --2- 172.21.15.64:0/4076326692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b5a421cc0 0x558b5a41bf20 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd794012140 tx=0x7fd794009c20 comp rx=0 tx=0).stop 2023-09-13T03:16:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.566+0000 7fd7bf212b80 1 -- 172.21.15.64:0/4076326692 shutdown_connections 2023-09-13T03:16:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.566+0000 7fd7bf212b80 1 --2- 172.21.15.64:0/4076326692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b5a4271a0 0x558b5a42c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.566+0000 7fd7bf212b80 1 --2- 172.21.15.64:0/4076326692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b5a421300 0x558b5a432530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.566+0000 7fd7bf212b80 1 --2- 172.21.15.64:0/4076326692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b5a421cc0 0x558b5a41bf20 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.566+0000 7fd7bf212b80 1 -- 172.21.15.64:0/4076326692 >> 172.21.15.64:0/4076326692 conn(0x558b5a396030 msgr2=0x558b5a426050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:16:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.566+0000 7fd7bf212b80 1 -- 172.21.15.64:0/4076326692 shutdown_connections 2023-09-13T03:16:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.566+0000 7fd7bf212b80 1 -- 172.21.15.64:0/4076326692 wait complete. 2023-09-13T03:16:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.567+0000 7fd7bf212b80 1 -- start start 2023-09-13T03:16:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.567+0000 7fd7bf212b80 1 Processor -- start 2023-09-13T03:16:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.567+0000 7fd7bf212b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b5a421300 0x558b5a4875f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.567+0000 7fd7bf212b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b5a421cc0 0x558b5a487b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.567+0000 7fd7bf212b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b5a4271a0 0x558b5a483230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.567+0000 7fd7bf212b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b5a483940 con 0x558b5a421300 2023-09-13T03:16:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.567+0000 7fd7bf212b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b5a483ab0 con 0x558b5a4271a0 2023-09-13T03:16:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.567+0000 7fd7bf212b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b5a483c70 con 0x558b5a421cc0 2023-09-13T03:16:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.568+0000 7fd7bf212b80 10 client.0 _ll_register_callbacks cb 0x7fd794004350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:16:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.568+0000 7fd7a959b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b5a421300 0x558b5a4875f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.568+0000 7fd7a959b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b5a421300 0x558b5a4875f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44474/0 (socket says 172.21.15.64:44474) 2023-09-13T03:16:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.568+0000 7fd7a959b700 1 -- 172.21.15.64:0/2987555242 learned_addr learned my addr 172.21.15.64:0/2987555242 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:16:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.568+0000 7fd7a8d9a700 1 --2- 172.21.15.64:0/2987555242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b5a421cc0 0x558b5a487b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.568+0000 7fd7a9d9c700 1 --2- 172.21.15.64:0/2987555242 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b5a4271a0 0x558b5a483230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.568+0000 7fd7a959b700 1 -- 172.21.15.64:0/2987555242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b5a421cc0 msgr2=0x558b5a487b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:16:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.568+0000 7fd7a959b700 1 --2- 172.21.15.64:0/2987555242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b5a421cc0 0x558b5a487b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.568+0000 7fd7a959b700 1 -- 172.21.15.64:0/2987555242 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b5a4271a0 msgr2=0x558b5a483230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:16:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.568+0000 7fd7a959b700 1 --2- 172.21.15.64:0/2987555242 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b5a4271a0 0x558b5a483230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.568+0000 7fd7a959b700 1 -- 172.21.15.64:0/2987555242 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b5a188c00 con 0x558b5a421300 2023-09-13T03:16:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.568+0000 7fd7a9d9c700 1 --2- 172.21.15.64:0/2987555242 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b5a4271a0 0x558b5a483230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:16:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.568+0000 7fd7a959b700 1 --2- 172.21.15.64:0/2987555242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b5a421300 0x558b5a4875f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0011f70 tx=0x7fd7a0011770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d4c61f56e36475a server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:16:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.568+0000 7fd78cff9700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:16:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.569+0000 7fd78cff9700 1 -- 172.21.15.64:0/2987555242 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a002c040 con 0x558b5a421300 2023-09-13T03:16:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.569+0000 7fd78cff9700 1 -- 172.21.15.64:0/2987555242 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd7a0019620 con 0x558b5a421300 2023-09-13T03:16:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.569+0000 7fd7bf212b80 1 -- 172.21.15.64:0/2987555242 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7a0019a30 con 0x558b5a421300 2023-09-13T03:16:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.569+0000 7fd78cff9700 1 -- 172.21.15.64:0/2987555242 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7a00197e0 con 0x558b5a421300 2023-09-13T03:16:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.569+0000 7fd7bf212b80 10 client.16323 fetch_fsmap learned FSMap version 34 2023-09-13T03:16:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.569+0000 7fd7bf212b80 1 -- 172.21.15.64:0/2987555242 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x558b5a19e250 con 0x558b5a421300 2023-09-13T03:16:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.569+0000 7fd78cff9700 1 -- 172.21.15.64:0/2987555242 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7a003e020 con 0x558b5a421300 2023-09-13T03:16:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.569+0000 7fd78cff9700 1 -- 172.21.15.64:0/2987555242 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 34) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fd7a0003ac0 con 0x558b5a421300 2023-09-13T03:16:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.569+0000 7fd7bf212b80 10 client.16323 fetch_fsmap finished waiting for FSMap version 34 2023-09-13T03:16:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.569+0000 7fd7bf212b80 10 client.16323 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:16:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.569+0000 7fd7bf212b80 20 client.16323 populate_metadata read hostname 'smithi064' 2023-09-13T03:16:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.569+0000 7fd7bf212b80 1 --2- 172.21.15.64:0/2987555242 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x558b5a490600 0x558b5a492a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.570+0000 7fd7bf212b80 4 client.16323 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:16:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.570+0000 7fd7bf212b80 1 -- 172.21.15.64:0/2987555242 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558b5a36c8c0 con 0x558b5a490600 2023-09-13T03:16:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.570+0000 7fd7a8d9a700 1 --2- 172.21.15.64:0/2987555242 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x558b5a490600 0x558b5a492a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.570+0000 7fd7a8d9a700 1 --2- 172.21.15.64:0/2987555242 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x558b5a490600 0x558b5a492a20 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4139a05557b52112 server_cookie=7350b0a99fbb8c06 in_seq=0 out_seq=0 2023-09-13T03:16:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.570+0000 7fd78cff9700 10 client.16323 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:16:04.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.596+0000 7fd78cff9700 1 -- 172.21.15.64:0/2987555242 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558b5a36c8c0 con 0x558b5a490600 2023-09-13T03:16:04.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.596+0000 7fd78cff9700 10 client.16323 handle_command_reply: tid=0 2023-09-13T03:16:04.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.596+0000 7fd7bf212b80 1 client.16323 shutdown 2023-09-13T03:16:04.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.597+0000 7fd7bf212b80 1 -- 172.21.15.64:0/2987555242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b5a421300 msgr2=0x558b5a4875f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:04.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.597+0000 7fd7bf212b80 1 --2- 172.21.15.64:0/2987555242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b5a421300 0x558b5a4875f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0011f70 tx=0x7fd7a0011770 comp rx=0 tx=0).stop 2023-09-13T03:16:04.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.597+0000 7fd7bf212b80 1 -- 172.21.15.64:0/2987555242 shutdown_connections 2023-09-13T03:16:04.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.597+0000 7fd7bf212b80 1 --2- 172.21.15.64:0/2987555242 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b5a4271a0 0x558b5a483230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:04.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.597+0000 7fd7bf212b80 1 --2- 172.21.15.64:0/2987555242 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x558b5a490600 0x558b5a492a20 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:04.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.597+0000 7fd7bf212b80 1 --2- 172.21.15.64:0/2987555242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b5a421300 0x558b5a4875f0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:04.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.597+0000 7fd7bf212b80 1 --2- 172.21.15.64:0/2987555242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b5a421cc0 0x558b5a487b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.597+0000 7fd7bf212b80 1 -- 172.21.15.64:0/2987555242 >> 172.21.15.64:0/2987555242 conn(0x558b5a396030 msgr2=0x558b5a4202f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:16:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.597+0000 7fd78cff9700 0 client.16323 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:16:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.597+0000 7fd7bf212b80 1 -- 172.21.15.64:0/2987555242 shutdown_connections 2023-09-13T03:16:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.597+0000 7fd7bf212b80 1 -- 172.21.15.64:0/2987555242 wait complete. 2023-09-13T03:16:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.597+0000 7fd7bf212b80 20 client.16323 trim_cache size 0 max 16384 2023-09-13T03:16:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:16:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:16:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:16:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:16:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.598+0000 7fd7ab7ff700 1 -- 172.21.15.64:0/2709874383 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fd7900a0830 msgr2=0x7fd7900a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.598+0000 7fd7ab7ff700 1 --2- 172.21.15.64:0/2709874383 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fd7900a0830 0x7fd7900a2cf0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd794009fd0 tx=0x7fd794013c30 comp rx=0 tx=0).stop 2023-09-13T03:16:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.598+0000 7fd7ab7ff700 1 -- 172.21.15.64:0/2709874383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7a410ce20 msgr2=0x7fd7a4198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.598+0000 7fd7ab7ff700 1 --2- 172.21.15.64:0/2709874383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7a410ce20 0x7fd7a4198840 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd79c009fd0 tx=0x7fd79c007930 comp rx=0 tx=0).stop 2023-09-13T03:16:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.598+0000 7fd7ab7ff700 1 -- 172.21.15.64:0/2709874383 shutdown_connections 2023-09-13T03:16:04.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.598+0000 7fd7ab7ff700 1 --2- 172.21.15.64:0/2709874383 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fd7900a0830 0x7fd7900a2cf0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:04.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.598+0000 7fd7ab7ff700 1 --2- 172.21.15.64:0/2709874383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7a40729d0 0x7fd7a419e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:04.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.598+0000 7fd7ab7ff700 1 --2- 172.21.15.64:0/2709874383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7a410ce20 0x7fd7a4198840 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:04.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.598+0000 7fd7ab7ff700 1 --2- 172.21.15.64:0/2709874383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7a40732f0 0x7fd7a419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:04.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.598+0000 7fd7ab7ff700 1 -- 172.21.15.64:0/2709874383 >> 172.21.15.64:0/2709874383 conn(0x7fd7a40fc050 msgr2=0x7fd7a4104880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:16:04.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.598+0000 7fd7ab7ff700 1 -- 172.21.15.64:0/2709874383 shutdown_connections 2023-09-13T03:16:04.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:04.598+0000 7fd7ab7ff700 1 -- 172.21.15.64:0/2709874383 wait complete. 2023-09-13T03:16:04.899 INFO:tasks.workunit.client.0.smithi064.stdout: 1 57503 2.11 MB/sec execute 9 sec latency 1328.649 ms 2023-09-13T03:16:04.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:16:04 smithi170 ceph-mon[134328]: pgmap v90: 97 pgs: 97 active+clean; 35 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 248 KiB/s wr, 69 op/s 2023-09-13T03:16:04.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:16:04 smithi064 ceph-mon[97675]: pgmap v90: 97 pgs: 97 active+clean; 35 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 248 KiB/s wr, 69 op/s 2023-09-13T03:16:04.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:16:04 smithi142 ceph-mon[134082]: pgmap v90: 97 pgs: 97 active+clean; 35 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 248 KiB/s wr, 69 op/s 2023-09-13T03:16:05.899 INFO:tasks.workunit.client.0.smithi064.stdout: 1 57503 1.90 MB/sec execute 10 sec latency 2328.699 ms 2023-09-13T03:16:06.899 INFO:tasks.workunit.client.0.smithi064.stdout: 1 57503 1.72 MB/sec execute 11 sec latency 3328.825 ms 2023-09-13T03:16:06.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:16:06 smithi170 ceph-mon[134328]: pgmap v91: 97 pgs: 97 active+clean; 35 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 305 KiB/s wr, 88 op/s 2023-09-13T03:16:06.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:16:06 smithi064 ceph-mon[97675]: pgmap v91: 97 pgs: 97 active+clean; 35 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 305 KiB/s wr, 88 op/s 2023-09-13T03:16:06.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:16:06 smithi142 ceph-mon[134082]: pgmap v91: 97 pgs: 97 active+clean; 35 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 305 KiB/s wr, 88 op/s 2023-09-13T03:16:07.899 INFO:tasks.workunit.client.0.smithi064.stdout: 1 57683 1.62 MB/sec execute 12 sec latency 4208.898 ms 2023-09-13T03:16:08.899 INFO:tasks.workunit.client.0.smithi064.stdout: 1 59776 2.41 MB/sec execute 13 sec latency 54.154 ms 2023-09-13T03:16:08.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:16:08 smithi170 ceph-mon[134328]: pgmap v92: 97 pgs: 97 active+clean; 35 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 273 KiB/s wr, 70 op/s 2023-09-13T03:16:08.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:16:08 smithi064 ceph-mon[97675]: pgmap v92: 97 pgs: 97 active+clean; 35 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 273 KiB/s wr, 70 op/s 2023-09-13T03:16:08.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:16:08 smithi142 ceph-mon[134082]: pgmap v92: 97 pgs: 97 active+clean; 35 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 273 KiB/s wr, 70 op/s 2023-09-13T03:16:09.899 INFO:tasks.workunit.client.0.smithi064.stdout: 1 60995 2.71 MB/sec execute 14 sec latency 107.859 ms 2023-09-13T03:16:10.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:16:10 smithi170 ceph-mon[134328]: pgmap v93: 97 pgs: 97 active+clean; 35 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 273 KiB/s wr, 70 op/s 2023-09-13T03:16:10.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:16:10 smithi064 ceph-mon[97675]: pgmap v93: 97 pgs: 97 active+clean; 35 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 273 KiB/s wr, 70 op/s 2023-09-13T03:16:10.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:16:10 smithi142 ceph-mon[134082]: pgmap v93: 97 pgs: 97 active+clean; 35 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 273 KiB/s wr, 70 op/s 2023-09-13T03:16:10.899 INFO:tasks.workunit.client.0.smithi064.stdout: 1 60995 2.53 MB/sec execute 15 sec latency 1107.962 ms 2023-09-13T03:16:11.899 INFO:tasks.workunit.client.0.smithi064.stdout: 1 60995 2.37 MB/sec execute 16 sec latency 2108.039 ms 2023-09-13T03:16:12.899 INFO:tasks.workunit.client.0.smithi064.stdout: 1 60995 2.23 MB/sec execute 17 sec latency 3108.111 ms 2023-09-13T03:16:12.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:16:12 smithi170 ceph-mon[134328]: pgmap v94: 97 pgs: 97 active+clean; 39 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 134 op/s 2023-09-13T03:16:12.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:16:12 smithi064 ceph-mon[97675]: pgmap v94: 97 pgs: 97 active+clean; 39 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 134 op/s 2023-09-13T03:16:12.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:16:12 smithi142 ceph-mon[134082]: pgmap v94: 97 pgs: 97 active+clean; 39 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 134 op/s 2023-09-13T03:16:13.899 INFO:tasks.workunit.client.0.smithi064.stdout: 1 61119 2.12 MB/sec execute 18 sec latency 3730.218 ms 2023-09-13T03:16:14.900 INFO:tasks.workunit.client.0.smithi064.stdout: 1 61119 2.01 MB/sec execute 19 sec latency 1277.189 ms 2023-09-13T03:16:14.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:16:14 smithi170 ceph-mon[134328]: pgmap v95: 97 pgs: 97 active+clean; 39 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 887 KiB/s wr, 83 op/s 2023-09-13T03:16:14.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:16:14 smithi064 ceph-mon[97675]: pgmap v95: 97 pgs: 97 active+clean; 39 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 887 KiB/s wr, 83 op/s 2023-09-13T03:16:14.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:16:14 smithi142 ceph-mon[134082]: pgmap v95: 97 pgs: 97 active+clean; 39 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 887 KiB/s wr, 83 op/s 2023-09-13T03:16:15.900 INFO:tasks.workunit.client.0.smithi064.stdout: 1 61119 1.91 MB/sec execute 20 sec latency 2277.275 ms 2023-09-13T03:16:16.902 INFO:tasks.workunit.client.0.smithi064.stdout: 1 61119 1.82 MB/sec execute 21 sec latency 3277.376 ms 2023-09-13T03:16:16.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:16:16 smithi170 ceph-mon[134328]: pgmap v96: 97 pgs: 97 active+clean; 40 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 932 KiB/s wr, 95 op/s 2023-09-13T03:16:16.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:16:16 smithi064 ceph-mon[97675]: pgmap v96: 97 pgs: 97 active+clean; 40 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 932 KiB/s wr, 95 op/s 2023-09-13T03:16:16.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:16:16 smithi142 ceph-mon[134082]: pgmap v96: 97 pgs: 97 active+clean; 40 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 932 KiB/s wr, 95 op/s 2023-09-13T03:16:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.762+0000 7f326ffff700 1 -- 172.21.15.64:0/175163281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32681005a0 msgr2=0x7f3268100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.762+0000 7f326ffff700 1 --2- 172.21.15.64:0/175163281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32681005a0 0x7f3268100a20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3258009920 tx=0x7f3258009c30 comp rx=0 tx=0).stop 2023-09-13T03:16:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.762+0000 7f326ffff700 1 -- 172.21.15.64:0/175163281 shutdown_connections 2023-09-13T03:16:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.762+0000 7f326ffff700 1 --2- 172.21.15.64:0/175163281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3268100f60 0x7f326810d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.762+0000 7f326ffff700 1 --2- 172.21.15.64:0/175163281 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3268106660 0x7f3268106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.762+0000 7f326ffff700 1 --2- 172.21.15.64:0/175163281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32681005a0 0x7f3268100a20 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.762+0000 7f326ffff700 1 -- 172.21.15.64:0/175163281 >> 172.21.15.64:0/175163281 conn(0x7f32680fc050 msgr2=0x7f32680fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:16:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.762+0000 7f326ffff700 1 -- 172.21.15.64:0/175163281 shutdown_connections 2023-09-13T03:16:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.762+0000 7f326ffff700 1 -- 172.21.15.64:0/175163281 wait complete. 2023-09-13T03:16:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.764+0000 7f326ffff700 1 Processor -- start 2023-09-13T03:16:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.764+0000 7f326ffff700 1 -- start start 2023-09-13T03:16:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.764+0000 7f326ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32681005a0 0x7f326819c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.765+0000 7f326ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3268100f60 0x7f326819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.765+0000 7f326ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3268106660 0x7f3268196550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.765+0000 7f326ffff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3268196bc0 con 0x7f32681005a0 2023-09-13T03:16:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.765+0000 7f326ffff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3268196d30 con 0x7f3268106660 2023-09-13T03:16:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.765+0000 7f326ffff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3268196ef0 con 0x7f3268100f60 2023-09-13T03:16:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.765+0000 7f326dd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32681005a0 0x7f326819c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.765+0000 7f326dd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32681005a0 0x7f326819c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34110/0 (socket says 172.21.15.64:34110) 2023-09-13T03:16:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.765+0000 7f326dd9b700 1 -- 172.21.15.64:0/4266511296 learned_addr learned my addr 172.21.15.64:0/4266511296 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:16:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.765+0000 7f326d59a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3268100f60 0x7f326819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.765+0000 7f326e59c700 1 --2- 172.21.15.64:0/4266511296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3268106660 0x7f3268196550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.765+0000 7f326dd9b700 1 -- 172.21.15.64:0/4266511296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3268100f60 msgr2=0x7f326819cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.765+0000 7f326dd9b700 1 --2- 172.21.15.64:0/4266511296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3268100f60 0x7f326819cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.765+0000 7f326dd9b700 1 -- 172.21.15.64:0/4266511296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3268106660 msgr2=0x7f3268196550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.765+0000 7f326dd9b700 1 --2- 172.21.15.64:0/4266511296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3268106660 0x7f3268196550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.765+0000 7f326dd9b700 1 -- 172.21.15.64:0/4266511296 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3260009d00 con 0x7f32681005a0 2023-09-13T03:16:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.765+0000 7f326d59a700 1 --2- 172.21.15.64:0/4266511296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3268100f60 0x7f326819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:16:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.766+0000 7f326dd9b700 1 --2- 172.21.15.64:0/4266511296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32681005a0 0x7f326819c6a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f326400e9a0 tx=0x7f326400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:16:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.766+0000 7f325effd700 1 -- 172.21.15.64:0/4266511296 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3264009790 con 0x7f32681005a0 2023-09-13T03:16:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.766+0000 7f326ffff700 1 -- 172.21.15.64:0/4266511296 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3258009520 con 0x7f32681005a0 2023-09-13T03:16:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.766+0000 7f325effd700 1 -- 172.21.15.64:0/4266511296 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3264010430 con 0x7f32681005a0 2023-09-13T03:16:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.766+0000 7f325effd700 1 -- 172.21.15.64:0/4266511296 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3264018680 con 0x7f32681005a0 2023-09-13T03:16:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.766+0000 7f326ffff700 1 -- 172.21.15.64:0/4266511296 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32681973c0 con 0x7f32681005a0 2023-09-13T03:16:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.769+0000 7f325effd700 1 -- 172.21.15.64:0/4266511296 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f32640105a0 con 0x7f32681005a0 2023-09-13T03:16:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.770+0000 7f328393fb80 1 Processor -- start 2023-09-13T03:16:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.770+0000 7f325effd700 1 --2- 172.21.15.64:0/4266511296 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f32540a07b0 0x7f32540a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.771+0000 7f325effd700 1 -- 172.21.15.64:0/4266511296 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f3264014070 con 0x7f32681005a0 2023-09-13T03:16:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.771+0000 7f328393fb80 1 -- start start 2023-09-13T03:16:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.771+0000 7f326d59a700 1 --2- 172.21.15.64:0/4266511296 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f32540a07b0 0x7f32540a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.771+0000 7f328393fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7dae805b0 0x55c7dae80990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.771+0000 7f328393fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7dae7a710 0x55c7dae7ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.771+0000 7f328393fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7dae7b0d0 0x55c7dae87340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.771+0000 7f328393fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7dadcf6d0 con 0x55c7dae7b0d0 2023-09-13T03:16:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.771+0000 7f328393fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7dadccff0 con 0x55c7dae7a710 2023-09-13T03:16:17.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.771+0000 7f328393fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7dadc88e0 con 0x55c7dae805b0 2023-09-13T03:16:17.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.771+0000 7f326dd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7dae805b0 0x55c7dae80990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:17.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.771+0000 7f326d59a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7dae7a710 0x55c7dae7ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:17.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.771+0000 7f326e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7dae7b0d0 0x55c7dae87340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:17.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.772+0000 7f326dd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7dae805b0 0x55c7dae80990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:49166/0 (socket says 172.21.15.64:49166) 2023-09-13T03:16:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.772+0000 7f326dd9b700 1 -- 172.21.15.64:0/1558004188 learned_addr learned my addr 172.21.15.64:0/1558004188 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:16:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.772+0000 7f326e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7dae7b0d0 0x55c7dae87340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34120/0 (socket says 172.21.15.64:34120) 2023-09-13T03:16:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.772+0000 7f326d59a700 1 --2- 172.21.15.64:0/4266511296 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f32540a07b0 0x7f32540a2c70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f325800c010 tx=0x7f3258019040 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:16:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.772+0000 7f326dd9b700 1 -- 172.21.15.64:0/1558004188 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7dae7a710 msgr2=0x55c7dae7ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:16:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.772+0000 7f326dd9b700 1 --2- 172.21.15.64:0/1558004188 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7dae7a710 0x55c7dae7ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.772+0000 7f326dd9b700 1 -- 172.21.15.64:0/1558004188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7dae7b0d0 msgr2=0x55c7dae87340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:16:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.772+0000 7f326dd9b700 1 --2- 172.21.15.64:0/1558004188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7dae7b0d0 0x55c7dae87340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.772+0000 7f326dd9b700 1 -- 172.21.15.64:0/1558004188 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7dabf4d80 con 0x55c7dae805b0 2023-09-13T03:16:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.772+0000 7f326e59c700 1 --2- 172.21.15.64:0/1558004188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7dae7b0d0 0x55c7dae87340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:16:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.772+0000 7f326dd9b700 1 --2- 172.21.15.64:0/1558004188 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7dae805b0 0x55c7dae80990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f32640959a0 tx=0x7f3264098e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fd34fb934e20ef4 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:16:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.772+0000 7f3253fff700 1 -- 172.21.15.64:0/1558004188 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326409d2d0 con 0x55c7dae805b0 2023-09-13T03:16:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.773+0000 7f3253fff700 1 -- 172.21.15.64:0/1558004188 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32640c9e90 con 0x55c7dae805b0 2023-09-13T03:16:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.773+0000 7f3253fff700 1 -- 172.21.15.64:0/1558004188 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3264099940 con 0x55c7dae805b0 2023-09-13T03:16:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.773+0000 7f328393fb80 1 -- 172.21.15.64:0/1558004188 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7dae805b0 msgr2=0x55c7dae80990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.773+0000 7f328393fb80 1 --2- 172.21.15.64:0/1558004188 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7dae805b0 0x55c7dae80990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f32640959a0 tx=0x7f3264098e80 comp rx=0 tx=0).stop 2023-09-13T03:16:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.773+0000 7f328393fb80 1 -- 172.21.15.64:0/1558004188 shutdown_connections 2023-09-13T03:16:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.773+0000 7f328393fb80 1 --2- 172.21.15.64:0/1558004188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7dae7b0d0 0x55c7dae87340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.773+0000 7f328393fb80 1 --2- 172.21.15.64:0/1558004188 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7dae805b0 0x55c7dae80990 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.773+0000 7f328393fb80 1 --2- 172.21.15.64:0/1558004188 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7dae7a710 0x55c7dae7ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.773+0000 7f328393fb80 1 -- 172.21.15.64:0/1558004188 >> 172.21.15.64:0/1558004188 conn(0x55c7dadeb030 msgr2=0x55c7dae71560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:16:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.773+0000 7f328393fb80 1 -- 172.21.15.64:0/1558004188 shutdown_connections 2023-09-13T03:16:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.773+0000 7f328393fb80 1 -- 172.21.15.64:0/1558004188 wait complete. 2023-09-13T03:16:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.774+0000 7f328393fb80 1 -- start start 2023-09-13T03:16:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.774+0000 7f328393fb80 1 Processor -- start 2023-09-13T03:16:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.775+0000 7f328393fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7dae7a710 0x55c7dae87200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.775+0000 7f328393fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7dae7b0d0 0x55c7dae84520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.775+0000 7f328393fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7dae805b0 0x55c7dae84b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.775+0000 7f328393fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7dad66b10 con 0x55c7dae7a710 2023-09-13T03:16:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.775+0000 7f328393fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7dad66810 con 0x55c7dae805b0 2023-09-13T03:16:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.775+0000 7f328393fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7dac4a8c0 con 0x55c7dae7b0d0 2023-09-13T03:16:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.775+0000 7f328393fb80 10 client.0 _ll_register_callbacks cb 0x55c7dacb0a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:16:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.775+0000 7f326dd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7dae7a710 0x55c7dae87200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.775+0000 7f326d59a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7dae7b0d0 0x55c7dae84520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.775+0000 7f326dd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7dae7a710 0x55c7dae87200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34122/0 (socket says 172.21.15.64:34122) 2023-09-13T03:16:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.775+0000 7f326dd9b700 1 -- 172.21.15.64:0/4174150894 learned_addr learned my addr 172.21.15.64:0/4174150894 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:16:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.775+0000 7f326d59a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7dae7b0d0 0x55c7dae84520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:49178/0 (socket says 172.21.15.64:49178) 2023-09-13T03:16:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.775+0000 7f326e59c700 1 --2- 172.21.15.64:0/4174150894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7dae805b0 0x55c7dae84b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.775+0000 7f326d59a700 1 -- 172.21.15.64:0/4174150894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7dae805b0 msgr2=0x55c7dae84b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:16:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.775+0000 7f326d59a700 1 --2- 172.21.15.64:0/4174150894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7dae805b0 0x55c7dae84b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.775+0000 7f326d59a700 1 -- 172.21.15.64:0/4174150894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7dae7a710 msgr2=0x55c7dae87200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:16:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.775+0000 7f326d59a700 1 --2- 172.21.15.64:0/4174150894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7dae7a710 0x55c7dae87200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.775+0000 7f326d59a700 1 -- 172.21.15.64:0/4174150894 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3260009160 con 0x55c7dae7b0d0 2023-09-13T03:16:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.775+0000 7f326dd9b700 1 --2- 172.21.15.64:0/4174150894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7dae7a710 0x55c7dae87200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:16:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.776+0000 7f326d59a700 1 --2- 172.21.15.64:0/4174150894 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7dae7b0d0 0x55c7dae84520 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f32580098f0 tx=0x7f3258009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a46b45756b8fa8a9 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:16:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.776+0000 7f32517fa700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:16:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.776+0000 7f32517fa700 1 -- 172.21.15.64:0/4174150894 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325800ba70 con 0x55c7dae7b0d0 2023-09-13T03:16:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.776+0000 7f328393fb80 1 -- 172.21.15.64:0/4174150894 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c7dad20490 con 0x55c7dae7b0d0 2023-09-13T03:16:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.776+0000 7f32517fa700 1 -- 172.21.15.64:0/4174150894 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f3258028040 con 0x55c7dae7b0d0 2023-09-13T03:16:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.776+0000 7f32517fa700 1 -- 172.21.15.64:0/4174150894 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f325800bbd0 con 0x55c7dae7b0d0 2023-09-13T03:16:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.776+0000 7f328393fb80 10 client.26374 fetch_fsmap learned FSMap version 34 2023-09-13T03:16:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.776+0000 7f328393fb80 1 -- 172.21.15.64:0/4174150894 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7f3258009520 con 0x55c7dae7b0d0 2023-09-13T03:16:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.777+0000 7f32517fa700 1 -- 172.21.15.64:0/4174150894 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c7dad20490 con 0x55c7dae7b0d0 2023-09-13T03:16:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.777+0000 7f32517fa700 1 -- 172.21.15.64:0/4174150894 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 34) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f325802d760 con 0x55c7dae7b0d0 2023-09-13T03:16:17.792 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.777+0000 7f328393fb80 10 client.26374 fetch_fsmap finished waiting for FSMap version 34 2023-09-13T03:16:17.792 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.777+0000 7f328393fb80 10 client.26374 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:16:17.792 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.777+0000 7f328393fb80 20 client.26374 populate_metadata read hostname 'smithi064' 2023-09-13T03:16:17.792 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.777+0000 7f328393fb80 1 --2- 172.21.15.64:0/4174150894 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55c7dae94790 0x55c7dae96bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:17.792 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.777+0000 7f328393fb80 4 client.26374 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:16:17.792 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.777+0000 7f328393fb80 1 -- 172.21.15.64:0/4174150894 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c7daddd840 con 0x55c7dae94790 2023-09-13T03:16:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.777+0000 7f326dd9b700 1 --2- 172.21.15.64:0/4174150894 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55c7dae94790 0x55c7dae96bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.778+0000 7f326dd9b700 1 --2- 172.21.15.64:0/4174150894 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55c7dae94790 0x55c7dae96bb0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2d807d9be7a9e40 server_cookie=582f4f5f334c3129 in_seq=0 out_seq=0 2023-09-13T03:16:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.778+0000 7f32517fa700 10 client.26374 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:16:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.779+0000 7f32517fa700 1 -- 172.21.15.64:0/4174150894 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c7daddd840 con 0x55c7dae94790 2023-09-13T03:16:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.779+0000 7f32517fa700 10 client.26374 handle_command_reply: tid=0 2023-09-13T03:16:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.779+0000 7f328393fb80 1 client.26374 shutdown 2023-09-13T03:16:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.779+0000 7f328393fb80 1 -- 172.21.15.64:0/4174150894 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7dae7b0d0 msgr2=0x55c7dae84520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.779+0000 7f328393fb80 1 --2- 172.21.15.64:0/4174150894 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7dae7b0d0 0x55c7dae84520 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f32580098f0 tx=0x7f3258009c30 comp rx=0 tx=0).stop 2023-09-13T03:16:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.779+0000 7f328393fb80 1 -- 172.21.15.64:0/4174150894 shutdown_connections 2023-09-13T03:16:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.779+0000 7f328393fb80 1 --2- 172.21.15.64:0/4174150894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7dae805b0 0x55c7dae84b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.779+0000 7f328393fb80 1 --2- 172.21.15.64:0/4174150894 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55c7dae94790 0x55c7dae96bb0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.779+0000 7f328393fb80 1 --2- 172.21.15.64:0/4174150894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7dae7a710 0x55c7dae87200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.779+0000 7f32517fa700 0 client.26374 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:16:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.779+0000 7f328393fb80 1 --2- 172.21.15.64:0/4174150894 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7dae7b0d0 0x55c7dae84520 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.779+0000 7f328393fb80 1 -- 172.21.15.64:0/4174150894 >> 172.21.15.64:0/4174150894 conn(0x55c7dadeb030 msgr2=0x55c7dae707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:16:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.779+0000 7f328393fb80 1 -- 172.21.15.64:0/4174150894 shutdown_connections 2023-09-13T03:16:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.779+0000 7f328393fb80 1 -- 172.21.15.64:0/4174150894 wait complete. 2023-09-13T03:16:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.779+0000 7f328393fb80 20 client.26374 trim_cache size 0 max 16384 2023-09-13T03:16:17.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.795+0000 7f328393fb80 1 Processor -- start 2023-09-13T03:16:17.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.795+0000 7f328393fb80 1 -- start start 2023-09-13T03:16:17.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.795+0000 7f328393fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7dae7a710 0x55c7dae83ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:17.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.795+0000 7f328393fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7dae7b0d0 0x55c7dae70f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:17.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.795+0000 7f328393fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7dae805b0 0x55c7dae86d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:17.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.795+0000 7f328393fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7daddd840 con 0x55c7dae7b0d0 2023-09-13T03:16:17.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.795+0000 7f328393fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7dad20490 con 0x55c7dae805b0 2023-09-13T03:16:17.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.795+0000 7f328393fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7dae6dfc0 con 0x55c7dae7a710 2023-09-13T03:16:17.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.795+0000 7f326dd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7dae7a710 0x55c7dae83ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:17.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.795+0000 7f326d59a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7dae7b0d0 0x55c7dae70f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:17.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.795+0000 7f326dd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7dae7a710 0x55c7dae83ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:49194/0 (socket says 172.21.15.64:49194) 2023-09-13T03:16:17.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.795+0000 7f326dd9b700 1 -- 172.21.15.64:0/1851815378 learned_addr learned my addr 172.21.15.64:0/1851815378 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:16:17.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.795+0000 7f326e59c700 1 --2- 172.21.15.64:0/1851815378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7dae805b0 0x55c7dae86d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:17.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.796+0000 7f326dd9b700 1 -- 172.21.15.64:0/1851815378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7dae805b0 msgr2=0x55c7dae86d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:16:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.796+0000 7f326dd9b700 1 --2- 172.21.15.64:0/1851815378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7dae805b0 0x55c7dae86d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.796+0000 7f326dd9b700 1 -- 172.21.15.64:0/1851815378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7dae7b0d0 msgr2=0x55c7dae70f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:16:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.796+0000 7f326dd9b700 1 --2- 172.21.15.64:0/1851815378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7dae7b0d0 0x55c7dae70f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.796+0000 7f326dd9b700 1 -- 172.21.15.64:0/1851815378 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7dabddc00 con 0x55c7dae7a710 2023-09-13T03:16:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.796+0000 7f326dd9b700 1 --2- 172.21.15.64:0/1851815378 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7dae7a710 0x55c7dae83ce0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3264098f70 tx=0x7f32640ae040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf67d9f91892d236 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:16:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.796+0000 7f325cff9700 1 -- 172.21.15.64:0/1851815378 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32640a1590 con 0x55c7dae7a710 2023-09-13T03:16:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.796+0000 7f325cff9700 1 -- 172.21.15.64:0/1851815378 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f326409d2d0 con 0x55c7dae7a710 2023-09-13T03:16:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.796+0000 7f325cff9700 1 -- 172.21.15.64:0/1851815378 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3264099960 con 0x55c7dae7a710 2023-09-13T03:16:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.797+0000 7f328393fb80 1 -- 172.21.15.64:0/1851815378 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7dae7a710 msgr2=0x55c7dae83ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.797+0000 7f328393fb80 1 --2- 172.21.15.64:0/1851815378 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7dae7a710 0x55c7dae83ce0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3264098f70 tx=0x7f32640ae040 comp rx=0 tx=0).stop 2023-09-13T03:16:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.797+0000 7f328393fb80 1 -- 172.21.15.64:0/1851815378 shutdown_connections 2023-09-13T03:16:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.797+0000 7f328393fb80 1 --2- 172.21.15.64:0/1851815378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7dae805b0 0x55c7dae86d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.797+0000 7f328393fb80 1 --2- 172.21.15.64:0/1851815378 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7dae7a710 0x55c7dae83ce0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.797+0000 7f328393fb80 1 --2- 172.21.15.64:0/1851815378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7dae7b0d0 0x55c7dae70f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.797+0000 7f328393fb80 1 -- 172.21.15.64:0/1851815378 >> 172.21.15.64:0/1851815378 conn(0x55c7dadeb030 msgr2=0x55c7dae7e560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:16:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.797+0000 7f328393fb80 1 -- 172.21.15.64:0/1851815378 shutdown_connections 2023-09-13T03:16:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.797+0000 7f328393fb80 1 -- 172.21.15.64:0/1851815378 wait complete. 2023-09-13T03:16:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.797+0000 7f328393fb80 1 -- start start 2023-09-13T03:16:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.797+0000 7f328393fb80 1 Processor -- start 2023-09-13T03:16:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.798+0000 7f328393fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7dae7a710 0x55c7dae8b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.798+0000 7f328393fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7dae7b0d0 0x55c7dae8b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:17.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.798+0000 7f328393fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7dae805b0 0x55c7dae89920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:17.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.798+0000 7f328393fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7dae89ef0 con 0x55c7dae7b0d0 2023-09-13T03:16:17.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.798+0000 7f328393fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7dae8a060 con 0x55c7dae805b0 2023-09-13T03:16:17.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.798+0000 7f328393fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7dae8a220 con 0x55c7dae7a710 2023-09-13T03:16:17.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.798+0000 7f328393fb80 10 client.0 _ll_register_callbacks cb 0x55c7dacb0a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:16:17.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.798+0000 7f326d59a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7dae7b0d0 0x55c7dae8b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:17.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.798+0000 7f326dd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7dae7a710 0x55c7dae8b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:17.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.798+0000 7f326dd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7dae7a710 0x55c7dae8b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:49210/0 (socket says 172.21.15.64:49210) 2023-09-13T03:16:17.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.798+0000 7f326dd9b700 1 -- 172.21.15.64:0/1778538662 learned_addr learned my addr 172.21.15.64:0/1778538662 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:16:17.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.798+0000 7f326e59c700 1 --2- 172.21.15.64:0/1778538662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7dae805b0 0x55c7dae89920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.799+0000 7f326dd9b700 1 -- 172.21.15.64:0/1778538662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7dae805b0 msgr2=0x55c7dae89920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:16:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.799+0000 7f326dd9b700 1 --2- 172.21.15.64:0/1778538662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7dae805b0 0x55c7dae89920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.799+0000 7f326dd9b700 1 -- 172.21.15.64:0/1778538662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7dae7b0d0 msgr2=0x55c7dae8b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:16:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.799+0000 7f326dd9b700 1 --2- 172.21.15.64:0/1778538662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7dae7b0d0 0x55c7dae8b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.799+0000 7f326dd9b700 1 -- 172.21.15.64:0/1778538662 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c7dabf4d80 con 0x55c7dae7a710 2023-09-13T03:16:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.799+0000 7f326d59a700 1 --2- 172.21.15.64:0/1778538662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7dae7b0d0 0x55c7dae8b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:16:17.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.799+0000 7f326dd9b700 1 --2- 172.21.15.64:0/1778538662 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7dae7a710 0x55c7dae8b2c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3264084010 tx=0x7f3264099be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90b71e7409868984 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:16:17.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.799+0000 7f32517fa700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:16:17.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.799+0000 7f32517fa700 1 -- 172.21.15.64:0/1778538662 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32640a1590 con 0x55c7dae7a710 2023-09-13T03:16:17.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.799+0000 7f328393fb80 1 -- 172.21.15.64:0/1778538662 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f32640c9e90 con 0x55c7dae7a710 2023-09-13T03:16:17.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.799+0000 7f32517fa700 1 -- 172.21.15.64:0/1778538662 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f32640b1040 con 0x55c7dae7a710 2023-09-13T03:16:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.799+0000 7f32517fa700 1 -- 172.21.15.64:0/1778538662 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3264098e00 con 0x55c7dae7a710 2023-09-13T03:16:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.799+0000 7f328393fb80 10 client.26380 fetch_fsmap learned FSMap version 34 2023-09-13T03:16:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.800+0000 7f328393fb80 1 -- 172.21.15.64:0/1778538662 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55c7dac1b1f0 con 0x55c7dae7a710 2023-09-13T03:16:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.800+0000 7f32517fa700 1 -- 172.21.15.64:0/1778538662 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32640c9e90 con 0x55c7dae7a710 2023-09-13T03:16:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.800+0000 7f32517fa700 1 -- 172.21.15.64:0/1778538662 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 34) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f32640a1080 con 0x55c7dae7a710 2023-09-13T03:16:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.800+0000 7f328393fb80 10 client.26380 fetch_fsmap finished waiting for FSMap version 34 2023-09-13T03:16:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.800+0000 7f328393fb80 10 client.26380 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:16:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.800+0000 7f328393fb80 20 client.26380 populate_metadata read hostname 'smithi064' 2023-09-13T03:16:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.800+0000 7f328393fb80 1 --2- 172.21.15.64:0/1778538662 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55c7daee93b0 0x55c7daeeb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.800+0000 7f328393fb80 4 client.26380 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:16:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.800+0000 7f328393fb80 1 -- 172.21.15.64:0/1778538662 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c7dadc18c0 con 0x55c7daee93b0 2023-09-13T03:16:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.800+0000 7f326d59a700 1 --2- 172.21.15.64:0/1778538662 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55c7daee93b0 0x55c7daeeb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.801+0000 7f326d59a700 1 --2- 172.21.15.64:0/1778538662 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55c7daee93b0 0x55c7daeeb7d0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d521e371f3328010 server_cookie=6069e4b58ea6a81b in_seq=0 out_seq=0 2023-09-13T03:16:17.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.801+0000 7f32517fa700 10 client.26380 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:16:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.828+0000 7f32517fa700 1 -- 172.21.15.64:0/1778538662 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c7dadc18c0 con 0x55c7daee93b0 2023-09-13T03:16:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.828+0000 7f32517fa700 10 client.26380 handle_command_reply: tid=0 2023-09-13T03:16:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.828+0000 7f328393fb80 1 client.26380 shutdown 2023-09-13T03:16:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.828+0000 7f328393fb80 1 -- 172.21.15.64:0/1778538662 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7dae7a710 msgr2=0x55c7dae8b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.828+0000 7f328393fb80 1 --2- 172.21.15.64:0/1778538662 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7dae7a710 0x55c7dae8b2c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3264084010 tx=0x7f3264099be0 comp rx=0 tx=0).stop 2023-09-13T03:16:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.828+0000 7f328393fb80 1 -- 172.21.15.64:0/1778538662 shutdown_connections 2023-09-13T03:16:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.828+0000 7f328393fb80 1 --2- 172.21.15.64:0/1778538662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7dae805b0 0x55c7dae89920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.828+0000 7f328393fb80 1 --2- 172.21.15.64:0/1778538662 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55c7daee93b0 0x55c7daeeb7d0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.828+0000 7f328393fb80 1 --2- 172.21.15.64:0/1778538662 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7dae7a710 0x55c7dae8b2c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.828+0000 7f328393fb80 1 --2- 172.21.15.64:0/1778538662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c7dae7b0d0 0x55c7dae8b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.828+0000 7f32517fa700 0 client.26380 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:16:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.828+0000 7f328393fb80 1 -- 172.21.15.64:0/1778538662 >> 172.21.15.64:0/1778538662 conn(0x55c7dadeb030 msgr2=0x55c7dae752f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:16:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.828+0000 7f328393fb80 1 -- 172.21.15.64:0/1778538662 shutdown_connections 2023-09-13T03:16:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.828+0000 7f328393fb80 1 -- 172.21.15.64:0/1778538662 wait complete. 2023-09-13T03:16:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.829+0000 7f328393fb80 20 client.26380 trim_cache size 0 max 16384 2023-09-13T03:16:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:16:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:16:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:16:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:16:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.829+0000 7f326ffff700 1 -- 172.21.15.64:0/4266511296 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f32540a07b0 msgr2=0x7f32540a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.829+0000 7f326ffff700 1 --2- 172.21.15.64:0/4266511296 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f32540a07b0 0x7f32540a2c70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f325800c010 tx=0x7f3258019040 comp rx=0 tx=0).stop 2023-09-13T03:16:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.829+0000 7f326ffff700 1 -- 172.21.15.64:0/4266511296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32681005a0 msgr2=0x7f326819c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.829+0000 7f326ffff700 1 --2- 172.21.15.64:0/4266511296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32681005a0 0x7f326819c6a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f326400e9a0 tx=0x7f326400ecb0 comp rx=0 tx=0).stop 2023-09-13T03:16:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.829+0000 7f326ffff700 1 -- 172.21.15.64:0/4266511296 shutdown_connections 2023-09-13T03:16:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.829+0000 7f326ffff700 1 --2- 172.21.15.64:0/4266511296 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f32540a07b0 0x7f32540a2c70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.829+0000 7f326ffff700 1 --2- 172.21.15.64:0/4266511296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3268106660 0x7f3268196550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.829+0000 7f326ffff700 1 --2- 172.21.15.64:0/4266511296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32681005a0 0x7f326819c6a0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.829+0000 7f326ffff700 1 --2- 172.21.15.64:0/4266511296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3268100f60 0x7f326819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.829+0000 7f326ffff700 1 -- 172.21.15.64:0/4266511296 >> 172.21.15.64:0/4266511296 conn(0x7f32680fc050 msgr2=0x7f32680fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:16:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.829+0000 7f326ffff700 1 -- 172.21.15.64:0/4266511296 shutdown_connections 2023-09-13T03:16:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:17.829+0000 7f326ffff700 1 -- 172.21.15.64:0/4266511296 wait complete. 2023-09-13T03:16:17.900 INFO:tasks.workunit.client.0.smithi064.stdout: 1 61119 1.74 MB/sec execute 22 sec latency 4277.469 ms 2023-09-13T03:16:18.900 INFO:tasks.workunit.client.0.smithi064.stdout: 1 61861 1.87 MB/sec execute 23 sec latency 4899.194 ms 2023-09-13T03:16:18.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:16:18 smithi170 ceph-mon[134328]: pgmap v97: 97 pgs: 97 active+clean; 40 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 875 KiB/s wr, 76 op/s 2023-09-13T03:16:18.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:16:18 smithi064 ceph-mon[97675]: pgmap v97: 97 pgs: 97 active+clean; 40 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 875 KiB/s wr, 76 op/s 2023-09-13T03:16:18.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:16:18 smithi142 ceph-mon[134082]: pgmap v97: 97 pgs: 97 active+clean; 40 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 875 KiB/s wr, 76 op/s 2023-09-13T03:16:19.900 INFO:tasks.workunit.client.0.smithi064.stdout: 1 64134 2.36 MB/sec execute 24 sec latency 15.810 ms 2023-09-13T03:16:20.900 INFO:tasks.workunit.client.0.smithi064.stdout: 1 64536 2.28 MB/sec execute 25 sec latency 487.446 ms 2023-09-13T03:16:20.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:16:20 smithi170 ceph-mon[134328]: pgmap v98: 97 pgs: 97 active+clean; 40 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 875 KiB/s wr, 76 op/s 2023-09-13T03:16:20.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:16:20 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:16:20.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:16:20 smithi064 ceph-mon[97675]: pgmap v98: 97 pgs: 97 active+clean; 40 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 875 KiB/s wr, 76 op/s 2023-09-13T03:16:20.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:16:20 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:16:20.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:16:20 smithi142 ceph-mon[134082]: pgmap v98: 97 pgs: 97 active+clean; 40 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 875 KiB/s wr, 76 op/s 2023-09-13T03:16:20.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:16:20 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:16:21.900 INFO:tasks.workunit.client.0.smithi064.stdout: 1 64536 2.19 MB/sec execute 26 sec latency 1487.558 ms 2023-09-13T03:16:21.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:16:21 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:16:21.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:16:21 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:16:21.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:16:21 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:16:21.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:16:21 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:16:21.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:16:21 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:16:21.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:16:21 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:16:21.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:16:21 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:16:21.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:16:21 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:16:21.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:16:21 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:16:21.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:16:21 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:16:21.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:16:21 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:16:21.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:16:21 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:16:22.900 INFO:tasks.workunit.client.0.smithi064.stdout: 1 64536 2.11 MB/sec execute 27 sec latency 2487.620 ms 2023-09-13T03:16:22.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:16:22 smithi170 ceph-mon[134328]: pgmap v99: 97 pgs: 97 active+clean; 42 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 125 op/s 2023-09-13T03:16:22.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:16:22 smithi064 ceph-mon[97675]: pgmap v99: 97 pgs: 97 active+clean; 42 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 125 op/s 2023-09-13T03:16:22.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:16:22 smithi142 ceph-mon[134082]: pgmap v99: 97 pgs: 97 active+clean; 42 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 125 op/s 2023-09-13T03:16:23.900 INFO:tasks.workunit.client.0.smithi064.stdout: 1 64592 2.04 MB/sec execute 28 sec latency 3113.263 ms 2023-09-13T03:16:24.901 INFO:tasks.workunit.client.0.smithi064.stdout: 1 64592 1.97 MB/sec execute 29 sec latency 1331.889 ms 2023-09-13T03:16:24.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:16:24 smithi170 ceph-mon[134328]: pgmap v100: 97 pgs: 97 active+clean; 42 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 292 KiB/s wr, 61 op/s 2023-09-13T03:16:24.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:16:24 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:16:24.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:16:24 smithi064 ceph-mon[97675]: pgmap v100: 97 pgs: 97 active+clean; 42 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 292 KiB/s wr, 61 op/s 2023-09-13T03:16:24.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:16:24 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:16:24.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:16:24 smithi142 ceph-mon[134082]: pgmap v100: 97 pgs: 97 active+clean; 42 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 292 KiB/s wr, 61 op/s 2023-09-13T03:16:24.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:16:24 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:16:25.901 INFO:tasks.workunit.client.0.smithi064.stdout: 1 64592 1.91 MB/sec execute 30 sec latency 2331.977 ms 2023-09-13T03:16:25.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:16:25 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:16:25.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:16:25 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:16:25.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:16:25 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:16:26.901 INFO:tasks.workunit.client.0.smithi064.stdout: 1 64592 1.84 MB/sec execute 31 sec latency 3332.141 ms 2023-09-13T03:16:26.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:16:26 smithi170 ceph-mon[134328]: pgmap v101: 97 pgs: 97 active+clean; 42 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 322 KiB/s wr, 72 op/s 2023-09-13T03:16:26.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:16:26 smithi064 ceph-mon[97675]: pgmap v101: 97 pgs: 97 active+clean; 42 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 322 KiB/s wr, 72 op/s 2023-09-13T03:16:26.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:16:26 smithi142 ceph-mon[134082]: pgmap v101: 97 pgs: 97 active+clean; 42 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 322 KiB/s wr, 72 op/s 2023-09-13T03:16:27.901 INFO:tasks.workunit.client.0.smithi064.stdout: 1 64592 1.79 MB/sec execute 32 sec latency 4332.282 ms 2023-09-13T03:16:28.901 INFO:tasks.workunit.client.0.smithi064.stdout: 1 66260 1.94 MB/sec execute 33 sec latency 4661.829 ms 2023-09-13T03:16:28.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:16:28 smithi170 ceph-mon[134328]: pgmap v102: 97 pgs: 97 active+clean; 42 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 276 KiB/s wr, 60 op/s 2023-09-13T03:16:28.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:16:28 smithi064 ceph-mon[97675]: pgmap v102: 97 pgs: 97 active+clean; 42 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 276 KiB/s wr, 60 op/s 2023-09-13T03:16:28.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:16:28 smithi142 ceph-mon[134082]: pgmap v102: 97 pgs: 97 active+clean; 42 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 276 KiB/s wr, 60 op/s 2023-09-13T03:16:29.901 INFO:tasks.workunit.client.0.smithi064.stdout: 1 67852 2.24 MB/sec execute 34 sec latency 11.423 ms 2023-09-13T03:16:30.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:16:30 smithi170 ceph-mon[134328]: pgmap v103: 97 pgs: 97 active+clean; 42 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 276 KiB/s wr, 60 op/s 2023-09-13T03:16:30.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:16:30 smithi064 ceph-mon[97675]: pgmap v103: 97 pgs: 97 active+clean; 42 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 276 KiB/s wr, 60 op/s 2023-09-13T03:16:30.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:16:30 smithi142 ceph-mon[134082]: pgmap v103: 97 pgs: 97 active+clean; 42 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 276 KiB/s wr, 60 op/s 2023-09-13T03:16:30.901 INFO:tasks.workunit.client.0.smithi064.stdout: 1 68211 2.19 MB/sec execute 35 sec latency 672.718 ms 2023-09-13T03:16:30.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:30.991+0000 7fb52abff700 1 -- 172.21.15.64:0/264022373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb524106660 msgr2=0x7fb524106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:30.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:30.991+0000 7fb52abff700 1 --2- 172.21.15.64:0/264022373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb524106660 0x7fb524106a40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb5180099c0 tx=0x7fb518009cd0 comp rx=0 tx=0).stop 2023-09-13T03:16:30.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:30.991+0000 7fb52abff700 1 -- 172.21.15.64:0/264022373 shutdown_connections 2023-09-13T03:16:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:30.991+0000 7fb52abff700 1 --2- 172.21.15.64:0/264022373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb524100f60 0x7fb52410d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:30.991+0000 7fb52abff700 1 --2- 172.21.15.64:0/264022373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb524106660 0x7fb524106a40 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:30.991+0000 7fb52abff700 1 --2- 172.21.15.64:0/264022373 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5241005a0 0x7fb524100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:30.991+0000 7fb52abff700 1 -- 172.21.15.64:0/264022373 >> 172.21.15.64:0/264022373 conn(0x7fb5240fc050 msgr2=0x7fb5240fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:16:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:30.991+0000 7fb52abff700 1 -- 172.21.15.64:0/264022373 shutdown_connections 2023-09-13T03:16:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:30.991+0000 7fb52abff700 1 -- 172.21.15.64:0/264022373 wait complete. 2023-09-13T03:16:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:30.993+0000 7fb52abff700 1 Processor -- start 2023-09-13T03:16:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:30.993+0000 7fb52abff700 1 -- start start 2023-09-13T03:16:30.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:30.993+0000 7fb52abff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5241005a0 0x7fb52419e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:30.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:30.993+0000 7fb52abff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb524100f60 0x7fb52419ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:30.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:30.994+0000 7fb52abff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb524106660 0x7fb524198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:30.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:30.994+0000 7fb52abff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb524198db0 con 0x7fb5241005a0 2023-09-13T03:16:30.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:30.994+0000 7fb52abff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb524198f20 con 0x7fb524106660 2023-09-13T03:16:30.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:30.994+0000 7fb52abff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5241990e0 con 0x7fb524100f60 2023-09-13T03:16:30.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:30.994+0000 7fb52899b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5241005a0 0x7fb52419e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:30.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:30.994+0000 7fb52899b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5241005a0 0x7fb52419e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40892/0 (socket says 172.21.15.64:40892) 2023-09-13T03:16:30.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:30.994+0000 7fb52899b700 1 -- 172.21.15.64:0/3567633966 learned_addr learned my addr 172.21.15.64:0/3567633966 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:16:30.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:30.994+0000 7fb52919c700 1 --2- 172.21.15.64:0/3567633966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb524106660 0x7fb524198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:30.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:30.994+0000 7fb523fff700 1 --2- 172.21.15.64:0/3567633966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb524100f60 0x7fb52419ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:30.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:30.994+0000 7fb52899b700 1 -- 172.21.15.64:0/3567633966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb524100f60 msgr2=0x7fb52419ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:30.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:30.994+0000 7fb52899b700 1 --2- 172.21.15.64:0/3567633966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb524100f60 0x7fb52419ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:30.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:30.994+0000 7fb52899b700 1 -- 172.21.15.64:0/3567633966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb524106660 msgr2=0x7fb524198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:31.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:30.994+0000 7fb52899b700 1 --2- 172.21.15.64:0/3567633966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb524106660 0x7fb524198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:31.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:30.994+0000 7fb52899b700 1 -- 172.21.15.64:0/3567633966 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5180095f0 con 0x7fb5241005a0 2023-09-13T03:16:31.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:30.994+0000 7fb523fff700 1 --2- 172.21.15.64:0/3567633966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb524100f60 0x7fb52419ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:16:31.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:30.995+0000 7fb52899b700 1 --2- 172.21.15.64:0/3567633966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5241005a0 0x7fb52419e8f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb51800b580 tx=0x7fb51800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:16:31.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:30.995+0000 7fb521ffb700 1 -- 172.21.15.64:0/3567633966 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51801d070 con 0x7fb5241005a0 2023-09-13T03:16:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:30.995+0000 7fb521ffb700 1 -- 172.21.15.64:0/3567633966 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb518005050 con 0x7fb5241005a0 2023-09-13T03:16:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:30.995+0000 7fb52abff700 1 -- 172.21.15.64:0/3567633966 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb524199360 con 0x7fb5241005a0 2023-09-13T03:16:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:30.995+0000 7fb521ffb700 1 -- 172.21.15.64:0/3567633966 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb518004a40 con 0x7fb5241005a0 2023-09-13T03:16:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:30.995+0000 7fb52abff700 1 -- 172.21.15.64:0/3567633966 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5241a58e0 con 0x7fb5241005a0 2023-09-13T03:16:31.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:30.998+0000 7fb521ffb700 1 -- 172.21.15.64:0/3567633966 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fb5180051c0 con 0x7fb5241005a0 2023-09-13T03:16:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.000+0000 7fb53e51cb80 1 Processor -- start 2023-09-13T03:16:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.001+0000 7fb53e51cb80 1 -- start start 2023-09-13T03:16:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.001+0000 7fb521ffb700 1 --2- 172.21.15.64:0/3567633966 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fb5140a0830 0x7fb5140a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.001+0000 7fb521ffb700 1 -- 172.21.15.64:0/3567633966 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fb5180ccd80 con 0x7fb5241005a0 2023-09-13T03:16:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.001+0000 7fb52919c700 1 --2- 172.21.15.64:0/3567633966 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fb5140a0830 0x7fb5140a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.001+0000 7fb53e51cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557af94d21a0 0x557af94d2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.001+0000 7fb53e51cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557af94cc300 0x557af94cc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.001+0000 7fb53e51cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557af94cccc0 0x557af94d8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.001+0000 7fb53e51cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x557af94256d0 con 0x557af94d21a0 2023-09-13T03:16:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.001+0000 7fb53e51cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557af9422ff0 con 0x557af94cc300 2023-09-13T03:16:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.001+0000 7fb53e51cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557af941e8e0 con 0x557af94cccc0 2023-09-13T03:16:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.001+0000 7fb52899b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557af94d21a0 0x557af94d2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.001+0000 7fb52899b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557af94d21a0 0x557af94d2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40904/0 (socket says 172.21.15.64:40904) 2023-09-13T03:16:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.001+0000 7fb52899b700 1 -- 172.21.15.64:0/4078548959 learned_addr learned my addr 172.21.15.64:0/4078548959 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:16:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.001+0000 7fb52919c700 1 --2- 172.21.15.64:0/4078548959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557af94cccc0 0x557af94d8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.002+0000 7fb523fff700 1 --2- 172.21.15.64:0/4078548959 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557af94cc300 0x557af94cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.002+0000 7fb52899b700 1 -- 172.21.15.64:0/4078548959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557af94cccc0 msgr2=0x557af94d8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:16:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.002+0000 7fb52899b700 1 --2- 172.21.15.64:0/4078548959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557af94cccc0 0x557af94d8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.002+0000 7fb52899b700 1 -- 172.21.15.64:0/4078548959 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557af94cc300 msgr2=0x557af94cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:16:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.002+0000 7fb52899b700 1 --2- 172.21.15.64:0/4078548959 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557af94cc300 0x557af94cc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:31.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.002+0000 7fb52899b700 1 -- 172.21.15.64:0/4078548959 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557af924ad80 con 0x557af94d21a0 2023-09-13T03:16:31.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.002+0000 7fb52919c700 1 --2- 172.21.15.64:0/3567633966 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fb5140a0830 0x7fb5140a2cf0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00f870 tx=0x7fb51c00fc30 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:16:31.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.002+0000 7fb52899b700 1 --2- 172.21.15.64:0/4078548959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557af94d21a0 0x557af94d2580 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb518096d20 tx=0x7fb518099ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f187182ff273a6a server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:16:31.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.002+0000 7fb50effd700 1 -- 172.21.15.64:0/4078548959 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51809e0e0 con 0x557af94d21a0 2023-09-13T03:16:31.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.002+0000 7fb50effd700 1 -- 172.21.15.64:0/4078548959 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5180a2290 con 0x557af94d21a0 2023-09-13T03:16:31.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.002+0000 7fb50effd700 1 -- 172.21.15.64:0/4078548959 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb518004e60 con 0x557af94d21a0 2023-09-13T03:16:31.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.003+0000 7fb53e51cb80 1 -- 172.21.15.64:0/4078548959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557af94d21a0 msgr2=0x557af94d2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:31.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.003+0000 7fb53e51cb80 1 --2- 172.21.15.64:0/4078548959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557af94d21a0 0x557af94d2580 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb518096d20 tx=0x7fb518099ce0 comp rx=0 tx=0).stop 2023-09-13T03:16:31.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.003+0000 7fb53e51cb80 1 -- 172.21.15.64:0/4078548959 shutdown_connections 2023-09-13T03:16:31.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.003+0000 7fb53e51cb80 1 --2- 172.21.15.64:0/4078548959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557af94cccc0 0x557af94d8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:31.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.003+0000 7fb53e51cb80 1 --2- 172.21.15.64:0/4078548959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557af94d21a0 0x557af94d2580 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:31.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.003+0000 7fb53e51cb80 1 --2- 172.21.15.64:0/4078548959 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557af94cc300 0x557af94cc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:31.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.003+0000 7fb53e51cb80 1 -- 172.21.15.64:0/4078548959 >> 172.21.15.64:0/4078548959 conn(0x557af9441030 msgr2=0x557af94c7560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:16:31.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.003+0000 7fb53e51cb80 1 -- 172.21.15.64:0/4078548959 shutdown_connections 2023-09-13T03:16:31.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.003+0000 7fb53e51cb80 1 -- 172.21.15.64:0/4078548959 wait complete. 2023-09-13T03:16:31.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.004+0000 7fb53e51cb80 1 -- start start 2023-09-13T03:16:31.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.004+0000 7fb53e51cb80 1 Processor -- start 2023-09-13T03:16:31.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.004+0000 7fb53e51cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557af94cc300 0x557af94dbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:31.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.004+0000 7fb53e51cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557af94cccc0 0x557af94dc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:31.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.004+0000 7fb53e51cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557af94d21a0 0x557af94d5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:31.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.004+0000 7fb53e51cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x557af93bcb10 con 0x557af94cccc0 2023-09-13T03:16:31.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.004+0000 7fb53e51cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557af93bc810 con 0x557af94d21a0 2023-09-13T03:16:31.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.005+0000 7fb53e51cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557af92a08c0 con 0x557af94cc300 2023-09-13T03:16:31.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.005+0000 7fb53e51cb80 10 client.0 _ll_register_callbacks cb 0x557af9306a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:16:31.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.005+0000 7fb523fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557af94cccc0 0x557af94dc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:31.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.005+0000 7fb523fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557af94cccc0 0x557af94dc030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40916/0 (socket says 172.21.15.64:40916) 2023-09-13T03:16:31.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.005+0000 7fb523fff700 1 -- 172.21.15.64:0/2547234516 learned_addr learned my addr 172.21.15.64:0/2547234516 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:16:31.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.005+0000 7fb52899b700 1 --2- 172.21.15.64:0/2547234516 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557af94cc300 0x557af94dbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:31.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.005+0000 7fb52919c700 1 --2- 172.21.15.64:0/2547234516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557af94d21a0 0x557af94d5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:31.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.005+0000 7fb523fff700 1 -- 172.21.15.64:0/2547234516 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557af94cc300 msgr2=0x557af94dbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:16:31.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.005+0000 7fb523fff700 1 --2- 172.21.15.64:0/2547234516 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557af94cc300 0x557af94dbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:31.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.005+0000 7fb523fff700 1 -- 172.21.15.64:0/2547234516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557af94d21a0 msgr2=0x557af94d5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:16:31.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.005+0000 7fb523fff700 1 --2- 172.21.15.64:0/2547234516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557af94d21a0 0x557af94d5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:31.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.005+0000 7fb523fff700 1 -- 172.21.15.64:0/2547234516 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5241a58e0 con 0x557af94cccc0 2023-09-13T03:16:31.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.005+0000 7fb523fff700 1 --2- 172.21.15.64:0/2547234516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557af94cccc0 0x557af94dc030 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb510012740 tx=0x7fb510012a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=630c96591294ae92 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:16:31.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.005+0000 7fb503fff700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:16:31.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.006+0000 7fb503fff700 1 -- 172.21.15.64:0/2547234516 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb510017070 con 0x557af94cccc0 2023-09-13T03:16:31.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.006+0000 7fb53e51cb80 1 -- 172.21.15.64:0/2547234516 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557af9376490 con 0x557af94cccc0 2023-09-13T03:16:31.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.006+0000 7fb503fff700 1 -- 172.21.15.64:0/2547234516 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fb5100117d0 con 0x557af94cccc0 2023-09-13T03:16:31.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.006+0000 7fb503fff700 1 -- 172.21.15.64:0/2547234516 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb510011990 con 0x557af94cccc0 2023-09-13T03:16:31.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.006+0000 7fb53e51cb80 10 client.16353 fetch_fsmap learned FSMap version 37 2023-09-13T03:16:31.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.006+0000 7fb53e51cb80 1 -- 172.21.15.64:0/2547234516 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x557af9233c00 con 0x557af94cccc0 2023-09-13T03:16:31.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.006+0000 7fb503fff700 1 -- 172.21.15.64:0/2547234516 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557af9376490 con 0x557af94cccc0 2023-09-13T03:16:31.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.006+0000 7fb503fff700 1 -- 172.21.15.64:0/2547234516 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 37) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fb51000fc50 con 0x557af94cccc0 2023-09-13T03:16:31.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.007+0000 7fb53e51cb80 10 client.16353 fetch_fsmap finished waiting for FSMap version 37 2023-09-13T03:16:31.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.007+0000 7fb53e51cb80 10 client.16353 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:16:31.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.007+0000 7fb53e51cb80 20 client.16353 populate_metadata read hostname 'smithi064' 2023-09-13T03:16:31.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.007+0000 7fb53e51cb80 1 --2- 172.21.15.64:0/2547234516 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x557af94e60d0 0x557af94e84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:31.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.007+0000 7fb53e51cb80 4 client.16353 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:16:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.007+0000 7fb53e51cb80 1 -- 172.21.15.64:0/2547234516 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557af9433840 con 0x557af94e60d0 2023-09-13T03:16:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.007+0000 7fb52899b700 1 --2- 172.21.15.64:0/2547234516 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x557af94e60d0 0x557af94e84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.007+0000 7fb52899b700 1 --2- 172.21.15.64:0/2547234516 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x557af94e60d0 0x557af94e84f0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82fbd061b86e7295 server_cookie=27d04518158fed13 in_seq=0 out_seq=0 2023-09-13T03:16:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.007+0000 7fb503fff700 10 client.16353 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:16:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.008+0000 7fb503fff700 1 -- 172.21.15.64:0/2547234516 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557af9433840 con 0x557af94e60d0 2023-09-13T03:16:31.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.008+0000 7fb503fff700 10 client.16353 handle_command_reply: tid=0 2023-09-13T03:16:31.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.008+0000 7fb53e51cb80 1 client.16353 shutdown 2023-09-13T03:16:31.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.008+0000 7fb53e51cb80 1 -- 172.21.15.64:0/2547234516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557af94cccc0 msgr2=0x557af94dc030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:31.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.008+0000 7fb53e51cb80 1 --2- 172.21.15.64:0/2547234516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557af94cccc0 0x557af94dc030 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb510012740 tx=0x7fb510012a50 comp rx=0 tx=0).stop 2023-09-13T03:16:31.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.009+0000 7fb53e51cb80 1 -- 172.21.15.64:0/2547234516 shutdown_connections 2023-09-13T03:16:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.009+0000 7fb53e51cb80 1 --2- 172.21.15.64:0/2547234516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557af94d21a0 0x557af94d5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.009+0000 7fb53e51cb80 1 --2- 172.21.15.64:0/2547234516 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x557af94e60d0 0x557af94e84f0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.009+0000 7fb53e51cb80 1 --2- 172.21.15.64:0/2547234516 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557af94cc300 0x557af94dbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.009+0000 7fb53e51cb80 1 --2- 172.21.15.64:0/2547234516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557af94cccc0 0x557af94dc030 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.009+0000 7fb503fff700 0 client.16353 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:16:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.009+0000 7fb53e51cb80 1 -- 172.21.15.64:0/2547234516 >> 172.21.15.64:0/2547234516 conn(0x557af9441030 msgr2=0x557af94c67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:16:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.009+0000 7fb53e51cb80 1 -- 172.21.15.64:0/2547234516 shutdown_connections 2023-09-13T03:16:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.009+0000 7fb53e51cb80 1 -- 172.21.15.64:0/2547234516 wait complete. 2023-09-13T03:16:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.009+0000 7fb53e51cb80 20 client.16353 trim_cache size 0 max 16384 2023-09-13T03:16:31.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.025+0000 7fb53e51cb80 1 Processor -- start 2023-09-13T03:16:31.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.025+0000 7fb53e51cb80 1 -- start start 2023-09-13T03:16:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.025+0000 7fb53e51cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557af94cc300 0x557af94c6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.025+0000 7fb53e51cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557af94cccc0 0x557af94c7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.025+0000 7fb53e51cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557af94d21a0 0x557af94d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.025+0000 7fb53e51cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x557af94e3630 con 0x557af94d21a0 2023-09-13T03:16:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.025+0000 7fb53e51cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557af94d1e00 con 0x557af94cc300 2023-09-13T03:16:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.025+0000 7fb53e51cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557af94d7320 con 0x557af94cccc0 2023-09-13T03:16:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.026+0000 7fb52919c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557af94d21a0 0x557af94d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.026+0000 7fb52919c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557af94d21a0 0x557af94d0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40926/0 (socket says 172.21.15.64:40926) 2023-09-13T03:16:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.026+0000 7fb52919c700 1 -- 172.21.15.64:0/1791881165 learned_addr learned my addr 172.21.15.64:0/1791881165 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:16:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.026+0000 7fb52899b700 1 --2- 172.21.15.64:0/1791881165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557af94cc300 0x557af94c6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.026+0000 7fb523fff700 1 --2- 172.21.15.64:0/1791881165 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557af94cccc0 0x557af94c7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.026+0000 7fb52919c700 1 -- 172.21.15.64:0/1791881165 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557af94cccc0 msgr2=0x557af94c7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:16:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.026+0000 7fb52919c700 1 --2- 172.21.15.64:0/1791881165 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557af94cccc0 0x557af94c7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.026+0000 7fb52919c700 1 -- 172.21.15.64:0/1791881165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557af94cc300 msgr2=0x557af94c6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:16:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.026+0000 7fb52919c700 1 --2- 172.21.15.64:0/1791881165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557af94cc300 0x557af94c6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.026+0000 7fb52919c700 1 -- 172.21.15.64:0/1791881165 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557af92711f0 con 0x557af94d21a0 2023-09-13T03:16:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.026+0000 7fb52899b700 1 --2- 172.21.15.64:0/1791881165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557af94cc300 0x557af94c6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:16:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.026+0000 7fb52919c700 1 --2- 172.21.15.64:0/1791881165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557af94d21a0 0x557af94d0e30 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb51c0060b0 tx=0x7fb51c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adbb3a2f86e7dfa0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:16:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.027+0000 7fb50f7fe700 1 -- 172.21.15.64:0/1791881165 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c00e040 con 0x557af94d21a0 2023-09-13T03:16:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.027+0000 7fb50f7fe700 1 -- 172.21.15.64:0/1791881165 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb51c00f040 con 0x557af94d21a0 2023-09-13T03:16:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.027+0000 7fb50f7fe700 1 -- 172.21.15.64:0/1791881165 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c0044c0 con 0x557af94d21a0 2023-09-13T03:16:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.027+0000 7fb53e51cb80 1 -- 172.21.15.64:0/1791881165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557af94d21a0 msgr2=0x557af94d0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.027+0000 7fb53e51cb80 1 --2- 172.21.15.64:0/1791881165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557af94d21a0 0x557af94d0e30 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb51c0060b0 tx=0x7fb51c01f000 comp rx=0 tx=0).stop 2023-09-13T03:16:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.027+0000 7fb53e51cb80 1 -- 172.21.15.64:0/1791881165 shutdown_connections 2023-09-13T03:16:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.027+0000 7fb53e51cb80 1 --2- 172.21.15.64:0/1791881165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557af94cc300 0x557af94c6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.027+0000 7fb53e51cb80 1 --2- 172.21.15.64:0/1791881165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557af94d21a0 0x557af94d0e30 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.027+0000 7fb53e51cb80 1 --2- 172.21.15.64:0/1791881165 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557af94cccc0 0x557af94c7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.027+0000 7fb53e51cb80 1 -- 172.21.15.64:0/1791881165 >> 172.21.15.64:0/1791881165 conn(0x557af9441030 msgr2=0x557af94d66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:16:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.027+0000 7fb53e51cb80 1 -- 172.21.15.64:0/1791881165 shutdown_connections 2023-09-13T03:16:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.027+0000 7fb53e51cb80 1 -- 172.21.15.64:0/1791881165 wait complete. 2023-09-13T03:16:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.028+0000 7fb53e51cb80 1 -- start start 2023-09-13T03:16:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.028+0000 7fb53e51cb80 1 Processor -- start 2023-09-13T03:16:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.028+0000 7fb53e51cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557af94cc300 0x557af94e1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.028+0000 7fb53e51cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557af94cccc0 0x557af94e2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.029+0000 7fb53e51cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557af94d21a0 0x557af94e02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.029+0000 7fb53e51cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x557af94e08c0 con 0x557af94cc300 2023-09-13T03:16:31.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.029+0000 7fb53e51cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557af94e0a30 con 0x557af94cccc0 2023-09-13T03:16:31.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.029+0000 7fb53e51cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557af94e0bf0 con 0x557af94d21a0 2023-09-13T03:16:31.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.029+0000 7fb53e51cb80 10 client.0 _ll_register_callbacks cb 0x7fb510009dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:16:31.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.029+0000 7fb52899b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557af94cc300 0x557af94e1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:31.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.029+0000 7fb523fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557af94cccc0 0x557af94e2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:31.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.029+0000 7fb52899b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557af94cc300 0x557af94e1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40930/0 (socket says 172.21.15.64:40930) 2023-09-13T03:16:31.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.029+0000 7fb52899b700 1 -- 172.21.15.64:0/3706125663 learned_addr learned my addr 172.21.15.64:0/3706125663 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:16:31.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.029+0000 7fb52919c700 1 --2- 172.21.15.64:0/3706125663 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557af94d21a0 0x557af94e02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:31.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.029+0000 7fb52899b700 1 -- 172.21.15.64:0/3706125663 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557af94d21a0 msgr2=0x557af94e02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:16:31.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.029+0000 7fb52899b700 1 --2- 172.21.15.64:0/3706125663 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557af94d21a0 0x557af94e02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:31.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.029+0000 7fb52899b700 1 -- 172.21.15.64:0/3706125663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557af94cccc0 msgr2=0x557af94e2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:16:31.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.029+0000 7fb52899b700 1 --2- 172.21.15.64:0/3706125663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557af94cccc0 0x557af94e2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:31.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.029+0000 7fb52899b700 1 -- 172.21.15.64:0/3706125663 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557af924ad80 con 0x557af94cc300 2023-09-13T03:16:31.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.029+0000 7fb523fff700 1 --2- 172.21.15.64:0/3706125663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557af94cccc0 0x557af94e2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:16:31.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.029+0000 7fb52899b700 1 --2- 172.21.15.64:0/3706125663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557af94cc300 0x557af94e1e30 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb518000c00 tx=0x7fb518009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a031fdd1e35eadd server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:16:31.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.029+0000 7fb503fff700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:16:31.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.030+0000 7fb503fff700 1 -- 172.21.15.64:0/3706125663 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5180986d0 con 0x557af94cc300 2023-09-13T03:16:31.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.030+0000 7fb53e51cb80 1 -- 172.21.15.64:0/3706125663 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb5180a2290 con 0x557af94cc300 2023-09-13T03:16:31.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.030+0000 7fb503fff700 1 -- 172.21.15.64:0/3706125663 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fb5180969c0 con 0x557af94cc300 2023-09-13T03:16:31.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.030+0000 7fb503fff700 1 -- 172.21.15.64:0/3706125663 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb518096b80 con 0x557af94cc300 2023-09-13T03:16:31.042 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.030+0000 7fb53e51cb80 10 client.16359 fetch_fsmap learned FSMap version 37 2023-09-13T03:16:31.042 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.030+0000 7fb53e51cb80 1 -- 172.21.15.64:0/3706125663 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x7fb51c00fdb0 con 0x557af94cc300 2023-09-13T03:16:31.042 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.030+0000 7fb503fff700 1 -- 172.21.15.64:0/3706125663 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5180a6080 con 0x557af94cc300 2023-09-13T03:16:31.042 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.030+0000 7fb503fff700 1 -- 172.21.15.64:0/3706125663 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 37) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fb5180997a0 con 0x557af94cc300 2023-09-13T03:16:31.042 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.030+0000 7fb53e51cb80 10 client.16359 fetch_fsmap finished waiting for FSMap version 37 2023-09-13T03:16:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.030+0000 7fb53e51cb80 10 client.16359 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:16:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.030+0000 7fb53e51cb80 20 client.16359 populate_metadata read hostname 'smithi064' 2023-09-13T03:16:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.030+0000 7fb53e51cb80 1 --2- 172.21.15.64:0/3706125663 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x557af953ad80 0x557af953d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.030+0000 7fb53e51cb80 4 client.16359 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:16:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.030+0000 7fb53e51cb80 1 -- 172.21.15.64:0/3706125663 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557af94178c0 con 0x557af953ad80 2023-09-13T03:16:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.031+0000 7fb523fff700 1 --2- 172.21.15.64:0/3706125663 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x557af953ad80 0x557af953d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.031+0000 7fb523fff700 1 --2- 172.21.15.64:0/3706125663 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x557af953ad80 0x557af953d1a0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f7502bff83c7403 server_cookie=10ebb38981019541 in_seq=0 out_seq=0 2023-09-13T03:16:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.031+0000 7fb503fff700 10 client.16359 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:16:31.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.054+0000 7fb503fff700 1 -- 172.21.15.64:0/3706125663 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557af94178c0 con 0x557af953ad80 2023-09-13T03:16:31.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.054+0000 7fb503fff700 10 client.16359 handle_command_reply: tid=0 2023-09-13T03:16:31.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.055+0000 7fb53e51cb80 1 client.16359 shutdown 2023-09-13T03:16:31.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.055+0000 7fb53e51cb80 1 -- 172.21.15.64:0/3706125663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557af94cc300 msgr2=0x557af94e1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.055+0000 7fb53e51cb80 1 --2- 172.21.15.64:0/3706125663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557af94cc300 0x557af94e1e30 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb518000c00 tx=0x7fb518009cd0 comp rx=0 tx=0).stop 2023-09-13T03:16:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.055+0000 7fb53e51cb80 1 -- 172.21.15.64:0/3706125663 shutdown_connections 2023-09-13T03:16:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.055+0000 7fb53e51cb80 1 --2- 172.21.15.64:0/3706125663 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x557af953ad80 0x557af953d1a0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.055+0000 7fb53e51cb80 1 --2- 172.21.15.64:0/3706125663 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557af94d21a0 0x557af94e02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.055+0000 7fb503fff700 0 client.16359 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:16:31.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.055+0000 7fb53e51cb80 1 --2- 172.21.15.64:0/3706125663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557af94cc300 0x557af94e1e30 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:31.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.055+0000 7fb53e51cb80 1 --2- 172.21.15.64:0/3706125663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557af94cccc0 0x557af94e2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:31.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.055+0000 7fb53e51cb80 1 -- 172.21.15.64:0/3706125663 >> 172.21.15.64:0/3706125663 conn(0x557af9441030 msgr2=0x557af94d4620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:16:31.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.055+0000 7fb53e51cb80 1 -- 172.21.15.64:0/3706125663 shutdown_connections 2023-09-13T03:16:31.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.055+0000 7fb53e51cb80 1 -- 172.21.15.64:0/3706125663 wait complete. 2023-09-13T03:16:31.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.055+0000 7fb53e51cb80 20 client.16359 trim_cache size 0 max 16384 2023-09-13T03:16:31.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:16:31.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:16:31.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:16:31.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:16:31.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.056+0000 7fb52abff700 1 -- 172.21.15.64:0/3567633966 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fb5140a0830 msgr2=0x7fb5140a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:31.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.056+0000 7fb52abff700 1 --2- 172.21.15.64:0/3567633966 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fb5140a0830 0x7fb5140a2cf0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00f870 tx=0x7fb51c00fc30 comp rx=0 tx=0).stop 2023-09-13T03:16:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.056+0000 7fb52abff700 1 -- 172.21.15.64:0/3567633966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5241005a0 msgr2=0x7fb52419e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.056+0000 7fb52abff700 1 --2- 172.21.15.64:0/3567633966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5241005a0 0x7fb52419e8f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb51800b580 tx=0x7fb51800bfd0 comp rx=0 tx=0).stop 2023-09-13T03:16:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.056+0000 7fb52abff700 1 -- 172.21.15.64:0/3567633966 shutdown_connections 2023-09-13T03:16:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.056+0000 7fb52abff700 1 --2- 172.21.15.64:0/3567633966 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fb5140a0830 0x7fb5140a2cf0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.056+0000 7fb52abff700 1 --2- 172.21.15.64:0/3567633966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb524106660 0x7fb524198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.056+0000 7fb52abff700 1 --2- 172.21.15.64:0/3567633966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5241005a0 0x7fb52419e8f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.056+0000 7fb52abff700 1 --2- 172.21.15.64:0/3567633966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb524100f60 0x7fb52419ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.056+0000 7fb52abff700 1 -- 172.21.15.64:0/3567633966 >> 172.21.15.64:0/3567633966 conn(0x7fb5240fc050 msgr2=0x7fb524072830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:16:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.056+0000 7fb52abff700 1 -- 172.21.15.64:0/3567633966 shutdown_connections 2023-09-13T03:16:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:31.056+0000 7fb52abff700 1 -- 172.21.15.64:0/3567633966 wait complete. 2023-09-13T03:16:31.901 INFO:tasks.workunit.client.0.smithi064.stdout: 1 68211 2.12 MB/sec execute 36 sec latency 1672.792 ms 2023-09-13T03:16:32.901 INFO:tasks.workunit.client.0.smithi064.stdout: 1 68211 2.07 MB/sec execute 37 sec latency 2672.911 ms 2023-09-13T03:16:32.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:16:32 smithi170 ceph-mon[134328]: pgmap v104: 97 pgs: 97 active+clean; 45 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 1.1 MiB/s wr, 125 op/s 2023-09-13T03:16:32.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:16:32 smithi064 ceph-mon[97675]: pgmap v104: 97 pgs: 97 active+clean; 45 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 1.1 MiB/s wr, 125 op/s 2023-09-13T03:16:32.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:16:32 smithi142 ceph-mon[134082]: pgmap v104: 97 pgs: 97 active+clean; 45 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 1.1 MiB/s wr, 125 op/s 2023-09-13T03:16:33.901 INFO:tasks.workunit.client.0.smithi064.stdout: 1 68929 2.14 MB/sec execute 38 sec latency 3299.112 ms 2023-09-13T03:16:34.902 INFO:tasks.workunit.client.0.smithi064.stdout: 1 71202 2.43 MB/sec execute 39 sec latency 12.843 ms 2023-09-13T03:16:34.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:16:34 smithi170 ceph-mon[134328]: pgmap v105: 97 pgs: 97 active+clean; 45 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 867 KiB/s wr, 76 op/s 2023-09-13T03:16:34.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:16:34 smithi064 ceph-mon[97675]: pgmap v105: 97 pgs: 97 active+clean; 45 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 867 KiB/s wr, 76 op/s 2023-09-13T03:16:34.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:16:34 smithi142 ceph-mon[134082]: pgmap v105: 97 pgs: 97 active+clean; 45 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 867 KiB/s wr, 76 op/s 2023-09-13T03:16:35.901 INFO:tasks.workunit.client.0.smithi064.stdout: 1 71629 2.38 MB/sec execute 40 sec latency 499.567 ms 2023-09-13T03:16:35.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:16:35 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:16:35.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:16:35 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:16:35.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:16:35 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:16:36.901 INFO:tasks.workunit.client.0.smithi064.stdout: 1 71629 2.32 MB/sec execute 41 sec latency 1499.631 ms 2023-09-13T03:16:36.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:16:36 smithi170 ceph-mon[134328]: pgmap v106: 97 pgs: 97 active+clean; 43 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 1.1 MiB/s wr, 124 op/s 2023-09-13T03:16:36.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:16:36 smithi064 ceph-mon[97675]: pgmap v106: 97 pgs: 97 active+clean; 43 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 1.1 MiB/s wr, 124 op/s 2023-09-13T03:16:36.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:16:36 smithi142 ceph-mon[134082]: pgmap v106: 97 pgs: 97 active+clean; 43 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 1.1 MiB/s wr, 124 op/s 2023-09-13T03:16:37.902 INFO:tasks.workunit.client.0.smithi064.stdout: 1 71685 2.27 MB/sec execute 42 sec latency 1504.411 ms 2023-09-13T03:16:38.902 INFO:tasks.workunit.client.0.smithi064.stdout: 1 72373 2.33 MB/sec execute 43 sec latency 1568.303 ms 2023-09-13T03:16:38.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:16:38 smithi170 ceph-mon[134328]: pgmap v107: 97 pgs: 97 active+clean; 43 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2023-09-13T03:16:38.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:16:38 smithi064 ceph-mon[97675]: pgmap v107: 97 pgs: 97 active+clean; 43 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2023-09-13T03:16:38.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:16:38 smithi142 ceph-mon[134082]: pgmap v107: 97 pgs: 97 active+clean; 43 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2023-09-13T03:16:39.902 INFO:tasks.workunit.client.0.smithi064.stdout: 1 74699 2.57 MB/sec execute 44 sec latency 28.274 ms 2023-09-13T03:16:39.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:16:39 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:16:39.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:16:39 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:16:39.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:16:39 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:16:40.902 INFO:tasks.workunit.client.0.smithi064.stdout: 1 75210 2.54 MB/sec execute 45 sec latency 397.325 ms 2023-09-13T03:16:40.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:16:40 smithi170 ceph-mon[134328]: pgmap v108: 97 pgs: 97 active+clean; 43 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2023-09-13T03:16:40.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:16:40 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:16:40.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:16:40 smithi064 ceph-mon[97675]: pgmap v108: 97 pgs: 97 active+clean; 43 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2023-09-13T03:16:40.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:16:40 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:16:40.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:16:40 smithi142 ceph-mon[134082]: pgmap v108: 97 pgs: 97 active+clean; 43 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2023-09-13T03:16:40.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:16:40 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:16:41.902 INFO:tasks.workunit.client.0.smithi064.stdout: 1 75210 2.49 MB/sec execute 46 sec latency 1397.470 ms 2023-09-13T03:16:42.902 INFO:tasks.workunit.client.0.smithi064.stdout: 1 75210 2.44 MB/sec execute 47 sec latency 2397.609 ms 2023-09-13T03:16:42.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:16:42 smithi170 ceph-mon[134328]: pgmap v109: 97 pgs: 97 active+clean; 46 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 158 op/s 2023-09-13T03:16:42.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:16:42 smithi064 ceph-mon[97675]: pgmap v109: 97 pgs: 97 active+clean; 46 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 158 op/s 2023-09-13T03:16:42.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:16:42 smithi142 ceph-mon[134082]: pgmap v109: 97 pgs: 97 active+clean; 46 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 158 op/s 2023-09-13T03:16:43.902 INFO:tasks.workunit.client.0.smithi064.stdout: 1 75266 2.39 MB/sec execute 48 sec latency 3025.827 ms 2023-09-13T03:16:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.218+0000 7f6fb19ff700 1 -- 172.21.15.64:0/1291680339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fac106650 msgr2=0x7f6fac106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.218+0000 7f6fb19ff700 1 --2- 172.21.15.64:0/1291680339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fac106650 0x7f6fac106a30 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c0099c0 tx=0x7f6f9c009cd0 comp rx=0 tx=0).stop 2023-09-13T03:16:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.219+0000 7f6fb19ff700 1 -- 172.21.15.64:0/1291680339 shutdown_connections 2023-09-13T03:16:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.219+0000 7f6fb19ff700 1 --2- 172.21.15.64:0/1291680339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6fac0683b0 0x7f6fac10d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.219+0000 7f6fb19ff700 1 --2- 172.21.15.64:0/1291680339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fac106650 0x7f6fac106a30 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:44.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.219+0000 7f6fb19ff700 1 --2- 172.21.15.64:0/1291680339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fac0679f0 0x7f6fac067e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:44.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.219+0000 7f6fb19ff700 1 -- 172.21.15.64:0/1291680339 >> 172.21.15.64:0/1291680339 conn(0x7f6fac074f30 msgr2=0x7f6fac075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:16:44.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.219+0000 7f6fb19ff700 1 -- 172.21.15.64:0/1291680339 shutdown_connections 2023-09-13T03:16:44.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.219+0000 7f6fb19ff700 1 -- 172.21.15.64:0/1291680339 wait complete. 2023-09-13T03:16:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.220+0000 7f6fb19ff700 1 Processor -- start 2023-09-13T03:16:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.220+0000 7f6fb19ff700 1 -- start start 2023-09-13T03:16:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.220+0000 7f6fb19ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6fac0679f0 0x7f6fac19e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.220+0000 7f6fb19ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fac0683b0 0x7f6fac19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.220+0000 7f6fb19ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fac106650 0x7f6fac198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.220+0000 7f6fb19ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac198de0 con 0x7f6fac106650 2023-09-13T03:16:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.220+0000 7f6fb19ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac198f50 con 0x7f6fac0683b0 2023-09-13T03:16:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.220+0000 7f6fb19ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac199110 con 0x7f6fac0679f0 2023-09-13T03:16:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.221+0000 7f6fab7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fac106650 0x7f6fac198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.221+0000 7f6fab7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fac106650 0x7f6fac198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60016/0 (socket says 172.21.15.64:60016) 2023-09-13T03:16:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.221+0000 7f6fab7fe700 1 -- 172.21.15.64:0/883531041 learned_addr learned my addr 172.21.15.64:0/883531041 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:16:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.221+0000 7f6faaffd700 1 --2- 172.21.15.64:0/883531041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6fac0679f0 0x7f6fac19e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.221+0000 7f6faa7fc700 1 --2- 172.21.15.64:0/883531041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fac0683b0 0x7f6fac19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.221+0000 7f6fab7fe700 1 -- 172.21.15.64:0/883531041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6fac0679f0 msgr2=0x7f6fac19e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.221+0000 7f6fab7fe700 1 --2- 172.21.15.64:0/883531041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6fac0679f0 0x7f6fac19e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.221+0000 7f6fab7fe700 1 -- 172.21.15.64:0/883531041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fac0683b0 msgr2=0x7f6fac19ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.221+0000 7f6fab7fe700 1 --2- 172.21.15.64:0/883531041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fac0683b0 0x7f6fac19ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.221+0000 7f6fab7fe700 1 -- 172.21.15.64:0/883531041 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f9c0095f0 con 0x7f6fac106650 2023-09-13T03:16:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.221+0000 7f6faa7fc700 1 --2- 172.21.15.64:0/883531041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fac0683b0 0x7f6fac19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:16:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.221+0000 7f6faaffd700 1 --2- 172.21.15.64:0/883531041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6fac0679f0 0x7f6fac19e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:16:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.221+0000 7f6fab7fe700 1 --2- 172.21.15.64:0/883531041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fac106650 0x7f6fac198610 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6fa400ef30 tx=0x7f6fa400c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:16:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.221+0000 7f6fb09fd700 1 -- 172.21.15.64:0/883531041 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4018070 con 0x7f6fac106650 2023-09-13T03:16:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.222+0000 7f6fb19ff700 1 -- 172.21.15.64:0/883531041 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fac199390 con 0x7f6fac106650 2023-09-13T03:16:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.222+0000 7f6fb09fd700 1 -- 172.21.15.64:0/883531041 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fa400f040 con 0x7f6fac106650 2023-09-13T03:16:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.222+0000 7f6fb09fd700 1 -- 172.21.15.64:0/883531041 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4003dc0 con 0x7f6fac106650 2023-09-13T03:16:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.222+0000 7f6fb19ff700 1 -- 172.21.15.64:0/883531041 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fac1a57c0 con 0x7f6fac106650 2023-09-13T03:16:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.225+0000 7f6fb09fd700 1 -- 172.21.15.64:0/883531041 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f6fa400caa0 con 0x7f6fac106650 2023-09-13T03:16:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.226+0000 7f6fb09fd700 1 --2- 172.21.15.64:0/883531041 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f6f940a07b0 0x7f6f940a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.226+0000 7f6fc53ccb80 1 Processor -- start 2023-09-13T03:16:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.226+0000 7f6fb09fd700 1 -- 172.21.15.64:0/883531041 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f6fa40cbc40 con 0x7f6fac106650 2023-09-13T03:16:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.227+0000 7f6fc53ccb80 1 -- start start 2023-09-13T03:16:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.227+0000 7f6faaffd700 1 --2- 172.21.15.64:0/883531041 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f6f940a07b0 0x7f6f940a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.227+0000 7f6fc53ccb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f4185f1a0 0x562f4185f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.227+0000 7f6fc53ccb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f41859300 0x562f41859780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.227+0000 7f6fc53ccb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f41859cc0 0x562f41865f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.227+0000 7f6fc53ccb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f417b26d0 con 0x562f41859300 2023-09-13T03:16:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.227+0000 7f6fc53ccb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f417afff0 con 0x562f4185f1a0 2023-09-13T03:16:44.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.227+0000 7f6fc53ccb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f417ab8e0 con 0x562f41859cc0 2023-09-13T03:16:44.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.227+0000 7f6faaffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f4185f1a0 0x562f4185f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:44.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.227+0000 7f6faa7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f41859300 0x562f41859780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:44.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.227+0000 7f6faa7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f41859300 0x562f41859780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60028/0 (socket says 172.21.15.64:60028) 2023-09-13T03:16:44.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.227+0000 7f6faa7fc700 1 -- 172.21.15.64:0/1414315442 learned_addr learned my addr 172.21.15.64:0/1414315442 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:16:44.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.228+0000 7f6fab7fe700 1 --2- 172.21.15.64:0/1414315442 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f41859cc0 0x562f41865f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:44.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.228+0000 7f6faaffd700 1 --2- 172.21.15.64:0/883531041 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f6f940a07b0 0x7f6f940a2c70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c009fd0 tx=0x7f6f9c01a040 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:16:44.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.228+0000 7f6faa7fc700 1 -- 172.21.15.64:0/1414315442 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f41859cc0 msgr2=0x562f41865f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:16:44.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.228+0000 7f6faa7fc700 1 --2- 172.21.15.64:0/1414315442 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f41859cc0 0x562f41865f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:44.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.228+0000 7f6faa7fc700 1 -- 172.21.15.64:0/1414315442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f4185f1a0 msgr2=0x562f4185f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:16:44.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.228+0000 7f6faa7fc700 1 --2- 172.21.15.64:0/1414315442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f4185f1a0 0x562f4185f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.228+0000 7f6faa7fc700 1 -- 172.21.15.64:0/1414315442 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f415d7d80 con 0x562f41859300 2023-09-13T03:16:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.228+0000 7f6faaffd700 1 --2- 172.21.15.64:0/1414315442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f4185f1a0 0x562f4185f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:16:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.228+0000 7f6faa7fc700 1 --2- 172.21.15.64:0/1414315442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f41859300 0x562f41859780 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0009d40 tx=0x7f6fa00116c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fee68672a5738dd8 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:16:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.228+0000 7f6f91ffb700 1 -- 172.21.15.64:0/1414315442 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0011e20 con 0x562f41859300 2023-09-13T03:16:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.228+0000 7f6f91ffb700 1 -- 172.21.15.64:0/1414315442 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fa0009e90 con 0x562f41859300 2023-09-13T03:16:44.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.228+0000 7f6f91ffb700 1 -- 172.21.15.64:0/1414315442 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0012450 con 0x562f41859300 2023-09-13T03:16:44.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.229+0000 7f6fc53ccb80 1 -- 172.21.15.64:0/1414315442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f41859300 msgr2=0x562f41859780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:44.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.229+0000 7f6fc53ccb80 1 --2- 172.21.15.64:0/1414315442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f41859300 0x562f41859780 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0009d40 tx=0x7f6fa00116c0 comp rx=0 tx=0).stop 2023-09-13T03:16:44.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.229+0000 7f6fc53ccb80 1 -- 172.21.15.64:0/1414315442 shutdown_connections 2023-09-13T03:16:44.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.229+0000 7f6fc53ccb80 1 --2- 172.21.15.64:0/1414315442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f4185f1a0 0x562f4185f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:44.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.229+0000 7f6fc53ccb80 1 --2- 172.21.15.64:0/1414315442 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f41859cc0 0x562f41865f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:44.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.229+0000 7f6fc53ccb80 1 --2- 172.21.15.64:0/1414315442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f41859300 0x562f41859780 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:44.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.229+0000 7f6fc53ccb80 1 -- 172.21.15.64:0/1414315442 >> 172.21.15.64:0/1414315442 conn(0x562f417ce030 msgr2=0x562f41854560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:16:44.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.229+0000 7f6fc53ccb80 1 -- 172.21.15.64:0/1414315442 shutdown_connections 2023-09-13T03:16:44.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.229+0000 7f6fc53ccb80 1 -- 172.21.15.64:0/1414315442 wait complete. 2023-09-13T03:16:44.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.230+0000 7f6fc53ccb80 1 -- start start 2023-09-13T03:16:44.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.230+0000 7f6fc53ccb80 1 Processor -- start 2023-09-13T03:16:44.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.231+0000 7f6fc53ccb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f41859300 0x562f41865d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.231+0000 7f6fc53ccb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f41859cc0 0x562f41863080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.231+0000 7f6fc53ccb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f4185f1a0 0x562f41863600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.231+0000 7f6fc53ccb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f41749b10 con 0x562f41859300 2023-09-13T03:16:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.231+0000 7f6fc53ccb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f41749810 con 0x562f41859cc0 2023-09-13T03:16:44.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.231+0000 7f6fc53ccb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f4162d8c0 con 0x562f4185f1a0 2023-09-13T03:16:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.231+0000 7f6fc53ccb80 10 client.0 _ll_register_callbacks cb 0x562f41693a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:16:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.231+0000 7f6faaffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f41859300 0x562f41865d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.231+0000 7f6faa7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f41859cc0 0x562f41863080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.231+0000 7f6fab7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f4185f1a0 0x562f41863600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.231+0000 7f6fab7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f4185f1a0 0x562f41863600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:52880/0 (socket says 172.21.15.64:52880) 2023-09-13T03:16:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.231+0000 7f6fab7fe700 1 -- 172.21.15.64:0/3134884227 learned_addr learned my addr 172.21.15.64:0/3134884227 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:16:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.231+0000 7f6faa7fc700 1 --2- 172.21.15.64:0/3134884227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f41859cc0 0x562f41863080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:56704/0 (socket says 172.21.15.64:56704) 2023-09-13T03:16:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.231+0000 7f6fab7fe700 1 -- 172.21.15.64:0/3134884227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f41859cc0 msgr2=0x562f41863080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:16:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.231+0000 7f6fab7fe700 1 --2- 172.21.15.64:0/3134884227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f41859cc0 0x562f41863080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.231+0000 7f6fab7fe700 1 -- 172.21.15.64:0/3134884227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f41859300 msgr2=0x562f41865d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:16:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.231+0000 7f6fab7fe700 1 --2- 172.21.15.64:0/3134884227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f41859300 0x562f41865d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.231+0000 7f6fab7fe700 1 -- 172.21.15.64:0/3134884227 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fac10dcd0 con 0x562f4185f1a0 2023-09-13T03:16:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.231+0000 7f6faa7fc700 1 --2- 172.21.15.64:0/3134884227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f41859cc0 0x562f41863080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:16:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.232+0000 7f6fab7fe700 1 --2- 172.21.15.64:0/3134884227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f4185f1a0 0x562f41863600 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4099100 tx=0x7f6fa4098dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be38f3dca1436532 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:16:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.232+0000 7f6f8affd700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:16:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.232+0000 7f6f8affd700 1 -- 172.21.15.64:0/3134884227 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa40c3520 con 0x562f4185f1a0 2023-09-13T03:16:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.232+0000 7f6fc53ccb80 1 -- 172.21.15.64:0/3134884227 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562f41703490 con 0x562f4185f1a0 2023-09-13T03:16:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.232+0000 7f6f8affd700 1 -- 172.21.15.64:0/3134884227 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f6fa401b320 con 0x562f4185f1a0 2023-09-13T03:16:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.232+0000 7f6f8affd700 1 -- 172.21.15.64:0/3134884227 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6fa4099870 con 0x562f4185f1a0 2023-09-13T03:16:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.232+0000 7f6fc53ccb80 10 client.26410 fetch_fsmap learned FSMap version 40 2023-09-13T03:16:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.232+0000 7f6fc53ccb80 1 -- 172.21.15.64:0/3134884227 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7f6fa0009970 con 0x562f4185f1a0 2023-09-13T03:16:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.232+0000 7f6f8affd700 1 -- 172.21.15.64:0/3134884227 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562f41703490 con 0x562f4185f1a0 2023-09-13T03:16:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.233+0000 7f6f8affd700 1 -- 172.21.15.64:0/3134884227 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6fa4098660 con 0x562f4185f1a0 2023-09-13T03:16:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.233+0000 7f6fc53ccb80 10 client.26410 fetch_fsmap finished waiting for FSMap version 40 2023-09-13T03:16:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.233+0000 7f6fc53ccb80 10 client.26410 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:16:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.233+0000 7f6fc53ccb80 20 client.26410 populate_metadata read hostname 'smithi064' 2023-09-13T03:16:44.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.233+0000 7f6fc53ccb80 1 --2- 172.21.15.64:0/3134884227 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x562f418733c0 0x562f418757b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:44.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.233+0000 7f6fc53ccb80 4 client.26410 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:16:44.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.233+0000 7f6fc53ccb80 1 -- 172.21.15.64:0/3134884227 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562f417c0840 con 0x562f418733c0 2023-09-13T03:16:44.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.233+0000 7f6faaffd700 1 --2- 172.21.15.64:0/3134884227 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x562f418733c0 0x562f418757b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:44.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.234+0000 7f6faaffd700 1 --2- 172.21.15.64:0/3134884227 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x562f418733c0 0x562f418757b0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2eb5c9ca86cfa73f server_cookie=18daf674bb6977f8 in_seq=0 out_seq=0 2023-09-13T03:16:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.234+0000 7f6f8affd700 10 client.26410 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:16:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.235+0000 7f6f8affd700 1 -- 172.21.15.64:0/3134884227 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562f417c0840 con 0x562f418733c0 2023-09-13T03:16:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.235+0000 7f6f8affd700 10 client.26410 handle_command_reply: tid=0 2023-09-13T03:16:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.235+0000 7f6fc53ccb80 1 client.26410 shutdown 2023-09-13T03:16:44.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.235+0000 7f6fc53ccb80 1 -- 172.21.15.64:0/3134884227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f4185f1a0 msgr2=0x562f41863600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:44.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.235+0000 7f6fc53ccb80 1 --2- 172.21.15.64:0/3134884227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f4185f1a0 0x562f41863600 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4099100 tx=0x7f6fa4098dd0 comp rx=0 tx=0).stop 2023-09-13T03:16:44.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.235+0000 7f6fc53ccb80 1 -- 172.21.15.64:0/3134884227 shutdown_connections 2023-09-13T03:16:44.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.235+0000 7f6fc53ccb80 1 --2- 172.21.15.64:0/3134884227 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x562f418733c0 0x562f418757b0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.235+0000 7f6fc53ccb80 1 --2- 172.21.15.64:0/3134884227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f4185f1a0 0x562f41863600 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.235+0000 7f6fc53ccb80 1 --2- 172.21.15.64:0/3134884227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f41859300 0x562f41865d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.235+0000 7f6fc53ccb80 1 --2- 172.21.15.64:0/3134884227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f41859cc0 0x562f41863080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.235+0000 7f6f8affd700 0 client.26410 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:16:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.235+0000 7f6fc53ccb80 1 -- 172.21.15.64:0/3134884227 >> 172.21.15.64:0/3134884227 conn(0x562f417ce030 msgr2=0x562f418537a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:16:44.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.235+0000 7f6fc53ccb80 1 -- 172.21.15.64:0/3134884227 shutdown_connections 2023-09-13T03:16:44.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.235+0000 7f6fc53ccb80 1 -- 172.21.15.64:0/3134884227 wait complete. 2023-09-13T03:16:44.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.235+0000 7f6fc53ccb80 20 client.26410 trim_cache size 0 max 16384 2023-09-13T03:16:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.251+0000 7f6fc53ccb80 1 Processor -- start 2023-09-13T03:16:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.251+0000 7f6fc53ccb80 1 -- start start 2023-09-13T03:16:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.251+0000 7f6fc53ccb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f41859300 0x562f4186a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.251+0000 7f6fc53ccb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f41859cc0 0x562f4185bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.251+0000 7f6fc53ccb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f4185f1a0 0x562f4185c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.251+0000 7f6fc53ccb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f41850fc0 con 0x562f4185f1a0 2023-09-13T03:16:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.251+0000 7f6fc53ccb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f4185a580 con 0x562f41859300 2023-09-13T03:16:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.251+0000 7f6fc53ccb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f4185ee00 con 0x562f41859cc0 2023-09-13T03:16:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.251+0000 7f6fab7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f4185f1a0 0x562f4185c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.251+0000 7f6faaffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f41859300 0x562f4186a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:44.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.251+0000 7f6fab7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f4185f1a0 0x562f4185c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60042/0 (socket says 172.21.15.64:60042) 2023-09-13T03:16:44.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.252+0000 7f6fab7fe700 1 -- 172.21.15.64:0/1506455037 learned_addr learned my addr 172.21.15.64:0/1506455037 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:16:44.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.252+0000 7f6faa7fc700 1 --2- 172.21.15.64:0/1506455037 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f41859cc0 0x562f4185bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:44.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.252+0000 7f6faaffd700 1 -- 172.21.15.64:0/1506455037 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f41859cc0 msgr2=0x562f4185bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:16:44.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.252+0000 7f6faaffd700 1 --2- 172.21.15.64:0/1506455037 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f41859cc0 0x562f4185bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:44.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.252+0000 7f6faaffd700 1 -- 172.21.15.64:0/1506455037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f4185f1a0 msgr2=0x562f4185c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:16:44.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.252+0000 7f6faaffd700 1 --2- 172.21.15.64:0/1506455037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f4185f1a0 0x562f4185c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:44.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.252+0000 7f6faaffd700 1 -- 172.21.15.64:0/1506455037 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f415c0c00 con 0x562f41859300 2023-09-13T03:16:44.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.252+0000 7f6fab7fe700 1 --2- 172.21.15.64:0/1506455037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f4185f1a0 0x562f4185c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:16:44.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.252+0000 7f6faaffd700 1 --2- 172.21.15.64:0/1506455037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f41859300 0x562f4186a520 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c0058e0 tx=0x7f6f9c023000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2dc3676238988e3 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:16:44.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.252+0000 7f6f927fc700 1 -- 172.21.15.64:0/1506455037 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c01d040 con 0x562f41859300 2023-09-13T03:16:44.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.253+0000 7f6f927fc700 1 -- 172.21.15.64:0/1506455037 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f9c00f840 con 0x562f41859300 2023-09-13T03:16:44.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.253+0000 7f6f927fc700 1 -- 172.21.15.64:0/1506455037 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c02f640 con 0x562f41859300 2023-09-13T03:16:44.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.253+0000 7f6fc53ccb80 1 -- 172.21.15.64:0/1506455037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f41859300 msgr2=0x562f4186a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:44.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.253+0000 7f6fc53ccb80 1 --2- 172.21.15.64:0/1506455037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f41859300 0x562f4186a520 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c0058e0 tx=0x7f6f9c023000 comp rx=0 tx=0).stop 2023-09-13T03:16:44.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.253+0000 7f6fc53ccb80 1 -- 172.21.15.64:0/1506455037 shutdown_connections 2023-09-13T03:16:44.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.253+0000 7f6fc53ccb80 1 --2- 172.21.15.64:0/1506455037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f41859300 0x562f4186a520 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:44.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.253+0000 7f6fc53ccb80 1 --2- 172.21.15.64:0/1506455037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f4185f1a0 0x562f4185c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:44.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.253+0000 7f6fc53ccb80 1 --2- 172.21.15.64:0/1506455037 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f41859cc0 0x562f4185bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:44.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.253+0000 7f6fc53ccb80 1 -- 172.21.15.64:0/1506455037 >> 172.21.15.64:0/1506455037 conn(0x562f417ce030 msgr2=0x562f4185e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:16:44.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.253+0000 7f6fc53ccb80 1 -- 172.21.15.64:0/1506455037 shutdown_connections 2023-09-13T03:16:44.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.253+0000 7f6fc53ccb80 1 -- 172.21.15.64:0/1506455037 wait complete. 2023-09-13T03:16:44.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.254+0000 7f6fc53ccb80 1 -- start start 2023-09-13T03:16:44.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.254+0000 7f6fc53ccb80 1 Processor -- start 2023-09-13T03:16:44.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.254+0000 7f6fc53ccb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f41859300 0x562f41869e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:44.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.254+0000 7f6fc53ccb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f41859cc0 0x562f4186a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:44.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.254+0000 7f6fc53ccb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f4185f1a0 0x562f41868510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:44.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.254+0000 7f6fc53ccb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f41868ae0 con 0x562f41859300 2023-09-13T03:16:44.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.254+0000 7f6fc53ccb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f41868c50 con 0x562f41859cc0 2023-09-13T03:16:44.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.254+0000 7f6fc53ccb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f41868e10 con 0x562f4185f1a0 2023-09-13T03:16:44.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.254+0000 7f6fc53ccb80 10 client.0 _ll_register_callbacks cb 0x7f6fa4098f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:16:44.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.255+0000 7f6faaffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f41859300 0x562f41869e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:44.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.255+0000 7f6faaffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f41859300 0x562f41869e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60048/0 (socket says 172.21.15.64:60048) 2023-09-13T03:16:44.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.255+0000 7f6faaffd700 1 -- 172.21.15.64:0/677484806 learned_addr learned my addr 172.21.15.64:0/677484806 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:16:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.255+0000 7f6faa7fc700 1 --2- 172.21.15.64:0/677484806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f41859cc0 0x562f4186a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.255+0000 7f6fab7fe700 1 --2- 172.21.15.64:0/677484806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f4185f1a0 0x562f41868510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.255+0000 7f6faaffd700 1 -- 172.21.15.64:0/677484806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f4185f1a0 msgr2=0x562f41868510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:16:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.255+0000 7f6faaffd700 1 --2- 172.21.15.64:0/677484806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f4185f1a0 0x562f41868510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.255+0000 7f6faaffd700 1 -- 172.21.15.64:0/677484806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f41859cc0 msgr2=0x562f4186a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:16:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.255+0000 7f6faaffd700 1 --2- 172.21.15.64:0/677484806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f41859cc0 0x562f4186a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.255+0000 7f6faaffd700 1 -- 172.21.15.64:0/677484806 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fa0009970 con 0x562f41859300 2023-09-13T03:16:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.255+0000 7f6faaffd700 1 --2- 172.21.15.64:0/677484806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f41859300 0x562f41869e60 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c0099c0 tx=0x7f6f9c005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1194a414a4ee3f1c server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:16:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.255+0000 7f6f8affd700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:16:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.256+0000 7f6f8affd700 1 -- 172.21.15.64:0/677484806 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c01d040 con 0x562f41859300 2023-09-13T03:16:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.256+0000 7f6f8affd700 1 -- 172.21.15.64:0/677484806 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f6f9c033150 con 0x562f41859300 2023-09-13T03:16:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.256+0000 7f6fc53ccb80 1 -- 172.21.15.64:0/677484806 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6fa0009e90 con 0x562f41859300 2023-09-13T03:16:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.256+0000 7f6f8affd700 1 -- 172.21.15.64:0/677484806 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f9c033310 con 0x562f41859300 2023-09-13T03:16:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.256+0000 7f6fc53ccb80 10 client.16377 fetch_fsmap learned FSMap version 40 2023-09-13T03:16:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.256+0000 7f6fc53ccb80 1 -- 172.21.15.64:0/677484806 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7f6f9c033b10 con 0x562f41859300 2023-09-13T03:16:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.256+0000 7f6f8affd700 1 -- 172.21.15.64:0/677484806 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fa0009e90 con 0x562f41859300 2023-09-13T03:16:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.256+0000 7f6f8affd700 1 -- 172.21.15.64:0/677484806 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6f9c04e770 con 0x562f41859300 2023-09-13T03:16:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.257+0000 7f6fc53ccb80 10 client.16377 fetch_fsmap finished waiting for FSMap version 40 2023-09-13T03:16:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.257+0000 7f6fc53ccb80 10 client.16377 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:16:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.257+0000 7f6fc53ccb80 20 client.16377 populate_metadata read hostname 'smithi064' 2023-09-13T03:16:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.257+0000 7f6fc53ccb80 1 --2- 172.21.15.64:0/677484806 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x562f418c7ec0 0x562f418ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.257+0000 7f6fc53ccb80 4 client.16377 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:16:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.257+0000 7f6fc53ccb80 1 -- 172.21.15.64:0/677484806 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562f417a48c0 con 0x562f418c7ec0 2023-09-13T03:16:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.257+0000 7f6faa7fc700 1 --2- 172.21.15.64:0/677484806 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x562f418c7ec0 0x562f418ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.258+0000 7f6f8affd700 10 client.16377 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:16:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.258+0000 7f6faa7fc700 1 --2- 172.21.15.64:0/677484806 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x562f418c7ec0 0x562f418ca2e0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e231c86fdbf8b8e2 server_cookie=bff83f088606a5b5 in_seq=0 out_seq=0 2023-09-13T03:16:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.290+0000 7f6f8affd700 1 -- 172.21.15.64:0/677484806 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562f417a48c0 con 0x562f418c7ec0 2023-09-13T03:16:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.290+0000 7f6f8affd700 10 client.16377 handle_command_reply: tid=0 2023-09-13T03:16:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.290+0000 7f6fc53ccb80 1 client.16377 shutdown 2023-09-13T03:16:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.291+0000 7f6fc53ccb80 1 -- 172.21.15.64:0/677484806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f41859300 msgr2=0x562f41869e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.291+0000 7f6fc53ccb80 1 --2- 172.21.15.64:0/677484806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f41859300 0x562f41869e60 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c0099c0 tx=0x7f6f9c005a00 comp rx=0 tx=0).stop 2023-09-13T03:16:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.291+0000 7f6fc53ccb80 1 -- 172.21.15.64:0/677484806 shutdown_connections 2023-09-13T03:16:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.291+0000 7f6fc53ccb80 1 --2- 172.21.15.64:0/677484806 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x562f418c7ec0 0x562f418ca2e0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.291+0000 7f6fc53ccb80 1 --2- 172.21.15.64:0/677484806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f4185f1a0 0x562f41868510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.291+0000 7f6fc53ccb80 1 --2- 172.21.15.64:0/677484806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f41859300 0x562f41869e60 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:44.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.291+0000 7f6f8affd700 0 client.16377 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:16:44.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.291+0000 7f6fc53ccb80 1 --2- 172.21.15.64:0/677484806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f41859cc0 0x562f4186a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:44.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.291+0000 7f6fc53ccb80 1 -- 172.21.15.64:0/677484806 >> 172.21.15.64:0/677484806 conn(0x562f417ce030 msgr2=0x562f41852f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:16:44.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.291+0000 7f6fc53ccb80 1 -- 172.21.15.64:0/677484806 shutdown_connections 2023-09-13T03:16:44.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.291+0000 7f6fc53ccb80 1 -- 172.21.15.64:0/677484806 wait complete. 2023-09-13T03:16:44.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.291+0000 7f6fc53ccb80 20 client.16377 trim_cache size 0 max 16384 2023-09-13T03:16:44.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:16:44.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:16:44.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:16:44.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:16:44.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.292+0000 7f6fb19ff700 1 -- 172.21.15.64:0/883531041 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f6f940a07b0 msgr2=0x7f6f940a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:44.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.292+0000 7f6fb19ff700 1 --2- 172.21.15.64:0/883531041 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f6f940a07b0 0x7f6f940a2c70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c009fd0 tx=0x7f6f9c01a040 comp rx=0 tx=0).stop 2023-09-13T03:16:44.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.292+0000 7f6fb19ff700 1 -- 172.21.15.64:0/883531041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fac106650 msgr2=0x7f6fac198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:44.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.292+0000 7f6fb19ff700 1 --2- 172.21.15.64:0/883531041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fac106650 0x7f6fac198610 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6fa400ef30 tx=0x7f6fa400c450 comp rx=0 tx=0).stop 2023-09-13T03:16:44.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.292+0000 7f6fb19ff700 1 -- 172.21.15.64:0/883531041 shutdown_connections 2023-09-13T03:16:44.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.292+0000 7f6fb19ff700 1 --2- 172.21.15.64:0/883531041 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f6f940a07b0 0x7f6f940a2c70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:44.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.292+0000 7f6fb19ff700 1 --2- 172.21.15.64:0/883531041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fac106650 0x7f6fac198610 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:44.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.292+0000 7f6fb19ff700 1 --2- 172.21.15.64:0/883531041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6fac0679f0 0x7f6fac19e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:44.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.292+0000 7f6fb19ff700 1 --2- 172.21.15.64:0/883531041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fac0683b0 0x7f6fac19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:44.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.292+0000 7f6fb19ff700 1 -- 172.21.15.64:0/883531041 >> 172.21.15.64:0/883531041 conn(0x7f6fac074f30 msgr2=0x7f6fac10b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:16:44.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.292+0000 7f6fb19ff700 1 -- 172.21.15.64:0/883531041 shutdown_connections 2023-09-13T03:16:44.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:44.292+0000 7f6fb19ff700 1 -- 172.21.15.64:0/883531041 wait complete. 2023-09-13T03:16:44.902 INFO:tasks.workunit.client.0.smithi064.stdout: 1 75266 2.34 MB/sec execute 49 sec latency 1330.303 ms 2023-09-13T03:16:44.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:16:44 smithi064 ceph-mon[97675]: pgmap v110: 97 pgs: 97 active+clean; 46 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 926 KiB/s rd, 512 KiB/s wr, 93 op/s 2023-09-13T03:16:44.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:16:44 smithi142 ceph-mon[134082]: pgmap v110: 97 pgs: 97 active+clean; 46 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 926 KiB/s rd, 512 KiB/s wr, 93 op/s 2023-09-13T03:16:45.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:16:44 smithi170 ceph-mon[134328]: pgmap v110: 97 pgs: 97 active+clean; 46 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 926 KiB/s rd, 512 KiB/s wr, 93 op/s 2023-09-13T03:16:45.902 INFO:tasks.workunit.client.0.smithi064.stdout: 1 75266 2.29 MB/sec execute 50 sec latency 2330.355 ms 2023-09-13T03:16:46.902 INFO:tasks.workunit.client.0.smithi064.stdout: 1 75266 2.25 MB/sec execute 51 sec latency 3330.442 ms 2023-09-13T03:16:46.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:16:46 smithi064 ceph-mon[97675]: pgmap v111: 97 pgs: 97 active+clean; 46 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 574 KiB/s wr, 115 op/s 2023-09-13T03:16:46.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:16:46 smithi142 ceph-mon[134082]: pgmap v111: 97 pgs: 97 active+clean; 46 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 574 KiB/s wr, 115 op/s 2023-09-13T03:16:47.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:16:46 smithi170 ceph-mon[134328]: pgmap v111: 97 pgs: 97 active+clean; 46 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 574 KiB/s wr, 115 op/s 2023-09-13T03:16:47.903 INFO:tasks.workunit.client.0.smithi064.stdout: 1 75266 2.20 MB/sec execute 52 sec latency 4330.594 ms 2023-09-13T03:16:48.902 INFO:tasks.workunit.client.0.smithi064.stdout: 1 75536 2.18 MB/sec execute 53 sec latency 4948.270 ms 2023-09-13T03:16:48.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:16:48 smithi142 ceph-mon[134082]: pgmap v112: 97 pgs: 97 active+clean; 46 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 326 KiB/s wr, 66 op/s 2023-09-13T03:16:49.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:16:48 smithi170 ceph-mon[134328]: pgmap v112: 97 pgs: 97 active+clean; 46 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 326 KiB/s wr, 66 op/s 2023-09-13T03:16:49.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:16:48 smithi064 ceph-mon[97675]: pgmap v112: 97 pgs: 97 active+clean; 46 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 326 KiB/s wr, 66 op/s 2023-09-13T03:16:49.902 INFO:tasks.workunit.client.0.smithi064.stdout: 1 78033 2.44 MB/sec execute 54 sec latency 12.337 ms 2023-09-13T03:16:50.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:16:50 smithi170 ceph-mon[134328]: pgmap v113: 97 pgs: 97 active+clean; 46 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 326 KiB/s wr, 66 op/s 2023-09-13T03:16:50.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:16:50 smithi064 ceph-mon[97675]: pgmap v113: 97 pgs: 97 active+clean; 46 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 326 KiB/s wr, 66 op/s 2023-09-13T03:16:50.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:16:50 smithi142 ceph-mon[134082]: pgmap v113: 97 pgs: 97 active+clean; 46 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 326 KiB/s wr, 66 op/s 2023-09-13T03:16:50.903 INFO:tasks.workunit.client.0.smithi064.stdout: 1 78753 2.43 MB/sec execute 55 sec latency 268.306 ms 2023-09-13T03:16:51.903 INFO:tasks.workunit.client.0.smithi064.stdout: 1 78753 2.39 MB/sec execute 56 sec latency 1268.438 ms 2023-09-13T03:16:52.903 INFO:tasks.workunit.client.0.smithi064.stdout: 1 78753 2.34 MB/sec execute 57 sec latency 2268.581 ms 2023-09-13T03:16:52.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:16:52 smithi170 ceph-mon[134328]: pgmap v114: 97 pgs: 97 active+clean; 47 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 1.3 MiB/s wr, 138 op/s 2023-09-13T03:16:52.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:16:52 smithi064 ceph-mon[97675]: pgmap v114: 97 pgs: 97 active+clean; 47 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 1.3 MiB/s wr, 138 op/s 2023-09-13T03:16:52.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:16:52 smithi142 ceph-mon[134082]: pgmap v114: 97 pgs: 97 active+clean; 47 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 1.3 MiB/s wr, 138 op/s 2023-09-13T03:16:53.903 INFO:tasks.workunit.client.0.smithi064.stdout: 1 78809 2.31 MB/sec execute 58 sec latency 2891.547 ms 2023-09-13T03:16:54.903 INFO:tasks.workunit.client.0.smithi064.stdout: 1 78809 2.27 MB/sec execute 59 sec latency 1333.492 ms 2023-09-13T03:16:54.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:16:54 smithi170 ceph-mon[134328]: pgmap v115: 97 pgs: 97 active+clean; 47 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2023-09-13T03:16:54.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:16:54 smithi064 ceph-mon[97675]: pgmap v115: 97 pgs: 97 active+clean; 47 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2023-09-13T03:16:54.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:16:54 smithi142 ceph-mon[134082]: pgmap v115: 97 pgs: 97 active+clean; 47 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2023-09-13T03:16:55.903 INFO:tasks.workunit.client.0.smithi064.stdout: 1 78809 2.23 MB/sec execute 60 sec latency 2333.650 ms 2023-09-13T03:16:56.904 INFO:tasks.workunit.client.0.smithi064.stdout: 1 78809 2.19 MB/sec execute 61 sec latency 3333.716 ms 2023-09-13T03:16:56.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:16:56 smithi170 ceph-mon[134328]: pgmap v116: 97 pgs: 97 active+clean; 48 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 1.1 MiB/s wr, 106 op/s 2023-09-13T03:16:56.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:16:56 smithi064 ceph-mon[97675]: pgmap v116: 97 pgs: 97 active+clean; 48 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 1.1 MiB/s wr, 106 op/s 2023-09-13T03:16:56.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:16:56 smithi142 ceph-mon[134082]: pgmap v116: 97 pgs: 97 active+clean; 48 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 1.1 MiB/s wr, 106 op/s 2023-09-13T03:16:57.454 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.453+0000 7f6a849ff700 1 -- 172.21.15.64:0/3149193551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a80100f60 msgr2=0x7f6a8010d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.453+0000 7f6a849ff700 1 --2- 172.21.15.64:0/3149193551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a80100f60 0x7f6a8010d420 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6a7800b0d0 tx=0x7f6a7800b3e0 comp rx=0 tx=0).stop 2023-09-13T03:16:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.454+0000 7f6a849ff700 1 -- 172.21.15.64:0/3149193551 shutdown_connections 2023-09-13T03:16:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.454+0000 7f6a849ff700 1 --2- 172.21.15.64:0/3149193551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a80106660 0x7f6a80106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:57.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.454+0000 7f6a849ff700 1 --2- 172.21.15.64:0/3149193551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a80100f60 0x7f6a8010d420 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:57.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.454+0000 7f6a849ff700 1 --2- 172.21.15.64:0/3149193551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a801005a0 0x7f6a80100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:57.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.454+0000 7f6a849ff700 1 -- 172.21.15.64:0/3149193551 >> 172.21.15.64:0/3149193551 conn(0x7f6a800fc0b0 msgr2=0x7f6a800fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:16:57.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.454+0000 7f6a849ff700 1 -- 172.21.15.64:0/3149193551 shutdown_connections 2023-09-13T03:16:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.454+0000 7f6a849ff700 1 -- 172.21.15.64:0/3149193551 wait complete. 2023-09-13T03:16:57.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.455+0000 7f6a849ff700 1 Processor -- start 2023-09-13T03:16:57.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.455+0000 7f6a849ff700 1 -- start start 2023-09-13T03:16:57.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.456+0000 7f6a849ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a801005a0 0x7f6a80072ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:57.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.456+0000 7f6a849ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a80100f60 0x7f6a80073020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:57.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.456+0000 7f6a849ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a80106660 0x7f6a801a4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:57.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.456+0000 7f6a849ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a801a4ea0 con 0x7f6a80106660 2023-09-13T03:16:57.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.456+0000 7f6a849ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a801a4fe0 con 0x7f6a801005a0 2023-09-13T03:16:57.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.456+0000 7f6a849ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a801a5150 con 0x7f6a80100f60 2023-09-13T03:16:57.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.456+0000 7f6a7e59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a801005a0 0x7f6a80072ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:57.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.456+0000 7f6a7e59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a801005a0 0x7f6a80072ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:52668/0 (socket says 172.21.15.64:52668) 2023-09-13T03:16:57.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.456+0000 7f6a7e59c700 1 -- 172.21.15.64:0/3678710847 learned_addr learned my addr 172.21.15.64:0/3678710847 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:16:57.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.456+0000 7f6a7ed9d700 1 --2- 172.21.15.64:0/3678710847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a80106660 0x7f6a801a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:57.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.456+0000 7f6a7dd9b700 1 --2- 172.21.15.64:0/3678710847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a80100f60 0x7f6a80073020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:57.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.457+0000 7f6a7e59c700 1 -- 172.21.15.64:0/3678710847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a80100f60 msgr2=0x7f6a80073020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:57.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.457+0000 7f6a7e59c700 1 --2- 172.21.15.64:0/3678710847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a80100f60 0x7f6a80073020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:57.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.457+0000 7f6a7e59c700 1 -- 172.21.15.64:0/3678710847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a80106660 msgr2=0x7f6a801a4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:57.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.457+0000 7f6a7e59c700 1 --2- 172.21.15.64:0/3678710847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a80106660 0x7f6a801a4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:57.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.457+0000 7f6a7e59c700 1 -- 172.21.15.64:0/3678710847 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a78009d00 con 0x7f6a801005a0 2023-09-13T03:16:57.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.457+0000 7f6a7e59c700 1 --2- 172.21.15.64:0/3678710847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a801005a0 0x7f6a80072ae0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6a7000e9a0 tx=0x7f6a7000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:16:57.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.457+0000 7f6a6f7fe700 1 -- 172.21.15.64:0/3678710847 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a70009790 con 0x7f6a801005a0 2023-09-13T03:16:57.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.457+0000 7f6a6f7fe700 1 -- 172.21.15.64:0/3678710847 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a7000cbd0 con 0x7f6a801005a0 2023-09-13T03:16:57.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.457+0000 7f6a849ff700 1 -- 172.21.15.64:0/3678710847 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a801a5430 con 0x7f6a801005a0 2023-09-13T03:16:57.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.457+0000 7f6a6f7fe700 1 -- 172.21.15.64:0/3678710847 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a70018680 con 0x7f6a801005a0 2023-09-13T03:16:57.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.457+0000 7f6a849ff700 1 -- 172.21.15.64:0/3678710847 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a801a5990 con 0x7f6a801005a0 2023-09-13T03:16:57.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.461+0000 7f6a6f7fe700 1 -- 172.21.15.64:0/3678710847 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f6a7000cd40 con 0x7f6a801005a0 2023-09-13T03:16:57.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.462+0000 7f6a9822bb80 1 Processor -- start 2023-09-13T03:16:57.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.462+0000 7f6a9822bb80 1 -- start start 2023-09-13T03:16:57.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.462+0000 7f6a6f7fe700 1 --2- 172.21.15.64:0/3678710847 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f6a680a0830 0x7f6a680a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:57.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.462+0000 7f6a6f7fe700 1 -- 172.21.15.64:0/3678710847 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f6a70014070 con 0x7f6a801005a0 2023-09-13T03:16:57.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.463+0000 7f6a9822bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56383749b1a0 0x56383749b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:57.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.463+0000 7f6a9822bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563837495300 0x563837495780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:57.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.463+0000 7f6a7ed9d700 1 --2- 172.21.15.64:0/3678710847 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f6a680a0830 0x7f6a680a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:57.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.463+0000 7f6a9822bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563837495cc0 0x5638374a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:57.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.463+0000 7f6a9822bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638373ee6d0 con 0x563837495cc0 2023-09-13T03:16:57.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.463+0000 7f6a9822bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638373ebff0 con 0x56383749b1a0 2023-09-13T03:16:57.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.463+0000 7f6a9822bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638373e78e0 con 0x563837495300 2023-09-13T03:16:57.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.463+0000 7f6a7ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563837495cc0 0x5638374a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:57.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.463+0000 7f6a7e59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56383749b1a0 0x56383749b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:57.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.463+0000 7f6a7e59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56383749b1a0 0x56383749b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:52684/0 (socket says 172.21.15.64:52684) 2023-09-13T03:16:57.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.463+0000 7f6a7e59c700 1 -- 172.21.15.64:0/355182622 learned_addr learned my addr 172.21.15.64:0/355182622 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:16:57.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.463+0000 7f6a7dd9b700 1 --2- 172.21.15.64:0/355182622 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563837495300 0x563837495780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.464+0000 7f6a7e59c700 1 -- 172.21.15.64:0/355182622 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563837495300 msgr2=0x563837495780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:16:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.464+0000 7f6a7e59c700 1 --2- 172.21.15.64:0/355182622 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563837495300 0x563837495780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.464+0000 7f6a7e59c700 1 -- 172.21.15.64:0/355182622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563837495cc0 msgr2=0x5638374a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:16:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.464+0000 7f6a7e59c700 1 --2- 172.21.15.64:0/355182622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563837495cc0 0x5638374a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.464+0000 7f6a7e59c700 1 -- 172.21.15.64:0/355182622 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563837213d80 con 0x56383749b1a0 2023-09-13T03:16:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.464+0000 7f6a7ed9d700 1 --2- 172.21.15.64:0/3678710847 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f6a680a0830 0x7f6a680a2cf0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6a78006280 tx=0x7f6a78008cf0 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:16:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.464+0000 7f6a7e59c700 1 --2- 172.21.15.64:0/355182622 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56383749b1a0 0x56383749b580 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6a700a1200 tx=0x7f6a70099280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ff6844602f2c2bb server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:16:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.464+0000 7f6a6cff9700 1 -- 172.21.15.64:0/355182622 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a70099dc0 con 0x56383749b1a0 2023-09-13T03:16:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.464+0000 7f6a6cff9700 1 -- 172.21.15.64:0/355182622 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a700996c0 con 0x56383749b1a0 2023-09-13T03:16:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.464+0000 7f6a6cff9700 1 -- 172.21.15.64:0/355182622 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a70099b60 con 0x56383749b1a0 2023-09-13T03:16:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.465+0000 7f6a9822bb80 1 -- 172.21.15.64:0/355182622 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56383749b1a0 msgr2=0x56383749b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.465+0000 7f6a9822bb80 1 --2- 172.21.15.64:0/355182622 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56383749b1a0 0x56383749b580 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6a700a1200 tx=0x7f6a70099280 comp rx=0 tx=0).stop 2023-09-13T03:16:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.465+0000 7f6a9822bb80 1 -- 172.21.15.64:0/355182622 shutdown_connections 2023-09-13T03:16:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.465+0000 7f6a9822bb80 1 --2- 172.21.15.64:0/355182622 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56383749b1a0 0x56383749b580 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.465+0000 7f6a9822bb80 1 --2- 172.21.15.64:0/355182622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563837495cc0 0x5638374a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:57.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.465+0000 7f6a9822bb80 1 --2- 172.21.15.64:0/355182622 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563837495300 0x563837495780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:57.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.465+0000 7f6a9822bb80 1 -- 172.21.15.64:0/355182622 >> 172.21.15.64:0/355182622 conn(0x56383740a030 msgr2=0x563837490560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:16:57.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.465+0000 7f6a9822bb80 1 -- 172.21.15.64:0/355182622 shutdown_connections 2023-09-13T03:16:57.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.465+0000 7f6a9822bb80 1 -- 172.21.15.64:0/355182622 wait complete. 2023-09-13T03:16:57.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.466+0000 7f6a9822bb80 1 -- start start 2023-09-13T03:16:57.477 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.466+0000 7f6a9822bb80 1 Processor -- start 2023-09-13T03:16:57.477 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.467+0000 7f6a9822bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563837495300 0x5638374a1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:57.477 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.467+0000 7f6a9822bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563837495cc0 0x56383749f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:57.478 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.467+0000 7f6a9822bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56383749b1a0 0x56383749f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:57.478 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.467+0000 7f6a9822bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x563837385b10 con 0x563837495cc0 2023-09-13T03:16:57.478 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.467+0000 7f6a9822bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563837385810 con 0x56383749b1a0 2023-09-13T03:16:57.478 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.467+0000 7f6a9822bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638372698c0 con 0x563837495300 2023-09-13T03:16:57.478 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.467+0000 7f6a9822bb80 10 client.0 _ll_register_callbacks cb 0x5638372cfa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:16:57.479 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.467+0000 7f6a7dd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563837495cc0 0x56383749f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:57.479 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.467+0000 7f6a7e59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563837495300 0x5638374a1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:57.479 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.467+0000 7f6a7dd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563837495cc0 0x56383749f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38794/0 (socket says 172.21.15.64:38794) 2023-09-13T03:16:57.479 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.467+0000 7f6a7e59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563837495300 0x5638374a1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:39036/0 (socket says 172.21.15.64:39036) 2023-09-13T03:16:57.479 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.467+0000 7f6a7dd9b700 1 -- 172.21.15.64:0/1620853 learned_addr learned my addr 172.21.15.64:0/1620853 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:16:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.467+0000 7f6a7ed9d700 1 --2- 172.21.15.64:0/1620853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56383749b1a0 0x56383749f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.468+0000 7f6a7dd9b700 1 -- 172.21.15.64:0/1620853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563837495300 msgr2=0x5638374a1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:16:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.468+0000 7f6a7dd9b700 1 --2- 172.21.15.64:0/1620853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563837495300 0x5638374a1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.468+0000 7f6a7dd9b700 1 -- 172.21.15.64:0/1620853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56383749b1a0 msgr2=0x56383749f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:16:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.468+0000 7f6a7dd9b700 1 --2- 172.21.15.64:0/1620853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56383749b1a0 0x56383749f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.468+0000 7f6a7dd9b700 1 -- 172.21.15.64:0/1620853 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a801a5990 con 0x563837495cc0 2023-09-13T03:16:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.468+0000 7f6a7dd9b700 1 --2- 172.21.15.64:0/1620853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563837495cc0 0x56383749f0f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f6a74012700 tx=0x7f6a74012a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c5b1e0a12b6cfbf server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:16:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.468+0000 7f6a65ffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:16:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.468+0000 7f6a65ffb700 1 -- 172.21.15.64:0/1620853 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a74017070 con 0x563837495cc0 2023-09-13T03:16:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.468+0000 7f6a9822bb80 1 -- 172.21.15.64:0/1620853 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56383733f490 con 0x563837495cc0 2023-09-13T03:16:57.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.468+0000 7f6a65ffb700 1 -- 172.21.15.64:0/1620853 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f6a74004320 con 0x563837495cc0 2023-09-13T03:16:57.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.468+0000 7f6a65ffb700 1 -- 172.21.15.64:0/1620853 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a740044e0 con 0x563837495cc0 2023-09-13T03:16:57.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.468+0000 7f6a9822bb80 10 client.16389 fetch_fsmap learned FSMap version 40 2023-09-13T03:16:57.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.468+0000 7f6a9822bb80 1 -- 172.21.15.64:0/1620853 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5638371fcc00 con 0x563837495cc0 2023-09-13T03:16:57.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.469+0000 7f6a65ffb700 1 -- 172.21.15.64:0/1620853 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56383733f490 con 0x563837495cc0 2023-09-13T03:16:57.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.469+0000 7f6a65ffb700 1 -- 172.21.15.64:0/1620853 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6a7400f840 con 0x563837495cc0 2023-09-13T03:16:57.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.469+0000 7f6a9822bb80 10 client.16389 fetch_fsmap finished waiting for FSMap version 40 2023-09-13T03:16:57.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.469+0000 7f6a9822bb80 10 client.16389 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:16:57.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.469+0000 7f6a9822bb80 20 client.16389 populate_metadata read hostname 'smithi064' 2023-09-13T03:16:57.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.469+0000 7f6a9822bb80 1 --2- 172.21.15.64:0/1620853 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5638374af4b0 0x5638374b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:57.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.469+0000 7f6a9822bb80 4 client.16389 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:16:57.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.469+0000 7f6a9822bb80 1 -- 172.21.15.64:0/1620853 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5638373fc840 con 0x5638374af4b0 2023-09-13T03:16:57.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.469+0000 7f6a7e59c700 1 --2- 172.21.15.64:0/1620853 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5638374af4b0 0x5638374b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:57.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.470+0000 7f6a65ffb700 10 client.16389 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:16:57.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.470+0000 7f6a7e59c700 1 --2- 172.21.15.64:0/1620853 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5638374af4b0 0x5638374b18d0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d664bec386e12cd server_cookie=4722d572e34c9a29 in_seq=0 out_seq=0 2023-09-13T03:16:57.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.471+0000 7f6a65ffb700 1 -- 172.21.15.64:0/1620853 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5638373fc840 con 0x5638374af4b0 2023-09-13T03:16:57.485 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.471+0000 7f6a65ffb700 10 client.16389 handle_command_reply: tid=0 2023-09-13T03:16:57.485 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.471+0000 7f6a9822bb80 1 client.16389 shutdown 2023-09-13T03:16:57.485 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.471+0000 7f6a9822bb80 1 -- 172.21.15.64:0/1620853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563837495cc0 msgr2=0x56383749f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:57.485 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.471+0000 7f6a9822bb80 1 --2- 172.21.15.64:0/1620853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563837495cc0 0x56383749f0f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f6a74012700 tx=0x7f6a74012a10 comp rx=0 tx=0).stop 2023-09-13T03:16:57.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.471+0000 7f6a9822bb80 1 -- 172.21.15.64:0/1620853 shutdown_connections 2023-09-13T03:16:57.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.471+0000 7f6a9822bb80 1 --2- 172.21.15.64:0/1620853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56383749b1a0 0x56383749f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:57.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.471+0000 7f6a9822bb80 1 --2- 172.21.15.64:0/1620853 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5638374af4b0 0x5638374b18d0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:57.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.471+0000 7f6a9822bb80 1 --2- 172.21.15.64:0/1620853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563837495300 0x5638374a1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:57.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.471+0000 7f6a9822bb80 1 --2- 172.21.15.64:0/1620853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563837495cc0 0x56383749f0f0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.471+0000 7f6a9822bb80 1 -- 172.21.15.64:0/1620853 >> 172.21.15.64:0/1620853 conn(0x56383740a030 msgr2=0x56383748f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:16:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.471+0000 7f6a65ffb700 0 client.16389 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:16:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.471+0000 7f6a9822bb80 1 -- 172.21.15.64:0/1620853 shutdown_connections 2023-09-13T03:16:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.471+0000 7f6a9822bb80 1 -- 172.21.15.64:0/1620853 wait complete. 2023-09-13T03:16:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.471+0000 7f6a9822bb80 20 client.16389 trim_cache size 0 max 16384 2023-09-13T03:16:57.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.486+0000 7f6a9822bb80 1 Processor -- start 2023-09-13T03:16:57.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.487+0000 7f6a9822bb80 1 -- start start 2023-09-13T03:16:57.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.487+0000 7f6a9822bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563837495300 0x56383748ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:57.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.487+0000 7f6a9822bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563837495cc0 0x563837499150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:57.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.487+0000 7f6a9822bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56383749b1a0 0x563837499690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:57.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.487+0000 7f6a9822bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56383748a8a0 con 0x56383749b1a0 2023-09-13T03:16:57.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.487+0000 7f6a9822bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56383748cfc0 con 0x563837495cc0 2023-09-13T03:16:57.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.487+0000 7f6a9822bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563837490ab0 con 0x563837495300 2023-09-13T03:16:57.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.487+0000 7f6a7ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56383749b1a0 0x563837499690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.487+0000 7f6a7ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56383749b1a0 0x563837499690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38808/0 (socket says 172.21.15.64:38808) 2023-09-13T03:16:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.487+0000 7f6a7e59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563837495300 0x56383748ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.487+0000 7f6a7ed9d700 1 -- 172.21.15.64:0/101766799 learned_addr learned my addr 172.21.15.64:0/101766799 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:16:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.487+0000 7f6a7dd9b700 1 --2- 172.21.15.64:0/101766799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563837495cc0 0x563837499150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.487+0000 7f6a7e59c700 1 -- 172.21.15.64:0/101766799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563837495cc0 msgr2=0x563837499150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:16:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.487+0000 7f6a7e59c700 1 --2- 172.21.15.64:0/101766799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563837495cc0 0x563837499150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.487+0000 7f6a7e59c700 1 -- 172.21.15.64:0/101766799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56383749b1a0 msgr2=0x563837499690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:16:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.487+0000 7f6a7e59c700 1 --2- 172.21.15.64:0/101766799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56383749b1a0 0x563837499690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.487+0000 7f6a7e59c700 1 -- 172.21.15.64:0/101766799 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56383723a1f0 con 0x563837495300 2023-09-13T03:16:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.488+0000 7f6a7e59c700 1 --2- 172.21.15.64:0/101766799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563837495300 0x56383748ff20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6a70095bc0 tx=0x7f6a7001f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad28392e22e5d01a server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:16:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.488+0000 7f6a6d7fa700 1 -- 172.21.15.64:0/101766799 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a70099dc0 con 0x563837495300 2023-09-13T03:16:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.488+0000 7f6a6d7fa700 1 -- 172.21.15.64:0/101766799 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a700a1730 con 0x563837495300 2023-09-13T03:16:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.488+0000 7f6a6d7fa700 1 -- 172.21.15.64:0/101766799 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a70099920 con 0x563837495300 2023-09-13T03:16:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.489+0000 7f6a9822bb80 1 -- 172.21.15.64:0/101766799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563837495300 msgr2=0x56383748ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.489+0000 7f6a9822bb80 1 --2- 172.21.15.64:0/101766799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563837495300 0x56383748ff20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6a70095bc0 tx=0x7f6a7001f000 comp rx=0 tx=0).stop 2023-09-13T03:16:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.489+0000 7f6a9822bb80 1 -- 172.21.15.64:0/101766799 shutdown_connections 2023-09-13T03:16:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.489+0000 7f6a9822bb80 1 --2- 172.21.15.64:0/101766799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56383749b1a0 0x563837499690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.489+0000 7f6a9822bb80 1 --2- 172.21.15.64:0/101766799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563837495300 0x56383748ff20 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.489+0000 7f6a9822bb80 1 --2- 172.21.15.64:0/101766799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563837495cc0 0x563837499150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.489+0000 7f6a9822bb80 1 -- 172.21.15.64:0/101766799 >> 172.21.15.64:0/101766799 conn(0x56383740a030 msgr2=0x5638374aac80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:16:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.489+0000 7f6a9822bb80 1 -- 172.21.15.64:0/101766799 shutdown_connections 2023-09-13T03:16:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.489+0000 7f6a9822bb80 1 -- 172.21.15.64:0/101766799 wait complete. 2023-09-13T03:16:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.489+0000 7f6a9822bb80 1 -- start start 2023-09-13T03:16:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.489+0000 7f6a9822bb80 1 Processor -- start 2023-09-13T03:16:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.490+0000 7f6a9822bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563837495300 0x5638374a4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.490+0000 7f6a9822bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563837495cc0 0x5638374a4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.490+0000 7f6a9822bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56383749b1a0 0x5638374a4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.490+0000 7f6a9822bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638374a6ac0 con 0x563837495300 2023-09-13T03:16:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.490+0000 7f6a9822bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638375022a0 con 0x563837495cc0 2023-09-13T03:16:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.490+0000 7f6a9822bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638375023e0 con 0x56383749b1a0 2023-09-13T03:16:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.490+0000 7f6a9822bb80 10 client.0 _ll_register_callbacks cb 0x7f6a74009d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:16:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.490+0000 7f6a7e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563837495300 0x5638374a4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.490+0000 7f6a7e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563837495300 0x5638374a4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38820/0 (socket says 172.21.15.64:38820) 2023-09-13T03:16:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.490+0000 7f6a7e59c700 1 -- 172.21.15.64:0/266895713 learned_addr learned my addr 172.21.15.64:0/266895713 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:16:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.490+0000 7f6a7ed9d700 1 --2- 172.21.15.64:0/266895713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56383749b1a0 0x5638374a4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.490+0000 7f6a7dd9b700 1 --2- 172.21.15.64:0/266895713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563837495cc0 0x5638374a4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.491+0000 7f6a7e59c700 1 -- 172.21.15.64:0/266895713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56383749b1a0 msgr2=0x5638374a4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:16:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.491+0000 7f6a7e59c700 1 --2- 172.21.15.64:0/266895713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56383749b1a0 0x5638374a4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.491+0000 7f6a7e59c700 1 -- 172.21.15.64:0/266895713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563837495cc0 msgr2=0x5638374a4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:16:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.491+0000 7f6a7e59c700 1 --2- 172.21.15.64:0/266895713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563837495cc0 0x5638374a4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.491+0000 7f6a7e59c700 1 -- 172.21.15.64:0/266895713 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a700b0040 con 0x563837495300 2023-09-13T03:16:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.491+0000 7f6a7dd9b700 1 --2- 172.21.15.64:0/266895713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563837495cc0 0x5638374a4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:16:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.491+0000 7f6a7e59c700 1 --2- 172.21.15.64:0/266895713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563837495300 0x5638374a4510 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6a7000c5b0 tx=0x7f6a700992f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d37556b82b575aa server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:16:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.491+0000 7f6a65ffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:16:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.491+0000 7f6a65ffb700 1 -- 172.21.15.64:0/266895713 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a70099920 con 0x563837495300 2023-09-13T03:16:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.491+0000 7f6a65ffb700 1 -- 172.21.15.64:0/266895713 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f6a70099a80 con 0x563837495300 2023-09-13T03:16:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.491+0000 7f6a9822bb80 1 -- 172.21.15.64:0/266895713 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a700996c0 con 0x563837495300 2023-09-13T03:16:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.491+0000 7f6a65ffb700 1 -- 172.21.15.64:0/266895713 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a70098bb0 con 0x563837495300 2023-09-13T03:16:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.491+0000 7f6a9822bb80 10 client.16395 fetch_fsmap learned FSMap version 40 2023-09-13T03:16:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.491+0000 7f6a9822bb80 1 -- 172.21.15.64:0/266895713 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x563837212250 con 0x563837495300 2023-09-13T03:16:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.492+0000 7f6a65ffb700 1 -- 172.21.15.64:0/266895713 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a700a5080 con 0x563837495300 2023-09-13T03:16:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.492+0000 7f6a65ffb700 1 -- 172.21.15.64:0/266895713 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6a70084150 con 0x563837495300 2023-09-13T03:16:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.492+0000 7f6a9822bb80 10 client.16395 fetch_fsmap finished waiting for FSMap version 40 2023-09-13T03:16:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.492+0000 7f6a9822bb80 10 client.16395 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:16:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.492+0000 7f6a9822bb80 20 client.16395 populate_metadata read hostname 'smithi064' 2023-09-13T03:16:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.492+0000 7f6a9822bb80 1 --2- 172.21.15.64:0/266895713 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x563837503670 0x563837505a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:16:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.492+0000 7f6a9822bb80 4 client.16395 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:16:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.492+0000 7f6a9822bb80 1 -- 172.21.15.64:0/266895713 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5638373e08c0 con 0x563837503670 2023-09-13T03:16:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.492+0000 7f6a7dd9b700 1 --2- 172.21.15.64:0/266895713 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x563837503670 0x563837505a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:16:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.493+0000 7f6a7dd9b700 1 --2- 172.21.15.64:0/266895713 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x563837503670 0x563837505a90 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cb8eecf2faafea9 server_cookie=aff8e9afbd8b1cec in_seq=0 out_seq=0 2023-09-13T03:16:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.493+0000 7f6a65ffb700 10 client.16395 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:16:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.519+0000 7f6a65ffb700 1 -- 172.21.15.64:0/266895713 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5638373e08c0 con 0x563837503670 2023-09-13T03:16:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.519+0000 7f6a65ffb700 10 client.16395 handle_command_reply: tid=0 2023-09-13T03:16:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.519+0000 7f6a9822bb80 1 client.16395 shutdown 2023-09-13T03:16:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.519+0000 7f6a9822bb80 1 -- 172.21.15.64:0/266895713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563837495300 msgr2=0x5638374a4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.519+0000 7f6a9822bb80 1 --2- 172.21.15.64:0/266895713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563837495300 0x5638374a4510 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6a7000c5b0 tx=0x7f6a700992f0 comp rx=0 tx=0).stop 2023-09-13T03:16:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.520+0000 7f6a9822bb80 1 -- 172.21.15.64:0/266895713 shutdown_connections 2023-09-13T03:16:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.520+0000 7f6a9822bb80 1 --2- 172.21.15.64:0/266895713 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x563837503670 0x563837505a90 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.520+0000 7f6a9822bb80 1 --2- 172.21.15.64:0/266895713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56383749b1a0 0x5638374a4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.520+0000 7f6a9822bb80 1 --2- 172.21.15.64:0/266895713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563837495300 0x5638374a4510 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.520+0000 7f6a65ffb700 0 client.16395 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:16:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.520+0000 7f6a9822bb80 1 --2- 172.21.15.64:0/266895713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563837495cc0 0x5638374a4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.520+0000 7f6a9822bb80 1 -- 172.21.15.64:0/266895713 >> 172.21.15.64:0/266895713 conn(0x56383740a030 msgr2=0x56383749a090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:16:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.520+0000 7f6a9822bb80 1 -- 172.21.15.64:0/266895713 shutdown_connections 2023-09-13T03:16:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.520+0000 7f6a9822bb80 1 -- 172.21.15.64:0/266895713 wait complete. 2023-09-13T03:16:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.520+0000 7f6a9822bb80 20 client.16395 trim_cache size 0 max 16384 2023-09-13T03:16:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:16:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:16:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:16:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:16:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.521+0000 7f6a849ff700 1 -- 172.21.15.64:0/3678710847 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f6a680a0830 msgr2=0x7f6a680a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.521+0000 7f6a849ff700 1 --2- 172.21.15.64:0/3678710847 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f6a680a0830 0x7f6a680a2cf0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6a78006280 tx=0x7f6a78008cf0 comp rx=0 tx=0).stop 2023-09-13T03:16:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.521+0000 7f6a849ff700 1 -- 172.21.15.64:0/3678710847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a801005a0 msgr2=0x7f6a80072ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:16:57.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.521+0000 7f6a849ff700 1 --2- 172.21.15.64:0/3678710847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a801005a0 0x7f6a80072ae0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6a7000e9a0 tx=0x7f6a7000ecb0 comp rx=0 tx=0).stop 2023-09-13T03:16:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.521+0000 7f6a849ff700 1 -- 172.21.15.64:0/3678710847 shutdown_connections 2023-09-13T03:16:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.521+0000 7f6a849ff700 1 --2- 172.21.15.64:0/3678710847 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f6a680a0830 0x7f6a680a2cf0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.521+0000 7f6a849ff700 1 --2- 172.21.15.64:0/3678710847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a801005a0 0x7f6a80072ae0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.521+0000 7f6a849ff700 1 --2- 172.21.15.64:0/3678710847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a80106660 0x7f6a801a4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.521+0000 7f6a849ff700 1 --2- 172.21.15.64:0/3678710847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a80100f60 0x7f6a80073020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:16:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.521+0000 7f6a849ff700 1 -- 172.21.15.64:0/3678710847 >> 172.21.15.64:0/3678710847 conn(0x7f6a800fc0b0 msgr2=0x7f6a801042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:16:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.521+0000 7f6a849ff700 1 -- 172.21.15.64:0/3678710847 shutdown_connections 2023-09-13T03:16:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:16:57.521+0000 7f6a849ff700 1 -- 172.21.15.64:0/3678710847 wait complete. 2023-09-13T03:16:57.904 INFO:tasks.workunit.client.0.smithi064.stdout: 1 78809 2.16 MB/sec execute 62 sec latency 4333.866 ms 2023-09-13T03:16:58.904 INFO:tasks.workunit.client.0.smithi064.stdout: 1 79496 2.20 MB/sec execute 63 sec latency 4952.397 ms 2023-09-13T03:16:58.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:16:58 smithi170 ceph-mon[134328]: pgmap v117: 97 pgs: 97 active+clean; 48 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 1.1 MiB/s wr, 84 op/s 2023-09-13T03:16:58.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:16:58 smithi064 ceph-mon[97675]: pgmap v117: 97 pgs: 97 active+clean; 48 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 1.1 MiB/s wr, 84 op/s 2023-09-13T03:16:58.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:16:58 smithi142 ceph-mon[134082]: pgmap v117: 97 pgs: 97 active+clean; 48 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 1.1 MiB/s wr, 84 op/s 2023-09-13T03:16:59.904 INFO:tasks.workunit.client.0.smithi064.stdout: 1 81748 2.36 MB/sec execute 64 sec latency 12.343 ms 2023-09-13T03:17:00.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:17:00 smithi170 ceph-mon[134328]: pgmap v118: 97 pgs: 97 active+clean; 48 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 1.1 MiB/s wr, 84 op/s 2023-09-13T03:17:00.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:17:00 smithi064 ceph-mon[97675]: pgmap v118: 97 pgs: 97 active+clean; 48 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 1.1 MiB/s wr, 84 op/s 2023-09-13T03:17:00.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:17:00 smithi142 ceph-mon[134082]: pgmap v118: 97 pgs: 97 active+clean; 48 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 1.1 MiB/s wr, 84 op/s 2023-09-13T03:17:00.904 INFO:tasks.workunit.client.0.smithi064.stdout: 1 82302 2.35 MB/sec execute 65 sec latency 381.506 ms 2023-09-13T03:17:01.904 INFO:tasks.workunit.client.0.smithi064.stdout: 1 82302 2.31 MB/sec execute 66 sec latency 1381.666 ms 2023-09-13T03:17:02.904 INFO:tasks.workunit.client.0.smithi064.stdout: 1 82302 2.28 MB/sec execute 67 sec latency 2381.752 ms 2023-09-13T03:17:02.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:17:02 smithi170 ceph-mon[134328]: pgmap v119: 97 pgs: 97 active+clean; 46 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 146 op/s 2023-09-13T03:17:02.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:17:02 smithi064 ceph-mon[97675]: pgmap v119: 97 pgs: 97 active+clean; 46 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 146 op/s 2023-09-13T03:17:02.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:17:02 smithi142 ceph-mon[134082]: pgmap v119: 97 pgs: 97 active+clean; 46 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 146 op/s 2023-09-13T03:17:03.904 INFO:tasks.workunit.client.0.smithi064.stdout: 1 82358 2.25 MB/sec execute 68 sec latency 3001.150 ms 2023-09-13T03:17:04.904 INFO:tasks.workunit.client.0.smithi064.stdout: 1 82358 2.22 MB/sec execute 69 sec latency 1330.571 ms 2023-09-13T03:17:04.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:17:04 smithi170 ceph-mon[134328]: pgmap v120: 97 pgs: 97 active+clean; 46 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 318 KiB/s wr, 75 op/s 2023-09-13T03:17:04.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:17:04 smithi064 ceph-mon[97675]: pgmap v120: 97 pgs: 97 active+clean; 46 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 318 KiB/s wr, 75 op/s 2023-09-13T03:17:04.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:17:04 smithi142 ceph-mon[134082]: pgmap v120: 97 pgs: 97 active+clean; 46 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 318 KiB/s wr, 75 op/s 2023-09-13T03:17:05.904 INFO:tasks.workunit.client.0.smithi064.stdout: 1 82358 2.18 MB/sec execute 70 sec latency 2330.639 ms 2023-09-13T03:17:06.897 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:17:06 smithi142 ceph-mon[134082]: pgmap v121: 97 pgs: 97 active+clean; 46 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 358 KiB/s wr, 87 op/s 2023-09-13T03:17:06.897 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:17:06 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:17:06.904 INFO:tasks.workunit.client.0.smithi064.stdout: 1 82358 2.15 MB/sec execute 71 sec latency 3330.709 ms 2023-09-13T03:17:06.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:17:06 smithi170 ceph-mon[134328]: pgmap v121: 97 pgs: 97 active+clean; 46 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 358 KiB/s wr, 87 op/s 2023-09-13T03:17:06.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:17:06 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:17:06.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:17:06 smithi064 ceph-mon[97675]: pgmap v121: 97 pgs: 97 active+clean; 46 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 358 KiB/s wr, 87 op/s 2023-09-13T03:17:06.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:17:06 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:17:07.905 INFO:tasks.workunit.client.0.smithi064.stdout: 1 82358 2.12 MB/sec execute 72 sec latency 4330.834 ms 2023-09-13T03:17:08.905 INFO:tasks.workunit.client.0.smithi064.stdout: 1 82426 2.10 MB/sec execute 73 sec latency 4949.206 ms 2023-09-13T03:17:08.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:17:08 smithi170 ceph-mon[134328]: pgmap v122: 97 pgs: 97 active+clean; 46 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 307 KiB/s wr, 74 op/s 2023-09-13T03:17:08.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:17:08 smithi064 ceph-mon[97675]: pgmap v122: 97 pgs: 97 active+clean; 46 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 307 KiB/s wr, 74 op/s 2023-09-13T03:17:08.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:17:08 smithi142 ceph-mon[134082]: pgmap v122: 97 pgs: 97 active+clean; 46 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 307 KiB/s wr, 74 op/s 2023-09-13T03:17:09.905 INFO:tasks.workunit.client.0.smithi064.stdout: 1 82728 2.09 MB/sec execute 74 sec latency 1139.854 ms 2023-09-13T03:17:10.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:17:10 smithi170 ceph-mon[134328]: pgmap v123: 97 pgs: 97 active+clean; 46 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 307 KiB/s wr, 74 op/s 2023-09-13T03:17:10.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:17:10 smithi064 ceph-mon[97675]: pgmap v123: 97 pgs: 97 active+clean; 46 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 307 KiB/s wr, 74 op/s 2023-09-13T03:17:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.675+0000 7fed637ff700 1 -- 172.21.15.64:0/4158677644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed5c106660 msgr2=0x7fed5c106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:17:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.675+0000 7fed637ff700 1 --2- 172.21.15.64:0/4158677644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed5c106660 0x7fed5c106a40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fed580099c0 tx=0x7fed58009cd0 comp rx=0 tx=0).stop 2023-09-13T03:17:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.675+0000 7fed637ff700 1 -- 172.21.15.64:0/4158677644 shutdown_connections 2023-09-13T03:17:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.675+0000 7fed637ff700 1 --2- 172.21.15.64:0/4158677644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed5c106660 0x7fed5c106a40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:10.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.675+0000 7fed637ff700 1 --2- 172.21.15.64:0/4158677644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fed5c100f60 0x7fed5c10d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:10.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.675+0000 7fed637ff700 1 --2- 172.21.15.64:0/4158677644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed5c1005a0 0x7fed5c100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:10.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.675+0000 7fed637ff700 1 -- 172.21.15.64:0/4158677644 >> 172.21.15.64:0/4158677644 conn(0x7fed5c0fc050 msgr2=0x7fed5c0fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:17:10.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.675+0000 7fed637ff700 1 -- 172.21.15.64:0/4158677644 shutdown_connections 2023-09-13T03:17:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.675+0000 7fed637ff700 1 -- 172.21.15.64:0/4158677644 wait complete. 2023-09-13T03:17:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.677+0000 7fed637ff700 1 Processor -- start 2023-09-13T03:17:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.677+0000 7fed637ff700 1 -- start start 2023-09-13T03:17:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.677+0000 7fed637ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fed5c1005a0 0x7fed5c19e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.677+0000 7fed637ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed5c100f60 0x7fed5c19ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.677+0000 7fed637ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed5c106660 0x7fed5c198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.677+0000 7fed637ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5c198e60 con 0x7fed5c106660 2023-09-13T03:17:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.677+0000 7fed637ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5c198fd0 con 0x7fed5c100f60 2023-09-13T03:17:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.677+0000 7fed637ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5c199190 con 0x7fed5c1005a0 2023-09-13T03:17:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.677+0000 7fed6159b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fed5c1005a0 0x7fed5c19e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.677+0000 7fed6159b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fed5c1005a0 0x7fed5c19e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:34382/0 (socket says 172.21.15.64:34382) 2023-09-13T03:17:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.677+0000 7fed6159b700 1 -- 172.21.15.64:0/1789037974 learned_addr learned my addr 172.21.15.64:0/1789037974 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:17:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.678+0000 7fed60d9a700 1 --2- 172.21.15.64:0/1789037974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed5c100f60 0x7fed5c19ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.678+0000 7fed61d9c700 1 --2- 172.21.15.64:0/1789037974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed5c106660 0x7fed5c198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.678+0000 7fed6159b700 1 -- 172.21.15.64:0/1789037974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed5c100f60 msgr2=0x7fed5c19ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:17:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.678+0000 7fed6159b700 1 --2- 172.21.15.64:0/1789037974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed5c100f60 0x7fed5c19ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.678+0000 7fed6159b700 1 -- 172.21.15.64:0/1789037974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed5c106660 msgr2=0x7fed5c198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:17:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.678+0000 7fed6159b700 1 --2- 172.21.15.64:0/1789037974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed5c106660 0x7fed5c198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.678+0000 7fed6159b700 1 -- 172.21.15.64:0/1789037974 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed580095f0 con 0x7fed5c1005a0 2023-09-13T03:17:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.678+0000 7fed60d9a700 1 --2- 172.21.15.64:0/1789037974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed5c100f60 0x7fed5c19ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:17:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.678+0000 7fed6159b700 1 --2- 172.21.15.64:0/1789037974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fed5c1005a0 0x7fed5c19e8c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fed5800b580 tx=0x7fed5800bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:17:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.678+0000 7fed527fc700 1 -- 172.21.15.64:0/1789037974 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5801d070 con 0x7fed5c1005a0 2023-09-13T03:17:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.678+0000 7fed527fc700 1 -- 172.21.15.64:0/1789037974 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed58005050 con 0x7fed5c1005a0 2023-09-13T03:17:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.678+0000 7fed527fc700 1 -- 172.21.15.64:0/1789037974 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed58004a40 con 0x7fed5c1005a0 2023-09-13T03:17:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.679+0000 7fed637ff700 1 -- 172.21.15.64:0/1789037974 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed5c199410 con 0x7fed5c1005a0 2023-09-13T03:17:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.679+0000 7fed637ff700 1 -- 172.21.15.64:0/1789037974 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed5c1a5840 con 0x7fed5c1005a0 2023-09-13T03:17:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.682+0000 7fed527fc700 1 -- 172.21.15.64:0/1789037974 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fed580051c0 con 0x7fed5c1005a0 2023-09-13T03:17:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.683+0000 7fed77211b80 1 Processor -- start 2023-09-13T03:17:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.683+0000 7fed527fc700 1 --2- 172.21.15.64:0/1789037974 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fed480a07b0 0x7fed480a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.683+0000 7fed527fc700 1 -- 172.21.15.64:0/1789037974 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fed580ccce0 con 0x7fed5c1005a0 2023-09-13T03:17:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.684+0000 7fed77211b80 1 -- start start 2023-09-13T03:17:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.684+0000 7fed60d9a700 1 --2- 172.21.15.64:0/1789037974 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fed480a07b0 0x7fed480a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.684+0000 7fed77211b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56211f2e21a0 0x56211f2e2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.684+0000 7fed77211b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56211f2dc300 0x56211f2dc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.684+0000 7fed77211b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56211f2dccc0 0x56211f2e8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.684+0000 7fed77211b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56211f2356d0 con 0x56211f2dc300 2023-09-13T03:17:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.684+0000 7fed77211b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56211f232ff0 con 0x56211f2dccc0 2023-09-13T03:17:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.684+0000 7fed77211b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56211f22e8e0 con 0x56211f2e21a0 2023-09-13T03:17:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.684+0000 7fed6159b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56211f2e21a0 0x56211f2e2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.684+0000 7fed61d9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56211f2dccc0 0x56211f2e8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.684+0000 7fed6159b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56211f2e21a0 0x56211f2e2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:34386/0 (socket says 172.21.15.64:34386) 2023-09-13T03:17:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.684+0000 7fed6159b700 1 -- 172.21.15.64:0/3495320409 learned_addr learned my addr 172.21.15.64:0/3495320409 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:17:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.684+0000 7fed60d9a700 1 --2- 172.21.15.64:0/3495320409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56211f2dc300 0x56211f2dc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.685+0000 7fed61d9c700 1 -- 172.21.15.64:0/3495320409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56211f2e21a0 msgr2=0x56211f2e2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:17:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.685+0000 7fed61d9c700 1 --2- 172.21.15.64:0/3495320409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56211f2e21a0 0x56211f2e2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.685+0000 7fed61d9c700 1 -- 172.21.15.64:0/3495320409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56211f2dc300 msgr2=0x56211f2dc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:17:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.685+0000 7fed61d9c700 1 --2- 172.21.15.64:0/3495320409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56211f2dc300 0x56211f2dc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.685+0000 7fed61d9c700 1 -- 172.21.15.64:0/3495320409 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56211f05ad80 con 0x56211f2dccc0 2023-09-13T03:17:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.685+0000 7fed60d9a700 1 --2- 172.21.15.64:0/1789037974 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fed480a07b0 0x7fed480a2c70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fed5c19a0d0 tx=0x7fed4c013c70 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:17:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.685+0000 7fed61d9c700 1 --2- 172.21.15.64:0/3495320409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56211f2dccc0 0x56211f2e8f30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fed54011550 tx=0x7fed54009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99185335339ed68c server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:17:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.685+0000 7fed477fe700 1 -- 172.21.15.64:0/3495320409 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5400e040 con 0x56211f2dccc0 2023-09-13T03:17:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.685+0000 7fed477fe700 1 -- 172.21.15.64:0/3495320409 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed5400f040 con 0x56211f2dccc0 2023-09-13T03:17:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.685+0000 7fed477fe700 1 -- 172.21.15.64:0/3495320409 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed54004310 con 0x56211f2dccc0 2023-09-13T03:17:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.686+0000 7fed77211b80 1 -- 172.21.15.64:0/3495320409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56211f2dccc0 msgr2=0x56211f2e8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:17:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.686+0000 7fed77211b80 1 --2- 172.21.15.64:0/3495320409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56211f2dccc0 0x56211f2e8f30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fed54011550 tx=0x7fed54009920 comp rx=0 tx=0).stop 2023-09-13T03:17:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.686+0000 7fed77211b80 1 -- 172.21.15.64:0/3495320409 shutdown_connections 2023-09-13T03:17:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.686+0000 7fed77211b80 1 --2- 172.21.15.64:0/3495320409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56211f2dccc0 0x56211f2e8f30 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.686+0000 7fed77211b80 1 --2- 172.21.15.64:0/3495320409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56211f2e21a0 0x56211f2e2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.686+0000 7fed77211b80 1 --2- 172.21.15.64:0/3495320409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56211f2dc300 0x56211f2dc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.686+0000 7fed77211b80 1 -- 172.21.15.64:0/3495320409 >> 172.21.15.64:0/3495320409 conn(0x56211f251030 msgr2=0x56211f2d7560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:17:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.686+0000 7fed77211b80 1 -- 172.21.15.64:0/3495320409 shutdown_connections 2023-09-13T03:17:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.686+0000 7fed77211b80 1 -- 172.21.15.64:0/3495320409 wait complete. 2023-09-13T03:17:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.687+0000 7fed77211b80 1 -- start start 2023-09-13T03:17:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.687+0000 7fed77211b80 1 Processor -- start 2023-09-13T03:17:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.687+0000 7fed77211b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56211f2dc300 0x56211f2e8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.687+0000 7fed77211b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56211f2dccc0 0x56211f2e6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.687+0000 7fed77211b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56211f2e21a0 0x56211f2e6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:10.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.687+0000 7fed77211b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56211f1ccb10 con 0x56211f2e21a0 2023-09-13T03:17:10.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.687+0000 7fed77211b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56211f1cc810 con 0x56211f2dc300 2023-09-13T03:17:10.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.687+0000 7fed77211b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56211f0b08c0 con 0x56211f2dccc0 2023-09-13T03:17:10.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.688+0000 7fed77211b80 10 client.0 _ll_register_callbacks cb 0x56211f116a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:17:10.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.688+0000 7fed6159b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56211f2dc300 0x56211f2e8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:10.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.688+0000 7fed6159b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56211f2dc300 0x56211f2e8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:60254/0 (socket says 172.21.15.64:60254) 2023-09-13T03:17:10.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.688+0000 7fed6159b700 1 -- 172.21.15.64:0/1562584614 learned_addr learned my addr 172.21.15.64:0/1562584614 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:17:10.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.688+0000 7fed61d9c700 1 --2- 172.21.15.64:0/1562584614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56211f2e21a0 0x56211f2e6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:10.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.688+0000 7fed60d9a700 1 --2- 172.21.15.64:0/1562584614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56211f2dccc0 0x56211f2e6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:10.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.688+0000 7fed6159b700 1 -- 172.21.15.64:0/1562584614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56211f2dccc0 msgr2=0x56211f2e6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:17:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.688+0000 7fed6159b700 1 --2- 172.21.15.64:0/1562584614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56211f2dccc0 0x56211f2e6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.688+0000 7fed6159b700 1 -- 172.21.15.64:0/1562584614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56211f2e21a0 msgr2=0x56211f2e6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:17:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.688+0000 7fed6159b700 1 --2- 172.21.15.64:0/1562584614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56211f2e21a0 0x56211f2e6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.688+0000 7fed6159b700 1 -- 172.21.15.64:0/1562584614 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed54011230 con 0x56211f2dc300 2023-09-13T03:17:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.688+0000 7fed6159b700 1 --2- 172.21.15.64:0/1562584614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56211f2dc300 0x56211f2e8d40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fed580036c0 tx=0x7fed5809a280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7081cceb71637a39 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:17:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.688+0000 7fed44ff9700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:17:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.688+0000 7fed44ff9700 1 -- 172.21.15.64:0/1562584614 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5809a670 con 0x56211f2dc300 2023-09-13T03:17:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.688+0000 7fed77211b80 1 -- 172.21.15.64:0/1562584614 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56211f186490 con 0x56211f2dc300 2023-09-13T03:17:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.689+0000 7fed44ff9700 1 -- 172.21.15.64:0/1562584614 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fed58099700 con 0x56211f2dc300 2023-09-13T03:17:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.689+0000 7fed44ff9700 1 -- 172.21.15.64:0/1562584614 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed580b8170 con 0x56211f2dc300 2023-09-13T03:17:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.689+0000 7fed77211b80 10 client.26447 fetch_fsmap learned FSMap version 41 2023-09-13T03:17:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.689+0000 7fed77211b80 1 -- 172.21.15.64:0/1562584614 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x56211f043c00 con 0x56211f2dc300 2023-09-13T03:17:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.690+0000 7fed44ff9700 1 -- 172.21.15.64:0/1562584614 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed58064080 con 0x56211f2dc300 2023-09-13T03:17:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.690+0000 7fed44ff9700 1 -- 172.21.15.64:0/1562584614 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 41) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fed58004ba0 con 0x56211f2dc300 2023-09-13T03:17:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.690+0000 7fed77211b80 10 client.26447 fetch_fsmap finished waiting for FSMap version 41 2023-09-13T03:17:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.690+0000 7fed77211b80 10 client.26447 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:17:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.690+0000 7fed77211b80 20 client.26447 populate_metadata read hostname 'smithi064' 2023-09-13T03:17:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.690+0000 7fed77211b80 1 --2- 172.21.15.64:0/1562584614 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56211f2f5fa0 0x56211f2f83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.690+0000 7fed77211b80 4 client.26447 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:17:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.690+0000 7fed77211b80 1 -- 172.21.15.64:0/1562584614 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56211f243840 con 0x56211f2f5fa0 2023-09-13T03:17:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.690+0000 7fed60d9a700 1 --2- 172.21.15.64:0/1562584614 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56211f2f5fa0 0x56211f2f83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.691+0000 7fed44ff9700 10 client.26447 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:17:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.691+0000 7fed60d9a700 1 --2- 172.21.15.64:0/1562584614 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56211f2f5fa0 0x56211f2f83c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da0b95adf4f4be72 server_cookie=129213609e245018 in_seq=0 out_seq=0 2023-09-13T03:17:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.691+0000 7fed44ff9700 1 -- 172.21.15.64:0/1562584614 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56211f243840 con 0x56211f2f5fa0 2023-09-13T03:17:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.691+0000 7fed44ff9700 10 client.26447 handle_command_reply: tid=0 2023-09-13T03:17:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.692+0000 7fed77211b80 1 client.26447 shutdown 2023-09-13T03:17:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.692+0000 7fed77211b80 1 -- 172.21.15.64:0/1562584614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56211f2dc300 msgr2=0x56211f2e8d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:17:10.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.692+0000 7fed77211b80 1 --2- 172.21.15.64:0/1562584614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56211f2dc300 0x56211f2e8d40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fed580036c0 tx=0x7fed5809a280 comp rx=0 tx=0).stop 2023-09-13T03:17:10.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.692+0000 7fed77211b80 1 -- 172.21.15.64:0/1562584614 shutdown_connections 2023-09-13T03:17:10.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.692+0000 7fed77211b80 1 --2- 172.21.15.64:0/1562584614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56211f2dc300 0x56211f2e8d40 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:10.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.692+0000 7fed77211b80 1 --2- 172.21.15.64:0/1562584614 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56211f2f5fa0 0x56211f2f83c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.692+0000 7fed77211b80 1 --2- 172.21.15.64:0/1562584614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56211f2e21a0 0x56211f2e6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.692+0000 7fed77211b80 1 --2- 172.21.15.64:0/1562584614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56211f2dccc0 0x56211f2e6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.692+0000 7fed44ff9700 0 client.26447 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:17:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.692+0000 7fed77211b80 1 -- 172.21.15.64:0/1562584614 >> 172.21.15.64:0/1562584614 conn(0x56211f251030 msgr2=0x56211f2d67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:17:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.692+0000 7fed77211b80 1 -- 172.21.15.64:0/1562584614 shutdown_connections 2023-09-13T03:17:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.692+0000 7fed77211b80 1 -- 172.21.15.64:0/1562584614 wait complete. 2023-09-13T03:17:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.692+0000 7fed77211b80 20 client.26447 trim_cache size 0 max 16384 2023-09-13T03:17:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.708+0000 7fed77211b80 1 Processor -- start 2023-09-13T03:17:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.708+0000 7fed77211b80 1 -- start start 2023-09-13T03:17:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.708+0000 7fed77211b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56211f2dc300 0x56211f2ed530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.708+0000 7fed77211b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56211f2dccc0 0x56211f2e88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.709+0000 7fed77211b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56211f2e21a0 0x56211f2d7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.709+0000 7fed77211b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56211f2d3fc0 con 0x56211f2dccc0 2023-09-13T03:17:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.709+0000 7fed77211b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56211f2d8d10 con 0x56211f2e21a0 2023-09-13T03:17:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.709+0000 7fed77211b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56211f2eda70 con 0x56211f2dc300 2023-09-13T03:17:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.709+0000 7fed60d9a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56211f2dccc0 0x56211f2e88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.709+0000 7fed6159b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56211f2dc300 0x56211f2ed530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.709+0000 7fed6159b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56211f2dc300 0x56211f2ed530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:34408/0 (socket says 172.21.15.64:34408) 2023-09-13T03:17:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.709+0000 7fed6159b700 1 -- 172.21.15.64:0/445089583 learned_addr learned my addr 172.21.15.64:0/445089583 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:17:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.709+0000 7fed61d9c700 1 --2- 172.21.15.64:0/445089583 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56211f2e21a0 0x56211f2d7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.709+0000 7fed6159b700 1 -- 172.21.15.64:0/445089583 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56211f2e21a0 msgr2=0x56211f2d7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:17:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.709+0000 7fed6159b700 1 --2- 172.21.15.64:0/445089583 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56211f2e21a0 0x56211f2d7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.709+0000 7fed6159b700 1 -- 172.21.15.64:0/445089583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56211f2dccc0 msgr2=0x56211f2e88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:17:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.709+0000 7fed6159b700 1 --2- 172.21.15.64:0/445089583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56211f2dccc0 0x56211f2e88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.709+0000 7fed6159b700 1 -- 172.21.15.64:0/445089583 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56211f0811f0 con 0x56211f2dc300 2023-09-13T03:17:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.709+0000 7fed6159b700 1 --2- 172.21.15.64:0/445089583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56211f2dc300 0x56211f2ed530 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fed58096710 tx=0x7fed5809a9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c2131239dbf0582 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:17:10.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.710+0000 7fed47fff700 1 -- 172.21.15.64:0/445089583 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed580b8170 con 0x56211f2dc300 2023-09-13T03:17:10.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.710+0000 7fed47fff700 1 -- 172.21.15.64:0/445089583 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56211f186490 con 0x56211f2dc300 2023-09-13T03:17:10.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.710+0000 7fed47fff700 1 -- 172.21.15.64:0/445089583 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed58099700 con 0x56211f2dc300 2023-09-13T03:17:10.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.710+0000 7fed77211b80 1 -- 172.21.15.64:0/445089583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56211f2dc300 msgr2=0x56211f2ed530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:17:10.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.710+0000 7fed77211b80 1 --2- 172.21.15.64:0/445089583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56211f2dc300 0x56211f2ed530 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fed58096710 tx=0x7fed5809a9c0 comp rx=0 tx=0).stop 2023-09-13T03:17:10.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.711+0000 7fed77211b80 1 -- 172.21.15.64:0/445089583 shutdown_connections 2023-09-13T03:17:10.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.711+0000 7fed77211b80 1 --2- 172.21.15.64:0/445089583 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56211f2e21a0 0x56211f2d7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:10.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.711+0000 7fed77211b80 1 --2- 172.21.15.64:0/445089583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56211f2dc300 0x56211f2ed530 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:10.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.711+0000 7fed77211b80 1 --2- 172.21.15.64:0/445089583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56211f2dccc0 0x56211f2e88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:10.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.711+0000 7fed77211b80 1 -- 172.21.15.64:0/445089583 >> 172.21.15.64:0/445089583 conn(0x56211f251030 msgr2=0x56211f2e0190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:17:10.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.711+0000 7fed77211b80 1 -- 172.21.15.64:0/445089583 shutdown_connections 2023-09-13T03:17:10.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.711+0000 7fed77211b80 1 -- 172.21.15.64:0/445089583 wait complete. 2023-09-13T03:17:10.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.711+0000 7fed77211b80 1 -- start start 2023-09-13T03:17:10.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.711+0000 7fed77211b80 1 Processor -- start 2023-09-13T03:17:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.712+0000 7fed77211b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56211f2dc300 0x56211f33df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.712+0000 7fed77211b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56211f2dccc0 0x56211f33e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.712+0000 7fed77211b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56211f2e21a0 0x56211f348de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.712+0000 7fed77211b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56211f349320 con 0x56211f2e21a0 2023-09-13T03:17:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.712+0000 7fed77211b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56211f349460 con 0x56211f2dc300 2023-09-13T03:17:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.712+0000 7fed77211b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56211f3495f0 con 0x56211f2dccc0 2023-09-13T03:17:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.712+0000 7fed77211b80 10 client.0 _ll_register_callbacks cb 0x56211f116a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:17:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.712+0000 7fed6159b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56211f2dc300 0x56211f33df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.712+0000 7fed6159b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56211f2dc300 0x56211f33df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:60278/0 (socket says 172.21.15.64:60278) 2023-09-13T03:17:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.713+0000 7fed6159b700 1 -- 172.21.15.64:0/2603426140 learned_addr learned my addr 172.21.15.64:0/2603426140 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:17:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.713+0000 7fed61d9c700 1 --2- 172.21.15.64:0/2603426140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56211f2e21a0 0x56211f348de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.713+0000 7fed60d9a700 1 --2- 172.21.15.64:0/2603426140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56211f2dccc0 0x56211f33e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.713+0000 7fed6159b700 1 -- 172.21.15.64:0/2603426140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56211f2dccc0 msgr2=0x56211f33e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:17:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.713+0000 7fed6159b700 1 --2- 172.21.15.64:0/2603426140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56211f2dccc0 0x56211f33e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.713+0000 7fed6159b700 1 -- 172.21.15.64:0/2603426140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56211f2e21a0 msgr2=0x56211f348de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:17:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.713+0000 7fed6159b700 1 --2- 172.21.15.64:0/2603426140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56211f2e21a0 0x56211f348de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.713+0000 7fed6159b700 1 -- 172.21.15.64:0/2603426140 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56211f043c00 con 0x56211f2dc300 2023-09-13T03:17:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.713+0000 7fed61d9c700 1 --2- 172.21.15.64:0/2603426140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56211f2e21a0 0x56211f348de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:17:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.713+0000 7fed6159b700 1 --2- 172.21.15.64:0/2603426140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56211f2dc300 0x56211f33df50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fed58096c80 tx=0x7fed580a2010 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d32414c87f154cd server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:17:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.713+0000 7fed44ff9700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:17:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.713+0000 7fed44ff9700 1 -- 172.21.15.64:0/2603426140 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed580cad30 con 0x56211f2dc300 2023-09-13T03:17:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.713+0000 7fed77211b80 1 -- 172.21.15.64:0/2603426140 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed5400f040 con 0x56211f2dc300 2023-09-13T03:17:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.713+0000 7fed44ff9700 1 -- 172.21.15.64:0/2603426140 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fed580b8170 con 0x56211f2dc300 2023-09-13T03:17:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.714+0000 7fed44ff9700 1 -- 172.21.15.64:0/2603426140 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed58099910 con 0x56211f2dc300 2023-09-13T03:17:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.714+0000 7fed77211b80 10 client.26453 fetch_fsmap learned FSMap version 41 2023-09-13T03:17:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.714+0000 7fed77211b80 1 -- 172.21.15.64:0/2603426140 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x56211f059250 con 0x56211f2dc300 2023-09-13T03:17:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.714+0000 7fed44ff9700 1 -- 172.21.15.64:0/2603426140 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed580a6020 con 0x56211f2dc300 2023-09-13T03:17:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.714+0000 7fed44ff9700 1 -- 172.21.15.64:0/2603426140 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 41) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fed580b2300 con 0x56211f2dc300 2023-09-13T03:17:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.714+0000 7fed77211b80 10 client.26453 fetch_fsmap finished waiting for FSMap version 41 2023-09-13T03:17:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.714+0000 7fed77211b80 10 client.26453 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:17:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.714+0000 7fed77211b80 20 client.26453 populate_metadata read hostname 'smithi064' 2023-09-13T03:17:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.714+0000 7fed77211b80 1 --2- 172.21.15.64:0/2603426140 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56211f34a850 0x56211f34cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.714+0000 7fed77211b80 4 client.26453 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:17:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.714+0000 7fed77211b80 1 -- 172.21.15.64:0/2603426140 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56211f2278c0 con 0x56211f34a850 2023-09-13T03:17:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.714+0000 7fed60d9a700 1 --2- 172.21.15.64:0/2603426140 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56211f34a850 0x56211f34cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.715+0000 7fed44ff9700 10 client.26453 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:17:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.715+0000 7fed60d9a700 1 --2- 172.21.15.64:0/2603426140 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56211f34a850 0x56211f34cc40 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc7797cb2d42d86e server_cookie=6b36cdaa0917275c in_seq=0 out_seq=0 2023-09-13T03:17:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.742+0000 7fed44ff9700 1 -- 172.21.15.64:0/2603426140 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56211f2278c0 con 0x56211f34a850 2023-09-13T03:17:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.742+0000 7fed44ff9700 10 client.26453 handle_command_reply: tid=0 2023-09-13T03:17:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.742+0000 7fed77211b80 1 client.26453 shutdown 2023-09-13T03:17:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.742+0000 7fed77211b80 1 -- 172.21.15.64:0/2603426140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56211f2dc300 msgr2=0x56211f33df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:17:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.742+0000 7fed77211b80 1 --2- 172.21.15.64:0/2603426140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56211f2dc300 0x56211f33df50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fed58096c80 tx=0x7fed580a2010 comp rx=0 tx=0).stop 2023-09-13T03:17:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.742+0000 7fed77211b80 1 -- 172.21.15.64:0/2603426140 shutdown_connections 2023-09-13T03:17:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.742+0000 7fed77211b80 1 --2- 172.21.15.64:0/2603426140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56211f2dc300 0x56211f33df50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.742+0000 7fed77211b80 1 --2- 172.21.15.64:0/2603426140 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56211f34a850 0x56211f34cc40 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.742+0000 7fed77211b80 1 --2- 172.21.15.64:0/2603426140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56211f2e21a0 0x56211f348de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.742+0000 7fed44ff9700 0 client.26453 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:17:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.742+0000 7fed77211b80 1 --2- 172.21.15.64:0/2603426140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56211f2dccc0 0x56211f33e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.742+0000 7fed77211b80 1 -- 172.21.15.64:0/2603426140 >> 172.21.15.64:0/2603426140 conn(0x56211f251030 msgr2=0x56211f2d7090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:17:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.742+0000 7fed77211b80 1 -- 172.21.15.64:0/2603426140 shutdown_connections 2023-09-13T03:17:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.742+0000 7fed77211b80 1 -- 172.21.15.64:0/2603426140 wait complete. 2023-09-13T03:17:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.742+0000 7fed77211b80 20 client.26453 trim_cache size 0 max 16384 2023-09-13T03:17:10.746 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:17:10.746 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:17:10.746 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:17:10.746 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:17:10.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.743+0000 7fed637ff700 1 -- 172.21.15.64:0/1789037974 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fed480a07b0 msgr2=0x7fed480a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:17:10.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.743+0000 7fed637ff700 1 --2- 172.21.15.64:0/1789037974 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fed480a07b0 0x7fed480a2c70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fed5c19a0d0 tx=0x7fed4c013c70 comp rx=0 tx=0).stop 2023-09-13T03:17:10.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.743+0000 7fed637ff700 1 -- 172.21.15.64:0/1789037974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fed5c1005a0 msgr2=0x7fed5c19e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:17:10.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.743+0000 7fed637ff700 1 --2- 172.21.15.64:0/1789037974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fed5c1005a0 0x7fed5c19e8c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fed5800b580 tx=0x7fed5800bfd0 comp rx=0 tx=0).stop 2023-09-13T03:17:10.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.743+0000 7fed637ff700 1 -- 172.21.15.64:0/1789037974 shutdown_connections 2023-09-13T03:17:10.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.743+0000 7fed637ff700 1 --2- 172.21.15.64:0/1789037974 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fed480a07b0 0x7fed480a2c70 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:10.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.743+0000 7fed637ff700 1 --2- 172.21.15.64:0/1789037974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed5c106660 0x7fed5c198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:10.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.743+0000 7fed637ff700 1 --2- 172.21.15.64:0/1789037974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fed5c1005a0 0x7fed5c19e8c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:10.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.743+0000 7fed637ff700 1 --2- 172.21.15.64:0/1789037974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed5c100f60 0x7fed5c19ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:10.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.743+0000 7fed637ff700 1 -- 172.21.15.64:0/1789037974 >> 172.21.15.64:0/1789037974 conn(0x7fed5c0fc050 msgr2=0x7fed5c0fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:17:10.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.743+0000 7fed637ff700 1 -- 172.21.15.64:0/1789037974 shutdown_connections 2023-09-13T03:17:10.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:10.743+0000 7fed637ff700 1 -- 172.21.15.64:0/1789037974 wait complete. 2023-09-13T03:17:10.750 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:17:10 smithi142 ceph-mon[134082]: pgmap v123: 97 pgs: 97 active+clean; 46 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 307 KiB/s wr, 74 op/s 2023-09-13T03:17:10.905 INFO:tasks.workunit.client.0.smithi064.stdout: 1 84842 2.21 MB/sec execute 75 sec latency 12.554 ms 2023-09-13T03:17:11.905 INFO:tasks.workunit.client.0.smithi064.stdout: 1 85967 2.26 MB/sec execute 76 sec latency 49.178 ms 2023-09-13T03:17:12.905 INFO:tasks.workunit.client.0.smithi064.stdout: 1 85967 2.24 MB/sec execute 77 sec latency 1049.340 ms 2023-09-13T03:17:12.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:17:12 smithi170 ceph-mon[134328]: pgmap v124: 97 pgs: 97 active+clean; 50 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 120 op/s 2023-09-13T03:17:12.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:17:12 smithi064 ceph-mon[97675]: pgmap v124: 97 pgs: 97 active+clean; 50 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 120 op/s 2023-09-13T03:17:12.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:17:12 smithi142 ceph-mon[134082]: pgmap v124: 97 pgs: 97 active+clean; 50 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 120 op/s 2023-09-13T03:17:13.905 INFO:tasks.workunit.client.0.smithi064.stdout: 1 86601 2.27 MB/sec execute 78 sec latency 1667.322 ms 2023-09-13T03:17:14.905 INFO:tasks.workunit.client.0.smithi064.stdout: 1 87937 2.31 MB/sec execute 79 sec latency 15.271 ms 2023-09-13T03:17:14.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:17:14 smithi170 ceph-mon[134328]: pgmap v125: 97 pgs: 97 active+clean; 50 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 771 KiB/s wr, 57 op/s 2023-09-13T03:17:14.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:17:14 smithi064 ceph-mon[97675]: pgmap v125: 97 pgs: 97 active+clean; 50 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 771 KiB/s wr, 57 op/s 2023-09-13T03:17:14.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:17:14 smithi142 ceph-mon[134082]: pgmap v125: 97 pgs: 97 active+clean; 50 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 771 KiB/s wr, 57 op/s 2023-09-13T03:17:15.905 INFO:tasks.workunit.client.0.smithi064.stdout: 1 89295 2.39 MB/sec execute 80 sec latency 12.226 ms 2023-09-13T03:17:16.902 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:17:16 smithi142 ceph-mon[134082]: pgmap v126: 97 pgs: 97 active+clean; 53 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 125 op/s 2023-09-13T03:17:16.905 INFO:tasks.workunit.client.0.smithi064.stdout: 1 89392 2.36 MB/sec execute 81 sec latency 924.496 ms 2023-09-13T03:17:16.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:17:16 smithi170 ceph-mon[134328]: pgmap v126: 97 pgs: 97 active+clean; 53 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 125 op/s 2023-09-13T03:17:16.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:17:16 smithi064 ceph-mon[97675]: pgmap v126: 97 pgs: 97 active+clean; 53 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 125 op/s 2023-09-13T03:17:17.906 INFO:tasks.workunit.client.0.smithi064.stdout: 1 89392 2.33 MB/sec execute 82 sec latency 1924.579 ms 2023-09-13T03:17:18.906 INFO:tasks.workunit.client.0.smithi064.stdout: 1 89448 2.30 MB/sec execute 83 sec latency 2544.390 ms 2023-09-13T03:17:18.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:17:18 smithi170 ceph-mon[134328]: pgmap v127: 97 pgs: 97 active+clean; 53 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2023-09-13T03:17:18.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:17:18 smithi064 ceph-mon[97675]: pgmap v127: 97 pgs: 97 active+clean; 53 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2023-09-13T03:17:18.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:17:18 smithi142 ceph-mon[134082]: pgmap v127: 97 pgs: 97 active+clean; 53 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2023-09-13T03:17:19.906 INFO:tasks.workunit.client.0.smithi064.stdout: 1 89448 2.27 MB/sec execute 84 sec latency 1335.759 ms 2023-09-13T03:17:20.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:17:20 smithi170 ceph-mon[134328]: pgmap v128: 97 pgs: 97 active+clean; 53 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2023-09-13T03:17:20.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:17:20 smithi064 ceph-mon[97675]: pgmap v128: 97 pgs: 97 active+clean; 53 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2023-09-13T03:17:20.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:17:20 smithi142 ceph-mon[134082]: pgmap v128: 97 pgs: 97 active+clean; 53 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2023-09-13T03:17:20.906 INFO:tasks.workunit.client.0.smithi064.stdout: 1 89448 2.25 MB/sec execute 85 sec latency 2335.836 ms 2023-09-13T03:17:21.906 INFO:tasks.workunit.client.0.smithi064.stdout: 1 89448 2.22 MB/sec execute 86 sec latency 3335.889 ms 2023-09-13T03:17:22.906 INFO:tasks.workunit.client.0.smithi064.stdout: 1 89448 2.20 MB/sec execute 87 sec latency 4335.937 ms 2023-09-13T03:17:22.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:17:22 smithi170 ceph-mon[134328]: pgmap v129: 97 pgs: 97 active+clean; 54 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 128 op/s 2023-09-13T03:17:22.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:17:22 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:17:22.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:17:22 smithi064 ceph-mon[97675]: pgmap v129: 97 pgs: 97 active+clean; 54 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 128 op/s 2023-09-13T03:17:22.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:17:22 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:17:22.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:17:22 smithi142 ceph-mon[134082]: pgmap v129: 97 pgs: 97 active+clean; 54 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 128 op/s 2023-09-13T03:17:22.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:17:22 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:17:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.903+0000 7fc5b91ff700 1 -- 172.21.15.64:0/3312532533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5b410cdd0 msgr2=0x7fc5b410f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:17:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.903+0000 7fc5b91ff700 1 --2- 172.21.15.64:0/3312532533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5b410cdd0 0x7fc5b410f240 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac009f90 tx=0x7fc5ac00b330 comp rx=0 tx=0).stop 2023-09-13T03:17:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.903+0000 7fc5b91ff700 1 -- 172.21.15.64:0/3312532533 shutdown_connections 2023-09-13T03:17:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.903+0000 7fc5b91ff700 1 --2- 172.21.15.64:0/3312532533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5b410cdd0 0x7fc5b410f240 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.903+0000 7fc5b91ff700 1 --2- 172.21.15.64:0/3312532533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5b4072a30 0x7fc5b4072e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.903+0000 7fc5b91ff700 1 --2- 172.21.15.64:0/3312532533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5b4073350 0x7fc5b410c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:23.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.903+0000 7fc5b91ff700 1 -- 172.21.15.64:0/3312532533 >> 172.21.15.64:0/3312532533 conn(0x7fc5b40fc060 msgr2=0x7fc5b40fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:17:23.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.904+0000 7fc5b91ff700 1 -- 172.21.15.64:0/3312532533 shutdown_connections 2023-09-13T03:17:23.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.904+0000 7fc5b91ff700 1 -- 172.21.15.64:0/3312532533 wait complete. 2023-09-13T03:17:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.905+0000 7fc5b91ff700 1 Processor -- start 2023-09-13T03:17:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.905+0000 7fc5b91ff700 1 -- start start 2023-09-13T03:17:23.907 INFO:tasks.workunit.client.0.smithi064.stdout: 1 89516 2.17 MB/sec execute 88 sec latency 4952.634 ms 2023-09-13T03:17:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.906+0000 7fc5b91ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5b4072a30 0x7fc5b419e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.906+0000 7fc5b91ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5b4073350 0x7fc5b419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.906+0000 7fc5b91ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5b410cdd0 0x7fc5b41987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.906+0000 7fc5b91ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b4198e60 con 0x7fc5b4072a30 2023-09-13T03:17:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.906+0000 7fc5b91ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b4198fd0 con 0x7fc5b410cdd0 2023-09-13T03:17:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.906+0000 7fc5b91ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b4199190 con 0x7fc5b4073350 2023-09-13T03:17:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.906+0000 7fc5b2d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5b4072a30 0x7fc5b419e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.906+0000 7fc5b259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5b4073350 0x7fc5b419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.906+0000 7fc5b2d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5b4072a30 0x7fc5b419e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52886/0 (socket says 172.21.15.64:52886) 2023-09-13T03:17:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.906+0000 7fc5b2d9d700 1 -- 172.21.15.64:0/3317679065 learned_addr learned my addr 172.21.15.64:0/3317679065 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:17:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.906+0000 7fc5b359e700 1 --2- 172.21.15.64:0/3317679065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5b410cdd0 0x7fc5b41987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.907+0000 7fc5b259c700 1 -- 172.21.15.64:0/3317679065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5b410cdd0 msgr2=0x7fc5b41987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:17:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.907+0000 7fc5b259c700 1 --2- 172.21.15.64:0/3317679065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5b410cdd0 0x7fc5b41987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.907+0000 7fc5b259c700 1 -- 172.21.15.64:0/3317679065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5b4072a30 msgr2=0x7fc5b419e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:17:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.907+0000 7fc5b259c700 1 --2- 172.21.15.64:0/3317679065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5b4072a30 0x7fc5b419e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.907+0000 7fc5b259c700 1 -- 172.21.15.64:0/3317679065 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ac009c40 con 0x7fc5b4073350 2023-09-13T03:17:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.907+0000 7fc5b2d9d700 1 --2- 172.21.15.64:0/3317679065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5b4072a30 0x7fc5b419e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:17:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.907+0000 7fc5b359e700 1 --2- 172.21.15.64:0/3317679065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5b410cdd0 0x7fc5b41987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:17:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.907+0000 7fc5b259c700 1 --2- 172.21.15.64:0/3317679065 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5b4073350 0x7fc5b419eed0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc5a800e9c0 tx=0x7fc5a800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:17:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.907+0000 7fc59bfff700 1 -- 172.21.15.64:0/3317679065 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a800c9c0 con 0x7fc5b4073350 2023-09-13T03:17:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.907+0000 7fc5b91ff700 1 -- 172.21.15.64:0/3317679065 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5b4199470 con 0x7fc5b4073350 2023-09-13T03:17:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.907+0000 7fc59bfff700 1 -- 172.21.15.64:0/3317679065 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5a8010430 con 0x7fc5b4073350 2023-09-13T03:17:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.907+0000 7fc59bfff700 1 -- 172.21.15.64:0/3317679065 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a8018680 con 0x7fc5b4073350 2023-09-13T03:17:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.907+0000 7fc5b91ff700 1 -- 172.21.15.64:0/3317679065 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5b41a5980 con 0x7fc5b4073350 2023-09-13T03:17:23.914 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:17:23 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:17:23.914 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:17:23 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:17:23.915 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:17:23 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:17:23.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.912+0000 7fc59bfff700 1 -- 172.21.15.64:0/3317679065 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fc5a8010860 con 0x7fc5b4073350 2023-09-13T03:17:23.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.913+0000 7fc5ccbc9b80 1 Processor -- start 2023-09-13T03:17:23.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.913+0000 7fc5ccbc9b80 1 -- start start 2023-09-13T03:17:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.913+0000 7fc5ccbc9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605526021a0 0x560552602580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.913+0000 7fc5ccbc9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605525fc300 0x5605525fc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.913+0000 7fc5ccbc9b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605525fccc0 0x560552608f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.913+0000 7fc5ccbc9b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605525556d0 con 0x5605525fccc0 2023-09-13T03:17:23.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.913+0000 7fc5ccbc9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560552552ff0 con 0x5605526021a0 2023-09-13T03:17:23.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.913+0000 7fc5ccbc9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56055254e8e0 con 0x5605525fc300 2023-09-13T03:17:23.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.914+0000 7fc5b259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605525fccc0 0x560552608f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:23.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.914+0000 7fc5b2d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605525fc300 0x5605525fc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:23.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.914+0000 7fc5b259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605525fccc0 0x560552608f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52902/0 (socket says 172.21.15.64:52902) 2023-09-13T03:17:23.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.914+0000 7fc5b259c700 1 -- 172.21.15.64:0/3136264979 learned_addr learned my addr 172.21.15.64:0/3136264979 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:17:23.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.914+0000 7fc5b359e700 1 --2- 172.21.15.64:0/3136264979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605526021a0 0x560552602580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:23.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.914+0000 7fc59bfff700 1 --2- 172.21.15.64:0/3317679065 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fc59c0a07b0 0x7fc59c0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:23.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.914+0000 7fc5b2d9d700 1 -- 172.21.15.64:0/3136264979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605526021a0 msgr2=0x560552602580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:17:23.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.914+0000 7fc5b2d9d700 1 --2- 172.21.15.64:0/3136264979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605526021a0 0x560552602580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:23.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.914+0000 7fc5b2d9d700 1 -- 172.21.15.64:0/3136264979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605525fccc0 msgr2=0x560552608f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:17:23.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.914+0000 7fc5b2d9d700 1 --2- 172.21.15.64:0/3136264979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605525fccc0 0x560552608f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:23.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.914+0000 7fc5b2d9d700 1 -- 172.21.15.64:0/3136264979 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56055237ad80 con 0x5605525fc300 2023-09-13T03:17:23.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.914+0000 7fc59bfff700 1 -- 172.21.15.64:0/3317679065 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc5a8014070 con 0x7fc5b4073350 2023-09-13T03:17:23.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.914+0000 7fc5b359e700 1 --2- 172.21.15.64:0/3317679065 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fc59c0a07b0 0x7fc59c0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:23.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.915+0000 7fc5b2d9d700 1 --2- 172.21.15.64:0/3136264979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605525fc300 0x5605525fc780 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4010480 tx=0x7fc5a4010790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d0220ab3956fc8c server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:17:23.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.915+0000 7fc5997fa700 1 -- 172.21.15.64:0/3136264979 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a4004a30 con 0x5605525fc300 2023-09-13T03:17:23.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.915+0000 7fc5997fa700 1 -- 172.21.15.64:0/3136264979 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5a4015070 con 0x5605525fc300 2023-09-13T03:17:23.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.915+0000 7fc5997fa700 1 -- 172.21.15.64:0/3136264979 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a401a6e0 con 0x5605525fc300 2023-09-13T03:17:23.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.916+0000 7fc5b359e700 1 --2- 172.21.15.64:0/3317679065 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fc59c0a07b0 0x7fc59c0a2c70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac007970 tx=0x7fc5ac009920 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:17:23.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.916+0000 7fc5ccbc9b80 1 -- 172.21.15.64:0/3136264979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605525fc300 msgr2=0x5605525fc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:17:23.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.916+0000 7fc5ccbc9b80 1 --2- 172.21.15.64:0/3136264979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605525fc300 0x5605525fc780 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4010480 tx=0x7fc5a4010790 comp rx=0 tx=0).stop 2023-09-13T03:17:23.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.916+0000 7fc5ccbc9b80 1 -- 172.21.15.64:0/3136264979 shutdown_connections 2023-09-13T03:17:23.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.916+0000 7fc5ccbc9b80 1 --2- 172.21.15.64:0/3136264979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605526021a0 0x560552602580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:23.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.916+0000 7fc5ccbc9b80 1 --2- 172.21.15.64:0/3136264979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605525fccc0 0x560552608f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:23.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.916+0000 7fc5ccbc9b80 1 --2- 172.21.15.64:0/3136264979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605525fc300 0x5605525fc780 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:23.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.916+0000 7fc5ccbc9b80 1 -- 172.21.15.64:0/3136264979 >> 172.21.15.64:0/3136264979 conn(0x560552571030 msgr2=0x5605525f7560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:17:23.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.916+0000 7fc5ccbc9b80 1 -- 172.21.15.64:0/3136264979 shutdown_connections 2023-09-13T03:17:23.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.916+0000 7fc5ccbc9b80 1 -- 172.21.15.64:0/3136264979 wait complete. 2023-09-13T03:17:23.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.916+0000 7fc5ccbc9b80 1 -- start start 2023-09-13T03:17:23.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.917+0000 7fc5ccbc9b80 1 Processor -- start 2023-09-13T03:17:23.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.917+0000 7fc5ccbc9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605525fc300 0x560552608de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:23.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.917+0000 7fc5ccbc9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605525fccc0 0x560552605ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:23.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.917+0000 7fc5ccbc9b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605526021a0 0x560552606570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:23.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.917+0000 7fc5ccbc9b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605524ecb10 con 0x5605526021a0 2023-09-13T03:17:23.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.917+0000 7fc5ccbc9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605524ec810 con 0x5605525fccc0 2023-09-13T03:17:23.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.917+0000 7fc5ccbc9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605523d08c0 con 0x5605525fc300 2023-09-13T03:17:23.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.917+0000 7fc5ccbc9b80 10 client.0 _ll_register_callbacks cb 0x560552436a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:17:23.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.917+0000 7fc5b2d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605525fc300 0x560552608de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:23.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.917+0000 7fc5b359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605526021a0 0x560552606570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:23.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.917+0000 7fc5b2d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605525fc300 0x560552608de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:55152/0 (socket says 172.21.15.64:55152) 2023-09-13T03:17:23.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.917+0000 7fc5b2d9d700 1 -- 172.21.15.64:0/3490442110 learned_addr learned my addr 172.21.15.64:0/3490442110 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:17:23.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.918+0000 7fc5b259c700 1 --2- 172.21.15.64:0/3490442110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605525fccc0 0x560552605ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:23.928 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:17:23 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:17:23.928 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:17:23 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:17:23.929 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:17:23 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:17:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.918+0000 7fc5b2d9d700 1 -- 172.21.15.64:0/3490442110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605525fccc0 msgr2=0x560552605ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:17:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.918+0000 7fc5b2d9d700 1 --2- 172.21.15.64:0/3490442110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605525fccc0 0x560552605ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.918+0000 7fc5b2d9d700 1 -- 172.21.15.64:0/3490442110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605526021a0 msgr2=0x560552606570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:17:23.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.918+0000 7fc5b2d9d700 1 --2- 172.21.15.64:0/3490442110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605526021a0 0x560552606570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:23.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.918+0000 7fc5b2d9d700 1 -- 172.21.15.64:0/3490442110 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a4007cd0 con 0x5605525fc300 2023-09-13T03:17:23.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.918+0000 7fc5b259c700 1 --2- 172.21.15.64:0/3490442110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605525fccc0 0x560552605ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:17:23.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.918+0000 7fc5b2d9d700 1 --2- 172.21.15.64:0/3490442110 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605525fc300 0x560552608de0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4007720 tx=0x7fc5a4010f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fc5a82554303ac5 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:17:23.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.918+0000 7fc596ff5700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:17:23.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.918+0000 7fc596ff5700 1 -- 172.21.15.64:0/3490442110 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a4004220 con 0x5605525fc300 2023-09-13T03:17:23.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.919+0000 7fc5ccbc9b80 1 -- 172.21.15.64:0/3490442110 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5605524a6490 con 0x5605525fc300 2023-09-13T03:17:23.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.919+0000 7fc596ff5700 1 -- 172.21.15.64:0/3490442110 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc5a4004380 con 0x5605525fc300 2023-09-13T03:17:23.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.919+0000 7fc596ff5700 1 -- 172.21.15.64:0/3490442110 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5a4004540 con 0x5605525fc300 2023-09-13T03:17:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.919+0000 7fc5ccbc9b80 10 client.26464 fetch_fsmap learned FSMap version 41 2023-09-13T03:17:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.919+0000 7fc5ccbc9b80 1 -- 172.21.15.64:0/3490442110 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x560552363c00 con 0x5605525fc300 2023-09-13T03:17:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.919+0000 7fc596ff5700 1 -- 172.21.15.64:0/3490442110 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5a4030020 con 0x5605525fc300 2023-09-13T03:17:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.919+0000 7fc596ff5700 1 -- 172.21.15.64:0/3490442110 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 41) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fc5a402a070 con 0x5605525fc300 2023-09-13T03:17:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.920+0000 7fc5ccbc9b80 10 client.26464 fetch_fsmap finished waiting for FSMap version 41 2023-09-13T03:17:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.920+0000 7fc5ccbc9b80 10 client.26464 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:17:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.920+0000 7fc5ccbc9b80 20 client.26464 populate_metadata read hostname 'smithi064' 2023-09-13T03:17:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.920+0000 7fc5ccbc9b80 1 --2- 172.21.15.64:0/3490442110 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x560552616230 0x560552618620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.920+0000 7fc5ccbc9b80 4 client.26464 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:17:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.920+0000 7fc5ccbc9b80 1 -- 172.21.15.64:0/3490442110 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560552563840 con 0x560552616230 2023-09-13T03:17:23.938 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.920+0000 7fc5b259c700 1 --2- 172.21.15.64:0/3490442110 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x560552616230 0x560552618620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:23.938 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.921+0000 7fc5b259c700 1 --2- 172.21.15.64:0/3490442110 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x560552616230 0x560552618620 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8213cc69bb8e51f server_cookie=baf6b09647aedbf9 in_seq=0 out_seq=0 2023-09-13T03:17:23.938 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.921+0000 7fc596ff5700 10 client.26464 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:17:23.938 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.921+0000 7fc596ff5700 1 -- 172.21.15.64:0/3490442110 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560552563840 con 0x560552616230 2023-09-13T03:17:23.938 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.921+0000 7fc596ff5700 10 client.26464 handle_command_reply: tid=0 2023-09-13T03:17:23.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.922+0000 7fc5ccbc9b80 1 client.26464 shutdown 2023-09-13T03:17:23.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.922+0000 7fc5ccbc9b80 1 -- 172.21.15.64:0/3490442110 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605525fc300 msgr2=0x560552608de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:17:23.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.922+0000 7fc5ccbc9b80 1 --2- 172.21.15.64:0/3490442110 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605525fc300 0x560552608de0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4007720 tx=0x7fc5a4010f60 comp rx=0 tx=0).stop 2023-09-13T03:17:23.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.922+0000 7fc5ccbc9b80 1 -- 172.21.15.64:0/3490442110 shutdown_connections 2023-09-13T03:17:23.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.922+0000 7fc5ccbc9b80 1 --2- 172.21.15.64:0/3490442110 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x560552616230 0x560552618620 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:23.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.922+0000 7fc5ccbc9b80 1 --2- 172.21.15.64:0/3490442110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605526021a0 0x560552606570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:23.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.922+0000 7fc5ccbc9b80 1 --2- 172.21.15.64:0/3490442110 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605525fc300 0x560552608de0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:23.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.922+0000 7fc5ccbc9b80 1 --2- 172.21.15.64:0/3490442110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605525fccc0 0x560552605ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:23.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.922+0000 7fc596ff5700 0 client.26464 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:17:23.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.922+0000 7fc5ccbc9b80 1 -- 172.21.15.64:0/3490442110 >> 172.21.15.64:0/3490442110 conn(0x560552571030 msgr2=0x5605525f67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:17:23.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.922+0000 7fc5ccbc9b80 1 -- 172.21.15.64:0/3490442110 shutdown_connections 2023-09-13T03:17:23.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.922+0000 7fc5ccbc9b80 1 -- 172.21.15.64:0/3490442110 wait complete. 2023-09-13T03:17:23.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.922+0000 7fc5ccbc9b80 20 client.26464 trim_cache size 0 max 16384 2023-09-13T03:17:23.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.937+0000 7fc5ccbc9b80 1 Processor -- start 2023-09-13T03:17:23.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.938+0000 7fc5ccbc9b80 1 -- start start 2023-09-13T03:17:23.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.938+0000 7fc5ccbc9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605525fc300 0x560552608910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:23.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.938+0000 7fc5ccbc9b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605525fccc0 0x560552600fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:23.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.938+0000 7fc5ccbc9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605526021a0 0x560552601510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:23.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.938+0000 7fc5ccbc9b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560552601ac0 con 0x5605525fccc0 2023-09-13T03:17:23.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.938+0000 7fc5ccbc9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605526045a0 con 0x5605525fc300 2023-09-13T03:17:23.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.938+0000 7fc5ccbc9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560552600bc0 con 0x5605526021a0 2023-09-13T03:17:23.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.938+0000 7fc5b259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605525fccc0 0x560552600fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:23.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.938+0000 7fc5b259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605525fccc0 0x560552600fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52922/0 (socket says 172.21.15.64:52922) 2023-09-13T03:17:23.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.938+0000 7fc5b259c700 1 -- 172.21.15.64:0/2176227726 learned_addr learned my addr 172.21.15.64:0/2176227726 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:17:23.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.938+0000 7fc5b2d9d700 1 --2- 172.21.15.64:0/2176227726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605525fc300 0x560552608910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.938+0000 7fc5b359e700 1 --2- 172.21.15.64:0/2176227726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605526021a0 0x560552601510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.939+0000 7fc5b259c700 1 -- 172.21.15.64:0/2176227726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605526021a0 msgr2=0x560552601510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:17:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.939+0000 7fc5b259c700 1 --2- 172.21.15.64:0/2176227726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605526021a0 0x560552601510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.939+0000 7fc5b259c700 1 -- 172.21.15.64:0/2176227726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605525fc300 msgr2=0x560552608910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:17:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.939+0000 7fc5b259c700 1 --2- 172.21.15.64:0/2176227726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605525fc300 0x560552608910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.939+0000 7fc5b259c700 1 -- 172.21.15.64:0/2176227726 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605523a11f0 con 0x5605525fccc0 2023-09-13T03:17:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.939+0000 7fc5b2d9d700 1 --2- 172.21.15.64:0/2176227726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605525fc300 0x560552608910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:17:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.939+0000 7fc5b259c700 1 --2- 172.21.15.64:0/2176227726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605525fccc0 0x560552600fd0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc5a800c5b0 tx=0x7fc5a80cb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d758eaf83ff482b7 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:17:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.939+0000 7fc599ffb700 1 -- 172.21.15.64:0/2176227726 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a80cf3b0 con 0x5605525fccc0 2023-09-13T03:17:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.939+0000 7fc599ffb700 1 -- 172.21.15.64:0/2176227726 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5a8098f90 con 0x5605525fccc0 2023-09-13T03:17:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.939+0000 7fc599ffb700 1 -- 172.21.15.64:0/2176227726 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a80a14c0 con 0x5605525fccc0 2023-09-13T03:17:23.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.940+0000 7fc5ccbc9b80 1 -- 172.21.15.64:0/2176227726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605525fccc0 msgr2=0x560552600fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:17:23.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.940+0000 7fc5ccbc9b80 1 --2- 172.21.15.64:0/2176227726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605525fccc0 0x560552600fd0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc5a800c5b0 tx=0x7fc5a80cb040 comp rx=0 tx=0).stop 2023-09-13T03:17:23.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.940+0000 7fc5ccbc9b80 1 -- 172.21.15.64:0/2176227726 shutdown_connections 2023-09-13T03:17:23.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.940+0000 7fc5ccbc9b80 1 --2- 172.21.15.64:0/2176227726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605525fc300 0x560552608910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:23.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.940+0000 7fc5ccbc9b80 1 --2- 172.21.15.64:0/2176227726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605526021a0 0x560552601510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:23.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.940+0000 7fc5ccbc9b80 1 --2- 172.21.15.64:0/2176227726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605525fccc0 0x560552600fd0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:23.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.940+0000 7fc5ccbc9b80 1 -- 172.21.15.64:0/2176227726 >> 172.21.15.64:0/2176227726 conn(0x560552571030 msgr2=0x560552600050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:17:23.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.940+0000 7fc5ccbc9b80 1 -- 172.21.15.64:0/2176227726 shutdown_connections 2023-09-13T03:17:23.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.940+0000 7fc5ccbc9b80 1 -- 172.21.15.64:0/2176227726 wait complete. 2023-09-13T03:17:23.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.940+0000 7fc5ccbc9b80 1 -- start start 2023-09-13T03:17:23.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.940+0000 7fc5ccbc9b80 1 Processor -- start 2023-09-13T03:17:23.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.941+0000 7fc5ccbc9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605525fc300 0x56055260d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:23.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.941+0000 7fc5ccbc9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605525fccc0 0x56055260d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:23.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.941+0000 7fc5ccbc9b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605526021a0 0x560552609760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:23.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.941+0000 7fc5ccbc9b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560552609e40 con 0x5605526021a0 2023-09-13T03:17:23.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.941+0000 7fc5ccbc9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560552609fb0 con 0x5605525fc300 2023-09-13T03:17:23.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.941+0000 7fc5ccbc9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56055260a170 con 0x5605525fccc0 2023-09-13T03:17:23.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.941+0000 7fc5ccbc9b80 10 client.0 _ll_register_callbacks cb 0x560552436a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:17:23.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.941+0000 7fc5b2d9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605525fc300 0x56055260d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:23.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.941+0000 7fc5b359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605526021a0 0x560552609760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:23.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.941+0000 7fc5b2d9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605525fc300 0x56055260d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:58536/0 (socket says 172.21.15.64:58536) 2023-09-13T03:17:23.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.941+0000 7fc5b2d9d700 1 -- 172.21.15.64:0/1208569112 learned_addr learned my addr 172.21.15.64:0/1208569112 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:17:23.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.941+0000 7fc5b259c700 1 --2- 172.21.15.64:0/1208569112 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605525fccc0 0x56055260d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:23.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.942+0000 7fc5b359e700 1 -- 172.21.15.64:0/1208569112 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605525fccc0 msgr2=0x56055260d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:17:23.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.942+0000 7fc5b359e700 1 --2- 172.21.15.64:0/1208569112 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605525fccc0 0x56055260d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:23.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.942+0000 7fc5b359e700 1 -- 172.21.15.64:0/1208569112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605525fc300 msgr2=0x56055260d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:17:23.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.942+0000 7fc5b359e700 1 --2- 172.21.15.64:0/1208569112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605525fc300 0x56055260d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:23.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.942+0000 7fc5b359e700 1 -- 172.21.15.64:0/1208569112 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560552363c00 con 0x5605526021a0 2023-09-13T03:17:23.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.942+0000 7fc5b2d9d700 1 --2- 172.21.15.64:0/1208569112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605525fc300 0x56055260d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:17:23.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.942+0000 7fc5b359e700 1 --2- 172.21.15.64:0/1208569112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605526021a0 0x560552609760 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc5b419f690 tx=0x7fc5ac0096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13aa057b2ff3dc60 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:17:23.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.942+0000 7fc596ff5700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:17:23.956 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.942+0000 7fc596ff5700 1 -- 172.21.15.64:0/1208569112 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac00e040 con 0x5605526021a0 2023-09-13T03:17:23.956 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.942+0000 7fc5ccbc9b80 1 -- 172.21.15.64:0/1208569112 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc5a4015070 con 0x5605526021a0 2023-09-13T03:17:23.956 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.942+0000 7fc596ff5700 1 -- 172.21.15.64:0/1208569112 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc5ac005460 con 0x5605526021a0 2023-09-13T03:17:23.956 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.943+0000 7fc596ff5700 1 -- 172.21.15.64:0/1208569112 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5ac005620 con 0x5605526021a0 2023-09-13T03:17:23.956 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.943+0000 7fc5ccbc9b80 10 client.16431 fetch_fsmap learned FSMap version 41 2023-09-13T03:17:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.943+0000 7fc5ccbc9b80 1 -- 172.21.15.64:0/1208569112 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7fc5b41a5980 con 0x5605526021a0 2023-09-13T03:17:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.943+0000 7fc596ff5700 1 -- 172.21.15.64:0/1208569112 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5a4015070 con 0x5605526021a0 2023-09-13T03:17:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.943+0000 7fc596ff5700 1 -- 172.21.15.64:0/1208569112 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 41) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fc5ac03dbc0 con 0x5605526021a0 2023-09-13T03:17:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.943+0000 7fc5ccbc9b80 10 client.16431 fetch_fsmap finished waiting for FSMap version 41 2023-09-13T03:17:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.943+0000 7fc5ccbc9b80 10 client.16431 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:17:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.943+0000 7fc5ccbc9b80 20 client.16431 populate_metadata read hostname 'smithi064' 2023-09-13T03:17:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.943+0000 7fc5ccbc9b80 1 --2- 172.21.15.64:0/1208569112 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56055266b3a0 0x56055266d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.943+0000 7fc5ccbc9b80 4 client.16431 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:17:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.943+0000 7fc5ccbc9b80 1 -- 172.21.15.64:0/1208569112 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5605525478c0 con 0x56055266b3a0 2023-09-13T03:17:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.943+0000 7fc5b2d9d700 1 --2- 172.21.15.64:0/1208569112 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56055266b3a0 0x56055266d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.944+0000 7fc5b2d9d700 1 --2- 172.21.15.64:0/1208569112 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56055266b3a0 0x56055266d7c0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c0de2ef5725ded0 server_cookie=357a4e930b90a9a5 in_seq=0 out_seq=0 2023-09-13T03:17:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.944+0000 7fc596ff5700 10 client.16431 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:17:23.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.974+0000 7fc596ff5700 1 -- 172.21.15.64:0/1208569112 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5605525478c0 con 0x56055266b3a0 2023-09-13T03:17:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.974+0000 7fc596ff5700 10 client.16431 handle_command_reply: tid=0 2023-09-13T03:17:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.975+0000 7fc5ccbc9b80 1 client.16431 shutdown 2023-09-13T03:17:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.975+0000 7fc5ccbc9b80 1 -- 172.21.15.64:0/1208569112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605526021a0 msgr2=0x560552609760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:17:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.975+0000 7fc5ccbc9b80 1 --2- 172.21.15.64:0/1208569112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605526021a0 0x560552609760 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc5b419f690 tx=0x7fc5ac0096c0 comp rx=0 tx=0).stop 2023-09-13T03:17:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.975+0000 7fc5ccbc9b80 1 -- 172.21.15.64:0/1208569112 shutdown_connections 2023-09-13T03:17:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.975+0000 7fc5ccbc9b80 1 --2- 172.21.15.64:0/1208569112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605525fc300 0x56055260d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.975+0000 7fc5ccbc9b80 1 --2- 172.21.15.64:0/1208569112 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56055266b3a0 0x56055266d7c0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.975+0000 7fc5ccbc9b80 1 --2- 172.21.15.64:0/1208569112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605526021a0 0x560552609760 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.975+0000 7fc5ccbc9b80 1 --2- 172.21.15.64:0/1208569112 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605525fccc0 0x56055260d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.975+0000 7fc596ff5700 0 client.16431 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:17:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.975+0000 7fc5ccbc9b80 1 -- 172.21.15.64:0/1208569112 >> 172.21.15.64:0/1208569112 conn(0x560552571030 msgr2=0x560552600050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:17:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.975+0000 7fc5ccbc9b80 1 -- 172.21.15.64:0/1208569112 shutdown_connections 2023-09-13T03:17:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.975+0000 7fc5ccbc9b80 1 -- 172.21.15.64:0/1208569112 wait complete. 2023-09-13T03:17:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.975+0000 7fc5ccbc9b80 20 client.16431 trim_cache size 0 max 16384 2023-09-13T03:17:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:17:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:17:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:17:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:17:23.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.976+0000 7fc5b91ff700 1 -- 172.21.15.64:0/3317679065 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fc59c0a07b0 msgr2=0x7fc59c0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:17:23.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.976+0000 7fc5b91ff700 1 --2- 172.21.15.64:0/3317679065 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fc59c0a07b0 0x7fc59c0a2c70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac007970 tx=0x7fc5ac009920 comp rx=0 tx=0).stop 2023-09-13T03:17:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.976+0000 7fc5b91ff700 1 -- 172.21.15.64:0/3317679065 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5b4073350 msgr2=0x7fc5b419eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:17:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.976+0000 7fc5b91ff700 1 --2- 172.21.15.64:0/3317679065 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5b4073350 0x7fc5b419eed0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc5a800e9c0 tx=0x7fc5a800ecd0 comp rx=0 tx=0).stop 2023-09-13T03:17:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.976+0000 7fc5b91ff700 1 -- 172.21.15.64:0/3317679065 shutdown_connections 2023-09-13T03:17:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.976+0000 7fc5b91ff700 1 --2- 172.21.15.64:0/3317679065 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fc59c0a07b0 0x7fc59c0a2c70 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.976+0000 7fc5b91ff700 1 --2- 172.21.15.64:0/3317679065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5b410cdd0 0x7fc5b41987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.976+0000 7fc5b91ff700 1 --2- 172.21.15.64:0/3317679065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5b4072a30 0x7fc5b419e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.976+0000 7fc5b91ff700 1 --2- 172.21.15.64:0/3317679065 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5b4073350 0x7fc5b419eed0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.976+0000 7fc5b91ff700 1 -- 172.21.15.64:0/3317679065 >> 172.21.15.64:0/3317679065 conn(0x7fc5b40fc060 msgr2=0x7fc5b410ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:17:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.976+0000 7fc5b91ff700 1 -- 172.21.15.64:0/3317679065 shutdown_connections 2023-09-13T03:17:23.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:23.976+0000 7fc5b91ff700 1 -- 172.21.15.64:0/3317679065 wait complete. 2023-09-13T03:17:23.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:17:23 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:17:23.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:17:23 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:17:23.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:17:23 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:17:24.906 INFO:tasks.workunit.client.0.smithi064.stdout: 1 89516 2.15 MB/sec execute 89 sec latency 1327.260 ms 2023-09-13T03:17:24.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:17:24 smithi064 ceph-mon[97675]: pgmap v130: 97 pgs: 97 active+clean; 54 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 429 KiB/s wr, 82 op/s 2023-09-13T03:17:24.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:17:24 smithi142 ceph-mon[134082]: pgmap v130: 97 pgs: 97 active+clean; 54 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 429 KiB/s wr, 82 op/s 2023-09-13T03:17:25.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:17:24 smithi170 ceph-mon[134328]: pgmap v130: 97 pgs: 97 active+clean; 54 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 429 KiB/s wr, 82 op/s 2023-09-13T03:17:25.906 INFO:tasks.workunit.client.0.smithi064.stdout: 1 89516 2.13 MB/sec execute 90 sec latency 2327.404 ms 2023-09-13T03:17:25.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:17:25 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:17:25.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:17:25 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:17:26.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:17:25 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:17:26.906 INFO:tasks.workunit.client.0.smithi064.stdout: 1 89516 2.10 MB/sec execute 91 sec latency 3327.489 ms 2023-09-13T03:17:26.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:17:26 smithi142 ceph-mon[134082]: pgmap v131: 97 pgs: 97 active+clean; 50 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 506 KiB/s wr, 104 op/s 2023-09-13T03:17:26.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:17:26 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:17:27.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:17:26 smithi170 ceph-mon[134328]: pgmap v131: 97 pgs: 97 active+clean; 50 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 506 KiB/s wr, 104 op/s 2023-09-13T03:17:27.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:17:26 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:17:27.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:17:26 smithi064 ceph-mon[97675]: pgmap v131: 97 pgs: 97 active+clean; 50 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 506 KiB/s wr, 104 op/s 2023-09-13T03:17:27.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:17:26 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:17:27.906 INFO:tasks.workunit.client.0.smithi064.stdout: 1 89516 2.08 MB/sec execute 92 sec latency 4327.585 ms 2023-09-13T03:17:28.906 INFO:tasks.workunit.client.0.smithi064.stdout: 1 90179 2.11 MB/sec execute 93 sec latency 4944.464 ms 2023-09-13T03:17:28.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:17:28 smithi142 ceph-mon[134082]: pgmap v132: 97 pgs: 97 active+clean; 50 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 143 KiB/s wr, 37 op/s 2023-09-13T03:17:29.159 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:17:28 smithi170 ceph-mon[134328]: pgmap v132: 97 pgs: 97 active+clean; 50 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 143 KiB/s wr, 37 op/s 2023-09-13T03:17:29.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:17:28 smithi064 ceph-mon[97675]: pgmap v132: 97 pgs: 97 active+clean; 50 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 143 KiB/s wr, 37 op/s 2023-09-13T03:17:29.907 INFO:tasks.workunit.client.0.smithi064.stdout: 1 90986 2.09 MB/sec execute 94 sec latency 620.798 ms 2023-09-13T03:17:30.907 INFO:tasks.workunit.client.0.smithi064.stdout: 1 92634 2.21 MB/sec execute 95 sec latency 11.421 ms 2023-09-13T03:17:30.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:17:30 smithi142 ceph-mon[134082]: pgmap v133: 97 pgs: 97 active+clean; 50 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 143 KiB/s wr, 37 op/s 2023-09-13T03:17:30.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:17:30 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:17:31.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:17:30 smithi170 ceph-mon[134328]: pgmap v133: 97 pgs: 97 active+clean; 50 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 143 KiB/s wr, 37 op/s 2023-09-13T03:17:31.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:17:30 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:17:31.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:17:30 smithi064 ceph-mon[97675]: pgmap v133: 97 pgs: 97 active+clean; 50 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 143 KiB/s wr, 37 op/s 2023-09-13T03:17:31.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:17:30 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:17:31.907 INFO:tasks.workunit.client.0.smithi064.stdout: 1 92936 2.19 MB/sec execute 96 sec latency 634.519 ms 2023-09-13T03:17:32.907 INFO:tasks.workunit.client.0.smithi064.stdout: 1 92936 2.17 MB/sec execute 97 sec latency 1634.652 ms 2023-09-13T03:17:32.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:17:32 smithi142 ceph-mon[134082]: pgmap v134: 97 pgs: 97 active+clean; 52 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 320 KiB/s wr, 68 op/s 2023-09-13T03:17:33.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:17:32 smithi170 ceph-mon[134328]: pgmap v134: 97 pgs: 97 active+clean; 52 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 320 KiB/s wr, 68 op/s 2023-09-13T03:17:33.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:17:32 smithi064 ceph-mon[97675]: pgmap v134: 97 pgs: 97 active+clean; 52 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 320 KiB/s wr, 68 op/s 2023-09-13T03:17:33.907 INFO:tasks.workunit.client.0.smithi064.stdout: 1 93491 2.16 MB/sec execute 98 sec latency 2254.303 ms 2023-09-13T03:17:34.907 INFO:tasks.workunit.client.0.smithi064.stdout: 1 93961 2.17 MB/sec execute 99 sec latency 726.121 ms 2023-09-13T03:17:34.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:17:34 smithi142 ceph-mon[134082]: pgmap v135: 97 pgs: 97 active+clean; 52 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 254 KiB/s wr, 53 op/s 2023-09-13T03:17:35.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:17:34 smithi170 ceph-mon[134328]: pgmap v135: 97 pgs: 97 active+clean; 52 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 254 KiB/s wr, 53 op/s 2023-09-13T03:17:35.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:17:34 smithi064 ceph-mon[97675]: pgmap v135: 97 pgs: 97 active+clean; 52 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 254 KiB/s wr, 53 op/s 2023-09-13T03:17:35.907 INFO:tasks.workunit.client.0.smithi064.stdout: 1 95163 2.22 MB/sec execute 100 sec latency 1165.654 ms 2023-09-13T03:17:36.907 INFO:tasks.workunit.client.0.smithi064.stdout: 1 96410 2.27 MB/sec execute 101 sec latency 10.112 ms 2023-09-13T03:17:36.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:17:36 smithi142 ceph-mon[134082]: pgmap v136: 97 pgs: 97 active+clean; 54 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 444 KiB/s wr, 87 op/s 2023-09-13T03:17:37.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.125+0000 7f93e19ff700 1 -- 172.21.15.64:0/964202853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93dc10cd60 msgr2=0x7f93dc10f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:17:37.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.125+0000 7f93e19ff700 1 --2- 172.21.15.64:0/964202853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93dc10cd60 0x7f93dc10f1d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f93d4009f90 tx=0x7f93d400b330 comp rx=0 tx=0).stop 2023-09-13T03:17:37.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.125+0000 7f93e19ff700 1 -- 172.21.15.64:0/964202853 shutdown_connections 2023-09-13T03:17:37.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.125+0000 7f93e19ff700 1 --2- 172.21.15.64:0/964202853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93dc072a30 0x7f93dc072e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:37.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.125+0000 7f93e19ff700 1 --2- 172.21.15.64:0/964202853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93dc10cd60 0x7f93dc10f1d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:37.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.125+0000 7f93e19ff700 1 --2- 172.21.15.64:0/964202853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93dc073350 0x7f93dc10c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:37.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.125+0000 7f93e19ff700 1 -- 172.21.15.64:0/964202853 >> 172.21.15.64:0/964202853 conn(0x7f93dc0fbff0 msgr2=0x7f93dc0fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:17:37.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.126+0000 7f93e19ff700 1 -- 172.21.15.64:0/964202853 shutdown_connections 2023-09-13T03:17:37.128 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.126+0000 7f93e19ff700 1 -- 172.21.15.64:0/964202853 wait complete. 2023-09-13T03:17:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.126+0000 7f93e19ff700 1 Processor -- start 2023-09-13T03:17:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.127+0000 7f93e19ff700 1 -- start start 2023-09-13T03:17:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.127+0000 7f93e19ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93dc072a30 0x7f93dc19e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.127+0000 7f93e19ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93dc073350 0x7f93dc19ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.127+0000 7f93e19ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93dc10cd60 0x7f93dc198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.127+0000 7f93e19ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93dc198e00 con 0x7f93dc10cd60 2023-09-13T03:17:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.127+0000 7f93e19ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93dc198f70 con 0x7f93dc073350 2023-09-13T03:17:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.127+0000 7f93e19ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93dc199130 con 0x7f93dc072a30 2023-09-13T03:17:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.127+0000 7f93db7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93dc10cd60 0x7f93dc198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.127+0000 7f93db7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93dc10cd60 0x7f93dc198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34684/0 (socket says 172.21.15.64:34684) 2023-09-13T03:17:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.127+0000 7f93db7fe700 1 -- 172.21.15.64:0/1292758019 learned_addr learned my addr 172.21.15.64:0/1292758019 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:17:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.127+0000 7f93daffd700 1 --2- 172.21.15.64:0/1292758019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93dc072a30 0x7f93dc19e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.127+0000 7f93da7fc700 1 --2- 172.21.15.64:0/1292758019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93dc073350 0x7f93dc19ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:37.133 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.128+0000 7f93db7fe700 1 -- 172.21.15.64:0/1292758019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93dc072a30 msgr2=0x7f93dc19e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:17:37.133 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.128+0000 7f93db7fe700 1 --2- 172.21.15.64:0/1292758019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93dc072a30 0x7f93dc19e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:37.133 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.128+0000 7f93db7fe700 1 -- 172.21.15.64:0/1292758019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93dc073350 msgr2=0x7f93dc19ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:17:37.133 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.128+0000 7f93db7fe700 1 --2- 172.21.15.64:0/1292758019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93dc073350 0x7f93dc19ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:37.133 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.128+0000 7f93db7fe700 1 -- 172.21.15.64:0/1292758019 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d4009c40 con 0x7f93dc10cd60 2023-09-13T03:17:37.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.128+0000 7f93db7fe700 1 --2- 172.21.15.64:0/1292758019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93dc10cd60 0x7f93dc198790 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f93d4009010 tx=0x7f93d4004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:17:37.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.128+0000 7f93e09fd700 1 -- 172.21.15.64:0/1292758019 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d400e040 con 0x7f93dc10cd60 2023-09-13T03:17:37.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.128+0000 7f93e19ff700 1 -- 172.21.15.64:0/1292758019 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93dc1993b0 con 0x7f93dc10cd60 2023-09-13T03:17:37.134 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.128+0000 7f93e09fd700 1 -- 172.21.15.64:0/1292758019 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93d4004e80 con 0x7f93dc10cd60 2023-09-13T03:17:37.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.128+0000 7f93e09fd700 1 -- 172.21.15.64:0/1292758019 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d4005040 con 0x7f93dc10cd60 2023-09-13T03:17:37.135 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.128+0000 7f93e19ff700 1 -- 172.21.15.64:0/1292758019 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93dc1a5710 con 0x7f93dc10cd60 2023-09-13T03:17:37.140 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.132+0000 7f93e09fd700 1 -- 172.21.15.64:0/1292758019 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f93d400bbf0 con 0x7f93dc10cd60 2023-09-13T03:17:37.140 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.133+0000 7f93f5314b80 1 Processor -- start 2023-09-13T03:17:37.140 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.133+0000 7f93f5314b80 1 -- start start 2023-09-13T03:17:37.141 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.133+0000 7f93e09fd700 1 --2- 172.21.15.64:0/1292758019 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f93c40a0830 0x7f93c40a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:37.141 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.133+0000 7f93e09fd700 1 -- 172.21.15.64:0/1292758019 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f93d40cd4c0 con 0x7f93dc10cd60 2023-09-13T03:17:37.141 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.133+0000 7f93f5314b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b342b11a0 0x558b342b1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:37.141 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.133+0000 7f93f5314b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b342ab300 0x558b342ab780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:37.141 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.133+0000 7f93f5314b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b342abcc0 0x558b342b7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:37.142 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.133+0000 7f93f5314b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b342046d0 con 0x558b342b11a0 2023-09-13T03:17:37.142 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.133+0000 7f93f5314b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b34201ff0 con 0x558b342ab300 2023-09-13T03:17:37.142 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.133+0000 7f93f5314b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b341fd8e0 con 0x558b342abcc0 2023-09-13T03:17:37.142 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.133+0000 7f93da7fc700 1 --2- 172.21.15.64:0/1292758019 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f93c40a0830 0x7f93c40a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:37.142 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.134+0000 7f93daffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b342b11a0 0x558b342b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:37.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.134+0000 7f93db7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b342abcc0 0x558b342b7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:37.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.134+0000 7f93db7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b342abcc0 0x558b342b7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:42148/0 (socket says 172.21.15.64:42148) 2023-09-13T03:17:37.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.134+0000 7f93db7fe700 1 -- 172.21.15.64:0/354494025 learned_addr learned my addr 172.21.15.64:0/354494025 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:17:37.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.134+0000 7f93da7fc700 1 --2- 172.21.15.64:0/354494025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b342ab300 0x558b342ab780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:37.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.134+0000 7f93db7fe700 1 -- 172.21.15.64:0/354494025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b342ab300 msgr2=0x558b342ab780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:17:37.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.134+0000 7f93db7fe700 1 --2- 172.21.15.64:0/354494025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b342ab300 0x558b342ab780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:37.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.134+0000 7f93db7fe700 1 -- 172.21.15.64:0/354494025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b342b11a0 msgr2=0x558b342b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:17:37.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.134+0000 7f93db7fe700 1 --2- 172.21.15.64:0/354494025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b342b11a0 0x558b342b1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:37.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.134+0000 7f93db7fe700 1 -- 172.21.15.64:0/354494025 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b34029d80 con 0x558b342abcc0 2023-09-13T03:17:37.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.134+0000 7f93db7fe700 1 --2- 172.21.15.64:0/354494025 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b342abcc0 0x558b342b7f30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f93d4009f60 tx=0x7f93d409a8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13e07c369e74214b server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:17:37.145 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.134+0000 7f93da7fc700 1 --2- 172.21.15.64:0/1292758019 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f93c40a0830 0x7f93c40a2cf0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x558b342a6060 tx=0x7f93d00097b0 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:17:37.145 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.135+0000 7f93c1ffb700 1 -- 172.21.15.64:0/354494025 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d409a9f0 con 0x558b342abcc0 2023-09-13T03:17:37.145 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.135+0000 7f93c1ffb700 1 -- 172.21.15.64:0/354494025 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93d409e820 con 0x558b342abcc0 2023-09-13T03:17:37.145 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.135+0000 7f93c1ffb700 1 -- 172.21.15.64:0/354494025 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d409ae50 con 0x558b342abcc0 2023-09-13T03:17:37.145 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.135+0000 7f93f5314b80 1 -- 172.21.15.64:0/354494025 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b342abcc0 msgr2=0x558b342b7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:17:37.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.135+0000 7f93f5314b80 1 --2- 172.21.15.64:0/354494025 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b342abcc0 0x558b342b7f30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f93d4009f60 tx=0x7f93d409a8f0 comp rx=0 tx=0).stop 2023-09-13T03:17:37.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.135+0000 7f93f5314b80 1 -- 172.21.15.64:0/354494025 shutdown_connections 2023-09-13T03:17:37.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.135+0000 7f93f5314b80 1 --2- 172.21.15.64:0/354494025 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b342abcc0 0x558b342b7f30 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:37.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.135+0000 7f93f5314b80 1 --2- 172.21.15.64:0/354494025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b342b11a0 0x558b342b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:37.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.135+0000 7f93f5314b80 1 --2- 172.21.15.64:0/354494025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b342ab300 0x558b342ab780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:37.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.135+0000 7f93f5314b80 1 -- 172.21.15.64:0/354494025 >> 172.21.15.64:0/354494025 conn(0x558b34220030 msgr2=0x558b342a6560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:17:37.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.135+0000 7f93f5314b80 1 -- 172.21.15.64:0/354494025 shutdown_connections 2023-09-13T03:17:37.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.135+0000 7f93f5314b80 1 -- 172.21.15.64:0/354494025 wait complete. 2023-09-13T03:17:37.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.136+0000 7f93f5314b80 1 -- start start 2023-09-13T03:17:37.147 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.136+0000 7f93f5314b80 1 Processor -- start 2023-09-13T03:17:37.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.137+0000 7f93f5314b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b342ab300 0x558b342bac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:37.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.137+0000 7f93f5314b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b342abcc0 0x558b342bb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:37.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.137+0000 7f93f5314b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b342b11a0 0x558b342b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:37.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.137+0000 7f93f5314b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b3419bb10 con 0x558b342ab300 2023-09-13T03:17:37.148 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.137+0000 7f93f5314b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b3419b810 con 0x558b342abcc0 2023-09-13T03:17:37.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.137+0000 7f93f5314b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b3407f8c0 con 0x558b342b11a0 2023-09-13T03:17:37.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.137+0000 7f93f5314b80 10 client.0 _ll_register_callbacks cb 0x558b340e5a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:17:37.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.137+0000 7f93daffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b342ab300 0x558b342bac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:37.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.137+0000 7f93daffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b342ab300 0x558b342bac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34706/0 (socket says 172.21.15.64:34706) 2023-09-13T03:17:37.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.137+0000 7f93daffd700 1 -- 172.21.15.64:0/3146339547 learned_addr learned my addr 172.21.15.64:0/3146339547 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:17:37.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.137+0000 7f93db7fe700 1 --2- 172.21.15.64:0/3146339547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b342b11a0 0x558b342b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:37.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.137+0000 7f93da7fc700 1 --2- 172.21.15.64:0/3146339547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b342abcc0 0x558b342bb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:37.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.138+0000 7f93daffd700 1 -- 172.21.15.64:0/3146339547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b342b11a0 msgr2=0x558b342b4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:17:37.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.138+0000 7f93daffd700 1 --2- 172.21.15.64:0/3146339547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b342b11a0 0x558b342b4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:37.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.138+0000 7f93daffd700 1 -- 172.21.15.64:0/3146339547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b342abcc0 msgr2=0x558b342bb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:17:37.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.138+0000 7f93daffd700 1 --2- 172.21.15.64:0/3146339547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b342abcc0 0x558b342bb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:37.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.138+0000 7f93daffd700 1 -- 172.21.15.64:0/3146339547 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93dc1a5710 con 0x558b342ab300 2023-09-13T03:17:37.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.138+0000 7f93db7fe700 1 --2- 172.21.15.64:0/3146339547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b342b11a0 0x558b342b4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:17:37.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.138+0000 7f93daffd700 1 --2- 172.21.15.64:0/3146339547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b342ab300 0x558b342bac70 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f93cc012860 tx=0x7f93cc012b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fee35fb307b65bfa server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:17:37.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.138+0000 7f93baffd700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:17:37.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.138+0000 7f93baffd700 1 -- 172.21.15.64:0/3146339547 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93cc017070 con 0x558b342ab300 2023-09-13T03:17:37.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.138+0000 7f93f5314b80 1 -- 172.21.15.64:0/3146339547 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558b34155490 con 0x558b342ab300 2023-09-13T03:17:37.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.138+0000 7f93baffd700 1 -- 172.21.15.64:0/3146339547 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f93cc00f540 con 0x558b342ab300 2023-09-13T03:17:37.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.139+0000 7f93baffd700 1 -- 172.21.15.64:0/3146339547 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93cc0122e0 con 0x558b342ab300 2023-09-13T03:17:37.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.139+0000 7f93f5314b80 10 client.16443 fetch_fsmap learned FSMap version 44 2023-09-13T03:17:37.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.139+0000 7f93f5314b80 1 -- 172.21.15.64:0/3146339547 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x558b34012c00 con 0x558b342ab300 2023-09-13T03:17:37.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.139+0000 7f93baffd700 1 -- 172.21.15.64:0/3146339547 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558b34155490 con 0x558b342ab300 2023-09-13T03:17:37.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.139+0000 7f93baffd700 1 -- 172.21.15.64:0/3146339547 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 44) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f93cc0108a0 con 0x558b342ab300 2023-09-13T03:17:37.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.139+0000 7f93f5314b80 10 client.16443 fetch_fsmap finished waiting for FSMap version 44 2023-09-13T03:17:37.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.139+0000 7f93f5314b80 10 client.16443 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:17:37.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.139+0000 7f93f5314b80 20 client.16443 populate_metadata read hostname 'smithi064' 2023-09-13T03:17:37.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.139+0000 7f93f5314b80 1 --2- 172.21.15.64:0/3146339547 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x558b342c5300 0x558b342c7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:37.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.139+0000 7f93f5314b80 4 client.16443 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:17:37.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.139+0000 7f93f5314b80 1 -- 172.21.15.64:0/3146339547 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558b34212840 con 0x558b342c5300 2023-09-13T03:17:37.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.139+0000 7f93da7fc700 1 --2- 172.21.15.64:0/3146339547 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x558b342c5300 0x558b342c7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:37.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.140+0000 7f93da7fc700 1 --2- 172.21.15.64:0/3146339547 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x558b342c5300 0x558b342c7720 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72899e0507a5fa2d server_cookie=15ac83760b23b9fe in_seq=0 out_seq=0 2023-09-13T03:17:37.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.140+0000 7f93baffd700 10 client.16443 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:17:37.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.141+0000 7f93baffd700 1 -- 172.21.15.64:0/3146339547 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558b34212840 con 0x558b342c5300 2023-09-13T03:17:37.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.141+0000 7f93baffd700 10 client.16443 handle_command_reply: tid=0 2023-09-13T03:17:37.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.141+0000 7f93f5314b80 1 client.16443 shutdown 2023-09-13T03:17:37.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.141+0000 7f93f5314b80 1 -- 172.21.15.64:0/3146339547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b342ab300 msgr2=0x558b342bac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:17:37.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.141+0000 7f93f5314b80 1 --2- 172.21.15.64:0/3146339547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b342ab300 0x558b342bac70 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f93cc012860 tx=0x7f93cc012b70 comp rx=0 tx=0).stop 2023-09-13T03:17:37.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.141+0000 7f93f5314b80 1 -- 172.21.15.64:0/3146339547 shutdown_connections 2023-09-13T03:17:37.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.141+0000 7f93f5314b80 1 --2- 172.21.15.64:0/3146339547 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x558b342c5300 0x558b342c7720 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:37.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.141+0000 7f93f5314b80 1 --2- 172.21.15.64:0/3146339547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b342b11a0 0x558b342b4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:37.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.141+0000 7f93f5314b80 1 --2- 172.21.15.64:0/3146339547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b342ab300 0x558b342bac70 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:37.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.141+0000 7f93baffd700 0 client.16443 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:17:37.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.141+0000 7f93f5314b80 1 --2- 172.21.15.64:0/3146339547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b342abcc0 0x558b342bb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:37.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.141+0000 7f93f5314b80 1 -- 172.21.15.64:0/3146339547 >> 172.21.15.64:0/3146339547 conn(0x558b34220030 msgr2=0x558b342a57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:17:37.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.141+0000 7f93f5314b80 1 -- 172.21.15.64:0/3146339547 shutdown_connections 2023-09-13T03:17:37.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.141+0000 7f93f5314b80 1 -- 172.21.15.64:0/3146339547 wait complete. 2023-09-13T03:17:37.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.141+0000 7f93f5314b80 20 client.16443 trim_cache size 0 max 16384 2023-09-13T03:17:37.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:17:36 smithi170 ceph-mon[134328]: pgmap v136: 97 pgs: 97 active+clean; 54 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 444 KiB/s wr, 87 op/s 2023-09-13T03:17:37.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.156+0000 7f93f5314b80 1 Processor -- start 2023-09-13T03:17:37.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.156+0000 7f93f5314b80 1 -- start start 2023-09-13T03:17:37.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.157+0000 7f93f5314b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b342ab300 0x558b342b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:37.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.157+0000 7f93f5314b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b342abcc0 0x558b342ba800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:37.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.157+0000 7f93f5314b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b342b11a0 0x558b342bad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:37.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.157+0000 7f93f5314b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b342b3490 con 0x558b342abcc0 2023-09-13T03:17:37.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.157+0000 7f93f5314b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b342aeb60 con 0x558b342ab300 2023-09-13T03:17:37.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.157+0000 7f93f5314b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b342afbe0 con 0x558b342b11a0 2023-09-13T03:17:37.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.157+0000 7f93daffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b342ab300 0x558b342b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:37.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.157+0000 7f93daffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b342ab300 0x558b342b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:45772/0 (socket says 172.21.15.64:45772) 2023-09-13T03:17:37.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.157+0000 7f93daffd700 1 -- 172.21.15.64:0/1791541129 learned_addr learned my addr 172.21.15.64:0/1791541129 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:17:37.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.157+0000 7f93da7fc700 1 --2- 172.21.15.64:0/1791541129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b342abcc0 0x558b342ba800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:37.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.157+0000 7f93db7fe700 1 --2- 172.21.15.64:0/1791541129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b342b11a0 0x558b342bad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:37.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.157+0000 7f93daffd700 1 -- 172.21.15.64:0/1791541129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b342b11a0 msgr2=0x558b342bad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:17:37.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.157+0000 7f93daffd700 1 --2- 172.21.15.64:0/1791541129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b342b11a0 0x558b342bad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.157+0000 7f93daffd700 1 -- 172.21.15.64:0/1791541129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b342abcc0 msgr2=0x558b342ba800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:17:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.157+0000 7f93daffd700 1 --2- 172.21.15.64:0/1791541129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b342abcc0 0x558b342ba800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.157+0000 7f93daffd700 1 -- 172.21.15.64:0/1791541129 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b340501f0 con 0x558b342ab300 2023-09-13T03:17:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.157+0000 7f93db7fe700 1 --2- 172.21.15.64:0/1791541129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b342b11a0 0x558b342bad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T03:17:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.158+0000 7f93daffd700 1 --2- 172.21.15.64:0/1791541129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b342ab300 0x558b342b0700 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f93cc009340 tx=0x7f93cc0123e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d77c5587e9b8bed6 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:17:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.158+0000 7f93c27fc700 1 -- 172.21.15.64:0/1791541129 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93cc017070 con 0x558b342ab300 2023-09-13T03:17:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.158+0000 7f93c27fc700 1 -- 172.21.15.64:0/1791541129 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93cc018040 con 0x558b342ab300 2023-09-13T03:17:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.158+0000 7f93c27fc700 1 -- 172.21.15.64:0/1791541129 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93cc003680 con 0x558b342ab300 2023-09-13T03:17:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.158+0000 7f93f5314b80 1 -- 172.21.15.64:0/1791541129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b342ab300 msgr2=0x558b342b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:17:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.158+0000 7f93f5314b80 1 --2- 172.21.15.64:0/1791541129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b342ab300 0x558b342b0700 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f93cc009340 tx=0x7f93cc0123e0 comp rx=0 tx=0).stop 2023-09-13T03:17:37.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.158+0000 7f93f5314b80 1 -- 172.21.15.64:0/1791541129 shutdown_connections 2023-09-13T03:17:37.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.158+0000 7f93f5314b80 1 --2- 172.21.15.64:0/1791541129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b342ab300 0x558b342b0700 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:37.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.158+0000 7f93f5314b80 1 --2- 172.21.15.64:0/1791541129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b342b11a0 0x558b342bad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:37.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.158+0000 7f93f5314b80 1 --2- 172.21.15.64:0/1791541129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b342abcc0 0x558b342ba800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:37.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.158+0000 7f93f5314b80 1 -- 172.21.15.64:0/1791541129 >> 172.21.15.64:0/1791541129 conn(0x558b34220030 msgr2=0x558b342afff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:17:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.158+0000 7f93f5314b80 1 -- 172.21.15.64:0/1791541129 shutdown_connections 2023-09-13T03:17:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.159+0000 7f93f5314b80 1 -- 172.21.15.64:0/1791541129 wait complete. 2023-09-13T03:17:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.159+0000 7f93f5314b80 1 -- start start 2023-09-13T03:17:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.159+0000 7f93f5314b80 1 Processor -- start 2023-09-13T03:17:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.160+0000 7f93f5314b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b342ab300 0x558b3430d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.160+0000 7f93f5314b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b342abcc0 0x558b3430d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.160+0000 7f93f5314b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b342b11a0 0x558b343121a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.160+0000 7f93f5314b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b343126e0 con 0x558b342ab300 2023-09-13T03:17:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.160+0000 7f93f5314b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b34312850 con 0x558b342abcc0 2023-09-13T03:17:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.160+0000 7f93f5314b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b34312a10 con 0x558b342b11a0 2023-09-13T03:17:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.160+0000 7f93f5314b80 10 client.0 _ll_register_callbacks cb 0x7f93cc009ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:17:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.160+0000 7f93daffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b342ab300 0x558b3430d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.160+0000 7f93da7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b342abcc0 0x558b3430d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.160+0000 7f93da7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b342abcc0 0x558b3430d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:45786/0 (socket says 172.21.15.64:45786) 2023-09-13T03:17:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.160+0000 7f93daffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b342ab300 0x558b3430d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34732/0 (socket says 172.21.15.64:34732) 2023-09-13T03:17:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.160+0000 7f93da7fc700 1 -- 172.21.15.64:0/1534294029 learned_addr learned my addr 172.21.15.64:0/1534294029 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:17:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.160+0000 7f93db7fe700 1 --2- 172.21.15.64:0/1534294029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b342b11a0 0x558b343121a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.160+0000 7f93da7fc700 1 -- 172.21.15.64:0/1534294029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b342b11a0 msgr2=0x558b343121a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:17:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.160+0000 7f93da7fc700 1 --2- 172.21.15.64:0/1534294029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b342b11a0 0x558b343121a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.160+0000 7f93da7fc700 1 -- 172.21.15.64:0/1534294029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b342ab300 msgr2=0x558b3430d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:17:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.160+0000 7f93da7fc700 1 --2- 172.21.15.64:0/1534294029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b342ab300 0x558b3430d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.160+0000 7f93da7fc700 1 -- 172.21.15.64:0/1534294029 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b34012c00 con 0x558b342abcc0 2023-09-13T03:17:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.161+0000 7f93da7fc700 1 --2- 172.21.15.64:0/1534294029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b342abcc0 0x558b3430d810 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f93d0005ed0 tx=0x7f93d0009b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5715a5d7408145 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:17:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.161+0000 7f93baffd700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:17:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.161+0000 7f93baffd700 1 -- 172.21.15.64:0/1534294029 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d0027040 con 0x558b342abcc0 2023-09-13T03:17:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.161+0000 7f93f5314b80 1 -- 172.21.15.64:0/1534294029 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f93d409e820 con 0x558b342abcc0 2023-09-13T03:17:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.161+0000 7f93baffd700 1 -- 172.21.15.64:0/1534294029 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f93d002a4a0 con 0x558b342abcc0 2023-09-13T03:17:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.161+0000 7f93baffd700 1 -- 172.21.15.64:0/1534294029 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93d0013360 con 0x558b342abcc0 2023-09-13T03:17:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.161+0000 7f93f5314b80 10 client.26489 fetch_fsmap learned FSMap version 44 2023-09-13T03:17:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.161+0000 7f93f5314b80 1 -- 172.21.15.64:0/1534294029 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x558b34028250 con 0x558b342abcc0 2023-09-13T03:17:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.162+0000 7f93baffd700 1 -- 172.21.15.64:0/1534294029 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93d409e820 con 0x558b342abcc0 2023-09-13T03:17:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.162+0000 7f93baffd700 1 -- 172.21.15.64:0/1534294029 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 44) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f93d0030020 con 0x558b342abcc0 2023-09-13T03:17:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.162+0000 7f93f5314b80 10 client.26489 fetch_fsmap finished waiting for FSMap version 44 2023-09-13T03:17:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.162+0000 7f93f5314b80 10 client.26489 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:17:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.162+0000 7f93f5314b80 20 client.26489 populate_metadata read hostname 'smithi064' 2023-09-13T03:17:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.162+0000 7f93f5314b80 1 --2- 172.21.15.64:0/1534294029 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x558b3431a2e0 0x558b3431c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.162+0000 7f93f5314b80 4 client.26489 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:17:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.162+0000 7f93f5314b80 1 -- 172.21.15.64:0/1534294029 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558b341f68c0 con 0x558b3431a2e0 2023-09-13T03:17:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.162+0000 7f93daffd700 1 --2- 172.21.15.64:0/1534294029 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x558b3431a2e0 0x558b3431c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.162+0000 7f93daffd700 1 --2- 172.21.15.64:0/1534294029 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x558b3431a2e0 0x558b3431c700 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a479a579cad9b785 server_cookie=41a509f1edccbe3d in_seq=0 out_seq=0 2023-09-13T03:17:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.162+0000 7f93baffd700 10 client.26489 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:17:37.178 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:17:36 smithi064 ceph-mon[97675]: pgmap v136: 97 pgs: 97 active+clean; 54 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 444 KiB/s wr, 87 op/s 2023-09-13T03:17:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.184+0000 7f93baffd700 1 -- 172.21.15.64:0/1534294029 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558b341f68c0 con 0x558b3431a2e0 2023-09-13T03:17:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.184+0000 7f93baffd700 10 client.26489 handle_command_reply: tid=0 2023-09-13T03:17:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.184+0000 7f93f5314b80 1 client.26489 shutdown 2023-09-13T03:17:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.184+0000 7f93f5314b80 1 -- 172.21.15.64:0/1534294029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b342abcc0 msgr2=0x558b3430d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:17:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.184+0000 7f93f5314b80 1 --2- 172.21.15.64:0/1534294029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b342abcc0 0x558b3430d810 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f93d0005ed0 tx=0x7f93d0009b20 comp rx=0 tx=0).stop 2023-09-13T03:17:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.184+0000 7f93f5314b80 1 -- 172.21.15.64:0/1534294029 shutdown_connections 2023-09-13T03:17:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.184+0000 7f93f5314b80 1 --2- 172.21.15.64:0/1534294029 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x558b3431a2e0 0x558b3431c700 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.184+0000 7f93f5314b80 1 --2- 172.21.15.64:0/1534294029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b342b11a0 0x558b343121a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.184+0000 7f93baffd700 0 client.26489 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:17:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.184+0000 7f93f5314b80 1 --2- 172.21.15.64:0/1534294029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b342ab300 0x558b3430d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.184+0000 7f93f5314b80 1 --2- 172.21.15.64:0/1534294029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b342abcc0 0x558b3430d810 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.184+0000 7f93f5314b80 1 -- 172.21.15.64:0/1534294029 >> 172.21.15.64:0/1534294029 conn(0x558b34220030 msgr2=0x558b342b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:17:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.184+0000 7f93f5314b80 1 -- 172.21.15.64:0/1534294029 shutdown_connections 2023-09-13T03:17:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.184+0000 7f93f5314b80 1 -- 172.21.15.64:0/1534294029 wait complete. 2023-09-13T03:17:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.185+0000 7f93f5314b80 20 client.26489 trim_cache size 0 max 16384 2023-09-13T03:17:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:17:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:17:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:17:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:17:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.185+0000 7f93e19ff700 1 -- 172.21.15.64:0/1292758019 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f93c40a0830 msgr2=0x7f93c40a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:17:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.185+0000 7f93e19ff700 1 --2- 172.21.15.64:0/1292758019 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f93c40a0830 0x7f93c40a2cf0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x558b342a6060 tx=0x7f93d00097b0 comp rx=0 tx=0).stop 2023-09-13T03:17:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.185+0000 7f93e19ff700 1 -- 172.21.15.64:0/1292758019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93dc10cd60 msgr2=0x7f93dc198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:17:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.185+0000 7f93e19ff700 1 --2- 172.21.15.64:0/1292758019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93dc10cd60 0x7f93dc198790 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f93d4009010 tx=0x7f93d4004a50 comp rx=0 tx=0).stop 2023-09-13T03:17:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.185+0000 7f93e19ff700 1 -- 172.21.15.64:0/1292758019 shutdown_connections 2023-09-13T03:17:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.185+0000 7f93e19ff700 1 --2- 172.21.15.64:0/1292758019 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f93c40a0830 0x7f93c40a2cf0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.185+0000 7f93e19ff700 1 --2- 172.21.15.64:0/1292758019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93dc10cd60 0x7f93dc198790 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.185+0000 7f93e19ff700 1 --2- 172.21.15.64:0/1292758019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93dc072a30 0x7f93dc19e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.185+0000 7f93e19ff700 1 --2- 172.21.15.64:0/1292758019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93dc073350 0x7f93dc19ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.185+0000 7f93e19ff700 1 -- 172.21.15.64:0/1292758019 >> 172.21.15.64:0/1292758019 conn(0x7f93dc0fbff0 msgr2=0x7f93dc1047c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:17:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.185+0000 7f93e19ff700 1 -- 172.21.15.64:0/1292758019 shutdown_connections 2023-09-13T03:17:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:37.185+0000 7f93e19ff700 1 -- 172.21.15.64:0/1292758019 wait complete. 2023-09-13T03:17:37.908 INFO:tasks.workunit.client.0.smithi064.stdout: 1 96539 2.25 MB/sec execute 102 sec latency 909.649 ms 2023-09-13T03:17:38.908 INFO:tasks.workunit.client.0.smithi064.stdout: 1 97226 2.27 MB/sec execute 103 sec latency 1526.221 ms 2023-09-13T03:17:38.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:17:38 smithi142 ceph-mon[134082]: pgmap v137: 97 pgs: 97 active+clean; 54 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 366 KiB/s wr, 65 op/s 2023-09-13T03:17:39.159 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:17:38 smithi170 ceph-mon[134328]: pgmap v137: 97 pgs: 97 active+clean; 54 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 366 KiB/s wr, 65 op/s 2023-09-13T03:17:39.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:17:38 smithi064 ceph-mon[97675]: pgmap v137: 97 pgs: 97 active+clean; 54 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 366 KiB/s wr, 65 op/s 2023-09-13T03:17:39.908 INFO:tasks.workunit.client.0.smithi064.stdout: 1 97508 2.25 MB/sec execute 104 sec latency 782.187 ms 2023-09-13T03:17:40.659 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:17:40 smithi170 ceph-mon[134328]: pgmap v138: 97 pgs: 97 active+clean; 54 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 366 KiB/s wr, 65 op/s 2023-09-13T03:17:40.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:17:40 smithi064 ceph-mon[97675]: pgmap v138: 97 pgs: 97 active+clean; 54 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 366 KiB/s wr, 65 op/s 2023-09-13T03:17:40.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:17:40 smithi142 ceph-mon[134082]: pgmap v138: 97 pgs: 97 active+clean; 54 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 366 KiB/s wr, 65 op/s 2023-09-13T03:17:40.908 INFO:tasks.workunit.client.0.smithi064.stdout: 1 98733 2.30 MB/sec execute 105 sec latency 1223.423 ms 2023-09-13T03:17:41.908 INFO:tasks.workunit.client.0.smithi064.stdout: 1 99981 2.34 MB/sec execute 106 sec latency 13.321 ms 2023-09-13T03:17:42.908 INFO:tasks.workunit.client.0.smithi064.stdout: 1 100028 2.32 MB/sec execute 107 sec latency 992.791 ms 2023-09-13T03:17:42.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:17:42 smithi170 ceph-mon[134328]: pgmap v139: 97 pgs: 97 active+clean; 54 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 764 KiB/s wr, 122 op/s 2023-09-13T03:17:42.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:17:42 smithi064 ceph-mon[97675]: pgmap v139: 97 pgs: 97 active+clean; 54 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 764 KiB/s wr, 122 op/s 2023-09-13T03:17:42.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:17:42 smithi142 ceph-mon[134082]: pgmap v139: 97 pgs: 97 active+clean; 54 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 764 KiB/s wr, 122 op/s 2023-09-13T03:17:43.908 INFO:tasks.workunit.client.0.smithi064.stdout: 1 100084 2.30 MB/sec execute 108 sec latency 1609.957 ms 2023-09-13T03:17:44.908 INFO:tasks.workunit.client.0.smithi064.stdout: 1 100084 2.28 MB/sec execute 109 sec latency 1340.261 ms 2023-09-13T03:17:44.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:17:44 smithi170 ceph-mon[134328]: pgmap v140: 97 pgs: 97 active+clean; 54 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 588 KiB/s wr, 91 op/s 2023-09-13T03:17:44.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:17:44 smithi064 ceph-mon[97675]: pgmap v140: 97 pgs: 97 active+clean; 54 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 588 KiB/s wr, 91 op/s 2023-09-13T03:17:44.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:17:44 smithi142 ceph-mon[134082]: pgmap v140: 97 pgs: 97 active+clean; 54 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 588 KiB/s wr, 91 op/s 2023-09-13T03:17:45.908 INFO:tasks.workunit.client.0.smithi064.stdout: 1 100084 2.26 MB/sec execute 110 sec latency 2340.409 ms 2023-09-13T03:17:46.909 INFO:tasks.workunit.client.0.smithi064.stdout: 1 100084 2.24 MB/sec execute 111 sec latency 3340.547 ms 2023-09-13T03:17:46.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:17:46 smithi170 ceph-mon[134328]: pgmap v141: 97 pgs: 97 active+clean; 60 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 132 op/s 2023-09-13T03:17:46.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:17:46 smithi064 ceph-mon[97675]: pgmap v141: 97 pgs: 97 active+clean; 60 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 132 op/s 2023-09-13T03:17:46.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:17:46 smithi142 ceph-mon[134082]: pgmap v141: 97 pgs: 97 active+clean; 60 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 132 op/s 2023-09-13T03:17:47.908 INFO:tasks.workunit.client.0.smithi064.stdout: 1 100084 2.22 MB/sec execute 112 sec latency 4340.703 ms 2023-09-13T03:17:48.909 INFO:tasks.workunit.client.0.smithi064.stdout: 1 100152 2.20 MB/sec execute 113 sec latency 4956.576 ms 2023-09-13T03:17:48.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:17:48 smithi170 ceph-mon[134328]: pgmap v142: 97 pgs: 97 active+clean; 60 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 1013 KiB/s wr, 97 op/s 2023-09-13T03:17:48.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:17:48 smithi064 ceph-mon[97675]: pgmap v142: 97 pgs: 97 active+clean; 60 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 1013 KiB/s wr, 97 op/s 2023-09-13T03:17:48.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:17:48 smithi142 ceph-mon[134082]: pgmap v142: 97 pgs: 97 active+clean; 60 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 1013 KiB/s wr, 97 op/s 2023-09-13T03:17:49.909 INFO:tasks.workunit.client.0.smithi064.stdout: 1 100152 2.18 MB/sec execute 114 sec latency 1331.021 ms 2023-09-13T03:17:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.347+0000 7f9d799ff700 1 -- 172.21.15.64:0/2885363167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d74106660 msgr2=0x7f9d74106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:17:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.347+0000 7f9d799ff700 1 --2- 172.21.15.64:0/2885363167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d74106660 0x7f9d74106a40 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9d640099c0 tx=0x7f9d64009cd0 comp rx=0 tx=0).stop 2023-09-13T03:17:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.347+0000 7f9d799ff700 1 -- 172.21.15.64:0/2885363167 shutdown_connections 2023-09-13T03:17:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.347+0000 7f9d799ff700 1 --2- 172.21.15.64:0/2885363167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d74100f60 0x7f9d7410d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.347+0000 7f9d799ff700 1 --2- 172.21.15.64:0/2885363167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d74106660 0x7f9d74106a40 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:50.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.347+0000 7f9d799ff700 1 --2- 172.21.15.64:0/2885363167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d741005a0 0x7f9d74100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:50.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.347+0000 7f9d799ff700 1 -- 172.21.15.64:0/2885363167 >> 172.21.15.64:0/2885363167 conn(0x7f9d740fc050 msgr2=0x7f9d740fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:17:50.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.347+0000 7f9d799ff700 1 -- 172.21.15.64:0/2885363167 shutdown_connections 2023-09-13T03:17:50.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.348+0000 7f9d799ff700 1 -- 172.21.15.64:0/2885363167 wait complete. 2023-09-13T03:17:50.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.349+0000 7f9d799ff700 1 Processor -- start 2023-09-13T03:17:50.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.349+0000 7f9d799ff700 1 -- start start 2023-09-13T03:17:50.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.349+0000 7f9d799ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d741005a0 0x7f9d7419c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:50.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.349+0000 7f9d799ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d74100f60 0x7f9d7419caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:50.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.349+0000 7f9d799ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d74106660 0x7f9d741963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:50.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.349+0000 7f9d799ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d74196a30 con 0x7f9d74100f60 2023-09-13T03:17:50.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.349+0000 7f9d799ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d74196ba0 con 0x7f9d74106660 2023-09-13T03:17:50.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.349+0000 7f9d799ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d74196d60 con 0x7f9d741005a0 2023-09-13T03:17:50.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.350+0000 7f9d727fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d74100f60 0x7f9d7419caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:50.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.350+0000 7f9d72ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d741005a0 0x7f9d7419c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:50.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.350+0000 7f9d72ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d741005a0 0x7f9d7419c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:49350/0 (socket says 172.21.15.64:49350) 2023-09-13T03:17:50.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.350+0000 7f9d727fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d74100f60 0x7f9d7419caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44974/0 (socket says 172.21.15.64:44974) 2023-09-13T03:17:50.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.350+0000 7f9d72ffd700 1 -- 172.21.15.64:0/2050151419 learned_addr learned my addr 172.21.15.64:0/2050151419 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:17:50.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.350+0000 7f9d737fe700 1 --2- 172.21.15.64:0/2050151419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d74106660 0x7f9d741963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:50.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.350+0000 7f9d727fc700 1 -- 172.21.15.64:0/2050151419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d741005a0 msgr2=0x7f9d7419c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:17:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.350+0000 7f9d727fc700 1 --2- 172.21.15.64:0/2050151419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d741005a0 0x7f9d7419c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.350+0000 7f9d727fc700 1 -- 172.21.15.64:0/2050151419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d74106660 msgr2=0x7f9d741963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:17:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.350+0000 7f9d727fc700 1 --2- 172.21.15.64:0/2050151419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d74106660 0x7f9d741963c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.350+0000 7f9d727fc700 1 -- 172.21.15.64:0/2050151419 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d640095f0 con 0x7f9d74100f60 2023-09-13T03:17:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.350+0000 7f9d737fe700 1 --2- 172.21.15.64:0/2050151419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d74106660 0x7f9d741963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:17:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.350+0000 7f9d72ffd700 1 --2- 172.21.15.64:0/2050151419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d741005a0 0x7f9d7419c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:17:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.350+0000 7f9d727fc700 1 --2- 172.21.15.64:0/2050151419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d74100f60 0x7f9d7419caa0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9d6800e9f0 tx=0x7f9d6800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:17:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.350+0000 7f9d789fd700 1 -- 172.21.15.64:0/2050151419 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6800eee0 con 0x7f9d74100f60 2023-09-13T03:17:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.351+0000 7f9d789fd700 1 -- 172.21.15.64:0/2050151419 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d6800cbd0 con 0x7f9d74100f60 2023-09-13T03:17:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.351+0000 7f9d799ff700 1 -- 172.21.15.64:0/2050151419 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d74197040 con 0x7f9d74100f60 2023-09-13T03:17:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.351+0000 7f9d789fd700 1 -- 172.21.15.64:0/2050151419 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d68018680 con 0x7f9d74100f60 2023-09-13T03:17:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.351+0000 7f9d799ff700 1 -- 172.21.15.64:0/2050151419 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d741a3550 con 0x7f9d74100f60 2023-09-13T03:17:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.354+0000 7f9d789fd700 1 -- 172.21.15.64:0/2050151419 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f9d6800cd40 con 0x7f9d74100f60 2023-09-13T03:17:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.356+0000 7f9d8d326b80 1 Processor -- start 2023-09-13T03:17:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.356+0000 7f9d8d326b80 1 -- start start 2023-09-13T03:17:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.356+0000 7f9d8d326b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56115b7c95b0 0x56115b7c9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:50.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.357+0000 7f9d8d326b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56115b7c3710 0x56115b7c3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:50.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.357+0000 7f9d8d326b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56115b7c40d0 0x56115b7d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:50.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.357+0000 7f9d789fd700 1 --2- 172.21.15.64:0/2050151419 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f9d5c0a0830 0x7f9d5c0a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:50.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.357+0000 7f9d8d326b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56115b7186d0 con 0x56115b7c40d0 2023-09-13T03:17:50.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.357+0000 7f9d8d326b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56115b715ff0 con 0x56115b7c95b0 2023-09-13T03:17:50.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.357+0000 7f9d8d326b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56115b7118e0 con 0x56115b7c3710 2023-09-13T03:17:50.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.357+0000 7f9d789fd700 1 -- 172.21.15.64:0/2050151419 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f9d68014070 con 0x7f9d74100f60 2023-09-13T03:17:50.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.357+0000 7f9d737fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56115b7c95b0 0x56115b7c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:50.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.357+0000 7f9d727fc700 1 --2- 172.21.15.64:0/2050151419 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f9d5c0a0830 0x7f9d5c0a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:50.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.357+0000 7f9d72ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56115b7c3710 0x56115b7c3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:50.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.357+0000 7f9d737fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56115b7c95b0 0x56115b7c9990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:34538/0 (socket says 172.21.15.64:34538) 2023-09-13T03:17:50.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.357+0000 7f9d737fe700 1 -- 172.21.15.64:0/1767746344 learned_addr learned my addr 172.21.15.64:0/1767746344 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:17:50.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.357+0000 7f9d737fe700 1 --2- 172.21.15.64:0/1767746344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56115b7c40d0 0x56115b7d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:50.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.357+0000 7f9d72ffd700 1 -- 172.21.15.64:0/1767746344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56115b7c95b0 msgr2=0x56115b7c9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:17:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.357+0000 7f9d72ffd700 1 --2- 172.21.15.64:0/1767746344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56115b7c95b0 0x56115b7c9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.357+0000 7f9d72ffd700 1 -- 172.21.15.64:0/1767746344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56115b7c40d0 msgr2=0x56115b7d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:17:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.357+0000 7f9d72ffd700 1 --2- 172.21.15.64:0/1767746344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56115b7c40d0 0x56115b7d0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.357+0000 7f9d72ffd700 1 -- 172.21.15.64:0/1767746344 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56115b53dd80 con 0x56115b7c3710 2023-09-13T03:17:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.358+0000 7f9d727fc700 1 --2- 172.21.15.64:0/2050151419 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f9d5c0a0830 0x7f9d5c0a2cf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9d68096060 tx=0x7f9d68005760 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:17:50.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.358+0000 7f9d72ffd700 1 --2- 172.21.15.64:0/1767746344 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56115b7c3710 0x56115b7c3b90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9d64009fd0 tx=0x7f9d64005bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c5fb0dc6b2bbd0f server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:17:50.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.358+0000 7f9d59ffb700 1 -- 172.21.15.64:0/1767746344 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6400be40 con 0x56115b7c3710 2023-09-13T03:17:50.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.358+0000 7f9d59ffb700 1 -- 172.21.15.64:0/1767746344 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d6401e040 con 0x56115b7c3710 2023-09-13T03:17:50.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.358+0000 7f9d59ffb700 1 -- 172.21.15.64:0/1767746344 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d64003e10 con 0x56115b7c3710 2023-09-13T03:17:50.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.359+0000 7f9d8d326b80 1 -- 172.21.15.64:0/1767746344 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56115b7c3710 msgr2=0x56115b7c3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:17:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.359+0000 7f9d8d326b80 1 --2- 172.21.15.64:0/1767746344 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56115b7c3710 0x56115b7c3b90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9d64009fd0 tx=0x7f9d64005bc0 comp rx=0 tx=0).stop 2023-09-13T03:17:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.359+0000 7f9d8d326b80 1 -- 172.21.15.64:0/1767746344 shutdown_connections 2023-09-13T03:17:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.359+0000 7f9d8d326b80 1 --2- 172.21.15.64:0/1767746344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56115b7c95b0 0x56115b7c9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.359+0000 7f9d8d326b80 1 --2- 172.21.15.64:0/1767746344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56115b7c40d0 0x56115b7d0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:50.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.359+0000 7f9d8d326b80 1 --2- 172.21.15.64:0/1767746344 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56115b7c3710 0x56115b7c3b90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:50.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.359+0000 7f9d8d326b80 1 -- 172.21.15.64:0/1767746344 >> 172.21.15.64:0/1767746344 conn(0x56115b734030 msgr2=0x56115b7ba560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:17:50.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.359+0000 7f9d8d326b80 1 -- 172.21.15.64:0/1767746344 shutdown_connections 2023-09-13T03:17:50.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.359+0000 7f9d8d326b80 1 -- 172.21.15.64:0/1767746344 wait complete. 2023-09-13T03:17:50.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.359+0000 7f9d8d326b80 1 -- start start 2023-09-13T03:17:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.360+0000 7f9d8d326b80 1 Processor -- start 2023-09-13T03:17:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.360+0000 7f9d8d326b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56115b7c3710 0x56115b7d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.360+0000 7f9d8d326b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56115b7c40d0 0x56115b7cd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.360+0000 7f9d8d326b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56115b7c95b0 0x56115b7cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.360+0000 7f9d8d326b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56115b6afb10 con 0x56115b7c3710 2023-09-13T03:17:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.360+0000 7f9d8d326b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56115b6af810 con 0x56115b7c95b0 2023-09-13T03:17:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.360+0000 7f9d8d326b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56115b5938c0 con 0x56115b7c40d0 2023-09-13T03:17:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.360+0000 7f9d8d326b80 10 client.0 _ll_register_callbacks cb 0x56115b5f9a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:17:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.360+0000 7f9d737fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56115b7c3710 0x56115b7d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.360+0000 7f9d72ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56115b7c40d0 0x56115b7cd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.361+0000 7f9d737fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56115b7c3710 0x56115b7d01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44998/0 (socket says 172.21.15.64:44998) 2023-09-13T03:17:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.361+0000 7f9d737fe700 1 -- 172.21.15.64:0/1037518520 learned_addr learned my addr 172.21.15.64:0/1037518520 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:17:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.361+0000 7f9d737fe700 1 --2- 172.21.15.64:0/1037518520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56115b7c95b0 0x56115b7cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.361+0000 7f9d737fe700 1 -- 172.21.15.64:0/1037518520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56115b7c40d0 msgr2=0x56115b7cd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:17:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.361+0000 7f9d737fe700 1 --2- 172.21.15.64:0/1037518520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56115b7c40d0 0x56115b7cd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.361+0000 7f9d737fe700 1 -- 172.21.15.64:0/1037518520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56115b7c95b0 msgr2=0x56115b7cdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:17:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.361+0000 7f9d737fe700 1 --2- 172.21.15.64:0/1037518520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56115b7c95b0 0x56115b7cdab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.361+0000 7f9d737fe700 1 -- 172.21.15.64:0/1037518520 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d7410dd20 con 0x56115b7c3710 2023-09-13T03:17:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.361+0000 7f9d72ffd700 1 --2- 172.21.15.64:0/1037518520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56115b7c40d0 0x56115b7cd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:17:50.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.361+0000 7f9d737fe700 1 --2- 172.21.15.64:0/1037518520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56115b7c3710 0x56115b7d01e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c012f70 tx=0x7f9d6c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81ae82394917200a server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:17:50.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.361+0000 7f9d52ffd700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:17:50.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.361+0000 7f9d52ffd700 1 -- 172.21.15.64:0/1037518520 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c00e040 con 0x56115b7c3710 2023-09-13T03:17:50.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.361+0000 7f9d8d326b80 1 -- 172.21.15.64:0/1037518520 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56115b669490 con 0x56115b7c3710 2023-09-13T03:17:50.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.361+0000 7f9d52ffd700 1 -- 172.21.15.64:0/1037518520 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f9d6c019a50 con 0x56115b7c3710 2023-09-13T03:17:50.376 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.362+0000 7f9d52ffd700 1 -- 172.21.15.64:0/1037518520 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d6c01ae80 con 0x56115b7c3710 2023-09-13T03:17:50.376 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.362+0000 7f9d8d326b80 10 client.16461 fetch_fsmap learned FSMap version 44 2023-09-13T03:17:50.376 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.362+0000 7f9d8d326b80 1 -- 172.21.15.64:0/1037518520 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x56115b526c00 con 0x56115b7c3710 2023-09-13T03:17:50.376 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.362+0000 7f9d52ffd700 1 -- 172.21.15.64:0/1037518520 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56115b669490 con 0x56115b7c3710 2023-09-13T03:17:50.376 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.362+0000 7f9d52ffd700 1 -- 172.21.15.64:0/1037518520 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 44) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f9d6c014030 con 0x56115b7c3710 2023-09-13T03:17:50.376 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.362+0000 7f9d8d326b80 10 client.16461 fetch_fsmap finished waiting for FSMap version 44 2023-09-13T03:17:50.377 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.362+0000 7f9d8d326b80 10 client.16461 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:17:50.377 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.362+0000 7f9d8d326b80 20 client.16461 populate_metadata read hostname 'smithi064' 2023-09-13T03:17:50.377 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.362+0000 7f9d8d326b80 1 --2- 172.21.15.64:0/1037518520 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56115b7dd830 0x56115b7dfc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:50.377 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.362+0000 7f9d8d326b80 4 client.16461 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:17:50.377 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.362+0000 7f9d8d326b80 1 -- 172.21.15.64:0/1037518520 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56115b726840 con 0x56115b7dd830 2023-09-13T03:17:50.378 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.363+0000 7f9d72ffd700 1 --2- 172.21.15.64:0/1037518520 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56115b7dd830 0x56115b7dfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:50.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.363+0000 7f9d52ffd700 10 client.16461 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:17:50.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.363+0000 7f9d72ffd700 1 --2- 172.21.15.64:0/1037518520 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56115b7dd830 0x56115b7dfc50 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbd5abf442c4f21a server_cookie=263f83768ea24e49 in_seq=0 out_seq=0 2023-09-13T03:17:50.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.364+0000 7f9d52ffd700 1 -- 172.21.15.64:0/1037518520 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56115b726840 con 0x56115b7dd830 2023-09-13T03:17:50.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.364+0000 7f9d52ffd700 10 client.16461 handle_command_reply: tid=0 2023-09-13T03:17:50.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.364+0000 7f9d8d326b80 1 client.16461 shutdown 2023-09-13T03:17:50.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.364+0000 7f9d8d326b80 1 -- 172.21.15.64:0/1037518520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56115b7c3710 msgr2=0x56115b7d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:17:50.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.364+0000 7f9d8d326b80 1 --2- 172.21.15.64:0/1037518520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56115b7c3710 0x56115b7d01e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c012f70 tx=0x7f9d6c00b040 comp rx=0 tx=0).stop 2023-09-13T03:17:50.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.364+0000 7f9d8d326b80 1 -- 172.21.15.64:0/1037518520 shutdown_connections 2023-09-13T03:17:50.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.364+0000 7f9d8d326b80 1 --2- 172.21.15.64:0/1037518520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56115b7c95b0 0x56115b7cdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:50.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.364+0000 7f9d8d326b80 1 --2- 172.21.15.64:0/1037518520 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56115b7dd830 0x56115b7dfc50 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:50.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.364+0000 7f9d8d326b80 1 --2- 172.21.15.64:0/1037518520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56115b7c3710 0x56115b7d01e0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:50.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.364+0000 7f9d8d326b80 1 --2- 172.21.15.64:0/1037518520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56115b7c40d0 0x56115b7cd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:50.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.364+0000 7f9d52ffd700 0 client.16461 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:17:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.364+0000 7f9d8d326b80 1 -- 172.21.15.64:0/1037518520 >> 172.21.15.64:0/1037518520 conn(0x56115b734030 msgr2=0x56115b7b97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:17:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.364+0000 7f9d8d326b80 1 -- 172.21.15.64:0/1037518520 shutdown_connections 2023-09-13T03:17:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.365+0000 7f9d8d326b80 1 -- 172.21.15.64:0/1037518520 wait complete. 2023-09-13T03:17:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.365+0000 7f9d8d326b80 20 client.16461 trim_cache size 0 max 16384 2023-09-13T03:17:50.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.380+0000 7f9d8d326b80 1 Processor -- start 2023-09-13T03:17:50.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.380+0000 7f9d8d326b80 1 -- start start 2023-09-13T03:17:50.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.380+0000 7f9d8d326b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56115b7c3710 0x56115b7d47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:50.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.380+0000 7f9d8d326b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56115b7c40d0 0x56115b7ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:50.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.380+0000 7f9d8d326b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56115b7c95b0 0x56115b7cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.380+0000 7f9d8d326b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56115b7c9050 con 0x56115b7c95b0 2023-09-13T03:17:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.380+0000 7f9d8d326b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56115b7bb930 con 0x56115b7c3710 2023-09-13T03:17:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.380+0000 7f9d8d326b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56115b7c70e0 con 0x56115b7c40d0 2023-09-13T03:17:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.381+0000 7f9d737fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56115b7c3710 0x56115b7d47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.381+0000 7f9d737fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56115b7c95b0 0x56115b7cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.381+0000 7f9d737fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56115b7c95b0 0x56115b7cfd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45006/0 (socket says 172.21.15.64:45006) 2023-09-13T03:17:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.381+0000 7f9d737fe700 1 -- 172.21.15.64:0/2430082195 learned_addr learned my addr 172.21.15.64:0/2430082195 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:17:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.381+0000 7f9d72ffd700 1 --2- 172.21.15.64:0/2430082195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56115b7c40d0 0x56115b7ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.381+0000 7f9d737fe700 1 -- 172.21.15.64:0/2430082195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56115b7c40d0 msgr2=0x56115b7ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:17:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.381+0000 7f9d737fe700 1 --2- 172.21.15.64:0/2430082195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56115b7c40d0 0x56115b7ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.381+0000 7f9d737fe700 1 -- 172.21.15.64:0/2430082195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56115b7c3710 msgr2=0x56115b7d47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:17:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.381+0000 7f9d737fe700 1 --2- 172.21.15.64:0/2430082195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56115b7c3710 0x56115b7d47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.381+0000 7f9d737fe700 1 -- 172.21.15.64:0/2430082195 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56115b5641f0 con 0x56115b7c95b0 2023-09-13T03:17:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.381+0000 7f9d737fe700 1 --2- 172.21.15.64:0/2430082195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56115b7c95b0 0x56115b7cfd80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c024b80 tx=0x7f9d6c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145ff25e7cc2c15e server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:17:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.382+0000 7f9d5a7fc700 1 -- 172.21.15.64:0/2430082195 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c00e040 con 0x56115b7c95b0 2023-09-13T03:17:50.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.382+0000 7f9d5a7fc700 1 -- 172.21.15.64:0/2430082195 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d6c0098a0 con 0x56115b7c95b0 2023-09-13T03:17:50.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.382+0000 7f9d5a7fc700 1 -- 172.21.15.64:0/2430082195 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c024620 con 0x56115b7c95b0 2023-09-13T03:17:50.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.382+0000 7f9d8d326b80 1 -- 172.21.15.64:0/2430082195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56115b7c95b0 msgr2=0x56115b7cfd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:17:50.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.382+0000 7f9d8d326b80 1 --2- 172.21.15.64:0/2430082195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56115b7c95b0 0x56115b7cfd80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c024b80 tx=0x7f9d6c000f40 comp rx=0 tx=0).stop 2023-09-13T03:17:50.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.382+0000 7f9d8d326b80 1 -- 172.21.15.64:0/2430082195 shutdown_connections 2023-09-13T03:17:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.382+0000 7f9d8d326b80 1 --2- 172.21.15.64:0/2430082195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56115b7c3710 0x56115b7d47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.382+0000 7f9d8d326b80 1 --2- 172.21.15.64:0/2430082195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56115b7c95b0 0x56115b7cfd80 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.382+0000 7f9d8d326b80 1 --2- 172.21.15.64:0/2430082195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56115b7c40d0 0x56115b7ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.382+0000 7f9d8d326b80 1 -- 172.21.15.64:0/2430082195 >> 172.21.15.64:0/2430082195 conn(0x56115b734030 msgr2=0x56115b7c8520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:17:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.382+0000 7f9d8d326b80 1 -- 172.21.15.64:0/2430082195 shutdown_connections 2023-09-13T03:17:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.382+0000 7f9d8d326b80 1 -- 172.21.15.64:0/2430082195 wait complete. 2023-09-13T03:17:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.383+0000 7f9d8d326b80 1 -- start start 2023-09-13T03:17:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.383+0000 7f9d8d326b80 1 Processor -- start 2023-09-13T03:17:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.383+0000 7f9d8d326b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56115b7c3710 0x56115b7c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.383+0000 7f9d8d326b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56115b7c40d0 0x56115b7c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.384+0000 7f9d8d326b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56115b7c95b0 0x56115b7c59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.384+0000 7f9d8d326b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56115b7c6090 con 0x56115b7c40d0 2023-09-13T03:17:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.384+0000 7f9d8d326b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56115b7c6200 con 0x56115b7c3710 2023-09-13T03:17:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.384+0000 7f9d8d326b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56115b7c63c0 con 0x56115b7c95b0 2023-09-13T03:17:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.384+0000 7f9d8d326b80 10 client.0 _ll_register_callbacks cb 0x56115b5f9a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:17:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.384+0000 7f9d737fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56115b7c3710 0x56115b7c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.384+0000 7f9d72ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56115b7c40d0 0x56115b7c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.384+0000 7f9d737fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56115b7c3710 0x56115b7c4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:34558/0 (socket says 172.21.15.64:34558) 2023-09-13T03:17:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.384+0000 7f9d737fe700 1 -- 172.21.15.64:0/3535987735 learned_addr learned my addr 172.21.15.64:0/3535987735 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:17:50.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.384+0000 7f9d737fe700 1 --2- 172.21.15.64:0/3535987735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56115b7c95b0 0x56115b7c59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:50.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.384+0000 7f9d72ffd700 1 -- 172.21.15.64:0/3535987735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56115b7c95b0 msgr2=0x56115b7c59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:17:50.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.384+0000 7f9d72ffd700 1 --2- 172.21.15.64:0/3535987735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56115b7c95b0 0x56115b7c59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:50.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.384+0000 7f9d72ffd700 1 -- 172.21.15.64:0/3535987735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56115b7c3710 msgr2=0x56115b7c4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:17:50.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.384+0000 7f9d72ffd700 1 --2- 172.21.15.64:0/3535987735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56115b7c3710 0x56115b7c4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:50.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.384+0000 7f9d72ffd700 1 -- 172.21.15.64:0/3535987735 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56115b526c00 con 0x56115b7c40d0 2023-09-13T03:17:50.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.384+0000 7f9d737fe700 1 --2- 172.21.15.64:0/3535987735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56115b7c95b0 0x56115b7c59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T03:17:50.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.384+0000 7f9d72ffd700 1 --2- 172.21.15.64:0/3535987735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56115b7c40d0 0x56115b7c54a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9d640036c0 tx=0x7f9d64003950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc7b757fa90dbd6 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:17:50.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.384+0000 7f9d52ffd700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:17:50.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.385+0000 7f9d52ffd700 1 -- 172.21.15.64:0/3535987735 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d640300b0 con 0x56115b7c40d0 2023-09-13T03:17:50.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.385+0000 7f9d8d326b80 1 -- 172.21.15.64:0/3535987735 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9d6401e040 con 0x56115b7c40d0 2023-09-13T03:17:50.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.385+0000 7f9d52ffd700 1 -- 172.21.15.64:0/3535987735 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f9d64034070 con 0x56115b7c40d0 2023-09-13T03:17:50.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.385+0000 7f9d52ffd700 1 -- 172.21.15.64:0/3535987735 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d64030210 con 0x56115b7c40d0 2023-09-13T03:17:50.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.385+0000 7f9d8d326b80 10 client.16467 fetch_fsmap learned FSMap version 44 2023-09-13T03:17:50.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.385+0000 7f9d8d326b80 1 -- 172.21.15.64:0/3535987735 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x56115b53c250 con 0x56115b7c40d0 2023-09-13T03:17:50.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.385+0000 7f9d52ffd700 1 -- 172.21.15.64:0/3535987735 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d6401e040 con 0x56115b7c40d0 2023-09-13T03:17:50.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.385+0000 7f9d52ffd700 1 -- 172.21.15.64:0/3535987735 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 44) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f9d6400f460 con 0x56115b7c40d0 2023-09-13T03:17:50.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.385+0000 7f9d8d326b80 10 client.16467 fetch_fsmap finished waiting for FSMap version 44 2023-09-13T03:17:50.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.385+0000 7f9d8d326b80 10 client.16467 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:17:50.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.385+0000 7f9d8d326b80 20 client.16467 populate_metadata read hostname 'smithi064' 2023-09-13T03:17:50.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.385+0000 7f9d8d326b80 1 --2- 172.21.15.64:0/3535987735 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56115b8323a0 0x56115b8347c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:17:50.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.385+0000 7f9d8d326b80 4 client.16467 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:17:50.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.385+0000 7f9d8d326b80 1 -- 172.21.15.64:0/3535987735 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56115b70a8c0 con 0x56115b8323a0 2023-09-13T03:17:50.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.386+0000 7f9d737fe700 1 --2- 172.21.15.64:0/3535987735 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56115b8323a0 0x56115b8347c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:17:50.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.386+0000 7f9d737fe700 1 --2- 172.21.15.64:0/3535987735 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56115b8323a0 0x56115b8347c0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d87bd53ff212369 server_cookie=3f7dbf57b3441bdd in_seq=0 out_seq=0 2023-09-13T03:17:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.386+0000 7f9d52ffd700 10 client.16467 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:17:50.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.412+0000 7f9d52ffd700 1 -- 172.21.15.64:0/3535987735 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56115b70a8c0 con 0x56115b8323a0 2023-09-13T03:17:50.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.412+0000 7f9d52ffd700 10 client.16467 handle_command_reply: tid=0 2023-09-13T03:17:50.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.412+0000 7f9d8d326b80 1 client.16467 shutdown 2023-09-13T03:17:50.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.412+0000 7f9d8d326b80 1 -- 172.21.15.64:0/3535987735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56115b7c40d0 msgr2=0x56115b7c54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:17:50.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.412+0000 7f9d8d326b80 1 --2- 172.21.15.64:0/3535987735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56115b7c40d0 0x56115b7c54a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9d640036c0 tx=0x7f9d64003950 comp rx=0 tx=0).stop 2023-09-13T03:17:50.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.412+0000 7f9d8d326b80 1 -- 172.21.15.64:0/3535987735 shutdown_connections 2023-09-13T03:17:50.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.412+0000 7f9d8d326b80 1 --2- 172.21.15.64:0/3535987735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56115b7c3710 0x56115b7c4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:50.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.412+0000 7f9d8d326b80 1 --2- 172.21.15.64:0/3535987735 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56115b8323a0 0x56115b8347c0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:50.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.412+0000 7f9d8d326b80 1 --2- 172.21.15.64:0/3535987735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56115b7c95b0 0x56115b7c59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:50.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.412+0000 7f9d8d326b80 1 --2- 172.21.15.64:0/3535987735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56115b7c40d0 0x56115b7c54a0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:50.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.412+0000 7f9d52ffd700 0 client.16467 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:17:50.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.412+0000 7f9d8d326b80 1 -- 172.21.15.64:0/3535987735 >> 172.21.15.64:0/3535987735 conn(0x56115b734030 msgr2=0x56115b7c75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:17:50.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.412+0000 7f9d8d326b80 1 -- 172.21.15.64:0/3535987735 shutdown_connections 2023-09-13T03:17:50.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.412+0000 7f9d8d326b80 1 -- 172.21.15.64:0/3535987735 wait complete. 2023-09-13T03:17:50.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.412+0000 7f9d8d326b80 20 client.16467 trim_cache size 0 max 16384 2023-09-13T03:17:50.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:17:50.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:17:50.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:17:50.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:17:50.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.413+0000 7f9d799ff700 1 -- 172.21.15.64:0/2050151419 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f9d5c0a0830 msgr2=0x7f9d5c0a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:17:50.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.413+0000 7f9d799ff700 1 --2- 172.21.15.64:0/2050151419 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f9d5c0a0830 0x7f9d5c0a2cf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9d68096060 tx=0x7f9d68005760 comp rx=0 tx=0).stop 2023-09-13T03:17:50.418 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.413+0000 7f9d799ff700 1 -- 172.21.15.64:0/2050151419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d74100f60 msgr2=0x7f9d7419caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:17:50.418 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.413+0000 7f9d799ff700 1 --2- 172.21.15.64:0/2050151419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d74100f60 0x7f9d7419caa0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9d6800e9f0 tx=0x7f9d6800ed00 comp rx=0 tx=0).stop 2023-09-13T03:17:50.418 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.413+0000 7f9d799ff700 1 -- 172.21.15.64:0/2050151419 shutdown_connections 2023-09-13T03:17:50.418 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.413+0000 7f9d799ff700 1 --2- 172.21.15.64:0/2050151419 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f9d5c0a0830 0x7f9d5c0a2cf0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.413+0000 7f9d799ff700 1 --2- 172.21.15.64:0/2050151419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d74106660 0x7f9d741963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.413+0000 7f9d799ff700 1 --2- 172.21.15.64:0/2050151419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d741005a0 0x7f9d7419c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.413+0000 7f9d799ff700 1 --2- 172.21.15.64:0/2050151419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d74100f60 0x7f9d7419caa0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:17:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.413+0000 7f9d799ff700 1 -- 172.21.15.64:0/2050151419 >> 172.21.15.64:0/2050151419 conn(0x7f9d740fc050 msgr2=0x7f9d740fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:17:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.413+0000 7f9d799ff700 1 -- 172.21.15.64:0/2050151419 shutdown_connections 2023-09-13T03:17:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:17:50.414+0000 7f9d799ff700 1 -- 172.21.15.64:0/2050151419 wait complete. 2023-09-13T03:17:50.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:17:50 smithi170 ceph-mon[134328]: pgmap v143: 97 pgs: 97 active+clean; 60 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 1013 KiB/s wr, 97 op/s 2023-09-13T03:17:50.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:17:50 smithi064 ceph-mon[97675]: pgmap v143: 97 pgs: 97 active+clean; 60 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 1013 KiB/s wr, 97 op/s 2023-09-13T03:17:50.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:17:50 smithi142 ceph-mon[134082]: pgmap v143: 97 pgs: 97 active+clean; 60 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 1013 KiB/s wr, 97 op/s 2023-09-13T03:17:50.909 INFO:tasks.workunit.client.0.smithi064.stdout: 1 100152 2.16 MB/sec execute 115 sec latency 2331.166 ms 2023-09-13T03:17:51.909 INFO:tasks.workunit.client.0.smithi064.stdout: 1 100152 2.14 MB/sec execute 116 sec latency 3331.261 ms 2023-09-13T03:17:52.909 INFO:tasks.workunit.client.0.smithi064.stdout: 1 100152 2.13 MB/sec execute 117 sec latency 4331.402 ms 2023-09-13T03:17:52.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:17:52 smithi170 ceph-mon[134328]: pgmap v144: 97 pgs: 97 active+clean; 55 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 1.1 MiB/s wr, 124 op/s 2023-09-13T03:17:52.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:17:52 smithi064 ceph-mon[97675]: pgmap v144: 97 pgs: 97 active+clean; 55 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 1.1 MiB/s wr, 124 op/s 2023-09-13T03:17:52.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:17:52 smithi142 ceph-mon[134082]: pgmap v144: 97 pgs: 97 active+clean; 55 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 1.1 MiB/s wr, 124 op/s 2023-09-13T03:17:53.909 INFO:tasks.workunit.client.0.smithi064.stdout: 1 100830 2.15 MB/sec execute 118 sec latency 4946.844 ms 2023-09-13T03:17:54.909 INFO:tasks.workunit.client.0.smithi064.stdout: 1 101053 2.13 MB/sec execute 119 sec latency 845.270 ms 2023-09-13T03:17:54.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:17:54 smithi170 ceph-mon[134328]: pgmap v145: 97 pgs: 97 active+clean; 55 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 685 KiB/s wr, 66 op/s 2023-09-13T03:17:54.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:17:54 smithi064 ceph-mon[97675]: pgmap v145: 97 pgs: 97 active+clean; 55 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 685 KiB/s wr, 66 op/s 2023-09-13T03:17:54.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:17:54 smithi142 ceph-mon[134082]: pgmap v145: 97 pgs: 97 active+clean; 55 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 685 KiB/s wr, 66 op/s 2023-09-13T03:17:55.910 INFO:tasks.workunit.client.0.smithi064.stdout: 1 101053 2.11 MB/sec execute 120 sec latency 1845.427 ms 2023-09-13T03:17:56.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:17:56 smithi170 ceph-mon[134328]: pgmap v146: 97 pgs: 97 active+clean; 56 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 784 KiB/s wr, 84 op/s 2023-09-13T03:17:56.910 INFO:tasks.workunit.client.0.smithi064.stdout: 1 101053 2.10 MB/sec execute 121 sec latency 2845.541 ms 2023-09-13T03:17:56.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:17:56 smithi064 ceph-mon[97675]: pgmap v146: 97 pgs: 97 active+clean; 56 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 784 KiB/s wr, 84 op/s 2023-09-13T03:17:56.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:17:56 smithi142 ceph-mon[134082]: pgmap v146: 97 pgs: 97 active+clean; 56 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 784 KiB/s wr, 84 op/s 2023-09-13T03:17:57.910 INFO:tasks.workunit.client.0.smithi064.stdout: 1 101053 2.08 MB/sec execute 122 sec latency 3845.703 ms 2023-09-13T03:17:58.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:17:58 smithi170 ceph-mon[134328]: pgmap v147: 97 pgs: 97 active+clean; 56 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 169 KiB/s wr, 43 op/s 2023-09-13T03:17:58.910 INFO:tasks.workunit.client.0.smithi064.stdout: 1 101053 2.06 MB/sec execute 123 sec latency 4845.790 ms 2023-09-13T03:17:58.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:17:58 smithi064 ceph-mon[97675]: pgmap v147: 97 pgs: 97 active+clean; 56 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 169 KiB/s wr, 43 op/s 2023-09-13T03:17:58.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:17:58 smithi142 ceph-mon[134082]: pgmap v147: 97 pgs: 97 active+clean; 56 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 169 KiB/s wr, 43 op/s 2023-09-13T03:17:59.910 INFO:tasks.workunit.client.0.smithi064.stdout: 1 102912 2.14 MB/sec execute 124 sec latency 4978.314 ms 2023-09-13T03:18:00.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:18:00 smithi170 ceph-mon[134328]: pgmap v148: 97 pgs: 97 active+clean; 56 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 169 KiB/s wr, 43 op/s 2023-09-13T03:18:00.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:18:00 smithi064 ceph-mon[97675]: pgmap v148: 97 pgs: 97 active+clean; 56 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 169 KiB/s wr, 43 op/s 2023-09-13T03:18:00.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:18:00 smithi142 ceph-mon[134082]: pgmap v148: 97 pgs: 97 active+clean; 56 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 169 KiB/s wr, 43 op/s 2023-09-13T03:18:00.910 INFO:tasks.workunit.client.0.smithi064.stdout: 1 103370 2.14 MB/sec execute 125 sec latency 436.457 ms 2023-09-13T03:18:01.910 INFO:tasks.workunit.client.0.smithi064.stdout: 1 103370 2.12 MB/sec execute 126 sec latency 1436.507 ms 2023-09-13T03:18:02.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:18:02 smithi170 ceph-mon[134328]: pgmap v149: 97 pgs: 97 active+clean; 57 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 328 KiB/s wr, 76 op/s 2023-09-13T03:18:02.910 INFO:tasks.workunit.client.0.smithi064.stdout: 1 103370 2.11 MB/sec execute 127 sec latency 2436.643 ms 2023-09-13T03:18:02.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:18:02 smithi064 ceph-mon[97675]: pgmap v149: 97 pgs: 97 active+clean; 57 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 328 KiB/s wr, 76 op/s 2023-09-13T03:18:02.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:18:02 smithi142 ceph-mon[134082]: pgmap v149: 97 pgs: 97 active+clean; 57 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 328 KiB/s wr, 76 op/s 2023-09-13T03:18:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.576+0000 7fe209bff700 1 -- 172.21.15.64:0/2386713611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe204106660 msgr2=0x7fe204106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.576+0000 7fe209bff700 1 --2- 172.21.15.64:0/2386713611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe204106660 0x7fe204106a40 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fe1f40099c0 tx=0x7fe1f4009cd0 comp rx=0 tx=0).stop 2023-09-13T03:18:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.576+0000 7fe209bff700 1 -- 172.21.15.64:0/2386713611 shutdown_connections 2023-09-13T03:18:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.576+0000 7fe209bff700 1 --2- 172.21.15.64:0/2386713611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe204100f60 0x7fe20410d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.576+0000 7fe209bff700 1 --2- 172.21.15.64:0/2386713611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe204106660 0x7fe204106a40 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.576+0000 7fe209bff700 1 --2- 172.21.15.64:0/2386713611 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2041005a0 0x7fe204100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.576+0000 7fe209bff700 1 -- 172.21.15.64:0/2386713611 >> 172.21.15.64:0/2386713611 conn(0x7fe2040fc050 msgr2=0x7fe2040fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:18:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.576+0000 7fe209bff700 1 -- 172.21.15.64:0/2386713611 shutdown_connections 2023-09-13T03:18:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.576+0000 7fe209bff700 1 -- 172.21.15.64:0/2386713611 wait complete. 2023-09-13T03:18:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.577+0000 7fe209bff700 1 Processor -- start 2023-09-13T03:18:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.577+0000 7fe209bff700 1 -- start start 2023-09-13T03:18:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.578+0000 7fe209bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2041005a0 0x7fe20419c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.578+0000 7fe209bff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe204100f60 0x7fe20419cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.578+0000 7fe209bff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe204106660 0x7fe204196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.578+0000 7fe209bff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe204196ac0 con 0x7fe204100f60 2023-09-13T03:18:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.578+0000 7fe209bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe204196c30 con 0x7fe2041005a0 2023-09-13T03:18:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.578+0000 7fe209bff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe204196df0 con 0x7fe204106660 2023-09-13T03:18:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.578+0000 7fe202ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe204100f60 0x7fe20419cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.578+0000 7fe203fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe204106660 0x7fe204196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.578+0000 7fe203fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe204106660 0x7fe204196450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:50694/0 (socket says 172.21.15.64:50694) 2023-09-13T03:18:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.578+0000 7fe203fff700 1 -- 172.21.15.64:0/4211125728 learned_addr learned my addr 172.21.15.64:0/4211125728 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:18:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.578+0000 7fe2037fe700 1 --2- 172.21.15.64:0/4211125728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2041005a0 0x7fe20419c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.579+0000 7fe202ffd700 1 -- 172.21.15.64:0/4211125728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe204106660 msgr2=0x7fe204196450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.579+0000 7fe202ffd700 1 --2- 172.21.15.64:0/4211125728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe204106660 0x7fe204196450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.579+0000 7fe202ffd700 1 -- 172.21.15.64:0/4211125728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2041005a0 msgr2=0x7fe20419c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.579+0000 7fe202ffd700 1 --2- 172.21.15.64:0/4211125728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2041005a0 0x7fe20419c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.579+0000 7fe202ffd700 1 -- 172.21.15.64:0/4211125728 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f40095f0 con 0x7fe204100f60 2023-09-13T03:18:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.579+0000 7fe203fff700 1 --2- 172.21.15.64:0/4211125728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe204106660 0x7fe204196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:18:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.579+0000 7fe202ffd700 1 --2- 172.21.15.64:0/4211125728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe204100f60 0x7fe20419cb30 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe1f800d750 tx=0x7fe1f800da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:18:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.579+0000 7fe200ff9700 1 -- 172.21.15.64:0/4211125728 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8004020 con 0x7fe204100f60 2023-09-13T03:18:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.579+0000 7fe209bff700 1 -- 172.21.15.64:0/4211125728 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2041970d0 con 0x7fe204100f60 2023-09-13T03:18:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.579+0000 7fe200ff9700 1 -- 172.21.15.64:0/4211125728 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1f8004830 con 0x7fe204100f60 2023-09-13T03:18:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.579+0000 7fe200ff9700 1 -- 172.21.15.64:0/4211125728 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f80054c0 con 0x7fe204100f60 2023-09-13T03:18:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.579+0000 7fe209bff700 1 -- 172.21.15.64:0/4211125728 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2041a35e0 con 0x7fe204100f60 2023-09-13T03:18:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.583+0000 7fe200ff9700 1 -- 172.21.15.64:0/4211125728 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fe1f80049a0 con 0x7fe204100f60 2023-09-13T03:18:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.584+0000 7fe200ff9700 1 --2- 172.21.15.64:0/4211125728 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fe1ec0a07b0 0x7fe1ec0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.584+0000 7fe21d5d8b80 1 Processor -- start 2023-09-13T03:18:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.584+0000 7fe200ff9700 1 -- 172.21.15.64:0/4211125728 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe1f80cb920 con 0x7fe204100f60 2023-09-13T03:18:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.584+0000 7fe21d5d8b80 1 -- start start 2023-09-13T03:18:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.584+0000 7fe2037fe700 1 --2- 172.21.15.64:0/4211125728 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fe1ec0a07b0 0x7fe1ec0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.585+0000 7fe21d5d8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614790e05b0 0x5614790e0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.585+0000 7fe21d5d8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614790da710 0x5614790dab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.585+0000 7fe21d5d8b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5614790db0d0 0x5614790e7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.585+0000 7fe21d5d8b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56147902f6d0 con 0x5614790db0d0 2023-09-13T03:18:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.585+0000 7fe21d5d8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56147902cff0 con 0x5614790da710 2023-09-13T03:18:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.585+0000 7fe21d5d8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614790288e0 con 0x5614790e05b0 2023-09-13T03:18:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.585+0000 7fe2037fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614790e05b0 0x5614790e0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.585+0000 7fe203fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5614790db0d0 0x5614790e7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.585+0000 7fe2037fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614790e05b0 0x5614790e0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:50704/0 (socket says 172.21.15.64:50704) 2023-09-13T03:18:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.585+0000 7fe203fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5614790db0d0 0x5614790e7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58782/0 (socket says 172.21.15.64:58782) 2023-09-13T03:18:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.585+0000 7fe2037fe700 1 -- 172.21.15.64:0/3361265764 learned_addr learned my addr 172.21.15.64:0/3361265764 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:18:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.585+0000 7fe202ffd700 1 --2- 172.21.15.64:0/3361265764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614790da710 0x5614790dab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.585+0000 7fe2037fe700 1 --2- 172.21.15.64:0/4211125728 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fe1ec0a07b0 0x7fe1ec0a2c70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4009990 tx=0x7fe1f4005fb0 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:18:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.585+0000 7fe203fff700 1 -- 172.21.15.64:0/3361265764 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614790e05b0 msgr2=0x5614790e0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:18:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.585+0000 7fe203fff700 1 --2- 172.21.15.64:0/3361265764 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614790e05b0 0x5614790e0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.585+0000 7fe203fff700 1 -- 172.21.15.64:0/3361265764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614790da710 msgr2=0x5614790dab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:18:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.585+0000 7fe203fff700 1 --2- 172.21.15.64:0/3361265764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614790da710 0x5614790dab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.585+0000 7fe203fff700 1 -- 172.21.15.64:0/3361265764 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561478e54d80 con 0x5614790db0d0 2023-09-13T03:18:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.586+0000 7fe203fff700 1 --2- 172.21.15.64:0/3361265764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5614790db0d0 0x5614790e7340 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe20419d2f0 tx=0x7fe1fc009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98d4a7117ef8a7de server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:18:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.586+0000 7fe1e9ffb700 1 -- 172.21.15.64:0/3361265764 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc004db0 con 0x5614790db0d0 2023-09-13T03:18:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.586+0000 7fe1e9ffb700 1 -- 172.21.15.64:0/3361265764 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1fc00f040 con 0x5614790db0d0 2023-09-13T03:18:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.586+0000 7fe1e9ffb700 1 -- 172.21.15.64:0/3361265764 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc0053c0 con 0x5614790db0d0 2023-09-13T03:18:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.587+0000 7fe21d5d8b80 1 -- 172.21.15.64:0/3361265764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5614790db0d0 msgr2=0x5614790e7340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.587+0000 7fe21d5d8b80 1 --2- 172.21.15.64:0/3361265764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5614790db0d0 0x5614790e7340 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe20419d2f0 tx=0x7fe1fc009920 comp rx=0 tx=0).stop 2023-09-13T03:18:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.587+0000 7fe21d5d8b80 1 -- 172.21.15.64:0/3361265764 shutdown_connections 2023-09-13T03:18:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.587+0000 7fe21d5d8b80 1 --2- 172.21.15.64:0/3361265764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5614790db0d0 0x5614790e7340 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.587+0000 7fe21d5d8b80 1 --2- 172.21.15.64:0/3361265764 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614790e05b0 0x5614790e0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.587+0000 7fe21d5d8b80 1 --2- 172.21.15.64:0/3361265764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614790da710 0x5614790dab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.587+0000 7fe21d5d8b80 1 -- 172.21.15.64:0/3361265764 >> 172.21.15.64:0/3361265764 conn(0x56147904b030 msgr2=0x5614790d1560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:18:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.587+0000 7fe21d5d8b80 1 -- 172.21.15.64:0/3361265764 shutdown_connections 2023-09-13T03:18:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.587+0000 7fe21d5d8b80 1 -- 172.21.15.64:0/3361265764 wait complete. 2023-09-13T03:18:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.587+0000 7fe21d5d8b80 1 -- start start 2023-09-13T03:18:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.588+0000 7fe21d5d8b80 1 Processor -- start 2023-09-13T03:18:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.588+0000 7fe21d5d8b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5614790da710 0x5614790e7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.588+0000 7fe21d5d8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614790db0d0 0x5614790e4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.588+0000 7fe21d5d8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614790e05b0 0x5614790e4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.588+0000 7fe21d5d8b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561478fc6b10 con 0x5614790da710 2023-09-13T03:18:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.588+0000 7fe21d5d8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561478fc6810 con 0x5614790e05b0 2023-09-13T03:18:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.588+0000 7fe21d5d8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561478eaa8c0 con 0x5614790db0d0 2023-09-13T03:18:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.588+0000 7fe21d5d8b80 10 client.0 _ll_register_callbacks cb 0x561478f10a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:18:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.589+0000 7fe2037fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5614790da710 0x5614790e7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.589+0000 7fe2037fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5614790da710 0x5614790e7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58788/0 (socket says 172.21.15.64:58788) 2023-09-13T03:18:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.589+0000 7fe2037fe700 1 -- 172.21.15.64:0/550345695 learned_addr learned my addr 172.21.15.64:0/550345695 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:18:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.589+0000 7fe202ffd700 1 --2- 172.21.15.64:0/550345695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614790db0d0 0x5614790e4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.589+0000 7fe203fff700 1 --2- 172.21.15.64:0/550345695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614790e05b0 0x5614790e4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.589+0000 7fe2037fe700 1 -- 172.21.15.64:0/550345695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614790db0d0 msgr2=0x5614790e4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:18:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.589+0000 7fe2037fe700 1 --2- 172.21.15.64:0/550345695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614790db0d0 0x5614790e4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.589+0000 7fe2037fe700 1 -- 172.21.15.64:0/550345695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614790e05b0 msgr2=0x5614790e4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:18:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.589+0000 7fe2037fe700 1 --2- 172.21.15.64:0/550345695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614790e05b0 0x5614790e4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.589+0000 7fe2037fe700 1 -- 172.21.15.64:0/550345695 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1fc0090f0 con 0x5614790da710 2023-09-13T03:18:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.589+0000 7fe202ffd700 1 --2- 172.21.15.64:0/550345695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614790db0d0 0x5614790e4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:18:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.589+0000 7fe203fff700 1 --2- 172.21.15.64:0/550345695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614790e05b0 0x5614790e4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:18:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.589+0000 7fe2037fe700 1 --2- 172.21.15.64:0/550345695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5614790da710 0x5614790e7150 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe1f40058e0 tx=0x7fe1f4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b37af83c4e957c48 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:18:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.589+0000 7fe1e2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:18:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.590+0000 7fe1e2ffd700 1 -- 172.21.15.64:0/550345695 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f400b990 con 0x5614790da710 2023-09-13T03:18:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.590+0000 7fe21d5d8b80 1 -- 172.21.15.64:0/550345695 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561478f80490 con 0x5614790da710 2023-09-13T03:18:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.590+0000 7fe1e2ffd700 1 -- 172.21.15.64:0/550345695 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe1f400baf0 con 0x5614790da710 2023-09-13T03:18:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.590+0000 7fe1e2ffd700 1 -- 172.21.15.64:0/550345695 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1f400bcb0 con 0x5614790da710 2023-09-13T03:18:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.590+0000 7fe21d5d8b80 10 client.16479 fetch_fsmap learned FSMap version 44 2023-09-13T03:18:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.590+0000 7fe21d5d8b80 1 -- 172.21.15.64:0/550345695 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x561478e3dc00 con 0x5614790da710 2023-09-13T03:18:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.590+0000 7fe1e2ffd700 1 -- 172.21.15.64:0/550345695 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561478f80490 con 0x5614790da710 2023-09-13T03:18:03.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.590+0000 7fe1e2ffd700 1 -- 172.21.15.64:0/550345695 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 44) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fe1f400f560 con 0x5614790da710 2023-09-13T03:18:03.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.590+0000 7fe21d5d8b80 10 client.16479 fetch_fsmap finished waiting for FSMap version 44 2023-09-13T03:18:03.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.590+0000 7fe21d5d8b80 10 client.16479 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:18:03.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.591+0000 7fe21d5d8b80 20 client.16479 populate_metadata read hostname 'smithi064' 2023-09-13T03:18:03.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.591+0000 7fe21d5d8b80 1 --2- 172.21.15.64:0/550345695 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5614790f4680 0x5614790f6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.591+0000 7fe21d5d8b80 4 client.16479 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:18:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.591+0000 7fe21d5d8b80 1 -- 172.21.15.64:0/550345695 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56147903d840 con 0x5614790f4680 2023-09-13T03:18:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.591+0000 7fe202ffd700 1 --2- 172.21.15.64:0/550345695 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5614790f4680 0x5614790f6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.591+0000 7fe202ffd700 1 --2- 172.21.15.64:0/550345695 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5614790f4680 0x5614790f6aa0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5bdcf980a1d8e6 server_cookie=66bf6975264d0c75 in_seq=0 out_seq=0 2023-09-13T03:18:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.591+0000 7fe1e2ffd700 10 client.16479 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:18:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.592+0000 7fe1e2ffd700 1 -- 172.21.15.64:0/550345695 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56147903d840 con 0x5614790f4680 2023-09-13T03:18:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.592+0000 7fe1e2ffd700 10 client.16479 handle_command_reply: tid=0 2023-09-13T03:18:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.592+0000 7fe21d5d8b80 1 client.16479 shutdown 2023-09-13T03:18:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.592+0000 7fe21d5d8b80 1 -- 172.21.15.64:0/550345695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5614790da710 msgr2=0x5614790e7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.592+0000 7fe21d5d8b80 1 --2- 172.21.15.64:0/550345695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5614790da710 0x5614790e7150 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe1f40058e0 tx=0x7fe1f4009310 comp rx=0 tx=0).stop 2023-09-13T03:18:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.592+0000 7fe21d5d8b80 1 -- 172.21.15.64:0/550345695 shutdown_connections 2023-09-13T03:18:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.592+0000 7fe21d5d8b80 1 --2- 172.21.15.64:0/550345695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614790e05b0 0x5614790e4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.592+0000 7fe21d5d8b80 1 --2- 172.21.15.64:0/550345695 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5614790f4680 0x5614790f6aa0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.592+0000 7fe21d5d8b80 1 --2- 172.21.15.64:0/550345695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5614790da710 0x5614790e7150 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.592+0000 7fe21d5d8b80 1 --2- 172.21.15.64:0/550345695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614790db0d0 0x5614790e4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.592+0000 7fe1e2ffd700 0 client.16479 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:18:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.592+0000 7fe21d5d8b80 1 -- 172.21.15.64:0/550345695 >> 172.21.15.64:0/550345695 conn(0x56147904b030 msgr2=0x5614790d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:18:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.592+0000 7fe21d5d8b80 1 -- 172.21.15.64:0/550345695 shutdown_connections 2023-09-13T03:18:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.593+0000 7fe21d5d8b80 1 -- 172.21.15.64:0/550345695 wait complete. 2023-09-13T03:18:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.593+0000 7fe21d5d8b80 20 client.16479 trim_cache size 0 max 16384 2023-09-13T03:18:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.608+0000 7fe21d5d8b80 1 Processor -- start 2023-09-13T03:18:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.608+0000 7fe21d5d8b80 1 -- start start 2023-09-13T03:18:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.608+0000 7fe21d5d8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614790da710 0x5614790eb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.608+0000 7fe21d5d8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614790db0d0 0x5614790dfba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.608+0000 7fe21d5d8b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5614790e05b0 0x5614790e6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.608+0000 7fe21d5d8b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614790db990 con 0x5614790e05b0 2023-09-13T03:18:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.608+0000 7fe21d5d8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614790cdfc0 con 0x5614790db0d0 2023-09-13T03:18:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.608+0000 7fe21d5d8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614790c8ee0 con 0x5614790da710 2023-09-13T03:18:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.609+0000 7fe203fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5614790e05b0 0x5614790e6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.609+0000 7fe203fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5614790e05b0 0x5614790e6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58794/0 (socket says 172.21.15.64:58794) 2023-09-13T03:18:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.609+0000 7fe203fff700 1 -- 172.21.15.64:0/930975331 learned_addr learned my addr 172.21.15.64:0/930975331 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:18:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.609+0000 7fe2037fe700 1 --2- 172.21.15.64:0/930975331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614790da710 0x5614790eb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.609+0000 7fe202ffd700 1 --2- 172.21.15.64:0/930975331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614790db0d0 0x5614790dfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.609+0000 7fe203fff700 1 -- 172.21.15.64:0/930975331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614790da710 msgr2=0x5614790eb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:18:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.609+0000 7fe203fff700 1 --2- 172.21.15.64:0/930975331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614790da710 0x5614790eb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:03.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.609+0000 7fe203fff700 1 -- 172.21.15.64:0/930975331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614790db0d0 msgr2=0x5614790dfba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:18:03.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.609+0000 7fe203fff700 1 --2- 172.21.15.64:0/930975331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614790db0d0 0x5614790dfba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:03.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.609+0000 7fe203fff700 1 -- 172.21.15.64:0/930975331 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561478e7b1f0 con 0x5614790e05b0 2023-09-13T03:18:03.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.609+0000 7fe203fff700 1 --2- 172.21.15.64:0/930975331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5614790e05b0 0x5614790e6c10 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc0090c0 tx=0x7fe1fc007540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c866dc73908e31f0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:18:03.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.610+0000 7fe1ea7fc700 1 -- 172.21.15.64:0/930975331 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc004db0 con 0x5614790e05b0 2023-09-13T03:18:03.619 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.610+0000 7fe1ea7fc700 1 -- 172.21.15.64:0/930975331 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1fc023040 con 0x5614790e05b0 2023-09-13T03:18:03.619 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.610+0000 7fe1ea7fc700 1 -- 172.21.15.64:0/930975331 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc015830 con 0x5614790e05b0 2023-09-13T03:18:03.619 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.610+0000 7fe21d5d8b80 1 -- 172.21.15.64:0/930975331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5614790e05b0 msgr2=0x5614790e6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:03.619 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.610+0000 7fe21d5d8b80 1 --2- 172.21.15.64:0/930975331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5614790e05b0 0x5614790e6c10 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc0090c0 tx=0x7fe1fc007540 comp rx=0 tx=0).stop 2023-09-13T03:18:03.619 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.610+0000 7fe21d5d8b80 1 -- 172.21.15.64:0/930975331 shutdown_connections 2023-09-13T03:18:03.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.610+0000 7fe21d5d8b80 1 --2- 172.21.15.64:0/930975331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5614790e05b0 0x5614790e6c10 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:03.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.610+0000 7fe21d5d8b80 1 --2- 172.21.15.64:0/930975331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614790da710 0x5614790eb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:03.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.610+0000 7fe21d5d8b80 1 --2- 172.21.15.64:0/930975331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614790db0d0 0x5614790dfba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:03.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.610+0000 7fe21d5d8b80 1 -- 172.21.15.64:0/930975331 >> 172.21.15.64:0/930975331 conn(0x56147904b030 msgr2=0x5614790d5b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:18:03.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.611+0000 7fe21d5d8b80 1 -- 172.21.15.64:0/930975331 shutdown_connections 2023-09-13T03:18:03.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.611+0000 7fe21d5d8b80 1 -- 172.21.15.64:0/930975331 wait complete. 2023-09-13T03:18:03.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.611+0000 7fe21d5d8b80 1 -- start start 2023-09-13T03:18:03.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.611+0000 7fe21d5d8b80 1 Processor -- start 2023-09-13T03:18:03.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.612+0000 7fe21d5d8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614790da710 0x5614790eb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:03.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.612+0000 7fe21d5d8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614790db0d0 0x5614790e9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:03.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.612+0000 7fe21d5d8b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5614790e05b0 0x5614790ea090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:03.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.612+0000 7fe21d5d8b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614790ea660 con 0x5614790e05b0 2023-09-13T03:18:03.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.612+0000 7fe21d5d8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614790ea7a0 con 0x5614790da710 2023-09-13T03:18:03.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.612+0000 7fe21d5d8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614790ea960 con 0x5614790db0d0 2023-09-13T03:18:03.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.612+0000 7fe21d5d8b80 10 client.0 _ll_register_callbacks cb 0x7fe1f4003ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:18:03.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.612+0000 7fe203fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5614790e05b0 0x5614790ea090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:03.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.612+0000 7fe202ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614790db0d0 0x5614790e9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:03.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.612+0000 7fe2037fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614790da710 0x5614790eb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:03.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.612+0000 7fe202ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614790db0d0 0x5614790e9b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:50742/0 (socket says 172.21.15.64:50742) 2023-09-13T03:18:03.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.612+0000 7fe202ffd700 1 -- 172.21.15.64:0/2915011910 learned_addr learned my addr 172.21.15.64:0/2915011910 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:18:03.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.612+0000 7fe203fff700 1 -- 172.21.15.64:0/2915011910 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614790db0d0 msgr2=0x5614790e9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:18:03.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.612+0000 7fe203fff700 1 --2- 172.21.15.64:0/2915011910 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614790db0d0 0x5614790e9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:03.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.612+0000 7fe203fff700 1 -- 172.21.15.64:0/2915011910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614790da710 msgr2=0x5614790eb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:18:03.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.612+0000 7fe203fff700 1 --2- 172.21.15.64:0/2915011910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614790da710 0x5614790eb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:03.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.612+0000 7fe203fff700 1 -- 172.21.15.64:0/2915011910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561478e54d80 con 0x5614790e05b0 2023-09-13T03:18:03.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.612+0000 7fe203fff700 1 --2- 172.21.15.64:0/2915011910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5614790e05b0 0x5614790ea090 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc009ed0 tx=0x7fe1fc00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eccf16b1ec7d6a7f server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:18:03.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.612+0000 7fe1e2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:18:03.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.613+0000 7fe1e2ffd700 1 -- 172.21.15.64:0/2915011910 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc030650 con 0x5614790e05b0 2023-09-13T03:18:03.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.613+0000 7fe1e2ffd700 1 -- 172.21.15.64:0/2915011910 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe1fc004dc0 con 0x5614790e05b0 2023-09-13T03:18:03.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.613+0000 7fe21d5d8b80 1 -- 172.21.15.64:0/2915011910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe1fc00f040 con 0x5614790e05b0 2023-09-13T03:18:03.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.613+0000 7fe1e2ffd700 1 -- 172.21.15.64:0/2915011910 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1fc0307b0 con 0x5614790e05b0 2023-09-13T03:18:03.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.613+0000 7fe21d5d8b80 10 client.16485 fetch_fsmap learned FSMap version 44 2023-09-13T03:18:03.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.613+0000 7fe21d5d8b80 1 -- 172.21.15.64:0/2915011910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x561478e53250 con 0x5614790e05b0 2023-09-13T03:18:03.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.613+0000 7fe1e2ffd700 1 -- 172.21.15.64:0/2915011910 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1fc03e020 con 0x5614790e05b0 2023-09-13T03:18:03.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.613+0000 7fe1e2ffd700 1 -- 172.21.15.64:0/2915011910 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 44) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fe1fc02e070 con 0x5614790e05b0 2023-09-13T03:18:03.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.614+0000 7fe21d5d8b80 10 client.16485 fetch_fsmap finished waiting for FSMap version 44 2023-09-13T03:18:03.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.614+0000 7fe21d5d8b80 10 client.16485 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:18:03.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.614+0000 7fe21d5d8b80 20 client.16485 populate_metadata read hostname 'smithi064' 2023-09-13T03:18:03.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.614+0000 7fe21d5d8b80 1 --2- 172.21.15.64:0/2915011910 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5614791495f0 0x56147914ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:03.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.614+0000 7fe21d5d8b80 4 client.16485 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:18:03.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.614+0000 7fe21d5d8b80 1 -- 172.21.15.64:0/2915011910 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5614790218c0 con 0x5614791495f0 2023-09-13T03:18:03.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.614+0000 7fe2037fe700 1 --2- 172.21.15.64:0/2915011910 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5614791495f0 0x56147914ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:03.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.614+0000 7fe1e2ffd700 10 client.16485 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:18:03.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.614+0000 7fe2037fe700 1 --2- 172.21.15.64:0/2915011910 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5614791495f0 0x56147914ba10 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4749746cb1d07c95 server_cookie=352c2fcc7b3c3074 in_seq=0 out_seq=0 2023-09-13T03:18:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.638+0000 7fe1e2ffd700 1 -- 172.21.15.64:0/2915011910 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5614790218c0 con 0x5614791495f0 2023-09-13T03:18:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.638+0000 7fe1e2ffd700 10 client.16485 handle_command_reply: tid=0 2023-09-13T03:18:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.638+0000 7fe21d5d8b80 1 client.16485 shutdown 2023-09-13T03:18:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.638+0000 7fe21d5d8b80 1 -- 172.21.15.64:0/2915011910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5614790e05b0 msgr2=0x5614790ea090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.638+0000 7fe21d5d8b80 1 --2- 172.21.15.64:0/2915011910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5614790e05b0 0x5614790ea090 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc009ed0 tx=0x7fe1fc00ec20 comp rx=0 tx=0).stop 2023-09-13T03:18:03.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.638+0000 7fe21d5d8b80 1 -- 172.21.15.64:0/2915011910 shutdown_connections 2023-09-13T03:18:03.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.638+0000 7fe21d5d8b80 1 --2- 172.21.15.64:0/2915011910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614790da710 0x5614790eb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:03.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.638+0000 7fe21d5d8b80 1 --2- 172.21.15.64:0/2915011910 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5614791495f0 0x56147914ba10 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:03.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.638+0000 7fe21d5d8b80 1 --2- 172.21.15.64:0/2915011910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5614790e05b0 0x5614790ea090 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:03.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.638+0000 7fe21d5d8b80 1 --2- 172.21.15.64:0/2915011910 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614790db0d0 0x5614790e9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:03.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.638+0000 7fe1e2ffd700 0 client.16485 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:18:03.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.638+0000 7fe21d5d8b80 1 -- 172.21.15.64:0/2915011910 >> 172.21.15.64:0/2915011910 conn(0x56147904b030 msgr2=0x5614790d5b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:18:03.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.638+0000 7fe21d5d8b80 1 -- 172.21.15.64:0/2915011910 shutdown_connections 2023-09-13T03:18:03.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.638+0000 7fe21d5d8b80 1 -- 172.21.15.64:0/2915011910 wait complete. 2023-09-13T03:18:03.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.638+0000 7fe21d5d8b80 20 client.16485 trim_cache size 0 max 16384 2023-09-13T03:18:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:18:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:18:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:18:03.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:18:03.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.639+0000 7fe209bff700 1 -- 172.21.15.64:0/4211125728 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fe1ec0a07b0 msgr2=0x7fe1ec0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:03.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.639+0000 7fe209bff700 1 --2- 172.21.15.64:0/4211125728 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fe1ec0a07b0 0x7fe1ec0a2c70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4009990 tx=0x7fe1f4005fb0 comp rx=0 tx=0).stop 2023-09-13T03:18:03.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.639+0000 7fe209bff700 1 -- 172.21.15.64:0/4211125728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe204100f60 msgr2=0x7fe20419cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:03.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.639+0000 7fe209bff700 1 --2- 172.21.15.64:0/4211125728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe204100f60 0x7fe20419cb30 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe1f800d750 tx=0x7fe1f800da60 comp rx=0 tx=0).stop 2023-09-13T03:18:03.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.639+0000 7fe209bff700 1 -- 172.21.15.64:0/4211125728 shutdown_connections 2023-09-13T03:18:03.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.639+0000 7fe209bff700 1 --2- 172.21.15.64:0/4211125728 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fe1ec0a07b0 0x7fe1ec0a2c70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.639+0000 7fe209bff700 1 --2- 172.21.15.64:0/4211125728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2041005a0 0x7fe20419c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.639+0000 7fe209bff700 1 --2- 172.21.15.64:0/4211125728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe204106660 0x7fe204196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.639+0000 7fe209bff700 1 --2- 172.21.15.64:0/4211125728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe204100f60 0x7fe20419cb30 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.639+0000 7fe209bff700 1 -- 172.21.15.64:0/4211125728 >> 172.21.15.64:0/4211125728 conn(0x7fe2040fc050 msgr2=0x7fe2040fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:18:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.639+0000 7fe209bff700 1 -- 172.21.15.64:0/4211125728 shutdown_connections 2023-09-13T03:18:03.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:03.639+0000 7fe209bff700 1 -- 172.21.15.64:0/4211125728 wait complete. 2023-09-13T03:18:03.910 INFO:tasks.workunit.client.0.smithi064.stdout: 1 103370 2.09 MB/sec execute 128 sec latency 3436.728 ms 2023-09-13T03:18:04.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:18:04 smithi170 ceph-mon[134328]: pgmap v150: 97 pgs: 97 active+clean; 57 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 258 KiB/s wr, 49 op/s 2023-09-13T03:18:04.911 INFO:tasks.workunit.client.0.smithi064.stdout: 1 103370 2.07 MB/sec execute 129 sec latency 4436.877 ms 2023-09-13T03:18:04.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:18:04 smithi064 ceph-mon[97675]: pgmap v150: 97 pgs: 97 active+clean; 57 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 258 KiB/s wr, 49 op/s 2023-09-13T03:18:04.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:18:04 smithi142 ceph-mon[134082]: pgmap v150: 97 pgs: 97 active+clean; 57 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 258 KiB/s wr, 49 op/s 2023-09-13T03:18:05.911 INFO:tasks.workunit.client.0.smithi064.stdout: 1 103576 2.06 MB/sec execute 130 sec latency 4881.216 ms 2023-09-13T03:18:06.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:18:06 smithi170 ceph-mon[134328]: pgmap v151: 97 pgs: 97 active+clean; 58 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 545 KiB/s wr, 85 op/s 2023-09-13T03:18:06.911 INFO:tasks.workunit.client.0.smithi064.stdout: 1 103576 2.04 MB/sec execute 131 sec latency 1377.237 ms 2023-09-13T03:18:06.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:18:06 smithi064 ceph-mon[97675]: pgmap v151: 97 pgs: 97 active+clean; 58 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 545 KiB/s wr, 85 op/s 2023-09-13T03:18:06.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:18:06 smithi142 ceph-mon[134082]: pgmap v151: 97 pgs: 97 active+clean; 58 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 545 KiB/s wr, 85 op/s 2023-09-13T03:18:07.911 INFO:tasks.workunit.client.0.smithi064.stdout: 1 103576 2.03 MB/sec execute 132 sec latency 2377.374 ms 2023-09-13T03:18:08.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:18:08 smithi170 ceph-mon[134328]: pgmap v152: 97 pgs: 97 active+clean; 58 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 445 KiB/s wr, 67 op/s 2023-09-13T03:18:08.911 INFO:tasks.workunit.client.0.smithi064.stdout: 1 103632 2.01 MB/sec execute 133 sec latency 2990.910 ms 2023-09-13T03:18:08.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:18:08 smithi064 ceph-mon[97675]: pgmap v152: 97 pgs: 97 active+clean; 58 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 445 KiB/s wr, 67 op/s 2023-09-13T03:18:08.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:18:08 smithi142 ceph-mon[134082]: pgmap v152: 97 pgs: 97 active+clean; 58 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 445 KiB/s wr, 67 op/s 2023-09-13T03:18:09.911 INFO:tasks.workunit.client.0.smithi064.stdout: 1 103632 2.00 MB/sec execute 134 sec latency 1339.047 ms 2023-09-13T03:18:10.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:18:10 smithi170 ceph-mon[134328]: pgmap v153: 97 pgs: 97 active+clean; 58 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 445 KiB/s wr, 67 op/s 2023-09-13T03:18:10.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:18:10 smithi064 ceph-mon[97675]: pgmap v153: 97 pgs: 97 active+clean; 58 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 445 KiB/s wr, 67 op/s 2023-09-13T03:18:10.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:18:10 smithi142 ceph-mon[134082]: pgmap v153: 97 pgs: 97 active+clean; 58 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 445 KiB/s wr, 67 op/s 2023-09-13T03:18:10.911 INFO:tasks.workunit.client.0.smithi064.stdout: 1 103632 1.98 MB/sec execute 135 sec latency 2339.187 ms 2023-09-13T03:18:11.912 INFO:tasks.workunit.client.0.smithi064.stdout: 1 103632 1.97 MB/sec execute 136 sec latency 3339.346 ms 2023-09-13T03:18:12.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:18:12 smithi170 ceph-mon[134328]: pgmap v154: 97 pgs: 97 active+clean; 57 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 698 KiB/s wr, 88 op/s 2023-09-13T03:18:12.911 INFO:tasks.workunit.client.0.smithi064.stdout: 1 103632 1.95 MB/sec execute 137 sec latency 4339.470 ms 2023-09-13T03:18:12.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:18:12 smithi064 ceph-mon[97675]: pgmap v154: 97 pgs: 97 active+clean; 57 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 698 KiB/s wr, 88 op/s 2023-09-13T03:18:12.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:18:12 smithi142 ceph-mon[134082]: pgmap v154: 97 pgs: 97 active+clean; 57 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 698 KiB/s wr, 88 op/s 2023-09-13T03:18:13.912 INFO:tasks.workunit.client.0.smithi064.stdout: 1 103700 1.94 MB/sec execute 138 sec latency 4952.798 ms 2023-09-13T03:18:14.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:18:14 smithi170 ceph-mon[134328]: pgmap v155: 97 pgs: 97 active+clean; 57 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 539 KiB/s wr, 56 op/s 2023-09-13T03:18:14.912 INFO:tasks.workunit.client.0.smithi064.stdout: 1 103700 1.93 MB/sec execute 139 sec latency 1332.434 ms 2023-09-13T03:18:14.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:18:14 smithi064 ceph-mon[97675]: pgmap v155: 97 pgs: 97 active+clean; 57 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 539 KiB/s wr, 56 op/s 2023-09-13T03:18:14.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:18:14 smithi142 ceph-mon[134082]: pgmap v155: 97 pgs: 97 active+clean; 57 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 539 KiB/s wr, 56 op/s 2023-09-13T03:18:15.914 INFO:tasks.workunit.client.0.smithi064.stdout: 1 103700 1.91 MB/sec execute 140 sec latency 2332.549 ms 2023-09-13T03:18:16.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.800+0000 7f363bdff700 1 -- 172.21.15.64:0/4286358698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36341005b0 msgr2=0x7f3634100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:16.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.800+0000 7f363bdff700 1 --2- 172.21.15.64:0/4286358698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36341005b0 0x7f3634100a30 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f36240099c0 tx=0x7f3624009cd0 comp rx=0 tx=0).stop 2023-09-13T03:18:16.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.800+0000 7f363bdff700 1 -- 172.21.15.64:0/4286358698 shutdown_connections 2023-09-13T03:18:16.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.800+0000 7f363bdff700 1 --2- 172.21.15.64:0/4286358698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3634106670 0x7f3634106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:16.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.800+0000 7f363bdff700 1 --2- 172.21.15.64:0/4286358698 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3634100f70 0x7f363410d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:16.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.800+0000 7f363bdff700 1 --2- 172.21.15.64:0/4286358698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36341005b0 0x7f3634100a30 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:16.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.800+0000 7f363bdff700 1 -- 172.21.15.64:0/4286358698 >> 172.21.15.64:0/4286358698 conn(0x7f36340fc060 msgr2=0x7f36340fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:18:16.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.800+0000 7f363bdff700 1 -- 172.21.15.64:0/4286358698 shutdown_connections 2023-09-13T03:18:16.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.800+0000 7f363bdff700 1 -- 172.21.15.64:0/4286358698 wait complete. 2023-09-13T03:18:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.801+0000 7f363bdff700 1 Processor -- start 2023-09-13T03:18:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.801+0000 7f363bdff700 1 -- start start 2023-09-13T03:18:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.802+0000 7f363bdff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36341005b0 0x7f3634072ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.802+0000 7f363bdff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3634100f70 0x7f3634073020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.802+0000 7f363bdff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3634106670 0x7f36341a4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.802+0000 7f363bdff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36341a4d60 con 0x7f3634100f70 2023-09-13T03:18:16.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.802+0000 7f363bdff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36341a4ea0 con 0x7f36341005b0 2023-09-13T03:18:16.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.802+0000 7f363bdff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36341a5010 con 0x7f3634106670 2023-09-13T03:18:16.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.802+0000 7f363939a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3634100f70 0x7f3634073020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:16.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.802+0000 7f363939a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3634100f70 0x7f3634073020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44884/0 (socket says 172.21.15.64:44884) 2023-09-13T03:18:16.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.802+0000 7f363939a700 1 -- 172.21.15.64:0/1528349715 learned_addr learned my addr 172.21.15.64:0/1528349715 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:18:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.802+0000 7f3639b9b700 1 --2- 172.21.15.64:0/1528349715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36341005b0 0x7f3634072ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.802+0000 7f363a39c700 1 --2- 172.21.15.64:0/1528349715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3634106670 0x7f36341a4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.802+0000 7f363939a700 1 -- 172.21.15.64:0/1528349715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3634106670 msgr2=0x7f36341a4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.802+0000 7f363939a700 1 --2- 172.21.15.64:0/1528349715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3634106670 0x7f36341a4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.802+0000 7f363939a700 1 -- 172.21.15.64:0/1528349715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36341005b0 msgr2=0x7f3634072ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.802+0000 7f363939a700 1 --2- 172.21.15.64:0/1528349715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36341005b0 0x7f3634072ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.802+0000 7f363939a700 1 -- 172.21.15.64:0/1528349715 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36240095f0 con 0x7f3634100f70 2023-09-13T03:18:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.803+0000 7f363939a700 1 --2- 172.21.15.64:0/1528349715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3634100f70 0x7f3634073020 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f36240098e0 tx=0x7f362400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:18:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.803+0000 7f362affd700 1 -- 172.21.15.64:0/1528349715 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362401d070 con 0x7f3634100f70 2023-09-13T03:18:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.803+0000 7f362affd700 1 -- 172.21.15.64:0/1528349715 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f362400fb80 con 0x7f3634100f70 2023-09-13T03:18:16.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.803+0000 7f363bdff700 1 -- 172.21.15.64:0/1528349715 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36341a5290 con 0x7f3634100f70 2023-09-13T03:18:16.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.803+0000 7f362affd700 1 -- 172.21.15.64:0/1528349715 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36240173f0 con 0x7f3634100f70 2023-09-13T03:18:16.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.803+0000 7f363bdff700 1 -- 172.21.15.64:0/1528349715 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36341a5710 con 0x7f3634100f70 2023-09-13T03:18:16.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.806+0000 7f362affd700 1 -- 172.21.15.64:0/1528349715 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f362400fcf0 con 0x7f3634100f70 2023-09-13T03:18:16.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.808+0000 7f364f7aeb80 1 Processor -- start 2023-09-13T03:18:16.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.808+0000 7f362affd700 1 --2- 172.21.15.64:0/1528349715 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f36200a0830 0x7f36200a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:16.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.808+0000 7f362affd700 1 -- 172.21.15.64:0/1528349715 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f36240ccb40 con 0x7f3634100f70 2023-09-13T03:18:16.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.808+0000 7f364f7aeb80 1 -- start start 2023-09-13T03:18:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.808+0000 7f3639b9b700 1 --2- 172.21.15.64:0/1528349715 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f36200a0830 0x7f36200a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.808+0000 7f364f7aeb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55803c0d71a0 0x55803c0d7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.808+0000 7f364f7aeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55803c0d1300 0x55803c0d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.808+0000 7f364f7aeb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55803c0d1cc0 0x55803c0ddf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.808+0000 7f364f7aeb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55803c02a6d0 con 0x55803c0d71a0 2023-09-13T03:18:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.808+0000 7f364f7aeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55803c027ff0 con 0x55803c0d1300 2023-09-13T03:18:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.808+0000 7f364f7aeb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55803c0238e0 con 0x55803c0d1cc0 2023-09-13T03:18:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.809+0000 7f3639b9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55803c0d71a0 0x55803c0d7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.809+0000 7f363939a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55803c0d1300 0x55803c0d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.809+0000 7f3639b9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55803c0d71a0 0x55803c0d7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44894/0 (socket says 172.21.15.64:44894) 2023-09-13T03:18:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.809+0000 7f3639b9b700 1 -- 172.21.15.64:0/2822779652 learned_addr learned my addr 172.21.15.64:0/2822779652 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:18:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.809+0000 7f363a39c700 1 --2- 172.21.15.64:0/2822779652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55803c0d1cc0 0x55803c0ddf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.809+0000 7f363939a700 1 -- 172.21.15.64:0/2822779652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55803c0d1cc0 msgr2=0x55803c0ddf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:18:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.809+0000 7f363939a700 1 --2- 172.21.15.64:0/2822779652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55803c0d1cc0 0x55803c0ddf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.809+0000 7f363939a700 1 -- 172.21.15.64:0/2822779652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55803c0d71a0 msgr2=0x55803c0d7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:18:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.809+0000 7f363939a700 1 --2- 172.21.15.64:0/2822779652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55803c0d71a0 0x55803c0d7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.809+0000 7f363939a700 1 -- 172.21.15.64:0/2822779652 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55803be4fd80 con 0x55803c0d1300 2023-09-13T03:18:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.809+0000 7f3639b9b700 1 --2- 172.21.15.64:0/1528349715 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f36200a0830 0x7f36200a2cf0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3630013650 tx=0x7f3630013a10 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:18:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.810+0000 7f363939a700 1 --2- 172.21.15.64:0/2822779652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55803c0d1300 0x55803c0d1780 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f36240ca280 tx=0x7f3624099f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbaf8efd36ee2d8d server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:18:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.810+0000 7f361ffff700 1 -- 172.21.15.64:0/2822779652 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362409dea0 con 0x55803c0d1300 2023-09-13T03:18:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.810+0000 7f361ffff700 1 -- 172.21.15.64:0/2822779652 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36240a2050 con 0x55803c0d1300 2023-09-13T03:18:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.810+0000 7f361ffff700 1 -- 172.21.15.64:0/2822779652 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3624099560 con 0x55803c0d1300 2023-09-13T03:18:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.810+0000 7f364f7aeb80 1 -- 172.21.15.64:0/2822779652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55803c0d1300 msgr2=0x55803c0d1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.810+0000 7f364f7aeb80 1 --2- 172.21.15.64:0/2822779652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55803c0d1300 0x55803c0d1780 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f36240ca280 tx=0x7f3624099f70 comp rx=0 tx=0).stop 2023-09-13T03:18:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.811+0000 7f364f7aeb80 1 -- 172.21.15.64:0/2822779652 shutdown_connections 2023-09-13T03:18:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.811+0000 7f364f7aeb80 1 --2- 172.21.15.64:0/2822779652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55803c0d1cc0 0x55803c0ddf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.811+0000 7f364f7aeb80 1 --2- 172.21.15.64:0/2822779652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55803c0d71a0 0x55803c0d7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.811+0000 7f364f7aeb80 1 --2- 172.21.15.64:0/2822779652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55803c0d1300 0x55803c0d1780 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.811+0000 7f364f7aeb80 1 -- 172.21.15.64:0/2822779652 >> 172.21.15.64:0/2822779652 conn(0x55803c046030 msgr2=0x55803c0cc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:18:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.811+0000 7f364f7aeb80 1 -- 172.21.15.64:0/2822779652 shutdown_connections 2023-09-13T03:18:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.811+0000 7f364f7aeb80 1 -- 172.21.15.64:0/2822779652 wait complete. 2023-09-13T03:18:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.811+0000 7f364f7aeb80 1 -- start start 2023-09-13T03:18:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.811+0000 7f364f7aeb80 1 Processor -- start 2023-09-13T03:18:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.812+0000 7f364f7aeb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55803c0d1300 0x55803c0e0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.812+0000 7f364f7aeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55803c0d1cc0 0x55803c0e1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.812+0000 7f364f7aeb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55803c0d71a0 0x55803c0daed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.812+0000 7f364f7aeb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55803bfc1b10 con 0x55803c0d1300 2023-09-13T03:18:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.812+0000 7f364f7aeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55803bfc1810 con 0x55803c0d1cc0 2023-09-13T03:18:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.812+0000 7f364f7aeb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55803bea58c0 con 0x55803c0d71a0 2023-09-13T03:18:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.812+0000 7f364f7aeb80 10 client.0 _ll_register_callbacks cb 0x55803bf0ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:18:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.812+0000 7f3639b9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55803c0d1300 0x55803c0e0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.812+0000 7f363939a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55803c0d1cc0 0x55803c0e1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.812+0000 7f3639b9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55803c0d1300 0x55803c0e0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44910/0 (socket says 172.21.15.64:44910) 2023-09-13T03:18:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.812+0000 7f3639b9b700 1 -- 172.21.15.64:0/313117114 learned_addr learned my addr 172.21.15.64:0/313117114 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:18:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.812+0000 7f363a39c700 1 --2- 172.21.15.64:0/313117114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55803c0d71a0 0x55803c0daed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.813+0000 7f363939a700 1 -- 172.21.15.64:0/313117114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55803c0d71a0 msgr2=0x55803c0daed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:18:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.813+0000 7f363939a700 1 --2- 172.21.15.64:0/313117114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55803c0d71a0 0x55803c0daed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.813+0000 7f363939a700 1 -- 172.21.15.64:0/313117114 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55803c0d1300 msgr2=0x55803c0e0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:18:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.813+0000 7f363939a700 1 --2- 172.21.15.64:0/313117114 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55803c0d1300 0x55803c0e0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.813+0000 7f363939a700 1 -- 172.21.15.64:0/313117114 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36341a5710 con 0x55803c0d1cc0 2023-09-13T03:18:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.813+0000 7f3639b9b700 1 --2- 172.21.15.64:0/313117114 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55803c0d1300 0x55803c0e0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:18:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.813+0000 7f363939a700 1 --2- 172.21.15.64:0/313117114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55803c0d1cc0 0x55803c0e1110 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f362409a920 tx=0x7f362409ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f94bd37bdd2bc3c server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:18:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.813+0000 7f361d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:18:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.813+0000 7f361d7fa700 1 -- 172.21.15.64:0/313117114 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362409dea0 con 0x55803c0d1cc0 2023-09-13T03:18:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.813+0000 7f364f7aeb80 1 -- 172.21.15.64:0/313117114 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55803bf7b490 con 0x55803c0d1cc0 2023-09-13T03:18:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.813+0000 7f361d7fa700 1 -- 172.21.15.64:0/313117114 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f362409a4e0 con 0x55803c0d1cc0 2023-09-13T03:18:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.813+0000 7f361d7fa700 1 -- 172.21.15.64:0/313117114 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36240a1bc0 con 0x55803c0d1cc0 2023-09-13T03:18:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.813+0000 7f364f7aeb80 10 client.26537 fetch_fsmap learned FSMap version 44 2023-09-13T03:18:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.813+0000 7f364f7aeb80 1 -- 172.21.15.64:0/313117114 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55803be38c00 con 0x55803c0d1cc0 2023-09-13T03:18:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.814+0000 7f361d7fa700 1 -- 172.21.15.64:0/313117114 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55803bf7b490 con 0x55803c0d1cc0 2023-09-13T03:18:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.814+0000 7f361d7fa700 1 -- 172.21.15.64:0/313117114 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 44) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f3624098fd0 con 0x55803c0d1cc0 2023-09-13T03:18:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.814+0000 7f364f7aeb80 10 client.26537 fetch_fsmap finished waiting for FSMap version 44 2023-09-13T03:18:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.814+0000 7f364f7aeb80 10 client.26537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:18:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.814+0000 7f364f7aeb80 20 client.26537 populate_metadata read hostname 'smithi064' 2023-09-13T03:18:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.814+0000 7f364f7aeb80 1 --2- 172.21.15.64:0/313117114 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55803c0ead60 0x55803c0ed180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.814+0000 7f364f7aeb80 4 client.26537 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:18:16.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.814+0000 7f364f7aeb80 1 -- 172.21.15.64:0/313117114 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55803c038840 con 0x55803c0ead60 2023-09-13T03:18:16.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.814+0000 7f3639b9b700 1 --2- 172.21.15.64:0/313117114 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55803c0ead60 0x55803c0ed180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:16.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.815+0000 7f3639b9b700 1 --2- 172.21.15.64:0/313117114 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55803c0ead60 0x55803c0ed180 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d153e29e6f427af server_cookie=a6a68800e3be49c6 in_seq=0 out_seq=0 2023-09-13T03:18:16.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.815+0000 7f361d7fa700 10 client.26537 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:18:16.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.816+0000 7f361d7fa700 1 -- 172.21.15.64:0/313117114 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55803c038840 con 0x55803c0ead60 2023-09-13T03:18:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.816+0000 7f361d7fa700 10 client.26537 handle_command_reply: tid=0 2023-09-13T03:18:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.816+0000 7f364f7aeb80 1 client.26537 shutdown 2023-09-13T03:18:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.816+0000 7f364f7aeb80 1 -- 172.21.15.64:0/313117114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55803c0d1cc0 msgr2=0x55803c0e1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.816+0000 7f364f7aeb80 1 --2- 172.21.15.64:0/313117114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55803c0d1cc0 0x55803c0e1110 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f362409a920 tx=0x7f362409ef90 comp rx=0 tx=0).stop 2023-09-13T03:18:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.816+0000 7f364f7aeb80 1 -- 172.21.15.64:0/313117114 shutdown_connections 2023-09-13T03:18:16.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.816+0000 7f364f7aeb80 1 --2- 172.21.15.64:0/313117114 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55803c0ead60 0x55803c0ed180 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:16.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.816+0000 7f364f7aeb80 1 --2- 172.21.15.64:0/313117114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55803c0d71a0 0x55803c0daed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:16.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.816+0000 7f364f7aeb80 1 --2- 172.21.15.64:0/313117114 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55803c0d1300 0x55803c0e0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:16.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.816+0000 7f364f7aeb80 1 --2- 172.21.15.64:0/313117114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55803c0d1cc0 0x55803c0e1110 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:16.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.816+0000 7f361d7fa700 0 client.26537 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:18:16.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.816+0000 7f364f7aeb80 1 -- 172.21.15.64:0/313117114 >> 172.21.15.64:0/313117114 conn(0x55803c046030 msgr2=0x55803c0cb7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:18:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.816+0000 7f364f7aeb80 1 -- 172.21.15.64:0/313117114 shutdown_connections 2023-09-13T03:18:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.816+0000 7f364f7aeb80 1 -- 172.21.15.64:0/313117114 wait complete. 2023-09-13T03:18:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.817+0000 7f364f7aeb80 20 client.26537 trim_cache size 0 max 16384 2023-09-13T03:18:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.832+0000 7f364f7aeb80 1 Processor -- start 2023-09-13T03:18:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.832+0000 7f364f7aeb80 1 -- start start 2023-09-13T03:18:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.832+0000 7f364f7aeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55803c0d1300 0x55803c0d6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:16.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.832+0000 7f364f7aeb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55803c0d1cc0 0x55803c0d3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:16.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.832+0000 7f364f7aeb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55803c0d71a0 0x55803c0d40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:16.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.832+0000 7f364f7aeb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55803c0d7c60 con 0x55803c0d1cc0 2023-09-13T03:18:16.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.832+0000 7f364f7aeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55803c0cdb80 con 0x55803c0d1300 2023-09-13T03:18:16.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.832+0000 7f364f7aeb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55803c0cdd60 con 0x55803c0d71a0 2023-09-13T03:18:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.833+0000 7f363939a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55803c0d1cc0 0x55803c0d3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.833+0000 7f363939a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55803c0d1cc0 0x55803c0d3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44926/0 (socket says 172.21.15.64:44926) 2023-09-13T03:18:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.833+0000 7f363939a700 1 -- 172.21.15.64:0/1040625229 learned_addr learned my addr 172.21.15.64:0/1040625229 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:18:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.833+0000 7f3639b9b700 1 --2- 172.21.15.64:0/1040625229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55803c0d1300 0x55803c0d6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.833+0000 7f363a39c700 1 --2- 172.21.15.64:0/1040625229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55803c0d71a0 0x55803c0d40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:16.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.833+0000 7f363939a700 1 -- 172.21.15.64:0/1040625229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55803c0d71a0 msgr2=0x55803c0d40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:18:16.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.833+0000 7f363939a700 1 --2- 172.21.15.64:0/1040625229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55803c0d71a0 0x55803c0d40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:16.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.833+0000 7f363939a700 1 -- 172.21.15.64:0/1040625229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55803c0d1300 msgr2=0x55803c0d6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:18:16.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.833+0000 7f363939a700 1 --2- 172.21.15.64:0/1040625229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55803c0d1300 0x55803c0d6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:16.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.833+0000 7f363939a700 1 -- 172.21.15.64:0/1040625229 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55803be761f0 con 0x55803c0d1cc0 2023-09-13T03:18:16.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.833+0000 7f363939a700 1 --2- 172.21.15.64:0/1040625229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55803c0d1cc0 0x55803c0d3b60 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3624009990 tx=0x7f3624099f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9932b7473caff50 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:18:16.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.833+0000 7f3628bff700 1 -- 172.21.15.64:0/1040625229 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36240999d0 con 0x55803c0d1cc0 2023-09-13T03:18:16.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.833+0000 7f3628bff700 1 -- 172.21.15.64:0/1040625229 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f362409dea0 con 0x55803c0d1cc0 2023-09-13T03:18:16.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.834+0000 7f3628bff700 1 -- 172.21.15.64:0/1040625229 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3624098fd0 con 0x55803c0d1cc0 2023-09-13T03:18:16.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.834+0000 7f364f7aeb80 1 -- 172.21.15.64:0/1040625229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55803c0d1cc0 msgr2=0x55803c0d3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.834+0000 7f364f7aeb80 1 --2- 172.21.15.64:0/1040625229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55803c0d1cc0 0x55803c0d3b60 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3624009990 tx=0x7f3624099f70 comp rx=0 tx=0).stop 2023-09-13T03:18:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.834+0000 7f364f7aeb80 1 -- 172.21.15.64:0/1040625229 shutdown_connections 2023-09-13T03:18:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.834+0000 7f364f7aeb80 1 --2- 172.21.15.64:0/1040625229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55803c0d1300 0x55803c0d6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.834+0000 7f364f7aeb80 1 --2- 172.21.15.64:0/1040625229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55803c0d71a0 0x55803c0d40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.834+0000 7f364f7aeb80 1 --2- 172.21.15.64:0/1040625229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55803c0d1cc0 0x55803c0d3b60 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:16.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.834+0000 7f364f7aeb80 1 -- 172.21.15.64:0/1040625229 >> 172.21.15.64:0/1040625229 conn(0x55803c046030 msgr2=0x55803c0d6110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:18:16.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.834+0000 7f364f7aeb80 1 -- 172.21.15.64:0/1040625229 shutdown_connections 2023-09-13T03:18:16.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.834+0000 7f364f7aeb80 1 -- 172.21.15.64:0/1040625229 wait complete. 2023-09-13T03:18:16.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.835+0000 7f364f7aeb80 1 -- start start 2023-09-13T03:18:16.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.835+0000 7f364f7aeb80 1 Processor -- start 2023-09-13T03:18:16.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.835+0000 7f364f7aeb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55803c0d1300 0x55803c132ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:16.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.835+0000 7f364f7aeb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55803c0d1cc0 0x55803c133020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:16.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.835+0000 7f364f7aeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55803c0d71a0 0x55803c13db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:16.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.835+0000 7f364f7aeb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55803c133790 con 0x55803c0d1300 2023-09-13T03:18:16.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.835+0000 7f364f7aeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55803c133900 con 0x55803c0d71a0 2023-09-13T03:18:16.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.835+0000 7f364f7aeb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55803c13e0a0 con 0x55803c0d1cc0 2023-09-13T03:18:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.835+0000 7f364f7aeb80 10 client.0 _ll_register_callbacks cb 0x7f3630009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:18:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.836+0000 7f3639b9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55803c0d1300 0x55803c132ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.836+0000 7f3639b9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55803c0d1300 0x55803c132ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44932/0 (socket says 172.21.15.64:44932) 2023-09-13T03:18:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.836+0000 7f3639b9b700 1 -- 172.21.15.64:0/2477311333 learned_addr learned my addr 172.21.15.64:0/2477311333 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:18:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.836+0000 7f363939a700 1 --2- 172.21.15.64:0/2477311333 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55803c0d1cc0 0x55803c133020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.836+0000 7f363a39c700 1 --2- 172.21.15.64:0/2477311333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55803c0d71a0 0x55803c13db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.836+0000 7f3639b9b700 1 -- 172.21.15.64:0/2477311333 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55803c0d1cc0 msgr2=0x55803c133020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:18:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.836+0000 7f3639b9b700 1 --2- 172.21.15.64:0/2477311333 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55803c0d1cc0 0x55803c133020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.836+0000 7f3639b9b700 1 -- 172.21.15.64:0/2477311333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55803c0d71a0 msgr2=0x55803c13db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:18:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.836+0000 7f3639b9b700 1 --2- 172.21.15.64:0/2477311333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55803c0d71a0 0x55803c13db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.836+0000 7f3639b9b700 1 -- 172.21.15.64:0/2477311333 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55803be38c00 con 0x55803c0d1300 2023-09-13T03:18:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.836+0000 7f363a39c700 1 --2- 172.21.15.64:0/2477311333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55803c0d71a0 0x55803c13db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:18:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.836+0000 7f3639b9b700 1 --2- 172.21.15.64:0/2477311333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55803c0d1300 0x55803c132ae0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x55803c0ed9f0 tx=0x7f36300047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=858288b5e3b2b974 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:18:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.836+0000 7f361d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:18:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.836+0000 7f361d7fa700 1 -- 172.21.15.64:0/2477311333 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3630018040 con 0x55803c0d1300 2023-09-13T03:18:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.836+0000 7f361d7fa700 1 -- 172.21.15.64:0/2477311333 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f363002b4a0 con 0x55803c0d1300 2023-09-13T03:18:16.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.837+0000 7f364f7aeb80 1 -- 172.21.15.64:0/2477311333 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f36240a2050 con 0x55803c0d1300 2023-09-13T03:18:16.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.837+0000 7f361d7fa700 1 -- 172.21.15.64:0/2477311333 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f363002bc90 con 0x55803c0d1300 2023-09-13T03:18:16.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.837+0000 7f364f7aeb80 10 client.16503 fetch_fsmap learned FSMap version 44 2023-09-13T03:18:16.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.837+0000 7f364f7aeb80 1 -- 172.21.15.64:0/2477311333 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55803be4e250 con 0x55803c0d1300 2023-09-13T03:18:16.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.837+0000 7f361d7fa700 1 -- 172.21.15.64:0/2477311333 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36240a2050 con 0x55803c0d1300 2023-09-13T03:18:16.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.837+0000 7f361d7fa700 1 -- 172.21.15.64:0/2477311333 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 44) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f3630020540 con 0x55803c0d1300 2023-09-13T03:18:16.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.837+0000 7f364f7aeb80 10 client.16503 fetch_fsmap finished waiting for FSMap version 44 2023-09-13T03:18:16.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.837+0000 7f364f7aeb80 10 client.16503 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:18:16.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.837+0000 7f364f7aeb80 20 client.16503 populate_metadata read hostname 'smithi064' 2023-09-13T03:18:16.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.837+0000 7f364f7aeb80 1 --2- 172.21.15.64:0/2477311333 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55803c13f860 0x55803c141c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:16.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.837+0000 7f364f7aeb80 4 client.16503 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:18:16.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.837+0000 7f364f7aeb80 1 -- 172.21.15.64:0/2477311333 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55803c01c8c0 con 0x55803c13f860 2023-09-13T03:18:16.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.838+0000 7f363939a700 1 --2- 172.21.15.64:0/2477311333 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55803c13f860 0x55803c141c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:16.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.838+0000 7f363939a700 1 --2- 172.21.15.64:0/2477311333 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55803c13f860 0x55803c141c80 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1816a46e8ce8553 server_cookie=cfafdaef05e0fb9c in_seq=0 out_seq=0 2023-09-13T03:18:16.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.838+0000 7f361d7fa700 10 client.16503 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:18:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.861+0000 7f361d7fa700 1 -- 172.21.15.64:0/2477311333 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55803c01c8c0 con 0x55803c13f860 2023-09-13T03:18:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.861+0000 7f361d7fa700 10 client.16503 handle_command_reply: tid=0 2023-09-13T03:18:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.861+0000 7f364f7aeb80 1 client.16503 shutdown 2023-09-13T03:18:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.861+0000 7f364f7aeb80 1 -- 172.21.15.64:0/2477311333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55803c0d1300 msgr2=0x55803c132ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.861+0000 7f364f7aeb80 1 --2- 172.21.15.64:0/2477311333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55803c0d1300 0x55803c132ae0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x55803c0ed9f0 tx=0x7f36300047c0 comp rx=0 tx=0).stop 2023-09-13T03:18:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.861+0000 7f364f7aeb80 1 -- 172.21.15.64:0/2477311333 shutdown_connections 2023-09-13T03:18:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.861+0000 7f364f7aeb80 1 --2- 172.21.15.64:0/2477311333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55803c0d71a0 0x55803c13db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.861+0000 7f364f7aeb80 1 --2- 172.21.15.64:0/2477311333 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55803c13f860 0x55803c141c80 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.861+0000 7f364f7aeb80 1 --2- 172.21.15.64:0/2477311333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55803c0d1300 0x55803c132ae0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.861+0000 7f364f7aeb80 1 --2- 172.21.15.64:0/2477311333 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55803c0d1cc0 0x55803c133020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.861+0000 7f361d7fa700 0 client.16503 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:18:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.861+0000 7f364f7aeb80 1 -- 172.21.15.64:0/2477311333 >> 172.21.15.64:0/2477311333 conn(0x55803c046030 msgr2=0x55803c0caf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:18:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.861+0000 7f364f7aeb80 1 -- 172.21.15.64:0/2477311333 shutdown_connections 2023-09-13T03:18:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.861+0000 7f364f7aeb80 1 -- 172.21.15.64:0/2477311333 wait complete. 2023-09-13T03:18:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.862+0000 7f364f7aeb80 20 client.16503 trim_cache size 0 max 16384 2023-09-13T03:18:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:18:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:18:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:18:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:18:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.862+0000 7f363bdff700 1 -- 172.21.15.64:0/1528349715 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f36200a0830 msgr2=0x7f36200a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.862+0000 7f363bdff700 1 --2- 172.21.15.64:0/1528349715 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f36200a0830 0x7f36200a2cf0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3630013650 tx=0x7f3630013a10 comp rx=0 tx=0).stop 2023-09-13T03:18:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.862+0000 7f363bdff700 1 -- 172.21.15.64:0/1528349715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3634100f70 msgr2=0x7f3634073020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.862+0000 7f363bdff700 1 --2- 172.21.15.64:0/1528349715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3634100f70 0x7f3634073020 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f36240098e0 tx=0x7f362400f690 comp rx=0 tx=0).stop 2023-09-13T03:18:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.862+0000 7f363bdff700 1 -- 172.21.15.64:0/1528349715 shutdown_connections 2023-09-13T03:18:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.862+0000 7f363bdff700 1 --2- 172.21.15.64:0/1528349715 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f36200a0830 0x7f36200a2cf0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.862+0000 7f363bdff700 1 --2- 172.21.15.64:0/1528349715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36341005b0 0x7f3634072ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.862+0000 7f363bdff700 1 --2- 172.21.15.64:0/1528349715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3634106670 0x7f36341a4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.862+0000 7f363bdff700 1 --2- 172.21.15.64:0/1528349715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3634100f70 0x7f3634073020 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.862+0000 7f363bdff700 1 -- 172.21.15.64:0/1528349715 >> 172.21.15.64:0/1528349715 conn(0x7f36340fc060 msgr2=0x7f36340fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:18:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.863+0000 7f363bdff700 1 -- 172.21.15.64:0/1528349715 shutdown_connections 2023-09-13T03:18:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:16.863+0000 7f363bdff700 1 -- 172.21.15.64:0/1528349715 wait complete. 2023-09-13T03:18:16.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:18:16 smithi170 ceph-mon[134328]: pgmap v156: 97 pgs: 97 active+clean; 58 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 924 KiB/s wr, 76 op/s 2023-09-13T03:18:16.912 INFO:tasks.workunit.client.0.smithi064.stdout: 1 103700 1.90 MB/sec execute 141 sec latency 3332.678 ms 2023-09-13T03:18:16.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:18:16 smithi064 ceph-mon[97675]: pgmap v156: 97 pgs: 97 active+clean; 58 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 924 KiB/s wr, 76 op/s 2023-09-13T03:18:16.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:18:16 smithi142 ceph-mon[134082]: pgmap v156: 97 pgs: 97 active+clean; 58 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 924 KiB/s wr, 76 op/s 2023-09-13T03:18:17.912 INFO:tasks.workunit.client.0.smithi064.stdout: 1 103700 1.89 MB/sec execute 142 sec latency 4332.819 ms 2023-09-13T03:18:18.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:18:18 smithi170 ceph-mon[134328]: pgmap v157: 97 pgs: 97 active+clean; 58 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 637 KiB/s wr, 41 op/s 2023-09-13T03:18:18.912 INFO:tasks.workunit.client.0.smithi064.stdout: 1 104358 1.90 MB/sec execute 143 sec latency 4950.709 ms 2023-09-13T03:18:18.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:18:18 smithi064 ceph-mon[97675]: pgmap v157: 97 pgs: 97 active+clean; 58 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 637 KiB/s wr, 41 op/s 2023-09-13T03:18:18.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:18:18 smithi142 ceph-mon[134082]: pgmap v157: 97 pgs: 97 active+clean; 58 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 637 KiB/s wr, 41 op/s 2023-09-13T03:18:19.912 INFO:tasks.workunit.client.0.smithi064.stdout: 1 104602 1.89 MB/sec execute 144 sec latency 819.130 ms 2023-09-13T03:18:20.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:18:20 smithi170 ceph-mon[134328]: pgmap v158: 97 pgs: 97 active+clean; 58 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 637 KiB/s wr, 41 op/s 2023-09-13T03:18:20.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:18:20 smithi064 ceph-mon[97675]: pgmap v158: 97 pgs: 97 active+clean; 58 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 637 KiB/s wr, 41 op/s 2023-09-13T03:18:20.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:18:20 smithi142 ceph-mon[134082]: pgmap v158: 97 pgs: 97 active+clean; 58 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 637 KiB/s wr, 41 op/s 2023-09-13T03:18:20.912 INFO:tasks.workunit.client.0.smithi064.stdout: 1 105779 1.92 MB/sec execute 145 sec latency 1256.139 ms 2023-09-13T03:18:21.913 INFO:tasks.workunit.client.0.smithi064.stdout: 1 107046 1.96 MB/sec execute 146 sec latency 11.449 ms 2023-09-13T03:18:22.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:18:22 smithi170 ceph-mon[134328]: pgmap v159: 97 pgs: 97 active+clean; 61 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 1.0 MiB/s wr, 78 op/s 2023-09-13T03:18:22.912 INFO:tasks.workunit.client.0.smithi064.stdout: 1 107173 1.95 MB/sec execute 147 sec latency 919.594 ms 2023-09-13T03:18:22.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:18:22 smithi064 ceph-mon[97675]: pgmap v159: 97 pgs: 97 active+clean; 61 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 1.0 MiB/s wr, 78 op/s 2023-09-13T03:18:22.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:18:22 smithi142 ceph-mon[134082]: pgmap v159: 97 pgs: 97 active+clean; 61 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 1.0 MiB/s wr, 78 op/s 2023-09-13T03:18:23.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:18:23 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:18:23.912 INFO:tasks.workunit.client.0.smithi064.stdout: 1 107241 1.94 MB/sec execute 148 sec latency 1529.656 ms 2023-09-13T03:18:23.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:18:23 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:18:23.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:18:23 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:18:24.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:18:24 smithi170 ceph-mon[134328]: pgmap v160: 97 pgs: 97 active+clean; 61 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 786 KiB/s wr, 57 op/s 2023-09-13T03:18:24.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:18:24 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:18:24.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:18:24 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:18:24.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:18:24 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:18:24.913 INFO:tasks.workunit.client.0.smithi064.stdout: 1 107241 1.93 MB/sec execute 149 sec latency 1342.207 ms 2023-09-13T03:18:24.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:18:24 smithi064 ceph-mon[97675]: pgmap v160: 97 pgs: 97 active+clean; 61 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 786 KiB/s wr, 57 op/s 2023-09-13T03:18:24.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:18:24 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:18:24.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:18:24 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:18:24.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:18:24 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:18:24.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:18:24 smithi142 ceph-mon[134082]: pgmap v160: 97 pgs: 97 active+clean; 61 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 786 KiB/s wr, 57 op/s 2023-09-13T03:18:25.002 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:18:24 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:18:25.002 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:18:24 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:18:25.003 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:18:24 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:18:25.913 INFO:tasks.workunit.client.0.smithi064.stdout: 1 107241 1.91 MB/sec execute 150 sec latency 2342.321 ms 2023-09-13T03:18:26.913 INFO:tasks.workunit.client.0.smithi064.stdout: 1 107241 1.90 MB/sec execute 151 sec latency 3342.449 ms 2023-09-13T03:18:26.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:18:26 smithi064 ceph-mon[97675]: pgmap v161: 97 pgs: 97 active+clean; 63 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 997 KiB/s wr, 91 op/s 2023-09-13T03:18:26.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:18:26 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:18:26.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:18:26 smithi142 ceph-mon[134082]: pgmap v161: 97 pgs: 97 active+clean; 63 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 997 KiB/s wr, 91 op/s 2023-09-13T03:18:26.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:18:26 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:18:27.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:18:26 smithi170 ceph-mon[134328]: pgmap v161: 97 pgs: 97 active+clean; 63 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 997 KiB/s wr, 91 op/s 2023-09-13T03:18:27.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:18:26 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:18:27.913 INFO:tasks.workunit.client.0.smithi064.stdout: 1 107241 1.89 MB/sec execute 152 sec latency 4342.524 ms 2023-09-13T03:18:28.913 INFO:tasks.workunit.client.0.smithi064.stdout: 1 107853 1.91 MB/sec execute 153 sec latency 4954.225 ms 2023-09-13T03:18:28.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:18:28 smithi064 ceph-mon[97675]: pgmap v162: 97 pgs: 97 active+clean; 63 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 612 KiB/s wr, 70 op/s 2023-09-13T03:18:28.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:18:28 smithi142 ceph-mon[134082]: pgmap v162: 97 pgs: 97 active+clean; 63 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 612 KiB/s wr, 70 op/s 2023-09-13T03:18:29.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:18:28 smithi170 ceph-mon[134328]: pgmap v162: 97 pgs: 97 active+clean; 63 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 612 KiB/s wr, 70 op/s 2023-09-13T03:18:29.913 INFO:tasks.workunit.client.0.smithi064.stdout: 1 107853 1.89 MB/sec execute 154 sec latency 1016.060 ms 2023-09-13T03:18:29.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:18:29 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:18:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.025+0000 7efccd1ff700 1 -- 172.21.15.64:0/4159184026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcc8106660 msgr2=0x7efcc8106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.025+0000 7efccd1ff700 1 --2- 172.21.15.64:0/4159184026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcc8106660 0x7efcc8106a40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7efcb80099c0 tx=0x7efcb8009cd0 comp rx=0 tx=0).stop 2023-09-13T03:18:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.025+0000 7efccd1ff700 1 -- 172.21.15.64:0/4159184026 shutdown_connections 2023-09-13T03:18:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.025+0000 7efccd1ff700 1 --2- 172.21.15.64:0/4159184026 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efcc8100f60 0x7efcc810d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.025+0000 7efccd1ff700 1 --2- 172.21.15.64:0/4159184026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcc8106660 0x7efcc8106a40 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.025+0000 7efccd1ff700 1 --2- 172.21.15.64:0/4159184026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcc81005a0 0x7efcc8100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.025+0000 7efccd1ff700 1 -- 172.21.15.64:0/4159184026 >> 172.21.15.64:0/4159184026 conn(0x7efcc80fc050 msgr2=0x7efcc80fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:18:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.026+0000 7efccd1ff700 1 -- 172.21.15.64:0/4159184026 shutdown_connections 2023-09-13T03:18:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.026+0000 7efccd1ff700 1 -- 172.21.15.64:0/4159184026 wait complete. 2023-09-13T03:18:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.027+0000 7efccd1ff700 1 Processor -- start 2023-09-13T03:18:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.027+0000 7efccd1ff700 1 -- start start 2023-09-13T03:18:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.027+0000 7efccd1ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcc81005a0 0x7efcc819e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.027+0000 7efccd1ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcc8100f60 0x7efcc819ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.027+0000 7efccd1ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efcc8106660 0x7efcc8198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.027+0000 7efccd1ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc8198c90 con 0x7efcc81005a0 2023-09-13T03:18:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.027+0000 7efccd1ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc8198e00 con 0x7efcc8100f60 2023-09-13T03:18:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.027+0000 7efccd1ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc8198fc0 con 0x7efcc8106660 2023-09-13T03:18:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.027+0000 7efcc6d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcc81005a0 0x7efcc819e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.028+0000 7efcc6d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcc81005a0 0x7efcc819e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53044/0 (socket says 172.21.15.64:53044) 2023-09-13T03:18:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.028+0000 7efcc6d9d700 1 -- 172.21.15.64:0/1599721137 learned_addr learned my addr 172.21.15.64:0/1599721137 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:18:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.028+0000 7efcc659c700 1 --2- 172.21.15.64:0/1599721137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcc8100f60 0x7efcc819ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:30.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.028+0000 7efcc759e700 1 --2- 172.21.15.64:0/1599721137 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efcc8106660 0x7efcc8198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:30.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.028+0000 7efcc6d9d700 1 -- 172.21.15.64:0/1599721137 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efcc8106660 msgr2=0x7efcc8198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:30.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.028+0000 7efcc6d9d700 1 --2- 172.21.15.64:0/1599721137 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efcc8106660 0x7efcc8198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:30.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.028+0000 7efcc6d9d700 1 -- 172.21.15.64:0/1599721137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcc8100f60 msgr2=0x7efcc819ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:30.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.028+0000 7efcc6d9d700 1 --2- 172.21.15.64:0/1599721137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcc8100f60 0x7efcc819ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:30.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.028+0000 7efcc6d9d700 1 -- 172.21.15.64:0/1599721137 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb80095f0 con 0x7efcc81005a0 2023-09-13T03:18:30.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.028+0000 7efcc6d9d700 1 --2- 172.21.15.64:0/1599721137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcc81005a0 0x7efcc819e7c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7efcb800b580 tx=0x7efcb800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:18:30.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.029+0000 7efcaffff700 1 -- 172.21.15.64:0/1599721137 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb801d070 con 0x7efcc81005a0 2023-09-13T03:18:30.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.029+0000 7efcaffff700 1 -- 172.21.15.64:0/1599721137 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcb8005050 con 0x7efcc81005a0 2023-09-13T03:18:30.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.029+0000 7efcaffff700 1 -- 172.21.15.64:0/1599721137 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb8004a40 con 0x7efcc81005a0 2023-09-13T03:18:30.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.029+0000 7efccd1ff700 1 -- 172.21.15.64:0/1599721137 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcc8199240 con 0x7efcc81005a0 2023-09-13T03:18:30.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.029+0000 7efccd1ff700 1 -- 172.21.15.64:0/1599721137 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcc81a57b0 con 0x7efcc81005a0 2023-09-13T03:18:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.032+0000 7efcaffff700 1 -- 172.21.15.64:0/1599721137 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7efcb8005430 con 0x7efcc81005a0 2023-09-13T03:18:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.033+0000 7efce0b46b80 1 Processor -- start 2023-09-13T03:18:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.033+0000 7efcaffff700 1 --2- 172.21.15.64:0/1599721137 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7efcb00a07b0 0x7efcb00a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.033+0000 7efce0b46b80 1 -- start start 2023-09-13T03:18:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.034+0000 7efcaffff700 1 -- 172.21.15.64:0/1599721137 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7efcb80ccd90 con 0x7efcc81005a0 2023-09-13T03:18:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.034+0000 7efcc759e700 1 --2- 172.21.15.64:0/1599721137 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7efcb00a07b0 0x7efcb00a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.034+0000 7efce0b46b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f3f7a01a0 0x558f3f7a0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.034+0000 7efce0b46b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f3f79a300 0x558f3f79a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.034+0000 7efce0b46b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558f3f79acc0 0x558f3f7a6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.034+0000 7efce0b46b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f3f6f36d0 con 0x558f3f79a300 2023-09-13T03:18:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.034+0000 7efce0b46b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f3f6f0ff0 con 0x558f3f7a01a0 2023-09-13T03:18:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.034+0000 7efce0b46b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f3f6ec8e0 con 0x558f3f79acc0 2023-09-13T03:18:30.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.034+0000 7efcc6d9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f3f7a01a0 0x558f3f7a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:30.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.034+0000 7efcc6d9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f3f7a01a0 0x558f3f7a0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:45026/0 (socket says 172.21.15.64:45026) 2023-09-13T03:18:30.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.034+0000 7efcc659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f3f79a300 0x558f3f79a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:30.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.034+0000 7efcc6d9d700 1 -- 172.21.15.64:0/2334459656 learned_addr learned my addr 172.21.15.64:0/2334459656 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:18:30.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.034+0000 7efcc759e700 1 --2- 172.21.15.64:0/2334459656 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558f3f79acc0 0x558f3f7a6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:30.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.034+0000 7efcc6d9d700 1 -- 172.21.15.64:0/2334459656 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558f3f79acc0 msgr2=0x558f3f7a6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:18:30.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.034+0000 7efcc6d9d700 1 --2- 172.21.15.64:0/2334459656 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558f3f79acc0 0x558f3f7a6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:30.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.035+0000 7efcc6d9d700 1 -- 172.21.15.64:0/2334459656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f3f79a300 msgr2=0x558f3f79a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:18:30.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.035+0000 7efcc6d9d700 1 --2- 172.21.15.64:0/2334459656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f3f79a300 0x558f3f79a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:30.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.035+0000 7efcc6d9d700 1 -- 172.21.15.64:0/2334459656 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f3f518d80 con 0x558f3f7a01a0 2023-09-13T03:18:30.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.035+0000 7efcc759e700 1 --2- 172.21.15.64:0/1599721137 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7efcb00a07b0 0x7efcb00a2c70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7efcc819f4c0 tx=0x7efcc0009ba0 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:18:30.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.035+0000 7efcc6d9d700 1 --2- 172.21.15.64:0/2334459656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f3f7a01a0 0x558f3f7a0580 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7efcb8096d30 tx=0x7efcb809a9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=752b397c7ceb13eb server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:18:30.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.035+0000 7efcad7fa700 1 -- 172.21.15.64:0/2334459656 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb809e0f0 con 0x558f3f7a01a0 2023-09-13T03:18:30.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.035+0000 7efcad7fa700 1 -- 172.21.15.64:0/2334459656 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcb80a22a0 con 0x558f3f7a01a0 2023-09-13T03:18:30.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.035+0000 7efcad7fa700 1 -- 172.21.15.64:0/2334459656 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb8099a10 con 0x558f3f7a01a0 2023-09-13T03:18:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.036+0000 7efce0b46b80 1 -- 172.21.15.64:0/2334459656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f3f7a01a0 msgr2=0x558f3f7a0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.036+0000 7efce0b46b80 1 --2- 172.21.15.64:0/2334459656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f3f7a01a0 0x558f3f7a0580 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7efcb8096d30 tx=0x7efcb809a9c0 comp rx=0 tx=0).stop 2023-09-13T03:18:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.036+0000 7efce0b46b80 1 -- 172.21.15.64:0/2334459656 shutdown_connections 2023-09-13T03:18:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.036+0000 7efce0b46b80 1 --2- 172.21.15.64:0/2334459656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f3f7a01a0 0x558f3f7a0580 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.036+0000 7efce0b46b80 1 --2- 172.21.15.64:0/2334459656 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558f3f79acc0 0x558f3f7a6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.036+0000 7efce0b46b80 1 --2- 172.21.15.64:0/2334459656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f3f79a300 0x558f3f79a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.036+0000 7efce0b46b80 1 -- 172.21.15.64:0/2334459656 >> 172.21.15.64:0/2334459656 conn(0x558f3f70f030 msgr2=0x558f3f795560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:18:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.036+0000 7efce0b46b80 1 -- 172.21.15.64:0/2334459656 shutdown_connections 2023-09-13T03:18:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.036+0000 7efce0b46b80 1 -- 172.21.15.64:0/2334459656 wait complete. 2023-09-13T03:18:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.037+0000 7efce0b46b80 1 -- start start 2023-09-13T03:18:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.037+0000 7efce0b46b80 1 Processor -- start 2023-09-13T03:18:30.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.037+0000 7efce0b46b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558f3f79a300 0x558f3f7a9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:30.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.037+0000 7efce0b46b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f3f79acc0 0x558f3f7aa0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:30.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.037+0000 7efce0b46b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f3f7a01a0 0x558f3f7a3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.037+0000 7efce0b46b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f3f68ab10 con 0x558f3f79acc0 2023-09-13T03:18:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.037+0000 7efce0b46b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f3f68a810 con 0x558f3f7a01a0 2023-09-13T03:18:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.037+0000 7efce0b46b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f3f56e8c0 con 0x558f3f79a300 2023-09-13T03:18:30.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.037+0000 7efce0b46b80 10 client.0 _ll_register_callbacks cb 0x558f3f5d4a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:18:30.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.038+0000 7efcc659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f3f79acc0 0x558f3f7aa0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:30.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.038+0000 7efcc659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f3f79acc0 0x558f3f7aa0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53058/0 (socket says 172.21.15.64:53058) 2023-09-13T03:18:30.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.038+0000 7efcc6d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558f3f79a300 0x558f3f7a9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:30.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.038+0000 7efcc759e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f3f7a01a0 0x558f3f7a3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:30.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.038+0000 7efcc659c700 1 -- 172.21.15.64:0/3126569910 learned_addr learned my addr 172.21.15.64:0/3126569910 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:18:30.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.038+0000 7efcc659c700 1 -- 172.21.15.64:0/3126569910 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558f3f79a300 msgr2=0x558f3f7a9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:18:30.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.038+0000 7efcc659c700 1 --2- 172.21.15.64:0/3126569910 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558f3f79a300 0x558f3f7a9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:30.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.038+0000 7efcc659c700 1 -- 172.21.15.64:0/3126569910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f3f7a01a0 msgr2=0x558f3f7a3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:18:30.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.038+0000 7efcc659c700 1 --2- 172.21.15.64:0/3126569910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f3f7a01a0 0x558f3f7a3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:30.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.038+0000 7efcc659c700 1 -- 172.21.15.64:0/3126569910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcc81a57b0 con 0x558f3f79acc0 2023-09-13T03:18:30.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.038+0000 7efcc6d9d700 1 --2- 172.21.15.64:0/3126569910 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558f3f79a300 0x558f3f7a9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:18:30.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.038+0000 7efcc759e700 1 --2- 172.21.15.64:0/3126569910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f3f7a01a0 0x558f3f7a3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:18:30.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.038+0000 7efcc659c700 1 --2- 172.21.15.64:0/3126569910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f3f79acc0 0x558f3f7aa0c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7efcbc0126d0 tx=0x7efcbc0129e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27d35c472b51adac server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:18:30.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.038+0000 7efcaaff5700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:18:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.039+0000 7efcaaff5700 1 -- 172.21.15.64:0/3126569910 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcbc017070 con 0x558f3f79acc0 2023-09-13T03:18:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.039+0000 7efce0b46b80 1 -- 172.21.15.64:0/3126569910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558f3f644490 con 0x558f3f79acc0 2023-09-13T03:18:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.039+0000 7efcaaff5700 1 -- 172.21.15.64:0/3126569910 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7efcbc010430 con 0x558f3f79acc0 2023-09-13T03:18:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.039+0000 7efcaaff5700 1 -- 172.21.15.64:0/3126569910 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7efcbc0105f0 con 0x558f3f79acc0 2023-09-13T03:18:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.039+0000 7efce0b46b80 10 client.16515 fetch_fsmap learned FSMap version 46 2023-09-13T03:18:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.039+0000 7efce0b46b80 1 -- 172.21.15.64:0/3126569910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x558f3f501c00 con 0x558f3f79acc0 2023-09-13T03:18:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.039+0000 7efcaaff5700 1 -- 172.21.15.64:0/3126569910 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558f3f644490 con 0x558f3f79acc0 2023-09-13T03:18:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.039+0000 7efcaaff5700 1 -- 172.21.15.64:0/3126569910 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 46) v1 ==== 7906+0+0 (secure 0 0 0) 0x7efcbc01c780 con 0x558f3f79acc0 2023-09-13T03:18:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.039+0000 7efce0b46b80 10 client.16515 fetch_fsmap finished waiting for FSMap version 46 2023-09-13T03:18:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.039+0000 7efce0b46b80 10 client.16515 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:18:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.039+0000 7efce0b46b80 20 client.16515 populate_metadata read hostname 'smithi064' 2023-09-13T03:18:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.039+0000 7efce0b46b80 1 --2- 172.21.15.64:0/3126569910 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x558f3f7b4260 0x558f3f7b6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.040+0000 7efce0b46b80 4 client.16515 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:18:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.040+0000 7efce0b46b80 1 -- 172.21.15.64:0/3126569910 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558f3f701840 con 0x558f3f7b4260 2023-09-13T03:18:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.040+0000 7efcc6d9d700 1 --2- 172.21.15.64:0/3126569910 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x558f3f7b4260 0x558f3f7b6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:30.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.040+0000 7efcc6d9d700 1 --2- 172.21.15.64:0/3126569910 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x558f3f7b4260 0x558f3f7b6680 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af5ccc122d42f445 server_cookie=142bb34bb1620e50 in_seq=0 out_seq=0 2023-09-13T03:18:30.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.041+0000 7efcaaff5700 10 client.16515 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:18:30.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.041+0000 7efcaaff5700 1 -- 172.21.15.64:0/3126569910 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558f3f701840 con 0x558f3f7b4260 2023-09-13T03:18:30.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.041+0000 7efcaaff5700 10 client.16515 handle_command_reply: tid=0 2023-09-13T03:18:30.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.041+0000 7efce0b46b80 1 client.16515 shutdown 2023-09-13T03:18:30.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.042+0000 7efce0b46b80 1 -- 172.21.15.64:0/3126569910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f3f79acc0 msgr2=0x558f3f7aa0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:30.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.042+0000 7efce0b46b80 1 --2- 172.21.15.64:0/3126569910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f3f79acc0 0x558f3f7aa0c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7efcbc0126d0 tx=0x7efcbc0129e0 comp rx=0 tx=0).stop 2023-09-13T03:18:30.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.042+0000 7efce0b46b80 1 -- 172.21.15.64:0/3126569910 shutdown_connections 2023-09-13T03:18:30.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.042+0000 7efce0b46b80 1 --2- 172.21.15.64:0/3126569910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f3f7a01a0 0x558f3f7a3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:30.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.042+0000 7efce0b46b80 1 --2- 172.21.15.64:0/3126569910 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x558f3f7b4260 0x558f3f7b6680 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:30.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.042+0000 7efce0b46b80 1 --2- 172.21.15.64:0/3126569910 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558f3f79a300 0x558f3f7a9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:30.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.042+0000 7efce0b46b80 1 --2- 172.21.15.64:0/3126569910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f3f79acc0 0x558f3f7aa0c0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:30.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.042+0000 7efcaaff5700 0 client.16515 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:18:30.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.042+0000 7efce0b46b80 1 -- 172.21.15.64:0/3126569910 >> 172.21.15.64:0/3126569910 conn(0x558f3f70f030 msgr2=0x558f3f7947a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:18:30.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.042+0000 7efce0b46b80 1 -- 172.21.15.64:0/3126569910 shutdown_connections 2023-09-13T03:18:30.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.042+0000 7efce0b46b80 1 -- 172.21.15.64:0/3126569910 wait complete. 2023-09-13T03:18:30.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.042+0000 7efce0b46b80 20 client.16515 trim_cache size 0 max 16384 2023-09-13T03:18:30.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.057+0000 7efce0b46b80 1 Processor -- start 2023-09-13T03:18:30.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.057+0000 7efce0b46b80 1 -- start start 2023-09-13T03:18:30.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.058+0000 7efce0b46b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558f3f79a300 0x558f3f7a3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:30.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.058+0000 7efce0b46b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f3f79acc0 0x558f3f799a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:30.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.058+0000 7efce0b46b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f3f7a01a0 0x558f3f794f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:30.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.058+0000 7efce0b46b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f3f79e9e0 con 0x558f3f7a01a0 2023-09-13T03:18:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.058+0000 7efce0b46b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f3f79b2c0 con 0x558f3f79acc0 2023-09-13T03:18:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.058+0000 7efce0b46b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f3f7b09a0 con 0x558f3f79a300 2023-09-13T03:18:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.058+0000 7efcc759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f3f7a01a0 0x558f3f794f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.058+0000 7efcc759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f3f7a01a0 0x558f3f794f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53064/0 (socket says 172.21.15.64:53064) 2023-09-13T03:18:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.058+0000 7efcc759e700 1 -- 172.21.15.64:0/356489734 learned_addr learned my addr 172.21.15.64:0/356489734 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:18:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.058+0000 7efcc6d9d700 1 --2- 172.21.15.64:0/356489734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558f3f79a300 0x558f3f7a3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.058+0000 7efcc659c700 1 --2- 172.21.15.64:0/356489734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f3f79acc0 0x558f3f799a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.058+0000 7efcc759e700 1 -- 172.21.15.64:0/356489734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558f3f79a300 msgr2=0x558f3f7a3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:18:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.058+0000 7efcc759e700 1 --2- 172.21.15.64:0/356489734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558f3f79a300 0x558f3f7a3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.058+0000 7efcc759e700 1 -- 172.21.15.64:0/356489734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f3f79acc0 msgr2=0x558f3f799a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:18:30.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.058+0000 7efcc759e700 1 --2- 172.21.15.64:0/356489734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f3f79acc0 0x558f3f799a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:30.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.058+0000 7efcc759e700 1 -- 172.21.15.64:0/356489734 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f3f53f1f0 con 0x558f3f7a01a0 2023-09-13T03:18:30.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.058+0000 7efcc659c700 1 --2- 172.21.15.64:0/356489734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f3f79acc0 0x558f3f799a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:18:30.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.059+0000 7efcc759e700 1 --2- 172.21.15.64:0/356489734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f3f7a01a0 0x558f3f794f20 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7efcc0008860 tx=0x7efcc0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e297718a7ee46c19 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:18:30.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.059+0000 7efcadffb700 1 -- 172.21.15.64:0/356489734 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc0004db0 con 0x558f3f7a01a0 2023-09-13T03:18:30.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.059+0000 7efcadffb700 1 -- 172.21.15.64:0/356489734 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcc000f040 con 0x558f3f7a01a0 2023-09-13T03:18:30.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.059+0000 7efcadffb700 1 -- 172.21.15.64:0/356489734 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc0018670 con 0x558f3f7a01a0 2023-09-13T03:18:30.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.060+0000 7efce0b46b80 1 -- 172.21.15.64:0/356489734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f3f7a01a0 msgr2=0x558f3f794f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:30.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.060+0000 7efce0b46b80 1 --2- 172.21.15.64:0/356489734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f3f7a01a0 0x558f3f794f20 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7efcc0008860 tx=0x7efcc0015000 comp rx=0 tx=0).stop 2023-09-13T03:18:30.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.060+0000 7efce0b46b80 1 -- 172.21.15.64:0/356489734 shutdown_connections 2023-09-13T03:18:30.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.060+0000 7efce0b46b80 1 --2- 172.21.15.64:0/356489734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f3f7a01a0 0x558f3f794f20 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:30.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.060+0000 7efce0b46b80 1 --2- 172.21.15.64:0/356489734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558f3f79a300 0x558f3f7a3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:30.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.060+0000 7efce0b46b80 1 --2- 172.21.15.64:0/356489734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f3f79acc0 0x558f3f799a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:30.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.060+0000 7efce0b46b80 1 -- 172.21.15.64:0/356489734 >> 172.21.15.64:0/356489734 conn(0x558f3f70f030 msgr2=0x558f3f7992f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:18:30.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.060+0000 7efce0b46b80 1 -- 172.21.15.64:0/356489734 shutdown_connections 2023-09-13T03:18:30.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.060+0000 7efce0b46b80 1 -- 172.21.15.64:0/356489734 wait complete. 2023-09-13T03:18:30.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.060+0000 7efce0b46b80 1 -- start start 2023-09-13T03:18:30.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.060+0000 7efce0b46b80 1 Processor -- start 2023-09-13T03:18:30.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.061+0000 7efce0b46b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558f3f79a300 0x558f3f801be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:30.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.061+0000 7efce0b46b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f3f79acc0 0x558f3f802120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:30.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.061+0000 7efce0b46b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f3f7a01a0 0x558f3f7fc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:30.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.061+0000 7efce0b46b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f3f7fc600 con 0x558f3f7a01a0 2023-09-13T03:18:30.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.061+0000 7efce0b46b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f3f7fc770 con 0x558f3f79acc0 2023-09-13T03:18:30.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.061+0000 7efce0b46b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f3f7fc930 con 0x558f3f79a300 2023-09-13T03:18:30.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.061+0000 7efce0b46b80 10 client.0 _ll_register_callbacks cb 0x7efcbc009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:18:30.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.061+0000 7efcc659c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f3f79acc0 0x558f3f802120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:30.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.061+0000 7efcc659c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f3f79acc0 0x558f3f802120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:45050/0 (socket says 172.21.15.64:45050) 2023-09-13T03:18:30.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.061+0000 7efcc659c700 1 -- 172.21.15.64:0/2770764436 learned_addr learned my addr 172.21.15.64:0/2770764436 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:18:30.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.061+0000 7efcc759e700 1 --2- 172.21.15.64:0/2770764436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f3f7a01a0 0x558f3f7fc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:30.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.061+0000 7efcc6d9d700 1 --2- 172.21.15.64:0/2770764436 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558f3f79a300 0x558f3f801be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:30.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.062+0000 7efcc659c700 1 -- 172.21.15.64:0/2770764436 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558f3f79a300 msgr2=0x558f3f801be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:18:30.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.062+0000 7efcc659c700 1 --2- 172.21.15.64:0/2770764436 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558f3f79a300 0x558f3f801be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:30.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.062+0000 7efcc659c700 1 -- 172.21.15.64:0/2770764436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f3f7a01a0 msgr2=0x558f3f7fc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:18:30.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.062+0000 7efcc659c700 1 --2- 172.21.15.64:0/2770764436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f3f7a01a0 0x558f3f7fc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:30.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.062+0000 7efcc659c700 1 -- 172.21.15.64:0/2770764436 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558f3f501c00 con 0x558f3f79acc0 2023-09-13T03:18:30.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.062+0000 7efcc659c700 1 --2- 172.21.15.64:0/2770764436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f3f79acc0 0x558f3f802120 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7efcbc005a80 tx=0x7efcbc01b400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1da80acd101aa7fa server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:18:30.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.062+0000 7efcaaff5700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:18:30.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.062+0000 7efcaaff5700 1 -- 172.21.15.64:0/2770764436 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcbc017040 con 0x558f3f79acc0 2023-09-13T03:18:30.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.062+0000 7efce0b46b80 1 -- 172.21.15.64:0/2770764436 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efcb80a22a0 con 0x558f3f79acc0 2023-09-13T03:18:30.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.062+0000 7efcaaff5700 1 -- 172.21.15.64:0/2770764436 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7efcbc0052b0 con 0x558f3f79acc0 2023-09-13T03:18:30.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.063+0000 7efcaaff5700 1 -- 172.21.15.64:0/2770764436 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7efcbc01c940 con 0x558f3f79acc0 2023-09-13T03:18:30.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.063+0000 7efce0b46b80 10 client.26561 fetch_fsmap learned FSMap version 46 2023-09-13T03:18:30.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.063+0000 7efce0b46b80 1 -- 172.21.15.64:0/2770764436 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7efcc0008510 con 0x558f3f79acc0 2023-09-13T03:18:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.063+0000 7efcaaff5700 1 -- 172.21.15.64:0/2770764436 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcb80a22a0 con 0x558f3f79acc0 2023-09-13T03:18:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.063+0000 7efcaaff5700 1 -- 172.21.15.64:0/2770764436 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 46) v1 ==== 7906+0+0 (secure 0 0 0) 0x7efcbc035050 con 0x558f3f79acc0 2023-09-13T03:18:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.063+0000 7efce0b46b80 10 client.26561 fetch_fsmap finished waiting for FSMap version 46 2023-09-13T03:18:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.063+0000 7efce0b46b80 10 client.26561 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:18:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.063+0000 7efce0b46b80 20 client.26561 populate_metadata read hostname 'smithi064' 2023-09-13T03:18:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.063+0000 7efce0b46b80 1 --2- 172.21.15.64:0/2770764436 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x558f3f809bd0 0x558f3f80bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.063+0000 7efce0b46b80 4 client.26561 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:18:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.063+0000 7efce0b46b80 1 -- 172.21.15.64:0/2770764436 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558f3f6e58c0 con 0x558f3f809bd0 2023-09-13T03:18:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.063+0000 7efcc6d9d700 1 --2- 172.21.15.64:0/2770764436 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x558f3f809bd0 0x558f3f80bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.064+0000 7efcc6d9d700 1 --2- 172.21.15.64:0/2770764436 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x558f3f809bd0 0x558f3f80bff0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2c2f7c691298b25 server_cookie=606f9a067c1542e5 in_seq=0 out_seq=0 2023-09-13T03:18:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.064+0000 7efcaaff5700 10 client.26561 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:18:30.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.086+0000 7efcaaff5700 1 -- 172.21.15.64:0/2770764436 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558f3f6e58c0 con 0x558f3f809bd0 2023-09-13T03:18:30.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.086+0000 7efcaaff5700 10 client.26561 handle_command_reply: tid=0 2023-09-13T03:18:30.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.086+0000 7efce0b46b80 1 client.26561 shutdown 2023-09-13T03:18:30.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.086+0000 7efce0b46b80 1 -- 172.21.15.64:0/2770764436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f3f79acc0 msgr2=0x558f3f802120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:30.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.086+0000 7efce0b46b80 1 --2- 172.21.15.64:0/2770764436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f3f79acc0 0x558f3f802120 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7efcbc005a80 tx=0x7efcbc01b400 comp rx=0 tx=0).stop 2023-09-13T03:18:30.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.086+0000 7efce0b46b80 1 -- 172.21.15.64:0/2770764436 shutdown_connections 2023-09-13T03:18:30.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.086+0000 7efce0b46b80 1 --2- 172.21.15.64:0/2770764436 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x558f3f809bd0 0x558f3f80bff0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:30.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.086+0000 7efce0b46b80 1 --2- 172.21.15.64:0/2770764436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f3f7a01a0 0x558f3f7fc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:30.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.086+0000 7efcaaff5700 0 client.26561 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:18:30.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.086+0000 7efce0b46b80 1 --2- 172.21.15.64:0/2770764436 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558f3f79a300 0x558f3f801be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:30.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.086+0000 7efce0b46b80 1 --2- 172.21.15.64:0/2770764436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558f3f79acc0 0x558f3f802120 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:30.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.086+0000 7efce0b46b80 1 -- 172.21.15.64:0/2770764436 >> 172.21.15.64:0/2770764436 conn(0x558f3f70f030 msgr2=0x558f3f7a3db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:18:30.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.086+0000 7efce0b46b80 1 -- 172.21.15.64:0/2770764436 shutdown_connections 2023-09-13T03:18:30.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.087+0000 7efce0b46b80 1 -- 172.21.15.64:0/2770764436 wait complete. 2023-09-13T03:18:30.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.087+0000 7efce0b46b80 20 client.26561 trim_cache size 0 max 16384 2023-09-13T03:18:30.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:18:30.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:18:30.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:18:30.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:18:30.096 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.087+0000 7efccd1ff700 1 -- 172.21.15.64:0/1599721137 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7efcb00a07b0 msgr2=0x7efcb00a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:30.096 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.087+0000 7efccd1ff700 1 --2- 172.21.15.64:0/1599721137 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7efcb00a07b0 0x7efcb00a2c70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7efcc819f4c0 tx=0x7efcc0009ba0 comp rx=0 tx=0).stop 2023-09-13T03:18:30.097 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.087+0000 7efccd1ff700 1 -- 172.21.15.64:0/1599721137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcc81005a0 msgr2=0x7efcc819e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:30.097 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.087+0000 7efccd1ff700 1 --2- 172.21.15.64:0/1599721137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcc81005a0 0x7efcc819e7c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7efcb800b580 tx=0x7efcb800f690 comp rx=0 tx=0).stop 2023-09-13T03:18:30.097 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.088+0000 7efccd1ff700 1 -- 172.21.15.64:0/1599721137 shutdown_connections 2023-09-13T03:18:30.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.088+0000 7efccd1ff700 1 --2- 172.21.15.64:0/1599721137 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7efcb00a07b0 0x7efcb00a2c70 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:30.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.088+0000 7efccd1ff700 1 --2- 172.21.15.64:0/1599721137 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efcc8106660 0x7efcc8198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:30.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.088+0000 7efccd1ff700 1 --2- 172.21.15.64:0/1599721137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcc81005a0 0x7efcc819e7c0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.088+0000 7efccd1ff700 1 --2- 172.21.15.64:0/1599721137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcc8100f60 0x7efcc819ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.088+0000 7efccd1ff700 1 -- 172.21.15.64:0/1599721137 >> 172.21.15.64:0/1599721137 conn(0x7efcc80fc050 msgr2=0x7efcc80fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:18:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.088+0000 7efccd1ff700 1 -- 172.21.15.64:0/1599721137 shutdown_connections 2023-09-13T03:18:30.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:30.088+0000 7efccd1ff700 1 -- 172.21.15.64:0/1599721137 wait complete. 2023-09-13T03:18:30.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:18:29 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:18:30.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:18:29 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:18:30.913 INFO:tasks.workunit.client.0.smithi064.stdout: 1 108142 1.88 MB/sec execute 155 sec latency 1447.417 ms 2023-09-13T03:18:30.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:18:30 smithi142 ceph-mon[134082]: pgmap v163: 97 pgs: 97 active+clean; 63 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 612 KiB/s wr, 70 op/s 2023-09-13T03:18:31.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:18:30 smithi170 ceph-mon[134328]: pgmap v163: 97 pgs: 97 active+clean; 63 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 612 KiB/s wr, 70 op/s 2023-09-13T03:18:31.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:18:30 smithi064 ceph-mon[97675]: pgmap v163: 97 pgs: 97 active+clean; 63 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 612 KiB/s wr, 70 op/s 2023-09-13T03:18:31.914 INFO:tasks.workunit.client.0.smithi064.stdout: 1 108142 1.87 MB/sec execute 156 sec latency 1337.524 ms 2023-09-13T03:18:32.914 INFO:tasks.workunit.client.0.smithi064.stdout: 1 108142 1.86 MB/sec execute 157 sec latency 2337.663 ms 2023-09-13T03:18:32.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:18:32 smithi142 ceph-mon[134082]: pgmap v164: 97 pgs: 97 active+clean; 60 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 726 KiB/s wr, 99 op/s 2023-09-13T03:18:33.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:18:32 smithi170 ceph-mon[134328]: pgmap v164: 97 pgs: 97 active+clean; 60 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 726 KiB/s wr, 99 op/s 2023-09-13T03:18:33.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:18:32 smithi064 ceph-mon[97675]: pgmap v164: 97 pgs: 97 active+clean; 60 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 726 KiB/s wr, 99 op/s 2023-09-13T03:18:33.915 INFO:tasks.workunit.client.0.smithi064.stdout: 1 108142 1.85 MB/sec execute 158 sec latency 3337.791 ms 2023-09-13T03:18:34.916 INFO:tasks.workunit.client.0.smithi064.stdout: 1 108142 1.83 MB/sec execute 159 sec latency 4337.902 ms 2023-09-13T03:18:34.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:18:34 smithi142 ceph-mon[134082]: pgmap v165: 97 pgs: 97 active+clean; 60 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 325 KiB/s wr, 62 op/s 2023-09-13T03:18:35.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:18:34 smithi170 ceph-mon[134328]: pgmap v165: 97 pgs: 97 active+clean; 60 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 325 KiB/s wr, 62 op/s 2023-09-13T03:18:35.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:18:34 smithi064 ceph-mon[97675]: pgmap v165: 97 pgs: 97 active+clean; 60 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 325 KiB/s wr, 62 op/s 2023-09-13T03:18:35.914 INFO:tasks.workunit.client.0.smithi064.stdout: 1 109367 1.87 MB/sec execute 160 sec latency 4773.599 ms 2023-09-13T03:18:36.914 INFO:tasks.workunit.client.0.smithi064.stdout: 1 110596 1.90 MB/sec execute 161 sec latency 13.721 ms 2023-09-13T03:18:36.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:18:36 smithi142 ceph-mon[134082]: pgmap v166: 97 pgs: 97 active+clean; 61 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 365 KiB/s wr, 76 op/s 2023-09-13T03:18:37.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:18:36 smithi170 ceph-mon[134328]: pgmap v166: 97 pgs: 97 active+clean; 61 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 365 KiB/s wr, 76 op/s 2023-09-13T03:18:37.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:18:36 smithi064 ceph-mon[97675]: pgmap v166: 97 pgs: 97 active+clean; 61 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 365 KiB/s wr, 76 op/s 2023-09-13T03:18:37.914 INFO:tasks.workunit.client.0.smithi064.stdout: 1 110791 1.89 MB/sec execute 162 sec latency 872.787 ms 2023-09-13T03:18:38.914 INFO:tasks.workunit.client.0.smithi064.stdout: 1 111453 1.91 MB/sec execute 163 sec latency 1482.457 ms 2023-09-13T03:18:38.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:18:38 smithi142 ceph-mon[134082]: pgmap v167: 97 pgs: 97 active+clean; 61 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 154 KiB/s wr, 43 op/s 2023-09-13T03:18:39.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:18:38 smithi170 ceph-mon[134328]: pgmap v167: 97 pgs: 97 active+clean; 61 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 154 KiB/s wr, 43 op/s 2023-09-13T03:18:39.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:18:38 smithi064 ceph-mon[97675]: pgmap v167: 97 pgs: 97 active+clean; 61 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 154 KiB/s wr, 43 op/s 2023-09-13T03:18:39.914 INFO:tasks.workunit.client.0.smithi064.stdout: 1 111692 1.90 MB/sec execute 164 sec latency 825.476 ms 2023-09-13T03:18:40.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:18:40 smithi170 ceph-mon[134328]: pgmap v168: 97 pgs: 97 active+clean; 61 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 154 KiB/s wr, 43 op/s 2023-09-13T03:18:40.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:18:40 smithi064 ceph-mon[97675]: pgmap v168: 97 pgs: 97 active+clean; 61 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 154 KiB/s wr, 43 op/s 2023-09-13T03:18:40.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:18:40 smithi142 ceph-mon[134082]: pgmap v168: 97 pgs: 97 active+clean; 61 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 154 KiB/s wr, 43 op/s 2023-09-13T03:18:40.914 INFO:tasks.workunit.client.0.smithi064.stdout: 1 112927 1.93 MB/sec execute 165 sec latency 1258.566 ms 2023-09-13T03:18:41.915 INFO:tasks.workunit.client.0.smithi064.stdout: 1 114210 1.96 MB/sec execute 166 sec latency 20.871 ms 2023-09-13T03:18:42.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:18:42 smithi170 ceph-mon[134328]: pgmap v169: 97 pgs: 97 active+clean; 63 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 463 KiB/s wr, 91 op/s 2023-09-13T03:18:42.915 INFO:tasks.workunit.client.0.smithi064.stdout: 1 114210 1.94 MB/sec execute 167 sec latency 1021.004 ms 2023-09-13T03:18:42.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:18:42 smithi064 ceph-mon[97675]: pgmap v169: 97 pgs: 97 active+clean; 63 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 463 KiB/s wr, 91 op/s 2023-09-13T03:18:42.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:18:42 smithi142 ceph-mon[134082]: pgmap v169: 97 pgs: 97 active+clean; 63 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 463 KiB/s wr, 91 op/s 2023-09-13T03:18:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.248+0000 7ff2c3dff700 1 -- 172.21.15.64:0/3946214153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2bc106670 msgr2=0x7ff2bc106a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.248+0000 7ff2c3dff700 1 --2- 172.21.15.64:0/3946214153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2bc106670 0x7ff2bc106a50 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff2b80099c0 tx=0x7ff2b8009cd0 comp rx=0 tx=0).stop 2023-09-13T03:18:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.248+0000 7ff2c3dff700 1 -- 172.21.15.64:0/3946214153 shutdown_connections 2023-09-13T03:18:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.248+0000 7ff2c3dff700 1 --2- 172.21.15.64:0/3946214153 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2bc100f70 0x7ff2bc10d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.248+0000 7ff2c3dff700 1 --2- 172.21.15.64:0/3946214153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2bc106670 0x7ff2bc106a50 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.248+0000 7ff2c3dff700 1 --2- 172.21.15.64:0/3946214153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2bc1005b0 0x7ff2bc100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.248+0000 7ff2c3dff700 1 -- 172.21.15.64:0/3946214153 >> 172.21.15.64:0/3946214153 conn(0x7ff2bc0fc060 msgr2=0x7ff2bc0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:18:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.248+0000 7ff2c3dff700 1 -- 172.21.15.64:0/3946214153 shutdown_connections 2023-09-13T03:18:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.248+0000 7ff2c3dff700 1 -- 172.21.15.64:0/3946214153 wait complete. 2023-09-13T03:18:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.249+0000 7ff2c3dff700 1 Processor -- start 2023-09-13T03:18:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.249+0000 7ff2c3dff700 1 -- start start 2023-09-13T03:18:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.250+0000 7ff2c3dff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2bc1005b0 0x7ff2bc19c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.250+0000 7ff2c3dff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2bc100f70 0x7ff2bc19cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.250+0000 7ff2c3dff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2bc106670 0x7ff2bc1964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.250+0000 7ff2c3dff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2bc196b50 con 0x7ff2bc106670 2023-09-13T03:18:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.250+0000 7ff2c3dff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2bc196cc0 con 0x7ff2bc100f70 2023-09-13T03:18:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.250+0000 7ff2c3dff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2bc196e80 con 0x7ff2bc1005b0 2023-09-13T03:18:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.250+0000 7ff2c239c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2bc106670 0x7ff2bc1964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.250+0000 7ff2c1b9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2bc1005b0 0x7ff2bc19c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.250+0000 7ff2c1b9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2bc1005b0 0x7ff2bc19c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:57344/0 (socket says 172.21.15.64:57344) 2023-09-13T03:18:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.250+0000 7ff2c239c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2bc106670 0x7ff2bc1964e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58692/0 (socket says 172.21.15.64:58692) 2023-09-13T03:18:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.250+0000 7ff2c1b9b700 1 -- 172.21.15.64:0/2135743824 learned_addr learned my addr 172.21.15.64:0/2135743824 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:18:43.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.250+0000 7ff2c139a700 1 --2- 172.21.15.64:0/2135743824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2bc100f70 0x7ff2bc19cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:43.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.250+0000 7ff2c239c700 1 -- 172.21.15.64:0/2135743824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2bc1005b0 msgr2=0x7ff2bc19c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:43.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.250+0000 7ff2c239c700 1 --2- 172.21.15.64:0/2135743824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2bc1005b0 0x7ff2bc19c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:43.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.250+0000 7ff2c239c700 1 -- 172.21.15.64:0/2135743824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2bc100f70 msgr2=0x7ff2bc19cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:43.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.250+0000 7ff2c239c700 1 --2- 172.21.15.64:0/2135743824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2bc100f70 0x7ff2bc19cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.250+0000 7ff2c239c700 1 -- 172.21.15.64:0/2135743824 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b80095f0 con 0x7ff2bc106670 2023-09-13T03:18:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.250+0000 7ff2c1b9b700 1 --2- 172.21.15.64:0/2135743824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2bc1005b0 0x7ff2bc19c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:18:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.251+0000 7ff2c239c700 1 --2- 172.21.15.64:0/2135743824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2bc106670 0x7ff2bc1964e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4007f10 tx=0x7ff2b400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:18:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.251+0000 7ff2b2ffd700 1 -- 172.21.15.64:0/2135743824 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b4004db0 con 0x7ff2bc106670 2023-09-13T03:18:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.251+0000 7ff2b2ffd700 1 -- 172.21.15.64:0/2135743824 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2b400f040 con 0x7ff2bc106670 2023-09-13T03:18:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.251+0000 7ff2b2ffd700 1 -- 172.21.15.64:0/2135743824 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b4005990 con 0x7ff2bc106670 2023-09-13T03:18:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.251+0000 7ff2c3dff700 1 -- 172.21.15.64:0/2135743824 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2bc197160 con 0x7ff2bc106670 2023-09-13T03:18:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.251+0000 7ff2c3dff700 1 -- 172.21.15.64:0/2135743824 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2bc1a3670 con 0x7ff2bc106670 2023-09-13T03:18:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.254+0000 7ff2b2ffd700 1 -- 172.21.15.64:0/2135743824 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7ff2b400db20 con 0x7ff2bc106670 2023-09-13T03:18:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.256+0000 7ff2d7764b80 1 Processor -- start 2023-09-13T03:18:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.257+0000 7ff2d7764b80 1 -- start start 2023-09-13T03:18:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.257+0000 7ff2d7764b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55936a3ae1a0 0x55936a3ae580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.257+0000 7ff2d7764b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55936a3a8300 0x55936a3a8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.257+0000 7ff2d7764b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55936a3a8cc0 0x55936a3b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.257+0000 7ff2d7764b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55936a3016d0 con 0x55936a3a8cc0 2023-09-13T03:18:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.257+0000 7ff2d7764b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55936a2feff0 con 0x55936a3a8300 2023-09-13T03:18:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.257+0000 7ff2d7764b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55936a2fa8e0 con 0x55936a3ae1a0 2023-09-13T03:18:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.257+0000 7ff2b2ffd700 1 --2- 172.21.15.64:0/2135743824 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7ff2a80a0830 0x7ff2a80a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.258+0000 7ff2b2ffd700 1 -- 172.21.15.64:0/2135743824 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7ff2b40cb750 con 0x7ff2bc106670 2023-09-13T03:18:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.258+0000 7ff2c139a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55936a3a8cc0 0x55936a3b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.258+0000 7ff2c239c700 1 --2- 172.21.15.64:0/2135743824 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7ff2a80a0830 0x7ff2a80a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.258+0000 7ff2c139a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55936a3a8cc0 0x55936a3b4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58700/0 (socket says 172.21.15.64:58700) 2023-09-13T03:18:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.258+0000 7ff2c139a700 1 -- 172.21.15.64:0/3584406567 learned_addr learned my addr 172.21.15.64:0/3584406567 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:18:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.258+0000 7ff2c1b9b700 1 --2- 172.21.15.64:0/3584406567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55936a3a8300 0x55936a3a8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.258+0000 7ff2c139a700 1 --2- 172.21.15.64:0/3584406567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55936a3ae1a0 0x55936a3ae580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.258+0000 7ff2c1b9b700 1 -- 172.21.15.64:0/3584406567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55936a3ae1a0 msgr2=0x55936a3ae580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:18:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.258+0000 7ff2c1b9b700 1 --2- 172.21.15.64:0/3584406567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55936a3ae1a0 0x55936a3ae580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.258+0000 7ff2c1b9b700 1 -- 172.21.15.64:0/3584406567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55936a3a8cc0 msgr2=0x55936a3b4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:18:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.258+0000 7ff2c1b9b700 1 --2- 172.21.15.64:0/3584406567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55936a3a8cc0 0x55936a3b4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.258+0000 7ff2c1b9b700 1 -- 172.21.15.64:0/3584406567 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55936a126d80 con 0x55936a3a8300 2023-09-13T03:18:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.258+0000 7ff2c139a700 1 --2- 172.21.15.64:0/3584406567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55936a3a8cc0 0x55936a3b4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:18:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.258+0000 7ff2c239c700 1 --2- 172.21.15.64:0/2135743824 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7ff2a80a0830 0x7ff2a80a2cf0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff2b40a0730 tx=0x7ff2b4009f90 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:18:43.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.258+0000 7ff2c1b9b700 1 --2- 172.21.15.64:0/3584406567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55936a3a8300 0x55936a3a8780 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8000c00 tx=0x7ff2b8005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75b806b532a291f6 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:18:43.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.259+0000 7ff2a7fff700 1 -- 172.21.15.64:0/3584406567 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b800bdc0 con 0x55936a3a8300 2023-09-13T03:18:43.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.259+0000 7ff2a7fff700 1 -- 172.21.15.64:0/3584406567 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2b801e040 con 0x55936a3a8300 2023-09-13T03:18:43.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.259+0000 7ff2a7fff700 1 -- 172.21.15.64:0/3584406567 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b80055b0 con 0x55936a3a8300 2023-09-13T03:18:43.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.260+0000 7ff2d7764b80 1 -- 172.21.15.64:0/3584406567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55936a3a8300 msgr2=0x55936a3a8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:43.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.260+0000 7ff2d7764b80 1 --2- 172.21.15.64:0/3584406567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55936a3a8300 0x55936a3a8780 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8000c00 tx=0x7ff2b8005fb0 comp rx=0 tx=0).stop 2023-09-13T03:18:43.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.260+0000 7ff2d7764b80 1 -- 172.21.15.64:0/3584406567 shutdown_connections 2023-09-13T03:18:43.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.260+0000 7ff2d7764b80 1 --2- 172.21.15.64:0/3584406567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55936a3a8cc0 0x55936a3b4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:43.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.260+0000 7ff2d7764b80 1 --2- 172.21.15.64:0/3584406567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55936a3ae1a0 0x55936a3ae580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:43.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.260+0000 7ff2d7764b80 1 --2- 172.21.15.64:0/3584406567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55936a3a8300 0x55936a3a8780 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.260+0000 7ff2d7764b80 1 -- 172.21.15.64:0/3584406567 >> 172.21.15.64:0/3584406567 conn(0x55936a31d030 msgr2=0x55936a3a3560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:18:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.260+0000 7ff2d7764b80 1 -- 172.21.15.64:0/3584406567 shutdown_connections 2023-09-13T03:18:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.260+0000 7ff2d7764b80 1 -- 172.21.15.64:0/3584406567 wait complete. 2023-09-13T03:18:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.260+0000 7ff2d7764b80 1 -- start start 2023-09-13T03:18:43.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.261+0000 7ff2d7764b80 1 Processor -- start 2023-09-13T03:18:43.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.261+0000 7ff2d7764b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55936a3a8300 0x55936a3b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:43.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.261+0000 7ff2d7764b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55936a3a8cc0 0x55936a3b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:43.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.261+0000 7ff2d7764b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55936a3ae1a0 0x55936a3b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:43.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.261+0000 7ff2d7764b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55936a298b10 con 0x55936a3a8cc0 2023-09-13T03:18:43.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.261+0000 7ff2d7764b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55936a298810 con 0x55936a3a8300 2023-09-13T03:18:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.261+0000 7ff2d7764b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55936a17c8c0 con 0x55936a3ae1a0 2023-09-13T03:18:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.261+0000 7ff2d7764b80 10 client.0 _ll_register_callbacks cb 0x55936a1e2a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:18:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.261+0000 7ff2c1b9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55936a3a8cc0 0x55936a3b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.261+0000 7ff2c1b9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55936a3a8cc0 0x55936a3b20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58706/0 (socket says 172.21.15.64:58706) 2023-09-13T03:18:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.261+0000 7ff2c1b9b700 1 -- 172.21.15.64:0/1913806026 learned_addr learned my addr 172.21.15.64:0/1913806026 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:18:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.262+0000 7ff2c139a700 1 --2- 172.21.15.64:0/1913806026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55936a3a8300 0x55936a3b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.262+0000 7ff2c1b9b700 1 -- 172.21.15.64:0/1913806026 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55936a3ae1a0 msgr2=0x55936a3b26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:18:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.262+0000 7ff2c1b9b700 1 --2- 172.21.15.64:0/1913806026 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55936a3ae1a0 0x55936a3b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.262+0000 7ff2c1b9b700 1 -- 172.21.15.64:0/1913806026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55936a3a8300 msgr2=0x55936a3b4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:18:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.262+0000 7ff2c1b9b700 1 --2- 172.21.15.64:0/1913806026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55936a3a8300 0x55936a3b4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.262+0000 7ff2c1b9b700 1 -- 172.21.15.64:0/1913806026 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2ac019040 con 0x55936a3a8cc0 2023-09-13T03:18:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.262+0000 7ff2c1b9b700 1 --2- 172.21.15.64:0/1913806026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55936a3a8cc0 0x55936a3b20c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8009d10 tx=0x7ff2b8004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7556b7b94b3fd08 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:18:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.262+0000 7ff2a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:18:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.262+0000 7ff2a57fa700 1 -- 172.21.15.64:0/1913806026 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b8005240 con 0x55936a3a8cc0 2023-09-13T03:18:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.262+0000 7ff2d7764b80 1 -- 172.21.15.64:0/1913806026 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55936a252490 con 0x55936a3a8cc0 2023-09-13T03:18:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.262+0000 7ff2a57fa700 1 -- 172.21.15.64:0/1913806026 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7ff2b80053a0 con 0x55936a3a8cc0 2023-09-13T03:18:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.263+0000 7ff2a57fa700 1 -- 172.21.15.64:0/1913806026 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2b8005560 con 0x55936a3a8cc0 2023-09-13T03:18:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.263+0000 7ff2d7764b80 10 client.16533 fetch_fsmap learned FSMap version 46 2023-09-13T03:18:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.263+0000 7ff2d7764b80 1 -- 172.21.15.64:0/1913806026 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7ff2bc10dd30 con 0x55936a3a8cc0 2023-09-13T03:18:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.263+0000 7ff2a57fa700 1 -- 172.21.15.64:0/1913806026 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55936a252490 con 0x55936a3a8cc0 2023-09-13T03:18:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.263+0000 7ff2a57fa700 1 -- 172.21.15.64:0/1913806026 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 46) v1 ==== 7906+0+0 (secure 0 0 0) 0x7ff2b8021070 con 0x55936a3a8cc0 2023-09-13T03:18:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.263+0000 7ff2d7764b80 10 client.16533 fetch_fsmap finished waiting for FSMap version 46 2023-09-13T03:18:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.263+0000 7ff2d7764b80 10 client.16533 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:18:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.263+0000 7ff2d7764b80 20 client.16533 populate_metadata read hostname 'smithi064' 2023-09-13T03:18:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.263+0000 7ff2d7764b80 1 --2- 172.21.15.64:0/1913806026 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55936a3c1ff0 0x55936a3c4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.263+0000 7ff2d7764b80 4 client.16533 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:18:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.263+0000 7ff2d7764b80 1 -- 172.21.15.64:0/1913806026 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55936a30f840 con 0x55936a3c1ff0 2023-09-13T03:18:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.264+0000 7ff2c139a700 1 --2- 172.21.15.64:0/1913806026 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55936a3c1ff0 0x55936a3c4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.264+0000 7ff2c139a700 1 --2- 172.21.15.64:0/1913806026 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55936a3c1ff0 0x55936a3c4410 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abe470a295ae73fb server_cookie=cf08dc7c6166d658 in_seq=0 out_seq=0 2023-09-13T03:18:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.264+0000 7ff2a57fa700 10 client.16533 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:18:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.265+0000 7ff2a57fa700 1 -- 172.21.15.64:0/1913806026 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55936a30f840 con 0x55936a3c1ff0 2023-09-13T03:18:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.265+0000 7ff2a57fa700 10 client.16533 handle_command_reply: tid=0 2023-09-13T03:18:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.265+0000 7ff2d7764b80 1 client.16533 shutdown 2023-09-13T03:18:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.265+0000 7ff2d7764b80 1 -- 172.21.15.64:0/1913806026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55936a3a8cc0 msgr2=0x55936a3b20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.265+0000 7ff2d7764b80 1 --2- 172.21.15.64:0/1913806026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55936a3a8cc0 0x55936a3b20c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8009d10 tx=0x7ff2b8004ac0 comp rx=0 tx=0).stop 2023-09-13T03:18:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.265+0000 7ff2d7764b80 1 -- 172.21.15.64:0/1913806026 shutdown_connections 2023-09-13T03:18:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.265+0000 7ff2d7764b80 1 --2- 172.21.15.64:0/1913806026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55936a3a8300 0x55936a3b4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.265+0000 7ff2d7764b80 1 --2- 172.21.15.64:0/1913806026 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55936a3c1ff0 0x55936a3c4410 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.265+0000 7ff2d7764b80 1 --2- 172.21.15.64:0/1913806026 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55936a3ae1a0 0x55936a3b26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.265+0000 7ff2d7764b80 1 --2- 172.21.15.64:0/1913806026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55936a3a8cc0 0x55936a3b20c0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.265+0000 7ff2a57fa700 0 client.16533 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:18:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.265+0000 7ff2d7764b80 1 -- 172.21.15.64:0/1913806026 >> 172.21.15.64:0/1913806026 conn(0x55936a31d030 msgr2=0x55936a3a27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:18:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.265+0000 7ff2d7764b80 1 -- 172.21.15.64:0/1913806026 shutdown_connections 2023-09-13T03:18:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.265+0000 7ff2d7764b80 1 -- 172.21.15.64:0/1913806026 wait complete. 2023-09-13T03:18:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.266+0000 7ff2d7764b80 20 client.16533 trim_cache size 0 max 16384 2023-09-13T03:18:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.281+0000 7ff2d7764b80 1 Processor -- start 2023-09-13T03:18:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.281+0000 7ff2d7764b80 1 -- start start 2023-09-13T03:18:43.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.281+0000 7ff2d7764b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55936a3a8300 0x55936a3b93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:43.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.281+0000 7ff2d7764b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55936a3a8cc0 0x55936a3b4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:43.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.281+0000 7ff2d7764b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55936a3ae1a0 0x55936a3aab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:43.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.281+0000 7ff2d7764b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55936a30f840 con 0x55936a3a8300 2023-09-13T03:18:43.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.281+0000 7ff2d7764b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55936a252490 con 0x55936a3ae1a0 2023-09-13T03:18:43.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.281+0000 7ff2d7764b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55936a39ae50 con 0x55936a3a8cc0 2023-09-13T03:18:43.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.281+0000 7ff2c139a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55936a3a8300 0x55936a3b93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:43.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.282+0000 7ff2c139a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55936a3a8300 0x55936a3b93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58718/0 (socket says 172.21.15.64:58718) 2023-09-13T03:18:43.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.282+0000 7ff2c139a700 1 -- 172.21.15.64:0/3495480982 learned_addr learned my addr 172.21.15.64:0/3495480982 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:18:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.282+0000 7ff2c1b9b700 1 --2- 172.21.15.64:0/3495480982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55936a3a8cc0 0x55936a3b4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.282+0000 7ff2c139a700 1 --2- 172.21.15.64:0/3495480982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55936a3ae1a0 0x55936a3aab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.282+0000 7ff2c139a700 1 -- 172.21.15.64:0/3495480982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55936a3a8cc0 msgr2=0x55936a3b4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:18:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.282+0000 7ff2c139a700 1 --2- 172.21.15.64:0/3495480982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55936a3a8cc0 0x55936a3b4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.282+0000 7ff2c139a700 1 -- 172.21.15.64:0/3495480982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55936a3ae1a0 msgr2=0x55936a3aab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:18:43.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.282+0000 7ff2c139a700 1 --2- 172.21.15.64:0/3495480982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55936a3ae1a0 0x55936a3aab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:43.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.282+0000 7ff2c139a700 1 -- 172.21.15.64:0/3495480982 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55936a10fc00 con 0x55936a3a8300 2023-09-13T03:18:43.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.282+0000 7ff2c139a700 1 --2- 172.21.15.64:0/3495480982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55936a3a8300 0x55936a3b93d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x55936a3c4d20 tx=0x7ff2ac0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd3f2f7b099279dd server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:18:43.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.282+0000 7ff2b0bff700 1 -- 172.21.15.64:0/3495480982 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac016040 con 0x55936a3a8300 2023-09-13T03:18:43.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.282+0000 7ff2b0bff700 1 -- 172.21.15.64:0/3495480982 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2ac011bd0 con 0x55936a3a8300 2023-09-13T03:18:43.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.282+0000 7ff2b0bff700 1 -- 172.21.15.64:0/3495480982 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac0039e0 con 0x55936a3a8300 2023-09-13T03:18:43.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.283+0000 7ff2d7764b80 1 -- 172.21.15.64:0/3495480982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55936a3a8300 msgr2=0x55936a3b93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:43.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.283+0000 7ff2d7764b80 1 --2- 172.21.15.64:0/3495480982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55936a3a8300 0x55936a3b93d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x55936a3c4d20 tx=0x7ff2ac0047c0 comp rx=0 tx=0).stop 2023-09-13T03:18:43.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.283+0000 7ff2d7764b80 1 -- 172.21.15.64:0/3495480982 shutdown_connections 2023-09-13T03:18:43.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.283+0000 7ff2d7764b80 1 --2- 172.21.15.64:0/3495480982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55936a3ae1a0 0x55936a3aab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.283+0000 7ff2d7764b80 1 --2- 172.21.15.64:0/3495480982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55936a3a8300 0x55936a3b93d0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.283+0000 7ff2d7764b80 1 --2- 172.21.15.64:0/3495480982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55936a3a8cc0 0x55936a3b4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.283+0000 7ff2d7764b80 1 -- 172.21.15.64:0/3495480982 >> 172.21.15.64:0/3495480982 conn(0x55936a31d030 msgr2=0x55936a3ad110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:18:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.283+0000 7ff2d7764b80 1 -- 172.21.15.64:0/3495480982 shutdown_connections 2023-09-13T03:18:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.283+0000 7ff2d7764b80 1 -- 172.21.15.64:0/3495480982 wait complete. 2023-09-13T03:18:43.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.284+0000 7ff2d7764b80 1 -- start start 2023-09-13T03:18:43.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.284+0000 7ff2d7764b80 1 Processor -- start 2023-09-13T03:18:43.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.284+0000 7ff2d7764b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55936a3a8300 0x55936a3b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:43.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.284+0000 7ff2d7764b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55936a3a8cc0 0x55936a3b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:43.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.284+0000 7ff2d7764b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55936a3ae1a0 0x55936a3b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:43.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.284+0000 7ff2d7764b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55936a3b12f0 con 0x55936a3a8300 2023-09-13T03:18:43.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.284+0000 7ff2d7764b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55936a3b1460 con 0x55936a3ae1a0 2023-09-13T03:18:43.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.284+0000 7ff2d7764b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55936a3b1620 con 0x55936a3a8cc0 2023-09-13T03:18:43.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.284+0000 7ff2d7764b80 10 client.0 _ll_register_callbacks cb 0x7ff2ac009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:18:43.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.284+0000 7ff2c139a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55936a3a8300 0x55936a3b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:43.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.285+0000 7ff2c139a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55936a3a8300 0x55936a3b1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58730/0 (socket says 172.21.15.64:58730) 2023-09-13T03:18:43.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.285+0000 7ff2c139a700 1 -- 172.21.15.64:0/4094968195 learned_addr learned my addr 172.21.15.64:0/4094968195 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:18:43.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.285+0000 7ff2c1b9b700 1 --2- 172.21.15.64:0/4094968195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55936a3a8cc0 0x55936a3b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:43.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.285+0000 7ff2c139a700 1 --2- 172.21.15.64:0/4094968195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55936a3ae1a0 0x55936a3b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:43.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.285+0000 7ff2c139a700 1 -- 172.21.15.64:0/4094968195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55936a3a8cc0 msgr2=0x55936a3b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:18:43.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.285+0000 7ff2c139a700 1 --2- 172.21.15.64:0/4094968195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55936a3a8cc0 0x55936a3b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:43.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.285+0000 7ff2c139a700 1 -- 172.21.15.64:0/4094968195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55936a3ae1a0 msgr2=0x55936a3b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:18:43.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.285+0000 7ff2c139a700 1 --2- 172.21.15.64:0/4094968195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55936a3ae1a0 0x55936a3b0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:43.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.285+0000 7ff2c139a700 1 -- 172.21.15.64:0/4094968195 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2ac022520 con 0x55936a3a8300 2023-09-13T03:18:43.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.285+0000 7ff2c1b9b700 1 --2- 172.21.15.64:0/4094968195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55936a3a8cc0 0x55936a3b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:18:43.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.285+0000 7ff2c139a700 1 --2- 172.21.15.64:0/4094968195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55936a3a8300 0x55936a3b1d30 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac011fb0 tx=0x7ff2ac0092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ffe82a85c705d74 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:18:43.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.285+0000 7ff2a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:18:43.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.285+0000 7ff2a57fa700 1 -- 172.21.15.64:0/4094968195 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac016040 con 0x55936a3a8300 2023-09-13T03:18:43.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.285+0000 7ff2a57fa700 1 -- 172.21.15.64:0/4094968195 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7ff2ac0119a0 con 0x55936a3a8300 2023-09-13T03:18:43.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.285+0000 7ff2d7764b80 1 -- 172.21.15.64:0/4094968195 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff2b801e040 con 0x55936a3a8300 2023-09-13T03:18:43.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.286+0000 7ff2a57fa700 1 -- 172.21.15.64:0/4094968195 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2ac003680 con 0x55936a3a8300 2023-09-13T03:18:43.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.286+0000 7ff2d7764b80 10 client.16539 fetch_fsmap learned FSMap version 46 2023-09-13T03:18:43.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.286+0000 7ff2d7764b80 1 -- 172.21.15.64:0/4094968195 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55936a14d1f0 con 0x55936a3a8300 2023-09-13T03:18:43.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.286+0000 7ff2a57fa700 1 -- 172.21.15.64:0/4094968195 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2ac03e020 con 0x55936a3a8300 2023-09-13T03:18:43.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.286+0000 7ff2a57fa700 1 -- 172.21.15.64:0/4094968195 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 46) v1 ==== 7906+0+0 (secure 0 0 0) 0x7ff2ac01f6a0 con 0x55936a3a8300 2023-09-13T03:18:43.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.286+0000 7ff2d7764b80 10 client.16539 fetch_fsmap finished waiting for FSMap version 46 2023-09-13T03:18:43.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.286+0000 7ff2d7764b80 10 client.16539 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:18:43.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.286+0000 7ff2d7764b80 20 client.16539 populate_metadata read hostname 'smithi064' 2023-09-13T03:18:43.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.286+0000 7ff2d7764b80 1 --2- 172.21.15.64:0/4094968195 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55936a4172a0 0x55936a4196c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:43.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.286+0000 7ff2d7764b80 4 client.16539 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:18:43.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.286+0000 7ff2d7764b80 1 -- 172.21.15.64:0/4094968195 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55936a2f38c0 con 0x55936a4172a0 2023-09-13T03:18:43.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.287+0000 7ff2c1b9b700 1 --2- 172.21.15.64:0/4094968195 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55936a4172a0 0x55936a4196c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:43.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.287+0000 7ff2c1b9b700 1 --2- 172.21.15.64:0/4094968195 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55936a4172a0 0x55936a4196c0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53e73421908392d8 server_cookie=ffac275e7fec9535 in_seq=0 out_seq=0 2023-09-13T03:18:43.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.287+0000 7ff2a57fa700 10 client.16539 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:18:43.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.327+0000 7ff2a57fa700 1 -- 172.21.15.64:0/4094968195 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55936a2f38c0 con 0x55936a4172a0 2023-09-13T03:18:43.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.327+0000 7ff2a57fa700 10 client.16539 handle_command_reply: tid=0 2023-09-13T03:18:43.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.327+0000 7ff2d7764b80 1 client.16539 shutdown 2023-09-13T03:18:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.327+0000 7ff2d7764b80 1 -- 172.21.15.64:0/4094968195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55936a3a8300 msgr2=0x55936a3b1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.327+0000 7ff2d7764b80 1 --2- 172.21.15.64:0/4094968195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55936a3a8300 0x55936a3b1d30 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac011fb0 tx=0x7ff2ac0092a0 comp rx=0 tx=0).stop 2023-09-13T03:18:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.327+0000 7ff2d7764b80 1 -- 172.21.15.64:0/4094968195 shutdown_connections 2023-09-13T03:18:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.327+0000 7ff2d7764b80 1 --2- 172.21.15.64:0/4094968195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55936a3ae1a0 0x55936a3b0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.327+0000 7ff2d7764b80 1 --2- 172.21.15.64:0/4094968195 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55936a4172a0 0x55936a4196c0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.327+0000 7ff2d7764b80 1 --2- 172.21.15.64:0/4094968195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55936a3a8300 0x55936a3b1d30 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.327+0000 7ff2d7764b80 1 --2- 172.21.15.64:0/4094968195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55936a3a8cc0 0x55936a3b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.327+0000 7ff2a57fa700 0 client.16539 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:18:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.327+0000 7ff2d7764b80 1 -- 172.21.15.64:0/4094968195 >> 172.21.15.64:0/4094968195 conn(0x55936a31d030 msgr2=0x55936a3a72f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:18:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.327+0000 7ff2d7764b80 1 -- 172.21.15.64:0/4094968195 shutdown_connections 2023-09-13T03:18:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.327+0000 7ff2d7764b80 1 -- 172.21.15.64:0/4094968195 wait complete. 2023-09-13T03:18:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.328+0000 7ff2d7764b80 20 client.16539 trim_cache size 0 max 16384 2023-09-13T03:18:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:18:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:18:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:18:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:18:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.328+0000 7ff2c3dff700 1 -- 172.21.15.64:0/2135743824 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7ff2a80a0830 msgr2=0x7ff2a80a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.328+0000 7ff2c3dff700 1 --2- 172.21.15.64:0/2135743824 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7ff2a80a0830 0x7ff2a80a2cf0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff2b40a0730 tx=0x7ff2b4009f90 comp rx=0 tx=0).stop 2023-09-13T03:18:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.328+0000 7ff2c3dff700 1 -- 172.21.15.64:0/2135743824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2bc106670 msgr2=0x7ff2bc1964e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.328+0000 7ff2c3dff700 1 --2- 172.21.15.64:0/2135743824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2bc106670 0x7ff2bc1964e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4007f10 tx=0x7ff2b400d3b0 comp rx=0 tx=0).stop 2023-09-13T03:18:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.328+0000 7ff2c3dff700 1 -- 172.21.15.64:0/2135743824 shutdown_connections 2023-09-13T03:18:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.328+0000 7ff2c3dff700 1 --2- 172.21.15.64:0/2135743824 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7ff2a80a0830 0x7ff2a80a2cf0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.328+0000 7ff2c3dff700 1 --2- 172.21.15.64:0/2135743824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2bc106670 0x7ff2bc1964e0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.328+0000 7ff2c3dff700 1 --2- 172.21.15.64:0/2135743824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2bc1005b0 0x7ff2bc19c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.328+0000 7ff2c3dff700 1 --2- 172.21.15.64:0/2135743824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2bc100f70 0x7ff2bc19cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.328+0000 7ff2c3dff700 1 -- 172.21.15.64:0/2135743824 >> 172.21.15.64:0/2135743824 conn(0x7ff2bc0fc060 msgr2=0x7ff2bc104240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:18:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.329+0000 7ff2c3dff700 1 -- 172.21.15.64:0/2135743824 shutdown_connections 2023-09-13T03:18:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:43.329+0000 7ff2c3dff700 1 -- 172.21.15.64:0/2135743824 wait complete. 2023-09-13T03:18:43.915 INFO:tasks.workunit.client.0.smithi064.stdout: 1 114266 1.93 MB/sec execute 168 sec latency 1632.447 ms 2023-09-13T03:18:44.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:18:44 smithi170 ceph-mon[134328]: pgmap v170: 97 pgs: 97 active+clean; 63 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 350 KiB/s wr, 61 op/s 2023-09-13T03:18:44.915 INFO:tasks.workunit.client.0.smithi064.stdout: 1 114266 1.92 MB/sec execute 169 sec latency 1347.152 ms 2023-09-13T03:18:44.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:18:44 smithi064 ceph-mon[97675]: pgmap v170: 97 pgs: 97 active+clean; 63 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 350 KiB/s wr, 61 op/s 2023-09-13T03:18:44.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:18:44 smithi142 ceph-mon[134082]: pgmap v170: 97 pgs: 97 active+clean; 63 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 350 KiB/s wr, 61 op/s 2023-09-13T03:18:45.915 INFO:tasks.workunit.client.0.smithi064.stdout: 1 114266 1.91 MB/sec execute 170 sec latency 2347.231 ms 2023-09-13T03:18:46.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:18:46 smithi170 ceph-mon[134328]: pgmap v171: 97 pgs: 97 active+clean; 61 MiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 557 KiB/s wr, 100 op/s 2023-09-13T03:18:46.915 INFO:tasks.workunit.client.0.smithi064.stdout: 1 114266 1.90 MB/sec execute 171 sec latency 3347.377 ms 2023-09-13T03:18:46.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:18:46 smithi064 ceph-mon[97675]: pgmap v171: 97 pgs: 97 active+clean; 61 MiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 557 KiB/s wr, 100 op/s 2023-09-13T03:18:46.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:18:46 smithi142 ceph-mon[134082]: pgmap v171: 97 pgs: 97 active+clean; 61 MiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 557 KiB/s wr, 100 op/s 2023-09-13T03:18:47.915 INFO:tasks.workunit.client.0.smithi064.stdout: 1 114266 1.89 MB/sec execute 172 sec latency 4347.518 ms 2023-09-13T03:18:48.915 INFO:tasks.workunit.client.0.smithi064.stdout: 1 114334 1.88 MB/sec execute 173 sec latency 4957.870 ms 2023-09-13T03:18:48.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:18:48 smithi064 ceph-mon[97675]: pgmap v172: 97 pgs: 97 active+clean; 61 MiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 516 KiB/s wr, 86 op/s 2023-09-13T03:18:48.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:18:48 smithi142 ceph-mon[134082]: pgmap v172: 97 pgs: 97 active+clean; 61 MiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 516 KiB/s wr, 86 op/s 2023-09-13T03:18:49.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:18:48 smithi170 ceph-mon[134328]: pgmap v172: 97 pgs: 97 active+clean; 61 MiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 516 KiB/s wr, 86 op/s 2023-09-13T03:18:49.915 INFO:tasks.workunit.client.0.smithi064.stdout: 1 114334 1.87 MB/sec execute 174 sec latency 1338.069 ms 2023-09-13T03:18:50.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:18:50 smithi170 ceph-mon[134328]: pgmap v173: 97 pgs: 97 active+clean; 61 MiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 516 KiB/s wr, 86 op/s 2023-09-13T03:18:50.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:18:50 smithi064 ceph-mon[97675]: pgmap v173: 97 pgs: 97 active+clean; 61 MiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 516 KiB/s wr, 86 op/s 2023-09-13T03:18:50.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:18:50 smithi142 ceph-mon[134082]: pgmap v173: 97 pgs: 97 active+clean; 61 MiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 516 KiB/s wr, 86 op/s 2023-09-13T03:18:50.915 INFO:tasks.workunit.client.0.smithi064.stdout: 1 114334 1.86 MB/sec execute 175 sec latency 2338.198 ms 2023-09-13T03:18:51.916 INFO:tasks.workunit.client.0.smithi064.stdout: 1 114334 1.85 MB/sec execute 176 sec latency 3338.326 ms 2023-09-13T03:18:52.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:18:52 smithi170 ceph-mon[134328]: pgmap v174: 97 pgs: 97 active+clean; 62 MiB data, 830 MiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 561 KiB/s wr, 97 op/s 2023-09-13T03:18:52.916 INFO:tasks.workunit.client.0.smithi064.stdout: 1 115235 1.86 MB/sec execute 177 sec latency 3573.393 ms 2023-09-13T03:18:52.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:18:52 smithi064 ceph-mon[97675]: pgmap v174: 97 pgs: 97 active+clean; 62 MiB data, 830 MiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 561 KiB/s wr, 97 op/s 2023-09-13T03:18:52.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:18:52 smithi142 ceph-mon[134082]: pgmap v174: 97 pgs: 97 active+clean; 62 MiB data, 830 MiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 561 KiB/s wr, 97 op/s 2023-09-13T03:18:53.916 INFO:tasks.workunit.client.0.smithi064.stdout: 1 115235 1.85 MB/sec execute 178 sec latency 1202.574 ms 2023-09-13T03:18:54.916 INFO:tasks.workunit.client.0.smithi064.stdout: 1 115235 1.84 MB/sec execute 179 sec latency 2202.672 ms 2023-09-13T03:18:54.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:18:54 smithi064 ceph-mon[97675]: pgmap v175: 97 pgs: 97 active+clean; 62 MiB data, 830 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 252 KiB/s wr, 49 op/s 2023-09-13T03:18:54.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:18:54 smithi142 ceph-mon[134082]: pgmap v175: 97 pgs: 97 active+clean; 62 MiB data, 830 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 252 KiB/s wr, 49 op/s 2023-09-13T03:18:55.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:18:54 smithi170 ceph-mon[134328]: pgmap v175: 97 pgs: 97 active+clean; 62 MiB data, 830 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 252 KiB/s wr, 49 op/s 2023-09-13T03:18:55.916 INFO:tasks.workunit.client.0.smithi064.stdout: 1 116442 1.87 MB/sec execute 180 sec latency 2637.033 ms 2023-09-13T03:18:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.483+0000 7fc7e4dff700 1 -- 172.21.15.64:0/1704641400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7e01005a0 msgr2=0x7fc7e0100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.483+0000 7fc7e4dff700 1 --2- 172.21.15.64:0/1704641400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7e01005a0 0x7fc7e0100a20 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc7d40099c0 tx=0x7fc7d4009cd0 comp rx=0 tx=0).stop 2023-09-13T03:18:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.483+0000 7fc7e4dff700 1 -- 172.21.15.64:0/1704641400 shutdown_connections 2023-09-13T03:18:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.483+0000 7fc7e4dff700 1 --2- 172.21.15.64:0/1704641400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7e0106660 0x7fc7e0106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:56.485 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.483+0000 7fc7e4dff700 1 --2- 172.21.15.64:0/1704641400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7e0100f60 0x7fc7e010d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:56.485 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.483+0000 7fc7e4dff700 1 --2- 172.21.15.64:0/1704641400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7e01005a0 0x7fc7e0100a20 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:56.485 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.483+0000 7fc7e4dff700 1 -- 172.21.15.64:0/1704641400 >> 172.21.15.64:0/1704641400 conn(0x7fc7e00fc050 msgr2=0x7fc7e00fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:18:56.485 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.483+0000 7fc7e4dff700 1 -- 172.21.15.64:0/1704641400 shutdown_connections 2023-09-13T03:18:56.485 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.483+0000 7fc7e4dff700 1 -- 172.21.15.64:0/1704641400 wait complete. 2023-09-13T03:18:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.484+0000 7fc7e4dff700 1 Processor -- start 2023-09-13T03:18:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.484+0000 7fc7e4dff700 1 -- start start 2023-09-13T03:18:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.485+0000 7fc7e4dff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7e01005a0 0x7fc7e019e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.485+0000 7fc7e4dff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7e0100f60 0x7fc7e019ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.485+0000 7fc7e4dff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7e0106660 0x7fc7e0198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.485+0000 7fc7e4dff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e0198cc0 con 0x7fc7e0106660 2023-09-13T03:18:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.485+0000 7fc7e4dff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e0198e30 con 0x7fc7e01005a0 2023-09-13T03:18:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.485+0000 7fc7e4dff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e0198ff0 con 0x7fc7e0100f60 2023-09-13T03:18:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.485+0000 7fc7de59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7e01005a0 0x7fc7e019e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.485+0000 7fc7de59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7e01005a0 0x7fc7e019e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:39130/0 (socket says 172.21.15.64:39130) 2023-09-13T03:18:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.485+0000 7fc7de59c700 1 -- 172.21.15.64:0/1493613547 learned_addr learned my addr 172.21.15.64:0/1493613547 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:18:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.485+0000 7fc7ddd9b700 1 --2- 172.21.15.64:0/1493613547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7e0100f60 0x7fc7e019ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.485+0000 7fc7ded9d700 1 --2- 172.21.15.64:0/1493613547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7e0106660 0x7fc7e0198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.485+0000 7fc7de59c700 1 -- 172.21.15.64:0/1493613547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7e0100f60 msgr2=0x7fc7e019ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.485+0000 7fc7de59c700 1 --2- 172.21.15.64:0/1493613547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7e0100f60 0x7fc7e019ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.486+0000 7fc7de59c700 1 -- 172.21.15.64:0/1493613547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7e0106660 msgr2=0x7fc7e0198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.486+0000 7fc7de59c700 1 --2- 172.21.15.64:0/1493613547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7e0106660 0x7fc7e0198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.486+0000 7fc7de59c700 1 -- 172.21.15.64:0/1493613547 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d8009c40 con 0x7fc7e01005a0 2023-09-13T03:18:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.486+0000 7fc7ddd9b700 1 --2- 172.21.15.64:0/1493613547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7e0100f60 0x7fc7e019ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:18:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.486+0000 7fc7ded9d700 1 --2- 172.21.15.64:0/1493613547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7e0106660 0x7fc7e0198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:18:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.486+0000 7fc7de59c700 1 --2- 172.21.15.64:0/1493613547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7e01005a0 0x7fc7e019e7f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc7d000e9a0 tx=0x7fc7d000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:18:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.486+0000 7fc7cf7fe700 1 -- 172.21.15.64:0/1493613547 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d0009790 con 0x7fc7e01005a0 2023-09-13T03:18:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.486+0000 7fc7e4dff700 1 -- 172.21.15.64:0/1493613547 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7d40095f0 con 0x7fc7e01005a0 2023-09-13T03:18:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.486+0000 7fc7cf7fe700 1 -- 172.21.15.64:0/1493613547 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7d000ce80 con 0x7fc7e01005a0 2023-09-13T03:18:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.486+0000 7fc7cf7fe700 1 -- 172.21.15.64:0/1493613547 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d0018680 con 0x7fc7e01005a0 2023-09-13T03:18:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.486+0000 7fc7e4dff700 1 -- 172.21.15.64:0/1493613547 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7e0199490 con 0x7fc7e01005a0 2023-09-13T03:18:56.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.491+0000 7fc7cf7fe700 1 -- 172.21.15.64:0/1493613547 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fc7d0010430 con 0x7fc7e01005a0 2023-09-13T03:18:56.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.492+0000 7fc7f87a5b80 1 Processor -- start 2023-09-13T03:18:56.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.492+0000 7fc7f87a5b80 1 -- start start 2023-09-13T03:18:56.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.492+0000 7fc7f87a5b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d50805d1a0 0x55d50805d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:56.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.493+0000 7fc7f87a5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d508057300 0x55d508057780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:56.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.493+0000 7fc7f87a5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d508057cc0 0x55d508063f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:56.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.493+0000 7fc7f87a5b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d507fb06d0 con 0x55d50805d1a0 2023-09-13T03:18:56.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.493+0000 7fc7f87a5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d507fadff0 con 0x55d508057300 2023-09-13T03:18:56.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.493+0000 7fc7f87a5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d507fa98e0 con 0x55d508057cc0 2023-09-13T03:18:56.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.493+0000 7fc7cf7fe700 1 --2- 172.21.15.64:0/1493613547 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fc7c80a07b0 0x7fc7c80a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:56.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.493+0000 7fc7de59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d508057300 0x55d508057780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:56.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.493+0000 7fc7ddd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d508057cc0 0x55d508063f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:56.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.493+0000 7fc7ddd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d508057cc0 0x55d508063f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:58728/0 (socket says 172.21.15.64:58728) 2023-09-13T03:18:56.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.493+0000 7fc7de59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d508057300 0x55d508057780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:39138/0 (socket says 172.21.15.64:39138) 2023-09-13T03:18:56.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.493+0000 7fc7ddd9b700 1 -- 172.21.15.64:0/2266180703 learned_addr learned my addr 172.21.15.64:0/2266180703 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:18:56.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.493+0000 7fc7ded9d700 1 --2- 172.21.15.64:0/2266180703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d50805d1a0 0x55d50805d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:56.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.493+0000 7fc7cf7fe700 1 -- 172.21.15.64:0/1493613547 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc7d0014070 con 0x7fc7e01005a0 2023-09-13T03:18:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.493+0000 7fc7ded9d700 1 --2- 172.21.15.64:0/1493613547 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fc7c80a07b0 0x7fc7c80a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.493+0000 7fc7ddd9b700 1 -- 172.21.15.64:0/2266180703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d508057300 msgr2=0x55d508057780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:18:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.493+0000 7fc7ddd9b700 1 --2- 172.21.15.64:0/2266180703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d508057300 0x55d508057780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.493+0000 7fc7ddd9b700 1 -- 172.21.15.64:0/2266180703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d50805d1a0 msgr2=0x55d50805d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:18:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.493+0000 7fc7ddd9b700 1 --2- 172.21.15.64:0/2266180703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d50805d1a0 0x55d50805d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.493+0000 7fc7ddd9b700 1 -- 172.21.15.64:0/2266180703 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d507dd5d80 con 0x55d508057cc0 2023-09-13T03:18:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.494+0000 7fc7ddd9b700 1 --2- 172.21.15.64:0/2266180703 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d508057cc0 0x55d508063f30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc7d40058e0 tx=0x7fc7d4005ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b691a7ab4eef653 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:18:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.494+0000 7fc7ccff9700 1 -- 172.21.15.64:0/2266180703 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d400bdc0 con 0x55d508057cc0 2023-09-13T03:18:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.494+0000 7fc7ded9d700 1 --2- 172.21.15.64:0/1493613547 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fc7c80a07b0 0x7fc7c80a2c70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8009fa0 tx=0x7fc7d800b040 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:18:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.494+0000 7fc7ccff9700 1 -- 172.21.15.64:0/2266180703 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7d401e040 con 0x55d508057cc0 2023-09-13T03:18:56.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.494+0000 7fc7ccff9700 1 -- 172.21.15.64:0/2266180703 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d4003e40 con 0x55d508057cc0 2023-09-13T03:18:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.495+0000 7fc7f87a5b80 1 -- 172.21.15.64:0/2266180703 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d508057cc0 msgr2=0x55d508063f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.495+0000 7fc7f87a5b80 1 --2- 172.21.15.64:0/2266180703 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d508057cc0 0x55d508063f30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc7d40058e0 tx=0x7fc7d4005ee0 comp rx=0 tx=0).stop 2023-09-13T03:18:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.495+0000 7fc7f87a5b80 1 -- 172.21.15.64:0/2266180703 shutdown_connections 2023-09-13T03:18:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.495+0000 7fc7f87a5b80 1 --2- 172.21.15.64:0/2266180703 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d508057cc0 0x55d508063f30 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.495+0000 7fc7f87a5b80 1 --2- 172.21.15.64:0/2266180703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d50805d1a0 0x55d50805d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.495+0000 7fc7f87a5b80 1 --2- 172.21.15.64:0/2266180703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d508057300 0x55d508057780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.495+0000 7fc7f87a5b80 1 -- 172.21.15.64:0/2266180703 >> 172.21.15.64:0/2266180703 conn(0x55d507fcc030 msgr2=0x55d508052560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:18:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.495+0000 7fc7f87a5b80 1 -- 172.21.15.64:0/2266180703 shutdown_connections 2023-09-13T03:18:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.495+0000 7fc7f87a5b80 1 -- 172.21.15.64:0/2266180703 wait complete. 2023-09-13T03:18:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.495+0000 7fc7f87a5b80 1 -- start start 2023-09-13T03:18:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.496+0000 7fc7f87a5b80 1 Processor -- start 2023-09-13T03:18:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.496+0000 7fc7f87a5b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d508057300 0x55d508063d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.496+0000 7fc7f87a5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d508057cc0 0x55d508061080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.496+0000 7fc7f87a5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d50805d1a0 0x55d508061620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.496+0000 7fc7f87a5b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d507f47b10 con 0x55d508057300 2023-09-13T03:18:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.496+0000 7fc7f87a5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d507f47810 con 0x55d508057cc0 2023-09-13T03:18:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.496+0000 7fc7f87a5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d507e2b8c0 con 0x55d50805d1a0 2023-09-13T03:18:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.496+0000 7fc7f87a5b80 10 client.0 _ll_register_callbacks cb 0x55d507e91a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:18:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.496+0000 7fc7de59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d508057300 0x55d508063d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.496+0000 7fc7de59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d508057300 0x55d508063d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44042/0 (socket says 172.21.15.64:44042) 2023-09-13T03:18:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.496+0000 7fc7de59c700 1 -- 172.21.15.64:0/4242710292 learned_addr learned my addr 172.21.15.64:0/4242710292 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:18:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.497+0000 7fc7ddd9b700 1 --2- 172.21.15.64:0/4242710292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d508057cc0 0x55d508061080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.497+0000 7fc7ded9d700 1 --2- 172.21.15.64:0/4242710292 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d50805d1a0 0x55d508061620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.497+0000 7fc7de59c700 1 -- 172.21.15.64:0/4242710292 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d50805d1a0 msgr2=0x55d508061620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:18:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.497+0000 7fc7de59c700 1 --2- 172.21.15.64:0/4242710292 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d50805d1a0 0x55d508061620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.497+0000 7fc7de59c700 1 -- 172.21.15.64:0/4242710292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d508057cc0 msgr2=0x55d508061080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:18:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.497+0000 7fc7de59c700 1 --2- 172.21.15.64:0/4242710292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d508057cc0 0x55d508061080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.497+0000 7fc7de59c700 1 -- 172.21.15.64:0/4242710292 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7e010dd20 con 0x55d508057300 2023-09-13T03:18:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.497+0000 7fc7ddd9b700 1 --2- 172.21.15.64:0/4242710292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d508057cc0 0x55d508061080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:18:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.497+0000 7fc7de59c700 1 --2- 172.21.15.64:0/4242710292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d508057300 0x55d508063d90 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0099320 tx=0x7fc7d0099d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aef84a6ca1ddc987 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:18:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.497+0000 7fc7c5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:18:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.497+0000 7fc7c5ffb700 1 -- 172.21.15.64:0/4242710292 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d009d450 con 0x55d508057300 2023-09-13T03:18:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.497+0000 7fc7c5ffb700 1 -- 172.21.15.64:0/4242710292 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc7d00a1690 con 0x55d508057300 2023-09-13T03:18:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.497+0000 7fc7f87a5b80 1 -- 172.21.15.64:0/4242710292 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d507f01490 con 0x55d508057300 2023-09-13T03:18:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.498+0000 7fc7c5ffb700 1 -- 172.21.15.64:0/4242710292 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7d00840b0 con 0x55d508057300 2023-09-13T03:18:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.498+0000 7fc7f87a5b80 10 client.16551 fetch_fsmap learned FSMap version 46 2023-09-13T03:18:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.498+0000 7fc7f87a5b80 1 -- 172.21.15.64:0/4242710292 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55d507dbec00 con 0x55d508057300 2023-09-13T03:18:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.498+0000 7fc7c5ffb700 1 -- 172.21.15.64:0/4242710292 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d507f01490 con 0x55d508057300 2023-09-13T03:18:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.498+0000 7fc7c5ffb700 1 -- 172.21.15.64:0/4242710292 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 46) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fc7d00a1200 con 0x55d508057300 2023-09-13T03:18:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.498+0000 7fc7f87a5b80 10 client.16551 fetch_fsmap finished waiting for FSMap version 46 2023-09-13T03:18:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.498+0000 7fc7f87a5b80 10 client.16551 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:18:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.498+0000 7fc7f87a5b80 20 client.16551 populate_metadata read hostname 'smithi064' 2023-09-13T03:18:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.498+0000 7fc7f87a5b80 1 --2- 172.21.15.64:0/4242710292 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55d508071310 0x55d508073730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.498+0000 7fc7f87a5b80 4 client.16551 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:18:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.498+0000 7fc7f87a5b80 1 -- 172.21.15.64:0/4242710292 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d507fbe840 con 0x55d508071310 2023-09-13T03:18:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.499+0000 7fc7ddd9b700 1 --2- 172.21.15.64:0/4242710292 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55d508071310 0x55d508073730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.499+0000 7fc7ddd9b700 1 --2- 172.21.15.64:0/4242710292 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55d508071310 0x55d508073730 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=644bc7fe4540788b server_cookie=d1092b141db4937 in_seq=0 out_seq=0 2023-09-13T03:18:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.499+0000 7fc7c5ffb700 10 client.16551 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:18:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.500+0000 7fc7c5ffb700 1 -- 172.21.15.64:0/4242710292 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d507fbe840 con 0x55d508071310 2023-09-13T03:18:56.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.500+0000 7fc7c5ffb700 10 client.16551 handle_command_reply: tid=0 2023-09-13T03:18:56.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.500+0000 7fc7f87a5b80 1 client.16551 shutdown 2023-09-13T03:18:56.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.500+0000 7fc7f87a5b80 1 -- 172.21.15.64:0/4242710292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d508057300 msgr2=0x55d508063d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:56.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.500+0000 7fc7f87a5b80 1 --2- 172.21.15.64:0/4242710292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d508057300 0x55d508063d90 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0099320 tx=0x7fc7d0099d20 comp rx=0 tx=0).stop 2023-09-13T03:18:56.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.500+0000 7fc7f87a5b80 1 -- 172.21.15.64:0/4242710292 shutdown_connections 2023-09-13T03:18:56.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.500+0000 7fc7f87a5b80 1 --2- 172.21.15.64:0/4242710292 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55d508071310 0x55d508073730 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:56.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.500+0000 7fc7f87a5b80 1 --2- 172.21.15.64:0/4242710292 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d50805d1a0 0x55d508061620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:56.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.500+0000 7fc7f87a5b80 1 --2- 172.21.15.64:0/4242710292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d508057300 0x55d508063d90 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:56.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.500+0000 7fc7f87a5b80 1 --2- 172.21.15.64:0/4242710292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d508057cc0 0x55d508061080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:56.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.500+0000 7fc7f87a5b80 1 -- 172.21.15.64:0/4242710292 >> 172.21.15.64:0/4242710292 conn(0x55d507fcc030 msgr2=0x55d5080517a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:18:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.500+0000 7fc7c5ffb700 0 client.16551 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:18:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.500+0000 7fc7f87a5b80 1 -- 172.21.15.64:0/4242710292 shutdown_connections 2023-09-13T03:18:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.500+0000 7fc7f87a5b80 1 -- 172.21.15.64:0/4242710292 wait complete. 2023-09-13T03:18:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.500+0000 7fc7f87a5b80 20 client.16551 trim_cache size 0 max 16384 2023-09-13T03:18:56.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.516+0000 7fc7f87a5b80 1 Processor -- start 2023-09-13T03:18:56.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.516+0000 7fc7f87a5b80 1 -- start start 2023-09-13T03:18:56.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.517+0000 7fc7f87a5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d508057300 0x55d508063930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:56.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.517+0000 7fc7f87a5b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d508057cc0 0x55d50805c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:56.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.517+0000 7fc7f87a5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d50805d1a0 0x55d50805c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:56.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.517+0000 7fc7f87a5b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d50805f650 con 0x55d508057cc0 2023-09-13T03:18:56.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.517+0000 7fc7f87a5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5080582f0 con 0x55d508057300 2023-09-13T03:18:56.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.517+0000 7fc7f87a5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d50805bca0 con 0x55d50805d1a0 2023-09-13T03:18:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.517+0000 7fc7de59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d508057300 0x55d508063930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.517+0000 7fc7de59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d508057300 0x55d508063930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:39152/0 (socket says 172.21.15.64:39152) 2023-09-13T03:18:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.517+0000 7fc7de59c700 1 -- 172.21.15.64:0/310144360 learned_addr learned my addr 172.21.15.64:0/310144360 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:18:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.517+0000 7fc7ddd9b700 1 --2- 172.21.15.64:0/310144360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d508057cc0 0x55d50805c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.517+0000 7fc7ded9d700 1 --2- 172.21.15.64:0/310144360 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d50805d1a0 0x55d50805c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.517+0000 7fc7de59c700 1 -- 172.21.15.64:0/310144360 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d50805d1a0 msgr2=0x55d50805c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:18:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.517+0000 7fc7de59c700 1 --2- 172.21.15.64:0/310144360 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d50805d1a0 0x55d50805c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.517+0000 7fc7de59c700 1 -- 172.21.15.64:0/310144360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d508057cc0 msgr2=0x55d50805c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:18:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.517+0000 7fc7de59c700 1 --2- 172.21.15.64:0/310144360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d508057cc0 0x55d50805c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.517+0000 7fc7de59c700 1 -- 172.21.15.64:0/310144360 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d507dfc1f0 con 0x55d508057300 2023-09-13T03:18:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.518+0000 7fc7de59c700 1 --2- 172.21.15.64:0/310144360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d508057300 0x55d508063930 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc7d000e970 tx=0x7fc7d0005b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5450a3de232b5541 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:18:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.518+0000 7fc7cd7fa700 1 -- 172.21.15.64:0/310144360 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d00954b0 con 0x55d508057300 2023-09-13T03:18:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.518+0000 7fc7cd7fa700 1 -- 172.21.15.64:0/310144360 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7d00a1690 con 0x55d508057300 2023-09-13T03:18:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.518+0000 7fc7cd7fa700 1 -- 172.21.15.64:0/310144360 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d00a1380 con 0x55d508057300 2023-09-13T03:18:56.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.518+0000 7fc7f87a5b80 1 -- 172.21.15.64:0/310144360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d508057300 msgr2=0x55d508063930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:56.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.518+0000 7fc7f87a5b80 1 --2- 172.21.15.64:0/310144360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d508057300 0x55d508063930 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc7d000e970 tx=0x7fc7d0005b40 comp rx=0 tx=0).stop 2023-09-13T03:18:56.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.518+0000 7fc7f87a5b80 1 -- 172.21.15.64:0/310144360 shutdown_connections 2023-09-13T03:18:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.518+0000 7fc7f87a5b80 1 --2- 172.21.15.64:0/310144360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d508057300 0x55d508063930 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.518+0000 7fc7f87a5b80 1 --2- 172.21.15.64:0/310144360 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d50805d1a0 0x55d50805c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.518+0000 7fc7f87a5b80 1 --2- 172.21.15.64:0/310144360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d508057cc0 0x55d50805c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.518+0000 7fc7f87a5b80 1 -- 172.21.15.64:0/310144360 >> 172.21.15.64:0/310144360 conn(0x55d507fcc030 msgr2=0x55d50805b190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:18:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.518+0000 7fc7f87a5b80 1 -- 172.21.15.64:0/310144360 shutdown_connections 2023-09-13T03:18:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.519+0000 7fc7f87a5b80 1 -- 172.21.15.64:0/310144360 wait complete. 2023-09-13T03:18:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.519+0000 7fc7f87a5b80 1 -- start start 2023-09-13T03:18:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.519+0000 7fc7f87a5b80 1 Processor -- start 2023-09-13T03:18:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.520+0000 7fc7f87a5b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d508057300 0x55d508064370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.520+0000 7fc7f87a5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d508057cc0 0x55d5080648b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.520+0000 7fc7f87a5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d50805d1a0 0x55d508064e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.520+0000 7fc7f87a5b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5080c4330 con 0x55d508057300 2023-09-13T03:18:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.520+0000 7fc7f87a5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5080c44a0 con 0x55d508057cc0 2023-09-13T03:18:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.520+0000 7fc7f87a5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5080c4660 con 0x55d50805d1a0 2023-09-13T03:18:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.520+0000 7fc7f87a5b80 10 client.0 _ll_register_callbacks cb 0x7fc7d0098f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:18:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.520+0000 7fc7ddd9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d508057cc0 0x55d5080648b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.520+0000 7fc7de59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d508057300 0x55d508064370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.520+0000 7fc7ddd9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d508057cc0 0x55d5080648b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:39162/0 (socket says 172.21.15.64:39162) 2023-09-13T03:18:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.520+0000 7fc7ddd9b700 1 -- 172.21.15.64:0/3191800375 learned_addr learned my addr 172.21.15.64:0/3191800375 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:18:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.520+0000 7fc7ded9d700 1 --2- 172.21.15.64:0/3191800375 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d50805d1a0 0x55d508064e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.520+0000 7fc7ddd9b700 1 -- 172.21.15.64:0/3191800375 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d50805d1a0 msgr2=0x55d508064e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:18:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.520+0000 7fc7ddd9b700 1 --2- 172.21.15.64:0/3191800375 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d50805d1a0 0x55d508064e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.520+0000 7fc7ddd9b700 1 -- 172.21.15.64:0/3191800375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d508057300 msgr2=0x55d508064370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:18:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.520+0000 7fc7ddd9b700 1 --2- 172.21.15.64:0/3191800375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d508057300 0x55d508064370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.520+0000 7fc7ddd9b700 1 -- 172.21.15.64:0/3191800375 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d507dbec00 con 0x55d508057cc0 2023-09-13T03:18:56.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.520+0000 7fc7de59c700 1 --2- 172.21.15.64:0/3191800375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d508057300 0x55d508064370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:18:56.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.521+0000 7fc7ddd9b700 1 --2- 172.21.15.64:0/3191800375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d508057cc0 0x55d5080648b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4005a80 tx=0x7fc7d400bfc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ac0c3701fbcd62e server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:18:56.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.521+0000 7fc7c5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:18:56.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.521+0000 7fc7c5ffb700 1 -- 172.21.15.64:0/3191800375 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d4031b40 con 0x55d508057cc0 2023-09-13T03:18:56.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.521+0000 7fc7c5ffb700 1 -- 172.21.15.64:0/3191800375 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc7d4031ca0 con 0x55d508057cc0 2023-09-13T03:18:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.521+0000 7fc7f87a5b80 1 -- 172.21.15.64:0/3191800375 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc7d401e040 con 0x55d508057cc0 2023-09-13T03:18:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.521+0000 7fc7c5ffb700 1 -- 172.21.15.64:0/3191800375 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7d4031e60 con 0x55d508057cc0 2023-09-13T03:18:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.521+0000 7fc7f87a5b80 10 client.26597 fetch_fsmap learned FSMap version 46 2023-09-13T03:18:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.521+0000 7fc7f87a5b80 1 -- 172.21.15.64:0/3191800375 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55d507dd4250 con 0x55d508057cc0 2023-09-13T03:18:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.522+0000 7fc7c5ffb700 1 -- 172.21.15.64:0/3191800375 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7d401e040 con 0x55d508057cc0 2023-09-13T03:18:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.522+0000 7fc7c5ffb700 1 -- 172.21.15.64:0/3191800375 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 46) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fc7d402d460 con 0x55d508057cc0 2023-09-13T03:18:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.522+0000 7fc7f87a5b80 10 client.26597 fetch_fsmap finished waiting for FSMap version 46 2023-09-13T03:18:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.522+0000 7fc7f87a5b80 10 client.26597 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:18:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.522+0000 7fc7f87a5b80 20 client.26597 populate_metadata read hostname 'smithi064' 2023-09-13T03:18:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.522+0000 7fc7f87a5b80 1 --2- 172.21.15.64:0/3191800375 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55d5080c5a90 0x55d5080c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:18:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.522+0000 7fc7f87a5b80 4 client.26597 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:18:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.522+0000 7fc7f87a5b80 1 -- 172.21.15.64:0/3191800375 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d507fa28c0 con 0x55d5080c5a90 2023-09-13T03:18:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.522+0000 7fc7de59c700 1 --2- 172.21.15.64:0/3191800375 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55d5080c5a90 0x55d5080c7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:18:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.522+0000 7fc7de59c700 1 --2- 172.21.15.64:0/3191800375 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55d5080c5a90 0x55d5080c7eb0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3442eebc5b8430e5 server_cookie=2c5d47cf8eb6a149 in_seq=0 out_seq=0 2023-09-13T03:18:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.522+0000 7fc7c5ffb700 10 client.26597 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:18:56.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.545+0000 7fc7c5ffb700 1 -- 172.21.15.64:0/3191800375 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d507fa28c0 con 0x55d5080c5a90 2023-09-13T03:18:56.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.545+0000 7fc7c5ffb700 10 client.26597 handle_command_reply: tid=0 2023-09-13T03:18:56.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.545+0000 7fc7f87a5b80 1 client.26597 shutdown 2023-09-13T03:18:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.545+0000 7fc7f87a5b80 1 -- 172.21.15.64:0/3191800375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d508057cc0 msgr2=0x55d5080648b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.545+0000 7fc7f87a5b80 1 --2- 172.21.15.64:0/3191800375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d508057cc0 0x55d5080648b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4005a80 tx=0x7fc7d400bfc0 comp rx=0 tx=0).stop 2023-09-13T03:18:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.545+0000 7fc7f87a5b80 1 -- 172.21.15.64:0/3191800375 shutdown_connections 2023-09-13T03:18:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.545+0000 7fc7f87a5b80 1 --2- 172.21.15.64:0/3191800375 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55d5080c5a90 0x55d5080c7eb0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.545+0000 7fc7f87a5b80 1 --2- 172.21.15.64:0/3191800375 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d50805d1a0 0x55d508064e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.545+0000 7fc7f87a5b80 1 --2- 172.21.15.64:0/3191800375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d508057300 0x55d508064370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.545+0000 7fc7f87a5b80 1 --2- 172.21.15.64:0/3191800375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d508057cc0 0x55d5080648b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.545+0000 7fc7c5ffb700 0 client.26597 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:18:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.545+0000 7fc7f87a5b80 1 -- 172.21.15.64:0/3191800375 >> 172.21.15.64:0/3191800375 conn(0x55d507fcc030 msgr2=0x55d50805c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:18:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.545+0000 7fc7f87a5b80 1 -- 172.21.15.64:0/3191800375 shutdown_connections 2023-09-13T03:18:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.545+0000 7fc7f87a5b80 1 -- 172.21.15.64:0/3191800375 wait complete. 2023-09-13T03:18:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.545+0000 7fc7f87a5b80 20 client.26597 trim_cache size 0 max 16384 2023-09-13T03:18:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:18:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:18:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:18:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:18:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.546+0000 7fc7e4dff700 1 -- 172.21.15.64:0/1493613547 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fc7c80a07b0 msgr2=0x7fc7c80a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.546+0000 7fc7e4dff700 1 --2- 172.21.15.64:0/1493613547 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fc7c80a07b0 0x7fc7c80a2c70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8009fa0 tx=0x7fc7d800b040 comp rx=0 tx=0).stop 2023-09-13T03:18:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.546+0000 7fc7e4dff700 1 -- 172.21.15.64:0/1493613547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7e01005a0 msgr2=0x7fc7e019e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:18:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.546+0000 7fc7e4dff700 1 --2- 172.21.15.64:0/1493613547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7e01005a0 0x7fc7e019e7f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc7d000e9a0 tx=0x7fc7d000ecb0 comp rx=0 tx=0).stop 2023-09-13T03:18:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.546+0000 7fc7e4dff700 1 -- 172.21.15.64:0/1493613547 shutdown_connections 2023-09-13T03:18:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.546+0000 7fc7e4dff700 1 --2- 172.21.15.64:0/1493613547 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fc7c80a07b0 0x7fc7c80a2c70 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.546+0000 7fc7e4dff700 1 --2- 172.21.15.64:0/1493613547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7e01005a0 0x7fc7e019e7f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.546+0000 7fc7e4dff700 1 --2- 172.21.15.64:0/1493613547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7e0106660 0x7fc7e0198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.546+0000 7fc7e4dff700 1 --2- 172.21.15.64:0/1493613547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7e0100f60 0x7fc7e019ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:18:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.546+0000 7fc7e4dff700 1 -- 172.21.15.64:0/1493613547 >> 172.21.15.64:0/1493613547 conn(0x7fc7e00fc050 msgr2=0x7fc7e010b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:18:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.546+0000 7fc7e4dff700 1 -- 172.21.15.64:0/1493613547 shutdown_connections 2023-09-13T03:18:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:18:56.546+0000 7fc7e4dff700 1 -- 172.21.15.64:0/1493613547 wait complete. 2023-09-13T03:18:56.916 INFO:tasks.workunit.client.0.smithi064.stdout: 1 117687 1.90 MB/sec execute 181 sec latency 16.498 ms 2023-09-13T03:18:56.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:18:56 smithi064 ceph-mon[97675]: pgmap v176: 97 pgs: 97 active+clean; 62 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 1.2 MiB/s wr, 101 op/s 2023-09-13T03:18:56.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:18:56 smithi142 ceph-mon[134082]: pgmap v176: 97 pgs: 97 active+clean; 62 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 1.2 MiB/s wr, 101 op/s 2023-09-13T03:18:57.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:18:56 smithi170 ceph-mon[134328]: pgmap v176: 97 pgs: 97 active+clean; 62 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 1.2 MiB/s wr, 101 op/s 2023-09-13T03:18:57.916 INFO:tasks.workunit.client.0.smithi064.stdout: 1 117756 1.89 MB/sec execute 182 sec latency 958.148 ms 2023-09-13T03:18:58.916 INFO:tasks.workunit.client.0.smithi064.stdout: 1 117812 1.88 MB/sec execute 183 sec latency 1566.288 ms 2023-09-13T03:18:58.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:18:58 smithi064 ceph-mon[97675]: pgmap v177: 97 pgs: 97 active+clean; 62 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 979 KiB/s wr, 62 op/s 2023-09-13T03:18:58.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:18:58 smithi142 ceph-mon[134082]: pgmap v177: 97 pgs: 97 active+clean; 62 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 979 KiB/s wr, 62 op/s 2023-09-13T03:18:59.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:18:58 smithi170 ceph-mon[134328]: pgmap v177: 97 pgs: 97 active+clean; 62 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 979 KiB/s wr, 62 op/s 2023-09-13T03:18:59.916 INFO:tasks.workunit.client.0.smithi064.stdout: 1 117812 1.87 MB/sec execute 184 sec latency 1347.198 ms 2023-09-13T03:19:00.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:19:00 smithi170 ceph-mon[134328]: pgmap v178: 97 pgs: 97 active+clean; 62 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 979 KiB/s wr, 62 op/s 2023-09-13T03:19:00.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:19:00 smithi064 ceph-mon[97675]: pgmap v178: 97 pgs: 97 active+clean; 62 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 979 KiB/s wr, 62 op/s 2023-09-13T03:19:00.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:19:00 smithi142 ceph-mon[134082]: pgmap v178: 97 pgs: 97 active+clean; 62 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 979 KiB/s wr, 62 op/s 2023-09-13T03:19:00.916 INFO:tasks.workunit.client.0.smithi064.stdout: 1 118117 1.87 MB/sec execute 185 sec latency 1872.231 ms 2023-09-13T03:19:01.917 INFO:tasks.workunit.client.0.smithi064.stdout: 1 118782 1.88 MB/sec execute 186 sec latency 376.066 ms 2023-09-13T03:19:02.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:19:02 smithi170 ceph-mon[134328]: pgmap v179: 97 pgs: 97 active+clean; 64 MiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2023-09-13T03:19:02.917 INFO:tasks.workunit.client.0.smithi064.stdout: 1 118782 1.87 MB/sec execute 187 sec latency 1376.203 ms 2023-09-13T03:19:02.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:19:02 smithi064 ceph-mon[97675]: pgmap v179: 97 pgs: 97 active+clean; 64 MiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2023-09-13T03:19:02.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:19:02 smithi142 ceph-mon[134082]: pgmap v179: 97 pgs: 97 active+clean; 64 MiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2023-09-13T03:19:03.917 INFO:tasks.workunit.client.0.smithi064.stdout: 1 118782 1.86 MB/sec execute 188 sec latency 2376.349 ms 2023-09-13T03:19:04.917 INFO:tasks.workunit.client.0.smithi064.stdout: 1 118782 1.85 MB/sec execute 189 sec latency 3376.545 ms 2023-09-13T03:19:04.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:19:04 smithi064 ceph-mon[97675]: pgmap v180: 97 pgs: 97 active+clean; 64 MiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 1.1 MiB/s wr, 101 op/s 2023-09-13T03:19:04.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:19:04 smithi142 ceph-mon[134082]: pgmap v180: 97 pgs: 97 active+clean; 64 MiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 1.1 MiB/s wr, 101 op/s 2023-09-13T03:19:05.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:19:04 smithi170 ceph-mon[134328]: pgmap v180: 97 pgs: 97 active+clean; 64 MiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 1.1 MiB/s wr, 101 op/s 2023-09-13T03:19:05.917 INFO:tasks.workunit.client.0.smithi064.stdout: 1 119985 1.87 MB/sec execute 190 sec latency 3807.673 ms 2023-09-13T03:19:06.920 INFO:tasks.workunit.client.0.smithi064.stdout: 1 121233 1.90 MB/sec execute 191 sec latency 13.053 ms 2023-09-13T03:19:06.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:19:06 smithi064 ceph-mon[97675]: pgmap v181: 97 pgs: 97 active+clean; 66 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 1.3 MiB/s wr, 142 op/s 2023-09-13T03:19:06.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:19:06 smithi142 ceph-mon[134082]: pgmap v181: 97 pgs: 97 active+clean; 66 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 1.3 MiB/s wr, 142 op/s 2023-09-13T03:19:07.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:19:06 smithi170 ceph-mon[134328]: pgmap v181: 97 pgs: 97 active+clean; 66 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 1.3 MiB/s wr, 142 op/s 2023-09-13T03:19:07.917 INFO:tasks.workunit.client.0.smithi064.stdout: 1 121424 1.89 MB/sec execute 192 sec latency 872.941 ms 2023-09-13T03:19:08.917 INFO:tasks.workunit.client.0.smithi064.stdout: 1 122043 1.91 MB/sec execute 193 sec latency 1480.699 ms 2023-09-13T03:19:08.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:19:08 smithi064 ceph-mon[97675]: pgmap v182: 97 pgs: 97 active+clean; 66 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 438 KiB/s wr, 90 op/s 2023-09-13T03:19:08.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:19:08 smithi142 ceph-mon[134082]: pgmap v182: 97 pgs: 97 active+clean; 66 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 438 KiB/s wr, 90 op/s 2023-09-13T03:19:09.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:19:08 smithi170 ceph-mon[134328]: pgmap v182: 97 pgs: 97 active+clean; 66 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 438 KiB/s wr, 90 op/s 2023-09-13T03:19:09.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.702+0000 7f1164bff700 1 -- 172.21.15.64:0/1536800571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1160100f60 msgr2=0x7f116010d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:19:09.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.702+0000 7f1164bff700 1 --2- 172.21.15.64:0/1536800571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1160100f60 0x7f116010d420 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f115800b0d0 tx=0x7f115800b3e0 comp rx=0 tx=0).stop 2023-09-13T03:19:09.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.702+0000 7f1164bff700 1 -- 172.21.15.64:0/1536800571 shutdown_connections 2023-09-13T03:19:09.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.702+0000 7f1164bff700 1 --2- 172.21.15.64:0/1536800571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1160106660 0x7f1160106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.702+0000 7f1164bff700 1 --2- 172.21.15.64:0/1536800571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1160100f60 0x7f116010d420 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.702+0000 7f1164bff700 1 --2- 172.21.15.64:0/1536800571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11601005a0 0x7f1160100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.702+0000 7f1164bff700 1 -- 172.21.15.64:0/1536800571 >> 172.21.15.64:0/1536800571 conn(0x7f11600fc050 msgr2=0x7f11600fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:19:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.702+0000 7f1164bff700 1 -- 172.21.15.64:0/1536800571 shutdown_connections 2023-09-13T03:19:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.702+0000 7f1164bff700 1 -- 172.21.15.64:0/1536800571 wait complete. 2023-09-13T03:19:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.704+0000 7f1164bff700 1 Processor -- start 2023-09-13T03:19:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.704+0000 7f1164bff700 1 -- start start 2023-09-13T03:19:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.704+0000 7f1164bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11601005a0 0x7f116019e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.704+0000 7f1164bff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1160100f60 0x7f116019ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.704+0000 7f1164bff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1160106660 0x7f1160198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.704+0000 7f1164bff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1160198e00 con 0x7f1160106660 2023-09-13T03:19:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.704+0000 7f1164bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1160198f70 con 0x7f11601005a0 2023-09-13T03:19:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.704+0000 7f1164bff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1160199130 con 0x7f1160100f60 2023-09-13T03:19:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.705+0000 7f115e59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11601005a0 0x7f116019e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.705+0000 7f115e59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11601005a0 0x7f116019e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:38758/0 (socket says 172.21.15.64:38758) 2023-09-13T03:19:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.705+0000 7f115e59c700 1 -- 172.21.15.64:0/142451409 learned_addr learned my addr 172.21.15.64:0/142451409 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:19:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.705+0000 7f115ed9d700 1 --2- 172.21.15.64:0/142451409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1160106660 0x7f1160198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.705+0000 7f115dd9b700 1 --2- 172.21.15.64:0/142451409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1160100f60 0x7f116019ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.705+0000 7f115e59c700 1 -- 172.21.15.64:0/142451409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1160100f60 msgr2=0x7f116019ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:19:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.705+0000 7f115e59c700 1 --2- 172.21.15.64:0/142451409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1160100f60 0x7f116019ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.705+0000 7f115e59c700 1 -- 172.21.15.64:0/142451409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1160106660 msgr2=0x7f1160198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:19:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.705+0000 7f115e59c700 1 --2- 172.21.15.64:0/142451409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1160106660 0x7f1160198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.705+0000 7f115e59c700 1 -- 172.21.15.64:0/142451409 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1158009d00 con 0x7f11601005a0 2023-09-13T03:19:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.705+0000 7f115dd9b700 1 --2- 172.21.15.64:0/142451409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1160100f60 0x7f116019ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:19:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.705+0000 7f115ed9d700 1 --2- 172.21.15.64:0/142451409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1160106660 0x7f1160198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:19:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.706+0000 7f115e59c700 1 --2- 172.21.15.64:0/142451409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11601005a0 0x7f116019e930 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f115000e9a0 tx=0x7f115000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:19:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.706+0000 7f114f7fe700 1 -- 172.21.15.64:0/142451409 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1150009790 con 0x7f11601005a0 2023-09-13T03:19:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.706+0000 7f1164bff700 1 -- 172.21.15.64:0/142451409 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1160199410 con 0x7f11601005a0 2023-09-13T03:19:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.706+0000 7f114f7fe700 1 -- 172.21.15.64:0/142451409 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f115000cbd0 con 0x7f11601005a0 2023-09-13T03:19:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.706+0000 7f114f7fe700 1 -- 172.21.15.64:0/142451409 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1150018680 con 0x7f11601005a0 2023-09-13T03:19:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.706+0000 7f1164bff700 1 -- 172.21.15.64:0/142451409 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11601a5920 con 0x7f11601005a0 2023-09-13T03:19:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.709+0000 7f114f7fe700 1 -- 172.21.15.64:0/142451409 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f115000cd40 con 0x7f11601005a0 2023-09-13T03:19:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.711+0000 7f1178564b80 1 Processor -- start 2023-09-13T03:19:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.711+0000 7f1178564b80 1 -- start start 2023-09-13T03:19:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.711+0000 7f1178564b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f4a4d81a0 0x556f4a4d8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.711+0000 7f1178564b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f4a4d2300 0x556f4a4d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.711+0000 7f1178564b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f4a4d2cc0 0x556f4a4def30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.711+0000 7f1178564b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f4a42b6d0 con 0x556f4a4d2300 2023-09-13T03:19:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.711+0000 7f1178564b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f4a428ff0 con 0x556f4a4d81a0 2023-09-13T03:19:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.711+0000 7f1178564b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f4a4248e0 con 0x556f4a4d2cc0 2023-09-13T03:19:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.712+0000 7f114f7fe700 1 --2- 172.21.15.64:0/142451409 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f11480a0830 0x7f11480a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.712+0000 7f115e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f4a4d2300 0x556f4a4d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.712+0000 7f115e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f4a4d2300 0x556f4a4d2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53204/0 (socket says 172.21.15.64:53204) 2023-09-13T03:19:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.712+0000 7f115e59c700 1 -- 172.21.15.64:0/58358530 learned_addr learned my addr 172.21.15.64:0/58358530 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:19:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.712+0000 7f115dd9b700 1 --2- 172.21.15.64:0/58358530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f4a4d2cc0 0x556f4a4def30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.712+0000 7f114f7fe700 1 -- 172.21.15.64:0/142451409 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f1150014070 con 0x7f11601005a0 2023-09-13T03:19:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.712+0000 7f115ed9d700 1 --2- 172.21.15.64:0/58358530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f4a4d81a0 0x556f4a4d8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.712+0000 7f115ed9d700 1 --2- 172.21.15.64:0/142451409 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f11480a0830 0x7f11480a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.712+0000 7f115e59c700 1 -- 172.21.15.64:0/58358530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f4a4d2cc0 msgr2=0x556f4a4def30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:19:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.712+0000 7f115e59c700 1 --2- 172.21.15.64:0/58358530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f4a4d2cc0 0x556f4a4def30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.712+0000 7f115e59c700 1 -- 172.21.15.64:0/58358530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f4a4d81a0 msgr2=0x556f4a4d8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:19:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.712+0000 7f115e59c700 1 --2- 172.21.15.64:0/58358530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f4a4d81a0 0x556f4a4d8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.712+0000 7f115e59c700 1 -- 172.21.15.64:0/58358530 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f4a250d80 con 0x556f4a4d2300 2023-09-13T03:19:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.712+0000 7f115dd9b700 1 --2- 172.21.15.64:0/58358530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f4a4d2cc0 0x556f4a4def30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:19:09.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.713+0000 7f115e59c700 1 --2- 172.21.15.64:0/58358530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f4a4d2300 0x556f4a4d2780 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f115000c5b0 tx=0x7f1150095400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c60fb3b3c9822809 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:19:09.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.713+0000 7f114cff9700 1 -- 172.21.15.64:0/58358530 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1150099dc0 con 0x556f4a4d2300 2023-09-13T03:19:09.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.713+0000 7f115ed9d700 1 --2- 172.21.15.64:0/142451409 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f11480a0830 0x7f11480a2cf0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1158009cd0 tx=0x7f115800b040 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:19:09.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.713+0000 7f114cff9700 1 -- 172.21.15.64:0/58358530 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11500996c0 con 0x556f4a4d2300 2023-09-13T03:19:09.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.713+0000 7f114cff9700 1 -- 172.21.15.64:0/58358530 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1150099b60 con 0x556f4a4d2300 2023-09-13T03:19:09.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.713+0000 7f1178564b80 1 -- 172.21.15.64:0/58358530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f4a4d2300 msgr2=0x556f4a4d2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:19:09.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.713+0000 7f1178564b80 1 --2- 172.21.15.64:0/58358530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f4a4d2300 0x556f4a4d2780 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f115000c5b0 tx=0x7f1150095400 comp rx=0 tx=0).stop 2023-09-13T03:19:09.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.714+0000 7f1178564b80 1 -- 172.21.15.64:0/58358530 shutdown_connections 2023-09-13T03:19:09.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.714+0000 7f1178564b80 1 --2- 172.21.15.64:0/58358530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f4a4d81a0 0x556f4a4d8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:09.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.714+0000 7f1178564b80 1 --2- 172.21.15.64:0/58358530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f4a4d2cc0 0x556f4a4def30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:09.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.714+0000 7f1178564b80 1 --2- 172.21.15.64:0/58358530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f4a4d2300 0x556f4a4d2780 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:09.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.714+0000 7f1178564b80 1 -- 172.21.15.64:0/58358530 >> 172.21.15.64:0/58358530 conn(0x556f4a447030 msgr2=0x556f4a4cd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:19:09.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.714+0000 7f1178564b80 1 -- 172.21.15.64:0/58358530 shutdown_connections 2023-09-13T03:19:09.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.714+0000 7f1178564b80 1 -- 172.21.15.64:0/58358530 wait complete. 2023-09-13T03:19:09.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.714+0000 7f1178564b80 1 -- start start 2023-09-13T03:19:09.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.714+0000 7f1178564b80 1 Processor -- start 2023-09-13T03:19:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.715+0000 7f1178564b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f4a4d2300 0x556f4a4deda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.715+0000 7f1178564b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f4a4d2cc0 0x556f4a4dc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.715+0000 7f1178564b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f4a4d81a0 0x556f4a4dc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.715+0000 7f1178564b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f4a3c2b10 con 0x556f4a4d2cc0 2023-09-13T03:19:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.715+0000 7f1178564b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f4a3c2810 con 0x556f4a4d81a0 2023-09-13T03:19:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.715+0000 7f1178564b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f4a2a68c0 con 0x556f4a4d2300 2023-09-13T03:19:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.715+0000 7f1178564b80 10 client.0 _ll_register_callbacks cb 0x556f4a30ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:19:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.715+0000 7f115dd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f4a4d2cc0 0x556f4a4dc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.715+0000 7f115e59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f4a4d2300 0x556f4a4deda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.715+0000 7f115e59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f4a4d2300 0x556f4a4deda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:42216/0 (socket says 172.21.15.64:42216) 2023-09-13T03:19:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.715+0000 7f115ed9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f4a4d81a0 0x556f4a4dc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.715+0000 7f115e59c700 1 -- 172.21.15.64:0/2807674445 learned_addr learned my addr 172.21.15.64:0/2807674445 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:19:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.716+0000 7f115e59c700 1 -- 172.21.15.64:0/2807674445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f4a4d81a0 msgr2=0x556f4a4dc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:19:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.716+0000 7f115e59c700 1 --2- 172.21.15.64:0/2807674445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f4a4d81a0 0x556f4a4dc6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.716+0000 7f115e59c700 1 -- 172.21.15.64:0/2807674445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f4a4d2cc0 msgr2=0x556f4a4dc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:19:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.716+0000 7f115e59c700 1 --2- 172.21.15.64:0/2807674445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f4a4d2cc0 0x556f4a4dc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.716+0000 7f115e59c700 1 -- 172.21.15.64:0/2807674445 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11540098f0 con 0x556f4a4d2300 2023-09-13T03:19:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.716+0000 7f115dd9b700 1 --2- 172.21.15.64:0/2807674445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f4a4d2cc0 0x556f4a4dc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:19:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.716+0000 7f115e59c700 1 --2- 172.21.15.64:0/2807674445 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f4a4d2300 0x556f4a4deda0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f11500cc310 tx=0x7f1150098de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab008fb437b562a0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:19:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.716+0000 7f1145ffb700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:19:09.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.716+0000 7f1145ffb700 1 -- 172.21.15.64:0/2807674445 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11500a1730 con 0x556f4a4d2300 2023-09-13T03:19:09.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.716+0000 7f1145ffb700 1 -- 172.21.15.64:0/2807674445 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f115009d4f0 con 0x556f4a4d2300 2023-09-13T03:19:09.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.716+0000 7f1178564b80 1 -- 172.21.15.64:0/2807674445 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556f4a37c490 con 0x556f4a4d2300 2023-09-13T03:19:09.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.716+0000 7f1145ffb700 1 -- 172.21.15.64:0/2807674445 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1150097a60 con 0x556f4a4d2300 2023-09-13T03:19:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.716+0000 7f1178564b80 10 client.26599 fetch_fsmap learned FSMap version 46 2023-09-13T03:19:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.716+0000 7f1178564b80 1 -- 172.21.15.64:0/2807674445 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7f11601a5920 con 0x556f4a4d2300 2023-09-13T03:19:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.717+0000 7f1145ffb700 1 -- 172.21.15.64:0/2807674445 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556f4a37c490 con 0x556f4a4d2300 2023-09-13T03:19:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.717+0000 7f1145ffb700 1 -- 172.21.15.64:0/2807674445 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 46) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f11500840d0 con 0x556f4a4d2300 2023-09-13T03:19:09.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.717+0000 7f1178564b80 10 client.26599 fetch_fsmap finished waiting for FSMap version 46 2023-09-13T03:19:09.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.717+0000 7f1178564b80 10 client.26599 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:19:09.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.717+0000 7f1178564b80 20 client.26599 populate_metadata read hostname 'smithi064' 2023-09-13T03:19:09.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.717+0000 7f1178564b80 1 --2- 172.21.15.64:0/2807674445 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x556f4a4ec050 0x556f4a4ee470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:09.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.717+0000 7f1178564b80 4 client.26599 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:19:09.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.717+0000 7f1178564b80 1 -- 172.21.15.64:0/2807674445 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556f4a439840 con 0x556f4a4ec050 2023-09-13T03:19:09.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.717+0000 7f115dd9b700 1 --2- 172.21.15.64:0/2807674445 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x556f4a4ec050 0x556f4a4ee470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:09.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.718+0000 7f1145ffb700 10 client.26599 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:19:09.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.718+0000 7f115dd9b700 1 --2- 172.21.15.64:0/2807674445 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x556f4a4ec050 0x556f4a4ee470 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fae51ef3c52546a server_cookie=b8b30b6b829c1a38 in_seq=0 out_seq=0 2023-09-13T03:19:09.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.718+0000 7f1145ffb700 1 -- 172.21.15.64:0/2807674445 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556f4a439840 con 0x556f4a4ec050 2023-09-13T03:19:09.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.718+0000 7f1145ffb700 10 client.26599 handle_command_reply: tid=0 2023-09-13T03:19:09.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.719+0000 7f1178564b80 1 client.26599 shutdown 2023-09-13T03:19:09.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.719+0000 7f1178564b80 1 -- 172.21.15.64:0/2807674445 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f4a4d2300 msgr2=0x556f4a4deda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:19:09.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.719+0000 7f1178564b80 1 --2- 172.21.15.64:0/2807674445 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f4a4d2300 0x556f4a4deda0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f11500cc310 tx=0x7f1150098de0 comp rx=0 tx=0).stop 2023-09-13T03:19:09.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.719+0000 7f1178564b80 1 -- 172.21.15.64:0/2807674445 shutdown_connections 2023-09-13T03:19:09.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.719+0000 7f1178564b80 1 --2- 172.21.15.64:0/2807674445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f4a4d81a0 0x556f4a4dc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:09.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.719+0000 7f1178564b80 1 --2- 172.21.15.64:0/2807674445 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x556f4a4ec050 0x556f4a4ee470 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:09.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.719+0000 7f1178564b80 1 --2- 172.21.15.64:0/2807674445 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f4a4d2300 0x556f4a4deda0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:09.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.719+0000 7f1178564b80 1 --2- 172.21.15.64:0/2807674445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f4a4d2cc0 0x556f4a4dc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:09.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.719+0000 7f1178564b80 1 -- 172.21.15.64:0/2807674445 >> 172.21.15.64:0/2807674445 conn(0x556f4a447030 msgr2=0x556f4a4cc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:19:09.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.719+0000 7f1145ffb700 0 client.26599 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:19:09.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.719+0000 7f1178564b80 1 -- 172.21.15.64:0/2807674445 shutdown_connections 2023-09-13T03:19:09.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.719+0000 7f1178564b80 1 -- 172.21.15.64:0/2807674445 wait complete. 2023-09-13T03:19:09.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.719+0000 7f1178564b80 20 client.26599 trim_cache size 0 max 16384 2023-09-13T03:19:09.746 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.734+0000 7f1178564b80 1 Processor -- start 2023-09-13T03:19:09.746 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.734+0000 7f1178564b80 1 -- start start 2023-09-13T03:19:09.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.735+0000 7f1178564b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f4a4d2300 0x556f4a4de8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:09.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.735+0000 7f1178564b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f4a4d2cc0 0x556f4a4d7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:09.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.735+0000 7f1178564b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f4a4d81a0 0x556f4a4d7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.735+0000 7f1178564b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f4a4dd150 con 0x556f4a4d2300 2023-09-13T03:19:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.735+0000 7f1178564b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f4a4d6d30 con 0x556f4a4d81a0 2023-09-13T03:19:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.735+0000 7f1178564b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f4a4eead0 con 0x556f4a4d2cc0 2023-09-13T03:19:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.735+0000 7f115e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f4a4d2300 0x556f4a4de8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.735+0000 7f115e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f4a4d2300 0x556f4a4de8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53228/0 (socket says 172.21.15.64:53228) 2023-09-13T03:19:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.735+0000 7f115e59c700 1 -- 172.21.15.64:0/2264546392 learned_addr learned my addr 172.21.15.64:0/2264546392 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:19:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.735+0000 7f115ed9d700 1 --2- 172.21.15.64:0/2264546392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f4a4d81a0 0x556f4a4d7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.735+0000 7f115dd9b700 1 --2- 172.21.15.64:0/2264546392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f4a4d2cc0 0x556f4a4d7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.735+0000 7f115e59c700 1 -- 172.21.15.64:0/2264546392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f4a4d2cc0 msgr2=0x556f4a4d7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:19:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.735+0000 7f115e59c700 1 --2- 172.21.15.64:0/2264546392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f4a4d2cc0 0x556f4a4d7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.735+0000 7f115e59c700 1 -- 172.21.15.64:0/2264546392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f4a4d81a0 msgr2=0x556f4a4d7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:19:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.735+0000 7f115e59c700 1 --2- 172.21.15.64:0/2264546392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f4a4d81a0 0x556f4a4d7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.735+0000 7f115e59c700 1 -- 172.21.15.64:0/2264546392 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f4a239c00 con 0x556f4a4d2300 2023-09-13T03:19:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.736+0000 7f115e59c700 1 --2- 172.21.15.64:0/2264546392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f4a4d2300 0x556f4a4de8d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f1150096130 tx=0x7f1150095400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b1fbe6c359efec0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:19:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.736+0000 7f114d7fa700 1 -- 172.21.15.64:0/2264546392 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1150098e50 con 0x556f4a4d2300 2023-09-13T03:19:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.736+0000 7f114d7fa700 1 -- 172.21.15.64:0/2264546392 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1150098710 con 0x556f4a4d2300 2023-09-13T03:19:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.736+0000 7f114d7fa700 1 -- 172.21.15.64:0/2264546392 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11500840d0 con 0x556f4a4d2300 2023-09-13T03:19:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.737+0000 7f1178564b80 1 -- 172.21.15.64:0/2264546392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f4a4d2300 msgr2=0x556f4a4de8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:19:09.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.737+0000 7f1178564b80 1 --2- 172.21.15.64:0/2264546392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f4a4d2300 0x556f4a4de8d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f1150096130 tx=0x7f1150095400 comp rx=0 tx=0).stop 2023-09-13T03:19:09.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.737+0000 7f1178564b80 1 -- 172.21.15.64:0/2264546392 shutdown_connections 2023-09-13T03:19:09.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.737+0000 7f1178564b80 1 --2- 172.21.15.64:0/2264546392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f4a4d81a0 0x556f4a4d7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:09.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.737+0000 7f1178564b80 1 --2- 172.21.15.64:0/2264546392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f4a4d2300 0x556f4a4de8d0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:09.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.737+0000 7f1178564b80 1 --2- 172.21.15.64:0/2264546392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f4a4d2cc0 0x556f4a4d7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:09.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.737+0000 7f1178564b80 1 -- 172.21.15.64:0/2264546392 >> 172.21.15.64:0/2264546392 conn(0x556f4a447030 msgr2=0x556f4a4d6250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:19:09.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.737+0000 7f1178564b80 1 -- 172.21.15.64:0/2264546392 shutdown_connections 2023-09-13T03:19:09.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.737+0000 7f1178564b80 1 -- 172.21.15.64:0/2264546392 wait complete. 2023-09-13T03:19:09.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.737+0000 7f1178564b80 1 -- start start 2023-09-13T03:19:09.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.737+0000 7f1178564b80 1 Processor -- start 2023-09-13T03:19:09.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.738+0000 7f1178564b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f4a4d2300 0x556f4a4e1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:09.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.738+0000 7f1178564b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f4a4d2cc0 0x556f4a4e1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:09.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.738+0000 7f1178564b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f4a4d81a0 0x556f4a4e1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:09.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.738+0000 7f1178564b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f4a53ef70 con 0x556f4a4d2cc0 2023-09-13T03:19:09.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.738+0000 7f1178564b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f4a53f0e0 con 0x556f4a4d81a0 2023-09-13T03:19:09.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.738+0000 7f1178564b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f4a53f2a0 con 0x556f4a4d2300 2023-09-13T03:19:09.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.738+0000 7f1178564b80 10 client.0 _ll_register_callbacks cb 0x7f1154009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:19:09.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.738+0000 7f115e59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f4a4d2300 0x556f4a4e1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:09.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.738+0000 7f115dd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f4a4d2cc0 0x556f4a4e1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:09.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.738+0000 7f115e59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f4a4d2300 0x556f4a4e1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:42246/0 (socket says 172.21.15.64:42246) 2023-09-13T03:19:09.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.738+0000 7f115e59c700 1 -- 172.21.15.64:0/1902550371 learned_addr learned my addr 172.21.15.64:0/1902550371 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:19:09.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.738+0000 7f115ed9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f4a4d81a0 0x556f4a4e1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:09.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.738+0000 7f115e59c700 1 -- 172.21.15.64:0/1902550371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f4a4d81a0 msgr2=0x556f4a4e1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:19:09.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.738+0000 7f115e59c700 1 --2- 172.21.15.64:0/1902550371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f4a4d81a0 0x556f4a4e1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:09.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.738+0000 7f115e59c700 1 -- 172.21.15.64:0/1902550371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f4a4d2cc0 msgr2=0x556f4a4e1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:19:09.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.738+0000 7f115e59c700 1 --2- 172.21.15.64:0/1902550371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f4a4d2cc0 0x556f4a4e1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:09.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.739+0000 7f115e59c700 1 -- 172.21.15.64:0/1902550371 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11601a5920 con 0x556f4a4d2300 2023-09-13T03:19:09.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.739+0000 7f115e59c700 1 --2- 172.21.15.64:0/1902550371 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f4a4d2300 0x556f4a4e1510 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1150095bc0 tx=0x7f1150099dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64f4d7fef94f9c83 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:19:09.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.739+0000 7f1145ffb700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:19:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.739+0000 7f1145ffb700 1 -- 172.21.15.64:0/1902550371 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115009d4f0 con 0x556f4a4d2300 2023-09-13T03:19:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.739+0000 7f1178564b80 1 -- 172.21.15.64:0/1902550371 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f11500996c0 con 0x556f4a4d2300 2023-09-13T03:19:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.739+0000 7f1145ffb700 1 -- 172.21.15.64:0/1902550371 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f11500a1730 con 0x556f4a4d2300 2023-09-13T03:19:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.739+0000 7f1145ffb700 1 -- 172.21.15.64:0/1902550371 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1150098dc0 con 0x556f4a4d2300 2023-09-13T03:19:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.739+0000 7f1178564b80 10 client.26605 fetch_fsmap learned FSMap version 46 2023-09-13T03:19:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.739+0000 7f1178564b80 1 -- 172.21.15.64:0/1902550371 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x556f4a2771f0 con 0x556f4a4d2300 2023-09-13T03:19:09.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.739+0000 7f1145ffb700 1 -- 172.21.15.64:0/1902550371 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11500996c0 con 0x556f4a4d2300 2023-09-13T03:19:09.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.740+0000 7f1145ffb700 1 -- 172.21.15.64:0/1902550371 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 46) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f115009e430 con 0x556f4a4d2300 2023-09-13T03:19:09.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.740+0000 7f1178564b80 10 client.26605 fetch_fsmap finished waiting for FSMap version 46 2023-09-13T03:19:09.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.740+0000 7f1178564b80 10 client.26605 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:19:09.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.740+0000 7f1178564b80 20 client.26605 populate_metadata read hostname 'smithi064' 2023-09-13T03:19:09.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.740+0000 7f1178564b80 1 --2- 172.21.15.64:0/1902550371 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x556f4a5405a0 0x556f4a5429c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:09.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.740+0000 7f1178564b80 4 client.26605 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:19:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.740+0000 7f1178564b80 1 -- 172.21.15.64:0/1902550371 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556f4a41d8c0 con 0x556f4a5405a0 2023-09-13T03:19:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.740+0000 7f115dd9b700 1 --2- 172.21.15.64:0/1902550371 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x556f4a5405a0 0x556f4a5429c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.740+0000 7f115dd9b700 1 --2- 172.21.15.64:0/1902550371 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x556f4a5405a0 0x556f4a5429c0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf8e3275c2f05d80 server_cookie=63b76d79300c3e81 in_seq=0 out_seq=0 2023-09-13T03:19:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.740+0000 7f1145ffb700 10 client.26605 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:19:09.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.760+0000 7f1145ffb700 1 -- 172.21.15.64:0/1902550371 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556f4a41d8c0 con 0x556f4a5405a0 2023-09-13T03:19:09.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.760+0000 7f1145ffb700 10 client.26605 handle_command_reply: tid=0 2023-09-13T03:19:09.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.760+0000 7f1178564b80 1 client.26605 shutdown 2023-09-13T03:19:09.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.760+0000 7f1178564b80 1 -- 172.21.15.64:0/1902550371 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f4a4d2300 msgr2=0x556f4a4e1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:19:09.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.760+0000 7f1178564b80 1 --2- 172.21.15.64:0/1902550371 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f4a4d2300 0x556f4a4e1510 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1150095bc0 tx=0x7f1150099dc0 comp rx=0 tx=0).stop 2023-09-13T03:19:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.760+0000 7f1178564b80 1 -- 172.21.15.64:0/1902550371 shutdown_connections 2023-09-13T03:19:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.760+0000 7f1178564b80 1 --2- 172.21.15.64:0/1902550371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f4a4d81a0 0x556f4a4e1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.760+0000 7f1178564b80 1 --2- 172.21.15.64:0/1902550371 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x556f4a5405a0 0x556f4a5429c0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.760+0000 7f1178564b80 1 --2- 172.21.15.64:0/1902550371 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f4a4d2300 0x556f4a4e1510 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.760+0000 7f1178564b80 1 --2- 172.21.15.64:0/1902550371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f4a4d2cc0 0x556f4a4e1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.760+0000 7f1178564b80 1 -- 172.21.15.64:0/1902550371 >> 172.21.15.64:0/1902550371 conn(0x556f4a447030 msgr2=0x556f4a4d6250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:19:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.760+0000 7f1145ffb700 0 client.26605 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:19:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.760+0000 7f1178564b80 1 -- 172.21.15.64:0/1902550371 shutdown_connections 2023-09-13T03:19:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.760+0000 7f1178564b80 1 -- 172.21.15.64:0/1902550371 wait complete. 2023-09-13T03:19:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.760+0000 7f1178564b80 20 client.26605 trim_cache size 0 max 16384 2023-09-13T03:19:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:19:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:19:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:19:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:19:09.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.761+0000 7f1164bff700 1 -- 172.21.15.64:0/142451409 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f11480a0830 msgr2=0x7f11480a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:19:09.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.761+0000 7f1164bff700 1 --2- 172.21.15.64:0/142451409 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f11480a0830 0x7f11480a2cf0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1158009cd0 tx=0x7f115800b040 comp rx=0 tx=0).stop 2023-09-13T03:19:09.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.761+0000 7f1164bff700 1 -- 172.21.15.64:0/142451409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11601005a0 msgr2=0x7f116019e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:19:09.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.761+0000 7f1164bff700 1 --2- 172.21.15.64:0/142451409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11601005a0 0x7f116019e930 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f115000e9a0 tx=0x7f115000ecb0 comp rx=0 tx=0).stop 2023-09-13T03:19:09.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.761+0000 7f1164bff700 1 -- 172.21.15.64:0/142451409 shutdown_connections 2023-09-13T03:19:09.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.761+0000 7f1164bff700 1 --2- 172.21.15.64:0/142451409 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f11480a0830 0x7f11480a2cf0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.761+0000 7f1164bff700 1 --2- 172.21.15.64:0/142451409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11601005a0 0x7f116019e930 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.761+0000 7f1164bff700 1 --2- 172.21.15.64:0/142451409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1160106660 0x7f1160198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.761+0000 7f1164bff700 1 --2- 172.21.15.64:0/142451409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1160100f60 0x7f116019ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.761+0000 7f1164bff700 1 -- 172.21.15.64:0/142451409 >> 172.21.15.64:0/142451409 conn(0x7f11600fc050 msgr2=0x7f116010b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:19:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.761+0000 7f1164bff700 1 -- 172.21.15.64:0/142451409 shutdown_connections 2023-09-13T03:19:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:09.761+0000 7f1164bff700 1 -- 172.21.15.64:0/142451409 wait complete. 2023-09-13T03:19:09.918 INFO:tasks.workunit.client.0.smithi064.stdout: 1 122325 1.90 MB/sec execute 194 sec latency 585.744 ms 2023-09-13T03:19:10.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:19:10 smithi170 ceph-mon[134328]: pgmap v183: 97 pgs: 97 active+clean; 66 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 438 KiB/s wr, 90 op/s 2023-09-13T03:19:10.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:19:10 smithi064 ceph-mon[97675]: pgmap v183: 97 pgs: 97 active+clean; 66 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 438 KiB/s wr, 90 op/s 2023-09-13T03:19:10.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:19:10 smithi142 ceph-mon[134082]: pgmap v183: 97 pgs: 97 active+clean; 66 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 438 KiB/s wr, 90 op/s 2023-09-13T03:19:10.918 INFO:tasks.workunit.client.0.smithi064.stdout: 1 123134 1.90 MB/sec execute 195 sec latency 1018.025 ms 2023-09-13T03:19:11.918 INFO:tasks.workunit.client.0.smithi064.stdout: 1 124644 1.95 MB/sec execute 196 sec latency 14.827 ms 2023-09-13T03:19:12.918 INFO:tasks.workunit.client.0.smithi064.stdout: 1 124849 1.94 MB/sec execute 197 sec latency 835.960 ms 2023-09-13T03:19:12.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:19:12 smithi064 ceph-mon[97675]: pgmap v184: 97 pgs: 97 active+clean; 65 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 755 KiB/s wr, 139 op/s 2023-09-13T03:19:12.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:19:12 smithi142 ceph-mon[134082]: pgmap v184: 97 pgs: 97 active+clean; 65 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 755 KiB/s wr, 139 op/s 2023-09-13T03:19:13.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:19:12 smithi170 ceph-mon[134328]: pgmap v184: 97 pgs: 97 active+clean; 65 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 755 KiB/s wr, 139 op/s 2023-09-13T03:19:13.918 INFO:tasks.workunit.client.0.smithi064.stdout: 1 124973 1.93 MB/sec execute 198 sec latency 1447.435 ms 2023-09-13T03:19:14.918 INFO:tasks.workunit.client.0.smithi064.stdout: 1 124973 1.92 MB/sec execute 199 sec latency 1288.919 ms 2023-09-13T03:19:14.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:19:14 smithi064 ceph-mon[97675]: pgmap v185: 97 pgs: 97 active+clean; 65 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 514 KiB/s wr, 89 op/s 2023-09-13T03:19:14.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:19:14 smithi142 ceph-mon[134082]: pgmap v185: 97 pgs: 97 active+clean; 65 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 514 KiB/s wr, 89 op/s 2023-09-13T03:19:15.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:19:14 smithi170 ceph-mon[134328]: pgmap v185: 97 pgs: 97 active+clean; 65 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 514 KiB/s wr, 89 op/s 2023-09-13T03:19:15.918 INFO:tasks.workunit.client.0.smithi064.stdout: 1 124973 1.91 MB/sec execute 200 sec latency 2289.053 ms 2023-09-13T03:19:16.919 INFO:tasks.workunit.client.0.smithi064.stdout: 1 124973 1.90 MB/sec execute 201 sec latency 3289.171 ms 2023-09-13T03:19:16.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:19:16 smithi142 ceph-mon[134082]: pgmap v186: 97 pgs: 97 active+clean; 67 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 743 KiB/s wr, 124 op/s 2023-09-13T03:19:17.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:19:16 smithi170 ceph-mon[134328]: pgmap v186: 97 pgs: 97 active+clean; 67 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 743 KiB/s wr, 124 op/s 2023-09-13T03:19:17.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:19:16 smithi064 ceph-mon[97675]: pgmap v186: 97 pgs: 97 active+clean; 67 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 743 KiB/s wr, 124 op/s 2023-09-13T03:19:17.918 INFO:tasks.workunit.client.0.smithi064.stdout: 1 125290 1.90 MB/sec execute 202 sec latency 4091.520 ms 2023-09-13T03:19:18.918 INFO:tasks.workunit.client.0.smithi064.stdout: 1 125878 1.91 MB/sec execute 203 sec latency 622.264 ms 2023-09-13T03:19:18.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:19:18 smithi142 ceph-mon[134082]: pgmap v187: 97 pgs: 97 active+clean; 67 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 545 KiB/s wr, 83 op/s 2023-09-13T03:19:19.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:19:18 smithi170 ceph-mon[134328]: pgmap v187: 97 pgs: 97 active+clean; 67 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 545 KiB/s wr, 83 op/s 2023-09-13T03:19:19.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:19:18 smithi064 ceph-mon[97675]: pgmap v187: 97 pgs: 97 active+clean; 67 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 545 KiB/s wr, 83 op/s 2023-09-13T03:19:19.919 INFO:tasks.workunit.client.0.smithi064.stdout: 1 125878 1.90 MB/sec execute 204 sec latency 1622.387 ms 2023-09-13T03:19:20.659 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:19:20 smithi170 ceph-mon[134328]: pgmap v188: 97 pgs: 97 active+clean; 67 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 545 KiB/s wr, 83 op/s 2023-09-13T03:19:20.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:19:20 smithi064 ceph-mon[97675]: pgmap v188: 97 pgs: 97 active+clean; 67 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 545 KiB/s wr, 83 op/s 2023-09-13T03:19:20.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:19:20 smithi142 ceph-mon[134082]: pgmap v188: 97 pgs: 97 active+clean; 67 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 545 KiB/s wr, 83 op/s 2023-09-13T03:19:20.919 INFO:tasks.workunit.client.0.smithi064.stdout: 1 127086 1.92 MB/sec execute 205 sec latency 2058.075 ms 2023-09-13T03:19:21.919 INFO:tasks.workunit.client.0.smithi064.stdout: 1 128316 1.95 MB/sec execute 206 sec latency 58.460 ms 2023-09-13T03:19:22.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:19:22 smithi064 ceph-mon[97675]: pgmap v189: 97 pgs: 97 active+clean; 68 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 135 op/s 2023-09-13T03:19:22.919 INFO:tasks.workunit.client.0.smithi064.stdout: 1 128393 1.94 MB/sec execute 207 sec latency 936.922 ms 2023-09-13T03:19:22.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.921+0000 7f0c4fdff700 1 -- 172.21.15.64:0/482496732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c480732f0 msgr2=0x7f0c4810c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:19:22.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.921+0000 7f0c4fdff700 1 --2- 172.21.15.64:0/482496732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c480732f0 0x7f0c4810c900 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0c38009990 tx=0x7f0c38009ca0 comp rx=0 tx=0).stop 2023-09-13T03:19:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.921+0000 7f0c4fdff700 1 -- 172.21.15.64:0/482496732 shutdown_connections 2023-09-13T03:19:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.921+0000 7f0c4fdff700 1 --2- 172.21.15.64:0/482496732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c4810ce40 0x7f0c4810f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.921+0000 7f0c4fdff700 1 --2- 172.21.15.64:0/482496732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c480729d0 0x7f0c48072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.921+0000 7f0c4fdff700 1 --2- 172.21.15.64:0/482496732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c480732f0 0x7f0c4810c900 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:22.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.921+0000 7f0c4fdff700 1 -- 172.21.15.64:0/482496732 >> 172.21.15.64:0/482496732 conn(0x7f0c480fc0b0 msgr2=0x7f0c480fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:19:22.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.921+0000 7f0c4fdff700 1 -- 172.21.15.64:0/482496732 shutdown_connections 2023-09-13T03:19:22.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.921+0000 7f0c4fdff700 1 -- 172.21.15.64:0/482496732 wait complete. 2023-09-13T03:19:22.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.922+0000 7f0c4fdff700 1 Processor -- start 2023-09-13T03:19:22.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.922+0000 7f0c4fdff700 1 -- start start 2023-09-13T03:19:22.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.922+0000 7f0c4fdff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c480729d0 0x7f0c4819e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:22.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.922+0000 7f0c4fdff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c480732f0 0x7f0c4819edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:22.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.922+0000 7f0c4fdff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c4810ce40 0x7f0c481986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:22.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.922+0000 7f0c4fdff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c48198d50 con 0x7f0c4810ce40 2023-09-13T03:19:22.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.922+0000 7f0c4fdff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c48198ec0 con 0x7f0c480732f0 2023-09-13T03:19:22.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.922+0000 7f0c4fdff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c48199080 con 0x7f0c480729d0 2023-09-13T03:19:22.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.923+0000 7f0c4db9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c480729d0 0x7f0c4819e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.923+0000 7f0c4e39c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c4810ce40 0x7f0c481986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.923+0000 7f0c4db9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c480729d0 0x7f0c4819e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:46428/0 (socket says 172.21.15.64:46428) 2023-09-13T03:19:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.923+0000 7f0c4db9b700 1 -- 172.21.15.64:0/3804130625 learned_addr learned my addr 172.21.15.64:0/3804130625 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:19:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.923+0000 7f0c4d39a700 1 --2- 172.21.15.64:0/3804130625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c480732f0 0x7f0c4819edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.923+0000 7f0c4e39c700 1 -- 172.21.15.64:0/3804130625 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c480729d0 msgr2=0x7f0c4819e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:19:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.923+0000 7f0c4e39c700 1 --2- 172.21.15.64:0/3804130625 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c480729d0 0x7f0c4819e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.923+0000 7f0c4e39c700 1 -- 172.21.15.64:0/3804130625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c480732f0 msgr2=0x7f0c4819edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:19:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.923+0000 7f0c4e39c700 1 --2- 172.21.15.64:0/3804130625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c480732f0 0x7f0c4819edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.923+0000 7f0c4e39c700 1 -- 172.21.15.64:0/3804130625 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c380095f0 con 0x7f0c4810ce40 2023-09-13T03:19:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.923+0000 7f0c4db9b700 1 --2- 172.21.15.64:0/3804130625 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c480729d0 0x7f0c4819e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:19:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.924+0000 7f0c4e39c700 1 --2- 172.21.15.64:0/3804130625 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c4810ce40 0x7f0c481986e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0c4000ef90 tx=0x7f0c4000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:19:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.924+0000 7f0c3effd700 1 -- 172.21.15.64:0/3804130625 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c40018070 con 0x7f0c4810ce40 2023-09-13T03:19:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.924+0000 7f0c4fdff700 1 -- 172.21.15.64:0/3804130625 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c48199300 con 0x7f0c4810ce40 2023-09-13T03:19:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.924+0000 7f0c3effd700 1 -- 172.21.15.64:0/3804130625 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c4000f040 con 0x7f0c4810ce40 2023-09-13T03:19:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.924+0000 7f0c3effd700 1 -- 172.21.15.64:0/3804130625 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4000cc30 con 0x7f0c4810ce40 2023-09-13T03:19:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.924+0000 7f0c4fdff700 1 -- 172.21.15.64:0/3804130625 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c481a5870 con 0x7f0c4810ce40 2023-09-13T03:19:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.929+0000 7f0c3effd700 1 -- 172.21.15.64:0/3804130625 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f0c40003a00 con 0x7f0c4810ce40 2023-09-13T03:19:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.930+0000 7f0c6378cb80 1 Processor -- start 2023-09-13T03:19:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.930+0000 7f0c6378cb80 1 -- start start 2023-09-13T03:19:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.930+0000 7f0c6378cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cce2c6c1a0 0x55cce2c6c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.930+0000 7f0c6378cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cce2c66300 0x55cce2c66780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.930+0000 7f0c6378cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cce2c66cc0 0x55cce2c72f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.930+0000 7f0c6378cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce2bbf6d0 con 0x55cce2c6c1a0 2023-09-13T03:19:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.930+0000 7f0c6378cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce2bbcff0 con 0x55cce2c66cc0 2023-09-13T03:19:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.930+0000 7f0c6378cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce2bb88e0 con 0x55cce2c66300 2023-09-13T03:19:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.930+0000 7f0c3effd700 1 --2- 172.21.15.64:0/3804130625 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f0c340a07b0 0x7f0c340a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.931+0000 7f0c3effd700 1 -- 172.21.15.64:0/3804130625 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f0c400cba70 con 0x7f0c4810ce40 2023-09-13T03:19:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.931+0000 7f0c4d39a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cce2c6c1a0 0x55cce2c6c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:22.937 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.931+0000 7f0c4e39c700 1 --2- 172.21.15.64:0/3804130625 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f0c340a07b0 0x7f0c340a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:22.937 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.931+0000 7f0c4d39a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cce2c6c1a0 0x55cce2c6c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47620/0 (socket says 172.21.15.64:47620) 2023-09-13T03:19:22.938 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.931+0000 7f0c4d39a700 1 -- 172.21.15.64:0/4280437380 learned_addr learned my addr 172.21.15.64:0/4280437380 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:19:22.938 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.931+0000 7f0c4db9b700 1 --2- 172.21.15.64:0/4280437380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cce2c66300 0x55cce2c66780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:22.938 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.931+0000 7f0c4d39a700 1 --2- 172.21.15.64:0/4280437380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cce2c66cc0 0x55cce2c72f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.931+0000 7f0c4db9b700 1 -- 172.21.15.64:0/4280437380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cce2c66cc0 msgr2=0x55cce2c72f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:19:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.931+0000 7f0c4db9b700 1 --2- 172.21.15.64:0/4280437380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cce2c66cc0 0x55cce2c72f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:22.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.931+0000 7f0c4db9b700 1 -- 172.21.15.64:0/4280437380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cce2c6c1a0 msgr2=0x55cce2c6c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:19:22.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.931+0000 7f0c4db9b700 1 --2- 172.21.15.64:0/4280437380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cce2c6c1a0 0x55cce2c6c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:22.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.931+0000 7f0c4db9b700 1 -- 172.21.15.64:0/4280437380 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cce29e4d80 con 0x55cce2c66300 2023-09-13T03:19:22.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.931+0000 7f0c4d39a700 1 --2- 172.21.15.64:0/4280437380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cce2c6c1a0 0x55cce2c6c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:19:22.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.931+0000 7f0c4e39c700 1 --2- 172.21.15.64:0/3804130625 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f0c340a07b0 0x7f0c340a2c70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0c4000c6f0 tx=0x7f0c40009040 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:19:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.932+0000 7f0c4db9b700 1 --2- 172.21.15.64:0/4280437380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cce2c66300 0x55cce2c66780 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0c4400f4c0 tx=0x7f0c4400f7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5662aad375f0fad8 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:19:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.932+0000 7f0c33fff700 1 -- 172.21.15.64:0/4280437380 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c44004a30 con 0x55cce2c66300 2023-09-13T03:19:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.932+0000 7f0c33fff700 1 -- 172.21.15.64:0/4280437380 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c4400fe50 con 0x55cce2c66300 2023-09-13T03:19:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.932+0000 7f0c33fff700 1 -- 172.21.15.64:0/4280437380 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c440108e0 con 0x55cce2c66300 2023-09-13T03:19:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.933+0000 7f0c6378cb80 1 -- 172.21.15.64:0/4280437380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cce2c66300 msgr2=0x55cce2c66780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:19:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.933+0000 7f0c6378cb80 1 --2- 172.21.15.64:0/4280437380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cce2c66300 0x55cce2c66780 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0c4400f4c0 tx=0x7f0c4400f7d0 comp rx=0 tx=0).stop 2023-09-13T03:19:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.933+0000 7f0c6378cb80 1 -- 172.21.15.64:0/4280437380 shutdown_connections 2023-09-13T03:19:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.933+0000 7f0c6378cb80 1 --2- 172.21.15.64:0/4280437380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cce2c66cc0 0x55cce2c72f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.933+0000 7f0c6378cb80 1 --2- 172.21.15.64:0/4280437380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cce2c6c1a0 0x55cce2c6c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.933+0000 7f0c6378cb80 1 --2- 172.21.15.64:0/4280437380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cce2c66300 0x55cce2c66780 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.933+0000 7f0c6378cb80 1 -- 172.21.15.64:0/4280437380 >> 172.21.15.64:0/4280437380 conn(0x55cce2bdb030 msgr2=0x55cce2c61560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:19:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.933+0000 7f0c6378cb80 1 -- 172.21.15.64:0/4280437380 shutdown_connections 2023-09-13T03:19:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.933+0000 7f0c6378cb80 1 -- 172.21.15.64:0/4280437380 wait complete. 2023-09-13T03:19:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.934+0000 7f0c6378cb80 1 -- start start 2023-09-13T03:19:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.934+0000 7f0c6378cb80 1 Processor -- start 2023-09-13T03:19:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.934+0000 7f0c6378cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cce2c66300 0x55cce2c72e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.934+0000 7f0c6378cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cce2c66cc0 0x55cce2c70140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.934+0000 7f0c6378cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cce2c6c1a0 0x55cce2c70770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.934+0000 7f0c6378cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce2b56b10 con 0x55cce2c66300 2023-09-13T03:19:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.934+0000 7f0c6378cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce2b56810 con 0x55cce2c6c1a0 2023-09-13T03:19:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.934+0000 7f0c6378cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce2a3a8c0 con 0x55cce2c66cc0 2023-09-13T03:19:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.934+0000 7f0c6378cb80 10 client.0 _ll_register_callbacks cb 0x55cce2aa0a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:19:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.935+0000 7f0c4d39a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cce2c66300 0x55cce2c72e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.935+0000 7f0c4d39a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cce2c66300 0x55cce2c72e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47634/0 (socket says 172.21.15.64:47634) 2023-09-13T03:19:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.935+0000 7f0c4d39a700 1 -- 172.21.15.64:0/3448260245 learned_addr learned my addr 172.21.15.64:0/3448260245 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:19:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.935+0000 7f0c4db9b700 1 --2- 172.21.15.64:0/3448260245 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cce2c66cc0 0x55cce2c70140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.935+0000 7f0c4d39a700 1 --2- 172.21.15.64:0/3448260245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cce2c6c1a0 0x55cce2c70770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.935+0000 7f0c4d39a700 1 -- 172.21.15.64:0/3448260245 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cce2c66cc0 msgr2=0x55cce2c70140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:19:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.935+0000 7f0c4d39a700 1 --2- 172.21.15.64:0/3448260245 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cce2c66cc0 0x55cce2c70140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.935+0000 7f0c4d39a700 1 -- 172.21.15.64:0/3448260245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cce2c6c1a0 msgr2=0x55cce2c70770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:19:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.935+0000 7f0c4d39a700 1 --2- 172.21.15.64:0/3448260245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cce2c6c1a0 0x55cce2c70770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:22.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.935+0000 7f0c4d39a700 1 -- 172.21.15.64:0/3448260245 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c38020040 con 0x55cce2c66300 2023-09-13T03:19:22.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.935+0000 7f0c4d39a700 1 --2- 172.21.15.64:0/3448260245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cce2c66300 0x55cce2c72e20 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0c380098e0 tx=0x7f0c38005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4aecd16cd7d263d server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:19:22.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.935+0000 7f0c317fa700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:19:22.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.936+0000 7f0c317fa700 1 -- 172.21.15.64:0/3448260245 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3801d070 con 0x55cce2c66300 2023-09-13T03:19:22.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.936+0000 7f0c6378cb80 1 -- 172.21.15.64:0/3448260245 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cce2b10490 con 0x55cce2c66300 2023-09-13T03:19:22.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.936+0000 7f0c317fa700 1 -- 172.21.15.64:0/3448260245 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f0c38005660 con 0x55cce2c66300 2023-09-13T03:19:22.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.936+0000 7f0c317fa700 1 -- 172.21.15.64:0/3448260245 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c38004500 con 0x55cce2c66300 2023-09-13T03:19:22.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.936+0000 7f0c6378cb80 10 client.16587 fetch_fsmap learned FSMap version 46 2023-09-13T03:19:22.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.936+0000 7f0c6378cb80 1 -- 172.21.15.64:0/3448260245 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7f0c4400ad30 con 0x55cce2c66300 2023-09-13T03:19:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.936+0000 7f0c317fa700 1 -- 172.21.15.64:0/3448260245 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cce2b10490 con 0x55cce2c66300 2023-09-13T03:19:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.936+0000 7f0c317fa700 1 -- 172.21.15.64:0/3448260245 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 46) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f0c3800f7c0 con 0x55cce2c66300 2023-09-13T03:19:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.936+0000 7f0c6378cb80 10 client.16587 fetch_fsmap finished waiting for FSMap version 46 2023-09-13T03:19:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.936+0000 7f0c6378cb80 10 client.16587 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:19:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.936+0000 7f0c6378cb80 20 client.16587 populate_metadata read hostname 'smithi064' 2023-09-13T03:19:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.937+0000 7f0c6378cb80 1 --2- 172.21.15.64:0/3448260245 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55cce2c80470 0x55cce2c82890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.937+0000 7f0c6378cb80 4 client.16587 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:19:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.937+0000 7f0c6378cb80 1 -- 172.21.15.64:0/3448260245 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cce2bcd840 con 0x55cce2c80470 2023-09-13T03:19:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.937+0000 7f0c4db9b700 1 --2- 172.21.15.64:0/3448260245 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55cce2c80470 0x55cce2c82890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.938+0000 7f0c317fa700 10 client.16587 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:19:22.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.938+0000 7f0c4db9b700 1 --2- 172.21.15.64:0/3448260245 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55cce2c80470 0x55cce2c82890 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=465d245a0087deca server_cookie=3a2c8e8c9dd41abb in_seq=0 out_seq=0 2023-09-13T03:19:22.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.939+0000 7f0c317fa700 1 -- 172.21.15.64:0/3448260245 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cce2bcd840 con 0x55cce2c80470 2023-09-13T03:19:22.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.939+0000 7f0c317fa700 10 client.16587 handle_command_reply: tid=0 2023-09-13T03:19:22.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.939+0000 7f0c6378cb80 1 client.16587 shutdown 2023-09-13T03:19:22.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.939+0000 7f0c6378cb80 1 -- 172.21.15.64:0/3448260245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cce2c66300 msgr2=0x55cce2c72e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:19:22.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.939+0000 7f0c6378cb80 1 --2- 172.21.15.64:0/3448260245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cce2c66300 0x55cce2c72e20 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0c380098e0 tx=0x7f0c38005a90 comp rx=0 tx=0).stop 2023-09-13T03:19:22.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.939+0000 7f0c6378cb80 1 -- 172.21.15.64:0/3448260245 shutdown_connections 2023-09-13T03:19:22.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.939+0000 7f0c6378cb80 1 --2- 172.21.15.64:0/3448260245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cce2c6c1a0 0x55cce2c70770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:22.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.939+0000 7f0c6378cb80 1 --2- 172.21.15.64:0/3448260245 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55cce2c80470 0x55cce2c82890 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:22.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.939+0000 7f0c6378cb80 1 --2- 172.21.15.64:0/3448260245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cce2c66300 0x55cce2c72e20 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.939+0000 7f0c6378cb80 1 --2- 172.21.15.64:0/3448260245 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cce2c66cc0 0x55cce2c70140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.939+0000 7f0c6378cb80 1 -- 172.21.15.64:0/3448260245 >> 172.21.15.64:0/3448260245 conn(0x55cce2bdb030 msgr2=0x55cce2c607a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:19:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.939+0000 7f0c317fa700 0 client.16587 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:19:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.940+0000 7f0c6378cb80 1 -- 172.21.15.64:0/3448260245 shutdown_connections 2023-09-13T03:19:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.940+0000 7f0c6378cb80 1 -- 172.21.15.64:0/3448260245 wait complete. 2023-09-13T03:19:22.957 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.940+0000 7f0c6378cb80 20 client.16587 trim_cache size 0 max 16384 2023-09-13T03:19:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.955+0000 7f0c6378cb80 1 Processor -- start 2023-09-13T03:19:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.955+0000 7f0c6378cb80 1 -- start start 2023-09-13T03:19:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.956+0000 7f0c6378cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cce2c66300 0x55cce2c6de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.956+0000 7f0c6378cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cce2c66cc0 0x55cce2c77520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.956+0000 7f0c6378cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cce2c6c1a0 0x55cce2c72950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.956+0000 7f0c6378cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce2c670a0 con 0x55cce2c6c1a0 2023-09-13T03:19:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.956+0000 7f0c6378cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce2c71150 con 0x55cce2c66300 2023-09-13T03:19:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.956+0000 7f0c6378cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce2c6be00 con 0x55cce2c66cc0 2023-09-13T03:19:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.956+0000 7f0c4d39a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cce2c66300 0x55cce2c6de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.956+0000 7f0c4d39a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cce2c66300 0x55cce2c6de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:43076/0 (socket says 172.21.15.64:43076) 2023-09-13T03:19:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.956+0000 7f0c4d39a700 1 -- 172.21.15.64:0/1321456738 learned_addr learned my addr 172.21.15.64:0/1321456738 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:19:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.956+0000 7f0c4d39a700 1 --2- 172.21.15.64:0/1321456738 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cce2c6c1a0 0x55cce2c72950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:22.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.956+0000 7f0c4db9b700 1 --2- 172.21.15.64:0/1321456738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cce2c66cc0 0x55cce2c77520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:22.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.956+0000 7f0c4d39a700 1 -- 172.21.15.64:0/1321456738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cce2c66cc0 msgr2=0x55cce2c77520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:19:22.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.956+0000 7f0c4d39a700 1 --2- 172.21.15.64:0/1321456738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cce2c66cc0 0x55cce2c77520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:22.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.956+0000 7f0c4d39a700 1 -- 172.21.15.64:0/1321456738 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cce2c6c1a0 msgr2=0x55cce2c72950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:19:22.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.956+0000 7f0c4d39a700 1 --2- 172.21.15.64:0/1321456738 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cce2c6c1a0 0x55cce2c72950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:22.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.956+0000 7f0c4d39a700 1 -- 172.21.15.64:0/1321456738 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cce29cdc00 con 0x55cce2c66300 2023-09-13T03:19:22.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.957+0000 7f0c4d39a700 1 --2- 172.21.15.64:0/1321456738 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cce2c66300 0x55cce2c6de60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0c38009990 tx=0x7f0c3801a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51d58f9ff03ed2a server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:19:22.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.957+0000 7f0c3cbff700 1 -- 172.21.15.64:0/1321456738 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3801d070 con 0x55cce2c66300 2023-09-13T03:19:22.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.957+0000 7f0c3cbff700 1 -- 172.21.15.64:0/1321456738 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c3801e040 con 0x55cce2c66300 2023-09-13T03:19:22.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.957+0000 7f0c3cbff700 1 -- 172.21.15.64:0/1321456738 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c38004740 con 0x55cce2c66300 2023-09-13T03:19:22.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.958+0000 7f0c6378cb80 1 -- 172.21.15.64:0/1321456738 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cce2c66300 msgr2=0x55cce2c6de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:19:22.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.958+0000 7f0c6378cb80 1 --2- 172.21.15.64:0/1321456738 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cce2c66300 0x55cce2c6de60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0c38009990 tx=0x7f0c3801a040 comp rx=0 tx=0).stop 2023-09-13T03:19:22.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.958+0000 7f0c6378cb80 1 -- 172.21.15.64:0/1321456738 shutdown_connections 2023-09-13T03:19:22.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.958+0000 7f0c6378cb80 1 --2- 172.21.15.64:0/1321456738 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cce2c66300 0x55cce2c6de60 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.958+0000 7f0c6378cb80 1 --2- 172.21.15.64:0/1321456738 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cce2c6c1a0 0x55cce2c72950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.958+0000 7f0c6378cb80 1 --2- 172.21.15.64:0/1321456738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cce2c66cc0 0x55cce2c77520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.958+0000 7f0c6378cb80 1 -- 172.21.15.64:0/1321456738 >> 172.21.15.64:0/1321456738 conn(0x55cce2bdb030 msgr2=0x55cce2c6a250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:19:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.958+0000 7f0c6378cb80 1 -- 172.21.15.64:0/1321456738 shutdown_connections 2023-09-13T03:19:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.958+0000 7f0c6378cb80 1 -- 172.21.15.64:0/1321456738 wait complete. 2023-09-13T03:19:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.958+0000 7f0c6378cb80 1 -- start start 2023-09-13T03:19:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.958+0000 7f0c6378cb80 1 Processor -- start 2023-09-13T03:19:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.959+0000 7f0c6378cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cce2c66300 0x55cce2c6fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.959+0000 7f0c6378cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cce2c66cc0 0x55cce2c700f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.959+0000 7f0c6378cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cce2c6c1a0 0x55cce2c6e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.959+0000 7f0c6378cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce2c6eeb0 con 0x55cce2c66300 2023-09-13T03:19:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.959+0000 7f0c6378cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce2c6f020 con 0x55cce2c66cc0 2023-09-13T03:19:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.959+0000 7f0c6378cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce2c6f1e0 con 0x55cce2c6c1a0 2023-09-13T03:19:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.959+0000 7f0c6378cb80 10 client.0 _ll_register_callbacks cb 0x55cce2aa0a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:19:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.959+0000 7f0c4db9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cce2c66cc0 0x55cce2c700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.959+0000 7f0c4db9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cce2c66cc0 0x55cce2c700f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:43080/0 (socket says 172.21.15.64:43080) 2023-09-13T03:19:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.959+0000 7f0c4db9b700 1 -- 172.21.15.64:0/3709207602 learned_addr learned my addr 172.21.15.64:0/3709207602 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:19:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.959+0000 7f0c4d39a700 1 --2- 172.21.15.64:0/3709207602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cce2c66300 0x55cce2c6fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.960+0000 7f0c4db9b700 1 -- 172.21.15.64:0/3709207602 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cce2c6c1a0 msgr2=0x55cce2c6e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:19:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.960+0000 7f0c4db9b700 1 --2- 172.21.15.64:0/3709207602 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cce2c6c1a0 0x55cce2c6e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.960+0000 7f0c4db9b700 1 -- 172.21.15.64:0/3709207602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cce2c66300 msgr2=0x55cce2c6fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:19:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.960+0000 7f0c4db9b700 1 --2- 172.21.15.64:0/3709207602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cce2c66300 0x55cce2c6fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.960+0000 7f0c4db9b700 1 -- 172.21.15.64:0/3709207602 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c4400ad30 con 0x55cce2c66cc0 2023-09-13T03:19:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.960+0000 7f0c4db9b700 1 --2- 172.21.15.64:0/3709207602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cce2c66cc0 0x55cce2c700f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0c4400a700 tx=0x7f0c44004aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebc84a68a5fcbfb2 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:19:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.960+0000 7f0c317fa700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:19:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.960+0000 7f0c317fa700 1 -- 172.21.15.64:0/3709207602 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c44029490 con 0x55cce2c66cc0 2023-09-13T03:19:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.960+0000 7f0c6378cb80 1 -- 172.21.15.64:0/3709207602 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0c4400fe50 con 0x55cce2c66cc0 2023-09-13T03:19:22.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.960+0000 7f0c317fa700 1 -- 172.21.15.64:0/3709207602 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f0c44029950 con 0x55cce2c66cc0 2023-09-13T03:19:22.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.961+0000 7f0c317fa700 1 -- 172.21.15.64:0/3709207602 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c44029b10 con 0x55cce2c66cc0 2023-09-13T03:19:22.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.961+0000 7f0c6378cb80 10 client.26633 fetch_fsmap learned FSMap version 46 2023-09-13T03:19:22.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.961+0000 7f0c6378cb80 1 -- 172.21.15.64:0/3709207602 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55cce2a0b1f0 con 0x55cce2c66cc0 2023-09-13T03:19:22.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.961+0000 7f0c317fa700 1 -- 172.21.15.64:0/3709207602 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c4400fe50 con 0x55cce2c66cc0 2023-09-13T03:19:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.961+0000 7f0c317fa700 1 -- 172.21.15.64:0/3709207602 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 46) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f0c4400f4f0 con 0x55cce2c66cc0 2023-09-13T03:19:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.961+0000 7f0c6378cb80 10 client.26633 fetch_fsmap finished waiting for FSMap version 46 2023-09-13T03:19:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.961+0000 7f0c6378cb80 10 client.26633 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:19:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.961+0000 7f0c6378cb80 20 client.26633 populate_metadata read hostname 'smithi064' 2023-09-13T03:19:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.961+0000 7f0c6378cb80 1 --2- 172.21.15.64:0/3709207602 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55cce2cd5200 0x55cce2cd7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.961+0000 7f0c6378cb80 4 client.26633 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:19:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.961+0000 7f0c6378cb80 1 -- 172.21.15.64:0/3709207602 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cce2bb18c0 con 0x55cce2cd5200 2023-09-13T03:19:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.961+0000 7f0c4d39a700 1 --2- 172.21.15.64:0/3709207602 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55cce2cd5200 0x55cce2cd7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.962+0000 7f0c4d39a700 1 --2- 172.21.15.64:0/3709207602 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55cce2cd5200 0x55cce2cd7620 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9b0864181be25b0 server_cookie=d30323849a187abc in_seq=0 out_seq=0 2023-09-13T03:19:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.962+0000 7f0c317fa700 10 client.26633 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:19:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.984+0000 7f0c317fa700 1 -- 172.21.15.64:0/3709207602 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cce2bb18c0 con 0x55cce2cd5200 2023-09-13T03:19:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.984+0000 7f0c317fa700 10 client.26633 handle_command_reply: tid=0 2023-09-13T03:19:22.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.984+0000 7f0c6378cb80 1 client.26633 shutdown 2023-09-13T03:19:22.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.984+0000 7f0c6378cb80 1 -- 172.21.15.64:0/3709207602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cce2c66cc0 msgr2=0x55cce2c700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:19:22.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.984+0000 7f0c6378cb80 1 --2- 172.21.15.64:0/3709207602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cce2c66cc0 0x55cce2c700f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0c4400a700 tx=0x7f0c44004aa0 comp rx=0 tx=0).stop 2023-09-13T03:19:22.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.985+0000 7f0c6378cb80 1 -- 172.21.15.64:0/3709207602 shutdown_connections 2023-09-13T03:19:22.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.985+0000 7f0c6378cb80 1 --2- 172.21.15.64:0/3709207602 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55cce2cd5200 0x55cce2cd7620 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:22.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.985+0000 7f0c6378cb80 1 --2- 172.21.15.64:0/3709207602 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cce2c6c1a0 0x55cce2c6e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:22.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.985+0000 7f0c317fa700 0 client.26633 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:19:22.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.985+0000 7f0c6378cb80 1 --2- 172.21.15.64:0/3709207602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cce2c66300 0x55cce2c6fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:22.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.985+0000 7f0c6378cb80 1 --2- 172.21.15.64:0/3709207602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cce2c66cc0 0x55cce2c700f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.985+0000 7f0c6378cb80 1 -- 172.21.15.64:0/3709207602 >> 172.21.15.64:0/3709207602 conn(0x55cce2bdb030 msgr2=0x55cce2c61090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:19:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.985+0000 7f0c6378cb80 1 -- 172.21.15.64:0/3709207602 shutdown_connections 2023-09-13T03:19:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.985+0000 7f0c6378cb80 1 -- 172.21.15.64:0/3709207602 wait complete. 2023-09-13T03:19:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.985+0000 7f0c6378cb80 20 client.26633 trim_cache size 0 max 16384 2023-09-13T03:19:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:19:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:19:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:19:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:19:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.986+0000 7f0c4fdff700 1 -- 172.21.15.64:0/3804130625 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f0c340a07b0 msgr2=0x7f0c340a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:19:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.986+0000 7f0c4fdff700 1 --2- 172.21.15.64:0/3804130625 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f0c340a07b0 0x7f0c340a2c70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0c4000c6f0 tx=0x7f0c40009040 comp rx=0 tx=0).stop 2023-09-13T03:19:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.986+0000 7f0c4fdff700 1 -- 172.21.15.64:0/3804130625 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c4810ce40 msgr2=0x7f0c481986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:19:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.986+0000 7f0c4fdff700 1 --2- 172.21.15.64:0/3804130625 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c4810ce40 0x7f0c481986e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0c4000ef90 tx=0x7f0c4000c3a0 comp rx=0 tx=0).stop 2023-09-13T03:19:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.986+0000 7f0c4fdff700 1 -- 172.21.15.64:0/3804130625 shutdown_connections 2023-09-13T03:19:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.986+0000 7f0c4fdff700 1 --2- 172.21.15.64:0/3804130625 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f0c340a07b0 0x7f0c340a2c70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.986+0000 7f0c4fdff700 1 --2- 172.21.15.64:0/3804130625 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c4810ce40 0x7f0c481986e0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.986+0000 7f0c4fdff700 1 --2- 172.21.15.64:0/3804130625 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c480729d0 0x7f0c4819e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.986+0000 7f0c4fdff700 1 --2- 172.21.15.64:0/3804130625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c480732f0 0x7f0c4819edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.986+0000 7f0c4fdff700 1 -- 172.21.15.64:0/3804130625 >> 172.21.15.64:0/3804130625 conn(0x7f0c480fc0b0 msgr2=0x7f0c481048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:19:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.986+0000 7f0c4fdff700 1 -- 172.21.15.64:0/3804130625 shutdown_connections 2023-09-13T03:19:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:22.986+0000 7f0c4fdff700 1 -- 172.21.15.64:0/3804130625 wait complete. 2023-09-13T03:19:22.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:19:22 smithi142 ceph-mon[134082]: pgmap v189: 97 pgs: 97 active+clean; 68 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 135 op/s 2023-09-13T03:19:23.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:19:22 smithi170 ceph-mon[134328]: pgmap v189: 97 pgs: 97 active+clean; 68 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 135 op/s 2023-09-13T03:19:23.919 INFO:tasks.workunit.client.0.smithi064.stdout: 1 128449 1.93 MB/sec execute 208 sec latency 1547.065 ms 2023-09-13T03:19:24.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:19:24 smithi064 ceph-mon[97675]: pgmap v190: 97 pgs: 97 active+clean; 68 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 1.2 MiB/s wr, 87 op/s 2023-09-13T03:19:24.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:19:24 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:19:24.919 INFO:tasks.workunit.client.0.smithi064.stdout: 1 128449 1.92 MB/sec execute 209 sec latency 1345.059 ms 2023-09-13T03:19:24.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:19:24 smithi142 ceph-mon[134082]: pgmap v190: 97 pgs: 97 active+clean; 68 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 1.2 MiB/s wr, 87 op/s 2023-09-13T03:19:24.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:19:24 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:19:25.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:19:24 smithi170 ceph-mon[134328]: pgmap v190: 97 pgs: 97 active+clean; 68 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 1.2 MiB/s wr, 87 op/s 2023-09-13T03:19:25.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:19:24 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:19:25.919 INFO:tasks.workunit.client.0.smithi064.stdout: 1 128449 1.91 MB/sec execute 210 sec latency 2345.188 ms 2023-09-13T03:19:25.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:19:25 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:19:25.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:19:25 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:19:25.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:19:25 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:19:26.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:19:25 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:19:26.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:19:25 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:19:26.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:19:25 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:19:26.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:19:25 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:19:26.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:19:25 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:19:26.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:19:25 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:19:26.919 INFO:tasks.workunit.client.0.smithi064.stdout: 1 128449 1.90 MB/sec execute 211 sec latency 3345.310 ms 2023-09-13T03:19:26.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:19:26 smithi142 ceph-mon[134082]: pgmap v191: 97 pgs: 97 active+clean; 70 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 1.4 MiB/s wr, 122 op/s 2023-09-13T03:19:27.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:19:26 smithi170 ceph-mon[134328]: pgmap v191: 97 pgs: 97 active+clean; 70 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 1.4 MiB/s wr, 122 op/s 2023-09-13T03:19:27.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:19:26 smithi064 ceph-mon[97675]: pgmap v191: 97 pgs: 97 active+clean; 70 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 1.4 MiB/s wr, 122 op/s 2023-09-13T03:19:27.920 INFO:tasks.workunit.client.0.smithi064.stdout: 1 128449 1.89 MB/sec execute 212 sec latency 4345.444 ms 2023-09-13T03:19:28.919 INFO:tasks.workunit.client.0.smithi064.stdout: 1 129137 1.91 MB/sec execute 213 sec latency 4952.332 ms 2023-09-13T03:19:28.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:19:28 smithi142 ceph-mon[134082]: pgmap v192: 97 pgs: 97 active+clean; 70 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 1.2 MiB/s wr, 87 op/s 2023-09-13T03:19:29.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:19:28 smithi170 ceph-mon[134328]: pgmap v192: 97 pgs: 97 active+clean; 70 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 1.2 MiB/s wr, 87 op/s 2023-09-13T03:19:29.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:19:28 smithi064 ceph-mon[97675]: pgmap v192: 97 pgs: 97 active+clean; 70 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 1.2 MiB/s wr, 87 op/s 2023-09-13T03:19:29.921 INFO:tasks.workunit.client.0.smithi064.stdout: 1 129419 1.90 MB/sec execute 214 sec latency 758.522 ms 2023-09-13T03:19:30.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:19:30 smithi170 ceph-mon[134328]: pgmap v193: 97 pgs: 97 active+clean; 70 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 1.2 MiB/s wr, 87 op/s 2023-09-13T03:19:30.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:19:30 smithi064 ceph-mon[97675]: pgmap v193: 97 pgs: 97 active+clean; 70 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 1.2 MiB/s wr, 87 op/s 2023-09-13T03:19:30.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:19:30 smithi142 ceph-mon[134082]: pgmap v193: 97 pgs: 97 active+clean; 70 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 1.2 MiB/s wr, 87 op/s 2023-09-13T03:19:30.920 INFO:tasks.workunit.client.0.smithi064.stdout: 1 130657 1.92 MB/sec execute 215 sec latency 1186.932 ms 2023-09-13T03:19:31.920 INFO:tasks.workunit.client.0.smithi064.stdout: 1 131879 1.95 MB/sec execute 216 sec latency 13.552 ms 2023-09-13T03:19:32.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:19:32 smithi064 ceph-mon[97675]: pgmap v194: 97 pgs: 97 active+clean; 72 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 112 op/s 2023-09-13T03:19:32.920 INFO:tasks.workunit.client.0.smithi064.stdout: 1 131999 1.94 MB/sec execute 217 sec latency 925.417 ms 2023-09-13T03:19:32.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:19:32 smithi142 ceph-mon[134082]: pgmap v194: 97 pgs: 97 active+clean; 72 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 112 op/s 2023-09-13T03:19:33.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:19:32 smithi170 ceph-mon[134328]: pgmap v194: 97 pgs: 97 active+clean; 72 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 112 op/s 2023-09-13T03:19:33.921 INFO:tasks.workunit.client.0.smithi064.stdout: 1 132067 1.93 MB/sec execute 218 sec latency 1532.430 ms 2023-09-13T03:19:34.920 INFO:tasks.workunit.client.0.smithi064.stdout: 1 132067 1.92 MB/sec execute 219 sec latency 1346.470 ms 2023-09-13T03:19:34.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:19:34 smithi142 ceph-mon[134082]: pgmap v195: 97 pgs: 97 active+clean; 72 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 350 KiB/s wr, 59 op/s 2023-09-13T03:19:35.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:19:34 smithi170 ceph-mon[134328]: pgmap v195: 97 pgs: 97 active+clean; 72 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 350 KiB/s wr, 59 op/s 2023-09-13T03:19:35.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:19:34 smithi064 ceph-mon[97675]: pgmap v195: 97 pgs: 97 active+clean; 72 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 350 KiB/s wr, 59 op/s 2023-09-13T03:19:35.920 INFO:tasks.workunit.client.0.smithi064.stdout: 1 132067 1.91 MB/sec execute 220 sec latency 2346.630 ms 2023-09-13T03:19:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.148+0000 7fbb599ff700 1 -- 172.21.15.64:0/1773356968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb54106660 msgr2=0x7fbb54106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:19:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.148+0000 7fbb599ff700 1 --2- 172.21.15.64:0/1773356968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb54106660 0x7fbb54106a40 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fbb440099c0 tx=0x7fbb44009cd0 comp rx=0 tx=0).stop 2023-09-13T03:19:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.148+0000 7fbb599ff700 1 -- 172.21.15.64:0/1773356968 shutdown_connections 2023-09-13T03:19:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.148+0000 7fbb599ff700 1 --2- 172.21.15.64:0/1773356968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb54100f60 0x7fbb5410d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.148+0000 7fbb599ff700 1 --2- 172.21.15.64:0/1773356968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb54106660 0x7fbb54106a40 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.148+0000 7fbb599ff700 1 --2- 172.21.15.64:0/1773356968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb541005a0 0x7fbb54100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.148+0000 7fbb599ff700 1 -- 172.21.15.64:0/1773356968 >> 172.21.15.64:0/1773356968 conn(0x7fbb540fc050 msgr2=0x7fbb540fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:19:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.148+0000 7fbb599ff700 1 -- 172.21.15.64:0/1773356968 shutdown_connections 2023-09-13T03:19:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.148+0000 7fbb599ff700 1 -- 172.21.15.64:0/1773356968 wait complete. 2023-09-13T03:19:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.149+0000 7fbb599ff700 1 Processor -- start 2023-09-13T03:19:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.149+0000 7fbb599ff700 1 -- start start 2023-09-13T03:19:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.150+0000 7fbb599ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb541005a0 0x7fbb5419c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.150+0000 7fbb599ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb54100f60 0x7fbb5419cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:36.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.150+0000 7fbb599ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb54106660 0x7fbb54196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:36.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.150+0000 7fbb599ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb54196ac0 con 0x7fbb54100f60 2023-09-13T03:19:36.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.150+0000 7fbb599ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb54196c30 con 0x7fbb54106660 2023-09-13T03:19:36.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.150+0000 7fbb599ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb54196df0 con 0x7fbb541005a0 2023-09-13T03:19:36.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.150+0000 7fbb527fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb54100f60 0x7fbb5419cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:36.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.150+0000 7fbb52ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb541005a0 0x7fbb5419c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:36.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.150+0000 7fbb527fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb54100f60 0x7fbb5419cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45100/0 (socket says 172.21.15.64:45100) 2023-09-13T03:19:36.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.150+0000 7fbb527fc700 1 -- 172.21.15.64:0/1275546708 learned_addr learned my addr 172.21.15.64:0/1275546708 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:19:36.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.150+0000 7fbb52ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb541005a0 0x7fbb5419c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:43226/0 (socket says 172.21.15.64:43226) 2023-09-13T03:19:36.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.150+0000 7fbb537fe700 1 --2- 172.21.15.64:0/1275546708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb54106660 0x7fbb54196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:36.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.151+0000 7fbb527fc700 1 -- 172.21.15.64:0/1275546708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb541005a0 msgr2=0x7fbb5419c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:19:36.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.151+0000 7fbb527fc700 1 --2- 172.21.15.64:0/1275546708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb541005a0 0x7fbb5419c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:36.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.151+0000 7fbb527fc700 1 -- 172.21.15.64:0/1275546708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb54106660 msgr2=0x7fbb54196450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:19:36.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.151+0000 7fbb527fc700 1 --2- 172.21.15.64:0/1275546708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb54106660 0x7fbb54196450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:36.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.151+0000 7fbb527fc700 1 -- 172.21.15.64:0/1275546708 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb440095f0 con 0x7fbb54100f60 2023-09-13T03:19:36.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.151+0000 7fbb52ffd700 1 --2- 172.21.15.64:0/1275546708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb541005a0 0x7fbb5419c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:19:36.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.151+0000 7fbb527fc700 1 --2- 172.21.15.64:0/1275546708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb54100f60 0x7fbb5419cb30 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800e9a0 tx=0x7fbb4800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:19:36.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.151+0000 7fbb589fd700 1 -- 172.21.15.64:0/1275546708 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb48009790 con 0x7fbb54100f60 2023-09-13T03:19:36.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.151+0000 7fbb589fd700 1 -- 172.21.15.64:0/1275546708 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb48010430 con 0x7fbb54100f60 2023-09-13T03:19:36.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.151+0000 7fbb589fd700 1 -- 172.21.15.64:0/1275546708 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb48018680 con 0x7fbb54100f60 2023-09-13T03:19:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.151+0000 7fbb599ff700 1 -- 172.21.15.64:0/1275546708 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb541970d0 con 0x7fbb54100f60 2023-09-13T03:19:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.151+0000 7fbb599ff700 1 -- 172.21.15.64:0/1275546708 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb541a35e0 con 0x7fbb54100f60 2023-09-13T03:19:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.154+0000 7fbb589fd700 1 -- 172.21.15.64:0/1275546708 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fbb480105a0 con 0x7fbb54100f60 2023-09-13T03:19:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.156+0000 7fbb6d295b80 1 Processor -- start 2023-09-13T03:19:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.156+0000 7fbb6d295b80 1 -- start start 2023-09-13T03:19:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.156+0000 7fbb589fd700 1 --2- 172.21.15.64:0/1275546708 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fbb3c0a0830 0x7fbb3c0a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.156+0000 7fbb589fd700 1 -- 172.21.15.64:0/1275546708 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fbb48014070 con 0x7fbb54100f60 2023-09-13T03:19:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.156+0000 7fbb537fe700 1 --2- 172.21.15.64:0/1275546708 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fbb3c0a0830 0x7fbb3c0a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.157+0000 7fbb6d295b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9333e35b0 0x55f9333e3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.157+0000 7fbb6d295b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9333dd710 0x55f9333ddb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.157+0000 7fbb6d295b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9333de0d0 0x55f9333ea340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.157+0000 7fbb6d295b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9333326d0 con 0x55f9333dd710 2023-09-13T03:19:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.157+0000 7fbb6d295b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f93332fff0 con 0x55f9333e35b0 2023-09-13T03:19:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.157+0000 7fbb6d295b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f93332b8e0 con 0x55f9333de0d0 2023-09-13T03:19:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.157+0000 7fbb527fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9333dd710 0x55f9333ddb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.157+0000 7fbb527fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9333dd710 0x55f9333ddb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45114/0 (socket says 172.21.15.64:45114) 2023-09-13T03:19:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.157+0000 7fbb527fc700 1 -- 172.21.15.64:0/2024857792 learned_addr learned my addr 172.21.15.64:0/2024857792 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:19:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.157+0000 7fbb52ffd700 1 --2- 172.21.15.64:0/2024857792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9333e35b0 0x55f9333e3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.157+0000 7fbb537fe700 1 --2- 172.21.15.64:0/2024857792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9333de0d0 0x55f9333ea340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.157+0000 7fbb527fc700 1 -- 172.21.15.64:0/2024857792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9333de0d0 msgr2=0x55f9333ea340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:19:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.157+0000 7fbb527fc700 1 --2- 172.21.15.64:0/2024857792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9333de0d0 0x55f9333ea340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.157+0000 7fbb527fc700 1 -- 172.21.15.64:0/2024857792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9333e35b0 msgr2=0x55f9333e3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:19:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.157+0000 7fbb527fc700 1 --2- 172.21.15.64:0/2024857792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9333e35b0 0x55f9333e3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.157+0000 7fbb527fc700 1 -- 172.21.15.64:0/2024857792 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f933157d80 con 0x55f9333dd710 2023-09-13T03:19:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.157+0000 7fbb52ffd700 1 --2- 172.21.15.64:0/2024857792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9333e35b0 0x55f9333e3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:19:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.157+0000 7fbb537fe700 1 --2- 172.21.15.64:0/1275546708 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fbb3c0a0830 0x7fbb3c0a2cf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbb5419d2f0 tx=0x7fbb4c00fc00 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:19:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.158+0000 7fbb527fc700 1 --2- 172.21.15.64:0/2024857792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9333dd710 0x55f9333ddb90 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fbb48096020 tx=0x7fbb48098fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c7edfc4a92edcd4 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:19:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.158+0000 7fbb39ffb700 1 -- 172.21.15.64:0/2024857792 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4809d3e0 con 0x55f9333dd710 2023-09-13T03:19:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.158+0000 7fbb39ffb700 1 -- 172.21.15.64:0/2024857792 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb480a1620 con 0x55f9333dd710 2023-09-13T03:19:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.158+0000 7fbb39ffb700 1 -- 172.21.15.64:0/2024857792 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb48099810 con 0x55f9333dd710 2023-09-13T03:19:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.159+0000 7fbb6d295b80 1 -- 172.21.15.64:0/2024857792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9333dd710 msgr2=0x55f9333ddb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:19:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.159+0000 7fbb6d295b80 1 --2- 172.21.15.64:0/2024857792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9333dd710 0x55f9333ddb90 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fbb48096020 tx=0x7fbb48098fe0 comp rx=0 tx=0).stop 2023-09-13T03:19:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.159+0000 7fbb6d295b80 1 -- 172.21.15.64:0/2024857792 shutdown_connections 2023-09-13T03:19:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.159+0000 7fbb6d295b80 1 --2- 172.21.15.64:0/2024857792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9333e35b0 0x55f9333e3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.159+0000 7fbb6d295b80 1 --2- 172.21.15.64:0/2024857792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9333de0d0 0x55f9333ea340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.159+0000 7fbb6d295b80 1 --2- 172.21.15.64:0/2024857792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9333dd710 0x55f9333ddb90 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.159+0000 7fbb6d295b80 1 -- 172.21.15.64:0/2024857792 >> 172.21.15.64:0/2024857792 conn(0x55f93334e030 msgr2=0x55f9333d4560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:19:36.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.159+0000 7fbb6d295b80 1 -- 172.21.15.64:0/2024857792 shutdown_connections 2023-09-13T03:19:36.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.159+0000 7fbb6d295b80 1 -- 172.21.15.64:0/2024857792 wait complete. 2023-09-13T03:19:36.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.159+0000 7fbb6d295b80 1 -- start start 2023-09-13T03:19:36.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.160+0000 7fbb6d295b80 1 Processor -- start 2023-09-13T03:19:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.160+0000 7fbb6d295b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9333dd710 0x55f9333ea340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.160+0000 7fbb6d295b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9333de0d0 0x55f9333e7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.160+0000 7fbb6d295b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9333e35b0 0x55f9333e7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.160+0000 7fbb6d295b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9332c9b10 con 0x55f9333e35b0 2023-09-13T03:19:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.160+0000 7fbb6d295b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9332c9810 con 0x55f9333de0d0 2023-09-13T03:19:36.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.160+0000 7fbb6d295b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9331ad8c0 con 0x55f9333dd710 2023-09-13T03:19:36.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.160+0000 7fbb6d295b80 10 client.0 _ll_register_callbacks cb 0x55f933213a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:19:36.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.160+0000 7fbb537fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9333e35b0 0x55f9333e7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:36.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.161+0000 7fbb52ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9333dd710 0x55f9333ea340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:36.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.161+0000 7fbb537fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9333e35b0 0x55f9333e7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45124/0 (socket says 172.21.15.64:45124) 2023-09-13T03:19:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.161+0000 7fbb537fe700 1 -- 172.21.15.64:0/2265438810 learned_addr learned my addr 172.21.15.64:0/2265438810 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:19:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.161+0000 7fbb527fc700 1 --2- 172.21.15.64:0/2265438810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9333de0d0 0x55f9333e7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.161+0000 7fbb537fe700 1 -- 172.21.15.64:0/2265438810 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9333dd710 msgr2=0x55f9333ea340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:19:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.161+0000 7fbb537fe700 1 --2- 172.21.15.64:0/2265438810 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9333dd710 0x55f9333ea340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.161+0000 7fbb537fe700 1 -- 172.21.15.64:0/2265438810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9333de0d0 msgr2=0x55f9333e7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:19:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.161+0000 7fbb537fe700 1 --2- 172.21.15.64:0/2265438810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9333de0d0 0x55f9333e7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:36.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.161+0000 7fbb537fe700 1 -- 172.21.15.64:0/2265438810 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb5410dd20 con 0x55f9333e35b0 2023-09-13T03:19:36.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.161+0000 7fbb52ffd700 1 --2- 172.21.15.64:0/2265438810 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9333dd710 0x55f9333ea340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:19:36.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.161+0000 7fbb527fc700 1 --2- 172.21.15.64:0/2265438810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9333de0d0 0x55f9333e7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:19:36.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.161+0000 7fbb537fe700 1 --2- 172.21.15.64:0/2265438810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9333e35b0 0x55f9333e7c90 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c018b60 tx=0x7fbb4c004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f57782b51c96e10 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:19:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.161+0000 7fbb32ffd700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:19:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.162+0000 7fbb32ffd700 1 -- 172.21.15.64:0/2265438810 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c00e040 con 0x55f9333e35b0 2023-09-13T03:19:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.162+0000 7fbb6d295b80 1 -- 172.21.15.64:0/2265438810 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f933283490 con 0x55f9333e35b0 2023-09-13T03:19:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.162+0000 7fbb32ffd700 1 -- 172.21.15.64:0/2265438810 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fbb4c018e70 con 0x55f9333e35b0 2023-09-13T03:19:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.162+0000 7fbb32ffd700 1 -- 172.21.15.64:0/2265438810 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb4c009400 con 0x55f9333e35b0 2023-09-13T03:19:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.162+0000 7fbb6d295b80 10 client.16605 fetch_fsmap learned FSMap version 46 2023-09-13T03:19:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.162+0000 7fbb6d295b80 1 -- 172.21.15.64:0/2265438810 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7fbb541a35e0 con 0x55f9333e35b0 2023-09-13T03:19:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.162+0000 7fbb32ffd700 1 -- 172.21.15.64:0/2265438810 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f933283490 con 0x55f9333e35b0 2023-09-13T03:19:36.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.163+0000 7fbb32ffd700 1 -- 172.21.15.64:0/2265438810 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 46) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fbb4c022070 con 0x55f9333e35b0 2023-09-13T03:19:36.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.163+0000 7fbb6d295b80 10 client.16605 fetch_fsmap finished waiting for FSMap version 46 2023-09-13T03:19:36.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.163+0000 7fbb6d295b80 10 client.16605 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:19:36.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.163+0000 7fbb6d295b80 20 client.16605 populate_metadata read hostname 'smithi064' 2023-09-13T03:19:36.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.163+0000 7fbb6d295b80 1 --2- 172.21.15.64:0/2265438810 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f9333f7890 0x55f9333f9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.163+0000 7fbb6d295b80 4 client.16605 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:19:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.163+0000 7fbb6d295b80 1 -- 172.21.15.64:0/2265438810 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f933340840 con 0x55f9333f7890 2023-09-13T03:19:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.163+0000 7fbb52ffd700 1 --2- 172.21.15.64:0/2265438810 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f9333f7890 0x55f9333f9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.164+0000 7fbb52ffd700 1 --2- 172.21.15.64:0/2265438810 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f9333f7890 0x55f9333f9cb0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7da9842f91a68f05 server_cookie=104527a41746b3bf in_seq=0 out_seq=0 2023-09-13T03:19:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.164+0000 7fbb32ffd700 10 client.16605 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:19:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.164+0000 7fbb32ffd700 1 -- 172.21.15.64:0/2265438810 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f933340840 con 0x55f9333f7890 2023-09-13T03:19:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.164+0000 7fbb32ffd700 10 client.16605 handle_command_reply: tid=0 2023-09-13T03:19:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.165+0000 7fbb6d295b80 1 client.16605 shutdown 2023-09-13T03:19:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.165+0000 7fbb6d295b80 1 -- 172.21.15.64:0/2265438810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9333e35b0 msgr2=0x55f9333e7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:19:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.165+0000 7fbb6d295b80 1 --2- 172.21.15.64:0/2265438810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9333e35b0 0x55f9333e7c90 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c018b60 tx=0x7fbb4c004b40 comp rx=0 tx=0).stop 2023-09-13T03:19:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.165+0000 7fbb6d295b80 1 -- 172.21.15.64:0/2265438810 shutdown_connections 2023-09-13T03:19:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.165+0000 7fbb6d295b80 1 --2- 172.21.15.64:0/2265438810 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f9333f7890 0x55f9333f9cb0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.165+0000 7fbb6d295b80 1 --2- 172.21.15.64:0/2265438810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9333e35b0 0x55f9333e7c90 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.165+0000 7fbb6d295b80 1 --2- 172.21.15.64:0/2265438810 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9333dd710 0x55f9333ea340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.165+0000 7fbb6d295b80 1 --2- 172.21.15.64:0/2265438810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9333de0d0 0x55f9333e7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.165+0000 7fbb32ffd700 0 client.16605 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:19:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.165+0000 7fbb6d295b80 1 -- 172.21.15.64:0/2265438810 >> 172.21.15.64:0/2265438810 conn(0x55f93334e030 msgr2=0x55f9333d37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:19:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.165+0000 7fbb6d295b80 1 -- 172.21.15.64:0/2265438810 shutdown_connections 2023-09-13T03:19:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.165+0000 7fbb6d295b80 1 -- 172.21.15.64:0/2265438810 wait complete. 2023-09-13T03:19:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.165+0000 7fbb6d295b80 20 client.16605 trim_cache size 0 max 16384 2023-09-13T03:19:36.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.180+0000 7fbb6d295b80 1 Processor -- start 2023-09-13T03:19:36.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.180+0000 7fbb6d295b80 1 -- start start 2023-09-13T03:19:36.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.181+0000 7fbb6d295b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9333dd710 0x55f9333e7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:36.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.181+0000 7fbb6d295b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9333de0d0 0x55f9333ece90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:36.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.181+0000 7fbb6d295b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9333e35b0 0x55f9333e1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:36.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.181+0000 7fbb6d295b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9333e8290 con 0x55f9333de0d0 2023-09-13T03:19:36.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.181+0000 7fbb6d295b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9333e3050 con 0x55f9333e35b0 2023-09-13T03:19:36.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.181+0000 7fbb6d295b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9333f4e90 con 0x55f9333dd710 2023-09-13T03:19:36.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.181+0000 7fbb527fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9333de0d0 0x55f9333ece90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:36.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.181+0000 7fbb52ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9333dd710 0x55f9333e7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:36.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.181+0000 7fbb527fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9333de0d0 0x55f9333ece90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45136/0 (socket says 172.21.15.64:45136) 2023-09-13T03:19:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.181+0000 7fbb52ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9333dd710 0x55f9333e7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:43242/0 (socket says 172.21.15.64:43242) 2023-09-13T03:19:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.181+0000 7fbb527fc700 1 -- 172.21.15.64:0/831354153 learned_addr learned my addr 172.21.15.64:0/831354153 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:19:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.181+0000 7fbb537fe700 1 --2- 172.21.15.64:0/831354153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9333e35b0 0x55f9333e1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.181+0000 7fbb52ffd700 1 -- 172.21.15.64:0/831354153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9333e35b0 msgr2=0x55f9333e1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:19:36.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.181+0000 7fbb52ffd700 1 --2- 172.21.15.64:0/831354153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9333e35b0 0x55f9333e1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:36.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.181+0000 7fbb52ffd700 1 -- 172.21.15.64:0/831354153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9333de0d0 msgr2=0x55f9333ece90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:19:36.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.181+0000 7fbb52ffd700 1 --2- 172.21.15.64:0/831354153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9333de0d0 0x55f9333ece90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:36.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.181+0000 7fbb52ffd700 1 -- 172.21.15.64:0/831354153 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f933140c00 con 0x55f9333dd710 2023-09-13T03:19:36.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.181+0000 7fbb537fe700 1 --2- 172.21.15.64:0/831354153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9333e35b0 0x55f9333e1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:19:36.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.181+0000 7fbb527fc700 1 --2- 172.21.15.64:0/831354153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9333de0d0 0x55f9333ece90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:19:36.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.182+0000 7fbb52ffd700 1 --2- 172.21.15.64:0/831354153 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9333dd710 0x55f9333e7c10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbb4400b580 tx=0x7fbb44005fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7152307fd095417c server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:19:36.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.182+0000 7fbb3a7fc700 1 -- 172.21.15.64:0/831354153 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb44004580 con 0x55f9333dd710 2023-09-13T03:19:36.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.182+0000 7fbb3a7fc700 1 -- 172.21.15.64:0/831354153 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb4401e040 con 0x55f9333dd710 2023-09-13T03:19:36.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.182+0000 7fbb3a7fc700 1 -- 172.21.15.64:0/831354153 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb44003a30 con 0x55f9333dd710 2023-09-13T03:19:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.183+0000 7fbb6d295b80 1 -- 172.21.15.64:0/831354153 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9333dd710 msgr2=0x55f9333e7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:19:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.183+0000 7fbb6d295b80 1 --2- 172.21.15.64:0/831354153 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9333dd710 0x55f9333e7c10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbb4400b580 tx=0x7fbb44005fb0 comp rx=0 tx=0).stop 2023-09-13T03:19:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.183+0000 7fbb6d295b80 1 -- 172.21.15.64:0/831354153 shutdown_connections 2023-09-13T03:19:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.183+0000 7fbb6d295b80 1 --2- 172.21.15.64:0/831354153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9333e35b0 0x55f9333e1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.183+0000 7fbb6d295b80 1 --2- 172.21.15.64:0/831354153 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9333dd710 0x55f9333e7c10 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.183+0000 7fbb6d295b80 1 --2- 172.21.15.64:0/831354153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9333de0d0 0x55f9333ece90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.183+0000 7fbb6d295b80 1 -- 172.21.15.64:0/831354153 >> 172.21.15.64:0/831354153 conn(0x55f93334e030 msgr2=0x55f9333fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:19:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.183+0000 7fbb6d295b80 1 -- 172.21.15.64:0/831354153 shutdown_connections 2023-09-13T03:19:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.183+0000 7fbb6d295b80 1 -- 172.21.15.64:0/831354153 wait complete. 2023-09-13T03:19:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.183+0000 7fbb6d295b80 1 -- start start 2023-09-13T03:19:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.183+0000 7fbb6d295b80 1 Processor -- start 2023-09-13T03:19:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.184+0000 7fbb6d295b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9333dd710 0x55f93343f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.184+0000 7fbb6d295b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9333de0d0 0x55f93343fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.184+0000 7fbb6d295b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9333e35b0 0x55f93344a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.184+0000 7fbb6d295b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f93344abd0 con 0x55f9333e35b0 2023-09-13T03:19:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.184+0000 7fbb6d295b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f93344ad10 con 0x55f9333dd710 2023-09-13T03:19:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.184+0000 7fbb6d295b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f93344ae80 con 0x55f9333de0d0 2023-09-13T03:19:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.184+0000 7fbb6d295b80 10 client.0 _ll_register_callbacks cb 0x55f933213a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:19:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.184+0000 7fbb527fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9333de0d0 0x55f93343fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.184+0000 7fbb527fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9333de0d0 0x55f93343fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:43246/0 (socket says 172.21.15.64:43246) 2023-09-13T03:19:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.184+0000 7fbb527fc700 1 -- 172.21.15.64:0/1169313958 learned_addr learned my addr 172.21.15.64:0/1169313958 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:19:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.184+0000 7fbb537fe700 1 --2- 172.21.15.64:0/1169313958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9333e35b0 0x55f93344a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.184+0000 7fbb52ffd700 1 --2- 172.21.15.64:0/1169313958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9333dd710 0x55f93343f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.184+0000 7fbb527fc700 1 -- 172.21.15.64:0/1169313958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9333dd710 msgr2=0x55f93343f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:19:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.184+0000 7fbb527fc700 1 --2- 172.21.15.64:0/1169313958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9333dd710 0x55f93343f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.184+0000 7fbb527fc700 1 -- 172.21.15.64:0/1169313958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9333e35b0 msgr2=0x55f93344a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:19:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.184+0000 7fbb527fc700 1 --2- 172.21.15.64:0/1169313958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9333e35b0 0x55f93344a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.184+0000 7fbb527fc700 1 -- 172.21.15.64:0/1169313958 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f933157d80 con 0x55f9333de0d0 2023-09-13T03:19:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.185+0000 7fbb527fc700 1 --2- 172.21.15.64:0/1169313958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9333de0d0 0x55f93343fd40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbb48099540 tx=0x7fbb48098f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efbdb3d1dfde38c1 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:19:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.185+0000 7fbb32ffd700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:19:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.185+0000 7fbb32ffd700 1 -- 172.21.15.64:0/1169313958 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb48099a10 con 0x55f9333de0d0 2023-09-13T03:19:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.185+0000 7fbb6d295b80 1 -- 172.21.15.64:0/1169313958 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbb480a1620 con 0x55f9333de0d0 2023-09-13T03:19:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.185+0000 7fbb32ffd700 1 -- 172.21.15.64:0/1169313958 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fbb4809d3e0 con 0x55f9333de0d0 2023-09-13T03:19:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.185+0000 7fbb32ffd700 1 -- 172.21.15.64:0/1169313958 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb48097950 con 0x55f9333de0d0 2023-09-13T03:19:36.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.185+0000 7fbb6d295b80 10 client.26635 fetch_fsmap learned FSMap version 46 2023-09-13T03:19:36.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.185+0000 7fbb6d295b80 1 -- 172.21.15.64:0/1169313958 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7fbb541a35e0 con 0x55f9333de0d0 2023-09-13T03:19:36.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.185+0000 7fbb32ffd700 1 -- 172.21.15.64:0/1169313958 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb480a1620 con 0x55f9333de0d0 2023-09-13T03:19:36.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.185+0000 7fbb32ffd700 1 -- 172.21.15.64:0/1169313958 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 46) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fbb480b1d20 con 0x55f9333de0d0 2023-09-13T03:19:36.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.186+0000 7fbb6d295b80 10 client.26635 fetch_fsmap finished waiting for FSMap version 46 2023-09-13T03:19:36.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.186+0000 7fbb6d295b80 10 client.26635 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:19:36.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.186+0000 7fbb6d295b80 20 client.26635 populate_metadata read hostname 'smithi064' 2023-09-13T03:19:36.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.186+0000 7fbb6d295b80 1 --2- 172.21.15.64:0/1169313958 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f93344c3a0 0x55f93344e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:36.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.186+0000 7fbb6d295b80 4 client.26635 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:19:36.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.186+0000 7fbb6d295b80 1 -- 172.21.15.64:0/1169313958 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f9333248c0 con 0x55f93344c3a0 2023-09-13T03:19:36.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.186+0000 7fbb52ffd700 1 --2- 172.21.15.64:0/1169313958 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f93344c3a0 0x55f93344e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:36.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.186+0000 7fbb52ffd700 1 --2- 172.21.15.64:0/1169313958 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f93344c3a0 0x55f93344e7c0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a2c8d851d4dfea8 server_cookie=d423d6fca962c494 in_seq=0 out_seq=0 2023-09-13T03:19:36.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.186+0000 7fbb32ffd700 10 client.26635 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:19:36.210 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.209+0000 7fbb32ffd700 1 -- 172.21.15.64:0/1169313958 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f9333248c0 con 0x55f93344c3a0 2023-09-13T03:19:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.209+0000 7fbb32ffd700 10 client.26635 handle_command_reply: tid=0 2023-09-13T03:19:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.209+0000 7fbb6d295b80 1 client.26635 shutdown 2023-09-13T03:19:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.210+0000 7fbb6d295b80 1 -- 172.21.15.64:0/1169313958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9333de0d0 msgr2=0x55f93343fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:19:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.210+0000 7fbb6d295b80 1 --2- 172.21.15.64:0/1169313958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9333de0d0 0x55f93343fd40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbb48099540 tx=0x7fbb48098f10 comp rx=0 tx=0).stop 2023-09-13T03:19:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.210+0000 7fbb6d295b80 1 -- 172.21.15.64:0/1169313958 shutdown_connections 2023-09-13T03:19:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.210+0000 7fbb6d295b80 1 --2- 172.21.15.64:0/1169313958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9333dd710 0x55f93343f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.210+0000 7fbb6d295b80 1 --2- 172.21.15.64:0/1169313958 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f93344c3a0 0x55f93344e7c0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.210+0000 7fbb6d295b80 1 --2- 172.21.15.64:0/1169313958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9333e35b0 0x55f93344a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.210+0000 7fbb6d295b80 1 --2- 172.21.15.64:0/1169313958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9333de0d0 0x55f93343fd40 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.210+0000 7fbb32ffd700 0 client.26635 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:19:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.210+0000 7fbb6d295b80 1 -- 172.21.15.64:0/1169313958 >> 172.21.15.64:0/1169313958 conn(0x55f93334e030 msgr2=0x55f9333e1640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:19:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.210+0000 7fbb6d295b80 1 -- 172.21.15.64:0/1169313958 shutdown_connections 2023-09-13T03:19:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.210+0000 7fbb6d295b80 1 -- 172.21.15.64:0/1169313958 wait complete. 2023-09-13T03:19:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.210+0000 7fbb6d295b80 20 client.26635 trim_cache size 0 max 16384 2023-09-13T03:19:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:19:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:19:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:19:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:19:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.211+0000 7fbb599ff700 1 -- 172.21.15.64:0/1275546708 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fbb3c0a0830 msgr2=0x7fbb3c0a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:19:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.211+0000 7fbb599ff700 1 --2- 172.21.15.64:0/1275546708 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fbb3c0a0830 0x7fbb3c0a2cf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbb5419d2f0 tx=0x7fbb4c00fc00 comp rx=0 tx=0).stop 2023-09-13T03:19:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.211+0000 7fbb599ff700 1 -- 172.21.15.64:0/1275546708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb54100f60 msgr2=0x7fbb5419cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:19:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.211+0000 7fbb599ff700 1 --2- 172.21.15.64:0/1275546708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb54100f60 0x7fbb5419cb30 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800e9a0 tx=0x7fbb4800ecb0 comp rx=0 tx=0).stop 2023-09-13T03:19:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.211+0000 7fbb599ff700 1 -- 172.21.15.64:0/1275546708 shutdown_connections 2023-09-13T03:19:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.211+0000 7fbb599ff700 1 --2- 172.21.15.64:0/1275546708 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fbb3c0a0830 0x7fbb3c0a2cf0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.211+0000 7fbb599ff700 1 --2- 172.21.15.64:0/1275546708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb54106660 0x7fbb54196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.211+0000 7fbb599ff700 1 --2- 172.21.15.64:0/1275546708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb541005a0 0x7fbb5419c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.211+0000 7fbb599ff700 1 --2- 172.21.15.64:0/1275546708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb54100f60 0x7fbb5419cb30 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.211+0000 7fbb599ff700 1 -- 172.21.15.64:0/1275546708 >> 172.21.15.64:0/1275546708 conn(0x7fbb540fc050 msgr2=0x7fbb540fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:19:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.211+0000 7fbb599ff700 1 -- 172.21.15.64:0/1275546708 shutdown_connections 2023-09-13T03:19:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:36.211+0000 7fbb599ff700 1 -- 172.21.15.64:0/1275546708 wait complete. 2023-09-13T03:19:36.921 INFO:tasks.workunit.client.0.smithi064.stdout: 1 132067 1.90 MB/sec execute 221 sec latency 3346.785 ms 2023-09-13T03:19:36.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:19:36 smithi142 ceph-mon[134082]: pgmap v196: 97 pgs: 97 active+clean; 74 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 562 KiB/s wr, 92 op/s 2023-09-13T03:19:37.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:19:36 smithi170 ceph-mon[134328]: pgmap v196: 97 pgs: 97 active+clean; 74 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 562 KiB/s wr, 92 op/s 2023-09-13T03:19:37.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:19:36 smithi064 ceph-mon[97675]: pgmap v196: 97 pgs: 97 active+clean; 74 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 562 KiB/s wr, 92 op/s 2023-09-13T03:19:37.921 INFO:tasks.workunit.client.0.smithi064.stdout: 1 132067 1.90 MB/sec execute 222 sec latency 4346.941 ms 2023-09-13T03:19:38.921 INFO:tasks.workunit.client.0.smithi064.stdout: 1 132968 1.91 MB/sec execute 223 sec latency 4612.392 ms 2023-09-13T03:19:38.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:19:38 smithi142 ceph-mon[134082]: pgmap v197: 97 pgs: 97 active+clean; 74 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 340 KiB/s wr, 57 op/s 2023-09-13T03:19:39.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:19:38 smithi170 ceph-mon[134328]: pgmap v197: 97 pgs: 97 active+clean; 74 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 340 KiB/s wr, 57 op/s 2023-09-13T03:19:39.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:19:38 smithi064 ceph-mon[97675]: pgmap v197: 97 pgs: 97 active+clean; 74 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 340 KiB/s wr, 57 op/s 2023-09-13T03:19:39.921 INFO:tasks.workunit.client.0.smithi064.stdout: 1 132968 1.90 MB/sec execute 224 sec latency 1157.637 ms 2023-09-13T03:19:40.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:19:40 smithi170 ceph-mon[134328]: pgmap v198: 97 pgs: 97 active+clean; 74 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 340 KiB/s wr, 57 op/s 2023-09-13T03:19:40.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:19:40 smithi064 ceph-mon[97675]: pgmap v198: 97 pgs: 97 active+clean; 74 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 340 KiB/s wr, 57 op/s 2023-09-13T03:19:40.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:19:40 smithi142 ceph-mon[134082]: pgmap v198: 97 pgs: 97 active+clean; 74 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 340 KiB/s wr, 57 op/s 2023-09-13T03:19:40.921 INFO:tasks.workunit.client.0.smithi064.stdout: 1 134196 1.92 MB/sec execute 225 sec latency 1587.453 ms 2023-09-13T03:19:41.921 INFO:tasks.workunit.client.0.smithi064.stdout: 1 135499 1.95 MB/sec execute 226 sec latency 17.170 ms 2023-09-13T03:19:42.921 INFO:tasks.workunit.client.0.smithi064.stdout: 1 135543 1.94 MB/sec execute 227 sec latency 970.284 ms 2023-09-13T03:19:42.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:19:42 smithi142 ceph-mon[134082]: pgmap v199: 97 pgs: 97 active+clean; 70 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 1.0 MiB/s wr, 105 op/s 2023-09-13T03:19:43.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:19:42 smithi170 ceph-mon[134328]: pgmap v199: 97 pgs: 97 active+clean; 70 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 1.0 MiB/s wr, 105 op/s 2023-09-13T03:19:43.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:19:42 smithi064 ceph-mon[97675]: pgmap v199: 97 pgs: 97 active+clean; 70 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 1.0 MiB/s wr, 105 op/s 2023-09-13T03:19:43.921 INFO:tasks.workunit.client.0.smithi064.stdout: 1 135611 1.93 MB/sec execute 228 sec latency 1575.385 ms 2023-09-13T03:19:44.921 INFO:tasks.workunit.client.0.smithi064.stdout: 1 135611 1.92 MB/sec execute 229 sec latency 1341.207 ms 2023-09-13T03:19:44.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:19:44 smithi142 ceph-mon[134082]: pgmap v200: 97 pgs: 97 active+clean; 70 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 921 KiB/s wr, 80 op/s 2023-09-13T03:19:45.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:19:44 smithi170 ceph-mon[134328]: pgmap v200: 97 pgs: 97 active+clean; 70 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 921 KiB/s wr, 80 op/s 2023-09-13T03:19:45.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:19:44 smithi064 ceph-mon[97675]: pgmap v200: 97 pgs: 97 active+clean; 70 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 921 KiB/s wr, 80 op/s 2023-09-13T03:19:45.921 INFO:tasks.workunit.client.0.smithi064.stdout: 1 135611 1.91 MB/sec execute 230 sec latency 2341.297 ms 2023-09-13T03:19:46.922 INFO:tasks.workunit.client.0.smithi064.stdout: 1 135611 1.90 MB/sec execute 231 sec latency 3341.438 ms 2023-09-13T03:19:46.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:19:46 smithi142 ceph-mon[134082]: pgmap v201: 97 pgs: 97 active+clean; 72 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 1.1 MiB/s wr, 114 op/s 2023-09-13T03:19:47.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:19:46 smithi170 ceph-mon[134328]: pgmap v201: 97 pgs: 97 active+clean; 72 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 1.1 MiB/s wr, 114 op/s 2023-09-13T03:19:47.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:19:46 smithi064 ceph-mon[97675]: pgmap v201: 97 pgs: 97 active+clean; 72 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 1.1 MiB/s wr, 114 op/s 2023-09-13T03:19:47.922 INFO:tasks.workunit.client.0.smithi064.stdout: 1 135611 1.90 MB/sec execute 232 sec latency 4341.569 ms 2023-09-13T03:19:48.922 INFO:tasks.workunit.client.0.smithi064.stdout: 1 136205 1.91 MB/sec execute 233 sec latency 4947.413 ms 2023-09-13T03:19:48.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:19:48 smithi142 ceph-mon[134082]: pgmap v202: 97 pgs: 97 active+clean; 72 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 921 KiB/s wr, 81 op/s 2023-09-13T03:19:49.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:19:48 smithi170 ceph-mon[134328]: pgmap v202: 97 pgs: 97 active+clean; 72 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 921 KiB/s wr, 81 op/s 2023-09-13T03:19:49.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:19:48 smithi064 ceph-mon[97675]: pgmap v202: 97 pgs: 97 active+clean; 72 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 921 KiB/s wr, 81 op/s 2023-09-13T03:19:49.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.360+0000 7f89cb1ff700 1 -- 172.21.15.64:0/233567780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89c4106660 msgr2=0x7f89c4106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:19:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.360+0000 7f89cb1ff700 1 --2- 172.21.15.64:0/233567780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89c4106660 0x7f89c4106a40 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f89b80099c0 tx=0x7f89b8009cd0 comp rx=0 tx=0).stop 2023-09-13T03:19:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.360+0000 7f89cb1ff700 1 -- 172.21.15.64:0/233567780 shutdown_connections 2023-09-13T03:19:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.360+0000 7f89cb1ff700 1 --2- 172.21.15.64:0/233567780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89c4100f60 0x7f89c410d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.360+0000 7f89cb1ff700 1 --2- 172.21.15.64:0/233567780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89c4106660 0x7f89c4106a40 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.360+0000 7f89cb1ff700 1 --2- 172.21.15.64:0/233567780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89c41005a0 0x7f89c4100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.360+0000 7f89cb1ff700 1 -- 172.21.15.64:0/233567780 >> 172.21.15.64:0/233567780 conn(0x7f89c40fc050 msgr2=0x7f89c40fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:19:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.360+0000 7f89cb1ff700 1 -- 172.21.15.64:0/233567780 shutdown_connections 2023-09-13T03:19:49.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.360+0000 7f89cb1ff700 1 -- 172.21.15.64:0/233567780 wait complete. 2023-09-13T03:19:49.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.361+0000 7f89cb1ff700 1 Processor -- start 2023-09-13T03:19:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.361+0000 7f89cb1ff700 1 -- start start 2023-09-13T03:19:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.362+0000 7f89cb1ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89c41005a0 0x7f89c419c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.362+0000 7f89cb1ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89c4100f60 0x7f89c419cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.362+0000 7f89cb1ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89c4106660 0x7f89c4196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.362+0000 7f89cb1ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c4196ac0 con 0x7f89c41005a0 2023-09-13T03:19:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.362+0000 7f89cb1ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c4196c30 con 0x7f89c4100f60 2023-09-13T03:19:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.362+0000 7f89cb1ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c4196df0 con 0x7f89c4106660 2023-09-13T03:19:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.362+0000 7f89c8f9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89c41005a0 0x7f89c419c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.362+0000 7f89c8f9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89c41005a0 0x7f89c419c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59296/0 (socket says 172.21.15.64:59296) 2023-09-13T03:19:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.362+0000 7f89c8f9b700 1 -- 172.21.15.64:0/4092402294 learned_addr learned my addr 172.21.15.64:0/4092402294 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:19:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.362+0000 7f89c3fff700 1 --2- 172.21.15.64:0/4092402294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89c4100f60 0x7f89c419cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.362+0000 7f89c979c700 1 --2- 172.21.15.64:0/4092402294 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89c4106660 0x7f89c4196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.363+0000 7f89c8f9b700 1 -- 172.21.15.64:0/4092402294 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89c4106660 msgr2=0x7f89c4196450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:19:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.363+0000 7f89c8f9b700 1 --2- 172.21.15.64:0/4092402294 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89c4106660 0x7f89c4196450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.363+0000 7f89c8f9b700 1 -- 172.21.15.64:0/4092402294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89c4100f60 msgr2=0x7f89c419cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:19:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.363+0000 7f89c8f9b700 1 --2- 172.21.15.64:0/4092402294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89c4100f60 0x7f89c419cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.363+0000 7f89c8f9b700 1 -- 172.21.15.64:0/4092402294 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b80095f0 con 0x7f89c41005a0 2023-09-13T03:19:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.363+0000 7f89c3fff700 1 --2- 172.21.15.64:0/4092402294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89c4100f60 0x7f89c419cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:19:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.363+0000 7f89c8f9b700 1 --2- 172.21.15.64:0/4092402294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89c41005a0 0x7f89c419c5f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f89b80098e0 tx=0x7f89b800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:19:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.363+0000 7f89c1ffb700 1 -- 172.21.15.64:0/4092402294 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b801d070 con 0x7f89c41005a0 2023-09-13T03:19:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.363+0000 7f89cb1ff700 1 -- 172.21.15.64:0/4092402294 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89c4197070 con 0x7f89c41005a0 2023-09-13T03:19:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.363+0000 7f89c1ffb700 1 -- 172.21.15.64:0/4092402294 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89b8022470 con 0x7f89c41005a0 2023-09-13T03:19:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.363+0000 7f89c1ffb700 1 -- 172.21.15.64:0/4092402294 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b8017650 con 0x7f89c41005a0 2023-09-13T03:19:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.363+0000 7f89cb1ff700 1 -- 172.21.15.64:0/4092402294 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89c41a35e0 con 0x7f89c41005a0 2023-09-13T03:19:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.367+0000 7f89c1ffb700 1 -- 172.21.15.64:0/4092402294 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f89b8022850 con 0x7f89c41005a0 2023-09-13T03:19:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.369+0000 7f89deb48b80 1 Processor -- start 2023-09-13T03:19:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.369+0000 7f89deb48b80 1 -- start start 2023-09-13T03:19:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.369+0000 7f89c1ffb700 1 --2- 172.21.15.64:0/4092402294 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f89b40a4bc0 0x7f89b40a7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.369+0000 7f89c1ffb700 1 -- 172.21.15.64:0/4092402294 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f89b80cca60 con 0x7f89c41005a0 2023-09-13T03:19:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.369+0000 7f89c979c700 1 --2- 172.21.15.64:0/4092402294 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f89b40a4bc0 0x7f89b40a7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.369+0000 7f89deb48b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583f92611a0 0x5583f9261580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.369+0000 7f89deb48b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5583f925b300 0x5583f925b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.369+0000 7f89deb48b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f925bcc0 0x5583f9267f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.369+0000 7f89deb48b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583f91b46d0 con 0x5583f925bcc0 2023-09-13T03:19:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.369+0000 7f89deb48b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583f91b1ff0 con 0x5583f92611a0 2023-09-13T03:19:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.369+0000 7f89deb48b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583f91ad8e0 con 0x5583f925b300 2023-09-13T03:19:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.370+0000 7f89c8f9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583f92611a0 0x5583f9261580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.370+0000 7f89c8f9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583f92611a0 0x5583f9261580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:40108/0 (socket says 172.21.15.64:40108) 2023-09-13T03:19:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.370+0000 7f89c8f9b700 1 -- 172.21.15.64:0/3890536084 learned_addr learned my addr 172.21.15.64:0/3890536084 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:19:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.370+0000 7f89c3fff700 1 --2- 172.21.15.64:0/3890536084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5583f925b300 0x5583f925b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.370+0000 7f89c979c700 1 --2- 172.21.15.64:0/3890536084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f925bcc0 0x5583f9267f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.370+0000 7f89c8f9b700 1 -- 172.21.15.64:0/3890536084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5583f925b300 msgr2=0x5583f925b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:19:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.370+0000 7f89c8f9b700 1 --2- 172.21.15.64:0/3890536084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5583f925b300 0x5583f925b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.370+0000 7f89c8f9b700 1 -- 172.21.15.64:0/3890536084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f925bcc0 msgr2=0x5583f9267f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:19:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.370+0000 7f89c8f9b700 1 --2- 172.21.15.64:0/3890536084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f925bcc0 0x5583f9267f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.370+0000 7f89c8f9b700 1 -- 172.21.15.64:0/3890536084 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583f8fd9d80 con 0x5583f92611a0 2023-09-13T03:19:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.370+0000 7f89c979c700 1 --2- 172.21.15.64:0/4092402294 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f89b40a4bc0 0x7f89b40a7080 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f89c419d2f0 tx=0x7f89bc00fc00 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:19:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.370+0000 7f89c8f9b700 1 --2- 172.21.15.64:0/3890536084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583f92611a0 0x5583f9261580 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f89b8096b10 tx=0x7f89b80964f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9d7e4f112923c12 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:19:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.371+0000 7f89aeffd700 1 -- 172.21.15.64:0/3890536084 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b809a0a0 con 0x5583f92611a0 2023-09-13T03:19:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.371+0000 7f89aeffd700 1 -- 172.21.15.64:0/3890536084 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89b809ded0 con 0x5583f92611a0 2023-09-13T03:19:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.371+0000 7f89aeffd700 1 -- 172.21.15.64:0/3890536084 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b8099590 con 0x5583f92611a0 2023-09-13T03:19:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.371+0000 7f89deb48b80 1 -- 172.21.15.64:0/3890536084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583f92611a0 msgr2=0x5583f9261580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:19:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.371+0000 7f89deb48b80 1 --2- 172.21.15.64:0/3890536084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583f92611a0 0x5583f9261580 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f89b8096b10 tx=0x7f89b80964f0 comp rx=0 tx=0).stop 2023-09-13T03:19:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.371+0000 7f89deb48b80 1 -- 172.21.15.64:0/3890536084 shutdown_connections 2023-09-13T03:19:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.371+0000 7f89deb48b80 1 --2- 172.21.15.64:0/3890536084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583f92611a0 0x5583f9261580 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.371+0000 7f89deb48b80 1 --2- 172.21.15.64:0/3890536084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f925bcc0 0x5583f9267f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.371+0000 7f89deb48b80 1 --2- 172.21.15.64:0/3890536084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5583f925b300 0x5583f925b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.371+0000 7f89deb48b80 1 -- 172.21.15.64:0/3890536084 >> 172.21.15.64:0/3890536084 conn(0x5583f91d0030 msgr2=0x5583f9256560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:19:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.372+0000 7f89deb48b80 1 -- 172.21.15.64:0/3890536084 shutdown_connections 2023-09-13T03:19:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.372+0000 7f89deb48b80 1 -- 172.21.15.64:0/3890536084 wait complete. 2023-09-13T03:19:49.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.372+0000 7f89deb48b80 1 -- start start 2023-09-13T03:19:49.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.372+0000 7f89deb48b80 1 Processor -- start 2023-09-13T03:19:49.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.373+0000 7f89deb48b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f925b300 0x5583f926abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:49.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.373+0000 7f89deb48b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5583f925bcc0 0x5583f926b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:49.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.373+0000 7f89deb48b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583f92611a0 0x5583f9264ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:49.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.373+0000 7f89deb48b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583f914bb10 con 0x5583f925b300 2023-09-13T03:19:49.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.373+0000 7f89deb48b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583f914b810 con 0x5583f92611a0 2023-09-13T03:19:49.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.373+0000 7f89deb48b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583f902f8c0 con 0x5583f925bcc0 2023-09-13T03:19:49.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.373+0000 7f89deb48b80 10 client.0 _ll_register_callbacks cb 0x5583f9095a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:19:49.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.373+0000 7f89c8f9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f925b300 0x5583f926abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:49.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.373+0000 7f89c979c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583f92611a0 0x5583f9264ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:49.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.373+0000 7f89c3fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5583f925bcc0 0x5583f926b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:49.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.373+0000 7f89c8f9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f925b300 0x5583f926abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59318/0 (socket says 172.21.15.64:59318) 2023-09-13T03:19:49.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.373+0000 7f89c8f9b700 1 -- 172.21.15.64:0/4285103676 learned_addr learned my addr 172.21.15.64:0/4285103676 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:19:49.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.374+0000 7f89c3fff700 1 -- 172.21.15.64:0/4285103676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583f92611a0 msgr2=0x5583f9264ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:19:49.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.374+0000 7f89c3fff700 1 --2- 172.21.15.64:0/4285103676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583f92611a0 0x5583f9264ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:49.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.374+0000 7f89c3fff700 1 -- 172.21.15.64:0/4285103676 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f925b300 msgr2=0x5583f926abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:19:49.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.374+0000 7f89c3fff700 1 --2- 172.21.15.64:0/4285103676 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f925b300 0x5583f926abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.374+0000 7f89c3fff700 1 -- 172.21.15.64:0/4285103676 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89c41a35e0 con 0x5583f925bcc0 2023-09-13T03:19:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.374+0000 7f89c8f9b700 1 --2- 172.21.15.64:0/4285103676 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f925b300 0x5583f926abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:19:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.374+0000 7f89c3fff700 1 --2- 172.21.15.64:0/4285103676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5583f925bcc0 0x5583f926b120 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f89c4105f20 tx=0x7f89b0009d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78880022ef63b6ff server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:19:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.374+0000 7f89a3fff700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:19:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.374+0000 7f89a3fff700 1 -- 172.21.15.64:0/4285103676 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b0017070 con 0x5583f925bcc0 2023-09-13T03:19:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.374+0000 7f89deb48b80 1 -- 172.21.15.64:0/4285103676 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5583f9105490 con 0x5583f925bcc0 2023-09-13T03:19:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.374+0000 7f89a3fff700 1 -- 172.21.15.64:0/4285103676 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f89b0004050 con 0x5583f925bcc0 2023-09-13T03:19:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.374+0000 7f89a3fff700 1 -- 172.21.15.64:0/4285103676 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89b0004210 con 0x5583f925bcc0 2023-09-13T03:19:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.374+0000 7f89deb48b80 10 client.26647 fetch_fsmap learned FSMap version 47 2023-09-13T03:19:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.374+0000 7f89deb48b80 1 -- 172.21.15.64:0/4285103676 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x5583f8fc2c00 con 0x5583f925bcc0 2023-09-13T03:19:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.375+0000 7f89a3fff700 1 -- 172.21.15.64:0/4285103676 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5583f9105490 con 0x5583f925bcc0 2023-09-13T03:19:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.375+0000 7f89a3fff700 1 -- 172.21.15.64:0/4285103676 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 47) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f89b00037a0 con 0x5583f925bcc0 2023-09-13T03:19:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.375+0000 7f89deb48b80 10 client.26647 fetch_fsmap finished waiting for FSMap version 47 2023-09-13T03:19:49.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.375+0000 7f89deb48b80 10 client.26647 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:19:49.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.375+0000 7f89deb48b80 20 client.26647 populate_metadata read hostname 'smithi064' 2023-09-13T03:19:49.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.375+0000 7f89deb48b80 1 --2- 172.21.15.64:0/4285103676 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5583f92750f0 0x5583f9277510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:49.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.375+0000 7f89deb48b80 4 client.26647 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:19:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.375+0000 7f89deb48b80 1 -- 172.21.15.64:0/4285103676 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5583f91c2840 con 0x5583f92750f0 2023-09-13T03:19:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.375+0000 7f89c8f9b700 1 --2- 172.21.15.64:0/4285103676 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5583f92750f0 0x5583f9277510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.376+0000 7f89c8f9b700 1 --2- 172.21.15.64:0/4285103676 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5583f92750f0 0x5583f9277510 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5891d3e6b997d739 server_cookie=9340d3ccbd21414 in_seq=0 out_seq=0 2023-09-13T03:19:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.376+0000 7f89a3fff700 10 client.26647 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:19:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.376+0000 7f89a3fff700 1 -- 172.21.15.64:0/4285103676 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5583f91c2840 con 0x5583f92750f0 2023-09-13T03:19:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.376+0000 7f89a3fff700 10 client.26647 handle_command_reply: tid=0 2023-09-13T03:19:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.377+0000 7f89deb48b80 1 client.26647 shutdown 2023-09-13T03:19:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.377+0000 7f89deb48b80 1 -- 172.21.15.64:0/4285103676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5583f925bcc0 msgr2=0x5583f926b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:19:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.377+0000 7f89deb48b80 1 --2- 172.21.15.64:0/4285103676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5583f925bcc0 0x5583f926b120 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f89c4105f20 tx=0x7f89b0009d40 comp rx=0 tx=0).stop 2023-09-13T03:19:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.377+0000 7f89deb48b80 1 -- 172.21.15.64:0/4285103676 shutdown_connections 2023-09-13T03:19:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.377+0000 7f89deb48b80 1 --2- 172.21.15.64:0/4285103676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583f92611a0 0x5583f9264ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.377+0000 7f89deb48b80 1 --2- 172.21.15.64:0/4285103676 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5583f92750f0 0x5583f9277510 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.377+0000 7f89deb48b80 1 --2- 172.21.15.64:0/4285103676 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f925b300 0x5583f926abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.377+0000 7f89deb48b80 1 --2- 172.21.15.64:0/4285103676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5583f925bcc0 0x5583f926b120 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.377+0000 7f89a3fff700 0 client.26647 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:19:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.377+0000 7f89deb48b80 1 -- 172.21.15.64:0/4285103676 >> 172.21.15.64:0/4285103676 conn(0x5583f91d0030 msgr2=0x5583f92557a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:19:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.377+0000 7f89deb48b80 1 -- 172.21.15.64:0/4285103676 shutdown_connections 2023-09-13T03:19:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.377+0000 7f89deb48b80 1 -- 172.21.15.64:0/4285103676 wait complete. 2023-09-13T03:19:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.377+0000 7f89deb48b80 20 client.26647 trim_cache size 0 max 16384 2023-09-13T03:19:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.392+0000 7f89deb48b80 1 Processor -- start 2023-09-13T03:19:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.392+0000 7f89deb48b80 1 -- start start 2023-09-13T03:19:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.393+0000 7f89deb48b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5583f925b300 0x5583f9260980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:49.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.393+0000 7f89deb48b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f925bcc0 0x5583f925aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:49.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.393+0000 7f89deb48b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583f92611a0 0x5583f9255f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:49.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.393+0000 7f89deb48b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583f9265e20 con 0x5583f925bcc0 2023-09-13T03:19:49.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.393+0000 7f89deb48b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583f9265300 con 0x5583f92611a0 2023-09-13T03:19:49.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.393+0000 7f89deb48b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583f9257d10 con 0x5583f925b300 2023-09-13T03:19:49.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.393+0000 7f89c3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f925bcc0 0x5583f925aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:49.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.393+0000 7f89c8f9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5583f925b300 0x5583f9260980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:49.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.393+0000 7f89c3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f925bcc0 0x5583f925aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59324/0 (socket says 172.21.15.64:59324) 2023-09-13T03:19:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.393+0000 7f89c3fff700 1 -- 172.21.15.64:0/3598833433 learned_addr learned my addr 172.21.15.64:0/3598833433 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:19:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.393+0000 7f89c8f9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5583f925b300 0x5583f9260980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:36216/0 (socket says 172.21.15.64:36216) 2023-09-13T03:19:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.393+0000 7f89c979c700 1 --2- 172.21.15.64:0/3598833433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583f92611a0 0x5583f9255f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.393+0000 7f89c8f9b700 1 -- 172.21.15.64:0/3598833433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583f92611a0 msgr2=0x5583f9255f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:19:49.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.393+0000 7f89c8f9b700 1 --2- 172.21.15.64:0/3598833433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583f92611a0 0x5583f9255f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:49.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.393+0000 7f89c8f9b700 1 -- 172.21.15.64:0/3598833433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f925bcc0 msgr2=0x5583f925aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:19:49.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.393+0000 7f89c8f9b700 1 --2- 172.21.15.64:0/3598833433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f925bcc0 0x5583f925aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:49.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.393+0000 7f89c8f9b700 1 -- 172.21.15.64:0/3598833433 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583f90001f0 con 0x5583f925b300 2023-09-13T03:19:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.393+0000 7f89c3fff700 1 --2- 172.21.15.64:0/3598833433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f925bcc0 0x5583f925aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:19:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.393+0000 7f89c8f9b700 1 --2- 172.21.15.64:0/3598833433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5583f925b300 0x5583f9260980 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f89b80a1b50 tx=0x7f89b80a0040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a73c9c96dba8ffbc server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:19:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.394+0000 7f89af7fe700 1 -- 172.21.15.64:0/3598833433 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b809a0a0 con 0x5583f925b300 2023-09-13T03:19:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.394+0000 7f89af7fe700 1 -- 172.21.15.64:0/3598833433 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89b8098440 con 0x5583f925b300 2023-09-13T03:19:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.394+0000 7f89af7fe700 1 -- 172.21.15.64:0/3598833433 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b80a1e50 con 0x5583f925b300 2023-09-13T03:19:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.394+0000 7f89deb48b80 1 -- 172.21.15.64:0/3598833433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5583f925b300 msgr2=0x5583f9260980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:19:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.394+0000 7f89deb48b80 1 --2- 172.21.15.64:0/3598833433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5583f925b300 0x5583f9260980 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f89b80a1b50 tx=0x7f89b80a0040 comp rx=0 tx=0).stop 2023-09-13T03:19:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.394+0000 7f89deb48b80 1 -- 172.21.15.64:0/3598833433 shutdown_connections 2023-09-13T03:19:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.394+0000 7f89deb48b80 1 --2- 172.21.15.64:0/3598833433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583f92611a0 0x5583f9255f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.394+0000 7f89deb48b80 1 --2- 172.21.15.64:0/3598833433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5583f925b300 0x5583f9260980 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.394+0000 7f89deb48b80 1 --2- 172.21.15.64:0/3598833433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f925bcc0 0x5583f925aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.394+0000 7f89deb48b80 1 -- 172.21.15.64:0/3598833433 >> 172.21.15.64:0/3598833433 conn(0x5583f91d0030 msgr2=0x5583f925a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:19:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.394+0000 7f89deb48b80 1 -- 172.21.15.64:0/3598833433 shutdown_connections 2023-09-13T03:19:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.394+0000 7f89deb48b80 1 -- 172.21.15.64:0/3598833433 wait complete. 2023-09-13T03:19:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.395+0000 7f89deb48b80 1 -- start start 2023-09-13T03:19:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.395+0000 7f89deb48b80 1 Processor -- start 2023-09-13T03:19:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.395+0000 7f89deb48b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f925b300 0x5583f9268e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:49.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.395+0000 7f89deb48b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5583f925bcc0 0x5583f92693c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:49.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.395+0000 7f89deb48b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583f92611a0 0x5583f92c1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:49.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.395+0000 7f89deb48b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583f92c24c0 con 0x5583f925b300 2023-09-13T03:19:49.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.395+0000 7f89deb48b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583f92c2630 con 0x5583f92611a0 2023-09-13T03:19:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.395+0000 7f89deb48b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583f92c27f0 con 0x5583f925bcc0 2023-09-13T03:19:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.395+0000 7f89deb48b80 10 client.0 _ll_register_callbacks cb 0x7f89b00127f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:19:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.396+0000 7f89c8f9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f925b300 0x5583f9268e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.396+0000 7f89c8f9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f925b300 0x5583f9268e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59332/0 (socket says 172.21.15.64:59332) 2023-09-13T03:19:49.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.396+0000 7f89c8f9b700 1 -- 172.21.15.64:0/1290548905 learned_addr learned my addr 172.21.15.64:0/1290548905 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:19:49.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.396+0000 7f89c3fff700 1 --2- 172.21.15.64:0/1290548905 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5583f925bcc0 0x5583f92693c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:49.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.396+0000 7f89c979c700 1 --2- 172.21.15.64:0/1290548905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583f92611a0 0x5583f92c1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:49.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.396+0000 7f89c8f9b700 1 -- 172.21.15.64:0/1290548905 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5583f925bcc0 msgr2=0x5583f92693c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:19:49.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.396+0000 7f89c8f9b700 1 --2- 172.21.15.64:0/1290548905 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5583f925bcc0 0x5583f92693c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:49.418 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.396+0000 7f89c8f9b700 1 -- 172.21.15.64:0/1290548905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583f92611a0 msgr2=0x5583f92c1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:19:49.418 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.396+0000 7f89c8f9b700 1 --2- 172.21.15.64:0/1290548905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583f92611a0 0x5583f92c1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:49.418 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.396+0000 7f89c8f9b700 1 -- 172.21.15.64:0/1290548905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5583f8fc2c00 con 0x5583f925b300 2023-09-13T03:19:49.418 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.396+0000 7f89c3fff700 1 --2- 172.21.15.64:0/1290548905 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5583f925bcc0 0x5583f92693c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:19:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.396+0000 7f89c8f9b700 1 --2- 172.21.15.64:0/1290548905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f925b300 0x5583f9268e80 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f89b809a030 tx=0x7f89b8005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf320a073fc5cd9 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:19:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.396+0000 7f89a3fff700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:19:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.396+0000 7f89a3fff700 1 -- 172.21.15.64:0/1290548905 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b809a500 con 0x5583f925b300 2023-09-13T03:19:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.397+0000 7f89a3fff700 1 -- 172.21.15.64:0/1290548905 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f89b80a1bf0 con 0x5583f925b300 2023-09-13T03:19:49.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.397+0000 7f89deb48b80 1 -- 172.21.15.64:0/1290548905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f89b809ded0 con 0x5583f925b300 2023-09-13T03:19:49.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.397+0000 7f89a3fff700 1 -- 172.21.15.64:0/1290548905 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89b80a1db0 con 0x5583f925b300 2023-09-13T03:19:49.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.397+0000 7f89deb48b80 10 client.16629 fetch_fsmap learned FSMap version 47 2023-09-13T03:19:49.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.397+0000 7f89deb48b80 1 -- 172.21.15.64:0/1290548905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x5583f8fd9d80 con 0x5583f925b300 2023-09-13T03:19:49.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.397+0000 7f89a3fff700 1 -- 172.21.15.64:0/1290548905 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89b809ded0 con 0x5583f925b300 2023-09-13T03:19:49.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.397+0000 7f89a3fff700 1 -- 172.21.15.64:0/1290548905 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 47) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f89b800fc70 con 0x5583f925b300 2023-09-13T03:19:49.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.397+0000 7f89deb48b80 10 client.16629 fetch_fsmap finished waiting for FSMap version 47 2023-09-13T03:19:49.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.397+0000 7f89deb48b80 10 client.16629 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:19:49.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.397+0000 7f89deb48b80 20 client.16629 populate_metadata read hostname 'smithi064' 2023-09-13T03:19:49.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.397+0000 7f89deb48b80 1 --2- 172.21.15.64:0/1290548905 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5583f92c9b50 0x5583f92cbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:19:49.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.397+0000 7f89deb48b80 4 client.16629 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:19:49.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.397+0000 7f89deb48b80 1 -- 172.21.15.64:0/1290548905 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5583f91a68c0 con 0x5583f92c9b50 2023-09-13T03:19:49.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.398+0000 7f89c3fff700 1 --2- 172.21.15.64:0/1290548905 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5583f92c9b50 0x5583f92cbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:19:49.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.398+0000 7f89c3fff700 1 --2- 172.21.15.64:0/1290548905 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5583f92c9b50 0x5583f92cbf70 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d217cb14e7cfdb10 server_cookie=639373783cb77712 in_seq=0 out_seq=0 2023-09-13T03:19:49.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.398+0000 7f89a3fff700 10 client.16629 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:19:49.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.418+0000 7f89a3fff700 1 -- 172.21.15.64:0/1290548905 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5583f91a68c0 con 0x5583f92c9b50 2023-09-13T03:19:49.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.418+0000 7f89a3fff700 10 client.16629 handle_command_reply: tid=0 2023-09-13T03:19:49.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.418+0000 7f89deb48b80 1 client.16629 shutdown 2023-09-13T03:19:49.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.419+0000 7f89deb48b80 1 -- 172.21.15.64:0/1290548905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f925b300 msgr2=0x5583f9268e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:19:49.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.419+0000 7f89deb48b80 1 --2- 172.21.15.64:0/1290548905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f925b300 0x5583f9268e80 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f89b809a030 tx=0x7f89b8005e80 comp rx=0 tx=0).stop 2023-09-13T03:19:49.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.419+0000 7f89deb48b80 1 -- 172.21.15.64:0/1290548905 shutdown_connections 2023-09-13T03:19:49.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.419+0000 7f89deb48b80 1 --2- 172.21.15.64:0/1290548905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583f92611a0 0x5583f92c1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:49.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.419+0000 7f89deb48b80 1 --2- 172.21.15.64:0/1290548905 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5583f92c9b50 0x5583f92cbf70 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:49.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.419+0000 7f89deb48b80 1 --2- 172.21.15.64:0/1290548905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f925b300 0x5583f9268e80 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:49.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.419+0000 7f89deb48b80 1 --2- 172.21.15.64:0/1290548905 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5583f925bcc0 0x5583f92693c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:49.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.419+0000 7f89a3fff700 0 client.16629 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:19:49.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.419+0000 7f89deb48b80 1 -- 172.21.15.64:0/1290548905 >> 172.21.15.64:0/1290548905 conn(0x5583f91d0030 msgr2=0x5583f92607f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:19:49.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.419+0000 7f89deb48b80 1 -- 172.21.15.64:0/1290548905 shutdown_connections 2023-09-13T03:19:49.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.419+0000 7f89deb48b80 1 -- 172.21.15.64:0/1290548905 wait complete. 2023-09-13T03:19:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.419+0000 7f89deb48b80 20 client.16629 trim_cache size 0 max 16384 2023-09-13T03:19:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:19:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:19:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:19:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:19:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.420+0000 7f89cb1ff700 1 -- 172.21.15.64:0/4092402294 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f89b40a4bc0 msgr2=0x7f89b40a7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:19:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.420+0000 7f89cb1ff700 1 --2- 172.21.15.64:0/4092402294 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f89b40a4bc0 0x7f89b40a7080 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f89c419d2f0 tx=0x7f89bc00fc00 comp rx=0 tx=0).stop 2023-09-13T03:19:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.420+0000 7f89cb1ff700 1 -- 172.21.15.64:0/4092402294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89c41005a0 msgr2=0x7f89c419c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:19:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.420+0000 7f89cb1ff700 1 --2- 172.21.15.64:0/4092402294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89c41005a0 0x7f89c419c5f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f89b80098e0 tx=0x7f89b800f690 comp rx=0 tx=0).stop 2023-09-13T03:19:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.420+0000 7f89cb1ff700 1 -- 172.21.15.64:0/4092402294 shutdown_connections 2023-09-13T03:19:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.420+0000 7f89cb1ff700 1 --2- 172.21.15.64:0/4092402294 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f89b40a4bc0 0x7f89b40a7080 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.420+0000 7f89cb1ff700 1 --2- 172.21.15.64:0/4092402294 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89c4106660 0x7f89c4196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.420+0000 7f89cb1ff700 1 --2- 172.21.15.64:0/4092402294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89c41005a0 0x7f89c419c5f0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.420+0000 7f89cb1ff700 1 --2- 172.21.15.64:0/4092402294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89c4100f60 0x7f89c419cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:19:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.420+0000 7f89cb1ff700 1 -- 172.21.15.64:0/4092402294 >> 172.21.15.64:0/4092402294 conn(0x7f89c40fc050 msgr2=0x7f89c40fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:19:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.420+0000 7f89cb1ff700 1 -- 172.21.15.64:0/4092402294 shutdown_connections 2023-09-13T03:19:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:19:49.420+0000 7f89cb1ff700 1 -- 172.21.15.64:0/4092402294 wait complete. 2023-09-13T03:19:49.922 INFO:tasks.workunit.client.0.smithi064.stdout: 1 136512 1.90 MB/sec execute 234 sec latency 759.229 ms 2023-09-13T03:19:49.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:19:49 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:19:50.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:19:49 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:19:50.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:19:49 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:19:50.922 INFO:tasks.workunit.client.0.smithi064.stdout: 1 137748 1.92 MB/sec execute 235 sec latency 1188.591 ms 2023-09-13T03:19:50.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:19:50 smithi142 ceph-mon[134082]: pgmap v203: 97 pgs: 97 active+clean; 72 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 921 KiB/s wr, 81 op/s 2023-09-13T03:19:51.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:19:50 smithi170 ceph-mon[134328]: pgmap v203: 97 pgs: 97 active+clean; 72 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 921 KiB/s wr, 81 op/s 2023-09-13T03:19:51.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:19:50 smithi064 ceph-mon[97675]: pgmap v203: 97 pgs: 97 active+clean; 72 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 921 KiB/s wr, 81 op/s 2023-09-13T03:19:51.922 INFO:tasks.workunit.client.0.smithi064.stdout: 1 139034 1.94 MB/sec execute 236 sec latency 17.265 ms 2023-09-13T03:19:52.922 INFO:tasks.workunit.client.0.smithi064.stdout: 1 139034 1.94 MB/sec execute 237 sec latency 1005.916 ms 2023-09-13T03:19:52.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:19:52 smithi142 ceph-mon[134082]: pgmap v204: 97 pgs: 97 active+clean; 74 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 1.0 MiB/s wr, 108 op/s 2023-09-13T03:19:53.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:19:52 smithi170 ceph-mon[134328]: pgmap v204: 97 pgs: 97 active+clean; 74 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 1.0 MiB/s wr, 108 op/s 2023-09-13T03:19:53.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:19:52 smithi064 ceph-mon[97675]: pgmap v204: 97 pgs: 97 active+clean; 74 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 1.0 MiB/s wr, 108 op/s 2023-09-13T03:19:53.922 INFO:tasks.workunit.client.0.smithi064.stdout: 1 139090 1.93 MB/sec execute 238 sec latency 1613.080 ms 2023-09-13T03:19:54.922 INFO:tasks.workunit.client.0.smithi064.stdout: 1 139090 1.92 MB/sec execute 239 sec latency 1347.229 ms 2023-09-13T03:19:54.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:19:54 smithi142 ceph-mon[134082]: pgmap v205: 97 pgs: 97 active+clean; 74 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 354 KiB/s wr, 60 op/s 2023-09-13T03:19:55.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:19:54 smithi170 ceph-mon[134328]: pgmap v205: 97 pgs: 97 active+clean; 74 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 354 KiB/s wr, 60 op/s 2023-09-13T03:19:55.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:19:54 smithi064 ceph-mon[97675]: pgmap v205: 97 pgs: 97 active+clean; 74 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 354 KiB/s wr, 60 op/s 2023-09-13T03:19:55.923 INFO:tasks.workunit.client.0.smithi064.stdout: 1 139090 1.91 MB/sec execute 240 sec latency 2347.352 ms 2023-09-13T03:19:56.923 INFO:tasks.workunit.client.0.smithi064.stdout: 1 139090 1.90 MB/sec execute 241 sec latency 3347.469 ms 2023-09-13T03:19:56.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:19:56 smithi142 ceph-mon[134082]: pgmap v206: 97 pgs: 97 active+clean; 76 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 542 KiB/s wr, 90 op/s 2023-09-13T03:19:57.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:19:56 smithi170 ceph-mon[134328]: pgmap v206: 97 pgs: 97 active+clean; 76 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 542 KiB/s wr, 90 op/s 2023-09-13T03:19:57.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:19:56 smithi064 ceph-mon[97675]: pgmap v206: 97 pgs: 97 active+clean; 76 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 542 KiB/s wr, 90 op/s 2023-09-13T03:19:57.923 INFO:tasks.workunit.client.0.smithi064.stdout: 1 139090 1.90 MB/sec execute 242 sec latency 4347.621 ms 2023-09-13T03:19:58.923 INFO:tasks.workunit.client.0.smithi064.stdout: 1 139777 1.91 MB/sec execute 243 sec latency 4955.176 ms 2023-09-13T03:19:58.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:19:58 smithi142 ceph-mon[134082]: pgmap v207: 97 pgs: 97 active+clean; 76 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 330 KiB/s wr, 56 op/s 2023-09-13T03:19:59.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:19:58 smithi170 ceph-mon[134328]: pgmap v207: 97 pgs: 97 active+clean; 76 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 330 KiB/s wr, 56 op/s 2023-09-13T03:19:59.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:19:58 smithi064 ceph-mon[97675]: pgmap v207: 97 pgs: 97 active+clean; 76 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 330 KiB/s wr, 56 op/s 2023-09-13T03:19:59.923 INFO:tasks.workunit.client.0.smithi064.stdout: 1 140059 1.90 MB/sec execute 244 sec latency 753.020 ms 2023-09-13T03:20:00.342 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:00 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mon-a[97651]: 2023-09-13T03:19:59.999+0000 7f9fd6939700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2023-09-13T03:20:00.342 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:00 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mon-a[97651]: 2023-09-13T03:19:59.999+0000 7f9fd6939700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2023-09-13T03:20:00.342 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:00 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mon-a[97651]: 2023-09-13T03:19:59.999+0000 7f9fd6939700 -1 log_channel(cluster) log [ERR] : mds.h(mds.2): Metadata damage detected 2023-09-13T03:20:00.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:20:00 smithi170 ceph-mon[134328]: pgmap v208: 97 pgs: 97 active+clean; 76 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 330 KiB/s wr, 56 op/s 2023-09-13T03:20:00.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:20:00 smithi170 ceph-mon[134328]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2023-09-13T03:20:00.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:20:00 smithi170 ceph-mon[134328]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2023-09-13T03:20:00.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:20:00 smithi170 ceph-mon[134328]: mds.h(mds.2): Metadata damage detected 2023-09-13T03:20:00.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:00 smithi064 ceph-mon[97675]: pgmap v208: 97 pgs: 97 active+clean; 76 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 330 KiB/s wr, 56 op/s 2023-09-13T03:20:00.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:00 smithi064 ceph-mon[97675]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2023-09-13T03:20:00.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:00 smithi064 ceph-mon[97675]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2023-09-13T03:20:00.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:00 smithi064 ceph-mon[97675]: mds.h(mds.2): Metadata damage detected 2023-09-13T03:20:00.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:20:00 smithi142 ceph-mon[134082]: pgmap v208: 97 pgs: 97 active+clean; 76 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 330 KiB/s wr, 56 op/s 2023-09-13T03:20:00.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:20:00 smithi142 ceph-mon[134082]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2023-09-13T03:20:00.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:20:00 smithi142 ceph-mon[134082]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2023-09-13T03:20:00.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:20:00 smithi142 ceph-mon[134082]: mds.h(mds.2): Metadata damage detected 2023-09-13T03:20:00.923 INFO:tasks.workunit.client.0.smithi064.stdout: 1 140691 1.90 MB/sec execute 245 sec latency 1183.710 ms 2023-09-13T03:20:01.924 INFO:tasks.workunit.client.0.smithi064.stdout: 1 142214 1.94 MB/sec execute 246 sec latency 23.069 ms 2023-09-13T03:20:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.569+0000 7ff0365ff700 1 -- 172.21.15.64:0/3130420268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff030106660 msgr2=0x7ff030106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.569+0000 7ff0365ff700 1 --2- 172.21.15.64:0/3130420268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff030106660 0x7ff030106a40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff020009990 tx=0x7ff020009ca0 comp rx=0 tx=0).stop 2023-09-13T03:20:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.569+0000 7ff0365ff700 1 -- 172.21.15.64:0/3130420268 shutdown_connections 2023-09-13T03:20:02.574 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.569+0000 7ff0365ff700 1 --2- 172.21.15.64:0/3130420268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff030100f60 0x7ff03010d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:02.574 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.569+0000 7ff0365ff700 1 --2- 172.21.15.64:0/3130420268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff030106660 0x7ff030106a40 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:02.574 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.569+0000 7ff0365ff700 1 --2- 172.21.15.64:0/3130420268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0301005a0 0x7ff030100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:02.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.569+0000 7ff0365ff700 1 -- 172.21.15.64:0/3130420268 >> 172.21.15.64:0/3130420268 conn(0x7ff0300fc050 msgr2=0x7ff0300fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:20:02.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.569+0000 7ff0365ff700 1 -- 172.21.15.64:0/3130420268 shutdown_connections 2023-09-13T03:20:02.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.569+0000 7ff0365ff700 1 -- 172.21.15.64:0/3130420268 wait complete. 2023-09-13T03:20:02.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.570+0000 7ff0365ff700 1 Processor -- start 2023-09-13T03:20:02.576 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.570+0000 7ff0365ff700 1 -- start start 2023-09-13T03:20:02.576 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.570+0000 7ff0365ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0301005a0 0x7ff030072ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:02.576 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.570+0000 7ff0365ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff030100f60 0x7ff030073020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.571+0000 7ff0365ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff030106660 0x7ff0301a4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.571+0000 7ff0365ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0301a4d80 con 0x7ff0301005a0 2023-09-13T03:20:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.571+0000 7ff0365ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0301a4ec0 con 0x7ff030100f60 2023-09-13T03:20:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.571+0000 7ff0365ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0301a5030 con 0x7ff030106660 2023-09-13T03:20:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.571+0000 7ff02ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0301005a0 0x7ff030072ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.571+0000 7ff02ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0301005a0 0x7ff030072ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41850/0 (socket says 172.21.15.64:41850) 2023-09-13T03:20:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.571+0000 7ff02ffff700 1 -- 172.21.15.64:0/1973032832 learned_addr learned my addr 172.21.15.64:0/1973032832 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:20:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.571+0000 7ff02f7fe700 1 --2- 172.21.15.64:0/1973032832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff030100f60 0x7ff030073020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.571+0000 7ff034b9c700 1 --2- 172.21.15.64:0/1973032832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff030106660 0x7ff0301a4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.571+0000 7ff02ffff700 1 -- 172.21.15.64:0/1973032832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff030106660 msgr2=0x7ff0301a4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.571+0000 7ff02ffff700 1 --2- 172.21.15.64:0/1973032832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff030106660 0x7ff0301a4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.571+0000 7ff02ffff700 1 -- 172.21.15.64:0/1973032832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff030100f60 msgr2=0x7ff030073020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.571+0000 7ff02ffff700 1 --2- 172.21.15.64:0/1973032832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff030100f60 0x7ff030073020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.571+0000 7ff02ffff700 1 -- 172.21.15.64:0/1973032832 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0200095f0 con 0x7ff0301005a0 2023-09-13T03:20:02.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.571+0000 7ff02f7fe700 1 --2- 172.21.15.64:0/1973032832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff030100f60 0x7ff030073020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:20:02.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.571+0000 7ff034b9c700 1 --2- 172.21.15.64:0/1973032832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff030106660 0x7ff0301a4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:20:02.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.571+0000 7ff02ffff700 1 --2- 172.21.15.64:0/1973032832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0301005a0 0x7ff030072ae0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff020005910 tx=0x7ff0200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:20:02.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.572+0000 7ff02d7fa700 1 -- 172.21.15.64:0/1973032832 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02001d070 con 0x7ff0301005a0 2023-09-13T03:20:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.572+0000 7ff02d7fa700 1 -- 172.21.15.64:0/1973032832 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff02000fb80 con 0x7ff0301005a0 2023-09-13T03:20:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.572+0000 7ff0365ff700 1 -- 172.21.15.64:0/1973032832 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0301a52b0 con 0x7ff0301005a0 2023-09-13T03:20:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.572+0000 7ff02d7fa700 1 -- 172.21.15.64:0/1973032832 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff020017650 con 0x7ff0301005a0 2023-09-13T03:20:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.572+0000 7ff0365ff700 1 -- 172.21.15.64:0/1973032832 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0301a56d0 con 0x7ff0301005a0 2023-09-13T03:20:02.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.575+0000 7ff02d7fa700 1 -- 172.21.15.64:0/1973032832 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7ff02000fcf0 con 0x7ff0301005a0 2023-09-13T03:20:02.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.576+0000 7ff049ed7b80 1 Processor -- start 2023-09-13T03:20:02.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.576+0000 7ff02d7fa700 1 --2- 172.21.15.64:0/1973032832 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7ff0180a07b0 0x7ff0180a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:02.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.577+0000 7ff049ed7b80 1 -- start start 2023-09-13T03:20:02.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.577+0000 7ff02d7fa700 1 -- 172.21.15.64:0/1973032832 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7ff0200cd7a0 con 0x7ff0301005a0 2023-09-13T03:20:02.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.577+0000 7ff034b9c700 1 --2- 172.21.15.64:0/1973032832 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7ff0180a07b0 0x7ff0180a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:02.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.577+0000 7ff049ed7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fcc1c8b1a0 0x55fcc1c8b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:02.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.577+0000 7ff049ed7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcc1c85300 0x55fcc1c85780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:02.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.577+0000 7ff049ed7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fcc1c85cc0 0x55fcc1c91f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:02.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.577+0000 7ff049ed7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc1bde6d0 con 0x55fcc1c8b1a0 2023-09-13T03:20:02.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.577+0000 7ff049ed7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc1bdbff0 con 0x55fcc1c85300 2023-09-13T03:20:02.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.577+0000 7ff049ed7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc1bd78e0 con 0x55fcc1c85cc0 2023-09-13T03:20:02.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.577+0000 7ff02ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fcc1c8b1a0 0x55fcc1c8b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:02.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.577+0000 7ff02ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fcc1c8b1a0 0x55fcc1c8b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41854/0 (socket says 172.21.15.64:41854) 2023-09-13T03:20:02.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.577+0000 7ff02ffff700 1 -- 172.21.15.64:0/2067687154 learned_addr learned my addr 172.21.15.64:0/2067687154 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:20:02.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.577+0000 7ff02f7fe700 1 --2- 172.21.15.64:0/2067687154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcc1c85300 0x55fcc1c85780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:02.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.577+0000 7ff034b9c700 1 --2- 172.21.15.64:0/2067687154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fcc1c85cc0 0x55fcc1c91f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:02.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.578+0000 7ff034b9c700 1 --2- 172.21.15.64:0/1973032832 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7ff0180a07b0 0x7ff0180a2c70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff0280093f0 tx=0x7ff02800b040 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:20:02.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.578+0000 7ff02ffff700 1 -- 172.21.15.64:0/2067687154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fcc1c85cc0 msgr2=0x55fcc1c91f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:20:02.599 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.578+0000 7ff02ffff700 1 --2- 172.21.15.64:0/2067687154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fcc1c85cc0 0x55fcc1c91f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:02.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.578+0000 7ff02ffff700 1 -- 172.21.15.64:0/2067687154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcc1c85300 msgr2=0x55fcc1c85780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:20:02.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.578+0000 7ff02ffff700 1 --2- 172.21.15.64:0/2067687154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcc1c85300 0x55fcc1c85780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:02.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.578+0000 7ff02ffff700 1 -- 172.21.15.64:0/2067687154 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fcc1a03d80 con 0x55fcc1c8b1a0 2023-09-13T03:20:02.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.578+0000 7ff02ffff700 1 --2- 172.21.15.64:0/2067687154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fcc1c8b1a0 0x55fcc1c8b580 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff020000c00 tx=0x7ff020009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dffff07fb8107b42 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:20:02.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.578+0000 7ff0167fc700 1 -- 172.21.15.64:0/2067687154 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02009adf0 con 0x55fcc1c8b1a0 2023-09-13T03:20:02.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.578+0000 7ff0167fc700 1 -- 172.21.15.64:0/2067687154 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff02009ec20 con 0x55fcc1c8b1a0 2023-09-13T03:20:02.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.578+0000 7ff0167fc700 1 -- 172.21.15.64:0/2067687154 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02009a500 con 0x55fcc1c8b1a0 2023-09-13T03:20:02.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.579+0000 7ff049ed7b80 1 -- 172.21.15.64:0/2067687154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fcc1c8b1a0 msgr2=0x55fcc1c8b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:02.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.579+0000 7ff049ed7b80 1 --2- 172.21.15.64:0/2067687154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fcc1c8b1a0 0x55fcc1c8b580 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff020000c00 tx=0x7ff020009310 comp rx=0 tx=0).stop 2023-09-13T03:20:02.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.579+0000 7ff049ed7b80 1 -- 172.21.15.64:0/2067687154 shutdown_connections 2023-09-13T03:20:02.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.579+0000 7ff049ed7b80 1 --2- 172.21.15.64:0/2067687154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fcc1c85cc0 0x55fcc1c91f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:02.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.579+0000 7ff049ed7b80 1 --2- 172.21.15.64:0/2067687154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fcc1c8b1a0 0x55fcc1c8b580 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:02.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.579+0000 7ff049ed7b80 1 --2- 172.21.15.64:0/2067687154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcc1c85300 0x55fcc1c85780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:02.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.579+0000 7ff049ed7b80 1 -- 172.21.15.64:0/2067687154 >> 172.21.15.64:0/2067687154 conn(0x55fcc1bfa030 msgr2=0x55fcc1c80560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:20:02.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.579+0000 7ff049ed7b80 1 -- 172.21.15.64:0/2067687154 shutdown_connections 2023-09-13T03:20:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.579+0000 7ff049ed7b80 1 -- 172.21.15.64:0/2067687154 wait complete. 2023-09-13T03:20:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.580+0000 7ff049ed7b80 1 -- start start 2023-09-13T03:20:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.580+0000 7ff049ed7b80 1 Processor -- start 2023-09-13T03:20:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.580+0000 7ff049ed7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fcc1c85300 0x55fcc1c94ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:02.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.580+0000 7ff049ed7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fcc1c85cc0 0x55fcc1c950e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:02.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.580+0000 7ff049ed7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcc1c8b1a0 0x55fcc1c8edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:02.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.580+0000 7ff049ed7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc1b75b10 con 0x55fcc1c85300 2023-09-13T03:20:02.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.580+0000 7ff049ed7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc1b75810 con 0x55fcc1c8b1a0 2023-09-13T03:20:02.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.580+0000 7ff049ed7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc1a598c0 con 0x55fcc1c85cc0 2023-09-13T03:20:02.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.581+0000 7ff049ed7b80 10 client.0 _ll_register_callbacks cb 0x55fcc1abfa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:20:02.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.581+0000 7ff02ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fcc1c85300 0x55fcc1c94ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:02.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.581+0000 7ff02f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fcc1c85cc0 0x55fcc1c950e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:02.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.581+0000 7ff034b9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcc1c8b1a0 0x55fcc1c8edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.581+0000 7ff02f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fcc1c85cc0 0x55fcc1c950e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:34872/0 (socket says 172.21.15.64:34872) 2023-09-13T03:20:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.581+0000 7ff034b9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcc1c8b1a0 0x55fcc1c8edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:59216/0 (socket says 172.21.15.64:59216) 2023-09-13T03:20:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.581+0000 7ff02f7fe700 1 -- 172.21.15.64:0/1327631905 learned_addr learned my addr 172.21.15.64:0/1327631905 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:20:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.581+0000 7ff034b9c700 1 -- 172.21.15.64:0/1327631905 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fcc1c85cc0 msgr2=0x55fcc1c950e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:20:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.581+0000 7ff034b9c700 1 --2- 172.21.15.64:0/1327631905 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fcc1c85cc0 0x55fcc1c950e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.581+0000 7ff034b9c700 1 -- 172.21.15.64:0/1327631905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fcc1c85300 msgr2=0x55fcc1c94ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:20:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.581+0000 7ff034b9c700 1 --2- 172.21.15.64:0/1327631905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fcc1c85300 0x55fcc1c94ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.581+0000 7ff034b9c700 1 -- 172.21.15.64:0/1327631905 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0301a56d0 con 0x55fcc1c8b1a0 2023-09-13T03:20:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.581+0000 7ff02f7fe700 1 --2- 172.21.15.64:0/1327631905 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fcc1c85cc0 0x55fcc1c950e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:20:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.581+0000 7ff02ffff700 1 --2- 172.21.15.64:0/1327631905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fcc1c85300 0x55fcc1c94ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:20:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.582+0000 7ff034b9c700 1 --2- 172.21.15.64:0/1327631905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcc1c8b1a0 0x55fcc1c8edf0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff028007b70 tx=0x7ff028004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a9f5ec4df90e2f7 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:20:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.582+0000 7ff00f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:20:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.582+0000 7ff00f7fe700 1 -- 172.21.15.64:0/1327631905 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02800e050 con 0x55fcc1c8b1a0 2023-09-13T03:20:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.582+0000 7ff049ed7b80 1 -- 172.21.15.64:0/1327631905 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fcc1b2f490 con 0x55fcc1c8b1a0 2023-09-13T03:20:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.582+0000 7ff00f7fe700 1 -- 172.21.15.64:0/1327631905 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7ff028015e10 con 0x55fcc1c8b1a0 2023-09-13T03:20:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.582+0000 7ff00f7fe700 1 -- 172.21.15.64:0/1327631905 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff028022420 con 0x55fcc1c8b1a0 2023-09-13T03:20:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.582+0000 7ff049ed7b80 10 client.26681 fetch_fsmap learned FSMap version 47 2023-09-13T03:20:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.582+0000 7ff049ed7b80 1 -- 172.21.15.64:0/1327631905 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55fcc19ecc00 con 0x55fcc1c8b1a0 2023-09-13T03:20:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.582+0000 7ff00f7fe700 1 -- 172.21.15.64:0/1327631905 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fcc1b2f490 con 0x55fcc1c8b1a0 2023-09-13T03:20:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.583+0000 7ff00f7fe700 1 -- 172.21.15.64:0/1327631905 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 47) v1 ==== 7914+0+0 (secure 0 0 0) 0x7ff028014560 con 0x55fcc1c8b1a0 2023-09-13T03:20:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.583+0000 7ff049ed7b80 10 client.26681 fetch_fsmap finished waiting for FSMap version 47 2023-09-13T03:20:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.583+0000 7ff049ed7b80 10 client.26681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:20:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.583+0000 7ff049ed7b80 20 client.26681 populate_metadata read hostname 'smithi064' 2023-09-13T03:20:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.583+0000 7ff049ed7b80 1 --2- 172.21.15.64:0/1327631905 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55fcc1c9f240 0x55fcc1ca1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.583+0000 7ff049ed7b80 4 client.26681 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:20:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.583+0000 7ff049ed7b80 1 -- 172.21.15.64:0/1327631905 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fcc1bec840 con 0x55fcc1c9f240 2023-09-13T03:20:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.583+0000 7ff02ffff700 1 --2- 172.21.15.64:0/1327631905 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55fcc1c9f240 0x55fcc1ca1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.584+0000 7ff02ffff700 1 --2- 172.21.15.64:0/1327631905 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55fcc1c9f240 0x55fcc1ca1660 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12ef2561d9424967 server_cookie=81622ab4a23d05ef in_seq=0 out_seq=0 2023-09-13T03:20:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.584+0000 7ff00f7fe700 10 client.26681 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:20:02.619 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.584+0000 7ff00f7fe700 1 -- 172.21.15.64:0/1327631905 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fcc1bec840 con 0x55fcc1c9f240 2023-09-13T03:20:02.619 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.584+0000 7ff00f7fe700 10 client.26681 handle_command_reply: tid=0 2023-09-13T03:20:02.619 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.584+0000 7ff049ed7b80 1 client.26681 shutdown 2023-09-13T03:20:02.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.584+0000 7ff049ed7b80 1 -- 172.21.15.64:0/1327631905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcc1c8b1a0 msgr2=0x55fcc1c8edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:02.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.584+0000 7ff049ed7b80 1 --2- 172.21.15.64:0/1327631905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcc1c8b1a0 0x55fcc1c8edf0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff028007b70 tx=0x7ff028004ad0 comp rx=0 tx=0).stop 2023-09-13T03:20:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.584+0000 7ff049ed7b80 1 -- 172.21.15.64:0/1327631905 shutdown_connections 2023-09-13T03:20:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.584+0000 7ff049ed7b80 1 --2- 172.21.15.64:0/1327631905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcc1c8b1a0 0x55fcc1c8edf0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.584+0000 7ff049ed7b80 1 --2- 172.21.15.64:0/1327631905 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55fcc1c9f240 0x55fcc1ca1660 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.584+0000 7ff049ed7b80 1 --2- 172.21.15.64:0/1327631905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fcc1c85300 0x55fcc1c94ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:02.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.584+0000 7ff00f7fe700 0 client.26681 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:20:02.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.584+0000 7ff049ed7b80 1 --2- 172.21.15.64:0/1327631905 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fcc1c85cc0 0x55fcc1c950e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:02.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.584+0000 7ff049ed7b80 1 -- 172.21.15.64:0/1327631905 >> 172.21.15.64:0/1327631905 conn(0x55fcc1bfa030 msgr2=0x55fcc1c7f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:20:02.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.585+0000 7ff049ed7b80 1 -- 172.21.15.64:0/1327631905 shutdown_connections 2023-09-13T03:20:02.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.585+0000 7ff049ed7b80 1 -- 172.21.15.64:0/1327631905 wait complete. 2023-09-13T03:20:02.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.585+0000 7ff049ed7b80 20 client.26681 trim_cache size 0 max 16384 2023-09-13T03:20:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.601+0000 7ff049ed7b80 1 Processor -- start 2023-09-13T03:20:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.601+0000 7ff049ed7b80 1 -- start start 2023-09-13T03:20:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.601+0000 7ff049ed7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fcc1c85300 0x55fcc1c8a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.601+0000 7ff049ed7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcc1c85cc0 0x55fcc1c7ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.601+0000 7ff049ed7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fcc1c8b1a0 0x55fcc1c80460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.601+0000 7ff049ed7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc1b2f490 con 0x55fcc1c85300 2023-09-13T03:20:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.601+0000 7ff049ed7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc1c860a0 con 0x55fcc1c85cc0 2023-09-13T03:20:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.601+0000 7ff049ed7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc1c9c6e0 con 0x55fcc1c8b1a0 2023-09-13T03:20:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.601+0000 7ff02ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fcc1c85300 0x55fcc1c8a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.601+0000 7ff034b9c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fcc1c8b1a0 0x55fcc1c80460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.601+0000 7ff034b9c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fcc1c8b1a0 0x55fcc1c80460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:34884/0 (socket says 172.21.15.64:34884) 2023-09-13T03:20:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.602+0000 7ff034b9c700 1 -- 172.21.15.64:0/4012281501 learned_addr learned my addr 172.21.15.64:0/4012281501 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:20:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.602+0000 7ff02f7fe700 1 --2- 172.21.15.64:0/4012281501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcc1c85cc0 0x55fcc1c7ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.602+0000 7ff034b9c700 1 -- 172.21.15.64:0/4012281501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcc1c85cc0 msgr2=0x55fcc1c7ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:20:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.602+0000 7ff034b9c700 1 --2- 172.21.15.64:0/4012281501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcc1c85cc0 0x55fcc1c7ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.602+0000 7ff034b9c700 1 -- 172.21.15.64:0/4012281501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fcc1c85300 msgr2=0x55fcc1c8a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:20:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.602+0000 7ff034b9c700 1 --2- 172.21.15.64:0/4012281501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fcc1c85300 0x55fcc1c8a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.602+0000 7ff034b9c700 1 -- 172.21.15.64:0/4012281501 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fcc1a2a1f0 con 0x55fcc1c8b1a0 2023-09-13T03:20:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.602+0000 7ff034b9c700 1 --2- 172.21.15.64:0/4012281501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fcc1c8b1a0 0x55fcc1c80460 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff0280092b0 tx=0x7ff0280099a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad2fcbb885130bab server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:20:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.602+0000 7ff016ffd700 1 -- 172.21.15.64:0/4012281501 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02800e050 con 0x55fcc1c8b1a0 2023-09-13T03:20:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.602+0000 7ff016ffd700 1 -- 172.21.15.64:0/4012281501 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff02800f040 con 0x55fcc1c8b1a0 2023-09-13T03:20:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.602+0000 7ff016ffd700 1 -- 172.21.15.64:0/4012281501 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff028035a20 con 0x55fcc1c8b1a0 2023-09-13T03:20:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.603+0000 7ff049ed7b80 1 -- 172.21.15.64:0/4012281501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fcc1c8b1a0 msgr2=0x55fcc1c80460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.603+0000 7ff049ed7b80 1 --2- 172.21.15.64:0/4012281501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fcc1c8b1a0 0x55fcc1c80460 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff0280092b0 tx=0x7ff0280099a0 comp rx=0 tx=0).stop 2023-09-13T03:20:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.603+0000 7ff049ed7b80 1 -- 172.21.15.64:0/4012281501 shutdown_connections 2023-09-13T03:20:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.603+0000 7ff049ed7b80 1 --2- 172.21.15.64:0/4012281501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fcc1c8b1a0 0x55fcc1c80460 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.603+0000 7ff049ed7b80 1 --2- 172.21.15.64:0/4012281501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fcc1c85300 0x55fcc1c8a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.603+0000 7ff049ed7b80 1 --2- 172.21.15.64:0/4012281501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcc1c85cc0 0x55fcc1c7ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.603+0000 7ff049ed7b80 1 -- 172.21.15.64:0/4012281501 >> 172.21.15.64:0/4012281501 conn(0x55fcc1bfa030 msgr2=0x55fcc1c8ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:20:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.603+0000 7ff049ed7b80 1 -- 172.21.15.64:0/4012281501 shutdown_connections 2023-09-13T03:20:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.603+0000 7ff049ed7b80 1 -- 172.21.15.64:0/4012281501 wait complete. 2023-09-13T03:20:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.604+0000 7ff049ed7b80 1 -- start start 2023-09-13T03:20:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.604+0000 7ff049ed7b80 1 Processor -- start 2023-09-13T03:20:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.604+0000 7ff049ed7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcc1c85300 0x55fcc1ce71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.604+0000 7ff049ed7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fcc1c85cc0 0x55fcc1ce76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.604+0000 7ff049ed7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fcc1c8b1a0 0x55fcc1ceb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.604+0000 7ff049ed7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc1ceb5b0 con 0x55fcc1c85cc0 2023-09-13T03:20:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.604+0000 7ff049ed7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc1ceb720 con 0x55fcc1c85300 2023-09-13T03:20:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.604+0000 7ff049ed7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcc1ceb8e0 con 0x55fcc1c8b1a0 2023-09-13T03:20:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.604+0000 7ff049ed7b80 10 client.0 _ll_register_callbacks cb 0x7ff0280157e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:20:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.605+0000 7ff034b9c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fcc1c8b1a0 0x55fcc1ceb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.605+0000 7ff034b9c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fcc1c8b1a0 0x55fcc1ceb070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:34894/0 (socket says 172.21.15.64:34894) 2023-09-13T03:20:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.605+0000 7ff034b9c700 1 -- 172.21.15.64:0/1694506100 learned_addr learned my addr 172.21.15.64:0/1694506100 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:20:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.605+0000 7ff02f7fe700 1 --2- 172.21.15.64:0/1694506100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fcc1c85cc0 0x55fcc1ce76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.605+0000 7ff02ffff700 1 --2- 172.21.15.64:0/1694506100 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcc1c85300 0x55fcc1ce71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.605+0000 7ff034b9c700 1 -- 172.21.15.64:0/1694506100 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcc1c85300 msgr2=0x55fcc1ce71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:20:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.605+0000 7ff034b9c700 1 --2- 172.21.15.64:0/1694506100 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcc1c85300 0x55fcc1ce71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.605+0000 7ff034b9c700 1 -- 172.21.15.64:0/1694506100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fcc1c85cc0 msgr2=0x55fcc1ce76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:20:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.605+0000 7ff034b9c700 1 --2- 172.21.15.64:0/1694506100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fcc1c85cc0 0x55fcc1ce76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.605+0000 7ff034b9c700 1 -- 172.21.15.64:0/1694506100 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fcc19ecc00 con 0x55fcc1c8b1a0 2023-09-13T03:20:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.605+0000 7ff034b9c700 1 --2- 172.21.15.64:0/1694506100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fcc1c8b1a0 0x55fcc1ceb070 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff028016040 tx=0x7ff028035560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2c6b1675a3c51e server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:20:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.605+0000 7ff00f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:20:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.606+0000 7ff00f7fe700 1 -- 172.21.15.64:0/1694506100 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02800e040 con 0x55fcc1c8b1a0 2023-09-13T03:20:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.606+0000 7ff049ed7b80 1 -- 172.21.15.64:0/1694506100 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff02009ec20 con 0x55fcc1c8b1a0 2023-09-13T03:20:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.606+0000 7ff00f7fe700 1 -- 172.21.15.64:0/1694506100 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8440+0+0 (secure 0 0 0) 0x7ff028004a70 con 0x55fcc1c8b1a0 2023-09-13T03:20:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.606+0000 7ff00f7fe700 1 -- 172.21.15.64:0/1694506100 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff028004c50 con 0x55fcc1c8b1a0 2023-09-13T03:20:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.606+0000 7ff049ed7b80 10 client.26671 fetch_fsmap learned FSMap version 47 2023-09-13T03:20:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.606+0000 7ff049ed7b80 1 -- 172.21.15.64:0/1694506100 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55fcc1a02250 con 0x55fcc1c8b1a0 2023-09-13T03:20:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.606+0000 7ff00f7fe700 1 -- 172.21.15.64:0/1694506100 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff02009ec20 con 0x55fcc1c8b1a0 2023-09-13T03:20:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.606+0000 7ff00f7fe700 1 -- 172.21.15.64:0/1694506100 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 47) v1 ==== 7914+0+0 (secure 0 0 0) 0x7ff028035c70 con 0x55fcc1c8b1a0 2023-09-13T03:20:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.606+0000 7ff049ed7b80 10 client.26671 fetch_fsmap finished waiting for FSMap version 47 2023-09-13T03:20:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.606+0000 7ff049ed7b80 10 client.26671 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:20:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.606+0000 7ff049ed7b80 20 client.26671 populate_metadata read hostname 'smithi064' 2023-09-13T03:20:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.606+0000 7ff049ed7b80 1 --2- 172.21.15.64:0/1694506100 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55fcc1cf3ea0 0x55fcc1cf62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.606+0000 7ff049ed7b80 4 client.26671 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:20:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.606+0000 7ff049ed7b80 1 -- 172.21.15.64:0/1694506100 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fcc1bd08c0 con 0x55fcc1cf3ea0 2023-09-13T03:20:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.607+0000 7ff02ffff700 1 --2- 172.21.15.64:0/1694506100 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55fcc1cf3ea0 0x55fcc1cf62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.607+0000 7ff02ffff700 1 --2- 172.21.15.64:0/1694506100 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55fcc1cf3ea0 0x55fcc1cf62c0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cd004630660af4b server_cookie=4c351f10b641c109 in_seq=0 out_seq=0 2023-09-13T03:20:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.607+0000 7ff00f7fe700 10 client.26671 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:20:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.626+0000 7ff00f7fe700 1 -- 172.21.15.64:0/1694506100 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fcc1bd08c0 con 0x55fcc1cf3ea0 2023-09-13T03:20:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.626+0000 7ff00f7fe700 10 client.26671 handle_command_reply: tid=0 2023-09-13T03:20:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.626+0000 7ff049ed7b80 1 client.26671 shutdown 2023-09-13T03:20:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.626+0000 7ff049ed7b80 1 -- 172.21.15.64:0/1694506100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fcc1c8b1a0 msgr2=0x55fcc1ceb070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.626+0000 7ff049ed7b80 1 --2- 172.21.15.64:0/1694506100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fcc1c8b1a0 0x55fcc1ceb070 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff028016040 tx=0x7ff028035560 comp rx=0 tx=0).stop 2023-09-13T03:20:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.626+0000 7ff049ed7b80 1 -- 172.21.15.64:0/1694506100 shutdown_connections 2023-09-13T03:20:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.626+0000 7ff049ed7b80 1 --2- 172.21.15.64:0/1694506100 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fcc1c85300 0x55fcc1ce71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.626+0000 7ff049ed7b80 1 --2- 172.21.15.64:0/1694506100 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55fcc1cf3ea0 0x55fcc1cf62c0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.626+0000 7ff049ed7b80 1 --2- 172.21.15.64:0/1694506100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fcc1c8b1a0 0x55fcc1ceb070 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.626+0000 7ff049ed7b80 1 --2- 172.21.15.64:0/1694506100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fcc1c85cc0 0x55fcc1ce76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.626+0000 7ff00f7fe700 0 client.26671 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:20:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.626+0000 7ff049ed7b80 1 -- 172.21.15.64:0/1694506100 >> 172.21.15.64:0/1694506100 conn(0x55fcc1bfa030 msgr2=0x55fcc1c842f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:20:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.626+0000 7ff049ed7b80 1 -- 172.21.15.64:0/1694506100 shutdown_connections 2023-09-13T03:20:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.627+0000 7ff049ed7b80 1 -- 172.21.15.64:0/1694506100 wait complete. 2023-09-13T03:20:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.627+0000 7ff049ed7b80 20 client.26671 trim_cache size 0 max 16384 2023-09-13T03:20:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:20:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:20:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:20:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:20:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.627+0000 7ff0365ff700 1 -- 172.21.15.64:0/1973032832 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7ff0180a07b0 msgr2=0x7ff0180a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.627+0000 7ff0365ff700 1 --2- 172.21.15.64:0/1973032832 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7ff0180a07b0 0x7ff0180a2c70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff0280093f0 tx=0x7ff02800b040 comp rx=0 tx=0).stop 2023-09-13T03:20:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.627+0000 7ff0365ff700 1 -- 172.21.15.64:0/1973032832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0301005a0 msgr2=0x7ff030072ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.627+0000 7ff0365ff700 1 --2- 172.21.15.64:0/1973032832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0301005a0 0x7ff030072ae0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff020005910 tx=0x7ff0200057f0 comp rx=0 tx=0).stop 2023-09-13T03:20:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.628+0000 7ff0365ff700 1 -- 172.21.15.64:0/1973032832 shutdown_connections 2023-09-13T03:20:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.628+0000 7ff0365ff700 1 --2- 172.21.15.64:0/1973032832 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7ff0180a07b0 0x7ff0180a2c70 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.628+0000 7ff0365ff700 1 --2- 172.21.15.64:0/1973032832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff030106660 0x7ff0301a4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.628+0000 7ff0365ff700 1 --2- 172.21.15.64:0/1973032832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0301005a0 0x7ff030072ae0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.628+0000 7ff0365ff700 1 --2- 172.21.15.64:0/1973032832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff030100f60 0x7ff030073020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.628+0000 7ff0365ff700 1 -- 172.21.15.64:0/1973032832 >> 172.21.15.64:0/1973032832 conn(0x7ff0300fc050 msgr2=0x7ff0301041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:20:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.628+0000 7ff0365ff700 1 -- 172.21.15.64:0/1973032832 shutdown_connections 2023-09-13T03:20:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:02.628+0000 7ff0365ff700 1 -- 172.21.15.64:0/1973032832 wait complete. 2023-09-13T03:20:02.924 INFO:tasks.workunit.client.0.smithi064.stdout: 1 142577 1.93 MB/sec execute 247 sec latency 576.781 ms 2023-09-13T03:20:02.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:20:02 smithi142 ceph-mon[134082]: pgmap v209: 97 pgs: 97 active+clean; 77 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 485 KiB/s wr, 84 op/s 2023-09-13T03:20:03.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:20:02 smithi170 ceph-mon[134328]: pgmap v209: 97 pgs: 97 active+clean; 77 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 485 KiB/s wr, 84 op/s 2023-09-13T03:20:03.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:02 smithi064 ceph-mon[97675]: pgmap v209: 97 pgs: 97 active+clean; 77 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 485 KiB/s wr, 84 op/s 2023-09-13T03:20:03.924 INFO:tasks.workunit.client.0.smithi064.stdout: 1 142633 1.93 MB/sec execute 248 sec latency 1183.465 ms 2023-09-13T03:20:04.924 INFO:tasks.workunit.client.0.smithi064.stdout: 1 142633 1.92 MB/sec execute 249 sec latency 1348.732 ms 2023-09-13T03:20:04.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:20:04 smithi142 ceph-mon[134082]: pgmap v210: 97 pgs: 97 active+clean; 77 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 343 KiB/s wr, 57 op/s 2023-09-13T03:20:05.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:20:04 smithi170 ceph-mon[134328]: pgmap v210: 97 pgs: 97 active+clean; 77 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 343 KiB/s wr, 57 op/s 2023-09-13T03:20:05.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:04 smithi064 ceph-mon[97675]: pgmap v210: 97 pgs: 97 active+clean; 77 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 343 KiB/s wr, 57 op/s 2023-09-13T03:20:05.924 INFO:tasks.workunit.client.0.smithi064.stdout: 1 142633 1.91 MB/sec execute 250 sec latency 2348.838 ms 2023-09-13T03:20:05.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:20:05 smithi142 ceph-mon[134082]: osdmap e87: 12 total, 12 up, 12 in 2023-09-13T03:20:06.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:20:05 smithi170 ceph-mon[134328]: osdmap e87: 12 total, 12 up, 12 in 2023-09-13T03:20:06.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:05 smithi064 ceph-mon[97675]: osdmap e87: 12 total, 12 up, 12 in 2023-09-13T03:20:06.924 INFO:tasks.workunit.client.0.smithi064.stdout: 1 142633 1.90 MB/sec execute 251 sec latency 3348.987 ms 2023-09-13T03:20:06.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:20:06 smithi142 ceph-mon[134082]: pgmap v212: 97 pgs: 97 active+clean; 74 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 598 KiB/s wr, 114 op/s 2023-09-13T03:20:07.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:20:06 smithi170 ceph-mon[134328]: pgmap v212: 97 pgs: 97 active+clean; 74 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 598 KiB/s wr, 114 op/s 2023-09-13T03:20:07.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:06 smithi064 ceph-mon[97675]: pgmap v212: 97 pgs: 97 active+clean; 74 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 598 KiB/s wr, 114 op/s 2023-09-13T03:20:07.924 INFO:tasks.workunit.client.0.smithi064.stdout: 1 142633 1.90 MB/sec execute 252 sec latency 4349.174 ms 2023-09-13T03:20:08.924 INFO:tasks.workunit.client.0.smithi064.stdout: 1 142701 1.89 MB/sec execute 253 sec latency 4954.561 ms 2023-09-13T03:20:08.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:20:08 smithi142 ceph-mon[134082]: pgmap v213: 97 pgs: 97 active+clean; 74 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 598 KiB/s wr, 114 op/s 2023-09-13T03:20:09.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:20:08 smithi170 ceph-mon[134328]: pgmap v213: 97 pgs: 97 active+clean; 74 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 598 KiB/s wr, 114 op/s 2023-09-13T03:20:09.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:08 smithi064 ceph-mon[97675]: pgmap v213: 97 pgs: 97 active+clean; 74 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 598 KiB/s wr, 114 op/s 2023-09-13T03:20:09.924 INFO:tasks.workunit.client.0.smithi064.stdout: 1 143603 1.90 MB/sec execute 254 sec latency 376.275 ms 2023-09-13T03:20:10.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:20:10 smithi170 ceph-mon[134328]: osdmap e88: 12 total, 12 up, 12 in 2023-09-13T03:20:10.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:20:10 smithi170 ceph-mon[134328]: pgmap v215: 97 pgs: 97 active+clean; 74 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 515 KiB/s wr, 100 op/s 2023-09-13T03:20:10.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:10 smithi064 ceph-mon[97675]: osdmap e88: 12 total, 12 up, 12 in 2023-09-13T03:20:10.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:10 smithi064 ceph-mon[97675]: pgmap v215: 97 pgs: 97 active+clean; 74 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 515 KiB/s wr, 100 op/s 2023-09-13T03:20:10.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:20:10 smithi142 ceph-mon[134082]: osdmap e88: 12 total, 12 up, 12 in 2023-09-13T03:20:10.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:20:10 smithi142 ceph-mon[134082]: pgmap v215: 97 pgs: 97 active+clean; 74 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 515 KiB/s wr, 100 op/s 2023-09-13T03:20:10.924 INFO:tasks.workunit.client.0.smithi064.stdout: 1 144809 1.92 MB/sec execute 255 sec latency 756.759 ms 2023-09-13T03:20:11.924 INFO:tasks.workunit.client.0.smithi064.stdout: 1 146061 1.94 MB/sec execute 256 sec latency 16.043 ms 2023-09-13T03:20:12.924 INFO:tasks.workunit.client.0.smithi064.stdout: 1 146126 1.93 MB/sec execute 257 sec latency 973.495 ms 2023-09-13T03:20:12.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:20:12 smithi142 ceph-mon[134082]: pgmap v216: 97 pgs: 97 active+clean; 76 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 181 op/s 2023-09-13T03:20:13.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:20:12 smithi170 ceph-mon[134328]: pgmap v216: 97 pgs: 97 active+clean; 76 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 181 op/s 2023-09-13T03:20:13.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:12 smithi064 ceph-mon[97675]: pgmap v216: 97 pgs: 97 active+clean; 76 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 181 op/s 2023-09-13T03:20:13.925 INFO:tasks.workunit.client.0.smithi064.stdout: 1 146182 1.93 MB/sec execute 258 sec latency 1577.519 ms 2023-09-13T03:20:14.925 INFO:tasks.workunit.client.0.smithi064.stdout: 1 146182 1.92 MB/sec execute 259 sec latency 1351.150 ms 2023-09-13T03:20:14.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:20:14 smithi142 ceph-mon[134082]: pgmap v217: 97 pgs: 97 active+clean; 76 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 1.7 MiB/s wr, 162 op/s 2023-09-13T03:20:15.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:20:14 smithi170 ceph-mon[134328]: pgmap v217: 97 pgs: 97 active+clean; 76 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 1.7 MiB/s wr, 162 op/s 2023-09-13T03:20:15.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:14 smithi064 ceph-mon[97675]: pgmap v217: 97 pgs: 97 active+clean; 76 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 1.7 MiB/s wr, 162 op/s 2023-09-13T03:20:15.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.793+0000 7f50acb9d700 1 -- 172.21.15.64:0/2309096463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50a8073350 msgr2=0x7f50a810c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:15.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.793+0000 7f50acb9d700 1 --2- 172.21.15.64:0/2309096463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50a8073350 0x7f50a810c690 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f509c0098f0 tx=0x7f509c009c00 comp rx=0 tx=0).stop 2023-09-13T03:20:15.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.793+0000 7f50acb9d700 1 -- 172.21.15.64:0/2309096463 shutdown_connections 2023-09-13T03:20:15.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.793+0000 7f50acb9d700 1 --2- 172.21.15.64:0/2309096463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50a810cbd0 0x7f50a810f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:15.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.793+0000 7f50acb9d700 1 --2- 172.21.15.64:0/2309096463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50a8072a30 0x7f50a8072e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:15.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.793+0000 7f50acb9d700 1 --2- 172.21.15.64:0/2309096463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50a8073350 0x7f50a810c690 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:15.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.793+0000 7f50acb9d700 1 -- 172.21.15.64:0/2309096463 >> 172.21.15.64:0/2309096463 conn(0x7f50a80fbe80 msgr2=0x7f50a80fe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:20:15.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.793+0000 7f50acb9d700 1 -- 172.21.15.64:0/2309096463 shutdown_connections 2023-09-13T03:20:15.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.793+0000 7f50acb9d700 1 -- 172.21.15.64:0/2309096463 wait complete. 2023-09-13T03:20:15.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.794+0000 7f50acb9d700 1 Processor -- start 2023-09-13T03:20:15.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.795+0000 7f50acb9d700 1 -- start start 2023-09-13T03:20:15.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.795+0000 7f50acb9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50a8072a30 0x7f50a819e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:15.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.795+0000 7f50acb9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50a8073350 0x7f50a819ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.795+0000 7f50acb9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50a810cbd0 0x7f50a81985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.795+0000 7f50acb9d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a8198c10 con 0x7f50a8072a30 2023-09-13T03:20:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.795+0000 7f50acb9d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a8198d80 con 0x7f50a810cbd0 2023-09-13T03:20:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.795+0000 7f50acb9d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a8198f40 con 0x7f50a8073350 2023-09-13T03:20:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.795+0000 7f50a77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50a8072a30 0x7f50a819e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.796+0000 7f50a77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50a8072a30 0x7f50a819e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54418/0 (socket says 172.21.15.64:54418) 2023-09-13T03:20:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.796+0000 7f50a77fe700 1 -- 172.21.15.64:0/1321323856 learned_addr learned my addr 172.21.15.64:0/1321323856 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:20:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.796+0000 7f50a6ffd700 1 --2- 172.21.15.64:0/1321323856 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50a8073350 0x7f50a819ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.796+0000 7f50a7fff700 1 --2- 172.21.15.64:0/1321323856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50a810cbd0 0x7f50a81985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.796+0000 7f50a77fe700 1 -- 172.21.15.64:0/1321323856 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50a8073350 msgr2=0x7f50a819ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.796+0000 7f50a77fe700 1 --2- 172.21.15.64:0/1321323856 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50a8073350 0x7f50a819ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.796+0000 7f50a77fe700 1 -- 172.21.15.64:0/1321323856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50a810cbd0 msgr2=0x7f50a81985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.796+0000 7f50a77fe700 1 --2- 172.21.15.64:0/1321323856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50a810cbd0 0x7f50a81985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.796+0000 7f50a77fe700 1 -- 172.21.15.64:0/1321323856 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f509c009520 con 0x7f50a8072a30 2023-09-13T03:20:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.796+0000 7f50a77fe700 1 --2- 172.21.15.64:0/1321323856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50a8072a30 0x7f50a819e530 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f509800e9c0 tx=0x7f509800ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:20:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.796+0000 7f50a4ff9700 1 -- 172.21.15.64:0/1321323856 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509800cbd0 con 0x7f50a8072a30 2023-09-13T03:20:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.796+0000 7f50a4ff9700 1 -- 172.21.15.64:0/1321323856 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f509800cd30 con 0x7f50a8072a30 2023-09-13T03:20:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.796+0000 7f50acb9d700 1 -- 172.21.15.64:0/1321323856 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a8199220 con 0x7f50a8072a30 2023-09-13T03:20:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.796+0000 7f50a4ff9700 1 -- 172.21.15.64:0/1321323856 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5098005580 con 0x7f50a8072a30 2023-09-13T03:20:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.797+0000 7f50acb9d700 1 -- 172.21.15.64:0/1321323856 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50a81a5520 con 0x7f50a8072a30 2023-09-13T03:20:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.800+0000 7f50a4ff9700 1 -- 172.21.15.64:0/1321323856 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f5098004750 con 0x7f50a8072a30 2023-09-13T03:20:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.801+0000 7f50c174bb80 1 Processor -- start 2023-09-13T03:20:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.801+0000 7f50c174bb80 1 -- start start 2023-09-13T03:20:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.801+0000 7f50c174bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56173f6a3290 0x56173f6a3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.801+0000 7f50c174bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56173f6a3bb0 0x56173f69d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.801+0000 7f50c174bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56173f69d9a0 0x56173f69dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.802+0000 7f50c174bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173f5f66d0 con 0x56173f6a3290 2023-09-13T03:20:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.802+0000 7f50c174bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173f5f3ff0 con 0x56173f69d9a0 2023-09-13T03:20:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.802+0000 7f50c174bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173f5ef8e0 con 0x56173f6a3bb0 2023-09-13T03:20:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.802+0000 7f50a4ff9700 1 --2- 172.21.15.64:0/1321323856 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f50900a0830 0x7f50900a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.802+0000 7f50a4ff9700 1 -- 172.21.15.64:0/1321323856 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f5098014070 con 0x7f50a8072a30 2023-09-13T03:20:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.802+0000 7f50a7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56173f6a3290 0x56173f6a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.802+0000 7f50a6ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56173f69d9a0 0x56173f69dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.802+0000 7f50a7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56173f6a3290 0x56173f6a3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55118/0 (socket says 172.21.15.64:55118) 2023-09-13T03:20:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.802+0000 7f50a7fff700 1 -- 172.21.15.64:0/1346713514 learned_addr learned my addr 172.21.15.64:0/1346713514 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:20:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.802+0000 7f50a77fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56173f6a3bb0 0x56173f69d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.802+0000 7f50a7fff700 1 --2- 172.21.15.64:0/1321323856 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f50900a0830 0x7f50900a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.802+0000 7f50a77fe700 1 -- 172.21.15.64:0/1346713514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56173f69d9a0 msgr2=0x56173f69dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:20:15.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.802+0000 7f50a77fe700 1 --2- 172.21.15.64:0/1346713514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56173f69d9a0 0x56173f69dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:15.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.802+0000 7f50a77fe700 1 -- 172.21.15.64:0/1346713514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56173f6a3290 msgr2=0x56173f6a3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:20:15.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.802+0000 7f50a77fe700 1 --2- 172.21.15.64:0/1346713514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56173f6a3290 0x56173f6a3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:15.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.802+0000 7f50a77fe700 1 -- 172.21.15.64:0/1346713514 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56173f41bd80 con 0x56173f6a3bb0 2023-09-13T03:20:15.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.803+0000 7f50a77fe700 1 --2- 172.21.15.64:0/1346713514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56173f6a3bb0 0x56173f69d460 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f50980c9950 tx=0x7f50980097d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9a9d7d5948d6fd9 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:20:15.816 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.803+0000 7f50a7fff700 1 --2- 172.21.15.64:0/1321323856 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f50900a0830 0x7f50900a2cf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f50a0009fd0 tx=0x7f50a000b040 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:20:15.817 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.803+0000 7f508dffb700 1 -- 172.21.15.64:0/1346713514 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5098099db0 con 0x56173f6a3bb0 2023-09-13T03:20:15.817 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.803+0000 7f508dffb700 1 -- 172.21.15.64:0/1346713514 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50980cc050 con 0x56173f6a3bb0 2023-09-13T03:20:15.817 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.803+0000 7f508dffb700 1 -- 172.21.15.64:0/1346713514 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50980c9bd0 con 0x56173f6a3bb0 2023-09-13T03:20:15.817 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.804+0000 7f50c174bb80 1 -- 172.21.15.64:0/1346713514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56173f6a3bb0 msgr2=0x56173f69d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:15.817 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.804+0000 7f50c174bb80 1 --2- 172.21.15.64:0/1346713514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56173f6a3bb0 0x56173f69d460 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f50980c9950 tx=0x7f50980097d0 comp rx=0 tx=0).stop 2023-09-13T03:20:15.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.804+0000 7f50c174bb80 1 -- 172.21.15.64:0/1346713514 shutdown_connections 2023-09-13T03:20:15.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.804+0000 7f50c174bb80 1 --2- 172.21.15.64:0/1346713514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56173f69d9a0 0x56173f69dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:15.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.804+0000 7f50c174bb80 1 --2- 172.21.15.64:0/1346713514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56173f6a3290 0x56173f6a3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:15.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.804+0000 7f50c174bb80 1 --2- 172.21.15.64:0/1346713514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56173f6a3bb0 0x56173f69d460 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.804+0000 7f50c174bb80 1 -- 172.21.15.64:0/1346713514 >> 172.21.15.64:0/1346713514 conn(0x56173f612030 msgr2=0x56173f6986c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:20:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.804+0000 7f50c174bb80 1 -- 172.21.15.64:0/1346713514 shutdown_connections 2023-09-13T03:20:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.804+0000 7f50c174bb80 1 -- 172.21.15.64:0/1346713514 wait complete. 2023-09-13T03:20:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.804+0000 7f50c174bb80 1 -- start start 2023-09-13T03:20:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.805+0000 7f50c174bb80 1 Processor -- start 2023-09-13T03:20:15.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.806+0000 7f50c174bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56173f69d9a0 0x56173f6a9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:15.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.806+0000 7f50c174bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56173f6a3290 0x56173f6a6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:15.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.806+0000 7f50c174bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56173f6a3bb0 0x56173f6a7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:15.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.806+0000 7f50c174bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173f58db10 con 0x56173f6a3bb0 2023-09-13T03:20:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.806+0000 7f50c174bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173f58d810 con 0x56173f69d9a0 2023-09-13T03:20:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.806+0000 7f50c174bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173f4718c0 con 0x56173f6a3290 2023-09-13T03:20:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.806+0000 7f50c174bb80 10 client.0 _ll_register_callbacks cb 0x56173f4d7a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:20:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.806+0000 7f50a6ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56173f6a3290 0x56173f6a6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.806+0000 7f50a7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56173f6a3bb0 0x56173f6a7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.806+0000 7f50a6ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56173f6a3290 0x56173f6a6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:46678/0 (socket says 172.21.15.64:46678) 2023-09-13T03:20:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.806+0000 7f50a6ffd700 1 -- 172.21.15.64:0/4253463641 learned_addr learned my addr 172.21.15.64:0/4253463641 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:20:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.806+0000 7f50a77fe700 1 --2- 172.21.15.64:0/4253463641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56173f69d9a0 0x56173f6a9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.806+0000 7f50a7fff700 1 -- 172.21.15.64:0/4253463641 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56173f6a3290 msgr2=0x56173f6a6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:20:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.806+0000 7f50a7fff700 1 --2- 172.21.15.64:0/4253463641 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56173f6a3290 0x56173f6a6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.806+0000 7f50a7fff700 1 -- 172.21.15.64:0/4253463641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56173f69d9a0 msgr2=0x56173f6a9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:20:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.807+0000 7f50a7fff700 1 --2- 172.21.15.64:0/4253463641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56173f69d9a0 0x56173f6a9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.807+0000 7f50a7fff700 1 -- 172.21.15.64:0/4253463641 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50a81a5520 con 0x56173f6a3bb0 2023-09-13T03:20:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.807+0000 7f50a77fe700 1 --2- 172.21.15.64:0/4253463641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56173f69d9a0 0x56173f6a9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:20:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.807+0000 7f50a7fff700 1 --2- 172.21.15.64:0/4253463641 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56173f6a3bb0 0x56173f6a7390 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f50a0019ff0 tx=0x7f50a0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=873fe02d48302a48 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:20:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.807+0000 7f5086ffd700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:20:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.807+0000 7f5086ffd700 1 -- 172.21.15.64:0/4253463641 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a0020070 con 0x56173f6a3bb0 2023-09-13T03:20:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.807+0000 7f50c174bb80 1 -- 172.21.15.64:0/4253463641 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56173f547490 con 0x56173f6a3bb0 2023-09-13T03:20:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.807+0000 7f5086ffd700 1 -- 172.21.15.64:0/4253463641 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f50a0007d80 con 0x56173f6a3bb0 2023-09-13T03:20:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.807+0000 7f5086ffd700 1 -- 172.21.15.64:0/4253463641 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50a000e040 con 0x56173f6a3bb0 2023-09-13T03:20:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.807+0000 7f50c174bb80 10 client.16659 fetch_fsmap learned FSMap version 48 2023-09-13T03:20:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.807+0000 7f50c174bb80 1 -- 172.21.15.64:0/4253463641 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x56173f404c00 con 0x56173f6a3bb0 2023-09-13T03:20:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.808+0000 7f5086ffd700 1 -- 172.21.15.64:0/4253463641 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50a0031080 con 0x56173f6a3bb0 2023-09-13T03:20:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.808+0000 7f5086ffd700 1 -- 172.21.15.64:0/4253463641 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 48) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f50a0004020 con 0x56173f6a3bb0 2023-09-13T03:20:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.808+0000 7f50c174bb80 10 client.16659 fetch_fsmap finished waiting for FSMap version 48 2023-09-13T03:20:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.808+0000 7f50c174bb80 10 client.16659 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:20:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.808+0000 7f50c174bb80 20 client.16659 populate_metadata read hostname 'smithi064' 2023-09-13T03:20:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.808+0000 7f50c174bb80 1 --2- 172.21.15.64:0/4253463641 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56173f6b7600 0x56173f6b9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.808+0000 7f50c174bb80 4 client.16659 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:20:15.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.808+0000 7f50c174bb80 1 -- 172.21.15.64:0/4253463641 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56173f604840 con 0x56173f6b7600 2023-09-13T03:20:15.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.808+0000 7f50a77fe700 1 --2- 172.21.15.64:0/4253463641 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56173f6b7600 0x56173f6b9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:15.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.809+0000 7f50a77fe700 1 --2- 172.21.15.64:0/4253463641 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56173f6b7600 0x56173f6b9a20 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f03ff16a2b08cf36 server_cookie=e9b676d06d8ecba3 in_seq=0 out_seq=0 2023-09-13T03:20:15.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.809+0000 7f5086ffd700 10 client.16659 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:20:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.810+0000 7f5086ffd700 1 -- 172.21.15.64:0/4253463641 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56173f604840 con 0x56173f6b7600 2023-09-13T03:20:15.835 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.810+0000 7f5086ffd700 10 client.16659 handle_command_reply: tid=0 2023-09-13T03:20:15.835 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.810+0000 7f50c174bb80 1 client.16659 shutdown 2023-09-13T03:20:15.835 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.810+0000 7f50c174bb80 1 -- 172.21.15.64:0/4253463641 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56173f6a3bb0 msgr2=0x56173f6a7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:15.836 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.810+0000 7f50c174bb80 1 --2- 172.21.15.64:0/4253463641 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56173f6a3bb0 0x56173f6a7390 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f50a0019ff0 tx=0x7f50a0004ad0 comp rx=0 tx=0).stop 2023-09-13T03:20:15.836 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.810+0000 7f50c174bb80 1 -- 172.21.15.64:0/4253463641 shutdown_connections 2023-09-13T03:20:15.836 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.810+0000 7f50c174bb80 1 --2- 172.21.15.64:0/4253463641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56173f69d9a0 0x56173f6a9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:15.836 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.810+0000 7f50c174bb80 1 --2- 172.21.15.64:0/4253463641 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56173f6b7600 0x56173f6b9a20 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:15.836 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.810+0000 7f50c174bb80 1 --2- 172.21.15.64:0/4253463641 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56173f6a3bb0 0x56173f6a7390 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.810+0000 7f50c174bb80 1 --2- 172.21.15.64:0/4253463641 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56173f6a3290 0x56173f6a6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.810+0000 7f50c174bb80 1 -- 172.21.15.64:0/4253463641 >> 172.21.15.64:0/4253463641 conn(0x56173f612030 msgr2=0x56173f697070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:20:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.810+0000 7f5086ffd700 0 client.16659 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:20:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.810+0000 7f50c174bb80 1 -- 172.21.15.64:0/4253463641 shutdown_connections 2023-09-13T03:20:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.810+0000 7f50c174bb80 1 -- 172.21.15.64:0/4253463641 wait complete. 2023-09-13T03:20:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.810+0000 7f50c174bb80 20 client.16659 trim_cache size 0 max 16384 2023-09-13T03:20:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.826+0000 7f50c174bb80 1 Processor -- start 2023-09-13T03:20:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.826+0000 7f50c174bb80 1 -- start start 2023-09-13T03:20:15.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.826+0000 7f50c174bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56173f69d9a0 0x56173f69d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:15.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.826+0000 7f50c174bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56173f6a3290 0x56173f6a6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:15.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.826+0000 7f50c174bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56173f6a3bb0 0x56173f6ae5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:15.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.826+0000 7f50c174bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173f6a3a40 con 0x56173f6a3bb0 2023-09-13T03:20:15.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.826+0000 7f50c174bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173f695120 con 0x56173f6a3290 2023-09-13T03:20:15.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.826+0000 7f50c174bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173f69e4f0 con 0x56173f69d9a0 2023-09-13T03:20:15.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.826+0000 7f50a7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56173f6a3bb0 0x56173f6ae5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:15.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.826+0000 7f50a6ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56173f6a3290 0x56173f6a6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:15.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.826+0000 7f50a7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56173f6a3bb0 0x56173f6ae5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55136/0 (socket says 172.21.15.64:55136) 2023-09-13T03:20:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.826+0000 7f50a7fff700 1 -- 172.21.15.64:0/4037790793 learned_addr learned my addr 172.21.15.64:0/4037790793 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:20:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.826+0000 7f50a6ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56173f6a3290 0x56173f6a6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:35616/0 (socket says 172.21.15.64:35616) 2023-09-13T03:20:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.826+0000 7f50a77fe700 1 --2- 172.21.15.64:0/4037790793 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56173f69d9a0 0x56173f69d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.827+0000 7f50a7fff700 1 -- 172.21.15.64:0/4037790793 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56173f69d9a0 msgr2=0x56173f69d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:20:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.827+0000 7f50a7fff700 1 --2- 172.21.15.64:0/4037790793 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56173f69d9a0 0x56173f69d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.827+0000 7f50a7fff700 1 -- 172.21.15.64:0/4037790793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56173f6a3290 msgr2=0x56173f6a6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:20:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.827+0000 7f50a7fff700 1 --2- 172.21.15.64:0/4037790793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56173f6a3290 0x56173f6a6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.827+0000 7f50a7fff700 1 -- 172.21.15.64:0/4037790793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56173f69e050 con 0x56173f6a3bb0 2023-09-13T03:20:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.827+0000 7f50a7fff700 1 --2- 172.21.15.64:0/4037790793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56173f6a3bb0 0x56173f6ae5b0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f50a0012d00 tx=0x7f50a0012720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb90aa11bedc3362 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:20:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.827+0000 7f508e7fc700 1 -- 172.21.15.64:0/4037790793 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a0034070 con 0x56173f6a3bb0 2023-09-13T03:20:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.827+0000 7f508e7fc700 1 -- 172.21.15.64:0/4037790793 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56173f547490 con 0x56173f6a3bb0 2023-09-13T03:20:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.827+0000 7f508e7fc700 1 -- 172.21.15.64:0/4037790793 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a0020070 con 0x56173f6a3bb0 2023-09-13T03:20:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.828+0000 7f50c174bb80 1 -- 172.21.15.64:0/4037790793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56173f6a3bb0 msgr2=0x56173f6ae5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.828+0000 7f50c174bb80 1 --2- 172.21.15.64:0/4037790793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56173f6a3bb0 0x56173f6ae5b0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f50a0012d00 tx=0x7f50a0012720 comp rx=0 tx=0).stop 2023-09-13T03:20:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.828+0000 7f50c174bb80 1 -- 172.21.15.64:0/4037790793 shutdown_connections 2023-09-13T03:20:15.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.828+0000 7f50c174bb80 1 --2- 172.21.15.64:0/4037790793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56173f6a3bb0 0x56173f6ae5b0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:15.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.828+0000 7f50c174bb80 1 --2- 172.21.15.64:0/4037790793 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56173f69d9a0 0x56173f69d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:15.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.828+0000 7f50c174bb80 1 --2- 172.21.15.64:0/4037790793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56173f6a3290 0x56173f6a6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:15.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.828+0000 7f50c174bb80 1 -- 172.21.15.64:0/4037790793 >> 172.21.15.64:0/4037790793 conn(0x56173f612030 msgr2=0x56173f6a6600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:20:15.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.828+0000 7f50c174bb80 1 -- 172.21.15.64:0/4037790793 shutdown_connections 2023-09-13T03:20:15.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.828+0000 7f50c174bb80 1 -- 172.21.15.64:0/4037790793 wait complete. 2023-09-13T03:20:15.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.828+0000 7f50c174bb80 1 -- start start 2023-09-13T03:20:15.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.829+0000 7f50c174bb80 1 Processor -- start 2023-09-13T03:20:15.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.830+0000 7f50c174bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56173f69d9a0 0x56173f6ae5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:15.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.830+0000 7f50c174bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56173f6a3290 0x56173f69ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:15.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.830+0000 7f50c174bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56173f6a3bb0 0x56173f69f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:15.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.830+0000 7f50c174bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173f69f7b0 con 0x56173f69d9a0 2023-09-13T03:20:15.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.830+0000 7f50c174bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173f69f920 con 0x56173f6a3290 2023-09-13T03:20:15.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.830+0000 7f50c174bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173f69fae0 con 0x56173f6a3bb0 2023-09-13T03:20:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.830+0000 7f50c174bb80 10 client.0 _ll_register_callbacks cb 0x7f50a00093e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:20:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.830+0000 7f50a77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56173f69d9a0 0x56173f6ae5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.830+0000 7f50a77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56173f69d9a0 0x56173f6ae5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55148/0 (socket says 172.21.15.64:55148) 2023-09-13T03:20:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.830+0000 7f50a77fe700 1 -- 172.21.15.64:0/2219724682 learned_addr learned my addr 172.21.15.64:0/2219724682 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:20:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.830+0000 7f50a7fff700 1 --2- 172.21.15.64:0/2219724682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56173f6a3bb0 0x56173f69f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.830+0000 7f50a6ffd700 1 --2- 172.21.15.64:0/2219724682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56173f6a3290 0x56173f69ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.830+0000 7f50a77fe700 1 -- 172.21.15.64:0/2219724682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56173f6a3bb0 msgr2=0x56173f69f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:20:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.830+0000 7f50a77fe700 1 --2- 172.21.15.64:0/2219724682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56173f6a3bb0 0x56173f69f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.830+0000 7f50a77fe700 1 -- 172.21.15.64:0/2219724682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56173f6a3290 msgr2=0x56173f69ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:20:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.830+0000 7f50a77fe700 1 --2- 172.21.15.64:0/2219724682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56173f6a3290 0x56173f69ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:15.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.830+0000 7f50a77fe700 1 -- 172.21.15.64:0/2219724682 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56173f404c00 con 0x56173f69d9a0 2023-09-13T03:20:15.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.830+0000 7f50a77fe700 1 --2- 172.21.15.64:0/2219724682 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56173f69d9a0 0x56173f6ae5c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f50980993b0 tx=0x7f5098009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e31990f93a970e0f server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:20:15.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.830+0000 7f5086ffd700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:20:15.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.831+0000 7f5086ffd700 1 -- 172.21.15.64:0/2219724682 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50980c9dd0 con 0x56173f69d9a0 2023-09-13T03:20:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.831+0000 7f5086ffd700 1 -- 172.21.15.64:0/2219724682 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f50980a1720 con 0x56173f69d9a0 2023-09-13T03:20:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.831+0000 7f50c174bb80 1 -- 172.21.15.64:0/2219724682 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f50980cc050 con 0x56173f69d9a0 2023-09-13T03:20:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.831+0000 7f5086ffd700 1 -- 172.21.15.64:0/2219724682 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f509809d4e0 con 0x56173f69d9a0 2023-09-13T03:20:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.831+0000 7f50c174bb80 10 client.16665 fetch_fsmap learned FSMap version 48 2023-09-13T03:20:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.831+0000 7f50c174bb80 1 -- 172.21.15.64:0/2219724682 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x56173f4421f0 con 0x56173f69d9a0 2023-09-13T03:20:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.831+0000 7f5086ffd700 1 -- 172.21.15.64:0/2219724682 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50980cc050 con 0x56173f69d9a0 2023-09-13T03:20:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.831+0000 7f5086ffd700 1 -- 172.21.15.64:0/2219724682 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 48) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f5098098ba0 con 0x56173f69d9a0 2023-09-13T03:20:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.831+0000 7f50c174bb80 10 client.16665 fetch_fsmap finished waiting for FSMap version 48 2023-09-13T03:20:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.831+0000 7f50c174bb80 10 client.16665 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:20:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.831+0000 7f50c174bb80 20 client.16665 populate_metadata read hostname 'smithi064' 2023-09-13T03:20:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.831+0000 7f50c174bb80 1 --2- 172.21.15.64:0/2219724682 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56173f70c7a0 0x56173f70ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.831+0000 7f50c174bb80 4 client.16665 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:20:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.831+0000 7f50c174bb80 1 -- 172.21.15.64:0/2219724682 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56173f5e88c0 con 0x56173f70c7a0 2023-09-13T03:20:15.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.832+0000 7f50a6ffd700 1 --2- 172.21.15.64:0/2219724682 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56173f70c7a0 0x56173f70ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:15.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.832+0000 7f50a6ffd700 1 --2- 172.21.15.64:0/2219724682 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56173f70c7a0 0x56173f70ebc0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8372fb5ed5e05731 server_cookie=d714baf847aea271 in_seq=0 out_seq=0 2023-09-13T03:20:15.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.832+0000 7f5086ffd700 10 client.16665 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:20:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.856+0000 7f5086ffd700 1 -- 172.21.15.64:0/2219724682 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56173f5e88c0 con 0x56173f70c7a0 2023-09-13T03:20:15.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.856+0000 7f5086ffd700 10 client.16665 handle_command_reply: tid=0 2023-09-13T03:20:15.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.856+0000 7f50c174bb80 1 client.16665 shutdown 2023-09-13T03:20:15.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.856+0000 7f50c174bb80 1 -- 172.21.15.64:0/2219724682 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56173f69d9a0 msgr2=0x56173f6ae5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:15.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.856+0000 7f50c174bb80 1 --2- 172.21.15.64:0/2219724682 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56173f69d9a0 0x56173f6ae5c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f50980993b0 tx=0x7f5098009290 comp rx=0 tx=0).stop 2023-09-13T03:20:15.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.856+0000 7f50c174bb80 1 -- 172.21.15.64:0/2219724682 shutdown_connections 2023-09-13T03:20:15.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.856+0000 7f50c174bb80 1 --2- 172.21.15.64:0/2219724682 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56173f70c7a0 0x56173f70ebc0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:15.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.856+0000 7f50c174bb80 1 --2- 172.21.15.64:0/2219724682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56173f6a3bb0 0x56173f69f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:15.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.856+0000 7f50c174bb80 1 --2- 172.21.15.64:0/2219724682 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56173f69d9a0 0x56173f6ae5c0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:15.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.856+0000 7f5086ffd700 0 client.16665 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:20:15.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.856+0000 7f50c174bb80 1 --2- 172.21.15.64:0/2219724682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56173f6a3290 0x56173f69ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:15.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.856+0000 7f50c174bb80 1 -- 172.21.15.64:0/2219724682 >> 172.21.15.64:0/2219724682 conn(0x56173f612030 msgr2=0x56173f6a4060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:20:15.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.857+0000 7f50c174bb80 1 -- 172.21.15.64:0/2219724682 shutdown_connections 2023-09-13T03:20:15.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.857+0000 7f50c174bb80 1 -- 172.21.15.64:0/2219724682 wait complete. 2023-09-13T03:20:15.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.857+0000 7f50c174bb80 20 client.16665 trim_cache size 0 max 16384 2023-09-13T03:20:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:20:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:20:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:20:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:20:15.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.857+0000 7f50acb9d700 1 -- 172.21.15.64:0/1321323856 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f50900a0830 msgr2=0x7f50900a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:15.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.857+0000 7f50acb9d700 1 --2- 172.21.15.64:0/1321323856 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f50900a0830 0x7f50900a2cf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f50a0009fd0 tx=0x7f50a000b040 comp rx=0 tx=0).stop 2023-09-13T03:20:15.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.857+0000 7f50acb9d700 1 -- 172.21.15.64:0/1321323856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50a8072a30 msgr2=0x7f50a819e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.857+0000 7f50acb9d700 1 --2- 172.21.15.64:0/1321323856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50a8072a30 0x7f50a819e530 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f509800e9c0 tx=0x7f509800ed80 comp rx=0 tx=0).stop 2023-09-13T03:20:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.858+0000 7f50acb9d700 1 -- 172.21.15.64:0/1321323856 shutdown_connections 2023-09-13T03:20:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.858+0000 7f50acb9d700 1 --2- 172.21.15.64:0/1321323856 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f50900a0830 0x7f50900a2cf0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.858+0000 7f50acb9d700 1 --2- 172.21.15.64:0/1321323856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50a810cbd0 0x7f50a81985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.858+0000 7f50acb9d700 1 --2- 172.21.15.64:0/1321323856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50a8072a30 0x7f50a819e530 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.858+0000 7f50acb9d700 1 --2- 172.21.15.64:0/1321323856 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50a8073350 0x7f50a819ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.858+0000 7f50acb9d700 1 -- 172.21.15.64:0/1321323856 >> 172.21.15.64:0/1321323856 conn(0x7f50a80fbe80 msgr2=0x7f50a810edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:20:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.858+0000 7f50acb9d700 1 -- 172.21.15.64:0/1321323856 shutdown_connections 2023-09-13T03:20:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:15.858+0000 7f50acb9d700 1 -- 172.21.15.64:0/1321323856 wait complete. 2023-09-13T03:20:15.925 INFO:tasks.workunit.client.0.smithi064.stdout: 1 146182 1.91 MB/sec execute 260 sec latency 2351.264 ms 2023-09-13T03:20:16.925 INFO:tasks.workunit.client.0.smithi064.stdout: 1 146182 1.90 MB/sec execute 261 sec latency 3351.374 ms 2023-09-13T03:20:16.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:20:16 smithi142 ceph-mon[134082]: pgmap v218: 97 pgs: 97 active+clean; 78 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 1.4 MiB/s wr, 102 op/s 2023-09-13T03:20:16.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:20:16 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:20:17.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:20:16 smithi170 ceph-mon[134328]: pgmap v218: 97 pgs: 97 active+clean; 78 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 1.4 MiB/s wr, 102 op/s 2023-09-13T03:20:17.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:20:16 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:20:17.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:16 smithi064 ceph-mon[97675]: pgmap v218: 97 pgs: 97 active+clean; 78 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 1.4 MiB/s wr, 102 op/s 2023-09-13T03:20:17.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:16 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:20:17.925 INFO:tasks.workunit.client.0.smithi064.stdout: 1 146182 1.90 MB/sec execute 262 sec latency 4351.489 ms 2023-09-13T03:20:17.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:20:17 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:20:18.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:20:17 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:20:18.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:17 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:20:18.925 INFO:tasks.workunit.client.0.smithi064.stdout: 1 146250 1.89 MB/sec execute 263 sec latency 4957.714 ms 2023-09-13T03:20:18.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:20:18 smithi142 ceph-mon[134082]: pgmap v219: 97 pgs: 97 active+clean; 78 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 1.4 MiB/s wr, 102 op/s 2023-09-13T03:20:19.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:20:18 smithi170 ceph-mon[134328]: pgmap v219: 97 pgs: 97 active+clean; 78 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 1.4 MiB/s wr, 102 op/s 2023-09-13T03:20:19.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:18 smithi064 ceph-mon[97675]: pgmap v219: 97 pgs: 97 active+clean; 78 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 1.4 MiB/s wr, 102 op/s 2023-09-13T03:20:19.925 INFO:tasks.workunit.client.0.smithi064.stdout: 1 146250 1.88 MB/sec execute 264 sec latency 1340.315 ms 2023-09-13T03:20:20.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:20:20 smithi170 ceph-mon[134328]: pgmap v220: 97 pgs: 97 active+clean; 78 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 1.4 MiB/s wr, 99 op/s 2023-09-13T03:20:20.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:20 smithi064 ceph-mon[97675]: pgmap v220: 97 pgs: 97 active+clean; 78 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 1.4 MiB/s wr, 99 op/s 2023-09-13T03:20:20.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:20:20 smithi142 ceph-mon[134082]: pgmap v220: 97 pgs: 97 active+clean; 78 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 1.4 MiB/s wr, 99 op/s 2023-09-13T03:20:20.925 INFO:tasks.workunit.client.0.smithi064.stdout: 1 146250 1.88 MB/sec execute 265 sec latency 2340.422 ms 2023-09-13T03:20:21.926 INFO:tasks.workunit.client.0.smithi064.stdout: 1 146250 1.87 MB/sec execute 266 sec latency 3340.551 ms 2023-09-13T03:20:22.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:20:22 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:20:22.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:20:22 smithi170 ceph-mon[134328]: pgmap v221: 97 pgs: 97 active+clean; 78 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 1.2 MiB/s wr, 100 op/s 2023-09-13T03:20:22.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:22 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:20:22.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:22 smithi064 ceph-mon[97675]: pgmap v221: 97 pgs: 97 active+clean; 78 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 1.2 MiB/s wr, 100 op/s 2023-09-13T03:20:22.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:20:22 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:20:22.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:20:22 smithi142 ceph-mon[134082]: pgmap v221: 97 pgs: 97 active+clean; 78 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 1.2 MiB/s wr, 100 op/s 2023-09-13T03:20:22.926 INFO:tasks.workunit.client.0.smithi064.stdout: 1 146250 1.86 MB/sec execute 267 sec latency 4340.700 ms 2023-09-13T03:20:23.926 INFO:tasks.workunit.client.0.smithi064.stdout: 1 146965 1.87 MB/sec execute 268 sec latency 4942.133 ms 2023-09-13T03:20:24.926 INFO:tasks.workunit.client.0.smithi064.stdout: 1 147151 1.87 MB/sec execute 269 sec latency 852.729 ms 2023-09-13T03:20:24.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:20:24 smithi142 ceph-mon[134082]: pgmap v222: 97 pgs: 97 active+clean; 78 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 262 KiB/s wr, 45 op/s 2023-09-13T03:20:25.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:20:24 smithi170 ceph-mon[134328]: pgmap v222: 97 pgs: 97 active+clean; 78 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 262 KiB/s wr, 45 op/s 2023-09-13T03:20:25.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:24 smithi064 ceph-mon[97675]: pgmap v222: 97 pgs: 97 active+clean; 78 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 262 KiB/s wr, 45 op/s 2023-09-13T03:20:25.926 INFO:tasks.workunit.client.0.smithi064.stdout: 1 148360 1.88 MB/sec execute 270 sec latency 1274.030 ms 2023-09-13T03:20:25.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:20:25 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:20:26.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:20:25 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:20:26.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:25 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:20:26.926 INFO:tasks.workunit.client.0.smithi064.stdout: 1 149600 1.90 MB/sec execute 271 sec latency 42.631 ms 2023-09-13T03:20:26.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:20:26 smithi142 ceph-mon[134082]: pgmap v223: 97 pgs: 97 active+clean; 79 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 370 KiB/s wr, 67 op/s 2023-09-13T03:20:26.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:20:26 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:20:26.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:20:26 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:20:26.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:20:26 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:20:27.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:20:26 smithi170 ceph-mon[134328]: pgmap v223: 97 pgs: 97 active+clean; 79 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 370 KiB/s wr, 67 op/s 2023-09-13T03:20:27.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:20:26 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:20:27.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:20:26 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:20:27.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:20:26 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:20:27.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:26 smithi064 ceph-mon[97675]: pgmap v223: 97 pgs: 97 active+clean; 79 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 370 KiB/s wr, 67 op/s 2023-09-13T03:20:27.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:26 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:20:27.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:26 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:20:27.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:26 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:20:27.926 INFO:tasks.workunit.client.0.smithi064.stdout: 1 149791 1.90 MB/sec execute 272 sec latency 821.829 ms 2023-09-13T03:20:28.926 INFO:tasks.workunit.client.0.smithi064.stdout: 1 150435 1.91 MB/sec execute 273 sec latency 1429.711 ms 2023-09-13T03:20:28.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:20:28 smithi142 ceph-mon[134082]: pgmap v224: 97 pgs: 97 active+clean; 79 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 161 KiB/s wr, 36 op/s 2023-09-13T03:20:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.015+0000 7f7c3b5ff700 1 -- 172.21.15.64:0/4110536731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c340732f0 msgr2=0x7f7c3410c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.015+0000 7f7c3b5ff700 1 --2- 172.21.15.64:0/4110536731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c340732f0 0x7f7c3410c8e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7c24009990 tx=0x7f7c24009ca0 comp rx=0 tx=0).stop 2023-09-13T03:20:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.015+0000 7f7c3b5ff700 1 -- 172.21.15.64:0/4110536731 shutdown_connections 2023-09-13T03:20:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.015+0000 7f7c3b5ff700 1 --2- 172.21.15.64:0/4110536731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c3410ce20 0x7f7c3410f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.015+0000 7f7c3b5ff700 1 --2- 172.21.15.64:0/4110536731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c340729d0 0x7f7c34072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.015+0000 7f7c3b5ff700 1 --2- 172.21.15.64:0/4110536731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c340732f0 0x7f7c3410c8e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.015+0000 7f7c3b5ff700 1 -- 172.21.15.64:0/4110536731 >> 172.21.15.64:0/4110536731 conn(0x7f7c340fc0b0 msgr2=0x7f7c340fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:20:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.015+0000 7f7c3b5ff700 1 -- 172.21.15.64:0/4110536731 shutdown_connections 2023-09-13T03:20:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.015+0000 7f7c3b5ff700 1 -- 172.21.15.64:0/4110536731 wait complete. 2023-09-13T03:20:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.016+0000 7f7c3b5ff700 1 Processor -- start 2023-09-13T03:20:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.017+0000 7f7c3b5ff700 1 -- start start 2023-09-13T03:20:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.017+0000 7f7c3b5ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c340729d0 0x7f7c3419e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.017+0000 7f7c3b5ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c340732f0 0x7f7c3419ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.017+0000 7f7c3b5ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c3410ce20 0x7f7c34198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.017+0000 7f7c3b5ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c34198db0 con 0x7f7c340729d0 2023-09-13T03:20:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.017+0000 7f7c3b5ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c34198f20 con 0x7f7c3410ce20 2023-09-13T03:20:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.017+0000 7f7c3b5ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c341990e0 con 0x7f7c340732f0 2023-09-13T03:20:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.017+0000 7f7c3939b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c340729d0 0x7f7c3419e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.017+0000 7f7c38b9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c340732f0 0x7f7c3419ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.017+0000 7f7c39b9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c3410ce20 0x7f7c34198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.017+0000 7f7c38b9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c340732f0 0x7f7c3419ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:42146/0 (socket says 172.21.15.64:42146) 2023-09-13T03:20:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.017+0000 7f7c39b9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c3410ce20 0x7f7c34198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:40866/0 (socket says 172.21.15.64:40866) 2023-09-13T03:20:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.017+0000 7f7c38b9a700 1 -- 172.21.15.64:0/951977612 learned_addr learned my addr 172.21.15.64:0/951977612 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:20:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.018+0000 7f7c39b9c700 1 -- 172.21.15.64:0/951977612 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c340732f0 msgr2=0x7f7c3419ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.018+0000 7f7c39b9c700 1 --2- 172.21.15.64:0/951977612 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c340732f0 0x7f7c3419ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.018+0000 7f7c39b9c700 1 -- 172.21.15.64:0/951977612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c340729d0 msgr2=0x7f7c3419e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.018+0000 7f7c39b9c700 1 --2- 172.21.15.64:0/951977612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c340729d0 0x7f7c3419e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.018+0000 7f7c39b9c700 1 -- 172.21.15.64:0/951977612 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c30009520 con 0x7f7c3410ce20 2023-09-13T03:20:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.018+0000 7f7c39b9c700 1 --2- 172.21.15.64:0/951977612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c3410ce20 0x7f7c34198740 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7c34069c60 tx=0x7f7c2c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:20:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.018+0000 7f7c2a7fc700 1 -- 172.21.15.64:0/951977612 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c003a00 con 0x7f7c3410ce20 2023-09-13T03:20:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.018+0000 7f7c3b5ff700 1 -- 172.21.15.64:0/951977612 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c2c009c40 con 0x7f7c3410ce20 2023-09-13T03:20:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.018+0000 7f7c2a7fc700 1 -- 172.21.15.64:0/951977612 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c2c00f040 con 0x7f7c3410ce20 2023-09-13T03:20:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.018+0000 7f7c2a7fc700 1 -- 172.21.15.64:0/951977612 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c004570 con 0x7f7c3410ce20 2023-09-13T03:20:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.018+0000 7f7c3b5ff700 1 -- 172.21.15.64:0/951977612 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c240095f0 con 0x7f7c3410ce20 2023-09-13T03:20:29.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.022+0000 7f7c2a7fc700 1 -- 172.21.15.64:0/951977612 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f7c2c003b60 con 0x7f7c3410ce20 2023-09-13T03:20:29.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.023+0000 7f7c2a7fc700 1 --2- 172.21.15.64:0/951977612 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f7c200a0830 0x7f7c200a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:29.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.023+0000 7f7c4ee71b80 1 Processor -- start 2023-09-13T03:20:29.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.023+0000 7f7c2a7fc700 1 -- 172.21.15.64:0/951977612 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f7c2c0cba70 con 0x7f7c3410ce20 2023-09-13T03:20:29.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.023+0000 7f7c3939b700 1 --2- 172.21.15.64:0/951977612 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f7c200a0830 0x7f7c200a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:29.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.024+0000 7f7c4ee71b80 1 -- start start 2023-09-13T03:20:29.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.024+0000 7f7c3939b700 1 --2- 172.21.15.64:0/951977612 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f7c200a0830 0x7f7c200a2cf0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7c30009df0 tx=0x7f7c30009260 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:20:29.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.024+0000 7f7c4ee71b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633fd5fc1a0 0x5633fd5fc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:29.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.024+0000 7f7c4ee71b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633fd5f6300 0x5633fd5f6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:29.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.024+0000 7f7c4ee71b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5633fd5f6cc0 0x5633fd602f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.024+0000 7f7c4ee71b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633fd54f6d0 con 0x5633fd5fc1a0 2023-09-13T03:20:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.024+0000 7f7c4ee71b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633fd54cff0 con 0x5633fd5f6300 2023-09-13T03:20:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.024+0000 7f7c4ee71b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633fd5488e0 con 0x5633fd5f6cc0 2023-09-13T03:20:29.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.025+0000 7f7c38b9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633fd5f6300 0x5633fd5f6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:29.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.025+0000 7f7c3939b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633fd5fc1a0 0x5633fd5fc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:29.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.025+0000 7f7c38b9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633fd5f6300 0x5633fd5f6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:40868/0 (socket says 172.21.15.64:40868) 2023-09-13T03:20:29.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.025+0000 7f7c38b9a700 1 -- 172.21.15.64:0/330178511 learned_addr learned my addr 172.21.15.64:0/330178511 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:20:29.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.025+0000 7f7c39b9c700 1 --2- 172.21.15.64:0/330178511 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5633fd5f6cc0 0x5633fd602f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:29.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.025+0000 7f7c38b9a700 1 -- 172.21.15.64:0/330178511 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5633fd5f6cc0 msgr2=0x5633fd602f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:20:29.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.025+0000 7f7c38b9a700 1 --2- 172.21.15.64:0/330178511 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5633fd5f6cc0 0x5633fd602f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:29.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.025+0000 7f7c38b9a700 1 -- 172.21.15.64:0/330178511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633fd5fc1a0 msgr2=0x5633fd5fc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:20:29.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.025+0000 7f7c38b9a700 1 --2- 172.21.15.64:0/330178511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633fd5fc1a0 0x5633fd5fc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:29.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.025+0000 7f7c38b9a700 1 -- 172.21.15.64:0/330178511 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633fd374d80 con 0x5633fd5f6300 2023-09-13T03:20:29.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.025+0000 7f7c39b9c700 1 --2- 172.21.15.64:0/330178511 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5633fd5f6cc0 0x5633fd602f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:20:29.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.025+0000 7f7c3939b700 1 --2- 172.21.15.64:0/330178511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633fd5fc1a0 0x5633fd5fc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:20:29.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.025+0000 7f7c38b9a700 1 --2- 172.21.15.64:0/330178511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633fd5f6300 0x5633fd5f6780 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7c24005f10 tx=0x7f7c24005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6287faa61093134c server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:20:29.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.026+0000 7f7c1f7fe700 1 -- 172.21.15.64:0/330178511 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2401d070 con 0x5633fd5f6300 2023-09-13T03:20:29.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.026+0000 7f7c1f7fe700 1 -- 172.21.15.64:0/330178511 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c2401e040 con 0x5633fd5f6300 2023-09-13T03:20:29.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.026+0000 7f7c1f7fe700 1 -- 172.21.15.64:0/330178511 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c24004af0 con 0x5633fd5f6300 2023-09-13T03:20:29.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.026+0000 7f7c4ee71b80 1 -- 172.21.15.64:0/330178511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633fd5f6300 msgr2=0x5633fd5f6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:29.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.026+0000 7f7c4ee71b80 1 --2- 172.21.15.64:0/330178511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633fd5f6300 0x5633fd5f6780 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7c24005f10 tx=0x7f7c24005c00 comp rx=0 tx=0).stop 2023-09-13T03:20:29.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.026+0000 7f7c4ee71b80 1 -- 172.21.15.64:0/330178511 shutdown_connections 2023-09-13T03:20:29.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.026+0000 7f7c4ee71b80 1 --2- 172.21.15.64:0/330178511 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5633fd5f6cc0 0x5633fd602f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:29.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.026+0000 7f7c4ee71b80 1 --2- 172.21.15.64:0/330178511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633fd5fc1a0 0x5633fd5fc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:29.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.026+0000 7f7c4ee71b80 1 --2- 172.21.15.64:0/330178511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633fd5f6300 0x5633fd5f6780 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:29.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.026+0000 7f7c4ee71b80 1 -- 172.21.15.64:0/330178511 >> 172.21.15.64:0/330178511 conn(0x5633fd56b030 msgr2=0x5633fd5f1560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:20:29.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.027+0000 7f7c4ee71b80 1 -- 172.21.15.64:0/330178511 shutdown_connections 2023-09-13T03:20:29.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.027+0000 7f7c4ee71b80 1 -- 172.21.15.64:0/330178511 wait complete. 2023-09-13T03:20:29.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.027+0000 7f7c4ee71b80 1 -- start start 2023-09-13T03:20:29.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.027+0000 7f7c4ee71b80 1 Processor -- start 2023-09-13T03:20:29.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.028+0000 7f7c4ee71b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633fd5f6300 0x5633fd602e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:29.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.028+0000 7f7c4ee71b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633fd5f6cc0 0x5633fd6000a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:29.042 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.028+0000 7f7c4ee71b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5633fd5fc1a0 0x5633fd6006d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:29.042 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.028+0000 7f7c4ee71b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633fd4e6b10 con 0x5633fd5f6cc0 2023-09-13T03:20:29.042 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.028+0000 7f7c4ee71b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633fd4e6810 con 0x5633fd5f6300 2023-09-13T03:20:29.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.028+0000 7f7c4ee71b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633fd3ca8c0 con 0x5633fd5fc1a0 2023-09-13T03:20:29.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.028+0000 7f7c4ee71b80 10 client.0 _ll_register_callbacks cb 0x5633fd430a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:20:29.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.029+0000 7f7c38b9a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633fd5f6cc0 0x5633fd6000a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:29.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.029+0000 7f7c3939b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633fd5f6300 0x5633fd602e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:29.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.029+0000 7f7c39b9c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5633fd5fc1a0 0x5633fd6006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:29.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.029+0000 7f7c3939b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633fd5f6300 0x5633fd602e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:40878/0 (socket says 172.21.15.64:40878) 2023-09-13T03:20:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.029+0000 7f7c38b9a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633fd5f6cc0 0x5633fd6000a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55932/0 (socket says 172.21.15.64:55932) 2023-09-13T03:20:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.029+0000 7f7c3939b700 1 -- 172.21.15.64:0/3071816454 learned_addr learned my addr 172.21.15.64:0/3071816454 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:20:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.029+0000 7f7c39b9c700 1 -- 172.21.15.64:0/3071816454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633fd5f6300 msgr2=0x5633fd602e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:20:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.029+0000 7f7c39b9c700 1 --2- 172.21.15.64:0/3071816454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633fd5f6300 0x5633fd602e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.029+0000 7f7c39b9c700 1 -- 172.21.15.64:0/3071816454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633fd5f6cc0 msgr2=0x5633fd6000a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:20:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.029+0000 7f7c39b9c700 1 --2- 172.21.15.64:0/3071816454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633fd5f6cc0 0x5633fd6000a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.029+0000 7f7c39b9c700 1 -- 172.21.15.64:0/3071816454 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c240095f0 con 0x5633fd5fc1a0 2023-09-13T03:20:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.029+0000 7f7c38b9a700 1 --2- 172.21.15.64:0/3071816454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633fd5f6cc0 0x5633fd6000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:20:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.029+0000 7f7c39b9c700 1 --2- 172.21.15.64:0/3071816454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5633fd5fc1a0 0x5633fd6006d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c0c9180 tx=0x7f7c2c0988d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53f3ca560503dba server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:20:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.029+0000 7f7c1cff9700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:20:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.029+0000 7f7c1cff9700 1 -- 172.21.15.64:0/3071816454 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c0b8040 con 0x5633fd5fc1a0 2023-09-13T03:20:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.029+0000 7f7c4ee71b80 1 -- 172.21.15.64:0/3071816454 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5633fd4a0490 con 0x5633fd5fc1a0 2023-09-13T03:20:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.029+0000 7f7c1cff9700 1 -- 172.21.15.64:0/3071816454 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f7c2c098f70 con 0x5633fd5fc1a0 2023-09-13T03:20:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.030+0000 7f7c1cff9700 1 -- 172.21.15.64:0/3071816454 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c2c0048e0 con 0x5633fd5fc1a0 2023-09-13T03:20:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.030+0000 7f7c4ee71b80 10 client.26701 fetch_fsmap learned FSMap version 50 2023-09-13T03:20:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.030+0000 7f7c4ee71b80 1 -- 172.21.15.64:0/3071816454 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x7f7c30017070 con 0x5633fd5fc1a0 2023-09-13T03:20:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.030+0000 7f7c1cff9700 1 -- 172.21.15.64:0/3071816454 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c2c07b020 con 0x5633fd5fc1a0 2023-09-13T03:20:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.030+0000 7f7c1cff9700 1 -- 172.21.15.64:0/3071816454 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 50) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f7c2c098460 con 0x5633fd5fc1a0 2023-09-13T03:20:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.030+0000 7f7c4ee71b80 10 client.26701 fetch_fsmap finished waiting for FSMap version 50 2023-09-13T03:20:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.030+0000 7f7c4ee71b80 10 client.26701 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:20:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.030+0000 7f7c4ee71b80 20 client.26701 populate_metadata read hostname 'smithi064' 2023-09-13T03:20:29.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.030+0000 7f7c4ee71b80 1 --2- 172.21.15.64:0/3071816454 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5633fd610410 0x5633fd612830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:29.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.030+0000 7f7c4ee71b80 4 client.26701 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:20:29.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.030+0000 7f7c4ee71b80 1 -- 172.21.15.64:0/3071816454 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5633fd55d840 con 0x5633fd610410 2023-09-13T03:20:29.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.031+0000 7f7c3939b700 1 --2- 172.21.15.64:0/3071816454 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5633fd610410 0x5633fd612830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:29.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.031+0000 7f7c3939b700 1 --2- 172.21.15.64:0/3071816454 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5633fd610410 0x5633fd612830 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c68b37e0a1d9648a server_cookie=dbef812da2595363 in_seq=0 out_seq=0 2023-09-13T03:20:29.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.031+0000 7f7c1cff9700 10 client.26701 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:20:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.032+0000 7f7c1cff9700 1 -- 172.21.15.64:0/3071816454 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5633fd55d840 con 0x5633fd610410 2023-09-13T03:20:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.032+0000 7f7c1cff9700 10 client.26701 handle_command_reply: tid=0 2023-09-13T03:20:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.032+0000 7f7c4ee71b80 1 client.26701 shutdown 2023-09-13T03:20:29.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.032+0000 7f7c4ee71b80 1 -- 172.21.15.64:0/3071816454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5633fd5fc1a0 msgr2=0x5633fd6006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:29.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.032+0000 7f7c4ee71b80 1 --2- 172.21.15.64:0/3071816454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5633fd5fc1a0 0x5633fd6006d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c0c9180 tx=0x7f7c2c0988d0 comp rx=0 tx=0).stop 2023-09-13T03:20:29.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.032+0000 7f7c4ee71b80 1 -- 172.21.15.64:0/3071816454 shutdown_connections 2023-09-13T03:20:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.032+0000 7f7c4ee71b80 1 --2- 172.21.15.64:0/3071816454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633fd5f6300 0x5633fd602e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.032+0000 7f7c4ee71b80 1 --2- 172.21.15.64:0/3071816454 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5633fd610410 0x5633fd612830 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.032+0000 7f7c4ee71b80 1 --2- 172.21.15.64:0/3071816454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5633fd5fc1a0 0x5633fd6006d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.032+0000 7f7c4ee71b80 1 --2- 172.21.15.64:0/3071816454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633fd5f6cc0 0x5633fd6000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.032+0000 7f7c4ee71b80 1 -- 172.21.15.64:0/3071816454 >> 172.21.15.64:0/3071816454 conn(0x5633fd56b030 msgr2=0x5633fd5f07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:20:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.032+0000 7f7c1cff9700 0 client.26701 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:20:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.032+0000 7f7c4ee71b80 1 -- 172.21.15.64:0/3071816454 shutdown_connections 2023-09-13T03:20:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.032+0000 7f7c4ee71b80 1 -- 172.21.15.64:0/3071816454 wait complete. 2023-09-13T03:20:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.033+0000 7f7c4ee71b80 20 client.26701 trim_cache size 0 max 16384 2023-09-13T03:20:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.052+0000 7f7c4ee71b80 1 Processor -- start 2023-09-13T03:20:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.052+0000 7f7c4ee71b80 1 -- start start 2023-09-13T03:20:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.052+0000 7f7c4ee71b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633fd5f6300 0x5633fd607440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.052+0000 7f7c4ee71b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633fd5f6cc0 0x5633fd5f8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.052+0000 7f7c4ee71b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5633fd5fc1a0 0x5633fd5f90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.052+0000 7f7c4ee71b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633fd55d840 con 0x5633fd5f6cc0 2023-09-13T03:20:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.052+0000 7f7c4ee71b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633fd5f2b80 con 0x5633fd5f6300 2023-09-13T03:20:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.052+0000 7f7c4ee71b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633fd60da20 con 0x5633fd5fc1a0 2023-09-13T03:20:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.052+0000 7f7c3939b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633fd5f6300 0x5633fd607440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.052+0000 7f7c3939b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633fd5f6300 0x5633fd607440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:40894/0 (socket says 172.21.15.64:40894) 2023-09-13T03:20:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.052+0000 7f7c3939b700 1 -- 172.21.15.64:0/4064373172 learned_addr learned my addr 172.21.15.64:0/4064373172 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:20:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.052+0000 7f7c38b9a700 1 --2- 172.21.15.64:0/4064373172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633fd5f6cc0 0x5633fd5f8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.052+0000 7f7c39b9c700 1 --2- 172.21.15.64:0/4064373172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5633fd5fc1a0 0x5633fd5f90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.053+0000 7f7c3939b700 1 -- 172.21.15.64:0/4064373172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5633fd5fc1a0 msgr2=0x5633fd5f90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:20:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.053+0000 7f7c3939b700 1 --2- 172.21.15.64:0/4064373172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5633fd5fc1a0 0x5633fd5f90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.053+0000 7f7c3939b700 1 -- 172.21.15.64:0/4064373172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633fd5f6cc0 msgr2=0x5633fd5f8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:20:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.053+0000 7f7c3939b700 1 --2- 172.21.15.64:0/4064373172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633fd5f6cc0 0x5633fd5f8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.053+0000 7f7c3939b700 1 -- 172.21.15.64:0/4064373172 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c3410ff60 con 0x5633fd5f6300 2023-09-13T03:20:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.053+0000 7f7c3939b700 1 --2- 172.21.15.64:0/4064373172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633fd5f6300 0x5633fd607440 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7c30011ab0 tx=0x7f7c30014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff37306083b976f7 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:20:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.053+0000 7f7c1ffff700 1 -- 172.21.15.64:0/4064373172 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c30022030 con 0x5633fd5f6300 2023-09-13T03:20:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.053+0000 7f7c1ffff700 1 -- 172.21.15.64:0/4064373172 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c30023040 con 0x5633fd5f6300 2023-09-13T03:20:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.053+0000 7f7c1ffff700 1 -- 172.21.15.64:0/4064373172 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3000e720 con 0x5633fd5f6300 2023-09-13T03:20:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.054+0000 7f7c4ee71b80 1 -- 172.21.15.64:0/4064373172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633fd5f6300 msgr2=0x5633fd607440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.054+0000 7f7c4ee71b80 1 --2- 172.21.15.64:0/4064373172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633fd5f6300 0x5633fd607440 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7c30011ab0 tx=0x7f7c30014000 comp rx=0 tx=0).stop 2023-09-13T03:20:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.054+0000 7f7c4ee71b80 1 -- 172.21.15.64:0/4064373172 shutdown_connections 2023-09-13T03:20:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.054+0000 7f7c4ee71b80 1 --2- 172.21.15.64:0/4064373172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633fd5f6300 0x5633fd607440 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.054+0000 7f7c4ee71b80 1 --2- 172.21.15.64:0/4064373172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5633fd5fc1a0 0x5633fd5f90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.054+0000 7f7c4ee71b80 1 --2- 172.21.15.64:0/4064373172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633fd5f6cc0 0x5633fd5f8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.054+0000 7f7c4ee71b80 1 -- 172.21.15.64:0/4064373172 >> 172.21.15.64:0/4064373172 conn(0x5633fd56b030 msgr2=0x5633fd60bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:20:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.054+0000 7f7c4ee71b80 1 -- 172.21.15.64:0/4064373172 shutdown_connections 2023-09-13T03:20:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.054+0000 7f7c4ee71b80 1 -- 172.21.15.64:0/4064373172 wait complete. 2023-09-13T03:20:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.055+0000 7f7c4ee71b80 1 -- start start 2023-09-13T03:20:29.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.055+0000 7f7c4ee71b80 1 Processor -- start 2023-09-13T03:20:29.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.056+0000 7f7c4ee71b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633fd5f6300 0x5633fd5ffb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:29.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.056+0000 7f7c4ee71b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633fd5f6cc0 0x5633fd600070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:29.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.056+0000 7f7c4ee71b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5633fd5fc1a0 0x5633fd5fe890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:29.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.056+0000 7f7c4ee71b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633fd5fee10 con 0x5633fd5f6300 2023-09-13T03:20:29.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.056+0000 7f7c4ee71b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633fd5fef50 con 0x5633fd5f6cc0 2023-09-13T03:20:29.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.056+0000 7f7c4ee71b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633fd5ff110 con 0x5633fd5fc1a0 2023-09-13T03:20:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.056+0000 7f7c4ee71b80 10 client.0 _ll_register_callbacks cb 0x7f7c2c098d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:20:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.056+0000 7f7c38b9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633fd5f6cc0 0x5633fd600070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.056+0000 7f7c39b9c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5633fd5fc1a0 0x5633fd5fe890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.056+0000 7f7c38b9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633fd5f6cc0 0x5633fd600070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:40900/0 (socket says 172.21.15.64:40900) 2023-09-13T03:20:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.056+0000 7f7c38b9a700 1 -- 172.21.15.64:0/1132811242 learned_addr learned my addr 172.21.15.64:0/1132811242 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:20:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.056+0000 7f7c39b9c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5633fd5fc1a0 0x5633fd5fe890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:42180/0 (socket says 172.21.15.64:42180) 2023-09-13T03:20:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.056+0000 7f7c3939b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633fd5f6300 0x5633fd5ffb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.056+0000 7f7c38b9a700 1 -- 172.21.15.64:0/1132811242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5633fd5fc1a0 msgr2=0x5633fd5fe890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:20:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.056+0000 7f7c38b9a700 1 --2- 172.21.15.64:0/1132811242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5633fd5fc1a0 0x5633fd5fe890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.056+0000 7f7c38b9a700 1 -- 172.21.15.64:0/1132811242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633fd5f6300 msgr2=0x5633fd5ffb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:20:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.056+0000 7f7c38b9a700 1 --2- 172.21.15.64:0/1132811242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633fd5f6300 0x5633fd5ffb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.057+0000 7f7c38b9a700 1 -- 172.21.15.64:0/1132811242 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c300041b0 con 0x5633fd5f6cc0 2023-09-13T03:20:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.057+0000 7f7c38b9a700 1 --2- 172.21.15.64:0/1132811242 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633fd5f6cc0 0x5633fd600070 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7c24000c00 tx=0x7f7c24005790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3b17756a1617e32 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:20:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.057+0000 7f7c1cff9700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:20:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.057+0000 7f7c1cff9700 1 -- 172.21.15.64:0/1132811242 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2401d040 con 0x5633fd5f6cc0 2023-09-13T03:20:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.057+0000 7f7c4ee71b80 1 -- 172.21.15.64:0/1132811242 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7c2401e040 con 0x5633fd5f6cc0 2023-09-13T03:20:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.057+0000 7f7c1cff9700 1 -- 172.21.15.64:0/1132811242 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f7c2400b2d0 con 0x5633fd5f6cc0 2023-09-13T03:20:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.057+0000 7f7c1cff9700 1 -- 172.21.15.64:0/1132811242 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c24004ab0 con 0x5633fd5f6cc0 2023-09-13T03:20:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.057+0000 7f7c4ee71b80 10 client.26723 fetch_fsmap learned FSMap version 50 2023-09-13T03:20:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.057+0000 7f7c4ee71b80 1 -- 172.21.15.64:0/1132811242 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x5633fd35dc00 con 0x5633fd5f6cc0 2023-09-13T03:20:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.058+0000 7f7c1cff9700 1 -- 172.21.15.64:0/1132811242 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c2404a020 con 0x5633fd5f6cc0 2023-09-13T03:20:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.058+0000 7f7c1cff9700 1 -- 172.21.15.64:0/1132811242 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 50) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f7c24046440 con 0x5633fd5f6cc0 2023-09-13T03:20:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.058+0000 7f7c4ee71b80 10 client.26723 fetch_fsmap finished waiting for FSMap version 50 2023-09-13T03:20:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.058+0000 7f7c4ee71b80 10 client.26723 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:20:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.058+0000 7f7c4ee71b80 20 client.26723 populate_metadata read hostname 'smithi064' 2023-09-13T03:20:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.058+0000 7f7c4ee71b80 1 --2- 172.21.15.64:0/1132811242 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5633fd664f00 0x5633fd667320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.058+0000 7f7c4ee71b80 4 client.26723 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:20:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.058+0000 7f7c4ee71b80 1 -- 172.21.15.64:0/1132811242 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5633fd5418c0 con 0x5633fd664f00 2023-09-13T03:20:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.058+0000 7f7c3939b700 1 --2- 172.21.15.64:0/1132811242 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5633fd664f00 0x5633fd667320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.059+0000 7f7c3939b700 1 --2- 172.21.15.64:0/1132811242 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5633fd664f00 0x5633fd667320 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=729dd67666605f91 server_cookie=ba7702864cb45ad6 in_seq=0 out_seq=0 2023-09-13T03:20:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.059+0000 7f7c1cff9700 10 client.26723 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:20:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.083+0000 7f7c1cff9700 1 -- 172.21.15.64:0/1132811242 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5633fd5418c0 con 0x5633fd664f00 2023-09-13T03:20:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.083+0000 7f7c1cff9700 10 client.26723 handle_command_reply: tid=0 2023-09-13T03:20:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.083+0000 7f7c4ee71b80 1 client.26723 shutdown 2023-09-13T03:20:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.083+0000 7f7c4ee71b80 1 -- 172.21.15.64:0/1132811242 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633fd5f6cc0 msgr2=0x5633fd600070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.083+0000 7f7c4ee71b80 1 --2- 172.21.15.64:0/1132811242 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633fd5f6cc0 0x5633fd600070 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7c24000c00 tx=0x7f7c24005790 comp rx=0 tx=0).stop 2023-09-13T03:20:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.083+0000 7f7c4ee71b80 1 -- 172.21.15.64:0/1132811242 shutdown_connections 2023-09-13T03:20:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.083+0000 7f7c4ee71b80 1 --2- 172.21.15.64:0/1132811242 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5633fd664f00 0x5633fd667320 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.083+0000 7f7c4ee71b80 1 --2- 172.21.15.64:0/1132811242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5633fd5fc1a0 0x5633fd5fe890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:29.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.083+0000 7f7c4ee71b80 1 --2- 172.21.15.64:0/1132811242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633fd5f6300 0x5633fd5ffb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:29.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.083+0000 7f7c1cff9700 0 client.26723 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:20:29.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.083+0000 7f7c4ee71b80 1 --2- 172.21.15.64:0/1132811242 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633fd5f6cc0 0x5633fd600070 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:29.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.083+0000 7f7c4ee71b80 1 -- 172.21.15.64:0/1132811242 >> 172.21.15.64:0/1132811242 conn(0x5633fd56b030 msgr2=0x5633fd5fa250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:20:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.083+0000 7f7c4ee71b80 1 -- 172.21.15.64:0/1132811242 shutdown_connections 2023-09-13T03:20:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.083+0000 7f7c4ee71b80 1 -- 172.21.15.64:0/1132811242 wait complete. 2023-09-13T03:20:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.084+0000 7f7c4ee71b80 20 client.26723 trim_cache size 0 max 16384 2023-09-13T03:20:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:20:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:20:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:20:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:20:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.084+0000 7f7c3b5ff700 1 -- 172.21.15.64:0/951977612 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f7c200a0830 msgr2=0x7f7c200a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.084+0000 7f7c3b5ff700 1 --2- 172.21.15.64:0/951977612 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f7c200a0830 0x7f7c200a2cf0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7c30009df0 tx=0x7f7c30009260 comp rx=0 tx=0).stop 2023-09-13T03:20:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.084+0000 7f7c3b5ff700 1 -- 172.21.15.64:0/951977612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c3410ce20 msgr2=0x7f7c34198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.084+0000 7f7c3b5ff700 1 --2- 172.21.15.64:0/951977612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c3410ce20 0x7f7c34198740 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7c34069c60 tx=0x7f7c2c00c3a0 comp rx=0 tx=0).stop 2023-09-13T03:20:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.084+0000 7f7c3b5ff700 1 -- 172.21.15.64:0/951977612 shutdown_connections 2023-09-13T03:20:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.084+0000 7f7c3b5ff700 1 --2- 172.21.15.64:0/951977612 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f7c200a0830 0x7f7c200a2cf0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.084+0000 7f7c3b5ff700 1 --2- 172.21.15.64:0/951977612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c3410ce20 0x7f7c34198740 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.084+0000 7f7c3b5ff700 1 --2- 172.21.15.64:0/951977612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c340729d0 0x7f7c3419e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.085+0000 7f7c3b5ff700 1 --2- 172.21.15.64:0/951977612 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c340732f0 0x7f7c3419ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.085+0000 7f7c3b5ff700 1 -- 172.21.15.64:0/951977612 >> 172.21.15.64:0/951977612 conn(0x7f7c340fc0b0 msgr2=0x7f7c3410ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:20:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.085+0000 7f7c3b5ff700 1 -- 172.21.15.64:0/951977612 shutdown_connections 2023-09-13T03:20:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:29.085+0000 7f7c3b5ff700 1 -- 172.21.15.64:0/951977612 wait complete. 2023-09-13T03:20:29.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:20:28 smithi170 ceph-mon[134328]: pgmap v224: 97 pgs: 97 active+clean; 79 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 161 KiB/s wr, 36 op/s 2023-09-13T03:20:29.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:28 smithi064 ceph-mon[97675]: pgmap v224: 97 pgs: 97 active+clean; 79 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 161 KiB/s wr, 36 op/s 2023-09-13T03:20:29.926 INFO:tasks.workunit.client.0.smithi064.stdout: 1 150692 1.90 MB/sec execute 274 sec latency 785.850 ms 2023-09-13T03:20:30.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:20:30 smithi170 ceph-mon[134328]: pgmap v225: 97 pgs: 97 active+clean; 79 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 161 KiB/s wr, 36 op/s 2023-09-13T03:20:30.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:30 smithi064 ceph-mon[97675]: pgmap v225: 97 pgs: 97 active+clean; 79 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 161 KiB/s wr, 36 op/s 2023-09-13T03:20:30.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:20:30 smithi142 ceph-mon[134082]: pgmap v225: 97 pgs: 97 active+clean; 79 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 161 KiB/s wr, 36 op/s 2023-09-13T03:20:30.926 INFO:tasks.workunit.client.0.smithi064.stdout: 1 151925 1.92 MB/sec execute 275 sec latency 1209.239 ms 2023-09-13T03:20:31.926 INFO:tasks.workunit.client.0.smithi064.stdout: 1 153217 1.94 MB/sec execute 276 sec latency 40.187 ms 2023-09-13T03:20:32.926 INFO:tasks.workunit.client.0.smithi064.stdout: 1 153217 1.93 MB/sec execute 277 sec latency 1040.229 ms 2023-09-13T03:20:32.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:20:32 smithi142 ceph-mon[134082]: pgmap v226: 97 pgs: 97 active+clean; 78 MiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 487 KiB/s wr, 88 op/s 2023-09-13T03:20:33.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:20:32 smithi170 ceph-mon[134328]: pgmap v226: 97 pgs: 97 active+clean; 78 MiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 487 KiB/s wr, 88 op/s 2023-09-13T03:20:33.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:32 smithi064 ceph-mon[97675]: pgmap v226: 97 pgs: 97 active+clean; 78 MiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 487 KiB/s wr, 88 op/s 2023-09-13T03:20:33.926 INFO:tasks.workunit.client.0.smithi064.stdout: 1 153801 1.93 MB/sec execute 278 sec latency 1643.364 ms 2023-09-13T03:20:34.926 INFO:tasks.workunit.client.0.smithi064.stdout: 1 154242 1.94 MB/sec execute 279 sec latency 702.352 ms 2023-09-13T03:20:34.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:20:34 smithi142 ceph-mon[134082]: pgmap v227: 97 pgs: 97 active+clean; 78 MiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 434 KiB/s wr, 73 op/s 2023-09-13T03:20:35.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:20:34 smithi170 ceph-mon[134328]: pgmap v227: 97 pgs: 97 active+clean; 78 MiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 434 KiB/s wr, 73 op/s 2023-09-13T03:20:35.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:34 smithi064 ceph-mon[97675]: pgmap v227: 97 pgs: 97 active+clean; 78 MiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 434 KiB/s wr, 73 op/s 2023-09-13T03:20:35.926 INFO:tasks.workunit.client.0.smithi064.stdout: 1 155477 1.96 MB/sec execute 280 sec latency 1126.749 ms 2023-09-13T03:20:36.927 INFO:tasks.workunit.client.0.smithi064.stdout: 1 156774 1.97 MB/sec execute 281 sec latency 17.257 ms 2023-09-13T03:20:36.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:20:36 smithi142 ceph-mon[134082]: pgmap v228: 97 pgs: 97 active+clean; 81 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 747 KiB/s wr, 120 op/s 2023-09-13T03:20:37.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:20:36 smithi170 ceph-mon[134328]: pgmap v228: 97 pgs: 97 active+clean; 81 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 747 KiB/s wr, 120 op/s 2023-09-13T03:20:37.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:36 smithi064 ceph-mon[97675]: pgmap v228: 97 pgs: 97 active+clean; 81 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 747 KiB/s wr, 120 op/s 2023-09-13T03:20:37.927 INFO:tasks.workunit.client.0.smithi064.stdout: 1 156815 1.97 MB/sec execute 282 sec latency 973.971 ms 2023-09-13T03:20:38.927 INFO:tasks.workunit.client.0.smithi064.stdout: 1 156883 1.96 MB/sec execute 283 sec latency 1575.714 ms 2023-09-13T03:20:38.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:20:38 smithi142 ceph-mon[134082]: pgmap v229: 97 pgs: 97 active+clean; 81 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 639 KiB/s wr, 98 op/s 2023-09-13T03:20:39.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:20:38 smithi170 ceph-mon[134328]: pgmap v229: 97 pgs: 97 active+clean; 81 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 639 KiB/s wr, 98 op/s 2023-09-13T03:20:39.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:38 smithi064 ceph-mon[97675]: pgmap v229: 97 pgs: 97 active+clean; 81 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 639 KiB/s wr, 98 op/s 2023-09-13T03:20:39.927 INFO:tasks.workunit.client.0.smithi064.stdout: 1 156883 1.95 MB/sec execute 284 sec latency 1344.343 ms 2023-09-13T03:20:40.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:20:40 smithi170 ceph-mon[134328]: pgmap v230: 97 pgs: 97 active+clean; 81 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 639 KiB/s wr, 98 op/s 2023-09-13T03:20:40.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:40 smithi064 ceph-mon[97675]: pgmap v230: 97 pgs: 97 active+clean; 81 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 639 KiB/s wr, 98 op/s 2023-09-13T03:20:40.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:20:40 smithi142 ceph-mon[134082]: pgmap v230: 97 pgs: 97 active+clean; 81 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 639 KiB/s wr, 98 op/s 2023-09-13T03:20:40.927 INFO:tasks.workunit.client.0.smithi064.stdout: 1 157785 1.96 MB/sec execute 285 sec latency 1584.656 ms 2023-09-13T03:20:41.927 INFO:tasks.workunit.client.0.smithi064.stdout: 1 157785 1.96 MB/sec execute 286 sec latency 1129.841 ms 2023-09-13T03:20:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.246+0000 7f5dc49ff700 1 -- 172.21.15.64:0/2131007145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5dc0106660 msgr2=0x7f5dc0106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.246+0000 7f5dc49ff700 1 --2- 172.21.15.64:0/2131007145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5dc0106660 0x7f5dc0106a40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5db00098f0 tx=0x7f5db0009c00 comp rx=0 tx=0).stop 2023-09-13T03:20:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.246+0000 7f5dc49ff700 1 -- 172.21.15.64:0/2131007145 shutdown_connections 2023-09-13T03:20:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.246+0000 7f5dc49ff700 1 --2- 172.21.15.64:0/2131007145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5dc0100f60 0x7f5dc010d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.246+0000 7f5dc49ff700 1 --2- 172.21.15.64:0/2131007145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5dc0106660 0x7f5dc0106a40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.246+0000 7f5dc49ff700 1 --2- 172.21.15.64:0/2131007145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5dc01005a0 0x7f5dc0100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.246+0000 7f5dc49ff700 1 -- 172.21.15.64:0/2131007145 >> 172.21.15.64:0/2131007145 conn(0x7f5dc00fc050 msgr2=0x7f5dc00fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:20:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.247+0000 7f5dc49ff700 1 -- 172.21.15.64:0/2131007145 shutdown_connections 2023-09-13T03:20:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.247+0000 7f5dc49ff700 1 -- 172.21.15.64:0/2131007145 wait complete. 2023-09-13T03:20:42.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.248+0000 7f5dc49ff700 1 Processor -- start 2023-09-13T03:20:42.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.248+0000 7f5dc49ff700 1 -- start start 2023-09-13T03:20:42.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.248+0000 7f5dc49ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5dc01005a0 0x7f5dc019e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:42.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.248+0000 7f5dc49ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5dc0100f60 0x7f5dc019ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:42.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.248+0000 7f5dc49ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5dc0106660 0x7f5dc01986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:42.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.248+0000 7f5dc49ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc0198d20 con 0x7f5dc01005a0 2023-09-13T03:20:42.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.248+0000 7f5dc49ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc0198e90 con 0x7f5dc0100f60 2023-09-13T03:20:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.248+0000 7f5dc49ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc0199050 con 0x7f5dc0106660 2023-09-13T03:20:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.248+0000 7f5dbe59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5dc01005a0 0x7f5dc019e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.249+0000 7f5dbe59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5dc01005a0 0x7f5dc019e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42662/0 (socket says 172.21.15.64:42662) 2023-09-13T03:20:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.249+0000 7f5dbe59c700 1 -- 172.21.15.64:0/232799339 learned_addr learned my addr 172.21.15.64:0/232799339 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:20:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.249+0000 7f5db7fff700 1 --2- 172.21.15.64:0/232799339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5dc0100f60 0x7f5dc019ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:42.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.249+0000 7f5dbed9d700 1 --2- 172.21.15.64:0/232799339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5dc0106660 0x7f5dc01986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:42.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.249+0000 7f5dbe59c700 1 -- 172.21.15.64:0/232799339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5dc0106660 msgr2=0x7f5dc01986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:42.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.249+0000 7f5dbe59c700 1 --2- 172.21.15.64:0/232799339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5dc0106660 0x7f5dc01986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:42.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.249+0000 7f5dbe59c700 1 -- 172.21.15.64:0/232799339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5dc0100f60 msgr2=0x7f5dc019ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:42.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.249+0000 7f5dbe59c700 1 --2- 172.21.15.64:0/232799339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5dc0100f60 0x7f5dc019ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:42.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.249+0000 7f5dbe59c700 1 -- 172.21.15.64:0/232799339 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da80095f0 con 0x7f5dc01005a0 2023-09-13T03:20:42.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.249+0000 7f5dbed9d700 1 --2- 172.21.15.64:0/232799339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5dc0106660 0x7f5dc01986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:20:42.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.249+0000 7f5dbe59c700 1 --2- 172.21.15.64:0/232799339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5dc01005a0 0x7f5dc019e850 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5db0009230 tx=0x7f5db000fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:20:42.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.249+0000 7f5db77fe700 1 -- 172.21.15.64:0/232799339 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db001c070 con 0x7f5dc01005a0 2023-09-13T03:20:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.249+0000 7f5dc49ff700 1 -- 172.21.15.64:0/232799339 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5db0009520 con 0x7f5dc01005a0 2023-09-13T03:20:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.249+0000 7f5db77fe700 1 -- 172.21.15.64:0/232799339 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5db00058e0 con 0x7f5dc01005a0 2023-09-13T03:20:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.249+0000 7f5db77fe700 1 -- 172.21.15.64:0/232799339 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db00038e0 con 0x7f5dc01005a0 2023-09-13T03:20:42.258 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.250+0000 7f5dc49ff700 1 -- 172.21.15.64:0/232799339 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc01a5660 con 0x7f5dc01005a0 2023-09-13T03:20:42.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.253+0000 7f5db77fe700 1 -- 172.21.15.64:0/232799339 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f5db0020030 con 0x7f5dc01005a0 2023-09-13T03:20:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.254+0000 7f5db77fe700 1 --2- 172.21.15.64:0/232799339 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f5dac0a07b0 0x7f5dac0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.254+0000 7f5dd840cb80 1 Processor -- start 2023-09-13T03:20:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.254+0000 7f5db77fe700 1 -- 172.21.15.64:0/232799339 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f5db00ccec0 con 0x7f5dc01005a0 2023-09-13T03:20:42.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.255+0000 7f5db7fff700 1 --2- 172.21.15.64:0/232799339 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f5dac0a07b0 0x7f5dac0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:42.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.255+0000 7f5dd840cb80 1 -- start start 2023-09-13T03:20:42.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.255+0000 7f5dd840cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a52868f9b0 0x55a52868fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:42.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.255+0000 7f5dd840cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a528689b10 0x55a528689f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:42.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.255+0000 7f5dd840cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52868a4d0 0x55a528696740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:42.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.255+0000 7f5dd840cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5285da6d0 con 0x55a52868a4d0 2023-09-13T03:20:42.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.255+0000 7f5dd840cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5285d7ff0 con 0x55a52868f9b0 2023-09-13T03:20:42.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.255+0000 7f5dd840cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5285d38e0 con 0x55a528689b10 2023-09-13T03:20:42.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.255+0000 7f5dbe59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a52868f9b0 0x55a52868fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:42.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.255+0000 7f5dbe59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a52868f9b0 0x55a52868fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:42042/0 (socket says 172.21.15.64:42042) 2023-09-13T03:20:42.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.255+0000 7f5dbe59c700 1 -- 172.21.15.64:0/2259535005 learned_addr learned my addr 172.21.15.64:0/2259535005 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:20:42.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.255+0000 7f5dbed9d700 1 --2- 172.21.15.64:0/2259535005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52868a4d0 0x55a528696740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:42.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.256+0000 7f5db7fff700 1 --2- 172.21.15.64:0/232799339 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f5dac0a07b0 0x7f5dac0a2c70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5dc00ffb10 tx=0x7f5da8009310 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:20:42.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.256+0000 7f5db7fff700 1 --2- 172.21.15.64:0/2259535005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a528689b10 0x55a528689f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:42.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.256+0000 7f5dbe59c700 1 -- 172.21.15.64:0/2259535005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a528689b10 msgr2=0x55a528689f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:20:42.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.256+0000 7f5dbe59c700 1 --2- 172.21.15.64:0/2259535005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a528689b10 0x55a528689f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:42.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.256+0000 7f5dbe59c700 1 -- 172.21.15.64:0/2259535005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52868a4d0 msgr2=0x55a528696740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:20:42.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.256+0000 7f5dbe59c700 1 --2- 172.21.15.64:0/2259535005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52868a4d0 0x55a528696740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:42.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.256+0000 7f5dbe59c700 1 -- 172.21.15.64:0/2259535005 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5283ffd80 con 0x55a52868f9b0 2023-09-13T03:20:42.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.256+0000 7f5db7fff700 1 --2- 172.21.15.64:0/2259535005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a528689b10 0x55a528689f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T03:20:42.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.256+0000 7f5dbe59c700 1 --2- 172.21.15.64:0/2259535005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a52868f9b0 0x55a52868fd90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5db00a1e70 tx=0x7f5db0005570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=307029a3e5bbaa64 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:20:42.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.256+0000 7f5db4ff9700 1 -- 172.21.15.64:0/2259535005 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db009aac0 con 0x55a52868f9b0 2023-09-13T03:20:42.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.256+0000 7f5db4ff9700 1 -- 172.21.15.64:0/2259535005 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5db009a3c0 con 0x55a52868f9b0 2023-09-13T03:20:42.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.256+0000 7f5db4ff9700 1 -- 172.21.15.64:0/2259535005 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db00051d0 con 0x55a52868f9b0 2023-09-13T03:20:42.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.257+0000 7f5dd840cb80 1 -- 172.21.15.64:0/2259535005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a52868f9b0 msgr2=0x55a52868fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:42.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.257+0000 7f5dd840cb80 1 --2- 172.21.15.64:0/2259535005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a52868f9b0 0x55a52868fd90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5db00a1e70 tx=0x7f5db0005570 comp rx=0 tx=0).stop 2023-09-13T03:20:42.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.257+0000 7f5dd840cb80 1 -- 172.21.15.64:0/2259535005 shutdown_connections 2023-09-13T03:20:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.257+0000 7f5dd840cb80 1 --2- 172.21.15.64:0/2259535005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a52868f9b0 0x55a52868fd90 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.257+0000 7f5dd840cb80 1 --2- 172.21.15.64:0/2259535005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52868a4d0 0x55a528696740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.257+0000 7f5dd840cb80 1 --2- 172.21.15.64:0/2259535005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a528689b10 0x55a528689f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.257+0000 7f5dd840cb80 1 -- 172.21.15.64:0/2259535005 >> 172.21.15.64:0/2259535005 conn(0x55a5285f6030 msgr2=0x55a52867c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:20:42.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.257+0000 7f5dd840cb80 1 -- 172.21.15.64:0/2259535005 shutdown_connections 2023-09-13T03:20:42.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.257+0000 7f5dd840cb80 1 -- 172.21.15.64:0/2259535005 wait complete. 2023-09-13T03:20:42.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.258+0000 7f5dd840cb80 1 -- start start 2023-09-13T03:20:42.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.258+0000 7f5dd840cb80 1 Processor -- start 2023-09-13T03:20:42.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.259+0000 7f5dd840cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a528689b10 0x55a528699300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:42.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.259+0000 7f5dd840cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52868a4d0 0x55a528699840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.259+0000 7f5dd840cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a52868f9b0 0x55a528693590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.259+0000 7f5dd840cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a528571b10 con 0x55a52868a4d0 2023-09-13T03:20:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.259+0000 7f5dd840cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a528571810 con 0x55a52868f9b0 2023-09-13T03:20:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.259+0000 7f5dd840cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5284558c0 con 0x55a528689b10 2023-09-13T03:20:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.259+0000 7f5dd840cb80 10 client.0 _ll_register_callbacks cb 0x55a5284bba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:20:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.259+0000 7f5db7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52868a4d0 0x55a528699840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.259+0000 7f5dbe59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a528689b10 0x55a528699300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.259+0000 7f5dbe59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a528689b10 0x55a528699300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:54896/0 (socket says 172.21.15.64:54896) 2023-09-13T03:20:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.259+0000 7f5db7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52868a4d0 0x55a528699840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42686/0 (socket says 172.21.15.64:42686) 2023-09-13T03:20:42.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.259+0000 7f5dbe59c700 1 -- 172.21.15.64:0/1901610737 learned_addr learned my addr 172.21.15.64:0/1901610737 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:20:42.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.259+0000 7f5dbed9d700 1 --2- 172.21.15.64:0/1901610737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a52868f9b0 0x55a528693590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:42.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.259+0000 7f5db7fff700 1 -- 172.21.15.64:0/1901610737 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a528689b10 msgr2=0x55a528699300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:20:42.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.259+0000 7f5db7fff700 1 --2- 172.21.15.64:0/1901610737 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a528689b10 0x55a528699300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:42.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.259+0000 7f5db7fff700 1 -- 172.21.15.64:0/1901610737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a52868f9b0 msgr2=0x55a528693590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:20:42.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.259+0000 7f5db7fff700 1 --2- 172.21.15.64:0/1901610737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a52868f9b0 0x55a528693590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:42.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.259+0000 7f5db7fff700 1 -- 172.21.15.64:0/1901610737 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc01a5660 con 0x55a52868a4d0 2023-09-13T03:20:42.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.259+0000 7f5dbe59c700 1 --2- 172.21.15.64:0/1901610737 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a528689b10 0x55a528699300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:20:42.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.260+0000 7f5db7fff700 1 --2- 172.21.15.64:0/1901610737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52868a4d0 0x55a528699840 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x55a52867c060 tx=0x7f5da80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c915bf5045de3 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:20:42.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.260+0000 7f5da5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:20:42.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.260+0000 7f5da5ffb700 1 -- 172.21.15.64:0/1901610737 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da8004230 con 0x55a52868a4d0 2023-09-13T03:20:42.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.260+0000 7f5dd840cb80 1 -- 172.21.15.64:0/1901610737 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a52852b490 con 0x55a52868a4d0 2023-09-13T03:20:42.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.260+0000 7f5da5ffb700 1 -- 172.21.15.64:0/1901610737 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f5da8004390 con 0x55a52868a4d0 2023-09-13T03:20:42.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.260+0000 7f5da5ffb700 1 -- 172.21.15.64:0/1901610737 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5da8004550 con 0x55a52868a4d0 2023-09-13T03:20:42.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.260+0000 7f5dd840cb80 10 client.16695 fetch_fsmap learned FSMap version 50 2023-09-13T03:20:42.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.260+0000 7f5dd840cb80 1 -- 172.21.15.64:0/1901610737 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55a5283e8c00 con 0x55a52868a4d0 2023-09-13T03:20:42.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.261+0000 7f5da5ffb700 1 -- 172.21.15.64:0/1901610737 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a52852b490 con 0x55a52868a4d0 2023-09-13T03:20:42.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.261+0000 7f5da5ffb700 1 -- 172.21.15.64:0/1901610737 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 50) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f5da800eb60 con 0x55a52868a4d0 2023-09-13T03:20:42.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.261+0000 7f5dd840cb80 10 client.16695 fetch_fsmap finished waiting for FSMap version 50 2023-09-13T03:20:42.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.261+0000 7f5dd840cb80 10 client.16695 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:20:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.261+0000 7f5dd840cb80 20 client.16695 populate_metadata read hostname 'smithi064' 2023-09-13T03:20:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.261+0000 7f5dd840cb80 1 --2- 172.21.15.64:0/1901610737 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55a5286a3870 0x55a5286a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.261+0000 7f5dd840cb80 4 client.16695 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:20:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.261+0000 7f5dd840cb80 1 -- 172.21.15.64:0/1901610737 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a5285e8840 con 0x55a5286a3870 2023-09-13T03:20:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.261+0000 7f5dbe59c700 1 --2- 172.21.15.64:0/1901610737 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55a5286a3870 0x55a5286a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.262+0000 7f5dbe59c700 1 --2- 172.21.15.64:0/1901610737 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55a5286a3870 0x55a5286a5c90 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb902c095e395b03 server_cookie=927c893c923c02ca in_seq=0 out_seq=0 2023-09-13T03:20:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.262+0000 7f5da5ffb700 10 client.16695 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:20:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.263+0000 7f5da5ffb700 1 -- 172.21.15.64:0/1901610737 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a5285e8840 con 0x55a5286a3870 2023-09-13T03:20:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.263+0000 7f5da5ffb700 10 client.16695 handle_command_reply: tid=0 2023-09-13T03:20:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.263+0000 7f5dd840cb80 1 client.16695 shutdown 2023-09-13T03:20:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.263+0000 7f5dd840cb80 1 -- 172.21.15.64:0/1901610737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52868a4d0 msgr2=0x55a528699840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.263+0000 7f5dd840cb80 1 --2- 172.21.15.64:0/1901610737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52868a4d0 0x55a528699840 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x55a52867c060 tx=0x7f5da80047c0 comp rx=0 tx=0).stop 2023-09-13T03:20:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.263+0000 7f5dd840cb80 1 -- 172.21.15.64:0/1901610737 shutdown_connections 2023-09-13T03:20:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.263+0000 7f5dd840cb80 1 --2- 172.21.15.64:0/1901610737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a52868f9b0 0x55a528693590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.263+0000 7f5dd840cb80 1 --2- 172.21.15.64:0/1901610737 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55a5286a3870 0x55a5286a5c90 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.263+0000 7f5dd840cb80 1 --2- 172.21.15.64:0/1901610737 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a528689b10 0x55a528699300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.263+0000 7f5dd840cb80 1 --2- 172.21.15.64:0/1901610737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52868a4d0 0x55a528699840 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.263+0000 7f5da5ffb700 0 client.16695 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:20:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.263+0000 7f5dd840cb80 1 -- 172.21.15.64:0/1901610737 >> 172.21.15.64:0/1901610737 conn(0x55a5285f6030 msgr2=0x55a52867b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:20:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.263+0000 7f5dd840cb80 1 -- 172.21.15.64:0/1901610737 shutdown_connections 2023-09-13T03:20:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.263+0000 7f5dd840cb80 1 -- 172.21.15.64:0/1901610737 wait complete. 2023-09-13T03:20:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.263+0000 7f5dd840cb80 20 client.16695 trim_cache size 0 max 16384 2023-09-13T03:20:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.279+0000 7f5dd840cb80 1 Processor -- start 2023-09-13T03:20:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.279+0000 7f5dd840cb80 1 -- start start 2023-09-13T03:20:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.279+0000 7f5dd840cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a528689b10 0x55a52867c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.279+0000 7f5dd840cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a52868a4d0 0x55a52868f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.279+0000 7f5dd840cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52868f9b0 0x55a528692820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.279+0000 7f5dd840cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a52868f670 con 0x55a52868f9b0 2023-09-13T03:20:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.279+0000 7f5dd840cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5285e8840 con 0x55a52868a4d0 2023-09-13T03:20:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.279+0000 7f5dd840cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a52852b490 con 0x55a528689b10 2023-09-13T03:20:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.279+0000 7f5dbed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52868f9b0 0x55a528692820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.280+0000 7f5dbed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52868f9b0 0x55a528692820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42690/0 (socket says 172.21.15.64:42690) 2023-09-13T03:20:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.280+0000 7f5dbed9d700 1 -- 172.21.15.64:0/4121610845 learned_addr learned my addr 172.21.15.64:0/4121610845 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:20:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.280+0000 7f5db7fff700 1 --2- 172.21.15.64:0/4121610845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a52868a4d0 0x55a52868f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.280+0000 7f5dbe59c700 1 --2- 172.21.15.64:0/4121610845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a528689b10 0x55a52867c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.280+0000 7f5dbed9d700 1 -- 172.21.15.64:0/4121610845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a528689b10 msgr2=0x55a52867c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:20:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.280+0000 7f5dbed9d700 1 --2- 172.21.15.64:0/4121610845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a528689b10 0x55a52867c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.280+0000 7f5dbed9d700 1 -- 172.21.15.64:0/4121610845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a52868a4d0 msgr2=0x55a52868f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:20:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.280+0000 7f5dbed9d700 1 --2- 172.21.15.64:0/4121610845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a52868a4d0 0x55a52868f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.280+0000 7f5dbed9d700 1 -- 172.21.15.64:0/4121610845 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5286aa8c0 con 0x55a52868f9b0 2023-09-13T03:20:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.280+0000 7f5dbed9d700 1 --2- 172.21.15.64:0/4121610845 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52868f9b0 0x55a528692820 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f5db8007670 tx=0x7f5db800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f0cc69d839587 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:20:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.281+0000 7f5db57fa700 1 -- 172.21.15.64:0/4121610845 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db800e040 con 0x55a52868f9b0 2023-09-13T03:20:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.281+0000 7f5db57fa700 1 -- 172.21.15.64:0/4121610845 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5db800f040 con 0x55a52868f9b0 2023-09-13T03:20:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.281+0000 7f5db57fa700 1 -- 172.21.15.64:0/4121610845 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db801f860 con 0x55a52868f9b0 2023-09-13T03:20:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.281+0000 7f5dd840cb80 1 -- 172.21.15.64:0/4121610845 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52868f9b0 msgr2=0x55a528692820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.281+0000 7f5dd840cb80 1 --2- 172.21.15.64:0/4121610845 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52868f9b0 0x55a528692820 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f5db8007670 tx=0x7f5db800b040 comp rx=0 tx=0).stop 2023-09-13T03:20:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.281+0000 7f5dd840cb80 1 -- 172.21.15.64:0/4121610845 shutdown_connections 2023-09-13T03:20:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.281+0000 7f5dd840cb80 1 --2- 172.21.15.64:0/4121610845 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52868f9b0 0x55a528692820 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.281+0000 7f5dd840cb80 1 --2- 172.21.15.64:0/4121610845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a528689b10 0x55a52867c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.281+0000 7f5dd840cb80 1 --2- 172.21.15.64:0/4121610845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a52868a4d0 0x55a52868f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.281+0000 7f5dd840cb80 1 -- 172.21.15.64:0/4121610845 >> 172.21.15.64:0/4121610845 conn(0x55a5285f6030 msgr2=0x55a528692e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:20:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.281+0000 7f5dd840cb80 1 -- 172.21.15.64:0/4121610845 shutdown_connections 2023-09-13T03:20:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.281+0000 7f5dd840cb80 1 -- 172.21.15.64:0/4121610845 wait complete. 2023-09-13T03:20:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.282+0000 7f5dd840cb80 1 -- start start 2023-09-13T03:20:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.282+0000 7f5dd840cb80 1 Processor -- start 2023-09-13T03:20:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.282+0000 7f5dd840cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a528689b10 0x55a5286eb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.282+0000 7f5dd840cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a52868a4d0 0x55a5286ebb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.282+0000 7f5dd840cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52868f9b0 0x55a5286f66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.282+0000 7f5dd840cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5286ec370 con 0x55a52868f9b0 2023-09-13T03:20:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.282+0000 7f5dd840cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5286ec4e0 con 0x55a528689b10 2023-09-13T03:20:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.282+0000 7f5dd840cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5286f6c70 con 0x55a52868a4d0 2023-09-13T03:20:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.282+0000 7f5dd840cb80 10 client.0 _ll_register_callbacks cb 0x7f5da801d4c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:20:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.283+0000 7f5dbed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52868f9b0 0x55a5286f66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.283+0000 7f5dbed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52868f9b0 0x55a5286f66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42696/0 (socket says 172.21.15.64:42696) 2023-09-13T03:20:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.283+0000 7f5dbe59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a528689b10 0x55a5286eb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.283+0000 7f5dbed9d700 1 -- 172.21.15.64:0/1823891658 learned_addr learned my addr 172.21.15.64:0/1823891658 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:20:42.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.283+0000 7f5db7fff700 1 --2- 172.21.15.64:0/1823891658 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a52868a4d0 0x55a5286ebb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:42.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.283+0000 7f5dbed9d700 1 -- 172.21.15.64:0/1823891658 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a52868a4d0 msgr2=0x55a5286ebb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:20:42.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.283+0000 7f5dbed9d700 1 --2- 172.21.15.64:0/1823891658 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a52868a4d0 0x55a5286ebb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:42.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.283+0000 7f5dbed9d700 1 -- 172.21.15.64:0/1823891658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a528689b10 msgr2=0x55a5286eb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:20:42.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.283+0000 7f5dbed9d700 1 --2- 172.21.15.64:0/1823891658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a528689b10 0x55a5286eb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:42.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.283+0000 7f5dbed9d700 1 -- 172.21.15.64:0/1823891658 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5db8009340 con 0x55a52868f9b0 2023-09-13T03:20:42.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.283+0000 7f5dbed9d700 1 --2- 172.21.15.64:0/1823891658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52868f9b0 0x55a5286f66b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5db8017a30 tx=0x7f5db8017b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20308e0126b4339b server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:20:42.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.283+0000 7f5da5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:20:42.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.284+0000 7f5da5ffb700 1 -- 172.21.15.64:0/1823891658 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db800e070 con 0x55a52868f9b0 2023-09-13T03:20:42.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.284+0000 7f5dd840cb80 1 -- 172.21.15.64:0/1823891658 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5db009a3c0 con 0x55a52868f9b0 2023-09-13T03:20:42.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.284+0000 7f5da5ffb700 1 -- 172.21.15.64:0/1823891658 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f5db801f810 con 0x55a52868f9b0 2023-09-13T03:20:42.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.284+0000 7f5da5ffb700 1 -- 172.21.15.64:0/1823891658 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5db801f9d0 con 0x55a52868f9b0 2023-09-13T03:20:42.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.284+0000 7f5dd840cb80 10 client.16701 fetch_fsmap learned FSMap version 50 2023-09-13T03:20:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.284+0000 7f5dd840cb80 1 -- 172.21.15.64:0/1823891658 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55a5284261f0 con 0x55a52868f9b0 2023-09-13T03:20:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.284+0000 7f5da5ffb700 1 -- 172.21.15.64:0/1823891658 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5db8034020 con 0x55a52868f9b0 2023-09-13T03:20:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.284+0000 7f5da5ffb700 1 -- 172.21.15.64:0/1823891658 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 50) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f5db801d020 con 0x55a52868f9b0 2023-09-13T03:20:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.284+0000 7f5dd840cb80 10 client.16701 fetch_fsmap finished waiting for FSMap version 50 2023-09-13T03:20:42.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.284+0000 7f5dd840cb80 10 client.16701 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:20:42.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.284+0000 7f5dd840cb80 20 client.16701 populate_metadata read hostname 'smithi064' 2023-09-13T03:20:42.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.284+0000 7f5dd840cb80 1 --2- 172.21.15.64:0/1823891658 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55a5286f8410 0x55a5286fa830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:42.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.284+0000 7f5dd840cb80 4 client.16701 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:20:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.284+0000 7f5dd840cb80 1 -- 172.21.15.64:0/1823891658 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a5285cc8c0 con 0x55a5286f8410 2023-09-13T03:20:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.285+0000 7f5dbe59c700 1 --2- 172.21.15.64:0/1823891658 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55a5286f8410 0x55a5286fa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.285+0000 7f5da5ffb700 10 client.16701 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:20:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.285+0000 7f5dbe59c700 1 --2- 172.21.15.64:0/1823891658 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55a5286f8410 0x55a5286fa830 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b13b131675ff76c server_cookie=e336f205cbbe814a in_seq=0 out_seq=0 2023-09-13T03:20:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.309+0000 7f5da5ffb700 1 -- 172.21.15.64:0/1823891658 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a5285cc8c0 con 0x55a5286f8410 2023-09-13T03:20:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.309+0000 7f5da5ffb700 10 client.16701 handle_command_reply: tid=0 2023-09-13T03:20:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.309+0000 7f5dd840cb80 1 client.16701 shutdown 2023-09-13T03:20:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.310+0000 7f5dd840cb80 1 -- 172.21.15.64:0/1823891658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52868f9b0 msgr2=0x55a5286f66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.310+0000 7f5dd840cb80 1 --2- 172.21.15.64:0/1823891658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52868f9b0 0x55a5286f66b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5db8017a30 tx=0x7f5db8017b10 comp rx=0 tx=0).stop 2023-09-13T03:20:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.310+0000 7f5dd840cb80 1 -- 172.21.15.64:0/1823891658 shutdown_connections 2023-09-13T03:20:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.310+0000 7f5dd840cb80 1 --2- 172.21.15.64:0/1823891658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a528689b10 0x55a5286eb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.310+0000 7f5dd840cb80 1 --2- 172.21.15.64:0/1823891658 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55a5286f8410 0x55a5286fa830 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.310+0000 7f5dd840cb80 1 --2- 172.21.15.64:0/1823891658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a52868f9b0 0x55a5286f66b0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.310+0000 7f5dd840cb80 1 --2- 172.21.15.64:0/1823891658 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a52868a4d0 0x55a5286ebb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.310+0000 7f5dd840cb80 1 -- 172.21.15.64:0/1823891658 >> 172.21.15.64:0/1823891658 conn(0x55a5285f6030 msgr2=0x55a52867af10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:20:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.310+0000 7f5da5ffb700 0 client.16701 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:20:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.310+0000 7f5dd840cb80 1 -- 172.21.15.64:0/1823891658 shutdown_connections 2023-09-13T03:20:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.310+0000 7f5dd840cb80 1 -- 172.21.15.64:0/1823891658 wait complete. 2023-09-13T03:20:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.310+0000 7f5dd840cb80 20 client.16701 trim_cache size 0 max 16384 2023-09-13T03:20:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:20:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:20:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:20:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:20:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.311+0000 7f5dc49ff700 1 -- 172.21.15.64:0/232799339 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f5dac0a07b0 msgr2=0x7f5dac0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.311+0000 7f5dc49ff700 1 --2- 172.21.15.64:0/232799339 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f5dac0a07b0 0x7f5dac0a2c70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5dc00ffb10 tx=0x7f5da8009310 comp rx=0 tx=0).stop 2023-09-13T03:20:42.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.311+0000 7f5dc49ff700 1 -- 172.21.15.64:0/232799339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5dc01005a0 msgr2=0x7f5dc019e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:42.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.311+0000 7f5dc49ff700 1 --2- 172.21.15.64:0/232799339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5dc01005a0 0x7f5dc019e850 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5db0009230 tx=0x7f5db000fa80 comp rx=0 tx=0).stop 2023-09-13T03:20:42.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.311+0000 7f5dc49ff700 1 -- 172.21.15.64:0/232799339 shutdown_connections 2023-09-13T03:20:42.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.311+0000 7f5dc49ff700 1 --2- 172.21.15.64:0/232799339 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f5dac0a07b0 0x7f5dac0a2c70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:42.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.311+0000 7f5dc49ff700 1 --2- 172.21.15.64:0/232799339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5dc0106660 0x7f5dc01986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:42.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.311+0000 7f5dc49ff700 1 --2- 172.21.15.64:0/232799339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5dc01005a0 0x7f5dc019e850 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:42.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.311+0000 7f5dc49ff700 1 --2- 172.21.15.64:0/232799339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5dc0100f60 0x7f5dc019ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:42.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.311+0000 7f5dc49ff700 1 -- 172.21.15.64:0/232799339 >> 172.21.15.64:0/232799339 conn(0x7f5dc00fc050 msgr2=0x7f5dc010b850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:20:42.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.311+0000 7f5dc49ff700 1 -- 172.21.15.64:0/232799339 shutdown_connections 2023-09-13T03:20:42.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:42.311+0000 7f5dc49ff700 1 -- 172.21.15.64:0/232799339 wait complete. 2023-09-13T03:20:42.927 INFO:tasks.workunit.client.0.smithi064.stdout: 1 157785 1.95 MB/sec execute 287 sec latency 2129.891 ms 2023-09-13T03:20:42.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:20:42 smithi142 ceph-mon[134082]: pgmap v231: 97 pgs: 97 active+clean; 83 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 176 op/s 2023-09-13T03:20:43.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:20:42 smithi170 ceph-mon[134328]: pgmap v231: 97 pgs: 97 active+clean; 83 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 176 op/s 2023-09-13T03:20:43.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:42 smithi064 ceph-mon[97675]: pgmap v231: 97 pgs: 97 active+clean; 83 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 176 op/s 2023-09-13T03:20:43.927 INFO:tasks.workunit.client.0.smithi064.stdout: 1 159783 1.99 MB/sec execute 288 sec latency 2189.215 ms 2023-09-13T03:20:44.927 INFO:tasks.workunit.client.0.smithi064.stdout: 1 160306 1.98 MB/sec execute 289 sec latency 448.361 ms 2023-09-13T03:20:44.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:20:44 smithi142 ceph-mon[134082]: pgmap v232: 97 pgs: 97 active+clean; 83 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 1.6 MiB/s wr, 124 op/s 2023-09-13T03:20:45.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:20:44 smithi170 ceph-mon[134328]: pgmap v232: 97 pgs: 97 active+clean; 83 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 1.6 MiB/s wr, 124 op/s 2023-09-13T03:20:45.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:44 smithi064 ceph-mon[97675]: pgmap v232: 97 pgs: 97 active+clean; 83 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 1.6 MiB/s wr, 124 op/s 2023-09-13T03:20:45.927 INFO:tasks.workunit.client.0.smithi064.stdout: 1 160306 1.98 MB/sec execute 290 sec latency 1448.458 ms 2023-09-13T03:20:45.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:20:45 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:20:46.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:20:45 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:20:46.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:45 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:20:46.927 INFO:tasks.workunit.client.0.smithi064.stdout: 1 160306 1.97 MB/sec execute 291 sec latency 2448.525 ms 2023-09-13T03:20:46.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:20:46 smithi142 ceph-mon[134082]: pgmap v233: 97 pgs: 97 active+clean; 81 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 167 op/s 2023-09-13T03:20:47.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:20:46 smithi170 ceph-mon[134328]: pgmap v233: 97 pgs: 97 active+clean; 81 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 167 op/s 2023-09-13T03:20:47.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:46 smithi064 ceph-mon[97675]: pgmap v233: 97 pgs: 97 active+clean; 81 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 167 op/s 2023-09-13T03:20:47.928 INFO:tasks.workunit.client.0.smithi064.stdout: 1 160306 1.96 MB/sec execute 292 sec latency 3448.663 ms 2023-09-13T03:20:48.928 INFO:tasks.workunit.client.0.smithi064.stdout: 1 160897 1.96 MB/sec execute 293 sec latency 4051.322 ms 2023-09-13T03:20:48.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:20:48 smithi142 ceph-mon[134082]: pgmap v234: 97 pgs: 97 active+clean; 81 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 1.5 MiB/s wr, 120 op/s 2023-09-13T03:20:49.159 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:20:48 smithi170 ceph-mon[134328]: pgmap v234: 97 pgs: 97 active+clean; 81 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 1.5 MiB/s wr, 120 op/s 2023-09-13T03:20:49.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:48 smithi064 ceph-mon[97675]: pgmap v234: 97 pgs: 97 active+clean; 81 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 1.5 MiB/s wr, 120 op/s 2023-09-13T03:20:49.928 INFO:tasks.workunit.client.0.smithi064.stdout: 1 161331 1.97 MB/sec execute 294 sec latency 727.392 ms 2023-09-13T03:20:50.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:20:50 smithi170 ceph-mon[134328]: pgmap v235: 97 pgs: 97 active+clean; 81 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 1.5 MiB/s wr, 120 op/s 2023-09-13T03:20:50.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:50 smithi064 ceph-mon[97675]: pgmap v235: 97 pgs: 97 active+clean; 81 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 1.5 MiB/s wr, 120 op/s 2023-09-13T03:20:50.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:20:50 smithi142 ceph-mon[134082]: pgmap v235: 97 pgs: 97 active+clean; 81 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 1.5 MiB/s wr, 120 op/s 2023-09-13T03:20:50.928 INFO:tasks.workunit.client.0.smithi064.stdout: 1 162561 1.99 MB/sec execute 295 sec latency 1151.116 ms 2023-09-13T03:20:51.928 INFO:tasks.workunit.client.0.smithi064.stdout: 1 163848 2.00 MB/sec execute 296 sec latency 12.925 ms 2023-09-13T03:20:52.928 INFO:tasks.workunit.client.0.smithi064.stdout: 1 163850 1.99 MB/sec execute 297 sec latency 999.926 ms 2023-09-13T03:20:52.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:20:52 smithi142 ceph-mon[134082]: pgmap v236: 97 pgs: 97 active+clean; 82 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 146 op/s 2023-09-13T03:20:53.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:20:52 smithi170 ceph-mon[134328]: pgmap v236: 97 pgs: 97 active+clean; 82 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 146 op/s 2023-09-13T03:20:53.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:52 smithi064 ceph-mon[97675]: pgmap v236: 97 pgs: 97 active+clean; 82 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 146 op/s 2023-09-13T03:20:53.928 INFO:tasks.workunit.client.0.smithi064.stdout: 1 163906 1.99 MB/sec execute 298 sec latency 1602.133 ms 2023-09-13T03:20:54.928 INFO:tasks.workunit.client.0.smithi064.stdout: 1 163906 1.98 MB/sec execute 299 sec latency 1350.750 ms 2023-09-13T03:20:54.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:20:54 smithi142 ceph-mon[134082]: pgmap v237: 97 pgs: 97 active+clean; 82 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 380 KiB/s wr, 68 op/s 2023-09-13T03:20:55.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:20:54 smithi170 ceph-mon[134328]: pgmap v237: 97 pgs: 97 active+clean; 82 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 380 KiB/s wr, 68 op/s 2023-09-13T03:20:55.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:54 smithi064 ceph-mon[97675]: pgmap v237: 97 pgs: 97 active+clean; 82 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 380 KiB/s wr, 68 op/s 2023-09-13T03:20:55.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.471+0000 7faa3e3ff700 1 -- 172.21.15.64:0/2895315091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa381005b0 msgr2=0x7faa38100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.471+0000 7faa3e3ff700 1 --2- 172.21.15.64:0/2895315091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa381005b0 0x7faa38100a30 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7faa2c0099c0 tx=0x7faa2c009cd0 comp rx=0 tx=0).stop 2023-09-13T03:20:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.471+0000 7faa3e3ff700 1 -- 172.21.15.64:0/2895315091 shutdown_connections 2023-09-13T03:20:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.471+0000 7faa3e3ff700 1 --2- 172.21.15.64:0/2895315091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa38100f70 0x7faa3810d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.471+0000 7faa3e3ff700 1 --2- 172.21.15.64:0/2895315091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa38106670 0x7faa38106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.471+0000 7faa3e3ff700 1 --2- 172.21.15.64:0/2895315091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa381005b0 0x7faa38100a30 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.471+0000 7faa3e3ff700 1 -- 172.21.15.64:0/2895315091 >> 172.21.15.64:0/2895315091 conn(0x7faa380fc060 msgr2=0x7faa380fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:20:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.472+0000 7faa3e3ff700 1 -- 172.21.15.64:0/2895315091 shutdown_connections 2023-09-13T03:20:55.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.472+0000 7faa3e3ff700 1 -- 172.21.15.64:0/2895315091 wait complete. 2023-09-13T03:20:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.473+0000 7faa3e3ff700 1 Processor -- start 2023-09-13T03:20:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.473+0000 7faa3e3ff700 1 -- start start 2023-09-13T03:20:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.473+0000 7faa3e3ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa381005b0 0x7faa3819e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:55.478 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.473+0000 7faa3e3ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa38100f70 0x7faa3819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:55.478 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.473+0000 7faa3e3ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa38106670 0x7faa38198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:55.478 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.473+0000 7faa3e3ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa38198da0 con 0x7faa38100f70 2023-09-13T03:20:55.478 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.473+0000 7faa3e3ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa38198f10 con 0x7faa38106670 2023-09-13T03:20:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.473+0000 7faa3e3ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa381990d0 con 0x7faa381005b0 2023-09-13T03:20:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.473+0000 7faa377fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa38100f70 0x7faa3819ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.474+0000 7faa377fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa38100f70 0x7faa3819ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55826/0 (socket says 172.21.15.64:55826) 2023-09-13T03:20:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.474+0000 7faa377fe700 1 -- 172.21.15.64:0/1460259470 learned_addr learned my addr 172.21.15.64:0/1460259470 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:20:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.474+0000 7faa37fff700 1 --2- 172.21.15.64:0/1460259470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa381005b0 0x7faa3819e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:55.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.474+0000 7faa3c99c700 1 --2- 172.21.15.64:0/1460259470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa38106670 0x7faa38198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:55.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.474+0000 7faa377fe700 1 -- 172.21.15.64:0/1460259470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa381005b0 msgr2=0x7faa3819e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:55.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.474+0000 7faa377fe700 1 --2- 172.21.15.64:0/1460259470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa381005b0 0x7faa3819e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:55.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.474+0000 7faa377fe700 1 -- 172.21.15.64:0/1460259470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa38106670 msgr2=0x7faa38198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:55.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.474+0000 7faa377fe700 1 --2- 172.21.15.64:0/1460259470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa38106670 0x7faa38198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:55.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.474+0000 7faa377fe700 1 -- 172.21.15.64:0/1460259470 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa2c0095f0 con 0x7faa38100f70 2023-09-13T03:20:55.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.474+0000 7faa3c99c700 1 --2- 172.21.15.64:0/1460259470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa38106670 0x7faa38198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:20:55.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.474+0000 7faa37fff700 1 --2- 172.21.15.64:0/1460259470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa381005b0 0x7faa3819e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:20:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.474+0000 7faa377fe700 1 --2- 172.21.15.64:0/1460259470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa38100f70 0x7faa3819ee10 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7faa2c00b580 tx=0x7faa2c00bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:20:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.474+0000 7faa357fa700 1 -- 172.21.15.64:0/1460259470 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c01d070 con 0x7faa38100f70 2023-09-13T03:20:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.474+0000 7faa357fa700 1 -- 172.21.15.64:0/1460259470 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa2c005050 con 0x7faa38100f70 2023-09-13T03:20:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.475+0000 7faa3e3ff700 1 -- 172.21.15.64:0/1460259470 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa38199350 con 0x7faa38100f70 2023-09-13T03:20:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.475+0000 7faa357fa700 1 -- 172.21.15.64:0/1460259470 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c004a40 con 0x7faa38100f70 2023-09-13T03:20:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.475+0000 7faa3e3ff700 1 -- 172.21.15.64:0/1460259470 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa381a58c0 con 0x7faa38100f70 2023-09-13T03:20:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.478+0000 7faa357fa700 1 -- 172.21.15.64:0/1460259470 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7faa2c0051c0 con 0x7faa38100f70 2023-09-13T03:20:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.479+0000 7faa357fa700 1 --2- 172.21.15.64:0/1460259470 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7faa200a0830 0x7faa200a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.479+0000 7faa51d4ab80 1 Processor -- start 2023-09-13T03:20:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.479+0000 7faa357fa700 1 -- 172.21.15.64:0/1460259470 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8484+0+0 (secure 0 0 0) 0x7faa2c0ccde0 con 0x7faa38100f70 2023-09-13T03:20:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.479+0000 7faa51d4ab80 1 -- start start 2023-09-13T03:20:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.479+0000 7faa37fff700 1 --2- 172.21.15.64:0/1460259470 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7faa200a0830 0x7faa200a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.480+0000 7faa51d4ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564928a181a0 0x564928a18580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.480+0000 7faa51d4ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564928a12300 0x564928a12780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.480+0000 7faa51d4ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564928a12cc0 0x564928a1ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.480+0000 7faa51d4ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56492896b6d0 con 0x564928a12cc0 2023-09-13T03:20:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.480+0000 7faa51d4ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564928968ff0 con 0x564928a181a0 2023-09-13T03:20:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.480+0000 7faa51d4ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649289648e0 con 0x564928a12300 2023-09-13T03:20:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.480+0000 7faa377fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564928a12300 0x564928a12780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.480+0000 7faa37fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564928a181a0 0x564928a18580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.480+0000 7faa3c99c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564928a12cc0 0x564928a1ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.480+0000 7faa377fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564928a12300 0x564928a12780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:52432/0 (socket says 172.21.15.64:52432) 2023-09-13T03:20:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.480+0000 7faa377fe700 1 -- 172.21.15.64:0/2481306819 learned_addr learned my addr 172.21.15.64:0/2481306819 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:20:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.480+0000 7faa37fff700 1 --2- 172.21.15.64:0/1460259470 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7faa200a0830 0x7faa200a2cf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7faa28009fd0 tx=0x7faa28009260 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:20:55.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.481+0000 7faa377fe700 1 -- 172.21.15.64:0/2481306819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564928a181a0 msgr2=0x564928a18580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:20:55.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.481+0000 7faa377fe700 1 --2- 172.21.15.64:0/2481306819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564928a181a0 0x564928a18580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:55.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.481+0000 7faa377fe700 1 -- 172.21.15.64:0/2481306819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564928a12cc0 msgr2=0x564928a1ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:20:55.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.481+0000 7faa377fe700 1 --2- 172.21.15.64:0/2481306819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564928a12cc0 0x564928a1ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:55.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.481+0000 7faa377fe700 1 -- 172.21.15.64:0/2481306819 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564928790d80 con 0x564928a12300 2023-09-13T03:20:55.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.481+0000 7faa377fe700 1 --2- 172.21.15.64:0/2481306819 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564928a12300 0x564928a12780 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7faa2c096d50 tx=0x7faa2c09aad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf32989a28b9fd79 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:20:55.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.481+0000 7faa1e7fc700 1 -- 172.21.15.64:0/2481306819 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c0a22c0 con 0x564928a12300 2023-09-13T03:20:55.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.481+0000 7faa1e7fc700 1 -- 172.21.15.64:0/2481306819 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa2c0a00c0 con 0x564928a12300 2023-09-13T03:20:55.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.481+0000 7faa1e7fc700 1 -- 172.21.15.64:0/2481306819 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c09a540 con 0x564928a12300 2023-09-13T03:20:55.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.482+0000 7faa51d4ab80 1 -- 172.21.15.64:0/2481306819 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564928a12300 msgr2=0x564928a12780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:55.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.482+0000 7faa51d4ab80 1 --2- 172.21.15.64:0/2481306819 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564928a12300 0x564928a12780 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7faa2c096d50 tx=0x7faa2c09aad0 comp rx=0 tx=0).stop 2023-09-13T03:20:55.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.482+0000 7faa51d4ab80 1 -- 172.21.15.64:0/2481306819 shutdown_connections 2023-09-13T03:20:55.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.482+0000 7faa51d4ab80 1 --2- 172.21.15.64:0/2481306819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564928a181a0 0x564928a18580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:55.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.482+0000 7faa51d4ab80 1 --2- 172.21.15.64:0/2481306819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564928a12cc0 0x564928a1ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.482+0000 7faa51d4ab80 1 --2- 172.21.15.64:0/2481306819 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564928a12300 0x564928a12780 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.482+0000 7faa51d4ab80 1 -- 172.21.15.64:0/2481306819 >> 172.21.15.64:0/2481306819 conn(0x564928987030 msgr2=0x564928a0d560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:20:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.482+0000 7faa51d4ab80 1 -- 172.21.15.64:0/2481306819 shutdown_connections 2023-09-13T03:20:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.482+0000 7faa51d4ab80 1 -- 172.21.15.64:0/2481306819 wait complete. 2023-09-13T03:20:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.483+0000 7faa51d4ab80 1 -- start start 2023-09-13T03:20:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.483+0000 7faa51d4ab80 1 Processor -- start 2023-09-13T03:20:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.483+0000 7faa51d4ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564928a12300 0x564928a21c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.483+0000 7faa51d4ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564928a12cc0 0x564928a22150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.483+0000 7faa51d4ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564928a181a0 0x564928a1bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.483+0000 7faa51d4ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564928902b10 con 0x564928a181a0 2023-09-13T03:20:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.483+0000 7faa51d4ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564928902810 con 0x564928a12cc0 2023-09-13T03:20:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.483+0000 7faa51d4ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649287e68c0 con 0x564928a12300 2023-09-13T03:20:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.483+0000 7faa51d4ab80 10 client.0 _ll_register_callbacks cb 0x56492884ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:20:55.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.484+0000 7faa3c99c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564928a181a0 0x564928a1bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:55.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.484+0000 7faa37fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564928a12300 0x564928a21c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:55.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.484+0000 7faa3c99c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564928a181a0 0x564928a1bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55850/0 (socket says 172.21.15.64:55850) 2023-09-13T03:20:55.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.484+0000 7faa3c99c700 1 -- 172.21.15.64:0/2460996375 learned_addr learned my addr 172.21.15.64:0/2460996375 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:20:55.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.484+0000 7faa377fe700 1 --2- 172.21.15.64:0/2460996375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564928a12cc0 0x564928a22150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:55.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.484+0000 7faa3c99c700 1 -- 172.21.15.64:0/2460996375 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564928a12300 msgr2=0x564928a21c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:20:55.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.484+0000 7faa3c99c700 1 --2- 172.21.15.64:0/2460996375 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564928a12300 0x564928a21c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:55.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.484+0000 7faa3c99c700 1 -- 172.21.15.64:0/2460996375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564928a12cc0 msgr2=0x564928a22150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:20:55.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.484+0000 7faa3c99c700 1 --2- 172.21.15.64:0/2460996375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564928a12cc0 0x564928a22150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:55.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.484+0000 7faa3c99c700 1 -- 172.21.15.64:0/2460996375 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa381a58c0 con 0x564928a181a0 2023-09-13T03:20:55.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.484+0000 7faa37fff700 1 --2- 172.21.15.64:0/2460996375 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564928a12300 0x564928a21c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:20:55.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.484+0000 7faa3c99c700 1 --2- 172.21.15.64:0/2460996375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564928a181a0 0x564928a1bf10 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7faa30007ab0 tx=0x7faa30007ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ada30e42a170e605 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:20:55.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.484+0000 7faa177fe700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:20:55.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.485+0000 7faa177fe700 1 -- 172.21.15.64:0/2460996375 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3000e030 con 0x564928a181a0 2023-09-13T03:20:55.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.485+0000 7faa51d4ab80 1 -- 172.21.15.64:0/2460996375 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5649288bc490 con 0x564928a181a0 2023-09-13T03:20:55.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.485+0000 7faa177fe700 1 -- 172.21.15.64:0/2460996375 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8484+0+0 (secure 0 0 0) 0x7faa30015860 con 0x564928a181a0 2023-09-13T03:20:55.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.485+0000 7faa177fe700 1 -- 172.21.15.64:0/2460996375 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa30015a20 con 0x564928a181a0 2023-09-13T03:20:55.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.485+0000 7faa51d4ab80 10 client.16713 fetch_fsmap learned FSMap version 51 2023-09-13T03:20:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.485+0000 7faa51d4ab80 1 -- 172.21.15.64:0/2460996375 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x564928779c00 con 0x564928a181a0 2023-09-13T03:20:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.485+0000 7faa177fe700 1 -- 172.21.15.64:0/2460996375 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5649288bc490 con 0x564928a181a0 2023-09-13T03:20:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.485+0000 7faa177fe700 1 -- 172.21.15.64:0/2460996375 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 51) v1 ==== 7910+0+0 (secure 0 0 0) 0x7faa3001e3e0 con 0x564928a181a0 2023-09-13T03:20:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.486+0000 7faa51d4ab80 10 client.16713 fetch_fsmap finished waiting for FSMap version 51 2023-09-13T03:20:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.486+0000 7faa51d4ab80 10 client.16713 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:20:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.486+0000 7faa51d4ab80 20 client.16713 populate_metadata read hostname 'smithi064' 2023-09-13T03:20:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.486+0000 7faa51d4ab80 1 --2- 172.21.15.64:0/2460996375 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x564928a2c1e0 0x564928a2e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.486+0000 7faa51d4ab80 4 client.16713 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:20:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.486+0000 7faa51d4ab80 1 -- 172.21.15.64:0/2460996375 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564928979840 con 0x564928a2c1e0 2023-09-13T03:20:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.486+0000 7faa37fff700 1 --2- 172.21.15.64:0/2460996375 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x564928a2c1e0 0x564928a2e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:55.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.487+0000 7faa37fff700 1 --2- 172.21.15.64:0/2460996375 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x564928a2c1e0 0x564928a2e600 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a3dcf1f56db6a96 server_cookie=a21537efb5c46d2d in_seq=0 out_seq=0 2023-09-13T03:20:55.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.487+0000 7faa177fe700 10 client.16713 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:20:55.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.487+0000 7faa177fe700 1 -- 172.21.15.64:0/2460996375 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564928979840 con 0x564928a2c1e0 2023-09-13T03:20:55.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.487+0000 7faa177fe700 10 client.16713 handle_command_reply: tid=0 2023-09-13T03:20:55.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.487+0000 7faa51d4ab80 1 client.16713 shutdown 2023-09-13T03:20:55.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.487+0000 7faa51d4ab80 1 -- 172.21.15.64:0/2460996375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564928a181a0 msgr2=0x564928a1bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:55.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.487+0000 7faa51d4ab80 1 --2- 172.21.15.64:0/2460996375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564928a181a0 0x564928a1bf10 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7faa30007ab0 tx=0x7faa30007ae0 comp rx=0 tx=0).stop 2023-09-13T03:20:55.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.488+0000 7faa51d4ab80 1 -- 172.21.15.64:0/2460996375 shutdown_connections 2023-09-13T03:20:55.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.488+0000 7faa51d4ab80 1 --2- 172.21.15.64:0/2460996375 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x564928a2c1e0 0x564928a2e600 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:55.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.488+0000 7faa51d4ab80 1 --2- 172.21.15.64:0/2460996375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564928a181a0 0x564928a1bf10 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:55.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.488+0000 7faa177fe700 0 client.16713 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:20:55.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.488+0000 7faa51d4ab80 1 --2- 172.21.15.64:0/2460996375 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564928a12300 0x564928a21c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:55.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.488+0000 7faa51d4ab80 1 --2- 172.21.15.64:0/2460996375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564928a12cc0 0x564928a22150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:55.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.488+0000 7faa51d4ab80 1 -- 172.21.15.64:0/2460996375 >> 172.21.15.64:0/2460996375 conn(0x564928987030 msgr2=0x564928a0c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:20:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.488+0000 7faa51d4ab80 1 -- 172.21.15.64:0/2460996375 shutdown_connections 2023-09-13T03:20:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.488+0000 7faa51d4ab80 1 -- 172.21.15.64:0/2460996375 wait complete. 2023-09-13T03:20:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.488+0000 7faa51d4ab80 20 client.16713 trim_cache size 0 max 16384 2023-09-13T03:20:55.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.503+0000 7faa51d4ab80 1 Processor -- start 2023-09-13T03:20:55.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.503+0000 7faa51d4ab80 1 -- start start 2023-09-13T03:20:55.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.504+0000 7faa51d4ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564928a12300 0x564928a11a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:55.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.504+0000 7faa51d4ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564928a12cc0 0x564928a217e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:55.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.504+0000 7faa51d4ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564928a181a0 0x564928a21d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:55.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.504+0000 7faa51d4ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564928a130a0 con 0x564928a181a0 2023-09-13T03:20:55.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.504+0000 7faa51d4ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564928a0e1c0 con 0x564928a12300 2023-09-13T03:20:55.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.504+0000 7faa51d4ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564928a04e00 con 0x564928a12cc0 2023-09-13T03:20:55.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.504+0000 7faa3c99c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564928a181a0 0x564928a21d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:55.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.504+0000 7faa3c99c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564928a181a0 0x564928a21d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55864/0 (socket says 172.21.15.64:55864) 2023-09-13T03:20:55.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.504+0000 7faa3c99c700 1 -- 172.21.15.64:0/1258141634 learned_addr learned my addr 172.21.15.64:0/1258141634 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:20:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.504+0000 7faa37fff700 1 --2- 172.21.15.64:0/1258141634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564928a12300 0x564928a11a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.504+0000 7faa377fe700 1 --2- 172.21.15.64:0/1258141634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564928a12cc0 0x564928a217e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.504+0000 7faa37fff700 1 -- 172.21.15.64:0/1258141634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564928a12cc0 msgr2=0x564928a217e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:20:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.504+0000 7faa37fff700 1 --2- 172.21.15.64:0/1258141634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564928a12cc0 0x564928a217e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:55.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.504+0000 7faa37fff700 1 -- 172.21.15.64:0/1258141634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564928a181a0 msgr2=0x564928a21d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:20:55.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.504+0000 7faa37fff700 1 --2- 172.21.15.64:0/1258141634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564928a181a0 0x564928a21d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:55.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.505+0000 7faa37fff700 1 -- 172.21.15.64:0/1258141634 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564928a331f0 con 0x564928a12300 2023-09-13T03:20:55.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.505+0000 7faa377fe700 1 --2- 172.21.15.64:0/1258141634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564928a12cc0 0x564928a217e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:20:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.505+0000 7faa3c99c700 1 --2- 172.21.15.64:0/1258141634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564928a181a0 0x564928a21d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:20:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.505+0000 7faa37fff700 1 --2- 172.21.15.64:0/1258141634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564928a12300 0x564928a11a00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x564928a2ee70 tx=0x7faa28019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86aad963daa316f0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:20:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.505+0000 7faa1effd700 1 -- 172.21.15.64:0/1258141634 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa28021070 con 0x564928a12300 2023-09-13T03:20:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.505+0000 7faa1effd700 1 -- 172.21.15.64:0/1258141634 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa28022040 con 0x564928a12300 2023-09-13T03:20:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.505+0000 7faa1effd700 1 -- 172.21.15.64:0/1258141634 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2801c890 con 0x564928a12300 2023-09-13T03:20:55.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.506+0000 7faa51d4ab80 1 -- 172.21.15.64:0/1258141634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564928a12300 msgr2=0x564928a11a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:55.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.506+0000 7faa51d4ab80 1 --2- 172.21.15.64:0/1258141634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564928a12300 0x564928a11a00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x564928a2ee70 tx=0x7faa28019000 comp rx=0 tx=0).stop 2023-09-13T03:20:55.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.506+0000 7faa51d4ab80 1 -- 172.21.15.64:0/1258141634 shutdown_connections 2023-09-13T03:20:55.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.506+0000 7faa51d4ab80 1 --2- 172.21.15.64:0/1258141634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564928a12300 0x564928a11a00 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:55.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.506+0000 7faa51d4ab80 1 --2- 172.21.15.64:0/1258141634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564928a181a0 0x564928a21d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:55.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.506+0000 7faa51d4ab80 1 --2- 172.21.15.64:0/1258141634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564928a12cc0 0x564928a217e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:55.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.506+0000 7faa51d4ab80 1 -- 172.21.15.64:0/1258141634 >> 172.21.15.64:0/1258141634 conn(0x564928987030 msgr2=0x564928a112f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:20:55.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.506+0000 7faa51d4ab80 1 -- 172.21.15.64:0/1258141634 shutdown_connections 2023-09-13T03:20:55.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.506+0000 7faa51d4ab80 1 -- 172.21.15.64:0/1258141634 wait complete. 2023-09-13T03:20:55.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.506+0000 7faa51d4ab80 1 -- start start 2023-09-13T03:20:55.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.507+0000 7faa51d4ab80 1 Processor -- start 2023-09-13T03:20:55.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.507+0000 7faa51d4ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564928a12300 0x564928a78f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:55.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.507+0000 7faa51d4ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564928a12cc0 0x564928a79450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:55.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.507+0000 7faa51d4ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564928a181a0 0x564928a73300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:55.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.507+0000 7faa51d4ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564928a738e0 con 0x564928a12300 2023-09-13T03:20:55.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.507+0000 7faa51d4ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564928a73a50 con 0x564928a181a0 2023-09-13T03:20:55.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.507+0000 7faa51d4ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564928a73c10 con 0x564928a12cc0 2023-09-13T03:20:55.523 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.507+0000 7faa51d4ab80 10 client.0 _ll_register_callbacks cb 0x7faa30009470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:20:55.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.507+0000 7faa37fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564928a12300 0x564928a78f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:55.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.507+0000 7faa37fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564928a12300 0x564928a78f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55872/0 (socket says 172.21.15.64:55872) 2023-09-13T03:20:55.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.507+0000 7faa3c99c700 1 --2- 172.21.15.64:0/2511487185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564928a181a0 0x564928a73300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:55.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.507+0000 7faa377fe700 1 --2- 172.21.15.64:0/2511487185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564928a12cc0 0x564928a79450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:55.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.507+0000 7faa37fff700 1 -- 172.21.15.64:0/2511487185 learned_addr learned my addr 172.21.15.64:0/2511487185 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:20:55.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.508+0000 7faa37fff700 1 -- 172.21.15.64:0/2511487185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564928a12cc0 msgr2=0x564928a79450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:20:55.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.508+0000 7faa37fff700 1 --2- 172.21.15.64:0/2511487185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564928a12cc0 0x564928a79450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:55.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.508+0000 7faa37fff700 1 -- 172.21.15.64:0/2511487185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564928a181a0 msgr2=0x564928a73300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:20:55.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.508+0000 7faa37fff700 1 --2- 172.21.15.64:0/2511487185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564928a181a0 0x564928a73300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:55.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.508+0000 7faa37fff700 1 -- 172.21.15.64:0/2511487185 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564928790d80 con 0x564928a12300 2023-09-13T03:20:55.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.508+0000 7faa377fe700 1 --2- 172.21.15.64:0/2511487185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564928a12cc0 0x564928a79450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:20:55.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.508+0000 7faa3c99c700 1 --2- 172.21.15.64:0/2511487185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564928a181a0 0x564928a73300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:20:55.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.508+0000 7faa37fff700 1 --2- 172.21.15.64:0/2511487185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564928a12300 0x564928a78f10 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7faa280058d0 tx=0x7faa28003d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db7156da6e49077 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:20:55.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.508+0000 7faa177fe700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:20:55.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.508+0000 7faa177fe700 1 -- 172.21.15.64:0/2511487185 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa28021050 con 0x564928a12300 2023-09-13T03:20:55.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.509+0000 7faa51d4ab80 1 -- 172.21.15.64:0/2511487185 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faa2c0a00c0 con 0x564928a12300 2023-09-13T03:20:55.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.509+0000 7faa177fe700 1 -- 172.21.15.64:0/2511487185 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8484+0+0 (secure 0 0 0) 0x7faa2802a2c0 con 0x564928a12300 2023-09-13T03:20:55.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.509+0000 7faa177fe700 1 -- 172.21.15.64:0/2511487185 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa2802a480 con 0x564928a12300 2023-09-13T03:20:55.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.509+0000 7faa51d4ab80 10 client.16719 fetch_fsmap learned FSMap version 51 2023-09-13T03:20:55.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.509+0000 7faa51d4ab80 1 -- 172.21.15.64:0/2511487185 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x564928779c00 con 0x564928a12300 2023-09-13T03:20:55.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.509+0000 7faa177fe700 1 -- 172.21.15.64:0/2511487185 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa2c0a00c0 con 0x564928a12300 2023-09-13T03:20:55.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.509+0000 7faa177fe700 1 -- 172.21.15.64:0/2511487185 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 51) v1 ==== 7910+0+0 (secure 0 0 0) 0x7faa2801c600 con 0x564928a12300 2023-09-13T03:20:55.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.509+0000 7faa51d4ab80 10 client.16719 fetch_fsmap finished waiting for FSMap version 51 2023-09-13T03:20:55.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.509+0000 7faa51d4ab80 10 client.16719 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:20:55.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.509+0000 7faa51d4ab80 20 client.16719 populate_metadata read hostname 'smithi064' 2023-09-13T03:20:55.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.509+0000 7faa51d4ab80 1 --2- 172.21.15.64:0/2511487185 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x564928a80e20 0x564928a83240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:20:55.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.509+0000 7faa51d4ab80 4 client.16719 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:20:55.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.509+0000 7faa51d4ab80 1 -- 172.21.15.64:0/2511487185 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56492895d8c0 con 0x564928a80e20 2023-09-13T03:20:55.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.510+0000 7faa377fe700 1 --2- 172.21.15.64:0/2511487185 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x564928a80e20 0x564928a83240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:20:55.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.510+0000 7faa377fe700 1 --2- 172.21.15.64:0/2511487185 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x564928a80e20 0x564928a83240 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76b483512f626c11 server_cookie=b3bb3459bfe9f591 in_seq=0 out_seq=0 2023-09-13T03:20:55.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.510+0000 7faa177fe700 10 client.16719 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:20:55.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.541+0000 7faa177fe700 1 -- 172.21.15.64:0/2511487185 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56492895d8c0 con 0x564928a80e20 2023-09-13T03:20:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.541+0000 7faa177fe700 10 client.16719 handle_command_reply: tid=0 2023-09-13T03:20:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.541+0000 7faa51d4ab80 1 client.16719 shutdown 2023-09-13T03:20:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.541+0000 7faa51d4ab80 1 -- 172.21.15.64:0/2511487185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564928a12300 msgr2=0x564928a78f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.541+0000 7faa51d4ab80 1 --2- 172.21.15.64:0/2511487185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564928a12300 0x564928a78f10 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7faa280058d0 tx=0x7faa28003d10 comp rx=0 tx=0).stop 2023-09-13T03:20:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.541+0000 7faa51d4ab80 1 -- 172.21.15.64:0/2511487185 shutdown_connections 2023-09-13T03:20:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.541+0000 7faa51d4ab80 1 --2- 172.21.15.64:0/2511487185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564928a181a0 0x564928a73300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.541+0000 7faa51d4ab80 1 --2- 172.21.15.64:0/2511487185 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x564928a80e20 0x564928a83240 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.542+0000 7faa51d4ab80 1 --2- 172.21.15.64:0/2511487185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564928a12300 0x564928a78f10 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.542+0000 7faa51d4ab80 1 --2- 172.21.15.64:0/2511487185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564928a12cc0 0x564928a79450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.542+0000 7faa177fe700 0 client.16719 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:20:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.542+0000 7faa51d4ab80 1 -- 172.21.15.64:0/2511487185 >> 172.21.15.64:0/2511487185 conn(0x564928987030 msgr2=0x564928a11b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:20:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.542+0000 7faa51d4ab80 1 -- 172.21.15.64:0/2511487185 shutdown_connections 2023-09-13T03:20:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.542+0000 7faa51d4ab80 1 -- 172.21.15.64:0/2511487185 wait complete. 2023-09-13T03:20:55.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.542+0000 7faa51d4ab80 20 client.16719 trim_cache size 0 max 16384 2023-09-13T03:20:55.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:20:55.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:20:55.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:20:55.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:20:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.543+0000 7faa3e3ff700 1 -- 172.21.15.64:0/1460259470 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7faa200a0830 msgr2=0x7faa200a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.543+0000 7faa3e3ff700 1 --2- 172.21.15.64:0/1460259470 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7faa200a0830 0x7faa200a2cf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7faa28009fd0 tx=0x7faa28009260 comp rx=0 tx=0).stop 2023-09-13T03:20:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.543+0000 7faa3e3ff700 1 -- 172.21.15.64:0/1460259470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa38100f70 msgr2=0x7faa3819ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:20:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.543+0000 7faa3e3ff700 1 --2- 172.21.15.64:0/1460259470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa38100f70 0x7faa3819ee10 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7faa2c00b580 tx=0x7faa2c00bfd0 comp rx=0 tx=0).stop 2023-09-13T03:20:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.543+0000 7faa3e3ff700 1 -- 172.21.15.64:0/1460259470 shutdown_connections 2023-09-13T03:20:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.543+0000 7faa3e3ff700 1 --2- 172.21.15.64:0/1460259470 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7faa200a0830 0x7faa200a2cf0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.543+0000 7faa3e3ff700 1 --2- 172.21.15.64:0/1460259470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa38106670 0x7faa38198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.543+0000 7faa3e3ff700 1 --2- 172.21.15.64:0/1460259470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa381005b0 0x7faa3819e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.543+0000 7faa3e3ff700 1 --2- 172.21.15.64:0/1460259470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa38100f70 0x7faa3819ee10 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:20:55.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.543+0000 7faa3e3ff700 1 -- 172.21.15.64:0/1460259470 >> 172.21.15.64:0/1460259470 conn(0x7faa380fc060 msgr2=0x7faa380fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:20:55.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.543+0000 7faa3e3ff700 1 -- 172.21.15.64:0/1460259470 shutdown_connections 2023-09-13T03:20:55.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:20:55.543+0000 7faa3e3ff700 1 -- 172.21.15.64:0/1460259470 wait complete. 2023-09-13T03:20:55.928 INFO:tasks.workunit.client.0.smithi064.stdout: 1 163906 1.98 MB/sec execute 300 sec latency 2350.858 ms 2023-09-13T03:20:56.929 INFO:tasks.workunit.client.0.smithi064.stdout: 1 163906 1.97 MB/sec execute 301 sec latency 3350.988 ms 2023-09-13T03:20:56.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:20:56 smithi142 ceph-mon[134082]: pgmap v238: 97 pgs: 97 active+clean; 84 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 590 KiB/s wr, 102 op/s 2023-09-13T03:20:57.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:20:56 smithi170 ceph-mon[134328]: pgmap v238: 97 pgs: 97 active+clean; 84 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 590 KiB/s wr, 102 op/s 2023-09-13T03:20:57.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:56 smithi064 ceph-mon[97675]: pgmap v238: 97 pgs: 97 active+clean; 84 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 590 KiB/s wr, 102 op/s 2023-09-13T03:20:57.929 INFO:tasks.workunit.client.0.smithi064.stdout: 1 163906 1.96 MB/sec execute 302 sec latency 4351.153 ms 2023-09-13T03:20:58.929 INFO:tasks.workunit.client.0.smithi064.stdout: 1 164593 1.97 MB/sec execute 303 sec latency 4950.337 ms 2023-09-13T03:20:58.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:20:58 smithi142 ceph-mon[134082]: pgmap v239: 97 pgs: 97 active+clean; 84 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 354 KiB/s wr, 60 op/s 2023-09-13T03:20:59.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:20:58 smithi170 ceph-mon[134328]: pgmap v239: 97 pgs: 97 active+clean; 84 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 354 KiB/s wr, 60 op/s 2023-09-13T03:20:59.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:20:58 smithi064 ceph-mon[97675]: pgmap v239: 97 pgs: 97 active+clean; 84 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 354 KiB/s wr, 60 op/s 2023-09-13T03:20:59.929 INFO:tasks.workunit.client.0.smithi064.stdout: 1 164875 1.97 MB/sec execute 304 sec latency 774.240 ms 2023-09-13T03:21:00.929 INFO:tasks.workunit.client.0.smithi064.stdout: 1 166101 1.98 MB/sec execute 305 sec latency 1197.014 ms 2023-09-13T03:21:00.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:21:00 smithi142 ceph-mon[134082]: pgmap v240: 97 pgs: 97 active+clean; 84 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 354 KiB/s wr, 60 op/s 2023-09-13T03:21:00.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:21:00 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:21:01.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:21:00 smithi170 ceph-mon[134328]: pgmap v240: 97 pgs: 97 active+clean; 84 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 354 KiB/s wr, 60 op/s 2023-09-13T03:21:01.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:21:00 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:21:01.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:21:00 smithi064 ceph-mon[97675]: pgmap v240: 97 pgs: 97 active+clean; 84 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 354 KiB/s wr, 60 op/s 2023-09-13T03:21:01.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:21:00 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:21:01.929 INFO:tasks.workunit.client.0.smithi064.stdout: 1 167115 2.00 MB/sec execute 306 sec latency 25.003 ms 2023-09-13T03:21:02.929 INFO:tasks.workunit.client.0.smithi064.stdout: 1 167399 1.99 MB/sec execute 307 sec latency 597.934 ms 2023-09-13T03:21:02.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:21:02 smithi142 ceph-mon[134082]: pgmap v241: 97 pgs: 97 active+clean; 86 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 497 KiB/s wr, 84 op/s 2023-09-13T03:21:03.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:21:02 smithi170 ceph-mon[134328]: pgmap v241: 97 pgs: 97 active+clean; 86 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 497 KiB/s wr, 84 op/s 2023-09-13T03:21:03.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:21:02 smithi064 ceph-mon[97675]: pgmap v241: 97 pgs: 97 active+clean; 86 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 497 KiB/s wr, 84 op/s 2023-09-13T03:21:03.929 INFO:tasks.workunit.client.0.smithi064.stdout: 1 167954 1.99 MB/sec execute 308 sec latency 1199.808 ms 2023-09-13T03:21:04.929 INFO:tasks.workunit.client.0.smithi064.stdout: 1 168201 2.00 MB/sec execute 309 sec latency 867.892 ms 2023-09-13T03:21:04.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:21:04 smithi142 ceph-mon[134082]: pgmap v242: 97 pgs: 97 active+clean; 86 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 353 KiB/s wr, 58 op/s 2023-09-13T03:21:05.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:21:04 smithi170 ceph-mon[134328]: pgmap v242: 97 pgs: 97 active+clean; 86 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 353 KiB/s wr, 58 op/s 2023-09-13T03:21:05.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:21:04 smithi064 ceph-mon[97675]: pgmap v242: 97 pgs: 97 active+clean; 86 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 353 KiB/s wr, 58 op/s 2023-09-13T03:21:05.929 INFO:tasks.workunit.client.0.smithi064.stdout: 1 168201 1.99 MB/sec execute 310 sec latency 1867.969 ms 2023-09-13T03:21:05.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:21:05 smithi142 ceph-mon[134082]: osdmap e89: 12 total, 12 up, 12 in 2023-09-13T03:21:06.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:21:05 smithi170 ceph-mon[134328]: osdmap e89: 12 total, 12 up, 12 in 2023-09-13T03:21:06.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:21:05 smithi064 ceph-mon[97675]: osdmap e89: 12 total, 12 up, 12 in 2023-09-13T03:21:06.930 INFO:tasks.workunit.client.0.smithi064.stdout: 1 168201 1.98 MB/sec execute 311 sec latency 2868.081 ms 2023-09-13T03:21:06.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:21:06 smithi142 ceph-mon[134082]: pgmap v244: 97 pgs: 97 active+clean; 88 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 680 KiB/s wr, 117 op/s 2023-09-13T03:21:07.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:21:06 smithi170 ceph-mon[134328]: pgmap v244: 97 pgs: 97 active+clean; 88 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 680 KiB/s wr, 117 op/s 2023-09-13T03:21:07.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:21:06 smithi064 ceph-mon[97675]: pgmap v244: 97 pgs: 97 active+clean; 88 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 680 KiB/s wr, 117 op/s 2023-09-13T03:21:07.930 INFO:tasks.workunit.client.0.smithi064.stdout: 1 168201 1.98 MB/sec execute 312 sec latency 3868.218 ms 2023-09-13T03:21:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.704+0000 7f46541ff700 1 -- 172.21.15.64:0/2529216432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f464c106670 msgr2=0x7f464c106a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:21:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.704+0000 7f46541ff700 1 --2- 172.21.15.64:0/2529216432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f464c106670 0x7f464c106a50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f46480099c0 tx=0x7f4648009cd0 comp rx=0 tx=0).stop 2023-09-13T03:21:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.704+0000 7f46541ff700 1 -- 172.21.15.64:0/2529216432 shutdown_connections 2023-09-13T03:21:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.704+0000 7f46541ff700 1 --2- 172.21.15.64:0/2529216432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f464c106670 0x7f464c106a50 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.704+0000 7f46541ff700 1 --2- 172.21.15.64:0/2529216432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f464c100f70 0x7f464c10d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.704+0000 7f46541ff700 1 --2- 172.21.15.64:0/2529216432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f464c1005b0 0x7f464c100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.704+0000 7f46541ff700 1 -- 172.21.15.64:0/2529216432 >> 172.21.15.64:0/2529216432 conn(0x7f464c0fc060 msgr2=0x7f464c0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:21:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.704+0000 7f46541ff700 1 -- 172.21.15.64:0/2529216432 shutdown_connections 2023-09-13T03:21:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.704+0000 7f46541ff700 1 -- 172.21.15.64:0/2529216432 wait complete. 2023-09-13T03:21:08.708 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.705+0000 7f46541ff700 1 Processor -- start 2023-09-13T03:21:08.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.705+0000 7f46541ff700 1 -- start start 2023-09-13T03:21:08.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.705+0000 7f46541ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f464c1005b0 0x7f464c19c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:08.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.705+0000 7f46541ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f464c100f70 0x7f464c19cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:08.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.705+0000 7f46541ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f464c106670 0x7f464c196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:08.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.706+0000 7f46541ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464c196aa0 con 0x7f464c1005b0 2023-09-13T03:21:08.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.706+0000 7f46541ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464c196c10 con 0x7f464c106670 2023-09-13T03:21:08.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.706+0000 7f46541ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464c196dd0 con 0x7f464c100f70 2023-09-13T03:21:08.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.706+0000 7f4651f9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f464c1005b0 0x7f464c19c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:08.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.706+0000 7f4651f9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f464c1005b0 0x7f464c19c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34622/0 (socket says 172.21.15.64:34622) 2023-09-13T03:21:08.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.706+0000 7f4651f9b700 1 -- 172.21.15.64:0/1531360058 learned_addr learned my addr 172.21.15.64:0/1531360058 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:21:08.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.706+0000 7f465179a700 1 --2- 172.21.15.64:0/1531360058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f464c100f70 0x7f464c19cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:08.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.706+0000 7f465279c700 1 --2- 172.21.15.64:0/1531360058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f464c106670 0x7f464c196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:08.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.706+0000 7f4651f9b700 1 -- 172.21.15.64:0/1531360058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f464c100f70 msgr2=0x7f464c19cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:21:08.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.706+0000 7f4651f9b700 1 --2- 172.21.15.64:0/1531360058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f464c100f70 0x7f464c19cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:08.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.706+0000 7f4651f9b700 1 -- 172.21.15.64:0/1531360058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f464c106670 msgr2=0x7f464c196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:21:08.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.706+0000 7f4651f9b700 1 --2- 172.21.15.64:0/1531360058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f464c106670 0x7f464c196430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:08.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.706+0000 7f4651f9b700 1 -- 172.21.15.64:0/1531360058 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4644009c40 con 0x7f464c1005b0 2023-09-13T03:21:08.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.707+0000 7f4651f9b700 1 --2- 172.21.15.64:0/1531360058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f464c1005b0 0x7f464c19c5d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4648000c00 tx=0x7f464800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:21:08.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.707+0000 7f4642ffd700 1 -- 172.21.15.64:0/1531360058 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464801d070 con 0x7f464c1005b0 2023-09-13T03:21:08.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.707+0000 7f46541ff700 1 -- 172.21.15.64:0/1531360058 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46480095f0 con 0x7f464c1005b0 2023-09-13T03:21:08.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.707+0000 7f4642ffd700 1 -- 172.21.15.64:0/1531360058 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4648005050 con 0x7f464c1005b0 2023-09-13T03:21:08.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.707+0000 7f4642ffd700 1 -- 172.21.15.64:0/1531360058 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4648004a90 con 0x7f464c1005b0 2023-09-13T03:21:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.707+0000 7f46541ff700 1 -- 172.21.15.64:0/1531360058 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f464c1a33e0 con 0x7f464c1005b0 2023-09-13T03:21:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.710+0000 7f4642ffd700 1 -- 172.21.15.64:0/1531360058 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f46480051c0 con 0x7f464c1005b0 2023-09-13T03:21:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.711+0000 7f4667ab5b80 1 Processor -- start 2023-09-13T03:21:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.712+0000 7f4667ab5b80 1 -- start start 2023-09-13T03:21:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.712+0000 7f4642ffd700 1 --2- 172.21.15.64:0/1531360058 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f46380a07b0 0x7f46380a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.712+0000 7f4642ffd700 1 -- 172.21.15.64:0/1531360058 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f46480cccf0 con 0x7f464c1005b0 2023-09-13T03:21:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.712+0000 7f465279c700 1 --2- 172.21.15.64:0/1531360058 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f46380a07b0 0x7f46380a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.712+0000 7f4667ab5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c8aea270 0x55c6c8ae6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.712+0000 7f4667ab5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6c8ae6840 0x55c6c8ae6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.712+0000 7f4667ab5b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6c8af8f60 0x55c6c8afb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.712+0000 7f4667ab5b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c8a3f6d0 con 0x55c6c8af8f60 2023-09-13T03:21:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.712+0000 7f4667ab5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c8a3cff0 con 0x55c6c8aea270 2023-09-13T03:21:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.712+0000 7f4667ab5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c8a388e0 con 0x55c6c8ae6840 2023-09-13T03:21:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.713+0000 7f465279c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6c8af8f60 0x55c6c8afb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.713+0000 7f4651f9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c8aea270 0x55c6c8ae6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.713+0000 7f465279c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6c8af8f60 0x55c6c8afb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34624/0 (socket says 172.21.15.64:34624) 2023-09-13T03:21:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.713+0000 7f465179a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6c8ae6840 0x55c6c8ae6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.713+0000 7f465279c700 1 -- 172.21.15.64:0/3648393069 learned_addr learned my addr 172.21.15.64:0/3648393069 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:21:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.713+0000 7f4651f9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c8aea270 0x55c6c8ae6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:47698/0 (socket says 172.21.15.64:47698) 2023-09-13T03:21:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.713+0000 7f465179a700 1 -- 172.21.15.64:0/3648393069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c8aea270 msgr2=0x55c6c8ae6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:21:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.713+0000 7f465179a700 1 --2- 172.21.15.64:0/3648393069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c8aea270 0x55c6c8ae6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.713+0000 7f465179a700 1 -- 172.21.15.64:0/3648393069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6c8af8f60 msgr2=0x55c6c8afb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:21:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.713+0000 7f465179a700 1 --2- 172.21.15.64:0/3648393069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6c8af8f60 0x55c6c8afb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.713+0000 7f465179a700 1 -- 172.21.15.64:0/3648393069 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6c8864d80 con 0x55c6c8ae6840 2023-09-13T03:21:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.713+0000 7f465279c700 1 --2- 172.21.15.64:0/1531360058 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f46380a07b0 0x7f46380a2c70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f464400faa0 tx=0x7f464400fe60 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:21:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.713+0000 7f4651f9b700 1 --2- 172.21.15.64:0/3648393069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c8aea270 0x55c6c8ae6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:21:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.713+0000 7f465179a700 1 --2- 172.21.15.64:0/3648393069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6c8ae6840 0x55c6c8ae6cc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f463c009cb0 tx=0x7f463c011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c76d286a013f49b1 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:21:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.714+0000 7f4637fff700 1 -- 172.21.15.64:0/3648393069 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463c011cc0 con 0x55c6c8ae6840 2023-09-13T03:21:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.714+0000 7f4637fff700 1 -- 172.21.15.64:0/3648393069 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f463c011e20 con 0x55c6c8ae6840 2023-09-13T03:21:08.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.714+0000 7f4637fff700 1 -- 172.21.15.64:0/3648393069 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463c005260 con 0x55c6c8ae6840 2023-09-13T03:21:08.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.714+0000 7f4667ab5b80 1 -- 172.21.15.64:0/3648393069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6c8ae6840 msgr2=0x55c6c8ae6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:21:08.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.714+0000 7f4667ab5b80 1 --2- 172.21.15.64:0/3648393069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6c8ae6840 0x55c6c8ae6cc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f463c009cb0 tx=0x7f463c011440 comp rx=0 tx=0).stop 2023-09-13T03:21:08.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.714+0000 7f4667ab5b80 1 -- 172.21.15.64:0/3648393069 shutdown_connections 2023-09-13T03:21:08.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.714+0000 7f4667ab5b80 1 --2- 172.21.15.64:0/3648393069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c8aea270 0x55c6c8ae6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:08.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.714+0000 7f4667ab5b80 1 --2- 172.21.15.64:0/3648393069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6c8af8f60 0x55c6c8afb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:08.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.714+0000 7f4667ab5b80 1 --2- 172.21.15.64:0/3648393069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6c8ae6840 0x55c6c8ae6cc0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:08.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.714+0000 7f4667ab5b80 1 -- 172.21.15.64:0/3648393069 >> 172.21.15.64:0/3648393069 conn(0x55c6c8a5b030 msgr2=0x55c6c8ae1560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:21:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.714+0000 7f4667ab5b80 1 -- 172.21.15.64:0/3648393069 shutdown_connections 2023-09-13T03:21:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.715+0000 7f4667ab5b80 1 -- 172.21.15.64:0/3648393069 wait complete. 2023-09-13T03:21:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.715+0000 7f4667ab5b80 1 -- start start 2023-09-13T03:21:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.715+0000 7f4667ab5b80 1 Processor -- start 2023-09-13T03:21:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.716+0000 7f4667ab5b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6c8ae6840 0x55c6c8af88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.716+0000 7f4667ab5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c8aea270 0x55c6c8af5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.716+0000 7f4667ab5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6c8af8f60 0x55c6c8af6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.716+0000 7f4667ab5b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c89d6b10 con 0x55c6c8ae6840 2023-09-13T03:21:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.716+0000 7f4667ab5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c89d6810 con 0x55c6c8aea270 2023-09-13T03:21:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.716+0000 7f4667ab5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c88ba8c0 con 0x55c6c8af8f60 2023-09-13T03:21:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.716+0000 7f4667ab5b80 10 client.0 _ll_register_callbacks cb 0x55c6c8920a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:21:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.716+0000 7f4651f9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6c8ae6840 0x55c6c8af88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.716+0000 7f4651f9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6c8ae6840 0x55c6c8af88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34628/0 (socket says 172.21.15.64:34628) 2023-09-13T03:21:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.716+0000 7f4651f9b700 1 -- 172.21.15.64:0/2376478074 learned_addr learned my addr 172.21.15.64:0/2376478074 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:21:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.716+0000 7f465279c700 1 --2- 172.21.15.64:0/2376478074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6c8af8f60 0x55c6c8af6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.716+0000 7f465179a700 1 --2- 172.21.15.64:0/2376478074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c8aea270 0x55c6c8af5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.716+0000 7f4651f9b700 1 -- 172.21.15.64:0/2376478074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6c8af8f60 msgr2=0x55c6c8af6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:21:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.716+0000 7f4651f9b700 1 --2- 172.21.15.64:0/2376478074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6c8af8f60 0x55c6c8af6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.716+0000 7f4651f9b700 1 -- 172.21.15.64:0/2376478074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c8aea270 msgr2=0x55c6c8af5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:21:08.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.716+0000 7f4651f9b700 1 --2- 172.21.15.64:0/2376478074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c8aea270 0x55c6c8af5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:08.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.716+0000 7f4651f9b700 1 -- 172.21.15.64:0/2376478074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f464c1a33e0 con 0x55c6c8ae6840 2023-09-13T03:21:08.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.717+0000 7f4651f9b700 1 --2- 172.21.15.64:0/2376478074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6c8ae6840 0x55c6c8af88b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f46480966f0 tx=0x7f4648099c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79bfd2e65a60af42 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:21:08.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.717+0000 7f46357fa700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:21:08.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.717+0000 7f46357fa700 1 -- 172.21.15.64:0/2376478074 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46480a21d0 con 0x55c6c8ae6840 2023-09-13T03:21:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.717+0000 7f4667ab5b80 1 -- 172.21.15.64:0/2376478074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6c8990490 con 0x55c6c8ae6840 2023-09-13T03:21:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.717+0000 7f46357fa700 1 -- 172.21.15.64:0/2376478074 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f4648004bf0 con 0x55c6c8ae6840 2023-09-13T03:21:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.717+0000 7f46357fa700 1 -- 172.21.15.64:0/2376478074 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46480b8170 con 0x55c6c8ae6840 2023-09-13T03:21:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.717+0000 7f4667ab5b80 10 client.16731 fetch_fsmap learned FSMap version 52 2023-09-13T03:21:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.717+0000 7f4667ab5b80 1 -- 172.21.15.64:0/2376478074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x7f463c0098e0 con 0x55c6c8ae6840 2023-09-13T03:21:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.717+0000 7f46357fa700 1 -- 172.21.15.64:0/2376478074 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4648064080 con 0x55c6c8ae6840 2023-09-13T03:21:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.718+0000 7f46357fa700 1 -- 172.21.15.64:0/2376478074 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 52) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f4648099750 con 0x55c6c8ae6840 2023-09-13T03:21:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.718+0000 7f4667ab5b80 10 client.16731 fetch_fsmap finished waiting for FSMap version 52 2023-09-13T03:21:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.718+0000 7f4667ab5b80 10 client.16731 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:21:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.718+0000 7f4667ab5b80 20 client.16731 populate_metadata read hostname 'smithi064' 2023-09-13T03:21:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.718+0000 7f4667ab5b80 1 --2- 172.21.15.64:0/2376478074 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55c6c8b087c0 0x55c6c8b0abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.718+0000 7f4667ab5b80 4 client.16731 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:21:08.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.718+0000 7f4667ab5b80 1 -- 172.21.15.64:0/2376478074 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6c8a4d840 con 0x55c6c8b087c0 2023-09-13T03:21:08.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.718+0000 7f465179a700 1 --2- 172.21.15.64:0/2376478074 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55c6c8b087c0 0x55c6c8b0abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:08.742 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.719+0000 7f465179a700 1 --2- 172.21.15.64:0/2376478074 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55c6c8b087c0 0x55c6c8b0abb0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3faab85ec3e3e9a server_cookie=97dedb49f0a4a8b7 in_seq=0 out_seq=0 2023-09-13T03:21:08.743 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.719+0000 7f46357fa700 10 client.16731 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:21:08.743 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.719+0000 7f46357fa700 1 -- 172.21.15.64:0/2376478074 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c6c8a4d840 con 0x55c6c8b087c0 2023-09-13T03:21:08.743 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.719+0000 7f46357fa700 10 client.16731 handle_command_reply: tid=0 2023-09-13T03:21:08.743 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.720+0000 7f4667ab5b80 1 client.16731 shutdown 2023-09-13T03:21:08.743 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.720+0000 7f4667ab5b80 1 -- 172.21.15.64:0/2376478074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6c8ae6840 msgr2=0x55c6c8af88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:21:08.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.720+0000 7f4667ab5b80 1 --2- 172.21.15.64:0/2376478074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6c8ae6840 0x55c6c8af88b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f46480966f0 tx=0x7f4648099c70 comp rx=0 tx=0).stop 2023-09-13T03:21:08.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.720+0000 7f4667ab5b80 1 -- 172.21.15.64:0/2376478074 shutdown_connections 2023-09-13T03:21:08.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.720+0000 7f4667ab5b80 1 --2- 172.21.15.64:0/2376478074 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55c6c8b087c0 0x55c6c8b0abb0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:08.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.720+0000 7f4667ab5b80 1 --2- 172.21.15.64:0/2376478074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6c8af8f60 0x55c6c8af6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:08.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.720+0000 7f4667ab5b80 1 --2- 172.21.15.64:0/2376478074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6c8ae6840 0x55c6c8af88b0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:08.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.720+0000 7f4667ab5b80 1 --2- 172.21.15.64:0/2376478074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c8aea270 0x55c6c8af5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:08.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.720+0000 7f46357fa700 0 client.16731 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:21:08.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.720+0000 7f4667ab5b80 1 -- 172.21.15.64:0/2376478074 >> 172.21.15.64:0/2376478074 conn(0x55c6c8a5b030 msgr2=0x55c6c8ae07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:21:08.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.720+0000 7f4667ab5b80 1 -- 172.21.15.64:0/2376478074 shutdown_connections 2023-09-13T03:21:08.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.720+0000 7f4667ab5b80 1 -- 172.21.15.64:0/2376478074 wait complete. 2023-09-13T03:21:08.746 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.720+0000 7f4667ab5b80 20 client.16731 trim_cache size 0 max 16384 2023-09-13T03:21:08.746 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.736+0000 7f4667ab5b80 1 Processor -- start 2023-09-13T03:21:08.746 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.736+0000 7f4667ab5b80 1 -- start start 2023-09-13T03:21:08.746 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.736+0000 7f4667ab5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c8ae6840 0x55c6c8affc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:08.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.736+0000 7f4667ab5b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6c8aea270 0x55c6c8ae5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:08.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.736+0000 7f4667ab5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6c8af8f60 0x55c6c8af8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:08.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.736+0000 7f4667ab5b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c8adb8a0 con 0x55c6c8aea270 2023-09-13T03:21:08.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.736+0000 7f4667ab5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c8addfc0 con 0x55c6c8ae6840 2023-09-13T03:21:08.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.736+0000 7f4667ab5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c8ae7580 con 0x55c6c8af8f60 2023-09-13T03:21:08.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.736+0000 7f465179a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6c8aea270 0x55c6c8ae5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:08.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.736+0000 7f465279c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6c8af8f60 0x55c6c8af8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:08.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.737+0000 7f465179a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6c8aea270 0x55c6c8ae5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34644/0 (socket says 172.21.15.64:34644) 2023-09-13T03:21:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.737+0000 7f465279c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6c8af8f60 0x55c6c8af8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:60444/0 (socket says 172.21.15.64:60444) 2023-09-13T03:21:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.737+0000 7f465179a700 1 -- 172.21.15.64:0/1398448795 learned_addr learned my addr 172.21.15.64:0/1398448795 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:21:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.737+0000 7f4651f9b700 1 --2- 172.21.15.64:0/1398448795 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c8ae6840 0x55c6c8affc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.737+0000 7f465179a700 1 -- 172.21.15.64:0/1398448795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6c8af8f60 msgr2=0x55c6c8af8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:21:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.737+0000 7f465179a700 1 --2- 172.21.15.64:0/1398448795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6c8af8f60 0x55c6c8af8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.737+0000 7f465179a700 1 -- 172.21.15.64:0/1398448795 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c8ae6840 msgr2=0x55c6c8affc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:21:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.737+0000 7f465179a700 1 --2- 172.21.15.64:0/1398448795 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c8ae6840 0x55c6c8affc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.737+0000 7f465179a700 1 -- 172.21.15.64:0/1398448795 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6c884dc00 con 0x55c6c8aea270 2023-09-13T03:21:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.737+0000 7f465179a700 1 --2- 172.21.15.64:0/1398448795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6c8aea270 0x55c6c8ae5e40 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f463c019000 tx=0x7f463c009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1adaaf05aac7cbf server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:21:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.737+0000 7f4640ff9700 1 -- 172.21.15.64:0/1398448795 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463c011b40 con 0x55c6c8aea270 2023-09-13T03:21:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.738+0000 7f4640ff9700 1 -- 172.21.15.64:0/1398448795 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f463c011ca0 con 0x55c6c8aea270 2023-09-13T03:21:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.738+0000 7f4640ff9700 1 -- 172.21.15.64:0/1398448795 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463c003dc0 con 0x55c6c8aea270 2023-09-13T03:21:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.738+0000 7f4667ab5b80 1 -- 172.21.15.64:0/1398448795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6c8aea270 msgr2=0x55c6c8ae5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:21:08.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.738+0000 7f4667ab5b80 1 --2- 172.21.15.64:0/1398448795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6c8aea270 0x55c6c8ae5e40 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f463c019000 tx=0x7f463c009710 comp rx=0 tx=0).stop 2023-09-13T03:21:08.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.738+0000 7f4667ab5b80 1 -- 172.21.15.64:0/1398448795 shutdown_connections 2023-09-13T03:21:08.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.738+0000 7f4667ab5b80 1 --2- 172.21.15.64:0/1398448795 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c8ae6840 0x55c6c8affc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:08.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.738+0000 7f4667ab5b80 1 --2- 172.21.15.64:0/1398448795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6c8af8f60 0x55c6c8af8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:08.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.738+0000 7f4667ab5b80 1 --2- 172.21.15.64:0/1398448795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6c8aea270 0x55c6c8ae5e40 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:08.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.738+0000 7f4667ab5b80 1 -- 172.21.15.64:0/1398448795 >> 172.21.15.64:0/1398448795 conn(0x55c6c8a5b030 msgr2=0x55c6c8ae5730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:21:08.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.738+0000 7f4667ab5b80 1 -- 172.21.15.64:0/1398448795 shutdown_connections 2023-09-13T03:21:08.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.738+0000 7f4667ab5b80 1 -- 172.21.15.64:0/1398448795 wait complete. 2023-09-13T03:21:08.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.739+0000 7f4667ab5b80 1 -- start start 2023-09-13T03:21:08.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.739+0000 7f4667ab5b80 1 Processor -- start 2023-09-13T03:21:08.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.740+0000 7f4667ab5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c8ae6840 0x55c6c8afe3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:08.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.740+0000 7f4667ab5b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6c8aea270 0x55c6c8afe920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:08.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.740+0000 7f4667ab5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6c8af8f60 0x55c6c8afee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:08.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.740+0000 7f4667ab5b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c8b5c640 con 0x55c6c8aea270 2023-09-13T03:21:08.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.740+0000 7f4667ab5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c8b5c7b0 con 0x55c6c8ae6840 2023-09-13T03:21:08.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.740+0000 7f4667ab5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c8b5c970 con 0x55c6c8af8f60 2023-09-13T03:21:08.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.740+0000 7f4667ab5b80 10 client.0 _ll_register_callbacks cb 0x7f464809a8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:21:08.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.740+0000 7f465179a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6c8aea270 0x55c6c8afe920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:08.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.740+0000 7f465179a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6c8aea270 0x55c6c8afe920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34656/0 (socket says 172.21.15.64:34656) 2023-09-13T03:21:08.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.740+0000 7f4651f9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c8ae6840 0x55c6c8afe3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:08.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.740+0000 7f465179a700 1 -- 172.21.15.64:0/3261561821 learned_addr learned my addr 172.21.15.64:0/3261561821 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:21:08.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.740+0000 7f465279c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6c8af8f60 0x55c6c8afee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:08.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.740+0000 7f4651f9b700 1 -- 172.21.15.64:0/3261561821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6c8af8f60 msgr2=0x55c6c8afee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:21:08.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.740+0000 7f4651f9b700 1 --2- 172.21.15.64:0/3261561821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6c8af8f60 0x55c6c8afee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.740+0000 7f4651f9b700 1 -- 172.21.15.64:0/3261561821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6c8aea270 msgr2=0x55c6c8afe920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:21:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.740+0000 7f4651f9b700 1 --2- 172.21.15.64:0/3261561821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6c8aea270 0x55c6c8afe920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.740+0000 7f4651f9b700 1 -- 172.21.15.64:0/3261561821 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f463c01d040 con 0x55c6c8ae6840 2023-09-13T03:21:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.740+0000 7f465279c700 1 --2- 172.21.15.64:0/3261561821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6c8af8f60 0x55c6c8afee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:21:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.740+0000 7f465179a700 1 --2- 172.21.15.64:0/3261561821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6c8aea270 0x55c6c8afe920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:21:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.741+0000 7f4651f9b700 1 --2- 172.21.15.64:0/3261561821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c8ae6840 0x55c6c8afe3e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4648096c60 tx=0x7f464809a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f8c1b14b05db036 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:21:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.741+0000 7f46357fa700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:21:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.741+0000 7f46357fa700 1 -- 172.21.15.64:0/3261561821 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46480b8170 con 0x55c6c8ae6840 2023-09-13T03:21:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.741+0000 7f4667ab5b80 1 -- 172.21.15.64:0/3261561821 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6c8a318c0 con 0x55c6c8ae6840 2023-09-13T03:21:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.741+0000 7f46357fa700 1 -- 172.21.15.64:0/3261561821 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f46480a21d0 con 0x55c6c8ae6840 2023-09-13T03:21:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.741+0000 7f46357fa700 1 -- 172.21.15.64:0/3261561821 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46480a1f60 con 0x55c6c8ae6840 2023-09-13T03:21:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.741+0000 7f4667ab5b80 10 client.26777 fetch_fsmap learned FSMap version 52 2023-09-13T03:21:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.741+0000 7f4667ab5b80 1 -- 172.21.15.64:0/3261561821 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x55c6c888b1f0 con 0x55c6c8ae6840 2023-09-13T03:21:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.741+0000 7f46357fa700 1 -- 172.21.15.64:0/3261561821 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c6c8a318c0 con 0x55c6c8ae6840 2023-09-13T03:21:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.742+0000 7f46357fa700 1 -- 172.21.15.64:0/3261561821 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 52) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f464807d030 con 0x55c6c8ae6840 2023-09-13T03:21:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.742+0000 7f4667ab5b80 10 client.26777 fetch_fsmap finished waiting for FSMap version 52 2023-09-13T03:21:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.742+0000 7f4667ab5b80 10 client.26777 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:21:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.742+0000 7f4667ab5b80 20 client.26777 populate_metadata read hostname 'smithi064' 2023-09-13T03:21:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.742+0000 7f4667ab5b80 1 --2- 172.21.15.64:0/3261561821 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55c6c8b5dda0 0x55c6c8b601c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.742+0000 7f4667ab5b80 4 client.26777 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:21:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.742+0000 7f4667ab5b80 1 -- 172.21.15.64:0/3261561821 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c6c89f2050 con 0x55c6c8b5dda0 2023-09-13T03:21:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.742+0000 7f465179a700 1 --2- 172.21.15.64:0/3261561821 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55c6c8b5dda0 0x55c6c8b601c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:08.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.743+0000 7f465179a700 1 --2- 172.21.15.64:0/3261561821 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55c6c8b5dda0 0x55c6c8b601c0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=833f6d46e542fc0c server_cookie=e25690d4b926553a in_seq=0 out_seq=0 2023-09-13T03:21:08.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.743+0000 7f46357fa700 10 client.26777 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:21:08.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.773+0000 7f46357fa700 1 -- 172.21.15.64:0/3261561821 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c6c89f2050 con 0x55c6c8b5dda0 2023-09-13T03:21:08.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.773+0000 7f46357fa700 10 client.26777 handle_command_reply: tid=0 2023-09-13T03:21:08.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.773+0000 7f4667ab5b80 1 client.26777 shutdown 2023-09-13T03:21:08.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.774+0000 7f4667ab5b80 1 -- 172.21.15.64:0/3261561821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c8ae6840 msgr2=0x55c6c8afe3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:21:08.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.774+0000 7f4667ab5b80 1 --2- 172.21.15.64:0/3261561821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c8ae6840 0x55c6c8afe3e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4648096c60 tx=0x7f464809a660 comp rx=0 tx=0).stop 2023-09-13T03:21:08.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.774+0000 7f4667ab5b80 1 -- 172.21.15.64:0/3261561821 shutdown_connections 2023-09-13T03:21:08.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.774+0000 7f4667ab5b80 1 --2- 172.21.15.64:0/3261561821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c8ae6840 0x55c6c8afe3e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:08.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.774+0000 7f4667ab5b80 1 --2- 172.21.15.64:0/3261561821 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55c6c8b5dda0 0x55c6c8b601c0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:08.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.774+0000 7f4667ab5b80 1 --2- 172.21.15.64:0/3261561821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6c8af8f60 0x55c6c8afee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.774+0000 7f4667ab5b80 1 --2- 172.21.15.64:0/3261561821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6c8aea270 0x55c6c8afe920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.774+0000 7f46357fa700 0 client.26777 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:21:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.774+0000 7f4667ab5b80 1 -- 172.21.15.64:0/3261561821 >> 172.21.15.64:0/3261561821 conn(0x55c6c8a5b030 msgr2=0x55c6c8ae3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:21:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.774+0000 7f4667ab5b80 1 -- 172.21.15.64:0/3261561821 shutdown_connections 2023-09-13T03:21:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.774+0000 7f4667ab5b80 1 -- 172.21.15.64:0/3261561821 wait complete. 2023-09-13T03:21:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.774+0000 7f4667ab5b80 20 client.26777 trim_cache size 0 max 16384 2023-09-13T03:21:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:21:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:21:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:21:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:21:08.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.775+0000 7f46541ff700 1 -- 172.21.15.64:0/1531360058 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f46380a07b0 msgr2=0x7f46380a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:21:08.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.775+0000 7f46541ff700 1 --2- 172.21.15.64:0/1531360058 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f46380a07b0 0x7f46380a2c70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f464400faa0 tx=0x7f464400fe60 comp rx=0 tx=0).stop 2023-09-13T03:21:08.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.775+0000 7f46541ff700 1 -- 172.21.15.64:0/1531360058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f464c1005b0 msgr2=0x7f464c19c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:21:08.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.775+0000 7f46541ff700 1 --2- 172.21.15.64:0/1531360058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f464c1005b0 0x7f464c19c5d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4648000c00 tx=0x7f464800bfd0 comp rx=0 tx=0).stop 2023-09-13T03:21:08.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.775+0000 7f46541ff700 1 -- 172.21.15.64:0/1531360058 shutdown_connections 2023-09-13T03:21:08.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.775+0000 7f46541ff700 1 --2- 172.21.15.64:0/1531360058 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f46380a07b0 0x7f46380a2c70 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:08.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.775+0000 7f46541ff700 1 --2- 172.21.15.64:0/1531360058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f464c106670 0x7f464c196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:08.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.775+0000 7f46541ff700 1 --2- 172.21.15.64:0/1531360058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f464c1005b0 0x7f464c19c5d0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:08.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.775+0000 7f46541ff700 1 --2- 172.21.15.64:0/1531360058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f464c100f70 0x7f464c19cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:08.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.775+0000 7f46541ff700 1 -- 172.21.15.64:0/1531360058 >> 172.21.15.64:0/1531360058 conn(0x7f464c0fc060 msgr2=0x7f464c104240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:21:08.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.775+0000 7f46541ff700 1 -- 172.21.15.64:0/1531360058 shutdown_connections 2023-09-13T03:21:08.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:08.775+0000 7f46541ff700 1 -- 172.21.15.64:0/1531360058 wait complete. 2023-09-13T03:21:08.930 INFO:tasks.workunit.client.0.smithi064.stdout: 1 168201 1.97 MB/sec execute 313 sec latency 4868.318 ms 2023-09-13T03:21:08.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:21:08 smithi142 ceph-mon[134082]: pgmap v245: 97 pgs: 97 active+clean; 88 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 680 KiB/s wr, 117 op/s 2023-09-13T03:21:09.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:21:08 smithi170 ceph-mon[134328]: pgmap v245: 97 pgs: 97 active+clean; 88 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 680 KiB/s wr, 117 op/s 2023-09-13T03:21:09.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:21:08 smithi064 ceph-mon[97675]: pgmap v245: 97 pgs: 97 active+clean; 88 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 680 KiB/s wr, 117 op/s 2023-09-13T03:21:09.930 INFO:tasks.workunit.client.0.smithi064.stdout: 1 168201 1.96 MB/sec execute 314 sec latency 5868.425 ms 2023-09-13T03:21:10.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:21:10 smithi170 ceph-mon[134328]: osdmap e90: 12 total, 12 up, 12 in 2023-09-13T03:21:10.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:21:10 smithi170 ceph-mon[134328]: pgmap v247: 97 pgs: 97 active+clean; 88 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 636 KiB/s wr, 110 op/s 2023-09-13T03:21:10.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:21:10 smithi064 ceph-mon[97675]: osdmap e90: 12 total, 12 up, 12 in 2023-09-13T03:21:10.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:21:10 smithi064 ceph-mon[97675]: pgmap v247: 97 pgs: 97 active+clean; 88 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 636 KiB/s wr, 110 op/s 2023-09-13T03:21:10.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:21:10 smithi142 ceph-mon[134082]: osdmap e90: 12 total, 12 up, 12 in 2023-09-13T03:21:10.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:21:10 smithi142 ceph-mon[134082]: pgmap v247: 97 pgs: 97 active+clean; 88 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 636 KiB/s wr, 110 op/s 2023-09-13T03:21:10.930 INFO:tasks.workunit.client.0.smithi064.stdout: 1 168201 1.96 MB/sec execute 315 sec latency 6868.565 ms 2023-09-13T03:21:11.930 INFO:tasks.workunit.client.0.smithi064.stdout: 1 168201 1.95 MB/sec execute 316 sec latency 7868.673 ms 2023-09-13T03:21:12.930 INFO:tasks.workunit.client.0.smithi064.stdout: 1 168201 1.95 MB/sec execute 317 sec latency 8868.810 ms 2023-09-13T03:21:12.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:21:12 smithi142 ceph-mon[134082]: pgmap v248: 97 pgs: 97 active+clean; 84 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 717 KiB/s wr, 132 op/s 2023-09-13T03:21:13.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:21:12 smithi170 ceph-mon[134328]: pgmap v248: 97 pgs: 97 active+clean; 84 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 717 KiB/s wr, 132 op/s 2023-09-13T03:21:13.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:21:12 smithi064 ceph-mon[97675]: pgmap v248: 97 pgs: 97 active+clean; 84 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 717 KiB/s wr, 132 op/s 2023-09-13T03:21:13.930 INFO:tasks.workunit.client.0.smithi064.stdout: 1 168201 1.94 MB/sec execute 318 sec latency 9868.993 ms 2023-09-13T03:21:14.931 INFO:tasks.workunit.client.0.smithi064.stdout: 1 168201 1.93 MB/sec execute 319 sec latency 10869.153 ms 2023-09-13T03:21:14.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:21:14 smithi142 ceph-mon[134082]: pgmap v249: 97 pgs: 97 active+clean; 84 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 642 KiB/s wr, 118 op/s 2023-09-13T03:21:15.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:21:14 smithi170 ceph-mon[134328]: pgmap v249: 97 pgs: 97 active+clean; 84 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 642 KiB/s wr, 118 op/s 2023-09-13T03:21:15.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:21:14 smithi064 ceph-mon[97675]: pgmap v249: 97 pgs: 97 active+clean; 84 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 642 KiB/s wr, 118 op/s 2023-09-13T03:21:15.931 INFO:tasks.workunit.client.0.smithi064.stdout: 1 168201 1.93 MB/sec execute 320 sec latency 11869.251 ms 2023-09-13T03:21:16.931 INFO:tasks.workunit.client.0.smithi064.stdout: 1 168201 1.92 MB/sec execute 321 sec latency 12869.385 ms 2023-09-13T03:21:16.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:21:16 smithi142 ceph-mon[134082]: pgmap v250: 97 pgs: 97 active+clean; 85 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 192 KiB/s wr, 31 op/s 2023-09-13T03:21:17.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:21:16 smithi170 ceph-mon[134328]: pgmap v250: 97 pgs: 97 active+clean; 85 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 192 KiB/s wr, 31 op/s 2023-09-13T03:21:17.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:21:16 smithi064 ceph-mon[97675]: pgmap v250: 97 pgs: 97 active+clean; 85 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 192 KiB/s wr, 31 op/s 2023-09-13T03:21:17.931 INFO:tasks.workunit.client.0.smithi064.stdout: 1 168201 1.91 MB/sec execute 322 sec latency 13869.555 ms 2023-09-13T03:21:18.931 INFO:tasks.workunit.client.0.smithi064.stdout: 1 168201 1.91 MB/sec execute 323 sec latency 14869.694 ms 2023-09-13T03:21:18.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:21:18 smithi142 ceph-mon[134082]: pgmap v251: 97 pgs: 97 active+clean; 85 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 192 KiB/s wr, 31 op/s 2023-09-13T03:21:19.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:21:18 smithi170 ceph-mon[134328]: pgmap v251: 97 pgs: 97 active+clean; 85 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 192 KiB/s wr, 31 op/s 2023-09-13T03:21:19.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:21:18 smithi064 ceph-mon[97675]: pgmap v251: 97 pgs: 97 active+clean; 85 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 192 KiB/s wr, 31 op/s 2023-09-13T03:21:19.931 INFO:tasks.workunit.client.0.smithi064.stdout: 1 168201 1.90 MB/sec execute 324 sec latency 15869.832 ms 2023-09-13T03:21:20.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:21:20 smithi170 ceph-mon[134328]: pgmap v252: 97 pgs: 97 active+clean; 85 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 187 KiB/s wr, 30 op/s 2023-09-13T03:21:20.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:21:20 smithi064 ceph-mon[97675]: pgmap v252: 97 pgs: 97 active+clean; 85 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 187 KiB/s wr, 30 op/s 2023-09-13T03:21:20.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:21:20 smithi142 ceph-mon[134082]: pgmap v252: 97 pgs: 97 active+clean; 85 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 187 KiB/s wr, 30 op/s 2023-09-13T03:21:20.931 INFO:tasks.workunit.client.0.smithi064.stdout: 1 168391 1.90 MB/sec execute 325 sec latency 16285.619 ms 2023-09-13T03:21:21.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:21:21 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:21:21.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:21:21 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:21:21.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:21:21 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:21:21.931 INFO:tasks.workunit.client.0.smithi064.stdout: 1 168391 1.89 MB/sec execute 326 sec latency 1416.582 ms 2023-09-13T03:21:21.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.935+0000 7f6c10fff700 1 -- 172.21.15.64:0/2629031418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c0c100f60 msgr2=0x7f6c0c10d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:21:21.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.935+0000 7f6c10fff700 1 --2- 172.21.15.64:0/2629031418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c0c100f60 0x7f6c0c10d420 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f6c0400b0d0 tx=0x7f6c0400b3e0 comp rx=0 tx=0).stop 2023-09-13T03:21:21.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.935+0000 7f6c10fff700 1 -- 172.21.15.64:0/2629031418 shutdown_connections 2023-09-13T03:21:21.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.935+0000 7f6c10fff700 1 --2- 172.21.15.64:0/2629031418 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c0c106660 0x7f6c0c106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.935+0000 7f6c10fff700 1 --2- 172.21.15.64:0/2629031418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c0c100f60 0x7f6c0c10d420 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.935+0000 7f6c10fff700 1 --2- 172.21.15.64:0/2629031418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c0c1005a0 0x7f6c0c100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.935+0000 7f6c10fff700 1 -- 172.21.15.64:0/2629031418 >> 172.21.15.64:0/2629031418 conn(0x7f6c0c0fc050 msgr2=0x7f6c0c0fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:21:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.935+0000 7f6c10fff700 1 -- 172.21.15.64:0/2629031418 shutdown_connections 2023-09-13T03:21:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.935+0000 7f6c10fff700 1 -- 172.21.15.64:0/2629031418 wait complete. 2023-09-13T03:21:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.936+0000 7f6c10fff700 1 Processor -- start 2023-09-13T03:21:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.936+0000 7f6c10fff700 1 -- start start 2023-09-13T03:21:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.937+0000 7f6c10fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c0c1005a0 0x7f6c0c072ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.937+0000 7f6c10fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c0c100f60 0x7f6c0c073020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.937+0000 7f6c10fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c0c106660 0x7f6c0c1a4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.937+0000 7f6c10fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c1a4e90 con 0x7f6c0c1005a0 2023-09-13T03:21:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.937+0000 7f6c10fff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c1a4fd0 con 0x7f6c0c106660 2023-09-13T03:21:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.937+0000 7f6c10fff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c1a5140 con 0x7f6c0c100f60 2023-09-13T03:21:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.937+0000 7f6c0a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c0c1005a0 0x7f6c0c072ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:21.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.937+0000 7f6c0a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c0c1005a0 0x7f6c0c072ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37422/0 (socket says 172.21.15.64:37422) 2023-09-13T03:21:21.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.937+0000 7f6c0a59c700 1 -- 172.21.15.64:0/3514000056 learned_addr learned my addr 172.21.15.64:0/3514000056 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:21:21.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.937+0000 7f6c09d9b700 1 --2- 172.21.15.64:0/3514000056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c0c100f60 0x7f6c0c073020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:21.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.937+0000 7f6c0ad9d700 1 --2- 172.21.15.64:0/3514000056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c0c106660 0x7f6c0c1a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:21.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.938+0000 7f6c09d9b700 1 -- 172.21.15.64:0/3514000056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c0c106660 msgr2=0x7f6c0c1a4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:21:21.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.938+0000 7f6c09d9b700 1 --2- 172.21.15.64:0/3514000056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c0c106660 0x7f6c0c1a4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:21.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.938+0000 7f6c09d9b700 1 -- 172.21.15.64:0/3514000056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c0c1005a0 msgr2=0x7f6c0c072ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:21:21.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.938+0000 7f6c09d9b700 1 --2- 172.21.15.64:0/3514000056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c0c1005a0 0x7f6c0c072ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:21.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.938+0000 7f6c09d9b700 1 -- 172.21.15.64:0/3514000056 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c04009d00 con 0x7f6c0c100f60 2023-09-13T03:21:21.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.938+0000 7f6c09d9b700 1 --2- 172.21.15.64:0/3514000056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c0c100f60 0x7f6c0c073020 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6c0000e9f0 tx=0x7f6c0000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:21:21.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.938+0000 7f6bfb7fe700 1 -- 172.21.15.64:0/3514000056 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0000c9c0 con 0x7f6c0c100f60 2023-09-13T03:21:21.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.938+0000 7f6c10fff700 1 -- 172.21.15.64:0/3514000056 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c0c1a5420 con 0x7f6c0c100f60 2023-09-13T03:21:21.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.938+0000 7f6bfb7fe700 1 -- 172.21.15.64:0/3514000056 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c00010430 con 0x7f6c0c100f60 2023-09-13T03:21:21.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.938+0000 7f6bfb7fe700 1 -- 172.21.15.64:0/3514000056 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c00018680 con 0x7f6c0c100f60 2023-09-13T03:21:21.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.938+0000 7f6c10fff700 1 -- 172.21.15.64:0/3514000056 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c0c1a5930 con 0x7f6c0c100f60 2023-09-13T03:21:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.942+0000 7f6bfb7fe700 1 -- 172.21.15.64:0/3514000056 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f6c00010860 con 0x7f6c0c100f60 2023-09-13T03:21:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.943+0000 7f6c24855b80 1 Processor -- start 2023-09-13T03:21:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.943+0000 7f6c24855b80 1 -- start start 2023-09-13T03:21:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.943+0000 7f6bfb7fe700 1 --2- 172.21.15.64:0/3514000056 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f6bf40a07b0 0x7f6bf40a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.944+0000 7f6bfb7fe700 1 -- 172.21.15.64:0/3514000056 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f6c00014070 con 0x7f6c0c100f60 2023-09-13T03:21:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.944+0000 7f6c24855b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f7baf21a0 0x560f7baf2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.944+0000 7f6c0ad9d700 1 --2- 172.21.15.64:0/3514000056 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f6bf40a07b0 0x7f6bf40a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.944+0000 7f6c24855b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f7baec300 0x560f7baec780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.944+0000 7f6c24855b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f7baeccc0 0x560f7baf8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.944+0000 7f6c24855b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f7ba456d0 con 0x560f7baf21a0 2023-09-13T03:21:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.944+0000 7f6c24855b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f7ba42ff0 con 0x560f7baeccc0 2023-09-13T03:21:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.944+0000 7f6c24855b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f7ba3e8e0 con 0x560f7baec300 2023-09-13T03:21:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.944+0000 7f6c0a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f7baf21a0 0x560f7baf2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.944+0000 7f6c0a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f7baf21a0 0x560f7baf2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37432/0 (socket says 172.21.15.64:37432) 2023-09-13T03:21:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.944+0000 7f6c0a59c700 1 -- 172.21.15.64:0/3255616068 learned_addr learned my addr 172.21.15.64:0/3255616068 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:21:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.944+0000 7f6c09d9b700 1 --2- 172.21.15.64:0/3255616068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f7baec300 0x560f7baec780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.944+0000 7f6c0ad9d700 1 --2- 172.21.15.64:0/3255616068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f7baeccc0 0x560f7baf8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.945+0000 7f6c0a59c700 1 -- 172.21.15.64:0/3255616068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f7baec300 msgr2=0x560f7baec780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:21:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.945+0000 7f6c0a59c700 1 --2- 172.21.15.64:0/3255616068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f7baec300 0x560f7baec780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.945+0000 7f6c0a59c700 1 -- 172.21.15.64:0/3255616068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f7baeccc0 msgr2=0x560f7baf8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:21:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.945+0000 7f6c0a59c700 1 --2- 172.21.15.64:0/3255616068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f7baeccc0 0x560f7baf8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.945+0000 7f6c0a59c700 1 -- 172.21.15.64:0/3255616068 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f7b86ad80 con 0x560f7baf21a0 2023-09-13T03:21:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.945+0000 7f6c09d9b700 1 --2- 172.21.15.64:0/3255616068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f7baec300 0x560f7baec780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:21:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.945+0000 7f6c0ad9d700 1 --2- 172.21.15.64:0/3255616068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f7baeccc0 0x560f7baf8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:21:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.945+0000 7f6c0ad9d700 1 --2- 172.21.15.64:0/3514000056 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f6bf40a07b0 0x7f6bf40a2c70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6c0400b770 tx=0x7f6c04009380 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:21:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.945+0000 7f6c0a59c700 1 --2- 172.21.15.64:0/3255616068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f7baf21a0 0x560f7baf2580 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc009dd0 tx=0x7f6bfc011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b262df7028edaa8b server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:21:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.945+0000 7f6bf8ff9700 1 -- 172.21.15.64:0/3255616068 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc004a30 con 0x560f7baf21a0 2023-09-13T03:21:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.945+0000 7f6bf8ff9700 1 -- 172.21.15.64:0/3255616068 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6bfc0054e0 con 0x560f7baf21a0 2023-09-13T03:21:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.945+0000 7f6bf8ff9700 1 -- 172.21.15.64:0/3255616068 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc0197f0 con 0x560f7baf21a0 2023-09-13T03:21:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.946+0000 7f6c24855b80 1 -- 172.21.15.64:0/3255616068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f7baf21a0 msgr2=0x560f7baf2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:21:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.946+0000 7f6c24855b80 1 --2- 172.21.15.64:0/3255616068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f7baf21a0 0x560f7baf2580 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc009dd0 tx=0x7f6bfc011720 comp rx=0 tx=0).stop 2023-09-13T03:21:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.946+0000 7f6c24855b80 1 -- 172.21.15.64:0/3255616068 shutdown_connections 2023-09-13T03:21:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.946+0000 7f6c24855b80 1 --2- 172.21.15.64:0/3255616068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f7baeccc0 0x560f7baf8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.946+0000 7f6c24855b80 1 --2- 172.21.15.64:0/3255616068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f7baf21a0 0x560f7baf2580 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.946+0000 7f6c24855b80 1 --2- 172.21.15.64:0/3255616068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f7baec300 0x560f7baec780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.946+0000 7f6c24855b80 1 -- 172.21.15.64:0/3255616068 >> 172.21.15.64:0/3255616068 conn(0x560f7ba61030 msgr2=0x560f7bae7560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:21:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.946+0000 7f6c24855b80 1 -- 172.21.15.64:0/3255616068 shutdown_connections 2023-09-13T03:21:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.946+0000 7f6c24855b80 1 -- 172.21.15.64:0/3255616068 wait complete. 2023-09-13T03:21:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.947+0000 7f6c24855b80 1 -- start start 2023-09-13T03:21:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.947+0000 7f6c24855b80 1 Processor -- start 2023-09-13T03:21:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.947+0000 7f6c24855b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f7baec300 0x560f7baf8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.947+0000 7f6c24855b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f7baeccc0 0x560f7baf6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.947+0000 7f6c24855b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f7baf21a0 0x560f7baf6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.947+0000 7f6c24855b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f7b9dcb10 con 0x560f7baeccc0 2023-09-13T03:21:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.947+0000 7f6c24855b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f7b9dc810 con 0x560f7baec300 2023-09-13T03:21:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.947+0000 7f6c24855b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f7b8c08c0 con 0x560f7baf21a0 2023-09-13T03:21:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.947+0000 7f6c24855b80 10 client.0 _ll_register_callbacks cb 0x560f7b926a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:21:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.948+0000 7f6c09d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f7baeccc0 0x560f7baf6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.948+0000 7f6c09d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f7baeccc0 0x560f7baf6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37436/0 (socket says 172.21.15.64:37436) 2023-09-13T03:21:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.948+0000 7f6c09d9b700 1 -- 172.21.15.64:0/1439645927 learned_addr learned my addr 172.21.15.64:0/1439645927 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:21:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.948+0000 7f6c0a59c700 1 --2- 172.21.15.64:0/1439645927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f7baec300 0x560f7baf8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.948+0000 7f6c0ad9d700 1 --2- 172.21.15.64:0/1439645927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f7baf21a0 0x560f7baf6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.948+0000 7f6c0a59c700 1 -- 172.21.15.64:0/1439645927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f7baf21a0 msgr2=0x560f7baf6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:21:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.948+0000 7f6c0a59c700 1 --2- 172.21.15.64:0/1439645927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f7baf21a0 0x560f7baf6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.948+0000 7f6c0a59c700 1 -- 172.21.15.64:0/1439645927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f7baeccc0 msgr2=0x560f7baf6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:21:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.948+0000 7f6c0a59c700 1 --2- 172.21.15.64:0/1439645927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f7baeccc0 0x560f7baf6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.948+0000 7f6c0a59c700 1 -- 172.21.15.64:0/1439645927 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c0c1a5930 con 0x560f7baec300 2023-09-13T03:21:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.948+0000 7f6c09d9b700 1 --2- 172.21.15.64:0/1439645927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f7baeccc0 0x560f7baf6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:21:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.948+0000 7f6c0a59c700 1 --2- 172.21.15.64:0/1439645927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f7baec300 0x560f7baf8d30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc01b040 tx=0x7f6bfc0099c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac47864a6667aef1 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:21:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.949+0000 7f6bf1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:21:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.949+0000 7f6bf1ffb700 1 -- 172.21.15.64:0/1439645927 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc005d60 con 0x560f7baec300 2023-09-13T03:21:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.949+0000 7f6c24855b80 1 -- 172.21.15.64:0/1439645927 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560f7b996490 con 0x560f7baec300 2023-09-13T03:21:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.949+0000 7f6bf1ffb700 1 -- 172.21.15.64:0/1439645927 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f6bfc004040 con 0x560f7baec300 2023-09-13T03:21:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.949+0000 7f6bf1ffb700 1 -- 172.21.15.64:0/1439645927 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6bfc005370 con 0x560f7baec300 2023-09-13T03:21:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.949+0000 7f6c24855b80 10 client.26789 fetch_fsmap learned FSMap version 53 2023-09-13T03:21:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.949+0000 7f6c24855b80 1 -- 172.21.15.64:0/1439645927 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x7f6bfc009a30 con 0x560f7baec300 2023-09-13T03:21:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.949+0000 7f6bf1ffb700 1 -- 172.21.15.64:0/1439645927 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560f7b996490 con 0x560f7baec300 2023-09-13T03:21:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.950+0000 7f6bf1ffb700 1 -- 172.21.15.64:0/1439645927 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 53) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f6bfc028020 con 0x560f7baec300 2023-09-13T03:21:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.950+0000 7f6c24855b80 10 client.26789 fetch_fsmap finished waiting for FSMap version 53 2023-09-13T03:21:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.950+0000 7f6c24855b80 10 client.26789 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:21:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.950+0000 7f6c24855b80 20 client.26789 populate_metadata read hostname 'smithi064' 2023-09-13T03:21:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.950+0000 7f6c24855b80 1 --2- 172.21.15.64:0/1439645927 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x560f7bb06200 0x560f7bb08610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.950+0000 7f6c24855b80 4 client.26789 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:21:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.950+0000 7f6c24855b80 1 -- 172.21.15.64:0/1439645927 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560f7ba53840 con 0x560f7bb06200 2023-09-13T03:21:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.950+0000 7f6c09d9b700 1 --2- 172.21.15.64:0/1439645927 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x560f7bb06200 0x560f7bb08610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.951+0000 7f6bf1ffb700 10 client.26789 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:21:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.951+0000 7f6c09d9b700 1 --2- 172.21.15.64:0/1439645927 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x560f7bb06200 0x560f7bb08610 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba10371c14e60e23 server_cookie=3b8ef6d45fa2d751 in_seq=0 out_seq=0 2023-09-13T03:21:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.952+0000 7f6bf1ffb700 1 -- 172.21.15.64:0/1439645927 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560f7ba53840 con 0x560f7bb06200 2023-09-13T03:21:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.952+0000 7f6bf1ffb700 10 client.26789 handle_command_reply: tid=0 2023-09-13T03:21:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.952+0000 7f6c24855b80 1 client.26789 shutdown 2023-09-13T03:21:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.952+0000 7f6c24855b80 1 -- 172.21.15.64:0/1439645927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f7baec300 msgr2=0x560f7baf8d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:21:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.952+0000 7f6c24855b80 1 --2- 172.21.15.64:0/1439645927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f7baec300 0x560f7baf8d30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc01b040 tx=0x7f6bfc0099c0 comp rx=0 tx=0).stop 2023-09-13T03:21:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.952+0000 7f6c24855b80 1 -- 172.21.15.64:0/1439645927 shutdown_connections 2023-09-13T03:21:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.952+0000 7f6c24855b80 1 --2- 172.21.15.64:0/1439645927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f7baec300 0x560f7baf8d30 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.952+0000 7f6c24855b80 1 --2- 172.21.15.64:0/1439645927 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x560f7bb06200 0x560f7bb08610 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.952+0000 7f6c24855b80 1 --2- 172.21.15.64:0/1439645927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f7baf21a0 0x560f7baf6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.952+0000 7f6c24855b80 1 --2- 172.21.15.64:0/1439645927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f7baeccc0 0x560f7baf6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.952+0000 7f6c24855b80 1 -- 172.21.15.64:0/1439645927 >> 172.21.15.64:0/1439645927 conn(0x560f7ba61030 msgr2=0x560f7bae67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:21:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.952+0000 7f6bf1ffb700 0 client.26789 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:21:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.952+0000 7f6c24855b80 1 -- 172.21.15.64:0/1439645927 shutdown_connections 2023-09-13T03:21:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.952+0000 7f6c24855b80 1 -- 172.21.15.64:0/1439645927 wait complete. 2023-09-13T03:21:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.952+0000 7f6c24855b80 20 client.26789 trim_cache size 0 max 16384 2023-09-13T03:21:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.967+0000 7f6c24855b80 1 Processor -- start 2023-09-13T03:21:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.967+0000 7f6c24855b80 1 -- start start 2023-09-13T03:21:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.968+0000 7f6c24855b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f7baec300 0x560f7bafd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.968+0000 7f6c24855b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f7baeccc0 0x560f7baeba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.968+0000 7f6c24855b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f7baf21a0 0x560f7bae6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.968+0000 7f6c24855b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f7ba53840 con 0x560f7baf21a0 2023-09-13T03:21:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.968+0000 7f6c24855b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f7b996490 con 0x560f7baeccc0 2023-09-13T03:21:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.968+0000 7f6c24855b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f7bae82c0 con 0x560f7baec300 2023-09-13T03:21:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.968+0000 7f6c0ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f7baf21a0 0x560f7bae6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.968+0000 7f6c09d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f7baeccc0 0x560f7baeba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.968+0000 7f6c0ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f7baf21a0 0x560f7bae6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37444/0 (socket says 172.21.15.64:37444) 2023-09-13T03:21:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.968+0000 7f6c0ad9d700 1 -- 172.21.15.64:0/4212394547 learned_addr learned my addr 172.21.15.64:0/4212394547 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:21:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.968+0000 7f6c0a59c700 1 --2- 172.21.15.64:0/4212394547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f7baec300 0x560f7bafd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.968+0000 7f6c0ad9d700 1 -- 172.21.15.64:0/4212394547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f7baec300 msgr2=0x560f7bafd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:21:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.968+0000 7f6c0ad9d700 1 --2- 172.21.15.64:0/4212394547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f7baec300 0x560f7bafd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.968+0000 7f6c0ad9d700 1 -- 172.21.15.64:0/4212394547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f7baeccc0 msgr2=0x560f7baeba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:21:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.968+0000 7f6c0ad9d700 1 --2- 172.21.15.64:0/4212394547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f7baeccc0 0x560f7baeba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:21.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.968+0000 7f6c0ad9d700 1 -- 172.21.15.64:0/4212394547 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f7b853c00 con 0x560f7baf21a0 2023-09-13T03:21:21.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.969+0000 7f6c0ad9d700 1 --2- 172.21.15.64:0/4212394547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f7baf21a0 0x560f7bae6f20 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6c0400bd30 tx=0x7f6c04024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e41fb3e22a51343 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:21:21.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.969+0000 7f6bf97fa700 1 -- 172.21.15.64:0/4212394547 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0400e040 con 0x560f7baf21a0 2023-09-13T03:21:21.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.969+0000 7f6bf97fa700 1 -- 172.21.15.64:0/4212394547 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c04007930 con 0x560f7baf21a0 2023-09-13T03:21:21.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.969+0000 7f6bf97fa700 1 -- 172.21.15.64:0/4212394547 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0401e7a0 con 0x560f7baf21a0 2023-09-13T03:21:21.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.969+0000 7f6c24855b80 1 -- 172.21.15.64:0/4212394547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f7baf21a0 msgr2=0x560f7bae6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:21:21.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.969+0000 7f6c24855b80 1 --2- 172.21.15.64:0/4212394547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f7baf21a0 0x560f7bae6f20 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6c0400bd30 tx=0x7f6c04024000 comp rx=0 tx=0).stop 2023-09-13T03:21:21.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.970+0000 7f6c24855b80 1 -- 172.21.15.64:0/4212394547 shutdown_connections 2023-09-13T03:21:21.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.970+0000 7f6c24855b80 1 --2- 172.21.15.64:0/4212394547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f7baf21a0 0x560f7bae6f20 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:21.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.970+0000 7f6c24855b80 1 --2- 172.21.15.64:0/4212394547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f7baec300 0x560f7bafd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:21.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.970+0000 7f6c24855b80 1 --2- 172.21.15.64:0/4212394547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f7baeccc0 0x560f7baeba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:21.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.970+0000 7f6c24855b80 1 -- 172.21.15.64:0/4212394547 >> 172.21.15.64:0/4212394547 conn(0x560f7ba61030 msgr2=0x560f7baeb2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:21:21.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.970+0000 7f6c24855b80 1 -- 172.21.15.64:0/4212394547 shutdown_connections 2023-09-13T03:21:21.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.970+0000 7f6c24855b80 1 -- 172.21.15.64:0/4212394547 wait complete. 2023-09-13T03:21:21.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.970+0000 7f6c24855b80 1 -- start start 2023-09-13T03:21:21.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.970+0000 7f6c24855b80 1 Processor -- start 2023-09-13T03:21:21.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.971+0000 7f6c24855b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f7baec300 0x560f7bb4e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:21.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.971+0000 7f6c24855b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f7baeccc0 0x560f7bb4e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:21.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.971+0000 7f6c24855b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f7baf21a0 0x560f7bb52300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:21.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.971+0000 7f6c24855b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f7bb52870 con 0x560f7baec300 2023-09-13T03:21:21.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.971+0000 7f6c24855b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f7bb529e0 con 0x560f7baf21a0 2023-09-13T03:21:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.971+0000 7f6c24855b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f7bb52ba0 con 0x560f7baeccc0 2023-09-13T03:21:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.971+0000 7f6c24855b80 10 client.0 _ll_register_callbacks cb 0x7f6c00095210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:21:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.971+0000 7f6c0a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f7baec300 0x560f7bb4e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.971+0000 7f6c0a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f7baec300 0x560f7bb4e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37456/0 (socket says 172.21.15.64:37456) 2023-09-13T03:21:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.971+0000 7f6c0a59c700 1 -- 172.21.15.64:0/3407793476 learned_addr learned my addr 172.21.15.64:0/3407793476 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:21:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.972+0000 7f6c09d9b700 1 --2- 172.21.15.64:0/3407793476 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f7baeccc0 0x560f7bb4e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.972+0000 7f6c0ad9d700 1 --2- 172.21.15.64:0/3407793476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f7baf21a0 0x560f7bb52300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.972+0000 7f6c0a59c700 1 -- 172.21.15.64:0/3407793476 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f7baeccc0 msgr2=0x560f7bb4e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:21:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.972+0000 7f6c0a59c700 1 --2- 172.21.15.64:0/3407793476 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f7baeccc0 0x560f7bb4e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.972+0000 7f6c0a59c700 1 -- 172.21.15.64:0/3407793476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f7baf21a0 msgr2=0x560f7bb52300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:21:21.984 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.972+0000 7f6c0a59c700 1 --2- 172.21.15.64:0/3407793476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f7baf21a0 0x560f7bb52300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:21.984 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.972+0000 7f6c0a59c700 1 -- 172.21.15.64:0/3407793476 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c0c10dd20 con 0x560f7baec300 2023-09-13T03:21:21.984 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.972+0000 7f6c0ad9d700 1 --2- 172.21.15.64:0/3407793476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f7baf21a0 0x560f7bb52300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:21:21.984 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.972+0000 7f6c0a59c700 1 --2- 172.21.15.64:0/3407793476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f7baec300 0x560f7bb4e1b0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc01b040 tx=0x7f6bfc004a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8eb6da80d2e0925 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:21:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.972+0000 7f6bf1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:21:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.972+0000 7f6bf1ffb700 1 -- 172.21.15.64:0/3407793476 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc022bd0 con 0x560f7baec300 2023-09-13T03:21:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.972+0000 7f6c24855b80 1 -- 172.21.15.64:0/3407793476 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6bfc0054e0 con 0x560f7baec300 2023-09-13T03:21:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.972+0000 7f6bf1ffb700 1 -- 172.21.15.64:0/3407793476 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f6bfc000e40 con 0x560f7baec300 2023-09-13T03:21:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.973+0000 7f6bf1ffb700 1 -- 172.21.15.64:0/3407793476 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6bfc022d30 con 0x560f7baec300 2023-09-13T03:21:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.973+0000 7f6c24855b80 10 client.16755 fetch_fsmap learned FSMap version 53 2023-09-13T03:21:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.973+0000 7f6c24855b80 1 -- 172.21.15.64:0/3407793476 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x560f7b8911f0 con 0x560f7baec300 2023-09-13T03:21:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.973+0000 7f6bf1ffb700 1 -- 172.21.15.64:0/3407793476 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6bfc0054e0 con 0x560f7baec300 2023-09-13T03:21:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.973+0000 7f6bf1ffb700 1 -- 172.21.15.64:0/3407793476 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 53) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f6bfc011790 con 0x560f7baec300 2023-09-13T03:21:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.973+0000 7f6c24855b80 10 client.16755 fetch_fsmap finished waiting for FSMap version 53 2023-09-13T03:21:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.973+0000 7f6c24855b80 10 client.16755 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:21:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.973+0000 7f6c24855b80 20 client.16755 populate_metadata read hostname 'smithi064' 2023-09-13T03:21:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.973+0000 7f6c24855b80 1 --2- 172.21.15.64:0/3407793476 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x560f7bb5b0f0 0x560f7bb5d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.973+0000 7f6c24855b80 4 client.16755 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:21:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.973+0000 7f6c24855b80 1 -- 172.21.15.64:0/3407793476 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560f7ba378c0 con 0x560f7bb5b0f0 2023-09-13T03:21:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.974+0000 7f6c09d9b700 1 --2- 172.21.15.64:0/3407793476 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x560f7bb5b0f0 0x560f7bb5d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.974+0000 7f6c09d9b700 1 --2- 172.21.15.64:0/3407793476 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x560f7bb5b0f0 0x560f7bb5d510 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e76c8b0d70a6021 server_cookie=f9a5dce489fe99c3 in_seq=0 out_seq=0 2023-09-13T03:21:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:21.974+0000 7f6bf1ffb700 10 client.16755 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:21:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:22.002+0000 7f6bf1ffb700 1 -- 172.21.15.64:0/3407793476 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560f7ba378c0 con 0x560f7bb5b0f0 2023-09-13T03:21:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:22.002+0000 7f6bf1ffb700 10 client.16755 handle_command_reply: tid=0 2023-09-13T03:21:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:22.002+0000 7f6c24855b80 1 client.16755 shutdown 2023-09-13T03:21:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:22.002+0000 7f6c24855b80 1 -- 172.21.15.64:0/3407793476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f7baec300 msgr2=0x560f7bb4e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:21:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:22.002+0000 7f6c24855b80 1 --2- 172.21.15.64:0/3407793476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f7baec300 0x560f7bb4e1b0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc01b040 tx=0x7f6bfc004a60 comp rx=0 tx=0).stop 2023-09-13T03:21:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:22.002+0000 7f6c24855b80 1 -- 172.21.15.64:0/3407793476 shutdown_connections 2023-09-13T03:21:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:22.002+0000 7f6c24855b80 1 --2- 172.21.15.64:0/3407793476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f7baf21a0 0x560f7bb52300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:22.002+0000 7f6c24855b80 1 --2- 172.21.15.64:0/3407793476 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x560f7bb5b0f0 0x560f7bb5d510 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:22.002+0000 7f6c24855b80 1 --2- 172.21.15.64:0/3407793476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f7baec300 0x560f7bb4e1b0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:22.002+0000 7f6c24855b80 1 --2- 172.21.15.64:0/3407793476 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f7baeccc0 0x560f7bb4e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:22.002+0000 7f6bf1ffb700 0 client.16755 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:21:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:22.002+0000 7f6c24855b80 1 -- 172.21.15.64:0/3407793476 >> 172.21.15.64:0/3407793476 conn(0x560f7ba61030 msgr2=0x560f7baebb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:21:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:22.002+0000 7f6c24855b80 1 -- 172.21.15.64:0/3407793476 shutdown_connections 2023-09-13T03:21:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:22.002+0000 7f6c24855b80 1 -- 172.21.15.64:0/3407793476 wait complete. 2023-09-13T03:21:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:22.003+0000 7f6c24855b80 20 client.16755 trim_cache size 0 max 16384 2023-09-13T03:21:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:21:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:21:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:21:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:21:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:22.003+0000 7f6c10fff700 1 -- 172.21.15.64:0/3514000056 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f6bf40a07b0 msgr2=0x7f6bf40a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:21:22.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:22.003+0000 7f6c10fff700 1 --2- 172.21.15.64:0/3514000056 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f6bf40a07b0 0x7f6bf40a2c70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6c0400b770 tx=0x7f6c04009380 comp rx=0 tx=0).stop 2023-09-13T03:21:22.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:22.003+0000 7f6c10fff700 1 -- 172.21.15.64:0/3514000056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c0c100f60 msgr2=0x7f6c0c073020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:21:22.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:22.003+0000 7f6c10fff700 1 --2- 172.21.15.64:0/3514000056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c0c100f60 0x7f6c0c073020 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6c0000e9f0 tx=0x7f6c0000ed00 comp rx=0 tx=0).stop 2023-09-13T03:21:22.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:22.003+0000 7f6c10fff700 1 -- 172.21.15.64:0/3514000056 shutdown_connections 2023-09-13T03:21:22.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:22.003+0000 7f6c10fff700 1 --2- 172.21.15.64:0/3514000056 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f6bf40a07b0 0x7f6bf40a2c70 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:22.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:22.003+0000 7f6c10fff700 1 --2- 172.21.15.64:0/3514000056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c0c106660 0x7f6c0c1a4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:22.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:22.003+0000 7f6c10fff700 1 --2- 172.21.15.64:0/3514000056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c0c1005a0 0x7f6c0c072ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:22.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:22.003+0000 7f6c10fff700 1 --2- 172.21.15.64:0/3514000056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c0c100f60 0x7f6c0c073020 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:22.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:22.003+0000 7f6c10fff700 1 -- 172.21.15.64:0/3514000056 >> 172.21.15.64:0/3514000056 conn(0x7f6c0c0fc050 msgr2=0x7f6c0c104230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:21:22.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:22.003+0000 7f6c10fff700 1 -- 172.21.15.64:0/3514000056 shutdown_connections 2023-09-13T03:21:22.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:22.003+0000 7f6c10fff700 1 -- 172.21.15.64:0/3514000056 wait complete. 2023-09-13T03:21:22.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:21:22 smithi170 ceph-mon[134328]: pgmap v253: 97 pgs: 97 active+clean; 85 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 436 KiB/s wr, 47 op/s 2023-09-13T03:21:22.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:21:22 smithi064 ceph-mon[97675]: pgmap v253: 97 pgs: 97 active+clean; 85 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 436 KiB/s wr, 47 op/s 2023-09-13T03:21:22.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:21:22 smithi142 ceph-mon[134082]: pgmap v253: 97 pgs: 97 active+clean; 85 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 436 KiB/s wr, 47 op/s 2023-09-13T03:21:22.931 INFO:tasks.workunit.client.0.smithi064.stdout: 1 168391 1.89 MB/sec execute 327 sec latency 2416.655 ms 2023-09-13T03:21:23.932 INFO:tasks.workunit.client.0.smithi064.stdout: 1 168391 1.88 MB/sec execute 328 sec latency 3416.765 ms 2023-09-13T03:21:24.932 INFO:tasks.workunit.client.0.smithi064.stdout: 1 168391 1.87 MB/sec execute 329 sec latency 4416.901 ms 2023-09-13T03:21:24.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:21:24 smithi142 ceph-mon[134082]: pgmap v254: 97 pgs: 97 active+clean; 85 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 381 KiB/s wr, 32 op/s 2023-09-13T03:21:25.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:21:24 smithi170 ceph-mon[134328]: pgmap v254: 97 pgs: 97 active+clean; 85 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 381 KiB/s wr, 32 op/s 2023-09-13T03:21:25.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:21:24 smithi064 ceph-mon[97675]: pgmap v254: 97 pgs: 97 active+clean; 85 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 381 KiB/s wr, 32 op/s 2023-09-13T03:21:25.932 INFO:tasks.workunit.client.0.smithi064.stdout: 1 168424 1.87 MB/sec execute 330 sec latency 5270.989 ms 2023-09-13T03:21:26.932 INFO:tasks.workunit.client.0.smithi064.stdout: 1 168424 1.86 MB/sec execute 331 sec latency 1126.642 ms 2023-09-13T03:21:26.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:21:26 smithi142 ceph-mon[134082]: pgmap v255: 97 pgs: 97 active+clean; 85 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 605 KiB/s wr, 52 op/s 2023-09-13T03:21:26.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:21:26 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:21:27.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:21:26 smithi170 ceph-mon[134328]: pgmap v255: 97 pgs: 97 active+clean; 85 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 605 KiB/s wr, 52 op/s 2023-09-13T03:21:27.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:21:26 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:21:27.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:21:26 smithi064 ceph-mon[97675]: pgmap v255: 97 pgs: 97 active+clean; 85 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 605 KiB/s wr, 52 op/s 2023-09-13T03:21:27.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:21:26 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:21:27.932 INFO:tasks.workunit.client.0.smithi064.stdout: 1 168424 1.86 MB/sec execute 332 sec latency 2126.752 ms 2023-09-13T03:21:27.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:21:27 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:21:27.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:21:27 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:21:27.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:21:27 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:21:28.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:21:27 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:21:28.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:21:27 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:21:28.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:21:27 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:21:28.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:21:27 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:21:28.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:21:27 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:21:28.170 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:21:27 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:21:28.932 INFO:tasks.workunit.client.0.smithi064.stdout: 1 168424 1.85 MB/sec execute 333 sec latency 3126.870 ms 2023-09-13T03:21:28.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:21:28 smithi142 ceph-mon[134082]: pgmap v256: 97 pgs: 97 active+clean; 85 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 499 KiB/s wr, 40 op/s 2023-09-13T03:21:29.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:21:28 smithi170 ceph-mon[134328]: pgmap v256: 97 pgs: 97 active+clean; 85 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 499 KiB/s wr, 40 op/s 2023-09-13T03:21:29.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:21:28 smithi064 ceph-mon[97675]: pgmap v256: 97 pgs: 97 active+clean; 85 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 499 KiB/s wr, 40 op/s 2023-09-13T03:21:29.932 INFO:tasks.workunit.client.0.smithi064.stdout: 1 168424 1.85 MB/sec execute 334 sec latency 4126.995 ms 2023-09-13T03:21:30.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:21:30 smithi170 ceph-mon[134328]: pgmap v257: 97 pgs: 97 active+clean; 85 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 499 KiB/s wr, 40 op/s 2023-09-13T03:21:30.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:21:30 smithi064 ceph-mon[97675]: pgmap v257: 97 pgs: 97 active+clean; 85 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 499 KiB/s wr, 40 op/s 2023-09-13T03:21:30.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:21:30 smithi142 ceph-mon[134082]: pgmap v257: 97 pgs: 97 active+clean; 85 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 499 KiB/s wr, 40 op/s 2023-09-13T03:21:30.932 INFO:tasks.workunit.client.0.smithi064.stdout: 1 169162 1.85 MB/sec execute 335 sec latency 4548.402 ms 2023-09-13T03:21:31.933 INFO:tasks.workunit.client.0.smithi064.stdout: 1 170685 1.88 MB/sec execute 336 sec latency 14.313 ms 2023-09-13T03:21:32.932 INFO:tasks.workunit.client.0.smithi064.stdout: 1 170995 1.87 MB/sec execute 337 sec latency 638.549 ms 2023-09-13T03:21:32.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:21:32 smithi142 ceph-mon[134082]: pgmap v258: 97 pgs: 97 active+clean; 85 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 584 KiB/s wr, 54 op/s 2023-09-13T03:21:33.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:21:32 smithi170 ceph-mon[134328]: pgmap v258: 97 pgs: 97 active+clean; 85 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 584 KiB/s wr, 54 op/s 2023-09-13T03:21:33.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:21:32 smithi064 ceph-mon[97675]: pgmap v258: 97 pgs: 97 active+clean; 85 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 584 KiB/s wr, 54 op/s 2023-09-13T03:21:33.933 INFO:tasks.workunit.client.0.smithi064.stdout: 1 171675 1.88 MB/sec execute 338 sec latency 1235.849 ms 2023-09-13T03:21:34.933 INFO:tasks.workunit.client.0.smithi064.stdout: 1 171965 1.88 MB/sec execute 339 sec latency 757.403 ms 2023-09-13T03:21:34.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:21:34 smithi142 ceph-mon[134082]: pgmap v259: 97 pgs: 97 active+clean; 85 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 308 KiB/s wr, 33 op/s 2023-09-13T03:21:35.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:21:34 smithi170 ceph-mon[134328]: pgmap v259: 97 pgs: 97 active+clean; 85 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 308 KiB/s wr, 33 op/s 2023-09-13T03:21:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.160+0000 7f02ec7ff700 1 -- 172.21.15.64:0/3493618888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02e40683b0 msgr2=0x7f02e410d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:21:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.160+0000 7f02ec7ff700 1 --2- 172.21.15.64:0/3493618888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02e40683b0 0x7f02e410d3d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f02dc00b0d0 tx=0x7f02dc00b3e0 comp rx=0 tx=0).stop 2023-09-13T03:21:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.160+0000 7f02ec7ff700 1 -- 172.21.15.64:0/3493618888 shutdown_connections 2023-09-13T03:21:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.160+0000 7f02ec7ff700 1 --2- 172.21.15.64:0/3493618888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02e40683b0 0x7f02e410d3d0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.160+0000 7f02ec7ff700 1 --2- 172.21.15.64:0/3493618888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02e4106650 0x7f02e4106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.160+0000 7f02ec7ff700 1 --2- 172.21.15.64:0/3493618888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02e40679f0 0x7f02e4067e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.160+0000 7f02ec7ff700 1 -- 172.21.15.64:0/3493618888 >> 172.21.15.64:0/3493618888 conn(0x7f02e4074f30 msgr2=0x7f02e4075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:21:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.160+0000 7f02ec7ff700 1 -- 172.21.15.64:0/3493618888 shutdown_connections 2023-09-13T03:21:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.160+0000 7f02ec7ff700 1 -- 172.21.15.64:0/3493618888 wait complete. 2023-09-13T03:21:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.161+0000 7f02ec7ff700 1 Processor -- start 2023-09-13T03:21:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.161+0000 7f02ec7ff700 1 -- start start 2023-09-13T03:21:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.162+0000 7f02ec7ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02e40679f0 0x7f02e419c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:35.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.162+0000 7f02ec7ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02e40683b0 0x7f02e419cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:35.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.162+0000 7f02ec7ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02e4106650 0x7f02e4196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:35.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.162+0000 7f02ec7ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e4196bf0 con 0x7f02e4106650 2023-09-13T03:21:35.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.162+0000 7f02ec7ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e4196d60 con 0x7f02e40683b0 2023-09-13T03:21:35.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.162+0000 7f02ec7ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e4196f20 con 0x7f02e40679f0 2023-09-13T03:21:35.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.162+0000 7f02ead9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02e4106650 0x7f02e4196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:35.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.162+0000 7f02ead9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02e4106650 0x7f02e4196580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51536/0 (socket says 172.21.15.64:51536) 2023-09-13T03:21:35.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.162+0000 7f02ead9c700 1 -- 172.21.15.64:0/4161480744 learned_addr learned my addr 172.21.15.64:0/4161480744 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:21:35.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.162+0000 7f02e9d9a700 1 --2- 172.21.15.64:0/4161480744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02e40683b0 0x7f02e419cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:35.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.162+0000 7f02ea59b700 1 --2- 172.21.15.64:0/4161480744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02e40679f0 0x7f02e419c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.163+0000 7f02ead9c700 1 -- 172.21.15.64:0/4161480744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02e40679f0 msgr2=0x7f02e419c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:21:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.163+0000 7f02ead9c700 1 --2- 172.21.15.64:0/4161480744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02e40679f0 0x7f02e419c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.163+0000 7f02ead9c700 1 -- 172.21.15.64:0/4161480744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02e40683b0 msgr2=0x7f02e419cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:21:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.163+0000 7f02ead9c700 1 --2- 172.21.15.64:0/4161480744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02e40683b0 0x7f02e419cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.163+0000 7f02ead9c700 1 -- 172.21.15.64:0/4161480744 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02dc009d00 con 0x7f02e4106650 2023-09-13T03:21:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.163+0000 7f02ead9c700 1 --2- 172.21.15.64:0/4161480744 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02e4106650 0x7f02e4196580 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f02dc007890 tx=0x7f02dc0078c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:21:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.163+0000 7f02db7fe700 1 -- 172.21.15.64:0/4161480744 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc00e040 con 0x7f02e4106650 2023-09-13T03:21:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.163+0000 7f02db7fe700 1 -- 172.21.15.64:0/4161480744 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02dc00bd30 con 0x7f02e4106650 2023-09-13T03:21:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.163+0000 7f02ec7ff700 1 -- 172.21.15.64:0/4161480744 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02e41971a0 con 0x7f02e4106650 2023-09-13T03:21:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.163+0000 7f02db7fe700 1 -- 172.21.15.64:0/4161480744 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc01d470 con 0x7f02e4106650 2023-09-13T03:21:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.163+0000 7f02ec7ff700 1 -- 172.21.15.64:0/4161480744 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02e41a3710 con 0x7f02e4106650 2023-09-13T03:21:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.167+0000 7f02db7fe700 1 -- 172.21.15.64:0/4161480744 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f02dc004a50 con 0x7f02e4106650 2023-09-13T03:21:35.171 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:21:34 smithi064 ceph-mon[97675]: pgmap v259: 97 pgs: 97 active+clean; 85 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 308 KiB/s wr, 33 op/s 2023-09-13T03:21:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.168+0000 7f03001a3b80 1 Processor -- start 2023-09-13T03:21:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.168+0000 7f03001a3b80 1 -- start start 2023-09-13T03:21:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.168+0000 7f02db7fe700 1 --2- 172.21.15.64:0/4161480744 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f02d00a07b0 0x7f02d00a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:35.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.168+0000 7f02db7fe700 1 -- 172.21.15.64:0/4161480744 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f02dc0cd130 con 0x7f02e4106650 2023-09-13T03:21:35.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.168+0000 7f03001a3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582f05245b0 0x5582f0524990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:35.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.168+0000 7f03001a3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582f051e710 0x5582f051eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:35.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.168+0000 7f03001a3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582f051f0d0 0x5582f052b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:35.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.168+0000 7f02e9d9a700 1 --2- 172.21.15.64:0/4161480744 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f02d00a07b0 0x7f02d00a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.168+0000 7f03001a3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582f04736d0 con 0x5582f051e710 2023-09-13T03:21:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.168+0000 7f03001a3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582f0470ff0 con 0x5582f05245b0 2023-09-13T03:21:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.168+0000 7f03001a3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582f046c8e0 con 0x5582f051f0d0 2023-09-13T03:21:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.169+0000 7f02ea59b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582f05245b0 0x5582f0524990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.169+0000 7f02ea59b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582f05245b0 0x5582f0524990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:54056/0 (socket says 172.21.15.64:54056) 2023-09-13T03:21:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.169+0000 7f02ea59b700 1 -- 172.21.15.64:0/4157241057 learned_addr learned my addr 172.21.15.64:0/4157241057 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:21:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.169+0000 7f02e9d9a700 1 --2- 172.21.15.64:0/4157241057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582f051e710 0x5582f051eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.169+0000 7f02ead9c700 1 --2- 172.21.15.64:0/4157241057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582f051f0d0 0x5582f052b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.169+0000 7f02ea59b700 1 -- 172.21.15.64:0/4157241057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582f051f0d0 msgr2=0x5582f052b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:21:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.169+0000 7f02ea59b700 1 --2- 172.21.15.64:0/4157241057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582f051f0d0 0x5582f052b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.169+0000 7f02ea59b700 1 -- 172.21.15.64:0/4157241057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582f051e710 msgr2=0x5582f051eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:21:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.169+0000 7f02ea59b700 1 --2- 172.21.15.64:0/4157241057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582f051e710 0x5582f051eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.169+0000 7f02ea59b700 1 -- 172.21.15.64:0/4157241057 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582f0298d80 con 0x5582f05245b0 2023-09-13T03:21:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.169+0000 7f02e9d9a700 1 --2- 172.21.15.64:0/4161480744 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f02d00a07b0 0x7f02d00a2c70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f02e41979f0 tx=0x7f02d4013740 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:21:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.170+0000 7f02ea59b700 1 --2- 172.21.15.64:0/4157241057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582f05245b0 0x5582f0524990 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f02e000bd70 tx=0x7f02e00106b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3520ff40be28c596 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:21:35.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.170+0000 7f02d8ff9700 1 -- 172.21.15.64:0/4157241057 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e0010e10 con 0x5582f05245b0 2023-09-13T03:21:35.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.170+0000 7f02d8ff9700 1 -- 172.21.15.64:0/4157241057 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02e0015070 con 0x5582f05245b0 2023-09-13T03:21:35.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.170+0000 7f02d8ff9700 1 -- 172.21.15.64:0/4157241057 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e00116a0 con 0x5582f05245b0 2023-09-13T03:21:35.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.170+0000 7f03001a3b80 1 -- 172.21.15.64:0/4157241057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582f05245b0 msgr2=0x5582f0524990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:21:35.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.170+0000 7f03001a3b80 1 --2- 172.21.15.64:0/4157241057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582f05245b0 0x5582f0524990 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f02e000bd70 tx=0x7f02e00106b0 comp rx=0 tx=0).stop 2023-09-13T03:21:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.171+0000 7f03001a3b80 1 -- 172.21.15.64:0/4157241057 shutdown_connections 2023-09-13T03:21:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.171+0000 7f03001a3b80 1 --2- 172.21.15.64:0/4157241057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582f05245b0 0x5582f0524990 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.171+0000 7f03001a3b80 1 --2- 172.21.15.64:0/4157241057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582f051f0d0 0x5582f052b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.171+0000 7f03001a3b80 1 --2- 172.21.15.64:0/4157241057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582f051e710 0x5582f051eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.171+0000 7f03001a3b80 1 -- 172.21.15.64:0/4157241057 >> 172.21.15.64:0/4157241057 conn(0x5582f048f030 msgr2=0x5582f0515560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:21:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.171+0000 7f03001a3b80 1 -- 172.21.15.64:0/4157241057 shutdown_connections 2023-09-13T03:21:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.171+0000 7f03001a3b80 1 -- 172.21.15.64:0/4157241057 wait complete. 2023-09-13T03:21:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.171+0000 7f03001a3b80 1 -- start start 2023-09-13T03:21:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.171+0000 7f03001a3b80 1 Processor -- start 2023-09-13T03:21:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.172+0000 7f03001a3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582f051e710 0x5582f052b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:35.184 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.172+0000 7f03001a3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582f051f0d0 0x5582f0528440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:35.184 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.172+0000 7f03001a3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582f05245b0 0x5582f0528a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:35.184 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.172+0000 7f03001a3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582f040ab10 con 0x5582f05245b0 2023-09-13T03:21:35.184 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.172+0000 7f03001a3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582f040a810 con 0x5582f051e710 2023-09-13T03:21:35.184 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.172+0000 7f03001a3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582f02ee8c0 con 0x5582f051f0d0 2023-09-13T03:21:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.172+0000 7f03001a3b80 10 client.0 _ll_register_callbacks cb 0x5582f0354a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:21:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.172+0000 7f02e9d9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582f051f0d0 0x5582f0528440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.172+0000 7f02ea59b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582f051e710 0x5582f052b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.172+0000 7f02ead9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582f05245b0 0x5582f0528a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.172+0000 7f02ea59b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582f051e710 0x5582f052b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:54064/0 (socket says 172.21.15.64:54064) 2023-09-13T03:21:35.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.172+0000 7f02ea59b700 1 -- 172.21.15.64:0/11267751 learned_addr learned my addr 172.21.15.64:0/11267751 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:21:35.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.173+0000 7f02ea59b700 1 -- 172.21.15.64:0/11267751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582f051f0d0 msgr2=0x5582f0528440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:21:35.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.173+0000 7f02ea59b700 1 --2- 172.21.15.64:0/11267751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582f051f0d0 0x5582f0528440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:35.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.173+0000 7f02ea59b700 1 -- 172.21.15.64:0/11267751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582f05245b0 msgr2=0x5582f0528a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:21:35.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.173+0000 7f02ea59b700 1 --2- 172.21.15.64:0/11267751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582f05245b0 0x5582f0528a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:35.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.173+0000 7f02ea59b700 1 -- 172.21.15.64:0/11267751 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02e000b9a0 con 0x5582f051e710 2023-09-13T03:21:35.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.173+0000 7f02ead9c700 1 --2- 172.21.15.64:0/11267751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582f05245b0 0x5582f0528a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:21:35.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.173+0000 7f02e9d9a700 1 --2- 172.21.15.64:0/11267751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582f051f0d0 0x5582f0528440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:21:35.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.173+0000 7f02ea59b700 1 --2- 172.21.15.64:0/11267751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582f051e710 0x5582f052b230 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f02e0000c00 tx=0x7f02e000bec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a53afbeef20ed76 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:21:35.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.173+0000 7f02cdffb700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:21:35.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.173+0000 7f02cdffb700 1 -- 172.21.15.64:0/11267751 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e0004230 con 0x5582f051e710 2023-09-13T03:21:35.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.173+0000 7f03001a3b80 1 -- 172.21.15.64:0/11267751 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5582f03c4490 con 0x5582f051e710 2023-09-13T03:21:35.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.173+0000 7f02cdffb700 1 -- 172.21.15.64:0/11267751 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f02e0017070 con 0x5582f051e710 2023-09-13T03:21:35.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.174+0000 7f02cdffb700 1 -- 172.21.15.64:0/11267751 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02e0004390 con 0x5582f051e710 2023-09-13T03:21:35.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.174+0000 7f03001a3b80 10 client.26807 fetch_fsmap learned FSMap version 53 2023-09-13T03:21:35.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.174+0000 7f03001a3b80 1 -- 172.21.15.64:0/11267751 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x5582f0281c00 con 0x5582f051e710 2023-09-13T03:21:35.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.174+0000 7f02cdffb700 1 -- 172.21.15.64:0/11267751 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5582f03c4490 con 0x5582f051e710 2023-09-13T03:21:35.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.174+0000 7f02cdffb700 1 -- 172.21.15.64:0/11267751 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 53) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f02e00039d0 con 0x5582f051e710 2023-09-13T03:21:35.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.174+0000 7f03001a3b80 10 client.26807 fetch_fsmap finished waiting for FSMap version 53 2023-09-13T03:21:35.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.174+0000 7f03001a3b80 10 client.26807 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:21:35.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.174+0000 7f03001a3b80 20 client.26807 populate_metadata read hostname 'smithi064' 2023-09-13T03:21:35.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.174+0000 7f03001a3b80 1 --2- 172.21.15.64:0/11267751 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5582f0538760 0x5582f053ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:35.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.175+0000 7f03001a3b80 4 client.26807 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:21:35.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.175+0000 7f03001a3b80 1 -- 172.21.15.64:0/11267751 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5582f0481840 con 0x5582f0538760 2023-09-13T03:21:35.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.175+0000 7f02e9d9a700 1 --2- 172.21.15.64:0/11267751 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5582f0538760 0x5582f053ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:35.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.175+0000 7f02e9d9a700 1 --2- 172.21.15.64:0/11267751 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5582f0538760 0x5582f053ab80 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=714e2623be1df3e server_cookie=35ee1c496f4e845d in_seq=0 out_seq=0 2023-09-13T03:21:35.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.175+0000 7f02cdffb700 10 client.26807 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:21:35.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.176+0000 7f02cdffb700 1 -- 172.21.15.64:0/11267751 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5582f0481840 con 0x5582f0538760 2023-09-13T03:21:35.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.176+0000 7f02cdffb700 10 client.26807 handle_command_reply: tid=0 2023-09-13T03:21:35.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.176+0000 7f03001a3b80 1 client.26807 shutdown 2023-09-13T03:21:35.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.176+0000 7f03001a3b80 1 -- 172.21.15.64:0/11267751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582f051e710 msgr2=0x5582f052b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:21:35.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.176+0000 7f03001a3b80 1 --2- 172.21.15.64:0/11267751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582f051e710 0x5582f052b230 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f02e0000c00 tx=0x7f02e000bec0 comp rx=0 tx=0).stop 2023-09-13T03:21:35.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.177+0000 7f03001a3b80 1 -- 172.21.15.64:0/11267751 shutdown_connections 2023-09-13T03:21:35.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.177+0000 7f03001a3b80 1 --2- 172.21.15.64:0/11267751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582f051e710 0x5582f052b230 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.177+0000 7f03001a3b80 1 --2- 172.21.15.64:0/11267751 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5582f0538760 0x5582f053ab80 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.177+0000 7f03001a3b80 1 --2- 172.21.15.64:0/11267751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582f05245b0 0x5582f0528a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.177+0000 7f03001a3b80 1 --2- 172.21.15.64:0/11267751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582f051f0d0 0x5582f0528440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.177+0000 7f02cdffb700 0 client.26807 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:21:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.177+0000 7f03001a3b80 1 -- 172.21.15.64:0/11267751 >> 172.21.15.64:0/11267751 conn(0x5582f048f030 msgr2=0x5582f05147a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:21:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.177+0000 7f03001a3b80 1 -- 172.21.15.64:0/11267751 shutdown_connections 2023-09-13T03:21:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.177+0000 7f03001a3b80 1 -- 172.21.15.64:0/11267751 wait complete. 2023-09-13T03:21:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.177+0000 7f03001a3b80 20 client.26807 trim_cache size 0 max 16384 2023-09-13T03:21:35.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.192+0000 7f03001a3b80 1 Processor -- start 2023-09-13T03:21:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.192+0000 7f03001a3b80 1 -- start start 2023-09-13T03:21:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.193+0000 7f03001a3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582f051e710 0x5582f051eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.193+0000 7f03001a3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582f051f0d0 0x5582f052ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.193+0000 7f03001a3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582f05245b0 0x5582f0522560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.193+0000 7f03001a3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582f0529400 con 0x5582f051e710 2023-09-13T03:21:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.193+0000 7f03001a3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582f0516380 con 0x5582f05245b0 2023-09-13T03:21:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.193+0000 7f03001a3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582f0511fc0 con 0x5582f051f0d0 2023-09-13T03:21:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.193+0000 7f02ea59b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582f051e710 0x5582f051eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.193+0000 7f02ea59b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582f051e710 0x5582f051eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51562/0 (socket says 172.21.15.64:51562) 2023-09-13T03:21:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.193+0000 7f02ea59b700 1 -- 172.21.15.64:0/3869549123 learned_addr learned my addr 172.21.15.64:0/3869549123 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:21:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.193+0000 7f02e9d9a700 1 --2- 172.21.15.64:0/3869549123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582f051f0d0 0x5582f052ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.193+0000 7f02ead9c700 1 --2- 172.21.15.64:0/3869549123 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582f05245b0 0x5582f0522560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.193+0000 7f02ea59b700 1 -- 172.21.15.64:0/3869549123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582f051f0d0 msgr2=0x5582f052ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:21:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.193+0000 7f02ea59b700 1 --2- 172.21.15.64:0/3869549123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582f051f0d0 0x5582f052ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.193+0000 7f02ea59b700 1 -- 172.21.15.64:0/3869549123 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582f05245b0 msgr2=0x5582f0522560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:21:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.193+0000 7f02ea59b700 1 --2- 172.21.15.64:0/3869549123 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582f05245b0 0x5582f0522560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.193+0000 7f02ea59b700 1 -- 172.21.15.64:0/3869549123 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582f02bf1f0 con 0x5582f051e710 2023-09-13T03:21:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.194+0000 7f02ea59b700 1 --2- 172.21.15.64:0/3869549123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582f051e710 0x5582f051eaf0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f02e000b450 tx=0x7f02e000b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2358cd8928d92c28 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:21:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.194+0000 7f02d97fa700 1 -- 172.21.15.64:0/3869549123 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e0003680 con 0x5582f051e710 2023-09-13T03:21:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.194+0000 7f02d97fa700 1 -- 172.21.15.64:0/3869549123 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02e0005070 con 0x5582f051e710 2023-09-13T03:21:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.194+0000 7f02d97fa700 1 -- 172.21.15.64:0/3869549123 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e001c680 con 0x5582f051e710 2023-09-13T03:21:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.194+0000 7f03001a3b80 1 -- 172.21.15.64:0/3869549123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582f051e710 msgr2=0x5582f051eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:21:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.194+0000 7f03001a3b80 1 --2- 172.21.15.64:0/3869549123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582f051e710 0x5582f051eaf0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f02e000b450 tx=0x7f02e000b630 comp rx=0 tx=0).stop 2023-09-13T03:21:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.194+0000 7f03001a3b80 1 -- 172.21.15.64:0/3869549123 shutdown_connections 2023-09-13T03:21:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.194+0000 7f03001a3b80 1 --2- 172.21.15.64:0/3869549123 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582f05245b0 0x5582f0522560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.194+0000 7f03001a3b80 1 --2- 172.21.15.64:0/3869549123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582f051e710 0x5582f051eaf0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.194+0000 7f03001a3b80 1 --2- 172.21.15.64:0/3869549123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582f051f0d0 0x5582f052ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.194+0000 7f03001a3b80 1 -- 172.21.15.64:0/3869549123 >> 172.21.15.64:0/3869549123 conn(0x5582f048f030 msgr2=0x5582f0534a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:21:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.194+0000 7f03001a3b80 1 -- 172.21.15.64:0/3869549123 shutdown_connections 2023-09-13T03:21:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.195+0000 7f03001a3b80 1 -- 172.21.15.64:0/3869549123 wait complete. 2023-09-13T03:21:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.195+0000 7f03001a3b80 1 -- start start 2023-09-13T03:21:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.195+0000 7f03001a3b80 1 Processor -- start 2023-09-13T03:21:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.195+0000 7f03001a3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582f051e710 0x5582f052d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.195+0000 7f03001a3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582f051f0d0 0x5582f052d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.196+0000 7f03001a3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582f05245b0 0x5582f052db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.196+0000 7f03001a3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582f058b630 con 0x5582f051f0d0 2023-09-13T03:21:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.196+0000 7f03001a3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582f058b7a0 con 0x5582f05245b0 2023-09-13T03:21:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.196+0000 7f03001a3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582f058b960 con 0x5582f051e710 2023-09-13T03:21:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.196+0000 7f03001a3b80 10 client.0 _ll_register_callbacks cb 0x7f02dc005320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:21:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.196+0000 7f02e9d9a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582f051f0d0 0x5582f052d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.196+0000 7f02ea59b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582f051e710 0x5582f052d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.196+0000 7f02ea59b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582f051e710 0x5582f052d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:38856/0 (socket says 172.21.15.64:38856) 2023-09-13T03:21:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.196+0000 7f02ea59b700 1 -- 172.21.15.64:0/1598239171 learned_addr learned my addr 172.21.15.64:0/1598239171 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:21:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.196+0000 7f02e9d9a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582f051f0d0 0x5582f052d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51570/0 (socket says 172.21.15.64:51570) 2023-09-13T03:21:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.196+0000 7f02ead9c700 1 --2- 172.21.15.64:0/1598239171 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582f05245b0 0x5582f052db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.196+0000 7f02e9d9a700 1 -- 172.21.15.64:0/1598239171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582f051e710 msgr2=0x5582f052d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:21:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.196+0000 7f02e9d9a700 1 --2- 172.21.15.64:0/1598239171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582f051e710 0x5582f052d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.196+0000 7f02e9d9a700 1 -- 172.21.15.64:0/1598239171 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582f05245b0 msgr2=0x5582f052db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:21:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.196+0000 7f02e9d9a700 1 --2- 172.21.15.64:0/1598239171 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582f05245b0 0x5582f052db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.196+0000 7f02e9d9a700 1 -- 172.21.15.64:0/1598239171 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5582f0281c00 con 0x5582f051f0d0 2023-09-13T03:21:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.196+0000 7f02ea59b700 1 --2- 172.21.15.64:0/1598239171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582f051e710 0x5582f052d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:21:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.197+0000 7f02e9d9a700 1 --2- 172.21.15.64:0/1598239171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582f051f0d0 0x5582f052d640 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x5582f0529af0 tx=0x7f02d40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af56519f5abaf7a6 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:21:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.197+0000 7f02cdffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:21:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.197+0000 7f02cdffb700 1 -- 172.21.15.64:0/1598239171 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d4023040 con 0x5582f051f0d0 2023-09-13T03:21:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.197+0000 7f03001a3b80 1 -- 172.21.15.64:0/1598239171 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f02e0015070 con 0x5582f051f0d0 2023-09-13T03:21:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.197+0000 7f02cdffb700 1 -- 172.21.15.64:0/1598239171 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f02d402b3f0 con 0x5582f051f0d0 2023-09-13T03:21:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.197+0000 7f02cdffb700 1 -- 172.21.15.64:0/1598239171 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02d4018070 con 0x5582f051f0d0 2023-09-13T03:21:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.197+0000 7f03001a3b80 10 client.16773 fetch_fsmap learned FSMap version 53 2023-09-13T03:21:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.197+0000 7f03001a3b80 1 -- 172.21.15.64:0/1598239171 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x5582f0297250 con 0x5582f051f0d0 2023-09-13T03:21:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.197+0000 7f02cdffb700 1 -- 172.21.15.64:0/1598239171 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02e0015070 con 0x5582f051f0d0 2023-09-13T03:21:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.198+0000 7f02cdffb700 1 -- 172.21.15.64:0/1598239171 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 53) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f02d4031070 con 0x5582f051f0d0 2023-09-13T03:21:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.198+0000 7f03001a3b80 10 client.16773 fetch_fsmap finished waiting for FSMap version 53 2023-09-13T03:21:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.198+0000 7f03001a3b80 10 client.16773 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:21:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.198+0000 7f03001a3b80 20 client.16773 populate_metadata read hostname 'smithi064' 2023-09-13T03:21:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.198+0000 7f03001a3b80 1 --2- 172.21.15.64:0/1598239171 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5582f058d0a0 0x5582f058f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.198+0000 7f03001a3b80 4 client.16773 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:21:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.198+0000 7f03001a3b80 1 -- 172.21.15.64:0/1598239171 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5582f04658c0 con 0x5582f058d0a0 2023-09-13T03:21:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.198+0000 7f02ea59b700 1 --2- 172.21.15.64:0/1598239171 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5582f058d0a0 0x5582f058f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.198+0000 7f02ea59b700 1 --2- 172.21.15.64:0/1598239171 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5582f058d0a0 0x5582f058f490 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d616958f711b093 server_cookie=43d9819c6a339749 in_seq=0 out_seq=0 2023-09-13T03:21:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.198+0000 7f02cdffb700 10 client.16773 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:21:35.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.225+0000 7f02cdffb700 1 -- 172.21.15.64:0/1598239171 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5582f04658c0 con 0x5582f058d0a0 2023-09-13T03:21:35.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.225+0000 7f02cdffb700 10 client.16773 handle_command_reply: tid=0 2023-09-13T03:21:35.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.225+0000 7f03001a3b80 1 client.16773 shutdown 2023-09-13T03:21:35.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.225+0000 7f03001a3b80 1 -- 172.21.15.64:0/1598239171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582f051f0d0 msgr2=0x5582f052d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:21:35.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.225+0000 7f03001a3b80 1 --2- 172.21.15.64:0/1598239171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582f051f0d0 0x5582f052d640 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x5582f0529af0 tx=0x7f02d40047c0 comp rx=0 tx=0).stop 2023-09-13T03:21:35.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.225+0000 7f03001a3b80 1 -- 172.21.15.64:0/1598239171 shutdown_connections 2023-09-13T03:21:35.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.225+0000 7f03001a3b80 1 --2- 172.21.15.64:0/1598239171 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582f05245b0 0x5582f052db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:35.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.225+0000 7f03001a3b80 1 --2- 172.21.15.64:0/1598239171 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5582f058d0a0 0x5582f058f490 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:35.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.225+0000 7f03001a3b80 1 --2- 172.21.15.64:0/1598239171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582f051e710 0x5582f052d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:35.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.225+0000 7f03001a3b80 1 --2- 172.21.15.64:0/1598239171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582f051f0d0 0x5582f052d640 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:35.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.225+0000 7f02cdffb700 0 client.16773 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:21:35.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.225+0000 7f03001a3b80 1 -- 172.21.15.64:0/1598239171 >> 172.21.15.64:0/1598239171 conn(0x5582f048f030 msgr2=0x5582f0522560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:21:35.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.225+0000 7f03001a3b80 1 -- 172.21.15.64:0/1598239171 shutdown_connections 2023-09-13T03:21:35.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.225+0000 7f03001a3b80 1 -- 172.21.15.64:0/1598239171 wait complete. 2023-09-13T03:21:35.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.225+0000 7f03001a3b80 20 client.16773 trim_cache size 0 max 16384 2023-09-13T03:21:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:21:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:21:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:21:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:21:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.226+0000 7f02ec7ff700 1 -- 172.21.15.64:0/4161480744 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f02d00a07b0 msgr2=0x7f02d00a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:21:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.226+0000 7f02ec7ff700 1 --2- 172.21.15.64:0/4161480744 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f02d00a07b0 0x7f02d00a2c70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f02e41979f0 tx=0x7f02d4013740 comp rx=0 tx=0).stop 2023-09-13T03:21:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.226+0000 7f02ec7ff700 1 -- 172.21.15.64:0/4161480744 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02e4106650 msgr2=0x7f02e4196580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:21:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.226+0000 7f02ec7ff700 1 --2- 172.21.15.64:0/4161480744 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02e4106650 0x7f02e4196580 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f02dc007890 tx=0x7f02dc0078c0 comp rx=0 tx=0).stop 2023-09-13T03:21:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.226+0000 7f02ec7ff700 1 -- 172.21.15.64:0/4161480744 shutdown_connections 2023-09-13T03:21:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.226+0000 7f02ec7ff700 1 --2- 172.21.15.64:0/4161480744 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f02d00a07b0 0x7f02d00a2c70 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.226+0000 7f02ec7ff700 1 --2- 172.21.15.64:0/4161480744 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02e4106650 0x7f02e4196580 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.226+0000 7f02ec7ff700 1 --2- 172.21.15.64:0/4161480744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02e40679f0 0x7f02e419c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.226+0000 7f02ec7ff700 1 --2- 172.21.15.64:0/4161480744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02e40683b0 0x7f02e419cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.226+0000 7f02ec7ff700 1 -- 172.21.15.64:0/4161480744 >> 172.21.15.64:0/4161480744 conn(0x7f02e4074f30 msgr2=0x7f02e41073f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:21:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.226+0000 7f02ec7ff700 1 -- 172.21.15.64:0/4161480744 shutdown_connections 2023-09-13T03:21:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:35.226+0000 7f02ec7ff700 1 -- 172.21.15.64:0/4161480744 wait complete. 2023-09-13T03:21:35.933 INFO:tasks.workunit.client.0.smithi064.stdout: 1 173194 1.89 MB/sec execute 340 sec latency 1174.483 ms 2023-09-13T03:21:36.933 INFO:tasks.workunit.client.0.smithi064.stdout: 1 174425 1.91 MB/sec execute 341 sec latency 14.897 ms 2023-09-13T03:21:36.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:21:36 smithi142 ceph-mon[134082]: pgmap v260: 97 pgs: 97 active+clean; 85 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 843 KiB/s wr, 97 op/s 2023-09-13T03:21:37.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:21:36 smithi170 ceph-mon[134328]: pgmap v260: 97 pgs: 97 active+clean; 85 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 843 KiB/s wr, 97 op/s 2023-09-13T03:21:37.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:21:36 smithi064 ceph-mon[97675]: pgmap v260: 97 pgs: 97 active+clean; 85 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 843 KiB/s wr, 97 op/s 2023-09-13T03:21:37.933 INFO:tasks.workunit.client.0.smithi064.stdout: 1 174489 1.90 MB/sec execute 342 sec latency 974.244 ms 2023-09-13T03:21:38.933 INFO:tasks.workunit.client.0.smithi064.stdout: 1 175060 1.90 MB/sec execute 343 sec latency 1574.099 ms 2023-09-13T03:21:38.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:21:38 smithi142 ceph-mon[134082]: pgmap v261: 97 pgs: 97 active+clean; 85 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 619 KiB/s wr, 77 op/s 2023-09-13T03:21:39.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:21:38 smithi170 ceph-mon[134328]: pgmap v261: 97 pgs: 97 active+clean; 85 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 619 KiB/s wr, 77 op/s 2023-09-13T03:21:39.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:21:38 smithi064 ceph-mon[97675]: pgmap v261: 97 pgs: 97 active+clean; 85 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 619 KiB/s wr, 77 op/s 2023-09-13T03:21:39.933 INFO:tasks.workunit.client.0.smithi064.stdout: 1 175514 1.90 MB/sec execute 344 sec latency 698.880 ms 2023-09-13T03:21:40.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:21:40 smithi170 ceph-mon[134328]: pgmap v262: 97 pgs: 97 active+clean; 85 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 619 KiB/s wr, 77 op/s 2023-09-13T03:21:40.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:21:40 smithi064 ceph-mon[97675]: pgmap v262: 97 pgs: 97 active+clean; 85 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 619 KiB/s wr, 77 op/s 2023-09-13T03:21:40.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:21:40 smithi142 ceph-mon[134082]: pgmap v262: 97 pgs: 97 active+clean; 85 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 619 KiB/s wr, 77 op/s 2023-09-13T03:21:40.933 INFO:tasks.workunit.client.0.smithi064.stdout: 1 176721 1.92 MB/sec execute 345 sec latency 1121.215 ms 2023-09-13T03:21:41.933 INFO:tasks.workunit.client.0.smithi064.stdout: 1 177958 1.93 MB/sec execute 346 sec latency 12.469 ms 2023-09-13T03:21:42.933 INFO:tasks.workunit.client.0.smithi064.stdout: 1 179058 1.94 MB/sec execute 347 sec latency 92.141 ms 2023-09-13T03:21:42.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:21:42 smithi142 ceph-mon[134082]: pgmap v263: 97 pgs: 97 active+clean; 88 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1002 KiB/s wr, 130 op/s 2023-09-13T03:21:43.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:21:42 smithi170 ceph-mon[134328]: pgmap v263: 97 pgs: 97 active+clean; 88 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1002 KiB/s wr, 130 op/s 2023-09-13T03:21:43.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:21:42 smithi064 ceph-mon[97675]: pgmap v263: 97 pgs: 97 active+clean; 88 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1002 KiB/s wr, 130 op/s 2023-09-13T03:21:43.933 INFO:tasks.workunit.client.0.smithi064.stdout: 1 179058 1.94 MB/sec execute 348 sec latency 1092.240 ms 2023-09-13T03:21:44.934 INFO:tasks.workunit.client.0.smithi064.stdout: 1 179058 1.93 MB/sec execute 349 sec latency 2092.380 ms 2023-09-13T03:21:44.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:21:44 smithi142 ceph-mon[134082]: pgmap v264: 97 pgs: 97 active+clean; 88 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 918 KiB/s wr, 116 op/s 2023-09-13T03:21:45.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:21:44 smithi170 ceph-mon[134328]: pgmap v264: 97 pgs: 97 active+clean; 88 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 918 KiB/s wr, 116 op/s 2023-09-13T03:21:45.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:21:44 smithi064 ceph-mon[97675]: pgmap v264: 97 pgs: 97 active+clean; 88 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 918 KiB/s wr, 116 op/s 2023-09-13T03:21:45.934 INFO:tasks.workunit.client.0.smithi064.stdout: 1 180264 1.94 MB/sec execute 350 sec latency 2513.857 ms 2023-09-13T03:21:46.934 INFO:tasks.workunit.client.0.smithi064.stdout: 1 181518 1.96 MB/sec execute 351 sec latency 15.705 ms 2023-09-13T03:21:46.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:21:46 smithi142 ceph-mon[134082]: pgmap v265: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.2 MiB/s wr, 163 op/s 2023-09-13T03:21:47.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:21:46 smithi170 ceph-mon[134328]: pgmap v265: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.2 MiB/s wr, 163 op/s 2023-09-13T03:21:47.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:21:46 smithi064 ceph-mon[97675]: pgmap v265: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.2 MiB/s wr, 163 op/s 2023-09-13T03:21:47.934 INFO:tasks.workunit.client.0.smithi064.stdout: 1 181635 1.96 MB/sec execute 352 sec latency 939.319 ms 2023-09-13T03:21:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.387+0000 7f4a13dff700 1 -- 172.21.15.64:0/3679439938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a0c1005b0 msgr2=0x7f4a0c100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:21:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.387+0000 7f4a13dff700 1 --2- 172.21.15.64:0/3679439938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a0c1005b0 0x7f4a0c100a30 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0099c0 tx=0x7f49fc009cd0 comp rx=0 tx=0).stop 2023-09-13T03:21:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.387+0000 7f4a13dff700 1 -- 172.21.15.64:0/3679439938 shutdown_connections 2023-09-13T03:21:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.387+0000 7f4a13dff700 1 --2- 172.21.15.64:0/3679439938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a0c106670 0x7f4a0c106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.387+0000 7f4a13dff700 1 --2- 172.21.15.64:0/3679439938 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a0c100f70 0x7f4a0c10d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.387+0000 7f4a13dff700 1 --2- 172.21.15.64:0/3679439938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a0c1005b0 0x7f4a0c100a30 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.387+0000 7f4a13dff700 1 -- 172.21.15.64:0/3679439938 >> 172.21.15.64:0/3679439938 conn(0x7f4a0c0fc060 msgr2=0x7f4a0c0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:21:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.387+0000 7f4a13dff700 1 -- 172.21.15.64:0/3679439938 shutdown_connections 2023-09-13T03:21:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.388+0000 7f4a13dff700 1 -- 172.21.15.64:0/3679439938 wait complete. 2023-09-13T03:21:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.389+0000 7f4a13dff700 1 Processor -- start 2023-09-13T03:21:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.389+0000 7f4a13dff700 1 -- start start 2023-09-13T03:21:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.389+0000 7f4a13dff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a0c1005b0 0x7f4a0c072ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.390+0000 7f4a13dff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a0c100f70 0x7f4a0c073020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.390+0000 7f4a13dff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a0c106670 0x7f4a0c1a4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.390+0000 7f4a13dff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0c1a4d60 con 0x7f4a0c100f70 2023-09-13T03:21:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.390+0000 7f4a13dff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0c1a4ea0 con 0x7f4a0c106670 2023-09-13T03:21:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.390+0000 7f4a13dff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0c1a5010 con 0x7f4a0c1005b0 2023-09-13T03:21:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.390+0000 7f4a1139a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a0c100f70 0x7f4a0c073020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:48.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.390+0000 7f4a1139a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a0c100f70 0x7f4a0c073020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59810/0 (socket says 172.21.15.64:59810) 2023-09-13T03:21:48.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.390+0000 7f4a1139a700 1 -- 172.21.15.64:0/1251869402 learned_addr learned my addr 172.21.15.64:0/1251869402 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:21:48.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.390+0000 7f4a1239c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a0c106670 0x7f4a0c1a4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:48.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.390+0000 7f4a11b9b700 1 --2- 172.21.15.64:0/1251869402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a0c1005b0 0x7f4a0c072ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:48.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.390+0000 7f4a1139a700 1 -- 172.21.15.64:0/1251869402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a0c1005b0 msgr2=0x7f4a0c072ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:21:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.390+0000 7f4a1139a700 1 --2- 172.21.15.64:0/1251869402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a0c1005b0 0x7f4a0c072ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.390+0000 7f4a1139a700 1 -- 172.21.15.64:0/1251869402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a0c106670 msgr2=0x7f4a0c1a4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:21:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.390+0000 7f4a1139a700 1 --2- 172.21.15.64:0/1251869402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a0c106670 0x7f4a0c1a4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.390+0000 7f4a1139a700 1 -- 172.21.15.64:0/1251869402 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49fc0095f0 con 0x7f4a0c100f70 2023-09-13T03:21:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.390+0000 7f4a1239c700 1 --2- 172.21.15.64:0/1251869402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a0c106670 0x7f4a0c1a4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:21:48.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.391+0000 7f4a1139a700 1 --2- 172.21.15.64:0/1251869402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a0c100f70 0x7f4a0c073020 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0098e0 tx=0x7f49fc00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:21:48.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.391+0000 7f4a02ffd700 1 -- 172.21.15.64:0/1251869402 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49fc01d070 con 0x7f4a0c100f70 2023-09-13T03:21:48.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.391+0000 7f4a13dff700 1 -- 172.21.15.64:0/1251869402 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a0c1a5290 con 0x7f4a0c100f70 2023-09-13T03:21:48.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.391+0000 7f4a02ffd700 1 -- 172.21.15.64:0/1251869402 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49fc022470 con 0x7f4a0c100f70 2023-09-13T03:21:48.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.391+0000 7f4a02ffd700 1 -- 172.21.15.64:0/1251869402 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49fc017650 con 0x7f4a0c100f70 2023-09-13T03:21:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.391+0000 7f4a13dff700 1 -- 172.21.15.64:0/1251869402 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a0c1a56b0 con 0x7f4a0c100f70 2023-09-13T03:21:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.394+0000 7f4a02ffd700 1 -- 172.21.15.64:0/1251869402 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f49fc022850 con 0x7f4a0c100f70 2023-09-13T03:21:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.396+0000 7f4a27722b80 1 Processor -- start 2023-09-13T03:21:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.396+0000 7f4a02ffd700 1 --2- 172.21.15.64:0/1251869402 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f49f80a07b0 0x7f49f80a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.396+0000 7f4a27722b80 1 -- start start 2023-09-13T03:21:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.396+0000 7f4a02ffd700 1 -- 172.21.15.64:0/1251869402 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f49fc0cca90 con 0x7f4a0c100f70 2023-09-13T03:21:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.396+0000 7f4a11b9b700 1 --2- 172.21.15.64:0/1251869402 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f49f80a07b0 0x7f49f80a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.396+0000 7f4a27722b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a740c1a0 0x5647a740c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.396+0000 7f4a27722b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647a7406300 0x5647a7406780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.396+0000 7f4a27722b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a7406cc0 0x5647a7412f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:48.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.396+0000 7f4a27722b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a735f6d0 con 0x5647a7406cc0 2023-09-13T03:21:48.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.396+0000 7f4a27722b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a735cff0 con 0x5647a740c1a0 2023-09-13T03:21:48.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.396+0000 7f4a27722b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a73588e0 con 0x5647a7406300 2023-09-13T03:21:48.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.397+0000 7f4a1239c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a7406cc0 0x5647a7412f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:48.402 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.397+0000 7f4a11b9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a740c1a0 0x5647a740c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.397+0000 7f4a1139a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647a7406300 0x5647a7406780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.397+0000 7f4a1239c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a7406cc0 0x5647a7412f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59814/0 (socket says 172.21.15.64:59814) 2023-09-13T03:21:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.397+0000 7f4a1239c700 1 -- 172.21.15.64:0/2879795486 learned_addr learned my addr 172.21.15.64:0/2879795486 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:21:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.397+0000 7f4a11b9b700 1 --2- 172.21.15.64:0/1251869402 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f49f80a07b0 0x7f49f80a2c70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4a08009cb0 tx=0x7f4a08009260 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:21:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.397+0000 7f4a1139a700 1 -- 172.21.15.64:0/2879795486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a740c1a0 msgr2=0x5647a740c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:21:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.397+0000 7f4a1139a700 1 --2- 172.21.15.64:0/2879795486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a740c1a0 0x5647a740c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.397+0000 7f4a1139a700 1 -- 172.21.15.64:0/2879795486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a7406cc0 msgr2=0x5647a7412f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:21:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.397+0000 7f4a1139a700 1 --2- 172.21.15.64:0/2879795486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a7406cc0 0x5647a7412f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.397+0000 7f4a1139a700 1 -- 172.21.15.64:0/2879795486 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647a7184d80 con 0x5647a7406300 2023-09-13T03:21:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.397+0000 7f4a1239c700 1 --2- 172.21.15.64:0/2879795486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a7406cc0 0x5647a7412f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:21:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.398+0000 7f4a1139a700 1 --2- 172.21.15.64:0/2879795486 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647a7406300 0x5647a7406780 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0ca2b0 tx=0x7f49fc099f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72562e6626a95902 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:21:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.398+0000 7f49f7fff700 1 -- 172.21.15.64:0/2879795486 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49fc09a0a0 con 0x5647a7406300 2023-09-13T03:21:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.398+0000 7f49f7fff700 1 -- 172.21.15.64:0/2879795486 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49fc09ded0 con 0x5647a7406300 2023-09-13T03:21:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.398+0000 7f49f7fff700 1 -- 172.21.15.64:0/2879795486 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49fc099590 con 0x5647a7406300 2023-09-13T03:21:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.398+0000 7f4a27722b80 1 -- 172.21.15.64:0/2879795486 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647a7406300 msgr2=0x5647a7406780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:21:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.398+0000 7f4a27722b80 1 --2- 172.21.15.64:0/2879795486 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647a7406300 0x5647a7406780 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0ca2b0 tx=0x7f49fc099f40 comp rx=0 tx=0).stop 2023-09-13T03:21:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.399+0000 7f4a27722b80 1 -- 172.21.15.64:0/2879795486 shutdown_connections 2023-09-13T03:21:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.399+0000 7f4a27722b80 1 --2- 172.21.15.64:0/2879795486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a740c1a0 0x5647a740c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.399+0000 7f4a27722b80 1 --2- 172.21.15.64:0/2879795486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a7406cc0 0x5647a7412f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.399+0000 7f4a27722b80 1 --2- 172.21.15.64:0/2879795486 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647a7406300 0x5647a7406780 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.399+0000 7f4a27722b80 1 -- 172.21.15.64:0/2879795486 >> 172.21.15.64:0/2879795486 conn(0x5647a737b030 msgr2=0x5647a7401560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:21:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.399+0000 7f4a27722b80 1 -- 172.21.15.64:0/2879795486 shutdown_connections 2023-09-13T03:21:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.399+0000 7f4a27722b80 1 -- 172.21.15.64:0/2879795486 wait complete. 2023-09-13T03:21:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.399+0000 7f4a27722b80 1 -- start start 2023-09-13T03:21:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.400+0000 7f4a27722b80 1 Processor -- start 2023-09-13T03:21:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.400+0000 7f4a27722b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a7406300 0x5647a7415b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.400+0000 7f4a27722b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647a7406cc0 0x5647a7416040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.400+0000 7f4a27722b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a740c1a0 0x5647a740fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.400+0000 7f4a27722b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a72f6b10 con 0x5647a740c1a0 2023-09-13T03:21:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.400+0000 7f4a27722b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a72f6810 con 0x5647a7406300 2023-09-13T03:21:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.400+0000 7f4a27722b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a71da8c0 con 0x5647a7406cc0 2023-09-13T03:21:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.400+0000 7f4a27722b80 10 client.0 _ll_register_callbacks cb 0x5647a7240a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:21:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.400+0000 7f4a1239c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a740c1a0 0x5647a740fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.400+0000 7f4a1139a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647a7406cc0 0x5647a7416040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.400+0000 7f4a11b9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a7406300 0x5647a7415b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.400+0000 7f4a1139a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647a7406cc0 0x5647a7416040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:49916/0 (socket says 172.21.15.64:49916) 2023-09-13T03:21:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.401+0000 7f4a1139a700 1 -- 172.21.15.64:0/2120787071 learned_addr learned my addr 172.21.15.64:0/2120787071 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:21:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.401+0000 7f4a1139a700 1 -- 172.21.15.64:0/2120787071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a7406300 msgr2=0x5647a7415b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:21:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.401+0000 7f4a1139a700 1 --2- 172.21.15.64:0/2120787071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a7406300 0x5647a7415b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:48.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.401+0000 7f4a1139a700 1 -- 172.21.15.64:0/2120787071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a740c1a0 msgr2=0x5647a740fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:21:48.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.401+0000 7f4a1139a700 1 --2- 172.21.15.64:0/2120787071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a740c1a0 0x5647a740fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:48.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.401+0000 7f4a1139a700 1 -- 172.21.15.64:0/2120787071 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a04003b90 con 0x5647a7406cc0 2023-09-13T03:21:48.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.401+0000 7f4a11b9b700 1 --2- 172.21.15.64:0/2120787071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a7406300 0x5647a7415b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:21:48.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.401+0000 7f4a1239c700 1 --2- 172.21.15.64:0/2120787071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a740c1a0 0x5647a740fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:21:48.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.401+0000 7f4a1139a700 1 --2- 172.21.15.64:0/2120787071 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647a7406cc0 0x5647a7416040 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0a1b50 tx=0x7f49fc0092b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14553841fc6c5f96 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:21:48.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.401+0000 7f49f57fa700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:21:48.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.401+0000 7f49f57fa700 1 -- 172.21.15.64:0/2120787071 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49fc0a2080 con 0x5647a7406cc0 2023-09-13T03:21:48.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.401+0000 7f4a27722b80 1 -- 172.21.15.64:0/2120787071 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5647a72b0490 con 0x5647a7406cc0 2023-09-13T03:21:48.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.401+0000 7f49f57fa700 1 -- 172.21.15.64:0/2120787071 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f49fc0b21a0 con 0x5647a7406cc0 2023-09-13T03:21:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.402+0000 7f49f57fa700 1 -- 172.21.15.64:0/2120787071 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49fc0b2360 con 0x5647a7406cc0 2023-09-13T03:21:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.402+0000 7f4a27722b80 10 client.26806 fetch_fsmap learned FSMap version 53 2023-09-13T03:21:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.402+0000 7f4a27722b80 1 -- 172.21.15.64:0/2120787071 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x7f4a0c1a56b0 con 0x5647a7406cc0 2023-09-13T03:21:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.402+0000 7f49f57fa700 1 -- 172.21.15.64:0/2120787071 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49fc07c080 con 0x5647a7406cc0 2023-09-13T03:21:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.402+0000 7f49f57fa700 1 -- 172.21.15.64:0/2120787071 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 53) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f49fc00fad0 con 0x5647a7406cc0 2023-09-13T03:21:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.402+0000 7f4a27722b80 10 client.26806 fetch_fsmap finished waiting for FSMap version 53 2023-09-13T03:21:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.402+0000 7f4a27722b80 10 client.26806 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:21:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.402+0000 7f4a27722b80 20 client.26806 populate_metadata read hostname 'smithi064' 2023-09-13T03:21:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.402+0000 7f4a27722b80 1 --2- 172.21.15.64:0/2120787071 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5647a741fb10 0x5647a7421f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.402+0000 7f4a27722b80 4 client.26806 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:21:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.402+0000 7f4a27722b80 1 -- 172.21.15.64:0/2120787071 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5647a736d840 con 0x5647a741fb10 2023-09-13T03:21:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.403+0000 7f4a11b9b700 1 --2- 172.21.15.64:0/2120787071 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5647a741fb10 0x5647a7421f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.403+0000 7f4a11b9b700 1 --2- 172.21.15.64:0/2120787071 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5647a741fb10 0x5647a7421f30 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=736fe1791eb9d6a1 server_cookie=c1ab84b540805277 in_seq=0 out_seq=0 2023-09-13T03:21:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.403+0000 7f49f57fa700 10 client.26806 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:21:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.404+0000 7f49f57fa700 1 -- 172.21.15.64:0/2120787071 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5647a736d840 con 0x5647a741fb10 2023-09-13T03:21:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.404+0000 7f49f57fa700 10 client.26806 handle_command_reply: tid=0 2023-09-13T03:21:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.404+0000 7f4a27722b80 1 client.26806 shutdown 2023-09-13T03:21:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.404+0000 7f4a27722b80 1 -- 172.21.15.64:0/2120787071 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647a7406cc0 msgr2=0x5647a7416040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:21:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.404+0000 7f4a27722b80 1 --2- 172.21.15.64:0/2120787071 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647a7406cc0 0x5647a7416040 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0a1b50 tx=0x7f49fc0092b0 comp rx=0 tx=0).stop 2023-09-13T03:21:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.404+0000 7f4a27722b80 1 -- 172.21.15.64:0/2120787071 shutdown_connections 2023-09-13T03:21:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.404+0000 7f4a27722b80 1 --2- 172.21.15.64:0/2120787071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a7406300 0x5647a7415b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.404+0000 7f4a27722b80 1 --2- 172.21.15.64:0/2120787071 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5647a741fb10 0x5647a7421f30 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.404+0000 7f4a27722b80 1 --2- 172.21.15.64:0/2120787071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a740c1a0 0x5647a740fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.404+0000 7f4a27722b80 1 --2- 172.21.15.64:0/2120787071 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647a7406cc0 0x5647a7416040 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.404+0000 7f4a27722b80 1 -- 172.21.15.64:0/2120787071 >> 172.21.15.64:0/2120787071 conn(0x5647a737b030 msgr2=0x5647a74007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:21:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.404+0000 7f49f57fa700 0 client.26806 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:21:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.404+0000 7f4a27722b80 1 -- 172.21.15.64:0/2120787071 shutdown_connections 2023-09-13T03:21:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.404+0000 7f4a27722b80 1 -- 172.21.15.64:0/2120787071 wait complete. 2023-09-13T03:21:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.405+0000 7f4a27722b80 20 client.26806 trim_cache size 0 max 16384 2023-09-13T03:21:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.420+0000 7f4a27722b80 1 Processor -- start 2023-09-13T03:21:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.420+0000 7f4a27722b80 1 -- start start 2023-09-13T03:21:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.420+0000 7f4a27722b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a7406300 0x5647a740fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.420+0000 7f4a27722b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647a7406cc0 0x5647a7400f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.420+0000 7f4a27722b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a740c1a0 0x5647a7401460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.420+0000 7f4a27722b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a7410bc0 con 0x5647a740c1a0 2023-09-13T03:21:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.420+0000 7f4a27722b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a7422590 con 0x5647a7406300 2023-09-13T03:21:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.420+0000 7f4a27722b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a740ac20 con 0x5647a7406cc0 2023-09-13T03:21:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.420+0000 7f4a1239c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a740c1a0 0x5647a7401460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.421+0000 7f4a1239c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a740c1a0 0x5647a7401460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59832/0 (socket says 172.21.15.64:59832) 2023-09-13T03:21:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.421+0000 7f4a1239c700 1 -- 172.21.15.64:0/3818993151 learned_addr learned my addr 172.21.15.64:0/3818993151 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:21:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.421+0000 7f4a11b9b700 1 --2- 172.21.15.64:0/3818993151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a7406300 0x5647a740fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.421+0000 7f4a1139a700 1 --2- 172.21.15.64:0/3818993151 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647a7406cc0 0x5647a7400f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.421+0000 7f4a1239c700 1 -- 172.21.15.64:0/3818993151 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647a7406cc0 msgr2=0x5647a7400f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:21:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.421+0000 7f4a1239c700 1 --2- 172.21.15.64:0/3818993151 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647a7406cc0 0x5647a7400f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.421+0000 7f4a1239c700 1 -- 172.21.15.64:0/3818993151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a7406300 msgr2=0x5647a740fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:21:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.421+0000 7f4a1239c700 1 --2- 172.21.15.64:0/3818993151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a7406300 0x5647a740fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.421+0000 7f4a1239c700 1 -- 172.21.15.64:0/3818993151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647a716dc00 con 0x5647a740c1a0 2023-09-13T03:21:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.421+0000 7f4a1139a700 1 --2- 172.21.15.64:0/3818993151 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647a7406cc0 0x5647a7400f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:21:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.421+0000 7f4a11b9b700 1 --2- 172.21.15.64:0/3818993151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a7406300 0x5647a740fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:21:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.421+0000 7f4a1239c700 1 --2- 172.21.15.64:0/3818993151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a740c1a0 0x5647a7401460 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f4a040046a0 tx=0x7f4a04009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=984df69c6bbdc682 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:21:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.421+0000 7f4a00bff700 1 -- 172.21.15.64:0/3818993151 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a04004db0 con 0x5647a740c1a0 2023-09-13T03:21:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.421+0000 7f4a00bff700 1 -- 172.21.15.64:0/3818993151 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a0400f040 con 0x5647a740c1a0 2023-09-13T03:21:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.421+0000 7f4a00bff700 1 -- 172.21.15.64:0/3818993151 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a04007bf0 con 0x5647a740c1a0 2023-09-13T03:21:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.422+0000 7f4a27722b80 1 -- 172.21.15.64:0/3818993151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a740c1a0 msgr2=0x5647a7401460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:21:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.422+0000 7f4a27722b80 1 --2- 172.21.15.64:0/3818993151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a740c1a0 0x5647a7401460 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f4a040046a0 tx=0x7f4a04009920 comp rx=0 tx=0).stop 2023-09-13T03:21:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.422+0000 7f4a27722b80 1 -- 172.21.15.64:0/3818993151 shutdown_connections 2023-09-13T03:21:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.422+0000 7f4a27722b80 1 --2- 172.21.15.64:0/3818993151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a7406300 0x5647a740fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.422+0000 7f4a27722b80 1 --2- 172.21.15.64:0/3818993151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a740c1a0 0x5647a7401460 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:48.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.422+0000 7f4a27722b80 1 --2- 172.21.15.64:0/3818993151 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647a7406cc0 0x5647a7400f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:48.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.422+0000 7f4a27722b80 1 -- 172.21.15.64:0/3818993151 >> 172.21.15.64:0/3818993151 conn(0x5647a737b030 msgr2=0x5647a7405b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:21:48.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.422+0000 7f4a27722b80 1 -- 172.21.15.64:0/3818993151 shutdown_connections 2023-09-13T03:21:48.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.422+0000 7f4a27722b80 1 -- 172.21.15.64:0/3818993151 wait complete. 2023-09-13T03:21:48.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.423+0000 7f4a27722b80 1 -- start start 2023-09-13T03:21:48.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.423+0000 7f4a27722b80 1 Processor -- start 2023-09-13T03:21:48.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.423+0000 7f4a27722b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a7406300 0x5647a746da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:48.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.423+0000 7f4a27722b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a7406cc0 0x5647a746dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:48.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.423+0000 7f4a27722b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647a740c1a0 0x5647a7467db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:48.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.423+0000 7f4a27722b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a7468390 con 0x5647a7406300 2023-09-13T03:21:48.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.423+0000 7f4a27722b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a7468500 con 0x5647a7406cc0 2023-09-13T03:21:48.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.423+0000 7f4a27722b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a74686c0 con 0x5647a740c1a0 2023-09-13T03:21:48.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.423+0000 7f4a27722b80 10 client.0 _ll_register_callbacks cb 0x7f4a04003a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:21:48.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.424+0000 7f4a11b9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a7406300 0x5647a746da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:48.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.424+0000 7f4a11b9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a7406300 0x5647a746da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59840/0 (socket says 172.21.15.64:59840) 2023-09-13T03:21:48.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.424+0000 7f4a11b9b700 1 -- 172.21.15.64:0/2217798551 learned_addr learned my addr 172.21.15.64:0/2217798551 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:21:48.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.424+0000 7f4a1139a700 1 --2- 172.21.15.64:0/2217798551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a7406cc0 0x5647a746dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:48.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.424+0000 7f4a1239c700 1 --2- 172.21.15.64:0/2217798551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647a740c1a0 0x5647a7467db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:48.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.424+0000 7f4a11b9b700 1 -- 172.21.15.64:0/2217798551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647a740c1a0 msgr2=0x5647a7467db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:21:48.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.424+0000 7f4a11b9b700 1 --2- 172.21.15.64:0/2217798551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647a740c1a0 0x5647a7467db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.424+0000 7f4a11b9b700 1 -- 172.21.15.64:0/2217798551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a7406cc0 msgr2=0x5647a746dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:21:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.424+0000 7f4a11b9b700 1 --2- 172.21.15.64:0/2217798551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a7406cc0 0x5647a746dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.424+0000 7f4a11b9b700 1 -- 172.21.15.64:0/2217798551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a04004350 con 0x5647a7406300 2023-09-13T03:21:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.424+0000 7f4a1139a700 1 --2- 172.21.15.64:0/2217798551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a7406cc0 0x5647a746dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:21:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.424+0000 7f4a11b9b700 1 --2- 172.21.15.64:0/2217798551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a7406300 0x5647a746da80 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x5647a7422800 tx=0x7f4a080047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5996fb59a8451883 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:21:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.424+0000 7f49f57fa700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:21:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.425+0000 7f49f57fa700 1 -- 172.21.15.64:0/2217798551 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0802b4a0 con 0x5647a7406300 2023-09-13T03:21:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.425+0000 7f4a27722b80 1 -- 172.21.15.64:0/2217798551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f49fc09ded0 con 0x5647a7406300 2023-09-13T03:21:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.425+0000 7f49f57fa700 1 -- 172.21.15.64:0/2217798551 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f4a08003680 con 0x5647a7406300 2023-09-13T03:21:48.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.425+0000 7f49f57fa700 1 -- 172.21.15.64:0/2217798551 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a08003840 con 0x5647a7406300 2023-09-13T03:21:48.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.425+0000 7f4a27722b80 10 client.16791 fetch_fsmap learned FSMap version 53 2023-09-13T03:21:48.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.425+0000 7f4a27722b80 1 -- 172.21.15.64:0/2217798551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x5647a71ab1f0 con 0x5647a7406300 2023-09-13T03:21:48.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.425+0000 7f49f57fa700 1 -- 172.21.15.64:0/2217798551 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49fc09ded0 con 0x5647a7406300 2023-09-13T03:21:48.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.425+0000 7f49f57fa700 1 -- 172.21.15.64:0/2217798551 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 53) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f4a0803d6e0 con 0x5647a7406300 2023-09-13T03:21:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.425+0000 7f4a27722b80 10 client.16791 fetch_fsmap finished waiting for FSMap version 53 2023-09-13T03:21:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.425+0000 7f4a27722b80 10 client.16791 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:21:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.425+0000 7f4a27722b80 20 client.16791 populate_metadata read hostname 'smithi064' 2023-09-13T03:21:48.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.425+0000 7f4a27722b80 1 --2- 172.21.15.64:0/2217798551 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5647a74758c0 0x5647a7477cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:21:48.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.425+0000 7f4a27722b80 4 client.16791 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:21:48.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.425+0000 7f4a27722b80 1 -- 172.21.15.64:0/2217798551 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5647a73518c0 con 0x5647a74758c0 2023-09-13T03:21:48.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.426+0000 7f4a1139a700 1 --2- 172.21.15.64:0/2217798551 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5647a74758c0 0x5647a7477cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:21:48.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.426+0000 7f4a1139a700 1 --2- 172.21.15.64:0/2217798551 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5647a74758c0 0x5647a7477cb0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4eff5d6982c9a4a server_cookie=9add05f05df0336a in_seq=0 out_seq=0 2023-09-13T03:21:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.426+0000 7f49f57fa700 10 client.16791 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:21:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.451+0000 7f49f57fa700 1 -- 172.21.15.64:0/2217798551 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5647a73518c0 con 0x5647a74758c0 2023-09-13T03:21:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.451+0000 7f49f57fa700 10 client.16791 handle_command_reply: tid=0 2023-09-13T03:21:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.451+0000 7f4a27722b80 1 client.16791 shutdown 2023-09-13T03:21:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.451+0000 7f4a27722b80 1 -- 172.21.15.64:0/2217798551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a7406300 msgr2=0x5647a746da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:21:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.451+0000 7f4a27722b80 1 --2- 172.21.15.64:0/2217798551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a7406300 0x5647a746da80 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x5647a7422800 tx=0x7f4a080047c0 comp rx=0 tx=0).stop 2023-09-13T03:21:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.451+0000 7f4a27722b80 1 -- 172.21.15.64:0/2217798551 shutdown_connections 2023-09-13T03:21:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.451+0000 7f4a27722b80 1 --2- 172.21.15.64:0/2217798551 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5647a74758c0 0x5647a7477cb0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.451+0000 7f4a27722b80 1 --2- 172.21.15.64:0/2217798551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5647a740c1a0 0x5647a7467db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.451+0000 7f4a27722b80 1 --2- 172.21.15.64:0/2217798551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647a7406300 0x5647a746da80 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.451+0000 7f49f57fa700 0 client.16791 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:21:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.451+0000 7f4a27722b80 1 --2- 172.21.15.64:0/2217798551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a7406cc0 0x5647a746dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.451+0000 7f4a27722b80 1 -- 172.21.15.64:0/2217798551 >> 172.21.15.64:0/2217798551 conn(0x5647a737b030 msgr2=0x5647a7405b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:21:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.451+0000 7f4a27722b80 1 -- 172.21.15.64:0/2217798551 shutdown_connections 2023-09-13T03:21:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.452+0000 7f4a27722b80 1 -- 172.21.15.64:0/2217798551 wait complete. 2023-09-13T03:21:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.452+0000 7f4a27722b80 20 client.16791 trim_cache size 0 max 16384 2023-09-13T03:21:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:21:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:21:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:21:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:21:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.452+0000 7f4a13dff700 1 -- 172.21.15.64:0/1251869402 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f49f80a07b0 msgr2=0x7f49f80a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:21:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.452+0000 7f4a13dff700 1 --2- 172.21.15.64:0/1251869402 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f49f80a07b0 0x7f49f80a2c70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4a08009cb0 tx=0x7f4a08009260 comp rx=0 tx=0).stop 2023-09-13T03:21:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.452+0000 7f4a13dff700 1 -- 172.21.15.64:0/1251869402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a0c100f70 msgr2=0x7f4a0c073020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:21:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.452+0000 7f4a13dff700 1 --2- 172.21.15.64:0/1251869402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a0c100f70 0x7f4a0c073020 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0098e0 tx=0x7f49fc00f690 comp rx=0 tx=0).stop 2023-09-13T03:21:48.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.452+0000 7f4a13dff700 1 -- 172.21.15.64:0/1251869402 shutdown_connections 2023-09-13T03:21:48.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.452+0000 7f4a13dff700 1 --2- 172.21.15.64:0/1251869402 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f49f80a07b0 0x7f49f80a2c70 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:48.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.452+0000 7f4a13dff700 1 --2- 172.21.15.64:0/1251869402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a0c106670 0x7f4a0c1a4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:48.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.452+0000 7f4a13dff700 1 --2- 172.21.15.64:0/1251869402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a0c1005b0 0x7f4a0c072ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:48.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.452+0000 7f4a13dff700 1 --2- 172.21.15.64:0/1251869402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a0c100f70 0x7f4a0c073020 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:21:48.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.452+0000 7f4a13dff700 1 -- 172.21.15.64:0/1251869402 >> 172.21.15.64:0/1251869402 conn(0x7f4a0c0fc060 msgr2=0x7f4a0c0fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:21:48.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.453+0000 7f4a13dff700 1 -- 172.21.15.64:0/1251869402 shutdown_connections 2023-09-13T03:21:48.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:21:48.453+0000 7f4a13dff700 1 -- 172.21.15.64:0/1251869402 wait complete. 2023-09-13T03:21:48.934 INFO:tasks.workunit.client.0.smithi064.stdout: 1 181703 1.95 MB/sec execute 353 sec latency 1533.928 ms 2023-09-13T03:21:48.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:21:48 smithi142 ceph-mon[134082]: pgmap v266: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 706 KiB/s wr, 100 op/s 2023-09-13T03:21:49.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:21:48 smithi170 ceph-mon[134328]: pgmap v266: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 706 KiB/s wr, 100 op/s 2023-09-13T03:21:49.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:21:48 smithi064 ceph-mon[97675]: pgmap v266: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 706 KiB/s wr, 100 op/s 2023-09-13T03:21:49.934 INFO:tasks.workunit.client.0.smithi064.stdout: 1 181703 1.94 MB/sec execute 354 sec latency 1352.896 ms 2023-09-13T03:21:49.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:21:49 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:21:50.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:21:49 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:21:50.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:21:49 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:21:50.934 INFO:tasks.workunit.client.0.smithi064.stdout: 1 181703 1.94 MB/sec execute 355 sec latency 2352.992 ms 2023-09-13T03:21:50.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:21:50 smithi142 ceph-mon[134082]: pgmap v267: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 706 KiB/s wr, 100 op/s 2023-09-13T03:21:51.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:21:50 smithi170 ceph-mon[134328]: pgmap v267: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 706 KiB/s wr, 100 op/s 2023-09-13T03:21:51.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:21:50 smithi064 ceph-mon[97675]: pgmap v267: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 706 KiB/s wr, 100 op/s 2023-09-13T03:21:51.934 INFO:tasks.workunit.client.0.smithi064.stdout: 1 181703 1.93 MB/sec execute 356 sec latency 3353.088 ms 2023-09-13T03:21:52.934 INFO:tasks.workunit.client.0.smithi064.stdout: 1 181703 1.93 MB/sec execute 357 sec latency 4353.166 ms 2023-09-13T03:21:52.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:21:52 smithi142 ceph-mon[134082]: pgmap v268: 97 pgs: 97 active+clean; 89 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 944 KiB/s wr, 138 op/s 2023-09-13T03:21:53.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:21:52 smithi170 ceph-mon[134328]: pgmap v268: 97 pgs: 97 active+clean; 89 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 944 KiB/s wr, 138 op/s 2023-09-13T03:21:53.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:21:52 smithi064 ceph-mon[97675]: pgmap v268: 97 pgs: 97 active+clean; 89 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 944 KiB/s wr, 138 op/s 2023-09-13T03:21:53.934 INFO:tasks.workunit.client.0.smithi064.stdout: 1 182365 1.94 MB/sec execute 358 sec latency 4948.025 ms 2023-09-13T03:21:54.935 INFO:tasks.workunit.client.0.smithi064.stdout: 1 182604 1.93 MB/sec execute 359 sec latency 808.009 ms 2023-09-13T03:21:54.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:21:54 smithi142 ceph-mon[134082]: pgmap v269: 97 pgs: 97 active+clean; 89 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 561 KiB/s wr, 85 op/s 2023-09-13T03:21:55.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:21:54 smithi170 ceph-mon[134328]: pgmap v269: 97 pgs: 97 active+clean; 89 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 561 KiB/s wr, 85 op/s 2023-09-13T03:21:55.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:21:54 smithi064 ceph-mon[97675]: pgmap v269: 97 pgs: 97 active+clean; 89 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 561 KiB/s wr, 85 op/s 2023-09-13T03:21:55.935 INFO:tasks.workunit.client.0.smithi064.stdout: 1 183886 1.95 MB/sec execute 360 sec latency 1225.038 ms 2023-09-13T03:21:56.935 INFO:tasks.workunit.client.0.smithi064.stdout: 1 185122 1.96 MB/sec execute 361 sec latency 24.872 ms 2023-09-13T03:21:56.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:21:56 smithi142 ceph-mon[134082]: pgmap v270: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 699 KiB/s wr, 111 op/s 2023-09-13T03:21:57.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:21:56 smithi170 ceph-mon[134328]: pgmap v270: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 699 KiB/s wr, 111 op/s 2023-09-13T03:21:57.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:21:56 smithi064 ceph-mon[97675]: pgmap v270: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 699 KiB/s wr, 111 op/s 2023-09-13T03:21:57.935 INFO:tasks.workunit.client.0.smithi064.stdout: 1 185178 1.95 MB/sec execute 362 sec latency 962.805 ms 2023-09-13T03:21:58.935 INFO:tasks.workunit.client.0.smithi064.stdout: 1 185640 1.95 MB/sec execute 363 sec latency 1556.901 ms 2023-09-13T03:21:58.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:21:58 smithi142 ceph-mon[134082]: pgmap v271: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 376 KiB/s wr, 64 op/s 2023-09-13T03:21:59.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:21:58 smithi170 ceph-mon[134328]: pgmap v271: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 376 KiB/s wr, 64 op/s 2023-09-13T03:21:59.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:21:58 smithi064 ceph-mon[97675]: pgmap v271: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 376 KiB/s wr, 64 op/s 2023-09-13T03:21:59.935 INFO:tasks.workunit.client.0.smithi064.stdout: 1 185908 1.96 MB/sec execute 364 sec latency 865.514 ms 2023-09-13T03:21:59.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:21:59 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:22:00.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:21:59 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:22:00.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:21:59 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:22:00.935 INFO:tasks.workunit.client.0.smithi064.stdout: 1 186148 1.95 MB/sec execute 365 sec latency 1284.634 ms 2023-09-13T03:22:00.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:22:00 smithi142 ceph-mon[134082]: pgmap v272: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 376 KiB/s wr, 64 op/s 2023-09-13T03:22:01.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:22:00 smithi170 ceph-mon[134328]: pgmap v272: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 376 KiB/s wr, 64 op/s 2023-09-13T03:22:01.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:22:00 smithi064 ceph-mon[97675]: pgmap v272: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 376 KiB/s wr, 64 op/s 2023-09-13T03:22:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.612+0000 7efe215ff700 1 -- 172.21.15.64:0/2869575308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe1c106670 msgr2=0x7efe1c106a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.612+0000 7efe215ff700 1 --2- 172.21.15.64:0/2869575308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe1c106670 0x7efe1c106a50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7efe0c0099c0 tx=0x7efe0c009cd0 comp rx=0 tx=0).stop 2023-09-13T03:22:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.612+0000 7efe215ff700 1 -- 172.21.15.64:0/2869575308 shutdown_connections 2023-09-13T03:22:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.612+0000 7efe215ff700 1 --2- 172.21.15.64:0/2869575308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe1c106670 0x7efe1c106a50 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.612+0000 7efe215ff700 1 --2- 172.21.15.64:0/2869575308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe1c100f70 0x7efe1c10d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.612+0000 7efe215ff700 1 --2- 172.21.15.64:0/2869575308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe1c1005b0 0x7efe1c100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.612+0000 7efe215ff700 1 -- 172.21.15.64:0/2869575308 >> 172.21.15.64:0/2869575308 conn(0x7efe1c0fc060 msgr2=0x7efe1c0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:22:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.612+0000 7efe215ff700 1 -- 172.21.15.64:0/2869575308 shutdown_connections 2023-09-13T03:22:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.612+0000 7efe215ff700 1 -- 172.21.15.64:0/2869575308 wait complete. 2023-09-13T03:22:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.613+0000 7efe215ff700 1 Processor -- start 2023-09-13T03:22:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.614+0000 7efe215ff700 1 -- start start 2023-09-13T03:22:01.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.614+0000 7efe215ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe1c1005b0 0x7efe1c072ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:01.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.614+0000 7efe215ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe1c100f70 0x7efe1c073020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:01.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.614+0000 7efe215ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe1c106670 0x7efe1c1a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:01.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.614+0000 7efe215ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1c1a4d50 con 0x7efe1c106670 2023-09-13T03:22:01.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.614+0000 7efe215ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1c1a4e90 con 0x7efe1c1005b0 2023-09-13T03:22:01.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.614+0000 7efe215ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1c1a5000 con 0x7efe1c100f70 2023-09-13T03:22:01.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.614+0000 7efe1affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe1c1005b0 0x7efe1c072ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:01.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.614+0000 7efe1b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe1c106670 0x7efe1c1a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:01.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.614+0000 7efe1affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe1c1005b0 0x7efe1c072ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:55228/0 (socket says 172.21.15.64:55228) 2023-09-13T03:22:01.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.614+0000 7efe1affd700 1 -- 172.21.15.64:0/208279820 learned_addr learned my addr 172.21.15.64:0/208279820 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:22:01.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.614+0000 7efe1a7fc700 1 --2- 172.21.15.64:0/208279820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe1c100f70 0x7efe1c073020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:01.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.615+0000 7efe1affd700 1 -- 172.21.15.64:0/208279820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe1c100f70 msgr2=0x7efe1c073020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:01.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.615+0000 7efe1affd700 1 --2- 172.21.15.64:0/208279820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe1c100f70 0x7efe1c073020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:01.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.615+0000 7efe1affd700 1 -- 172.21.15.64:0/208279820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe1c106670 msgr2=0x7efe1c1a4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:01.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.615+0000 7efe1affd700 1 --2- 172.21.15.64:0/208279820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe1c106670 0x7efe1c1a4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:01.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.615+0000 7efe1affd700 1 -- 172.21.15.64:0/208279820 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe0c0095f0 con 0x7efe1c1005b0 2023-09-13T03:22:01.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.615+0000 7efe1affd700 1 --2- 172.21.15.64:0/208279820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe1c1005b0 0x7efe1c072ae0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7efe0c00b580 tx=0x7efe0c00bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:22:01.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.615+0000 7efe03fff700 1 -- 172.21.15.64:0/208279820 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0c01d070 con 0x7efe1c1005b0 2023-09-13T03:22:01.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.615+0000 7efe03fff700 1 -- 172.21.15.64:0/208279820 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe0c005050 con 0x7efe1c1005b0 2023-09-13T03:22:01.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.615+0000 7efe215ff700 1 -- 172.21.15.64:0/208279820 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe1c1a5280 con 0x7efe1c1005b0 2023-09-13T03:22:01.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.615+0000 7efe03fff700 1 -- 172.21.15.64:0/208279820 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0c004a40 con 0x7efe1c1005b0 2023-09-13T03:22:01.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.616+0000 7efe215ff700 1 -- 172.21.15.64:0/208279820 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe1c1a56a0 con 0x7efe1c1005b0 2023-09-13T03:22:01.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.619+0000 7efe03fff700 1 -- 172.21.15.64:0/208279820 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7efe0c0051c0 con 0x7efe1c1005b0 2023-09-13T03:22:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.620+0000 7efe3500db80 1 Processor -- start 2023-09-13T03:22:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.620+0000 7efe3500db80 1 -- start start 2023-09-13T03:22:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.620+0000 7efe3500db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56313e5371a0 0x56313e537580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.620+0000 7efe03fff700 1 --2- 172.21.15.64:0/208279820 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7efe040a07b0 0x7efe040a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.620+0000 7efe3500db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56313e531300 0x56313e531780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.620+0000 7efe3500db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56313e531cc0 0x56313e53df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.620+0000 7efe3500db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56313e48a6d0 con 0x56313e531300 2023-09-13T03:22:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.620+0000 7efe3500db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56313e487ff0 con 0x56313e5371a0 2023-09-13T03:22:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.620+0000 7efe3500db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56313e4838e0 con 0x56313e531cc0 2023-09-13T03:22:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.621+0000 7efe03fff700 1 -- 172.21.15.64:0/208279820 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8484+0+0 (secure 0 0 0) 0x7efe0c0cccb0 con 0x7efe1c1005b0 2023-09-13T03:22:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.621+0000 7efe1affd700 1 --2- 172.21.15.64:0/208279820 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7efe040a07b0 0x7efe040a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.621+0000 7efe1a7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56313e531300 0x56313e531780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.621+0000 7efe1b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56313e5371a0 0x56313e537580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.621+0000 7efe1a7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56313e531300 0x56313e531780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58390/0 (socket says 172.21.15.64:58390) 2023-09-13T03:22:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.621+0000 7efe1a7fc700 1 -- 172.21.15.64:0/2230084912 learned_addr learned my addr 172.21.15.64:0/2230084912 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:22:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.621+0000 7efe1b7fe700 1 --2- 172.21.15.64:0/2230084912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56313e531cc0 0x56313e53df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.621+0000 7efe1a7fc700 1 -- 172.21.15.64:0/2230084912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56313e531cc0 msgr2=0x56313e53df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:22:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.621+0000 7efe1a7fc700 1 --2- 172.21.15.64:0/2230084912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56313e531cc0 0x56313e53df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.621+0000 7efe1a7fc700 1 -- 172.21.15.64:0/2230084912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56313e5371a0 msgr2=0x56313e537580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:22:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.621+0000 7efe1a7fc700 1 --2- 172.21.15.64:0/2230084912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56313e5371a0 0x56313e537580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.621+0000 7efe1a7fc700 1 -- 172.21.15.64:0/2230084912 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56313e2afd80 con 0x56313e531300 2023-09-13T03:22:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.621+0000 7efe1b7fe700 1 --2- 172.21.15.64:0/2230084912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56313e5371a0 0x56313e537580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:22:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.621+0000 7efe1affd700 1 --2- 172.21.15.64:0/208279820 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7efe040a07b0 0x7efe040a2c70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efe0c096050 tx=0x7efe0c009cd0 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:22:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.622+0000 7efe1a7fc700 1 --2- 172.21.15.64:0/2230084912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56313e531300 0x56313e531780 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7efe10009d20 tx=0x7efe10011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d7ebdd8e1e47a6 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:22:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.622+0000 7efe017fa700 1 -- 172.21.15.64:0/2230084912 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe10019630 con 0x56313e531300 2023-09-13T03:22:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.622+0000 7efe017fa700 1 -- 172.21.15.64:0/2230084912 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe10019c50 con 0x56313e531300 2023-09-13T03:22:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.622+0000 7efe017fa700 1 -- 172.21.15.64:0/2230084912 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1000eb70 con 0x56313e531300 2023-09-13T03:22:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.623+0000 7efe3500db80 1 -- 172.21.15.64:0/2230084912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56313e531300 msgr2=0x56313e531780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.623+0000 7efe3500db80 1 --2- 172.21.15.64:0/2230084912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56313e531300 0x56313e531780 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7efe10009d20 tx=0x7efe10011720 comp rx=0 tx=0).stop 2023-09-13T03:22:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.623+0000 7efe3500db80 1 -- 172.21.15.64:0/2230084912 shutdown_connections 2023-09-13T03:22:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.623+0000 7efe3500db80 1 --2- 172.21.15.64:0/2230084912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56313e5371a0 0x56313e537580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.623+0000 7efe3500db80 1 --2- 172.21.15.64:0/2230084912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56313e531cc0 0x56313e53df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.623+0000 7efe3500db80 1 --2- 172.21.15.64:0/2230084912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56313e531300 0x56313e531780 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.623+0000 7efe3500db80 1 -- 172.21.15.64:0/2230084912 >> 172.21.15.64:0/2230084912 conn(0x56313e4a6030 msgr2=0x56313e52c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:22:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.623+0000 7efe3500db80 1 -- 172.21.15.64:0/2230084912 shutdown_connections 2023-09-13T03:22:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.623+0000 7efe3500db80 1 -- 172.21.15.64:0/2230084912 wait complete. 2023-09-13T03:22:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.623+0000 7efe3500db80 1 -- start start 2023-09-13T03:22:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.624+0000 7efe3500db80 1 Processor -- start 2023-09-13T03:22:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.624+0000 7efe3500db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56313e531300 0x56313e53ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.624+0000 7efe3500db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56313e531cc0 0x56313e53b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.624+0000 7efe3500db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56313e5371a0 0x56313e53b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.624+0000 7efe3500db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56313e421b10 con 0x56313e531300 2023-09-13T03:22:01.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.624+0000 7efe3500db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56313e421810 con 0x56313e5371a0 2023-09-13T03:22:01.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.624+0000 7efe3500db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56313e3058c0 con 0x56313e531cc0 2023-09-13T03:22:01.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.624+0000 7efe3500db80 10 client.0 _ll_register_callbacks cb 0x56313e36ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:22:01.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.624+0000 7efe1b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56313e531300 0x56313e53ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:01.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.624+0000 7efe1a7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56313e531cc0 0x56313e53b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:01.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.624+0000 7efe1b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56313e531300 0x56313e53ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58402/0 (socket says 172.21.15.64:58402) 2023-09-13T03:22:01.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.624+0000 7efe1b7fe700 1 -- 172.21.15.64:0/4075900569 learned_addr learned my addr 172.21.15.64:0/4075900569 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:22:01.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.625+0000 7efe1b7fe700 1 --2- 172.21.15.64:0/4075900569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56313e5371a0 0x56313e53b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:01.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.625+0000 7efe1a7fc700 1 -- 172.21.15.64:0/4075900569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56313e5371a0 msgr2=0x56313e53b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:22:01.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.625+0000 7efe1a7fc700 1 --2- 172.21.15.64:0/4075900569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56313e5371a0 0x56313e53b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:01.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.625+0000 7efe1a7fc700 1 -- 172.21.15.64:0/4075900569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56313e531300 msgr2=0x56313e53ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:22:01.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.625+0000 7efe1a7fc700 1 --2- 172.21.15.64:0/4075900569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56313e531300 0x56313e53ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:01.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.625+0000 7efe1a7fc700 1 -- 172.21.15.64:0/4075900569 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe14008850 con 0x56313e531cc0 2023-09-13T03:22:01.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.625+0000 7efe1a7fc700 1 --2- 172.21.15.64:0/4075900569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56313e531cc0 0x56313e53b080 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7efe10009d20 tx=0x7efe10019a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=509a92216576bb76 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:22:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.625+0000 7efdfeff5700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:22:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.625+0000 7efdfeff5700 1 -- 172.21.15.64:0/4075900569 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe10004230 con 0x56313e531cc0 2023-09-13T03:22:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.625+0000 7efe3500db80 1 -- 172.21.15.64:0/4075900569 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56313e3db490 con 0x56313e531cc0 2023-09-13T03:22:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.625+0000 7efdfeff5700 1 -- 172.21.15.64:0/4075900569 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8484+0+0 (secure 0 0 0) 0x7efe10019dc0 con 0x56313e531cc0 2023-09-13T03:22:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.626+0000 7efdfeff5700 1 -- 172.21.15.64:0/4075900569 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe10005ce0 con 0x56313e531cc0 2023-09-13T03:22:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.626+0000 7efe3500db80 10 client.26824 fetch_fsmap learned FSMap version 55 2023-09-13T03:22:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.626+0000 7efe3500db80 1 -- 172.21.15.64:0/4075900569 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x7efe10009980 con 0x56313e531cc0 2023-09-13T03:22:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.626+0000 7efdfeff5700 1 -- 172.21.15.64:0/4075900569 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56313e3db490 con 0x56313e531cc0 2023-09-13T03:22:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.626+0000 7efdfeff5700 1 -- 172.21.15.64:0/4075900569 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 55) v1 ==== 7898+0+0 (secure 0 0 0) 0x7efe10004890 con 0x56313e531cc0 2023-09-13T03:22:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.626+0000 7efe3500db80 10 client.26824 fetch_fsmap finished waiting for FSMap version 55 2023-09-13T03:22:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.626+0000 7efe3500db80 10 client.26824 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:22:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.626+0000 7efe3500db80 20 client.26824 populate_metadata read hostname 'smithi064' 2023-09-13T03:22:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.626+0000 7efe3500db80 1 --2- 172.21.15.64:0/4075900569 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56313e54b2d0 0x56313e54d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.626+0000 7efe3500db80 4 client.26824 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:22:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.626+0000 7efe3500db80 1 -- 172.21.15.64:0/4075900569 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56313e498840 con 0x56313e54b2d0 2023-09-13T03:22:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.627+0000 7efe1b7fe700 1 --2- 172.21.15.64:0/4075900569 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56313e54b2d0 0x56313e54d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.627+0000 7efe1b7fe700 1 --2- 172.21.15.64:0/4075900569 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56313e54b2d0 0x56313e54d6f0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d214914f4053e9a server_cookie=fbc9f96223035262 in_seq=0 out_seq=0 2023-09-13T03:22:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.627+0000 7efdfeff5700 10 client.26824 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:22:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.628+0000 7efdfeff5700 1 -- 172.21.15.64:0/4075900569 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x7efe14026730 con 0x56313e54b2d0 2023-09-13T03:22:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.628+0000 7efdfeff5700 10 client.26824 handle_command_reply: tid=0 2023-09-13T03:22:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.628+0000 7efe3500db80 1 client.26824 shutdown 2023-09-13T03:22:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.628+0000 7efe3500db80 1 -- 172.21.15.64:0/4075900569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56313e531cc0 msgr2=0x56313e53b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.628+0000 7efe3500db80 1 --2- 172.21.15.64:0/4075900569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56313e531cc0 0x56313e53b080 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7efe10009d20 tx=0x7efe10019a30 comp rx=0 tx=0).stop 2023-09-13T03:22:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.628+0000 7efe3500db80 1 -- 172.21.15.64:0/4075900569 shutdown_connections 2023-09-13T03:22:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.628+0000 7efe3500db80 1 --2- 172.21.15.64:0/4075900569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56313e5371a0 0x56313e53b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:01.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.628+0000 7efe3500db80 1 --2- 172.21.15.64:0/4075900569 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56313e54b2d0 0x56313e54d6f0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:01.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.628+0000 7efe3500db80 1 --2- 172.21.15.64:0/4075900569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56313e531300 0x56313e53ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:01.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.628+0000 7efe3500db80 1 --2- 172.21.15.64:0/4075900569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56313e531cc0 0x56313e53b080 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:01.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.628+0000 7efdfeff5700 0 client.26824 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:22:01.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.628+0000 7efe3500db80 1 -- 172.21.15.64:0/4075900569 >> 172.21.15.64:0/4075900569 conn(0x56313e4a6030 msgr2=0x56313e52b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:22:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.628+0000 7efe3500db80 1 -- 172.21.15.64:0/4075900569 shutdown_connections 2023-09-13T03:22:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.628+0000 7efe3500db80 1 -- 172.21.15.64:0/4075900569 wait complete. 2023-09-13T03:22:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.629+0000 7efe3500db80 20 client.26824 trim_cache size 0 max 16384 2023-09-13T03:22:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.644+0000 7efe3500db80 1 Processor -- start 2023-09-13T03:22:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.644+0000 7efe3500db80 1 -- start start 2023-09-13T03:22:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.644+0000 7efe3500db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56313e531300 0x56313e542520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.644+0000 7efe3500db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56313e531cc0 0x56313e533b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.644+0000 7efe3500db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56313e5371a0 0x56313e53d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.644+0000 7efe3500db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56313e542a60 con 0x56313e531cc0 2023-09-13T03:22:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.644+0000 7efe3500db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56313e547cc0 con 0x56313e5371a0 2023-09-13T03:22:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.644+0000 7efe3500db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56313e535c70 con 0x56313e531300 2023-09-13T03:22:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.645+0000 7efe1a7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56313e531cc0 0x56313e533b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.645+0000 7efe1a7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56313e531cc0 0x56313e533b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58404/0 (socket says 172.21.15.64:58404) 2023-09-13T03:22:01.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.645+0000 7efe1a7fc700 1 -- 172.21.15.64:0/3402642970 learned_addr learned my addr 172.21.15.64:0/3402642970 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:22:01.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.645+0000 7efe1b7fe700 1 --2- 172.21.15.64:0/3402642970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56313e531300 0x56313e542520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:01.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.645+0000 7efe1b7fe700 1 --2- 172.21.15.64:0/3402642970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56313e5371a0 0x56313e53d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:01.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.645+0000 7efe1a7fc700 1 -- 172.21.15.64:0/3402642970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56313e531300 msgr2=0x56313e542520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:22:01.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.645+0000 7efe1a7fc700 1 --2- 172.21.15.64:0/3402642970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56313e531300 0x56313e542520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:01.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.645+0000 7efe1a7fc700 1 -- 172.21.15.64:0/3402642970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56313e5371a0 msgr2=0x56313e53d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:22:01.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.645+0000 7efe1a7fc700 1 --2- 172.21.15.64:0/3402642970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56313e5371a0 0x56313e53d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:01.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.645+0000 7efe1a7fc700 1 -- 172.21.15.64:0/3402642970 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56313e298c00 con 0x56313e531cc0 2023-09-13T03:22:01.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.645+0000 7efe1b7fe700 1 --2- 172.21.15.64:0/3402642970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56313e5371a0 0x56313e53d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T03:22:01.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.645+0000 7efe1a7fc700 1 --2- 172.21.15.64:0/3402642970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56313e531cc0 0x56313e533b60 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7efe1001c040 tx=0x7efe10005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5f9d3b711913376 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:22:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.645+0000 7efe01ffb700 1 -- 172.21.15.64:0/3402642970 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1000ea90 con 0x56313e531cc0 2023-09-13T03:22:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.645+0000 7efe01ffb700 1 -- 172.21.15.64:0/3402642970 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe10019aa0 con 0x56313e531cc0 2023-09-13T03:22:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.645+0000 7efe01ffb700 1 -- 172.21.15.64:0/3402642970 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1000ee50 con 0x56313e531cc0 2023-09-13T03:22:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.646+0000 7efe3500db80 1 -- 172.21.15.64:0/3402642970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56313e531cc0 msgr2=0x56313e533b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.646+0000 7efe3500db80 1 --2- 172.21.15.64:0/3402642970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56313e531cc0 0x56313e533b60 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7efe1001c040 tx=0x7efe10005a00 comp rx=0 tx=0).stop 2023-09-13T03:22:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.646+0000 7efe3500db80 1 -- 172.21.15.64:0/3402642970 shutdown_connections 2023-09-13T03:22:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.646+0000 7efe3500db80 1 --2- 172.21.15.64:0/3402642970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56313e5371a0 0x56313e53d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.646+0000 7efe3500db80 1 --2- 172.21.15.64:0/3402642970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56313e531300 0x56313e542520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.646+0000 7efe3500db80 1 --2- 172.21.15.64:0/3402642970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56313e531cc0 0x56313e533b60 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.646+0000 7efe3500db80 1 -- 172.21.15.64:0/3402642970 >> 172.21.15.64:0/3402642970 conn(0x56313e4a6030 msgr2=0x56313e536080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:22:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.646+0000 7efe3500db80 1 -- 172.21.15.64:0/3402642970 shutdown_connections 2023-09-13T03:22:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.646+0000 7efe3500db80 1 -- 172.21.15.64:0/3402642970 wait complete. 2023-09-13T03:22:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.646+0000 7efe3500db80 1 -- start start 2023-09-13T03:22:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.647+0000 7efe3500db80 1 Processor -- start 2023-09-13T03:22:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.647+0000 7efe3500db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56313e531300 0x56313e52b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.647+0000 7efe3500db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56313e531cc0 0x56313e52b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:01.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.647+0000 7efe3500db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56313e5371a0 0x56313e52bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:01.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.647+0000 7efe3500db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56313e59f0c0 con 0x56313e531cc0 2023-09-13T03:22:01.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.647+0000 7efe3500db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56313e59f200 con 0x56313e531300 2023-09-13T03:22:01.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.647+0000 7efe3500db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56313e59f340 con 0x56313e5371a0 2023-09-13T03:22:01.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.647+0000 7efe3500db80 10 client.0 _ll_register_callbacks cb 0x7efe140091e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:22:01.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.647+0000 7efe1a7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56313e531cc0 0x56313e52b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:01.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.647+0000 7efe1a7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56313e531cc0 0x56313e52b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58418/0 (socket says 172.21.15.64:58418) 2023-09-13T03:22:01.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.647+0000 7efe1a7fc700 1 -- 172.21.15.64:0/1372442830 learned_addr learned my addr 172.21.15.64:0/1372442830 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:22:01.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.648+0000 7efe1b7fe700 1 --2- 172.21.15.64:0/1372442830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56313e531300 0x56313e52b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:01.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.648+0000 7efe1b7fe700 1 --2- 172.21.15.64:0/1372442830 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56313e5371a0 0x56313e52bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.648+0000 7efe1a7fc700 1 -- 172.21.15.64:0/1372442830 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56313e5371a0 msgr2=0x56313e52bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:22:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.648+0000 7efe1a7fc700 1 --2- 172.21.15.64:0/1372442830 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56313e5371a0 0x56313e52bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.648+0000 7efe1a7fc700 1 -- 172.21.15.64:0/1372442830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56313e531300 msgr2=0x56313e52b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:22:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.648+0000 7efe1a7fc700 1 --2- 172.21.15.64:0/1372442830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56313e531300 0x56313e52b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.648+0000 7efe1a7fc700 1 -- 172.21.15.64:0/1372442830 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe10009980 con 0x56313e531cc0 2023-09-13T03:22:01.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.648+0000 7efe1b7fe700 1 --2- 172.21.15.64:0/1372442830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56313e531300 0x56313e52b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:22:01.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.648+0000 7efe1a7fc700 1 --2- 172.21.15.64:0/1372442830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56313e531cc0 0x56313e52b8a0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7efe1000e7e0 tx=0x7efe10004df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbe3cb2e00d4a6ed server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:22:01.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.648+0000 7efdfeff5700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:22:01.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.648+0000 7efdfeff5700 1 -- 172.21.15.64:0/1372442830 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1000eca0 con 0x56313e531cc0 2023-09-13T03:22:01.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.648+0000 7efe3500db80 1 -- 172.21.15.64:0/1372442830 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efe10019c50 con 0x56313e531cc0 2023-09-13T03:22:01.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.648+0000 7efdfeff5700 1 -- 172.21.15.64:0/1372442830 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8484+0+0 (secure 0 0 0) 0x7efe10000e40 con 0x56313e531cc0 2023-09-13T03:22:01.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.649+0000 7efdfeff5700 1 -- 172.21.15.64:0/1372442830 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe1000ee00 con 0x56313e531cc0 2023-09-13T03:22:01.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.649+0000 7efe3500db80 10 client.16809 fetch_fsmap learned FSMap version 55 2023-09-13T03:22:01.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.649+0000 7efe3500db80 1 -- 172.21.15.64:0/1372442830 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x56313e2d61f0 con 0x56313e531cc0 2023-09-13T03:22:01.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.649+0000 7efdfeff5700 1 -- 172.21.15.64:0/1372442830 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe10041020 con 0x56313e531cc0 2023-09-13T03:22:01.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.649+0000 7efdfeff5700 1 -- 172.21.15.64:0/1372442830 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 55) v1 ==== 7898+0+0 (secure 0 0 0) 0x7efe100226d0 con 0x56313e531cc0 2023-09-13T03:22:01.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.649+0000 7efe3500db80 10 client.16809 fetch_fsmap finished waiting for FSMap version 55 2023-09-13T03:22:01.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.649+0000 7efe3500db80 10 client.16809 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:22:01.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.649+0000 7efe3500db80 20 client.16809 populate_metadata read hostname 'smithi064' 2023-09-13T03:22:01.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.649+0000 7efe3500db80 1 --2- 172.21.15.64:0/1372442830 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56313e5a05c0 0x56313e5a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:01.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.649+0000 7efe3500db80 4 client.16809 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:22:01.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.649+0000 7efe3500db80 1 -- 172.21.15.64:0/1372442830 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56313e47c8c0 con 0x56313e5a05c0 2023-09-13T03:22:01.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.649+0000 7efe1b7fe700 1 --2- 172.21.15.64:0/1372442830 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56313e5a05c0 0x56313e5a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:01.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.650+0000 7efe1b7fe700 1 --2- 172.21.15.64:0/1372442830 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56313e5a05c0 0x56313e5a29e0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=711f3f1f3056d83d server_cookie=e95f12f51c329b56 in_seq=0 out_seq=0 2023-09-13T03:22:01.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.650+0000 7efdfeff5700 10 client.16809 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:22:01.673 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.673+0000 7efdfeff5700 1 -- 172.21.15.64:0/1372442830 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56313e47c8c0 con 0x56313e5a05c0 2023-09-13T03:22:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.673+0000 7efdfeff5700 10 client.16809 handle_command_reply: tid=0 2023-09-13T03:22:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.673+0000 7efe3500db80 1 client.16809 shutdown 2023-09-13T03:22:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.673+0000 7efe3500db80 1 -- 172.21.15.64:0/1372442830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56313e531cc0 msgr2=0x56313e52b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.673+0000 7efe3500db80 1 --2- 172.21.15.64:0/1372442830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56313e531cc0 0x56313e52b8a0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7efe1000e7e0 tx=0x7efe10004df0 comp rx=0 tx=0).stop 2023-09-13T03:22:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.673+0000 7efe3500db80 1 -- 172.21.15.64:0/1372442830 shutdown_connections 2023-09-13T03:22:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.673+0000 7efe3500db80 1 --2- 172.21.15.64:0/1372442830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56313e531300 0x56313e52b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.673+0000 7efe3500db80 1 --2- 172.21.15.64:0/1372442830 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56313e5a05c0 0x56313e5a29e0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.673+0000 7efe3500db80 1 --2- 172.21.15.64:0/1372442830 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56313e5371a0 0x56313e52bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.673+0000 7efe3500db80 1 --2- 172.21.15.64:0/1372442830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56313e531cc0 0x56313e52b8a0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.673+0000 7efdfeff5700 0 client.16809 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:22:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.673+0000 7efe3500db80 1 -- 172.21.15.64:0/1372442830 >> 172.21.15.64:0/1372442830 conn(0x56313e4a6030 msgr2=0x56313e5302f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:22:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.673+0000 7efe3500db80 1 -- 172.21.15.64:0/1372442830 shutdown_connections 2023-09-13T03:22:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.673+0000 7efe3500db80 1 -- 172.21.15.64:0/1372442830 wait complete. 2023-09-13T03:22:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.673+0000 7efe3500db80 20 client.16809 trim_cache size 0 max 16384 2023-09-13T03:22:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:22:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:22:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:22:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:22:01.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.673+0000 7efe215ff700 1 -- 172.21.15.64:0/208279820 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7efe040a07b0 msgr2=0x7efe040a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:01.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.673+0000 7efe215ff700 1 --2- 172.21.15.64:0/208279820 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7efe040a07b0 0x7efe040a2c70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efe0c096050 tx=0x7efe0c009cd0 comp rx=0 tx=0).stop 2023-09-13T03:22:01.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.673+0000 7efe215ff700 1 -- 172.21.15.64:0/208279820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe1c1005b0 msgr2=0x7efe1c072ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:01.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.673+0000 7efe215ff700 1 --2- 172.21.15.64:0/208279820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe1c1005b0 0x7efe1c072ae0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7efe0c00b580 tx=0x7efe0c00bfd0 comp rx=0 tx=0).stop 2023-09-13T03:22:01.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.674+0000 7efe215ff700 1 -- 172.21.15.64:0/208279820 shutdown_connections 2023-09-13T03:22:01.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.674+0000 7efe215ff700 1 --2- 172.21.15.64:0/208279820 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7efe040a07b0 0x7efe040a2c70 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:01.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.674+0000 7efe215ff700 1 --2- 172.21.15.64:0/208279820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe1c1005b0 0x7efe1c072ae0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:01.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.674+0000 7efe215ff700 1 --2- 172.21.15.64:0/208279820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe1c106670 0x7efe1c1a4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:01.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.674+0000 7efe215ff700 1 --2- 172.21.15.64:0/208279820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe1c100f70 0x7efe1c073020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:01.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.674+0000 7efe215ff700 1 -- 172.21.15.64:0/208279820 >> 172.21.15.64:0/208279820 conn(0x7efe1c0fc060 msgr2=0x7efe1c104240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:22:01.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.674+0000 7efe215ff700 1 -- 172.21.15.64:0/208279820 shutdown_connections 2023-09-13T03:22:01.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:01.674+0000 7efe215ff700 1 -- 172.21.15.64:0/208279820 wait complete. 2023-09-13T03:22:01.935 INFO:tasks.workunit.client.0.smithi064.stdout: 1 186148 1.95 MB/sec execute 366 sec latency 1050.637 ms 2023-09-13T03:22:02.935 INFO:tasks.workunit.client.0.smithi064.stdout: 1 187830 1.97 MB/sec execute 367 sec latency 1215.510 ms 2023-09-13T03:22:02.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:22:02 smithi142 ceph-mon[134082]: pgmap v273: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 661 KiB/s wr, 104 op/s 2023-09-13T03:22:03.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:22:02 smithi170 ceph-mon[134328]: pgmap v273: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 661 KiB/s wr, 104 op/s 2023-09-13T03:22:03.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:22:02 smithi064 ceph-mon[97675]: pgmap v273: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 661 KiB/s wr, 104 op/s 2023-09-13T03:22:03.935 INFO:tasks.workunit.client.0.smithi064.stdout: 1 188918 1.98 MB/sec execute 368 sec latency 22.641 ms 2023-09-13T03:22:04.936 INFO:tasks.workunit.client.0.smithi064.stdout: 1 189698 1.98 MB/sec execute 369 sec latency 336.845 ms 2023-09-13T03:22:04.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:22:04 smithi142 ceph-mon[134082]: pgmap v274: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 422 KiB/s wr, 66 op/s 2023-09-13T03:22:05.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:22:04 smithi170 ceph-mon[134328]: pgmap v274: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 422 KiB/s wr, 66 op/s 2023-09-13T03:22:05.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:22:04 smithi064 ceph-mon[97675]: pgmap v274: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 422 KiB/s wr, 66 op/s 2023-09-13T03:22:05.936 INFO:tasks.workunit.client.0.smithi064.stdout: 1 190937 2.00 MB/sec execute 370 sec latency 753.341 ms 2023-09-13T03:22:06.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:22:05 smithi170 ceph-mon[134328]: osdmap e91: 12 total, 12 up, 12 in 2023-09-13T03:22:06.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:22:05 smithi064 ceph-mon[97675]: osdmap e91: 12 total, 12 up, 12 in 2023-09-13T03:22:06.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:22:05 smithi142 ceph-mon[134082]: osdmap e91: 12 total, 12 up, 12 in 2023-09-13T03:22:06.936 INFO:tasks.workunit.client.0.smithi064.stdout: 1 192213 2.01 MB/sec execute 371 sec latency 19.393 ms 2023-09-13T03:22:07.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:22:06 smithi170 ceph-mon[134328]: pgmap v276: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 191 op/s 2023-09-13T03:22:07.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:22:06 smithi064 ceph-mon[97675]: pgmap v276: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 191 op/s 2023-09-13T03:22:07.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:22:06 smithi142 ceph-mon[134082]: pgmap v276: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 191 op/s 2023-09-13T03:22:07.937 INFO:tasks.workunit.client.0.smithi064.stdout: 1 192213 2.00 MB/sec execute 372 sec latency 1017.607 ms 2023-09-13T03:22:08.936 INFO:tasks.workunit.client.0.smithi064.stdout: 1 192798 2.00 MB/sec execute 373 sec latency 1613.185 ms 2023-09-13T03:22:09.159 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:22:08 smithi170 ceph-mon[134328]: pgmap v277: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 191 op/s 2023-09-13T03:22:09.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:22:08 smithi064 ceph-mon[97675]: pgmap v277: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 191 op/s 2023-09-13T03:22:09.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:22:08 smithi142 ceph-mon[134082]: pgmap v277: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 191 op/s 2023-09-13T03:22:09.936 INFO:tasks.workunit.client.0.smithi064.stdout: 1 193043 2.01 MB/sec execute 374 sec latency 874.774 ms 2023-09-13T03:22:10.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:22:10 smithi170 ceph-mon[134328]: osdmap e92: 12 total, 12 up, 12 in 2023-09-13T03:22:10.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:22:10 smithi170 ceph-mon[134328]: pgmap v279: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 178 op/s 2023-09-13T03:22:10.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:22:10 smithi064 ceph-mon[97675]: osdmap e92: 12 total, 12 up, 12 in 2023-09-13T03:22:10.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:22:10 smithi064 ceph-mon[97675]: pgmap v279: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 178 op/s 2023-09-13T03:22:10.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:22:10 smithi142 ceph-mon[134082]: osdmap e92: 12 total, 12 up, 12 in 2023-09-13T03:22:10.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:22:10 smithi142 ceph-mon[134082]: pgmap v279: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 178 op/s 2023-09-13T03:22:10.936 INFO:tasks.workunit.client.0.smithi064.stdout: 1 193043 2.00 MB/sec execute 375 sec latency 1874.891 ms 2023-09-13T03:22:11.936 INFO:tasks.workunit.client.0.smithi064.stdout: 1 193043 2.00 MB/sec execute 376 sec latency 2874.991 ms 2023-09-13T03:22:12.936 INFO:tasks.workunit.client.0.smithi064.stdout: 1 193043 1.99 MB/sec execute 377 sec latency 3875.121 ms 2023-09-13T03:22:12.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:22:12 smithi142 ceph-mon[134082]: pgmap v280: 97 pgs: 97 active+clean; 95 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 242 op/s 2023-09-13T03:22:13.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:22:12 smithi170 ceph-mon[134328]: pgmap v280: 97 pgs: 97 active+clean; 95 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 242 op/s 2023-09-13T03:22:13.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:22:12 smithi064 ceph-mon[97675]: pgmap v280: 97 pgs: 97 active+clean; 95 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 242 op/s 2023-09-13T03:22:13.936 INFO:tasks.workunit.client.0.smithi064.stdout: 1 193043 1.99 MB/sec execute 378 sec latency 4875.254 ms 2023-09-13T03:22:14.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.836+0000 7f53449ff700 1 -- 172.21.15.64:0/2727200039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53401005a0 msgr2=0x7f5340100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:14.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.836+0000 7f53449ff700 1 --2- 172.21.15.64:0/2727200039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53401005a0 0x7f5340100a20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f53280098f0 tx=0x7f5328009c00 comp rx=0 tx=0).stop 2023-09-13T03:22:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.836+0000 7f53449ff700 1 -- 172.21.15.64:0/2727200039 shutdown_connections 2023-09-13T03:22:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.836+0000 7f53449ff700 1 --2- 172.21.15.64:0/2727200039 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5340106660 0x7f5340106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.836+0000 7f53449ff700 1 --2- 172.21.15.64:0/2727200039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5340100f60 0x7f534010d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.836+0000 7f53449ff700 1 --2- 172.21.15.64:0/2727200039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53401005a0 0x7f5340100a20 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.836+0000 7f53449ff700 1 -- 172.21.15.64:0/2727200039 >> 172.21.15.64:0/2727200039 conn(0x7f53400fc050 msgr2=0x7f53400fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:22:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.837+0000 7f53449ff700 1 -- 172.21.15.64:0/2727200039 shutdown_connections 2023-09-13T03:22:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.837+0000 7f53449ff700 1 -- 172.21.15.64:0/2727200039 wait complete. 2023-09-13T03:22:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.837+0000 7f53449ff700 1 Processor -- start 2023-09-13T03:22:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.838+0000 7f53449ff700 1 -- start start 2023-09-13T03:22:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.838+0000 7f53449ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53401005a0 0x7f534019e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.838+0000 7f53449ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5340100f60 0x7f534019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.838+0000 7f53449ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5340106660 0x7f5340198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.838+0000 7f53449ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5340198ce0 con 0x7f5340100f60 2023-09-13T03:22:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.838+0000 7f53449ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5340198e50 con 0x7f53401005a0 2023-09-13T03:22:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.838+0000 7f53449ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5340199010 con 0x7f5340106660 2023-09-13T03:22:14.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.838+0000 7f5337fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5340100f60 0x7f534019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:14.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.838+0000 7f5337fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5340100f60 0x7f534019ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53744/0 (socket says 172.21.15.64:53744) 2023-09-13T03:22:14.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.838+0000 7f5337fff700 1 -- 172.21.15.64:0/2392424791 learned_addr learned my addr 172.21.15.64:0/2392424791 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:22:14.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.839+0000 7f533ed9d700 1 --2- 172.21.15.64:0/2392424791 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5340106660 0x7f5340198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:14.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.839+0000 7f533e59c700 1 --2- 172.21.15.64:0/2392424791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53401005a0 0x7f534019e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:14.843 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.839+0000 7f5337fff700 1 -- 172.21.15.64:0/2392424791 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5340106660 msgr2=0x7f5340198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:14.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.839+0000 7f5337fff700 1 --2- 172.21.15.64:0/2392424791 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5340106660 0x7f5340198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:14.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.839+0000 7f5337fff700 1 -- 172.21.15.64:0/2392424791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53401005a0 msgr2=0x7f534019e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:14.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.839+0000 7f5337fff700 1 --2- 172.21.15.64:0/2392424791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53401005a0 0x7f534019e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:14.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.839+0000 7f5337fff700 1 -- 172.21.15.64:0/2392424791 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5338009d00 con 0x7f5340100f60 2023-09-13T03:22:14.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.839+0000 7f5337fff700 1 --2- 172.21.15.64:0/2392424791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5340100f60 0x7f534019ed50 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f5328009810 tx=0x7f532800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:22:14.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.839+0000 7f53377fe700 1 -- 172.21.15.64:0/2392424791 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532801c070 con 0x7f5340100f60 2023-09-13T03:22:14.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.839+0000 7f53449ff700 1 -- 172.21.15.64:0/2392424791 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5328009520 con 0x7f5340100f60 2023-09-13T03:22:14.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.839+0000 7f53377fe700 1 -- 172.21.15.64:0/2392424791 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f532800bb20 con 0x7f5340100f60 2023-09-13T03:22:14.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.839+0000 7f53377fe700 1 -- 172.21.15.64:0/2392424791 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5328004930 con 0x7f5340100f60 2023-09-13T03:22:14.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.840+0000 7f53449ff700 1 -- 172.21.15.64:0/2392424791 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53401a5620 con 0x7f5340100f60 2023-09-13T03:22:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.843+0000 7f53377fe700 1 -- 172.21.15.64:0/2392424791 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f5328020070 con 0x7f5340100f60 2023-09-13T03:22:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.844+0000 7f53377fe700 1 --2- 172.21.15.64:0/2392424791 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f532c0a07b0 0x7f532c0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.844+0000 7f5358330b80 1 Processor -- start 2023-09-13T03:22:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.844+0000 7f53377fe700 1 -- 172.21.15.64:0/2392424791 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f53280ccc70 con 0x7f5340100f60 2023-09-13T03:22:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.844+0000 7f533e59c700 1 --2- 172.21.15.64:0/2392424791 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f532c0a07b0 0x7f532c0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.844+0000 7f5358330b80 1 -- start start 2023-09-13T03:22:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.845+0000 7f5358330b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615814141a0 0x561581414580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.845+0000 7f5358330b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56158140e300 0x56158140e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.845+0000 7f5358330b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56158140ecc0 0x56158141af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.845+0000 7f5358330b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615813676d0 con 0x56158140ecc0 2023-09-13T03:22:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.845+0000 7f5358330b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561581364ff0 con 0x5615814141a0 2023-09-13T03:22:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.845+0000 7f5358330b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615813608e0 con 0x56158140e300 2023-09-13T03:22:14.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.845+0000 7f533e59c700 1 --2- 172.21.15.64:0/2392424791 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f532c0a07b0 0x7f532c0a2c70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5330009c60 tx=0x7f5330009260 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:22:14.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.845+0000 7f533e59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615814141a0 0x561581414580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:14.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.845+0000 7f5337fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56158140e300 0x56158140e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:14.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.845+0000 7f533e59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615814141a0 0x561581414580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:41116/0 (socket says 172.21.15.64:41116) 2023-09-13T03:22:14.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.845+0000 7f533e59c700 1 -- 172.21.15.64:0/2303449278 learned_addr learned my addr 172.21.15.64:0/2303449278 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:22:14.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.845+0000 7f533ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56158140ecc0 0x56158141af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:14.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.846+0000 7f533e59c700 1 -- 172.21.15.64:0/2303449278 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56158140e300 msgr2=0x56158140e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:22:14.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.846+0000 7f533e59c700 1 --2- 172.21.15.64:0/2303449278 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56158140e300 0x56158140e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:14.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.846+0000 7f533e59c700 1 -- 172.21.15.64:0/2303449278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56158140ecc0 msgr2=0x56158141af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:22:14.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.846+0000 7f533e59c700 1 --2- 172.21.15.64:0/2303449278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56158140ecc0 0x56158141af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:14.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.846+0000 7f533e59c700 1 -- 172.21.15.64:0/2303449278 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56158118cd80 con 0x5615814141a0 2023-09-13T03:22:14.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.846+0000 7f5337fff700 1 --2- 172.21.15.64:0/2303449278 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56158140e300 0x56158140e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:22:14.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.846+0000 7f533ed9d700 1 --2- 172.21.15.64:0/2303449278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56158140ecc0 0x56158141af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:22:14.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.846+0000 7f533e59c700 1 --2- 172.21.15.64:0/2303449278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615814141a0 0x561581414580 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f533000e430 tx=0x7f5330014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdeb5be9f614b35a server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:22:14.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.846+0000 7f5334ff9700 1 -- 172.21.15.64:0/2303449278 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533000edd0 con 0x5615814141a0 2023-09-13T03:22:14.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.847+0000 7f5334ff9700 1 -- 172.21.15.64:0/2303449278 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5330021070 con 0x5615814141a0 2023-09-13T03:22:14.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.847+0000 7f5334ff9700 1 -- 172.21.15.64:0/2303449278 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53300256c0 con 0x5615814141a0 2023-09-13T03:22:14.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.847+0000 7f5358330b80 1 -- 172.21.15.64:0/2303449278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615814141a0 msgr2=0x561581414580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:14.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.847+0000 7f5358330b80 1 --2- 172.21.15.64:0/2303449278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615814141a0 0x561581414580 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f533000e430 tx=0x7f5330014000 comp rx=0 tx=0).stop 2023-09-13T03:22:14.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.847+0000 7f5358330b80 1 -- 172.21.15.64:0/2303449278 shutdown_connections 2023-09-13T03:22:14.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.847+0000 7f5358330b80 1 --2- 172.21.15.64:0/2303449278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5615814141a0 0x561581414580 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:14.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.847+0000 7f5358330b80 1 --2- 172.21.15.64:0/2303449278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56158140ecc0 0x56158141af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:14.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.847+0000 7f5358330b80 1 --2- 172.21.15.64:0/2303449278 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56158140e300 0x56158140e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:14.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.847+0000 7f5358330b80 1 -- 172.21.15.64:0/2303449278 >> 172.21.15.64:0/2303449278 conn(0x561581383030 msgr2=0x561581409560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:22:14.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.847+0000 7f5358330b80 1 -- 172.21.15.64:0/2303449278 shutdown_connections 2023-09-13T03:22:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.848+0000 7f5358330b80 1 -- 172.21.15.64:0/2303449278 wait complete. 2023-09-13T03:22:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.848+0000 7f5358330b80 1 -- start start 2023-09-13T03:22:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.848+0000 7f5358330b80 1 Processor -- start 2023-09-13T03:22:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.849+0000 7f5358330b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56158140e300 0x56158141ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:14.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.849+0000 7f5358330b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56158140ecc0 0x5615814180c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:14.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.849+0000 7f5358330b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615814141a0 0x561581418600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:14.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.849+0000 7f5358330b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615812feb10 con 0x5615814141a0 2023-09-13T03:22:14.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.849+0000 7f5358330b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615812fe810 con 0x56158140e300 2023-09-13T03:22:14.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.849+0000 7f5358330b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615811e28c0 con 0x56158140ecc0 2023-09-13T03:22:14.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.849+0000 7f5358330b80 10 client.0 _ll_register_callbacks cb 0x561581248a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:22:14.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.849+0000 7f533ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615814141a0 0x561581418600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:14.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.849+0000 7f533e59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56158140e300 0x56158141ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:14.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.849+0000 7f533ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615814141a0 0x561581418600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53748/0 (socket says 172.21.15.64:53748) 2023-09-13T03:22:14.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.849+0000 7f533ed9d700 1 -- 172.21.15.64:0/2806875901 learned_addr learned my addr 172.21.15.64:0/2806875901 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:22:14.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.849+0000 7f5337fff700 1 --2- 172.21.15.64:0/2806875901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56158140ecc0 0x5615814180c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:14.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.850+0000 7f533ed9d700 1 -- 172.21.15.64:0/2806875901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56158140ecc0 msgr2=0x5615814180c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:22:14.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.850+0000 7f533ed9d700 1 --2- 172.21.15.64:0/2806875901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56158140ecc0 0x5615814180c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:14.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.850+0000 7f533ed9d700 1 -- 172.21.15.64:0/2806875901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56158140e300 msgr2=0x56158141ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:22:14.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.850+0000 7f533ed9d700 1 --2- 172.21.15.64:0/2806875901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56158140e300 0x56158141ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:14.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.850+0000 7f533ed9d700 1 -- 172.21.15.64:0/2806875901 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53401a5620 con 0x5615814141a0 2023-09-13T03:22:14.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.850+0000 7f533e59c700 1 --2- 172.21.15.64:0/2806875901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56158140e300 0x56158141ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:22:14.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.850+0000 7f533ed9d700 1 --2- 172.21.15.64:0/2806875901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615814141a0 0x561581418600 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5338004660 tx=0x7f5338004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83e789d51c78296a server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:22:14.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.850+0000 7f5325ffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:22:14.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.850+0000 7f5325ffb700 1 -- 172.21.15.64:0/2806875901 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5338004db0 con 0x5615814141a0 2023-09-13T03:22:14.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.850+0000 7f5358330b80 1 -- 172.21.15.64:0/2806875901 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5615812b8490 con 0x5615814141a0 2023-09-13T03:22:14.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.850+0000 7f5325ffb700 1 -- 172.21.15.64:0/2806875901 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f5338005020 con 0x5615814141a0 2023-09-13T03:22:14.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.850+0000 7f5325ffb700 1 -- 172.21.15.64:0/2806875901 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5338003e50 con 0x5615814141a0 2023-09-13T03:22:14.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.850+0000 7f5358330b80 10 client.16821 fetch_fsmap learned FSMap version 55 2023-09-13T03:22:14.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.850+0000 7f5358330b80 1 -- 172.21.15.64:0/2806875901 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x7f5338009160 con 0x5615814141a0 2023-09-13T03:22:14.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.851+0000 7f5325ffb700 1 -- 172.21.15.64:0/2806875901 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5615812b8490 con 0x5615814141a0 2023-09-13T03:22:14.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.851+0000 7f5325ffb700 1 -- 172.21.15.64:0/2806875901 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 55) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f533801c030 con 0x5615814141a0 2023-09-13T03:22:14.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.851+0000 7f5358330b80 10 client.16821 fetch_fsmap finished waiting for FSMap version 55 2023-09-13T03:22:14.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.851+0000 7f5358330b80 10 client.16821 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:22:14.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.851+0000 7f5358330b80 20 client.16821 populate_metadata read hostname 'smithi064' 2023-09-13T03:22:14.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.851+0000 7f5358330b80 1 --2- 172.21.15.64:0/2806875901 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x561581428390 0x56158142a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:14.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.851+0000 7f5358330b80 4 client.16821 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:22:14.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.851+0000 7f5358330b80 1 -- 172.21.15.64:0/2806875901 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561581375840 con 0x561581428390 2023-09-13T03:22:14.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.851+0000 7f533e59c700 1 --2- 172.21.15.64:0/2806875901 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x561581428390 0x56158142a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:14.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.852+0000 7f533e59c700 1 --2- 172.21.15.64:0/2806875901 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x561581428390 0x56158142a7b0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44d9dbd16276caa2 server_cookie=a3bf139f1c48be1d in_seq=0 out_seq=0 2023-09-13T03:22:14.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.852+0000 7f5325ffb700 10 client.16821 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:22:14.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.853+0000 7f5325ffb700 1 -- 172.21.15.64:0/2806875901 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561581375840 con 0x561581428390 2023-09-13T03:22:14.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.853+0000 7f5325ffb700 10 client.16821 handle_command_reply: tid=0 2023-09-13T03:22:14.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.853+0000 7f5358330b80 1 client.16821 shutdown 2023-09-13T03:22:14.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.853+0000 7f5358330b80 1 -- 172.21.15.64:0/2806875901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615814141a0 msgr2=0x561581418600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:14.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.853+0000 7f5358330b80 1 --2- 172.21.15.64:0/2806875901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615814141a0 0x561581418600 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5338004660 tx=0x7f5338004970 comp rx=0 tx=0).stop 2023-09-13T03:22:14.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.853+0000 7f5358330b80 1 -- 172.21.15.64:0/2806875901 shutdown_connections 2023-09-13T03:22:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.853+0000 7f5358330b80 1 --2- 172.21.15.64:0/2806875901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56158140e300 0x56158141ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.853+0000 7f5358330b80 1 --2- 172.21.15.64:0/2806875901 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x561581428390 0x56158142a7b0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.853+0000 7f5358330b80 1 --2- 172.21.15.64:0/2806875901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615814141a0 0x561581418600 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.853+0000 7f5358330b80 1 --2- 172.21.15.64:0/2806875901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56158140ecc0 0x5615814180c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.853+0000 7f5325ffb700 0 client.16821 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:22:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.853+0000 7f5358330b80 1 -- 172.21.15.64:0/2806875901 >> 172.21.15.64:0/2806875901 conn(0x561581383030 msgr2=0x5615814087a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:22:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.853+0000 7f5358330b80 1 -- 172.21.15.64:0/2806875901 shutdown_connections 2023-09-13T03:22:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.853+0000 7f5358330b80 1 -- 172.21.15.64:0/2806875901 wait complete. 2023-09-13T03:22:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.853+0000 7f5358330b80 20 client.16821 trim_cache size 0 max 16384 2023-09-13T03:22:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.868+0000 7f5358330b80 1 Processor -- start 2023-09-13T03:22:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.869+0000 7f5358330b80 1 -- start start 2023-09-13T03:22:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.869+0000 7f5358330b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561581383030 0x56158141f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.869+0000 7f5358330b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56158140e300 0x561581408f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.869+0000 7f5358330b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56158140ecc0 0x561581409460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.869+0000 7f5358330b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615814038a0 con 0x56158140ecc0 2023-09-13T03:22:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.869+0000 7f5358330b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561581418be0 con 0x56158140e300 2023-09-13T03:22:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.869+0000 7f5358330b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56158140f580 con 0x561581383030 2023-09-13T03:22:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.869+0000 7f533ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56158140ecc0 0x561581409460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.869+0000 7f533ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56158140ecc0 0x561581409460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53750/0 (socket says 172.21.15.64:53750) 2023-09-13T03:22:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.869+0000 7f533ed9d700 1 -- 172.21.15.64:0/1007846991 learned_addr learned my addr 172.21.15.64:0/1007846991 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:22:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.869+0000 7f533e59c700 1 --2- 172.21.15.64:0/1007846991 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561581383030 0x56158141f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.869+0000 7f5337fff700 1 --2- 172.21.15.64:0/1007846991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56158140e300 0x561581408f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.869+0000 7f533ed9d700 1 -- 172.21.15.64:0/1007846991 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561581383030 msgr2=0x56158141f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:22:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.869+0000 7f533ed9d700 1 --2- 172.21.15.64:0/1007846991 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561581383030 0x56158141f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.869+0000 7f533ed9d700 1 -- 172.21.15.64:0/1007846991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56158140e300 msgr2=0x561581408f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:22:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.869+0000 7f533ed9d700 1 --2- 172.21.15.64:0/1007846991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56158140e300 0x561581408f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.869+0000 7f533ed9d700 1 -- 172.21.15.64:0/1007846991 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5330017070 con 0x56158140ecc0 2023-09-13T03:22:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.869+0000 7f5337fff700 1 --2- 172.21.15.64:0/1007846991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56158140e300 0x561581408f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:22:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.870+0000 7f533ed9d700 1 --2- 172.21.15.64:0/1007846991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56158140ecc0 0x561581409460 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5338009cd0 tx=0x7f53380078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a90ec6ef6e1d40c server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:22:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.870+0000 7f53357fa700 1 -- 172.21.15.64:0/1007846991 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5338004a40 con 0x56158140ecc0 2023-09-13T03:22:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.870+0000 7f53357fa700 1 -- 172.21.15.64:0/1007846991 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f533800f040 con 0x56158140ecc0 2023-09-13T03:22:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.870+0000 7f53357fa700 1 -- 172.21.15.64:0/1007846991 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5338016640 con 0x56158140ecc0 2023-09-13T03:22:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.871+0000 7f5358330b80 1 -- 172.21.15.64:0/1007846991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56158140ecc0 msgr2=0x561581409460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.871+0000 7f5358330b80 1 --2- 172.21.15.64:0/1007846991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56158140ecc0 0x561581409460 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5338009cd0 tx=0x7f53380078a0 comp rx=0 tx=0).stop 2023-09-13T03:22:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.871+0000 7f5358330b80 1 -- 172.21.15.64:0/1007846991 shutdown_connections 2023-09-13T03:22:14.877 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.871+0000 7f5358330b80 1 --2- 172.21.15.64:0/1007846991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56158140ecc0 0x561581409460 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:14.877 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.871+0000 7f5358330b80 1 --2- 172.21.15.64:0/1007846991 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561581383030 0x56158141f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:14.877 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.871+0000 7f5358330b80 1 --2- 172.21.15.64:0/1007846991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56158140e300 0x561581408f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:14.877 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.871+0000 7f5358330b80 1 -- 172.21.15.64:0/1007846991 >> 172.21.15.64:0/1007846991 conn(0x561581412880 msgr2=0x56158140db80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:22:14.877 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.871+0000 7f5358330b80 1 -- 172.21.15.64:0/1007846991 shutdown_connections 2023-09-13T03:22:14.877 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.871+0000 7f5358330b80 1 -- 172.21.15.64:0/1007846991 wait complete. 2023-09-13T03:22:14.878 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.871+0000 7f5358330b80 1 -- start start 2023-09-13T03:22:14.878 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.871+0000 7f5358330b80 1 Processor -- start 2023-09-13T03:22:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.872+0000 7f5358330b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561581383030 0x561581475880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.872+0000 7f5358330b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56158140e300 0x561581475dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:14.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.872+0000 7f5358330b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56158140ecc0 0x561581470110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:14.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.872+0000 7f5358330b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561581470760 con 0x56158140e300 2023-09-13T03:22:14.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.872+0000 7f5358330b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615814708d0 con 0x56158140ecc0 2023-09-13T03:22:14.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.872+0000 7f5358330b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561581470a90 con 0x561581383030 2023-09-13T03:22:14.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.872+0000 7f5358330b80 10 client.0 _ll_register_callbacks cb 0x561581248a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:22:14.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.872+0000 7f5337fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56158140e300 0x561581475dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:14.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.872+0000 7f533e59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561581383030 0x561581475880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:14.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.872+0000 7f533e59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561581383030 0x561581475880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:53906/0 (socket says 172.21.15.64:53906) 2023-09-13T03:22:14.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.872+0000 7f533e59c700 1 -- 172.21.15.64:0/1673695595 learned_addr learned my addr 172.21.15.64:0/1673695595 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:22:14.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.872+0000 7f533ed9d700 1 --2- 172.21.15.64:0/1673695595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56158140ecc0 0x561581470110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:14.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.872+0000 7f5337fff700 1 -- 172.21.15.64:0/1673695595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561581383030 msgr2=0x561581475880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:22:14.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.872+0000 7f5337fff700 1 --2- 172.21.15.64:0/1673695595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561581383030 0x561581475880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:14.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.872+0000 7f5337fff700 1 -- 172.21.15.64:0/1673695595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56158140ecc0 msgr2=0x561581470110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:22:14.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.872+0000 7f5337fff700 1 --2- 172.21.15.64:0/1673695595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56158140ecc0 0x561581470110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:14.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.872+0000 7f5337fff700 1 -- 172.21.15.64:0/1673695595 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5338009160 con 0x56158140e300 2023-09-13T03:22:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.873+0000 7f5337fff700 1 --2- 172.21.15.64:0/1673695595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56158140e300 0x561581475dc0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f53280ca380 tx=0x7f5328005570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed3e0c5c5446adba server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:22:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.873+0000 7f5325ffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:22:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.873+0000 7f5325ffb700 1 -- 172.21.15.64:0/1673695595 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53280b1070 con 0x56158140e300 2023-09-13T03:22:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.873+0000 7f5358330b80 1 -- 172.21.15.64:0/1673695595 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5330021070 con 0x56158140e300 2023-09-13T03:22:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.873+0000 7f5325ffb700 1 -- 172.21.15.64:0/1673695595 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f532809a170 con 0x56158140e300 2023-09-13T03:22:14.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.873+0000 7f5325ffb700 1 -- 172.21.15.64:0/1673695595 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53280053e0 con 0x56158140e300 2023-09-13T03:22:14.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.873+0000 7f5358330b80 10 client.16827 fetch_fsmap learned FSMap version 55 2023-09-13T03:22:14.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.873+0000 7f5358330b80 1 -- 172.21.15.64:0/1673695595 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x561581175c00 con 0x56158140e300 2023-09-13T03:22:14.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.873+0000 7f5325ffb700 1 -- 172.21.15.64:0/1673695595 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5330021070 con 0x56158140e300 2023-09-13T03:22:14.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.874+0000 7f5325ffb700 1 -- 172.21.15.64:0/1673695595 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 55) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f532809a3d0 con 0x56158140e300 2023-09-13T03:22:14.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.874+0000 7f5358330b80 10 client.16827 fetch_fsmap finished waiting for FSMap version 55 2023-09-13T03:22:14.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.874+0000 7f5358330b80 10 client.16827 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:22:14.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.874+0000 7f5358330b80 20 client.16827 populate_metadata read hostname 'smithi064' 2023-09-13T03:22:14.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.874+0000 7f5358330b80 1 --2- 172.21.15.64:0/1673695595 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5615814141a0 0x56158147f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:14.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.874+0000 7f5358330b80 4 client.16827 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:22:14.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.874+0000 7f5358330b80 1 -- 172.21.15.64:0/1673695595 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5615813598c0 con 0x5615814141a0 2023-09-13T03:22:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.874+0000 7f533e59c700 1 --2- 172.21.15.64:0/1673695595 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5615814141a0 0x56158147f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.875+0000 7f533e59c700 1 --2- 172.21.15.64:0/1673695595 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5615814141a0 0x56158147f410 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d695af469ffc7ff server_cookie=9368702e0d9a5a0b in_seq=0 out_seq=0 2023-09-13T03:22:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.875+0000 7f5325ffb700 10 client.16827 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:22:14.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.903+0000 7f5325ffb700 1 -- 172.21.15.64:0/1673695595 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5615813598c0 con 0x5615814141a0 2023-09-13T03:22:14.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.903+0000 7f5325ffb700 10 client.16827 handle_command_reply: tid=0 2023-09-13T03:22:14.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.903+0000 7f5358330b80 1 client.16827 shutdown 2023-09-13T03:22:14.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.903+0000 7f5358330b80 1 -- 172.21.15.64:0/1673695595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56158140e300 msgr2=0x561581475dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:14.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.903+0000 7f5358330b80 1 --2- 172.21.15.64:0/1673695595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56158140e300 0x561581475dc0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f53280ca380 tx=0x7f5328005570 comp rx=0 tx=0).stop 2023-09-13T03:22:14.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.904+0000 7f5358330b80 1 -- 172.21.15.64:0/1673695595 shutdown_connections 2023-09-13T03:22:14.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.904+0000 7f5358330b80 1 --2- 172.21.15.64:0/1673695595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56158140ecc0 0x561581470110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:14.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.904+0000 7f5358330b80 1 --2- 172.21.15.64:0/1673695595 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5615814141a0 0x56158147f410 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:14.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.904+0000 7f5358330b80 1 --2- 172.21.15.64:0/1673695595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561581383030 0x561581475880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:14.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.904+0000 7f5358330b80 1 --2- 172.21.15.64:0/1673695595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56158140e300 0x561581475dc0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.904+0000 7f5325ffb700 0 client.16827 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:22:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.904+0000 7f5358330b80 1 -- 172.21.15.64:0/1673695595 >> 172.21.15.64:0/1673695595 conn(0x561581412880 msgr2=0x56158140db80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:22:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.904+0000 7f5358330b80 1 -- 172.21.15.64:0/1673695595 shutdown_connections 2023-09-13T03:22:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.904+0000 7f5358330b80 1 -- 172.21.15.64:0/1673695595 wait complete. 2023-09-13T03:22:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.904+0000 7f5358330b80 20 client.16827 trim_cache size 0 max 16384 2023-09-13T03:22:14.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:22:14.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:22:14.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:22:14.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:22:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.905+0000 7f53449ff700 1 -- 172.21.15.64:0/2392424791 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f532c0a07b0 msgr2=0x7f532c0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.905+0000 7f53449ff700 1 --2- 172.21.15.64:0/2392424791 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f532c0a07b0 0x7f532c0a2c70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5330009c60 tx=0x7f5330009260 comp rx=0 tx=0).stop 2023-09-13T03:22:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.905+0000 7f53449ff700 1 -- 172.21.15.64:0/2392424791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5340100f60 msgr2=0x7f534019ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.905+0000 7f53449ff700 1 --2- 172.21.15.64:0/2392424791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5340100f60 0x7f534019ed50 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f5328009810 tx=0x7f532800fa00 comp rx=0 tx=0).stop 2023-09-13T03:22:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.905+0000 7f53449ff700 1 -- 172.21.15.64:0/2392424791 shutdown_connections 2023-09-13T03:22:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.905+0000 7f53449ff700 1 --2- 172.21.15.64:0/2392424791 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f532c0a07b0 0x7f532c0a2c70 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.905+0000 7f53449ff700 1 --2- 172.21.15.64:0/2392424791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53401005a0 0x7f534019e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:14.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.905+0000 7f53449ff700 1 --2- 172.21.15.64:0/2392424791 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5340106660 0x7f5340198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:14.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.905+0000 7f53449ff700 1 --2- 172.21.15.64:0/2392424791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5340100f60 0x7f534019ed50 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:14.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.905+0000 7f53449ff700 1 -- 172.21.15.64:0/2392424791 >> 172.21.15.64:0/2392424791 conn(0x7f53400fc050 msgr2=0x7f5340072830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:22:14.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.905+0000 7f53449ff700 1 -- 172.21.15.64:0/2392424791 shutdown_connections 2023-09-13T03:22:14.911 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:14.905+0000 7f53449ff700 1 -- 172.21.15.64:0/2392424791 wait complete. 2023-09-13T03:22:14.936 INFO:tasks.workunit.client.0.smithi064.stdout: 1 193043 1.98 MB/sec execute 379 sec latency 5875.336 ms 2023-09-13T03:22:14.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:22:14 smithi142 ceph-mon[134082]: pgmap v281: 97 pgs: 97 active+clean; 95 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 218 op/s 2023-09-13T03:22:15.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:22:14 smithi170 ceph-mon[134328]: pgmap v281: 97 pgs: 97 active+clean; 95 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 218 op/s 2023-09-13T03:22:15.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:22:14 smithi064 ceph-mon[97675]: pgmap v281: 97 pgs: 97 active+clean; 95 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 218 op/s 2023-09-13T03:22:15.937 INFO:tasks.workunit.client.0.smithi064.stdout: 1 193043 1.97 MB/sec execute 380 sec latency 6875.450 ms 2023-09-13T03:22:16.937 INFO:tasks.workunit.client.0.smithi064.stdout: 1 193043 1.97 MB/sec execute 381 sec latency 7875.569 ms 2023-09-13T03:22:16.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:22:16 smithi142 ceph-mon[134082]: pgmap v282: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 516 KiB/s wr, 71 op/s 2023-09-13T03:22:17.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:22:16 smithi170 ceph-mon[134328]: pgmap v282: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 516 KiB/s wr, 71 op/s 2023-09-13T03:22:17.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:22:16 smithi064 ceph-mon[97675]: pgmap v282: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 516 KiB/s wr, 71 op/s 2023-09-13T03:22:17.937 INFO:tasks.workunit.client.0.smithi064.stdout: 1 193043 1.96 MB/sec execute 382 sec latency 8875.696 ms 2023-09-13T03:22:18.937 INFO:tasks.workunit.client.0.smithi064.stdout: 1 193043 1.96 MB/sec execute 383 sec latency 9875.799 ms 2023-09-13T03:22:18.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:22:18 smithi142 ceph-mon[134082]: pgmap v283: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 516 KiB/s wr, 71 op/s 2023-09-13T03:22:19.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:22:18 smithi170 ceph-mon[134328]: pgmap v283: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 516 KiB/s wr, 71 op/s 2023-09-13T03:22:19.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:22:18 smithi064 ceph-mon[97675]: pgmap v283: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 516 KiB/s wr, 71 op/s 2023-09-13T03:22:19.937 INFO:tasks.workunit.client.0.smithi064.stdout: 1 193043 1.95 MB/sec execute 384 sec latency 10875.933 ms 2023-09-13T03:22:20.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:22:20 smithi170 ceph-mon[134328]: pgmap v284: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 502 KiB/s wr, 69 op/s 2023-09-13T03:22:20.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:22:20 smithi064 ceph-mon[97675]: pgmap v284: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 502 KiB/s wr, 69 op/s 2023-09-13T03:22:20.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:22:20 smithi142 ceph-mon[134082]: pgmap v284: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 502 KiB/s wr, 69 op/s 2023-09-13T03:22:20.937 INFO:tasks.workunit.client.0.smithi064.stdout: 1 193215 1.95 MB/sec execute 385 sec latency 11713.056 ms 2023-09-13T03:22:21.937 INFO:tasks.workunit.client.0.smithi064.stdout: 1 193239 1.94 MB/sec execute 386 sec latency 983.604 ms 2023-09-13T03:22:22.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:22:22 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:22:22.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:22:22 smithi170 ceph-mon[134328]: pgmap v285: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 440 KiB/s wr, 65 op/s 2023-09-13T03:22:22.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:22:22 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:22:22.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:22:22 smithi064 ceph-mon[97675]: pgmap v285: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 440 KiB/s wr, 65 op/s 2023-09-13T03:22:22.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:22:22 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:22:22.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:22:22 smithi142 ceph-mon[134082]: pgmap v285: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 440 KiB/s wr, 65 op/s 2023-09-13T03:22:22.937 INFO:tasks.workunit.client.0.smithi064.stdout: 1 193239 1.94 MB/sec execute 387 sec latency 1983.758 ms 2023-09-13T03:22:23.938 INFO:tasks.workunit.client.0.smithi064.stdout: 1 193239 1.93 MB/sec execute 388 sec latency 2983.906 ms 2023-09-13T03:22:24.938 INFO:tasks.workunit.client.0.smithi064.stdout: 1 193239 1.93 MB/sec execute 389 sec latency 3984.064 ms 2023-09-13T03:22:24.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:22:24 smithi142 ceph-mon[134082]: pgmap v286: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 130 KiB/s wr, 23 op/s 2023-09-13T03:22:25.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:22:24 smithi170 ceph-mon[134328]: pgmap v286: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 130 KiB/s wr, 23 op/s 2023-09-13T03:22:25.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:22:24 smithi064 ceph-mon[97675]: pgmap v286: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 130 KiB/s wr, 23 op/s 2023-09-13T03:22:25.938 INFO:tasks.workunit.client.0.smithi064.stdout: 1 194467 1.94 MB/sec execute 390 sec latency 4397.846 ms 2023-09-13T03:22:26.938 INFO:tasks.workunit.client.0.smithi064.stdout: 1 195690 1.96 MB/sec execute 391 sec latency 16.211 ms 2023-09-13T03:22:26.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:22:26 smithi142 ceph-mon[134082]: pgmap v287: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 177 KiB/s wr, 36 op/s 2023-09-13T03:22:27.159 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:22:26 smithi170 ceph-mon[134328]: pgmap v287: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 177 KiB/s wr, 36 op/s 2023-09-13T03:22:27.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:22:26 smithi064 ceph-mon[97675]: pgmap v287: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 177 KiB/s wr, 36 op/s 2023-09-13T03:22:27.938 INFO:tasks.workunit.client.0.smithi064.stdout: 1 195819 1.95 MB/sec execute 392 sec latency 914.461 ms 2023-09-13T03:22:27.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:22:27 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:22:27.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:22:27 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:22:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.055+0000 7fb41d3ff700 1 -- 172.21.15.64:0/106794497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4180732f0 msgr2=0x7fb41810c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.055+0000 7fb41d3ff700 1 --2- 172.21.15.64:0/106794497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4180732f0 0x7fb41810c920 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009990 tx=0x7fb40c009ca0 comp rx=0 tx=0).stop 2023-09-13T03:22:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.056+0000 7fb41d3ff700 1 -- 172.21.15.64:0/106794497 shutdown_connections 2023-09-13T03:22:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.056+0000 7fb41d3ff700 1 --2- 172.21.15.64:0/106794497 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb41810ce60 0x7fb41810f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.056+0000 7fb41d3ff700 1 --2- 172.21.15.64:0/106794497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4180729d0 0x7fb418072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.056+0000 7fb41d3ff700 1 --2- 172.21.15.64:0/106794497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4180732f0 0x7fb41810c920 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.056+0000 7fb41d3ff700 1 -- 172.21.15.64:0/106794497 >> 172.21.15.64:0/106794497 conn(0x7fb4180fc0d0 msgr2=0x7fb4180fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:22:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.056+0000 7fb41d3ff700 1 -- 172.21.15.64:0/106794497 shutdown_connections 2023-09-13T03:22:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.056+0000 7fb41d3ff700 1 -- 172.21.15.64:0/106794497 wait complete. 2023-09-13T03:22:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.057+0000 7fb41d3ff700 1 Processor -- start 2023-09-13T03:22:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.057+0000 7fb41d3ff700 1 -- start start 2023-09-13T03:22:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.057+0000 7fb41d3ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4180729d0 0x7fb41819e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.057+0000 7fb41d3ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4180732f0 0x7fb41819ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.057+0000 7fb41d3ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb41810ce60 0x7fb4181987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.057+0000 7fb41d3ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb418198e10 con 0x7fb4180729d0 2023-09-13T03:22:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.057+0000 7fb41d3ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb418198f80 con 0x7fb4180732f0 2023-09-13T03:22:28.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.057+0000 7fb41d3ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb418199140 con 0x7fb41810ce60 2023-09-13T03:22:28.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.058+0000 7fb4167fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4180732f0 0x7fb41819ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:28.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.058+0000 7fb416ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4180729d0 0x7fb41819e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:28.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.058+0000 7fb4167fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4180732f0 0x7fb41819ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:32800/0 (socket says 172.21.15.64:32800) 2023-09-13T03:22:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.058+0000 7fb4167fc700 1 -- 172.21.15.64:0/2020586202 learned_addr learned my addr 172.21.15.64:0/2020586202 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:22:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.058+0000 7fb4177fe700 1 --2- 172.21.15.64:0/2020586202 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb41810ce60 0x7fb4181987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.058+0000 7fb4167fc700 1 -- 172.21.15.64:0/2020586202 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb41810ce60 msgr2=0x7fb4181987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.058+0000 7fb4167fc700 1 --2- 172.21.15.64:0/2020586202 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb41810ce60 0x7fb4181987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:28.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.058+0000 7fb4167fc700 1 -- 172.21.15.64:0/2020586202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4180729d0 msgr2=0x7fb41819e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:28.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.058+0000 7fb4167fc700 1 --2- 172.21.15.64:0/2020586202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4180729d0 0x7fb41819e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:28.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.058+0000 7fb4167fc700 1 -- 172.21.15.64:0/2020586202 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb40c0095f0 con 0x7fb4180732f0 2023-09-13T03:22:28.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.058+0000 7fb4167fc700 1 --2- 172.21.15.64:0/2020586202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4180732f0 0x7fb41819ee80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb40c005910 tx=0x7fb40c004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:22:28.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.058+0000 7fb3fffff700 1 -- 172.21.15.64:0/2020586202 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40c01d070 con 0x7fb4180732f0 2023-09-13T03:22:28.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.059+0000 7fb3fffff700 1 -- 172.21.15.64:0/2020586202 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb40c022470 con 0x7fb4180732f0 2023-09-13T03:22:28.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.059+0000 7fb3fffff700 1 -- 172.21.15.64:0/2020586202 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40c017650 con 0x7fb4180732f0 2023-09-13T03:22:28.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.059+0000 7fb41d3ff700 1 -- 172.21.15.64:0/2020586202 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4181993c0 con 0x7fb4180732f0 2023-09-13T03:22:28.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.059+0000 7fb41d3ff700 1 -- 172.21.15.64:0/2020586202 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4181a5930 con 0x7fb4180732f0 2023-09-13T03:22:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.063+0000 7fb3fffff700 1 -- 172.21.15.64:0/2020586202 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fb40c0225e0 con 0x7fb4180732f0 2023-09-13T03:22:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.065+0000 7fb430d69b80 1 Processor -- start 2023-09-13T03:22:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.065+0000 7fb430d69b80 1 -- start start 2023-09-13T03:22:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.065+0000 7fb430d69b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca1a7421a0 0x55ca1a742580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.065+0000 7fb3fffff700 1 --2- 172.21.15.64:0/2020586202 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fb4000a0830 0x7fb4000a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.065+0000 7fb430d69b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca1a73c300 0x55ca1a73c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.065+0000 7fb430d69b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ca1a73ccc0 0x55ca1a748f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.065+0000 7fb430d69b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca1a6956d0 con 0x55ca1a7421a0 2023-09-13T03:22:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.065+0000 7fb430d69b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca1a692ff0 con 0x55ca1a73c300 2023-09-13T03:22:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.065+0000 7fb430d69b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca1a68e8e0 con 0x55ca1a73ccc0 2023-09-13T03:22:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.066+0000 7fb3fffff700 1 -- 172.21.15.64:0/2020586202 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fb40c0cd180 con 0x7fb4180732f0 2023-09-13T03:22:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.066+0000 7fb416ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca1a7421a0 0x55ca1a742580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.066+0000 7fb4177fe700 1 --2- 172.21.15.64:0/2020586202 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fb4000a0830 0x7fb4000a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.066+0000 7fb416ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca1a7421a0 0x55ca1a742580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49086/0 (socket says 172.21.15.64:49086) 2023-09-13T03:22:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.066+0000 7fb416ffd700 1 -- 172.21.15.64:0/2594772675 learned_addr learned my addr 172.21.15.64:0/2594772675 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:22:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.066+0000 7fb4167fc700 1 --2- 172.21.15.64:0/2594772675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca1a73c300 0x55ca1a73c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.066+0000 7fb4177fe700 1 --2- 172.21.15.64:0/2594772675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ca1a73ccc0 0x55ca1a748f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.066+0000 7fb416ffd700 1 -- 172.21.15.64:0/2594772675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ca1a73ccc0 msgr2=0x55ca1a748f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:22:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.066+0000 7fb416ffd700 1 --2- 172.21.15.64:0/2594772675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ca1a73ccc0 0x55ca1a748f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.066+0000 7fb416ffd700 1 -- 172.21.15.64:0/2594772675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca1a73c300 msgr2=0x55ca1a73c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:22:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.066+0000 7fb416ffd700 1 --2- 172.21.15.64:0/2594772675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca1a73c300 0x55ca1a73c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.066+0000 7fb416ffd700 1 -- 172.21.15.64:0/2594772675 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca1a4bad80 con 0x55ca1a7421a0 2023-09-13T03:22:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.066+0000 7fb4167fc700 1 --2- 172.21.15.64:0/2594772675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca1a73c300 0x55ca1a73c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:22:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.067+0000 7fb4177fe700 1 --2- 172.21.15.64:0/2020586202 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fb4000a0830 0x7fb4000a2cf0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb410009ec0 tx=0x7fb410011c60 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:22:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.067+0000 7fb416ffd700 1 --2- 172.21.15.64:0/2594772675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca1a7421a0 0x55ca1a742580 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb408009d80 tx=0x7fb408011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b7a5d33ef4a8d6 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:22:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.067+0000 7fb3fd7fa700 1 -- 172.21.15.64:0/2594772675 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb408004230 con 0x55ca1a7421a0 2023-09-13T03:22:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.067+0000 7fb3fd7fa700 1 -- 172.21.15.64:0/2594772675 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb408004390 con 0x55ca1a7421a0 2023-09-13T03:22:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.067+0000 7fb3fd7fa700 1 -- 172.21.15.64:0/2594772675 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4080197a0 con 0x55ca1a7421a0 2023-09-13T03:22:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.068+0000 7fb430d69b80 1 -- 172.21.15.64:0/2594772675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca1a7421a0 msgr2=0x55ca1a742580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.068+0000 7fb430d69b80 1 --2- 172.21.15.64:0/2594772675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca1a7421a0 0x55ca1a742580 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb408009d80 tx=0x7fb408011720 comp rx=0 tx=0).stop 2023-09-13T03:22:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.068+0000 7fb430d69b80 1 -- 172.21.15.64:0/2594772675 shutdown_connections 2023-09-13T03:22:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.068+0000 7fb430d69b80 1 --2- 172.21.15.64:0/2594772675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ca1a73ccc0 0x55ca1a748f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.068+0000 7fb430d69b80 1 --2- 172.21.15.64:0/2594772675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca1a7421a0 0x55ca1a742580 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.068+0000 7fb430d69b80 1 --2- 172.21.15.64:0/2594772675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca1a73c300 0x55ca1a73c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.068+0000 7fb430d69b80 1 -- 172.21.15.64:0/2594772675 >> 172.21.15.64:0/2594772675 conn(0x55ca1a6b1030 msgr2=0x55ca1a737560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:22:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.068+0000 7fb430d69b80 1 -- 172.21.15.64:0/2594772675 shutdown_connections 2023-09-13T03:22:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.068+0000 7fb430d69b80 1 -- 172.21.15.64:0/2594772675 wait complete. 2023-09-13T03:22:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.068+0000 7fb430d69b80 1 -- start start 2023-09-13T03:22:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.069+0000 7fb430d69b80 1 Processor -- start 2023-09-13T03:22:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.069+0000 7fb430d69b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca1a73c300 0x55ca1a748df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.069+0000 7fb430d69b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca1a73ccc0 0x55ca1a746070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.069+0000 7fb430d69b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ca1a7421a0 0x55ca1a746610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.069+0000 7fb430d69b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca1a62cb10 con 0x55ca1a73ccc0 2023-09-13T03:22:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.069+0000 7fb430d69b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca1a62c810 con 0x55ca1a73c300 2023-09-13T03:22:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.069+0000 7fb430d69b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca1a5108c0 con 0x55ca1a7421a0 2023-09-13T03:22:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.069+0000 7fb430d69b80 10 client.0 _ll_register_callbacks cb 0x55ca1a576a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:22:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.070+0000 7fb4167fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca1a73ccc0 0x55ca1a746070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.070+0000 7fb4167fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca1a73ccc0 0x55ca1a746070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49102/0 (socket says 172.21.15.64:49102) 2023-09-13T03:22:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.070+0000 7fb4167fc700 1 -- 172.21.15.64:0/3350482562 learned_addr learned my addr 172.21.15.64:0/3350482562 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:22:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.070+0000 7fb416ffd700 1 --2- 172.21.15.64:0/3350482562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca1a73c300 0x55ca1a748df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.070+0000 7fb4177fe700 1 --2- 172.21.15.64:0/3350482562 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ca1a7421a0 0x55ca1a746610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.070+0000 7fb416ffd700 1 -- 172.21.15.64:0/3350482562 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ca1a7421a0 msgr2=0x55ca1a746610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:22:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.070+0000 7fb416ffd700 1 --2- 172.21.15.64:0/3350482562 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ca1a7421a0 0x55ca1a746610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.070+0000 7fb416ffd700 1 -- 172.21.15.64:0/3350482562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca1a73ccc0 msgr2=0x55ca1a746070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:22:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.070+0000 7fb416ffd700 1 --2- 172.21.15.64:0/3350482562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca1a73ccc0 0x55ca1a746070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.070+0000 7fb416ffd700 1 -- 172.21.15.64:0/3350482562 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4080099e0 con 0x55ca1a73c300 2023-09-13T03:22:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.070+0000 7fb416ffd700 1 --2- 172.21.15.64:0/3350482562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca1a73c300 0x55ca1a748df0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb408003f10 tx=0x7fb4080038b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4eb177748f6e7c7e server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:22:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.070+0000 7fb3faff5700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:22:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.071+0000 7fb3faff5700 1 -- 172.21.15.64:0/3350482562 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb408003d30 con 0x55ca1a73c300 2023-09-13T03:22:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.071+0000 7fb3faff5700 1 -- 172.21.15.64:0/3350482562 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fb408005ce0 con 0x55ca1a73c300 2023-09-13T03:22:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.071+0000 7fb430d69b80 1 -- 172.21.15.64:0/3350482562 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ca1a5e6490 con 0x55ca1a73c300 2023-09-13T03:22:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.071+0000 7fb3faff5700 1 -- 172.21.15.64:0/3350482562 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb408005040 con 0x55ca1a73c300 2023-09-13T03:22:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.071+0000 7fb430d69b80 10 client.26876 fetch_fsmap learned FSMap version 57 2023-09-13T03:22:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.071+0000 7fb430d69b80 1 -- 172.21.15.64:0/3350482562 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x55ca1a4a3c00 con 0x55ca1a73c300 2023-09-13T03:22:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.071+0000 7fb3faff5700 1 -- 172.21.15.64:0/3350482562 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ca1a5e6490 con 0x55ca1a73c300 2023-09-13T03:22:28.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.072+0000 7fb3faff5700 1 -- 172.21.15.64:0/3350482562 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 57) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fb408021040 con 0x55ca1a73c300 2023-09-13T03:22:28.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.072+0000 7fb430d69b80 10 client.26876 fetch_fsmap finished waiting for FSMap version 57 2023-09-13T03:22:28.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.072+0000 7fb430d69b80 10 client.26876 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:22:28.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.072+0000 7fb430d69b80 20 client.26876 populate_metadata read hostname 'smithi064' 2023-09-13T03:22:28.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.072+0000 7fb430d69b80 1 --2- 172.21.15.64:0/3350482562 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55ca1a756310 0x55ca1a758730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:28.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.072+0000 7fb430d69b80 4 client.26876 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:22:28.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.072+0000 7fb430d69b80 1 -- 172.21.15.64:0/3350482562 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ca1a6a3840 con 0x55ca1a756310 2023-09-13T03:22:28.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.072+0000 7fb4167fc700 1 --2- 172.21.15.64:0/3350482562 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55ca1a756310 0x55ca1a758730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:28.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.072+0000 7fb4167fc700 1 --2- 172.21.15.64:0/3350482562 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55ca1a756310 0x55ca1a758730 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=660a4d10446b2789 server_cookie=91acb6757d4d3f6f in_seq=0 out_seq=0 2023-09-13T03:22:28.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.072+0000 7fb3faff5700 10 client.26876 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:22:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.073+0000 7fb3faff5700 1 -- 172.21.15.64:0/3350482562 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ca1a6a3840 con 0x55ca1a756310 2023-09-13T03:22:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.073+0000 7fb3faff5700 10 client.26876 handle_command_reply: tid=0 2023-09-13T03:22:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.073+0000 7fb430d69b80 1 client.26876 shutdown 2023-09-13T03:22:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.073+0000 7fb430d69b80 1 -- 172.21.15.64:0/3350482562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca1a73c300 msgr2=0x55ca1a748df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.073+0000 7fb430d69b80 1 --2- 172.21.15.64:0/3350482562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca1a73c300 0x55ca1a748df0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb408003f10 tx=0x7fb4080038b0 comp rx=0 tx=0).stop 2023-09-13T03:22:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.073+0000 7fb430d69b80 1 -- 172.21.15.64:0/3350482562 shutdown_connections 2023-09-13T03:22:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.073+0000 7fb430d69b80 1 --2- 172.21.15.64:0/3350482562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca1a73c300 0x55ca1a748df0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.073+0000 7fb430d69b80 1 --2- 172.21.15.64:0/3350482562 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55ca1a756310 0x55ca1a758730 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.073+0000 7fb430d69b80 1 --2- 172.21.15.64:0/3350482562 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ca1a7421a0 0x55ca1a746610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.073+0000 7fb430d69b80 1 --2- 172.21.15.64:0/3350482562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca1a73ccc0 0x55ca1a746070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.073+0000 7fb3faff5700 0 client.26876 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:22:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.073+0000 7fb430d69b80 1 -- 172.21.15.64:0/3350482562 >> 172.21.15.64:0/3350482562 conn(0x55ca1a6b1030 msgr2=0x55ca1a7367a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:22:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.074+0000 7fb430d69b80 1 -- 172.21.15.64:0/3350482562 shutdown_connections 2023-09-13T03:22:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.074+0000 7fb430d69b80 1 -- 172.21.15.64:0/3350482562 wait complete. 2023-09-13T03:22:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.074+0000 7fb430d69b80 20 client.26876 trim_cache size 0 max 16384 2023-09-13T03:22:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.089+0000 7fb430d69b80 1 Processor -- start 2023-09-13T03:22:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.090+0000 7fb430d69b80 1 -- start start 2023-09-13T03:22:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.090+0000 7fb430d69b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca1a73c300 0x55ca1a74d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.090+0000 7fb430d69b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ca1a73ccc0 0x55ca1a748920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.090+0000 7fb430d69b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca1a7421a0 0x55ca1a73eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.090+0000 7fb430d69b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca1a740ca0 con 0x55ca1a7421a0 2023-09-13T03:22:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.090+0000 7fb430d69b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca1a75d450 con 0x55ca1a73c300 2023-09-13T03:22:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.090+0000 7fb430d69b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca1a75d610 con 0x55ca1a73ccc0 2023-09-13T03:22:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.090+0000 7fb4177fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca1a7421a0 0x55ca1a73eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.090+0000 7fb4177fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca1a7421a0 0x55ca1a73eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49114/0 (socket says 172.21.15.64:49114) 2023-09-13T03:22:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.090+0000 7fb4177fe700 1 -- 172.21.15.64:0/3081146318 learned_addr learned my addr 172.21.15.64:0/3081146318 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:22:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.090+0000 7fb416ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca1a73c300 0x55ca1a74d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.090+0000 7fb4167fc700 1 --2- 172.21.15.64:0/3081146318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ca1a73ccc0 0x55ca1a748920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.091+0000 7fb416ffd700 1 -- 172.21.15.64:0/3081146318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ca1a73ccc0 msgr2=0x55ca1a748920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:22:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.091+0000 7fb416ffd700 1 --2- 172.21.15.64:0/3081146318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ca1a73ccc0 0x55ca1a748920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.091+0000 7fb416ffd700 1 -- 172.21.15.64:0/3081146318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca1a7421a0 msgr2=0x55ca1a73eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:22:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.091+0000 7fb416ffd700 1 --2- 172.21.15.64:0/3081146318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca1a7421a0 0x55ca1a73eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.091+0000 7fb416ffd700 1 -- 172.21.15.64:0/3081146318 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca1a4e11f0 con 0x55ca1a73c300 2023-09-13T03:22:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.091+0000 7fb4177fe700 1 --2- 172.21.15.64:0/3081146318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca1a7421a0 0x55ca1a73eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:22:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.091+0000 7fb416ffd700 1 --2- 172.21.15.64:0/3081146318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca1a73c300 0x55ca1a74d520 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb408031040 tx=0x7fb408011c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e624685b2806c8f server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:22:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.091+0000 7fb3fdffb700 1 -- 172.21.15.64:0/3081146318 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb408003920 con 0x55ca1a73c300 2023-09-13T03:22:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.091+0000 7fb3fdffb700 1 -- 172.21.15.64:0/3081146318 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb408005c40 con 0x55ca1a73c300 2023-09-13T03:22:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.091+0000 7fb3fdffb700 1 -- 172.21.15.64:0/3081146318 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb408005470 con 0x55ca1a73c300 2023-09-13T03:22:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.092+0000 7fb430d69b80 1 -- 172.21.15.64:0/3081146318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca1a73c300 msgr2=0x55ca1a74d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.092+0000 7fb430d69b80 1 --2- 172.21.15.64:0/3081146318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca1a73c300 0x55ca1a74d520 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb408031040 tx=0x7fb408011c50 comp rx=0 tx=0).stop 2023-09-13T03:22:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.092+0000 7fb430d69b80 1 -- 172.21.15.64:0/3081146318 shutdown_connections 2023-09-13T03:22:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.092+0000 7fb430d69b80 1 --2- 172.21.15.64:0/3081146318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca1a73c300 0x55ca1a74d520 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.092+0000 7fb430d69b80 1 --2- 172.21.15.64:0/3081146318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca1a7421a0 0x55ca1a73eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.092+0000 7fb430d69b80 1 --2- 172.21.15.64:0/3081146318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ca1a73ccc0 0x55ca1a748920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.092+0000 7fb430d69b80 1 -- 172.21.15.64:0/3081146318 >> 172.21.15.64:0/3081146318 conn(0x55ca1a6b1030 msgr2=0x55ca1a752660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:22:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.092+0000 7fb430d69b80 1 -- 172.21.15.64:0/3081146318 shutdown_connections 2023-09-13T03:22:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.092+0000 7fb430d69b80 1 -- 172.21.15.64:0/3081146318 wait complete. 2023-09-13T03:22:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.092+0000 7fb430d69b80 1 -- start start 2023-09-13T03:22:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.093+0000 7fb430d69b80 1 Processor -- start 2023-09-13T03:22:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.093+0000 7fb430d69b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ca1a73c300 0x55ca1a74cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.093+0000 7fb430d69b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca1a73ccc0 0x55ca1a74d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.093+0000 7fb430d69b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca1a7421a0 0x55ca1a749700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.093+0000 7fb430d69b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca1a749da0 con 0x55ca1a73ccc0 2023-09-13T03:22:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.093+0000 7fb430d69b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca1a749f10 con 0x55ca1a7421a0 2023-09-13T03:22:28.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.093+0000 7fb430d69b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca1a74a0d0 con 0x55ca1a73c300 2023-09-13T03:22:28.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.093+0000 7fb430d69b80 10 client.0 _ll_register_callbacks cb 0x7fb408009c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:22:28.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.093+0000 7fb4177fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca1a7421a0 0x55ca1a749700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:28.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.094+0000 7fb4177fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca1a7421a0 0x55ca1a749700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:32830/0 (socket says 172.21.15.64:32830) 2023-09-13T03:22:28.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.094+0000 7fb4177fe700 1 -- 172.21.15.64:0/3524877824 learned_addr learned my addr 172.21.15.64:0/3524877824 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:22:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.094+0000 7fb4167fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca1a73ccc0 0x55ca1a74d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.094+0000 7fb416ffd700 1 --2- 172.21.15.64:0/3524877824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ca1a73c300 0x55ca1a74cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.094+0000 7fb4177fe700 1 -- 172.21.15.64:0/3524877824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ca1a73c300 msgr2=0x55ca1a74cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:22:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.094+0000 7fb4177fe700 1 --2- 172.21.15.64:0/3524877824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ca1a73c300 0x55ca1a74cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.094+0000 7fb4177fe700 1 -- 172.21.15.64:0/3524877824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca1a73ccc0 msgr2=0x55ca1a74d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:22:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.094+0000 7fb4177fe700 1 --2- 172.21.15.64:0/3524877824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca1a73ccc0 0x55ca1a74d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.094+0000 7fb4177fe700 1 -- 172.21.15.64:0/3524877824 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb410011de0 con 0x55ca1a7421a0 2023-09-13T03:22:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.094+0000 7fb416ffd700 1 --2- 172.21.15.64:0/3524877824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ca1a73c300 0x55ca1a74cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:22:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.094+0000 7fb4177fe700 1 --2- 172.21.15.64:0/3524877824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca1a7421a0 0x55ca1a749700 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb410018da0 tx=0x7fb410004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=582fa8d1e12ba3c2 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:22:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.094+0000 7fb3faff5700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:22:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.095+0000 7fb3faff5700 1 -- 172.21.15.64:0/3524877824 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb410020040 con 0x55ca1a7421a0 2023-09-13T03:22:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.095+0000 7fb430d69b80 1 -- 172.21.15.64:0/3524877824 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb408004390 con 0x55ca1a7421a0 2023-09-13T03:22:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.095+0000 7fb3faff5700 1 -- 172.21.15.64:0/3524877824 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fb410007d20 con 0x55ca1a7421a0 2023-09-13T03:22:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.095+0000 7fb3faff5700 1 -- 172.21.15.64:0/3524877824 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb41000e040 con 0x55ca1a7421a0 2023-09-13T03:22:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.095+0000 7fb430d69b80 10 client.26882 fetch_fsmap learned FSMap version 57 2023-09-13T03:22:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.095+0000 7fb430d69b80 1 -- 172.21.15.64:0/3524877824 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x55ca1a4a3c00 con 0x55ca1a7421a0 2023-09-13T03:22:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.095+0000 7fb3faff5700 1 -- 172.21.15.64:0/3524877824 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb408004390 con 0x55ca1a7421a0 2023-09-13T03:22:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.095+0000 7fb3faff5700 1 -- 172.21.15.64:0/3524877824 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 57) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fb4100044d0 con 0x55ca1a7421a0 2023-09-13T03:22:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.095+0000 7fb430d69b80 10 client.26882 fetch_fsmap finished waiting for FSMap version 57 2023-09-13T03:22:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.095+0000 7fb430d69b80 10 client.26882 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:22:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.095+0000 7fb430d69b80 20 client.26882 populate_metadata read hostname 'smithi064' 2023-09-13T03:22:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.095+0000 7fb430d69b80 1 --2- 172.21.15.64:0/3524877824 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55ca1a7ab400 0x55ca1a7ad820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.095+0000 7fb430d69b80 4 client.26882 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:22:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.095+0000 7fb430d69b80 1 -- 172.21.15.64:0/3524877824 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ca1a6878c0 con 0x55ca1a7ab400 2023-09-13T03:22:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.096+0000 7fb416ffd700 1 --2- 172.21.15.64:0/3524877824 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55ca1a7ab400 0x55ca1a7ad820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.096+0000 7fb3faff5700 10 client.26882 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:22:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.096+0000 7fb416ffd700 1 --2- 172.21.15.64:0/3524877824 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55ca1a7ab400 0x55ca1a7ad820 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc3252eec6ff6662 server_cookie=c4918b0a4c3ba902 in_seq=0 out_seq=0 2023-09-13T03:22:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.119+0000 7fb3faff5700 1 -- 172.21.15.64:0/3524877824 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ca1a6878c0 con 0x55ca1a7ab400 2023-09-13T03:22:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.119+0000 7fb3faff5700 10 client.26882 handle_command_reply: tid=0 2023-09-13T03:22:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.119+0000 7fb430d69b80 1 client.26882 shutdown 2023-09-13T03:22:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.119+0000 7fb430d69b80 1 -- 172.21.15.64:0/3524877824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca1a7421a0 msgr2=0x55ca1a749700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.119+0000 7fb430d69b80 1 --2- 172.21.15.64:0/3524877824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca1a7421a0 0x55ca1a749700 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb410018da0 tx=0x7fb410004b40 comp rx=0 tx=0).stop 2023-09-13T03:22:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.119+0000 7fb430d69b80 1 -- 172.21.15.64:0/3524877824 shutdown_connections 2023-09-13T03:22:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.119+0000 7fb430d69b80 1 --2- 172.21.15.64:0/3524877824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca1a7421a0 0x55ca1a749700 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.119+0000 7fb430d69b80 1 --2- 172.21.15.64:0/3524877824 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55ca1a7ab400 0x55ca1a7ad820 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.119+0000 7fb430d69b80 1 --2- 172.21.15.64:0/3524877824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ca1a73c300 0x55ca1a74cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.119+0000 7fb430d69b80 1 --2- 172.21.15.64:0/3524877824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ca1a73ccc0 0x55ca1a74d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.119+0000 7fb3faff5700 0 client.26882 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:22:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.119+0000 7fb430d69b80 1 -- 172.21.15.64:0/3524877824 >> 172.21.15.64:0/3524877824 conn(0x55ca1a6b1030 msgr2=0x55ca1a7410b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:22:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.119+0000 7fb430d69b80 1 -- 172.21.15.64:0/3524877824 shutdown_connections 2023-09-13T03:22:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.120+0000 7fb430d69b80 1 -- 172.21.15.64:0/3524877824 wait complete. 2023-09-13T03:22:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.120+0000 7fb430d69b80 20 client.26882 trim_cache size 0 max 16384 2023-09-13T03:22:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:22:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:22:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:22:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:22:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.120+0000 7fb41d3ff700 1 -- 172.21.15.64:0/2020586202 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fb4000a0830 msgr2=0x7fb4000a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.120+0000 7fb41d3ff700 1 --2- 172.21.15.64:0/2020586202 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fb4000a0830 0x7fb4000a2cf0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb410009ec0 tx=0x7fb410011c60 comp rx=0 tx=0).stop 2023-09-13T03:22:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.120+0000 7fb41d3ff700 1 -- 172.21.15.64:0/2020586202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4180732f0 msgr2=0x7fb41819ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.120+0000 7fb41d3ff700 1 --2- 172.21.15.64:0/2020586202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4180732f0 0x7fb41819ee80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb40c005910 tx=0x7fb40c004710 comp rx=0 tx=0).stop 2023-09-13T03:22:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.120+0000 7fb41d3ff700 1 -- 172.21.15.64:0/2020586202 shutdown_connections 2023-09-13T03:22:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.120+0000 7fb41d3ff700 1 --2- 172.21.15.64:0/2020586202 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fb4000a0830 0x7fb4000a2cf0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.120+0000 7fb41d3ff700 1 --2- 172.21.15.64:0/2020586202 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb41810ce60 0x7fb4181987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.120+0000 7fb41d3ff700 1 --2- 172.21.15.64:0/2020586202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4180729d0 0x7fb41819e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.120+0000 7fb41d3ff700 1 --2- 172.21.15.64:0/2020586202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4180732f0 0x7fb41819ee80 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.120+0000 7fb41d3ff700 1 -- 172.21.15.64:0/2020586202 >> 172.21.15.64:0/2020586202 conn(0x7fb4180fc0d0 msgr2=0x7fb41810efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:22:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.121+0000 7fb41d3ff700 1 -- 172.21.15.64:0/2020586202 shutdown_connections 2023-09-13T03:22:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:28.121+0000 7fb41d3ff700 1 -- 172.21.15.64:0/2020586202 wait complete. 2023-09-13T03:22:28.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:22:27 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:22:28.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:22:27 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:22:28.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:22:27 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:22:28.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:22:27 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:22:28.938 INFO:tasks.workunit.client.0.smithi064.stdout: 1 196506 1.96 MB/sec execute 393 sec latency 1503.965 ms 2023-09-13T03:22:28.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:22:28 smithi142 ceph-mon[134082]: pgmap v288: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 58 KiB/s wr, 19 op/s 2023-09-13T03:22:28.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:22:28 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:22:28.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:22:28 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:22:28.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:22:28 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:22:29.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:22:28 smithi170 ceph-mon[134328]: pgmap v288: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 58 KiB/s wr, 19 op/s 2023-09-13T03:22:29.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:22:28 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:22:29.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:22:28 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:22:29.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:22:28 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:22:29.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:22:28 smithi064 ceph-mon[97675]: pgmap v288: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 58 KiB/s wr, 19 op/s 2023-09-13T03:22:29.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:22:28 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:22:29.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:22:28 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:22:29.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:22:28 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:22:29.938 INFO:tasks.workunit.client.0.smithi064.stdout: 1 196788 1.95 MB/sec execute 394 sec latency 652.706 ms 2023-09-13T03:22:30.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:22:30 smithi170 ceph-mon[134328]: pgmap v289: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 58 KiB/s wr, 19 op/s 2023-09-13T03:22:30.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:22:30 smithi064 ceph-mon[97675]: pgmap v289: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 58 KiB/s wr, 19 op/s 2023-09-13T03:22:30.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:22:30 smithi142 ceph-mon[134082]: pgmap v289: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 58 KiB/s wr, 19 op/s 2023-09-13T03:22:30.938 INFO:tasks.workunit.client.0.smithi064.stdout: 1 198023 1.97 MB/sec execute 395 sec latency 1067.642 ms 2023-09-13T03:22:31.938 INFO:tasks.workunit.client.0.smithi064.stdout: 1 199240 1.98 MB/sec execute 396 sec latency 19.071 ms 2023-09-13T03:22:32.659 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:22:32 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:22:32.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:22:32 smithi170 ceph-mon[134328]: pgmap v290: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 390 KiB/s wr, 70 op/s 2023-09-13T03:22:32.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:22:32 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:22:32.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:22:32 smithi064 ceph-mon[97675]: pgmap v290: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 390 KiB/s wr, 70 op/s 2023-09-13T03:22:32.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:22:32 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:22:32.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:22:32 smithi142 ceph-mon[134082]: pgmap v290: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 390 KiB/s wr, 70 op/s 2023-09-13T03:22:32.938 INFO:tasks.workunit.client.0.smithi064.stdout: 1 200331 1.99 MB/sec execute 397 sec latency 119.088 ms 2023-09-13T03:22:33.938 INFO:tasks.workunit.client.0.smithi064.stdout: 1 200331 1.98 MB/sec execute 398 sec latency 1119.204 ms 2023-09-13T03:22:34.939 INFO:tasks.workunit.client.0.smithi064.stdout: 1 200331 1.98 MB/sec execute 399 sec latency 2119.294 ms 2023-09-13T03:22:34.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:22:34 smithi142 ceph-mon[134082]: pgmap v291: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 379 KiB/s wr, 64 op/s 2023-09-13T03:22:35.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:22:34 smithi170 ceph-mon[134328]: pgmap v291: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 379 KiB/s wr, 64 op/s 2023-09-13T03:22:35.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:22:34 smithi064 ceph-mon[97675]: pgmap v291: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 379 KiB/s wr, 64 op/s 2023-09-13T03:22:35.939 INFO:tasks.workunit.client.0.smithi064.stdout: 1 201524 1.99 MB/sec execute 400 sec latency 2540.360 ms 2023-09-13T03:22:36.939 INFO:tasks.workunit.client.0.smithi064.stdout: 1 202741 2.00 MB/sec execute 401 sec latency 11.371 ms 2023-09-13T03:22:36.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:22:36 smithi142 ceph-mon[134082]: pgmap v292: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 991 KiB/s wr, 118 op/s 2023-09-13T03:22:37.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:22:36 smithi170 ceph-mon[134328]: pgmap v292: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 991 KiB/s wr, 118 op/s 2023-09-13T03:22:37.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:22:36 smithi064 ceph-mon[97675]: pgmap v292: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 991 KiB/s wr, 118 op/s 2023-09-13T03:22:37.939 INFO:tasks.workunit.client.0.smithi064.stdout: 1 202852 2.00 MB/sec execute 402 sec latency 903.686 ms 2023-09-13T03:22:38.939 INFO:tasks.workunit.client.0.smithi064.stdout: 1 202976 1.99 MB/sec execute 403 sec latency 1496.979 ms 2023-09-13T03:22:38.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:22:38 smithi142 ceph-mon[134082]: pgmap v293: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 944 KiB/s wr, 105 op/s 2023-09-13T03:22:39.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:22:38 smithi170 ceph-mon[134328]: pgmap v293: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 944 KiB/s wr, 105 op/s 2023-09-13T03:22:39.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:22:38 smithi064 ceph-mon[97675]: pgmap v293: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 944 KiB/s wr, 105 op/s 2023-09-13T03:22:39.939 INFO:tasks.workunit.client.0.smithi064.stdout: 1 202976 1.99 MB/sec execute 404 sec latency 1290.280 ms 2023-09-13T03:22:40.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:22:40 smithi170 ceph-mon[134328]: pgmap v294: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 944 KiB/s wr, 105 op/s 2023-09-13T03:22:40.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:22:40 smithi064 ceph-mon[97675]: pgmap v294: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 944 KiB/s wr, 105 op/s 2023-09-13T03:22:40.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:22:40 smithi142 ceph-mon[134082]: pgmap v294: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 944 KiB/s wr, 105 op/s 2023-09-13T03:22:40.939 INFO:tasks.workunit.client.0.smithi064.stdout: 1 202976 1.98 MB/sec execute 405 sec latency 2290.353 ms 2023-09-13T03:22:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.281+0000 7f7af5bff700 1 -- 172.21.15.64:0/1812179677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7af0106660 msgr2=0x7f7af0106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.281+0000 7f7af5bff700 1 --2- 172.21.15.64:0/1812179677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7af0106660 0x7f7af0106a40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0009920 tx=0x7f7ae0009c30 comp rx=0 tx=0).stop 2023-09-13T03:22:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.281+0000 7f7af5bff700 1 -- 172.21.15.64:0/1812179677 shutdown_connections 2023-09-13T03:22:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.281+0000 7f7af5bff700 1 --2- 172.21.15.64:0/1812179677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7af0106660 0x7f7af0106a40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.281+0000 7f7af5bff700 1 --2- 172.21.15.64:0/1812179677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7af0100f60 0x7f7af010d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.281+0000 7f7af5bff700 1 --2- 172.21.15.64:0/1812179677 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7af01005a0 0x7f7af0100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.281+0000 7f7af5bff700 1 -- 172.21.15.64:0/1812179677 >> 172.21.15.64:0/1812179677 conn(0x7f7af00fc050 msgr2=0x7f7af00fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:22:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.281+0000 7f7af5bff700 1 -- 172.21.15.64:0/1812179677 shutdown_connections 2023-09-13T03:22:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.281+0000 7f7af5bff700 1 -- 172.21.15.64:0/1812179677 wait complete. 2023-09-13T03:22:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.282+0000 7f7af5bff700 1 Processor -- start 2023-09-13T03:22:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.282+0000 7f7af5bff700 1 -- start start 2023-09-13T03:22:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.283+0000 7f7af5bff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7af01005a0 0x7f7af0072ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.283+0000 7f7af5bff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7af0100f60 0x7f7af0073020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.283+0000 7f7af5bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7af0106660 0x7f7af01a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.283+0000 7f7af5bff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af01a4e20 con 0x7f7af01005a0 2023-09-13T03:22:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.283+0000 7f7af5bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af01a4f60 con 0x7f7af0106660 2023-09-13T03:22:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.283+0000 7f7af5bff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af01a50d0 con 0x7f7af0100f60 2023-09-13T03:22:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.283+0000 7f7aef7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7af01005a0 0x7f7af0072ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.283+0000 7f7aeeffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7af0100f60 0x7f7af0073020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.283+0000 7f7aeeffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7af0100f60 0x7f7af0073020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:55850/0 (socket says 172.21.15.64:55850) 2023-09-13T03:22:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.283+0000 7f7aeeffd700 1 -- 172.21.15.64:0/292920506 learned_addr learned my addr 172.21.15.64:0/292920506 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:22:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.283+0000 7f7aeffff700 1 --2- 172.21.15.64:0/292920506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7af0106660 0x7f7af01a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.284+0000 7f7aeeffd700 1 -- 172.21.15.64:0/292920506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7af0106660 msgr2=0x7f7af01a48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.284+0000 7f7aeeffd700 1 --2- 172.21.15.64:0/292920506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7af0106660 0x7f7af01a48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.284+0000 7f7aeeffd700 1 -- 172.21.15.64:0/292920506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7af01005a0 msgr2=0x7f7af0072ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.284+0000 7f7aeeffd700 1 --2- 172.21.15.64:0/292920506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7af01005a0 0x7f7af0072ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.284+0000 7f7aeeffd700 1 -- 172.21.15.64:0/292920506 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae0009520 con 0x7f7af0100f60 2023-09-13T03:22:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.284+0000 7f7aef7fe700 1 --2- 172.21.15.64:0/292920506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7af01005a0 0x7f7af0072ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:22:41.290 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.284+0000 7f7aeeffd700 1 --2- 172.21.15.64:0/292920506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7af0100f60 0x7f7af0073020 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7ae400e9c0 tx=0x7f7ae400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:22:41.290 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.284+0000 7f7aecff9700 1 -- 172.21.15.64:0/292920506 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4004a30 con 0x7f7af0100f60 2023-09-13T03:22:41.290 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.284+0000 7f7af5bff700 1 -- 172.21.15.64:0/292920506 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7af01a53b0 con 0x7f7af0100f60 2023-09-13T03:22:41.290 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.284+0000 7f7aecff9700 1 -- 172.21.15.64:0/292920506 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ae400cc80 con 0x7f7af0100f60 2023-09-13T03:22:41.290 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.284+0000 7f7aecff9700 1 -- 172.21.15.64:0/292920506 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae40054e0 con 0x7f7af0100f60 2023-09-13T03:22:41.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.284+0000 7f7af5bff700 1 -- 172.21.15.64:0/292920506 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7af01a58c0 con 0x7f7af0100f60 2023-09-13T03:22:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.288+0000 7f7aecff9700 1 -- 172.21.15.64:0/292920506 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f7ae4004b90 con 0x7f7af0100f60 2023-09-13T03:22:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.289+0000 7f7b095a6b80 1 Processor -- start 2023-09-13T03:22:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.289+0000 7f7b095a6b80 1 -- start start 2023-09-13T03:22:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.289+0000 7f7aecff9700 1 --2- 172.21.15.64:0/292920506 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f7ad80a07b0 0x7f7ad80a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.290+0000 7f7aecff9700 1 -- 172.21.15.64:0/292920506 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f7ae4014070 con 0x7f7af0100f60 2023-09-13T03:22:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.290+0000 7f7aeffff700 1 --2- 172.21.15.64:0/292920506 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f7ad80a07b0 0x7f7ad80a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.290+0000 7f7b095a6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5813891a0 0x55f581389580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.290+0000 7f7b095a6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f581383300 0x55f581383780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.290+0000 7f7b095a6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581383cc0 0x55f58138ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.290+0000 7f7b095a6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5812dc6d0 con 0x55f581383300 2023-09-13T03:22:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.290+0000 7f7b095a6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5812d9ff0 con 0x55f581383cc0 2023-09-13T03:22:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.290+0000 7f7b095a6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5812d58e0 con 0x55f5813891a0 2023-09-13T03:22:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.290+0000 7f7aeeffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f581383300 0x55f581383780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.290+0000 7f7aef7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5813891a0 0x55f581389580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.290+0000 7f7aeeffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f581383300 0x55f581383780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34914/0 (socket says 172.21.15.64:34914) 2023-09-13T03:22:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.290+0000 7f7aeeffd700 1 -- 172.21.15.64:0/1083775250 learned_addr learned my addr 172.21.15.64:0/1083775250 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:22:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.290+0000 7f7aeffff700 1 --2- 172.21.15.64:0/1083775250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581383cc0 0x55f58138ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.291+0000 7f7aeeffd700 1 -- 172.21.15.64:0/1083775250 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5813891a0 msgr2=0x55f581389580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:22:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.291+0000 7f7aeeffd700 1 --2- 172.21.15.64:0/1083775250 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5813891a0 0x55f581389580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.291+0000 7f7aeeffd700 1 -- 172.21.15.64:0/1083775250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581383cc0 msgr2=0x55f58138ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:22:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.291+0000 7f7aeeffd700 1 --2- 172.21.15.64:0/1083775250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581383cc0 0x55f58138ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:41.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.291+0000 7f7aeffff700 1 --2- 172.21.15.64:0/292920506 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f7ad80a07b0 0x7f7ad80a2c70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7ae800f590 tx=0x7f7ae800b040 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:22:41.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.291+0000 7f7aeeffd700 1 -- 172.21.15.64:0/1083775250 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f581101d80 con 0x55f581383300 2023-09-13T03:22:41.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.291+0000 7f7aeeffd700 1 --2- 172.21.15.64:0/1083775250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f581383300 0x55f581383780 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4095a20 tx=0x7f7ae4099400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24c07c779f55ca0a server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:22:41.298 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.291+0000 7f7ad5ffb700 1 -- 172.21.15.64:0/1083775250 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4095040 con 0x55f581383300 2023-09-13T03:22:41.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.291+0000 7f7ad5ffb700 1 -- 172.21.15.64:0/1083775250 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ae409d170 con 0x55f581383300 2023-09-13T03:22:41.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.291+0000 7f7ad5ffb700 1 -- 172.21.15.64:0/1083775250 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4095c30 con 0x55f581383300 2023-09-13T03:22:41.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.292+0000 7f7b095a6b80 1 -- 172.21.15.64:0/1083775250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f581383300 msgr2=0x55f581383780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:41.299 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.292+0000 7f7b095a6b80 1 --2- 172.21.15.64:0/1083775250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f581383300 0x55f581383780 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4095a20 tx=0x7f7ae4099400 comp rx=0 tx=0).stop 2023-09-13T03:22:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.292+0000 7f7b095a6b80 1 -- 172.21.15.64:0/1083775250 shutdown_connections 2023-09-13T03:22:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.292+0000 7f7b095a6b80 1 --2- 172.21.15.64:0/1083775250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581383cc0 0x55f58138ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.292+0000 7f7b095a6b80 1 --2- 172.21.15.64:0/1083775250 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5813891a0 0x55f581389580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.292+0000 7f7b095a6b80 1 --2- 172.21.15.64:0/1083775250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f581383300 0x55f581383780 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.292+0000 7f7b095a6b80 1 -- 172.21.15.64:0/1083775250 >> 172.21.15.64:0/1083775250 conn(0x55f5812f8030 msgr2=0x55f58137e560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:22:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.292+0000 7f7b095a6b80 1 -- 172.21.15.64:0/1083775250 shutdown_connections 2023-09-13T03:22:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.292+0000 7f7b095a6b80 1 -- 172.21.15.64:0/1083775250 wait complete. 2023-09-13T03:22:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.293+0000 7f7b095a6b80 1 -- start start 2023-09-13T03:22:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.293+0000 7f7b095a6b80 1 Processor -- start 2023-09-13T03:22:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.294+0000 7f7b095a6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581383300 0x55f58138fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.294+0000 7f7b095a6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f581383cc0 0x55f58138d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.294+0000 7f7b095a6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5813891a0 0x55f58138d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.294+0000 7f7b095a6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f581273b10 con 0x55f581383cc0 2023-09-13T03:22:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.294+0000 7f7b095a6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f581273810 con 0x55f581383300 2023-09-13T03:22:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.294+0000 7f7b095a6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5811578c0 con 0x55f5813891a0 2023-09-13T03:22:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.294+0000 7f7b095a6b80 10 client.0 _ll_register_callbacks cb 0x55f5811bda20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:22:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.294+0000 7f7aeeffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f581383cc0 0x55f58138d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.294+0000 7f7aef7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581383300 0x55f58138fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.294+0000 7f7aeeffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f581383cc0 0x55f58138d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34916/0 (socket says 172.21.15.64:34916) 2023-09-13T03:22:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.294+0000 7f7aeeffd700 1 -- 172.21.15.64:0/3415595678 learned_addr learned my addr 172.21.15.64:0/3415595678 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:22:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.294+0000 7f7aeffff700 1 --2- 172.21.15.64:0/3415595678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5813891a0 0x55f58138d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.295+0000 7f7aeeffd700 1 -- 172.21.15.64:0/3415595678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5813891a0 msgr2=0x55f58138d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:22:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.295+0000 7f7aeeffd700 1 --2- 172.21.15.64:0/3415595678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5813891a0 0x55f58138d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.295+0000 7f7aeeffd700 1 -- 172.21.15.64:0/3415595678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581383300 msgr2=0x55f58138fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:22:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.295+0000 7f7aeeffd700 1 --2- 172.21.15.64:0/3415595678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581383300 0x55f58138fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.295+0000 7f7aeeffd700 1 -- 172.21.15.64:0/3415595678 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7af01a58c0 con 0x55f581383cc0 2023-09-13T03:22:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.295+0000 7f7aeeffd700 1 --2- 172.21.15.64:0/3415595678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f581383cc0 0x55f58138d110 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4020000 tx=0x7f7ae4005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c13eee2f10163d4c server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:22:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.295+0000 7f7aceffd700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:22:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.295+0000 7f7aceffd700 1 -- 172.21.15.64:0/3415595678 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4095040 con 0x55f581383cc0 2023-09-13T03:22:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.295+0000 7f7b095a6b80 1 -- 172.21.15.64:0/3415595678 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f5812ea840 con 0x55f581383cc0 2023-09-13T03:22:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.295+0000 7f7aceffd700 1 -- 172.21.15.64:0/3415595678 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f7ae40ace40 con 0x55f581383cc0 2023-09-13T03:22:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.295+0000 7f7aceffd700 1 -- 172.21.15.64:0/3415595678 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ae409d350 con 0x55f581383cc0 2023-09-13T03:22:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.295+0000 7f7b095a6b80 10 client.16857 fetch_fsmap learned FSMap version 58 2023-09-13T03:22:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.295+0000 7f7b095a6b80 1 -- 172.21.15.64:0/3415595678 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55f5810eac00 con 0x55f581383cc0 2023-09-13T03:22:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.296+0000 7f7aceffd700 1 -- 172.21.15.64:0/3415595678 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f5812ea840 con 0x55f581383cc0 2023-09-13T03:22:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.296+0000 7f7aceffd700 1 -- 172.21.15.64:0/3415595678 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 58) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f7ae4099810 con 0x55f581383cc0 2023-09-13T03:22:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.296+0000 7f7b095a6b80 10 client.16857 fetch_fsmap finished waiting for FSMap version 58 2023-09-13T03:22:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.296+0000 7f7b095a6b80 10 client.16857 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:22:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.296+0000 7f7b095a6b80 20 client.16857 populate_metadata read hostname 'smithi064' 2023-09-13T03:22:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.296+0000 7f7b095a6b80 1 --2- 172.21.15.64:0/3415595678 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f58139cf00 0x55f58139f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.296+0000 7f7b095a6b80 4 client.16857 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:22:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.296+0000 7f7b095a6b80 1 -- 172.21.15.64:0/3415595678 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f5812ce8c0 con 0x55f58139cf00 2023-09-13T03:22:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.296+0000 7f7aef7fe700 1 --2- 172.21.15.64:0/3415595678 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f58139cf00 0x55f58139f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.297+0000 7f7aef7fe700 1 --2- 172.21.15.64:0/3415595678 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f58139cf00 0x55f58139f2f0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6ee641bfe26a0a3 server_cookie=450d2d86b41c49ce in_seq=0 out_seq=0 2023-09-13T03:22:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.297+0000 7f7aceffd700 10 client.16857 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:22:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.298+0000 7f7aceffd700 1 -- 172.21.15.64:0/3415595678 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f5812ce8c0 con 0x55f58139cf00 2023-09-13T03:22:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.298+0000 7f7aceffd700 10 client.16857 handle_command_reply: tid=0 2023-09-13T03:22:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.298+0000 7f7b095a6b80 1 client.16857 shutdown 2023-09-13T03:22:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.298+0000 7f7b095a6b80 1 -- 172.21.15.64:0/3415595678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f581383cc0 msgr2=0x55f58138d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.298+0000 7f7b095a6b80 1 --2- 172.21.15.64:0/3415595678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f581383cc0 0x55f58138d110 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4020000 tx=0x7f7ae4005870 comp rx=0 tx=0).stop 2023-09-13T03:22:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.298+0000 7f7b095a6b80 1 -- 172.21.15.64:0/3415595678 shutdown_connections 2023-09-13T03:22:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.298+0000 7f7b095a6b80 1 --2- 172.21.15.64:0/3415595678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581383300 0x55f58138fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.298+0000 7f7b095a6b80 1 --2- 172.21.15.64:0/3415595678 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f58139cf00 0x55f58139f2f0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.298+0000 7f7b095a6b80 1 --2- 172.21.15.64:0/3415595678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5813891a0 0x55f58138d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.298+0000 7f7b095a6b80 1 --2- 172.21.15.64:0/3415595678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f581383cc0 0x55f58138d110 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.298+0000 7f7b095a6b80 1 -- 172.21.15.64:0/3415595678 >> 172.21.15.64:0/3415595678 conn(0x55f5812f8030 msgr2=0x55f58137d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:22:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.298+0000 7f7aceffd700 0 client.16857 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:22:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.298+0000 7f7b095a6b80 1 -- 172.21.15.64:0/3415595678 shutdown_connections 2023-09-13T03:22:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.298+0000 7f7b095a6b80 1 -- 172.21.15.64:0/3415595678 wait complete. 2023-09-13T03:22:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.298+0000 7f7b095a6b80 20 client.16857 trim_cache size 0 max 16384 2023-09-13T03:22:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.314+0000 7f7b095a6b80 1 Processor -- start 2023-09-13T03:22:41.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.314+0000 7f7b095a6b80 1 -- start start 2023-09-13T03:22:41.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.314+0000 7f7b095a6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f581383300 0x55f581389580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:41.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.314+0000 7f7b095a6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581383cc0 0x55f581394520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:41.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.314+0000 7f7b095a6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f5813891a0 0x55f58137df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:41.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.314+0000 7f7b095a6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5813889a0 con 0x55f5813891a0 2023-09-13T03:22:41.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.314+0000 7f7b095a6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f581394a60 con 0x55f581383cc0 2023-09-13T03:22:41.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.314+0000 7f7b095a6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f58138b670 con 0x55f581383300 2023-09-13T03:22:41.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.315+0000 7f7aeffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f5813891a0 0x55f58137df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:41.322 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.315+0000 7f7aeffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f5813891a0 0x55f58137df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34930/0 (socket says 172.21.15.64:34930) 2023-09-13T03:22:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.315+0000 7f7aeffff700 1 -- 172.21.15.64:0/3117060545 learned_addr learned my addr 172.21.15.64:0/3117060545 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:22:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.315+0000 7f7aeeffd700 1 --2- 172.21.15.64:0/3117060545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581383cc0 0x55f581394520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.315+0000 7f7aef7fe700 1 --2- 172.21.15.64:0/3117060545 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f581383300 0x55f581389580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.315+0000 7f7aeffff700 1 -- 172.21.15.64:0/3117060545 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f581383300 msgr2=0x55f581389580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:22:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.315+0000 7f7aeffff700 1 --2- 172.21.15.64:0/3117060545 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f581383300 0x55f581389580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:41.324 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.315+0000 7f7aeffff700 1 -- 172.21.15.64:0/3117060545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581383cc0 msgr2=0x55f581394520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:22:41.324 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.315+0000 7f7aeffff700 1 --2- 172.21.15.64:0/3117060545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581383cc0 0x55f581394520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:41.324 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.315+0000 7f7aeffff700 1 -- 172.21.15.64:0/3117060545 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5811281f0 con 0x55f5813891a0 2023-09-13T03:22:41.324 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.315+0000 7f7aeeffd700 1 --2- 172.21.15.64:0/3117060545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581383cc0 0x55f581394520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:22:41.324 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.315+0000 7f7aeffff700 1 --2- 172.21.15.64:0/3117060545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f5813891a0 0x55f58137df20 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f7ae80093b0 tx=0x7f7ae8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3084730bcd291a89 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:22:41.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.316+0000 7f7ad67fc700 1 -- 172.21.15.64:0/3117060545 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae800e040 con 0x55f5813891a0 2023-09-13T03:22:41.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.316+0000 7f7ad67fc700 1 -- 172.21.15.64:0/3117060545 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ae800f040 con 0x55f5813891a0 2023-09-13T03:22:41.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.316+0000 7f7ad67fc700 1 -- 172.21.15.64:0/3117060545 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae8017440 con 0x55f5813891a0 2023-09-13T03:22:41.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.316+0000 7f7b095a6b80 1 -- 172.21.15.64:0/3117060545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f5813891a0 msgr2=0x55f58137df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:41.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.316+0000 7f7b095a6b80 1 --2- 172.21.15.64:0/3117060545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f5813891a0 0x55f58137df20 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f7ae80093b0 tx=0x7f7ae8019000 comp rx=0 tx=0).stop 2023-09-13T03:22:41.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.316+0000 7f7b095a6b80 1 -- 172.21.15.64:0/3117060545 shutdown_connections 2023-09-13T03:22:41.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.316+0000 7f7b095a6b80 1 --2- 172.21.15.64:0/3117060545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f5813891a0 0x55f58137df20 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:41.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.316+0000 7f7b095a6b80 1 --2- 172.21.15.64:0/3117060545 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f581383300 0x55f581389580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:41.326 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.316+0000 7f7b095a6b80 1 --2- 172.21.15.64:0/3117060545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581383cc0 0x55f581394520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:41.327 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.316+0000 7f7b095a6b80 1 -- 172.21.15.64:0/3117060545 >> 172.21.15.64:0/3117060545 conn(0x55f5812f8030 msgr2=0x55f5813871f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:22:41.327 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.316+0000 7f7b095a6b80 1 -- 172.21.15.64:0/3117060545 shutdown_connections 2023-09-13T03:22:41.327 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.316+0000 7f7b095a6b80 1 -- 172.21.15.64:0/3117060545 wait complete. 2023-09-13T03:22:41.327 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.317+0000 7f7b095a6b80 1 -- start start 2023-09-13T03:22:41.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.317+0000 7f7b095a6b80 1 Processor -- start 2023-09-13T03:22:41.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.317+0000 7f7b095a6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f581383300 0x55f581393d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:41.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.317+0000 7f7b095a6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581383cc0 0x55f581394240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:41.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.317+0000 7f7b095a6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f5813891a0 0x55f581392510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:41.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.318+0000 7f7b095a6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f581392ae0 con 0x55f5813891a0 2023-09-13T03:22:41.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.318+0000 7f7b095a6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f581392c50 con 0x55f581383cc0 2023-09-13T03:22:41.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.318+0000 7f7b095a6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f581392e10 con 0x55f581383300 2023-09-13T03:22:41.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.318+0000 7f7b095a6b80 10 client.0 _ll_register_callbacks cb 0x7f7ae80097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:22:41.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.318+0000 7f7aef7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f581383300 0x55f581393d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:41.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.318+0000 7f7aef7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f581383300 0x55f581393d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:55886/0 (socket says 172.21.15.64:55886) 2023-09-13T03:22:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.318+0000 7f7aef7fe700 1 -- 172.21.15.64:0/3037856181 learned_addr learned my addr 172.21.15.64:0/3037856181 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:22:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.318+0000 7f7aeffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f5813891a0 0x55f581392510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.318+0000 7f7aeeffd700 1 --2- 172.21.15.64:0/3037856181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581383cc0 0x55f581394240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.318+0000 7f7aef7fe700 1 -- 172.21.15.64:0/3037856181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581383cc0 msgr2=0x55f581394240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:22:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.318+0000 7f7aef7fe700 1 --2- 172.21.15.64:0/3037856181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581383cc0 0x55f581394240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.318+0000 7f7aef7fe700 1 -- 172.21.15.64:0/3037856181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f5813891a0 msgr2=0x55f581392510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:22:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.318+0000 7f7aef7fe700 1 --2- 172.21.15.64:0/3037856181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f5813891a0 0x55f581392510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.318+0000 7f7aef7fe700 1 -- 172.21.15.64:0/3037856181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f5810eac00 con 0x55f581383300 2023-09-13T03:22:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.319+0000 7f7aef7fe700 1 --2- 172.21.15.64:0/3037856181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f581383300 0x55f581393d00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0000c00 tx=0x7f7ae000f960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd3b60e815c4d299 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:22:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.319+0000 7f7aceffd700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:22:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.319+0000 7f7aceffd700 1 -- 172.21.15.64:0/3037856181 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae0029040 con 0x55f581383300 2023-09-13T03:22:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.319+0000 7f7b095a6b80 1 -- 172.21.15.64:0/3037856181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f58122d490 con 0x55f581383300 2023-09-13T03:22:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.319+0000 7f7aceffd700 1 -- 172.21.15.64:0/3037856181 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f7ae00052b0 con 0x55f581383300 2023-09-13T03:22:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.319+0000 7f7aceffd700 1 -- 172.21.15.64:0/3037856181 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ae001c040 con 0x55f581383300 2023-09-13T03:22:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.319+0000 7f7b095a6b80 10 client.26884 fetch_fsmap learned FSMap version 58 2023-09-13T03:22:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.319+0000 7f7b095a6b80 1 -- 172.21.15.64:0/3037856181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x7f7ae8009060 con 0x55f581383300 2023-09-13T03:22:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.319+0000 7f7aceffd700 1 -- 172.21.15.64:0/3037856181 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f58122d490 con 0x55f581383300 2023-09-13T03:22:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.320+0000 7f7aceffd700 1 -- 172.21.15.64:0/3037856181 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 58) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f7ae0015020 con 0x55f581383300 2023-09-13T03:22:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.320+0000 7f7b095a6b80 10 client.26884 fetch_fsmap finished waiting for FSMap version 58 2023-09-13T03:22:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.320+0000 7f7b095a6b80 10 client.26884 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:22:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.320+0000 7f7b095a6b80 20 client.26884 populate_metadata read hostname 'smithi064' 2023-09-13T03:22:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.320+0000 7f7b095a6b80 1 --2- 172.21.15.64:0/3037856181 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f5813f2060 0x55f5813f4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.320+0000 7f7b095a6b80 4 client.26884 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:22:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.320+0000 7f7b095a6b80 1 -- 172.21.15.64:0/3037856181 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f58128f050 con 0x55f5813f2060 2023-09-13T03:22:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.320+0000 7f7aeeffd700 1 --2- 172.21.15.64:0/3037856181 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f5813f2060 0x55f5813f4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.320+0000 7f7aceffd700 10 client.26884 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:22:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.320+0000 7f7aeeffd700 1 --2- 172.21.15.64:0/3037856181 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f5813f2060 0x55f5813f4480 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37d55078cbd7ea8c server_cookie=9383c6ec92e4e8b8 in_seq=0 out_seq=0 2023-09-13T03:22:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.350+0000 7f7aceffd700 1 -- 172.21.15.64:0/3037856181 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f58128f050 con 0x55f5813f2060 2023-09-13T03:22:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.350+0000 7f7aceffd700 10 client.26884 handle_command_reply: tid=0 2023-09-13T03:22:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.350+0000 7f7b095a6b80 1 client.26884 shutdown 2023-09-13T03:22:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.350+0000 7f7b095a6b80 1 -- 172.21.15.64:0/3037856181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f581383300 msgr2=0x55f581393d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.350+0000 7f7b095a6b80 1 --2- 172.21.15.64:0/3037856181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f581383300 0x55f581393d00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0000c00 tx=0x7f7ae000f960 comp rx=0 tx=0).stop 2023-09-13T03:22:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.350+0000 7f7b095a6b80 1 -- 172.21.15.64:0/3037856181 shutdown_connections 2023-09-13T03:22:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.350+0000 7f7b095a6b80 1 --2- 172.21.15.64:0/3037856181 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f5813f2060 0x55f5813f4480 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.350+0000 7f7b095a6b80 1 --2- 172.21.15.64:0/3037856181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f5813891a0 0x55f581392510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:41.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.350+0000 7f7b095a6b80 1 --2- 172.21.15.64:0/3037856181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f581383300 0x55f581393d00 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:41.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.350+0000 7f7b095a6b80 1 --2- 172.21.15.64:0/3037856181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f581383cc0 0x55f581394240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:41.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.350+0000 7f7aceffd700 0 client.26884 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:22:41.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.350+0000 7f7b095a6b80 1 -- 172.21.15.64:0/3037856181 >> 172.21.15.64:0/3037856181 conn(0x55f5812f8030 msgr2=0x55f5813822f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:22:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.350+0000 7f7b095a6b80 1 -- 172.21.15.64:0/3037856181 shutdown_connections 2023-09-13T03:22:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.350+0000 7f7b095a6b80 1 -- 172.21.15.64:0/3037856181 wait complete. 2023-09-13T03:22:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.350+0000 7f7b095a6b80 20 client.26884 trim_cache size 0 max 16384 2023-09-13T03:22:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:22:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:22:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:22:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:22:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.351+0000 7f7af5bff700 1 -- 172.21.15.64:0/292920506 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f7ad80a07b0 msgr2=0x7f7ad80a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.351+0000 7f7af5bff700 1 --2- 172.21.15.64:0/292920506 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f7ad80a07b0 0x7f7ad80a2c70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7ae800f590 tx=0x7f7ae800b040 comp rx=0 tx=0).stop 2023-09-13T03:22:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.351+0000 7f7af5bff700 1 -- 172.21.15.64:0/292920506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7af0100f60 msgr2=0x7f7af0073020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.351+0000 7f7af5bff700 1 --2- 172.21.15.64:0/292920506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7af0100f60 0x7f7af0073020 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7ae400e9c0 tx=0x7f7ae400ecd0 comp rx=0 tx=0).stop 2023-09-13T03:22:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.351+0000 7f7af5bff700 1 -- 172.21.15.64:0/292920506 shutdown_connections 2023-09-13T03:22:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.351+0000 7f7af5bff700 1 --2- 172.21.15.64:0/292920506 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f7ad80a07b0 0x7f7ad80a2c70 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.351+0000 7f7af5bff700 1 --2- 172.21.15.64:0/292920506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7af0106660 0x7f7af01a48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.351+0000 7f7af5bff700 1 --2- 172.21.15.64:0/292920506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7af01005a0 0x7f7af0072ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.351+0000 7f7af5bff700 1 --2- 172.21.15.64:0/292920506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7af0100f60 0x7f7af0073020 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.351+0000 7f7af5bff700 1 -- 172.21.15.64:0/292920506 >> 172.21.15.64:0/292920506 conn(0x7f7af00fc050 msgr2=0x7f7af01042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:22:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.351+0000 7f7af5bff700 1 -- 172.21.15.64:0/292920506 shutdown_connections 2023-09-13T03:22:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:41.351+0000 7f7af5bff700 1 -- 172.21.15.64:0/292920506 wait complete. 2023-09-13T03:22:41.939 INFO:tasks.workunit.client.0.smithi064.stdout: 1 202976 1.98 MB/sec execute 406 sec latency 3290.477 ms 2023-09-13T03:22:42.940 INFO:tasks.workunit.client.0.smithi064.stdout: 1 202976 1.97 MB/sec execute 407 sec latency 4290.604 ms 2023-09-13T03:22:42.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:22:42 smithi142 ceph-mon[134082]: pgmap v295: 97 pgs: 97 active+clean; 99 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 145 op/s 2023-09-13T03:22:43.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:22:42 smithi170 ceph-mon[134328]: pgmap v295: 97 pgs: 97 active+clean; 99 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 145 op/s 2023-09-13T03:22:43.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:22:42 smithi064 ceph-mon[97675]: pgmap v295: 97 pgs: 97 active+clean; 99 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 145 op/s 2023-09-13T03:22:43.939 INFO:tasks.workunit.client.0.smithi064.stdout: 1 203639 1.98 MB/sec execute 408 sec latency 4885.773 ms 2023-09-13T03:22:44.940 INFO:tasks.workunit.client.0.smithi064.stdout: 1 203878 1.98 MB/sec execute 409 sec latency 803.285 ms 2023-09-13T03:22:44.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:22:44 smithi142 ceph-mon[134082]: pgmap v296: 97 pgs: 97 active+clean; 99 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 845 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2023-09-13T03:22:45.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:22:44 smithi170 ceph-mon[134328]: pgmap v296: 97 pgs: 97 active+clean; 99 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 845 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2023-09-13T03:22:45.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:22:44 smithi064 ceph-mon[97675]: pgmap v296: 97 pgs: 97 active+clean; 99 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 845 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2023-09-13T03:22:45.940 INFO:tasks.workunit.client.0.smithi064.stdout: 1 205116 1.99 MB/sec execute 410 sec latency 1217.879 ms 2023-09-13T03:22:46.940 INFO:tasks.workunit.client.0.smithi064.stdout: 1 206349 2.00 MB/sec execute 411 sec latency 17.463 ms 2023-09-13T03:22:46.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:22:46 smithi142 ceph-mon[134082]: pgmap v297: 97 pgs: 97 active+clean; 98 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 131 op/s 2023-09-13T03:22:47.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:22:46 smithi170 ceph-mon[134328]: pgmap v297: 97 pgs: 97 active+clean; 98 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 131 op/s 2023-09-13T03:22:47.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:22:46 smithi064 ceph-mon[97675]: pgmap v297: 97 pgs: 97 active+clean; 98 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 131 op/s 2023-09-13T03:22:47.940 INFO:tasks.workunit.client.0.smithi064.stdout: 1 206520 2.00 MB/sec execute 412 sec latency 862.184 ms 2023-09-13T03:22:48.940 INFO:tasks.workunit.client.0.smithi064.stdout: 1 207225 2.00 MB/sec execute 413 sec latency 1452.501 ms 2023-09-13T03:22:48.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:22:48 smithi142 ceph-mon[134082]: pgmap v298: 97 pgs: 97 active+clean; 98 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 648 KiB/s wr, 77 op/s 2023-09-13T03:22:49.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:22:48 smithi170 ceph-mon[134328]: pgmap v298: 97 pgs: 97 active+clean; 98 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 648 KiB/s wr, 77 op/s 2023-09-13T03:22:49.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:22:48 smithi064 ceph-mon[97675]: pgmap v298: 97 pgs: 97 active+clean; 98 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 648 KiB/s wr, 77 op/s 2023-09-13T03:22:49.940 INFO:tasks.workunit.client.0.smithi064.stdout: 1 207421 2.00 MB/sec execute 414 sec latency 840.151 ms 2023-09-13T03:22:50.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:22:50 smithi170 ceph-mon[134328]: pgmap v299: 97 pgs: 97 active+clean; 98 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 648 KiB/s wr, 77 op/s 2023-09-13T03:22:50.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:22:50 smithi064 ceph-mon[97675]: pgmap v299: 97 pgs: 97 active+clean; 98 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 648 KiB/s wr, 77 op/s 2023-09-13T03:22:50.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:22:50 smithi142 ceph-mon[134082]: pgmap v299: 97 pgs: 97 active+clean; 98 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 648 KiB/s wr, 77 op/s 2023-09-13T03:22:50.940 INFO:tasks.workunit.client.0.smithi064.stdout: 1 208621 2.01 MB/sec execute 415 sec latency 1253.571 ms 2023-09-13T03:22:51.666 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:22:51 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:22:51.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:22:51 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:22:51.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:22:51 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:22:51.940 INFO:tasks.workunit.client.0.smithi064.stdout: 1 209872 2.02 MB/sec execute 416 sec latency 18.130 ms 2023-09-13T03:22:52.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:22:52 smithi170 ceph-mon[134328]: pgmap v300: 97 pgs: 97 active+clean; 101 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 980 KiB/s wr, 123 op/s 2023-09-13T03:22:52.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:22:52 smithi064 ceph-mon[97675]: pgmap v300: 97 pgs: 97 active+clean; 101 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 980 KiB/s wr, 123 op/s 2023-09-13T03:22:52.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:22:52 smithi142 ceph-mon[134082]: pgmap v300: 97 pgs: 97 active+clean; 101 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 980 KiB/s wr, 123 op/s 2023-09-13T03:22:52.940 INFO:tasks.workunit.client.0.smithi064.stdout: 1 209945 2.02 MB/sec execute 417 sec latency 960.402 ms 2023-09-13T03:22:53.941 INFO:tasks.workunit.client.0.smithi064.stdout: 1 210001 2.01 MB/sec execute 418 sec latency 1553.068 ms 2023-09-13T03:22:54.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.512+0000 7f1f32fff700 1 -- 172.21.15.64:0/2753958961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f2c106670 msgr2=0x7f1f2c106a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:54.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.512+0000 7f1f32fff700 1 --2- 172.21.15.64:0/2753958961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f2c106670 0x7f1f2c106a50 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f1f200099c0 tx=0x7f1f20009cd0 comp rx=0 tx=0).stop 2023-09-13T03:22:54.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.512+0000 7f1f32fff700 1 -- 172.21.15.64:0/2753958961 shutdown_connections 2023-09-13T03:22:54.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.512+0000 7f1f32fff700 1 --2- 172.21.15.64:0/2753958961 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f2c100f70 0x7f1f2c10d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:54.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.512+0000 7f1f32fff700 1 --2- 172.21.15.64:0/2753958961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f2c106670 0x7f1f2c106a50 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:54.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.512+0000 7f1f32fff700 1 --2- 172.21.15.64:0/2753958961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f2c1005b0 0x7f1f2c100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:54.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.512+0000 7f1f32fff700 1 -- 172.21.15.64:0/2753958961 >> 172.21.15.64:0/2753958961 conn(0x7f1f2c0fc060 msgr2=0x7f1f2c0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:22:54.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.512+0000 7f1f32fff700 1 -- 172.21.15.64:0/2753958961 shutdown_connections 2023-09-13T03:22:54.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.512+0000 7f1f32fff700 1 -- 172.21.15.64:0/2753958961 wait complete. 2023-09-13T03:22:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.513+0000 7f1f32fff700 1 Processor -- start 2023-09-13T03:22:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.513+0000 7f1f32fff700 1 -- start start 2023-09-13T03:22:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.514+0000 7f1f32fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f2c1005b0 0x7f1f2c19e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.514+0000 7f1f32fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f2c100f70 0x7f1f2c19edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.514+0000 7f1f32fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f2c106670 0x7f1f2c198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.514+0000 7f1f32fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c198d80 con 0x7f1f2c100f70 2023-09-13T03:22:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.514+0000 7f1f32fff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c198ef0 con 0x7f1f2c106670 2023-09-13T03:22:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.514+0000 7f1f32fff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c1990b0 con 0x7f1f2c1005b0 2023-09-13T03:22:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.514+0000 7f1f2bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f2c100f70 0x7f1f2c19edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.514+0000 7f1f30d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f2c1005b0 0x7f1f2c19e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.514+0000 7f1f2bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f2c100f70 0x7f1f2c19edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33332/0 (socket says 172.21.15.64:33332) 2023-09-13T03:22:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.514+0000 7f1f2bfff700 1 -- 172.21.15.64:0/3804241481 learned_addr learned my addr 172.21.15.64:0/3804241481 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:22:54.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.514+0000 7f1f3159c700 1 --2- 172.21.15.64:0/3804241481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f2c106670 0x7f1f2c198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:54.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.515+0000 7f1f2bfff700 1 -- 172.21.15.64:0/3804241481 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f2c1005b0 msgr2=0x7f1f2c19e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:54.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.515+0000 7f1f2bfff700 1 --2- 172.21.15.64:0/3804241481 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f2c1005b0 0x7f1f2c19e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:54.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.515+0000 7f1f2bfff700 1 -- 172.21.15.64:0/3804241481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f2c106670 msgr2=0x7f1f2c198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.515+0000 7f1f2bfff700 1 --2- 172.21.15.64:0/3804241481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f2c106670 0x7f1f2c198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.515+0000 7f1f2bfff700 1 -- 172.21.15.64:0/3804241481 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f200095f0 con 0x7f1f2c100f70 2023-09-13T03:22:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.515+0000 7f1f30d9b700 1 --2- 172.21.15.64:0/3804241481 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f2c1005b0 0x7f1f2c19e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:22:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.515+0000 7f1f3159c700 1 --2- 172.21.15.64:0/3804241481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f2c106670 0x7f1f2c198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:22:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.515+0000 7f1f2bfff700 1 --2- 172.21.15.64:0/3804241481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f2c100f70 0x7f1f2c19edf0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f1f1800e9a0 tx=0x7f1f1800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:22:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.515+0000 7f1f29ffb700 1 -- 172.21.15.64:0/3804241481 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f18009790 con 0x7f1f2c100f70 2023-09-13T03:22:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.515+0000 7f1f29ffb700 1 -- 172.21.15.64:0/3804241481 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f18010430 con 0x7f1f2c100f70 2023-09-13T03:22:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.515+0000 7f1f29ffb700 1 -- 172.21.15.64:0/3804241481 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f18018680 con 0x7f1f2c100f70 2023-09-13T03:22:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.515+0000 7f1f32fff700 1 -- 172.21.15.64:0/3804241481 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f2c199390 con 0x7f1f2c100f70 2023-09-13T03:22:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.515+0000 7f1f32fff700 1 -- 172.21.15.64:0/3804241481 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f2c1a58a0 con 0x7f1f2c100f70 2023-09-13T03:22:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.519+0000 7f1f29ffb700 1 -- 172.21.15.64:0/3804241481 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f1f180105a0 con 0x7f1f2c100f70 2023-09-13T03:22:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.520+0000 7f1f29ffb700 1 --2- 172.21.15.64:0/3804241481 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f1f1c0a0830 0x7f1f1c0a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.520+0000 7f1f29ffb700 1 -- 172.21.15.64:0/3804241481 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f1f18014070 con 0x7f1f2c100f70 2023-09-13T03:22:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.520+0000 7f1f469c1b80 1 Processor -- start 2023-09-13T03:22:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.520+0000 7f1f30d9b700 1 --2- 172.21.15.64:0/3804241481 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f1f1c0a0830 0x7f1f1c0a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.520+0000 7f1f469c1b80 1 -- start start 2023-09-13T03:22:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.521+0000 7f1f469c1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56126d3d51a0 0x56126d3d5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.521+0000 7f1f469c1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56126d3cf300 0x56126d3cf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.521+0000 7f1f469c1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56126d3cfcc0 0x56126d3dbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.521+0000 7f1f469c1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56126d3286d0 con 0x56126d3d51a0 2023-09-13T03:22:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.521+0000 7f1f469c1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56126d325ff0 con 0x56126d3cf300 2023-09-13T03:22:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.521+0000 7f1f469c1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56126d3218e0 con 0x56126d3cfcc0 2023-09-13T03:22:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.521+0000 7f1f30d9b700 1 --2- 172.21.15.64:0/3804241481 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f1f1c0a0830 0x7f1f1c0a2cf0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1f20009990 tx=0x7f1f2001a040 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:22:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.521+0000 7f1f2bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56126d3cf300 0x56126d3cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.521+0000 7f1f30d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56126d3d51a0 0x56126d3d5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.521+0000 7f1f2bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56126d3cf300 0x56126d3cf780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:44214/0 (socket says 172.21.15.64:44214) 2023-09-13T03:22:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.521+0000 7f1f2bfff700 1 -- 172.21.15.64:0/3303900435 learned_addr learned my addr 172.21.15.64:0/3303900435 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:22:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.521+0000 7f1f3159c700 1 --2- 172.21.15.64:0/3303900435 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56126d3cfcc0 0x56126d3dbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.522+0000 7f1f2bfff700 1 -- 172.21.15.64:0/3303900435 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56126d3cfcc0 msgr2=0x56126d3dbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:22:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.522+0000 7f1f2bfff700 1 --2- 172.21.15.64:0/3303900435 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56126d3cfcc0 0x56126d3dbf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.522+0000 7f1f2bfff700 1 -- 172.21.15.64:0/3303900435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56126d3d51a0 msgr2=0x56126d3d5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:22:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.522+0000 7f1f2bfff700 1 --2- 172.21.15.64:0/3303900435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56126d3d51a0 0x56126d3d5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.522+0000 7f1f2bfff700 1 -- 172.21.15.64:0/3303900435 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56126d14dd80 con 0x56126d3cf300 2023-09-13T03:22:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.522+0000 7f1f2bfff700 1 --2- 172.21.15.64:0/3303900435 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56126d3cf300 0x56126d3cf780 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1f18096020 tx=0x7f1f18098fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb5932e84cd984aa server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:22:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.522+0000 7f1f16ffd700 1 -- 172.21.15.64:0/3303900435 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1809d3e0 con 0x56126d3cf300 2023-09-13T03:22:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.522+0000 7f1f16ffd700 1 -- 172.21.15.64:0/3303900435 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f180a1620 con 0x56126d3cf300 2023-09-13T03:22:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.522+0000 7f1f16ffd700 1 -- 172.21.15.64:0/3303900435 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f18099810 con 0x56126d3cf300 2023-09-13T03:22:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.523+0000 7f1f469c1b80 1 -- 172.21.15.64:0/3303900435 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56126d3cf300 msgr2=0x56126d3cf780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.523+0000 7f1f469c1b80 1 --2- 172.21.15.64:0/3303900435 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56126d3cf300 0x56126d3cf780 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1f18096020 tx=0x7f1f18098fe0 comp rx=0 tx=0).stop 2023-09-13T03:22:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.523+0000 7f1f469c1b80 1 -- 172.21.15.64:0/3303900435 shutdown_connections 2023-09-13T03:22:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.523+0000 7f1f469c1b80 1 --2- 172.21.15.64:0/3303900435 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56126d3cfcc0 0x56126d3dbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.523+0000 7f1f469c1b80 1 --2- 172.21.15.64:0/3303900435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56126d3d51a0 0x56126d3d5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.523+0000 7f1f469c1b80 1 --2- 172.21.15.64:0/3303900435 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56126d3cf300 0x56126d3cf780 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.523+0000 7f1f469c1b80 1 -- 172.21.15.64:0/3303900435 >> 172.21.15.64:0/3303900435 conn(0x56126d344030 msgr2=0x56126d3ca560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:22:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.523+0000 7f1f469c1b80 1 -- 172.21.15.64:0/3303900435 shutdown_connections 2023-09-13T03:22:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.523+0000 7f1f469c1b80 1 -- 172.21.15.64:0/3303900435 wait complete. 2023-09-13T03:22:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.523+0000 7f1f469c1b80 1 -- start start 2023-09-13T03:22:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.524+0000 7f1f469c1b80 1 Processor -- start 2023-09-13T03:22:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.524+0000 7f1f469c1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56126d3cf300 0x56126d3dbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.524+0000 7f1f469c1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56126d3cfcc0 0x56126d3d9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.524+0000 7f1f469c1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56126d3d51a0 0x56126d3d9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.524+0000 7f1f469c1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56126d2bfb10 con 0x56126d3cfcc0 2023-09-13T03:22:54.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.524+0000 7f1f469c1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56126d2bf810 con 0x56126d3cf300 2023-09-13T03:22:54.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.524+0000 7f1f469c1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56126d1a38c0 con 0x56126d3d51a0 2023-09-13T03:22:54.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.524+0000 7f1f469c1b80 10 client.0 _ll_register_callbacks cb 0x56126d209a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:22:54.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.525+0000 7f1f30d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56126d3cf300 0x56126d3dbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:54.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.525+0000 7f1f2bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56126d3cfcc0 0x56126d3d9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:54.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.525+0000 7f1f30d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56126d3cf300 0x56126d3dbed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:44230/0 (socket says 172.21.15.64:44230) 2023-09-13T03:22:54.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.525+0000 7f1f30d9b700 1 -- 172.21.15.64:0/2908563896 learned_addr learned my addr 172.21.15.64:0/2908563896 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:22:54.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.525+0000 7f1f3159c700 1 --2- 172.21.15.64:0/2908563896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56126d3d51a0 0x56126d3d9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:54.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.525+0000 7f1f30d9b700 1 -- 172.21.15.64:0/2908563896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56126d3d51a0 msgr2=0x56126d3d9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:22:54.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.525+0000 7f1f30d9b700 1 --2- 172.21.15.64:0/2908563896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56126d3d51a0 0x56126d3d9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.525+0000 7f1f30d9b700 1 -- 172.21.15.64:0/2908563896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56126d3cfcc0 msgr2=0x56126d3d9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:22:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.525+0000 7f1f30d9b700 1 --2- 172.21.15.64:0/2908563896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56126d3cfcc0 0x56126d3d9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.525+0000 7f1f30d9b700 1 -- 172.21.15.64:0/2908563896 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f2c1a58a0 con 0x56126d3cf300 2023-09-13T03:22:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.525+0000 7f1f2bfff700 1 --2- 172.21.15.64:0/2908563896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56126d3cfcc0 0x56126d3d9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:22:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.525+0000 7f1f30d9b700 1 --2- 172.21.15.64:0/2908563896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56126d3cf300 0x56126d3dbed0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1f20009fd0 tx=0x7f1f20009cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20fa7e3cb702181d server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:22:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.525+0000 7f1f0bfff700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:22:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.526+0000 7f1f0bfff700 1 -- 172.21.15.64:0/2908563896 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2002b040 con 0x56126d3cf300 2023-09-13T03:22:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.526+0000 7f1f469c1b80 1 -- 172.21.15.64:0/2908563896 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56126d279490 con 0x56126d3cf300 2023-09-13T03:22:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.526+0000 7f1f0bfff700 1 -- 172.21.15.64:0/2908563896 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f1f20005550 con 0x56126d3cf300 2023-09-13T03:22:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.526+0000 7f1f0bfff700 1 -- 172.21.15.64:0/2908563896 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f20005710 con 0x56126d3cf300 2023-09-13T03:22:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.526+0000 7f1f469c1b80 10 client.26912 fetch_fsmap learned FSMap version 59 2023-09-13T03:22:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.526+0000 7f1f469c1b80 1 -- 172.21.15.64:0/2908563896 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x56126d136c00 con 0x56126d3cf300 2023-09-13T03:22:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.527+0000 7f1f0bfff700 1 -- 172.21.15.64:0/2908563896 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56126d279490 con 0x56126d3cf300 2023-09-13T03:22:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.527+0000 7f1f0bfff700 1 -- 172.21.15.64:0/2908563896 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 59) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f1f200245c0 con 0x56126d3cf300 2023-09-13T03:22:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.527+0000 7f1f469c1b80 10 client.26912 fetch_fsmap finished waiting for FSMap version 59 2023-09-13T03:22:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.527+0000 7f1f469c1b80 10 client.26912 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:22:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.527+0000 7f1f469c1b80 20 client.26912 populate_metadata read hostname 'smithi064' 2023-09-13T03:22:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.527+0000 7f1f469c1b80 1 --2- 172.21.15.64:0/2908563896 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56126d3e93f0 0x56126d3eb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.527+0000 7f1f469c1b80 4 client.26912 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:22:54.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.527+0000 7f1f469c1b80 1 -- 172.21.15.64:0/2908563896 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56126d336840 con 0x56126d3e93f0 2023-09-13T03:22:54.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.527+0000 7f1f2bfff700 1 --2- 172.21.15.64:0/2908563896 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56126d3e93f0 0x56126d3eb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:54.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.528+0000 7f1f2bfff700 1 --2- 172.21.15.64:0/2908563896 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56126d3e93f0 0x56126d3eb810 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83ab6edb494a6ee8 server_cookie=b9636438f306eed4 in_seq=0 out_seq=0 2023-09-13T03:22:54.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.528+0000 7f1f0bfff700 10 client.26912 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:22:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.529+0000 7f1f0bfff700 1 -- 172.21.15.64:0/2908563896 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56126d336840 con 0x56126d3e93f0 2023-09-13T03:22:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.529+0000 7f1f0bfff700 10 client.26912 handle_command_reply: tid=0 2023-09-13T03:22:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.529+0000 7f1f469c1b80 1 client.26912 shutdown 2023-09-13T03:22:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.529+0000 7f1f469c1b80 1 -- 172.21.15.64:0/2908563896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56126d3cf300 msgr2=0x56126d3dbed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.529+0000 7f1f469c1b80 1 --2- 172.21.15.64:0/2908563896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56126d3cf300 0x56126d3dbed0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1f20009fd0 tx=0x7f1f20009cd0 comp rx=0 tx=0).stop 2023-09-13T03:22:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.529+0000 7f1f469c1b80 1 -- 172.21.15.64:0/2908563896 shutdown_connections 2023-09-13T03:22:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.529+0000 7f1f469c1b80 1 --2- 172.21.15.64:0/2908563896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56126d3cf300 0x56126d3dbed0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.529+0000 7f1f469c1b80 1 --2- 172.21.15.64:0/2908563896 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56126d3e93f0 0x56126d3eb810 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.529+0000 7f1f469c1b80 1 --2- 172.21.15.64:0/2908563896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56126d3d51a0 0x56126d3d9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.529+0000 7f1f0bfff700 0 client.26912 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:22:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.529+0000 7f1f469c1b80 1 --2- 172.21.15.64:0/2908563896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56126d3cfcc0 0x56126d3d9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.529+0000 7f1f469c1b80 1 -- 172.21.15.64:0/2908563896 >> 172.21.15.64:0/2908563896 conn(0x56126d344030 msgr2=0x56126d3c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:22:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.529+0000 7f1f469c1b80 1 -- 172.21.15.64:0/2908563896 shutdown_connections 2023-09-13T03:22:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.529+0000 7f1f469c1b80 1 -- 172.21.15.64:0/2908563896 wait complete. 2023-09-13T03:22:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.529+0000 7f1f469c1b80 20 client.26912 trim_cache size 0 max 16384 2023-09-13T03:22:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.545+0000 7f1f469c1b80 1 Processor -- start 2023-09-13T03:22:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.545+0000 7f1f469c1b80 1 -- start start 2023-09-13T03:22:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.545+0000 7f1f469c1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56126d3cf300 0x56126d3dba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.545+0000 7f1f469c1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56126d3cfcc0 0x56126d3d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.545+0000 7f1f469c1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56126d3d51a0 0x56126d3d3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.545+0000 7f1f469c1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56126d3d0580 con 0x56126d3cfcc0 2023-09-13T03:22:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.545+0000 7f1f469c1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56126d3c6fc0 con 0x56126d3cf300 2023-09-13T03:22:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.545+0000 7f1f469c1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56126d3da730 con 0x56126d3d51a0 2023-09-13T03:22:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.546+0000 7f1f2bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56126d3cfcc0 0x56126d3d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.546+0000 7f1f2bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56126d3cfcc0 0x56126d3d31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33370/0 (socket says 172.21.15.64:33370) 2023-09-13T03:22:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.546+0000 7f1f2bfff700 1 -- 172.21.15.64:0/2543522729 learned_addr learned my addr 172.21.15.64:0/2543522729 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:22:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.546+0000 7f1f30d9b700 1 --2- 172.21.15.64:0/2543522729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56126d3cf300 0x56126d3dba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.546+0000 7f1f3159c700 1 --2- 172.21.15.64:0/2543522729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56126d3d51a0 0x56126d3d3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.546+0000 7f1f2bfff700 1 -- 172.21.15.64:0/2543522729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56126d3d51a0 msgr2=0x56126d3d3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:22:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.546+0000 7f1f2bfff700 1 --2- 172.21.15.64:0/2543522729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56126d3d51a0 0x56126d3d3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.546+0000 7f1f2bfff700 1 -- 172.21.15.64:0/2543522729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56126d3cf300 msgr2=0x56126d3dba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:22:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.546+0000 7f1f2bfff700 1 --2- 172.21.15.64:0/2543522729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56126d3cf300 0x56126d3dba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.546+0000 7f1f2bfff700 1 -- 172.21.15.64:0/2543522729 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56126d1741f0 con 0x56126d3cfcc0 2023-09-13T03:22:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.546+0000 7f1f2bfff700 1 --2- 172.21.15.64:0/2543522729 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56126d3cfcc0 0x56126d3d31f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f1f180c9e10 tx=0x7f1f180cb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2663a42a95cfde76 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:22:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.546+0000 7f1f177fe700 1 -- 172.21.15.64:0/2543522729 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f18097950 con 0x56126d3cfcc0 2023-09-13T03:22:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.547+0000 7f1f177fe700 1 -- 172.21.15.64:0/2543522729 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f180b0070 con 0x56126d3cfcc0 2023-09-13T03:22:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.547+0000 7f1f177fe700 1 -- 172.21.15.64:0/2543522729 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f18099810 con 0x56126d3cfcc0 2023-09-13T03:22:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.547+0000 7f1f469c1b80 1 -- 172.21.15.64:0/2543522729 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56126d3cfcc0 msgr2=0x56126d3d31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.547+0000 7f1f469c1b80 1 --2- 172.21.15.64:0/2543522729 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56126d3cfcc0 0x56126d3d31f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f1f180c9e10 tx=0x7f1f180cb040 comp rx=0 tx=0).stop 2023-09-13T03:22:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.547+0000 7f1f469c1b80 1 -- 172.21.15.64:0/2543522729 shutdown_connections 2023-09-13T03:22:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.547+0000 7f1f469c1b80 1 --2- 172.21.15.64:0/2543522729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56126d3cf300 0x56126d3dba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.547+0000 7f1f469c1b80 1 --2- 172.21.15.64:0/2543522729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56126d3d51a0 0x56126d3d3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.547+0000 7f1f469c1b80 1 --2- 172.21.15.64:0/2543522729 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56126d3cfcc0 0x56126d3d31f0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.547+0000 7f1f469c1b80 1 -- 172.21.15.64:0/2543522729 >> 172.21.15.64:0/2543522729 conn(0x56126d344030 msgr2=0x56126d3e5730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:22:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.547+0000 7f1f469c1b80 1 -- 172.21.15.64:0/2543522729 shutdown_connections 2023-09-13T03:22:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.547+0000 7f1f469c1b80 1 -- 172.21.15.64:0/2543522729 wait complete. 2023-09-13T03:22:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.548+0000 7f1f469c1b80 1 -- start start 2023-09-13T03:22:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.548+0000 7f1f469c1b80 1 Processor -- start 2023-09-13T03:22:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.548+0000 7f1f469c1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56126d3cf300 0x56126d3de2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.548+0000 7f1f469c1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56126d3cfcc0 0x56126d3de7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.548+0000 7f1f469c1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56126d3d51a0 0x56126d436210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.548+0000 7f1f469c1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56126d436750 con 0x56126d3cf300 2023-09-13T03:22:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.548+0000 7f1f469c1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56126d4368c0 con 0x56126d3cfcc0 2023-09-13T03:22:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.548+0000 7f1f469c1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56126d436a80 con 0x56126d3d51a0 2023-09-13T03:22:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.548+0000 7f1f469c1b80 10 client.0 _ll_register_callbacks cb 0x56126d209a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:22:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.548+0000 7f1f30d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56126d3cf300 0x56126d3de2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.549+0000 7f1f30d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56126d3cf300 0x56126d3de2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33386/0 (socket says 172.21.15.64:33386) 2023-09-13T03:22:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.549+0000 7f1f30d9b700 1 -- 172.21.15.64:0/4057926385 learned_addr learned my addr 172.21.15.64:0/4057926385 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:22:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.549+0000 7f1f2bfff700 1 --2- 172.21.15.64:0/4057926385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56126d3cfcc0 0x56126d3de7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.549+0000 7f1f3159c700 1 --2- 172.21.15.64:0/4057926385 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56126d3d51a0 0x56126d436210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.549+0000 7f1f30d9b700 1 -- 172.21.15.64:0/4057926385 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56126d3d51a0 msgr2=0x56126d436210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:22:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.549+0000 7f1f30d9b700 1 --2- 172.21.15.64:0/4057926385 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56126d3d51a0 0x56126d436210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.549+0000 7f1f30d9b700 1 -- 172.21.15.64:0/4057926385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56126d3cfcc0 msgr2=0x56126d3de7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:22:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.549+0000 7f1f30d9b700 1 --2- 172.21.15.64:0/4057926385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56126d3cfcc0 0x56126d3de7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.549+0000 7f1f30d9b700 1 -- 172.21.15.64:0/4057926385 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56126d14dd80 con 0x56126d3cf300 2023-09-13T03:22:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.549+0000 7f1f30d9b700 1 --2- 172.21.15.64:0/4057926385 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56126d3cf300 0x56126d3de2a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f1f20024a70 tx=0x7f1f20024e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb1e2f2e88e242a9 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:22:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.549+0000 7f1f0bfff700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:22:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.550+0000 7f1f0bfff700 1 -- 172.21.15.64:0/4057926385 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2002b070 con 0x56126d3cf300 2023-09-13T03:22:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.550+0000 7f1f0bfff700 1 -- 172.21.15.64:0/4057926385 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f1f2000be70 con 0x56126d3cf300 2023-09-13T03:22:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.550+0000 7f1f469c1b80 1 -- 172.21.15.64:0/4057926385 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f180a1620 con 0x56126d3cf300 2023-09-13T03:22:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.550+0000 7f1f0bfff700 1 -- 172.21.15.64:0/4057926385 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f20024620 con 0x56126d3cf300 2023-09-13T03:22:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.550+0000 7f1f469c1b80 10 client.16881 fetch_fsmap learned FSMap version 59 2023-09-13T03:22:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.550+0000 7f1f469c1b80 1 -- 172.21.15.64:0/4057926385 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x56126d14c250 con 0x56126d3cf300 2023-09-13T03:22:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.550+0000 7f1f0bfff700 1 -- 172.21.15.64:0/4057926385 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f2004e020 con 0x56126d3cf300 2023-09-13T03:22:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.550+0000 7f1f0bfff700 1 -- 172.21.15.64:0/4057926385 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 59) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f1f20039ab0 con 0x56126d3cf300 2023-09-13T03:22:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.550+0000 7f1f469c1b80 10 client.16881 fetch_fsmap finished waiting for FSMap version 59 2023-09-13T03:22:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.550+0000 7f1f469c1b80 10 client.16881 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:22:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.550+0000 7f1f469c1b80 20 client.16881 populate_metadata read hostname 'smithi064' 2023-09-13T03:22:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.550+0000 7f1f469c1b80 1 --2- 172.21.15.64:0/4057926385 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56126d43ddd0 0x56126d4401f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:22:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.550+0000 7f1f469c1b80 4 client.16881 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:22:54.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.550+0000 7f1f469c1b80 1 -- 172.21.15.64:0/4057926385 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56126d31a8c0 con 0x56126d43ddd0 2023-09-13T03:22:54.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.551+0000 7f1f2bfff700 1 --2- 172.21.15.64:0/4057926385 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56126d43ddd0 0x56126d4401f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:22:54.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.551+0000 7f1f2bfff700 1 --2- 172.21.15.64:0/4057926385 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56126d43ddd0 0x56126d4401f0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b12272d5d4c5b8b server_cookie=7792510c6fc69602 in_seq=0 out_seq=0 2023-09-13T03:22:54.566 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.551+0000 7f1f0bfff700 10 client.16881 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:22:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.582+0000 7f1f0bfff700 1 -- 172.21.15.64:0/4057926385 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56126d31a8c0 con 0x56126d43ddd0 2023-09-13T03:22:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.582+0000 7f1f0bfff700 10 client.16881 handle_command_reply: tid=0 2023-09-13T03:22:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.582+0000 7f1f469c1b80 1 client.16881 shutdown 2023-09-13T03:22:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.582+0000 7f1f469c1b80 1 -- 172.21.15.64:0/4057926385 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56126d3cf300 msgr2=0x56126d3de2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.582+0000 7f1f469c1b80 1 --2- 172.21.15.64:0/4057926385 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56126d3cf300 0x56126d3de2a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f1f20024a70 tx=0x7f1f20024e90 comp rx=0 tx=0).stop 2023-09-13T03:22:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.582+0000 7f1f469c1b80 1 -- 172.21.15.64:0/4057926385 shutdown_connections 2023-09-13T03:22:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.582+0000 7f1f469c1b80 1 --2- 172.21.15.64:0/4057926385 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56126d43ddd0 0x56126d4401f0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.582+0000 7f1f469c1b80 1 --2- 172.21.15.64:0/4057926385 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56126d3d51a0 0x56126d436210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.582+0000 7f1f469c1b80 1 --2- 172.21.15.64:0/4057926385 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56126d3cf300 0x56126d3de2a0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.582+0000 7f1f469c1b80 1 --2- 172.21.15.64:0/4057926385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56126d3cfcc0 0x56126d3de7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.582+0000 7f1f0bfff700 0 client.16881 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:22:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.582+0000 7f1f469c1b80 1 -- 172.21.15.64:0/4057926385 >> 172.21.15.64:0/4057926385 conn(0x56126d344030 msgr2=0x56126d3d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:22:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.582+0000 7f1f469c1b80 1 -- 172.21.15.64:0/4057926385 shutdown_connections 2023-09-13T03:22:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.582+0000 7f1f469c1b80 1 -- 172.21.15.64:0/4057926385 wait complete. 2023-09-13T03:22:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.582+0000 7f1f469c1b80 20 client.16881 trim_cache size 0 max 16384 2023-09-13T03:22:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:22:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:22:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:22:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:22:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.582+0000 7f1f32fff700 1 -- 172.21.15.64:0/3804241481 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f1f1c0a0830 msgr2=0x7f1f1c0a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.582+0000 7f1f32fff700 1 --2- 172.21.15.64:0/3804241481 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f1f1c0a0830 0x7f1f1c0a2cf0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1f20009990 tx=0x7f1f2001a040 comp rx=0 tx=0).stop 2023-09-13T03:22:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.582+0000 7f1f32fff700 1 -- 172.21.15.64:0/3804241481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f2c100f70 msgr2=0x7f1f2c19edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:22:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.582+0000 7f1f32fff700 1 --2- 172.21.15.64:0/3804241481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f2c100f70 0x7f1f2c19edf0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f1f1800e9a0 tx=0x7f1f1800ecb0 comp rx=0 tx=0).stop 2023-09-13T03:22:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.582+0000 7f1f32fff700 1 -- 172.21.15.64:0/3804241481 shutdown_connections 2023-09-13T03:22:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.582+0000 7f1f32fff700 1 --2- 172.21.15.64:0/3804241481 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f1f1c0a0830 0x7f1f1c0a2cf0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.582+0000 7f1f32fff700 1 --2- 172.21.15.64:0/3804241481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f2c106670 0x7f1f2c198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.582+0000 7f1f32fff700 1 --2- 172.21.15.64:0/3804241481 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f2c1005b0 0x7f1f2c19e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.582+0000 7f1f32fff700 1 --2- 172.21.15.64:0/3804241481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f2c100f70 0x7f1f2c19edf0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:22:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.582+0000 7f1f32fff700 1 -- 172.21.15.64:0/3804241481 >> 172.21.15.64:0/3804241481 conn(0x7f1f2c0fc060 msgr2=0x7f1f2c10b860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:22:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.583+0000 7f1f32fff700 1 -- 172.21.15.64:0/3804241481 shutdown_connections 2023-09-13T03:22:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:22:54.583+0000 7f1f32fff700 1 -- 172.21.15.64:0/3804241481 wait complete. 2023-09-13T03:22:54.941 INFO:tasks.workunit.client.0.smithi064.stdout: 1 210001 2.01 MB/sec execute 419 sec latency 1362.372 ms 2023-09-13T03:22:54.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:22:54 smithi142 ceph-mon[134082]: pgmap v301: 97 pgs: 97 active+clean; 101 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 497 KiB/s wr, 82 op/s 2023-09-13T03:22:55.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:22:54 smithi170 ceph-mon[134328]: pgmap v301: 97 pgs: 97 active+clean; 101 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 497 KiB/s wr, 82 op/s 2023-09-13T03:22:55.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:22:54 smithi064 ceph-mon[97675]: pgmap v301: 97 pgs: 97 active+clean; 101 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 497 KiB/s wr, 82 op/s 2023-09-13T03:22:55.941 INFO:tasks.workunit.client.0.smithi064.stdout: 1 210001 2.00 MB/sec execute 420 sec latency 2362.455 ms 2023-09-13T03:22:56.941 INFO:tasks.workunit.client.0.smithi064.stdout: 1 210001 2.00 MB/sec execute 421 sec latency 3362.589 ms 2023-09-13T03:22:56.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:22:56 smithi142 ceph-mon[134082]: pgmap v302: 97 pgs: 97 active+clean; 99 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 739 KiB/s wr, 123 op/s 2023-09-13T03:22:57.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:22:56 smithi170 ceph-mon[134328]: pgmap v302: 97 pgs: 97 active+clean; 99 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 739 KiB/s wr, 123 op/s 2023-09-13T03:22:57.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:22:56 smithi064 ceph-mon[97675]: pgmap v302: 97 pgs: 97 active+clean; 99 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 739 KiB/s wr, 123 op/s 2023-09-13T03:22:57.941 INFO:tasks.workunit.client.0.smithi064.stdout: 1 210001 1.99 MB/sec execute 422 sec latency 4362.720 ms 2023-09-13T03:22:58.941 INFO:tasks.workunit.client.0.smithi064.stdout: 1 210069 1.99 MB/sec execute 423 sec latency 4950.592 ms 2023-09-13T03:22:58.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:22:58 smithi142 ceph-mon[134082]: pgmap v303: 97 pgs: 97 active+clean; 99 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 573 KiB/s wr, 86 op/s 2023-09-13T03:22:59.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:22:58 smithi170 ceph-mon[134328]: pgmap v303: 97 pgs: 97 active+clean; 99 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 573 KiB/s wr, 86 op/s 2023-09-13T03:22:59.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:22:58 smithi064 ceph-mon[97675]: pgmap v303: 97 pgs: 97 active+clean; 99 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 573 KiB/s wr, 86 op/s 2023-09-13T03:22:59.941 INFO:tasks.workunit.client.0.smithi064.stdout: 1 210069 1.98 MB/sec execute 424 sec latency 1358.951 ms 2023-09-13T03:23:00.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:00 smithi170 ceph-mon[134328]: pgmap v304: 97 pgs: 97 active+clean; 99 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 573 KiB/s wr, 86 op/s 2023-09-13T03:23:00.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:00 smithi064 ceph-mon[97675]: pgmap v304: 97 pgs: 97 active+clean; 99 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 573 KiB/s wr, 86 op/s 2023-09-13T03:23:00.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:00 smithi142 ceph-mon[134082]: pgmap v304: 97 pgs: 97 active+clean; 99 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 573 KiB/s wr, 86 op/s 2023-09-13T03:23:00.942 INFO:tasks.workunit.client.0.smithi064.stdout: 1 210257 1.98 MB/sec execute 425 sec latency 2210.135 ms 2023-09-13T03:23:01.942 INFO:tasks.workunit.client.0.smithi064.stdout: 1 210970 1.99 MB/sec execute 426 sec latency 217.103 ms 2023-09-13T03:23:02.942 INFO:tasks.workunit.client.0.smithi064.stdout: 1 210970 1.98 MB/sec execute 427 sec latency 1217.246 ms 2023-09-13T03:23:02.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:02 smithi142 ceph-mon[134082]: pgmap v305: 97 pgs: 97 active+clean; 102 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 846 KiB/s wr, 101 op/s 2023-09-13T03:23:03.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:02 smithi170 ceph-mon[134328]: pgmap v305: 97 pgs: 97 active+clean; 102 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 846 KiB/s wr, 101 op/s 2023-09-13T03:23:03.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:02 smithi064 ceph-mon[97675]: pgmap v305: 97 pgs: 97 active+clean; 102 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 846 KiB/s wr, 101 op/s 2023-09-13T03:23:03.942 INFO:tasks.workunit.client.0.smithi064.stdout: 1 210970 1.98 MB/sec execute 428 sec latency 2217.350 ms 2023-09-13T03:23:04.942 INFO:tasks.workunit.client.0.smithi064.stdout: 1 210970 1.97 MB/sec execute 429 sec latency 3217.486 ms 2023-09-13T03:23:04.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:04 smithi142 ceph-mon[134082]: pgmap v306: 97 pgs: 97 active+clean; 102 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 514 KiB/s wr, 56 op/s 2023-09-13T03:23:05.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:04 smithi170 ceph-mon[134328]: pgmap v306: 97 pgs: 97 active+clean; 102 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 514 KiB/s wr, 56 op/s 2023-09-13T03:23:05.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:04 smithi064 ceph-mon[97675]: pgmap v306: 97 pgs: 97 active+clean; 102 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 514 KiB/s wr, 56 op/s 2023-09-13T03:23:05.942 INFO:tasks.workunit.client.0.smithi064.stdout: 1 212171 1.98 MB/sec execute 430 sec latency 3635.384 ms 2023-09-13T03:23:05.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:05 smithi142 ceph-mon[134082]: osdmap e93: 12 total, 12 up, 12 in 2023-09-13T03:23:06.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:05 smithi170 ceph-mon[134328]: osdmap e93: 12 total, 12 up, 12 in 2023-09-13T03:23:06.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:05 smithi064 ceph-mon[97675]: osdmap e93: 12 total, 12 up, 12 in 2023-09-13T03:23:06.942 INFO:tasks.workunit.client.0.smithi064.stdout: 1 213421 2.00 MB/sec execute 431 sec latency 17.218 ms 2023-09-13T03:23:06.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:06 smithi142 ceph-mon[134082]: pgmap v308: 97 pgs: 97 active+clean; 100 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 948 KiB/s wr, 98 op/s 2023-09-13T03:23:07.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:06 smithi170 ceph-mon[134328]: pgmap v308: 97 pgs: 97 active+clean; 100 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 948 KiB/s wr, 98 op/s 2023-09-13T03:23:07.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:06 smithi064 ceph-mon[97675]: pgmap v308: 97 pgs: 97 active+clean; 100 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 948 KiB/s wr, 98 op/s 2023-09-13T03:23:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.745+0000 7fd3955ff700 1 -- 172.21.15.64:0/3267028656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3900679f0 msgr2=0x7fd390067e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:23:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.745+0000 7fd3955ff700 1 --2- 172.21.15.64:0/3267028656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3900679f0 0x7fd390067e70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd3840099c0 tx=0x7fd384009cd0 comp rx=0 tx=0).stop 2023-09-13T03:23:07.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.745+0000 7fd3955ff700 1 -- 172.21.15.64:0/3267028656 shutdown_connections 2023-09-13T03:23:07.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.745+0000 7fd3955ff700 1 --2- 172.21.15.64:0/3267028656 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3900683b0 0x7fd39010d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:07.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.745+0000 7fd3955ff700 1 --2- 172.21.15.64:0/3267028656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd390106650 0x7fd390106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:07.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.745+0000 7fd3955ff700 1 --2- 172.21.15.64:0/3267028656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3900679f0 0x7fd390067e70 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:07.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.745+0000 7fd3955ff700 1 -- 172.21.15.64:0/3267028656 >> 172.21.15.64:0/3267028656 conn(0x7fd390074f30 msgr2=0x7fd390075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:23:07.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.746+0000 7fd3955ff700 1 -- 172.21.15.64:0/3267028656 shutdown_connections 2023-09-13T03:23:07.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.746+0000 7fd3955ff700 1 -- 172.21.15.64:0/3267028656 wait complete. 2023-09-13T03:23:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.746+0000 7fd3955ff700 1 Processor -- start 2023-09-13T03:23:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.747+0000 7fd3955ff700 1 -- start start 2023-09-13T03:23:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.747+0000 7fd3955ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3900679f0 0x7fd3900ffa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.747+0000 7fd3955ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3900683b0 0x7fd3900fff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.747+0000 7fd3955ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd390106650 0x7fd3901a4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.747+0000 7fd3955ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3901a4d90 con 0x7fd3900679f0 2023-09-13T03:23:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.747+0000 7fd3955ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3901a4ed0 con 0x7fd390106650 2023-09-13T03:23:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.747+0000 7fd3955ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3901a5060 con 0x7fd3900683b0 2023-09-13T03:23:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.747+0000 7fd38effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3900679f0 0x7fd3900ffa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.747+0000 7fd38effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3900679f0 0x7fd3900ffa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58312/0 (socket says 172.21.15.64:58312) 2023-09-13T03:23:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.747+0000 7fd38effd700 1 -- 172.21.15.64:0/2608902915 learned_addr learned my addr 172.21.15.64:0/2608902915 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:23:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.747+0000 7fd38e7fc700 1 --2- 172.21.15.64:0/2608902915 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3900683b0 0x7fd3900fff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.747+0000 7fd38f7fe700 1 --2- 172.21.15.64:0/2608902915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd390106650 0x7fd3901a4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.748+0000 7fd38effd700 1 -- 172.21.15.64:0/2608902915 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3900683b0 msgr2=0x7fd3900fff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:23:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.748+0000 7fd38effd700 1 --2- 172.21.15.64:0/2608902915 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3900683b0 0x7fd3900fff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.748+0000 7fd38effd700 1 -- 172.21.15.64:0/2608902915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd390106650 msgr2=0x7fd3901a4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:23:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.748+0000 7fd38effd700 1 --2- 172.21.15.64:0/2608902915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd390106650 0x7fd3901a4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.748+0000 7fd38effd700 1 -- 172.21.15.64:0/2608902915 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd380009520 con 0x7fd3900679f0 2023-09-13T03:23:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.748+0000 7fd38e7fc700 1 --2- 172.21.15.64:0/2608902915 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3900683b0 0x7fd3900fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:23:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.748+0000 7fd38effd700 1 --2- 172.21.15.64:0/2608902915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3900679f0 0x7fd3900ffa10 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd38000eb60 tx=0x7fd38000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:23:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.748+0000 7fd377fff700 1 -- 172.21.15.64:0/2608902915 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38000cbd0 con 0x7fd3900679f0 2023-09-13T03:23:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.749+0000 7fd377fff700 1 -- 172.21.15.64:0/2608902915 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3800044e0 con 0x7fd3900679f0 2023-09-13T03:23:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.749+0000 7fd3955ff700 1 -- 172.21.15.64:0/2608902915 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3840095f0 con 0x7fd3900679f0 2023-09-13T03:23:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.749+0000 7fd377fff700 1 -- 172.21.15.64:0/2608902915 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd380003df0 con 0x7fd3900679f0 2023-09-13T03:23:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.749+0000 7fd3955ff700 1 -- 172.21.15.64:0/2608902915 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3901a5690 con 0x7fd3900679f0 2023-09-13T03:23:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.754+0000 7fd377fff700 1 -- 172.21.15.64:0/2608902915 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fd380004000 con 0x7fd3900679f0 2023-09-13T03:23:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.755+0000 7fd3a8e6db80 1 Processor -- start 2023-09-13T03:23:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.755+0000 7fd377fff700 1 --2- 172.21.15.64:0/2608902915 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fd3780a07b0 0x7fd3780a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.755+0000 7fd3a8e6db80 1 -- start start 2023-09-13T03:23:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.755+0000 7fd377fff700 1 -- 172.21.15.64:0/2608902915 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8528+0+0 (secure 0 0 0) 0x7fd380014070 con 0x7fd3900679f0 2023-09-13T03:23:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.755+0000 7fd38f7fe700 1 --2- 172.21.15.64:0/2608902915 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fd3780a07b0 0x7fd3780a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.756+0000 7fd3a8e6db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab86b371a0 0x55ab86b37580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.756+0000 7fd3a8e6db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab86b31300 0x55ab86b31780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.756+0000 7fd3a8e6db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab86b31cc0 0x55ab86b3df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.756+0000 7fd3a8e6db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab86a8a6d0 con 0x55ab86b31cc0 2023-09-13T03:23:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.756+0000 7fd3a8e6db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab86a87ff0 con 0x55ab86b31300 2023-09-13T03:23:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.756+0000 7fd3a8e6db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab86a838e0 con 0x55ab86b371a0 2023-09-13T03:23:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.756+0000 7fd38effd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab86b371a0 0x55ab86b37580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.756+0000 7fd38effd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab86b371a0 0x55ab86b37580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:47962/0 (socket says 172.21.15.64:47962) 2023-09-13T03:23:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.756+0000 7fd38effd700 1 -- 172.21.15.64:0/1337123989 learned_addr learned my addr 172.21.15.64:0/1337123989 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:23:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.756+0000 7fd38e7fc700 1 --2- 172.21.15.64:0/1337123989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab86b31300 0x55ab86b31780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.756+0000 7fd38f7fe700 1 --2- 172.21.15.64:0/1337123989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab86b31cc0 0x55ab86b3df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.756+0000 7fd38effd700 1 -- 172.21.15.64:0/1337123989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab86b31300 msgr2=0x55ab86b31780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:23:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.756+0000 7fd38effd700 1 --2- 172.21.15.64:0/1337123989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab86b31300 0x55ab86b31780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.756+0000 7fd38effd700 1 -- 172.21.15.64:0/1337123989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab86b31cc0 msgr2=0x55ab86b3df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:23:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.756+0000 7fd38effd700 1 --2- 172.21.15.64:0/1337123989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab86b31cc0 0x55ab86b3df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.756+0000 7fd38effd700 1 -- 172.21.15.64:0/1337123989 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab868afd80 con 0x55ab86b371a0 2023-09-13T03:23:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.756+0000 7fd38f7fe700 1 --2- 172.21.15.64:0/1337123989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab86b31cc0 0x55ab86b3df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T03:23:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.756+0000 7fd38f7fe700 1 --2- 172.21.15.64:0/2608902915 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fd3780a07b0 0x7fd3780a2c70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd38800f540 tx=0x7fd38800b040 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:23:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.757+0000 7fd38effd700 1 --2- 172.21.15.64:0/1337123989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab86b371a0 0x55ab86b37580 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd380009230 tx=0x7fd380099630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df8bc3aa238a4494 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:23:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.757+0000 7fd3757fa700 1 -- 172.21.15.64:0/1337123989 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd380099790 con 0x55ab86b371a0 2023-09-13T03:23:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.757+0000 7fd3757fa700 1 -- 172.21.15.64:0/1337123989 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd38009d5c0 con 0x55ab86b371a0 2023-09-13T03:23:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.757+0000 7fd3757fa700 1 -- 172.21.15.64:0/1337123989 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38009d3d0 con 0x55ab86b371a0 2023-09-13T03:23:07.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.758+0000 7fd3a8e6db80 1 -- 172.21.15.64:0/1337123989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab86b371a0 msgr2=0x55ab86b37580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:23:07.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.758+0000 7fd3a8e6db80 1 --2- 172.21.15.64:0/1337123989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab86b371a0 0x55ab86b37580 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd380009230 tx=0x7fd380099630 comp rx=0 tx=0).stop 2023-09-13T03:23:07.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.758+0000 7fd3a8e6db80 1 -- 172.21.15.64:0/1337123989 shutdown_connections 2023-09-13T03:23:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.758+0000 7fd3a8e6db80 1 --2- 172.21.15.64:0/1337123989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab86b31cc0 0x55ab86b3df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.758+0000 7fd3a8e6db80 1 --2- 172.21.15.64:0/1337123989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab86b371a0 0x55ab86b37580 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.758+0000 7fd3a8e6db80 1 --2- 172.21.15.64:0/1337123989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab86b31300 0x55ab86b31780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.758+0000 7fd3a8e6db80 1 -- 172.21.15.64:0/1337123989 >> 172.21.15.64:0/1337123989 conn(0x55ab86aa6030 msgr2=0x55ab86b2c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:23:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.758+0000 7fd3a8e6db80 1 -- 172.21.15.64:0/1337123989 shutdown_connections 2023-09-13T03:23:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.758+0000 7fd3a8e6db80 1 -- 172.21.15.64:0/1337123989 wait complete. 2023-09-13T03:23:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.758+0000 7fd3a8e6db80 1 -- start start 2023-09-13T03:23:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.759+0000 7fd3a8e6db80 1 Processor -- start 2023-09-13T03:23:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.759+0000 7fd3a8e6db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab86b31300 0x55ab86b3de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.759+0000 7fd3a8e6db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab86b31cc0 0x55ab86b3b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.759+0000 7fd3a8e6db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab86b371a0 0x55ab86b3b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.760+0000 7fd3a8e6db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab86a21b10 con 0x55ab86b31cc0 2023-09-13T03:23:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.760+0000 7fd3a8e6db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab86a21810 con 0x55ab86b371a0 2023-09-13T03:23:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.760+0000 7fd3a8e6db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab869058c0 con 0x55ab86b31300 2023-09-13T03:23:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.760+0000 7fd3a8e6db80 10 client.0 _ll_register_callbacks cb 0x55ab8696ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:23:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.760+0000 7fd38effd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab86b31300 0x55ab86b3de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.760+0000 7fd38e7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab86b31cc0 0x55ab86b3b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.760+0000 7fd38f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab86b371a0 0x55ab86b3b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.760+0000 7fd38effd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab86b31300 0x55ab86b3de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:47974/0 (socket says 172.21.15.64:47974) 2023-09-13T03:23:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.760+0000 7fd38effd700 1 -- 172.21.15.64:0/502115638 learned_addr learned my addr 172.21.15.64:0/502115638 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:23:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.760+0000 7fd38effd700 1 -- 172.21.15.64:0/502115638 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab86b371a0 msgr2=0x55ab86b3b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:23:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.760+0000 7fd38effd700 1 --2- 172.21.15.64:0/502115638 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab86b371a0 0x55ab86b3b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.760+0000 7fd38effd700 1 -- 172.21.15.64:0/502115638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab86b31cc0 msgr2=0x55ab86b3b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:23:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.760+0000 7fd38effd700 1 --2- 172.21.15.64:0/502115638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab86b31cc0 0x55ab86b3b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.760+0000 7fd38effd700 1 -- 172.21.15.64:0/502115638 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3901a5690 con 0x55ab86b31300 2023-09-13T03:23:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.760+0000 7fd38effd700 1 --2- 172.21.15.64:0/502115638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab86b31300 0x55ab86b3de10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd380021040 tx=0x7fd3800c9a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f4cbc9730b2272e server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:23:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.760+0000 7fd372ff5700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:23:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.761+0000 7fd372ff5700 1 -- 172.21.15.64:0/502115638 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3800978d0 con 0x55ab86b31300 2023-09-13T03:23:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.761+0000 7fd372ff5700 1 -- 172.21.15.64:0/502115638 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8528+0+0 (secure 0 0 0) 0x7fd38009d380 con 0x55ab86b31300 2023-09-13T03:23:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.761+0000 7fd3a8e6db80 1 -- 172.21.15.64:0/502115638 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ab869db490 con 0x55ab86b31300 2023-09-13T03:23:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.761+0000 7fd372ff5700 1 -- 172.21.15.64:0/502115638 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd380099790 con 0x55ab86b31300 2023-09-13T03:23:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.761+0000 7fd3a8e6db80 10 client.26914 fetch_fsmap learned FSMap version 59 2023-09-13T03:23:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.761+0000 7fd3a8e6db80 1 -- 172.21.15.64:0/502115638 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x55ab86898c00 con 0x55ab86b31300 2023-09-13T03:23:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.761+0000 7fd372ff5700 1 -- 172.21.15.64:0/502115638 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ab869db490 con 0x55ab86b31300 2023-09-13T03:23:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.761+0000 7fd372ff5700 1 -- 172.21.15.64:0/502115638 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 59) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fd380065020 con 0x55ab86b31300 2023-09-13T03:23:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.761+0000 7fd3a8e6db80 10 client.26914 fetch_fsmap finished waiting for FSMap version 59 2023-09-13T03:23:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.761+0000 7fd3a8e6db80 10 client.26914 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:23:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.761+0000 7fd3a8e6db80 20 client.26914 populate_metadata read hostname 'smithi064' 2023-09-13T03:23:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.761+0000 7fd3a8e6db80 1 --2- 172.21.15.64:0/502115638 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55ab86b4aec0 0x55ab86b4d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.761+0000 7fd3a8e6db80 4 client.26914 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:23:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.761+0000 7fd3a8e6db80 1 -- 172.21.15.64:0/502115638 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ab86a98840 con 0x55ab86b4aec0 2023-09-13T03:23:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.762+0000 7fd38e7fc700 1 --2- 172.21.15.64:0/502115638 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55ab86b4aec0 0x55ab86b4d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.762+0000 7fd38e7fc700 1 --2- 172.21.15.64:0/502115638 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55ab86b4aec0 0x55ab86b4d2b0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2467af9e14d74650 server_cookie=8fd20f199074c8f0 in_seq=0 out_seq=0 2023-09-13T03:23:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.762+0000 7fd372ff5700 10 client.26914 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:23:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.763+0000 7fd372ff5700 1 -- 172.21.15.64:0/502115638 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ab86a98840 con 0x55ab86b4aec0 2023-09-13T03:23:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.763+0000 7fd372ff5700 10 client.26914 handle_command_reply: tid=0 2023-09-13T03:23:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.763+0000 7fd3a8e6db80 1 client.26914 shutdown 2023-09-13T03:23:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.763+0000 7fd3a8e6db80 1 -- 172.21.15.64:0/502115638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab86b31300 msgr2=0x55ab86b3de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:23:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.763+0000 7fd3a8e6db80 1 --2- 172.21.15.64:0/502115638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab86b31300 0x55ab86b3de10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd380021040 tx=0x7fd3800c9a30 comp rx=0 tx=0).stop 2023-09-13T03:23:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.763+0000 7fd3a8e6db80 1 -- 172.21.15.64:0/502115638 shutdown_connections 2023-09-13T03:23:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.763+0000 7fd3a8e6db80 1 --2- 172.21.15.64:0/502115638 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab86b371a0 0x55ab86b3b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.763+0000 7fd3a8e6db80 1 --2- 172.21.15.64:0/502115638 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55ab86b4aec0 0x55ab86b4d2b0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.763+0000 7fd3a8e6db80 1 --2- 172.21.15.64:0/502115638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab86b31300 0x55ab86b3de10 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.763+0000 7fd3a8e6db80 1 --2- 172.21.15.64:0/502115638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab86b31cc0 0x55ab86b3b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.763+0000 7fd372ff5700 0 client.26914 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:23:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.763+0000 7fd3a8e6db80 1 -- 172.21.15.64:0/502115638 >> 172.21.15.64:0/502115638 conn(0x55ab86aa6030 msgr2=0x55ab86b2b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:23:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.763+0000 7fd3a8e6db80 1 -- 172.21.15.64:0/502115638 shutdown_connections 2023-09-13T03:23:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.763+0000 7fd3a8e6db80 1 -- 172.21.15.64:0/502115638 wait complete. 2023-09-13T03:23:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.764+0000 7fd3a8e6db80 20 client.26914 trim_cache size 0 max 16384 2023-09-13T03:23:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.780+0000 7fd3a8e6db80 1 Processor -- start 2023-09-13T03:23:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.780+0000 7fd3a8e6db80 1 -- start start 2023-09-13T03:23:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.780+0000 7fd3a8e6db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab86b31300 0x55ab86b3d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.780+0000 7fd3a8e6db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab86b31cc0 0x55ab86b35150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.780+0000 7fd3a8e6db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab86b371a0 0x55ab86b35690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.780+0000 7fd3a8e6db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab86b36960 con 0x55ab86b31cc0 2023-09-13T03:23:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.780+0000 7fd3a8e6db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab86b39690 con 0x55ab86b371a0 2023-09-13T03:23:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.780+0000 7fd3a8e6db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab86b35cc0 con 0x55ab86b31300 2023-09-13T03:23:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.781+0000 7fd38effd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab86b31300 0x55ab86b3d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.781+0000 7fd38e7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab86b31cc0 0x55ab86b35150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.781+0000 7fd38effd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab86b31300 0x55ab86b3d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:47982/0 (socket says 172.21.15.64:47982) 2023-09-13T03:23:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.781+0000 7fd38effd700 1 -- 172.21.15.64:0/2401952140 learned_addr learned my addr 172.21.15.64:0/2401952140 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:23:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.781+0000 7fd38f7fe700 1 --2- 172.21.15.64:0/2401952140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab86b371a0 0x55ab86b35690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.781+0000 7fd38effd700 1 -- 172.21.15.64:0/2401952140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab86b371a0 msgr2=0x55ab86b35690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:23:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.781+0000 7fd38effd700 1 --2- 172.21.15.64:0/2401952140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab86b371a0 0x55ab86b35690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.781+0000 7fd38effd700 1 -- 172.21.15.64:0/2401952140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab86b31cc0 msgr2=0x55ab86b35150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:23:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.781+0000 7fd38effd700 1 --2- 172.21.15.64:0/2401952140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab86b31cc0 0x55ab86b35150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.781+0000 7fd38effd700 1 -- 172.21.15.64:0/2401952140 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab868d61f0 con 0x55ab86b31300 2023-09-13T03:23:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.781+0000 7fd38effd700 1 --2- 172.21.15.64:0/2401952140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab86b31300 0x55ab86b3d940 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd380096200 tx=0x7fd380003890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d8e5f79a604d04d server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:23:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.781+0000 7fd375ffb700 1 -- 172.21.15.64:0/2401952140 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3800978d0 con 0x55ab86b31300 2023-09-13T03:23:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.781+0000 7fd375ffb700 1 -- 172.21.15.64:0/2401952140 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd38009d380 con 0x55ab86b31300 2023-09-13T03:23:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.781+0000 7fd375ffb700 1 -- 172.21.15.64:0/2401952140 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3800652b0 con 0x55ab86b31300 2023-09-13T03:23:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.782+0000 7fd3a8e6db80 1 -- 172.21.15.64:0/2401952140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab86b31300 msgr2=0x55ab86b3d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:23:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.782+0000 7fd3a8e6db80 1 --2- 172.21.15.64:0/2401952140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab86b31300 0x55ab86b3d940 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd380096200 tx=0x7fd380003890 comp rx=0 tx=0).stop 2023-09-13T03:23:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.782+0000 7fd3a8e6db80 1 -- 172.21.15.64:0/2401952140 shutdown_connections 2023-09-13T03:23:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.782+0000 7fd3a8e6db80 1 --2- 172.21.15.64:0/2401952140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab86b371a0 0x55ab86b35690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.782+0000 7fd3a8e6db80 1 --2- 172.21.15.64:0/2401952140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab86b31300 0x55ab86b3d940 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.782+0000 7fd3a8e6db80 1 --2- 172.21.15.64:0/2401952140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab86b31cc0 0x55ab86b35150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.782+0000 7fd3a8e6db80 1 -- 172.21.15.64:0/2401952140 >> 172.21.15.64:0/2401952140 conn(0x55ab86aa6030 msgr2=0x55ab86b47760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:23:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.782+0000 7fd3a8e6db80 1 -- 172.21.15.64:0/2401952140 shutdown_connections 2023-09-13T03:23:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.782+0000 7fd3a8e6db80 1 -- 172.21.15.64:0/2401952140 wait complete. 2023-09-13T03:23:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.783+0000 7fd3a8e6db80 1 -- start start 2023-09-13T03:23:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.783+0000 7fd3a8e6db80 1 Processor -- start 2023-09-13T03:23:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.783+0000 7fd3a8e6db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab86b31300 0x55ab86b40700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.783+0000 7fd3a8e6db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab86b31cc0 0x55ab86b40c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.783+0000 7fd3a8e6db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab86b371a0 0x55ab86b9dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.783+0000 7fd3a8e6db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab86b9e1a0 con 0x55ab86b371a0 2023-09-13T03:23:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.783+0000 7fd3a8e6db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab86b9e2e0 con 0x55ab86b31cc0 2023-09-13T03:23:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.783+0000 7fd3a8e6db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab86b9e4a0 con 0x55ab86b31300 2023-09-13T03:23:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.784+0000 7fd3a8e6db80 10 client.0 _ll_register_callbacks cb 0x55ab8696ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:23:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.784+0000 7fd38e7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab86b31cc0 0x55ab86b40c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.784+0000 7fd38effd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab86b31300 0x55ab86b40700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.784+0000 7fd38effd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab86b31300 0x55ab86b40700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:47988/0 (socket says 172.21.15.64:47988) 2023-09-13T03:23:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.784+0000 7fd38e7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab86b31cc0 0x55ab86b40c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:43770/0 (socket says 172.21.15.64:43770) 2023-09-13T03:23:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.784+0000 7fd38effd700 1 -- 172.21.15.64:0/2058391966 learned_addr learned my addr 172.21.15.64:0/2058391966 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:23:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.784+0000 7fd38f7fe700 1 --2- 172.21.15.64:0/2058391966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab86b371a0 0x55ab86b9dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.784+0000 7fd38effd700 1 -- 172.21.15.64:0/2058391966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab86b31cc0 msgr2=0x55ab86b40c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:23:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.784+0000 7fd38effd700 1 --2- 172.21.15.64:0/2058391966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab86b31cc0 0x55ab86b40c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.784+0000 7fd38effd700 1 -- 172.21.15.64:0/2058391966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab86b371a0 msgr2=0x55ab86b9dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:23:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.784+0000 7fd38effd700 1 --2- 172.21.15.64:0/2058391966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab86b371a0 0x55ab86b9dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.784+0000 7fd38effd700 1 -- 172.21.15.64:0/2058391966 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ab86898c00 con 0x55ab86b31300 2023-09-13T03:23:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.784+0000 7fd38f7fe700 1 --2- 172.21.15.64:0/2058391966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab86b371a0 0x55ab86b9dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:23:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.784+0000 7fd38effd700 1 --2- 172.21.15.64:0/2058391966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab86b31300 0x55ab86b40700 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd38000c5b0 tx=0x7fd3800a16d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8af37c6a430e8eb6 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:23:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.784+0000 7fd372ff5700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:23:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.784+0000 7fd372ff5700 1 -- 172.21.15.64:0/2058391966 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3800978d0 con 0x55ab86b31300 2023-09-13T03:23:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.784+0000 7fd372ff5700 1 -- 172.21.15.64:0/2058391966 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8528+0+0 (secure 0 0 0) 0x7fd380099790 con 0x55ab86b31300 2023-09-13T03:23:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.784+0000 7fd3a8e6db80 1 -- 172.21.15.64:0/2058391966 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd38009d5c0 con 0x55ab86b31300 2023-09-13T03:23:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.785+0000 7fd372ff5700 1 -- 172.21.15.64:0/2058391966 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd380099c00 con 0x55ab86b31300 2023-09-13T03:23:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.785+0000 7fd3a8e6db80 10 client.26920 fetch_fsmap learned FSMap version 59 2023-09-13T03:23:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.785+0000 7fd3a8e6db80 1 -- 172.21.15.64:0/2058391966 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x55ab868ae250 con 0x55ab86b31300 2023-09-13T03:23:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.785+0000 7fd372ff5700 1 -- 172.21.15.64:0/2058391966 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd38009d5c0 con 0x55ab86b31300 2023-09-13T03:23:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.785+0000 7fd372ff5700 1 -- 172.21.15.64:0/2058391966 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 59) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fd38007e030 con 0x55ab86b31300 2023-09-13T03:23:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.785+0000 7fd3a8e6db80 10 client.26920 fetch_fsmap finished waiting for FSMap version 59 2023-09-13T03:23:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.785+0000 7fd3a8e6db80 10 client.26920 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:23:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.785+0000 7fd3a8e6db80 20 client.26920 populate_metadata read hostname 'smithi064' 2023-09-13T03:23:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.785+0000 7fd3a8e6db80 1 --2- 172.21.15.64:0/2058391966 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55ab86b9f6a0 0x55ab86ba1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.785+0000 7fd3a8e6db80 4 client.26920 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:23:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.785+0000 7fd3a8e6db80 1 -- 172.21.15.64:0/2058391966 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ab86a7c8c0 con 0x55ab86b9f6a0 2023-09-13T03:23:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.785+0000 7fd38e7fc700 1 --2- 172.21.15.64:0/2058391966 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55ab86b9f6a0 0x55ab86ba1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.786+0000 7fd38e7fc700 1 --2- 172.21.15.64:0/2058391966 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55ab86b9f6a0 0x55ab86ba1a90 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8708fbe0217ec4ce server_cookie=259d377252d206d5 in_seq=0 out_seq=0 2023-09-13T03:23:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.786+0000 7fd372ff5700 10 client.26920 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:23:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.812+0000 7fd372ff5700 1 -- 172.21.15.64:0/2058391966 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ab86a7c8c0 con 0x55ab86b9f6a0 2023-09-13T03:23:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.812+0000 7fd372ff5700 10 client.26920 handle_command_reply: tid=0 2023-09-13T03:23:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.812+0000 7fd3a8e6db80 1 client.26920 shutdown 2023-09-13T03:23:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.812+0000 7fd3a8e6db80 1 -- 172.21.15.64:0/2058391966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab86b31300 msgr2=0x55ab86b40700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:23:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.812+0000 7fd3a8e6db80 1 --2- 172.21.15.64:0/2058391966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab86b31300 0x55ab86b40700 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd38000c5b0 tx=0x7fd3800a16d0 comp rx=0 tx=0).stop 2023-09-13T03:23:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.812+0000 7fd3a8e6db80 1 -- 172.21.15.64:0/2058391966 shutdown_connections 2023-09-13T03:23:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.812+0000 7fd3a8e6db80 1 --2- 172.21.15.64:0/2058391966 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55ab86b9f6a0 0x55ab86ba1a90 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.812+0000 7fd3a8e6db80 1 --2- 172.21.15.64:0/2058391966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ab86b371a0 0x55ab86b9dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.812+0000 7fd372ff5700 0 client.26920 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:23:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.812+0000 7fd3a8e6db80 1 --2- 172.21.15.64:0/2058391966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab86b31300 0x55ab86b40700 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.812+0000 7fd3a8e6db80 1 --2- 172.21.15.64:0/2058391966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab86b31cc0 0x55ab86b40c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.812+0000 7fd3a8e6db80 1 -- 172.21.15.64:0/2058391966 >> 172.21.15.64:0/2058391966 conn(0x55ab86aa6030 msgr2=0x55ab86b35150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:23:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.812+0000 7fd3a8e6db80 1 -- 172.21.15.64:0/2058391966 shutdown_connections 2023-09-13T03:23:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.812+0000 7fd3a8e6db80 1 -- 172.21.15.64:0/2058391966 wait complete. 2023-09-13T03:23:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.812+0000 7fd3a8e6db80 20 client.26920 trim_cache size 0 max 16384 2023-09-13T03:23:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:23:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:23:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:23:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:23:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.813+0000 7fd3955ff700 1 -- 172.21.15.64:0/2608902915 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fd3780a07b0 msgr2=0x7fd3780a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:23:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.813+0000 7fd3955ff700 1 --2- 172.21.15.64:0/2608902915 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fd3780a07b0 0x7fd3780a2c70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd38800f540 tx=0x7fd38800b040 comp rx=0 tx=0).stop 2023-09-13T03:23:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.813+0000 7fd3955ff700 1 -- 172.21.15.64:0/2608902915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3900679f0 msgr2=0x7fd3900ffa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:23:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.813+0000 7fd3955ff700 1 --2- 172.21.15.64:0/2608902915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3900679f0 0x7fd3900ffa10 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd38000eb60 tx=0x7fd38000ee70 comp rx=0 tx=0).stop 2023-09-13T03:23:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.813+0000 7fd3955ff700 1 -- 172.21.15.64:0/2608902915 shutdown_connections 2023-09-13T03:23:07.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.813+0000 7fd3955ff700 1 --2- 172.21.15.64:0/2608902915 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fd3780a07b0 0x7fd3780a2c70 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:07.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.813+0000 7fd3955ff700 1 --2- 172.21.15.64:0/2608902915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd390106650 0x7fd3901a4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:07.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.813+0000 7fd3955ff700 1 --2- 172.21.15.64:0/2608902915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3900679f0 0x7fd3900ffa10 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:07.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.813+0000 7fd3955ff700 1 --2- 172.21.15.64:0/2608902915 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3900683b0 0x7fd3900fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:07.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.813+0000 7fd3955ff700 1 -- 172.21.15.64:0/2608902915 >> 172.21.15.64:0/2608902915 conn(0x7fd390074f30 msgr2=0x7fd3900fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:23:07.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.813+0000 7fd3955ff700 1 -- 172.21.15.64:0/2608902915 shutdown_connections 2023-09-13T03:23:07.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:07.813+0000 7fd3955ff700 1 -- 172.21.15.64:0/2608902915 wait complete. 2023-09-13T03:23:07.942 INFO:tasks.workunit.client.0.smithi064.stdout: 1 214512 2.00 MB/sec execute 432 sec latency 133.590 ms 2023-09-13T03:23:08.942 INFO:tasks.workunit.client.0.smithi064.stdout: 1 216509 2.03 MB/sec execute 433 sec latency 178.618 ms 2023-09-13T03:23:08.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:08 smithi142 ceph-mon[134082]: pgmap v309: 97 pgs: 97 active+clean; 100 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 948 KiB/s wr, 98 op/s 2023-09-13T03:23:09.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:08 smithi170 ceph-mon[134328]: pgmap v309: 97 pgs: 97 active+clean; 100 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 948 KiB/s wr, 98 op/s 2023-09-13T03:23:09.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:08 smithi064 ceph-mon[97675]: pgmap v309: 97 pgs: 97 active+clean; 100 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 948 KiB/s wr, 98 op/s 2023-09-13T03:23:09.942 INFO:tasks.workunit.client.0.smithi064.stdout: 1 217558 2.03 MB/sec execute 434 sec latency 18.944 ms 2023-09-13T03:23:10.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:10 smithi170 ceph-mon[134328]: osdmap e94: 12 total, 12 up, 12 in 2023-09-13T03:23:10.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:10 smithi170 ceph-mon[134328]: pgmap v311: 97 pgs: 97 active+clean; 100 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 776 KiB/s wr, 100 op/s 2023-09-13T03:23:10.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:10 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:23:10.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:10 smithi064 ceph-mon[97675]: osdmap e94: 12 total, 12 up, 12 in 2023-09-13T03:23:10.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:10 smithi064 ceph-mon[97675]: pgmap v311: 97 pgs: 97 active+clean; 100 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 776 KiB/s wr, 100 op/s 2023-09-13T03:23:10.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:10 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:23:10.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:10 smithi142 ceph-mon[134082]: osdmap e94: 12 total, 12 up, 12 in 2023-09-13T03:23:10.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:10 smithi142 ceph-mon[134082]: pgmap v311: 97 pgs: 97 active+clean; 100 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 776 KiB/s wr, 100 op/s 2023-09-13T03:23:10.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:10 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:23:10.942 INFO:tasks.workunit.client.0.smithi064.stdout: 1 218268 2.03 MB/sec execute 435 sec latency 520.822 ms 2023-09-13T03:23:11.943 INFO:tasks.workunit.client.0.smithi064.stdout: 1 218268 2.03 MB/sec execute 436 sec latency 1520.909 ms 2023-09-13T03:23:12.943 INFO:tasks.workunit.client.0.smithi064.stdout: 1 218268 2.02 MB/sec execute 437 sec latency 2521.048 ms 2023-09-13T03:23:12.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:12 smithi142 ceph-mon[134082]: pgmap v312: 97 pgs: 97 active+clean; 101 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 252 op/s 2023-09-13T03:23:13.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:12 smithi170 ceph-mon[134328]: pgmap v312: 97 pgs: 97 active+clean; 101 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 252 op/s 2023-09-13T03:23:13.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:12 smithi064 ceph-mon[97675]: pgmap v312: 97 pgs: 97 active+clean; 101 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 252 op/s 2023-09-13T03:23:13.943 INFO:tasks.workunit.client.0.smithi064.stdout: 1 218268 2.02 MB/sec execute 438 sec latency 3521.153 ms 2023-09-13T03:23:14.943 INFO:tasks.workunit.client.0.smithi064.stdout: 1 218268 2.02 MB/sec execute 439 sec latency 4521.283 ms 2023-09-13T03:23:14.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:14 smithi142 ceph-mon[134082]: pgmap v313: 97 pgs: 97 active+clean; 101 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 226 op/s 2023-09-13T03:23:15.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:14 smithi170 ceph-mon[134328]: pgmap v313: 97 pgs: 97 active+clean; 101 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 226 op/s 2023-09-13T03:23:15.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:14 smithi064 ceph-mon[97675]: pgmap v313: 97 pgs: 97 active+clean; 101 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 226 op/s 2023-09-13T03:23:15.943 INFO:tasks.workunit.client.0.smithi064.stdout: 1 218268 2.01 MB/sec execute 440 sec latency 5521.451 ms 2023-09-13T03:23:16.943 INFO:tasks.workunit.client.0.smithi064.stdout: 1 218268 2.01 MB/sec execute 441 sec latency 6521.605 ms 2023-09-13T03:23:16.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:16 smithi142 ceph-mon[134082]: pgmap v314: 97 pgs: 97 active+clean; 100 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 140 op/s 2023-09-13T03:23:17.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:16 smithi170 ceph-mon[134328]: pgmap v314: 97 pgs: 97 active+clean; 100 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 140 op/s 2023-09-13T03:23:17.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:16 smithi064 ceph-mon[97675]: pgmap v314: 97 pgs: 97 active+clean; 100 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 140 op/s 2023-09-13T03:23:17.943 INFO:tasks.workunit.client.0.smithi064.stdout: 1 218268 2.00 MB/sec execute 442 sec latency 7521.770 ms 2023-09-13T03:23:18.944 INFO:tasks.workunit.client.0.smithi064.stdout: 1 218268 2.00 MB/sec execute 443 sec latency 8521.909 ms 2023-09-13T03:23:18.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:18 smithi142 ceph-mon[134082]: pgmap v315: 97 pgs: 97 active+clean; 100 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 140 op/s 2023-09-13T03:23:19.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:18 smithi170 ceph-mon[134328]: pgmap v315: 97 pgs: 97 active+clean; 100 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 140 op/s 2023-09-13T03:23:19.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:18 smithi064 ceph-mon[97675]: pgmap v315: 97 pgs: 97 active+clean; 100 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 140 op/s 2023-09-13T03:23:19.944 INFO:tasks.workunit.client.0.smithi064.stdout: 1 218268 1.99 MB/sec execute 444 sec latency 9522.050 ms 2023-09-13T03:23:20.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:20 smithi170 ceph-mon[134328]: pgmap v316: 97 pgs: 97 active+clean; 100 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 136 op/s 2023-09-13T03:23:20.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:20 smithi064 ceph-mon[97675]: pgmap v316: 97 pgs: 97 active+clean; 100 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 136 op/s 2023-09-13T03:23:20.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:20 smithi142 ceph-mon[134082]: pgmap v316: 97 pgs: 97 active+clean; 100 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 136 op/s 2023-09-13T03:23:20.944 INFO:tasks.workunit.client.0.smithi064.stdout: 1 218268 1.99 MB/sec execute 445 sec latency 10522.140 ms 2023-09-13T03:23:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.974+0000 7fce773ff700 1 -- 172.21.15.64:0/1752180557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce70100f70 msgr2=0x7fce7010d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:23:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.974+0000 7fce773ff700 1 --2- 172.21.15.64:0/1752180557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce70100f70 0x7fce7010d430 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fce6800b0d0 tx=0x7fce6800b3e0 comp rx=0 tx=0).stop 2023-09-13T03:23:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.974+0000 7fce773ff700 1 -- 172.21.15.64:0/1752180557 shutdown_connections 2023-09-13T03:23:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.974+0000 7fce773ff700 1 --2- 172.21.15.64:0/1752180557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce70100f70 0x7fce7010d430 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.974+0000 7fce773ff700 1 --2- 172.21.15.64:0/1752180557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce70106670 0x7fce70106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.974+0000 7fce773ff700 1 --2- 172.21.15.64:0/1752180557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce701005b0 0x7fce70100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.974+0000 7fce773ff700 1 -- 172.21.15.64:0/1752180557 >> 172.21.15.64:0/1752180557 conn(0x7fce700fc060 msgr2=0x7fce700fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:23:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.974+0000 7fce773ff700 1 -- 172.21.15.64:0/1752180557 shutdown_connections 2023-09-13T03:23:20.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.974+0000 7fce773ff700 1 -- 172.21.15.64:0/1752180557 wait complete. 2023-09-13T03:23:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.975+0000 7fce773ff700 1 Processor -- start 2023-09-13T03:23:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.975+0000 7fce773ff700 1 -- start start 2023-09-13T03:23:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.976+0000 7fce773ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce701005b0 0x7fce701967a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.976+0000 7fce773ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce70100f70 0x7fce70196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.976+0000 7fce773ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce70106670 0x7fce7019b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.976+0000 7fce773ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7019b950 con 0x7fce701005b0 2023-09-13T03:23:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.976+0000 7fce773ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7019bac0 con 0x7fce70106670 2023-09-13T03:23:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.976+0000 7fce773ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7019bc80 con 0x7fce70100f70 2023-09-13T03:23:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.976+0000 7fce7519b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce701005b0 0x7fce701967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.976+0000 7fce7499a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce70100f70 0x7fce70196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.976+0000 7fce7519b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce701005b0 0x7fce701967a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47608/0 (socket says 172.21.15.64:47608) 2023-09-13T03:23:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.976+0000 7fce7519b700 1 -- 172.21.15.64:0/57167160 learned_addr learned my addr 172.21.15.64:0/57167160 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:23:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.976+0000 7fce7499a700 1 --2- 172.21.15.64:0/57167160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce70100f70 0x7fce70196ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:43030/0 (socket says 172.21.15.64:43030) 2023-09-13T03:23:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.976+0000 7fce7599c700 1 --2- 172.21.15.64:0/57167160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce70106670 0x7fce7019b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.977+0000 7fce7499a700 1 -- 172.21.15.64:0/57167160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce70106670 msgr2=0x7fce7019b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:23:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.977+0000 7fce7499a700 1 --2- 172.21.15.64:0/57167160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce70106670 0x7fce7019b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.977+0000 7fce7499a700 1 -- 172.21.15.64:0/57167160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce701005b0 msgr2=0x7fce701967a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:23:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.977+0000 7fce7499a700 1 --2- 172.21.15.64:0/57167160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce701005b0 0x7fce701967a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.977+0000 7fce7499a700 1 -- 172.21.15.64:0/57167160 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce68009d00 con 0x7fce70100f70 2023-09-13T03:23:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.977+0000 7fce7499a700 1 --2- 172.21.15.64:0/57167160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce70100f70 0x7fce70196ce0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fce6000e9f0 tx=0x7fce6000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:23:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.977+0000 7fce667fc700 1 -- 172.21.15.64:0/57167160 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6000c9c0 con 0x7fce70100f70 2023-09-13T03:23:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.977+0000 7fce667fc700 1 -- 172.21.15.64:0/57167160 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce60010430 con 0x7fce70100f70 2023-09-13T03:23:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.977+0000 7fce773ff700 1 -- 172.21.15.64:0/57167160 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce7019bf60 con 0x7fce70100f70 2023-09-13T03:23:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.977+0000 7fce667fc700 1 -- 172.21.15.64:0/57167160 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce60018680 con 0x7fce70100f70 2023-09-13T03:23:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.977+0000 7fce773ff700 1 -- 172.21.15.64:0/57167160 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce701a3810 con 0x7fce70100f70 2023-09-13T03:23:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.981+0000 7fce667fc700 1 -- 172.21.15.64:0/57167160 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fce60010860 con 0x7fce70100f70 2023-09-13T03:23:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.982+0000 7fce8adddb80 1 Processor -- start 2023-09-13T03:23:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.982+0000 7fce8adddb80 1 -- start start 2023-09-13T03:23:20.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.982+0000 7fce8adddb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d801281a0 0x562d80128580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:20.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.983+0000 7fce8adddb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d80122300 0x562d80122780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:20.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.983+0000 7fce8adddb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562d80122cc0 0x562d8012ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:20.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.983+0000 7fce8adddb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d8007b6d0 con 0x562d80122300 2023-09-13T03:23:20.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.983+0000 7fce8adddb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d80078ff0 con 0x562d801281a0 2023-09-13T03:23:20.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.983+0000 7fce8adddb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d800748e0 con 0x562d80122cc0 2023-09-13T03:23:20.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.983+0000 7fce7519b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d80122300 0x562d80122780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:20.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.983+0000 7fce7599c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d801281a0 0x562d80128580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:20.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.983+0000 7fce7599c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d801281a0 0x562d80128580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:34854/0 (socket says 172.21.15.64:34854) 2023-09-13T03:23:20.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.983+0000 7fce7599c700 1 -- 172.21.15.64:0/4015744515 learned_addr learned my addr 172.21.15.64:0/4015744515 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:23:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.983+0000 7fce7499a700 1 --2- 172.21.15.64:0/4015744515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562d80122cc0 0x562d8012ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.983+0000 7fce7519b700 1 -- 172.21.15.64:0/4015744515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562d80122cc0 msgr2=0x562d8012ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:23:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.983+0000 7fce7519b700 1 --2- 172.21.15.64:0/4015744515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562d80122cc0 0x562d8012ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.983+0000 7fce7519b700 1 -- 172.21.15.64:0/4015744515 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d801281a0 msgr2=0x562d80128580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:23:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.983+0000 7fce7519b700 1 --2- 172.21.15.64:0/4015744515 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d801281a0 0x562d80128580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.983+0000 7fce7519b700 1 -- 172.21.15.64:0/4015744515 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d7fea0d80 con 0x562d80122300 2023-09-13T03:23:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.983+0000 7fce7599c700 1 --2- 172.21.15.64:0/4015744515 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d801281a0 0x562d80128580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:23:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.983+0000 7fce667fc700 1 --2- 172.21.15.64:0/57167160 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fce5c0a07b0 0x7fce5c0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.984+0000 7fce667fc700 1 -- 172.21.15.64:0/57167160 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fce60014070 con 0x7fce70100f70 2023-09-13T03:23:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.984+0000 7fce7519b700 1 --2- 172.21.15.64:0/4015744515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d80122300 0x562d80122780 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fce6c009d10 tx=0x7fce6c011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79030dfd8b0c8826 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:23:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.984+0000 7fce7599c700 1 --2- 172.21.15.64:0/57167160 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fce5c0a07b0 0x7fce5c0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.984+0000 7fce5b7fe700 1 -- 172.21.15.64:0/4015744515 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c004250 con 0x562d80122300 2023-09-13T03:23:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.984+0000 7fce5b7fe700 1 -- 172.21.15.64:0/4015744515 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce6c004cf0 con 0x562d80122300 2023-09-13T03:23:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.984+0000 7fce5b7fe700 1 -- 172.21.15.64:0/4015744515 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c0055e0 con 0x562d80122300 2023-09-13T03:23:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.985+0000 7fce8adddb80 1 -- 172.21.15.64:0/4015744515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d80122300 msgr2=0x562d80122780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:23:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.985+0000 7fce7599c700 1 --2- 172.21.15.64:0/57167160 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fce5c0a07b0 0x7fce5c0a2c70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fce68009500 tx=0x7fce6800b040 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:23:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.985+0000 7fce8adddb80 1 --2- 172.21.15.64:0/4015744515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d80122300 0x562d80122780 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fce6c009d10 tx=0x7fce6c011720 comp rx=0 tx=0).stop 2023-09-13T03:23:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.985+0000 7fce8adddb80 1 -- 172.21.15.64:0/4015744515 shutdown_connections 2023-09-13T03:23:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.985+0000 7fce8adddb80 1 --2- 172.21.15.64:0/4015744515 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d801281a0 0x562d80128580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.985+0000 7fce8adddb80 1 --2- 172.21.15.64:0/4015744515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562d80122cc0 0x562d8012ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.985+0000 7fce8adddb80 1 --2- 172.21.15.64:0/4015744515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d80122300 0x562d80122780 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.985+0000 7fce8adddb80 1 -- 172.21.15.64:0/4015744515 >> 172.21.15.64:0/4015744515 conn(0x562d80097030 msgr2=0x562d8011d560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:23:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.985+0000 7fce8adddb80 1 -- 172.21.15.64:0/4015744515 shutdown_connections 2023-09-13T03:23:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.985+0000 7fce8adddb80 1 -- 172.21.15.64:0/4015744515 wait complete. 2023-09-13T03:23:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.986+0000 7fce8adddb80 1 -- start start 2023-09-13T03:23:20.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.986+0000 7fce8adddb80 1 Processor -- start 2023-09-13T03:23:20.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.986+0000 7fce8adddb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d80122300 0x562d8012ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:20.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.986+0000 7fce8adddb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d80122cc0 0x562d8012c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:20.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.986+0000 7fce8adddb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562d801281a0 0x562d8012c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:20.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.986+0000 7fce8adddb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d80012b10 con 0x562d80122300 2023-09-13T03:23:20.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.986+0000 7fce8adddb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d80012810 con 0x562d80122cc0 2023-09-13T03:23:20.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.986+0000 7fce8adddb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d7fef68c0 con 0x562d801281a0 2023-09-13T03:23:20.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.986+0000 7fce8adddb80 10 client.0 _ll_register_callbacks cb 0x562d7ff5ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:23:20.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.986+0000 7fce7519b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d80122300 0x562d8012ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:20.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.987+0000 7fce7519b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d80122300 0x562d8012ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47634/0 (socket says 172.21.15.64:47634) 2023-09-13T03:23:20.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.987+0000 7fce7519b700 1 -- 172.21.15.64:0/1713378202 learned_addr learned my addr 172.21.15.64:0/1713378202 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:23:20.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.987+0000 7fce7499a700 1 --2- 172.21.15.64:0/1713378202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d80122cc0 0x562d8012c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:20.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.987+0000 7fce7599c700 1 --2- 172.21.15.64:0/1713378202 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562d801281a0 0x562d8012c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:20.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.987+0000 7fce7519b700 1 -- 172.21.15.64:0/1713378202 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562d801281a0 msgr2=0x562d8012c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:23:20.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.987+0000 7fce7519b700 1 --2- 172.21.15.64:0/1713378202 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562d801281a0 0x562d8012c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:20.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.987+0000 7fce7519b700 1 -- 172.21.15.64:0/1713378202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d80122cc0 msgr2=0x562d8012c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:23:21.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.987+0000 7fce7519b700 1 --2- 172.21.15.64:0/1713378202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d80122cc0 0x562d8012c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:21.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.987+0000 7fce7519b700 1 -- 172.21.15.64:0/1713378202 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce7010dd30 con 0x562d80122300 2023-09-13T03:23:21.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.987+0000 7fce7499a700 1 --2- 172.21.15.64:0/1713378202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d80122cc0 0x562d8012c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:23:21.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.987+0000 7fce7519b700 1 --2- 172.21.15.64:0/1713378202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d80122300 0x562d8012ed60 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0096b0 tx=0x7fce6c0045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ce160864da137e server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:23:21.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.987+0000 7fce58ff9700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:23:21.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.988+0000 7fce58ff9700 1 -- 172.21.15.64:0/1713378202 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c004750 con 0x562d80122300 2023-09-13T03:23:21.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.988+0000 7fce8adddb80 1 -- 172.21.15.64:0/1713378202 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562d7ffcc490 con 0x562d80122300 2023-09-13T03:23:21.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.988+0000 7fce58ff9700 1 -- 172.21.15.64:0/1713378202 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fce6c0048b0 con 0x562d80122300 2023-09-13T03:23:21.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.988+0000 7fce58ff9700 1 -- 172.21.15.64:0/1713378202 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce6c004a70 con 0x562d80122300 2023-09-13T03:23:21.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.988+0000 7fce8adddb80 10 client.16908 fetch_fsmap learned FSMap version 60 2023-09-13T03:23:21.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.988+0000 7fce8adddb80 1 -- 172.21.15.64:0/1713378202 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x7fce6c009970 con 0x562d80122300 2023-09-13T03:23:21.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.988+0000 7fce58ff9700 1 -- 172.21.15.64:0/1713378202 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562d7ffcc490 con 0x562d80122300 2023-09-13T03:23:21.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.988+0000 7fce58ff9700 1 -- 172.21.15.64:0/1713378202 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 60) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fce6c003b60 con 0x562d80122300 2023-09-13T03:23:21.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.989+0000 7fce8adddb80 10 client.16908 fetch_fsmap finished waiting for FSMap version 60 2023-09-13T03:23:21.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.989+0000 7fce8adddb80 10 client.16908 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:23:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.989+0000 7fce8adddb80 20 client.16908 populate_metadata read hostname 'smithi064' 2023-09-13T03:23:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.989+0000 7fce8adddb80 1 --2- 172.21.15.64:0/1713378202 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x562d80140690 0x562d80142ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.989+0000 7fce8adddb80 4 client.16908 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:23:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.989+0000 7fce8adddb80 1 -- 172.21.15.64:0/1713378202 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d80089840 con 0x562d80140690 2023-09-13T03:23:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.989+0000 7fce7499a700 1 --2- 172.21.15.64:0/1713378202 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x562d80140690 0x562d80142ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.990+0000 7fce7499a700 1 --2- 172.21.15.64:0/1713378202 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x562d80140690 0x562d80142ab0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dac0f8ddfb2eb9b server_cookie=6326279eaf4f3d43 in_seq=0 out_seq=0 2023-09-13T03:23:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.990+0000 7fce58ff9700 10 client.16908 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:23:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.990+0000 7fce58ff9700 1 -- 172.21.15.64:0/1713378202 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562d80089840 con 0x562d80140690 2023-09-13T03:23:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.991+0000 7fce58ff9700 10 client.16908 handle_command_reply: tid=0 2023-09-13T03:23:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.991+0000 7fce8adddb80 1 client.16908 shutdown 2023-09-13T03:23:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.991+0000 7fce8adddb80 1 -- 172.21.15.64:0/1713378202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d80122300 msgr2=0x562d8012ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:23:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.991+0000 7fce8adddb80 1 --2- 172.21.15.64:0/1713378202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d80122300 0x562d8012ed60 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0096b0 tx=0x7fce6c0045f0 comp rx=0 tx=0).stop 2023-09-13T03:23:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.991+0000 7fce8adddb80 1 -- 172.21.15.64:0/1713378202 shutdown_connections 2023-09-13T03:23:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.991+0000 7fce8adddb80 1 --2- 172.21.15.64:0/1713378202 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x562d80140690 0x562d80142ab0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.991+0000 7fce8adddb80 1 --2- 172.21.15.64:0/1713378202 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562d801281a0 0x562d8012c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.991+0000 7fce8adddb80 1 --2- 172.21.15.64:0/1713378202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d80122300 0x562d8012ed60 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.991+0000 7fce8adddb80 1 --2- 172.21.15.64:0/1713378202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d80122cc0 0x562d8012c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.991+0000 7fce58ff9700 0 client.16908 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:23:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.991+0000 7fce8adddb80 1 -- 172.21.15.64:0/1713378202 >> 172.21.15.64:0/1713378202 conn(0x562d80097030 msgr2=0x562d8011c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:23:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.991+0000 7fce8adddb80 1 -- 172.21.15.64:0/1713378202 shutdown_connections 2023-09-13T03:23:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.991+0000 7fce8adddb80 1 -- 172.21.15.64:0/1713378202 wait complete. 2023-09-13T03:23:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:20.991+0000 7fce8adddb80 20 client.16908 trim_cache size 0 max 16384 2023-09-13T03:23:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.007+0000 7fce8adddb80 1 Processor -- start 2023-09-13T03:23:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.007+0000 7fce8adddb80 1 -- start start 2023-09-13T03:23:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.007+0000 7fce8adddb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d80122300 0x562d80137970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.007+0000 7fce8adddb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562d80122cc0 0x562d8012e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.007+0000 7fce8adddb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d801281a0 0x562d801261f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.007+0000 7fce8adddb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d8012d610 con 0x562d801281a0 2023-09-13T03:23:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.007+0000 7fce8adddb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d80127c70 con 0x562d80122300 2023-09-13T03:23:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.007+0000 7fce8adddb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d80137ee0 con 0x562d80122cc0 2023-09-13T03:23:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.007+0000 7fce7599c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d801281a0 0x562d801261f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.007+0000 7fce7599c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d801281a0 0x562d801261f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47646/0 (socket says 172.21.15.64:47646) 2023-09-13T03:23:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.007+0000 7fce7599c700 1 -- 172.21.15.64:0/2356521335 learned_addr learned my addr 172.21.15.64:0/2356521335 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:23:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.007+0000 7fce7519b700 1 --2- 172.21.15.64:0/2356521335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d80122300 0x562d80137970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.008+0000 7fce7499a700 1 --2- 172.21.15.64:0/2356521335 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562d80122cc0 0x562d8012e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.008+0000 7fce7599c700 1 -- 172.21.15.64:0/2356521335 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562d80122cc0 msgr2=0x562d8012e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:23:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.008+0000 7fce7599c700 1 --2- 172.21.15.64:0/2356521335 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562d80122cc0 0x562d8012e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.008+0000 7fce7599c700 1 -- 172.21.15.64:0/2356521335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d80122300 msgr2=0x562d80137970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:23:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.008+0000 7fce7599c700 1 --2- 172.21.15.64:0/2356521335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d80122300 0x562d80137970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.008+0000 7fce7599c700 1 -- 172.21.15.64:0/2356521335 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d7fe89c00 con 0x562d801281a0 2023-09-13T03:23:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.008+0000 7fce7499a700 1 --2- 172.21.15.64:0/2356521335 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562d80122cc0 0x562d8012e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:23:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.008+0000 7fce7599c700 1 --2- 172.21.15.64:0/2356521335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d801281a0 0x562d801261f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fce68008ee0 tx=0x7fce68020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba5be5f0e54650 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:23:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.008+0000 7fce5bfff700 1 -- 172.21.15.64:0/2356521335 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6800e040 con 0x562d801281a0 2023-09-13T03:23:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.008+0000 7fce5bfff700 1 -- 172.21.15.64:0/2356521335 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce6801d420 con 0x562d801281a0 2023-09-13T03:23:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.008+0000 7fce5bfff700 1 -- 172.21.15.64:0/2356521335 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce68008f10 con 0x562d801281a0 2023-09-13T03:23:21.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.009+0000 7fce8adddb80 1 -- 172.21.15.64:0/2356521335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d801281a0 msgr2=0x562d801261f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:23:21.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.009+0000 7fce8adddb80 1 --2- 172.21.15.64:0/2356521335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d801281a0 0x562d801261f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fce68008ee0 tx=0x7fce68020000 comp rx=0 tx=0).stop 2023-09-13T03:23:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.009+0000 7fce8adddb80 1 -- 172.21.15.64:0/2356521335 shutdown_connections 2023-09-13T03:23:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.009+0000 7fce8adddb80 1 --2- 172.21.15.64:0/2356521335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d80122300 0x562d80137970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.009+0000 7fce8adddb80 1 --2- 172.21.15.64:0/2356521335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d801281a0 0x562d801261f0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.009+0000 7fce8adddb80 1 --2- 172.21.15.64:0/2356521335 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562d80122cc0 0x562d8012e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.009+0000 7fce8adddb80 1 -- 172.21.15.64:0/2356521335 >> 172.21.15.64:0/2356521335 conn(0x562d80097030 msgr2=0x562d801256b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:23:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.009+0000 7fce8adddb80 1 -- 172.21.15.64:0/2356521335 shutdown_connections 2023-09-13T03:23:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.009+0000 7fce8adddb80 1 -- 172.21.15.64:0/2356521335 wait complete. 2023-09-13T03:23:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.010+0000 7fce8adddb80 1 -- start start 2023-09-13T03:23:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.010+0000 7fce8adddb80 1 Processor -- start 2023-09-13T03:23:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.010+0000 7fce8adddb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562d80122300 0x562d8011cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.010+0000 7fce8adddb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d80122cc0 0x562d8011d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.010+0000 7fce8adddb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d801281a0 0x562d8011bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.010+0000 7fce8adddb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d8011c490 con 0x562d801281a0 2023-09-13T03:23:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.010+0000 7fce8adddb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d8011c600 con 0x562d80122cc0 2023-09-13T03:23:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.010+0000 7fce8adddb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d8011c7c0 con 0x562d80122300 2023-09-13T03:23:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.010+0000 7fce8adddb80 10 client.0 _ll_register_callbacks cb 0x7fce60005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:23:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.011+0000 7fce7599c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d801281a0 0x562d8011bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.011+0000 7fce7499a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d80122cc0 0x562d8011d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.011+0000 7fce7599c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d801281a0 0x562d8011bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47652/0 (socket says 172.21.15.64:47652) 2023-09-13T03:23:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.011+0000 7fce7599c700 1 -- 172.21.15.64:0/4192930146 learned_addr learned my addr 172.21.15.64:0/4192930146 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:23:21.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.011+0000 7fce7519b700 1 --2- 172.21.15.64:0/4192930146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562d80122300 0x562d8011cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:21.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.011+0000 7fce7599c700 1 -- 172.21.15.64:0/4192930146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562d80122300 msgr2=0x562d8011cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:23:21.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.011+0000 7fce7599c700 1 --2- 172.21.15.64:0/4192930146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562d80122300 0x562d8011cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:21.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.011+0000 7fce7599c700 1 -- 172.21.15.64:0/4192930146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d80122cc0 msgr2=0x562d8011d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:23:21.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.011+0000 7fce7599c700 1 --2- 172.21.15.64:0/4192930146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d80122cc0 0x562d8011d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:21.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.011+0000 7fce7599c700 1 -- 172.21.15.64:0/4192930146 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce68023070 con 0x562d801281a0 2023-09-13T03:23:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.011+0000 7fce7599c700 1 --2- 172.21.15.64:0/4192930146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d801281a0 0x562d8011bf10 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fce6800b0d0 tx=0x7fce68007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c0e9ee2b7898c0e server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:23:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.011+0000 7fce58ff9700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:23:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.012+0000 7fce58ff9700 1 -- 172.21.15.64:0/4192930146 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6800e040 con 0x562d801281a0 2023-09-13T03:23:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.012+0000 7fce8adddb80 1 -- 172.21.15.64:0/4192930146 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce6c004cf0 con 0x562d801281a0 2023-09-13T03:23:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.012+0000 7fce58ff9700 1 -- 172.21.15.64:0/4192930146 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fce68028610 con 0x562d801281a0 2023-09-13T03:23:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.012+0000 7fce58ff9700 1 -- 172.21.15.64:0/4192930146 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce68008f10 con 0x562d801281a0 2023-09-13T03:23:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.012+0000 7fce8adddb80 10 client.16914 fetch_fsmap learned FSMap version 60 2023-09-13T03:23:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.012+0000 7fce8adddb80 1 -- 172.21.15.64:0/4192930146 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x562d7fec71f0 con 0x562d801281a0 2023-09-13T03:23:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.012+0000 7fce58ff9700 1 -- 172.21.15.64:0/4192930146 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce68046020 con 0x562d801281a0 2023-09-13T03:23:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.012+0000 7fce58ff9700 1 -- 172.21.15.64:0/4192930146 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 60) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fce68039680 con 0x562d801281a0 2023-09-13T03:23:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.013+0000 7fce8adddb80 10 client.16914 fetch_fsmap finished waiting for FSMap version 60 2023-09-13T03:23:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.013+0000 7fce8adddb80 10 client.16914 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:23:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.013+0000 7fce8adddb80 20 client.16914 populate_metadata read hostname 'smithi064' 2023-09-13T03:23:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.013+0000 7fce8adddb80 1 --2- 172.21.15.64:0/4192930146 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x562d80195170 0x562d80197560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.013+0000 7fce8adddb80 4 client.16914 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:23:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.013+0000 7fce8adddb80 1 -- 172.21.15.64:0/4192930146 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562d8006d8c0 con 0x562d80195170 2023-09-13T03:23:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.013+0000 7fce7519b700 1 --2- 172.21.15.64:0/4192930146 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x562d80195170 0x562d80197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.013+0000 7fce7519b700 1 --2- 172.21.15.64:0/4192930146 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x562d80195170 0x562d80197560 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad32fadaa71d1182 server_cookie=62fbd5d37b38b76a in_seq=0 out_seq=0 2023-09-13T03:23:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:21.013+0000 7fce58ff9700 10 client.16914 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:23:21.945 INFO:tasks.workunit.client.0.smithi064.stdout: 1 218268 1.98 MB/sec execute 446 sec latency 11522.240 ms 2023-09-13T03:23:22.944 INFO:tasks.workunit.client.0.smithi064.stdout: 1 218268 1.98 MB/sec execute 447 sec latency 12522.303 ms 2023-09-13T03:23:22.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:22 smithi142 ceph-mon[134082]: pgmap v317: 97 pgs: 97 active+clean; 103 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 131 op/s 2023-09-13T03:23:23.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:22 smithi170 ceph-mon[134328]: pgmap v317: 97 pgs: 97 active+clean; 103 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 131 op/s 2023-09-13T03:23:23.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:22 smithi064 ceph-mon[97675]: pgmap v317: 97 pgs: 97 active+clean; 103 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 131 op/s 2023-09-13T03:23:23.944 INFO:tasks.workunit.client.0.smithi064.stdout: 1 218268 1.97 MB/sec execute 448 sec latency 13522.391 ms 2023-09-13T03:23:24.944 INFO:tasks.workunit.client.0.smithi064.stdout: 1 218268 1.97 MB/sec execute 449 sec latency 14522.488 ms 2023-09-13T03:23:24.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:24 smithi142 ceph-mon[134082]: pgmap v318: 97 pgs: 97 active+clean; 103 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 302 KiB/s wr, 29 op/s 2023-09-13T03:23:25.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:24 smithi170 ceph-mon[134328]: pgmap v318: 97 pgs: 97 active+clean; 103 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 302 KiB/s wr, 29 op/s 2023-09-13T03:23:25.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:24 smithi064 ceph-mon[97675]: pgmap v318: 97 pgs: 97 active+clean; 103 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 302 KiB/s wr, 29 op/s 2023-09-13T03:23:25.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:25.358+0000 7fce58ff9700 1 -- 172.21.15.64:0/4192930146 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562d8006d8c0 con 0x562d80195170 2023-09-13T03:23:25.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:25.358+0000 7fce58ff9700 10 client.16914 handle_command_reply: tid=0 2023-09-13T03:23:25.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:25.358+0000 7fce8adddb80 1 client.16914 shutdown 2023-09-13T03:23:25.360 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:25.358+0000 7fce8adddb80 1 -- 172.21.15.64:0/4192930146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d801281a0 msgr2=0x562d8011bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:23:25.360 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:25.358+0000 7fce8adddb80 1 --2- 172.21.15.64:0/4192930146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d801281a0 0x562d8011bf10 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fce6800b0d0 tx=0x7fce68007750 comp rx=0 tx=0).stop 2023-09-13T03:23:25.360 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:25.358+0000 7fce8adddb80 1 -- 172.21.15.64:0/4192930146 shutdown_connections 2023-09-13T03:23:25.360 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:25.358+0000 7fce8adddb80 1 --2- 172.21.15.64:0/4192930146 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x562d80195170 0x562d80197560 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:25.360 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:25.358+0000 7fce8adddb80 1 --2- 172.21.15.64:0/4192930146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d801281a0 0x562d8011bf10 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:25.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:25.358+0000 7fce8adddb80 1 --2- 172.21.15.64:0/4192930146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562d80122300 0x562d8011cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:25.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:25.358+0000 7fce8adddb80 1 --2- 172.21.15.64:0/4192930146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d80122cc0 0x562d8011d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:25.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:25.358+0000 7fce8adddb80 1 -- 172.21.15.64:0/4192930146 >> 172.21.15.64:0/4192930146 conn(0x562d80097030 msgr2=0x562d801261f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:23:25.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:25.358+0000 7fce58ff9700 0 client.16914 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:23:25.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:25.359+0000 7fce8adddb80 1 -- 172.21.15.64:0/4192930146 shutdown_connections 2023-09-13T03:23:25.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:25.359+0000 7fce8adddb80 1 -- 172.21.15.64:0/4192930146 wait complete. 2023-09-13T03:23:25.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:25.359+0000 7fce8adddb80 20 client.16914 trim_cache size 0 max 16384 2023-09-13T03:23:25.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:23:25.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:23:25.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:23:25.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:23:25.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:25.360+0000 7fce773ff700 1 -- 172.21.15.64:0/57167160 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fce5c0a07b0 msgr2=0x7fce5c0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:23:25.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:25.360+0000 7fce773ff700 1 --2- 172.21.15.64:0/57167160 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fce5c0a07b0 0x7fce5c0a2c70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fce68009500 tx=0x7fce6800b040 comp rx=0 tx=0).stop 2023-09-13T03:23:25.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:25.360+0000 7fce773ff700 1 -- 172.21.15.64:0/57167160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce70100f70 msgr2=0x7fce70196ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:23:25.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:25.360+0000 7fce773ff700 1 --2- 172.21.15.64:0/57167160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce70100f70 0x7fce70196ce0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fce6000e9f0 tx=0x7fce6000ed00 comp rx=0 tx=0).stop 2023-09-13T03:23:25.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:25.360+0000 7fce773ff700 1 -- 172.21.15.64:0/57167160 shutdown_connections 2023-09-13T03:23:25.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:25.360+0000 7fce773ff700 1 --2- 172.21.15.64:0/57167160 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fce5c0a07b0 0x7fce5c0a2c70 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:25.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:25.360+0000 7fce773ff700 1 --2- 172.21.15.64:0/57167160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce70106670 0x7fce7019b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:25.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:25.360+0000 7fce773ff700 1 --2- 172.21.15.64:0/57167160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce701005b0 0x7fce701967a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:25.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:25.360+0000 7fce773ff700 1 --2- 172.21.15.64:0/57167160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce70100f70 0x7fce70196ce0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:25.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:25.360+0000 7fce773ff700 1 -- 172.21.15.64:0/57167160 >> 172.21.15.64:0/57167160 conn(0x7fce700fc060 msgr2=0x7fce700fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:23:25.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:25.360+0000 7fce773ff700 1 -- 172.21.15.64:0/57167160 shutdown_connections 2023-09-13T03:23:25.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:25.360+0000 7fce773ff700 1 -- 172.21.15.64:0/57167160 wait complete. 2023-09-13T03:23:25.944 INFO:tasks.workunit.client.0.smithi064.stdout: 1 219427 1.98 MB/sec execute 450 sec latency 14936.563 ms 2023-09-13T03:23:26.944 INFO:tasks.workunit.client.0.smithi064.stdout: 1 220688 1.99 MB/sec execute 451 sec latency 17.919 ms 2023-09-13T03:23:26.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:26 smithi142 ceph-mon[134082]: pgmap v319: 97 pgs: 97 active+clean; 103 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 694 KiB/s wr, 58 op/s 2023-09-13T03:23:27.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:26 smithi170 ceph-mon[134328]: pgmap v319: 97 pgs: 97 active+clean; 103 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 694 KiB/s wr, 58 op/s 2023-09-13T03:23:27.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:26 smithi064 ceph-mon[97675]: pgmap v319: 97 pgs: 97 active+clean; 103 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 694 KiB/s wr, 58 op/s 2023-09-13T03:23:27.944 INFO:tasks.workunit.client.0.smithi064.stdout: 1 220688 1.99 MB/sec execute 452 sec latency 1015.276 ms 2023-09-13T03:23:28.944 INFO:tasks.workunit.client.0.smithi064.stdout: 1 221375 1.99 MB/sec execute 453 sec latency 1600.342 ms 2023-09-13T03:23:28.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:28 smithi142 ceph-mon[134082]: pgmap v320: 97 pgs: 97 active+clean; 103 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 650 KiB/s wr, 43 op/s 2023-09-13T03:23:28.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:28 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:23:29.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:28 smithi170 ceph-mon[134328]: pgmap v320: 97 pgs: 97 active+clean; 103 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 650 KiB/s wr, 43 op/s 2023-09-13T03:23:29.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:28 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:23:29.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:28 smithi064 ceph-mon[97675]: pgmap v320: 97 pgs: 97 active+clean; 103 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 650 KiB/s wr, 43 op/s 2023-09-13T03:23:29.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:28 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:23:29.944 INFO:tasks.workunit.client.0.smithi064.stdout: 1 221657 1.99 MB/sec execute 454 sec latency 543.266 ms 2023-09-13T03:23:30.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:30 smithi170 ceph-mon[134328]: pgmap v321: 97 pgs: 97 active+clean; 103 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 650 KiB/s wr, 43 op/s 2023-09-13T03:23:30.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:30 smithi064 ceph-mon[97675]: pgmap v321: 97 pgs: 97 active+clean; 103 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 650 KiB/s wr, 43 op/s 2023-09-13T03:23:30.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:30 smithi142 ceph-mon[134082]: pgmap v321: 97 pgs: 97 active+clean; 103 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 650 KiB/s wr, 43 op/s 2023-09-13T03:23:30.945 INFO:tasks.workunit.client.0.smithi064.stdout: 1 222064 1.99 MB/sec execute 455 sec latency 1379.974 ms 2023-09-13T03:23:31.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:31 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:23:31.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:31 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:23:31.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:31 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:23:31.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:31 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:23:31.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:31 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:23:31.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:31 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:23:31.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:31 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:23:31.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:31 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:23:31.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:31 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:23:31.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:31 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:23:31.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:31 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:23:31.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:31 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:23:31.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:31 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:23:31.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:31 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:23:31.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:31 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:23:31.945 INFO:tasks.workunit.client.0.smithi064.stdout: 1 223751 2.01 MB/sec execute 456 sec latency 19.076 ms 2023-09-13T03:23:32.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:32 smithi064 ceph-mon[97675]: pgmap v322: 97 pgs: 97 active+clean; 103 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 926 KiB/s wr, 85 op/s 2023-09-13T03:23:32.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:32 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:23:32.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:32 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:23:32.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:32 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:23:32.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:32 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:23:32.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:32 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:23:32.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:32 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:23:32.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:32 smithi064 ceph-mon[97675]: Adjusting osd_memory_target on smithi142 to 196.2M 2023-09-13T03:23:32.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:32 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:23:32.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:32 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:23:32.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:32 smithi064 ceph-mon[97675]: Unable to set osd_memory_target on smithi142 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-09-13T03:23:32.671 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:32 smithi064 ceph-mon[97675]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-13T03:23:32.671 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:32 smithi064 ceph-mon[97675]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-13T03:23:32.671 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:32 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:23:32.671 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:32 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:23:32.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:32 smithi142 ceph-mon[134082]: pgmap v322: 97 pgs: 97 active+clean; 103 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 926 KiB/s wr, 85 op/s 2023-09-13T03:23:32.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:32 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:23:32.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:32 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:23:32.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:32 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:23:32.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:32 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:23:32.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:32 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:23:32.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:32 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:23:32.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:32 smithi142 ceph-mon[134082]: Adjusting osd_memory_target on smithi142 to 196.2M 2023-09-13T03:23:32.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:32 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:23:32.747 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:32 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:23:32.748 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:32 smithi142 ceph-mon[134082]: Unable to set osd_memory_target on smithi142 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-09-13T03:23:32.748 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:32 smithi142 ceph-mon[134082]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-13T03:23:32.748 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:32 smithi142 ceph-mon[134082]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-13T03:23:32.748 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:32 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:23:32.748 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:32 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:23:32.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:32 smithi170 ceph-mon[134328]: pgmap v322: 97 pgs: 97 active+clean; 103 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 926 KiB/s wr, 85 op/s 2023-09-13T03:23:32.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:32 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:23:32.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:32 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:23:32.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:32 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:23:32.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:32 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:23:32.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:32 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:23:32.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:32 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:23:32.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:32 smithi170 ceph-mon[134328]: Adjusting osd_memory_target on smithi142 to 196.2M 2023-09-13T03:23:32.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:32 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:23:32.913 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:32 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:23:32.913 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:32 smithi170 ceph-mon[134328]: Unable to set osd_memory_target on smithi142 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-09-13T03:23:32.913 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:32 smithi170 ceph-mon[134328]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-13T03:23:32.913 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:32 smithi170 ceph-mon[134328]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-13T03:23:32.913 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:32 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:23:32.914 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:32 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:23:32.945 INFO:tasks.workunit.client.0.smithi064.stdout: 1 224236 2.01 MB/sec execute 457 sec latency 386.617 ms 2023-09-13T03:23:33.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:33 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:23:33.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:33 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:23:33.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:33 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:23:33.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:33 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:23:33.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:33 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:23:33.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:33 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:23:33.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:33 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:23:33.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:33 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:23:33.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:33 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:23:33.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:33 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:23:33.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:33 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:23:33.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:33 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:23:33.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:33 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:23:33.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:33 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:23:33.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:33 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:23:33.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:33 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:23:33.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:33 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:23:33.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:33 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-13T03:23:33.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:33 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:23:33.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:33 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:23:33.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:33 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:23:33.945 INFO:tasks.workunit.client.0.smithi064.stdout: 1 224304 2.00 MB/sec execute 458 sec latency 972.870 ms 2023-09-13T03:23:34.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:34 smithi064 ceph-mon[97675]: Adjusting osd_memory_target on smithi064 to 220.9M 2023-09-13T03:23:34.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:34 smithi064 ceph-mon[97675]: Unable to set osd_memory_target on smithi064 to 231682662: error parsing value: Value '231682662' is below minimum 939524096 2023-09-13T03:23:34.670 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:34 smithi064 ceph-mon[97675]: pgmap v323: 97 pgs: 97 active+clean; 103 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 668 KiB/s wr, 71 op/s 2023-09-13T03:23:34.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:34 smithi142 ceph-mon[134082]: Adjusting osd_memory_target on smithi064 to 220.9M 2023-09-13T03:23:34.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:34 smithi142 ceph-mon[134082]: Unable to set osd_memory_target on smithi064 to 231682662: error parsing value: Value '231682662' is below minimum 939524096 2023-09-13T03:23:34.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:34 smithi142 ceph-mon[134082]: pgmap v323: 97 pgs: 97 active+clean; 103 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 668 KiB/s wr, 71 op/s 2023-09-13T03:23:34.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:34 smithi170 ceph-mon[134328]: Adjusting osd_memory_target on smithi064 to 220.9M 2023-09-13T03:23:34.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:34 smithi170 ceph-mon[134328]: Unable to set osd_memory_target on smithi064 to 231682662: error parsing value: Value '231682662' is below minimum 939524096 2023-09-13T03:23:34.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:34 smithi170 ceph-mon[134328]: pgmap v323: 97 pgs: 97 active+clean; 103 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 668 KiB/s wr, 71 op/s 2023-09-13T03:23:34.945 INFO:tasks.workunit.client.0.smithi064.stdout: 1 224304 2.00 MB/sec execute 459 sec latency 1367.406 ms 2023-09-13T03:23:35.945 INFO:tasks.workunit.client.0.smithi064.stdout: 1 224304 2.00 MB/sec execute 460 sec latency 2367.542 ms 2023-09-13T03:23:36.945 INFO:tasks.workunit.client.0.smithi064.stdout: 1 224304 1.99 MB/sec execute 461 sec latency 3367.677 ms 2023-09-13T03:23:36.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:36 smithi142 ceph-mon[134082]: pgmap v324: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 935 KiB/s wr, 109 op/s 2023-09-13T03:23:37.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:36 smithi170 ceph-mon[134328]: pgmap v324: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 935 KiB/s wr, 109 op/s 2023-09-13T03:23:37.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:36 smithi064 ceph-mon[97675]: pgmap v324: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 935 KiB/s wr, 109 op/s 2023-09-13T03:23:37.945 INFO:tasks.workunit.client.0.smithi064.stdout: 1 224304 1.99 MB/sec execute 462 sec latency 4367.830 ms 2023-09-13T03:23:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.529+0000 7f67703ff700 1 -- 172.21.15.64:0/2307495943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6768106650 msgr2=0x7f6768106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:23:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.529+0000 7f67703ff700 1 --2- 172.21.15.64:0/2307495943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6768106650 0x7f6768106a30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f67640099c0 tx=0x7f6764009cd0 comp rx=0 tx=0).stop 2023-09-13T03:23:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.529+0000 7f67703ff700 1 -- 172.21.15.64:0/2307495943 shutdown_connections 2023-09-13T03:23:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.529+0000 7f67703ff700 1 --2- 172.21.15.64:0/2307495943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6768106650 0x7f6768106a30 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.529+0000 7f67703ff700 1 --2- 172.21.15.64:0/2307495943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67680683b0 0x7f676810d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.529+0000 7f67703ff700 1 --2- 172.21.15.64:0/2307495943 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67680679f0 0x7f6768067e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.529+0000 7f67703ff700 1 -- 172.21.15.64:0/2307495943 >> 172.21.15.64:0/2307495943 conn(0x7f6768074f30 msgr2=0x7f6768075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:23:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.529+0000 7f67703ff700 1 -- 172.21.15.64:0/2307495943 shutdown_connections 2023-09-13T03:23:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.529+0000 7f67703ff700 1 -- 172.21.15.64:0/2307495943 wait complete. 2023-09-13T03:23:38.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.530+0000 7f67703ff700 1 Processor -- start 2023-09-13T03:23:38.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.531+0000 7f67703ff700 1 -- start start 2023-09-13T03:23:38.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.531+0000 7f67703ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67680679f0 0x7f676819e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:38.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.531+0000 7f67703ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67680683b0 0x7f676819eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:38.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.531+0000 7f67703ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6768106650 0x7f6768198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:38.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.531+0000 7f67703ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6768198d90 con 0x7f67680683b0 2023-09-13T03:23:38.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.531+0000 7f67703ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6768198f00 con 0x7f67680679f0 2023-09-13T03:23:38.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.531+0000 7f67703ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67681990c0 con 0x7f6768106650 2023-09-13T03:23:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.531+0000 7f676e19b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67680679f0 0x7f676819e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.531+0000 7f676e19b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67680679f0 0x7f676819e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:38854/0 (socket says 172.21.15.64:38854) 2023-09-13T03:23:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.531+0000 7f676e19b700 1 -- 172.21.15.64:0/397305468 learned_addr learned my addr 172.21.15.64:0/397305468 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:23:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.531+0000 7f676e99c700 1 --2- 172.21.15.64:0/397305468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6768106650 0x7f6768198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.532+0000 7f676d99a700 1 --2- 172.21.15.64:0/397305468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67680683b0 0x7f676819eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.532+0000 7f676e19b700 1 -- 172.21.15.64:0/397305468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6768106650 msgr2=0x7f6768198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:23:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.532+0000 7f676e19b700 1 --2- 172.21.15.64:0/397305468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6768106650 0x7f6768198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.532+0000 7f676e19b700 1 -- 172.21.15.64:0/397305468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67680683b0 msgr2=0x7f676819eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:23:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.532+0000 7f676e19b700 1 --2- 172.21.15.64:0/397305468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67680683b0 0x7f676819eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.532+0000 7f676e19b700 1 -- 172.21.15.64:0/397305468 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6758009520 con 0x7f67680679f0 2023-09-13T03:23:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.532+0000 7f676e19b700 1 --2- 172.21.15.64:0/397305468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67680679f0 0x7f676819e860 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6764009990 tx=0x7f676400f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:23:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.532+0000 7f675f7fe700 1 -- 172.21.15.64:0/397305468 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676401d070 con 0x7f67680679f0 2023-09-13T03:23:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.532+0000 7f675f7fe700 1 -- 172.21.15.64:0/397305468 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f676400fe90 con 0x7f67680679f0 2023-09-13T03:23:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.532+0000 7f675f7fe700 1 -- 172.21.15.64:0/397305468 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6764021e70 con 0x7f67680679f0 2023-09-13T03:23:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.532+0000 7f67703ff700 1 -- 172.21.15.64:0/397305468 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67640095f0 con 0x7f67680679f0 2023-09-13T03:23:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.533+0000 7f67703ff700 1 -- 172.21.15.64:0/397305468 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6768199580 con 0x7f67680679f0 2023-09-13T03:23:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.538+0000 7f675f7fe700 1 -- 172.21.15.64:0/397305468 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f67640215a0 con 0x7f67680679f0 2023-09-13T03:23:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.538+0000 7f6783cb0b80 1 Processor -- start 2023-09-13T03:23:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.539+0000 7f6783cb0b80 1 -- start start 2023-09-13T03:23:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.539+0000 7f6783cb0b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56437ebdd1a0 0x56437ebdd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.539+0000 7f6783cb0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56437ebd7300 0x56437ebd7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.539+0000 7f6783cb0b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56437ebd7cc0 0x56437ebe3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.539+0000 7f6783cb0b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56437eb306d0 con 0x56437ebd7cc0 2023-09-13T03:23:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.539+0000 7f6783cb0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56437eb2dff0 con 0x56437ebd7300 2023-09-13T03:23:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.539+0000 7f6783cb0b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56437eb298e0 con 0x56437ebdd1a0 2023-09-13T03:23:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.539+0000 7f676e99c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56437ebdd1a0 0x56437ebdd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.540+0000 7f676e99c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56437ebdd1a0 0x56437ebdd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:45766/0 (socket says 172.21.15.64:45766) 2023-09-13T03:23:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.540+0000 7f676e99c700 1 -- 172.21.15.64:0/1511023823 learned_addr learned my addr 172.21.15.64:0/1511023823 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:23:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.540+0000 7f676d99a700 1 --2- 172.21.15.64:0/1511023823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56437ebd7cc0 0x56437ebe3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.540+0000 7f676e19b700 1 --2- 172.21.15.64:0/1511023823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56437ebd7300 0x56437ebd7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.540+0000 7f676e99c700 1 -- 172.21.15.64:0/1511023823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56437ebd7300 msgr2=0x56437ebd7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:23:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.540+0000 7f676e99c700 1 --2- 172.21.15.64:0/1511023823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56437ebd7300 0x56437ebd7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.540+0000 7f676e99c700 1 -- 172.21.15.64:0/1511023823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56437ebd7cc0 msgr2=0x56437ebe3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:23:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.540+0000 7f676e99c700 1 --2- 172.21.15.64:0/1511023823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56437ebd7cc0 0x56437ebe3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.540+0000 7f676e99c700 1 -- 172.21.15.64:0/1511023823 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56437e955d80 con 0x56437ebdd1a0 2023-09-13T03:23:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.540+0000 7f676e99c700 1 --2- 172.21.15.64:0/1511023823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56437ebdd1a0 0x56437ebdd580 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6760011680 tx=0x7f6760009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=169d0772e4345917 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:23:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.541+0000 7f675f7fe700 1 --2- 172.21.15.64:0/397305468 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f67540a0870 0x7f67540a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.541+0000 7f675cbfe700 1 -- 172.21.15.64:0/1511023823 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676000e040 con 0x56437ebdd1a0 2023-09-13T03:23:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.541+0000 7f675cbfe700 1 -- 172.21.15.64:0/1511023823 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f676000f040 con 0x56437ebdd1a0 2023-09-13T03:23:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.541+0000 7f675cbfe700 1 -- 172.21.15.64:0/1511023823 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67600048c0 con 0x56437ebdd1a0 2023-09-13T03:23:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.541+0000 7f675f7fe700 1 -- 172.21.15.64:0/397305468 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f67640ccb20 con 0x7f67680679f0 2023-09-13T03:23:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.541+0000 7f676e19b700 1 --2- 172.21.15.64:0/397305468 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f67540a0870 0x7f67540a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:38.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.541+0000 7f6783cb0b80 1 -- 172.21.15.64:0/1511023823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56437ebdd1a0 msgr2=0x56437ebdd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:23:38.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.541+0000 7f6783cb0b80 1 --2- 172.21.15.64:0/1511023823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56437ebdd1a0 0x56437ebdd580 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6760011680 tx=0x7f6760009920 comp rx=0 tx=0).stop 2023-09-13T03:23:38.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.541+0000 7f676e19b700 1 --2- 172.21.15.64:0/397305468 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f67540a0870 0x7f67540a2d30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x56437ebd2060 tx=0x7f67640d0590 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:23:38.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.542+0000 7f6783cb0b80 1 -- 172.21.15.64:0/1511023823 shutdown_connections 2023-09-13T03:23:38.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.542+0000 7f6783cb0b80 1 --2- 172.21.15.64:0/1511023823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56437ebd7cc0 0x56437ebe3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:38.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.542+0000 7f6783cb0b80 1 --2- 172.21.15.64:0/1511023823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56437ebdd1a0 0x56437ebdd580 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:38.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.542+0000 7f6783cb0b80 1 --2- 172.21.15.64:0/1511023823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56437ebd7300 0x56437ebd7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:38.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.542+0000 7f6783cb0b80 1 -- 172.21.15.64:0/1511023823 >> 172.21.15.64:0/1511023823 conn(0x56437eb4c030 msgr2=0x56437ebd2560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:23:38.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.542+0000 7f6783cb0b80 1 -- 172.21.15.64:0/1511023823 shutdown_connections 2023-09-13T03:23:38.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.542+0000 7f6783cb0b80 1 -- 172.21.15.64:0/1511023823 wait complete. 2023-09-13T03:23:38.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.542+0000 7f6783cb0b80 1 -- start start 2023-09-13T03:23:38.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.543+0000 7f6783cb0b80 1 Processor -- start 2023-09-13T03:23:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.543+0000 7f6783cb0b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56437ebd7300 0x56437ebe3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.543+0000 7f6783cb0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56437ebd7cc0 0x56437ebe1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.543+0000 7f6783cb0b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56437ebdd1a0 0x56437ebe15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.543+0000 7f6783cb0b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56437eac7b10 con 0x56437ebd7300 2023-09-13T03:23:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.543+0000 7f6783cb0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56437eac7810 con 0x56437ebd7cc0 2023-09-13T03:23:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.543+0000 7f6783cb0b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56437e9ab8c0 con 0x56437ebdd1a0 2023-09-13T03:23:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.543+0000 7f6783cb0b80 10 client.0 _ll_register_callbacks cb 0x56437ea11a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:23:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.544+0000 7f676d99a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56437ebd7cc0 0x56437ebe1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.544+0000 7f676d99a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56437ebd7cc0 0x56437ebe1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:38876/0 (socket says 172.21.15.64:38876) 2023-09-13T03:23:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.544+0000 7f676d99a700 1 -- 172.21.15.64:0/1108127747 learned_addr learned my addr 172.21.15.64:0/1108127747 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:23:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.544+0000 7f676e99c700 1 --2- 172.21.15.64:0/1108127747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56437ebd7300 0x56437ebe3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.544+0000 7f676e99c700 1 --2- 172.21.15.64:0/1108127747 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56437ebdd1a0 0x56437ebe15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.544+0000 7f676d99a700 1 -- 172.21.15.64:0/1108127747 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56437ebdd1a0 msgr2=0x56437ebe15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:23:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.544+0000 7f676d99a700 1 --2- 172.21.15.64:0/1108127747 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56437ebdd1a0 0x56437ebe15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.544+0000 7f676d99a700 1 -- 172.21.15.64:0/1108127747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56437ebd7300 msgr2=0x56437ebe3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:23:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.544+0000 7f676d99a700 1 --2- 172.21.15.64:0/1108127747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56437ebd7300 0x56437ebe3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.544+0000 7f676d99a700 1 -- 172.21.15.64:0/1108127747 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6760011330 con 0x56437ebd7cc0 2023-09-13T03:23:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.544+0000 7f676e99c700 1 --2- 172.21.15.64:0/1108127747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56437ebd7300 0x56437ebe3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:23:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.545+0000 7f676d99a700 1 --2- 172.21.15.64:0/1108127747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56437ebd7cc0 0x56437ebe1000 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6758011950 tx=0x7f6758011c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a531ba923302557 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:23:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.545+0000 7f6751ffb700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:23:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.545+0000 7f6751ffb700 1 -- 172.21.15.64:0/1108127747 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6758016070 con 0x56437ebd7cc0 2023-09-13T03:23:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.545+0000 7f6751ffb700 1 -- 172.21.15.64:0/1108127747 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f675800f850 con 0x56437ebd7cc0 2023-09-13T03:23:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.545+0000 7f6783cb0b80 1 -- 172.21.15.64:0/1108127747 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56437ea81490 con 0x56437ebd7cc0 2023-09-13T03:23:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.546+0000 7f6751ffb700 1 -- 172.21.15.64:0/1108127747 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f675800fa10 con 0x56437ebd7cc0 2023-09-13T03:23:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.546+0000 7f6783cb0b80 10 client.26966 fetch_fsmap learned FSMap version 61 2023-09-13T03:23:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.546+0000 7f6783cb0b80 1 -- 172.21.15.64:0/1108127747 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x56437e93ec00 con 0x56437ebd7cc0 2023-09-13T03:23:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.546+0000 7f6751ffb700 1 -- 172.21.15.64:0/1108127747 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56437ea81490 con 0x56437ebd7cc0 2023-09-13T03:23:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.546+0000 7f6751ffb700 1 -- 172.21.15.64:0/1108127747 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 61) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f675801a3f0 con 0x56437ebd7cc0 2023-09-13T03:23:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.546+0000 7f6783cb0b80 10 client.26966 fetch_fsmap finished waiting for FSMap version 61 2023-09-13T03:23:38.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.547+0000 7f6783cb0b80 10 client.26966 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:23:38.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.547+0000 7f6783cb0b80 20 client.26966 populate_metadata read hostname 'smithi064' 2023-09-13T03:23:38.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.547+0000 7f6783cb0b80 1 --2- 172.21.15.64:0/1108127747 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56437ebf1480 0x56437ebf38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:38.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.547+0000 7f6783cb0b80 4 client.26966 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:23:38.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.547+0000 7f6783cb0b80 1 -- 172.21.15.64:0/1108127747 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56437eb3e840 con 0x56437ebf1480 2023-09-13T03:23:38.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.547+0000 7f676e99c700 1 --2- 172.21.15.64:0/1108127747 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56437ebf1480 0x56437ebf38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:38.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.548+0000 7f676e99c700 1 --2- 172.21.15.64:0/1108127747 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56437ebf1480 0x56437ebf38a0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71c21b5db61e7e75 server_cookie=c1e8f476c272a525 in_seq=0 out_seq=0 2023-09-13T03:23:38.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.548+0000 7f6751ffb700 10 client.26966 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:23:38.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.548+0000 7f6751ffb700 1 -- 172.21.15.64:0/1108127747 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56437eb3e840 con 0x56437ebf1480 2023-09-13T03:23:38.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.548+0000 7f6751ffb700 10 client.26966 handle_command_reply: tid=0 2023-09-13T03:23:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.548+0000 7f6783cb0b80 1 client.26966 shutdown 2023-09-13T03:23:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.549+0000 7f6783cb0b80 1 -- 172.21.15.64:0/1108127747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56437ebd7cc0 msgr2=0x56437ebe1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:23:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.549+0000 7f6783cb0b80 1 --2- 172.21.15.64:0/1108127747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56437ebd7cc0 0x56437ebe1000 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6758011950 tx=0x7f6758011c60 comp rx=0 tx=0).stop 2023-09-13T03:23:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.549+0000 7f6783cb0b80 1 -- 172.21.15.64:0/1108127747 shutdown_connections 2023-09-13T03:23:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.549+0000 7f6783cb0b80 1 --2- 172.21.15.64:0/1108127747 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56437ebf1480 0x56437ebf38a0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:38.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.549+0000 7f6783cb0b80 1 --2- 172.21.15.64:0/1108127747 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56437ebdd1a0 0x56437ebe15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:38.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.549+0000 7f6783cb0b80 1 --2- 172.21.15.64:0/1108127747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56437ebd7300 0x56437ebe3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:38.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.549+0000 7f6783cb0b80 1 --2- 172.21.15.64:0/1108127747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56437ebd7cc0 0x56437ebe1000 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:38.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.549+0000 7f6783cb0b80 1 -- 172.21.15.64:0/1108127747 >> 172.21.15.64:0/1108127747 conn(0x56437eb4c030 msgr2=0x56437ebd17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:23:38.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.549+0000 7f6751ffb700 0 client.26966 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:23:38.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.549+0000 7f6783cb0b80 1 -- 172.21.15.64:0/1108127747 shutdown_connections 2023-09-13T03:23:38.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.549+0000 7f6783cb0b80 1 -- 172.21.15.64:0/1108127747 wait complete. 2023-09-13T03:23:38.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.550+0000 7f6783cb0b80 20 client.26966 trim_cache size 0 max 16384 2023-09-13T03:23:38.567 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.567+0000 7f6783cb0b80 1 Processor -- start 2023-09-13T03:23:38.568 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.567+0000 7f6783cb0b80 1 -- start start 2023-09-13T03:23:38.568 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.567+0000 7f6783cb0b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56437ebd7300 0x56437ebe8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:38.568 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.567+0000 7f6783cb0b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56437ebd7cc0 0x56437ebe3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:38.568 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.567+0000 7f6783cb0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56437ebdd1a0 0x56437ebdbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:38.568 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.567+0000 7f6783cb0b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56437ebeea30 con 0x56437ebd7300 2023-09-13T03:23:38.569 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.567+0000 7f6783cb0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56437ebe8a60 con 0x56437ebdd1a0 2023-09-13T03:23:38.569 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.567+0000 7f6783cb0b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56437ebdf5b0 con 0x56437ebd7cc0 2023-09-13T03:23:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.568+0000 7f676d99a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56437ebd7cc0 0x56437ebe3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.568+0000 7f676d99a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56437ebd7cc0 0x56437ebe3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:45782/0 (socket says 172.21.15.64:45782) 2023-09-13T03:23:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.568+0000 7f676d99a700 1 -- 172.21.15.64:0/2133595280 learned_addr learned my addr 172.21.15.64:0/2133595280 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:23:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.568+0000 7f676e99c700 1 --2- 172.21.15.64:0/2133595280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56437ebd7300 0x56437ebe8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:38.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.568+0000 7f676e99c700 1 --2- 172.21.15.64:0/2133595280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56437ebdd1a0 0x56437ebdbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:38.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.568+0000 7f676d99a700 1 -- 172.21.15.64:0/2133595280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56437ebdd1a0 msgr2=0x56437ebdbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:23:38.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.568+0000 7f676d99a700 1 --2- 172.21.15.64:0/2133595280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56437ebdd1a0 0x56437ebdbff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:38.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.568+0000 7f676d99a700 1 -- 172.21.15.64:0/2133595280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56437ebd7300 msgr2=0x56437ebe8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:23:38.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.568+0000 7f676d99a700 1 --2- 172.21.15.64:0/2133595280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56437ebd7300 0x56437ebe8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:38.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.568+0000 7f676d99a700 1 -- 172.21.15.64:0/2133595280 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56437e97c1f0 con 0x56437ebd7cc0 2023-09-13T03:23:38.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.568+0000 7f676d99a700 1 --2- 172.21.15.64:0/2133595280 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56437ebd7cc0 0x56437ebe3920 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f675801a8a0 tx=0x7f675801bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a5762dfb4dc45f5 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:23:38.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.568+0000 7f675d3ff700 1 -- 172.21.15.64:0/2133595280 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6758016070 con 0x56437ebd7cc0 2023-09-13T03:23:38.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.569+0000 7f675d3ff700 1 -- 172.21.15.64:0/2133595280 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f675801a3f0 con 0x56437ebd7cc0 2023-09-13T03:23:38.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.569+0000 7f675d3ff700 1 -- 172.21.15.64:0/2133595280 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67580050b0 con 0x56437ebd7cc0 2023-09-13T03:23:38.574 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.569+0000 7f6783cb0b80 1 -- 172.21.15.64:0/2133595280 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56437ebd7cc0 msgr2=0x56437ebe3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:23:38.574 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.569+0000 7f6783cb0b80 1 --2- 172.21.15.64:0/2133595280 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56437ebd7cc0 0x56437ebe3920 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f675801a8a0 tx=0x7f675801bd80 comp rx=0 tx=0).stop 2023-09-13T03:23:38.574 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.569+0000 7f6783cb0b80 1 -- 172.21.15.64:0/2133595280 shutdown_connections 2023-09-13T03:23:38.574 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.569+0000 7f6783cb0b80 1 --2- 172.21.15.64:0/2133595280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56437ebdd1a0 0x56437ebdbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:38.574 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.569+0000 7f6783cb0b80 1 --2- 172.21.15.64:0/2133595280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56437ebd7300 0x56437ebe8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:38.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.569+0000 7f6783cb0b80 1 --2- 172.21.15.64:0/2133595280 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56437ebd7cc0 0x56437ebe3920 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:38.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.569+0000 7f6783cb0b80 1 -- 172.21.15.64:0/2133595280 >> 172.21.15.64:0/2133595280 conn(0x56437eb4c030 msgr2=0x56437ebd2090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:23:38.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.569+0000 7f6783cb0b80 1 -- 172.21.15.64:0/2133595280 shutdown_connections 2023-09-13T03:23:38.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.569+0000 7f6783cb0b80 1 -- 172.21.15.64:0/2133595280 wait complete. 2023-09-13T03:23:38.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.570+0000 7f6783cb0b80 1 -- start start 2023-09-13T03:23:38.576 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.570+0000 7f6783cb0b80 1 Processor -- start 2023-09-13T03:23:38.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.571+0000 7f6783cb0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56437ebd7300 0x56437ebe6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:38.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.571+0000 7f6783cb0b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56437ebd7cc0 0x56437ebe6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:38.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.571+0000 7f6783cb0b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56437ebdd1a0 0x56437ebe6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:38.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.571+0000 7f6783cb0b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56437ec44300 con 0x56437ebd7cc0 2023-09-13T03:23:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.571+0000 7f6783cb0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56437ec44470 con 0x56437ebd7300 2023-09-13T03:23:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.571+0000 7f6783cb0b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56437ec44630 con 0x56437ebdd1a0 2023-09-13T03:23:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.571+0000 7f6783cb0b80 10 client.0 _ll_register_callbacks cb 0x7f6758011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:23:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.571+0000 7f676d99a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56437ebd7cc0 0x56437ebe6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.571+0000 7f676e99c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56437ebdd1a0 0x56437ebe6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:38.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.571+0000 7f676e99c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56437ebdd1a0 0x56437ebe6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:45798/0 (socket says 172.21.15.64:45798) 2023-09-13T03:23:38.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.571+0000 7f676e99c700 1 -- 172.21.15.64:0/906557888 learned_addr learned my addr 172.21.15.64:0/906557888 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:23:38.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.571+0000 7f676e99c700 1 --2- 172.21.15.64:0/906557888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56437ebd7300 0x56437ebe6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:38.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.572+0000 7f676d99a700 1 -- 172.21.15.64:0/906557888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56437ebdd1a0 msgr2=0x56437ebe6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:23:38.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.572+0000 7f676d99a700 1 --2- 172.21.15.64:0/906557888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56437ebdd1a0 0x56437ebe6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:38.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.572+0000 7f676d99a700 1 -- 172.21.15.64:0/906557888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56437ebd7300 msgr2=0x56437ebe6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:23:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.572+0000 7f676d99a700 1 --2- 172.21.15.64:0/906557888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56437ebd7300 0x56437ebe6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.572+0000 7f676d99a700 1 -- 172.21.15.64:0/906557888 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56437e93ec00 con 0x56437ebd7cc0 2023-09-13T03:23:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.572+0000 7f676e99c700 1 --2- 172.21.15.64:0/906557888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56437ebdd1a0 0x56437ebe6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:23:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.572+0000 7f676d99a700 1 --2- 172.21.15.64:0/906557888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56437ebd7cc0 0x56437ebe6a50 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x56437eaf18a0 tx=0x7f675800fb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca095ff0b2d12cbf server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:23:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.572+0000 7f6751ffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:23:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.572+0000 7f6751ffb700 1 -- 172.21.15.64:0/906557888 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6758016070 con 0x56437ebd7cc0 2023-09-13T03:23:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.572+0000 7f6783cb0b80 1 -- 172.21.15.64:0/906557888 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f676000f040 con 0x56437ebd7cc0 2023-09-13T03:23:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.572+0000 7f6751ffb700 1 -- 172.21.15.64:0/906557888 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f67580050b0 con 0x56437ebd7cc0 2023-09-13T03:23:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.573+0000 7f6751ffb700 1 -- 172.21.15.64:0/906557888 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f675801a560 con 0x56437ebd7cc0 2023-09-13T03:23:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.573+0000 7f6783cb0b80 10 client.16932 fetch_fsmap learned FSMap version 61 2023-09-13T03:23:38.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.573+0000 7f6783cb0b80 1 -- 172.21.15.64:0/906557888 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x56437e954250 con 0x56437ebd7cc0 2023-09-13T03:23:38.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.573+0000 7f6751ffb700 1 -- 172.21.15.64:0/906557888 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6758044020 con 0x56437ebd7cc0 2023-09-13T03:23:38.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.573+0000 7f6751ffb700 1 -- 172.21.15.64:0/906557888 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 61) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f6758003920 con 0x56437ebd7cc0 2023-09-13T03:23:38.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.573+0000 7f6783cb0b80 10 client.16932 fetch_fsmap finished waiting for FSMap version 61 2023-09-13T03:23:38.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.573+0000 7f6783cb0b80 10 client.16932 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:23:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.573+0000 7f6783cb0b80 20 client.16932 populate_metadata read hostname 'smithi064' 2023-09-13T03:23:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.573+0000 7f6783cb0b80 1 --2- 172.21.15.64:0/906557888 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56437ec45a60 0x56437ec47e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.573+0000 7f6783cb0b80 4 client.16932 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:23:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.573+0000 7f6783cb0b80 1 -- 172.21.15.64:0/906557888 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56437eb228c0 con 0x56437ec45a60 2023-09-13T03:23:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.574+0000 7f676e99c700 1 --2- 172.21.15.64:0/906557888 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56437ec45a60 0x56437ec47e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.574+0000 7f676e99c700 1 --2- 172.21.15.64:0/906557888 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56437ec45a60 0x56437ec47e50 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2be21e5b64e64943 server_cookie=987aef42fbfb9f5 in_seq=0 out_seq=0 2023-09-13T03:23:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.574+0000 7f6751ffb700 10 client.16932 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:23:38.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.600+0000 7f6751ffb700 1 -- 172.21.15.64:0/906557888 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56437eb228c0 con 0x56437ec45a60 2023-09-13T03:23:38.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.600+0000 7f6751ffb700 10 client.16932 handle_command_reply: tid=0 2023-09-13T03:23:38.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.600+0000 7f6783cb0b80 1 client.16932 shutdown 2023-09-13T03:23:38.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.601+0000 7f6783cb0b80 1 -- 172.21.15.64:0/906557888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56437ebd7cc0 msgr2=0x56437ebe6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:23:38.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.601+0000 7f6783cb0b80 1 --2- 172.21.15.64:0/906557888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56437ebd7cc0 0x56437ebe6a50 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x56437eaf18a0 tx=0x7f675800fb10 comp rx=0 tx=0).stop 2023-09-13T03:23:38.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.601+0000 7f6783cb0b80 1 -- 172.21.15.64:0/906557888 shutdown_connections 2023-09-13T03:23:38.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.601+0000 7f6783cb0b80 1 --2- 172.21.15.64:0/906557888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56437ebd7300 0x56437ebe6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:38.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.601+0000 7f6783cb0b80 1 --2- 172.21.15.64:0/906557888 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x56437ec45a60 0x56437ec47e50 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:38.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.601+0000 7f6783cb0b80 1 --2- 172.21.15.64:0/906557888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56437ebdd1a0 0x56437ebe6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:38.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.601+0000 7f6783cb0b80 1 --2- 172.21.15.64:0/906557888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56437ebd7cc0 0x56437ebe6a50 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.601+0000 7f6783cb0b80 1 -- 172.21.15.64:0/906557888 >> 172.21.15.64:0/906557888 conn(0x56437eb4c030 msgr2=0x56437ebdbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:23:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.601+0000 7f6751ffb700 0 client.16932 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:23:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.601+0000 7f6783cb0b80 1 -- 172.21.15.64:0/906557888 shutdown_connections 2023-09-13T03:23:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.601+0000 7f6783cb0b80 1 -- 172.21.15.64:0/906557888 wait complete. 2023-09-13T03:23:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.601+0000 7f6783cb0b80 20 client.16932 trim_cache size 0 max 16384 2023-09-13T03:23:38.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:23:38.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:23:38.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:23:38.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:23:38.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.602+0000 7f67703ff700 1 -- 172.21.15.64:0/397305468 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f67540a0870 msgr2=0x7f67540a2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:23:38.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.602+0000 7f67703ff700 1 --2- 172.21.15.64:0/397305468 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f67540a0870 0x7f67540a2d30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x56437ebd2060 tx=0x7f67640d0590 comp rx=0 tx=0).stop 2023-09-13T03:23:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.602+0000 7f67703ff700 1 -- 172.21.15.64:0/397305468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67680679f0 msgr2=0x7f676819e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:23:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.602+0000 7f67703ff700 1 --2- 172.21.15.64:0/397305468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67680679f0 0x7f676819e860 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6764009990 tx=0x7f676400f690 comp rx=0 tx=0).stop 2023-09-13T03:23:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.602+0000 7f67703ff700 1 -- 172.21.15.64:0/397305468 shutdown_connections 2023-09-13T03:23:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.602+0000 7f67703ff700 1 --2- 172.21.15.64:0/397305468 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f67540a0870 0x7f67540a2d30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.602+0000 7f67703ff700 1 --2- 172.21.15.64:0/397305468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67680679f0 0x7f676819e860 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.602+0000 7f67703ff700 1 --2- 172.21.15.64:0/397305468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6768106650 0x7f6768198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.602+0000 7f67703ff700 1 --2- 172.21.15.64:0/397305468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67680683b0 0x7f676819eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.602+0000 7f67703ff700 1 -- 172.21.15.64:0/397305468 >> 172.21.15.64:0/397305468 conn(0x7f6768074f30 msgr2=0x7f67680fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:23:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.602+0000 7f67703ff700 1 -- 172.21.15.64:0/397305468 shutdown_connections 2023-09-13T03:23:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:38.602+0000 7f67703ff700 1 -- 172.21.15.64:0/397305468 wait complete. 2023-09-13T03:23:38.946 INFO:tasks.workunit.client.0.smithi064.stdout: 1 224965 1.99 MB/sec execute 463 sec latency 4952.726 ms 2023-09-13T03:23:38.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:38 smithi142 ceph-mon[134082]: pgmap v325: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 543 KiB/s wr, 80 op/s 2023-09-13T03:23:39.159 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:38 smithi170 ceph-mon[134328]: pgmap v325: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 543 KiB/s wr, 80 op/s 2023-09-13T03:23:39.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:38 smithi064 ceph-mon[97675]: pgmap v325: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 543 KiB/s wr, 80 op/s 2023-09-13T03:23:39.946 INFO:tasks.workunit.client.0.smithi064.stdout: 1 225205 1.99 MB/sec execute 464 sec latency 809.834 ms 2023-09-13T03:23:40.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:40 smithi170 ceph-mon[134328]: pgmap v326: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 543 KiB/s wr, 80 op/s 2023-09-13T03:23:40.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:40 smithi064 ceph-mon[97675]: pgmap v326: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 543 KiB/s wr, 80 op/s 2023-09-13T03:23:40.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:40 smithi142 ceph-mon[134082]: pgmap v326: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 543 KiB/s wr, 80 op/s 2023-09-13T03:23:40.946 INFO:tasks.workunit.client.0.smithi064.stdout: 1 225604 1.99 MB/sec execute 465 sec latency 1650.436 ms 2023-09-13T03:23:41.946 INFO:tasks.workunit.client.0.smithi064.stdout: 1 227359 2.01 MB/sec execute 466 sec latency 12.826 ms 2023-09-13T03:23:42.946 INFO:tasks.workunit.client.0.smithi064.stdout: 1 228575 2.02 MB/sec execute 467 sec latency 37.050 ms 2023-09-13T03:23:42.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:42 smithi142 ceph-mon[134082]: pgmap v327: 97 pgs: 97 active+clean; 108 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 1.1 MiB/s wr, 116 op/s 2023-09-13T03:23:43.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:42 smithi170 ceph-mon[134328]: pgmap v327: 97 pgs: 97 active+clean; 108 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 1.1 MiB/s wr, 116 op/s 2023-09-13T03:23:43.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:42 smithi064 ceph-mon[97675]: pgmap v327: 97 pgs: 97 active+clean; 108 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 1.1 MiB/s wr, 116 op/s 2023-09-13T03:23:43.946 INFO:tasks.workunit.client.0.smithi064.stdout: 1 228748 2.01 MB/sec execute 468 sec latency 873.165 ms 2023-09-13T03:23:44.946 INFO:tasks.workunit.client.0.smithi064.stdout: 1 228751 2.01 MB/sec execute 469 sec latency 1855.435 ms 2023-09-13T03:23:44.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:44 smithi142 ceph-mon[134082]: pgmap v328: 97 pgs: 97 active+clean; 108 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 868 KiB/s wr, 74 op/s 2023-09-13T03:23:45.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:44 smithi170 ceph-mon[134328]: pgmap v328: 97 pgs: 97 active+clean; 108 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 868 KiB/s wr, 74 op/s 2023-09-13T03:23:45.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:44 smithi064 ceph-mon[97675]: pgmap v328: 97 pgs: 97 active+clean; 108 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 868 KiB/s wr, 74 op/s 2023-09-13T03:23:45.946 INFO:tasks.workunit.client.0.smithi064.stdout: 1 230760 2.03 MB/sec execute 470 sec latency 21.176 ms 2023-09-13T03:23:46.946 INFO:tasks.workunit.client.0.smithi064.stdout: 1 231856 2.03 MB/sec execute 471 sec latency 18.686 ms 2023-09-13T03:23:46.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:46 smithi142 ceph-mon[134082]: pgmap v329: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2023-09-13T03:23:47.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:46 smithi170 ceph-mon[134328]: pgmap v329: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2023-09-13T03:23:47.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:46 smithi064 ceph-mon[97675]: pgmap v329: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2023-09-13T03:23:47.946 INFO:tasks.workunit.client.0.smithi064.stdout: 1 232296 2.04 MB/sec execute 472 sec latency 709.093 ms 2023-09-13T03:23:48.946 INFO:tasks.workunit.client.0.smithi064.stdout: 1 232296 2.03 MB/sec execute 473 sec latency 1709.142 ms 2023-09-13T03:23:48.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:48 smithi142 ceph-mon[134082]: pgmap v330: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 1.1 MiB/s wr, 100 op/s 2023-09-13T03:23:49.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:48 smithi170 ceph-mon[134328]: pgmap v330: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 1.1 MiB/s wr, 100 op/s 2023-09-13T03:23:49.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:48 smithi064 ceph-mon[97675]: pgmap v330: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 1.1 MiB/s wr, 100 op/s 2023-09-13T03:23:49.946 INFO:tasks.workunit.client.0.smithi064.stdout: 1 232296 2.03 MB/sec execute 474 sec latency 2709.242 ms 2023-09-13T03:23:50.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:50 smithi064 ceph-mon[97675]: pgmap v331: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 1.1 MiB/s wr, 100 op/s 2023-09-13T03:23:50.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:50 smithi142 ceph-mon[134082]: pgmap v331: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 1.1 MiB/s wr, 100 op/s 2023-09-13T03:23:50.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:50 smithi170 ceph-mon[134328]: pgmap v331: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 1.1 MiB/s wr, 100 op/s 2023-09-13T03:23:50.947 INFO:tasks.workunit.client.0.smithi064.stdout: 1 232696 2.02 MB/sec execute 475 sec latency 3541.618 ms 2023-09-13T03:23:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.756+0000 7f3ba8bff700 1 -- 172.21.15.64:0/1190525661 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ba41005b0 msgr2=0x7f3ba4100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:23:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.756+0000 7f3ba8bff700 1 --2- 172.21.15.64:0/1190525661 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ba41005b0 0x7f3ba4100a30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c009990 tx=0x7f3b8c009ca0 comp rx=0 tx=0).stop 2023-09-13T03:23:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.756+0000 7f3ba8bff700 1 -- 172.21.15.64:0/1190525661 shutdown_connections 2023-09-13T03:23:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.756+0000 7f3ba8bff700 1 --2- 172.21.15.64:0/1190525661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ba4106670 0x7f3ba4106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.756+0000 7f3ba8bff700 1 --2- 172.21.15.64:0/1190525661 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ba4100f70 0x7f3ba410d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.756+0000 7f3ba8bff700 1 --2- 172.21.15.64:0/1190525661 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ba41005b0 0x7f3ba4100a30 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.756+0000 7f3ba8bff700 1 -- 172.21.15.64:0/1190525661 >> 172.21.15.64:0/1190525661 conn(0x7f3ba40fc060 msgr2=0x7f3ba40fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:23:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.756+0000 7f3ba8bff700 1 -- 172.21.15.64:0/1190525661 shutdown_connections 2023-09-13T03:23:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.756+0000 7f3ba8bff700 1 -- 172.21.15.64:0/1190525661 wait complete. 2023-09-13T03:23:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.757+0000 7f3ba8bff700 1 Processor -- start 2023-09-13T03:23:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.757+0000 7f3ba8bff700 1 -- start start 2023-09-13T03:23:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.758+0000 7f3ba8bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ba41005b0 0x7f3ba419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.758+0000 7f3ba8bff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ba4100f70 0x7f3ba419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.758+0000 7f3ba8bff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ba4106670 0x7f3ba4198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.758+0000 7f3ba8bff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba4198d00 con 0x7f3ba4106670 2023-09-13T03:23:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.758+0000 7f3ba8bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba4198e70 con 0x7f3ba41005b0 2023-09-13T03:23:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.758+0000 7f3ba8bff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba4199030 con 0x7f3ba4100f70 2023-09-13T03:23:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.758+0000 7f3ba259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ba41005b0 0x7f3ba419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.758+0000 7f3ba259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ba41005b0 0x7f3ba419e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:51320/0 (socket says 172.21.15.64:51320) 2023-09-13T03:23:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.758+0000 7f3b9bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ba4100f70 0x7f3ba419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.758+0000 7f3ba259c700 1 -- 172.21.15.64:0/2469926730 learned_addr learned my addr 172.21.15.64:0/2469926730 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:23:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.758+0000 7f3ba2d9d700 1 --2- 172.21.15.64:0/2469926730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ba4106670 0x7f3ba4198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.758+0000 7f3ba259c700 1 -- 172.21.15.64:0/2469926730 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ba4100f70 msgr2=0x7f3ba419ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:23:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.758+0000 7f3ba259c700 1 --2- 172.21.15.64:0/2469926730 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ba4100f70 0x7f3ba419ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.759+0000 7f3ba259c700 1 -- 172.21.15.64:0/2469926730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ba4106670 msgr2=0x7f3ba4198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:23:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.759+0000 7f3ba259c700 1 --2- 172.21.15.64:0/2469926730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ba4106670 0x7f3ba4198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.759+0000 7f3ba259c700 1 -- 172.21.15.64:0/2469926730 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b94009520 con 0x7f3ba41005b0 2023-09-13T03:23:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.759+0000 7f3ba259c700 1 --2- 172.21.15.64:0/2469926730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ba41005b0 0x7f3ba419e830 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3b9400eb30 tx=0x7f3b9400ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:23:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.759+0000 7f3b9b7fe700 1 -- 172.21.15.64:0/2469926730 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b94009880 con 0x7f3ba41005b0 2023-09-13T03:23:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.759+0000 7f3b9b7fe700 1 -- 172.21.15.64:0/2469926730 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b940044e0 con 0x7f3ba41005b0 2023-09-13T03:23:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.759+0000 7f3b9b7fe700 1 -- 172.21.15.64:0/2469926730 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b94003d80 con 0x7f3ba41005b0 2023-09-13T03:23:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.759+0000 7f3ba8bff700 1 -- 172.21.15.64:0/2469926730 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b8c0095f0 con 0x7f3ba41005b0 2023-09-13T03:23:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.759+0000 7f3ba8bff700 1 -- 172.21.15.64:0/2469926730 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba41a5640 con 0x7f3ba41005b0 2023-09-13T03:23:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.763+0000 7f3b9b7fe700 1 -- 172.21.15.64:0/2469926730 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f3b94003f20 con 0x7f3ba41005b0 2023-09-13T03:23:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.764+0000 7f3b9b7fe700 1 --2- 172.21.15.64:0/2469926730 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f3b900a07b0 0x7f3b900a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.764+0000 7f3b9b7fe700 1 -- 172.21.15.64:0/2469926730 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f3b94014070 con 0x7f3ba41005b0 2023-09-13T03:23:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.764+0000 7f3bbc55fb80 1 Processor -- start 2023-09-13T03:23:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.764+0000 7f3b9bfff700 1 --2- 172.21.15.64:0/2469926730 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f3b900a07b0 0x7f3b900a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.765+0000 7f3bbc55fb80 1 -- start start 2023-09-13T03:23:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.765+0000 7f3bbc55fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e14411a0 0x5582e1441580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.765+0000 7f3bbc55fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582e143b300 0x5582e143b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.765+0000 7f3bbc55fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e143bcc0 0x5582e1447f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.765+0000 7f3bbc55fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582e13946d0 con 0x5582e143bcc0 2023-09-13T03:23:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.765+0000 7f3bbc55fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582e1391ff0 con 0x5582e14411a0 2023-09-13T03:23:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.765+0000 7f3bbc55fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582e138d8e0 con 0x5582e143b300 2023-09-13T03:23:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.765+0000 7f3b9bfff700 1 --2- 172.21.15.64:0/2469926730 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f3b900a07b0 0x7f3b900a2c70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3ba40ffad0 tx=0x7f3b8c005fb0 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:23:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.765+0000 7f3ba259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e14411a0 0x5582e1441580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.765+0000 7f3ba259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e14411a0 0x5582e1441580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:51328/0 (socket says 172.21.15.64:51328) 2023-09-13T03:23:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.765+0000 7f3b9bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582e143b300 0x5582e143b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.765+0000 7f3ba259c700 1 -- 172.21.15.64:0/2378049844 learned_addr learned my addr 172.21.15.64:0/2378049844 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:23:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.766+0000 7f3ba2d9d700 1 --2- 172.21.15.64:0/2378049844 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e143bcc0 0x5582e1447f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.766+0000 7f3ba259c700 1 -- 172.21.15.64:0/2378049844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582e143b300 msgr2=0x5582e143b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:23:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.766+0000 7f3ba259c700 1 --2- 172.21.15.64:0/2378049844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582e143b300 0x5582e143b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.766+0000 7f3ba259c700 1 -- 172.21.15.64:0/2378049844 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e143bcc0 msgr2=0x5582e1447f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:23:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.766+0000 7f3ba259c700 1 --2- 172.21.15.64:0/2378049844 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e143bcc0 0x5582e1447f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.766+0000 7f3ba259c700 1 -- 172.21.15.64:0/2378049844 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582e11b9d80 con 0x5582e14411a0 2023-09-13T03:23:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.766+0000 7f3ba2d9d700 1 --2- 172.21.15.64:0/2378049844 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e143bcc0 0x5582e1447f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T03:23:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.766+0000 7f3ba259c700 1 --2- 172.21.15.64:0/2378049844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e14411a0 0x5582e1441580 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3b940a14e0 tx=0x7f3b94099380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=171d723c3ff8545a server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:23:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.766+0000 7f3b98ff9700 1 -- 172.21.15.64:0/2378049844 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b940b90d0 con 0x5582e14411a0 2023-09-13T03:23:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.766+0000 7f3b98ff9700 1 -- 172.21.15.64:0/2378049844 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b940999a0 con 0x5582e14411a0 2023-09-13T03:23:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.766+0000 7f3b98ff9700 1 -- 172.21.15.64:0/2378049844 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b94099c00 con 0x5582e14411a0 2023-09-13T03:23:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.767+0000 7f3bbc55fb80 1 -- 172.21.15.64:0/2378049844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e14411a0 msgr2=0x5582e1441580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:23:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.767+0000 7f3bbc55fb80 1 --2- 172.21.15.64:0/2378049844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e14411a0 0x5582e1441580 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3b940a14e0 tx=0x7f3b94099380 comp rx=0 tx=0).stop 2023-09-13T03:23:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.767+0000 7f3bbc55fb80 1 -- 172.21.15.64:0/2378049844 shutdown_connections 2023-09-13T03:23:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.767+0000 7f3bbc55fb80 1 --2- 172.21.15.64:0/2378049844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e14411a0 0x5582e1441580 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.767+0000 7f3bbc55fb80 1 --2- 172.21.15.64:0/2378049844 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e143bcc0 0x5582e1447f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.767+0000 7f3bbc55fb80 1 --2- 172.21.15.64:0/2378049844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582e143b300 0x5582e143b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.767+0000 7f3bbc55fb80 1 -- 172.21.15.64:0/2378049844 >> 172.21.15.64:0/2378049844 conn(0x5582e13b0030 msgr2=0x5582e1436560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:23:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.767+0000 7f3bbc55fb80 1 -- 172.21.15.64:0/2378049844 shutdown_connections 2023-09-13T03:23:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.767+0000 7f3bbc55fb80 1 -- 172.21.15.64:0/2378049844 wait complete. 2023-09-13T03:23:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.768+0000 7f3bbc55fb80 1 -- start start 2023-09-13T03:23:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.768+0000 7f3bbc55fb80 1 Processor -- start 2023-09-13T03:23:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.769+0000 7f3bbc55fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582e143b300 0x5582e144ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.769+0000 7f3bbc55fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e143bcc0 0x5582e144b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.769+0000 7f3bbc55fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e14411a0 0x5582e1445020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.769+0000 7f3bbc55fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582e132bb10 con 0x5582e14411a0 2023-09-13T03:23:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.769+0000 7f3bbc55fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582e132b810 con 0x5582e143bcc0 2023-09-13T03:23:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.769+0000 7f3bbc55fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582e120f8c0 con 0x5582e143b300 2023-09-13T03:23:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.769+0000 7f3bbc55fb80 10 client.0 _ll_register_callbacks cb 0x5582e1275a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:23:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.769+0000 7f3b9bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e143bcc0 0x5582e144b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:51.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.769+0000 7f3ba259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582e143b300 0x5582e144ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:51.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.769+0000 7f3b9bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e143bcc0 0x5582e144b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:51336/0 (socket says 172.21.15.64:51336) 2023-09-13T03:23:51.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.769+0000 7f3b9bfff700 1 -- 172.21.15.64:0/2753570648 learned_addr learned my addr 172.21.15.64:0/2753570648 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:23:51.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.769+0000 7f3ba2d9d700 1 --2- 172.21.15.64:0/2753570648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e14411a0 0x5582e1445020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:51.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.769+0000 7f3b9bfff700 1 -- 172.21.15.64:0/2753570648 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582e143b300 msgr2=0x5582e144ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:23:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.769+0000 7f3b9bfff700 1 --2- 172.21.15.64:0/2753570648 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582e143b300 0x5582e144ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.769+0000 7f3b9bfff700 1 -- 172.21.15.64:0/2753570648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e14411a0 msgr2=0x5582e1445020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:23:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.769+0000 7f3b9bfff700 1 --2- 172.21.15.64:0/2753570648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e14411a0 0x5582e1445020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.769+0000 7f3b9bfff700 1 -- 172.21.15.64:0/2753570648 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba41a5640 con 0x5582e143bcc0 2023-09-13T03:23:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.769+0000 7f3ba259c700 1 --2- 172.21.15.64:0/2753570648 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582e143b300 0x5582e144ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:23:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.770+0000 7f3b9bfff700 1 --2- 172.21.15.64:0/2753570648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e143bcc0 0x5582e144b2d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c0098e0 tx=0x7f3b8c01a050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d581b2f9163d8d1 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:23:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.770+0000 7f3b89ffb700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:23:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.770+0000 7f3b89ffb700 1 -- 172.21.15.64:0/2753570648 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c00f870 con 0x5582e143bcc0 2023-09-13T03:23:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.770+0000 7f3b89ffb700 1 -- 172.21.15.64:0/2753570648 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f3b8c00f9d0 con 0x5582e143bcc0 2023-09-13T03:23:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.770+0000 7f3bbc55fb80 1 -- 172.21.15.64:0/2753570648 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5582e12e5490 con 0x5582e143bcc0 2023-09-13T03:23:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.770+0000 7f3b89ffb700 1 -- 172.21.15.64:0/2753570648 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b8c00fb90 con 0x5582e143bcc0 2023-09-13T03:23:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.770+0000 7f3bbc55fb80 10 client.26984 fetch_fsmap learned FSMap version 61 2023-09-13T03:23:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.770+0000 7f3bbc55fb80 1 -- 172.21.15.64:0/2753570648 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x5582e11a2c00 con 0x5582e143bcc0 2023-09-13T03:23:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.771+0000 7f3b89ffb700 1 -- 172.21.15.64:0/2753570648 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b8c03c020 con 0x5582e143bcc0 2023-09-13T03:23:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.771+0000 7f3b89ffb700 1 -- 172.21.15.64:0/2753570648 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 61) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f3b8c018400 con 0x5582e143bcc0 2023-09-13T03:23:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.771+0000 7f3bbc55fb80 10 client.26984 fetch_fsmap finished waiting for FSMap version 61 2023-09-13T03:23:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.771+0000 7f3bbc55fb80 10 client.26984 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:23:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.771+0000 7f3bbc55fb80 20 client.26984 populate_metadata read hostname 'smithi064' 2023-09-13T03:23:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.771+0000 7f3bbc55fb80 1 --2- 172.21.15.64:0/2753570648 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5582e14551f0 0x5582e1457610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.771+0000 7f3bbc55fb80 4 client.26984 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:23:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.771+0000 7f3bbc55fb80 1 -- 172.21.15.64:0/2753570648 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5582e13a2840 con 0x5582e14551f0 2023-09-13T03:23:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.771+0000 7f3ba259c700 1 --2- 172.21.15.64:0/2753570648 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5582e14551f0 0x5582e1457610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.772+0000 7f3ba259c700 1 --2- 172.21.15.64:0/2753570648 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5582e14551f0 0x5582e1457610 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0c6df827ddfb974 server_cookie=58a8351fb3d330a in_seq=0 out_seq=0 2023-09-13T03:23:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.772+0000 7f3b89ffb700 10 client.26984 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:23:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.772+0000 7f3b89ffb700 1 -- 172.21.15.64:0/2753570648 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5582e13a2840 con 0x5582e14551f0 2023-09-13T03:23:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.772+0000 7f3b89ffb700 10 client.26984 handle_command_reply: tid=0 2023-09-13T03:23:51.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.772+0000 7f3bbc55fb80 1 client.26984 shutdown 2023-09-13T03:23:51.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.773+0000 7f3bbc55fb80 1 -- 172.21.15.64:0/2753570648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e143bcc0 msgr2=0x5582e144b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:23:51.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.773+0000 7f3bbc55fb80 1 --2- 172.21.15.64:0/2753570648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e143bcc0 0x5582e144b2d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c0098e0 tx=0x7f3b8c01a050 comp rx=0 tx=0).stop 2023-09-13T03:23:51.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.773+0000 7f3bbc55fb80 1 -- 172.21.15.64:0/2753570648 shutdown_connections 2023-09-13T03:23:51.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.773+0000 7f3bbc55fb80 1 --2- 172.21.15.64:0/2753570648 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5582e14551f0 0x5582e1457610 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:51.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.773+0000 7f3bbc55fb80 1 --2- 172.21.15.64:0/2753570648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e14411a0 0x5582e1445020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:51.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.773+0000 7f3bbc55fb80 1 --2- 172.21.15.64:0/2753570648 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582e143b300 0x5582e144ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:51.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.773+0000 7f3bbc55fb80 1 --2- 172.21.15.64:0/2753570648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e143bcc0 0x5582e144b2d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:51.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.773+0000 7f3b89ffb700 0 client.26984 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:23:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.773+0000 7f3bbc55fb80 1 -- 172.21.15.64:0/2753570648 >> 172.21.15.64:0/2753570648 conn(0x5582e13b0030 msgr2=0x5582e14357a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:23:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.773+0000 7f3bbc55fb80 1 -- 172.21.15.64:0/2753570648 shutdown_connections 2023-09-13T03:23:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.773+0000 7f3bbc55fb80 1 -- 172.21.15.64:0/2753570648 wait complete. 2023-09-13T03:23:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.773+0000 7f3bbc55fb80 20 client.26984 trim_cache size 0 max 16384 2023-09-13T03:23:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.789+0000 7f3bbc55fb80 1 Processor -- start 2023-09-13T03:23:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.789+0000 7f3bbc55fb80 1 -- start start 2023-09-13T03:23:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.789+0000 7f3bbc55fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e143b300 0x5582e1440680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.789+0000 7f3bbc55fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e143bcc0 0x5582e143aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.789+0000 7f3bbc55fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582e14411a0 0x5582e1435f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.789+0000 7f3bbc55fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582e1437930 con 0x5582e143bcc0 2023-09-13T03:23:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.789+0000 7f3bbc55fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582e1437ca0 con 0x5582e143b300 2023-09-13T03:23:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.789+0000 7f3bbc55fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582e14328b0 con 0x5582e14411a0 2023-09-13T03:23:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.790+0000 7f3ba259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e143b300 0x5582e1440680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.790+0000 7f3b9bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e143bcc0 0x5582e143aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.790+0000 7f3ba2d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582e14411a0 0x5582e1435f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.790+0000 7f3ba259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e143b300 0x5582e1440680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:51350/0 (socket says 172.21.15.64:51350) 2023-09-13T03:23:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.790+0000 7f3ba259c700 1 -- 172.21.15.64:0/767012187 learned_addr learned my addr 172.21.15.64:0/767012187 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:23:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.790+0000 7f3ba2d9d700 1 -- 172.21.15.64:0/767012187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e143b300 msgr2=0x5582e1440680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:23:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.790+0000 7f3ba2d9d700 1 --2- 172.21.15.64:0/767012187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e143b300 0x5582e1440680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.790+0000 7f3ba2d9d700 1 -- 172.21.15.64:0/767012187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e143bcc0 msgr2=0x5582e143aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:23:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.790+0000 7f3ba2d9d700 1 --2- 172.21.15.64:0/767012187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e143bcc0 0x5582e143aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.790+0000 7f3ba2d9d700 1 -- 172.21.15.64:0/767012187 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582e11e01f0 con 0x5582e14411a0 2023-09-13T03:23:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.790+0000 7f3b9bfff700 1 --2- 172.21.15.64:0/767012187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e143bcc0 0x5582e143aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:23:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.790+0000 7f3ba2d9d700 1 --2- 172.21.15.64:0/767012187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582e14411a0 0x5582e1435f20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c009470 tx=0x7f3b9c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b03d455b4c36a0eb server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:23:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.791+0000 7f3b997fa700 1 -- 172.21.15.64:0/767012187 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c00e040 con 0x5582e14411a0 2023-09-13T03:23:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.791+0000 7f3b997fa700 1 -- 172.21.15.64:0/767012187 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b9c00f040 con 0x5582e14411a0 2023-09-13T03:23:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.791+0000 7f3b997fa700 1 -- 172.21.15.64:0/767012187 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c016960 con 0x5582e14411a0 2023-09-13T03:23:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.791+0000 7f3bbc55fb80 1 -- 172.21.15.64:0/767012187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582e14411a0 msgr2=0x5582e1435f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:23:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.791+0000 7f3bbc55fb80 1 --2- 172.21.15.64:0/767012187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582e14411a0 0x5582e1435f20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c009470 tx=0x7f3b9c00b040 comp rx=0 tx=0).stop 2023-09-13T03:23:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.791+0000 7f3bbc55fb80 1 -- 172.21.15.64:0/767012187 shutdown_connections 2023-09-13T03:23:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.791+0000 7f3bbc55fb80 1 --2- 172.21.15.64:0/767012187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e143b300 0x5582e1440680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.791+0000 7f3bbc55fb80 1 --2- 172.21.15.64:0/767012187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582e14411a0 0x5582e1435f20 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.791+0000 7f3bbc55fb80 1 --2- 172.21.15.64:0/767012187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e143bcc0 0x5582e143aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.791+0000 7f3bbc55fb80 1 -- 172.21.15.64:0/767012187 >> 172.21.15.64:0/767012187 conn(0x5582e13b0030 msgr2=0x5582e143a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:23:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.791+0000 7f3bbc55fb80 1 -- 172.21.15.64:0/767012187 shutdown_connections 2023-09-13T03:23:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.792+0000 7f3bbc55fb80 1 -- 172.21.15.64:0/767012187 wait complete. 2023-09-13T03:23:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.792+0000 7f3bbc55fb80 1 -- start start 2023-09-13T03:23:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.792+0000 7f3bbc55fb80 1 Processor -- start 2023-09-13T03:23:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.793+0000 7f3bbc55fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e143b300 0x5582e14a2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.793+0000 7f3bbc55fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582e143bcc0 0x5582e14a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.793+0000 7f3bbc55fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e14411a0 0x5582e149cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.793+0000 7f3bbc55fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582e149d6c0 con 0x5582e143b300 2023-09-13T03:23:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.793+0000 7f3bbc55fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582e149d830 con 0x5582e14411a0 2023-09-13T03:23:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.793+0000 7f3bbc55fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582e149d9f0 con 0x5582e143bcc0 2023-09-13T03:23:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.793+0000 7f3bbc55fb80 10 client.0 _ll_register_callbacks cb 0x5582e1275a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:23:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.793+0000 7f3ba2d9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e14411a0 0x5582e149cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.793+0000 7f3ba259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e143b300 0x5582e14a2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.793+0000 7f3ba2d9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e14411a0 0x5582e149cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:51354/0 (socket says 172.21.15.64:51354) 2023-09-13T03:23:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.793+0000 7f3ba2d9d700 1 -- 172.21.15.64:0/2522093775 learned_addr learned my addr 172.21.15.64:0/2522093775 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:23:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.793+0000 7f3b9bfff700 1 --2- 172.21.15.64:0/2522093775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582e143bcc0 0x5582e14a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.793+0000 7f3ba2d9d700 1 -- 172.21.15.64:0/2522093775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582e143bcc0 msgr2=0x5582e14a2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:23:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.793+0000 7f3ba2d9d700 1 --2- 172.21.15.64:0/2522093775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582e143bcc0 0x5582e14a2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.793+0000 7f3ba2d9d700 1 -- 172.21.15.64:0/2522093775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e143b300 msgr2=0x5582e14a2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:23:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.793+0000 7f3ba2d9d700 1 --2- 172.21.15.64:0/2522093775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e143b300 0x5582e14a2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.793+0000 7f3ba2d9d700 1 -- 172.21.15.64:0/2522093775 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5582e11a2c00 con 0x5582e14411a0 2023-09-13T03:23:51.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.794+0000 7f3ba2d9d700 1 --2- 172.21.15.64:0/2522093775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e14411a0 0x5582e149cfb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c009010 tx=0x7f3b9c016770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55c55ce149959c43 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:23:51.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.794+0000 7f3b89ffb700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:23:51.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.794+0000 7f3b89ffb700 1 -- 172.21.15.64:0/2522093775 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c00e040 con 0x5582e14411a0 2023-09-13T03:23:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.794+0000 7f3bbc55fb80 1 -- 172.21.15.64:0/2522093775 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b940999a0 con 0x5582e14411a0 2023-09-13T03:23:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.794+0000 7f3b89ffb700 1 -- 172.21.15.64:0/2522093775 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f3b9c016da0 con 0x5582e14411a0 2023-09-13T03:23:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.794+0000 7f3b89ffb700 1 -- 172.21.15.64:0/2522093775 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b9c0163a0 con 0x5582e14411a0 2023-09-13T03:23:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.794+0000 7f3bbc55fb80 10 client.26990 fetch_fsmap learned FSMap version 61 2023-09-13T03:23:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.794+0000 7f3bbc55fb80 1 -- 172.21.15.64:0/2522093775 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x7f3b9c009150 con 0x5582e14411a0 2023-09-13T03:23:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.794+0000 7f3b89ffb700 1 -- 172.21.15.64:0/2522093775 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b940999a0 con 0x5582e14411a0 2023-09-13T03:23:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.795+0000 7f3b89ffb700 1 -- 172.21.15.64:0/2522093775 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 61) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f3b9c014050 con 0x5582e14411a0 2023-09-13T03:23:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.795+0000 7f3bbc55fb80 10 client.26990 fetch_fsmap finished waiting for FSMap version 61 2023-09-13T03:23:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.795+0000 7f3bbc55fb80 10 client.26990 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:23:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.795+0000 7f3bbc55fb80 20 client.26990 populate_metadata read hostname 'smithi064' 2023-09-13T03:23:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.795+0000 7f3bbc55fb80 1 --2- 172.21.15.64:0/2522093775 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5582e14aa500 0x5582e14ac920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:23:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.795+0000 7f3bbc55fb80 4 client.26990 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:23:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.795+0000 7f3bbc55fb80 1 -- 172.21.15.64:0/2522093775 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5582e13868c0 con 0x5582e14aa500 2023-09-13T03:23:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.795+0000 7f3ba259c700 1 --2- 172.21.15.64:0/2522093775 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5582e14aa500 0x5582e14ac920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:23:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.796+0000 7f3ba259c700 1 --2- 172.21.15.64:0/2522093775 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5582e14aa500 0x5582e14ac920 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e456299e1840147 server_cookie=16ce14ef0c48b321 in_seq=0 out_seq=0 2023-09-13T03:23:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.796+0000 7f3b89ffb700 10 client.26990 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:23:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.823+0000 7f3b89ffb700 1 -- 172.21.15.64:0/2522093775 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5582e13868c0 con 0x5582e14aa500 2023-09-13T03:23:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.823+0000 7f3b89ffb700 10 client.26990 handle_command_reply: tid=0 2023-09-13T03:23:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.823+0000 7f3bbc55fb80 1 client.26990 shutdown 2023-09-13T03:23:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.823+0000 7f3bbc55fb80 1 -- 172.21.15.64:0/2522093775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e14411a0 msgr2=0x5582e149cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:23:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.823+0000 7f3bbc55fb80 1 --2- 172.21.15.64:0/2522093775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e14411a0 0x5582e149cfb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c009010 tx=0x7f3b9c016770 comp rx=0 tx=0).stop 2023-09-13T03:23:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.823+0000 7f3bbc55fb80 1 -- 172.21.15.64:0/2522093775 shutdown_connections 2023-09-13T03:23:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.823+0000 7f3bbc55fb80 1 --2- 172.21.15.64:0/2522093775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e14411a0 0x5582e149cfb0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.823+0000 7f3bbc55fb80 1 --2- 172.21.15.64:0/2522093775 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5582e14aa500 0x5582e14ac920 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.823+0000 7f3bbc55fb80 1 --2- 172.21.15.64:0/2522093775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e143b300 0x5582e14a2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.823+0000 7f3bbc55fb80 1 --2- 172.21.15.64:0/2522093775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582e143bcc0 0x5582e14a2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.823+0000 7f3b89ffb700 0 client.26990 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:23:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.823+0000 7f3bbc55fb80 1 -- 172.21.15.64:0/2522093775 >> 172.21.15.64:0/2522093775 conn(0x5582e13b0030 msgr2=0x5582e1441d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:23:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.823+0000 7f3bbc55fb80 1 -- 172.21.15.64:0/2522093775 shutdown_connections 2023-09-13T03:23:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.824+0000 7f3bbc55fb80 1 -- 172.21.15.64:0/2522093775 wait complete. 2023-09-13T03:23:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.824+0000 7f3bbc55fb80 20 client.26990 trim_cache size 0 max 16384 2023-09-13T03:23:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:23:51.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:23:51.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:23:51.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:23:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.824+0000 7f3ba8bff700 1 -- 172.21.15.64:0/2469926730 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f3b900a07b0 msgr2=0x7f3b900a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:23:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.824+0000 7f3ba8bff700 1 --2- 172.21.15.64:0/2469926730 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f3b900a07b0 0x7f3b900a2c70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3ba40ffad0 tx=0x7f3b8c005fb0 comp rx=0 tx=0).stop 2023-09-13T03:23:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.824+0000 7f3ba8bff700 1 -- 172.21.15.64:0/2469926730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ba41005b0 msgr2=0x7f3ba419e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:23:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.824+0000 7f3ba8bff700 1 --2- 172.21.15.64:0/2469926730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ba41005b0 0x7f3ba419e830 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3b9400eb30 tx=0x7f3b9400ee40 comp rx=0 tx=0).stop 2023-09-13T03:23:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.825+0000 7f3ba8bff700 1 -- 172.21.15.64:0/2469926730 shutdown_connections 2023-09-13T03:23:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.825+0000 7f3ba8bff700 1 --2- 172.21.15.64:0/2469926730 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f3b900a07b0 0x7f3b900a2c70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.825+0000 7f3ba8bff700 1 --2- 172.21.15.64:0/2469926730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ba41005b0 0x7f3ba419e830 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.825+0000 7f3ba8bff700 1 --2- 172.21.15.64:0/2469926730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ba4106670 0x7f3ba4198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.825+0000 7f3ba8bff700 1 --2- 172.21.15.64:0/2469926730 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ba4100f70 0x7f3ba419ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:23:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.825+0000 7f3ba8bff700 1 -- 172.21.15.64:0/2469926730 >> 172.21.15.64:0/2469926730 conn(0x7f3ba40fc060 msgr2=0x7f3ba4072830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:23:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.825+0000 7f3ba8bff700 1 -- 172.21.15.64:0/2469926730 shutdown_connections 2023-09-13T03:23:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:23:51.825+0000 7f3ba8bff700 1 -- 172.21.15.64:0/2469926730 wait complete. 2023-09-13T03:23:51.946 INFO:tasks.workunit.client.0.smithi064.stdout: 1 234403 2.05 MB/sec execute 476 sec latency 45.431 ms 2023-09-13T03:23:52.946 INFO:tasks.workunit.client.0.smithi064.stdout: 1 235573 2.05 MB/sec execute 477 sec latency 27.248 ms 2023-09-13T03:23:52.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:52 smithi142 ceph-mon[134082]: pgmap v332: 97 pgs: 97 active+clean; 108 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2023-09-13T03:23:53.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:52 smithi170 ceph-mon[134328]: pgmap v332: 97 pgs: 97 active+clean; 108 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2023-09-13T03:23:53.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:52 smithi064 ceph-mon[97675]: pgmap v332: 97 pgs: 97 active+clean; 108 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2023-09-13T03:23:53.947 INFO:tasks.workunit.client.0.smithi064.stdout: 1 235839 2.05 MB/sec execute 478 sec latency 796.562 ms 2023-09-13T03:23:54.947 INFO:tasks.workunit.client.0.smithi064.stdout: 1 235839 2.05 MB/sec execute 479 sec latency 1796.732 ms 2023-09-13T03:23:54.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:54 smithi142 ceph-mon[134082]: pgmap v333: 97 pgs: 97 active+clean; 108 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 761 KiB/s wr, 101 op/s 2023-09-13T03:23:55.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:54 smithi170 ceph-mon[134328]: pgmap v333: 97 pgs: 97 active+clean; 108 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 761 KiB/s wr, 101 op/s 2023-09-13T03:23:55.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:54 smithi064 ceph-mon[97675]: pgmap v333: 97 pgs: 97 active+clean; 108 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 761 KiB/s wr, 101 op/s 2023-09-13T03:23:55.948 INFO:tasks.workunit.client.0.smithi064.stdout: 1 236287 2.04 MB/sec execute 480 sec latency 2627.683 ms 2023-09-13T03:23:56.947 INFO:tasks.workunit.client.0.smithi064.stdout: 1 237974 2.07 MB/sec execute 481 sec latency 13.660 ms 2023-09-13T03:23:56.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:56 smithi142 ceph-mon[134082]: pgmap v334: 97 pgs: 97 active+clean; 108 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 155 op/s 2023-09-13T03:23:57.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:56 smithi170 ceph-mon[134328]: pgmap v334: 97 pgs: 97 active+clean; 108 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 155 op/s 2023-09-13T03:23:57.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:56 smithi064 ceph-mon[97675]: pgmap v334: 97 pgs: 97 active+clean; 108 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 155 op/s 2023-09-13T03:23:57.947 INFO:tasks.workunit.client.0.smithi064.stdout: 1 239106 2.07 MB/sec execute 482 sec latency 25.545 ms 2023-09-13T03:23:58.947 INFO:tasks.workunit.client.0.smithi064.stdout: 1 240254 2.07 MB/sec execute 483 sec latency 232.804 ms 2023-09-13T03:23:59.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:23:58 smithi170 ceph-mon[134328]: pgmap v335: 97 pgs: 97 active+clean; 108 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 582 KiB/s wr, 90 op/s 2023-09-13T03:23:59.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:23:58 smithi064 ceph-mon[97675]: pgmap v335: 97 pgs: 97 active+clean; 108 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 582 KiB/s wr, 90 op/s 2023-09-13T03:23:59.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:23:58 smithi142 ceph-mon[134082]: pgmap v335: 97 pgs: 97 active+clean; 108 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 582 KiB/s wr, 90 op/s 2023-09-13T03:23:59.947 INFO:tasks.workunit.client.0.smithi064.stdout: 1 241695 2.09 MB/sec execute 484 sec latency 11.987 ms 2023-09-13T03:24:00.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:24:00 smithi170 ceph-mon[134328]: pgmap v336: 97 pgs: 97 active+clean; 108 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 582 KiB/s wr, 90 op/s 2023-09-13T03:24:00.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:24:00 smithi064 ceph-mon[97675]: pgmap v336: 97 pgs: 97 active+clean; 108 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 582 KiB/s wr, 90 op/s 2023-09-13T03:24:00.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:24:00 smithi142 ceph-mon[134082]: pgmap v336: 97 pgs: 97 active+clean; 108 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 582 KiB/s wr, 90 op/s 2023-09-13T03:24:00.947 INFO:tasks.workunit.client.0.smithi064.stdout: 1 242931 2.10 MB/sec execute 485 sec latency 43.058 ms 2023-09-13T03:24:01.948 INFO:tasks.workunit.client.0.smithi064.stdout: 1 242931 2.10 MB/sec execute 486 sec latency 1043.175 ms 2023-09-13T03:24:02.948 INFO:tasks.workunit.client.0.smithi064.stdout: 1 242931 2.09 MB/sec execute 487 sec latency 2043.307 ms 2023-09-13T03:24:02.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:24:02 smithi142 ceph-mon[134082]: pgmap v337: 97 pgs: 97 active+clean; 116 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 170 op/s 2023-09-13T03:24:03.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:24:02 smithi170 ceph-mon[134328]: pgmap v337: 97 pgs: 97 active+clean; 116 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 170 op/s 2023-09-13T03:24:03.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:24:02 smithi064 ceph-mon[97675]: pgmap v337: 97 pgs: 97 active+clean; 116 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 170 op/s 2023-09-13T03:24:03.948 INFO:tasks.workunit.client.0.smithi064.stdout: 1 242931 2.09 MB/sec execute 488 sec latency 3043.457 ms 2023-09-13T03:24:04.948 INFO:tasks.workunit.client.0.smithi064.stdout: 1 242931 2.08 MB/sec execute 489 sec latency 4043.529 ms 2023-09-13T03:24:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.986+0000 7f38551ff700 1 -- 172.21.15.64:0/1001541734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3850100f60 msgr2=0x7f385010d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.986+0000 7f38551ff700 1 --2- 172.21.15.64:0/1001541734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3850100f60 0x7f385010d420 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f384800b0d0 tx=0x7f384800b3e0 comp rx=0 tx=0).stop 2023-09-13T03:24:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.986+0000 7f38551ff700 1 -- 172.21.15.64:0/1001541734 shutdown_connections 2023-09-13T03:24:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.986+0000 7f38551ff700 1 --2- 172.21.15.64:0/1001541734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3850106660 0x7f3850106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.986+0000 7f38551ff700 1 --2- 172.21.15.64:0/1001541734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3850100f60 0x7f385010d420 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.986+0000 7f38551ff700 1 --2- 172.21.15.64:0/1001541734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38501005a0 0x7f3850100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.986+0000 7f38551ff700 1 -- 172.21.15.64:0/1001541734 >> 172.21.15.64:0/1001541734 conn(0x7f38500fc050 msgr2=0x7f38500fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:24:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.986+0000 7f38551ff700 1 -- 172.21.15.64:0/1001541734 shutdown_connections 2023-09-13T03:24:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.986+0000 7f38551ff700 1 -- 172.21.15.64:0/1001541734 wait complete. 2023-09-13T03:24:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.987+0000 7f38551ff700 1 Processor -- start 2023-09-13T03:24:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.987+0000 7f38551ff700 1 -- start start 2023-09-13T03:24:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.988+0000 7f38551ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38501005a0 0x7f385019e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.988+0000 7f38551ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3850100f60 0x7f385019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.988+0000 7f38551ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3850106660 0x7f3850198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.988+0000 7f38551ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3850198e80 con 0x7f3850100f60 2023-09-13T03:24:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.988+0000 7f38551ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3850198ff0 con 0x7f38501005a0 2023-09-13T03:24:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.988+0000 7f38551ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38501991b0 con 0x7f3850106660 2023-09-13T03:24:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.988+0000 7f3847fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3850100f60 0x7f385019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.988+0000 7f3847fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3850100f60 0x7f385019eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59026/0 (socket says 172.21.15.64:59026) 2023-09-13T03:24:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.988+0000 7f3847fff700 1 -- 172.21.15.64:0/642492150 learned_addr learned my addr 172.21.15.64:0/642492150 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:24:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.988+0000 7f384f59e700 1 --2- 172.21.15.64:0/642492150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3850106660 0x7f3850198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.988+0000 7f384ed9d700 1 --2- 172.21.15.64:0/642492150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38501005a0 0x7f385019e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.988+0000 7f3847fff700 1 -- 172.21.15.64:0/642492150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3850106660 msgr2=0x7f3850198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.988+0000 7f3847fff700 1 --2- 172.21.15.64:0/642492150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3850106660 0x7f3850198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.988+0000 7f3847fff700 1 -- 172.21.15.64:0/642492150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38501005a0 msgr2=0x7f385019e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.988+0000 7f3847fff700 1 --2- 172.21.15.64:0/642492150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38501005a0 0x7f385019e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:04.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.988+0000 7f3847fff700 1 -- 172.21.15.64:0/642492150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3848009d00 con 0x7f3850100f60 2023-09-13T03:24:04.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.989+0000 7f3847fff700 1 --2- 172.21.15.64:0/642492150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3850100f60 0x7f385019eef0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f383800e9a0 tx=0x7f383800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:24:04.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.989+0000 7f384cd99700 1 -- 172.21.15.64:0/642492150 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3838009790 con 0x7f3850100f60 2023-09-13T03:24:04.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.989+0000 7f384cd99700 1 -- 172.21.15.64:0/642492150 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3838010430 con 0x7f3850100f60 2023-09-13T03:24:04.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.989+0000 7f38551ff700 1 -- 172.21.15.64:0/642492150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3850199490 con 0x7f3850100f60 2023-09-13T03:24:04.996 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.989+0000 7f384cd99700 1 -- 172.21.15.64:0/642492150 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3838018680 con 0x7f3850100f60 2023-09-13T03:24:04.996 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.989+0000 7f38551ff700 1 -- 172.21.15.64:0/642492150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38501a59a0 con 0x7f3850100f60 2023-09-13T03:24:04.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:24:04 smithi142 ceph-mon[134082]: pgmap v338: 97 pgs: 97 active+clean; 116 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2023-09-13T03:24:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.992+0000 7f384cd99700 1 -- 172.21.15.64:0/642492150 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f38380105a0 con 0x7f3850100f60 2023-09-13T03:24:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.994+0000 7f3868c08b80 1 Processor -- start 2023-09-13T03:24:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.995+0000 7f3868c08b80 1 -- start start 2023-09-13T03:24:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.995+0000 7f384cd99700 1 --2- 172.21.15.64:0/642492150 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f383c0a07b0 0x7f383c0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.995+0000 7f3868c08b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df8008e1a0 0x55df8008e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.995+0000 7f3868c08b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df80088300 0x55df80088780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.995+0000 7f3868c08b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df80088cc0 0x55df80094f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.995+0000 7f3868c08b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df7ffe16d0 con 0x55df80088300 2023-09-13T03:24:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.995+0000 7f3868c08b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df7ffdeff0 con 0x55df80088cc0 2023-09-13T03:24:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.995+0000 7f3868c08b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df7ffda8e0 con 0x55df8008e1a0 2023-09-13T03:24:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.995+0000 7f384cd99700 1 -- 172.21.15.64:0/642492150 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f3838014070 con 0x7f3850100f60 2023-09-13T03:24:05.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.996+0000 7f384f59e700 1 --2- 172.21.15.64:0/642492150 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f383c0a07b0 0x7f383c0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:05.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.996+0000 7f3847fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df80088300 0x55df80088780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:05.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.996+0000 7f384ed9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df8008e1a0 0x55df8008e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:05.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.996+0000 7f384ed9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df8008e1a0 0x55df8008e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:36434/0 (socket says 172.21.15.64:36434) 2023-09-13T03:24:05.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.996+0000 7f3847fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df80088300 0x55df80088780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59042/0 (socket says 172.21.15.64:59042) 2023-09-13T03:24:05.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.996+0000 7f384ed9d700 1 -- 172.21.15.64:0/1178421448 learned_addr learned my addr 172.21.15.64:0/1178421448 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:24:05.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.996+0000 7f384f59e700 1 --2- 172.21.15.64:0/1178421448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df80088cc0 0x55df80094f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:05.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.996+0000 7f384ed9d700 1 -- 172.21.15.64:0/1178421448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df80088cc0 msgr2=0x55df80094f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:24:05.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.996+0000 7f384ed9d700 1 --2- 172.21.15.64:0/1178421448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df80088cc0 0x55df80094f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:05.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.996+0000 7f384ed9d700 1 -- 172.21.15.64:0/1178421448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df80088300 msgr2=0x55df80088780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:24:05.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.996+0000 7f384ed9d700 1 --2- 172.21.15.64:0/1178421448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df80088300 0x55df80088780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:05.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.996+0000 7f384ed9d700 1 -- 172.21.15.64:0/1178421448 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df7fe06d80 con 0x55df8008e1a0 2023-09-13T03:24:05.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.996+0000 7f384f59e700 1 --2- 172.21.15.64:0/1178421448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df80088cc0 0x55df80094f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:24:05.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.996+0000 7f384ed9d700 1 --2- 172.21.15.64:0/1178421448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df8008e1a0 0x55df8008e580 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3840009d00 tx=0x7f3840011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f08bb2ac84d3b43d server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:24:05.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.996+0000 7f384f59e700 1 --2- 172.21.15.64:0/642492150 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f383c0a07b0 0x7f383c0a2c70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3848009cd0 tx=0x7f3848009c00 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:24:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.997+0000 7f38457fa700 1 -- 172.21.15.64:0/1178421448 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3840004230 con 0x55df8008e1a0 2023-09-13T03:24:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.997+0000 7f38457fa700 1 -- 172.21.15.64:0/1178421448 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3840004390 con 0x55df8008e1a0 2023-09-13T03:24:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.997+0000 7f38457fa700 1 -- 172.21.15.64:0/1178421448 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3840019880 con 0x55df8008e1a0 2023-09-13T03:24:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.997+0000 7f3868c08b80 1 -- 172.21.15.64:0/1178421448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df8008e1a0 msgr2=0x55df8008e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.997+0000 7f3868c08b80 1 --2- 172.21.15.64:0/1178421448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df8008e1a0 0x55df8008e580 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3840009d00 tx=0x7f3840011720 comp rx=0 tx=0).stop 2023-09-13T03:24:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.997+0000 7f3868c08b80 1 -- 172.21.15.64:0/1178421448 shutdown_connections 2023-09-13T03:24:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.997+0000 7f3868c08b80 1 --2- 172.21.15.64:0/1178421448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df80088cc0 0x55df80094f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.997+0000 7f3868c08b80 1 --2- 172.21.15.64:0/1178421448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df8008e1a0 0x55df8008e580 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.997+0000 7f3868c08b80 1 --2- 172.21.15.64:0/1178421448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df80088300 0x55df80088780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.997+0000 7f3868c08b80 1 -- 172.21.15.64:0/1178421448 >> 172.21.15.64:0/1178421448 conn(0x55df7fffd030 msgr2=0x55df80083560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:24:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.997+0000 7f3868c08b80 1 -- 172.21.15.64:0/1178421448 shutdown_connections 2023-09-13T03:24:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.997+0000 7f3868c08b80 1 -- 172.21.15.64:0/1178421448 wait complete. 2023-09-13T03:24:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.998+0000 7f3868c08b80 1 -- start start 2023-09-13T03:24:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.998+0000 7f3868c08b80 1 Processor -- start 2023-09-13T03:24:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.999+0000 7f3868c08b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df80088300 0x55df80094d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.999+0000 7f3868c08b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df80088cc0 0x55df80091fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.999+0000 7f3868c08b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df8008e1a0 0x55df800924e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.999+0000 7f3868c08b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df7ff78b10 con 0x55df8008e1a0 2023-09-13T03:24:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.999+0000 7f3868c08b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df7ff78810 con 0x55df80088300 2023-09-13T03:24:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.999+0000 7f3868c08b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df7fe5c8c0 con 0x55df80088cc0 2023-09-13T03:24:05.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.999+0000 7f3868c08b80 10 client.0 _ll_register_callbacks cb 0x55df7fec2a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:24:05.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.999+0000 7f384f59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df8008e1a0 0x55df800924e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:05.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.999+0000 7f384ed9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df80088300 0x55df80094d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:05.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.999+0000 7f384ed9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df80088300 0x55df80094d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:46450/0 (socket says 172.21.15.64:46450) 2023-09-13T03:24:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.999+0000 7f3847fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df80088cc0 0x55df80091fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.999+0000 7f384ed9d700 1 -- 172.21.15.64:0/304883146 learned_addr learned my addr 172.21.15.64:0/304883146 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:24:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:04.999+0000 7f384f59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df8008e1a0 0x55df800924e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59050/0 (socket says 172.21.15.64:59050) 2023-09-13T03:24:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.000+0000 7f384ed9d700 1 -- 172.21.15.64:0/304883146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df80088cc0 msgr2=0x55df80091fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:24:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.000+0000 7f384ed9d700 1 --2- 172.21.15.64:0/304883146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df80088cc0 0x55df80091fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.000+0000 7f384ed9d700 1 -- 172.21.15.64:0/304883146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df8008e1a0 msgr2=0x55df800924e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:24:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.000+0000 7f384ed9d700 1 --2- 172.21.15.64:0/304883146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df8008e1a0 0x55df800924e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.000+0000 7f384ed9d700 1 -- 172.21.15.64:0/304883146 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3840009960 con 0x55df80088300 2023-09-13T03:24:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.000+0000 7f384f59e700 1 --2- 172.21.15.64:0/304883146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df8008e1a0 0x55df800924e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:24:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.000+0000 7f3847fff700 1 --2- 172.21.15.64:0/304883146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df80088cc0 0x55df80091fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:24:05.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.000+0000 7f384ed9d700 1 --2- 172.21.15.64:0/304883146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df80088300 0x55df80094d20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3840000c00 tx=0x7f38400038b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc69d75adfcf43e2 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:24:05.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.000+0000 7f38367fc700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:24:05.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.000+0000 7f38367fc700 1 -- 172.21.15.64:0/304883146 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3840003aa0 con 0x55df80088300 2023-09-13T03:24:05.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.000+0000 7f3868c08b80 1 -- 172.21.15.64:0/304883146 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55df7ff32490 con 0x55df80088300 2023-09-13T03:24:05.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.000+0000 7f38367fc700 1 -- 172.21.15.64:0/304883146 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f3840005ce0 con 0x55df80088300 2023-09-13T03:24:05.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.001+0000 7f38367fc700 1 -- 172.21.15.64:0/304883146 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3840005250 con 0x55df80088300 2023-09-13T03:24:05.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.001+0000 7f3868c08b80 10 client.27002 fetch_fsmap learned FSMap version 61 2023-09-13T03:24:05.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.001+0000 7f3868c08b80 1 -- 172.21.15.64:0/304883146 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x55df7fdefc00 con 0x55df80088300 2023-09-13T03:24:05.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.001+0000 7f38367fc700 1 -- 172.21.15.64:0/304883146 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55df7ff32490 con 0x55df80088300 2023-09-13T03:24:05.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.001+0000 7f38367fc700 1 -- 172.21.15.64:0/304883146 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 61) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f3840028020 con 0x55df80088300 2023-09-13T03:24:05.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.001+0000 7f3868c08b80 10 client.27002 fetch_fsmap finished waiting for FSMap version 61 2023-09-13T03:24:05.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.001+0000 7f3868c08b80 10 client.27002 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:24:05.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.001+0000 7f3868c08b80 20 client.27002 populate_metadata read hostname 'smithi064' 2023-09-13T03:24:05.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.001+0000 7f3868c08b80 1 --2- 172.21.15.64:0/304883146 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55df800a22a0 0x55df800a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:05.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.001+0000 7f3868c08b80 4 client.27002 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:24:05.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.001+0000 7f3868c08b80 1 -- 172.21.15.64:0/304883146 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df7ffef840 con 0x55df800a22a0 2023-09-13T03:24:05.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.002+0000 7f3847fff700 1 --2- 172.21.15.64:0/304883146 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55df800a22a0 0x55df800a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:05.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.002+0000 7f3847fff700 1 --2- 172.21.15.64:0/304883146 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55df800a22a0 0x55df800a46c0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=509b1a1eab348198 server_cookie=a330721f7669d030 in_seq=0 out_seq=0 2023-09-13T03:24:05.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.002+0000 7f38367fc700 10 client.27002 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:24:05.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.003+0000 7f38367fc700 1 -- 172.21.15.64:0/304883146 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55df7ffef840 con 0x55df800a22a0 2023-09-13T03:24:05.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.003+0000 7f38367fc700 10 client.27002 handle_command_reply: tid=0 2023-09-13T03:24:05.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.003+0000 7f3868c08b80 1 client.27002 shutdown 2023-09-13T03:24:05.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.003+0000 7f3868c08b80 1 -- 172.21.15.64:0/304883146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df80088300 msgr2=0x55df80094d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:05.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.003+0000 7f3868c08b80 1 --2- 172.21.15.64:0/304883146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df80088300 0x55df80094d20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3840000c00 tx=0x7f38400038b0 comp rx=0 tx=0).stop 2023-09-13T03:24:05.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.003+0000 7f3868c08b80 1 -- 172.21.15.64:0/304883146 shutdown_connections 2023-09-13T03:24:05.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.003+0000 7f3868c08b80 1 --2- 172.21.15.64:0/304883146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df80088300 0x55df80094d20 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:05.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.003+0000 7f3868c08b80 1 --2- 172.21.15.64:0/304883146 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55df800a22a0 0x55df800a46c0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:05.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.003+0000 7f3868c08b80 1 --2- 172.21.15.64:0/304883146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df8008e1a0 0x55df800924e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:05.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.003+0000 7f38367fc700 0 client.27002 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:24:05.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.003+0000 7f3868c08b80 1 --2- 172.21.15.64:0/304883146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df80088cc0 0x55df80091fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:05.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.003+0000 7f3868c08b80 1 -- 172.21.15.64:0/304883146 >> 172.21.15.64:0/304883146 conn(0x55df7fffd030 msgr2=0x55df800827a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:24:05.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.004+0000 7f3868c08b80 1 -- 172.21.15.64:0/304883146 shutdown_connections 2023-09-13T03:24:05.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.004+0000 7f3868c08b80 1 -- 172.21.15.64:0/304883146 wait complete. 2023-09-13T03:24:05.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.004+0000 7f3868c08b80 20 client.27002 trim_cache size 0 max 16384 2023-09-13T03:24:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.019+0000 7f3868c08b80 1 Processor -- start 2023-09-13T03:24:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.019+0000 7f3868c08b80 1 -- start start 2023-09-13T03:24:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.020+0000 7f3868c08b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df80088300 0x55df80099530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.020+0000 7f3868c08b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df80088cc0 0x55df80087a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.020+0000 7f3868c08b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df8008e1a0 0x55df80082f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.020+0000 7f3868c08b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df8007ffc0 con 0x55df80088300 2023-09-13T03:24:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.020+0000 7f3868c08b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df8008e930 con 0x55df8008e1a0 2023-09-13T03:24:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.020+0000 7f3868c08b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df80099a70 con 0x55df80088cc0 2023-09-13T03:24:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.020+0000 7f384ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df80088300 0x55df80099530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.020+0000 7f384ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df80088300 0x55df80099530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59052/0 (socket says 172.21.15.64:59052) 2023-09-13T03:24:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.020+0000 7f384ed9d700 1 -- 172.21.15.64:0/2008349300 learned_addr learned my addr 172.21.15.64:0/2008349300 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:24:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.020+0000 7f3847fff700 1 --2- 172.21.15.64:0/2008349300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df80088cc0 0x55df80087a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.020+0000 7f384f59e700 1 --2- 172.21.15.64:0/2008349300 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df8008e1a0 0x55df80082f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.020+0000 7f384ed9d700 1 -- 172.21.15.64:0/2008349300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df80088cc0 msgr2=0x55df80087a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:24:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.020+0000 7f384ed9d700 1 --2- 172.21.15.64:0/2008349300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df80088cc0 0x55df80087a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:05.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.020+0000 7f384ed9d700 1 -- 172.21.15.64:0/2008349300 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df8008e1a0 msgr2=0x55df80082f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:24:05.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.020+0000 7f384ed9d700 1 --2- 172.21.15.64:0/2008349300 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df8008e1a0 0x55df80082f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:05.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.020+0000 7f384ed9d700 1 -- 172.21.15.64:0/2008349300 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df7fe2d1f0 con 0x55df80088300 2023-09-13T03:24:05.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.020+0000 7f384f59e700 1 --2- 172.21.15.64:0/2008349300 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df8008e1a0 0x55df80082f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:24:05.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.021+0000 7f384ed9d700 1 --2- 172.21.15.64:0/2008349300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df80088300 0x55df80099530 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3840005a60 tx=0x7f3840003ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=109af7cf999b232a server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:24:05.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.021+0000 7f3845ffb700 1 -- 172.21.15.64:0/2008349300 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3840015070 con 0x55df80088300 2023-09-13T03:24:05.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.021+0000 7f3845ffb700 1 -- 172.21.15.64:0/2008349300 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3840003920 con 0x55df80088300 2023-09-13T03:24:05.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.021+0000 7f3845ffb700 1 -- 172.21.15.64:0/2008349300 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3840004760 con 0x55df80088300 2023-09-13T03:24:05.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.021+0000 7f3868c08b80 1 -- 172.21.15.64:0/2008349300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df80088300 msgr2=0x55df80099530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:05.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.021+0000 7f3868c08b80 1 --2- 172.21.15.64:0/2008349300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df80088300 0x55df80099530 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3840005a60 tx=0x7f3840003ea0 comp rx=0 tx=0).stop 2023-09-13T03:24:05.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.021+0000 7f3868c08b80 1 -- 172.21.15.64:0/2008349300 shutdown_connections 2023-09-13T03:24:05.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.021+0000 7f3868c08b80 1 --2- 172.21.15.64:0/2008349300 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df8008e1a0 0x55df80082f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:05.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.021+0000 7f3868c08b80 1 --2- 172.21.15.64:0/2008349300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df80088300 0x55df80099530 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:05.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.021+0000 7f3868c08b80 1 --2- 172.21.15.64:0/2008349300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df80088cc0 0x55df80087a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:05.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.021+0000 7f3868c08b80 1 -- 172.21.15.64:0/2008349300 >> 172.21.15.64:0/2008349300 conn(0x55df7fffd030 msgr2=0x55df800872f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:24:05.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.021+0000 7f3868c08b80 1 -- 172.21.15.64:0/2008349300 shutdown_connections 2023-09-13T03:24:05.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.022+0000 7f3868c08b80 1 -- 172.21.15.64:0/2008349300 wait complete. 2023-09-13T03:24:05.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.022+0000 7f3868c08b80 1 -- start start 2023-09-13T03:24:05.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.022+0000 7f3868c08b80 1 Processor -- start 2023-09-13T03:24:05.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.023+0000 7f3868c08b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df80088300 0x55df800ea230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:05.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.023+0000 7f3868c08b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df80088cc0 0x55df800ea770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:05.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.023+0000 7f3868c08b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df8008e1a0 0x55df800ef100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:05.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.023+0000 7f3868c08b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df800ef640 con 0x55df80088300 2023-09-13T03:24:05.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.023+0000 7f3868c08b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df800ef7b0 con 0x55df80088cc0 2023-09-13T03:24:05.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.023+0000 7f3868c08b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df800ef920 con 0x55df8008e1a0 2023-09-13T03:24:05.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.023+0000 7f3868c08b80 10 client.0 _ll_register_callbacks cb 0x7f3838010b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:24:05.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.023+0000 7f384ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df80088300 0x55df800ea230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:05.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.023+0000 7f384ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df80088300 0x55df800ea230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59054/0 (socket says 172.21.15.64:59054) 2023-09-13T03:24:05.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.023+0000 7f384ed9d700 1 -- 172.21.15.64:0/1229626842 learned_addr learned my addr 172.21.15.64:0/1229626842 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:24:05.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.023+0000 7f3847fff700 1 --2- 172.21.15.64:0/1229626842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df80088cc0 0x55df800ea770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.023+0000 7f384f59e700 1 --2- 172.21.15.64:0/1229626842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df8008e1a0 0x55df800ef100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.023+0000 7f384ed9d700 1 -- 172.21.15.64:0/1229626842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df8008e1a0 msgr2=0x55df800ef100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:24:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.023+0000 7f384ed9d700 1 --2- 172.21.15.64:0/1229626842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df8008e1a0 0x55df800ef100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.023+0000 7f384ed9d700 1 -- 172.21.15.64:0/1229626842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df80088cc0 msgr2=0x55df800ea770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:24:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.023+0000 7f384ed9d700 1 --2- 172.21.15.64:0/1229626842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df80088cc0 0x55df800ea770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.023+0000 7f384ed9d700 1 -- 172.21.15.64:0/1229626842 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df7fdefc00 con 0x55df80088300 2023-09-13T03:24:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.023+0000 7f384f59e700 1 --2- 172.21.15.64:0/1229626842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df8008e1a0 0x55df800ef100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:24:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.024+0000 7f384ed9d700 1 --2- 172.21.15.64:0/1229626842 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df80088300 0x55df800ea230 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3840022620 tx=0x7f3840022990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c29b8454d5aed608 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:24:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.024+0000 7f38367fc700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:24:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.024+0000 7f38367fc700 1 -- 172.21.15.64:0/1229626842 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3840015070 con 0x55df80088300 2023-09-13T03:24:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.024+0000 7f38367fc700 1 -- 172.21.15.64:0/1229626842 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f3840000e40 con 0x55df80088300 2023-09-13T03:24:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.024+0000 7f3868c08b80 1 -- 172.21.15.64:0/1229626842 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55df800895c0 con 0x55df80088300 2023-09-13T03:24:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.024+0000 7f38367fc700 1 -- 172.21.15.64:0/1229626842 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3840005810 con 0x55df80088300 2023-09-13T03:24:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.024+0000 7f3868c08b80 10 client.16965 fetch_fsmap learned FSMap version 61 2023-09-13T03:24:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.024+0000 7f3868c08b80 1 -- 172.21.15.64:0/1229626842 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x55df7fe05250 con 0x55df80088300 2023-09-13T03:24:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.024+0000 7f38367fc700 1 -- 172.21.15.64:0/1229626842 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55df800895c0 con 0x55df80088300 2023-09-13T03:24:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.024+0000 7f38367fc700 1 -- 172.21.15.64:0/1229626842 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 61) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f384003b420 con 0x55df80088300 2023-09-13T03:24:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.025+0000 7f3868c08b80 10 client.16965 fetch_fsmap finished waiting for FSMap version 61 2023-09-13T03:24:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.025+0000 7f3868c08b80 10 client.16965 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:24:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.025+0000 7f3868c08b80 20 client.16965 populate_metadata read hostname 'smithi064' 2023-09-13T03:24:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.025+0000 7f3868c08b80 1 --2- 172.21.15.64:0/1229626842 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55df800f6c50 0x55df800f9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.025+0000 7f3868c08b80 4 client.16965 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:24:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.025+0000 7f3868c08b80 1 -- 172.21.15.64:0/1229626842 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f3840004390 con 0x55df800f6c50 2023-09-13T03:24:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.025+0000 7f3847fff700 1 --2- 172.21.15.64:0/1229626842 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55df800f6c50 0x55df800f9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.025+0000 7f3847fff700 1 --2- 172.21.15.64:0/1229626842 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55df800f6c50 0x55df800f9040 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0fea7c0197713f4 server_cookie=436f9b519abee5d4 in_seq=0 out_seq=0 2023-09-13T03:24:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.025+0000 7f38367fc700 10 client.16965 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:24:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.060+0000 7f38367fc700 1 -- 172.21.15.64:0/1229626842 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f3840004390 con 0x55df800f6c50 2023-09-13T03:24:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.060+0000 7f38367fc700 10 client.16965 handle_command_reply: tid=0 2023-09-13T03:24:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.060+0000 7f3868c08b80 1 client.16965 shutdown 2023-09-13T03:24:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.060+0000 7f3868c08b80 1 -- 172.21.15.64:0/1229626842 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df80088300 msgr2=0x55df800ea230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.060+0000 7f3868c08b80 1 --2- 172.21.15.64:0/1229626842 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df80088300 0x55df800ea230 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3840022620 tx=0x7f3840022990 comp rx=0 tx=0).stop 2023-09-13T03:24:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.060+0000 7f3868c08b80 1 -- 172.21.15.64:0/1229626842 shutdown_connections 2023-09-13T03:24:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.060+0000 7f3868c08b80 1 --2- 172.21.15.64:0/1229626842 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55df800f6c50 0x55df800f9040 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.060+0000 7f3868c08b80 1 --2- 172.21.15.64:0/1229626842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df8008e1a0 0x55df800ef100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.060+0000 7f3868c08b80 1 --2- 172.21.15.64:0/1229626842 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df80088300 0x55df800ea230 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.060+0000 7f3868c08b80 1 --2- 172.21.15.64:0/1229626842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df80088cc0 0x55df800ea770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.060+0000 7f38367fc700 0 client.16965 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:24:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.060+0000 7f3868c08b80 1 -- 172.21.15.64:0/1229626842 >> 172.21.15.64:0/1229626842 conn(0x55df7fffd030 msgr2=0x55df80087b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:24:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.060+0000 7f3868c08b80 1 -- 172.21.15.64:0/1229626842 shutdown_connections 2023-09-13T03:24:05.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.060+0000 7f3868c08b80 1 -- 172.21.15.64:0/1229626842 wait complete. 2023-09-13T03:24:05.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.060+0000 7f3868c08b80 20 client.16965 trim_cache size 0 max 16384 2023-09-13T03:24:05.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:24:05.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:24:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:24:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:24:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.061+0000 7f38551ff700 1 -- 172.21.15.64:0/642492150 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f383c0a07b0 msgr2=0x7f383c0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.061+0000 7f38551ff700 1 --2- 172.21.15.64:0/642492150 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f383c0a07b0 0x7f383c0a2c70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3848009cd0 tx=0x7f3848009c00 comp rx=0 tx=0).stop 2023-09-13T03:24:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.061+0000 7f38551ff700 1 -- 172.21.15.64:0/642492150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3850100f60 msgr2=0x7f385019eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.061+0000 7f38551ff700 1 --2- 172.21.15.64:0/642492150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3850100f60 0x7f385019eef0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f383800e9a0 tx=0x7f383800ecb0 comp rx=0 tx=0).stop 2023-09-13T03:24:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.061+0000 7f38551ff700 1 -- 172.21.15.64:0/642492150 shutdown_connections 2023-09-13T03:24:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.061+0000 7f38551ff700 1 --2- 172.21.15.64:0/642492150 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f383c0a07b0 0x7f383c0a2c70 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.061+0000 7f38551ff700 1 --2- 172.21.15.64:0/642492150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38501005a0 0x7f385019e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.061+0000 7f38551ff700 1 --2- 172.21.15.64:0/642492150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3850106660 0x7f3850198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.061+0000 7f38551ff700 1 --2- 172.21.15.64:0/642492150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3850100f60 0x7f385019eef0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.061+0000 7f38551ff700 1 -- 172.21.15.64:0/642492150 >> 172.21.15.64:0/642492150 conn(0x7f38500fc050 msgr2=0x7f38500fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:24:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.061+0000 7f38551ff700 1 -- 172.21.15.64:0/642492150 shutdown_connections 2023-09-13T03:24:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:05.061+0000 7f38551ff700 1 -- 172.21.15.64:0/642492150 wait complete. 2023-09-13T03:24:05.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:24:04 smithi170 ceph-mon[134328]: pgmap v338: 97 pgs: 97 active+clean; 116 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2023-09-13T03:24:05.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:24:04 smithi064 ceph-mon[97675]: pgmap v338: 97 pgs: 97 active+clean; 116 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2023-09-13T03:24:05.948 INFO:tasks.workunit.client.0.smithi064.stdout: 1 243536 2.08 MB/sec execute 490 sec latency 4452.245 ms 2023-09-13T03:24:06.948 INFO:tasks.workunit.client.0.smithi064.stdout: 1 245111 2.10 MB/sec execute 491 sec latency 15.678 ms 2023-09-13T03:24:07.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:24:06 smithi170 ceph-mon[134328]: pgmap v339: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 961 KiB/s rd, 1.5 MiB/s wr, 183 op/s 2023-09-13T03:24:07.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:24:06 smithi064 ceph-mon[97675]: pgmap v339: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 961 KiB/s rd, 1.5 MiB/s wr, 183 op/s 2023-09-13T03:24:07.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:24:06 smithi142 ceph-mon[134082]: pgmap v339: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 961 KiB/s rd, 1.5 MiB/s wr, 183 op/s 2023-09-13T03:24:07.948 INFO:tasks.workunit.client.0.smithi064.stdout: 1 246282 2.11 MB/sec execute 492 sec latency 22.281 ms 2023-09-13T03:24:08.948 INFO:tasks.workunit.client.0.smithi064.stdout: 1 246478 2.10 MB/sec execute 493 sec latency 736.386 ms 2023-09-13T03:24:09.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:24:08 smithi170 ceph-mon[134328]: pgmap v340: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2023-09-13T03:24:09.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:24:08 smithi064 ceph-mon[97675]: pgmap v340: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2023-09-13T03:24:09.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:24:08 smithi142 ceph-mon[134082]: pgmap v340: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2023-09-13T03:24:09.948 INFO:tasks.workunit.client.0.smithi064.stdout: 1 246478 2.10 MB/sec execute 494 sec latency 1736.547 ms 2023-09-13T03:24:10.948 INFO:tasks.workunit.client.0.smithi064.stdout: 1 246860 2.10 MB/sec execute 495 sec latency 2580.900 ms 2023-09-13T03:24:11.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:24:10 smithi170 ceph-mon[134328]: pgmap v341: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2023-09-13T03:24:11.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:24:10 smithi170 ceph-mon[134328]: osdmap e95: 12 total, 12 up, 12 in 2023-09-13T03:24:11.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:24:10 smithi064 ceph-mon[97675]: pgmap v341: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2023-09-13T03:24:11.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:24:10 smithi064 ceph-mon[97675]: osdmap e95: 12 total, 12 up, 12 in 2023-09-13T03:24:11.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:24:10 smithi142 ceph-mon[134082]: pgmap v341: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2023-09-13T03:24:11.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:24:10 smithi142 ceph-mon[134082]: osdmap e95: 12 total, 12 up, 12 in 2023-09-13T03:24:11.948 INFO:tasks.workunit.client.0.smithi064.stdout: 1 248608 2.12 MB/sec execute 496 sec latency 18.682 ms 2023-09-13T03:24:12.949 INFO:tasks.workunit.client.0.smithi064.stdout: 1 249739 2.13 MB/sec execute 497 sec latency 23.387 ms 2023-09-13T03:24:13.081 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:24:12 smithi170 ceph-mon[134328]: pgmap v343: 97 pgs: 97 active+clean; 110 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 959 KiB/s wr, 137 op/s 2023-09-13T03:24:13.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:24:12 smithi064 ceph-mon[97675]: pgmap v343: 97 pgs: 97 active+clean; 110 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 959 KiB/s wr, 137 op/s 2023-09-13T03:24:13.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:24:12 smithi142 ceph-mon[134082]: pgmap v343: 97 pgs: 97 active+clean; 110 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 959 KiB/s wr, 137 op/s 2023-09-13T03:24:13.949 INFO:tasks.workunit.client.0.smithi064.stdout: 1 250021 2.12 MB/sec execute 498 sec latency 553.353 ms 2023-09-13T03:24:14.949 INFO:tasks.workunit.client.0.smithi064.stdout: 1 250021 2.12 MB/sec execute 499 sec latency 1553.496 ms 2023-09-13T03:24:15.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:24:14 smithi170 ceph-mon[134328]: pgmap v344: 97 pgs: 97 active+clean; 110 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 959 KiB/s wr, 137 op/s 2023-09-13T03:24:15.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:24:14 smithi170 ceph-mon[134328]: osdmap e96: 12 total, 12 up, 12 in 2023-09-13T03:24:15.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:24:14 smithi064 ceph-mon[97675]: pgmap v344: 97 pgs: 97 active+clean; 110 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 959 KiB/s wr, 137 op/s 2023-09-13T03:24:15.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:24:14 smithi064 ceph-mon[97675]: osdmap e96: 12 total, 12 up, 12 in 2023-09-13T03:24:15.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:24:14 smithi142 ceph-mon[134082]: pgmap v344: 97 pgs: 97 active+clean; 110 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 959 KiB/s wr, 137 op/s 2023-09-13T03:24:15.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:24:14 smithi142 ceph-mon[134082]: osdmap e96: 12 total, 12 up, 12 in 2023-09-13T03:24:15.949 INFO:tasks.workunit.client.0.smithi064.stdout: 1 250392 2.11 MB/sec execute 500 sec latency 2396.051 ms 2023-09-13T03:24:16.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:24:16 smithi170 ceph-mon[134328]: pgmap v346: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 170 op/s 2023-09-13T03:24:16.949 INFO:tasks.workunit.client.0.smithi064.stdout: 1 252178 2.14 MB/sec execute 501 sec latency 14.153 ms 2023-09-13T03:24:17.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:24:16 smithi064 ceph-mon[97675]: pgmap v346: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 170 op/s 2023-09-13T03:24:17.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:24:16 smithi142 ceph-mon[134082]: pgmap v346: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 170 op/s 2023-09-13T03:24:17.949 INFO:tasks.workunit.client.0.smithi064.stdout: 1 253348 2.14 MB/sec execute 502 sec latency 23.591 ms 2023-09-13T03:24:18.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.215+0000 7fbfe53ff700 1 -- 172.21.15.64:0/2830305088 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfe01005b0 msgr2=0x7fbfe0100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:18.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.215+0000 7fbfe53ff700 1 --2- 172.21.15.64:0/2830305088 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfe01005b0 0x7fbfe0100a30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4009990 tx=0x7fbfd4009ca0 comp rx=0 tx=0).stop 2023-09-13T03:24:18.217 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.215+0000 7fbfe53ff700 1 -- 172.21.15.64:0/2830305088 shutdown_connections 2023-09-13T03:24:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.215+0000 7fbfe53ff700 1 --2- 172.21.15.64:0/2830305088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfe0100f70 0x7fbfe010d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.215+0000 7fbfe53ff700 1 --2- 172.21.15.64:0/2830305088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfe0106670 0x7fbfe0106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.215+0000 7fbfe53ff700 1 --2- 172.21.15.64:0/2830305088 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfe01005b0 0x7fbfe0100a30 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.215+0000 7fbfe53ff700 1 -- 172.21.15.64:0/2830305088 >> 172.21.15.64:0/2830305088 conn(0x7fbfe00fc060 msgr2=0x7fbfe00fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:24:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.215+0000 7fbfe53ff700 1 -- 172.21.15.64:0/2830305088 shutdown_connections 2023-09-13T03:24:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.215+0000 7fbfe53ff700 1 -- 172.21.15.64:0/2830305088 wait complete. 2023-09-13T03:24:18.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.216+0000 7fbfe53ff700 1 Processor -- start 2023-09-13T03:24:18.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.217+0000 7fbfe53ff700 1 -- start start 2023-09-13T03:24:18.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.217+0000 7fbfe53ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfe01005b0 0x7fbfe0072ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:18.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.217+0000 7fbfe53ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfe0100f70 0x7fbfe0073020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:18.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.217+0000 7fbfe53ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfe0106670 0x7fbfe01a4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:18.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.217+0000 7fbfe53ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe01a4db0 con 0x7fbfe0100f70 2023-09-13T03:24:18.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.217+0000 7fbfe53ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe01a4ef0 con 0x7fbfe01005b0 2023-09-13T03:24:18.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.217+0000 7fbfe53ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe01a5030 con 0x7fbfe0106670 2023-09-13T03:24:18.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.217+0000 7fbfdf7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfe0106670 0x7fbfe01a4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:18.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.217+0000 7fbfdeffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfe01005b0 0x7fbfe0072ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:18.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.217+0000 7fbfdf7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfe0106670 0x7fbfe01a4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:47468/0 (socket says 172.21.15.64:47468) 2023-09-13T03:24:18.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.217+0000 7fbfdeffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfe01005b0 0x7fbfe0072ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:49578/0 (socket says 172.21.15.64:49578) 2023-09-13T03:24:18.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.217+0000 7fbfdf7fe700 1 -- 172.21.15.64:0/2899118737 learned_addr learned my addr 172.21.15.64:0/2899118737 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:24:18.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.217+0000 7fbfde7fc700 1 --2- 172.21.15.64:0/2899118737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfe0100f70 0x7fbfe0073020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:18.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.218+0000 7fbfdeffd700 1 -- 172.21.15.64:0/2899118737 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfe0106670 msgr2=0x7fbfe01a4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:18.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.218+0000 7fbfdeffd700 1 --2- 172.21.15.64:0/2899118737 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfe0106670 0x7fbfe01a4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:18.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.218+0000 7fbfdeffd700 1 -- 172.21.15.64:0/2899118737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfe0100f70 msgr2=0x7fbfe0073020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:18.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.218+0000 7fbfdeffd700 1 --2- 172.21.15.64:0/2899118737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfe0100f70 0x7fbfe0073020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:18.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.218+0000 7fbfdeffd700 1 -- 172.21.15.64:0/2899118737 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd8009c40 con 0x7fbfe01005b0 2023-09-13T03:24:18.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.218+0000 7fbfdeffd700 1 --2- 172.21.15.64:0/2899118737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfe01005b0 0x7fbfe0072ae0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbfd000e9f0 tx=0x7fbfd000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:24:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.218+0000 7fbfc7fff700 1 -- 172.21.15.64:0/2899118737 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd000c9c0 con 0x7fbfe01005b0 2023-09-13T03:24:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.218+0000 7fbfe53ff700 1 -- 172.21.15.64:0/2899118737 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfd40095f0 con 0x7fbfe01005b0 2023-09-13T03:24:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.218+0000 7fbfc7fff700 1 -- 172.21.15.64:0/2899118737 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfd0010430 con 0x7fbfe01005b0 2023-09-13T03:24:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.218+0000 7fbfc7fff700 1 -- 172.21.15.64:0/2899118737 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0018680 con 0x7fbfe01005b0 2023-09-13T03:24:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.218+0000 7fbfe53ff700 1 -- 172.21.15.64:0/2899118737 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe01a5660 con 0x7fbfe01005b0 2023-09-13T03:24:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.222+0000 7fbfc7fff700 1 -- 172.21.15.64:0/2899118737 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fbfd0010860 con 0x7fbfe01005b0 2023-09-13T03:24:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.223+0000 7fbff8c3cb80 1 Processor -- start 2023-09-13T03:24:18.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.223+0000 7fbff8c3cb80 1 -- start start 2023-09-13T03:24:18.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.223+0000 7fbfc7fff700 1 --2- 172.21.15.64:0/2899118737 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fbfc80a07b0 0x7fbfc80a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:18.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.223+0000 7fbfc7fff700 1 -- 172.21.15.64:0/2899118737 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fbfd0014070 con 0x7fbfe01005b0 2023-09-13T03:24:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.223+0000 7fbff8c3cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55945fac01a0 0x55945fac0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.223+0000 7fbff8c3cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55945faba300 0x55945faba780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.223+0000 7fbff8c3cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55945fabacc0 0x55945fac6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.223+0000 7fbff8c3cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945fa136d0 con 0x55945fac01a0 2023-09-13T03:24:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.223+0000 7fbff8c3cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945fa10ff0 con 0x55945faba300 2023-09-13T03:24:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.223+0000 7fbff8c3cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945fa0c8e0 con 0x55945fabacc0 2023-09-13T03:24:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.223+0000 7fbfdf7fe700 1 --2- 172.21.15.64:0/2899118737 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fbfc80a07b0 0x7fbfc80a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.224+0000 7fbfde7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55945faba300 0x55945faba780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.224+0000 7fbfde7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55945faba300 0x55945faba780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:49584/0 (socket says 172.21.15.64:49584) 2023-09-13T03:24:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.224+0000 7fbfdeffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55945fac01a0 0x55945fac0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.224+0000 7fbfde7fc700 1 -- 172.21.15.64:0/62566821 learned_addr learned my addr 172.21.15.64:0/62566821 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:24:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.224+0000 7fbfdf7fe700 1 --2- 172.21.15.64:0/62566821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55945fabacc0 0x55945fac6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.224+0000 7fbfde7fc700 1 -- 172.21.15.64:0/62566821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55945fabacc0 msgr2=0x55945fac6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:24:18.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.224+0000 7fbfde7fc700 1 --2- 172.21.15.64:0/62566821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55945fabacc0 0x55945fac6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:18.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.224+0000 7fbfde7fc700 1 -- 172.21.15.64:0/62566821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55945fac01a0 msgr2=0x55945fac0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:24:18.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.224+0000 7fbfde7fc700 1 --2- 172.21.15.64:0/62566821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55945fac01a0 0x55945fac0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:18.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.224+0000 7fbfde7fc700 1 -- 172.21.15.64:0/62566821 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55945f838d80 con 0x55945faba300 2023-09-13T03:24:18.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.224+0000 7fbfdeffd700 1 --2- 172.21.15.64:0/62566821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55945fac01a0 0x55945fac0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:24:18.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.224+0000 7fbfdf7fe700 1 --2- 172.21.15.64:0/2899118737 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fbfc80a07b0 0x7fbfc80a2c70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8009550 tx=0x7fbfd8007c60 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:24:18.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.224+0000 7fbfde7fc700 1 --2- 172.21.15.64:0/62566821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55945faba300 0x55945faba780 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4015000 tx=0x7fbfd4005ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cd18bda0f24a278 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:24:18.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.225+0000 7fbfc57fa700 1 -- 172.21.15.64:0/62566821 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd401d070 con 0x55945faba300 2023-09-13T03:24:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.225+0000 7fbfc57fa700 1 -- 172.21.15.64:0/62566821 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfd401e040 con 0x55945faba300 2023-09-13T03:24:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.225+0000 7fbfc57fa700 1 -- 172.21.15.64:0/62566821 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd40047f0 con 0x55945faba300 2023-09-13T03:24:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.225+0000 7fbff8c3cb80 1 -- 172.21.15.64:0/62566821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55945faba300 msgr2=0x55945faba780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.225+0000 7fbff8c3cb80 1 --2- 172.21.15.64:0/62566821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55945faba300 0x55945faba780 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4015000 tx=0x7fbfd4005ea0 comp rx=0 tx=0).stop 2023-09-13T03:24:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.226+0000 7fbff8c3cb80 1 -- 172.21.15.64:0/62566821 shutdown_connections 2023-09-13T03:24:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.226+0000 7fbff8c3cb80 1 --2- 172.21.15.64:0/62566821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55945fabacc0 0x55945fac6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.226+0000 7fbff8c3cb80 1 --2- 172.21.15.64:0/62566821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55945fac01a0 0x55945fac0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.226+0000 7fbff8c3cb80 1 --2- 172.21.15.64:0/62566821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55945faba300 0x55945faba780 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.226+0000 7fbff8c3cb80 1 -- 172.21.15.64:0/62566821 >> 172.21.15.64:0/62566821 conn(0x55945fa2f030 msgr2=0x55945fab5560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:24:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.226+0000 7fbff8c3cb80 1 -- 172.21.15.64:0/62566821 shutdown_connections 2023-09-13T03:24:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.226+0000 7fbff8c3cb80 1 -- 172.21.15.64:0/62566821 wait complete. 2023-09-13T03:24:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.226+0000 7fbff8c3cb80 1 -- start start 2023-09-13T03:24:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.226+0000 7fbff8c3cb80 1 Processor -- start 2023-09-13T03:24:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.227+0000 7fbff8c3cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55945faba300 0x55945fac6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.227+0000 7fbff8c3cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55945fabacc0 0x55945fac40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.227+0000 7fbff8c3cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55945fac01a0 0x55945fac46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.227+0000 7fbff8c3cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945f9aab10 con 0x55945fabacc0 2023-09-13T03:24:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.227+0000 7fbff8c3cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945f9aa810 con 0x55945fac01a0 2023-09-13T03:24:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.227+0000 7fbff8c3cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945f88e8c0 con 0x55945faba300 2023-09-13T03:24:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.227+0000 7fbff8c3cb80 10 client.0 _ll_register_callbacks cb 0x55945f8f4a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:24:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.227+0000 7fbfdeffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55945faba300 0x55945fac6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.227+0000 7fbfdf7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55945fac01a0 0x55945fac46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.227+0000 7fbfde7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55945fabacc0 0x55945fac40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.227+0000 7fbfdf7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55945fac01a0 0x55945fac46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:49592/0 (socket says 172.21.15.64:49592) 2023-09-13T03:24:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.227+0000 7fbfdf7fe700 1 -- 172.21.15.64:0/825429912 learned_addr learned my addr 172.21.15.64:0/825429912 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:24:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.227+0000 7fbfdeffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55945faba300 0x55945fac6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:47494/0 (socket says 172.21.15.64:47494) 2023-09-13T03:24:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.228+0000 7fbfdf7fe700 1 -- 172.21.15.64:0/825429912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55945faba300 msgr2=0x55945fac6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:24:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.228+0000 7fbfdf7fe700 1 --2- 172.21.15.64:0/825429912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55945faba300 0x55945fac6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.228+0000 7fbfdf7fe700 1 -- 172.21.15.64:0/825429912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55945fabacc0 msgr2=0x55945fac40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:24:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.228+0000 7fbfdf7fe700 1 --2- 172.21.15.64:0/825429912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55945fabacc0 0x55945fac40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.228+0000 7fbfdf7fe700 1 -- 172.21.15.64:0/825429912 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe01a5660 con 0x55945fac01a0 2023-09-13T03:24:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.228+0000 7fbfdeffd700 1 --2- 172.21.15.64:0/825429912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55945faba300 0x55945fac6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:24:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.228+0000 7fbfdf7fe700 1 --2- 172.21.15.64:0/825429912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55945fac01a0 0x55945fac46d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbfd801a6c0 tx=0x7fbfd8004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bff06585ff73bcce server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:24:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.228+0000 7fbfc2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:24:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.228+0000 7fbfc2ff5700 1 -- 172.21.15.64:0/825429912 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8007ea0 con 0x55945fac01a0 2023-09-13T03:24:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.228+0000 7fbff8c3cb80 1 -- 172.21.15.64:0/825429912 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55945f964490 con 0x55945fac01a0 2023-09-13T03:24:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.228+0000 7fbfc2ff5700 1 -- 172.21.15.64:0/825429912 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fbfd8020050 con 0x55945fac01a0 2023-09-13T03:24:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.229+0000 7fbfc2ff5700 1 -- 172.21.15.64:0/825429912 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfd8018ee0 con 0x55945fac01a0 2023-09-13T03:24:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.229+0000 7fbff8c3cb80 10 client.27020 fetch_fsmap learned FSMap version 61 2023-09-13T03:24:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.229+0000 7fbff8c3cb80 1 -- 172.21.15.64:0/825429912 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x7fbfe010dd30 con 0x55945fac01a0 2023-09-13T03:24:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.229+0000 7fbfc2ff5700 1 -- 172.21.15.64:0/825429912 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55945f964490 con 0x55945fac01a0 2023-09-13T03:24:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.229+0000 7fbfc2ff5700 1 -- 172.21.15.64:0/825429912 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 61) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fbfd8015900 con 0x55945fac01a0 2023-09-13T03:24:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.229+0000 7fbff8c3cb80 10 client.27020 fetch_fsmap finished waiting for FSMap version 61 2023-09-13T03:24:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.229+0000 7fbff8c3cb80 10 client.27020 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:24:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.229+0000 7fbff8c3cb80 20 client.27020 populate_metadata read hostname 'smithi064' 2023-09-13T03:24:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.229+0000 7fbff8c3cb80 1 --2- 172.21.15.64:0/825429912 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55945fad43b0 0x55945fad67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.229+0000 7fbff8c3cb80 4 client.27020 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:24:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.229+0000 7fbff8c3cb80 1 -- 172.21.15.64:0/825429912 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55945fa21840 con 0x55945fad43b0 2023-09-13T03:24:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.229+0000 7fbfdeffd700 1 --2- 172.21.15.64:0/825429912 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55945fad43b0 0x55945fad67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.230+0000 7fbfdeffd700 1 --2- 172.21.15.64:0/825429912 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55945fad43b0 0x55945fad67d0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71f2b7da243f7024 server_cookie=b19bbce188aec10e in_seq=0 out_seq=0 2023-09-13T03:24:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.230+0000 7fbfc2ff5700 10 client.27020 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:24:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.230+0000 7fbfc2ff5700 1 -- 172.21.15.64:0/825429912 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55945fa21840 con 0x55945fad43b0 2023-09-13T03:24:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.230+0000 7fbfc2ff5700 10 client.27020 handle_command_reply: tid=0 2023-09-13T03:24:18.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.231+0000 7fbff8c3cb80 1 client.27020 shutdown 2023-09-13T03:24:18.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.231+0000 7fbff8c3cb80 1 -- 172.21.15.64:0/825429912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55945fac01a0 msgr2=0x55945fac46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:18.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.231+0000 7fbff8c3cb80 1 --2- 172.21.15.64:0/825429912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55945fac01a0 0x55945fac46d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbfd801a6c0 tx=0x7fbfd8004b40 comp rx=0 tx=0).stop 2023-09-13T03:24:18.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.231+0000 7fbff8c3cb80 1 -- 172.21.15.64:0/825429912 shutdown_connections 2023-09-13T03:24:18.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.231+0000 7fbff8c3cb80 1 --2- 172.21.15.64:0/825429912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55945fac01a0 0x55945fac46d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:18.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.231+0000 7fbff8c3cb80 1 --2- 172.21.15.64:0/825429912 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55945fad43b0 0x55945fad67d0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:18.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.231+0000 7fbff8c3cb80 1 --2- 172.21.15.64:0/825429912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55945faba300 0x55945fac6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:18.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.231+0000 7fbff8c3cb80 1 --2- 172.21.15.64:0/825429912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55945fabacc0 0x55945fac40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:18.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.231+0000 7fbfc2ff5700 0 client.27020 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:24:18.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.231+0000 7fbff8c3cb80 1 -- 172.21.15.64:0/825429912 >> 172.21.15.64:0/825429912 conn(0x55945fa2f030 msgr2=0x55945fab47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:24:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.231+0000 7fbff8c3cb80 1 -- 172.21.15.64:0/825429912 shutdown_connections 2023-09-13T03:24:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.231+0000 7fbff8c3cb80 1 -- 172.21.15.64:0/825429912 wait complete. 2023-09-13T03:24:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.231+0000 7fbff8c3cb80 20 client.27020 trim_cache size 0 max 16384 2023-09-13T03:24:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.248+0000 7fbff8c3cb80 1 Processor -- start 2023-09-13T03:24:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.248+0000 7fbff8c3cb80 1 -- start start 2023-09-13T03:24:18.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.248+0000 7fbff8c3cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55945faba300 0x55945facb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:18.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.248+0000 7fbff8c3cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55945fabacc0 0x55945fac69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:18.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.248+0000 7fbff8c3cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55945fac01a0 0x55945fabcb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:18.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.248+0000 7fbff8c3cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945fab6b80 con 0x55945faba300 2023-09-13T03:24:18.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.248+0000 7fbff8c3cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945fad1a40 con 0x55945fac01a0 2023-09-13T03:24:18.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.248+0000 7fbff8c3cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945fac56b0 con 0x55945fabacc0 2023-09-13T03:24:18.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.248+0000 7fbfdf7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55945fac01a0 0x55945fabcb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:18.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.248+0000 7fbfde7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55945fabacc0 0x55945fac69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:18.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.248+0000 7fbfdf7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55945fac01a0 0x55945fabcb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:49600/0 (socket says 172.21.15.64:49600) 2023-09-13T03:24:18.270 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.248+0000 7fbfdf7fe700 1 -- 172.21.15.64:0/59492627 learned_addr learned my addr 172.21.15.64:0/59492627 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:24:18.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.248+0000 7fbfde7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55945fabacc0 0x55945fac69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:47508/0 (socket says 172.21.15.64:47508) 2023-09-13T03:24:18.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.248+0000 7fbfdeffd700 1 --2- 172.21.15.64:0/59492627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55945faba300 0x55945facb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:18.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.249+0000 7fbfdf7fe700 1 -- 172.21.15.64:0/59492627 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55945fabacc0 msgr2=0x55945fac69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:24:18.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.249+0000 7fbfdf7fe700 1 --2- 172.21.15.64:0/59492627 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55945fabacc0 0x55945fac69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:18.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.249+0000 7fbfdf7fe700 1 -- 172.21.15.64:0/59492627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55945faba300 msgr2=0x55945facb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:24:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.249+0000 7fbfdf7fe700 1 --2- 172.21.15.64:0/59492627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55945faba300 0x55945facb420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.249+0000 7fbfdf7fe700 1 -- 172.21.15.64:0/59492627 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55945f821c00 con 0x55945fac01a0 2023-09-13T03:24:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.249+0000 7fbfdeffd700 1 --2- 172.21.15.64:0/59492627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55945faba300 0x55945facb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:24:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.249+0000 7fbfdf7fe700 1 --2- 172.21.15.64:0/59492627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55945fac01a0 0x55945fabcb60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8009100 tx=0x7fbfd8018740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d89b559e19cd026 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:24:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.249+0000 7fbfc5ffb700 1 -- 172.21.15.64:0/59492627 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8015ed0 con 0x55945fac01a0 2023-09-13T03:24:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.249+0000 7fbfc5ffb700 1 -- 172.21.15.64:0/59492627 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfd800f040 con 0x55945fac01a0 2023-09-13T03:24:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.249+0000 7fbfc5ffb700 1 -- 172.21.15.64:0/59492627 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8015670 con 0x55945fac01a0 2023-09-13T03:24:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.250+0000 7fbff8c3cb80 1 -- 172.21.15.64:0/59492627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55945fac01a0 msgr2=0x55945fabcb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.250+0000 7fbff8c3cb80 1 --2- 172.21.15.64:0/59492627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55945fac01a0 0x55945fabcb60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8009100 tx=0x7fbfd8018740 comp rx=0 tx=0).stop 2023-09-13T03:24:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.250+0000 7fbff8c3cb80 1 -- 172.21.15.64:0/59492627 shutdown_connections 2023-09-13T03:24:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.250+0000 7fbff8c3cb80 1 --2- 172.21.15.64:0/59492627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55945fac01a0 0x55945fabcb60 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.250+0000 7fbff8c3cb80 1 --2- 172.21.15.64:0/59492627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55945faba300 0x55945facb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.250+0000 7fbff8c3cb80 1 --2- 172.21.15.64:0/59492627 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55945fabacc0 0x55945fac69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.250+0000 7fbff8c3cb80 1 -- 172.21.15.64:0/59492627 >> 172.21.15.64:0/59492627 conn(0x55945fa2f030 msgr2=0x55945fabe190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:24:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.250+0000 7fbff8c3cb80 1 -- 172.21.15.64:0/59492627 shutdown_connections 2023-09-13T03:24:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.250+0000 7fbff8c3cb80 1 -- 172.21.15.64:0/59492627 wait complete. 2023-09-13T03:24:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.250+0000 7fbff8c3cb80 1 -- start start 2023-09-13T03:24:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.251+0000 7fbff8c3cb80 1 Processor -- start 2023-09-13T03:24:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.251+0000 7fbff8c3cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55945faba300 0x55945fac3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.251+0000 7fbff8c3cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55945fabacc0 0x55945fac2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.252+0000 7fbff8c3cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55945fac01a0 0x55945fac2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.252+0000 7fbff8c3cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945fac43e0 con 0x55945fabacc0 2023-09-13T03:24:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.252+0000 7fbff8c3cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945fac32f0 con 0x55945faba300 2023-09-13T03:24:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.252+0000 7fbff8c3cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945fac34b0 con 0x55945fac01a0 2023-09-13T03:24:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.252+0000 7fbff8c3cb80 10 client.0 _ll_register_callbacks cb 0x7fbfd8018b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:24:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.252+0000 7fbfdeffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55945faba300 0x55945fac3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.252+0000 7fbfdeffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55945faba300 0x55945fac3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:49604/0 (socket says 172.21.15.64:49604) 2023-09-13T03:24:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.252+0000 7fbfdeffd700 1 -- 172.21.15.64:0/3944838581 learned_addr learned my addr 172.21.15.64:0/3944838581 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:24:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.252+0000 7fbfdf7fe700 1 --2- 172.21.15.64:0/3944838581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55945fac01a0 0x55945fac2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.252+0000 7fbfde7fc700 1 --2- 172.21.15.64:0/3944838581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55945fabacc0 0x55945fac2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.252+0000 7fbfdf7fe700 1 -- 172.21.15.64:0/3944838581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55945faba300 msgr2=0x55945fac3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:24:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.252+0000 7fbfdf7fe700 1 --2- 172.21.15.64:0/3944838581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55945faba300 0x55945fac3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.252+0000 7fbfdf7fe700 1 -- 172.21.15.64:0/3944838581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55945fabacc0 msgr2=0x55945fac2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:24:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.252+0000 7fbfdf7fe700 1 --2- 172.21.15.64:0/3944838581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55945fabacc0 0x55945fac2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.252+0000 7fbfdf7fe700 1 -- 172.21.15.64:0/3944838581 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe010dd30 con 0x55945fac01a0 2023-09-13T03:24:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.252+0000 7fbfdeffd700 1 --2- 172.21.15.64:0/3944838581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55945faba300 0x55945fac3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:24:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.252+0000 7fbfde7fc700 1 --2- 172.21.15.64:0/3944838581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55945fabacc0 0x55945fac2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:24:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.252+0000 7fbfdf7fe700 1 --2- 172.21.15.64:0/3944838581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55945fac01a0 0x55945fac2db0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8015c60 tx=0x7fbfd8009ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9284e82a5d685fc server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:24:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.252+0000 7fbfc2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:24:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.253+0000 7fbfc2ff5700 1 -- 172.21.15.64:0/3944838581 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd80075e0 con 0x55945fac01a0 2023-09-13T03:24:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.253+0000 7fbff8c3cb80 1 -- 172.21.15.64:0/3944838581 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbfd401e040 con 0x55945fac01a0 2023-09-13T03:24:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.253+0000 7fbfc2ff5700 1 -- 172.21.15.64:0/3944838581 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fbfd8020040 con 0x55945fac01a0 2023-09-13T03:24:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.253+0000 7fbfc2ff5700 1 -- 172.21.15.64:0/3944838581 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfd8025a80 con 0x55945fac01a0 2023-09-13T03:24:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.253+0000 7fbff8c3cb80 10 client.27010 fetch_fsmap learned FSMap version 61 2023-09-13T03:24:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.253+0000 7fbff8c3cb80 1 -- 172.21.15.64:0/3944838581 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x55945f85f1f0 con 0x55945fac01a0 2023-09-13T03:24:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.253+0000 7fbfc2ff5700 1 -- 172.21.15.64:0/3944838581 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfd8050020 con 0x55945fac01a0 2023-09-13T03:24:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.254+0000 7fbfc2ff5700 1 -- 172.21.15.64:0/3944838581 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 61) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fbfd8037670 con 0x55945fac01a0 2023-09-13T03:24:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.254+0000 7fbff8c3cb80 10 client.27010 fetch_fsmap finished waiting for FSMap version 61 2023-09-13T03:24:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.254+0000 7fbff8c3cb80 10 client.27010 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:24:18.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.254+0000 7fbff8c3cb80 20 client.27010 populate_metadata read hostname 'smithi064' 2023-09-13T03:24:18.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.254+0000 7fbff8c3cb80 1 --2- 172.21.15.64:0/3944838581 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55945fb28f00 0x55945fb2b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:18.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.254+0000 7fbff8c3cb80 4 client.27010 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:24:18.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.254+0000 7fbff8c3cb80 1 -- 172.21.15.64:0/3944838581 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55945fa058c0 con 0x55945fb28f00 2023-09-13T03:24:18.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.254+0000 7fbfdeffd700 1 --2- 172.21.15.64:0/3944838581 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55945fb28f00 0x55945fb2b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.254+0000 7fbfdeffd700 1 --2- 172.21.15.64:0/3944838581 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55945fb28f00 0x55945fb2b2f0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc4f9b9aaba61a28 server_cookie=4e2f7ab2658425e5 in_seq=0 out_seq=0 2023-09-13T03:24:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.254+0000 7fbfc2ff5700 10 client.27010 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:24:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.288+0000 7fbfc2ff5700 1 -- 172.21.15.64:0/3944838581 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55945fa058c0 con 0x55945fb28f00 2023-09-13T03:24:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.288+0000 7fbfc2ff5700 10 client.27010 handle_command_reply: tid=0 2023-09-13T03:24:18.290 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.288+0000 7fbff8c3cb80 1 client.27010 shutdown 2023-09-13T03:24:18.290 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.288+0000 7fbff8c3cb80 1 -- 172.21.15.64:0/3944838581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55945fac01a0 msgr2=0x55945fac2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:18.290 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.288+0000 7fbff8c3cb80 1 --2- 172.21.15.64:0/3944838581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55945fac01a0 0x55945fac2db0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8015c60 tx=0x7fbfd8009ad0 comp rx=0 tx=0).stop 2023-09-13T03:24:18.290 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.288+0000 7fbff8c3cb80 1 -- 172.21.15.64:0/3944838581 shutdown_connections 2023-09-13T03:24:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.288+0000 7fbff8c3cb80 1 --2- 172.21.15.64:0/3944838581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55945faba300 0x55945fac3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.288+0000 7fbff8c3cb80 1 --2- 172.21.15.64:0/3944838581 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55945fb28f00 0x55945fb2b2f0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.288+0000 7fbff8c3cb80 1 --2- 172.21.15.64:0/3944838581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55945fac01a0 0x55945fac2db0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.288+0000 7fbff8c3cb80 1 --2- 172.21.15.64:0/3944838581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55945fabacc0 0x55945fac2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.288+0000 7fbfc2ff5700 0 client.27010 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:24:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.288+0000 7fbff8c3cb80 1 -- 172.21.15.64:0/3944838581 >> 172.21.15.64:0/3944838581 conn(0x55945fa2f030 msgr2=0x55945fab5090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:24:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.289+0000 7fbff8c3cb80 1 -- 172.21.15.64:0/3944838581 shutdown_connections 2023-09-13T03:24:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.289+0000 7fbff8c3cb80 1 -- 172.21.15.64:0/3944838581 wait complete. 2023-09-13T03:24:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.289+0000 7fbff8c3cb80 20 client.27010 trim_cache size 0 max 16384 2023-09-13T03:24:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:24:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:24:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:24:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:24:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.289+0000 7fbfe53ff700 1 -- 172.21.15.64:0/2899118737 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fbfc80a07b0 msgr2=0x7fbfc80a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.289+0000 7fbfe53ff700 1 --2- 172.21.15.64:0/2899118737 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fbfc80a07b0 0x7fbfc80a2c70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8009550 tx=0x7fbfd8007c60 comp rx=0 tx=0).stop 2023-09-13T03:24:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.289+0000 7fbfe53ff700 1 -- 172.21.15.64:0/2899118737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfe01005b0 msgr2=0x7fbfe0072ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.289+0000 7fbfe53ff700 1 --2- 172.21.15.64:0/2899118737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfe01005b0 0x7fbfe0072ae0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbfd000e9f0 tx=0x7fbfd000ed00 comp rx=0 tx=0).stop 2023-09-13T03:24:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.289+0000 7fbfe53ff700 1 -- 172.21.15.64:0/2899118737 shutdown_connections 2023-09-13T03:24:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.289+0000 7fbfe53ff700 1 --2- 172.21.15.64:0/2899118737 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fbfc80a07b0 0x7fbfc80a2c70 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.289+0000 7fbfe53ff700 1 --2- 172.21.15.64:0/2899118737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfe01005b0 0x7fbfe0072ae0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.289+0000 7fbfe53ff700 1 --2- 172.21.15.64:0/2899118737 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfe0106670 0x7fbfe01a4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.289+0000 7fbfe53ff700 1 --2- 172.21.15.64:0/2899118737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfe0100f70 0x7fbfe0073020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.289+0000 7fbfe53ff700 1 -- 172.21.15.64:0/2899118737 >> 172.21.15.64:0/2899118737 conn(0x7fbfe00fc060 msgr2=0x7fbfe00fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:24:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.290+0000 7fbfe53ff700 1 -- 172.21.15.64:0/2899118737 shutdown_connections 2023-09-13T03:24:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:18.290+0000 7fbfe53ff700 1 -- 172.21.15.64:0/2899118737 wait complete. 2023-09-13T03:24:18.949 INFO:tasks.workunit.client.0.smithi064.stdout: 1 254438 2.14 MB/sec execute 503 sec latency 29.660 ms 2023-09-13T03:24:19.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:24:18 smithi170 ceph-mon[134328]: pgmap v347: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 170 op/s 2023-09-13T03:24:19.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:24:18 smithi064 ceph-mon[97675]: pgmap v347: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 170 op/s 2023-09-13T03:24:19.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:24:18 smithi142 ceph-mon[134082]: pgmap v347: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 170 op/s 2023-09-13T03:24:19.949 INFO:tasks.workunit.client.0.smithi064.stdout: 1 255857 2.16 MB/sec execute 504 sec latency 11.894 ms 2023-09-13T03:24:20.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:24:20 smithi064 ceph-mon[97675]: pgmap v348: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 997 KiB/s wr, 137 op/s 2023-09-13T03:24:20.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:24:20 smithi142 ceph-mon[134082]: pgmap v348: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 997 KiB/s wr, 137 op/s 2023-09-13T03:24:20.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:24:20 smithi170 ceph-mon[134328]: pgmap v348: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 997 KiB/s wr, 137 op/s 2023-09-13T03:24:20.949 INFO:tasks.workunit.client.0.smithi064.stdout: 1 257107 2.17 MB/sec execute 505 sec latency 28.031 ms 2023-09-13T03:24:21.949 INFO:tasks.workunit.client.0.smithi064.stdout: 1 259177 2.19 MB/sec execute 506 sec latency 16.672 ms 2023-09-13T03:24:22.950 INFO:tasks.workunit.client.0.smithi064.stdout: 1 260369 2.20 MB/sec execute 507 sec latency 22.944 ms 2023-09-13T03:24:22.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:24:22 smithi142 ceph-mon[134082]: pgmap v349: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 180 op/s 2023-09-13T03:24:23.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:24:22 smithi170 ceph-mon[134328]: pgmap v349: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 180 op/s 2023-09-13T03:24:23.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:24:22 smithi064 ceph-mon[97675]: pgmap v349: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 180 op/s 2023-09-13T03:24:23.949 INFO:tasks.workunit.client.0.smithi064.stdout: 1 261410 2.20 MB/sec execute 508 sec latency 18.307 ms 2023-09-13T03:24:24.949 INFO:tasks.workunit.client.0.smithi064.stdout: 1 262910 2.22 MB/sec execute 509 sec latency 11.109 ms 2023-09-13T03:24:25.010 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:24:24 smithi170 ceph-mon[134328]: pgmap v350: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 180 op/s 2023-09-13T03:24:25.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:24:24 smithi064 ceph-mon[97675]: pgmap v350: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 180 op/s 2023-09-13T03:24:25.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:24:24 smithi142 ceph-mon[134082]: pgmap v350: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 180 op/s 2023-09-13T03:24:25.950 INFO:tasks.workunit.client.0.smithi064.stdout: 1 264018 2.22 MB/sec execute 510 sec latency 35.170 ms 2023-09-13T03:24:26.950 INFO:tasks.workunit.client.0.smithi064.stdout: 1 265874 2.24 MB/sec execute 511 sec latency 27.958 ms 2023-09-13T03:24:27.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:24:26 smithi170 ceph-mon[134328]: pgmap v351: 97 pgs: 97 active+clean; 118 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 249 op/s 2023-09-13T03:24:27.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:24:26 smithi064 ceph-mon[97675]: pgmap v351: 97 pgs: 97 active+clean; 118 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 249 op/s 2023-09-13T03:24:27.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:24:26 smithi142 ceph-mon[134082]: pgmap v351: 97 pgs: 97 active+clean; 118 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 249 op/s 2023-09-13T03:24:27.950 INFO:tasks.workunit.client.0.smithi064.stdout: 1 266935 2.24 MB/sec execute 512 sec latency 20.156 ms 2023-09-13T03:24:28.950 INFO:tasks.workunit.client.0.smithi064.stdout: 1 267979 2.25 MB/sec execute 513 sec latency 18.186 ms 2023-09-13T03:24:29.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:24:28 smithi170 ceph-mon[134328]: pgmap v352: 97 pgs: 97 active+clean; 118 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 185 op/s 2023-09-13T03:24:29.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:24:28 smithi064 ceph-mon[97675]: pgmap v352: 97 pgs: 97 active+clean; 118 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 185 op/s 2023-09-13T03:24:29.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:24:28 smithi142 ceph-mon[134082]: pgmap v352: 97 pgs: 97 active+clean; 118 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 185 op/s 2023-09-13T03:24:29.950 INFO:tasks.workunit.client.0.smithi064.stdout: 1 269847 2.27 MB/sec execute 514 sec latency 14.678 ms 2023-09-13T03:24:30.668 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:24:30 smithi064 ceph-mon[97675]: pgmap v353: 97 pgs: 97 active+clean; 118 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 185 op/s 2023-09-13T03:24:30.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:24:30 smithi142 ceph-mon[134082]: pgmap v353: 97 pgs: 97 active+clean; 118 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 185 op/s 2023-09-13T03:24:30.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:24:30 smithi170 ceph-mon[134328]: pgmap v353: 97 pgs: 97 active+clean; 118 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 185 op/s 2023-09-13T03:24:30.950 INFO:tasks.workunit.client.0.smithi064.stdout: 1 271005 2.27 MB/sec execute 515 sec latency 13.494 ms 2023-09-13T03:24:31.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.451+0000 7f4cd8dff700 1 -- 172.21.15.64:0/183734091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cd41087b0 msgr2=0x7f4cd4108b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:31.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.451+0000 7f4cd8dff700 1 --2- 172.21.15.64:0/183734091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cd41087b0 0x7f4cd4108b90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4009990 tx=0x7f4cc4009ca0 comp rx=0 tx=0).stop 2023-09-13T03:24:31.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.451+0000 7f4cd8dff700 1 -- 172.21.15.64:0/183734091 shutdown_connections 2023-09-13T03:24:31.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.451+0000 7f4cd8dff700 1 --2- 172.21.15.64:0/183734091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cd4103160 0x7f4cd410f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:31.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.451+0000 7f4cd8dff700 1 --2- 172.21.15.64:0/183734091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cd41087b0 0x7f4cd4108b90 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:31.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.451+0000 7f4cd8dff700 1 --2- 172.21.15.64:0/183734091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cd41027a0 0x7f4cd4102c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:31.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.451+0000 7f4cd8dff700 1 -- 172.21.15.64:0/183734091 >> 172.21.15.64:0/183734091 conn(0x7f4cd40fe250 msgr2=0x7f4cd4100670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:24:31.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.451+0000 7f4cd8dff700 1 -- 172.21.15.64:0/183734091 shutdown_connections 2023-09-13T03:24:31.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.451+0000 7f4cd8dff700 1 -- 172.21.15.64:0/183734091 wait complete. 2023-09-13T03:24:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.452+0000 7f4cd8dff700 1 Processor -- start 2023-09-13T03:24:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.452+0000 7f4cd8dff700 1 -- start start 2023-09-13T03:24:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.453+0000 7f4cd8dff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cd41027a0 0x7f4cd419e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.453+0000 7f4cd8dff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cd4103160 0x7f4cd419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.453+0000 7f4cd8dff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cd41087b0 0x7f4cd4198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.453+0000 7f4cd8dff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd4198cf0 con 0x7f4cd41087b0 2023-09-13T03:24:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.453+0000 7f4cd8dff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd4198e60 con 0x7f4cd41027a0 2023-09-13T03:24:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.453+0000 7f4cd8dff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd4199020 con 0x7f4cd4103160 2023-09-13T03:24:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.453+0000 7f4cd259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cd41027a0 0x7f4cd419e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.453+0000 7f4cd1d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cd4103160 0x7f4cd419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.453+0000 7f4cd259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cd41027a0 0x7f4cd419e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:37798/0 (socket says 172.21.15.64:37798) 2023-09-13T03:24:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.453+0000 7f4cd259c700 1 -- 172.21.15.64:0/1014070416 learned_addr learned my addr 172.21.15.64:0/1014070416 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:24:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.453+0000 7f4cd2d9d700 1 --2- 172.21.15.64:0/1014070416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cd41087b0 0x7f4cd4198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.453+0000 7f4cd259c700 1 -- 172.21.15.64:0/1014070416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cd4103160 msgr2=0x7f4cd419ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.454+0000 7f4cd259c700 1 --2- 172.21.15.64:0/1014070416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cd4103160 0x7f4cd419ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.454+0000 7f4cd259c700 1 -- 172.21.15.64:0/1014070416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cd41087b0 msgr2=0x7f4cd4198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.454+0000 7f4cd259c700 1 --2- 172.21.15.64:0/1014070416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cd41087b0 0x7f4cd4198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.454+0000 7f4cd259c700 1 -- 172.21.15.64:0/1014070416 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc8009520 con 0x7f4cd41027a0 2023-09-13T03:24:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.454+0000 7f4cd1d9b700 1 --2- 172.21.15.64:0/1014070416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cd4103160 0x7f4cd419ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:24:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.454+0000 7f4cd259c700 1 --2- 172.21.15.64:0/1014070416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cd41027a0 0x7f4cd419e820 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4000c00 tx=0x7f4cc4004790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:24:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.454+0000 7f4cc37fe700 1 -- 172.21.15.64:0/1014070416 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc401d070 con 0x7f4cd41027a0 2023-09-13T03:24:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.454+0000 7f4cd8dff700 1 -- 172.21.15.64:0/1014070416 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cc40095f0 con 0x7f4cd41027a0 2023-09-13T03:24:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.454+0000 7f4cc37fe700 1 -- 172.21.15.64:0/1014070416 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cc4021410 con 0x7f4cd41027a0 2023-09-13T03:24:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.454+0000 7f4cc37fe700 1 -- 172.21.15.64:0/1014070416 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc4017650 con 0x7f4cd41027a0 2023-09-13T03:24:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.454+0000 7f4cd8dff700 1 -- 172.21.15.64:0/1014070416 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cd41a5630 con 0x7f4cd41027a0 2023-09-13T03:24:31.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.458+0000 7f4cc37fe700 1 -- 172.21.15.64:0/1014070416 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f4cc4021580 con 0x7f4cd41027a0 2023-09-13T03:24:31.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.459+0000 7f4cec682b80 1 Processor -- start 2023-09-13T03:24:31.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.459+0000 7f4cec682b80 1 -- start start 2023-09-13T03:24:31.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.459+0000 7f4cc37fe700 1 --2- 172.21.15.64:0/1014070416 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f4cbc0a07b0 0x7f4cbc0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.460+0000 7f4cc37fe700 1 -- 172.21.15.64:0/1014070416 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f4cc40cd2b0 con 0x7f4cd41027a0 2023-09-13T03:24:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.460+0000 7f4cd2d9d700 1 --2- 172.21.15.64:0/1014070416 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f4cbc0a07b0 0x7f4cbc0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.460+0000 7f4cec682b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a230cf01a0 0x55a230cf0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.460+0000 7f4cec682b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a230cea300 0x55a230cea780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.460+0000 7f4cec682b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a230ceacc0 0x55a230cf6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.460+0000 7f4cec682b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a230c436d0 con 0x55a230cf01a0 2023-09-13T03:24:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.460+0000 7f4cec682b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a230c40ff0 con 0x55a230cea300 2023-09-13T03:24:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.460+0000 7f4cec682b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a230c3c8e0 con 0x55a230ceacc0 2023-09-13T03:24:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.460+0000 7f4cd2d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a230ceacc0 0x55a230cf6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.460+0000 7f4cd1d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a230cea300 0x55a230cea780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:31.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.460+0000 7f4cd1d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a230cea300 0x55a230cea780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:37802/0 (socket says 172.21.15.64:37802) 2023-09-13T03:24:31.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.460+0000 7f4cd1d9b700 1 -- 172.21.15.64:0/741642588 learned_addr learned my addr 172.21.15.64:0/741642588 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:24:31.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.460+0000 7f4cd259c700 1 --2- 172.21.15.64:0/741642588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a230cf01a0 0x55a230cf0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:31.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.461+0000 7f4cd1d9b700 1 -- 172.21.15.64:0/741642588 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a230ceacc0 msgr2=0x55a230cf6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:24:31.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.461+0000 7f4cd1d9b700 1 --2- 172.21.15.64:0/741642588 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a230ceacc0 0x55a230cf6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:31.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.461+0000 7f4cd1d9b700 1 -- 172.21.15.64:0/741642588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a230cf01a0 msgr2=0x55a230cf0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:24:31.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.461+0000 7f4cd1d9b700 1 --2- 172.21.15.64:0/741642588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a230cf01a0 0x55a230cf0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:31.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.461+0000 7f4cd1d9b700 1 -- 172.21.15.64:0/741642588 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a230a68d80 con 0x55a230cea300 2023-09-13T03:24:31.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.461+0000 7f4cd259c700 1 --2- 172.21.15.64:0/741642588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a230cf01a0 0x55a230cf0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:24:31.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.461+0000 7f4cd2d9d700 1 --2- 172.21.15.64:0/1014070416 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f4cbc0a07b0 0x7f4cbc0a2c70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4cd419f520 tx=0x7f4ccc013460 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:24:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.461+0000 7f4cd1d9b700 1 --2- 172.21.15.64:0/741642588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a230cea300 0x55a230cea780 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8011810 tx=0x7f4cc8011b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2db10315c9bfd887 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:24:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.461+0000 7f4cc0ff9700 1 -- 172.21.15.64:0/741642588 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc800f480 con 0x55a230cea300 2023-09-13T03:24:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.461+0000 7f4cc0ff9700 1 -- 172.21.15.64:0/741642588 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cc800faa0 con 0x55a230cea300 2023-09-13T03:24:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.461+0000 7f4cc0ff9700 1 -- 172.21.15.64:0/741642588 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8019a30 con 0x55a230cea300 2023-09-13T03:24:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.462+0000 7f4cec682b80 1 -- 172.21.15.64:0/741642588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a230cea300 msgr2=0x55a230cea780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.462+0000 7f4cec682b80 1 --2- 172.21.15.64:0/741642588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a230cea300 0x55a230cea780 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8011810 tx=0x7f4cc8011b20 comp rx=0 tx=0).stop 2023-09-13T03:24:31.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.462+0000 7f4cec682b80 1 -- 172.21.15.64:0/741642588 shutdown_connections 2023-09-13T03:24:31.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.462+0000 7f4cec682b80 1 --2- 172.21.15.64:0/741642588 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a230ceacc0 0x55a230cf6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:31.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.462+0000 7f4cec682b80 1 --2- 172.21.15.64:0/741642588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a230cf01a0 0x55a230cf0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:31.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.462+0000 7f4cec682b80 1 --2- 172.21.15.64:0/741642588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a230cea300 0x55a230cea780 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:31.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.462+0000 7f4cec682b80 1 -- 172.21.15.64:0/741642588 >> 172.21.15.64:0/741642588 conn(0x55a230c5f030 msgr2=0x55a230ce5560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:24:31.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.462+0000 7f4cec682b80 1 -- 172.21.15.64:0/741642588 shutdown_connections 2023-09-13T03:24:31.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.462+0000 7f4cec682b80 1 -- 172.21.15.64:0/741642588 wait complete. 2023-09-13T03:24:31.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.463+0000 7f4cec682b80 1 -- start start 2023-09-13T03:24:31.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.463+0000 7f4cec682b80 1 Processor -- start 2023-09-13T03:24:31.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.464+0000 7f4cec682b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a230cea300 0x55a230cf6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:31.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.464+0000 7f4cec682b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a230ceacc0 0x55a230cf40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:31.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.464+0000 7f4cec682b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a230cf01a0 0x55a230cf4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:31.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.464+0000 7f4cec682b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a230bdab10 con 0x55a230ceacc0 2023-09-13T03:24:31.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.464+0000 7f4cec682b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a230bda810 con 0x55a230cf01a0 2023-09-13T03:24:31.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.464+0000 7f4cec682b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a230abe8c0 con 0x55a230cea300 2023-09-13T03:24:31.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.464+0000 7f4cec682b80 10 client.0 _ll_register_callbacks cb 0x55a230b24a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:24:31.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.464+0000 7f4cd259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a230cea300 0x55a230cf6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:31.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.464+0000 7f4cd2d9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a230cf01a0 0x55a230cf4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:31.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.464+0000 7f4cd259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a230cea300 0x55a230cf6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:42006/0 (socket says 172.21.15.64:42006) 2023-09-13T03:24:31.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.464+0000 7f4cd259c700 1 -- 172.21.15.64:0/4088149511 learned_addr learned my addr 172.21.15.64:0/4088149511 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:24:31.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.464+0000 7f4cd1d9b700 1 --2- 172.21.15.64:0/4088149511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a230ceacc0 0x55a230cf40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.465+0000 7f4cd259c700 1 -- 172.21.15.64:0/4088149511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a230cf01a0 msgr2=0x55a230cf4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:24:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.465+0000 7f4cd259c700 1 --2- 172.21.15.64:0/4088149511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a230cf01a0 0x55a230cf4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.465+0000 7f4cd259c700 1 -- 172.21.15.64:0/4088149511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a230ceacc0 msgr2=0x55a230cf40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:24:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.465+0000 7f4cd259c700 1 --2- 172.21.15.64:0/4088149511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a230ceacc0 0x55a230cf40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.465+0000 7f4cd259c700 1 -- 172.21.15.64:0/4088149511 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cd41a5630 con 0x55a230cea300 2023-09-13T03:24:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.465+0000 7f4cd2d9d700 1 --2- 172.21.15.64:0/4088149511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a230cf01a0 0x55a230cf4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:24:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.465+0000 7f4cd1d9b700 1 --2- 172.21.15.64:0/4088149511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a230ceacc0 0x55a230cf40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:24:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.465+0000 7f4cd259c700 1 --2- 172.21.15.64:0/4088149511 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a230cea300 0x55a230cf6d80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4097220 tx=0x7f4cc409a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2d93eb03de2e31b server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:24:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.465+0000 7f4cb9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:24:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.465+0000 7f4cb9ffb700 1 -- 172.21.15.64:0/4088149511 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc409a7b0 con 0x55a230cea300 2023-09-13T03:24:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.465+0000 7f4cb9ffb700 1 -- 172.21.15.64:0/4088149511 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f4cc409e5e0 con 0x55a230cea300 2023-09-13T03:24:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.465+0000 7f4cec682b80 1 -- 172.21.15.64:0/4088149511 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a230b94490 con 0x55a230cea300 2023-09-13T03:24:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.465+0000 7f4cb9ffb700 1 -- 172.21.15.64:0/4088149511 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4cc40a2790 con 0x55a230cea300 2023-09-13T03:24:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.465+0000 7f4cec682b80 10 client.27022 fetch_fsmap learned FSMap version 61 2023-09-13T03:24:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.465+0000 7f4cec682b80 1 -- 172.21.15.64:0/4088149511 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x7f4cc8011440 con 0x55a230cea300 2023-09-13T03:24:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.466+0000 7f4cb9ffb700 1 -- 172.21.15.64:0/4088149511 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a230b94490 con 0x55a230cea300 2023-09-13T03:24:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.466+0000 7f4cb9ffb700 1 -- 172.21.15.64:0/4088149511 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 61) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f4cc40b2d20 con 0x55a230cea300 2023-09-13T03:24:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.466+0000 7f4cec682b80 10 client.27022 fetch_fsmap finished waiting for FSMap version 61 2023-09-13T03:24:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.466+0000 7f4cec682b80 10 client.27022 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:24:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.466+0000 7f4cec682b80 20 client.27022 populate_metadata read hostname 'smithi064' 2023-09-13T03:24:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.466+0000 7f4cec682b80 1 --2- 172.21.15.64:0/4088149511 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55a230d04040 0x55a230d06430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.466+0000 7f4cec682b80 4 client.27022 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:24:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.466+0000 7f4cec682b80 1 -- 172.21.15.64:0/4088149511 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a230c51840 con 0x55a230d04040 2023-09-13T03:24:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.466+0000 7f4cd1d9b700 1 --2- 172.21.15.64:0/4088149511 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55a230d04040 0x55a230d06430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.467+0000 7f4cd1d9b700 1 --2- 172.21.15.64:0/4088149511 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55a230d04040 0x55a230d06430 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e116658f0675415b server_cookie=ecb967d74dc099b in_seq=0 out_seq=0 2023-09-13T03:24:31.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.467+0000 7f4cb9ffb700 10 client.27022 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:24:31.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.468+0000 7f4cb9ffb700 1 -- 172.21.15.64:0/4088149511 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a230c51840 con 0x55a230d04040 2023-09-13T03:24:31.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.468+0000 7f4cb9ffb700 10 client.27022 handle_command_reply: tid=0 2023-09-13T03:24:31.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.468+0000 7f4cec682b80 1 client.27022 shutdown 2023-09-13T03:24:31.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.468+0000 7f4cec682b80 1 -- 172.21.15.64:0/4088149511 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a230cea300 msgr2=0x55a230cf6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:31.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.468+0000 7f4cec682b80 1 --2- 172.21.15.64:0/4088149511 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a230cea300 0x55a230cf6d80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4097220 tx=0x7f4cc409a110 comp rx=0 tx=0).stop 2023-09-13T03:24:31.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.468+0000 7f4cec682b80 1 -- 172.21.15.64:0/4088149511 shutdown_connections 2023-09-13T03:24:31.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.468+0000 7f4cec682b80 1 --2- 172.21.15.64:0/4088149511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a230cf01a0 0x55a230cf4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:31.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.468+0000 7f4cec682b80 1 --2- 172.21.15.64:0/4088149511 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55a230d04040 0x55a230d06430 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:31.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.468+0000 7f4cec682b80 1 --2- 172.21.15.64:0/4088149511 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a230cea300 0x55a230cf6d80 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:31.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.468+0000 7f4cec682b80 1 --2- 172.21.15.64:0/4088149511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a230ceacc0 0x55a230cf40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.468+0000 7f4cec682b80 1 -- 172.21.15.64:0/4088149511 >> 172.21.15.64:0/4088149511 conn(0x55a230c5f030 msgr2=0x55a230ce47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:24:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.468+0000 7f4cb9ffb700 0 client.27022 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:24:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.468+0000 7f4cec682b80 1 -- 172.21.15.64:0/4088149511 shutdown_connections 2023-09-13T03:24:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.468+0000 7f4cec682b80 1 -- 172.21.15.64:0/4088149511 wait complete. 2023-09-13T03:24:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.468+0000 7f4cec682b80 20 client.27022 trim_cache size 0 max 16384 2023-09-13T03:24:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.485+0000 7f4cec682b80 1 Processor -- start 2023-09-13T03:24:31.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.485+0000 7f4cec682b80 1 -- start start 2023-09-13T03:24:31.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.485+0000 7f4cec682b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a230cea300 0x55a230cfb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:31.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.485+0000 7f4cec682b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a230ceacc0 0x55a230cf68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:31.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.485+0000 7f4cec682b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a230cf01a0 0x55a230cecb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:31.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.485+0000 7f4cec682b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a230cf5500 con 0x55a230cea300 2023-09-13T03:24:31.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.485+0000 7f4cec682b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a230ce6b40 con 0x55a230ceacc0 2023-09-13T03:24:31.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.485+0000 7f4cec682b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a230c51840 con 0x55a230cf01a0 2023-09-13T03:24:31.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.486+0000 7f4cd259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a230cea300 0x55a230cfb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:31.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.486+0000 7f4cd1d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a230ceacc0 0x55a230cf68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.486+0000 7f4cd2d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a230cf01a0 0x55a230cecb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.486+0000 7f4cd1d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a230ceacc0 0x55a230cf68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:37814/0 (socket says 172.21.15.64:37814) 2023-09-13T03:24:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.486+0000 7f4cd1d9b700 1 -- 172.21.15.64:0/1806554749 learned_addr learned my addr 172.21.15.64:0/1806554749 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:24:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.486+0000 7f4cd259c700 1 -- 172.21.15.64:0/1806554749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a230cf01a0 msgr2=0x55a230cecb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:24:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.486+0000 7f4cd259c700 1 --2- 172.21.15.64:0/1806554749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a230cf01a0 0x55a230cecb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.486+0000 7f4cd259c700 1 -- 172.21.15.64:0/1806554749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a230ceacc0 msgr2=0x55a230cf68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:24:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.486+0000 7f4cd259c700 1 --2- 172.21.15.64:0/1806554749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a230ceacc0 0x55a230cf68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.486+0000 7f4cd259c700 1 -- 172.21.15.64:0/1806554749 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a230a51c00 con 0x55a230cea300 2023-09-13T03:24:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.486+0000 7f4cd1d9b700 1 --2- 172.21.15.64:0/1806554749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a230ceacc0 0x55a230cf68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:24:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.486+0000 7f4cd259c700 1 --2- 172.21.15.64:0/1806554749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a230cea300 0x55a230cfb520 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f4cc409a4b0 tx=0x7f4cc4021c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84fe82daae89f69f server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:24:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.487+0000 7f4cc17fa700 1 -- 172.21.15.64:0/1806554749 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc40a2420 con 0x55a230cea300 2023-09-13T03:24:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.487+0000 7f4cc17fa700 1 -- 172.21.15.64:0/1806554749 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cc40b1070 con 0x55a230cea300 2023-09-13T03:24:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.487+0000 7f4cc17fa700 1 -- 172.21.15.64:0/1806554749 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc409ac70 con 0x55a230cea300 2023-09-13T03:24:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.487+0000 7f4cec682b80 1 -- 172.21.15.64:0/1806554749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a230cea300 msgr2=0x55a230cfb520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.487+0000 7f4cec682b80 1 --2- 172.21.15.64:0/1806554749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a230cea300 0x55a230cfb520 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f4cc409a4b0 tx=0x7f4cc4021c00 comp rx=0 tx=0).stop 2023-09-13T03:24:31.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.487+0000 7f4cec682b80 1 -- 172.21.15.64:0/1806554749 shutdown_connections 2023-09-13T03:24:31.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.487+0000 7f4cec682b80 1 --2- 172.21.15.64:0/1806554749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a230cf01a0 0x55a230cecb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:31.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.487+0000 7f4cec682b80 1 --2- 172.21.15.64:0/1806554749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a230cea300 0x55a230cfb520 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:31.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.487+0000 7f4cec682b80 1 --2- 172.21.15.64:0/1806554749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a230ceacc0 0x55a230cf68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:31.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.487+0000 7f4cec682b80 1 -- 172.21.15.64:0/1806554749 >> 172.21.15.64:0/1806554749 conn(0x55a230c5f030 msgr2=0x55a230cee250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:24:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.487+0000 7f4cec682b80 1 -- 172.21.15.64:0/1806554749 shutdown_connections 2023-09-13T03:24:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.488+0000 7f4cec682b80 1 -- 172.21.15.64:0/1806554749 wait complete. 2023-09-13T03:24:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.488+0000 7f4cec682b80 1 -- start start 2023-09-13T03:24:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.488+0000 7f4cec682b80 1 Processor -- start 2023-09-13T03:24:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.489+0000 7f4cec682b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a230cea300 0x55a230cfb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.489+0000 7f4cec682b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a230ceacc0 0x55a230cf2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.489+0000 7f4cec682b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a230cf01a0 0x55a230cf2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.489+0000 7f4cec682b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a230cf34a0 con 0x55a230ceacc0 2023-09-13T03:24:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.489+0000 7f4cec682b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a230cf3610 con 0x55a230cf01a0 2023-09-13T03:24:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.489+0000 7f4cec682b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a230cf37d0 con 0x55a230cea300 2023-09-13T03:24:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.489+0000 7f4cec682b80 10 client.0 _ll_register_callbacks cb 0x7f4cc4096ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:24:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.489+0000 7f4cd259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a230cea300 0x55a230cfb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.489+0000 7f4cd1d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a230ceacc0 0x55a230cf2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.489+0000 7f4cd259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a230cea300 0x55a230cfb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:42022/0 (socket says 172.21.15.64:42022) 2023-09-13T03:24:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.489+0000 7f4cd259c700 1 -- 172.21.15.64:0/1867137898 learned_addr learned my addr 172.21.15.64:0/1867137898 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:24:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.489+0000 7f4cd2d9d700 1 --2- 172.21.15.64:0/1867137898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a230cf01a0 0x55a230cf2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.489+0000 7f4cd259c700 1 -- 172.21.15.64:0/1867137898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a230cf01a0 msgr2=0x55a230cf2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:24:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.489+0000 7f4cd259c700 1 --2- 172.21.15.64:0/1867137898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a230cf01a0 0x55a230cf2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.489+0000 7f4cd259c700 1 -- 172.21.15.64:0/1867137898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a230ceacc0 msgr2=0x55a230cf2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:24:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.489+0000 7f4cd259c700 1 --2- 172.21.15.64:0/1867137898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a230ceacc0 0x55a230cf2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.489+0000 7f4cd259c700 1 -- 172.21.15.64:0/1867137898 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a230a68d80 con 0x55a230cea300 2023-09-13T03:24:31.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.489+0000 7f4cd1d9b700 1 --2- 172.21.15.64:0/1867137898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a230ceacc0 0x55a230cf2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:24:31.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.489+0000 7f4cd2d9d700 1 --2- 172.21.15.64:0/1867137898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a230cf01a0 0x55a230cf2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:24:31.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.490+0000 7f4cd259c700 1 --2- 172.21.15.64:0/1867137898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a230cea300 0x55a230cfb720 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4098b00 tx=0x7f4cc409aff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5fe9fe2c76edd06 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:24:31.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.490+0000 7f4cb9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:24:31.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.490+0000 7f4cb9ffb700 1 -- 172.21.15.64:0/1867137898 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc4099ca0 con 0x55a230cea300 2023-09-13T03:24:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.490+0000 7f4cb9ffb700 1 -- 172.21.15.64:0/1867137898 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f4cc409a7b0 con 0x55a230cea300 2023-09-13T03:24:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.490+0000 7f4cec682b80 1 -- 172.21.15.64:0/1867137898 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4cc800faa0 con 0x55a230cea300 2023-09-13T03:24:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.490+0000 7f4cb9ffb700 1 -- 172.21.15.64:0/1867137898 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4cc40a2790 con 0x55a230cea300 2023-09-13T03:24:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.490+0000 7f4cec682b80 10 client.27028 fetch_fsmap learned FSMap version 61 2023-09-13T03:24:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.490+0000 7f4cec682b80 1 -- 172.21.15.64:0/1867137898 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x7f4cc8011440 con 0x55a230cea300 2023-09-13T03:24:31.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.490+0000 7f4cb9ffb700 1 -- 172.21.15.64:0/1867137898 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cc800faa0 con 0x55a230cea300 2023-09-13T03:24:31.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.490+0000 7f4cb9ffb700 1 -- 172.21.15.64:0/1867137898 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 61) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f4cc4096ec0 con 0x55a230cea300 2023-09-13T03:24:31.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.490+0000 7f4cec682b80 10 client.27028 fetch_fsmap finished waiting for FSMap version 61 2023-09-13T03:24:31.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.490+0000 7f4cec682b80 10 client.27028 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:24:31.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.490+0000 7f4cec682b80 20 client.27028 populate_metadata read hostname 'smithi064' 2023-09-13T03:24:31.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.490+0000 7f4cec682b80 1 --2- 172.21.15.64:0/1867137898 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55a230d58250 0x55a230d5a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.490+0000 7f4cec682b80 4 client.27028 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:24:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.490+0000 7f4cec682b80 1 -- 172.21.15.64:0/1867137898 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a230c358c0 con 0x55a230d58250 2023-09-13T03:24:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.491+0000 7f4cd1d9b700 1 --2- 172.21.15.64:0/1867137898 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55a230d58250 0x55a230d5a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.491+0000 7f4cd1d9b700 1 --2- 172.21.15.64:0/1867137898 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55a230d58250 0x55a230d5a640 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca60b532fcb0d9d2 server_cookie=d9f6650e16f2d01c in_seq=0 out_seq=0 2023-09-13T03:24:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.491+0000 7f4cb9ffb700 10 client.27028 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:24:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.528+0000 7f4cb9ffb700 1 -- 172.21.15.64:0/1867137898 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a230c358c0 con 0x55a230d58250 2023-09-13T03:24:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.528+0000 7f4cb9ffb700 10 client.27028 handle_command_reply: tid=0 2023-09-13T03:24:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.528+0000 7f4cec682b80 1 client.27028 shutdown 2023-09-13T03:24:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.528+0000 7f4cec682b80 1 -- 172.21.15.64:0/1867137898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a230cea300 msgr2=0x55a230cfb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.528+0000 7f4cec682b80 1 --2- 172.21.15.64:0/1867137898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a230cea300 0x55a230cfb720 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4098b00 tx=0x7f4cc409aff0 comp rx=0 tx=0).stop 2023-09-13T03:24:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.529+0000 7f4cec682b80 1 -- 172.21.15.64:0/1867137898 shutdown_connections 2023-09-13T03:24:31.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.529+0000 7f4cec682b80 1 --2- 172.21.15.64:0/1867137898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a230cf01a0 0x55a230cf2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:31.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.529+0000 7f4cec682b80 1 --2- 172.21.15.64:0/1867137898 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55a230d58250 0x55a230d5a640 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:31.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.529+0000 7f4cec682b80 1 --2- 172.21.15.64:0/1867137898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a230cea300 0x55a230cfb720 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:31.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.529+0000 7f4cec682b80 1 --2- 172.21.15.64:0/1867137898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a230ceacc0 0x55a230cf2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:31.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.529+0000 7f4cb9ffb700 0 client.27028 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:24:31.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.529+0000 7f4cec682b80 1 -- 172.21.15.64:0/1867137898 >> 172.21.15.64:0/1867137898 conn(0x55a230c5f030 msgr2=0x55a230ce5090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:24:31.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.529+0000 7f4cec682b80 1 -- 172.21.15.64:0/1867137898 shutdown_connections 2023-09-13T03:24:31.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.529+0000 7f4cec682b80 1 -- 172.21.15.64:0/1867137898 wait complete. 2023-09-13T03:24:31.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.529+0000 7f4cec682b80 20 client.27028 trim_cache size 0 max 16384 2023-09-13T03:24:31.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:24:31.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:24:31.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:24:31.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:24:31.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.530+0000 7f4cd8dff700 1 -- 172.21.15.64:0/1014070416 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f4cbc0a07b0 msgr2=0x7f4cbc0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:31.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.530+0000 7f4cd8dff700 1 --2- 172.21.15.64:0/1014070416 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f4cbc0a07b0 0x7f4cbc0a2c70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4cd419f520 tx=0x7f4ccc013460 comp rx=0 tx=0).stop 2023-09-13T03:24:31.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.530+0000 7f4cd8dff700 1 -- 172.21.15.64:0/1014070416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cd41027a0 msgr2=0x7f4cd419e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:31.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.530+0000 7f4cd8dff700 1 --2- 172.21.15.64:0/1014070416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cd41027a0 0x7f4cd419e820 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4000c00 tx=0x7f4cc4004790 comp rx=0 tx=0).stop 2023-09-13T03:24:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.530+0000 7f4cd8dff700 1 -- 172.21.15.64:0/1014070416 shutdown_connections 2023-09-13T03:24:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.530+0000 7f4cd8dff700 1 --2- 172.21.15.64:0/1014070416 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f4cbc0a07b0 0x7f4cbc0a2c70 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.530+0000 7f4cd8dff700 1 --2- 172.21.15.64:0/1014070416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cd41027a0 0x7f4cd419e820 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.530+0000 7f4cd8dff700 1 --2- 172.21.15.64:0/1014070416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cd41087b0 0x7f4cd4198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.530+0000 7f4cd8dff700 1 --2- 172.21.15.64:0/1014070416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cd4103160 0x7f4cd419ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.530+0000 7f4cd8dff700 1 -- 172.21.15.64:0/1014070416 >> 172.21.15.64:0/1014070416 conn(0x7f4cd40fe250 msgr2=0x7f4cd40fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:24:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.530+0000 7f4cd8dff700 1 -- 172.21.15.64:0/1014070416 shutdown_connections 2023-09-13T03:24:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:31.530+0000 7f4cd8dff700 1 -- 172.21.15.64:0/1014070416 wait complete. 2023-09-13T03:24:31.950 INFO:tasks.workunit.client.0.smithi064.stdout: 1 272526 2.28 MB/sec execute 516 sec latency 61.047 ms 2023-09-13T03:24:32.950 INFO:tasks.workunit.client.0.smithi064.stdout: 1 273771 2.29 MB/sec execute 517 sec latency 21.365 ms 2023-09-13T03:24:33.083 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:24:32 smithi170 ceph-mon[134328]: pgmap v354: 97 pgs: 97 active+clean; 124 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 268 op/s 2023-09-13T03:24:33.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:24:32 smithi064 ceph-mon[97675]: pgmap v354: 97 pgs: 97 active+clean; 124 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 268 op/s 2023-09-13T03:24:33.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:24:32 smithi142 ceph-mon[134082]: pgmap v354: 97 pgs: 97 active+clean; 124 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 268 op/s 2023-09-13T03:24:33.950 INFO:tasks.workunit.client.0.smithi064.stdout: 1 274843 2.30 MB/sec execute 518 sec latency 22.410 ms 2023-09-13T03:24:34.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:24:33 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:24:34.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:24:33 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:24:34.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:24:33 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:24:34.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:24:33 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:24:34.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:24:33 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:24:34.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:24:33 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:24:34.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:24:33 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:24:34.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:24:33 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:24:34.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:24:33 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:24:34.950 INFO:tasks.workunit.client.0.smithi064.stdout: 1 276359 2.31 MB/sec execute 519 sec latency 23.441 ms 2023-09-13T03:24:35.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:24:34 smithi170 ceph-mon[134328]: pgmap v355: 97 pgs: 97 active+clean; 124 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 166 op/s 2023-09-13T03:24:35.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:24:34 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:24:35.077 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:24:34 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:24:35.077 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:24:34 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:24:35.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:24:34 smithi064 ceph-mon[97675]: pgmap v355: 97 pgs: 97 active+clean; 124 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 166 op/s 2023-09-13T03:24:35.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:24:34 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:24:35.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:24:34 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:24:35.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:24:34 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:24:35.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:24:34 smithi142 ceph-mon[134082]: pgmap v355: 97 pgs: 97 active+clean; 124 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 166 op/s 2023-09-13T03:24:35.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:24:34 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:24:35.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:24:34 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:24:35.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:24:34 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:24:35.950 INFO:tasks.workunit.client.0.smithi064.stdout: 1 277474 2.32 MB/sec execute 520 sec latency 15.847 ms 2023-09-13T03:24:36.950 INFO:tasks.workunit.client.0.smithi064.stdout: 1 279135 2.33 MB/sec execute 521 sec latency 26.571 ms 2023-09-13T03:24:37.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:24:36 smithi170 ceph-mon[134328]: pgmap v356: 97 pgs: 97 active+clean; 121 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 269 op/s 2023-09-13T03:24:37.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:24:36 smithi064 ceph-mon[97675]: pgmap v356: 97 pgs: 97 active+clean; 121 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 269 op/s 2023-09-13T03:24:37.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:24:36 smithi142 ceph-mon[134082]: pgmap v356: 97 pgs: 97 active+clean; 121 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 269 op/s 2023-09-13T03:24:37.950 INFO:tasks.workunit.client.0.smithi064.stdout: 1 280641 2.34 MB/sec execute 522 sec latency 13.483 ms 2023-09-13T03:24:38.950 INFO:tasks.workunit.client.0.smithi064.stdout: 1 281774 2.35 MB/sec execute 523 sec latency 24.391 ms 2023-09-13T03:24:39.159 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:24:38 smithi170 ceph-mon[134328]: pgmap v357: 97 pgs: 97 active+clean; 121 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 186 op/s 2023-09-13T03:24:39.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:24:38 smithi064 ceph-mon[97675]: pgmap v357: 97 pgs: 97 active+clean; 121 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 186 op/s 2023-09-13T03:24:39.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:24:38 smithi142 ceph-mon[134082]: pgmap v357: 97 pgs: 97 active+clean; 121 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 186 op/s 2023-09-13T03:24:39.951 INFO:tasks.workunit.client.0.smithi064.stdout: 1 283170 2.36 MB/sec execute 524 sec latency 17.218 ms 2023-09-13T03:24:40.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:24:40 smithi142 ceph-mon[134082]: pgmap v358: 97 pgs: 97 active+clean; 121 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 186 op/s 2023-09-13T03:24:40.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:24:40 smithi170 ceph-mon[134328]: pgmap v358: 97 pgs: 97 active+clean; 121 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 186 op/s 2023-09-13T03:24:40.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:24:40 smithi064 ceph-mon[97675]: pgmap v358: 97 pgs: 97 active+clean; 121 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 186 op/s 2023-09-13T03:24:40.951 INFO:tasks.workunit.client.0.smithi064.stdout: 1 284359 2.37 MB/sec execute 525 sec latency 11.605 ms 2023-09-13T03:24:41.951 INFO:tasks.workunit.client.0.smithi064.stdout: 1 286046 2.37 MB/sec execute 526 sec latency 31.456 ms 2023-09-13T03:24:42.951 INFO:tasks.workunit.client.0.smithi064.stdout: 1 287625 2.39 MB/sec execute 527 sec latency 12.612 ms 2023-09-13T03:24:43.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:24:42 smithi170 ceph-mon[134328]: pgmap v359: 97 pgs: 97 active+clean; 127 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 264 op/s 2023-09-13T03:24:43.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:24:42 smithi064 ceph-mon[97675]: pgmap v359: 97 pgs: 97 active+clean; 127 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 264 op/s 2023-09-13T03:24:43.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:24:42 smithi142 ceph-mon[134082]: pgmap v359: 97 pgs: 97 active+clean; 127 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 264 op/s 2023-09-13T03:24:43.951 INFO:tasks.workunit.client.0.smithi064.stdout: 1 288851 2.40 MB/sec execute 528 sec latency 17.964 ms 2023-09-13T03:24:44.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.686+0000 7fd264bff700 1 -- 172.21.15.64:0/1174376208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd260106660 msgr2=0x7fd260106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:44.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.686+0000 7fd264bff700 1 --2- 172.21.15.64:0/1174376208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd260106660 0x7fd260106a40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd2500099c0 tx=0x7fd250009cd0 comp rx=0 tx=0).stop 2023-09-13T03:24:44.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.686+0000 7fd264bff700 1 -- 172.21.15.64:0/1174376208 shutdown_connections 2023-09-13T03:24:44.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.686+0000 7fd264bff700 1 --2- 172.21.15.64:0/1174376208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd260106660 0x7fd260106a40 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:44.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.686+0000 7fd264bff700 1 --2- 172.21.15.64:0/1174376208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd260100f60 0x7fd26010d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:44.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.686+0000 7fd264bff700 1 --2- 172.21.15.64:0/1174376208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2601005a0 0x7fd260100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:44.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.686+0000 7fd264bff700 1 -- 172.21.15.64:0/1174376208 >> 172.21.15.64:0/1174376208 conn(0x7fd2600fc050 msgr2=0x7fd2600fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:24:44.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.686+0000 7fd264bff700 1 -- 172.21.15.64:0/1174376208 shutdown_connections 2023-09-13T03:24:44.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.686+0000 7fd264bff700 1 -- 172.21.15.64:0/1174376208 wait complete. 2023-09-13T03:24:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.687+0000 7fd264bff700 1 Processor -- start 2023-09-13T03:24:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.688+0000 7fd264bff700 1 -- start start 2023-09-13T03:24:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.688+0000 7fd264bff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2601005a0 0x7fd26019c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.688+0000 7fd264bff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd260100f60 0x7fd26019cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.688+0000 7fd264bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd260106660 0x7fd260196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.688+0000 7fd264bff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd260196ac0 con 0x7fd2601005a0 2023-09-13T03:24:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.688+0000 7fd264bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd260196c30 con 0x7fd260106660 2023-09-13T03:24:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.688+0000 7fd264bff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd260196df0 con 0x7fd260100f60 2023-09-13T03:24:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.688+0000 7fd25e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2601005a0 0x7fd26019c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.689+0000 7fd257fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd260100f60 0x7fd26019cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.689+0000 7fd25ed9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd260106660 0x7fd260196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.689+0000 7fd25ed9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd260106660 0x7fd260196450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:39146/0 (socket says 172.21.15.64:39146) 2023-09-13T03:24:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.689+0000 7fd257fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd260100f60 0x7fd26019cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:60684/0 (socket says 172.21.15.64:60684) 2023-09-13T03:24:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.689+0000 7fd25ed9d700 1 -- 172.21.15.64:0/1577936855 learned_addr learned my addr 172.21.15.64:0/1577936855 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:24:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.689+0000 7fd25ed9d700 1 -- 172.21.15.64:0/1577936855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd260100f60 msgr2=0x7fd26019cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.689+0000 7fd25ed9d700 1 --2- 172.21.15.64:0/1577936855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd260100f60 0x7fd26019cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.689+0000 7fd25ed9d700 1 -- 172.21.15.64:0/1577936855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2601005a0 msgr2=0x7fd26019c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.689+0000 7fd25ed9d700 1 --2- 172.21.15.64:0/1577936855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2601005a0 0x7fd26019c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.689+0000 7fd25ed9d700 1 -- 172.21.15.64:0/1577936855 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2500095f0 con 0x7fd260106660 2023-09-13T03:24:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.689+0000 7fd25e59c700 1 --2- 172.21.15.64:0/1577936855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2601005a0 0x7fd26019c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:24:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.689+0000 7fd257fff700 1 --2- 172.21.15.64:0/1577936855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd260100f60 0x7fd26019cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:24:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.689+0000 7fd25ed9d700 1 --2- 172.21.15.64:0/1577936855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd260106660 0x7fd260196450 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd25800ef30 tx=0x7fd25800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:24:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.690+0000 7fd2577fe700 1 -- 172.21.15.64:0/1577936855 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd258018070 con 0x7fd260106660 2023-09-13T03:24:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.690+0000 7fd2577fe700 1 -- 172.21.15.64:0/1577936855 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd25800f040 con 0x7fd260106660 2023-09-13T03:24:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.690+0000 7fd2577fe700 1 -- 172.21.15.64:0/1577936855 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25800cd00 con 0x7fd260106660 2023-09-13T03:24:44.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.690+0000 7fd264bff700 1 -- 172.21.15.64:0/1577936855 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd260197070 con 0x7fd260106660 2023-09-13T03:24:44.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.690+0000 7fd264bff700 1 -- 172.21.15.64:0/1577936855 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd260072a10 con 0x7fd260106660 2023-09-13T03:24:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.693+0000 7fd2577fe700 1 -- 172.21.15.64:0/1577936855 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fd258003a00 con 0x7fd260106660 2023-09-13T03:24:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.694+0000 7fd27846ab80 1 Processor -- start 2023-09-13T03:24:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.695+0000 7fd27846ab80 1 -- start start 2023-09-13T03:24:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.695+0000 7fd2577fe700 1 --2- 172.21.15.64:0/1577936855 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fd24c0a0830 0x7fd24c0a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.695+0000 7fd27846ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557ccd0e1a0 0x5557ccd0e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.695+0000 7fd27846ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5557ccd08300 0x5557ccd08780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.695+0000 7fd27846ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557ccd08cc0 0x5557ccd14f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.695+0000 7fd27846ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557ccc616d0 con 0x5557ccd08cc0 2023-09-13T03:24:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.695+0000 7fd27846ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557ccc5eff0 con 0x5557ccd0e1a0 2023-09-13T03:24:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.695+0000 7fd27846ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557ccc5a8e0 con 0x5557ccd08300 2023-09-13T03:24:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.695+0000 7fd2577fe700 1 -- 172.21.15.64:0/1577936855 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fd2580cba60 con 0x7fd260106660 2023-09-13T03:24:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.695+0000 7fd25e59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557ccd0e1a0 0x5557ccd0e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.695+0000 7fd25e59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557ccd0e1a0 0x5557ccd0e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:39158/0 (socket says 172.21.15.64:39158) 2023-09-13T03:24:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.695+0000 7fd25e59c700 1 -- 172.21.15.64:0/1961588576 learned_addr learned my addr 172.21.15.64:0/1961588576 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:24:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.695+0000 7fd257fff700 1 --2- 172.21.15.64:0/1961588576 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5557ccd08300 0x5557ccd08780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.695+0000 7fd25ed9d700 1 --2- 172.21.15.64:0/1961588576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557ccd08cc0 0x5557ccd14f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.696+0000 7fd257fff700 1 --2- 172.21.15.64:0/1577936855 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fd24c0a0830 0x7fd24c0a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.696+0000 7fd25e59c700 1 -- 172.21.15.64:0/1961588576 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5557ccd08300 msgr2=0x5557ccd08780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:24:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.696+0000 7fd25e59c700 1 --2- 172.21.15.64:0/1961588576 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5557ccd08300 0x5557ccd08780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.696+0000 7fd25e59c700 1 -- 172.21.15.64:0/1961588576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557ccd08cc0 msgr2=0x5557ccd14f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:24:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.696+0000 7fd25e59c700 1 --2- 172.21.15.64:0/1961588576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557ccd08cc0 0x5557ccd14f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.696+0000 7fd25e59c700 1 -- 172.21.15.64:0/1961588576 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557cca86d80 con 0x5557ccd0e1a0 2023-09-13T03:24:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.696+0000 7fd25e59c700 1 --2- 172.21.15.64:0/1961588576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557ccd0e1a0 0x5557ccd0e580 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd2500036c0 tx=0x7fd250005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f51f1dcf47439a0d server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:24:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.696+0000 7fd257fff700 1 --2- 172.21.15.64:0/1577936855 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fd24c0a0830 0x7fd24c0a2cf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x5557ccd03060 tx=0x7fd248009260 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:24:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.696+0000 7fd254ff9700 1 -- 172.21.15.64:0/1961588576 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25002c070 con 0x5557ccd0e1a0 2023-09-13T03:24:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.697+0000 7fd254ff9700 1 -- 172.21.15.64:0/1961588576 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd250004cf0 con 0x5557ccd0e1a0 2023-09-13T03:24:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.697+0000 7fd254ff9700 1 -- 172.21.15.64:0/1961588576 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd250021040 con 0x5557ccd0e1a0 2023-09-13T03:24:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.697+0000 7fd27846ab80 1 -- 172.21.15.64:0/1961588576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557ccd0e1a0 msgr2=0x5557ccd0e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.697+0000 7fd27846ab80 1 --2- 172.21.15.64:0/1961588576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557ccd0e1a0 0x5557ccd0e580 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd2500036c0 tx=0x7fd250005fb0 comp rx=0 tx=0).stop 2023-09-13T03:24:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.697+0000 7fd27846ab80 1 -- 172.21.15.64:0/1961588576 shutdown_connections 2023-09-13T03:24:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.697+0000 7fd27846ab80 1 --2- 172.21.15.64:0/1961588576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557ccd0e1a0 0x5557ccd0e580 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.697+0000 7fd27846ab80 1 --2- 172.21.15.64:0/1961588576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557ccd08cc0 0x5557ccd14f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.697+0000 7fd27846ab80 1 --2- 172.21.15.64:0/1961588576 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5557ccd08300 0x5557ccd08780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.697+0000 7fd27846ab80 1 -- 172.21.15.64:0/1961588576 >> 172.21.15.64:0/1961588576 conn(0x5557ccc7d030 msgr2=0x5557ccd03560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:24:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.698+0000 7fd27846ab80 1 -- 172.21.15.64:0/1961588576 shutdown_connections 2023-09-13T03:24:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.698+0000 7fd27846ab80 1 -- 172.21.15.64:0/1961588576 wait complete. 2023-09-13T03:24:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.698+0000 7fd27846ab80 1 -- start start 2023-09-13T03:24:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.698+0000 7fd27846ab80 1 Processor -- start 2023-09-13T03:24:44.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.699+0000 7fd27846ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5557ccd08300 0x5557ccd14e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:44.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.699+0000 7fd27846ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557ccd08cc0 0x5557ccd12030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:44.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.699+0000 7fd27846ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557ccd0e1a0 0x5557ccd125f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:44.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.699+0000 7fd27846ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557ccbf8b10 con 0x5557ccd0e1a0 2023-09-13T03:24:44.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.699+0000 7fd27846ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557ccbf8810 con 0x5557ccd08cc0 2023-09-13T03:24:44.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.699+0000 7fd27846ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557ccadc8c0 con 0x5557ccd08300 2023-09-13T03:24:44.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.699+0000 7fd27846ab80 10 client.0 _ll_register_callbacks cb 0x5557ccb42a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:24:44.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.699+0000 7fd257fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557ccd08cc0 0x5557ccd12030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:44.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.699+0000 7fd25ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557ccd0e1a0 0x5557ccd125f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:44.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.699+0000 7fd25e59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5557ccd08300 0x5557ccd14e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:44.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.699+0000 7fd257fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557ccd08cc0 0x5557ccd12030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:39162/0 (socket says 172.21.15.64:39162) 2023-09-13T03:24:44.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.699+0000 7fd257fff700 1 -- 172.21.15.64:0/2959635181 learned_addr learned my addr 172.21.15.64:0/2959635181 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:24:44.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.699+0000 7fd25ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557ccd0e1a0 0x5557ccd125f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42686/0 (socket says 172.21.15.64:42686) 2023-09-13T03:24:44.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.699+0000 7fd25e59c700 1 -- 172.21.15.64:0/2959635181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557ccd08cc0 msgr2=0x5557ccd12030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:24:44.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.700+0000 7fd25e59c700 1 --2- 172.21.15.64:0/2959635181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557ccd08cc0 0x5557ccd12030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:44.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.700+0000 7fd25e59c700 1 -- 172.21.15.64:0/2959635181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557ccd0e1a0 msgr2=0x5557ccd125f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:24:44.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.700+0000 7fd25e59c700 1 --2- 172.21.15.64:0/2959635181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557ccd0e1a0 0x5557ccd125f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:44.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.700+0000 7fd25e59c700 1 -- 172.21.15.64:0/2959635181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd26010dd20 con 0x5557ccd08300 2023-09-13T03:24:44.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.700+0000 7fd257fff700 1 --2- 172.21.15.64:0/2959635181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557ccd08cc0 0x5557ccd12030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:24:44.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.700+0000 7fd25ed9d700 1 --2- 172.21.15.64:0/2959635181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557ccd0e1a0 0x5557ccd125f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:24:44.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.700+0000 7fd25e59c700 1 --2- 172.21.15.64:0/2959635181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5557ccd08300 0x5557ccd14e20 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd250009990 tx=0x7fd2500059c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34609f5b157a5a54 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:24:44.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.700+0000 7fd245ffb700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:24:44.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.700+0000 7fd245ffb700 1 -- 172.21.15.64:0/2959635181 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25002c040 con 0x5557ccd08300 2023-09-13T03:24:44.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.700+0000 7fd245ffb700 1 -- 172.21.15.64:0/2959635181 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fd250005360 con 0x5557ccd08300 2023-09-13T03:24:44.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.700+0000 7fd27846ab80 1 -- 172.21.15.64:0/2959635181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5557ccbb2490 con 0x5557ccd08300 2023-09-13T03:24:44.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.700+0000 7fd245ffb700 1 -- 172.21.15.64:0/2959635181 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd250021040 con 0x5557ccd08300 2023-09-13T03:24:44.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.700+0000 7fd27846ab80 10 client.27040 fetch_fsmap learned FSMap version 61 2023-09-13T03:24:44.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.700+0000 7fd27846ab80 1 -- 172.21.15.64:0/2959635181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x5557cca6fc00 con 0x5557ccd08300 2023-09-13T03:24:44.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.701+0000 7fd245ffb700 1 -- 172.21.15.64:0/2959635181 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5557ccbb2490 con 0x5557ccd08300 2023-09-13T03:24:44.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.701+0000 7fd245ffb700 1 -- 172.21.15.64:0/2959635181 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 61) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fd250024030 con 0x5557ccd08300 2023-09-13T03:24:44.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.701+0000 7fd27846ab80 10 client.27040 fetch_fsmap finished waiting for FSMap version 61 2023-09-13T03:24:44.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.701+0000 7fd27846ab80 10 client.27040 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:24:44.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.701+0000 7fd27846ab80 20 client.27040 populate_metadata read hostname 'smithi064' 2023-09-13T03:24:44.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.701+0000 7fd27846ab80 1 --2- 172.21.15.64:0/2959635181 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5557ccd21e90 0x5557ccd24280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:44.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.701+0000 7fd27846ab80 4 client.27040 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:24:44.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.701+0000 7fd27846ab80 1 -- 172.21.15.64:0/2959635181 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5557ccc6f840 con 0x5557ccd21e90 2023-09-13T03:24:44.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.701+0000 7fd257fff700 1 --2- 172.21.15.64:0/2959635181 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5557ccd21e90 0x5557ccd24280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:44.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.702+0000 7fd257fff700 1 --2- 172.21.15.64:0/2959635181 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5557ccd21e90 0x5557ccd24280 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b107a278945b631c server_cookie=530b018122f4ce11 in_seq=0 out_seq=0 2023-09-13T03:24:44.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.702+0000 7fd245ffb700 10 client.27040 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:24:44.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.703+0000 7fd245ffb700 1 -- 172.21.15.64:0/2959635181 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5557ccc6f840 con 0x5557ccd21e90 2023-09-13T03:24:44.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.703+0000 7fd245ffb700 10 client.27040 handle_command_reply: tid=0 2023-09-13T03:24:44.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.703+0000 7fd27846ab80 1 client.27040 shutdown 2023-09-13T03:24:44.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.703+0000 7fd27846ab80 1 -- 172.21.15.64:0/2959635181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5557ccd08300 msgr2=0x5557ccd14e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:44.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.703+0000 7fd27846ab80 1 --2- 172.21.15.64:0/2959635181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5557ccd08300 0x5557ccd14e20 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd250009990 tx=0x7fd2500059c0 comp rx=0 tx=0).stop 2023-09-13T03:24:44.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.703+0000 7fd27846ab80 1 -- 172.21.15.64:0/2959635181 shutdown_connections 2023-09-13T03:24:44.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.703+0000 7fd27846ab80 1 --2- 172.21.15.64:0/2959635181 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5557ccd21e90 0x5557ccd24280 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:44.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.703+0000 7fd27846ab80 1 --2- 172.21.15.64:0/2959635181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557ccd0e1a0 0x5557ccd125f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:44.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.703+0000 7fd27846ab80 1 --2- 172.21.15.64:0/2959635181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5557ccd08300 0x5557ccd14e20 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:44.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.703+0000 7fd27846ab80 1 --2- 172.21.15.64:0/2959635181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557ccd08cc0 0x5557ccd12030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:44.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.703+0000 7fd245ffb700 0 client.27040 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:24:44.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.703+0000 7fd27846ab80 1 -- 172.21.15.64:0/2959635181 >> 172.21.15.64:0/2959635181 conn(0x5557ccc7d030 msgr2=0x5557ccd027a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:24:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.703+0000 7fd27846ab80 1 -- 172.21.15.64:0/2959635181 shutdown_connections 2023-09-13T03:24:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.703+0000 7fd27846ab80 1 -- 172.21.15.64:0/2959635181 wait complete. 2023-09-13T03:24:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.703+0000 7fd27846ab80 20 client.27040 trim_cache size 0 max 16384 2023-09-13T03:24:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.719+0000 7fd27846ab80 1 Processor -- start 2023-09-13T03:24:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.719+0000 7fd27846ab80 1 -- start start 2023-09-13T03:24:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.719+0000 7fd27846ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557ccd08300 0x5557ccd14950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.719+0000 7fd27846ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557ccd08cc0 0x5557ccd03090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.719+0000 7fd27846ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5557ccd0e1a0 0x5557ccd035d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.719+0000 7fd27846ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557ccd12ff0 con 0x5557ccd08cc0 2023-09-13T03:24:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.719+0000 7fd27846ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557ccd0e970 con 0x5557ccd08300 2023-09-13T03:24:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.719+0000 7fd27846ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557cccfffc0 con 0x5557ccd0e1a0 2023-09-13T03:24:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.719+0000 7fd25e59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557ccd08300 0x5557ccd14950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.719+0000 7fd25e59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557ccd08300 0x5557ccd14950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:39168/0 (socket says 172.21.15.64:39168) 2023-09-13T03:24:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.719+0000 7fd25e59c700 1 -- 172.21.15.64:0/295766197 learned_addr learned my addr 172.21.15.64:0/295766197 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:24:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.719+0000 7fd257fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557ccd08cc0 0x5557ccd03090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.719+0000 7fd25ed9d700 1 --2- 172.21.15.64:0/295766197 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5557ccd0e1a0 0x5557ccd035d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:44.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.720+0000 7fd25e59c700 1 -- 172.21.15.64:0/295766197 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5557ccd0e1a0 msgr2=0x5557ccd035d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:24:44.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.720+0000 7fd25e59c700 1 --2- 172.21.15.64:0/295766197 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5557ccd0e1a0 0x5557ccd035d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:44.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.720+0000 7fd25e59c700 1 -- 172.21.15.64:0/295766197 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557ccd08cc0 msgr2=0x5557ccd03090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:24:44.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.720+0000 7fd25e59c700 1 --2- 172.21.15.64:0/295766197 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557ccd08cc0 0x5557ccd03090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:44.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.720+0000 7fd25e59c700 1 -- 172.21.15.64:0/295766197 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557ccaad1f0 con 0x5557ccd08300 2023-09-13T03:24:44.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.720+0000 7fd25e59c700 1 --2- 172.21.15.64:0/295766197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557ccd08300 0x5557ccd14950 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd25000b580 tx=0x7fd250009cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7652fd4a2b9c18a7 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:24:44.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.720+0000 7fd2557fa700 1 -- 172.21.15.64:0/295766197 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd250021070 con 0x5557ccd08300 2023-09-13T03:24:44.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.720+0000 7fd2557fa700 1 -- 172.21.15.64:0/295766197 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd250028910 con 0x5557ccd08300 2023-09-13T03:24:44.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.720+0000 7fd2557fa700 1 -- 172.21.15.64:0/295766197 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25002c040 con 0x5557ccd08300 2023-09-13T03:24:44.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.721+0000 7fd27846ab80 1 -- 172.21.15.64:0/295766197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557ccd08300 msgr2=0x5557ccd14950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:44.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.721+0000 7fd27846ab80 1 --2- 172.21.15.64:0/295766197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557ccd08300 0x5557ccd14950 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd25000b580 tx=0x7fd250009cd0 comp rx=0 tx=0).stop 2023-09-13T03:24:44.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.721+0000 7fd27846ab80 1 -- 172.21.15.64:0/295766197 shutdown_connections 2023-09-13T03:24:44.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.721+0000 7fd27846ab80 1 --2- 172.21.15.64:0/295766197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557ccd08300 0x5557ccd14950 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:44.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.721+0000 7fd27846ab80 1 --2- 172.21.15.64:0/295766197 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5557ccd0e1a0 0x5557ccd035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:44.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.721+0000 7fd27846ab80 1 --2- 172.21.15.64:0/295766197 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557ccd08cc0 0x5557ccd03090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:44.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.721+0000 7fd27846ab80 1 -- 172.21.15.64:0/295766197 >> 172.21.15.64:0/295766197 conn(0x5557ccc7d030 msgr2=0x5557ccd0c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:24:44.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.721+0000 7fd27846ab80 1 -- 172.21.15.64:0/295766197 shutdown_connections 2023-09-13T03:24:44.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.721+0000 7fd27846ab80 1 -- 172.21.15.64:0/295766197 wait complete. 2023-09-13T03:24:44.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.722+0000 7fd27846ab80 1 -- start start 2023-09-13T03:24:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.722+0000 7fd27846ab80 1 Processor -- start 2023-09-13T03:24:44.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.722+0000 7fd27846ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5557ccd08300 0x5557ccd19410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:44.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.722+0000 7fd27846ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557ccd08cc0 0x5557ccd17840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:44.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.722+0000 7fd27846ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557ccd0e1a0 0x5557ccd17d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:44.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.722+0000 7fd27846ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557ccd18480 con 0x5557ccd08cc0 2023-09-13T03:24:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.722+0000 7fd27846ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557ccd185f0 con 0x5557ccd0e1a0 2023-09-13T03:24:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.722+0000 7fd27846ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557ccd75ca0 con 0x5557ccd08300 2023-09-13T03:24:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.722+0000 7fd27846ab80 10 client.0 _ll_register_callbacks cb 0x7fd258009410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:24:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.723+0000 7fd25e59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5557ccd08300 0x5557ccd19410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.723+0000 7fd25e59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5557ccd08300 0x5557ccd19410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:60724/0 (socket says 172.21.15.64:60724) 2023-09-13T03:24:44.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.723+0000 7fd25e59c700 1 -- 172.21.15.64:0/1220755281 learned_addr learned my addr 172.21.15.64:0/1220755281 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:24:44.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.723+0000 7fd25ed9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557ccd0e1a0 0x5557ccd17d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:44.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.723+0000 7fd257fff700 1 --2- 172.21.15.64:0/1220755281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557ccd08cc0 0x5557ccd17840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:44.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.723+0000 7fd25e59c700 1 -- 172.21.15.64:0/1220755281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557ccd0e1a0 msgr2=0x5557ccd17d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:24:44.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.723+0000 7fd25e59c700 1 --2- 172.21.15.64:0/1220755281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557ccd0e1a0 0x5557ccd17d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.723+0000 7fd25e59c700 1 -- 172.21.15.64:0/1220755281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557ccd08cc0 msgr2=0x5557ccd17840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:24:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.723+0000 7fd25e59c700 1 --2- 172.21.15.64:0/1220755281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557ccd08cc0 0x5557ccd17840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.723+0000 7fd25e59c700 1 -- 172.21.15.64:0/1220755281 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5557cca6fc00 con 0x5557ccd08300 2023-09-13T03:24:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.723+0000 7fd25e59c700 1 --2- 172.21.15.64:0/1220755281 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5557ccd08300 0x5557ccd19410 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd250000c00 tx=0x7fd2500059c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=523ff3b080311d7c server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:24:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.723+0000 7fd245ffb700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:24:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.724+0000 7fd245ffb700 1 -- 172.21.15.64:0/1220755281 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25002c070 con 0x5557ccd08300 2023-09-13T03:24:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.724+0000 7fd27846ab80 1 -- 172.21.15.64:0/1220755281 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd250004cf0 con 0x5557ccd08300 2023-09-13T03:24:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.724+0000 7fd245ffb700 1 -- 172.21.15.64:0/1220755281 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fd250000e40 con 0x5557ccd08300 2023-09-13T03:24:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.724+0000 7fd245ffb700 1 -- 172.21.15.64:0/1220755281 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd250021040 con 0x5557ccd08300 2023-09-13T03:24:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.724+0000 7fd27846ab80 10 client.27046 fetch_fsmap learned FSMap version 61 2023-09-13T03:24:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.724+0000 7fd27846ab80 1 -- 172.21.15.64:0/1220755281 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x5557cca85250 con 0x5557ccd08300 2023-09-13T03:24:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.724+0000 7fd245ffb700 1 -- 172.21.15.64:0/1220755281 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd250004cf0 con 0x5557ccd08300 2023-09-13T03:24:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.724+0000 7fd245ffb700 1 -- 172.21.15.64:0/1220755281 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 61) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fd25004d070 con 0x5557ccd08300 2023-09-13T03:24:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.724+0000 7fd27846ab80 10 client.27046 fetch_fsmap finished waiting for FSMap version 61 2023-09-13T03:24:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.724+0000 7fd27846ab80 10 client.27046 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:24:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.724+0000 7fd27846ab80 20 client.27046 populate_metadata read hostname 'smithi064' 2023-09-13T03:24:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.725+0000 7fd27846ab80 1 --2- 172.21.15.64:0/1220755281 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5557ccd76dc0 0x5557ccd791b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.725+0000 7fd27846ab80 4 client.27046 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:24:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.725+0000 7fd27846ab80 1 -- 172.21.15.64:0/1220755281 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5557ccc538c0 con 0x5557ccd76dc0 2023-09-13T03:24:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.725+0000 7fd257fff700 1 --2- 172.21.15.64:0/1220755281 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5557ccd76dc0 0x5557ccd791b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.725+0000 7fd245ffb700 10 client.27046 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:24:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.725+0000 7fd257fff700 1 --2- 172.21.15.64:0/1220755281 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5557ccd76dc0 0x5557ccd791b0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4343591be812200a server_cookie=fcbcb705a038cf1e in_seq=0 out_seq=0 2023-09-13T03:24:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.763+0000 7fd245ffb700 1 -- 172.21.15.64:0/1220755281 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5557ccc538c0 con 0x5557ccd76dc0 2023-09-13T03:24:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.763+0000 7fd245ffb700 10 client.27046 handle_command_reply: tid=0 2023-09-13T03:24:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.763+0000 7fd27846ab80 1 client.27046 shutdown 2023-09-13T03:24:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.763+0000 7fd27846ab80 1 -- 172.21.15.64:0/1220755281 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5557ccd08300 msgr2=0x5557ccd19410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.763+0000 7fd27846ab80 1 --2- 172.21.15.64:0/1220755281 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5557ccd08300 0x5557ccd19410 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd250000c00 tx=0x7fd2500059c0 comp rx=0 tx=0).stop 2023-09-13T03:24:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.763+0000 7fd27846ab80 1 -- 172.21.15.64:0/1220755281 shutdown_connections 2023-09-13T03:24:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.763+0000 7fd27846ab80 1 --2- 172.21.15.64:0/1220755281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557ccd0e1a0 0x5557ccd17d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.763+0000 7fd27846ab80 1 --2- 172.21.15.64:0/1220755281 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5557ccd76dc0 0x5557ccd791b0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.763+0000 7fd27846ab80 1 --2- 172.21.15.64:0/1220755281 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5557ccd08300 0x5557ccd19410 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.763+0000 7fd27846ab80 1 --2- 172.21.15.64:0/1220755281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5557ccd08cc0 0x5557ccd17840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.763+0000 7fd245ffb700 0 client.27046 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:24:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.763+0000 7fd27846ab80 1 -- 172.21.15.64:0/1220755281 >> 172.21.15.64:0/1220755281 conn(0x5557ccc7d030 msgr2=0x5557ccd0c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:24:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.763+0000 7fd27846ab80 1 -- 172.21.15.64:0/1220755281 shutdown_connections 2023-09-13T03:24:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.763+0000 7fd27846ab80 1 -- 172.21.15.64:0/1220755281 wait complete. 2023-09-13T03:24:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.763+0000 7fd27846ab80 20 client.27046 trim_cache size 0 max 16384 2023-09-13T03:24:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:24:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:24:44.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:24:44.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:24:44.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.764+0000 7fd264bff700 1 -- 172.21.15.64:0/1577936855 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fd24c0a0830 msgr2=0x7fd24c0a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:44.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.764+0000 7fd264bff700 1 --2- 172.21.15.64:0/1577936855 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fd24c0a0830 0x7fd24c0a2cf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x5557ccd03060 tx=0x7fd248009260 comp rx=0 tx=0).stop 2023-09-13T03:24:44.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.764+0000 7fd264bff700 1 -- 172.21.15.64:0/1577936855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd260106660 msgr2=0x7fd260196450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:44.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.764+0000 7fd264bff700 1 --2- 172.21.15.64:0/1577936855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd260106660 0x7fd260196450 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd25800ef30 tx=0x7fd25800c3a0 comp rx=0 tx=0).stop 2023-09-13T03:24:44.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.764+0000 7fd264bff700 1 -- 172.21.15.64:0/1577936855 shutdown_connections 2023-09-13T03:24:44.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.764+0000 7fd264bff700 1 --2- 172.21.15.64:0/1577936855 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fd24c0a0830 0x7fd24c0a2cf0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.764+0000 7fd264bff700 1 --2- 172.21.15.64:0/1577936855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd260106660 0x7fd260196450 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.764+0000 7fd264bff700 1 --2- 172.21.15.64:0/1577936855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2601005a0 0x7fd26019c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.764+0000 7fd264bff700 1 --2- 172.21.15.64:0/1577936855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd260100f60 0x7fd26019cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.764+0000 7fd264bff700 1 -- 172.21.15.64:0/1577936855 >> 172.21.15.64:0/1577936855 conn(0x7fd2600fc050 msgr2=0x7fd2600fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:24:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.764+0000 7fd264bff700 1 -- 172.21.15.64:0/1577936855 shutdown_connections 2023-09-13T03:24:44.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:44.765+0000 7fd264bff700 1 -- 172.21.15.64:0/1577936855 wait complete. 2023-09-13T03:24:44.951 INFO:tasks.workunit.client.0.smithi064.stdout: 1 290707 2.41 MB/sec execute 529 sec latency 28.954 ms 2023-09-13T03:24:45.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:24:44 smithi170 ceph-mon[134328]: pgmap v360: 97 pgs: 97 active+clean; 127 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 181 op/s 2023-09-13T03:24:45.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:24:44 smithi064 ceph-mon[97675]: pgmap v360: 97 pgs: 97 active+clean; 127 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 181 op/s 2023-09-13T03:24:45.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:24:44 smithi142 ceph-mon[134082]: pgmap v360: 97 pgs: 97 active+clean; 127 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 181 op/s 2023-09-13T03:24:45.951 INFO:tasks.workunit.client.0.smithi064.stdout: 1 291750 2.42 MB/sec execute 530 sec latency 14.141 ms 2023-09-13T03:24:46.951 INFO:tasks.workunit.client.0.smithi064.stdout: 1 292701 2.42 MB/sec execute 531 sec latency 193.574 ms 2023-09-13T03:24:47.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:24:46 smithi170 ceph-mon[134328]: pgmap v361: 97 pgs: 97 active+clean; 125 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 288 op/s 2023-09-13T03:24:47.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:24:46 smithi064 ceph-mon[97675]: pgmap v361: 97 pgs: 97 active+clean; 125 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 288 op/s 2023-09-13T03:24:47.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:24:46 smithi142 ceph-mon[134082]: pgmap v361: 97 pgs: 97 active+clean; 125 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 288 op/s 2023-09-13T03:24:47.952 INFO:tasks.workunit.client.0.smithi064.stdout: 1 294570 2.44 MB/sec execute 532 sec latency 222.343 ms 2023-09-13T03:24:48.951 INFO:tasks.workunit.client.0.smithi064.stdout: 1 295656 2.44 MB/sec execute 533 sec latency 21.742 ms 2023-09-13T03:24:49.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:24:48 smithi170 ceph-mon[134328]: pgmap v362: 97 pgs: 97 active+clean; 125 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 185 op/s 2023-09-13T03:24:49.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:24:48 smithi064 ceph-mon[97675]: pgmap v362: 97 pgs: 97 active+clean; 125 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 185 op/s 2023-09-13T03:24:49.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:24:48 smithi142 ceph-mon[134082]: pgmap v362: 97 pgs: 97 active+clean; 125 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 185 op/s 2023-09-13T03:24:49.951 INFO:tasks.workunit.client.0.smithi064.stdout: 1 297444 2.46 MB/sec execute 534 sec latency 25.206 ms 2023-09-13T03:24:50.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:24:50 smithi142 ceph-mon[134082]: pgmap v363: 97 pgs: 97 active+clean; 125 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 185 op/s 2023-09-13T03:24:50.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:24:50 smithi170 ceph-mon[134328]: pgmap v363: 97 pgs: 97 active+clean; 125 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 185 op/s 2023-09-13T03:24:50.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:24:50 smithi064 ceph-mon[97675]: pgmap v363: 97 pgs: 97 active+clean; 125 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 185 op/s 2023-09-13T03:24:50.951 INFO:tasks.workunit.client.0.smithi064.stdout: 1 298640 2.47 MB/sec execute 535 sec latency 17.113 ms 2023-09-13T03:24:51.951 INFO:tasks.workunit.client.0.smithi064.stdout: 1 300274 2.47 MB/sec execute 536 sec latency 48.650 ms 2023-09-13T03:24:52.951 INFO:tasks.workunit.client.0.smithi064.stdout: 1 301860 2.49 MB/sec execute 537 sec latency 13.748 ms 2023-09-13T03:24:53.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:24:52 smithi170 ceph-mon[134328]: pgmap v364: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 272 op/s 2023-09-13T03:24:53.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:24:52 smithi064 ceph-mon[97675]: pgmap v364: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 272 op/s 2023-09-13T03:24:53.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:24:52 smithi142 ceph-mon[134082]: pgmap v364: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 272 op/s 2023-09-13T03:24:53.951 INFO:tasks.workunit.client.0.smithi064.stdout: 1 303214 2.50 MB/sec execute 538 sec latency 30.335 ms 2023-09-13T03:24:54.952 INFO:tasks.workunit.client.0.smithi064.stdout: 1 305293 2.52 MB/sec execute 539 sec latency 16.304 ms 2023-09-13T03:24:55.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:24:54 smithi170 ceph-mon[134328]: pgmap v365: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 194 op/s 2023-09-13T03:24:55.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:24:54 smithi064 ceph-mon[97675]: pgmap v365: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 194 op/s 2023-09-13T03:24:55.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:24:54 smithi142 ceph-mon[134082]: pgmap v365: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 194 op/s 2023-09-13T03:24:55.952 INFO:tasks.workunit.client.0.smithi064.stdout: 1 306044 2.51 MB/sec execute 540 sec latency 207.929 ms 2023-09-13T03:24:56.952 INFO:tasks.workunit.client.0.smithi064.stdout: 1 307455 2.52 MB/sec execute 541 sec latency 32.130 ms 2023-09-13T03:24:57.159 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:24:56 smithi170 ceph-mon[134328]: pgmap v366: 97 pgs: 97 active+clean; 137 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 294 op/s 2023-09-13T03:24:57.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:24:56 smithi064 ceph-mon[97675]: pgmap v366: 97 pgs: 97 active+clean; 137 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 294 op/s 2023-09-13T03:24:57.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:24:56 smithi142 ceph-mon[134082]: pgmap v366: 97 pgs: 97 active+clean; 137 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 294 op/s 2023-09-13T03:24:57.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.916+0000 7fad8b59e700 1 -- 172.21.15.64:0/1640398574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad8c106470 msgr2=0x7fad8c106850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:57.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.916+0000 7fad8b59e700 1 --2- 172.21.15.64:0/1640398574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad8c106470 0x7fad8c106850 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fad7c009990 tx=0x7fad7c009ca0 comp rx=0 tx=0).stop 2023-09-13T03:24:57.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.916+0000 7fad8b59e700 1 -- 172.21.15.64:0/1640398574 shutdown_connections 2023-09-13T03:24:57.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.916+0000 7fad8b59e700 1 --2- 172.21.15.64:0/1640398574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad8c106470 0x7fad8c106850 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:57.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.916+0000 7fad8b59e700 1 --2- 172.21.15.64:0/1640398574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad8c100d90 0x7fad8c10d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:57.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.916+0000 7fad8b59e700 1 --2- 172.21.15.64:0/1640398574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad8c1003d0 0x7fad8c100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:57.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.916+0000 7fad8b59e700 1 -- 172.21.15.64:0/1640398574 >> 172.21.15.64:0/1640398574 conn(0x7fad8c0fbf00 msgr2=0x7fad8c0fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:24:57.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.916+0000 7fad8b59e700 1 -- 172.21.15.64:0/1640398574 shutdown_connections 2023-09-13T03:24:57.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.916+0000 7fad8b59e700 1 -- 172.21.15.64:0/1640398574 wait complete. 2023-09-13T03:24:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.917+0000 7fad8b59e700 1 Processor -- start 2023-09-13T03:24:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.917+0000 7fad8b59e700 1 -- start start 2023-09-13T03:24:57.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.918+0000 7fad8b59e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad8c1003d0 0x7fad8c196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:57.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.918+0000 7fad8b59e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad8c100d90 0x7fad8c196970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:57.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.918+0000 7fad8b59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad8c106470 0x7fad8c19add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:57.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.918+0000 7fad8b59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c19b410 con 0x7fad8c106470 2023-09-13T03:24:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.918+0000 7fad8b59e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c19b580 con 0x7fad8c100d90 2023-09-13T03:24:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.918+0000 7fad8b59e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c19b740 con 0x7fad8c1003d0 2023-09-13T03:24:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.918+0000 7fad8a59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad8c1003d0 0x7fad8c196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.918+0000 7fad89d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad8c100d90 0x7fad8c196970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.918+0000 7fad89d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad8c100d90 0x7fad8c196970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:51322/0 (socket says 172.21.15.64:51322) 2023-09-13T03:24:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.918+0000 7fad89d9b700 1 -- 172.21.15.64:0/1125995285 learned_addr learned my addr 172.21.15.64:0/1125995285 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:24:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.918+0000 7fad8a59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad8c1003d0 0x7fad8c196430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:50220/0 (socket says 172.21.15.64:50220) 2023-09-13T03:24:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.918+0000 7fad8ad9d700 1 --2- 172.21.15.64:0/1125995285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad8c106470 0x7fad8c19add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.919+0000 7fad89d9b700 1 -- 172.21.15.64:0/1125995285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad8c1003d0 msgr2=0x7fad8c196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.919+0000 7fad89d9b700 1 --2- 172.21.15.64:0/1125995285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad8c1003d0 0x7fad8c196430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:57.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.919+0000 7fad89d9b700 1 -- 172.21.15.64:0/1125995285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad8c106470 msgr2=0x7fad8c19add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:57.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.919+0000 7fad89d9b700 1 --2- 172.21.15.64:0/1125995285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad8c106470 0x7fad8c19add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:57.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.919+0000 7fad89d9b700 1 -- 172.21.15.64:0/1125995285 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad80009520 con 0x7fad8c100d90 2023-09-13T03:24:57.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.919+0000 7fad89d9b700 1 --2- 172.21.15.64:0/1125995285 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad8c100d90 0x7fad8c196970 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fad8000eb30 tx=0x7fad8000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:24:57.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.919+0000 7fad7b7fe700 1 -- 172.21.15.64:0/1125995285 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad80009880 con 0x7fad8c100d90 2023-09-13T03:24:57.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.919+0000 7fad7b7fe700 1 -- 172.21.15.64:0/1125995285 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad800044e0 con 0x7fad8c100d90 2023-09-13T03:24:57.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.919+0000 7fad8b59e700 1 -- 172.21.15.64:0/1125995285 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad7c0095f0 con 0x7fad8c100d90 2023-09-13T03:24:57.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.919+0000 7fad7b7fe700 1 -- 172.21.15.64:0/1125995285 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad80003d80 con 0x7fad8c100d90 2023-09-13T03:24:57.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.920+0000 7fad8b59e700 1 -- 172.21.15.64:0/1125995285 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad8c072830 con 0x7fad8c100d90 2023-09-13T03:24:57.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.923+0000 7fad7b7fe700 1 -- 172.21.15.64:0/1125995285 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fad80003f20 con 0x7fad8c100d90 2023-09-13T03:24:57.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.924+0000 7fada4578b80 1 Processor -- start 2023-09-13T03:24:57.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.924+0000 7fada4578b80 1 -- start start 2023-09-13T03:24:57.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.925+0000 7fada4578b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df95134290 0x55df95134670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:57.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.925+0000 7fada4578b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df95134bb0 0x55df9512e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:57.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.925+0000 7fada4578b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df9512e9a0 0x55df9512ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:57.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.925+0000 7fad7b7fe700 1 --2- 172.21.15.64:0/1125995285 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fad740a07b0 0x7fad740a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:57.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.925+0000 7fada4578b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df950876d0 con 0x55df9512e9a0 2023-09-13T03:24:57.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.925+0000 7fada4578b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df95084ff0 con 0x55df95134bb0 2023-09-13T03:24:57.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.925+0000 7fada4578b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df950808e0 con 0x55df95134290 2023-09-13T03:24:57.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.925+0000 7fad7b7fe700 1 -- 172.21.15.64:0/1125995285 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fad80014070 con 0x7fad8c100d90 2023-09-13T03:24:57.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.925+0000 7fad8ad9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df95134290 0x55df95134670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:57.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.925+0000 7fad8ad9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df95134290 0x55df95134670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:50228/0 (socket says 172.21.15.64:50228) 2023-09-13T03:24:57.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.925+0000 7fad8ad9d700 1 -- 172.21.15.64:0/3568182767 learned_addr learned my addr 172.21.15.64:0/3568182767 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:24:57.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.925+0000 7fad8a59c700 1 --2- 172.21.15.64:0/3568182767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df95134bb0 0x55df9512e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:57.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.925+0000 7fad8ad9d700 1 --2- 172.21.15.64:0/3568182767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df9512e9a0 0x55df9512ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:57.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.925+0000 7fad89d9b700 1 --2- 172.21.15.64:0/1125995285 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fad740a07b0 0x7fad740a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:57.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.926+0000 7fad8a59c700 1 -- 172.21.15.64:0/3568182767 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df95134290 msgr2=0x55df95134670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:24:57.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.926+0000 7fad8a59c700 1 --2- 172.21.15.64:0/3568182767 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df95134290 0x55df95134670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:57.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.926+0000 7fad8a59c700 1 -- 172.21.15.64:0/3568182767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df9512e9a0 msgr2=0x55df9512ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:24:57.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.926+0000 7fad8a59c700 1 --2- 172.21.15.64:0/3568182767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df9512e9a0 0x55df9512ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:57.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.926+0000 7fad8a59c700 1 -- 172.21.15.64:0/3568182767 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df94eacd80 con 0x55df95134bb0 2023-09-13T03:24:57.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.926+0000 7fad89d9b700 1 --2- 172.21.15.64:0/1125995285 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fad740a07b0 0x7fad740a2c70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fad800c9ca0 tx=0x7fad80009f60 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:24:57.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.926+0000 7fad8a59c700 1 --2- 172.21.15.64:0/3568182767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df95134bb0 0x55df9512e460 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fad7c000c00 tx=0x7fad7c005ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85b0dd63a66c3df0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:24:57.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.926+0000 7fad78ff9700 1 -- 172.21.15.64:0/3568182767 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c021070 con 0x55df95134bb0 2023-09-13T03:24:57.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.926+0000 7fad78ff9700 1 -- 172.21.15.64:0/3568182767 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad7c022040 con 0x55df95134bb0 2023-09-13T03:24:57.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.926+0000 7fad78ff9700 1 -- 172.21.15.64:0/3568182767 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c005310 con 0x55df95134bb0 2023-09-13T03:24:57.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.927+0000 7fada4578b80 1 -- 172.21.15.64:0/3568182767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df95134bb0 msgr2=0x55df9512e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:57.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.927+0000 7fada4578b80 1 --2- 172.21.15.64:0/3568182767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df95134bb0 0x55df9512e460 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fad7c000c00 tx=0x7fad7c005ea0 comp rx=0 tx=0).stop 2023-09-13T03:24:57.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.927+0000 7fada4578b80 1 -- 172.21.15.64:0/3568182767 shutdown_connections 2023-09-13T03:24:57.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.927+0000 7fada4578b80 1 --2- 172.21.15.64:0/3568182767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df9512e9a0 0x55df9512ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:57.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.927+0000 7fada4578b80 1 --2- 172.21.15.64:0/3568182767 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df95134290 0x55df95134670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:57.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.927+0000 7fada4578b80 1 --2- 172.21.15.64:0/3568182767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df95134bb0 0x55df9512e460 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:57.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.927+0000 7fada4578b80 1 -- 172.21.15.64:0/3568182767 >> 172.21.15.64:0/3568182767 conn(0x55df950a3030 msgr2=0x55df951296c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:24:57.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.927+0000 7fada4578b80 1 -- 172.21.15.64:0/3568182767 shutdown_connections 2023-09-13T03:24:57.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.927+0000 7fada4578b80 1 -- 172.21.15.64:0/3568182767 wait complete. 2023-09-13T03:24:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.928+0000 7fada4578b80 1 -- start start 2023-09-13T03:24:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.928+0000 7fada4578b80 1 Processor -- start 2023-09-13T03:24:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.929+0000 7fada4578b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df9512e9a0 0x55df95137ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.929+0000 7fada4578b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df95134290 0x55df951383e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.929+0000 7fada4578b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df95134bb0 0x55df95138920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.929+0000 7fada4578b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df9501eb10 con 0x55df95134bb0 2023-09-13T03:24:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.929+0000 7fada4578b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df9501e810 con 0x55df9512e9a0 2023-09-13T03:24:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.929+0000 7fada4578b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df94f028c0 con 0x55df95134290 2023-09-13T03:24:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.929+0000 7fada4578b80 10 client.0 _ll_register_callbacks cb 0x55df94f68a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:24:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.929+0000 7fad8a59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df95134290 0x55df951383e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.929+0000 7fad8a59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df95134290 0x55df951383e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:50236/0 (socket says 172.21.15.64:50236) 2023-09-13T03:24:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.930+0000 7fad8a59c700 1 -- 172.21.15.64:0/1666808619 learned_addr learned my addr 172.21.15.64:0/1666808619 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:24:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.930+0000 7fad8ad9d700 1 --2- 172.21.15.64:0/1666808619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df9512e9a0 0x55df95137ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.930+0000 7fad8ad9d700 1 --2- 172.21.15.64:0/1666808619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df95134bb0 0x55df95138920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.930+0000 7fad8a59c700 1 -- 172.21.15.64:0/1666808619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df9512e9a0 msgr2=0x55df95137ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:24:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.930+0000 7fad8a59c700 1 --2- 172.21.15.64:0/1666808619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df9512e9a0 0x55df95137ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.930+0000 7fad8a59c700 1 -- 172.21.15.64:0/1666808619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df95134bb0 msgr2=0x55df95138920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:24:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.930+0000 7fad8a59c700 1 --2- 172.21.15.64:0/1666808619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df95134bb0 0x55df95138920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.930+0000 7fad8a59c700 1 -- 172.21.15.64:0/1666808619 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad8c10dba0 con 0x55df95134290 2023-09-13T03:24:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.930+0000 7fad8a59c700 1 --2- 172.21.15.64:0/1666808619 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df95134290 0x55df951383e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fad7c0098e0 tx=0x7fad7c01cea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=606687b05513f400 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:24:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.930+0000 7fad71ffb700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:24:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.930+0000 7fad71ffb700 1 -- 172.21.15.64:0/1666808619 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c021040 con 0x55df95134290 2023-09-13T03:24:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.931+0000 7fada4578b80 1 -- 172.21.15.64:0/1666808619 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55df94fd8490 con 0x55df95134290 2023-09-13T03:24:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.931+0000 7fad71ffb700 1 -- 172.21.15.64:0/1666808619 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fad7c00bc60 con 0x55df95134290 2023-09-13T03:24:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.931+0000 7fad71ffb700 1 -- 172.21.15.64:0/1666808619 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad7c005bc0 con 0x55df95134290 2023-09-13T03:24:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.931+0000 7fada4578b80 10 client.27058 fetch_fsmap learned FSMap version 61 2023-09-13T03:24:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.931+0000 7fada4578b80 1 -- 172.21.15.64:0/1666808619 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x55df94e95c00 con 0x55df95134290 2023-09-13T03:24:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.931+0000 7fad71ffb700 1 -- 172.21.15.64:0/1666808619 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55df94fd8490 con 0x55df95134290 2023-09-13T03:24:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.931+0000 7fad71ffb700 1 -- 172.21.15.64:0/1666808619 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 61) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fad7c041560 con 0x55df95134290 2023-09-13T03:24:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.931+0000 7fada4578b80 10 client.27058 fetch_fsmap finished waiting for FSMap version 61 2023-09-13T03:24:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.931+0000 7fada4578b80 10 client.27058 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:24:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.931+0000 7fada4578b80 20 client.27058 populate_metadata read hostname 'smithi064' 2023-09-13T03:24:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.931+0000 7fada4578b80 1 --2- 172.21.15.64:0/1666808619 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55df951480d0 0x55df9514a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.931+0000 7fada4578b80 4 client.27058 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:24:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.931+0000 7fada4578b80 1 -- 172.21.15.64:0/1666808619 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df95095840 con 0x55df951480d0 2023-09-13T03:24:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.932+0000 7fad8ad9d700 1 --2- 172.21.15.64:0/1666808619 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55df951480d0 0x55df9514a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.932+0000 7fad8ad9d700 1 --2- 172.21.15.64:0/1666808619 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55df951480d0 0x55df9514a4f0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c5310a491b63a server_cookie=2f23971fd69fc2e7 in_seq=0 out_seq=0 2023-09-13T03:24:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.932+0000 7fad71ffb700 10 client.27058 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:24:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.933+0000 7fad71ffb700 1 -- 172.21.15.64:0/1666808619 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55df95095840 con 0x55df951480d0 2023-09-13T03:24:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.933+0000 7fad71ffb700 10 client.27058 handle_command_reply: tid=0 2023-09-13T03:24:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.933+0000 7fada4578b80 1 client.27058 shutdown 2023-09-13T03:24:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.933+0000 7fada4578b80 1 -- 172.21.15.64:0/1666808619 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df95134290 msgr2=0x55df951383e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:57.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.933+0000 7fada4578b80 1 --2- 172.21.15.64:0/1666808619 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df95134290 0x55df951383e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fad7c0098e0 tx=0x7fad7c01cea0 comp rx=0 tx=0).stop 2023-09-13T03:24:57.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.933+0000 7fada4578b80 1 -- 172.21.15.64:0/1666808619 shutdown_connections 2023-09-13T03:24:57.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.933+0000 7fada4578b80 1 --2- 172.21.15.64:0/1666808619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df9512e9a0 0x55df95137ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:57.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.933+0000 7fada4578b80 1 --2- 172.21.15.64:0/1666808619 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55df951480d0 0x55df9514a4f0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:57.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.933+0000 7fada4578b80 1 --2- 172.21.15.64:0/1666808619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df95134bb0 0x55df95138920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:57.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.933+0000 7fad71ffb700 0 client.27058 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:24:57.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.933+0000 7fada4578b80 1 --2- 172.21.15.64:0/1666808619 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df95134290 0x55df951383e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:57.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.933+0000 7fada4578b80 1 -- 172.21.15.64:0/1666808619 >> 172.21.15.64:0/1666808619 conn(0x55df950a3030 msgr2=0x55df95128900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:24:57.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.933+0000 7fada4578b80 1 -- 172.21.15.64:0/1666808619 shutdown_connections 2023-09-13T03:24:57.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.933+0000 7fada4578b80 1 -- 172.21.15.64:0/1666808619 wait complete. 2023-09-13T03:24:57.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.934+0000 7fada4578b80 20 client.27058 trim_cache size 0 max 16384 2023-09-13T03:24:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.950+0000 7fada4578b80 1 Processor -- start 2023-09-13T03:24:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.950+0000 7fada4578b80 1 -- start start 2023-09-13T03:24:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.950+0000 7fada4578b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df9512e9a0 0x55df9512e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.950+0000 7fada4578b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df95134290 0x55df9513f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.950+0000 7fada4578b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df95134bb0 0x55df95133670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.950+0000 7fada4578b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df9512a330 con 0x55df95134290 2023-09-13T03:24:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.950+0000 7fada4578b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df9512afc0 con 0x55df95134bb0 2023-09-13T03:24:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.950+0000 7fada4578b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df9512f720 con 0x55df9512e9a0 2023-09-13T03:24:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.950+0000 7fad8ad9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df9512e9a0 0x55df9512e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.950+0000 7fad8ad9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df9512e9a0 0x55df9512e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:50250/0 (socket says 172.21.15.64:50250) 2023-09-13T03:24:57.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.950+0000 7fad8ad9d700 1 -- 172.21.15.64:0/1676500959 learned_addr learned my addr 172.21.15.64:0/1676500959 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:24:57.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.950+0000 7fad8ad9d700 1 --2- 172.21.15.64:0/1676500959 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df95134bb0 0x55df95133670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:57.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.950+0000 7fad8a59c700 1 --2- 172.21.15.64:0/1676500959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df95134290 0x55df9513f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.951+0000 7fad8a59c700 1 -- 172.21.15.64:0/1676500959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df9512e9a0 msgr2=0x55df9512e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:24:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.951+0000 7fad8a59c700 1 --2- 172.21.15.64:0/1676500959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df9512e9a0 0x55df9512e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.951+0000 7fad8a59c700 1 -- 172.21.15.64:0/1676500959 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df95134bb0 msgr2=0x55df95133670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:24:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.951+0000 7fad8a59c700 1 --2- 172.21.15.64:0/1676500959 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df95134bb0 0x55df95133670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.951+0000 7fad8a59c700 1 -- 172.21.15.64:0/1676500959 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df95137cb0 con 0x55df95134290 2023-09-13T03:24:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.951+0000 7fad8a59c700 1 --2- 172.21.15.64:0/1676500959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df95134290 0x55df9513f5b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fad7c041920 tx=0x7fad7c041d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68f29bb37bcf9048 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:24:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.951+0000 7fad797fa700 1 -- 172.21.15.64:0/1676500959 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c021070 con 0x55df95134290 2023-09-13T03:24:57.956 INFO:tasks.workunit.client.0.smithi064.stdout: 1 308950 2.54 MB/sec execute 542 sec latency 15.752 ms 2023-09-13T03:24:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.951+0000 7fad797fa700 1 -- 172.21.15.64:0/1676500959 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad7c005bc0 con 0x55df95134290 2023-09-13T03:24:57.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.951+0000 7fad797fa700 1 -- 172.21.15.64:0/1676500959 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c0386c0 con 0x55df95134290 2023-09-13T03:24:57.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.952+0000 7fada4578b80 1 -- 172.21.15.64:0/1676500959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df95134290 msgr2=0x55df9513f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:57.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.952+0000 7fada4578b80 1 --2- 172.21.15.64:0/1676500959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df95134290 0x55df9513f5b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fad7c041920 tx=0x7fad7c041d00 comp rx=0 tx=0).stop 2023-09-13T03:24:57.959 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.952+0000 7fada4578b80 1 -- 172.21.15.64:0/1676500959 shutdown_connections 2023-09-13T03:24:57.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.952+0000 7fada4578b80 1 --2- 172.21.15.64:0/1676500959 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df95134bb0 0x55df95133670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:57.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.952+0000 7fada4578b80 1 --2- 172.21.15.64:0/1676500959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df9512e9a0 0x55df9512e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:57.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.952+0000 7fada4578b80 1 --2- 172.21.15.64:0/1676500959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df95134290 0x55df9513f5b0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:57.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.952+0000 7fada4578b80 1 -- 172.21.15.64:0/1676500959 >> 172.21.15.64:0/1676500959 conn(0x55df950a3030 msgr2=0x55df95132f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:24:57.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.952+0000 7fada4578b80 1 -- 172.21.15.64:0/1676500959 shutdown_connections 2023-09-13T03:24:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.952+0000 7fada4578b80 1 -- 172.21.15.64:0/1676500959 wait complete. 2023-09-13T03:24:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.953+0000 7fada4578b80 1 -- start start 2023-09-13T03:24:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.953+0000 7fada4578b80 1 Processor -- start 2023-09-13T03:24:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.953+0000 7fada4578b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df9512e9a0 0x55df95131000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.953+0000 7fada4578b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df95134290 0x55df95131540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:57.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.953+0000 7fada4578b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df95134bb0 0x55df9512fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:57.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.953+0000 7fada4578b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df95130230 con 0x55df95134290 2023-09-13T03:24:57.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.953+0000 7fada4578b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df951303a0 con 0x55df95134bb0 2023-09-13T03:24:57.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.953+0000 7fada4578b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df95130560 con 0x55df9512e9a0 2023-09-13T03:24:57.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.954+0000 7fada4578b80 10 client.0 _ll_register_callbacks cb 0x7fad84008630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:24:57.962 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.954+0000 7fad8ad9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df9512e9a0 0x55df95131000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.954+0000 7fad8ad9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df9512e9a0 0x55df95131000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:50254/0 (socket says 172.21.15.64:50254) 2023-09-13T03:24:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.954+0000 7fad8ad9d700 1 -- 172.21.15.64:0/4266064440 learned_addr learned my addr 172.21.15.64:0/4266064440 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:24:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.954+0000 7fad8a59c700 1 --2- 172.21.15.64:0/4266064440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df95134290 0x55df95131540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.954+0000 7fad8ad9d700 1 --2- 172.21.15.64:0/4266064440 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df95134bb0 0x55df9512fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.954+0000 7fad8ad9d700 1 -- 172.21.15.64:0/4266064440 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df95134bb0 msgr2=0x55df9512fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:24:57.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.954+0000 7fad8ad9d700 1 --2- 172.21.15.64:0/4266064440 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df95134bb0 0x55df9512fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:57.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.954+0000 7fad8ad9d700 1 -- 172.21.15.64:0/4266064440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df95134290 msgr2=0x55df95131540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:24:57.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.954+0000 7fad8ad9d700 1 --2- 172.21.15.64:0/4266064440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df95134290 0x55df95131540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:57.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.954+0000 7fad8ad9d700 1 -- 172.21.15.64:0/4266064440 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df94e95c00 con 0x55df9512e9a0 2023-09-13T03:24:57.964 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.954+0000 7fad8a59c700 1 --2- 172.21.15.64:0/4266064440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df95134290 0x55df95131540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:24:57.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.954+0000 7fad8ad9d700 1 --2- 172.21.15.64:0/4266064440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df9512e9a0 0x55df95131000 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fad84025bf0 tx=0x7fad7c00b630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbf788a6007e4693 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:24:57.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.954+0000 7fad71ffb700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:24:57.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.955+0000 7fad71ffb700 1 -- 172.21.15.64:0/4266064440 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad84020040 con 0x55df9512e9a0 2023-09-13T03:24:57.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.955+0000 7fada4578b80 1 -- 172.21.15.64:0/4266064440 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55df950798c0 con 0x55df9512e9a0 2023-09-13T03:24:57.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.955+0000 7fad71ffb700 1 -- 172.21.15.64:0/4266064440 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fad84008ef0 con 0x55df9512e9a0 2023-09-13T03:24:57.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.955+0000 7fad71ffb700 1 -- 172.21.15.64:0/4266064440 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad8400e040 con 0x55df9512e9a0 2023-09-13T03:24:57.966 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.955+0000 7fada4578b80 10 client.27064 fetch_fsmap learned FSMap version 61 2023-09-13T03:24:57.966 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.955+0000 7fada4578b80 1 -- 172.21.15.64:0/4266064440 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x55df94ed31f0 con 0x55df9512e9a0 2023-09-13T03:24:57.966 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.955+0000 7fad71ffb700 1 -- 172.21.15.64:0/4266064440 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55df950798c0 con 0x55df9512e9a0 2023-09-13T03:24:57.966 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.955+0000 7fad71ffb700 1 -- 172.21.15.64:0/4266064440 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 61) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fad84032760 con 0x55df9512e9a0 2023-09-13T03:24:57.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.955+0000 7fada4578b80 10 client.27064 fetch_fsmap finished waiting for FSMap version 61 2023-09-13T03:24:57.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.955+0000 7fada4578b80 10 client.27064 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:24:57.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.956+0000 7fada4578b80 20 client.27064 populate_metadata read hostname 'smithi064' 2023-09-13T03:24:57.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.956+0000 7fada4578b80 1 --2- 172.21.15.64:0/4266064440 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55df9519cdd0 0x55df9519f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:24:57.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.956+0000 7fada4578b80 4 client.27064 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:24:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.956+0000 7fada4578b80 1 -- 172.21.15.64:0/4266064440 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55df9503a050 con 0x55df9519cdd0 2023-09-13T03:24:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.956+0000 7fad8a59c700 1 --2- 172.21.15.64:0/4266064440 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55df9519cdd0 0x55df9519f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:24:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.956+0000 7fad8a59c700 1 --2- 172.21.15.64:0/4266064440 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55df9519cdd0 0x55df9519f1f0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b6152fcf79b3aef server_cookie=aaea9cd67615d603 in_seq=0 out_seq=0 2023-09-13T03:24:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:57.956+0000 7fad71ffb700 10 client.27064 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:24:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:58.000+0000 7fad71ffb700 1 -- 172.21.15.64:0/4266064440 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55df9503a050 con 0x55df9519cdd0 2023-09-13T03:24:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:58.000+0000 7fad71ffb700 10 client.27064 handle_command_reply: tid=0 2023-09-13T03:24:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:58.000+0000 7fada4578b80 1 client.27064 shutdown 2023-09-13T03:24:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:58.000+0000 7fada4578b80 1 -- 172.21.15.64:0/4266064440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df9512e9a0 msgr2=0x55df95131000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:58.000+0000 7fada4578b80 1 --2- 172.21.15.64:0/4266064440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df9512e9a0 0x55df95131000 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fad84025bf0 tx=0x7fad7c00b630 comp rx=0 tx=0).stop 2023-09-13T03:24:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:58.000+0000 7fada4578b80 1 -- 172.21.15.64:0/4266064440 shutdown_connections 2023-09-13T03:24:58.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:58.000+0000 7fada4578b80 1 --2- 172.21.15.64:0/4266064440 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df95134bb0 0x55df9512fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:58.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:58.000+0000 7fada4578b80 1 --2- 172.21.15.64:0/4266064440 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55df9519cdd0 0x55df9519f1f0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:58.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:58.000+0000 7fada4578b80 1 --2- 172.21.15.64:0/4266064440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df9512e9a0 0x55df95131000 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:58.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:58.000+0000 7fada4578b80 1 --2- 172.21.15.64:0/4266064440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df95134290 0x55df95131540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:58.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:58.000+0000 7fad71ffb700 0 client.27064 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:24:58.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:58.000+0000 7fada4578b80 1 -- 172.21.15.64:0/4266064440 >> 172.21.15.64:0/4266064440 conn(0x55df950a3030 msgr2=0x55df95135820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:24:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:58.001+0000 7fada4578b80 1 -- 172.21.15.64:0/4266064440 shutdown_connections 2023-09-13T03:24:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:58.001+0000 7fada4578b80 1 -- 172.21.15.64:0/4266064440 wait complete. 2023-09-13T03:24:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:58.001+0000 7fada4578b80 20 client.27064 trim_cache size 0 max 16384 2023-09-13T03:24:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:24:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:24:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:24:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:24:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:58.002+0000 7fad8b59e700 1 -- 172.21.15.64:0/1125995285 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fad740a07b0 msgr2=0x7fad740a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:58.002+0000 7fad8b59e700 1 --2- 172.21.15.64:0/1125995285 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fad740a07b0 0x7fad740a2c70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fad800c9ca0 tx=0x7fad80009f60 comp rx=0 tx=0).stop 2023-09-13T03:24:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:58.002+0000 7fad8b59e700 1 -- 172.21.15.64:0/1125995285 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad8c100d90 msgr2=0x7fad8c196970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:24:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:58.002+0000 7fad8b59e700 1 --2- 172.21.15.64:0/1125995285 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad8c100d90 0x7fad8c196970 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fad8000eb30 tx=0x7fad8000ee40 comp rx=0 tx=0).stop 2023-09-13T03:24:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:58.002+0000 7fad8b59e700 1 -- 172.21.15.64:0/1125995285 shutdown_connections 2023-09-13T03:24:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:58.002+0000 7fad8b59e700 1 --2- 172.21.15.64:0/1125995285 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fad740a07b0 0x7fad740a2c70 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:58.002+0000 7fad8b59e700 1 --2- 172.21.15.64:0/1125995285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad8c106470 0x7fad8c19add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:58.002+0000 7fad8b59e700 1 --2- 172.21.15.64:0/1125995285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad8c1003d0 0x7fad8c196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:58.002+0000 7fad8b59e700 1 --2- 172.21.15.64:0/1125995285 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad8c100d90 0x7fad8c196970 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:24:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:58.002+0000 7fad8b59e700 1 -- 172.21.15.64:0/1125995285 >> 172.21.15.64:0/1125995285 conn(0x7fad8c0fbf00 msgr2=0x7fad8c0fc520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:24:58.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:58.002+0000 7fad8b59e700 1 -- 172.21.15.64:0/1125995285 shutdown_connections 2023-09-13T03:24:58.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:24:58.002+0000 7fad8b59e700 1 -- 172.21.15.64:0/1125995285 wait complete. 2023-09-13T03:24:58.952 INFO:tasks.workunit.client.0.smithi064.stdout: 1 310143 2.54 MB/sec execute 543 sec latency 39.495 ms 2023-09-13T03:24:59.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:24:58 smithi170 ceph-mon[134328]: pgmap v367: 97 pgs: 97 active+clean; 137 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 187 op/s 2023-09-13T03:24:59.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:24:58 smithi064 ceph-mon[97675]: pgmap v367: 97 pgs: 97 active+clean; 137 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 187 op/s 2023-09-13T03:24:59.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:24:58 smithi142 ceph-mon[134082]: pgmap v367: 97 pgs: 97 active+clean; 137 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 187 op/s 2023-09-13T03:24:59.952 INFO:tasks.workunit.client.0.smithi064.stdout: 1 312102 2.56 MB/sec execute 544 sec latency 50.560 ms 2023-09-13T03:25:00.952 INFO:tasks.workunit.client.0.smithi064.stdout: 1 313063 2.56 MB/sec execute 545 sec latency 26.558 ms 2023-09-13T03:25:01.159 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:00 smithi170 ceph-mon[134328]: pgmap v368: 97 pgs: 97 active+clean; 137 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 187 op/s 2023-09-13T03:25:01.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:00 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:25:01.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:00 smithi064 ceph-mon[97675]: pgmap v368: 97 pgs: 97 active+clean; 137 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 187 op/s 2023-09-13T03:25:01.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:00 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:25:01.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:00 smithi142 ceph-mon[134082]: pgmap v368: 97 pgs: 97 active+clean; 137 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 187 op/s 2023-09-13T03:25:01.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:00 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:25:01.952 INFO:tasks.workunit.client.0.smithi064.stdout: 1 313063 2.56 MB/sec execute 546 sec latency 1026.721 ms 2023-09-13T03:25:02.952 INFO:tasks.workunit.client.0.smithi064.stdout: 1 313063 2.55 MB/sec execute 547 sec latency 2026.851 ms 2023-09-13T03:25:03.162 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:02 smithi170 ceph-mon[134328]: pgmap v369: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.2 MiB/s wr, 298 op/s 2023-09-13T03:25:03.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:02 smithi064 ceph-mon[97675]: pgmap v369: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.2 MiB/s wr, 298 op/s 2023-09-13T03:25:03.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:02 smithi142 ceph-mon[134082]: pgmap v369: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.2 MiB/s wr, 298 op/s 2023-09-13T03:25:03.952 INFO:tasks.workunit.client.0.smithi064.stdout: 1 313188 2.55 MB/sec execute 548 sec latency 2617.777 ms 2023-09-13T03:25:04.953 INFO:tasks.workunit.client.0.smithi064.stdout: 1 314731 2.56 MB/sec execute 549 sec latency 26.762 ms 2023-09-13T03:25:05.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:04 smithi170 ceph-mon[134328]: pgmap v370: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 211 op/s 2023-09-13T03:25:05.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:04 smithi064 ceph-mon[97675]: pgmap v370: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 211 op/s 2023-09-13T03:25:05.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:04 smithi142 ceph-mon[134082]: pgmap v370: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 211 op/s 2023-09-13T03:25:05.952 INFO:tasks.workunit.client.0.smithi064.stdout: 1 316153 2.57 MB/sec execute 550 sec latency 11.585 ms 2023-09-13T03:25:06.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:05 smithi170 ceph-mon[134328]: osdmap e97: 12 total, 12 up, 12 in 2023-09-13T03:25:06.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:05 smithi064 ceph-mon[97675]: osdmap e97: 12 total, 12 up, 12 in 2023-09-13T03:25:06.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:05 smithi142 ceph-mon[134082]: osdmap e97: 12 total, 12 up, 12 in 2023-09-13T03:25:06.952 INFO:tasks.workunit.client.0.smithi064.stdout: 1 317363 2.58 MB/sec execute 551 sec latency 22.032 ms 2023-09-13T03:25:07.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:06 smithi170 ceph-mon[134328]: pgmap v372: 97 pgs: 97 active+clean; 136 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 243 op/s 2023-09-13T03:25:07.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:06 smithi064 ceph-mon[97675]: pgmap v372: 97 pgs: 97 active+clean; 136 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 243 op/s 2023-09-13T03:25:07.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:06 smithi142 ceph-mon[134082]: pgmap v372: 97 pgs: 97 active+clean; 136 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 243 op/s 2023-09-13T03:25:07.953 INFO:tasks.workunit.client.0.smithi064.stdout: 1 319336 2.59 MB/sec execute 552 sec latency 20.833 ms 2023-09-13T03:25:08.953 INFO:tasks.workunit.client.0.smithi064.stdout: 1 320425 2.60 MB/sec execute 553 sec latency 19.844 ms 2023-09-13T03:25:09.159 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:08 smithi170 ceph-mon[134328]: pgmap v373: 97 pgs: 97 active+clean; 136 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 243 op/s 2023-09-13T03:25:09.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:08 smithi064 ceph-mon[97675]: pgmap v373: 97 pgs: 97 active+clean; 136 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 243 op/s 2023-09-13T03:25:09.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:08 smithi142 ceph-mon[134082]: pgmap v373: 97 pgs: 97 active+clean; 136 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 243 op/s 2023-09-13T03:25:09.953 INFO:tasks.workunit.client.0.smithi064.stdout: 1 321387 2.60 MB/sec execute 554 sec latency 49.267 ms 2023-09-13T03:25:10.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:10 smithi142 ceph-mon[134082]: osdmap e98: 12 total, 12 up, 12 in 2023-09-13T03:25:10.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:10 smithi142 ceph-mon[134082]: pgmap v375: 97 pgs: 97 active+clean; 136 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 1.3 MiB/s wr, 138 op/s 2023-09-13T03:25:10.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:10 smithi170 ceph-mon[134328]: osdmap e98: 12 total, 12 up, 12 in 2023-09-13T03:25:10.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:10 smithi170 ceph-mon[134328]: pgmap v375: 97 pgs: 97 active+clean; 136 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 1.3 MiB/s wr, 138 op/s 2023-09-13T03:25:10.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:10 smithi064 ceph-mon[97675]: osdmap e98: 12 total, 12 up, 12 in 2023-09-13T03:25:10.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:10 smithi064 ceph-mon[97675]: pgmap v375: 97 pgs: 97 active+clean; 136 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 1.3 MiB/s wr, 138 op/s 2023-09-13T03:25:10.952 INFO:tasks.workunit.client.0.smithi064.stdout: 1 323083 2.62 MB/sec execute 555 sec latency 52.479 ms 2023-09-13T03:25:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.156+0000 7faef87ff700 1 -- 172.21.15.64:0/3136367969 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faef0100530 msgr2=0x7faef01009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:25:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.156+0000 7faef87ff700 1 --2- 172.21.15.64:0/3136367969 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faef0100530 0x7faef01009b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7faee0009990 tx=0x7faee0009ca0 comp rx=0 tx=0).stop 2023-09-13T03:25:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.156+0000 7faef87ff700 1 -- 172.21.15.64:0/3136367969 shutdown_connections 2023-09-13T03:25:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.156+0000 7faef87ff700 1 --2- 172.21.15.64:0/3136367969 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faef01065f0 0x7faef01069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.156+0000 7faef87ff700 1 --2- 172.21.15.64:0/3136367969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faef0100ef0 0x7faef010d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.156+0000 7faef87ff700 1 --2- 172.21.15.64:0/3136367969 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faef0100530 0x7faef01009b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.156+0000 7faef87ff700 1 -- 172.21.15.64:0/3136367969 >> 172.21.15.64:0/3136367969 conn(0x7faef00fbfc0 msgr2=0x7faef00fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:25:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.156+0000 7faef87ff700 1 -- 172.21.15.64:0/3136367969 shutdown_connections 2023-09-13T03:25:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.156+0000 7faef87ff700 1 -- 172.21.15.64:0/3136367969 wait complete. 2023-09-13T03:25:11.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.157+0000 7faef87ff700 1 Processor -- start 2023-09-13T03:25:11.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.157+0000 7faef87ff700 1 -- start start 2023-09-13T03:25:11.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.158+0000 7faef87ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faef0100530 0x7faef0196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:11.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.158+0000 7faef87ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faef0100ef0 0x7faef0196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:11.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.158+0000 7faef87ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faef01065f0 0x7faef0199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:11.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.158+0000 7faef87ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef019a580 con 0x7faef01065f0 2023-09-13T03:25:11.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.158+0000 7faef87ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef019a6f0 con 0x7faef0100ef0 2023-09-13T03:25:11.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.158+0000 7faef87ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef019a8b0 con 0x7faef0100530 2023-09-13T03:25:11.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.158+0000 7faef659b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faef0100530 0x7faef0196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:11.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.158+0000 7faef659b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faef0100530 0x7faef0196690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:45022/0 (socket says 172.21.15.64:45022) 2023-09-13T03:25:11.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.158+0000 7faef659b700 1 -- 172.21.15.64:0/964627600 learned_addr learned my addr 172.21.15.64:0/964627600 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:25:11.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.158+0000 7faef5d9a700 1 --2- 172.21.15.64:0/964627600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faef0100ef0 0x7faef0196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:11.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.159+0000 7faef6d9c700 1 --2- 172.21.15.64:0/964627600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faef01065f0 0x7faef0199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:11.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.159+0000 7faef5d9a700 1 -- 172.21.15.64:0/964627600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faef0100530 msgr2=0x7faef0196690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:25:11.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.159+0000 7faef5d9a700 1 --2- 172.21.15.64:0/964627600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faef0100530 0x7faef0196690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:11.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.159+0000 7faef5d9a700 1 -- 172.21.15.64:0/964627600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faef01065f0 msgr2=0x7faef0199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:25:11.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.159+0000 7faef5d9a700 1 --2- 172.21.15.64:0/964627600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faef01065f0 0x7faef0199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:11.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.159+0000 7faef5d9a700 1 -- 172.21.15.64:0/964627600 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee00095f0 con 0x7faef0100ef0 2023-09-13T03:25:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.159+0000 7faef6d9c700 1 --2- 172.21.15.64:0/964627600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faef01065f0 0x7faef0199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T03:25:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.159+0000 7faef659b700 1 --2- 172.21.15.64:0/964627600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faef0100530 0x7faef0196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:25:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.159+0000 7faef5d9a700 1 --2- 172.21.15.64:0/964627600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faef0100ef0 0x7faef0196bd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7faee0000c00 tx=0x7faee0005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:25:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.159+0000 7faee77fe700 1 -- 172.21.15.64:0/964627600 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee001d070 con 0x7faef0100ef0 2023-09-13T03:25:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.159+0000 7faee77fe700 1 -- 172.21.15.64:0/964627600 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faee0021410 con 0x7faef0100ef0 2023-09-13T03:25:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.159+0000 7faef87ff700 1 -- 172.21.15.64:0/964627600 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faef019ab30 con 0x7faef0100ef0 2023-09-13T03:25:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.159+0000 7faee77fe700 1 -- 172.21.15.64:0/964627600 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee0021ea0 con 0x7faef0100ef0 2023-09-13T03:25:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.160+0000 7faef87ff700 1 -- 172.21.15.64:0/964627600 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faef019ae00 con 0x7faef0100ef0 2023-09-13T03:25:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.163+0000 7faee77fe700 1 -- 172.21.15.64:0/964627600 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7faee0021580 con 0x7faef0100ef0 2023-09-13T03:25:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.164+0000 7faf0c1d6b80 1 Processor -- start 2023-09-13T03:25:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.164+0000 7faf0c1d6b80 1 -- start start 2023-09-13T03:25:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.164+0000 7faf0c1d6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565fdf4d1a0 0x5565fdf4d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.164+0000 7faf0c1d6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565fdf47300 0x5565fdf47780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.164+0000 7faf0c1d6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565fdf47cc0 0x5565fdf53f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.164+0000 7faf0c1d6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565fdea06d0 con 0x5565fdf47300 2023-09-13T03:25:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.164+0000 7faf0c1d6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565fde9dff0 con 0x5565fdf47cc0 2023-09-13T03:25:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.164+0000 7faf0c1d6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565fde998e0 con 0x5565fdf4d1a0 2023-09-13T03:25:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.165+0000 7faef659b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565fdf47300 0x5565fdf47780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.165+0000 7faef6d9c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565fdf4d1a0 0x5565fdf4d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.165+0000 7faef6d9c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565fdf4d1a0 0x5565fdf4d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:45038/0 (socket says 172.21.15.64:45038) 2023-09-13T03:25:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.165+0000 7faef659b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565fdf47300 0x5565fdf47780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56104/0 (socket says 172.21.15.64:56104) 2023-09-13T03:25:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.165+0000 7faef6d9c700 1 -- 172.21.15.64:0/220197015 learned_addr learned my addr 172.21.15.64:0/220197015 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:25:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.165+0000 7faee77fe700 1 --2- 172.21.15.64:0/964627600 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7faedc0a07b0 0x7faedc0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:11.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.165+0000 7faef5d9a700 1 --2- 172.21.15.64:0/220197015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565fdf47cc0 0x5565fdf53f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:11.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.165+0000 7faef6d9c700 1 --2- 172.21.15.64:0/964627600 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7faedc0a07b0 0x7faedc0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:11.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.165+0000 7faee77fe700 1 -- 172.21.15.64:0/964627600 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7faee00cd1d0 con 0x7faef0100ef0 2023-09-13T03:25:11.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.165+0000 7faef659b700 1 -- 172.21.15.64:0/220197015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565fdf4d1a0 msgr2=0x5565fdf4d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:25:11.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.165+0000 7faef659b700 1 --2- 172.21.15.64:0/220197015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565fdf4d1a0 0x5565fdf4d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:11.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.165+0000 7faef659b700 1 -- 172.21.15.64:0/220197015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565fdf47cc0 msgr2=0x5565fdf53f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:25:11.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.165+0000 7faef659b700 1 --2- 172.21.15.64:0/220197015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565fdf47cc0 0x5565fdf53f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:11.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.165+0000 7faef659b700 1 -- 172.21.15.64:0/220197015 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565fdcc5d80 con 0x5565fdf47300 2023-09-13T03:25:11.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.166+0000 7faef659b700 1 --2- 172.21.15.64:0/220197015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565fdf47300 0x5565fdf47780 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7faeec0114c0 tx=0x7faeec0117d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af0ad23522b41fd server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:25:11.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.166+0000 7faef6d9c700 1 --2- 172.21.15.64:0/964627600 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7faedc0a07b0 0x7faedc0a2c70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7faee8009550 tx=0x7faee8004b40 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:25:11.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.166+0000 7faee4ff9700 1 -- 172.21.15.64:0/220197015 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeec004230 con 0x5565fdf47300 2023-09-13T03:25:11.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.166+0000 7faee4ff9700 1 -- 172.21.15.64:0/220197015 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faeec004390 con 0x5565fdf47300 2023-09-13T03:25:11.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.166+0000 7faee4ff9700 1 -- 172.21.15.64:0/220197015 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeec00e880 con 0x5565fdf47300 2023-09-13T03:25:11.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.167+0000 7faf0c1d6b80 1 -- 172.21.15.64:0/220197015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565fdf47300 msgr2=0x5565fdf47780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:25:11.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.167+0000 7faf0c1d6b80 1 --2- 172.21.15.64:0/220197015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565fdf47300 0x5565fdf47780 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7faeec0114c0 tx=0x7faeec0117d0 comp rx=0 tx=0).stop 2023-09-13T03:25:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.167+0000 7faf0c1d6b80 1 -- 172.21.15.64:0/220197015 shutdown_connections 2023-09-13T03:25:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.167+0000 7faf0c1d6b80 1 --2- 172.21.15.64:0/220197015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565fdf47cc0 0x5565fdf53f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.167+0000 7faf0c1d6b80 1 --2- 172.21.15.64:0/220197015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565fdf4d1a0 0x5565fdf4d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.167+0000 7faf0c1d6b80 1 --2- 172.21.15.64:0/220197015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565fdf47300 0x5565fdf47780 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:11.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.167+0000 7faf0c1d6b80 1 -- 172.21.15.64:0/220197015 >> 172.21.15.64:0/220197015 conn(0x5565fdebc030 msgr2=0x5565fdf42560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:25:11.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.167+0000 7faf0c1d6b80 1 -- 172.21.15.64:0/220197015 shutdown_connections 2023-09-13T03:25:11.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.167+0000 7faf0c1d6b80 1 -- 172.21.15.64:0/220197015 wait complete. 2023-09-13T03:25:11.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.167+0000 7faf0c1d6b80 1 -- start start 2023-09-13T03:25:11.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.168+0000 7faf0c1d6b80 1 Processor -- start 2023-09-13T03:25:11.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.168+0000 7faf0c1d6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565fdf47300 0x5565fdf53d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:11.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.168+0000 7faf0c1d6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565fdf47cc0 0x5565fdf50fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:11.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.168+0000 7faf0c1d6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565fdf4d1a0 0x5565fdf514e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:11.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.168+0000 7faf0c1d6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565fde37b10 con 0x5565fdf4d1a0 2023-09-13T03:25:11.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.168+0000 7faf0c1d6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565fde37810 con 0x5565fdf47cc0 2023-09-13T03:25:11.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.168+0000 7faf0c1d6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565fdd1b8c0 con 0x5565fdf47300 2023-09-13T03:25:11.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.168+0000 7faf0c1d6b80 10 client.0 _ll_register_callbacks cb 0x5565fdd81a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:25:11.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.168+0000 7faef5d9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565fdf47cc0 0x5565fdf50fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:11.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.168+0000 7faef659b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565fdf47300 0x5565fdf53d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:11.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.168+0000 7faef6d9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565fdf4d1a0 0x5565fdf514e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:11.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.168+0000 7faef5d9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565fdf47cc0 0x5565fdf50fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:38816/0 (socket says 172.21.15.64:38816) 2023-09-13T03:25:11.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.168+0000 7faef5d9a700 1 -- 172.21.15.64:0/2049233958 learned_addr learned my addr 172.21.15.64:0/2049233958 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:25:11.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.169+0000 7faef5d9a700 1 -- 172.21.15.64:0/2049233958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565fdf47300 msgr2=0x5565fdf53d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:25:11.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.169+0000 7faef5d9a700 1 --2- 172.21.15.64:0/2049233958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565fdf47300 0x5565fdf53d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:11.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.169+0000 7faef5d9a700 1 -- 172.21.15.64:0/2049233958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565fdf4d1a0 msgr2=0x5565fdf514e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:25:11.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.169+0000 7faef5d9a700 1 --2- 172.21.15.64:0/2049233958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565fdf4d1a0 0x5565fdf514e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:11.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.169+0000 7faef5d9a700 1 -- 172.21.15.64:0/2049233958 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeec009ce0 con 0x5565fdf47cc0 2023-09-13T03:25:11.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.169+0000 7faef5d9a700 1 --2- 172.21.15.64:0/2049233958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565fdf47cc0 0x5565fdf50fa0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7faee0097140 tx=0x7faee0069d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bd62aeb03e103e9 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:25:11.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.169+0000 7faed9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:25:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.169+0000 7faed9ffb700 1 -- 172.21.15.64:0/2049233958 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee009ab30 con 0x5565fdf47cc0 2023-09-13T03:25:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.169+0000 7faed9ffb700 1 -- 172.21.15.64:0/2049233958 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7faee009add0 con 0x5565fdf47cc0 2023-09-13T03:25:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.169+0000 7faf0c1d6b80 1 -- 172.21.15.64:0/2049233958 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5565fddf1490 con 0x5565fdf47cc0 2023-09-13T03:25:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.169+0000 7faed9ffb700 1 -- 172.21.15.64:0/2049233958 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7faee009a030 con 0x5565fdf47cc0 2023-09-13T03:25:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.169+0000 7faf0c1d6b80 10 client.27092 fetch_fsmap learned FSMap version 62 2023-09-13T03:25:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.169+0000 7faf0c1d6b80 1 -- 172.21.15.64:0/2049233958 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) v3 -- 0x5565fdcaec00 con 0x5565fdf47cc0 2023-09-13T03:25:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.170+0000 7faed9ffb700 1 -- 172.21.15.64:0/2049233958 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5565fddf1490 con 0x5565fdf47cc0 2023-09-13T03:25:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.170+0000 7faed9ffb700 1 -- 172.21.15.64:0/2049233958 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 62) v1 ==== 7914+0+0 (secure 0 0 0) 0x7faee009e180 con 0x5565fdf47cc0 2023-09-13T03:25:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.170+0000 7faf0c1d6b80 10 client.27092 fetch_fsmap finished waiting for FSMap version 62 2023-09-13T03:25:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.170+0000 7faf0c1d6b80 10 client.27092 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:25:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.170+0000 7faf0c1d6b80 20 client.27092 populate_metadata read hostname 'smithi064' 2023-09-13T03:25:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.170+0000 7faf0c1d6b80 1 --2- 172.21.15.64:0/2049233958 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5565fdf60f90 0x5565fdf633b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.170+0000 7faf0c1d6b80 4 client.27092 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:25:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.170+0000 7faf0c1d6b80 1 -- 172.21.15.64:0/2049233958 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5565fdeae840 con 0x5565fdf60f90 2023-09-13T03:25:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.170+0000 7faef659b700 1 --2- 172.21.15.64:0/2049233958 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5565fdf60f90 0x5565fdf633b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.171+0000 7faef659b700 1 --2- 172.21.15.64:0/2049233958 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5565fdf60f90 0x5565fdf633b0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2b1ba8f99db270f server_cookie=d3987b0e310f2594 in_seq=0 out_seq=0 2023-09-13T03:25:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.171+0000 7faed9ffb700 10 client.27092 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:25:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.171+0000 7faed9ffb700 1 -- 172.21.15.64:0/2049233958 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5565fdeae840 con 0x5565fdf60f90 2023-09-13T03:25:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.171+0000 7faed9ffb700 10 client.27092 handle_command_reply: tid=0 2023-09-13T03:25:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.171+0000 7faf0c1d6b80 1 client.27092 shutdown 2023-09-13T03:25:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.172+0000 7faf0c1d6b80 1 -- 172.21.15.64:0/2049233958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565fdf47cc0 msgr2=0x5565fdf50fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:25:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.172+0000 7faf0c1d6b80 1 --2- 172.21.15.64:0/2049233958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565fdf47cc0 0x5565fdf50fa0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7faee0097140 tx=0x7faee0069d30 comp rx=0 tx=0).stop 2023-09-13T03:25:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.172+0000 7faf0c1d6b80 1 -- 172.21.15.64:0/2049233958 shutdown_connections 2023-09-13T03:25:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.172+0000 7faf0c1d6b80 1 --2- 172.21.15.64:0/2049233958 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5565fdf60f90 0x5565fdf633b0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.172+0000 7faf0c1d6b80 1 --2- 172.21.15.64:0/2049233958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565fdf4d1a0 0x5565fdf514e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.172+0000 7faf0c1d6b80 1 --2- 172.21.15.64:0/2049233958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565fdf47300 0x5565fdf53d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.172+0000 7faf0c1d6b80 1 --2- 172.21.15.64:0/2049233958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565fdf47cc0 0x5565fdf50fa0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.172+0000 7faed9ffb700 0 client.27092 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:25:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.172+0000 7faf0c1d6b80 1 -- 172.21.15.64:0/2049233958 >> 172.21.15.64:0/2049233958 conn(0x5565fdebc030 msgr2=0x5565fdf417a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:25:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.172+0000 7faf0c1d6b80 1 -- 172.21.15.64:0/2049233958 shutdown_connections 2023-09-13T03:25:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.172+0000 7faf0c1d6b80 1 -- 172.21.15.64:0/2049233958 wait complete. 2023-09-13T03:25:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.172+0000 7faf0c1d6b80 20 client.27092 trim_cache size 0 max 16384 2023-09-13T03:25:11.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.188+0000 7faf0c1d6b80 1 Processor -- start 2023-09-13T03:25:11.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.188+0000 7faf0c1d6b80 1 -- start start 2023-09-13T03:25:11.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.189+0000 7faf0c1d6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565fdf47300 0x5565fdf4c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:11.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.189+0000 7faf0c1d6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565fdf47cc0 0x5565fdf58520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:11.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.189+0000 7faf0c1d6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565fdf4d1a0 0x5565fdf49b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:11.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.189+0000 7faf0c1d6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565fdeae840 con 0x5565fdf47300 2023-09-13T03:25:11.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.189+0000 7faf0c1d6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565fddf1490 con 0x5565fdf4d1a0 2023-09-13T03:25:11.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.189+0000 7faf0c1d6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565fdf3c8a0 con 0x5565fdf47cc0 2023-09-13T03:25:11.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.189+0000 7faef5d9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565fdf47cc0 0x5565fdf58520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:11.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.189+0000 7faef659b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565fdf47300 0x5565fdf4c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:11.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.189+0000 7faef6d9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565fdf4d1a0 0x5565fdf49b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:11.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.189+0000 7faef5d9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565fdf47cc0 0x5565fdf58520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:45062/0 (socket says 172.21.15.64:45062) 2023-09-13T03:25:11.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.189+0000 7faef5d9a700 1 -- 172.21.15.64:0/3861939968 learned_addr learned my addr 172.21.15.64:0/3861939968 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:25:11.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.189+0000 7faef5d9a700 1 -- 172.21.15.64:0/3861939968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565fdf4d1a0 msgr2=0x5565fdf49b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:25:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.189+0000 7faef5d9a700 1 --2- 172.21.15.64:0/3861939968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565fdf4d1a0 0x5565fdf49b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.189+0000 7faef5d9a700 1 -- 172.21.15.64:0/3861939968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565fdf47300 msgr2=0x5565fdf4c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:25:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.189+0000 7faef5d9a700 1 --2- 172.21.15.64:0/3861939968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565fdf47300 0x5565fdf4c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.189+0000 7faef5d9a700 1 -- 172.21.15.64:0/3861939968 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565fdcec1f0 con 0x5565fdf47cc0 2023-09-13T03:25:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.190+0000 7faef5d9a700 1 --2- 172.21.15.64:0/3861939968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565fdf47cc0 0x5565fdf58520 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7faee0009990 tx=0x7faee0021b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c914eebad639d028 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:25:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.190+0000 7faee57fa700 1 -- 172.21.15.64:0/3861939968 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee009a030 con 0x5565fdf47cc0 2023-09-13T03:25:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.190+0000 7faee57fa700 1 -- 172.21.15.64:0/3861939968 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faee00cad70 con 0x5565fdf47cc0 2023-09-13T03:25:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.190+0000 7faee57fa700 1 -- 172.21.15.64:0/3861939968 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee009e500 con 0x5565fdf47cc0 2023-09-13T03:25:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.190+0000 7faf0c1d6b80 1 -- 172.21.15.64:0/3861939968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565fdf47cc0 msgr2=0x5565fdf58520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:25:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.190+0000 7faf0c1d6b80 1 --2- 172.21.15.64:0/3861939968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565fdf47cc0 0x5565fdf58520 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7faee0009990 tx=0x7faee0021b40 comp rx=0 tx=0).stop 2023-09-13T03:25:11.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.191+0000 7faf0c1d6b80 1 -- 172.21.15.64:0/3861939968 shutdown_connections 2023-09-13T03:25:11.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.191+0000 7faf0c1d6b80 1 --2- 172.21.15.64:0/3861939968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565fdf4d1a0 0x5565fdf49b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:11.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.191+0000 7faf0c1d6b80 1 --2- 172.21.15.64:0/3861939968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565fdf47300 0x5565fdf4c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:11.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.191+0000 7faf0c1d6b80 1 --2- 172.21.15.64:0/3861939968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565fdf47cc0 0x5565fdf58520 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:11.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.191+0000 7faf0c1d6b80 1 -- 172.21.15.64:0/3861939968 >> 172.21.15.64:0/3861939968 conn(0x5565fdebc030 msgr2=0x5565fdf4c080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:25:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.191+0000 7faf0c1d6b80 1 -- 172.21.15.64:0/3861939968 shutdown_connections 2023-09-13T03:25:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.191+0000 7faf0c1d6b80 1 -- 172.21.15.64:0/3861939968 wait complete. 2023-09-13T03:25:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.191+0000 7faf0c1d6b80 1 -- start start 2023-09-13T03:25:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.191+0000 7faf0c1d6b80 1 Processor -- start 2023-09-13T03:25:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.192+0000 7faf0c1d6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565fdf47300 0x5565fdf58560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.192+0000 7faf0c1d6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565fdf47cc0 0x5565fdf56960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.192+0000 7faf0c1d6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565fdf4d1a0 0x5565fdf56ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.192+0000 7faf0c1d6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565fdf575b0 con 0x5565fdf4d1a0 2023-09-13T03:25:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.192+0000 7faf0c1d6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565fdf57720 con 0x5565fdf47300 2023-09-13T03:25:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.192+0000 7faf0c1d6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565fdfb4d90 con 0x5565fdf47cc0 2023-09-13T03:25:11.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.192+0000 7faf0c1d6b80 10 client.0 _ll_register_callbacks cb 0x5565fdd81a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:25:11.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.192+0000 7faef659b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565fdf47300 0x5565fdf58560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:11.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.192+0000 7faef659b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565fdf47300 0x5565fdf58560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:38830/0 (socket says 172.21.15.64:38830) 2023-09-13T03:25:11.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.192+0000 7faef659b700 1 -- 172.21.15.64:0/3235952420 learned_addr learned my addr 172.21.15.64:0/3235952420 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:25:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.192+0000 7faef5d9a700 1 --2- 172.21.15.64:0/3235952420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565fdf47cc0 0x5565fdf56960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.192+0000 7faef6d9c700 1 --2- 172.21.15.64:0/3235952420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565fdf4d1a0 0x5565fdf56ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.193+0000 7faef659b700 1 -- 172.21.15.64:0/3235952420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565fdf47cc0 msgr2=0x5565fdf56960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:25:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.193+0000 7faef659b700 1 --2- 172.21.15.64:0/3235952420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565fdf47cc0 0x5565fdf56960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.193+0000 7faef659b700 1 -- 172.21.15.64:0/3235952420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565fdf4d1a0 msgr2=0x5565fdf56ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:25:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.193+0000 7faef659b700 1 --2- 172.21.15.64:0/3235952420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565fdf4d1a0 0x5565fdf56ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.193+0000 7faef659b700 1 -- 172.21.15.64:0/3235952420 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5565fdcaec00 con 0x5565fdf47300 2023-09-13T03:25:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.193+0000 7faef659b700 1 --2- 172.21.15.64:0/3235952420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565fdf47300 0x5565fdf58560 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7faeec0114c0 tx=0x7faeec011c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b410b05b971492d7 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:25:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.193+0000 7faed9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:25:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.193+0000 7faed9ffb700 1 -- 172.21.15.64:0/3235952420 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeec029040 con 0x5565fdf47300 2023-09-13T03:25:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.193+0000 7faf0c1d6b80 1 -- 172.21.15.64:0/3235952420 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faeec004390 con 0x5565fdf47300 2023-09-13T03:25:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.193+0000 7faed9ffb700 1 -- 172.21.15.64:0/3235952420 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7faeec005690 con 0x5565fdf47300 2023-09-13T03:25:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.193+0000 7faed9ffb700 1 -- 172.21.15.64:0/3235952420 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7faeec011230 con 0x5565fdf47300 2023-09-13T03:25:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.193+0000 7faf0c1d6b80 10 client.27098 fetch_fsmap learned FSMap version 62 2023-09-13T03:25:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.194+0000 7faf0c1d6b80 1 -- 172.21.15.64:0/3235952420 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) v3 -- 0x5565fdcc4250 con 0x5565fdf47300 2023-09-13T03:25:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.194+0000 7faed9ffb700 1 -- 172.21.15.64:0/3235952420 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faeec004390 con 0x5565fdf47300 2023-09-13T03:25:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.194+0000 7faed9ffb700 1 -- 172.21.15.64:0/3235952420 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 62) v1 ==== 7914+0+0 (secure 0 0 0) 0x7faeec0114f0 con 0x5565fdf47300 2023-09-13T03:25:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.194+0000 7faf0c1d6b80 10 client.27098 fetch_fsmap finished waiting for FSMap version 62 2023-09-13T03:25:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.194+0000 7faf0c1d6b80 10 client.27098 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:25:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.194+0000 7faf0c1d6b80 20 client.27098 populate_metadata read hostname 'smithi064' 2023-09-13T03:25:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.194+0000 7faf0c1d6b80 1 --2- 172.21.15.64:0/3235952420 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5565fdfb6380 0x5565fdfb87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.194+0000 7faf0c1d6b80 4 client.27098 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:25:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.194+0000 7faf0c1d6b80 1 -- 172.21.15.64:0/3235952420 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5565fde928c0 con 0x5565fdfb6380 2023-09-13T03:25:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.194+0000 7faef5d9a700 1 --2- 172.21.15.64:0/3235952420 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5565fdfb6380 0x5565fdfb87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.195+0000 7faef5d9a700 1 --2- 172.21.15.64:0/3235952420 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5565fdfb6380 0x5565fdfb87a0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=622df18063ddfbf3 server_cookie=c968151f7ae898b0 in_seq=0 out_seq=0 2023-09-13T03:25:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.195+0000 7faed9ffb700 10 client.27098 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:25:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.227+0000 7faed9ffb700 1 -- 172.21.15.64:0/3235952420 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5565fde928c0 con 0x5565fdfb6380 2023-09-13T03:25:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.227+0000 7faed9ffb700 10 client.27098 handle_command_reply: tid=0 2023-09-13T03:25:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.227+0000 7faf0c1d6b80 1 client.27098 shutdown 2023-09-13T03:25:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.227+0000 7faf0c1d6b80 1 -- 172.21.15.64:0/3235952420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565fdf47300 msgr2=0x5565fdf58560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:25:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.227+0000 7faf0c1d6b80 1 --2- 172.21.15.64:0/3235952420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565fdf47300 0x5565fdf58560 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7faeec0114c0 tx=0x7faeec011c50 comp rx=0 tx=0).stop 2023-09-13T03:25:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.227+0000 7faf0c1d6b80 1 -- 172.21.15.64:0/3235952420 shutdown_connections 2023-09-13T03:25:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.227+0000 7faf0c1d6b80 1 --2- 172.21.15.64:0/3235952420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565fdf47300 0x5565fdf58560 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.227+0000 7faf0c1d6b80 1 --2- 172.21.15.64:0/3235952420 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5565fdfb6380 0x5565fdfb87a0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.227+0000 7faf0c1d6b80 1 --2- 172.21.15.64:0/3235952420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565fdf4d1a0 0x5565fdf56ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.227+0000 7faf0c1d6b80 1 --2- 172.21.15.64:0/3235952420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5565fdf47cc0 0x5565fdf56960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.227+0000 7faed9ffb700 0 client.27098 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:25:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.227+0000 7faf0c1d6b80 1 -- 172.21.15.64:0/3235952420 >> 172.21.15.64:0/3235952420 conn(0x5565fdebc030 msgr2=0x5565fdf462f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:25:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.228+0000 7faf0c1d6b80 1 -- 172.21.15.64:0/3235952420 shutdown_connections 2023-09-13T03:25:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.228+0000 7faf0c1d6b80 1 -- 172.21.15.64:0/3235952420 wait complete. 2023-09-13T03:25:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.228+0000 7faf0c1d6b80 20 client.27098 trim_cache size 0 max 16384 2023-09-13T03:25:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:25:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:25:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:25:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:25:11.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.229+0000 7faef87ff700 1 -- 172.21.15.64:0/964627600 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7faedc0a07b0 msgr2=0x7faedc0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:25:11.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.229+0000 7faef87ff700 1 --2- 172.21.15.64:0/964627600 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7faedc0a07b0 0x7faedc0a2c70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7faee8009550 tx=0x7faee8004b40 comp rx=0 tx=0).stop 2023-09-13T03:25:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.229+0000 7faef87ff700 1 -- 172.21.15.64:0/964627600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faef0100ef0 msgr2=0x7faef0196bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:25:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.229+0000 7faef87ff700 1 --2- 172.21.15.64:0/964627600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faef0100ef0 0x7faef0196bd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7faee0000c00 tx=0x7faee0005740 comp rx=0 tx=0).stop 2023-09-13T03:25:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.229+0000 7faef87ff700 1 -- 172.21.15.64:0/964627600 shutdown_connections 2023-09-13T03:25:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.229+0000 7faef87ff700 1 --2- 172.21.15.64:0/964627600 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7faedc0a07b0 0x7faedc0a2c70 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.229+0000 7faef87ff700 1 --2- 172.21.15.64:0/964627600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faef01065f0 0x7faef0199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.229+0000 7faef87ff700 1 --2- 172.21.15.64:0/964627600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faef0100530 0x7faef0196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.229+0000 7faef87ff700 1 --2- 172.21.15.64:0/964627600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faef0100ef0 0x7faef0196bd0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.229+0000 7faef87ff700 1 -- 172.21.15.64:0/964627600 >> 172.21.15.64:0/964627600 conn(0x7faef00fbfc0 msgr2=0x7faef00fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:25:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.229+0000 7faef87ff700 1 -- 172.21.15.64:0/964627600 shutdown_connections 2023-09-13T03:25:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:11.229+0000 7faef87ff700 1 -- 172.21.15.64:0/964627600 wait complete. 2023-09-13T03:25:11.954 INFO:tasks.workunit.client.0.smithi064.stdout: 1 324251 2.62 MB/sec execute 556 sec latency 21.249 ms 2023-09-13T03:25:12.955 INFO:tasks.workunit.client.0.smithi064.stdout: 1 325989 2.63 MB/sec execute 557 sec latency 55.846 ms 2023-09-13T03:25:13.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:12 smithi170 ceph-mon[134328]: pgmap v376: 97 pgs: 97 active+clean; 143 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 273 op/s 2023-09-13T03:25:13.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:12 smithi064 ceph-mon[97675]: pgmap v376: 97 pgs: 97 active+clean; 143 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 273 op/s 2023-09-13T03:25:13.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:12 smithi142 ceph-mon[134082]: pgmap v376: 97 pgs: 97 active+clean; 143 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 273 op/s 2023-09-13T03:25:13.955 INFO:tasks.workunit.client.0.smithi064.stdout: 1 326943 2.64 MB/sec execute 558 sec latency 133.085 ms 2023-09-13T03:25:14.955 INFO:tasks.workunit.client.0.smithi064.stdout: 1 326943 2.63 MB/sec execute 559 sec latency 1133.230 ms 2023-09-13T03:25:15.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:14 smithi170 ceph-mon[134328]: pgmap v377: 97 pgs: 97 active+clean; 143 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 244 op/s 2023-09-13T03:25:15.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:14 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:25:15.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:14 smithi064 ceph-mon[97675]: pgmap v377: 97 pgs: 97 active+clean; 143 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 244 op/s 2023-09-13T03:25:15.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:14 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:25:15.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:14 smithi142 ceph-mon[134082]: pgmap v377: 97 pgs: 97 active+clean; 143 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 244 op/s 2023-09-13T03:25:15.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:14 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:25:15.955 INFO:tasks.workunit.client.0.smithi064.stdout: 1 326943 2.63 MB/sec execute 560 sec latency 2133.350 ms 2023-09-13T03:25:16.955 INFO:tasks.workunit.client.0.smithi064.stdout: 1 326943 2.62 MB/sec execute 561 sec latency 3133.463 ms 2023-09-13T03:25:17.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:16 smithi170 ceph-mon[134328]: pgmap v378: 97 pgs: 97 active+clean; 142 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 211 op/s 2023-09-13T03:25:17.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:16 smithi064 ceph-mon[97675]: pgmap v378: 97 pgs: 97 active+clean; 142 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 211 op/s 2023-09-13T03:25:17.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:16 smithi142 ceph-mon[134082]: pgmap v378: 97 pgs: 97 active+clean; 142 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 211 op/s 2023-09-13T03:25:17.955 INFO:tasks.workunit.client.0.smithi064.stdout: 1 326943 2.62 MB/sec execute 562 sec latency 4133.606 ms 2023-09-13T03:25:18.955 INFO:tasks.workunit.client.0.smithi064.stdout: 1 327722 2.62 MB/sec execute 563 sec latency 4712.788 ms 2023-09-13T03:25:19.159 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:18 smithi170 ceph-mon[134328]: pgmap v379: 97 pgs: 97 active+clean; 142 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 211 op/s 2023-09-13T03:25:19.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:18 smithi064 ceph-mon[97675]: pgmap v379: 97 pgs: 97 active+clean; 142 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 211 op/s 2023-09-13T03:25:19.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:18 smithi142 ceph-mon[134082]: pgmap v379: 97 pgs: 97 active+clean; 142 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 211 op/s 2023-09-13T03:25:19.955 INFO:tasks.workunit.client.0.smithi064.stdout: 1 329660 2.64 MB/sec execute 564 sec latency 18.658 ms 2023-09-13T03:25:20.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:20 smithi142 ceph-mon[134082]: pgmap v380: 97 pgs: 97 active+clean; 142 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 205 op/s 2023-09-13T03:25:20.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:20 smithi170 ceph-mon[134328]: pgmap v380: 97 pgs: 97 active+clean; 142 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 205 op/s 2023-09-13T03:25:20.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:20 smithi064 ceph-mon[97675]: pgmap v380: 97 pgs: 97 active+clean; 142 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 205 op/s 2023-09-13T03:25:20.955 INFO:tasks.workunit.client.0.smithi064.stdout: 1 330788 2.64 MB/sec execute 565 sec latency 14.484 ms 2023-09-13T03:25:21.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:21 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:25:21.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:21 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:25:21.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:21 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:25:21.955 INFO:tasks.workunit.client.0.smithi064.stdout: 1 332659 2.65 MB/sec execute 566 sec latency 21.597 ms 2023-09-13T03:25:22.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:22 smithi142 ceph-mon[134082]: pgmap v381: 97 pgs: 97 active+clean; 145 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 241 op/s 2023-09-13T03:25:22.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:22 smithi170 ceph-mon[134328]: pgmap v381: 97 pgs: 97 active+clean; 145 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 241 op/s 2023-09-13T03:25:22.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:22 smithi064 ceph-mon[97675]: pgmap v381: 97 pgs: 97 active+clean; 145 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 241 op/s 2023-09-13T03:25:22.955 INFO:tasks.workunit.client.0.smithi064.stdout: 1 333867 2.66 MB/sec execute 567 sec latency 49.239 ms 2023-09-13T03:25:23.956 INFO:tasks.workunit.client.0.smithi064.stdout: 1 335203 2.67 MB/sec execute 568 sec latency 32.931 ms 2023-09-13T03:25:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.385+0000 7fee5f7ff700 1 -- 172.21.15.64:0/2354363483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee580679f0 msgr2=0x7fee58067e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:25:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.385+0000 7fee5f7ff700 1 --2- 172.21.15.64:0/2354363483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee580679f0 0x7fee58067e70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fee48009990 tx=0x7fee48009ca0 comp rx=0 tx=0).stop 2023-09-13T03:25:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.386+0000 7fee5f7ff700 1 -- 172.21.15.64:0/2354363483 shutdown_connections 2023-09-13T03:25:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.386+0000 7fee5f7ff700 1 --2- 172.21.15.64:0/2354363483 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee58106650 0x7fee58106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.386+0000 7fee5f7ff700 1 --2- 172.21.15.64:0/2354363483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee580683b0 0x7fee5810d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.386+0000 7fee5f7ff700 1 --2- 172.21.15.64:0/2354363483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee580679f0 0x7fee58067e70 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.386+0000 7fee5f7ff700 1 -- 172.21.15.64:0/2354363483 >> 172.21.15.64:0/2354363483 conn(0x7fee58074f30 msgr2=0x7fee58075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:25:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.386+0000 7fee5f7ff700 1 -- 172.21.15.64:0/2354363483 shutdown_connections 2023-09-13T03:25:24.389 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.386+0000 7fee5f7ff700 1 -- 172.21.15.64:0/2354363483 wait complete. 2023-09-13T03:25:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.387+0000 7fee5f7ff700 1 Processor -- start 2023-09-13T03:25:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.387+0000 7fee5f7ff700 1 -- start start 2023-09-13T03:25:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.387+0000 7fee5f7ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee580679f0 0x7fee5819ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.387+0000 7fee5f7ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee580683b0 0x7fee5819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.387+0000 7fee5f7ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee58106650 0x7fee58198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.387+0000 7fee5f7ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee58199400 con 0x7fee580679f0 2023-09-13T03:25:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.387+0000 7fee5f7ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee58199570 con 0x7fee58106650 2023-09-13T03:25:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.387+0000 7fee5f7ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee58199730 con 0x7fee580683b0 2023-09-13T03:25:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.387+0000 7fee5cd9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee580683b0 0x7fee5819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.387+0000 7fee5d59b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee580679f0 0x7fee5819ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.388+0000 7fee5dd9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee58106650 0x7fee58198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.388+0000 7fee5cd9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee580683b0 0x7fee5819ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:58860/0 (socket says 172.21.15.64:58860) 2023-09-13T03:25:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.388+0000 7fee5cd9a700 1 -- 172.21.15.64:0/1807242840 learned_addr learned my addr 172.21.15.64:0/1807242840 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:25:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.388+0000 7fee5cd9a700 1 -- 172.21.15.64:0/1807242840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee58106650 msgr2=0x7fee58198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:25:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.388+0000 7fee5cd9a700 1 --2- 172.21.15.64:0/1807242840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee58106650 0x7fee58198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.388+0000 7fee5cd9a700 1 -- 172.21.15.64:0/1807242840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee580679f0 msgr2=0x7fee5819ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:25:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.388+0000 7fee5cd9a700 1 --2- 172.21.15.64:0/1807242840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee580679f0 0x7fee5819ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.388+0000 7fee5cd9a700 1 -- 172.21.15.64:0/1807242840 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee480095f0 con 0x7fee580683b0 2023-09-13T03:25:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.388+0000 7fee5cd9a700 1 --2- 172.21.15.64:0/1807242840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee580683b0 0x7fee5819ef70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fee48005910 tx=0x7fee48005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:25:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.388+0000 7fee4e7fc700 1 -- 172.21.15.64:0/1807242840 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4801d070 con 0x7fee580683b0 2023-09-13T03:25:24.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.388+0000 7fee4e7fc700 1 -- 172.21.15.64:0/1807242840 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee48022470 con 0x7fee580683b0 2023-09-13T03:25:24.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.389+0000 7fee5f7ff700 1 -- 172.21.15.64:0/1807242840 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee581999b0 con 0x7fee580683b0 2023-09-13T03:25:24.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.389+0000 7fee4e7fc700 1 -- 172.21.15.64:0/1807242840 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee48017650 con 0x7fee580683b0 2023-09-13T03:25:24.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.389+0000 7fee5f7ff700 1 -- 172.21.15.64:0/1807242840 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee581a5a20 con 0x7fee580683b0 2023-09-13T03:25:24.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.392+0000 7fee4e7fc700 1 -- 172.21.15.64:0/1807242840 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fee480225e0 con 0x7fee580683b0 2023-09-13T03:25:24.405 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.394+0000 7fee73040b80 1 Processor -- start 2023-09-13T03:25:24.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.394+0000 7fee73040b80 1 -- start start 2023-09-13T03:25:24.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.394+0000 7fee4e7fc700 1 --2- 172.21.15.64:0/1807242840 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fee440a07b0 0x7fee440a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:24.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.394+0000 7fee73040b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a9f7f1a0 0x55b6a9f7f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:24.406 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.394+0000 7fee73040b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6a9f79300 0x55b6a9f79780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:24.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.394+0000 7fee73040b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6a9f79cc0 0x55b6a9f85f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:24.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.394+0000 7fee73040b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6a9ed26d0 con 0x55b6a9f79cc0 2023-09-13T03:25:24.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.394+0000 7fee73040b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6a9ecfff0 con 0x55b6a9f79300 2023-09-13T03:25:24.407 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.394+0000 7fee73040b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6a9ecb8e0 con 0x55b6a9f7f1a0 2023-09-13T03:25:24.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.394+0000 7fee4e7fc700 1 -- 172.21.15.64:0/1807242840 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fee480cd410 con 0x7fee580683b0 2023-09-13T03:25:24.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.395+0000 7fee5d59b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a9f7f1a0 0x55b6a9f7f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:24.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.395+0000 7fee5dd9c700 1 --2- 172.21.15.64:0/1807242840 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fee440a07b0 0x7fee440a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:24.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.395+0000 7fee5d59b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a9f7f1a0 0x55b6a9f7f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:58872/0 (socket says 172.21.15.64:58872) 2023-09-13T03:25:24.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.395+0000 7fee5d59b700 1 -- 172.21.15.64:0/1158169693 learned_addr learned my addr 172.21.15.64:0/1158169693 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:25:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.395+0000 7fee5dd9c700 1 --2- 172.21.15.64:0/1158169693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6a9f79cc0 0x55b6a9f85f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.395+0000 7fee5cd9a700 1 --2- 172.21.15.64:0/1158169693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6a9f79300 0x55b6a9f79780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.395+0000 7fee5d59b700 1 -- 172.21.15.64:0/1158169693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6a9f79300 msgr2=0x55b6a9f79780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:25:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.395+0000 7fee5d59b700 1 --2- 172.21.15.64:0/1158169693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6a9f79300 0x55b6a9f79780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:24.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.395+0000 7fee5d59b700 1 -- 172.21.15.64:0/1158169693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6a9f79cc0 msgr2=0x55b6a9f85f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:25:24.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.395+0000 7fee5d59b700 1 --2- 172.21.15.64:0/1158169693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6a9f79cc0 0x55b6a9f85f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:24.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.395+0000 7fee5d59b700 1 -- 172.21.15.64:0/1158169693 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6a9cf7d80 con 0x55b6a9f7f1a0 2023-09-13T03:25:24.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.395+0000 7fee5cd9a700 1 --2- 172.21.15.64:0/1158169693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6a9f79300 0x55b6a9f79780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:25:24.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.395+0000 7fee5d59b700 1 --2- 172.21.15.64:0/1158169693 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a9f7f1a0 0x55b6a9f7f580 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fee54009d80 tx=0x7fee54011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3265a7d896470513 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:25:24.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.396+0000 7fee437fe700 1 -- 172.21.15.64:0/1158169693 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee54004230 con 0x55b6a9f7f1a0 2023-09-13T03:25:24.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.396+0000 7fee5dd9c700 1 --2- 172.21.15.64:0/1807242840 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fee440a07b0 0x7fee440a2c70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fee50009ec0 tx=0x7fee500092f0 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:25:24.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.396+0000 7fee437fe700 1 -- 172.21.15.64:0/1158169693 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee54004390 con 0x55b6a9f7f1a0 2023-09-13T03:25:24.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.396+0000 7fee437fe700 1 -- 172.21.15.64:0/1158169693 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee54019800 con 0x55b6a9f7f1a0 2023-09-13T03:25:24.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.396+0000 7fee73040b80 1 -- 172.21.15.64:0/1158169693 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a9f7f1a0 msgr2=0x55b6a9f7f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:25:24.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.396+0000 7fee73040b80 1 --2- 172.21.15.64:0/1158169693 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a9f7f1a0 0x55b6a9f7f580 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fee54009d80 tx=0x7fee54011720 comp rx=0 tx=0).stop 2023-09-13T03:25:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.397+0000 7fee73040b80 1 -- 172.21.15.64:0/1158169693 shutdown_connections 2023-09-13T03:25:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.397+0000 7fee73040b80 1 --2- 172.21.15.64:0/1158169693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6a9f79cc0 0x55b6a9f85f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.397+0000 7fee73040b80 1 --2- 172.21.15.64:0/1158169693 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a9f7f1a0 0x55b6a9f7f580 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.397+0000 7fee73040b80 1 --2- 172.21.15.64:0/1158169693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6a9f79300 0x55b6a9f79780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:24.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.397+0000 7fee73040b80 1 -- 172.21.15.64:0/1158169693 >> 172.21.15.64:0/1158169693 conn(0x55b6a9eee030 msgr2=0x55b6a9f74560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:25:24.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.397+0000 7fee73040b80 1 -- 172.21.15.64:0/1158169693 shutdown_connections 2023-09-13T03:25:24.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.397+0000 7fee73040b80 1 -- 172.21.15.64:0/1158169693 wait complete. 2023-09-13T03:25:24.414 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.397+0000 7fee73040b80 1 -- start start 2023-09-13T03:25:24.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.397+0000 7fee73040b80 1 Processor -- start 2023-09-13T03:25:24.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.398+0000 7fee73040b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a9f79300 0x55b6a9f85cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:24.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.398+0000 7fee73040b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6a9f79cc0 0x55b6a9f82ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:24.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.398+0000 7fee73040b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6a9f7f1a0 0x55b6a9f83530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:24.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.398+0000 7fee73040b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6a9e69b10 con 0x55b6a9f7f1a0 2023-09-13T03:25:24.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.398+0000 7fee73040b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6a9e69810 con 0x55b6a9f79cc0 2023-09-13T03:25:24.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.398+0000 7fee73040b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6a9d4d8c0 con 0x55b6a9f79300 2023-09-13T03:25:24.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.398+0000 7fee73040b80 10 client.0 _ll_register_callbacks cb 0x55b6a9db3a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:25:24.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.398+0000 7fee5cd9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6a9f79cc0 0x55b6a9f82ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:24.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.398+0000 7fee5d59b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a9f79300 0x55b6a9f85cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:24.418 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.398+0000 7fee5cd9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6a9f79cc0 0x55b6a9f82ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:44950/0 (socket says 172.21.15.64:44950) 2023-09-13T03:25:24.418 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.398+0000 7fee5cd9a700 1 -- 172.21.15.64:0/1723573777 learned_addr learned my addr 172.21.15.64:0/1723573777 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:25:24.418 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.398+0000 7fee5dd9c700 1 --2- 172.21.15.64:0/1723573777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6a9f7f1a0 0x55b6a9f83530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:24.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.399+0000 7fee5cd9a700 1 -- 172.21.15.64:0/1723573777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a9f79300 msgr2=0x55b6a9f85cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:25:24.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.399+0000 7fee5cd9a700 1 --2- 172.21.15.64:0/1723573777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a9f79300 0x55b6a9f85cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:24.419 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.399+0000 7fee5cd9a700 1 -- 172.21.15.64:0/1723573777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6a9f7f1a0 msgr2=0x55b6a9f83530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:25:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.399+0000 7fee5cd9a700 1 --2- 172.21.15.64:0/1723573777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6a9f7f1a0 0x55b6a9f83530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.399+0000 7fee5cd9a700 1 -- 172.21.15.64:0/1723573777 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee540099e0 con 0x55b6a9f79cc0 2023-09-13T03:25:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.399+0000 7fee5cd9a700 1 --2- 172.21.15.64:0/1723573777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6a9f79cc0 0x55b6a9f82ff0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fee4809a730 tx=0x7fee4801ffd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9497c1a21bf642ca server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:25:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.399+0000 7fee40ff9700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:25:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.399+0000 7fee40ff9700 1 -- 172.21.15.64:0/1723573777 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4809a390 con 0x55b6a9f79cc0 2023-09-13T03:25:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.399+0000 7fee73040b80 1 -- 172.21.15.64:0/1723573777 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b6a9e23490 con 0x55b6a9f79cc0 2023-09-13T03:25:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.399+0000 7fee40ff9700 1 -- 172.21.15.64:0/1723573777 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fee48015300 con 0x55b6a9f79cc0 2023-09-13T03:25:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.399+0000 7fee40ff9700 1 -- 172.21.15.64:0/1723573777 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee4809b130 con 0x55b6a9f79cc0 2023-09-13T03:25:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.399+0000 7fee73040b80 10 client.27110 fetch_fsmap learned FSMap version 64 2023-09-13T03:25:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.399+0000 7fee73040b80 1 -- 172.21.15.64:0/1723573777 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) v3 -- 0x55b6a9ce0c00 con 0x55b6a9f79cc0 2023-09-13T03:25:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.400+0000 7fee40ff9700 1 -- 172.21.15.64:0/1723573777 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b6a9e23490 con 0x55b6a9f79cc0 2023-09-13T03:25:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.400+0000 7fee40ff9700 1 -- 172.21.15.64:0/1723573777 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 64) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fee4801f2d0 con 0x55b6a9f79cc0 2023-09-13T03:25:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.400+0000 7fee73040b80 10 client.27110 fetch_fsmap finished waiting for FSMap version 64 2023-09-13T03:25:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.400+0000 7fee73040b80 10 client.27110 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:25:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.400+0000 7fee73040b80 20 client.27110 populate_metadata read hostname 'smithi064' 2023-09-13T03:25:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.400+0000 7fee73040b80 1 --2- 172.21.15.64:0/1723573777 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55b6a9f93000 0x55b6a9f953f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.400+0000 7fee73040b80 4 client.27110 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:25:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.400+0000 7fee73040b80 1 -- 172.21.15.64:0/1723573777 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b6a9ee0840 con 0x55b6a9f93000 2023-09-13T03:25:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.400+0000 7fee5d59b700 1 --2- 172.21.15.64:0/1723573777 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55b6a9f93000 0x55b6a9f953f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.401+0000 7fee5d59b700 1 --2- 172.21.15.64:0/1723573777 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55b6a9f93000 0x55b6a9f953f0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=347c4889c50c4f07 server_cookie=1741d9dbeb84e71a in_seq=0 out_seq=0 2023-09-13T03:25:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.401+0000 7fee40ff9700 10 client.27110 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:25:24.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.401+0000 7fee40ff9700 1 -- 172.21.15.64:0/1723573777 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b6a9ee0840 con 0x55b6a9f93000 2023-09-13T03:25:24.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.401+0000 7fee40ff9700 10 client.27110 handle_command_reply: tid=0 2023-09-13T03:25:24.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.401+0000 7fee73040b80 1 client.27110 shutdown 2023-09-13T03:25:24.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.402+0000 7fee73040b80 1 -- 172.21.15.64:0/1723573777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6a9f79cc0 msgr2=0x55b6a9f82ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:25:24.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.402+0000 7fee73040b80 1 --2- 172.21.15.64:0/1723573777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6a9f79cc0 0x55b6a9f82ff0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fee4809a730 tx=0x7fee4801ffd0 comp rx=0 tx=0).stop 2023-09-13T03:25:24.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.402+0000 7fee73040b80 1 -- 172.21.15.64:0/1723573777 shutdown_connections 2023-09-13T03:25:24.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.402+0000 7fee73040b80 1 --2- 172.21.15.64:0/1723573777 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55b6a9f93000 0x55b6a9f953f0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:24.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.402+0000 7fee73040b80 1 --2- 172.21.15.64:0/1723573777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6a9f7f1a0 0x55b6a9f83530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:24.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.402+0000 7fee73040b80 1 --2- 172.21.15.64:0/1723573777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a9f79300 0x55b6a9f85cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:24.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.402+0000 7fee40ff9700 0 client.27110 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:25:24.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.402+0000 7fee73040b80 1 --2- 172.21.15.64:0/1723573777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6a9f79cc0 0x55b6a9f82ff0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:24.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.402+0000 7fee73040b80 1 -- 172.21.15.64:0/1723573777 >> 172.21.15.64:0/1723573777 conn(0x55b6a9eee030 msgr2=0x55b6a9f737a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:25:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.402+0000 7fee73040b80 1 -- 172.21.15.64:0/1723573777 shutdown_connections 2023-09-13T03:25:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.402+0000 7fee73040b80 1 -- 172.21.15.64:0/1723573777 wait complete. 2023-09-13T03:25:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.402+0000 7fee73040b80 20 client.27110 trim_cache size 0 max 16384 2023-09-13T03:25:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.417+0000 7fee73040b80 1 Processor -- start 2023-09-13T03:25:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.418+0000 7fee73040b80 1 -- start start 2023-09-13T03:25:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.418+0000 7fee73040b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6a9f79300 0x55b6a9f78a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.418+0000 7fee73040b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a9f79cc0 0x55b6a9f73f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.418+0000 7fee73040b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6a9f7f1a0 0x55b6a9f74460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.418+0000 7fee73040b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6a9f7a580 con 0x55b6a9f7f1a0 2023-09-13T03:25:24.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.418+0000 7fee73040b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6a9f70fc0 con 0x55b6a9f79300 2023-09-13T03:25:24.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.418+0000 7fee73040b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6a9f75d10 con 0x55b6a9f79cc0 2023-09-13T03:25:24.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.418+0000 7fee5d59b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6a9f79300 0x55b6a9f78a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:24.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.418+0000 7fee5dd9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6a9f7f1a0 0x55b6a9f74460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:24.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.418+0000 7fee5d59b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6a9f79300 0x55b6a9f78a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:44964/0 (socket says 172.21.15.64:44964) 2023-09-13T03:25:24.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.418+0000 7fee5d59b700 1 -- 172.21.15.64:0/1161664380 learned_addr learned my addr 172.21.15.64:0/1161664380 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:25:24.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.418+0000 7fee5cd9a700 1 --2- 172.21.15.64:0/1161664380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a9f79cc0 0x55b6a9f73f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.419+0000 7fee5d59b700 1 -- 172.21.15.64:0/1161664380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a9f79cc0 msgr2=0x55b6a9f73f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:25:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.419+0000 7fee5d59b700 1 --2- 172.21.15.64:0/1161664380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a9f79cc0 0x55b6a9f73f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.419+0000 7fee5d59b700 1 -- 172.21.15.64:0/1161664380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6a9f7f1a0 msgr2=0x55b6a9f74460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:25:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.419+0000 7fee5d59b700 1 --2- 172.21.15.64:0/1161664380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6a9f7f1a0 0x55b6a9f74460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.419+0000 7fee5d59b700 1 -- 172.21.15.64:0/1161664380 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6a9f9a020 con 0x55b6a9f79300 2023-09-13T03:25:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.419+0000 7fee5d59b700 1 --2- 172.21.15.64:0/1161664380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6a9f79300 0x55b6a9f78a00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fee54014040 tx=0x7fee54005fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c99f09f92a16ac47 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:25:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.419+0000 7fee43fff700 1 -- 172.21.15.64:0/1161664380 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee54029070 con 0x55b6a9f79300 2023-09-13T03:25:24.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.419+0000 7fee43fff700 1 -- 172.21.15.64:0/1161664380 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee54009ed0 con 0x55b6a9f79300 2023-09-13T03:25:24.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.419+0000 7fee43fff700 1 -- 172.21.15.64:0/1161664380 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee54019420 con 0x55b6a9f79300 2023-09-13T03:25:24.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.420+0000 7fee73040b80 1 -- 172.21.15.64:0/1161664380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6a9f79300 msgr2=0x55b6a9f78a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:25:24.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.420+0000 7fee73040b80 1 --2- 172.21.15.64:0/1161664380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6a9f79300 0x55b6a9f78a00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fee54014040 tx=0x7fee54005fa0 comp rx=0 tx=0).stop 2023-09-13T03:25:24.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.420+0000 7fee73040b80 1 -- 172.21.15.64:0/1161664380 shutdown_connections 2023-09-13T03:25:24.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.420+0000 7fee73040b80 1 --2- 172.21.15.64:0/1161664380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6a9f79300 0x55b6a9f78a00 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:24.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.420+0000 7fee73040b80 1 --2- 172.21.15.64:0/1161664380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6a9f7f1a0 0x55b6a9f74460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:24.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.420+0000 7fee73040b80 1 --2- 172.21.15.64:0/1161664380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a9f79cc0 0x55b6a9f73f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:24.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.420+0000 7fee73040b80 1 -- 172.21.15.64:0/1161664380 >> 172.21.15.64:0/1161664380 conn(0x55b6a9eee030 msgr2=0x55b6a9f782f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:25:24.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.420+0000 7fee73040b80 1 -- 172.21.15.64:0/1161664380 shutdown_connections 2023-09-13T03:25:24.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.420+0000 7fee73040b80 1 -- 172.21.15.64:0/1161664380 wait complete. 2023-09-13T03:25:24.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.421+0000 7fee73040b80 1 -- start start 2023-09-13T03:25:24.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.421+0000 7fee73040b80 1 Processor -- start 2023-09-13T03:25:24.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.421+0000 7fee73040b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a9f79300 0x55b6a9f88b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:24.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.421+0000 7fee73040b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6a9f79cc0 0x55b6a9f89050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:24.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.421+0000 7fee73040b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6a9f7f1a0 0x55b6a9f89590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:24.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.421+0000 7fee73040b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6a9f8aad0 con 0x55b6a9f7f1a0 2023-09-13T03:25:24.445 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.421+0000 7fee73040b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6a9fe6e10 con 0x55b6a9f79cc0 2023-09-13T03:25:24.445 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.421+0000 7fee73040b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6a9fe6f50 con 0x55b6a9f79300 2023-09-13T03:25:24.445 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.422+0000 7fee73040b80 10 client.0 _ll_register_callbacks cb 0x55b6a9db3a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:25:24.445 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.422+0000 7fee5cd9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6a9f79cc0 0x55b6a9f89050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:24.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.422+0000 7fee5d59b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a9f79300 0x55b6a9f88b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:24.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.422+0000 7fee5cd9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6a9f79cc0 0x55b6a9f89050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:44974/0 (socket says 172.21.15.64:44974) 2023-09-13T03:25:24.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.422+0000 7fee5cd9a700 1 -- 172.21.15.64:0/1285830716 learned_addr learned my addr 172.21.15.64:0/1285830716 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:25:24.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.422+0000 7fee5dd9c700 1 --2- 172.21.15.64:0/1285830716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6a9f7f1a0 0x55b6a9f89590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:24.446 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.422+0000 7fee5d59b700 1 -- 172.21.15.64:0/1285830716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6a9f79cc0 msgr2=0x55b6a9f89050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:25:24.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.422+0000 7fee5d59b700 1 --2- 172.21.15.64:0/1285830716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6a9f79cc0 0x55b6a9f89050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:24.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.422+0000 7fee5d59b700 1 -- 172.21.15.64:0/1285830716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6a9f7f1a0 msgr2=0x55b6a9f89590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:25:24.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.422+0000 7fee5d59b700 1 --2- 172.21.15.64:0/1285830716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6a9f7f1a0 0x55b6a9f89590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:24.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.422+0000 7fee5d59b700 1 -- 172.21.15.64:0/1285830716 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b6a9cf7d80 con 0x55b6a9f79300 2023-09-13T03:25:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.422+0000 7fee5cd9a700 1 --2- 172.21.15.64:0/1285830716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6a9f79cc0 0x55b6a9f89050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:25:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.422+0000 7fee5d59b700 1 --2- 172.21.15.64:0/1285830716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a9f79300 0x55b6a9f88b10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fee54000c00 tx=0x7fee54004800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2dbb0acfeec4d40 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:25:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.422+0000 7fee40ff9700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:25:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.423+0000 7fee40ff9700 1 -- 172.21.15.64:0/1285830716 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee54029070 con 0x55b6a9f79300 2023-09-13T03:25:24.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.423+0000 7fee73040b80 1 -- 172.21.15.64:0/1285830716 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fee54004390 con 0x55b6a9f79300 2023-09-13T03:25:24.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.423+0000 7fee40ff9700 1 -- 172.21.15.64:0/1285830716 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fee540050c0 con 0x55b6a9f79300 2023-09-13T03:25:24.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.423+0000 7fee40ff9700 1 -- 172.21.15.64:0/1285830716 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee54005280 con 0x55b6a9f79300 2023-09-13T03:25:24.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.423+0000 7fee73040b80 10 client.27100 fetch_fsmap learned FSMap version 64 2023-09-13T03:25:24.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.423+0000 7fee73040b80 1 -- 172.21.15.64:0/1285830716 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) v3 -- 0x55b6a9d1e1f0 con 0x55b6a9f79300 2023-09-13T03:25:24.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.423+0000 7fee40ff9700 1 -- 172.21.15.64:0/1285830716 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee54042020 con 0x55b6a9f79300 2023-09-13T03:25:24.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.423+0000 7fee40ff9700 1 -- 172.21.15.64:0/1285830716 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 64) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fee5403d070 con 0x55b6a9f79300 2023-09-13T03:25:24.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.423+0000 7fee73040b80 10 client.27100 fetch_fsmap finished waiting for FSMap version 64 2023-09-13T03:25:24.450 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.423+0000 7fee73040b80 10 client.27100 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:25:24.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.423+0000 7fee73040b80 20 client.27100 populate_metadata read hostname 'smithi064' 2023-09-13T03:25:24.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.423+0000 7fee73040b80 1 --2- 172.21.15.64:0/1285830716 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55b6a9fe8370 0x55b6a9fea760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:24.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.424+0000 7fee73040b80 4 client.27100 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:25:24.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.424+0000 7fee73040b80 1 -- 172.21.15.64:0/1285830716 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b6a9ec48c0 con 0x55b6a9fe8370 2023-09-13T03:25:24.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.424+0000 7fee5cd9a700 1 --2- 172.21.15.64:0/1285830716 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55b6a9fe8370 0x55b6a9fea760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:24.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.424+0000 7fee40ff9700 10 client.27100 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:25:24.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.424+0000 7fee5cd9a700 1 --2- 172.21.15.64:0/1285830716 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55b6a9fe8370 0x55b6a9fea760 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15e05fbfa3db03f2 server_cookie=f3c5c5bc2b5e2887 in_seq=0 out_seq=0 2023-09-13T03:25:24.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.462+0000 7fee40ff9700 1 -- 172.21.15.64:0/1285830716 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b6a9ec48c0 con 0x55b6a9fe8370 2023-09-13T03:25:24.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.462+0000 7fee40ff9700 10 client.27100 handle_command_reply: tid=0 2023-09-13T03:25:24.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.462+0000 7fee73040b80 1 client.27100 shutdown 2023-09-13T03:25:24.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.462+0000 7fee73040b80 1 -- 172.21.15.64:0/1285830716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a9f79300 msgr2=0x55b6a9f88b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:25:24.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.462+0000 7fee73040b80 1 --2- 172.21.15.64:0/1285830716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a9f79300 0x55b6a9f88b10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fee54000c00 tx=0x7fee54004800 comp rx=0 tx=0).stop 2023-09-13T03:25:24.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.462+0000 7fee73040b80 1 -- 172.21.15.64:0/1285830716 shutdown_connections 2023-09-13T03:25:24.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.462+0000 7fee73040b80 1 --2- 172.21.15.64:0/1285830716 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55b6a9fe8370 0x55b6a9fea760 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:24.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.462+0000 7fee73040b80 1 --2- 172.21.15.64:0/1285830716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6a9f7f1a0 0x55b6a9f89590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.462+0000 7fee73040b80 1 --2- 172.21.15.64:0/1285830716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b6a9f79300 0x55b6a9f88b10 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.462+0000 7fee73040b80 1 --2- 172.21.15.64:0/1285830716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6a9f79cc0 0x55b6a9f89050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.462+0000 7fee73040b80 1 -- 172.21.15.64:0/1285830716 >> 172.21.15.64:0/1285830716 conn(0x55b6a9eee030 msgr2=0x55b6a9f78b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:25:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.462+0000 7fee40ff9700 0 client.27100 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:25:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.462+0000 7fee73040b80 1 -- 172.21.15.64:0/1285830716 shutdown_connections 2023-09-13T03:25:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.463+0000 7fee73040b80 1 -- 172.21.15.64:0/1285830716 wait complete. 2023-09-13T03:25:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.463+0000 7fee73040b80 20 client.27100 trim_cache size 0 max 16384 2023-09-13T03:25:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:25:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:25:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:25:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:25:24.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.464+0000 7fee5f7ff700 1 -- 172.21.15.64:0/1807242840 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fee440a07b0 msgr2=0x7fee440a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:25:24.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.464+0000 7fee5f7ff700 1 --2- 172.21.15.64:0/1807242840 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fee440a07b0 0x7fee440a2c70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fee50009ec0 tx=0x7fee500092f0 comp rx=0 tx=0).stop 2023-09-13T03:25:24.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.464+0000 7fee5f7ff700 1 -- 172.21.15.64:0/1807242840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee580683b0 msgr2=0x7fee5819ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:25:24.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.464+0000 7fee5f7ff700 1 --2- 172.21.15.64:0/1807242840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee580683b0 0x7fee5819ef70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fee48005910 tx=0x7fee48005720 comp rx=0 tx=0).stop 2023-09-13T03:25:24.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.464+0000 7fee5f7ff700 1 -- 172.21.15.64:0/1807242840 shutdown_connections 2023-09-13T03:25:24.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.464+0000 7fee5f7ff700 1 --2- 172.21.15.64:0/1807242840 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fee440a07b0 0x7fee440a2c70 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:24.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.464+0000 7fee5f7ff700 1 --2- 172.21.15.64:0/1807242840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee58106650 0x7fee58198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:24.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.464+0000 7fee5f7ff700 1 --2- 172.21.15.64:0/1807242840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee580679f0 0x7fee5819ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:24.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.464+0000 7fee5f7ff700 1 --2- 172.21.15.64:0/1807242840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee580683b0 0x7fee5819ef70 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:24.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.464+0000 7fee5f7ff700 1 -- 172.21.15.64:0/1807242840 >> 172.21.15.64:0/1807242840 conn(0x7fee58074f30 msgr2=0x7fee580fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:25:24.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.464+0000 7fee5f7ff700 1 -- 172.21.15.64:0/1807242840 shutdown_connections 2023-09-13T03:25:24.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:24.464+0000 7fee5f7ff700 1 -- 172.21.15.64:0/1807242840 wait complete. 2023-09-13T03:25:24.956 INFO:tasks.workunit.client.0.smithi064.stdout: 1 337190 2.69 MB/sec execute 569 sec latency 17.861 ms 2023-09-13T03:25:25.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:24 smithi170 ceph-mon[134328]: pgmap v382: 97 pgs: 97 active+clean; 145 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 1.7 MiB/s wr, 151 op/s 2023-09-13T03:25:25.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:24 smithi064 ceph-mon[97675]: pgmap v382: 97 pgs: 97 active+clean; 145 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 1.7 MiB/s wr, 151 op/s 2023-09-13T03:25:25.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:24 smithi142 ceph-mon[134082]: pgmap v382: 97 pgs: 97 active+clean; 145 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 1.7 MiB/s wr, 151 op/s 2023-09-13T03:25:25.956 INFO:tasks.workunit.client.0.smithi064.stdout: 1 338147 2.69 MB/sec execute 570 sec latency 17.319 ms 2023-09-13T03:25:26.956 INFO:tasks.workunit.client.0.smithi064.stdout: 1 339736 2.70 MB/sec execute 571 sec latency 21.220 ms 2023-09-13T03:25:27.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:26 smithi170 ceph-mon[134328]: pgmap v383: 97 pgs: 97 active+clean; 143 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 264 op/s 2023-09-13T03:25:27.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:26 smithi064 ceph-mon[97675]: pgmap v383: 97 pgs: 97 active+clean; 143 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 264 op/s 2023-09-13T03:25:27.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:26 smithi142 ceph-mon[134082]: pgmap v383: 97 pgs: 97 active+clean; 143 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 264 op/s 2023-09-13T03:25:27.956 INFO:tasks.workunit.client.0.smithi064.stdout: 1 340980 2.71 MB/sec execute 572 sec latency 17.916 ms 2023-09-13T03:25:28.956 INFO:tasks.workunit.client.0.smithi064.stdout: 1 342379 2.71 MB/sec execute 573 sec latency 25.837 ms 2023-09-13T03:25:29.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:28 smithi170 ceph-mon[134328]: pgmap v384: 97 pgs: 97 active+clean; 143 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 2.1 MiB/s wr, 177 op/s 2023-09-13T03:25:29.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:28 smithi064 ceph-mon[97675]: pgmap v384: 97 pgs: 97 active+clean; 143 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 2.1 MiB/s wr, 177 op/s 2023-09-13T03:25:29.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:28 smithi142 ceph-mon[134082]: pgmap v384: 97 pgs: 97 active+clean; 143 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 2.1 MiB/s wr, 177 op/s 2023-09-13T03:25:29.956 INFO:tasks.workunit.client.0.smithi064.stdout: 1 344276 2.73 MB/sec execute 574 sec latency 72.174 ms 2023-09-13T03:25:30.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:30 smithi142 ceph-mon[134082]: pgmap v385: 97 pgs: 97 active+clean; 143 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 2.1 MiB/s wr, 177 op/s 2023-09-13T03:25:30.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:30 smithi170 ceph-mon[134328]: pgmap v385: 97 pgs: 97 active+clean; 143 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 2.1 MiB/s wr, 177 op/s 2023-09-13T03:25:30.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:30 smithi064 ceph-mon[97675]: pgmap v385: 97 pgs: 97 active+clean; 143 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 2.1 MiB/s wr, 177 op/s 2023-09-13T03:25:30.956 INFO:tasks.workunit.client.0.smithi064.stdout: 1 345140 2.73 MB/sec execute 575 sec latency 16.265 ms 2023-09-13T03:25:31.956 INFO:tasks.workunit.client.0.smithi064.stdout: 1 346102 2.73 MB/sec execute 576 sec latency 50.694 ms 2023-09-13T03:25:32.956 INFO:tasks.workunit.client.0.smithi064.stdout: 1 347769 2.75 MB/sec execute 577 sec latency 72.040 ms 2023-09-13T03:25:33.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:32 smithi170 ceph-mon[134328]: pgmap v386: 97 pgs: 97 active+clean; 149 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 275 op/s 2023-09-13T03:25:33.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:32 smithi064 ceph-mon[97675]: pgmap v386: 97 pgs: 97 active+clean; 149 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 275 op/s 2023-09-13T03:25:33.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:32 smithi142 ceph-mon[134082]: pgmap v386: 97 pgs: 97 active+clean; 149 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 275 op/s 2023-09-13T03:25:33.956 INFO:tasks.workunit.client.0.smithi064.stdout: 1 348789 2.75 MB/sec execute 578 sec latency 17.344 ms 2023-09-13T03:25:34.956 INFO:tasks.workunit.client.0.smithi064.stdout: 1 350529 2.76 MB/sec execute 579 sec latency 30.831 ms 2023-09-13T03:25:35.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:34 smithi170 ceph-mon[134328]: pgmap v387: 97 pgs: 97 active+clean; 149 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 210 op/s 2023-09-13T03:25:35.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:34 smithi064 ceph-mon[97675]: pgmap v387: 97 pgs: 97 active+clean; 149 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 210 op/s 2023-09-13T03:25:35.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:34 smithi142 ceph-mon[134082]: pgmap v387: 97 pgs: 97 active+clean; 149 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 210 op/s 2023-09-13T03:25:35.957 INFO:tasks.workunit.client.0.smithi064.stdout: 1 351550 2.77 MB/sec execute 580 sec latency 150.169 ms 2023-09-13T03:25:36.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:35 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:25:36.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:35 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:25:36.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:35 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:25:36.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:35 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:25:36.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:35 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:25:36.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:35 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:25:36.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:35 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:25:36.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:35 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:25:36.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:35 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:25:36.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:35 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:25:36.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:35 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:25:36.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:35 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:25:36.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:35 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:25:36.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:35 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:25:36.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:35 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:25:36.170 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:35 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:25:36.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:35 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:25:36.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:35 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:25:36.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:35 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:25:36.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:35 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:25:36.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:35 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:25:36.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:35 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:25:36.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:35 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:25:36.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:35 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:25:36.956 INFO:tasks.workunit.client.0.smithi064.stdout: 1 352627 2.77 MB/sec execute 581 sec latency 98.134 ms 2023-09-13T03:25:37.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:36 smithi170 ceph-mon[134328]: pgmap v388: 97 pgs: 97 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 295 op/s 2023-09-13T03:25:37.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:36 smithi064 ceph-mon[97675]: pgmap v388: 97 pgs: 97 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 295 op/s 2023-09-13T03:25:37.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:36 smithi142 ceph-mon[134082]: pgmap v388: 97 pgs: 97 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 295 op/s 2023-09-13T03:25:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.634+0000 7fa85c1ff700 1 -- 172.21.15.64:0/3448281379 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa854106670 msgr2=0x7fa854106a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:25:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.634+0000 7fa85c1ff700 1 --2- 172.21.15.64:0/3448281379 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa854106670 0x7fa854106a50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa850009990 tx=0x7fa850009ca0 comp rx=0 tx=0).stop 2023-09-13T03:25:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.634+0000 7fa85c1ff700 1 -- 172.21.15.64:0/3448281379 shutdown_connections 2023-09-13T03:25:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.634+0000 7fa85c1ff700 1 --2- 172.21.15.64:0/3448281379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa854100f70 0x7fa85410d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.634+0000 7fa85c1ff700 1 --2- 172.21.15.64:0/3448281379 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa854106670 0x7fa854106a50 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.634+0000 7fa85c1ff700 1 --2- 172.21.15.64:0/3448281379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8541005b0 0x7fa854100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.634+0000 7fa85c1ff700 1 -- 172.21.15.64:0/3448281379 >> 172.21.15.64:0/3448281379 conn(0x7fa8540fc060 msgr2=0x7fa8540fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:25:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.634+0000 7fa85c1ff700 1 -- 172.21.15.64:0/3448281379 shutdown_connections 2023-09-13T03:25:37.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.634+0000 7fa85c1ff700 1 -- 172.21.15.64:0/3448281379 wait complete. 2023-09-13T03:25:37.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.635+0000 7fa85c1ff700 1 Processor -- start 2023-09-13T03:25:37.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.636+0000 7fa85c1ff700 1 -- start start 2023-09-13T03:25:37.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.636+0000 7fa85c1ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8541005b0 0x7fa85419e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:37.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.636+0000 7fa85c1ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa854100f70 0x7fa85419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:37.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.636+0000 7fa85c1ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa854106670 0x7fa8541986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:37.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.636+0000 7fa85c1ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa854198d10 con 0x7fa8541005b0 2023-09-13T03:25:37.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.636+0000 7fa85c1ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa854198e80 con 0x7fa854100f70 2023-09-13T03:25:37.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.636+0000 7fa85c1ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa854199040 con 0x7fa854106670 2023-09-13T03:25:37.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.636+0000 7fa85979a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa854100f70 0x7fa85419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:37.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.636+0000 7fa859f9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8541005b0 0x7fa85419e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.636+0000 7fa85979a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa854100f70 0x7fa85419ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:40456/0 (socket says 172.21.15.64:40456) 2023-09-13T03:25:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.636+0000 7fa85a79c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa854106670 0x7fa8541986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.636+0000 7fa85979a700 1 -- 172.21.15.64:0/1960763921 learned_addr learned my addr 172.21.15.64:0/1960763921 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:25:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.637+0000 7fa85979a700 1 -- 172.21.15.64:0/1960763921 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa854106670 msgr2=0x7fa8541986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:25:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.637+0000 7fa85979a700 1 --2- 172.21.15.64:0/1960763921 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa854106670 0x7fa8541986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:37.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.637+0000 7fa85979a700 1 -- 172.21.15.64:0/1960763921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8541005b0 msgr2=0x7fa85419e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:25:37.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.637+0000 7fa85979a700 1 --2- 172.21.15.64:0/1960763921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8541005b0 0x7fa85419e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:37.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.637+0000 7fa85979a700 1 -- 172.21.15.64:0/1960763921 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8500095f0 con 0x7fa854100f70 2023-09-13T03:25:37.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.637+0000 7fa85a79c700 1 --2- 172.21.15.64:0/1960763921 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa854106670 0x7fa8541986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:25:37.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.637+0000 7fa85979a700 1 --2- 172.21.15.64:0/1960763921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa854100f70 0x7fa85419ed80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa84400e9c0 tx=0x7fa84400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:25:37.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.637+0000 7fa84affd700 1 -- 172.21.15.64:0/1960763921 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8440097b0 con 0x7fa854100f70 2023-09-13T03:25:37.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.637+0000 7fa85c1ff700 1 -- 172.21.15.64:0/1960763921 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa854199320 con 0x7fa854100f70 2023-09-13T03:25:37.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.637+0000 7fa84affd700 1 -- 172.21.15.64:0/1960763921 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa84400ee60 con 0x7fa854100f70 2023-09-13T03:25:37.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.637+0000 7fa84affd700 1 -- 172.21.15.64:0/1960763921 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8440054f0 con 0x7fa854100f70 2023-09-13T03:25:37.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.637+0000 7fa85c1ff700 1 -- 172.21.15.64:0/1960763921 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8541a5830 con 0x7fa854100f70 2023-09-13T03:25:37.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.641+0000 7fa84affd700 1 -- 172.21.15.64:0/1960763921 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fa844004750 con 0x7fa854100f70 2023-09-13T03:25:37.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.642+0000 7fa86faeeb80 1 Processor -- start 2023-09-13T03:25:37.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.643+0000 7fa86faeeb80 1 -- start start 2023-09-13T03:25:37.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.643+0000 7fa84affd700 1 --2- 172.21.15.64:0/1960763921 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fa8400a07b0 0x7fa8400a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:37.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.643+0000 7fa84affd700 1 -- 172.21.15.64:0/1960763921 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fa844014070 con 0x7fa854100f70 2023-09-13T03:25:37.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.643+0000 7fa86faeeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a56d01a1a0 0x55a56d01a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:37.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.643+0000 7fa86faeeb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a56d014300 0x55a56d014780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:37.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.643+0000 7fa86faeeb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a56d014cc0 0x55a56d020f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:37.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.643+0000 7fa85a79c700 1 --2- 172.21.15.64:0/1960763921 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fa8400a07b0 0x7fa8400a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:37.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.643+0000 7fa86faeeb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a56cf6d6d0 con 0x55a56d014300 2023-09-13T03:25:37.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.643+0000 7fa86faeeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a56cf6aff0 con 0x55a56d01a1a0 2023-09-13T03:25:37.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.643+0000 7fa86faeeb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a56cf668e0 con 0x55a56d014cc0 2023-09-13T03:25:37.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.643+0000 7fa859f9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a56d01a1a0 0x55a56d01a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:37.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.643+0000 7fa859f9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a56d01a1a0 0x55a56d01a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:40460/0 (socket says 172.21.15.64:40460) 2023-09-13T03:25:37.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.644+0000 7fa859f9b700 1 -- 172.21.15.64:0/215339449 learned_addr learned my addr 172.21.15.64:0/215339449 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:25:37.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.644+0000 7fa85a79c700 1 --2- 172.21.15.64:0/215339449 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a56d014cc0 0x55a56d020f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:37.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.644+0000 7fa85979a700 1 --2- 172.21.15.64:0/215339449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a56d014300 0x55a56d014780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:37.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.644+0000 7fa859f9b700 1 -- 172.21.15.64:0/215339449 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a56d014cc0 msgr2=0x55a56d020f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:25:37.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.644+0000 7fa859f9b700 1 --2- 172.21.15.64:0/215339449 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a56d014cc0 0x55a56d020f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:37.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.644+0000 7fa859f9b700 1 -- 172.21.15.64:0/215339449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a56d014300 msgr2=0x55a56d014780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:25:37.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.644+0000 7fa859f9b700 1 --2- 172.21.15.64:0/215339449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a56d014300 0x55a56d014780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:37.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.644+0000 7fa859f9b700 1 -- 172.21.15.64:0/215339449 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a56cd92d80 con 0x55a56d01a1a0 2023-09-13T03:25:37.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.644+0000 7fa85a79c700 1 --2- 172.21.15.64:0/215339449 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a56d014cc0 0x55a56d020f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:25:37.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.644+0000 7fa85979a700 1 --2- 172.21.15.64:0/215339449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a56d014300 0x55a56d014780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:25:37.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.644+0000 7fa85a79c700 1 --2- 172.21.15.64:0/1960763921 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fa8400a07b0 0x7fa8400a2c70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa85419f540 tx=0x7fa84c0045c0 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:25:37.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.644+0000 7fa859f9b700 1 --2- 172.21.15.64:0/215339449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a56d01a1a0 0x55a56d01a580 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa850005ea0 tx=0x7fa850005ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc6bb929b6e724ab server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:25:37.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.644+0000 7fa83ffff700 1 -- 172.21.15.64:0/215339449 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85001d070 con 0x55a56d01a1a0 2023-09-13T03:25:37.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.644+0000 7fa83ffff700 1 -- 172.21.15.64:0/215339449 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa85001e040 con 0x55a56d01a1a0 2023-09-13T03:25:37.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.644+0000 7fa83ffff700 1 -- 172.21.15.64:0/215339449 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa850004930 con 0x55a56d01a1a0 2023-09-13T03:25:37.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.645+0000 7fa86faeeb80 1 -- 172.21.15.64:0/215339449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a56d01a1a0 msgr2=0x55a56d01a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:25:37.652 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.645+0000 7fa86faeeb80 1 --2- 172.21.15.64:0/215339449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a56d01a1a0 0x55a56d01a580 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa850005ea0 tx=0x7fa850005ed0 comp rx=0 tx=0).stop 2023-09-13T03:25:37.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.645+0000 7fa86faeeb80 1 -- 172.21.15.64:0/215339449 shutdown_connections 2023-09-13T03:25:37.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.645+0000 7fa86faeeb80 1 --2- 172.21.15.64:0/215339449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a56d01a1a0 0x55a56d01a580 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:37.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.645+0000 7fa86faeeb80 1 --2- 172.21.15.64:0/215339449 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a56d014cc0 0x55a56d020f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:37.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.645+0000 7fa86faeeb80 1 --2- 172.21.15.64:0/215339449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a56d014300 0x55a56d014780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:37.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.645+0000 7fa86faeeb80 1 -- 172.21.15.64:0/215339449 >> 172.21.15.64:0/215339449 conn(0x55a56cf89030 msgr2=0x55a56d00f560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:25:37.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.645+0000 7fa86faeeb80 1 -- 172.21.15.64:0/215339449 shutdown_connections 2023-09-13T03:25:37.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.645+0000 7fa86faeeb80 1 -- 172.21.15.64:0/215339449 wait complete. 2023-09-13T03:25:37.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.646+0000 7fa86faeeb80 1 -- start start 2023-09-13T03:25:37.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.646+0000 7fa86faeeb80 1 Processor -- start 2023-09-13T03:25:37.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.647+0000 7fa86faeeb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a56d014300 0x55a56d020dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:37.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.647+0000 7fa86faeeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a56d014cc0 0x55a56d01dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:37.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.647+0000 7fa86faeeb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a56d01a1a0 0x55a56d01e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:37.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.647+0000 7fa86faeeb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a56cf04b10 con 0x55a56d014300 2023-09-13T03:25:37.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.647+0000 7fa86faeeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a56cf04810 con 0x55a56d014cc0 2023-09-13T03:25:37.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.647+0000 7fa86faeeb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a56cde88c0 con 0x55a56d01a1a0 2023-09-13T03:25:37.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.647+0000 7fa86faeeb80 10 client.0 _ll_register_callbacks cb 0x55a56ce4ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:25:37.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.647+0000 7fa859f9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a56d014300 0x55a56d020dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.647+0000 7fa859f9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a56d014300 0x55a56d020dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33300/0 (socket says 172.21.15.64:33300) 2023-09-13T03:25:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.647+0000 7fa859f9b700 1 -- 172.21.15.64:0/2500683336 learned_addr learned my addr 172.21.15.64:0/2500683336 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:25:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.647+0000 7fa85a79c700 1 --2- 172.21.15.64:0/2500683336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a56d01a1a0 0x55a56d01e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.647+0000 7fa85979a700 1 --2- 172.21.15.64:0/2500683336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a56d014cc0 0x55a56d01dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.648+0000 7fa85a79c700 1 -- 172.21.15.64:0/2500683336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a56d014cc0 msgr2=0x55a56d01dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:25:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.648+0000 7fa85a79c700 1 --2- 172.21.15.64:0/2500683336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a56d014cc0 0x55a56d01dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:37.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.648+0000 7fa85a79c700 1 -- 172.21.15.64:0/2500683336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a56d014300 msgr2=0x55a56d020dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:25:37.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.648+0000 7fa85a79c700 1 --2- 172.21.15.64:0/2500683336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a56d014300 0x55a56d020dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:37.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.648+0000 7fa85a79c700 1 -- 172.21.15.64:0/2500683336 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa85410dd30 con 0x55a56d01a1a0 2023-09-13T03:25:37.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.648+0000 7fa85a79c700 1 --2- 172.21.15.64:0/2500683336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a56d01a1a0 0x55a56d01e560 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa84c0049c0 tx=0x7fa84c004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97ae30f1fb686315 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:25:37.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.648+0000 7fa83d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:25:37.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.648+0000 7fa83d7fa700 1 -- 172.21.15.64:0/2500683336 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84c01ace0 con 0x55a56d01a1a0 2023-09-13T03:25:37.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.648+0000 7fa86faeeb80 1 -- 172.21.15.64:0/2500683336 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a56cebe490 con 0x55a56d01a1a0 2023-09-13T03:25:37.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.648+0000 7fa83d7fa700 1 -- 172.21.15.64:0/2500683336 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fa84c01ae40 con 0x55a56d01a1a0 2023-09-13T03:25:37.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.648+0000 7fa83d7fa700 1 -- 172.21.15.64:0/2500683336 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa84c024420 con 0x55a56d01a1a0 2023-09-13T03:25:37.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.648+0000 7fa86faeeb80 10 client.27112 fetch_fsmap learned FSMap version 64 2023-09-13T03:25:37.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.648+0000 7fa86faeeb80 1 -- 172.21.15.64:0/2500683336 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) v3 -- 0x55a56cd7bc00 con 0x55a56d01a1a0 2023-09-13T03:25:37.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.649+0000 7fa83d7fa700 1 -- 172.21.15.64:0/2500683336 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a56cebe490 con 0x55a56d01a1a0 2023-09-13T03:25:37.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.649+0000 7fa83d7fa700 1 -- 172.21.15.64:0/2500683336 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 64) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fa84c018460 con 0x55a56d01a1a0 2023-09-13T03:25:37.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.649+0000 7fa86faeeb80 10 client.27112 fetch_fsmap finished waiting for FSMap version 64 2023-09-13T03:25:37.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.649+0000 7fa86faeeb80 10 client.27112 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:25:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.649+0000 7fa86faeeb80 20 client.27112 populate_metadata read hostname 'smithi064' 2023-09-13T03:25:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.649+0000 7fa86faeeb80 1 --2- 172.21.15.64:0/2500683336 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55a56d02e360 0x55a56d030780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.649+0000 7fa86faeeb80 4 client.27112 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:25:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.649+0000 7fa86faeeb80 1 -- 172.21.15.64:0/2500683336 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a56cf7b840 con 0x55a56d02e360 2023-09-13T03:25:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.649+0000 7fa859f9b700 1 --2- 172.21.15.64:0/2500683336 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55a56d02e360 0x55a56d030780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.650+0000 7fa859f9b700 1 --2- 172.21.15.64:0/2500683336 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55a56d02e360 0x55a56d030780 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b3c849cd75cd59 server_cookie=69074544948dc747 in_seq=0 out_seq=0 2023-09-13T03:25:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.650+0000 7fa83d7fa700 10 client.27112 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:25:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.651+0000 7fa83d7fa700 1 -- 172.21.15.64:0/2500683336 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a56cf7b840 con 0x55a56d02e360 2023-09-13T03:25:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.651+0000 7fa83d7fa700 10 client.27112 handle_command_reply: tid=0 2023-09-13T03:25:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.651+0000 7fa86faeeb80 1 client.27112 shutdown 2023-09-13T03:25:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.651+0000 7fa86faeeb80 1 -- 172.21.15.64:0/2500683336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a56d01a1a0 msgr2=0x55a56d01e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:25:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.651+0000 7fa86faeeb80 1 --2- 172.21.15.64:0/2500683336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a56d01a1a0 0x55a56d01e560 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa84c0049c0 tx=0x7fa84c004b40 comp rx=0 tx=0).stop 2023-09-13T03:25:37.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.651+0000 7fa86faeeb80 1 -- 172.21.15.64:0/2500683336 shutdown_connections 2023-09-13T03:25:37.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.651+0000 7fa86faeeb80 1 --2- 172.21.15.64:0/2500683336 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55a56d02e360 0x55a56d030780 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:37.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.651+0000 7fa86faeeb80 1 --2- 172.21.15.64:0/2500683336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a56d01a1a0 0x55a56d01e560 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:37.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.651+0000 7fa86faeeb80 1 --2- 172.21.15.64:0/2500683336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a56d014300 0x55a56d020dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:37.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.651+0000 7fa86faeeb80 1 --2- 172.21.15.64:0/2500683336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a56d014cc0 0x55a56d01dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.651+0000 7fa83d7fa700 0 client.27112 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:25:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.651+0000 7fa86faeeb80 1 -- 172.21.15.64:0/2500683336 >> 172.21.15.64:0/2500683336 conn(0x55a56cf89030 msgr2=0x55a56d00e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:25:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.652+0000 7fa86faeeb80 1 -- 172.21.15.64:0/2500683336 shutdown_connections 2023-09-13T03:25:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.652+0000 7fa86faeeb80 1 -- 172.21.15.64:0/2500683336 wait complete. 2023-09-13T03:25:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.652+0000 7fa86faeeb80 20 client.27112 trim_cache size 0 max 16384 2023-09-13T03:25:37.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.668+0000 7fa86faeeb80 1 Processor -- start 2023-09-13T03:25:37.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.668+0000 7fa86faeeb80 1 -- start start 2023-09-13T03:25:37.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.668+0000 7fa86faeeb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a56d014300 0x55a56d013a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:37.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.668+0000 7fa86faeeb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a56d014cc0 0x55a56d020900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:37.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.668+0000 7fa86faeeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a56d01a1a0 0x55a56d00ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:37.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.668+0000 7fa86faeeb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a56d01c590 con 0x55a56d014cc0 2023-09-13T03:25:37.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.668+0000 7fa86faeeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a56d018be0 con 0x55a56d01a1a0 2023-09-13T03:25:37.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.668+0000 7fa86faeeb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a56d01eb60 con 0x55a56d014300 2023-09-13T03:25:37.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.669+0000 7fa859f9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a56d014300 0x55a56d013a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:37.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.669+0000 7fa859f9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a56d014300 0x55a56d013a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:35106/0 (socket says 172.21.15.64:35106) 2023-09-13T03:25:37.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.669+0000 7fa859f9b700 1 -- 172.21.15.64:0/1013506424 learned_addr learned my addr 172.21.15.64:0/1013506424 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:25:37.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.669+0000 7fa85a79c700 1 --2- 172.21.15.64:0/1013506424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a56d01a1a0 0x55a56d00ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:37.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.669+0000 7fa85979a700 1 --2- 172.21.15.64:0/1013506424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a56d014cc0 0x55a56d020900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.669+0000 7fa859f9b700 1 -- 172.21.15.64:0/1013506424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a56d01a1a0 msgr2=0x55a56d00ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:25:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.669+0000 7fa859f9b700 1 --2- 172.21.15.64:0/1013506424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a56d01a1a0 0x55a56d00ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.669+0000 7fa859f9b700 1 -- 172.21.15.64:0/1013506424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a56d014cc0 msgr2=0x55a56d020900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:25:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.669+0000 7fa859f9b700 1 --2- 172.21.15.64:0/1013506424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a56d014cc0 0x55a56d020900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.669+0000 7fa859f9b700 1 -- 172.21.15.64:0/1013506424 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a56cdb91f0 con 0x55a56d014300 2023-09-13T03:25:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.669+0000 7fa85a79c700 1 --2- 172.21.15.64:0/1013506424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a56d01a1a0 0x55a56d00ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:25:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.669+0000 7fa859f9b700 1 --2- 172.21.15.64:0/1013506424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a56d014300 0x55a56d013a00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa850009990 tx=0x7fa85001a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f337a601c828c1e server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:25:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.670+0000 7fa848ff9700 1 -- 172.21.15.64:0/1013506424 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa850032070 con 0x55a56d014300 2023-09-13T03:25:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.670+0000 7fa848ff9700 1 -- 172.21.15.64:0/1013506424 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8500055e0 con 0x55a56d014300 2023-09-13T03:25:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.670+0000 7fa848ff9700 1 -- 172.21.15.64:0/1013506424 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85001d040 con 0x55a56d014300 2023-09-13T03:25:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.670+0000 7fa86faeeb80 1 -- 172.21.15.64:0/1013506424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a56d014300 msgr2=0x55a56d013a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:25:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.670+0000 7fa86faeeb80 1 --2- 172.21.15.64:0/1013506424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a56d014300 0x55a56d013a00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa850009990 tx=0x7fa85001a040 comp rx=0 tx=0).stop 2023-09-13T03:25:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.670+0000 7fa86faeeb80 1 -- 172.21.15.64:0/1013506424 shutdown_connections 2023-09-13T03:25:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.670+0000 7fa86faeeb80 1 --2- 172.21.15.64:0/1013506424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a56d01a1a0 0x55a56d00ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.670+0000 7fa86faeeb80 1 --2- 172.21.15.64:0/1013506424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a56d014300 0x55a56d013a00 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.670+0000 7fa86faeeb80 1 --2- 172.21.15.64:0/1013506424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a56d014cc0 0x55a56d020900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.670+0000 7fa86faeeb80 1 -- 172.21.15.64:0/1013506424 >> 172.21.15.64:0/1013506424 conn(0x55a56cf89030 msgr2=0x55a56d0132f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:25:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.670+0000 7fa86faeeb80 1 -- 172.21.15.64:0/1013506424 shutdown_connections 2023-09-13T03:25:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.671+0000 7fa86faeeb80 1 -- 172.21.15.64:0/1013506424 wait complete. 2023-09-13T03:25:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.671+0000 7fa86faeeb80 1 -- start start 2023-09-13T03:25:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.671+0000 7fa86faeeb80 1 Processor -- start 2023-09-13T03:25:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.672+0000 7fa86faeeb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a56d014300 0x55a56d0233e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.672+0000 7fa86faeeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a56d014cc0 0x55a56d023920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.672+0000 7fa86faeeb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a56d01a1a0 0x55a56d023e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.672+0000 7fa86faeeb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a56d0899c0 con 0x55a56d01a1a0 2023-09-13T03:25:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.672+0000 7fa86faeeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a56d089b00 con 0x55a56d014cc0 2023-09-13T03:25:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.672+0000 7fa86faeeb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a56d089c70 con 0x55a56d014300 2023-09-13T03:25:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.672+0000 7fa86faeeb80 10 client.0 _ll_register_callbacks cb 0x7fa84c00e960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:25:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.672+0000 7fa85979a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a56d014cc0 0x55a56d023920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:37.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.672+0000 7fa859f9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a56d014300 0x55a56d0233e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:37.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.672+0000 7fa85979a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a56d014cc0 0x55a56d023920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:40494/0 (socket says 172.21.15.64:40494) 2023-09-13T03:25:37.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.672+0000 7fa859f9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a56d014300 0x55a56d0233e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:35108/0 (socket says 172.21.15.64:35108) 2023-09-13T03:25:37.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.672+0000 7fa85979a700 1 -- 172.21.15.64:0/2642235475 learned_addr learned my addr 172.21.15.64:0/2642235475 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:25:37.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.672+0000 7fa85a79c700 1 --2- 172.21.15.64:0/2642235475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a56d01a1a0 0x55a56d023e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:37.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.672+0000 7fa85979a700 1 -- 172.21.15.64:0/2642235475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a56d014300 msgr2=0x55a56d0233e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:25:37.682 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.672+0000 7fa85979a700 1 --2- 172.21.15.64:0/2642235475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a56d014300 0x55a56d0233e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:37.682 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.672+0000 7fa85979a700 1 -- 172.21.15.64:0/2642235475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a56d01a1a0 msgr2=0x55a56d023e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:25:37.682 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.672+0000 7fa85979a700 1 --2- 172.21.15.64:0/2642235475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a56d01a1a0 0x55a56d023e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:37.682 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.672+0000 7fa85979a700 1 -- 172.21.15.64:0/2642235475 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a56cd7bc00 con 0x55a56d014cc0 2023-09-13T03:25:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.672+0000 7fa85979a700 1 --2- 172.21.15.64:0/2642235475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a56d014cc0 0x55a56d023920 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa8440c99e0 tx=0x7fa844099ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c7cab00278b1eee server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:25:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.672+0000 7fa83d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:25:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.673+0000 7fa83d7fa700 1 -- 172.21.15.64:0/2642235475 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8440a1700 con 0x55a56d014cc0 2023-09-13T03:25:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.673+0000 7fa86faeeb80 1 -- 172.21.15.64:0/2642235475 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa85001e040 con 0x55a56d014cc0 2023-09-13T03:25:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.673+0000 7fa83d7fa700 1 -- 172.21.15.64:0/2642235475 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fa84409d120 con 0x55a56d014cc0 2023-09-13T03:25:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.673+0000 7fa83d7fa700 1 -- 172.21.15.64:0/2642235475 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa84409d300 con 0x55a56d014cc0 2023-09-13T03:25:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.673+0000 7fa86faeeb80 10 client.27134 fetch_fsmap learned FSMap version 64 2023-09-13T03:25:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.673+0000 7fa86faeeb80 1 -- 172.21.15.64:0/2642235475 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) v3 -- 0x55a56cd92d80 con 0x55a56d014cc0 2023-09-13T03:25:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.673+0000 7fa83d7fa700 1 -- 172.21.15.64:0/2642235475 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa85001e040 con 0x55a56d014cc0 2023-09-13T03:25:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.673+0000 7fa83d7fa700 1 -- 172.21.15.64:0/2642235475 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 64) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fa8440998f0 con 0x55a56d014cc0 2023-09-13T03:25:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.674+0000 7fa86faeeb80 10 client.27134 fetch_fsmap finished waiting for FSMap version 64 2023-09-13T03:25:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.674+0000 7fa86faeeb80 10 client.27134 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:25:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.674+0000 7fa86faeeb80 20 client.27134 populate_metadata read hostname 'smithi064' 2023-09-13T03:25:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.674+0000 7fa86faeeb80 1 --2- 172.21.15.64:0/2642235475 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55a56d08b390 0x55a56d08d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.674+0000 7fa86faeeb80 4 client.27134 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:25:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.674+0000 7fa86faeeb80 1 -- 172.21.15.64:0/2642235475 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a56cf5f8c0 con 0x55a56d08b390 2023-09-13T03:25:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.674+0000 7fa859f9b700 1 --2- 172.21.15.64:0/2642235475 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55a56d08b390 0x55a56d08d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.674+0000 7fa859f9b700 1 --2- 172.21.15.64:0/2642235475 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55a56d08b390 0x55a56d08d7b0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33b635c9b0326116 server_cookie=ded29756ac54b933 in_seq=0 out_seq=0 2023-09-13T03:25:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.674+0000 7fa83d7fa700 10 client.27134 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:25:37.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.715+0000 7fa83d7fa700 1 -- 172.21.15.64:0/2642235475 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a56cf5f8c0 con 0x55a56d08b390 2023-09-13T03:25:37.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.716+0000 7fa83d7fa700 10 client.27134 handle_command_reply: tid=0 2023-09-13T03:25:37.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.716+0000 7fa86faeeb80 1 client.27134 shutdown 2023-09-13T03:25:37.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.716+0000 7fa86faeeb80 1 -- 172.21.15.64:0/2642235475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a56d014cc0 msgr2=0x55a56d023920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:25:37.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.716+0000 7fa86faeeb80 1 --2- 172.21.15.64:0/2642235475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a56d014cc0 0x55a56d023920 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa8440c99e0 tx=0x7fa844099ed0 comp rx=0 tx=0).stop 2023-09-13T03:25:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.716+0000 7fa86faeeb80 1 -- 172.21.15.64:0/2642235475 shutdown_connections 2023-09-13T03:25:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.716+0000 7fa86faeeb80 1 --2- 172.21.15.64:0/2642235475 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55a56d08b390 0x55a56d08d7b0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.716+0000 7fa86faeeb80 1 --2- 172.21.15.64:0/2642235475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a56d01a1a0 0x55a56d023e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.716+0000 7fa86faeeb80 1 --2- 172.21.15.64:0/2642235475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a56d014300 0x55a56d0233e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.716+0000 7fa86faeeb80 1 --2- 172.21.15.64:0/2642235475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a56d014cc0 0x55a56d023920 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:37.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.716+0000 7fa86faeeb80 1 -- 172.21.15.64:0/2642235475 >> 172.21.15.64:0/2642235475 conn(0x55a56cf89030 msgr2=0x55a56d013b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:25:37.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.716+0000 7fa83d7fa700 0 client.27134 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:25:37.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.716+0000 7fa86faeeb80 1 -- 172.21.15.64:0/2642235475 shutdown_connections 2023-09-13T03:25:37.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.716+0000 7fa86faeeb80 1 -- 172.21.15.64:0/2642235475 wait complete. 2023-09-13T03:25:37.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.716+0000 7fa86faeeb80 20 client.27134 trim_cache size 0 max 16384 2023-09-13T03:25:37.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:25:37.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:25:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:25:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:25:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.717+0000 7fa85c1ff700 1 -- 172.21.15.64:0/1960763921 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fa8400a07b0 msgr2=0x7fa8400a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:25:37.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.717+0000 7fa85c1ff700 1 --2- 172.21.15.64:0/1960763921 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fa8400a07b0 0x7fa8400a2c70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa85419f540 tx=0x7fa84c0045c0 comp rx=0 tx=0).stop 2023-09-13T03:25:37.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.717+0000 7fa85c1ff700 1 -- 172.21.15.64:0/1960763921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa854100f70 msgr2=0x7fa85419ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:25:37.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.717+0000 7fa85c1ff700 1 --2- 172.21.15.64:0/1960763921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa854100f70 0x7fa85419ed80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa84400e9c0 tx=0x7fa84400ecd0 comp rx=0 tx=0).stop 2023-09-13T03:25:37.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.717+0000 7fa85c1ff700 1 -- 172.21.15.64:0/1960763921 shutdown_connections 2023-09-13T03:25:37.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.717+0000 7fa85c1ff700 1 --2- 172.21.15.64:0/1960763921 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fa8400a07b0 0x7fa8400a2c70 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:37.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.717+0000 7fa85c1ff700 1 --2- 172.21.15.64:0/1960763921 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa854106670 0x7fa8541986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:37.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.717+0000 7fa85c1ff700 1 --2- 172.21.15.64:0/1960763921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8541005b0 0x7fa85419e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:37.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.717+0000 7fa85c1ff700 1 --2- 172.21.15.64:0/1960763921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa854100f70 0x7fa85419ed80 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:37.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.717+0000 7fa85c1ff700 1 -- 172.21.15.64:0/1960763921 >> 172.21.15.64:0/1960763921 conn(0x7fa8540fc060 msgr2=0x7fa8540fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:25:37.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.718+0000 7fa85c1ff700 1 -- 172.21.15.64:0/1960763921 shutdown_connections 2023-09-13T03:25:37.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:37.718+0000 7fa85c1ff700 1 -- 172.21.15.64:0/1960763921 wait complete. 2023-09-13T03:25:37.957 INFO:tasks.workunit.client.0.smithi064.stdout: 1 354149 2.78 MB/sec execute 582 sec latency 60.883 ms 2023-09-13T03:25:38.957 INFO:tasks.workunit.client.0.smithi064.stdout: 1 355433 2.79 MB/sec execute 583 sec latency 19.322 ms 2023-09-13T03:25:39.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:38 smithi170 ceph-mon[134328]: pgmap v389: 97 pgs: 97 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 182 op/s 2023-09-13T03:25:39.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:38 smithi064 ceph-mon[97675]: pgmap v389: 97 pgs: 97 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 182 op/s 2023-09-13T03:25:39.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:38 smithi142 ceph-mon[134082]: pgmap v389: 97 pgs: 97 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 182 op/s 2023-09-13T03:25:39.957 INFO:tasks.workunit.client.0.smithi064.stdout: 1 357291 2.80 MB/sec execute 584 sec latency 21.337 ms 2023-09-13T03:25:40.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:40 smithi142 ceph-mon[134082]: pgmap v390: 97 pgs: 97 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 182 op/s 2023-09-13T03:25:40.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:40 smithi170 ceph-mon[134328]: pgmap v390: 97 pgs: 97 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 182 op/s 2023-09-13T03:25:40.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:40 smithi064 ceph-mon[97675]: pgmap v390: 97 pgs: 97 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 182 op/s 2023-09-13T03:25:40.956 INFO:tasks.workunit.client.0.smithi064.stdout: 1 358723 2.81 MB/sec execute 585 sec latency 16.695 ms 2023-09-13T03:25:41.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:41 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:25:41.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:41 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:25:41.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:41 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:25:41.957 INFO:tasks.workunit.client.0.smithi064.stdout: 1 359898 2.81 MB/sec execute 586 sec latency 21.625 ms 2023-09-13T03:25:42.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:42 smithi142 ceph-mon[134082]: pgmap v391: 97 pgs: 97 active+clean; 153 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 292 op/s 2023-09-13T03:25:42.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:42 smithi170 ceph-mon[134328]: pgmap v391: 97 pgs: 97 active+clean; 153 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 292 op/s 2023-09-13T03:25:42.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:42 smithi064 ceph-mon[97675]: pgmap v391: 97 pgs: 97 active+clean; 153 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 292 op/s 2023-09-13T03:25:42.957 INFO:tasks.workunit.client.0.smithi064.stdout: 1 361623 2.83 MB/sec execute 587 sec latency 29.363 ms 2023-09-13T03:25:43.957 INFO:tasks.workunit.client.0.smithi064.stdout: 1 362715 2.83 MB/sec execute 588 sec latency 16.474 ms 2023-09-13T03:25:44.957 INFO:tasks.workunit.client.0.smithi064.stdout: 1 364665 2.84 MB/sec execute 589 sec latency 22.766 ms 2023-09-13T03:25:45.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:44 smithi170 ceph-mon[134328]: pgmap v392: 97 pgs: 97 active+clean; 153 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 195 op/s 2023-09-13T03:25:45.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:44 smithi064 ceph-mon[97675]: pgmap v392: 97 pgs: 97 active+clean; 153 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 195 op/s 2023-09-13T03:25:45.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:44 smithi142 ceph-mon[134082]: pgmap v392: 97 pgs: 97 active+clean; 153 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 195 op/s 2023-09-13T03:25:45.957 INFO:tasks.workunit.client.0.smithi064.stdout: 1 365882 2.85 MB/sec execute 590 sec latency 10.648 ms 2023-09-13T03:25:46.957 INFO:tasks.workunit.client.0.smithi064.stdout: 1 367035 2.86 MB/sec execute 591 sec latency 21.509 ms 2023-09-13T03:25:47.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:46 smithi170 ceph-mon[134328]: pgmap v393: 97 pgs: 97 active+clean; 159 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.1 MiB/s wr, 291 op/s 2023-09-13T03:25:47.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:46 smithi064 ceph-mon[97675]: pgmap v393: 97 pgs: 97 active+clean; 159 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.1 MiB/s wr, 291 op/s 2023-09-13T03:25:47.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:46 smithi142 ceph-mon[134082]: pgmap v393: 97 pgs: 97 active+clean; 159 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.1 MiB/s wr, 291 op/s 2023-09-13T03:25:47.957 INFO:tasks.workunit.client.0.smithi064.stdout: 1 368715 2.87 MB/sec execute 592 sec latency 15.413 ms 2023-09-13T03:25:48.957 INFO:tasks.workunit.client.0.smithi064.stdout: 1 369797 2.87 MB/sec execute 593 sec latency 17.804 ms 2023-09-13T03:25:49.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:48 smithi170 ceph-mon[134328]: pgmap v394: 97 pgs: 97 active+clean; 159 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 206 op/s 2023-09-13T03:25:49.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:48 smithi064 ceph-mon[97675]: pgmap v394: 97 pgs: 97 active+clean; 159 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 206 op/s 2023-09-13T03:25:49.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:48 smithi142 ceph-mon[134082]: pgmap v394: 97 pgs: 97 active+clean; 159 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 206 op/s 2023-09-13T03:25:49.957 INFO:tasks.workunit.client.0.smithi064.stdout: 1 371502 2.88 MB/sec execute 594 sec latency 28.005 ms 2023-09-13T03:25:50.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:50 smithi142 ceph-mon[134082]: pgmap v395: 97 pgs: 97 active+clean; 159 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 206 op/s 2023-09-13T03:25:50.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.889+0000 7f83127ff700 1 -- 172.21.15.64:0/4008727155 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f830c0729a0 msgr2=0x7f830c072d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:25:50.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.889+0000 7f83127ff700 1 --2- 172.21.15.64:0/4008727155 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f830c0729a0 0x7f830c072d80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f82fc009990 tx=0x7f82fc009ca0 comp rx=0 tx=0).stop 2023-09-13T03:25:50.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.890+0000 7f83127ff700 1 -- 172.21.15.64:0/4008727155 shutdown_connections 2023-09-13T03:25:50.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.890+0000 7f83127ff700 1 --2- 172.21.15.64:0/4008727155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f830c111590 0x7f830c113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:50.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.890+0000 7f83127ff700 1 --2- 172.21.15.64:0/4008727155 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f830c0729a0 0x7f830c072d80 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.890+0000 7f83127ff700 1 --2- 172.21.15.64:0/4008727155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f830c0732c0 0x7f830c111050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.890+0000 7f83127ff700 1 -- 172.21.15.64:0/4008727155 >> 172.21.15.64:0/4008727155 conn(0x7f830c0fc050 msgr2=0x7f830c0fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:25:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.890+0000 7f83127ff700 1 -- 172.21.15.64:0/4008727155 shutdown_connections 2023-09-13T03:25:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.890+0000 7f83127ff700 1 -- 172.21.15.64:0/4008727155 wait complete. 2023-09-13T03:25:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.891+0000 7f83127ff700 1 Processor -- start 2023-09-13T03:25:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.891+0000 7f83127ff700 1 -- start start 2023-09-13T03:25:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.891+0000 7f83127ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f830c0729a0 0x7f830c19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.891+0000 7f83127ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f830c0732c0 0x7f830c19d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.891+0000 7f83127ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f830c111590 0x7f830c1a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.891+0000 7f83127ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830c1a0db0 con 0x7f830c111590 2023-09-13T03:25:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.891+0000 7f83127ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830c1a0f20 con 0x7f830c0732c0 2023-09-13T03:25:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.891+0000 7f83127ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830c1a10e0 con 0x7f830c0729a0 2023-09-13T03:25:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.892+0000 7f830bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f830c0729a0 0x7f830c19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.892+0000 7f830b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f830c0732c0 0x7f830c19d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.892+0000 7f830bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f830c0729a0 0x7f830c19ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:54924/0 (socket says 172.21.15.64:54924) 2023-09-13T03:25:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.892+0000 7f830bfff700 1 -- 172.21.15.64:0/2813617980 learned_addr learned my addr 172.21.15.64:0/2813617980 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:25:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.892+0000 7f8310d9c700 1 --2- 172.21.15.64:0/2813617980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f830c111590 0x7f830c1a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.892+0000 7f830bfff700 1 -- 172.21.15.64:0/2813617980 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f830c0732c0 msgr2=0x7f830c19d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:25:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.892+0000 7f830bfff700 1 --2- 172.21.15.64:0/2813617980 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f830c0732c0 0x7f830c19d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.892+0000 7f830bfff700 1 -- 172.21.15.64:0/2813617980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f830c111590 msgr2=0x7f830c1a07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:25:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.892+0000 7f830bfff700 1 --2- 172.21.15.64:0/2813617980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f830c111590 0x7f830c1a07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.892+0000 7f830bfff700 1 -- 172.21.15.64:0/2813617980 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82fc0095f0 con 0x7f830c0729a0 2023-09-13T03:25:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.892+0000 7f830bfff700 1 --2- 172.21.15.64:0/2813617980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f830c0729a0 0x7f830c19ccb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f82fc005910 tx=0x7f82fc004710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:25:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.893+0000 7f83097fa700 1 -- 172.21.15.64:0/2813617980 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82fc01d070 con 0x7f830c0729a0 2023-09-13T03:25:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.893+0000 7f83097fa700 1 -- 172.21.15.64:0/2813617980 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82fc022470 con 0x7f830c0729a0 2023-09-13T03:25:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.893+0000 7f83127ff700 1 -- 172.21.15.64:0/2813617980 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f830c1a1360 con 0x7f830c0729a0 2023-09-13T03:25:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.893+0000 7f83097fa700 1 -- 172.21.15.64:0/2813617980 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82fc017650 con 0x7f830c0729a0 2023-09-13T03:25:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.893+0000 7f83127ff700 1 -- 172.21.15.64:0/2813617980 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f830c1a9d20 con 0x7f830c0729a0 2023-09-13T03:25:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.896+0000 7f83097fa700 1 -- 172.21.15.64:0/2813617980 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f82fc0225e0 con 0x7f830c0729a0 2023-09-13T03:25:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.897+0000 7f832608fb80 1 Processor -- start 2023-09-13T03:25:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.898+0000 7f832608fb80 1 -- start start 2023-09-13T03:25:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.898+0000 7f832608fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f4ef5bc1a0 0x55f4ef5bc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.898+0000 7f832608fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4ef5b6300 0x55f4ef5b6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.898+0000 7f832608fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4ef5b6cc0 0x55f4ef5c2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.898+0000 7f832608fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4ef50f6d0 con 0x55f4ef5bc1a0 2023-09-13T03:25:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.898+0000 7f832608fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4ef50cff0 con 0x55f4ef5b6cc0 2023-09-13T03:25:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.898+0000 7f832608fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4ef5088e0 con 0x55f4ef5b6300 2023-09-13T03:25:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.898+0000 7f83097fa700 1 --2- 172.21.15.64:0/2813617980 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f82f40a0830 0x7f82f40a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.898+0000 7f830b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4ef5b6cc0 0x55f4ef5c2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.898+0000 7f830bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4ef5b6300 0x55f4ef5b6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.898+0000 7f830b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4ef5b6cc0 0x55f4ef5c2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:55012/0 (socket says 172.21.15.64:55012) 2023-09-13T03:25:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.898+0000 7f830b7fe700 1 -- 172.21.15.64:0/2545466990 learned_addr learned my addr 172.21.15.64:0/2545466990 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:25:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.898+0000 7f8310d9c700 1 --2- 172.21.15.64:0/2545466990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f4ef5bc1a0 0x55f4ef5bc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.899+0000 7f8310d9c700 1 --2- 172.21.15.64:0/2813617980 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f82f40a0830 0x7f82f40a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.899+0000 7f83097fa700 1 -- 172.21.15.64:0/2813617980 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f82fc0cd180 con 0x7f830c0729a0 2023-09-13T03:25:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.899+0000 7f830bfff700 1 -- 172.21.15.64:0/2545466990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4ef5b6cc0 msgr2=0x55f4ef5c2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:25:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.899+0000 7f830bfff700 1 --2- 172.21.15.64:0/2545466990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4ef5b6cc0 0x55f4ef5c2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.899+0000 7f830bfff700 1 -- 172.21.15.64:0/2545466990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f4ef5bc1a0 msgr2=0x55f4ef5bc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:25:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.899+0000 7f830bfff700 1 --2- 172.21.15.64:0/2545466990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f4ef5bc1a0 0x55f4ef5bc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.899+0000 7f830bfff700 1 -- 172.21.15.64:0/2545466990 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4ef334d80 con 0x55f4ef5b6300 2023-09-13T03:25:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.899+0000 7f830b7fe700 1 --2- 172.21.15.64:0/2545466990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4ef5b6cc0 0x55f4ef5c2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:25:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.899+0000 7f8310d9c700 1 --2- 172.21.15.64:0/2545466990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f4ef5bc1a0 0x55f4ef5bc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:25:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.899+0000 7f830bfff700 1 --2- 172.21.15.64:0/2545466990 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4ef5b6300 0x55f4ef5b6780 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f82fc0ca890 tx=0x7f82fc022cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0be59b47eefcc9 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:25:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.899+0000 7f82f27fc700 1 -- 172.21.15.64:0/2545466990 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82fc099fe0 con 0x55f4ef5b6300 2023-09-13T03:25:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.899+0000 7f82f27fc700 1 -- 172.21.15.64:0/2545466990 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82fc09a680 con 0x55f4ef5b6300 2023-09-13T03:25:50.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.899+0000 7f82f27fc700 1 -- 172.21.15.64:0/2545466990 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82fc09a8e0 con 0x55f4ef5b6300 2023-09-13T03:25:50.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.900+0000 7f8310d9c700 1 --2- 172.21.15.64:0/2813617980 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f82f40a0830 0x7f82f40a2cf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8304009fd0 tx=0x7f830400b040 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:25:50.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.900+0000 7f832608fb80 1 -- 172.21.15.64:0/2545466990 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4ef5b6300 msgr2=0x55f4ef5b6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:25:50.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.900+0000 7f832608fb80 1 --2- 172.21.15.64:0/2545466990 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4ef5b6300 0x55f4ef5b6780 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f82fc0ca890 tx=0x7f82fc022cd0 comp rx=0 tx=0).stop 2023-09-13T03:25:50.908 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.900+0000 7f832608fb80 1 -- 172.21.15.64:0/2545466990 shutdown_connections 2023-09-13T03:25:50.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.900+0000 7f832608fb80 1 --2- 172.21.15.64:0/2545466990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4ef5b6cc0 0x55f4ef5c2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:50.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.900+0000 7f832608fb80 1 --2- 172.21.15.64:0/2545466990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f4ef5bc1a0 0x55f4ef5bc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:50.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.900+0000 7f832608fb80 1 --2- 172.21.15.64:0/2545466990 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4ef5b6300 0x55f4ef5b6780 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:50.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.900+0000 7f832608fb80 1 -- 172.21.15.64:0/2545466990 >> 172.21.15.64:0/2545466990 conn(0x55f4ef52b030 msgr2=0x55f4ef5b1560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:25:50.909 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.900+0000 7f832608fb80 1 -- 172.21.15.64:0/2545466990 shutdown_connections 2023-09-13T03:25:50.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.900+0000 7f832608fb80 1 -- 172.21.15.64:0/2545466990 wait complete. 2023-09-13T03:25:50.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.901+0000 7f832608fb80 1 -- start start 2023-09-13T03:25:50.910 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.901+0000 7f832608fb80 1 Processor -- start 2023-09-13T03:25:50.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:50 smithi170 ceph-mon[134328]: pgmap v395: 97 pgs: 97 active+clean; 159 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 206 op/s 2023-09-13T03:25:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.902+0000 7f832608fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f4ef5b6300 0x55f4ef5c5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.902+0000 7f832608fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4ef5b6cc0 0x55f4ef5c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.902+0000 7f832608fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4ef5bc1a0 0x55f4ef5bfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.902+0000 7f832608fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4ef4a6b10 con 0x55f4ef5b6300 2023-09-13T03:25:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.902+0000 7f832608fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4ef4a6810 con 0x55f4ef5bc1a0 2023-09-13T03:25:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.902+0000 7f832608fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4ef38a8c0 con 0x55f4ef5b6cc0 2023-09-13T03:25:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.902+0000 7f832608fb80 10 client.0 _ll_register_callbacks cb 0x55f4ef3f0a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:25:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.902+0000 7f830b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4ef5b6cc0 0x55f4ef5c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.902+0000 7f8310d9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4ef5bc1a0 0x55f4ef5bfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.902+0000 7f830b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4ef5b6cc0 0x55f4ef5c6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:54948/0 (socket says 172.21.15.64:54948) 2023-09-13T03:25:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.902+0000 7f830b7fe700 1 -- 172.21.15.64:0/3970601615 learned_addr learned my addr 172.21.15.64:0/3970601615 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:25:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.902+0000 7f830bfff700 1 --2- 172.21.15.64:0/3970601615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f4ef5b6300 0x55f4ef5c5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.903+0000 7f830b7fe700 1 -- 172.21.15.64:0/3970601615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4ef5bc1a0 msgr2=0x55f4ef5bfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:25:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.903+0000 7f830b7fe700 1 --2- 172.21.15.64:0/3970601615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4ef5bc1a0 0x55f4ef5bfd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.903+0000 7f830b7fe700 1 -- 172.21.15.64:0/3970601615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f4ef5b6300 msgr2=0x55f4ef5c5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:25:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.903+0000 7f830b7fe700 1 --2- 172.21.15.64:0/3970601615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f4ef5b6300 0x55f4ef5c5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.903+0000 7f830b7fe700 1 -- 172.21.15.64:0/3970601615 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8300009920 con 0x55f4ef5b6cc0 2023-09-13T03:25:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.903+0000 7f8310d9c700 1 --2- 172.21.15.64:0/3970601615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4ef5bc1a0 0x55f4ef5bfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:25:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.903+0000 7f830b7fe700 1 --2- 172.21.15.64:0/3970601615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4ef5b6cc0 0x55f4ef5c6010 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8300012a10 tx=0x7f8300012d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b001251776528105 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:25:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.903+0000 7f82eb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:25:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.903+0000 7f82eb7fe700 1 -- 172.21.15.64:0/3970601615 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8300017070 con 0x55f4ef5b6cc0 2023-09-13T03:25:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.903+0000 7f82eb7fe700 1 -- 172.21.15.64:0/3970601615 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f8300004020 con 0x55f4ef5b6cc0 2023-09-13T03:25:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.903+0000 7f832608fb80 1 -- 172.21.15.64:0/3970601615 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f4ef460490 con 0x55f4ef5b6cc0 2023-09-13T03:25:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.903+0000 7f82eb7fe700 1 -- 172.21.15.64:0/3970601615 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83000041e0 con 0x55f4ef5b6cc0 2023-09-13T03:25:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.904+0000 7f832608fb80 10 client.27130 fetch_fsmap learned FSMap version 65 2023-09-13T03:25:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.904+0000 7f832608fb80 1 -- 172.21.15.64:0/3970601615 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) v3 -- 0x7f830c1a9d20 con 0x55f4ef5b6cc0 2023-09-13T03:25:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.904+0000 7f82eb7fe700 1 -- 172.21.15.64:0/3970601615 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f4ef460490 con 0x55f4ef5b6cc0 2023-09-13T03:25:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.904+0000 7f82eb7fe700 1 -- 172.21.15.64:0/3970601615 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 65) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f83000037a0 con 0x55f4ef5b6cc0 2023-09-13T03:25:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.904+0000 7f832608fb80 10 client.27130 fetch_fsmap finished waiting for FSMap version 65 2023-09-13T03:25:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.904+0000 7f832608fb80 10 client.27130 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:25:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.904+0000 7f832608fb80 20 client.27130 populate_metadata read hostname 'smithi064' 2023-09-13T03:25:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.904+0000 7f832608fb80 1 --2- 172.21.15.64:0/3970601615 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f4ef5d0060 0x55f4ef5d2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.904+0000 7f832608fb80 4 client.27130 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:25:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.904+0000 7f832608fb80 1 -- 172.21.15.64:0/3970601615 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f4ef51d840 con 0x55f4ef5d0060 2023-09-13T03:25:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.904+0000 7f830bfff700 1 --2- 172.21.15.64:0/3970601615 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f4ef5d0060 0x55f4ef5d2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.905+0000 7f830bfff700 1 --2- 172.21.15.64:0/3970601615 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f4ef5d0060 0x55f4ef5d2480 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8e1fe20f66112f6 server_cookie=b955cc75fa704768 in_seq=0 out_seq=0 2023-09-13T03:25:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.905+0000 7f82eb7fe700 10 client.27130 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:25:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.905+0000 7f82eb7fe700 1 -- 172.21.15.64:0/3970601615 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f4ef51d840 con 0x55f4ef5d0060 2023-09-13T03:25:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.905+0000 7f82eb7fe700 10 client.27130 handle_command_reply: tid=0 2023-09-13T03:25:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.906+0000 7f832608fb80 1 client.27130 shutdown 2023-09-13T03:25:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.906+0000 7f832608fb80 1 -- 172.21.15.64:0/3970601615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4ef5b6cc0 msgr2=0x55f4ef5c6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:25:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.906+0000 7f832608fb80 1 --2- 172.21.15.64:0/3970601615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4ef5b6cc0 0x55f4ef5c6010 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8300012a10 tx=0x7f8300012d20 comp rx=0 tx=0).stop 2023-09-13T03:25:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.906+0000 7f832608fb80 1 -- 172.21.15.64:0/3970601615 shutdown_connections 2023-09-13T03:25:50.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.906+0000 7f832608fb80 1 --2- 172.21.15.64:0/3970601615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4ef5bc1a0 0x55f4ef5bfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.906+0000 7f832608fb80 1 --2- 172.21.15.64:0/3970601615 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f4ef5d0060 0x55f4ef5d2480 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.906+0000 7f832608fb80 1 --2- 172.21.15.64:0/3970601615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f4ef5b6300 0x55f4ef5c5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.906+0000 7f832608fb80 1 --2- 172.21.15.64:0/3970601615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4ef5b6cc0 0x55f4ef5c6010 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.906+0000 7f82eb7fe700 0 client.27130 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:25:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.906+0000 7f832608fb80 1 -- 172.21.15.64:0/3970601615 >> 172.21.15.64:0/3970601615 conn(0x55f4ef52b030 msgr2=0x55f4ef5b07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:25:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.906+0000 7f832608fb80 1 -- 172.21.15.64:0/3970601615 shutdown_connections 2023-09-13T03:25:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.906+0000 7f832608fb80 1 -- 172.21.15.64:0/3970601615 wait complete. 2023-09-13T03:25:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.906+0000 7f832608fb80 20 client.27130 trim_cache size 0 max 16384 2023-09-13T03:25:50.927 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:50 smithi064 ceph-mon[97675]: pgmap v395: 97 pgs: 97 active+clean; 159 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 206 op/s 2023-09-13T03:25:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.923+0000 7f832608fb80 1 Processor -- start 2023-09-13T03:25:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.923+0000 7f832608fb80 1 -- start start 2023-09-13T03:25:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.924+0000 7f832608fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4ef5b6300 0x55f4ef5d2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.924+0000 7f832608fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4ef5b6cc0 0x55f4ef5b0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.924+0000 7f832608fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f4ef5bc1a0 0x55f4ef5b1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.924+0000 7f832608fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4ef5c1300 con 0x55f4ef5bc1a0 2023-09-13T03:25:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.924+0000 7f832608fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4ef5b2d10 con 0x55f4ef5b6300 2023-09-13T03:25:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.924+0000 7f832608fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4ef5be280 con 0x55f4ef5b6cc0 2023-09-13T03:25:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.924+0000 7f830bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4ef5b6300 0x55f4ef5d2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.924+0000 7f830b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4ef5b6cc0 0x55f4ef5b0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.924+0000 7f8310d9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f4ef5bc1a0 0x55f4ef5b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.924+0000 7f830b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4ef5b6cc0 0x55f4ef5b0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:54954/0 (socket says 172.21.15.64:54954) 2023-09-13T03:25:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.924+0000 7f8310d9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f4ef5bc1a0 0x55f4ef5b1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53000/0 (socket says 172.21.15.64:53000) 2023-09-13T03:25:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.924+0000 7f8310d9c700 1 -- 172.21.15.64:0/917851518 learned_addr learned my addr 172.21.15.64:0/917851518 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:25:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.924+0000 7f830bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4ef5b6300 0x55f4ef5d2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:55026/0 (socket says 172.21.15.64:55026) 2023-09-13T03:25:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.924+0000 7f830bfff700 1 -- 172.21.15.64:0/917851518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4ef5b6cc0 msgr2=0x55f4ef5b0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:25:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.924+0000 7f830bfff700 1 --2- 172.21.15.64:0/917851518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4ef5b6cc0 0x55f4ef5b0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.924+0000 7f830bfff700 1 -- 172.21.15.64:0/917851518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f4ef5bc1a0 msgr2=0x55f4ef5b1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:25:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.924+0000 7f830bfff700 1 --2- 172.21.15.64:0/917851518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f4ef5bc1a0 0x55f4ef5b1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.924+0000 7f830bfff700 1 -- 172.21.15.64:0/917851518 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4ef31dc00 con 0x55f4ef5b6300 2023-09-13T03:25:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.925+0000 7f830bfff700 1 --2- 172.21.15.64:0/917851518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4ef5b6300 0x55f4ef5d2450 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f82fc09a810 tx=0x7f82fc000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88d289bcc9599de0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:25:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.925+0000 7f82f2ffd700 1 -- 172.21.15.64:0/917851518 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82fc099fe0 con 0x55f4ef5b6300 2023-09-13T03:25:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.925+0000 7f82f2ffd700 1 -- 172.21.15.64:0/917851518 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82fc09ad80 con 0x55f4ef5b6300 2023-09-13T03:25:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.925+0000 7f82f2ffd700 1 -- 172.21.15.64:0/917851518 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82fc09e170 con 0x55f4ef5b6300 2023-09-13T03:25:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.925+0000 7f832608fb80 1 -- 172.21.15.64:0/917851518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4ef5b6300 msgr2=0x55f4ef5d2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:25:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.925+0000 7f832608fb80 1 --2- 172.21.15.64:0/917851518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4ef5b6300 0x55f4ef5d2450 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f82fc09a810 tx=0x7f82fc000bc0 comp rx=0 tx=0).stop 2023-09-13T03:25:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.926+0000 7f832608fb80 1 -- 172.21.15.64:0/917851518 shutdown_connections 2023-09-13T03:25:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.926+0000 7f832608fb80 1 --2- 172.21.15.64:0/917851518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4ef5b6300 0x55f4ef5d2450 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.926+0000 7f832608fb80 1 --2- 172.21.15.64:0/917851518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f4ef5bc1a0 0x55f4ef5b1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.926+0000 7f832608fb80 1 --2- 172.21.15.64:0/917851518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4ef5b6cc0 0x55f4ef5b0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.926+0000 7f832608fb80 1 -- 172.21.15.64:0/917851518 >> 172.21.15.64:0/917851518 conn(0x55f4ef52b030 msgr2=0x55f4ef5c06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:25:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.926+0000 7f832608fb80 1 -- 172.21.15.64:0/917851518 shutdown_connections 2023-09-13T03:25:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.926+0000 7f832608fb80 1 -- 172.21.15.64:0/917851518 wait complete. 2023-09-13T03:25:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.926+0000 7f832608fb80 1 -- start start 2023-09-13T03:25:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.926+0000 7f832608fb80 1 Processor -- start 2023-09-13T03:25:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.927+0000 7f832608fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f4ef5b6300 0x55f4ef5ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.927+0000 7f832608fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4ef5b6cc0 0x55f4ef5ca800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.927+0000 7f832608fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4ef5bc1a0 0x55f4ef622e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.927+0000 7f832608fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4ef6233d0 con 0x55f4ef5b6300 2023-09-13T03:25:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.927+0000 7f832608fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4ef623510 con 0x55f4ef5b6cc0 2023-09-13T03:25:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.927+0000 7f832608fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4ef6236a0 con 0x55f4ef5bc1a0 2023-09-13T03:25:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.927+0000 7f832608fb80 10 client.0 _ll_register_callbacks cb 0x7f83000124e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:25:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.927+0000 7f830b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4ef5b6cc0 0x55f4ef5ca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.927+0000 7f830b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4ef5b6cc0 0x55f4ef5ca800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:55030/0 (socket says 172.21.15.64:55030) 2023-09-13T03:25:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.927+0000 7f830bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f4ef5b6300 0x55f4ef5ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.927+0000 7f8310d9c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4ef5bc1a0 0x55f4ef622e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.927+0000 7f830b7fe700 1 -- 172.21.15.64:0/165101113 learned_addr learned my addr 172.21.15.64:0/165101113 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:25:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.927+0000 7f830b7fe700 1 -- 172.21.15.64:0/165101113 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4ef5bc1a0 msgr2=0x55f4ef622e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:25:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.927+0000 7f830b7fe700 1 --2- 172.21.15.64:0/165101113 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4ef5bc1a0 0x55f4ef622e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.927+0000 7f830b7fe700 1 -- 172.21.15.64:0/165101113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f4ef5b6300 msgr2=0x55f4ef5ca2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:25:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.927+0000 7f830b7fe700 1 --2- 172.21.15.64:0/165101113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f4ef5b6300 0x55f4ef5ca2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.927+0000 7f830b7fe700 1 -- 172.21.15.64:0/165101113 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f4ef334d80 con 0x55f4ef5b6cc0 2023-09-13T03:25:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.928+0000 7f830b7fe700 1 --2- 172.21.15.64:0/165101113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4ef5b6cc0 0x55f4ef5ca800 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8300012730 tx=0x7f8300009b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e612de86a956e483 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:25:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.928+0000 7f82eb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:25:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.928+0000 7f82eb7fe700 1 -- 172.21.15.64:0/165101113 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8300017040 con 0x55f4ef5b6cc0 2023-09-13T03:25:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.928+0000 7f832608fb80 1 -- 172.21.15.64:0/165101113 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f82fc09a680 con 0x55f4ef5b6cc0 2023-09-13T03:25:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.928+0000 7f82eb7fe700 1 -- 172.21.15.64:0/165101113 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f830000fde0 con 0x55f4ef5b6cc0 2023-09-13T03:25:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.928+0000 7f82eb7fe700 1 -- 172.21.15.64:0/165101113 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83000050c0 con 0x55f4ef5b6cc0 2023-09-13T03:25:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.928+0000 7f832608fb80 10 client.27152 fetch_fsmap learned FSMap version 65 2023-09-13T03:25:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.928+0000 7f832608fb80 1 -- 172.21.15.64:0/165101113 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) v3 -- 0x55f4ef35b1f0 con 0x55f4ef5b6cc0 2023-09-13T03:25:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.929+0000 7f82eb7fe700 1 -- 172.21.15.64:0/165101113 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82fc09a680 con 0x55f4ef5b6cc0 2023-09-13T03:25:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.929+0000 7f82eb7fe700 1 -- 172.21.15.64:0/165101113 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 65) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f830001c850 con 0x55f4ef5b6cc0 2023-09-13T03:25:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.929+0000 7f832608fb80 10 client.27152 fetch_fsmap finished waiting for FSMap version 65 2023-09-13T03:25:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.929+0000 7f832608fb80 10 client.27152 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:25:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.929+0000 7f832608fb80 20 client.27152 populate_metadata read hostname 'smithi064' 2023-09-13T03:25:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.929+0000 7f832608fb80 1 --2- 172.21.15.64:0/165101113 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f4ef624e30 0x55f4ef627250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:25:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.929+0000 7f832608fb80 4 client.27152 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:25:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.929+0000 7f832608fb80 1 -- 172.21.15.64:0/165101113 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f4ef5018c0 con 0x55f4ef624e30 2023-09-13T03:25:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.929+0000 7f830bfff700 1 --2- 172.21.15.64:0/165101113 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f4ef624e30 0x55f4ef627250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:25:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.929+0000 7f830bfff700 1 --2- 172.21.15.64:0/165101113 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f4ef624e30 0x55f4ef627250 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=705d7d2752131f6c server_cookie=765c97b318663465 in_seq=0 out_seq=0 2023-09-13T03:25:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.929+0000 7f82eb7fe700 10 client.27152 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:25:50.957 INFO:tasks.workunit.client.0.smithi064.stdout: 1 372900 2.89 MB/sec execute 595 sec latency 28.779 ms 2023-09-13T03:25:50.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.973+0000 7f82eb7fe700 1 -- 172.21.15.64:0/165101113 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f4ef5018c0 con 0x55f4ef624e30 2023-09-13T03:25:50.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.973+0000 7f82eb7fe700 10 client.27152 handle_command_reply: tid=0 2023-09-13T03:25:50.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.973+0000 7f832608fb80 1 client.27152 shutdown 2023-09-13T03:25:50.975 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.973+0000 7f832608fb80 1 -- 172.21.15.64:0/165101113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4ef5b6cc0 msgr2=0x55f4ef5ca800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:25:50.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.973+0000 7f832608fb80 1 --2- 172.21.15.64:0/165101113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4ef5b6cc0 0x55f4ef5ca800 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8300012730 tx=0x7f8300009b20 comp rx=0 tx=0).stop 2023-09-13T03:25:50.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.973+0000 7f832608fb80 1 -- 172.21.15.64:0/165101113 shutdown_connections 2023-09-13T03:25:50.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.973+0000 7f832608fb80 1 --2- 172.21.15.64:0/165101113 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f4ef624e30 0x55f4ef627250 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:50.976 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.974+0000 7f832608fb80 1 --2- 172.21.15.64:0/165101113 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f4ef5bc1a0 0x55f4ef622e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:50.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.974+0000 7f832608fb80 1 --2- 172.21.15.64:0/165101113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f4ef5b6300 0x55f4ef5ca2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:50.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.974+0000 7f82eb7fe700 0 client.27152 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:25:50.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.974+0000 7f832608fb80 1 --2- 172.21.15.64:0/165101113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4ef5b6cc0 0x55f4ef5ca800 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:50.977 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.974+0000 7f832608fb80 1 -- 172.21.15.64:0/165101113 >> 172.21.15.64:0/165101113 conn(0x55f4ef52b030 msgr2=0x55f4ef5bae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:25:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.974+0000 7f832608fb80 1 -- 172.21.15.64:0/165101113 shutdown_connections 2023-09-13T03:25:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.974+0000 7f832608fb80 1 -- 172.21.15.64:0/165101113 wait complete. 2023-09-13T03:25:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.974+0000 7f832608fb80 20 client.27152 trim_cache size 0 max 16384 2023-09-13T03:25:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:25:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:25:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:25:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:25:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.975+0000 7f83127ff700 1 -- 172.21.15.64:0/2813617980 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f82f40a0830 msgr2=0x7f82f40a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:25:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.975+0000 7f83127ff700 1 --2- 172.21.15.64:0/2813617980 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f82f40a0830 0x7f82f40a2cf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8304009fd0 tx=0x7f830400b040 comp rx=0 tx=0).stop 2023-09-13T03:25:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.975+0000 7f83127ff700 1 -- 172.21.15.64:0/2813617980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f830c0729a0 msgr2=0x7f830c19ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:25:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.975+0000 7f83127ff700 1 --2- 172.21.15.64:0/2813617980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f830c0729a0 0x7f830c19ccb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f82fc005910 tx=0x7f82fc004710 comp rx=0 tx=0).stop 2023-09-13T03:25:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.977+0000 7f83127ff700 1 -- 172.21.15.64:0/2813617980 shutdown_connections 2023-09-13T03:25:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.977+0000 7f83127ff700 1 --2- 172.21.15.64:0/2813617980 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f82f40a0830 0x7f82f40a2cf0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.977+0000 7f83127ff700 1 --2- 172.21.15.64:0/2813617980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f830c111590 0x7f830c1a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.977+0000 7f83127ff700 1 --2- 172.21.15.64:0/2813617980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f830c0729a0 0x7f830c19ccb0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.977+0000 7f83127ff700 1 --2- 172.21.15.64:0/2813617980 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f830c0732c0 0x7f830c19d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:25:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.977+0000 7f83127ff700 1 -- 172.21.15.64:0/2813617980 >> 172.21.15.64:0/2813617980 conn(0x7f830c0fc050 msgr2=0x7f830c0fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:25:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.977+0000 7f83127ff700 1 -- 172.21.15.64:0/2813617980 shutdown_connections 2023-09-13T03:25:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:25:50.977+0000 7f83127ff700 1 -- 172.21.15.64:0/2813617980 wait complete. 2023-09-13T03:25:51.957 INFO:tasks.workunit.client.0.smithi064.stdout: 1 374050 2.90 MB/sec execute 596 sec latency 29.937 ms 2023-09-13T03:25:52.958 INFO:tasks.workunit.client.0.smithi064.stdout: 1 375441 2.90 MB/sec execute 597 sec latency 31.739 ms 2023-09-13T03:25:53.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:52 smithi170 ceph-mon[134328]: pgmap v396: 97 pgs: 97 active+clean; 166 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 285 op/s 2023-09-13T03:25:53.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:52 smithi064 ceph-mon[97675]: pgmap v396: 97 pgs: 97 active+clean; 166 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 285 op/s 2023-09-13T03:25:53.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:52 smithi142 ceph-mon[134082]: pgmap v396: 97 pgs: 97 active+clean; 166 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 285 op/s 2023-09-13T03:25:53.958 INFO:tasks.workunit.client.0.smithi064.stdout: 1 376636 2.91 MB/sec execute 598 sec latency 21.993 ms 2023-09-13T03:25:54.958 INFO:tasks.workunit.client.0.smithi064.stdout: 1 378444 2.92 MB/sec execute 599 sec latency 18.977 ms 2023-09-13T03:25:55.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:54 smithi170 ceph-mon[134328]: pgmap v397: 97 pgs: 97 active+clean; 166 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 175 op/s 2023-09-13T03:25:55.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:54 smithi064 ceph-mon[97675]: pgmap v397: 97 pgs: 97 active+clean; 166 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 175 op/s 2023-09-13T03:25:55.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:54 smithi142 ceph-mon[134082]: pgmap v397: 97 pgs: 97 active+clean; 166 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 175 op/s 2023-09-13T03:25:55.958 INFO:tasks.workunit.client.0.smithi064.stdout: 1 cleanup 600 sec 2023-09-13T03:25:56.174 INFO:tasks.workunit.client.0.smithi064.stdout: 0 cleanup 600 sec 2023-09-13T03:25:56.174 INFO:tasks.workunit.client.0.smithi064.stdout: 2023-09-13T03:25:56.175 INFO:tasks.workunit.client.0.smithi064.stdout: Operation Count AvgLat MaxLat 2023-09-13T03:25:56.175 INFO:tasks.workunit.client.0.smithi064.stdout: ---------------------------------------- 2023-09-13T03:25:56.175 INFO:tasks.workunit.client.0.smithi064.stdout: NTCreateX 56355 6.762 16285.606 2023-09-13T03:25:56.175 INFO:tasks.workunit.client.0.smithi064.stdout: Close 41400 0.005 0.194 2023-09-13T03:25:56.175 INFO:tasks.workunit.client.0.smithi064.stdout: Rename 2390 42.499 4978.298 2023-09-13T03:25:56.176 INFO:tasks.workunit.client.0.smithi064.stdout: Unlink 11366 2.619 4712.773 2023-09-13T03:25:56.176 INFO:tasks.workunit.client.0.smithi064.stdout: Qpathinfo 51110 1.237 222.334 2023-09-13T03:25:56.176 INFO:tasks.workunit.client.0.smithi064.stdout: Qfileinfo 8920 0.002 0.031 2023-09-13T03:25:56.176 INFO:tasks.workunit.client.0.smithi064.stdout: Qfsinfo 9368 0.262 10.988 2023-09-13T03:25:56.176 INFO:tasks.workunit.client.0.smithi064.stdout: Sfileinfo 4593 0.063 8.942 2023-09-13T03:25:56.176 INFO:tasks.workunit.client.0.smithi064.stdout: Find 19753 0.278 32.907 2023-09-13T03:25:56.177 INFO:tasks.workunit.client.0.smithi064.stdout: WriteX 27863 0.114 36.304 2023-09-13T03:25:56.177 INFO:tasks.workunit.client.0.smithi064.stdout: ReadX 88508 0.020 2.327 2023-09-13T03:25:56.177 INFO:tasks.workunit.client.0.smithi064.stdout: LockX 184 0.770 5.898 2023-09-13T03:25:56.177 INFO:tasks.workunit.client.0.smithi064.stdout: UnlockX 184 0.716 2.233 2023-09-13T03:25:56.177 INFO:tasks.workunit.client.0.smithi064.stdout: Flush 3944 3.004 72.169 2023-09-13T03:25:56.178 INFO:tasks.workunit.client.0.smithi064.stdout: 2023-09-13T03:25:56.178 INFO:tasks.workunit.client.0.smithi064.stdout:Throughput 2.91611 MB/sec 1 clients 1 procs max_latency=16285.619 ms 2023-09-13T03:25:56.178 INFO:tasks.workunit.client.0.smithi064.stderr:+ dbench 10 2023-09-13T03:25:56.178 INFO:tasks.workunit.client.0.smithi064.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2023-09-13T03:25:56.179 INFO:tasks.workunit.client.0.smithi064.stdout: 2023-09-13T03:25:56.179 INFO:tasks.workunit.client.0.smithi064.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2023-09-13T03:25:56.179 INFO:tasks.workunit.client.0.smithi064.stdout:9 of 10 processes prepared for launch 0 sec 2023-09-13T03:25:56.279 INFO:tasks.workunit.client.0.smithi064.stdout:10 of 10 processes prepared for launch 0 sec 2023-09-13T03:25:56.279 INFO:tasks.workunit.client.0.smithi064.stdout:releasing clients 2023-09-13T03:25:57.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:56 smithi170 ceph-mon[134328]: pgmap v398: 97 pgs: 97 active+clean; 163 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.1 MiB/s wr, 287 op/s 2023-09-13T03:25:57.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:56 smithi064 ceph-mon[97675]: pgmap v398: 97 pgs: 97 active+clean; 163 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.1 MiB/s wr, 287 op/s 2023-09-13T03:25:57.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:56 smithi142 ceph-mon[134082]: pgmap v398: 97 pgs: 97 active+clean; 163 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.1 MiB/s wr, 287 op/s 2023-09-13T03:25:57.279 INFO:tasks.workunit.client.0.smithi064.stdout: 10 369 132.26 MB/sec warmup 1 sec latency 516.523 ms 2023-09-13T03:25:58.280 INFO:tasks.workunit.client.0.smithi064.stdout: 10 729 99.72 MB/sec warmup 2 sec latency 1516.632 ms 2023-09-13T03:25:59.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:58 smithi170 ceph-mon[134328]: pgmap v399: 97 pgs: 97 active+clean; 163 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 191 op/s 2023-09-13T03:25:59.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:58 smithi064 ceph-mon[97675]: pgmap v399: 97 pgs: 97 active+clean; 163 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 191 op/s 2023-09-13T03:25:59.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:58 smithi142 ceph-mon[134082]: pgmap v399: 97 pgs: 97 active+clean; 163 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 191 op/s 2023-09-13T03:25:59.279 INFO:tasks.workunit.client.0.smithi064.stdout: 10 1134 72.53 MB/sec warmup 3 sec latency 1803.578 ms 2023-09-13T03:26:00.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:25:59 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:26:00.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:25:59 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:26:00.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:25:59 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:26:00.279 INFO:tasks.workunit.client.0.smithi064.stdout: 10 2138 65.84 MB/sec warmup 4 sec latency 145.361 ms 2023-09-13T03:26:01.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:26:00 smithi170 ceph-mon[134328]: pgmap v400: 97 pgs: 97 active+clean; 163 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 191 op/s 2023-09-13T03:26:01.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:26:00 smithi064 ceph-mon[97675]: pgmap v400: 97 pgs: 97 active+clean; 163 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 191 op/s 2023-09-13T03:26:01.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:26:00 smithi142 ceph-mon[134082]: pgmap v400: 97 pgs: 97 active+clean; 163 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 191 op/s 2023-09-13T03:26:01.280 INFO:tasks.workunit.client.0.smithi064.stdout: 10 2264 54.26 MB/sec warmup 5 sec latency 550.309 ms 2023-09-13T03:26:02.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:26:01 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:26:02.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:26:01 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:26:02.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:26:01 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:26:02.280 INFO:tasks.workunit.client.0.smithi064.stdout: 10 2264 45.22 MB/sec warmup 6 sec latency 1550.464 ms 2023-09-13T03:26:03.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:26:02 smithi170 ceph-mon[134328]: pgmap v401: 97 pgs: 97 active+clean; 174 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.8 MiB/s wr, 427 op/s 2023-09-13T03:26:03.181 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:26:02 smithi064 ceph-mon[97675]: pgmap v401: 97 pgs: 97 active+clean; 174 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.8 MiB/s wr, 427 op/s 2023-09-13T03:26:03.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:26:02 smithi142 ceph-mon[134082]: pgmap v401: 97 pgs: 97 active+clean; 174 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.8 MiB/s wr, 427 op/s 2023-09-13T03:26:03.280 INFO:tasks.workunit.client.0.smithi064.stdout: 10 2264 38.76 MB/sec warmup 7 sec latency 2550.540 ms 2023-09-13T03:26:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.135+0000 7f51cadff700 1 -- 172.21.15.64:0/2742209132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51c41005a0 msgr2=0x7f51c4100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:26:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.135+0000 7f51cadff700 1 --2- 172.21.15.64:0/2742209132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51c41005a0 0x7f51c4100a20 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f51b0009920 tx=0x7f51b0009c30 comp rx=0 tx=0).stop 2023-09-13T03:26:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.136+0000 7f51cadff700 1 -- 172.21.15.64:0/2742209132 shutdown_connections 2023-09-13T03:26:04.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.136+0000 7f51cadff700 1 --2- 172.21.15.64:0/2742209132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51c4106660 0x7f51c4106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:04.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.136+0000 7f51cadff700 1 --2- 172.21.15.64:0/2742209132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51c4100f60 0x7f51c410d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:04.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.136+0000 7f51cadff700 1 --2- 172.21.15.64:0/2742209132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51c41005a0 0x7f51c4100a20 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:04.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.136+0000 7f51cadff700 1 -- 172.21.15.64:0/2742209132 >> 172.21.15.64:0/2742209132 conn(0x7f51c40fc0b0 msgr2=0x7f51c40fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:26:04.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.136+0000 7f51cadff700 1 -- 172.21.15.64:0/2742209132 shutdown_connections 2023-09-13T03:26:04.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.136+0000 7f51cadff700 1 -- 172.21.15.64:0/2742209132 wait complete. 2023-09-13T03:26:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.137+0000 7f51cadff700 1 Processor -- start 2023-09-13T03:26:04.141 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.137+0000 7f51cadff700 1 -- start start 2023-09-13T03:26:04.141 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.138+0000 7f51cadff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51c41005a0 0x7f51c4196790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:04.141 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.138+0000 7f51cadff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51c4100f60 0x7f51c4196cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:04.141 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.138+0000 7f51cadff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51c4106660 0x7f51c419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:04.141 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.138+0000 7f51cadff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c419b940 con 0x7f51c4106660 2023-09-13T03:26:04.142 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.138+0000 7f51cadff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c419bab0 con 0x7f51c41005a0 2023-09-13T03:26:04.142 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.138+0000 7f51cadff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c419bc70 con 0x7f51c4100f60 2023-09-13T03:26:04.142 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.138+0000 7f51c8b9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51c41005a0 0x7f51c4196790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:04.142 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.138+0000 7f51c8b9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51c41005a0 0x7f51c4196790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:45142/0 (socket says 172.21.15.64:45142) 2023-09-13T03:26:04.142 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.138+0000 7f51c8b9b700 1 -- 172.21.15.64:0/809478011 learned_addr learned my addr 172.21.15.64:0/809478011 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:26:04.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.138+0000 7f51c939c700 1 --2- 172.21.15.64:0/809478011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51c4106660 0x7f51c419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:04.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.138+0000 7f51c3fff700 1 --2- 172.21.15.64:0/809478011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51c4100f60 0x7f51c4196cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:04.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.138+0000 7f51c8b9b700 1 -- 172.21.15.64:0/809478011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51c4100f60 msgr2=0x7f51c4196cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:26:04.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.138+0000 7f51c8b9b700 1 --2- 172.21.15.64:0/809478011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51c4100f60 0x7f51c4196cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:04.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.138+0000 7f51c8b9b700 1 -- 172.21.15.64:0/809478011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51c4106660 msgr2=0x7f51c419b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:26:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.138+0000 7f51c8b9b700 1 --2- 172.21.15.64:0/809478011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51c4106660 0x7f51c419b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.138+0000 7f51c8b9b700 1 -- 172.21.15.64:0/809478011 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b80095f0 con 0x7f51c41005a0 2023-09-13T03:26:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.139+0000 7f51c8b9b700 1 --2- 172.21.15.64:0/809478011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51c41005a0 0x7f51c4196790 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f51b800ec10 tx=0x7f51b800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:26:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.139+0000 7f51c1ffb700 1 -- 172.21.15.64:0/809478011 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b800cc90 con 0x7f51c41005a0 2023-09-13T03:26:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.139+0000 7f51c1ffb700 1 -- 172.21.15.64:0/809478011 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51b80044e0 con 0x7f51c41005a0 2023-09-13T03:26:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.139+0000 7f51cadff700 1 -- 172.21.15.64:0/809478011 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51b0009520 con 0x7f51c41005a0 2023-09-13T03:26:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.139+0000 7f51c1ffb700 1 -- 172.21.15.64:0/809478011 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b8010430 con 0x7f51c41005a0 2023-09-13T03:26:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.139+0000 7f51cadff700 1 -- 172.21.15.64:0/809478011 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c41a3620 con 0x7f51c41005a0 2023-09-13T03:26:04.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.142+0000 7f51c1ffb700 1 -- 172.21.15.64:0/809478011 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f51b8003fb0 con 0x7f51c41005a0 2023-09-13T03:26:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.144+0000 7f51de6bcb80 1 Processor -- start 2023-09-13T03:26:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.144+0000 7f51de6bcb80 1 -- start start 2023-09-13T03:26:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.145+0000 7f51de6bcb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7062bf5b0 0x55e7062bf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.145+0000 7f51de6bcb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7062b9710 0x55e7062b9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.145+0000 7f51de6bcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7062ba0d0 0x55e7062c6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.145+0000 7f51de6bcb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e70620e6d0 con 0x55e7062bf5b0 2023-09-13T03:26:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.145+0000 7f51de6bcb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e70620bff0 con 0x55e7062b9710 2023-09-13T03:26:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.145+0000 7f51de6bcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7062078e0 con 0x55e7062ba0d0 2023-09-13T03:26:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.145+0000 7f51c1ffb700 1 --2- 172.21.15.64:0/809478011 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f51b40a07b0 0x7f51b40a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:04.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.145+0000 7f51c8b9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7062b9710 0x55e7062b9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.145+0000 7f51c939c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7062bf5b0 0x55e7062bf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.145+0000 7f51c3fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7062ba0d0 0x55e7062c6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.145+0000 7f51c1ffb700 1 -- 172.21.15.64:0/809478011 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f51b8014070 con 0x7f51c41005a0 2023-09-13T03:26:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.145+0000 7f51c3fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7062ba0d0 0x55e7062c6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:50824/0 (socket says 172.21.15.64:50824) 2023-09-13T03:26:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.145+0000 7f51c8b9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7062b9710 0x55e7062b9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:45156/0 (socket says 172.21.15.64:45156) 2023-09-13T03:26:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.145+0000 7f51c3fff700 1 -- 172.21.15.64:0/2045757275 learned_addr learned my addr 172.21.15.64:0/2045757275 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:26:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.145+0000 7f51c939c700 1 --2- 172.21.15.64:0/809478011 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f51b40a07b0 0x7f51b40a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.145+0000 7f51c8b9b700 1 -- 172.21.15.64:0/2045757275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7062ba0d0 msgr2=0x55e7062c6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:26:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.145+0000 7f51c8b9b700 1 --2- 172.21.15.64:0/2045757275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7062ba0d0 0x55e7062c6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.145+0000 7f51c8b9b700 1 -- 172.21.15.64:0/2045757275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7062bf5b0 msgr2=0x55e7062bf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:26:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.145+0000 7f51c8b9b700 1 --2- 172.21.15.64:0/2045757275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7062bf5b0 0x55e7062bf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.146+0000 7f51c8b9b700 1 -- 172.21.15.64:0/2045757275 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e706033d80 con 0x55e7062b9710 2023-09-13T03:26:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.146+0000 7f51c3fff700 1 --2- 172.21.15.64:0/2045757275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7062ba0d0 0x55e7062c6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:26:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.146+0000 7f51c8b9b700 1 --2- 172.21.15.64:0/2045757275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7062b9710 0x55e7062b9b90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f51b80c9bf0 tx=0x7f51b8095dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40fa7f58779420af server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:26:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.146+0000 7f51c939c700 1 --2- 172.21.15.64:0/809478011 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f51b40a07b0 0x7f51b40a2c70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f51bc009ec0 tx=0x7f51bc00b040 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:26:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.146+0000 7f51aeffd700 1 -- 172.21.15.64:0/2045757275 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b8016330 con 0x55e7062b9710 2023-09-13T03:26:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.146+0000 7f51aeffd700 1 -- 172.21.15.64:0/2045757275 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51b809a050 con 0x55e7062b9710 2023-09-13T03:26:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.146+0000 7f51aeffd700 1 -- 172.21.15.64:0/2045757275 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b8096060 con 0x55e7062b9710 2023-09-13T03:26:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.147+0000 7f51de6bcb80 1 -- 172.21.15.64:0/2045757275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7062b9710 msgr2=0x55e7062b9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:26:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.147+0000 7f51de6bcb80 1 --2- 172.21.15.64:0/2045757275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7062b9710 0x55e7062b9b90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f51b80c9bf0 tx=0x7f51b8095dc0 comp rx=0 tx=0).stop 2023-09-13T03:26:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.147+0000 7f51de6bcb80 1 -- 172.21.15.64:0/2045757275 shutdown_connections 2023-09-13T03:26:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.147+0000 7f51de6bcb80 1 --2- 172.21.15.64:0/2045757275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7062ba0d0 0x55e7062c6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.147+0000 7f51de6bcb80 1 --2- 172.21.15.64:0/2045757275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7062bf5b0 0x55e7062bf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.147+0000 7f51de6bcb80 1 --2- 172.21.15.64:0/2045757275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7062b9710 0x55e7062b9b90 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.147+0000 7f51de6bcb80 1 -- 172.21.15.64:0/2045757275 >> 172.21.15.64:0/2045757275 conn(0x55e70622a030 msgr2=0x55e7062b0560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:26:04.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.147+0000 7f51de6bcb80 1 -- 172.21.15.64:0/2045757275 shutdown_connections 2023-09-13T03:26:04.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.147+0000 7f51de6bcb80 1 -- 172.21.15.64:0/2045757275 wait complete. 2023-09-13T03:26:04.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.147+0000 7f51de6bcb80 1 -- start start 2023-09-13T03:26:04.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.148+0000 7f51de6bcb80 1 Processor -- start 2023-09-13T03:26:04.158 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.148+0000 7f51de6bcb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7062b9710 0x55e7062c6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.148+0000 7f51de6bcb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7062ba0d0 0x55e7062c3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.148+0000 7f51de6bcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7062bf5b0 0x55e7062c3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.148+0000 7f51de6bcb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7061a5b10 con 0x55e7062ba0d0 2023-09-13T03:26:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.148+0000 7f51de6bcb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7061a5810 con 0x55e7062b9710 2023-09-13T03:26:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.148+0000 7f51de6bcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7060898c0 con 0x55e7062bf5b0 2023-09-13T03:26:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.148+0000 7f51de6bcb80 10 client.0 _ll_register_callbacks cb 0x55e7060efa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:26:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.148+0000 7f51c8b9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7062b9710 0x55e7062c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.149+0000 7f51c8b9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7062b9710 0x55e7062c6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:45164/0 (socket says 172.21.15.64:45164) 2023-09-13T03:26:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.149+0000 7f51c8b9b700 1 -- 172.21.15.64:0/806543891 learned_addr learned my addr 172.21.15.64:0/806543891 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:26:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.149+0000 7f51c3fff700 1 --2- 172.21.15.64:0/806543891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7062ba0d0 0x55e7062c3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.149+0000 7f51c939c700 1 --2- 172.21.15.64:0/806543891 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7062bf5b0 0x55e7062c3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.149+0000 7f51c8b9b700 1 -- 172.21.15.64:0/806543891 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7062bf5b0 msgr2=0x55e7062c3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:26:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.149+0000 7f51c8b9b700 1 --2- 172.21.15.64:0/806543891 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7062bf5b0 0x55e7062c3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.149+0000 7f51c8b9b700 1 -- 172.21.15.64:0/806543891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7062ba0d0 msgr2=0x55e7062c3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:26:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.149+0000 7f51c8b9b700 1 --2- 172.21.15.64:0/806543891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7062ba0d0 0x55e7062c3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.149+0000 7f51c8b9b700 1 -- 172.21.15.64:0/806543891 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c410dd20 con 0x55e7062b9710 2023-09-13T03:26:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.149+0000 7f51c939c700 1 --2- 172.21.15.64:0/806543891 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7062bf5b0 0x55e7062c3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:26:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.149+0000 7f51c8b9b700 1 --2- 172.21.15.64:0/806543891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7062b9710 0x55e7062c6220 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f51b800c5b0 tx=0x7f51b8003910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9aa318122878d703 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:26:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.149+0000 7f51a3fff700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:26:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.149+0000 7f51a3fff700 1 -- 172.21.15.64:0/806543891 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b8099950 con 0x55e7062b9710 2023-09-13T03:26:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.149+0000 7f51de6bcb80 1 -- 172.21.15.64:0/806543891 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e70615f490 con 0x55e7062b9710 2023-09-13T03:26:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.149+0000 7f51a3fff700 1 -- 172.21.15.64:0/806543891 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f51b809d780 con 0x55e7062b9710 2023-09-13T03:26:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.150+0000 7f51a3fff700 1 -- 172.21.15.64:0/806543891 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51b80a19c0 con 0x55e7062b9710 2023-09-13T03:26:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.150+0000 7f51de6bcb80 10 client.27164 fetch_fsmap learned FSMap version 68 2023-09-13T03:26:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.150+0000 7f51de6bcb80 1 -- 172.21.15.64:0/806543891 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x7f51c41a3620 con 0x55e7062b9710 2023-09-13T03:26:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.150+0000 7f51a3fff700 1 -- 172.21.15.64:0/806543891 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51b807c020 con 0x55e7062b9710 2023-09-13T03:26:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.150+0000 7f51a3fff700 1 -- 172.21.15.64:0/806543891 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 68) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f51b80a1530 con 0x55e7062b9710 2023-09-13T03:26:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.150+0000 7f51de6bcb80 10 client.27164 fetch_fsmap finished waiting for FSMap version 68 2023-09-13T03:26:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.150+0000 7f51de6bcb80 10 client.27164 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:26:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.150+0000 7f51de6bcb80 20 client.27164 populate_metadata read hostname 'smithi064' 2023-09-13T03:26:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.150+0000 7f51de6bcb80 1 --2- 172.21.15.64:0/806543891 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55e7062d33b0 0x55e7062d57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.150+0000 7f51de6bcb80 4 client.27164 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:26:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.150+0000 7f51de6bcb80 1 -- 172.21.15.64:0/806543891 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e70621c840 con 0x55e7062d33b0 2023-09-13T03:26:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.150+0000 7f51c3fff700 1 --2- 172.21.15.64:0/806543891 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55e7062d33b0 0x55e7062d57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:04.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.151+0000 7f51c3fff700 1 --2- 172.21.15.64:0/806543891 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55e7062d33b0 0x55e7062d57a0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=830f6cd447284342 server_cookie=a69854d31b733611 in_seq=0 out_seq=0 2023-09-13T03:26:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.151+0000 7f51a3fff700 10 client.27164 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:26:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.152+0000 7f51a3fff700 1 -- 172.21.15.64:0/806543891 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e70621c840 con 0x55e7062d33b0 2023-09-13T03:26:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.152+0000 7f51a3fff700 10 client.27164 handle_command_reply: tid=0 2023-09-13T03:26:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.152+0000 7f51de6bcb80 1 client.27164 shutdown 2023-09-13T03:26:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.152+0000 7f51de6bcb80 1 -- 172.21.15.64:0/806543891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7062b9710 msgr2=0x55e7062c6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:26:04.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.152+0000 7f51de6bcb80 1 --2- 172.21.15.64:0/806543891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7062b9710 0x55e7062c6220 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f51b800c5b0 tx=0x7f51b8003910 comp rx=0 tx=0).stop 2023-09-13T03:26:04.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.153+0000 7f51de6bcb80 1 -- 172.21.15.64:0/806543891 shutdown_connections 2023-09-13T03:26:04.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.153+0000 7f51de6bcb80 1 --2- 172.21.15.64:0/806543891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7062b9710 0x55e7062c6220 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:04.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.153+0000 7f51de6bcb80 1 --2- 172.21.15.64:0/806543891 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55e7062d33b0 0x55e7062d57a0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:04.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.153+0000 7f51de6bcb80 1 --2- 172.21.15.64:0/806543891 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7062bf5b0 0x55e7062c3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.153+0000 7f51de6bcb80 1 --2- 172.21.15.64:0/806543891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7062ba0d0 0x55e7062c3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.153+0000 7f51de6bcb80 1 -- 172.21.15.64:0/806543891 >> 172.21.15.64:0/806543891 conn(0x55e70622a030 msgr2=0x55e7062af7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:26:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.153+0000 7f51a3fff700 0 client.27164 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:26:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.153+0000 7f51de6bcb80 1 -- 172.21.15.64:0/806543891 shutdown_connections 2023-09-13T03:26:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.153+0000 7f51de6bcb80 1 -- 172.21.15.64:0/806543891 wait complete. 2023-09-13T03:26:04.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.153+0000 7f51de6bcb80 20 client.27164 trim_cache size 0 max 16384 2023-09-13T03:26:04.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.170+0000 7f51de6bcb80 1 Processor -- start 2023-09-13T03:26:04.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.170+0000 7f51de6bcb80 1 -- start start 2023-09-13T03:26:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.171+0000 7f51de6bcb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7062b9710 0x55e7062b1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.171+0000 7f51de6bcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7062ba0d0 0x55e7062c2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.171+0000 7f51de6bcb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7062bf5b0 0x55e7062c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.171+0000 7f51de6bcb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7062c4b90 con 0x55e7062bf5b0 2023-09-13T03:26:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.171+0000 7f51de6bcb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7062bd1f0 con 0x55e7062b9710 2023-09-13T03:26:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.171+0000 7f51de6bcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7062c1ad0 con 0x55e7062ba0d0 2023-09-13T03:26:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.171+0000 7f51c3fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7062ba0d0 0x55e7062c2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:04.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.171+0000 7f51c939c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7062bf5b0 0x55e7062c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:04.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.171+0000 7f51c3fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7062ba0d0 0x55e7062c2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:50844/0 (socket says 172.21.15.64:50844) 2023-09-13T03:26:04.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.171+0000 7f51c3fff700 1 -- 172.21.15.64:0/329872193 learned_addr learned my addr 172.21.15.64:0/329872193 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:26:04.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.171+0000 7f51c8b9b700 1 --2- 172.21.15.64:0/329872193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7062b9710 0x55e7062b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:04.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.171+0000 7f51c3fff700 1 -- 172.21.15.64:0/329872193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7062b9710 msgr2=0x55e7062b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:26:04.177 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.171+0000 7f51c3fff700 1 --2- 172.21.15.64:0/329872193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7062b9710 0x55e7062b1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.171+0000 7f51c3fff700 1 -- 172.21.15.64:0/329872193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7062bf5b0 msgr2=0x55e7062c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:26:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.171+0000 7f51c3fff700 1 --2- 172.21.15.64:0/329872193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7062bf5b0 0x55e7062c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.171+0000 7f51c3fff700 1 -- 172.21.15.64:0/329872193 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e70601cc00 con 0x55e7062ba0d0 2023-09-13T03:26:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.172+0000 7f51c3fff700 1 --2- 172.21.15.64:0/329872193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7062ba0d0 0x55e7062c2d00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f51b0000c00 tx=0x7f51b000f9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c27befe12396d205 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:26:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.172+0000 7f51af7fe700 1 -- 172.21.15.64:0/329872193 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b0028cf0 con 0x55e7062ba0d0 2023-09-13T03:26:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.172+0000 7f51af7fe700 1 -- 172.21.15.64:0/329872193 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51b001d040 con 0x55e7062ba0d0 2023-09-13T03:26:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.172+0000 7f51af7fe700 1 -- 172.21.15.64:0/329872193 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b0005040 con 0x55e7062ba0d0 2023-09-13T03:26:04.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.173+0000 7f51de6bcb80 1 -- 172.21.15.64:0/329872193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7062ba0d0 msgr2=0x55e7062c2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:26:04.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.173+0000 7f51de6bcb80 1 --2- 172.21.15.64:0/329872193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7062ba0d0 0x55e7062c2d00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f51b0000c00 tx=0x7f51b000f9c0 comp rx=0 tx=0).stop 2023-09-13T03:26:04.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.173+0000 7f51de6bcb80 1 -- 172.21.15.64:0/329872193 shutdown_connections 2023-09-13T03:26:04.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.173+0000 7f51de6bcb80 1 --2- 172.21.15.64:0/329872193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7062b9710 0x55e7062b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:04.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.173+0000 7f51de6bcb80 1 --2- 172.21.15.64:0/329872193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7062bf5b0 0x55e7062c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.173+0000 7f51de6bcb80 1 --2- 172.21.15.64:0/329872193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7062ba0d0 0x55e7062c2d00 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.173+0000 7f51de6bcb80 1 -- 172.21.15.64:0/329872193 >> 172.21.15.64:0/329872193 conn(0x55e70622a030 msgr2=0x55e7062cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:26:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.173+0000 7f51de6bcb80 1 -- 172.21.15.64:0/329872193 shutdown_connections 2023-09-13T03:26:04.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.173+0000 7f51de6bcb80 1 -- 172.21.15.64:0/329872193 wait complete. 2023-09-13T03:26:04.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.173+0000 7f51de6bcb80 1 -- start start 2023-09-13T03:26:04.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.173+0000 7f51de6bcb80 1 Processor -- start 2023-09-13T03:26:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.174+0000 7f51de6bcb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7062b9710 0x55e7062c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.174+0000 7f51de6bcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7062ba0d0 0x55e7062c9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.174+0000 7f51de6bcb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7062bf5b0 0x55e706320310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.174+0000 7f51de6bcb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e706320850 con 0x55e7062bf5b0 2023-09-13T03:26:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.174+0000 7f51de6bcb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e706320990 con 0x55e7062b9710 2023-09-13T03:26:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.174+0000 7f51de6bcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e706320b50 con 0x55e7062ba0d0 2023-09-13T03:26:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.174+0000 7f51de6bcb80 10 client.0 _ll_register_callbacks cb 0x7f51bc009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:26:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.174+0000 7f51c3fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7062ba0d0 0x55e7062c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.174+0000 7f51c939c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7062bf5b0 0x55e706320310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.174+0000 7f51c3fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7062ba0d0 0x55e7062c9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:50856/0 (socket says 172.21.15.64:50856) 2023-09-13T03:26:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.174+0000 7f51c3fff700 1 -- 172.21.15.64:0/207173499 learned_addr learned my addr 172.21.15.64:0/207173499 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:26:04.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.174+0000 7f51c8b9b700 1 --2- 172.21.15.64:0/207173499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7062b9710 0x55e7062c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:04.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.174+0000 7f51c3fff700 1 -- 172.21.15.64:0/207173499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7062b9710 msgr2=0x55e7062c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:26:04.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.174+0000 7f51c3fff700 1 --2- 172.21.15.64:0/207173499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7062b9710 0x55e7062c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:04.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.174+0000 7f51c3fff700 1 -- 172.21.15.64:0/207173499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7062bf5b0 msgr2=0x55e706320310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:26:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.174+0000 7f51c3fff700 1 --2- 172.21.15.64:0/207173499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7062bf5b0 0x55e706320310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.174+0000 7f51c3fff700 1 -- 172.21.15.64:0/207173499 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b000f780 con 0x55e7062ba0d0 2023-09-13T03:26:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.175+0000 7f51c3fff700 1 --2- 172.21.15.64:0/207173499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7062ba0d0 0x55e7062c9050 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f51b00059d0 tx=0x7f51b000b360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d16ce63b43544cf server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:26:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.175+0000 7f51a3fff700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:26:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.175+0000 7f51a3fff700 1 -- 172.21.15.64:0/207173499 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b0028160 con 0x55e7062ba0d0 2023-09-13T03:26:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.175+0000 7f51de6bcb80 1 -- 172.21.15.64:0/207173499 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51b809a050 con 0x55e7062ba0d0 2023-09-13T03:26:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.175+0000 7f51a3fff700 1 -- 172.21.15.64:0/207173499 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f51b0000e40 con 0x55e7062ba0d0 2023-09-13T03:26:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.175+0000 7f51a3fff700 1 -- 172.21.15.64:0/207173499 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51b00282c0 con 0x55e7062ba0d0 2023-09-13T03:26:04.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.175+0000 7f51de6bcb80 10 client.27154 fetch_fsmap learned FSMap version 68 2023-09-13T03:26:04.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.175+0000 7f51de6bcb80 1 -- 172.21.15.64:0/207173499 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x55e70605a1f0 con 0x55e7062ba0d0 2023-09-13T03:26:04.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.175+0000 7f51a3fff700 1 -- 172.21.15.64:0/207173499 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51b0040020 con 0x55e7062ba0d0 2023-09-13T03:26:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.176+0000 7f51a3fff700 1 -- 172.21.15.64:0/207173499 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 68) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f51b0009950 con 0x55e7062ba0d0 2023-09-13T03:26:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.176+0000 7f51de6bcb80 10 client.27154 fetch_fsmap finished waiting for FSMap version 68 2023-09-13T03:26:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.176+0000 7f51de6bcb80 10 client.27154 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:26:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.176+0000 7f51de6bcb80 20 client.27154 populate_metadata read hostname 'smithi064' 2023-09-13T03:26:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.176+0000 7f51de6bcb80 1 --2- 172.21.15.64:0/207173499 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55e706327e90 0x55e70632a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.176+0000 7f51de6bcb80 4 client.27154 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:26:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.176+0000 7f51de6bcb80 1 -- 172.21.15.64:0/207173499 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e7062008c0 con 0x55e706327e90 2023-09-13T03:26:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.176+0000 7f51c8b9b700 1 --2- 172.21.15.64:0/207173499 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55e706327e90 0x55e70632a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.176+0000 7f51c8b9b700 1 --2- 172.21.15.64:0/207173499 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55e706327e90 0x55e70632a2b0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1408b6a93ffddfb3 server_cookie=13b2aca11ffc9f65 in_seq=0 out_seq=0 2023-09-13T03:26:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.176+0000 7f51a3fff700 10 client.27154 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:26:04.280 INFO:tasks.workunit.client.0.smithi064.stdout: 10 2300 34.21 MB/sec warmup 8 sec latency 3373.071 ms 2023-09-13T03:26:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.581+0000 7f51a3fff700 1 -- 172.21.15.64:0/207173499 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e7062008c0 con 0x55e706327e90 2023-09-13T03:26:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.582+0000 7f51a3fff700 10 client.27154 handle_command_reply: tid=0 2023-09-13T03:26:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.582+0000 7f51de6bcb80 1 client.27154 shutdown 2023-09-13T03:26:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.582+0000 7f51de6bcb80 1 -- 172.21.15.64:0/207173499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7062ba0d0 msgr2=0x55e7062c9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:26:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.582+0000 7f51de6bcb80 1 --2- 172.21.15.64:0/207173499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7062ba0d0 0x55e7062c9050 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f51b00059d0 tx=0x7f51b000b360 comp rx=0 tx=0).stop 2023-09-13T03:26:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.582+0000 7f51de6bcb80 1 -- 172.21.15.64:0/207173499 shutdown_connections 2023-09-13T03:26:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.582+0000 7f51de6bcb80 1 --2- 172.21.15.64:0/207173499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7062b9710 0x55e7062c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.582+0000 7f51de6bcb80 1 --2- 172.21.15.64:0/207173499 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55e706327e90 0x55e70632a2b0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.582+0000 7f51de6bcb80 1 --2- 172.21.15.64:0/207173499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e7062bf5b0 0x55e706320310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.582+0000 7f51de6bcb80 1 --2- 172.21.15.64:0/207173499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7062ba0d0 0x55e7062c9050 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.582+0000 7f51a3fff700 0 client.27154 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:26:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.582+0000 7f51de6bcb80 1 -- 172.21.15.64:0/207173499 >> 172.21.15.64:0/207173499 conn(0x55e70622a030 msgr2=0x55e7062bd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:26:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.582+0000 7f51de6bcb80 1 -- 172.21.15.64:0/207173499 shutdown_connections 2023-09-13T03:26:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.582+0000 7f51de6bcb80 1 -- 172.21.15.64:0/207173499 wait complete. 2023-09-13T03:26:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.583+0000 7f51de6bcb80 20 client.27154 trim_cache size 0 max 16384 2023-09-13T03:26:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:26:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:26:04.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:26:04.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:26:04.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.583+0000 7f51cadff700 1 -- 172.21.15.64:0/809478011 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f51b40a07b0 msgr2=0x7f51b40a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:26:04.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.583+0000 7f51cadff700 1 --2- 172.21.15.64:0/809478011 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f51b40a07b0 0x7f51b40a2c70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f51bc009ec0 tx=0x7f51bc00b040 comp rx=0 tx=0).stop 2023-09-13T03:26:04.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.583+0000 7f51cadff700 1 -- 172.21.15.64:0/809478011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51c41005a0 msgr2=0x7f51c4196790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:26:04.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.583+0000 7f51cadff700 1 --2- 172.21.15.64:0/809478011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51c41005a0 0x7f51c4196790 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f51b800ec10 tx=0x7f51b800ef20 comp rx=0 tx=0).stop 2023-09-13T03:26:04.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.584+0000 7f51cadff700 1 -- 172.21.15.64:0/809478011 shutdown_connections 2023-09-13T03:26:04.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.584+0000 7f51cadff700 1 --2- 172.21.15.64:0/809478011 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f51b40a07b0 0x7f51b40a2c70 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:04.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.584+0000 7f51cadff700 1 --2- 172.21.15.64:0/809478011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51c41005a0 0x7f51c4196790 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:04.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.584+0000 7f51cadff700 1 --2- 172.21.15.64:0/809478011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f51c4106660 0x7f51c419b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:04.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.584+0000 7f51cadff700 1 --2- 172.21.15.64:0/809478011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51c4100f60 0x7f51c4196cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:04.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.584+0000 7f51cadff700 1 -- 172.21.15.64:0/809478011 >> 172.21.15.64:0/809478011 conn(0x7f51c40fc0b0 msgr2=0x7f51c40fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:26:04.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.584+0000 7f51cadff700 1 -- 172.21.15.64:0/809478011 shutdown_connections 2023-09-13T03:26:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:04.584+0000 7f51cadff700 1 -- 172.21.15.64:0/809478011 wait complete. 2023-09-13T03:26:05.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:26:04 smithi170 ceph-mon[134328]: pgmap v402: 97 pgs: 97 active+clean; 174 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.0 MiB/s wr, 348 op/s 2023-09-13T03:26:05.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:26:04 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:26:05.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:26:04 smithi064 ceph-mon[97675]: pgmap v402: 97 pgs: 97 active+clean; 174 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.0 MiB/s wr, 348 op/s 2023-09-13T03:26:05.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:26:04 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:26:05.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:26:04 smithi142 ceph-mon[134082]: pgmap v402: 97 pgs: 97 active+clean; 174 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.0 MiB/s wr, 348 op/s 2023-09-13T03:26:05.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:26:04 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:26:05.280 INFO:tasks.workunit.client.0.smithi064.stdout: 10 2741 35.43 MB/sec warmup 9 sec latency 241.453 ms 2023-09-13T03:26:06.280 INFO:tasks.workunit.client.0.smithi064.stdout: 10 3364 36.92 MB/sec warmup 10 sec latency 129.495 ms 2023-09-13T03:26:07.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:26:06 smithi170 ceph-mon[134328]: pgmap v403: 97 pgs: 97 active+clean; 349 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 22 MiB/s wr, 650 op/s 2023-09-13T03:26:07.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:26:06 smithi064 ceph-mon[97675]: pgmap v403: 97 pgs: 97 active+clean; 349 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 22 MiB/s wr, 650 op/s 2023-09-13T03:26:07.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:26:06 smithi142 ceph-mon[134082]: pgmap v403: 97 pgs: 97 active+clean; 349 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 22 MiB/s wr, 650 op/s 2023-09-13T03:26:07.280 INFO:tasks.workunit.client.0.smithi064.stdout: 10 3615 34.92 MB/sec warmup 11 sec latency 72.516 ms 2023-09-13T03:26:08.280 INFO:tasks.workunit.client.0.smithi064.stdout: 10 3817 32.24 MB/sec warmup 12 sec latency 110.050 ms 2023-09-13T03:26:09.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:26:08 smithi170 ceph-mon[134328]: pgmap v404: 97 pgs: 97 active+clean; 349 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 21 MiB/s wr, 538 op/s 2023-09-13T03:26:09.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:26:08 smithi064 ceph-mon[97675]: pgmap v404: 97 pgs: 97 active+clean; 349 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 21 MiB/s wr, 538 op/s 2023-09-13T03:26:09.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:26:08 smithi142 ceph-mon[134082]: pgmap v404: 97 pgs: 97 active+clean; 349 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 21 MiB/s wr, 538 op/s 2023-09-13T03:26:09.280 INFO:tasks.workunit.client.0.smithi064.stdout: 10 4069 30.03 MB/sec warmup 13 sec latency 189.675 ms 2023-09-13T03:26:10.280 INFO:tasks.workunit.client.0.smithi064.stdout: 10 4484 29.00 MB/sec warmup 14 sec latency 130.584 ms 2023-09-13T03:26:11.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:26:10 smithi170 ceph-mon[134328]: pgmap v405: 97 pgs: 97 active+clean; 349 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 21 MiB/s wr, 538 op/s 2023-09-13T03:26:11.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:26:10 smithi170 ceph-mon[134328]: osdmap e99: 12 total, 12 up, 12 in 2023-09-13T03:26:11.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:26:10 smithi064 ceph-mon[97675]: pgmap v405: 97 pgs: 97 active+clean; 349 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 21 MiB/s wr, 538 op/s 2023-09-13T03:26:11.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:26:10 smithi064 ceph-mon[97675]: osdmap e99: 12 total, 12 up, 12 in 2023-09-13T03:26:11.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:26:10 smithi142 ceph-mon[134082]: pgmap v405: 97 pgs: 97 active+clean; 349 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 21 MiB/s wr, 538 op/s 2023-09-13T03:26:11.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:26:10 smithi142 ceph-mon[134082]: osdmap e99: 12 total, 12 up, 12 in 2023-09-13T03:26:11.280 INFO:tasks.workunit.client.0.smithi064.stdout: 10 5070 29.17 MB/sec warmup 15 sec latency 166.128 ms 2023-09-13T03:26:12.280 INFO:tasks.workunit.client.0.smithi064.stdout: 10 5770 29.02 MB/sec warmup 16 sec latency 57.493 ms 2023-09-13T03:26:13.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:26:12 smithi170 ceph-mon[134328]: pgmap v407: 97 pgs: 97 active+clean; 362 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 21 MiB/s wr, 504 op/s 2023-09-13T03:26:13.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:26:12 smithi064 ceph-mon[97675]: pgmap v407: 97 pgs: 97 active+clean; 362 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 21 MiB/s wr, 504 op/s 2023-09-13T03:26:13.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:26:12 smithi142 ceph-mon[134082]: pgmap v407: 97 pgs: 97 active+clean; 362 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 21 MiB/s wr, 504 op/s 2023-09-13T03:26:13.280 INFO:tasks.workunit.client.0.smithi064.stdout: 10 6437 30.52 MB/sec warmup 17 sec latency 62.514 ms 2023-09-13T03:26:14.280 INFO:tasks.workunit.client.0.smithi064.stdout: 10 6923 31.46 MB/sec warmup 18 sec latency 62.417 ms 2023-09-13T03:26:15.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:26:14 smithi170 ceph-mon[134328]: pgmap v408: 97 pgs: 97 active+clean; 362 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 21 MiB/s wr, 504 op/s 2023-09-13T03:26:15.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:26:14 smithi170 ceph-mon[134328]: osdmap e100: 12 total, 12 up, 12 in 2023-09-13T03:26:15.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:26:14 smithi064 ceph-mon[97675]: pgmap v408: 97 pgs: 97 active+clean; 362 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 21 MiB/s wr, 504 op/s 2023-09-13T03:26:15.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:26:14 smithi064 ceph-mon[97675]: osdmap e100: 12 total, 12 up, 12 in 2023-09-13T03:26:15.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:26:14 smithi142 ceph-mon[134082]: pgmap v408: 97 pgs: 97 active+clean; 362 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 21 MiB/s wr, 504 op/s 2023-09-13T03:26:15.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:26:14 smithi142 ceph-mon[134082]: osdmap e100: 12 total, 12 up, 12 in 2023-09-13T03:26:15.281 INFO:tasks.workunit.client.0.smithi064.stdout: 10 7093 30.09 MB/sec warmup 19 sec latency 426.509 ms 2023-09-13T03:26:16.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:26:15 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:26:16.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:26:15 smithi170 ceph-mon[134328]: pgmap v410: 97 pgs: 97 active+clean; 374 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 340 op/s 2023-09-13T03:26:16.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:26:15 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:26:16.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:26:15 smithi064 ceph-mon[97675]: pgmap v410: 97 pgs: 97 active+clean; 374 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 340 op/s 2023-09-13T03:26:16.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:26:15 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:26:16.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:26:15 smithi142 ceph-mon[134082]: pgmap v410: 97 pgs: 97 active+clean; 374 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 340 op/s 2023-09-13T03:26:16.281 INFO:tasks.workunit.client.0.smithi064.stdout: 10 7313 28.73 MB/sec warmup 20 sec latency 1426.568 ms 2023-09-13T03:26:17.281 INFO:tasks.workunit.client.0.smithi064.stdout: 10 7710 27.81 MB/sec warmup 21 sec latency 2426.615 ms 2023-09-13T03:26:17.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.754+0000 7efef359d700 1 -- 172.21.15.64:0/3788642605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efeec106480 msgr2=0x7efeec106860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:26:17.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.754+0000 7efef359d700 1 --2- 172.21.15.64:0/3788642605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efeec106480 0x7efeec106860 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7efee8009990 tx=0x7efee8009ca0 comp rx=0 tx=0).stop 2023-09-13T03:26:17.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.754+0000 7efef359d700 1 -- 172.21.15.64:0/3788642605 shutdown_connections 2023-09-13T03:26:17.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.754+0000 7efef359d700 1 --2- 172.21.15.64:0/3788642605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efeec0683b0 0x7efeec10d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:17.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.754+0000 7efef359d700 1 --2- 172.21.15.64:0/3788642605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efeec106480 0x7efeec106860 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:17.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.754+0000 7efef359d700 1 --2- 172.21.15.64:0/3788642605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efeec0679f0 0x7efeec067e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:17.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.754+0000 7efef359d700 1 -- 172.21.15.64:0/3788642605 >> 172.21.15.64:0/3788642605 conn(0x7efeec074dc0 msgr2=0x7efeec0751d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:26:17.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.754+0000 7efef359d700 1 -- 172.21.15.64:0/3788642605 shutdown_connections 2023-09-13T03:26:17.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.754+0000 7efef359d700 1 -- 172.21.15.64:0/3788642605 wait complete. 2023-09-13T03:26:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.755+0000 7efef359d700 1 Processor -- start 2023-09-13T03:26:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.755+0000 7efef359d700 1 -- start start 2023-09-13T03:26:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.756+0000 7efef359d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efeec0679f0 0x7efeec19e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.756+0000 7efef359d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efeec0683b0 0x7efeec19e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.756+0000 7efef359d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efeec106480 0x7efeec1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.756+0000 7efef359d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeec198c50 con 0x7efeec106480 2023-09-13T03:26:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.756+0000 7efef359d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeec198dc0 con 0x7efeec0679f0 2023-09-13T03:26:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.756+0000 7efef359d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeec198f80 con 0x7efeec0683b0 2023-09-13T03:26:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.756+0000 7efef259b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efeec0679f0 0x7efeec19e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.756+0000 7efef259b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efeec0679f0 0x7efeec19e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:55716/0 (socket says 172.21.15.64:55716) 2023-09-13T03:26:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.756+0000 7efef259b700 1 -- 172.21.15.64:0/1468059465 learned_addr learned my addr 172.21.15.64:0/1468059465 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:26:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.756+0000 7efef1d9a700 1 --2- 172.21.15.64:0/1468059465 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efeec0683b0 0x7efeec19e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.756+0000 7efef2d9c700 1 --2- 172.21.15.64:0/1468059465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efeec106480 0x7efeec1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.757+0000 7efef259b700 1 -- 172.21.15.64:0/1468059465 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efeec0683b0 msgr2=0x7efeec19e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:26:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.757+0000 7efef259b700 1 --2- 172.21.15.64:0/1468059465 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efeec0683b0 0x7efeec19e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.757+0000 7efef259b700 1 -- 172.21.15.64:0/1468059465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efeec106480 msgr2=0x7efeec1985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:26:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.757+0000 7efef259b700 1 --2- 172.21.15.64:0/1468059465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efeec106480 0x7efeec1985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.757+0000 7efef259b700 1 -- 172.21.15.64:0/1468059465 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee80095f0 con 0x7efeec0679f0 2023-09-13T03:26:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.757+0000 7efef2d9c700 1 --2- 172.21.15.64:0/1468059465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efeec106480 0x7efeec1985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:26:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.757+0000 7efef259b700 1 --2- 172.21.15.64:0/1468059465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efeec0679f0 0x7efeec19e3e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7efee8000c00 tx=0x7efee8005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:26:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.757+0000 7efee37fe700 1 -- 172.21.15.64:0/1468059465 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee801d070 con 0x7efeec0679f0 2023-09-13T03:26:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.757+0000 7efef359d700 1 -- 172.21.15.64:0/1468059465 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeec199200 con 0x7efeec0679f0 2023-09-13T03:26:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.757+0000 7efee37fe700 1 -- 172.21.15.64:0/1468059465 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efee8022470 con 0x7efeec0679f0 2023-09-13T03:26:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.757+0000 7efee37fe700 1 -- 172.21.15.64:0/1468059465 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee8017650 con 0x7efeec0679f0 2023-09-13T03:26:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.757+0000 7efef359d700 1 -- 172.21.15.64:0/1468059465 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeec1a5480 con 0x7efeec0679f0 2023-09-13T03:26:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.763+0000 7efee37fe700 1 -- 172.21.15.64:0/1468059465 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7efee80225e0 con 0x7efeec0679f0 2023-09-13T03:26:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.763+0000 7eff080c8b80 1 Processor -- start 2023-09-13T03:26:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.763+0000 7eff080c8b80 1 -- start start 2023-09-13T03:26:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.763+0000 7eff080c8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564af7577290 0x564af7577670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.763+0000 7eff080c8b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af7577bb0 0x564af7571460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.763+0000 7eff080c8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564af75719a0 0x564af7571de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.764+0000 7eff080c8b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564af74ca6d0 con 0x564af7577bb0 2023-09-13T03:26:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.764+0000 7eff080c8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564af74c7ff0 con 0x564af75719a0 2023-09-13T03:26:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.764+0000 7eff080c8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564af74c38e0 con 0x564af7577290 2023-09-13T03:26:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.764+0000 7efef2d9c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564af7577290 0x564af7577670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.764+0000 7efef2d9c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564af7577290 0x564af7577670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:56844/0 (socket says 172.21.15.64:56844) 2023-09-13T03:26:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.764+0000 7efef2d9c700 1 -- 172.21.15.64:0/708032396 learned_addr learned my addr 172.21.15.64:0/708032396 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:26:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.764+0000 7efef259b700 1 --2- 172.21.15.64:0/708032396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af7577bb0 0x564af7571460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.764+0000 7efef1d9a700 1 --2- 172.21.15.64:0/708032396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564af75719a0 0x564af7571de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.764+0000 7efef2d9c700 1 -- 172.21.15.64:0/708032396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564af75719a0 msgr2=0x564af7571de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:26:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.764+0000 7efef2d9c700 1 --2- 172.21.15.64:0/708032396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564af75719a0 0x564af7571de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.764+0000 7efef2d9c700 1 -- 172.21.15.64:0/708032396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af7577bb0 msgr2=0x564af7571460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:26:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.764+0000 7efef2d9c700 1 --2- 172.21.15.64:0/708032396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af7577bb0 0x564af7571460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.764+0000 7efef2d9c700 1 -- 172.21.15.64:0/708032396 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564af72efd80 con 0x564af7577290 2023-09-13T03:26:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.764+0000 7efef2d9c700 1 --2- 172.21.15.64:0/708032396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564af7577290 0x564af7577670 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7efee4009440 tx=0x7efee4009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5f42856893ab455 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:26:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.765+0000 7efee37fe700 1 --2- 172.21.15.64:0/1468059465 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7efed80a07b0 0x7efed80a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.765+0000 7efee0ff9700 1 -- 172.21.15.64:0/708032396 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee400e070 con 0x564af7577290 2023-09-13T03:26:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.765+0000 7efee0ff9700 1 -- 172.21.15.64:0/708032396 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efee400f040 con 0x564af7577290 2023-09-13T03:26:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.765+0000 7efee0ff9700 1 -- 172.21.15.64:0/708032396 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee4014b20 con 0x564af7577290 2023-09-13T03:26:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.765+0000 7efee37fe700 1 -- 172.21.15.64:0/1468059465 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7efee80cd4f0 con 0x7efeec0679f0 2023-09-13T03:26:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.765+0000 7efef259b700 1 --2- 172.21.15.64:0/1468059465 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7efed80a07b0 0x7efed80a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.765+0000 7eff080c8b80 1 -- 172.21.15.64:0/708032396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564af7577290 msgr2=0x564af7577670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:26:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.765+0000 7eff080c8b80 1 --2- 172.21.15.64:0/708032396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564af7577290 0x564af7577670 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7efee4009440 tx=0x7efee4009920 comp rx=0 tx=0).stop 2023-09-13T03:26:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.766+0000 7efef259b700 1 --2- 172.21.15.64:0/1468059465 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7efed80a07b0 0x7efed80a2c70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7efee80cad10 tx=0x7efee801a040 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:26:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.766+0000 7eff080c8b80 1 -- 172.21.15.64:0/708032396 shutdown_connections 2023-09-13T03:26:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.766+0000 7eff080c8b80 1 --2- 172.21.15.64:0/708032396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564af75719a0 0x564af7571de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.766+0000 7eff080c8b80 1 --2- 172.21.15.64:0/708032396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564af7577290 0x564af7577670 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.766+0000 7eff080c8b80 1 --2- 172.21.15.64:0/708032396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af7577bb0 0x564af7571460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.766+0000 7eff080c8b80 1 -- 172.21.15.64:0/708032396 >> 172.21.15.64:0/708032396 conn(0x564af74e6030 msgr2=0x564af756c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:26:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.766+0000 7eff080c8b80 1 -- 172.21.15.64:0/708032396 shutdown_connections 2023-09-13T03:26:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.766+0000 7eff080c8b80 1 -- 172.21.15.64:0/708032396 wait complete. 2023-09-13T03:26:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.766+0000 7eff080c8b80 1 -- start start 2023-09-13T03:26:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.767+0000 7eff080c8b80 1 Processor -- start 2023-09-13T03:26:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.767+0000 7eff080c8b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af75719a0 0x564af757ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.768+0000 7eff080c8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564af7577290 0x564af757b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.768+0000 7eff080c8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564af7577bb0 0x564af757b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.768+0000 7eff080c8b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564af7461b10 con 0x564af75719a0 2023-09-13T03:26:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.768+0000 7eff080c8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564af7461810 con 0x564af7577290 2023-09-13T03:26:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.768+0000 7eff080c8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564af73458c0 con 0x564af7577bb0 2023-09-13T03:26:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.768+0000 7eff080c8b80 10 client.0 _ll_register_callbacks cb 0x564af73aba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:26:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.768+0000 7efef1d9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564af7577290 0x564af757b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.768+0000 7efef2d9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af75719a0 0x564af757ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.768+0000 7efef1d9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564af7577290 0x564af757b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:55736/0 (socket says 172.21.15.64:55736) 2023-09-13T03:26:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.768+0000 7efef2d9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af75719a0 0x564af757ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37530/0 (socket says 172.21.15.64:37530) 2023-09-13T03:26:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.768+0000 7efef1d9a700 1 -- 172.21.15.64:0/1940581025 learned_addr learned my addr 172.21.15.64:0/1940581025 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:26:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.768+0000 7efef2d9c700 1 --2- 172.21.15.64:0/1940581025 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564af7577bb0 0x564af757b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.768+0000 7efef1d9a700 1 -- 172.21.15.64:0/1940581025 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564af7577bb0 msgr2=0x564af757b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:26:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.768+0000 7efef1d9a700 1 --2- 172.21.15.64:0/1940581025 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564af7577bb0 0x564af757b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.768+0000 7efef1d9a700 1 -- 172.21.15.64:0/1940581025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af75719a0 msgr2=0x564af757ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:26:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.768+0000 7efef1d9a700 1 --2- 172.21.15.64:0/1940581025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af75719a0 0x564af757ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:17.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.768+0000 7efef1d9a700 1 -- 172.21.15.64:0/1940581025 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee40090f0 con 0x564af7577290 2023-09-13T03:26:17.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.769+0000 7efef1d9a700 1 --2- 172.21.15.64:0/1940581025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564af7577290 0x564af757b370 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7efedc009fd0 tx=0x7efedc011a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6187aa671d6ba30f server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:26:17.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.769+0000 7efed5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:26:17.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.769+0000 7efed5ffb700 1 -- 172.21.15.64:0/1940581025 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc016070 con 0x564af7577290 2023-09-13T03:26:17.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.769+0000 7eff080c8b80 1 -- 172.21.15.64:0/1940581025 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564af741b490 con 0x564af7577290 2023-09-13T03:26:17.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.769+0000 7efed5ffb700 1 -- 172.21.15.64:0/1940581025 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7efedc00f690 con 0x564af7577290 2023-09-13T03:26:17.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.773+0000 7efed5ffb700 1 -- 172.21.15.64:0/1940581025 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7efedc00f890 con 0x564af7577290 2023-09-13T03:26:17.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.773+0000 7eff080c8b80 10 client.27182 fetch_fsmap learned FSMap version 69 2023-09-13T03:26:17.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.773+0000 7eff080c8b80 1 -- 172.21.15.64:0/1940581025 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) v3 -- 0x564af72d8c00 con 0x564af7577290 2023-09-13T03:26:17.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.773+0000 7efed5ffb700 1 -- 172.21.15.64:0/1940581025 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564af741b490 con 0x564af7577290 2023-09-13T03:26:17.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.773+0000 7efed5ffb700 1 -- 172.21.15.64:0/1940581025 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 69) v1 ==== 7930+0+0 (secure 0 0 0) 0x7efedc01aca0 con 0x564af7577290 2023-09-13T03:26:17.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.773+0000 7eff080c8b80 10 client.27182 fetch_fsmap finished waiting for FSMap version 69 2023-09-13T03:26:17.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.773+0000 7eff080c8b80 10 client.27182 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:26:17.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.773+0000 7eff080c8b80 20 client.27182 populate_metadata read hostname 'smithi064' 2023-09-13T03:26:17.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.773+0000 7eff080c8b80 1 --2- 172.21.15.64:0/1940581025 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x564af758b6e0 0x564af758db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.773+0000 7eff080c8b80 4 client.27182 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:26:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.773+0000 7eff080c8b80 1 -- 172.21.15.64:0/1940581025 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564af74d8840 con 0x564af758b6e0 2023-09-13T03:26:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.774+0000 7efef2d9c700 1 --2- 172.21.15.64:0/1940581025 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x564af758b6e0 0x564af758db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.774+0000 7efef2d9c700 1 --2- 172.21.15.64:0/1940581025 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x564af758b6e0 0x564af758db00 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62d068fdb5f9bf61 server_cookie=c4c59c648029fce1 in_seq=0 out_seq=0 2023-09-13T03:26:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.774+0000 7efed5ffb700 10 client.27182 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:26:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.776+0000 7efed5ffb700 1 -- 172.21.15.64:0/1940581025 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564af74d8840 con 0x564af758b6e0 2023-09-13T03:26:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.776+0000 7efed5ffb700 10 client.27182 handle_command_reply: tid=0 2023-09-13T03:26:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.776+0000 7eff080c8b80 1 client.27182 shutdown 2023-09-13T03:26:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.776+0000 7eff080c8b80 1 -- 172.21.15.64:0/1940581025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564af7577290 msgr2=0x564af757b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:26:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.776+0000 7eff080c8b80 1 --2- 172.21.15.64:0/1940581025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564af7577290 0x564af757b370 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7efedc009fd0 tx=0x7efedc011a10 comp rx=0 tx=0).stop 2023-09-13T03:26:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.776+0000 7eff080c8b80 1 -- 172.21.15.64:0/1940581025 shutdown_connections 2023-09-13T03:26:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.776+0000 7eff080c8b80 1 --2- 172.21.15.64:0/1940581025 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x564af758b6e0 0x564af758db00 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.776+0000 7eff080c8b80 1 --2- 172.21.15.64:0/1940581025 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564af7577bb0 0x564af757b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.776+0000 7eff080c8b80 1 --2- 172.21.15.64:0/1940581025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af75719a0 0x564af757ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.776+0000 7eff080c8b80 1 --2- 172.21.15.64:0/1940581025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564af7577290 0x564af757b370 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.776+0000 7efed5ffb700 0 client.27182 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:26:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.776+0000 7eff080c8b80 1 -- 172.21.15.64:0/1940581025 >> 172.21.15.64:0/1940581025 conn(0x564af74e6030 msgr2=0x564af756b070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:26:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.777+0000 7eff080c8b80 1 -- 172.21.15.64:0/1940581025 shutdown_connections 2023-09-13T03:26:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.777+0000 7eff080c8b80 1 -- 172.21.15.64:0/1940581025 wait complete. 2023-09-13T03:26:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.777+0000 7eff080c8b80 20 client.27182 trim_cache size 0 max 16384 2023-09-13T03:26:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.796+0000 7eff080c8b80 1 Processor -- start 2023-09-13T03:26:17.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.796+0000 7eff080c8b80 1 -- start start 2023-09-13T03:26:17.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.796+0000 7eff080c8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564af75719a0 0x564af7571460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:17.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.796+0000 7eff080c8b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af7577290 0x564af7582610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:17.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.796+0000 7eff080c8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564af7577bb0 0x564af7576630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:17.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.796+0000 7eff080c8b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564af7582c10 con 0x564af7577290 2023-09-13T03:26:17.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.796+0000 7eff080c8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564af757def0 con 0x564af75719a0 2023-09-13T03:26:17.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.796+0000 7eff080c8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564af7579390 con 0x564af7577bb0 2023-09-13T03:26:17.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.797+0000 7efef1d9a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af7577290 0x564af7582610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:17.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.797+0000 7efef2d9c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564af7577bb0 0x564af7576630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:17.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.797+0000 7efef2d9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564af75719a0 0x564af7571460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:17.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.797+0000 7efef2d9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564af75719a0 0x564af7571460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:55742/0 (socket says 172.21.15.64:55742) 2023-09-13T03:26:17.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.797+0000 7efef1d9a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af7577290 0x564af7582610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37544/0 (socket says 172.21.15.64:37544) 2023-09-13T03:26:17.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.797+0000 7efef2d9c700 1 -- 172.21.15.64:0/2630510270 learned_addr learned my addr 172.21.15.64:0/2630510270 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:26:17.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.797+0000 7efef2d9c700 1 -- 172.21.15.64:0/2630510270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564af7577bb0 msgr2=0x564af7576630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:26:17.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.797+0000 7efef2d9c700 1 --2- 172.21.15.64:0/2630510270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564af7577bb0 0x564af7576630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:17.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.797+0000 7efef2d9c700 1 -- 172.21.15.64:0/2630510270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af7577290 msgr2=0x564af7582610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:26:17.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.797+0000 7efef2d9c700 1 --2- 172.21.15.64:0/2630510270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af7577290 0x564af7582610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.797+0000 7efef2d9c700 1 -- 172.21.15.64:0/2630510270 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564af73161f0 con 0x564af75719a0 2023-09-13T03:26:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.797+0000 7efef2d9c700 1 --2- 172.21.15.64:0/2630510270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564af75719a0 0x564af7571460 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x564af7587430 tx=0x7efee400b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95fb91cc96563000 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:26:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.798+0000 7efee17fa700 1 -- 172.21.15.64:0/2630510270 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee4028070 con 0x564af75719a0 2023-09-13T03:26:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.798+0000 7efee17fa700 1 -- 172.21.15.64:0/2630510270 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efee402b530 con 0x564af75719a0 2023-09-13T03:26:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.798+0000 7efee17fa700 1 -- 172.21.15.64:0/2630510270 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee400e040 con 0x564af75719a0 2023-09-13T03:26:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.798+0000 7eff080c8b80 1 -- 172.21.15.64:0/2630510270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564af75719a0 msgr2=0x564af7571460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:26:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.798+0000 7eff080c8b80 1 --2- 172.21.15.64:0/2630510270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564af75719a0 0x564af7571460 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x564af7587430 tx=0x7efee400b040 comp rx=0 tx=0).stop 2023-09-13T03:26:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.798+0000 7eff080c8b80 1 -- 172.21.15.64:0/2630510270 shutdown_connections 2023-09-13T03:26:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.798+0000 7eff080c8b80 1 --2- 172.21.15.64:0/2630510270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564af75719a0 0x564af7571460 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.798+0000 7eff080c8b80 1 --2- 172.21.15.64:0/2630510270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564af7577bb0 0x564af7576630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.798+0000 7eff080c8b80 1 --2- 172.21.15.64:0/2630510270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af7577290 0x564af7582610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.798+0000 7eff080c8b80 1 -- 172.21.15.64:0/2630510270 >> 172.21.15.64:0/2630510270 conn(0x564af74e6030 msgr2=0x564af7575f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:26:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.799+0000 7eff080c8b80 1 -- 172.21.15.64:0/2630510270 shutdown_connections 2023-09-13T03:26:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.799+0000 7eff080c8b80 1 -- 172.21.15.64:0/2630510270 wait complete. 2023-09-13T03:26:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.799+0000 7eff080c8b80 1 -- start start 2023-09-13T03:26:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.799+0000 7eff080c8b80 1 Processor -- start 2023-09-13T03:26:17.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.800+0000 7eff080c8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564af75719a0 0x564af7573350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:17.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.800+0000 7eff080c8b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af7577290 0x564af7573890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.800+0000 7eff080c8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564af7577bb0 0x564af7573dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.800+0000 7eff080c8b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564af75df580 con 0x564af7577290 2023-09-13T03:26:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.800+0000 7eff080c8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564af75df6f0 con 0x564af7577bb0 2023-09-13T03:26:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.800+0000 7eff080c8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564af75df8b0 con 0x564af75719a0 2023-09-13T03:26:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.800+0000 7eff080c8b80 10 client.0 _ll_register_callbacks cb 0x7efedc009dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:26:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.800+0000 7efef2d9c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564af75719a0 0x564af7573350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.800+0000 7efef1d9a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af7577290 0x564af7573890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.800+0000 7efef2d9c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564af75719a0 0x564af7573350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:56862/0 (socket says 172.21.15.64:56862) 2023-09-13T03:26:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.800+0000 7efef2d9c700 1 -- 172.21.15.64:0/3268941139 learned_addr learned my addr 172.21.15.64:0/3268941139 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:26:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.801+0000 7efef2d9c700 1 --2- 172.21.15.64:0/3268941139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564af7577bb0 0x564af7573dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.801+0000 7efef2d9c700 1 -- 172.21.15.64:0/3268941139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564af7577bb0 msgr2=0x564af7573dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:26:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.801+0000 7efef2d9c700 1 --2- 172.21.15.64:0/3268941139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564af7577bb0 0x564af7573dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.801+0000 7efef2d9c700 1 -- 172.21.15.64:0/3268941139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af7577290 msgr2=0x564af7573890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:26:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.801+0000 7efef2d9c700 1 --2- 172.21.15.64:0/3268941139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af7577290 0x564af7573890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:17.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.801+0000 7efef2d9c700 1 -- 172.21.15.64:0/3268941139 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564af72efd80 con 0x564af75719a0 2023-09-13T03:26:17.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.801+0000 7efef1d9a700 1 --2- 172.21.15.64:0/3268941139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af7577290 0x564af7573890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:26:17.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.801+0000 7efef2d9c700 1 --2- 172.21.15.64:0/3268941139 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564af75719a0 0x564af7573350 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7efee4009da0 tx=0x7efee402b3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd56db542d5b79ed server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:26:17.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.801+0000 7efed5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:26:17.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.801+0000 7efed5ffb700 1 -- 172.21.15.64:0/3268941139 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee4028040 con 0x564af75719a0 2023-09-13T03:26:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.801+0000 7eff080c8b80 1 -- 172.21.15.64:0/3268941139 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efee400f040 con 0x564af75719a0 2023-09-13T03:26:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.802+0000 7efed5ffb700 1 -- 172.21.15.64:0/3268941139 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7efee4003e00 con 0x564af75719a0 2023-09-13T03:26:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.802+0000 7efed5ffb700 1 -- 172.21.15.64:0/3268941139 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7efee400e040 con 0x564af75719a0 2023-09-13T03:26:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.802+0000 7eff080c8b80 10 client.27172 fetch_fsmap learned FSMap version 69 2023-09-13T03:26:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.802+0000 7eff080c8b80 1 -- 172.21.15.64:0/3268941139 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) v3 -- 0x564af72ee250 con 0x564af75719a0 2023-09-13T03:26:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.802+0000 7efed5ffb700 1 -- 172.21.15.64:0/3268941139 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efee400f040 con 0x564af75719a0 2023-09-13T03:26:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.802+0000 7efed5ffb700 1 -- 172.21.15.64:0/3268941139 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 69) v1 ==== 7930+0+0 (secure 0 0 0) 0x7efee4033070 con 0x564af75719a0 2023-09-13T03:26:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.802+0000 7eff080c8b80 10 client.27172 fetch_fsmap finished waiting for FSMap version 69 2023-09-13T03:26:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.802+0000 7eff080c8b80 10 client.27172 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:26:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.802+0000 7eff080c8b80 20 client.27172 populate_metadata read hostname 'smithi064' 2023-09-13T03:26:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.802+0000 7eff080c8b80 1 --2- 172.21.15.64:0/3268941139 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x564af75e0c80 0x564af75e30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.802+0000 7eff080c8b80 4 client.27172 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:26:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.802+0000 7eff080c8b80 1 -- 172.21.15.64:0/3268941139 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564af74bc8c0 con 0x564af75e0c80 2023-09-13T03:26:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.803+0000 7efef1d9a700 1 --2- 172.21.15.64:0/3268941139 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x564af75e0c80 0x564af75e30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.803+0000 7efed5ffb700 10 client.27172 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:26:17.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:17.803+0000 7efef1d9a700 1 --2- 172.21.15.64:0/3268941139 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x564af75e0c80 0x564af75e30a0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e42fea24faae59e5 server_cookie=8e2fc0e7898696cf in_seq=0 out_seq=0 2023-09-13T03:26:18.281 INFO:tasks.workunit.client.0.smithi064.stdout: 10 8286 28.40 MB/sec warmup 22 sec latency 3426.691 ms 2023-09-13T03:26:19.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:26:18 smithi170 ceph-mon[134328]: pgmap v411: 97 pgs: 97 active+clean; 374 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 340 op/s 2023-09-13T03:26:19.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:26:18 smithi064 ceph-mon[97675]: pgmap v411: 97 pgs: 97 active+clean; 374 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 340 op/s 2023-09-13T03:26:19.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:26:18 smithi142 ceph-mon[134082]: pgmap v411: 97 pgs: 97 active+clean; 374 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 340 op/s 2023-09-13T03:26:19.281 INFO:tasks.workunit.client.0.smithi064.stdout: 10 8955 27.96 MB/sec warmup 23 sec latency 4426.752 ms 2023-09-13T03:26:19.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:19.460+0000 7efed5ffb700 1 -- 172.21.15.64:0/3268941139 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564af74bc8c0 con 0x564af75e0c80 2023-09-13T03:26:19.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:19.460+0000 7efed5ffb700 10 client.27172 handle_command_reply: tid=0 2023-09-13T03:26:19.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:19.460+0000 7eff080c8b80 1 client.27172 shutdown 2023-09-13T03:26:19.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:19.461+0000 7eff080c8b80 1 -- 172.21.15.64:0/3268941139 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564af75719a0 msgr2=0x564af7573350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:26:19.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:19.461+0000 7eff080c8b80 1 --2- 172.21.15.64:0/3268941139 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564af75719a0 0x564af7573350 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7efee4009da0 tx=0x7efee402b3a0 comp rx=0 tx=0).stop 2023-09-13T03:26:19.466 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:19.461+0000 7eff080c8b80 1 -- 172.21.15.64:0/3268941139 shutdown_connections 2023-09-13T03:26:19.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:19.461+0000 7eff080c8b80 1 --2- 172.21.15.64:0/3268941139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564af7577bb0 0x564af7573dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:19.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:19.461+0000 7eff080c8b80 1 --2- 172.21.15.64:0/3268941139 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x564af75e0c80 0x564af75e30a0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:19.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:19.461+0000 7eff080c8b80 1 --2- 172.21.15.64:0/3268941139 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564af75719a0 0x564af7573350 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:19.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:19.461+0000 7eff080c8b80 1 --2- 172.21.15.64:0/3268941139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af7577290 0x564af7573890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:19.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:19.461+0000 7eff080c8b80 1 -- 172.21.15.64:0/3268941139 >> 172.21.15.64:0/3268941139 conn(0x564af74e6030 msgr2=0x564af7579e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:26:19.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:19.461+0000 7efed5ffb700 0 client.27172 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:26:19.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:19.462+0000 7eff080c8b80 1 -- 172.21.15.64:0/3268941139 shutdown_connections 2023-09-13T03:26:19.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:19.462+0000 7eff080c8b80 1 -- 172.21.15.64:0/3268941139 wait complete. 2023-09-13T03:26:19.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:19.462+0000 7eff080c8b80 20 client.27172 trim_cache size 0 max 16384 2023-09-13T03:26:19.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:26:19.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:26:19.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:26:19.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:26:19.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:19.463+0000 7efef359d700 1 -- 172.21.15.64:0/1468059465 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7efed80a07b0 msgr2=0x7efed80a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:26:19.470 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:19.463+0000 7efef359d700 1 --2- 172.21.15.64:0/1468059465 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7efed80a07b0 0x7efed80a2c70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7efee80cad10 tx=0x7efee801a040 comp rx=0 tx=0).stop 2023-09-13T03:26:19.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:19.463+0000 7efef359d700 1 -- 172.21.15.64:0/1468059465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efeec0679f0 msgr2=0x7efeec19e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:26:19.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:19.463+0000 7efef359d700 1 --2- 172.21.15.64:0/1468059465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efeec0679f0 0x7efeec19e3e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7efee8000c00 tx=0x7efee8005740 comp rx=0 tx=0).stop 2023-09-13T03:26:19.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:19.463+0000 7efef359d700 1 -- 172.21.15.64:0/1468059465 shutdown_connections 2023-09-13T03:26:19.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:19.463+0000 7efef359d700 1 --2- 172.21.15.64:0/1468059465 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7efed80a07b0 0x7efed80a2c70 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:19.471 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:19.463+0000 7efef359d700 1 --2- 172.21.15.64:0/1468059465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efeec0679f0 0x7efeec19e3e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:19.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:19.463+0000 7efef359d700 1 --2- 172.21.15.64:0/1468059465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efeec106480 0x7efeec1985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:19.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:19.463+0000 7efef359d700 1 --2- 172.21.15.64:0/1468059465 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efeec0683b0 0x7efeec19e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:19.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:19.463+0000 7efef359d700 1 -- 172.21.15.64:0/1468059465 >> 172.21.15.64:0/1468059465 conn(0x7efeec074dc0 msgr2=0x7efeec0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:26:19.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:19.464+0000 7efef359d700 1 -- 172.21.15.64:0/1468059465 shutdown_connections 2023-09-13T03:26:19.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:19.464+0000 7efef359d700 1 -- 172.21.15.64:0/1468059465 wait complete. 2023-09-13T03:26:20.282 INFO:tasks.workunit.client.0.smithi064.stdout: 10 9794 29.33 MB/sec warmup 24 sec latency 4540.351 ms 2023-09-13T03:26:21.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:26:20 smithi170 ceph-mon[134328]: pgmap v412: 97 pgs: 97 active+clean; 374 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 275 op/s 2023-09-13T03:26:21.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:26:20 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:26:21.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:26:20 smithi064 ceph-mon[97675]: pgmap v412: 97 pgs: 97 active+clean; 374 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 275 op/s 2023-09-13T03:26:21.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:26:20 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:26:21.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:26:20 smithi142 ceph-mon[134082]: pgmap v412: 97 pgs: 97 active+clean; 374 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 275 op/s 2023-09-13T03:26:21.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:26:20 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:26:21.282 INFO:tasks.workunit.client.0.smithi064.stdout: 10 10704 30.10 MB/sec warmup 25 sec latency 53.369 ms 2023-09-13T03:26:22.282 INFO:tasks.workunit.client.0.smithi064.stdout: 10 12127 31.53 MB/sec warmup 26 sec latency 51.196 ms 2023-09-13T03:26:23.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:26:22 smithi170 ceph-mon[134328]: pgmap v413: 97 pgs: 97 active+clean; 371 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 564 op/s 2023-09-13T03:26:23.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:26:22 smithi064 ceph-mon[97675]: pgmap v413: 97 pgs: 97 active+clean; 371 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 564 op/s 2023-09-13T03:26:23.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:26:22 smithi142 ceph-mon[134082]: pgmap v413: 97 pgs: 97 active+clean; 371 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 564 op/s 2023-09-13T03:26:23.282 INFO:tasks.workunit.client.0.smithi064.stdout: 10 13479 33.47 MB/sec warmup 27 sec latency 54.578 ms 2023-09-13T03:26:24.282 INFO:tasks.workunit.client.0.smithi064.stdout: 10 14412 33.92 MB/sec warmup 28 sec latency 117.242 ms 2023-09-13T03:26:25.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:26:24 smithi170 ceph-mon[134328]: pgmap v414: 97 pgs: 97 active+clean; 371 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 564 op/s 2023-09-13T03:26:25.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:26:24 smithi064 ceph-mon[97675]: pgmap v414: 97 pgs: 97 active+clean; 371 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 564 op/s 2023-09-13T03:26:25.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:26:24 smithi142 ceph-mon[134082]: pgmap v414: 97 pgs: 97 active+clean; 371 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 564 op/s 2023-09-13T03:26:25.282 INFO:tasks.workunit.client.0.smithi064.stdout: 10 15830 35.30 MB/sec warmup 29 sec latency 34.597 ms 2023-09-13T03:26:26.282 INFO:tasks.workunit.client.0.smithi064.stdout: 10 17010 36.33 MB/sec warmup 30 sec latency 29.485 ms 2023-09-13T03:26:27.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:26:26 smithi170 ceph-mon[134328]: pgmap v415: 97 pgs: 97 active+clean; 408 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.2 MiB/s wr, 870 op/s 2023-09-13T03:26:27.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:26:26 smithi064 ceph-mon[97675]: pgmap v415: 97 pgs: 97 active+clean; 408 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.2 MiB/s wr, 870 op/s 2023-09-13T03:26:27.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:26:26 smithi142 ceph-mon[134082]: pgmap v415: 97 pgs: 97 active+clean; 408 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.2 MiB/s wr, 870 op/s 2023-09-13T03:26:27.282 INFO:tasks.workunit.client.0.smithi064.stdout: 10 18071 36.94 MB/sec warmup 31 sec latency 71.261 ms 2023-09-13T03:26:28.282 INFO:tasks.workunit.client.0.smithi064.stdout: 10 19292 37.90 MB/sec warmup 32 sec latency 40.046 ms 2023-09-13T03:26:29.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:26:28 smithi170 ceph-mon[134328]: pgmap v416: 97 pgs: 97 active+clean; 408 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.6 MiB/s wr, 709 op/s 2023-09-13T03:26:29.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:26:28 smithi064 ceph-mon[97675]: pgmap v416: 97 pgs: 97 active+clean; 408 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.6 MiB/s wr, 709 op/s 2023-09-13T03:26:29.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:26:28 smithi142 ceph-mon[134082]: pgmap v416: 97 pgs: 97 active+clean; 408 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.6 MiB/s wr, 709 op/s 2023-09-13T03:26:29.282 INFO:tasks.workunit.client.0.smithi064.stdout: 10 20480 38.80 MB/sec warmup 33 sec latency 43.712 ms 2023-09-13T03:26:30.282 INFO:tasks.workunit.client.0.smithi064.stdout: 10 21533 39.09 MB/sec warmup 34 sec latency 86.140 ms 2023-09-13T03:26:30.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:26:30 smithi142 ceph-mon[134082]: pgmap v417: 97 pgs: 97 active+clean; 408 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.6 MiB/s wr, 709 op/s 2023-09-13T03:26:30.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:26:30 smithi170 ceph-mon[134328]: pgmap v417: 97 pgs: 97 active+clean; 408 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.6 MiB/s wr, 709 op/s 2023-09-13T03:26:30.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:26:30 smithi064 ceph-mon[97675]: pgmap v417: 97 pgs: 97 active+clean; 408 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.6 MiB/s wr, 709 op/s 2023-09-13T03:26:31.282 INFO:tasks.workunit.client.0.smithi064.stdout: 10 22733 40.00 MB/sec warmup 35 sec latency 35.907 ms 2023-09-13T03:26:32.285 INFO:tasks.workunit.client.0.smithi064.stdout: 10 23936 40.63 MB/sec warmup 36 sec latency 29.744 ms 2023-09-13T03:26:32.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.664+0000 7fbc519ff700 1 -- 172.21.15.64:0/3776764929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc4c115690 msgr2=0x7fbc4c117a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:26:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.664+0000 7fbc519ff700 1 --2- 172.21.15.64:0/3776764929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc4c115690 0x7fbc4c117a80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbc4400b0a0 tx=0x7fbc4400b3b0 comp rx=0 tx=0).stop 2023-09-13T03:26:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.665+0000 7fbc519ff700 1 -- 172.21.15.64:0/3776764929 shutdown_connections 2023-09-13T03:26:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.665+0000 7fbc519ff700 1 --2- 172.21.15.64:0/3776764929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc4c115690 0x7fbc4c117a80 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.665+0000 7fbc519ff700 1 --2- 172.21.15.64:0/3776764929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc4c10ad10 0x7fbc4c107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.665+0000 7fbc519ff700 1 --2- 172.21.15.64:0/3776764929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc4c107b40 0x7fbc4c107fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.665+0000 7fbc519ff700 1 -- 172.21.15.64:0/3776764929 >> 172.21.15.64:0/3776764929 conn(0x7fbc4c06cb00 msgr2=0x7fbc4c06cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:26:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.665+0000 7fbc519ff700 1 -- 172.21.15.64:0/3776764929 shutdown_connections 2023-09-13T03:26:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.665+0000 7fbc519ff700 1 -- 172.21.15.64:0/3776764929 wait complete. 2023-09-13T03:26:32.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.666+0000 7fbc519ff700 1 Processor -- start 2023-09-13T03:26:32.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.667+0000 7fbc519ff700 1 -- start start 2023-09-13T03:26:32.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.667+0000 7fbc519ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc4c107b40 0x7fbc4c1a1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:32.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.667+0000 7fbc519ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc4c10ad10 0x7fbc4c1a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:32.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.667+0000 7fbc519ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc4c115690 0x7fbc4c1a5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:32.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.667+0000 7fbc519ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4c1a63f0 con 0x7fbc4c107b40 2023-09-13T03:26:32.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.667+0000 7fbc519ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4c1a6560 con 0x7fbc4c10ad10 2023-09-13T03:26:32.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.667+0000 7fbc519ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4c1a6720 con 0x7fbc4c115690 2023-09-13T03:26:32.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.668+0000 7fbc43fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc4c10ad10 0x7fbc4c1a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:32.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.668+0000 7fbc43fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc4c10ad10 0x7fbc4c1a1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:45886/0 (socket says 172.21.15.64:45886) 2023-09-13T03:26:32.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.668+0000 7fbc43fff700 1 -- 172.21.15.64:0/1123234263 learned_addr learned my addr 172.21.15.64:0/1123234263 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:26:32.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.668+0000 7fbc4affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc4c107b40 0x7fbc4c1a1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:32.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.668+0000 7fbc4b7fe700 1 --2- 172.21.15.64:0/1123234263 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc4c115690 0x7fbc4c1a5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:32.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.668+0000 7fbc4affd700 1 -- 172.21.15.64:0/1123234263 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc4c115690 msgr2=0x7fbc4c1a5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:26:32.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.668+0000 7fbc4affd700 1 --2- 172.21.15.64:0/1123234263 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc4c115690 0x7fbc4c1a5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:32.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.668+0000 7fbc4affd700 1 -- 172.21.15.64:0/1123234263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc4c10ad10 msgr2=0x7fbc4c1a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:26:32.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.668+0000 7fbc4affd700 1 --2- 172.21.15.64:0/1123234263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc4c10ad10 0x7fbc4c1a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:32.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.668+0000 7fbc4affd700 1 -- 172.21.15.64:0/1123234263 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc44009d00 con 0x7fbc4c107b40 2023-09-13T03:26:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.669+0000 7fbc4affd700 1 --2- 172.21.15.64:0/1123234263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc4c107b40 0x7fbc4c1a1200 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c00e9f0 tx=0x7fbc3c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:26:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.669+0000 7fbc509fd700 1 -- 172.21.15.64:0/1123234263 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c00c9c0 con 0x7fbc4c107b40 2023-09-13T03:26:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.669+0000 7fbc509fd700 1 -- 172.21.15.64:0/1123234263 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc3c010430 con 0x7fbc4c107b40 2023-09-13T03:26:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.669+0000 7fbc519ff700 1 -- 172.21.15.64:0/1123234263 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc4c1a6a00 con 0x7fbc4c107b40 2023-09-13T03:26:32.674 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.669+0000 7fbc509fd700 1 -- 172.21.15.64:0/1123234263 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c018680 con 0x7fbc4c107b40 2023-09-13T03:26:32.674 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.669+0000 7fbc519ff700 1 -- 172.21.15.64:0/1123234263 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc4c1ae240 con 0x7fbc4c107b40 2023-09-13T03:26:32.674 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.674+0000 7fbc509fd700 1 -- 172.21.15.64:0/1123234263 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fbc3c010860 con 0x7fbc4c107b40 2023-09-13T03:26:32.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.675+0000 7fbc653d7b80 1 Processor -- start 2023-09-13T03:26:32.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.675+0000 7fbc653d7b80 1 -- start start 2023-09-13T03:26:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.675+0000 7fbc653d7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641dd37c1a0 0x5641dd37c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.676+0000 7fbc653d7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5641dd376300 0x5641dd376780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.676+0000 7fbc653d7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641dd376cc0 0x5641dd382f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.676+0000 7fbc653d7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641dd2cf6d0 con 0x5641dd37c1a0 2023-09-13T03:26:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.676+0000 7fbc653d7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641dd2ccff0 con 0x5641dd376cc0 2023-09-13T03:26:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.676+0000 7fbc653d7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641dd2c88e0 con 0x5641dd376300 2023-09-13T03:26:32.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.676+0000 7fbc4affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5641dd376300 0x5641dd376780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:32.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.676+0000 7fbc4b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641dd37c1a0 0x5641dd37c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:32.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.676+0000 7fbc4affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5641dd376300 0x5641dd376780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:35700/0 (socket says 172.21.15.64:35700) 2023-09-13T03:26:32.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.676+0000 7fbc4affd700 1 -- 172.21.15.64:0/877724723 learned_addr learned my addr 172.21.15.64:0/877724723 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:26:32.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.676+0000 7fbc43fff700 1 --2- 172.21.15.64:0/877724723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641dd376cc0 0x5641dd382f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:32.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.677+0000 7fbc4affd700 1 -- 172.21.15.64:0/877724723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641dd376cc0 msgr2=0x5641dd382f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:26:32.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.677+0000 7fbc509fd700 1 --2- 172.21.15.64:0/1123234263 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fbc380a07b0 0x7fbc380a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:32.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.678+0000 7fbc4affd700 1 --2- 172.21.15.64:0/877724723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641dd376cc0 0x5641dd382f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:32.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.678+0000 7fbc509fd700 1 -- 172.21.15.64:0/1123234263 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fbc3c014070 con 0x7fbc4c107b40 2023-09-13T03:26:32.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.678+0000 7fbc4affd700 1 -- 172.21.15.64:0/877724723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641dd37c1a0 msgr2=0x5641dd37c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:26:32.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.678+0000 7fbc4affd700 1 --2- 172.21.15.64:0/877724723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641dd37c1a0 0x5641dd37c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:32.681 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.678+0000 7fbc4affd700 1 -- 172.21.15.64:0/877724723 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641dd0f4d80 con 0x5641dd376300 2023-09-13T03:26:32.682 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.678+0000 7fbc43fff700 1 --2- 172.21.15.64:0/877724723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641dd376cc0 0x5641dd382f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:26:32.682 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.678+0000 7fbc4b7fe700 1 --2- 172.21.15.64:0/877724723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641dd37c1a0 0x5641dd37c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:26:32.682 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.678+0000 7fbc4affd700 1 --2- 172.21.15.64:0/877724723 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5641dd376300 0x5641dd376780 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c0c96e0 tx=0x7fbc3c010c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2079311167248685 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:26:32.682 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.678+0000 7fbc4b7fe700 1 --2- 172.21.15.64:0/1123234263 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fbc380a07b0 0x7fbc380a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:32.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.679+0000 7fbc41ffb700 1 -- 172.21.15.64:0/877724723 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c095840 con 0x5641dd376300 2023-09-13T03:26:32.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.679+0000 7fbc41ffb700 1 -- 172.21.15.64:0/877724723 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc3c0992d0 con 0x5641dd376300 2023-09-13T03:26:32.683 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.679+0000 7fbc41ffb700 1 -- 172.21.15.64:0/877724723 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c099590 con 0x5641dd376300 2023-09-13T03:26:32.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.680+0000 7fbc653d7b80 1 -- 172.21.15.64:0/877724723 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5641dd376300 msgr2=0x5641dd376780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:26:32.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.680+0000 7fbc653d7b80 1 --2- 172.21.15.64:0/877724723 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5641dd376300 0x5641dd376780 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c0c96e0 tx=0x7fbc3c010c60 comp rx=0 tx=0).stop 2023-09-13T03:26:32.684 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.680+0000 7fbc4b7fe700 1 --2- 172.21.15.64:0/1123234263 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fbc380a07b0 0x7fbc380a2c70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbc4400b780 tx=0x7fbc440099e0 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:26:32.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.680+0000 7fbc653d7b80 1 -- 172.21.15.64:0/877724723 shutdown_connections 2023-09-13T03:26:32.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.680+0000 7fbc653d7b80 1 --2- 172.21.15.64:0/877724723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641dd376cc0 0x5641dd382f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:32.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.680+0000 7fbc653d7b80 1 --2- 172.21.15.64:0/877724723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641dd37c1a0 0x5641dd37c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:32.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.680+0000 7fbc653d7b80 1 --2- 172.21.15.64:0/877724723 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5641dd376300 0x5641dd376780 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:32.685 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.680+0000 7fbc653d7b80 1 -- 172.21.15.64:0/877724723 >> 172.21.15.64:0/877724723 conn(0x5641dd2eb030 msgr2=0x5641dd371560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:26:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.680+0000 7fbc653d7b80 1 -- 172.21.15.64:0/877724723 shutdown_connections 2023-09-13T03:26:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.680+0000 7fbc653d7b80 1 -- 172.21.15.64:0/877724723 wait complete. 2023-09-13T03:26:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.681+0000 7fbc653d7b80 1 -- start start 2023-09-13T03:26:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.681+0000 7fbc653d7b80 1 Processor -- start 2023-09-13T03:26:32.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.683+0000 7fbc653d7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5641dd376300 0x5641dd382ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:32.687 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.683+0000 7fbc653d7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641dd376cc0 0x5641dd380170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:32.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.683+0000 7fbc653d7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641dd37c1a0 0x5641dd380860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:32.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.683+0000 7fbc653d7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641dd266b10 con 0x5641dd37c1a0 2023-09-13T03:26:32.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.683+0000 7fbc653d7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641dd266810 con 0x5641dd376cc0 2023-09-13T03:26:32.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.683+0000 7fbc653d7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641dd14a8c0 con 0x5641dd376300 2023-09-13T03:26:32.688 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.684+0000 7fbc653d7b80 10 client.0 _ll_register_callbacks cb 0x5641dd1b0a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:26:32.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.684+0000 7fbc43fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641dd376cc0 0x5641dd380170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:32.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.684+0000 7fbc43fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641dd376cc0 0x5641dd380170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:45910/0 (socket says 172.21.15.64:45910) 2023-09-13T03:26:32.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.684+0000 7fbc43fff700 1 -- 172.21.15.64:0/1164981503 learned_addr learned my addr 172.21.15.64:0/1164981503 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:26:32.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.684+0000 7fbc43fff700 1 -- 172.21.15.64:0/1164981503 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5641dd376300 msgr2=0x5641dd382ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-13T03:26:32.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.684+0000 7fbc43fff700 1 --2- 172.21.15.64:0/1164981503 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5641dd376300 0x5641dd382ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:32.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.684+0000 7fbc43fff700 1 -- 172.21.15.64:0/1164981503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641dd37c1a0 msgr2=0x5641dd380860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:26:32.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.684+0000 7fbc43fff700 1 --2- 172.21.15.64:0/1164981503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641dd37c1a0 0x5641dd380860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:32.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.684+0000 7fbc43fff700 1 -- 172.21.15.64:0/1164981503 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc4c1186c0 con 0x5641dd376cc0 2023-09-13T03:26:32.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.685+0000 7fbc43fff700 1 --2- 172.21.15.64:0/1164981503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641dd376cc0 0x5641dd380170 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbc34009fd0 tx=0x7fbc34012a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc6a7b35f3d70e11 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:26:32.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.685+0000 7fbc2affd700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:26:32.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.685+0000 7fbc2affd700 1 -- 172.21.15.64:0/1164981503 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc34017070 con 0x5641dd376cc0 2023-09-13T03:26:32.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.685+0000 7fbc2affd700 1 -- 172.21.15.64:0/1164981503 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fbc34004050 con 0x5641dd376cc0 2023-09-13T03:26:32.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.686+0000 7fbc653d7b80 1 -- 172.21.15.64:0/1164981503 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5641dd220490 con 0x5641dd376cc0 2023-09-13T03:26:32.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.686+0000 7fbc2affd700 1 -- 172.21.15.64:0/1164981503 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc34004210 con 0x5641dd376cc0 2023-09-13T03:26:32.695 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.686+0000 7fbc653d7b80 10 client.27200 fetch_fsmap learned FSMap version 70 2023-09-13T03:26:32.695 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.686+0000 7fbc653d7b80 1 -- 172.21.15.64:0/1164981503 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x7fbc4c1ae240 con 0x5641dd376cc0 2023-09-13T03:26:32.695 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.686+0000 7fbc2affd700 1 -- 172.21.15.64:0/1164981503 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5641dd220490 con 0x5641dd376cc0 2023-09-13T03:26:32.695 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.687+0000 7fbc2affd700 1 -- 172.21.15.64:0/1164981503 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 70) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fbc3400f5f0 con 0x5641dd376cc0 2023-09-13T03:26:32.695 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.687+0000 7fbc653d7b80 10 client.27200 fetch_fsmap finished waiting for FSMap version 70 2023-09-13T03:26:32.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.687+0000 7fbc653d7b80 10 client.27200 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:26:32.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.687+0000 7fbc653d7b80 20 client.27200 populate_metadata read hostname 'smithi064' 2023-09-13T03:26:32.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.687+0000 7fbc653d7b80 1 --2- 172.21.15.64:0/1164981503 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5641dd398d80 0x5641dd39b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:32.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.687+0000 7fbc653d7b80 4 client.27200 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:26:32.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.687+0000 7fbc653d7b80 1 -- 172.21.15.64:0/1164981503 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5641dd2dd840 con 0x5641dd398d80 2023-09-13T03:26:32.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.688+0000 7fbc4affd700 1 --2- 172.21.15.64:0/1164981503 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5641dd398d80 0x5641dd39b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:32.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.689+0000 7fbc4affd700 1 --2- 172.21.15.64:0/1164981503 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5641dd398d80 0x5641dd39b1a0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e01a2e26ad32c2a3 server_cookie=429489db417c3532 in_seq=0 out_seq=0 2023-09-13T03:26:32.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.689+0000 7fbc2affd700 10 client.27200 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:26:32.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.690+0000 7fbc2affd700 1 -- 172.21.15.64:0/1164981503 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5641dd2dd840 con 0x5641dd398d80 2023-09-13T03:26:32.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.690+0000 7fbc2affd700 10 client.27200 handle_command_reply: tid=0 2023-09-13T03:26:32.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.690+0000 7fbc653d7b80 1 client.27200 shutdown 2023-09-13T03:26:32.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.690+0000 7fbc653d7b80 1 -- 172.21.15.64:0/1164981503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641dd376cc0 msgr2=0x5641dd380170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:26:32.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.690+0000 7fbc653d7b80 1 --2- 172.21.15.64:0/1164981503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641dd376cc0 0x5641dd380170 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbc34009fd0 tx=0x7fbc34012a60 comp rx=0 tx=0).stop 2023-09-13T03:26:32.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.690+0000 7fbc653d7b80 1 -- 172.21.15.64:0/1164981503 shutdown_connections 2023-09-13T03:26:32.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.690+0000 7fbc653d7b80 1 --2- 172.21.15.64:0/1164981503 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5641dd398d80 0x5641dd39b1a0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:32.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.690+0000 7fbc653d7b80 1 --2- 172.21.15.64:0/1164981503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641dd37c1a0 0x5641dd380860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:32.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.690+0000 7fbc653d7b80 1 --2- 172.21.15.64:0/1164981503 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5641dd376300 0x5641dd382ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:32.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.690+0000 7fbc653d7b80 1 --2- 172.21.15.64:0/1164981503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641dd376cc0 0x5641dd380170 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:32.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.690+0000 7fbc653d7b80 1 -- 172.21.15.64:0/1164981503 >> 172.21.15.64:0/1164981503 conn(0x5641dd2eb030 msgr2=0x5641dd3707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:26:32.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.691+0000 7fbc2affd700 0 client.27200 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:26:32.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.691+0000 7fbc653d7b80 1 -- 172.21.15.64:0/1164981503 shutdown_connections 2023-09-13T03:26:32.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.691+0000 7fbc653d7b80 1 -- 172.21.15.64:0/1164981503 wait complete. 2023-09-13T03:26:32.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.691+0000 7fbc653d7b80 20 client.27200 trim_cache size 0 max 16384 2023-09-13T03:26:32.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.712+0000 7fbc653d7b80 1 Processor -- start 2023-09-13T03:26:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.712+0000 7fbc653d7b80 1 -- start start 2023-09-13T03:26:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.712+0000 7fbc653d7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641dd376300 0x5641dd37ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.712+0000 7fbc653d7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5641dd376cc0 0x5641dd385ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.712+0000 7fbc653d7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641dd37c1a0 0x5641dd37f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.712+0000 7fbc653d7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641dd37be00 con 0x5641dd37c1a0 2023-09-13T03:26:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.712+0000 7fbc653d7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641dd372930 con 0x5641dd376300 2023-09-13T03:26:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.712+0000 7fbc653d7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641dd372d60 con 0x5641dd376cc0 2023-09-13T03:26:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.713+0000 7fbc4affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641dd376300 0x5641dd37ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.713+0000 7fbc43fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5641dd376cc0 0x5641dd385ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.713+0000 7fbc4affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641dd376300 0x5641dd37ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:45914/0 (socket says 172.21.15.64:45914) 2023-09-13T03:26:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.713+0000 7fbc4affd700 1 -- 172.21.15.64:0/220546317 learned_addr learned my addr 172.21.15.64:0/220546317 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:26:32.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.713+0000 7fbc4b7fe700 1 --2- 172.21.15.64:0/220546317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641dd37c1a0 0x5641dd37f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:32.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.713+0000 7fbc4affd700 1 -- 172.21.15.64:0/220546317 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5641dd376cc0 msgr2=0x5641dd385ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:26:32.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.713+0000 7fbc4affd700 1 --2- 172.21.15.64:0/220546317 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5641dd376cc0 0x5641dd385ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:32.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.713+0000 7fbc4affd700 1 -- 172.21.15.64:0/220546317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641dd37c1a0 msgr2=0x5641dd37f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:26:32.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.713+0000 7fbc4affd700 1 --2- 172.21.15.64:0/220546317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641dd37c1a0 0x5641dd37f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:32.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.713+0000 7fbc4affd700 1 -- 172.21.15.64:0/220546317 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641dd0ddc00 con 0x5641dd376300 2023-09-13T03:26:32.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.714+0000 7fbc4affd700 1 --2- 172.21.15.64:0/220546317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641dd376300 0x5641dd37ddf0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c01e000 tx=0x7fbc3c0cf550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35df44bd52b791d6 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:26:32.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.714+0000 7fbc427fc700 1 -- 172.21.15.64:0/220546317 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c09d170 con 0x5641dd376300 2023-09-13T03:26:32.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.714+0000 7fbc427fc700 1 -- 172.21.15.64:0/220546317 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc3c099040 con 0x5641dd376300 2023-09-13T03:26:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.714+0000 7fbc427fc700 1 -- 172.21.15.64:0/220546317 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c099500 con 0x5641dd376300 2023-09-13T03:26:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.715+0000 7fbc653d7b80 1 -- 172.21.15.64:0/220546317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641dd376300 msgr2=0x5641dd37ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:26:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.715+0000 7fbc653d7b80 1 --2- 172.21.15.64:0/220546317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641dd376300 0x5641dd37ddf0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c01e000 tx=0x7fbc3c0cf550 comp rx=0 tx=0).stop 2023-09-13T03:26:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.715+0000 7fbc653d7b80 1 -- 172.21.15.64:0/220546317 shutdown_connections 2023-09-13T03:26:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.715+0000 7fbc653d7b80 1 --2- 172.21.15.64:0/220546317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641dd376300 0x5641dd37ddf0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:32.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.715+0000 7fbc653d7b80 1 --2- 172.21.15.64:0/220546317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641dd37c1a0 0x5641dd37f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:32.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.715+0000 7fbc653d7b80 1 --2- 172.21.15.64:0/220546317 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5641dd376cc0 0x5641dd385ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:32.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.715+0000 7fbc653d7b80 1 -- 172.21.15.64:0/220546317 >> 172.21.15.64:0/220546317 conn(0x5641dd2eb030 msgr2=0x5641dd371090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:26:32.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.715+0000 7fbc653d7b80 1 -- 172.21.15.64:0/220546317 shutdown_connections 2023-09-13T03:26:32.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.715+0000 7fbc653d7b80 1 -- 172.21.15.64:0/220546317 wait complete. 2023-09-13T03:26:32.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.716+0000 7fbc653d7b80 1 -- start start 2023-09-13T03:26:32.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.716+0000 7fbc653d7b80 1 Processor -- start 2023-09-13T03:26:32.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.716+0000 7fbc653d7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5641dd376300 0x5641dd3852e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:32.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.716+0000 7fbc653d7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641dd376cc0 0x5641dd385820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:32.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.717+0000 7fbc653d7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641dd37c1a0 0x5641dd3e4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:32.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.717+0000 7fbc653d7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641dd3e4810 con 0x5641dd37c1a0 2023-09-13T03:26:32.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.717+0000 7fbc653d7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641dd3e4980 con 0x5641dd376cc0 2023-09-13T03:26:32.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.717+0000 7fbc653d7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641dd3e4b40 con 0x5641dd376300 2023-09-13T03:26:32.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.717+0000 7fbc653d7b80 10 client.0 _ll_register_callbacks cb 0x7fbc34009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:26:32.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.717+0000 7fbc43fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641dd376cc0 0x5641dd385820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:32.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.717+0000 7fbc4affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5641dd376300 0x5641dd3852e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:32.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.717+0000 7fbc43fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641dd376cc0 0x5641dd385820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:45918/0 (socket says 172.21.15.64:45918) 2023-09-13T03:26:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.717+0000 7fbc43fff700 1 -- 172.21.15.64:0/2443162789 learned_addr learned my addr 172.21.15.64:0/2443162789 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:26:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.717+0000 7fbc4b7fe700 1 --2- 172.21.15.64:0/2443162789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641dd37c1a0 0x5641dd3e4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.717+0000 7fbc4affd700 1 -- 172.21.15.64:0/2443162789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641dd376cc0 msgr2=0x5641dd385820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:26:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.717+0000 7fbc4affd700 1 --2- 172.21.15.64:0/2443162789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641dd376cc0 0x5641dd385820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.717+0000 7fbc4affd700 1 -- 172.21.15.64:0/2443162789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641dd37c1a0 msgr2=0x5641dd3e4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:26:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.717+0000 7fbc4affd700 1 --2- 172.21.15.64:0/2443162789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641dd37c1a0 0x5641dd3e4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.717+0000 7fbc4affd700 1 -- 172.21.15.64:0/2443162789 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5641dd0f4d80 con 0x5641dd376300 2023-09-13T03:26:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.718+0000 7fbc4affd700 1 --2- 172.21.15.64:0/2443162789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5641dd376300 0x5641dd3852e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c01e000 tx=0x7fbc3c098f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=753aa765e69f55fa server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:26:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.718+0000 7fbc2affd700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:26:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.718+0000 7fbc2affd700 1 -- 172.21.15.64:0/2443162789 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c099500 con 0x5641dd376300 2023-09-13T03:26:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.718+0000 7fbc2affd700 1 -- 172.21.15.64:0/2443162789 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fbc3c09d330 con 0x5641dd376300 2023-09-13T03:26:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.718+0000 7fbc653d7b80 1 -- 172.21.15.64:0/2443162789 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbc3c0992d0 con 0x5641dd376300 2023-09-13T03:26:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.718+0000 7fbc2affd700 1 -- 172.21.15.64:0/2443162789 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc3c0840b0 con 0x5641dd376300 2023-09-13T03:26:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.718+0000 7fbc653d7b80 10 client.27187 fetch_fsmap learned FSMap version 70 2023-09-13T03:26:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.718+0000 7fbc653d7b80 1 -- 172.21.15.64:0/2443162789 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x5641dd11b1f0 con 0x5641dd376300 2023-09-13T03:26:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.719+0000 7fbc2affd700 1 -- 172.21.15.64:0/2443162789 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc3c0992d0 con 0x5641dd376300 2023-09-13T03:26:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.719+0000 7fbc2affd700 1 -- 172.21.15.64:0/2443162789 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 70) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fbc3c0989f0 con 0x5641dd376300 2023-09-13T03:26:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.719+0000 7fbc653d7b80 10 client.27187 fetch_fsmap finished waiting for FSMap version 70 2023-09-13T03:26:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.719+0000 7fbc653d7b80 10 client.27187 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:26:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.719+0000 7fbc653d7b80 20 client.27187 populate_metadata read hostname 'smithi064' 2023-09-13T03:26:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.719+0000 7fbc653d7b80 1 --2- 172.21.15.64:0/2443162789 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5641dd3ed0c0 0x5641dd3ef4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:32.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.719+0000 7fbc653d7b80 4 client.27187 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:26:32.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.719+0000 7fbc653d7b80 1 -- 172.21.15.64:0/2443162789 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5641dd2c18c0 con 0x5641dd3ed0c0 2023-09-13T03:26:32.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.720+0000 7fbc43fff700 1 --2- 172.21.15.64:0/2443162789 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5641dd3ed0c0 0x5641dd3ef4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:32.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.721+0000 7fbc43fff700 1 --2- 172.21.15.64:0/2443162789 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5641dd3ed0c0 0x5641dd3ef4b0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66c5fd8a85fadb33 server_cookie=dbd349f9a9f1c23e in_seq=0 out_seq=0 2023-09-13T03:26:32.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.721+0000 7fbc2affd700 10 client.27187 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:26:32.941 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.940+0000 7fbc2affd700 1 -- 172.21.15.64:0/2443162789 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5641dd2c18c0 con 0x5641dd3ed0c0 2023-09-13T03:26:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.940+0000 7fbc2affd700 10 client.27187 handle_command_reply: tid=0 2023-09-13T03:26:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.940+0000 7fbc653d7b80 1 client.27187 shutdown 2023-09-13T03:26:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.940+0000 7fbc653d7b80 1 -- 172.21.15.64:0/2443162789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5641dd376300 msgr2=0x5641dd3852e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:26:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.940+0000 7fbc653d7b80 1 --2- 172.21.15.64:0/2443162789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5641dd376300 0x5641dd3852e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c01e000 tx=0x7fbc3c098f60 comp rx=0 tx=0).stop 2023-09-13T03:26:32.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.940+0000 7fbc653d7b80 1 -- 172.21.15.64:0/2443162789 shutdown_connections 2023-09-13T03:26:32.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.940+0000 7fbc653d7b80 1 --2- 172.21.15.64:0/2443162789 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5641dd3ed0c0 0x5641dd3ef4b0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:32.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.941+0000 7fbc2affd700 0 client.27187 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:26:32.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.940+0000 7fbc653d7b80 1 --2- 172.21.15.64:0/2443162789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641dd37c1a0 0x5641dd3e4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:32.943 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.941+0000 7fbc653d7b80 1 --2- 172.21.15.64:0/2443162789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5641dd376300 0x5641dd3852e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.941+0000 7fbc653d7b80 1 --2- 172.21.15.64:0/2443162789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641dd376cc0 0x5641dd385820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.941+0000 7fbc653d7b80 1 -- 172.21.15.64:0/2443162789 >> 172.21.15.64:0/2443162789 conn(0x5641dd2eb030 msgr2=0x5641dd3752f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:26:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.941+0000 7fbc653d7b80 1 -- 172.21.15.64:0/2443162789 shutdown_connections 2023-09-13T03:26:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.941+0000 7fbc653d7b80 1 -- 172.21.15.64:0/2443162789 wait complete. 2023-09-13T03:26:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.941+0000 7fbc653d7b80 20 client.27187 trim_cache size 0 max 16384 2023-09-13T03:26:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:26:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:26:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:26:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:26:32.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.942+0000 7fbc519ff700 1 -- 172.21.15.64:0/1123234263 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fbc380a07b0 msgr2=0x7fbc380a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:26:32.946 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.942+0000 7fbc519ff700 1 --2- 172.21.15.64:0/1123234263 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fbc380a07b0 0x7fbc380a2c70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbc4400b780 tx=0x7fbc440099e0 comp rx=0 tx=0).stop 2023-09-13T03:26:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.942+0000 7fbc519ff700 1 -- 172.21.15.64:0/1123234263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc4c107b40 msgr2=0x7fbc4c1a1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:26:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.942+0000 7fbc519ff700 1 --2- 172.21.15.64:0/1123234263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc4c107b40 0x7fbc4c1a1200 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c00e9f0 tx=0x7fbc3c00ed00 comp rx=0 tx=0).stop 2023-09-13T03:26:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.942+0000 7fbc519ff700 1 -- 172.21.15.64:0/1123234263 shutdown_connections 2023-09-13T03:26:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.942+0000 7fbc519ff700 1 --2- 172.21.15.64:0/1123234263 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fbc380a07b0 0x7fbc380a2c70 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.942+0000 7fbc519ff700 1 --2- 172.21.15.64:0/1123234263 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc4c115690 0x7fbc4c1a5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.942+0000 7fbc519ff700 1 --2- 172.21.15.64:0/1123234263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc4c107b40 0x7fbc4c1a1200 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.942+0000 7fbc519ff700 1 --2- 172.21.15.64:0/1123234263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc4c10ad10 0x7fbc4c1a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.942+0000 7fbc519ff700 1 -- 172.21.15.64:0/1123234263 >> 172.21.15.64:0/1123234263 conn(0x7fbc4c06cb00 msgr2=0x7fbc4c070fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:26:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.942+0000 7fbc519ff700 1 -- 172.21.15.64:0/1123234263 shutdown_connections 2023-09-13T03:26:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:32.942+0000 7fbc519ff700 1 -- 172.21.15.64:0/1123234263 wait complete. 2023-09-13T03:26:33.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:26:32 smithi170 ceph-mon[134328]: pgmap v418: 97 pgs: 97 active+clean; 449 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.4 MiB/s wr, 1.05k op/s 2023-09-13T03:26:33.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:26:32 smithi064 ceph-mon[97675]: pgmap v418: 97 pgs: 97 active+clean; 449 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.4 MiB/s wr, 1.05k op/s 2023-09-13T03:26:33.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:26:32 smithi142 ceph-mon[134082]: pgmap v418: 97 pgs: 97 active+clean; 449 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.4 MiB/s wr, 1.05k op/s 2023-09-13T03:26:33.282 INFO:tasks.workunit.client.0.smithi064.stdout: 10 24921 41.05 MB/sec warmup 37 sec latency 164.681 ms 2023-09-13T03:26:34.282 INFO:tasks.workunit.client.0.smithi064.stdout: 10 26145 41.77 MB/sec warmup 38 sec latency 48.487 ms 2023-09-13T03:26:35.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:26:34 smithi170 ceph-mon[134328]: pgmap v419: 97 pgs: 97 active+clean; 449 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 7.4 MiB/s wr, 693 op/s 2023-09-13T03:26:35.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:26:34 smithi064 ceph-mon[97675]: pgmap v419: 97 pgs: 97 active+clean; 449 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 7.4 MiB/s wr, 693 op/s 2023-09-13T03:26:35.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:26:34 smithi142 ceph-mon[134082]: pgmap v419: 97 pgs: 97 active+clean; 449 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 7.4 MiB/s wr, 693 op/s 2023-09-13T03:26:35.282 INFO:tasks.workunit.client.0.smithi064.stdout: 10 27160 41.84 MB/sec warmup 39 sec latency 78.575 ms 2023-09-13T03:26:36.283 INFO:tasks.workunit.client.0.smithi064.stdout: 10 28172 42.32 MB/sec warmup 40 sec latency 84.688 ms 2023-09-13T03:26:37.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:26:36 smithi170 ceph-mon[134328]: pgmap v420: 97 pgs: 97 active+clean; 484 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 12 MiB/s wr, 1.14k op/s 2023-09-13T03:26:37.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:26:36 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:26:37.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:26:36 smithi064 ceph-mon[97675]: pgmap v420: 97 pgs: 97 active+clean; 484 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 12 MiB/s wr, 1.14k op/s 2023-09-13T03:26:37.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:26:36 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:26:37.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:26:36 smithi142 ceph-mon[134082]: pgmap v420: 97 pgs: 97 active+clean; 484 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 12 MiB/s wr, 1.14k op/s 2023-09-13T03:26:37.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:26:36 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:26:37.283 INFO:tasks.workunit.client.0.smithi064.stdout: 10 29313 42.72 MB/sec warmup 41 sec latency 42.249 ms 2023-09-13T03:26:38.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:26:37 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:26:38.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:26:37 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:26:38.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:26:37 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:26:38.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:26:37 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:26:38.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:26:37 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:26:38.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:26:37 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:26:38.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:26:37 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:26:38.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:26:37 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:26:38.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:26:37 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:26:38.283 INFO:tasks.workunit.client.0.smithi064.stdout: 10 30423 43.23 MB/sec warmup 42 sec latency 34.573 ms 2023-09-13T03:26:39.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:26:38 smithi170 ceph-mon[134328]: pgmap v421: 97 pgs: 97 active+clean; 484 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.1 MiB/s wr, 796 op/s 2023-09-13T03:26:39.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:26:38 smithi064 ceph-mon[97675]: pgmap v421: 97 pgs: 97 active+clean; 484 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.1 MiB/s wr, 796 op/s 2023-09-13T03:26:39.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:26:38 smithi142 ceph-mon[134082]: pgmap v421: 97 pgs: 97 active+clean; 484 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.1 MiB/s wr, 796 op/s 2023-09-13T03:26:39.283 INFO:tasks.workunit.client.0.smithi064.stdout: 10 31438 43.43 MB/sec warmup 43 sec latency 64.521 ms 2023-09-13T03:26:40.283 INFO:tasks.workunit.client.0.smithi064.stdout: 10 32353 43.60 MB/sec warmup 44 sec latency 219.747 ms 2023-09-13T03:26:40.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:26:40 smithi142 ceph-mon[134082]: pgmap v422: 97 pgs: 97 active+clean; 484 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.1 MiB/s wr, 796 op/s 2023-09-13T03:26:40.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:26:40 smithi170 ceph-mon[134328]: pgmap v422: 97 pgs: 97 active+clean; 484 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.1 MiB/s wr, 796 op/s 2023-09-13T03:26:40.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:26:40 smithi064 ceph-mon[97675]: pgmap v422: 97 pgs: 97 active+clean; 484 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.1 MiB/s wr, 796 op/s 2023-09-13T03:26:41.283 INFO:tasks.workunit.client.0.smithi064.stdout: 10 33432 43.80 MB/sec warmup 45 sec latency 119.774 ms 2023-09-13T03:26:42.283 INFO:tasks.workunit.client.0.smithi064.stdout: 10 34573 44.29 MB/sec warmup 46 sec latency 45.084 ms 2023-09-13T03:26:43.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:26:42 smithi170 ceph-mon[134328]: pgmap v423: 97 pgs: 97 active+clean; 539 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 14 MiB/s wr, 1.13k op/s 2023-09-13T03:26:43.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:26:42 smithi064 ceph-mon[97675]: pgmap v423: 97 pgs: 97 active+clean; 539 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 14 MiB/s wr, 1.13k op/s 2023-09-13T03:26:43.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:26:42 smithi142 ceph-mon[134082]: pgmap v423: 97 pgs: 97 active+clean; 539 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 14 MiB/s wr, 1.13k op/s 2023-09-13T03:26:43.283 INFO:tasks.workunit.client.0.smithi064.stdout: 10 35554 44.45 MB/sec warmup 47 sec latency 66.095 ms 2023-09-13T03:26:44.283 INFO:tasks.workunit.client.0.smithi064.stdout: 10 36762 44.79 MB/sec warmup 48 sec latency 32.952 ms 2023-09-13T03:26:45.162 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:26:44 smithi170 ceph-mon[134328]: pgmap v424: 97 pgs: 97 active+clean; 539 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 9.9 MiB/s wr, 786 op/s 2023-09-13T03:26:45.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:26:44 smithi064 ceph-mon[97675]: pgmap v424: 97 pgs: 97 active+clean; 539 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 9.9 MiB/s wr, 786 op/s 2023-09-13T03:26:45.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:26:44 smithi142 ceph-mon[134082]: pgmap v424: 97 pgs: 97 active+clean; 539 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 9.9 MiB/s wr, 786 op/s 2023-09-13T03:26:45.283 INFO:tasks.workunit.client.0.smithi064.stdout: 10 37845 45.34 MB/sec warmup 49 sec latency 39.804 ms 2023-09-13T03:26:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.229+0000 7fefa95ff700 1 -- 172.21.15.64:0/2007640495 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefa41031a0 msgr2=0x7fefa410f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:26:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.229+0000 7fefa95ff700 1 --2- 172.21.15.64:0/2007640495 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefa41031a0 0x7fefa410f6b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fef9c00b0a0 tx=0x7fef9c00b3b0 comp rx=0 tx=0).stop 2023-09-13T03:26:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.229+0000 7fefa95ff700 1 -- 172.21.15.64:0/2007640495 shutdown_connections 2023-09-13T03:26:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.229+0000 7fefa95ff700 1 --2- 172.21.15.64:0/2007640495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefa41087f0 0x7fefa4108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.229+0000 7fefa95ff700 1 --2- 172.21.15.64:0/2007640495 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefa41031a0 0x7fefa410f6b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.229+0000 7fefa95ff700 1 --2- 172.21.15.64:0/2007640495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefa41027e0 0x7fefa4102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.229+0000 7fefa95ff700 1 -- 172.21.15.64:0/2007640495 >> 172.21.15.64:0/2007640495 conn(0x7fefa40fe2b0 msgr2=0x7fefa41006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:26:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.230+0000 7fefa95ff700 1 -- 172.21.15.64:0/2007640495 shutdown_connections 2023-09-13T03:26:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.230+0000 7fefa95ff700 1 -- 172.21.15.64:0/2007640495 wait complete. 2023-09-13T03:26:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.231+0000 7fefa95ff700 1 Processor -- start 2023-09-13T03:26:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.231+0000 7fefa95ff700 1 -- start start 2023-09-13T03:26:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.232+0000 7fefa95ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefa41027e0 0x7fefa419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.232+0000 7fefa95ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefa41031a0 0x7fefa419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.232+0000 7fefa95ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefa41087f0 0x7fefa4198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.232+0000 7fefa95ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa4198d00 con 0x7fefa41031a0 2023-09-13T03:26:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.232+0000 7fefa95ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa4198e70 con 0x7fefa41087f0 2023-09-13T03:26:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.232+0000 7fefa95ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa4199030 con 0x7fefa41027e0 2023-09-13T03:26:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.232+0000 7fefa37fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefa41087f0 0x7fefa4198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.232+0000 7fefa2ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefa41027e0 0x7fefa419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.232+0000 7fefa37fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefa41087f0 0x7fefa4198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:45158/0 (socket says 172.21.15.64:45158) 2023-09-13T03:26:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.232+0000 7fefa37fe700 1 -- 172.21.15.64:0/3741327045 learned_addr learned my addr 172.21.15.64:0/3741327045 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:26:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.232+0000 7fefa27fc700 1 --2- 172.21.15.64:0/3741327045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefa41031a0 0x7fefa419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.232+0000 7fefa37fe700 1 -- 172.21.15.64:0/3741327045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefa41027e0 msgr2=0x7fefa419e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:26:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.232+0000 7fefa37fe700 1 --2- 172.21.15.64:0/3741327045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefa41027e0 0x7fefa419e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.232+0000 7fefa37fe700 1 -- 172.21.15.64:0/3741327045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefa41031a0 msgr2=0x7fefa419ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:26:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.232+0000 7fefa37fe700 1 --2- 172.21.15.64:0/3741327045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefa41031a0 0x7fefa419ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.232+0000 7fefa37fe700 1 -- 172.21.15.64:0/3741327045 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef9c009d00 con 0x7fefa41087f0 2023-09-13T03:26:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.232+0000 7fefa27fc700 1 --2- 172.21.15.64:0/3741327045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefa41031a0 0x7fefa419ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T03:26:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.233+0000 7fefa37fe700 1 --2- 172.21.15.64:0/3741327045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefa41087f0 0x7fefa4198690 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fef9c009fd0 tx=0x7fef9c007b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:26:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.233+0000 7fef8bfff700 1 -- 172.21.15.64:0/3741327045 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9c00e040 con 0x7fefa41087f0 2023-09-13T03:26:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.233+0000 7fef8bfff700 1 -- 172.21.15.64:0/3741327045 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef9c012dc0 con 0x7fefa41087f0 2023-09-13T03:26:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.233+0000 7fef8bfff700 1 -- 172.21.15.64:0/3741327045 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9c01b450 con 0x7fefa41087f0 2023-09-13T03:26:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.233+0000 7fefa95ff700 1 -- 172.21.15.64:0/3741327045 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefa41992b0 con 0x7fefa41087f0 2023-09-13T03:26:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.233+0000 7fefa95ff700 1 -- 172.21.15.64:0/3741327045 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefa4074c20 con 0x7fefa41087f0 2023-09-13T03:26:46.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.236+0000 7fef8bfff700 1 -- 172.21.15.64:0/3741327045 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fef9c012900 con 0x7fefa41087f0 2023-09-13T03:26:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.238+0000 7fef8bfff700 1 --2- 172.21.15.64:0/3741327045 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fef8c0a07b0 0x7fef8c0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.238+0000 7fefbcedcb80 1 Processor -- start 2023-09-13T03:26:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.238+0000 7fef8bfff700 1 -- 172.21.15.64:0/3741327045 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fef9c0cd880 con 0x7fefa41087f0 2023-09-13T03:26:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.238+0000 7fefbcedcb80 1 -- start start 2023-09-13T03:26:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.238+0000 7fefa2ffd700 1 --2- 172.21.15.64:0/3741327045 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fef8c0a07b0 0x7fef8c0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:46.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.239+0000 7fefbcedcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b4b8c5c1a0 0x55b4b8c5c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:46.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.239+0000 7fefbcedcb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4b8c56300 0x55b4b8c56780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:46.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.239+0000 7fefbcedcb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4b8c56cc0 0x55b4b8c62f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:46.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.239+0000 7fefbcedcb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4b8baf6d0 con 0x55b4b8c56300 2023-09-13T03:26:46.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.239+0000 7fefbcedcb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4b8bacff0 con 0x55b4b8c56cc0 2023-09-13T03:26:46.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.239+0000 7fefbcedcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4b8ba88e0 con 0x55b4b8c5c1a0 2023-09-13T03:26:46.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.239+0000 7fefa37fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4b8c56cc0 0x55b4b8c62f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.239+0000 7fefa37fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4b8c56cc0 0x55b4b8c62f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:45166/0 (socket says 172.21.15.64:45166) 2023-09-13T03:26:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.239+0000 7fefa37fe700 1 -- 172.21.15.64:0/1648460165 learned_addr learned my addr 172.21.15.64:0/1648460165 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:26:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.239+0000 7fefa2ffd700 1 --2- 172.21.15.64:0/3741327045 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fef8c0a07b0 0x7fef8c0a2c70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fef94005950 tx=0x7fef9400b410 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:26:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.239+0000 7fefa27fc700 1 --2- 172.21.15.64:0/1648460165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4b8c56300 0x55b4b8c56780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.239+0000 7fefa2ffd700 1 --2- 172.21.15.64:0/1648460165 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b4b8c5c1a0 0x55b4b8c5c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.239+0000 7fefa37fe700 1 -- 172.21.15.64:0/1648460165 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b4b8c5c1a0 msgr2=0x55b4b8c5c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:26:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.239+0000 7fefa37fe700 1 --2- 172.21.15.64:0/1648460165 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b4b8c5c1a0 0x55b4b8c5c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.239+0000 7fefa37fe700 1 -- 172.21.15.64:0/1648460165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4b8c56300 msgr2=0x55b4b8c56780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:26:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.239+0000 7fefa37fe700 1 --2- 172.21.15.64:0/1648460165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4b8c56300 0x55b4b8c56780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.239+0000 7fefa37fe700 1 -- 172.21.15.64:0/1648460165 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4b89d4d80 con 0x55b4b8c56cc0 2023-09-13T03:26:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.239+0000 7fefa27fc700 1 --2- 172.21.15.64:0/1648460165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4b8c56300 0x55b4b8c56780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:26:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.240+0000 7fefa37fe700 1 --2- 172.21.15.64:0/1648460165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4b8c56cc0 0x55b4b8c62f30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fef9c0977f0 tx=0x7fef9c0099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cb939d8f6271f9e server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:26:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.240+0000 7fef897fa700 1 -- 172.21.15.64:0/1648460165 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9c019350 con 0x55b4b8c56cc0 2023-09-13T03:26:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.240+0000 7fef897fa700 1 -- 172.21.15.64:0/1648460165 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef9c09b480 con 0x55b4b8c56cc0 2023-09-13T03:26:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.240+0000 7fef897fa700 1 -- 172.21.15.64:0/1648460165 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9c097490 con 0x55b4b8c56cc0 2023-09-13T03:26:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.240+0000 7fefbcedcb80 1 -- 172.21.15.64:0/1648460165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4b8c56cc0 msgr2=0x55b4b8c62f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:26:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.240+0000 7fefbcedcb80 1 --2- 172.21.15.64:0/1648460165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4b8c56cc0 0x55b4b8c62f30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fef9c0977f0 tx=0x7fef9c0099e0 comp rx=0 tx=0).stop 2023-09-13T03:26:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.241+0000 7fefbcedcb80 1 -- 172.21.15.64:0/1648460165 shutdown_connections 2023-09-13T03:26:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.241+0000 7fefbcedcb80 1 --2- 172.21.15.64:0/1648460165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4b8c56cc0 0x55b4b8c62f30 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.241+0000 7fefbcedcb80 1 --2- 172.21.15.64:0/1648460165 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b4b8c5c1a0 0x55b4b8c5c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.241+0000 7fefbcedcb80 1 --2- 172.21.15.64:0/1648460165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4b8c56300 0x55b4b8c56780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.241+0000 7fefbcedcb80 1 -- 172.21.15.64:0/1648460165 >> 172.21.15.64:0/1648460165 conn(0x55b4b8bcb030 msgr2=0x55b4b8c51560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:26:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.241+0000 7fefbcedcb80 1 -- 172.21.15.64:0/1648460165 shutdown_connections 2023-09-13T03:26:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.241+0000 7fefbcedcb80 1 -- 172.21.15.64:0/1648460165 wait complete. 2023-09-13T03:26:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.241+0000 7fefbcedcb80 1 -- start start 2023-09-13T03:26:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.242+0000 7fefbcedcb80 1 Processor -- start 2023-09-13T03:26:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.243+0000 7fefbcedcb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4b8c56300 0x55b4b8c65d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.243+0000 7fefbcedcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b4b8c56cc0 0x55b4b8c66240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.243+0000 7fefbcedcb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4b8c5c1a0 0x55b4b8c60000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.243+0000 7fefbcedcb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4b8b46b10 con 0x55b4b8c56300 2023-09-13T03:26:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.243+0000 7fefbcedcb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4b8b46810 con 0x55b4b8c5c1a0 2023-09-13T03:26:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.243+0000 7fefbcedcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4b8a2a8c0 con 0x55b4b8c56cc0 2023-09-13T03:26:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.243+0000 7fefa2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4b8c56300 0x55b4b8c65d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.243+0000 7fefbcedcb80 10 client.0 _ll_register_callbacks cb 0x55b4b8a90a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:26:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.243+0000 7fefa2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4b8c56300 0x55b4b8c65d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60886/0 (socket says 172.21.15.64:60886) 2023-09-13T03:26:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.243+0000 7fefa2ffd700 1 -- 172.21.15.64:0/3793364095 learned_addr learned my addr 172.21.15.64:0/3793364095 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:26:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.243+0000 7fefa37fe700 1 --2- 172.21.15.64:0/3793364095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4b8c5c1a0 0x55b4b8c60000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.243+0000 7fefa27fc700 1 --2- 172.21.15.64:0/3793364095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b4b8c56cc0 0x55b4b8c66240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.243+0000 7fefa2ffd700 1 -- 172.21.15.64:0/3793364095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b4b8c56cc0 msgr2=0x55b4b8c66240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:26:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.243+0000 7fefa2ffd700 1 --2- 172.21.15.64:0/3793364095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b4b8c56cc0 0x55b4b8c66240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.243+0000 7fefa2ffd700 1 -- 172.21.15.64:0/3793364095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4b8c5c1a0 msgr2=0x55b4b8c60000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:26:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.243+0000 7fefa2ffd700 1 --2- 172.21.15.64:0/3793364095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4b8c5c1a0 0x55b4b8c60000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.243+0000 7fefa2ffd700 1 -- 172.21.15.64:0/3793364095 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefa4074c20 con 0x55b4b8c56300 2023-09-13T03:26:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.243+0000 7fefa27fc700 1 --2- 172.21.15.64:0/3793364095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b4b8c56cc0 0x55b4b8c66240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:26:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.244+0000 7fefa2ffd700 1 --2- 172.21.15.64:0/3793364095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4b8c56300 0x55b4b8c65d00 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fef940116c0 tx=0x7fef940047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9da4fecd004e1df server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:26:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.244+0000 7fef86ff5700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:26:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.244+0000 7fef86ff5700 1 -- 172.21.15.64:0/3793364095 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef94004a30 con 0x55b4b8c56300 2023-09-13T03:26:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.244+0000 7fefbcedcb80 1 -- 172.21.15.64:0/3793364095 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b4b8b00490 con 0x55b4b8c56300 2023-09-13T03:26:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.244+0000 7fef86ff5700 1 -- 172.21.15.64:0/3793364095 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fef94017070 con 0x55b4b8c56300 2023-09-13T03:26:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.244+0000 7fef86ff5700 1 -- 172.21.15.64:0/3793364095 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef94004b90 con 0x55b4b8c56300 2023-09-13T03:26:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.244+0000 7fefbcedcb80 10 client.17163 fetch_fsmap learned FSMap version 70 2023-09-13T03:26:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.244+0000 7fefbcedcb80 1 -- 172.21.15.64:0/3793364095 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x55b4b89bdc00 con 0x55b4b8c56300 2023-09-13T03:26:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.244+0000 7fef86ff5700 1 -- 172.21.15.64:0/3793364095 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b4b8b00490 con 0x55b4b8c56300 2023-09-13T03:26:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.245+0000 7fef86ff5700 1 -- 172.21.15.64:0/3793364095 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 70) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fef9401f6f0 con 0x55b4b8c56300 2023-09-13T03:26:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.245+0000 7fefbcedcb80 10 client.17163 fetch_fsmap finished waiting for FSMap version 70 2023-09-13T03:26:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.245+0000 7fefbcedcb80 10 client.17163 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:26:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.245+0000 7fefbcedcb80 20 client.17163 populate_metadata read hostname 'smithi064' 2023-09-13T03:26:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.245+0000 7fefbcedcb80 1 --2- 172.21.15.64:0/3793364095 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55b4b8c813c0 0x55b4b8c837e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.245+0000 7fefbcedcb80 4 client.17163 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:26:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.245+0000 7fefbcedcb80 1 -- 172.21.15.64:0/3793364095 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b4b8bbd840 con 0x55b4b8c813c0 2023-09-13T03:26:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.245+0000 7fefa27fc700 1 --2- 172.21.15.64:0/3793364095 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55b4b8c813c0 0x55b4b8c837e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.246+0000 7fefa27fc700 1 --2- 172.21.15.64:0/3793364095 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55b4b8c813c0 0x55b4b8c837e0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ea2b3599cd15448 server_cookie=f89b081531f727dc in_seq=0 out_seq=0 2023-09-13T03:26:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.246+0000 7fef86ff5700 10 client.17163 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:26:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.246+0000 7fef86ff5700 1 -- 172.21.15.64:0/3793364095 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b4b8bbd840 con 0x55b4b8c813c0 2023-09-13T03:26:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.246+0000 7fef86ff5700 10 client.17163 handle_command_reply: tid=0 2023-09-13T03:26:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.246+0000 7fefbcedcb80 1 client.17163 shutdown 2023-09-13T03:26:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.246+0000 7fefbcedcb80 1 -- 172.21.15.64:0/3793364095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4b8c56300 msgr2=0x55b4b8c65d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:26:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.246+0000 7fefbcedcb80 1 --2- 172.21.15.64:0/3793364095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4b8c56300 0x55b4b8c65d00 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fef940116c0 tx=0x7fef940047c0 comp rx=0 tx=0).stop 2023-09-13T03:26:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.247+0000 7fefbcedcb80 1 -- 172.21.15.64:0/3793364095 shutdown_connections 2023-09-13T03:26:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.247+0000 7fefbcedcb80 1 --2- 172.21.15.64:0/3793364095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4b8c5c1a0 0x55b4b8c60000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.247+0000 7fefbcedcb80 1 --2- 172.21.15.64:0/3793364095 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55b4b8c813c0 0x55b4b8c837e0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.247+0000 7fefbcedcb80 1 --2- 172.21.15.64:0/3793364095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4b8c56300 0x55b4b8c65d00 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.247+0000 7fefbcedcb80 1 --2- 172.21.15.64:0/3793364095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b4b8c56cc0 0x55b4b8c66240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.247+0000 7fef86ff5700 0 client.17163 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:26:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.247+0000 7fefbcedcb80 1 -- 172.21.15.64:0/3793364095 >> 172.21.15.64:0/3793364095 conn(0x55b4b8bcb030 msgr2=0x55b4b8c507a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:26:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.247+0000 7fefbcedcb80 1 -- 172.21.15.64:0/3793364095 shutdown_connections 2023-09-13T03:26:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.247+0000 7fefbcedcb80 1 -- 172.21.15.64:0/3793364095 wait complete. 2023-09-13T03:26:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.247+0000 7fefbcedcb80 20 client.17163 trim_cache size 0 max 16384 2023-09-13T03:26:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.262+0000 7fefbcedcb80 1 Processor -- start 2023-09-13T03:26:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.263+0000 7fefbcedcb80 1 -- start start 2023-09-13T03:26:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.263+0000 7fefbcedcb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4b8c56300 0x55b4b8c51710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.263+0000 7fefbcedcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b4b8c56cc0 0x55b4b8c5dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.263+0000 7fefbcedcb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4b8c5c1a0 0x55b4b8c60130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.263+0000 7fefbcedcb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4b8b00490 con 0x55b4b8c5c1a0 2023-09-13T03:26:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.263+0000 7fefbcedcb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4b8c4b8a0 con 0x55b4b8c56300 2023-09-13T03:26:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.263+0000 7fefbcedcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4b8c59db0 con 0x55b4b8c56cc0 2023-09-13T03:26:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.263+0000 7fefa27fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b4b8c56cc0 0x55b4b8c5dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.263+0000 7fefa37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4b8c5c1a0 0x55b4b8c60130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.263+0000 7fefa27fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b4b8c56cc0 0x55b4b8c5dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:52738/0 (socket says 172.21.15.64:52738) 2023-09-13T03:26:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.263+0000 7fefa27fc700 1 -- 172.21.15.64:0/296957838 learned_addr learned my addr 172.21.15.64:0/296957838 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:26:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.263+0000 7fefa2ffd700 1 --2- 172.21.15.64:0/296957838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4b8c56300 0x55b4b8c51710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.264+0000 7fefa27fc700 1 -- 172.21.15.64:0/296957838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4b8c56300 msgr2=0x55b4b8c51710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:26:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.264+0000 7fefa27fc700 1 --2- 172.21.15.64:0/296957838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4b8c56300 0x55b4b8c51710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.264+0000 7fefa27fc700 1 -- 172.21.15.64:0/296957838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4b8c5c1a0 msgr2=0x55b4b8c60130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:26:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.264+0000 7fefa27fc700 1 --2- 172.21.15.64:0/296957838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4b8c5c1a0 0x55b4b8c60130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.264+0000 7fefa27fc700 1 -- 172.21.15.64:0/296957838 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4b8c883d0 con 0x55b4b8c56cc0 2023-09-13T03:26:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.264+0000 7fefa37fe700 1 --2- 172.21.15.64:0/296957838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4b8c5c1a0 0x55b4b8c60130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:26:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.264+0000 7fefa27fc700 1 --2- 172.21.15.64:0/296957838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b4b8c56cc0 0x55b4b8c5dc50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fefa4108010 tx=0x7fef98015700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cc6d60971031731 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:26:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.264+0000 7fef89ffb700 1 -- 172.21.15.64:0/296957838 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef98015b00 con 0x55b4b8c56cc0 2023-09-13T03:26:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.264+0000 7fef89ffb700 1 -- 172.21.15.64:0/296957838 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef9800c040 con 0x55b4b8c56cc0 2023-09-13T03:26:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.264+0000 7fef89ffb700 1 -- 172.21.15.64:0/296957838 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef980205f0 con 0x55b4b8c56cc0 2023-09-13T03:26:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.265+0000 7fefbcedcb80 1 -- 172.21.15.64:0/296957838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b4b8c56cc0 msgr2=0x55b4b8c5dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:26:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.265+0000 7fefbcedcb80 1 --2- 172.21.15.64:0/296957838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b4b8c56cc0 0x55b4b8c5dc50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fefa4108010 tx=0x7fef98015700 comp rx=0 tx=0).stop 2023-09-13T03:26:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.265+0000 7fefbcedcb80 1 -- 172.21.15.64:0/296957838 shutdown_connections 2023-09-13T03:26:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.265+0000 7fefbcedcb80 1 --2- 172.21.15.64:0/296957838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4b8c56300 0x55b4b8c51710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.265+0000 7fefbcedcb80 1 --2- 172.21.15.64:0/296957838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4b8c5c1a0 0x55b4b8c60130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.265+0000 7fefbcedcb80 1 --2- 172.21.15.64:0/296957838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b4b8c56cc0 0x55b4b8c5dc50 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.265+0000 7fefbcedcb80 1 -- 172.21.15.64:0/296957838 >> 172.21.15.64:0/296957838 conn(0x55b4b8bcb030 msgr2=0x55b4b8c552f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:26:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.265+0000 7fefbcedcb80 1 -- 172.21.15.64:0/296957838 shutdown_connections 2023-09-13T03:26:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.265+0000 7fefbcedcb80 1 -- 172.21.15.64:0/296957838 wait complete. 2023-09-13T03:26:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.265+0000 7fefbcedcb80 1 -- start start 2023-09-13T03:26:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.266+0000 7fefbcedcb80 1 Processor -- start 2023-09-13T03:26:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.266+0000 7fefbcedcb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4b8c56300 0x55b4b8ccf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.266+0000 7fefbcedcb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4b8c56cc0 0x55b4b8ccf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.266+0000 7fefbcedcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b4b8c5c1a0 0x55b4b8cc9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.266+0000 7fefbcedcb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4b8cc9960 con 0x55b4b8c56300 2023-09-13T03:26:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.266+0000 7fefbcedcb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4b8cc9ad0 con 0x55b4b8c56cc0 2023-09-13T03:26:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.266+0000 7fefbcedcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4b8cc9c90 con 0x55b4b8c5c1a0 2023-09-13T03:26:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.266+0000 7fefbcedcb80 10 client.0 _ll_register_callbacks cb 0x7fef94010d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:26:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.266+0000 7fefa2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4b8c56300 0x55b4b8ccf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.266+0000 7fefa27fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4b8c56cc0 0x55b4b8ccf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.266+0000 7fefa27fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4b8c56cc0 0x55b4b8ccf560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:45196/0 (socket says 172.21.15.64:45196) 2023-09-13T03:26:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.267+0000 7fefa27fc700 1 -- 172.21.15.64:0/2276080519 learned_addr learned my addr 172.21.15.64:0/2276080519 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:26:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.266+0000 7fefa37fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b4b8c5c1a0 0x55b4b8cc9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.267+0000 7fefa2ffd700 1 -- 172.21.15.64:0/2276080519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b4b8c5c1a0 msgr2=0x55b4b8cc9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:26:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.267+0000 7fefa2ffd700 1 --2- 172.21.15.64:0/2276080519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b4b8c5c1a0 0x55b4b8cc9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.267+0000 7fefa2ffd700 1 -- 172.21.15.64:0/2276080519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4b8c56cc0 msgr2=0x55b4b8ccf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:26:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.267+0000 7fefa2ffd700 1 --2- 172.21.15.64:0/2276080519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4b8c56cc0 0x55b4b8ccf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.267+0000 7fefa2ffd700 1 -- 172.21.15.64:0/2276080519 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b4b89d4d80 con 0x55b4b8c56300 2023-09-13T03:26:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.267+0000 7fefa27fc700 1 --2- 172.21.15.64:0/2276080519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4b8c56cc0 0x55b4b8ccf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:26:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.267+0000 7fefa37fe700 1 --2- 172.21.15.64:0/2276080519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b4b8c5c1a0 0x55b4b8cc9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:26:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.267+0000 7fefa2ffd700 1 --2- 172.21.15.64:0/2276080519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4b8c56300 0x55b4b8ccf020 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fef9401fa70 tx=0x7fef9400b500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=966152d545879803 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:26:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.267+0000 7fef86ff5700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:26:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.267+0000 7fef86ff5700 1 -- 172.21.15.64:0/2276080519 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9401f840 con 0x55b4b8c56300 2023-09-13T03:26:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.267+0000 7fefbcedcb80 1 -- 172.21.15.64:0/2276080519 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fef9c09b480 con 0x55b4b8c56300 2023-09-13T03:26:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.267+0000 7fef86ff5700 1 -- 172.21.15.64:0/2276080519 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fef94017070 con 0x55b4b8c56300 2023-09-13T03:26:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.268+0000 7fef86ff5700 1 -- 172.21.15.64:0/2276080519 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef94004830 con 0x55b4b8c56300 2023-09-13T03:26:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.268+0000 7fefbcedcb80 10 client.17169 fetch_fsmap learned FSMap version 70 2023-09-13T03:26:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.268+0000 7fefbcedcb80 1 -- 172.21.15.64:0/2276080519 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x7fef9800abe0 con 0x55b4b8c56300 2023-09-13T03:26:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.268+0000 7fef86ff5700 1 -- 172.21.15.64:0/2276080519 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef94045020 con 0x55b4b8c56300 2023-09-13T03:26:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.268+0000 7fef86ff5700 1 -- 172.21.15.64:0/2276080519 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 70) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fef94027420 con 0x55b4b8c56300 2023-09-13T03:26:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.268+0000 7fefbcedcb80 10 client.17169 fetch_fsmap finished waiting for FSMap version 70 2023-09-13T03:26:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.268+0000 7fefbcedcb80 10 client.17169 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:26:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.268+0000 7fefbcedcb80 20 client.17169 populate_metadata read hostname 'smithi064' 2023-09-13T03:26:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.268+0000 7fefbcedcb80 1 --2- 172.21.15.64:0/2276080519 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55b4b8cd6f20 0x55b4b8cd9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.268+0000 7fefbcedcb80 4 client.17169 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:26:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.268+0000 7fefbcedcb80 1 -- 172.21.15.64:0/2276080519 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b4b8ba18c0 con 0x55b4b8cd6f20 2023-09-13T03:26:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.269+0000 7fefa27fc700 1 --2- 172.21.15.64:0/2276080519 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55b4b8cd6f20 0x55b4b8cd9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.269+0000 7fefa27fc700 1 --2- 172.21.15.64:0/2276080519 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55b4b8cd6f20 0x55b4b8cd9340 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cbf54e8b7c32c16 server_cookie=c02065381d2dc9b in_seq=0 out_seq=0 2023-09-13T03:26:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.269+0000 7fef86ff5700 10 client.17169 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:26:46.283 INFO:tasks.workunit.client.0.smithi064.stdout: 10 38216 44.74 MB/sec warmup 50 sec latency 515.738 ms 2023-09-13T03:26:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.599+0000 7fef86ff5700 1 -- 172.21.15.64:0/2276080519 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b4b8ba18c0 con 0x55b4b8cd6f20 2023-09-13T03:26:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.599+0000 7fef86ff5700 10 client.17169 handle_command_reply: tid=0 2023-09-13T03:26:46.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.599+0000 7fefbcedcb80 1 client.17169 shutdown 2023-09-13T03:26:46.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.600+0000 7fefbcedcb80 1 -- 172.21.15.64:0/2276080519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4b8c56300 msgr2=0x55b4b8ccf020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:26:46.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.600+0000 7fefbcedcb80 1 --2- 172.21.15.64:0/2276080519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4b8c56300 0x55b4b8ccf020 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fef9401fa70 tx=0x7fef9400b500 comp rx=0 tx=0).stop 2023-09-13T03:26:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.600+0000 7fefbcedcb80 1 -- 172.21.15.64:0/2276080519 shutdown_connections 2023-09-13T03:26:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.600+0000 7fefbcedcb80 1 --2- 172.21.15.64:0/2276080519 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55b4b8cd6f20 0x55b4b8cd9340 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.600+0000 7fefbcedcb80 1 --2- 172.21.15.64:0/2276080519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b4b8c5c1a0 0x55b4b8cc9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.600+0000 7fefbcedcb80 1 --2- 172.21.15.64:0/2276080519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4b8c56300 0x55b4b8ccf020 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.600+0000 7fef86ff5700 0 client.17169 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:26:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.600+0000 7fefbcedcb80 1 --2- 172.21.15.64:0/2276080519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4b8c56cc0 0x55b4b8ccf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.600+0000 7fefbcedcb80 1 -- 172.21.15.64:0/2276080519 >> 172.21.15.64:0/2276080519 conn(0x55b4b8bcb030 msgr2=0x55b4b8c55b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:26:46.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.600+0000 7fefbcedcb80 1 -- 172.21.15.64:0/2276080519 shutdown_connections 2023-09-13T03:26:46.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.601+0000 7fefbcedcb80 1 -- 172.21.15.64:0/2276080519 wait complete. 2023-09-13T03:26:46.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.601+0000 7fefbcedcb80 20 client.17169 trim_cache size 0 max 16384 2023-09-13T03:26:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:26:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:26:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:26:46.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:26:46.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.601+0000 7fefa95ff700 1 -- 172.21.15.64:0/3741327045 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fef8c0a07b0 msgr2=0x7fef8c0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:26:46.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.601+0000 7fefa95ff700 1 --2- 172.21.15.64:0/3741327045 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fef8c0a07b0 0x7fef8c0a2c70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fef94005950 tx=0x7fef9400b410 comp rx=0 tx=0).stop 2023-09-13T03:26:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.602+0000 7fefa95ff700 1 -- 172.21.15.64:0/3741327045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefa41087f0 msgr2=0x7fefa4198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:26:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.602+0000 7fefa95ff700 1 --2- 172.21.15.64:0/3741327045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefa41087f0 0x7fefa4198690 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fef9c009fd0 tx=0x7fef9c007b60 comp rx=0 tx=0).stop 2023-09-13T03:26:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.602+0000 7fefa95ff700 1 -- 172.21.15.64:0/3741327045 shutdown_connections 2023-09-13T03:26:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.602+0000 7fefa95ff700 1 --2- 172.21.15.64:0/3741327045 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fef8c0a07b0 0x7fef8c0a2c70 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.602+0000 7fefa95ff700 1 --2- 172.21.15.64:0/3741327045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefa41087f0 0x7fefa4198690 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.602+0000 7fefa95ff700 1 --2- 172.21.15.64:0/3741327045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefa41027e0 0x7fefa419e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.602+0000 7fefa95ff700 1 --2- 172.21.15.64:0/3741327045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefa41031a0 0x7fefa419ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.602+0000 7fefa95ff700 1 -- 172.21.15.64:0/3741327045 >> 172.21.15.64:0/3741327045 conn(0x7fefa40fe2b0 msgr2=0x7fefa40ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:26:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.602+0000 7fefa95ff700 1 -- 172.21.15.64:0/3741327045 shutdown_connections 2023-09-13T03:26:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:46.602+0000 7fefa95ff700 1 -- 172.21.15.64:0/3741327045 wait complete. 2023-09-13T03:26:47.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:26:46 smithi170 ceph-mon[134328]: pgmap v425: 97 pgs: 97 active+clean; 552 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 1.12k op/s 2023-09-13T03:26:47.170 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:26:46 smithi064 ceph-mon[97675]: pgmap v425: 97 pgs: 97 active+clean; 552 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 1.12k op/s 2023-09-13T03:26:47.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:26:46 smithi142 ceph-mon[134082]: pgmap v425: 97 pgs: 97 active+clean; 552 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 1.12k op/s 2023-09-13T03:26:47.283 INFO:tasks.workunit.client.0.smithi064.stdout: 10 38911 44.42 MB/sec warmup 51 sec latency 235.455 ms 2023-09-13T03:26:48.283 INFO:tasks.workunit.client.0.smithi064.stdout: 10 40149 44.89 MB/sec warmup 52 sec latency 52.466 ms 2023-09-13T03:26:49.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:26:48 smithi170 ceph-mon[134328]: pgmap v426: 97 pgs: 97 active+clean; 552 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 8.8 MiB/s wr, 670 op/s 2023-09-13T03:26:49.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:26:48 smithi064 ceph-mon[97675]: pgmap v426: 97 pgs: 97 active+clean; 552 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 8.8 MiB/s wr, 670 op/s 2023-09-13T03:26:49.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:26:48 smithi142 ceph-mon[134082]: pgmap v426: 97 pgs: 97 active+clean; 552 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 8.8 MiB/s wr, 670 op/s 2023-09-13T03:26:49.284 INFO:tasks.workunit.client.0.smithi064.stdout: 10 41262 45.32 MB/sec warmup 53 sec latency 44.335 ms 2023-09-13T03:26:50.284 INFO:tasks.workunit.client.0.smithi064.stdout: 10 42306 45.44 MB/sec warmup 54 sec latency 56.637 ms 2023-09-13T03:26:50.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:26:50 smithi142 ceph-mon[134082]: pgmap v427: 97 pgs: 97 active+clean; 552 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 8.8 MiB/s wr, 670 op/s 2023-09-13T03:26:50.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:26:50 smithi170 ceph-mon[134328]: pgmap v427: 97 pgs: 97 active+clean; 552 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 8.8 MiB/s wr, 670 op/s 2023-09-13T03:26:50.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:26:50 smithi064 ceph-mon[97675]: pgmap v427: 97 pgs: 97 active+clean; 552 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 8.8 MiB/s wr, 670 op/s 2023-09-13T03:26:51.284 INFO:tasks.workunit.client.0.smithi064.stdout: 10 43501 45.86 MB/sec warmup 55 sec latency 24.633 ms 2023-09-13T03:26:52.284 INFO:tasks.workunit.client.0.smithi064.stdout: 10 43548 45.11 MB/sec warmup 56 sec latency 741.266 ms 2023-09-13T03:26:53.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:26:52 smithi170 ceph-mon[134328]: pgmap v428: 97 pgs: 97 active+clean; 563 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 17 MiB/s wr, 1.23k op/s 2023-09-13T03:26:53.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:26:52 smithi064 ceph-mon[97675]: pgmap v428: 97 pgs: 97 active+clean; 563 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 17 MiB/s wr, 1.23k op/s 2023-09-13T03:26:53.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:26:52 smithi142 ceph-mon[134082]: pgmap v428: 97 pgs: 97 active+clean; 563 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 17 MiB/s wr, 1.23k op/s 2023-09-13T03:26:53.284 INFO:tasks.workunit.client.0.smithi064.stdout: 10 43848 44.52 MB/sec warmup 57 sec latency 822.681 ms 2023-09-13T03:26:54.285 INFO:tasks.workunit.client.0.smithi064.stdout: 10 44481 44.31 MB/sec warmup 58 sec latency 145.854 ms 2023-09-13T03:26:55.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:26:54 smithi170 ceph-mon[134328]: pgmap v429: 97 pgs: 97 active+clean; 563 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 12 MiB/s wr, 894 op/s 2023-09-13T03:26:55.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:26:54 smithi064 ceph-mon[97675]: pgmap v429: 97 pgs: 97 active+clean; 563 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 12 MiB/s wr, 894 op/s 2023-09-13T03:26:55.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:26:54 smithi142 ceph-mon[134082]: pgmap v429: 97 pgs: 97 active+clean; 563 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 12 MiB/s wr, 894 op/s 2023-09-13T03:26:55.285 INFO:tasks.workunit.client.0.smithi064.stdout: 10 45464 44.55 MB/sec warmup 59 sec latency 68.124 ms 2023-09-13T03:26:56.285 INFO:tasks.workunit.client.0.smithi064.stdout: 10 46542 44.67 MB/sec warmup 60 sec latency 50.461 ms 2023-09-13T03:26:57.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:26:56 smithi170 ceph-mon[134328]: pgmap v430: 97 pgs: 97 active+clean; 560 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 32 MiB/s wr, 1.71k op/s 2023-09-13T03:26:57.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:26:56 smithi064 ceph-mon[97675]: pgmap v430: 97 pgs: 97 active+clean; 560 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 32 MiB/s wr, 1.71k op/s 2023-09-13T03:26:57.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:26:56 smithi142 ceph-mon[134082]: pgmap v430: 97 pgs: 97 active+clean; 560 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 32 MiB/s wr, 1.71k op/s 2023-09-13T03:26:57.285 INFO:tasks.workunit.client.0.smithi064.stdout: 10 47626 44.95 MB/sec warmup 61 sec latency 58.859 ms 2023-09-13T03:26:58.285 INFO:tasks.workunit.client.0.smithi064.stdout: 10 48771 45.12 MB/sec warmup 62 sec latency 41.939 ms 2023-09-13T03:26:59.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:26:58 smithi170 ceph-mon[134328]: pgmap v431: 97 pgs: 97 active+clean; 560 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 29 MiB/s wr, 1.38k op/s 2023-09-13T03:26:59.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:26:58 smithi064 ceph-mon[97675]: pgmap v431: 97 pgs: 97 active+clean; 560 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 29 MiB/s wr, 1.38k op/s 2023-09-13T03:26:59.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:26:58 smithi142 ceph-mon[134082]: pgmap v431: 97 pgs: 97 active+clean; 560 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 29 MiB/s wr, 1.38k op/s 2023-09-13T03:26:59.285 INFO:tasks.workunit.client.0.smithi064.stdout: 10 49973 45.55 MB/sec warmup 63 sec latency 40.848 ms 2023-09-13T03:26:59.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.793+0000 7fb0c9fff700 1 -- 172.21.15.64:0/65709580 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0c4106650 msgr2=0x7fb0c4106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:26:59.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.793+0000 7fb0c9fff700 1 --2- 172.21.15.64:0/65709580 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0c4106650 0x7fb0c4106a30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4009990 tx=0x7fb0b4009ca0 comp rx=0 tx=0).stop 2023-09-13T03:26:59.794 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.793+0000 7fb0c9fff700 1 -- 172.21.15.64:0/65709580 shutdown_connections 2023-09-13T03:26:59.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.793+0000 7fb0c9fff700 1 --2- 172.21.15.64:0/65709580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0c40683b0 0x7fb0c410d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:59.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.793+0000 7fb0c9fff700 1 --2- 172.21.15.64:0/65709580 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0c4106650 0x7fb0c4106a30 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:59.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.793+0000 7fb0c9fff700 1 --2- 172.21.15.64:0/65709580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0c40679f0 0x7fb0c4067e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:59.795 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.793+0000 7fb0c9fff700 1 -- 172.21.15.64:0/65709580 >> 172.21.15.64:0/65709580 conn(0x7fb0c4074f30 msgr2=0x7fb0c4075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:26:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.793+0000 7fb0c9fff700 1 -- 172.21.15.64:0/65709580 shutdown_connections 2023-09-13T03:26:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.793+0000 7fb0c9fff700 1 -- 172.21.15.64:0/65709580 wait complete. 2023-09-13T03:26:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.794+0000 7fb0c9fff700 1 Processor -- start 2023-09-13T03:26:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.795+0000 7fb0c9fff700 1 -- start start 2023-09-13T03:26:59.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.795+0000 7fb0c9fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0c40679f0 0x7fb0c419c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:59.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.795+0000 7fb0c9fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0c40683b0 0x7fb0c419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:59.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.795+0000 7fb0c9fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0c4106650 0x7fb0c4196570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:59.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.795+0000 7fb0c9fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c4196be0 con 0x7fb0c40683b0 2023-09-13T03:26:59.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.795+0000 7fb0c9fff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c4196d50 con 0x7fb0c4106650 2023-09-13T03:26:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.795+0000 7fb0c9fff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c4196f10 con 0x7fb0c40679f0 2023-09-13T03:26:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.795+0000 7fb0c37fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0c40679f0 0x7fb0c419c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.795+0000 7fb0c37fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0c40679f0 0x7fb0c419c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:57144/0 (socket says 172.21.15.64:57144) 2023-09-13T03:26:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.795+0000 7fb0c37fe700 1 -- 172.21.15.64:0/2038762149 learned_addr learned my addr 172.21.15.64:0/2038762149 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:26:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.796+0000 7fb0c3fff700 1 --2- 172.21.15.64:0/2038762149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0c4106650 0x7fb0c4196570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.796+0000 7fb0c2ffd700 1 --2- 172.21.15.64:0/2038762149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0c40683b0 0x7fb0c419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.796+0000 7fb0c37fe700 1 -- 172.21.15.64:0/2038762149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0c4106650 msgr2=0x7fb0c4196570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:26:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.796+0000 7fb0c37fe700 1 --2- 172.21.15.64:0/2038762149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0c4106650 0x7fb0c4196570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.796+0000 7fb0c37fe700 1 -- 172.21.15.64:0/2038762149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0c40683b0 msgr2=0x7fb0c419cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:26:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.796+0000 7fb0c37fe700 1 --2- 172.21.15.64:0/2038762149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0c40683b0 0x7fb0c419cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:59.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.796+0000 7fb0c37fe700 1 -- 172.21.15.64:0/2038762149 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b40095f0 con 0x7fb0c40679f0 2023-09-13T03:26:59.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.796+0000 7fb0c37fe700 1 --2- 172.21.15.64:0/2038762149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0c40679f0 0x7fb0c419c6c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400b680 tx=0x7fb0b4005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:26:59.801 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.797+0000 7fb0c0ff9700 1 -- 172.21.15.64:0/2038762149 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b401d070 con 0x7fb0c40679f0 2023-09-13T03:26:59.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.797+0000 7fb0c0ff9700 1 -- 172.21.15.64:0/2038762149 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0b4021410 con 0x7fb0c40679f0 2023-09-13T03:26:59.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.798+0000 7fb0c0ff9700 1 -- 172.21.15.64:0/2038762149 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4017650 con 0x7fb0c40679f0 2023-09-13T03:26:59.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.799+0000 7fb0c9fff700 1 -- 172.21.15.64:0/2038762149 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0c4197190 con 0x7fb0c40679f0 2023-09-13T03:26:59.802 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.799+0000 7fb0c9fff700 1 -- 172.21.15.64:0/2038762149 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0c406a200 con 0x7fb0c40679f0 2023-09-13T03:26:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.803+0000 7fb0c0ff9700 1 -- 172.21.15.64:0/2038762149 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fb0b400fb10 con 0x7fb0c40679f0 2023-09-13T03:26:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.804+0000 7fb0dd912b80 1 Processor -- start 2023-09-13T03:26:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.804+0000 7fb0dd912b80 1 -- start start 2023-09-13T03:26:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.805+0000 7fb0dd912b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558635954270 0x558635950300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.805+0000 7fb0dd912b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558635950840 0x558635950cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.805+0000 7fb0dd912b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558635962f60 0x558635965350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.805+0000 7fb0dd912b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586358a96d0 con 0x558635954270 2023-09-13T03:26:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.805+0000 7fb0dd912b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586358a6ff0 con 0x558635962f60 2023-09-13T03:26:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.805+0000 7fb0dd912b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586358a28e0 con 0x558635950840 2023-09-13T03:26:59.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.805+0000 7fb0c2ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558635962f60 0x558635965350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:59.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.805+0000 7fb0c2ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558635962f60 0x558635965350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:50258/0 (socket says 172.21.15.64:50258) 2023-09-13T03:26:59.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.805+0000 7fb0c2ffd700 1 -- 172.21.15.64:0/4159569528 learned_addr learned my addr 172.21.15.64:0/4159569528 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:26:59.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.806+0000 7fb0c3fff700 1 --2- 172.21.15.64:0/4159569528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558635954270 0x558635950300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:59.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.806+0000 7fb0c0ff9700 1 --2- 172.21.15.64:0/2038762149 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fb0ac0a0900 0x7fb0ac0a2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.806+0000 7fb0c0ff9700 1 -- 172.21.15.64:0/2038762149 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fb0b40ccfe0 con 0x7fb0c40679f0 2023-09-13T03:26:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.806+0000 7fb0c2ffd700 1 -- 172.21.15.64:0/4159569528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558635950840 msgr2=0x558635950cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:26:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.806+0000 7fb0c2ffd700 1 --2- 172.21.15.64:0/4159569528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558635950840 0x558635950cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.806+0000 7fb0c2ffd700 1 -- 172.21.15.64:0/4159569528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558635954270 msgr2=0x558635950300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:26:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.806+0000 7fb0c2ffd700 1 --2- 172.21.15.64:0/4159569528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558635954270 0x558635950300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:59.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.806+0000 7fb0c2ffd700 1 -- 172.21.15.64:0/4159569528 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586356ced80 con 0x558635962f60 2023-09-13T03:26:59.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.806+0000 7fb0c3fff700 1 --2- 172.21.15.64:0/2038762149 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fb0ac0a0900 0x7fb0ac0a2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:59.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.807+0000 7fb0c2ffd700 1 --2- 172.21.15.64:0/4159569528 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558635962f60 0x558635965350 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8009d20 tx=0x7fb0b8011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b4d9b9dfb13301b server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:26:59.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.807+0000 7fb0a9ffb700 1 -- 172.21.15.64:0/4159569528 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8019630 con 0x558635962f60 2023-09-13T03:26:59.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.807+0000 7fb0a9ffb700 1 -- 172.21.15.64:0/4159569528 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0b8019c50 con 0x558635962f60 2023-09-13T03:26:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.807+0000 7fb0a9ffb700 1 -- 172.21.15.64:0/4159569528 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b800ec30 con 0x558635962f60 2023-09-13T03:26:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.808+0000 7fb0dd912b80 1 -- 172.21.15.64:0/4159569528 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558635962f60 msgr2=0x558635965350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:26:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.808+0000 7fb0c3fff700 1 --2- 172.21.15.64:0/2038762149 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fb0ac0a0900 0x7fb0ac0a2dc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb0c419d3c0 tx=0x7fb0bc00b040 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:26:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.808+0000 7fb0dd912b80 1 --2- 172.21.15.64:0/4159569528 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558635962f60 0x558635965350 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8009d20 tx=0x7fb0b8011720 comp rx=0 tx=0).stop 2023-09-13T03:26:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.808+0000 7fb0dd912b80 1 -- 172.21.15.64:0/4159569528 shutdown_connections 2023-09-13T03:26:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.808+0000 7fb0dd912b80 1 --2- 172.21.15.64:0/4159569528 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558635962f60 0x558635965350 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.808+0000 7fb0dd912b80 1 --2- 172.21.15.64:0/4159569528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558635954270 0x558635950300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.808+0000 7fb0dd912b80 1 --2- 172.21.15.64:0/4159569528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558635950840 0x558635950cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.808+0000 7fb0dd912b80 1 -- 172.21.15.64:0/4159569528 >> 172.21.15.64:0/4159569528 conn(0x5586358c5030 msgr2=0x55863594b560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:26:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.809+0000 7fb0dd912b80 1 -- 172.21.15.64:0/4159569528 shutdown_connections 2023-09-13T03:26:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.809+0000 7fb0dd912b80 1 -- 172.21.15.64:0/4159569528 wait complete. 2023-09-13T03:26:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.809+0000 7fb0dd912b80 1 -- start start 2023-09-13T03:26:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.810+0000 7fb0dd912b80 1 Processor -- start 2023-09-13T03:26:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.810+0000 7fb0dd912b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558635950840 0x5586359629c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.810+0000 7fb0dd912b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558635954270 0x55863595fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.810+0000 7fb0dd912b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558635962f60 0x558635960380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.810+0000 7fb0dd912b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x558635840b10 con 0x558635962f60 2023-09-13T03:26:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.810+0000 7fb0dd912b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558635840810 con 0x558635954270 2023-09-13T03:26:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.810+0000 7fb0dd912b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586357248c0 con 0x558635950840 2023-09-13T03:26:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.810+0000 7fb0dd912b80 10 client.0 _ll_register_callbacks cb 0x55863578aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:26:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.811+0000 7fb0c3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558635962f60 0x558635960380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.811+0000 7fb0c3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558635962f60 0x558635960380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36966/0 (socket says 172.21.15.64:36966) 2023-09-13T03:26:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.811+0000 7fb0c3fff700 1 -- 172.21.15.64:0/2282780393 learned_addr learned my addr 172.21.15.64:0/2282780393 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:26:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.811+0000 7fb0c37fe700 1 --2- 172.21.15.64:0/2282780393 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558635950840 0x5586359629c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.811+0000 7fb0c2ffd700 1 --2- 172.21.15.64:0/2282780393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558635954270 0x55863595fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.811+0000 7fb0c3fff700 1 -- 172.21.15.64:0/2282780393 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558635950840 msgr2=0x5586359629c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:26:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.811+0000 7fb0c3fff700 1 --2- 172.21.15.64:0/2282780393 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558635950840 0x5586359629c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:59.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.811+0000 7fb0c3fff700 1 -- 172.21.15.64:0/2282780393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558635954270 msgr2=0x55863595fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:26:59.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.811+0000 7fb0c3fff700 1 --2- 172.21.15.64:0/2282780393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558635954270 0x55863595fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:59.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.811+0000 7fb0c3fff700 1 -- 172.21.15.64:0/2282780393 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0b8009980 con 0x558635962f60 2023-09-13T03:26:59.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.811+0000 7fb0c2ffd700 1 --2- 172.21.15.64:0/2282780393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558635954270 0x55863595fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:26:59.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.812+0000 7fb0c3fff700 1 --2- 172.21.15.64:0/2282780393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558635962f60 0x558635960380 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc016c00 tx=0x7fb0bc009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5971016dae671624 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:26:59.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.812+0000 7fb0a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:26:59.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.812+0000 7fb0a2ffd700 1 -- 172.21.15.64:0/2282780393 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc015800 con 0x558635962f60 2023-09-13T03:26:59.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.812+0000 7fb0dd912b80 1 -- 172.21.15.64:0/2282780393 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5586357fa490 con 0x558635962f60 2023-09-13T03:26:59.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.812+0000 7fb0a2ffd700 1 -- 172.21.15.64:0/2282780393 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fb0bc015960 con 0x558635962f60 2023-09-13T03:26:59.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.813+0000 7fb0a2ffd700 1 -- 172.21.15.64:0/2282780393 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0bc015b20 con 0x558635962f60 2023-09-13T03:26:59.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.813+0000 7fb0dd912b80 10 client.17181 fetch_fsmap learned FSMap version 70 2023-09-13T03:26:59.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.813+0000 7fb0dd912b80 1 -- 172.21.15.64:0/2282780393 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x5586356b7c00 con 0x558635962f60 2023-09-13T03:26:59.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.813+0000 7fb0a2ffd700 1 -- 172.21.15.64:0/2282780393 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0bc02e020 con 0x558635962f60 2023-09-13T03:26:59.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.813+0000 7fb0a2ffd700 1 -- 172.21.15.64:0/2282780393 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 70) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb0bc0203a0 con 0x558635962f60 2023-09-13T03:26:59.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.813+0000 7fb0dd912b80 10 client.17181 fetch_fsmap finished waiting for FSMap version 70 2023-09-13T03:26:59.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.813+0000 7fb0dd912b80 10 client.17181 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:26:59.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.813+0000 7fb0dd912b80 20 client.17181 populate_metadata read hostname 'smithi064' 2023-09-13T03:26:59.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.813+0000 7fb0dd912b80 1 --2- 172.21.15.64:0/2282780393 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x558635972da0 0x5586359751c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:59.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.813+0000 7fb0dd912b80 4 client.17181 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:26:59.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.813+0000 7fb0dd912b80 1 -- 172.21.15.64:0/2282780393 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5586358b7840 con 0x558635972da0 2023-09-13T03:26:59.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.814+0000 7fb0c37fe700 1 --2- 172.21.15.64:0/2282780393 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x558635972da0 0x5586359751c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:59.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.815+0000 7fb0c37fe700 1 --2- 172.21.15.64:0/2282780393 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x558635972da0 0x5586359751c0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30cf76b3c5b72166 server_cookie=59168d0c4ab38d18 in_seq=0 out_seq=0 2023-09-13T03:26:59.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.815+0000 7fb0a2ffd700 10 client.17181 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:26:59.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.819+0000 7fb0a2ffd700 1 -- 172.21.15.64:0/2282780393 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5586358b7840 con 0x558635972da0 2023-09-13T03:26:59.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.819+0000 7fb0a2ffd700 10 client.17181 handle_command_reply: tid=0 2023-09-13T03:26:59.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.819+0000 7fb0dd912b80 1 client.17181 shutdown 2023-09-13T03:26:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.820+0000 7fb0dd912b80 1 -- 172.21.15.64:0/2282780393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558635962f60 msgr2=0x558635960380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:26:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.820+0000 7fb0dd912b80 1 --2- 172.21.15.64:0/2282780393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558635962f60 0x558635960380 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc016c00 tx=0x7fb0bc009040 comp rx=0 tx=0).stop 2023-09-13T03:26:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.820+0000 7fb0dd912b80 1 -- 172.21.15.64:0/2282780393 shutdown_connections 2023-09-13T03:26:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.820+0000 7fb0dd912b80 1 --2- 172.21.15.64:0/2282780393 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x558635972da0 0x5586359751c0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.820+0000 7fb0dd912b80 1 --2- 172.21.15.64:0/2282780393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558635962f60 0x558635960380 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.820+0000 7fb0dd912b80 1 --2- 172.21.15.64:0/2282780393 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558635950840 0x5586359629c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.820+0000 7fb0a2ffd700 0 client.17181 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:26:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.820+0000 7fb0dd912b80 1 --2- 172.21.15.64:0/2282780393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558635954270 0x55863595fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.820+0000 7fb0dd912b80 1 -- 172.21.15.64:0/2282780393 >> 172.21.15.64:0/2282780393 conn(0x5586358c5030 msgr2=0x55863594a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:26:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.820+0000 7fb0dd912b80 1 -- 172.21.15.64:0/2282780393 shutdown_connections 2023-09-13T03:26:59.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.820+0000 7fb0dd912b80 1 -- 172.21.15.64:0/2282780393 wait complete. 2023-09-13T03:26:59.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.820+0000 7fb0dd912b80 20 client.17181 trim_cache size 0 max 16384 2023-09-13T03:26:59.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.845+0000 7fb0dd912b80 1 Processor -- start 2023-09-13T03:26:59.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.845+0000 7fb0dd912b80 1 -- start start 2023-09-13T03:26:59.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.845+0000 7fb0dd912b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558635950840 0x55863594af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:59.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.845+0000 7fb0dd912b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558635954270 0x55863594b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:59.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.845+0000 7fb0dd912b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558635962f60 0x55863595f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:59.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.845+0000 7fb0dd912b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x558635970290 con 0x558635954270 2023-09-13T03:26:59.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.845+0000 7fb0dd912b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558635947fc0 con 0x558635962f60 2023-09-13T03:26:59.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.845+0000 7fb0dd912b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558635960bd0 con 0x558635950840 2023-09-13T03:26:59.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.846+0000 7fb0c3fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558635962f60 0x55863595f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:59.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.846+0000 7fb0c3fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558635962f60 0x55863595f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:50270/0 (socket says 172.21.15.64:50270) 2023-09-13T03:26:59.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.846+0000 7fb0c3fff700 1 -- 172.21.15.64:0/1582344011 learned_addr learned my addr 172.21.15.64:0/1582344011 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:26:59.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.846+0000 7fb0c37fe700 1 --2- 172.21.15.64:0/1582344011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558635950840 0x55863594af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:59.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.846+0000 7fb0c3fff700 1 -- 172.21.15.64:0/1582344011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558635950840 msgr2=0x55863594af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:26:59.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.846+0000 7fb0c3fff700 1 --2- 172.21.15.64:0/1582344011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558635950840 0x55863594af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:59.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.846+0000 7fb0c3fff700 1 -- 172.21.15.64:0/1582344011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558635954270 msgr2=0x55863594b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:26:59.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.846+0000 7fb0c3fff700 1 --2- 172.21.15.64:0/1582344011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558635954270 0x55863594b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:59.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.846+0000 7fb0c3fff700 1 -- 172.21.15.64:0/1582344011 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586356f51f0 con 0x558635962f60 2023-09-13T03:26:59.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.847+0000 7fb0c3fff700 1 --2- 172.21.15.64:0/1582344011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558635962f60 0x55863595f400 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc020780 tx=0x7fb0bc00afb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7b801f5443501b3 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:26:59.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.847+0000 7fb0aa7fc700 1 -- 172.21.15.64:0/1582344011 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc036070 con 0x558635962f60 2023-09-13T03:26:59.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.848+0000 7fb0aa7fc700 1 -- 172.21.15.64:0/1582344011 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0bc02e070 con 0x558635962f60 2023-09-13T03:26:59.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.848+0000 7fb0aa7fc700 1 -- 172.21.15.64:0/1582344011 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc0158b0 con 0x558635962f60 2023-09-13T03:26:59.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.849+0000 7fb0dd912b80 1 -- 172.21.15.64:0/1582344011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558635962f60 msgr2=0x55863595f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:26:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.849+0000 7fb0dd912b80 1 --2- 172.21.15.64:0/1582344011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558635962f60 0x55863595f400 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc020780 tx=0x7fb0bc00afb0 comp rx=0 tx=0).stop 2023-09-13T03:26:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.849+0000 7fb0dd912b80 1 -- 172.21.15.64:0/1582344011 shutdown_connections 2023-09-13T03:26:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.849+0000 7fb0dd912b80 1 --2- 172.21.15.64:0/1582344011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558635962f60 0x55863595f400 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.849+0000 7fb0dd912b80 1 --2- 172.21.15.64:0/1582344011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558635950840 0x55863594af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.849+0000 7fb0dd912b80 1 --2- 172.21.15.64:0/1582344011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558635954270 0x55863594b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.849+0000 7fb0dd912b80 1 -- 172.21.15.64:0/1582344011 >> 172.21.15.64:0/1582344011 conn(0x5586358c5030 msgr2=0x55863594e840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:26:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.850+0000 7fb0dd912b80 1 -- 172.21.15.64:0/1582344011 shutdown_connections 2023-09-13T03:26:59.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.850+0000 7fb0dd912b80 1 -- 172.21.15.64:0/1582344011 wait complete. 2023-09-13T03:26:59.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.851+0000 7fb0dd912b80 1 -- start start 2023-09-13T03:26:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.851+0000 7fb0dd912b80 1 Processor -- start 2023-09-13T03:26:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.852+0000 7fb0dd912b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558635950840 0x558635969c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.852+0000 7fb0dd912b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558635954270 0x558635968090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.852+0000 7fb0dd912b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558635962f60 0x5586359685d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.852+0000 7fb0dd912b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x558635968ca0 con 0x558635962f60 2023-09-13T03:26:59.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.852+0000 7fb0dd912b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558635968e10 con 0x558635950840 2023-09-13T03:26:59.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.852+0000 7fb0c37fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558635950840 0x558635969c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:59.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.852+0000 7fb0c37fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558635950840 0x558635969c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:50284/0 (socket says 172.21.15.64:50284) 2023-09-13T03:26:59.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.852+0000 7fb0c37fe700 1 -- 172.21.15.64:0/1501054531 learned_addr learned my addr 172.21.15.64:0/1501054531 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:26:59.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.852+0000 7fb0dd912b80 1 -- 172.21.15.64:0/1501054531 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558635954710 con 0x558635954270 2023-09-13T03:26:59.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.853+0000 7fb0dd912b80 10 client.0 _ll_register_callbacks cb 0x7fb0bc0167b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:26:59.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.853+0000 7fb0c37fe700 1 -- 172.21.15.64:0/1501054531 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558635954270 msgr2=0x558635968090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-13T03:26:59.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.853+0000 7fb0c37fe700 1 --2- 172.21.15.64:0/1501054531 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558635954270 0x558635968090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:59.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.853+0000 7fb0c37fe700 1 -- 172.21.15.64:0/1501054531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558635962f60 msgr2=0x5586359685d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:26:59.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.853+0000 7fb0c37fe700 1 --2- 172.21.15.64:0/1501054531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558635962f60 0x5586359685d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:26:59.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.853+0000 7fb0c37fe700 1 -- 172.21.15.64:0/1501054531 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5586356b7c00 con 0x558635950840 2023-09-13T03:26:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.853+0000 7fb0c37fe700 1 --2- 172.21.15.64:0/1501054531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558635950840 0x558635969c70 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4000c00 tx=0x7fb0b4096210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14effcc15bb66b90 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:26:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.853+0000 7fb0a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:26:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.854+0000 7fb0a2ffd700 1 -- 172.21.15.64:0/1501054531 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b40b6070 con 0x558635950840 2023-09-13T03:26:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.854+0000 7fb0a2ffd700 1 -- 172.21.15.64:0/1501054531 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fb0b40cae40 con 0x558635950840 2023-09-13T03:26:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.854+0000 7fb0dd912b80 1 -- 172.21.15.64:0/1501054531 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb0b8019c50 con 0x558635950840 2023-09-13T03:26:59.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.854+0000 7fb0a2ffd700 1 -- 172.21.15.64:0/1501054531 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0b409a4e0 con 0x558635950840 2023-09-13T03:26:59.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.855+0000 7fb0dd912b80 10 client.27239 fetch_fsmap learned FSMap version 70 2023-09-13T03:26:59.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.855+0000 7fb0dd912b80 1 -- 172.21.15.64:0/1501054531 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x5586356cd250 con 0x558635950840 2023-09-13T03:26:59.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.855+0000 7fb0a2ffd700 1 -- 172.21.15.64:0/1501054531 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0b8019c50 con 0x558635950840 2023-09-13T03:26:59.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.856+0000 7fb0a2ffd700 1 -- 172.21.15.64:0/1501054531 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 70) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb0b40b82b0 con 0x558635950840 2023-09-13T03:26:59.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.856+0000 7fb0dd912b80 10 client.27239 fetch_fsmap finished waiting for FSMap version 70 2023-09-13T03:26:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.856+0000 7fb0dd912b80 10 client.27239 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:26:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.856+0000 7fb0dd912b80 20 client.27239 populate_metadata read hostname 'smithi064' 2023-09-13T03:26:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.856+0000 7fb0dd912b80 1 --2- 172.21.15.64:0/1501054531 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x558635955a60 0x558635957ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:26:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.856+0000 7fb0dd912b80 4 client.27239 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:26:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.856+0000 7fb0dd912b80 1 -- 172.21.15.64:0/1501054531 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55863589b8c0 con 0x558635955a60 2023-09-13T03:26:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.856+0000 7fb0c2ffd700 1 --2- 172.21.15.64:0/1501054531 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x558635955a60 0x558635957ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:26:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.857+0000 7fb0c2ffd700 1 --2- 172.21.15.64:0/1501054531 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x558635955a60 0x558635957ea0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1ca5097ebbbd77f server_cookie=73918f9d6492dad1 in_seq=0 out_seq=0 2023-09-13T03:26:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:26:59.857+0000 7fb0a2ffd700 10 client.27239 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:27:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:00.024+0000 7fb0a2ffd700 1 -- 172.21.15.64:0/1501054531 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55863589b8c0 con 0x558635955a60 2023-09-13T03:27:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:00.024+0000 7fb0a2ffd700 10 client.27239 handle_command_reply: tid=0 2023-09-13T03:27:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:00.024+0000 7fb0dd912b80 1 client.27239 shutdown 2023-09-13T03:27:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:00.025+0000 7fb0dd912b80 1 -- 172.21.15.64:0/1501054531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558635950840 msgr2=0x558635969c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:27:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:00.025+0000 7fb0dd912b80 1 --2- 172.21.15.64:0/1501054531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558635950840 0x558635969c70 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4000c00 tx=0x7fb0b4096210 comp rx=0 tx=0).stop 2023-09-13T03:27:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:00.026+0000 7fb0dd912b80 1 -- 172.21.15.64:0/1501054531 shutdown_connections 2023-09-13T03:27:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:00.026+0000 7fb0dd912b80 1 --2- 172.21.15.64:0/1501054531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558635950840 0x558635969c70 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:00.026+0000 7fb0dd912b80 1 --2- 172.21.15.64:0/1501054531 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x558635955a60 0x558635957ea0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:00.026+0000 7fb0dd912b80 1 --2- 172.21.15.64:0/1501054531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558635962f60 0x5586359685d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:00.026+0000 7fb0dd912b80 1 --2- 172.21.15.64:0/1501054531 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558635954270 0x558635968090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:00.026+0000 7fb0a2ffd700 0 client.27239 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:27:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:00.026+0000 7fb0dd912b80 1 -- 172.21.15.64:0/1501054531 >> 172.21.15.64:0/1501054531 conn(0x5586358c5030 msgr2=0x55863595cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:27:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:00.026+0000 7fb0dd912b80 1 -- 172.21.15.64:0/1501054531 shutdown_connections 2023-09-13T03:27:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:00.026+0000 7fb0dd912b80 1 -- 172.21.15.64:0/1501054531 wait complete. 2023-09-13T03:27:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:00.027+0000 7fb0dd912b80 20 client.27239 trim_cache size 0 max 16384 2023-09-13T03:27:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:27:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:27:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:27:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:27:00.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:00.027+0000 7fb0c9fff700 1 -- 172.21.15.64:0/2038762149 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fb0ac0a0900 msgr2=0x7fb0ac0a2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:27:00.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:00.027+0000 7fb0c9fff700 1 --2- 172.21.15.64:0/2038762149 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fb0ac0a0900 0x7fb0ac0a2dc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb0c419d3c0 tx=0x7fb0bc00b040 comp rx=0 tx=0).stop 2023-09-13T03:27:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:00.027+0000 7fb0c9fff700 1 -- 172.21.15.64:0/2038762149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0c40679f0 msgr2=0x7fb0c419c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:27:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:00.027+0000 7fb0c9fff700 1 --2- 172.21.15.64:0/2038762149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0c40679f0 0x7fb0c419c6c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400b680 tx=0x7fb0b4005740 comp rx=0 tx=0).stop 2023-09-13T03:27:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:00.027+0000 7fb0c9fff700 1 -- 172.21.15.64:0/2038762149 shutdown_connections 2023-09-13T03:27:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:00.027+0000 7fb0c9fff700 1 --2- 172.21.15.64:0/2038762149 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fb0ac0a0900 0x7fb0ac0a2dc0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:00.027+0000 7fb0c9fff700 1 --2- 172.21.15.64:0/2038762149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0c4106650 0x7fb0c4196570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:00.028+0000 7fb0c9fff700 1 --2- 172.21.15.64:0/2038762149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0c40679f0 0x7fb0c419c6c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:00.028+0000 7fb0c9fff700 1 --2- 172.21.15.64:0/2038762149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0c40683b0 0x7fb0c419cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:00.028+0000 7fb0c9fff700 1 -- 172.21.15.64:0/2038762149 >> 172.21.15.64:0/2038762149 conn(0x7fb0c4074f30 msgr2=0x7fb0c40fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:27:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:00.028+0000 7fb0c9fff700 1 -- 172.21.15.64:0/2038762149 shutdown_connections 2023-09-13T03:27:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:00.028+0000 7fb0c9fff700 1 -- 172.21.15.64:0/2038762149 wait complete. 2023-09-13T03:27:00.285 INFO:tasks.workunit.client.0.smithi064.stdout: 10 50731 45.50 MB/sec warmup 64 sec latency 127.355 ms 2023-09-13T03:27:00.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:27:00 smithi142 ceph-mon[134082]: pgmap v432: 97 pgs: 97 active+clean; 560 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 29 MiB/s wr, 1.38k op/s 2023-09-13T03:27:00.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:27:00 smithi170 ceph-mon[134328]: pgmap v432: 97 pgs: 97 active+clean; 560 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 29 MiB/s wr, 1.38k op/s 2023-09-13T03:27:00.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:27:00 smithi064 ceph-mon[97675]: pgmap v432: 97 pgs: 97 active+clean; 560 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 29 MiB/s wr, 1.38k op/s 2023-09-13T03:27:01.285 INFO:tasks.workunit.client.0.smithi064.stdout: 10 51640 45.36 MB/sec warmup 65 sec latency 47.869 ms 2023-09-13T03:27:02.285 INFO:tasks.workunit.client.0.smithi064.stdout: 10 52088 45.05 MB/sec warmup 66 sec latency 144.722 ms 2023-09-13T03:27:03.162 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:27:02 smithi170 ceph-mon[134328]: pgmap v433: 97 pgs: 97 active+clean; 587 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 32 MiB/s wr, 1.76k op/s 2023-09-13T03:27:03.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:27:02 smithi064 ceph-mon[97675]: pgmap v433: 97 pgs: 97 active+clean; 587 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 32 MiB/s wr, 1.76k op/s 2023-09-13T03:27:03.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:27:02 smithi142 ceph-mon[134082]: pgmap v433: 97 pgs: 97 active+clean; 587 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 32 MiB/s wr, 1.76k op/s 2023-09-13T03:27:03.285 INFO:tasks.workunit.client.0.smithi064.stdout: 10 52990 45.23 MB/sec warmup 67 sec latency 108.993 ms 2023-09-13T03:27:04.285 INFO:tasks.workunit.client.0.smithi064.stdout: 10 53926 45.35 MB/sec warmup 68 sec latency 42.611 ms 2023-09-13T03:27:05.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:27:04 smithi170 ceph-mon[134328]: pgmap v434: 97 pgs: 97 active+clean; 587 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 24 MiB/s wr, 1.21k op/s 2023-09-13T03:27:05.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:27:04 smithi064 ceph-mon[97675]: pgmap v434: 97 pgs: 97 active+clean; 587 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 24 MiB/s wr, 1.21k op/s 2023-09-13T03:27:05.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:27:04 smithi142 ceph-mon[134082]: pgmap v434: 97 pgs: 97 active+clean; 587 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 24 MiB/s wr, 1.21k op/s 2023-09-13T03:27:05.285 INFO:tasks.workunit.client.0.smithi064.stdout: 10 54683 45.25 MB/sec warmup 69 sec latency 102.748 ms 2023-09-13T03:27:06.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:27:05 smithi170 ceph-mon[134328]: osdmap e101: 12 total, 12 up, 12 in 2023-09-13T03:27:06.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:27:05 smithi064 ceph-mon[97675]: osdmap e101: 12 total, 12 up, 12 in 2023-09-13T03:27:06.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:27:05 smithi142 ceph-mon[134082]: osdmap e101: 12 total, 12 up, 12 in 2023-09-13T03:27:06.285 INFO:tasks.workunit.client.0.smithi064.stdout: 10 55726 45.45 MB/sec warmup 70 sec latency 348.701 ms 2023-09-13T03:27:07.162 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:27:06 smithi170 ceph-mon[134328]: pgmap v436: 97 pgs: 97 active+clean; 614 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.1 MiB/s wr, 1.02k op/s 2023-09-13T03:27:07.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:27:06 smithi064 ceph-mon[97675]: pgmap v436: 97 pgs: 97 active+clean; 614 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.1 MiB/s wr, 1.02k op/s 2023-09-13T03:27:07.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:27:06 smithi142 ceph-mon[134082]: pgmap v436: 97 pgs: 97 active+clean; 614 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.1 MiB/s wr, 1.02k op/s 2023-09-13T03:27:07.285 INFO:tasks.workunit.client.0.smithi064.stdout: 10 56585 45.42 MB/sec warmup 71 sec latency 1348.751 ms 2023-09-13T03:27:08.286 INFO:tasks.workunit.client.0.smithi064.stdout: 10 57524 45.46 MB/sec warmup 72 sec latency 2348.819 ms 2023-09-13T03:27:09.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:27:08 smithi170 ceph-mon[134328]: pgmap v437: 97 pgs: 97 active+clean; 614 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.1 MiB/s wr, 1.02k op/s 2023-09-13T03:27:09.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:27:08 smithi064 ceph-mon[97675]: pgmap v437: 97 pgs: 97 active+clean; 614 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.1 MiB/s wr, 1.02k op/s 2023-09-13T03:27:09.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:27:08 smithi142 ceph-mon[134082]: pgmap v437: 97 pgs: 97 active+clean; 614 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.1 MiB/s wr, 1.02k op/s 2023-09-13T03:27:09.286 INFO:tasks.workunit.client.0.smithi064.stdout: 10 58611 45.65 MB/sec warmup 73 sec latency 3348.862 ms 2023-09-13T03:27:10.286 INFO:tasks.workunit.client.0.smithi064.stdout: 10 59683 45.89 MB/sec warmup 74 sec latency 4348.921 ms 2023-09-13T03:27:10.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:27:10 smithi142 ceph-mon[134082]: osdmap e102: 12 total, 12 up, 12 in 2023-09-13T03:27:10.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:27:10 smithi142 ceph-mon[134082]: pgmap v439: 97 pgs: 97 active+clean; 614 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 5.3 MiB/s wr, 691 op/s 2023-09-13T03:27:10.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:27:10 smithi170 ceph-mon[134328]: osdmap e102: 12 total, 12 up, 12 in 2023-09-13T03:27:10.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:27:10 smithi170 ceph-mon[134328]: pgmap v439: 97 pgs: 97 active+clean; 614 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 5.3 MiB/s wr, 691 op/s 2023-09-13T03:27:10.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:27:10 smithi064 ceph-mon[97675]: osdmap e102: 12 total, 12 up, 12 in 2023-09-13T03:27:10.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:27:10 smithi064 ceph-mon[97675]: pgmap v439: 97 pgs: 97 active+clean; 614 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 5.3 MiB/s wr, 691 op/s 2023-09-13T03:27:11.287 INFO:tasks.workunit.client.0.smithi064.stdout: 10 60666 45.98 MB/sec warmup 75 sec latency 5350.142 ms 2023-09-13T03:27:12.287 INFO:tasks.workunit.client.0.smithi064.stdout: 10 61495 45.95 MB/sec warmup 76 sec latency 6350.196 ms 2023-09-13T03:27:13.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:27:12 smithi170 ceph-mon[134328]: pgmap v440: 97 pgs: 97 active+clean; 648 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 10 MiB/s wr, 1.14k op/s 2023-09-13T03:27:13.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:27:12 smithi064 ceph-mon[97675]: pgmap v440: 97 pgs: 97 active+clean; 648 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 10 MiB/s wr, 1.14k op/s 2023-09-13T03:27:13.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.231+0000 7fe3705ff700 1 -- 172.21.15.64:0/431143633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe368106680 msgr2=0x7fe368106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:27:13.231 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.231+0000 7fe3705ff700 1 --2- 172.21.15.64:0/431143633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe368106680 0x7fe368106a60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe3600099c0 tx=0x7fe360009cd0 comp rx=0 tx=0).stop 2023-09-13T03:27:13.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.232+0000 7fe3705ff700 1 -- 172.21.15.64:0/431143633 shutdown_connections 2023-09-13T03:27:13.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.232+0000 7fe3705ff700 1 --2- 172.21.15.64:0/431143633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe368106680 0x7fe368106a60 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:13.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.232+0000 7fe3705ff700 1 --2- 172.21.15.64:0/431143633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe368100f80 0x7fe36810d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:13.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.232+0000 7fe3705ff700 1 --2- 172.21.15.64:0/431143633 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3681005c0 0x7fe368100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:13.232 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.232+0000 7fe3705ff700 1 -- 172.21.15.64:0/431143633 >> 172.21.15.64:0/431143633 conn(0x7fe3680fc0b0 msgr2=0x7fe3680fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:27:13.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.232+0000 7fe3705ff700 1 -- 172.21.15.64:0/431143633 shutdown_connections 2023-09-13T03:27:13.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.232+0000 7fe3705ff700 1 -- 172.21.15.64:0/431143633 wait complete. 2023-09-13T03:27:13.233 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.233+0000 7fe3705ff700 1 Processor -- start 2023-09-13T03:27:13.234 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.234+0000 7fe3705ff700 1 -- start start 2023-09-13T03:27:13.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.234+0000 7fe3705ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3681005c0 0x7fe368072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:13.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.234+0000 7fe3705ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe368100f80 0x7fe368072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:13.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.235+0000 7fe3705ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe368106680 0x7fe368073380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:13.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.235+0000 7fe3705ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3681a9110 con 0x7fe368106680 2023-09-13T03:27:13.235 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.235+0000 7fe3705ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3681a9280 con 0x7fe368100f80 2023-09-13T03:27:13.236 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.235+0000 7fe3705ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3681a9440 con 0x7fe3681005c0 2023-09-13T03:27:13.236 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.235+0000 7fe36db9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe368100f80 0x7fe368072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:13.236 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.235+0000 7fe36db9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe368100f80 0x7fe368072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:42532/0 (socket says 172.21.15.64:42532) 2023-09-13T03:27:13.236 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.235+0000 7fe36db9a700 1 -- 172.21.15.64:0/4133593054 learned_addr learned my addr 172.21.15.64:0/4133593054 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:27:13.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.235+0000 7fe36e39b700 1 --2- 172.21.15.64:0/4133593054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3681005c0 0x7fe368072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:13.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.235+0000 7fe36db9a700 1 -- 172.21.15.64:0/4133593054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3681005c0 msgr2=0x7fe368072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:27:13.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.235+0000 7fe36db9a700 1 --2- 172.21.15.64:0/4133593054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3681005c0 0x7fe368072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:13.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.235+0000 7fe36db9a700 1 -- 172.21.15.64:0/4133593054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe368106680 msgr2=0x7fe368073380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:27:13.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.235+0000 7fe36db9a700 1 --2- 172.21.15.64:0/4133593054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe368106680 0x7fe368073380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:13.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.235+0000 7fe36db9a700 1 -- 172.21.15.64:0/4133593054 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3600095f0 con 0x7fe368100f80 2023-09-13T03:27:13.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.236+0000 7fe36db9a700 1 --2- 172.21.15.64:0/4133593054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe368100f80 0x7fe368072e40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe35c0060b0 tx=0x7fe35c00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:27:13.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.237+0000 7fe3677fe700 1 -- 172.21.15.64:0/4133593054 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c00ddd0 con 0x7fe368100f80 2023-09-13T03:27:13.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.237+0000 7fe3677fe700 1 -- 172.21.15.64:0/4133593054 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe35c00f040 con 0x7fe368100f80 2023-09-13T03:27:13.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.237+0000 7fe3677fe700 1 -- 172.21.15.64:0/4133593054 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c014720 con 0x7fe368100f80 2023-09-13T03:27:13.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.237+0000 7fe3705ff700 1 -- 172.21.15.64:0/4133593054 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3681a9720 con 0x7fe368100f80 2023-09-13T03:27:13.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.237+0000 7fe3705ff700 1 -- 172.21.15.64:0/4133593054 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3681a9be0 con 0x7fe368100f80 2023-09-13T03:27:13.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.242+0000 7fe3677fe700 1 -- 172.21.15.64:0/4133593054 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fe35c0159c0 con 0x7fe368100f80 2023-09-13T03:27:13.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.244+0000 7fe3677fe700 1 --2- 172.21.15.64:0/4133593054 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fe3500a08b0 0x7fe3500a2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:13.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.244+0000 7fe3677fe700 1 -- 172.21.15.64:0/4133593054 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fe35c0cba90 con 0x7fe368100f80 2023-09-13T03:27:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.244+0000 7fe36e39b700 1 --2- 172.21.15.64:0/4133593054 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fe3500a08b0 0x7fe3500a2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.244+0000 7fe383e95b80 1 Processor -- start 2023-09-13T03:27:13.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.245+0000 7fe383e95b80 1 -- start start 2023-09-13T03:27:13.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:27:12 smithi142 ceph-mon[134082]: pgmap v440: 97 pgs: 97 active+clean; 648 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 10 MiB/s wr, 1.14k op/s 2023-09-13T03:27:13.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.245+0000 7fe383e95b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d74dccf1a0 0x55d74dccf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:13.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.245+0000 7fe383e95b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d74dcc9300 0x55d74dcc9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:13.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.245+0000 7fe383e95b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d74dcc9cc0 0x55d74dcd5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:13.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.245+0000 7fe383e95b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d74dc226d0 con 0x55d74dcc9300 2023-09-13T03:27:13.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.245+0000 7fe383e95b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d74dc1fff0 con 0x55d74dccf1a0 2023-09-13T03:27:13.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.245+0000 7fe383e95b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d74dc1b8e0 con 0x55d74dcc9cc0 2023-09-13T03:27:13.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.246+0000 7fe36e39b700 1 --2- 172.21.15.64:0/4133593054 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fe3500a08b0 0x7fe3500a2d70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe360009990 tx=0x7fe360005fb0 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:27:13.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.246+0000 7fe36e39b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d74dccf1a0 0x55d74dccf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:13.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.246+0000 7fe36e39b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d74dccf1a0 0x55d74dccf580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:42542/0 (socket says 172.21.15.64:42542) 2023-09-13T03:27:13.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.246+0000 7fe36e39b700 1 -- 172.21.15.64:0/4250442506 learned_addr learned my addr 172.21.15.64:0/4250442506 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:27:13.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.246+0000 7fe36eb9c700 1 --2- 172.21.15.64:0/4250442506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d74dcc9cc0 0x55d74dcd5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:13.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.246+0000 7fe36db9a700 1 --2- 172.21.15.64:0/4250442506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d74dcc9300 0x55d74dcc9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:13.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.246+0000 7fe36e39b700 1 -- 172.21.15.64:0/4250442506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d74dcc9cc0 msgr2=0x55d74dcd5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:27:13.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.246+0000 7fe36e39b700 1 --2- 172.21.15.64:0/4250442506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d74dcc9cc0 0x55d74dcd5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:13.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.246+0000 7fe36e39b700 1 -- 172.21.15.64:0/4250442506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d74dcc9300 msgr2=0x55d74dcc9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:27:13.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.246+0000 7fe36e39b700 1 --2- 172.21.15.64:0/4250442506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d74dcc9300 0x55d74dcc9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:13.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.246+0000 7fe36e39b700 1 -- 172.21.15.64:0/4250442506 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d74da47d80 con 0x55d74dccf1a0 2023-09-13T03:27:13.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.246+0000 7fe36eb9c700 1 --2- 172.21.15.64:0/4250442506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d74dcc9cc0 0x55d74dcd5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:27:13.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.247+0000 7fe36e39b700 1 --2- 172.21.15.64:0/4250442506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d74dccf1a0 0x55d74dccf580 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe3600095a0 tx=0x7fe36001f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad70541d1d85e67a server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:27:13.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.247+0000 7fe364bfe700 1 -- 172.21.15.64:0/4250442506 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36002b070 con 0x55d74dccf1a0 2023-09-13T03:27:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.247+0000 7fe364bfe700 1 -- 172.21.15.64:0/4250442506 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe36001e040 con 0x55d74dccf1a0 2023-09-13T03:27:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.247+0000 7fe364bfe700 1 -- 172.21.15.64:0/4250442506 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe360004710 con 0x55d74dccf1a0 2023-09-13T03:27:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.248+0000 7fe383e95b80 1 -- 172.21.15.64:0/4250442506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d74dccf1a0 msgr2=0x55d74dccf580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:27:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.248+0000 7fe383e95b80 1 --2- 172.21.15.64:0/4250442506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d74dccf1a0 0x55d74dccf580 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe3600095a0 tx=0x7fe36001f000 comp rx=0 tx=0).stop 2023-09-13T03:27:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.248+0000 7fe383e95b80 1 -- 172.21.15.64:0/4250442506 shutdown_connections 2023-09-13T03:27:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.248+0000 7fe383e95b80 1 --2- 172.21.15.64:0/4250442506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d74dccf1a0 0x55d74dccf580 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.248+0000 7fe383e95b80 1 --2- 172.21.15.64:0/4250442506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d74dcc9cc0 0x55d74dcd5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.248+0000 7fe383e95b80 1 --2- 172.21.15.64:0/4250442506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d74dcc9300 0x55d74dcc9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.248+0000 7fe383e95b80 1 -- 172.21.15.64:0/4250442506 >> 172.21.15.64:0/4250442506 conn(0x55d74dc3e030 msgr2=0x55d74dcc4560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:27:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.248+0000 7fe383e95b80 1 -- 172.21.15.64:0/4250442506 shutdown_connections 2023-09-13T03:27:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.248+0000 7fe383e95b80 1 -- 172.21.15.64:0/4250442506 wait complete. 2023-09-13T03:27:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.249+0000 7fe383e95b80 1 -- start start 2023-09-13T03:27:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.249+0000 7fe383e95b80 1 Processor -- start 2023-09-13T03:27:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.250+0000 7fe383e95b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d74dcc9300 0x55d74dcd5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.250+0000 7fe383e95b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d74dcc9cc0 0x55d74dcd30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.250+0000 7fe383e95b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d74dccf1a0 0x55d74dcd3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.250+0000 7fe383e95b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d74dbb9b10 con 0x55d74dcc9300 2023-09-13T03:27:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.250+0000 7fe383e95b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d74dbb9810 con 0x55d74dcc9cc0 2023-09-13T03:27:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.250+0000 7fe383e95b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d74da9d8c0 con 0x55d74dccf1a0 2023-09-13T03:27:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.250+0000 7fe383e95b80 10 client.0 _ll_register_callbacks cb 0x55d74db03a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:27:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.250+0000 7fe36db9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d74dcc9cc0 0x55d74dcd30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.250+0000 7fe36db9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d74dcc9cc0 0x55d74dcd30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:42544/0 (socket says 172.21.15.64:42544) 2023-09-13T03:27:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.250+0000 7fe36db9a700 1 -- 172.21.15.64:0/2041781252 learned_addr learned my addr 172.21.15.64:0/2041781252 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:27:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.250+0000 7fe36e39b700 1 --2- 172.21.15.64:0/2041781252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d74dcc9300 0x55d74dcd5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.251+0000 7fe36db9a700 1 -- 172.21.15.64:0/2041781252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d74dccf1a0 msgr2=0x55d74dcd3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:27:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.251+0000 7fe36db9a700 1 --2- 172.21.15.64:0/2041781252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d74dccf1a0 0x55d74dcd3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:13.258 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.251+0000 7fe36db9a700 1 -- 172.21.15.64:0/2041781252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d74dcc9300 msgr2=0x55d74dcd5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:27:13.258 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.251+0000 7fe36db9a700 1 --2- 172.21.15.64:0/2041781252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d74dcc9300 0x55d74dcd5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:13.258 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.251+0000 7fe36db9a700 1 -- 172.21.15.64:0/2041781252 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe36810dd40 con 0x55d74dcc9cc0 2023-09-13T03:27:13.258 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.251+0000 7fe36db9a700 1 --2- 172.21.15.64:0/2041781252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d74dcc9cc0 0x55d74dcd30b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe35c0c9200 tx=0x7fe35c099600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dff04baab231d6a0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:27:13.258 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.251+0000 7fe34dffb700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:27:13.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.251+0000 7fe34dffb700 1 -- 172.21.15.64:0/2041781252 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c0a0f70 con 0x55d74dcc9cc0 2023-09-13T03:27:13.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.251+0000 7fe34dffb700 1 -- 172.21.15.64:0/2041781252 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fe35c0b1250 con 0x55d74dcc9cc0 2023-09-13T03:27:13.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.252+0000 7fe383e95b80 1 -- 172.21.15.64:0/2041781252 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d74db73490 con 0x55d74dcc9cc0 2023-09-13T03:27:13.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.252+0000 7fe34dffb700 1 -- 172.21.15.64:0/2041781252 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe35c0983f0 con 0x55d74dcc9cc0 2023-09-13T03:27:13.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.253+0000 7fe383e95b80 10 client.27251 fetch_fsmap learned FSMap version 70 2023-09-13T03:27:13.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.253+0000 7fe383e95b80 1 -- 172.21.15.64:0/2041781252 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x55d74da30c00 con 0x55d74dcc9cc0 2023-09-13T03:27:13.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.253+0000 7fe34dffb700 1 -- 172.21.15.64:0/2041781252 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d74db73490 con 0x55d74dcc9cc0 2023-09-13T03:27:13.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.253+0000 7fe34dffb700 1 -- 172.21.15.64:0/2041781252 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 70) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe35c094cf0 con 0x55d74dcc9cc0 2023-09-13T03:27:13.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.254+0000 7fe383e95b80 10 client.27251 fetch_fsmap finished waiting for FSMap version 70 2023-09-13T03:27:13.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.254+0000 7fe383e95b80 10 client.27251 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:27:13.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.254+0000 7fe383e95b80 20 client.27251 populate_metadata read hostname 'smithi064' 2023-09-13T03:27:13.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.254+0000 7fe383e95b80 1 --2- 172.21.15.64:0/2041781252 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55d74dce3410 0x55d74dce5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:13.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.254+0000 7fe383e95b80 4 client.27251 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:27:13.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.254+0000 7fe383e95b80 1 -- 172.21.15.64:0/2041781252 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d74dc30840 con 0x55d74dce3410 2023-09-13T03:27:13.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.254+0000 7fe36e39b700 1 --2- 172.21.15.64:0/2041781252 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55d74dce3410 0x55d74dce5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:13.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.255+0000 7fe36e39b700 1 --2- 172.21.15.64:0/2041781252 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55d74dce3410 0x55d74dce5830 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=983f15b54a5a8ead server_cookie=8595407efd27f647 in_seq=0 out_seq=0 2023-09-13T03:27:13.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.255+0000 7fe34dffb700 10 client.27251 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:27:13.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.256+0000 7fe34dffb700 1 -- 172.21.15.64:0/2041781252 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d74dc30840 con 0x55d74dce3410 2023-09-13T03:27:13.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.256+0000 7fe34dffb700 10 client.27251 handle_command_reply: tid=0 2023-09-13T03:27:13.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.257+0000 7fe383e95b80 1 client.27251 shutdown 2023-09-13T03:27:13.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.258+0000 7fe383e95b80 1 -- 172.21.15.64:0/2041781252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d74dcc9cc0 msgr2=0x55d74dcd30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:27:13.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.258+0000 7fe383e95b80 1 --2- 172.21.15.64:0/2041781252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d74dcc9cc0 0x55d74dcd30b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe35c0c9200 tx=0x7fe35c099600 comp rx=0 tx=0).stop 2023-09-13T03:27:13.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.258+0000 7fe383e95b80 1 -- 172.21.15.64:0/2041781252 shutdown_connections 2023-09-13T03:27:13.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.258+0000 7fe383e95b80 1 --2- 172.21.15.64:0/2041781252 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55d74dce3410 0x55d74dce5830 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:13.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.258+0000 7fe383e95b80 1 --2- 172.21.15.64:0/2041781252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d74dccf1a0 0x55d74dcd3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:13.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.258+0000 7fe383e95b80 1 --2- 172.21.15.64:0/2041781252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d74dcc9300 0x55d74dcd5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:13.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.258+0000 7fe383e95b80 1 --2- 172.21.15.64:0/2041781252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d74dcc9cc0 0x55d74dcd30b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:13.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.258+0000 7fe383e95b80 1 -- 172.21.15.64:0/2041781252 >> 172.21.15.64:0/2041781252 conn(0x55d74dc3e030 msgr2=0x55d74dcc37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:27:13.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.258+0000 7fe34dffb700 0 client.27251 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:27:13.265 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.258+0000 7fe383e95b80 1 -- 172.21.15.64:0/2041781252 shutdown_connections 2023-09-13T03:27:13.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.259+0000 7fe383e95b80 1 -- 172.21.15.64:0/2041781252 wait complete. 2023-09-13T03:27:13.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.260+0000 7fe383e95b80 20 client.27251 trim_cache size 0 max 16384 2023-09-13T03:27:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.280+0000 7fe383e95b80 1 Processor -- start 2023-09-13T03:27:13.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.280+0000 7fe383e95b80 1 -- start start 2023-09-13T03:27:13.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.280+0000 7fe383e95b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d74dcc9300 0x55d74dcda460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:13.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.281+0000 7fe383e95b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d74dccf1a0 0x55d74dcc8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:13.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.281+0000 7fe383e95b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d74dcd0df0 0x55d74dcc3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:13.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.281+0000 7fe383e95b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d74dcccdb0 con 0x55d74dccf1a0 2023-09-13T03:27:13.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.281+0000 7fe383e95b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d74dce09f0 con 0x55d74dcc9300 2023-09-13T03:27:13.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.281+0000 7fe383e95b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d74dccee00 con 0x55d74dcd0df0 2023-09-13T03:27:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.281+0000 7fe36e39b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d74dcc9300 0x55d74dcda460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.281+0000 7fe36e39b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d74dcc9300 0x55d74dcda460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:42556/0 (socket says 172.21.15.64:42556) 2023-09-13T03:27:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.281+0000 7fe36e39b700 1 -- 172.21.15.64:0/4276094621 learned_addr learned my addr 172.21.15.64:0/4276094621 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:27:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.281+0000 7fe36db9a700 1 --2- 172.21.15.64:0/4276094621 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d74dccf1a0 0x55d74dcc8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.281+0000 7fe36e39b700 1 -- 172.21.15.64:0/4276094621 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d74dcd0df0 msgr2=0x55d74dcc3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:27:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.281+0000 7fe36e39b700 1 --2- 172.21.15.64:0/4276094621 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d74dcd0df0 0x55d74dcc3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.281+0000 7fe36e39b700 1 -- 172.21.15.64:0/4276094621 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d74dccf1a0 msgr2=0x55d74dcc8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:27:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.281+0000 7fe36e39b700 1 --2- 172.21.15.64:0/4276094621 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d74dccf1a0 0x55d74dcc8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.281+0000 7fe36e39b700 1 -- 172.21.15.64:0/4276094621 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d74da6e1f0 con 0x55d74dcc9300 2023-09-13T03:27:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.281+0000 7fe36db9a700 1 --2- 172.21.15.64:0/4276094621 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d74dccf1a0 0x55d74dcc8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T03:27:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.282+0000 7fe36e39b700 1 --2- 172.21.15.64:0/4276094621 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d74dcc9300 0x55d74dcda460 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe3600099c0 tx=0x7fe360022000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9747676d4492d1a server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:27:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.282+0000 7fe3653ff700 1 -- 172.21.15.64:0/4276094621 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe360034040 con 0x55d74dcc9300 2023-09-13T03:27:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.282+0000 7fe3653ff700 1 -- 172.21.15.64:0/4276094621 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe36000b980 con 0x55d74dcc9300 2023-09-13T03:27:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.282+0000 7fe3653ff700 1 -- 172.21.15.64:0/4276094621 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36002b040 con 0x55d74dcc9300 2023-09-13T03:27:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.283+0000 7fe383e95b80 1 -- 172.21.15.64:0/4276094621 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d74dcc9300 msgr2=0x55d74dcda460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:27:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.283+0000 7fe383e95b80 1 --2- 172.21.15.64:0/4276094621 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d74dcc9300 0x55d74dcda460 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe3600099c0 tx=0x7fe360022000 comp rx=0 tx=0).stop 2023-09-13T03:27:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.283+0000 7fe383e95b80 1 -- 172.21.15.64:0/4276094621 shutdown_connections 2023-09-13T03:27:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.283+0000 7fe383e95b80 1 --2- 172.21.15.64:0/4276094621 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d74dcc9300 0x55d74dcda460 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.283+0000 7fe383e95b80 1 --2- 172.21.15.64:0/4276094621 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d74dcd0df0 0x55d74dcc3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.283+0000 7fe383e95b80 1 --2- 172.21.15.64:0/4276094621 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d74dccf1a0 0x55d74dcc8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.283+0000 7fe383e95b80 1 -- 172.21.15.64:0/4276094621 >> 172.21.15.64:0/4276094621 conn(0x55d74dc3e030 msgr2=0x55d74dcc82f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:27:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.283+0000 7fe383e95b80 1 -- 172.21.15.64:0/4276094621 shutdown_connections 2023-09-13T03:27:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.283+0000 7fe383e95b80 1 -- 172.21.15.64:0/4276094621 wait complete. 2023-09-13T03:27:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.284+0000 7fe383e95b80 1 -- start start 2023-09-13T03:27:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.284+0000 7fe383e95b80 1 Processor -- start 2023-09-13T03:27:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.285+0000 7fe383e95b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d74dcc9300 0x55d74dcd8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.285+0000 7fe383e95b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d74dccf1a0 0x55d74dcd9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.285+0000 7fe383e95b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d74dcd0df0 0x55d74dcd9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.285+0000 7fe383e95b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d74dd372e0 con 0x55d74dccf1a0 2023-09-13T03:27:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.285+0000 7fe383e95b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d74dd37420 con 0x55d74dcd0df0 2023-09-13T03:27:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.285+0000 7fe383e95b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d74dd485f0 con 0x55d74dcc9300 2023-09-13T03:27:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.285+0000 7fe383e95b80 10 client.0 _ll_register_callbacks cb 0x7fe35c098a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:27:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.285+0000 7fe36e39b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d74dcc9300 0x55d74dcd8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.285+0000 7fe36e39b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d74dcc9300 0x55d74dcd8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:36198/0 (socket says 172.21.15.64:36198) 2023-09-13T03:27:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.285+0000 7fe36e39b700 1 -- 172.21.15.64:0/1080342466 learned_addr learned my addr 172.21.15.64:0/1080342466 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:27:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.285+0000 7fe36eb9c700 1 --2- 172.21.15.64:0/1080342466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d74dcd0df0 0x55d74dcd9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.285+0000 7fe36db9a700 1 --2- 172.21.15.64:0/1080342466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d74dccf1a0 0x55d74dcd9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.285+0000 7fe36e39b700 1 -- 172.21.15.64:0/1080342466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d74dcd0df0 msgr2=0x55d74dcd9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:27:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.285+0000 7fe36e39b700 1 --2- 172.21.15.64:0/1080342466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d74dcd0df0 0x55d74dcd9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.285+0000 7fe36e39b700 1 -- 172.21.15.64:0/1080342466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d74dccf1a0 msgr2=0x55d74dcd9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:27:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.285+0000 7fe36e39b700 1 --2- 172.21.15.64:0/1080342466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d74dccf1a0 0x55d74dcd9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.285+0000 7fe36e39b700 1 -- 172.21.15.64:0/1080342466 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d74da47d80 con 0x55d74dcc9300 2023-09-13T03:27:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.286+0000 7fe36e39b700 1 --2- 172.21.15.64:0/1080342466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d74dcc9300 0x55d74dcd8bf0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe360004080 tx=0x7fe36003a420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cb9de7bea963026 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:27:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.286+0000 7fe34dffb700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:27:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.286+0000 7fe34dffb700 1 -- 172.21.15.64:0/1080342466 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36002b040 con 0x55d74dcc9300 2023-09-13T03:27:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.286+0000 7fe34dffb700 1 -- 172.21.15.64:0/1080342466 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fe36003ab70 con 0x55d74dcc9300 2023-09-13T03:27:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.286+0000 7fe383e95b80 1 -- 172.21.15.64:0/1080342466 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe36001e040 con 0x55d74dcc9300 2023-09-13T03:27:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.286+0000 7fe34dffb700 1 -- 172.21.15.64:0/1080342466 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe360034040 con 0x55d74dcc9300 2023-09-13T03:27:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.286+0000 7fe383e95b80 10 client.27229 fetch_fsmap learned FSMap version 70 2023-09-13T03:27:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.286+0000 7fe383e95b80 1 -- 172.21.15.64:0/1080342466 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x55d74da46250 con 0x55d74dcc9300 2023-09-13T03:27:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.287+0000 7fe34dffb700 1 -- 172.21.15.64:0/1080342466 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe360052020 con 0x55d74dcc9300 2023-09-13T03:27:13.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.287+0000 7fe34dffb700 1 -- 172.21.15.64:0/1080342466 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 70) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe36000f580 con 0x55d74dcc9300 2023-09-13T03:27:13.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.287+0000 7fe383e95b80 10 client.27229 fetch_fsmap finished waiting for FSMap version 70 2023-09-13T03:27:13.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.287+0000 7fe383e95b80 10 client.27229 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:27:13.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.287+0000 7fe383e95b80 20 client.27229 populate_metadata read hostname 'smithi064' 2023-09-13T03:27:13.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.287+0000 7fe383e95b80 1 --2- 172.21.15.64:0/1080342466 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55d74dd497f0 0x55d74dd4bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:13.295 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.287+0000 7fe383e95b80 4 client.27229 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:27:13.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.287+0000 7fe383e95b80 1 -- 172.21.15.64:0/1080342466 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d74dc148c0 con 0x55d74dd497f0 2023-09-13T03:27:13.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.288+0000 7fe36db9a700 1 --2- 172.21.15.64:0/1080342466 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55d74dd497f0 0x55d74dd4bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:13.296 INFO:tasks.workunit.client.0.smithi064.stdout: 10 62663 45.97 MB/sec warmup 77 sec latency 7350.259 ms 2023-09-13T03:27:13.296 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.288+0000 7fe34dffb700 10 client.27229 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:27:13.297 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.288+0000 7fe36db9a700 1 --2- 172.21.15.64:0/1080342466 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55d74dd497f0 0x55d74dd4bc10 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb57900f160ab267 server_cookie=510db8adf379c609 in_seq=0 out_seq=0 2023-09-13T03:27:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.425+0000 7fe34dffb700 1 -- 172.21.15.64:0/1080342466 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d74dc148c0 con 0x55d74dd497f0 2023-09-13T03:27:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.425+0000 7fe34dffb700 10 client.27229 handle_command_reply: tid=0 2023-09-13T03:27:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.425+0000 7fe383e95b80 1 client.27229 shutdown 2023-09-13T03:27:13.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.427+0000 7fe383e95b80 1 -- 172.21.15.64:0/1080342466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d74dcc9300 msgr2=0x55d74dcd8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:27:13.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.427+0000 7fe383e95b80 1 --2- 172.21.15.64:0/1080342466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d74dcc9300 0x55d74dcd8bf0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe360004080 tx=0x7fe36003a420 comp rx=0 tx=0).stop 2023-09-13T03:27:13.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.427+0000 7fe383e95b80 1 -- 172.21.15.64:0/1080342466 shutdown_connections 2023-09-13T03:27:13.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.427+0000 7fe383e95b80 1 --2- 172.21.15.64:0/1080342466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d74dcd0df0 0x55d74dcd9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:13.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.427+0000 7fe383e95b80 1 --2- 172.21.15.64:0/1080342466 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55d74dd497f0 0x55d74dd4bc10 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.427+0000 7fe383e95b80 1 --2- 172.21.15.64:0/1080342466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d74dcc9300 0x55d74dcd8bf0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.427+0000 7fe383e95b80 1 --2- 172.21.15.64:0/1080342466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d74dccf1a0 0x55d74dcd9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.427+0000 7fe383e95b80 1 -- 172.21.15.64:0/1080342466 >> 172.21.15.64:0/1080342466 conn(0x55d74dc3e030 msgr2=0x55d74dcc8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:27:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.427+0000 7fe34dffb700 0 client.27229 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:27:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.427+0000 7fe383e95b80 1 -- 172.21.15.64:0/1080342466 shutdown_connections 2023-09-13T03:27:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.428+0000 7fe383e95b80 1 -- 172.21.15.64:0/1080342466 wait complete. 2023-09-13T03:27:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.430+0000 7fe383e95b80 20 client.27229 trim_cache size 0 max 16384 2023-09-13T03:27:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:27:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:27:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:27:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:27:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.432+0000 7fe3705ff700 1 -- 172.21.15.64:0/4133593054 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fe3500a08b0 msgr2=0x7fe3500a2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:27:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.432+0000 7fe3705ff700 1 --2- 172.21.15.64:0/4133593054 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fe3500a08b0 0x7fe3500a2d70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe360009990 tx=0x7fe360005fb0 comp rx=0 tx=0).stop 2023-09-13T03:27:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.432+0000 7fe3705ff700 1 -- 172.21.15.64:0/4133593054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe368100f80 msgr2=0x7fe368072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:27:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.432+0000 7fe3705ff700 1 --2- 172.21.15.64:0/4133593054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe368100f80 0x7fe368072e40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe35c0060b0 tx=0x7fe35c00d3b0 comp rx=0 tx=0).stop 2023-09-13T03:27:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.436+0000 7fe3705ff700 1 -- 172.21.15.64:0/4133593054 shutdown_connections 2023-09-13T03:27:13.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.436+0000 7fe3705ff700 1 --2- 172.21.15.64:0/4133593054 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fe3500a08b0 0x7fe3500a2d70 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:13.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.436+0000 7fe3705ff700 1 --2- 172.21.15.64:0/4133593054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe368106680 0x7fe368073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:13.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.436+0000 7fe3705ff700 1 --2- 172.21.15.64:0/4133593054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3681005c0 0x7fe368072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:13.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.436+0000 7fe3705ff700 1 --2- 172.21.15.64:0/4133593054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe368100f80 0x7fe368072e40 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.436+0000 7fe3705ff700 1 -- 172.21.15.64:0/4133593054 >> 172.21.15.64:0/4133593054 conn(0x7fe3680fc0b0 msgr2=0x7fe368104250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:27:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.436+0000 7fe3705ff700 1 -- 172.21.15.64:0/4133593054 shutdown_connections 2023-09-13T03:27:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:13.436+0000 7fe3705ff700 1 -- 172.21.15.64:0/4133593054 wait complete. 2023-09-13T03:27:14.287 INFO:tasks.workunit.client.0.smithi064.stdout: 10 63694 46.34 MB/sec warmup 78 sec latency 8350.337 ms 2023-09-13T03:27:15.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:27:14 smithi170 ceph-mon[134328]: pgmap v441: 97 pgs: 97 active+clean; 648 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.0 MiB/s wr, 1.02k op/s 2023-09-13T03:27:15.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:27:14 smithi064 ceph-mon[97675]: pgmap v441: 97 pgs: 97 active+clean; 648 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.0 MiB/s wr, 1.02k op/s 2023-09-13T03:27:15.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:27:14 smithi142 ceph-mon[134082]: pgmap v441: 97 pgs: 97 active+clean; 648 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.0 MiB/s wr, 1.02k op/s 2023-09-13T03:27:15.287 INFO:tasks.workunit.client.0.smithi064.stdout: 10 64421 46.17 MB/sec warmup 79 sec latency 9350.385 ms 2023-09-13T03:27:16.287 INFO:tasks.workunit.client.0.smithi064.stdout: 10 65213 46.05 MB/sec warmup 80 sec latency 9697.082 ms 2023-09-13T03:27:17.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:27:16 smithi170 ceph-mon[134328]: pgmap v442: 97 pgs: 97 active+clean; 679 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.2 MiB/s wr, 811 op/s 2023-09-13T03:27:17.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:27:16 smithi064 ceph-mon[97675]: pgmap v442: 97 pgs: 97 active+clean; 679 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.2 MiB/s wr, 811 op/s 2023-09-13T03:27:17.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:27:16 smithi142 ceph-mon[134082]: pgmap v442: 97 pgs: 97 active+clean; 679 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.2 MiB/s wr, 811 op/s 2023-09-13T03:27:17.287 INFO:tasks.workunit.client.0.smithi064.stdout: 10 66312 46.25 MB/sec warmup 81 sec latency 131.385 ms 2023-09-13T03:27:18.287 INFO:tasks.workunit.client.0.smithi064.stdout: 10 67275 46.34 MB/sec warmup 82 sec latency 87.579 ms 2023-09-13T03:27:19.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:27:18 smithi170 ceph-mon[134328]: pgmap v443: 97 pgs: 97 active+clean; 679 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.2 MiB/s wr, 811 op/s 2023-09-13T03:27:19.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:27:18 smithi064 ceph-mon[97675]: pgmap v443: 97 pgs: 97 active+clean; 679 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.2 MiB/s wr, 811 op/s 2023-09-13T03:27:19.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:27:18 smithi142 ceph-mon[134082]: pgmap v443: 97 pgs: 97 active+clean; 679 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.2 MiB/s wr, 811 op/s 2023-09-13T03:27:19.287 INFO:tasks.workunit.client.0.smithi064.stdout: 10 68279 46.48 MB/sec warmup 83 sec latency 40.711 ms 2023-09-13T03:27:20.287 INFO:tasks.workunit.client.0.smithi064.stdout: 10 69348 46.57 MB/sec warmup 84 sec latency 38.690 ms 2023-09-13T03:27:20.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:27:20 smithi142 ceph-mon[134082]: pgmap v444: 97 pgs: 97 active+clean; 679 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.9 MiB/s wr, 789 op/s 2023-09-13T03:27:20.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:27:20 smithi170 ceph-mon[134328]: pgmap v444: 97 pgs: 97 active+clean; 679 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.9 MiB/s wr, 789 op/s 2023-09-13T03:27:20.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:27:20 smithi064 ceph-mon[97675]: pgmap v444: 97 pgs: 97 active+clean; 679 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.9 MiB/s wr, 789 op/s 2023-09-13T03:27:21.288 INFO:tasks.workunit.client.0.smithi064.stdout: 10 70406 46.67 MB/sec warmup 85 sec latency 47.807 ms 2023-09-13T03:27:22.288 INFO:tasks.workunit.client.0.smithi064.stdout: 10 71460 46.88 MB/sec warmup 86 sec latency 55.650 ms 2023-09-13T03:27:23.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:27:22 smithi170 ceph-mon[134328]: pgmap v445: 97 pgs: 97 active+clean; 716 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 11 MiB/s wr, 1.02k op/s 2023-09-13T03:27:23.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:27:22 smithi064 ceph-mon[97675]: pgmap v445: 97 pgs: 97 active+clean; 716 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 11 MiB/s wr, 1.02k op/s 2023-09-13T03:27:23.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:27:22 smithi142 ceph-mon[134082]: pgmap v445: 97 pgs: 97 active+clean; 716 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 11 MiB/s wr, 1.02k op/s 2023-09-13T03:27:23.288 INFO:tasks.workunit.client.0.smithi064.stdout: 10 71991 46.69 MB/sec warmup 87 sec latency 195.025 ms 2023-09-13T03:27:24.288 INFO:tasks.workunit.client.0.smithi064.stdout: 10 72770 46.45 MB/sec warmup 88 sec latency 113.649 ms 2023-09-13T03:27:25.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:27:24 smithi170 ceph-mon[134328]: pgmap v446: 97 pgs: 97 active+clean; 716 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.9 MiB/s wr, 720 op/s 2023-09-13T03:27:25.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:27:24 smithi064 ceph-mon[97675]: pgmap v446: 97 pgs: 97 active+clean; 716 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.9 MiB/s wr, 720 op/s 2023-09-13T03:27:25.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:27:24 smithi142 ceph-mon[134082]: pgmap v446: 97 pgs: 97 active+clean; 716 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.9 MiB/s wr, 720 op/s 2023-09-13T03:27:25.288 INFO:tasks.workunit.client.0.smithi064.stdout: 10 74090 46.73 MB/sec warmup 89 sec latency 37.696 ms 2023-09-13T03:27:26.288 INFO:tasks.workunit.client.0.smithi064.stdout: 10 75085 47.02 MB/sec warmup 90 sec latency 47.794 ms 2023-09-13T03:27:26.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.614+0000 7f36ed39d700 1 -- 172.21.15.64:0/1027705777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36e8106480 msgr2=0x7f36e8106860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:27:26.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.614+0000 7f36ed39d700 1 --2- 172.21.15.64:0/1027705777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36e8106480 0x7f36e8106860 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f36d8009990 tx=0x7f36d8009ca0 comp rx=0 tx=0).stop 2023-09-13T03:27:26.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.614+0000 7f36ed39d700 1 -- 172.21.15.64:0/1027705777 shutdown_connections 2023-09-13T03:27:26.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.614+0000 7f36ed39d700 1 --2- 172.21.15.64:0/1027705777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36e8106480 0x7f36e8106860 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:26.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.614+0000 7f36ed39d700 1 --2- 172.21.15.64:0/1027705777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36e80683b0 0x7f36e810d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:26.616 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.614+0000 7f36ed39d700 1 --2- 172.21.15.64:0/1027705777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36e80679f0 0x7f36e8067e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.614+0000 7f36ed39d700 1 -- 172.21.15.64:0/1027705777 >> 172.21.15.64:0/1027705777 conn(0x7f36e8074dc0 msgr2=0x7f36e80751d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:27:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.614+0000 7f36ed39d700 1 -- 172.21.15.64:0/1027705777 shutdown_connections 2023-09-13T03:27:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.614+0000 7f36ed39d700 1 -- 172.21.15.64:0/1027705777 wait complete. 2023-09-13T03:27:26.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.615+0000 7f36ed39d700 1 Processor -- start 2023-09-13T03:27:26.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.616+0000 7f36ed39d700 1 -- start start 2023-09-13T03:27:26.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.616+0000 7f36ed39d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36e80679f0 0x7f36e80ff590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:26.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.616+0000 7f36ed39d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36e80683b0 0x7f36e80ffad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:26.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.616+0000 7f36ed39d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36e8106480 0x7f36e81a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:26.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.616+0000 7f36ed39d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e8100300 con 0x7f36e80679f0 2023-09-13T03:27:26.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.616+0000 7f36ed39d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e81a4b80 con 0x7f36e80683b0 2023-09-13T03:27:26.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.616+0000 7f36ed39d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e81a4cc0 con 0x7f36e8106480 2023-09-13T03:27:26.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.616+0000 7f36e77fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36e80683b0 0x7f36e80ffad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:26.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.616+0000 7f36e7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36e80679f0 0x7f36e80ff590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:26.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.616+0000 7f36e77fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36e80683b0 0x7f36e80ffad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:58620/0 (socket says 172.21.15.64:58620) 2023-09-13T03:27:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.616+0000 7f36e77fe700 1 -- 172.21.15.64:0/2567807943 learned_addr learned my addr 172.21.15.64:0/2567807943 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:27:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.616+0000 7f36ecb9c700 1 --2- 172.21.15.64:0/2567807943 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36e8106480 0x7f36e81a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.617+0000 7f36e77fe700 1 -- 172.21.15.64:0/2567807943 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36e8106480 msgr2=0x7f36e81a4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:27:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.617+0000 7f36e77fe700 1 --2- 172.21.15.64:0/2567807943 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36e8106480 0x7f36e81a4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:26.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.617+0000 7f36e77fe700 1 -- 172.21.15.64:0/2567807943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36e80679f0 msgr2=0x7f36e80ff590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:27:26.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.617+0000 7f36e77fe700 1 --2- 172.21.15.64:0/2567807943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36e80679f0 0x7f36e80ff590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:26.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.617+0000 7f36e77fe700 1 -- 172.21.15.64:0/2567807943 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d80095f0 con 0x7f36e80683b0 2023-09-13T03:27:26.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.617+0000 7f36e77fe700 1 --2- 172.21.15.64:0/2567807943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36e80683b0 0x7f36e80ffad0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f36dc00e9a0 tx=0x7f36dc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:27:26.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.617+0000 7f36e57fa700 1 -- 172.21.15.64:0/2567807943 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36dc009790 con 0x7f36e80683b0 2023-09-13T03:27:26.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.617+0000 7f36e57fa700 1 -- 172.21.15.64:0/2567807943 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36dc010430 con 0x7f36e80683b0 2023-09-13T03:27:26.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.617+0000 7f36ed39d700 1 -- 172.21.15.64:0/2567807943 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36e81a4f20 con 0x7f36e80683b0 2023-09-13T03:27:26.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.618+0000 7f36e57fa700 1 -- 172.21.15.64:0/2567807943 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36dc018680 con 0x7f36e80683b0 2023-09-13T03:27:26.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.618+0000 7f36ed39d700 1 -- 172.21.15.64:0/2567807943 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36e81a53f0 con 0x7f36e80683b0 2023-09-13T03:27:26.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.622+0000 7f36e57fa700 1 -- 172.21.15.64:0/2567807943 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f36dc0105a0 con 0x7f36e80683b0 2023-09-13T03:27:26.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.622+0000 7f3701fe1b80 1 Processor -- start 2023-09-13T03:27:26.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.623+0000 7f3701fe1b80 1 -- start start 2023-09-13T03:27:26.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.623+0000 7f3701fe1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fdad2b290 0x559fdad2b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:26.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.623+0000 7f3701fe1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559fdad2bbb0 0x559fdad25460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:26.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.623+0000 7f3701fe1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fdad259a0 0x559fdad25de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.623+0000 7f3701fe1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fdac7e6d0 con 0x559fdad259a0 2023-09-13T03:27:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.623+0000 7f3701fe1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fdac7bff0 con 0x559fdad2b290 2023-09-13T03:27:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.623+0000 7f3701fe1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fdac778e0 con 0x559fdad2bbb0 2023-09-13T03:27:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.623+0000 7f36ecb9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fdad2b290 0x559fdad2b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.623+0000 7f36ecb9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fdad2b290 0x559fdad2b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:58624/0 (socket says 172.21.15.64:58624) 2023-09-13T03:27:26.631 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.624+0000 7f36ecb9c700 1 -- 172.21.15.64:0/734582175 learned_addr learned my addr 172.21.15.64:0/734582175 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:27:26.631 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.624+0000 7f36e7fff700 1 --2- 172.21.15.64:0/734582175 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559fdad2bbb0 0x559fdad25460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:26.631 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.624+0000 7f36e77fe700 1 --2- 172.21.15.64:0/734582175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fdad259a0 0x559fdad25de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:26.631 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.624+0000 7f36ecb9c700 1 -- 172.21.15.64:0/734582175 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559fdad2bbb0 msgr2=0x559fdad25460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:27:26.631 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.624+0000 7f36ecb9c700 1 --2- 172.21.15.64:0/734582175 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559fdad2bbb0 0x559fdad25460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:26.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.624+0000 7f36ecb9c700 1 -- 172.21.15.64:0/734582175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fdad259a0 msgr2=0x559fdad25de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:27:26.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.624+0000 7f36ecb9c700 1 --2- 172.21.15.64:0/734582175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fdad259a0 0x559fdad25de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:26.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.624+0000 7f36ecb9c700 1 -- 172.21.15.64:0/734582175 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559fdaaa3d80 con 0x559fdad2b290 2023-09-13T03:27:26.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.624+0000 7f36e7fff700 1 --2- 172.21.15.64:0/734582175 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559fdad2bbb0 0x559fdad25460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:27:26.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.624+0000 7f36e57fa700 1 --2- 172.21.15.64:0/2567807943 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f36d00a0830 0x7f36d00a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:26.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.624+0000 7f36e57fa700 1 -- 172.21.15.64:0/2567807943 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f36dc014070 con 0x7f36e80683b0 2023-09-13T03:27:26.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.624+0000 7f36e77fe700 1 --2- 172.21.15.64:0/734582175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fdad259a0 0x559fdad25de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T03:27:26.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.624+0000 7f36ecb9c700 1 --2- 172.21.15.64:0/734582175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fdad2b290 0x559fdad2b670 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f36e0011600 tx=0x7f36e0009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c69988dd796e44bb server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:27:26.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.624+0000 7f36e7fff700 1 --2- 172.21.15.64:0/2567807943 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f36d00a0830 0x7f36d00a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:26.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.625+0000 7f36ce7fc700 1 -- 172.21.15.64:0/734582175 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e000e040 con 0x559fdad2b290 2023-09-13T03:27:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.625+0000 7f36ce7fc700 1 -- 172.21.15.64:0/734582175 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36e000f040 con 0x559fdad2b290 2023-09-13T03:27:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.625+0000 7f36ce7fc700 1 -- 172.21.15.64:0/734582175 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e0004870 con 0x559fdad2b290 2023-09-13T03:27:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.626+0000 7f36e7fff700 1 --2- 172.21.15.64:0/2567807943 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f36d00a0830 0x7f36d00a2cf0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f36d8000c00 tx=0x7f36d8005960 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:27:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.626+0000 7f3701fe1b80 1 -- 172.21.15.64:0/734582175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fdad2b290 msgr2=0x559fdad2b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:27:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.626+0000 7f3701fe1b80 1 --2- 172.21.15.64:0/734582175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fdad2b290 0x559fdad2b670 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f36e0011600 tx=0x7f36e0009920 comp rx=0 tx=0).stop 2023-09-13T03:27:26.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.626+0000 7f3701fe1b80 1 -- 172.21.15.64:0/734582175 shutdown_connections 2023-09-13T03:27:26.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.626+0000 7f3701fe1b80 1 --2- 172.21.15.64:0/734582175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fdad2b290 0x559fdad2b670 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:26.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.626+0000 7f3701fe1b80 1 --2- 172.21.15.64:0/734582175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fdad259a0 0x559fdad25de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:26.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.626+0000 7f3701fe1b80 1 --2- 172.21.15.64:0/734582175 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559fdad2bbb0 0x559fdad25460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:26.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.626+0000 7f3701fe1b80 1 -- 172.21.15.64:0/734582175 >> 172.21.15.64:0/734582175 conn(0x559fdac9a030 msgr2=0x559fdad206c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:27:26.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.626+0000 7f3701fe1b80 1 -- 172.21.15.64:0/734582175 shutdown_connections 2023-09-13T03:27:26.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.626+0000 7f3701fe1b80 1 -- 172.21.15.64:0/734582175 wait complete. 2023-09-13T03:27:26.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.627+0000 7f3701fe1b80 1 -- start start 2023-09-13T03:27:26.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.627+0000 7f3701fe1b80 1 Processor -- start 2023-09-13T03:27:26.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.627+0000 7f3701fe1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fdad259a0 0x559fdad31bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:26.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.627+0000 7f3701fe1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559fdad2b290 0x559fdad2ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:26.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.627+0000 7f3701fe1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fdad2bbb0 0x559fdad2f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:26.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.627+0000 7f3701fe1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fdac15b10 con 0x559fdad2bbb0 2023-09-13T03:27:26.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.627+0000 7f3701fe1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fdac15810 con 0x559fdad259a0 2023-09-13T03:27:26.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.627+0000 7f3701fe1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fdaaf98c0 con 0x559fdad2b290 2023-09-13T03:27:26.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.627+0000 7f3701fe1b80 10 client.0 _ll_register_callbacks cb 0x559fdab5fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:27:26.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.628+0000 7f36ecb9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fdad2bbb0 0x559fdad2f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:26.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.628+0000 7f36ecb9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fdad2bbb0 0x559fdad2f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46456/0 (socket says 172.21.15.64:46456) 2023-09-13T03:27:26.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.628+0000 7f36e77fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559fdad2b290 0x559fdad2ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:26.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.628+0000 7f36ecb9c700 1 -- 172.21.15.64:0/216892750 learned_addr learned my addr 172.21.15.64:0/216892750 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:27:26.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.628+0000 7f36ecb9c700 1 -- 172.21.15.64:0/216892750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559fdad2b290 msgr2=0x559fdad2ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:27:26.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.628+0000 7f36ecb9c700 1 --2- 172.21.15.64:0/216892750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559fdad2b290 0x559fdad2ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:26.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.628+0000 7f36ecb9c700 1 -- 172.21.15.64:0/216892750 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fdad259a0 msgr2=0x559fdad31bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:27:26.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.628+0000 7f36ecb9c700 1 --2- 172.21.15.64:0/216892750 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fdad259a0 0x559fdad31bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:26.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.628+0000 7f36ecb9c700 1 -- 172.21.15.64:0/216892750 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36e00112b0 con 0x559fdad2bbb0 2023-09-13T03:27:26.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.629+0000 7f36ecb9c700 1 --2- 172.21.15.64:0/216892750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fdad2bbb0 0x559fdad2f540 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f36e0003c60 tx=0x7f36e0003c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d4f0a874b3ebe29 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:27:26.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.629+0000 7f36c77fe700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:27:26.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.629+0000 7f36c77fe700 1 -- 172.21.15.64:0/216892750 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e000e030 con 0x559fdad2bbb0 2023-09-13T03:27:26.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.629+0000 7f3701fe1b80 1 -- 172.21.15.64:0/216892750 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559fdabcf490 con 0x559fdad2bbb0 2023-09-13T03:27:26.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.629+0000 7f36c77fe700 1 -- 172.21.15.64:0/216892750 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f36e00092e0 con 0x559fdad2bbb0 2023-09-13T03:27:26.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.629+0000 7f36c77fe700 1 -- 172.21.15.64:0/216892750 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36e00048b0 con 0x559fdad2bbb0 2023-09-13T03:27:26.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.629+0000 7f3701fe1b80 10 client.17202 fetch_fsmap learned FSMap version 70 2023-09-13T03:27:26.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.629+0000 7f3701fe1b80 1 -- 172.21.15.64:0/216892750 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x559fdaa8cc00 con 0x559fdad2bbb0 2023-09-13T03:27:26.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.629+0000 7f36c77fe700 1 -- 172.21.15.64:0/216892750 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559fdabcf490 con 0x559fdad2bbb0 2023-09-13T03:27:26.643 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.630+0000 7f36c77fe700 1 -- 172.21.15.64:0/216892750 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 70) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f36e0024020 con 0x559fdad2bbb0 2023-09-13T03:27:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.630+0000 7f3701fe1b80 10 client.17202 fetch_fsmap finished waiting for FSMap version 70 2023-09-13T03:27:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.630+0000 7f3701fe1b80 10 client.17202 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:27:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.630+0000 7f3701fe1b80 20 client.17202 populate_metadata read hostname 'smithi064' 2023-09-13T03:27:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.630+0000 7f3701fe1b80 1 --2- 172.21.15.64:0/216892750 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x559fdad3f4a0 0x559fdad418c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.630+0000 7f3701fe1b80 4 client.17202 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:27:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.630+0000 7f3701fe1b80 1 -- 172.21.15.64:0/216892750 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559fdac8c840 con 0x559fdad3f4a0 2023-09-13T03:27:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.630+0000 7f36e7fff700 1 --2- 172.21.15.64:0/216892750 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x559fdad3f4a0 0x559fdad418c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.631+0000 7f36c77fe700 10 client.17202 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:27:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.631+0000 7f36e7fff700 1 --2- 172.21.15.64:0/216892750 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x559fdad3f4a0 0x559fdad418c0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21b87c952714e7f5 server_cookie=6fc666c2d5f13844 in_seq=0 out_seq=0 2023-09-13T03:27:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.633+0000 7f36c77fe700 1 -- 172.21.15.64:0/216892750 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559fdac8c840 con 0x559fdad3f4a0 2023-09-13T03:27:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.633+0000 7f36c77fe700 10 client.17202 handle_command_reply: tid=0 2023-09-13T03:27:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.633+0000 7f3701fe1b80 1 client.17202 shutdown 2023-09-13T03:27:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.633+0000 7f3701fe1b80 1 -- 172.21.15.64:0/216892750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fdad2bbb0 msgr2=0x559fdad2f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:27:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.633+0000 7f3701fe1b80 1 --2- 172.21.15.64:0/216892750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fdad2bbb0 0x559fdad2f540 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f36e0003c60 tx=0x7f36e0003c90 comp rx=0 tx=0).stop 2023-09-13T03:27:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.633+0000 7f3701fe1b80 1 -- 172.21.15.64:0/216892750 shutdown_connections 2023-09-13T03:27:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.633+0000 7f3701fe1b80 1 --2- 172.21.15.64:0/216892750 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fdad259a0 0x559fdad31bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.633+0000 7f3701fe1b80 1 --2- 172.21.15.64:0/216892750 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x559fdad3f4a0 0x559fdad418c0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.633+0000 7f3701fe1b80 1 --2- 172.21.15.64:0/216892750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fdad2bbb0 0x559fdad2f540 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.633+0000 7f3701fe1b80 1 --2- 172.21.15.64:0/216892750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559fdad2b290 0x559fdad2ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.633+0000 7f3701fe1b80 1 -- 172.21.15.64:0/216892750 >> 172.21.15.64:0/216892750 conn(0x559fdac9a030 msgr2=0x559fdad1f900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:27:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.633+0000 7f36c77fe700 0 client.17202 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:27:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.633+0000 7f3701fe1b80 1 -- 172.21.15.64:0/216892750 shutdown_connections 2023-09-13T03:27:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.634+0000 7f3701fe1b80 1 -- 172.21.15.64:0/216892750 wait complete. 2023-09-13T03:27:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.634+0000 7f3701fe1b80 20 client.17202 trim_cache size 0 max 16384 2023-09-13T03:27:26.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.654+0000 7f3701fe1b80 1 Processor -- start 2023-09-13T03:27:26.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.654+0000 7f3701fe1b80 1 -- start start 2023-09-13T03:27:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.655+0000 7f3701fe1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fdad259a0 0x559fdad36610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.655+0000 7f3701fe1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559fdad2b290 0x559fdad2a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.655+0000 7f3701fe1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fdad2bbb0 0x559fdad2e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.655+0000 7f3701fe1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fdad2f8e0 con 0x559fdad2bbb0 2023-09-13T03:27:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.655+0000 7f3701fe1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fdad36b50 con 0x559fdad259a0 2023-09-13T03:27:26.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.655+0000 7f3701fe1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fdad2ae90 con 0x559fdad2b290 2023-09-13T03:27:26.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.655+0000 7f36e7fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fdad259a0 0x559fdad36610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:26.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.655+0000 7f36e7fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fdad259a0 0x559fdad36610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:58646/0 (socket says 172.21.15.64:58646) 2023-09-13T03:27:26.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.655+0000 7f36e7fff700 1 -- 172.21.15.64:0/3642487874 learned_addr learned my addr 172.21.15.64:0/3642487874 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:27:26.657 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.655+0000 7f36ecb9c700 1 --2- 172.21.15.64:0/3642487874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fdad2bbb0 0x559fdad2e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:26.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.656+0000 7f36e7fff700 1 -- 172.21.15.64:0/3642487874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559fdad2b290 msgr2=0x559fdad2a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:27:26.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.656+0000 7f36e7fff700 1 --2- 172.21.15.64:0/3642487874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559fdad2b290 0x559fdad2a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:26.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.656+0000 7f36e7fff700 1 -- 172.21.15.64:0/3642487874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fdad2bbb0 msgr2=0x559fdad2e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:27:26.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.656+0000 7f36e7fff700 1 --2- 172.21.15.64:0/3642487874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fdad2bbb0 0x559fdad2e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:26.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.656+0000 7f36e7fff700 1 -- 172.21.15.64:0/3642487874 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559fdaaca1f0 con 0x559fdad259a0 2023-09-13T03:27:26.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.656+0000 7f36e7fff700 1 --2- 172.21.15.64:0/3642487874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fdad259a0 0x559fdad36610 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f36d8006010 tx=0x7f36d8009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1f3ab94613f396 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:27:26.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.657+0000 7f36ceffd700 1 -- 172.21.15.64:0/3642487874 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d801d050 con 0x559fdad259a0 2023-09-13T03:27:26.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.657+0000 7f36ceffd700 1 -- 172.21.15.64:0/3642487874 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36d801e040 con 0x559fdad259a0 2023-09-13T03:27:26.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.657+0000 7f36ceffd700 1 -- 172.21.15.64:0/3642487874 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d8004d90 con 0x559fdad259a0 2023-09-13T03:27:26.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.657+0000 7f3701fe1b80 1 -- 172.21.15.64:0/3642487874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fdad259a0 msgr2=0x559fdad36610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:27:26.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.657+0000 7f3701fe1b80 1 --2- 172.21.15.64:0/3642487874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fdad259a0 0x559fdad36610 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f36d8006010 tx=0x7f36d8009310 comp rx=0 tx=0).stop 2023-09-13T03:27:26.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.658+0000 7f3701fe1b80 1 -- 172.21.15.64:0/3642487874 shutdown_connections 2023-09-13T03:27:26.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.658+0000 7f3701fe1b80 1 --2- 172.21.15.64:0/3642487874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fdad259a0 0x559fdad36610 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.658+0000 7f3701fe1b80 1 --2- 172.21.15.64:0/3642487874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fdad2bbb0 0x559fdad2e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.658+0000 7f3701fe1b80 1 --2- 172.21.15.64:0/3642487874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559fdad2b290 0x559fdad2a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.658+0000 7f3701fe1b80 1 -- 172.21.15.64:0/3642487874 >> 172.21.15.64:0/3642487874 conn(0x559fdac9a030 msgr2=0x559fdad29ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:27:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.658+0000 7f3701fe1b80 1 -- 172.21.15.64:0/3642487874 shutdown_connections 2023-09-13T03:27:26.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.658+0000 7f3701fe1b80 1 -- 172.21.15.64:0/3642487874 wait complete. 2023-09-13T03:27:26.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.658+0000 7f3701fe1b80 1 -- start start 2023-09-13T03:27:26.663 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.659+0000 7f3701fe1b80 1 Processor -- start 2023-09-13T03:27:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.659+0000 7f3701fe1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fdad259a0 0x559fdad282f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.659+0000 7f3701fe1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fdad2b290 0x559fdad26cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:26.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.659+0000 7f3701fe1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559fdad2bbb0 0x559fdad27230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:26.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.659+0000 7f3701fe1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fdad27770 con 0x559fdad259a0 2023-09-13T03:27:26.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.659+0000 7f3701fe1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fdad278e0 con 0x559fdad2b290 2023-09-13T03:27:26.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.659+0000 7f3701fe1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fdad27aa0 con 0x559fdad2bbb0 2023-09-13T03:27:26.666 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.659+0000 7f3701fe1b80 10 client.0 _ll_register_callbacks cb 0x7f36e00114d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:27:26.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.660+0000 7f36e7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fdad259a0 0x559fdad282f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:26.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.660+0000 7f36e7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fdad259a0 0x559fdad282f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46480/0 (socket says 172.21.15.64:46480) 2023-09-13T03:27:26.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.660+0000 7f36e7fff700 1 -- 172.21.15.64:0/1246461829 learned_addr learned my addr 172.21.15.64:0/1246461829 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:27:26.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.660+0000 7f36e77fe700 1 --2- 172.21.15.64:0/1246461829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fdad2b290 0x559fdad26cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:26.667 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.660+0000 7f36ecb9c700 1 --2- 172.21.15.64:0/1246461829 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559fdad2bbb0 0x559fdad27230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:26.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.660+0000 7f36e7fff700 1 -- 172.21.15.64:0/1246461829 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559fdad2bbb0 msgr2=0x559fdad27230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:27:26.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.660+0000 7f36e7fff700 1 --2- 172.21.15.64:0/1246461829 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559fdad2bbb0 0x559fdad27230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:26.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.660+0000 7f36e7fff700 1 -- 172.21.15.64:0/1246461829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fdad2b290 msgr2=0x559fdad26cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:27:26.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.660+0000 7f36e7fff700 1 --2- 172.21.15.64:0/1246461829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fdad2b290 0x559fdad26cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:26.668 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.660+0000 7f36e7fff700 1 -- 172.21.15.64:0/1246461829 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36e810db50 con 0x559fdad259a0 2023-09-13T03:27:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.661+0000 7f36e7fff700 1 --2- 172.21.15.64:0/1246461829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fdad259a0 0x559fdad282f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f36d800c010 tx=0x7f36d800b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a060a8b1d271a1 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:27:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.661+0000 7f36c77fe700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:27:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.661+0000 7f36c77fe700 1 -- 172.21.15.64:0/1246461829 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d803f070 con 0x559fdad259a0 2023-09-13T03:27:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.661+0000 7f36c77fe700 1 -- 172.21.15.64:0/1246461829 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f36d8004500 con 0x559fdad259a0 2023-09-13T03:27:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.661+0000 7f3701fe1b80 1 -- 172.21.15.64:0/1246461829 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f36e000f040 con 0x559fdad259a0 2023-09-13T03:27:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.662+0000 7f36c77fe700 1 -- 172.21.15.64:0/1246461829 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36d801d040 con 0x559fdad259a0 2023-09-13T03:27:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.662+0000 7f3701fe1b80 10 client.17208 fetch_fsmap learned FSMap version 70 2023-09-13T03:27:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.662+0000 7f3701fe1b80 1 -- 172.21.15.64:0/1246461829 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x559fdaaa2250 con 0x559fdad259a0 2023-09-13T03:27:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.662+0000 7f36c77fe700 1 -- 172.21.15.64:0/1246461829 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36e000f040 con 0x559fdad259a0 2023-09-13T03:27:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.662+0000 7f36c77fe700 1 -- 172.21.15.64:0/1246461829 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 70) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f36d803b470 con 0x559fdad259a0 2023-09-13T03:27:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.662+0000 7f3701fe1b80 10 client.17208 fetch_fsmap finished waiting for FSMap version 70 2023-09-13T03:27:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.662+0000 7f3701fe1b80 10 client.17208 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:27:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.662+0000 7f3701fe1b80 20 client.17208 populate_metadata read hostname 'smithi064' 2023-09-13T03:27:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.662+0000 7f3701fe1b80 1 --2- 172.21.15.64:0/1246461829 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x559fdad942e0 0x559fdad96700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.662+0000 7f3701fe1b80 4 client.17208 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:27:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.662+0000 7f3701fe1b80 1 -- 172.21.15.64:0/1246461829 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559fdac708c0 con 0x559fdad942e0 2023-09-13T03:27:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.663+0000 7f36e77fe700 1 --2- 172.21.15.64:0/1246461829 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x559fdad942e0 0x559fdad96700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.663+0000 7f36e77fe700 1 --2- 172.21.15.64:0/1246461829 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x559fdad942e0 0x559fdad96700 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1935da59d84d8787 server_cookie=8485eaedd5efc4a1 in_seq=0 out_seq=0 2023-09-13T03:27:26.673 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.664+0000 7f36c77fe700 10 client.17208 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:27:26.876 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.875+0000 7f36c77fe700 1 -- 172.21.15.64:0/1246461829 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559fdac708c0 con 0x559fdad942e0 2023-09-13T03:27:26.877 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.875+0000 7f36c77fe700 10 client.17208 handle_command_reply: tid=0 2023-09-13T03:27:26.877 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.875+0000 7f3701fe1b80 1 client.17208 shutdown 2023-09-13T03:27:26.877 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.876+0000 7f3701fe1b80 1 -- 172.21.15.64:0/1246461829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fdad259a0 msgr2=0x559fdad282f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:27:26.877 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.876+0000 7f3701fe1b80 1 --2- 172.21.15.64:0/1246461829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fdad259a0 0x559fdad282f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f36d800c010 tx=0x7f36d800b9d0 comp rx=0 tx=0).stop 2023-09-13T03:27:26.878 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.876+0000 7f3701fe1b80 1 -- 172.21.15.64:0/1246461829 shutdown_connections 2023-09-13T03:27:26.878 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.876+0000 7f3701fe1b80 1 --2- 172.21.15.64:0/1246461829 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x559fdad942e0 0x559fdad96700 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:26.878 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.876+0000 7f3701fe1b80 1 --2- 172.21.15.64:0/1246461829 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559fdad2bbb0 0x559fdad27230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:26.878 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.876+0000 7f3701fe1b80 1 --2- 172.21.15.64:0/1246461829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fdad259a0 0x559fdad282f0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:26.878 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.876+0000 7f36c77fe700 0 client.17208 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:27:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.876+0000 7f3701fe1b80 1 --2- 172.21.15.64:0/1246461829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fdad2b290 0x559fdad26cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.876+0000 7f3701fe1b80 1 -- 172.21.15.64:0/1246461829 >> 172.21.15.64:0/1246461829 conn(0x559fdac9a030 msgr2=0x559fdad2a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:27:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.876+0000 7f3701fe1b80 1 -- 172.21.15.64:0/1246461829 shutdown_connections 2023-09-13T03:27:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.876+0000 7f3701fe1b80 1 -- 172.21.15.64:0/1246461829 wait complete. 2023-09-13T03:27:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.876+0000 7f3701fe1b80 20 client.17208 trim_cache size 0 max 16384 2023-09-13T03:27:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:27:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:27:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:27:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:27:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.877+0000 7f36ed39d700 1 -- 172.21.15.64:0/2567807943 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f36d00a0830 msgr2=0x7f36d00a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:27:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.877+0000 7f36ed39d700 1 --2- 172.21.15.64:0/2567807943 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f36d00a0830 0x7f36d00a2cf0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f36d8000c00 tx=0x7f36d8005960 comp rx=0 tx=0).stop 2023-09-13T03:27:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.877+0000 7f36ed39d700 1 -- 172.21.15.64:0/2567807943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36e80683b0 msgr2=0x7f36e80ffad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:27:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.877+0000 7f36ed39d700 1 --2- 172.21.15.64:0/2567807943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36e80683b0 0x7f36e80ffad0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f36dc00e9a0 tx=0x7f36dc00ecb0 comp rx=0 tx=0).stop 2023-09-13T03:27:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.877+0000 7f36ed39d700 1 -- 172.21.15.64:0/2567807943 shutdown_connections 2023-09-13T03:27:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.877+0000 7f36ed39d700 1 --2- 172.21.15.64:0/2567807943 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f36d00a0830 0x7f36d00a2cf0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.877+0000 7f36ed39d700 1 --2- 172.21.15.64:0/2567807943 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36e8106480 0x7f36e81a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.877+0000 7f36ed39d700 1 --2- 172.21.15.64:0/2567807943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36e80679f0 0x7f36e80ff590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.877+0000 7f36ed39d700 1 --2- 172.21.15.64:0/2567807943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36e80683b0 0x7f36e80ffad0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.877+0000 7f36ed39d700 1 -- 172.21.15.64:0/2567807943 >> 172.21.15.64:0/2567807943 conn(0x7f36e8074dc0 msgr2=0x7f36e80fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:27:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.877+0000 7f36ed39d700 1 -- 172.21.15.64:0/2567807943 shutdown_connections 2023-09-13T03:27:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:26.877+0000 7f36ed39d700 1 -- 172.21.15.64:0/2567807943 wait complete. 2023-09-13T03:27:27.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:27:26 smithi170 ceph-mon[134328]: pgmap v447: 97 pgs: 97 active+clean; 747 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 14 MiB/s wr, 1.12k op/s 2023-09-13T03:27:27.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:27:26 smithi064 ceph-mon[97675]: pgmap v447: 97 pgs: 97 active+clean; 747 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 14 MiB/s wr, 1.12k op/s 2023-09-13T03:27:27.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:27:26 smithi142 ceph-mon[134082]: pgmap v447: 97 pgs: 97 active+clean; 747 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 14 MiB/s wr, 1.12k op/s 2023-09-13T03:27:27.288 INFO:tasks.workunit.client.0.smithi064.stdout: 10 75618 46.73 MB/sec warmup 91 sec latency 140.359 ms 2023-09-13T03:27:28.289 INFO:tasks.workunit.client.0.smithi064.stdout: 10 76162 46.52 MB/sec warmup 92 sec latency 233.076 ms 2023-09-13T03:27:29.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:27:28 smithi170 ceph-mon[134328]: pgmap v448: 97 pgs: 97 active+clean; 747 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 745 op/s 2023-09-13T03:27:29.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:27:28 smithi064 ceph-mon[97675]: pgmap v448: 97 pgs: 97 active+clean; 747 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 745 op/s 2023-09-13T03:27:29.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:27:28 smithi142 ceph-mon[134082]: pgmap v448: 97 pgs: 97 active+clean; 747 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 745 op/s 2023-09-13T03:27:29.289 INFO:tasks.workunit.client.0.smithi064.stdout: 10 77214 46.55 MB/sec warmup 93 sec latency 60.255 ms 2023-09-13T03:27:30.289 INFO:tasks.workunit.client.0.smithi064.stdout: 10 78346 46.79 MB/sec warmup 94 sec latency 50.001 ms 2023-09-13T03:27:30.746 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:27:30 smithi142 ceph-mon[134082]: pgmap v449: 97 pgs: 97 active+clean; 747 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 745 op/s 2023-09-13T03:27:30.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:27:30 smithi170 ceph-mon[134328]: pgmap v449: 97 pgs: 97 active+clean; 747 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 745 op/s 2023-09-13T03:27:30.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:27:30 smithi064 ceph-mon[97675]: pgmap v449: 97 pgs: 97 active+clean; 747 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 745 op/s 2023-09-13T03:27:31.289 INFO:tasks.workunit.client.0.smithi064.stdout: 10 79170 46.74 MB/sec warmup 95 sec latency 56.498 ms 2023-09-13T03:27:32.289 INFO:tasks.workunit.client.0.smithi064.stdout: 10 80265 46.90 MB/sec warmup 96 sec latency 62.255 ms 2023-09-13T03:27:33.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:27:32 smithi170 ceph-mon[134328]: pgmap v450: 97 pgs: 97 active+clean; 768 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 17 MiB/s wr, 1.23k op/s 2023-09-13T03:27:33.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:27:32 smithi064 ceph-mon[97675]: pgmap v450: 97 pgs: 97 active+clean; 768 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 17 MiB/s wr, 1.23k op/s 2023-09-13T03:27:33.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:27:32 smithi142 ceph-mon[134082]: pgmap v450: 97 pgs: 97 active+clean; 768 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 17 MiB/s wr, 1.23k op/s 2023-09-13T03:27:33.289 INFO:tasks.workunit.client.0.smithi064.stdout: 10 81285 46.98 MB/sec warmup 97 sec latency 45.503 ms 2023-09-13T03:27:34.290 INFO:tasks.workunit.client.0.smithi064.stdout: 10 81931 46.81 MB/sec warmup 98 sec latency 176.439 ms 2023-09-13T03:27:35.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:27:34 smithi170 ceph-mon[134328]: pgmap v451: 97 pgs: 97 active+clean; 768 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 886 op/s 2023-09-13T03:27:35.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:27:34 smithi064 ceph-mon[97675]: pgmap v451: 97 pgs: 97 active+clean; 768 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 886 op/s 2023-09-13T03:27:35.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:27:34 smithi142 ceph-mon[134082]: pgmap v451: 97 pgs: 97 active+clean; 768 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 886 op/s 2023-09-13T03:27:35.290 INFO:tasks.workunit.client.0.smithi064.stdout: 10 82944 46.92 MB/sec warmup 99 sec latency 66.955 ms 2023-09-13T03:27:36.290 INFO:tasks.workunit.client.0.smithi064.stdout: 10 83851 46.92 MB/sec warmup 100 sec latency 71.583 ms 2023-09-13T03:27:37.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:27:36 smithi170 ceph-mon[134328]: pgmap v452: 97 pgs: 97 active+clean; 797 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 19 MiB/s wr, 1.29k op/s 2023-09-13T03:27:37.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:27:36 smithi064 ceph-mon[97675]: pgmap v452: 97 pgs: 97 active+clean; 797 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 19 MiB/s wr, 1.29k op/s 2023-09-13T03:27:37.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:27:36 smithi142 ceph-mon[134082]: pgmap v452: 97 pgs: 97 active+clean; 797 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 19 MiB/s wr, 1.29k op/s 2023-09-13T03:27:37.290 INFO:tasks.workunit.client.0.smithi064.stdout: 10 84994 47.05 MB/sec warmup 101 sec latency 38.098 ms 2023-09-13T03:27:38.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:27:37 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:27:38.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:27:37 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:27:38.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:27:37 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:27:38.290 INFO:tasks.workunit.client.0.smithi064.stdout: 10 85922 47.14 MB/sec warmup 102 sec latency 91.319 ms 2023-09-13T03:27:39.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:27:38 smithi170 ceph-mon[134328]: pgmap v453: 97 pgs: 97 active+clean; 797 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 891 op/s 2023-09-13T03:27:39.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:27:38 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:27:39.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:27:38 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:27:39.162 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:27:38 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:27:39.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:27:38 smithi064 ceph-mon[97675]: pgmap v453: 97 pgs: 97 active+clean; 797 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 891 op/s 2023-09-13T03:27:39.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:27:38 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:27:39.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:27:38 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:27:39.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:27:38 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:27:39.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:27:38 smithi142 ceph-mon[134082]: pgmap v453: 97 pgs: 97 active+clean; 797 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 891 op/s 2023-09-13T03:27:39.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:27:38 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:27:39.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:27:38 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:27:39.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:27:38 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:27:39.291 INFO:tasks.workunit.client.0.smithi064.stdout: 10 86388 46.91 MB/sec warmup 103 sec latency 211.507 ms 2023-09-13T03:27:40.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.051+0000 7f145fbff700 1 -- 172.21.15.64:0/3137762982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1458106650 msgr2=0x7f1458106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:27:40.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.052+0000 7f145fbff700 1 --2- 172.21.15.64:0/3137762982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1458106650 0x7f1458106a30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f14540099c0 tx=0x7f1454009cd0 comp rx=0 tx=0).stop 2023-09-13T03:27:40.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.052+0000 7f145fbff700 1 -- 172.21.15.64:0/3137762982 shutdown_connections 2023-09-13T03:27:40.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.052+0000 7f145fbff700 1 --2- 172.21.15.64:0/3137762982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1458106650 0x7f1458106a30 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:40.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.052+0000 7f145fbff700 1 --2- 172.21.15.64:0/3137762982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14580683b0 0x7f145810d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:40.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.052+0000 7f145fbff700 1 --2- 172.21.15.64:0/3137762982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14580679f0 0x7f1458067e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:40.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.052+0000 7f145fbff700 1 -- 172.21.15.64:0/3137762982 >> 172.21.15.64:0/3137762982 conn(0x7f1458074f30 msgr2=0x7f1458075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:27:40.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.052+0000 7f145fbff700 1 -- 172.21.15.64:0/3137762982 shutdown_connections 2023-09-13T03:27:40.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.052+0000 7f145fbff700 1 -- 172.21.15.64:0/3137762982 wait complete. 2023-09-13T03:27:40.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.053+0000 7f145fbff700 1 Processor -- start 2023-09-13T03:27:40.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.053+0000 7f145fbff700 1 -- start start 2023-09-13T03:27:40.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.054+0000 7f145fbff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14580679f0 0x7f14581137f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:40.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.054+0000 7f145fbff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14580683b0 0x7f145810fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:40.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.054+0000 7f145fbff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1458106650 0x7f14581102c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:40.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.054+0000 7f145fbff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14581108c0 con 0x7f14580679f0 2023-09-13T03:27:40.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.054+0000 7f145fbff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1458110a30 con 0x7f14580683b0 2023-09-13T03:27:40.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.054+0000 7f145fbff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1458110bf0 con 0x7f1458106650 2023-09-13T03:27:40.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.054+0000 7f145d99b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14580679f0 0x7f14581137f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:40.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.054+0000 7f145d99b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14580679f0 0x7f14581137f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46360/0 (socket says 172.21.15.64:46360) 2023-09-13T03:27:40.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.055+0000 7f145d99b700 1 -- 172.21.15.64:0/3546952385 learned_addr learned my addr 172.21.15.64:0/3546952385 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:27:40.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.055+0000 7f145e19c700 1 --2- 172.21.15.64:0/3546952385 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1458106650 0x7f14581102c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:40.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.055+0000 7f145d19a700 1 --2- 172.21.15.64:0/3546952385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14580683b0 0x7f145810fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:40.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.055+0000 7f145d99b700 1 -- 172.21.15.64:0/3546952385 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1458106650 msgr2=0x7f14581102c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:27:40.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.055+0000 7f145d99b700 1 --2- 172.21.15.64:0/3546952385 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1458106650 0x7f14581102c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:40.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.055+0000 7f145d99b700 1 -- 172.21.15.64:0/3546952385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14580683b0 msgr2=0x7f145810fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:27:40.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.055+0000 7f145d99b700 1 --2- 172.21.15.64:0/3546952385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14580683b0 0x7f145810fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.055+0000 7f145d99b700 1 -- 172.21.15.64:0/3546952385 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14540095f0 con 0x7f14580679f0 2023-09-13T03:27:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.055+0000 7f145d99b700 1 --2- 172.21.15.64:0/3546952385 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14580679f0 0x7f14581137f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f145400b580 tx=0x7f145400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:27:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.056+0000 7f144effd700 1 -- 172.21.15.64:0/3546952385 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145401d070 con 0x7f14580679f0 2023-09-13T03:27:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.056+0000 7f144effd700 1 -- 172.21.15.64:0/3546952385 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1454005510 con 0x7f14580679f0 2023-09-13T03:27:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.056+0000 7f144effd700 1 -- 172.21.15.64:0/3546952385 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1454004a80 con 0x7f14580679f0 2023-09-13T03:27:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.056+0000 7f145fbff700 1 -- 172.21.15.64:0/3546952385 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14581a9760 con 0x7f14580679f0 2023-09-13T03:27:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.056+0000 7f145fbff700 1 -- 172.21.15.64:0/3546952385 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14581a9b30 con 0x7f14580679f0 2023-09-13T03:27:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.060+0000 7f144effd700 1 -- 172.21.15.64:0/3546952385 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f1454004c60 con 0x7f14580679f0 2023-09-13T03:27:40.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.062+0000 7f14735b1b80 1 Processor -- start 2023-09-13T03:27:40.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.063+0000 7f144effd700 1 --2- 172.21.15.64:0/3546952385 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f14440a0830 0x7f14440a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:40.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.063+0000 7f14735b1b80 1 -- start start 2023-09-13T03:27:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.063+0000 7f144effd700 1 -- 172.21.15.64:0/3546952385 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f14540ccdb0 con 0x7f14580679f0 2023-09-13T03:27:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.063+0000 7f145e19c700 1 --2- 172.21.15.64:0/3546952385 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f14440a0830 0x7f14440a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.063+0000 7f14735b1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a032c9a1a0 0x55a032c9a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.063+0000 7f14735b1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a032c94300 0x55a032c94780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.063+0000 7f14735b1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a032c94cc0 0x55a032ca0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.063+0000 7f14735b1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a032bed6d0 con 0x55a032c94cc0 2023-09-13T03:27:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.063+0000 7f14735b1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a032beaff0 con 0x55a032c9a1a0 2023-09-13T03:27:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.063+0000 7f14735b1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a032be68e0 con 0x55a032c94300 2023-09-13T03:27:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.064+0000 7f145d99b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a032c9a1a0 0x55a032c9a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.064+0000 7f145d99b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a032c9a1a0 0x55a032c9a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:59330/0 (socket says 172.21.15.64:59330) 2023-09-13T03:27:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.064+0000 7f145d99b700 1 -- 172.21.15.64:0/3025754019 learned_addr learned my addr 172.21.15.64:0/3025754019 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:27:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.064+0000 7f145e19c700 1 --2- 172.21.15.64:0/3025754019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a032c94cc0 0x55a032ca0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:40.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.064+0000 7f145e19c700 1 --2- 172.21.15.64:0/3546952385 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f14440a0830 0x7f14440a2cf0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f145000f590 tx=0x7f1450009920 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:27:40.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.064+0000 7f145d99b700 1 -- 172.21.15.64:0/3025754019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a032c94300 msgr2=0x55a032c94780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:27:40.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.064+0000 7f145d99b700 1 --2- 172.21.15.64:0/3025754019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a032c94300 0x55a032c94780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:40.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.064+0000 7f145d99b700 1 -- 172.21.15.64:0/3025754019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a032c94cc0 msgr2=0x55a032ca0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:27:40.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.064+0000 7f145d99b700 1 --2- 172.21.15.64:0/3025754019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a032c94cc0 0x55a032ca0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:40.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.064+0000 7f145d99b700 1 -- 172.21.15.64:0/3025754019 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a032a12d80 con 0x55a032c9a1a0 2023-09-13T03:27:40.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.064+0000 7f145e19c700 1 --2- 172.21.15.64:0/3025754019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a032c94cc0 0x55a032ca0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:27:40.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.065+0000 7f145d99b700 1 --2- 172.21.15.64:0/3025754019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a032c9a1a0 0x55a032c9a580 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1454009990 tx=0x7f1454099df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9871e0b756eb4375 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:27:40.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.065+0000 7f1443fff700 1 -- 172.21.15.64:0/3025754019 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145409a9b0 con 0x55a032c9a1a0 2023-09-13T03:27:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.065+0000 7f1443fff700 1 -- 172.21.15.64:0/3025754019 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f145409a2b0 con 0x55a032c9a1a0 2023-09-13T03:27:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.065+0000 7f1443fff700 1 -- 172.21.15.64:0/3025754019 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145409a510 con 0x55a032c9a1a0 2023-09-13T03:27:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.066+0000 7f14735b1b80 1 -- 172.21.15.64:0/3025754019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a032c9a1a0 msgr2=0x55a032c9a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:27:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.066+0000 7f14735b1b80 1 --2- 172.21.15.64:0/3025754019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a032c9a1a0 0x55a032c9a580 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1454009990 tx=0x7f1454099df0 comp rx=0 tx=0).stop 2023-09-13T03:27:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.066+0000 7f14735b1b80 1 -- 172.21.15.64:0/3025754019 shutdown_connections 2023-09-13T03:27:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.066+0000 7f14735b1b80 1 --2- 172.21.15.64:0/3025754019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a032c9a1a0 0x55a032c9a580 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.066+0000 7f14735b1b80 1 --2- 172.21.15.64:0/3025754019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a032c94cc0 0x55a032ca0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.066+0000 7f14735b1b80 1 --2- 172.21.15.64:0/3025754019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a032c94300 0x55a032c94780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.066+0000 7f14735b1b80 1 -- 172.21.15.64:0/3025754019 >> 172.21.15.64:0/3025754019 conn(0x55a032c09030 msgr2=0x55a032c8f560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:27:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.067+0000 7f14735b1b80 1 -- 172.21.15.64:0/3025754019 shutdown_connections 2023-09-13T03:27:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.067+0000 7f14735b1b80 1 -- 172.21.15.64:0/3025754019 wait complete. 2023-09-13T03:27:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.067+0000 7f14735b1b80 1 -- start start 2023-09-13T03:27:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.068+0000 7f14735b1b80 1 Processor -- start 2023-09-13T03:27:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.070+0000 7f14735b1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a032c94300 0x55a032ca3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.070+0000 7f14735b1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a032c94cc0 0x55a032ca4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.070+0000 7f14735b1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a032c9a1a0 0x55a032c9de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.070+0000 7f14735b1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a032b84b10 con 0x55a032c9a1a0 2023-09-13T03:27:40.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.070+0000 7f14735b1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a032b84810 con 0x55a032c94300 2023-09-13T03:27:40.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.070+0000 7f14735b1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a032a688c0 con 0x55a032c94cc0 2023-09-13T03:27:40.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.070+0000 7f14735b1b80 10 client.0 _ll_register_callbacks cb 0x55a032acea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:27:40.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.070+0000 7f145d99b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a032c94300 0x55a032ca3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:40.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.070+0000 7f145d99b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a032c94300 0x55a032ca3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:59336/0 (socket says 172.21.15.64:59336) 2023-09-13T03:27:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.070+0000 7f145d99b700 1 -- 172.21.15.64:0/2309257586 learned_addr learned my addr 172.21.15.64:0/2309257586 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:27:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.071+0000 7f145d99b700 1 -- 172.21.15.64:0/2309257586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a032c94cc0 msgr2=0x55a032ca4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:27:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.071+0000 7f145d99b700 1 --2- 172.21.15.64:0/2309257586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a032c94cc0 0x55a032ca4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.071+0000 7f145d99b700 1 -- 172.21.15.64:0/2309257586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a032c9a1a0 msgr2=0x55a032c9de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:27:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.071+0000 7f145d99b700 1 --2- 172.21.15.64:0/2309257586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a032c9a1a0 0x55a032c9de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.071+0000 7f145d99b700 1 -- 172.21.15.64:0/2309257586 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1450019460 con 0x55a032c94300 2023-09-13T03:27:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.071+0000 7f145d99b700 1 --2- 172.21.15.64:0/2309257586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a032c94300 0x55a032ca3bc0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1454096030 tx=0x7f145400fde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c19e3247eb90d2e server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:27:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.071+0000 7f14417fa700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:27:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.071+0000 7f14417fa700 1 -- 172.21.15.64:0/2309257586 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14540a2290 con 0x55a032c94300 2023-09-13T03:27:40.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.071+0000 7f14417fa700 1 -- 172.21.15.64:0/2309257586 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f14540999b0 con 0x55a032c94300 2023-09-13T03:27:40.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.072+0000 7f14735b1b80 1 -- 172.21.15.64:0/2309257586 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a032b3e490 con 0x55a032c94300 2023-09-13T03:27:40.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.072+0000 7f14417fa700 1 -- 172.21.15.64:0/2309257586 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14540b81a0 con 0x55a032c94300 2023-09-13T03:27:40.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.072+0000 7f14735b1b80 10 client.27281 fetch_fsmap learned FSMap version 70 2023-09-13T03:27:40.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.072+0000 7f14735b1b80 1 -- 172.21.15.64:0/2309257586 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x7f14581a9b30 con 0x55a032c94300 2023-09-13T03:27:40.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.072+0000 7f14417fa700 1 -- 172.21.15.64:0/2309257586 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a032b3e490 con 0x55a032c94300 2023-09-13T03:27:40.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.073+0000 7f14417fa700 1 -- 172.21.15.64:0/2309257586 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 70) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1454005050 con 0x55a032c94300 2023-09-13T03:27:40.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.073+0000 7f14735b1b80 10 client.27281 fetch_fsmap finished waiting for FSMap version 70 2023-09-13T03:27:40.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.073+0000 7f14735b1b80 10 client.27281 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:27:40.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.073+0000 7f14735b1b80 20 client.27281 populate_metadata read hostname 'smithi064' 2023-09-13T03:27:40.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.073+0000 7f14735b1b80 1 --2- 172.21.15.64:0/2309257586 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55a032cadc90 0x55a032cb00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:40.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.073+0000 7f14735b1b80 4 client.27281 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:27:40.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.073+0000 7f14735b1b80 1 -- 172.21.15.64:0/2309257586 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a032bfb840 con 0x55a032cadc90 2023-09-13T03:27:40.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.073+0000 7f145d19a700 1 --2- 172.21.15.64:0/2309257586 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55a032cadc90 0x55a032cb00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:40.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.074+0000 7f145d19a700 1 --2- 172.21.15.64:0/2309257586 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55a032cadc90 0x55a032cb00b0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d60588abb28cf1be server_cookie=85c7ace4d9333c51 in_seq=0 out_seq=0 2023-09-13T03:27:40.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.074+0000 7f14417fa700 10 client.27281 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:27:40.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.075+0000 7f14417fa700 1 -- 172.21.15.64:0/2309257586 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a032bfb840 con 0x55a032cadc90 2023-09-13T03:27:40.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.075+0000 7f14417fa700 10 client.27281 handle_command_reply: tid=0 2023-09-13T03:27:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.075+0000 7f14735b1b80 1 client.27281 shutdown 2023-09-13T03:27:40.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.076+0000 7f14735b1b80 1 -- 172.21.15.64:0/2309257586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a032c94300 msgr2=0x55a032ca3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:27:40.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.076+0000 7f14735b1b80 1 --2- 172.21.15.64:0/2309257586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a032c94300 0x55a032ca3bc0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1454096030 tx=0x7f145400fde0 comp rx=0 tx=0).stop 2023-09-13T03:27:40.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.076+0000 7f14735b1b80 1 -- 172.21.15.64:0/2309257586 shutdown_connections 2023-09-13T03:27:40.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.076+0000 7f14735b1b80 1 --2- 172.21.15.64:0/2309257586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a032c94300 0x55a032ca3bc0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.076+0000 7f14735b1b80 1 --2- 172.21.15.64:0/2309257586 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55a032cadc90 0x55a032cb00b0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.076+0000 7f14735b1b80 1 --2- 172.21.15.64:0/2309257586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a032c9a1a0 0x55a032c9de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.076+0000 7f14735b1b80 1 --2- 172.21.15.64:0/2309257586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a032c94cc0 0x55a032ca4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.076+0000 7f14735b1b80 1 -- 172.21.15.64:0/2309257586 >> 172.21.15.64:0/2309257586 conn(0x55a032c09030 msgr2=0x55a032c8e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:27:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.076+0000 7f14417fa700 0 client.27281 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:27:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.076+0000 7f14735b1b80 1 -- 172.21.15.64:0/2309257586 shutdown_connections 2023-09-13T03:27:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.076+0000 7f14735b1b80 1 -- 172.21.15.64:0/2309257586 wait complete. 2023-09-13T03:27:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.077+0000 7f14735b1b80 20 client.27281 trim_cache size 0 max 16384 2023-09-13T03:27:40.096 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.096+0000 7f14735b1b80 1 Processor -- start 2023-09-13T03:27:40.097 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.096+0000 7f14735b1b80 1 -- start start 2023-09-13T03:27:40.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.096+0000 7f14735b1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a032c94300 0x55a032c9ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:40.098 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.097+0000 7f14735b1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a032c94cc0 0x55a032ca3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.097+0000 7f14735b1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a032c9a1a0 0x55a032ca3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.097+0000 7f14735b1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a032c90140 con 0x55a032c94cc0 2023-09-13T03:27:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.097+0000 7f14735b1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a032c898a0 con 0x55a032c9a1a0 2023-09-13T03:27:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.097+0000 7f14735b1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a032c95140 con 0x55a032c94300 2023-09-13T03:27:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.097+0000 7f145d99b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a032c94300 0x55a032c9ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.097+0000 7f145e19c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a032c9a1a0 0x55a032ca3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.097+0000 7f145d99b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a032c94300 0x55a032c9ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:49312/0 (socket says 172.21.15.64:49312) 2023-09-13T03:27:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.097+0000 7f145d99b700 1 -- 172.21.15.64:0/3746932552 learned_addr learned my addr 172.21.15.64:0/3746932552 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:27:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.097+0000 7f145d19a700 1 --2- 172.21.15.64:0/3746932552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a032c94cc0 0x55a032ca3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.097+0000 7f145d99b700 1 -- 172.21.15.64:0/3746932552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a032c9a1a0 msgr2=0x55a032ca3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:27:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.097+0000 7f145d99b700 1 --2- 172.21.15.64:0/3746932552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a032c9a1a0 0x55a032ca3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.097+0000 7f145d99b700 1 -- 172.21.15.64:0/3746932552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a032c94cc0 msgr2=0x55a032ca3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:27:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.097+0000 7f145d99b700 1 --2- 172.21.15.64:0/3746932552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a032c94cc0 0x55a032ca3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.097+0000 7f145d99b700 1 -- 172.21.15.64:0/3746932552 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a032cb4ca0 con 0x55a032c94300 2023-09-13T03:27:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.097+0000 7f145e19c700 1 --2- 172.21.15.64:0/3746932552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a032c9a1a0 0x55a032ca3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:27:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.098+0000 7f145d99b700 1 --2- 172.21.15.64:0/3746932552 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a032c94300 0x55a032c9ddc0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1454009990 tx=0x7f1454099df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fff193099a167790 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:27:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.098+0000 7f144cbff700 1 -- 172.21.15.64:0/3746932552 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145409a9b0 con 0x55a032c94300 2023-09-13T03:27:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.098+0000 7f144cbff700 1 -- 172.21.15.64:0/3746932552 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14540a2290 con 0x55a032c94300 2023-09-13T03:27:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.098+0000 7f144cbff700 1 -- 172.21.15.64:0/3746932552 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1454005050 con 0x55a032c94300 2023-09-13T03:27:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.099+0000 7f14735b1b80 1 -- 172.21.15.64:0/3746932552 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a032c94300 msgr2=0x55a032c9ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:27:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.099+0000 7f14735b1b80 1 --2- 172.21.15.64:0/3746932552 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a032c94300 0x55a032c9ddc0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1454009990 tx=0x7f1454099df0 comp rx=0 tx=0).stop 2023-09-13T03:27:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.099+0000 7f14735b1b80 1 -- 172.21.15.64:0/3746932552 shutdown_connections 2023-09-13T03:27:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.099+0000 7f14735b1b80 1 --2- 172.21.15.64:0/3746932552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a032c9a1a0 0x55a032ca3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.099+0000 7f14735b1b80 1 --2- 172.21.15.64:0/3746932552 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a032c94300 0x55a032c9ddc0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.099+0000 7f14735b1b80 1 --2- 172.21.15.64:0/3746932552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a032c94cc0 0x55a032ca3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.099+0000 7f14735b1b80 1 -- 172.21.15.64:0/3746932552 >> 172.21.15.64:0/3746932552 conn(0x55a032c09030 msgr2=0x55a032c9d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:27:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.099+0000 7f14735b1b80 1 -- 172.21.15.64:0/3746932552 shutdown_connections 2023-09-13T03:27:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.099+0000 7f14735b1b80 1 -- 172.21.15.64:0/3746932552 wait complete. 2023-09-13T03:27:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.099+0000 7f14735b1b80 1 -- start start 2023-09-13T03:27:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.100+0000 7f14735b1b80 1 Processor -- start 2023-09-13T03:27:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.100+0000 7f14735b1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a032c94300 0x55a032cfa320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.100+0000 7f14735b1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a032c94cc0 0x55a032cfa860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.100+0000 7f14735b1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a032c9a1a0 0x55a032ca32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:40.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.100+0000 7f14735b1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a032ca38d0 con 0x55a032c94300 2023-09-13T03:27:40.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.100+0000 7f14735b1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a032ca3a40 con 0x55a032c94cc0 2023-09-13T03:27:40.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.100+0000 7f14735b1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a032ca3c00 con 0x55a032c9a1a0 2023-09-13T03:27:40.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.100+0000 7f14735b1b80 10 client.0 _ll_register_callbacks cb 0x55a032acea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:27:40.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.101+0000 7f145d19a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a032c94cc0 0x55a032cfa860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:40.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.101+0000 7f145d99b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a032c94300 0x55a032cfa320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.101+0000 7f145d19a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a032c94cc0 0x55a032cfa860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:59354/0 (socket says 172.21.15.64:59354) 2023-09-13T03:27:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.101+0000 7f145d19a700 1 -- 172.21.15.64:0/1853145806 learned_addr learned my addr 172.21.15.64:0/1853145806 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:27:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.101+0000 7f145e19c700 1 --2- 172.21.15.64:0/1853145806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a032c9a1a0 0x55a032ca32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.101+0000 7f145d99b700 1 -- 172.21.15.64:0/1853145806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a032c9a1a0 msgr2=0x55a032ca32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:27:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.101+0000 7f145d99b700 1 --2- 172.21.15.64:0/1853145806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a032c9a1a0 0x55a032ca32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.101+0000 7f145d99b700 1 -- 172.21.15.64:0/1853145806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a032c94cc0 msgr2=0x55a032cfa860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:27:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.101+0000 7f145d99b700 1 --2- 172.21.15.64:0/1853145806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a032c94cc0 0x55a032cfa860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.101+0000 7f145d99b700 1 -- 172.21.15.64:0/1853145806 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14500195f0 con 0x55a032c94300 2023-09-13T03:27:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.101+0000 7f145d19a700 1 --2- 172.21.15.64:0/1853145806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a032c94cc0 0x55a032cfa860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:27:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.102+0000 7f145d99b700 1 --2- 172.21.15.64:0/1853145806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a032c94300 0x55a032cfa320 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f1454099fb0 tx=0x7f1454005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9e067ed9024249 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:27:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.102+0000 7f14417fa700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:27:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.103+0000 7f14417fa700 1 -- 172.21.15.64:0/1853145806 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14540963c0 con 0x55a032c94300 2023-09-13T03:27:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.103+0000 7f14417fa700 1 -- 172.21.15.64:0/1853145806 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f145409a9b0 con 0x55a032c94300 2023-09-13T03:27:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.103+0000 7f14735b1b80 1 -- 172.21.15.64:0/1853145806 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f145409a2b0 con 0x55a032c94300 2023-09-13T03:27:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.103+0000 7f14417fa700 1 -- 172.21.15.64:0/1853145806 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14540a1e00 con 0x55a032c94300 2023-09-13T03:27:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.103+0000 7f14735b1b80 10 client.17223 fetch_fsmap learned FSMap version 70 2023-09-13T03:27:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.103+0000 7f14735b1b80 1 -- 172.21.15.64:0/1853145806 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x7f14581a9b30 con 0x55a032c94300 2023-09-13T03:27:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.103+0000 7f14417fa700 1 -- 172.21.15.64:0/1853145806 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f145409a2b0 con 0x55a032c94300 2023-09-13T03:27:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.103+0000 7f14417fa700 1 -- 172.21.15.64:0/1853145806 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 70) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1454083ab0 con 0x55a032c94300 2023-09-13T03:27:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.103+0000 7f14735b1b80 10 client.17223 fetch_fsmap finished waiting for FSMap version 70 2023-09-13T03:27:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.103+0000 7f14735b1b80 10 client.17223 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:27:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.103+0000 7f14735b1b80 20 client.17223 populate_metadata read hostname 'smithi064' 2023-09-13T03:27:40.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.104+0000 7f14735b1b80 1 --2- 172.21.15.64:0/1853145806 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55a032d01d50 0x55a032d04170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:40.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.104+0000 7f14735b1b80 4 client.17223 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:27:40.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.104+0000 7f14735b1b80 1 -- 172.21.15.64:0/1853145806 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a032bdf8c0 con 0x55a032d01d50 2023-09-13T03:27:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.105+0000 7f145d19a700 1 --2- 172.21.15.64:0/1853145806 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55a032d01d50 0x55a032d04170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.106+0000 7f145d19a700 1 --2- 172.21.15.64:0/1853145806 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55a032d01d50 0x55a032d04170 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29bc303fdd17609c server_cookie=d5b2a768315391b in_seq=0 out_seq=0 2023-09-13T03:27:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.106+0000 7f14417fa700 10 client.17223 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:27:40.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.281+0000 7f14417fa700 1 -- 172.21.15.64:0/1853145806 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a032bdf8c0 con 0x55a032d01d50 2023-09-13T03:27:40.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.282+0000 7f14417fa700 10 client.17223 handle_command_reply: tid=0 2023-09-13T03:27:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.282+0000 7f14735b1b80 1 client.17223 shutdown 2023-09-13T03:27:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.282+0000 7f14735b1b80 1 -- 172.21.15.64:0/1853145806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a032c94300 msgr2=0x55a032cfa320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:27:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.282+0000 7f14735b1b80 1 --2- 172.21.15.64:0/1853145806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a032c94300 0x55a032cfa320 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f1454099fb0 tx=0x7f1454005c70 comp rx=0 tx=0).stop 2023-09-13T03:27:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.282+0000 7f14735b1b80 1 -- 172.21.15.64:0/1853145806 shutdown_connections 2023-09-13T03:27:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.282+0000 7f14735b1b80 1 --2- 172.21.15.64:0/1853145806 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55a032d01d50 0x55a032d04170 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.282+0000 7f14735b1b80 1 --2- 172.21.15.64:0/1853145806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a032c9a1a0 0x55a032ca32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.282+0000 7f14735b1b80 1 --2- 172.21.15.64:0/1853145806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a032c94300 0x55a032cfa320 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.282+0000 7f14735b1b80 1 --2- 172.21.15.64:0/1853145806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a032c94cc0 0x55a032cfa860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.282+0000 7f14735b1b80 1 -- 172.21.15.64:0/1853145806 >> 172.21.15.64:0/1853145806 conn(0x55a032c09030 msgr2=0x55a032c99060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:27:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.282+0000 7f14417fa700 0 client.17223 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:27:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.282+0000 7f14735b1b80 1 -- 172.21.15.64:0/1853145806 shutdown_connections 2023-09-13T03:27:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.282+0000 7f14735b1b80 1 -- 172.21.15.64:0/1853145806 wait complete. 2023-09-13T03:27:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.283+0000 7f14735b1b80 20 client.17223 trim_cache size 0 max 16384 2023-09-13T03:27:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:27:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:27:40.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:27:40.287 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:27:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.283+0000 7f145fbff700 1 -- 172.21.15.64:0/3546952385 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f14440a0830 msgr2=0x7f14440a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:27:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.283+0000 7f145fbff700 1 --2- 172.21.15.64:0/3546952385 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f14440a0830 0x7f14440a2cf0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f145000f590 tx=0x7f1450009920 comp rx=0 tx=0).stop 2023-09-13T03:27:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.283+0000 7f145fbff700 1 -- 172.21.15.64:0/3546952385 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14580679f0 msgr2=0x7f14581137f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:27:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.283+0000 7f145fbff700 1 --2- 172.21.15.64:0/3546952385 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14580679f0 0x7f14581137f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f145400b580 tx=0x7f145400f690 comp rx=0 tx=0).stop 2023-09-13T03:27:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.284+0000 7f145fbff700 1 -- 172.21.15.64:0/3546952385 shutdown_connections 2023-09-13T03:27:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.284+0000 7f145fbff700 1 --2- 172.21.15.64:0/3546952385 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f14440a0830 0x7f14440a2cf0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.284+0000 7f145fbff700 1 --2- 172.21.15.64:0/3546952385 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1458106650 0x7f14581102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.284+0000 7f145fbff700 1 --2- 172.21.15.64:0/3546952385 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14580679f0 0x7f14581137f0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.284+0000 7f145fbff700 1 --2- 172.21.15.64:0/3546952385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14580683b0 0x7f145810fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.284+0000 7f145fbff700 1 -- 172.21.15.64:0/3546952385 >> 172.21.15.64:0/3546952385 conn(0x7f1458074f30 msgr2=0x7f14580fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:27:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.284+0000 7f145fbff700 1 -- 172.21.15.64:0/3546952385 shutdown_connections 2023-09-13T03:27:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:40.284+0000 7f145fbff700 1 -- 172.21.15.64:0/3546952385 wait complete. 2023-09-13T03:27:40.290 INFO:tasks.workunit.client.0.smithi064.stdout: 10 87114 46.82 MB/sec warmup 104 sec latency 103.109 ms 2023-09-13T03:27:40.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:27:40 smithi142 ceph-mon[134082]: pgmap v454: 97 pgs: 97 active+clean; 797 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 891 op/s 2023-09-13T03:27:40.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:27:40 smithi170 ceph-mon[134328]: pgmap v454: 97 pgs: 97 active+clean; 797 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 891 op/s 2023-09-13T03:27:40.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:27:40 smithi064 ceph-mon[97675]: pgmap v454: 97 pgs: 97 active+clean; 797 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 891 op/s 2023-09-13T03:27:41.291 INFO:tasks.workunit.client.0.smithi064.stdout: 10 88128 46.85 MB/sec warmup 105 sec latency 45.123 ms 2023-09-13T03:27:42.291 INFO:tasks.workunit.client.0.smithi064.stdout: 10 89426 47.12 MB/sec warmup 106 sec latency 52.608 ms 2023-09-13T03:27:43.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:27:42 smithi170 ceph-mon[134328]: pgmap v455: 97 pgs: 97 active+clean; 813 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 19 MiB/s wr, 1.36k op/s 2023-09-13T03:27:43.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:27:42 smithi064 ceph-mon[97675]: pgmap v455: 97 pgs: 97 active+clean; 813 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 19 MiB/s wr, 1.36k op/s 2023-09-13T03:27:43.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:27:42 smithi142 ceph-mon[134082]: pgmap v455: 97 pgs: 97 active+clean; 813 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 19 MiB/s wr, 1.36k op/s 2023-09-13T03:27:43.291 INFO:tasks.workunit.client.0.smithi064.stdout: 10 90280 47.15 MB/sec warmup 107 sec latency 49.054 ms 2023-09-13T03:27:44.291 INFO:tasks.workunit.client.0.smithi064.stdout: 10 90870 46.95 MB/sec warmup 108 sec latency 220.736 ms 2023-09-13T03:27:45.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:27:44 smithi170 ceph-mon[134328]: pgmap v456: 97 pgs: 97 active+clean; 813 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 876 op/s 2023-09-13T03:27:45.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:27:44 smithi064 ceph-mon[97675]: pgmap v456: 97 pgs: 97 active+clean; 813 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 876 op/s 2023-09-13T03:27:45.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:27:44 smithi142 ceph-mon[134082]: pgmap v456: 97 pgs: 97 active+clean; 813 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 876 op/s 2023-09-13T03:27:45.291 INFO:tasks.workunit.client.0.smithi064.stdout: 10 91928 47.00 MB/sec warmup 109 sec latency 44.808 ms 2023-09-13T03:27:46.291 INFO:tasks.workunit.client.0.smithi064.stdout: 10 93031 47.20 MB/sec warmup 110 sec latency 55.654 ms 2023-09-13T03:27:47.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:27:46 smithi170 ceph-mon[134328]: pgmap v457: 97 pgs: 97 active+clean; 840 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 19 MiB/s wr, 1.30k op/s 2023-09-13T03:27:47.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:27:46 smithi064 ceph-mon[97675]: pgmap v457: 97 pgs: 97 active+clean; 840 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 19 MiB/s wr, 1.30k op/s 2023-09-13T03:27:47.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:27:46 smithi142 ceph-mon[134082]: pgmap v457: 97 pgs: 97 active+clean; 840 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 19 MiB/s wr, 1.30k op/s 2023-09-13T03:27:47.291 INFO:tasks.workunit.client.0.smithi064.stdout: 10 93846 47.14 MB/sec warmup 111 sec latency 60.001 ms 2023-09-13T03:27:48.291 INFO:tasks.workunit.client.0.smithi064.stdout: 10 94950 47.21 MB/sec warmup 112 sec latency 45.859 ms 2023-09-13T03:27:49.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:27:48 smithi170 ceph-mon[134328]: pgmap v458: 97 pgs: 97 active+clean; 840 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 901 op/s 2023-09-13T03:27:49.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:27:48 smithi064 ceph-mon[97675]: pgmap v458: 97 pgs: 97 active+clean; 840 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 901 op/s 2023-09-13T03:27:49.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:27:48 smithi142 ceph-mon[134082]: pgmap v458: 97 pgs: 97 active+clean; 840 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 901 op/s 2023-09-13T03:27:49.291 INFO:tasks.workunit.client.0.smithi064.stdout: 10 95640 47.12 MB/sec warmup 113 sec latency 160.789 ms 2023-09-13T03:27:50.291 INFO:tasks.workunit.client.0.smithi064.stdout: 10 96522 47.18 MB/sec warmup 114 sec latency 68.315 ms 2023-09-13T03:27:50.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:27:50 smithi142 ceph-mon[134082]: pgmap v459: 97 pgs: 97 active+clean; 840 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 901 op/s 2023-09-13T03:27:50.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:27:50 smithi170 ceph-mon[134328]: pgmap v459: 97 pgs: 97 active+clean; 840 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 901 op/s 2023-09-13T03:27:50.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:27:50 smithi064 ceph-mon[97675]: pgmap v459: 97 pgs: 97 active+clean; 840 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 901 op/s 2023-09-13T03:27:51.291 INFO:tasks.workunit.client.0.smithi064.stdout: 10 97551 47.21 MB/sec warmup 115 sec latency 32.647 ms 2023-09-13T03:27:52.292 INFO:tasks.workunit.client.0.smithi064.stdout: 10 98595 47.34 MB/sec warmup 116 sec latency 40.879 ms 2023-09-13T03:27:53.163 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:27:52 smithi170 ceph-mon[134328]: pgmap v460: 97 pgs: 97 active+clean; 867 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 20 MiB/s wr, 1.30k op/s 2023-09-13T03:27:53.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:27:52 smithi064 ceph-mon[97675]: pgmap v460: 97 pgs: 97 active+clean; 867 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 20 MiB/s wr, 1.30k op/s 2023-09-13T03:27:53.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:27:52 smithi142 ceph-mon[134082]: pgmap v460: 97 pgs: 97 active+clean; 867 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 20 MiB/s wr, 1.30k op/s 2023-09-13T03:27:53.292 INFO:tasks.workunit.client.0.smithi064.stdout: 10 99510 47.29 MB/sec warmup 117 sec latency 64.146 ms 2023-09-13T03:27:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.509+0000 7fce553ff700 1 -- 172.21.15.64:0/3139542145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce50067f30 msgr2=0x7fce500683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:27:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.509+0000 7fce553ff700 1 --2- 172.21.15.64:0/3139542145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce50067f30 0x7fce500683b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fce44009990 tx=0x7fce44009ca0 comp rx=0 tx=0).stop 2023-09-13T03:27:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.509+0000 7fce553ff700 1 -- 172.21.15.64:0/3139542145 shutdown_connections 2023-09-13T03:27:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.509+0000 7fce553ff700 1 --2- 172.21.15.64:0/3139542145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce5010cdd0 0x7fce5010f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.509+0000 7fce553ff700 1 --2- 172.21.15.64:0/3139542145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce50101470 0x7fce500679f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.509+0000 7fce553ff700 1 --2- 172.21.15.64:0/3139542145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce50067f30 0x7fce500683b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.509+0000 7fce553ff700 1 -- 172.21.15.64:0/3139542145 >> 172.21.15.64:0/3139542145 conn(0x7fce50074f30 msgr2=0x7fce50075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:27:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.509+0000 7fce553ff700 1 -- 172.21.15.64:0/3139542145 shutdown_connections 2023-09-13T03:27:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.510+0000 7fce553ff700 1 -- 172.21.15.64:0/3139542145 wait complete. 2023-09-13T03:27:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.511+0000 7fce553ff700 1 Processor -- start 2023-09-13T03:27:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.512+0000 7fce553ff700 1 -- start start 2023-09-13T03:27:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.512+0000 7fce553ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce50067f30 0x7fce5019e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.512+0000 7fce553ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce50101470 0x7fce5019ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.512+0000 7fce553ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce5010cdd0 0x7fce50198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.512+0000 7fce553ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce50198dc0 con 0x7fce50101470 2023-09-13T03:27:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.512+0000 7fce553ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce50198f30 con 0x7fce50067f30 2023-09-13T03:27:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.512+0000 7fce553ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce501990f0 con 0x7fce5010cdd0 2023-09-13T03:27:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.513+0000 7fce4f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce5010cdd0 0x7fce50198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:53.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.513+0000 7fce4f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce5010cdd0 0x7fce50198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:37028/0 (socket says 172.21.15.64:37028) 2023-09-13T03:27:53.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.513+0000 7fce4f7fe700 1 -- 172.21.15.64:0/728463913 learned_addr learned my addr 172.21.15.64:0/728463913 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:27:53.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.513+0000 7fce4e7fc700 1 --2- 172.21.15.64:0/728463913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce50101470 0x7fce5019ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:53.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.513+0000 7fce4f7fe700 1 -- 172.21.15.64:0/728463913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce50067f30 msgr2=0x7fce5019e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:27:53.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.513+0000 7fce4f7fe700 1 --2- 172.21.15.64:0/728463913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce50067f30 0x7fce5019e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.513+0000 7fce4f7fe700 1 -- 172.21.15.64:0/728463913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce50101470 msgr2=0x7fce5019ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:27:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.513+0000 7fce4f7fe700 1 --2- 172.21.15.64:0/728463913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce50101470 0x7fce5019ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.513+0000 7fce4f7fe700 1 -- 172.21.15.64:0/728463913 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce440095f0 con 0x7fce5010cdd0 2023-09-13T03:27:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.514+0000 7fce4f7fe700 1 --2- 172.21.15.64:0/728463913 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce5010cdd0 0x7fce50198750 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fce4800ef30 tx=0x7fce4800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:27:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.514+0000 7fce37fff700 1 -- 172.21.15.64:0/728463913 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce48018070 con 0x7fce5010cdd0 2023-09-13T03:27:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.514+0000 7fce37fff700 1 -- 172.21.15.64:0/728463913 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce4800f040 con 0x7fce5010cdd0 2023-09-13T03:27:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.514+0000 7fce553ff700 1 -- 172.21.15.64:0/728463913 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce50199370 con 0x7fce5010cdd0 2023-09-13T03:27:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.514+0000 7fce553ff700 1 -- 172.21.15.64:0/728463913 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce5006a200 con 0x7fce5010cdd0 2023-09-13T03:27:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.514+0000 7fce37fff700 1 -- 172.21.15.64:0/728463913 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4800cdb0 con 0x7fce5010cdd0 2023-09-13T03:27:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.522+0000 7fce37fff700 1 -- 172.21.15.64:0/728463913 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fce48004ad0 con 0x7fce5010cdd0 2023-09-13T03:27:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.523+0000 7fce68e19b80 1 Processor -- start 2023-09-13T03:27:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.524+0000 7fce68e19b80 1 -- start start 2023-09-13T03:27:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.524+0000 7fce68e19b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581d65969b0 0x5581d6596d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.524+0000 7fce68e19b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d6590b10 0x5581d6590f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.524+0000 7fce68e19b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581d65914d0 0x5581d659d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.524+0000 7fce68e19b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581d64e16d0 con 0x5581d65969b0 2023-09-13T03:27:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.524+0000 7fce68e19b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581d64deff0 con 0x5581d65914d0 2023-09-13T03:27:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.524+0000 7fce68e19b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581d64da8e0 con 0x5581d6590b10 2023-09-13T03:27:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.524+0000 7fce37fff700 1 --2- 172.21.15.64:0/728463913 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fce380a0830 0x7fce380a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.525+0000 7fce4f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581d65914d0 0x5581d659d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.525+0000 7fce4f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581d65914d0 0x5581d659d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:52526/0 (socket says 172.21.15.64:52526) 2023-09-13T03:27:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.525+0000 7fce4f7fe700 1 -- 172.21.15.64:0/2714231760 learned_addr learned my addr 172.21.15.64:0/2714231760 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:27:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.525+0000 7fce4e7fc700 1 --2- 172.21.15.64:0/2714231760 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d6590b10 0x5581d6590f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.525+0000 7fce4e7fc700 1 --2- 172.21.15.64:0/728463913 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fce380a0830 0x7fce380a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.526+0000 7fce4f7fe700 1 -- 172.21.15.64:0/2714231760 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d6590b10 msgr2=0x5581d6590f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:27:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.526+0000 7fce4f7fe700 1 --2- 172.21.15.64:0/2714231760 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d6590b10 0x5581d6590f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.526+0000 7fce4f7fe700 1 -- 172.21.15.64:0/2714231760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581d65969b0 msgr2=0x5581d6596d90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-13T03:27:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.526+0000 7fce4f7fe700 1 --2- 172.21.15.64:0/2714231760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581d65969b0 0x5581d6596d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.526+0000 7fce37fff700 1 -- 172.21.15.64:0/728463913 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fce480cba00 con 0x7fce5010cdd0 2023-09-13T03:27:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.526+0000 7fce4f7fe700 1 -- 172.21.15.64:0/2714231760 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581d6306d80 con 0x5581d65914d0 2023-09-13T03:27:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.526+0000 7fce4f7fe700 1 --2- 172.21.15.64:0/2714231760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581d65914d0 0x5581d659d740 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fce480959d0 tx=0x7fce480988c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b67e5408e587adfc server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:27:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.527+0000 7fce357fa700 1 -- 172.21.15.64:0/2714231760 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce48099660 con 0x5581d65914d0 2023-09-13T03:27:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.528+0000 7fce357fa700 1 -- 172.21.15.64:0/2714231760 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce48098f60 con 0x5581d65914d0 2023-09-13T03:27:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.528+0000 7fce357fa700 1 -- 172.21.15.64:0/2714231760 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce48098450 con 0x5581d65914d0 2023-09-13T03:27:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.528+0000 7fce68e19b80 1 -- 172.21.15.64:0/2714231760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581d65914d0 msgr2=0x5581d659d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:27:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.528+0000 7fce68e19b80 1 --2- 172.21.15.64:0/2714231760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581d65914d0 0x5581d659d740 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fce480959d0 tx=0x7fce480988c0 comp rx=0 tx=0).stop 2023-09-13T03:27:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.531+0000 7fce4e7fc700 1 --2- 172.21.15.64:0/728463913 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fce380a0830 0x7fce380a2cf0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fce4400b2e0 tx=0x7fce4400b610 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:27:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.531+0000 7fce68e19b80 1 -- 172.21.15.64:0/2714231760 shutdown_connections 2023-09-13T03:27:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.531+0000 7fce68e19b80 1 --2- 172.21.15.64:0/2714231760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581d65914d0 0x5581d659d740 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.531+0000 7fce68e19b80 1 --2- 172.21.15.64:0/2714231760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581d65969b0 0x5581d6596d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.531+0000 7fce68e19b80 1 --2- 172.21.15.64:0/2714231760 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d6590b10 0x5581d6590f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.531+0000 7fce68e19b80 1 -- 172.21.15.64:0/2714231760 >> 172.21.15.64:0/2714231760 conn(0x5581d64fd030 msgr2=0x5581d6583560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:27:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.532+0000 7fce68e19b80 1 -- 172.21.15.64:0/2714231760 shutdown_connections 2023-09-13T03:27:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.532+0000 7fce68e19b80 1 -- 172.21.15.64:0/2714231760 wait complete. 2023-09-13T03:27:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.533+0000 7fce68e19b80 1 -- start start 2023-09-13T03:27:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.533+0000 7fce68e19b80 1 Processor -- start 2023-09-13T03:27:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.534+0000 7fce68e19b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581d6590b10 0x5581d659d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.534+0000 7fce68e19b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581d65914d0 0x5581d659a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.534+0000 7fce68e19b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d65969b0 0x5581d659ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.534+0000 7fce68e19b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581d6478b10 con 0x5581d6590b10 2023-09-13T03:27:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.534+0000 7fce68e19b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581d6478810 con 0x5581d65914d0 2023-09-13T03:27:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.534+0000 7fce68e19b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581d635c8c0 con 0x5581d65969b0 2023-09-13T03:27:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.534+0000 7fce68e19b80 10 client.0 _ll_register_callbacks cb 0x5581d63c2a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:27:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.535+0000 7fce4e7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581d65914d0 0x5581d659a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.535+0000 7fce4effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581d6590b10 0x5581d659d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.535+0000 7fce4f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d65969b0 0x5581d659ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.535+0000 7fce4effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581d6590b10 0x5581d659d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58422/0 (socket says 172.21.15.64:58422) 2023-09-13T03:27:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.535+0000 7fce4effd700 1 -- 172.21.15.64:0/413846451 learned_addr learned my addr 172.21.15.64:0/413846451 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:27:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.535+0000 7fce4f7fe700 1 --2- 172.21.15.64:0/413846451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d65969b0 0x5581d659ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:37052/0 (socket says 172.21.15.64:37052) 2023-09-13T03:27:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.536+0000 7fce4e7fc700 1 -- 172.21.15.64:0/413846451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d65969b0 msgr2=0x5581d659ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:27:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.536+0000 7fce4e7fc700 1 --2- 172.21.15.64:0/413846451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d65969b0 0x5581d659ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.536+0000 7fce4e7fc700 1 -- 172.21.15.64:0/413846451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581d6590b10 msgr2=0x5581d659d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:27:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.536+0000 7fce4e7fc700 1 --2- 172.21.15.64:0/413846451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581d6590b10 0x5581d659d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.536+0000 7fce4e7fc700 1 -- 172.21.15.64:0/413846451 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce5006a200 con 0x5581d65914d0 2023-09-13T03:27:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.536+0000 7fce4effd700 1 --2- 172.21.15.64:0/413846451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581d6590b10 0x5581d659d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:27:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.536+0000 7fce4f7fe700 1 --2- 172.21.15.64:0/413846451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d65969b0 0x5581d659ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:27:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.538+0000 7fce4e7fc700 1 --2- 172.21.15.64:0/413846451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581d65914d0 0x5581d659a840 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fce4400b680 tx=0x7fce44009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2099380dd3f9c123 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:27:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.538+0000 7fce32ff5700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:27:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.539+0000 7fce32ff5700 1 -- 172.21.15.64:0/413846451 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4402d070 con 0x5581d65914d0 2023-09-13T03:27:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.539+0000 7fce68e19b80 1 -- 172.21.15.64:0/413846451 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5581d6432490 con 0x5581d65914d0 2023-09-13T03:27:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.539+0000 7fce32ff5700 1 -- 172.21.15.64:0/413846451 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fce44028070 con 0x5581d65914d0 2023-09-13T03:27:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.539+0000 7fce32ff5700 1 -- 172.21.15.64:0/413846451 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce4401d070 con 0x5581d65914d0 2023-09-13T03:27:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.540+0000 7fce68e19b80 10 client.27296 fetch_fsmap learned FSMap version 70 2023-09-13T03:27:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.540+0000 7fce68e19b80 1 -- 172.21.15.64:0/413846451 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x5581d62efc00 con 0x5581d65914d0 2023-09-13T03:27:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.540+0000 7fce32ff5700 1 -- 172.21.15.64:0/413846451 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5581d6432490 con 0x5581d65914d0 2023-09-13T03:27:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.542+0000 7fce32ff5700 1 -- 172.21.15.64:0/413846451 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 70) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fce4400f460 con 0x5581d65914d0 2023-09-13T03:27:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.542+0000 7fce68e19b80 10 client.27296 fetch_fsmap finished waiting for FSMap version 70 2023-09-13T03:27:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.542+0000 7fce68e19b80 10 client.27296 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:27:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.542+0000 7fce68e19b80 20 client.27296 populate_metadata read hostname 'smithi064' 2023-09-13T03:27:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.542+0000 7fce68e19b80 1 --2- 172.21.15.64:0/413846451 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5581d65b3400 0x5581d65b5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.542+0000 7fce68e19b80 4 client.27296 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:27:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.542+0000 7fce68e19b80 1 -- 172.21.15.64:0/413846451 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5581d64ef840 con 0x5581d65b3400 2023-09-13T03:27:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.543+0000 7fce4effd700 1 --2- 172.21.15.64:0/413846451 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5581d65b3400 0x5581d65b5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.544+0000 7fce32ff5700 10 client.27296 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:27:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.544+0000 7fce4effd700 1 --2- 172.21.15.64:0/413846451 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5581d65b3400 0x5581d65b5820 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f73f595ce321775 server_cookie=d7f77d11928d8993 in_seq=0 out_seq=0 2023-09-13T03:27:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.546+0000 7fce32ff5700 1 -- 172.21.15.64:0/413846451 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5581d64ef840 con 0x5581d65b3400 2023-09-13T03:27:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.546+0000 7fce32ff5700 10 client.27296 handle_command_reply: tid=0 2023-09-13T03:27:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.547+0000 7fce68e19b80 1 client.27296 shutdown 2023-09-13T03:27:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.548+0000 7fce68e19b80 1 -- 172.21.15.64:0/413846451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581d65914d0 msgr2=0x5581d659a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:27:53.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.548+0000 7fce68e19b80 1 --2- 172.21.15.64:0/413846451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581d65914d0 0x5581d659a840 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fce4400b680 tx=0x7fce44009ca0 comp rx=0 tx=0).stop 2023-09-13T03:27:53.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.548+0000 7fce68e19b80 1 -- 172.21.15.64:0/413846451 shutdown_connections 2023-09-13T03:27:53.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.548+0000 7fce68e19b80 1 --2- 172.21.15.64:0/413846451 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5581d65b3400 0x5581d65b5820 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:53.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.548+0000 7fce68e19b80 1 --2- 172.21.15.64:0/413846451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d65969b0 0x5581d659ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:53.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.548+0000 7fce68e19b80 1 --2- 172.21.15.64:0/413846451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581d6590b10 0x5581d659d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:53.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.548+0000 7fce68e19b80 1 --2- 172.21.15.64:0/413846451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581d65914d0 0x5581d659a840 secure :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fce4400b680 tx=0x7fce44009ca0 comp rx=0 tx=0).stop 2023-09-13T03:27:53.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.548+0000 7fce68e19b80 1 -- 172.21.15.64:0/413846451 >> 172.21.15.64:0/413846451 conn(0x5581d64fd030 msgr2=0x5581d65827a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:27:53.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.548+0000 7fce32ff5700 0 client.27296 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:27:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.548+0000 7fce68e19b80 1 -- 172.21.15.64:0/413846451 shutdown_connections 2023-09-13T03:27:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.548+0000 7fce68e19b80 1 -- 172.21.15.64:0/413846451 wait complete. 2023-09-13T03:27:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.549+0000 7fce68e19b80 20 client.27296 trim_cache size 0 max 16384 2023-09-13T03:27:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.568+0000 7fce68e19b80 1 Processor -- start 2023-09-13T03:27:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.569+0000 7fce68e19b80 1 -- start start 2023-09-13T03:27:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.569+0000 7fce68e19b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581d6590b10 0x5581d658c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.569+0000 7fce68e19b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581d65969b0 0x5581d6599db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.569+0000 7fce68e19b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d6598590 0x5581d659a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.569+0000 7fce68e19b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581d6590860 con 0x5581d65969b0 2023-09-13T03:27:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.569+0000 7fce68e19b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581d6591dc0 con 0x5581d6590b10 2023-09-13T03:27:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.569+0000 7fce68e19b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581d659b3a0 con 0x5581d6598590 2023-09-13T03:27:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.569+0000 7fce4effd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581d6590b10 0x5581d658c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:53.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.569+0000 7fce4e7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581d65969b0 0x5581d6599db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:53.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.569+0000 7fce4effd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581d6590b10 0x5581d658c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:52550/0 (socket says 172.21.15.64:52550) 2023-09-13T03:27:53.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.569+0000 7fce4effd700 1 -- 172.21.15.64:0/839822464 learned_addr learned my addr 172.21.15.64:0/839822464 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:27:53.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.569+0000 7fce4f7fe700 1 --2- 172.21.15.64:0/839822464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d6598590 0x5581d659a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.570+0000 7fce4effd700 1 -- 172.21.15.64:0/839822464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d6598590 msgr2=0x5581d659a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:27:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.570+0000 7fce4effd700 1 --2- 172.21.15.64:0/839822464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d6598590 0x5581d659a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.570+0000 7fce4effd700 1 -- 172.21.15.64:0/839822464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581d65969b0 msgr2=0x5581d6599db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:27:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.570+0000 7fce4effd700 1 --2- 172.21.15.64:0/839822464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581d65969b0 0x5581d6599db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.570+0000 7fce4effd700 1 -- 172.21.15.64:0/839822464 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581d65ba450 con 0x5581d6590b10 2023-09-13T03:27:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.570+0000 7fce4effd700 1 --2- 172.21.15.64:0/839822464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581d6590b10 0x5581d658c3b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fce4000beb0 tx=0x7fce4000b420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b0edb664c91f12 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:27:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.571+0000 7fce35ffb700 1 -- 172.21.15.64:0/839822464 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce40016040 con 0x5581d6590b10 2023-09-13T03:27:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.571+0000 7fce35ffb700 1 -- 172.21.15.64:0/839822464 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce4001d070 con 0x5581d6590b10 2023-09-13T03:27:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.571+0000 7fce35ffb700 1 -- 172.21.15.64:0/839822464 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce40010690 con 0x5581d6590b10 2023-09-13T03:27:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.571+0000 7fce68e19b80 1 -- 172.21.15.64:0/839822464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581d6590b10 msgr2=0x5581d658c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:27:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.571+0000 7fce68e19b80 1 --2- 172.21.15.64:0/839822464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581d6590b10 0x5581d658c3b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fce4000beb0 tx=0x7fce4000b420 comp rx=0 tx=0).stop 2023-09-13T03:27:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.572+0000 7fce68e19b80 1 -- 172.21.15.64:0/839822464 shutdown_connections 2023-09-13T03:27:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.572+0000 7fce68e19b80 1 --2- 172.21.15.64:0/839822464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581d6590b10 0x5581d658c3b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.572+0000 7fce68e19b80 1 --2- 172.21.15.64:0/839822464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d6598590 0x5581d659a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.572+0000 7fce68e19b80 1 --2- 172.21.15.64:0/839822464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581d65969b0 0x5581d6599db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.572+0000 7fce68e19b80 1 -- 172.21.15.64:0/839822464 >> 172.21.15.64:0/839822464 conn(0x5581d64fd030 msgr2=0x5581d65b5410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:27:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.572+0000 7fce68e19b80 1 -- 172.21.15.64:0/839822464 shutdown_connections 2023-09-13T03:27:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.572+0000 7fce68e19b80 1 -- 172.21.15.64:0/839822464 wait complete. 2023-09-13T03:27:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.572+0000 7fce68e19b80 1 -- start start 2023-09-13T03:27:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.573+0000 7fce68e19b80 1 Processor -- start 2023-09-13T03:27:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.573+0000 7fce68e19b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581d6590b10 0x5581d6588390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.573+0000 7fce68e19b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d65969b0 0x5581d65888d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.573+0000 7fce68e19b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581d6598590 0x5581d6606220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.573+0000 7fce68e19b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581d6589100 con 0x5581d6598590 2023-09-13T03:27:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.573+0000 7fce68e19b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581d6606760 con 0x5581d6590b10 2023-09-13T03:27:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.573+0000 7fce68e19b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581d66068a0 con 0x5581d65969b0 2023-09-13T03:27:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.573+0000 7fce68e19b80 10 client.0 _ll_register_callbacks cb 0x7fce44028740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:27:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.574+0000 7fce4e7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d65969b0 0x5581d65888d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.574+0000 7fce4e7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d65969b0 0x5581d65888d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:37074/0 (socket says 172.21.15.64:37074) 2023-09-13T03:27:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.574+0000 7fce4e7fc700 1 -- 172.21.15.64:0/2139809546 learned_addr learned my addr 172.21.15.64:0/2139809546 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:27:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.574+0000 7fce4e7fc700 1 -- 172.21.15.64:0/2139809546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581d6590b10 msgr2=0x5581d6588390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-13T03:27:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.574+0000 7fce4e7fc700 1 --2- 172.21.15.64:0/2139809546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581d6590b10 0x5581d6588390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.574+0000 7fce4e7fc700 1 -- 172.21.15.64:0/2139809546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581d6598590 msgr2=0x5581d6606220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:27:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.574+0000 7fce4e7fc700 1 --2- 172.21.15.64:0/2139809546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581d6598590 0x5581d6606220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.574+0000 7fce4e7fc700 1 -- 172.21.15.64:0/2139809546 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce4000b940 con 0x5581d65969b0 2023-09-13T03:27:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.574+0000 7fce4e7fc700 1 --2- 172.21.15.64:0/2139809546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d65969b0 0x5581d65888d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x5581d657ae40 tx=0x7fce4400f8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d316c35faa91c11e server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:27:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.574+0000 7fce32ff5700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:27:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.575+0000 7fce32ff5700 1 -- 172.21.15.64:0/2139809546 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4402d070 con 0x5581d65969b0 2023-09-13T03:27:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.575+0000 7fce32ff5700 1 -- 172.21.15.64:0/2139809546 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fce4400fc70 con 0x5581d65969b0 2023-09-13T03:27:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.575+0000 7fce68e19b80 1 -- 172.21.15.64:0/2139809546 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce48098f60 con 0x5581d65969b0 2023-09-13T03:27:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.576+0000 7fce32ff5700 1 -- 172.21.15.64:0/2139809546 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce4400fe50 con 0x5581d65969b0 2023-09-13T03:27:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.576+0000 7fce68e19b80 10 client.27274 fetch_fsmap learned FSMap version 70 2023-09-13T03:27:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.576+0000 7fce68e19b80 1 -- 172.21.15.64:0/2139809546 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x5581d632d1f0 con 0x5581d65969b0 2023-09-13T03:27:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.576+0000 7fce32ff5700 1 -- 172.21.15.64:0/2139809546 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce48098f60 con 0x5581d65969b0 2023-09-13T03:27:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.576+0000 7fce32ff5700 1 -- 172.21.15.64:0/2139809546 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 70) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fce44041070 con 0x5581d65969b0 2023-09-13T03:27:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.576+0000 7fce68e19b80 10 client.27274 fetch_fsmap finished waiting for FSMap version 70 2023-09-13T03:27:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.576+0000 7fce68e19b80 10 client.27274 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:27:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.576+0000 7fce68e19b80 20 client.27274 populate_metadata read hostname 'smithi064' 2023-09-13T03:27:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.576+0000 7fce68e19b80 1 --2- 172.21.15.64:0/2139809546 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5581d6607bf0 0x5581d660a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:27:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.576+0000 7fce68e19b80 4 client.27274 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:27:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.576+0000 7fce68e19b80 1 -- 172.21.15.64:0/2139809546 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5581d64d38c0 con 0x5581d6607bf0 2023-09-13T03:27:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.577+0000 7fce4effd700 1 --2- 172.21.15.64:0/2139809546 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5581d6607bf0 0x5581d660a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:27:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.578+0000 7fce4effd700 1 --2- 172.21.15.64:0/2139809546 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5581d6607bf0 0x5581d660a030 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bab10b3f8ff18b8b server_cookie=42cd8084782c3eec in_seq=0 out_seq=0 2023-09-13T03:27:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.578+0000 7fce32ff5700 10 client.27274 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:27:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.764+0000 7fce32ff5700 1 -- 172.21.15.64:0/2139809546 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5581d64d38c0 con 0x5581d6607bf0 2023-09-13T03:27:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.764+0000 7fce32ff5700 10 client.27274 handle_command_reply: tid=0 2023-09-13T03:27:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.764+0000 7fce68e19b80 1 client.27274 shutdown 2023-09-13T03:27:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.765+0000 7fce68e19b80 1 -- 172.21.15.64:0/2139809546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d65969b0 msgr2=0x5581d65888d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:27:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.765+0000 7fce68e19b80 1 --2- 172.21.15.64:0/2139809546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d65969b0 0x5581d65888d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x5581d657ae40 tx=0x7fce4400f8e0 comp rx=0 tx=0).stop 2023-09-13T03:27:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.765+0000 7fce68e19b80 1 -- 172.21.15.64:0/2139809546 shutdown_connections 2023-09-13T03:27:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.765+0000 7fce68e19b80 1 --2- 172.21.15.64:0/2139809546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581d6590b10 0x5581d6588390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.765+0000 7fce68e19b80 1 --2- 172.21.15.64:0/2139809546 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5581d6607bf0 0x5581d660a030 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.765+0000 7fce68e19b80 1 --2- 172.21.15.64:0/2139809546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5581d6598590 0x5581d6606220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.765+0000 7fce68e19b80 1 --2- 172.21.15.64:0/2139809546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d65969b0 0x5581d65888d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.765+0000 7fce68e19b80 1 -- 172.21.15.64:0/2139809546 >> 172.21.15.64:0/2139809546 conn(0x5581d64fd030 msgr2=0x5581d6594970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:27:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.765+0000 7fce32ff5700 0 client.27274 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:27:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.766+0000 7fce68e19b80 1 -- 172.21.15.64:0/2139809546 shutdown_connections 2023-09-13T03:27:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.766+0000 7fce68e19b80 1 -- 172.21.15.64:0/2139809546 wait complete. 2023-09-13T03:27:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.769+0000 7fce68e19b80 20 client.27274 trim_cache size 0 max 16384 2023-09-13T03:27:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:27:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:27:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:27:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:27:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.770+0000 7fce553ff700 1 -- 172.21.15.64:0/728463913 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fce380a0830 msgr2=0x7fce380a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:27:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.770+0000 7fce553ff700 1 --2- 172.21.15.64:0/728463913 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fce380a0830 0x7fce380a2cf0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fce4400b2e0 tx=0x7fce4400b610 comp rx=0 tx=0).stop 2023-09-13T03:27:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.770+0000 7fce553ff700 1 -- 172.21.15.64:0/728463913 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce5010cdd0 msgr2=0x7fce50198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:27:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.770+0000 7fce553ff700 1 --2- 172.21.15.64:0/728463913 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce5010cdd0 0x7fce50198750 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fce4800ef30 tx=0x7fce4800c3a0 comp rx=0 tx=0).stop 2023-09-13T03:27:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.770+0000 7fce553ff700 1 -- 172.21.15.64:0/728463913 shutdown_connections 2023-09-13T03:27:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.770+0000 7fce553ff700 1 --2- 172.21.15.64:0/728463913 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fce380a0830 0x7fce380a2cf0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.770+0000 7fce553ff700 1 --2- 172.21.15.64:0/728463913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce50067f30 0x7fce5019e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.770+0000 7fce553ff700 1 --2- 172.21.15.64:0/728463913 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce5010cdd0 0x7fce50198750 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.770+0000 7fce553ff700 1 --2- 172.21.15.64:0/728463913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce50101470 0x7fce5019ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:27:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.770+0000 7fce553ff700 1 -- 172.21.15.64:0/728463913 >> 172.21.15.64:0/728463913 conn(0x7fce50074f30 msgr2=0x7fce500fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:27:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.771+0000 7fce553ff700 1 -- 172.21.15.64:0/728463913 shutdown_connections 2023-09-13T03:27:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:27:53.771+0000 7fce553ff700 1 -- 172.21.15.64:0/728463913 wait complete. 2023-09-13T03:27:54.292 INFO:tasks.workunit.client.0.smithi064.stdout: 10 100430 47.30 MB/sec warmup 118 sec latency 150.984 ms 2023-09-13T03:27:55.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:27:54 smithi170 ceph-mon[134328]: pgmap v461: 97 pgs: 97 active+clean; 867 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 824 op/s 2023-09-13T03:27:55.162 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:27:54 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:27:55.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:27:54 smithi064 ceph-mon[97675]: pgmap v461: 97 pgs: 97 active+clean; 867 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 824 op/s 2023-09-13T03:27:55.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:27:54 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:27:55.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:27:54 smithi142 ceph-mon[134082]: pgmap v461: 97 pgs: 97 active+clean; 867 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 824 op/s 2023-09-13T03:27:55.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:27:54 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:27:55.292 INFO:tasks.workunit.client.0.smithi064.stdout: 10 101289 47.36 MB/sec warmup 119 sec latency 279.005 ms 2023-09-13T03:27:57.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:27:56 smithi170 ceph-mon[134328]: pgmap v462: 97 pgs: 97 active+clean; 885 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 20 MiB/s wr, 1.30k op/s 2023-09-13T03:27:57.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:27:56 smithi064 ceph-mon[97675]: pgmap v462: 97 pgs: 97 active+clean; 885 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 20 MiB/s wr, 1.30k op/s 2023-09-13T03:27:57.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:27:56 smithi142 ceph-mon[134082]: pgmap v462: 97 pgs: 97 active+clean; 885 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 20 MiB/s wr, 1.30k op/s 2023-09-13T03:27:57.292 INFO:tasks.workunit.client.0.smithi064.stdout: 10 102890 31.18 MB/sec execute 1 sec latency 61.525 ms 2023-09-13T03:27:58.293 INFO:tasks.workunit.client.0.smithi064.stdout: 10 103978 41.27 MB/sec execute 2 sec latency 151.351 ms 2023-09-13T03:27:59.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:27:58 smithi170 ceph-mon[134328]: pgmap v463: 97 pgs: 97 active+clean; 885 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 873 op/s 2023-09-13T03:27:59.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:27:58 smithi064 ceph-mon[97675]: pgmap v463: 97 pgs: 97 active+clean; 885 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 873 op/s 2023-09-13T03:27:59.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:27:58 smithi142 ceph-mon[134082]: pgmap v463: 97 pgs: 97 active+clean; 885 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 873 op/s 2023-09-13T03:27:59.293 INFO:tasks.workunit.client.0.smithi064.stdout: 10 105057 50.04 MB/sec execute 3 sec latency 166.092 ms 2023-09-13T03:28:00.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:27:59 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:28:00.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:27:59 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:28:00.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:27:59 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:28:00.293 INFO:tasks.workunit.client.0.smithi064.stdout: 10 105559 44.96 MB/sec execute 4 sec latency 242.311 ms 2023-09-13T03:28:01.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:28:00 smithi170 ceph-mon[134328]: pgmap v464: 97 pgs: 97 active+clean; 885 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 873 op/s 2023-09-13T03:28:01.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:28:00 smithi064 ceph-mon[97675]: pgmap v464: 97 pgs: 97 active+clean; 885 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 873 op/s 2023-09-13T03:28:01.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:28:00 smithi142 ceph-mon[134082]: pgmap v464: 97 pgs: 97 active+clean; 885 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 873 op/s 2023-09-13T03:28:01.293 INFO:tasks.workunit.client.0.smithi064.stdout: 10 106134 42.37 MB/sec execute 5 sec latency 109.979 ms 2023-09-13T03:28:02.293 INFO:tasks.workunit.client.0.smithi064.stdout: 10 106575 38.04 MB/sec execute 6 sec latency 129.271 ms 2023-09-13T03:28:02.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:28:02 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:28:02.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:28:02 smithi170 ceph-mon[134328]: pgmap v465: 97 pgs: 97 active+clean; 918 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 20 MiB/s wr, 1.25k op/s 2023-09-13T03:28:02.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:28:02 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:28:02.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:28:02 smithi064 ceph-mon[97675]: pgmap v465: 97 pgs: 97 active+clean; 918 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 20 MiB/s wr, 1.25k op/s 2023-09-13T03:28:02.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:28:02 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:28:02.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:28:02 smithi142 ceph-mon[134082]: pgmap v465: 97 pgs: 97 active+clean; 918 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 20 MiB/s wr, 1.25k op/s 2023-09-13T03:28:03.293 INFO:tasks.workunit.client.0.smithi064.stdout: 10 107256 37.40 MB/sec execute 7 sec latency 104.309 ms 2023-09-13T03:28:04.294 INFO:tasks.workunit.client.0.smithi064.stdout: 10 108334 41.28 MB/sec execute 8 sec latency 164.162 ms 2023-09-13T03:28:05.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:28:04 smithi170 ceph-mon[134328]: pgmap v466: 97 pgs: 97 active+clean; 918 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 856 op/s 2023-09-13T03:28:05.172 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:28:04 smithi064 ceph-mon[97675]: pgmap v466: 97 pgs: 97 active+clean; 918 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 856 op/s 2023-09-13T03:28:05.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:28:04 smithi142 ceph-mon[134082]: pgmap v466: 97 pgs: 97 active+clean; 918 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 856 op/s 2023-09-13T03:28:05.295 INFO:tasks.workunit.client.0.smithi064.stdout: 10 108782 39.36 MB/sec execute 9 sec latency 323.073 ms 2023-09-13T03:28:06.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:28:05 smithi170 ceph-mon[134328]: osdmap e103: 12 total, 12 up, 12 in 2023-09-13T03:28:06.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:28:05 smithi064 ceph-mon[97675]: osdmap e103: 12 total, 12 up, 12 in 2023-09-13T03:28:06.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:28:05 smithi142 ceph-mon[134082]: osdmap e103: 12 total, 12 up, 12 in 2023-09-13T03:28:06.295 INFO:tasks.workunit.client.0.smithi064.stdout: 10 109193 37.88 MB/sec execute 10 sec latency 88.811 ms 2023-09-13T03:28:06.838 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:28:06 smithi170 ceph-mon[134328]: pgmap v468: 97 pgs: 16 active+clean+snaptrim, 48 active+clean, 33 active+clean+snaptrim_wait; 954 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 1.05k op/s 2023-09-13T03:28:06.839 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:28:06 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:28:06.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:06.996+0000 7f0812bff700 1 -- 172.21.15.64:0/3006240919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f080c0729a0 msgr2=0x7f080c072d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:28:06.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:06.996+0000 7f0812bff700 1 --2- 172.21.15.64:0/3006240919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f080c0729a0 0x7f080c072d80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0800009990 tx=0x7f0800009ca0 comp rx=0 tx=0).stop 2023-09-13T03:28:06.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:06.997+0000 7f0812bff700 1 -- 172.21.15.64:0/3006240919 shutdown_connections 2023-09-13T03:28:06.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:06.997+0000 7f0812bff700 1 --2- 172.21.15.64:0/3006240919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f080c0729a0 0x7f080c072d80 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:06.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:06.997+0000 7f0812bff700 1 --2- 172.21.15.64:0/3006240919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f080c1115a0 0x7f080c113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:06.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:06.997+0000 7f0812bff700 1 --2- 172.21.15.64:0/3006240919 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f080c0732c0 0x7f080c111060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:06.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:06.997+0000 7f0812bff700 1 -- 172.21.15.64:0/3006240919 >> 172.21.15.64:0/3006240919 conn(0x7f080c0fc060 msgr2=0x7f080c0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:28:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:06.997+0000 7f0812bff700 1 -- 172.21.15.64:0/3006240919 shutdown_connections 2023-09-13T03:28:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:06.997+0000 7f0812bff700 1 -- 172.21.15.64:0/3006240919 wait complete. 2023-09-13T03:28:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:06.999+0000 7f0812bff700 1 Processor -- start 2023-09-13T03:28:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:06.999+0000 7f0812bff700 1 -- start start 2023-09-13T03:28:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:06.999+0000 7f0812bff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f080c0729a0 0x7f080c1109f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:06.999+0000 7f0812bff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f080c0732c0 0x7f080c110f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:06.999+0000 7f0812bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f080c1115a0 0x7f080c10a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:06.999+0000 7f0812bff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080c10b090 con 0x7f080c0729a0 2023-09-13T03:28:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:06.999+0000 7f0812bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080c10b200 con 0x7f080c1115a0 2023-09-13T03:28:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:06.999+0000 7f0812bff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080c10b3c0 con 0x7f080c0732c0 2023-09-13T03:28:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.000+0000 7f080bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f080c0732c0 0x7f080c110f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:07.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.000+0000 7f080bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f080c0732c0 0x7f080c110f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:54014/0 (socket says 172.21.15.64:54014) 2023-09-13T03:28:07.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.000+0000 7f080bfff700 1 -- 172.21.15.64:0/4096648327 learned_addr learned my addr 172.21.15.64:0/4096648327 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:28:07.003 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.000+0000 7f081119c700 1 --2- 172.21.15.64:0/4096648327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f080c1115a0 0x7f080c10a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.000+0000 7f080bfff700 1 -- 172.21.15.64:0/4096648327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f080c1115a0 msgr2=0x7f080c10a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:28:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.000+0000 7f080bfff700 1 --2- 172.21.15.64:0/4096648327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f080c1115a0 0x7f080c10a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.000+0000 7f080bfff700 1 -- 172.21.15.64:0/4096648327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f080c0729a0 msgr2=0x7f080c1109f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:28:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.000+0000 7f080bfff700 1 --2- 172.21.15.64:0/4096648327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f080c0729a0 0x7f080c1109f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.000+0000 7f080bfff700 1 -- 172.21.15.64:0/4096648327 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f8009520 con 0x7f080c0732c0 2023-09-13T03:28:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.001+0000 7f080bfff700 1 --2- 172.21.15.64:0/4096648327 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f080c0732c0 0x7f080c110f30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f07f800eb30 tx=0x7f07f800ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:28:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.002+0000 7f0809ffb700 1 -- 172.21.15.64:0/4096648327 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f8009880 con 0x7f080c0732c0 2023-09-13T03:28:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.002+0000 7f0812bff700 1 -- 172.21.15.64:0/4096648327 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08000095f0 con 0x7f080c0732c0 2023-09-13T03:28:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.002+0000 7f0812bff700 1 -- 172.21.15.64:0/4096648327 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f080c1b2440 con 0x7f080c0732c0 2023-09-13T03:28:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.002+0000 7f0809ffb700 1 -- 172.21.15.64:0/4096648327 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07f80044e0 con 0x7f080c0732c0 2023-09-13T03:28:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.002+0000 7f0809ffb700 1 -- 172.21.15.64:0/4096648327 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f8003e90 con 0x7f080c0732c0 2023-09-13T03:28:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.008+0000 7f0826559b80 1 Processor -- start 2023-09-13T03:28:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.009+0000 7f0826559b80 1 -- start start 2023-09-13T03:28:07.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.009+0000 7f0826559b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f677cda1a0 0x55f677cda580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:07.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.009+0000 7f0826559b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f677cd4300 0x55f677cd4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:07.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.009+0000 7f0826559b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f677cd4cc0 0x55f677ce0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.009+0000 7f0826559b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f677c2d6d0 con 0x55f677cd4300 2023-09-13T03:28:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.009+0000 7f0826559b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f677c2aff0 con 0x55f677cd4cc0 2023-09-13T03:28:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.009+0000 7f0826559b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f677c268e0 con 0x55f677cda1a0 2023-09-13T03:28:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.010+0000 7f0809ffb700 1 -- 172.21.15.64:0/4096648327 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f07f8003680 con 0x7f080c0732c0 2023-09-13T03:28:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.010+0000 7f081119c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f677cda1a0 0x55f677cda580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.010+0000 7f081119c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f677cda1a0 0x55f677cda580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:54016/0 (socket says 172.21.15.64:54016) 2023-09-13T03:28:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.010+0000 7f081119c700 1 -- 172.21.15.64:0/4055142045 learned_addr learned my addr 172.21.15.64:0/4055142045 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:28:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.010+0000 7f080bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f677cd4cc0 0x55f677ce0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.011+0000 7f081099b700 1 --2- 172.21.15.64:0/4055142045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f677cd4300 0x55f677cd4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.011+0000 7f081119c700 1 -- 172.21.15.64:0/4055142045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f677cd4cc0 msgr2=0x55f677ce0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:28:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.011+0000 7f081119c700 1 --2- 172.21.15.64:0/4055142045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f677cd4cc0 0x55f677ce0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:07.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.011+0000 7f081119c700 1 -- 172.21.15.64:0/4055142045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f677cd4300 msgr2=0x55f677cd4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:28:07.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.011+0000 7f081119c700 1 --2- 172.21.15.64:0/4055142045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f677cd4300 0x55f677cd4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:07.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.011+0000 7f081119c700 1 -- 172.21.15.64:0/4055142045 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f677a52d80 con 0x55f677cda1a0 2023-09-13T03:28:07.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.011+0000 7f081119c700 1 --2- 172.21.15.64:0/4055142045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f677cda1a0 0x55f677cda580 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f080400a400 tx=0x7f0804009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a04e22c3d262c8a2 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:28:07.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.011+0000 7f07f6ffd700 1 -- 172.21.15.64:0/4055142045 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0804010040 con 0x55f677cda1a0 2023-09-13T03:28:07.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.012+0000 7f07f6ffd700 1 -- 172.21.15.64:0/4055142045 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0804014ae0 con 0x55f677cda1a0 2023-09-13T03:28:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.012+0000 7f07f6ffd700 1 -- 172.21.15.64:0/4055142045 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0804013af0 con 0x55f677cda1a0 2023-09-13T03:28:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.012+0000 7f0809ffb700 1 --2- 172.21.15.64:0/4096648327 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f07fc0a08b0 0x7f07fc0a2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.012+0000 7f0809ffb700 1 -- 172.21.15.64:0/4096648327 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f07f8014070 con 0x7f080c0732c0 2023-09-13T03:28:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.013+0000 7f081099b700 1 --2- 172.21.15.64:0/4096648327 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f07fc0a08b0 0x7f07fc0a2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.013+0000 7f0826559b80 1 -- 172.21.15.64:0/4055142045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f677cda1a0 msgr2=0x55f677cda580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:28:07.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.013+0000 7f0826559b80 1 --2- 172.21.15.64:0/4055142045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f677cda1a0 0x55f677cda580 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f080400a400 tx=0x7f0804009650 comp rx=0 tx=0).stop 2023-09-13T03:28:07.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.014+0000 7f081099b700 1 --2- 172.21.15.64:0/4096648327 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f07fc0a08b0 0x7f07fc0a2d70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f080000b2e0 tx=0x7f080000b610 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:28:07.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.014+0000 7f0826559b80 1 -- 172.21.15.64:0/4055142045 shutdown_connections 2023-09-13T03:28:07.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.014+0000 7f0826559b80 1 --2- 172.21.15.64:0/4055142045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f677cd4cc0 0x55f677ce0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:07.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.014+0000 7f0826559b80 1 --2- 172.21.15.64:0/4055142045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f677cda1a0 0x55f677cda580 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:07.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.014+0000 7f0826559b80 1 --2- 172.21.15.64:0/4055142045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f677cd4300 0x55f677cd4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:07.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.014+0000 7f0826559b80 1 -- 172.21.15.64:0/4055142045 >> 172.21.15.64:0/4055142045 conn(0x55f677c49030 msgr2=0x55f677ccf560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:28:07.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.017+0000 7f0826559b80 1 -- 172.21.15.64:0/4055142045 shutdown_connections 2023-09-13T03:28:07.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.017+0000 7f0826559b80 1 -- 172.21.15.64:0/4055142045 wait complete. 2023-09-13T03:28:07.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.018+0000 7f0826559b80 1 -- start start 2023-09-13T03:28:07.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.018+0000 7f0826559b80 1 Processor -- start 2023-09-13T03:28:07.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.019+0000 7f0826559b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f677cd4300 0x55f677ce0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:07.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.019+0000 7f0826559b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f677cd4cc0 0x55f677cde020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:07.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.019+0000 7f0826559b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f677cda1a0 0x55f677cde5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.019+0000 7f0826559b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f677bc4b10 con 0x55f677cda1a0 2023-09-13T03:28:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.019+0000 7f0826559b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f677bc4810 con 0x55f677cd4300 2023-09-13T03:28:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.019+0000 7f0826559b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f677aa88c0 con 0x55f677cd4cc0 2023-09-13T03:28:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.020+0000 7f0826559b80 10 client.0 _ll_register_callbacks cb 0x55f677b0ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:28:07.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.020+0000 7f081099b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f677cd4300 0x55f677ce0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.020+0000 7f081099b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f677cd4300 0x55f677ce0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:34692/0 (socket says 172.21.15.64:34692) 2023-09-13T03:28:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.020+0000 7f081099b700 1 -- 172.21.15.64:0/316093247 learned_addr learned my addr 172.21.15.64:0/316093247 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:28:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.020+0000 7f080bfff700 1 --2- 172.21.15.64:0/316093247 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f677cd4cc0 0x55f677cde020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.020+0000 7f081099b700 1 -- 172.21.15.64:0/316093247 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f677cd4cc0 msgr2=0x55f677cde020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:28:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.020+0000 7f081099b700 1 --2- 172.21.15.64:0/316093247 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f677cd4cc0 0x55f677cde020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:07.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.020+0000 7f081099b700 1 -- 172.21.15.64:0/316093247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f677cda1a0 msgr2=0x55f677cde5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:28:07.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.020+0000 7f081119c700 1 --2- 172.21.15.64:0/316093247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f677cda1a0 0x55f677cde5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:07.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.020+0000 7f081099b700 1 --2- 172.21.15.64:0/316093247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f677cda1a0 0x55f677cde5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:07.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.020+0000 7f081099b700 1 -- 172.21.15.64:0/316093247 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0804009c60 con 0x55f677cd4300 2023-09-13T03:28:07.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.021+0000 7f081119c700 1 --2- 172.21.15.64:0/316093247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f677cda1a0 0x55f677cde5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T03:28:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.021+0000 7f081099b700 1 --2- 172.21.15.64:0/316093247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f677cd4300 0x55f677ce0e10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0800015000 tx=0x7f0800009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d216b5d859a1c212 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:28:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.021+0000 7f07ebfff700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:28:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.022+0000 7f07ebfff700 1 -- 172.21.15.64:0/316093247 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080002b070 con 0x55f677cd4300 2023-09-13T03:28:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.022+0000 7f0826559b80 1 -- 172.21.15.64:0/316093247 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f677b7e490 con 0x55f677cd4300 2023-09-13T03:28:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.022+0000 7f07ebfff700 1 -- 172.21.15.64:0/316093247 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f0800004810 con 0x55f677cd4300 2023-09-13T03:28:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.022+0000 7f07ebfff700 1 -- 172.21.15.64:0/316093247 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08000049d0 con 0x55f677cd4300 2023-09-13T03:28:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.022+0000 7f0826559b80 10 client.27311 fetch_fsmap learned FSMap version 74 2023-09-13T03:28:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.022+0000 7f0826559b80 1 -- 172.21.15.64:0/316093247 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x55f677a3bc00 con 0x55f677cd4300 2023-09-13T03:28:07.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.022+0000 7f07ebfff700 1 -- 172.21.15.64:0/316093247 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f080003c020 con 0x55f677cd4300 2023-09-13T03:28:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.023+0000 7f07ebfff700 1 -- 172.21.15.64:0/316093247 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 74) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f080000fc40 con 0x55f677cd4300 2023-09-13T03:28:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.023+0000 7f0826559b80 10 client.27311 fetch_fsmap finished waiting for FSMap version 74 2023-09-13T03:28:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.023+0000 7f0826559b80 10 client.27311 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:28:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.023+0000 7f0826559b80 20 client.27311 populate_metadata read hostname 'smithi064' 2023-09-13T03:28:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.023+0000 7f0826559b80 1 --2- 172.21.15.64:0/316093247 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f677cee500 0x55f677cf0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.023+0000 7f0826559b80 4 client.27311 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:28:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.023+0000 7f0826559b80 1 -- 172.21.15.64:0/316093247 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f677c3b840 con 0x55f677cee500 2023-09-13T03:28:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.023+0000 7f080bfff700 1 --2- 172.21.15.64:0/316093247 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f677cee500 0x55f677cf0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.024+0000 7f080bfff700 1 --2- 172.21.15.64:0/316093247 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f677cee500 0x55f677cf0920 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=639719f49bf74c93 server_cookie=752bcc674843ccda in_seq=0 out_seq=0 2023-09-13T03:28:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.024+0000 7f07ebfff700 10 client.27311 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:28:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.026+0000 7f07ebfff700 1 -- 172.21.15.64:0/316093247 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f677c3b840 con 0x55f677cee500 2023-09-13T03:28:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.026+0000 7f07ebfff700 10 client.27311 handle_command_reply: tid=0 2023-09-13T03:28:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.026+0000 7f0826559b80 1 client.27311 shutdown 2023-09-13T03:28:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.027+0000 7f0826559b80 1 -- 172.21.15.64:0/316093247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f677cd4300 msgr2=0x55f677ce0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:28:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.027+0000 7f0826559b80 1 --2- 172.21.15.64:0/316093247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f677cd4300 0x55f677ce0e10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0800015000 tx=0x7f0800009310 comp rx=0 tx=0).stop 2023-09-13T03:28:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.027+0000 7f0826559b80 1 -- 172.21.15.64:0/316093247 shutdown_connections 2023-09-13T03:28:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.027+0000 7f0826559b80 1 --2- 172.21.15.64:0/316093247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f677cd4300 0x55f677ce0e10 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.027+0000 7f0826559b80 1 --2- 172.21.15.64:0/316093247 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f677cee500 0x55f677cf0920 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.027+0000 7f0826559b80 1 --2- 172.21.15.64:0/316093247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f677cda1a0 0x55f677cde5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.027+0000 7f0826559b80 1 --2- 172.21.15.64:0/316093247 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f677cd4cc0 0x55f677cde020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.027+0000 7f07ebfff700 0 client.27311 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:28:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.027+0000 7f0826559b80 1 -- 172.21.15.64:0/316093247 >> 172.21.15.64:0/316093247 conn(0x55f677c49030 msgr2=0x55f677cce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:28:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.027+0000 7f0826559b80 1 -- 172.21.15.64:0/316093247 shutdown_connections 2023-09-13T03:28:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.027+0000 7f0826559b80 1 -- 172.21.15.64:0/316093247 wait complete. 2023-09-13T03:28:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.028+0000 7f0826559b80 20 client.27311 trim_cache size 0 max 16384 2023-09-13T03:28:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.051+0000 7f0826559b80 1 Processor -- start 2023-09-13T03:28:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.051+0000 7f0826559b80 1 -- start start 2023-09-13T03:28:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.051+0000 7f0826559b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f677cd4300 0x55f677cddfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.051+0000 7f0826559b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f677cd4cc0 0x55f677ce0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.051+0000 7f0826559b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f677cda1a0 0x55f677ccef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.051+0000 7f0826559b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f677cd5290 con 0x55f677cd4300 2023-09-13T03:28:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.051+0000 7f0826559b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f677cd0380 con 0x55f677cd4cc0 2023-09-13T03:28:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.051+0000 7f0826559b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f677ccbfc0 con 0x55f677cda1a0 2023-09-13T03:28:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.052+0000 7f080bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f677cd4cc0 0x55f677ce0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.052+0000 7f081099b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f677cd4300 0x55f677cddfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.052+0000 7f080bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f677cd4cc0 0x55f677ce0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:34698/0 (socket says 172.21.15.64:34698) 2023-09-13T03:28:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.052+0000 7f080bfff700 1 -- 172.21.15.64:0/449849050 learned_addr learned my addr 172.21.15.64:0/449849050 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:28:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.052+0000 7f081119c700 1 --2- 172.21.15.64:0/449849050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f677cda1a0 0x55f677ccef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.053+0000 7f081119c700 1 -- 172.21.15.64:0/449849050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f677cd4cc0 msgr2=0x55f677ce0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:28:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.053+0000 7f081119c700 1 --2- 172.21.15.64:0/449849050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f677cd4cc0 0x55f677ce0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.053+0000 7f081119c700 1 -- 172.21.15.64:0/449849050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f677cd4300 msgr2=0x55f677cddfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:28:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.053+0000 7f081119c700 1 --2- 172.21.15.64:0/449849050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f677cd4300 0x55f677cddfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.053+0000 7f081119c700 1 -- 172.21.15.64:0/449849050 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f677a791f0 con 0x55f677cda1a0 2023-09-13T03:28:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.053+0000 7f081119c700 1 --2- 172.21.15.64:0/449849050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f677cda1a0 0x55f677ccef20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0804009c30 tx=0x7f08040093f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2fa1d75196193f server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:28:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.054+0000 7f07f77fe700 1 -- 172.21.15.64:0/449849050 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0804010070 con 0x55f677cda1a0 2023-09-13T03:28:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.054+0000 7f07f77fe700 1 -- 172.21.15.64:0/449849050 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0804027460 con 0x55f677cda1a0 2023-09-13T03:28:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.054+0000 7f07f77fe700 1 -- 172.21.15.64:0/449849050 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080401e620 con 0x55f677cda1a0 2023-09-13T03:28:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.055+0000 7f0826559b80 1 -- 172.21.15.64:0/449849050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f677cda1a0 msgr2=0x55f677ccef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:28:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.055+0000 7f0826559b80 1 --2- 172.21.15.64:0/449849050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f677cda1a0 0x55f677ccef20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0804009c30 tx=0x7f08040093f0 comp rx=0 tx=0).stop 2023-09-13T03:28:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.055+0000 7f0826559b80 1 -- 172.21.15.64:0/449849050 shutdown_connections 2023-09-13T03:28:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.055+0000 7f0826559b80 1 --2- 172.21.15.64:0/449849050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f677cda1a0 0x55f677ccef20 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.055+0000 7f0826559b80 1 --2- 172.21.15.64:0/449849050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f677cd4300 0x55f677cddfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.055+0000 7f0826559b80 1 --2- 172.21.15.64:0/449849050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f677cd4cc0 0x55f677ce0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.055+0000 7f0826559b80 1 -- 172.21.15.64:0/449849050 >> 172.21.15.64:0/449849050 conn(0x55f677c49030 msgr2=0x55f677cdd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:28:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.056+0000 7f0826559b80 1 -- 172.21.15.64:0/449849050 shutdown_connections 2023-09-13T03:28:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.056+0000 7f0826559b80 1 -- 172.21.15.64:0/449849050 wait complete. 2023-09-13T03:28:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.056+0000 7f0826559b80 1 -- start start 2023-09-13T03:28:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.056+0000 7f0826559b80 1 Processor -- start 2023-09-13T03:28:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.057+0000 7f0826559b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f677cd4300 0x55f677ce5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.057+0000 7f0826559b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f677cd4cc0 0x55f677ce3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.057+0000 7f0826559b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f677cda1a0 0x55f677ce3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.057+0000 7f0826559b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f677ce4560 con 0x55f677cd4cc0 2023-09-13T03:28:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.057+0000 7f0826559b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f677ce46d0 con 0x55f677cda1a0 2023-09-13T03:28:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.057+0000 7f0826559b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f677d42380 con 0x55f677cd4300 2023-09-13T03:28:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.057+0000 7f0826559b80 10 client.0 _ll_register_callbacks cb 0x7f080000bac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:28:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.058+0000 7f081099b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f677cd4300 0x55f677ce5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.058+0000 7f081119c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f677cda1a0 0x55f677ce3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.058+0000 7f081119c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f677cda1a0 0x55f677ce3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:34704/0 (socket says 172.21.15.64:34704) 2023-09-13T03:28:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.058+0000 7f081119c700 1 -- 172.21.15.64:0/1832166417 learned_addr learned my addr 172.21.15.64:0/1832166417 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:28:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.058+0000 7f080bfff700 1 --2- 172.21.15.64:0/1832166417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f677cd4cc0 0x55f677ce3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.059+0000 7f081119c700 1 -- 172.21.15.64:0/1832166417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f677cd4300 msgr2=0x55f677ce5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:28:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.059+0000 7f081119c700 1 --2- 172.21.15.64:0/1832166417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f677cd4300 0x55f677ce5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.059+0000 7f081119c700 1 -- 172.21.15.64:0/1832166417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f677cd4cc0 msgr2=0x55f677ce3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:28:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.059+0000 7f081119c700 1 --2- 172.21.15.64:0/1832166417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f677cd4cc0 0x55f677ce3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.059+0000 7f081119c700 1 -- 172.21.15.64:0/1832166417 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f677a52d80 con 0x55f677cda1a0 2023-09-13T03:28:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.059+0000 7f081119c700 1 --2- 172.21.15.64:0/1832166417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f677cda1a0 0x55f677ce3e80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0804004ae0 tx=0x7f08040275d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46bb0f262ad4223 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:28:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.059+0000 7f07ebfff700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:28:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.059+0000 7f07ebfff700 1 -- 172.21.15.64:0/1832166417 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0804010040 con 0x55f677cda1a0 2023-09-13T03:28:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.059+0000 7f07ebfff700 1 -- 172.21.15.64:0/1832166417 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f0804006e40 con 0x55f677cda1a0 2023-09-13T03:28:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.060+0000 7f0826559b80 1 -- 172.21.15.64:0/1832166417 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0804014ae0 con 0x55f677cda1a0 2023-09-13T03:28:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.060+0000 7f07ebfff700 1 -- 172.21.15.64:0/1832166417 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0804014620 con 0x55f677cda1a0 2023-09-13T03:28:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.060+0000 7f0826559b80 10 client.27317 fetch_fsmap learned FSMap version 74 2023-09-13T03:28:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.060+0000 7f0826559b80 1 -- 172.21.15.64:0/1832166417 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x55f677a51250 con 0x55f677cda1a0 2023-09-13T03:28:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.060+0000 7f07ebfff700 1 -- 172.21.15.64:0/1832166417 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0804014ae0 con 0x55f677cda1a0 2023-09-13T03:28:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.060+0000 7f07ebfff700 1 -- 172.21.15.64:0/1832166417 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 74) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f0804008030 con 0x55f677cda1a0 2023-09-13T03:28:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.061+0000 7f0826559b80 10 client.27317 fetch_fsmap finished waiting for FSMap version 74 2023-09-13T03:28:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.061+0000 7f0826559b80 10 client.27317 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:28:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.061+0000 7f0826559b80 20 client.27317 populate_metadata read hostname 'smithi064' 2023-09-13T03:28:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.061+0000 7f0826559b80 1 --2- 172.21.15.64:0/1832166417 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f677d435a0 0x55f677d459c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.061+0000 7f0826559b80 4 client.27317 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:28:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.061+0000 7f0826559b80 1 -- 172.21.15.64:0/1832166417 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f677c1f8c0 con 0x55f677d435a0 2023-09-13T03:28:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.061+0000 7f081099b700 1 --2- 172.21.15.64:0/1832166417 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f677d435a0 0x55f677d459c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.062+0000 7f081099b700 1 --2- 172.21.15.64:0/1832166417 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f677d435a0 0x55f677d459c0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23cdebfb39484f94 server_cookie=37c67a002fce4228 in_seq=0 out_seq=0 2023-09-13T03:28:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.062+0000 7f07ebfff700 10 client.27317 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:28:07.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:28:06 smithi064 ceph-mon[97675]: pgmap v468: 97 pgs: 16 active+clean+snaptrim, 48 active+clean, 33 active+clean+snaptrim_wait; 954 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 1.05k op/s 2023-09-13T03:28:07.170 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:28:06 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:28:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.236+0000 7f07ebfff700 1 -- 172.21.15.64:0/1832166417 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f677c1f8c0 con 0x55f677d435a0 2023-09-13T03:28:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.236+0000 7f07ebfff700 10 client.27317 handle_command_reply: tid=0 2023-09-13T03:28:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.236+0000 7f0826559b80 1 client.27317 shutdown 2023-09-13T03:28:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.237+0000 7f0826559b80 1 -- 172.21.15.64:0/1832166417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f677cda1a0 msgr2=0x55f677ce3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:28:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.237+0000 7f0826559b80 1 --2- 172.21.15.64:0/1832166417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f677cda1a0 0x55f677ce3e80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0804004ae0 tx=0x7f08040275d0 comp rx=0 tx=0).stop 2023-09-13T03:28:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.237+0000 7f0826559b80 1 -- 172.21.15.64:0/1832166417 shutdown_connections 2023-09-13T03:28:07.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.237+0000 7f0826559b80 1 --2- 172.21.15.64:0/1832166417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f677cda1a0 0x55f677ce3e80 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:07.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.237+0000 7f0826559b80 1 --2- 172.21.15.64:0/1832166417 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55f677d435a0 0x55f677d459c0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:07.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.237+0000 7f0826559b80 1 --2- 172.21.15.64:0/1832166417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f677cd4300 0x55f677ce5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:07.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.237+0000 7f0826559b80 1 --2- 172.21.15.64:0/1832166417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f677cd4cc0 0x55f677ce3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:07.242 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.237+0000 7f0826559b80 1 -- 172.21.15.64:0/1832166417 >> 172.21.15.64:0/1832166417 conn(0x55f677c49030 msgr2=0x55f677cdc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:28:07.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.237+0000 7f07ebfff700 0 client.27317 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:28:07.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.237+0000 7f0826559b80 1 -- 172.21.15.64:0/1832166417 shutdown_connections 2023-09-13T03:28:07.243 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.238+0000 7f0826559b80 1 -- 172.21.15.64:0/1832166417 wait complete. 2023-09-13T03:28:07.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.241+0000 7f0826559b80 20 client.27317 trim_cache size 0 max 16384 2023-09-13T03:28:07.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:28:07.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:28:07.244 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:28:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:28:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.242+0000 7f0812bff700 1 -- 172.21.15.64:0/4096648327 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f07fc0a08b0 msgr2=0x7f07fc0a2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:28:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.242+0000 7f0812bff700 1 --2- 172.21.15.64:0/4096648327 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f07fc0a08b0 0x7f07fc0a2d70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f080000b2e0 tx=0x7f080000b610 comp rx=0 tx=0).stop 2023-09-13T03:28:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.242+0000 7f0812bff700 1 -- 172.21.15.64:0/4096648327 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f080c0732c0 msgr2=0x7f080c110f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:28:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.242+0000 7f0812bff700 1 --2- 172.21.15.64:0/4096648327 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f080c0732c0 0x7f080c110f30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f07f800eb30 tx=0x7f07f800ee40 comp rx=0 tx=0).stop 2023-09-13T03:28:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.243+0000 7f0812bff700 1 -- 172.21.15.64:0/4096648327 shutdown_connections 2023-09-13T03:28:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.243+0000 7f0812bff700 1 --2- 172.21.15.64:0/4096648327 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f07fc0a08b0 0x7f07fc0a2d70 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.243+0000 7f0812bff700 1 --2- 172.21.15.64:0/4096648327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f080c1115a0 0x7f080c10a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.243+0000 7f0812bff700 1 --2- 172.21.15.64:0/4096648327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f080c0729a0 0x7f080c1109f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.243+0000 7f0812bff700 1 --2- 172.21.15.64:0/4096648327 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f080c0732c0 0x7f080c110f30 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.243+0000 7f0812bff700 1 -- 172.21.15.64:0/4096648327 >> 172.21.15.64:0/4096648327 conn(0x7f080c0fc060 msgr2=0x7f080c0fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:28:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.244+0000 7f0812bff700 1 -- 172.21.15.64:0/4096648327 shutdown_connections 2023-09-13T03:28:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:07.244+0000 7f0812bff700 1 -- 172.21.15.64:0/4096648327 wait complete. 2023-09-13T03:28:07.248 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:28:06 smithi142 ceph-mon[134082]: pgmap v468: 97 pgs: 16 active+clean+snaptrim, 48 active+clean, 33 active+clean+snaptrim_wait; 954 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 1.05k op/s 2023-09-13T03:28:07.248 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:28:06 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:28:07.299 INFO:tasks.workunit.client.0.smithi064.stdout: 10 110028 37.87 MB/sec execute 11 sec latency 146.865 ms 2023-09-13T03:28:08.299 INFO:tasks.workunit.client.0.smithi064.stdout: 10 111286 40.47 MB/sec execute 12 sec latency 39.655 ms 2023-09-13T03:28:09.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:28:08 smithi170 ceph-mon[134328]: pgmap v469: 97 pgs: 16 active+clean+snaptrim, 48 active+clean, 33 active+clean+snaptrim_wait; 954 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 1.05k op/s 2023-09-13T03:28:09.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:28:08 smithi064 ceph-mon[97675]: pgmap v469: 97 pgs: 16 active+clean+snaptrim, 48 active+clean, 33 active+clean+snaptrim_wait; 954 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 1.05k op/s 2023-09-13T03:28:09.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:28:08 smithi142 ceph-mon[134082]: pgmap v469: 97 pgs: 16 active+clean+snaptrim, 48 active+clean, 33 active+clean+snaptrim_wait; 954 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 1.05k op/s 2023-09-13T03:28:09.299 INFO:tasks.workunit.client.0.smithi064.stdout: 10 112161 41.51 MB/sec execute 13 sec latency 54.571 ms 2023-09-13T03:28:10.299 INFO:tasks.workunit.client.0.smithi064.stdout: 10 112611 39.90 MB/sec execute 14 sec latency 293.407 ms 2023-09-13T03:28:10.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:28:10 smithi170 ceph-mon[134328]: pgmap v470: 97 pgs: 16 active+clean+snaptrim, 48 active+clean, 33 active+clean+snaptrim_wait; 954 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 1.05k op/s 2023-09-13T03:28:10.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:28:10 smithi064 ceph-mon[97675]: pgmap v470: 97 pgs: 16 active+clean+snaptrim, 48 active+clean, 33 active+clean+snaptrim_wait; 954 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 1.05k op/s 2023-09-13T03:28:10.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:28:10 smithi142 ceph-mon[134082]: pgmap v470: 97 pgs: 16 active+clean+snaptrim, 48 active+clean, 33 active+clean+snaptrim_wait; 954 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 1.05k op/s 2023-09-13T03:28:11.299 INFO:tasks.workunit.client.0.smithi064.stdout: 10 113400 40.09 MB/sec execute 15 sec latency 105.254 ms 2023-09-13T03:28:12.300 INFO:tasks.workunit.client.0.smithi064.stdout: 10 114312 40.45 MB/sec execute 16 sec latency 91.025 ms 2023-09-13T03:28:13.164 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:28:12 smithi170 ceph-mon[134328]: pgmap v471: 97 pgs: 97 active+clean; 957 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 1.13k op/s 2023-09-13T03:28:13.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:28:12 smithi064 ceph-mon[97675]: pgmap v471: 97 pgs: 97 active+clean; 957 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 1.13k op/s 2023-09-13T03:28:13.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:28:12 smithi142 ceph-mon[134082]: pgmap v471: 97 pgs: 97 active+clean; 957 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 1.13k op/s 2023-09-13T03:28:13.300 INFO:tasks.workunit.client.0.smithi064.stdout: 10 115261 41.34 MB/sec execute 17 sec latency 48.204 ms 2023-09-13T03:28:14.300 INFO:tasks.workunit.client.0.smithi064.stdout: 10 116128 41.36 MB/sec execute 18 sec latency 140.993 ms 2023-09-13T03:28:14.855 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:28:14 smithi170 ceph-mon[134328]: pgmap v472: 97 pgs: 97 active+clean; 957 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 1.13k op/s 2023-09-13T03:28:14.855 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:28:14 smithi170 ceph-mon[134328]: osdmap e104: 12 total, 12 up, 12 in 2023-09-13T03:28:15.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:28:14 smithi064 ceph-mon[97675]: pgmap v472: 97 pgs: 97 active+clean; 957 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 1.13k op/s 2023-09-13T03:28:15.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:28:14 smithi064 ceph-mon[97675]: osdmap e104: 12 total, 12 up, 12 in 2023-09-13T03:28:15.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:28:14 smithi142 ceph-mon[134082]: pgmap v472: 97 pgs: 97 active+clean; 957 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 1.13k op/s 2023-09-13T03:28:15.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:28:14 smithi142 ceph-mon[134082]: osdmap e104: 12 total, 12 up, 12 in 2023-09-13T03:28:15.300 INFO:tasks.workunit.client.0.smithi064.stdout: 10 116732 41.13 MB/sec execute 19 sec latency 255.879 ms 2023-09-13T03:28:16.300 INFO:tasks.workunit.client.0.smithi064.stdout: 10 117417 40.96 MB/sec execute 20 sec latency 162.178 ms 2023-09-13T03:28:17.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:28:16 smithi170 ceph-mon[134328]: pgmap v474: 97 pgs: 97 active+clean; 1009 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 985 op/s 2023-09-13T03:28:17.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:28:16 smithi064 ceph-mon[97675]: pgmap v474: 97 pgs: 97 active+clean; 1009 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 985 op/s 2023-09-13T03:28:17.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:28:16 smithi142 ceph-mon[134082]: pgmap v474: 97 pgs: 97 active+clean; 1009 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 985 op/s 2023-09-13T03:28:17.300 INFO:tasks.workunit.client.0.smithi064.stdout: 10 118476 41.63 MB/sec execute 21 sec latency 39.415 ms 2023-09-13T03:28:18.301 INFO:tasks.workunit.client.0.smithi064.stdout: 10 119373 41.96 MB/sec execute 22 sec latency 99.298 ms 2023-09-13T03:28:19.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:28:18 smithi170 ceph-mon[134328]: pgmap v475: 97 pgs: 97 active+clean; 1009 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 985 op/s 2023-09-13T03:28:19.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:28:18 smithi064 ceph-mon[97675]: pgmap v475: 97 pgs: 97 active+clean; 1009 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 985 op/s 2023-09-13T03:28:19.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:28:18 smithi142 ceph-mon[134082]: pgmap v475: 97 pgs: 97 active+clean; 1009 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 985 op/s 2023-09-13T03:28:19.300 INFO:tasks.workunit.client.0.smithi064.stdout: 10 120333 42.08 MB/sec execute 23 sec latency 57.532 ms 2023-09-13T03:28:20.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:28:19 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:28:20.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:28:19 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:28:20.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:28:19 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:28:20.301 INFO:tasks.workunit.client.0.smithi064.stdout: 10 121065 42.33 MB/sec execute 24 sec latency 235.836 ms 2023-09-13T03:28:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.481+0000 7fe9775ff700 1 -- 172.21.15.64:0/1904977364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9700ff760 msgr2=0x7fe9700ffbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:28:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.481+0000 7fe9775ff700 1 --2- 172.21.15.64:0/1904977364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9700ff760 0x7fe9700ffbe0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fe96000fb20 tx=0x7fe96000fe30 comp rx=0 tx=0).stop 2023-09-13T03:28:20.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.482+0000 7fe9775ff700 1 -- 172.21.15.64:0/1904977364 shutdown_connections 2023-09-13T03:28:20.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.482+0000 7fe9775ff700 1 --2- 172.21.15.64:0/1904977364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe970100120 0x7fe97010c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.482+0000 7fe9775ff700 1 --2- 172.21.15.64:0/1904977364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9701055b0 0x7fe970105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.482+0000 7fe9775ff700 1 --2- 172.21.15.64:0/1904977364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9700ff760 0x7fe9700ffbe0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.482+0000 7fe9775ff700 1 -- 172.21.15.64:0/1904977364 >> 172.21.15.64:0/1904977364 conn(0x7fe970074f30 msgr2=0x7fe970075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:28:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.483+0000 7fe9775ff700 1 -- 172.21.15.64:0/1904977364 shutdown_connections 2023-09-13T03:28:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.483+0000 7fe9775ff700 1 -- 172.21.15.64:0/1904977364 wait complete. 2023-09-13T03:28:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.484+0000 7fe9775ff700 1 Processor -- start 2023-09-13T03:28:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.484+0000 7fe9775ff700 1 -- start start 2023-09-13T03:28:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.484+0000 7fe9775ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9700ff760 0x7fe97006b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.485+0000 7fe9775ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe970100120 0x7fe97006bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.485+0000 7fe9775ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9701055b0 0x7fe970070430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.485+0000 7fe9775ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe970070970 con 0x7fe9700ff760 2023-09-13T03:28:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.485+0000 7fe9775ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe970070ae0 con 0x7fe9701055b0 2023-09-13T03:28:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.485+0000 7fe9775ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe970070ca0 con 0x7fe970100120 2023-09-13T03:28:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.485+0000 7fe975b9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9701055b0 0x7fe970070430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.485+0000 7fe975b9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9701055b0 0x7fe970070430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:58056/0 (socket says 172.21.15.64:58056) 2023-09-13T03:28:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.485+0000 7fe975b9c700 1 -- 172.21.15.64:0/2933104379 learned_addr learned my addr 172.21.15.64:0/2933104379 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:28:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.486+0000 7fe974b9a700 1 --2- 172.21.15.64:0/2933104379 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe970100120 0x7fe97006bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.486+0000 7fe975b9c700 1 -- 172.21.15.64:0/2933104379 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe970100120 msgr2=0x7fe97006bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:28:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.486+0000 7fe975b9c700 1 --2- 172.21.15.64:0/2933104379 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe970100120 0x7fe97006bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.486+0000 7fe975b9c700 1 -- 172.21.15.64:0/2933104379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9700ff760 msgr2=0x7fe97006b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:28:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.486+0000 7fe975b9c700 1 --2- 172.21.15.64:0/2933104379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9700ff760 0x7fe97006b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:20.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.486+0000 7fe975b9c700 1 -- 172.21.15.64:0/2933104379 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe96000f750 con 0x7fe9701055b0 2023-09-13T03:28:20.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.486+0000 7fe975b9c700 1 --2- 172.21.15.64:0/2933104379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9701055b0 0x7fe970070430 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe968009210 tx=0x7fe968009520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:28:20.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.487+0000 7fe9667fc700 1 -- 172.21.15.64:0/2933104379 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe968004830 con 0x7fe9701055b0 2023-09-13T03:28:20.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.487+0000 7fe9667fc700 1 -- 172.21.15.64:0/2933104379 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe968004e50 con 0x7fe9701055b0 2023-09-13T03:28:20.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.487+0000 7fe9667fc700 1 -- 172.21.15.64:0/2933104379 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe968011490 con 0x7fe9701055b0 2023-09-13T03:28:20.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.487+0000 7fe9775ff700 1 -- 172.21.15.64:0/2933104379 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe970070f80 con 0x7fe9701055b0 2023-09-13T03:28:20.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.487+0000 7fe9775ff700 1 -- 172.21.15.64:0/2933104379 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9701badc0 con 0x7fe9701055b0 2023-09-13T03:28:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.493+0000 7fe98af70b80 1 Processor -- start 2023-09-13T03:28:20.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.495+0000 7fe98af70b80 1 -- start start 2023-09-13T03:28:20.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.496+0000 7fe98af70b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628c37691a0 0x5628c3769580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:20.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.496+0000 7fe98af70b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5628c3763300 0x5628c3763780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:20.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.496+0000 7fe98af70b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5628c3763cc0 0x5628c376ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:20.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.496+0000 7fe98af70b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628c36bc6d0 con 0x5628c3763cc0 2023-09-13T03:28:20.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.496+0000 7fe98af70b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628c36b9ff0 con 0x5628c37691a0 2023-09-13T03:28:20.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.496+0000 7fe98af70b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628c36b58e0 con 0x5628c3763300 2023-09-13T03:28:20.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.496+0000 7fe9667fc700 1 -- 172.21.15.64:0/2933104379 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fe9680115f0 con 0x7fe9701055b0 2023-09-13T03:28:20.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.497+0000 7fe974b9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628c37691a0 0x5628c3769580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:20.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.497+0000 7fe974b9a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628c37691a0 0x5628c3769580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:58058/0 (socket says 172.21.15.64:58058) 2023-09-13T03:28:20.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.497+0000 7fe974b9a700 1 -- 172.21.15.64:0/4200294291 learned_addr learned my addr 172.21.15.64:0/4200294291 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:28:20.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.497+0000 7fe974b9a700 1 --2- 172.21.15.64:0/4200294291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5628c3763cc0 0x5628c376ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:20.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.498+0000 7fe974b9a700 1 -- 172.21.15.64:0/4200294291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5628c3763300 msgr2=0x5628c3763780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:28:20.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.498+0000 7fe974b9a700 1 --2- 172.21.15.64:0/4200294291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5628c3763300 0x5628c3763780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:20.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.498+0000 7fe974b9a700 1 -- 172.21.15.64:0/4200294291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5628c3763cc0 msgr2=0x5628c376ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:28:20.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.498+0000 7fe974b9a700 1 --2- 172.21.15.64:0/4200294291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5628c3763cc0 0x5628c376ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:20.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.498+0000 7fe974b9a700 1 -- 172.21.15.64:0/4200294291 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628c34e1d80 con 0x5628c37691a0 2023-09-13T03:28:20.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.498+0000 7fe974b9a700 1 --2- 172.21.15.64:0/4200294291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628c37691a0 0x5628c3769580 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe96000fb20 tx=0x7fe96001b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4061bc7e9fec9c9 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:28:20.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.499+0000 7fe95b7fe700 1 -- 172.21.15.64:0/4200294291 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96001e040 con 0x5628c37691a0 2023-09-13T03:28:20.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.499+0000 7fe95b7fe700 1 -- 172.21.15.64:0/4200294291 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe96001f040 con 0x5628c37691a0 2023-09-13T03:28:20.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.499+0000 7fe95b7fe700 1 -- 172.21.15.64:0/4200294291 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9600041b0 con 0x5628c37691a0 2023-09-13T03:28:20.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.500+0000 7fe9667fc700 1 --2- 172.21.15.64:0/2933104379 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fe95c0a09c0 0x7fe95c0a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:20.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.500+0000 7fe9667fc700 1 -- 172.21.15.64:0/2933104379 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fe9680cb9a0 con 0x7fe9701055b0 2023-09-13T03:28:20.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.500+0000 7fe98af70b80 1 -- 172.21.15.64:0/4200294291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628c37691a0 msgr2=0x5628c3769580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:28:20.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.500+0000 7fe98af70b80 1 --2- 172.21.15.64:0/4200294291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628c37691a0 0x5628c3769580 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe96000fb20 tx=0x7fe96001b040 comp rx=0 tx=0).stop 2023-09-13T03:28:20.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.500+0000 7fe98af70b80 1 -- 172.21.15.64:0/4200294291 shutdown_connections 2023-09-13T03:28:20.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.500+0000 7fe98af70b80 1 --2- 172.21.15.64:0/4200294291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628c37691a0 0x5628c3769580 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:20.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.500+0000 7fe98af70b80 1 --2- 172.21.15.64:0/4200294291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5628c3763cc0 0x5628c376ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:20.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.500+0000 7fe98af70b80 1 --2- 172.21.15.64:0/4200294291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5628c3763300 0x5628c3763780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:20.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.500+0000 7fe98af70b80 1 -- 172.21.15.64:0/4200294291 >> 172.21.15.64:0/4200294291 conn(0x5628c36d8030 msgr2=0x5628c375e560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:28:20.504 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.501+0000 7fe97539b700 1 --2- 172.21.15.64:0/2933104379 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fe95c0a09c0 0x7fe95c0a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:20.504 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.501+0000 7fe98af70b80 1 -- 172.21.15.64:0/4200294291 shutdown_connections 2023-09-13T03:28:20.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.501+0000 7fe98af70b80 1 -- 172.21.15.64:0/4200294291 wait complete. 2023-09-13T03:28:20.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.501+0000 7fe97539b700 1 --2- 172.21.15.64:0/2933104379 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fe95c0a09c0 0x7fe95c0a2e80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe96c004b10 tx=0x7fe96c006040 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:28:20.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.502+0000 7fe98af70b80 1 -- start start 2023-09-13T03:28:20.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.503+0000 7fe98af70b80 1 Processor -- start 2023-09-13T03:28:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.503+0000 7fe98af70b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5628c3763300 0x5628c3772e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.503+0000 7fe98af70b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5628c3763cc0 0x5628c37733d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.503+0000 7fe98af70b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628c37691a0 0x5628c376d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.503+0000 7fe98af70b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628c3653b10 con 0x5628c3763300 2023-09-13T03:28:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.503+0000 7fe98af70b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628c3653810 con 0x5628c37691a0 2023-09-13T03:28:20.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.503+0000 7fe98af70b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628c35378c0 con 0x5628c3763cc0 2023-09-13T03:28:20.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.503+0000 7fe98af70b80 10 client.0 _ll_register_callbacks cb 0x5628c359da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:28:20.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.504+0000 7fe975b9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628c37691a0 0x5628c376d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:20.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.504+0000 7fe975b9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628c37691a0 0x5628c376d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:58060/0 (socket says 172.21.15.64:58060) 2023-09-13T03:28:20.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.504+0000 7fe975b9c700 1 -- 172.21.15.64:0/3742349823 learned_addr learned my addr 172.21.15.64:0/3742349823 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:28:20.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.507+0000 7fe97539b700 1 --2- 172.21.15.64:0/3742349823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5628c3763300 0x5628c3772e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:20.509 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.507+0000 7fe97539b700 1 -- 172.21.15.64:0/3742349823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5628c3763cc0 msgr2=0x5628c37733d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-13T03:28:20.509 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.507+0000 7fe97539b700 1 --2- 172.21.15.64:0/3742349823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5628c3763cc0 0x5628c37733d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:20.509 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.507+0000 7fe97539b700 1 -- 172.21.15.64:0/3742349823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628c37691a0 msgr2=0x5628c376d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:28:20.509 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.507+0000 7fe97539b700 1 --2- 172.21.15.64:0/3742349823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628c37691a0 0x5628c376d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:20.509 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.507+0000 7fe97539b700 1 -- 172.21.15.64:0/3742349823 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe97010cbc0 con 0x5628c3763300 2023-09-13T03:28:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.510+0000 7fe97539b700 1 --2- 172.21.15.64:0/3742349823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5628c3763300 0x5628c3772e90 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe96c0060b0 tx=0x7fe96c00d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=883630ea9ab47a8c server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:28:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.510+0000 7fe958ff9700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-09-13T03:28:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.511+0000 7fe958ff9700 1 -- 172.21.15.64:0/3742349823 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c024070 con 0x5628c3763300 2023-09-13T03:28:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.511+0000 7fe98af70b80 1 -- 172.21.15.64:0/3742349823 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5628c360d490 con 0x5628c3763300 2023-09-13T03:28:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.511+0000 7fe958ff9700 1 -- 172.21.15.64:0/3742349823 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fe96c004bb0 con 0x5628c3763300 2023-09-13T03:28:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.511+0000 7fe958ff9700 1 -- 172.21.15.64:0/3742349823 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe96c010040 con 0x5628c3763300 2023-09-13T03:28:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.511+0000 7fe98af70b80 10 client.17253 fetch_fsmap learned FSMap version 75 2023-09-13T03:28:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.511+0000 7fe98af70b80 1 -- 172.21.15.64:0/3742349823 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=4+}) v3 -- 0x5628c34cac00 con 0x5628c3763300 2023-09-13T03:28:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.512+0000 7fe958ff9700 1 -- 172.21.15.64:0/3742349823 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5628c360d490 con 0x5628c3763300 2023-09-13T03:28:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.512+0000 7fe958ff9700 1 -- 172.21.15.64:0/3742349823 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 75) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fe96c027020 con 0x5628c3763300 2023-09-13T03:28:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.512+0000 7fe98af70b80 10 client.17253 fetch_fsmap finished waiting for FSMap version 75 2023-09-13T03:28:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.512+0000 7fe98af70b80 10 client.17253 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:28:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.512+0000 7fe98af70b80 20 client.17253 populate_metadata read hostname 'smithi064' 2023-09-13T03:28:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.512+0000 7fe98af70b80 1 --2- 172.21.15.64:0/3742349823 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5628c377d500 0x5628c377f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.512+0000 7fe98af70b80 4 client.17253 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:28:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.512+0000 7fe98af70b80 1 -- 172.21.15.64:0/3742349823 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5628c36ca840 con 0x5628c377d500 2023-09-13T03:28:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.513+0000 7fe974b9a700 1 --2- 172.21.15.64:0/3742349823 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5628c377d500 0x5628c377f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.514+0000 7fe974b9a700 1 --2- 172.21.15.64:0/3742349823 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5628c377d500 0x5628c377f920 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b39b20a5cc8f2768 server_cookie=526dd1d806218b96 in_seq=0 out_seq=0 2023-09-13T03:28:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.514+0000 7fe958ff9700 10 client.17253 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:28:20.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.515+0000 7fe958ff9700 1 -- 172.21.15.64:0/3742349823 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5628c36ca840 con 0x5628c377d500 2023-09-13T03:28:20.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.515+0000 7fe958ff9700 10 client.17253 handle_command_reply: tid=0 2023-09-13T03:28:20.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.515+0000 7fe98af70b80 1 client.17253 shutdown 2023-09-13T03:28:20.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.517+0000 7fe98af70b80 1 -- 172.21.15.64:0/3742349823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5628c3763300 msgr2=0x5628c3772e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:28:20.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.517+0000 7fe98af70b80 1 --2- 172.21.15.64:0/3742349823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5628c3763300 0x5628c3772e90 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe96c0060b0 tx=0x7fe96c00d040 comp rx=0 tx=0).stop 2023-09-13T03:28:20.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.517+0000 7fe98af70b80 1 -- 172.21.15.64:0/3742349823 shutdown_connections 2023-09-13T03:28:20.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.517+0000 7fe98af70b80 1 --2- 172.21.15.64:0/3742349823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628c37691a0 0x5628c376d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:20.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.517+0000 7fe98af70b80 1 --2- 172.21.15.64:0/3742349823 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5628c377d500 0x5628c377f920 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:20.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.517+0000 7fe98af70b80 1 --2- 172.21.15.64:0/3742349823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5628c3763300 0x5628c3772e90 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:20.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.517+0000 7fe98af70b80 1 --2- 172.21.15.64:0/3742349823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5628c3763cc0 0x5628c37733d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:20.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.517+0000 7fe98af70b80 1 -- 172.21.15.64:0/3742349823 >> 172.21.15.64:0/3742349823 conn(0x5628c36d8030 msgr2=0x5628c375d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:28:20.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.517+0000 7fe958ff9700 0 client.17253 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:28:20.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.518+0000 7fe98af70b80 1 -- 172.21.15.64:0/3742349823 shutdown_connections 2023-09-13T03:28:20.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.518+0000 7fe98af70b80 1 -- 172.21.15.64:0/3742349823 wait complete. 2023-09-13T03:28:20.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.519+0000 7fe98af70b80 20 client.17253 trim_cache size 0 max 16384 2023-09-13T03:28:20.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.542+0000 7fe98af70b80 1 Processor -- start 2023-09-13T03:28:20.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.543+0000 7fe98af70b80 1 -- start start 2023-09-13T03:28:20.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.543+0000 7fe98af70b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628c3763300 0x5628c376d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:20.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.543+0000 7fe98af70b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5628c3763cc0 0x5628c3762a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:20.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.543+0000 7fe98af70b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5628c37691a0 0x5628c375df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:20.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.543+0000 7fe98af70b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628c375afc0 con 0x5628c37691a0 2023-09-13T03:28:20.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.543+0000 7fe98af70b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628c376e6c0 con 0x5628c3763300 2023-09-13T03:28:20.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.543+0000 7fe98af70b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628c375eac0 con 0x5628c3763cc0 2023-09-13T03:28:20.545 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.543+0000 7fe97539b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628c3763300 0x5628c376d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:20.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.544+0000 7fe975b9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5628c37691a0 0x5628c375df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:20.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.544+0000 7fe97539b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628c3763300 0x5628c376d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:58066/0 (socket says 172.21.15.64:58066) 2023-09-13T03:28:20.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.544+0000 7fe97539b700 1 -- 172.21.15.64:0/945667489 learned_addr learned my addr 172.21.15.64:0/945667489 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:28:20.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.544+0000 7fe97539b700 1 -- 172.21.15.64:0/945667489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5628c3763cc0 msgr2=0x5628c3762a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:28:20.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.544+0000 7fe97539b700 1 --2- 172.21.15.64:0/945667489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5628c3763cc0 0x5628c3762a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:20.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.544+0000 7fe97539b700 1 -- 172.21.15.64:0/945667489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5628c37691a0 msgr2=0x5628c375df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:28:20.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.544+0000 7fe97539b700 1 --2- 172.21.15.64:0/945667489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5628c37691a0 0x5628c375df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:20.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.544+0000 7fe97539b700 1 -- 172.21.15.64:0/945667489 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628c35081f0 con 0x5628c3763300 2023-09-13T03:28:20.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.544+0000 7fe975b9c700 1 --2- 172.21.15.64:0/945667489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5628c37691a0 0x5628c375df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:28:20.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.544+0000 7fe97539b700 1 --2- 172.21.15.64:0/945667489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628c3763300 0x5628c376d210 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00aac0 tx=0x7fe96c020c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b680bfeaa37527a server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:28:20.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.544+0000 7fe95bfff700 1 -- 172.21.15.64:0/945667489 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c035070 con 0x5628c3763300 2023-09-13T03:28:20.548 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.545+0000 7fe95bfff700 1 -- 172.21.15.64:0/945667489 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe96c01c040 con 0x5628c3763300 2023-09-13T03:28:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.545+0000 7fe95bfff700 1 -- 172.21.15.64:0/945667489 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c024050 con 0x5628c3763300 2023-09-13T03:28:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.545+0000 7fe98af70b80 1 -- 172.21.15.64:0/945667489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628c3763300 msgr2=0x5628c376d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:28:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.545+0000 7fe98af70b80 1 --2- 172.21.15.64:0/945667489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628c3763300 0x5628c376d210 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00aac0 tx=0x7fe96c020c00 comp rx=0 tx=0).stop 2023-09-13T03:28:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.547+0000 7fe98af70b80 1 -- 172.21.15.64:0/945667489 shutdown_connections 2023-09-13T03:28:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.547+0000 7fe98af70b80 1 --2- 172.21.15.64:0/945667489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628c3763300 0x5628c376d210 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:20.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.547+0000 7fe98af70b80 1 --2- 172.21.15.64:0/945667489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5628c37691a0 0x5628c375df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:20.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.547+0000 7fe98af70b80 1 --2- 172.21.15.64:0/945667489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5628c3763cc0 0x5628c3762a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:20.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.547+0000 7fe98af70b80 1 -- 172.21.15.64:0/945667489 >> 172.21.15.64:0/945667489 conn(0x5628c36d8030 msgr2=0x5628c37622f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:28:20.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.547+0000 7fe98af70b80 1 -- 172.21.15.64:0/945667489 shutdown_connections 2023-09-13T03:28:20.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.547+0000 7fe98af70b80 1 -- 172.21.15.64:0/945667489 wait complete. 2023-09-13T03:28:20.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.548+0000 7fe98af70b80 1 -- start start 2023-09-13T03:28:20.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.548+0000 7fe98af70b80 1 Processor -- start 2023-09-13T03:28:20.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.548+0000 7fe98af70b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628c3763300 0x5628c37ca590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:20.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.548+0000 7fe98af70b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5628c3763cc0 0x5628c37caad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:20.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.548+0000 7fe98af70b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5628c37691a0 0x5628c37c9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:20.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.548+0000 7fe98af70b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628c37c9890 con 0x5628c3763cc0 2023-09-13T03:28:20.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.548+0000 7fe98af70b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628c37c9a00 con 0x5628c3763300 2023-09-13T03:28:20.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.548+0000 7fe98af70b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628c37c9bc0 con 0x5628c37691a0 2023-09-13T03:28:20.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.549+0000 7fe98af70b80 10 client.0 _ll_register_callbacks cb 0x7fe96c00aed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:28:20.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.549+0000 7fe97539b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628c3763300 0x5628c37ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:20.553 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.549+0000 7fe97539b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628c3763300 0x5628c37ca590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:58072/0 (socket says 172.21.15.64:58072) 2023-09-13T03:28:20.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.549+0000 7fe97539b700 1 -- 172.21.15.64:0/1964443191 learned_addr learned my addr 172.21.15.64:0/1964443191 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:28:20.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.549+0000 7fe97539b700 1 -- 172.21.15.64:0/1964443191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5628c37691a0 msgr2=0x5628c37c9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:28:20.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.549+0000 7fe97539b700 1 --2- 172.21.15.64:0/1964443191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5628c37691a0 0x5628c37c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:20.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.549+0000 7fe97539b700 1 -- 172.21.15.64:0/1964443191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5628c3763cc0 msgr2=0x5628c37caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:28:20.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.549+0000 7fe97539b700 1 --2- 172.21.15.64:0/1964443191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5628c3763cc0 0x5628c37caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:20.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.549+0000 7fe97539b700 1 -- 172.21.15.64:0/1964443191 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9701badc0 con 0x5628c3763300 2023-09-13T03:28:20.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.550+0000 7fe97539b700 1 --2- 172.21.15.64:0/1964443191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628c3763300 0x5628c37ca590 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe96c020a10 tx=0x7fe96c00b1b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37e2cf4137a1fc4 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:28:20.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.550+0000 7fe958ff9700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:28:20.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.550+0000 7fe958ff9700 1 -- 172.21.15.64:0/1964443191 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c024070 con 0x5628c3763300 2023-09-13T03:28:20.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.550+0000 7fe98af70b80 1 -- 172.21.15.64:0/1964443191 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe96001f040 con 0x5628c3763300 2023-09-13T03:28:20.555 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.550+0000 7fe958ff9700 1 -- 172.21.15.64:0/1964443191 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fe96c007740 con 0x5628c3763300 2023-09-13T03:28:20.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.551+0000 7fe958ff9700 1 -- 172.21.15.64:0/1964443191 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe96c035040 con 0x5628c3763300 2023-09-13T03:28:20.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.551+0000 7fe98af70b80 10 client.27335 fetch_fsmap learned FSMap version 75 2023-09-13T03:28:20.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.551+0000 7fe98af70b80 1 -- 172.21.15.64:0/1964443191 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=4+}) v3 -- 0x5628c34cac00 con 0x5628c3763300 2023-09-13T03:28:20.556 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.551+0000 7fe958ff9700 1 -- 172.21.15.64:0/1964443191 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe96c046020 con 0x5628c3763300 2023-09-13T03:28:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.551+0000 7fe958ff9700 1 -- 172.21.15.64:0/1964443191 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 75) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fe96c041070 con 0x5628c3763300 2023-09-13T03:28:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.551+0000 7fe98af70b80 10 client.27335 fetch_fsmap finished waiting for FSMap version 75 2023-09-13T03:28:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.551+0000 7fe98af70b80 10 client.27335 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:28:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.551+0000 7fe98af70b80 20 client.27335 populate_metadata read hostname 'smithi064' 2023-09-13T03:28:20.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.551+0000 7fe98af70b80 1 --2- 172.21.15.64:0/1964443191 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5628c37d22d0 0x5628c37d46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:20.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.551+0000 7fe98af70b80 4 client.27335 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:28:20.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.551+0000 7fe98af70b80 1 -- 172.21.15.64:0/1964443191 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5628c36ae8c0 con 0x5628c37d22d0 2023-09-13T03:28:20.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.551+0000 7fe974b9a700 1 --2- 172.21.15.64:0/1964443191 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5628c37d22d0 0x5628c37d46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:20.558 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.552+0000 7fe974b9a700 1 --2- 172.21.15.64:0/1964443191 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5628c37d22d0 0x5628c37d46f0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19acfaeb70676690 server_cookie=6933ff7c04c60200 in_seq=0 out_seq=0 2023-09-13T03:28:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.552+0000 7fe958ff9700 10 client.27335 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:28:20.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.776+0000 7fe958ff9700 1 -- 172.21.15.64:0/1964443191 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5628c36ae8c0 con 0x5628c37d22d0 2023-09-13T03:28:20.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.776+0000 7fe958ff9700 10 client.27335 handle_command_reply: tid=0 2023-09-13T03:28:20.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.776+0000 7fe98af70b80 1 client.27335 shutdown 2023-09-13T03:28:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.777+0000 7fe98af70b80 1 -- 172.21.15.64:0/1964443191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628c3763300 msgr2=0x5628c37ca590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:28:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.777+0000 7fe98af70b80 1 --2- 172.21.15.64:0/1964443191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628c3763300 0x5628c37ca590 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe96c020a10 tx=0x7fe96c00b1b0 comp rx=0 tx=0).stop 2023-09-13T03:28:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.777+0000 7fe98af70b80 1 -- 172.21.15.64:0/1964443191 shutdown_connections 2023-09-13T03:28:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.777+0000 7fe98af70b80 1 --2- 172.21.15.64:0/1964443191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5628c3763300 0x5628c37ca590 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.777+0000 7fe98af70b80 1 --2- 172.21.15.64:0/1964443191 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5628c37d22d0 0x5628c37d46f0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.777+0000 7fe98af70b80 1 --2- 172.21.15.64:0/1964443191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5628c37691a0 0x5628c37c9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.777+0000 7fe98af70b80 1 --2- 172.21.15.64:0/1964443191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5628c3763cc0 0x5628c37caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.777+0000 7fe98af70b80 1 -- 172.21.15.64:0/1964443191 >> 172.21.15.64:0/1964443191 conn(0x5628c36d8030 msgr2=0x5628c376d080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:28:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.777+0000 7fe958ff9700 0 client.27335 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:28:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.777+0000 7fe98af70b80 1 -- 172.21.15.64:0/1964443191 shutdown_connections 2023-09-13T03:28:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.777+0000 7fe98af70b80 1 -- 172.21.15.64:0/1964443191 wait complete. 2023-09-13T03:28:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.778+0000 7fe98af70b80 20 client.27335 trim_cache size 0 max 16384 2023-09-13T03:28:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:28:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:28:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:28:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:28:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.779+0000 7fe9775ff700 1 -- 172.21.15.64:0/2933104379 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fe95c0a09c0 msgr2=0x7fe95c0a2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:28:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.779+0000 7fe9775ff700 1 --2- 172.21.15.64:0/2933104379 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fe95c0a09c0 0x7fe95c0a2e80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe96c004b10 tx=0x7fe96c006040 comp rx=0 tx=0).stop 2023-09-13T03:28:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.779+0000 7fe9775ff700 1 -- 172.21.15.64:0/2933104379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9701055b0 msgr2=0x7fe970070430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:28:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.779+0000 7fe9775ff700 1 --2- 172.21.15.64:0/2933104379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9701055b0 0x7fe970070430 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe968009210 tx=0x7fe968009520 comp rx=0 tx=0).stop 2023-09-13T03:28:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.779+0000 7fe9775ff700 1 -- 172.21.15.64:0/2933104379 shutdown_connections 2023-09-13T03:28:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.779+0000 7fe9775ff700 1 --2- 172.21.15.64:0/2933104379 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fe95c0a09c0 0x7fe95c0a2e80 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.779+0000 7fe9775ff700 1 --2- 172.21.15.64:0/2933104379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9701055b0 0x7fe970070430 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.779+0000 7fe9775ff700 1 --2- 172.21.15.64:0/2933104379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9700ff760 0x7fe97006b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.779+0000 7fe9775ff700 1 --2- 172.21.15.64:0/2933104379 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe970100120 0x7fe97006bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.779+0000 7fe9775ff700 1 -- 172.21.15.64:0/2933104379 >> 172.21.15.64:0/2933104379 conn(0x7fe970074f30 msgr2=0x7fe9700fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:28:20.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.779+0000 7fe9775ff700 1 -- 172.21.15.64:0/2933104379 shutdown_connections 2023-09-13T03:28:20.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:20.780+0000 7fe9775ff700 1 -- 172.21.15.64:0/2933104379 wait complete. 2023-09-13T03:28:21.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:28:20 smithi170 ceph-mon[134328]: pgmap v476: 97 pgs: 97 active+clean; 1009 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 985 op/s 2023-09-13T03:28:21.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:28:20 smithi064 ceph-mon[97675]: pgmap v476: 97 pgs: 97 active+clean; 1009 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 985 op/s 2023-09-13T03:28:21.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:28:20 smithi142 ceph-mon[134082]: pgmap v476: 97 pgs: 97 active+clean; 1009 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 985 op/s 2023-09-13T03:28:21.301 INFO:tasks.workunit.client.0.smithi064.stdout: 10 121740 42.21 MB/sec execute 25 sec latency 120.996 ms 2023-09-13T03:28:22.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:28:21 smithi170 ceph-mon[134328]: pgmap v477: 97 pgs: 97 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 953 op/s 2023-09-13T03:28:22.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:28:21 smithi064 ceph-mon[97675]: pgmap v477: 97 pgs: 97 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 953 op/s 2023-09-13T03:28:22.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:28:21 smithi142 ceph-mon[134082]: pgmap v477: 97 pgs: 97 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 953 op/s 2023-09-13T03:28:22.301 INFO:tasks.workunit.client.0.smithi064.stdout: 10 122764 42.73 MB/sec execute 26 sec latency 41.435 ms 2023-09-13T03:28:23.301 INFO:tasks.workunit.client.0.smithi064.stdout: 10 123583 42.66 MB/sec execute 27 sec latency 88.611 ms 2023-09-13T03:28:24.302 INFO:tasks.workunit.client.0.smithi064.stdout: 10 124526 42.80 MB/sec execute 28 sec latency 54.843 ms 2023-09-13T03:28:25.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:28:24 smithi170 ceph-mon[134328]: pgmap v478: 97 pgs: 97 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 953 op/s 2023-09-13T03:28:25.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:28:24 smithi064 ceph-mon[97675]: pgmap v478: 97 pgs: 97 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 953 op/s 2023-09-13T03:28:25.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:28:24 smithi142 ceph-mon[134082]: pgmap v478: 97 pgs: 97 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 953 op/s 2023-09-13T03:28:25.301 INFO:tasks.workunit.client.0.smithi064.stdout: 10 125320 42.72 MB/sec execute 29 sec latency 224.321 ms 2023-09-13T03:28:26.302 INFO:tasks.workunit.client.0.smithi064.stdout: 10 126152 43.03 MB/sec execute 30 sec latency 180.618 ms 2023-09-13T03:28:27.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:28:26 smithi170 ceph-mon[134328]: pgmap v479: 97 pgs: 97 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 22 MiB/s wr, 1.29k op/s 2023-09-13T03:28:27.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:28:26 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:28:27.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:28:26 smithi064 ceph-mon[97675]: pgmap v479: 97 pgs: 97 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 22 MiB/s wr, 1.29k op/s 2023-09-13T03:28:27.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:28:26 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:28:27.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:28:26 smithi142 ceph-mon[134082]: pgmap v479: 97 pgs: 97 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 22 MiB/s wr, 1.29k op/s 2023-09-13T03:28:27.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:28:26 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:28:27.301 INFO:tasks.workunit.client.0.smithi064.stdout: 10 127048 43.12 MB/sec execute 31 sec latency 49.233 ms 2023-09-13T03:28:28.301 INFO:tasks.workunit.client.0.smithi064.stdout: 10 127873 43.35 MB/sec execute 32 sec latency 100.054 ms 2023-09-13T03:28:29.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:28:28 smithi170 ceph-mon[134328]: pgmap v480: 97 pgs: 97 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 836 op/s 2023-09-13T03:28:29.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:28:28 smithi064 ceph-mon[97675]: pgmap v480: 97 pgs: 97 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 836 op/s 2023-09-13T03:28:29.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:28:28 smithi142 ceph-mon[134082]: pgmap v480: 97 pgs: 97 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 836 op/s 2023-09-13T03:28:29.302 INFO:tasks.workunit.client.0.smithi064.stdout: 10 128800 43.27 MB/sec execute 33 sec latency 42.475 ms 2023-09-13T03:28:30.302 INFO:tasks.workunit.client.0.smithi064.stdout: 10 130035 43.98 MB/sec execute 34 sec latency 47.166 ms 2023-09-13T03:28:30.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:28:30 smithi170 ceph-mon[134328]: pgmap v481: 97 pgs: 97 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 836 op/s 2023-09-13T03:28:30.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:28:30 smithi064 ceph-mon[97675]: pgmap v481: 97 pgs: 97 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 836 op/s 2023-09-13T03:28:30.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:28:30 smithi142 ceph-mon[134082]: pgmap v481: 97 pgs: 97 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 836 op/s 2023-09-13T03:28:31.302 INFO:tasks.workunit.client.0.smithi064.stdout: 10 130822 44.11 MB/sec execute 35 sec latency 260.931 ms 2023-09-13T03:28:32.302 INFO:tasks.workunit.client.0.smithi064.stdout: 10 131479 43.82 MB/sec execute 36 sec latency 75.101 ms 2023-09-13T03:28:33.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:28:32 smithi170 ceph-mon[134328]: pgmap v482: 97 pgs: 97 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 19 MiB/s wr, 1.21k op/s 2023-09-13T03:28:33.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:28:32 smithi064 ceph-mon[97675]: pgmap v482: 97 pgs: 97 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 19 MiB/s wr, 1.21k op/s 2023-09-13T03:28:33.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:28:32 smithi142 ceph-mon[134082]: pgmap v482: 97 pgs: 97 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 19 MiB/s wr, 1.21k op/s 2023-09-13T03:28:33.302 INFO:tasks.workunit.client.0.smithi064.stdout: 10 132559 44.15 MB/sec execute 37 sec latency 33.120 ms 2023-09-13T03:28:34.005 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.004+0000 7fcd41bff700 1 -- 172.21.15.64:0/899000716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd3c10cf80 msgr2=0x7fcd3c10f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:28:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.004+0000 7fcd41bff700 1 --2- 172.21.15.64:0/899000716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd3c10cf80 0x7fcd3c10f370 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fcd3400b0a0 tx=0x7fcd3400b3b0 comp rx=0 tx=0).stop 2023-09-13T03:28:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.005+0000 7fcd41bff700 1 -- 172.21.15.64:0/899000716 shutdown_connections 2023-09-13T03:28:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.005+0000 7fcd41bff700 1 --2- 172.21.15.64:0/899000716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd3c10cf80 0x7fcd3c10f370 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.005+0000 7fcd41bff700 1 --2- 172.21.15.64:0/899000716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd3c0fea40 0x7fcd3c0fee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.005+0000 7fcd41bff700 1 --2- 172.21.15.64:0/899000716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd3c0ff360 0x7fcd3c10ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.005+0000 7fcd41bff700 1 -- 172.21.15.64:0/899000716 >> 172.21.15.64:0/899000716 conn(0x7fcd3c074370 msgr2=0x7fcd3c074780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:28:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.005+0000 7fcd41bff700 1 -- 172.21.15.64:0/899000716 shutdown_connections 2023-09-13T03:28:34.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.006+0000 7fcd41bff700 1 -- 172.21.15.64:0/899000716 wait complete. 2023-09-13T03:28:34.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.007+0000 7fcd41bff700 1 Processor -- start 2023-09-13T03:28:34.008 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.007+0000 7fcd41bff700 1 -- start start 2023-09-13T03:28:34.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.007+0000 7fcd41bff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd3c0fea40 0x7fcd3c070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:34.009 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.008+0000 7fcd41bff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd3c0ff360 0x7fcd3c071530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.008+0000 7fcd41bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd3c10cf80 0x7fcd3c071a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.008+0000 7fcd41bff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3c10ce00 con 0x7fcd3c0fea40 2023-09-13T03:28:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.008+0000 7fcd41bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3c1b1c10 con 0x7fcd3c10cf80 2023-09-13T03:28:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.008+0000 7fcd41bff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3c1b1d80 con 0x7fcd3c0ff360 2023-09-13T03:28:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.008+0000 7fcd3affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd3c0ff360 0x7fcd3c071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.008+0000 7fcd3bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd3c10cf80 0x7fcd3c071a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.008+0000 7fcd3affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd3c0ff360 0x7fcd3c071530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:57878/0 (socket says 172.21.15.64:57878) 2023-09-13T03:28:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.008+0000 7fcd3affd700 1 -- 172.21.15.64:0/189560654 learned_addr learned my addr 172.21.15.64:0/189560654 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:28:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.008+0000 7fcd3bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd3c10cf80 0x7fcd3c071a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:51156/0 (socket says 172.21.15.64:51156) 2023-09-13T03:28:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.009+0000 7fcd3affd700 1 -- 172.21.15.64:0/189560654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd3c10cf80 msgr2=0x7fcd3c071a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:28:34.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.009+0000 7fcd3affd700 1 --2- 172.21.15.64:0/189560654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd3c10cf80 0x7fcd3c071a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:34.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.009+0000 7fcd3affd700 1 -- 172.21.15.64:0/189560654 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd3c0fea40 msgr2=0x7fcd3c070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:28:34.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.009+0000 7fcd3affd700 1 --2- 172.21.15.64:0/189560654 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd3c0fea40 0x7fcd3c070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:34.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.009+0000 7fcd3affd700 1 -- 172.21.15.64:0/189560654 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd34009d00 con 0x7fcd3c0ff360 2023-09-13T03:28:34.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.009+0000 7fcd3bfff700 1 --2- 172.21.15.64:0/189560654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd3c10cf80 0x7fcd3c071a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:28:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.009+0000 7fcd3affd700 1 --2- 172.21.15.64:0/189560654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd3c0ff360 0x7fcd3c071530 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcd3000c9e0 tx=0x7fcd3000ccf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:28:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.009+0000 7fcd38ff9700 1 -- 172.21.15.64:0/189560654 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd30007ac0 con 0x7fcd3c0ff360 2023-09-13T03:28:34.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.009+0000 7fcd38ff9700 1 -- 172.21.15.64:0/189560654 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd30007c20 con 0x7fcd3c0ff360 2023-09-13T03:28:34.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.010+0000 7fcd41bff700 1 -- 172.21.15.64:0/189560654 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd3c1b2060 con 0x7fcd3c0ff360 2023-09-13T03:28:34.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.010+0000 7fcd38ff9700 1 -- 172.21.15.64:0/189560654 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd30018660 con 0x7fcd3c0ff360 2023-09-13T03:28:34.014 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.010+0000 7fcd41bff700 1 -- 172.21.15.64:0/189560654 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd3c1b25c0 con 0x7fcd3c0ff360 2023-09-13T03:28:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.016+0000 7fcd38ff9700 1 -- 172.21.15.64:0/189560654 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fcd3000f450 con 0x7fcd3c0ff360 2023-09-13T03:28:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.020+0000 7fcd38ff9700 1 --2- 172.21.15.64:0/189560654 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fcd240a0870 0x7fcd240a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.020+0000 7fcd38ff9700 1 -- 172.21.15.64:0/189560654 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fcd300cbe90 con 0x7fcd3c0ff360 2023-09-13T03:28:34.022 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.020+0000 7fcd3b7fe700 1 --2- 172.21.15.64:0/189560654 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fcd240a0870 0x7fcd240a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:34.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.022+0000 7fcd3b7fe700 1 --2- 172.21.15.64:0/189560654 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fcd240a0870 0x7fcd240a2d30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c005950 tx=0x7fcd2c0058e0 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:28:34.023 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.022+0000 7fcd55462b80 1 Processor -- start 2023-09-13T03:28:34.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.023+0000 7fcd55462b80 1 -- start start 2023-09-13T03:28:34.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.023+0000 7fcd55462b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5563117e71a0 0x5563117e7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:34.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.023+0000 7fcd55462b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5563117e1300 0x5563117e1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:34.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.023+0000 7fcd55462b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563117e1cc0 0x5563117edf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:34.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.023+0000 7fcd55462b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55631173a6d0 con 0x5563117e1300 2023-09-13T03:28:34.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.023+0000 7fcd55462b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556311737ff0 con 0x5563117e71a0 2023-09-13T03:28:34.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.023+0000 7fcd55462b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563117338e0 con 0x5563117e1cc0 2023-09-13T03:28:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.023+0000 7fcd3b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5563117e71a0 0x5563117e7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.023+0000 7fcd3b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5563117e71a0 0x5563117e7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:51162/0 (socket says 172.21.15.64:51162) 2023-09-13T03:28:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.023+0000 7fcd3b7fe700 1 -- 172.21.15.64:0/47025235 learned_addr learned my addr 172.21.15.64:0/47025235 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:28:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.024+0000 7fcd3b7fe700 1 -- 172.21.15.64:0/47025235 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563117e1cc0 msgr2=0x5563117edf30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-13T03:28:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.024+0000 7fcd3b7fe700 1 --2- 172.21.15.64:0/47025235 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563117e1cc0 0x5563117edf30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.024+0000 7fcd3b7fe700 1 -- 172.21.15.64:0/47025235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5563117e1300 msgr2=0x5563117e1780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-13T03:28:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.024+0000 7fcd3b7fe700 1 --2- 172.21.15.64:0/47025235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5563117e1300 0x5563117e1780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.024+0000 7fcd3b7fe700 1 -- 172.21.15.64:0/47025235 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55631155fd80 con 0x5563117e71a0 2023-09-13T03:28:34.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.024+0000 7fcd3b7fe700 1 --2- 172.21.15.64:0/47025235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5563117e71a0 0x5563117e7580 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c009270 tx=0x7fcd2c010000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9545129a7e7ed4d1 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:28:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.024+0000 7fcd21ffb700 1 -- 172.21.15.64:0/47025235 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c010ac0 con 0x5563117e71a0 2023-09-13T03:28:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.024+0000 7fcd21ffb700 1 -- 172.21.15.64:0/47025235 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd2c010c20 con 0x5563117e71a0 2023-09-13T03:28:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.024+0000 7fcd21ffb700 1 -- 172.21.15.64:0/47025235 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c00f620 con 0x5563117e71a0 2023-09-13T03:28:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.025+0000 7fcd55462b80 1 -- 172.21.15.64:0/47025235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5563117e71a0 msgr2=0x5563117e7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:28:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.025+0000 7fcd55462b80 1 --2- 172.21.15.64:0/47025235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5563117e71a0 0x5563117e7580 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c009270 tx=0x7fcd2c010000 comp rx=0 tx=0).stop 2023-09-13T03:28:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.025+0000 7fcd55462b80 1 -- 172.21.15.64:0/47025235 shutdown_connections 2023-09-13T03:28:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.025+0000 7fcd55462b80 1 --2- 172.21.15.64:0/47025235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5563117e71a0 0x5563117e7580 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.025+0000 7fcd55462b80 1 --2- 172.21.15.64:0/47025235 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563117e1cc0 0x5563117edf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.025+0000 7fcd55462b80 1 --2- 172.21.15.64:0/47025235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5563117e1300 0x5563117e1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.025+0000 7fcd55462b80 1 -- 172.21.15.64:0/47025235 >> 172.21.15.64:0/47025235 conn(0x556311756030 msgr2=0x5563117dc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:28:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.026+0000 7fcd55462b80 1 -- 172.21.15.64:0/47025235 shutdown_connections 2023-09-13T03:28:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.026+0000 7fcd55462b80 1 -- 172.21.15.64:0/47025235 wait complete. 2023-09-13T03:28:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.026+0000 7fcd55462b80 1 -- start start 2023-09-13T03:28:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.027+0000 7fcd55462b80 1 Processor -- start 2023-09-13T03:28:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.027+0000 7fcd55462b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563117e1cc0 0x5563117edf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.027+0000 7fcd55462b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5563117e71a0 0x5563117eb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.028+0000 7fcd55462b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5563117eb850 0x5563117f7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.028+0000 7fcd55462b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563116d1b10 con 0x5563117eb850 2023-09-13T03:28:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.028+0000 7fcd55462b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563116d1810 con 0x5563117e71a0 2023-09-13T03:28:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.028+0000 7fcd55462b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563115b58c0 con 0x5563117e1cc0 2023-09-13T03:28:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.028+0000 7fcd55462b80 10 client.0 _ll_register_callbacks cb 0x55631161ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:28:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.028+0000 7fcd3b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563117e1cc0 0x5563117edf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.028+0000 7fcd3b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563117e1cc0 0x5563117edf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:57894/0 (socket says 172.21.15.64:57894) 2023-09-13T03:28:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.028+0000 7fcd3b7fe700 1 -- 172.21.15.64:0/2861009806 learned_addr learned my addr 172.21.15.64:0/2861009806 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:28:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.028+0000 7fcd3bfff700 1 --2- 172.21.15.64:0/2861009806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5563117eb850 0x5563117f7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.028+0000 7fcd3affd700 1 --2- 172.21.15.64:0/2861009806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5563117e71a0 0x5563117eb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.028+0000 7fcd3b7fe700 1 -- 172.21.15.64:0/2861009806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5563117e71a0 msgr2=0x5563117eb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:28:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.029+0000 7fcd3b7fe700 1 --2- 172.21.15.64:0/2861009806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5563117e71a0 0x5563117eb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.029+0000 7fcd3b7fe700 1 -- 172.21.15.64:0/2861009806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5563117eb850 msgr2=0x5563117f7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:28:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.029+0000 7fcd3b7fe700 1 --2- 172.21.15.64:0/2861009806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5563117eb850 0x5563117f7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.029+0000 7fcd3b7fe700 1 -- 172.21.15.64:0/2861009806 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd2c00d960 con 0x5563117e1cc0 2023-09-13T03:28:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.029+0000 7fcd3affd700 1 --2- 172.21.15.64:0/2861009806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5563117e71a0 0x5563117eb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T03:28:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.029+0000 7fcd3b7fe700 1 --2- 172.21.15.64:0/2861009806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563117e1cc0 0x5563117edf30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c012040 tx=0x7fcd2c01db00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83431e2149d5b65 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:28:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.029+0000 7fcd1affd700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:28:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.030+0000 7fcd1affd700 1 -- 172.21.15.64:0/2861009806 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c01d640 con 0x5563117e1cc0 2023-09-13T03:28:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.030+0000 7fcd1affd700 1 -- 172.21.15.64:0/2861009806 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fcd2c02d070 con 0x5563117e1cc0 2023-09-13T03:28:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.032+0000 7fcd55462b80 1 -- 172.21.15.64:0/2861009806 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55631168b490 con 0x5563117e1cc0 2023-09-13T03:28:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.032+0000 7fcd1affd700 1 -- 172.21.15.64:0/2861009806 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd2c01d7a0 con 0x5563117e1cc0 2023-09-13T03:28:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.032+0000 7fcd55462b80 10 client.27304 fetch_fsmap learned FSMap version 76 2023-09-13T03:28:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.032+0000 7fcd55462b80 1 -- 172.21.15.64:0/2861009806 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x556311548c00 con 0x5563117e1cc0 2023-09-13T03:28:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.032+0000 7fcd1affd700 1 -- 172.21.15.64:0/2861009806 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55631168b490 con 0x5563117e1cc0 2023-09-13T03:28:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.033+0000 7fcd1affd700 1 -- 172.21.15.64:0/2861009806 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 76) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fcd2c03c420 con 0x5563117e1cc0 2023-09-13T03:28:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.033+0000 7fcd55462b80 10 client.27304 fetch_fsmap finished waiting for FSMap version 76 2023-09-13T03:28:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.033+0000 7fcd55462b80 10 client.27304 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:28:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.033+0000 7fcd55462b80 20 client.27304 populate_metadata read hostname 'smithi064' 2023-09-13T03:28:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.033+0000 7fcd55462b80 1 --2- 172.21.15.64:0/2861009806 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5563117fb7d0 0x5563117fdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.033+0000 7fcd55462b80 4 client.27304 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:28:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.033+0000 7fcd55462b80 1 -- 172.21.15.64:0/2861009806 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556311748840 con 0x5563117fb7d0 2023-09-13T03:28:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.034+0000 7fcd3affd700 1 --2- 172.21.15.64:0/2861009806 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5563117fb7d0 0x5563117fdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.034+0000 7fcd3affd700 1 --2- 172.21.15.64:0/2861009806 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5563117fb7d0 0x5563117fdbf0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75827890bb6acdb2 server_cookie=bfa8063be1c55320 in_seq=0 out_seq=0 2023-09-13T03:28:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.034+0000 7fcd1affd700 10 client.27304 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:28:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.035+0000 7fcd1affd700 1 -- 172.21.15.64:0/2861009806 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556311748840 con 0x5563117fb7d0 2023-09-13T03:28:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.035+0000 7fcd1affd700 10 client.27304 handle_command_reply: tid=0 2023-09-13T03:28:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.035+0000 7fcd55462b80 1 client.27304 shutdown 2023-09-13T03:28:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.036+0000 7fcd55462b80 1 -- 172.21.15.64:0/2861009806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563117e1cc0 msgr2=0x5563117edf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:28:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.036+0000 7fcd55462b80 1 --2- 172.21.15.64:0/2861009806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563117e1cc0 0x5563117edf30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c012040 tx=0x7fcd2c01db00 comp rx=0 tx=0).stop 2023-09-13T03:28:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.036+0000 7fcd55462b80 1 -- 172.21.15.64:0/2861009806 shutdown_connections 2023-09-13T03:28:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.036+0000 7fcd55462b80 1 --2- 172.21.15.64:0/2861009806 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x5563117fb7d0 0x5563117fdbf0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.036+0000 7fcd55462b80 1 --2- 172.21.15.64:0/2861009806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5563117eb850 0x5563117f7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.036+0000 7fcd55462b80 1 --2- 172.21.15.64:0/2861009806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563117e1cc0 0x5563117edf30 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.036+0000 7fcd55462b80 1 --2- 172.21.15.64:0/2861009806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5563117e71a0 0x5563117eb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.036+0000 7fcd55462b80 1 -- 172.21.15.64:0/2861009806 >> 172.21.15.64:0/2861009806 conn(0x556311756030 msgr2=0x5563117db7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:28:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.036+0000 7fcd1affd700 0 client.27304 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:28:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.036+0000 7fcd55462b80 1 -- 172.21.15.64:0/2861009806 shutdown_connections 2023-09-13T03:28:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.036+0000 7fcd55462b80 1 -- 172.21.15.64:0/2861009806 wait complete. 2023-09-13T03:28:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.037+0000 7fcd55462b80 20 client.27304 trim_cache size 0 max 16384 2023-09-13T03:28:34.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.065+0000 7fcd55462b80 1 Processor -- start 2023-09-13T03:28:34.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.066+0000 7fcd55462b80 1 -- start start 2023-09-13T03:28:34.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.066+0000 7fcd55462b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563117e1cc0 0x5563117f0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:34.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.066+0000 7fcd55462b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5563117e71a0 0x5563117dbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:34.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.066+0000 7fcd55462b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5563117eb850 0x5563117ea770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:34.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.066+0000 7fcd55462b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563117f8ed0 con 0x5563117eb850 2023-09-13T03:28:34.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.066+0000 7fcd55462b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563117d8fc0 con 0x5563117e71a0 2023-09-13T03:28:34.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.066+0000 7fcd55462b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563117ec790 con 0x5563117e1cc0 2023-09-13T03:28:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.067+0000 7fcd3b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563117e1cc0 0x5563117f0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.067+0000 7fcd3b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563117e1cc0 0x5563117f0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:57906/0 (socket says 172.21.15.64:57906) 2023-09-13T03:28:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.067+0000 7fcd3b7fe700 1 -- 172.21.15.64:0/1620424297 learned_addr learned my addr 172.21.15.64:0/1620424297 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:28:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.067+0000 7fcd3b7fe700 1 -- 172.21.15.64:0/1620424297 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5563117e71a0 msgr2=0x5563117dbf20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-13T03:28:34.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.067+0000 7fcd3b7fe700 1 --2- 172.21.15.64:0/1620424297 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5563117e71a0 0x5563117dbf20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:34.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.067+0000 7fcd3b7fe700 1 -- 172.21.15.64:0/1620424297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5563117eb850 msgr2=0x5563117ea770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:28:34.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.067+0000 7fcd3b7fe700 1 --2- 172.21.15.64:0/1620424297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5563117eb850 0x5563117ea770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:34.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.067+0000 7fcd3b7fe700 1 -- 172.21.15.64:0/1620424297 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563115861f0 con 0x5563117e1cc0 2023-09-13T03:28:34.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.068+0000 7fcd3b7fe700 1 --2- 172.21.15.64:0/1620424297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563117e1cc0 0x5563117f0af0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c004710 tx=0x7fcd2c004740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5b8b4bd0be4c3d1 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:28:34.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.068+0000 7fcd227fc700 1 -- 172.21.15.64:0/1620424297 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c03c620 con 0x5563117e1cc0 2023-09-13T03:28:34.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.068+0000 7fcd227fc700 1 -- 172.21.15.64:0/1620424297 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd2c02b070 con 0x5563117e1cc0 2023-09-13T03:28:34.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.068+0000 7fcd227fc700 1 -- 172.21.15.64:0/1620424297 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c032880 con 0x5563117e1cc0 2023-09-13T03:28:34.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.069+0000 7fcd55462b80 1 -- 172.21.15.64:0/1620424297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563117e1cc0 msgr2=0x5563117f0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:28:34.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.069+0000 7fcd55462b80 1 --2- 172.21.15.64:0/1620424297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563117e1cc0 0x5563117f0af0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c004710 tx=0x7fcd2c004740 comp rx=0 tx=0).stop 2023-09-13T03:28:34.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.070+0000 7fcd55462b80 1 -- 172.21.15.64:0/1620424297 shutdown_connections 2023-09-13T03:28:34.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.070+0000 7fcd55462b80 1 --2- 172.21.15.64:0/1620424297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5563117eb850 0x5563117ea770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:34.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.070+0000 7fcd55462b80 1 --2- 172.21.15.64:0/1620424297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563117e1cc0 0x5563117f0af0 secure :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c004710 tx=0x7fcd2c004740 comp rx=0 tx=0).stop 2023-09-13T03:28:34.073 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.070+0000 7fcd55462b80 1 --2- 172.21.15.64:0/1620424297 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5563117e71a0 0x5563117dbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:34.073 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.070+0000 7fcd55462b80 1 -- 172.21.15.64:0/1620424297 >> 172.21.15.64:0/1620424297 conn(0x556311756030 msgr2=0x5563117e5190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:28:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.076+0000 7fcd55462b80 1 -- 172.21.15.64:0/1620424297 shutdown_connections 2023-09-13T03:28:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.076+0000 7fcd55462b80 1 -- 172.21.15.64:0/1620424297 wait complete. 2023-09-13T03:28:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.076+0000 7fcd55462b80 1 -- start start 2023-09-13T03:28:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.077+0000 7fcd55462b80 1 Processor -- start 2023-09-13T03:28:34.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.077+0000 7fcd55462b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5563117e71a0 0x556311843550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:34.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.077+0000 7fcd55462b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563117eb850 0x556311843a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:34.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.077+0000 7fcd55462b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556311844070 0x55631184e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:34.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.077+0000 7fcd55462b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55631184ec90 con 0x556311844070 2023-09-13T03:28:34.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.077+0000 7fcd55462b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55631184ee00 con 0x5563117e71a0 2023-09-13T03:28:34.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.077+0000 7fcd55462b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55631184efc0 con 0x5563117eb850 2023-09-13T03:28:34.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.078+0000 7fcd55462b80 10 client.0 _ll_register_callbacks cb 0x7fcd2c00db80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:28:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.078+0000 7fcd3affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563117eb850 0x556311843a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.078+0000 7fcd3affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563117eb850 0x556311843a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:57908/0 (socket says 172.21.15.64:57908) 2023-09-13T03:28:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.078+0000 7fcd3affd700 1 -- 172.21.15.64:0/1633224132 learned_addr learned my addr 172.21.15.64:0/1633224132 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:28:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.078+0000 7fcd3b7fe700 1 --2- 172.21.15.64:0/1633224132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5563117e71a0 0x556311843550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.079+0000 7fcd3bfff700 1 --2- 172.21.15.64:0/1633224132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556311844070 0x55631184e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.079+0000 7fcd3b7fe700 1 -- 172.21.15.64:0/1633224132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563117eb850 msgr2=0x556311843a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:28:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.079+0000 7fcd3b7fe700 1 --2- 172.21.15.64:0/1633224132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563117eb850 0x556311843a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.079+0000 7fcd3b7fe700 1 -- 172.21.15.64:0/1633224132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556311844070 msgr2=0x55631184e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:28:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.079+0000 7fcd3b7fe700 1 --2- 172.21.15.64:0/1633224132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556311844070 0x55631184e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.079+0000 7fcd3b7fe700 1 -- 172.21.15.64:0/1633224132 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556311548c00 con 0x5563117e71a0 2023-09-13T03:28:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.080+0000 7fcd3affd700 1 --2- 172.21.15.64:0/1633224132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563117eb850 0x556311843a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:28:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.080+0000 7fcd3b7fe700 1 --2- 172.21.15.64:0/1633224132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5563117e71a0 0x556311843550 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c013000 tx=0x7fcd2c03cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a9769226a92d177 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:28:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.080+0000 7fcd1affd700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:28:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.080+0000 7fcd1affd700 1 -- 172.21.15.64:0/1633224132 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c01d850 con 0x5563117e71a0 2023-09-13T03:28:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.080+0000 7fcd55462b80 1 -- 172.21.15.64:0/1633224132 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5563117f78e0 con 0x5563117e71a0 2023-09-13T03:28:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.080+0000 7fcd1affd700 1 -- 172.21.15.64:0/1633224132 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fcd2c02d050 con 0x5563117e71a0 2023-09-13T03:28:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.080+0000 7fcd1affd700 1 -- 172.21.15.64:0/1633224132 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd2c010e40 con 0x5563117e71a0 2023-09-13T03:28:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.080+0000 7fcd55462b80 10 client.27347 fetch_fsmap learned FSMap version 76 2023-09-13T03:28:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.080+0000 7fcd55462b80 1 -- 172.21.15.64:0/1633224132 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x55631155e250 con 0x5563117e71a0 2023-09-13T03:28:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.081+0000 7fcd1affd700 1 -- 172.21.15.64:0/1633224132 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5563117f78e0 con 0x5563117e71a0 2023-09-13T03:28:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.081+0000 7fcd1affd700 1 -- 172.21.15.64:0/1633224132 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 76) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fcd2c043030 con 0x5563117e71a0 2023-09-13T03:28:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.082+0000 7fcd55462b80 10 client.27347 fetch_fsmap finished waiting for FSMap version 76 2023-09-13T03:28:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.082+0000 7fcd55462b80 10 client.27347 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:28:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.082+0000 7fcd55462b80 20 client.27347 populate_metadata read hostname 'smithi064' 2023-09-13T03:28:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.082+0000 7fcd55462b80 1 --2- 172.21.15.64:0/1633224132 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x556311850200 0x5563118525f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.082+0000 7fcd55462b80 4 client.27347 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:28:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.082+0000 7fcd55462b80 1 -- 172.21.15.64:0/1633224132 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7fcd2c010c20 con 0x556311850200 2023-09-13T03:28:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.083+0000 7fcd3affd700 1 --2- 172.21.15.64:0/1633224132 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x556311850200 0x5563118525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.086+0000 7fcd3affd700 1 --2- 172.21.15.64:0/1633224132 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x556311850200 0x5563118525f0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ed648ac9121ea4b server_cookie=623155f5a141266e in_seq=0 out_seq=0 2023-09-13T03:28:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.086+0000 7fcd1affd700 10 client.27347 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:28:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.270+0000 7fcd1affd700 1 -- 172.21.15.64:0/1633224132 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7fcd2c010c20 con 0x556311850200 2023-09-13T03:28:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.270+0000 7fcd1affd700 10 client.27347 handle_command_reply: tid=0 2023-09-13T03:28:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.270+0000 7fcd55462b80 1 client.27347 shutdown 2023-09-13T03:28:34.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.270+0000 7fcd55462b80 1 -- 172.21.15.64:0/1633224132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5563117e71a0 msgr2=0x556311843550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:28:34.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.271+0000 7fcd55462b80 1 --2- 172.21.15.64:0/1633224132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5563117e71a0 0x556311843550 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c013000 tx=0x7fcd2c03cc40 comp rx=0 tx=0).stop 2023-09-13T03:28:34.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.271+0000 7fcd55462b80 1 -- 172.21.15.64:0/1633224132 shutdown_connections 2023-09-13T03:28:34.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.271+0000 7fcd55462b80 1 --2- 172.21.15.64:0/1633224132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5563117e71a0 0x556311843550 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.271+0000 7fcd55462b80 1 --2- 172.21.15.64:0/1633224132 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x556311850200 0x5563118525f0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.271+0000 7fcd55462b80 1 --2- 172.21.15.64:0/1633224132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556311844070 0x55631184e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.271+0000 7fcd55462b80 1 --2- 172.21.15.64:0/1633224132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563117eb850 0x556311843a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.271+0000 7fcd55462b80 1 -- 172.21.15.64:0/1633224132 >> 172.21.15.64:0/1633224132 conn(0x556311756030 msgr2=0x5563117e02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:28:34.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.272+0000 7fcd1affd700 0 client.27347 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:28:34.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.272+0000 7fcd55462b80 1 -- 172.21.15.64:0/1633224132 shutdown_connections 2023-09-13T03:28:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.272+0000 7fcd55462b80 1 -- 172.21.15.64:0/1633224132 wait complete. 2023-09-13T03:28:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.273+0000 7fcd55462b80 20 client.27347 trim_cache size 0 max 16384 2023-09-13T03:28:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:28:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:28:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:28:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:28:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.274+0000 7fcd41bff700 1 -- 172.21.15.64:0/189560654 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fcd240a0870 msgr2=0x7fcd240a2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:28:34.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.274+0000 7fcd41bff700 1 --2- 172.21.15.64:0/189560654 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fcd240a0870 0x7fcd240a2d30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c005950 tx=0x7fcd2c0058e0 comp rx=0 tx=0).stop 2023-09-13T03:28:34.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.274+0000 7fcd41bff700 1 -- 172.21.15.64:0/189560654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd3c0ff360 msgr2=0x7fcd3c071530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:28:34.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.274+0000 7fcd41bff700 1 --2- 172.21.15.64:0/189560654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd3c0ff360 0x7fcd3c071530 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcd3000c9e0 tx=0x7fcd3000ccf0 comp rx=0 tx=0).stop 2023-09-13T03:28:34.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.275+0000 7fcd41bff700 1 -- 172.21.15.64:0/189560654 shutdown_connections 2023-09-13T03:28:34.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.275+0000 7fcd41bff700 1 --2- 172.21.15.64:0/189560654 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fcd240a0870 0x7fcd240a2d30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.275+0000 7fcd41bff700 1 --2- 172.21.15.64:0/189560654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd3c10cf80 0x7fcd3c071a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.275+0000 7fcd41bff700 1 --2- 172.21.15.64:0/189560654 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd3c0fea40 0x7fcd3c070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.275+0000 7fcd41bff700 1 --2- 172.21.15.64:0/189560654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd3c0ff360 0x7fcd3c071530 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.275+0000 7fcd41bff700 1 -- 172.21.15.64:0/189560654 >> 172.21.15.64:0/189560654 conn(0x7fcd3c074370 msgr2=0x7fcd3c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:28:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.277+0000 7fcd41bff700 1 -- 172.21.15.64:0/189560654 shutdown_connections 2023-09-13T03:28:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:34.277+0000 7fcd41bff700 1 -- 172.21.15.64:0/189560654 wait complete. 2023-09-13T03:28:34.302 INFO:tasks.workunit.client.0.smithi064.stdout: 10 133506 44.25 MB/sec execute 38 sec latency 120.833 ms 2023-09-13T03:28:35.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:28:34 smithi170 ceph-mon[134328]: pgmap v483: 97 pgs: 97 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 795 op/s 2023-09-13T03:28:35.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:28:34 smithi064 ceph-mon[97675]: pgmap v483: 97 pgs: 97 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 795 op/s 2023-09-13T03:28:35.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:28:34 smithi142 ceph-mon[134082]: pgmap v483: 97 pgs: 97 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 795 op/s 2023-09-13T03:28:35.303 INFO:tasks.workunit.client.0.smithi064.stdout: 10 134570 44.80 MB/sec execute 39 sec latency 39.974 ms 2023-09-13T03:28:36.303 INFO:tasks.workunit.client.0.smithi064.stdout: 10 135169 44.49 MB/sec execute 40 sec latency 200.015 ms 2023-09-13T03:28:37.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:28:36 smithi170 ceph-mon[134328]: pgmap v484: 97 pgs: 97 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 1.04k op/s 2023-09-13T03:28:37.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:28:36 smithi064 ceph-mon[97675]: pgmap v484: 97 pgs: 97 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 1.04k op/s 2023-09-13T03:28:37.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:28:36 smithi142 ceph-mon[134082]: pgmap v484: 97 pgs: 97 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 1.04k op/s 2023-09-13T03:28:37.303 INFO:tasks.workunit.client.0.smithi064.stdout: 10 135860 44.14 MB/sec execute 41 sec latency 144.610 ms 2023-09-13T03:28:38.303 INFO:tasks.workunit.client.0.smithi064.stdout: 10 137133 44.69 MB/sec execute 42 sec latency 46.637 ms 2023-09-13T03:28:39.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:28:38 smithi170 ceph-mon[134328]: pgmap v485: 97 pgs: 97 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 813 op/s 2023-09-13T03:28:39.171 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:28:38 smithi064 ceph-mon[97675]: pgmap v485: 97 pgs: 97 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 813 op/s 2023-09-13T03:28:39.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:28:38 smithi142 ceph-mon[134082]: pgmap v485: 97 pgs: 97 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 813 op/s 2023-09-13T03:28:39.303 INFO:tasks.workunit.client.0.smithi064.stdout: 10 138189 45.37 MB/sec execute 43 sec latency 37.688 ms 2023-09-13T03:28:40.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:28:39 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:28:40.187 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:28:39 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:28:40.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:28:39 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:28:40.303 INFO:tasks.workunit.client.0.smithi064.stdout: 10 138768 44.81 MB/sec execute 44 sec latency 58.784 ms 2023-09-13T03:28:41.170 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:28:40 smithi064 ceph-mon[97675]: pgmap v486: 97 pgs: 97 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 813 op/s 2023-09-13T03:28:41.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:28:40 smithi142 ceph-mon[134082]: pgmap v486: 97 pgs: 97 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 813 op/s 2023-09-13T03:28:41.303 INFO:tasks.workunit.client.0.smithi064.stdout: 10 139367 44.36 MB/sec execute 45 sec latency 260.051 ms 2023-09-13T03:28:41.391 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:28:40 smithi170 ceph-mon[134328]: pgmap v486: 97 pgs: 97 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 813 op/s 2023-09-13T03:28:42.304 INFO:tasks.workunit.client.0.smithi064.stdout: 10 140207 44.36 MB/sec execute 46 sec latency 90.932 ms 2023-09-13T03:28:42.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:28:42 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:28:42.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:28:42 smithi170 ceph-mon[134328]: pgmap v487: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 16 MiB/s wr, 1.17k op/s 2023-09-13T03:28:42.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:28:42 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:28:42.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:28:42 smithi064 ceph-mon[97675]: pgmap v487: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 16 MiB/s wr, 1.17k op/s 2023-09-13T03:28:42.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:28:42 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:28:42.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:28:42 smithi142 ceph-mon[134082]: pgmap v487: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 16 MiB/s wr, 1.17k op/s 2023-09-13T03:28:43.305 INFO:tasks.workunit.client.0.smithi064.stdout: 10 141313 44.76 MB/sec execute 47 sec latency 45.267 ms 2023-09-13T03:28:43.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:28:43 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:28:43.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:28:43 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:28:43.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:28:43 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:28:43.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:28:43 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:28:43.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:28:43 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:28:43.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:28:43 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:28:43.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:28:43 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:28:43.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:28:43 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:28:43.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:28:43 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:28:44.305 INFO:tasks.workunit.client.0.smithi064.stdout: 10 142131 44.89 MB/sec execute 48 sec latency 43.117 ms 2023-09-13T03:28:44.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:28:44 smithi170 ceph-mon[134328]: pgmap v488: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 797 op/s 2023-09-13T03:28:44.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:28:44 smithi064 ceph-mon[97675]: pgmap v488: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 797 op/s 2023-09-13T03:28:44.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:28:44 smithi142 ceph-mon[134082]: pgmap v488: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 797 op/s 2023-09-13T03:28:45.305 INFO:tasks.workunit.client.0.smithi064.stdout: 10 142844 44.65 MB/sec execute 49 sec latency 93.739 ms 2023-09-13T03:28:46.305 INFO:tasks.workunit.client.0.smithi064.stdout: 10 143491 44.38 MB/sec execute 50 sec latency 190.663 ms 2023-09-13T03:28:47.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:28:46 smithi170 ceph-mon[134328]: pgmap v489: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 956 op/s 2023-09-13T03:28:47.170 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:28:46 smithi064 ceph-mon[97675]: pgmap v489: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 956 op/s 2023-09-13T03:28:47.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:28:46 smithi142 ceph-mon[134082]: pgmap v489: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 956 op/s 2023-09-13T03:28:47.305 INFO:tasks.workunit.client.0.smithi064.stdout: 10 144386 44.52 MB/sec execute 51 sec latency 76.228 ms 2023-09-13T03:28:47.479 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.478+0000 7fdaa4fff700 1 -- 172.21.15.64:0/4137940274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdaa00729d0 msgr2=0x7fdaa0072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:28:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.478+0000 7fdaa4fff700 1 --2- 172.21.15.64:0/4137940274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdaa00729d0 0x7fdaa0072db0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fda90009990 tx=0x7fda90009ca0 comp rx=0 tx=0).stop 2023-09-13T03:28:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.478+0000 7fdaa4fff700 1 -- 172.21.15.64:0/4137940274 shutdown_connections 2023-09-13T03:28:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.478+0000 7fdaa4fff700 1 --2- 172.21.15.64:0/4137940274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdaa00729d0 0x7fdaa0072db0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.478+0000 7fdaa4fff700 1 --2- 172.21.15.64:0/4137940274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdaa010ce40 0x7fdaa010f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.478+0000 7fdaa4fff700 1 --2- 172.21.15.64:0/4137940274 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdaa00732f0 0x7fdaa010c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.478+0000 7fdaa4fff700 1 -- 172.21.15.64:0/4137940274 >> 172.21.15.64:0/4137940274 conn(0x7fdaa00fc0b0 msgr2=0x7fdaa00fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:28:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.479+0000 7fdaa4fff700 1 -- 172.21.15.64:0/4137940274 shutdown_connections 2023-09-13T03:28:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.479+0000 7fdaa4fff700 1 -- 172.21.15.64:0/4137940274 wait complete. 2023-09-13T03:28:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.480+0000 7fdaa4fff700 1 Processor -- start 2023-09-13T03:28:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.480+0000 7fdaa4fff700 1 -- start start 2023-09-13T03:28:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.480+0000 7fdaa4fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdaa00729d0 0x7fdaa0107fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.480+0000 7fdaa4fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdaa00732f0 0x7fdaa0108520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.480+0000 7fdaa4fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdaa010ce40 0x7fdaa0102e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.481+0000 7fdaa4fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa0103640 con 0x7fdaa00729d0 2023-09-13T03:28:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.481+0000 7fdaa4fff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa01037b0 con 0x7fdaa00732f0 2023-09-13T03:28:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.481+0000 7fdaa4fff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa0103970 con 0x7fdaa010ce40 2023-09-13T03:28:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.481+0000 7fda9ed9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdaa010ce40 0x7fdaa0102e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.481+0000 7fda9dd9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdaa00732f0 0x7fdaa0108520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:47.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.481+0000 7fda9e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdaa00729d0 0x7fdaa0107fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:47.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.481+0000 7fda9ed9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdaa010ce40 0x7fdaa0102e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:38590/0 (socket says 172.21.15.64:38590) 2023-09-13T03:28:47.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.481+0000 7fda9ed9d700 1 -- 172.21.15.64:0/2012520295 learned_addr learned my addr 172.21.15.64:0/2012520295 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:28:47.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.481+0000 7fda9dd9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdaa00732f0 0x7fdaa0108520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:53358/0 (socket says 172.21.15.64:53358) 2023-09-13T03:28:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.481+0000 7fda9ed9d700 1 -- 172.21.15.64:0/2012520295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdaa00732f0 msgr2=0x7fdaa0108520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:28:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.481+0000 7fda9ed9d700 1 --2- 172.21.15.64:0/2012520295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdaa00732f0 0x7fdaa0108520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.481+0000 7fda9ed9d700 1 -- 172.21.15.64:0/2012520295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdaa00729d0 msgr2=0x7fdaa0107fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:28:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.481+0000 7fda9ed9d700 1 --2- 172.21.15.64:0/2012520295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdaa00729d0 0x7fdaa0107fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.481+0000 7fda9ed9d700 1 -- 172.21.15.64:0/2012520295 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda900095f0 con 0x7fdaa010ce40 2023-09-13T03:28:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.482+0000 7fda9ed9d700 1 --2- 172.21.15.64:0/2012520295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdaa010ce40 0x7fdaa0102e70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fda98007ec0 tx=0x7fda9800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:28:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.484+0000 7fda8f7fe700 1 -- 172.21.15.64:0/2012520295 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda98017070 con 0x7fdaa010ce40 2023-09-13T03:28:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.484+0000 7fdaa4fff700 1 -- 172.21.15.64:0/2012520295 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaa0103c50 con 0x7fdaa010ce40 2023-09-13T03:28:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.484+0000 7fda8f7fe700 1 -- 172.21.15.64:0/2012520295 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda9800f040 con 0x7fdaa010ce40 2023-09-13T03:28:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.484+0000 7fda8f7fe700 1 -- 172.21.15.64:0/2012520295 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda98004280 con 0x7fdaa010ce40 2023-09-13T03:28:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.484+0000 7fdaa4fff700 1 -- 172.21.15.64:0/2012520295 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaa010c050 con 0x7fdaa010ce40 2023-09-13T03:28:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.488+0000 7fda8f7fe700 1 -- 172.21.15.64:0/2012520295 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fda9800db20 con 0x7fdaa010ce40 2023-09-13T03:28:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.489+0000 7fdab89a4b80 1 Processor -- start 2023-09-13T03:28:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.489+0000 7fdab89a4b80 1 -- start start 2023-09-13T03:28:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.490+0000 7fdab89a4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ee9fd81a0 0x555ee9fd8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.490+0000 7fdab89a4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ee9fd2300 0x555ee9fd2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.490+0000 7fdab89a4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ee9fd2cc0 0x555ee9fdef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.490+0000 7fdab89a4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ee9f2b6d0 con 0x555ee9fd2300 2023-09-13T03:28:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.490+0000 7fdab89a4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ee9f28ff0 con 0x555ee9fd2cc0 2023-09-13T03:28:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.490+0000 7fdab89a4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ee9f248e0 con 0x555ee9fd81a0 2023-09-13T03:28:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.490+0000 7fda9dd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ee9fd81a0 0x555ee9fd8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.490+0000 7fda9dd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ee9fd81a0 0x555ee9fd8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:38606/0 (socket says 172.21.15.64:38606) 2023-09-13T03:28:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.490+0000 7fda9dd9b700 1 -- 172.21.15.64:0/3015940266 learned_addr learned my addr 172.21.15.64:0/3015940266 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:28:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.490+0000 7fda9dd9b700 1 --2- 172.21.15.64:0/3015940266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ee9fd2cc0 0x555ee9fdef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.491+0000 7fda9e59c700 1 --2- 172.21.15.64:0/3015940266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ee9fd2300 0x555ee9fd2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.491+0000 7fda9dd9b700 1 -- 172.21.15.64:0/3015940266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ee9fd81a0 msgr2=0x555ee9fd8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:28:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.491+0000 7fda9dd9b700 1 --2- 172.21.15.64:0/3015940266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ee9fd81a0 0x555ee9fd8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.491+0000 7fda9dd9b700 1 -- 172.21.15.64:0/3015940266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ee9fd2300 msgr2=0x555ee9fd2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:28:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.491+0000 7fda9dd9b700 1 --2- 172.21.15.64:0/3015940266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ee9fd2300 0x555ee9fd2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.491+0000 7fda9dd9b700 1 -- 172.21.15.64:0/3015940266 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ee9d50d80 con 0x555ee9fd2cc0 2023-09-13T03:28:47.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.491+0000 7fda9e59c700 1 --2- 172.21.15.64:0/3015940266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ee9fd2300 0x555ee9fd2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T03:28:47.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.491+0000 7fda8f7fe700 1 --2- 172.21.15.64:0/2012520295 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fda880a07b0 0x7fda880a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:47.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.491+0000 7fda9dd9b700 1 --2- 172.21.15.64:0/3015940266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ee9fd2cc0 0x555ee9fdef30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fda940095f0 tx=0x7fda94009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa3207d0ec7a0830 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:28:47.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.491+0000 7fda8f7fe700 1 -- 172.21.15.64:0/2012520295 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fda980cb4d0 con 0x7fdaa010ce40 2023-09-13T03:28:47.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.491+0000 7fda9e59c700 1 --2- 172.21.15.64:0/2012520295 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fda880a07b0 0x7fda880a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:47.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.491+0000 7fda8cff9700 1 -- 172.21.15.64:0/3015940266 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda94011ce0 con 0x555ee9fd2cc0 2023-09-13T03:28:47.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.492+0000 7fda8cff9700 1 -- 172.21.15.64:0/3015940266 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda94011e40 con 0x555ee9fd2cc0 2023-09-13T03:28:47.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.492+0000 7fda8cff9700 1 -- 172.21.15.64:0/3015940266 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda94005280 con 0x555ee9fd2cc0 2023-09-13T03:28:47.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.493+0000 7fdab89a4b80 1 -- 172.21.15.64:0/3015940266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ee9fd2cc0 msgr2=0x555ee9fdef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:28:47.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.493+0000 7fdab89a4b80 1 --2- 172.21.15.64:0/3015940266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ee9fd2cc0 0x555ee9fdef30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fda940095f0 tx=0x7fda94009260 comp rx=0 tx=0).stop 2023-09-13T03:28:47.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.493+0000 7fda9e59c700 1 --2- 172.21.15.64:0/2012520295 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fda880a07b0 0x7fda880a2c70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x555ee9fcd060 tx=0x7fda90005fd0 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:28:47.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.493+0000 7fdab89a4b80 1 -- 172.21.15.64:0/3015940266 shutdown_connections 2023-09-13T03:28:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.493+0000 7fdab89a4b80 1 --2- 172.21.15.64:0/3015940266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ee9fd2cc0 0x555ee9fdef30 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.493+0000 7fdab89a4b80 1 --2- 172.21.15.64:0/3015940266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ee9fd81a0 0x555ee9fd8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.493+0000 7fdab89a4b80 1 --2- 172.21.15.64:0/3015940266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ee9fd2300 0x555ee9fd2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.493+0000 7fdab89a4b80 1 -- 172.21.15.64:0/3015940266 >> 172.21.15.64:0/3015940266 conn(0x555ee9f47030 msgr2=0x555ee9fcd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:28:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.493+0000 7fdab89a4b80 1 -- 172.21.15.64:0/3015940266 shutdown_connections 2023-09-13T03:28:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.493+0000 7fdab89a4b80 1 -- 172.21.15.64:0/3015940266 wait complete. 2023-09-13T03:28:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.493+0000 7fdab89a4b80 1 -- start start 2023-09-13T03:28:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.494+0000 7fdab89a4b80 1 Processor -- start 2023-09-13T03:28:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.495+0000 7fdab89a4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ee9fd2300 0x555ee9fdef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.495+0000 7fdab89a4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ee9fd2cc0 0x555ee9fdc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:47.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.495+0000 7fdab89a4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ee9fd81a0 0x555ee9fdc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:47.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.495+0000 7fdab89a4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ee9ec2b10 con 0x555ee9fd81a0 2023-09-13T03:28:47.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.495+0000 7fdab89a4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ee9ec2810 con 0x555ee9fd2300 2023-09-13T03:28:47.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.495+0000 7fdab89a4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ee9da68c0 con 0x555ee9fd2cc0 2023-09-13T03:28:47.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.495+0000 7fdab89a4b80 10 client.0 _ll_register_callbacks cb 0x555ee9e0ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:28:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.495+0000 7fda9e59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ee9fd2300 0x555ee9fdef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.495+0000 7fda9e59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ee9fd2300 0x555ee9fdef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:53366/0 (socket says 172.21.15.64:53366) 2023-09-13T03:28:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.495+0000 7fda9e59c700 1 -- 172.21.15.64:0/2561232868 learned_addr learned my addr 172.21.15.64:0/2561232868 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:28:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.495+0000 7fda9ed9d700 1 --2- 172.21.15.64:0/2561232868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ee9fd81a0 0x555ee9fdc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.495+0000 7fda9e59c700 1 -- 172.21.15.64:0/2561232868 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ee9fd2cc0 msgr2=0x555ee9fdc210 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-13T03:28:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.496+0000 7fda9dd9b700 1 --2- 172.21.15.64:0/2561232868 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ee9fd2cc0 0x555ee9fdc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.496+0000 7fda9e59c700 1 --2- 172.21.15.64:0/2561232868 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ee9fd2cc0 0x555ee9fdc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.496+0000 7fda9e59c700 1 -- 172.21.15.64:0/2561232868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ee9fd81a0 msgr2=0x555ee9fdc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:28:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.496+0000 7fda9e59c700 1 --2- 172.21.15.64:0/2561232868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ee9fd81a0 0x555ee9fdc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.496+0000 7fda9e59c700 1 -- 172.21.15.64:0/2561232868 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda94019040 con 0x555ee9fd2300 2023-09-13T03:28:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.496+0000 7fda9ed9d700 1 --2- 172.21.15.64:0/2561232868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ee9fd81a0 0x555ee9fdc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-13T03:28:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.496+0000 7fda9e59c700 1 --2- 172.21.15.64:0/2561232868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ee9fd2300 0x555ee9fdef20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fda90015000 tx=0x7fda90009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9586dd993c63ffcb server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:28:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.496+0000 7fda85ffb700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:28:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.497+0000 7fda85ffb700 1 -- 172.21.15.64:0/2561232868 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9002b070 con 0x555ee9fd2300 2023-09-13T03:28:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.497+0000 7fda85ffb700 1 -- 172.21.15.64:0/2561232868 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fda90004aa0 con 0x555ee9fd2300 2023-09-13T03:28:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.497+0000 7fdab89a4b80 1 -- 172.21.15.64:0/2561232868 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555ee9e7c490 con 0x555ee9fd2300 2023-09-13T03:28:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.497+0000 7fda85ffb700 1 -- 172.21.15.64:0/2561232868 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda90005700 con 0x555ee9fd2300 2023-09-13T03:28:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.497+0000 7fdab89a4b80 10 client.27359 fetch_fsmap learned FSMap version 77 2023-09-13T03:28:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.497+0000 7fdab89a4b80 1 -- 172.21.15.64:0/2561232868 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) v3 -- 0x555ee9d39c00 con 0x555ee9fd2300 2023-09-13T03:28:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.498+0000 7fda85ffb700 1 -- 172.21.15.64:0/2561232868 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555ee9e7c490 con 0x555ee9fd2300 2023-09-13T03:28:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.498+0000 7fda85ffb700 1 -- 172.21.15.64:0/2561232868 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 77) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fda90037070 con 0x555ee9fd2300 2023-09-13T03:28:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.498+0000 7fdab89a4b80 10 client.27359 fetch_fsmap finished waiting for FSMap version 77 2023-09-13T03:28:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.498+0000 7fdab89a4b80 10 client.27359 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:28:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.498+0000 7fdab89a4b80 20 client.27359 populate_metadata read hostname 'smithi064' 2023-09-13T03:28:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.498+0000 7fdab89a4b80 1 --2- 172.21.15.64:0/2561232868 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x555ee9fec4c0 0x555ee9fee8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.498+0000 7fdab89a4b80 4 client.27359 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:28:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.498+0000 7fdab89a4b80 1 -- 172.21.15.64:0/2561232868 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555ee9f39840 con 0x555ee9fec4c0 2023-09-13T03:28:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.498+0000 7fda9dd9b700 1 --2- 172.21.15.64:0/2561232868 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x555ee9fec4c0 0x555ee9fee8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.499+0000 7fda9dd9b700 1 --2- 172.21.15.64:0/2561232868 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x555ee9fec4c0 0x555ee9fee8e0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4854d7da5330942 server_cookie=200e66d322c8562c in_seq=0 out_seq=0 2023-09-13T03:28:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.499+0000 7fda85ffb700 10 client.27359 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:28:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.500+0000 7fda85ffb700 1 -- 172.21.15.64:0/2561232868 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555ee9f39840 con 0x555ee9fec4c0 2023-09-13T03:28:47.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.500+0000 7fda85ffb700 10 client.27359 handle_command_reply: tid=0 2023-09-13T03:28:47.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.500+0000 7fdab89a4b80 1 client.27359 shutdown 2023-09-13T03:28:47.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.500+0000 7fdab89a4b80 1 -- 172.21.15.64:0/2561232868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ee9fd2300 msgr2=0x555ee9fdef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:28:47.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.500+0000 7fdab89a4b80 1 --2- 172.21.15.64:0/2561232868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ee9fd2300 0x555ee9fdef20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fda90015000 tx=0x7fda90009ca0 comp rx=0 tx=0).stop 2023-09-13T03:28:47.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.500+0000 7fdab89a4b80 1 -- 172.21.15.64:0/2561232868 shutdown_connections 2023-09-13T03:28:47.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.500+0000 7fdab89a4b80 1 --2- 172.21.15.64:0/2561232868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ee9fd2300 0x555ee9fdef20 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:47.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.500+0000 7fdab89a4b80 1 --2- 172.21.15.64:0/2561232868 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x555ee9fec4c0 0x555ee9fee8e0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:47.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.500+0000 7fdab89a4b80 1 --2- 172.21.15.64:0/2561232868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ee9fd81a0 0x555ee9fdc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:47.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.500+0000 7fdab89a4b80 1 --2- 172.21.15.64:0/2561232868 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ee9fd2cc0 0x555ee9fdc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:47.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.500+0000 7fdab89a4b80 1 -- 172.21.15.64:0/2561232868 >> 172.21.15.64:0/2561232868 conn(0x555ee9f47030 msgr2=0x555ee9fcc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:28:47.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.500+0000 7fda85ffb700 0 client.27359 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:28:47.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.501+0000 7fdab89a4b80 1 -- 172.21.15.64:0/2561232868 shutdown_connections 2023-09-13T03:28:47.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.501+0000 7fdab89a4b80 1 -- 172.21.15.64:0/2561232868 wait complete. 2023-09-13T03:28:47.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.501+0000 7fdab89a4b80 20 client.27359 trim_cache size 0 max 16384 2023-09-13T03:28:47.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.524+0000 7fdab89a4b80 1 Processor -- start 2023-09-13T03:28:47.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.524+0000 7fdab89a4b80 1 -- start start 2023-09-13T03:28:47.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.525+0000 7fdab89a4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ee9fd2300 0x555ee9fd9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:47.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.525+0000 7fdab89a4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ee9fd2cc0 0x555ee9fdeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:47.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.525+0000 7fdab89a4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ee9fd81a0 0x555ee9fcd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:47.527 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.525+0000 7fdab89a4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ee9fd7b80 con 0x555ee9fd81a0 2023-09-13T03:28:47.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.525+0000 7fdab89a4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ee9fced60 con 0x555ee9fd2300 2023-09-13T03:28:47.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.525+0000 7fdab89a4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ee9fda770 con 0x555ee9fd2cc0 2023-09-13T03:28:47.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.525+0000 7fda9dd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ee9fd2cc0 0x555ee9fdeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:47.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.525+0000 7fda9dd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ee9fd2cc0 0x555ee9fdeac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:38624/0 (socket says 172.21.15.64:38624) 2023-09-13T03:28:47.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.525+0000 7fda9dd9b700 1 -- 172.21.15.64:0/1949951822 learned_addr learned my addr 172.21.15.64:0/1949951822 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:28:47.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.525+0000 7fda9e59c700 1 --2- 172.21.15.64:0/1949951822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ee9fd2300 0x555ee9fd9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:47.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.525+0000 7fda9ed9d700 1 --2- 172.21.15.64:0/1949951822 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ee9fd81a0 0x555ee9fcd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:47.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.526+0000 7fda9e59c700 1 -- 172.21.15.64:0/1949951822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ee9fd2cc0 msgr2=0x555ee9fdeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:28:47.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.526+0000 7fda9e59c700 1 --2- 172.21.15.64:0/1949951822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ee9fd2cc0 0x555ee9fdeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:47.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.526+0000 7fda9e59c700 1 -- 172.21.15.64:0/1949951822 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ee9fd81a0 msgr2=0x555ee9fcd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:28:47.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.526+0000 7fda9e59c700 1 --2- 172.21.15.64:0/1949951822 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ee9fd81a0 0x555ee9fcd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:47.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.526+0000 7fda9e59c700 1 -- 172.21.15.64:0/1949951822 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ee9d771f0 con 0x555ee9fd2300 2023-09-13T03:28:47.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.526+0000 7fda9e59c700 1 --2- 172.21.15.64:0/1949951822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ee9fd2300 0x555ee9fd9d80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fda90000c00 tx=0x7fda90004820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a8d929f5ef4bff7 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:28:47.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.526+0000 7fda8d7fa700 1 -- 172.21.15.64:0/1949951822 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9002b070 con 0x555ee9fd2300 2023-09-13T03:28:47.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.526+0000 7fda8d7fa700 1 -- 172.21.15.64:0/1949951822 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda900254f0 con 0x555ee9fd2300 2023-09-13T03:28:47.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.526+0000 7fda8d7fa700 1 -- 172.21.15.64:0/1949951822 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda90025e40 con 0x555ee9fd2300 2023-09-13T03:28:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.527+0000 7fdab89a4b80 1 -- 172.21.15.64:0/1949951822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ee9fd2300 msgr2=0x555ee9fd9d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:28:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.527+0000 7fdab89a4b80 1 --2- 172.21.15.64:0/1949951822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ee9fd2300 0x555ee9fd9d80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fda90000c00 tx=0x7fda90004820 comp rx=0 tx=0).stop 2023-09-13T03:28:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.528+0000 7fdab89a4b80 1 -- 172.21.15.64:0/1949951822 shutdown_connections 2023-09-13T03:28:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.528+0000 7fdab89a4b80 1 --2- 172.21.15.64:0/1949951822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ee9fd2300 0x555ee9fd9d80 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.528+0000 7fdab89a4b80 1 --2- 172.21.15.64:0/1949951822 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ee9fd81a0 0x555ee9fcd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.528+0000 7fdab89a4b80 1 --2- 172.21.15.64:0/1949951822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ee9fd2cc0 0x555ee9fdeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.528+0000 7fdab89a4b80 1 -- 172.21.15.64:0/1949951822 >> 172.21.15.64:0/1949951822 conn(0x555ee9f47030 msgr2=0x555ee9fd63d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:28:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.528+0000 7fdab89a4b80 1 -- 172.21.15.64:0/1949951822 shutdown_connections 2023-09-13T03:28:47.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.528+0000 7fdab89a4b80 1 -- 172.21.15.64:0/1949951822 wait complete. 2023-09-13T03:28:47.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.529+0000 7fdab89a4b80 1 -- start start 2023-09-13T03:28:47.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.529+0000 7fdab89a4b80 1 Processor -- start 2023-09-13T03:28:47.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.530+0000 7fdab89a4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ee9fd2300 0x555ee9fe1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:47.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.530+0000 7fdab89a4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ee9fd2cc0 0x555ee9fe1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:47.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.530+0000 7fdab89a4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ee9fd81a0 0x555eea03f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:47.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.530+0000 7fdab89a4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ee9fe2420 con 0x555ee9fd2300 2023-09-13T03:28:47.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.530+0000 7fdab89a4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eea03f850 con 0x555ee9fd2cc0 2023-09-13T03:28:47.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.530+0000 7fdab89a4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eea03fa10 con 0x555ee9fd81a0 2023-09-13T03:28:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.530+0000 7fdab89a4b80 10 client.0 _ll_register_callbacks cb 0x555ee9e0ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:28:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.530+0000 7fda9ed9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ee9fd81a0 0x555eea03f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.530+0000 7fda9ed9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ee9fd81a0 0x555eea03f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:38630/0 (socket says 172.21.15.64:38630) 2023-09-13T03:28:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.530+0000 7fda9ed9d700 1 -- 172.21.15.64:0/3135940558 learned_addr learned my addr 172.21.15.64:0/3135940558 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:28:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.530+0000 7fda9dd9b700 1 --2- 172.21.15.64:0/3135940558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ee9fd2cc0 0x555ee9fe1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.530+0000 7fda9e59c700 1 --2- 172.21.15.64:0/3135940558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ee9fd2300 0x555ee9fe1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.531+0000 7fda9ed9d700 1 -- 172.21.15.64:0/3135940558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ee9fd2cc0 msgr2=0x555ee9fe1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:28:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.531+0000 7fda9ed9d700 1 --2- 172.21.15.64:0/3135940558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ee9fd2cc0 0x555ee9fe1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.531+0000 7fda9ed9d700 1 -- 172.21.15.64:0/3135940558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ee9fd2300 msgr2=0x555ee9fe1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:28:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.531+0000 7fda9ed9d700 1 --2- 172.21.15.64:0/3135940558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ee9fd2300 0x555ee9fe1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.531+0000 7fda9ed9d700 1 -- 172.21.15.64:0/3135940558 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555ee9d39c00 con 0x555ee9fd81a0 2023-09-13T03:28:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.531+0000 7fda9e59c700 1 --2- 172.21.15.64:0/3135940558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ee9fd2300 0x555ee9fe1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:28:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.531+0000 7fda9ed9d700 1 --2- 172.21.15.64:0/3135940558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ee9fd81a0 0x555eea03f310 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fda98098960 tx=0x7fda98003e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=347feea9a41c72a5 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:28:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.531+0000 7fda85ffb700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:28:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.531+0000 7fda85ffb700 1 -- 172.21.15.64:0/3135940558 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda980b8070 con 0x555ee9fd81a0 2023-09-13T03:28:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.531+0000 7fda85ffb700 1 -- 172.21.15.64:0/3135940558 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fda980990d0 con 0x555ee9fd81a0 2023-09-13T03:28:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.532+0000 7fdab89a4b80 1 -- 172.21.15.64:0/3135940558 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda94011e40 con 0x555ee9fd81a0 2023-09-13T03:28:47.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.532+0000 7fda85ffb700 1 -- 172.21.15.64:0/3135940558 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda980989d0 con 0x555ee9fd81a0 2023-09-13T03:28:47.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.532+0000 7fdab89a4b80 10 client.27325 fetch_fsmap learned FSMap version 77 2023-09-13T03:28:47.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.532+0000 7fdab89a4b80 1 -- 172.21.15.64:0/3135940558 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) v3 -- 0x555ee9d4f250 con 0x555ee9fd81a0 2023-09-13T03:28:47.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.532+0000 7fda85ffb700 1 -- 172.21.15.64:0/3135940558 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda94011e40 con 0x555ee9fd81a0 2023-09-13T03:28:47.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.532+0000 7fda85ffb700 1 -- 172.21.15.64:0/3135940558 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 77) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fda980ae030 con 0x555ee9fd81a0 2023-09-13T03:28:47.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.532+0000 7fdab89a4b80 10 client.27325 fetch_fsmap finished waiting for FSMap version 77 2023-09-13T03:28:47.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.532+0000 7fdab89a4b80 10 client.27325 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:28:47.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.532+0000 7fdab89a4b80 20 client.27325 populate_metadata read hostname 'smithi064' 2023-09-13T03:28:47.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.532+0000 7fdab89a4b80 1 --2- 172.21.15.64:0/3135940558 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x555eea041290 0x555eea0436b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:28:47.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.532+0000 7fdab89a4b80 4 client.27325 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:28:47.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.532+0000 7fdab89a4b80 1 -- 172.21.15.64:0/3135940558 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555ee9f1d8c0 con 0x555eea041290 2023-09-13T03:28:47.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.533+0000 7fda9e59c700 1 --2- 172.21.15.64:0/3135940558 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x555eea041290 0x555eea0436b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:28:47.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.533+0000 7fda9e59c700 1 --2- 172.21.15.64:0/3135940558 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x555eea041290 0x555eea0436b0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d7ed37b0ce2cb1d server_cookie=f31066aca74bbed2 in_seq=0 out_seq=0 2023-09-13T03:28:47.544 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.533+0000 7fda85ffb700 10 client.27325 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:28:47.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.763+0000 7fda85ffb700 1 -- 172.21.15.64:0/3135940558 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555ee9f1d8c0 con 0x555eea041290 2023-09-13T03:28:47.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.763+0000 7fda85ffb700 10 client.27325 handle_command_reply: tid=0 2023-09-13T03:28:47.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.763+0000 7fdab89a4b80 1 client.27325 shutdown 2023-09-13T03:28:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.765+0000 7fdab89a4b80 1 -- 172.21.15.64:0/3135940558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ee9fd81a0 msgr2=0x555eea03f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:28:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.765+0000 7fdab89a4b80 1 --2- 172.21.15.64:0/3135940558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ee9fd81a0 0x555eea03f310 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fda98098960 tx=0x7fda98003e50 comp rx=0 tx=0).stop 2023-09-13T03:28:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.768+0000 7fdab89a4b80 1 -- 172.21.15.64:0/3135940558 shutdown_connections 2023-09-13T03:28:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.768+0000 7fdab89a4b80 1 --2- 172.21.15.64:0/3135940558 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x555eea041290 0x555eea0436b0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.768+0000 7fdab89a4b80 1 --2- 172.21.15.64:0/3135940558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ee9fd81a0 0x555eea03f310 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.768+0000 7fdab89a4b80 1 --2- 172.21.15.64:0/3135940558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ee9fd2300 0x555ee9fe1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.768+0000 7fdab89a4b80 1 --2- 172.21.15.64:0/3135940558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ee9fd2cc0 0x555ee9fe1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.768+0000 7fdab89a4b80 1 -- 172.21.15.64:0/3135940558 >> 172.21.15.64:0/3135940558 conn(0x555ee9f47030 msgr2=0x555ee9fcd090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:28:47.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.768+0000 7fda85ffb700 0 client.27325 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:28:47.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.770+0000 7fdab89a4b80 1 -- 172.21.15.64:0/3135940558 shutdown_connections 2023-09-13T03:28:47.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.770+0000 7fdab89a4b80 1 -- 172.21.15.64:0/3135940558 wait complete. 2023-09-13T03:28:47.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.771+0000 7fdab89a4b80 20 client.27325 trim_cache size 0 max 16384 2023-09-13T03:28:47.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:28:47.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:28:47.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:28:47.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:28:47.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.772+0000 7fdaa4fff700 1 -- 172.21.15.64:0/2012520295 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fda880a07b0 msgr2=0x7fda880a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:28:47.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.772+0000 7fdaa4fff700 1 --2- 172.21.15.64:0/2012520295 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fda880a07b0 0x7fda880a2c70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x555ee9fcd060 tx=0x7fda90005fd0 comp rx=0 tx=0).stop 2023-09-13T03:28:47.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.772+0000 7fdaa4fff700 1 -- 172.21.15.64:0/2012520295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdaa010ce40 msgr2=0x7fdaa0102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:28:47.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.772+0000 7fdaa4fff700 1 --2- 172.21.15.64:0/2012520295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdaa010ce40 0x7fdaa0102e70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fda98007ec0 tx=0x7fda9800d3b0 comp rx=0 tx=0).stop 2023-09-13T03:28:47.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.772+0000 7fdaa4fff700 1 -- 172.21.15.64:0/2012520295 shutdown_connections 2023-09-13T03:28:47.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.772+0000 7fdaa4fff700 1 --2- 172.21.15.64:0/2012520295 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fda880a07b0 0x7fda880a2c70 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:47.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.772+0000 7fdaa4fff700 1 --2- 172.21.15.64:0/2012520295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdaa010ce40 0x7fdaa0102e70 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:47.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.772+0000 7fdaa4fff700 1 --2- 172.21.15.64:0/2012520295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdaa00729d0 0x7fdaa0107fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:47.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.772+0000 7fdaa4fff700 1 --2- 172.21.15.64:0/2012520295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdaa00732f0 0x7fdaa0108520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:28:47.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.772+0000 7fdaa4fff700 1 -- 172.21.15.64:0/2012520295 >> 172.21.15.64:0/2012520295 conn(0x7fdaa00fc0b0 msgr2=0x7fdaa010f040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:28:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.773+0000 7fdaa4fff700 1 -- 172.21.15.64:0/2012520295 shutdown_connections 2023-09-13T03:28:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:28:47.773+0000 7fdaa4fff700 1 -- 172.21.15.64:0/2012520295 wait complete. 2023-09-13T03:28:48.305 INFO:tasks.workunit.client.0.smithi064.stdout: 10 145287 44.53 MB/sec execute 52 sec latency 135.913 ms 2023-09-13T03:28:49.166 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:28:48 smithi170 ceph-mon[134328]: pgmap v490: 97 pgs: 97 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 911 op/s 2023-09-13T03:28:49.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:28:48 smithi064 ceph-mon[97675]: pgmap v490: 97 pgs: 97 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 911 op/s 2023-09-13T03:28:49.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:28:48 smithi142 ceph-mon[134082]: pgmap v490: 97 pgs: 97 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 911 op/s 2023-09-13T03:28:49.305 INFO:tasks.workunit.client.0.smithi064.stdout: 10 146132 44.69 MB/sec execute 53 sec latency 50.359 ms 2023-09-13T03:28:50.305 INFO:tasks.workunit.client.0.smithi064.stdout: 10 146782 44.50 MB/sec execute 54 sec latency 128.073 ms 2023-09-13T03:28:50.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:28:50 smithi170 ceph-mon[134328]: pgmap v491: 97 pgs: 97 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 716 op/s 2023-09-13T03:28:50.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:28:50 smithi064 ceph-mon[97675]: pgmap v491: 97 pgs: 97 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 716 op/s 2023-09-13T03:28:50.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:28:50 smithi142 ceph-mon[134082]: pgmap v491: 97 pgs: 97 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 716 op/s 2023-09-13T03:28:51.310 INFO:tasks.workunit.client.0.smithi064.stdout: 10 147360 44.24 MB/sec execute 55 sec latency 190.219 ms 2023-09-13T03:28:52.310 INFO:tasks.workunit.client.0.smithi064.stdout: 10 148163 44.12 MB/sec execute 56 sec latency 229.856 ms 2023-09-13T03:28:53.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:28:52 smithi170 ceph-mon[134328]: pgmap v492: 97 pgs: 97 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 18 MiB/s wr, 1.15k op/s 2023-09-13T03:28:53.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:28:52 smithi064 ceph-mon[97675]: pgmap v492: 97 pgs: 97 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 18 MiB/s wr, 1.15k op/s 2023-09-13T03:28:53.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:28:52 smithi142 ceph-mon[134082]: pgmap v492: 97 pgs: 97 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 18 MiB/s wr, 1.15k op/s 2023-09-13T03:28:53.310 INFO:tasks.workunit.client.0.smithi064.stdout: 10 149229 44.50 MB/sec execute 57 sec latency 54.925 ms 2023-09-13T03:28:54.310 INFO:tasks.workunit.client.0.smithi064.stdout: 10 150088 44.64 MB/sec execute 58 sec latency 53.900 ms 2023-09-13T03:28:55.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:28:54 smithi170 ceph-mon[134328]: pgmap v493: 97 pgs: 97 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 12 MiB/s wr, 791 op/s 2023-09-13T03:28:55.170 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:28:54 smithi064 ceph-mon[97675]: pgmap v493: 97 pgs: 97 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 12 MiB/s wr, 791 op/s 2023-09-13T03:28:55.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:28:54 smithi142 ceph-mon[134082]: pgmap v493: 97 pgs: 97 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 12 MiB/s wr, 791 op/s 2023-09-13T03:28:55.310 INFO:tasks.workunit.client.0.smithi064.stdout: 10 150862 44.43 MB/sec execute 59 sec latency 49.506 ms 2023-09-13T03:28:56.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:28:55 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:28:56.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:28:55 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:28:56.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:28:55 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:28:56.313 INFO:tasks.workunit.client.0.smithi064.stdout: 10 151737 44.34 MB/sec execute 60 sec latency 101.189 ms 2023-09-13T03:28:57.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:28:56 smithi170 ceph-mon[134328]: pgmap v494: 97 pgs: 97 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 996 op/s 2023-09-13T03:28:57.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:28:56 smithi064 ceph-mon[97675]: pgmap v494: 97 pgs: 97 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 996 op/s 2023-09-13T03:28:57.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:28:56 smithi142 ceph-mon[134082]: pgmap v494: 97 pgs: 97 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 996 op/s 2023-09-13T03:28:57.312 INFO:tasks.workunit.client.0.smithi064.stdout: 10 152624 44.47 MB/sec execute 61 sec latency 275.782 ms 2023-09-13T03:28:58.312 INFO:tasks.workunit.client.0.smithi064.stdout: 10 153429 44.61 MB/sec execute 62 sec latency 69.802 ms 2023-09-13T03:28:59.162 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:28:58 smithi170 ceph-mon[134328]: pgmap v495: 97 pgs: 97 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 1.01k op/s 2023-09-13T03:28:59.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:28:58 smithi064 ceph-mon[97675]: pgmap v495: 97 pgs: 97 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 1.01k op/s 2023-09-13T03:28:59.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:28:58 smithi142 ceph-mon[134082]: pgmap v495: 97 pgs: 97 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 1.01k op/s 2023-09-13T03:28:59.313 INFO:tasks.workunit.client.0.smithi064.stdout: 10 154200 44.55 MB/sec execute 63 sec latency 44.428 ms 2023-09-13T03:29:00.313 INFO:tasks.workunit.client.0.smithi064.stdout: 10 154967 44.28 MB/sec execute 64 sec latency 83.080 ms 2023-09-13T03:29:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.010+0000 7f55965ff700 1 -- 172.21.15.64:0/3674162877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f559010afb0 msgr2=0x7f559010b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.010+0000 7f55965ff700 1 --2- 172.21.15.64:0/3674162877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f559010afb0 0x7f559010b430 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5584009990 tx=0x7f5584009ca0 comp rx=0 tx=0).stop 2023-09-13T03:29:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.010+0000 7f55965ff700 1 -- 172.21.15.64:0/3674162877 shutdown_connections 2023-09-13T03:29:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.010+0000 7f55965ff700 1 --2- 172.21.15.64:0/3674162877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f559010b970 0x7f5590117e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.010+0000 7f55965ff700 1 --2- 172.21.15.64:0/3674162877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5590111070 0x7f5590111450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.010+0000 7f55965ff700 1 --2- 172.21.15.64:0/3674162877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f559010afb0 0x7f559010b430 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.010+0000 7f55965ff700 1 -- 172.21.15.64:0/3674162877 >> 172.21.15.64:0/3674162877 conn(0x7f559006b3d0 msgr2=0x7f5590109610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.011+0000 7f55965ff700 1 -- 172.21.15.64:0/3674162877 shutdown_connections 2023-09-13T03:29:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.011+0000 7f55965ff700 1 -- 172.21.15.64:0/3674162877 wait complete. 2023-09-13T03:29:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.013+0000 7f55965ff700 1 Processor -- start 2023-09-13T03:29:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.014+0000 7f55965ff700 1 -- start start 2023-09-13T03:29:01.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.015+0000 7f55965ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f559010afb0 0x7f5590117a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:01.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.015+0000 7f55965ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f559010b970 0x7f55901148c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:01.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.015+0000 7f55965ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5590111070 0x7f5590114e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:01.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.015+0000 7f55965ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55901153d0 con 0x7f5590111070 2023-09-13T03:29:01.016 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.015+0000 7f55965ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5590115540 con 0x7f559010afb0 2023-09-13T03:29:01.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.015+0000 7f55965ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5590115700 con 0x7f559010b970 2023-09-13T03:29:01.017 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.015+0000 7f558ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f559010afb0 0x7f5590117a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:01.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.016+0000 7f558ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f559010afb0 0x7f5590117a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:54272/0 (socket says 172.21.15.64:54272) 2023-09-13T03:29:01.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.016+0000 7f558ffff700 1 -- 172.21.15.64:0/297502517 learned_addr learned my addr 172.21.15.64:0/297502517 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:29:01.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.016+0000 7f558f7fe700 1 --2- 172.21.15.64:0/297502517 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f559010b970 0x7f55901148c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:01.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.016+0000 7f558ffff700 1 -- 172.21.15.64:0/297502517 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f559010b970 msgr2=0x7f55901148c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:01.018 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.016+0000 7f558ffff700 1 --2- 172.21.15.64:0/297502517 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f559010b970 0x7f55901148c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.016+0000 7f558ffff700 1 -- 172.21.15.64:0/297502517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5590111070 msgr2=0x7f5590114e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.016+0000 7f558ffff700 1 --2- 172.21.15.64:0/297502517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5590111070 0x7f5590114e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.016+0000 7f558ffff700 1 -- 172.21.15.64:0/297502517 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55840095f0 con 0x7f559010afb0 2023-09-13T03:29:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.017+0000 7f558ffff700 1 --2- 172.21.15.64:0/297502517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f559010afb0 0x7f5590117a70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f558000b560 tx=0x7f558000b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.017+0000 7f558d7fa700 1 -- 172.21.15.64:0/297502517 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5580011830 con 0x7f559010afb0 2023-09-13T03:29:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.017+0000 7f558d7fa700 1 -- 172.21.15.64:0/297502517 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5580011e50 con 0x7f559010afb0 2023-09-13T03:29:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.017+0000 7f558d7fa700 1 -- 172.21.15.64:0/297502517 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5580010e70 con 0x7f559010afb0 2023-09-13T03:29:01.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.017+0000 7f55965ff700 1 -- 172.21.15.64:0/297502517 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55901a9760 con 0x7f559010afb0 2023-09-13T03:29:01.021 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.018+0000 7f55965ff700 1 -- 172.21.15.64:0/297502517 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55901a9ca0 con 0x7f559010afb0 2023-09-13T03:29:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.023+0000 7f55a9e6bb80 1 Processor -- start 2023-09-13T03:29:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.024+0000 7f55a9e6bb80 1 -- start start 2023-09-13T03:29:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.025+0000 7f55a9e6bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e77aea81a0 0x55e77aea8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.025+0000 7f55a9e6bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e77aea2300 0x55e77aea2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.025+0000 7f55a9e6bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e77aea2cc0 0x55e77aeaef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.025+0000 7f55a9e6bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e77adfb6d0 con 0x55e77aea81a0 2023-09-13T03:29:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.025+0000 7f55a9e6bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e77adf8ff0 con 0x55e77aea2300 2023-09-13T03:29:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.025+0000 7f55a9e6bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e77adf48e0 con 0x55e77aea2cc0 2023-09-13T03:29:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.026+0000 7f5594b9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e77aea81a0 0x55e77aea8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.026+0000 7f5594b9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e77aea81a0 0x55e77aea8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36726/0 (socket says 172.21.15.64:36726) 2023-09-13T03:29:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.026+0000 7f5594b9c700 1 -- 172.21.15.64:0/357208374 learned_addr learned my addr 172.21.15.64:0/357208374 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:29:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.026+0000 7f5594b9c700 1 -- 172.21.15.64:0/357208374 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e77aea2cc0 msgr2=0x55e77aeaef30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-13T03:29:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.026+0000 7f5594b9c700 1 --2- 172.21.15.64:0/357208374 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e77aea2cc0 0x55e77aeaef30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.026+0000 7f5594b9c700 1 -- 172.21.15.64:0/357208374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e77aea2300 msgr2=0x55e77aea2780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-13T03:29:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.026+0000 7f5594b9c700 1 --2- 172.21.15.64:0/357208374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e77aea2300 0x55e77aea2780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:01.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.026+0000 7f5594b9c700 1 -- 172.21.15.64:0/357208374 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e77ac20d80 con 0x55e77aea81a0 2023-09-13T03:29:01.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.027+0000 7f5594b9c700 1 --2- 172.21.15.64:0/357208374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e77aea81a0 0x55e77aea8580 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f558800e400 tx=0x7f558800e710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc57de75e83f7e7e server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:01.029 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.028+0000 7f558d7fa700 1 -- 172.21.15.64:0/297502517 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f5580010510 con 0x7f559010afb0 2023-09-13T03:29:01.031 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.030+0000 7f558d7fa700 1 --2- 172.21.15.64:0/297502517 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f55780a08b0 0x7f55780a2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.031+0000 7f558d7fa700 1 -- 172.21.15.64:0/297502517 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f55800cb910 con 0x7f559010afb0 2023-09-13T03:29:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.031+0000 7f55767fc700 1 -- 172.21.15.64:0/357208374 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5588018070 con 0x55e77aea81a0 2023-09-13T03:29:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.031+0000 7f55767fc700 1 -- 172.21.15.64:0/357208374 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f558800f040 con 0x55e77aea81a0 2023-09-13T03:29:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.031+0000 7f55767fc700 1 -- 172.21.15.64:0/357208374 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5588004280 con 0x55e77aea81a0 2023-09-13T03:29:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.031+0000 7f55a9e6bb80 1 -- 172.21.15.64:0/357208374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e77aea81a0 msgr2=0x55e77aea8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.031+0000 7f55a9e6bb80 1 --2- 172.21.15.64:0/357208374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e77aea81a0 0x55e77aea8580 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f558800e400 tx=0x7f558800e710 comp rx=0 tx=0).stop 2023-09-13T03:29:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.032+0000 7f55a9e6bb80 1 -- 172.21.15.64:0/357208374 shutdown_connections 2023-09-13T03:29:01.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.032+0000 7f55a9e6bb80 1 --2- 172.21.15.64:0/357208374 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e77aea2cc0 0x55e77aeaef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:01.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.032+0000 7f55a9e6bb80 1 --2- 172.21.15.64:0/357208374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e77aea81a0 0x55e77aea8580 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:01.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.032+0000 7f55a9e6bb80 1 --2- 172.21.15.64:0/357208374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e77aea2300 0x55e77aea2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:01.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.032+0000 7f55a9e6bb80 1 -- 172.21.15.64:0/357208374 >> 172.21.15.64:0/357208374 conn(0x55e77ae17030 msgr2=0x55e77ae9d560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:01.034 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.033+0000 7f558ffff700 1 --2- 172.21.15.64:0/297502517 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f55780a08b0 0x7f55780a2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.033+0000 7f55a9e6bb80 1 -- 172.21.15.64:0/357208374 shutdown_connections 2023-09-13T03:29:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.034+0000 7f55a9e6bb80 1 -- 172.21.15.64:0/357208374 wait complete. 2023-09-13T03:29:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.034+0000 7f55a9e6bb80 1 -- start start 2023-09-13T03:29:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.034+0000 7f558ffff700 1 --2- 172.21.15.64:0/297502517 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f55780a08b0 0x7f55780a2d70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x55e77ae9d060 tx=0x7f5580004750 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.034+0000 7f55a9e6bb80 1 Processor -- start 2023-09-13T03:29:01.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.035+0000 7f55a9e6bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e77aea2300 0x55e77aeaee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:01.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.035+0000 7f55a9e6bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e77aea2cc0 0x55e77aeac0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:01.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.035+0000 7f55a9e6bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e77aea81a0 0x55e77aeac770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:01.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.035+0000 7f55a9e6bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e77ad92b10 con 0x55e77aea81a0 2023-09-13T03:29:01.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.035+0000 7f55a9e6bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e77ad92810 con 0x55e77aea2cc0 2023-09-13T03:29:01.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.035+0000 7f55a9e6bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e77ac768c0 con 0x55e77aea2300 2023-09-13T03:29:01.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.035+0000 7f55a9e6bb80 10 client.0 _ll_register_callbacks cb 0x55e77acdca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:29:01.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.036+0000 7f558f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e77aea2cc0 0x55e77aeac0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:01.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.036+0000 7f5594b9c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e77aea2300 0x55e77aeaee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:01.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.036+0000 7f5594b9c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e77aea2300 0x55e77aeaee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:48884/0 (socket says 172.21.15.64:48884) 2023-09-13T03:29:01.039 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.036+0000 7f5594b9c700 1 -- 172.21.15.64:0/1608480975 learned_addr learned my addr 172.21.15.64:0/1608480975 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:29:01.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.037+0000 7f5594b9c700 1 -- 172.21.15.64:0/1608480975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e77aea2cc0 msgr2=0x55e77aeac0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:29:01.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.037+0000 7f5594b9c700 1 --2- 172.21.15.64:0/1608480975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e77aea2cc0 0x55e77aeac0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:01.040 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.037+0000 7f5594b9c700 1 -- 172.21.15.64:0/1608480975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e77aea81a0 msgr2=0x55e77aeac770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:29:01.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.037+0000 7f5594b9c700 1 --2- 172.21.15.64:0/1608480975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e77aea81a0 0x55e77aeac770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:01.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.037+0000 7f5594b9c700 1 -- 172.21.15.64:0/1608480975 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f558800e140 con 0x55e77aea2300 2023-09-13T03:29:01.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.037+0000 7f5594b9c700 1 --2- 172.21.15.64:0/1608480975 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e77aea2300 0x55e77aeaee70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5588004ee0 tx=0x7f5588009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7033020208c755e9 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:01.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.037+0000 7f556f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-13T03:29:01.041 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.038+0000 7f556f7fe700 1 -- 172.21.15.64:0/1608480975 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5588018070 con 0x55e77aea2300 2023-09-13T03:29:01.042 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.038+0000 7f556f7fe700 1 -- 172.21.15.64:0/1608480975 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f5588004000 con 0x55e77aea2300 2023-09-13T03:29:01.042 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.038+0000 7f55a9e6bb80 1 -- 172.21.15.64:0/1608480975 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e77ad4c490 con 0x55e77aea2300 2023-09-13T03:29:01.042 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.039+0000 7f556f7fe700 1 -- 172.21.15.64:0/1608480975 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55880041c0 con 0x55e77aea2300 2023-09-13T03:29:01.042 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.039+0000 7f55a9e6bb80 10 client.27334 fetch_fsmap learned FSMap version 79 2023-09-13T03:29:01.042 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.039+0000 7f55a9e6bb80 1 -- 172.21.15.64:0/1608480975 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=4+}) v3 -- 0x55e77ac09c00 con 0x55e77aea2300 2023-09-13T03:29:01.043 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.040+0000 7f556f7fe700 1 -- 172.21.15.64:0/1608480975 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e77ad4c490 con 0x55e77aea2300 2023-09-13T03:29:01.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.040+0000 7f556f7fe700 1 -- 172.21.15.64:0/1608480975 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 79) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f5588007af0 con 0x55e77aea2300 2023-09-13T03:29:01.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.040+0000 7f55a9e6bb80 10 client.27334 fetch_fsmap finished waiting for FSMap version 79 2023-09-13T03:29:01.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.040+0000 7f55a9e6bb80 10 client.27334 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:29:01.044 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.040+0000 7f55a9e6bb80 20 client.27334 populate_metadata read hostname 'smithi064' 2023-09-13T03:29:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.040+0000 7f55a9e6bb80 1 --2- 172.21.15.64:0/1608480975 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55e77aebc290 0x55e77aebe6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.040+0000 7f55a9e6bb80 4 client.27334 mds_command: new command op to 24686 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-13T03:29:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.040+0000 7f55a9e6bb80 1 -- 172.21.15.64:0/1608480975 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e77ae09840 con 0x55e77aebc290 2023-09-13T03:29:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.041+0000 7f558f7fe700 1 --2- 172.21.15.64:0/1608480975 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55e77aebc290 0x55e77aebe6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.042+0000 7f558f7fe700 1 --2- 172.21.15.64:0/1608480975 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55e77aebc290 0x55e77aebe6b0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f1e6d13509d462e server_cookie=aecdbb904e75e9c4 in_seq=0 out_seq=0 2023-09-13T03:29:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.042+0000 7f556f7fe700 10 client.27334 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:29:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.043+0000 7f556f7fe700 1 -- 172.21.15.64:0/1608480975 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e77ae09840 con 0x55e77aebc290 2023-09-13T03:29:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.043+0000 7f556f7fe700 10 client.27334 handle_command_reply: tid=0 2023-09-13T03:29:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.043+0000 7f55a9e6bb80 1 client.27334 shutdown 2023-09-13T03:29:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.044+0000 7f55a9e6bb80 1 -- 172.21.15.64:0/1608480975 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e77aea2300 msgr2=0x55e77aeaee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.044+0000 7f55a9e6bb80 1 --2- 172.21.15.64:0/1608480975 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e77aea2300 0x55e77aeaee70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5588004ee0 tx=0x7f5588009920 comp rx=0 tx=0).stop 2023-09-13T03:29:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.044+0000 7f55a9e6bb80 1 -- 172.21.15.64:0/1608480975 shutdown_connections 2023-09-13T03:29:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.044+0000 7f55a9e6bb80 1 --2- 172.21.15.64:0/1608480975 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55e77aebc290 0x55e77aebe6b0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.044+0000 7f55a9e6bb80 1 --2- 172.21.15.64:0/1608480975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e77aea81a0 0x55e77aeac770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.044+0000 7f55a9e6bb80 1 --2- 172.21.15.64:0/1608480975 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e77aea2300 0x55e77aeaee70 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.044+0000 7f55a9e6bb80 1 --2- 172.21.15.64:0/1608480975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e77aea2cc0 0x55e77aeac0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.044+0000 7f55a9e6bb80 1 -- 172.21.15.64:0/1608480975 >> 172.21.15.64:0/1608480975 conn(0x55e77ae17030 msgr2=0x55e77ae9c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.044+0000 7f556f7fe700 0 client.27334 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:29:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.044+0000 7f55a9e6bb80 1 -- 172.21.15.64:0/1608480975 shutdown_connections 2023-09-13T03:29:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.044+0000 7f55a9e6bb80 1 -- 172.21.15.64:0/1608480975 wait complete. 2023-09-13T03:29:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.045+0000 7f55a9e6bb80 20 client.27334 trim_cache size 0 max 16384 2023-09-13T03:29:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.068+0000 7f55a9e6bb80 1 Processor -- start 2023-09-13T03:29:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.069+0000 7f55a9e6bb80 1 -- start start 2023-09-13T03:29:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.069+0000 7f55a9e6bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e77aea2300 0x55e77aeaea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.069+0000 7f55a9e6bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e77aea2cc0 0x55e77aeab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.069+0000 7f55a9e6bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e77aea81a0 0x55e77aeabe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.069+0000 7f55a9e6bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e77ae978a0 con 0x55e77aea81a0 2023-09-13T03:29:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.069+0000 7f55a9e6bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e77aead080 con 0x55e77aea2300 2023-09-13T03:29:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.069+0000 7f55a9e6bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e77aead6d0 con 0x55e77aea2cc0 2023-09-13T03:29:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.070+0000 7f558f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e77aea2cc0 0x55e77aeab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.070+0000 7f5594b9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e77aea2300 0x55e77aeaea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:01.073 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.070+0000 7f558f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e77aea2cc0 0x55e77aeab8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:48886/0 (socket says 172.21.15.64:48886) 2023-09-13T03:29:01.073 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.070+0000 7f558f7fe700 1 -- 172.21.15.64:0/3920927735 learned_addr learned my addr 172.21.15.64:0/3920927735 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:29:01.073 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.070+0000 7f558f7fe700 1 -- 172.21.15.64:0/3920927735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e77aea2300 msgr2=0x55e77aeaea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:29:01.073 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.070+0000 7f558f7fe700 1 --2- 172.21.15.64:0/3920927735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e77aea2300 0x55e77aeaea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:01.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.070+0000 7f558f7fe700 1 -- 172.21.15.64:0/3920927735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e77aea81a0 msgr2=0x55e77aeabe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:29:01.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.070+0000 7f558f7fe700 1 --2- 172.21.15.64:0/3920927735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e77aea81a0 0x55e77aeabe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:01.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.070+0000 7f558f7fe700 1 -- 172.21.15.64:0/3920927735 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e77ac471f0 con 0x55e77aea2cc0 2023-09-13T03:29:01.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.071+0000 7f558f7fe700 1 --2- 172.21.15.64:0/3920927735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e77aea2cc0 0x55e77aeab8d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f55840095c0 tx=0x7f5584005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12f5b62f37414d55 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:01.074 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.071+0000 7f5576ffd700 1 -- 172.21.15.64:0/3920927735 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5584025990 con 0x55e77aea2cc0 2023-09-13T03:29:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.071+0000 7f5576ffd700 1 -- 172.21.15.64:0/3920927735 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f558401e070 con 0x55e77aea2cc0 2023-09-13T03:29:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.071+0000 7f5576ffd700 1 -- 172.21.15.64:0/3920927735 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558400f7c0 con 0x55e77aea2cc0 2023-09-13T03:29:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.072+0000 7f55a9e6bb80 1 -- 172.21.15.64:0/3920927735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e77aea2cc0 msgr2=0x55e77aeab8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.072+0000 7f55a9e6bb80 1 --2- 172.21.15.64:0/3920927735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e77aea2cc0 0x55e77aeab8d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f55840095c0 tx=0x7f5584005fd0 comp rx=0 tx=0).stop 2023-09-13T03:29:01.076 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.072+0000 7f55a9e6bb80 1 -- 172.21.15.64:0/3920927735 shutdown_connections 2023-09-13T03:29:01.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.072+0000 7f55a9e6bb80 1 --2- 172.21.15.64:0/3920927735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e77aea2300 0x55e77aeaea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:01.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.072+0000 7f55a9e6bb80 1 --2- 172.21.15.64:0/3920927735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e77aea81a0 0x55e77aeabe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:01.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.072+0000 7f55a9e6bb80 1 --2- 172.21.15.64:0/3920927735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e77aea2cc0 0x55e77aeab8d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:01.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.072+0000 7f55a9e6bb80 1 -- 172.21.15.64:0/3920927735 >> 172.21.15.64:0/3920927735 conn(0x55e77ae17030 msgr2=0x55e77ae9d090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:01.077 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.072+0000 7f55a9e6bb80 1 -- 172.21.15.64:0/3920927735 shutdown_connections 2023-09-13T03:29:01.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.073+0000 7f55a9e6bb80 1 -- 172.21.15.64:0/3920927735 wait complete. 2023-09-13T03:29:01.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.073+0000 7f55a9e6bb80 1 -- start start 2023-09-13T03:29:01.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.073+0000 7f55a9e6bb80 1 Processor -- start 2023-09-13T03:29:01.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.074+0000 7f55a9e6bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e77aea2300 0x55e77aeb1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:01.078 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.074+0000 7f55a9e6bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e77aea2cc0 0x55e77aeb20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:01.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.074+0000 7f55a9e6bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e77aea81a0 0x55e77aeb2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:01.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.074+0000 7f55a9e6bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e77af100f0 con 0x55e77aea2cc0 2023-09-13T03:29:01.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.074+0000 7f55a9e6bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e77af10230 con 0x55e77aea2300 2023-09-13T03:29:01.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.074+0000 7f55a9e6bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e77af103f0 con 0x55e77aea81a0 2023-09-13T03:29:01.079 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.074+0000 7f55a9e6bb80 10 client.0 _ll_register_callbacks cb 0x7f558800e2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-13T03:29:01.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.075+0000 7f5594b9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e77aea2300 0x55e77aeb1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:01.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.075+0000 7f5594b9c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e77aea2300 0x55e77aeb1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:54294/0 (socket says 172.21.15.64:54294) 2023-09-13T03:29:01.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.075+0000 7f5594b9c700 1 -- 172.21.15.64:0/2569374682 learned_addr learned my addr 172.21.15.64:0/2569374682 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:29:01.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.075+0000 7f5594b9c700 1 --2- 172.21.15.64:0/2569374682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e77aea81a0 0x55e77aeb2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:01.080 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.076+0000 7f5594b9c700 1 -- 172.21.15.64:0/2569374682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e77aea81a0 msgr2=0x55e77aeb2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:29:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.076+0000 7f5594b9c700 1 --2- 172.21.15.64:0/2569374682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e77aea81a0 0x55e77aeb2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.076+0000 7f5594b9c700 1 -- 172.21.15.64:0/2569374682 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e77aea2cc0 msgr2=0x55e77aeb20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-13T03:29:01.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.076+0000 7f5594b9c700 1 --2- 172.21.15.64:0/2569374682 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e77aea2cc0 0x55e77aeb20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:01.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.076+0000 7f5594b9c700 1 -- 172.21.15.64:0/2569374682 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5590118700 con 0x55e77aea2300 2023-09-13T03:29:01.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.076+0000 7f5594b9c700 1 --2- 172.21.15.64:0/2569374682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e77aea2300 0x55e77aeb1ba0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f558800ec10 tx=0x7f558800e710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df88a5490a17d987 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:01.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.076+0000 7f556f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2023-09-13T03:29:01.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.077+0000 7f556f7fe700 1 -- 172.21.15.64:0/2569374682 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5588018040 con 0x55e77aea2300 2023-09-13T03:29:01.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.077+0000 7f556f7fe700 1 -- 172.21.15.64:0/2569374682 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f5588007910 con 0x55e77aea2300 2023-09-13T03:29:01.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.077+0000 7f55a9e6bb80 1 -- 172.21.15.64:0/2569374682 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f558800f040 con 0x55e77aea2300 2023-09-13T03:29:01.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.077+0000 7f556f7fe700 1 -- 172.21.15.64:0/2569374682 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5588007ad0 con 0x55e77aea2300 2023-09-13T03:29:01.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.077+0000 7f55a9e6bb80 10 client.27377 fetch_fsmap learned FSMap version 79 2023-09-13T03:29:01.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.077+0000 7f55a9e6bb80 1 -- 172.21.15.64:0/2569374682 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=4+}) v3 -- 0x55e77ac1f250 con 0x55e77aea2300 2023-09-13T03:29:01.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.078+0000 7f556f7fe700 1 -- 172.21.15.64:0/2569374682 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f558800f040 con 0x55e77aea2300 2023-09-13T03:29:01.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.078+0000 7f556f7fe700 1 -- 172.21.15.64:0/2569374682 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 79) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f558800e830 con 0x55e77aea2300 2023-09-13T03:29:01.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.078+0000 7f55a9e6bb80 10 client.27377 fetch_fsmap finished waiting for FSMap version 79 2023-09-13T03:29:01.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.078+0000 7f55a9e6bb80 10 client.27377 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-13T03:29:01.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.078+0000 7f55a9e6bb80 20 client.27377 populate_metadata read hostname 'smithi064' 2023-09-13T03:29:01.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.078+0000 7f55a9e6bb80 1 --2- 172.21.15.64:0/2569374682 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55e77af338f0 0x55e77af35d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:01.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.078+0000 7f55a9e6bb80 4 client.27377 mds_command: new command op to 24686 tid=0[{"prefix": "flush journal"}] 2023-09-13T03:29:01.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.078+0000 7f55a9e6bb80 1 -- 172.21.15.64:0/2569374682 --> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e77aded8c0 con 0x55e77af338f0 2023-09-13T03:29:01.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.079+0000 7f558f7fe700 1 --2- 172.21.15.64:0/2569374682 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55e77af338f0 0x55e77af35d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:01.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.081+0000 7f558f7fe700 1 --2- 172.21.15.64:0/2569374682 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55e77af338f0 0x55e77af35d10 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73532c3565abe681 server_cookie=71a869bc88cbc2d1 in_seq=0 out_seq=0 2023-09-13T03:29:01.086 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.081+0000 7f556f7fe700 10 client.27377 ms_handle_connect on v2:172.21.15.142:6838/3645998616 2023-09-13T03:29:01.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:00 smithi170 ceph-mon[134328]: pgmap v496: 97 pgs: 97 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 12 MiB/s wr, 809 op/s 2023-09-13T03:29:01.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:00 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:29:01.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:00 smithi064 ceph-mon[97675]: pgmap v496: 97 pgs: 97 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 12 MiB/s wr, 809 op/s 2023-09-13T03:29:01.170 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:00 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:29:01.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:00 smithi142 ceph-mon[134082]: pgmap v496: 97 pgs: 97 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 12 MiB/s wr, 809 op/s 2023-09-13T03:29:01.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:00 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:29:01.313 INFO:tasks.workunit.client.0.smithi064.stdout: 10 155408 43.92 MB/sec execute 65 sec latency 206.264 ms 2023-09-13T03:29:01.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.340+0000 7f556f7fe700 1 -- 172.21.15.64:0/2569374682 <== mds.0 v2:172.21.15.142:6838/3645998616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e77aded8c0 con 0x55e77af338f0 2023-09-13T03:29:01.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.340+0000 7f556f7fe700 10 client.27377 handle_command_reply: tid=0 2023-09-13T03:29:01.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.340+0000 7f55a9e6bb80 1 client.27377 shutdown 2023-09-13T03:29:01.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.340+0000 7f55a9e6bb80 1 -- 172.21.15.64:0/2569374682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e77aea2300 msgr2=0x55e77aeb1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:01.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.340+0000 7f55a9e6bb80 1 --2- 172.21.15.64:0/2569374682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e77aea2300 0x55e77aeb1ba0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f558800ec10 tx=0x7f558800e710 comp rx=0 tx=0).stop 2023-09-13T03:29:01.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.340+0000 7f55a9e6bb80 1 -- 172.21.15.64:0/2569374682 shutdown_connections 2023-09-13T03:29:01.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.340+0000 7f55a9e6bb80 1 --2- 172.21.15.64:0/2569374682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e77aea2300 0x55e77aeb1ba0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:01.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.340+0000 7f55a9e6bb80 1 --2- 172.21.15.64:0/2569374682 >> [v2:172.21.15.142:6838/3645998616,v1:172.21.15.142:6839/3645998616] conn(0x55e77af338f0 0x55e77af35d10 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:01.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.340+0000 7f55a9e6bb80 1 --2- 172.21.15.64:0/2569374682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e77aea81a0 0x55e77aeb2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:01.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.340+0000 7f55a9e6bb80 1 --2- 172.21.15.64:0/2569374682 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e77aea2cc0 0x55e77aeb20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:01.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.340+0000 7f55a9e6bb80 1 -- 172.21.15.64:0/2569374682 >> 172.21.15.64:0/2569374682 conn(0x55e77ae17030 msgr2=0x55e77aea12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:01.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.340+0000 7f556f7fe700 0 client.27377 ms_handle_reset on v2:172.21.15.142:6838/3645998616 2023-09-13T03:29:01.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.341+0000 7f55a9e6bb80 1 -- 172.21.15.64:0/2569374682 shutdown_connections 2023-09-13T03:29:01.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.341+0000 7f55a9e6bb80 1 -- 172.21.15.64:0/2569374682 wait complete. 2023-09-13T03:29:01.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.341+0000 7f55a9e6bb80 20 client.27377 trim_cache size 0 max 16384 2023-09-13T03:29:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2023-09-13T03:29:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2023-09-13T03:29:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2023-09-13T03:29:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2023-09-13T03:29:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.350+0000 7f55965ff700 1 -- 172.21.15.64:0/297502517 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f55780a08b0 msgr2=0x7f55780a2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.350+0000 7f55965ff700 1 --2- 172.21.15.64:0/297502517 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f55780a08b0 0x7f55780a2d70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x55e77ae9d060 tx=0x7f5580004750 comp rx=0 tx=0).stop 2023-09-13T03:29:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.350+0000 7f55965ff700 1 -- 172.21.15.64:0/297502517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f559010afb0 msgr2=0x7f5590117a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.350+0000 7f55965ff700 1 --2- 172.21.15.64:0/297502517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f559010afb0 0x7f5590117a70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f558000b560 tx=0x7f558000b870 comp rx=0 tx=0).stop 2023-09-13T03:29:01.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.353+0000 7f55965ff700 1 -- 172.21.15.64:0/297502517 shutdown_connections 2023-09-13T03:29:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.353+0000 7f55965ff700 1 --2- 172.21.15.64:0/297502517 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f55780a08b0 0x7f55780a2d70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.353+0000 7f55965ff700 1 --2- 172.21.15.64:0/297502517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f559010afb0 0x7f5590117a70 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.353+0000 7f55965ff700 1 --2- 172.21.15.64:0/297502517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5590111070 0x7f5590114e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.353+0000 7f55965ff700 1 --2- 172.21.15.64:0/297502517 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f559010b970 0x7f55901148c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.353+0000 7f55965ff700 1 -- 172.21.15.64:0/297502517 >> 172.21.15.64:0/297502517 conn(0x7f559006b3d0 msgr2=0x7f55901097d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.369+0000 7f55965ff700 1 -- 172.21.15.64:0/297502517 shutdown_connections 2023-09-13T03:29:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2023-09-13T03:29:01.369+0000 7f55965ff700 1 -- 172.21.15.64:0/297502517 wait complete. 2023-09-13T03:29:02.313 INFO:tasks.workunit.client.0.smithi064.stdout: 10 155672 43.49 MB/sec execute 66 sec latency 600.378 ms 2023-09-13T03:29:03.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:02 smithi170 ceph-mon[134328]: pgmap v497: 97 pgs: 97 active+clean; 1.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 18 MiB/s wr, 1.24k op/s 2023-09-13T03:29:03.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:02 smithi064 ceph-mon[97675]: pgmap v497: 97 pgs: 97 active+clean; 1.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 18 MiB/s wr, 1.24k op/s 2023-09-13T03:29:03.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:02 smithi142 ceph-mon[134082]: pgmap v497: 97 pgs: 97 active+clean; 1.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 18 MiB/s wr, 1.24k op/s 2023-09-13T03:29:03.313 INFO:tasks.workunit.client.0.smithi064.stdout: 10 156513 43.68 MB/sec execute 67 sec latency 158.586 ms 2023-09-13T03:29:04.313 INFO:tasks.workunit.client.0.smithi064.stdout: 10 157175 43.52 MB/sec execute 68 sec latency 104.072 ms 2023-09-13T03:29:05.075 INFO:tasks.workunit.client.0.smithi064.stdout:[175347] write failed on handle 13903 (Resource temporarily unavailable) 2023-09-13T03:29:05.076 INFO:tasks.workunit.client.0.smithi064.stdout:Child failed with status 1 2023-09-13T03:29:05.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-13T03:29:05.112 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2023-09-13T03:29:05.112 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-09-13T03:29:05.162 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:04 smithi170 ceph-mon[134328]: pgmap v498: 97 pgs: 97 active+clean; 1.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 12 MiB/s wr, 802 op/s 2023-09-13T03:29:05.163 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:04 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:29:05.171 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:04 smithi064 ceph-mon[97675]: pgmap v498: 97 pgs: 97 active+clean; 1.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 12 MiB/s wr, 802 op/s 2023-09-13T03:29:05.171 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:04 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:29:05.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:04 smithi142 ceph-mon[134082]: pgmap v498: 97 pgs: 97 active+clean; 1.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 12 MiB/s wr, 802 op/s 2023-09-13T03:29:05.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:04 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:29:06.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:05 smithi170 ceph-mon[134328]: osdmap e105: 12 total, 12 up, 12 in 2023-09-13T03:29:06.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:05 smithi064 ceph-mon[97675]: osdmap e105: 12 total, 12 up, 12 in 2023-09-13T03:29:06.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:05 smithi142 ceph-mon[134082]: osdmap e105: 12 total, 12 up, 12 in 2023-09-13T03:29:06.683 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_batrick_ceph_3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_batrick_ceph_3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_batrick_ceph_3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi064 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2023-09-13T03:29:06.784 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=9b5adaaf424f46be8df3f3ab259b8cb6 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_batrick_ceph_3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_batrick_ceph_3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_batrick_ceph_3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi064 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2023-09-13T03:29:06.786 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2023-09-13T03:29:06.797 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi064.front.sepia.ceph.com 2023-09-13T03:29:06.811 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2023-09-13T03:29:06.826 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2023-09-13T03:29:06.827 INFO:teuthology.task.exec:Executing custom commands... 2023-09-13T03:29:06.827 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi064.front.sepia.ceph.com 2023-09-13T03:29:06.828 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2023-09-13T03:29:06.926 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2023-09-13T03:29:06.987 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:06 smithi064 ceph-mon[97675]: pgmap v500: 97 pgs: 9 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 73 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 15 MiB/s wr, 992 op/s 2023-09-13T03:29:07.003 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-09-13T03:29:07.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:06 smithi170 ceph-mon[134328]: pgmap v500: 97 pgs: 9 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 73 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 15 MiB/s wr, 992 op/s 2023-09-13T03:29:07.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.225+0000 7fcf993ff700 1 -- 172.21.15.64:0/1072968400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf94100f70 msgr2=0x7fcf9410d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:07.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.225+0000 7fcf993ff700 1 --2- 172.21.15.64:0/1072968400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf94100f70 0x7fcf9410d430 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c009f90 tx=0x7fcf8c00b330 comp rx=0 tx=0).stop 2023-09-13T03:29:07.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.225+0000 7fcf993ff700 1 -- 172.21.15.64:0/1072968400 shutdown_connections 2023-09-13T03:29:07.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.225+0000 7fcf993ff700 1 --2- 172.21.15.64:0/1072968400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf94106670 0x7fcf94106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:07.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.225+0000 7fcf993ff700 1 --2- 172.21.15.64:0/1072968400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf94100f70 0x7fcf9410d430 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:07.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.225+0000 7fcf993ff700 1 --2- 172.21.15.64:0/1072968400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf941005b0 0x7fcf94100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:07.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.225+0000 7fcf993ff700 1 -- 172.21.15.64:0/1072968400 >> 172.21.15.64:0/1072968400 conn(0x7fcf940fc060 msgr2=0x7fcf940fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:07.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.225+0000 7fcf993ff700 1 -- 172.21.15.64:0/1072968400 shutdown_connections 2023-09-13T03:29:07.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.225+0000 7fcf993ff700 1 -- 172.21.15.64:0/1072968400 wait complete. 2023-09-13T03:29:07.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.226+0000 7fcf993ff700 1 Processor -- start 2023-09-13T03:29:07.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.226+0000 7fcf993ff700 1 -- start start 2023-09-13T03:29:07.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.227+0000 7fcf993ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf941005b0 0x7fcf9419c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:07.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.227+0000 7fcf993ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf94100f70 0x7fcf9419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:07.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.227+0000 7fcf993ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf94106670 0x7fcf94196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:07.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.227+0000 7fcf993ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf94196cb0 con 0x7fcf94106670 2023-09-13T03:29:07.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.227+0000 7fcf993ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf94196e20 con 0x7fcf941005b0 2023-09-13T03:29:07.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.227+0000 7fcf993ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf94196fe0 con 0x7fcf94100f70 2023-09-13T03:29:07.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.227+0000 7fcf927fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf94100f70 0x7fcf9419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:07.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.227+0000 7fcf927fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf94100f70 0x7fcf9419cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:35634/0 (socket says 172.21.15.64:35634) 2023-09-13T03:29:07.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.227+0000 7fcf927fc700 1 -- 172.21.15.64:0/66649450 learned_addr learned my addr 172.21.15.64:0/66649450 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:29:07.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.227+0000 7fcf937fe700 1 --2- 172.21.15.64:0/66649450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf94106670 0x7fcf94196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:07.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.227+0000 7fcf92ffd700 1 --2- 172.21.15.64:0/66649450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf941005b0 0x7fcf9419c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:07.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.227+0000 7fcf927fc700 1 -- 172.21.15.64:0/66649450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf941005b0 msgr2=0x7fcf9419c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:07.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.227+0000 7fcf927fc700 1 --2- 172.21.15.64:0/66649450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf941005b0 0x7fcf9419c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:07.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.227+0000 7fcf927fc700 1 -- 172.21.15.64:0/66649450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf94106670 msgr2=0x7fcf94196640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:07.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.227+0000 7fcf927fc700 1 --2- 172.21.15.64:0/66649450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf94106670 0x7fcf94196640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:07.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.228+0000 7fcf927fc700 1 -- 172.21.15.64:0/66649450 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf840095f0 con 0x7fcf94100f70 2023-09-13T03:29:07.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.228+0000 7fcf927fc700 1 --2- 172.21.15.64:0/66649450 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf94100f70 0x7fcf9419cd20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fcf8800e9a0 tx=0x7fcf8800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:07.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.228+0000 7fcf7bfff700 1 -- 172.21.15.64:0/66649450 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf88009790 con 0x7fcf94100f70 2023-09-13T03:29:07.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.228+0000 7fcf7bfff700 1 -- 172.21.15.64:0/66649450 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf88010430 con 0x7fcf94100f70 2023-09-13T03:29:07.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.228+0000 7fcf993ff700 1 -- 172.21.15.64:0/66649450 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf8c009c40 con 0x7fcf94100f70 2023-09-13T03:29:07.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.228+0000 7fcf7bfff700 1 -- 172.21.15.64:0/66649450 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf88018680 con 0x7fcf94100f70 2023-09-13T03:29:07.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.228+0000 7fcf993ff700 1 -- 172.21.15.64:0/66649450 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf941a35f0 con 0x7fcf94100f70 2023-09-13T03:29:07.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.229+0000 7fcf993ff700 1 -- 172.21.15.64:0/66649450 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf940ffcd0 con 0x7fcf94100f70 2023-09-13T03:29:07.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.231+0000 7fcf7bfff700 1 -- 172.21.15.64:0/66649450 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fcf880105a0 con 0x7fcf94100f70 2023-09-13T03:29:07.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.232+0000 7fcf7bfff700 1 --2- 172.21.15.64:0/66649450 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fcf7c0a07b0 0x7fcf7c0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:07.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.233+0000 7fcf7bfff700 1 -- 172.21.15.64:0/66649450 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8528+0+0 (secure 0 0 0) 0x7fcf88014070 con 0x7fcf94100f70 2023-09-13T03:29:07.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.233+0000 7fcf92ffd700 1 --2- 172.21.15.64:0/66649450 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fcf7c0a07b0 0x7fcf7c0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:07.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.233+0000 7fcf92ffd700 1 --2- 172.21.15.64:0/66649450 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fcf7c0a07b0 0x7fcf7c0a2c70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcf84005fd0 tx=0x7fcf84009310 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:07.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.235+0000 7fcf7bfff700 1 -- 172.21.15.64:0/66649450 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fcf88095c20 con 0x7fcf94100f70 2023-09-13T03:29:07.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:06 smithi142 ceph-mon[134082]: pgmap v500: 97 pgs: 9 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 73 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 15 MiB/s wr, 992 op/s 2023-09-13T03:29:07.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.509+0000 7fcf993ff700 1 -- 172.21.15.64:0/66649450 --> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7fcf94110590 con 0x7fcf7c0a07b0 2023-09-13T03:29:07.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.521+0000 7fcf7bfff700 1 -- 172.21.15.64:0/66649450 <== mgr.15507 v2:172.21.15.64:6800/3860981465 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fcf94110590 con 0x7fcf7c0a07b0 2023-09-13T03:29:07.522 INFO:teuthology.orchestra.run.smithi064.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1M", "retention": {"h": 3, "M": 6}, "start": "2023-09-13T00:00:00", "created": "2023-09-13T03:13:10", "first": "2023-09-13T03:14:00", "last": "2023-09-13T03:29:00", "last_pruned": "2023-09-13T03:29:00", "created_count": 16, "pruned_count": 10, "active": true} 2023-09-13T03:29:07.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.524+0000 7fcf993ff700 1 -- 172.21.15.64:0/66649450 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fcf7c0a07b0 msgr2=0x7fcf7c0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:07.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.524+0000 7fcf993ff700 1 --2- 172.21.15.64:0/66649450 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fcf7c0a07b0 0x7fcf7c0a2c70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcf84005fd0 tx=0x7fcf84009310 comp rx=0 tx=0).stop 2023-09-13T03:29:07.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.524+0000 7fcf993ff700 1 -- 172.21.15.64:0/66649450 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf94100f70 msgr2=0x7fcf9419cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:07.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.524+0000 7fcf993ff700 1 --2- 172.21.15.64:0/66649450 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf94100f70 0x7fcf9419cd20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fcf8800e9a0 tx=0x7fcf8800ecb0 comp rx=0 tx=0).stop 2023-09-13T03:29:07.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.524+0000 7fcf993ff700 1 -- 172.21.15.64:0/66649450 shutdown_connections 2023-09-13T03:29:07.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.524+0000 7fcf993ff700 1 --2- 172.21.15.64:0/66649450 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fcf7c0a07b0 0x7fcf7c0a2c70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:07.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.524+0000 7fcf993ff700 1 --2- 172.21.15.64:0/66649450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf941005b0 0x7fcf9419c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:07.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.524+0000 7fcf993ff700 1 --2- 172.21.15.64:0/66649450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf94106670 0x7fcf94196640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:07.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.524+0000 7fcf993ff700 1 --2- 172.21.15.64:0/66649450 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf94100f70 0x7fcf9419cd20 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:07.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.524+0000 7fcf993ff700 1 -- 172.21.15.64:0/66649450 >> 172.21.15.64:0/66649450 conn(0x7fcf940fc060 msgr2=0x7fcf940fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:07.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.524+0000 7fcf993ff700 1 -- 172.21.15.64:0/66649450 shutdown_connections 2023-09-13T03:29:07.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.525+0000 7fcf993ff700 1 -- 172.21.15.64:0/66649450 wait complete. 2023-09-13T03:29:07.550 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-09-13T03:29:07.760 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.759+0000 7f021adff700 1 -- 172.21.15.64:0/1172312161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0214100f60 msgr2=0x7f021410d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:07.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.759+0000 7f021adff700 1 --2- 172.21.15.64:0/1172312161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0214100f60 0x7f021410d420 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f020c009f90 tx=0x7f020c00b330 comp rx=0 tx=0).stop 2023-09-13T03:29:07.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.760+0000 7f021adff700 1 -- 172.21.15.64:0/1172312161 shutdown_connections 2023-09-13T03:29:07.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.760+0000 7f021adff700 1 --2- 172.21.15.64:0/1172312161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0214100f60 0x7f021410d420 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:07.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.760+0000 7f021adff700 1 --2- 172.21.15.64:0/1172312161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0214106660 0x7f0214106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:07.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.760+0000 7f021adff700 1 --2- 172.21.15.64:0/1172312161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02141005a0 0x7f0214100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:07.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.760+0000 7f021adff700 1 -- 172.21.15.64:0/1172312161 >> 172.21.15.64:0/1172312161 conn(0x7f02140fc050 msgr2=0x7f02140fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:07.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.760+0000 7f021adff700 1 -- 172.21.15.64:0/1172312161 shutdown_connections 2023-09-13T03:29:07.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.760+0000 7f021adff700 1 -- 172.21.15.64:0/1172312161 wait complete. 2023-09-13T03:29:07.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.761+0000 7f021adff700 1 Processor -- start 2023-09-13T03:29:07.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.761+0000 7f021adff700 1 -- start start 2023-09-13T03:29:07.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.761+0000 7f021adff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02141005a0 0x7f021419c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:07.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.761+0000 7f021adff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0214100f60 0x7f021419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:07.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.761+0000 7f021adff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0214106660 0x7f0214196610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:07.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.761+0000 7f021adff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0214196de0 con 0x7f0214106660 2023-09-13T03:29:07.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.761+0000 7f021adff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0214196f50 con 0x7f02141005a0 2023-09-13T03:29:07.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.761+0000 7f021adff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0214197110 con 0x7f0214100f60 2023-09-13T03:29:07.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.761+0000 7f0218b9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02141005a0 0x7f021419c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:07.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.761+0000 7f021939c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0214106660 0x7f0214196610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:07.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.761+0000 7f0218b9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02141005a0 0x7f021419c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:54492/0 (socket says 172.21.15.64:54492) 2023-09-13T03:29:07.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.761+0000 7f021939c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0214106660 0x7f0214196610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48842/0 (socket says 172.21.15.64:48842) 2023-09-13T03:29:07.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.761+0000 7f0218b9b700 1 -- 172.21.15.64:0/3418640928 learned_addr learned my addr 172.21.15.64:0/3418640928 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:29:07.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.762+0000 7f0213fff700 1 --2- 172.21.15.64:0/3418640928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0214100f60 0x7f021419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:07.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.762+0000 7f021939c700 1 -- 172.21.15.64:0/3418640928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0214100f60 msgr2=0x7f021419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:07.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.762+0000 7f021939c700 1 --2- 172.21.15.64:0/3418640928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0214100f60 0x7f021419cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:07.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.762+0000 7f021939c700 1 -- 172.21.15.64:0/3418640928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02141005a0 msgr2=0x7f021419c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:07.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.762+0000 7f021939c700 1 --2- 172.21.15.64:0/3418640928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02141005a0 0x7f021419c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:07.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.762+0000 7f021939c700 1 -- 172.21.15.64:0/3418640928 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02000095f0 con 0x7f0214106660 2023-09-13T03:29:07.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.762+0000 7f0218b9b700 1 --2- 172.21.15.64:0/3418640928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02141005a0 0x7f021419c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:29:07.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.762+0000 7f021939c700 1 --2- 172.21.15.64:0/3418640928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0214106660 0x7f0214196610 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f020c00b6c0 tx=0x7f020c004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:07.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.762+0000 7f0211ffb700 1 -- 172.21.15.64:0/3418640928 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020c00e040 con 0x7f0214106660 2023-09-13T03:29:07.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.762+0000 7f021adff700 1 -- 172.21.15.64:0/3418640928 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f020c009c40 con 0x7f0214106660 2023-09-13T03:29:07.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.763+0000 7f021adff700 1 -- 172.21.15.64:0/3418640928 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02141a35e0 con 0x7f0214106660 2023-09-13T03:29:07.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.763+0000 7f0211ffb700 1 -- 172.21.15.64:0/3418640928 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f020c008e80 con 0x7f0214106660 2023-09-13T03:29:07.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.763+0000 7f0211ffb700 1 -- 172.21.15.64:0/3418640928 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020c005040 con 0x7f0214106660 2023-09-13T03:29:07.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.764+0000 7f021adff700 1 -- 172.21.15.64:0/3418640928 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02140ffcc0 con 0x7f0214106660 2023-09-13T03:29:07.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.766+0000 7f0211ffb700 1 -- 172.21.15.64:0/3418640928 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f020c01d470 con 0x7f0214106660 2023-09-13T03:29:07.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.768+0000 7f0211ffb700 1 --2- 172.21.15.64:0/3418640928 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f02040a0830 0x7f02040a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:07.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.768+0000 7f0211ffb700 1 -- 172.21.15.64:0/3418640928 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f020c0cd530 con 0x7f0214106660 2023-09-13T03:29:07.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.768+0000 7f0218b9b700 1 --2- 172.21.15.64:0/3418640928 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f02040a0830 0x7f02040a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:07.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.769+0000 7f0218b9b700 1 --2- 172.21.15.64:0/3418640928 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f02040a0830 0x7f02040a2cf0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0208011550 tx=0x7f0208009260 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:07.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:07.771+0000 7f0211ffb700 1 -- 172.21.15.64:0/3418640928 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f020c096d70 con 0x7f0214106660 2023-09-13T03:29:08.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.019+0000 7f021adff700 1 -- 172.21.15.64:0/3418640928 --> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f0214197e60 con 0x7f02040a0830 2023-09-13T03:29:08.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.029+0000 7f0211ffb700 1 -- 172.21.15.64:0/3418640928 <== mgr.15507 v2:172.21.15.64:6800/3860981465 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f0214197e60 con 0x7f02040a0830 2023-09-13T03:29:08.029 INFO:teuthology.orchestra.run.smithi064.stdout:/ 1M 6M3h 2023-09-13T03:29:08.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.032+0000 7f021adff700 1 -- 172.21.15.64:0/3418640928 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f02040a0830 msgr2=0x7f02040a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:08.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.032+0000 7f021adff700 1 --2- 172.21.15.64:0/3418640928 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f02040a0830 0x7f02040a2cf0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0208011550 tx=0x7f0208009260 comp rx=0 tx=0).stop 2023-09-13T03:29:08.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.032+0000 7f021adff700 1 -- 172.21.15.64:0/3418640928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0214106660 msgr2=0x7f0214196610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:08.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.032+0000 7f021adff700 1 --2- 172.21.15.64:0/3418640928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0214106660 0x7f0214196610 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f020c00b6c0 tx=0x7f020c004ce0 comp rx=0 tx=0).stop 2023-09-13T03:29:08.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.032+0000 7f021adff700 1 -- 172.21.15.64:0/3418640928 shutdown_connections 2023-09-13T03:29:08.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.032+0000 7f021adff700 1 --2- 172.21.15.64:0/3418640928 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f02040a0830 0x7f02040a2cf0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:08.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.032+0000 7f021adff700 1 --2- 172.21.15.64:0/3418640928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02141005a0 0x7f021419c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:08.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.032+0000 7f021adff700 1 --2- 172.21.15.64:0/3418640928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0214106660 0x7f0214196610 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:08.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.032+0000 7f021adff700 1 --2- 172.21.15.64:0/3418640928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0214100f60 0x7f021419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:08.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.032+0000 7f021adff700 1 -- 172.21.15.64:0/3418640928 >> 172.21.15.64:0/3418640928 conn(0x7f02140fc050 msgr2=0x7f02140fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:08.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.033+0000 7f021adff700 1 -- 172.21.15.64:0/3418640928 shutdown_connections 2023-09-13T03:29:08.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.033+0000 7f021adff700 1 -- 172.21.15.64:0/3418640928 wait complete. 2023-09-13T03:29:08.055 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-09-13T03:29:08.066 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-09-13T03:29:08.067 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_batrick_ceph_3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/github.com_batrick_ceph_3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2023-09-13T03:29:08.071 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-09-13T03:29:08.094 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-13T03:29:08.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.317+0000 7fb1027ff700 1 -- 172.21.15.64:0/2649817612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0fc100590 msgr2=0x7fb0fc100a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:08.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.317+0000 7fb1027ff700 1 --2- 172.21.15.64:0/2649817612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0fc100590 0x7fb0fc100a10 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb0f00099c0 tx=0x7fb0f0009cd0 comp rx=0 tx=0).stop 2023-09-13T03:29:08.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.317+0000 7fb1027ff700 1 -- 172.21.15.64:0/2649817612 shutdown_connections 2023-09-13T03:29:08.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.317+0000 7fb1027ff700 1 --2- 172.21.15.64:0/2649817612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0fc100f50 0x7fb0fc10d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:08.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.317+0000 7fb1027ff700 1 --2- 172.21.15.64:0/2649817612 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0fc106650 0x7fb0fc106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:08.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.317+0000 7fb1027ff700 1 --2- 172.21.15.64:0/2649817612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0fc100590 0x7fb0fc100a10 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:08.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.317+0000 7fb1027ff700 1 -- 172.21.15.64:0/2649817612 >> 172.21.15.64:0/2649817612 conn(0x7fb0fc0fc020 msgr2=0x7fb0fc0fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:08.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.317+0000 7fb1027ff700 1 -- 172.21.15.64:0/2649817612 shutdown_connections 2023-09-13T03:29:08.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.317+0000 7fb1027ff700 1 -- 172.21.15.64:0/2649817612 wait complete. 2023-09-13T03:29:08.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.318+0000 7fb1027ff700 1 Processor -- start 2023-09-13T03:29:08.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.318+0000 7fb1027ff700 1 -- start start 2023-09-13T03:29:08.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.318+0000 7fb1027ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0fc100590 0x7fb0fc19e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:08.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.318+0000 7fb1027ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0fc100f50 0x7fb0fc19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:08.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.318+0000 7fb1027ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0fc106650 0x7fb0fc198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:08.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.318+0000 7fb1027ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc198c90 con 0x7fb0fc106650 2023-09-13T03:29:08.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.318+0000 7fb1027ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc198e00 con 0x7fb0fc100590 2023-09-13T03:29:08.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.318+0000 7fb1027ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc198fc0 con 0x7fb0fc100f50 2023-09-13T03:29:08.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.319+0000 7fb0fbfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0fc100590 0x7fb0fc19e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:08.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.319+0000 7fb0fbfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0fc100590 0x7fb0fc19e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:54514/0 (socket says 172.21.15.64:54514) 2023-09-13T03:29:08.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.319+0000 7fb0fbfff700 1 -- 172.21.15.64:0/2494678073 learned_addr learned my addr 172.21.15.64:0/2494678073 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:29:08.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.319+0000 7fb0fb7fe700 1 --2- 172.21.15.64:0/2494678073 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0fc100f50 0x7fb0fc19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:08.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.319+0000 7fb100d9c700 1 --2- 172.21.15.64:0/2494678073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0fc106650 0x7fb0fc198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:08.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.319+0000 7fb0fbfff700 1 -- 172.21.15.64:0/2494678073 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0fc100f50 msgr2=0x7fb0fc19eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:08.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.319+0000 7fb0fbfff700 1 --2- 172.21.15.64:0/2494678073 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0fc100f50 0x7fb0fc19eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:08.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.319+0000 7fb0fbfff700 1 -- 172.21.15.64:0/2494678073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0fc106650 msgr2=0x7fb0fc198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:08.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.319+0000 7fb0fbfff700 1 --2- 172.21.15.64:0/2494678073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0fc106650 0x7fb0fc198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:08.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.319+0000 7fb0fbfff700 1 -- 172.21.15.64:0/2494678073 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f00095f0 con 0x7fb0fc100590 2023-09-13T03:29:08.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.319+0000 7fb0fbfff700 1 --2- 172.21.15.64:0/2494678073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0fc100590 0x7fb0fc19e5b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec00e9a0 tx=0x7fb0ec00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:08.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.320+0000 7fb0f97fa700 1 -- 172.21.15.64:0/2494678073 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec009790 con 0x7fb0fc100590 2023-09-13T03:29:08.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.320+0000 7fb0f97fa700 1 -- 172.21.15.64:0/2494678073 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0ec010430 con 0x7fb0fc100590 2023-09-13T03:29:08.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.320+0000 7fb0f97fa700 1 -- 172.21.15.64:0/2494678073 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec018680 con 0x7fb0fc100590 2023-09-13T03:29:08.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.320+0000 7fb1027ff700 1 -- 172.21.15.64:0/2494678073 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0fc1992a0 con 0x7fb0fc100590 2023-09-13T03:29:08.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.320+0000 7fb1027ff700 1 -- 172.21.15.64:0/2494678073 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0fc1a55a0 con 0x7fb0fc100590 2023-09-13T03:29:08.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.321+0000 7fb1027ff700 1 -- 172.21.15.64:0/2494678073 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0fc0673f0 con 0x7fb0fc100590 2023-09-13T03:29:08.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.323+0000 7fb0f97fa700 1 -- 172.21.15.64:0/2494678073 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fb0ec0105a0 con 0x7fb0fc100590 2023-09-13T03:29:08.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.324+0000 7fb0f97fa700 1 --2- 172.21.15.64:0/2494678073 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fb0e40a0870 0x7fb0e40a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:08.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.324+0000 7fb0f97fa700 1 -- 172.21.15.64:0/2494678073 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8528+0+0 (secure 0 0 0) 0x7fb0ec014070 con 0x7fb0fc100590 2023-09-13T03:29:08.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.324+0000 7fb0fb7fe700 1 --2- 172.21.15.64:0/2494678073 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fb0e40a0870 0x7fb0e40a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:08.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.325+0000 7fb0fb7fe700 1 --2- 172.21.15.64:0/2494678073 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fb0e40a0870 0x7fb0e40a2d30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0009990 tx=0x7fb0f0005fb0 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:08.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.327+0000 7fb0f97fa700 1 -- 172.21.15.64:0/2494678073 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb0ec095c20 con 0x7fb0fc100590 2023-09-13T03:29:08.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.602+0000 7fb1027ff700 1 -- 172.21.15.64:0/2494678073 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0fc199b30 con 0x7fb0fc100590 2023-09-13T03:29:08.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.604+0000 7fb0f97fa700 1 -- 172.21.15.64:0/2494678073 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+23010 (secure 0 0 0) 0x7fb0ec099240 con 0x7fb0fc100590 2023-09-13T03:29:08.605 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:29:08.605 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":105,"fsid":"f8dbced4-51e1-11ee-9ab7-7b867c8bd7da","created":"2023-09-13T03:03:24.167202+0000","modified":"2023-09-13T03:29:04.878346+0000","last_up_change":"2023-09-13T03:10:22.503529+0000","last_in_change":"2023-09-13T03:10:05.678720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T03:06:58.987479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-13T03:11:08.589202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-13T03:11:09.924659+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3997c114-241b-48c4-8feb-653c99209470","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6803","nonce":3940892135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6805","nonce":3940892135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6809","nonce":3940892135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6807","nonce":3940892135}]},"public_addr":"172.21.15.64:6803/3940892135","cluster_addr":"172.21.15.64:6805/3940892135","heartbeat_back_addr":"172.21.15.64:6809/3940892135","heartbeat_front_addr":"172.21.15.64:6807/3940892135","state":["exists","up"]},{"osd":1,"uuid":"94cfae85-322e-4ed2-a6e6-e40349bb14bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6801","nonce":4188149333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6803","nonce":4188149333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6807","nonce":4188149333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6805","nonce":4188149333}]},"public_addr":"172.21.15.142:6801/4188149333","cluster_addr":"172.21.15.142:6803/4188149333","heartbeat_back_addr":"172.21.15.142:6807/4188149333","heartbeat_front_addr":"172.21.15.142:6805/4188149333","state":["exists","up"]},{"osd":2,"uuid":"b3aae17e-96db-4008-8c27-f733c46b30ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6801","nonce":2406669338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6803","nonce":2406669338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6807","nonce":2406669338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6805","nonce":2406669338}]},"public_addr":"172.21.15.170:6801/2406669338","cluster_addr":"172.21.15.170:6803/2406669338","heartbeat_back_addr":"172.21.15.170:6807/2406669338","heartbeat_front_addr":"172.21.15.170:6805/2406669338","state":["exists","up"]},{"osd":3,"uuid":"7fa8d8cb-2ab7-48a7-8b49-5e2903d6779f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6811","nonce":3558318707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6813","nonce":3558318707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6817","nonce":3558318707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6815","nonce":3558318707}]},"public_addr":"172.21.15.64:6811/3558318707","cluster_addr":"172.21.15.64:6813/3558318707","heartbeat_back_addr":"172.21.15.64:6817/3558318707","heartbeat_front_addr":"172.21.15.64:6815/3558318707","state":["exists","up"]},{"osd":4,"uuid":"8c7a9c9f-f919-48d3-9db2-214a25bb5603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6809","nonce":1811142576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6811","nonce":1811142576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6815","nonce":1811142576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6813","nonce":1811142576}]},"public_addr":"172.21.15.142:6809/1811142576","cluster_addr":"172.21.15.142:6811/1811142576","heartbeat_back_addr":"172.21.15.142:6815/1811142576","heartbeat_front_addr":"172.21.15.142:6813/1811142576","state":["exists","up"]},{"osd":5,"uuid":"0c6101c6-2065-4022-a751-d3f216d7f478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6809","nonce":2903841453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6811","nonce":2903841453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6815","nonce":2903841453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6813","nonce":2903841453}]},"public_addr":"172.21.15.170:6809/2903841453","cluster_addr":"172.21.15.170:6811/2903841453","heartbeat_back_addr":"172.21.15.170:6815/2903841453","heartbeat_front_addr":"172.21.15.170:6813/2903841453","state":["exists","up"]},{"osd":6,"uuid":"5cf3e52e-d155-41b1-9194-db047d26a953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6819","nonce":1436584490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6821","nonce":1436584490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6825","nonce":1436584490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6823","nonce":1436584490}]},"public_addr":"172.21.15.64:6819/1436584490","cluster_addr":"172.21.15.64:6821/1436584490","heartbeat_back_addr":"172.21.15.64:6825/1436584490","heartbeat_front_addr":"172.21.15.64:6823/1436584490","state":["exists","up"]},{"osd":7,"uuid":"e97323e3-badd-41a7-af9e-e78a81a64046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6817","nonce":3726793534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6819","nonce":3726793534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6823","nonce":3726793534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6821","nonce":3726793534}]},"public_addr":"172.21.15.142:6817/3726793534","cluster_addr":"172.21.15.142:6819/3726793534","heartbeat_back_addr":"172.21.15.142:6823/3726793534","heartbeat_front_addr":"172.21.15.142:6821/3726793534","state":["exists","up"]},{"osd":8,"uuid":"83d2835c-ed26-493d-9222-c2894ec0e071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6817","nonce":4272513873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6819","nonce":4272513873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6823","nonce":4272513873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6821","nonce":4272513873}]},"public_addr":"172.21.15.170:6817/4272513873","cluster_addr":"172.21.15.170:6819/4272513873","heartbeat_back_addr":"172.21.15.170:6823/4272513873","heartbeat_front_addr":"172.21.15.170:6821/4272513873","state":["exists","up"]},{"osd":9,"uuid":"cb936bdd-1bec-4236-aa2e-a24adcac11e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6827","nonce":2641175211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6829","nonce":2641175211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6833","nonce":2641175211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6831","nonce":2641175211}]},"public_addr":"172.21.15.64:6827/2641175211","cluster_addr":"172.21.15.64:6829/2641175211","heartbeat_back_addr":"172.21.15.64:6833/2641175211","heartbeat_front_addr":"172.21.15.64:6831/2641175211","state":["exists","up"]},{"osd":10,"uuid":"85512a1c-6171-43d4-9c15-c588a8b71be0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6825","nonce":936499217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6827","nonce":936499217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6831","nonce":936499217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6829","nonce":936499217}]},"public_addr":"172.21.15.142:6825/936499217","cluster_addr":"172.21.15.142:6827/936499217","heartbeat_back_addr":"172.21.15.142:6831/936499217","heartbeat_front_addr":"172.21.15.142:6829/936499217","state":["exists","up"]},{"osd":11,"uuid":"9b8417dc-5d6c-4917-94a9-b032c5bc33d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6825","nonce":3605017060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6827","nonce":3605017060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6831","nonce":3605017060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6829","nonce":3605017060}]},"public_addr":"172.21.15.170:6825/3605017060","cluster_addr":"172.21.15.170:6827/3605017060","heartbeat_back_addr":"172.21.15.170:6831/3605017060","heartbeat_front_addr":"172.21.15.170:6829/3605017060","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:14.229044+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:36.147777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:56.655298+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:07:23.107741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:07:44.663770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:05.577432+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:31.168031+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:52.791055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:09:13.658546+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:09:39.150360+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:10:00.601735+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:10:21.407555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6801/873679308":"2023-09-14T03:13:09.492864+0000","172.21.15.64:0/1170163026":"2023-09-14T03:13:09.492864+0000","172.21.15.64:0/3771142265":"2023-09-14T03:13:09.492864+0000","172.21.15.64:0/4083942699":"2023-09-14T03:13:09.492864+0000","172.21.15.64:6800/2955227347":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/1373223118":"2023-09-14T03:04:22.355817+0000","172.21.15.64:6800/873679308":"2023-09-14T03:13:09.492864+0000","172.21.15.64:0/791645251":"2023-09-14T03:13:09.492864+0000","172.21.15.64:0/271194613":"2023-09-14T03:04:22.355817+0000","172.21.15.64:6801/3998091347":"2023-09-14T03:03:56.442541+0000","172.21.15.64:6800/3998091347":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/920818572":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/735789005":"2023-09-14T03:03:56.442541+0000","172.21.15.64:6801/2955227347":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/1975923178":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/1500151178":"2023-09-14T03:04:22.355817+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T03:29:08.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.606+0000 7fb1027ff700 1 -- 172.21.15.64:0/2494678073 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fb0e40a0870 msgr2=0x7fb0e40a2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:08.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.606+0000 7fb1027ff700 1 --2- 172.21.15.64:0/2494678073 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fb0e40a0870 0x7fb0e40a2d30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0009990 tx=0x7fb0f0005fb0 comp rx=0 tx=0).stop 2023-09-13T03:29:08.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.606+0000 7fb1027ff700 1 -- 172.21.15.64:0/2494678073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0fc100590 msgr2=0x7fb0fc19e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:08.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.606+0000 7fb1027ff700 1 --2- 172.21.15.64:0/2494678073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0fc100590 0x7fb0fc19e5b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec00e9a0 tx=0x7fb0ec00ecb0 comp rx=0 tx=0).stop 2023-09-13T03:29:08.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.606+0000 7fb1027ff700 1 -- 172.21.15.64:0/2494678073 shutdown_connections 2023-09-13T03:29:08.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.606+0000 7fb1027ff700 1 --2- 172.21.15.64:0/2494678073 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fb0e40a0870 0x7fb0e40a2d30 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:08.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.606+0000 7fb1027ff700 1 --2- 172.21.15.64:0/2494678073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0fc100590 0x7fb0fc19e5b0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:08.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.606+0000 7fb1027ff700 1 --2- 172.21.15.64:0/2494678073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0fc106650 0x7fb0fc198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:08.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.606+0000 7fb1027ff700 1 --2- 172.21.15.64:0/2494678073 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0fc100f50 0x7fb0fc19eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:08.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.606+0000 7fb1027ff700 1 -- 172.21.15.64:0/2494678073 >> 172.21.15.64:0/2494678073 conn(0x7fb0fc0fc020 msgr2=0x7fb0fc0fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:08.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.607+0000 7fb1027ff700 1 -- 172.21.15.64:0/2494678073 shutdown_connections 2023-09-13T03:29:08.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.607+0000 7fb1027ff700 1 -- 172.21.15.64:0/2494678073 wait complete. 2023-09-13T03:29:08.628 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-13T03:06:58.987479+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-13T03:11:08.589202+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-13T03:11:09.924659+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 105, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-13T03:29:08.629 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-13T03:29:08.839 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.837+0000 7fcea0dff700 1 -- 172.21.15.64:0/778275413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce9c100f50 msgr2=0x7fce9c10d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:08.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.837+0000 7fcea0dff700 1 --2- 172.21.15.64:0/778275413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce9c100f50 0x7fce9c10d410 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fce9400b0d0 tx=0x7fce9400b3e0 comp rx=0 tx=0).stop 2023-09-13T03:29:08.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.837+0000 7fcea0dff700 1 -- 172.21.15.64:0/778275413 shutdown_connections 2023-09-13T03:29:08.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.837+0000 7fcea0dff700 1 --2- 172.21.15.64:0/778275413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce9c100f50 0x7fce9c10d410 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:08.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.837+0000 7fcea0dff700 1 --2- 172.21.15.64:0/778275413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce9c106650 0x7fce9c106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:08.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.837+0000 7fcea0dff700 1 --2- 172.21.15.64:0/778275413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce9c100590 0x7fce9c100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:08.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.837+0000 7fcea0dff700 1 -- 172.21.15.64:0/778275413 >> 172.21.15.64:0/778275413 conn(0x7fce9c0fc060 msgr2=0x7fce9c0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:08.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.837+0000 7fcea0dff700 1 -- 172.21.15.64:0/778275413 shutdown_connections 2023-09-13T03:29:08.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.837+0000 7fcea0dff700 1 -- 172.21.15.64:0/778275413 wait complete. 2023-09-13T03:29:08.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.838+0000 7fcea0dff700 1 Processor -- start 2023-09-13T03:29:08.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.838+0000 7fcea0dff700 1 -- start start 2023-09-13T03:29:08.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.839+0000 7fcea0dff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce9c100590 0x7fce9c19e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:08.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.839+0000 7fcea0dff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce9c100f50 0x7fce9c19ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:08.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.839+0000 7fcea0dff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce9c106650 0x7fce9c198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:08.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.839+0000 7fcea0dff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9c198d80 con 0x7fce9c100f50 2023-09-13T03:29:08.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.839+0000 7fcea0dff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9c198ef0 con 0x7fce9c106650 2023-09-13T03:29:08.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.839+0000 7fcea0dff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9c1990b0 con 0x7fce9c100590 2023-09-13T03:29:08.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.839+0000 7fce93fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce9c100f50 0x7fce9c19ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:08.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.839+0000 7fce9ad9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce9c106650 0x7fce9c198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:08.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.839+0000 7fce9ad9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce9c106650 0x7fce9c198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:54528/0 (socket says 172.21.15.64:54528) 2023-09-13T03:29:08.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.839+0000 7fce9ad9d700 1 -- 172.21.15.64:0/1471298843 learned_addr learned my addr 172.21.15.64:0/1471298843 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:29:08.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.839+0000 7fce9a59c700 1 --2- 172.21.15.64:0/1471298843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce9c100590 0x7fce9c19e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:08.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.839+0000 7fce9ad9d700 1 -- 172.21.15.64:0/1471298843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce9c100590 msgr2=0x7fce9c19e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:08.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.839+0000 7fce9ad9d700 1 --2- 172.21.15.64:0/1471298843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce9c100590 0x7fce9c19e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:08.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.839+0000 7fce9ad9d700 1 -- 172.21.15.64:0/1471298843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce9c100f50 msgr2=0x7fce9c19ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:08.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.839+0000 7fce9ad9d700 1 --2- 172.21.15.64:0/1471298843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce9c100f50 0x7fce9c19ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:08.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.839+0000 7fce9ad9d700 1 -- 172.21.15.64:0/1471298843 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce94009d00 con 0x7fce9c106650 2023-09-13T03:29:08.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.839+0000 7fce9ad9d700 1 --2- 172.21.15.64:0/1471298843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce9c106650 0x7fce9c198710 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fce94007870 tx=0x7fce940078a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:08.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.840+0000 7fce937fe700 1 -- 172.21.15.64:0/1471298843 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9400e040 con 0x7fce9c106650 2023-09-13T03:29:08.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.840+0000 7fce937fe700 1 -- 172.21.15.64:0/1471298843 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce94005070 con 0x7fce9c106650 2023-09-13T03:29:08.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.840+0000 7fcea0dff700 1 -- 172.21.15.64:0/1471298843 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce9c199330 con 0x7fce9c106650 2023-09-13T03:29:08.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.840+0000 7fce937fe700 1 -- 172.21.15.64:0/1471298843 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9401b6c0 con 0x7fce9c106650 2023-09-13T03:29:08.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.840+0000 7fcea0dff700 1 -- 172.21.15.64:0/1471298843 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce9c1a5690 con 0x7fce9c106650 2023-09-13T03:29:08.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.841+0000 7fcea0dff700 1 -- 172.21.15.64:0/1471298843 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce9c067380 con 0x7fce9c106650 2023-09-13T03:29:08.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.843+0000 7fce937fe700 1 -- 172.21.15.64:0/1471298843 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fce940051e0 con 0x7fce9c106650 2023-09-13T03:29:08.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.845+0000 7fce937fe700 1 --2- 172.21.15.64:0/1471298843 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fce880a0830 0x7fce880a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:08.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.845+0000 7fce937fe700 1 -- 172.21.15.64:0/1471298843 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8528+0+0 (secure 0 0 0) 0x7fce940cd330 con 0x7fce9c106650 2023-09-13T03:29:08.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.845+0000 7fce9a59c700 1 --2- 172.21.15.64:0/1471298843 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fce880a0830 0x7fce880a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:08.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.845+0000 7fce9a59c700 1 --2- 172.21.15.64:0/1471298843 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fce880a0830 0x7fce880a2cf0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fce8c009bf0 tx=0x7fce8c009260 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:08.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:08.847+0000 7fce937fe700 1 -- 172.21.15.64:0/1471298843 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fce94096b70 con 0x7fce9c106650 2023-09-13T03:29:09.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.061+0000 7fcea0dff700 1 -- 172.21.15.64:0/1471298843 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fce9c110550 con 0x7fce9c106650 2023-09-13T03:29:09.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.061+0000 7fce937fe700 1 -- 172.21.15.64:0/1471298843 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) v1 ==== 93+0+10 (secure 0 0 0) 0x7fce9409a190 con 0x7fce9c106650 2023-09-13T03:29:09.063 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-09-13T03:29:09.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.064+0000 7fcea0dff700 1 -- 172.21.15.64:0/1471298843 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fce880a0830 msgr2=0x7fce880a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:09.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.064+0000 7fcea0dff700 1 --2- 172.21.15.64:0/1471298843 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fce880a0830 0x7fce880a2cf0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fce8c009bf0 tx=0x7fce8c009260 comp rx=0 tx=0).stop 2023-09-13T03:29:09.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.064+0000 7fcea0dff700 1 -- 172.21.15.64:0/1471298843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce9c106650 msgr2=0x7fce9c198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:09.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.064+0000 7fcea0dff700 1 --2- 172.21.15.64:0/1471298843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce9c106650 0x7fce9c198710 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fce94007870 tx=0x7fce940078a0 comp rx=0 tx=0).stop 2023-09-13T03:29:09.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.064+0000 7fcea0dff700 1 -- 172.21.15.64:0/1471298843 shutdown_connections 2023-09-13T03:29:09.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.064+0000 7fcea0dff700 1 --2- 172.21.15.64:0/1471298843 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fce880a0830 0x7fce880a2cf0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:09.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.064+0000 7fcea0dff700 1 --2- 172.21.15.64:0/1471298843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce9c106650 0x7fce9c198710 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:09.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.064+0000 7fcea0dff700 1 --2- 172.21.15.64:0/1471298843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce9c100590 0x7fce9c19e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:09.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.064+0000 7fcea0dff700 1 --2- 172.21.15.64:0/1471298843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce9c100f50 0x7fce9c19ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:09.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.064+0000 7fcea0dff700 1 -- 172.21.15.64:0/1471298843 >> 172.21.15.64:0/1471298843 conn(0x7fce9c0fc060 msgr2=0x7fce9c10b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:09.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.065+0000 7fcea0dff700 1 -- 172.21.15.64:0/1471298843 shutdown_connections 2023-09-13T03:29:09.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.065+0000 7fcea0dff700 1 -- 172.21.15.64:0/1471298843 wait complete. 2023-09-13T03:29:09.086 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-13T03:29:09.146 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:08 smithi064 ceph-mon[97675]: from='client.27346 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:29:09.147 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:08 smithi064 ceph-mon[97675]: pgmap v501: 97 pgs: 10 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 72 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 15 MiB/s wr, 1.05k op/s 2023-09-13T03:29:09.147 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:08 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2494678073' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:29:09.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:08 smithi170 ceph-mon[134328]: from='client.27346 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:29:09.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:08 smithi170 ceph-mon[134328]: pgmap v501: 97 pgs: 10 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 72 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 15 MiB/s wr, 1.05k op/s 2023-09-13T03:29:09.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:08 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/2494678073' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:29:09.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:08 smithi142 ceph-mon[134082]: from='client.27346 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:29:09.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:08 smithi142 ceph-mon[134082]: pgmap v501: 97 pgs: 10 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 72 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 15 MiB/s wr, 1.05k op/s 2023-09-13T03:29:09.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:08 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/2494678073' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:29:09.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.293+0000 7f0b82fff700 1 -- 172.21.15.64:0/2314722789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b7c073350 msgr2=0x7f0b7c10c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:09.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.293+0000 7f0b82fff700 1 --2- 172.21.15.64:0/2314722789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b7c073350 0x7f0b7c10c850 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0b68009990 tx=0x7f0b68009ca0 comp rx=0 tx=0).stop 2023-09-13T03:29:09.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.293+0000 7f0b82fff700 1 -- 172.21.15.64:0/2314722789 shutdown_connections 2023-09-13T03:29:09.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.293+0000 7f0b82fff700 1 --2- 172.21.15.64:0/2314722789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b7c10cd90 0x7f0b7c10f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:09.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.293+0000 7f0b82fff700 1 --2- 172.21.15.64:0/2314722789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b7c072a30 0x7f0b7c072e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:09.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.293+0000 7f0b82fff700 1 --2- 172.21.15.64:0/2314722789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b7c073350 0x7f0b7c10c850 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:09.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.293+0000 7f0b82fff700 1 -- 172.21.15.64:0/2314722789 >> 172.21.15.64:0/2314722789 conn(0x7f0b7c0fc080 msgr2=0x7f0b7c0fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:09.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.293+0000 7f0b82fff700 1 -- 172.21.15.64:0/2314722789 shutdown_connections 2023-09-13T03:29:09.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.293+0000 7f0b82fff700 1 -- 172.21.15.64:0/2314722789 wait complete. 2023-09-13T03:29:09.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.294+0000 7f0b82fff700 1 Processor -- start 2023-09-13T03:29:09.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.294+0000 7f0b82fff700 1 -- start start 2023-09-13T03:29:09.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.295+0000 7f0b82fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b7c072a30 0x7f0b7c19e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:09.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.295+0000 7f0b82fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b7c073350 0x7f0b7c19ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:09.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.295+0000 7f0b82fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b7c10cd90 0x7f0b7c1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:09.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.295+0000 7f0b82fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7c198d50 con 0x7f0b7c10cd90 2023-09-13T03:29:09.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.295+0000 7f0b82fff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7c198ec0 con 0x7f0b7c073350 2023-09-13T03:29:09.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.295+0000 7f0b82fff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7c199080 con 0x7f0b7c072a30 2023-09-13T03:29:09.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.295+0000 7f0b80d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b7c072a30 0x7f0b7c19e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:09.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.295+0000 7f0b7bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b7c073350 0x7f0b7c19ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:09.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.295+0000 7f0b80d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b7c072a30 0x7f0b7c19e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:35712/0 (socket says 172.21.15.64:35712) 2023-09-13T03:29:09.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.295+0000 7f0b80d9b700 1 -- 172.21.15.64:0/994010859 learned_addr learned my addr 172.21.15.64:0/994010859 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:29:09.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.295+0000 7f0b8159c700 1 --2- 172.21.15.64:0/994010859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b7c10cd90 0x7f0b7c1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:09.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.295+0000 7f0b80d9b700 1 -- 172.21.15.64:0/994010859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b7c073350 msgr2=0x7f0b7c19ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:09.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.295+0000 7f0b80d9b700 1 --2- 172.21.15.64:0/994010859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b7c073350 0x7f0b7c19ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:09.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.295+0000 7f0b80d9b700 1 -- 172.21.15.64:0/994010859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b7c10cd90 msgr2=0x7f0b7c1986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:09.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.295+0000 7f0b80d9b700 1 --2- 172.21.15.64:0/994010859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b7c10cd90 0x7f0b7c1986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:09.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.295+0000 7f0b80d9b700 1 -- 172.21.15.64:0/994010859 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b70009520 con 0x7f0b7c072a30 2023-09-13T03:29:09.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.296+0000 7f0b80d9b700 1 --2- 172.21.15.64:0/994010859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b7c072a30 0x7f0b7c19e670 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0b7000eb30 tx=0x7f0b7000ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:09.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.296+0000 7f0b79ffb700 1 -- 172.21.15.64:0/994010859 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b70009880 con 0x7f0b7c072a30 2023-09-13T03:29:09.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.296+0000 7f0b82fff700 1 -- 172.21.15.64:0/994010859 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b74009c40 con 0x7f0b7c072a30 2023-09-13T03:29:09.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.296+0000 7f0b79ffb700 1 -- 172.21.15.64:0/994010859 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b700044e0 con 0x7f0b7c072a30 2023-09-13T03:29:09.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.296+0000 7f0b79ffb700 1 -- 172.21.15.64:0/994010859 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b70003ce0 con 0x7f0b7c072a30 2023-09-13T03:29:09.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.296+0000 7f0b82fff700 1 -- 172.21.15.64:0/994010859 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b680095f0 con 0x7f0b7c072a30 2023-09-13T03:29:09.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.297+0000 7f0b82fff700 1 -- 172.21.15.64:0/994010859 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b7c067380 con 0x7f0b7c072a30 2023-09-13T03:29:09.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.300+0000 7f0b79ffb700 1 -- 172.21.15.64:0/994010859 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f0b7000cd60 con 0x7f0b7c072a30 2023-09-13T03:29:09.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.301+0000 7f0b79ffb700 1 --2- 172.21.15.64:0/994010859 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f0b6c0a07b0 0x7f0b6c0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:09.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.301+0000 7f0b79ffb700 1 -- 172.21.15.64:0/994010859 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f0b70014070 con 0x7f0b7c072a30 2023-09-13T03:29:09.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.301+0000 7f0b7bfff700 1 --2- 172.21.15.64:0/994010859 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f0b6c0a07b0 0x7f0b6c0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:09.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.302+0000 7f0b7bfff700 1 --2- 172.21.15.64:0/994010859 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f0b6c0a07b0 0x7f0b6c0a2c70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0b6800b2e0 tx=0x7f0b68005960 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:09.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.303+0000 7f0b79ffb700 1 -- 172.21.15.64:0/994010859 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0b70095c70 con 0x7f0b7c072a30 2023-09-13T03:29:09.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.519+0000 7f0b82fff700 1 -- 172.21.15.64:0/994010859 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0b7c1127b0 con 0x7f0b7c072a30 2023-09-13T03:29:09.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.519+0000 7f0b79ffb700 1 -- 172.21.15.64:0/994010859 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v105) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0b70099290 con 0x7f0b7c072a30 2023-09-13T03:29:09.521 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-09-13T03:29:09.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.522+0000 7f0b82fff700 1 -- 172.21.15.64:0/994010859 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f0b6c0a07b0 msgr2=0x7f0b6c0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:09.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.522+0000 7f0b82fff700 1 --2- 172.21.15.64:0/994010859 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f0b6c0a07b0 0x7f0b6c0a2c70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0b6800b2e0 tx=0x7f0b68005960 comp rx=0 tx=0).stop 2023-09-13T03:29:09.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.522+0000 7f0b82fff700 1 -- 172.21.15.64:0/994010859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b7c072a30 msgr2=0x7f0b7c19e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:09.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.522+0000 7f0b82fff700 1 --2- 172.21.15.64:0/994010859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b7c072a30 0x7f0b7c19e670 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0b7000eb30 tx=0x7f0b7000ee40 comp rx=0 tx=0).stop 2023-09-13T03:29:09.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.522+0000 7f0b82fff700 1 -- 172.21.15.64:0/994010859 shutdown_connections 2023-09-13T03:29:09.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.522+0000 7f0b82fff700 1 --2- 172.21.15.64:0/994010859 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f0b6c0a07b0 0x7f0b6c0a2c70 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:09.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.522+0000 7f0b82fff700 1 --2- 172.21.15.64:0/994010859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b7c10cd90 0x7f0b7c1986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:09.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.522+0000 7f0b82fff700 1 --2- 172.21.15.64:0/994010859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b7c072a30 0x7f0b7c19e670 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:09.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.522+0000 7f0b82fff700 1 --2- 172.21.15.64:0/994010859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b7c073350 0x7f0b7c19ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:09.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.522+0000 7f0b82fff700 1 -- 172.21.15.64:0/994010859 >> 172.21.15.64:0/994010859 conn(0x7f0b7c0fc080 msgr2=0x7f0b7c10ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:09.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.522+0000 7f0b82fff700 1 -- 172.21.15.64:0/994010859 shutdown_connections 2023-09-13T03:29:09.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.522+0000 7f0b82fff700 1 -- 172.21.15.64:0/994010859 wait complete. 2023-09-13T03:29:09.546 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-13T03:29:09.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.783+0000 7fe3b79ff700 1 -- 172.21.15.64:0/1672936780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3b0072a30 msgr2=0x7fe3b0072e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:09.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.783+0000 7fe3b79ff700 1 --2- 172.21.15.64:0/1672936780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3b0072a30 0x7fe3b0072e10 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac009990 tx=0x7fe3ac009ca0 comp rx=0 tx=0).stop 2023-09-13T03:29:09.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.783+0000 7fe3b79ff700 1 -- 172.21.15.64:0/1672936780 shutdown_connections 2023-09-13T03:29:09.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.783+0000 7fe3b79ff700 1 --2- 172.21.15.64:0/1672936780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3b010cdb0 0x7fe3b010f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:09.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.783+0000 7fe3b79ff700 1 --2- 172.21.15.64:0/1672936780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3b0072a30 0x7fe3b0072e10 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:09.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.783+0000 7fe3b79ff700 1 --2- 172.21.15.64:0/1672936780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3b0073350 0x7fe3b010c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:09.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.783+0000 7fe3b79ff700 1 -- 172.21.15.64:0/1672936780 >> 172.21.15.64:0/1672936780 conn(0x7fe3b00fc060 msgr2=0x7fe3b00fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:09.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.783+0000 7fe3b79ff700 1 -- 172.21.15.64:0/1672936780 shutdown_connections 2023-09-13T03:29:09.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.783+0000 7fe3b79ff700 1 -- 172.21.15.64:0/1672936780 wait complete. 2023-09-13T03:29:09.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.784+0000 7fe3b79ff700 1 Processor -- start 2023-09-13T03:29:09.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.784+0000 7fe3b79ff700 1 -- start start 2023-09-13T03:29:09.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.785+0000 7fe3b79ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3b0072a30 0x7fe3b019e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:09.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.785+0000 7fe3b79ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3b0073350 0x7fe3b019ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:09.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.785+0000 7fe3b79ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3b010cdb0 0x7fe3b0198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:09.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.785+0000 7fe3b79ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b0198da0 con 0x7fe3b0073350 2023-09-13T03:29:09.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.785+0000 7fe3b79ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b0198f10 con 0x7fe3b0072a30 2023-09-13T03:29:09.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.785+0000 7fe3b79ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b01990d0 con 0x7fe3b010cdb0 2023-09-13T03:29:09.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.785+0000 7fe3b579b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3b0072a30 0x7fe3b019e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:09.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.785+0000 7fe3b579b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3b0072a30 0x7fe3b019e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:54568/0 (socket says 172.21.15.64:54568) 2023-09-13T03:29:09.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.785+0000 7fe3b579b700 1 -- 172.21.15.64:0/804413372 learned_addr learned my addr 172.21.15.64:0/804413372 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:29:09.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.785+0000 7fe3b4f9a700 1 --2- 172.21.15.64:0/804413372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3b0073350 0x7fe3b019ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:09.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.785+0000 7fe3b5f9c700 1 --2- 172.21.15.64:0/804413372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3b010cdb0 0x7fe3b0198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:09.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.785+0000 7fe3b5f9c700 1 -- 172.21.15.64:0/804413372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3b0072a30 msgr2=0x7fe3b019e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:09.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.785+0000 7fe3b5f9c700 1 --2- 172.21.15.64:0/804413372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3b0072a30 0x7fe3b019e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:09.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.786+0000 7fe3b5f9c700 1 -- 172.21.15.64:0/804413372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3b0073350 msgr2=0x7fe3b019ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:09.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.786+0000 7fe3b5f9c700 1 --2- 172.21.15.64:0/804413372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3b0073350 0x7fe3b019ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:09.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.786+0000 7fe3b5f9c700 1 -- 172.21.15.64:0/804413372 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a0009520 con 0x7fe3b010cdb0 2023-09-13T03:29:09.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.786+0000 7fe3b579b700 1 --2- 172.21.15.64:0/804413372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3b0072a30 0x7fe3b019e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:29:09.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.786+0000 7fe3b5f9c700 1 --2- 172.21.15.64:0/804413372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3b010cdb0 0x7fe3b0198730 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe3a800efb0 tx=0x7fe3a800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:09.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.786+0000 7fe3a67fc700 1 -- 172.21.15.64:0/804413372 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8003a00 con 0x7fe3b010cdb0 2023-09-13T03:29:09.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.786+0000 7fe3b79ff700 1 -- 172.21.15.64:0/804413372 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3a8009c40 con 0x7fe3b010cdb0 2023-09-13T03:29:09.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.786+0000 7fe3a67fc700 1 -- 172.21.15.64:0/804413372 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3a800f040 con 0x7fe3b010cdb0 2023-09-13T03:29:09.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.786+0000 7fe3a67fc700 1 -- 172.21.15.64:0/804413372 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8004550 con 0x7fe3b010cdb0 2023-09-13T03:29:09.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.786+0000 7fe3b79ff700 1 -- 172.21.15.64:0/804413372 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3ac0095f0 con 0x7fe3b010cdb0 2023-09-13T03:29:09.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.787+0000 7fe3b79ff700 1 -- 172.21.15.64:0/804413372 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3b0067380 con 0x7fe3b010cdb0 2023-09-13T03:29:09.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.789+0000 7fe3a67fc700 1 -- 172.21.15.64:0/804413372 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fe3a8003bf0 con 0x7fe3b010cdb0 2023-09-13T03:29:09.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.791+0000 7fe3a67fc700 1 --2- 172.21.15.64:0/804413372 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fe39c0a07b0 0x7fe39c0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:09.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.791+0000 7fe3a67fc700 1 -- 172.21.15.64:0/804413372 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8528+0+0 (secure 0 0 0) 0x7fe3a80cbd70 con 0x7fe3b010cdb0 2023-09-13T03:29:09.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.791+0000 7fe3b579b700 1 --2- 172.21.15.64:0/804413372 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fe39c0a07b0 0x7fe39c0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:09.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.792+0000 7fe3b579b700 1 --2- 172.21.15.64:0/804413372 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fe39c0a07b0 0x7fe39c0a2c70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac006010 tx=0x7fe3ac00b260 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:09.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:09.793+0000 7fe3a67fc700 1 -- 172.21.15.64:0/804413372 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe3a80955b0 con 0x7fe3b010cdb0 2023-09-13T03:29:10.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.004+0000 7fe3b79ff700 1 -- 172.21.15.64:0/804413372 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe3b0199c40 con 0x7fe3b010cdb0 2023-09-13T03:29:10.006 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.005+0000 7fe3a67fc700 1 -- 172.21.15.64:0/804413372 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v105) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe3a8098bd0 con 0x7fe3b010cdb0 2023-09-13T03:29:10.006 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-09-13T03:29:10.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.007+0000 7fe3b79ff700 1 -- 172.21.15.64:0/804413372 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fe39c0a07b0 msgr2=0x7fe39c0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:10.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.007+0000 7fe3b79ff700 1 --2- 172.21.15.64:0/804413372 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fe39c0a07b0 0x7fe39c0a2c70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac006010 tx=0x7fe3ac00b260 comp rx=0 tx=0).stop 2023-09-13T03:29:10.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.007+0000 7fe3b79ff700 1 -- 172.21.15.64:0/804413372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3b010cdb0 msgr2=0x7fe3b0198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:10.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.007+0000 7fe3b79ff700 1 --2- 172.21.15.64:0/804413372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3b010cdb0 0x7fe3b0198730 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe3a800efb0 tx=0x7fe3a800c3a0 comp rx=0 tx=0).stop 2023-09-13T03:29:10.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.007+0000 7fe3b79ff700 1 -- 172.21.15.64:0/804413372 shutdown_connections 2023-09-13T03:29:10.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.007+0000 7fe3b79ff700 1 --2- 172.21.15.64:0/804413372 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fe39c0a07b0 0x7fe39c0a2c70 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:10.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.007+0000 7fe3b79ff700 1 --2- 172.21.15.64:0/804413372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3b0072a30 0x7fe3b019e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:10.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.007+0000 7fe3b79ff700 1 --2- 172.21.15.64:0/804413372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3b010cdb0 0x7fe3b0198730 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:10.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.007+0000 7fe3b79ff700 1 --2- 172.21.15.64:0/804413372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3b0073350 0x7fe3b019ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:10.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.008+0000 7fe3b79ff700 1 -- 172.21.15.64:0/804413372 >> 172.21.15.64:0/804413372 conn(0x7fe3b00fc060 msgr2=0x7fe3b010efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:10.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.008+0000 7fe3b79ff700 1 -- 172.21.15.64:0/804413372 shutdown_connections 2023-09-13T03:29:10.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.008+0000 7fe3b79ff700 1 -- 172.21.15.64:0/804413372 wait complete. 2023-09-13T03:29:10.028 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-09-13T03:29:10.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:09 smithi170 ceph-mon[134328]: from='client.17292 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:29:10.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:09 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:29:10.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:09 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/1471298843' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T03:29:10.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:09 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/994010859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T03:29:10.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:09 smithi064 ceph-mon[97675]: from='client.17292 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:29:10.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:09 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:29:10.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:09 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/1471298843' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T03:29:10.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:09 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/994010859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T03:29:10.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.232+0000 7f07a63ff700 1 -- 172.21.15.64:0/1774987722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07a0100590 msgr2=0x7f07a0100a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:10.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.232+0000 7f07a63ff700 1 --2- 172.21.15.64:0/1774987722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07a0100590 0x7f07a0100a10 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f07880099c0 tx=0x7f0788009cd0 comp rx=0 tx=0).stop 2023-09-13T03:29:10.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.233+0000 7f07a63ff700 1 -- 172.21.15.64:0/1774987722 shutdown_connections 2023-09-13T03:29:10.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.233+0000 7f07a63ff700 1 --2- 172.21.15.64:0/1774987722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07a0106650 0x7f07a0106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:10.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.233+0000 7f07a63ff700 1 --2- 172.21.15.64:0/1774987722 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07a0100f50 0x7f07a010d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:10.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.233+0000 7f07a63ff700 1 --2- 172.21.15.64:0/1774987722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07a0100590 0x7f07a0100a10 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:10.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.233+0000 7f07a63ff700 1 -- 172.21.15.64:0/1774987722 >> 172.21.15.64:0/1774987722 conn(0x7f07a00fc020 msgr2=0x7f07a00fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:10.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.233+0000 7f07a63ff700 1 -- 172.21.15.64:0/1774987722 shutdown_connections 2023-09-13T03:29:10.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.233+0000 7f07a63ff700 1 -- 172.21.15.64:0/1774987722 wait complete. 2023-09-13T03:29:10.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.234+0000 7f07a63ff700 1 Processor -- start 2023-09-13T03:29:10.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.234+0000 7f07a63ff700 1 -- start start 2023-09-13T03:29:10.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.234+0000 7f07a63ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07a0100590 0x7f07a0072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:10.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.234+0000 7f07a63ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07a0100f50 0x7f07a0072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:10.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.234+0000 7f07a63ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07a0106650 0x7f07a01a47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:10.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.234+0000 7f07a63ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a0073670 con 0x7f07a0100590 2023-09-13T03:29:10.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.234+0000 7f07a63ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a01a4d20 con 0x7f07a0100f50 2023-09-13T03:29:10.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.234+0000 7f07a63ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a01a4e60 con 0x7f07a0106650 2023-09-13T03:29:10.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.235+0000 7f079ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07a0100590 0x7f07a0072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:10.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.235+0000 7f0797fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07a0100f50 0x7f07a0072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:10.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.235+0000 7f0797fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07a0100f50 0x7f07a0072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:54586/0 (socket says 172.21.15.64:54586) 2023-09-13T03:29:10.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.235+0000 7f0797fff700 1 -- 172.21.15.64:0/2662790034 learned_addr learned my addr 172.21.15.64:0/2662790034 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:29:10.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.235+0000 7f07a499c700 1 --2- 172.21.15.64:0/2662790034 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07a0106650 0x7f07a01a47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:10.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.235+0000 7f0797fff700 1 -- 172.21.15.64:0/2662790034 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07a0106650 msgr2=0x7f07a01a47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:10.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.235+0000 7f0797fff700 1 --2- 172.21.15.64:0/2662790034 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07a0106650 0x7f07a01a47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:10.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.235+0000 7f0797fff700 1 -- 172.21.15.64:0/2662790034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07a0100590 msgr2=0x7f07a0072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:10.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.235+0000 7f0797fff700 1 --2- 172.21.15.64:0/2662790034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07a0100590 0x7f07a0072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:10.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.235+0000 7f0797fff700 1 -- 172.21.15.64:0/2662790034 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07880095f0 con 0x7f07a0100f50 2023-09-13T03:29:10.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.235+0000 7f079ffff700 1 --2- 172.21.15.64:0/2662790034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07a0100590 0x7f07a0072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:29:10.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.235+0000 7f0797fff700 1 --2- 172.21.15.64:0/2662790034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07a0100f50 0x7f07a0072e40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f078800b580 tx=0x7f078800bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:10.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.236+0000 7f079dffb700 1 -- 172.21.15.64:0/2662790034 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078801d070 con 0x7f07a0100f50 2023-09-13T03:29:10.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.236+0000 7f07a63ff700 1 -- 172.21.15.64:0/2662790034 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07a01a5060 con 0x7f07a0100f50 2023-09-13T03:29:10.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.236+0000 7f07a63ff700 1 -- 172.21.15.64:0/2662790034 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a01a54e0 con 0x7f07a0100f50 2023-09-13T03:29:10.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.236+0000 7f079dffb700 1 -- 172.21.15.64:0/2662790034 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0788005050 con 0x7f07a0100f50 2023-09-13T03:29:10.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.236+0000 7f079dffb700 1 -- 172.21.15.64:0/2662790034 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07880049b0 con 0x7f07a0100f50 2023-09-13T03:29:10.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.237+0000 7f07a63ff700 1 -- 172.21.15.64:0/2662790034 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07a00673f0 con 0x7f07a0100f50 2023-09-13T03:29:10.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:09 smithi142 ceph-mon[134082]: from='client.17292 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-13T03:29:10.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:09 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:29:10.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:09 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/1471298843' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T03:29:10.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:09 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/994010859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T03:29:10.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.239+0000 7f079dffb700 1 -- 172.21.15.64:0/2662790034 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f0788005480 con 0x7f07a0100f50 2023-09-13T03:29:10.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.240+0000 7f079dffb700 1 --2- 172.21.15.64:0/2662790034 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f078c0a0870 0x7f078c0a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:10.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.241+0000 7f079dffb700 1 -- 172.21.15.64:0/2662790034 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f07880cd050 con 0x7f07a0100f50 2023-09-13T03:29:10.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.241+0000 7f079ffff700 1 --2- 172.21.15.64:0/2662790034 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f078c0a0870 0x7f078c0a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:10.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.241+0000 7f079ffff700 1 --2- 172.21.15.64:0/2662790034 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f078c0a0870 0x7f078c0a2d30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0790011490 tx=0x7f0790009260 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:10.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.243+0000 7f079dffb700 1 -- 172.21.15.64:0/2662790034 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0788096890 con 0x7f07a0100f50 2023-09-13T03:29:10.530 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.529+0000 7f07a63ff700 1 -- 172.21.15.64:0/2662790034 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f07a01105a0 con 0x7f07a0100f50 2023-09-13T03:29:10.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.538+0000 7f079dffb700 1 -- 172.21.15.64:0/2662790034 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) v1 ==== 81+0+730426 (secure 0 0 0) 0x7f0788099eb0 con 0x7f07a0100f50 2023-09-13T03:29:10.550 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:29:10.550 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-13T03:29:10.550 INFO:teuthology.orchestra.run.smithi064.stdout: "epoch": 26, 2023-09-13T03:29:10.551 INFO:teuthology.orchestra.run.smithi064.stdout: "active_gid": 15507, 2023-09-13T03:29:10.551 INFO:teuthology.orchestra.run.smithi064.stdout: "active_name": "x", 2023-09-13T03:29:10.551 INFO:teuthology.orchestra.run.smithi064.stdout: "active_addrs": { 2023-09-13T03:29:10.551 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2023-09-13T03:29:10.552 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:10.552 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2023-09-13T03:29:10.552 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:6800", 2023-09-13T03:29:10.553 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 3860981465 2023-09-13T03:29:10.553 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.553 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:10.554 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-09-13T03:29:10.554 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:6801", 2023-09-13T03:29:10.554 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 3860981465 2023-09-13T03:29:10.555 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:10.555 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-09-13T03:29:10.555 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.556 INFO:teuthology.orchestra.run.smithi064.stdout: "active_addr": "172.21.15.64:6801/3860981465", 2023-09-13T03:29:10.556 INFO:teuthology.orchestra.run.smithi064.stdout: "active_change": "2023-09-13T03:13:09.493086+0000", 2023-09-13T03:29:10.556 INFO:teuthology.orchestra.run.smithi064.stdout: "active_mgr_features": 4540138322906710015, 2023-09-13T03:29:10.557 INFO:teuthology.orchestra.run.smithi064.stdout: "available": true, 2023-09-13T03:29:10.557 INFO:teuthology.orchestra.run.smithi064.stdout: "standbys": [ 2023-09-13T03:29:10.557 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:10.558 INFO:teuthology.orchestra.run.smithi064.stdout: "gid": 25526, 2023-09-13T03:29:10.558 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "y", 2023-09-13T03:29:10.558 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr_features": 4540138322906710015, 2023-09-13T03:29:10.559 INFO:teuthology.orchestra.run.smithi064.stdout: "available_modules": [ 2023-09-13T03:29:10.559 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:10.559 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "alerts", 2023-09-13T03:29:10.559 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:10.560 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:10.560 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:10.560 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2023-09-13T03:29:10.561 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2023-09-13T03:29:10.561 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:10.561 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.561 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:10.562 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2023-09-13T03:29:10.562 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.563 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.563 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.563 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How frequently to reexamine health status", 2023-09-13T03:29:10.563 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.564 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.564 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.564 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.565 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:10.565 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:10.565 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:10.565 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.566 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:10.566 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:10.566 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.567 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.567 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:10.567 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:10.568 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:10.568 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:10.568 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:10.568 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:10.569 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:10.569 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:10.569 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:10.569 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.570 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.570 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.570 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.571 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:10.571 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:10.571 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:10.571 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.572 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:10.572 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:10.572 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.573 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.573 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.573 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:10.574 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.574 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.574 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.575 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.575 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:10.575 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:10.576 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:10.576 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.576 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:10.576 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:10.577 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.577 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.577 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:10.578 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:10.578 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:10.578 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:10.579 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:10.579 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:10.579 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:10.580 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:10.580 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:10.580 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.581 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.581 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.581 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.581 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:10.582 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:10.582 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:10.582 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.583 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:10.583 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:10.583 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.584 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.584 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.584 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:10.585 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.585 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.585 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.586 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.586 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_destination": { 2023-09-13T03:29:10.586 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_destination", 2023-09-13T03:29:10.587 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:10.587 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.587 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:10.588 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:10.588 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.588 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.589 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.589 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Email address to send alerts to", 2023-09-13T03:29:10.589 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.589 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.590 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.590 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.590 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_from_name": { 2023-09-13T03:29:10.591 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_from_name", 2023-09-13T03:29:10.591 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:10.591 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.591 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:10.592 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "Ceph", 2023-09-13T03:29:10.592 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.592 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.592 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.593 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Email From: name", 2023-09-13T03:29:10.593 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.593 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.594 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.594 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.594 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_host": { 2023-09-13T03:29:10.595 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_host", 2023-09-13T03:29:10.595 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:10.595 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.596 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:10.596 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:10.596 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.596 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.597 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.597 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP server", 2023-09-13T03:29:10.597 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.598 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.598 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.598 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.599 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_password": { 2023-09-13T03:29:10.599 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_password", 2023-09-13T03:29:10.599 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:10.600 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.600 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:10.600 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:10.600 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.601 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.601 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.601 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Password to authenticate with", 2023-09-13T03:29:10.602 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.602 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.602 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.602 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.603 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_port": { 2023-09-13T03:29:10.603 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_port", 2023-09-13T03:29:10.603 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:10.604 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.604 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:10.604 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "465", 2023-09-13T03:29:10.604 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.605 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.605 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.605 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP port", 2023-09-13T03:29:10.606 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.606 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.606 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.607 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.607 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_sender": { 2023-09-13T03:29:10.607 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_sender", 2023-09-13T03:29:10.607 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:10.608 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.608 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:10.608 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:10.609 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.609 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.609 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.610 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP envelope sender", 2023-09-13T03:29:10.610 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.610 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.611 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.611 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.611 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_ssl": { 2023-09-13T03:29:10.612 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_ssl", 2023-09-13T03:29:10.612 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:10.612 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.612 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:10.613 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:10.613 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.613 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.614 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.614 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use SSL to connect to SMTP server", 2023-09-13T03:29:10.614 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.615 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.615 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.615 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.616 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_user": { 2023-09-13T03:29:10.616 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_user", 2023-09-13T03:29:10.616 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:10.617 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.617 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:10.617 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:10.617 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.618 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.618 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.618 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "User to authenticate as", 2023-09-13T03:29:10.619 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.619 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.619 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.620 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:10.620 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:10.620 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.620 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:10.621 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "balancer", 2023-09-13T03:29:10.621 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:10.621 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:10.622 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:10.622 INFO:teuthology.orchestra.run.smithi064.stdout: "active": { 2023-09-13T03:29:10.622 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "active", 2023-09-13T03:29:10.623 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:10.623 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.623 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:10.624 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:10.624 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.624 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.625 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.625 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "automatically balance PGs across cluster", 2023-09-13T03:29:10.625 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.625 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.626 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.626 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.626 INFO:teuthology.orchestra.run.smithi064.stdout: "begin_time": { 2023-09-13T03:29:10.627 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "begin_time", 2023-09-13T03:29:10.627 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:10.628 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.628 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:10.628 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0000", 2023-09-13T03:29:10.629 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.629 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.630 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.630 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "beginning time of day to automatically balance", 2023-09-13T03:29:10.631 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-13T03:29:10.631 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.631 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.632 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.632 INFO:teuthology.orchestra.run.smithi064.stdout: "begin_weekday": { 2023-09-13T03:29:10.633 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "begin_weekday", 2023-09-13T03:29:10.633 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2023-09-13T03:29:10.633 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.634 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:10.634 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2023-09-13T03:29:10.634 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0", 2023-09-13T03:29:10.635 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "6", 2023-09-13T03:29:10.635 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.635 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-09-13T03:29:10.635 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-13T03:29:10.636 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.636 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.636 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.637 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_max_iterations": { 2023-09-13T03:29:10.637 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_max_iterations", 2023-09-13T03:29:10.637 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2023-09-13T03:29:10.638 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.638 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:10.638 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "25", 2023-09-13T03:29:10.639 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2023-09-13T03:29:10.639 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "250", 2023-09-13T03:29:10.639 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.640 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-09-13T03:29:10.640 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.640 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.641 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.641 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.641 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_metrics": { 2023-09-13T03:29:10.642 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_metrics", 2023-09-13T03:29:10.642 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:10.642 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.643 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:10.643 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "pgs,objects,bytes", 2023-09-13T03:29:10.643 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.644 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.644 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.644 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-09-13T03:29:10.645 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-09-13T03:29:10.645 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.646 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.646 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.646 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_step": { 2023-09-13T03:29:10.647 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_step", 2023-09-13T03:29:10.647 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2023-09-13T03:29:10.647 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.648 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:10.648 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0.5", 2023-09-13T03:29:10.648 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0.001", 2023-09-13T03:29:10.649 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "0.999", 2023-09-13T03:29:10.649 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.649 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "aggressiveness of optimization", 2023-09-13T03:29:10.650 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-09-13T03:29:10.650 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.650 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.651 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.651 INFO:teuthology.orchestra.run.smithi064.stdout: "end_time": { 2023-09-13T03:29:10.651 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "end_time", 2023-09-13T03:29:10.652 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:10.652 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.652 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:10.653 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "2359", 2023-09-13T03:29:10.653 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.654 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.654 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.654 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "ending time of day to automatically balance", 2023-09-13T03:29:10.655 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-13T03:29:10.655 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.655 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.656 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.656 INFO:teuthology.orchestra.run.smithi064.stdout: "end_weekday": { 2023-09-13T03:29:10.656 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "end_weekday", 2023-09-13T03:29:10.657 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2023-09-13T03:29:10.657 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.657 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:10.658 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2023-09-13T03:29:10.658 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0", 2023-09-13T03:29:10.658 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "6", 2023-09-13T03:29:10.659 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.659 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-09-13T03:29:10.659 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-13T03:29:10.660 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.660 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.660 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.661 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:10.661 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:10.661 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:10.662 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.662 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:10.662 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:10.663 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.663 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.663 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:10.664 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:10.664 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:10.664 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:10.665 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:10.665 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:10.665 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:10.666 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:10.666 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:10.666 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.667 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.667 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.667 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.668 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:10.668 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:10.668 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:10.669 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.669 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:10.669 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:10.670 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.670 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.670 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.671 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:10.671 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.671 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.672 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.672 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.672 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:10.673 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:10.673 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:10.673 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.674 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:10.674 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:10.674 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.675 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.675 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:10.675 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:10.675 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:10.676 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:10.676 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:10.676 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:10.677 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:10.677 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:10.677 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:10.678 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.678 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.678 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.679 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.679 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:10.679 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:10.680 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:10.680 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.680 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:10.680 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:10.681 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.681 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.681 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.682 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:10.682 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.682 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.683 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.683 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.683 INFO:teuthology.orchestra.run.smithi064.stdout: "min_score": { 2023-09-13T03:29:10.683 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "min_score", 2023-09-13T03:29:10.684 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2023-09-13T03:29:10.684 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.684 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:10.685 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2023-09-13T03:29:10.685 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.685 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.685 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.686 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-09-13T03:29:10.686 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.687 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.687 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.687 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.688 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": { 2023-09-13T03:29:10.688 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mode", 2023-09-13T03:29:10.688 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:10.688 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.689 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:10.689 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "upmap", 2023-09-13T03:29:10.689 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.690 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.690 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:10.690 INFO:teuthology.orchestra.run.smithi064.stdout: "crush-compat", 2023-09-13T03:29:10.691 INFO:teuthology.orchestra.run.smithi064.stdout: "none", 2023-09-13T03:29:10.691 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap" 2023-09-13T03:29:10.692 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:10.692 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Balancer mode", 2023-09-13T03:29:10.692 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.693 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.693 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.693 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.694 INFO:teuthology.orchestra.run.smithi064.stdout: "pool_ids": { 2023-09-13T03:29:10.694 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pool_ids", 2023-09-13T03:29:10.695 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:10.695 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.695 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:10.695 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:10.696 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.696 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.696 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.697 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-09-13T03:29:10.697 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.697 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.698 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.698 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.698 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2023-09-13T03:29:10.699 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2023-09-13T03:29:10.699 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:10.699 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.700 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:10.700 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2023-09-13T03:29:10.700 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.701 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.701 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.701 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-09-13T03:29:10.702 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.702 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.702 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.702 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.703 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap_max_deviation": { 2023-09-13T03:29:10.703 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "upmap_max_deviation", 2023-09-13T03:29:10.703 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:10.704 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.704 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:10.704 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2023-09-13T03:29:10.705 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2023-09-13T03:29:10.705 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.705 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.706 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "deviation below which no optimization is attempted", 2023-09-13T03:29:10.706 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-09-13T03:29:10.706 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.706 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.707 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.707 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap_max_optimizations": { 2023-09-13T03:29:10.707 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "upmap_max_optimizations", 2023-09-13T03:29:10.708 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2023-09-13T03:29:10.708 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.708 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:10.708 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2023-09-13T03:29:10.709 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.709 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.709 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.710 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-09-13T03:29:10.710 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.710 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.711 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.711 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:10.711 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:10.711 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.712 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:10.712 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephadm", 2023-09-13T03:29:10.712 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:10.713 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:10.713 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:10.713 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_down_multiplier": { 2023-09-13T03:29:10.714 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_down_multiplier", 2023-09-13T03:29:10.714 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2023-09-13T03:29:10.714 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.714 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.715 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3.0", 2023-09-13T03:29:10.715 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.715 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.716 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.716 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-09-13T03:29:10.716 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.717 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.717 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.717 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.717 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_refresh_rate": { 2023-09-13T03:29:10.718 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_refresh_rate", 2023-09-13T03:29:10.718 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:10.718 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.719 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.719 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "20", 2023-09-13T03:29:10.719 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.720 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.720 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.720 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-09-13T03:29:10.720 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.721 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.721 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.721 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.722 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_starting_port": { 2023-09-13T03:29:10.722 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_starting_port", 2023-09-13T03:29:10.722 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:10.723 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.723 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.723 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "4721", 2023-09-13T03:29:10.723 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.724 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.724 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.724 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-09-13T03:29:10.725 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.725 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.725 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.726 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.726 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_ptrace": { 2023-09-13T03:29:10.726 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_ptrace", 2023-09-13T03:29:10.727 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:10.727 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.727 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.728 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:10.728 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.728 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.729 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.729 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-09-13T03:29:10.729 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-09-13T03:29:10.729 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.730 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.730 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.730 INFO:teuthology.orchestra.run.smithi064.stdout: "autotune_interval": { 2023-09-13T03:29:10.731 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "autotune_interval", 2023-09-13T03:29:10.731 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:10.731 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.732 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.732 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2023-09-13T03:29:10.732 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.732 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.733 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.733 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to autotune daemon memory", 2023-09-13T03:29:10.733 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.734 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.734 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.734 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.735 INFO:teuthology.orchestra.run.smithi064.stdout: "autotune_memory_target_ratio": { 2023-09-13T03:29:10.735 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "autotune_memory_target_ratio", 2023-09-13T03:29:10.735 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2023-09-13T03:29:10.736 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.736 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.736 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0.7", 2023-09-13T03:29:10.737 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.737 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.737 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.738 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-09-13T03:29:10.738 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.738 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.739 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.739 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.739 INFO:teuthology.orchestra.run.smithi064.stdout: "cgroups_split": { 2023-09-13T03:29:10.740 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cgroups_split", 2023-09-13T03:29:10.740 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:10.740 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.741 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.741 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:10.741 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.741 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.742 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.742 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-09-13T03:29:10.742 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.743 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.743 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.744 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.744 INFO:teuthology.orchestra.run.smithi064.stdout: "config_checks_enabled": { 2023-09-13T03:29:10.744 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_checks_enabled", 2023-09-13T03:29:10.744 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:10.745 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.745 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.745 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:10.746 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.746 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.746 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.747 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-09-13T03:29:10.747 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.747 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.748 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.748 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.748 INFO:teuthology.orchestra.run.smithi064.stdout: "config_dashboard": { 2023-09-13T03:29:10.748 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_dashboard", 2023-09-13T03:29:10.749 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:10.749 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.749 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.750 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:10.750 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.750 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.751 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.751 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-09-13T03:29:10.751 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.752 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.752 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.752 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.752 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_alertmanager": { 2023-09-13T03:29:10.753 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_alertmanager", 2023-09-13T03:29:10.753 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:10.753 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.754 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.754 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-09-13T03:29:10.754 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.755 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.755 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.755 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2023-09-13T03:29:10.756 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.756 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.756 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.757 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.757 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_base": { 2023-09-13T03:29:10.757 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_base", 2023-09-13T03:29:10.757 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:10.758 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.758 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:10.758 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/ceph", 2023-09-13T03:29:10.759 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.759 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.759 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.760 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Container image name, without the tag", 2023-09-13T03:29:10.760 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.760 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.760 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.761 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.761 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_elasticsearch": { 2023-09-13T03:29:10.761 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_elasticsearch", 2023-09-13T03:29:10.762 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:10.762 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.762 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.763 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-09-13T03:29:10.763 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.763 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.763 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.764 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "elasticsearch container image", 2023-09-13T03:29:10.764 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.764 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.765 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.765 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.765 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_grafana": { 2023-09-13T03:29:10.766 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_grafana", 2023-09-13T03:29:10.766 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:10.766 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.766 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.767 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-09-13T03:29:10.767 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.767 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.768 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.768 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2023-09-13T03:29:10.768 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.769 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.769 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.769 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.769 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_haproxy": { 2023-09-13T03:29:10.770 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_haproxy", 2023-09-13T03:29:10.770 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:10.770 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.771 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.771 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-09-13T03:29:10.771 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.772 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.772 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.772 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "HAproxy container image", 2023-09-13T03:29:10.773 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.773 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.773 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.773 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.774 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_agent": { 2023-09-13T03:29:10.774 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_agent", 2023-09-13T03:29:10.774 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:10.775 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.775 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.775 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-09-13T03:29:10.776 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.776 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.776 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.776 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger agent container image", 2023-09-13T03:29:10.777 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.777 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.777 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.778 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.778 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_collector": { 2023-09-13T03:29:10.778 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_collector", 2023-09-13T03:29:10.779 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:10.779 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.779 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.780 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-09-13T03:29:10.780 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.780 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.781 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.781 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger collector container image", 2023-09-13T03:29:10.781 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.781 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.782 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.782 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.782 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_query": { 2023-09-13T03:29:10.783 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_query", 2023-09-13T03:29:10.783 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:10.783 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.784 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.784 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-09-13T03:29:10.784 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.785 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.785 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.785 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger query container image", 2023-09-13T03:29:10.785 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.786 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.786 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.786 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.787 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_keepalived": { 2023-09-13T03:29:10.787 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_keepalived", 2023-09-13T03:29:10.787 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:10.788 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.788 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.788 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-09-13T03:29:10.788 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.789 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.789 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.789 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Keepalived container image", 2023-09-13T03:29:10.790 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.790 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.790 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.791 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.791 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_loki": { 2023-09-13T03:29:10.791 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_loki", 2023-09-13T03:29:10.792 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:10.792 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.792 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.792 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-09-13T03:29:10.793 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.793 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.793 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.794 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Loki container image", 2023-09-13T03:29:10.794 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.794 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.795 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.795 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.795 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_node_exporter": { 2023-09-13T03:29:10.795 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_node_exporter", 2023-09-13T03:29:10.796 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:10.796 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.796 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.797 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-09-13T03:29:10.797 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.797 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.798 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.798 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2023-09-13T03:29:10.798 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.799 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.799 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.799 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.800 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_nvmeof": { 2023-09-13T03:29:10.800 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_nvmeof", 2023-09-13T03:29:10.800 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:10.801 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.801 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.801 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-09-13T03:29:10.802 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.802 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.802 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.803 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Nvme-of container image", 2023-09-13T03:29:10.803 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.803 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.804 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.804 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.804 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_prometheus": { 2023-09-13T03:29:10.805 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_prometheus", 2023-09-13T03:29:10.805 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:10.805 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.805 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.806 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-09-13T03:29:10.806 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.806 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.807 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.807 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2023-09-13T03:29:10.807 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.808 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.808 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.808 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.808 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_promtail": { 2023-09-13T03:29:10.809 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_promtail", 2023-09-13T03:29:10.809 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:10.809 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.810 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.810 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-09-13T03:29:10.810 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.811 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.811 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.811 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Promtail container image", 2023-09-13T03:29:10.811 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.812 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.812 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.812 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.813 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_snmp_gateway": { 2023-09-13T03:29:10.813 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_snmp_gateway", 2023-09-13T03:29:10.813 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:10.813 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.814 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.814 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-09-13T03:29:10.814 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.815 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.815 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.815 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SNMP Gateway container image", 2023-09-13T03:29:10.816 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.816 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.816 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.817 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.817 INFO:teuthology.orchestra.run.smithi064.stdout: "container_init": { 2023-09-13T03:29:10.817 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_init", 2023-09-13T03:29:10.817 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:10.818 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.818 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.818 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:10.819 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.819 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.819 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.820 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Run podman/docker with `--init`", 2023-09-13T03:29:10.820 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.820 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.821 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.821 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.821 INFO:teuthology.orchestra.run.smithi064.stdout: "daemon_cache_timeout": { 2023-09-13T03:29:10.821 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "daemon_cache_timeout", 2023-09-13T03:29:10.822 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:10.822 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.822 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.823 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2023-09-13T03:29:10.823 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.823 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.824 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.824 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-09-13T03:29:10.824 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.824 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.825 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.825 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.825 INFO:teuthology.orchestra.run.smithi064.stdout: "default_cephadm_command_timeout": { 2023-09-13T03:29:10.826 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "default_cephadm_command_timeout", 2023-09-13T03:29:10.826 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:10.826 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.827 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.827 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "900", 2023-09-13T03:29:10.827 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.828 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.828 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.828 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-09-13T03:29:10.829 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.829 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.829 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.830 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.830 INFO:teuthology.orchestra.run.smithi064.stdout: "default_registry": { 2023-09-13T03:29:10.830 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "default_registry", 2023-09-13T03:29:10.831 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:10.831 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.831 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.832 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io", 2023-09-13T03:29:10.832 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.832 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.832 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.833 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-09-13T03:29:10.833 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.833 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.834 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.834 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.834 INFO:teuthology.orchestra.run.smithi064.stdout: "device_cache_timeout": { 2023-09-13T03:29:10.835 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_cache_timeout", 2023-09-13T03:29:10.835 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:10.835 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.835 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.836 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1800", 2023-09-13T03:29:10.836 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.836 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.837 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.837 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache device inventory", 2023-09-13T03:29:10.837 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.838 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.838 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.838 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.839 INFO:teuthology.orchestra.run.smithi064.stdout: "device_enhanced_scan": { 2023-09-13T03:29:10.839 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_enhanced_scan", 2023-09-13T03:29:10.839 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:10.839 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.840 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.840 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:10.840 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.841 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.841 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.841 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use libstoragemgmt during device scans", 2023-09-13T03:29:10.842 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.842 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.842 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.842 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.843 INFO:teuthology.orchestra.run.smithi064.stdout: "facts_cache_timeout": { 2023-09-13T03:29:10.843 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "facts_cache_timeout", 2023-09-13T03:29:10.843 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:10.844 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.844 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.844 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2023-09-13T03:29:10.845 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.845 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.845 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.846 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache host facts data", 2023-09-13T03:29:10.846 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.846 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.846 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.847 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.847 INFO:teuthology.orchestra.run.smithi064.stdout: "host_check_interval": { 2023-09-13T03:29:10.847 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "host_check_interval", 2023-09-13T03:29:10.848 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:10.848 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.848 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.849 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2023-09-13T03:29:10.849 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.849 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.849 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.850 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to perform a host check", 2023-09-13T03:29:10.850 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.850 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.851 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.851 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.851 INFO:teuthology.orchestra.run.smithi064.stdout: "inventory_list_all": { 2023-09-13T03:29:10.852 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "inventory_list_all", 2023-09-13T03:29:10.852 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:10.852 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.853 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.853 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:10.853 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.853 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.854 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.854 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-09-13T03:29:10.854 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.855 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.855 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.855 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.855 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:10.856 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:10.856 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:10.856 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.857 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:10.857 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:10.857 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.858 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.858 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:10.858 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:10.859 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:10.859 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:10.859 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:10.860 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:10.860 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:10.860 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:10.861 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:10.861 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.861 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.862 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.862 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.863 INFO:teuthology.orchestra.run.smithi064.stdout: "log_refresh_metadata": { 2023-09-13T03:29:10.863 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_refresh_metadata", 2023-09-13T03:29:10.863 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:10.863 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.864 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.864 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:10.864 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.865 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.865 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.865 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-09-13T03:29:10.866 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.866 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.866 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.866 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.867 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:10.867 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:10.867 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:10.868 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.868 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.868 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:10.868 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.869 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.869 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.869 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-09-13T03:29:10.870 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.870 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.870 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.871 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.871 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:10.871 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:10.872 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:10.872 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.872 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:10.872 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:10.873 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.873 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.873 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:10.874 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:10.874 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:10.874 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:10.875 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:10.875 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:10.875 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:10.875 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:10.876 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:10.876 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.876 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.877 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.877 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.877 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:10.878 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:10.878 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:10.878 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.879 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:10.879 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:10.879 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.880 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.880 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.880 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:10.881 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.881 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.881 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.881 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.882 INFO:teuthology.orchestra.run.smithi064.stdout: "manage_etc_ceph_ceph_conf": { 2023-09-13T03:29:10.882 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-09-13T03:29:10.882 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:10.883 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.883 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.883 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:10.884 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.884 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.884 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.884 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-09-13T03:29:10.885 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.885 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.885 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.886 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.886 INFO:teuthology.orchestra.run.smithi064.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-09-13T03:29:10.886 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-09-13T03:29:10.887 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:10.887 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.887 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.887 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "*", 2023-09-13T03:29:10.888 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.888 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.888 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.889 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-09-13T03:29:10.889 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.889 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.889 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.890 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.890 INFO:teuthology.orchestra.run.smithi064.stdout: "max_count_per_host": { 2023-09-13T03:29:10.890 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_count_per_host", 2023-09-13T03:29:10.891 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:10.891 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.891 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.892 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2023-09-13T03:29:10.892 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.892 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.892 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.893 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "max number of daemons per service per host", 2023-09-13T03:29:10.893 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.893 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.894 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.894 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.894 INFO:teuthology.orchestra.run.smithi064.stdout: "max_osd_draining_count": { 2023-09-13T03:29:10.895 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_osd_draining_count", 2023-09-13T03:29:10.895 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:10.895 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.895 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.896 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2023-09-13T03:29:10.896 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.896 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.897 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.897 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-09-13T03:29:10.897 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.897 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.898 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.898 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.898 INFO:teuthology.orchestra.run.smithi064.stdout: "migration_current": { 2023-09-13T03:29:10.899 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "migration_current", 2023-09-13T03:29:10.899 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:10.899 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.900 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.900 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:10.900 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.900 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.901 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.901 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "internal - do not modify", 2023-09-13T03:29:10.901 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.902 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.902 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.902 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.903 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": { 2023-09-13T03:29:10.903 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mode", 2023-09-13T03:29:10.903 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:10.904 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.904 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.904 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "root", 2023-09-13T03:29:10.905 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.905 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.905 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:10.906 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm-package", 2023-09-13T03:29:10.906 INFO:teuthology.orchestra.run.smithi064.stdout: "root" 2023-09-13T03:29:10.906 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:10.906 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "mode for remote execution of cephadm", 2023-09-13T03:29:10.907 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.907 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.907 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.908 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.908 INFO:teuthology.orchestra.run.smithi064.stdout: "prometheus_alerts_path": { 2023-09-13T03:29:10.908 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prometheus_alerts_path", 2023-09-13T03:29:10.908 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:10.909 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.909 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.909 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-09-13T03:29:10.910 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.910 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.910 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.911 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-09-13T03:29:10.911 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.911 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.911 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.912 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.912 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_insecure": { 2023-09-13T03:29:10.912 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_insecure", 2023-09-13T03:29:10.913 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:10.913 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.913 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.914 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:10.914 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.914 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.914 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.915 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-09-13T03:29:10.915 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.915 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.916 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.916 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.916 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_password": { 2023-09-13T03:29:10.916 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_password", 2023-09-13T03:29:10.917 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:10.917 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.917 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.918 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:10.918 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.918 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.919 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.919 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-09-13T03:29:10.919 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.919 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.920 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.920 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.920 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_url": { 2023-09-13T03:29:10.921 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_url", 2023-09-13T03:29:10.921 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:10.921 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.922 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.922 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:10.922 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.923 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.923 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.923 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-09-13T03:29:10.923 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.924 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.924 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.924 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.925 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_username": { 2023-09-13T03:29:10.925 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_username", 2023-09-13T03:29:10.925 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:10.926 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.926 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.926 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:10.927 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.927 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.927 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.928 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-09-13T03:29:10.928 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.928 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.928 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.929 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.929 INFO:teuthology.orchestra.run.smithi064.stdout: "secure_monitoring_stack": { 2023-09-13T03:29:10.929 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "secure_monitoring_stack", 2023-09-13T03:29:10.930 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:10.930 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.930 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.931 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:10.931 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.931 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.931 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.932 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-09-13T03:29:10.932 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.932 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.933 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.933 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.933 INFO:teuthology.orchestra.run.smithi064.stdout: "service_discovery_port": { 2023-09-13T03:29:10.933 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "service_discovery_port", 2023-09-13T03:29:10.934 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:10.934 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.934 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.935 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8765", 2023-09-13T03:29:10.935 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.935 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.936 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.936 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "cephadm service discovery port", 2023-09-13T03:29:10.936 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.937 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.937 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.937 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.938 INFO:teuthology.orchestra.run.smithi064.stdout: "ssh_config_file": { 2023-09-13T03:29:10.938 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssh_config_file", 2023-09-13T03:29:10.938 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:10.939 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.939 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.939 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:10.940 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.940 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.940 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.941 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-09-13T03:29:10.941 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.941 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.941 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.942 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.942 INFO:teuthology.orchestra.run.smithi064.stdout: "use_agent": { 2023-09-13T03:29:10.942 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "use_agent", 2023-09-13T03:29:10.943 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:10.943 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.943 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.944 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:10.944 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.944 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.945 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.945 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-09-13T03:29:10.945 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.945 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.946 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.946 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.946 INFO:teuthology.orchestra.run.smithi064.stdout: "use_repo_digest": { 2023-09-13T03:29:10.947 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "use_repo_digest", 2023-09-13T03:29:10.947 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:10.947 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.948 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.948 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:10.948 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.948 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.949 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.949 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-09-13T03:29:10.949 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.950 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.950 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.950 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.951 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_failed_host_check": { 2023-09-13T03:29:10.951 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_failed_host_check", 2023-09-13T03:29:10.951 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:10.951 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.952 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.952 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:10.952 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.953 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.953 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.953 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if the host check fails", 2023-09-13T03:29:10.953 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.954 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.954 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.954 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.955 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_stray_daemons": { 2023-09-13T03:29:10.955 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_stray_daemons", 2023-09-13T03:29:10.955 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:10.955 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.956 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.956 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:10.956 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.957 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.957 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.957 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-09-13T03:29:10.958 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.958 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.958 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.959 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.959 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_stray_hosts": { 2023-09-13T03:29:10.959 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_stray_hosts", 2023-09-13T03:29:10.960 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:10.960 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.961 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:10.961 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:10.961 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.961 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.962 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.962 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-09-13T03:29:10.962 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.963 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.963 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.963 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:10.964 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:10.964 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.964 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:10.965 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crash", 2023-09-13T03:29:10.965 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:10.965 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:10.966 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:10.966 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:10.966 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:10.967 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:10.967 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.967 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:10.968 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:10.968 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.968 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.969 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:10.969 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:10.969 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:10.970 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:10.970 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:10.971 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:10.971 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:10.971 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:10.972 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:10.972 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.972 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.972 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.973 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.973 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:10.973 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:10.974 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:10.974 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.974 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:10.975 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:10.975 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.975 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.975 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.976 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:10.976 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.976 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.977 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.977 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.977 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:10.978 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:10.978 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:10.978 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.978 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:10.979 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:10.979 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.979 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.980 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:10.980 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:10.980 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:10.981 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:10.981 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:10.981 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:10.982 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:10.982 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:10.982 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:10.983 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.983 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.983 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.983 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.984 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:10.984 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:10.984 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:10.985 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.985 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:10.985 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:10.985 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.986 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.986 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.986 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:10.987 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.987 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.987 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.988 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.988 INFO:teuthology.orchestra.run.smithi064.stdout: "retain_interval": { 2023-09-13T03:29:10.988 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "retain_interval", 2023-09-13T03:29:10.988 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:10.989 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.989 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:10.989 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "31536000", 2023-09-13T03:29:10.990 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.990 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.990 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.991 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long to retain crashes before pruning them", 2023-09-13T03:29:10.991 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.991 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.992 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.992 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.992 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_recent_interval": { 2023-09-13T03:29:10.993 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_recent_interval", 2023-09-13T03:29:10.993 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:10.993 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:10.994 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:10.994 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1209600", 2023-09-13T03:29:10.994 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:10.995 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:10.995 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:10.995 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "time interval in which to warn about recent crashes", 2023-09-13T03:29:10.995 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:10.996 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:10.996 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:10.996 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:10.996 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:10.997 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:10.997 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:10.997 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "dashboard", 2023-09-13T03:29:10.998 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:10.998 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:10.998 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:10.999 INFO:teuthology.orchestra.run.smithi064.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-09-13T03:29:10.999 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-09-13T03:29:10.999 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:11.000 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.000 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.000 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2023-09-13T03:29:11.000 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.001 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.001 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.001 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.002 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.002 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.002 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.003 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.003 INFO:teuthology.orchestra.run.smithi064.stdout: "ALERTMANAGER_API_HOST": { 2023-09-13T03:29:11.003 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ALERTMANAGER_API_HOST", 2023-09-13T03:29:11.004 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.004 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.004 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.005 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.005 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.005 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.005 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.006 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.006 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.006 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.007 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.007 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.007 INFO:teuthology.orchestra.run.smithi064.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-09-13T03:29:11.008 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-09-13T03:29:11.008 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.008 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.008 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.009 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:11.009 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.009 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.010 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.010 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.010 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.011 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.011 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.011 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.011 INFO:teuthology.orchestra.run.smithi064.stdout: "AUDIT_API_ENABLED": { 2023-09-13T03:29:11.012 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "AUDIT_API_ENABLED", 2023-09-13T03:29:11.012 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.012 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.013 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.013 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.013 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.014 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.014 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.014 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.014 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.015 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.015 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.015 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.016 INFO:teuthology.orchestra.run.smithi064.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-09-13T03:29:11.016 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-09-13T03:29:11.016 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.017 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.017 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.017 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:11.017 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.018 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.018 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.018 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.019 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.019 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.019 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.020 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.020 INFO:teuthology.orchestra.run.smithi064.stdout: "ENABLE_BROWSABLE_API": { 2023-09-13T03:29:11.020 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ENABLE_BROWSABLE_API", 2023-09-13T03:29:11.021 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.021 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.021 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.021 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:11.022 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.022 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.022 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.023 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.023 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.023 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.024 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.024 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.024 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-09-13T03:29:11.025 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-09-13T03:29:11.025 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.025 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.025 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.026 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:11.026 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.026 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.027 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.027 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.027 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.028 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.028 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.028 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.028 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-09-13T03:29:11.029 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-09-13T03:29:11.029 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.029 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.030 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.030 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:11.030 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.031 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.031 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.031 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.031 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.032 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.032 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.032 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.033 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-09-13T03:29:11.033 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-09-13T03:29:11.033 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.034 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.034 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.034 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:11.034 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.035 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.035 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.035 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.036 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.036 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.036 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.036 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.037 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-09-13T03:29:11.037 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-09-13T03:29:11.037 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.038 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.038 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.038 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:11.039 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.039 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.039 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.039 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.040 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.040 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.040 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.041 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.041 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_NFS": { 2023-09-13T03:29:11.041 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-09-13T03:29:11.042 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.042 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.042 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.042 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:11.043 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.043 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.043 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.044 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.044 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.044 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.044 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.045 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.045 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_RBD": { 2023-09-13T03:29:11.045 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-09-13T03:29:11.046 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.046 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.046 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.047 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:11.047 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.047 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.047 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.048 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.048 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.048 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.049 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.049 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.049 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_RGW": { 2023-09-13T03:29:11.050 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-09-13T03:29:11.050 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.050 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.051 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.051 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:11.051 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.052 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.052 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.052 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.052 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.053 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.053 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.053 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.054 INFO:teuthology.orchestra.run.smithi064.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-09-13T03:29:11.054 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-09-13T03:29:11.054 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.055 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.055 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.055 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.055 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.056 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.056 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.056 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.057 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.057 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.057 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.058 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.058 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_PASSWORD": { 2023-09-13T03:29:11.058 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_PASSWORD", 2023-09-13T03:29:11.058 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.059 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.059 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.059 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2023-09-13T03:29:11.060 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.060 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.060 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.061 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.061 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.061 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.061 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.062 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.062 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-09-13T03:29:11.062 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-09-13T03:29:11.063 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.063 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.063 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.064 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:11.064 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.064 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.065 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.065 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.066 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.066 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.066 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.067 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.067 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_URL": { 2023-09-13T03:29:11.067 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_URL", 2023-09-13T03:29:11.068 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.068 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.068 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.068 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.069 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.069 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.069 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.070 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.070 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.070 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.071 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.071 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.071 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_USERNAME": { 2023-09-13T03:29:11.072 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_USERNAME", 2023-09-13T03:29:11.072 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.072 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.073 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.073 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2023-09-13T03:29:11.073 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.074 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.074 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.074 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.075 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.075 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.075 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.076 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.076 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-09-13T03:29:11.076 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-09-13T03:29:11.076 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.077 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.077 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.077 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.078 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.078 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.078 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.078 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.079 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.079 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.079 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.080 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.080 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-09-13T03:29:11.080 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-09-13T03:29:11.080 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.081 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.081 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.081 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.082 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.082 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.082 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.082 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.083 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.083 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.083 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.084 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.084 INFO:teuthology.orchestra.run.smithi064.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-09-13T03:29:11.084 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-09-13T03:29:11.085 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.085 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.085 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.085 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:11.086 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.086 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.086 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.087 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.087 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.087 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.088 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.088 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.088 INFO:teuthology.orchestra.run.smithi064.stdout: "ISSUE_TRACKER_API_KEY": { 2023-09-13T03:29:11.088 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-09-13T03:29:11.089 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.089 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.089 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.090 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.090 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.090 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.091 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.091 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.091 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.091 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.092 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.092 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.092 INFO:teuthology.orchestra.run.smithi064.stdout: "PROMETHEUS_API_HOST": { 2023-09-13T03:29:11.093 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PROMETHEUS_API_HOST", 2023-09-13T03:29:11.093 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.093 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.093 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.094 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.094 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.094 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.095 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.095 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.095 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.095 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.096 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.096 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.096 INFO:teuthology.orchestra.run.smithi064.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-09-13T03:29:11.096 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-09-13T03:29:11.097 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.097 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.097 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.098 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:11.098 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.098 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.098 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.099 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.099 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.099 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.100 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.100 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.100 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-09-13T03:29:11.100 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-09-13T03:29:11.101 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.101 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.101 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.101 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.102 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.102 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.102 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.103 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.103 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.103 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.103 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.104 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.104 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-09-13T03:29:11.104 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-09-13T03:29:11.105 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.105 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.105 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.105 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.106 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.106 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.106 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.107 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.107 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.107 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.108 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.108 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.108 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-09-13T03:29:11.108 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-09-13T03:29:11.109 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.109 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.109 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.110 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:11.110 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.110 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.111 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.111 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.111 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.111 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.112 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.112 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.112 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-09-13T03:29:11.113 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-09-13T03:29:11.113 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.113 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.113 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.114 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:11.114 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.114 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.115 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.115 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.115 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.116 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.116 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.116 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.116 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-09-13T03:29:11.117 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-09-13T03:29:11.117 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.117 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.118 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.118 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.118 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.119 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.119 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.119 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.119 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.120 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.120 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.120 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.121 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-09-13T03:29:11.121 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-09-13T03:29:11.121 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.121 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.122 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.122 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.122 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.123 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.123 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.123 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.124 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.124 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.124 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.124 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.125 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-09-13T03:29:11.125 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-09-13T03:29:11.125 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.126 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.126 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.126 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.127 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.127 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.127 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.128 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.128 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.128 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.129 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.129 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.129 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_ENABLED": { 2023-09-13T03:29:11.130 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_ENABLED", 2023-09-13T03:29:11.130 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.130 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.131 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.131 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:11.131 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.132 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.132 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.132 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.132 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.133 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.133 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.134 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-09-13T03:29:11.134 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-09-13T03:29:11.134 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.135 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.135 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.135 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-09-13T03:29:11.136 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.136 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.136 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.137 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.137 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.137 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.137 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.138 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-09-13T03:29:11.138 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-09-13T03:29:11.139 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:11.139 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.139 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.140 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2023-09-13T03:29:11.140 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.140 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.140 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.141 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.141 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.141 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.142 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.142 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.142 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-09-13T03:29:11.142 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-09-13T03:29:11.143 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:11.143 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.143 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.144 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8", 2023-09-13T03:29:11.144 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.144 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.144 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.145 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.145 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.145 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.145 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.146 INFO:teuthology.orchestra.run.smithi064.stdout: "REST_REQUESTS_TIMEOUT": { 2023-09-13T03:29:11.146 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-09-13T03:29:11.147 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:11.147 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.147 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.148 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "45", 2023-09-13T03:29:11.148 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.148 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.148 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.149 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.149 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.149 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.150 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.150 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_ACCESS_KEY": { 2023-09-13T03:29:11.151 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_ACCESS_KEY", 2023-09-13T03:29:11.151 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.151 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.151 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.152 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.152 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.152 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.152 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.153 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.153 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.153 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.154 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.154 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-09-13T03:29:11.155 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-09-13T03:29:11.155 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.155 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.155 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.156 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2023-09-13T03:29:11.156 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.156 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.156 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.157 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.157 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.157 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.158 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.158 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.158 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_SECRET_KEY": { 2023-09-13T03:29:11.159 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_SECRET_KEY", 2023-09-13T03:29:11.159 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.159 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.160 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.160 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.160 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.161 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.161 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.161 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.161 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.162 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.162 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.162 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.163 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_SSL_VERIFY": { 2023-09-13T03:29:11.163 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_SSL_VERIFY", 2023-09-13T03:29:11.163 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.164 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.164 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.164 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:11.165 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.165 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.165 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.165 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.166 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.166 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.166 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.167 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.167 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-09-13T03:29:11.167 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-09-13T03:29:11.168 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:11.168 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.168 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.169 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2023-09-13T03:29:11.169 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.169 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.170 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.170 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.170 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.171 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.171 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.171 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.172 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-09-13T03:29:11.172 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-09-13T03:29:11.172 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:11.172 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.173 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.173 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2023-09-13T03:29:11.173 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.174 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.174 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.174 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.175 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.175 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.175 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.176 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.176 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-09-13T03:29:11.176 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-09-13T03:29:11.176 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:11.177 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.177 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.177 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2023-09-13T03:29:11.178 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.178 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.178 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.178 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.179 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.179 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.179 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.180 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.180 INFO:teuthology.orchestra.run.smithi064.stdout: "cross_origin_url": { 2023-09-13T03:29:11.180 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cross_origin_url", 2023-09-13T03:29:11.180 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.181 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.181 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.181 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.182 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.182 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.182 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.182 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.183 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.183 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.183 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.184 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.184 INFO:teuthology.orchestra.run.smithi064.stdout: "crt_file": { 2023-09-13T03:29:11.184 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crt_file", 2023-09-13T03:29:11.184 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.185 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.185 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.185 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.185 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.186 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.186 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.186 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.187 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.187 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.187 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.188 INFO:teuthology.orchestra.run.smithi064.stdout: "debug": { 2023-09-13T03:29:11.188 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "debug", 2023-09-13T03:29:11.188 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.189 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.189 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.189 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.190 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.190 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.190 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.191 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable/disable debug options", 2023-09-13T03:29:11.191 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.191 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.191 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.192 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.192 INFO:teuthology.orchestra.run.smithi064.stdout: "jwt_token_ttl": { 2023-09-13T03:29:11.192 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "jwt_token_ttl", 2023-09-13T03:29:11.193 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:11.193 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.193 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.193 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "28800", 2023-09-13T03:29:11.194 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.194 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.194 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.195 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.195 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.195 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.195 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.196 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.196 INFO:teuthology.orchestra.run.smithi064.stdout: "key_file": { 2023-09-13T03:29:11.196 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "key_file", 2023-09-13T03:29:11.196 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.197 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.197 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.197 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.198 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.198 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.198 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.198 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.199 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.199 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.199 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.200 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.200 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:11.200 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:11.200 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.201 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.201 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.201 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.201 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.202 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.202 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.202 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.203 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.203 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.203 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.204 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.204 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.204 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.204 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.205 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.205 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.205 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.206 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.206 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:11.206 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:11.207 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.207 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.207 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.208 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.208 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.208 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.208 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.209 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.209 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.209 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.210 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.210 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.210 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:11.211 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:11.211 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.211 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.211 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.212 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:11.212 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.212 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.213 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.213 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.213 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.213 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.214 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.214 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.214 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.215 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.215 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.215 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.215 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.216 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.216 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.216 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:11.217 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:11.217 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.217 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.217 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.218 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.218 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.218 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.219 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.219 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.219 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.220 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.220 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.220 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.221 INFO:teuthology.orchestra.run.smithi064.stdout: "motd": { 2023-09-13T03:29:11.221 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "motd", 2023-09-13T03:29:11.221 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.222 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.222 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.222 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.223 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.223 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.223 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.223 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "The message of the day", 2023-09-13T03:29:11.224 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.224 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.224 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.225 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.225 INFO:teuthology.orchestra.run.smithi064.stdout: "redirect_resolve_ip_addr": { 2023-09-13T03:29:11.226 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "redirect_resolve_ip_addr", 2023-09-13T03:29:11.226 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.226 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.227 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.227 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.227 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.227 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.228 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.228 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.228 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.229 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.229 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.229 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.230 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2023-09-13T03:29:11.230 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2023-09-13T03:29:11.230 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.230 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.231 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.231 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "::", 2023-09-13T03:29:11.231 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.232 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.232 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.232 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.233 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.233 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.233 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.233 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.234 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2023-09-13T03:29:11.234 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2023-09-13T03:29:11.234 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:11.234 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.235 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.235 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8080", 2023-09-13T03:29:11.235 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.236 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.236 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.236 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.236 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.237 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.237 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.237 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.238 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl": { 2023-09-13T03:29:11.238 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl", 2023-09-13T03:29:11.238 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.239 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.239 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.239 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:11.240 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.240 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.240 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.241 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.241 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.241 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.242 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.242 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.242 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl_server_port": { 2023-09-13T03:29:11.242 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl_server_port", 2023-09-13T03:29:11.243 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:11.243 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.243 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.244 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8443", 2023-09-13T03:29:11.244 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.244 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.244 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.245 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.245 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.245 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.246 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.246 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.246 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_behaviour": { 2023-09-13T03:29:11.247 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_behaviour", 2023-09-13T03:29:11.247 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.247 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.247 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.248 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "redirect", 2023-09-13T03:29:11.248 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.248 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.249 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.249 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.249 INFO:teuthology.orchestra.run.smithi064.stdout: "redirect" 2023-09-13T03:29:11.250 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.250 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.250 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.251 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.251 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.251 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.252 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_error_status_code": { 2023-09-13T03:29:11.252 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_error_status_code", 2023-09-13T03:29:11.252 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:11.252 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.253 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.253 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "500", 2023-09-13T03:29:11.253 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "400", 2023-09-13T03:29:11.254 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "599", 2023-09-13T03:29:11.254 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.254 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.255 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.255 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.255 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.256 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.256 INFO:teuthology.orchestra.run.smithi064.stdout: "url_prefix": { 2023-09-13T03:29:11.256 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "url_prefix", 2023-09-13T03:29:11.257 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.257 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.257 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.257 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.258 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.258 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.258 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.259 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.259 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.259 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.260 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.260 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:11.260 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:11.261 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.261 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:11.261 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "devicehealth", 2023-09-13T03:29:11.262 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:11.262 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:11.262 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:11.263 INFO:teuthology.orchestra.run.smithi064.stdout: "enable_monitoring": { 2023-09-13T03:29:11.263 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enable_monitoring", 2023-09-13T03:29:11.263 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.264 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.264 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.264 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:11.265 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.265 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.265 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.265 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "monitor device health metrics", 2023-09-13T03:29:11.266 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.266 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.266 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.267 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.267 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:11.267 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:11.268 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.268 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.268 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.269 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.269 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.269 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.269 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.270 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.270 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.271 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.271 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.271 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.271 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.272 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.272 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.272 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.273 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.273 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.273 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.274 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:11.274 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:11.274 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.275 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.275 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.275 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.276 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.276 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.276 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.277 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.277 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.277 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.278 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.278 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.278 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:11.278 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:11.279 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.279 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.279 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.280 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:11.280 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.280 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.281 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.281 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.281 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.282 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.282 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.282 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.282 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.283 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.283 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.283 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.284 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.284 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.284 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.285 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:11.285 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:11.285 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.286 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.286 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.286 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.287 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.287 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.287 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.287 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.288 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.288 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.288 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.289 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.289 INFO:teuthology.orchestra.run.smithi064.stdout: "mark_out_threshold": { 2023-09-13T03:29:11.289 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mark_out_threshold", 2023-09-13T03:29:11.290 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:11.290 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.291 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.291 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "2419200", 2023-09-13T03:29:11.291 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.291 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.292 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.292 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-09-13T03:29:11.292 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.293 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.293 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.293 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.293 INFO:teuthology.orchestra.run.smithi064.stdout: "pool_name": { 2023-09-13T03:29:11.294 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pool_name", 2023-09-13T03:29:11.294 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.295 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.295 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.295 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "device_health_metrics", 2023-09-13T03:29:11.295 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.296 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.296 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.296 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "name of pool in which to store device health metrics", 2023-09-13T03:29:11.297 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.297 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.297 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.298 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.298 INFO:teuthology.orchestra.run.smithi064.stdout: "retention_period": { 2023-09-13T03:29:11.298 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "retention_period", 2023-09-13T03:29:11.299 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:11.299 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.299 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.299 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15552000", 2023-09-13T03:29:11.300 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.300 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.300 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.301 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long to retain device health metrics", 2023-09-13T03:29:11.301 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.301 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.301 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.302 INFO:teuthology.orchestra.run.smithi064.stdout: "scrape_frequency": { 2023-09-13T03:29:11.302 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "scrape_frequency", 2023-09-13T03:29:11.303 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:11.303 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.303 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.304 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "86400", 2023-09-13T03:29:11.304 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.304 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.304 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.305 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to scrape device health metrics", 2023-09-13T03:29:11.305 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.305 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.306 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.306 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.306 INFO:teuthology.orchestra.run.smithi064.stdout: "self_heal": { 2023-09-13T03:29:11.307 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "self_heal", 2023-09-13T03:29:11.307 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.307 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.308 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.308 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:11.308 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.309 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.309 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.309 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-09-13T03:29:11.309 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.310 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.310 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.310 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.311 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2023-09-13T03:29:11.311 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2023-09-13T03:29:11.311 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:11.312 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.312 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.312 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2023-09-13T03:29:11.312 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.313 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.313 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.313 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to wake up and check device health", 2023-09-13T03:29:11.314 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.314 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.314 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.315 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.315 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_threshold": { 2023-09-13T03:29:11.315 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_threshold", 2023-09-13T03:29:11.316 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:11.316 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.316 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.316 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "7257600", 2023-09-13T03:29:11.317 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.317 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.317 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.318 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-09-13T03:29:11.318 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.318 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.319 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.319 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:11.319 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:11.320 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.320 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:11.320 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "diskprediction_local", 2023-09-13T03:29:11.321 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:11.321 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:11.321 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:11.322 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:11.322 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:11.322 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.323 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.323 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.323 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.323 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.324 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.324 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.324 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.325 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.325 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.325 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.325 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.326 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.326 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.326 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.327 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.327 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.327 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.328 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.328 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:11.328 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:11.328 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.329 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.329 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.329 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.330 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.330 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.330 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.331 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.331 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.331 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.332 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.332 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.332 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:11.332 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:11.333 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.333 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.333 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.334 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:11.334 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.334 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.334 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.335 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.335 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.335 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.335 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.336 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.336 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.336 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.337 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.337 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.337 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.337 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.338 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.338 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:11.338 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:11.339 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.339 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.339 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.339 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.340 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.340 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.340 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.341 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.341 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.341 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.342 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.342 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.342 INFO:teuthology.orchestra.run.smithi064.stdout: "predict_interval": { 2023-09-13T03:29:11.343 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "predict_interval", 2023-09-13T03:29:11.343 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.343 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.343 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.344 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "86400", 2023-09-13T03:29:11.344 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.344 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.345 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.345 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.345 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.346 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.346 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.346 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.347 INFO:teuthology.orchestra.run.smithi064.stdout: "predictor_model": { 2023-09-13T03:29:11.347 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "predictor_model", 2023-09-13T03:29:11.347 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.347 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.348 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.348 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "prophetstor", 2023-09-13T03:29:11.348 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.349 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.349 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.349 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.350 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.350 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.350 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.351 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.351 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2023-09-13T03:29:11.351 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2023-09-13T03:29:11.351 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.352 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.352 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.352 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2023-09-13T03:29:11.353 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.353 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.353 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.353 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.354 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.354 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.354 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.355 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:11.355 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:11.355 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.356 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:11.356 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "influx", 2023-09-13T03:29:11.356 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": false, 2023-09-13T03:29:11.356 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "influxdb python module not found", 2023-09-13T03:29:11.357 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:11.357 INFO:teuthology.orchestra.run.smithi064.stdout: "batch_size": { 2023-09-13T03:29:11.357 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "batch_size", 2023-09-13T03:29:11.358 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:11.358 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.358 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.359 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5000", 2023-09-13T03:29:11.359 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.359 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.360 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.360 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-09-13T03:29:11.360 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.361 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.361 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.361 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.361 INFO:teuthology.orchestra.run.smithi064.stdout: "database": { 2023-09-13T03:29:11.362 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "database", 2023-09-13T03:29:11.362 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.362 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.363 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.363 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "ceph", 2023-09-13T03:29:11.363 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.364 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.364 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.364 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-09-13T03:29:11.364 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.365 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.365 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.365 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.365 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": { 2023-09-13T03:29:11.366 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "hostname", 2023-09-13T03:29:11.366 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.366 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.367 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.367 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.367 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.368 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.368 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.368 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB server hostname", 2023-09-13T03:29:11.369 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.369 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.369 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.369 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.370 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2023-09-13T03:29:11.370 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2023-09-13T03:29:11.370 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:11.371 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.371 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.371 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "30", 2023-09-13T03:29:11.372 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "5", 2023-09-13T03:29:11.372 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.372 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.373 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-09-13T03:29:11.373 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.373 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.373 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.374 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.374 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:11.374 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:11.375 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.375 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.375 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.375 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.376 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.376 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.376 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.376 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.377 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.377 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.377 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.378 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.378 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.378 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.379 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.379 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.379 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.379 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.380 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.380 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:11.381 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:11.381 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.381 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.381 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.382 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.382 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.382 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.383 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.383 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.383 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.383 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.384 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.384 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.384 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:11.384 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:11.385 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.385 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.386 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.386 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:11.386 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.387 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.387 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.387 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.388 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.388 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.388 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.388 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.389 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.389 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.389 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.390 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.390 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.390 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.391 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.391 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:11.391 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:11.392 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.392 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.392 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.393 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.393 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.393 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.393 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.394 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.394 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.395 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.395 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.395 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.396 INFO:teuthology.orchestra.run.smithi064.stdout: "password": { 2023-09-13T03:29:11.396 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "password", 2023-09-13T03:29:11.396 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.397 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.397 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.397 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.398 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.398 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.398 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.399 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "password of InfluxDB server user", 2023-09-13T03:29:11.399 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.399 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.400 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.400 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.400 INFO:teuthology.orchestra.run.smithi064.stdout: "port": { 2023-09-13T03:29:11.401 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "port", 2023-09-13T03:29:11.401 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:11.401 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.401 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.402 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8086", 2023-09-13T03:29:11.402 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.402 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.403 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.403 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB server port", 2023-09-13T03:29:11.403 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.404 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.404 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.404 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.404 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl": { 2023-09-13T03:29:11.405 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl", 2023-09-13T03:29:11.405 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.406 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.406 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.406 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "false", 2023-09-13T03:29:11.406 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.407 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.407 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.407 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-09-13T03:29:11.408 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.408 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.408 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.409 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.409 INFO:teuthology.orchestra.run.smithi064.stdout: "threads": { 2023-09-13T03:29:11.410 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "threads", 2023-09-13T03:29:11.410 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:11.410 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.411 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.411 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2023-09-13T03:29:11.411 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2023-09-13T03:29:11.411 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "32", 2023-09-13T03:29:11.412 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.412 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-09-13T03:29:11.413 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.413 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.413 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.414 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.414 INFO:teuthology.orchestra.run.smithi064.stdout: "username": { 2023-09-13T03:29:11.414 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "username", 2023-09-13T03:29:11.415 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.415 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.415 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.416 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.416 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.416 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.416 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.417 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "username of InfluxDB server user", 2023-09-13T03:29:11.417 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.417 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.418 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.418 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.418 INFO:teuthology.orchestra.run.smithi064.stdout: "verify_ssl": { 2023-09-13T03:29:11.418 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "verify_ssl", 2023-09-13T03:29:11.419 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.419 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.419 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.420 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "true", 2023-09-13T03:29:11.420 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.420 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.421 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.421 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-09-13T03:29:11.421 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.421 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.422 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.422 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:11.422 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:11.423 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.423 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:11.423 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "insights", 2023-09-13T03:29:11.424 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:11.424 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:11.424 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:11.425 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:11.425 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:11.425 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.426 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.426 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.426 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.427 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.427 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.427 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.427 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.428 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.428 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.428 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.429 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.429 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.429 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.430 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.430 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.430 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.431 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.431 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.431 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:11.432 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:11.432 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.432 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.432 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.433 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.433 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.434 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.434 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.434 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.434 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.435 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.435 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.435 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.436 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:11.436 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:11.436 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.436 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.437 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.437 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:11.437 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.438 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.438 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.438 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.439 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.439 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.439 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.439 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.440 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.440 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.440 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.441 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.441 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.441 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.442 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.442 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:11.442 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:11.442 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.443 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.443 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.443 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.444 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.444 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.444 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.445 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.445 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.445 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.446 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.446 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:11.446 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:11.447 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.447 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:11.447 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "iostat", 2023-09-13T03:29:11.448 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:11.448 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:11.448 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:11.448 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:11.449 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:11.449 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.449 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.450 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.450 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.450 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.451 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.451 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.451 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.451 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.452 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.452 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.452 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.453 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.453 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.453 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.453 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.454 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.454 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.454 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.455 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:11.455 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:11.455 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.455 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.456 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.456 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.456 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.457 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.457 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.457 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.458 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.458 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.458 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.458 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.459 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:11.459 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:11.459 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.460 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.460 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.460 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:11.461 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.461 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.461 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.462 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.462 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.462 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.463 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.463 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.463 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.463 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.464 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.464 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.464 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.465 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.465 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.465 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:11.465 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:11.466 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.466 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.467 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.467 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.467 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.468 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.468 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.468 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.468 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.469 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.469 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.469 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:11.470 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:11.470 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.470 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:11.471 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "k8sevents", 2023-09-13T03:29:11.471 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:11.471 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:11.471 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:11.472 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_event_retention_days": { 2023-09-13T03:29:11.472 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ceph_event_retention_days", 2023-09-13T03:29:11.472 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:11.472 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.473 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.473 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "7", 2023-09-13T03:29:11.473 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.474 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.474 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.474 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Days to hold ceph event information within local cache", 2023-09-13T03:29:11.475 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.475 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.475 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.475 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.476 INFO:teuthology.orchestra.run.smithi064.stdout: "config_check_secs": { 2023-09-13T03:29:11.476 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_check_secs", 2023-09-13T03:29:11.476 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:11.476 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.477 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.477 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2023-09-13T03:29:11.477 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "10", 2023-09-13T03:29:11.478 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.478 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.478 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-09-13T03:29:11.479 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.479 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.479 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.480 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.480 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:11.480 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:11.481 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.481 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.481 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.482 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.482 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.482 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.483 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.483 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.483 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.484 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.484 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.484 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.484 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.485 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.485 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.485 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.486 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.486 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.486 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.487 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:11.487 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:11.487 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.488 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.488 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.488 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.489 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.489 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.489 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.490 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.490 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.490 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.491 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.491 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.491 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:11.492 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:11.492 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.492 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.493 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.493 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:11.493 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.494 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.494 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.494 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.494 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.495 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.495 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.495 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.496 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.496 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.496 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.496 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.497 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.497 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.497 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.498 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:11.498 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:11.498 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.499 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.499 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.499 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.500 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.500 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.500 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.500 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.501 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.501 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.501 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.502 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:11.502 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:11.503 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.503 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:11.504 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "localpool", 2023-09-13T03:29:11.504 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:11.504 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:11.504 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:11.505 INFO:teuthology.orchestra.run.smithi064.stdout: "failure_domain": { 2023-09-13T03:29:11.505 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "failure_domain", 2023-09-13T03:29:11.505 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.506 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.506 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.507 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "host", 2023-09-13T03:29:11.507 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.507 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.508 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.508 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "failure domain for any created local pool", 2023-09-13T03:29:11.508 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-09-13T03:29:11.509 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.509 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.509 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.510 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:11.510 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:11.510 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.511 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.511 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.512 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.512 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.513 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.513 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.514 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.514 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.515 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.515 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.516 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.516 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.516 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.516 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.517 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.517 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.517 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.518 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.518 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:11.518 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:11.518 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.519 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.519 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.519 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.520 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.520 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.520 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.520 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.521 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.521 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.521 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.522 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.522 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:11.522 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:11.522 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.523 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.523 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.523 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:11.524 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.524 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.524 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.524 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.525 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.525 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.525 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.526 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.526 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.526 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.526 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.527 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.527 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.527 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.528 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.528 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:11.528 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:11.528 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.529 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.529 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.529 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.530 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.530 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.530 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.530 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.531 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.531 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.531 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.532 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.532 INFO:teuthology.orchestra.run.smithi064.stdout: "min_size": { 2023-09-13T03:29:11.532 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "min_size", 2023-09-13T03:29:11.532 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:11.533 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.533 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.533 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.533 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.534 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.534 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.534 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default min_size for any created local pool", 2023-09-13T03:29:11.535 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-09-13T03:29:11.535 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.535 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.535 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.536 INFO:teuthology.orchestra.run.smithi064.stdout: "num_rep": { 2023-09-13T03:29:11.536 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "num_rep", 2023-09-13T03:29:11.536 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:11.537 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.537 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.537 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3", 2023-09-13T03:29:11.537 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.538 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.538 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.538 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default replica count for any created local pool", 2023-09-13T03:29:11.538 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.539 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.539 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.539 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.540 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_num": { 2023-09-13T03:29:11.540 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pg_num", 2023-09-13T03:29:11.540 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:11.540 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.541 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.541 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "128", 2023-09-13T03:29:11.541 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.542 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.542 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.542 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default pg_num for any created local pool", 2023-09-13T03:29:11.542 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.543 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.543 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.543 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.544 INFO:teuthology.orchestra.run.smithi064.stdout: "prefix": { 2023-09-13T03:29:11.544 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prefix", 2023-09-13T03:29:11.544 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.544 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.545 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.545 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.545 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.545 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.546 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.546 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "name prefix for any created local pool", 2023-09-13T03:29:11.546 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.547 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.547 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.548 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.549 INFO:teuthology.orchestra.run.smithi064.stdout: "subtree": { 2023-09-13T03:29:11.549 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "subtree", 2023-09-13T03:29:11.550 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.550 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.550 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.550 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "rack", 2023-09-13T03:29:11.551 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.551 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.551 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.552 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "CRUSH level for which to create a local pool", 2023-09-13T03:29:11.552 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-09-13T03:29:11.552 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.553 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.553 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:11.553 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:11.553 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.554 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:11.554 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mds_autoscaler", 2023-09-13T03:29:11.554 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:11.555 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:11.555 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:11.555 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:11.555 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:11.556 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.556 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.556 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.557 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.557 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.557 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.557 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.558 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.558 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.558 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.559 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.559 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.559 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.559 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.560 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.560 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.560 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.560 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.561 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.561 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:11.561 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:11.562 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.562 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.562 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.562 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.563 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.563 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.563 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.564 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.564 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.564 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.564 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.565 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.565 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:11.565 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:11.566 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.566 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.566 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.566 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:11.567 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.567 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.567 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.567 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.568 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.568 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.568 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.569 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.569 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.569 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.569 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.570 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.570 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.570 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.571 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.571 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:11.571 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:11.571 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.572 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.572 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.572 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.572 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.573 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.573 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.573 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.573 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.574 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.574 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.574 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:11.575 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:11.575 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.575 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:11.575 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mirroring", 2023-09-13T03:29:11.576 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:11.576 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:11.576 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:11.577 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:11.577 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:11.577 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.577 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.578 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.578 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.578 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.579 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.579 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.579 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.579 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.580 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.580 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.580 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.581 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.581 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.581 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.581 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.582 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.582 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.582 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.583 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:11.583 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:11.583 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.583 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.584 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.584 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.584 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.585 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.585 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.585 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.586 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.586 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.586 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.586 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.587 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:11.587 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:11.587 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.587 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.588 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.588 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:11.588 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.589 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.589 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.589 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.589 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.590 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.590 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.590 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.591 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.591 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.591 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.591 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.592 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.592 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.592 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.592 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:11.593 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:11.593 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.593 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.593 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.594 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.594 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.594 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.595 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.595 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.595 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.595 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.596 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.596 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:11.596 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:11.597 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.597 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:11.597 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "nfs", 2023-09-13T03:29:11.597 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:11.598 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:11.598 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:11.598 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:11.599 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:11.599 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.599 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.599 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.600 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.600 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.600 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.600 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.601 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.601 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.601 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.602 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.602 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.602 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.603 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.603 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.603 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.603 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.604 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.604 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.604 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:11.604 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:11.605 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.605 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.605 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.606 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.606 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.606 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.606 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.607 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.607 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.607 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.608 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.608 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.608 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:11.608 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:11.609 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.609 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.609 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.609 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:11.610 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.610 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.610 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.611 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.611 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.611 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.611 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.612 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.612 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.612 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.612 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.613 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.613 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.613 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.613 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.614 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:11.614 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:11.614 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.615 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.615 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.615 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.615 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.616 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.616 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.616 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.616 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.617 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.617 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.617 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:11.618 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:11.618 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.618 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:11.618 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "orchestrator", 2023-09-13T03:29:11.619 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:11.619 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:11.619 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:11.620 INFO:teuthology.orchestra.run.smithi064.stdout: "fail_fs": { 2023-09-13T03:29:11.620 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "fail_fs", 2023-09-13T03:29:11.620 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.620 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.621 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.621 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.621 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.621 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.622 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.622 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-09-13T03:29:11.622 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.623 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.623 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.623 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.623 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:11.624 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:11.624 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.624 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.624 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.625 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.625 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.625 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.626 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.626 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.626 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.626 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.627 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.627 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.627 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.627 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.628 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.628 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.628 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.629 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.629 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.629 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:11.629 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:11.630 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.630 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.630 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.630 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.631 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.631 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.631 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.632 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.632 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.632 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.632 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.633 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.633 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:11.633 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:11.633 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.634 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.634 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.634 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:11.635 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.635 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.635 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.635 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.636 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.636 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.636 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.637 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.637 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.637 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.637 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.638 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.638 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.638 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.639 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.639 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:11.639 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:11.639 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.640 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.640 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.640 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.640 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.641 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.641 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.641 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.642 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.642 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.642 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.642 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.643 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator": { 2023-09-13T03:29:11.643 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "orchestrator", 2023-09-13T03:29:11.643 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.644 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.644 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.644 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.644 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.645 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.645 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.645 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm", 2023-09-13T03:29:11.645 INFO:teuthology.orchestra.run.smithi064.stdout: "rook", 2023-09-13T03:29:11.646 INFO:teuthology.orchestra.run.smithi064.stdout: "test_orchestrator" 2023-09-13T03:29:11.646 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.646 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Orchestrator backend", 2023-09-13T03:29:11.647 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.647 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.647 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.647 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:11.648 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:11.648 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.648 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:11.649 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "osd_perf_query", 2023-09-13T03:29:11.649 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:11.649 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:11.649 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:11.650 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:11.650 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:11.650 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.650 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.651 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.651 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.651 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.652 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.652 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.652 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.652 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.653 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.653 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.653 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.653 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.654 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.654 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.654 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.655 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.655 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.655 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.655 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:11.656 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:11.656 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.656 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.656 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.657 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.657 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.657 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.658 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.658 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.658 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.658 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.659 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.659 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.659 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:11.659 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:11.660 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.660 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.660 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.661 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:11.661 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.661 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.661 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.662 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.662 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.662 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.662 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.663 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.663 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.663 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.664 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.664 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.664 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.664 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.665 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.665 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:11.665 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:11.665 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.666 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.666 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.666 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.667 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.667 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.667 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.667 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.668 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.668 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.668 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.669 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:11.669 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:11.669 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.669 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:11.670 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "osd_support", 2023-09-13T03:29:11.670 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:11.670 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:11.671 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:11.671 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:11.671 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:11.671 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.672 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.672 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.672 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.673 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.673 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.673 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.674 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.674 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.674 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.674 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.675 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.675 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.675 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.676 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.676 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.676 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.676 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.677 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.677 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:11.677 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:11.677 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.678 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.678 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.678 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.679 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.679 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.679 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.679 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.680 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.680 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.680 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.680 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.681 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:11.681 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:11.681 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.682 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.682 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.682 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:11.682 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.683 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.683 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.683 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.683 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.684 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.684 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.684 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.685 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.685 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.685 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.685 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.686 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.686 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.686 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.686 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:11.687 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:11.687 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.687 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.688 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.688 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.688 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.688 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.689 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.689 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.689 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.689 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.690 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.690 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:11.690 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:11.691 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.691 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:11.691 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pg_autoscaler", 2023-09-13T03:29:11.691 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:11.692 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:11.692 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:11.692 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:11.693 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:11.693 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.693 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.693 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.694 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.694 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.694 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.695 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.695 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.695 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.695 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.696 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.696 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.696 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.696 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.697 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.697 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.697 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.698 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.698 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.698 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:11.698 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:11.699 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.699 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.699 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.700 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.700 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.700 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.700 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.701 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.701 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.701 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.701 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.702 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.702 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:11.702 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:11.703 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.703 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.703 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.703 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:11.704 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.704 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.704 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.705 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.705 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.705 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.705 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.706 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.706 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.706 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.707 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.707 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.707 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.707 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.708 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.708 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:11.708 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:11.708 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.709 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.709 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.709 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.710 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.710 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.710 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.710 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.711 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.711 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.711 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.712 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.712 INFO:teuthology.orchestra.run.smithi064.stdout: "noautoscale": { 2023-09-13T03:29:11.712 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "noautoscale", 2023-09-13T03:29:11.712 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.713 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.713 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.713 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.714 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.714 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.714 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.714 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "global autoscale flag", 2023-09-13T03:29:11.715 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-09-13T03:29:11.715 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.715 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.716 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.716 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2023-09-13T03:29:11.716 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2023-09-13T03:29:11.716 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:11.717 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.717 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.717 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2023-09-13T03:29:11.718 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.718 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.718 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.718 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.719 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.719 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.719 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.720 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.720 INFO:teuthology.orchestra.run.smithi064.stdout: "threshold": { 2023-09-13T03:29:11.720 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "threshold", 2023-09-13T03:29:11.720 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2023-09-13T03:29:11.721 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.721 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.721 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3.0", 2023-09-13T03:29:11.721 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1.0", 2023-09-13T03:29:11.722 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.722 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.722 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "scaling threshold", 2023-09-13T03:29:11.723 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-09-13T03:29:11.723 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.723 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.723 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:11.724 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:11.724 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.724 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:11.725 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "progress", 2023-09-13T03:29:11.725 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:11.725 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:11.725 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:11.726 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_pg_recovery_event": { 2023-09-13T03:29:11.726 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_pg_recovery_event", 2023-09-13T03:29:11.726 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.727 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.727 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.727 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.727 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.728 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.728 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.728 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow the module to show pg recovery progress", 2023-09-13T03:29:11.729 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.729 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.729 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.729 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.730 INFO:teuthology.orchestra.run.smithi064.stdout: "enabled": { 2023-09-13T03:29:11.730 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enabled", 2023-09-13T03:29:11.730 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.731 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.731 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.731 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:11.731 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.732 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.732 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.732 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.732 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.733 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.733 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.733 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.734 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:11.734 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:11.734 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.734 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.735 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.735 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.735 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.736 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.736 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.736 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.736 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.737 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.737 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.737 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.738 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.738 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.738 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.739 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.739 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.739 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.739 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.740 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:11.740 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:11.740 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.740 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.741 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.741 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.741 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.742 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.742 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.742 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.742 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.743 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.743 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.743 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.744 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:11.744 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:11.744 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.744 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.745 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.745 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:11.745 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.745 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.746 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.746 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.746 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.747 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.747 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.747 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.747 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.748 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.748 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.748 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.749 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.749 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.749 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.749 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:11.750 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:11.750 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.750 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.751 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.751 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.751 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.751 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.752 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.752 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.752 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.752 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.753 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.753 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.753 INFO:teuthology.orchestra.run.smithi064.stdout: "max_completed_events": { 2023-09-13T03:29:11.754 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_completed_events", 2023-09-13T03:29:11.754 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:11.754 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.754 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.755 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "50", 2023-09-13T03:29:11.755 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.755 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.756 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.756 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "number of past completed events to remember", 2023-09-13T03:29:11.756 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.756 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.757 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.757 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.757 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2023-09-13T03:29:11.757 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2023-09-13T03:29:11.758 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:11.758 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.758 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.759 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2023-09-13T03:29:11.759 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.759 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.759 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.760 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long the module is going to sleep", 2023-09-13T03:29:11.760 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.760 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.761 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.761 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:11.761 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:11.761 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.762 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:11.762 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prometheus", 2023-09-13T03:29:11.762 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:11.763 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:11.763 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:11.763 INFO:teuthology.orchestra.run.smithi064.stdout: "cache": { 2023-09-13T03:29:11.763 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cache", 2023-09-13T03:29:11.764 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.764 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.764 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.764 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:11.765 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.765 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.765 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.766 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.766 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.766 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.766 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.767 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.767 INFO:teuthology.orchestra.run.smithi064.stdout: "exclude_perf_counters": { 2023-09-13T03:29:11.767 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "exclude_perf_counters", 2023-09-13T03:29:11.767 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.768 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.768 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.768 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:11.768 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.769 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.769 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.769 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-09-13T03:29:11.770 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-09-13T03:29:11.770 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.770 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.770 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.771 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:11.771 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:11.771 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.771 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.772 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.772 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.772 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.772 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.773 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.773 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.773 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.774 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.774 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.774 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.774 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.775 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.775 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.775 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.775 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.776 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.776 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.776 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:11.777 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:11.777 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.777 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.777 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.778 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.778 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.778 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.779 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.779 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.779 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.779 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.780 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.780 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.780 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:11.781 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:11.781 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.781 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.781 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.782 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:11.782 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.782 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.782 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.783 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.783 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.783 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.784 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.784 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.784 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.784 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.785 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.785 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.785 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.786 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.786 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.786 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:11.786 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:11.787 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.787 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.787 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.788 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.788 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.788 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.788 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.789 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.789 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.789 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.790 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.790 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.790 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_stats_pools": { 2023-09-13T03:29:11.790 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_stats_pools", 2023-09-13T03:29:11.791 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.791 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.791 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.792 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.792 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.792 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.792 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.793 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.793 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.793 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.793 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.794 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.794 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_stats_pools_refresh_interval": { 2023-09-13T03:29:11.794 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-09-13T03:29:11.795 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:11.795 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.795 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.795 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "300", 2023-09-13T03:29:11.796 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.796 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.796 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.796 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.797 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.797 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.797 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.798 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.798 INFO:teuthology.orchestra.run.smithi064.stdout: "scrape_interval": { 2023-09-13T03:29:11.798 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "scrape_interval", 2023-09-13T03:29:11.798 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2023-09-13T03:29:11.799 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.799 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.799 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15.0", 2023-09-13T03:29:11.800 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.800 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.800 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.800 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.801 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.801 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.801 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.801 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.802 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2023-09-13T03:29:11.802 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2023-09-13T03:29:11.802 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.803 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.803 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.803 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "::", 2023-09-13T03:29:11.803 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.804 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.804 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.804 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-09-13T03:29:11.804 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.805 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.805 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.805 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.805 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2023-09-13T03:29:11.806 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2023-09-13T03:29:11.806 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:11.806 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.807 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.807 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "9283", 2023-09-13T03:29:11.807 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.807 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.808 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.808 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-09-13T03:29:11.808 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.809 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.809 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.809 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.809 INFO:teuthology.orchestra.run.smithi064.stdout: "stale_cache_strategy": { 2023-09-13T03:29:11.810 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "stale_cache_strategy", 2023-09-13T03:29:11.810 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.810 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.811 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.811 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "log", 2023-09-13T03:29:11.811 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.811 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.812 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.812 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.812 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.812 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.813 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.813 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.813 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_behaviour": { 2023-09-13T03:29:11.814 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_behaviour", 2023-09-13T03:29:11.814 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.814 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.815 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.815 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "default", 2023-09-13T03:29:11.815 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.815 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.816 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.816 INFO:teuthology.orchestra.run.smithi064.stdout: "default", 2023-09-13T03:29:11.816 INFO:teuthology.orchestra.run.smithi064.stdout: "error" 2023-09-13T03:29:11.817 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.817 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.817 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.817 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.818 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.818 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.818 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_error_status_code": { 2023-09-13T03:29:11.819 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_error_status_code", 2023-09-13T03:29:11.819 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:11.819 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.819 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.820 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "500", 2023-09-13T03:29:11.820 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "400", 2023-09-13T03:29:11.820 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "599", 2023-09-13T03:29:11.820 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.821 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.821 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.821 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.821 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.822 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:11.822 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:11.822 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.823 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:11.823 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_support", 2023-09-13T03:29:11.823 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:11.823 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:11.824 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:11.824 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:11.824 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:11.825 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.825 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.825 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.825 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.826 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.826 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.826 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.827 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.827 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.827 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.827 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.828 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.828 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.828 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.828 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.829 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.829 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.829 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.830 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.830 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:11.830 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:11.831 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.831 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.831 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.831 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.832 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.832 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.832 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.832 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.833 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.833 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.833 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.833 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.834 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:11.834 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:11.834 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.835 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.835 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.835 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:11.835 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.836 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.836 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.836 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.836 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.837 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.837 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.837 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.837 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.838 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.838 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.838 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.839 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.839 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.839 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.839 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:11.839 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:11.840 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.840 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.840 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.840 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.841 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.841 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.841 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.841 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.842 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.842 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.842 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.843 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.843 INFO:teuthology.orchestra.run.smithi064.stdout: "max_concurrent_snap_create": { 2023-09-13T03:29:11.843 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_concurrent_snap_create", 2023-09-13T03:29:11.843 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:11.844 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.844 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.844 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2023-09-13T03:29:11.844 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.845 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.845 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.845 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.845 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.846 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.846 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.846 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.847 INFO:teuthology.orchestra.run.smithi064.stdout: "mirror_snapshot_schedule": { 2023-09-13T03:29:11.847 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mirror_snapshot_schedule", 2023-09-13T03:29:11.847 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.847 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.848 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.848 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.848 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.848 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.849 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.849 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.849 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.850 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.850 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.850 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.850 INFO:teuthology.orchestra.run.smithi064.stdout: "trash_purge_schedule": { 2023-09-13T03:29:11.851 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "trash_purge_schedule", 2023-09-13T03:29:11.851 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.851 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.851 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.852 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.852 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.852 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.853 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.853 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.853 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.853 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.854 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.854 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:11.854 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:11.855 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.855 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:11.855 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "restful", 2023-09-13T03:29:11.855 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:11.856 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:11.856 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:11.856 INFO:teuthology.orchestra.run.smithi064.stdout: "enable_auth": { 2023-09-13T03:29:11.856 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enable_auth", 2023-09-13T03:29:11.857 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.857 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.857 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.857 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:11.858 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.858 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.858 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.859 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.859 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.859 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.859 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.859 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.860 INFO:teuthology.orchestra.run.smithi064.stdout: "key_file": { 2023-09-13T03:29:11.860 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "key_file", 2023-09-13T03:29:11.860 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.860 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.861 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.861 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.861 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.861 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.862 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.862 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.862 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.863 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.863 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.863 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.863 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:11.864 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:11.864 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.864 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.864 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.864 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.865 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.865 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.865 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.865 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.866 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.866 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.866 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.867 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.867 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.867 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.867 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.868 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.868 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.868 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.868 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.869 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:11.869 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:11.869 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.870 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.870 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.870 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.870 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.871 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.871 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.871 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.872 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.872 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.872 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.872 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.873 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:11.873 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:11.873 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.873 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.874 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.874 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:11.874 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.875 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.875 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.875 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.875 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.876 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.876 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.876 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.876 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.877 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.877 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.877 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.877 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.878 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.878 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.878 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:11.879 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:11.879 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.879 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.879 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.880 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.880 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.880 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.881 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.881 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.881 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.881 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.882 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.882 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.882 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2023-09-13T03:29:11.883 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2023-09-13T03:29:11.883 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.883 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.884 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.884 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.884 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.884 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.885 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.885 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.885 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.886 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.886 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.886 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.886 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2023-09-13T03:29:11.887 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2023-09-13T03:29:11.887 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.887 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.887 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.888 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.888 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.888 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.888 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.889 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.889 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.889 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.889 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.890 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:11.890 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:11.890 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.891 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:11.891 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rgw", 2023-09-13T03:29:11.892 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:11.893 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:11.893 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:11.893 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:11.893 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:11.894 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.894 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.894 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.895 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.895 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.895 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.895 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.896 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.896 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.896 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.896 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.897 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.897 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.897 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.897 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.898 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.898 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.898 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.898 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.899 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:11.899 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:11.899 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.899 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.900 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.900 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.900 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.900 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.901 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.901 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.901 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.902 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.902 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.902 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.902 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:11.903 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:11.903 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.903 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.903 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.904 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:11.904 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.904 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.904 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.905 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.905 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.905 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.905 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.906 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.906 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.906 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.906 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.907 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.907 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.907 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.907 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.908 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:11.908 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:11.908 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.909 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.909 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.909 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.909 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.910 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.910 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.910 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.910 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.911 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.911 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.911 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:11.911 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:11.912 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.912 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:11.912 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rook", 2023-09-13T03:29:11.912 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:11.913 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:11.913 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:11.913 INFO:teuthology.orchestra.run.smithi064.stdout: "drive_group_interval": { 2023-09-13T03:29:11.913 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "drive_group_interval", 2023-09-13T03:29:11.914 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2023-09-13T03:29:11.914 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.914 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.915 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "300.0", 2023-09-13T03:29:11.915 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.915 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.915 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.916 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-09-13T03:29:11.916 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.916 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.916 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.917 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.917 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:11.917 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:11.917 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.918 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.918 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.918 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.918 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.919 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.919 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.919 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.920 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.920 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.920 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.920 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.921 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.921 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.921 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.921 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.922 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.922 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.922 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.922 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:11.923 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:11.923 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.923 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.924 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.924 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.924 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.924 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.925 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.925 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.925 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.926 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.926 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.926 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.926 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:11.927 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:11.927 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.927 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.928 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.928 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:11.928 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.928 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.929 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.929 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.929 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.929 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.930 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.930 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.930 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.930 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.931 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.931 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.931 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.931 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.932 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.932 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:11.932 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:11.932 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.933 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.933 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.933 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.933 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.934 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.934 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.934 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.934 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.935 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.935 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.935 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.935 INFO:teuthology.orchestra.run.smithi064.stdout: "storage_class": { 2023-09-13T03:29:11.936 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "storage_class", 2023-09-13T03:29:11.936 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.936 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.937 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.937 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "local", 2023-09-13T03:29:11.937 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.937 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.938 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.938 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-09-13T03:29:11.938 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.938 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.939 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.939 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:11.939 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:11.939 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.940 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:11.940 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "selftest", 2023-09-13T03:29:11.940 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:11.940 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:11.941 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:11.941 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:11.941 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:11.941 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.942 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.942 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.942 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.943 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.943 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.943 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.943 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.944 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.944 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.944 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.944 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.945 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.945 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.945 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.945 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.946 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.946 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.946 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.946 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:11.947 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:11.947 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.947 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.948 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.948 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.948 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.948 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.949 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.949 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.949 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.949 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.950 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.950 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.950 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:11.951 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:11.951 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.951 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.951 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.952 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:11.952 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.952 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.952 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:11.953 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:11.953 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:11.953 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:11.954 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:11.954 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:11.954 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:11.954 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:11.955 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.955 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.955 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.955 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.956 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.956 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:11.956 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:11.957 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.957 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.957 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:11.957 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:11.958 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.958 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.958 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.958 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.959 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.959 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.959 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.960 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.960 INFO:teuthology.orchestra.run.smithi064.stdout: "roption1": { 2023-09-13T03:29:11.960 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "roption1", 2023-09-13T03:29:11.960 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.961 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.961 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.961 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.961 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.962 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.962 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.962 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.962 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.963 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.963 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.963 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.963 INFO:teuthology.orchestra.run.smithi064.stdout: "roption2": { 2023-09-13T03:29:11.964 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "roption2", 2023-09-13T03:29:11.964 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.964 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.964 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.965 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "xyz", 2023-09-13T03:29:11.965 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.965 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.966 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.966 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.966 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.966 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.967 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.967 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.967 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption1": { 2023-09-13T03:29:11.967 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption1", 2023-09-13T03:29:11.968 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.968 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.968 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.968 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.969 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.969 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.969 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.970 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.970 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.970 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.970 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.971 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.971 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption2": { 2023-09-13T03:29:11.971 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption2", 2023-09-13T03:29:11.972 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:11.972 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.972 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.972 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.973 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.973 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.973 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.973 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.974 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.974 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.974 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.975 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.975 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption3": { 2023-09-13T03:29:11.975 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption3", 2023-09-13T03:29:11.975 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2023-09-13T03:29:11.976 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.976 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.976 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.976 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.977 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.977 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.977 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.977 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.978 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.978 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.978 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.979 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption4": { 2023-09-13T03:29:11.979 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption4", 2023-09-13T03:29:11.979 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.979 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.980 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.980 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.980 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.980 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.981 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.981 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.981 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.982 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.982 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.982 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.982 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption5": { 2023-09-13T03:29:11.983 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption5", 2023-09-13T03:29:11.983 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.983 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.983 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.984 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.984 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.984 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.985 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.985 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.985 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.985 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.986 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.986 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.986 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption6": { 2023-09-13T03:29:11.987 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption6", 2023-09-13T03:29:11.987 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:11.987 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.987 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.988 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:11.988 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.988 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.989 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.989 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.989 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.989 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.990 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.990 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.990 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption7": { 2023-09-13T03:29:11.991 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption7", 2023-09-13T03:29:11.991 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:11.991 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.991 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.992 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.992 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2023-09-13T03:29:11.992 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "42", 2023-09-13T03:29:11.992 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.993 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.993 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.993 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.993 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.994 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.994 INFO:teuthology.orchestra.run.smithi064.stdout: "testkey": { 2023-09-13T03:29:11.994 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testkey", 2023-09-13T03:29:11.995 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.995 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.995 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.995 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.996 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:11.996 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:11.996 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:11.996 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:11.997 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:11.997 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:11.997 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:11.997 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:11.998 INFO:teuthology.orchestra.run.smithi064.stdout: "testlkey": { 2023-09-13T03:29:11.998 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testlkey", 2023-09-13T03:29:11.998 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:11.999 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:11.999 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:11.999 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:11.999 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.000 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.000 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.000 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.000 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.000 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.001 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.001 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.001 INFO:teuthology.orchestra.run.smithi064.stdout: "testnewline": { 2023-09-13T03:29:12.002 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testnewline", 2023-09-13T03:29:12.002 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.002 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.002 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.003 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.003 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.003 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.003 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.004 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.004 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.004 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.004 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.005 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:12.005 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:12.005 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.005 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:12.006 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snap_schedule", 2023-09-13T03:29:12.006 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:12.006 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:12.006 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:12.007 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_m_granularity": { 2023-09-13T03:29:12.007 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_m_granularity", 2023-09-13T03:29:12.007 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.007 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.008 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.008 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.008 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.008 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.009 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.009 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow minute scheduled snapshots", 2023-09-13T03:29:12.009 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.010 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.010 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.010 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.010 INFO:teuthology.orchestra.run.smithi064.stdout: "dump_on_update": { 2023-09-13T03:29:12.011 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "dump_on_update", 2023-09-13T03:29:12.011 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.011 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.011 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.012 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.012 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.012 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.013 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.013 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "dump database to debug log on update", 2023-09-13T03:29:12.013 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.013 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.014 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.014 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.014 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:12.015 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:12.015 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.015 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.015 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.016 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.016 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.016 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.017 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:12.017 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:12.017 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:12.017 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:12.018 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:12.018 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:12.018 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:12.019 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:12.019 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.019 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.019 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.020 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.020 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.020 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:12.020 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:12.021 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.021 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.021 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.022 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.022 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.022 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.022 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.023 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.023 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.023 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.024 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.024 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.024 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:12.024 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:12.025 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.025 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.025 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.026 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:12.026 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.026 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.026 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:12.027 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:12.027 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:12.027 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:12.028 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:12.028 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:12.028 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:12.028 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:12.029 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.029 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.029 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.029 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.030 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.030 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:12.030 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:12.031 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.031 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.031 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.031 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.032 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.032 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.032 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.033 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.033 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.033 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.033 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.034 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:12.034 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:12.034 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.035 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:12.035 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "stats", 2023-09-13T03:29:12.035 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:12.035 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:12.036 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:12.036 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:12.036 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:12.036 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.037 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.037 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.037 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.038 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.038 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.038 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:12.038 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:12.039 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:12.039 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:12.039 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:12.040 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:12.040 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:12.040 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:12.040 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.041 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.041 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.041 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.042 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.042 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:12.042 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:12.043 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.043 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.043 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.043 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.044 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.044 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.044 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.044 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.045 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.045 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.045 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.046 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.046 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:12.046 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:12.046 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.047 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.047 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.047 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:12.048 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.048 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.048 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:12.048 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:12.049 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:12.049 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:12.049 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:12.050 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:12.050 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:12.050 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:12.050 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.051 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.051 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.051 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.051 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.052 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:12.052 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:12.052 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.053 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.053 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.053 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.053 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.054 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.054 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.054 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.055 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.055 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.055 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.055 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:12.056 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:12.056 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.056 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:12.056 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "status", 2023-09-13T03:29:12.057 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:12.057 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:12.057 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:12.058 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:12.058 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:12.058 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.058 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.059 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.059 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.059 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.059 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.060 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:12.060 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:12.060 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:12.061 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:12.061 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:12.061 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:12.061 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:12.062 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:12.062 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.062 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.063 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.063 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.063 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.063 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:12.064 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:12.064 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.064 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.065 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.065 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.065 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.065 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.066 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.066 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.066 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.067 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.067 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.067 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.067 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:12.068 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:12.068 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.068 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.069 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.069 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:12.069 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.069 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.070 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:12.070 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:12.070 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:12.070 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:12.071 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:12.071 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:12.071 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:12.072 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:12.072 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.072 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.072 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.073 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.073 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.073 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:12.073 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:12.074 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.074 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.074 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.075 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.075 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.075 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.075 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.076 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.076 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.076 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.077 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.077 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:12.077 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:12.077 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.078 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:12.078 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "telegraf", 2023-09-13T03:29:12.078 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:12.079 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:12.079 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:12.079 INFO:teuthology.orchestra.run.smithi064.stdout: "address": { 2023-09-13T03:29:12.079 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "address", 2023-09-13T03:29:12.080 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.080 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.080 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.080 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-09-13T03:29:12.081 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.081 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.081 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.082 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.082 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.082 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.082 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.083 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.083 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2023-09-13T03:29:12.083 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2023-09-13T03:29:12.083 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:12.084 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.084 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.084 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15", 2023-09-13T03:29:12.085 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.085 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.085 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.085 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.086 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.086 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.086 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.087 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.087 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:12.087 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:12.087 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.088 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.088 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.088 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.089 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.089 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.089 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:12.089 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:12.090 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:12.090 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:12.090 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:12.090 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:12.091 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:12.091 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:12.091 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.092 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.092 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.092 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.092 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.093 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:12.093 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:12.093 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.094 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.094 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.094 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.095 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.095 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.095 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.097 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.097 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.098 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.098 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.098 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.099 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:12.099 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:12.099 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.100 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.100 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.100 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:12.100 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.101 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.101 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:12.101 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:12.102 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:12.102 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:12.102 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:12.103 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:12.103 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:12.103 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:12.104 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.104 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.104 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.105 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.105 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.105 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:12.106 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:12.106 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.106 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.107 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.107 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.107 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.108 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.108 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.108 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.109 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.109 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.109 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.110 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:12.110 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:12.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.111 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:12.111 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "telemetry", 2023-09-13T03:29:12.111 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:12.112 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:12.112 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:12.112 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_basic": { 2023-09-13T03:29:12.113 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_basic", 2023-09-13T03:29:12.113 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.113 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.114 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.114 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:12.114 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.115 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.115 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.115 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share basic cluster information (size, version)", 2023-09-13T03:29:12.116 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.116 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.116 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.117 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.117 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_crash": { 2023-09-13T03:29:12.117 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_crash", 2023-09-13T03:29:12.117 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.118 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.118 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.118 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:12.119 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.119 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.119 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.119 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-09-13T03:29:12.120 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.120 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.120 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.121 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.121 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_device": { 2023-09-13T03:29:12.121 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_device", 2023-09-13T03:29:12.121 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.122 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.122 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.122 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:12.123 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.123 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.123 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.123 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-09-13T03:29:12.124 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.124 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.124 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.125 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.125 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_ident": { 2023-09-13T03:29:12.125 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_ident", 2023-09-13T03:29:12.126 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.126 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.126 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.126 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.127 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.127 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.127 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.127 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-09-13T03:29:12.128 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.128 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.128 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.129 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.129 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_perf": { 2023-09-13T03:29:12.129 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_perf", 2023-09-13T03:29:12.129 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.130 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.130 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.130 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.131 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.131 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.131 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.131 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share various performance metrics of a cluster", 2023-09-13T03:29:12.132 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.132 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.132 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.133 INFO:teuthology.orchestra.run.smithi064.stdout: "contact": { 2023-09-13T03:29:12.133 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "contact", 2023-09-13T03:29:12.133 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.134 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.134 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.134 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.135 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.135 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.135 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.135 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.136 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.136 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.136 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.137 INFO:teuthology.orchestra.run.smithi064.stdout: "description": { 2023-09-13T03:29:12.137 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "description", 2023-09-13T03:29:12.137 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.138 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.138 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.138 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.139 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.139 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.139 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.139 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.140 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.140 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.140 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.141 INFO:teuthology.orchestra.run.smithi064.stdout: "device_url": { 2023-09-13T03:29:12.141 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_url", 2023-09-13T03:29:12.141 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.142 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.142 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.142 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-09-13T03:29:12.143 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.143 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.143 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.143 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.144 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.144 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.144 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.144 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.145 INFO:teuthology.orchestra.run.smithi064.stdout: "enabled": { 2023-09-13T03:29:12.145 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enabled", 2023-09-13T03:29:12.145 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.145 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.146 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.146 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.146 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.147 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.147 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.147 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.148 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.148 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.148 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.148 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.149 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2023-09-13T03:29:12.149 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2023-09-13T03:29:12.149 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:12.149 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.150 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.150 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "24", 2023-09-13T03:29:12.150 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "8", 2023-09-13T03:29:12.151 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.151 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.151 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.151 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.152 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.152 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.152 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.153 INFO:teuthology.orchestra.run.smithi064.stdout: "last_opt_revision": { 2023-09-13T03:29:12.153 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "last_opt_revision", 2023-09-13T03:29:12.153 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:12.153 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.154 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.154 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1", 2023-09-13T03:29:12.154 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.154 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.155 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.155 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.155 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.156 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.156 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.156 INFO:teuthology.orchestra.run.smithi064.stdout: "leaderboard": { 2023-09-13T03:29:12.157 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "leaderboard", 2023-09-13T03:29:12.157 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.157 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.157 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.158 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.158 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.158 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.159 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.159 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.159 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.159 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.160 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.160 INFO:teuthology.orchestra.run.smithi064.stdout: "leaderboard_description": { 2023-09-13T03:29:12.161 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "leaderboard_description", 2023-09-13T03:29:12.161 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.161 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.161 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.162 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.162 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.162 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.162 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.163 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.163 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.163 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.164 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.164 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.164 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:12.164 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:12.165 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.165 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.165 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.166 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.166 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.166 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.166 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:12.167 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:12.167 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:12.167 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:12.168 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:12.168 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:12.168 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:12.169 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:12.169 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.169 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.170 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.170 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.171 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:12.171 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:12.171 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.172 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.172 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.172 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.172 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.173 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.173 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.173 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.174 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.174 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.174 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.175 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.175 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:12.175 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:12.175 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.176 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.176 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.176 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:12.177 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.177 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.177 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:12.177 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:12.178 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:12.178 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:12.178 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:12.178 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:12.179 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:12.179 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:12.179 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.179 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.180 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.180 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.180 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.181 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:12.181 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:12.181 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.181 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.182 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.182 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.182 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.182 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.183 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.183 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.183 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.184 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.184 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.184 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.184 INFO:teuthology.orchestra.run.smithi064.stdout: "organization": { 2023-09-13T03:29:12.185 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "organization", 2023-09-13T03:29:12.185 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.185 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.186 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.186 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.186 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.186 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.187 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.187 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.187 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.187 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.188 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.188 INFO:teuthology.orchestra.run.smithi064.stdout: "proxy": { 2023-09-13T03:29:12.188 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "proxy", 2023-09-13T03:29:12.189 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.189 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.189 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.189 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.190 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.190 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.190 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.191 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.191 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.191 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.191 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.191 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.192 INFO:teuthology.orchestra.run.smithi064.stdout: "url": { 2023-09-13T03:29:12.192 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "url", 2023-09-13T03:29:12.192 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.192 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.193 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.193 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-09-13T03:29:12.193 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.194 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.194 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.194 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.194 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.195 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.195 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.195 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:12.195 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:12.196 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.196 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:12.196 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "test_orchestrator", 2023-09-13T03:29:12.196 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:12.197 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:12.197 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:12.197 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:12.197 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:12.198 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.198 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.198 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.199 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.199 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.199 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.199 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:12.200 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:12.200 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:12.200 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:12.200 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:12.201 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:12.201 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:12.201 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:12.201 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.202 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.202 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.202 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.203 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.203 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:12.203 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:12.203 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.204 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.204 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.204 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.204 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.205 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.205 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.205 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.206 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.206 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.206 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.206 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.207 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:12.207 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:12.207 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.208 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.208 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.208 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:12.208 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.209 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.209 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:12.209 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:12.209 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:12.210 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:12.210 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:12.210 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:12.211 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:12.211 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:12.211 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.211 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.212 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.212 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.212 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.213 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:12.213 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:12.213 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.213 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.214 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.214 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.214 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.215 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.215 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.215 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.215 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.216 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.216 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.216 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:12.217 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:12.217 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.217 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:12.217 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "volumes", 2023-09-13T03:29:12.218 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:12.218 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:12.218 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:12.219 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:12.219 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:12.219 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.220 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.220 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.220 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.220 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.221 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.221 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:12.221 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:12.221 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:12.222 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:12.222 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:12.222 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:12.223 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:12.223 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:12.223 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.224 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.224 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.224 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.224 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.225 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:12.225 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:12.225 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.226 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.226 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.226 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.226 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.227 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.227 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.227 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.227 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.228 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.228 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.228 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.229 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:12.229 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:12.229 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.229 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.230 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.230 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:12.230 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.231 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.231 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:12.231 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:12.231 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:12.232 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:12.232 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:12.232 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:12.232 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:12.233 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:12.233 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.233 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.234 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.234 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.234 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.234 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:12.235 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:12.235 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.235 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.236 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.236 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.236 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.236 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.237 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.237 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.237 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.237 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.238 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.238 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.238 INFO:teuthology.orchestra.run.smithi064.stdout: "max_concurrent_clones": { 2023-09-13T03:29:12.239 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_concurrent_clones", 2023-09-13T03:29:12.239 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:12.239 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.239 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.240 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "4", 2023-09-13T03:29:12.240 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.240 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.241 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.241 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Number of asynchronous cloner threads", 2023-09-13T03:29:12.241 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.241 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.242 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.242 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.242 INFO:teuthology.orchestra.run.smithi064.stdout: "snapshot_clone_delay": { 2023-09-13T03:29:12.243 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snapshot_clone_delay", 2023-09-13T03:29:12.243 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:12.243 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.243 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.244 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2023-09-13T03:29:12.244 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.244 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.245 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.245 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-09-13T03:29:12.245 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.245 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.246 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.246 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:12.246 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:12.247 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.247 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:12.247 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix", 2023-09-13T03:29:12.247 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:12.248 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:12.248 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:12.248 INFO:teuthology.orchestra.run.smithi064.stdout: "discovery_interval": { 2023-09-13T03:29:12.249 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "discovery_interval", 2023-09-13T03:29:12.249 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2023-09-13T03:29:12.249 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.249 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.250 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "100", 2023-09-13T03:29:12.250 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.250 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.250 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.251 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.251 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.251 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.252 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.252 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.252 INFO:teuthology.orchestra.run.smithi064.stdout: "identifier": { 2023-09-13T03:29:12.252 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "identifier", 2023-09-13T03:29:12.253 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.253 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.253 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.253 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.254 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.254 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.254 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.255 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.255 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.255 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.255 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.256 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.256 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2023-09-13T03:29:12.256 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2023-09-13T03:29:12.256 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:12.257 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.257 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.257 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2023-09-13T03:29:12.257 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.258 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.258 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.258 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.259 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.259 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.259 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.259 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.260 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:12.260 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:12.260 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.261 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.261 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.261 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.261 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.262 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.262 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:12.262 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:12.262 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:12.263 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:12.263 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:12.263 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:12.264 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:12.264 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:12.264 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.264 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.265 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.265 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.265 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.266 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:12.266 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:12.266 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.266 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.267 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.267 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.267 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.267 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.268 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.268 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.268 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.269 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.269 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.269 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.269 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:12.270 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:12.270 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.270 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.271 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.271 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:12.271 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.271 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.272 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:12.272 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:12.272 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:12.272 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:12.273 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:12.273 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:12.273 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:12.273 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:12.274 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.274 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.274 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.275 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.275 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.275 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:12.276 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:12.276 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.276 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.276 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.277 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.277 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.277 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.277 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.278 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.278 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.278 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.279 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.279 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.279 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_host": { 2023-09-13T03:29:12.279 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_host", 2023-09-13T03:29:12.280 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.280 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.280 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.281 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.281 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.281 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.281 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.282 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.282 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.282 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.283 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.283 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.283 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_port": { 2023-09-13T03:29:12.283 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_port", 2023-09-13T03:29:12.284 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:12.284 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.284 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.285 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10051", 2023-09-13T03:29:12.285 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.285 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.285 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.286 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.286 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.286 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.286 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.287 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.287 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_sender": { 2023-09-13T03:29:12.287 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_sender", 2023-09-13T03:29:12.288 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.288 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.288 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.288 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-09-13T03:29:12.289 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.289 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.289 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.290 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.290 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.290 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.290 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.291 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:12.291 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:12.291 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:12.292 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-09-13T03:29:12.292 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.292 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:12.292 INFO:teuthology.orchestra.run.smithi064.stdout: "gid": 25537, 2023-09-13T03:29:12.293 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "z", 2023-09-13T03:29:12.293 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr_features": 4540138322906710015, 2023-09-13T03:29:12.293 INFO:teuthology.orchestra.run.smithi064.stdout: "available_modules": [ 2023-09-13T03:29:12.293 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:12.294 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "alerts", 2023-09-13T03:29:12.294 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:12.294 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:12.295 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:12.295 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2023-09-13T03:29:12.295 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2023-09-13T03:29:12.295 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:12.296 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.296 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.296 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2023-09-13T03:29:12.296 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.297 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.297 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.297 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How frequently to reexamine health status", 2023-09-13T03:29:12.298 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.298 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.298 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.301 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:12.301 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:12.301 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.302 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.302 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.302 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.303 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.303 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.303 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:12.303 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:12.304 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:12.304 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:12.304 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:12.304 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:12.305 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:12.305 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:12.305 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.305 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.306 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.306 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.306 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.307 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:12.307 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:12.307 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.307 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.308 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.308 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.308 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.309 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.309 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.309 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.309 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.310 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.310 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.310 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.311 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:12.311 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:12.311 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.311 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.312 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.312 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:12.312 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.313 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.313 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:12.313 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:12.313 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:12.314 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:12.314 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:12.314 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:12.315 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:12.315 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:12.315 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.315 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.316 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.316 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.316 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.317 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:12.317 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:12.317 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.317 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.318 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.318 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.318 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.319 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.319 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.319 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.320 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.320 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.320 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.320 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.321 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_destination": { 2023-09-13T03:29:12.321 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_destination", 2023-09-13T03:29:12.321 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.321 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.322 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.322 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.322 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.323 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.323 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.323 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Email address to send alerts to", 2023-09-13T03:29:12.323 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.324 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.324 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.324 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.324 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_from_name": { 2023-09-13T03:29:12.325 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_from_name", 2023-09-13T03:29:12.325 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.325 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.326 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.326 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "Ceph", 2023-09-13T03:29:12.326 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.326 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.327 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.327 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Email From: name", 2023-09-13T03:29:12.327 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.328 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.328 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.328 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.328 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_host": { 2023-09-13T03:29:12.329 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_host", 2023-09-13T03:29:12.329 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.329 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.329 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.330 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.330 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.330 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.331 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.331 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP server", 2023-09-13T03:29:12.331 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.331 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.332 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.332 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.332 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_password": { 2023-09-13T03:29:12.333 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_password", 2023-09-13T03:29:12.333 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.333 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.333 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.334 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.334 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.334 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.335 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.335 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Password to authenticate with", 2023-09-13T03:29:12.335 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.335 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.336 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.336 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.336 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_port": { 2023-09-13T03:29:12.337 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_port", 2023-09-13T03:29:12.337 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:12.337 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.337 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.338 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "465", 2023-09-13T03:29:12.338 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.338 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.339 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.339 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP port", 2023-09-13T03:29:12.339 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.339 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.340 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.340 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_sender": { 2023-09-13T03:29:12.341 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_sender", 2023-09-13T03:29:12.341 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.341 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.341 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.342 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.342 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.342 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.343 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.343 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP envelope sender", 2023-09-13T03:29:12.343 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.343 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.344 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.344 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.344 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_ssl": { 2023-09-13T03:29:12.344 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_ssl", 2023-09-13T03:29:12.345 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.345 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.345 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.346 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:12.346 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.346 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.346 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.347 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use SSL to connect to SMTP server", 2023-09-13T03:29:12.347 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.347 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.347 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.348 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.348 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_user": { 2023-09-13T03:29:12.348 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_user", 2023-09-13T03:29:12.349 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.349 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.349 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.349 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.350 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.350 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.350 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.351 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "User to authenticate as", 2023-09-13T03:29:12.351 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.351 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.351 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.352 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:12.352 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:12.352 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.353 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:12.353 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "balancer", 2023-09-13T03:29:12.353 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:12.353 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:12.354 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:12.354 INFO:teuthology.orchestra.run.smithi064.stdout: "active": { 2023-09-13T03:29:12.354 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "active", 2023-09-13T03:29:12.355 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.355 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.355 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.355 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:12.356 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.356 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.356 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.357 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "automatically balance PGs across cluster", 2023-09-13T03:29:12.357 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.357 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.357 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.358 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.358 INFO:teuthology.orchestra.run.smithi064.stdout: "begin_time": { 2023-09-13T03:29:12.358 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "begin_time", 2023-09-13T03:29:12.359 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.359 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.359 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.360 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0000", 2023-09-13T03:29:12.360 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.360 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.360 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.361 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "beginning time of day to automatically balance", 2023-09-13T03:29:12.361 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-13T03:29:12.361 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.362 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.362 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.362 INFO:teuthology.orchestra.run.smithi064.stdout: "begin_weekday": { 2023-09-13T03:29:12.362 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "begin_weekday", 2023-09-13T03:29:12.363 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2023-09-13T03:29:12.363 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.363 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.364 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2023-09-13T03:29:12.364 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0", 2023-09-13T03:29:12.364 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "6", 2023-09-13T03:29:12.364 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.365 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-09-13T03:29:12.365 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-13T03:29:12.365 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.366 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.366 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.366 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_max_iterations": { 2023-09-13T03:29:12.366 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_max_iterations", 2023-09-13T03:29:12.367 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2023-09-13T03:29:12.367 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.367 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.367 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "25", 2023-09-13T03:29:12.368 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2023-09-13T03:29:12.368 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "250", 2023-09-13T03:29:12.368 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.369 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-09-13T03:29:12.369 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.369 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.369 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.370 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.370 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_metrics": { 2023-09-13T03:29:12.370 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_metrics", 2023-09-13T03:29:12.371 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.371 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.371 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.371 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "pgs,objects,bytes", 2023-09-13T03:29:12.372 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.372 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.372 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.373 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-09-13T03:29:12.373 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-09-13T03:29:12.373 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.373 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.374 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.374 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_step": { 2023-09-13T03:29:12.374 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_step", 2023-09-13T03:29:12.375 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2023-09-13T03:29:12.375 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.375 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.375 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0.5", 2023-09-13T03:29:12.376 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0.001", 2023-09-13T03:29:12.376 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "0.999", 2023-09-13T03:29:12.376 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.377 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "aggressiveness of optimization", 2023-09-13T03:29:12.377 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-09-13T03:29:12.377 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.377 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.378 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.378 INFO:teuthology.orchestra.run.smithi064.stdout: "end_time": { 2023-09-13T03:29:12.378 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "end_time", 2023-09-13T03:29:12.379 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.379 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.379 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.379 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "2359", 2023-09-13T03:29:12.380 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.380 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.380 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.381 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "ending time of day to automatically balance", 2023-09-13T03:29:12.381 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-13T03:29:12.381 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.381 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.382 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.382 INFO:teuthology.orchestra.run.smithi064.stdout: "end_weekday": { 2023-09-13T03:29:12.382 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "end_weekday", 2023-09-13T03:29:12.383 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2023-09-13T03:29:12.383 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.383 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.383 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2023-09-13T03:29:12.384 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0", 2023-09-13T03:29:12.384 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "6", 2023-09-13T03:29:12.384 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.385 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-09-13T03:29:12.385 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-13T03:29:12.385 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.385 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.386 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.386 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:12.386 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:12.387 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.387 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.387 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.387 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.388 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.388 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.388 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:12.388 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:12.389 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:12.389 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:12.389 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:12.390 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:12.390 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:12.390 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:12.390 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.391 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.391 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.391 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.391 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.392 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:12.392 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:12.392 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.393 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.393 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.393 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.393 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.394 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.394 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.394 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.395 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.395 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.395 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.395 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.396 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:12.396 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:12.396 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.397 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.397 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.397 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:12.397 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.398 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.398 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:12.398 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:12.398 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:12.399 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:12.399 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:12.399 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:12.400 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:12.400 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:12.400 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.400 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.401 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.401 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.401 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.402 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:12.402 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:12.402 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.402 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.403 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.403 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.403 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.403 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.404 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.404 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.404 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.405 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.405 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.405 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.406 INFO:teuthology.orchestra.run.smithi064.stdout: "min_score": { 2023-09-13T03:29:12.406 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "min_score", 2023-09-13T03:29:12.406 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2023-09-13T03:29:12.406 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.407 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.407 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2023-09-13T03:29:12.407 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.408 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.408 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.408 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-09-13T03:29:12.408 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.409 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.409 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.409 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.409 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": { 2023-09-13T03:29:12.410 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mode", 2023-09-13T03:29:12.410 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.410 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.410 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.411 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "upmap", 2023-09-13T03:29:12.411 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.411 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.412 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:12.412 INFO:teuthology.orchestra.run.smithi064.stdout: "crush-compat", 2023-09-13T03:29:12.412 INFO:teuthology.orchestra.run.smithi064.stdout: "none", 2023-09-13T03:29:12.412 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap" 2023-09-13T03:29:12.413 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:12.413 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Balancer mode", 2023-09-13T03:29:12.413 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.413 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.414 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.414 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.414 INFO:teuthology.orchestra.run.smithi064.stdout: "pool_ids": { 2023-09-13T03:29:12.415 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pool_ids", 2023-09-13T03:29:12.415 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.415 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.415 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.416 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.416 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.416 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.416 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.417 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-09-13T03:29:12.417 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.417 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.417 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.418 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.418 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2023-09-13T03:29:12.418 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2023-09-13T03:29:12.419 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:12.419 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.419 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.419 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2023-09-13T03:29:12.420 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.420 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.420 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.421 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-09-13T03:29:12.421 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.421 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.421 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.422 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.422 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap_max_deviation": { 2023-09-13T03:29:12.422 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "upmap_max_deviation", 2023-09-13T03:29:12.422 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:12.423 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.423 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.423 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2023-09-13T03:29:12.423 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2023-09-13T03:29:12.424 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.424 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.424 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "deviation below which no optimization is attempted", 2023-09-13T03:29:12.425 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-09-13T03:29:12.425 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.425 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.425 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.426 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap_max_optimizations": { 2023-09-13T03:29:12.426 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "upmap_max_optimizations", 2023-09-13T03:29:12.426 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2023-09-13T03:29:12.426 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.427 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.427 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2023-09-13T03:29:12.427 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.427 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.428 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.428 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-09-13T03:29:12.428 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.429 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.429 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.429 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:12.429 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:12.430 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.430 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:12.430 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephadm", 2023-09-13T03:29:12.431 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:12.431 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:12.431 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:12.431 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_down_multiplier": { 2023-09-13T03:29:12.432 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_down_multiplier", 2023-09-13T03:29:12.432 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2023-09-13T03:29:12.432 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.432 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.433 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3.0", 2023-09-13T03:29:12.433 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.433 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.434 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.434 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-09-13T03:29:12.434 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.434 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.435 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.435 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.435 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_refresh_rate": { 2023-09-13T03:29:12.436 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_refresh_rate", 2023-09-13T03:29:12.436 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:12.436 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.436 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.437 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "20", 2023-09-13T03:29:12.437 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.437 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.438 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.438 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-09-13T03:29:12.438 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.438 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.439 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.439 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.439 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_starting_port": { 2023-09-13T03:29:12.440 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_starting_port", 2023-09-13T03:29:12.440 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:12.440 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.440 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.441 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "4721", 2023-09-13T03:29:12.441 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.441 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.441 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.442 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-09-13T03:29:12.442 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.442 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.443 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.443 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.443 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_ptrace": { 2023-09-13T03:29:12.443 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_ptrace", 2023-09-13T03:29:12.444 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.444 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.444 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.444 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.445 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.445 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.445 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.445 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-09-13T03:29:12.446 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-09-13T03:29:12.446 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.446 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.447 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.447 INFO:teuthology.orchestra.run.smithi064.stdout: "autotune_interval": { 2023-09-13T03:29:12.447 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "autotune_interval", 2023-09-13T03:29:12.447 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:12.448 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.448 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.448 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2023-09-13T03:29:12.449 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.449 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.449 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.449 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to autotune daemon memory", 2023-09-13T03:29:12.450 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.450 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.450 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.451 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.451 INFO:teuthology.orchestra.run.smithi064.stdout: "autotune_memory_target_ratio": { 2023-09-13T03:29:12.451 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "autotune_memory_target_ratio", 2023-09-13T03:29:12.451 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2023-09-13T03:29:12.452 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.452 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.452 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0.7", 2023-09-13T03:29:12.452 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.453 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.453 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.453 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-09-13T03:29:12.454 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.454 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.454 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.454 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.455 INFO:teuthology.orchestra.run.smithi064.stdout: "cgroups_split": { 2023-09-13T03:29:12.455 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cgroups_split", 2023-09-13T03:29:12.455 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.456 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.456 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.456 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:12.456 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.457 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.457 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.457 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-09-13T03:29:12.458 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.458 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.458 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.458 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.459 INFO:teuthology.orchestra.run.smithi064.stdout: "config_checks_enabled": { 2023-09-13T03:29:12.459 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_checks_enabled", 2023-09-13T03:29:12.459 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.460 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.460 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.460 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.460 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.461 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.461 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.461 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-09-13T03:29:12.461 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.462 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.462 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.462 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.463 INFO:teuthology.orchestra.run.smithi064.stdout: "config_dashboard": { 2023-09-13T03:29:12.463 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_dashboard", 2023-09-13T03:29:12.463 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.463 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.464 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.464 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:12.464 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.464 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.465 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.465 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-09-13T03:29:12.465 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.466 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.466 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.466 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.466 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_alertmanager": { 2023-09-13T03:29:12.467 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_alertmanager", 2023-09-13T03:29:12.467 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.467 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.468 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.468 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-09-13T03:29:12.468 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.468 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.469 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.469 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2023-09-13T03:29:12.469 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.470 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.470 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.470 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.470 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_base": { 2023-09-13T03:29:12.471 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_base", 2023-09-13T03:29:12.471 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.471 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.471 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.472 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/ceph", 2023-09-13T03:29:12.472 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.472 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.473 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.473 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Container image name, without the tag", 2023-09-13T03:29:12.473 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.473 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.474 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.474 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.474 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_elasticsearch": { 2023-09-13T03:29:12.474 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_elasticsearch", 2023-09-13T03:29:12.475 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.475 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.475 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.476 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-09-13T03:29:12.476 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.476 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.476 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.477 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "elasticsearch container image", 2023-09-13T03:29:12.477 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.477 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.477 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.478 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.478 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_grafana": { 2023-09-13T03:29:12.478 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_grafana", 2023-09-13T03:29:12.479 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.479 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.479 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.479 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-09-13T03:29:12.480 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.480 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.480 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.481 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2023-09-13T03:29:12.481 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.481 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.481 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.482 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.482 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_haproxy": { 2023-09-13T03:29:12.482 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_haproxy", 2023-09-13T03:29:12.482 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.483 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.483 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.483 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-09-13T03:29:12.484 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.484 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.484 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.484 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "HAproxy container image", 2023-09-13T03:29:12.485 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.485 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.485 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.485 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.486 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_agent": { 2023-09-13T03:29:12.486 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_agent", 2023-09-13T03:29:12.486 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.487 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.487 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.487 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-09-13T03:29:12.487 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.488 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.488 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.488 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger agent container image", 2023-09-13T03:29:12.489 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.489 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.489 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.489 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.490 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_collector": { 2023-09-13T03:29:12.490 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_collector", 2023-09-13T03:29:12.490 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.491 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.491 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.491 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-09-13T03:29:12.491 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.492 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.492 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.492 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger collector container image", 2023-09-13T03:29:12.492 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.493 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.493 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.493 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.494 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_query": { 2023-09-13T03:29:12.494 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_query", 2023-09-13T03:29:12.494 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.494 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.495 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.495 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-09-13T03:29:12.495 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.496 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.496 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.496 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger query container image", 2023-09-13T03:29:12.496 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.497 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.497 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.497 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.497 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_keepalived": { 2023-09-13T03:29:12.498 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_keepalived", 2023-09-13T03:29:12.498 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.498 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.499 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.499 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-09-13T03:29:12.499 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.499 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.500 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.500 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Keepalived container image", 2023-09-13T03:29:12.500 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.501 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.501 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.501 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.501 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_loki": { 2023-09-13T03:29:12.502 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_loki", 2023-09-13T03:29:12.502 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.502 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.503 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.503 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-09-13T03:29:12.504 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.505 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.505 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.505 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Loki container image", 2023-09-13T03:29:12.506 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.506 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.506 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.507 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.507 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_node_exporter": { 2023-09-13T03:29:12.507 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_node_exporter", 2023-09-13T03:29:12.507 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.508 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.508 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.508 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-09-13T03:29:12.508 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.509 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.509 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.509 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2023-09-13T03:29:12.510 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.510 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.510 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.511 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.511 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_nvmeof": { 2023-09-13T03:29:12.511 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_nvmeof", 2023-09-13T03:29:12.511 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.512 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.512 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.512 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-09-13T03:29:12.512 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.513 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.513 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.513 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Nvme-of container image", 2023-09-13T03:29:12.514 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.514 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.514 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.515 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.515 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_prometheus": { 2023-09-13T03:29:12.515 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_prometheus", 2023-09-13T03:29:12.515 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.516 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.516 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.516 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-09-13T03:29:12.517 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.517 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.517 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.517 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2023-09-13T03:29:12.518 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.518 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.518 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.519 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.519 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_promtail": { 2023-09-13T03:29:12.519 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_promtail", 2023-09-13T03:29:12.519 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.520 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.520 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.520 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-09-13T03:29:12.520 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.521 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.521 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.521 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Promtail container image", 2023-09-13T03:29:12.522 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.522 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.522 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.522 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.523 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_snmp_gateway": { 2023-09-13T03:29:12.523 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_snmp_gateway", 2023-09-13T03:29:12.523 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.524 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.524 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.524 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-09-13T03:29:12.524 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.525 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.525 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.525 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SNMP Gateway container image", 2023-09-13T03:29:12.526 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.526 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.526 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.526 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.527 INFO:teuthology.orchestra.run.smithi064.stdout: "container_init": { 2023-09-13T03:29:12.527 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_init", 2023-09-13T03:29:12.527 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.528 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.528 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.528 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:12.528 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.529 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.529 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.529 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Run podman/docker with `--init`", 2023-09-13T03:29:12.530 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.530 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.530 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.530 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.531 INFO:teuthology.orchestra.run.smithi064.stdout: "daemon_cache_timeout": { 2023-09-13T03:29:12.531 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "daemon_cache_timeout", 2023-09-13T03:29:12.531 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:12.532 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.532 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.532 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2023-09-13T03:29:12.532 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.533 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.533 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.533 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-09-13T03:29:12.533 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.534 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.534 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.534 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.535 INFO:teuthology.orchestra.run.smithi064.stdout: "default_cephadm_command_timeout": { 2023-09-13T03:29:12.535 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "default_cephadm_command_timeout", 2023-09-13T03:29:12.535 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:12.535 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.536 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.536 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "900", 2023-09-13T03:29:12.536 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.536 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.537 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.537 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-09-13T03:29:12.537 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.538 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.538 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.538 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.538 INFO:teuthology.orchestra.run.smithi064.stdout: "default_registry": { 2023-09-13T03:29:12.539 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "default_registry", 2023-09-13T03:29:12.539 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.539 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.540 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.540 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io", 2023-09-13T03:29:12.540 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.540 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.541 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.541 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-09-13T03:29:12.541 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.542 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.542 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.542 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.542 INFO:teuthology.orchestra.run.smithi064.stdout: "device_cache_timeout": { 2023-09-13T03:29:12.543 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_cache_timeout", 2023-09-13T03:29:12.543 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:12.543 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.543 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.544 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1800", 2023-09-13T03:29:12.544 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.544 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.545 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.545 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache device inventory", 2023-09-13T03:29:12.545 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.545 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.546 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.546 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.546 INFO:teuthology.orchestra.run.smithi064.stdout: "device_enhanced_scan": { 2023-09-13T03:29:12.547 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_enhanced_scan", 2023-09-13T03:29:12.547 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.547 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.547 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.548 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.548 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.548 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.549 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.549 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use libstoragemgmt during device scans", 2023-09-13T03:29:12.549 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.550 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.550 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.550 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.550 INFO:teuthology.orchestra.run.smithi064.stdout: "facts_cache_timeout": { 2023-09-13T03:29:12.551 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "facts_cache_timeout", 2023-09-13T03:29:12.551 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:12.551 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.552 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.552 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2023-09-13T03:29:12.552 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.552 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.553 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.553 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache host facts data", 2023-09-13T03:29:12.553 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.553 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.554 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.554 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.554 INFO:teuthology.orchestra.run.smithi064.stdout: "host_check_interval": { 2023-09-13T03:29:12.555 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "host_check_interval", 2023-09-13T03:29:12.555 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:12.555 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.555 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.556 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2023-09-13T03:29:12.556 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.556 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.557 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.557 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to perform a host check", 2023-09-13T03:29:12.557 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.557 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.558 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.558 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.558 INFO:teuthology.orchestra.run.smithi064.stdout: "inventory_list_all": { 2023-09-13T03:29:12.558 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "inventory_list_all", 2023-09-13T03:29:12.559 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.559 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.559 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.560 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.560 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.560 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.560 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.561 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-09-13T03:29:12.561 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.561 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.561 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.562 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.562 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:12.562 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:12.563 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.563 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.563 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.563 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.564 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.564 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.564 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:12.564 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:12.565 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:12.565 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:12.565 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:12.565 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:12.566 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:12.566 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:12.566 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.566 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.567 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.567 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.567 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.567 INFO:teuthology.orchestra.run.smithi064.stdout: "log_refresh_metadata": { 2023-09-13T03:29:12.568 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_refresh_metadata", 2023-09-13T03:29:12.568 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.568 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.569 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.569 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.569 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.569 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.570 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.570 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-09-13T03:29:12.570 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.571 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.571 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.571 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.571 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:12.572 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:12.572 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.572 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.572 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.573 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:12.573 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.573 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.573 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.574 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-09-13T03:29:12.574 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.574 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.574 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.575 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.575 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:12.575 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:12.576 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.576 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.576 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.576 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:12.577 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.577 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.577 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:12.577 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:12.578 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:12.578 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:12.578 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:12.579 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:12.579 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:12.579 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:12.579 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.580 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.580 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.580 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.580 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.581 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:12.581 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:12.581 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.582 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.582 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.582 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.582 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.583 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.583 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.583 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.583 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.584 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.584 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.584 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.585 INFO:teuthology.orchestra.run.smithi064.stdout: "manage_etc_ceph_ceph_conf": { 2023-09-13T03:29:12.585 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-09-13T03:29:12.585 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.585 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.586 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.586 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.586 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.587 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.587 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.587 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-09-13T03:29:12.587 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.588 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.588 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.588 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.588 INFO:teuthology.orchestra.run.smithi064.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-09-13T03:29:12.589 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-09-13T03:29:12.589 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.589 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.590 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.590 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "*", 2023-09-13T03:29:12.590 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.590 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.591 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.591 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-09-13T03:29:12.591 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.591 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.592 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.592 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.592 INFO:teuthology.orchestra.run.smithi064.stdout: "max_count_per_host": { 2023-09-13T03:29:12.593 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_count_per_host", 2023-09-13T03:29:12.593 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:12.593 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.593 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.594 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2023-09-13T03:29:12.594 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.594 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.594 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.595 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "max number of daemons per service per host", 2023-09-13T03:29:12.595 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.595 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.595 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.596 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.596 INFO:teuthology.orchestra.run.smithi064.stdout: "max_osd_draining_count": { 2023-09-13T03:29:12.596 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_osd_draining_count", 2023-09-13T03:29:12.596 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:12.597 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.597 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.597 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2023-09-13T03:29:12.597 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.598 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.598 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.598 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-09-13T03:29:12.599 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.599 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.599 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.599 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.600 INFO:teuthology.orchestra.run.smithi064.stdout: "migration_current": { 2023-09-13T03:29:12.600 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "migration_current", 2023-09-13T03:29:12.600 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:12.600 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.601 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.601 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.601 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.601 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.602 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.602 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "internal - do not modify", 2023-09-13T03:29:12.602 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.602 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.603 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.603 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.603 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": { 2023-09-13T03:29:12.604 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mode", 2023-09-13T03:29:12.604 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.604 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.604 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.605 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "root", 2023-09-13T03:29:12.605 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.605 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.606 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:12.606 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm-package", 2023-09-13T03:29:12.606 INFO:teuthology.orchestra.run.smithi064.stdout: "root" 2023-09-13T03:29:12.606 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:12.607 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "mode for remote execution of cephadm", 2023-09-13T03:29:12.607 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.607 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.608 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.608 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.608 INFO:teuthology.orchestra.run.smithi064.stdout: "prometheus_alerts_path": { 2023-09-13T03:29:12.608 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prometheus_alerts_path", 2023-09-13T03:29:12.609 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.609 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.609 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.610 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-09-13T03:29:12.610 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.610 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.610 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.611 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-09-13T03:29:12.611 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.611 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.612 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.612 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.612 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_insecure": { 2023-09-13T03:29:12.612 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_insecure", 2023-09-13T03:29:12.613 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.613 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.613 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.614 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.614 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.614 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.614 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.615 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-09-13T03:29:12.615 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.615 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.615 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.616 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.616 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_password": { 2023-09-13T03:29:12.616 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_password", 2023-09-13T03:29:12.617 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.617 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.617 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.617 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.618 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.618 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.618 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.619 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-09-13T03:29:12.619 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.619 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.619 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.620 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.620 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_url": { 2023-09-13T03:29:12.620 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_url", 2023-09-13T03:29:12.621 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.621 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.621 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.621 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.622 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.622 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.622 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.622 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-09-13T03:29:12.623 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.623 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.623 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.624 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.624 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_username": { 2023-09-13T03:29:12.624 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_username", 2023-09-13T03:29:12.624 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.625 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.625 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.625 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.625 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.626 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.626 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.626 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-09-13T03:29:12.627 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.627 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.627 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.628 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.628 INFO:teuthology.orchestra.run.smithi064.stdout: "secure_monitoring_stack": { 2023-09-13T03:29:12.628 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "secure_monitoring_stack", 2023-09-13T03:29:12.628 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.629 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.629 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.629 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.629 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.630 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.630 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.630 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-09-13T03:29:12.631 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.631 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.631 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.631 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.632 INFO:teuthology.orchestra.run.smithi064.stdout: "service_discovery_port": { 2023-09-13T03:29:12.632 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "service_discovery_port", 2023-09-13T03:29:12.632 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:12.632 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.633 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.633 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8765", 2023-09-13T03:29:12.633 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.633 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.634 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.634 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "cephadm service discovery port", 2023-09-13T03:29:12.634 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.635 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.635 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.635 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.635 INFO:teuthology.orchestra.run.smithi064.stdout: "ssh_config_file": { 2023-09-13T03:29:12.636 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssh_config_file", 2023-09-13T03:29:12.636 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.636 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.636 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.637 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.637 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.637 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.637 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.638 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-09-13T03:29:12.638 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.638 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.639 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.639 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.639 INFO:teuthology.orchestra.run.smithi064.stdout: "use_agent": { 2023-09-13T03:29:12.639 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "use_agent", 2023-09-13T03:29:12.640 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.640 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.640 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.640 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.641 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.641 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.641 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.641 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-09-13T03:29:12.642 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.642 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.642 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.642 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.643 INFO:teuthology.orchestra.run.smithi064.stdout: "use_repo_digest": { 2023-09-13T03:29:12.643 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "use_repo_digest", 2023-09-13T03:29:12.643 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.644 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.644 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.644 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:12.644 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.645 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.645 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.645 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-09-13T03:29:12.645 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.646 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.646 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.646 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.647 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_failed_host_check": { 2023-09-13T03:29:12.647 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_failed_host_check", 2023-09-13T03:29:12.647 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.648 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.648 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.648 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:12.648 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.649 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.649 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.649 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if the host check fails", 2023-09-13T03:29:12.650 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.650 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.650 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.650 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.651 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_stray_daemons": { 2023-09-13T03:29:12.651 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_stray_daemons", 2023-09-13T03:29:12.651 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.651 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.652 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.652 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:12.652 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.652 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.653 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.653 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-09-13T03:29:12.653 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.653 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.654 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.654 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.654 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_stray_hosts": { 2023-09-13T03:29:12.655 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_stray_hosts", 2023-09-13T03:29:12.655 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.655 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.655 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.656 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:12.656 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.656 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.656 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.657 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-09-13T03:29:12.657 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.657 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.658 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.658 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:12.658 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:12.659 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.659 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:12.659 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crash", 2023-09-13T03:29:12.659 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:12.660 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:12.660 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:12.660 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:12.660 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:12.661 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.661 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.661 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.661 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.662 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.662 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.662 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:12.663 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:12.663 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:12.663 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:12.663 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:12.664 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:12.664 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:12.664 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:12.664 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.665 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.665 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.665 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.665 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.666 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:12.666 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:12.666 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.666 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.667 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.667 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.667 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.668 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.668 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.668 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.668 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.669 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.669 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.669 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.670 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:12.670 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:12.670 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.670 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.671 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.671 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:12.671 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.671 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.672 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:12.672 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:12.672 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:12.672 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:12.673 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:12.673 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:12.673 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:12.673 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:12.674 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.674 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.674 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.675 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.675 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.675 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:12.675 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:12.676 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.676 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.676 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.676 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.677 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.677 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.677 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.677 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.678 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.678 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.678 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.679 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.679 INFO:teuthology.orchestra.run.smithi064.stdout: "retain_interval": { 2023-09-13T03:29:12.679 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "retain_interval", 2023-09-13T03:29:12.679 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:12.680 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.680 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.680 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "31536000", 2023-09-13T03:29:12.680 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.680 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.681 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.681 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long to retain crashes before pruning them", 2023-09-13T03:29:12.681 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.682 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.682 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.682 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.682 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_recent_interval": { 2023-09-13T03:29:12.683 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_recent_interval", 2023-09-13T03:29:12.683 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:12.683 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.683 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.684 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1209600", 2023-09-13T03:29:12.684 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.684 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.685 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.685 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "time interval in which to warn about recent crashes", 2023-09-13T03:29:12.685 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.685 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.686 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.686 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:12.686 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:12.686 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.687 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:12.687 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "dashboard", 2023-09-13T03:29:12.687 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:12.688 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:12.688 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:12.688 INFO:teuthology.orchestra.run.smithi064.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-09-13T03:29:12.688 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-09-13T03:29:12.689 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:12.689 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.689 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.689 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2023-09-13T03:29:12.690 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.690 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.690 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.691 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.691 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.691 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.691 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.692 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.692 INFO:teuthology.orchestra.run.smithi064.stdout: "ALERTMANAGER_API_HOST": { 2023-09-13T03:29:12.692 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ALERTMANAGER_API_HOST", 2023-09-13T03:29:12.693 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.693 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.693 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.693 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.694 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.694 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.694 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.694 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.695 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.695 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.695 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.695 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.696 INFO:teuthology.orchestra.run.smithi064.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-09-13T03:29:12.696 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-09-13T03:29:12.696 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.696 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.697 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.697 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:12.697 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.697 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.698 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.698 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.698 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.699 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.699 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.699 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.699 INFO:teuthology.orchestra.run.smithi064.stdout: "AUDIT_API_ENABLED": { 2023-09-13T03:29:12.700 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "AUDIT_API_ENABLED", 2023-09-13T03:29:12.700 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.700 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.700 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.701 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.701 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.701 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.701 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.702 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.702 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.702 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.703 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.703 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.703 INFO:teuthology.orchestra.run.smithi064.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-09-13T03:29:12.703 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-09-13T03:29:12.703 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.704 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.704 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.704 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:12.705 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.705 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.705 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.705 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.706 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.706 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.706 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.706 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.707 INFO:teuthology.orchestra.run.smithi064.stdout: "ENABLE_BROWSABLE_API": { 2023-09-13T03:29:12.707 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ENABLE_BROWSABLE_API", 2023-09-13T03:29:12.707 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.708 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.708 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.708 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:12.708 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.709 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.709 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.709 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.709 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.710 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.710 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.710 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.710 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-09-13T03:29:12.711 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-09-13T03:29:12.711 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.711 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.711 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.712 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:12.712 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.712 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.712 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.713 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.713 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.713 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.713 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.714 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.714 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-09-13T03:29:12.714 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-09-13T03:29:12.715 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.715 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.715 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.715 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:12.716 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.716 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.716 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.717 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.717 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.717 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.717 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.718 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.718 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-09-13T03:29:12.718 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-09-13T03:29:12.718 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.719 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.719 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.719 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:12.719 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.720 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.720 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.720 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.720 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.721 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.721 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.721 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.722 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-09-13T03:29:12.722 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-09-13T03:29:12.722 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.722 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.723 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.723 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:12.723 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.723 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.724 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.724 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.724 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.724 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.725 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.725 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.725 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_NFS": { 2023-09-13T03:29:12.726 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-09-13T03:29:12.726 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.726 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.726 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.727 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:12.727 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.727 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.727 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.728 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.728 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.728 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.728 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.729 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.729 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_RBD": { 2023-09-13T03:29:12.729 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-09-13T03:29:12.730 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.730 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.730 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.730 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:12.731 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.731 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.731 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.732 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.732 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.732 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.732 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.733 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.733 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_RGW": { 2023-09-13T03:29:12.733 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-09-13T03:29:12.733 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.734 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.734 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.734 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:12.735 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.735 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.735 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.735 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.736 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.736 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.736 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.736 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.737 INFO:teuthology.orchestra.run.smithi064.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-09-13T03:29:12.737 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-09-13T03:29:12.737 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.738 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.738 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.738 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.738 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.739 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.739 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.739 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.739 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.740 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.740 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.740 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.741 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_PASSWORD": { 2023-09-13T03:29:12.741 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_PASSWORD", 2023-09-13T03:29:12.741 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.741 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.742 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.742 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2023-09-13T03:29:12.742 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.742 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.743 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.743 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.743 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.743 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.744 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.744 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.744 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-09-13T03:29:12.745 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-09-13T03:29:12.745 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.745 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.745 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.746 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:12.746 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.746 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.746 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.747 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.747 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.747 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.748 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.748 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.748 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_URL": { 2023-09-13T03:29:12.748 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_URL", 2023-09-13T03:29:12.749 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.749 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.749 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.749 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.750 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.750 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.750 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.751 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.751 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.751 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.751 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.752 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.752 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_USERNAME": { 2023-09-13T03:29:12.752 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_USERNAME", 2023-09-13T03:29:12.753 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.753 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.753 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.753 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2023-09-13T03:29:12.754 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.754 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.754 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.754 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.755 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.755 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.755 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.756 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.756 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-09-13T03:29:12.756 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-09-13T03:29:12.756 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.757 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.757 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.757 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.758 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.758 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.758 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.758 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.759 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.759 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.759 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.759 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.760 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-09-13T03:29:12.760 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-09-13T03:29:12.760 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.761 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.761 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.761 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.761 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.762 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.762 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.762 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.763 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.763 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.763 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.763 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.764 INFO:teuthology.orchestra.run.smithi064.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-09-13T03:29:12.764 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-09-13T03:29:12.764 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.764 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.765 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.765 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:12.765 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.765 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.766 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.766 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.766 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.766 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.767 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.767 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.767 INFO:teuthology.orchestra.run.smithi064.stdout: "ISSUE_TRACKER_API_KEY": { 2023-09-13T03:29:12.768 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-09-13T03:29:12.768 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.768 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.768 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.769 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.769 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.769 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.769 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.770 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.770 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.770 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.771 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.771 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.771 INFO:teuthology.orchestra.run.smithi064.stdout: "PROMETHEUS_API_HOST": { 2023-09-13T03:29:12.771 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PROMETHEUS_API_HOST", 2023-09-13T03:29:12.772 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.772 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.772 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.772 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.773 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.773 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.773 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.773 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.774 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.774 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.774 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.775 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.775 INFO:teuthology.orchestra.run.smithi064.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-09-13T03:29:12.775 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-09-13T03:29:12.775 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.776 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.776 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.776 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:12.776 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.777 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.777 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.777 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.777 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.778 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.778 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.778 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.779 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-09-13T03:29:12.779 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-09-13T03:29:12.779 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.779 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.780 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.780 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.780 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.780 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.781 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.781 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.781 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.781 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.782 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.782 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.782 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-09-13T03:29:12.783 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-09-13T03:29:12.783 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.783 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.783 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.784 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.784 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.784 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.785 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.785 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.785 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.785 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.786 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.786 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.786 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-09-13T03:29:12.786 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-09-13T03:29:12.787 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.787 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.787 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.787 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:12.788 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.788 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.788 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.789 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.789 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.789 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.789 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.790 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.790 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-09-13T03:29:12.790 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-09-13T03:29:12.791 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.791 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.791 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.791 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:12.792 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.792 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.792 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.792 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.793 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.793 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.793 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.793 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.794 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-09-13T03:29:12.794 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-09-13T03:29:12.794 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.794 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.795 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.795 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.795 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.796 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.796 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.796 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.796 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.797 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.797 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.797 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.797 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-09-13T03:29:12.798 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-09-13T03:29:12.798 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.798 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.799 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.799 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.799 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.799 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.800 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.800 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.800 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.801 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.801 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.801 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.801 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-09-13T03:29:12.802 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-09-13T03:29:12.802 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.802 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.802 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.803 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.803 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.803 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.804 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.804 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.804 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.804 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.805 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.805 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.805 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_ENABLED": { 2023-09-13T03:29:12.806 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_ENABLED", 2023-09-13T03:29:12.806 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.806 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.806 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.807 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:12.807 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.807 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.808 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.808 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.808 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.808 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.809 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.809 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.809 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-09-13T03:29:12.810 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-09-13T03:29:12.810 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.810 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.810 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.811 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-09-13T03:29:12.811 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.811 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.811 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.812 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.812 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.812 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.812 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.813 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.813 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-09-13T03:29:12.813 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-09-13T03:29:12.813 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:12.814 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.814 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.814 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2023-09-13T03:29:12.815 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.815 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.815 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.815 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.816 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.816 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.816 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.816 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.817 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-09-13T03:29:12.817 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-09-13T03:29:12.817 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:12.817 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.818 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.818 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8", 2023-09-13T03:29:12.818 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.819 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.819 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.819 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.819 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.820 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.820 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.820 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.820 INFO:teuthology.orchestra.run.smithi064.stdout: "REST_REQUESTS_TIMEOUT": { 2023-09-13T03:29:12.821 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-09-13T03:29:12.821 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:12.821 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.822 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.822 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "45", 2023-09-13T03:29:12.822 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.822 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.823 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.823 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.823 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.823 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.824 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.824 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.824 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_ACCESS_KEY": { 2023-09-13T03:29:12.825 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_ACCESS_KEY", 2023-09-13T03:29:12.825 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.825 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.825 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.826 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.826 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.826 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.826 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.827 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.827 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.827 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.828 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.828 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.828 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-09-13T03:29:12.828 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-09-13T03:29:12.829 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.829 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.829 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.829 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2023-09-13T03:29:12.830 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.830 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.830 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.830 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.831 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.831 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.831 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.832 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.832 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_SECRET_KEY": { 2023-09-13T03:29:12.832 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_SECRET_KEY", 2023-09-13T03:29:12.832 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.833 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.833 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.833 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.834 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.834 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.834 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.834 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.835 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.835 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.835 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.835 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.836 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_SSL_VERIFY": { 2023-09-13T03:29:12.836 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_SSL_VERIFY", 2023-09-13T03:29:12.836 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.836 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.837 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.837 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:12.837 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.838 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.838 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.838 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.838 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.839 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.839 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.839 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.839 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-09-13T03:29:12.840 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-09-13T03:29:12.840 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:12.840 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.840 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.841 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2023-09-13T03:29:12.841 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.841 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.841 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.842 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.842 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.842 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.843 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.843 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.843 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-09-13T03:29:12.843 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-09-13T03:29:12.844 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:12.844 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.844 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.844 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2023-09-13T03:29:12.845 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.845 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.845 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.845 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.846 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.846 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.846 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.847 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.847 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-09-13T03:29:12.847 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-09-13T03:29:12.847 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:12.848 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.848 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.848 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2023-09-13T03:29:12.848 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.849 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.849 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.849 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.849 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.850 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.850 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.850 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.851 INFO:teuthology.orchestra.run.smithi064.stdout: "cross_origin_url": { 2023-09-13T03:29:12.851 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cross_origin_url", 2023-09-13T03:29:12.851 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.851 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.852 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.852 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.852 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.853 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.853 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.853 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.853 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.854 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.854 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.854 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.854 INFO:teuthology.orchestra.run.smithi064.stdout: "crt_file": { 2023-09-13T03:29:12.855 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crt_file", 2023-09-13T03:29:12.855 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.855 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.856 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.856 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.856 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.857 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.857 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.857 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.857 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.858 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.858 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.858 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.858 INFO:teuthology.orchestra.run.smithi064.stdout: "debug": { 2023-09-13T03:29:12.859 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "debug", 2023-09-13T03:29:12.859 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.859 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.860 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.860 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.860 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.860 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.861 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.861 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable/disable debug options", 2023-09-13T03:29:12.861 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.862 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.862 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.862 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.862 INFO:teuthology.orchestra.run.smithi064.stdout: "jwt_token_ttl": { 2023-09-13T03:29:12.863 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "jwt_token_ttl", 2023-09-13T03:29:12.863 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:12.863 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.864 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.864 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "28800", 2023-09-13T03:29:12.864 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.864 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.865 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.865 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.865 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.866 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.866 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.866 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.866 INFO:teuthology.orchestra.run.smithi064.stdout: "key_file": { 2023-09-13T03:29:12.867 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "key_file", 2023-09-13T03:29:12.867 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.867 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.867 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.868 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.868 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.868 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.869 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.869 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.869 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.869 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.870 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.870 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.870 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:12.871 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:12.871 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.871 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.871 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.872 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.872 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.872 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.872 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:12.873 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:12.873 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:12.873 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:12.874 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:12.874 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:12.874 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:12.874 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:12.875 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.875 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.875 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.875 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.876 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.876 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:12.876 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:12.876 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.877 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.877 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.877 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.877 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.878 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.878 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.878 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.879 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.879 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.879 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.879 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.880 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:12.880 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:12.880 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.880 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.881 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.881 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:12.881 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.881 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.882 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:12.882 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:12.882 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:12.883 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:12.883 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:12.883 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:12.883 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:12.884 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:12.884 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.884 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.885 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.885 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.885 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.885 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:12.886 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:12.886 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.886 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.887 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.887 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.887 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.887 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.888 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.888 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.888 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.889 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.889 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.889 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.889 INFO:teuthology.orchestra.run.smithi064.stdout: "motd": { 2023-09-13T03:29:12.890 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "motd", 2023-09-13T03:29:12.890 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.890 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.891 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.891 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.891 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.891 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.892 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.892 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "The message of the day", 2023-09-13T03:29:12.892 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.893 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.893 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.893 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.893 INFO:teuthology.orchestra.run.smithi064.stdout: "redirect_resolve_ip_addr": { 2023-09-13T03:29:12.894 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "redirect_resolve_ip_addr", 2023-09-13T03:29:12.894 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.894 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.894 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.895 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.895 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.895 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.895 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.896 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.896 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.896 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.896 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.897 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.897 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2023-09-13T03:29:12.897 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2023-09-13T03:29:12.898 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.898 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.898 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.898 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "::", 2023-09-13T03:29:12.899 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.899 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.899 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.900 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.900 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.900 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.900 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.901 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.901 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2023-09-13T03:29:12.901 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2023-09-13T03:29:12.901 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:12.902 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.902 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.902 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8080", 2023-09-13T03:29:12.903 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.903 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.903 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.903 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.904 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.904 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.904 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.905 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.905 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl": { 2023-09-13T03:29:12.905 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl", 2023-09-13T03:29:12.905 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.906 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.906 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.906 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:12.907 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.907 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.907 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.907 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.908 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.908 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.908 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.908 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.909 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl_server_port": { 2023-09-13T03:29:12.909 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl_server_port", 2023-09-13T03:29:12.909 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:12.910 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.910 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.910 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8443", 2023-09-13T03:29:12.910 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.911 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.911 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.911 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.911 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.912 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.912 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.912 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.912 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_behaviour": { 2023-09-13T03:29:12.913 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_behaviour", 2023-09-13T03:29:12.913 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.913 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.913 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.914 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "redirect", 2023-09-13T03:29:12.914 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.914 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.915 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:12.915 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:12.915 INFO:teuthology.orchestra.run.smithi064.stdout: "redirect" 2023-09-13T03:29:12.915 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:12.916 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.916 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.916 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.916 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.917 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.917 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_error_status_code": { 2023-09-13T03:29:12.917 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_error_status_code", 2023-09-13T03:29:12.917 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:12.918 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.918 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.918 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "500", 2023-09-13T03:29:12.919 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "400", 2023-09-13T03:29:12.919 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "599", 2023-09-13T03:29:12.919 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.919 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.920 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.920 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.920 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.921 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.921 INFO:teuthology.orchestra.run.smithi064.stdout: "url_prefix": { 2023-09-13T03:29:12.921 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "url_prefix", 2023-09-13T03:29:12.921 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.922 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.922 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.922 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.923 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.923 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.923 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.923 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.924 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.924 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.924 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.924 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:12.925 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:12.925 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.925 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:12.925 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "devicehealth", 2023-09-13T03:29:12.926 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:12.926 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:12.926 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:12.927 INFO:teuthology.orchestra.run.smithi064.stdout: "enable_monitoring": { 2023-09-13T03:29:12.927 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enable_monitoring", 2023-09-13T03:29:12.927 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.927 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.928 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.928 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:12.928 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.928 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.929 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.929 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "monitor device health metrics", 2023-09-13T03:29:12.929 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.929 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.930 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.930 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.930 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:12.931 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:12.931 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.931 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.931 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.932 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.932 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.932 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.932 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:12.933 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:12.933 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:12.933 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:12.934 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:12.934 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:12.934 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:12.934 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:12.935 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.935 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.935 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.935 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.936 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.936 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:12.936 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:12.936 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.937 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.937 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.937 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.937 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.938 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.938 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.938 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.938 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.939 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.939 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.939 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.940 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:12.940 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:12.940 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.940 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.941 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.941 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:12.941 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.941 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.942 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:12.942 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:12.942 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:12.943 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:12.943 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:12.943 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:12.943 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:12.944 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:12.944 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.944 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.945 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.945 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.945 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.945 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:12.946 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:12.946 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.946 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.946 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.947 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.947 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.947 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.948 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.948 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.948 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.948 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.949 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.949 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.949 INFO:teuthology.orchestra.run.smithi064.stdout: "mark_out_threshold": { 2023-09-13T03:29:12.950 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mark_out_threshold", 2023-09-13T03:29:12.950 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:12.950 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.950 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.951 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "2419200", 2023-09-13T03:29:12.951 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.951 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.951 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.952 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-09-13T03:29:12.952 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.952 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.952 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.953 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.953 INFO:teuthology.orchestra.run.smithi064.stdout: "pool_name": { 2023-09-13T03:29:12.953 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pool_name", 2023-09-13T03:29:12.953 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.954 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.954 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.954 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "device_health_metrics", 2023-09-13T03:29:12.955 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.955 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.955 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.955 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "name of pool in which to store device health metrics", 2023-09-13T03:29:12.956 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.956 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.956 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.956 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.957 INFO:teuthology.orchestra.run.smithi064.stdout: "retention_period": { 2023-09-13T03:29:12.957 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "retention_period", 2023-09-13T03:29:12.957 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:12.957 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.958 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.958 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15552000", 2023-09-13T03:29:12.958 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.958 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.959 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.959 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long to retain device health metrics", 2023-09-13T03:29:12.959 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.960 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.960 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.960 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.960 INFO:teuthology.orchestra.run.smithi064.stdout: "scrape_frequency": { 2023-09-13T03:29:12.961 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "scrape_frequency", 2023-09-13T03:29:12.961 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:12.961 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.961 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.962 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "86400", 2023-09-13T03:29:12.962 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.962 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.962 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.963 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to scrape device health metrics", 2023-09-13T03:29:12.963 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.963 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.963 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.964 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.964 INFO:teuthology.orchestra.run.smithi064.stdout: "self_heal": { 2023-09-13T03:29:12.964 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "self_heal", 2023-09-13T03:29:12.964 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.965 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.965 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.965 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:12.965 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.966 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.966 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.966 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-09-13T03:29:12.967 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.967 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.967 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.967 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.968 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2023-09-13T03:29:12.968 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2023-09-13T03:29:12.968 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:12.969 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.969 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.969 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2023-09-13T03:29:12.969 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.970 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.970 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.970 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to wake up and check device health", 2023-09-13T03:29:12.971 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.971 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.971 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.971 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.971 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_threshold": { 2023-09-13T03:29:12.972 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_threshold", 2023-09-13T03:29:12.972 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:12.972 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.973 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.973 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "7257600", 2023-09-13T03:29:12.973 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.973 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.974 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.974 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-09-13T03:29:12.974 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.974 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.975 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.975 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:12.975 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:12.975 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.976 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:12.976 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "diskprediction_local", 2023-09-13T03:29:12.976 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:12.976 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:12.977 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:12.977 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:12.977 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:12.977 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.978 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.978 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.978 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:12.978 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.979 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.979 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:12.979 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:12.980 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:12.980 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:12.980 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:12.980 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:12.981 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:12.981 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:12.981 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.981 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.982 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.982 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.982 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.983 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:12.983 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:12.983 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.983 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.984 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.984 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.984 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.985 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.985 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.985 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.985 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.986 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.986 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.986 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.986 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:12.987 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:12.987 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.987 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.988 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.988 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:12.988 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.988 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.989 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:12.989 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:12.989 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:12.989 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:12.990 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:12.990 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:12.990 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:12.991 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:12.991 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.991 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.991 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.992 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.992 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.992 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:12.992 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:12.993 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:12.993 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.993 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:12.993 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:12.994 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.994 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.994 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.994 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.995 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.995 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.995 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.996 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:12.996 INFO:teuthology.orchestra.run.smithi064.stdout: "predict_interval": { 2023-09-13T03:29:12.996 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "predict_interval", 2023-09-13T03:29:12.996 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:12.997 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:12.997 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:12.997 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "86400", 2023-09-13T03:29:12.997 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:12.998 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:12.998 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:12.998 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:12.998 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:12.999 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:12.999 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:12.999 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.000 INFO:teuthology.orchestra.run.smithi064.stdout: "predictor_model": { 2023-09-13T03:29:13.000 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "predictor_model", 2023-09-13T03:29:13.000 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.000 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.001 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.001 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "prophetstor", 2023-09-13T03:29:13.001 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.002 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.002 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.002 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.002 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.003 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.003 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.003 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.004 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2023-09-13T03:29:13.004 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2023-09-13T03:29:13.004 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.004 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.005 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.005 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2023-09-13T03:29:13.005 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.005 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.006 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.006 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.006 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.007 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.007 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.007 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.007 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.008 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.008 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:13.008 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "influx", 2023-09-13T03:29:13.009 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": false, 2023-09-13T03:29:13.009 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "influxdb python module not found", 2023-09-13T03:29:13.009 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:13.009 INFO:teuthology.orchestra.run.smithi064.stdout: "batch_size": { 2023-09-13T03:29:13.010 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "batch_size", 2023-09-13T03:29:13.010 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:13.010 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.010 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.011 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5000", 2023-09-13T03:29:13.011 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.011 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.012 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.012 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-09-13T03:29:13.012 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.012 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.013 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.013 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.013 INFO:teuthology.orchestra.run.smithi064.stdout: "database": { 2023-09-13T03:29:13.013 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "database", 2023-09-13T03:29:13.014 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.014 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.014 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.015 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "ceph", 2023-09-13T03:29:13.015 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.015 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.015 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.016 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-09-13T03:29:13.016 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.016 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.017 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.017 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.017 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": { 2023-09-13T03:29:13.017 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "hostname", 2023-09-13T03:29:13.018 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.018 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.018 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.019 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.019 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.019 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.019 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.020 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB server hostname", 2023-09-13T03:29:13.020 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.020 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.020 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.021 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.021 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2023-09-13T03:29:13.021 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2023-09-13T03:29:13.022 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:13.022 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.022 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.022 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "30", 2023-09-13T03:29:13.023 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "5", 2023-09-13T03:29:13.023 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.023 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.023 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-09-13T03:29:13.024 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.024 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.024 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.025 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.025 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:13.025 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:13.025 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.026 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.026 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.026 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.026 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.027 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.027 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.027 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.027 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.028 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.028 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.028 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.029 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.029 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.029 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.029 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.030 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.030 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.030 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.030 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:13.031 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:13.031 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.031 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.031 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.032 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.032 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.032 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.033 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.033 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.033 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.033 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.034 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.034 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.034 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:13.035 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:13.035 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.035 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.035 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.036 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:13.036 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.036 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.036 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.037 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.037 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.037 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.037 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.038 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.038 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.038 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.039 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.039 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.039 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.039 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.040 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.040 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:13.040 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:13.040 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.041 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.041 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.041 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.042 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.042 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.042 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.042 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.043 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.043 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.043 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.043 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.044 INFO:teuthology.orchestra.run.smithi064.stdout: "password": { 2023-09-13T03:29:13.044 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "password", 2023-09-13T03:29:13.044 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.044 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.045 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.045 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.045 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.045 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.046 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.046 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "password of InfluxDB server user", 2023-09-13T03:29:13.046 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.047 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.047 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.047 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.048 INFO:teuthology.orchestra.run.smithi064.stdout: "port": { 2023-09-13T03:29:13.048 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "port", 2023-09-13T03:29:13.048 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:13.048 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.049 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.049 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8086", 2023-09-13T03:29:13.049 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.049 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.050 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.050 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB server port", 2023-09-13T03:29:13.050 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.051 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.051 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.051 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.051 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl": { 2023-09-13T03:29:13.052 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl", 2023-09-13T03:29:13.052 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.052 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.052 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.053 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "false", 2023-09-13T03:29:13.053 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.053 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.053 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.054 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-09-13T03:29:13.054 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.054 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.055 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.055 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.055 INFO:teuthology.orchestra.run.smithi064.stdout: "threads": { 2023-09-13T03:29:13.055 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "threads", 2023-09-13T03:29:13.056 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:13.056 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.056 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.056 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2023-09-13T03:29:13.057 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2023-09-13T03:29:13.057 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "32", 2023-09-13T03:29:13.057 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.057 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-09-13T03:29:13.058 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.058 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.058 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.058 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.059 INFO:teuthology.orchestra.run.smithi064.stdout: "username": { 2023-09-13T03:29:13.059 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "username", 2023-09-13T03:29:13.059 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.060 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.060 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.060 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.060 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.061 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.061 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.061 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "username of InfluxDB server user", 2023-09-13T03:29:13.061 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.062 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.062 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.062 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.063 INFO:teuthology.orchestra.run.smithi064.stdout: "verify_ssl": { 2023-09-13T03:29:13.063 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "verify_ssl", 2023-09-13T03:29:13.063 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.063 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.064 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.064 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "true", 2023-09-13T03:29:13.064 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.064 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.065 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.065 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-09-13T03:29:13.065 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.066 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.066 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.066 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.066 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.067 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.067 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:13.067 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "insights", 2023-09-13T03:29:13.067 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:13.068 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:13.068 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:13.068 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:13.069 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:13.069 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.069 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.069 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.070 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.070 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.070 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.071 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.071 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.071 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.071 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.072 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.072 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.072 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.072 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.073 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.073 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.073 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.073 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.074 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.074 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:13.074 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:13.075 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.075 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.075 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.075 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.076 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.076 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.076 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.077 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.077 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.077 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.077 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.078 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.078 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:13.078 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:13.079 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.079 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.079 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.079 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:13.080 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.080 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.080 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.081 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.081 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.081 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.081 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.082 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.082 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.082 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.082 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.083 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.083 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.083 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.084 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.084 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:13.084 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:13.084 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.085 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.085 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.085 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.085 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.086 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.086 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.086 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.087 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.087 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.087 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.088 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.088 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.088 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.088 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:13.089 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "iostat", 2023-09-13T03:29:13.089 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:13.089 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:13.089 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:13.090 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:13.090 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:13.090 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.090 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.091 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.091 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.091 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.092 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.092 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.092 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.092 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.093 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.093 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.093 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.093 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.094 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.094 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.094 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.095 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.095 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.095 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.095 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:13.096 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:13.096 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.096 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.096 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.097 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.097 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.097 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.097 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.098 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.098 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.098 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.099 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.099 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.099 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:13.100 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:13.100 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.100 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.100 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.101 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:13.101 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.101 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.101 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.102 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.102 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.102 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.102 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.103 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.103 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.103 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.103 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.104 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.104 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.104 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.105 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.105 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:13.105 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:13.105 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.106 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.106 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.106 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.106 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.107 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.107 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.107 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.107 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.108 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.108 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.108 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.108 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.109 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.109 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:13.109 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "k8sevents", 2023-09-13T03:29:13.110 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:13.110 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:13.110 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:13.110 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_event_retention_days": { 2023-09-13T03:29:13.111 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ceph_event_retention_days", 2023-09-13T03:29:13.111 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:13.111 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.111 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.112 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "7", 2023-09-13T03:29:13.112 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.112 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.112 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.113 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Days to hold ceph event information within local cache", 2023-09-13T03:29:13.113 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.113 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.114 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.114 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.114 INFO:teuthology.orchestra.run.smithi064.stdout: "config_check_secs": { 2023-09-13T03:29:13.114 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_check_secs", 2023-09-13T03:29:13.115 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:13.115 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.115 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.115 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2023-09-13T03:29:13.116 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "10", 2023-09-13T03:29:13.116 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.116 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.117 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-09-13T03:29:13.117 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.117 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.117 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.118 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.118 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:13.118 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:13.119 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.119 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.119 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.119 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.120 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.120 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.120 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.121 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.121 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.121 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.121 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.122 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.122 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.122 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.122 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.123 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.123 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.123 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.124 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.124 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:13.124 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:13.124 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.125 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.125 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.125 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.125 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.126 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.126 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.126 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.126 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.127 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.127 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.127 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.128 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:13.128 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:13.128 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.128 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.129 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.129 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:13.129 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.130 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.130 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.130 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.130 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.131 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.131 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.131 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.131 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.132 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.132 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.132 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.133 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.133 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.133 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:13.134 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:13.134 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.134 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.134 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.135 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.135 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.135 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.135 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.136 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.136 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.136 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.136 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.137 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.137 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.137 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:13.138 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "localpool", 2023-09-13T03:29:13.138 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:13.138 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:13.138 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:13.139 INFO:teuthology.orchestra.run.smithi064.stdout: "failure_domain": { 2023-09-13T03:29:13.139 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "failure_domain", 2023-09-13T03:29:13.139 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.139 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.140 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.140 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "host", 2023-09-13T03:29:13.140 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.140 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.141 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.141 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "failure domain for any created local pool", 2023-09-13T03:29:13.141 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-09-13T03:29:13.142 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.142 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.142 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.142 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:13.143 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:13.143 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.143 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.143 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.144 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.144 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.144 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.144 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.145 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.145 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.145 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.146 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.146 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.146 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.146 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.147 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.147 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.147 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.148 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.148 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.148 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:13.148 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:13.149 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.149 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.149 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.150 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.150 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.150 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.150 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.151 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.151 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.151 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.151 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.152 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.152 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:13.152 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:13.152 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.153 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.153 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.153 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:13.154 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.154 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.154 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.154 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.155 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.155 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.155 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.156 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.156 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.156 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.156 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.157 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.157 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.157 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.157 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.158 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:13.158 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:13.158 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.159 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.159 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.159 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.159 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.160 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.160 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.160 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.160 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.161 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.161 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.161 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.162 INFO:teuthology.orchestra.run.smithi064.stdout: "min_size": { 2023-09-13T03:29:13.162 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "min_size", 2023-09-13T03:29:13.162 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:13.162 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.163 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.163 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.163 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.164 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.164 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.164 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default min_size for any created local pool", 2023-09-13T03:29:13.164 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-09-13T03:29:13.165 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.165 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.165 INFO:teuthology.orchestra.run.smithi064.stdout: "num_rep": { 2023-09-13T03:29:13.166 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "num_rep", 2023-09-13T03:29:13.166 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:13.166 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.167 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.167 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3", 2023-09-13T03:29:13.167 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.168 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.168 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.168 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default replica count for any created local pool", 2023-09-13T03:29:13.168 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.169 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.169 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.169 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.169 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_num": { 2023-09-13T03:29:13.170 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pg_num", 2023-09-13T03:29:13.170 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:13.170 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.171 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.171 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "128", 2023-09-13T03:29:13.171 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.171 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.172 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.172 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default pg_num for any created local pool", 2023-09-13T03:29:13.172 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.172 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.173 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.173 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.173 INFO:teuthology.orchestra.run.smithi064.stdout: "prefix": { 2023-09-13T03:29:13.173 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prefix", 2023-09-13T03:29:13.174 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.174 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.174 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.175 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.175 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.175 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.175 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.176 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "name prefix for any created local pool", 2023-09-13T03:29:13.176 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.176 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.177 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.177 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.177 INFO:teuthology.orchestra.run.smithi064.stdout: "subtree": { 2023-09-13T03:29:13.177 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "subtree", 2023-09-13T03:29:13.178 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.178 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.178 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.178 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "rack", 2023-09-13T03:29:13.179 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.179 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.179 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.180 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "CRUSH level for which to create a local pool", 2023-09-13T03:29:13.180 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-09-13T03:29:13.180 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.180 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.181 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.181 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.181 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:13.182 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mds_autoscaler", 2023-09-13T03:29:13.182 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:13.182 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:13.183 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:13.183 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:13.183 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:13.183 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.184 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.184 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.184 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.184 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.185 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.185 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.185 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.186 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.186 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.186 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.186 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.187 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.187 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.187 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.188 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.188 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.188 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.189 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:13.189 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:13.189 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.190 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.190 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.190 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.190 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.191 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.191 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.191 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.192 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.192 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.192 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.192 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.193 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:13.193 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:13.193 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.193 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.194 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.194 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:13.194 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.194 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.195 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.195 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.195 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.195 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.196 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.196 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.196 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.197 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.197 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.197 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.197 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.198 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.198 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.198 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:13.199 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:13.199 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.199 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.199 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.200 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.200 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.200 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.201 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.201 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.201 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.201 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.202 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.202 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.202 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.203 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.203 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:13.203 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mirroring", 2023-09-13T03:29:13.203 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:13.204 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:13.204 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:13.204 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:13.204 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:13.205 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.205 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.205 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.206 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.206 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.206 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.206 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.207 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.207 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.207 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.208 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.208 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.208 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.209 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.209 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.209 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.209 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.210 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.210 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.210 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:13.210 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:13.211 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.211 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.211 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.211 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.212 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.212 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.212 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.213 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.213 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.213 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.213 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.214 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.214 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:13.214 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:13.215 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.215 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.215 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.215 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:13.216 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.216 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.216 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.216 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.217 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.217 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.217 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.218 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.218 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.218 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.219 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.219 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.219 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.219 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.220 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.220 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:13.220 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:13.220 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.221 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.221 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.221 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.222 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.222 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.222 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.223 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.223 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.223 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.223 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.224 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.224 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.224 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.225 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:13.225 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "nfs", 2023-09-13T03:29:13.225 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:13.225 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:13.226 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:13.226 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:13.226 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:13.227 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.227 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.227 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.227 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.228 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.228 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.228 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.229 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.229 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.229 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.230 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.230 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.230 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.230 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.231 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.231 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.231 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.231 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.232 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.232 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:13.232 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:13.233 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.233 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.233 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.233 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.234 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.234 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.234 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.234 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.235 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.235 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.235 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.236 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.236 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:13.236 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:13.236 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.237 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.237 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.237 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:13.237 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.238 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.238 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.238 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.239 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.239 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.239 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.240 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.240 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.240 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.240 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.241 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.241 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.241 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.241 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.242 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:13.242 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:13.242 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.243 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.243 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.243 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.244 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.244 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.244 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.244 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.245 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.245 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.245 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.245 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.246 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.246 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.246 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:13.247 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "orchestrator", 2023-09-13T03:29:13.247 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:13.247 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:13.247 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:13.248 INFO:teuthology.orchestra.run.smithi064.stdout: "fail_fs": { 2023-09-13T03:29:13.248 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "fail_fs", 2023-09-13T03:29:13.248 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.249 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.249 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.249 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.249 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.250 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.250 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.250 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-09-13T03:29:13.250 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.251 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.251 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.251 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.252 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:13.252 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:13.252 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.252 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.253 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.253 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.253 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.253 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.254 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.254 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.254 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.255 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.255 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.255 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.255 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.256 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.256 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.256 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.257 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.257 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.257 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.257 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:13.258 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:13.258 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.258 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.259 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.259 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.259 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.259 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.260 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.260 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.260 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.261 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.261 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.261 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.261 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:13.262 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:13.262 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.262 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.262 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.263 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:13.263 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.263 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.264 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.264 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.264 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.264 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.265 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.265 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.265 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.266 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.266 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.266 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.266 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.267 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.267 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.267 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:13.267 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:13.268 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.268 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.268 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.269 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.269 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.269 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.269 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.270 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.270 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.270 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.271 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.271 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.271 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator": { 2023-09-13T03:29:13.271 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "orchestrator", 2023-09-13T03:29:13.272 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.272 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.272 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.273 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.273 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.273 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.274 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.274 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm", 2023-09-13T03:29:13.274 INFO:teuthology.orchestra.run.smithi064.stdout: "rook", 2023-09-13T03:29:13.274 INFO:teuthology.orchestra.run.smithi064.stdout: "test_orchestrator" 2023-09-13T03:29:13.275 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.275 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Orchestrator backend", 2023-09-13T03:29:13.275 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.276 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.276 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.276 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.276 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.277 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.277 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:13.277 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "osd_perf_query", 2023-09-13T03:29:13.278 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:13.278 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:13.278 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:13.278 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:13.279 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:13.279 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.279 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.280 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.280 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.280 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.280 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.281 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.281 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.281 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.281 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.282 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.282 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.282 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.283 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.283 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.283 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.283 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.284 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.284 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.284 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:13.285 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:13.285 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.285 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.285 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.286 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.286 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.286 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.287 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.287 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.287 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.287 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.288 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.288 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.288 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:13.289 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:13.289 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.289 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.289 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.290 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:13.290 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.290 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.290 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.291 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.291 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.291 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.292 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.292 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.292 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.292 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.293 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.293 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.293 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.293 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.294 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.294 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:13.294 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:13.295 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.295 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.295 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.295 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.296 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.296 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.296 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.297 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.297 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.297 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.297 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.298 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.298 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.298 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.299 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:13.299 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "osd_support", 2023-09-13T03:29:13.299 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:13.299 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:13.300 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:13.300 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:13.300 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:13.300 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.301 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.301 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.301 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.301 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.302 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.302 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.302 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.303 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.303 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.303 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.303 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.304 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.304 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.304 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.305 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.305 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.305 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.305 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.306 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:13.306 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:13.306 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.306 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.307 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.307 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.307 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.308 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.308 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.308 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.308 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.309 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.309 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.309 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.309 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:13.310 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:13.310 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.310 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.311 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.311 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:13.311 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.312 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.312 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.312 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.312 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.313 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.313 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.313 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.313 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.314 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.314 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.314 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.315 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.315 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.315 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.315 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:13.316 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:13.316 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.316 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.316 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.317 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.317 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.317 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.318 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.318 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.318 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.319 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.319 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.319 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.319 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.320 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.320 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:13.320 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pg_autoscaler", 2023-09-13T03:29:13.320 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:13.321 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:13.321 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:13.321 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:13.322 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:13.322 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.322 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.322 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.323 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.323 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.323 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.323 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.324 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.324 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.324 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.325 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.325 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.325 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.326 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.326 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.326 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.326 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.327 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.327 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.327 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:13.328 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:13.328 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.328 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.328 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.329 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.329 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.329 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.330 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.330 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.330 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.331 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.331 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.333 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.333 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:13.333 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:13.334 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.334 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.334 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.334 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:13.335 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.335 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.335 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.336 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.336 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.336 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.336 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.337 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.337 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.337 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.338 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.338 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.338 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.338 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.339 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.339 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:13.339 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:13.340 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.340 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.340 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.340 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.341 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.341 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.341 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.342 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.342 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.342 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.342 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.343 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.343 INFO:teuthology.orchestra.run.smithi064.stdout: "noautoscale": { 2023-09-13T03:29:13.343 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "noautoscale", 2023-09-13T03:29:13.344 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.344 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.344 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.344 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.345 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.345 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.345 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.346 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "global autoscale flag", 2023-09-13T03:29:13.346 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-09-13T03:29:13.346 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.346 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.347 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.347 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2023-09-13T03:29:13.347 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2023-09-13T03:29:13.347 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:13.348 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.348 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.348 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2023-09-13T03:29:13.349 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.349 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.349 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.349 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.350 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.350 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.350 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.351 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.351 INFO:teuthology.orchestra.run.smithi064.stdout: "threshold": { 2023-09-13T03:29:13.351 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "threshold", 2023-09-13T03:29:13.351 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2023-09-13T03:29:13.352 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.352 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.352 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3.0", 2023-09-13T03:29:13.352 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1.0", 2023-09-13T03:29:13.353 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.353 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.353 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "scaling threshold", 2023-09-13T03:29:13.353 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-09-13T03:29:13.354 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.354 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.354 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.355 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.355 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.355 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:13.356 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "progress", 2023-09-13T03:29:13.356 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:13.356 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:13.356 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:13.357 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_pg_recovery_event": { 2023-09-13T03:29:13.357 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_pg_recovery_event", 2023-09-13T03:29:13.357 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.357 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.358 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.358 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.358 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.359 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.359 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.359 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow the module to show pg recovery progress", 2023-09-13T03:29:13.359 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.360 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.360 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.360 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.360 INFO:teuthology.orchestra.run.smithi064.stdout: "enabled": { 2023-09-13T03:29:13.361 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enabled", 2023-09-13T03:29:13.361 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.361 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.361 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.362 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:13.362 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.362 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.363 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.363 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.363 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.363 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.364 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.364 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.364 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:13.365 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:13.365 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.365 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.365 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.366 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.366 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.366 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.367 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.367 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.367 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.367 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.368 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.368 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.368 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.369 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.369 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.369 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.369 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.370 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.370 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.370 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:13.371 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:13.371 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.371 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.371 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.372 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.372 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.372 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.373 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.373 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.373 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.373 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.374 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.374 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.374 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:13.375 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:13.375 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.375 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.375 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.376 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:13.376 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.376 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.377 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.377 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.377 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.377 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.378 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.378 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.378 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.379 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.379 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.379 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.380 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.380 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.380 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.380 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:13.381 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:13.381 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.381 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.382 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.382 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.382 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.383 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.383 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.383 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.383 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.384 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.384 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.384 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.385 INFO:teuthology.orchestra.run.smithi064.stdout: "max_completed_events": { 2023-09-13T03:29:13.385 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_completed_events", 2023-09-13T03:29:13.385 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:13.385 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.386 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.386 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "50", 2023-09-13T03:29:13.386 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.387 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.387 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.387 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "number of past completed events to remember", 2023-09-13T03:29:13.387 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.388 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.388 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.388 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.389 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2023-09-13T03:29:13.389 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2023-09-13T03:29:13.389 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:13.389 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.390 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.390 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2023-09-13T03:29:13.390 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.391 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.391 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.391 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long the module is going to sleep", 2023-09-13T03:29:13.391 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.392 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.392 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.392 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.393 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.393 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.393 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:13.394 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prometheus", 2023-09-13T03:29:13.394 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:13.394 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:13.394 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:13.395 INFO:teuthology.orchestra.run.smithi064.stdout: "cache": { 2023-09-13T03:29:13.395 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cache", 2023-09-13T03:29:13.395 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.396 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.396 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.396 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:13.396 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.397 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.397 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.397 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.398 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.398 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.398 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.399 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.399 INFO:teuthology.orchestra.run.smithi064.stdout: "exclude_perf_counters": { 2023-09-13T03:29:13.399 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "exclude_perf_counters", 2023-09-13T03:29:13.399 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.400 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.400 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.400 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:13.400 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.401 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.401 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.401 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-09-13T03:29:13.402 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-09-13T03:29:13.402 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.402 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.402 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.403 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:13.403 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:13.403 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.404 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.404 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.404 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.404 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.405 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.405 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.405 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.406 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.406 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.406 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.406 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.407 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.407 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.407 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.408 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.408 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.408 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.408 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.409 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:13.409 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:13.409 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.409 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.410 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.410 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.410 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.411 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.411 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.411 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.411 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.412 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.412 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.412 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.413 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:13.413 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:13.413 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.413 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.414 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.414 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:13.414 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.415 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.415 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.415 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.415 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.416 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.416 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.416 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.416 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.417 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.417 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.417 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.418 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.418 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.418 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.419 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:13.419 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:13.419 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.419 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.420 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.420 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.420 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.420 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.421 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.421 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.421 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.422 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.422 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.422 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.422 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_stats_pools": { 2023-09-13T03:29:13.423 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_stats_pools", 2023-09-13T03:29:13.423 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.423 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.423 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.424 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.424 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.424 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.425 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.425 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.425 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.425 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.426 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.426 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.426 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_stats_pools_refresh_interval": { 2023-09-13T03:29:13.427 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-09-13T03:29:13.427 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:13.427 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.427 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.428 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "300", 2023-09-13T03:29:13.428 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.428 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.429 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.429 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.429 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.429 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.430 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.430 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.430 INFO:teuthology.orchestra.run.smithi064.stdout: "scrape_interval": { 2023-09-13T03:29:13.431 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "scrape_interval", 2023-09-13T03:29:13.431 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2023-09-13T03:29:13.431 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.432 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.432 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15.0", 2023-09-13T03:29:13.432 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.432 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.433 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.433 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.433 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.433 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.434 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.434 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.434 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2023-09-13T03:29:13.435 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2023-09-13T03:29:13.435 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.435 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.435 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.436 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "::", 2023-09-13T03:29:13.436 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.436 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.437 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.437 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-09-13T03:29:13.437 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.437 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.438 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.438 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.438 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2023-09-13T03:29:13.439 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2023-09-13T03:29:13.439 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:13.439 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.439 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.440 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "9283", 2023-09-13T03:29:13.440 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.440 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.440 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.441 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-09-13T03:29:13.441 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.441 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.442 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.442 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.442 INFO:teuthology.orchestra.run.smithi064.stdout: "stale_cache_strategy": { 2023-09-13T03:29:13.442 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "stale_cache_strategy", 2023-09-13T03:29:13.443 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.443 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.443 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.444 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "log", 2023-09-13T03:29:13.444 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.444 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.444 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.445 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.445 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.445 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.446 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.446 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.446 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_behaviour": { 2023-09-13T03:29:13.446 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_behaviour", 2023-09-13T03:29:13.447 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.447 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.447 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.448 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "default", 2023-09-13T03:29:13.448 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.448 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.448 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.449 INFO:teuthology.orchestra.run.smithi064.stdout: "default", 2023-09-13T03:29:13.449 INFO:teuthology.orchestra.run.smithi064.stdout: "error" 2023-09-13T03:29:13.449 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.450 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.450 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.450 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.450 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.451 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.451 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_error_status_code": { 2023-09-13T03:29:13.451 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_error_status_code", 2023-09-13T03:29:13.452 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:13.452 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.452 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.453 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "500", 2023-09-13T03:29:13.453 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "400", 2023-09-13T03:29:13.453 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "599", 2023-09-13T03:29:13.453 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.454 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.454 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.454 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.454 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.455 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.455 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.455 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.456 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:13.456 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_support", 2023-09-13T03:29:13.456 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:13.456 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:13.457 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:13.457 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:13.457 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:13.457 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.458 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.458 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.458 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.459 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.459 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.459 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.459 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.460 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.460 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.460 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.461 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.461 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.461 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.461 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.462 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.462 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.462 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.463 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.463 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:13.463 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:13.463 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.464 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.464 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.464 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.464 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.465 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.465 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.465 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.466 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.466 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.466 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.466 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.467 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:13.467 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:13.467 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.468 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.468 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.468 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:13.468 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.469 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.469 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.469 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.470 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.470 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.470 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.470 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.471 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.471 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.471 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.472 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.472 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.472 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.473 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.473 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:13.473 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:13.474 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.474 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.474 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.474 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.475 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.475 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.475 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.476 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.476 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.476 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.476 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.477 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.477 INFO:teuthology.orchestra.run.smithi064.stdout: "max_concurrent_snap_create": { 2023-09-13T03:29:13.477 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_concurrent_snap_create", 2023-09-13T03:29:13.478 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:13.478 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.478 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.478 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2023-09-13T03:29:13.479 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.479 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.479 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.480 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.480 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.480 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.480 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.481 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.481 INFO:teuthology.orchestra.run.smithi064.stdout: "mirror_snapshot_schedule": { 2023-09-13T03:29:13.481 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mirror_snapshot_schedule", 2023-09-13T03:29:13.482 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.482 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.482 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.483 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.483 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.483 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.483 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.484 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.484 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.484 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.484 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.485 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.485 INFO:teuthology.orchestra.run.smithi064.stdout: "trash_purge_schedule": { 2023-09-13T03:29:13.485 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "trash_purge_schedule", 2023-09-13T03:29:13.485 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.486 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.486 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.486 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.487 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.487 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.487 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.487 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.488 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.488 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.488 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.488 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.489 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.489 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.489 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:13.490 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "restful", 2023-09-13T03:29:13.490 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:13.490 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:13.490 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:13.491 INFO:teuthology.orchestra.run.smithi064.stdout: "enable_auth": { 2023-09-13T03:29:13.491 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enable_auth", 2023-09-13T03:29:13.491 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.492 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.492 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.492 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:13.492 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.493 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.493 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.493 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.494 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.494 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.494 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.495 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.495 INFO:teuthology.orchestra.run.smithi064.stdout: "key_file": { 2023-09-13T03:29:13.495 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "key_file", 2023-09-13T03:29:13.496 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.496 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.496 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.496 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.497 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.497 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.497 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.498 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.498 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.498 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.499 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.499 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.499 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:13.499 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:13.500 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.500 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.500 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.501 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.501 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.501 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.501 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.502 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.502 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.502 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.503 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.503 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.503 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.504 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.504 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.504 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.504 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.505 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.505 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.505 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:13.506 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:13.506 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.506 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.506 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.507 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.507 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.507 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.507 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.508 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.508 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.508 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.509 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.509 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.509 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:13.509 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:13.510 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.510 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.510 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.510 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:13.511 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.511 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.511 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.512 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.512 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.512 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.512 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.513 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.513 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.513 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.514 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.514 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.514 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.514 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.515 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.515 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:13.515 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:13.515 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.516 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.516 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.516 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.517 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.517 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.517 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.517 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.518 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.518 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.518 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.518 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.519 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2023-09-13T03:29:13.519 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2023-09-13T03:29:13.519 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.520 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.520 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.520 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.520 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.521 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.521 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.521 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.521 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.522 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.522 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.522 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.523 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2023-09-13T03:29:13.523 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2023-09-13T03:29:13.523 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.523 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.524 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.524 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.524 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.524 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.525 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.525 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.525 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.526 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.526 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.526 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.526 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.527 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.527 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:13.527 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rgw", 2023-09-13T03:29:13.528 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:13.528 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:13.528 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:13.528 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:13.529 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:13.529 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.529 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.529 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.530 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.530 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.530 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.531 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.531 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.531 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.532 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.532 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.532 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.532 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.533 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.533 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.533 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.534 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.534 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.534 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.534 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:13.535 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:13.535 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.535 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.535 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.536 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.536 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.536 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.537 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.537 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.537 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.537 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.538 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.538 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.538 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:13.539 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:13.539 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.539 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.540 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.540 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:13.540 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.540 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.541 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.541 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.541 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.542 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.542 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.542 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.543 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.543 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.543 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.543 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.544 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.544 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.544 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.544 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:13.545 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:13.545 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.545 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.546 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.546 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.546 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.547 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.547 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.547 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.547 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.548 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.548 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.548 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.549 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.549 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.549 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:13.549 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rook", 2023-09-13T03:29:13.550 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:13.550 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:13.550 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:13.550 INFO:teuthology.orchestra.run.smithi064.stdout: "drive_group_interval": { 2023-09-13T03:29:13.551 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "drive_group_interval", 2023-09-13T03:29:13.551 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2023-09-13T03:29:13.551 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.552 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.552 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "300.0", 2023-09-13T03:29:13.552 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.552 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.553 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.553 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-09-13T03:29:13.553 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.553 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.554 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.554 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.554 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:13.555 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:13.555 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.555 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.555 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.556 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.556 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.556 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.556 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.557 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.557 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.557 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.558 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.558 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.558 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.559 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.559 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.559 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.559 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.560 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.560 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.560 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:13.560 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:13.561 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.561 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.561 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.561 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.562 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.562 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.562 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.562 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.563 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.563 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.563 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.563 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.564 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:13.564 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:13.564 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.564 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.565 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.565 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:13.565 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.565 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.566 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.566 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.566 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.567 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.567 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.567 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.567 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.568 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.568 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.568 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.568 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.569 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.569 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.569 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:13.570 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:13.570 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.570 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.570 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.571 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.571 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.571 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.572 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.572 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.572 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.572 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.573 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.573 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.573 INFO:teuthology.orchestra.run.smithi064.stdout: "storage_class": { 2023-09-13T03:29:13.573 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "storage_class", 2023-09-13T03:29:13.574 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.574 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.574 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.575 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "local", 2023-09-13T03:29:13.575 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.575 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.575 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.576 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-09-13T03:29:13.576 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.576 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.576 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.577 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.577 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.577 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.578 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:13.578 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "selftest", 2023-09-13T03:29:13.578 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:13.578 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:13.579 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:13.579 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:13.579 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:13.580 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.580 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.580 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.580 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.581 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.581 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.581 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.581 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.582 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.582 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.582 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.582 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.583 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.583 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.583 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.584 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.584 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.584 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.584 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.585 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:13.585 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:13.585 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.585 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.586 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.586 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.586 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.587 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.587 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.587 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.587 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.588 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.588 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.588 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.588 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:13.589 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:13.589 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.589 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.590 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.590 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:13.590 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.590 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.591 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.591 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.591 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.591 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.592 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.592 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.592 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.593 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.593 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.593 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.593 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.594 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.594 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.594 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:13.595 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:13.595 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.595 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.595 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.596 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.596 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.596 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.597 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.597 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.597 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.597 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.598 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.598 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.598 INFO:teuthology.orchestra.run.smithi064.stdout: "roption1": { 2023-09-13T03:29:13.598 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "roption1", 2023-09-13T03:29:13.599 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.599 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.599 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.600 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.600 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.600 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.601 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.601 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.601 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.601 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.602 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.602 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.602 INFO:teuthology.orchestra.run.smithi064.stdout: "roption2": { 2023-09-13T03:29:13.602 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "roption2", 2023-09-13T03:29:13.603 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.603 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.603 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.604 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "xyz", 2023-09-13T03:29:13.604 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.604 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.604 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.605 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.605 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.605 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.605 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.606 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.606 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption1": { 2023-09-13T03:29:13.606 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption1", 2023-09-13T03:29:13.607 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.607 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.607 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.607 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.608 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.608 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.608 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.608 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.609 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.609 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.609 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.610 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.610 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption2": { 2023-09-13T03:29:13.610 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption2", 2023-09-13T03:29:13.610 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:13.611 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.611 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.611 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.611 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.612 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.612 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.612 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.612 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.613 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.613 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.613 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.613 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption3": { 2023-09-13T03:29:13.614 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption3", 2023-09-13T03:29:13.614 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2023-09-13T03:29:13.614 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.615 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.615 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.615 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.615 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.616 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.616 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.616 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.616 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.617 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.617 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.617 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption4": { 2023-09-13T03:29:13.617 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption4", 2023-09-13T03:29:13.618 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.618 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.618 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.618 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.619 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.619 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.619 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.620 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.620 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.620 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.620 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.621 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.621 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption5": { 2023-09-13T03:29:13.621 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption5", 2023-09-13T03:29:13.621 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.622 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.622 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.622 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.623 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.623 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.623 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.623 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.624 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.624 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.624 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.625 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.625 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption6": { 2023-09-13T03:29:13.625 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption6", 2023-09-13T03:29:13.625 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.625 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.626 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.626 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:13.626 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.627 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.627 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.627 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.627 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.628 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.628 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.628 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.629 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption7": { 2023-09-13T03:29:13.629 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption7", 2023-09-13T03:29:13.629 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:13.629 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.630 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.630 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.630 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2023-09-13T03:29:13.630 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "42", 2023-09-13T03:29:13.631 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.631 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.631 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.631 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.632 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.632 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.632 INFO:teuthology.orchestra.run.smithi064.stdout: "testkey": { 2023-09-13T03:29:13.632 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testkey", 2023-09-13T03:29:13.633 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.633 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.633 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.633 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.634 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.634 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.634 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.634 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.635 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.635 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.635 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.635 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.636 INFO:teuthology.orchestra.run.smithi064.stdout: "testlkey": { 2023-09-13T03:29:13.636 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testlkey", 2023-09-13T03:29:13.636 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.636 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.637 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.637 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.637 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.637 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.638 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.638 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.638 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.639 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.639 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.639 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.639 INFO:teuthology.orchestra.run.smithi064.stdout: "testnewline": { 2023-09-13T03:29:13.640 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testnewline", 2023-09-13T03:29:13.640 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.640 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.641 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.641 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.641 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.641 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.642 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.642 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.642 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.642 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.643 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.643 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.643 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.643 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.644 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:13.644 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snap_schedule", 2023-09-13T03:29:13.644 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:13.644 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:13.645 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:13.645 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_m_granularity": { 2023-09-13T03:29:13.645 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_m_granularity", 2023-09-13T03:29:13.645 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.646 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.646 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.646 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.647 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.647 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.647 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.647 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow minute scheduled snapshots", 2023-09-13T03:29:13.648 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.648 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.648 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.649 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.649 INFO:teuthology.orchestra.run.smithi064.stdout: "dump_on_update": { 2023-09-13T03:29:13.649 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "dump_on_update", 2023-09-13T03:29:13.649 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.650 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.650 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.650 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.651 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.651 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.651 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.651 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "dump database to debug log on update", 2023-09-13T03:29:13.652 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.652 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.652 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.652 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.653 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:13.653 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:13.653 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.654 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.654 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.654 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.654 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.655 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.655 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.655 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.655 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.656 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.656 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.656 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.657 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.657 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.657 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.657 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.658 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.658 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.658 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.658 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:13.659 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:13.659 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.659 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.660 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.660 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.660 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.660 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.661 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.661 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.661 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.662 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.662 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.662 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.662 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:13.663 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:13.663 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.663 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.663 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.664 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:13.664 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.664 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.665 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.665 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.665 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.665 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.666 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.666 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.666 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.666 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.667 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.667 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.667 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.668 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.668 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.668 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:13.668 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:13.669 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.669 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.669 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.669 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.670 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.670 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.670 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.671 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.671 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.671 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.671 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.672 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.672 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.672 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.672 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:13.673 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "stats", 2023-09-13T03:29:13.673 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:13.673 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:13.674 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:13.674 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:13.674 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:13.674 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.675 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.675 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.675 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.676 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.676 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.676 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.676 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.677 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.677 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.677 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.677 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.678 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.678 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.678 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.679 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.679 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.679 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.679 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.680 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:13.680 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:13.680 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.680 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.681 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.681 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.681 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.682 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.682 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.682 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.682 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.683 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.683 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.683 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.684 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:13.684 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:13.684 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.684 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.685 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.685 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:13.685 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.686 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.686 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.686 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.686 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.687 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.687 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.687 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.687 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.688 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.688 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.688 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.689 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.689 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.689 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.689 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:13.690 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:13.690 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.690 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.691 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.691 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.691 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.691 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.692 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.692 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.692 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.692 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.693 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.693 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.693 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.693 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.694 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:13.694 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "status", 2023-09-13T03:29:13.694 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:13.695 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:13.695 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:13.695 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:13.695 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:13.696 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.696 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.696 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.696 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.697 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.697 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.697 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.698 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.698 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.698 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.698 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.699 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.699 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.699 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.700 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.700 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.700 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.700 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.701 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.701 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:13.701 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:13.702 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.702 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.702 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.702 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.703 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.703 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.703 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.704 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.704 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.704 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.704 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.705 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.705 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:13.705 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:13.705 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.706 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.706 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.706 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:13.707 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.707 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.707 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.707 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.708 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.708 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.708 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.709 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.709 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.709 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.709 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.710 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.710 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.710 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.711 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.711 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:13.711 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:13.711 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.712 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.712 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.712 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.712 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.713 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.713 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.713 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.714 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.714 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.714 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.714 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.715 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.715 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.715 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:13.716 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "telegraf", 2023-09-13T03:29:13.716 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:13.716 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:13.716 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:13.717 INFO:teuthology.orchestra.run.smithi064.stdout: "address": { 2023-09-13T03:29:13.717 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "address", 2023-09-13T03:29:13.717 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.717 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.718 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.718 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-09-13T03:29:13.718 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.719 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.719 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.719 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.719 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.720 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.720 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.720 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.721 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2023-09-13T03:29:13.721 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2023-09-13T03:29:13.721 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:13.721 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.722 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.722 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15", 2023-09-13T03:29:13.722 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.723 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.723 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.723 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.723 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.724 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.724 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.724 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.725 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:13.725 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:13.725 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.725 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.726 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.726 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.726 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.726 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.727 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.727 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.727 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.728 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.728 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.728 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.728 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.729 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.729 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.729 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.730 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.730 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.730 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.730 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:13.731 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:13.731 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.731 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.732 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.732 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.732 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.732 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.733 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.733 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.733 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.734 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.734 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.734 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.734 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:13.735 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:13.735 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.735 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.736 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.736 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:13.736 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.736 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.737 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.737 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.737 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.737 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.738 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.738 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.738 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.739 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.739 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.739 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.740 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.740 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.740 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.740 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:13.741 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:13.741 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.741 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.741 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.742 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.742 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.742 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.743 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.743 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.743 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.743 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.744 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.744 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.744 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.745 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.745 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:13.745 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "telemetry", 2023-09-13T03:29:13.745 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:13.746 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:13.746 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:13.746 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_basic": { 2023-09-13T03:29:13.747 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_basic", 2023-09-13T03:29:13.747 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.747 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.747 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.748 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:13.748 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.748 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.749 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.749 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share basic cluster information (size, version)", 2023-09-13T03:29:13.749 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.749 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.750 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.750 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.750 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_crash": { 2023-09-13T03:29:13.751 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_crash", 2023-09-13T03:29:13.751 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.751 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.751 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.752 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:13.752 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.752 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.753 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.753 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-09-13T03:29:13.753 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.753 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.754 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.754 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.754 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_device": { 2023-09-13T03:29:13.755 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_device", 2023-09-13T03:29:13.755 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.755 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.755 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.756 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:13.756 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.756 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.756 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.757 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-09-13T03:29:13.757 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.757 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.758 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.758 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.758 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_ident": { 2023-09-13T03:29:13.758 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_ident", 2023-09-13T03:29:13.759 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.759 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.759 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.760 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.760 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.760 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.760 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.761 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-09-13T03:29:13.761 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.761 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.762 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.762 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.762 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_perf": { 2023-09-13T03:29:13.763 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_perf", 2023-09-13T03:29:13.763 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.763 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.763 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.764 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.764 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.764 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.764 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.765 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share various performance metrics of a cluster", 2023-09-13T03:29:13.765 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.765 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.765 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.766 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.766 INFO:teuthology.orchestra.run.smithi064.stdout: "contact": { 2023-09-13T03:29:13.766 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "contact", 2023-09-13T03:29:13.767 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.767 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.767 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.767 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.768 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.768 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.768 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.769 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.769 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.769 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.769 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.770 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.770 INFO:teuthology.orchestra.run.smithi064.stdout: "description": { 2023-09-13T03:29:13.770 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "description", 2023-09-13T03:29:13.771 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.771 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.771 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.771 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.772 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.772 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.772 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.773 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.773 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.773 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.773 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.774 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.774 INFO:teuthology.orchestra.run.smithi064.stdout: "device_url": { 2023-09-13T03:29:13.774 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_url", 2023-09-13T03:29:13.775 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.775 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.775 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.775 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-09-13T03:29:13.776 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.776 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.776 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.777 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.777 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.777 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.777 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.778 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.778 INFO:teuthology.orchestra.run.smithi064.stdout: "enabled": { 2023-09-13T03:29:13.778 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enabled", 2023-09-13T03:29:13.779 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.779 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.779 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.780 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.780 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.780 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.780 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.781 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.781 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.781 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.781 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.782 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.782 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2023-09-13T03:29:13.782 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2023-09-13T03:29:13.782 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:13.783 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.783 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.783 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "24", 2023-09-13T03:29:13.783 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "8", 2023-09-13T03:29:13.784 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.784 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.784 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.784 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.785 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.785 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.785 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.785 INFO:teuthology.orchestra.run.smithi064.stdout: "last_opt_revision": { 2023-09-13T03:29:13.786 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "last_opt_revision", 2023-09-13T03:29:13.786 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:13.786 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.786 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.787 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1", 2023-09-13T03:29:13.787 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.787 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.787 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.788 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.788 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.788 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.789 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.789 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.789 INFO:teuthology.orchestra.run.smithi064.stdout: "leaderboard": { 2023-09-13T03:29:13.789 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "leaderboard", 2023-09-13T03:29:13.790 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.790 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.790 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.791 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.791 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.791 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.791 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.792 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.792 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.792 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.792 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.793 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.793 INFO:teuthology.orchestra.run.smithi064.stdout: "leaderboard_description": { 2023-09-13T03:29:13.793 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "leaderboard_description", 2023-09-13T03:29:13.794 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.794 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.794 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.794 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.795 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.795 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.795 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.795 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.796 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.796 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.796 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.797 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.797 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:13.797 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:13.797 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.798 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.798 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.798 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.799 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.799 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.799 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.799 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.800 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.800 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.800 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.800 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.801 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.801 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.801 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.802 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.802 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.802 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.802 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.803 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:13.803 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:13.803 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.803 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.804 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.804 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.804 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.805 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.805 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.805 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.805 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.806 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.806 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.806 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.806 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:13.807 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:13.807 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.807 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.808 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.808 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:13.808 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.808 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.809 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.809 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.809 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.809 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.810 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.810 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.810 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.811 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.811 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.811 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.811 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.812 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.812 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.812 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:13.813 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:13.813 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.813 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.813 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.814 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.814 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.814 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.814 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.815 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.815 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.815 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.816 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.816 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.816 INFO:teuthology.orchestra.run.smithi064.stdout: "organization": { 2023-09-13T03:29:13.816 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "organization", 2023-09-13T03:29:13.817 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.817 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.817 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.817 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.818 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.818 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.818 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.819 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.819 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.819 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.819 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.820 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.820 INFO:teuthology.orchestra.run.smithi064.stdout: "proxy": { 2023-09-13T03:29:13.820 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "proxy", 2023-09-13T03:29:13.820 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.821 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.821 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.821 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.822 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.822 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.822 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.822 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.823 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.823 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.823 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.824 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.824 INFO:teuthology.orchestra.run.smithi064.stdout: "url": { 2023-09-13T03:29:13.824 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "url", 2023-09-13T03:29:13.824 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.825 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.825 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.825 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-09-13T03:29:13.826 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.826 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.826 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.826 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.827 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.827 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.827 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.828 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.828 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.828 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.828 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:13.829 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "test_orchestrator", 2023-09-13T03:29:13.829 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:13.829 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:13.829 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:13.830 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:13.830 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:13.830 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.831 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.831 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.831 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.831 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.832 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.832 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.832 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.833 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.833 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.833 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.833 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.834 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.834 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.834 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.835 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.835 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.835 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.835 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.836 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:13.836 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:13.836 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.836 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.837 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.837 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.837 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.837 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.838 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.838 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.838 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.839 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.839 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.839 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.839 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:13.840 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:13.840 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.840 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.840 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.841 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:13.841 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.841 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.841 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.842 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.842 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.842 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.842 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.843 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.843 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.843 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.843 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.844 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.844 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.844 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.845 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.845 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:13.845 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:13.845 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.846 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.846 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.846 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.846 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.847 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.847 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.847 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.847 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.848 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.848 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.848 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.849 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.849 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.849 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:13.849 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "volumes", 2023-09-13T03:29:13.850 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:13.850 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:13.850 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:13.851 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:13.851 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:13.851 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.851 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.852 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.852 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.852 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.853 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.853 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.853 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.853 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.854 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.854 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.854 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.854 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.855 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.855 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.855 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.856 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.856 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.856 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.856 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:13.857 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:13.857 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.857 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.857 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.858 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.858 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.858 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.859 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.859 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.859 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.859 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.860 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.860 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.860 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:13.860 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:13.861 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.861 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.861 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.861 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:13.862 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.862 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.862 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.863 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.863 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.863 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.863 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.864 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.864 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.864 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.865 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.865 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.865 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.865 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.866 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.866 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:13.866 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:13.867 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.867 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.867 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.867 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.868 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.868 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.868 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.868 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.869 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.869 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.869 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.870 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.870 INFO:teuthology.orchestra.run.smithi064.stdout: "max_concurrent_clones": { 2023-09-13T03:29:13.870 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_concurrent_clones", 2023-09-13T03:29:13.870 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:13.871 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.871 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.871 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "4", 2023-09-13T03:29:13.872 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.872 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.872 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.872 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Number of asynchronous cloner threads", 2023-09-13T03:29:13.873 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.873 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.873 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.873 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.874 INFO:teuthology.orchestra.run.smithi064.stdout: "snapshot_clone_delay": { 2023-09-13T03:29:13.874 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snapshot_clone_delay", 2023-09-13T03:29:13.874 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:13.875 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.875 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.875 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2023-09-13T03:29:13.876 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.876 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.876 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.876 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-09-13T03:29:13.877 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.877 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.877 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.877 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.878 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.878 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.878 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:13.879 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix", 2023-09-13T03:29:13.879 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:13.879 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:13.879 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:13.880 INFO:teuthology.orchestra.run.smithi064.stdout: "discovery_interval": { 2023-09-13T03:29:13.880 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "discovery_interval", 2023-09-13T03:29:13.880 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2023-09-13T03:29:13.881 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.881 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.881 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "100", 2023-09-13T03:29:13.881 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.881 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.882 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.882 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.882 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.883 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.883 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.883 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.883 INFO:teuthology.orchestra.run.smithi064.stdout: "identifier": { 2023-09-13T03:29:13.884 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "identifier", 2023-09-13T03:29:13.884 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.884 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.884 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.885 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.885 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.885 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.885 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.886 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.886 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.886 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.887 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.887 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.887 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2023-09-13T03:29:13.887 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2023-09-13T03:29:13.888 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:13.888 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.888 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.888 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2023-09-13T03:29:13.889 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.889 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.889 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.889 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.890 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.890 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.890 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.891 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.891 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:13.891 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:13.891 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.892 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.892 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.892 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.892 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.893 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.893 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.893 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.893 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.894 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.894 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.894 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.895 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.895 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.895 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.895 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.896 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.896 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.896 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.896 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:13.897 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:13.897 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.897 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.898 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.898 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.898 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.898 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.899 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.899 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.899 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.900 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.900 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.900 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.900 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:13.901 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:13.901 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.901 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.901 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.902 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:13.902 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.902 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.902 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.903 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.903 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.903 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.903 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.904 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.904 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.904 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.904 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.905 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.905 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.905 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.905 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.906 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:13.906 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:13.906 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.907 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.907 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.907 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.907 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.908 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.908 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.908 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.908 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.909 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.909 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.909 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.909 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_host": { 2023-09-13T03:29:13.910 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_host", 2023-09-13T03:29:13.910 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.910 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.911 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.911 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.911 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.911 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.912 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.912 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.912 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.913 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.913 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.913 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.913 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_port": { 2023-09-13T03:29:13.914 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_port", 2023-09-13T03:29:13.914 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:13.914 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.914 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.915 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10051", 2023-09-13T03:29:13.915 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.915 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.915 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.916 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.916 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.916 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.917 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.917 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.917 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_sender": { 2023-09-13T03:29:13.917 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_sender", 2023-09-13T03:29:13.918 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.918 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.918 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:13.919 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-09-13T03:29:13.919 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.919 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.919 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.920 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.920 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.920 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.921 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.921 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.921 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.921 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.922 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-09-13T03:29:13.922 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.922 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.922 INFO:teuthology.orchestra.run.smithi064.stdout: "modules": [ 2023-09-13T03:29:13.923 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm", 2023-09-13T03:29:13.923 INFO:teuthology.orchestra.run.smithi064.stdout: "dashboard", 2023-09-13T03:29:13.923 INFO:teuthology.orchestra.run.smithi064.stdout: "iostat", 2023-09-13T03:29:13.924 INFO:teuthology.orchestra.run.smithi064.stdout: "nfs", 2023-09-13T03:29:13.924 INFO:teuthology.orchestra.run.smithi064.stdout: "restful", 2023-09-13T03:29:13.924 INFO:teuthology.orchestra.run.smithi064.stdout: "snap_schedule" 2023-09-13T03:29:13.924 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.925 INFO:teuthology.orchestra.run.smithi064.stdout: "available_modules": [ 2023-09-13T03:29:13.925 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:13.925 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "alerts", 2023-09-13T03:29:13.925 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:13.926 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:13.926 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:13.926 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2023-09-13T03:29:13.927 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2023-09-13T03:29:13.927 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:13.927 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.927 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.928 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2023-09-13T03:29:13.928 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.928 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.928 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.929 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How frequently to reexamine health status", 2023-09-13T03:29:13.929 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.929 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.930 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.930 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.930 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:13.930 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:13.931 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.931 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.931 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.931 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.932 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.932 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.932 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.932 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.933 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.933 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.933 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.933 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.934 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.934 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.934 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.935 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.935 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.935 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.935 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.936 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:13.936 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:13.936 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.936 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.937 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.937 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.937 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.937 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.938 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.938 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.938 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.939 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.939 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.939 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.939 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:13.940 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:13.940 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.940 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.941 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.941 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:13.941 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.941 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.942 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:13.942 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:13.942 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:13.942 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:13.943 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:13.943 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:13.943 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:13.944 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:13.944 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.944 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.944 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.945 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.945 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.945 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:13.945 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:13.946 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.946 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.946 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.949 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:13.949 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.949 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.949 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.950 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:13.950 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.950 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.951 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.951 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.951 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_destination": { 2023-09-13T03:29:13.951 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_destination", 2023-09-13T03:29:13.952 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.952 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.952 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.952 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.953 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.953 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.953 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.953 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Email address to send alerts to", 2023-09-13T03:29:13.954 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.954 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.954 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.955 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.955 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_from_name": { 2023-09-13T03:29:13.955 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_from_name", 2023-09-13T03:29:13.955 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.955 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.956 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.956 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "Ceph", 2023-09-13T03:29:13.956 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.956 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.957 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.957 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Email From: name", 2023-09-13T03:29:13.957 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.957 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.958 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.958 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.958 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_host": { 2023-09-13T03:29:13.959 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_host", 2023-09-13T03:29:13.959 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.959 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.959 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.960 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.960 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.960 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.961 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.961 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP server", 2023-09-13T03:29:13.961 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.961 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.962 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.962 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.962 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_password": { 2023-09-13T03:29:13.962 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_password", 2023-09-13T03:29:13.963 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.963 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.963 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.964 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.964 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.964 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.964 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.965 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Password to authenticate with", 2023-09-13T03:29:13.965 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.965 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.965 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.966 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.966 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_port": { 2023-09-13T03:29:13.966 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_port", 2023-09-13T03:29:13.967 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:13.967 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.967 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.967 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "465", 2023-09-13T03:29:13.968 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.968 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.968 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.969 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP port", 2023-09-13T03:29:13.969 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.969 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.969 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.970 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.970 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_sender": { 2023-09-13T03:29:13.970 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_sender", 2023-09-13T03:29:13.970 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.971 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.971 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.971 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.971 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.972 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.972 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.972 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP envelope sender", 2023-09-13T03:29:13.973 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.973 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.973 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.973 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.973 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_ssl": { 2023-09-13T03:29:13.974 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_ssl", 2023-09-13T03:29:13.974 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.974 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.975 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.975 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:13.975 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.975 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.976 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.976 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use SSL to connect to SMTP server", 2023-09-13T03:29:13.976 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.977 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.977 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.977 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.977 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_user": { 2023-09-13T03:29:13.978 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_user", 2023-09-13T03:29:13.978 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.978 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.978 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.979 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:13.979 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.979 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.979 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.980 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "User to authenticate as", 2023-09-13T03:29:13.980 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.980 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.981 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.981 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.981 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:13.981 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.982 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:13.982 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "balancer", 2023-09-13T03:29:13.982 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:13.982 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:13.983 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:13.983 INFO:teuthology.orchestra.run.smithi064.stdout: "active": { 2023-09-13T03:29:13.983 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "active", 2023-09-13T03:29:13.984 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:13.984 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.984 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.984 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:13.985 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.985 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.985 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.985 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "automatically balance PGs across cluster", 2023-09-13T03:29:13.986 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.986 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.986 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.986 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.987 INFO:teuthology.orchestra.run.smithi064.stdout: "begin_time": { 2023-09-13T03:29:13.987 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "begin_time", 2023-09-13T03:29:13.987 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.988 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.988 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.988 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0000", 2023-09-13T03:29:13.988 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:13.989 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:13.989 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.989 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "beginning time of day to automatically balance", 2023-09-13T03:29:13.990 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-13T03:29:13.990 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.990 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.990 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.991 INFO:teuthology.orchestra.run.smithi064.stdout: "begin_weekday": { 2023-09-13T03:29:13.991 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "begin_weekday", 2023-09-13T03:29:13.991 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2023-09-13T03:29:13.991 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.992 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.992 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2023-09-13T03:29:13.992 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0", 2023-09-13T03:29:13.993 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "6", 2023-09-13T03:29:13.993 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.993 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-09-13T03:29:13.993 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-13T03:29:13.994 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.994 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.994 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.995 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_max_iterations": { 2023-09-13T03:29:13.995 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_max_iterations", 2023-09-13T03:29:13.995 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2023-09-13T03:29:13.995 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.996 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:13.996 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "25", 2023-09-13T03:29:13.996 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2023-09-13T03:29:13.996 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "250", 2023-09-13T03:29:13.997 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:13.997 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-09-13T03:29:13.997 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:13.997 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:13.998 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:13.998 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:13.998 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_metrics": { 2023-09-13T03:29:13.999 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_metrics", 2023-09-13T03:29:13.999 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:13.999 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:13.999 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.000 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "pgs,objects,bytes", 2023-09-13T03:29:14.000 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.000 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.000 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.001 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-09-13T03:29:14.001 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-09-13T03:29:14.001 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.001 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.002 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.002 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_step": { 2023-09-13T03:29:14.002 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_step", 2023-09-13T03:29:14.003 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2023-09-13T03:29:14.003 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.003 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.003 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0.5", 2023-09-13T03:29:14.004 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0.001", 2023-09-13T03:29:14.004 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "0.999", 2023-09-13T03:29:14.004 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.004 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "aggressiveness of optimization", 2023-09-13T03:29:14.005 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-09-13T03:29:14.005 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.005 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.005 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.006 INFO:teuthology.orchestra.run.smithi064.stdout: "end_time": { 2023-09-13T03:29:14.006 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "end_time", 2023-09-13T03:29:14.006 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.006 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.007 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.007 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "2359", 2023-09-13T03:29:14.007 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.007 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.008 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.008 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "ending time of day to automatically balance", 2023-09-13T03:29:14.008 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-13T03:29:14.008 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.009 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.009 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.009 INFO:teuthology.orchestra.run.smithi064.stdout: "end_weekday": { 2023-09-13T03:29:14.010 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "end_weekday", 2023-09-13T03:29:14.010 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2023-09-13T03:29:14.010 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.010 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.011 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2023-09-13T03:29:14.011 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0", 2023-09-13T03:29:14.011 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "6", 2023-09-13T03:29:14.012 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.012 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-09-13T03:29:14.012 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-13T03:29:14.012 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.013 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.013 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.013 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:14.014 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:14.014 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.014 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.014 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.015 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:14.015 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.015 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.015 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:14.016 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:14.016 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:14.016 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:14.016 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:14.017 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:14.017 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:14.017 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:14.017 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.018 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.018 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.018 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.019 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.019 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:14.019 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:14.019 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.020 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.020 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.020 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.020 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.021 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.021 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.021 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.021 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.022 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.022 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.022 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.023 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:14.023 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:14.023 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.023 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.024 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.024 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:14.024 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.025 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.025 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:14.025 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:14.025 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:14.026 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:14.026 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:14.026 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:14.026 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:14.027 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:14.027 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.027 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.027 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.028 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.028 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.028 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:14.029 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:14.029 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.029 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.029 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.030 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.030 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.030 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.030 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.031 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.031 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.031 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.032 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.032 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.032 INFO:teuthology.orchestra.run.smithi064.stdout: "min_score": { 2023-09-13T03:29:14.032 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "min_score", 2023-09-13T03:29:14.033 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2023-09-13T03:29:14.033 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.033 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.033 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2023-09-13T03:29:14.034 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.034 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.034 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.035 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-09-13T03:29:14.035 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.035 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.035 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.036 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.036 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": { 2023-09-13T03:29:14.036 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mode", 2023-09-13T03:29:14.036 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.037 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.037 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.037 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "upmap", 2023-09-13T03:29:14.038 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.038 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.038 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:14.038 INFO:teuthology.orchestra.run.smithi064.stdout: "crush-compat", 2023-09-13T03:29:14.039 INFO:teuthology.orchestra.run.smithi064.stdout: "none", 2023-09-13T03:29:14.039 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap" 2023-09-13T03:29:14.039 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:14.039 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Balancer mode", 2023-09-13T03:29:14.040 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.040 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.040 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.040 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.041 INFO:teuthology.orchestra.run.smithi064.stdout: "pool_ids": { 2023-09-13T03:29:14.041 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pool_ids", 2023-09-13T03:29:14.041 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.042 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.042 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.042 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:14.042 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.043 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.043 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.043 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-09-13T03:29:14.044 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.044 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.044 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.044 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.045 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2023-09-13T03:29:14.045 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2023-09-13T03:29:14.045 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:14.045 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.046 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.046 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2023-09-13T03:29:14.046 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.047 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.047 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.047 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-09-13T03:29:14.047 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.048 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.048 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.048 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.048 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap_max_deviation": { 2023-09-13T03:29:14.049 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "upmap_max_deviation", 2023-09-13T03:29:14.049 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:14.049 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.049 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.050 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2023-09-13T03:29:14.050 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2023-09-13T03:29:14.050 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.051 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.051 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "deviation below which no optimization is attempted", 2023-09-13T03:29:14.051 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-09-13T03:29:14.051 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.052 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.052 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.052 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap_max_optimizations": { 2023-09-13T03:29:14.052 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "upmap_max_optimizations", 2023-09-13T03:29:14.053 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2023-09-13T03:29:14.053 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.053 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.053 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2023-09-13T03:29:14.054 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.054 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.054 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.055 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-09-13T03:29:14.055 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.055 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.055 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.056 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:14.056 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:14.056 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.056 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:14.057 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephadm", 2023-09-13T03:29:14.057 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:14.057 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:14.057 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:14.058 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_down_multiplier": { 2023-09-13T03:29:14.058 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_down_multiplier", 2023-09-13T03:29:14.058 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2023-09-13T03:29:14.059 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.059 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.059 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3.0", 2023-09-13T03:29:14.060 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.060 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.060 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.060 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-09-13T03:29:14.061 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.061 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.061 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.061 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.062 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_refresh_rate": { 2023-09-13T03:29:14.062 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_refresh_rate", 2023-09-13T03:29:14.062 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:14.063 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.063 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.063 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "20", 2023-09-13T03:29:14.063 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.064 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.064 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.064 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-09-13T03:29:14.064 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.065 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.065 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.065 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.065 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_starting_port": { 2023-09-13T03:29:14.066 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_starting_port", 2023-09-13T03:29:14.066 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:14.066 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.066 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.067 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "4721", 2023-09-13T03:29:14.067 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.067 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.067 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.068 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-09-13T03:29:14.068 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.068 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.069 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.069 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.069 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_ptrace": { 2023-09-13T03:29:14.069 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_ptrace", 2023-09-13T03:29:14.070 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.070 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.070 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.070 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.071 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.071 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.071 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.071 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-09-13T03:29:14.072 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-09-13T03:29:14.072 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.072 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.072 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.073 INFO:teuthology.orchestra.run.smithi064.stdout: "autotune_interval": { 2023-09-13T03:29:14.073 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "autotune_interval", 2023-09-13T03:29:14.073 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:14.073 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.074 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.074 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2023-09-13T03:29:14.074 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.075 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.075 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.075 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to autotune daemon memory", 2023-09-13T03:29:14.075 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.076 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.076 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.076 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.076 INFO:teuthology.orchestra.run.smithi064.stdout: "autotune_memory_target_ratio": { 2023-09-13T03:29:14.077 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "autotune_memory_target_ratio", 2023-09-13T03:29:14.077 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2023-09-13T03:29:14.077 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.077 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.078 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0.7", 2023-09-13T03:29:14.078 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.078 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.079 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.079 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-09-13T03:29:14.079 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.079 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.080 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.080 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.080 INFO:teuthology.orchestra.run.smithi064.stdout: "cgroups_split": { 2023-09-13T03:29:14.081 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cgroups_split", 2023-09-13T03:29:14.081 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.081 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.081 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.082 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:14.082 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.082 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.082 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.083 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-09-13T03:29:14.083 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.083 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.083 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.084 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.084 INFO:teuthology.orchestra.run.smithi064.stdout: "config_checks_enabled": { 2023-09-13T03:29:14.084 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_checks_enabled", 2023-09-13T03:29:14.085 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.085 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.085 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.085 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.086 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.086 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.086 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.086 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-09-13T03:29:14.087 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.087 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.087 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.087 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.088 INFO:teuthology.orchestra.run.smithi064.stdout: "config_dashboard": { 2023-09-13T03:29:14.088 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_dashboard", 2023-09-13T03:29:14.088 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.088 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.089 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.089 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:14.089 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.089 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.090 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.090 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-09-13T03:29:14.090 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.090 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.091 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.091 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.091 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_alertmanager": { 2023-09-13T03:29:14.091 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_alertmanager", 2023-09-13T03:29:14.092 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.092 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.092 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.092 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-09-13T03:29:14.093 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.093 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.093 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.093 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2023-09-13T03:29:14.094 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.094 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.094 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.095 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.095 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_base": { 2023-09-13T03:29:14.095 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_base", 2023-09-13T03:29:14.095 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.096 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.096 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.096 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/ceph", 2023-09-13T03:29:14.096 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.097 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.097 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.097 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Container image name, without the tag", 2023-09-13T03:29:14.097 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.098 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.098 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.098 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.099 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_elasticsearch": { 2023-09-13T03:29:14.099 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_elasticsearch", 2023-09-13T03:29:14.099 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.099 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.100 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.100 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-09-13T03:29:14.100 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.100 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.101 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.101 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "elasticsearch container image", 2023-09-13T03:29:14.101 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.102 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.102 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.102 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.102 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_grafana": { 2023-09-13T03:29:14.103 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_grafana", 2023-09-13T03:29:14.103 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.103 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.103 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.104 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-09-13T03:29:14.104 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.104 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.105 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.105 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2023-09-13T03:29:14.105 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.105 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.106 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.106 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_haproxy": { 2023-09-13T03:29:14.106 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_haproxy", 2023-09-13T03:29:14.107 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.107 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.107 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.108 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-09-13T03:29:14.108 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.108 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.108 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.109 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "HAproxy container image", 2023-09-13T03:29:14.109 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.109 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.109 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.110 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_agent": { 2023-09-13T03:29:14.110 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_agent", 2023-09-13T03:29:14.110 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.111 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.111 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.111 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-09-13T03:29:14.112 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.112 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.112 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.112 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger agent container image", 2023-09-13T03:29:14.113 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.113 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.113 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.113 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.114 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_collector": { 2023-09-13T03:29:14.114 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_collector", 2023-09-13T03:29:14.114 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.115 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.115 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.115 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-09-13T03:29:14.115 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.116 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.116 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.116 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger collector container image", 2023-09-13T03:29:14.116 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.117 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.117 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.117 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.118 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_query": { 2023-09-13T03:29:14.118 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_query", 2023-09-13T03:29:14.118 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.118 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.119 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.119 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-09-13T03:29:14.119 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.120 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.120 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.120 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger query container image", 2023-09-13T03:29:14.120 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.121 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.121 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.121 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.121 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_keepalived": { 2023-09-13T03:29:14.122 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_keepalived", 2023-09-13T03:29:14.122 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.122 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.123 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.123 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-09-13T03:29:14.123 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.123 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.124 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.124 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Keepalived container image", 2023-09-13T03:29:14.124 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.125 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.125 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.125 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.125 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_loki": { 2023-09-13T03:29:14.126 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_loki", 2023-09-13T03:29:14.126 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.126 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.126 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.127 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-09-13T03:29:14.127 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.127 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.127 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.128 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Loki container image", 2023-09-13T03:29:14.128 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.128 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.129 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.129 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.129 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_node_exporter": { 2023-09-13T03:29:14.129 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_node_exporter", 2023-09-13T03:29:14.130 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.130 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.130 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.130 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-09-13T03:29:14.131 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.131 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.131 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.132 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2023-09-13T03:29:14.132 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.132 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.132 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.133 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_nvmeof": { 2023-09-13T03:29:14.133 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_nvmeof", 2023-09-13T03:29:14.133 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.134 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.134 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.134 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-09-13T03:29:14.135 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.135 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.135 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.135 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Nvme-of container image", 2023-09-13T03:29:14.136 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.136 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.136 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.136 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.137 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_prometheus": { 2023-09-13T03:29:14.137 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_prometheus", 2023-09-13T03:29:14.137 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.138 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.138 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.138 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-09-13T03:29:14.138 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.139 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.139 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.139 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2023-09-13T03:29:14.139 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.140 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.140 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.140 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.141 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_promtail": { 2023-09-13T03:29:14.141 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_promtail", 2023-09-13T03:29:14.141 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.141 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.142 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.142 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-09-13T03:29:14.142 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.142 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.143 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.143 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Promtail container image", 2023-09-13T03:29:14.143 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.144 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.144 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.144 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.144 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_snmp_gateway": { 2023-09-13T03:29:14.145 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_snmp_gateway", 2023-09-13T03:29:14.145 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.145 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.145 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.146 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-09-13T03:29:14.146 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.146 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.147 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.147 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SNMP Gateway container image", 2023-09-13T03:29:14.147 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.147 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.148 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.148 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.148 INFO:teuthology.orchestra.run.smithi064.stdout: "container_init": { 2023-09-13T03:29:14.148 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_init", 2023-09-13T03:29:14.149 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.149 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.149 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.149 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:14.150 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.150 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.150 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.151 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Run podman/docker with `--init`", 2023-09-13T03:29:14.151 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.151 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.151 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.152 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.152 INFO:teuthology.orchestra.run.smithi064.stdout: "daemon_cache_timeout": { 2023-09-13T03:29:14.152 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "daemon_cache_timeout", 2023-09-13T03:29:14.152 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:14.153 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.153 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.153 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2023-09-13T03:29:14.154 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.154 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.154 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.154 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-09-13T03:29:14.155 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.155 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.155 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.156 INFO:teuthology.orchestra.run.smithi064.stdout: "default_cephadm_command_timeout": { 2023-09-13T03:29:14.156 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "default_cephadm_command_timeout", 2023-09-13T03:29:14.156 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:14.157 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.157 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.157 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "900", 2023-09-13T03:29:14.157 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.158 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.158 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.158 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-09-13T03:29:14.158 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.159 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.159 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.159 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.160 INFO:teuthology.orchestra.run.smithi064.stdout: "default_registry": { 2023-09-13T03:29:14.160 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "default_registry", 2023-09-13T03:29:14.160 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.160 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.160 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.161 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io", 2023-09-13T03:29:14.161 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.161 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.162 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.162 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-09-13T03:29:14.162 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.162 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.163 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.163 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.163 INFO:teuthology.orchestra.run.smithi064.stdout: "device_cache_timeout": { 2023-09-13T03:29:14.163 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_cache_timeout", 2023-09-13T03:29:14.164 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:14.164 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.164 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.164 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1800", 2023-09-13T03:29:14.165 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.165 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.165 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.165 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache device inventory", 2023-09-13T03:29:14.166 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.166 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.166 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.166 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.167 INFO:teuthology.orchestra.run.smithi064.stdout: "device_enhanced_scan": { 2023-09-13T03:29:14.167 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_enhanced_scan", 2023-09-13T03:29:14.167 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.168 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.168 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.168 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.168 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.169 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.169 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.169 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use libstoragemgmt during device scans", 2023-09-13T03:29:14.169 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.170 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.170 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.170 INFO:teuthology.orchestra.run.smithi064.stdout: "facts_cache_timeout": { 2023-09-13T03:29:14.171 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "facts_cache_timeout", 2023-09-13T03:29:14.171 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:14.171 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.171 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.172 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2023-09-13T03:29:14.172 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.172 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.172 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.173 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache host facts data", 2023-09-13T03:29:14.173 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.173 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.173 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.174 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.174 INFO:teuthology.orchestra.run.smithi064.stdout: "host_check_interval": { 2023-09-13T03:29:14.174 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "host_check_interval", 2023-09-13T03:29:14.175 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:14.175 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.175 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.175 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2023-09-13T03:29:14.175 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.176 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.176 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.176 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to perform a host check", 2023-09-13T03:29:14.177 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.177 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.177 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.178 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.178 INFO:teuthology.orchestra.run.smithi064.stdout: "inventory_list_all": { 2023-09-13T03:29:14.178 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "inventory_list_all", 2023-09-13T03:29:14.178 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.179 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.179 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.179 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.179 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.180 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.180 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.180 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-09-13T03:29:14.180 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.181 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.181 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.181 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:14.182 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:14.182 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.182 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.183 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.183 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:14.183 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.183 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.184 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:14.184 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:14.184 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:14.185 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:14.185 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:14.185 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:14.185 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:14.186 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:14.186 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.186 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.186 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.187 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.187 INFO:teuthology.orchestra.run.smithi064.stdout: "log_refresh_metadata": { 2023-09-13T03:29:14.188 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_refresh_metadata", 2023-09-13T03:29:14.188 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.188 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.188 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.189 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.189 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.189 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.189 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.190 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-09-13T03:29:14.190 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.190 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.191 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.191 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.191 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:14.191 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:14.192 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.192 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.192 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.192 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:14.193 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.193 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.193 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.194 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-09-13T03:29:14.194 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.194 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.194 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.195 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.195 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:14.195 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:14.195 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.196 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.196 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.196 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:14.196 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.197 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.197 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:14.197 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:14.198 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:14.198 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:14.198 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:14.198 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:14.199 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:14.199 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:14.199 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.199 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.200 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.200 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.200 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.200 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:14.201 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:14.201 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.201 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.201 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.202 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.202 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.202 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.203 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.203 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.203 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.203 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.204 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.204 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.204 INFO:teuthology.orchestra.run.smithi064.stdout: "manage_etc_ceph_ceph_conf": { 2023-09-13T03:29:14.204 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-09-13T03:29:14.205 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.205 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.205 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.205 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.206 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.206 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.206 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.207 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-09-13T03:29:14.207 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.207 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.207 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.208 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.208 INFO:teuthology.orchestra.run.smithi064.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-09-13T03:29:14.208 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-09-13T03:29:14.208 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.209 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.209 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.209 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "*", 2023-09-13T03:29:14.209 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.210 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.210 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.210 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-09-13T03:29:14.211 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.211 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.211 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.211 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.212 INFO:teuthology.orchestra.run.smithi064.stdout: "max_count_per_host": { 2023-09-13T03:29:14.212 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_count_per_host", 2023-09-13T03:29:14.212 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:14.212 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.213 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.213 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2023-09-13T03:29:14.213 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.214 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.214 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.214 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "max number of daemons per service per host", 2023-09-13T03:29:14.214 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.215 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.215 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.215 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.215 INFO:teuthology.orchestra.run.smithi064.stdout: "max_osd_draining_count": { 2023-09-13T03:29:14.216 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_osd_draining_count", 2023-09-13T03:29:14.216 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:14.216 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.217 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.217 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2023-09-13T03:29:14.217 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.217 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.218 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.218 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-09-13T03:29:14.218 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.219 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.219 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.219 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.219 INFO:teuthology.orchestra.run.smithi064.stdout: "migration_current": { 2023-09-13T03:29:14.220 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "migration_current", 2023-09-13T03:29:14.220 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:14.220 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.220 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.221 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:14.221 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.221 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.221 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.222 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "internal - do not modify", 2023-09-13T03:29:14.222 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.222 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.223 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.223 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.223 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": { 2023-09-13T03:29:14.223 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mode", 2023-09-13T03:29:14.224 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.224 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.224 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.225 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "root", 2023-09-13T03:29:14.225 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.225 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.225 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:14.226 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm-package", 2023-09-13T03:29:14.226 INFO:teuthology.orchestra.run.smithi064.stdout: "root" 2023-09-13T03:29:14.226 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:14.227 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "mode for remote execution of cephadm", 2023-09-13T03:29:14.227 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.227 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.227 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.228 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.228 INFO:teuthology.orchestra.run.smithi064.stdout: "prometheus_alerts_path": { 2023-09-13T03:29:14.228 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prometheus_alerts_path", 2023-09-13T03:29:14.228 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.229 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.229 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.229 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-09-13T03:29:14.230 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.230 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.230 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.230 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-09-13T03:29:14.231 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.231 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.231 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.232 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.232 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_insecure": { 2023-09-13T03:29:14.232 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_insecure", 2023-09-13T03:29:14.232 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.233 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.233 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.233 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.233 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.234 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.234 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.234 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-09-13T03:29:14.235 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.235 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.235 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.235 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.236 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_password": { 2023-09-13T03:29:14.236 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_password", 2023-09-13T03:29:14.236 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.236 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.237 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.237 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:14.237 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.237 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.238 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.238 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-09-13T03:29:14.238 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.239 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.239 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.239 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.239 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_url": { 2023-09-13T03:29:14.240 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_url", 2023-09-13T03:29:14.240 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.240 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.240 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.241 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:14.241 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.241 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.241 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.242 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-09-13T03:29:14.242 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.242 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.243 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.243 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.243 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_username": { 2023-09-13T03:29:14.243 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_username", 2023-09-13T03:29:14.244 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.244 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.244 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.245 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:14.245 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.245 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.245 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.246 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-09-13T03:29:14.246 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.246 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.246 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.247 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.247 INFO:teuthology.orchestra.run.smithi064.stdout: "secure_monitoring_stack": { 2023-09-13T03:29:14.247 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "secure_monitoring_stack", 2023-09-13T03:29:14.248 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.248 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.248 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.249 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.249 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.249 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.250 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.250 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-09-13T03:29:14.250 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.251 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.251 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.251 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.251 INFO:teuthology.orchestra.run.smithi064.stdout: "service_discovery_port": { 2023-09-13T03:29:14.252 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "service_discovery_port", 2023-09-13T03:29:14.252 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:14.252 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.253 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.253 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8765", 2023-09-13T03:29:14.253 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.253 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.254 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.254 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "cephadm service discovery port", 2023-09-13T03:29:14.254 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.255 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.255 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.255 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.256 INFO:teuthology.orchestra.run.smithi064.stdout: "ssh_config_file": { 2023-09-13T03:29:14.256 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssh_config_file", 2023-09-13T03:29:14.256 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.257 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.257 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.257 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:14.257 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.258 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.258 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.258 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-09-13T03:29:14.259 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.259 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.259 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.260 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.260 INFO:teuthology.orchestra.run.smithi064.stdout: "use_agent": { 2023-09-13T03:29:14.260 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "use_agent", 2023-09-13T03:29:14.260 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.261 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.261 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.261 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.262 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.262 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.262 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.263 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-09-13T03:29:14.263 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.263 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.263 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.264 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.264 INFO:teuthology.orchestra.run.smithi064.stdout: "use_repo_digest": { 2023-09-13T03:29:14.264 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "use_repo_digest", 2023-09-13T03:29:14.265 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.265 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.265 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.266 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:14.266 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.266 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.267 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.267 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-09-13T03:29:14.267 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.267 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.268 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.268 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.268 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_failed_host_check": { 2023-09-13T03:29:14.269 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_failed_host_check", 2023-09-13T03:29:14.269 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.269 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.269 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.270 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:14.270 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.270 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.270 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.271 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if the host check fails", 2023-09-13T03:29:14.271 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.271 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.271 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.272 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.272 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_stray_daemons": { 2023-09-13T03:29:14.272 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_stray_daemons", 2023-09-13T03:29:14.272 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.273 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.273 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.273 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:14.273 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.274 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.274 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.274 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-09-13T03:29:14.275 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.275 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.275 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.275 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.276 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_stray_hosts": { 2023-09-13T03:29:14.276 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_stray_hosts", 2023-09-13T03:29:14.276 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.276 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.277 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.277 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:14.277 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.277 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.278 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.278 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-09-13T03:29:14.278 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.279 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.279 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.279 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:14.279 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:14.280 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.280 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:14.280 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crash", 2023-09-13T03:29:14.280 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:14.281 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:14.281 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:14.281 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:14.281 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:14.282 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.282 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.282 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.283 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:14.283 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.283 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.283 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:14.284 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:14.284 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:14.284 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:14.284 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:14.285 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:14.285 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:14.285 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:14.286 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.286 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.286 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.286 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.287 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.287 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:14.287 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:14.287 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.288 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.288 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.288 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.289 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.289 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.289 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.289 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.290 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.290 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.290 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.290 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.291 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:14.291 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:14.291 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.292 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.292 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.292 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:14.292 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.293 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.293 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:14.293 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:14.293 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:14.294 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:14.294 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:14.294 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:14.294 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:14.295 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:14.295 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.295 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.295 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.296 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.296 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.296 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:14.296 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:14.297 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.297 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.297 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.297 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.298 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.298 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.298 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.299 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.299 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.299 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.299 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.300 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.300 INFO:teuthology.orchestra.run.smithi064.stdout: "retain_interval": { 2023-09-13T03:29:14.300 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "retain_interval", 2023-09-13T03:29:14.300 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:14.301 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.301 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.301 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "31536000", 2023-09-13T03:29:14.301 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.302 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.302 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.302 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long to retain crashes before pruning them", 2023-09-13T03:29:14.303 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.303 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.303 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.304 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_recent_interval": { 2023-09-13T03:29:14.304 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_recent_interval", 2023-09-13T03:29:14.304 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:14.304 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.305 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.305 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1209600", 2023-09-13T03:29:14.305 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.305 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.306 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.306 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "time interval in which to warn about recent crashes", 2023-09-13T03:29:14.306 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.306 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.307 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.307 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:14.307 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:14.308 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.308 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:14.308 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "dashboard", 2023-09-13T03:29:14.308 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:14.309 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:14.309 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:14.309 INFO:teuthology.orchestra.run.smithi064.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-09-13T03:29:14.309 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-09-13T03:29:14.310 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:14.310 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.310 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.310 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2023-09-13T03:29:14.311 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.311 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.311 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.312 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.312 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.312 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.312 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.313 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.313 INFO:teuthology.orchestra.run.smithi064.stdout: "ALERTMANAGER_API_HOST": { 2023-09-13T03:29:14.313 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ALERTMANAGER_API_HOST", 2023-09-13T03:29:14.313 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.314 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.314 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.314 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:14.314 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.315 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.315 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.315 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.316 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.316 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.316 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.316 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.317 INFO:teuthology.orchestra.run.smithi064.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-09-13T03:29:14.317 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-09-13T03:29:14.317 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.317 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.318 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.318 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:14.318 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.319 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.319 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.319 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.320 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.320 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.320 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.320 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.321 INFO:teuthology.orchestra.run.smithi064.stdout: "AUDIT_API_ENABLED": { 2023-09-13T03:29:14.321 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "AUDIT_API_ENABLED", 2023-09-13T03:29:14.321 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.321 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.322 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.322 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.322 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.323 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.323 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.323 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.323 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.324 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.324 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.324 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.325 INFO:teuthology.orchestra.run.smithi064.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-09-13T03:29:14.325 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-09-13T03:29:14.325 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.325 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.326 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.326 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:14.326 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.327 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.327 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.327 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.327 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.328 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.328 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.328 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.329 INFO:teuthology.orchestra.run.smithi064.stdout: "ENABLE_BROWSABLE_API": { 2023-09-13T03:29:14.329 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ENABLE_BROWSABLE_API", 2023-09-13T03:29:14.329 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.329 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.330 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.330 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:14.330 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.330 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.331 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.331 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.331 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.332 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.332 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.332 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.332 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-09-13T03:29:14.333 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-09-13T03:29:14.333 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.333 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.333 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.334 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:14.334 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.334 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.335 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.335 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.335 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.335 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.336 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.336 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.336 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-09-13T03:29:14.336 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-09-13T03:29:14.337 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.337 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.337 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.337 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:14.338 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.338 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.338 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.339 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.339 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.339 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.339 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.340 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-09-13T03:29:14.340 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-09-13T03:29:14.340 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.341 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.341 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.341 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:14.342 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.342 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.342 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.342 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.343 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.343 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.343 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.344 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.344 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-09-13T03:29:14.344 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-09-13T03:29:14.344 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.345 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.345 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.345 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:14.345 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.346 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.346 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.346 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.346 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.347 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.347 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.347 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.347 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_NFS": { 2023-09-13T03:29:14.348 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-09-13T03:29:14.348 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.348 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.349 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.349 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:14.349 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.349 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.350 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.353 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.353 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.353 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.354 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.354 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.354 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_RBD": { 2023-09-13T03:29:14.355 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-09-13T03:29:14.355 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.355 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.355 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.356 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:14.356 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.356 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.357 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.357 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.357 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.357 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.358 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.358 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.358 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_RGW": { 2023-09-13T03:29:14.359 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-09-13T03:29:14.359 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.359 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.359 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.360 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:14.360 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.360 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.360 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.361 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.361 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.361 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.362 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.362 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.362 INFO:teuthology.orchestra.run.smithi064.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-09-13T03:29:14.362 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-09-13T03:29:14.363 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.363 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.363 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.364 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:14.364 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.364 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.364 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.365 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.365 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.365 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.365 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.366 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.366 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_PASSWORD": { 2023-09-13T03:29:14.366 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_PASSWORD", 2023-09-13T03:29:14.367 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.367 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.367 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.367 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2023-09-13T03:29:14.368 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.368 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.368 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.368 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.369 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.369 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.369 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.370 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.370 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-09-13T03:29:14.370 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-09-13T03:29:14.370 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.371 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.371 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.371 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:14.371 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.372 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.372 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.372 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.372 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.373 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.373 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.373 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.374 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_URL": { 2023-09-13T03:29:14.374 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_URL", 2023-09-13T03:29:14.374 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.374 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.375 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.375 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:14.375 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.376 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.376 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.376 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.376 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.377 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.377 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.377 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.377 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_USERNAME": { 2023-09-13T03:29:14.378 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_USERNAME", 2023-09-13T03:29:14.378 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.378 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.379 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.379 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2023-09-13T03:29:14.379 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.379 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.380 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.380 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.380 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.381 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.381 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.381 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.381 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-09-13T03:29:14.382 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-09-13T03:29:14.382 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.382 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.383 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.383 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:14.383 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.383 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.384 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.384 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.384 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.384 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.385 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.385 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.385 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-09-13T03:29:14.386 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-09-13T03:29:14.386 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.386 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.386 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.387 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.387 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.387 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.387 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.388 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.388 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.388 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.388 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.389 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.389 INFO:teuthology.orchestra.run.smithi064.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-09-13T03:29:14.389 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-09-13T03:29:14.389 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.390 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.390 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.390 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:14.390 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.391 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.391 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.391 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.391 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.392 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.392 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.392 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.392 INFO:teuthology.orchestra.run.smithi064.stdout: "ISSUE_TRACKER_API_KEY": { 2023-09-13T03:29:14.393 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-09-13T03:29:14.393 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.393 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.394 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.394 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:14.394 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.394 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.395 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.395 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.395 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.395 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.396 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.396 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.396 INFO:teuthology.orchestra.run.smithi064.stdout: "PROMETHEUS_API_HOST": { 2023-09-13T03:29:14.397 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PROMETHEUS_API_HOST", 2023-09-13T03:29:14.397 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.397 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.397 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.398 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:14.398 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.398 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.399 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.399 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.399 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.399 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.400 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.400 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.400 INFO:teuthology.orchestra.run.smithi064.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-09-13T03:29:14.400 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-09-13T03:29:14.401 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.401 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.401 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.401 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:14.402 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.402 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.402 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.403 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.403 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.403 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.403 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.404 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.404 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-09-13T03:29:14.404 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-09-13T03:29:14.405 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.405 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.405 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.405 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.406 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.406 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.406 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.406 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.407 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.407 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.407 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.408 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.408 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-09-13T03:29:14.408 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-09-13T03:29:14.408 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.409 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.409 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.409 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.409 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.410 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.410 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.410 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.411 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.411 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.411 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.411 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.412 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-09-13T03:29:14.412 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-09-13T03:29:14.412 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.413 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.413 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.413 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:14.413 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.414 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.414 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.414 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.415 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.415 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.415 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.415 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.416 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-09-13T03:29:14.416 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-09-13T03:29:14.416 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.416 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.417 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.417 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:14.417 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.417 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.418 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.418 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.418 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.419 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.419 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.419 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.419 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-09-13T03:29:14.420 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-09-13T03:29:14.420 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.420 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.420 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.421 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.421 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.421 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.421 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.422 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.422 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.422 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.423 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.423 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.423 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-09-13T03:29:14.423 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-09-13T03:29:14.424 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.424 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.424 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.425 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.425 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.425 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.425 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.426 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.426 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.426 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.426 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.427 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.427 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-09-13T03:29:14.427 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-09-13T03:29:14.428 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.428 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.428 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.428 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.429 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.429 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.429 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.430 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.430 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.430 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.430 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.431 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.431 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_ENABLED": { 2023-09-13T03:29:14.431 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_ENABLED", 2023-09-13T03:29:14.432 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.432 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.432 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.432 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:14.433 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.433 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.433 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.434 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.434 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.434 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.434 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.435 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.435 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-09-13T03:29:14.435 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-09-13T03:29:14.435 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.436 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.436 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.436 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-09-13T03:29:14.437 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.437 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.437 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.437 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.438 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.438 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.438 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.438 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.439 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-09-13T03:29:14.439 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-09-13T03:29:14.439 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:14.440 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.440 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.440 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2023-09-13T03:29:14.440 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.441 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.441 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.441 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.441 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.442 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.442 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.442 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.443 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-09-13T03:29:14.443 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-09-13T03:29:14.443 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:14.443 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.444 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.444 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8", 2023-09-13T03:29:14.444 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.445 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.445 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.445 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.445 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.446 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.446 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.446 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.447 INFO:teuthology.orchestra.run.smithi064.stdout: "REST_REQUESTS_TIMEOUT": { 2023-09-13T03:29:14.447 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-09-13T03:29:14.447 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:14.447 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.448 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.448 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "45", 2023-09-13T03:29:14.448 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.448 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.449 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.449 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.449 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.450 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.450 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.450 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.450 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_ACCESS_KEY": { 2023-09-13T03:29:14.451 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_ACCESS_KEY", 2023-09-13T03:29:14.451 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.451 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.452 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.452 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:14.452 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.452 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.453 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.453 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.453 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.453 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.454 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.454 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.454 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-09-13T03:29:14.455 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-09-13T03:29:14.455 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.455 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.455 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.456 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2023-09-13T03:29:14.456 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.456 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.456 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.457 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.457 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.457 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.458 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.458 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.458 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_SECRET_KEY": { 2023-09-13T03:29:14.458 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_SECRET_KEY", 2023-09-13T03:29:14.459 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.459 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.459 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.460 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:14.460 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.460 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.460 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.461 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.461 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.461 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.461 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.462 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.462 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_SSL_VERIFY": { 2023-09-13T03:29:14.462 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_SSL_VERIFY", 2023-09-13T03:29:14.463 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.463 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.463 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.464 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:14.464 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.464 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.464 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.465 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.465 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.465 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.465 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.466 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.466 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-09-13T03:29:14.466 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-09-13T03:29:14.467 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:14.467 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.467 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.468 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2023-09-13T03:29:14.468 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.468 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.468 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.469 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.469 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.469 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.469 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.470 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.470 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-09-13T03:29:14.470 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-09-13T03:29:14.471 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:14.471 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.471 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.471 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2023-09-13T03:29:14.472 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.472 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.472 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.473 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.473 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.473 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.473 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.474 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.474 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-09-13T03:29:14.474 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-09-13T03:29:14.474 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:14.475 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.475 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.475 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2023-09-13T03:29:14.476 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.476 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.476 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.476 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.477 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.477 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.477 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.477 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.478 INFO:teuthology.orchestra.run.smithi064.stdout: "cross_origin_url": { 2023-09-13T03:29:14.478 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cross_origin_url", 2023-09-13T03:29:14.478 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.479 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.479 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.479 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:14.480 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.480 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.480 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.480 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.481 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.481 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.481 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.483 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.483 INFO:teuthology.orchestra.run.smithi064.stdout: "crt_file": { 2023-09-13T03:29:14.483 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crt_file", 2023-09-13T03:29:14.484 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.484 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.484 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.485 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:14.485 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.485 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.485 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.486 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.487 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.487 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.487 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.487 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.488 INFO:teuthology.orchestra.run.smithi064.stdout: "debug": { 2023-09-13T03:29:14.488 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "debug", 2023-09-13T03:29:14.488 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.488 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.489 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.489 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.489 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.490 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.490 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.490 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable/disable debug options", 2023-09-13T03:29:14.490 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.491 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.491 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.491 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.492 INFO:teuthology.orchestra.run.smithi064.stdout: "jwt_token_ttl": { 2023-09-13T03:29:14.492 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "jwt_token_ttl", 2023-09-13T03:29:14.492 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:14.492 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.493 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.493 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "28800", 2023-09-13T03:29:14.493 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.493 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.494 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.494 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.494 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.495 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.495 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.495 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.495 INFO:teuthology.orchestra.run.smithi064.stdout: "key_file": { 2023-09-13T03:29:14.496 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "key_file", 2023-09-13T03:29:14.496 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.496 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.496 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.497 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:14.497 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.497 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.498 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.498 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.498 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.498 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.499 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.499 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.499 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:14.500 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:14.500 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.500 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.500 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.501 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:14.501 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.501 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.501 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:14.502 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:14.502 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:14.502 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:14.503 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:14.503 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:14.503 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:14.503 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:14.504 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.504 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.504 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.504 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.505 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.505 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:14.505 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:14.506 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.506 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.506 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.506 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.507 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.507 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.507 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.507 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.508 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.508 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.508 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.508 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.509 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:14.509 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:14.509 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.510 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.510 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.510 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:14.510 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.511 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.511 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:14.511 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:14.511 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:14.512 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:14.512 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:14.512 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:14.513 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:14.513 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:14.513 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.513 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.514 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.514 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.514 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.515 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:14.515 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:14.515 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.515 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.516 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.516 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.516 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.516 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.517 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.517 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.517 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.518 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.518 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.518 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.519 INFO:teuthology.orchestra.run.smithi064.stdout: "motd": { 2023-09-13T03:29:14.519 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "motd", 2023-09-13T03:29:14.519 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.519 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.520 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.520 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:14.520 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.520 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.521 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.521 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "The message of the day", 2023-09-13T03:29:14.521 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.522 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.522 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.522 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.522 INFO:teuthology.orchestra.run.smithi064.stdout: "redirect_resolve_ip_addr": { 2023-09-13T03:29:14.523 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "redirect_resolve_ip_addr", 2023-09-13T03:29:14.523 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.523 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.524 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.524 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.524 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.524 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.525 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.525 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.525 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.525 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.526 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.526 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.526 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2023-09-13T03:29:14.527 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2023-09-13T03:29:14.527 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.527 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.527 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.528 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "::", 2023-09-13T03:29:14.528 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.528 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.528 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.529 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.529 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.529 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.530 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.530 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.530 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2023-09-13T03:29:14.530 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2023-09-13T03:29:14.531 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:14.531 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.531 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.532 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8080", 2023-09-13T03:29:14.532 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.532 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.532 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.533 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.533 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.533 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.533 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.534 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.534 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl": { 2023-09-13T03:29:14.534 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl", 2023-09-13T03:29:14.535 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.535 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.535 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.535 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:14.536 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.536 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.536 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.537 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.537 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.537 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.537 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.538 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.538 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl_server_port": { 2023-09-13T03:29:14.538 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl_server_port", 2023-09-13T03:29:14.538 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:14.539 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.539 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.539 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8443", 2023-09-13T03:29:14.539 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.540 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.540 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.540 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.541 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.541 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.541 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.541 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.542 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_behaviour": { 2023-09-13T03:29:14.542 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_behaviour", 2023-09-13T03:29:14.542 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.543 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.543 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.543 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "redirect", 2023-09-13T03:29:14.543 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.544 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.544 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:14.544 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:14.544 INFO:teuthology.orchestra.run.smithi064.stdout: "redirect" 2023-09-13T03:29:14.545 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:14.545 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.545 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.546 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.546 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.546 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.546 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_error_status_code": { 2023-09-13T03:29:14.547 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_error_status_code", 2023-09-13T03:29:14.547 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:14.547 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.547 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.548 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "500", 2023-09-13T03:29:14.548 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "400", 2023-09-13T03:29:14.548 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "599", 2023-09-13T03:29:14.549 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.549 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.549 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.549 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.550 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.550 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.550 INFO:teuthology.orchestra.run.smithi064.stdout: "url_prefix": { 2023-09-13T03:29:14.551 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "url_prefix", 2023-09-13T03:29:14.551 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.551 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.551 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.552 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:14.552 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.552 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.552 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.553 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.553 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.553 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.553 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.554 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:14.554 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:14.554 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.555 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:14.555 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "devicehealth", 2023-09-13T03:29:14.555 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:14.555 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:14.556 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:14.556 INFO:teuthology.orchestra.run.smithi064.stdout: "enable_monitoring": { 2023-09-13T03:29:14.556 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enable_monitoring", 2023-09-13T03:29:14.556 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.557 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.557 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.557 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:14.558 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.558 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.558 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.559 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "monitor device health metrics", 2023-09-13T03:29:14.559 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.559 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.559 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.560 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.560 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:14.560 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:14.560 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.561 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.561 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.561 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:14.561 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.562 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.562 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:14.562 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:14.563 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:14.563 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:14.563 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:14.563 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:14.564 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:14.564 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:14.564 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.565 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.565 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.565 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.565 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.566 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:14.566 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:14.566 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.566 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.567 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.567 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.567 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.568 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.568 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.568 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.568 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.569 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.569 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.569 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.570 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:14.570 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:14.570 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.570 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.571 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.571 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:14.571 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.571 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.572 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:14.572 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:14.572 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:14.572 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:14.573 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:14.573 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:14.573 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:14.574 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:14.574 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.574 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.574 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.575 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.575 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.575 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:14.575 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:14.576 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.576 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.576 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.577 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.577 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.577 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.577 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.578 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.578 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.578 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.578 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.579 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.579 INFO:teuthology.orchestra.run.smithi064.stdout: "mark_out_threshold": { 2023-09-13T03:29:14.579 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mark_out_threshold", 2023-09-13T03:29:14.580 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:14.580 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.580 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.580 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "2419200", 2023-09-13T03:29:14.581 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.581 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.581 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.581 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-09-13T03:29:14.582 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.582 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.582 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.583 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.583 INFO:teuthology.orchestra.run.smithi064.stdout: "pool_name": { 2023-09-13T03:29:14.583 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pool_name", 2023-09-13T03:29:14.583 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.584 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.584 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.584 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "device_health_metrics", 2023-09-13T03:29:14.585 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.585 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.585 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.585 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "name of pool in which to store device health metrics", 2023-09-13T03:29:14.586 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.586 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.586 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.586 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.587 INFO:teuthology.orchestra.run.smithi064.stdout: "retention_period": { 2023-09-13T03:29:14.587 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "retention_period", 2023-09-13T03:29:14.587 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:14.588 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.588 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.588 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15552000", 2023-09-13T03:29:14.588 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.589 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.589 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.589 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long to retain device health metrics", 2023-09-13T03:29:14.589 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.590 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.590 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.590 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.591 INFO:teuthology.orchestra.run.smithi064.stdout: "scrape_frequency": { 2023-09-13T03:29:14.591 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "scrape_frequency", 2023-09-13T03:29:14.591 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:14.591 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.592 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.592 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "86400", 2023-09-13T03:29:14.592 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.592 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.593 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.593 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to scrape device health metrics", 2023-09-13T03:29:14.593 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.594 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.594 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.594 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.594 INFO:teuthology.orchestra.run.smithi064.stdout: "self_heal": { 2023-09-13T03:29:14.595 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "self_heal", 2023-09-13T03:29:14.595 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.595 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.596 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.596 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:14.596 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.596 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.597 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.597 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-09-13T03:29:14.597 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.597 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.598 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.598 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.598 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2023-09-13T03:29:14.599 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2023-09-13T03:29:14.599 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:14.599 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.599 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.600 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2023-09-13T03:29:14.600 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.600 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.601 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.601 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to wake up and check device health", 2023-09-13T03:29:14.601 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.601 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.602 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.602 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.602 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_threshold": { 2023-09-13T03:29:14.603 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_threshold", 2023-09-13T03:29:14.603 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:14.603 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.603 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.604 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "7257600", 2023-09-13T03:29:14.604 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.604 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.605 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.605 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-09-13T03:29:14.605 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.605 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.606 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.606 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:14.606 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:14.606 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.607 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:14.607 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "diskprediction_local", 2023-09-13T03:29:14.607 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:14.608 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:14.608 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:14.608 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:14.608 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:14.609 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.609 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.609 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.609 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:14.610 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.610 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.610 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:14.611 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:14.611 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:14.611 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:14.611 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:14.612 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:14.612 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:14.612 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:14.613 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.613 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.613 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.613 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.614 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.614 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:14.614 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:14.615 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.615 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.615 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.615 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.616 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.616 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.616 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.616 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.617 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.617 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.617 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.618 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.618 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:14.618 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:14.618 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.619 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.619 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.619 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:14.620 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.620 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.620 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:14.620 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:14.621 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:14.621 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:14.621 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:14.622 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:14.622 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:14.622 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:14.622 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.623 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.623 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.623 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.623 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.624 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:14.624 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:14.624 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.625 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.625 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.625 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.625 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.626 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.626 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.626 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.627 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.627 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.627 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.627 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.628 INFO:teuthology.orchestra.run.smithi064.stdout: "predict_interval": { 2023-09-13T03:29:14.628 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "predict_interval", 2023-09-13T03:29:14.628 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.629 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.629 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.629 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "86400", 2023-09-13T03:29:14.629 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.630 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.630 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.630 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.631 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.631 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.631 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.631 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.632 INFO:teuthology.orchestra.run.smithi064.stdout: "predictor_model": { 2023-09-13T03:29:14.632 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "predictor_model", 2023-09-13T03:29:14.632 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.633 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.633 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.633 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "prophetstor", 2023-09-13T03:29:14.633 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.634 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.634 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.634 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.634 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.635 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.635 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.635 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.636 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2023-09-13T03:29:14.636 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2023-09-13T03:29:14.636 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.636 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.637 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.637 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2023-09-13T03:29:14.637 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.638 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.638 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.638 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.638 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.639 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.639 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.639 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:14.640 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:14.640 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.640 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:14.640 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "influx", 2023-09-13T03:29:14.641 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": false, 2023-09-13T03:29:14.641 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "influxdb python module not found", 2023-09-13T03:29:14.641 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:14.641 INFO:teuthology.orchestra.run.smithi064.stdout: "batch_size": { 2023-09-13T03:29:14.642 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "batch_size", 2023-09-13T03:29:14.642 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:14.642 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.643 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.643 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5000", 2023-09-13T03:29:14.643 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.643 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.644 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.644 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-09-13T03:29:14.644 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.645 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.645 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.645 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.645 INFO:teuthology.orchestra.run.smithi064.stdout: "database": { 2023-09-13T03:29:14.646 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "database", 2023-09-13T03:29:14.646 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.646 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.647 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.647 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "ceph", 2023-09-13T03:29:14.647 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.648 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.648 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.648 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-09-13T03:29:14.648 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.649 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.649 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.649 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.649 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": { 2023-09-13T03:29:14.650 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "hostname", 2023-09-13T03:29:14.650 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.650 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.651 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.651 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:14.651 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.652 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.652 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.652 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB server hostname", 2023-09-13T03:29:14.652 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.653 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.653 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.653 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.654 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2023-09-13T03:29:14.654 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2023-09-13T03:29:14.654 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:14.655 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.655 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.655 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "30", 2023-09-13T03:29:14.655 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "5", 2023-09-13T03:29:14.656 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.656 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.656 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-09-13T03:29:14.656 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.657 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.657 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.657 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.658 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:14.658 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:14.658 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.658 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.659 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.659 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:14.659 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.659 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.660 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:14.660 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:14.660 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:14.661 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:14.661 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:14.661 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:14.661 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:14.662 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:14.662 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.662 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.662 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.663 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.663 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.663 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:14.664 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:14.664 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.664 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.664 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.665 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.665 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.665 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.665 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.666 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.666 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.666 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.667 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.667 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.667 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:14.668 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:14.668 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.668 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.669 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.669 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:14.669 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.669 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.670 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:14.670 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:14.670 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:14.671 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:14.671 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:14.671 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:14.671 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:14.672 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:14.672 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.672 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.672 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.673 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.673 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.673 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:14.673 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:14.674 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.674 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.674 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.675 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.675 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.675 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.676 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.676 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.676 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.676 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.677 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.677 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.677 INFO:teuthology.orchestra.run.smithi064.stdout: "password": { 2023-09-13T03:29:14.677 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "password", 2023-09-13T03:29:14.678 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.678 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.678 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.679 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:14.679 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.679 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.679 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.680 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "password of InfluxDB server user", 2023-09-13T03:29:14.680 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.680 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.680 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.681 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.681 INFO:teuthology.orchestra.run.smithi064.stdout: "port": { 2023-09-13T03:29:14.681 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "port", 2023-09-13T03:29:14.682 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:14.682 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.682 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.682 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8086", 2023-09-13T03:29:14.683 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.683 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.683 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.683 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB server port", 2023-09-13T03:29:14.684 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.684 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.684 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.685 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.685 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl": { 2023-09-13T03:29:14.685 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl", 2023-09-13T03:29:14.685 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.686 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.686 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.686 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "false", 2023-09-13T03:29:14.686 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.687 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.687 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.687 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-09-13T03:29:14.688 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.688 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.688 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.688 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.689 INFO:teuthology.orchestra.run.smithi064.stdout: "threads": { 2023-09-13T03:29:14.689 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "threads", 2023-09-13T03:29:14.689 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:14.690 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.690 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.690 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2023-09-13T03:29:14.690 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2023-09-13T03:29:14.691 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "32", 2023-09-13T03:29:14.691 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.691 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-09-13T03:29:14.692 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.692 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.692 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.692 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.693 INFO:teuthology.orchestra.run.smithi064.stdout: "username": { 2023-09-13T03:29:14.693 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "username", 2023-09-13T03:29:14.693 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.694 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.694 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.694 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:14.694 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.695 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.695 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.695 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "username of InfluxDB server user", 2023-09-13T03:29:14.695 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.696 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.696 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.696 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.697 INFO:teuthology.orchestra.run.smithi064.stdout: "verify_ssl": { 2023-09-13T03:29:14.697 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "verify_ssl", 2023-09-13T03:29:14.697 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.697 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.698 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.698 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "true", 2023-09-13T03:29:14.698 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.699 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.699 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.699 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-09-13T03:29:14.699 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.700 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.700 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.700 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:14.701 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:14.701 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.701 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:14.701 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "insights", 2023-09-13T03:29:14.702 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:14.702 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:14.702 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:14.703 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:14.703 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:14.703 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.703 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.704 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.704 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:14.704 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.705 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.705 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:14.705 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:14.706 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:14.706 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:14.706 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:14.706 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:14.707 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:14.707 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:14.707 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.708 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.708 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.708 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.709 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.709 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:14.709 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:14.710 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.710 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.710 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.711 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.711 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.711 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.711 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.712 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.712 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.712 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.712 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.713 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.713 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:14.713 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:14.714 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.714 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.714 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.715 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:14.715 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.715 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.715 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:14.716 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:14.716 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:14.716 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:14.717 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:14.717 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:14.717 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:14.717 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:14.718 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.718 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.718 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.719 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.719 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.719 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:14.719 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:14.720 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.720 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.720 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.721 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.721 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.721 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.721 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.722 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.722 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.722 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.723 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.723 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:14.723 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:14.723 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.724 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:14.724 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "iostat", 2023-09-13T03:29:14.724 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:14.725 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:14.725 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:14.725 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:14.726 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:14.726 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.726 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.726 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.727 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:14.727 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.727 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.727 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:14.728 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:14.728 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:14.728 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:14.729 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:14.729 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:14.729 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:14.730 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:14.730 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.730 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.730 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.731 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.731 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:14.731 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:14.732 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.732 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.732 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.733 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.733 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.733 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.733 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.734 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.734 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.734 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.735 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.735 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.735 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:14.735 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:14.736 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.736 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.736 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.736 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:14.737 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.737 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.737 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:14.738 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:14.738 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:14.738 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:14.738 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:14.739 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:14.739 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:14.739 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:14.739 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.740 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.740 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.740 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.741 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.741 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:14.741 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:14.742 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.742 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.742 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.742 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.743 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.743 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.743 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.744 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.744 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.744 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.745 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.745 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:14.745 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:14.745 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.746 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:14.746 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "k8sevents", 2023-09-13T03:29:14.746 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:14.747 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:14.747 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:14.747 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_event_retention_days": { 2023-09-13T03:29:14.747 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ceph_event_retention_days", 2023-09-13T03:29:14.748 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:14.748 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.748 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.749 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "7", 2023-09-13T03:29:14.749 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.749 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.749 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.750 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Days to hold ceph event information within local cache", 2023-09-13T03:29:14.750 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.750 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.751 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.751 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.751 INFO:teuthology.orchestra.run.smithi064.stdout: "config_check_secs": { 2023-09-13T03:29:14.751 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_check_secs", 2023-09-13T03:29:14.752 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:14.752 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.752 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.753 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2023-09-13T03:29:14.753 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "10", 2023-09-13T03:29:14.753 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.753 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.754 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-09-13T03:29:14.754 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.754 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.755 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.755 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.755 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:14.755 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:14.756 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.756 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.756 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.757 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:14.757 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.757 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.757 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:14.758 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:14.758 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:14.758 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:14.759 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:14.759 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:14.759 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:14.759 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:14.760 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.760 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.760 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.761 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.761 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.761 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:14.761 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:14.762 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.765 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.765 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.766 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.766 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.766 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.766 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.767 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.767 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.767 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.768 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.768 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.768 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:14.768 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:14.769 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.769 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.769 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.770 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:14.770 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.770 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.770 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:14.771 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:14.771 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:14.771 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:14.771 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:14.772 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:14.772 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:14.772 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:14.773 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.773 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.773 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.773 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.774 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.774 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:14.774 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:14.775 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.775 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.775 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.775 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.776 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.776 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.776 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.776 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.777 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.777 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.777 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.777 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:14.778 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:14.778 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.778 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:14.779 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "localpool", 2023-09-13T03:29:14.779 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:14.779 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:14.779 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:14.780 INFO:teuthology.orchestra.run.smithi064.stdout: "failure_domain": { 2023-09-13T03:29:14.780 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "failure_domain", 2023-09-13T03:29:14.780 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.781 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.781 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.781 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "host", 2023-09-13T03:29:14.781 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.782 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.782 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.782 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "failure domain for any created local pool", 2023-09-13T03:29:14.783 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-09-13T03:29:14.783 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.783 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.784 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.784 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:14.784 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:14.784 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.785 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.785 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.785 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:14.786 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.786 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.786 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:14.786 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:14.787 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:14.787 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:14.787 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:14.788 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:14.788 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:14.788 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:14.788 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.789 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.789 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.789 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.790 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.790 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:14.790 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:14.791 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.791 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.791 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.791 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.792 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.792 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.792 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.792 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.793 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.793 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.793 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.794 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.794 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:14.794 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:14.795 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.795 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.795 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.795 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:14.796 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.796 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.796 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:14.796 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:14.797 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:14.797 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:14.797 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:14.798 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:14.798 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:14.798 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:14.799 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.799 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.799 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.799 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.800 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.800 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:14.800 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:14.801 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.801 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.801 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.801 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.802 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.802 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.802 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.802 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.803 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.803 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.803 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.804 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.804 INFO:teuthology.orchestra.run.smithi064.stdout: "min_size": { 2023-09-13T03:29:14.804 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "min_size", 2023-09-13T03:29:14.804 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:14.805 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.805 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.805 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:14.806 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.806 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.806 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.806 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default min_size for any created local pool", 2023-09-13T03:29:14.807 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-09-13T03:29:14.807 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.807 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.808 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.808 INFO:teuthology.orchestra.run.smithi064.stdout: "num_rep": { 2023-09-13T03:29:14.808 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "num_rep", 2023-09-13T03:29:14.808 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:14.809 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.809 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.809 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3", 2023-09-13T03:29:14.810 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.810 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.810 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.810 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default replica count for any created local pool", 2023-09-13T03:29:14.811 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.811 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.811 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.812 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.812 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_num": { 2023-09-13T03:29:14.812 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pg_num", 2023-09-13T03:29:14.812 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:14.813 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.813 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.813 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "128", 2023-09-13T03:29:14.814 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.814 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.814 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.814 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default pg_num for any created local pool", 2023-09-13T03:29:14.815 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.815 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.815 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.816 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.816 INFO:teuthology.orchestra.run.smithi064.stdout: "prefix": { 2023-09-13T03:29:14.816 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prefix", 2023-09-13T03:29:14.816 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.817 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.817 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.817 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:14.818 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.818 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.818 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.818 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "name prefix for any created local pool", 2023-09-13T03:29:14.819 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.819 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.819 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.820 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.820 INFO:teuthology.orchestra.run.smithi064.stdout: "subtree": { 2023-09-13T03:29:14.820 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "subtree", 2023-09-13T03:29:14.820 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.821 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.821 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.821 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "rack", 2023-09-13T03:29:14.822 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.822 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.822 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.822 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "CRUSH level for which to create a local pool", 2023-09-13T03:29:14.823 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-09-13T03:29:14.823 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.823 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.824 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:14.824 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:14.824 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.825 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:14.825 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mds_autoscaler", 2023-09-13T03:29:14.825 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:14.825 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:14.826 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:14.826 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:14.826 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:14.827 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.827 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.827 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.827 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:14.828 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.828 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.828 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:14.828 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:14.829 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:14.829 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:14.829 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:14.830 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:14.830 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:14.830 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:14.831 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.831 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.831 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.831 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.832 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.832 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:14.832 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:14.833 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.833 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.833 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.834 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.834 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.834 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.834 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.835 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.835 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.835 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.836 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.836 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.836 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:14.836 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:14.837 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.837 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.837 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.838 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:14.838 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.838 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.838 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:14.839 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:14.839 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:14.839 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:14.840 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:14.840 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:14.840 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:14.840 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:14.841 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.841 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.841 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.842 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.842 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.842 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:14.843 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:14.843 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.843 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.843 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.844 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.844 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.844 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.845 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.845 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.845 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.845 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.846 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.846 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:14.846 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:14.847 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.847 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:14.847 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mirroring", 2023-09-13T03:29:14.848 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:14.848 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:14.848 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:14.848 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:14.849 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:14.849 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.849 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.850 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.850 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:14.850 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.850 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.851 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:14.851 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:14.851 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:14.852 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:14.852 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:14.852 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:14.852 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:14.853 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:14.853 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.853 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.854 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.854 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.854 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.854 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:14.855 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:14.855 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.855 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.856 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.856 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.856 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.857 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.857 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.857 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.858 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.858 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.858 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.858 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.859 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:14.859 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:14.859 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.860 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.860 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.860 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:14.860 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.861 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.861 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:14.861 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:14.862 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:14.862 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:14.862 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:14.862 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:14.863 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:14.863 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:14.863 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.863 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.864 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.864 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.864 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.864 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:14.865 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:14.865 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.865 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.866 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.866 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.866 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.867 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.867 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.867 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.867 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.868 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.868 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.868 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:14.869 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:14.869 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.869 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:14.869 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "nfs", 2023-09-13T03:29:14.870 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:14.870 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:14.870 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:14.871 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:14.871 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:14.871 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.872 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.872 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.872 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:14.872 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.873 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.873 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:14.873 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:14.874 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:14.874 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:14.874 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:14.875 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:14.875 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:14.875 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:14.875 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.876 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.876 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.876 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.876 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.877 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:14.877 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:14.877 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.878 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.878 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.878 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.878 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.879 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.879 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.879 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.880 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.880 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.880 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.881 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.881 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:14.881 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:14.882 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.882 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.882 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.882 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:14.883 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.883 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.883 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:14.884 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:14.884 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:14.884 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:14.884 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:14.885 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:14.885 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:14.885 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:14.886 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.886 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.886 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.886 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.887 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.887 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:14.887 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:14.888 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.888 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.888 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.888 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.889 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.889 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.889 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.890 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.890 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.890 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.890 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.891 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:14.891 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:14.891 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.892 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:14.892 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "orchestrator", 2023-09-13T03:29:14.892 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:14.892 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:14.893 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:14.893 INFO:teuthology.orchestra.run.smithi064.stdout: "fail_fs": { 2023-09-13T03:29:14.893 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "fail_fs", 2023-09-13T03:29:14.894 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.894 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.894 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.894 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.895 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.895 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.895 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.895 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-09-13T03:29:14.896 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.896 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.896 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.897 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.897 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:14.897 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:14.897 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.898 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.898 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.898 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:14.899 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.899 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.899 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:14.899 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:14.900 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:14.900 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:14.900 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:14.901 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:14.901 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:14.901 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:14.901 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.902 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.902 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.902 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.903 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.903 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:14.903 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:14.903 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.904 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.904 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.904 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.904 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.905 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.905 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.905 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.906 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.906 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.906 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.906 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.907 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:14.907 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:14.907 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.908 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.908 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.908 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:14.908 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.909 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.909 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:14.909 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:14.909 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:14.910 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:14.910 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:14.910 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:14.911 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:14.911 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:14.911 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.911 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.912 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.912 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.912 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.913 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:14.913 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:14.913 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.913 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.914 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.914 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.914 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.915 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.915 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.915 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.915 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.916 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.916 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.916 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.916 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator": { 2023-09-13T03:29:14.917 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "orchestrator", 2023-09-13T03:29:14.917 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.917 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.918 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.918 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:14.918 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.919 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.919 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:14.919 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm", 2023-09-13T03:29:14.919 INFO:teuthology.orchestra.run.smithi064.stdout: "rook", 2023-09-13T03:29:14.920 INFO:teuthology.orchestra.run.smithi064.stdout: "test_orchestrator" 2023-09-13T03:29:14.920 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:14.920 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Orchestrator backend", 2023-09-13T03:29:14.921 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.921 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.921 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.921 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:14.922 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:14.922 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.922 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:14.922 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "osd_perf_query", 2023-09-13T03:29:14.923 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:14.923 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:14.923 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:14.924 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:14.924 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:14.924 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.924 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.925 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.925 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:14.925 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.925 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.926 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:14.926 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:14.926 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:14.927 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:14.927 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:14.927 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:14.927 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:14.928 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:14.928 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.928 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.928 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.929 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.929 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.929 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:14.930 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:14.930 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.930 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.931 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.931 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.931 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.931 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.932 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.932 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.932 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.933 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.933 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.933 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.933 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:14.934 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:14.934 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.934 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.935 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.935 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:14.935 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.935 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.936 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:14.936 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:14.936 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:14.937 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:14.937 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:14.937 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:14.937 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:14.938 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:14.938 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.938 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.939 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.939 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.939 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.940 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:14.940 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:14.940 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.940 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.941 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.941 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.941 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.941 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.942 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.942 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.942 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.943 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.943 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.943 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:14.943 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:14.944 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.944 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:14.944 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "osd_support", 2023-09-13T03:29:14.945 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:14.945 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:14.945 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:14.946 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:14.946 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:14.946 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.946 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.947 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.947 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:14.947 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.947 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.948 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:14.948 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:14.948 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:14.949 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:14.949 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:14.949 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:14.949 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:14.950 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:14.950 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.950 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.951 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.951 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.951 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.951 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:14.952 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:14.952 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.952 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.953 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.953 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.953 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.953 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.954 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.954 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.954 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.954 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.955 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.955 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.955 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:14.955 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:14.956 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.956 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.956 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.956 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:14.957 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.957 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.957 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:14.957 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:14.958 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:14.958 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:14.958 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:14.959 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:14.959 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:14.959 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:14.959 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.960 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.960 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.960 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.960 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.961 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:14.961 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:14.961 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.962 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.962 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.962 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.962 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.963 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.963 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.963 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.964 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.964 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.964 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.964 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:14.965 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:14.965 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.965 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:14.965 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pg_autoscaler", 2023-09-13T03:29:14.966 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:14.966 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:14.966 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:14.967 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:14.967 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:14.967 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.967 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.968 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.968 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:14.968 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.969 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.969 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:14.969 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:14.969 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:14.970 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:14.970 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:14.970 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:14.970 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:14.971 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:14.971 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.971 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.971 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.972 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.972 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.972 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:14.972 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:14.973 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.973 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.973 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.973 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.974 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.974 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.974 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.975 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.975 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.975 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.975 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.976 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.976 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:14.976 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:14.976 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:14.977 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.977 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.977 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:14.977 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.978 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.978 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:14.978 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:14.978 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:14.979 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:14.979 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:14.979 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:14.980 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:14.980 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:14.980 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.980 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.981 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.981 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.981 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.981 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:14.982 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:14.982 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.982 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.982 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:14.983 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.983 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.983 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.984 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.984 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.984 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.984 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.985 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.985 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.985 INFO:teuthology.orchestra.run.smithi064.stdout: "noautoscale": { 2023-09-13T03:29:14.986 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "noautoscale", 2023-09-13T03:29:14.986 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.986 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.986 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.987 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:14.987 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.987 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.987 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.988 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "global autoscale flag", 2023-09-13T03:29:14.988 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-09-13T03:29:14.988 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.989 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.989 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.989 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2023-09-13T03:29:14.989 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2023-09-13T03:29:14.990 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:14.990 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.990 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.991 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2023-09-13T03:29:14.991 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:14.991 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.991 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.992 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:14.992 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:14.992 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.992 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.993 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.993 INFO:teuthology.orchestra.run.smithi064.stdout: "threshold": { 2023-09-13T03:29:14.993 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "threshold", 2023-09-13T03:29:14.993 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2023-09-13T03:29:14.994 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:14.994 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:14.994 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3.0", 2023-09-13T03:29:14.995 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1.0", 2023-09-13T03:29:14.995 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:14.995 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:14.995 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "scaling threshold", 2023-09-13T03:29:14.996 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-09-13T03:29:14.996 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:14.996 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:14.996 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:14.997 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:14.997 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:14.997 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:14.997 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "progress", 2023-09-13T03:29:14.998 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:14.998 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:14.998 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:14.999 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_pg_recovery_event": { 2023-09-13T03:29:14.999 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_pg_recovery_event", 2023-09-13T03:29:14.999 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:14.999 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.000 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.000 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:15.000 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.000 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.001 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.001 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow the module to show pg recovery progress", 2023-09-13T03:29:15.001 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.001 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.002 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.002 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.002 INFO:teuthology.orchestra.run.smithi064.stdout: "enabled": { 2023-09-13T03:29:15.002 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enabled", 2023-09-13T03:29:15.003 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.003 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.003 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.003 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:15.004 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.004 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.004 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.004 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.005 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.005 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.005 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.005 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.006 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:15.006 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:15.006 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.007 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.007 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.007 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:15.007 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.008 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.008 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:15.008 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:15.009 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:15.009 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:15.009 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:15.009 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:15.010 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:15.010 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:15.010 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.011 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.011 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.011 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.011 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.012 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:15.012 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:15.012 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.013 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.013 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.013 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:15.013 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.014 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.014 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.014 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.015 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.015 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.015 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.015 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.016 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:15.016 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:15.016 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.017 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.017 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.017 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:15.017 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.018 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.018 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:15.018 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:15.019 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:15.019 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:15.019 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:15.019 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:15.020 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:15.020 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:15.020 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.020 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.021 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.021 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.021 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.022 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:15.022 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:15.022 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.022 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.023 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.023 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:15.023 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.023 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.024 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.024 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.024 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.024 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.025 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.025 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.025 INFO:teuthology.orchestra.run.smithi064.stdout: "max_completed_events": { 2023-09-13T03:29:15.025 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_completed_events", 2023-09-13T03:29:15.026 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:15.026 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.026 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.027 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "50", 2023-09-13T03:29:15.027 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.027 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.027 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.028 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "number of past completed events to remember", 2023-09-13T03:29:15.028 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.028 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.029 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.029 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.029 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2023-09-13T03:29:15.029 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2023-09-13T03:29:15.030 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:15.030 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.030 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.031 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2023-09-13T03:29:15.031 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.031 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.031 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.032 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long the module is going to sleep", 2023-09-13T03:29:15.032 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.032 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.032 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.033 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:15.033 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:15.033 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.034 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:15.034 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prometheus", 2023-09-13T03:29:15.034 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:15.034 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:15.035 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:15.035 INFO:teuthology.orchestra.run.smithi064.stdout: "cache": { 2023-09-13T03:29:15.035 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cache", 2023-09-13T03:29:15.036 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.036 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.036 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.036 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:15.037 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.037 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.037 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.038 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.038 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.038 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.038 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.039 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.039 INFO:teuthology.orchestra.run.smithi064.stdout: "exclude_perf_counters": { 2023-09-13T03:29:15.039 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "exclude_perf_counters", 2023-09-13T03:29:15.039 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.040 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.040 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.040 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:15.040 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.041 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.041 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.041 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-09-13T03:29:15.042 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-09-13T03:29:15.042 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.042 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.042 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.043 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:15.043 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:15.043 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.043 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.044 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.044 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:15.044 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.044 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.045 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:15.045 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:15.045 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:15.046 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:15.046 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:15.046 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:15.047 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:15.047 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:15.047 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.047 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.048 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.048 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.048 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.049 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:15.049 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:15.049 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.050 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.050 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.051 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:15.051 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.051 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.052 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.052 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.053 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.053 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.053 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.054 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.054 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:15.055 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:15.055 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.055 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.055 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.056 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:15.056 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.056 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.057 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:15.057 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:15.057 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:15.057 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:15.058 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:15.058 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:15.058 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:15.059 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:15.059 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.059 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.059 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.059 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.060 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.060 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:15.060 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:15.060 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.061 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.061 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.061 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:15.062 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.062 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.062 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.062 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.063 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.063 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.063 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.063 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.064 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_stats_pools": { 2023-09-13T03:29:15.064 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_stats_pools", 2023-09-13T03:29:15.064 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.065 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.065 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.065 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:15.065 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.066 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.066 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.066 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.066 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.067 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.067 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.067 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.067 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_stats_pools_refresh_interval": { 2023-09-13T03:29:15.068 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-09-13T03:29:15.068 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:15.068 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.068 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.069 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "300", 2023-09-13T03:29:15.069 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.069 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.070 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.070 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.070 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.070 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.071 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.071 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.071 INFO:teuthology.orchestra.run.smithi064.stdout: "scrape_interval": { 2023-09-13T03:29:15.072 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "scrape_interval", 2023-09-13T03:29:15.072 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2023-09-13T03:29:15.072 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.072 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.073 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15.0", 2023-09-13T03:29:15.073 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.073 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.073 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.074 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.074 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.074 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.075 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.075 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.075 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2023-09-13T03:29:15.075 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2023-09-13T03:29:15.076 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.076 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.076 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.076 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "::", 2023-09-13T03:29:15.077 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.077 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.077 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.078 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-09-13T03:29:15.078 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.078 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.078 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.079 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.079 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2023-09-13T03:29:15.079 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2023-09-13T03:29:15.080 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:15.080 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.080 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.080 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "9283", 2023-09-13T03:29:15.081 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.081 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.081 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.082 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-09-13T03:29:15.082 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.082 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.082 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.083 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.083 INFO:teuthology.orchestra.run.smithi064.stdout: "stale_cache_strategy": { 2023-09-13T03:29:15.083 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "stale_cache_strategy", 2023-09-13T03:29:15.083 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.084 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.084 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.084 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "log", 2023-09-13T03:29:15.085 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.085 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.085 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.085 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.086 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.086 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.086 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.087 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.087 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_behaviour": { 2023-09-13T03:29:15.087 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_behaviour", 2023-09-13T03:29:15.087 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.087 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.088 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.088 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "default", 2023-09-13T03:29:15.088 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.089 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.089 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:15.089 INFO:teuthology.orchestra.run.smithi064.stdout: "default", 2023-09-13T03:29:15.089 INFO:teuthology.orchestra.run.smithi064.stdout: "error" 2023-09-13T03:29:15.090 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:15.090 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.090 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.091 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.091 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.091 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.091 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_error_status_code": { 2023-09-13T03:29:15.092 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_error_status_code", 2023-09-13T03:29:15.092 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:15.092 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.093 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.093 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "500", 2023-09-13T03:29:15.093 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "400", 2023-09-13T03:29:15.093 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "599", 2023-09-13T03:29:15.094 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.094 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.094 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.094 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.095 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.095 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:15.095 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:15.095 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.096 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:15.096 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_support", 2023-09-13T03:29:15.096 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:15.097 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:15.097 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:15.097 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:15.097 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:15.098 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.098 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.098 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.099 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:15.099 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.099 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.099 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:15.100 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:15.100 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:15.100 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:15.101 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:15.101 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:15.101 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:15.101 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:15.102 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.102 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.102 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.103 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.103 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.103 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:15.103 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:15.104 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.104 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.104 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.104 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:15.105 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.105 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.105 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.106 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.106 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.106 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.106 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.107 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:15.107 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:15.107 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.108 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.108 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.108 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:15.109 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.109 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.109 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:15.109 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:15.110 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:15.110 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:15.110 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:15.110 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:15.111 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:15.111 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:15.111 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.112 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.112 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.112 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.112 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.113 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:15.113 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:15.113 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.114 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.114 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.114 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:15.114 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.115 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.115 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.115 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.116 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.116 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.116 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.116 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.117 INFO:teuthology.orchestra.run.smithi064.stdout: "max_concurrent_snap_create": { 2023-09-13T03:29:15.117 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_concurrent_snap_create", 2023-09-13T03:29:15.117 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:15.117 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.118 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.118 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2023-09-13T03:29:15.118 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.119 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.119 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.119 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.119 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.120 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.120 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.120 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.120 INFO:teuthology.orchestra.run.smithi064.stdout: "mirror_snapshot_schedule": { 2023-09-13T03:29:15.121 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mirror_snapshot_schedule", 2023-09-13T03:29:15.121 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.121 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.122 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.122 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:15.122 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.122 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.123 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.123 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.123 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.123 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.124 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.124 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.124 INFO:teuthology.orchestra.run.smithi064.stdout: "trash_purge_schedule": { 2023-09-13T03:29:15.125 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "trash_purge_schedule", 2023-09-13T03:29:15.125 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.125 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.125 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.126 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:15.126 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.126 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.126 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.127 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.127 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.127 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.128 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.128 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:15.128 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:15.128 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.129 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:15.129 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "restful", 2023-09-13T03:29:15.129 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:15.129 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:15.130 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:15.130 INFO:teuthology.orchestra.run.smithi064.stdout: "enable_auth": { 2023-09-13T03:29:15.130 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enable_auth", 2023-09-13T03:29:15.131 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.131 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.131 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.131 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:15.132 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.132 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.132 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.133 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.133 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.133 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.133 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.134 INFO:teuthology.orchestra.run.smithi064.stdout: "key_file": { 2023-09-13T03:29:15.134 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "key_file", 2023-09-13T03:29:15.135 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.135 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.135 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.135 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:15.136 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.136 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.136 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.136 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.137 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.137 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.137 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.138 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:15.138 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:15.139 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.139 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.139 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.139 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:15.140 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.140 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.140 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:15.140 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:15.141 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:15.141 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:15.141 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:15.141 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:15.142 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:15.142 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:15.142 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.143 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.143 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.143 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.143 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.144 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:15.144 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:15.144 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.144 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.145 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.145 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:15.145 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.146 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.146 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.146 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.146 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.147 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.147 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.148 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:15.148 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:15.148 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.148 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.149 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.149 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:15.149 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.149 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.150 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:15.150 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:15.150 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:15.151 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:15.151 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:15.151 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:15.151 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:15.152 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:15.152 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.152 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.152 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.153 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.153 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.153 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:15.154 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:15.154 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.154 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.154 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.155 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:15.155 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.155 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.155 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.156 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.156 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.156 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.157 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.157 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.157 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2023-09-13T03:29:15.157 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2023-09-13T03:29:15.158 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.158 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.158 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.159 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:15.159 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.159 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.159 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.160 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.160 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.160 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.160 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.161 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.161 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2023-09-13T03:29:15.161 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2023-09-13T03:29:15.162 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.162 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.162 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.162 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:15.163 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.163 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.163 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.164 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.164 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.164 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.164 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.165 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:15.165 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:15.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.165 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:15.166 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rgw", 2023-09-13T03:29:15.166 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:15.166 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:15.167 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:15.167 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:15.167 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:15.167 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.168 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.168 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.168 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:15.168 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.169 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.169 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:15.169 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:15.170 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:15.170 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:15.170 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:15.170 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:15.171 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:15.171 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:15.171 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.171 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.172 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.172 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.172 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.173 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:15.173 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:15.173 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.173 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.174 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.174 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:15.174 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.175 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.176 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.177 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.177 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.177 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.178 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.178 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.178 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:15.179 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:15.179 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.179 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.179 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.180 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:15.180 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.180 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.180 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:15.181 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:15.181 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:15.181 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:15.181 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:15.182 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:15.182 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:15.182 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:15.183 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.183 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.183 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.183 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.184 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.184 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:15.184 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:15.185 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.185 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.185 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.185 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:15.186 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.186 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.186 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.187 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.187 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.187 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.187 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.188 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:15.188 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:15.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.188 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:15.189 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rook", 2023-09-13T03:29:15.189 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:15.189 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:15.189 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:15.190 INFO:teuthology.orchestra.run.smithi064.stdout: "drive_group_interval": { 2023-09-13T03:29:15.190 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "drive_group_interval", 2023-09-13T03:29:15.190 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2023-09-13T03:29:15.191 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.191 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.191 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "300.0", 2023-09-13T03:29:15.192 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.192 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.192 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.192 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-09-13T03:29:15.193 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.193 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.193 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.193 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.194 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:15.194 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:15.194 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.195 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.195 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.195 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:15.195 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.196 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.196 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:15.196 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:15.196 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:15.197 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:15.197 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:15.197 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:15.198 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:15.198 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:15.198 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.199 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.199 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.199 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.199 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.200 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:15.200 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:15.200 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.200 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.201 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.201 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:15.201 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.202 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.202 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.202 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.202 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.203 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.203 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.203 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.204 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:15.204 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:15.204 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.204 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.205 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.205 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:15.205 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.206 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.206 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:15.206 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:15.206 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:15.207 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:15.207 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:15.207 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:15.207 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:15.208 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:15.208 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.208 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.209 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.209 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.209 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.209 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:15.210 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:15.210 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.210 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.211 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.211 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:15.211 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.211 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.212 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.212 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.212 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.213 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.213 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.213 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.213 INFO:teuthology.orchestra.run.smithi064.stdout: "storage_class": { 2023-09-13T03:29:15.214 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "storage_class", 2023-09-13T03:29:15.214 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.214 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.215 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.215 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "local", 2023-09-13T03:29:15.215 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.215 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.216 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.216 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-09-13T03:29:15.216 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.217 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.217 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.217 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:15.217 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:15.218 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.218 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:15.218 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "selftest", 2023-09-13T03:29:15.218 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:15.219 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:15.219 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:15.219 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:15.220 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:15.220 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.220 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.220 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.221 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:15.221 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.221 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.221 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:15.222 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:15.222 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:15.222 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:15.223 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:15.223 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:15.223 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:15.223 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:15.224 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.224 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.224 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.224 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.225 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.225 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:15.225 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:15.225 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.226 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.226 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.226 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:15.227 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.227 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.227 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.227 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.228 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.228 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.228 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.229 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.229 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:15.229 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:15.229 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.230 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.230 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.230 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:15.231 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.231 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.231 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:15.231 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:15.232 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:15.232 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:15.232 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:15.232 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:15.233 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:15.233 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:15.233 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.234 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.234 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.234 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.234 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.235 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:15.235 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:15.235 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.236 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.236 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.236 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:15.236 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.237 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.237 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.237 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.238 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.238 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.238 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.238 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.239 INFO:teuthology.orchestra.run.smithi064.stdout: "roption1": { 2023-09-13T03:29:15.239 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "roption1", 2023-09-13T03:29:15.239 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.239 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.240 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.240 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:15.240 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.241 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.241 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.241 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.241 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.242 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.242 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.242 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.243 INFO:teuthology.orchestra.run.smithi064.stdout: "roption2": { 2023-09-13T03:29:15.243 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "roption2", 2023-09-13T03:29:15.243 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.243 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.244 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.244 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "xyz", 2023-09-13T03:29:15.244 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.244 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.245 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.245 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.245 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.246 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.246 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.246 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.246 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption1": { 2023-09-13T03:29:15.247 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption1", 2023-09-13T03:29:15.247 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.247 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.248 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.248 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:15.248 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.248 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.249 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.249 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.249 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.250 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.250 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.250 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.250 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption2": { 2023-09-13T03:29:15.251 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption2", 2023-09-13T03:29:15.251 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:15.251 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.251 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.252 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:15.252 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.252 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.252 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.253 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.253 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.253 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.254 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.254 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.254 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption3": { 2023-09-13T03:29:15.254 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption3", 2023-09-13T03:29:15.255 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2023-09-13T03:29:15.255 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.255 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.256 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:15.256 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.256 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.256 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.257 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.257 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.257 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.257 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.258 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.258 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption4": { 2023-09-13T03:29:15.258 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption4", 2023-09-13T03:29:15.259 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.259 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.259 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.259 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:15.260 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.260 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.260 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.261 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.261 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.261 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.261 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.262 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.262 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption5": { 2023-09-13T03:29:15.262 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption5", 2023-09-13T03:29:15.262 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.263 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.263 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.263 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:15.264 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.264 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.264 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.264 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.265 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.265 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.265 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.266 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.266 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption6": { 2023-09-13T03:29:15.266 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption6", 2023-09-13T03:29:15.266 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.267 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.267 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.267 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:15.268 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.268 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.268 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.268 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.269 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.269 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.269 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.270 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.270 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption7": { 2023-09-13T03:29:15.270 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption7", 2023-09-13T03:29:15.270 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:15.271 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.271 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.271 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:15.272 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2023-09-13T03:29:15.272 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "42", 2023-09-13T03:29:15.272 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.272 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.273 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.273 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.273 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.274 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.274 INFO:teuthology.orchestra.run.smithi064.stdout: "testkey": { 2023-09-13T03:29:15.274 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testkey", 2023-09-13T03:29:15.274 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.275 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.275 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.275 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:15.276 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.276 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.276 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.276 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.277 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.277 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.277 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.278 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.278 INFO:teuthology.orchestra.run.smithi064.stdout: "testlkey": { 2023-09-13T03:29:15.278 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testlkey", 2023-09-13T03:29:15.279 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.279 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.279 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.279 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:15.280 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.280 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.280 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.281 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.281 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.281 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.281 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.282 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.282 INFO:teuthology.orchestra.run.smithi064.stdout: "testnewline": { 2023-09-13T03:29:15.282 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testnewline", 2023-09-13T03:29:15.283 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.283 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.283 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.283 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:15.284 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.284 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.284 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.284 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.285 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.285 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.285 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.286 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:15.286 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:15.286 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.286 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:15.287 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snap_schedule", 2023-09-13T03:29:15.287 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:15.287 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:15.288 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:15.288 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_m_granularity": { 2023-09-13T03:29:15.288 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_m_granularity", 2023-09-13T03:29:15.288 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.289 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.289 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.289 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:15.289 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.290 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.290 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.290 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow minute scheduled snapshots", 2023-09-13T03:29:15.291 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.291 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.291 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.291 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.292 INFO:teuthology.orchestra.run.smithi064.stdout: "dump_on_update": { 2023-09-13T03:29:15.292 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "dump_on_update", 2023-09-13T03:29:15.292 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.292 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.293 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.293 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:15.293 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.293 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.294 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.294 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "dump database to debug log on update", 2023-09-13T03:29:15.294 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.295 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.295 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.295 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.295 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:15.296 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:15.296 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.296 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.296 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.297 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:15.297 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.297 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.298 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:15.298 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:15.298 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:15.298 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:15.299 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:15.299 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:15.299 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:15.299 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:15.300 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.300 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.300 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.301 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.301 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:15.301 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:15.302 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.302 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.302 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.303 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:15.303 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.303 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.303 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.304 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.304 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.304 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.304 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.305 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.305 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:15.305 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:15.306 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.306 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.306 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.306 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:15.307 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.307 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.307 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:15.308 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:15.308 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:15.308 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:15.308 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:15.309 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:15.309 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:15.309 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:15.310 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.310 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.310 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.310 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.311 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.311 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:15.311 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:15.312 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.312 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.312 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.312 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:15.313 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.313 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.313 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.314 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.314 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.314 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.314 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.315 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:15.315 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:15.315 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.316 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:15.316 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "stats", 2023-09-13T03:29:15.316 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:15.316 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:15.317 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:15.317 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:15.317 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:15.317 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.318 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.318 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.318 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:15.319 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.319 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.319 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:15.319 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:15.320 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:15.320 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:15.320 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:15.321 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:15.321 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:15.321 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:15.321 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.322 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.322 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.322 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.323 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.323 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:15.323 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:15.323 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.324 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.324 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.324 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:15.324 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.325 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.325 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.325 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.326 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.326 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.326 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.326 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.327 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:15.327 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:15.327 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.327 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.328 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.328 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:15.328 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.329 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.329 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:15.329 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:15.329 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:15.330 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:15.330 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:15.330 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:15.331 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:15.331 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:15.331 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.331 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.332 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.332 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.332 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.332 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:15.333 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:15.333 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.333 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.334 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.334 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:15.334 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.334 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.335 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.335 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.335 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.336 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.336 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.336 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:15.336 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:15.337 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.337 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:15.337 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "status", 2023-09-13T03:29:15.337 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:15.338 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:15.338 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:15.338 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:15.339 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:15.339 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.339 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.339 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.340 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:15.340 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.340 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.341 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:15.341 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:15.341 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:15.341 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:15.342 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:15.342 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:15.342 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:15.343 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:15.343 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.343 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.343 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.344 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.344 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.344 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:15.344 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:15.345 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.345 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.345 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.346 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:15.346 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.346 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.347 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.347 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.347 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.347 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.348 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.348 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.348 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:15.348 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:15.349 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.349 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.349 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.350 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:15.350 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.350 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.350 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:15.351 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:15.351 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:15.351 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:15.351 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:15.352 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:15.352 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:15.352 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:15.353 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.353 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.353 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.353 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.354 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.354 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:15.354 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:15.354 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.355 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.355 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.355 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:15.356 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.356 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.356 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.356 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.357 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.357 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.357 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.358 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:15.358 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:15.358 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.358 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:15.359 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "telegraf", 2023-09-13T03:29:15.359 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:15.359 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:15.359 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:15.360 INFO:teuthology.orchestra.run.smithi064.stdout: "address": { 2023-09-13T03:29:15.360 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "address", 2023-09-13T03:29:15.360 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.361 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.361 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.361 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-09-13T03:29:15.361 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.362 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.362 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.362 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.363 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.363 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.363 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.363 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.364 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2023-09-13T03:29:15.364 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2023-09-13T03:29:15.364 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:15.365 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.365 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.365 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15", 2023-09-13T03:29:15.365 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.366 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.366 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.366 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.366 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.367 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.367 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.367 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.368 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:15.368 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:15.368 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.368 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.369 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.369 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:15.369 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.370 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.370 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:15.370 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:15.370 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:15.371 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:15.371 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:15.371 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:15.372 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:15.372 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:15.372 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.372 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.373 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.373 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.373 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.373 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:15.374 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:15.374 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.374 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.375 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.375 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:15.375 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.376 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.376 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.376 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.376 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.377 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.377 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.377 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.377 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:15.378 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:15.378 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.378 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.379 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.379 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:15.379 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.379 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.380 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:15.380 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:15.380 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:15.381 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:15.381 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:15.381 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:15.381 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:15.382 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:15.382 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.382 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.383 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.383 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.383 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.383 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:15.384 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:15.384 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.384 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.384 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.385 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:15.385 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.385 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.386 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.386 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.386 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.386 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.387 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.387 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:15.387 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:15.388 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.388 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:15.388 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "telemetry", 2023-09-13T03:29:15.388 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:15.389 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:15.389 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:15.389 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_basic": { 2023-09-13T03:29:15.389 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_basic", 2023-09-13T03:29:15.390 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.390 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.390 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.391 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:15.391 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.391 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.391 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.392 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share basic cluster information (size, version)", 2023-09-13T03:29:15.392 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.392 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.393 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.393 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.393 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_crash": { 2023-09-13T03:29:15.393 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_crash", 2023-09-13T03:29:15.394 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.394 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.394 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.395 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:15.395 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.395 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.395 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.396 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-09-13T03:29:15.396 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.396 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.397 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.397 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.397 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_device": { 2023-09-13T03:29:15.397 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_device", 2023-09-13T03:29:15.398 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.398 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.398 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.399 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2023-09-13T03:29:15.399 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.399 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.399 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.400 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-09-13T03:29:15.400 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.400 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.400 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.401 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.401 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_ident": { 2023-09-13T03:29:15.401 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_ident", 2023-09-13T03:29:15.401 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.402 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.402 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.402 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:15.403 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.403 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.403 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.403 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-09-13T03:29:15.404 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.404 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.404 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.405 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.405 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_perf": { 2023-09-13T03:29:15.405 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_perf", 2023-09-13T03:29:15.405 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.406 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.406 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.406 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:15.406 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.407 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.407 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.407 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share various performance metrics of a cluster", 2023-09-13T03:29:15.408 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.408 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.408 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.409 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.409 INFO:teuthology.orchestra.run.smithi064.stdout: "contact": { 2023-09-13T03:29:15.409 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "contact", 2023-09-13T03:29:15.409 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.410 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.410 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.410 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:15.411 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.411 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.411 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.412 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.412 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.412 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.412 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.413 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.413 INFO:teuthology.orchestra.run.smithi064.stdout: "description": { 2023-09-13T03:29:15.413 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "description", 2023-09-13T03:29:15.414 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.414 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.414 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.414 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:15.415 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.415 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.415 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.416 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.416 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.416 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.416 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.417 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.417 INFO:teuthology.orchestra.run.smithi064.stdout: "device_url": { 2023-09-13T03:29:15.417 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_url", 2023-09-13T03:29:15.418 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.418 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.418 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.418 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-09-13T03:29:15.419 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.419 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.419 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.420 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.420 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.420 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.420 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.421 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.421 INFO:teuthology.orchestra.run.smithi064.stdout: "enabled": { 2023-09-13T03:29:15.421 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enabled", 2023-09-13T03:29:15.422 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.422 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.422 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.423 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:15.423 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.423 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.423 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.424 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.424 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.424 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.425 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.425 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.425 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2023-09-13T03:29:15.425 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2023-09-13T03:29:15.426 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:15.426 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.426 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.427 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "24", 2023-09-13T03:29:15.427 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "8", 2023-09-13T03:29:15.427 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.427 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.428 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.428 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.428 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.429 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.429 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.429 INFO:teuthology.orchestra.run.smithi064.stdout: "last_opt_revision": { 2023-09-13T03:29:15.429 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "last_opt_revision", 2023-09-13T03:29:15.430 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:15.430 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.430 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.431 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1", 2023-09-13T03:29:15.431 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.431 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.431 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.432 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.432 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.432 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.433 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.433 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.433 INFO:teuthology.orchestra.run.smithi064.stdout: "leaderboard": { 2023-09-13T03:29:15.433 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "leaderboard", 2023-09-13T03:29:15.434 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.434 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.434 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.434 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:15.435 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.435 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.435 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.436 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.436 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.436 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.437 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.437 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.437 INFO:teuthology.orchestra.run.smithi064.stdout: "leaderboard_description": { 2023-09-13T03:29:15.437 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "leaderboard_description", 2023-09-13T03:29:15.438 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.438 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.438 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.439 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:15.439 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.439 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.439 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.440 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.440 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.440 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.441 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.441 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.441 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:15.441 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:15.442 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.442 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.442 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.443 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:15.443 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.443 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.444 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:15.444 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:15.444 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:15.444 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:15.445 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:15.445 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:15.445 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:15.446 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:15.446 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.446 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.446 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.447 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.447 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.447 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:15.448 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:15.448 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.448 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.448 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.449 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:15.449 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.449 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.450 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.450 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.450 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.450 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.451 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.451 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.451 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:15.452 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:15.452 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.452 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.452 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.453 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:15.453 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.453 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.454 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:15.454 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:15.454 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:15.454 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:15.455 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:15.455 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:15.455 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:15.456 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:15.456 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.456 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.456 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.457 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.457 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.457 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:15.457 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:15.458 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.458 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.458 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.459 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:15.459 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.459 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.459 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.460 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.460 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.460 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.461 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.461 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.461 INFO:teuthology.orchestra.run.smithi064.stdout: "organization": { 2023-09-13T03:29:15.462 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "organization", 2023-09-13T03:29:15.462 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.462 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.462 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.463 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:15.463 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.463 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.463 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.464 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.464 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.464 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.465 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.465 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.465 INFO:teuthology.orchestra.run.smithi064.stdout: "proxy": { 2023-09-13T03:29:15.465 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "proxy", 2023-09-13T03:29:15.466 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.466 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.466 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.467 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:15.467 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.467 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.467 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.468 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.468 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.468 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.468 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.469 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.469 INFO:teuthology.orchestra.run.smithi064.stdout: "url": { 2023-09-13T03:29:15.469 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "url", 2023-09-13T03:29:15.470 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.470 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.470 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.470 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-09-13T03:29:15.471 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.471 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.471 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.472 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.472 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.472 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.472 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.473 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:15.473 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:15.473 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.473 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:15.474 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "test_orchestrator", 2023-09-13T03:29:15.474 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:15.474 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:15.475 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:15.475 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:15.475 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:15.476 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.476 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.476 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.476 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:15.477 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.477 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.477 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:15.478 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:15.478 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:15.478 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:15.478 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:15.479 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:15.479 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:15.479 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:15.479 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.480 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.480 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.480 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.481 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.481 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:15.481 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:15.481 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.482 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.482 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.482 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:15.483 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.483 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.483 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.484 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.484 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.484 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.484 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.485 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.485 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:15.485 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:15.485 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.486 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.486 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.486 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:15.487 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.487 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.487 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:15.488 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:15.488 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:15.488 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:15.488 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:15.489 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:15.489 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:15.489 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:15.489 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.490 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.490 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.490 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.490 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.491 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:15.491 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:15.491 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.492 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.492 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.492 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:15.493 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.493 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.493 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.493 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.494 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.494 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.494 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.494 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:15.495 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:15.495 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.495 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:15.496 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "volumes", 2023-09-13T03:29:15.496 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:15.496 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:15.497 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:15.497 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:15.497 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:15.497 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.498 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.498 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.498 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:15.499 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.499 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.499 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:15.499 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:15.500 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:15.500 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:15.500 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:15.501 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:15.501 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:15.501 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:15.501 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.502 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.502 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.502 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.503 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.503 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:15.503 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:15.503 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.504 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.504 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.504 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:15.505 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.505 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.505 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.505 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.506 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.506 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.506 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.507 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.507 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:15.507 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:15.507 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.508 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.508 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.508 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:15.508 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.509 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.509 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:15.509 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:15.510 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:15.510 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:15.510 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:15.510 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:15.511 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:15.511 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:15.511 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.511 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.512 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.512 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.512 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.512 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:15.513 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:15.513 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.513 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.514 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.514 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:15.514 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.514 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.515 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.515 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.515 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.516 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.516 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.516 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.516 INFO:teuthology.orchestra.run.smithi064.stdout: "max_concurrent_clones": { 2023-09-13T03:29:15.517 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_concurrent_clones", 2023-09-13T03:29:15.517 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:15.517 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.517 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.518 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "4", 2023-09-13T03:29:15.518 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.518 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.519 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.519 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Number of asynchronous cloner threads", 2023-09-13T03:29:15.519 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.520 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.520 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.520 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.520 INFO:teuthology.orchestra.run.smithi064.stdout: "snapshot_clone_delay": { 2023-09-13T03:29:15.521 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snapshot_clone_delay", 2023-09-13T03:29:15.521 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:15.521 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.522 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.522 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2023-09-13T03:29:15.522 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.522 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.523 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.523 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-09-13T03:29:15.523 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.524 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.524 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.524 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:15.524 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:15.525 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.525 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:15.525 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix", 2023-09-13T03:29:15.525 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2023-09-13T03:29:15.526 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2023-09-13T03:29:15.526 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2023-09-13T03:29:15.526 INFO:teuthology.orchestra.run.smithi064.stdout: "discovery_interval": { 2023-09-13T03:29:15.527 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "discovery_interval", 2023-09-13T03:29:15.527 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2023-09-13T03:29:15.527 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.527 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.528 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "100", 2023-09-13T03:29:15.528 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.528 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.528 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.529 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.529 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.529 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.530 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.530 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.530 INFO:teuthology.orchestra.run.smithi064.stdout: "identifier": { 2023-09-13T03:29:15.530 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "identifier", 2023-09-13T03:29:15.531 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.531 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.531 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.532 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:15.532 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.532 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.532 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.533 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.533 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.533 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.533 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.534 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.534 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2023-09-13T03:29:15.534 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2023-09-13T03:29:15.535 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2023-09-13T03:29:15.535 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.535 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.535 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2023-09-13T03:29:15.536 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.536 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.536 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.536 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.537 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.537 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.537 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.538 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.538 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2023-09-13T03:29:15.538 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2023-09-13T03:29:15.538 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.539 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.539 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.539 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:15.540 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.540 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.540 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:15.540 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:15.541 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:15.541 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:15.541 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:15.541 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:15.542 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:15.542 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:15.542 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.543 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.543 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.543 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.543 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.544 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2023-09-13T03:29:15.544 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2023-09-13T03:29:15.544 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.545 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.545 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.545 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:15.545 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.546 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.546 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.546 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.547 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.547 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.547 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.547 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.548 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2023-09-13T03:29:15.548 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2023-09-13T03:29:15.548 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.548 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.549 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.549 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2023-09-13T03:29:15.549 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.550 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.550 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2023-09-13T03:29:15.550 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2023-09-13T03:29:15.550 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2023-09-13T03:29:15.551 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2023-09-13T03:29:15.551 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2023-09-13T03:29:15.551 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2023-09-13T03:29:15.552 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2023-09-13T03:29:15.552 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:15.552 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.552 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.553 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.553 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.553 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.553 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2023-09-13T03:29:15.554 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2023-09-13T03:29:15.554 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2023-09-13T03:29:15.554 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.555 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2023-09-13T03:29:15.555 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2023-09-13T03:29:15.555 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.555 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.556 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.556 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.556 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.557 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.557 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.557 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.557 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_host": { 2023-09-13T03:29:15.558 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_host", 2023-09-13T03:29:15.558 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.558 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.559 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.559 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2023-09-13T03:29:15.559 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.559 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.560 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.560 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.560 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.561 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.561 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.561 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.561 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_port": { 2023-09-13T03:29:15.562 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_port", 2023-09-13T03:29:15.562 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2023-09-13T03:29:15.562 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.562 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.563 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10051", 2023-09-13T03:29:15.563 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.563 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.564 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.564 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.564 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.564 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.565 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.565 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.565 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_sender": { 2023-09-13T03:29:15.565 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_sender", 2023-09-13T03:29:15.566 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2023-09-13T03:29:15.566 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2023-09-13T03:29:15.566 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2023-09-13T03:29:15.567 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-09-13T03:29:15.567 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2023-09-13T03:29:15.567 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2023-09-13T03:29:15.567 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2023-09-13T03:29:15.568 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2023-09-13T03:29:15.568 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2023-09-13T03:29:15.568 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2023-09-13T03:29:15.569 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2023-09-13T03:29:15.569 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:15.569 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:15.569 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:15.570 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:15.570 INFO:teuthology.orchestra.run.smithi064.stdout: "services": { 2023-09-13T03:29:15.570 INFO:teuthology.orchestra.run.smithi064.stdout: "dashboard": "https://172.21.15.64:8443/" 2023-09-13T03:29:15.571 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.571 INFO:teuthology.orchestra.run.smithi064.stdout: "always_on_modules": { 2023-09-13T03:29:15.571 INFO:teuthology.orchestra.run.smithi064.stdout: "octopus": [ 2023-09-13T03:29:15.571 INFO:teuthology.orchestra.run.smithi064.stdout: "balancer", 2023-09-13T03:29:15.572 INFO:teuthology.orchestra.run.smithi064.stdout: "crash", 2023-09-13T03:29:15.572 INFO:teuthology.orchestra.run.smithi064.stdout: "devicehealth", 2023-09-13T03:29:15.572 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator", 2023-09-13T03:29:15.572 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_autoscaler", 2023-09-13T03:29:15.573 INFO:teuthology.orchestra.run.smithi064.stdout: "progress", 2023-09-13T03:29:15.573 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_support", 2023-09-13T03:29:15.573 INFO:teuthology.orchestra.run.smithi064.stdout: "status", 2023-09-13T03:29:15.573 INFO:teuthology.orchestra.run.smithi064.stdout: "telemetry", 2023-09-13T03:29:15.574 INFO:teuthology.orchestra.run.smithi064.stdout: "volumes" 2023-09-13T03:29:15.574 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:15.574 INFO:teuthology.orchestra.run.smithi064.stdout: "pacific": [ 2023-09-13T03:29:15.575 INFO:teuthology.orchestra.run.smithi064.stdout: "balancer", 2023-09-13T03:29:15.575 INFO:teuthology.orchestra.run.smithi064.stdout: "crash", 2023-09-13T03:29:15.575 INFO:teuthology.orchestra.run.smithi064.stdout: "devicehealth", 2023-09-13T03:29:15.575 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator", 2023-09-13T03:29:15.576 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_autoscaler", 2023-09-13T03:29:15.576 INFO:teuthology.orchestra.run.smithi064.stdout: "progress", 2023-09-13T03:29:15.576 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_support", 2023-09-13T03:29:15.576 INFO:teuthology.orchestra.run.smithi064.stdout: "status", 2023-09-13T03:29:15.577 INFO:teuthology.orchestra.run.smithi064.stdout: "telemetry", 2023-09-13T03:29:15.577 INFO:teuthology.orchestra.run.smithi064.stdout: "volumes" 2023-09-13T03:29:15.577 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:15.577 INFO:teuthology.orchestra.run.smithi064.stdout: "quincy": [ 2023-09-13T03:29:15.578 INFO:teuthology.orchestra.run.smithi064.stdout: "balancer", 2023-09-13T03:29:15.578 INFO:teuthology.orchestra.run.smithi064.stdout: "crash", 2023-09-13T03:29:15.578 INFO:teuthology.orchestra.run.smithi064.stdout: "devicehealth", 2023-09-13T03:29:15.578 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator", 2023-09-13T03:29:15.579 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_autoscaler", 2023-09-13T03:29:15.579 INFO:teuthology.orchestra.run.smithi064.stdout: "progress", 2023-09-13T03:29:15.579 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_support", 2023-09-13T03:29:15.580 INFO:teuthology.orchestra.run.smithi064.stdout: "status", 2023-09-13T03:29:15.580 INFO:teuthology.orchestra.run.smithi064.stdout: "telemetry", 2023-09-13T03:29:15.580 INFO:teuthology.orchestra.run.smithi064.stdout: "volumes" 2023-09-13T03:29:15.580 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-09-13T03:29:15.581 INFO:teuthology.orchestra.run.smithi064.stdout: "reef": [ 2023-09-13T03:29:15.581 INFO:teuthology.orchestra.run.smithi064.stdout: "balancer", 2023-09-13T03:29:15.581 INFO:teuthology.orchestra.run.smithi064.stdout: "crash", 2023-09-13T03:29:15.581 INFO:teuthology.orchestra.run.smithi064.stdout: "devicehealth", 2023-09-13T03:29:15.582 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator", 2023-09-13T03:29:15.582 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_autoscaler", 2023-09-13T03:29:15.582 INFO:teuthology.orchestra.run.smithi064.stdout: "progress", 2023-09-13T03:29:15.583 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_support", 2023-09-13T03:29:15.583 INFO:teuthology.orchestra.run.smithi064.stdout: "status", 2023-09-13T03:29:15.584 INFO:teuthology.orchestra.run.smithi064.stdout: "telemetry", 2023-09-13T03:29:15.585 INFO:teuthology.orchestra.run.smithi064.stdout: "volumes" 2023-09-13T03:29:15.585 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-09-13T03:29:15.585 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.586 INFO:teuthology.orchestra.run.smithi064.stdout: "last_failure_osd_epoch": 86, 2023-09-13T03:29:15.586 INFO:teuthology.orchestra.run.smithi064.stdout: "active_clients": [ 2023-09-13T03:29:15.586 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:15.586 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "devicehealth", 2023-09-13T03:29:15.587 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2023-09-13T03:29:15.587 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:15.587 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2023-09-13T03:29:15.587 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:0", 2023-09-13T03:29:15.588 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 2140779455 2023-09-13T03:29:15.588 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:15.588 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-09-13T03:29:15.589 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.589 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:15.589 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "libcephsqlite", 2023-09-13T03:29:15.589 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2023-09-13T03:29:15.590 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:15.590 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2023-09-13T03:29:15.590 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:0", 2023-09-13T03:29:15.591 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 3693350729 2023-09-13T03:29:15.591 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:15.591 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-09-13T03:29:15.591 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.592 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:15.592 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_support", 2023-09-13T03:29:15.592 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2023-09-13T03:29:15.593 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:15.593 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2023-09-13T03:29:15.593 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:0", 2023-09-13T03:29:15.593 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 3079230338 2023-09-13T03:29:15.594 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:15.594 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-09-13T03:29:15.594 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.595 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:15.595 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snap_schedule", 2023-09-13T03:29:15.595 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2023-09-13T03:29:15.595 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:15.596 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2023-09-13T03:29:15.596 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:0", 2023-09-13T03:29:15.596 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 291324981 2023-09-13T03:29:15.596 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:15.597 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-09-13T03:29:15.597 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.597 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:15.598 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snap_schedule", 2023-09-13T03:29:15.598 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2023-09-13T03:29:15.598 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:15.598 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2023-09-13T03:29:15.599 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:0", 2023-09-13T03:29:15.599 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 2640758781 2023-09-13T03:29:15.599 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:15.600 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-09-13T03:29:15.600 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:15.600 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:15.600 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "volumes", 2023-09-13T03:29:15.601 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2023-09-13T03:29:15.601 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-09-13T03:29:15.601 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2023-09-13T03:29:15.602 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:0", 2023-09-13T03:29:15.602 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 1570621811 2023-09-13T03:29:15.602 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:15.602 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-09-13T03:29:15.603 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:15.603 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-09-13T03:29:15.603 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-13T03:29:15.605 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:10 smithi142 ceph-mon[134082]: pgmap v502: 97 pgs: 10 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 72 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 15 MiB/s wr, 1.05k op/s 2023-09-13T03:29:15.606 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:10 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:29:15.606 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:10 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/804413372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T03:29:15.606 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:10 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/2662790034' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-09-13T03:29:15.607 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:11 smithi142 ceph-mon[134082]: pgmap v503: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 11 MiB/s wr, 665 op/s 2023-09-13T03:29:15.607 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:14 smithi142 ceph-mon[134082]: pgmap v504: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 11 MiB/s wr, 682 op/s 2023-09-13T03:29:15.607 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:14 smithi142 ceph-mon[134082]: osdmap e106: 12 total, 12 up, 12 in 2023-09-13T03:29:15.607 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:10 smithi170 ceph-mon[134328]: pgmap v502: 97 pgs: 10 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 72 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 15 MiB/s wr, 1.05k op/s 2023-09-13T03:29:15.608 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:10 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:29:15.608 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:10 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/804413372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T03:29:15.608 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:10 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/2662790034' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-09-13T03:29:15.609 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:11 smithi170 ceph-mon[134328]: pgmap v503: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 11 MiB/s wr, 665 op/s 2023-09-13T03:29:15.609 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:14 smithi170 ceph-mon[134328]: pgmap v504: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 11 MiB/s wr, 682 op/s 2023-09-13T03:29:15.609 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:14 smithi170 ceph-mon[134328]: osdmap e106: 12 total, 12 up, 12 in 2023-09-13T03:29:15.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.546+0000 7f07a63ff700 1 -- 172.21.15.64:0/2662790034 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f078c0a0870 msgr2=0x7f078c0a2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:15.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.546+0000 7f07a63ff700 1 --2- 172.21.15.64:0/2662790034 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f078c0a0870 0x7f078c0a2d30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0790011490 tx=0x7f0790009260 comp rx=0 tx=0).stop 2023-09-13T03:29:15.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.546+0000 7f07a63ff700 1 -- 172.21.15.64:0/2662790034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07a0100f50 msgr2=0x7f07a0072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:15.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.546+0000 7f07a63ff700 1 --2- 172.21.15.64:0/2662790034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07a0100f50 0x7f07a0072e40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f078800b580 tx=0x7f078800bfd0 comp rx=0 tx=0).stop 2023-09-13T03:29:15.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.546+0000 7f07a63ff700 1 -- 172.21.15.64:0/2662790034 shutdown_connections 2023-09-13T03:29:15.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.546+0000 7f07a63ff700 1 --2- 172.21.15.64:0/2662790034 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f078c0a0870 0x7f078c0a2d30 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:15.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.546+0000 7f07a63ff700 1 --2- 172.21.15.64:0/2662790034 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07a0106650 0x7f07a01a47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:15.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.546+0000 7f07a63ff700 1 --2- 172.21.15.64:0/2662790034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07a0100590 0x7f07a0072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:15.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.546+0000 7f07a63ff700 1 --2- 172.21.15.64:0/2662790034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07a0100f50 0x7f07a0072e40 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:15.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.546+0000 7f07a63ff700 1 -- 172.21.15.64:0/2662790034 >> 172.21.15.64:0/2662790034 conn(0x7f07a00fc020 msgr2=0x7f07a00fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:15.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.546+0000 7f07a63ff700 1 -- 172.21.15.64:0/2662790034 shutdown_connections 2023-09-13T03:29:15.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:10.546+0000 7f07a63ff700 1 -- 172.21.15.64:0/2662790034 wait complete. 2023-09-13T03:29:15.620 DEBUG:tasks.check_counter:Getting stats from a 2023-09-13T03:29:15.620 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph daemon mds.a perf dump 2023-09-13T03:29:15.621 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:10 smithi064 ceph-mon[97675]: pgmap v502: 97 pgs: 10 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 72 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 15 MiB/s wr, 1.05k op/s 2023-09-13T03:29:15.622 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:10 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:29:15.622 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:10 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/804413372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T03:29:15.622 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:10 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2662790034' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-09-13T03:29:15.623 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:11 smithi064 ceph-mon[97675]: pgmap v503: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 11 MiB/s wr, 665 op/s 2023-09-13T03:29:15.623 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:14 smithi064 ceph-mon[97675]: pgmap v504: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 11 MiB/s wr, 682 op/s 2023-09-13T03:29:15.623 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:14 smithi064 ceph-mon[97675]: osdmap e106: 12 total, 12 up, 12 in 2023-09-13T03:29:16.382 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:29:16.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:16 smithi064 ceph-mon[97675]: pgmap v506: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 9.7 MiB/s wr, 483 op/s 2023-09-13T03:29:17.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:16 smithi170 ceph-mon[134328]: pgmap v506: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 9.7 MiB/s wr, 483 op/s 2023-09-13T03:29:17.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:16 smithi142 ceph-mon[134082]: pgmap v506: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 9.7 MiB/s wr, 483 op/s 2023-09-13T03:29:18.133 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-13T03:29:18.133 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-0": { 2023-09-13T03:29:18.134 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 2106, 2023-09-13T03:29:18.134 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 2106, 2023-09-13T03:29:18.134 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 55152991, 2023-09-13T03:29:18.134 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 654171, 2023-09-13T03:29:18.135 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 8, 2023-09-13T03:29:18.135 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 4, 2023-09-13T03:29:18.135 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.329023474, 2023-09-13T03:29:18.135 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.077117877, 2023-09-13T03:29:18.136 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.156914969, 2023-09-13T03:29:18.136 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.067738501, 2023-09-13T03:29:18.136 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T03:29:18.137 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 2105, 2023-09-13T03:29:18.137 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.113735693, 2023-09-13T03:29:18.137 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000054031 2023-09-13T03:29:18.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.138 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2023-09-13T03:29:18.138 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:18.138 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:18.139 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:18.139 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.140 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-09-13T03:29:18.140 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 448 2023-09-13T03:29:18.140 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.140 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-1": { 2023-09-13T03:29:18.141 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 1597, 2023-09-13T03:29:18.141 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 1530, 2023-09-13T03:29:18.141 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 102952815, 2023-09-13T03:29:18.141 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 557260, 2023-09-13T03:29:18.142 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 6, 2023-09-13T03:29:18.142 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 5, 2023-09-13T03:29:18.142 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.353171427, 2023-09-13T03:29:18.143 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.064907437, 2023-09-13T03:29:18.143 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.239060467, 2023-09-13T03:29:18.143 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.046822789, 2023-09-13T03:29:18.144 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T03:29:18.144 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 1529, 2023-09-13T03:29:18.144 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.091953963, 2023-09-13T03:29:18.145 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000060139 2023-09-13T03:29:18.145 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.145 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2023-09-13T03:29:18.145 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:18.146 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:18.146 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:18.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.146 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 1967760, 2023-09-13T03:29:18.147 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 181504 2023-09-13T03:29:18.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.147 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-2": { 2023-09-13T03:29:18.148 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 520, 2023-09-13T03:29:18.148 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 741, 2023-09-13T03:29:18.148 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 51643674, 2023-09-13T03:29:18.148 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 709898, 2023-09-13T03:29:18.149 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 9, 2023-09-13T03:29:18.149 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 6, 2023-09-13T03:29:18.149 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.144509819, 2023-09-13T03:29:18.149 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.036752066, 2023-09-13T03:29:18.150 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.083015407, 2023-09-13T03:29:18.150 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.014972565, 2023-09-13T03:29:18.151 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T03:29:18.151 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 738, 2023-09-13T03:29:18.151 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.056802212, 2023-09-13T03:29:18.152 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000076967 2023-09-13T03:29:18.152 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.152 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2023-09-13T03:29:18.153 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:18.153 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:18.153 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:18.153 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.154 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 320, 2023-09-13T03:29:18.154 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 548800 2023-09-13T03:29:18.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.155 INFO:teuthology.orchestra.run.smithi064.stdout: "cct": { 2023-09-13T03:29:18.155 INFO:teuthology.orchestra.run.smithi064.stdout: "total_workers": 1, 2023-09-13T03:29:18.155 INFO:teuthology.orchestra.run.smithi064.stdout: "unhealthy_workers": 0 2023-09-13T03:29:18.155 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.156 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-MDSRank": { 2023-09-13T03:29:18.156 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2023-09-13T03:29:18.156 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2023-09-13T03:29:18.156 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 6291, 2023-09-13T03:29:18.157 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.196961659, 2023-09-13T03:29:18.157 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000031308 2023-09-13T03:29:18.157 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:18.157 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.158 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-PurgeQueue": { 2023-09-13T03:29:18.158 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2023-09-13T03:29:18.158 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2023-09-13T03:29:18.159 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:18.159 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:18.159 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:18.159 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:18.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.160 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": { 2023-09-13T03:29:18.160 INFO:teuthology.orchestra.run.smithi064.stdout: "request": 0, 2023-09-13T03:29:18.161 INFO:teuthology.orchestra.run.smithi064.stdout: "reply": 0, 2023-09-13T03:29:18.161 INFO:teuthology.orchestra.run.smithi064.stdout: "reply_latency": { 2023-09-13T03:29:18.161 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:18.161 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:18.162 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:18.162 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.162 INFO:teuthology.orchestra.run.smithi064.stdout: "slow_reply": 0, 2023-09-13T03:29:18.163 INFO:teuthology.orchestra.run.smithi064.stdout: "forward": 0, 2023-09-13T03:29:18.163 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_fetch_complete": 0, 2023-09-13T03:29:18.163 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_fetch_keys": 0, 2023-09-13T03:29:18.163 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_commit": 0, 2023-09-13T03:29:18.164 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_split": 0, 2023-09-13T03:29:18.164 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_merge": 0, 2023-09-13T03:29:18.164 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes": 12575, 2023-09-13T03:29:18.164 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_top": 35, 2023-09-13T03:29:18.165 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_bottom": 11526, 2023-09-13T03:29:18.165 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_pin_tail": 1014, 2023-09-13T03:29:18.165 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_pinned": 12526, 2023-09-13T03:29:18.166 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_expired": 1, 2023-09-13T03:29:18.166 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_with_caps": 0, 2023-09-13T03:29:18.166 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": 0, 2023-09-13T03:29:18.166 INFO:teuthology.orchestra.run.smithi064.stdout: "subtrees": 6, 2023-09-13T03:29:18.167 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse": 0, 2023-09-13T03:29:18.167 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_hit": 0, 2023-09-13T03:29:18.167 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_forward": 0, 2023-09-13T03:29:18.167 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_discover": 0, 2023-09-13T03:29:18.168 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_dir_fetch": 0, 2023-09-13T03:29:18.168 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_remote_ino": 0, 2023-09-13T03:29:18.168 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_lock": 0, 2023-09-13T03:29:18.169 INFO:teuthology.orchestra.run.smithi064.stdout: "load_cent": 0, 2023-09-13T03:29:18.169 INFO:teuthology.orchestra.run.smithi064.stdout: "q": 0, 2023-09-13T03:29:18.169 INFO:teuthology.orchestra.run.smithi064.stdout: "exported": 0, 2023-09-13T03:29:18.170 INFO:teuthology.orchestra.run.smithi064.stdout: "exported_inodes": 0, 2023-09-13T03:29:18.170 INFO:teuthology.orchestra.run.smithi064.stdout: "imported": 0, 2023-09-13T03:29:18.170 INFO:teuthology.orchestra.run.smithi064.stdout: "imported_inodes": 0, 2023-09-13T03:29:18.170 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_dir_fetch": 0, 2023-09-13T03:29:18.171 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_backtrace_fetch": 0, 2023-09-13T03:29:18.171 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_peer_discover": 0, 2023-09-13T03:29:18.171 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rfiles": 0, 2023-09-13T03:29:18.171 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rbytes": 0, 2023-09-13T03:29:18.172 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rsnaps": 0, 2023-09-13T03:29:18.172 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_backtrace_fetch": 0, 2023-09-13T03:29:18.172 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_set_tag": 0, 2023-09-13T03:29:18.173 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_backtrace_repaired": 0, 2023-09-13T03:29:18.173 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_inotable_repaired": 0, 2023-09-13T03:29:18.173 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dir_inodes": 0, 2023-09-13T03:29:18.173 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dir_base_inodes": 0, 2023-09-13T03:29:18.174 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dirfrag_rstats": 0, 2023-09-13T03:29:18.174 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_file_inodes": 0, 2023-09-13T03:29:18.174 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_inode_file_caps": 0, 2023-09-13T03:29:18.175 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_revoke": 0, 2023-09-13T03:29:18.175 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_grant": 0, 2023-09-13T03:29:18.175 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_trunc": 0, 2023-09-13T03:29:18.175 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-13T03:29:18.176 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-13T03:29:18.176 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_caps": 0, 2023-09-13T03:29:18.176 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_caps_dirty": 0, 2023-09-13T03:29:18.176 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_cap_release": 0, 2023-09-13T03:29:18.177 INFO:teuthology.orchestra.run.smithi064.stdout: "process_request_cap_release": 0 2023-09-13T03:29:18.177 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.177 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_cache": { 2023-09-13T03:29:18.178 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_update": 0, 2023-09-13T03:29:18.178 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_update_receipt": 0, 2023-09-13T03:29:18.178 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_try_discover": 0, 2023-09-13T03:29:18.178 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_send_discover": 0, 2023-09-13T03:29:18.179 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_handle_discover": 0, 2023-09-13T03:29:18.179 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays": 0, 2023-09-13T03:29:18.179 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays_delayed": 0, 2023-09-13T03:29:18.180 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays_enqueuing": 0, 2023-09-13T03:29:18.180 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_created": 0, 2023-09-13T03:29:18.180 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_enqueued": 0, 2023-09-13T03:29:18.180 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_reintegrated": 0, 2023-09-13T03:29:18.181 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_migrated": 0, 2023-09-13T03:29:18.181 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_processing": 0, 2023-09-13T03:29:18.181 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_enqueued": 0, 2023-09-13T03:29:18.181 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_prioritized": 0, 2023-09-13T03:29:18.182 INFO:teuthology.orchestra.run.smithi064.stdout: "recovery_started": 0, 2023-09-13T03:29:18.182 INFO:teuthology.orchestra.run.smithi064.stdout: "recovery_completed": 0, 2023-09-13T03:29:18.182 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_enqueue_scrub": 0, 2023-09-13T03:29:18.183 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_exportdir": 0, 2023-09-13T03:29:18.183 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_flush": 0, 2023-09-13T03:29:18.183 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_fragmentdir": 0, 2023-09-13T03:29:18.184 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_fragstats": 0, 2023-09-13T03:29:18.184 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_inodestats": 0 2023-09-13T03:29:18.184 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.184 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_log": { 2023-09-13T03:29:18.185 INFO:teuthology.orchestra.run.smithi064.stdout: "evlrg": 0, 2023-09-13T03:29:18.185 INFO:teuthology.orchestra.run.smithi064.stdout: "evadd": 0, 2023-09-13T03:29:18.185 INFO:teuthology.orchestra.run.smithi064.stdout: "evex": 0, 2023-09-13T03:29:18.186 INFO:teuthology.orchestra.run.smithi064.stdout: "evtrm": 0, 2023-09-13T03:29:18.186 INFO:teuthology.orchestra.run.smithi064.stdout: "ev": 60158, 2023-09-13T03:29:18.187 INFO:teuthology.orchestra.run.smithi064.stdout: "evexg": 0, 2023-09-13T03:29:18.187 INFO:teuthology.orchestra.run.smithi064.stdout: "evexd": 0, 2023-09-13T03:29:18.188 INFO:teuthology.orchestra.run.smithi064.stdout: "segadd": 0, 2023-09-13T03:29:18.188 INFO:teuthology.orchestra.run.smithi064.stdout: "segex": 0, 2023-09-13T03:29:18.188 INFO:teuthology.orchestra.run.smithi064.stdout: "segtrm": 0, 2023-09-13T03:29:18.189 INFO:teuthology.orchestra.run.smithi064.stdout: "seg": 83, 2023-09-13T03:29:18.189 INFO:teuthology.orchestra.run.smithi064.stdout: "segmjr": 6, 2023-09-13T03:29:18.190 INFO:teuthology.orchestra.run.smithi064.stdout: "segexg": 0, 2023-09-13T03:29:18.190 INFO:teuthology.orchestra.run.smithi064.stdout: "segexd": 0, 2023-09-13T03:29:18.191 INFO:teuthology.orchestra.run.smithi064.stdout: "expos": 4194304, 2023-09-13T03:29:18.191 INFO:teuthology.orchestra.run.smithi064.stdout: "wrpos": 210895296, 2023-09-13T03:29:18.192 INFO:teuthology.orchestra.run.smithi064.stdout: "rdpos": 210887381, 2023-09-13T03:29:18.192 INFO:teuthology.orchestra.run.smithi064.stdout: "jlat": { 2023-09-13T03:29:18.192 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:18.193 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:18.193 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:18.193 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.194 INFO:teuthology.orchestra.run.smithi064.stdout: "replayed": 60158 2023-09-13T03:29:18.194 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.194 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_mem": { 2023-09-13T03:29:18.195 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 1225, 2023-09-13T03:29:18.195 INFO:teuthology.orchestra.run.smithi064.stdout: "ino+": 12721, 2023-09-13T03:29:18.195 INFO:teuthology.orchestra.run.smithi064.stdout: "ino-": 11496, 2023-09-13T03:29:18.195 INFO:teuthology.orchestra.run.smithi064.stdout: "dir": 83, 2023-09-13T03:29:18.196 INFO:teuthology.orchestra.run.smithi064.stdout: "dir+": 146, 2023-09-13T03:29:18.196 INFO:teuthology.orchestra.run.smithi064.stdout: "dir-": 63, 2023-09-13T03:29:18.196 INFO:teuthology.orchestra.run.smithi064.stdout: "dn": 12575, 2023-09-13T03:29:18.196 INFO:teuthology.orchestra.run.smithi064.stdout: "dn+": 12924, 2023-09-13T03:29:18.197 INFO:teuthology.orchestra.run.smithi064.stdout: "dn-": 349, 2023-09-13T03:29:18.197 INFO:teuthology.orchestra.run.smithi064.stdout: "cap": 0, 2023-09-13T03:29:18.197 INFO:teuthology.orchestra.run.smithi064.stdout: "cap+": 0, 2023-09-13T03:29:18.198 INFO:teuthology.orchestra.run.smithi064.stdout: "cap-": 0, 2023-09-13T03:29:18.198 INFO:teuthology.orchestra.run.smithi064.stdout: "rss": 68508, 2023-09-13T03:29:18.198 INFO:teuthology.orchestra.run.smithi064.stdout: "heap": 207152 2023-09-13T03:29:18.198 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.199 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_server": { 2023-09-13T03:29:18.199 INFO:teuthology.orchestra.run.smithi064.stdout: "dispatch_client_request": 0, 2023-09-13T03:29:18.199 INFO:teuthology.orchestra.run.smithi064.stdout: "dispatch_server_request": 0, 2023-09-13T03:29:18.200 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_request": 0, 2023-09-13T03:29:18.200 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_session": 0, 2023-09-13T03:29:18.200 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_peer_request": 0, 2023-09-13T03:29:18.200 INFO:teuthology.orchestra.run.smithi064.stdout: "req_create_latency": { 2023-09-13T03:29:18.201 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:18.201 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:18.201 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:18.202 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.202 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getattr_latency": { 2023-09-13T03:29:18.202 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:18.202 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:18.203 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:18.203 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.203 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getfilelock_latency": { 2023-09-13T03:29:18.204 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:18.204 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:18.204 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:18.204 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.205 INFO:teuthology.orchestra.run.smithi064.stdout: "req_link_latency": { 2023-09-13T03:29:18.205 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:18.205 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:18.205 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:18.206 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.206 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookup_latency": { 2023-09-13T03:29:18.206 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:18.207 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:18.207 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:18.207 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.207 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookuphash_latency": { 2023-09-13T03:29:18.208 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:18.208 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:18.208 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:18.208 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.209 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupino_latency": { 2023-09-13T03:29:18.209 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:18.209 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:18.210 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:18.210 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.210 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupname_latency": { 2023-09-13T03:29:18.210 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:18.211 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:18.211 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:18.211 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.211 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupparent_latency": { 2023-09-13T03:29:18.212 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:18.212 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:18.212 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:18.212 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.213 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupsnap_latency": { 2023-09-13T03:29:18.213 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:18.213 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:18.213 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:18.214 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.214 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lssnap_latency": { 2023-09-13T03:29:18.214 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:18.215 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:18.215 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:18.215 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.215 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mkdir_latency": { 2023-09-13T03:29:18.216 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:18.216 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:18.216 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:18.216 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.217 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mknod_latency": { 2023-09-13T03:29:18.217 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:18.217 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:18.217 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:18.218 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.218 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mksnap_latency": { 2023-09-13T03:29:18.218 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:18.219 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:18.219 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:18.219 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.219 INFO:teuthology.orchestra.run.smithi064.stdout: "req_open_latency": { 2023-09-13T03:29:18.219 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:18.220 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:18.220 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:18.220 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.220 INFO:teuthology.orchestra.run.smithi064.stdout: "req_readdir_latency": { 2023-09-13T03:29:18.221 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:18.221 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:18.221 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:18.221 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.222 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rename_latency": { 2023-09-13T03:29:18.222 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:18.223 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:18.223 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:18.224 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.224 INFO:teuthology.orchestra.run.smithi064.stdout: "req_renamesnap_latency": { 2023-09-13T03:29:18.224 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:18.225 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:18.225 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:18.225 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.225 INFO:teuthology.orchestra.run.smithi064.stdout: "req_snapdiff_latency": { 2023-09-13T03:29:18.226 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:18.226 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:18.227 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:18.227 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.227 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmdir_latency": { 2023-09-13T03:29:18.228 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:18.228 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:18.228 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:18.228 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.229 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmsnap_latency": { 2023-09-13T03:29:18.229 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:18.229 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:18.230 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:18.230 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.231 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmxattr_latency": { 2023-09-13T03:29:18.231 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:18.232 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:18.232 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:18.233 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.233 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setattr_latency": { 2023-09-13T03:29:18.234 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:18.234 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:18.235 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:18.235 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.235 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setdirlayout_latency": { 2023-09-13T03:29:18.235 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:18.236 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:18.236 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:18.236 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.236 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setfilelock_latency": { 2023-09-13T03:29:18.237 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:18.237 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:18.237 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:18.237 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.238 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setlayout_latency": { 2023-09-13T03:29:18.238 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:18.238 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:18.239 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:18.239 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.239 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setxattr_latency": { 2023-09-13T03:29:18.240 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:18.240 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:18.240 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:18.240 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.241 INFO:teuthology.orchestra.run.smithi064.stdout: "req_symlink_latency": { 2023-09-13T03:29:18.241 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:18.241 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:18.241 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:18.242 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.242 INFO:teuthology.orchestra.run.smithi064.stdout: "req_unlink_latency": { 2023-09-13T03:29:18.242 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:18.243 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:18.243 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:18.243 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.243 INFO:teuthology.orchestra.run.smithi064.stdout: "cap_revoke_eviction": 0, 2023-09-13T03:29:18.244 INFO:teuthology.orchestra.run.smithi064.stdout: "cap_acquisition_throttle": 0, 2023-09-13T03:29:18.244 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getvxattr_latency": { 2023-09-13T03:29:18.244 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:18.245 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:18.245 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:18.245 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:18.245 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.246 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_sessions": { 2023-09-13T03:29:18.246 INFO:teuthology.orchestra.run.smithi064.stdout: "session_count": 2, 2023-09-13T03:29:18.246 INFO:teuthology.orchestra.run.smithi064.stdout: "session_add": 4, 2023-09-13T03:29:18.247 INFO:teuthology.orchestra.run.smithi064.stdout: "session_remove": 2, 2023-09-13T03:29:18.247 INFO:teuthology.orchestra.run.smithi064.stdout: "sessions_open": 4, 2023-09-13T03:29:18.247 INFO:teuthology.orchestra.run.smithi064.stdout: "sessions_stale": 0, 2023-09-13T03:29:18.247 INFO:teuthology.orchestra.run.smithi064.stdout: "total_load": 0, 2023-09-13T03:29:18.248 INFO:teuthology.orchestra.run.smithi064.stdout: "average_load": 0, 2023-09-13T03:29:18.248 INFO:teuthology.orchestra.run.smithi064.stdout: "avg_session_uptime": 4121, 2023-09-13T03:29:18.248 INFO:teuthology.orchestra.run.smithi064.stdout: "mdthresh_evicted": 0 2023-09-13T03:29:18.248 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.249 INFO:teuthology.orchestra.run.smithi064.stdout: "mempool": { 2023-09-13T03:29:18.249 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_bytes": 0, 2023-09-13T03:29:18.249 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_items": 0, 2023-09-13T03:29:18.249 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_bytes": 0, 2023-09-13T03:29:18.250 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_items": 0, 2023-09-13T03:29:18.250 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_bytes": 0, 2023-09-13T03:29:18.250 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_items": 0, 2023-09-13T03:29:18.251 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-13T03:29:18.251 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_items": 0, 2023-09-13T03:29:18.253 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-13T03:29:18.253 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_items": 0, 2023-09-13T03:29:18.253 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_bytes": 0, 2023-09-13T03:29:18.254 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_items": 0, 2023-09-13T03:29:18.254 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-13T03:29:18.254 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_items": 0, 2023-09-13T03:29:18.255 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_bytes": 0, 2023-09-13T03:29:18.255 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_items": 0, 2023-09-13T03:29:18.255 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_bytes": 0, 2023-09-13T03:29:18.255 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_items": 0, 2023-09-13T03:29:18.256 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-13T03:29:18.256 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_items": 0, 2023-09-13T03:29:18.256 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-13T03:29:18.257 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_items": 0, 2023-09-13T03:29:18.257 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_bytes": 0, 2023-09-13T03:29:18.257 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_items": 0, 2023-09-13T03:29:18.257 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_bytes": 0, 2023-09-13T03:29:18.258 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_items": 0, 2023-09-13T03:29:18.258 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-13T03:29:18.258 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_items": 0, 2023-09-13T03:29:18.259 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_bytes": 0, 2023-09-13T03:29:18.259 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_items": 0, 2023-09-13T03:29:18.259 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_bytes": 0, 2023-09-13T03:29:18.259 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_items": 0, 2023-09-13T03:29:18.260 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_bytes": 0, 2023-09-13T03:29:18.260 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_items": 0, 2023-09-13T03:29:18.260 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_bytes": 0, 2023-09-13T03:29:18.260 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_items": 0, 2023-09-13T03:29:18.261 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_bytes": 284697, 2023-09-13T03:29:18.261 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_items": 1389, 2023-09-13T03:29:18.261 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_bytes": 0, 2023-09-13T03:29:18.262 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_items": 0, 2023-09-13T03:29:18.262 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_bytes": 0, 2023-09-13T03:29:18.262 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_items": 0, 2023-09-13T03:29:18.263 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_bytes": 0, 2023-09-13T03:29:18.263 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_items": 0, 2023-09-13T03:29:18.263 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_bytes": 0, 2023-09-13T03:29:18.264 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_items": 0, 2023-09-13T03:29:18.264 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_bytes": 7936, 2023-09-13T03:29:18.264 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_items": 165, 2023-09-13T03:29:18.264 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_bytes": 0, 2023-09-13T03:29:18.265 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_items": 0, 2023-09-13T03:29:18.265 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_bytes": 0, 2023-09-13T03:29:18.265 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_items": 0, 2023-09-13T03:29:18.265 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_bytes": 11876879, 2023-09-13T03:29:18.266 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_items": 162742, 2023-09-13T03:29:18.266 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_bytes": 0, 2023-09-13T03:29:18.266 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_items": 0, 2023-09-13T03:29:18.267 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_bytes": 0, 2023-09-13T03:29:18.267 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_items": 0 2023-09-13T03:29:18.267 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.267 INFO:teuthology.orchestra.run.smithi064.stdout: "objecter": { 2023-09-13T03:29:18.268 INFO:teuthology.orchestra.run.smithi064.stdout: "op_active": 0, 2023-09-13T03:29:18.268 INFO:teuthology.orchestra.run.smithi064.stdout: "op_laggy": 0, 2023-09-13T03:29:18.268 INFO:teuthology.orchestra.run.smithi064.stdout: "op_send": 3828, 2023-09-13T03:29:18.268 INFO:teuthology.orchestra.run.smithi064.stdout: "op_send_bytes": 21, 2023-09-13T03:29:18.269 INFO:teuthology.orchestra.run.smithi064.stdout: "op_resend": 3, 2023-09-13T03:29:18.269 INFO:teuthology.orchestra.run.smithi064.stdout: "op_reply": 3828, 2023-09-13T03:29:18.269 INFO:teuthology.orchestra.run.smithi064.stdout: "op_latency": { 2023-09-13T03:29:18.270 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 3828, 2023-09-13T03:29:18.270 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 4.990903190, 2023-09-13T03:29:18.270 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.001303788 2023-09-13T03:29:18.271 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.271 INFO:teuthology.orchestra.run.smithi064.stdout: "op_inflight": 0, 2023-09-13T03:29:18.271 INFO:teuthology.orchestra.run.smithi064.stdout: "oplen_avg": { 2023-09-13T03:29:18.272 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 3828, 2023-09-13T03:29:18.272 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 3829 2023-09-13T03:29:18.272 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.272 INFO:teuthology.orchestra.run.smithi064.stdout: "op": 3828, 2023-09-13T03:29:18.273 INFO:teuthology.orchestra.run.smithi064.stdout: "op_r": 3828, 2023-09-13T03:29:18.273 INFO:teuthology.orchestra.run.smithi064.stdout: "op_w": 0, 2023-09-13T03:29:18.273 INFO:teuthology.orchestra.run.smithi064.stdout: "op_rmw": 0, 2023-09-13T03:29:18.274 INFO:teuthology.orchestra.run.smithi064.stdout: "op_pg": 0, 2023-09-13T03:29:18.274 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_stat": 2200, 2023-09-13T03:29:18.274 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_create": 0, 2023-09-13T03:29:18.275 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_read": 1626, 2023-09-13T03:29:18.275 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_write": 0, 2023-09-13T03:29:18.275 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_writefull": 0, 2023-09-13T03:29:18.275 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_writesame": 0, 2023-09-13T03:29:18.276 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_append": 0, 2023-09-13T03:29:18.276 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_zero": 0, 2023-09-13T03:29:18.276 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_truncate": 0, 2023-09-13T03:29:18.276 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_delete": 0, 2023-09-13T03:29:18.277 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_mapext": 0, 2023-09-13T03:29:18.277 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_sparse_read": 0, 2023-09-13T03:29:18.277 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_clonerange": 0, 2023-09-13T03:29:18.278 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_getxattr": 1, 2023-09-13T03:29:18.278 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_setxattr": 0, 2023-09-13T03:29:18.278 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_cmpxattr": 0, 2023-09-13T03:29:18.279 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_rmxattr": 0, 2023-09-13T03:29:18.279 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_resetxattrs": 0, 2023-09-13T03:29:18.279 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_call": 0, 2023-09-13T03:29:18.279 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_watch": 0, 2023-09-13T03:29:18.280 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_notify": 0, 2023-09-13T03:29:18.280 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_src_cmpxattr": 0, 2023-09-13T03:29:18.280 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_pgls": 0, 2023-09-13T03:29:18.281 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_pgls_filter": 0, 2023-09-13T03:29:18.281 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_other": 0, 2023-09-13T03:29:18.281 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_active": 0, 2023-09-13T03:29:18.281 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_send": 0, 2023-09-13T03:29:18.282 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_resend": 0, 2023-09-13T03:29:18.282 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_ping": 0, 2023-09-13T03:29:18.282 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_active": 0, 2023-09-13T03:29:18.283 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_send": 0, 2023-09-13T03:29:18.283 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_resend": 0, 2023-09-13T03:29:18.283 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_active": 0, 2023-09-13T03:29:18.284 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_send": 0, 2023-09-13T03:29:18.284 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_resend": 0, 2023-09-13T03:29:18.284 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_active": 0, 2023-09-13T03:29:18.284 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_send": 0, 2023-09-13T03:29:18.285 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_resend": 0, 2023-09-13T03:29:18.285 INFO:teuthology.orchestra.run.smithi064.stdout: "command_active": 0, 2023-09-13T03:29:18.285 INFO:teuthology.orchestra.run.smithi064.stdout: "command_send": 0, 2023-09-13T03:29:18.285 INFO:teuthology.orchestra.run.smithi064.stdout: "command_resend": 0, 2023-09-13T03:29:18.285 INFO:teuthology.orchestra.run.smithi064.stdout: "map_epoch": 106, 2023-09-13T03:29:18.286 INFO:teuthology.orchestra.run.smithi064.stdout: "map_full": 0, 2023-09-13T03:29:18.286 INFO:teuthology.orchestra.run.smithi064.stdout: "map_inc": 23, 2023-09-13T03:29:18.286 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_sessions": 11, 2023-09-13T03:29:18.287 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_session_open": 12, 2023-09-13T03:29:18.287 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_session_close": 1, 2023-09-13T03:29:18.287 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_laggy": 0, 2023-09-13T03:29:18.287 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_wr": 0, 2023-09-13T03:29:18.288 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_rd": 2, 2023-09-13T03:29:18.288 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_del": 0 2023-09-13T03:29:18.288 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.288 INFO:teuthology.orchestra.run.smithi064.stdout: "oft": { 2023-09-13T03:29:18.289 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_objs": 0, 2023-09-13T03:29:18.289 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_kv_pairs": 0, 2023-09-13T03:29:18.289 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_updates": 0, 2023-09-13T03:29:18.289 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_removes": 0 2023-09-13T03:29:18.290 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.290 INFO:teuthology.orchestra.run.smithi064.stdout: "osd-slow-ops": { 2023-09-13T03:29:18.290 INFO:teuthology.orchestra.run.smithi064.stdout: "slow_ops_count": 0 2023-09-13T03:29:18.291 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.291 INFO:teuthology.orchestra.run.smithi064.stdout: "purge_queue": { 2023-09-13T03:29:18.291 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_ops": 0, 2023-09-13T03:29:18.291 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_ops_high_water": 0, 2023-09-13T03:29:18.292 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing": 0, 2023-09-13T03:29:18.292 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_high_water": 0, 2023-09-13T03:29:18.292 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executed": 0, 2023-09-13T03:29:18.292 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_item_in_journal": 0 2023-09-13T03:29:18.293 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.293 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-13T03:29:18.293 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-09-13T03:29:18.293 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2023-09-13T03:29:18.294 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-09-13T03:29:18.294 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 4219, 2023-09-13T03:29:18.294 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 209347375, 2023-09-13T03:29:18.295 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:18.295 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 4219, 2023-09-13T03:29:18.295 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2023-09-13T03:29:18.295 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2023-09-13T03:29:18.296 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 4219, 2023-09-13T03:29:18.296 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 209347375, 2023-09-13T03:29:18.296 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-09-13T03:29:18.296 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:18.297 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:18.297 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:18.297 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:18.297 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.298 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-objecter_bytes": { 2023-09-13T03:29:18.298 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-09-13T03:29:18.299 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2023-09-13T03:29:18.299 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-09-13T03:29:18.299 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2023-09-13T03:29:18.299 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2023-09-13T03:29:18.300 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:18.300 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2023-09-13T03:29:18.300 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 3828, 2023-09-13T03:29:18.301 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 206717432, 2023-09-13T03:29:18.301 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 3828, 2023-09-13T03:29:18.301 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 206717432, 2023-09-13T03:29:18.301 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-09-13T03:29:18.302 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:18.302 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:18.302 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:18.303 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:18.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.303 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-objecter_ops": { 2023-09-13T03:29:18.303 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-09-13T03:29:18.304 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 1024, 2023-09-13T03:29:18.304 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-09-13T03:29:18.304 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2023-09-13T03:29:18.305 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2023-09-13T03:29:18.305 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:18.305 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2023-09-13T03:29:18.305 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 3828, 2023-09-13T03:29:18.306 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 3828, 2023-09-13T03:29:18.306 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 3828, 2023-09-13T03:29:18.306 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 3828, 2023-09-13T03:29:18.307 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-09-13T03:29:18.307 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:18.307 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:18.308 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:18.308 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:18.308 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.308 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-write_buf_throttle": { 2023-09-13T03:29:18.309 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-09-13T03:29:18.309 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 3758096384, 2023-09-13T03:29:18.309 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-09-13T03:29:18.310 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2023-09-13T03:29:18.310 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2023-09-13T03:29:18.310 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:18.311 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2023-09-13T03:29:18.311 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2023-09-13T03:29:18.311 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2023-09-13T03:29:18.311 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 0, 2023-09-13T03:29:18.312 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 0, 2023-09-13T03:29:18.312 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-09-13T03:29:18.312 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:18.312 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:18.313 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:18.313 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:18.313 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:18.313 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-write_buf_throttle-0x5608efa20a00": { 2023-09-13T03:29:18.314 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-09-13T03:29:18.314 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 3758096384, 2023-09-13T03:29:18.314 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-09-13T03:29:18.315 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2023-09-13T03:29:18.315 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2023-09-13T03:29:18.315 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:18.315 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2023-09-13T03:29:18.316 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2023-09-13T03:29:18.316 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2023-09-13T03:29:18.316 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 0, 2023-09-13T03:29:18.316 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 0, 2023-09-13T03:29:18.317 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-09-13T03:29:18.317 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:18.317 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:18.318 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:18.318 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:18.318 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:18.318 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-13T03:29:18.603 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2023-09-13T03:29:18.603 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2023-09-13T03:29:18.603 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2023-09-13T03:29:18.604 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2023-09-13T03:29:18.604 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2023-09-13T03:29:18.604 DEBUG:tasks.check_counter:Getting stats from d 2023-09-13T03:29:18.605 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph daemon mds.d perf dump 2023-09-13T03:29:19.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:18 smithi170 ceph-mon[134328]: pgmap v507: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 8.7 MiB/s wr, 301 op/s 2023-09-13T03:29:19.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:18 smithi064 ceph-mon[97675]: pgmap v507: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 8.7 MiB/s wr, 301 op/s 2023-09-13T03:29:19.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:18 smithi142 ceph-mon[134082]: pgmap v507: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 8.7 MiB/s wr, 301 op/s 2023-09-13T03:29:19.340 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:29:20.755 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-13T03:29:20.756 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-0": { 2023-09-13T03:29:20.756 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 0, 2023-09-13T03:29:20.756 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 2, 2023-09-13T03:29:20.756 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 0, 2023-09-13T03:29:20.757 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 0, 2023-09-13T03:29:20.757 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 4, 2023-09-13T03:29:20.757 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 0, 2023-09-13T03:29:20.758 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.004137478, 2023-09-13T03:29:20.758 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.000000000, 2023-09-13T03:29:20.758 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.001340483, 2023-09-13T03:29:20.759 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-13T03:29:20.759 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T03:29:20.759 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:20.760 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:20.760 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:20.760 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:20.760 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2023-09-13T03:29:20.761 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:20.761 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:20.761 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:20.761 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:20.762 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-13T03:29:20.762 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 0 2023-09-13T03:29:20.762 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:20.763 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-1": { 2023-09-13T03:29:20.763 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 310, 2023-09-13T03:29:20.763 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 299, 2023-09-13T03:29:20.764 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 1577728, 2023-09-13T03:29:20.764 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 179744, 2023-09-13T03:29:20.764 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 2, 2023-09-13T03:29:20.765 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 1, 2023-09-13T03:29:20.765 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.064453413, 2023-09-13T03:29:20.765 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.023916941, 2023-09-13T03:29:20.765 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.043317834, 2023-09-13T03:29:20.766 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.006666039, 2023-09-13T03:29:20.766 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T03:29:20.766 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 298, 2023-09-13T03:29:20.767 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.021047152, 2023-09-13T03:29:20.767 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000070628 2023-09-13T03:29:20.767 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:20.767 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2023-09-13T03:29:20.768 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:20.768 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:20.768 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:20.769 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:20.769 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 1577728, 2023-09-13T03:29:20.769 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 179744 2023-09-13T03:29:20.770 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:20.770 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-2": { 2023-09-13T03:29:20.770 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 7, 2023-09-13T03:29:20.771 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 231, 2023-09-13T03:29:20.771 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 3424, 2023-09-13T03:29:20.771 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 269904, 2023-09-13T03:29:20.771 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 6, 2023-09-13T03:29:20.772 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 3, 2023-09-13T03:29:20.772 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.027200767, 2023-09-13T03:29:20.772 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.018568432, 2023-09-13T03:29:20.772 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.004092420, 2023-09-13T03:29:20.773 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-13T03:29:20.773 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T03:29:20.773 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 229, 2023-09-13T03:29:20.773 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.029457362, 2023-09-13T03:29:20.774 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000128634 2023-09-13T03:29:20.774 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:20.774 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2023-09-13T03:29:20.775 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:20.775 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:20.775 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:20.776 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:20.776 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 3424, 2023-09-13T03:29:20.776 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 269904 2023-09-13T03:29:20.776 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:20.777 INFO:teuthology.orchestra.run.smithi064.stdout: "cct": { 2023-09-13T03:29:20.777 INFO:teuthology.orchestra.run.smithi064.stdout: "total_workers": 0, 2023-09-13T03:29:20.777 INFO:teuthology.orchestra.run.smithi064.stdout: "unhealthy_workers": 0 2023-09-13T03:29:20.777 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:20.778 INFO:teuthology.orchestra.run.smithi064.stdout: "mempool": { 2023-09-13T03:29:20.778 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_bytes": 0, 2023-09-13T03:29:20.778 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_items": 0, 2023-09-13T03:29:20.778 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_bytes": 0, 2023-09-13T03:29:20.779 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_items": 0, 2023-09-13T03:29:20.779 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_bytes": 0, 2023-09-13T03:29:20.779 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_items": 0, 2023-09-13T03:29:20.780 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-13T03:29:20.780 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_items": 0, 2023-09-13T03:29:20.780 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-13T03:29:20.781 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_items": 0, 2023-09-13T03:29:20.781 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_bytes": 0, 2023-09-13T03:29:20.781 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_items": 0, 2023-09-13T03:29:20.782 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-13T03:29:20.782 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_items": 0, 2023-09-13T03:29:20.782 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_bytes": 0, 2023-09-13T03:29:20.782 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_items": 0, 2023-09-13T03:29:20.783 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_bytes": 0, 2023-09-13T03:29:20.783 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_items": 0, 2023-09-13T03:29:20.783 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-13T03:29:20.784 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_items": 0, 2023-09-13T03:29:20.784 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-13T03:29:20.784 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_items": 0, 2023-09-13T03:29:20.784 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_bytes": 0, 2023-09-13T03:29:20.785 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_items": 0, 2023-09-13T03:29:20.785 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_bytes": 0, 2023-09-13T03:29:20.785 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_items": 0, 2023-09-13T03:29:20.785 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-13T03:29:20.786 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_items": 0, 2023-09-13T03:29:20.786 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_bytes": 0, 2023-09-13T03:29:20.786 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_items": 0, 2023-09-13T03:29:20.787 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_bytes": 0, 2023-09-13T03:29:20.787 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_items": 0, 2023-09-13T03:29:20.787 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_bytes": 0, 2023-09-13T03:29:20.787 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_items": 0, 2023-09-13T03:29:20.788 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_bytes": 0, 2023-09-13T03:29:20.788 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_items": 0, 2023-09-13T03:29:20.788 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_bytes": 148714, 2023-09-13T03:29:20.789 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_items": 30, 2023-09-13T03:29:20.789 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_bytes": 0, 2023-09-13T03:29:20.789 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_items": 0, 2023-09-13T03:29:20.789 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_bytes": 0, 2023-09-13T03:29:20.790 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_items": 0, 2023-09-13T03:29:20.790 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_bytes": 0, 2023-09-13T03:29:20.790 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_items": 0, 2023-09-13T03:29:20.791 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_bytes": 0, 2023-09-13T03:29:20.791 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_items": 0, 2023-09-13T03:29:20.791 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_bytes": 0, 2023-09-13T03:29:20.792 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_items": 0, 2023-09-13T03:29:20.792 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_bytes": 0, 2023-09-13T03:29:20.792 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_items": 0, 2023-09-13T03:29:20.792 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_bytes": 0, 2023-09-13T03:29:20.793 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_items": 0, 2023-09-13T03:29:20.793 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_bytes": 928, 2023-09-13T03:29:20.793 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_items": 2, 2023-09-13T03:29:20.794 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_bytes": 0, 2023-09-13T03:29:20.794 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_items": 0, 2023-09-13T03:29:20.794 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_bytes": 0, 2023-09-13T03:29:20.795 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_items": 0 2023-09-13T03:29:20.795 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:20.795 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-13T03:29:20.795 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-09-13T03:29:20.796 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2023-09-13T03:29:20.796 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-09-13T03:29:20.796 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 313, 2023-09-13T03:29:20.797 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 1534025, 2023-09-13T03:29:20.797 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:20.797 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 313, 2023-09-13T03:29:20.797 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2023-09-13T03:29:20.798 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2023-09-13T03:29:20.798 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 313, 2023-09-13T03:29:20.798 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 1534025, 2023-09-13T03:29:20.799 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-09-13T03:29:20.799 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:20.799 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:20.799 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:20.800 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:20.800 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:20.800 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-13T03:29:20.801 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:20 smithi064 ceph-mon[97675]: pgmap v508: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 8.7 MiB/s wr, 301 op/s 2023-09-13T03:29:20.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:20 smithi170 ceph-mon[134328]: pgmap v508: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 8.7 MiB/s wr, 301 op/s 2023-09-13T03:29:20.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:20 smithi142 ceph-mon[134082]: pgmap v508: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 8.7 MiB/s wr, 301 op/s 2023-09-13T03:29:21.239 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2023-09-13T03:29:21.239 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2023-09-13T03:29:21.239 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2023-09-13T03:29:21.240 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2023-09-13T03:29:21.240 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2023-09-13T03:29:21.240 DEBUG:tasks.check_counter:Getting stats from g 2023-09-13T03:29:21.241 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph daemon mds.g perf dump 2023-09-13T03:29:22.128 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:29:22.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:22 smithi064 ceph-mon[97675]: pgmap v509: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 8.4 MiB/s wr, 299 op/s 2023-09-13T03:29:23.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:22 smithi170 ceph-mon[134328]: pgmap v509: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 8.4 MiB/s wr, 299 op/s 2023-09-13T03:29:23.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:22 smithi142 ceph-mon[134082]: pgmap v509: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 8.4 MiB/s wr, 299 op/s 2023-09-13T03:29:23.586 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-13T03:29:23.587 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-0": { 2023-09-13T03:29:23.587 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 1987, 2023-09-13T03:29:23.587 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 1920, 2023-09-13T03:29:23.588 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 61806224, 2023-09-13T03:29:23.588 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 679108, 2023-09-13T03:29:23.588 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 7, 2023-09-13T03:29:23.588 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 3, 2023-09-13T03:29:23.589 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.326264247, 2023-09-13T03:29:23.589 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.107105757, 2023-09-13T03:29:23.589 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.153958152, 2023-09-13T03:29:23.590 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.052910440, 2023-09-13T03:29:23.590 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T03:29:23.590 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 1919, 2023-09-13T03:29:23.590 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.143862230, 2023-09-13T03:29:23.591 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000074967 2023-09-13T03:29:23.591 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.591 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2023-09-13T03:29:23.591 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:23.592 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:23.592 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:23.592 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.593 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 1968304, 2023-09-13T03:29:23.593 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 182752 2023-09-13T03:29:23.593 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.593 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-1": { 2023-09-13T03:29:23.594 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 1653, 2023-09-13T03:29:23.594 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 1874, 2023-09-13T03:29:23.594 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 80113956, 2023-09-13T03:29:23.594 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 1150459, 2023-09-13T03:29:23.595 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 10, 2023-09-13T03:29:23.595 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 7, 2023-09-13T03:29:23.595 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.290899767, 2023-09-13T03:29:23.596 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.057650997, 2023-09-13T03:29:23.596 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.168440107, 2023-09-13T03:29:23.596 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.047315732, 2023-09-13T03:29:23.596 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T03:29:23.597 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 1872, 2023-09-13T03:29:23.597 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.092301615, 2023-09-13T03:29:23.597 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000049306 2023-09-13T03:29:23.597 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.598 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2023-09-13T03:29:23.598 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:23.598 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:23.599 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:23.599 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.599 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 3424, 2023-09-13T03:29:23.599 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 638496 2023-09-13T03:29:23.600 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.600 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-2": { 2023-09-13T03:29:23.600 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 725, 2023-09-13T03:29:23.601 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 727, 2023-09-13T03:29:23.601 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 117631911, 2023-09-13T03:29:23.601 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 225487, 2023-09-13T03:29:23.601 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 7, 2023-09-13T03:29:23.602 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 5, 2023-09-13T03:29:23.602 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.244661782, 2023-09-13T03:29:23.602 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.020602823, 2023-09-13T03:29:23.602 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.195494847, 2023-09-13T03:29:23.603 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.021473830, 2023-09-13T03:29:23.603 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T03:29:23.603 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 725, 2023-09-13T03:29:23.604 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.042355558, 2023-09-13T03:29:23.604 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000058421 2023-09-13T03:29:23.604 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.604 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2023-09-13T03:29:23.605 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:23.605 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:23.605 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:23.606 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.606 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-13T03:29:23.606 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 0 2023-09-13T03:29:23.606 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.607 INFO:teuthology.orchestra.run.smithi064.stdout: "cct": { 2023-09-13T03:29:23.607 INFO:teuthology.orchestra.run.smithi064.stdout: "total_workers": 1, 2023-09-13T03:29:23.607 INFO:teuthology.orchestra.run.smithi064.stdout: "unhealthy_workers": 0 2023-09-13T03:29:23.608 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.608 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-MDSRank": { 2023-09-13T03:29:23.608 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2023-09-13T03:29:23.608 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2023-09-13T03:29:23.609 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 6556, 2023-09-13T03:29:23.609 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.184066687, 2023-09-13T03:29:23.609 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000028076 2023-09-13T03:29:23.609 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:23.610 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.610 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-PurgeQueue": { 2023-09-13T03:29:23.610 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2023-09-13T03:29:23.611 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2023-09-13T03:29:23.611 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:23.611 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:23.611 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:23.612 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:23.612 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.612 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": { 2023-09-13T03:29:23.612 INFO:teuthology.orchestra.run.smithi064.stdout: "request": 0, 2023-09-13T03:29:23.613 INFO:teuthology.orchestra.run.smithi064.stdout: "reply": 0, 2023-09-13T03:29:23.613 INFO:teuthology.orchestra.run.smithi064.stdout: "reply_latency": { 2023-09-13T03:29:23.613 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:23.613 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:23.614 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:23.614 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.614 INFO:teuthology.orchestra.run.smithi064.stdout: "slow_reply": 0, 2023-09-13T03:29:23.615 INFO:teuthology.orchestra.run.smithi064.stdout: "forward": 0, 2023-09-13T03:29:23.615 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_fetch_complete": 0, 2023-09-13T03:29:23.615 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_fetch_keys": 0, 2023-09-13T03:29:23.615 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_commit": 0, 2023-09-13T03:29:23.616 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_split": 0, 2023-09-13T03:29:23.616 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_merge": 0, 2023-09-13T03:29:23.616 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes": 15751, 2023-09-13T03:29:23.617 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_top": 20, 2023-09-13T03:29:23.617 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_bottom": 11987, 2023-09-13T03:29:23.617 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_pin_tail": 3744, 2023-09-13T03:29:23.617 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_pinned": 15722, 2023-09-13T03:29:23.618 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_expired": 6, 2023-09-13T03:29:23.618 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_with_caps": 0, 2023-09-13T03:29:23.618 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": 0, 2023-09-13T03:29:23.619 INFO:teuthology.orchestra.run.smithi064.stdout: "subtrees": 5, 2023-09-13T03:29:23.619 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse": 0, 2023-09-13T03:29:23.619 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_hit": 0, 2023-09-13T03:29:23.619 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_forward": 0, 2023-09-13T03:29:23.620 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_discover": 0, 2023-09-13T03:29:23.620 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_dir_fetch": 0, 2023-09-13T03:29:23.620 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_remote_ino": 0, 2023-09-13T03:29:23.621 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_lock": 0, 2023-09-13T03:29:23.621 INFO:teuthology.orchestra.run.smithi064.stdout: "load_cent": 0, 2023-09-13T03:29:23.621 INFO:teuthology.orchestra.run.smithi064.stdout: "q": 0, 2023-09-13T03:29:23.621 INFO:teuthology.orchestra.run.smithi064.stdout: "exported": 0, 2023-09-13T03:29:23.622 INFO:teuthology.orchestra.run.smithi064.stdout: "exported_inodes": 0, 2023-09-13T03:29:23.622 INFO:teuthology.orchestra.run.smithi064.stdout: "imported": 0, 2023-09-13T03:29:23.622 INFO:teuthology.orchestra.run.smithi064.stdout: "imported_inodes": 0, 2023-09-13T03:29:23.622 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_dir_fetch": 0, 2023-09-13T03:29:23.623 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_backtrace_fetch": 0, 2023-09-13T03:29:23.623 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_peer_discover": 0, 2023-09-13T03:29:23.623 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rfiles": 0, 2023-09-13T03:29:23.623 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rbytes": 0, 2023-09-13T03:29:23.624 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rsnaps": 0, 2023-09-13T03:29:23.624 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_backtrace_fetch": 0, 2023-09-13T03:29:23.624 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_set_tag": 0, 2023-09-13T03:29:23.625 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_backtrace_repaired": 0, 2023-09-13T03:29:23.625 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_inotable_repaired": 0, 2023-09-13T03:29:23.625 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dir_inodes": 0, 2023-09-13T03:29:23.625 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dir_base_inodes": 0, 2023-09-13T03:29:23.626 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dirfrag_rstats": 0, 2023-09-13T03:29:23.626 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_file_inodes": 0, 2023-09-13T03:29:23.626 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_inode_file_caps": 0, 2023-09-13T03:29:23.626 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_revoke": 0, 2023-09-13T03:29:23.627 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_grant": 0, 2023-09-13T03:29:23.627 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_trunc": 0, 2023-09-13T03:29:23.627 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-13T03:29:23.628 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-13T03:29:23.628 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_caps": 0, 2023-09-13T03:29:23.628 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_caps_dirty": 0, 2023-09-13T03:29:23.628 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_cap_release": 0, 2023-09-13T03:29:23.629 INFO:teuthology.orchestra.run.smithi064.stdout: "process_request_cap_release": 0 2023-09-13T03:29:23.629 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.629 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_cache": { 2023-09-13T03:29:23.630 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_update": 0, 2023-09-13T03:29:23.630 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_update_receipt": 0, 2023-09-13T03:29:23.630 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_try_discover": 0, 2023-09-13T03:29:23.630 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_send_discover": 0, 2023-09-13T03:29:23.631 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_handle_discover": 0, 2023-09-13T03:29:23.631 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays": 0, 2023-09-13T03:29:23.631 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays_delayed": 0, 2023-09-13T03:29:23.632 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays_enqueuing": 0, 2023-09-13T03:29:23.632 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_created": 0, 2023-09-13T03:29:23.632 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_enqueued": 0, 2023-09-13T03:29:23.632 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_reintegrated": 0, 2023-09-13T03:29:23.633 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_migrated": 0, 2023-09-13T03:29:23.633 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_processing": 0, 2023-09-13T03:29:23.633 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_enqueued": 0, 2023-09-13T03:29:23.633 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_prioritized": 0, 2023-09-13T03:29:23.634 INFO:teuthology.orchestra.run.smithi064.stdout: "recovery_started": 0, 2023-09-13T03:29:23.634 INFO:teuthology.orchestra.run.smithi064.stdout: "recovery_completed": 0, 2023-09-13T03:29:23.634 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_enqueue_scrub": 0, 2023-09-13T03:29:23.635 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_exportdir": 0, 2023-09-13T03:29:23.635 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_flush": 0, 2023-09-13T03:29:23.635 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_fragmentdir": 0, 2023-09-13T03:29:23.635 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_fragstats": 0, 2023-09-13T03:29:23.636 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_inodestats": 0 2023-09-13T03:29:23.636 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.636 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_log": { 2023-09-13T03:29:23.636 INFO:teuthology.orchestra.run.smithi064.stdout: "evlrg": 0, 2023-09-13T03:29:23.637 INFO:teuthology.orchestra.run.smithi064.stdout: "evadd": 0, 2023-09-13T03:29:23.637 INFO:teuthology.orchestra.run.smithi064.stdout: "evex": 0, 2023-09-13T03:29:23.637 INFO:teuthology.orchestra.run.smithi064.stdout: "evtrm": 0, 2023-09-13T03:29:23.637 INFO:teuthology.orchestra.run.smithi064.stdout: "ev": 76761, 2023-09-13T03:29:23.638 INFO:teuthology.orchestra.run.smithi064.stdout: "evexg": 0, 2023-09-13T03:29:23.638 INFO:teuthology.orchestra.run.smithi064.stdout: "evexd": 0, 2023-09-13T03:29:23.638 INFO:teuthology.orchestra.run.smithi064.stdout: "segadd": 0, 2023-09-13T03:29:23.639 INFO:teuthology.orchestra.run.smithi064.stdout: "segex": 0, 2023-09-13T03:29:23.639 INFO:teuthology.orchestra.run.smithi064.stdout: "segtrm": 0, 2023-09-13T03:29:23.639 INFO:teuthology.orchestra.run.smithi064.stdout: "seg": 106, 2023-09-13T03:29:23.639 INFO:teuthology.orchestra.run.smithi064.stdout: "segmjr": 8, 2023-09-13T03:29:23.640 INFO:teuthology.orchestra.run.smithi064.stdout: "segexg": 0, 2023-09-13T03:29:23.640 INFO:teuthology.orchestra.run.smithi064.stdout: "segexd": 0, 2023-09-13T03:29:23.640 INFO:teuthology.orchestra.run.smithi064.stdout: "expos": 4194304, 2023-09-13T03:29:23.641 INFO:teuthology.orchestra.run.smithi064.stdout: "wrpos": 260670034, 2023-09-13T03:29:23.641 INFO:teuthology.orchestra.run.smithi064.stdout: "rdpos": 260662582, 2023-09-13T03:29:23.641 INFO:teuthology.orchestra.run.smithi064.stdout: "jlat": { 2023-09-13T03:29:23.641 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:23.642 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:23.642 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:23.642 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.642 INFO:teuthology.orchestra.run.smithi064.stdout: "replayed": 76761 2023-09-13T03:29:23.643 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.643 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_mem": { 2023-09-13T03:29:23.643 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 1234, 2023-09-13T03:29:23.644 INFO:teuthology.orchestra.run.smithi064.stdout: "ino+": 15744, 2023-09-13T03:29:23.644 INFO:teuthology.orchestra.run.smithi064.stdout: "ino-": 14510, 2023-09-13T03:29:23.644 INFO:teuthology.orchestra.run.smithi064.stdout: "dir": 93, 2023-09-13T03:29:23.644 INFO:teuthology.orchestra.run.smithi064.stdout: "dir+": 152, 2023-09-13T03:29:23.645 INFO:teuthology.orchestra.run.smithi064.stdout: "dir-": 59, 2023-09-13T03:29:23.645 INFO:teuthology.orchestra.run.smithi064.stdout: "dn": 15751, 2023-09-13T03:29:23.645 INFO:teuthology.orchestra.run.smithi064.stdout: "dn+": 15940, 2023-09-13T03:29:23.645 INFO:teuthology.orchestra.run.smithi064.stdout: "dn-": 189, 2023-09-13T03:29:23.646 INFO:teuthology.orchestra.run.smithi064.stdout: "cap": 0, 2023-09-13T03:29:23.646 INFO:teuthology.orchestra.run.smithi064.stdout: "cap+": 0, 2023-09-13T03:29:23.646 INFO:teuthology.orchestra.run.smithi064.stdout: "cap-": 0, 2023-09-13T03:29:23.647 INFO:teuthology.orchestra.run.smithi064.stdout: "rss": 69156, 2023-09-13T03:29:23.647 INFO:teuthology.orchestra.run.smithi064.stdout: "heap": 207152 2023-09-13T03:29:23.647 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.647 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_server": { 2023-09-13T03:29:23.648 INFO:teuthology.orchestra.run.smithi064.stdout: "dispatch_client_request": 0, 2023-09-13T03:29:23.648 INFO:teuthology.orchestra.run.smithi064.stdout: "dispatch_server_request": 0, 2023-09-13T03:29:23.648 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_request": 0, 2023-09-13T03:29:23.649 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_session": 0, 2023-09-13T03:29:23.649 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_peer_request": 0, 2023-09-13T03:29:23.649 INFO:teuthology.orchestra.run.smithi064.stdout: "req_create_latency": { 2023-09-13T03:29:23.649 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:23.650 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:23.650 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:23.650 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.650 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getattr_latency": { 2023-09-13T03:29:23.651 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:23.651 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:23.651 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:23.652 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.652 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getfilelock_latency": { 2023-09-13T03:29:23.652 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:23.652 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:23.653 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:23.653 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.653 INFO:teuthology.orchestra.run.smithi064.stdout: "req_link_latency": { 2023-09-13T03:29:23.653 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:23.654 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:23.654 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:23.654 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.655 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookup_latency": { 2023-09-13T03:29:23.655 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:23.655 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:23.655 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:23.656 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.656 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookuphash_latency": { 2023-09-13T03:29:23.656 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:23.656 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:23.657 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:23.657 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.657 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupino_latency": { 2023-09-13T03:29:23.657 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:23.658 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:23.658 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:23.658 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.659 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupname_latency": { 2023-09-13T03:29:23.659 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:23.659 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:23.659 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:23.660 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.660 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupparent_latency": { 2023-09-13T03:29:23.660 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:23.660 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:23.661 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:23.661 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.661 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupsnap_latency": { 2023-09-13T03:29:23.662 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:23.662 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:23.662 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:23.662 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.663 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lssnap_latency": { 2023-09-13T03:29:23.663 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:23.663 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:23.663 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:23.664 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.664 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mkdir_latency": { 2023-09-13T03:29:23.664 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:23.664 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:23.665 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:23.665 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.665 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mknod_latency": { 2023-09-13T03:29:23.665 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:23.666 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:23.666 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:23.666 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.667 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mksnap_latency": { 2023-09-13T03:29:23.667 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:23.667 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:23.667 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:23.668 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.668 INFO:teuthology.orchestra.run.smithi064.stdout: "req_open_latency": { 2023-09-13T03:29:23.668 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:23.668 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:23.669 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:23.669 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.669 INFO:teuthology.orchestra.run.smithi064.stdout: "req_readdir_latency": { 2023-09-13T03:29:23.669 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:23.670 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:23.670 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:23.670 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.670 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rename_latency": { 2023-09-13T03:29:23.671 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:23.671 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:23.671 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:23.672 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.672 INFO:teuthology.orchestra.run.smithi064.stdout: "req_renamesnap_latency": { 2023-09-13T03:29:23.672 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:23.672 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:23.673 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:23.673 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.673 INFO:teuthology.orchestra.run.smithi064.stdout: "req_snapdiff_latency": { 2023-09-13T03:29:23.673 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:23.674 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:23.674 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:23.674 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.675 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmdir_latency": { 2023-09-13T03:29:23.675 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:23.675 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:23.675 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:23.676 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.676 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmsnap_latency": { 2023-09-13T03:29:23.676 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:23.676 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:23.677 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:23.677 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.677 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmxattr_latency": { 2023-09-13T03:29:23.677 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:23.678 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:23.678 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:23.678 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.679 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setattr_latency": { 2023-09-13T03:29:23.679 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:23.679 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:23.679 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:23.680 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.680 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setdirlayout_latency": { 2023-09-13T03:29:23.680 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:23.680 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:23.681 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:23.681 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.681 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setfilelock_latency": { 2023-09-13T03:29:23.681 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:23.682 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:23.682 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:23.682 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.683 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setlayout_latency": { 2023-09-13T03:29:23.683 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:23.683 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:23.683 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:23.684 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.684 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setxattr_latency": { 2023-09-13T03:29:23.684 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:23.684 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:23.685 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:23.685 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.685 INFO:teuthology.orchestra.run.smithi064.stdout: "req_symlink_latency": { 2023-09-13T03:29:23.685 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:23.685 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:23.686 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:23.686 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.686 INFO:teuthology.orchestra.run.smithi064.stdout: "req_unlink_latency": { 2023-09-13T03:29:23.687 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:23.687 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:23.687 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:23.687 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.688 INFO:teuthology.orchestra.run.smithi064.stdout: "cap_revoke_eviction": 0, 2023-09-13T03:29:23.688 INFO:teuthology.orchestra.run.smithi064.stdout: "cap_acquisition_throttle": 0, 2023-09-13T03:29:23.688 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getvxattr_latency": { 2023-09-13T03:29:23.688 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:23.689 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:23.689 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:23.689 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:23.689 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.690 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_sessions": { 2023-09-13T03:29:23.690 INFO:teuthology.orchestra.run.smithi064.stdout: "session_count": 1, 2023-09-13T03:29:23.690 INFO:teuthology.orchestra.run.smithi064.stdout: "session_add": 2, 2023-09-13T03:29:23.690 INFO:teuthology.orchestra.run.smithi064.stdout: "session_remove": 1, 2023-09-13T03:29:23.691 INFO:teuthology.orchestra.run.smithi064.stdout: "sessions_open": 2, 2023-09-13T03:29:23.691 INFO:teuthology.orchestra.run.smithi064.stdout: "sessions_stale": 0, 2023-09-13T03:29:23.691 INFO:teuthology.orchestra.run.smithi064.stdout: "total_load": 0, 2023-09-13T03:29:23.691 INFO:teuthology.orchestra.run.smithi064.stdout: "average_load": 0, 2023-09-13T03:29:23.692 INFO:teuthology.orchestra.run.smithi064.stdout: "avg_session_uptime": 4128, 2023-09-13T03:29:23.692 INFO:teuthology.orchestra.run.smithi064.stdout: "mdthresh_evicted": 0 2023-09-13T03:29:23.692 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.693 INFO:teuthology.orchestra.run.smithi064.stdout: "mempool": { 2023-09-13T03:29:23.693 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_bytes": 0, 2023-09-13T03:29:23.693 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_items": 0, 2023-09-13T03:29:23.693 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_bytes": 0, 2023-09-13T03:29:23.694 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_items": 0, 2023-09-13T03:29:23.694 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_bytes": 0, 2023-09-13T03:29:23.694 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_items": 0, 2023-09-13T03:29:23.694 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-13T03:29:23.695 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_items": 0, 2023-09-13T03:29:23.695 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-13T03:29:23.695 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_items": 0, 2023-09-13T03:29:23.695 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_bytes": 0, 2023-09-13T03:29:23.696 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_items": 0, 2023-09-13T03:29:23.696 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-13T03:29:23.696 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_items": 0, 2023-09-13T03:29:23.697 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_bytes": 0, 2023-09-13T03:29:23.697 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_items": 0, 2023-09-13T03:29:23.697 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_bytes": 0, 2023-09-13T03:29:23.697 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_items": 0, 2023-09-13T03:29:23.698 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-13T03:29:23.698 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_items": 0, 2023-09-13T03:29:23.698 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-13T03:29:23.698 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_items": 0, 2023-09-13T03:29:23.699 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_bytes": 0, 2023-09-13T03:29:23.699 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_items": 0, 2023-09-13T03:29:23.699 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_bytes": 0, 2023-09-13T03:29:23.699 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_items": 0, 2023-09-13T03:29:23.700 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-13T03:29:23.700 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_items": 0, 2023-09-13T03:29:23.700 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_bytes": 0, 2023-09-13T03:29:23.700 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_items": 0, 2023-09-13T03:29:23.701 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_bytes": 0, 2023-09-13T03:29:23.701 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_items": 0, 2023-09-13T03:29:23.701 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_bytes": 0, 2023-09-13T03:29:23.701 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_items": 0, 2023-09-13T03:29:23.702 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_bytes": 0, 2023-09-13T03:29:23.702 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_items": 0, 2023-09-13T03:29:23.702 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_bytes": 284185, 2023-09-13T03:29:23.702 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_items": 1395, 2023-09-13T03:29:23.703 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_bytes": 0, 2023-09-13T03:29:23.703 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_items": 0, 2023-09-13T03:29:23.703 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_bytes": 0, 2023-09-13T03:29:23.703 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_items": 0, 2023-09-13T03:29:23.704 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_bytes": 0, 2023-09-13T03:29:23.704 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_items": 0, 2023-09-13T03:29:23.704 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_bytes": 0, 2023-09-13T03:29:23.705 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_items": 0, 2023-09-13T03:29:23.705 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_bytes": 7936, 2023-09-13T03:29:23.705 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_items": 165, 2023-09-13T03:29:23.705 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_bytes": 0, 2023-09-13T03:29:23.705 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_items": 0, 2023-09-13T03:29:23.706 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_bytes": 0, 2023-09-13T03:29:23.706 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_items": 0, 2023-09-13T03:29:23.706 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_bytes": 14135065, 2023-09-13T03:29:23.707 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_items": 167400, 2023-09-13T03:29:23.707 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_bytes": 0, 2023-09-13T03:29:23.707 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_items": 0, 2023-09-13T03:29:23.707 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_bytes": 0, 2023-09-13T03:29:23.708 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_items": 0 2023-09-13T03:29:23.708 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.708 INFO:teuthology.orchestra.run.smithi064.stdout: "objecter": { 2023-09-13T03:29:23.708 INFO:teuthology.orchestra.run.smithi064.stdout: "op_active": 0, 2023-09-13T03:29:23.709 INFO:teuthology.orchestra.run.smithi064.stdout: "op_laggy": 0, 2023-09-13T03:29:23.709 INFO:teuthology.orchestra.run.smithi064.stdout: "op_send": 3967, 2023-09-13T03:29:23.709 INFO:teuthology.orchestra.run.smithi064.stdout: "op_send_bytes": 21, 2023-09-13T03:29:23.709 INFO:teuthology.orchestra.run.smithi064.stdout: "op_resend": 3, 2023-09-13T03:29:23.710 INFO:teuthology.orchestra.run.smithi064.stdout: "op_reply": 3967, 2023-09-13T03:29:23.710 INFO:teuthology.orchestra.run.smithi064.stdout: "op_latency": { 2023-09-13T03:29:23.710 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 3967, 2023-09-13T03:29:23.710 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 5.448894295, 2023-09-13T03:29:23.711 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.001373555 2023-09-13T03:29:23.711 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.711 INFO:teuthology.orchestra.run.smithi064.stdout: "op_inflight": 0, 2023-09-13T03:29:23.711 INFO:teuthology.orchestra.run.smithi064.stdout: "oplen_avg": { 2023-09-13T03:29:23.712 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 3967, 2023-09-13T03:29:23.712 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 3968 2023-09-13T03:29:23.712 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.712 INFO:teuthology.orchestra.run.smithi064.stdout: "op": 3967, 2023-09-13T03:29:23.713 INFO:teuthology.orchestra.run.smithi064.stdout: "op_r": 3967, 2023-09-13T03:29:23.713 INFO:teuthology.orchestra.run.smithi064.stdout: "op_w": 0, 2023-09-13T03:29:23.713 INFO:teuthology.orchestra.run.smithi064.stdout: "op_rmw": 0, 2023-09-13T03:29:23.713 INFO:teuthology.orchestra.run.smithi064.stdout: "op_pg": 0, 2023-09-13T03:29:23.714 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_stat": 2271, 2023-09-13T03:29:23.714 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_create": 0, 2023-09-13T03:29:23.714 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_read": 1694, 2023-09-13T03:29:23.715 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_write": 0, 2023-09-13T03:29:23.715 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_writefull": 0, 2023-09-13T03:29:23.715 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_writesame": 0, 2023-09-13T03:29:23.715 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_append": 0, 2023-09-13T03:29:23.716 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_zero": 0, 2023-09-13T03:29:23.716 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_truncate": 0, 2023-09-13T03:29:23.716 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_delete": 0, 2023-09-13T03:29:23.717 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_mapext": 0, 2023-09-13T03:29:23.717 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_sparse_read": 0, 2023-09-13T03:29:23.717 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_clonerange": 0, 2023-09-13T03:29:23.717 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_getxattr": 1, 2023-09-13T03:29:23.718 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_setxattr": 0, 2023-09-13T03:29:23.718 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_cmpxattr": 0, 2023-09-13T03:29:23.718 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_rmxattr": 0, 2023-09-13T03:29:23.718 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_resetxattrs": 0, 2023-09-13T03:29:23.719 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_call": 0, 2023-09-13T03:29:23.719 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_watch": 0, 2023-09-13T03:29:23.719 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_notify": 0, 2023-09-13T03:29:23.720 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_src_cmpxattr": 0, 2023-09-13T03:29:23.720 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_pgls": 0, 2023-09-13T03:29:23.720 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_pgls_filter": 0, 2023-09-13T03:29:23.720 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_other": 0, 2023-09-13T03:29:23.721 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_active": 0, 2023-09-13T03:29:23.721 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_send": 0, 2023-09-13T03:29:23.721 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_resend": 0, 2023-09-13T03:29:23.721 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_ping": 0, 2023-09-13T03:29:23.722 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_active": 0, 2023-09-13T03:29:23.722 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_send": 0, 2023-09-13T03:29:23.722 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_resend": 0, 2023-09-13T03:29:23.723 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_active": 0, 2023-09-13T03:29:23.723 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_send": 0, 2023-09-13T03:29:23.723 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_resend": 0, 2023-09-13T03:29:23.723 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_active": 0, 2023-09-13T03:29:23.724 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_send": 0, 2023-09-13T03:29:23.724 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_resend": 0, 2023-09-13T03:29:23.724 INFO:teuthology.orchestra.run.smithi064.stdout: "command_active": 0, 2023-09-13T03:29:23.724 INFO:teuthology.orchestra.run.smithi064.stdout: "command_send": 0, 2023-09-13T03:29:23.725 INFO:teuthology.orchestra.run.smithi064.stdout: "command_resend": 0, 2023-09-13T03:29:23.725 INFO:teuthology.orchestra.run.smithi064.stdout: "map_epoch": 106, 2023-09-13T03:29:23.725 INFO:teuthology.orchestra.run.smithi064.stdout: "map_full": 0, 2023-09-13T03:29:23.725 INFO:teuthology.orchestra.run.smithi064.stdout: "map_inc": 23, 2023-09-13T03:29:23.726 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_sessions": 11, 2023-09-13T03:29:23.726 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_session_open": 12, 2023-09-13T03:29:23.726 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_session_close": 1, 2023-09-13T03:29:23.726 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_laggy": 0, 2023-09-13T03:29:23.727 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_wr": 0, 2023-09-13T03:29:23.727 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_rd": 2, 2023-09-13T03:29:23.727 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_del": 0 2023-09-13T03:29:23.727 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.728 INFO:teuthology.orchestra.run.smithi064.stdout: "oft": { 2023-09-13T03:29:23.728 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_objs": 0, 2023-09-13T03:29:23.728 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_kv_pairs": 0, 2023-09-13T03:29:23.728 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_updates": 0, 2023-09-13T03:29:23.729 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_removes": 0 2023-09-13T03:29:23.729 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.729 INFO:teuthology.orchestra.run.smithi064.stdout: "osd-slow-ops": { 2023-09-13T03:29:23.730 INFO:teuthology.orchestra.run.smithi064.stdout: "slow_ops_count": 0 2023-09-13T03:29:23.730 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.730 INFO:teuthology.orchestra.run.smithi064.stdout: "purge_queue": { 2023-09-13T03:29:23.730 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_ops": 0, 2023-09-13T03:29:23.731 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_ops_high_water": 0, 2023-09-13T03:29:23.731 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing": 0, 2023-09-13T03:29:23.731 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_high_water": 0, 2023-09-13T03:29:23.731 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executed": 0, 2023-09-13T03:29:23.732 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_item_in_journal": 0 2023-09-13T03:29:23.732 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.732 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-13T03:29:23.732 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-09-13T03:29:23.733 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2023-09-13T03:29:23.733 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-09-13T03:29:23.733 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 4361, 2023-09-13T03:29:23.733 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 259137059, 2023-09-13T03:29:23.734 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:23.734 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 4361, 2023-09-13T03:29:23.734 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2023-09-13T03:29:23.734 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2023-09-13T03:29:23.735 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 4361, 2023-09-13T03:29:23.735 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 259137059, 2023-09-13T03:29:23.735 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-09-13T03:29:23.735 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:23.736 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:23.736 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:23.736 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:23.736 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.737 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-objecter_bytes": { 2023-09-13T03:29:23.737 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-09-13T03:29:23.737 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2023-09-13T03:29:23.737 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-09-13T03:29:23.738 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2023-09-13T03:29:23.738 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2023-09-13T03:29:23.738 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:23.738 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2023-09-13T03:29:23.739 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 3967, 2023-09-13T03:29:23.739 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 256482705, 2023-09-13T03:29:23.739 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 3967, 2023-09-13T03:29:23.740 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 256482705, 2023-09-13T03:29:23.740 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-09-13T03:29:23.740 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:23.740 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:23.741 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:23.741 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:23.741 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.741 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-objecter_ops": { 2023-09-13T03:29:23.742 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-09-13T03:29:23.742 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 1024, 2023-09-13T03:29:23.742 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-09-13T03:29:23.743 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2023-09-13T03:29:23.743 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2023-09-13T03:29:23.743 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:23.743 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2023-09-13T03:29:23.744 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 3967, 2023-09-13T03:29:23.744 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 3967, 2023-09-13T03:29:23.744 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 3967, 2023-09-13T03:29:23.744 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 3967, 2023-09-13T03:29:23.745 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-09-13T03:29:23.745 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:23.745 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:23.745 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:23.746 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:23.746 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.746 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-write_buf_throttle": { 2023-09-13T03:29:23.747 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-09-13T03:29:23.747 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 3758096384, 2023-09-13T03:29:23.747 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-09-13T03:29:23.747 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2023-09-13T03:29:23.748 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2023-09-13T03:29:23.748 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:23.748 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2023-09-13T03:29:23.748 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2023-09-13T03:29:23.749 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2023-09-13T03:29:23.749 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 0, 2023-09-13T03:29:23.749 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 0, 2023-09-13T03:29:23.749 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-09-13T03:29:23.750 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:23.750 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:23.750 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:23.750 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:23.751 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:23.751 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-write_buf_throttle-0x56219238ca00": { 2023-09-13T03:29:23.751 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-09-13T03:29:23.751 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 3758096384, 2023-09-13T03:29:23.752 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-09-13T03:29:23.752 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2023-09-13T03:29:23.752 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2023-09-13T03:29:23.753 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:23.753 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2023-09-13T03:29:23.753 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2023-09-13T03:29:23.753 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2023-09-13T03:29:23.754 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 0, 2023-09-13T03:29:23.754 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 0, 2023-09-13T03:29:23.754 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-09-13T03:29:23.754 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:23.755 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:23.755 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:23.755 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:23.755 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:23.756 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-13T03:29:23.929 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2023-09-13T03:29:23.929 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2023-09-13T03:29:23.929 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2023-09-13T03:29:23.930 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2023-09-13T03:29:23.930 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2023-09-13T03:29:23.930 DEBUG:tasks.check_counter:Getting stats from j 2023-09-13T03:29:23.931 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph daemon mds.j perf dump 2023-09-13T03:29:24.712 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-13T03:29:25.128 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:24 smithi064 ceph-mon[97675]: pgmap v510: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 8.6 MiB/s wr, 310 op/s 2023-09-13T03:29:25.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:24 smithi170 ceph-mon[134328]: pgmap v510: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 8.6 MiB/s wr, 310 op/s 2023-09-13T03:29:25.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:24 smithi142 ceph-mon[134082]: pgmap v510: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 8.6 MiB/s wr, 310 op/s 2023-09-13T03:29:26.262 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-09-13T03:29:26.263 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-0": { 2023-09-13T03:29:26.263 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 0, 2023-09-13T03:29:26.263 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 2, 2023-09-13T03:29:26.264 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 0, 2023-09-13T03:29:26.264 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 0, 2023-09-13T03:29:26.264 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 4, 2023-09-13T03:29:26.265 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 0, 2023-09-13T03:29:26.265 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.004041502, 2023-09-13T03:29:26.265 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.000000000, 2023-09-13T03:29:26.265 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.000841299, 2023-09-13T03:29:26.266 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-13T03:29:26.266 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T03:29:26.266 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:26.267 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:26.267 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:26.267 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:26.267 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2023-09-13T03:29:26.268 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:26.268 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:26.268 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:26.268 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:26.269 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-13T03:29:26.269 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 0 2023-09-13T03:29:26.269 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:26.269 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-1": { 2023-09-13T03:29:26.270 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 312, 2023-09-13T03:29:26.270 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 301, 2023-09-13T03:29:26.270 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 1578272, 2023-09-13T03:29:26.271 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 180992, 2023-09-13T03:29:26.271 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 2, 2023-09-13T03:29:26.271 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 1, 2023-09-13T03:29:26.271 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.065464990, 2023-09-13T03:29:26.272 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.024591801, 2023-09-13T03:29:26.272 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.043710122, 2023-09-13T03:29:26.272 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.006539917, 2023-09-13T03:29:26.272 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T03:29:26.273 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 300, 2023-09-13T03:29:26.273 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.041274424, 2023-09-13T03:29:26.273 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000137581 2023-09-13T03:29:26.274 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:26.274 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2023-09-13T03:29:26.274 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:26.274 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:26.275 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:26.275 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:26.275 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 1578272, 2023-09-13T03:29:26.276 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 180992 2023-09-13T03:29:26.276 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:26.276 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-2": { 2023-09-13T03:29:26.276 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 7, 2023-09-13T03:29:26.277 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 232, 2023-09-13T03:29:26.277 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 3424, 2023-09-13T03:29:26.277 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 270080, 2023-09-13T03:29:26.278 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 6, 2023-09-13T03:29:26.278 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 3, 2023-09-13T03:29:26.278 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.026980148, 2023-09-13T03:29:26.278 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.017893400, 2023-09-13T03:29:26.279 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.004375575, 2023-09-13T03:29:26.279 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-13T03:29:26.279 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T03:29:26.279 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 230, 2023-09-13T03:29:26.280 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.031279942, 2023-09-13T03:29:26.280 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000135999 2023-09-13T03:29:26.280 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:26.281 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2023-09-13T03:29:26.281 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:26.281 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:26.281 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:26.282 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:26.282 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 3424, 2023-09-13T03:29:26.282 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 270080 2023-09-13T03:29:26.283 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:26.283 INFO:teuthology.orchestra.run.smithi064.stdout: "cct": { 2023-09-13T03:29:26.283 INFO:teuthology.orchestra.run.smithi064.stdout: "total_workers": 0, 2023-09-13T03:29:26.283 INFO:teuthology.orchestra.run.smithi064.stdout: "unhealthy_workers": 0 2023-09-13T03:29:26.283 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:26.284 INFO:teuthology.orchestra.run.smithi064.stdout: "mempool": { 2023-09-13T03:29:26.284 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_bytes": 0, 2023-09-13T03:29:26.284 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_items": 0, 2023-09-13T03:29:26.285 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_bytes": 0, 2023-09-13T03:29:26.285 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_items": 0, 2023-09-13T03:29:26.285 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_bytes": 0, 2023-09-13T03:29:26.285 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_items": 0, 2023-09-13T03:29:26.286 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-13T03:29:26.286 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_items": 0, 2023-09-13T03:29:26.286 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-13T03:29:26.287 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_items": 0, 2023-09-13T03:29:26.287 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_bytes": 0, 2023-09-13T03:29:26.287 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_items": 0, 2023-09-13T03:29:26.287 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-13T03:29:26.288 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_items": 0, 2023-09-13T03:29:26.288 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_bytes": 0, 2023-09-13T03:29:26.288 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_items": 0, 2023-09-13T03:29:26.288 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_bytes": 0, 2023-09-13T03:29:26.289 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_items": 0, 2023-09-13T03:29:26.289 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-13T03:29:26.289 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_items": 0, 2023-09-13T03:29:26.290 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-13T03:29:26.290 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_items": 0, 2023-09-13T03:29:26.290 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_bytes": 0, 2023-09-13T03:29:26.291 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_items": 0, 2023-09-13T03:29:26.291 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_bytes": 0, 2023-09-13T03:29:26.291 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_items": 0, 2023-09-13T03:29:26.291 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-13T03:29:26.292 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_items": 0, 2023-09-13T03:29:26.292 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_bytes": 0, 2023-09-13T03:29:26.292 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_items": 0, 2023-09-13T03:29:26.292 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_bytes": 0, 2023-09-13T03:29:26.293 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_items": 0, 2023-09-13T03:29:26.293 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_bytes": 0, 2023-09-13T03:29:26.293 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_items": 0, 2023-09-13T03:29:26.294 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_bytes": 0, 2023-09-13T03:29:26.294 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_items": 0, 2023-09-13T03:29:26.294 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_bytes": 148714, 2023-09-13T03:29:26.294 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_items": 30, 2023-09-13T03:29:26.295 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_bytes": 0, 2023-09-13T03:29:26.295 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_items": 0, 2023-09-13T03:29:26.295 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_bytes": 0, 2023-09-13T03:29:26.296 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_items": 0, 2023-09-13T03:29:26.296 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_bytes": 0, 2023-09-13T03:29:26.296 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_items": 0, 2023-09-13T03:29:26.296 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_bytes": 0, 2023-09-13T03:29:26.297 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_items": 0, 2023-09-13T03:29:26.297 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_bytes": 0, 2023-09-13T03:29:26.297 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_items": 0, 2023-09-13T03:29:26.298 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_bytes": 0, 2023-09-13T03:29:26.298 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_items": 0, 2023-09-13T03:29:26.298 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_bytes": 0, 2023-09-13T03:29:26.298 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_items": 0, 2023-09-13T03:29:26.299 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_bytes": 928, 2023-09-13T03:29:26.299 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_items": 2, 2023-09-13T03:29:26.299 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_bytes": 0, 2023-09-13T03:29:26.300 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_items": 0, 2023-09-13T03:29:26.300 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_bytes": 0, 2023-09-13T03:29:26.300 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_items": 0 2023-09-13T03:29:26.300 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-09-13T03:29:26.301 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-13T03:29:26.301 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-09-13T03:29:26.301 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2023-09-13T03:29:26.302 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-09-13T03:29:26.302 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 315, 2023-09-13T03:29:26.302 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 1534285, 2023-09-13T03:29:26.302 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:26.303 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 315, 2023-09-13T03:29:26.303 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2023-09-13T03:29:26.303 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2023-09-13T03:29:26.303 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 315, 2023-09-13T03:29:26.304 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 1534285, 2023-09-13T03:29:26.304 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-09-13T03:29:26.304 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-09-13T03:29:26.304 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-09-13T03:29:26.305 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-09-13T03:29:26.305 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:26.305 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-09-13T03:29:26.306 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-09-13T03:29:26.707 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-09-13T03:29:26.707 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-09-13T03:29:26.708 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2023-09-13T03:29:26.708 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2023-09-13T03:29:26.708 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2023-09-13T03:29:26.709 DEBUG:tasks.check_counter:Getting stats from b 2023-09-13T03:29:26.709 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph daemon mds.b perf dump 2023-09-13T03:29:27.163 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:26 smithi170 ceph-mon[134328]: pgmap v511: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 9.5 MiB/s wr, 345 op/s 2023-09-13T03:29:27.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:26 smithi064 ceph-mon[97675]: pgmap v511: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 9.5 MiB/s wr, 345 op/s 2023-09-13T03:29:27.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:26 smithi142 ceph-mon[134082]: pgmap v511: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 9.5 MiB/s wr, 345 op/s 2023-09-13T03:29:27.370 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-13T03:29:28.643 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2023-09-13T03:29:28.643 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2023-09-13T03:29:28.644 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 58571, 2023-09-13T03:29:28.644 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 68197, 2023-09-13T03:29:28.644 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 15898685, 2023-09-13T03:29:28.645 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 155481072, 2023-09-13T03:29:28.645 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 16, 2023-09-13T03:29:28.645 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 5, 2023-09-13T03:29:28.645 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 3.852006539, 2023-09-13T03:29:28.646 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 1.394624642, 2023-09-13T03:29:28.646 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 1.643631619, 2023-09-13T03:29:28.646 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.868902888, 2023-09-13T03:29:28.647 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T03:29:28.647 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 68193, 2023-09-13T03:29:28.647 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 16.412428556, 2023-09-13T03:29:28.647 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000240676 2023-09-13T03:29:28.648 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.648 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2023-09-13T03:29:28.648 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 73520, 2023-09-13T03:29:28.649 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.012202791, 2023-09-13T03:29:28.649 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000165 2023-09-13T03:29:28.649 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.649 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 160, 2023-09-13T03:29:28.650 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 434016 2023-09-13T03:29:28.650 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.650 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2023-09-13T03:29:28.651 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 148525, 2023-09-13T03:29:28.651 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 162882, 2023-09-13T03:29:28.651 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 40218862, 2023-09-13T03:29:28.651 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 189274591, 2023-09-13T03:29:28.652 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 32, 2023-09-13T03:29:28.652 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 6, 2023-09-13T03:29:28.652 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 8.654352425, 2023-09-13T03:29:28.652 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 3.495840652, 2023-09-13T03:29:28.653 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 4.948539405, 2023-09-13T03:29:28.653 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.976120106, 2023-09-13T03:29:28.653 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T03:29:28.654 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 162881, 2023-09-13T03:29:28.654 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 13.244547375, 2023-09-13T03:29:28.654 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000081314 2023-09-13T03:29:28.654 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.655 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2023-09-13T03:29:28.655 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 258072, 2023-09-13T03:29:28.655 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.036686630, 2023-09-13T03:29:28.656 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000142 2023-09-13T03:29:28.656 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.656 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 536096, 2023-09-13T03:29:28.656 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 281488 2023-09-13T03:29:28.657 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.657 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2023-09-13T03:29:28.657 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 232347, 2023-09-13T03:29:28.658 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 315313, 2023-09-13T03:29:28.658 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 71839115, 2023-09-13T03:29:28.658 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 264897451, 2023-09-13T03:29:28.658 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 164, 2023-09-13T03:29:28.659 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 6, 2023-09-13T03:29:28.659 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 11.846308581, 2023-09-13T03:29:28.659 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 5.331917143, 2023-09-13T03:29:28.659 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 7.841670751, 2023-09-13T03:29:28.660 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.431741454, 2023-09-13T03:29:28.660 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T03:29:28.660 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 315313, 2023-09-13T03:29:28.660 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 22.829265814, 2023-09-13T03:29:28.661 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000072401 2023-09-13T03:29:28.661 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.661 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2023-09-13T03:29:28.661 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 323311, 2023-09-13T03:29:28.662 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.068755987, 2023-09-13T03:29:28.662 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000212 2023-09-13T03:29:28.662 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.662 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 2095216, 2023-09-13T03:29:28.663 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 230608 2023-09-13T03:29:28.663 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.663 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2023-09-13T03:29:28.663 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 1, 2023-09-13T03:29:28.664 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2023-09-13T03:29:28.664 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.664 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-MDSRank": { 2023-09-13T03:29:28.664 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2023-09-13T03:29:28.665 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2023-09-13T03:29:28.665 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 72329, 2023-09-13T03:29:28.665 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 59.626215043, 2023-09-13T03:29:28.665 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000824374 2023-09-13T03:29:28.666 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-09-13T03:29:28.666 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.666 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-PurgeQueue": { 2023-09-13T03:29:28.666 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2023-09-13T03:29:28.667 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2023-09-13T03:29:28.667 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 25061, 2023-09-13T03:29:28.667 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 4.303491643, 2023-09-13T03:29:28.667 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000171720 2023-09-13T03:29:28.668 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-09-13T03:29:28.668 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.668 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2023-09-13T03:29:28.669 INFO:teuthology.orchestra.run.smithi142.stdout: "request": 109890, 2023-09-13T03:29:28.669 INFO:teuthology.orchestra.run.smithi142.stdout: "reply": 94725, 2023-09-13T03:29:28.669 INFO:teuthology.orchestra.run.smithi142.stdout: "reply_latency": { 2023-09-13T03:29:28.669 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 94725, 2023-09-13T03:29:28.670 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 707.209325775, 2023-09-13T03:29:28.670 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.007465920 2023-09-13T03:29:28.670 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.670 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_reply": 0, 2023-09-13T03:29:28.671 INFO:teuthology.orchestra.run.smithi142.stdout: "forward": 15165, 2023-09-13T03:29:28.671 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_complete": 47, 2023-09-13T03:29:28.671 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_keys": 2, 2023-09-13T03:29:28.672 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_commit": 1828, 2023-09-13T03:29:28.672 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_split": 9, 2023-09-13T03:29:28.672 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_merge": 7, 2023-09-13T03:29:28.672 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes": 2054, 2023-09-13T03:29:28.673 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_top": 454, 2023-09-13T03:29:28.673 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_bottom": 1528, 2023-09-13T03:29:28.673 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pin_tail": 72, 2023-09-13T03:29:28.673 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pinned": 1405, 2023-09-13T03:29:28.674 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_expired": 3419, 2023-09-13T03:29:28.674 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_with_caps": 505, 2023-09-13T03:29:28.674 INFO:teuthology.orchestra.run.smithi142.stdout: "caps": 505, 2023-09-13T03:29:28.674 INFO:teuthology.orchestra.run.smithi142.stdout: "subtrees": 20, 2023-09-13T03:29:28.675 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse": 172083, 2023-09-13T03:29:28.675 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_hit": 105676, 2023-09-13T03:29:28.675 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_forward": 15146, 2023-09-13T03:29:28.675 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_discover": 172, 2023-09-13T03:29:28.676 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_dir_fetch": 30, 2023-09-13T03:29:28.676 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_remote_ino": 0, 2023-09-13T03:29:28.676 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_lock": 1418, 2023-09-13T03:29:28.676 INFO:teuthology.orchestra.run.smithi142.stdout: "load_cent": 62234, 2023-09-13T03:29:28.677 INFO:teuthology.orchestra.run.smithi142.stdout: "q": 7, 2023-09-13T03:29:28.677 INFO:teuthology.orchestra.run.smithi142.stdout: "exported": 26, 2023-09-13T03:29:28.677 INFO:teuthology.orchestra.run.smithi142.stdout: "exported_inodes": 2283, 2023-09-13T03:29:28.678 INFO:teuthology.orchestra.run.smithi142.stdout: "imported": 22, 2023-09-13T03:29:28.678 INFO:teuthology.orchestra.run.smithi142.stdout: "imported_inodes": 783, 2023-09-13T03:29:28.678 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_dir_fetch": 0, 2023-09-13T03:29:28.679 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_backtrace_fetch": 101, 2023-09-13T03:29:28.679 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_peer_discover": 0, 2023-09-13T03:29:28.679 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rfiles": 693, 2023-09-13T03:29:28.679 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rbytes": 211724109, 2023-09-13T03:29:28.680 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rsnaps": 6, 2023-09-13T03:29:28.680 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_fetch": 101, 2023-09-13T03:29:28.680 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_set_tag": 0, 2023-09-13T03:29:28.680 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_repaired": 0, 2023-09-13T03:29:28.681 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_inotable_repaired": 0, 2023-09-13T03:29:28.681 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_inodes": 44, 2023-09-13T03:29:28.681 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_base_inodes": 4, 2023-09-13T03:29:28.682 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dirfrag_rstats": 40, 2023-09-13T03:29:28.682 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_file_inodes": 57, 2023-09-13T03:29:28.682 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_inode_file_caps": 17, 2023-09-13T03:29:28.682 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_revoke": 17537, 2023-09-13T03:29:28.683 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_grant": 57481, 2023-09-13T03:29:28.683 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_trunc": 0, 2023-09-13T03:29:28.683 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flushsnap_ack": 1949, 2023-09-13T03:29:28.684 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-13T03:29:28.684 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps": 36265, 2023-09-13T03:29:28.684 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps_dirty": 14201, 2023-09-13T03:29:28.684 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_cap_release": 3835, 2023-09-13T03:29:28.685 INFO:teuthology.orchestra.run.smithi142.stdout: "process_request_cap_release": 57400 2023-09-13T03:29:28.685 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.685 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_cache": { 2023-09-13T03:29:28.686 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update": 17, 2023-09-13T03:29:28.686 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update_receipt": 30, 2023-09-13T03:29:28.686 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_try_discover": 20, 2023-09-13T03:29:28.686 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_send_discover": 182, 2023-09-13T03:29:28.687 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_handle_discover": 174, 2023-09-13T03:29:28.687 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays": 350, 2023-09-13T03:29:28.687 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_delayed": 0, 2023-09-13T03:29:28.687 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_enqueuing": 0, 2023-09-13T03:29:28.688 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_created": 13727, 2023-09-13T03:29:28.688 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_enqueued": 13377, 2023-09-13T03:29:28.688 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_reintegrated": 0, 2023-09-13T03:29:28.689 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_migrated": 0, 2023-09-13T03:29:28.689 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_processing": 0, 2023-09-13T03:29:28.689 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_enqueued": 0, 2023-09-13T03:29:28.689 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_prioritized": 0, 2023-09-13T03:29:28.690 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_started": 0, 2023-09-13T03:29:28.690 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_completed": 0, 2023-09-13T03:29:28.690 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_enqueue_scrub": 4, 2023-09-13T03:29:28.691 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_exportdir": 29, 2023-09-13T03:29:28.691 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_flush": 0, 2023-09-13T03:29:28.691 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragmentdir": 16, 2023-09-13T03:29:28.691 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragstats": 0, 2023-09-13T03:29:28.692 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_inodestats": 0 2023-09-13T03:29:28.692 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.692 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_log": { 2023-09-13T03:29:28.692 INFO:teuthology.orchestra.run.smithi142.stdout: "evlrg": 0, 2023-09-13T03:29:28.693 INFO:teuthology.orchestra.run.smithi142.stdout: "evadd": 83672, 2023-09-13T03:29:28.693 INFO:teuthology.orchestra.run.smithi142.stdout: "evex": 81565, 2023-09-13T03:29:28.693 INFO:teuthology.orchestra.run.smithi142.stdout: "evtrm": 81565, 2023-09-13T03:29:28.694 INFO:teuthology.orchestra.run.smithi142.stdout: "ev": 2107, 2023-09-13T03:29:28.694 INFO:teuthology.orchestra.run.smithi142.stdout: "evexg": 0, 2023-09-13T03:29:28.694 INFO:teuthology.orchestra.run.smithi142.stdout: "evexd": 4093, 2023-09-13T03:29:28.694 INFO:teuthology.orchestra.run.smithi142.stdout: "segadd": 185, 2023-09-13T03:29:28.695 INFO:teuthology.orchestra.run.smithi142.stdout: "segex": 182, 2023-09-13T03:29:28.695 INFO:teuthology.orchestra.run.smithi142.stdout: "segtrm": 182, 2023-09-13T03:29:28.695 INFO:teuthology.orchestra.run.smithi142.stdout: "seg": 3, 2023-09-13T03:29:28.695 INFO:teuthology.orchestra.run.smithi142.stdout: "segmjr": 1, 2023-09-13T03:29:28.696 INFO:teuthology.orchestra.run.smithi142.stdout: "segexg": 0, 2023-09-13T03:29:28.696 INFO:teuthology.orchestra.run.smithi142.stdout: "segexd": 5, 2023-09-13T03:29:28.696 INFO:teuthology.orchestra.run.smithi142.stdout: "expos": 320478286, 2023-09-13T03:29:28.696 INFO:teuthology.orchestra.run.smithi142.stdout: "wrpos": 325428843, 2023-09-13T03:29:28.697 INFO:teuthology.orchestra.run.smithi142.stdout: "rdpos": 0, 2023-09-13T03:29:28.697 INFO:teuthology.orchestra.run.smithi142.stdout: "jlat": { 2023-09-13T03:29:28.697 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 38413, 2023-09-13T03:29:28.697 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 177.250724892, 2023-09-13T03:29:28.698 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.004614342 2023-09-13T03:29:28.698 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.701 INFO:teuthology.orchestra.run.smithi142.stdout: "replayed": 0 2023-09-13T03:29:28.702 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.702 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_mem": { 2023-09-13T03:29:28.702 INFO:teuthology.orchestra.run.smithi142.stdout: "ino": 953, 2023-09-13T03:29:28.702 INFO:teuthology.orchestra.run.smithi142.stdout: "ino+": 16637, 2023-09-13T03:29:28.703 INFO:teuthology.orchestra.run.smithi142.stdout: "ino-": 15684, 2023-09-13T03:29:28.703 INFO:teuthology.orchestra.run.smithi142.stdout: "dir": 135, 2023-09-13T03:29:28.703 INFO:teuthology.orchestra.run.smithi142.stdout: "dir+": 292, 2023-09-13T03:29:28.703 INFO:teuthology.orchestra.run.smithi142.stdout: "dir-": 157, 2023-09-13T03:29:28.704 INFO:teuthology.orchestra.run.smithi142.stdout: "dn": 2055, 2023-09-13T03:29:28.704 INFO:teuthology.orchestra.run.smithi142.stdout: "dn+": 30710, 2023-09-13T03:29:28.704 INFO:teuthology.orchestra.run.smithi142.stdout: "dn-": 28655, 2023-09-13T03:29:28.705 INFO:teuthology.orchestra.run.smithi142.stdout: "cap": 505, 2023-09-13T03:29:28.705 INFO:teuthology.orchestra.run.smithi142.stdout: "cap+": 15656, 2023-09-13T03:29:28.705 INFO:teuthology.orchestra.run.smithi142.stdout: "cap-": 15151, 2023-09-13T03:29:28.705 INFO:teuthology.orchestra.run.smithi142.stdout: "rss": 80840, 2023-09-13T03:29:28.706 INFO:teuthology.orchestra.run.smithi142.stdout: "heap": 207152 2023-09-13T03:29:28.706 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.706 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_server": { 2023-09-13T03:29:28.706 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_client_request": 162082, 2023-09-13T03:29:28.707 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_server_request": 15450, 2023-09-13T03:29:28.707 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_request": 109890, 2023-09-13T03:29:28.707 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_session": 1922, 2023-09-13T03:29:28.708 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_peer_request": 34615, 2023-09-13T03:29:28.708 INFO:teuthology.orchestra.run.smithi142.stdout: "req_create_latency": { 2023-09-13T03:29:28.708 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 14491, 2023-09-13T03:29:28.708 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 90.152088954, 2023-09-13T03:29:28.709 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.006221246 2023-09-13T03:29:28.709 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.709 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getattr_latency": { 2023-09-13T03:29:28.709 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 16634, 2023-09-13T03:29:28.710 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 95.730633250, 2023-09-13T03:29:28.710 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.005755118 2023-09-13T03:29:28.710 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.711 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getfilelock_latency": { 2023-09-13T03:29:28.711 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:28.711 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:28.711 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:28.712 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.712 INFO:teuthology.orchestra.run.smithi142.stdout: "req_link_latency": { 2023-09-13T03:29:28.712 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:28.712 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:28.713 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:28.713 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.713 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookup_latency": { 2023-09-13T03:29:28.713 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 21148, 2023-09-13T03:29:28.714 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 260.530306103, 2023-09-13T03:29:28.714 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.012319382 2023-09-13T03:29:28.714 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.715 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookuphash_latency": { 2023-09-13T03:29:28.715 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:28.715 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:28.715 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:28.716 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.716 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupino_latency": { 2023-09-13T03:29:28.716 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1, 2023-09-13T03:29:28.716 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000211925, 2023-09-13T03:29:28.717 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000211925 2023-09-13T03:29:28.717 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.717 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupname_latency": { 2023-09-13T03:29:28.717 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:28.718 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:28.718 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:28.718 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.719 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupparent_latency": { 2023-09-13T03:29:28.719 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:28.719 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:28.719 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:28.720 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.720 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupsnap_latency": { 2023-09-13T03:29:28.720 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:28.720 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:28.721 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:28.721 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.721 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lssnap_latency": { 2023-09-13T03:29:28.722 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 16, 2023-09-13T03:29:28.722 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.004535198, 2023-09-13T03:29:28.722 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000283449 2023-09-13T03:29:28.722 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.723 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mkdir_latency": { 2023-09-13T03:29:28.723 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 119, 2023-09-13T03:29:28.723 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.796929257, 2023-09-13T03:29:28.724 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.006696884 2023-09-13T03:29:28.724 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.724 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mknod_latency": { 2023-09-13T03:29:28.724 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:28.725 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:28.725 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:28.725 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.725 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mksnap_latency": { 2023-09-13T03:29:28.726 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 16, 2023-09-13T03:29:28.726 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 7.673823889, 2023-09-13T03:29:28.726 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.479613993 2023-09-13T03:29:28.726 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.727 INFO:teuthology.orchestra.run.smithi142.stdout: "req_open_latency": { 2023-09-13T03:29:28.727 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 9, 2023-09-13T03:29:28.727 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.011762633, 2023-09-13T03:29:28.727 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001306959 2023-09-13T03:29:28.728 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.728 INFO:teuthology.orchestra.run.smithi142.stdout: "req_readdir_latency": { 2023-09-13T03:29:28.728 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 25147, 2023-09-13T03:29:28.729 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 153.845881523, 2023-09-13T03:29:28.729 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.006117862 2023-09-13T03:29:28.729 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.729 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rename_latency": { 2023-09-13T03:29:28.730 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 3045, 2023-09-13T03:29:28.730 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 15.223751911, 2023-09-13T03:29:28.730 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.004999590 2023-09-13T03:29:28.730 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.731 INFO:teuthology.orchestra.run.smithi142.stdout: "req_renamesnap_latency": { 2023-09-13T03:29:28.731 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:28.731 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:28.732 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:28.732 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.732 INFO:teuthology.orchestra.run.smithi142.stdout: "req_snapdiff_latency": { 2023-09-13T03:29:28.732 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:28.733 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:28.733 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:28.733 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.733 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmdir_latency": { 2023-09-13T03:29:28.734 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1, 2023-09-13T03:29:28.734 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000819861, 2023-09-13T03:29:28.734 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000819861 2023-09-13T03:29:28.734 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.735 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmsnap_latency": { 2023-09-13T03:29:28.735 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 10, 2023-09-13T03:29:28.735 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 7.861886584, 2023-09-13T03:29:28.735 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.786188658 2023-09-13T03:29:28.736 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.736 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmxattr_latency": { 2023-09-13T03:29:28.736 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:28.736 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:28.737 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:28.737 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.737 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setattr_latency": { 2023-09-13T03:29:28.737 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 27, 2023-09-13T03:29:28.738 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 1.852244271, 2023-09-13T03:29:28.738 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.068601639 2023-09-13T03:29:28.738 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.739 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setdirlayout_latency": { 2023-09-13T03:29:28.739 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:28.739 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:28.739 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:28.740 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.740 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setfilelock_latency": { 2023-09-13T03:29:28.740 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 328, 2023-09-13T03:29:28.740 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.931179685, 2023-09-13T03:29:28.741 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.002838962 2023-09-13T03:29:28.741 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.741 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setlayout_latency": { 2023-09-13T03:29:28.741 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:28.741 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:28.742 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:28.742 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.742 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setxattr_latency": { 2023-09-13T03:29:28.743 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 6, 2023-09-13T03:29:28.743 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.016708153, 2023-09-13T03:29:28.743 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.002784692 2023-09-13T03:29:28.743 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.744 INFO:teuthology.orchestra.run.smithi142.stdout: "req_symlink_latency": { 2023-09-13T03:29:28.744 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:28.744 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:28.745 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:28.745 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.745 INFO:teuthology.orchestra.run.smithi142.stdout: "req_unlink_latency": { 2023-09-13T03:29:28.745 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 13727, 2023-09-13T03:29:28.745 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 72.576562578, 2023-09-13T03:29:28.746 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.005287139 2023-09-13T03:29:28.746 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.746 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_revoke_eviction": 0, 2023-09-13T03:29:28.747 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_acquisition_throttle": 0, 2023-09-13T03:29:28.747 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getvxattr_latency": { 2023-09-13T03:29:28.747 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:28.747 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:28.748 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:28.748 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-09-13T03:29:28.748 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.748 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_sessions": { 2023-09-13T03:29:28.749 INFO:teuthology.orchestra.run.smithi142.stdout: "session_count": 2, 2023-09-13T03:29:28.749 INFO:teuthology.orchestra.run.smithi142.stdout: "session_add": 5, 2023-09-13T03:29:28.749 INFO:teuthology.orchestra.run.smithi142.stdout: "session_remove": 3, 2023-09-13T03:29:28.749 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_open": 2, 2023-09-13T03:29:28.750 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_stale": 0, 2023-09-13T03:29:28.750 INFO:teuthology.orchestra.run.smithi142.stdout: "total_load": 25276, 2023-09-13T03:29:28.750 INFO:teuthology.orchestra.run.smithi142.stdout: "average_load": 12638, 2023-09-13T03:29:28.750 INFO:teuthology.orchestra.run.smithi142.stdout: "avg_session_uptime": 958, 2023-09-13T03:29:28.751 INFO:teuthology.orchestra.run.smithi142.stdout: "mdthresh_evicted": 0 2023-09-13T03:29:28.751 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.751 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2023-09-13T03:29:28.751 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 360, 2023-09-13T03:29:28.752 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 360, 2023-09-13T03:29:28.752 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2023-09-13T03:29:28.752 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2023-09-13T03:29:28.753 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2023-09-13T03:29:28.753 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2023-09-13T03:29:28.753 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-13T03:29:28.753 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2023-09-13T03:29:28.754 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-13T03:29:28.754 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2023-09-13T03:29:28.754 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2023-09-13T03:29:28.754 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2023-09-13T03:29:28.755 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-13T03:29:28.755 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2023-09-13T03:29:28.755 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2023-09-13T03:29:28.756 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2023-09-13T03:29:28.756 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2023-09-13T03:29:28.756 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2023-09-13T03:29:28.756 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-13T03:29:28.757 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2023-09-13T03:29:28.757 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-13T03:29:28.757 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2023-09-13T03:29:28.757 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2023-09-13T03:29:28.758 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2023-09-13T03:29:28.758 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2023-09-13T03:29:28.758 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2023-09-13T03:29:28.759 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-13T03:29:28.759 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2023-09-13T03:29:28.759 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2023-09-13T03:29:28.760 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2023-09-13T03:29:28.760 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2023-09-13T03:29:28.760 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2023-09-13T03:29:28.760 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2023-09-13T03:29:28.761 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2023-09-13T03:29:28.761 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2023-09-13T03:29:28.761 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2023-09-13T03:29:28.762 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 355745, 2023-09-13T03:29:28.762 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 905, 2023-09-13T03:29:28.762 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 176, 2023-09-13T03:29:28.762 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 2, 2023-09-13T03:29:28.763 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2023-09-13T03:29:28.763 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2023-09-13T03:29:28.763 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2023-09-13T03:29:28.764 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2023-09-13T03:29:28.764 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2023-09-13T03:29:28.764 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2023-09-13T03:29:28.764 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 7936, 2023-09-13T03:29:28.765 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 165, 2023-09-13T03:29:28.765 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2023-09-13T03:29:28.765 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2023-09-13T03:29:28.765 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2023-09-13T03:29:28.766 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2023-09-13T03:29:28.766 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 4319072, 2023-09-13T03:29:28.766 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 85396, 2023-09-13T03:29:28.767 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2023-09-13T03:29:28.767 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2023-09-13T03:29:28.767 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2023-09-13T03:29:28.767 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2023-09-13T03:29:28.768 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.768 INFO:teuthology.orchestra.run.smithi142.stdout: "objecter": { 2023-09-13T03:29:28.768 INFO:teuthology.orchestra.run.smithi142.stdout: "op_active": 0, 2023-09-13T03:29:28.769 INFO:teuthology.orchestra.run.smithi142.stdout: "op_laggy": 0, 2023-09-13T03:29:28.769 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send": 63747, 2023-09-13T03:29:28.769 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send_bytes": 338169194, 2023-09-13T03:29:28.769 INFO:teuthology.orchestra.run.smithi142.stdout: "op_resend": 37, 2023-09-13T03:29:28.770 INFO:teuthology.orchestra.run.smithi142.stdout: "op_reply": 63747, 2023-09-13T03:29:28.770 INFO:teuthology.orchestra.run.smithi142.stdout: "op_latency": { 2023-09-13T03:29:28.770 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 63747, 2023-09-13T03:29:28.771 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 810.062143990, 2023-09-13T03:29:28.771 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.012707455 2023-09-13T03:29:28.771 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.771 INFO:teuthology.orchestra.run.smithi142.stdout: "op_inflight": 0, 2023-09-13T03:29:28.772 INFO:teuthology.orchestra.run.smithi142.stdout: "oplen_avg": { 2023-09-13T03:29:28.772 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 63747, 2023-09-13T03:29:28.772 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 82708 2023-09-13T03:29:28.772 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.773 INFO:teuthology.orchestra.run.smithi142.stdout: "op": 63747, 2023-09-13T03:29:28.773 INFO:teuthology.orchestra.run.smithi142.stdout: "op_r": 706, 2023-09-13T03:29:28.773 INFO:teuthology.orchestra.run.smithi142.stdout: "op_w": 63041, 2023-09-13T03:29:28.774 INFO:teuthology.orchestra.run.smithi142.stdout: "op_rmw": 0, 2023-09-13T03:29:28.774 INFO:teuthology.orchestra.run.smithi142.stdout: "op_pg": 0, 2023-09-13T03:29:28.774 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_stat": 1619, 2023-09-13T03:29:28.775 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_create": 7286, 2023-09-13T03:29:28.775 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_read": 533, 2023-09-13T03:29:28.775 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_write": 38942, 2023-09-13T03:29:28.775 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writefull": 586, 2023-09-13T03:29:28.776 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writesame": 0, 2023-09-13T03:29:28.776 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_append": 0, 2023-09-13T03:29:28.776 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_zero": 0, 2023-09-13T03:29:28.776 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_truncate": 9, 2023-09-13T03:29:28.777 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_delete": 14079, 2023-09-13T03:29:28.777 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_mapext": 0, 2023-09-13T03:29:28.777 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_sparse_read": 0, 2023-09-13T03:29:28.777 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_clonerange": 0, 2023-09-13T03:29:28.778 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_getxattr": 152, 2023-09-13T03:29:28.778 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_setxattr": 14572, 2023-09-13T03:29:28.778 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_cmpxattr": 0, 2023-09-13T03:29:28.779 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_rmxattr": 0, 2023-09-13T03:29:28.779 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_resetxattrs": 0, 2023-09-13T03:29:28.779 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_call": 0, 2023-09-13T03:29:28.779 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_watch": 0, 2023-09-13T03:29:28.780 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_notify": 0, 2023-09-13T03:29:28.780 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_src_cmpxattr": 0, 2023-09-13T03:29:28.780 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls": 0, 2023-09-13T03:29:28.780 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls_filter": 0, 2023-09-13T03:29:28.781 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_other": 0, 2023-09-13T03:29:28.781 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_active": 0, 2023-09-13T03:29:28.781 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_send": 0, 2023-09-13T03:29:28.781 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_resend": 0, 2023-09-13T03:29:28.782 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_ping": 0, 2023-09-13T03:29:28.782 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_active": 0, 2023-09-13T03:29:28.782 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_send": 0, 2023-09-13T03:29:28.783 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_resend": 0, 2023-09-13T03:29:28.783 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_active": 0, 2023-09-13T03:29:28.783 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_send": 0, 2023-09-13T03:29:28.783 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_resend": 0, 2023-09-13T03:29:28.784 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_active": 0, 2023-09-13T03:29:28.784 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_send": 0, 2023-09-13T03:29:28.784 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_resend": 0, 2023-09-13T03:29:28.785 INFO:teuthology.orchestra.run.smithi142.stdout: "command_active": 0, 2023-09-13T03:29:28.785 INFO:teuthology.orchestra.run.smithi142.stdout: "command_send": 0, 2023-09-13T03:29:28.785 INFO:teuthology.orchestra.run.smithi142.stdout: "command_resend": 0, 2023-09-13T03:29:28.785 INFO:teuthology.orchestra.run.smithi142.stdout: "map_epoch": 106, 2023-09-13T03:29:28.786 INFO:teuthology.orchestra.run.smithi142.stdout: "map_full": 0, 2023-09-13T03:29:28.786 INFO:teuthology.orchestra.run.smithi142.stdout: "map_inc": 24, 2023-09-13T03:29:28.786 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_sessions": 12, 2023-09-13T03:29:28.786 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_open": 12, 2023-09-13T03:29:28.787 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_close": 0, 2023-09-13T03:29:28.787 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_laggy": 0, 2023-09-13T03:29:28.787 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_wr": 3954, 2023-09-13T03:29:28.788 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_rd": 117, 2023-09-13T03:29:28.788 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_del": 859 2023-09-13T03:29:28.788 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.788 INFO:teuthology.orchestra.run.smithi142.stdout: "oft": { 2023-09-13T03:29:28.789 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_objs": 1, 2023-09-13T03:29:28.789 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_kv_pairs": 0, 2023-09-13T03:29:28.789 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_updates": 0, 2023-09-13T03:29:28.790 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_removes": 0 2023-09-13T03:29:28.790 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.790 INFO:teuthology.orchestra.run.smithi142.stdout: "osd-slow-ops": { 2023-09-13T03:29:28.790 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_ops_count": 1 2023-09-13T03:29:28.791 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.791 INFO:teuthology.orchestra.run.smithi142.stdout: "purge_queue": { 2023-09-13T03:29:28.791 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops": 0, 2023-09-13T03:29:28.792 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops_high_water": 6, 2023-09-13T03:29:28.792 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing": 0, 2023-09-13T03:29:28.792 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_high_water": 6, 2023-09-13T03:29:28.792 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed": 13377, 2023-09-13T03:29:28.793 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_item_in_journal": 0 2023-09-13T03:29:28.793 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.793 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-13T03:29:28.794 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2023-09-13T03:29:28.794 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2023-09-13T03:29:28.794 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2023-09-13T03:29:28.794 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 439439, 2023-09-13T03:29:28.795 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 90814093, 2023-09-13T03:29:28.795 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:28.795 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 439439, 2023-09-13T03:29:28.796 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2023-09-13T03:29:28.796 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2023-09-13T03:29:28.796 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 439439, 2023-09-13T03:29:28.796 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 90814093, 2023-09-13T03:29:28.797 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2023-09-13T03:29:28.797 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:28.797 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:28.797 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:28.798 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-09-13T03:29:28.798 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.798 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_bytes": { 2023-09-13T03:29:28.799 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2023-09-13T03:29:28.799 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2023-09-13T03:29:28.799 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2023-09-13T03:29:28.799 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2023-09-13T03:29:28.800 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2023-09-13T03:29:28.800 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:28.800 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2023-09-13T03:29:28.801 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 63747, 2023-09-13T03:29:28.801 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 339520790, 2023-09-13T03:29:28.801 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 63747, 2023-09-13T03:29:28.801 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 339520790, 2023-09-13T03:29:28.802 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2023-09-13T03:29:28.802 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:28.802 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:28.802 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:28.803 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-09-13T03:29:28.803 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.803 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_ops": { 2023-09-13T03:29:28.804 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2023-09-13T03:29:28.804 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 1024, 2023-09-13T03:29:28.804 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2023-09-13T03:29:28.804 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2023-09-13T03:29:28.805 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2023-09-13T03:29:28.805 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:28.805 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2023-09-13T03:29:28.805 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 63747, 2023-09-13T03:29:28.806 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 63747, 2023-09-13T03:29:28.806 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 63747, 2023-09-13T03:29:28.806 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 63747, 2023-09-13T03:29:28.807 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2023-09-13T03:29:28.807 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:28.807 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:28.807 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:28.808 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-09-13T03:29:28.808 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.808 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle": { 2023-09-13T03:29:28.809 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2023-09-13T03:29:28.809 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2023-09-13T03:29:28.809 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2023-09-13T03:29:28.809 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 13377, 2023-09-13T03:29:28.810 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 1352949, 2023-09-13T03:29:28.810 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:28.810 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 13377, 2023-09-13T03:29:28.811 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2023-09-13T03:29:28.811 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2023-09-13T03:29:28.811 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 529, 2023-09-13T03:29:28.812 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 1352949, 2023-09-13T03:29:28.812 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2023-09-13T03:29:28.812 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:28.812 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:28.813 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:28.813 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-09-13T03:29:28.813 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:28.813 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle-0x5603047ea6e0": { 2023-09-13T03:29:28.814 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 311359, 2023-09-13T03:29:28.814 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2023-09-13T03:29:28.814 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2023-09-13T03:29:28.815 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 83672, 2023-09-13T03:29:28.815 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 321234539, 2023-09-13T03:29:28.815 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:28.815 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 83672, 2023-09-13T03:29:28.816 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2023-09-13T03:29:28.816 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2023-09-13T03:29:28.816 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 38413, 2023-09-13T03:29:28.816 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 320923180, 2023-09-13T03:29:28.817 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2023-09-13T03:29:28.817 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:28.817 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:28.818 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:28.818 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-09-13T03:29:28.818 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-09-13T03:29:28.819 INFO:teuthology.orchestra.run.smithi142.stdout:} 2023-09-13T03:29:28.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:28 smithi142 ceph-mon[134082]: pgmap v512: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 8.3 MiB/s wr, 316 op/s 2023-09-13T03:29:29.027 INFO:tasks.check_counter:Daemon mds.b mds.exported=26 2023-09-13T03:29:29.027 INFO:tasks.check_counter:Daemon mds.b mds.imported=22 2023-09-13T03:29:29.028 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2023-09-13T03:29:29.028 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=16 2023-09-13T03:29:29.028 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=10 2023-09-13T03:29:29.029 DEBUG:tasks.check_counter:Getting stats from e 2023-09-13T03:29:29.029 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph daemon mds.e perf dump 2023-09-13T03:29:29.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:28 smithi170 ceph-mon[134328]: pgmap v512: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 8.3 MiB/s wr, 316 op/s 2023-09-13T03:29:29.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:28 smithi064 ceph-mon[97675]: pgmap v512: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 8.3 MiB/s wr, 316 op/s 2023-09-13T03:29:29.616 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-13T03:29:30.838 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:30 smithi142 ceph-mon[134082]: pgmap v513: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 6.1 MiB/s wr, 250 op/s 2023-09-13T03:29:30.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:30 smithi170 ceph-mon[134328]: pgmap v513: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 6.1 MiB/s wr, 250 op/s 2023-09-13T03:29:30.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:30 smithi064 ceph-mon[97675]: pgmap v513: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 6.1 MiB/s wr, 250 op/s 2023-09-13T03:29:31.177 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2023-09-13T03:29:31.177 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2023-09-13T03:29:31.178 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 2286, 2023-09-13T03:29:31.178 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 2286, 2023-09-13T03:29:31.178 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 126503647, 2023-09-13T03:29:31.179 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 710160, 2023-09-13T03:29:31.179 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 8, 2023-09-13T03:29:31.179 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 4, 2023-09-13T03:29:31.179 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.245641061, 2023-09-13T03:29:31.180 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.065318784, 2023-09-13T03:29:31.180 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.100410311, 2023-09-13T03:29:31.180 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.061710154, 2023-09-13T03:29:31.180 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T03:29:31.181 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 2285, 2023-09-13T03:29:31.181 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.079681345, 2023-09-13T03:29:31.181 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000034871 2023-09-13T03:29:31.181 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.182 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2023-09-13T03:29:31.182 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:31.182 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:31.183 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:31.183 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.183 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-09-13T03:29:31.183 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 448 2023-09-13T03:29:31.184 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.184 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2023-09-13T03:29:31.184 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 1484, 2023-09-13T03:29:31.185 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 1417, 2023-09-13T03:29:31.185 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 109724548, 2023-09-13T03:29:31.185 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 524139, 2023-09-13T03:29:31.185 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 6, 2023-09-13T03:29:31.186 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 5, 2023-09-13T03:29:31.186 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.196694627, 2023-09-13T03:29:31.186 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.036360729, 2023-09-13T03:29:31.186 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.127763243, 2023-09-13T03:29:31.187 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.036053406, 2023-09-13T03:29:31.187 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T03:29:31.187 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1416, 2023-09-13T03:29:31.187 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.053086549, 2023-09-13T03:29:31.188 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000037490 2023-09-13T03:29:31.188 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.188 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2023-09-13T03:29:31.189 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:31.189 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:31.189 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:31.190 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.190 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 1969824, 2023-09-13T03:29:31.190 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 186016 2023-09-13T03:29:31.190 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.191 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2023-09-13T03:29:31.191 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 804, 2023-09-13T03:29:31.191 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 1031, 2023-09-13T03:29:31.191 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 88283187, 2023-09-13T03:29:31.192 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 804031, 2023-09-13T03:29:31.192 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 8, 2023-09-13T03:29:31.192 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 6, 2023-09-13T03:29:31.192 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.123271126, 2023-09-13T03:29:31.193 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.022122620, 2023-09-13T03:29:31.193 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.070431189, 2023-09-13T03:29:31.193 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.022809833, 2023-09-13T03:29:31.194 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T03:29:31.194 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1028, 2023-09-13T03:29:31.194 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.043380146, 2023-09-13T03:29:31.194 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000042198 2023-09-13T03:29:31.195 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.195 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2023-09-13T03:29:31.195 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:31.195 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:31.196 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:31.196 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.196 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 320, 2023-09-13T03:29:31.197 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 554608 2023-09-13T03:29:31.197 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.197 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2023-09-13T03:29:31.197 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 1, 2023-09-13T03:29:31.198 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2023-09-13T03:29:31.198 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.198 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-MDSRank": { 2023-09-13T03:29:31.199 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2023-09-13T03:29:31.199 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2023-09-13T03:29:31.199 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 7051, 2023-09-13T03:29:31.199 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.336360341, 2023-09-13T03:29:31.200 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000047703 2023-09-13T03:29:31.200 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-09-13T03:29:31.200 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.201 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-PurgeQueue": { 2023-09-13T03:29:31.201 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2023-09-13T03:29:31.201 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2023-09-13T03:29:31.201 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:31.202 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:31.202 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:31.202 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-09-13T03:29:31.203 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.203 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2023-09-13T03:29:31.203 INFO:teuthology.orchestra.run.smithi142.stdout: "request": 0, 2023-09-13T03:29:31.203 INFO:teuthology.orchestra.run.smithi142.stdout: "reply": 0, 2023-09-13T03:29:31.204 INFO:teuthology.orchestra.run.smithi142.stdout: "reply_latency": { 2023-09-13T03:29:31.204 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:31.204 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:31.204 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:31.205 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.205 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_reply": 0, 2023-09-13T03:29:31.205 INFO:teuthology.orchestra.run.smithi142.stdout: "forward": 0, 2023-09-13T03:29:31.206 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_complete": 0, 2023-09-13T03:29:31.206 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_keys": 0, 2023-09-13T03:29:31.206 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_commit": 0, 2023-09-13T03:29:31.206 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_split": 0, 2023-09-13T03:29:31.207 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_merge": 0, 2023-09-13T03:29:31.207 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes": 13461, 2023-09-13T03:29:31.207 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_top": 8708, 2023-09-13T03:29:31.208 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_bottom": 4747, 2023-09-13T03:29:31.208 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pin_tail": 6, 2023-09-13T03:29:31.208 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pinned": 1044, 2023-09-13T03:29:31.208 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_expired": 1021, 2023-09-13T03:29:31.209 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_with_caps": 0, 2023-09-13T03:29:31.209 INFO:teuthology.orchestra.run.smithi142.stdout: "caps": 0, 2023-09-13T03:29:31.209 INFO:teuthology.orchestra.run.smithi142.stdout: "subtrees": 19, 2023-09-13T03:29:31.210 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse": 0, 2023-09-13T03:29:31.210 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_hit": 0, 2023-09-13T03:29:31.210 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_forward": 0, 2023-09-13T03:29:31.210 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_discover": 0, 2023-09-13T03:29:31.211 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_dir_fetch": 0, 2023-09-13T03:29:31.211 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_remote_ino": 0, 2023-09-13T03:29:31.211 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_lock": 0, 2023-09-13T03:29:31.212 INFO:teuthology.orchestra.run.smithi142.stdout: "load_cent": 0, 2023-09-13T03:29:31.212 INFO:teuthology.orchestra.run.smithi142.stdout: "q": 0, 2023-09-13T03:29:31.212 INFO:teuthology.orchestra.run.smithi142.stdout: "exported": 0, 2023-09-13T03:29:31.212 INFO:teuthology.orchestra.run.smithi142.stdout: "exported_inodes": 0, 2023-09-13T03:29:31.213 INFO:teuthology.orchestra.run.smithi142.stdout: "imported": 0, 2023-09-13T03:29:31.213 INFO:teuthology.orchestra.run.smithi142.stdout: "imported_inodes": 0, 2023-09-13T03:29:31.213 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_dir_fetch": 0, 2023-09-13T03:29:31.213 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_backtrace_fetch": 0, 2023-09-13T03:29:31.214 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_peer_discover": 0, 2023-09-13T03:29:31.214 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rfiles": 693, 2023-09-13T03:29:31.214 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rbytes": 211724109, 2023-09-13T03:29:31.215 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rsnaps": 6, 2023-09-13T03:29:31.215 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_fetch": 0, 2023-09-13T03:29:31.215 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_set_tag": 0, 2023-09-13T03:29:31.215 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_repaired": 0, 2023-09-13T03:29:31.216 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_inotable_repaired": 0, 2023-09-13T03:29:31.216 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_inodes": 0, 2023-09-13T03:29:31.216 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_base_inodes": 0, 2023-09-13T03:29:31.216 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dirfrag_rstats": 0, 2023-09-13T03:29:31.217 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_file_inodes": 0, 2023-09-13T03:29:31.217 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_inode_file_caps": 0, 2023-09-13T03:29:31.217 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_revoke": 0, 2023-09-13T03:29:31.218 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_grant": 0, 2023-09-13T03:29:31.218 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_trunc": 0, 2023-09-13T03:29:31.218 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-13T03:29:31.219 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-13T03:29:31.219 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps": 0, 2023-09-13T03:29:31.219 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps_dirty": 0, 2023-09-13T03:29:31.219 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_cap_release": 0, 2023-09-13T03:29:31.220 INFO:teuthology.orchestra.run.smithi142.stdout: "process_request_cap_release": 0 2023-09-13T03:29:31.220 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.220 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_cache": { 2023-09-13T03:29:31.220 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update": 0, 2023-09-13T03:29:31.221 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update_receipt": 0, 2023-09-13T03:29:31.221 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_try_discover": 0, 2023-09-13T03:29:31.221 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_send_discover": 0, 2023-09-13T03:29:31.222 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_handle_discover": 0, 2023-09-13T03:29:31.222 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays": 0, 2023-09-13T03:29:31.222 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_delayed": 0, 2023-09-13T03:29:31.222 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_enqueuing": 0, 2023-09-13T03:29:31.223 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_created": 0, 2023-09-13T03:29:31.223 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_enqueued": 0, 2023-09-13T03:29:31.223 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_reintegrated": 0, 2023-09-13T03:29:31.223 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_migrated": 0, 2023-09-13T03:29:31.224 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_processing": 0, 2023-09-13T03:29:31.224 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_enqueued": 0, 2023-09-13T03:29:31.224 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_prioritized": 0, 2023-09-13T03:29:31.225 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_started": 0, 2023-09-13T03:29:31.225 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_completed": 0, 2023-09-13T03:29:31.225 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_enqueue_scrub": 0, 2023-09-13T03:29:31.225 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_exportdir": 0, 2023-09-13T03:29:31.226 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_flush": 0, 2023-09-13T03:29:31.226 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragmentdir": 0, 2023-09-13T03:29:31.226 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragstats": 0, 2023-09-13T03:29:31.226 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_inodestats": 0 2023-09-13T03:29:31.227 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.227 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_log": { 2023-09-13T03:29:31.227 INFO:teuthology.orchestra.run.smithi142.stdout: "evlrg": 0, 2023-09-13T03:29:31.228 INFO:teuthology.orchestra.run.smithi142.stdout: "evadd": 0, 2023-09-13T03:29:31.228 INFO:teuthology.orchestra.run.smithi142.stdout: "evex": 0, 2023-09-13T03:29:31.228 INFO:teuthology.orchestra.run.smithi142.stdout: "evtrm": 0, 2023-09-13T03:29:31.229 INFO:teuthology.orchestra.run.smithi142.stdout: "ev": 83802, 2023-09-13T03:29:31.229 INFO:teuthology.orchestra.run.smithi142.stdout: "evexg": 0, 2023-09-13T03:29:31.229 INFO:teuthology.orchestra.run.smithi142.stdout: "evexd": 0, 2023-09-13T03:29:31.229 INFO:teuthology.orchestra.run.smithi142.stdout: "segadd": 0, 2023-09-13T03:29:31.230 INFO:teuthology.orchestra.run.smithi142.stdout: "segex": 0, 2023-09-13T03:29:31.230 INFO:teuthology.orchestra.run.smithi142.stdout: "segtrm": 0, 2023-09-13T03:29:31.230 INFO:teuthology.orchestra.run.smithi142.stdout: "seg": 3, 2023-09-13T03:29:31.231 INFO:teuthology.orchestra.run.smithi142.stdout: "segmjr": 73, 2023-09-13T03:29:31.231 INFO:teuthology.orchestra.run.smithi142.stdout: "segexg": 0, 2023-09-13T03:29:31.231 INFO:teuthology.orchestra.run.smithi142.stdout: "segexd": 0, 2023-09-13T03:29:31.231 INFO:teuthology.orchestra.run.smithi142.stdout: "expos": 320478286, 2023-09-13T03:29:31.232 INFO:teuthology.orchestra.run.smithi142.stdout: "wrpos": 325582026, 2023-09-13T03:29:31.232 INFO:teuthology.orchestra.run.smithi142.stdout: "rdpos": 325546622, 2023-09-13T03:29:31.232 INFO:teuthology.orchestra.run.smithi142.stdout: "jlat": { 2023-09-13T03:29:31.232 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:31.233 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:31.233 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:31.233 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.234 INFO:teuthology.orchestra.run.smithi142.stdout: "replayed": 83802 2023-09-13T03:29:31.234 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.234 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_mem": { 2023-09-13T03:29:31.235 INFO:teuthology.orchestra.run.smithi142.stdout: "ino": 773, 2023-09-13T03:29:31.235 INFO:teuthology.orchestra.run.smithi142.stdout: "ino+": 16126, 2023-09-13T03:29:31.235 INFO:teuthology.orchestra.run.smithi142.stdout: "ino-": 15353, 2023-09-13T03:29:31.235 INFO:teuthology.orchestra.run.smithi142.stdout: "dir": 77, 2023-09-13T03:29:31.236 INFO:teuthology.orchestra.run.smithi142.stdout: "dir+": 223, 2023-09-13T03:29:31.236 INFO:teuthology.orchestra.run.smithi142.stdout: "dir-": 146, 2023-09-13T03:29:31.236 INFO:teuthology.orchestra.run.smithi142.stdout: "dn": 13461, 2023-09-13T03:29:31.236 INFO:teuthology.orchestra.run.smithi142.stdout: "dn+": 17008, 2023-09-13T03:29:31.237 INFO:teuthology.orchestra.run.smithi142.stdout: "dn-": 3547, 2023-09-13T03:29:31.237 INFO:teuthology.orchestra.run.smithi142.stdout: "cap": 0, 2023-09-13T03:29:31.237 INFO:teuthology.orchestra.run.smithi142.stdout: "cap+": 0, 2023-09-13T03:29:31.237 INFO:teuthology.orchestra.run.smithi142.stdout: "cap-": 0, 2023-09-13T03:29:31.238 INFO:teuthology.orchestra.run.smithi142.stdout: "rss": 69444, 2023-09-13T03:29:31.238 INFO:teuthology.orchestra.run.smithi142.stdout: "heap": 207152 2023-09-13T03:29:31.238 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.239 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_server": { 2023-09-13T03:29:31.239 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_client_request": 0, 2023-09-13T03:29:31.239 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_server_request": 0, 2023-09-13T03:29:31.239 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_request": 0, 2023-09-13T03:29:31.240 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_session": 0, 2023-09-13T03:29:31.240 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_peer_request": 0, 2023-09-13T03:29:31.240 INFO:teuthology.orchestra.run.smithi142.stdout: "req_create_latency": { 2023-09-13T03:29:31.241 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:31.241 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:31.241 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:31.241 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.242 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getattr_latency": { 2023-09-13T03:29:31.242 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:31.242 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:31.243 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:31.243 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.243 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getfilelock_latency": { 2023-09-13T03:29:31.243 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:31.244 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:31.244 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:31.244 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.245 INFO:teuthology.orchestra.run.smithi142.stdout: "req_link_latency": { 2023-09-13T03:29:31.245 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:31.245 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:31.245 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:31.246 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.246 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookup_latency": { 2023-09-13T03:29:31.246 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:31.247 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:31.247 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:31.247 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.247 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookuphash_latency": { 2023-09-13T03:29:31.248 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:31.248 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:31.248 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:31.248 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.249 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupino_latency": { 2023-09-13T03:29:31.249 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:31.249 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:31.249 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:31.250 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.250 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupname_latency": { 2023-09-13T03:29:31.250 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:31.251 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:31.251 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:31.251 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.251 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupparent_latency": { 2023-09-13T03:29:31.252 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:31.252 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:31.252 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:31.252 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.253 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupsnap_latency": { 2023-09-13T03:29:31.253 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:31.253 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:31.253 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:31.254 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.254 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lssnap_latency": { 2023-09-13T03:29:31.254 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:31.255 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:31.255 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:31.255 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.255 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mkdir_latency": { 2023-09-13T03:29:31.256 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:31.256 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:31.256 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:31.256 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.257 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mknod_latency": { 2023-09-13T03:29:31.257 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:31.257 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:31.258 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:31.258 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.258 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mksnap_latency": { 2023-09-13T03:29:31.259 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:31.259 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:31.259 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:31.259 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.260 INFO:teuthology.orchestra.run.smithi142.stdout: "req_open_latency": { 2023-09-13T03:29:31.260 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:31.260 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:31.260 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:31.261 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.261 INFO:teuthology.orchestra.run.smithi142.stdout: "req_readdir_latency": { 2023-09-13T03:29:31.261 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:31.262 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:31.262 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:31.262 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.262 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rename_latency": { 2023-09-13T03:29:31.263 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:31.263 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:31.263 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:31.263 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.264 INFO:teuthology.orchestra.run.smithi142.stdout: "req_renamesnap_latency": { 2023-09-13T03:29:31.264 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:31.264 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:31.265 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:31.265 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.265 INFO:teuthology.orchestra.run.smithi142.stdout: "req_snapdiff_latency": { 2023-09-13T03:29:31.265 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:31.266 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:31.266 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:31.266 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.266 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmdir_latency": { 2023-09-13T03:29:31.267 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:31.267 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:31.267 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:31.267 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.268 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmsnap_latency": { 2023-09-13T03:29:31.268 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:31.268 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:31.268 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:31.269 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.269 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmxattr_latency": { 2023-09-13T03:29:31.269 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:31.270 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:31.270 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:31.270 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.270 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setattr_latency": { 2023-09-13T03:29:31.271 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:31.271 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:31.271 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:31.271 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.272 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setdirlayout_latency": { 2023-09-13T03:29:31.272 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:31.272 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:31.272 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:31.273 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.273 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setfilelock_latency": { 2023-09-13T03:29:31.273 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:31.273 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:31.274 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:31.274 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.274 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setlayout_latency": { 2023-09-13T03:29:31.275 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:31.275 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:31.275 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:31.275 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.276 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setxattr_latency": { 2023-09-13T03:29:31.276 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:31.276 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:31.276 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:31.277 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.277 INFO:teuthology.orchestra.run.smithi142.stdout: "req_symlink_latency": { 2023-09-13T03:29:31.277 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:31.277 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:31.278 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:31.278 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.278 INFO:teuthology.orchestra.run.smithi142.stdout: "req_unlink_latency": { 2023-09-13T03:29:31.279 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:31.279 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:31.279 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:31.279 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.280 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_revoke_eviction": 0, 2023-09-13T03:29:31.280 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_acquisition_throttle": 0, 2023-09-13T03:29:31.280 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getvxattr_latency": { 2023-09-13T03:29:31.281 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:31.281 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:31.281 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:31.281 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-09-13T03:29:31.282 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.282 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_sessions": { 2023-09-13T03:29:31.282 INFO:teuthology.orchestra.run.smithi142.stdout: "session_count": 2, 2023-09-13T03:29:31.283 INFO:teuthology.orchestra.run.smithi142.stdout: "session_add": 5, 2023-09-13T03:29:31.283 INFO:teuthology.orchestra.run.smithi142.stdout: "session_remove": 3, 2023-09-13T03:29:31.283 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_open": 4, 2023-09-13T03:29:31.283 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_stale": 0, 2023-09-13T03:29:31.284 INFO:teuthology.orchestra.run.smithi142.stdout: "total_load": 0, 2023-09-13T03:29:31.284 INFO:teuthology.orchestra.run.smithi142.stdout: "average_load": 0, 2023-09-13T03:29:31.284 INFO:teuthology.orchestra.run.smithi142.stdout: "avg_session_uptime": 4597, 2023-09-13T03:29:31.284 INFO:teuthology.orchestra.run.smithi142.stdout: "mdthresh_evicted": 0 2023-09-13T03:29:31.285 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.285 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2023-09-13T03:29:31.285 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 0, 2023-09-13T03:29:31.285 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 0, 2023-09-13T03:29:31.286 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2023-09-13T03:29:31.286 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2023-09-13T03:29:31.286 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2023-09-13T03:29:31.286 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2023-09-13T03:29:31.287 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-13T03:29:31.287 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2023-09-13T03:29:31.287 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-13T03:29:31.288 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2023-09-13T03:29:31.288 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2023-09-13T03:29:31.288 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2023-09-13T03:29:31.288 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-13T03:29:31.289 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2023-09-13T03:29:31.289 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2023-09-13T03:29:31.289 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2023-09-13T03:29:31.290 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2023-09-13T03:29:31.290 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2023-09-13T03:29:31.290 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-13T03:29:31.290 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2023-09-13T03:29:31.291 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-13T03:29:31.291 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2023-09-13T03:29:31.291 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2023-09-13T03:29:31.291 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2023-09-13T03:29:31.292 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2023-09-13T03:29:31.292 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2023-09-13T03:29:31.292 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-13T03:29:31.293 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2023-09-13T03:29:31.293 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2023-09-13T03:29:31.293 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2023-09-13T03:29:31.293 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2023-09-13T03:29:31.294 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2023-09-13T03:29:31.294 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2023-09-13T03:29:31.294 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2023-09-13T03:29:31.295 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2023-09-13T03:29:31.295 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2023-09-13T03:29:31.295 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 274885, 2023-09-13T03:29:31.295 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 1079, 2023-09-13T03:29:31.296 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 0, 2023-09-13T03:29:31.296 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 0, 2023-09-13T03:29:31.296 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2023-09-13T03:29:31.296 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2023-09-13T03:29:31.297 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2023-09-13T03:29:31.297 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2023-09-13T03:29:31.297 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2023-09-13T03:29:31.298 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2023-09-13T03:29:31.298 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 7936, 2023-09-13T03:29:31.298 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 165, 2023-09-13T03:29:31.298 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2023-09-13T03:29:31.299 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2023-09-13T03:29:31.299 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2023-09-13T03:29:31.299 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2023-09-13T03:29:31.300 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 11666066, 2023-09-13T03:29:31.300 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 116872, 2023-09-13T03:29:31.300 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2023-09-13T03:29:31.300 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2023-09-13T03:29:31.301 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2023-09-13T03:29:31.301 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2023-09-13T03:29:31.301 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.301 INFO:teuthology.orchestra.run.smithi142.stdout: "objecter": { 2023-09-13T03:29:31.302 INFO:teuthology.orchestra.run.smithi142.stdout: "op_active": 0, 2023-09-13T03:29:31.302 INFO:teuthology.orchestra.run.smithi142.stdout: "op_laggy": 0, 2023-09-13T03:29:31.302 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send": 4171, 2023-09-13T03:29:31.303 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send_bytes": 26, 2023-09-13T03:29:31.303 INFO:teuthology.orchestra.run.smithi142.stdout: "op_resend": 4, 2023-09-13T03:29:31.303 INFO:teuthology.orchestra.run.smithi142.stdout: "op_reply": 4171, 2023-09-13T03:29:31.303 INFO:teuthology.orchestra.run.smithi142.stdout: "op_latency": { 2023-09-13T03:29:31.304 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 4171, 2023-09-13T03:29:31.304 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 4.197922934, 2023-09-13T03:29:31.304 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001006454 2023-09-13T03:29:31.305 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.305 INFO:teuthology.orchestra.run.smithi142.stdout: "op_inflight": 0, 2023-09-13T03:29:31.305 INFO:teuthology.orchestra.run.smithi142.stdout: "oplen_avg": { 2023-09-13T03:29:31.305 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 4171, 2023-09-13T03:29:31.306 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 4172 2023-09-13T03:29:31.306 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.306 INFO:teuthology.orchestra.run.smithi142.stdout: "op": 4171, 2023-09-13T03:29:31.306 INFO:teuthology.orchestra.run.smithi142.stdout: "op_r": 4171, 2023-09-13T03:29:31.307 INFO:teuthology.orchestra.run.smithi142.stdout: "op_w": 0, 2023-09-13T03:29:31.307 INFO:teuthology.orchestra.run.smithi142.stdout: "op_rmw": 0, 2023-09-13T03:29:31.307 INFO:teuthology.orchestra.run.smithi142.stdout: "op_pg": 0, 2023-09-13T03:29:31.307 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_stat": 2245, 2023-09-13T03:29:31.308 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_create": 0, 2023-09-13T03:29:31.308 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_read": 1923, 2023-09-13T03:29:31.308 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_write": 0, 2023-09-13T03:29:31.309 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writefull": 0, 2023-09-13T03:29:31.309 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writesame": 0, 2023-09-13T03:29:31.309 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_append": 0, 2023-09-13T03:29:31.309 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_zero": 0, 2023-09-13T03:29:31.310 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_truncate": 0, 2023-09-13T03:29:31.310 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_delete": 0, 2023-09-13T03:29:31.310 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_mapext": 0, 2023-09-13T03:29:31.311 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_sparse_read": 0, 2023-09-13T03:29:31.311 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_clonerange": 0, 2023-09-13T03:29:31.311 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_getxattr": 2, 2023-09-13T03:29:31.311 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_setxattr": 0, 2023-09-13T03:29:31.312 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_cmpxattr": 0, 2023-09-13T03:29:31.312 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_rmxattr": 0, 2023-09-13T03:29:31.312 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_resetxattrs": 0, 2023-09-13T03:29:31.312 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_call": 0, 2023-09-13T03:29:31.313 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_watch": 0, 2023-09-13T03:29:31.313 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_notify": 0, 2023-09-13T03:29:31.313 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_src_cmpxattr": 0, 2023-09-13T03:29:31.313 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls": 0, 2023-09-13T03:29:31.314 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls_filter": 0, 2023-09-13T03:29:31.314 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_other": 0, 2023-09-13T03:29:31.314 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_active": 0, 2023-09-13T03:29:31.314 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_send": 0, 2023-09-13T03:29:31.315 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_resend": 0, 2023-09-13T03:29:31.315 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_ping": 0, 2023-09-13T03:29:31.315 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_active": 0, 2023-09-13T03:29:31.315 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_send": 0, 2023-09-13T03:29:31.316 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_resend": 0, 2023-09-13T03:29:31.316 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_active": 0, 2023-09-13T03:29:31.316 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_send": 0, 2023-09-13T03:29:31.317 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_resend": 0, 2023-09-13T03:29:31.317 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_active": 0, 2023-09-13T03:29:31.317 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_send": 0, 2023-09-13T03:29:31.317 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_resend": 0, 2023-09-13T03:29:31.318 INFO:teuthology.orchestra.run.smithi142.stdout: "command_active": 0, 2023-09-13T03:29:31.318 INFO:teuthology.orchestra.run.smithi142.stdout: "command_send": 0, 2023-09-13T03:29:31.318 INFO:teuthology.orchestra.run.smithi142.stdout: "command_resend": 0, 2023-09-13T03:29:31.319 INFO:teuthology.orchestra.run.smithi142.stdout: "map_epoch": 106, 2023-09-13T03:29:31.319 INFO:teuthology.orchestra.run.smithi142.stdout: "map_full": 0, 2023-09-13T03:29:31.319 INFO:teuthology.orchestra.run.smithi142.stdout: "map_inc": 23, 2023-09-13T03:29:31.319 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_sessions": 11, 2023-09-13T03:29:31.320 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_open": 11, 2023-09-13T03:29:31.320 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_close": 0, 2023-09-13T03:29:31.320 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_laggy": 0, 2023-09-13T03:29:31.320 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_wr": 0, 2023-09-13T03:29:31.321 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_rd": 2, 2023-09-13T03:29:31.321 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_del": 0 2023-09-13T03:29:31.321 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.321 INFO:teuthology.orchestra.run.smithi142.stdout: "oft": { 2023-09-13T03:29:31.322 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_objs": 0, 2023-09-13T03:29:31.322 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_kv_pairs": 0, 2023-09-13T03:29:31.322 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_updates": 0, 2023-09-13T03:29:31.323 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_removes": 0 2023-09-13T03:29:31.323 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.323 INFO:teuthology.orchestra.run.smithi142.stdout: "osd-slow-ops": { 2023-09-13T03:29:31.323 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_ops_count": 0 2023-09-13T03:29:31.324 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.324 INFO:teuthology.orchestra.run.smithi142.stdout: "purge_queue": { 2023-09-13T03:29:31.324 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops": 0, 2023-09-13T03:29:31.324 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops_high_water": 0, 2023-09-13T03:29:31.325 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing": 0, 2023-09-13T03:29:31.325 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_high_water": 0, 2023-09-13T03:29:31.325 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed": 0, 2023-09-13T03:29:31.325 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_item_in_journal": 0 2023-09-13T03:29:31.326 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.326 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-13T03:29:31.326 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2023-09-13T03:29:31.326 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2023-09-13T03:29:31.327 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2023-09-13T03:29:31.327 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 4570, 2023-09-13T03:29:31.327 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 324077277, 2023-09-13T03:29:31.328 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:31.328 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 4570, 2023-09-13T03:29:31.328 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2023-09-13T03:29:31.328 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2023-09-13T03:29:31.329 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 4570, 2023-09-13T03:29:31.329 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 324077277, 2023-09-13T03:29:31.329 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2023-09-13T03:29:31.330 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:31.330 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:31.330 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:31.330 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-09-13T03:29:31.331 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.331 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_bytes": { 2023-09-13T03:29:31.331 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2023-09-13T03:29:31.331 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2023-09-13T03:29:31.332 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2023-09-13T03:29:31.332 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2023-09-13T03:29:31.332 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2023-09-13T03:29:31.333 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:31.333 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2023-09-13T03:29:31.333 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 4171, 2023-09-13T03:29:31.333 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 321388900, 2023-09-13T03:29:31.334 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 4171, 2023-09-13T03:29:31.334 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 321388900, 2023-09-13T03:29:31.334 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2023-09-13T03:29:31.335 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:31.335 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:31.335 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:31.336 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-09-13T03:29:31.336 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.336 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_ops": { 2023-09-13T03:29:31.336 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2023-09-13T03:29:31.337 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 1024, 2023-09-13T03:29:31.337 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2023-09-13T03:29:31.337 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2023-09-13T03:29:31.337 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2023-09-13T03:29:31.338 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:31.338 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2023-09-13T03:29:31.338 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 4171, 2023-09-13T03:29:31.338 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 4171, 2023-09-13T03:29:31.339 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 4171, 2023-09-13T03:29:31.339 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 4171, 2023-09-13T03:29:31.339 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2023-09-13T03:29:31.339 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:31.340 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:31.340 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:31.340 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-09-13T03:29:31.341 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.341 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle": { 2023-09-13T03:29:31.341 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2023-09-13T03:29:31.341 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2023-09-13T03:29:31.342 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2023-09-13T03:29:31.342 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2023-09-13T03:29:31.342 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2023-09-13T03:29:31.343 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:31.343 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2023-09-13T03:29:31.343 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2023-09-13T03:29:31.343 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2023-09-13T03:29:31.344 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 0, 2023-09-13T03:29:31.344 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 0, 2023-09-13T03:29:31.344 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2023-09-13T03:29:31.344 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:31.345 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:31.345 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:31.345 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-09-13T03:29:31.345 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:31.346 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle-0x56443ac28960": { 2023-09-13T03:29:31.346 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2023-09-13T03:29:31.346 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2023-09-13T03:29:31.347 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2023-09-13T03:29:31.347 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2023-09-13T03:29:31.347 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2023-09-13T03:29:31.347 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:31.348 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2023-09-13T03:29:31.348 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2023-09-13T03:29:31.348 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2023-09-13T03:29:31.348 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 0, 2023-09-13T03:29:31.349 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 0, 2023-09-13T03:29:31.349 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2023-09-13T03:29:31.349 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:31.349 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:31.350 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:31.350 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-09-13T03:29:31.350 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-09-13T03:29:31.351 INFO:teuthology.orchestra.run.smithi142.stdout:} 2023-09-13T03:29:31.586 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2023-09-13T03:29:31.587 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2023-09-13T03:29:31.587 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2023-09-13T03:29:31.587 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2023-09-13T03:29:31.587 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2023-09-13T03:29:31.588 DEBUG:tasks.check_counter:Getting stats from h 2023-09-13T03:29:31.588 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph daemon mds.h perf dump 2023-09-13T03:29:32.150 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-13T03:29:32.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:32 smithi142 ceph-mon[134082]: pgmap v514: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.7 MiB/s wr, 340 op/s 2023-09-13T03:29:33.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:32 smithi170 ceph-mon[134328]: pgmap v514: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.7 MiB/s wr, 340 op/s 2023-09-13T03:29:33.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:32 smithi064 ceph-mon[97675]: pgmap v514: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.7 MiB/s wr, 340 op/s 2023-09-13T03:29:33.521 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2023-09-13T03:29:33.522 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2023-09-13T03:29:33.522 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 26269, 2023-09-13T03:29:33.522 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 26254, 2023-09-13T03:29:33.523 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 6787242, 2023-09-13T03:29:33.523 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 109983763, 2023-09-13T03:29:33.523 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 18, 2023-09-13T03:29:33.523 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 3, 2023-09-13T03:29:33.524 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 1.699519844, 2023-09-13T03:29:33.524 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.629766816, 2023-09-13T03:29:33.524 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.657211533, 2023-09-13T03:29:33.525 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.416192913, 2023-09-13T03:29:33.525 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T03:29:33.525 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 26252, 2023-09-13T03:29:33.525 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 2.139023002, 2023-09-13T03:29:33.526 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000081480 2023-09-13T03:29:33.526 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.526 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2023-09-13T03:29:33.527 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 22319, 2023-09-13T03:29:33.527 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.003801999, 2023-09-13T03:29:33.527 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000170 2023-09-13T03:29:33.528 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.528 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 1824, 2023-09-13T03:29:33.528 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 1616 2023-09-13T03:29:33.528 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.529 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2023-09-13T03:29:33.529 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 30317, 2023-09-13T03:29:33.529 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 29031, 2023-09-13T03:29:33.529 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 10573808, 2023-09-13T03:29:33.530 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 93638664, 2023-09-13T03:29:33.530 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 11, 2023-09-13T03:29:33.530 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 6, 2023-09-13T03:29:33.530 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 2.015051439, 2023-09-13T03:29:33.531 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.702217011, 2023-09-13T03:29:33.531 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.843081009, 2023-09-13T03:29:33.531 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.520476024, 2023-09-13T03:29:33.531 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T03:29:33.532 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 29030, 2023-09-13T03:29:33.532 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 1.808370683, 2023-09-13T03:29:33.532 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000062293 2023-09-13T03:29:33.532 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.533 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2023-09-13T03:29:33.533 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 29954, 2023-09-13T03:29:33.533 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.005287198, 2023-09-13T03:29:33.533 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000176 2023-09-13T03:29:33.534 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.534 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 1992016, 2023-09-13T03:29:33.534 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 642576 2023-09-13T03:29:33.534 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.535 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2023-09-13T03:29:33.535 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 207333, 2023-09-13T03:29:33.535 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 272293, 2023-09-13T03:29:33.536 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 65719983, 2023-09-13T03:29:33.536 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 203294792, 2023-09-13T03:29:33.536 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 18, 2023-09-13T03:29:33.536 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 7, 2023-09-13T03:29:33.536 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 10.904858053, 2023-09-13T03:29:33.537 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 5.055576803, 2023-09-13T03:29:33.537 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 7.214231776, 2023-09-13T03:29:33.537 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.330044688, 2023-09-13T03:29:33.537 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T03:29:33.538 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 272291, 2023-09-13T03:29:33.538 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 16.368943942, 2023-09-13T03:29:33.538 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000060115 2023-09-13T03:29:33.539 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.539 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2023-09-13T03:29:33.539 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 308151, 2023-09-13T03:29:33.539 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.064093083, 2023-09-13T03:29:33.540 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000207 2023-09-13T03:29:33.540 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.540 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 4928, 2023-09-13T03:29:33.540 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 126848 2023-09-13T03:29:33.541 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.541 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2023-09-13T03:29:33.541 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 1, 2023-09-13T03:29:33.541 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2023-09-13T03:29:33.542 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.542 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-MDSRank": { 2023-09-13T03:29:33.542 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2023-09-13T03:29:33.543 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2023-09-13T03:29:33.543 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 39323, 2023-09-13T03:29:33.543 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 39.476704267, 2023-09-13T03:29:33.543 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001003908 2023-09-13T03:29:33.544 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-09-13T03:29:33.544 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.544 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-PurgeQueue": { 2023-09-13T03:29:33.545 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2023-09-13T03:29:33.545 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2023-09-13T03:29:33.545 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 23750, 2023-09-13T03:29:33.545 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 2.552456572, 2023-09-13T03:29:33.546 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000107471 2023-09-13T03:29:33.546 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-09-13T03:29:33.546 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.547 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2023-09-13T03:29:33.547 INFO:teuthology.orchestra.run.smithi142.stdout: "request": 82014, 2023-09-13T03:29:33.547 INFO:teuthology.orchestra.run.smithi142.stdout: "reply": 75009, 2023-09-13T03:29:33.548 INFO:teuthology.orchestra.run.smithi142.stdout: "reply_latency": { 2023-09-13T03:29:33.548 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 75009, 2023-09-13T03:29:33.548 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 259.308059233, 2023-09-13T03:29:33.548 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.003457025 2023-09-13T03:29:33.549 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.549 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_reply": 0, 2023-09-13T03:29:33.549 INFO:teuthology.orchestra.run.smithi142.stdout: "forward": 7005, 2023-09-13T03:29:33.549 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_complete": 10, 2023-09-13T03:29:33.550 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_keys": 0, 2023-09-13T03:29:33.550 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_commit": 106, 2023-09-13T03:29:33.550 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_split": 11, 2023-09-13T03:29:33.551 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_merge": 7, 2023-09-13T03:29:33.551 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes": 2549, 2023-09-13T03:29:33.551 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_top": 633, 2023-09-13T03:29:33.551 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_bottom": 1718, 2023-09-13T03:29:33.552 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pin_tail": 198, 2023-09-13T03:29:33.552 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pinned": 1640, 2023-09-13T03:29:33.552 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_expired": 347, 2023-09-13T03:29:33.553 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_with_caps": 374, 2023-09-13T03:29:33.553 INFO:teuthology.orchestra.run.smithi142.stdout: "caps": 374, 2023-09-13T03:29:33.553 INFO:teuthology.orchestra.run.smithi142.stdout: "subtrees": 16, 2023-09-13T03:29:33.553 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse": 127025, 2023-09-13T03:29:33.554 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_hit": 72815, 2023-09-13T03:29:33.554 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_forward": 7001, 2023-09-13T03:29:33.554 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_discover": 57, 2023-09-13T03:29:33.554 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_dir_fetch": 6, 2023-09-13T03:29:33.555 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_remote_ino": 0, 2023-09-13T03:29:33.555 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_lock": 1374, 2023-09-13T03:29:33.555 INFO:teuthology.orchestra.run.smithi142.stdout: "load_cent": 55890, 2023-09-13T03:29:33.556 INFO:teuthology.orchestra.run.smithi142.stdout: "q": 0, 2023-09-13T03:29:33.556 INFO:teuthology.orchestra.run.smithi142.stdout: "exported": 8, 2023-09-13T03:29:33.556 INFO:teuthology.orchestra.run.smithi142.stdout: "exported_inodes": 252, 2023-09-13T03:29:33.556 INFO:teuthology.orchestra.run.smithi142.stdout: "imported": 9, 2023-09-13T03:29:33.557 INFO:teuthology.orchestra.run.smithi142.stdout: "imported_inodes": 557, 2023-09-13T03:29:33.557 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_dir_fetch": 0, 2023-09-13T03:29:33.557 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_backtrace_fetch": 32, 2023-09-13T03:29:33.558 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_peer_discover": 0, 2023-09-13T03:29:33.558 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rfiles": 693, 2023-09-13T03:29:33.558 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rbytes": 211724109, 2023-09-13T03:29:33.558 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rsnaps": 6, 2023-09-13T03:29:33.559 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_fetch": 32, 2023-09-13T03:29:33.559 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_set_tag": 0, 2023-09-13T03:29:33.559 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_repaired": 0, 2023-09-13T03:29:33.560 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_inotable_repaired": 0, 2023-09-13T03:29:33.560 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_inodes": 0, 2023-09-13T03:29:33.560 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_base_inodes": 0, 2023-09-13T03:29:33.560 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dirfrag_rstats": 0, 2023-09-13T03:29:33.560 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_file_inodes": 32, 2023-09-13T03:29:33.561 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_inode_file_caps": 2, 2023-09-13T03:29:33.561 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_revoke": 9816, 2023-09-13T03:29:33.561 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_grant": 61056, 2023-09-13T03:29:33.562 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_trunc": 0, 2023-09-13T03:29:33.562 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flushsnap_ack": 1989, 2023-09-13T03:29:33.562 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-13T03:29:33.562 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps": 27145, 2023-09-13T03:29:33.563 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps_dirty": 12646, 2023-09-13T03:29:33.563 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_cap_release": 3953, 2023-09-13T03:29:33.563 INFO:teuthology.orchestra.run.smithi142.stdout: "process_request_cap_release": 51779 2023-09-13T03:29:33.563 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.564 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_cache": { 2023-09-13T03:29:33.564 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update": 12, 2023-09-13T03:29:33.564 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update_receipt": 81, 2023-09-13T03:29:33.564 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_try_discover": 53, 2023-09-13T03:29:33.565 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_send_discover": 107, 2023-09-13T03:29:33.565 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_handle_discover": 50, 2023-09-13T03:29:33.565 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays": 174, 2023-09-13T03:29:33.565 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_delayed": 0, 2023-09-13T03:29:33.566 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_enqueuing": 0, 2023-09-13T03:29:33.566 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_created": 12845, 2023-09-13T03:29:33.566 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_enqueued": 12671, 2023-09-13T03:29:33.567 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_reintegrated": 0, 2023-09-13T03:29:33.567 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_migrated": 0, 2023-09-13T03:29:33.567 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_processing": 0, 2023-09-13T03:29:33.567 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_enqueued": 0, 2023-09-13T03:29:33.568 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_prioritized": 0, 2023-09-13T03:29:33.568 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_started": 0, 2023-09-13T03:29:33.568 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_completed": 0, 2023-09-13T03:29:33.568 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_enqueue_scrub": 0, 2023-09-13T03:29:33.569 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_exportdir": 11, 2023-09-13T03:29:33.569 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_flush": 0, 2023-09-13T03:29:33.569 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragmentdir": 18, 2023-09-13T03:29:33.569 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragstats": 0, 2023-09-13T03:29:33.570 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_inodestats": 0 2023-09-13T03:29:33.570 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.570 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_log": { 2023-09-13T03:29:33.570 INFO:teuthology.orchestra.run.smithi142.stdout: "evlrg": 0, 2023-09-13T03:29:33.571 INFO:teuthology.orchestra.run.smithi142.stdout: "evadd": 65264, 2023-09-13T03:29:33.571 INFO:teuthology.orchestra.run.smithi142.stdout: "evex": 0, 2023-09-13T03:29:33.571 INFO:teuthology.orchestra.run.smithi142.stdout: "evtrm": 0, 2023-09-13T03:29:33.571 INFO:teuthology.orchestra.run.smithi142.stdout: "ev": 65264, 2023-09-13T03:29:33.572 INFO:teuthology.orchestra.run.smithi142.stdout: "evexg": 0, 2023-09-13T03:29:33.572 INFO:teuthology.orchestra.run.smithi142.stdout: "evexd": 0, 2023-09-13T03:29:33.572 INFO:teuthology.orchestra.run.smithi142.stdout: "segadd": 95, 2023-09-13T03:29:33.572 INFO:teuthology.orchestra.run.smithi142.stdout: "segex": 0, 2023-09-13T03:29:33.573 INFO:teuthology.orchestra.run.smithi142.stdout: "segtrm": 0, 2023-09-13T03:29:33.573 INFO:teuthology.orchestra.run.smithi142.stdout: "seg": 95, 2023-09-13T03:29:33.573 INFO:teuthology.orchestra.run.smithi142.stdout: "segmjr": 7, 2023-09-13T03:29:33.573 INFO:teuthology.orchestra.run.smithi142.stdout: "segexg": 0, 2023-09-13T03:29:33.574 INFO:teuthology.orchestra.run.smithi142.stdout: "segexd": 0, 2023-09-13T03:29:33.574 INFO:teuthology.orchestra.run.smithi142.stdout: "expos": 4194304, 2023-09-13T03:29:33.574 INFO:teuthology.orchestra.run.smithi142.stdout: "wrpos": 224627639, 2023-09-13T03:29:33.575 INFO:teuthology.orchestra.run.smithi142.stdout: "rdpos": 0, 2023-09-13T03:29:33.575 INFO:teuthology.orchestra.run.smithi142.stdout: "jlat": { 2023-09-13T03:29:33.575 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 21474, 2023-09-13T03:29:33.575 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 123.581032488, 2023-09-13T03:29:33.576 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.005754914 2023-09-13T03:29:33.576 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.576 INFO:teuthology.orchestra.run.smithi142.stdout: "replayed": 0 2023-09-13T03:29:33.576 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.577 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_mem": { 2023-09-13T03:29:33.577 INFO:teuthology.orchestra.run.smithi142.stdout: "ino": 870, 2023-09-13T03:29:33.577 INFO:teuthology.orchestra.run.smithi142.stdout: "ino+": 13724, 2023-09-13T03:29:33.577 INFO:teuthology.orchestra.run.smithi142.stdout: "ino-": 12854, 2023-09-13T03:29:33.578 INFO:teuthology.orchestra.run.smithi142.stdout: "dir": 156, 2023-09-13T03:29:33.578 INFO:teuthology.orchestra.run.smithi142.stdout: "dir+": 265, 2023-09-13T03:29:33.578 INFO:teuthology.orchestra.run.smithi142.stdout: "dir-": 109, 2023-09-13T03:29:33.579 INFO:teuthology.orchestra.run.smithi142.stdout: "dn": 2423, 2023-09-13T03:29:33.579 INFO:teuthology.orchestra.run.smithi142.stdout: "dn+": 24803, 2023-09-13T03:29:33.579 INFO:teuthology.orchestra.run.smithi142.stdout: "dn-": 22380, 2023-09-13T03:29:33.579 INFO:teuthology.orchestra.run.smithi142.stdout: "cap": 307, 2023-09-13T03:29:33.580 INFO:teuthology.orchestra.run.smithi142.stdout: "cap+": 14074, 2023-09-13T03:29:33.580 INFO:teuthology.orchestra.run.smithi142.stdout: "cap-": 13767, 2023-09-13T03:29:33.580 INFO:teuthology.orchestra.run.smithi142.stdout: "rss": 71140, 2023-09-13T03:29:33.581 INFO:teuthology.orchestra.run.smithi142.stdout: "heap": 198960 2023-09-13T03:29:33.581 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.581 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_server": { 2023-09-13T03:29:33.581 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_client_request": 115077, 2023-09-13T03:29:33.582 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_server_request": 1855, 2023-09-13T03:29:33.582 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_request": 82014, 2023-09-13T03:29:33.582 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_session": 2098, 2023-09-13T03:29:33.583 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_peer_request": 7931, 2023-09-13T03:29:33.583 INFO:teuthology.orchestra.run.smithi142.stdout: "req_create_latency": { 2023-09-13T03:29:33.583 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 12811, 2023-09-13T03:29:33.583 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 25.208783484, 2023-09-13T03:29:33.584 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001967745 2023-09-13T03:29:33.584 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.584 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getattr_latency": { 2023-09-13T03:29:33.585 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1771, 2023-09-13T03:29:33.585 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 34.067068724, 2023-09-13T03:29:33.585 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.019236063 2023-09-13T03:29:33.585 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.586 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getfilelock_latency": { 2023-09-13T03:29:33.586 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:33.586 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:33.587 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:33.587 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.587 INFO:teuthology.orchestra.run.smithi142.stdout: "req_link_latency": { 2023-09-13T03:29:33.587 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:33.588 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:33.588 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:33.588 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.589 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookup_latency": { 2023-09-13T03:29:33.589 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 20455, 2023-09-13T03:29:33.589 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 27.149691280, 2023-09-13T03:29:33.589 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001327288 2023-09-13T03:29:33.590 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.590 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookuphash_latency": { 2023-09-13T03:29:33.590 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:33.591 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:33.591 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:33.591 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.592 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupino_latency": { 2023-09-13T03:29:33.592 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:33.592 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:33.592 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:33.593 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.593 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupname_latency": { 2023-09-13T03:29:33.593 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:33.594 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:33.594 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:33.594 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.594 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupparent_latency": { 2023-09-13T03:29:33.595 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:33.595 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:33.595 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:33.596 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.596 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupsnap_latency": { 2023-09-13T03:29:33.596 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:33.596 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:33.597 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:33.597 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.597 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lssnap_latency": { 2023-09-13T03:29:33.598 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:33.598 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:33.598 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:33.599 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.599 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mkdir_latency": { 2023-09-13T03:29:33.599 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:33.599 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:33.600 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:33.600 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.600 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mknod_latency": { 2023-09-13T03:29:33.600 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:33.601 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:33.601 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:33.601 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.602 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mksnap_latency": { 2023-09-13T03:29:33.602 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:33.602 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:33.602 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:33.603 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.603 INFO:teuthology.orchestra.run.smithi142.stdout: "req_open_latency": { 2023-09-13T03:29:33.603 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:33.604 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:33.604 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:33.604 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.604 INFO:teuthology.orchestra.run.smithi142.stdout: "req_readdir_latency": { 2023-09-13T03:29:33.605 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 23137, 2023-09-13T03:29:33.605 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 111.690608247, 2023-09-13T03:29:33.605 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.004827359 2023-09-13T03:29:33.606 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.606 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rename_latency": { 2023-09-13T03:29:33.606 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 3213, 2023-09-13T03:29:33.606 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 34.383800648, 2023-09-13T03:29:33.607 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.010701463 2023-09-13T03:29:33.607 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.607 INFO:teuthology.orchestra.run.smithi142.stdout: "req_renamesnap_latency": { 2023-09-13T03:29:33.608 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:33.608 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:33.608 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:33.608 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.609 INFO:teuthology.orchestra.run.smithi142.stdout: "req_snapdiff_latency": { 2023-09-13T03:29:33.609 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:33.609 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:33.609 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:33.610 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.610 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmdir_latency": { 2023-09-13T03:29:33.610 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:33.610 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:33.611 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:33.611 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.611 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmsnap_latency": { 2023-09-13T03:29:33.611 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:33.612 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:33.612 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:33.612 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.612 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmxattr_latency": { 2023-09-13T03:29:33.613 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:33.613 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:33.613 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:33.613 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.614 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setattr_latency": { 2023-09-13T03:29:33.614 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1, 2023-09-13T03:29:33.614 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.013850233, 2023-09-13T03:29:33.615 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.013850233 2023-09-13T03:29:33.615 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.615 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setdirlayout_latency": { 2023-09-13T03:29:33.617 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:33.618 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:33.618 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:33.619 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.619 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setfilelock_latency": { 2023-09-13T03:29:33.619 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 776, 2023-09-13T03:29:33.620 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.509513892, 2023-09-13T03:29:33.620 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000656590 2023-09-13T03:29:33.620 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.620 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setlayout_latency": { 2023-09-13T03:29:33.621 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:33.621 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:33.621 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:33.621 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.622 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setxattr_latency": { 2023-09-13T03:29:33.622 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:33.622 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:33.623 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:33.623 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.623 INFO:teuthology.orchestra.run.smithi142.stdout: "req_symlink_latency": { 2023-09-13T03:29:33.623 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:33.624 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:33.624 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:33.624 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.625 INFO:teuthology.orchestra.run.smithi142.stdout: "req_unlink_latency": { 2023-09-13T03:29:33.625 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 12845, 2023-09-13T03:29:33.625 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 26.284742725, 2023-09-13T03:29:33.625 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.002046301 2023-09-13T03:29:33.626 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.626 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_revoke_eviction": 0, 2023-09-13T03:29:33.626 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_acquisition_throttle": 0, 2023-09-13T03:29:33.627 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getvxattr_latency": { 2023-09-13T03:29:33.627 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:33.628 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:33.628 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:33.629 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-09-13T03:29:33.629 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.630 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_sessions": { 2023-09-13T03:29:33.630 INFO:teuthology.orchestra.run.smithi142.stdout: "session_count": 1, 2023-09-13T03:29:33.630 INFO:teuthology.orchestra.run.smithi142.stdout: "session_add": 4, 2023-09-13T03:29:33.631 INFO:teuthology.orchestra.run.smithi142.stdout: "session_remove": 3, 2023-09-13T03:29:33.631 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_open": 1, 2023-09-13T03:29:33.631 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_stale": 0, 2023-09-13T03:29:33.631 INFO:teuthology.orchestra.run.smithi142.stdout: "total_load": 23913, 2023-09-13T03:29:33.632 INFO:teuthology.orchestra.run.smithi142.stdout: "average_load": 23913, 2023-09-13T03:29:33.632 INFO:teuthology.orchestra.run.smithi142.stdout: "avg_session_uptime": 932, 2023-09-13T03:29:33.632 INFO:teuthology.orchestra.run.smithi142.stdout: "mdthresh_evicted": 0 2023-09-13T03:29:33.632 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.633 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2023-09-13T03:29:33.633 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 0, 2023-09-13T03:29:33.633 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 0, 2023-09-13T03:29:33.634 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2023-09-13T03:29:33.634 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2023-09-13T03:29:33.634 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2023-09-13T03:29:33.635 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2023-09-13T03:29:33.635 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-13T03:29:33.635 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2023-09-13T03:29:33.635 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-13T03:29:33.636 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2023-09-13T03:29:33.636 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2023-09-13T03:29:33.636 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2023-09-13T03:29:33.637 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-13T03:29:33.637 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2023-09-13T03:29:33.637 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2023-09-13T03:29:33.637 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2023-09-13T03:29:33.638 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2023-09-13T03:29:33.638 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2023-09-13T03:29:33.638 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-13T03:29:33.639 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2023-09-13T03:29:33.639 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-13T03:29:33.639 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2023-09-13T03:29:33.640 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2023-09-13T03:29:33.640 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2023-09-13T03:29:33.640 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2023-09-13T03:29:33.640 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2023-09-13T03:29:33.641 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-13T03:29:33.641 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2023-09-13T03:29:33.641 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2023-09-13T03:29:33.642 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2023-09-13T03:29:33.642 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2023-09-13T03:29:33.642 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2023-09-13T03:29:33.643 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2023-09-13T03:29:33.643 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2023-09-13T03:29:33.643 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2023-09-13T03:29:33.643 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2023-09-13T03:29:33.644 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 2669314, 2023-09-13T03:29:33.644 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 1504, 2023-09-13T03:29:33.644 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 176, 2023-09-13T03:29:33.645 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 2, 2023-09-13T03:29:33.645 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2023-09-13T03:29:33.645 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2023-09-13T03:29:33.645 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2023-09-13T03:29:33.646 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2023-09-13T03:29:33.646 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2023-09-13T03:29:33.646 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2023-09-13T03:29:33.647 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 7936, 2023-09-13T03:29:33.647 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 165, 2023-09-13T03:29:33.647 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2023-09-13T03:29:33.648 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2023-09-13T03:29:33.648 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2023-09-13T03:29:33.648 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2023-09-13T03:29:33.648 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 4942307, 2023-09-13T03:29:33.649 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 101595, 2023-09-13T03:29:33.649 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2023-09-13T03:29:33.649 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2023-09-13T03:29:33.649 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2023-09-13T03:29:33.650 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2023-09-13T03:29:33.650 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.650 INFO:teuthology.orchestra.run.smithi142.stdout: "objecter": { 2023-09-13T03:29:33.651 INFO:teuthology.orchestra.run.smithi142.stdout: "op_active": 0, 2023-09-13T03:29:33.651 INFO:teuthology.orchestra.run.smithi142.stdout: "op_laggy": 0, 2023-09-13T03:29:33.651 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send": 36125, 2023-09-13T03:29:33.651 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send_bytes": 223740673, 2023-09-13T03:29:33.652 INFO:teuthology.orchestra.run.smithi142.stdout: "op_resend": 27, 2023-09-13T03:29:33.652 INFO:teuthology.orchestra.run.smithi142.stdout: "op_reply": 36125, 2023-09-13T03:29:33.652 INFO:teuthology.orchestra.run.smithi142.stdout: "op_latency": { 2023-09-13T03:29:33.653 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 36125, 2023-09-13T03:29:33.653 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 200.170330836, 2023-09-13T03:29:33.653 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.005541047 2023-09-13T03:29:33.654 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.654 INFO:teuthology.orchestra.run.smithi142.stdout: "op_inflight": 0, 2023-09-13T03:29:33.654 INFO:teuthology.orchestra.run.smithi142.stdout: "oplen_avg": { 2023-09-13T03:29:33.655 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 36125, 2023-09-13T03:29:33.655 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 36595 2023-09-13T03:29:33.655 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.655 INFO:teuthology.orchestra.run.smithi142.stdout: "op": 36125, 2023-09-13T03:29:33.656 INFO:teuthology.orchestra.run.smithi142.stdout: "op_r": 630, 2023-09-13T03:29:33.656 INFO:teuthology.orchestra.run.smithi142.stdout: "op_w": 35495, 2023-09-13T03:29:33.656 INFO:teuthology.orchestra.run.smithi142.stdout: "op_rmw": 0, 2023-09-13T03:29:33.657 INFO:teuthology.orchestra.run.smithi142.stdout: "op_pg": 0, 2023-09-13T03:29:33.657 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_stat": 0, 2023-09-13T03:29:33.657 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_create": 10, 2023-09-13T03:29:33.657 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_read": 576, 2023-09-13T03:29:33.658 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_write": 22050, 2023-09-13T03:29:33.658 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writefull": 352, 2023-09-13T03:29:33.658 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writesame": 0, 2023-09-13T03:29:33.659 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_append": 0, 2023-09-13T03:29:33.659 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_zero": 0, 2023-09-13T03:29:33.659 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_truncate": 11, 2023-09-13T03:29:33.659 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_delete": 12781, 2023-09-13T03:29:33.660 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_mapext": 0, 2023-09-13T03:29:33.660 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_sparse_read": 0, 2023-09-13T03:29:33.660 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_clonerange": 0, 2023-09-13T03:29:33.661 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_getxattr": 41, 2023-09-13T03:29:33.661 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_setxattr": 20, 2023-09-13T03:29:33.661 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_cmpxattr": 0, 2023-09-13T03:29:33.661 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_rmxattr": 0, 2023-09-13T03:29:33.662 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_resetxattrs": 0, 2023-09-13T03:29:33.662 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_call": 0, 2023-09-13T03:29:33.662 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_watch": 0, 2023-09-13T03:29:33.663 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_notify": 0, 2023-09-13T03:29:33.663 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_src_cmpxattr": 0, 2023-09-13T03:29:33.663 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls": 0, 2023-09-13T03:29:33.663 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls_filter": 0, 2023-09-13T03:29:33.664 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_other": 0, 2023-09-13T03:29:33.664 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_active": 0, 2023-09-13T03:29:33.664 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_send": 0, 2023-09-13T03:29:33.664 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_resend": 0, 2023-09-13T03:29:33.665 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_ping": 0, 2023-09-13T03:29:33.665 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_active": 0, 2023-09-13T03:29:33.665 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_send": 0, 2023-09-13T03:29:33.665 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_resend": 0, 2023-09-13T03:29:33.666 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_active": 0, 2023-09-13T03:29:33.666 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_send": 0, 2023-09-13T03:29:33.666 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_resend": 0, 2023-09-13T03:29:33.667 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_active": 0, 2023-09-13T03:29:33.667 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_send": 0, 2023-09-13T03:29:33.667 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_resend": 0, 2023-09-13T03:29:33.667 INFO:teuthology.orchestra.run.smithi142.stdout: "command_active": 0, 2023-09-13T03:29:33.668 INFO:teuthology.orchestra.run.smithi142.stdout: "command_send": 0, 2023-09-13T03:29:33.668 INFO:teuthology.orchestra.run.smithi142.stdout: "command_resend": 0, 2023-09-13T03:29:33.668 INFO:teuthology.orchestra.run.smithi142.stdout: "map_epoch": 106, 2023-09-13T03:29:33.669 INFO:teuthology.orchestra.run.smithi142.stdout: "map_full": 0, 2023-09-13T03:29:33.669 INFO:teuthology.orchestra.run.smithi142.stdout: "map_inc": 23, 2023-09-13T03:29:33.669 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_sessions": 12, 2023-09-13T03:29:33.670 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_open": 12, 2023-09-13T03:29:33.670 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_close": 0, 2023-09-13T03:29:33.670 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_laggy": 0, 2023-09-13T03:29:33.671 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_wr": 551, 2023-09-13T03:29:33.671 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_rd": 32, 2023-09-13T03:29:33.671 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_del": 171 2023-09-13T03:29:33.672 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.672 INFO:teuthology.orchestra.run.smithi142.stdout: "oft": { 2023-09-13T03:29:33.672 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_objs": 1, 2023-09-13T03:29:33.673 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_kv_pairs": 0, 2023-09-13T03:29:33.673 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_updates": 0, 2023-09-13T03:29:33.673 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_removes": 0 2023-09-13T03:29:33.673 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.674 INFO:teuthology.orchestra.run.smithi142.stdout: "osd-slow-ops": { 2023-09-13T03:29:33.674 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_ops_count": 0 2023-09-13T03:29:33.674 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.674 INFO:teuthology.orchestra.run.smithi142.stdout: "purge_queue": { 2023-09-13T03:29:33.675 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops": 0, 2023-09-13T03:29:33.675 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops_high_water": 6, 2023-09-13T03:29:33.675 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing": 0, 2023-09-13T03:29:33.675 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_high_water": 6, 2023-09-13T03:29:33.676 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed": 12671, 2023-09-13T03:29:33.676 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_item_in_journal": 0 2023-09-13T03:29:33.676 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.676 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-13T03:29:33.677 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2023-09-13T03:29:33.677 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2023-09-13T03:29:33.677 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2023-09-13T03:29:33.677 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 263915, 2023-09-13T03:29:33.678 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 61169740, 2023-09-13T03:29:33.678 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:33.678 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 263915, 2023-09-13T03:29:33.679 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2023-09-13T03:29:33.679 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2023-09-13T03:29:33.679 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 263915, 2023-09-13T03:29:33.679 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 61169740, 2023-09-13T03:29:33.680 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2023-09-13T03:29:33.680 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:33.680 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:33.680 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:33.681 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-09-13T03:29:33.681 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.681 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_bytes": { 2023-09-13T03:29:33.681 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2023-09-13T03:29:33.682 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2023-09-13T03:29:33.682 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2023-09-13T03:29:33.682 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2023-09-13T03:29:33.683 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2023-09-13T03:29:33.683 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:33.683 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2023-09-13T03:29:33.683 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 36125, 2023-09-13T03:29:33.684 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 225021322, 2023-09-13T03:29:33.684 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 36125, 2023-09-13T03:29:33.684 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 225021322, 2023-09-13T03:29:33.685 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2023-09-13T03:29:33.685 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:33.685 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:33.685 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:33.686 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-09-13T03:29:33.686 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.686 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_ops": { 2023-09-13T03:29:33.687 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2023-09-13T03:29:33.687 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 1024, 2023-09-13T03:29:33.687 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2023-09-13T03:29:33.687 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2023-09-13T03:29:33.688 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2023-09-13T03:29:33.688 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:33.688 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2023-09-13T03:29:33.689 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 36125, 2023-09-13T03:29:33.689 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 36125, 2023-09-13T03:29:33.689 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 36125, 2023-09-13T03:29:33.689 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 36125, 2023-09-13T03:29:33.690 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2023-09-13T03:29:33.690 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:33.691 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:33.691 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:33.692 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-09-13T03:29:33.692 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.692 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle": { 2023-09-13T03:29:33.693 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2023-09-13T03:29:33.693 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2023-09-13T03:29:33.693 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2023-09-13T03:29:33.694 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 12671, 2023-09-13T03:29:33.694 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 1281187, 2023-09-13T03:29:33.694 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:33.695 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 12671, 2023-09-13T03:29:33.695 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2023-09-13T03:29:33.695 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2023-09-13T03:29:33.695 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 576, 2023-09-13T03:29:33.696 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 1281187, 2023-09-13T03:29:33.696 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2023-09-13T03:29:33.696 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:33.696 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:33.697 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:33.697 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-09-13T03:29:33.697 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:33.697 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle-0x55845e0386e0": { 2023-09-13T03:29:33.698 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 41306, 2023-09-13T03:29:33.698 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2023-09-13T03:29:33.698 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2023-09-13T03:29:33.699 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 65264, 2023-09-13T03:29:33.699 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 220433335, 2023-09-13T03:29:33.699 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:33.700 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 65264, 2023-09-13T03:29:33.700 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2023-09-13T03:29:33.700 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2023-09-13T03:29:33.700 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 21474, 2023-09-13T03:29:33.701 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 220392029, 2023-09-13T03:29:33.701 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2023-09-13T03:29:33.701 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:33.701 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:33.702 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:33.702 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-09-13T03:29:33.702 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-09-13T03:29:33.702 INFO:teuthology.orchestra.run.smithi142.stdout:} 2023-09-13T03:29:33.932 INFO:tasks.check_counter:Daemon mds.h mds.exported=8 2023-09-13T03:29:33.932 INFO:tasks.check_counter:Daemon mds.h mds.imported=9 2023-09-13T03:29:33.933 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2023-09-13T03:29:33.933 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2023-09-13T03:29:33.933 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2023-09-13T03:29:33.934 DEBUG:tasks.check_counter:Getting stats from k 2023-09-13T03:29:33.934 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph daemon mds.k perf dump 2023-09-13T03:29:34.489 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-13T03:29:34.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:34 smithi142 ceph-mon[134082]: pgmap v515: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 6.2 MiB/s wr, 250 op/s 2023-09-13T03:29:35.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:34 smithi170 ceph-mon[134328]: pgmap v515: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 6.2 MiB/s wr, 250 op/s 2023-09-13T03:29:35.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:34 smithi064 ceph-mon[97675]: pgmap v515: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 6.2 MiB/s wr, 250 op/s 2023-09-13T03:29:35.864 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2023-09-13T03:29:35.865 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2023-09-13T03:29:35.865 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 1012, 2023-09-13T03:29:35.866 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 1014, 2023-09-13T03:29:35.866 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 65464340, 2023-09-13T03:29:35.866 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 314796, 2023-09-13T03:29:35.867 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 9, 2023-09-13T03:29:35.867 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 4, 2023-09-13T03:29:35.867 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.121786557, 2023-09-13T03:29:35.868 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.025290489, 2023-09-13T03:29:35.868 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.059987674, 2023-09-13T03:29:35.869 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.027951038, 2023-09-13T03:29:35.869 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T03:29:35.869 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1012, 2023-09-13T03:29:35.870 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.040953299, 2023-09-13T03:29:35.870 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000040467 2023-09-13T03:29:35.871 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.871 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2023-09-13T03:29:35.871 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:35.871 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:35.872 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:35.872 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.872 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-13T03:29:35.873 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 0 2023-09-13T03:29:35.873 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.874 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2023-09-13T03:29:35.874 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 1297, 2023-09-13T03:29:35.875 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 1525, 2023-09-13T03:29:35.875 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 100995831, 2023-09-13T03:29:35.875 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 1044091, 2023-09-13T03:29:35.876 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 9, 2023-09-13T03:29:35.876 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 6, 2023-09-13T03:29:35.876 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.175355516, 2023-09-13T03:29:35.877 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.034307435, 2023-09-13T03:29:35.877 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.089983396, 2023-09-13T03:29:35.877 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.038459682, 2023-09-13T03:29:35.877 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T03:29:35.878 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1523, 2023-09-13T03:29:35.878 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.058577067, 2023-09-13T03:29:35.879 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000038461 2023-09-13T03:29:35.879 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.879 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2023-09-13T03:29:35.880 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:35.880 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:35.880 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:35.881 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.881 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 3424, 2023-09-13T03:29:35.881 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 642896 2023-09-13T03:29:35.882 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.882 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2023-09-13T03:29:35.882 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 2117, 2023-09-13T03:29:35.883 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 2049, 2023-09-13T03:29:35.883 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 57028177, 2023-09-13T03:29:35.883 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 721715, 2023-09-13T03:29:35.884 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 5, 2023-09-13T03:29:35.884 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 4, 2023-09-13T03:29:35.884 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.244948234, 2023-09-13T03:29:35.885 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.063533710, 2023-09-13T03:29:35.885 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.108394164, 2023-09-13T03:29:35.886 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.067331644, 2023-09-13T03:29:35.886 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T03:29:35.886 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 2048, 2023-09-13T03:29:35.887 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.087371373, 2023-09-13T03:29:35.887 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000042661 2023-09-13T03:29:35.887 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.888 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2023-09-13T03:29:35.888 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:35.888 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:35.889 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:35.889 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.889 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 1976848, 2023-09-13T03:29:35.889 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 187728 2023-09-13T03:29:35.890 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.890 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2023-09-13T03:29:35.891 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 1, 2023-09-13T03:29:35.891 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2023-09-13T03:29:35.891 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.891 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-MDSRank": { 2023-09-13T03:29:35.892 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2023-09-13T03:29:35.892 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2023-09-13T03:29:35.892 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 6694, 2023-09-13T03:29:35.893 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.175909604, 2023-09-13T03:29:35.893 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000026278 2023-09-13T03:29:35.893 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-09-13T03:29:35.894 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.894 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-PurgeQueue": { 2023-09-13T03:29:35.894 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2023-09-13T03:29:35.895 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2023-09-13T03:29:35.895 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:35.895 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:35.896 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:35.896 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-09-13T03:29:35.897 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.897 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2023-09-13T03:29:35.897 INFO:teuthology.orchestra.run.smithi142.stdout: "request": 0, 2023-09-13T03:29:35.897 INFO:teuthology.orchestra.run.smithi142.stdout: "reply": 0, 2023-09-13T03:29:35.898 INFO:teuthology.orchestra.run.smithi142.stdout: "reply_latency": { 2023-09-13T03:29:35.898 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:35.898 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:35.899 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:35.900 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.900 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_reply": 0, 2023-09-13T03:29:35.901 INFO:teuthology.orchestra.run.smithi142.stdout: "forward": 0, 2023-09-13T03:29:35.901 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_complete": 0, 2023-09-13T03:29:35.902 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_keys": 0, 2023-09-13T03:29:35.902 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_commit": 0, 2023-09-13T03:29:35.903 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_split": 0, 2023-09-13T03:29:35.904 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_merge": 0, 2023-09-13T03:29:35.905 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes": 13552, 2023-09-13T03:29:35.905 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_top": 83, 2023-09-13T03:29:35.906 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_bottom": 10609, 2023-09-13T03:29:35.906 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pin_tail": 2860, 2023-09-13T03:29:35.907 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pinned": 13433, 2023-09-13T03:29:35.908 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_expired": 11, 2023-09-13T03:29:35.908 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_with_caps": 0, 2023-09-13T03:29:35.909 INFO:teuthology.orchestra.run.smithi142.stdout: "caps": 0, 2023-09-13T03:29:35.909 INFO:teuthology.orchestra.run.smithi142.stdout: "subtrees": 7, 2023-09-13T03:29:35.909 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse": 0, 2023-09-13T03:29:35.910 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_hit": 0, 2023-09-13T03:29:35.910 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_forward": 0, 2023-09-13T03:29:35.910 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_discover": 0, 2023-09-13T03:29:35.910 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_dir_fetch": 0, 2023-09-13T03:29:35.911 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_remote_ino": 0, 2023-09-13T03:29:35.911 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_lock": 0, 2023-09-13T03:29:35.912 INFO:teuthology.orchestra.run.smithi142.stdout: "load_cent": 0, 2023-09-13T03:29:35.912 INFO:teuthology.orchestra.run.smithi142.stdout: "q": 0, 2023-09-13T03:29:35.912 INFO:teuthology.orchestra.run.smithi142.stdout: "exported": 0, 2023-09-13T03:29:35.913 INFO:teuthology.orchestra.run.smithi142.stdout: "exported_inodes": 0, 2023-09-13T03:29:35.913 INFO:teuthology.orchestra.run.smithi142.stdout: "imported": 0, 2023-09-13T03:29:35.913 INFO:teuthology.orchestra.run.smithi142.stdout: "imported_inodes": 0, 2023-09-13T03:29:35.913 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_dir_fetch": 0, 2023-09-13T03:29:35.914 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_backtrace_fetch": 0, 2023-09-13T03:29:35.914 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_peer_discover": 0, 2023-09-13T03:29:35.915 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rfiles": 0, 2023-09-13T03:29:35.915 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rbytes": 0, 2023-09-13T03:29:35.915 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rsnaps": 0, 2023-09-13T03:29:35.915 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_fetch": 0, 2023-09-13T03:29:35.916 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_set_tag": 0, 2023-09-13T03:29:35.916 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_repaired": 0, 2023-09-13T03:29:35.917 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_inotable_repaired": 0, 2023-09-13T03:29:35.917 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_inodes": 0, 2023-09-13T03:29:35.917 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_base_inodes": 0, 2023-09-13T03:29:35.918 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dirfrag_rstats": 0, 2023-09-13T03:29:35.918 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_file_inodes": 0, 2023-09-13T03:29:35.919 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_inode_file_caps": 0, 2023-09-13T03:29:35.919 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_revoke": 0, 2023-09-13T03:29:35.919 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_grant": 0, 2023-09-13T03:29:35.920 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_trunc": 0, 2023-09-13T03:29:35.920 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-13T03:29:35.921 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-13T03:29:35.921 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps": 0, 2023-09-13T03:29:35.922 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps_dirty": 0, 2023-09-13T03:29:35.922 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_cap_release": 0, 2023-09-13T03:29:35.922 INFO:teuthology.orchestra.run.smithi142.stdout: "process_request_cap_release": 0 2023-09-13T03:29:35.923 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.923 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_cache": { 2023-09-13T03:29:35.923 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update": 0, 2023-09-13T03:29:35.924 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update_receipt": 0, 2023-09-13T03:29:35.924 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_try_discover": 0, 2023-09-13T03:29:35.925 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_send_discover": 0, 2023-09-13T03:29:35.925 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_handle_discover": 0, 2023-09-13T03:29:35.926 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays": 0, 2023-09-13T03:29:35.926 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_delayed": 0, 2023-09-13T03:29:35.926 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_enqueuing": 0, 2023-09-13T03:29:35.927 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_created": 0, 2023-09-13T03:29:35.927 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_enqueued": 0, 2023-09-13T03:29:35.927 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_reintegrated": 0, 2023-09-13T03:29:35.928 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_migrated": 0, 2023-09-13T03:29:35.928 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_processing": 0, 2023-09-13T03:29:35.928 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_enqueued": 0, 2023-09-13T03:29:35.929 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_prioritized": 0, 2023-09-13T03:29:35.929 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_started": 0, 2023-09-13T03:29:35.929 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_completed": 0, 2023-09-13T03:29:35.930 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_enqueue_scrub": 0, 2023-09-13T03:29:35.930 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_exportdir": 0, 2023-09-13T03:29:35.931 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_flush": 0, 2023-09-13T03:29:35.931 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragmentdir": 0, 2023-09-13T03:29:35.931 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragstats": 0, 2023-09-13T03:29:35.931 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_inodestats": 0 2023-09-13T03:29:35.932 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.932 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_log": { 2023-09-13T03:29:35.933 INFO:teuthology.orchestra.run.smithi142.stdout: "evlrg": 0, 2023-09-13T03:29:35.933 INFO:teuthology.orchestra.run.smithi142.stdout: "evadd": 0, 2023-09-13T03:29:35.933 INFO:teuthology.orchestra.run.smithi142.stdout: "evex": 0, 2023-09-13T03:29:35.934 INFO:teuthology.orchestra.run.smithi142.stdout: "evtrm": 0, 2023-09-13T03:29:35.934 INFO:teuthology.orchestra.run.smithi142.stdout: "ev": 65195, 2023-09-13T03:29:35.934 INFO:teuthology.orchestra.run.smithi142.stdout: "evexg": 0, 2023-09-13T03:29:35.935 INFO:teuthology.orchestra.run.smithi142.stdout: "evexd": 0, 2023-09-13T03:29:35.935 INFO:teuthology.orchestra.run.smithi142.stdout: "segadd": 0, 2023-09-13T03:29:35.935 INFO:teuthology.orchestra.run.smithi142.stdout: "segex": 0, 2023-09-13T03:29:35.936 INFO:teuthology.orchestra.run.smithi142.stdout: "segtrm": 0, 2023-09-13T03:29:35.936 INFO:teuthology.orchestra.run.smithi142.stdout: "seg": 95, 2023-09-13T03:29:35.937 INFO:teuthology.orchestra.run.smithi142.stdout: "segmjr": 7, 2023-09-13T03:29:35.937 INFO:teuthology.orchestra.run.smithi142.stdout: "segexg": 0, 2023-09-13T03:29:35.938 INFO:teuthology.orchestra.run.smithi142.stdout: "segexd": 0, 2023-09-13T03:29:35.938 INFO:teuthology.orchestra.run.smithi142.stdout: "expos": 4194304, 2023-09-13T03:29:35.938 INFO:teuthology.orchestra.run.smithi142.stdout: "wrpos": 224586333, 2023-09-13T03:29:35.939 INFO:teuthology.orchestra.run.smithi142.stdout: "rdpos": 224581384, 2023-09-13T03:29:35.939 INFO:teuthology.orchestra.run.smithi142.stdout: "jlat": { 2023-09-13T03:29:35.940 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:35.940 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:35.941 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:35.941 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.942 INFO:teuthology.orchestra.run.smithi142.stdout: "replayed": 65195 2023-09-13T03:29:35.942 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.942 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_mem": { 2023-09-13T03:29:35.942 INFO:teuthology.orchestra.run.smithi142.stdout: "ino": 809, 2023-09-13T03:29:35.943 INFO:teuthology.orchestra.run.smithi142.stdout: "ino+": 13581, 2023-09-13T03:29:35.943 INFO:teuthology.orchestra.run.smithi142.stdout: "ino-": 12772, 2023-09-13T03:29:35.944 INFO:teuthology.orchestra.run.smithi142.stdout: "dir": 82, 2023-09-13T03:29:35.944 INFO:teuthology.orchestra.run.smithi142.stdout: "dir+": 155, 2023-09-13T03:29:35.944 INFO:teuthology.orchestra.run.smithi142.stdout: "dir-": 73, 2023-09-13T03:29:35.945 INFO:teuthology.orchestra.run.smithi142.stdout: "dn": 13552, 2023-09-13T03:29:35.946 INFO:teuthology.orchestra.run.smithi142.stdout: "dn+": 13813, 2023-09-13T03:29:35.946 INFO:teuthology.orchestra.run.smithi142.stdout: "dn-": 261, 2023-09-13T03:29:35.946 INFO:teuthology.orchestra.run.smithi142.stdout: "cap": 0, 2023-09-13T03:29:35.947 INFO:teuthology.orchestra.run.smithi142.stdout: "cap+": 0, 2023-09-13T03:29:35.947 INFO:teuthology.orchestra.run.smithi142.stdout: "cap-": 0, 2023-09-13T03:29:35.947 INFO:teuthology.orchestra.run.smithi142.stdout: "rss": 67768, 2023-09-13T03:29:35.948 INFO:teuthology.orchestra.run.smithi142.stdout: "heap": 207152 2023-09-13T03:29:35.948 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.948 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_server": { 2023-09-13T03:29:35.949 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_client_request": 0, 2023-09-13T03:29:35.949 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_server_request": 0, 2023-09-13T03:29:35.949 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_request": 0, 2023-09-13T03:29:35.950 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_session": 0, 2023-09-13T03:29:35.950 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_peer_request": 0, 2023-09-13T03:29:35.950 INFO:teuthology.orchestra.run.smithi142.stdout: "req_create_latency": { 2023-09-13T03:29:35.951 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:35.951 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:35.951 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:35.952 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.952 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getattr_latency": { 2023-09-13T03:29:35.952 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:35.953 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:35.953 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:35.953 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.954 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getfilelock_latency": { 2023-09-13T03:29:35.954 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:35.954 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:35.955 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:35.955 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.956 INFO:teuthology.orchestra.run.smithi142.stdout: "req_link_latency": { 2023-09-13T03:29:35.956 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:35.956 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:35.956 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:35.957 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.957 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookup_latency": { 2023-09-13T03:29:35.957 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:35.958 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:35.958 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:35.959 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.959 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookuphash_latency": { 2023-09-13T03:29:35.959 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:35.960 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:35.960 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:35.960 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.961 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupino_latency": { 2023-09-13T03:29:35.961 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:35.962 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:35.962 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:35.962 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.963 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupname_latency": { 2023-09-13T03:29:35.963 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:35.963 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:35.964 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:35.964 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.964 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupparent_latency": { 2023-09-13T03:29:35.965 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:35.965 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:35.966 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:35.966 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.966 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupsnap_latency": { 2023-09-13T03:29:35.967 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:35.967 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:35.967 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:35.968 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.968 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lssnap_latency": { 2023-09-13T03:29:35.968 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:35.969 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:35.969 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:35.969 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.970 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mkdir_latency": { 2023-09-13T03:29:35.970 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:35.970 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:35.971 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:35.971 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.972 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mknod_latency": { 2023-09-13T03:29:35.972 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:35.972 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:35.972 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:35.973 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.973 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mksnap_latency": { 2023-09-13T03:29:35.974 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:35.974 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:35.975 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:35.975 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.976 INFO:teuthology.orchestra.run.smithi142.stdout: "req_open_latency": { 2023-09-13T03:29:35.976 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:35.976 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:35.976 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:35.977 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.977 INFO:teuthology.orchestra.run.smithi142.stdout: "req_readdir_latency": { 2023-09-13T03:29:35.977 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:35.978 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:35.978 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:35.978 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.979 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rename_latency": { 2023-09-13T03:29:35.979 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:35.980 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:35.980 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:35.981 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.981 INFO:teuthology.orchestra.run.smithi142.stdout: "req_renamesnap_latency": { 2023-09-13T03:29:35.982 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:35.982 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:35.982 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:35.982 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.983 INFO:teuthology.orchestra.run.smithi142.stdout: "req_snapdiff_latency": { 2023-09-13T03:29:35.983 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:35.983 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:35.984 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:35.984 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.984 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmdir_latency": { 2023-09-13T03:29:35.985 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:35.985 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:35.986 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:35.986 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.986 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmsnap_latency": { 2023-09-13T03:29:35.987 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:35.987 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:35.987 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:35.988 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.988 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmxattr_latency": { 2023-09-13T03:29:35.988 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:35.989 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:35.989 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:35.989 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.990 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setattr_latency": { 2023-09-13T03:29:35.990 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:35.990 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:35.991 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:35.991 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.991 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setdirlayout_latency": { 2023-09-13T03:29:35.992 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:35.992 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:35.992 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:35.993 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.993 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setfilelock_latency": { 2023-09-13T03:29:35.994 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:35.994 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:35.994 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:35.995 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.995 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setlayout_latency": { 2023-09-13T03:29:35.995 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:35.996 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:35.996 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:35.996 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.997 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setxattr_latency": { 2023-09-13T03:29:35.997 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:35.997 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:35.997 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:35.998 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:35.998 INFO:teuthology.orchestra.run.smithi142.stdout: "req_symlink_latency": { 2023-09-13T03:29:35.999 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:35.999 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:35.999 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:35.999 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:36.000 INFO:teuthology.orchestra.run.smithi142.stdout: "req_unlink_latency": { 2023-09-13T03:29:36.000 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:36.000 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:36.001 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:36.001 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:36.002 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_revoke_eviction": 0, 2023-09-13T03:29:36.002 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_acquisition_throttle": 0, 2023-09-13T03:29:36.002 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getvxattr_latency": { 2023-09-13T03:29:36.003 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:36.003 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:36.003 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:36.004 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-09-13T03:29:36.004 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:36.004 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_sessions": { 2023-09-13T03:29:36.005 INFO:teuthology.orchestra.run.smithi142.stdout: "session_count": 1, 2023-09-13T03:29:36.005 INFO:teuthology.orchestra.run.smithi142.stdout: "session_add": 4, 2023-09-13T03:29:36.005 INFO:teuthology.orchestra.run.smithi142.stdout: "session_remove": 3, 2023-09-13T03:29:36.006 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_open": 3, 2023-09-13T03:29:36.006 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_stale": 0, 2023-09-13T03:29:36.006 INFO:teuthology.orchestra.run.smithi142.stdout: "total_load": 0, 2023-09-13T03:29:36.007 INFO:teuthology.orchestra.run.smithi142.stdout: "average_load": 0, 2023-09-13T03:29:36.007 INFO:teuthology.orchestra.run.smithi142.stdout: "avg_session_uptime": 6414, 2023-09-13T03:29:36.007 INFO:teuthology.orchestra.run.smithi142.stdout: "mdthresh_evicted": 0 2023-09-13T03:29:36.008 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:36.008 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2023-09-13T03:29:36.008 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 0, 2023-09-13T03:29:36.008 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 0, 2023-09-13T03:29:36.009 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2023-09-13T03:29:36.009 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2023-09-13T03:29:36.010 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2023-09-13T03:29:36.010 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2023-09-13T03:29:36.011 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-13T03:29:36.011 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2023-09-13T03:29:36.012 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-13T03:29:36.012 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2023-09-13T03:29:36.012 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2023-09-13T03:29:36.012 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2023-09-13T03:29:36.013 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-13T03:29:36.013 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2023-09-13T03:29:36.013 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2023-09-13T03:29:36.014 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2023-09-13T03:29:36.014 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2023-09-13T03:29:36.014 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2023-09-13T03:29:36.015 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-13T03:29:36.015 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2023-09-13T03:29:36.015 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-13T03:29:36.016 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2023-09-13T03:29:36.016 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2023-09-13T03:29:36.016 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2023-09-13T03:29:36.017 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2023-09-13T03:29:36.017 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2023-09-13T03:29:36.017 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-13T03:29:36.017 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2023-09-13T03:29:36.018 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2023-09-13T03:29:36.018 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2023-09-13T03:29:36.019 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2023-09-13T03:29:36.019 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2023-09-13T03:29:36.019 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2023-09-13T03:29:36.020 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2023-09-13T03:29:36.020 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2023-09-13T03:29:36.020 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2023-09-13T03:29:36.021 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 262422, 2023-09-13T03:29:36.021 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 968, 2023-09-13T03:29:36.021 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 0, 2023-09-13T03:29:36.022 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 0, 2023-09-13T03:29:36.022 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2023-09-13T03:29:36.023 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2023-09-13T03:29:36.023 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2023-09-13T03:29:36.024 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2023-09-13T03:29:36.024 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2023-09-13T03:29:36.024 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2023-09-13T03:29:36.025 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 7936, 2023-09-13T03:29:36.025 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 165, 2023-09-13T03:29:36.025 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2023-09-13T03:29:36.025 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2023-09-13T03:29:36.026 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2023-09-13T03:29:36.026 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2023-09-13T03:29:36.027 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 11552365, 2023-09-13T03:29:36.027 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 102303, 2023-09-13T03:29:36.027 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2023-09-13T03:29:36.028 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2023-09-13T03:29:36.028 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2023-09-13T03:29:36.028 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2023-09-13T03:29:36.029 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:36.029 INFO:teuthology.orchestra.run.smithi142.stdout: "objecter": { 2023-09-13T03:29:36.029 INFO:teuthology.orchestra.run.smithi142.stdout: "op_active": 0, 2023-09-13T03:29:36.029 INFO:teuthology.orchestra.run.smithi142.stdout: "op_laggy": 0, 2023-09-13T03:29:36.030 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send": 4019, 2023-09-13T03:29:36.030 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send_bytes": 21, 2023-09-13T03:29:36.030 INFO:teuthology.orchestra.run.smithi142.stdout: "op_resend": 3, 2023-09-13T03:29:36.031 INFO:teuthology.orchestra.run.smithi142.stdout: "op_reply": 4019, 2023-09-13T03:29:36.031 INFO:teuthology.orchestra.run.smithi142.stdout: "op_latency": { 2023-09-13T03:29:36.031 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 4019, 2023-09-13T03:29:36.032 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 3.873928896, 2023-09-13T03:29:36.032 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000963903 2023-09-13T03:29:36.033 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:36.033 INFO:teuthology.orchestra.run.smithi142.stdout: "op_inflight": 0, 2023-09-13T03:29:36.033 INFO:teuthology.orchestra.run.smithi142.stdout: "oplen_avg": { 2023-09-13T03:29:36.034 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 4019, 2023-09-13T03:29:36.034 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 4020 2023-09-13T03:29:36.034 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:36.035 INFO:teuthology.orchestra.run.smithi142.stdout: "op": 4019, 2023-09-13T03:29:36.035 INFO:teuthology.orchestra.run.smithi142.stdout: "op_r": 4019, 2023-09-13T03:29:36.035 INFO:teuthology.orchestra.run.smithi142.stdout: "op_w": 0, 2023-09-13T03:29:36.035 INFO:teuthology.orchestra.run.smithi142.stdout: "op_rmw": 0, 2023-09-13T03:29:36.036 INFO:teuthology.orchestra.run.smithi142.stdout: "op_pg": 0, 2023-09-13T03:29:36.036 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_stat": 2303, 2023-09-13T03:29:36.037 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_create": 0, 2023-09-13T03:29:36.037 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_read": 1714, 2023-09-13T03:29:36.037 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_write": 0, 2023-09-13T03:29:36.037 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writefull": 0, 2023-09-13T03:29:36.038 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writesame": 0, 2023-09-13T03:29:36.038 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_append": 0, 2023-09-13T03:29:36.039 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_zero": 0, 2023-09-13T03:29:36.039 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_truncate": 0, 2023-09-13T03:29:36.040 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_delete": 0, 2023-09-13T03:29:36.040 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_mapext": 0, 2023-09-13T03:29:36.040 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_sparse_read": 0, 2023-09-13T03:29:36.041 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_clonerange": 0, 2023-09-13T03:29:36.041 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_getxattr": 1, 2023-09-13T03:29:36.041 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_setxattr": 0, 2023-09-13T03:29:36.042 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_cmpxattr": 0, 2023-09-13T03:29:36.042 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_rmxattr": 0, 2023-09-13T03:29:36.042 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_resetxattrs": 0, 2023-09-13T03:29:36.043 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_call": 0, 2023-09-13T03:29:36.043 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_watch": 0, 2023-09-13T03:29:36.043 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_notify": 0, 2023-09-13T03:29:36.043 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_src_cmpxattr": 0, 2023-09-13T03:29:36.044 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls": 0, 2023-09-13T03:29:36.044 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls_filter": 0, 2023-09-13T03:29:36.044 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_other": 0, 2023-09-13T03:29:36.045 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_active": 0, 2023-09-13T03:29:36.045 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_send": 0, 2023-09-13T03:29:36.045 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_resend": 0, 2023-09-13T03:29:36.046 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_ping": 0, 2023-09-13T03:29:36.046 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_active": 0, 2023-09-13T03:29:36.047 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_send": 0, 2023-09-13T03:29:36.047 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_resend": 0, 2023-09-13T03:29:36.047 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_active": 0, 2023-09-13T03:29:36.048 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_send": 0, 2023-09-13T03:29:36.048 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_resend": 0, 2023-09-13T03:29:36.049 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_active": 0, 2023-09-13T03:29:36.049 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_send": 0, 2023-09-13T03:29:36.049 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_resend": 0, 2023-09-13T03:29:36.050 INFO:teuthology.orchestra.run.smithi142.stdout: "command_active": 0, 2023-09-13T03:29:36.050 INFO:teuthology.orchestra.run.smithi142.stdout: "command_send": 0, 2023-09-13T03:29:36.050 INFO:teuthology.orchestra.run.smithi142.stdout: "command_resend": 0, 2023-09-13T03:29:36.051 INFO:teuthology.orchestra.run.smithi142.stdout: "map_epoch": 106, 2023-09-13T03:29:36.051 INFO:teuthology.orchestra.run.smithi142.stdout: "map_full": 0, 2023-09-13T03:29:36.051 INFO:teuthology.orchestra.run.smithi142.stdout: "map_inc": 23, 2023-09-13T03:29:36.052 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_sessions": 10, 2023-09-13T03:29:36.052 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_open": 11, 2023-09-13T03:29:36.052 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_close": 1, 2023-09-13T03:29:36.053 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_laggy": 0, 2023-09-13T03:29:36.053 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_wr": 0, 2023-09-13T03:29:36.053 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_rd": 2, 2023-09-13T03:29:36.054 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_del": 0 2023-09-13T03:29:36.054 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:36.055 INFO:teuthology.orchestra.run.smithi142.stdout: "oft": { 2023-09-13T03:29:36.055 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_objs": 0, 2023-09-13T03:29:36.055 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_kv_pairs": 0, 2023-09-13T03:29:36.056 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_updates": 0, 2023-09-13T03:29:36.056 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_removes": 0 2023-09-13T03:29:36.056 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:36.057 INFO:teuthology.orchestra.run.smithi142.stdout: "osd-slow-ops": { 2023-09-13T03:29:36.057 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_ops_count": 0 2023-09-13T03:29:36.057 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:36.058 INFO:teuthology.orchestra.run.smithi142.stdout: "purge_queue": { 2023-09-13T03:29:36.058 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops": 0, 2023-09-13T03:29:36.058 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops_high_water": 0, 2023-09-13T03:29:36.059 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing": 0, 2023-09-13T03:29:36.059 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_high_water": 0, 2023-09-13T03:29:36.059 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed": 0, 2023-09-13T03:29:36.060 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_item_in_journal": 0 2023-09-13T03:29:36.060 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:36.061 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-13T03:29:36.061 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2023-09-13T03:29:36.061 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2023-09-13T03:29:36.062 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2023-09-13T03:29:36.062 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 4422, 2023-09-13T03:29:36.063 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 223067359, 2023-09-13T03:29:36.063 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:36.063 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 4422, 2023-09-13T03:29:36.063 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2023-09-13T03:29:36.064 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2023-09-13T03:29:36.064 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 4422, 2023-09-13T03:29:36.064 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 223067359, 2023-09-13T03:29:36.065 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2023-09-13T03:29:36.065 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:36.065 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:36.066 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:36.066 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-09-13T03:29:36.066 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:36.067 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_bytes": { 2023-09-13T03:29:36.067 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2023-09-13T03:29:36.067 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2023-09-13T03:29:36.068 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2023-09-13T03:29:36.068 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2023-09-13T03:29:36.068 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2023-09-13T03:29:36.069 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:36.069 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2023-09-13T03:29:36.069 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 4019, 2023-09-13T03:29:36.070 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 220397214, 2023-09-13T03:29:36.070 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 4019, 2023-09-13T03:29:36.070 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 220397214, 2023-09-13T03:29:36.071 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2023-09-13T03:29:36.071 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:36.071 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:36.072 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:36.072 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-09-13T03:29:36.073 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:36.073 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_ops": { 2023-09-13T03:29:36.073 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2023-09-13T03:29:36.073 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 1024, 2023-09-13T03:29:36.074 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2023-09-13T03:29:36.074 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2023-09-13T03:29:36.074 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2023-09-13T03:29:36.075 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:36.075 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2023-09-13T03:29:36.075 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 4019, 2023-09-13T03:29:36.076 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 4019, 2023-09-13T03:29:36.076 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 4019, 2023-09-13T03:29:36.077 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 4019, 2023-09-13T03:29:36.077 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2023-09-13T03:29:36.077 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:36.078 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:36.078 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:36.079 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-09-13T03:29:36.079 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:36.080 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle": { 2023-09-13T03:29:36.080 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2023-09-13T03:29:36.080 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2023-09-13T03:29:36.080 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2023-09-13T03:29:36.081 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2023-09-13T03:29:36.081 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2023-09-13T03:29:36.081 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:36.082 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2023-09-13T03:29:36.082 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2023-09-13T03:29:36.082 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2023-09-13T03:29:36.083 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 0, 2023-09-13T03:29:36.083 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 0, 2023-09-13T03:29:36.083 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2023-09-13T03:29:36.083 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:36.084 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:36.084 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:36.084 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-09-13T03:29:36.085 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2023-09-13T03:29:36.085 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle-0x564219a44960": { 2023-09-13T03:29:36.085 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2023-09-13T03:29:36.086 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2023-09-13T03:29:36.086 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2023-09-13T03:29:36.086 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2023-09-13T03:29:36.087 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2023-09-13T03:29:36.087 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:36.087 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2023-09-13T03:29:36.088 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2023-09-13T03:29:36.088 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2023-09-13T03:29:36.088 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 0, 2023-09-13T03:29:36.089 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 0, 2023-09-13T03:29:36.089 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2023-09-13T03:29:36.089 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2023-09-13T03:29:36.090 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2023-09-13T03:29:36.090 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2023-09-13T03:29:36.090 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-09-13T03:29:36.090 INFO:teuthology.orchestra.run.smithi142.stdout: } 2023-09-13T03:29:36.091 INFO:teuthology.orchestra.run.smithi142.stdout:} 2023-09-13T03:29:36.346 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2023-09-13T03:29:36.347 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2023-09-13T03:29:36.347 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2023-09-13T03:29:36.348 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2023-09-13T03:29:36.348 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2023-09-13T03:29:36.348 DEBUG:tasks.check_counter:Getting stats from c 2023-09-13T03:29:36.349 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph daemon mds.c perf dump 2023-09-13T03:29:37.059 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-13T03:29:37.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:36 smithi170 ceph-mon[134328]: pgmap v516: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 7.0 MiB/s wr, 268 op/s 2023-09-13T03:29:37.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:36 smithi170 ceph-mon[134328]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:29:37.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:36 smithi064 ceph-mon[97675]: pgmap v516: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 7.0 MiB/s wr, 268 op/s 2023-09-13T03:29:37.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:36 smithi064 ceph-mon[97675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:29:37.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:36 smithi142 ceph-mon[134082]: pgmap v516: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 7.0 MiB/s wr, 268 op/s 2023-09-13T03:29:37.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:36 smithi142 ceph-mon[134082]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-13T03:29:39.093 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-09-13T03:29:39.093 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-0": { 2023-09-13T03:29:39.094 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 20787, 2023-09-13T03:29:39.094 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 21897, 2023-09-13T03:29:39.094 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 6206427, 2023-09-13T03:29:39.095 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 69867158, 2023-09-13T03:29:39.095 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 16, 2023-09-13T03:29:39.095 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 5, 2023-09-13T03:29:39.095 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 1.608360008, 2023-09-13T03:29:39.096 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.709489883, 2023-09-13T03:29:39.096 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.701538622, 2023-09-13T03:29:39.096 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.263094789, 2023-09-13T03:29:39.096 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T03:29:39.097 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 21894, 2023-09-13T03:29:39.097 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 3.625199733, 2023-09-13T03:29:39.097 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000165579 2023-09-13T03:29:39.098 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.098 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2023-09-13T03:29:39.098 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 25010, 2023-09-13T03:29:39.098 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.005199607, 2023-09-13T03:29:39.099 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000207 2023-09-13T03:29:39.099 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.099 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 7216, 2023-09-13T03:29:39.099 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 453024 2023-09-13T03:29:39.100 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.100 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-1": { 2023-09-13T03:29:39.100 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 124479, 2023-09-13T03:29:39.101 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 196649, 2023-09-13T03:29:39.101 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 39708614, 2023-09-13T03:29:39.101 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 269077162, 2023-09-13T03:29:39.101 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 23, 2023-09-13T03:29:39.102 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 8, 2023-09-13T03:29:39.102 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 7.273694924, 2023-09-13T03:29:39.102 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 3.415441355, 2023-09-13T03:29:39.103 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 4.607848019, 2023-09-13T03:29:39.103 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.565179729, 2023-09-13T03:29:39.103 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T03:29:39.103 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 196644, 2023-09-13T03:29:39.104 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 33.767365355, 2023-09-13T03:29:39.104 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000171718 2023-09-13T03:29:39.104 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.104 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2023-09-13T03:29:39.105 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 138842, 2023-09-13T03:29:39.105 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.044520244, 2023-09-13T03:29:39.105 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000320 2023-09-13T03:29:39.105 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.106 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 2912, 2023-09-13T03:29:39.106 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 127328 2023-09-13T03:29:39.106 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.106 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-2": { 2023-09-13T03:29:39.107 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 74581, 2023-09-13T03:29:39.107 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 65290, 2023-09-13T03:29:39.107 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 25295723, 2023-09-13T03:29:39.108 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 54954351, 2023-09-13T03:29:39.108 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 8, 2023-09-13T03:29:39.108 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 5, 2023-09-13T03:29:39.108 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 3.937680670, 2023-09-13T03:29:39.109 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 1.564906766, 2023-09-13T03:29:39.109 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 2.722365707, 2023-09-13T03:29:39.109 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.225124715, 2023-09-13T03:29:39.109 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T03:29:39.110 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 65293, 2023-09-13T03:29:39.110 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 7.628815118, 2023-09-13T03:29:39.110 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000116839 2023-09-13T03:29:39.111 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.111 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2023-09-13T03:29:39.111 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 121874, 2023-09-13T03:29:39.111 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.019695222, 2023-09-13T03:29:39.112 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000161 2023-09-13T03:29:39.112 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.112 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 1993552, 2023-09-13T03:29:39.112 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 192048 2023-09-13T03:29:39.113 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.113 INFO:teuthology.orchestra.run.smithi170.stdout: "cct": { 2023-09-13T03:29:39.113 INFO:teuthology.orchestra.run.smithi170.stdout: "total_workers": 1, 2023-09-13T03:29:39.114 INFO:teuthology.orchestra.run.smithi170.stdout: "unhealthy_workers": 0 2023-09-13T03:29:39.114 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.115 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-MDSRank": { 2023-09-13T03:29:39.115 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2023-09-13T03:29:39.115 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2023-09-13T03:29:39.115 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 32577, 2023-09-13T03:29:39.116 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 43.816794024, 2023-09-13T03:29:39.116 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001345022 2023-09-13T03:29:39.116 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-09-13T03:29:39.116 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.117 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-PurgeQueue": { 2023-09-13T03:29:39.117 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2023-09-13T03:29:39.117 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2023-09-13T03:29:39.118 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 20681, 2023-09-13T03:29:39.118 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 4.355757627, 2023-09-13T03:29:39.118 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000210616 2023-09-13T03:29:39.119 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-09-13T03:29:39.119 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.119 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": { 2023-09-13T03:29:39.120 INFO:teuthology.orchestra.run.smithi170.stdout: "request": 79662, 2023-09-13T03:29:39.120 INFO:teuthology.orchestra.run.smithi170.stdout: "reply": 74552, 2023-09-13T03:29:39.120 INFO:teuthology.orchestra.run.smithi170.stdout: "reply_latency": { 2023-09-13T03:29:39.121 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 74552, 2023-09-13T03:29:39.121 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 285.116931514, 2023-09-13T03:29:39.121 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.003824403 2023-09-13T03:29:39.122 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.122 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_reply": 0, 2023-09-13T03:29:39.122 INFO:teuthology.orchestra.run.smithi170.stdout: "forward": 5110, 2023-09-13T03:29:39.123 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_complete": 17, 2023-09-13T03:29:39.123 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_keys": 0, 2023-09-13T03:29:39.123 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_commit": 116, 2023-09-13T03:29:39.124 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_split": 12, 2023-09-13T03:29:39.124 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_merge": 9, 2023-09-13T03:29:39.124 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes": 2015, 2023-09-13T03:29:39.125 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_top": 618, 2023-09-13T03:29:39.125 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_bottom": 1006, 2023-09-13T03:29:39.125 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pin_tail": 391, 2023-09-13T03:29:39.126 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pinned": 1130, 2023-09-13T03:29:39.126 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_expired": 500, 2023-09-13T03:29:39.126 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_with_caps": 281, 2023-09-13T03:29:39.127 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": 281, 2023-09-13T03:29:39.127 INFO:teuthology.orchestra.run.smithi170.stdout: "subtrees": 15, 2023-09-13T03:29:39.128 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse": 112137, 2023-09-13T03:29:39.128 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_hit": 67246, 2023-09-13T03:29:39.128 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_forward": 5100, 2023-09-13T03:29:39.129 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_discover": 60, 2023-09-13T03:29:39.129 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_dir_fetch": 7, 2023-09-13T03:29:39.129 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_remote_ino": 0, 2023-09-13T03:29:39.130 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_lock": 798, 2023-09-13T03:29:39.130 INFO:teuthology.orchestra.run.smithi170.stdout: "load_cent": 12728, 2023-09-13T03:29:39.130 INFO:teuthology.orchestra.run.smithi170.stdout: "q": 1, 2023-09-13T03:29:39.130 INFO:teuthology.orchestra.run.smithi170.stdout: "exported": 9, 2023-09-13T03:29:39.131 INFO:teuthology.orchestra.run.smithi170.stdout: "exported_inodes": 334, 2023-09-13T03:29:39.131 INFO:teuthology.orchestra.run.smithi170.stdout: "imported": 9, 2023-09-13T03:29:39.131 INFO:teuthology.orchestra.run.smithi170.stdout: "imported_inodes": 482, 2023-09-13T03:29:39.132 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_dir_fetch": 0, 2023-09-13T03:29:39.132 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_backtrace_fetch": 0, 2023-09-13T03:29:39.132 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_peer_discover": 1, 2023-09-13T03:29:39.132 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rfiles": 693, 2023-09-13T03:29:39.133 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rbytes": 211724109, 2023-09-13T03:29:39.133 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rsnaps": 6, 2023-09-13T03:29:39.133 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_fetch": 0, 2023-09-13T03:29:39.133 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_set_tag": 0, 2023-09-13T03:29:39.134 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_repaired": 0, 2023-09-13T03:29:39.134 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_inotable_repaired": 0, 2023-09-13T03:29:39.134 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_inodes": 0, 2023-09-13T03:29:39.135 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_base_inodes": 0, 2023-09-13T03:29:39.135 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dirfrag_rstats": 0, 2023-09-13T03:29:39.135 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_file_inodes": 0, 2023-09-13T03:29:39.135 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_inode_file_caps": 2, 2023-09-13T03:29:39.136 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_revoke": 5845, 2023-09-13T03:29:39.136 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_grant": 55181, 2023-09-13T03:29:39.136 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_trunc": 0, 2023-09-13T03:29:39.136 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flushsnap_ack": 1980, 2023-09-13T03:29:39.137 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-13T03:29:39.137 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps": 22437, 2023-09-13T03:29:39.137 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps_dirty": 11982, 2023-09-13T03:29:39.137 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_cap_release": 3648, 2023-09-13T03:29:39.138 INFO:teuthology.orchestra.run.smithi170.stdout: "process_request_cap_release": 50391 2023-09-13T03:29:39.138 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.138 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_cache": { 2023-09-13T03:29:39.139 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update": 18, 2023-09-13T03:29:39.139 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update_receipt": 70, 2023-09-13T03:29:39.139 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_try_discover": 46, 2023-09-13T03:29:39.139 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_send_discover": 105, 2023-09-13T03:29:39.140 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_handle_discover": 94, 2023-09-13T03:29:39.140 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays": 152, 2023-09-13T03:29:39.140 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_delayed": 0, 2023-09-13T03:29:39.140 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_enqueuing": 0, 2023-09-13T03:29:39.141 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_created": 11940, 2023-09-13T03:29:39.141 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_enqueued": 11788, 2023-09-13T03:29:39.141 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_reintegrated": 0, 2023-09-13T03:29:39.142 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_migrated": 0, 2023-09-13T03:29:39.142 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_processing": 0, 2023-09-13T03:29:39.142 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_enqueued": 0, 2023-09-13T03:29:39.142 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_prioritized": 0, 2023-09-13T03:29:39.143 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_started": 0, 2023-09-13T03:29:39.143 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_completed": 0, 2023-09-13T03:29:39.143 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_enqueue_scrub": 0, 2023-09-13T03:29:39.144 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_exportdir": 14, 2023-09-13T03:29:39.144 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_flush": 0, 2023-09-13T03:29:39.144 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragmentdir": 21, 2023-09-13T03:29:39.144 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragstats": 0, 2023-09-13T03:29:39.145 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_inodestats": 0 2023-09-13T03:29:39.145 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.145 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_log": { 2023-09-13T03:29:39.146 INFO:teuthology.orchestra.run.smithi170.stdout: "evlrg": 0, 2023-09-13T03:29:39.146 INFO:teuthology.orchestra.run.smithi170.stdout: "evadd": 61292, 2023-09-13T03:29:39.146 INFO:teuthology.orchestra.run.smithi170.stdout: "evex": 0, 2023-09-13T03:29:39.146 INFO:teuthology.orchestra.run.smithi170.stdout: "evtrm": 0, 2023-09-13T03:29:39.147 INFO:teuthology.orchestra.run.smithi170.stdout: "ev": 61292, 2023-09-13T03:29:39.147 INFO:teuthology.orchestra.run.smithi170.stdout: "evexg": 0, 2023-09-13T03:29:39.147 INFO:teuthology.orchestra.run.smithi170.stdout: "evexd": 0, 2023-09-13T03:29:39.147 INFO:teuthology.orchestra.run.smithi170.stdout: "segadd": 85, 2023-09-13T03:29:39.148 INFO:teuthology.orchestra.run.smithi170.stdout: "segex": 0, 2023-09-13T03:29:39.148 INFO:teuthology.orchestra.run.smithi170.stdout: "segtrm": 0, 2023-09-13T03:29:39.148 INFO:teuthology.orchestra.run.smithi170.stdout: "seg": 85, 2023-09-13T03:29:39.149 INFO:teuthology.orchestra.run.smithi170.stdout: "segmjr": 6, 2023-09-13T03:29:39.149 INFO:teuthology.orchestra.run.smithi170.stdout: "segexg": 0, 2023-09-13T03:29:39.149 INFO:teuthology.orchestra.run.smithi170.stdout: "segexd": 0, 2023-09-13T03:29:39.149 INFO:teuthology.orchestra.run.smithi170.stdout: "expos": 4194304, 2023-09-13T03:29:39.150 INFO:teuthology.orchestra.run.smithi170.stdout: "wrpos": 214255478, 2023-09-13T03:29:39.150 INFO:teuthology.orchestra.run.smithi170.stdout: "rdpos": 0, 2023-09-13T03:29:39.150 INFO:teuthology.orchestra.run.smithi170.stdout: "jlat": { 2023-09-13T03:29:39.150 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 17931, 2023-09-13T03:29:39.151 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 112.321870973, 2023-09-13T03:29:39.151 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.006264116 2023-09-13T03:29:39.151 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.151 INFO:teuthology.orchestra.run.smithi170.stdout: "replayed": 0 2023-09-13T03:29:39.152 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.152 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_mem": { 2023-09-13T03:29:39.152 INFO:teuthology.orchestra.run.smithi170.stdout: "ino": 824, 2023-09-13T03:29:39.153 INFO:teuthology.orchestra.run.smithi170.stdout: "ino+": 12923, 2023-09-13T03:29:39.153 INFO:teuthology.orchestra.run.smithi170.stdout: "ino-": 12099, 2023-09-13T03:29:39.153 INFO:teuthology.orchestra.run.smithi170.stdout: "dir": 157, 2023-09-13T03:29:39.153 INFO:teuthology.orchestra.run.smithi170.stdout: "dir+": 268, 2023-09-13T03:29:39.154 INFO:teuthology.orchestra.run.smithi170.stdout: "dir-": 111, 2023-09-13T03:29:39.154 INFO:teuthology.orchestra.run.smithi170.stdout: "dn": 1947, 2023-09-13T03:29:39.154 INFO:teuthology.orchestra.run.smithi170.stdout: "dn+": 25378, 2023-09-13T03:29:39.154 INFO:teuthology.orchestra.run.smithi170.stdout: "dn-": 23431, 2023-09-13T03:29:39.155 INFO:teuthology.orchestra.run.smithi170.stdout: "cap": 265, 2023-09-13T03:29:39.155 INFO:teuthology.orchestra.run.smithi170.stdout: "cap+": 13455, 2023-09-13T03:29:39.155 INFO:teuthology.orchestra.run.smithi170.stdout: "cap-": 13190, 2023-09-13T03:29:39.155 INFO:teuthology.orchestra.run.smithi170.stdout: "rss": 73916, 2023-09-13T03:29:39.156 INFO:teuthology.orchestra.run.smithi170.stdout: "heap": 198960 2023-09-13T03:29:39.156 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.156 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_server": { 2023-09-13T03:29:39.157 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_client_request": 104081, 2023-09-13T03:29:39.157 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_server_request": 431, 2023-09-13T03:29:39.157 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_request": 79662, 2023-09-13T03:29:39.157 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_session": 1535, 2023-09-13T03:29:39.158 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_peer_request": 2441, 2023-09-13T03:29:39.158 INFO:teuthology.orchestra.run.smithi170.stdout: "req_create_latency": { 2023-09-13T03:29:39.158 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 11924, 2023-09-13T03:29:39.158 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 40.547487951, 2023-09-13T03:29:39.159 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.003400493 2023-09-13T03:29:39.159 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.159 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getattr_latency": { 2023-09-13T03:29:39.160 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 299, 2023-09-13T03:29:39.160 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 5.187122705, 2023-09-13T03:29:39.160 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.017348236 2023-09-13T03:29:39.160 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.161 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getfilelock_latency": { 2023-09-13T03:29:39.161 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:39.161 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:39.161 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:39.162 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.162 INFO:teuthology.orchestra.run.smithi170.stdout: "req_link_latency": { 2023-09-13T03:29:39.162 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:39.163 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:39.163 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:39.163 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.163 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookup_latency": { 2023-09-13T03:29:39.164 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 19908, 2023-09-13T03:29:39.164 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 41.421480623, 2023-09-13T03:29:39.164 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.002080644 2023-09-13T03:29:39.165 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.165 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookuphash_latency": { 2023-09-13T03:29:39.165 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:39.165 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:39.166 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:39.166 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.166 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupino_latency": { 2023-09-13T03:29:39.167 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:39.167 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:39.167 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:39.167 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.168 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupname_latency": { 2023-09-13T03:29:39.168 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:39.168 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:39.168 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:39.169 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.169 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupparent_latency": { 2023-09-13T03:29:39.169 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:39.170 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:39.170 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:39.170 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.170 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupsnap_latency": { 2023-09-13T03:29:39.171 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:39.171 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:39.171 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:39.171 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.172 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lssnap_latency": { 2023-09-13T03:29:39.172 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:39.172 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:39.173 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:39.173 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.173 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mkdir_latency": { 2023-09-13T03:29:39.173 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:39.174 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:39.174 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:39.174 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.175 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mknod_latency": { 2023-09-13T03:29:39.175 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:39.175 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:39.176 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:39.176 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.176 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mksnap_latency": { 2023-09-13T03:29:39.176 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:39.177 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:39.177 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:39.177 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.178 INFO:teuthology.orchestra.run.smithi170.stdout: "req_open_latency": { 2023-09-13T03:29:39.178 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:39.178 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:39.178 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:39.179 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.179 INFO:teuthology.orchestra.run.smithi170.stdout: "req_readdir_latency": { 2023-09-13T03:29:39.179 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 27499, 2023-09-13T03:29:39.179 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 131.289091769, 2023-09-13T03:29:39.180 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.004774322 2023-09-13T03:29:39.180 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.180 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rename_latency": { 2023-09-13T03:29:39.181 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2637, 2023-09-13T03:29:39.181 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 35.080630384, 2023-09-13T03:29:39.181 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.013303234 2023-09-13T03:29:39.181 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.182 INFO:teuthology.orchestra.run.smithi170.stdout: "req_renamesnap_latency": { 2023-09-13T03:29:39.182 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:39.182 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:39.183 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:39.183 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.183 INFO:teuthology.orchestra.run.smithi170.stdout: "req_snapdiff_latency": { 2023-09-13T03:29:39.183 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:39.184 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:39.184 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:39.184 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.185 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmdir_latency": { 2023-09-13T03:29:39.185 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:39.185 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:39.185 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:39.186 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.186 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmsnap_latency": { 2023-09-13T03:29:39.186 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:39.187 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:39.187 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:39.187 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.187 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmxattr_latency": { 2023-09-13T03:29:39.188 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:39.188 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:39.188 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:39.189 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.189 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setattr_latency": { 2023-09-13T03:29:39.189 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1, 2023-09-13T03:29:39.189 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.007999441, 2023-09-13T03:29:39.190 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.007999441 2023-09-13T03:29:39.190 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.190 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setdirlayout_latency": { 2023-09-13T03:29:39.191 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:39.191 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:39.191 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:39.191 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.192 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setfilelock_latency": { 2023-09-13T03:29:39.192 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 344, 2023-09-13T03:29:39.192 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.420452924, 2023-09-13T03:29:39.193 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001222246 2023-09-13T03:29:39.193 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.193 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setlayout_latency": { 2023-09-13T03:29:39.193 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:39.194 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:39.194 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:39.194 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.195 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setxattr_latency": { 2023-09-13T03:29:39.195 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:39.195 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:39.195 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:39.196 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.196 INFO:teuthology.orchestra.run.smithi170.stdout: "req_symlink_latency": { 2023-09-13T03:29:39.196 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:39.197 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:39.197 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:39.197 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.198 INFO:teuthology.orchestra.run.smithi170.stdout: "req_unlink_latency": { 2023-09-13T03:29:39.198 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 11940, 2023-09-13T03:29:39.198 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 31.162665717, 2023-09-13T03:29:39.198 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.002609938 2023-09-13T03:29:39.199 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.199 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_revoke_eviction": 0, 2023-09-13T03:29:39.199 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_acquisition_throttle": 0, 2023-09-13T03:29:39.200 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getvxattr_latency": { 2023-09-13T03:29:39.200 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:39.200 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:39.201 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:39.201 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-09-13T03:29:39.201 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.202 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_sessions": { 2023-09-13T03:29:39.202 INFO:teuthology.orchestra.run.smithi170.stdout: "session_count": 2, 2023-09-13T03:29:39.202 INFO:teuthology.orchestra.run.smithi170.stdout: "session_add": 4, 2023-09-13T03:29:39.203 INFO:teuthology.orchestra.run.smithi170.stdout: "session_remove": 2, 2023-09-13T03:29:39.203 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_open": 2, 2023-09-13T03:29:39.203 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_stale": 0, 2023-09-13T03:29:39.204 INFO:teuthology.orchestra.run.smithi170.stdout: "total_load": 25314, 2023-09-13T03:29:39.204 INFO:teuthology.orchestra.run.smithi170.stdout: "average_load": 12657, 2023-09-13T03:29:39.204 INFO:teuthology.orchestra.run.smithi170.stdout: "avg_session_uptime": 959, 2023-09-13T03:29:39.204 INFO:teuthology.orchestra.run.smithi170.stdout: "mdthresh_evicted": 0 2023-09-13T03:29:39.205 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.205 INFO:teuthology.orchestra.run.smithi170.stdout: "mempool": { 2023-09-13T03:29:39.205 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_bytes": 0, 2023-09-13T03:29:39.206 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_items": 0, 2023-09-13T03:29:39.206 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_bytes": 0, 2023-09-13T03:29:39.206 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_items": 0, 2023-09-13T03:29:39.206 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_bytes": 0, 2023-09-13T03:29:39.207 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_items": 0, 2023-09-13T03:29:39.207 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-13T03:29:39.207 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_items": 0, 2023-09-13T03:29:39.207 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-13T03:29:39.208 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_items": 0, 2023-09-13T03:29:39.208 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_bytes": 0, 2023-09-13T03:29:39.208 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_items": 0, 2023-09-13T03:29:39.209 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-13T03:29:39.209 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_items": 0, 2023-09-13T03:29:39.209 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_bytes": 0, 2023-09-13T03:29:39.209 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_items": 0, 2023-09-13T03:29:39.210 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_bytes": 0, 2023-09-13T03:29:39.210 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_items": 0, 2023-09-13T03:29:39.210 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-13T03:29:39.211 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_items": 0, 2023-09-13T03:29:39.211 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-13T03:29:39.211 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_items": 0, 2023-09-13T03:29:39.211 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_bytes": 0, 2023-09-13T03:29:39.212 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_items": 0, 2023-09-13T03:29:39.212 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_bytes": 0, 2023-09-13T03:29:39.212 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_items": 0, 2023-09-13T03:29:39.212 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-13T03:29:39.213 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_items": 0, 2023-09-13T03:29:39.213 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_bytes": 0, 2023-09-13T03:29:39.213 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_items": 0, 2023-09-13T03:29:39.214 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_bytes": 0, 2023-09-13T03:29:39.214 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_items": 0, 2023-09-13T03:29:39.214 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_bytes": 0, 2023-09-13T03:29:39.214 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_items": 0, 2023-09-13T03:29:39.215 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_bytes": 0, 2023-09-13T03:29:39.215 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_items": 0, 2023-09-13T03:29:39.215 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_bytes": 2180444, 2023-09-13T03:29:39.216 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_items": 1398, 2023-09-13T03:29:39.216 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_bytes": 440, 2023-09-13T03:29:39.216 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_items": 5, 2023-09-13T03:29:39.216 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_bytes": 0, 2023-09-13T03:29:39.217 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_items": 0, 2023-09-13T03:29:39.217 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_bytes": 0, 2023-09-13T03:29:39.217 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_items": 0, 2023-09-13T03:29:39.218 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_bytes": 0, 2023-09-13T03:29:39.218 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_items": 0, 2023-09-13T03:29:39.218 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_bytes": 7936, 2023-09-13T03:29:39.218 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_items": 165, 2023-09-13T03:29:39.219 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_bytes": 0, 2023-09-13T03:29:39.219 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_items": 0, 2023-09-13T03:29:39.219 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_bytes": 0, 2023-09-13T03:29:39.220 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_items": 0, 2023-09-13T03:29:39.220 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_bytes": 4629626, 2023-09-13T03:29:39.220 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_items": 95506, 2023-09-13T03:29:39.220 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_bytes": 0, 2023-09-13T03:29:39.221 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_items": 0, 2023-09-13T03:29:39.221 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_bytes": 0, 2023-09-13T03:29:39.221 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_items": 0 2023-09-13T03:29:39.222 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.222 INFO:teuthology.orchestra.run.smithi170.stdout: "objecter": { 2023-09-13T03:29:39.222 INFO:teuthology.orchestra.run.smithi170.stdout: "op_active": 0, 2023-09-13T03:29:39.222 INFO:teuthology.orchestra.run.smithi170.stdout: "op_laggy": 0, 2023-09-13T03:29:39.223 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send": 31497, 2023-09-13T03:29:39.223 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send_bytes": 212731340, 2023-09-13T03:29:39.223 INFO:teuthology.orchestra.run.smithi170.stdout: "op_resend": 28, 2023-09-13T03:29:39.224 INFO:teuthology.orchestra.run.smithi170.stdout: "op_reply": 31497, 2023-09-13T03:29:39.224 INFO:teuthology.orchestra.run.smithi170.stdout: "op_latency": { 2023-09-13T03:29:39.224 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 31497, 2023-09-13T03:29:39.225 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 191.968444169, 2023-09-13T03:29:39.225 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.006094816 2023-09-13T03:29:39.225 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.226 INFO:teuthology.orchestra.run.smithi170.stdout: "op_inflight": 0, 2023-09-13T03:29:39.227 INFO:teuthology.orchestra.run.smithi170.stdout: "oplen_avg": { 2023-09-13T03:29:39.227 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 31497, 2023-09-13T03:29:39.227 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 31982 2023-09-13T03:29:39.228 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.228 INFO:teuthology.orchestra.run.smithi170.stdout: "op": 31497, 2023-09-13T03:29:39.228 INFO:teuthology.orchestra.run.smithi170.stdout: "op_r": 540, 2023-09-13T03:29:39.229 INFO:teuthology.orchestra.run.smithi170.stdout: "op_w": 30957, 2023-09-13T03:29:39.229 INFO:teuthology.orchestra.run.smithi170.stdout: "op_rmw": 0, 2023-09-13T03:29:39.229 INFO:teuthology.orchestra.run.smithi170.stdout: "op_pg": 0, 2023-09-13T03:29:39.229 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_stat": 0, 2023-09-13T03:29:39.230 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_create": 10, 2023-09-13T03:29:39.230 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_read": 499, 2023-09-13T03:29:39.230 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_write": 18430, 2023-09-13T03:29:39.231 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writefull": 310, 2023-09-13T03:29:39.231 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writesame": 0, 2023-09-13T03:29:39.231 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_append": 0, 2023-09-13T03:29:39.231 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_zero": 0, 2023-09-13T03:29:39.232 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_truncate": 12, 2023-09-13T03:29:39.232 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_delete": 11894, 2023-09-13T03:29:39.232 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_mapext": 0, 2023-09-13T03:29:39.232 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_sparse_read": 0, 2023-09-13T03:29:39.233 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_clonerange": 0, 2023-09-13T03:29:39.233 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_getxattr": 12, 2023-09-13T03:29:39.233 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_setxattr": 20, 2023-09-13T03:29:39.234 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_cmpxattr": 0, 2023-09-13T03:29:39.234 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_rmxattr": 0, 2023-09-13T03:29:39.234 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_resetxattrs": 0, 2023-09-13T03:29:39.234 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_call": 0, 2023-09-13T03:29:39.235 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_watch": 0, 2023-09-13T03:29:39.235 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_notify": 0, 2023-09-13T03:29:39.235 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_src_cmpxattr": 0, 2023-09-13T03:29:39.235 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls": 0, 2023-09-13T03:29:39.236 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls_filter": 0, 2023-09-13T03:29:39.236 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_other": 0, 2023-09-13T03:29:39.236 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_active": 0, 2023-09-13T03:29:39.237 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_send": 0, 2023-09-13T03:29:39.237 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_resend": 0, 2023-09-13T03:29:39.237 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_ping": 0, 2023-09-13T03:29:39.238 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_active": 0, 2023-09-13T03:29:39.238 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_send": 0, 2023-09-13T03:29:39.238 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_resend": 0, 2023-09-13T03:29:39.238 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_active": 0, 2023-09-13T03:29:39.239 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_send": 0, 2023-09-13T03:29:39.239 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_resend": 0, 2023-09-13T03:29:39.239 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_active": 0, 2023-09-13T03:29:39.239 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_send": 0, 2023-09-13T03:29:39.240 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_resend": 0, 2023-09-13T03:29:39.240 INFO:teuthology.orchestra.run.smithi170.stdout: "command_active": 0, 2023-09-13T03:29:39.240 INFO:teuthology.orchestra.run.smithi170.stdout: "command_send": 0, 2023-09-13T03:29:39.241 INFO:teuthology.orchestra.run.smithi170.stdout: "command_resend": 0, 2023-09-13T03:29:39.241 INFO:teuthology.orchestra.run.smithi170.stdout: "map_epoch": 106, 2023-09-13T03:29:39.242 INFO:teuthology.orchestra.run.smithi170.stdout: "map_full": 0, 2023-09-13T03:29:39.242 INFO:teuthology.orchestra.run.smithi170.stdout: "map_inc": 23, 2023-09-13T03:29:39.243 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_sessions": 12, 2023-09-13T03:29:39.243 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_open": 12, 2023-09-13T03:29:39.243 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_close": 0, 2023-09-13T03:29:39.244 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_laggy": 0, 2023-09-13T03:29:39.244 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_wr": 575, 2023-09-13T03:29:39.245 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_rd": 58, 2023-09-13T03:29:39.245 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_del": 162 2023-09-13T03:29:39.245 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.246 INFO:teuthology.orchestra.run.smithi170.stdout: "oft": { 2023-09-13T03:29:39.246 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_objs": 1, 2023-09-13T03:29:39.246 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_kv_pairs": 0, 2023-09-13T03:29:39.247 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_updates": 0, 2023-09-13T03:29:39.247 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_removes": 0 2023-09-13T03:29:39.247 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.248 INFO:teuthology.orchestra.run.smithi170.stdout: "osd-slow-ops": { 2023-09-13T03:29:39.248 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_ops_count": 3 2023-09-13T03:29:39.248 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.248 INFO:teuthology.orchestra.run.smithi170.stdout: "purge_queue": { 2023-09-13T03:29:39.249 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops": 0, 2023-09-13T03:29:39.249 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops_high_water": 6, 2023-09-13T03:29:39.249 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing": 0, 2023-09-13T03:29:39.250 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_high_water": 6, 2023-09-13T03:29:39.250 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed": 11788, 2023-09-13T03:29:39.250 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_item_in_journal": 0 2023-09-13T03:29:39.250 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.251 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-13T03:29:39.251 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2023-09-13T03:29:39.251 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2023-09-13T03:29:39.252 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2023-09-13T03:29:39.252 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 219843, 2023-09-13T03:29:39.252 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 53131419, 2023-09-13T03:29:39.252 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:39.253 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 219843, 2023-09-13T03:29:39.253 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2023-09-13T03:29:39.253 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2023-09-13T03:29:39.253 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 219843, 2023-09-13T03:29:39.254 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 53131419, 2023-09-13T03:29:39.254 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2023-09-13T03:29:39.254 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:39.255 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:39.255 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:39.255 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-09-13T03:29:39.255 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.256 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_bytes": { 2023-09-13T03:29:39.256 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2023-09-13T03:29:39.256 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2023-09-13T03:29:39.257 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2023-09-13T03:29:39.257 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2023-09-13T03:29:39.257 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2023-09-13T03:29:39.258 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:39.258 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2023-09-13T03:29:39.258 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 31497, 2023-09-13T03:29:39.259 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 213922467, 2023-09-13T03:29:39.259 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 31497, 2023-09-13T03:29:39.260 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 213922467, 2023-09-13T03:29:39.260 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2023-09-13T03:29:39.261 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:39.261 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:39.262 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:39.262 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-09-13T03:29:39.262 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.263 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_ops": { 2023-09-13T03:29:39.263 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2023-09-13T03:29:39.263 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 1024, 2023-09-13T03:29:39.263 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2023-09-13T03:29:39.264 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2023-09-13T03:29:39.264 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2023-09-13T03:29:39.264 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:39.265 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2023-09-13T03:29:39.265 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 31497, 2023-09-13T03:29:39.265 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 31497, 2023-09-13T03:29:39.265 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 31497, 2023-09-13T03:29:39.266 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 31497, 2023-09-13T03:29:39.266 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2023-09-13T03:29:39.266 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:39.266 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:39.267 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:39.267 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-09-13T03:29:39.267 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.268 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle": { 2023-09-13T03:29:39.268 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2023-09-13T03:29:39.268 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2023-09-13T03:29:39.268 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2023-09-13T03:29:39.269 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 11788, 2023-09-13T03:29:39.269 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 1192148, 2023-09-13T03:29:39.269 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:39.269 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 11788, 2023-09-13T03:29:39.270 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2023-09-13T03:29:39.270 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2023-09-13T03:29:39.270 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 499, 2023-09-13T03:29:39.271 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 1192148, 2023-09-13T03:29:39.271 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2023-09-13T03:29:39.271 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:39.271 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:39.272 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:39.272 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-09-13T03:29:39.272 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:39.273 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle-0x55b955af86e0": { 2023-09-13T03:29:39.273 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 345974, 2023-09-13T03:29:39.273 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2023-09-13T03:29:39.273 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2023-09-13T03:29:39.274 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 61292, 2023-09-13T03:29:39.274 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 210061174, 2023-09-13T03:29:39.275 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:39.275 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 61292, 2023-09-13T03:29:39.275 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2023-09-13T03:29:39.276 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2023-09-13T03:29:39.276 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 17931, 2023-09-13T03:29:39.277 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 209715200, 2023-09-13T03:29:39.277 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2023-09-13T03:29:39.277 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:39.278 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:39.278 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:39.279 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-09-13T03:29:39.279 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-09-13T03:29:39.280 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-09-13T03:29:39.281 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:38 smithi142 ceph-mon[134082]: pgmap v517: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 5.9 MiB/s wr, 233 op/s 2023-09-13T03:29:39.281 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:38 smithi064 ceph-mon[97675]: pgmap v517: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 5.9 MiB/s wr, 233 op/s 2023-09-13T03:29:39.281 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:38 smithi170 ceph-mon[134328]: pgmap v517: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 5.9 MiB/s wr, 233 op/s 2023-09-13T03:29:39.532 INFO:tasks.check_counter:Daemon mds.c mds.exported=9 2023-09-13T03:29:39.533 INFO:tasks.check_counter:Daemon mds.c mds.imported=9 2023-09-13T03:29:39.533 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2023-09-13T03:29:39.534 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2023-09-13T03:29:39.534 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2023-09-13T03:29:39.534 DEBUG:tasks.check_counter:Getting stats from f 2023-09-13T03:29:39.535 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph daemon mds.f perf dump 2023-09-13T03:29:40.104 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-13T03:29:40.615 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:40 smithi170 ceph-mon[134328]: pgmap v518: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 4.6 MiB/s wr, 178 op/s 2023-09-13T03:29:40.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:40 smithi064 ceph-mon[97675]: pgmap v518: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 4.6 MiB/s wr, 178 op/s 2023-09-13T03:29:40.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:40 smithi142 ceph-mon[134082]: pgmap v518: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 4.6 MiB/s wr, 178 op/s 2023-09-13T03:29:41.400 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-09-13T03:29:41.401 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-0": { 2023-09-13T03:29:41.401 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 151428, 2023-09-13T03:29:41.401 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 235208, 2023-09-13T03:29:41.402 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 48094742, 2023-09-13T03:29:41.402 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 251712535, 2023-09-13T03:29:41.402 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 25, 2023-09-13T03:29:41.403 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 7, 2023-09-13T03:29:41.403 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 8.513216526, 2023-09-13T03:29:41.403 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 4.024477589, 2023-09-13T03:29:41.404 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 5.577592026, 2023-09-13T03:29:41.404 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.509333634, 2023-09-13T03:29:41.404 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T03:29:41.404 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 235203, 2023-09-13T03:29:41.405 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 36.552116232, 2023-09-13T03:29:41.405 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000155406 2023-09-13T03:29:41.405 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.406 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2023-09-13T03:29:41.406 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 186801, 2023-09-13T03:29:41.406 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.051738056, 2023-09-13T03:29:41.406 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000276 2023-09-13T03:29:41.407 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.407 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-09-13T03:29:41.407 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 448 2023-09-13T03:29:41.407 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.408 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-1": { 2023-09-13T03:29:41.408 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 89841, 2023-09-13T03:29:41.408 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 88660, 2023-09-13T03:29:41.409 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 26755508, 2023-09-13T03:29:41.409 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 93323696, 2023-09-13T03:29:41.409 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 12, 2023-09-13T03:29:41.410 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 5, 2023-09-13T03:29:41.410 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 4.930906889, 2023-09-13T03:29:41.410 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 1.816969549, 2023-09-13T03:29:41.410 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 3.044088309, 2023-09-13T03:29:41.411 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.554390801, 2023-09-13T03:29:41.411 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T03:29:41.411 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 88661, 2023-09-13T03:29:41.412 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 7.802831810, 2023-09-13T03:29:41.412 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000088007 2023-09-13T03:29:41.412 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.413 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2023-09-13T03:29:41.413 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 152033, 2023-09-13T03:29:41.413 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.027911933, 2023-09-13T03:29:41.413 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000183 2023-09-13T03:29:41.414 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.414 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 1072, 2023-09-13T03:29:41.414 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 124672 2023-09-13T03:29:41.414 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.415 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-2": { 2023-09-13T03:29:41.415 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 33872, 2023-09-13T03:29:41.415 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 37326, 2023-09-13T03:29:41.416 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 11230846, 2023-09-13T03:29:41.416 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 117888866, 2023-09-13T03:29:41.416 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 8, 2023-09-13T03:29:41.416 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 6, 2023-09-13T03:29:41.417 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 2.841092119, 2023-09-13T03:29:41.417 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 1.204517194, 2023-09-13T03:29:41.417 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 1.159735390, 2023-09-13T03:29:41.418 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.577417190, 2023-09-13T03:29:41.418 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T03:29:41.418 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 37325, 2023-09-13T03:29:41.419 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 4.265905295, 2023-09-13T03:29:41.419 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000114290 2023-09-13T03:29:41.419 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.420 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2023-09-13T03:29:41.420 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 41304, 2023-09-13T03:29:41.420 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.008101014, 2023-09-13T03:29:41.421 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000196 2023-09-13T03:29:41.421 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.421 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 1988208, 2023-09-13T03:29:41.422 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 628736 2023-09-13T03:29:41.422 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.422 INFO:teuthology.orchestra.run.smithi170.stdout: "cct": { 2023-09-13T03:29:41.423 INFO:teuthology.orchestra.run.smithi170.stdout: "total_workers": 1, 2023-09-13T03:29:41.423 INFO:teuthology.orchestra.run.smithi170.stdout: "unhealthy_workers": 0 2023-09-13T03:29:41.423 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.423 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-MDSRank": { 2023-09-13T03:29:41.424 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2023-09-13T03:29:41.424 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2023-09-13T03:29:41.424 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 43063, 2023-09-13T03:29:41.424 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 52.579873481, 2023-09-13T03:29:41.425 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001220998 2023-09-13T03:29:41.425 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-09-13T03:29:41.425 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.426 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-PurgeQueue": { 2023-09-13T03:29:41.426 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2023-09-13T03:29:41.426 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2023-09-13T03:29:41.426 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 26556, 2023-09-13T03:29:41.427 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 3.970746522, 2023-09-13T03:29:41.427 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000149523 2023-09-13T03:29:41.427 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-09-13T03:29:41.428 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.428 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": { 2023-09-13T03:29:41.428 INFO:teuthology.orchestra.run.smithi170.stdout: "request": 87586, 2023-09-13T03:29:41.429 INFO:teuthology.orchestra.run.smithi170.stdout: "reply": 87413, 2023-09-13T03:29:41.429 INFO:teuthology.orchestra.run.smithi170.stdout: "reply_latency": { 2023-09-13T03:29:41.429 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 87413, 2023-09-13T03:29:41.429 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 301.285774684, 2023-09-13T03:29:41.430 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.003446692 2023-09-13T03:29:41.430 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.430 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_reply": 0, 2023-09-13T03:29:41.431 INFO:teuthology.orchestra.run.smithi170.stdout: "forward": 173, 2023-09-13T03:29:41.431 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_complete": 16, 2023-09-13T03:29:41.431 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_keys": 0, 2023-09-13T03:29:41.431 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_commit": 121, 2023-09-13T03:29:41.432 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_split": 13, 2023-09-13T03:29:41.432 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_merge": 5, 2023-09-13T03:29:41.433 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes": 2503, 2023-09-13T03:29:41.433 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_top": 591, 2023-09-13T03:29:41.433 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_bottom": 1569, 2023-09-13T03:29:41.433 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pin_tail": 343, 2023-09-13T03:29:41.434 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pinned": 1549, 2023-09-13T03:29:41.434 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_expired": 241, 2023-09-13T03:29:41.434 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_with_caps": 270, 2023-09-13T03:29:41.434 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": 270, 2023-09-13T03:29:41.435 INFO:teuthology.orchestra.run.smithi170.stdout: "subtrees": 16, 2023-09-13T03:29:41.435 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse": 149228, 2023-09-13T03:29:41.435 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_hit": 83263, 2023-09-13T03:29:41.436 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_forward": 168, 2023-09-13T03:29:41.436 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_discover": 62, 2023-09-13T03:29:41.436 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_dir_fetch": 10, 2023-09-13T03:29:41.436 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_remote_ino": 0, 2023-09-13T03:29:41.437 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_lock": 1800, 2023-09-13T03:29:41.437 INFO:teuthology.orchestra.run.smithi170.stdout: "load_cent": 8363, 2023-09-13T03:29:41.437 INFO:teuthology.orchestra.run.smithi170.stdout: "q": 0, 2023-09-13T03:29:41.438 INFO:teuthology.orchestra.run.smithi170.stdout: "exported": 6, 2023-09-13T03:29:41.438 INFO:teuthology.orchestra.run.smithi170.stdout: "exported_inodes": 177, 2023-09-13T03:29:41.438 INFO:teuthology.orchestra.run.smithi170.stdout: "imported": 9, 2023-09-13T03:29:41.439 INFO:teuthology.orchestra.run.smithi170.stdout: "imported_inodes": 328, 2023-09-13T03:29:41.439 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_dir_fetch": 0, 2023-09-13T03:29:41.439 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_backtrace_fetch": 0, 2023-09-13T03:29:41.439 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_peer_discover": 0, 2023-09-13T03:29:41.440 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rfiles": 693, 2023-09-13T03:29:41.440 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rbytes": 211762747, 2023-09-13T03:29:41.440 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rsnaps": 6, 2023-09-13T03:29:41.441 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_fetch": 0, 2023-09-13T03:29:41.441 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_set_tag": 0, 2023-09-13T03:29:41.441 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_repaired": 0, 2023-09-13T03:29:41.441 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_inotable_repaired": 0, 2023-09-13T03:29:41.442 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_inodes": 0, 2023-09-13T03:29:41.442 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_base_inodes": 0, 2023-09-13T03:29:41.442 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dirfrag_rstats": 0, 2023-09-13T03:29:41.443 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_file_inodes": 0, 2023-09-13T03:29:41.443 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_inode_file_caps": 6, 2023-09-13T03:29:41.443 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_revoke": 3800, 2023-09-13T03:29:41.444 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_grant": 63050, 2023-09-13T03:29:41.444 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_trunc": 0, 2023-09-13T03:29:41.444 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flushsnap_ack": 2221, 2023-09-13T03:29:41.444 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-13T03:29:41.445 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps": 27486, 2023-09-13T03:29:41.445 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps_dirty": 17617, 2023-09-13T03:29:41.446 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_cap_release": 4606, 2023-09-13T03:29:41.446 INFO:teuthology.orchestra.run.smithi170.stdout: "process_request_cap_release": 63244 2023-09-13T03:29:41.447 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.447 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_cache": { 2023-09-13T03:29:41.447 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update": 8, 2023-09-13T03:29:41.448 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update_receipt": 80, 2023-09-13T03:29:41.448 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_try_discover": 53, 2023-09-13T03:29:41.448 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_send_discover": 116, 2023-09-13T03:29:41.449 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_handle_discover": 206, 2023-09-13T03:29:41.449 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays": 142, 2023-09-13T03:29:41.449 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_delayed": 0, 2023-09-13T03:29:41.450 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_enqueuing": 0, 2023-09-13T03:29:41.450 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_created": 14995, 2023-09-13T03:29:41.450 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_enqueued": 14853, 2023-09-13T03:29:41.451 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_reintegrated": 0, 2023-09-13T03:29:41.451 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_migrated": 0, 2023-09-13T03:29:41.451 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_processing": 0, 2023-09-13T03:29:41.451 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_enqueued": 0, 2023-09-13T03:29:41.452 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_prioritized": 0, 2023-09-13T03:29:41.452 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_started": 0, 2023-09-13T03:29:41.453 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_completed": 0, 2023-09-13T03:29:41.453 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_enqueue_scrub": 0, 2023-09-13T03:29:41.454 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_exportdir": 9, 2023-09-13T03:29:41.454 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_flush": 0, 2023-09-13T03:29:41.454 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragmentdir": 19, 2023-09-13T03:29:41.455 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragstats": 0, 2023-09-13T03:29:41.455 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_inodestats": 0 2023-09-13T03:29:41.456 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.456 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_log": { 2023-09-13T03:29:41.457 INFO:teuthology.orchestra.run.smithi170.stdout: "evlrg": 0, 2023-09-13T03:29:41.457 INFO:teuthology.orchestra.run.smithi170.stdout: "evadd": 77746, 2023-09-13T03:29:41.457 INFO:teuthology.orchestra.run.smithi170.stdout: "evex": 0, 2023-09-13T03:29:41.458 INFO:teuthology.orchestra.run.smithi170.stdout: "evtrm": 0, 2023-09-13T03:29:41.458 INFO:teuthology.orchestra.run.smithi170.stdout: "ev": 77746, 2023-09-13T03:29:41.458 INFO:teuthology.orchestra.run.smithi170.stdout: "evexg": 0, 2023-09-13T03:29:41.459 INFO:teuthology.orchestra.run.smithi170.stdout: "evexd": 0, 2023-09-13T03:29:41.459 INFO:teuthology.orchestra.run.smithi170.stdout: "segadd": 107, 2023-09-13T03:29:41.459 INFO:teuthology.orchestra.run.smithi170.stdout: "segex": 0, 2023-09-13T03:29:41.460 INFO:teuthology.orchestra.run.smithi170.stdout: "segtrm": 0, 2023-09-13T03:29:41.460 INFO:teuthology.orchestra.run.smithi170.stdout: "seg": 107, 2023-09-13T03:29:41.460 INFO:teuthology.orchestra.run.smithi170.stdout: "segmjr": 8, 2023-09-13T03:29:41.460 INFO:teuthology.orchestra.run.smithi170.stdout: "segexg": 0, 2023-09-13T03:29:41.461 INFO:teuthology.orchestra.run.smithi170.stdout: "segexd": 0, 2023-09-13T03:29:41.461 INFO:teuthology.orchestra.run.smithi170.stdout: "expos": 4194304, 2023-09-13T03:29:41.461 INFO:teuthology.orchestra.run.smithi170.stdout: "wrpos": 264146690, 2023-09-13T03:29:41.461 INFO:teuthology.orchestra.run.smithi170.stdout: "rdpos": 0, 2023-09-13T03:29:41.462 INFO:teuthology.orchestra.run.smithi170.stdout: "jlat": { 2023-09-13T03:29:41.462 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 23377, 2023-09-13T03:29:41.462 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 120.423887107, 2023-09-13T03:29:41.463 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.005151383 2023-09-13T03:29:41.463 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.463 INFO:teuthology.orchestra.run.smithi170.stdout: "replayed": 0 2023-09-13T03:29:41.463 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.464 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_mem": { 2023-09-13T03:29:41.464 INFO:teuthology.orchestra.run.smithi170.stdout: "ino": 920, 2023-09-13T03:29:41.464 INFO:teuthology.orchestra.run.smithi170.stdout: "ino+": 15913, 2023-09-13T03:29:41.465 INFO:teuthology.orchestra.run.smithi170.stdout: "ino-": 14993, 2023-09-13T03:29:41.465 INFO:teuthology.orchestra.run.smithi170.stdout: "dir": 174, 2023-09-13T03:29:41.465 INFO:teuthology.orchestra.run.smithi170.stdout: "dir+": 291, 2023-09-13T03:29:41.465 INFO:teuthology.orchestra.run.smithi170.stdout: "dir-": 117, 2023-09-13T03:29:41.466 INFO:teuthology.orchestra.run.smithi170.stdout: "dn": 2503, 2023-09-13T03:29:41.466 INFO:teuthology.orchestra.run.smithi170.stdout: "dn+": 29086, 2023-09-13T03:29:41.466 INFO:teuthology.orchestra.run.smithi170.stdout: "dn-": 26583, 2023-09-13T03:29:41.466 INFO:teuthology.orchestra.run.smithi170.stdout: "cap": 270, 2023-09-13T03:29:41.467 INFO:teuthology.orchestra.run.smithi170.stdout: "cap+": 16123, 2023-09-13T03:29:41.467 INFO:teuthology.orchestra.run.smithi170.stdout: "cap-": 15853, 2023-09-13T03:29:41.468 INFO:teuthology.orchestra.run.smithi170.stdout: "rss": 72600, 2023-09-13T03:29:41.468 INFO:teuthology.orchestra.run.smithi170.stdout: "heap": 198960 2023-09-13T03:29:41.469 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.469 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_server": { 2023-09-13T03:29:41.469 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_client_request": 130936, 2023-09-13T03:29:41.470 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_server_request": 20, 2023-09-13T03:29:41.470 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_request": 87586, 2023-09-13T03:29:41.471 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_session": 2688, 2023-09-13T03:29:41.471 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_peer_request": 6732, 2023-09-13T03:29:41.472 INFO:teuthology.orchestra.run.smithi170.stdout: "req_create_latency": { 2023-09-13T03:29:41.472 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 14956, 2023-09-13T03:29:41.473 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 46.722020057, 2023-09-13T03:29:41.473 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.003123964 2023-09-13T03:29:41.473 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.474 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getattr_latency": { 2023-09-13T03:29:41.474 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 107, 2023-09-13T03:29:41.475 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.249470814, 2023-09-13T03:29:41.475 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.002331502 2023-09-13T03:29:41.476 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.476 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getfilelock_latency": { 2023-09-13T03:29:41.477 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:41.477 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:41.477 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:41.478 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.478 INFO:teuthology.orchestra.run.smithi170.stdout: "req_link_latency": { 2023-09-13T03:29:41.479 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:41.479 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:41.480 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:41.480 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.480 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookup_latency": { 2023-09-13T03:29:41.481 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 27758, 2023-09-13T03:29:41.481 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 36.809839928, 2023-09-13T03:29:41.482 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001326098 2023-09-13T03:29:41.482 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.482 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookuphash_latency": { 2023-09-13T03:29:41.483 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:41.483 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:41.484 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:41.484 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.485 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupino_latency": { 2023-09-13T03:29:41.485 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1, 2023-09-13T03:29:41.485 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000125367, 2023-09-13T03:29:41.486 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000125367 2023-09-13T03:29:41.486 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.487 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupname_latency": { 2023-09-13T03:29:41.487 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:41.488 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:41.488 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:41.489 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.489 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupparent_latency": { 2023-09-13T03:29:41.489 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:41.490 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:41.490 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:41.491 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.491 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupsnap_latency": { 2023-09-13T03:29:41.492 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:41.492 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:41.492 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:41.493 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.493 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lssnap_latency": { 2023-09-13T03:29:41.494 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:41.494 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:41.495 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:41.495 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.496 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mkdir_latency": { 2023-09-13T03:29:41.496 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:41.496 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:41.497 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:41.497 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.498 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mknod_latency": { 2023-09-13T03:29:41.498 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:41.499 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:41.499 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:41.500 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.500 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mksnap_latency": { 2023-09-13T03:29:41.501 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:41.501 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:41.501 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:41.502 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.502 INFO:teuthology.orchestra.run.smithi170.stdout: "req_open_latency": { 2023-09-13T03:29:41.503 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 3, 2023-09-13T03:29:41.503 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.001698356, 2023-09-13T03:29:41.503 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000566118 2023-09-13T03:29:41.503 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.504 INFO:teuthology.orchestra.run.smithi170.stdout: "req_readdir_latency": { 2023-09-13T03:29:41.504 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 25664, 2023-09-13T03:29:41.504 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 127.872447914, 2023-09-13T03:29:41.505 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.004982561 2023-09-13T03:29:41.505 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.505 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rename_latency": { 2023-09-13T03:29:41.505 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 3526, 2023-09-13T03:29:41.506 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 42.969730872, 2023-09-13T03:29:41.506 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.012186537 2023-09-13T03:29:41.506 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.506 INFO:teuthology.orchestra.run.smithi170.stdout: "req_renamesnap_latency": { 2023-09-13T03:29:41.507 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:41.507 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:41.507 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:41.508 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.508 INFO:teuthology.orchestra.run.smithi170.stdout: "req_snapdiff_latency": { 2023-09-13T03:29:41.508 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:41.508 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:41.509 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:41.509 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.509 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmdir_latency": { 2023-09-13T03:29:41.510 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:41.510 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:41.510 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:41.510 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.511 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmsnap_latency": { 2023-09-13T03:29:41.511 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:41.511 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:41.511 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:41.512 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.512 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmxattr_latency": { 2023-09-13T03:29:41.512 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:41.513 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:41.513 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:41.513 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.513 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setattr_latency": { 2023-09-13T03:29:41.514 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 59, 2023-09-13T03:29:41.514 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.257613154, 2023-09-13T03:29:41.514 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.004366324 2023-09-13T03:29:41.515 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.515 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setdirlayout_latency": { 2023-09-13T03:29:41.515 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:41.515 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:41.516 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:41.516 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.516 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setfilelock_latency": { 2023-09-13T03:29:41.517 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 344, 2023-09-13T03:29:41.517 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.956337041, 2023-09-13T03:29:41.517 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.002780049 2023-09-13T03:29:41.517 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.518 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setlayout_latency": { 2023-09-13T03:29:41.518 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:41.518 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:41.519 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:41.519 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.519 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setxattr_latency": { 2023-09-13T03:29:41.519 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:41.520 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:41.520 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:41.520 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.521 INFO:teuthology.orchestra.run.smithi170.stdout: "req_symlink_latency": { 2023-09-13T03:29:41.521 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:41.521 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:41.521 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:41.522 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.522 INFO:teuthology.orchestra.run.smithi170.stdout: "req_unlink_latency": { 2023-09-13T03:29:41.522 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 14995, 2023-09-13T03:29:41.522 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 45.446491181, 2023-09-13T03:29:41.523 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.003030776 2023-09-13T03:29:41.523 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.523 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_revoke_eviction": 0, 2023-09-13T03:29:41.524 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_acquisition_throttle": 0, 2023-09-13T03:29:41.524 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getvxattr_latency": { 2023-09-13T03:29:41.524 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:41.524 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:41.525 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:41.525 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-09-13T03:29:41.525 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.526 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_sessions": { 2023-09-13T03:29:41.526 INFO:teuthology.orchestra.run.smithi170.stdout: "session_count": 1, 2023-09-13T03:29:41.526 INFO:teuthology.orchestra.run.smithi170.stdout: "session_add": 2, 2023-09-13T03:29:41.526 INFO:teuthology.orchestra.run.smithi170.stdout: "session_remove": 1, 2023-09-13T03:29:41.527 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_open": 1, 2023-09-13T03:29:41.527 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_stale": 0, 2023-09-13T03:29:41.528 INFO:teuthology.orchestra.run.smithi170.stdout: "total_load": 28062, 2023-09-13T03:29:41.528 INFO:teuthology.orchestra.run.smithi170.stdout: "average_load": 28062, 2023-09-13T03:29:41.529 INFO:teuthology.orchestra.run.smithi170.stdout: "avg_session_uptime": 1008, 2023-09-13T03:29:41.529 INFO:teuthology.orchestra.run.smithi170.stdout: "mdthresh_evicted": 0 2023-09-13T03:29:41.530 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.530 INFO:teuthology.orchestra.run.smithi170.stdout: "mempool": { 2023-09-13T03:29:41.530 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_bytes": 0, 2023-09-13T03:29:41.531 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_items": 0, 2023-09-13T03:29:41.531 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_bytes": 0, 2023-09-13T03:29:41.532 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_items": 0, 2023-09-13T03:29:41.532 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_bytes": 0, 2023-09-13T03:29:41.533 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_items": 0, 2023-09-13T03:29:41.533 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-13T03:29:41.534 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_items": 0, 2023-09-13T03:29:41.534 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-13T03:29:41.535 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_items": 0, 2023-09-13T03:29:41.535 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_bytes": 0, 2023-09-13T03:29:41.536 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_items": 0, 2023-09-13T03:29:41.536 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-13T03:29:41.537 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_items": 0, 2023-09-13T03:29:41.537 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_bytes": 0, 2023-09-13T03:29:41.538 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_items": 0, 2023-09-13T03:29:41.538 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_bytes": 0, 2023-09-13T03:29:41.539 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_items": 0, 2023-09-13T03:29:41.539 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-13T03:29:41.540 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_items": 0, 2023-09-13T03:29:41.541 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-13T03:29:41.541 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_items": 0, 2023-09-13T03:29:41.542 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_bytes": 0, 2023-09-13T03:29:41.543 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_items": 0, 2023-09-13T03:29:41.543 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_bytes": 0, 2023-09-13T03:29:41.544 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_items": 0, 2023-09-13T03:29:41.545 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-13T03:29:41.545 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_items": 0, 2023-09-13T03:29:41.546 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_bytes": 0, 2023-09-13T03:29:41.546 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_items": 0, 2023-09-13T03:29:41.547 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_bytes": 0, 2023-09-13T03:29:41.547 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_items": 0, 2023-09-13T03:29:41.548 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_bytes": 0, 2023-09-13T03:29:41.548 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_items": 0, 2023-09-13T03:29:41.549 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_bytes": 0, 2023-09-13T03:29:41.549 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_items": 0, 2023-09-13T03:29:41.549 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_bytes": 1877085, 2023-09-13T03:29:41.550 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_items": 1241, 2023-09-13T03:29:41.550 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_bytes": 264, 2023-09-13T03:29:41.550 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_items": 3, 2023-09-13T03:29:41.551 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_bytes": 0, 2023-09-13T03:29:41.551 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_items": 0, 2023-09-13T03:29:41.551 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_bytes": 0, 2023-09-13T03:29:41.551 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_items": 0, 2023-09-13T03:29:41.552 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_bytes": 0, 2023-09-13T03:29:41.552 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_items": 0, 2023-09-13T03:29:41.552 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_bytes": 7936, 2023-09-13T03:29:41.553 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_items": 165, 2023-09-13T03:29:41.553 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_bytes": 0, 2023-09-13T03:29:41.553 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_items": 0, 2023-09-13T03:29:41.553 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_bytes": 0, 2023-09-13T03:29:41.554 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_items": 0, 2023-09-13T03:29:41.554 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_bytes": 4939423, 2023-09-13T03:29:41.554 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_items": 85388, 2023-09-13T03:29:41.555 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_bytes": 0, 2023-09-13T03:29:41.555 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_items": 0, 2023-09-13T03:29:41.555 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_bytes": 0, 2023-09-13T03:29:41.555 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_items": 0 2023-09-13T03:29:41.556 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.556 INFO:teuthology.orchestra.run.smithi170.stdout: "objecter": { 2023-09-13T03:29:41.556 INFO:teuthology.orchestra.run.smithi170.stdout: "op_active": 0, 2023-09-13T03:29:41.556 INFO:teuthology.orchestra.run.smithi170.stdout: "op_laggy": 0, 2023-09-13T03:29:41.557 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send": 40451, 2023-09-13T03:29:41.557 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send_bytes": 263656568, 2023-09-13T03:29:41.557 INFO:teuthology.orchestra.run.smithi170.stdout: "op_resend": 28, 2023-09-13T03:29:41.558 INFO:teuthology.orchestra.run.smithi170.stdout: "op_reply": 40451, 2023-09-13T03:29:41.558 INFO:teuthology.orchestra.run.smithi170.stdout: "op_latency": { 2023-09-13T03:29:41.558 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 40451, 2023-09-13T03:29:41.558 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 213.412933282, 2023-09-13T03:29:41.559 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.005275838 2023-09-13T03:29:41.559 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.559 INFO:teuthology.orchestra.run.smithi170.stdout: "op_inflight": 0, 2023-09-13T03:29:41.560 INFO:teuthology.orchestra.run.smithi170.stdout: "oplen_avg": { 2023-09-13T03:29:41.560 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 40451, 2023-09-13T03:29:41.560 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 40982 2023-09-13T03:29:41.560 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.561 INFO:teuthology.orchestra.run.smithi170.stdout: "op": 40451, 2023-09-13T03:29:41.561 INFO:teuthology.orchestra.run.smithi170.stdout: "op_r": 703, 2023-09-13T03:29:41.561 INFO:teuthology.orchestra.run.smithi170.stdout: "op_w": 39748, 2023-09-13T03:29:41.561 INFO:teuthology.orchestra.run.smithi170.stdout: "op_rmw": 0, 2023-09-13T03:29:41.562 INFO:teuthology.orchestra.run.smithi170.stdout: "op_pg": 0, 2023-09-13T03:29:41.562 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_stat": 0, 2023-09-13T03:29:41.562 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_create": 10, 2023-09-13T03:29:41.563 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_read": 672, 2023-09-13T03:29:41.563 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_write": 24049, 2023-09-13T03:29:41.563 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writefull": 400, 2023-09-13T03:29:41.563 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writesame": 0, 2023-09-13T03:29:41.564 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_append": 0, 2023-09-13T03:29:41.564 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_zero": 0, 2023-09-13T03:29:41.564 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_truncate": 13, 2023-09-13T03:29:41.564 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_delete": 14966, 2023-09-13T03:29:41.565 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_mapext": 0, 2023-09-13T03:29:41.565 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_sparse_read": 0, 2023-09-13T03:29:41.565 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_clonerange": 0, 2023-09-13T03:29:41.566 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_getxattr": 15, 2023-09-13T03:29:41.566 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_setxattr": 20, 2023-09-13T03:29:41.566 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_cmpxattr": 0, 2023-09-13T03:29:41.566 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_rmxattr": 0, 2023-09-13T03:29:41.567 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_resetxattrs": 0, 2023-09-13T03:29:41.567 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_call": 0, 2023-09-13T03:29:41.567 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_watch": 0, 2023-09-13T03:29:41.568 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_notify": 0, 2023-09-13T03:29:41.568 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_src_cmpxattr": 0, 2023-09-13T03:29:41.568 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls": 0, 2023-09-13T03:29:41.568 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls_filter": 0, 2023-09-13T03:29:41.569 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_other": 0, 2023-09-13T03:29:41.569 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_active": 0, 2023-09-13T03:29:41.569 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_send": 0, 2023-09-13T03:29:41.569 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_resend": 0, 2023-09-13T03:29:41.570 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_ping": 0, 2023-09-13T03:29:41.570 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_active": 0, 2023-09-13T03:29:41.570 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_send": 0, 2023-09-13T03:29:41.571 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_resend": 0, 2023-09-13T03:29:41.571 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_active": 0, 2023-09-13T03:29:41.571 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_send": 0, 2023-09-13T03:29:41.571 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_resend": 0, 2023-09-13T03:29:41.572 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_active": 0, 2023-09-13T03:29:41.572 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_send": 0, 2023-09-13T03:29:41.572 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_resend": 0, 2023-09-13T03:29:41.572 INFO:teuthology.orchestra.run.smithi170.stdout: "command_active": 0, 2023-09-13T03:29:41.573 INFO:teuthology.orchestra.run.smithi170.stdout: "command_send": 0, 2023-09-13T03:29:41.573 INFO:teuthology.orchestra.run.smithi170.stdout: "command_resend": 0, 2023-09-13T03:29:41.573 INFO:teuthology.orchestra.run.smithi170.stdout: "map_epoch": 106, 2023-09-13T03:29:41.574 INFO:teuthology.orchestra.run.smithi170.stdout: "map_full": 0, 2023-09-13T03:29:41.574 INFO:teuthology.orchestra.run.smithi170.stdout: "map_inc": 23, 2023-09-13T03:29:41.574 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_sessions": 12, 2023-09-13T03:29:41.574 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_open": 12, 2023-09-13T03:29:41.575 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_close": 0, 2023-09-13T03:29:41.575 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_laggy": 0, 2023-09-13T03:29:41.575 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_wr": 599, 2023-09-13T03:29:41.575 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_rd": 47, 2023-09-13T03:29:41.576 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_del": 191 2023-09-13T03:29:41.576 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.576 INFO:teuthology.orchestra.run.smithi170.stdout: "oft": { 2023-09-13T03:29:41.577 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_objs": 1, 2023-09-13T03:29:41.577 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_kv_pairs": 0, 2023-09-13T03:29:41.577 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_updates": 0, 2023-09-13T03:29:41.577 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_removes": 0 2023-09-13T03:29:41.578 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.578 INFO:teuthology.orchestra.run.smithi170.stdout: "osd-slow-ops": { 2023-09-13T03:29:41.578 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_ops_count": 1 2023-09-13T03:29:41.579 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.579 INFO:teuthology.orchestra.run.smithi170.stdout: "purge_queue": { 2023-09-13T03:29:41.579 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops": 0, 2023-09-13T03:29:41.579 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops_high_water": 6, 2023-09-13T03:29:41.580 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing": 0, 2023-09-13T03:29:41.580 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_high_water": 6, 2023-09-13T03:29:41.580 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed": 14853, 2023-09-13T03:29:41.580 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_item_in_journal": 0 2023-09-13T03:29:41.581 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.581 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-13T03:29:41.581 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2023-09-13T03:29:41.582 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2023-09-13T03:29:41.582 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2023-09-13T03:29:41.582 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 275137, 2023-09-13T03:29:41.582 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 63274141, 2023-09-13T03:29:41.583 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:41.583 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 275137, 2023-09-13T03:29:41.583 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2023-09-13T03:29:41.583 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2023-09-13T03:29:41.584 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 275137, 2023-09-13T03:29:41.584 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 63274141, 2023-09-13T03:29:41.584 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2023-09-13T03:29:41.585 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:41.585 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:41.585 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:41.585 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-09-13T03:29:41.586 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.586 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_bytes": { 2023-09-13T03:29:41.586 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2023-09-13T03:29:41.586 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2023-09-13T03:29:41.587 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2023-09-13T03:29:41.587 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2023-09-13T03:29:41.587 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2023-09-13T03:29:41.588 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:41.588 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2023-09-13T03:29:41.588 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 40451, 2023-09-13T03:29:41.588 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 265157688, 2023-09-13T03:29:41.589 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 40451, 2023-09-13T03:29:41.589 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 265157688, 2023-09-13T03:29:41.589 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2023-09-13T03:29:41.590 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:41.590 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:41.590 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:41.590 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-09-13T03:29:41.591 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.591 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_ops": { 2023-09-13T03:29:41.591 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2023-09-13T03:29:41.592 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 1024, 2023-09-13T03:29:41.592 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2023-09-13T03:29:41.592 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2023-09-13T03:29:41.592 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2023-09-13T03:29:41.593 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:41.593 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2023-09-13T03:29:41.593 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 40451, 2023-09-13T03:29:41.594 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 40451, 2023-09-13T03:29:41.594 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 40451, 2023-09-13T03:29:41.594 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 40451, 2023-09-13T03:29:41.595 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2023-09-13T03:29:41.595 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:41.595 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:41.595 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:41.596 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-09-13T03:29:41.596 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.596 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle": { 2023-09-13T03:29:41.597 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2023-09-13T03:29:41.597 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2023-09-13T03:29:41.597 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2023-09-13T03:29:41.597 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 14853, 2023-09-13T03:29:41.598 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 1501849, 2023-09-13T03:29:41.598 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:41.598 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 14853, 2023-09-13T03:29:41.599 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2023-09-13T03:29:41.599 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2023-09-13T03:29:41.599 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 672, 2023-09-13T03:29:41.599 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 1501849, 2023-09-13T03:29:41.600 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2023-09-13T03:29:41.600 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:41.600 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:41.601 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:41.601 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-09-13T03:29:41.601 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:41.601 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle-0x55e8ae01e6e0": { 2023-09-13T03:29:41.602 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 21154, 2023-09-13T03:29:41.602 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2023-09-13T03:29:41.602 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2023-09-13T03:29:41.603 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 77746, 2023-09-13T03:29:41.603 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 259952386, 2023-09-13T03:29:41.603 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:41.603 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 77746, 2023-09-13T03:29:41.604 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2023-09-13T03:29:41.604 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2023-09-13T03:29:41.604 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 23377, 2023-09-13T03:29:41.605 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 259931232, 2023-09-13T03:29:41.605 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2023-09-13T03:29:41.605 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:41.605 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:41.606 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:41.606 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-09-13T03:29:41.606 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-09-13T03:29:41.607 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-09-13T03:29:41.759 INFO:tasks.check_counter:Daemon mds.f mds.exported=6 2023-09-13T03:29:41.760 INFO:tasks.check_counter:Daemon mds.f mds.imported=9 2023-09-13T03:29:41.760 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2023-09-13T03:29:41.760 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2023-09-13T03:29:41.761 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2023-09-13T03:29:41.761 DEBUG:tasks.check_counter:Getting stats from i 2023-09-13T03:29:41.761 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph daemon mds.i perf dump 2023-09-13T03:29:42.317 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-13T03:29:43.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:42 smithi170 ceph-mon[134328]: pgmap v519: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 6.0 MiB/s wr, 227 op/s 2023-09-13T03:29:43.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:42 smithi064 ceph-mon[97675]: pgmap v519: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 6.0 MiB/s wr, 227 op/s 2023-09-13T03:29:43.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:42 smithi142 ceph-mon[134082]: pgmap v519: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 6.0 MiB/s wr, 227 op/s 2023-09-13T03:29:43.523 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-09-13T03:29:43.524 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-0": { 2023-09-13T03:29:43.524 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 150342, 2023-09-13T03:29:43.525 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 231043, 2023-09-13T03:29:43.525 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 48129486, 2023-09-13T03:29:43.525 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 221559812, 2023-09-13T03:29:43.526 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 17, 2023-09-13T03:29:43.526 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 6, 2023-09-13T03:29:43.526 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 8.387795822, 2023-09-13T03:29:43.527 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 4.035073125, 2023-09-13T03:29:43.527 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 5.387414618, 2023-09-13T03:29:43.527 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.381405931, 2023-09-13T03:29:43.527 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T03:29:43.528 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 231040, 2023-09-13T03:29:43.528 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 33.499350192, 2023-09-13T03:29:43.528 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000144993 2023-09-13T03:29:43.529 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.529 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2023-09-13T03:29:43.529 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 189408, 2023-09-13T03:29:43.529 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.053650963, 2023-09-13T03:29:43.530 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000283 2023-09-13T03:29:43.530 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.530 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 2005136, 2023-09-13T03:29:43.531 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 196272 2023-09-13T03:29:43.531 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.531 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-1": { 2023-09-13T03:29:43.532 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 117885, 2023-09-13T03:29:43.532 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 111268, 2023-09-13T03:29:43.532 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 34892525, 2023-09-13T03:29:43.532 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 134373635, 2023-09-13T03:29:43.533 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 11, 2023-09-13T03:29:43.533 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 6, 2023-09-13T03:29:43.533 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 6.662351158, 2023-09-13T03:29:43.534 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 2.625177885, 2023-09-13T03:29:43.534 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 4.217197115, 2023-09-13T03:29:43.534 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.614889688, 2023-09-13T03:29:43.534 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T03:29:43.535 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 111271, 2023-09-13T03:29:43.535 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 10.813975065, 2023-09-13T03:29:43.535 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000097185 2023-09-13T03:29:43.535 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.536 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2023-09-13T03:29:43.536 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 200626, 2023-09-13T03:29:43.536 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.032502661, 2023-09-13T03:29:43.537 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000162 2023-09-13T03:29:43.537 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.537 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 5296, 2023-09-13T03:29:43.537 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 132944 2023-09-13T03:29:43.538 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.538 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-2": { 2023-09-13T03:29:43.538 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 13721, 2023-09-13T03:29:43.539 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 13926, 2023-09-13T03:29:43.539 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 4899674, 2023-09-13T03:29:43.539 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 97628643, 2023-09-13T03:29:43.540 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 20, 2023-09-13T03:29:43.540 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 7, 2023-09-13T03:29:43.540 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 1.230053818, 2023-09-13T03:29:43.540 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.423942281, 2023-09-13T03:29:43.541 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.226863878, 2023-09-13T03:29:43.541 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.504892482, 2023-09-13T03:29:43.541 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T03:29:43.542 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 13921, 2023-09-13T03:29:43.542 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 1.608183845, 2023-09-13T03:29:43.542 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000115522 2023-09-13T03:29:43.543 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.543 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2023-09-13T03:29:43.543 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 16, 2023-09-13T03:29:43.543 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000002138, 2023-09-13T03:29:43.544 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000133 2023-09-13T03:29:43.544 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.544 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 1072, 2023-09-13T03:29:43.544 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 436928 2023-09-13T03:29:43.545 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.545 INFO:teuthology.orchestra.run.smithi170.stdout: "cct": { 2023-09-13T03:29:43.545 INFO:teuthology.orchestra.run.smithi170.stdout: "total_workers": 1, 2023-09-13T03:29:43.546 INFO:teuthology.orchestra.run.smithi170.stdout: "unhealthy_workers": 0 2023-09-13T03:29:43.546 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.546 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-MDSRank": { 2023-09-13T03:29:43.546 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2023-09-13T03:29:43.547 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2023-09-13T03:29:43.547 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 37289, 2023-09-13T03:29:43.547 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 49.401375058, 2023-09-13T03:29:43.548 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001324824 2023-09-13T03:29:43.548 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-09-13T03:29:43.548 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.548 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-PurgeQueue": { 2023-09-13T03:29:43.549 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2023-09-13T03:29:43.549 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2023-09-13T03:29:43.549 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 25679, 2023-09-13T03:29:43.549 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 4.339864880, 2023-09-13T03:29:43.550 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000169004 2023-09-13T03:29:43.550 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-09-13T03:29:43.550 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.551 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": { 2023-09-13T03:29:43.551 INFO:teuthology.orchestra.run.smithi170.stdout: "request": 97074, 2023-09-13T03:29:43.551 INFO:teuthology.orchestra.run.smithi170.stdout: "reply": 90180, 2023-09-13T03:29:43.551 INFO:teuthology.orchestra.run.smithi170.stdout: "reply_latency": { 2023-09-13T03:29:43.552 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 90180, 2023-09-13T03:29:43.552 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 446.793163624, 2023-09-13T03:29:43.552 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.004954459 2023-09-13T03:29:43.553 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.553 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_reply": 0, 2023-09-13T03:29:43.553 INFO:teuthology.orchestra.run.smithi170.stdout: "forward": 6894, 2023-09-13T03:29:43.553 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_complete": 6, 2023-09-13T03:29:43.554 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_keys": 0, 2023-09-13T03:29:43.554 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_commit": 71, 2023-09-13T03:29:43.554 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_split": 7, 2023-09-13T03:29:43.555 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_merge": 4, 2023-09-13T03:29:43.555 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes": 2095, 2023-09-13T03:29:43.555 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_top": 706, 2023-09-13T03:29:43.556 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_bottom": 1063, 2023-09-13T03:29:43.556 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pin_tail": 326, 2023-09-13T03:29:43.556 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pinned": 1086, 2023-09-13T03:29:43.556 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_expired": 510, 2023-09-13T03:29:43.557 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_with_caps": 189, 2023-09-13T03:29:43.557 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": 189, 2023-09-13T03:29:43.557 INFO:teuthology.orchestra.run.smithi170.stdout: "subtrees": 18, 2023-09-13T03:29:43.558 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse": 146815, 2023-09-13T03:29:43.558 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_hit": 82442, 2023-09-13T03:29:43.558 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_forward": 6890, 2023-09-13T03:29:43.559 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_discover": 121, 2023-09-13T03:29:43.559 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_dir_fetch": 4, 2023-09-13T03:29:43.559 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_remote_ino": 0, 2023-09-13T03:29:43.559 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_lock": 652, 2023-09-13T03:29:43.560 INFO:teuthology.orchestra.run.smithi170.stdout: "load_cent": 6947, 2023-09-13T03:29:43.560 INFO:teuthology.orchestra.run.smithi170.stdout: "q": 0, 2023-09-13T03:29:43.560 INFO:teuthology.orchestra.run.smithi170.stdout: "exported": 7, 2023-09-13T03:29:43.561 INFO:teuthology.orchestra.run.smithi170.stdout: "exported_inodes": 217, 2023-09-13T03:29:43.561 INFO:teuthology.orchestra.run.smithi170.stdout: "imported": 7, 2023-09-13T03:29:43.561 INFO:teuthology.orchestra.run.smithi170.stdout: "imported_inodes": 1113, 2023-09-13T03:29:43.561 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_dir_fetch": 0, 2023-09-13T03:29:43.562 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_backtrace_fetch": 11, 2023-09-13T03:29:43.562 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_peer_discover": 0, 2023-09-13T03:29:43.562 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rfiles": 693, 2023-09-13T03:29:43.562 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rbytes": 211762747, 2023-09-13T03:29:43.563 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rsnaps": 6, 2023-09-13T03:29:43.563 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_fetch": 11, 2023-09-13T03:29:43.563 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_set_tag": 0, 2023-09-13T03:29:43.564 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_repaired": 0, 2023-09-13T03:29:43.564 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_inotable_repaired": 0, 2023-09-13T03:29:43.564 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_inodes": 0, 2023-09-13T03:29:43.564 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_base_inodes": 0, 2023-09-13T03:29:43.565 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dirfrag_rstats": 0, 2023-09-13T03:29:43.565 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_file_inodes": 11, 2023-09-13T03:29:43.565 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_inode_file_caps": 5, 2023-09-13T03:29:43.565 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_revoke": 7899, 2023-09-13T03:29:43.566 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_grant": 57650, 2023-09-13T03:29:43.566 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_trunc": 0, 2023-09-13T03:29:43.566 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flushsnap_ack": 1661, 2023-09-13T03:29:43.567 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-13T03:29:43.567 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps": 20063, 2023-09-13T03:29:43.567 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps_dirty": 9404, 2023-09-13T03:29:43.568 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_cap_release": 3240, 2023-09-13T03:29:43.568 INFO:teuthology.orchestra.run.smithi170.stdout: "process_request_cap_release": 66287 2023-09-13T03:29:43.568 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.568 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_cache": { 2023-09-13T03:29:43.569 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update": 47, 2023-09-13T03:29:43.569 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update_receipt": 49, 2023-09-13T03:29:43.569 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_try_discover": 36, 2023-09-13T03:29:43.570 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_send_discover": 142, 2023-09-13T03:29:43.570 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_handle_discover": 128, 2023-09-13T03:29:43.570 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays": 30, 2023-09-13T03:29:43.570 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_delayed": 0, 2023-09-13T03:29:43.571 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_enqueuing": 0, 2023-09-13T03:29:43.571 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_created": 14463, 2023-09-13T03:29:43.571 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_enqueued": 14433, 2023-09-13T03:29:43.571 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_reintegrated": 0, 2023-09-13T03:29:43.572 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_migrated": 0, 2023-09-13T03:29:43.572 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_processing": 0, 2023-09-13T03:29:43.572 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_enqueued": 0, 2023-09-13T03:29:43.573 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_prioritized": 0, 2023-09-13T03:29:43.573 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_started": 0, 2023-09-13T03:29:43.573 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_completed": 0, 2023-09-13T03:29:43.574 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_enqueue_scrub": 0, 2023-09-13T03:29:43.574 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_exportdir": 12, 2023-09-13T03:29:43.574 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_flush": 0, 2023-09-13T03:29:43.574 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragmentdir": 11, 2023-09-13T03:29:43.575 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragstats": 0, 2023-09-13T03:29:43.575 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_inodestats": 0 2023-09-13T03:29:43.575 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.575 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_log": { 2023-09-13T03:29:43.576 INFO:teuthology.orchestra.run.smithi170.stdout: "evlrg": 0, 2023-09-13T03:29:43.576 INFO:teuthology.orchestra.run.smithi170.stdout: "evadd": 66393, 2023-09-13T03:29:43.576 INFO:teuthology.orchestra.run.smithi170.stdout: "evex": 0, 2023-09-13T03:29:43.576 INFO:teuthology.orchestra.run.smithi170.stdout: "evtrm": 0, 2023-09-13T03:29:43.577 INFO:teuthology.orchestra.run.smithi170.stdout: "ev": 66393, 2023-09-13T03:29:43.577 INFO:teuthology.orchestra.run.smithi170.stdout: "evexg": 0, 2023-09-13T03:29:43.577 INFO:teuthology.orchestra.run.smithi170.stdout: "evexd": 0, 2023-09-13T03:29:43.578 INFO:teuthology.orchestra.run.smithi170.stdout: "segadd": 88, 2023-09-13T03:29:43.578 INFO:teuthology.orchestra.run.smithi170.stdout: "segex": 0, 2023-09-13T03:29:43.578 INFO:teuthology.orchestra.run.smithi170.stdout: "segtrm": 0, 2023-09-13T03:29:43.578 INFO:teuthology.orchestra.run.smithi170.stdout: "seg": 88, 2023-09-13T03:29:43.579 INFO:teuthology.orchestra.run.smithi170.stdout: "segmjr": 7, 2023-09-13T03:29:43.579 INFO:teuthology.orchestra.run.smithi170.stdout: "segexg": 0, 2023-09-13T03:29:43.579 INFO:teuthology.orchestra.run.smithi170.stdout: "segexd": 0, 2023-09-13T03:29:43.580 INFO:teuthology.orchestra.run.smithi170.stdout: "expos": 4194304, 2023-09-13T03:29:43.580 INFO:teuthology.orchestra.run.smithi170.stdout: "wrpos": 256983492, 2023-09-13T03:29:43.580 INFO:teuthology.orchestra.run.smithi170.stdout: "rdpos": 0, 2023-09-13T03:29:43.581 INFO:teuthology.orchestra.run.smithi170.stdout: "jlat": { 2023-09-13T03:29:43.581 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 20448, 2023-09-13T03:29:43.581 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 117.582545845, 2023-09-13T03:29:43.581 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.005750320 2023-09-13T03:29:43.582 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.582 INFO:teuthology.orchestra.run.smithi170.stdout: "replayed": 0 2023-09-13T03:29:43.582 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.583 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_mem": { 2023-09-13T03:29:43.583 INFO:teuthology.orchestra.run.smithi170.stdout: "ino": 1121, 2023-09-13T03:29:43.583 INFO:teuthology.orchestra.run.smithi170.stdout: "ino+": 15917, 2023-09-13T03:29:43.584 INFO:teuthology.orchestra.run.smithi170.stdout: "ino-": 14796, 2023-09-13T03:29:43.584 INFO:teuthology.orchestra.run.smithi170.stdout: "dir": 146, 2023-09-13T03:29:43.584 INFO:teuthology.orchestra.run.smithi170.stdout: "dir+": 215, 2023-09-13T03:29:43.584 INFO:teuthology.orchestra.run.smithi170.stdout: "dir-": 69, 2023-09-13T03:29:43.585 INFO:teuthology.orchestra.run.smithi170.stdout: "dn": 2040, 2023-09-13T03:29:43.585 INFO:teuthology.orchestra.run.smithi170.stdout: "dn+": 28481, 2023-09-13T03:29:43.585 INFO:teuthology.orchestra.run.smithi170.stdout: "dn-": 26441, 2023-09-13T03:29:43.586 INFO:teuthology.orchestra.run.smithi170.stdout: "cap": 189, 2023-09-13T03:29:43.586 INFO:teuthology.orchestra.run.smithi170.stdout: "cap+": 15397, 2023-09-13T03:29:43.586 INFO:teuthology.orchestra.run.smithi170.stdout: "cap-": 15208, 2023-09-13T03:29:43.587 INFO:teuthology.orchestra.run.smithi170.stdout: "rss": 71228, 2023-09-13T03:29:43.587 INFO:teuthology.orchestra.run.smithi170.stdout: "heap": 198960 2023-09-13T03:29:43.587 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.587 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_server": { 2023-09-13T03:29:43.588 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_client_request": 137492, 2023-09-13T03:29:43.588 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_server_request": 3978, 2023-09-13T03:29:43.588 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_request": 97074, 2023-09-13T03:29:43.589 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_session": 1333, 2023-09-13T03:29:43.589 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_peer_request": 11447, 2023-09-13T03:29:43.589 INFO:teuthology.orchestra.run.smithi170.stdout: "req_create_latency": { 2023-09-13T03:29:43.589 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 14481, 2023-09-13T03:29:43.590 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 49.506478815, 2023-09-13T03:29:43.590 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.003418719 2023-09-13T03:29:43.590 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.591 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getattr_latency": { 2023-09-13T03:29:43.591 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 5255, 2023-09-13T03:29:43.591 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 71.131166508, 2023-09-13T03:29:43.592 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.013535902 2023-09-13T03:29:43.592 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.592 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getfilelock_latency": { 2023-09-13T03:29:43.593 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:43.593 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:43.593 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:43.593 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.594 INFO:teuthology.orchestra.run.smithi170.stdout: "req_link_latency": { 2023-09-13T03:29:43.594 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:43.594 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:43.595 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:43.595 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.595 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookup_latency": { 2023-09-13T03:29:43.595 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 20463, 2023-09-13T03:29:43.596 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 142.061914571, 2023-09-13T03:29:43.596 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.006942379 2023-09-13T03:29:43.596 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.597 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookuphash_latency": { 2023-09-13T03:29:43.597 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:43.597 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:43.598 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:43.598 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.598 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupino_latency": { 2023-09-13T03:29:43.598 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:43.599 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:43.599 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:43.599 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.600 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupname_latency": { 2023-09-13T03:29:43.600 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:43.600 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:43.601 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:43.601 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.601 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupparent_latency": { 2023-09-13T03:29:43.601 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:43.602 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:43.602 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:43.602 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.603 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupsnap_latency": { 2023-09-13T03:29:43.603 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:43.603 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:43.603 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:43.604 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.604 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lssnap_latency": { 2023-09-13T03:29:43.604 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:43.604 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:43.605 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:43.605 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.605 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mkdir_latency": { 2023-09-13T03:29:43.606 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:43.606 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:43.606 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:43.606 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.607 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mknod_latency": { 2023-09-13T03:29:43.607 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:43.607 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:43.607 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:43.608 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.608 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mksnap_latency": { 2023-09-13T03:29:43.608 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:43.609 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:43.609 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:43.609 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.610 INFO:teuthology.orchestra.run.smithi170.stdout: "req_open_latency": { 2023-09-13T03:29:43.610 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:43.610 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:43.610 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:43.611 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.611 INFO:teuthology.orchestra.run.smithi170.stdout: "req_readdir_latency": { 2023-09-13T03:29:43.611 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 33220, 2023-09-13T03:29:43.612 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 131.298244190, 2023-09-13T03:29:43.612 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.003952385 2023-09-13T03:29:43.612 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.613 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rename_latency": { 2023-09-13T03:29:43.613 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1896, 2023-09-13T03:29:43.613 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 8.104498485, 2023-09-13T03:29:43.613 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.004274524 2023-09-13T03:29:43.614 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.614 INFO:teuthology.orchestra.run.smithi170.stdout: "req_renamesnap_latency": { 2023-09-13T03:29:43.614 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:43.614 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:43.615 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:43.615 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.615 INFO:teuthology.orchestra.run.smithi170.stdout: "req_snapdiff_latency": { 2023-09-13T03:29:43.615 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:43.616 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:43.616 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:43.616 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.617 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmdir_latency": { 2023-09-13T03:29:43.617 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:43.617 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:43.618 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:43.618 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.618 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmsnap_latency": { 2023-09-13T03:29:43.619 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:43.619 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:43.619 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:43.619 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.620 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmxattr_latency": { 2023-09-13T03:29:43.620 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:43.620 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:43.621 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:43.621 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.621 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setattr_latency": { 2023-09-13T03:29:43.621 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2, 2023-09-13T03:29:43.622 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.004864748, 2023-09-13T03:29:43.622 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.002432374 2023-09-13T03:29:43.622 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.623 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setdirlayout_latency": { 2023-09-13T03:29:43.623 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:43.623 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:43.624 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:43.624 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.624 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setfilelock_latency": { 2023-09-13T03:29:43.625 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 400, 2023-09-13T03:29:43.625 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 1.051660770, 2023-09-13T03:29:43.625 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.002629151 2023-09-13T03:29:43.625 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.626 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setlayout_latency": { 2023-09-13T03:29:43.626 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:43.626 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:43.627 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:43.627 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.627 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setxattr_latency": { 2023-09-13T03:29:43.628 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:43.628 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:43.628 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:43.628 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.629 INFO:teuthology.orchestra.run.smithi170.stdout: "req_symlink_latency": { 2023-09-13T03:29:43.629 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:43.629 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:43.630 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:43.630 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.630 INFO:teuthology.orchestra.run.smithi170.stdout: "req_unlink_latency": { 2023-09-13T03:29:43.630 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 14463, 2023-09-13T03:29:43.631 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 43.634335537, 2023-09-13T03:29:43.631 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.003016962 2023-09-13T03:29:43.631 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.631 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_revoke_eviction": 0, 2023-09-13T03:29:43.632 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_acquisition_throttle": 0, 2023-09-13T03:29:43.632 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getvxattr_latency": { 2023-09-13T03:29:43.632 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:43.633 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:43.633 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:43.633 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-09-13T03:29:43.634 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.634 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_sessions": { 2023-09-13T03:29:43.634 INFO:teuthology.orchestra.run.smithi170.stdout: "session_count": 2, 2023-09-13T03:29:43.635 INFO:teuthology.orchestra.run.smithi170.stdout: "session_add": 5, 2023-09-13T03:29:43.635 INFO:teuthology.orchestra.run.smithi170.stdout: "session_remove": 3, 2023-09-13T03:29:43.637 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_open": 2, 2023-09-13T03:29:43.637 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_stale": 0, 2023-09-13T03:29:43.638 INFO:teuthology.orchestra.run.smithi170.stdout: "total_load": 26763, 2023-09-13T03:29:43.638 INFO:teuthology.orchestra.run.smithi170.stdout: "average_load": 13381, 2023-09-13T03:29:43.638 INFO:teuthology.orchestra.run.smithi170.stdout: "avg_session_uptime": 940, 2023-09-13T03:29:43.638 INFO:teuthology.orchestra.run.smithi170.stdout: "mdthresh_evicted": 0 2023-09-13T03:29:43.639 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.639 INFO:teuthology.orchestra.run.smithi170.stdout: "mempool": { 2023-09-13T03:29:43.639 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_bytes": 0, 2023-09-13T03:29:43.640 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_items": 0, 2023-09-13T03:29:43.640 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_bytes": 0, 2023-09-13T03:29:43.640 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_items": 0, 2023-09-13T03:29:43.641 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_bytes": 0, 2023-09-13T03:29:43.641 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_items": 0, 2023-09-13T03:29:43.641 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-13T03:29:43.641 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_items": 0, 2023-09-13T03:29:43.642 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-13T03:29:43.642 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_items": 0, 2023-09-13T03:29:43.642 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_bytes": 0, 2023-09-13T03:29:43.643 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_items": 0, 2023-09-13T03:29:43.643 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-13T03:29:43.643 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_items": 0, 2023-09-13T03:29:43.643 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_bytes": 0, 2023-09-13T03:29:43.644 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_items": 0, 2023-09-13T03:29:43.644 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_bytes": 0, 2023-09-13T03:29:43.644 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_items": 0, 2023-09-13T03:29:43.645 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-13T03:29:43.645 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_items": 0, 2023-09-13T03:29:43.645 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-13T03:29:43.645 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_items": 0, 2023-09-13T03:29:43.646 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_bytes": 0, 2023-09-13T03:29:43.646 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_items": 0, 2023-09-13T03:29:43.646 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_bytes": 0, 2023-09-13T03:29:43.647 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_items": 0, 2023-09-13T03:29:43.647 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-13T03:29:43.647 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_items": 0, 2023-09-13T03:29:43.647 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_bytes": 0, 2023-09-13T03:29:43.648 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_items": 0, 2023-09-13T03:29:43.648 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_bytes": 0, 2023-09-13T03:29:43.648 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_items": 0, 2023-09-13T03:29:43.648 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_bytes": 0, 2023-09-13T03:29:43.649 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_items": 0, 2023-09-13T03:29:43.649 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_bytes": 0, 2023-09-13T03:29:43.649 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_items": 0, 2023-09-13T03:29:43.649 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_bytes": 782517, 2023-09-13T03:29:43.650 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_items": 1268, 2023-09-13T03:29:43.650 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_bytes": 264, 2023-09-13T03:29:43.650 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_items": 3, 2023-09-13T03:29:43.651 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_bytes": 0, 2023-09-13T03:29:43.651 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_items": 0, 2023-09-13T03:29:43.651 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_bytes": 0, 2023-09-13T03:29:43.652 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_items": 0, 2023-09-13T03:29:43.652 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_bytes": 0, 2023-09-13T03:29:43.652 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_items": 0, 2023-09-13T03:29:43.652 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_bytes": 7936, 2023-09-13T03:29:43.653 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_items": 165, 2023-09-13T03:29:43.653 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_bytes": 0, 2023-09-13T03:29:43.653 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_items": 0, 2023-09-13T03:29:43.653 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_bytes": 0, 2023-09-13T03:29:43.654 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_items": 0, 2023-09-13T03:29:43.654 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_bytes": 4751762, 2023-09-13T03:29:43.654 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_items": 56534, 2023-09-13T03:29:43.655 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_bytes": 0, 2023-09-13T03:29:43.655 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_items": 0, 2023-09-13T03:29:43.655 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_bytes": 0, 2023-09-13T03:29:43.655 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_items": 0 2023-09-13T03:29:43.656 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.656 INFO:teuthology.orchestra.run.smithi170.stdout: "objecter": { 2023-09-13T03:29:43.656 INFO:teuthology.orchestra.run.smithi170.stdout: "op_active": 0, 2023-09-13T03:29:43.657 INFO:teuthology.orchestra.run.smithi170.stdout: "op_laggy": 0, 2023-09-13T03:29:43.657 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send": 36760, 2023-09-13T03:29:43.657 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send_bytes": 255821333, 2023-09-13T03:29:43.658 INFO:teuthology.orchestra.run.smithi170.stdout: "op_resend": 33, 2023-09-13T03:29:43.658 INFO:teuthology.orchestra.run.smithi170.stdout: "op_reply": 36760, 2023-09-13T03:29:43.658 INFO:teuthology.orchestra.run.smithi170.stdout: "op_latency": { 2023-09-13T03:29:43.658 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 36760, 2023-09-13T03:29:43.659 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 201.624214120, 2023-09-13T03:29:43.659 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.005484880 2023-09-13T03:29:43.659 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.660 INFO:teuthology.orchestra.run.smithi170.stdout: "op_inflight": 0, 2023-09-13T03:29:43.660 INFO:teuthology.orchestra.run.smithi170.stdout: "oplen_avg": { 2023-09-13T03:29:43.660 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 36760, 2023-09-13T03:29:43.661 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 37209 2023-09-13T03:29:43.661 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.661 INFO:teuthology.orchestra.run.smithi170.stdout: "op": 36760, 2023-09-13T03:29:43.661 INFO:teuthology.orchestra.run.smithi170.stdout: "op_r": 595, 2023-09-13T03:29:43.662 INFO:teuthology.orchestra.run.smithi170.stdout: "op_w": 36165, 2023-09-13T03:29:43.662 INFO:teuthology.orchestra.run.smithi170.stdout: "op_rmw": 0, 2023-09-13T03:29:43.662 INFO:teuthology.orchestra.run.smithi170.stdout: "op_pg": 0, 2023-09-13T03:29:43.663 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_stat": 0, 2023-09-13T03:29:43.663 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_create": 10, 2023-09-13T03:29:43.663 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_read": 573, 2023-09-13T03:29:43.664 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_write": 21021, 2023-09-13T03:29:43.664 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writefull": 341, 2023-09-13T03:29:43.664 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writesame": 0, 2023-09-13T03:29:43.664 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_append": 0, 2023-09-13T03:29:43.665 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_zero": 0, 2023-09-13T03:29:43.665 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_truncate": 7, 2023-09-13T03:29:43.665 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_delete": 14527, 2023-09-13T03:29:43.666 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_mapext": 0, 2023-09-13T03:29:43.666 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_sparse_read": 0, 2023-09-13T03:29:43.666 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_clonerange": 0, 2023-09-13T03:29:43.666 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_getxattr": 16, 2023-09-13T03:29:43.667 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_setxattr": 20, 2023-09-13T03:29:43.667 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_cmpxattr": 0, 2023-09-13T03:29:43.667 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_rmxattr": 0, 2023-09-13T03:29:43.668 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_resetxattrs": 0, 2023-09-13T03:29:43.668 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_call": 0, 2023-09-13T03:29:43.668 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_watch": 0, 2023-09-13T03:29:43.669 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_notify": 0, 2023-09-13T03:29:43.669 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_src_cmpxattr": 0, 2023-09-13T03:29:43.669 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls": 0, 2023-09-13T03:29:43.669 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls_filter": 0, 2023-09-13T03:29:43.670 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_other": 0, 2023-09-13T03:29:43.670 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_active": 0, 2023-09-13T03:29:43.670 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_send": 0, 2023-09-13T03:29:43.671 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_resend": 0, 2023-09-13T03:29:43.671 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_ping": 0, 2023-09-13T03:29:43.671 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_active": 0, 2023-09-13T03:29:43.672 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_send": 0, 2023-09-13T03:29:43.672 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_resend": 0, 2023-09-13T03:29:43.672 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_active": 0, 2023-09-13T03:29:43.672 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_send": 0, 2023-09-13T03:29:43.673 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_resend": 0, 2023-09-13T03:29:43.673 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_active": 0, 2023-09-13T03:29:43.673 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_send": 0, 2023-09-13T03:29:43.673 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_resend": 0, 2023-09-13T03:29:43.674 INFO:teuthology.orchestra.run.smithi170.stdout: "command_active": 0, 2023-09-13T03:29:43.674 INFO:teuthology.orchestra.run.smithi170.stdout: "command_send": 0, 2023-09-13T03:29:43.674 INFO:teuthology.orchestra.run.smithi170.stdout: "command_resend": 0, 2023-09-13T03:29:43.675 INFO:teuthology.orchestra.run.smithi170.stdout: "map_epoch": 106, 2023-09-13T03:29:43.675 INFO:teuthology.orchestra.run.smithi170.stdout: "map_full": 0, 2023-09-13T03:29:43.675 INFO:teuthology.orchestra.run.smithi170.stdout: "map_inc": 23, 2023-09-13T03:29:43.675 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_sessions": 12, 2023-09-13T03:29:43.676 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_open": 12, 2023-09-13T03:29:43.676 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_close": 0, 2023-09-13T03:29:43.676 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_laggy": 0, 2023-09-13T03:29:43.676 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_wr": 496, 2023-09-13T03:29:43.677 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_rd": 17, 2023-09-13T03:29:43.677 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_del": 181 2023-09-13T03:29:43.677 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.678 INFO:teuthology.orchestra.run.smithi170.stdout: "oft": { 2023-09-13T03:29:43.678 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_objs": 1, 2023-09-13T03:29:43.678 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_kv_pairs": 0, 2023-09-13T03:29:43.678 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_updates": 0, 2023-09-13T03:29:43.679 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_removes": 0 2023-09-13T03:29:43.679 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.679 INFO:teuthology.orchestra.run.smithi170.stdout: "osd-slow-ops": { 2023-09-13T03:29:43.679 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_ops_count": 1 2023-09-13T03:29:43.680 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.680 INFO:teuthology.orchestra.run.smithi170.stdout: "purge_queue": { 2023-09-13T03:29:43.680 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops": 0, 2023-09-13T03:29:43.680 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops_high_water": 6, 2023-09-13T03:29:43.681 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing": 0, 2023-09-13T03:29:43.681 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_high_water": 6, 2023-09-13T03:29:43.681 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed": 14433, 2023-09-13T03:29:43.681 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_item_in_journal": 0 2023-09-13T03:29:43.682 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.682 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-13T03:29:43.682 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2023-09-13T03:29:43.683 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2023-09-13T03:29:43.683 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2023-09-13T03:29:43.683 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 281944, 2023-09-13T03:29:43.683 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 64539399, 2023-09-13T03:29:43.684 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:43.684 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 281944, 2023-09-13T03:29:43.684 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2023-09-13T03:29:43.684 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2023-09-13T03:29:43.685 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 281944, 2023-09-13T03:29:43.685 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 64539399, 2023-09-13T03:29:43.685 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2023-09-13T03:29:43.685 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:43.686 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:43.686 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:43.686 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-09-13T03:29:43.686 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.687 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_bytes": { 2023-09-13T03:29:43.687 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2023-09-13T03:29:43.687 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2023-09-13T03:29:43.688 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2023-09-13T03:29:43.688 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2023-09-13T03:29:43.688 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2023-09-13T03:29:43.688 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:43.689 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2023-09-13T03:29:43.689 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 36760, 2023-09-13T03:29:43.689 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 257279807, 2023-09-13T03:29:43.689 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 36760, 2023-09-13T03:29:43.690 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 257279807, 2023-09-13T03:29:43.690 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2023-09-13T03:29:43.690 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:43.691 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:43.691 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:43.691 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-09-13T03:29:43.691 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.692 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_ops": { 2023-09-13T03:29:43.692 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2023-09-13T03:29:43.692 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 1024, 2023-09-13T03:29:43.693 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2023-09-13T03:29:43.693 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2023-09-13T03:29:43.693 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2023-09-13T03:29:43.693 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:43.694 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2023-09-13T03:29:43.694 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 36760, 2023-09-13T03:29:43.694 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 36760, 2023-09-13T03:29:43.695 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 36760, 2023-09-13T03:29:43.695 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 36760, 2023-09-13T03:29:43.695 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2023-09-13T03:29:43.695 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:43.696 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:43.696 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:43.696 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-09-13T03:29:43.696 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.697 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle": { 2023-09-13T03:29:43.697 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2023-09-13T03:29:43.697 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2023-09-13T03:29:43.698 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2023-09-13T03:29:43.698 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 14433, 2023-09-13T03:29:43.698 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 1458725, 2023-09-13T03:29:43.699 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:43.699 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 14433, 2023-09-13T03:29:43.699 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2023-09-13T03:29:43.699 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2023-09-13T03:29:43.700 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 573, 2023-09-13T03:29:43.700 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 1458725, 2023-09-13T03:29:43.700 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2023-09-13T03:29:43.701 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:43.701 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:43.701 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:43.701 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-09-13T03:29:43.702 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:43.702 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle-0x5595062066e0": { 2023-09-13T03:29:43.702 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2023-09-13T03:29:43.703 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2023-09-13T03:29:43.703 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2023-09-13T03:29:43.703 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 66393, 2023-09-13T03:29:43.703 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 252789188, 2023-09-13T03:29:43.704 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:43.704 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 66393, 2023-09-13T03:29:43.704 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2023-09-13T03:29:43.705 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2023-09-13T03:29:43.705 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 20448, 2023-09-13T03:29:43.705 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 252789188, 2023-09-13T03:29:43.705 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2023-09-13T03:29:43.706 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:43.706 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:43.706 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:43.707 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-09-13T03:29:43.707 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-09-13T03:29:43.707 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-09-13T03:29:44.092 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:43 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:29:44.161 INFO:tasks.check_counter:Daemon mds.i mds.exported=7 2023-09-13T03:29:44.162 INFO:tasks.check_counter:Daemon mds.i mds.imported=7 2023-09-13T03:29:44.162 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2023-09-13T03:29:44.162 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2023-09-13T03:29:44.163 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2023-09-13T03:29:44.163 DEBUG:tasks.check_counter:Getting stats from l 2023-09-13T03:29:44.163 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2110e00747b31a0d73768c9e5229da18b26b8aa0 shell --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da -- ceph daemon mds.l perf dump 2023-09-13T03:29:44.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:43 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:29:44.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:43 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-13T03:29:44.715 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-13T03:29:45.128 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:44 smithi170 ceph-mon[134328]: pgmap v520: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 3.6 MiB/s wr, 146 op/s 2023-09-13T03:29:45.128 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:44 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:29:45.129 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:44 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:29:45.129 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:44 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:29:45.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:44 smithi064 ceph-mon[97675]: pgmap v520: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 3.6 MiB/s wr, 146 op/s 2023-09-13T03:29:45.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:44 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:29:45.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:44 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:29:45.169 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:44 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:29:45.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:44 smithi142 ceph-mon[134082]: pgmap v520: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 3.6 MiB/s wr, 146 op/s 2023-09-13T03:29:45.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:44 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-13T03:29:45.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:44 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-13T03:29:45.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:44 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' 2023-09-13T03:29:45.934 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-09-13T03:29:45.934 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-0": { 2023-09-13T03:29:45.935 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 1789, 2023-09-13T03:29:45.935 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 1719, 2023-09-13T03:29:45.935 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 48558954, 2023-09-13T03:29:45.936 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 619850, 2023-09-13T03:29:45.936 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 7, 2023-09-13T03:29:45.936 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 3, 2023-09-13T03:29:45.936 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.222724906, 2023-09-13T03:29:45.937 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.054411401, 2023-09-13T03:29:45.937 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.104904886, 2023-09-13T03:29:45.937 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.060110801, 2023-09-13T03:29:45.937 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T03:29:45.938 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1719, 2023-09-13T03:29:45.938 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.114286855, 2023-09-13T03:29:45.938 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000066484 2023-09-13T03:29:45.939 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:45.939 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2023-09-13T03:29:45.939 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:45.939 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:45.940 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:45.940 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:45.940 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 1980608, 2023-09-13T03:29:45.941 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 190048 2023-09-13T03:29:45.941 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:45.941 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-1": { 2023-09-13T03:29:45.941 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 676, 2023-09-13T03:29:45.942 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 908, 2023-09-13T03:29:45.942 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 75670047, 2023-09-13T03:29:45.942 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 769916, 2023-09-13T03:29:45.943 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 9, 2023-09-13T03:29:45.944 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 7, 2023-09-13T03:29:45.944 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.113232933, 2023-09-13T03:29:45.944 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.021161353, 2023-09-13T03:29:45.945 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.061527779, 2023-09-13T03:29:45.945 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.023185783, 2023-09-13T03:29:45.945 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T03:29:45.945 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 905, 2023-09-13T03:29:45.946 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.063081343, 2023-09-13T03:29:45.946 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000069703 2023-09-13T03:29:45.946 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:45.947 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2023-09-13T03:29:45.947 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:45.947 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:45.948 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:45.948 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:45.948 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 320, 2023-09-13T03:29:45.948 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 560240 2023-09-13T03:29:45.949 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:45.949 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-2": { 2023-09-13T03:29:45.949 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 2002, 2023-09-13T03:29:45.949 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 2004, 2023-09-13T03:29:45.950 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 131687278, 2023-09-13T03:29:45.950 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 622629, 2023-09-13T03:29:45.950 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 7, 2023-09-13T03:29:45.951 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 5, 2023-09-13T03:29:45.951 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.211082524, 2023-09-13T03:29:45.951 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.045219267, 2023-09-13T03:29:45.952 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.103265604, 2023-09-13T03:29:45.952 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.051041136, 2023-09-13T03:29:45.952 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2023-09-13T03:29:45.952 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2002, 2023-09-13T03:29:45.953 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.101662645, 2023-09-13T03:29:45.953 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000050780 2023-09-13T03:29:45.953 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:45.953 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2023-09-13T03:29:45.954 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:45.954 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:45.954 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:45.955 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:45.955 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-13T03:29:45.955 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 0 2023-09-13T03:29:45.955 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:45.956 INFO:teuthology.orchestra.run.smithi170.stdout: "cct": { 2023-09-13T03:29:45.956 INFO:teuthology.orchestra.run.smithi170.stdout: "total_workers": 1, 2023-09-13T03:29:45.956 INFO:teuthology.orchestra.run.smithi170.stdout: "unhealthy_workers": 0 2023-09-13T03:29:45.956 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:45.957 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-MDSRank": { 2023-09-13T03:29:45.957 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2023-09-13T03:29:45.957 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2023-09-13T03:29:45.958 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 6780, 2023-09-13T03:29:45.958 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.179526625, 2023-09-13T03:29:45.958 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000026478 2023-09-13T03:29:45.958 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-09-13T03:29:45.959 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:45.959 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-PurgeQueue": { 2023-09-13T03:29:45.959 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2023-09-13T03:29:45.960 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2023-09-13T03:29:45.960 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:45.960 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:45.960 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:45.961 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-09-13T03:29:45.961 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:45.961 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": { 2023-09-13T03:29:45.962 INFO:teuthology.orchestra.run.smithi170.stdout: "request": 0, 2023-09-13T03:29:45.962 INFO:teuthology.orchestra.run.smithi170.stdout: "reply": 0, 2023-09-13T03:29:45.962 INFO:teuthology.orchestra.run.smithi170.stdout: "reply_latency": { 2023-09-13T03:29:45.962 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:45.963 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:45.963 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:45.963 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:45.963 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_reply": 0, 2023-09-13T03:29:45.964 INFO:teuthology.orchestra.run.smithi170.stdout: "forward": 0, 2023-09-13T03:29:45.964 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_complete": 0, 2023-09-13T03:29:45.964 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_keys": 0, 2023-09-13T03:29:45.965 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_commit": 0, 2023-09-13T03:29:45.965 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_split": 0, 2023-09-13T03:29:45.965 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_merge": 0, 2023-09-13T03:29:45.965 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes": 15759, 2023-09-13T03:29:45.966 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_top": 426, 2023-09-13T03:29:45.966 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_bottom": 14552, 2023-09-13T03:29:45.966 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pin_tail": 781, 2023-09-13T03:29:45.967 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pinned": 15150, 2023-09-13T03:29:45.967 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_expired": 38, 2023-09-13T03:29:45.967 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_with_caps": 0, 2023-09-13T03:29:45.967 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": 0, 2023-09-13T03:29:45.968 INFO:teuthology.orchestra.run.smithi170.stdout: "subtrees": 11, 2023-09-13T03:29:45.968 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse": 0, 2023-09-13T03:29:45.968 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_hit": 0, 2023-09-13T03:29:45.969 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_forward": 0, 2023-09-13T03:29:45.969 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_discover": 0, 2023-09-13T03:29:45.969 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_dir_fetch": 0, 2023-09-13T03:29:45.970 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_remote_ino": 0, 2023-09-13T03:29:45.970 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_lock": 0, 2023-09-13T03:29:45.970 INFO:teuthology.orchestra.run.smithi170.stdout: "load_cent": 0, 2023-09-13T03:29:45.970 INFO:teuthology.orchestra.run.smithi170.stdout: "q": 0, 2023-09-13T03:29:45.971 INFO:teuthology.orchestra.run.smithi170.stdout: "exported": 0, 2023-09-13T03:29:45.971 INFO:teuthology.orchestra.run.smithi170.stdout: "exported_inodes": 0, 2023-09-13T03:29:45.971 INFO:teuthology.orchestra.run.smithi170.stdout: "imported": 0, 2023-09-13T03:29:45.972 INFO:teuthology.orchestra.run.smithi170.stdout: "imported_inodes": 0, 2023-09-13T03:29:45.972 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_dir_fetch": 0, 2023-09-13T03:29:45.972 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_backtrace_fetch": 0, 2023-09-13T03:29:45.973 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_peer_discover": 0, 2023-09-13T03:29:45.973 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rfiles": 0, 2023-09-13T03:29:45.973 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rbytes": 0, 2023-09-13T03:29:45.973 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rsnaps": 0, 2023-09-13T03:29:45.974 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_fetch": 0, 2023-09-13T03:29:45.974 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_set_tag": 0, 2023-09-13T03:29:45.974 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_repaired": 0, 2023-09-13T03:29:45.975 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_inotable_repaired": 0, 2023-09-13T03:29:45.975 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_inodes": 0, 2023-09-13T03:29:45.975 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_base_inodes": 0, 2023-09-13T03:29:45.975 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dirfrag_rstats": 0, 2023-09-13T03:29:45.976 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_file_inodes": 0, 2023-09-13T03:29:45.976 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_inode_file_caps": 0, 2023-09-13T03:29:45.976 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_revoke": 0, 2023-09-13T03:29:45.976 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_grant": 0, 2023-09-13T03:29:45.977 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_trunc": 0, 2023-09-13T03:29:45.977 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-13T03:29:45.977 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-13T03:29:45.978 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps": 0, 2023-09-13T03:29:45.978 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps_dirty": 0, 2023-09-13T03:29:45.978 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_cap_release": 0, 2023-09-13T03:29:45.979 INFO:teuthology.orchestra.run.smithi170.stdout: "process_request_cap_release": 0 2023-09-13T03:29:45.979 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:45.979 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_cache": { 2023-09-13T03:29:45.979 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update": 0, 2023-09-13T03:29:45.980 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update_receipt": 0, 2023-09-13T03:29:45.980 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_try_discover": 0, 2023-09-13T03:29:45.980 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_send_discover": 0, 2023-09-13T03:29:45.981 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_handle_discover": 0, 2023-09-13T03:29:45.981 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays": 0, 2023-09-13T03:29:45.981 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_delayed": 0, 2023-09-13T03:29:45.981 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_enqueuing": 0, 2023-09-13T03:29:45.982 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_created": 0, 2023-09-13T03:29:45.982 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_enqueued": 0, 2023-09-13T03:29:45.982 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_reintegrated": 0, 2023-09-13T03:29:45.983 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_migrated": 0, 2023-09-13T03:29:45.983 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_processing": 0, 2023-09-13T03:29:45.983 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_enqueued": 0, 2023-09-13T03:29:45.983 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_prioritized": 0, 2023-09-13T03:29:45.984 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_started": 0, 2023-09-13T03:29:45.984 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_completed": 0, 2023-09-13T03:29:45.984 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_enqueue_scrub": 0, 2023-09-13T03:29:45.984 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_exportdir": 0, 2023-09-13T03:29:45.985 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_flush": 0, 2023-09-13T03:29:45.985 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragmentdir": 0, 2023-09-13T03:29:45.985 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragstats": 0, 2023-09-13T03:29:45.985 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_inodestats": 0 2023-09-13T03:29:45.986 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:45.986 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_log": { 2023-09-13T03:29:45.986 INFO:teuthology.orchestra.run.smithi170.stdout: "evlrg": 0, 2023-09-13T03:29:45.987 INFO:teuthology.orchestra.run.smithi170.stdout: "evadd": 0, 2023-09-13T03:29:45.987 INFO:teuthology.orchestra.run.smithi170.stdout: "evex": 0, 2023-09-13T03:29:45.987 INFO:teuthology.orchestra.run.smithi170.stdout: "evtrm": 0, 2023-09-13T03:29:45.987 INFO:teuthology.orchestra.run.smithi170.stdout: "ev": 66394, 2023-09-13T03:29:45.988 INFO:teuthology.orchestra.run.smithi170.stdout: "evexg": 0, 2023-09-13T03:29:45.988 INFO:teuthology.orchestra.run.smithi170.stdout: "evexd": 0, 2023-09-13T03:29:45.988 INFO:teuthology.orchestra.run.smithi170.stdout: "segadd": 0, 2023-09-13T03:29:45.988 INFO:teuthology.orchestra.run.smithi170.stdout: "segex": 0, 2023-09-13T03:29:45.989 INFO:teuthology.orchestra.run.smithi170.stdout: "segtrm": 0, 2023-09-13T03:29:45.989 INFO:teuthology.orchestra.run.smithi170.stdout: "seg": 88, 2023-09-13T03:29:45.989 INFO:teuthology.orchestra.run.smithi170.stdout: "segmjr": 7, 2023-09-13T03:29:45.990 INFO:teuthology.orchestra.run.smithi170.stdout: "segexg": 0, 2023-09-13T03:29:45.990 INFO:teuthology.orchestra.run.smithi170.stdout: "segexd": 0, 2023-09-13T03:29:45.990 INFO:teuthology.orchestra.run.smithi170.stdout: "expos": 4194304, 2023-09-13T03:29:45.990 INFO:teuthology.orchestra.run.smithi170.stdout: "wrpos": 256991902, 2023-09-13T03:29:45.991 INFO:teuthology.orchestra.run.smithi170.stdout: "rdpos": 256983492, 2023-09-13T03:29:45.991 INFO:teuthology.orchestra.run.smithi170.stdout: "jlat": { 2023-09-13T03:29:45.991 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:45.992 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:45.992 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:45.992 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:45.992 INFO:teuthology.orchestra.run.smithi170.stdout: "replayed": 66394 2023-09-13T03:29:45.993 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:45.993 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_mem": { 2023-09-13T03:29:45.993 INFO:teuthology.orchestra.run.smithi170.stdout: "ino": 1103, 2023-09-13T03:29:45.993 INFO:teuthology.orchestra.run.smithi170.stdout: "ino+": 15711, 2023-09-13T03:29:45.994 INFO:teuthology.orchestra.run.smithi170.stdout: "ino-": 14608, 2023-09-13T03:29:45.994 INFO:teuthology.orchestra.run.smithi170.stdout: "dir": 90, 2023-09-13T03:29:45.994 INFO:teuthology.orchestra.run.smithi170.stdout: "dir+": 122, 2023-09-13T03:29:45.994 INFO:teuthology.orchestra.run.smithi170.stdout: "dir-": 32, 2023-09-13T03:29:45.995 INFO:teuthology.orchestra.run.smithi170.stdout: "dn": 15759, 2023-09-13T03:29:45.995 INFO:teuthology.orchestra.run.smithi170.stdout: "dn+": 15985, 2023-09-13T03:29:45.995 INFO:teuthology.orchestra.run.smithi170.stdout: "dn-": 226, 2023-09-13T03:29:45.996 INFO:teuthology.orchestra.run.smithi170.stdout: "cap": 0, 2023-09-13T03:29:45.996 INFO:teuthology.orchestra.run.smithi170.stdout: "cap+": 0, 2023-09-13T03:29:45.996 INFO:teuthology.orchestra.run.smithi170.stdout: "cap-": 0, 2023-09-13T03:29:45.996 INFO:teuthology.orchestra.run.smithi170.stdout: "rss": 69004, 2023-09-13T03:29:45.997 INFO:teuthology.orchestra.run.smithi170.stdout: "heap": 207152 2023-09-13T03:29:45.997 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:45.997 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_server": { 2023-09-13T03:29:45.997 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_client_request": 0, 2023-09-13T03:29:45.998 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_server_request": 0, 2023-09-13T03:29:45.998 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_request": 0, 2023-09-13T03:29:45.998 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_session": 0, 2023-09-13T03:29:45.998 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_peer_request": 0, 2023-09-13T03:29:45.999 INFO:teuthology.orchestra.run.smithi170.stdout: "req_create_latency": { 2023-09-13T03:29:45.999 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:45.999 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:46.000 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:46.000 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:46.000 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getattr_latency": { 2023-09-13T03:29:46.001 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:46.001 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:46.001 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:46.002 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:46.002 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getfilelock_latency": { 2023-09-13T03:29:46.002 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:46.003 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:46.003 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:46.003 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:46.004 INFO:teuthology.orchestra.run.smithi170.stdout: "req_link_latency": { 2023-09-13T03:29:46.004 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:46.004 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:46.005 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:46.005 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:46.005 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookup_latency": { 2023-09-13T03:29:46.006 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:46.006 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:46.006 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:46.006 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:46.007 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookuphash_latency": { 2023-09-13T03:29:46.007 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:46.007 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:46.008 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:46.008 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:46.008 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupino_latency": { 2023-09-13T03:29:46.009 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:46.009 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:46.009 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:46.009 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:46.010 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupname_latency": { 2023-09-13T03:29:46.010 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:46.010 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:46.011 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:46.011 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:46.011 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupparent_latency": { 2023-09-13T03:29:46.012 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:46.012 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:46.012 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:46.012 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:46.013 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupsnap_latency": { 2023-09-13T03:29:46.013 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:46.013 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:46.014 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:46.014 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:46.014 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lssnap_latency": { 2023-09-13T03:29:46.014 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:46.015 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:46.015 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:46.015 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:46.016 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mkdir_latency": { 2023-09-13T03:29:46.016 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:46.016 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:46.017 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:46.017 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:46.017 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mknod_latency": { 2023-09-13T03:29:46.017 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:46.018 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:46.018 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:46.018 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:46.019 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mksnap_latency": { 2023-09-13T03:29:46.019 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:46.019 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:46.019 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:46.020 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:46.020 INFO:teuthology.orchestra.run.smithi170.stdout: "req_open_latency": { 2023-09-13T03:29:46.020 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:46.021 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:46.021 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:46.021 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:46.022 INFO:teuthology.orchestra.run.smithi170.stdout: "req_readdir_latency": { 2023-09-13T03:29:46.022 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:46.022 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:46.022 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:46.023 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:46.023 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rename_latency": { 2023-09-13T03:29:46.023 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:46.024 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:46.024 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:46.024 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:46.025 INFO:teuthology.orchestra.run.smithi170.stdout: "req_renamesnap_latency": { 2023-09-13T03:29:46.025 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:46.025 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:46.025 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:46.026 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:46.026 INFO:teuthology.orchestra.run.smithi170.stdout: "req_snapdiff_latency": { 2023-09-13T03:29:46.026 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:46.027 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:46.027 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:46.027 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:46.028 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmdir_latency": { 2023-09-13T03:29:46.028 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:46.028 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:46.028 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:46.029 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:46.029 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmsnap_latency": { 2023-09-13T03:29:46.029 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:46.030 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:46.030 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:46.030 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:46.031 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmxattr_latency": { 2023-09-13T03:29:46.031 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:46.031 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:46.031 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:46.032 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:46.032 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setattr_latency": { 2023-09-13T03:29:46.032 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:46.033 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:46.033 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:46.033 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:46.033 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setdirlayout_latency": { 2023-09-13T03:29:46.034 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:46.034 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:46.034 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:46.035 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:46.035 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setfilelock_latency": { 2023-09-13T03:29:46.035 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:46.036 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:46.036 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:46.036 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:46.036 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setlayout_latency": { 2023-09-13T03:29:46.037 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:46.037 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:46.037 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:46.038 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:46.038 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setxattr_latency": { 2023-09-13T03:29:46.038 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:46.039 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:46.039 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:46.039 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:46.039 INFO:teuthology.orchestra.run.smithi170.stdout: "req_symlink_latency": { 2023-09-13T03:29:46.040 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:46.040 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:46.040 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:46.041 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:46.041 INFO:teuthology.orchestra.run.smithi170.stdout: "req_unlink_latency": { 2023-09-13T03:29:46.041 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:46.041 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:46.042 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:46.042 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:46.042 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_revoke_eviction": 0, 2023-09-13T03:29:46.043 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_acquisition_throttle": 0, 2023-09-13T03:29:46.043 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getvxattr_latency": { 2023-09-13T03:29:46.043 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:46.044 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:46.044 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:46.044 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-09-13T03:29:46.044 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:46.045 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_sessions": { 2023-09-13T03:29:46.045 INFO:teuthology.orchestra.run.smithi170.stdout: "session_count": 2, 2023-09-13T03:29:46.045 INFO:teuthology.orchestra.run.smithi170.stdout: "session_add": 5, 2023-09-13T03:29:46.046 INFO:teuthology.orchestra.run.smithi170.stdout: "session_remove": 3, 2023-09-13T03:29:46.046 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_open": 4, 2023-09-13T03:29:46.046 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_stale": 0, 2023-09-13T03:29:46.047 INFO:teuthology.orchestra.run.smithi170.stdout: "total_load": 0, 2023-09-13T03:29:46.047 INFO:teuthology.orchestra.run.smithi170.stdout: "average_load": 0, 2023-09-13T03:29:46.047 INFO:teuthology.orchestra.run.smithi170.stdout: "avg_session_uptime": 4641, 2023-09-13T03:29:46.047 INFO:teuthology.orchestra.run.smithi170.stdout: "mdthresh_evicted": 0 2023-09-13T03:29:46.048 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:46.048 INFO:teuthology.orchestra.run.smithi170.stdout: "mempool": { 2023-09-13T03:29:46.048 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_bytes": 0, 2023-09-13T03:29:46.049 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_items": 0, 2023-09-13T03:29:46.049 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_bytes": 0, 2023-09-13T03:29:46.049 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_items": 0, 2023-09-13T03:29:46.049 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_bytes": 0, 2023-09-13T03:29:46.050 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_items": 0, 2023-09-13T03:29:46.050 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-13T03:29:46.050 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_items": 0, 2023-09-13T03:29:46.051 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-13T03:29:46.051 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_items": 0, 2023-09-13T03:29:46.051 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_bytes": 0, 2023-09-13T03:29:46.052 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_items": 0, 2023-09-13T03:29:46.052 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-13T03:29:46.052 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_items": 0, 2023-09-13T03:29:46.052 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_bytes": 0, 2023-09-13T03:29:46.053 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_items": 0, 2023-09-13T03:29:46.053 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_bytes": 0, 2023-09-13T03:29:46.053 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_items": 0, 2023-09-13T03:29:46.054 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-13T03:29:46.054 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_items": 0, 2023-09-13T03:29:46.054 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-13T03:29:46.055 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_items": 0, 2023-09-13T03:29:46.055 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_bytes": 0, 2023-09-13T03:29:46.055 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_items": 0, 2023-09-13T03:29:46.056 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_bytes": 0, 2023-09-13T03:29:46.056 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_items": 0, 2023-09-13T03:29:46.056 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-13T03:29:46.056 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_items": 0, 2023-09-13T03:29:46.057 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_bytes": 0, 2023-09-13T03:29:46.057 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_items": 0, 2023-09-13T03:29:46.057 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_bytes": 0, 2023-09-13T03:29:46.058 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_items": 0, 2023-09-13T03:29:46.058 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_bytes": 0, 2023-09-13T03:29:46.058 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_items": 0, 2023-09-13T03:29:46.059 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_bytes": 0, 2023-09-13T03:29:46.059 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_items": 0, 2023-09-13T03:29:46.059 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_bytes": 283961, 2023-09-13T03:29:46.059 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_items": 1368, 2023-09-13T03:29:46.060 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_bytes": 0, 2023-09-13T03:29:46.060 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_items": 0, 2023-09-13T03:29:46.060 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_bytes": 0, 2023-09-13T03:29:46.061 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_items": 0, 2023-09-13T03:29:46.061 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_bytes": 0, 2023-09-13T03:29:46.061 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_items": 0, 2023-09-13T03:29:46.062 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_bytes": 0, 2023-09-13T03:29:46.062 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_items": 0, 2023-09-13T03:29:46.062 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_bytes": 7936, 2023-09-13T03:29:46.063 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_items": 165, 2023-09-13T03:29:46.063 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_bytes": 0, 2023-09-13T03:29:46.063 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_items": 0, 2023-09-13T03:29:46.063 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_bytes": 0, 2023-09-13T03:29:46.064 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_items": 0, 2023-09-13T03:29:46.064 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_bytes": 13811375, 2023-09-13T03:29:46.064 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_items": 93898, 2023-09-13T03:29:46.065 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_bytes": 0, 2023-09-13T03:29:46.065 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_items": 0, 2023-09-13T03:29:46.065 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_bytes": 0, 2023-09-13T03:29:46.066 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_items": 0 2023-09-13T03:29:46.066 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:46.066 INFO:teuthology.orchestra.run.smithi170.stdout: "objecter": { 2023-09-13T03:29:46.066 INFO:teuthology.orchestra.run.smithi170.stdout: "op_active": 0, 2023-09-13T03:29:46.067 INFO:teuthology.orchestra.run.smithi170.stdout: "op_laggy": 0, 2023-09-13T03:29:46.067 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send": 4058, 2023-09-13T03:29:46.067 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send_bytes": 21, 2023-09-13T03:29:46.068 INFO:teuthology.orchestra.run.smithi170.stdout: "op_resend": 3, 2023-09-13T03:29:46.068 INFO:teuthology.orchestra.run.smithi170.stdout: "op_reply": 4058, 2023-09-13T03:29:46.068 INFO:teuthology.orchestra.run.smithi170.stdout: "op_latency": { 2023-09-13T03:29:46.069 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 4058, 2023-09-13T03:29:46.069 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 4.560891574, 2023-09-13T03:29:46.069 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001123925 2023-09-13T03:29:46.069 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:46.070 INFO:teuthology.orchestra.run.smithi170.stdout: "op_inflight": 0, 2023-09-13T03:29:46.070 INFO:teuthology.orchestra.run.smithi170.stdout: "oplen_avg": { 2023-09-13T03:29:46.070 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 4058, 2023-09-13T03:29:46.071 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 4059 2023-09-13T03:29:46.071 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:46.071 INFO:teuthology.orchestra.run.smithi170.stdout: "op": 4058, 2023-09-13T03:29:46.071 INFO:teuthology.orchestra.run.smithi170.stdout: "op_r": 4058, 2023-09-13T03:29:46.072 INFO:teuthology.orchestra.run.smithi170.stdout: "op_w": 0, 2023-09-13T03:29:46.072 INFO:teuthology.orchestra.run.smithi170.stdout: "op_rmw": 0, 2023-09-13T03:29:46.072 INFO:teuthology.orchestra.run.smithi170.stdout: "op_pg": 0, 2023-09-13T03:29:46.073 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_stat": 2332, 2023-09-13T03:29:46.073 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_create": 0, 2023-09-13T03:29:46.073 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_read": 1724, 2023-09-13T03:29:46.074 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_write": 0, 2023-09-13T03:29:46.074 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writefull": 0, 2023-09-13T03:29:46.074 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writesame": 0, 2023-09-13T03:29:46.074 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_append": 0, 2023-09-13T03:29:46.075 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_zero": 0, 2023-09-13T03:29:46.075 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_truncate": 0, 2023-09-13T03:29:46.075 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_delete": 0, 2023-09-13T03:29:46.076 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_mapext": 0, 2023-09-13T03:29:46.076 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_sparse_read": 0, 2023-09-13T03:29:46.076 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_clonerange": 0, 2023-09-13T03:29:46.077 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_getxattr": 1, 2023-09-13T03:29:46.077 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_setxattr": 0, 2023-09-13T03:29:46.077 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_cmpxattr": 0, 2023-09-13T03:29:46.077 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_rmxattr": 0, 2023-09-13T03:29:46.078 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_resetxattrs": 0, 2023-09-13T03:29:46.078 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_call": 0, 2023-09-13T03:29:46.078 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_watch": 0, 2023-09-13T03:29:46.079 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_notify": 0, 2023-09-13T03:29:46.079 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_src_cmpxattr": 0, 2023-09-13T03:29:46.079 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls": 0, 2023-09-13T03:29:46.080 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls_filter": 0, 2023-09-13T03:29:46.080 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_other": 0, 2023-09-13T03:29:46.080 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_active": 0, 2023-09-13T03:29:46.080 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_send": 0, 2023-09-13T03:29:46.081 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_resend": 0, 2023-09-13T03:29:46.081 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_ping": 0, 2023-09-13T03:29:46.081 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_active": 0, 2023-09-13T03:29:46.082 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_send": 0, 2023-09-13T03:29:46.082 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_resend": 0, 2023-09-13T03:29:46.082 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_active": 0, 2023-09-13T03:29:46.082 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_send": 0, 2023-09-13T03:29:46.083 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_resend": 0, 2023-09-13T03:29:46.083 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_active": 0, 2023-09-13T03:29:46.083 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_send": 0, 2023-09-13T03:29:46.084 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_resend": 0, 2023-09-13T03:29:46.084 INFO:teuthology.orchestra.run.smithi170.stdout: "command_active": 0, 2023-09-13T03:29:46.084 INFO:teuthology.orchestra.run.smithi170.stdout: "command_send": 0, 2023-09-13T03:29:46.085 INFO:teuthology.orchestra.run.smithi170.stdout: "command_resend": 0, 2023-09-13T03:29:46.085 INFO:teuthology.orchestra.run.smithi170.stdout: "map_epoch": 106, 2023-09-13T03:29:46.085 INFO:teuthology.orchestra.run.smithi170.stdout: "map_full": 0, 2023-09-13T03:29:46.085 INFO:teuthology.orchestra.run.smithi170.stdout: "map_inc": 23, 2023-09-13T03:29:46.086 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_sessions": 11, 2023-09-13T03:29:46.086 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_open": 12, 2023-09-13T03:29:46.086 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_close": 1, 2023-09-13T03:29:46.087 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_laggy": 0, 2023-09-13T03:29:46.087 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_wr": 0, 2023-09-13T03:29:46.087 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_rd": 2, 2023-09-13T03:29:46.088 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_del": 0 2023-09-13T03:29:46.088 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:46.088 INFO:teuthology.orchestra.run.smithi170.stdout: "oft": { 2023-09-13T03:29:46.088 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_objs": 0, 2023-09-13T03:29:46.089 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_kv_pairs": 0, 2023-09-13T03:29:46.089 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_updates": 0, 2023-09-13T03:29:46.089 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_removes": 0 2023-09-13T03:29:46.090 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:46.090 INFO:teuthology.orchestra.run.smithi170.stdout: "osd-slow-ops": { 2023-09-13T03:29:46.090 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_ops_count": 0 2023-09-13T03:29:46.091 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:46.092 INFO:teuthology.orchestra.run.smithi170.stdout: "purge_queue": { 2023-09-13T03:29:46.092 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops": 0, 2023-09-13T03:29:46.093 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops_high_water": 0, 2023-09-13T03:29:46.093 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing": 0, 2023-09-13T03:29:46.093 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_high_water": 0, 2023-09-13T03:29:46.094 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed": 0, 2023-09-13T03:29:46.094 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_item_in_journal": 0 2023-09-13T03:29:46.094 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:46.095 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-13T03:29:46.095 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2023-09-13T03:29:46.095 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2023-09-13T03:29:46.095 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2023-09-13T03:29:46.096 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 4463, 2023-09-13T03:29:46.096 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 255491487, 2023-09-13T03:29:46.096 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:46.097 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 4463, 2023-09-13T03:29:46.097 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2023-09-13T03:29:46.097 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2023-09-13T03:29:46.097 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 4463, 2023-09-13T03:29:46.098 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 255491487, 2023-09-13T03:29:46.098 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2023-09-13T03:29:46.098 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:46.099 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:46.099 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:46.099 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-09-13T03:29:46.100 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:46.100 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_bytes": { 2023-09-13T03:29:46.100 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2023-09-13T03:29:46.100 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2023-09-13T03:29:46.101 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2023-09-13T03:29:46.101 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2023-09-13T03:29:46.101 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2023-09-13T03:29:46.102 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:46.102 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2023-09-13T03:29:46.102 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 4058, 2023-09-13T03:29:46.103 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 252813141, 2023-09-13T03:29:46.103 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 4058, 2023-09-13T03:29:46.103 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 252813141, 2023-09-13T03:29:46.103 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2023-09-13T03:29:46.104 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:46.104 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:46.104 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:46.105 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-09-13T03:29:46.105 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:46.105 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_ops": { 2023-09-13T03:29:46.106 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2023-09-13T03:29:46.106 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 1024, 2023-09-13T03:29:46.106 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2023-09-13T03:29:46.107 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2023-09-13T03:29:46.107 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2023-09-13T03:29:46.107 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:46.107 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2023-09-13T03:29:46.108 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 4058, 2023-09-13T03:29:46.108 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 4058, 2023-09-13T03:29:46.108 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 4058, 2023-09-13T03:29:46.109 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 4058, 2023-09-13T03:29:46.109 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2023-09-13T03:29:46.109 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:46.110 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:46.110 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:46.110 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-09-13T03:29:46.110 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:46.111 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle": { 2023-09-13T03:29:46.111 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2023-09-13T03:29:46.111 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2023-09-13T03:29:46.112 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2023-09-13T03:29:46.112 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2023-09-13T03:29:46.112 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2023-09-13T03:29:46.112 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:46.113 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2023-09-13T03:29:46.113 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2023-09-13T03:29:46.113 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2023-09-13T03:29:46.114 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 0, 2023-09-13T03:29:46.114 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 0, 2023-09-13T03:29:46.114 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2023-09-13T03:29:46.115 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:46.115 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:46.115 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:46.115 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-09-13T03:29:46.116 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-09-13T03:29:46.116 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle-0x55efbf650960": { 2023-09-13T03:29:46.116 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2023-09-13T03:29:46.117 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2023-09-13T03:29:46.117 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2023-09-13T03:29:46.117 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2023-09-13T03:29:46.117 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2023-09-13T03:29:46.118 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2023-09-13T03:29:46.118 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2023-09-13T03:29:46.118 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2023-09-13T03:29:46.119 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2023-09-13T03:29:46.119 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 0, 2023-09-13T03:29:46.119 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 0, 2023-09-13T03:29:46.120 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2023-09-13T03:29:46.120 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-09-13T03:29:46.120 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-09-13T03:29:46.120 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-09-13T03:29:46.121 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-09-13T03:29:46.121 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-09-13T03:29:46.121 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-09-13T03:29:46.370 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2023-09-13T03:29:46.370 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2023-09-13T03:29:46.371 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2023-09-13T03:29:46.371 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2023-09-13T03:29:46.371 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2023-09-13T03:29:46.372 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-09-13T03:29:46.384 INFO:tasks.kclient:Unmounting kernel clients... 2023-09-13T03:29:46.385 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-13T03:29:46.385 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-13T03:29:46.435 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-13T03:29:46.436 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-13T03:29:46.483 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-13T03:29:46.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.732+0000 7f76e33ff700 1 -- 172.21.15.64:0/727133807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76dc100590 msgr2=0x7f76dc100a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:46.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.732+0000 7f76e33ff700 1 --2- 172.21.15.64:0/727133807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76dc100590 0x7f76dc100a10 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f76cc0099c0 tx=0x7f76cc009cd0 comp rx=0 tx=0).stop 2023-09-13T03:29:46.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.732+0000 7f76e33ff700 1 -- 172.21.15.64:0/727133807 shutdown_connections 2023-09-13T03:29:46.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.732+0000 7f76e33ff700 1 --2- 172.21.15.64:0/727133807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76dc106650 0x7f76dc106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:46.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.732+0000 7f76e33ff700 1 --2- 172.21.15.64:0/727133807 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76dc100f50 0x7f76dc10d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:46.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.732+0000 7f76e33ff700 1 --2- 172.21.15.64:0/727133807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76dc100590 0x7f76dc100a10 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:46.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.732+0000 7f76e33ff700 1 -- 172.21.15.64:0/727133807 >> 172.21.15.64:0/727133807 conn(0x7f76dc0fc020 msgr2=0x7f76dc0fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:46.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.732+0000 7f76e33ff700 1 -- 172.21.15.64:0/727133807 shutdown_connections 2023-09-13T03:29:46.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.732+0000 7f76e33ff700 1 -- 172.21.15.64:0/727133807 wait complete. 2023-09-13T03:29:46.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.733+0000 7f76e33ff700 1 Processor -- start 2023-09-13T03:29:46.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.733+0000 7f76e33ff700 1 -- start start 2023-09-13T03:29:46.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.733+0000 7f76e33ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76dc100590 0x7f76dc072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:46.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.733+0000 7f76e33ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76dc100f50 0x7f76dc072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:46.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.733+0000 7f76e33ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76dc106650 0x7f76dc1a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:46.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.733+0000 7f76e33ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc073670 con 0x7f76dc100590 2023-09-13T03:29:46.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.733+0000 7f76e33ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc1a4ca0 con 0x7f76dc106650 2023-09-13T03:29:46.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.733+0000 7f76e33ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc1a4de0 con 0x7f76dc100f50 2023-09-13T03:29:46.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.733+0000 7f76e119b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76dc100590 0x7f76dc072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:46.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.734+0000 7f76e119b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76dc100590 0x7f76dc072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44190/0 (socket says 172.21.15.64:44190) 2023-09-13T03:29:46.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.734+0000 7f76e119b700 1 -- 172.21.15.64:0/3702141178 learned_addr learned my addr 172.21.15.64:0/3702141178 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:29:46.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.734+0000 7f76e099a700 1 --2- 172.21.15.64:0/3702141178 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76dc100f50 0x7f76dc072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:46.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.734+0000 7f76e199c700 1 --2- 172.21.15.64:0/3702141178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76dc106650 0x7f76dc1a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:46.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.734+0000 7f76e119b700 1 -- 172.21.15.64:0/3702141178 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76dc100f50 msgr2=0x7f76dc072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:46.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.734+0000 7f76e119b700 1 --2- 172.21.15.64:0/3702141178 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76dc100f50 0x7f76dc072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:46.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.734+0000 7f76e119b700 1 -- 172.21.15.64:0/3702141178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76dc106650 msgr2=0x7f76dc1a4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:46.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.734+0000 7f76e119b700 1 --2- 172.21.15.64:0/3702141178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76dc106650 0x7f76dc1a4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:46.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.734+0000 7f76e119b700 1 -- 172.21.15.64:0/3702141178 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d8009520 con 0x7f76dc100590 2023-09-13T03:29:46.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.734+0000 7f76e199c700 1 --2- 172.21.15.64:0/3702141178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76dc106650 0x7f76dc1a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:29:46.741 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.734+0000 7f76e119b700 1 --2- 172.21.15.64:0/3702141178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76dc100590 0x7f76dc072900 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f76d800eb30 tx=0x7f76d800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:46.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.734+0000 7f76d27fc700 1 -- 172.21.15.64:0/3702141178 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d8009880 con 0x7f76dc100590 2023-09-13T03:29:46.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.734+0000 7f76d27fc700 1 -- 172.21.15.64:0/3702141178 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76d80044e0 con 0x7f76dc100590 2023-09-13T03:29:46.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.734+0000 7f76e33ff700 1 -- 172.21.15.64:0/3702141178 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76cc0095f0 con 0x7f76dc100590 2023-09-13T03:29:46.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.735+0000 7f76d27fc700 1 -- 172.21.15.64:0/3702141178 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d8003e90 con 0x7f76dc100590 2023-09-13T03:29:46.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.735+0000 7f76e33ff700 1 -- 172.21.15.64:0/3702141178 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76dc1a52d0 con 0x7f76dc100590 2023-09-13T03:29:46.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.736+0000 7f76e33ff700 1 -- 172.21.15.64:0/3702141178 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76dc0673f0 con 0x7f76dc100590 2023-09-13T03:29:46.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.737+0000 7f76d27fc700 1 -- 172.21.15.64:0/3702141178 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f76d8004000 con 0x7f76dc100590 2023-09-13T03:29:46.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.739+0000 7f76d27fc700 1 --2- 172.21.15.64:0/3702141178 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f76c80a07f0 0x7f76c80a2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:46.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.739+0000 7f76d27fc700 1 -- 172.21.15.64:0/3702141178 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f76d8014070 con 0x7f76dc100590 2023-09-13T03:29:46.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.739+0000 7f76e099a700 1 --2- 172.21.15.64:0/3702141178 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f76c80a07f0 0x7f76c80a2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:46.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.740+0000 7f76e099a700 1 --2- 172.21.15.64:0/3702141178 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f76c80a07f0 0x7f76c80a2cb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f76cc009990 tx=0x7f76cc005fb0 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:46.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.743+0000 7f76d27fc700 1 -- 172.21.15.64:0/3702141178 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f76d8095b50 con 0x7f76dc100590 2023-09-13T03:29:46.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.955+0000 7f76e33ff700 1 -- 172.21.15.64:0/3702141178 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76dc1106f0 con 0x7f76dc100590 2023-09-13T03:29:46.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.957+0000 7f76d27fc700 1 -- 172.21.15.64:0/3702141178 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+22942 (secure 0 0 0) 0x7f76d8099170 con 0x7f76dc100590 2023-09-13T03:29:46.957 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:29:46.957 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":106,"fsid":"f8dbced4-51e1-11ee-9ab7-7b867c8bd7da","created":"2023-09-13T03:03:24.167202+0000","modified":"2023-09-13T03:29:14.506662+0000","last_up_change":"2023-09-13T03:10:22.503529+0000","last_in_change":"2023-09-13T03:10:05.678720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T03:06:58.987479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-13T03:11:08.589202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-13T03:11:09.924659+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3997c114-241b-48c4-8feb-653c99209470","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6803","nonce":3940892135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6805","nonce":3940892135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6809","nonce":3940892135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6807","nonce":3940892135}]},"public_addr":"172.21.15.64:6803/3940892135","cluster_addr":"172.21.15.64:6805/3940892135","heartbeat_back_addr":"172.21.15.64:6809/3940892135","heartbeat_front_addr":"172.21.15.64:6807/3940892135","state":["exists","up"]},{"osd":1,"uuid":"94cfae85-322e-4ed2-a6e6-e40349bb14bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6801","nonce":4188149333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6803","nonce":4188149333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6807","nonce":4188149333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6805","nonce":4188149333}]},"public_addr":"172.21.15.142:6801/4188149333","cluster_addr":"172.21.15.142:6803/4188149333","heartbeat_back_addr":"172.21.15.142:6807/4188149333","heartbeat_front_addr":"172.21.15.142:6805/4188149333","state":["exists","up"]},{"osd":2,"uuid":"b3aae17e-96db-4008-8c27-f733c46b30ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6801","nonce":2406669338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6803","nonce":2406669338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6807","nonce":2406669338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6805","nonce":2406669338}]},"public_addr":"172.21.15.170:6801/2406669338","cluster_addr":"172.21.15.170:6803/2406669338","heartbeat_back_addr":"172.21.15.170:6807/2406669338","heartbeat_front_addr":"172.21.15.170:6805/2406669338","state":["exists","up"]},{"osd":3,"uuid":"7fa8d8cb-2ab7-48a7-8b49-5e2903d6779f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6811","nonce":3558318707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6813","nonce":3558318707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6817","nonce":3558318707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6815","nonce":3558318707}]},"public_addr":"172.21.15.64:6811/3558318707","cluster_addr":"172.21.15.64:6813/3558318707","heartbeat_back_addr":"172.21.15.64:6817/3558318707","heartbeat_front_addr":"172.21.15.64:6815/3558318707","state":["exists","up"]},{"osd":4,"uuid":"8c7a9c9f-f919-48d3-9db2-214a25bb5603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6809","nonce":1811142576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6811","nonce":1811142576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6815","nonce":1811142576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6813","nonce":1811142576}]},"public_addr":"172.21.15.142:6809/1811142576","cluster_addr":"172.21.15.142:6811/1811142576","heartbeat_back_addr":"172.21.15.142:6815/1811142576","heartbeat_front_addr":"172.21.15.142:6813/1811142576","state":["exists","up"]},{"osd":5,"uuid":"0c6101c6-2065-4022-a751-d3f216d7f478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6809","nonce":2903841453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6811","nonce":2903841453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6815","nonce":2903841453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6813","nonce":2903841453}]},"public_addr":"172.21.15.170:6809/2903841453","cluster_addr":"172.21.15.170:6811/2903841453","heartbeat_back_addr":"172.21.15.170:6815/2903841453","heartbeat_front_addr":"172.21.15.170:6813/2903841453","state":["exists","up"]},{"osd":6,"uuid":"5cf3e52e-d155-41b1-9194-db047d26a953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6819","nonce":1436584490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6821","nonce":1436584490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6825","nonce":1436584490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6823","nonce":1436584490}]},"public_addr":"172.21.15.64:6819/1436584490","cluster_addr":"172.21.15.64:6821/1436584490","heartbeat_back_addr":"172.21.15.64:6825/1436584490","heartbeat_front_addr":"172.21.15.64:6823/1436584490","state":["exists","up"]},{"osd":7,"uuid":"e97323e3-badd-41a7-af9e-e78a81a64046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6817","nonce":3726793534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6819","nonce":3726793534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6823","nonce":3726793534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6821","nonce":3726793534}]},"public_addr":"172.21.15.142:6817/3726793534","cluster_addr":"172.21.15.142:6819/3726793534","heartbeat_back_addr":"172.21.15.142:6823/3726793534","heartbeat_front_addr":"172.21.15.142:6821/3726793534","state":["exists","up"]},{"osd":8,"uuid":"83d2835c-ed26-493d-9222-c2894ec0e071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6817","nonce":4272513873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6819","nonce":4272513873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6823","nonce":4272513873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6821","nonce":4272513873}]},"public_addr":"172.21.15.170:6817/4272513873","cluster_addr":"172.21.15.170:6819/4272513873","heartbeat_back_addr":"172.21.15.170:6823/4272513873","heartbeat_front_addr":"172.21.15.170:6821/4272513873","state":["exists","up"]},{"osd":9,"uuid":"cb936bdd-1bec-4236-aa2e-a24adcac11e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6827","nonce":2641175211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6829","nonce":2641175211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6833","nonce":2641175211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6831","nonce":2641175211}]},"public_addr":"172.21.15.64:6827/2641175211","cluster_addr":"172.21.15.64:6829/2641175211","heartbeat_back_addr":"172.21.15.64:6833/2641175211","heartbeat_front_addr":"172.21.15.64:6831/2641175211","state":["exists","up"]},{"osd":10,"uuid":"85512a1c-6171-43d4-9c15-c588a8b71be0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6825","nonce":936499217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6827","nonce":936499217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6831","nonce":936499217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6829","nonce":936499217}]},"public_addr":"172.21.15.142:6825/936499217","cluster_addr":"172.21.15.142:6827/936499217","heartbeat_back_addr":"172.21.15.142:6831/936499217","heartbeat_front_addr":"172.21.15.142:6829/936499217","state":["exists","up"]},{"osd":11,"uuid":"9b8417dc-5d6c-4917-94a9-b032c5bc33d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6825","nonce":3605017060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6827","nonce":3605017060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6831","nonce":3605017060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6829","nonce":3605017060}]},"public_addr":"172.21.15.170:6825/3605017060","cluster_addr":"172.21.15.170:6827/3605017060","heartbeat_back_addr":"172.21.15.170:6831/3605017060","heartbeat_front_addr":"172.21.15.170:6829/3605017060","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:14.229044+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:36.147777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:56.655298+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:07:23.107741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:07:44.663770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:05.577432+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:31.168031+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:52.791055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:09:13.658546+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:09:39.150360+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:10:00.601735+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:10:21.407555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6801/873679308":"2023-09-14T03:13:09.492864+0000","172.21.15.64:0/1170163026":"2023-09-14T03:13:09.492864+0000","172.21.15.64:0/3771142265":"2023-09-14T03:13:09.492864+0000","172.21.15.64:0/4083942699":"2023-09-14T03:13:09.492864+0000","172.21.15.64:6800/2955227347":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/1373223118":"2023-09-14T03:04:22.355817+0000","172.21.15.64:6800/873679308":"2023-09-14T03:13:09.492864+0000","172.21.15.64:0/791645251":"2023-09-14T03:13:09.492864+0000","172.21.15.64:0/271194613":"2023-09-14T03:04:22.355817+0000","172.21.15.64:6801/3998091347":"2023-09-14T03:03:56.442541+0000","172.21.15.64:6800/3998091347":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/920818572":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/735789005":"2023-09-14T03:03:56.442541+0000","172.21.15.64:6801/2955227347":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/1975923178":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/1500151178":"2023-09-14T03:04:22.355817+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T03:29:46.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.959+0000 7f76e33ff700 1 -- 172.21.15.64:0/3702141178 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f76c80a07f0 msgr2=0x7f76c80a2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:46.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.959+0000 7f76e33ff700 1 --2- 172.21.15.64:0/3702141178 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f76c80a07f0 0x7f76c80a2cb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f76cc009990 tx=0x7f76cc005fb0 comp rx=0 tx=0).stop 2023-09-13T03:29:46.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.959+0000 7f76e33ff700 1 -- 172.21.15.64:0/3702141178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76dc100590 msgr2=0x7f76dc072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:46.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.959+0000 7f76e33ff700 1 --2- 172.21.15.64:0/3702141178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76dc100590 0x7f76dc072900 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f76d800eb30 tx=0x7f76d800ee40 comp rx=0 tx=0).stop 2023-09-13T03:29:46.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.959+0000 7f76e33ff700 1 -- 172.21.15.64:0/3702141178 shutdown_connections 2023-09-13T03:29:46.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.959+0000 7f76e33ff700 1 --2- 172.21.15.64:0/3702141178 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f76c80a07f0 0x7f76c80a2cb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:46.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.959+0000 7f76e33ff700 1 --2- 172.21.15.64:0/3702141178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76dc106650 0x7f76dc1a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:46.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.959+0000 7f76e33ff700 1 --2- 172.21.15.64:0/3702141178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76dc100590 0x7f76dc072900 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:46.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.959+0000 7f76e33ff700 1 --2- 172.21.15.64:0/3702141178 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76dc100f50 0x7f76dc072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:46.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.959+0000 7f76e33ff700 1 -- 172.21.15.64:0/3702141178 >> 172.21.15.64:0/3702141178 conn(0x7f76dc0fc020 msgr2=0x7f76dc0fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:46.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.959+0000 7f76e33ff700 1 -- 172.21.15.64:0/3702141178 shutdown_connections 2023-09-13T03:29:46.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:46.959+0000 7f76e33ff700 1 -- 172.21.15.64:0/3702141178 wait complete. 2023-09-13T03:29:46.978 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-13T03:06:58.987479+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-13T03:11:08.589202+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-13T03:11:09.924659+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 105, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-13T03:29:46.979 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-13T03:29:47.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:46 smithi170 ceph-mon[134328]: pgmap v521: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 3.3 MiB/s wr, 139 op/s 2023-09-13T03:29:47.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:46 smithi064 ceph-mon[97675]: pgmap v521: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 3.3 MiB/s wr, 139 op/s 2023-09-13T03:29:47.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.188+0000 7f7eb57ff700 1 -- 172.21.15.64:0/1836243264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eb0100f30 msgr2=0x7f7eb010d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:47.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.188+0000 7f7eb57ff700 1 --2- 172.21.15.64:0/1836243264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eb0100f30 0x7f7eb010d3f0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f7ea800b0d0 tx=0x7f7ea800b3e0 comp rx=0 tx=0).stop 2023-09-13T03:29:47.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.189+0000 7f7eb57ff700 1 -- 172.21.15.64:0/1836243264 shutdown_connections 2023-09-13T03:29:47.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.189+0000 7f7eb57ff700 1 --2- 172.21.15.64:0/1836243264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eb0100f30 0x7f7eb010d3f0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:47.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.189+0000 7f7eb57ff700 1 --2- 172.21.15.64:0/1836243264 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7eb0106630 0x7f7eb0106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:47.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.189+0000 7f7eb57ff700 1 --2- 172.21.15.64:0/1836243264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7eb0100570 0x7f7eb01009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:47.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.189+0000 7f7eb57ff700 1 -- 172.21.15.64:0/1836243264 >> 172.21.15.64:0/1836243264 conn(0x7f7eb00fc080 msgr2=0x7f7eb00fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:47.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.189+0000 7f7eb57ff700 1 -- 172.21.15.64:0/1836243264 shutdown_connections 2023-09-13T03:29:47.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.189+0000 7f7eb57ff700 1 -- 172.21.15.64:0/1836243264 wait complete. 2023-09-13T03:29:47.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.190+0000 7f7eb57ff700 1 Processor -- start 2023-09-13T03:29:47.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.190+0000 7f7eb57ff700 1 -- start start 2023-09-13T03:29:47.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.190+0000 7f7eb57ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7eb0100570 0x7f7eb01966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:47.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.190+0000 7f7eb57ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eb0100f30 0x7f7eb0196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:47.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.190+0000 7f7eb57ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7eb0106630 0x7f7eb0199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:47.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.190+0000 7f7eb57ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb019a5d0 con 0x7f7eb0100f30 2023-09-13T03:29:47.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.190+0000 7f7eb57ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb019a740 con 0x7f7eb0100570 2023-09-13T03:29:47.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.190+0000 7f7eb57ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb019a900 con 0x7f7eb0106630 2023-09-13T03:29:47.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.190+0000 7f7eae7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eb0100f30 0x7f7eb0196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:47.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.190+0000 7f7eae7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eb0100f30 0x7f7eb0196c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44210/0 (socket says 172.21.15.64:44210) 2023-09-13T03:29:47.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.190+0000 7f7eae7fc700 1 -- 172.21.15.64:0/4135293492 learned_addr learned my addr 172.21.15.64:0/4135293492 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:29:47.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.190+0000 7f7eaeffd700 1 --2- 172.21.15.64:0/4135293492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7eb0100570 0x7f7eb01966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:47.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.191+0000 7f7eaf7fe700 1 --2- 172.21.15.64:0/4135293492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7eb0106630 0x7f7eb0199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:47.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.191+0000 7f7eae7fc700 1 -- 172.21.15.64:0/4135293492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7eb0106630 msgr2=0x7f7eb0199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:47.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.191+0000 7f7eae7fc700 1 --2- 172.21.15.64:0/4135293492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7eb0106630 0x7f7eb0199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:47.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.191+0000 7f7eae7fc700 1 -- 172.21.15.64:0/4135293492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7eb0100570 msgr2=0x7f7eb01966e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:47.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.191+0000 7f7eae7fc700 1 --2- 172.21.15.64:0/4135293492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7eb0100570 0x7f7eb01966e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:47.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.191+0000 7f7eae7fc700 1 -- 172.21.15.64:0/4135293492 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea8009d00 con 0x7f7eb0100f30 2023-09-13T03:29:47.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.191+0000 7f7eae7fc700 1 --2- 172.21.15.64:0/4135293492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eb0100f30 0x7f7eb0196c20 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f7ea400d980 tx=0x7f7ea400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:47.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.191+0000 7f7e97fff700 1 -- 172.21.15.64:0/4135293492 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea4004220 con 0x7f7eb0100f30 2023-09-13T03:29:47.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.191+0000 7f7e97fff700 1 -- 172.21.15.64:0/4135293492 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ea4007640 con 0x7f7eb0100f30 2023-09-13T03:29:47.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.191+0000 7f7eb57ff700 1 -- 172.21.15.64:0/4135293492 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eb019abe0 con 0x7f7eb0100f30 2023-09-13T03:29:47.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.191+0000 7f7e97fff700 1 -- 172.21.15.64:0/4135293492 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea4005690 con 0x7f7eb0100f30 2023-09-13T03:29:47.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.192+0000 7f7eb57ff700 1 -- 172.21.15.64:0/4135293492 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eb01a3540 con 0x7f7eb0100f30 2023-09-13T03:29:47.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.192+0000 7f7eb57ff700 1 -- 172.21.15.64:0/4135293492 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eb0067380 con 0x7f7eb0100f30 2023-09-13T03:29:47.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.194+0000 7f7e97fff700 1 -- 172.21.15.64:0/4135293492 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f7ea4004750 con 0x7f7eb0100f30 2023-09-13T03:29:47.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.196+0000 7f7e97fff700 1 --2- 172.21.15.64:0/4135293492 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f7e980a07b0 0x7f7e980a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:47.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.196+0000 7f7e97fff700 1 -- 172.21.15.64:0/4135293492 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f7ea40cbe10 con 0x7f7eb0100f30 2023-09-13T03:29:47.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.196+0000 7f7eaeffd700 1 --2- 172.21.15.64:0/4135293492 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f7e980a07b0 0x7f7e980a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:47.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.197+0000 7f7eaeffd700 1 --2- 172.21.15.64:0/4135293492 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f7e980a07b0 0x7f7e980a2c70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0005950 tx=0x7f7ea000b410 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:47.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.200+0000 7f7e97fff700 1 -- 172.21.15.64:0/4135293492 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f7ea40cf030 con 0x7f7eb0100f30 2023-09-13T03:29:47.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:46 smithi142 ceph-mon[134082]: pgmap v521: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 3.3 MiB/s wr, 139 op/s 2023-09-13T03:29:47.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.414+0000 7f7eb57ff700 1 -- 172.21.15.64:0/4135293492 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7eb01105d0 con 0x7f7eb0100f30 2023-09-13T03:29:47.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.414+0000 7f7e97fff700 1 -- 172.21.15.64:0/4135293492 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7ea40160a0 con 0x7f7eb0100f30 2023-09-13T03:29:47.415 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-09-13T03:29:47.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.417+0000 7f7eb57ff700 1 -- 172.21.15.64:0/4135293492 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f7e980a07b0 msgr2=0x7f7e980a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:47.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.417+0000 7f7eb57ff700 1 --2- 172.21.15.64:0/4135293492 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f7e980a07b0 0x7f7e980a2c70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0005950 tx=0x7f7ea000b410 comp rx=0 tx=0).stop 2023-09-13T03:29:47.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.417+0000 7f7eb57ff700 1 -- 172.21.15.64:0/4135293492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eb0100f30 msgr2=0x7f7eb0196c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:47.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.417+0000 7f7eb57ff700 1 --2- 172.21.15.64:0/4135293492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eb0100f30 0x7f7eb0196c20 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f7ea400d980 tx=0x7f7ea400dc90 comp rx=0 tx=0).stop 2023-09-13T03:29:47.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.417+0000 7f7eb57ff700 1 -- 172.21.15.64:0/4135293492 shutdown_connections 2023-09-13T03:29:47.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.417+0000 7f7eb57ff700 1 --2- 172.21.15.64:0/4135293492 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f7e980a07b0 0x7f7e980a2c70 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:47.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.417+0000 7f7eb57ff700 1 --2- 172.21.15.64:0/4135293492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7eb0100570 0x7f7eb01966e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:47.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.417+0000 7f7eb57ff700 1 --2- 172.21.15.64:0/4135293492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7eb0106630 0x7f7eb0199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:47.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.417+0000 7f7eb57ff700 1 --2- 172.21.15.64:0/4135293492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eb0100f30 0x7f7eb0196c20 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:47.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.417+0000 7f7eb57ff700 1 -- 172.21.15.64:0/4135293492 >> 172.21.15.64:0/4135293492 conn(0x7f7eb00fc080 msgr2=0x7f7eb00fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:47.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.417+0000 7f7eb57ff700 1 -- 172.21.15.64:0/4135293492 shutdown_connections 2023-09-13T03:29:47.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.417+0000 7f7eb57ff700 1 -- 172.21.15.64:0/4135293492 wait complete. 2023-09-13T03:29:47.436 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-13T03:29:47.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.643+0000 7f5795dff700 1 -- 172.21.15.64:0/2797929085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5790106630 msgr2=0x7f5790106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:47.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.643+0000 7f5795dff700 1 --2- 172.21.15.64:0/2797929085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5790106630 0x7f5790106a10 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f57800099c0 tx=0x7f5780009cd0 comp rx=0 tx=0).stop 2023-09-13T03:29:47.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.643+0000 7f5795dff700 1 -- 172.21.15.64:0/2797929085 shutdown_connections 2023-09-13T03:29:47.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.643+0000 7f5795dff700 1 --2- 172.21.15.64:0/2797929085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5790100f30 0x7f579010d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:47.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.643+0000 7f5795dff700 1 --2- 172.21.15.64:0/2797929085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5790106630 0x7f5790106a10 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:47.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.643+0000 7f5795dff700 1 --2- 172.21.15.64:0/2797929085 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5790100570 0x7f57901009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:47.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.643+0000 7f5795dff700 1 -- 172.21.15.64:0/2797929085 >> 172.21.15.64:0/2797929085 conn(0x7f57900fc080 msgr2=0x7f57900fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:47.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.643+0000 7f5795dff700 1 -- 172.21.15.64:0/2797929085 shutdown_connections 2023-09-13T03:29:47.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.643+0000 7f5795dff700 1 -- 172.21.15.64:0/2797929085 wait complete. 2023-09-13T03:29:47.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.644+0000 7f5795dff700 1 Processor -- start 2023-09-13T03:29:47.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.644+0000 7f5795dff700 1 -- start start 2023-09-13T03:29:47.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.645+0000 7f5795dff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5790100570 0x7f579019e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:47.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.645+0000 7f5795dff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5790100f30 0x7f579019eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:47.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.645+0000 7f5795dff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5790106630 0x7f5790198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:47.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.645+0000 7f5795dff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5790198cd0 con 0x7f5790106630 2023-09-13T03:29:47.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.645+0000 7f5795dff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5790198e40 con 0x7f5790100f30 2023-09-13T03:29:47.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.645+0000 7f5795dff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5790199000 con 0x7f5790100570 2023-09-13T03:29:47.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.645+0000 7f578ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5790106630 0x7f5790198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:47.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.645+0000 7f578ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5790106630 0x7f5790198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44224/0 (socket says 172.21.15.64:44224) 2023-09-13T03:29:47.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.645+0000 7f578ffff700 1 -- 172.21.15.64:0/368948433 learned_addr learned my addr 172.21.15.64:0/368948433 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:29:47.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.645+0000 7f578f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5790100570 0x7f579019e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:47.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.645+0000 7f578effd700 1 --2- 172.21.15.64:0/368948433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5790100f30 0x7f579019eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:47.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.645+0000 7f578ffff700 1 -- 172.21.15.64:0/368948433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5790100570 msgr2=0x7f579019e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:47.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.645+0000 7f578ffff700 1 --2- 172.21.15.64:0/368948433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5790100570 0x7f579019e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:47.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.645+0000 7f578ffff700 1 -- 172.21.15.64:0/368948433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5790100f30 msgr2=0x7f579019eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:47.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.645+0000 7f578ffff700 1 --2- 172.21.15.64:0/368948433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5790100f30 0x7f579019eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:47.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.645+0000 7f578ffff700 1 -- 172.21.15.64:0/368948433 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57800095f0 con 0x7f5790106630 2023-09-13T03:29:47.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.646+0000 7f578ffff700 1 --2- 172.21.15.64:0/368948433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5790106630 0x7f5790198660 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f578800ef30 tx=0x7f578800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:47.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.646+0000 7f578cff9700 1 -- 172.21.15.64:0/368948433 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5788018070 con 0x7f5790106630 2023-09-13T03:29:47.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.646+0000 7f578cff9700 1 -- 172.21.15.64:0/368948433 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f578800f040 con 0x7f5790106630 2023-09-13T03:29:47.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.646+0000 7f5795dff700 1 -- 172.21.15.64:0/368948433 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5790199280 con 0x7f5790106630 2023-09-13T03:29:47.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.646+0000 7f578cff9700 1 -- 172.21.15.64:0/368948433 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578800cd00 con 0x7f5790106630 2023-09-13T03:29:47.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.646+0000 7f5795dff700 1 -- 172.21.15.64:0/368948433 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57901a55e0 con 0x7f5790106630 2023-09-13T03:29:47.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.647+0000 7f5795dff700 1 -- 172.21.15.64:0/368948433 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5790067380 con 0x7f5790106630 2023-09-13T03:29:47.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.649+0000 7f578cff9700 1 -- 172.21.15.64:0/368948433 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f5788003a00 con 0x7f5790106630 2023-09-13T03:29:47.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.651+0000 7f578cff9700 1 --2- 172.21.15.64:0/368948433 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f57780a07b0 0x7f57780a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:47.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.652+0000 7f578cff9700 1 -- 172.21.15.64:0/368948433 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f57880cbb50 con 0x7f5790106630 2023-09-13T03:29:47.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.652+0000 7f578f7fe700 1 --2- 172.21.15.64:0/368948433 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f57780a07b0 0x7f57780a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:47.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.652+0000 7f578f7fe700 1 --2- 172.21.15.64:0/368948433 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f57780a07b0 0x7f57780a2c70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5780009990 tx=0x7f5780005fb0 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:47.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.656+0000 7f578cff9700 1 -- 172.21.15.64:0/368948433 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5788095390 con 0x7f5790106630 2023-09-13T03:29:47.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.868+0000 7f5795dff700 1 -- 172.21.15.64:0/368948433 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5790110620 con 0x7f5790106630 2023-09-13T03:29:47.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.869+0000 7f578cff9700 1 -- 172.21.15.64:0/368948433 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v106) v1 ==== 104+0+11 (secure 0 0 0) 0x7f57880989b0 con 0x7f5790106630 2023-09-13T03:29:47.869 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-09-13T03:29:47.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.871+0000 7f5795dff700 1 -- 172.21.15.64:0/368948433 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f57780a07b0 msgr2=0x7f57780a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:47.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.871+0000 7f5795dff700 1 --2- 172.21.15.64:0/368948433 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f57780a07b0 0x7f57780a2c70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5780009990 tx=0x7f5780005fb0 comp rx=0 tx=0).stop 2023-09-13T03:29:47.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.871+0000 7f5795dff700 1 -- 172.21.15.64:0/368948433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5790106630 msgr2=0x7f5790198660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:47.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.871+0000 7f5795dff700 1 --2- 172.21.15.64:0/368948433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5790106630 0x7f5790198660 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f578800ef30 tx=0x7f578800c3a0 comp rx=0 tx=0).stop 2023-09-13T03:29:47.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.871+0000 7f5795dff700 1 -- 172.21.15.64:0/368948433 shutdown_connections 2023-09-13T03:29:47.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.871+0000 7f5795dff700 1 --2- 172.21.15.64:0/368948433 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f57780a07b0 0x7f57780a2c70 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:47.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.871+0000 7f5795dff700 1 --2- 172.21.15.64:0/368948433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5790106630 0x7f5790198660 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:47.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.871+0000 7f5795dff700 1 --2- 172.21.15.64:0/368948433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5790100570 0x7f579019e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:47.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.871+0000 7f5795dff700 1 --2- 172.21.15.64:0/368948433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5790100f30 0x7f579019eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:47.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.871+0000 7f5795dff700 1 -- 172.21.15.64:0/368948433 >> 172.21.15.64:0/368948433 conn(0x7f57900fc080 msgr2=0x7f57900fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:47.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.871+0000 7f5795dff700 1 -- 172.21.15.64:0/368948433 shutdown_connections 2023-09-13T03:29:47.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:47.871+0000 7f5795dff700 1 -- 172.21.15.64:0/368948433 wait complete. 2023-09-13T03:29:47.890 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-13T03:29:48.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.098+0000 7f3faffff700 1 -- 172.21.15.64:0/1877644609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fa8100590 msgr2=0x7f3fa8100a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:48.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.098+0000 7f3faffff700 1 --2- 172.21.15.64:0/1877644609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fa8100590 0x7f3fa8100a10 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f3f980099c0 tx=0x7f3f98009cd0 comp rx=0 tx=0).stop 2023-09-13T03:29:48.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.099+0000 7f3faffff700 1 -- 172.21.15.64:0/1877644609 shutdown_connections 2023-09-13T03:29:48.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.099+0000 7f3faffff700 1 --2- 172.21.15.64:0/1877644609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fa8100f50 0x7f3fa810d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:48.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.099+0000 7f3faffff700 1 --2- 172.21.15.64:0/1877644609 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3fa8106650 0x7f3fa8106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:48.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.099+0000 7f3faffff700 1 --2- 172.21.15.64:0/1877644609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fa8100590 0x7f3fa8100a10 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:48.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.099+0000 7f3faffff700 1 -- 172.21.15.64:0/1877644609 >> 172.21.15.64:0/1877644609 conn(0x7f3fa80fc060 msgr2=0x7f3fa80fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:48.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.099+0000 7f3faffff700 1 -- 172.21.15.64:0/1877644609 shutdown_connections 2023-09-13T03:29:48.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.099+0000 7f3faffff700 1 -- 172.21.15.64:0/1877644609 wait complete. 2023-09-13T03:29:48.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.100+0000 7f3faffff700 1 Processor -- start 2023-09-13T03:29:48.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.100+0000 7f3faffff700 1 -- start start 2023-09-13T03:29:48.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.100+0000 7f3faffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fa8100590 0x7f3fa8072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:48.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.100+0000 7f3faffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fa8100f50 0x7f3fa8072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:48.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.100+0000 7f3faffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3fa8106650 0x7f3fa81a4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:48.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.100+0000 7f3faffff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa8073670 con 0x7f3fa8100f50 2023-09-13T03:29:48.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.100+0000 7f3faffff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa81a4d60 con 0x7f3fa8100590 2023-09-13T03:29:48.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.100+0000 7f3faffff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa81a4ea0 con 0x7f3fa8106650 2023-09-13T03:29:48.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.100+0000 7f3fad59a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fa8100f50 0x7f3fa8072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:48.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.100+0000 7f3fadd9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fa8100590 0x7f3fa8072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:48.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.100+0000 7f3fadd9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fa8100590 0x7f3fa8072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.64:43456/0 (socket says 172.21.15.64:43456) 2023-09-13T03:29:48.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.100+0000 7f3fadd9b700 1 -- 172.21.15.64:0/370951338 learned_addr learned my addr 172.21.15.64:0/370951338 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:29:48.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.101+0000 7f3fae59c700 1 --2- 172.21.15.64:0/370951338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3fa8106650 0x7f3fa81a4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:48.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.101+0000 7f3fad59a700 1 -- 172.21.15.64:0/370951338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3fa8106650 msgr2=0x7f3fa81a4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:48.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.101+0000 7f3fad59a700 1 --2- 172.21.15.64:0/370951338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3fa8106650 0x7f3fa81a4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:48.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.101+0000 7f3fad59a700 1 -- 172.21.15.64:0/370951338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fa8100590 msgr2=0x7f3fa8072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:48.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.101+0000 7f3fad59a700 1 --2- 172.21.15.64:0/370951338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fa8100590 0x7f3fa8072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:48.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.101+0000 7f3fad59a700 1 -- 172.21.15.64:0/370951338 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f980095f0 con 0x7f3fa8100f50 2023-09-13T03:29:48.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.101+0000 7f3fadd9b700 1 --2- 172.21.15.64:0/370951338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fa8100590 0x7f3fa8072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:29:48.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.101+0000 7f3fad59a700 1 --2- 172.21.15.64:0/370951338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fa8100f50 0x7f3fa8072e40 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f3f9800b580 tx=0x7f3f9800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:48.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.101+0000 7f3f9effd700 1 -- 172.21.15.64:0/370951338 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9801d070 con 0x7f3fa8100f50 2023-09-13T03:29:48.109 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.101+0000 7f3f9effd700 1 -- 172.21.15.64:0/370951338 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f98005020 con 0x7f3fa8100f50 2023-09-13T03:29:48.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.101+0000 7f3f9effd700 1 -- 172.21.15.64:0/370951338 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f98004a90 con 0x7f3fa8100f50 2023-09-13T03:29:48.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.101+0000 7f3faffff700 1 -- 172.21.15.64:0/370951338 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fa81a50a0 con 0x7f3fa8100f50 2023-09-13T03:29:48.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.102+0000 7f3faffff700 1 -- 172.21.15.64:0/370951338 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fa81a5460 con 0x7f3fa8100f50 2023-09-13T03:29:48.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.103+0000 7f3faffff700 1 -- 172.21.15.64:0/370951338 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fa8067380 con 0x7f3fa8100f50 2023-09-13T03:29:48.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.104+0000 7f3f9effd700 1 -- 172.21.15.64:0/370951338 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f3f98005400 con 0x7f3fa8100f50 2023-09-13T03:29:48.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.106+0000 7f3f9effd700 1 --2- 172.21.15.64:0/370951338 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f3f940a0830 0x7f3f940a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:48.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.106+0000 7f3f9effd700 1 -- 172.21.15.64:0/370951338 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f3f980cce90 con 0x7f3fa8100f50 2023-09-13T03:29:48.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.106+0000 7f3fadd9b700 1 --2- 172.21.15.64:0/370951338 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f3f940a0830 0x7f3f940a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:48.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.107+0000 7f3fadd9b700 1 --2- 172.21.15.64:0/370951338 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f3f940a0830 0x7f3f940a2cf0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4011550 tx=0x7f3fa4009260 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:48.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.108+0000 7f3f9effd700 1 -- 172.21.15.64:0/370951338 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f3f980966d0 con 0x7f3fa8100f50 2023-09-13T03:29:48.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:47 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/3702141178' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:29:48.160 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:47 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/4135293492' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T03:29:48.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:47 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/3702141178' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:29:48.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:47 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/4135293492' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T03:29:48.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:47 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/3702141178' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:29:48.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:47 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/4135293492' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-13T03:29:48.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.322+0000 7f3faffff700 1 -- 172.21.15.64:0/370951338 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3fa81a56b0 con 0x7f3fa8100f50 2023-09-13T03:29:48.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.323+0000 7f3f9effd700 1 -- 172.21.15.64:0/370951338 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v106) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3f98099cf0 con 0x7f3fa8100f50 2023-09-13T03:29:48.323 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-09-13T03:29:48.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.325+0000 7f3faffff700 1 -- 172.21.15.64:0/370951338 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f3f940a0830 msgr2=0x7f3f940a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:48.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.325+0000 7f3faffff700 1 --2- 172.21.15.64:0/370951338 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f3f940a0830 0x7f3f940a2cf0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4011550 tx=0x7f3fa4009260 comp rx=0 tx=0).stop 2023-09-13T03:29:48.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.325+0000 7f3faffff700 1 -- 172.21.15.64:0/370951338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fa8100f50 msgr2=0x7f3fa8072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:48.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.325+0000 7f3faffff700 1 --2- 172.21.15.64:0/370951338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fa8100f50 0x7f3fa8072e40 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f3f9800b580 tx=0x7f3f9800f690 comp rx=0 tx=0).stop 2023-09-13T03:29:48.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.325+0000 7f3faffff700 1 -- 172.21.15.64:0/370951338 shutdown_connections 2023-09-13T03:29:48.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.325+0000 7f3faffff700 1 --2- 172.21.15.64:0/370951338 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f3f940a0830 0x7f3f940a2cf0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:48.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.325+0000 7f3faffff700 1 --2- 172.21.15.64:0/370951338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fa8100590 0x7f3fa8072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:48.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.325+0000 7f3faffff700 1 --2- 172.21.15.64:0/370951338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3fa8106650 0x7f3fa81a4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:48.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.325+0000 7f3faffff700 1 --2- 172.21.15.64:0/370951338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fa8100f50 0x7f3fa8072e40 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:48.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.326+0000 7f3faffff700 1 -- 172.21.15.64:0/370951338 >> 172.21.15.64:0/370951338 conn(0x7f3fa80fc060 msgr2=0x7f3fa80fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:48.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.326+0000 7f3faffff700 1 -- 172.21.15.64:0/370951338 shutdown_connections 2023-09-13T03:29:48.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.326+0000 7f3faffff700 1 -- 172.21.15.64:0/370951338 wait complete. 2023-09-13T03:29:48.344 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-13T03:29:48.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.548+0000 7fb3bbdff700 1 -- 172.21.15.64:0/3182290798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3b4106630 msgr2=0x7fb3b4106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:48.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.548+0000 7fb3bbdff700 1 --2- 172.21.15.64:0/3182290798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3b4106630 0x7fb3b4106a10 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fb3b00099c0 tx=0x7fb3b0009cd0 comp rx=0 tx=0).stop 2023-09-13T03:29:48.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.549+0000 7fb3bbdff700 1 -- 172.21.15.64:0/3182290798 shutdown_connections 2023-09-13T03:29:48.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.549+0000 7fb3bbdff700 1 --2- 172.21.15.64:0/3182290798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3b4100f30 0x7fb3b410d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:48.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.549+0000 7fb3bbdff700 1 --2- 172.21.15.64:0/3182290798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3b4106630 0x7fb3b4106a10 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:48.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.549+0000 7fb3bbdff700 1 --2- 172.21.15.64:0/3182290798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3b4100570 0x7fb3b41009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:48.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.549+0000 7fb3bbdff700 1 -- 172.21.15.64:0/3182290798 >> 172.21.15.64:0/3182290798 conn(0x7fb3b40fc080 msgr2=0x7fb3b40fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:48.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.549+0000 7fb3bbdff700 1 -- 172.21.15.64:0/3182290798 shutdown_connections 2023-09-13T03:29:48.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.549+0000 7fb3bbdff700 1 -- 172.21.15.64:0/3182290798 wait complete. 2023-09-13T03:29:48.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.550+0000 7fb3bbdff700 1 Processor -- start 2023-09-13T03:29:48.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.550+0000 7fb3bbdff700 1 -- start start 2023-09-13T03:29:48.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.550+0000 7fb3bbdff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3b4100570 0x7fb3b419e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:48.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.550+0000 7fb3bbdff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3b4100f30 0x7fb3b419eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:48.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.550+0000 7fb3bbdff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3b4106630 0x7fb3b4198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:48.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.550+0000 7fb3bbdff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b4198c80 con 0x7fb3b4100f30 2023-09-13T03:29:48.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.550+0000 7fb3bbdff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b4198df0 con 0x7fb3b4100570 2023-09-13T03:29:48.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.550+0000 7fb3bbdff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b4198fb0 con 0x7fb3b4106630 2023-09-13T03:29:48.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.551+0000 7fb3b9b9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3b4100570 0x7fb3b419e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:48.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.551+0000 7fb3b939a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3b4100f30 0x7fb3b419eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:48.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.551+0000 7fb3b939a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3b4100f30 0x7fb3b419eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44262/0 (socket says 172.21.15.64:44262) 2023-09-13T03:29:48.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.551+0000 7fb3b939a700 1 -- 172.21.15.64:0/1924196932 learned_addr learned my addr 172.21.15.64:0/1924196932 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:29:48.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.551+0000 7fb3ba39c700 1 --2- 172.21.15.64:0/1924196932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3b4106630 0x7fb3b4198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:48.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.551+0000 7fb3b939a700 1 -- 172.21.15.64:0/1924196932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3b4106630 msgr2=0x7fb3b4198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:48.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.551+0000 7fb3b939a700 1 --2- 172.21.15.64:0/1924196932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3b4106630 0x7fb3b4198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:48.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.551+0000 7fb3b939a700 1 -- 172.21.15.64:0/1924196932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3b4100570 msgr2=0x7fb3b419e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:48.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.551+0000 7fb3b939a700 1 --2- 172.21.15.64:0/1924196932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3b4100570 0x7fb3b419e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:48.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.551+0000 7fb3b939a700 1 -- 172.21.15.64:0/1924196932 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b00095f0 con 0x7fb3b4100f30 2023-09-13T03:29:48.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.551+0000 7fb3b9b9b700 1 --2- 172.21.15.64:0/1924196932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3b4100570 0x7fb3b419e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:29:48.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.551+0000 7fb3b939a700 1 --2- 172.21.15.64:0/1924196932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3b4100f30 0x7fb3b419eae0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fb3a400e9a0 tx=0x7fb3a400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:48.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.552+0000 7fb3aaffd700 1 -- 172.21.15.64:0/1924196932 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a4009790 con 0x7fb3b4100f30 2023-09-13T03:29:48.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.552+0000 7fb3aaffd700 1 -- 172.21.15.64:0/1924196932 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3a4010430 con 0x7fb3b4100f30 2023-09-13T03:29:48.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.552+0000 7fb3bbdff700 1 -- 172.21.15.64:0/1924196932 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3b4199290 con 0x7fb3b4100f30 2023-09-13T03:29:48.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.552+0000 7fb3aaffd700 1 -- 172.21.15.64:0/1924196932 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a4018680 con 0x7fb3b4100f30 2023-09-13T03:29:48.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.552+0000 7fb3bbdff700 1 -- 172.21.15.64:0/1924196932 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b41a5590 con 0x7fb3b4100f30 2023-09-13T03:29:48.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.553+0000 7fb3bbdff700 1 -- 172.21.15.64:0/1924196932 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3b4067380 con 0x7fb3b4100f30 2023-09-13T03:29:48.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.555+0000 7fb3aaffd700 1 -- 172.21.15.64:0/1924196932 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fb3a40105a0 con 0x7fb3b4100f30 2023-09-13T03:29:48.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.557+0000 7fb3aaffd700 1 --2- 172.21.15.64:0/1924196932 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fb3a00a0830 0x7fb3a00a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:48.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.558+0000 7fb3aaffd700 1 -- 172.21.15.64:0/1924196932 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fb3a4014070 con 0x7fb3b4100f30 2023-09-13T03:29:48.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.558+0000 7fb3b9b9b700 1 --2- 172.21.15.64:0/1924196932 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fb3a00a0830 0x7fb3a00a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:48.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.558+0000 7fb3b9b9b700 1 --2- 172.21.15.64:0/1924196932 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fb3a00a0830 0x7fb3a00a2cf0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0009990 tx=0x7fb3b001a040 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:48.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.562+0000 7fb3aaffd700 1 -- 172.21.15.64:0/1924196932 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb3a4051e10 con 0x7fb3b4100f30 2023-09-13T03:29:48.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.790+0000 7fb3bbdff700 1 -- 172.21.15.64:0/1924196932 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb3b4199b20 con 0x7fb3b4100f30 2023-09-13T03:29:48.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.790+0000 7fb3aaffd700 1 -- 172.21.15.64:0/1924196932 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v106) v1 ==== 81+0+940 (secure 0 0 0) 0x7fb3a40958f0 con 0x7fb3b4100f30 2023-09-13T03:29:48.791 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6801/873679308 2023-09-14T03:13:09.492864+0000 2023-09-13T03:29:48.792 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1170163026 2023-09-14T03:13:09.492864+0000 2023-09-13T03:29:48.792 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/3771142265 2023-09-14T03:13:09.492864+0000 2023-09-13T03:29:48.792 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/4083942699 2023-09-14T03:13:09.492864+0000 2023-09-13T03:29:48.793 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6800/2955227347 2023-09-14T03:04:22.355817+0000 2023-09-13T03:29:48.793 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1373223118 2023-09-14T03:04:22.355817+0000 2023-09-13T03:29:48.793 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6800/873679308 2023-09-14T03:13:09.492864+0000 2023-09-13T03:29:48.794 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/791645251 2023-09-14T03:13:09.492864+0000 2023-09-13T03:29:48.794 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/271194613 2023-09-14T03:04:22.355817+0000 2023-09-13T03:29:48.794 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6801/3998091347 2023-09-14T03:03:56.442541+0000 2023-09-13T03:29:48.794 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6800/3998091347 2023-09-14T03:03:56.442541+0000 2023-09-13T03:29:48.795 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/920818572 2023-09-14T03:03:56.442541+0000 2023-09-13T03:29:48.795 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/735789005 2023-09-14T03:03:56.442541+0000 2023-09-13T03:29:48.795 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6801/2955227347 2023-09-14T03:04:22.355817+0000 2023-09-13T03:29:48.796 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1975923178 2023-09-14T03:03:56.442541+0000 2023-09-13T03:29:48.796 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1500151178 2023-09-14T03:04:22.355817+0000 2023-09-13T03:29:48.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.793+0000 7fb3bbdff700 1 -- 172.21.15.64:0/1924196932 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fb3a00a0830 msgr2=0x7fb3a00a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:48.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.793+0000 7fb3bbdff700 1 --2- 172.21.15.64:0/1924196932 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fb3a00a0830 0x7fb3a00a2cf0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0009990 tx=0x7fb3b001a040 comp rx=0 tx=0).stop 2023-09-13T03:29:48.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.793+0000 7fb3bbdff700 1 -- 172.21.15.64:0/1924196932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3b4100f30 msgr2=0x7fb3b419eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:48.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.793+0000 7fb3bbdff700 1 --2- 172.21.15.64:0/1924196932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3b4100f30 0x7fb3b419eae0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fb3a400e9a0 tx=0x7fb3a400ecb0 comp rx=0 tx=0).stop 2023-09-13T03:29:48.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.793+0000 7fb3bbdff700 1 -- 172.21.15.64:0/1924196932 shutdown_connections 2023-09-13T03:29:48.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.793+0000 7fb3bbdff700 1 --2- 172.21.15.64:0/1924196932 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fb3a00a0830 0x7fb3a00a2cf0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:48.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.793+0000 7fb3bbdff700 1 --2- 172.21.15.64:0/1924196932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3b4100570 0x7fb3b419e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:48.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.793+0000 7fb3bbdff700 1 --2- 172.21.15.64:0/1924196932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3b4106630 0x7fb3b4198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:48.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.793+0000 7fb3bbdff700 1 --2- 172.21.15.64:0/1924196932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3b4100f30 0x7fb3b419eae0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:48.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.793+0000 7fb3bbdff700 1 -- 172.21.15.64:0/1924196932 >> 172.21.15.64:0/1924196932 conn(0x7fb3b40fc080 msgr2=0x7fb3b40fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:48.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.793+0000 7fb3bbdff700 1 -- 172.21.15.64:0/1924196932 shutdown_connections 2023-09-13T03:29:48.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:48.793+0000 7fb3bbdff700 1 -- 172.21.15.64:0/1924196932 wait complete. 2023-09-13T03:29:48.801 INFO:teuthology.orchestra.run.smithi064.stderr:listed 16 entries 2023-09-13T03:29:48.813 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-09-13T03:29:48.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T03:29:48.814 DEBUG:teuthology.orchestra.run.smithi064:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-09-13T03:29:49.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:48 smithi170 ceph-mon[134328]: pgmap v522: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 2.1 MiB/s wr, 98 op/s 2023-09-13T03:29:49.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:48 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/368948433' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T03:29:49.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:48 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/370951338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T03:29:49.162 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:48 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/1924196932' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-13T03:29:49.173 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:48 smithi064 ceph-mon[97675]: pgmap v522: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 2.1 MiB/s wr, 98 op/s 2023-09-13T03:29:49.173 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:48 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/368948433' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T03:29:49.174 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:48 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/370951338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T03:29:49.174 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:48 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/1924196932' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-13T03:29:49.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:48 smithi142 ceph-mon[134082]: pgmap v522: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 2.1 MiB/s wr, 98 op/s 2023-09-13T03:29:49.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:48 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/368948433' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-13T03:29:49.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:48 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/370951338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-13T03:29:49.248 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:48 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/1924196932' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-13T03:29:49.307 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-09-13T03:29:49.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T03:29:49.308 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-09-13T03:29:49.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T03:29:49.489 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2023-09-13T03:29:49.646 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-09-13T03:29:49.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T03:29:49.648 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-13T03:29:49.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T03:29:49.801 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link delete ceph-brx 2023-09-13T03:29:49.972 INFO:tasks.kclient:Unmounting kernel clients... 2023-09-13T03:29:49.973 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-09-13T03:29:49.973 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-13T03:29:50.020 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345616k,nr_inodes=4086404,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275012k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/ef53b9237f008f7467187d48bea33aa53c32942fa4a34d12eee444a3663ba58f/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/OEB5XYXJ6BXUFGWGDAADVEFKNC:/var/lib/containers/storage/overlay/l/J2PLHJN7HVO2PLTM3OAGDXMI2B,upperdir=/var/lib/containers/storage/overlay/ef53b9237f008f7467187d48bea33aa53c32942fa4a34d12eee444a3663ba58f/diff,workdir=/var/lib/containers/storage/overlay/ef53b9237f008f7467187d48bea33aa53c32942fa4a34d12eee444a3663ba58f/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/ce4601be4804cdbfce7068c6836991445a4617fc9328e68788bb32a21f03f3a9/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/OEB5XYXJ6BXUFGWGDAADVEFKNC:/var/lib/containers/storage/overlay/l/J2PLHJN7HVO2PLTM3OAGDXMI2B,upperdir=/var/lib/containers/storage/overlay/ce4601be4804cdbfce7068c6836991445a4617fc9328e68788bb32a21f03f3a9/diff,workdir=/var/lib/containers/storage/overlay/ce4601be4804cdbfce7068c6836991445a4617fc9328e68788bb32a21f03f3a9/work,redirect_dir=on,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275012k,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0 overlay /var/lib/containers/storage/overlay/f42591e044bec6d27d8f6aced8d424422fcfc9e09b501c6b0fec384ca03f3b8a/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/OEB5XYXJ6BXUFGWGDAADVEFKNC:/var/lib/containers/storage/overlay/l/J2PLHJN7HVO2PLTM3OAGDXMI2B,upperdir=/var/lib/containers/storage/overlay/f42591e044bec6d27d8f6aced8d424422fcfc9e09b501c6b0fec384ca03f3b8a/diff,workdir=/var/lib/containers/storage/overlay/f42591e044bec6d27d8f6aced8d424422fcfc9e09b501c6b0fec384ca03f3b8a/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/f640bda255daf3a9831438f06b95b25672e0ff27625a9c70614643654c8b40c0/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/OEB5XYXJ6BXUFGWGDAADVEFKNC:/var/lib/containers/storage/overlay/l/J2PLHJN7HVO2PLTM3OAGDXMI2B,upperdir=/var/lib/containers/storage/overlay/f640bda255daf3a9831438f06b95b25672e0ff27625a9c70614643654c8b40c0/diff,workdir=/var/lib/containers/storage/overlay/f640bda255daf3a9831438f06b95b25672e0ff27625a9c70614643654c8b40c0/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/056e6588c86db30f07ea57433844ee73c68a14521bcdbf6c588f4261d1dbfcf2/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/OEB5XYXJ6BXUFGWGDAADVEFKNC:/var/lib/containers/storage/overlay/l/J2PLHJN7HVO2PLTM3OAGDXMI2B,upperdir=/var/lib/containers/storage/overlay/056e6588c86db30f07ea57433844ee73c68a14521bcdbf6c588f4261d1dbfcf2/diff,workdir=/var/lib/containers/storage/overlay/056e6588c86db30f07ea57433844ee73c68a14521bcdbf6c588f4261d1dbfcf2/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/678de7bcb605f2e31043dfe42caf71d9f9c2c5a729535738da013cff385e1a38/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/OEB5XYXJ6BXUFGWGDAADVEFKNC:/var/lib/containers/storage/overlay/l/J2PLHJN7HVO2PLTM3OAGDXMI2B,upperdir=/var/lib/containers/storage/overlay/678de7bcb605f2e31043dfe42caf71d9f9c2c5a729535738da013cff385e1a38/diff,workdir=/var/lib/containers/storage/overlay/678de7bcb605f2e31043dfe42caf71d9f9c2c5a729535738da013cff385e1a38/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/e75327f12857aa3bf6e1eb2e48ca2a0f746caf5e315728137c33a30a2250abce/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/OEB5XYXJ6BXUFGWGDAADVEFKNC:/var/lib/containers/storage/overlay/l/J2PLHJN7HVO2PLTM3OAGDXMI2B,upperdir=/var/lib/containers/storage/overlay/e75327f12857aa3bf6e1eb2e48ca2a0f746caf5e315728137c33a30a2250abce/diff,workdir=/var/lib/containers/storage/overlay/e75327f12857aa3bf6e1eb2e48ca2a0f746caf5e315728137c33a30a2250abce/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/5d8d4a249c484dc0688d41a8945c9e0b9d90ac7f3fc539e48c09e213d002c835/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/OEB5XYXJ6BXUFGWGDAADVEFKNC:/var/lib/containers/storage/overlay/l/J2PLHJN7HVO2PLTM3OAGDXMI2B,upperdir=/var/lib/containers/storage/overlay/5d8d4a249c484dc0688d41a8945c9e0b9d90ac7f3fc539e48c09e213d002c835/diff,workdir=/var/lib/containers/storage/overlay/5d8d4a249c484dc0688d41a8945c9e0b9d90ac7f3fc539e48c09e213d002c835/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/10d003c17ac6250af142135feaf1b67b428723eb3df1e4eb4796caab76680c64/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/OEB5XYXJ6BXUFGWGDAADVEFKNC:/var/lib/containers/storage/overlay/l/J2PLHJN7HVO2PLTM3OAGDXMI2B,upperdir=/var/lib/containers/storage/overlay/10d003c17ac6250af142135feaf1b67b428723eb3df1e4eb4796caab76680c64/diff,workdir=/var/lib/containers/storage/overlay/10d003c17ac6250af142135feaf1b67b428723eb3df1e4eb4796caab76680c64/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/ec7a7c0a2471de085b08c59a6dd69830dedf1647063f58de6a1800f2f0ee1315/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/OEB5XYXJ6BXUFGWGDAADVEFKNC:/var/lib/containers/storage/overlay/l/J2PLHJN7HVO2PLTM3OAGDXMI2B,upperdir=/var/lib/containers/storage/overlay/ec7a7c0a2471de085b08c59a6dd69830dedf1647063f58de6a1800f2f0ee1315/diff,workdir=/var/lib/containers/storage/overlay/ec7a7c0a2471de085b08c59a6dd69830dedf1647063f58de6a1800f2f0ee1315/work,redirect_dir=on,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 2023-09-13T03:29:50.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T03:29:50.021 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2023-09-13T03:29:50.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-13T03:29:50.104 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link delete ceph-brx 2023-09-13T03:29:50.220 INFO:teuthology.orchestra.run.smithi064.stderr:Cannot find device "ceph-brx" 2023-09-13T03:29:50.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-13T03:29:50.223 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-09-13T03:29:50.234 INFO:tasks.cephadm:Teardown begin 2023-09-13T03:29:50.234 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-09-13T03:29:50.235 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T03:29:50.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.438+0000 7fad12bff700 1 -- 172.21.15.64:0/4278453325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad0c106680 msgr2=0x7fad0c106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:50.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.438+0000 7fad12bff700 1 --2- 172.21.15.64:0/4278453325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad0c106680 0x7fad0c106a60 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fad000099c0 tx=0x7fad00009cd0 comp rx=0 tx=0).stop 2023-09-13T03:29:50.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.439+0000 7fad12bff700 1 -- 172.21.15.64:0/4278453325 shutdown_connections 2023-09-13T03:29:50.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.439+0000 7fad12bff700 1 --2- 172.21.15.64:0/4278453325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad0c100f80 0x7fad0c10d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:50.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.439+0000 7fad12bff700 1 --2- 172.21.15.64:0/4278453325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad0c106680 0x7fad0c106a60 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:50.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.439+0000 7fad12bff700 1 --2- 172.21.15.64:0/4278453325 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad0c1005c0 0x7fad0c100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:50.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.439+0000 7fad12bff700 1 -- 172.21.15.64:0/4278453325 >> 172.21.15.64:0/4278453325 conn(0x7fad0c0fc0b0 msgr2=0x7fad0c0fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:50.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.439+0000 7fad12bff700 1 -- 172.21.15.64:0/4278453325 shutdown_connections 2023-09-13T03:29:50.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.439+0000 7fad12bff700 1 -- 172.21.15.64:0/4278453325 wait complete. 2023-09-13T03:29:50.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.440+0000 7fad12bff700 1 Processor -- start 2023-09-13T03:29:50.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.440+0000 7fad12bff700 1 -- start start 2023-09-13T03:29:50.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.440+0000 7fad12bff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad0c1005c0 0x7fad0c19e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:50.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.440+0000 7fad12bff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad0c100f80 0x7fad0c19eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:50.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.440+0000 7fad12bff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad0c106680 0x7fad0c198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:50.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.440+0000 7fad12bff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c198cb0 con 0x7fad0c1005c0 2023-09-13T03:29:50.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.440+0000 7fad12bff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c198e20 con 0x7fad0c100f80 2023-09-13T03:29:50.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.440+0000 7fad12bff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c198fe0 con 0x7fad0c106680 2023-09-13T03:29:50.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.441+0000 7fad1099b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad0c1005c0 0x7fad0c19e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:50.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.441+0000 7fad1099b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad0c1005c0 0x7fad0c19e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44280/0 (socket says 172.21.15.64:44280) 2023-09-13T03:29:50.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.441+0000 7fad1099b700 1 -- 172.21.15.64:0/93095690 learned_addr learned my addr 172.21.15.64:0/93095690 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:29:50.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.441+0000 7fad0bfff700 1 --2- 172.21.15.64:0/93095690 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad0c100f80 0x7fad0c19eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:50.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.441+0000 7fad1119c700 1 --2- 172.21.15.64:0/93095690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad0c106680 0x7fad0c198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:50.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.441+0000 7fad1099b700 1 -- 172.21.15.64:0/93095690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad0c106680 msgr2=0x7fad0c198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:50.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.441+0000 7fad1099b700 1 --2- 172.21.15.64:0/93095690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad0c106680 0x7fad0c198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:50.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.441+0000 7fad1099b700 1 -- 172.21.15.64:0/93095690 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad0c100f80 msgr2=0x7fad0c19eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:50.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.441+0000 7fad1099b700 1 --2- 172.21.15.64:0/93095690 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad0c100f80 0x7fad0c19eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:50.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.441+0000 7fad1099b700 1 -- 172.21.15.64:0/93095690 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf8009520 con 0x7fad0c1005c0 2023-09-13T03:29:50.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.441+0000 7fad1099b700 1 --2- 172.21.15.64:0/93095690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad0c1005c0 0x7fad0c19e5d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fad0000b580 tx=0x7fad0000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:50.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.441+0000 7fad09ffb700 1 -- 172.21.15.64:0/93095690 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0001d070 con 0x7fad0c1005c0 2023-09-13T03:29:50.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.442+0000 7fad09ffb700 1 -- 172.21.15.64:0/93095690 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad00005050 con 0x7fad0c1005c0 2023-09-13T03:29:50.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.442+0000 7fad12bff700 1 -- 172.21.15.64:0/93095690 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad000095f0 con 0x7fad0c1005c0 2023-09-13T03:29:50.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.442+0000 7fad09ffb700 1 -- 172.21.15.64:0/93095690 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad00004a40 con 0x7fad0c1005c0 2023-09-13T03:29:50.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.442+0000 7fad12bff700 1 -- 172.21.15.64:0/93095690 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad0c1a53e0 con 0x7fad0c1005c0 2023-09-13T03:29:50.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.443+0000 7fad12bff700 1 -- 172.21.15.64:0/93095690 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad0c0673f0 con 0x7fad0c1005c0 2023-09-13T03:29:50.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.446+0000 7fad09ffb700 1 -- 172.21.15.64:0/93095690 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fad000051c0 con 0x7fad0c1005c0 2023-09-13T03:29:50.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.448+0000 7fad09ffb700 1 --2- 172.21.15.64:0/93095690 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7facfc0a0870 0x7facfc0a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:50.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.448+0000 7fad09ffb700 1 -- 172.21.15.64:0/93095690 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fad000cd060 con 0x7fad0c1005c0 2023-09-13T03:29:50.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.448+0000 7fad0bfff700 1 --2- 172.21.15.64:0/93095690 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7facfc0a0870 0x7facfc0a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:50.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.448+0000 7fad0bfff700 1 --2- 172.21.15.64:0/93095690 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7facfc0a0870 0x7facfc0a2d30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fad0c0ff6e0 tx=0x7facf8009260 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:50.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.450+0000 7fad09ffb700 1 -- 172.21.15.64:0/93095690 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fad000968a0 con 0x7fad0c1005c0 2023-09-13T03:29:50.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.714+0000 7fad12bff700 1 -- 172.21.15.64:0/93095690 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad0c199b50 con 0x7fad0c1005c0 2023-09-13T03:29:50.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.714+0000 7fad09ffb700 1 -- 172.21.15.64:0/93095690 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+11371 (secure 0 0 0) 0x7fad00096020 con 0x7fad0c1005c0 2023-09-13T03:29:50.715 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:29:50.716 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":83,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2234182985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2234182985},{"type":"v1","addr":"172.21.15.64:6835","nonce":2234182985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4053841873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4053841873},{"type":"v1","addr":"172.21.15.64:6837","nonce":4053841873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":83,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-13T03:11:10.490260+0000","modified":"2023-09-13T03:29:35.383933+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24686,"mds_1":24649,"mds_2":24635,"mds_3":24634,"mds_4":24622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6839/99523374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":99523374},{"type":"v1","addr":"172.21.15.64:6839","nonce":99523374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14649":{"gid":14649,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6841/1866582875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":1866582875},{"type":"v1","addr":"172.21.15.64:6841","nonce":1866582875}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/101906999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":101906999},{"type":"v1","addr":"172.21.15.142:6833","nonce":101906999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24574":{"gid":24574,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/1897366227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1897366227},{"type":"v1","addr":"172.21.15.170:6833","nonce":1897366227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24611":{"gid":24611,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2167779918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2167779918},{"type":"v1","addr":"172.21.15.142:6835","nonce":2167779918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24622":{"gid":24622,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6835/1493371121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1493371121},{"type":"v1","addr":"172.21.15.170:6835","nonce":1493371121}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24634":{"gid":24634,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.170:6837/1726587275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":1726587275},{"type":"v1","addr":"172.21.15.170:6837","nonce":1726587275}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24635":{"gid":24635,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":52,"addr":"172.21.15.142:6837/741173268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":741173268},{"type":"v1","addr":"172.21.15.142:6837","nonce":741173268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24649":{"gid":24649,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.170:6839/2189703596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2189703596},{"type":"v1","addr":"172.21.15.170:6839","nonce":2189703596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24686":{"gid":24686,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/3645998616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645998616},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645998616}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-13T03:29:50.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.717+0000 7fad12bff700 1 -- 172.21.15.64:0/93095690 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7facfc0a0870 msgr2=0x7facfc0a2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:50.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.717+0000 7fad12bff700 1 --2- 172.21.15.64:0/93095690 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7facfc0a0870 0x7facfc0a2d30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fad0c0ff6e0 tx=0x7facf8009260 comp rx=0 tx=0).stop 2023-09-13T03:29:50.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.717+0000 7fad12bff700 1 -- 172.21.15.64:0/93095690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad0c1005c0 msgr2=0x7fad0c19e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:50.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.717+0000 7fad12bff700 1 --2- 172.21.15.64:0/93095690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad0c1005c0 0x7fad0c19e5d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fad0000b580 tx=0x7fad0000bfd0 comp rx=0 tx=0).stop 2023-09-13T03:29:50.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.717+0000 7fad12bff700 1 -- 172.21.15.64:0/93095690 shutdown_connections 2023-09-13T03:29:50.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.717+0000 7fad12bff700 1 --2- 172.21.15.64:0/93095690 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7facfc0a0870 0x7facfc0a2d30 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:50.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.717+0000 7fad12bff700 1 --2- 172.21.15.64:0/93095690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad0c106680 0x7fad0c198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:50.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.717+0000 7fad12bff700 1 --2- 172.21.15.64:0/93095690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad0c1005c0 0x7fad0c19e5d0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:50.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.717+0000 7fad12bff700 1 --2- 172.21.15.64:0/93095690 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad0c100f80 0x7fad0c19eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:50.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.717+0000 7fad12bff700 1 -- 172.21.15.64:0/93095690 >> 172.21.15.64:0/93095690 conn(0x7fad0c0fc0b0 msgr2=0x7fad0c072830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:50.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.717+0000 7fad12bff700 1 -- 172.21.15.64:0/93095690 shutdown_connections 2023-09-13T03:29:50.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.717+0000 7fad12bff700 1 -- 172.21.15.64:0/93095690 wait complete. 2023-09-13T03:29:50.721 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 83 2023-09-13T03:29:50.737 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-13T03:29:50.800 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:50 smithi064 ceph-mon[97675]: pgmap v523: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 1.5 MiB/s wr, 67 op/s 2023-09-13T03:29:50.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:50 smithi170 ceph-mon[134328]: pgmap v523: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 1.5 MiB/s wr, 67 op/s 2023-09-13T03:29:50.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.943+0000 7fecdcdff700 1 -- 172.21.15.64:0/568621122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecd8106650 msgr2=0x7fecd8106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:50.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.943+0000 7fecdcdff700 1 --2- 172.21.15.64:0/568621122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecd8106650 0x7fecd8106a30 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fecc80099c0 tx=0x7fecc8009cd0 comp rx=0 tx=0).stop 2023-09-13T03:29:50.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.943+0000 7fecdcdff700 1 -- 172.21.15.64:0/568621122 shutdown_connections 2023-09-13T03:29:50.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.943+0000 7fecdcdff700 1 --2- 172.21.15.64:0/568621122 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecd8100f50 0x7fecd810d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:50.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.943+0000 7fecdcdff700 1 --2- 172.21.15.64:0/568621122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecd8106650 0x7fecd8106a30 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:50.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.943+0000 7fecdcdff700 1 --2- 172.21.15.64:0/568621122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecd8100590 0x7fecd8100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:50.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.943+0000 7fecdcdff700 1 -- 172.21.15.64:0/568621122 >> 172.21.15.64:0/568621122 conn(0x7fecd80fc020 msgr2=0x7fecd80fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:50.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.943+0000 7fecdcdff700 1 -- 172.21.15.64:0/568621122 shutdown_connections 2023-09-13T03:29:50.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.943+0000 7fecdcdff700 1 -- 172.21.15.64:0/568621122 wait complete. 2023-09-13T03:29:50.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.944+0000 7fecdcdff700 1 Processor -- start 2023-09-13T03:29:50.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.944+0000 7fecdcdff700 1 -- start start 2023-09-13T03:29:50.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.945+0000 7fecdcdff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecd8100590 0x7fecd81966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:50.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.945+0000 7fecdcdff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecd8100f50 0x7fecd8196c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:50.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.945+0000 7fecdcdff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecd8106650 0x7fecd8199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:50.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.945+0000 7fecdcdff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd819a5b0 con 0x7fecd8100f50 2023-09-13T03:29:50.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.945+0000 7fecdcdff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd819a720 con 0x7fecd8106650 2023-09-13T03:29:50.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.945+0000 7fecdcdff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd819a8e0 con 0x7fecd8100590 2023-09-13T03:29:50.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.945+0000 7feccffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecd8100f50 0x7fecd8196c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:50.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.945+0000 7feccffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecd8100f50 0x7fecd8196c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44302/0 (socket says 172.21.15.64:44302) 2023-09-13T03:29:50.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.945+0000 7feccffff700 1 -- 172.21.15.64:0/2092670556 learned_addr learned my addr 172.21.15.64:0/2092670556 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:29:50.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.945+0000 7fecd6d9d700 1 --2- 172.21.15.64:0/2092670556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecd8106650 0x7fecd8199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:50.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.945+0000 7fecd659c700 1 --2- 172.21.15.64:0/2092670556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecd8100590 0x7fecd81966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:50.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.945+0000 7feccffff700 1 -- 172.21.15.64:0/2092670556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecd8100590 msgr2=0x7fecd81966c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:50.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.945+0000 7feccffff700 1 --2- 172.21.15.64:0/2092670556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecd8100590 0x7fecd81966c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:50.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.945+0000 7feccffff700 1 -- 172.21.15.64:0/2092670556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecd8106650 msgr2=0x7fecd8199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:50.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.945+0000 7feccffff700 1 --2- 172.21.15.64:0/2092670556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecd8106650 0x7fecd8199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:50.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.945+0000 7feccffff700 1 -- 172.21.15.64:0/2092670556 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc80095f0 con 0x7fecd8100f50 2023-09-13T03:29:50.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.945+0000 7fecd6d9d700 1 --2- 172.21.15.64:0/2092670556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecd8106650 0x7fecd8199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:29:50.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.946+0000 7feccffff700 1 --2- 172.21.15.64:0/2092670556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecd8100f50 0x7fecd8196c00 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fecc000e9a0 tx=0x7fecc000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:50.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.946+0000 7feccf7fe700 1 -- 172.21.15.64:0/2092670556 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc0009790 con 0x7fecd8100f50 2023-09-13T03:29:50.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.946+0000 7fecdcdff700 1 -- 172.21.15.64:0/2092670556 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecd819abc0 con 0x7fecd8100f50 2023-09-13T03:29:50.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.946+0000 7feccf7fe700 1 -- 172.21.15.64:0/2092670556 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fecc0010430 con 0x7fecd8100f50 2023-09-13T03:29:50.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.946+0000 7feccf7fe700 1 -- 172.21.15.64:0/2092670556 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc0018680 con 0x7fecd8100f50 2023-09-13T03:29:50.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.946+0000 7fecdcdff700 1 -- 172.21.15.64:0/2092670556 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecd81a3520 con 0x7fecd8100f50 2023-09-13T03:29:50.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.947+0000 7fecdcdff700 1 -- 172.21.15.64:0/2092670556 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecd80673f0 con 0x7fecd8100f50 2023-09-13T03:29:50.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.949+0000 7feccf7fe700 1 -- 172.21.15.64:0/2092670556 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fecc00105a0 con 0x7fecd8100f50 2023-09-13T03:29:50.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.950+0000 7feccf7fe700 1 --2- 172.21.15.64:0/2092670556 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fecc40a07f0 0x7fecc40a2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:50.956 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.951+0000 7feccf7fe700 1 -- 172.21.15.64:0/2092670556 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fecc0014070 con 0x7fecd8100f50 2023-09-13T03:29:50.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.951+0000 7fecd659c700 1 --2- 172.21.15.64:0/2092670556 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fecc40a07f0 0x7fecc40a2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:50.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.951+0000 7fecd659c700 1 --2- 172.21.15.64:0/2092670556 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fecc40a07f0 0x7fecc40a2cb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fecc8009990 tx=0x7fecc8005fb0 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:50.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:50.954+0000 7feccf7fe700 1 -- 172.21.15.64:0/2092670556 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fecc0051e10 con 0x7fecd8100f50 2023-09-13T03:29:50.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:50 smithi142 ceph-mon[134082]: pgmap v523: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 1.5 MiB/s wr, 67 op/s 2023-09-13T03:29:51.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.217+0000 7fecdcdff700 1 -- 172.21.15.64:0/2092670556 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fecd8197a10 con 0x7fecd8100f50 2023-09-13T03:29:51.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.217+0000 7feccf7fe700 1 -- 172.21.15.64:0/2092670556 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+11371 (secure 0 0 0) 0x7fecc0095940 con 0x7fecd8100f50 2023-09-13T03:29:51.218 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:29:51.219 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":83,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2234182985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2234182985},{"type":"v1","addr":"172.21.15.64:6835","nonce":2234182985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4053841873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4053841873},{"type":"v1","addr":"172.21.15.64:6837","nonce":4053841873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":83,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-13T03:11:10.490260+0000","modified":"2023-09-13T03:29:35.383933+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24686,"mds_1":24649,"mds_2":24635,"mds_3":24634,"mds_4":24622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6839/99523374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":99523374},{"type":"v1","addr":"172.21.15.64:6839","nonce":99523374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14649":{"gid":14649,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6841/1866582875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":1866582875},{"type":"v1","addr":"172.21.15.64:6841","nonce":1866582875}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/101906999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":101906999},{"type":"v1","addr":"172.21.15.142:6833","nonce":101906999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24574":{"gid":24574,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/1897366227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1897366227},{"type":"v1","addr":"172.21.15.170:6833","nonce":1897366227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24611":{"gid":24611,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2167779918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2167779918},{"type":"v1","addr":"172.21.15.142:6835","nonce":2167779918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24622":{"gid":24622,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.170:6835/1493371121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1493371121},{"type":"v1","addr":"172.21.15.170:6835","nonce":1493371121}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24634":{"gid":24634,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.170:6837/1726587275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":1726587275},{"type":"v1","addr":"172.21.15.170:6837","nonce":1726587275}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24635":{"gid":24635,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":52,"addr":"172.21.15.142:6837/741173268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":741173268},{"type":"v1","addr":"172.21.15.142:6837","nonce":741173268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24649":{"gid":24649,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.170:6839/2189703596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2189703596},{"type":"v1","addr":"172.21.15.170:6839","nonce":2189703596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24686":{"gid":24686,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/3645998616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645998616},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645998616}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-13T03:29:51.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.220+0000 7fecdcdff700 1 -- 172.21.15.64:0/2092670556 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fecc40a07f0 msgr2=0x7fecc40a2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:51.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.220+0000 7fecdcdff700 1 --2- 172.21.15.64:0/2092670556 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fecc40a07f0 0x7fecc40a2cb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fecc8009990 tx=0x7fecc8005fb0 comp rx=0 tx=0).stop 2023-09-13T03:29:51.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.220+0000 7fecdcdff700 1 -- 172.21.15.64:0/2092670556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecd8100f50 msgr2=0x7fecd8196c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:51.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.220+0000 7fecdcdff700 1 --2- 172.21.15.64:0/2092670556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecd8100f50 0x7fecd8196c00 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fecc000e9a0 tx=0x7fecc000ecb0 comp rx=0 tx=0).stop 2023-09-13T03:29:51.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.220+0000 7fecdcdff700 1 -- 172.21.15.64:0/2092670556 shutdown_connections 2023-09-13T03:29:51.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.220+0000 7fecdcdff700 1 --2- 172.21.15.64:0/2092670556 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fecc40a07f0 0x7fecc40a2cb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:51.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.220+0000 7fecdcdff700 1 --2- 172.21.15.64:0/2092670556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecd8106650 0x7fecd8199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:51.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.220+0000 7fecdcdff700 1 --2- 172.21.15.64:0/2092670556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecd8100590 0x7fecd81966c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:51.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.220+0000 7fecdcdff700 1 --2- 172.21.15.64:0/2092670556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecd8100f50 0x7fecd8196c00 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:51.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.220+0000 7fecdcdff700 1 -- 172.21.15.64:0/2092670556 >> 172.21.15.64:0/2092670556 conn(0x7fecd80fc020 msgr2=0x7fecd80fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:51.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.220+0000 7fecdcdff700 1 -- 172.21.15.64:0/2092670556 shutdown_connections 2023-09-13T03:29:51.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.220+0000 7fecdcdff700 1 -- 172.21.15.64:0/2092670556 wait complete. 2023-09-13T03:29:51.224 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 83 2023-09-13T03:29:51.239 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-13T03:29:51.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.444+0000 7fad871ff700 1 -- 172.21.15.64:0/2927915198 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad80106650 msgr2=0x7fad80106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:51.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.444+0000 7fad871ff700 1 --2- 172.21.15.64:0/2927915198 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad80106650 0x7fad80106a30 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fad740099c0 tx=0x7fad74009cd0 comp rx=0 tx=0).stop 2023-09-13T03:29:51.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.444+0000 7fad871ff700 1 -- 172.21.15.64:0/2927915198 shutdown_connections 2023-09-13T03:29:51.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.444+0000 7fad871ff700 1 --2- 172.21.15.64:0/2927915198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad80100f50 0x7fad8010d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:51.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.444+0000 7fad871ff700 1 --2- 172.21.15.64:0/2927915198 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad80106650 0x7fad80106a30 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:51.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.444+0000 7fad871ff700 1 --2- 172.21.15.64:0/2927915198 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad80100590 0x7fad80100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:51.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.444+0000 7fad871ff700 1 -- 172.21.15.64:0/2927915198 >> 172.21.15.64:0/2927915198 conn(0x7fad800fc020 msgr2=0x7fad800fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:51.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.444+0000 7fad871ff700 1 -- 172.21.15.64:0/2927915198 shutdown_connections 2023-09-13T03:29:51.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.444+0000 7fad871ff700 1 -- 172.21.15.64:0/2927915198 wait complete. 2023-09-13T03:29:51.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.445+0000 7fad871ff700 1 Processor -- start 2023-09-13T03:29:51.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.445+0000 7fad871ff700 1 -- start start 2023-09-13T03:29:51.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.445+0000 7fad871ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad80100590 0x7fad80072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:51.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.446+0000 7fad871ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad80100f50 0x7fad80072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:51.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.446+0000 7fad871ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad80106650 0x7fad801a4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:51.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.446+0000 7fad871ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad80073670 con 0x7fad80100590 2023-09-13T03:29:51.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.446+0000 7fad871ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad801a4d40 con 0x7fad80106650 2023-09-13T03:29:51.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.446+0000 7fad871ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad801a4e80 con 0x7fad80100f50 2023-09-13T03:29:51.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.446+0000 7fad84f9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad80100590 0x7fad80072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:51.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.446+0000 7fad84f9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad80100590 0x7fad80072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44318/0 (socket says 172.21.15.64:44318) 2023-09-13T03:29:51.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.446+0000 7fad84f9b700 1 -- 172.21.15.64:0/3172393072 learned_addr learned my addr 172.21.15.64:0/3172393072 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:29:51.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.446+0000 7fad7ffff700 1 --2- 172.21.15.64:0/3172393072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad80100f50 0x7fad80072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:51.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.446+0000 7fad8579c700 1 --2- 172.21.15.64:0/3172393072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad80106650 0x7fad801a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:51.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.446+0000 7fad84f9b700 1 -- 172.21.15.64:0/3172393072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad80100f50 msgr2=0x7fad80072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:51.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.446+0000 7fad84f9b700 1 --2- 172.21.15.64:0/3172393072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad80100f50 0x7fad80072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:51.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.446+0000 7fad84f9b700 1 -- 172.21.15.64:0/3172393072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad80106650 msgr2=0x7fad801a4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:51.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.446+0000 7fad84f9b700 1 --2- 172.21.15.64:0/3172393072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad80106650 0x7fad801a4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:51.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.446+0000 7fad84f9b700 1 -- 172.21.15.64:0/3172393072 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad740095f0 con 0x7fad80100590 2023-09-13T03:29:51.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.446+0000 7fad84f9b700 1 --2- 172.21.15.64:0/3172393072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad80100590 0x7fad80072900 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fad74009990 tx=0x7fad740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:51.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.447+0000 7fad7dffb700 1 -- 172.21.15.64:0/3172393072 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7401d070 con 0x7fad80100590 2023-09-13T03:29:51.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.447+0000 7fad7dffb700 1 -- 172.21.15.64:0/3172393072 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad74004010 con 0x7fad80100590 2023-09-13T03:29:51.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.447+0000 7fad7dffb700 1 -- 172.21.15.64:0/3172393072 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7400fd30 con 0x7fad80100590 2023-09-13T03:29:51.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.447+0000 7fad871ff700 1 -- 172.21.15.64:0/3172393072 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad801a5080 con 0x7fad80100590 2023-09-13T03:29:51.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.447+0000 7fad871ff700 1 -- 172.21.15.64:0/3172393072 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad801a5450 con 0x7fad80100590 2023-09-13T03:29:51.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.448+0000 7fad871ff700 1 -- 172.21.15.64:0/3172393072 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad800673f0 con 0x7fad80100590 2023-09-13T03:29:51.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.450+0000 7fad7dffb700 1 -- 172.21.15.64:0/3172393072 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fad7400f460 con 0x7fad80100590 2023-09-13T03:29:51.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.452+0000 7fad7dffb700 1 --2- 172.21.15.64:0/3172393072 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fad700a07f0 0x7fad700a2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:51.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.452+0000 7fad7dffb700 1 -- 172.21.15.64:0/3172393072 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fad740ccfd0 con 0x7fad80100590 2023-09-13T03:29:51.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.452+0000 7fad7ffff700 1 --2- 172.21.15.64:0/3172393072 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fad700a07f0 0x7fad700a2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:51.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.453+0000 7fad7ffff700 1 --2- 172.21.15.64:0/3172393072 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fad700a07f0 0x7fad700a2cb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fad800740f0 tx=0x7fad6c00b5b0 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:51.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.456+0000 7fad7dffb700 1 -- 172.21.15.64:0/3172393072 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fad74096870 con 0x7fad80100590 2023-09-13T03:29:51.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.670+0000 7fad871ff700 1 -- 172.21.15.64:0/3172393072 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad80073e30 con 0x7fad80100590 2023-09-13T03:29:51.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.671+0000 7fad7dffb700 1 -- 172.21.15.64:0/3172393072 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+22942 (secure 0 0 0) 0x7fad74099e90 con 0x7fad80100590 2023-09-13T03:29:51.672 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-09-13T03:29:51.673 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":106,"fsid":"f8dbced4-51e1-11ee-9ab7-7b867c8bd7da","created":"2023-09-13T03:03:24.167202+0000","modified":"2023-09-13T03:29:14.506662+0000","last_up_change":"2023-09-13T03:10:22.503529+0000","last_in_change":"2023-09-13T03:10:05.678720+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-13T03:06:58.987479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-13T03:11:08.589202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-13T03:11:09.924659+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3997c114-241b-48c4-8feb-653c99209470","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6803","nonce":3940892135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6805","nonce":3940892135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6809","nonce":3940892135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3940892135},{"type":"v1","addr":"172.21.15.64:6807","nonce":3940892135}]},"public_addr":"172.21.15.64:6803/3940892135","cluster_addr":"172.21.15.64:6805/3940892135","heartbeat_back_addr":"172.21.15.64:6809/3940892135","heartbeat_front_addr":"172.21.15.64:6807/3940892135","state":["exists","up"]},{"osd":1,"uuid":"94cfae85-322e-4ed2-a6e6-e40349bb14bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6801","nonce":4188149333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6803","nonce":4188149333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6807","nonce":4188149333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4188149333},{"type":"v1","addr":"172.21.15.142:6805","nonce":4188149333}]},"public_addr":"172.21.15.142:6801/4188149333","cluster_addr":"172.21.15.142:6803/4188149333","heartbeat_back_addr":"172.21.15.142:6807/4188149333","heartbeat_front_addr":"172.21.15.142:6805/4188149333","state":["exists","up"]},{"osd":2,"uuid":"b3aae17e-96db-4008-8c27-f733c46b30ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6801","nonce":2406669338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6803","nonce":2406669338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6807","nonce":2406669338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2406669338},{"type":"v1","addr":"172.21.15.170:6805","nonce":2406669338}]},"public_addr":"172.21.15.170:6801/2406669338","cluster_addr":"172.21.15.170:6803/2406669338","heartbeat_back_addr":"172.21.15.170:6807/2406669338","heartbeat_front_addr":"172.21.15.170:6805/2406669338","state":["exists","up"]},{"osd":3,"uuid":"7fa8d8cb-2ab7-48a7-8b49-5e2903d6779f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6811","nonce":3558318707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6813","nonce":3558318707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6817","nonce":3558318707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":3558318707},{"type":"v1","addr":"172.21.15.64:6815","nonce":3558318707}]},"public_addr":"172.21.15.64:6811/3558318707","cluster_addr":"172.21.15.64:6813/3558318707","heartbeat_back_addr":"172.21.15.64:6817/3558318707","heartbeat_front_addr":"172.21.15.64:6815/3558318707","state":["exists","up"]},{"osd":4,"uuid":"8c7a9c9f-f919-48d3-9db2-214a25bb5603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6809","nonce":1811142576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6811","nonce":1811142576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6815","nonce":1811142576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1811142576},{"type":"v1","addr":"172.21.15.142:6813","nonce":1811142576}]},"public_addr":"172.21.15.142:6809/1811142576","cluster_addr":"172.21.15.142:6811/1811142576","heartbeat_back_addr":"172.21.15.142:6815/1811142576","heartbeat_front_addr":"172.21.15.142:6813/1811142576","state":["exists","up"]},{"osd":5,"uuid":"0c6101c6-2065-4022-a751-d3f216d7f478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6809","nonce":2903841453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6811","nonce":2903841453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6815","nonce":2903841453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2903841453},{"type":"v1","addr":"172.21.15.170:6813","nonce":2903841453}]},"public_addr":"172.21.15.170:6809/2903841453","cluster_addr":"172.21.15.170:6811/2903841453","heartbeat_back_addr":"172.21.15.170:6815/2903841453","heartbeat_front_addr":"172.21.15.170:6813/2903841453","state":["exists","up"]},{"osd":6,"uuid":"5cf3e52e-d155-41b1-9194-db047d26a953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6819","nonce":1436584490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6821","nonce":1436584490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6825","nonce":1436584490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1436584490},{"type":"v1","addr":"172.21.15.64:6823","nonce":1436584490}]},"public_addr":"172.21.15.64:6819/1436584490","cluster_addr":"172.21.15.64:6821/1436584490","heartbeat_back_addr":"172.21.15.64:6825/1436584490","heartbeat_front_addr":"172.21.15.64:6823/1436584490","state":["exists","up"]},{"osd":7,"uuid":"e97323e3-badd-41a7-af9e-e78a81a64046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6817","nonce":3726793534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6819","nonce":3726793534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6823","nonce":3726793534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3726793534},{"type":"v1","addr":"172.21.15.142:6821","nonce":3726793534}]},"public_addr":"172.21.15.142:6817/3726793534","cluster_addr":"172.21.15.142:6819/3726793534","heartbeat_back_addr":"172.21.15.142:6823/3726793534","heartbeat_front_addr":"172.21.15.142:6821/3726793534","state":["exists","up"]},{"osd":8,"uuid":"83d2835c-ed26-493d-9222-c2894ec0e071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6817","nonce":4272513873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6819","nonce":4272513873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6823","nonce":4272513873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4272513873},{"type":"v1","addr":"172.21.15.170:6821","nonce":4272513873}]},"public_addr":"172.21.15.170:6817/4272513873","cluster_addr":"172.21.15.170:6819/4272513873","heartbeat_back_addr":"172.21.15.170:6823/4272513873","heartbeat_front_addr":"172.21.15.170:6821/4272513873","state":["exists","up"]},{"osd":9,"uuid":"cb936bdd-1bec-4236-aa2e-a24adcac11e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6827","nonce":2641175211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6829","nonce":2641175211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6833","nonce":2641175211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2641175211},{"type":"v1","addr":"172.21.15.64:6831","nonce":2641175211}]},"public_addr":"172.21.15.64:6827/2641175211","cluster_addr":"172.21.15.64:6829/2641175211","heartbeat_back_addr":"172.21.15.64:6833/2641175211","heartbeat_front_addr":"172.21.15.64:6831/2641175211","state":["exists","up"]},{"osd":10,"uuid":"85512a1c-6171-43d4-9c15-c588a8b71be0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6825","nonce":936499217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6827","nonce":936499217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6831","nonce":936499217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":936499217},{"type":"v1","addr":"172.21.15.142:6829","nonce":936499217}]},"public_addr":"172.21.15.142:6825/936499217","cluster_addr":"172.21.15.142:6827/936499217","heartbeat_back_addr":"172.21.15.142:6831/936499217","heartbeat_front_addr":"172.21.15.142:6829/936499217","state":["exists","up"]},{"osd":11,"uuid":"9b8417dc-5d6c-4917-94a9-b032c5bc33d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6825","nonce":3605017060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6827","nonce":3605017060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6831","nonce":3605017060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3605017060},{"type":"v1","addr":"172.21.15.170:6829","nonce":3605017060}]},"public_addr":"172.21.15.170:6825/3605017060","cluster_addr":"172.21.15.170:6827/3605017060","heartbeat_back_addr":"172.21.15.170:6831/3605017060","heartbeat_front_addr":"172.21.15.170:6829/3605017060","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:14.229044+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:36.147777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:06:56.655298+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:07:23.107741+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:07:44.663770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:05.577432+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:31.168031+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:08:52.791055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:09:13.658546+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:09:39.150360+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:10:00.601735+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-13T03:10:21.407555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:6801/873679308":"2023-09-14T03:13:09.492864+0000","172.21.15.64:0/1170163026":"2023-09-14T03:13:09.492864+0000","172.21.15.64:0/3771142265":"2023-09-14T03:13:09.492864+0000","172.21.15.64:0/4083942699":"2023-09-14T03:13:09.492864+0000","172.21.15.64:6800/2955227347":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/1373223118":"2023-09-14T03:04:22.355817+0000","172.21.15.64:6800/873679308":"2023-09-14T03:13:09.492864+0000","172.21.15.64:0/791645251":"2023-09-14T03:13:09.492864+0000","172.21.15.64:0/271194613":"2023-09-14T03:04:22.355817+0000","172.21.15.64:6801/3998091347":"2023-09-14T03:03:56.442541+0000","172.21.15.64:6800/3998091347":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/920818572":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/735789005":"2023-09-14T03:03:56.442541+0000","172.21.15.64:6801/2955227347":"2023-09-14T03:04:22.355817+0000","172.21.15.64:0/1975923178":"2023-09-14T03:03:56.442541+0000","172.21.15.64:0/1500151178":"2023-09-14T03:04:22.355817+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-13T03:29:51.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.674+0000 7fad871ff700 1 -- 172.21.15.64:0/3172393072 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fad700a07f0 msgr2=0x7fad700a2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:51.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.674+0000 7fad871ff700 1 --2- 172.21.15.64:0/3172393072 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fad700a07f0 0x7fad700a2cb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fad800740f0 tx=0x7fad6c00b5b0 comp rx=0 tx=0).stop 2023-09-13T03:29:51.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.674+0000 7fad871ff700 1 -- 172.21.15.64:0/3172393072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad80100590 msgr2=0x7fad80072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:51.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.674+0000 7fad871ff700 1 --2- 172.21.15.64:0/3172393072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad80100590 0x7fad80072900 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fad74009990 tx=0x7fad740057f0 comp rx=0 tx=0).stop 2023-09-13T03:29:51.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.674+0000 7fad871ff700 1 -- 172.21.15.64:0/3172393072 shutdown_connections 2023-09-13T03:29:51.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.674+0000 7fad871ff700 1 --2- 172.21.15.64:0/3172393072 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fad700a07f0 0x7fad700a2cb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:51.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.674+0000 7fad871ff700 1 --2- 172.21.15.64:0/3172393072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad80106650 0x7fad801a4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:51.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.674+0000 7fad871ff700 1 --2- 172.21.15.64:0/3172393072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad80100590 0x7fad80072900 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:51.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.674+0000 7fad871ff700 1 --2- 172.21.15.64:0/3172393072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad80100f50 0x7fad80072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:51.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.674+0000 7fad871ff700 1 -- 172.21.15.64:0/3172393072 >> 172.21.15.64:0/3172393072 conn(0x7fad800fc020 msgr2=0x7fad800fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:51.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.674+0000 7fad871ff700 1 -- 172.21.15.64:0/3172393072 shutdown_connections 2023-09-13T03:29:51.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.674+0000 7fad871ff700 1 -- 172.21.15.64:0/3172393072 wait complete. 2023-09-13T03:29:51.694 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-09-13T03:29:51.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.898+0000 7fedd15ff700 1 -- 172.21.15.64:0/2145398913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedcc106630 msgr2=0x7fedcc106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:51.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.898+0000 7fedd15ff700 1 --2- 172.21.15.64:0/2145398913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedcc106630 0x7fedcc106a10 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fedbc0099c0 tx=0x7fedbc009cd0 comp rx=0 tx=0).stop 2023-09-13T03:29:51.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.899+0000 7fedd15ff700 1 -- 172.21.15.64:0/2145398913 shutdown_connections 2023-09-13T03:29:51.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.899+0000 7fedd15ff700 1 --2- 172.21.15.64:0/2145398913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedcc100f30 0x7fedcc10d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:51.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.899+0000 7fedd15ff700 1 --2- 172.21.15.64:0/2145398913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedcc106630 0x7fedcc106a10 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:51.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.899+0000 7fedd15ff700 1 --2- 172.21.15.64:0/2145398913 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fedcc100570 0x7fedcc1009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:51.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.899+0000 7fedd15ff700 1 -- 172.21.15.64:0/2145398913 >> 172.21.15.64:0/2145398913 conn(0x7fedcc0fc080 msgr2=0x7fedcc0fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:51.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.899+0000 7fedd15ff700 1 -- 172.21.15.64:0/2145398913 shutdown_connections 2023-09-13T03:29:51.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.899+0000 7fedd15ff700 1 -- 172.21.15.64:0/2145398913 wait complete. 2023-09-13T03:29:51.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.900+0000 7fedd15ff700 1 Processor -- start 2023-09-13T03:29:51.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.900+0000 7fedd15ff700 1 -- start start 2023-09-13T03:29:51.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.900+0000 7fedd15ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedcc100570 0x7fedcc196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:51.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.900+0000 7fedd15ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fedcc100f30 0x7fedcc196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:51.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.900+0000 7fedd15ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedcc106630 0x7fedcc199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:51.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.900+0000 7fedd15ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc19a580 con 0x7fedcc100570 2023-09-13T03:29:51.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.900+0000 7fedd15ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc19a6f0 con 0x7fedcc106630 2023-09-13T03:29:51.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.900+0000 7fedd15ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc19a8b0 con 0x7fedcc100f30 2023-09-13T03:29:51.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.900+0000 7fedcaffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedcc100570 0x7fedcc196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:51.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.900+0000 7fedcaffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedcc100570 0x7fedcc196690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44332/0 (socket says 172.21.15.64:44332) 2023-09-13T03:29:51.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.900+0000 7fedcaffd700 1 -- 172.21.15.64:0/3494864482 learned_addr learned my addr 172.21.15.64:0/3494864482 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:29:51.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.901+0000 7fedca7fc700 1 --2- 172.21.15.64:0/3494864482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fedcc100f30 0x7fedcc196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:51.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.901+0000 7fedcb7fe700 1 --2- 172.21.15.64:0/3494864482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedcc106630 0x7fedcc199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:51.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.901+0000 7fedcaffd700 1 -- 172.21.15.64:0/3494864482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fedcc100f30 msgr2=0x7fedcc196bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:51.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.901+0000 7fedcaffd700 1 --2- 172.21.15.64:0/3494864482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fedcc100f30 0x7fedcc196bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:51.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.901+0000 7fedcaffd700 1 -- 172.21.15.64:0/3494864482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedcc106630 msgr2=0x7fedcc199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:51.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.901+0000 7fedcaffd700 1 --2- 172.21.15.64:0/3494864482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedcc106630 0x7fedcc199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:51.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.901+0000 7fedcaffd700 1 -- 172.21.15.64:0/3494864482 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedbc0095f0 con 0x7fedcc100570 2023-09-13T03:29:51.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.901+0000 7fedcb7fe700 1 --2- 172.21.15.64:0/3494864482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedcc106630 0x7fedcc199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-13T03:29:51.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.901+0000 7fedcaffd700 1 --2- 172.21.15.64:0/3494864482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedcc100570 0x7fedcc196690 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fedbc009990 tx=0x7fedbc00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:51.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.901+0000 7fedb3fff700 1 -- 172.21.15.64:0/3494864482 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc01d070 con 0x7fedcc100570 2023-09-13T03:29:51.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.901+0000 7fedb3fff700 1 -- 172.21.15.64:0/3494864482 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fedbc0054f0 con 0x7fedcc100570 2023-09-13T03:29:51.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.901+0000 7fedd15ff700 1 -- 172.21.15.64:0/3494864482 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedcc19ab30 con 0x7fedcc100570 2023-09-13T03:29:51.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.901+0000 7fedb3fff700 1 -- 172.21.15.64:0/3494864482 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc004ae0 con 0x7fedcc100570 2023-09-13T03:29:51.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.902+0000 7fedd15ff700 1 -- 172.21.15.64:0/3494864482 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedcc19ae00 con 0x7fedcc100570 2023-09-13T03:29:51.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.903+0000 7fedd15ff700 1 -- 172.21.15.64:0/3494864482 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedcc067380 con 0x7fedcc100570 2023-09-13T03:29:51.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:51 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/93095690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:29:51.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:51 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/2092670556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:29:51.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.904+0000 7fedb3fff700 1 -- 172.21.15.64:0/3494864482 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fedbc00fb80 con 0x7fedcc100570 2023-09-13T03:29:51.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.906+0000 7fedb3fff700 1 --2- 172.21.15.64:0/3494864482 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fedb40a07b0 0x7fedb40a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:51.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.906+0000 7fedb3fff700 1 -- 172.21.15.64:0/3494864482 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fedbc0ccd60 con 0x7fedcc100570 2023-09-13T03:29:51.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.906+0000 7fedca7fc700 1 --2- 172.21.15.64:0/3494864482 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fedb40a07b0 0x7fedb40a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:51.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.907+0000 7fedca7fc700 1 --2- 172.21.15.64:0/3494864482 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fedb40a07b0 0x7fedb40a2c70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fedcc197ca0 tx=0x7fedc00073d0 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:51.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:51.910+0000 7fedb3fff700 1 -- 172.21.15.64:0/3494864482 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fedbc096720 con 0x7fedcc100570 2023-09-13T03:29:51.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:51 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/93095690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:29:51.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:51 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2092670556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:29:51.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:51 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/93095690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:29:51.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:51 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/2092670556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-13T03:29:52.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.166+0000 7fedd15ff700 1 -- 172.21.15.64:0/3494864482 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fedcc110620 con 0x7fedcc100570 2023-09-13T03:29:52.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.168+0000 7fedb3fff700 1 -- 172.21.15.64:0/3494864482 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v84) v1 ==== 161+0+0 (secure 0 0 0) 0x7fedbc026090 con 0x7fedcc100570 2023-09-13T03:29:52.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.172+0000 7fedd15ff700 1 -- 172.21.15.64:0/3494864482 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fedb40a07b0 msgr2=0x7fedb40a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:52.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.172+0000 7fedd15ff700 1 --2- 172.21.15.64:0/3494864482 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fedb40a07b0 0x7fedb40a2c70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fedcc197ca0 tx=0x7fedc00073d0 comp rx=0 tx=0).stop 2023-09-13T03:29:52.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.172+0000 7fedd15ff700 1 -- 172.21.15.64:0/3494864482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedcc100570 msgr2=0x7fedcc196690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:52.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.172+0000 7fedd15ff700 1 --2- 172.21.15.64:0/3494864482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedcc100570 0x7fedcc196690 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fedbc009990 tx=0x7fedbc00f690 comp rx=0 tx=0).stop 2023-09-13T03:29:52.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.172+0000 7fedd15ff700 1 -- 172.21.15.64:0/3494864482 shutdown_connections 2023-09-13T03:29:52.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.172+0000 7fedd15ff700 1 --2- 172.21.15.64:0/3494864482 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7fedb40a07b0 0x7fedb40a2c70 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:52.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.172+0000 7fedd15ff700 1 --2- 172.21.15.64:0/3494864482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedcc106630 0x7fedcc199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:52.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.172+0000 7fedd15ff700 1 --2- 172.21.15.64:0/3494864482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedcc100570 0x7fedcc196690 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:52.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.172+0000 7fedd15ff700 1 --2- 172.21.15.64:0/3494864482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fedcc100f30 0x7fedcc196bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:52.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.172+0000 7fedd15ff700 1 -- 172.21.15.64:0/3494864482 >> 172.21.15.64:0/3494864482 conn(0x7fedcc0fc080 msgr2=0x7fedcc0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:52.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.173+0000 7fedd15ff700 1 -- 172.21.15.64:0/3494864482 shutdown_connections 2023-09-13T03:29:52.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.173+0000 7fedd15ff700 1 -- 172.21.15.64:0/3494864482 wait complete. 2023-09-13T03:29:52.176 INFO:teuthology.orchestra.run.smithi064.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-09-13T03:29:52.194 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-09-13T03:29:52.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.415+0000 7f323ddff700 1 -- 172.21.15.64:0/3700876804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3238106650 msgr2=0x7f3238106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:52.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.415+0000 7f323ddff700 1 --2- 172.21.15.64:0/3700876804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3238106650 0x7f3238106a30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f32280099c0 tx=0x7f3228009cd0 comp rx=0 tx=0).stop 2023-09-13T03:29:52.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.415+0000 7f323ddff700 1 -- 172.21.15.64:0/3700876804 shutdown_connections 2023-09-13T03:29:52.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.415+0000 7f323ddff700 1 --2- 172.21.15.64:0/3700876804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3238106650 0x7f3238106a30 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:52.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.415+0000 7f323ddff700 1 --2- 172.21.15.64:0/3700876804 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3238100f50 0x7f323810d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:52.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.415+0000 7f323ddff700 1 --2- 172.21.15.64:0/3700876804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3238100590 0x7f3238100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:52.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.415+0000 7f323ddff700 1 -- 172.21.15.64:0/3700876804 >> 172.21.15.64:0/3700876804 conn(0x7f32380fc060 msgr2=0x7f32380fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:52.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.415+0000 7f323ddff700 1 -- 172.21.15.64:0/3700876804 shutdown_connections 2023-09-13T03:29:52.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.415+0000 7f323ddff700 1 -- 172.21.15.64:0/3700876804 wait complete. 2023-09-13T03:29:52.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.416+0000 7f323ddff700 1 Processor -- start 2023-09-13T03:29:52.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.416+0000 7f323ddff700 1 -- start start 2023-09-13T03:29:52.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.417+0000 7f323ddff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3238100590 0x7f3238072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:52.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.417+0000 7f323ddff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3238100f50 0x7f3238072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:52.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.417+0000 7f323ddff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3238106650 0x7f32381a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:52.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.417+0000 7f323ddff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3238073670 con 0x7f3238100590 2023-09-13T03:29:52.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.417+0000 7f323ddff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32381a4d30 con 0x7f3238100f50 2023-09-13T03:29:52.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.417+0000 7f323ddff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32381a4e70 con 0x7f3238106650 2023-09-13T03:29:52.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.417+0000 7f32377fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3238100590 0x7f3238072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:52.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.417+0000 7f32377fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3238100590 0x7f3238072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44384/0 (socket says 172.21.15.64:44384) 2023-09-13T03:29:52.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.417+0000 7f32377fe700 1 -- 172.21.15.64:0/1339447554 learned_addr learned my addr 172.21.15.64:0/1339447554 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:29:52.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.417+0000 7f3237fff700 1 --2- 172.21.15.64:0/1339447554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3238106650 0x7f32381a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:52.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.417+0000 7f3236ffd700 1 --2- 172.21.15.64:0/1339447554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3238100f50 0x7f3238072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:52.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.417+0000 7f32377fe700 1 -- 172.21.15.64:0/1339447554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3238106650 msgr2=0x7f32381a47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:52.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.417+0000 7f32377fe700 1 --2- 172.21.15.64:0/1339447554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3238106650 0x7f32381a47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:52.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.417+0000 7f32377fe700 1 -- 172.21.15.64:0/1339447554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3238100f50 msgr2=0x7f3238072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:52.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.417+0000 7f32377fe700 1 --2- 172.21.15.64:0/1339447554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3238100f50 0x7f3238072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:52.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.417+0000 7f32377fe700 1 -- 172.21.15.64:0/1339447554 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32280095f0 con 0x7f3238100590 2023-09-13T03:29:52.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.418+0000 7f32377fe700 1 --2- 172.21.15.64:0/1339447554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3238100590 0x7f3238072900 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f322800b580 tx=0x7f322800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:52.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.418+0000 7f3234ff9700 1 -- 172.21.15.64:0/1339447554 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322801d070 con 0x7f3238100590 2023-09-13T03:29:52.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.418+0000 7f323ddff700 1 -- 172.21.15.64:0/1339447554 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32381a5070 con 0x7f3238100590 2023-09-13T03:29:52.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.418+0000 7f3234ff9700 1 -- 172.21.15.64:0/1339447554 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3228022470 con 0x7f3238100590 2023-09-13T03:29:52.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.418+0000 7f3234ff9700 1 -- 172.21.15.64:0/1339447554 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3228017650 con 0x7f3238100590 2023-09-13T03:29:52.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.418+0000 7f323ddff700 1 -- 172.21.15.64:0/1339447554 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32381a5490 con 0x7f3238100590 2023-09-13T03:29:52.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.419+0000 7f323ddff700 1 -- 172.21.15.64:0/1339447554 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3238067380 con 0x7f3238100590 2023-09-13T03:29:52.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.421+0000 7f3234ff9700 1 -- 172.21.15.64:0/1339447554 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f32280225e0 con 0x7f3238100590 2023-09-13T03:29:52.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.422+0000 7f3234ff9700 1 --2- 172.21.15.64:0/1339447554 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f32200a07b0 0x7f32200a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:52.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.422+0000 7f3234ff9700 1 -- 172.21.15.64:0/1339447554 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8870+0+0 (secure 0 0 0) 0x7f32280ccfe0 con 0x7f3238100590 2023-09-13T03:29:52.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.423+0000 7f3236ffd700 1 --2- 172.21.15.64:0/1339447554 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f32200a07b0 0x7f32200a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:52.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.423+0000 7f3236ffd700 1 --2- 172.21.15.64:0/1339447554 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f32200a07b0 0x7f32200a2c70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f32380740f0 tx=0x7f322c009260 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:52.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.426+0000 7f3234ff9700 1 -- 172.21.15.64:0/1339447554 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f3228096700 con 0x7f3238100590 2023-09-13T03:29:52.495 INFO:journalctl@ceph.mds.k.smithi142.stdout:Sep 13 03:29:52 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-k[169640]: ignoring --setuser ceph since I am not root 2023-09-13T03:29:52.496 INFO:journalctl@ceph.mds.k.smithi142.stdout:Sep 13 03:29:52 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-k[169640]: ignoring --setgroup ceph since I am not root 2023-09-13T03:29:52.496 INFO:journalctl@ceph.mds.k.smithi142.stdout:Sep 13 03:29:52 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-k[169640]: starting mds.k at 2023-09-13T03:29:52.497 INFO:journalctl@ceph.mds.b.smithi142.stdout:Sep 13 03:29:52 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-b[170662]: ignoring --setuser ceph since I am not root 2023-09-13T03:29:52.497 INFO:journalctl@ceph.mds.b.smithi142.stdout:Sep 13 03:29:52 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-b[170662]: ignoring --setgroup ceph since I am not root 2023-09-13T03:29:52.497 INFO:journalctl@ceph.mds.b.smithi142.stdout:Sep 13 03:29:52 smithi142 ceph-mds[170686]: -- 172.21.15.142:0/119804498 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555f07ad11e0 con 0x555f07aa7000 2023-09-13T03:29:52.498 INFO:journalctl@ceph.mds.b.smithi142.stdout:Sep 13 03:29:52 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-b[170662]: starting mds.b at 2023-09-13T03:29:52.498 INFO:journalctl@ceph.mds.e.smithi142.stdout:Sep 13 03:29:52 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-e[169981]: ignoring --setuser ceph since I am not root 2023-09-13T03:29:52.499 INFO:journalctl@ceph.mds.e.smithi142.stdout:Sep 13 03:29:52 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-e[169981]: ignoring --setgroup ceph since I am not root 2023-09-13T03:29:52.499 INFO:journalctl@ceph.mds.e.smithi142.stdout:Sep 13 03:29:52 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-e[169981]: starting mds.e at 2023-09-13T03:29:52.499 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:29:52 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-h[170323]: ignoring --setuser ceph since I am not root 2023-09-13T03:29:52.500 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:29:52 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-h[170323]: ignoring --setgroup ceph since I am not root 2023-09-13T03:29:52.500 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:29:52 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-h[170323]: starting mds.h at 2023-09-13T03:29:52.556 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:52 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mon-a[97651]: 2023-09-13T03:29:52.166+0000 7f9fd4134700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-13T03:29:52.556 INFO:journalctl@ceph.mds.c.smithi170.stdout:Sep 13 03:29:52 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-c[171964]: ignoring --setuser ceph since I am not root 2023-09-13T03:29:52.557 INFO:journalctl@ceph.mds.c.smithi170.stdout:Sep 13 03:29:52 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-c[171964]: ignoring --setgroup ceph since I am not root 2023-09-13T03:29:52.557 INFO:journalctl@ceph.mds.c.smithi170.stdout:Sep 13 03:29:52 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-c[171964]: starting mds.c at 2023-09-13T03:29:52.558 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 13 03:29:52 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-g[152178]: ignoring --setuser ceph since I am not root 2023-09-13T03:29:52.558 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 13 03:29:52 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-g[152178]: ignoring --setgroup ceph since I am not root 2023-09-13T03:29:52.558 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 13 03:29:52 smithi064 ceph-mds[152220]: -- 172.21.15.64:0/2160332988 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5650e2a591e0 con 0x5650e2a2e800 2023-09-13T03:29:52.559 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 13 03:29:52 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-g[152178]: starting mds.g at 2023-09-13T03:29:52.559 INFO:journalctl@ceph.mds.f.smithi170.stdout:Sep 13 03:29:52 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-f[171621]: ignoring --setuser ceph since I am not root 2023-09-13T03:29:52.559 INFO:journalctl@ceph.mds.f.smithi170.stdout:Sep 13 03:29:52 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-f[171621]: ignoring --setgroup ceph since I am not root 2023-09-13T03:29:52.560 INFO:journalctl@ceph.mds.f.smithi170.stdout:Sep 13 03:29:52 smithi170 ceph-mds[171645]: -- 172.21.15.170:0/2861782291 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55edea32d1e0 con 0x55edea302800 2023-09-13T03:29:52.560 INFO:journalctl@ceph.mds.f.smithi170.stdout:Sep 13 03:29:52 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-f[171621]: starting mds.f at 2023-09-13T03:29:52.561 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 13 03:29:52 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-a[152587]: ignoring --setuser ceph since I am not root 2023-09-13T03:29:52.561 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 13 03:29:52 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-a[152587]: ignoring --setgroup ceph since I am not root 2023-09-13T03:29:52.561 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 13 03:29:52 smithi064 ceph-mds[152612]: -- 172.21.15.64:0/3130895258 <== mon.2 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560fd5d251e0 con 0x560fd5cfb000 2023-09-13T03:29:52.562 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 13 03:29:52 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-a[152587]: starting mds.a at 2023-09-13T03:29:52.562 INFO:journalctl@ceph.mds.i.smithi170.stdout:Sep 13 03:29:52 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-i[172301]: ignoring --setuser ceph since I am not root 2023-09-13T03:29:52.562 INFO:journalctl@ceph.mds.i.smithi170.stdout:Sep 13 03:29:52 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-i[172301]: ignoring --setgroup ceph since I am not root 2023-09-13T03:29:52.563 INFO:journalctl@ceph.mds.i.smithi170.stdout:Sep 13 03:29:52 smithi170 ceph-mds[172326]: -- 172.21.15.170:0/2100491116 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5628074751e0 con 0x56280744a800 2023-09-13T03:29:52.563 INFO:journalctl@ceph.mds.i.smithi170.stdout:Sep 13 03:29:52 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-i[172301]: starting mds.i at 2023-09-13T03:29:52.563 INFO:journalctl@ceph.mds.l.smithi170.stdout:Sep 13 03:29:52 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-l[171280]: ignoring --setuser ceph since I am not root 2023-09-13T03:29:52.564 INFO:journalctl@ceph.mds.l.smithi170.stdout:Sep 13 03:29:52 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-l[171280]: ignoring --setgroup ceph since I am not root 2023-09-13T03:29:52.564 INFO:journalctl@ceph.mds.l.smithi170.stdout:Sep 13 03:29:52 smithi170 ceph-mds[171305]: -- 172.21.15.170:0/952001136 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56389ec5b1e0 con 0x56389ec30800 2023-09-13T03:29:52.564 INFO:journalctl@ceph.mds.l.smithi170.stdout:Sep 13 03:29:52 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-l[171280]: starting mds.l at 2023-09-13T03:29:52.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:52.692+0000 7f323ddff700 1 -- 172.21.15.64:0/1339447554 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3238110700 con 0x7f3238100590 2023-09-13T03:29:52.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:52 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/3172393072' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:29:52.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:52 smithi170 ceph-mon[134328]: pgmap v524: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 1.9 MiB/s wr, 115 op/s 2023-09-13T03:29:52.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:52 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/3494864482' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-09-13T03:29:52.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:52 smithi170 ceph-mon[134328]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-09-13T03:29:52.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:52 smithi170 ceph-mon[134328]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-13T03:29:52.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:52 smithi170 ceph-mon[134328]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2023-09-13T03:29:52.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:52 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/3494864482' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-09-13T03:29:52.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:52 smithi170 ceph-mon[134328]: osdmap e107: 12 total, 12 up, 12 in 2023-09-13T03:29:52.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:52 smithi170 ceph-mon[134328]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-09-13T03:29:52.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:52 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/3172393072' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:29:52.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:52 smithi064 ceph-mon[97675]: pgmap v524: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 1.9 MiB/s wr, 115 op/s 2023-09-13T03:29:52.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:52 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/3494864482' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-09-13T03:29:52.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:52 smithi064 ceph-mon[97675]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-09-13T03:29:52.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:52 smithi064 ceph-mon[97675]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-13T03:29:52.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:52 smithi064 ceph-mon[97675]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2023-09-13T03:29:52.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:52 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/3494864482' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-09-13T03:29:52.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:52 smithi064 ceph-mon[97675]: osdmap e107: 12 total, 12 up, 12 in 2023-09-13T03:29:52.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:52 smithi064 ceph-mon[97675]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-09-13T03:29:52.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:52 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/3172393072' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-13T03:29:52.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:52 smithi142 ceph-mon[134082]: pgmap v524: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 1.9 MiB/s wr, 115 op/s 2023-09-13T03:29:52.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:52 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/3494864482' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-09-13T03:29:52.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:52 smithi142 ceph-mon[134082]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-09-13T03:29:52.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:52 smithi142 ceph-mon[134082]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-13T03:29:52.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:52 smithi142 ceph-mon[134082]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2023-09-13T03:29:52.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:52 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/3494864482' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-09-13T03:29:52.998 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:52 smithi142 ceph-mon[134082]: osdmap e107: 12 total, 12 up, 12 in 2023-09-13T03:29:52.998 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:52 smithi142 ceph-mon[134082]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-09-13T03:29:53.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.170+0000 7f3234ff9700 1 -- 172.21.15.64:0/1339447554 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v85) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3228099d20 con 0x7f3238100590 2023-09-13T03:29:53.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.173+0000 7f323ddff700 1 -- 172.21.15.64:0/1339447554 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f32200a07b0 msgr2=0x7f32200a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:53.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.173+0000 7f323ddff700 1 --2- 172.21.15.64:0/1339447554 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f32200a07b0 0x7f32200a2c70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f32380740f0 tx=0x7f322c009260 comp rx=0 tx=0).stop 2023-09-13T03:29:53.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.173+0000 7f323ddff700 1 -- 172.21.15.64:0/1339447554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3238100590 msgr2=0x7f3238072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:53.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.173+0000 7f323ddff700 1 --2- 172.21.15.64:0/1339447554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3238100590 0x7f3238072900 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f322800b580 tx=0x7f322800bfd0 comp rx=0 tx=0).stop 2023-09-13T03:29:53.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.173+0000 7f323ddff700 1 -- 172.21.15.64:0/1339447554 shutdown_connections 2023-09-13T03:29:53.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.173+0000 7f323ddff700 1 --2- 172.21.15.64:0/1339447554 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f32200a07b0 0x7f32200a2c70 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:53.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.173+0000 7f323ddff700 1 --2- 172.21.15.64:0/1339447554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3238106650 0x7f32381a47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:53.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.173+0000 7f323ddff700 1 --2- 172.21.15.64:0/1339447554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3238100590 0x7f3238072900 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:53.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.173+0000 7f323ddff700 1 --2- 172.21.15.64:0/1339447554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3238100f50 0x7f3238072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:53.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.173+0000 7f323ddff700 1 -- 172.21.15.64:0/1339447554 >> 172.21.15.64:0/1339447554 conn(0x7f32380fc060 msgr2=0x7f3238104220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:53.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.174+0000 7f323ddff700 1 -- 172.21.15.64:0/1339447554 shutdown_connections 2023-09-13T03:29:53.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.174+0000 7f323ddff700 1 -- 172.21.15.64:0/1339447554 wait complete. 2023-09-13T03:29:53.199 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-09-13T03:29:53.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.407+0000 7f80973ff700 1 -- 172.21.15.64:0/1215143046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8090106650 msgr2=0x7f8090106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:53.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.407+0000 7f80973ff700 1 --2- 172.21.15.64:0/1215143046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8090106650 0x7f8090106a30 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f808c0099c0 tx=0x7f808c009cd0 comp rx=0 tx=0).stop 2023-09-13T03:29:53.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.407+0000 7f80973ff700 1 -- 172.21.15.64:0/1215143046 shutdown_connections 2023-09-13T03:29:53.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.407+0000 7f80973ff700 1 --2- 172.21.15.64:0/1215143046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8090100f50 0x7f809010d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:53.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.407+0000 7f80973ff700 1 --2- 172.21.15.64:0/1215143046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8090106650 0x7f8090106a30 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:53.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.407+0000 7f80973ff700 1 --2- 172.21.15.64:0/1215143046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8090100590 0x7f8090100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:53.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.407+0000 7f80973ff700 1 -- 172.21.15.64:0/1215143046 >> 172.21.15.64:0/1215143046 conn(0x7f80900fc060 msgr2=0x7f80900fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:53.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.407+0000 7f80973ff700 1 -- 172.21.15.64:0/1215143046 shutdown_connections 2023-09-13T03:29:53.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.407+0000 7f80973ff700 1 -- 172.21.15.64:0/1215143046 wait complete. 2023-09-13T03:29:53.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.408+0000 7f80973ff700 1 Processor -- start 2023-09-13T03:29:53.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.408+0000 7f80973ff700 1 -- start start 2023-09-13T03:29:53.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.409+0000 7f80973ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8090100590 0x7f8090072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:53.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.409+0000 7f80973ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8090100f50 0x7f8090072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:53.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.409+0000 7f80973ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8090106650 0x7f8090073380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:53.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.409+0000 7f80973ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80901a4a50 con 0x7f8090100590 2023-09-13T03:29:53.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.409+0000 7f80973ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80901a4bc0 con 0x7f8090100f50 2023-09-13T03:29:53.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.409+0000 7f80973ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80901a4d80 con 0x7f8090106650 2023-09-13T03:29:53.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.409+0000 7f809519b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8090100590 0x7f8090072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:53.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.409+0000 7f809519b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8090100590 0x7f8090072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44402/0 (socket says 172.21.15.64:44402) 2023-09-13T03:29:53.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.409+0000 7f809519b700 1 -- 172.21.15.64:0/2592892323 learned_addr learned my addr 172.21.15.64:0/2592892323 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:29:53.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.409+0000 7f809499a700 1 --2- 172.21.15.64:0/2592892323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8090100f50 0x7f8090072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:53.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.409+0000 7f809599c700 1 --2- 172.21.15.64:0/2592892323 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8090106650 0x7f8090073380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:53.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.409+0000 7f809519b700 1 -- 172.21.15.64:0/2592892323 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8090106650 msgr2=0x7f8090073380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:53.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.409+0000 7f809519b700 1 --2- 172.21.15.64:0/2592892323 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8090106650 0x7f8090073380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:53.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.409+0000 7f809519b700 1 -- 172.21.15.64:0/2592892323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8090100f50 msgr2=0x7f8090072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:53.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.409+0000 7f809519b700 1 --2- 172.21.15.64:0/2592892323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8090100f50 0x7f8090072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:53.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.409+0000 7f809519b700 1 -- 172.21.15.64:0/2592892323 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f808c0095f0 con 0x7f8090100590 2023-09-13T03:29:53.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.409+0000 7f809499a700 1 --2- 172.21.15.64:0/2592892323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8090100f50 0x7f8090072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-13T03:29:53.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.410+0000 7f809519b700 1 --2- 172.21.15.64:0/2592892323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8090100590 0x7f8090072900 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f808c0098e0 tx=0x7f808c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:53.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.410+0000 7f80867fc700 1 -- 172.21.15.64:0/2592892323 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808c01d070 con 0x7f8090100590 2023-09-13T03:29:53.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.410+0000 7f80867fc700 1 -- 172.21.15.64:0/2592892323 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f808c022470 con 0x7f8090100590 2023-09-13T03:29:53.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.410+0000 7f80973ff700 1 -- 172.21.15.64:0/2592892323 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80901a5000 con 0x7f8090100590 2023-09-13T03:29:53.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.410+0000 7f80867fc700 1 -- 172.21.15.64:0/2592892323 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808c017650 con 0x7f8090100590 2023-09-13T03:29:53.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.410+0000 7f80973ff700 1 -- 172.21.15.64:0/2592892323 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80901a5420 con 0x7f8090100590 2023-09-13T03:29:53.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.411+0000 7f80973ff700 1 -- 172.21.15.64:0/2592892323 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8090067380 con 0x7f8090100590 2023-09-13T03:29:53.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.413+0000 7f80867fc700 1 -- 172.21.15.64:0/2592892323 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f808c022850 con 0x7f8090100590 2023-09-13T03:29:53.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.414+0000 7f80867fc700 1 --2- 172.21.15.64:0/2592892323 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f807c0a07b0 0x7f807c0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:53.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.415+0000 7f80867fc700 1 -- 172.21.15.64:0/2592892323 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8870+0+0 (secure 0 0 0) 0x7f808c0ccc20 con 0x7f8090100590 2023-09-13T03:29:53.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.415+0000 7f809499a700 1 --2- 172.21.15.64:0/2592892323 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f807c0a07b0 0x7f807c0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:53.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.416+0000 7f809499a700 1 --2- 172.21.15.64:0/2592892323 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f807c0a07b0 0x7f807c0a2c70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f80900740a0 tx=0x7f8080009260 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:53.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.419+0000 7f80867fc700 1 -- 172.21.15.64:0/2592892323 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f808c0963f0 con 0x7f8090100590 2023-09-13T03:29:53.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:53.631+0000 7f80973ff700 1 -- 172.21.15.64:0/2592892323 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8090073de0 con 0x7f8090100590 2023-09-13T03:29:53.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:53 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/1339447554' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-09-13T03:29:53.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:53 smithi170 ceph-mon[134328]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-09-13T03:29:53.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:53 smithi170 ceph-mon[134328]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-09-13T03:29:53.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:53 smithi170 ceph-mon[134328]: Cluster is now healthy 2023-09-13T03:29:53.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:53 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/1339447554' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-09-13T03:29:53.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:53 smithi170 ceph-mon[134328]: mds.? [v2:172.21.15.142:6838/2930351881,v1:172.21.15.142:6839/2930351881] up:boot 2023-09-13T03:29:53.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:53 smithi170 ceph-mon[134328]: mds.? [v2:172.21.15.170:6835/3555101774,v1:172.21.15.170:6838/3555101774] up:boot 2023-09-13T03:29:53.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:53 smithi170 ceph-mon[134328]: mds.? [v2:172.21.15.170:6836/3989877943,v1:172.21.15.170:6839/3989877943] up:boot 2023-09-13T03:29:53.913 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:53 smithi170 ceph-mon[134328]: mds.? [v2:172.21.15.142:6832/2340694862,v1:172.21.15.142:6834/2340694862] up:boot 2023-09-13T03:29:53.913 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:53 smithi170 ceph-mon[134328]: mds.? [v2:172.21.15.142:6836/3241697726,v1:172.21.15.142:6837/3241697726] up:boot 2023-09-13T03:29:53.913 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:53 smithi170 ceph-mon[134328]: mds.? [v2:172.21.15.142:6833/3818381154,v1:172.21.15.142:6835/3818381154] up:boot 2023-09-13T03:29:53.914 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:53 smithi170 ceph-mon[134328]: mds.? [v2:172.21.15.170:6832/2542078740,v1:172.21.15.170:6833/2542078740] up:boot 2023-09-13T03:29:53.914 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:53 smithi170 ceph-mon[134328]: mds.? [v2:172.21.15.170:6834/2027564550,v1:172.21.15.170:6837/2027564550] up:boot 2023-09-13T03:29:53.914 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:53 smithi170 ceph-mon[134328]: mds.? [v2:172.21.15.64:6838/2484770868,v1:172.21.15.64:6839/2484770868] up:boot 2023-09-13T03:29:53.914 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:53 smithi170 ceph-mon[134328]: mds.? [v2:172.21.15.64:6840/3487381289,v1:172.21.15.64:6841/3487381289] up:boot 2023-09-13T03:29:53.915 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:53 smithi170 ceph-mon[134328]: fsmap 12 up:standby 2023-09-13T03:29:53.915 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:53 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-13T03:29:53.915 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:53 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-13T03:29:53.916 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:53 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-13T03:29:53.916 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:53 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-13T03:29:53.916 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:53 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-13T03:29:53.916 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:53 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-13T03:29:53.917 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:53 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-13T03:29:53.917 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:53 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-13T03:29:53.917 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:53 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-13T03:29:53.917 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:53 smithi170 ceph-mon[134328]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-13T03:29:53.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:53 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/1339447554' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-09-13T03:29:53.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:53 smithi064 ceph-mon[97675]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-09-13T03:29:53.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:53 smithi064 ceph-mon[97675]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-09-13T03:29:53.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:53 smithi064 ceph-mon[97675]: Cluster is now healthy 2023-09-13T03:29:53.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:53 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/1339447554' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-09-13T03:29:53.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:53 smithi064 ceph-mon[97675]: mds.? [v2:172.21.15.142:6838/2930351881,v1:172.21.15.142:6839/2930351881] up:boot 2023-09-13T03:29:53.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:53 smithi064 ceph-mon[97675]: mds.? [v2:172.21.15.170:6835/3555101774,v1:172.21.15.170:6838/3555101774] up:boot 2023-09-13T03:29:53.920 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:53 smithi064 ceph-mon[97675]: mds.? [v2:172.21.15.170:6836/3989877943,v1:172.21.15.170:6839/3989877943] up:boot 2023-09-13T03:29:53.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:53 smithi064 ceph-mon[97675]: mds.? [v2:172.21.15.142:6832/2340694862,v1:172.21.15.142:6834/2340694862] up:boot 2023-09-13T03:29:53.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:53 smithi064 ceph-mon[97675]: mds.? [v2:172.21.15.142:6836/3241697726,v1:172.21.15.142:6837/3241697726] up:boot 2023-09-13T03:29:53.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:53 smithi064 ceph-mon[97675]: mds.? [v2:172.21.15.142:6833/3818381154,v1:172.21.15.142:6835/3818381154] up:boot 2023-09-13T03:29:53.921 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:53 smithi064 ceph-mon[97675]: mds.? [v2:172.21.15.170:6832/2542078740,v1:172.21.15.170:6833/2542078740] up:boot 2023-09-13T03:29:53.922 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:53 smithi064 ceph-mon[97675]: mds.? [v2:172.21.15.170:6834/2027564550,v1:172.21.15.170:6837/2027564550] up:boot 2023-09-13T03:29:53.922 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:53 smithi064 ceph-mon[97675]: mds.? [v2:172.21.15.64:6838/2484770868,v1:172.21.15.64:6839/2484770868] up:boot 2023-09-13T03:29:53.922 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:53 smithi064 ceph-mon[97675]: mds.? [v2:172.21.15.64:6840/3487381289,v1:172.21.15.64:6841/3487381289] up:boot 2023-09-13T03:29:53.923 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:53 smithi064 ceph-mon[97675]: fsmap 12 up:standby 2023-09-13T03:29:53.923 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:53 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-13T03:29:53.923 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:53 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-13T03:29:53.923 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:53 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-13T03:29:53.924 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:53 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-13T03:29:53.924 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:53 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-13T03:29:53.924 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:53 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-13T03:29:53.925 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:53 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-13T03:29:53.925 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:53 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-13T03:29:53.925 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:53 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-13T03:29:53.925 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:53 smithi064 ceph-mon[97675]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-13T03:29:53.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:53 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/1339447554' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-09-13T03:29:53.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:53 smithi142 ceph-mon[134082]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-09-13T03:29:53.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:53 smithi142 ceph-mon[134082]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-09-13T03:29:53.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:53 smithi142 ceph-mon[134082]: Cluster is now healthy 2023-09-13T03:29:53.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:53 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/1339447554' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-09-13T03:29:53.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:53 smithi142 ceph-mon[134082]: mds.? [v2:172.21.15.142:6838/2930351881,v1:172.21.15.142:6839/2930351881] up:boot 2023-09-13T03:29:53.997 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:53 smithi142 ceph-mon[134082]: mds.? [v2:172.21.15.170:6835/3555101774,v1:172.21.15.170:6838/3555101774] up:boot 2023-09-13T03:29:53.998 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:53 smithi142 ceph-mon[134082]: mds.? [v2:172.21.15.170:6836/3989877943,v1:172.21.15.170:6839/3989877943] up:boot 2023-09-13T03:29:53.998 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:53 smithi142 ceph-mon[134082]: mds.? [v2:172.21.15.142:6832/2340694862,v1:172.21.15.142:6834/2340694862] up:boot 2023-09-13T03:29:53.998 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:53 smithi142 ceph-mon[134082]: mds.? [v2:172.21.15.142:6836/3241697726,v1:172.21.15.142:6837/3241697726] up:boot 2023-09-13T03:29:53.998 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:53 smithi142 ceph-mon[134082]: mds.? [v2:172.21.15.142:6833/3818381154,v1:172.21.15.142:6835/3818381154] up:boot 2023-09-13T03:29:53.999 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:53 smithi142 ceph-mon[134082]: mds.? [v2:172.21.15.170:6832/2542078740,v1:172.21.15.170:6833/2542078740] up:boot 2023-09-13T03:29:53.999 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:53 smithi142 ceph-mon[134082]: mds.? [v2:172.21.15.170:6834/2027564550,v1:172.21.15.170:6837/2027564550] up:boot 2023-09-13T03:29:53.999 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:53 smithi142 ceph-mon[134082]: mds.? [v2:172.21.15.64:6838/2484770868,v1:172.21.15.64:6839/2484770868] up:boot 2023-09-13T03:29:54.000 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:53 smithi142 ceph-mon[134082]: mds.? [v2:172.21.15.64:6840/3487381289,v1:172.21.15.64:6841/3487381289] up:boot 2023-09-13T03:29:54.000 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:53 smithi142 ceph-mon[134082]: fsmap 12 up:standby 2023-09-13T03:29:54.000 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:53 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-13T03:29:54.001 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:53 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-13T03:29:54.001 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:53 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-13T03:29:54.001 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:53 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-13T03:29:54.002 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:53 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-13T03:29:54.002 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:53 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-13T03:29:54.002 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:53 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-13T03:29:54.002 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:53 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-13T03:29:54.003 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:53 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-13T03:29:54.003 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:53 smithi142 ceph-mon[134082]: from='mgr.15507 172.21.15.64:0/2897092240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-13T03:29:54.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.562+0000 7f80867fc700 1 -- 172.21.15.64:0/2592892323 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v108) v1 ==== 181+0+0 (secure 0 0 0) 0x7f808c099a10 con 0x7f8090100590 2023-09-13T03:29:54.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.566+0000 7f80973ff700 1 -- 172.21.15.64:0/2592892323 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f807c0a07b0 msgr2=0x7f807c0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:54.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.566+0000 7f80973ff700 1 --2- 172.21.15.64:0/2592892323 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f807c0a07b0 0x7f807c0a2c70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f80900740a0 tx=0x7f8080009260 comp rx=0 tx=0).stop 2023-09-13T03:29:54.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.566+0000 7f80973ff700 1 -- 172.21.15.64:0/2592892323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8090100590 msgr2=0x7f8090072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:54.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.566+0000 7f80973ff700 1 --2- 172.21.15.64:0/2592892323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8090100590 0x7f8090072900 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f808c0098e0 tx=0x7f808c00f690 comp rx=0 tx=0).stop 2023-09-13T03:29:54.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.567+0000 7f80973ff700 1 -- 172.21.15.64:0/2592892323 shutdown_connections 2023-09-13T03:29:54.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.567+0000 7f80973ff700 1 --2- 172.21.15.64:0/2592892323 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f807c0a07b0 0x7f807c0a2c70 secure :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f80900740a0 tx=0x7f8080009260 comp rx=0 tx=0).stop 2023-09-13T03:29:54.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.567+0000 7f80973ff700 1 --2- 172.21.15.64:0/2592892323 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8090106650 0x7f8090073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:54.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.567+0000 7f80973ff700 1 --2- 172.21.15.64:0/2592892323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8090100590 0x7f8090072900 secure :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f808c0098e0 tx=0x7f808c00f690 comp rx=0 tx=0).stop 2023-09-13T03:29:54.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.567+0000 7f80973ff700 1 --2- 172.21.15.64:0/2592892323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8090100f50 0x7f8090072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:54.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.567+0000 7f80973ff700 1 -- 172.21.15.64:0/2592892323 >> 172.21.15.64:0/2592892323 conn(0x7f80900fc060 msgr2=0x7f8090104220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:54.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.569+0000 7f80973ff700 1 -- 172.21.15.64:0/2592892323 shutdown_connections 2023-09-13T03:29:54.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.571+0000 7f80973ff700 1 -- 172.21.15.64:0/2592892323 wait complete. 2023-09-13T03:29:54.581 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' removed 2023-09-13T03:29:54.603 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-09-13T03:29:54.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.870+0000 7f92fdfff700 1 -- 172.21.15.64:0/3244015533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92f80ffe20 msgr2=0x7f92f81042e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:54.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.870+0000 7f92fdfff700 1 --2- 172.21.15.64:0/3244015533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92f80ffe20 0x7f92f81042e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f92ec009990 tx=0x7f92ec009ca0 comp rx=0 tx=0).stop 2023-09-13T03:29:54.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.870+0000 7f92fdfff700 1 -- 172.21.15.64:0/3244015533 shutdown_connections 2023-09-13T03:29:54.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.870+0000 7f92fdfff700 1 --2- 172.21.15.64:0/3244015533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92f8104900 0x7f92f8104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:54.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.870+0000 7f92fdfff700 1 --2- 172.21.15.64:0/3244015533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92f80ff500 0x7f92f80ff8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:54.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.870+0000 7f92fdfff700 1 --2- 172.21.15.64:0/3244015533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92f80ffe20 0x7f92f81042e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:54.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.870+0000 7f92fdfff700 1 -- 172.21.15.64:0/3244015533 >> 172.21.15.64:0/3244015533 conn(0x7f92f80faf50 msgr2=0x7f92f80fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:54.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.870+0000 7f92fdfff700 1 -- 172.21.15.64:0/3244015533 shutdown_connections 2023-09-13T03:29:54.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.871+0000 7f92fdfff700 1 -- 172.21.15.64:0/3244015533 wait complete. 2023-09-13T03:29:54.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.871+0000 7f92fdfff700 1 Processor -- start 2023-09-13T03:29:54.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.872+0000 7f92fdfff700 1 -- start start 2023-09-13T03:29:54.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.872+0000 7f92fdfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92f80ff500 0x7f92f81a2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:54.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.872+0000 7f92fdfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92f80ffe20 0x7f92f81a3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:54.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.872+0000 7f92fdfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92f8104900 0x7f92f819cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:54.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.872+0000 7f92fdfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f819d1a0 con 0x7f92f80ff500 2023-09-13T03:29:54.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.872+0000 7f92fdfff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f819d310 con 0x7f92f8104900 2023-09-13T03:29:54.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.872+0000 7f92fdfff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f819d4d0 con 0x7f92f80ffe20 2023-09-13T03:29:54.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.872+0000 7f92f77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92f80ff500 0x7f92f81a2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:54.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.872+0000 7f92f6ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92f80ffe20 0x7f92f81a3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:54.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.872+0000 7f92f6ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92f80ffe20 0x7f92f81a3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.64:36634/0 (socket says 172.21.15.64:36634) 2023-09-13T03:29:54.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.872+0000 7f92f6ffd700 1 -- 172.21.15.64:0/2042028416 learned_addr learned my addr 172.21.15.64:0/2042028416 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-09-13T03:29:54.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.873+0000 7f92f7fff700 1 --2- 172.21.15.64:0/2042028416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92f8104900 0x7f92f819cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:54.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.873+0000 7f92f6ffd700 1 -- 172.21.15.64:0/2042028416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92f8104900 msgr2=0x7f92f819cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:54.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.873+0000 7f92f6ffd700 1 --2- 172.21.15.64:0/2042028416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92f8104900 0x7f92f819cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:54.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.873+0000 7f92f6ffd700 1 -- 172.21.15.64:0/2042028416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92f80ff500 msgr2=0x7f92f81a2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:54.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.873+0000 7f92f6ffd700 1 --2- 172.21.15.64:0/2042028416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92f80ff500 0x7f92f81a2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:54.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.873+0000 7f92f6ffd700 1 -- 172.21.15.64:0/2042028416 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ec0095f0 con 0x7f92f80ffe20 2023-09-13T03:29:54.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.873+0000 7f92f6ffd700 1 --2- 172.21.15.64:0/2042028416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92f80ffe20 0x7f92f81a3000 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f92ec005910 tx=0x7f92ec005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:54.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.873+0000 7f92f4ff9700 1 -- 172.21.15.64:0/2042028416 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ec01d070 con 0x7f92f80ffe20 2023-09-13T03:29:54.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.874+0000 7f92f4ff9700 1 -- 172.21.15.64:0/2042028416 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92ec022470 con 0x7f92f80ffe20 2023-09-13T03:29:54.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.874+0000 7f92fdfff700 1 -- 172.21.15.64:0/2042028416 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92f819d750 con 0x7f92f80ffe20 2023-09-13T03:29:54.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.874+0000 7f92f4ff9700 1 -- 172.21.15.64:0/2042028416 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ec017650 con 0x7f92f80ffe20 2023-09-13T03:29:54.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.874+0000 7f92fdfff700 1 -- 172.21.15.64:0/2042028416 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92f81a9ab0 con 0x7f92f80ffe20 2023-09-13T03:29:54.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.875+0000 7f92fdfff700 1 -- 172.21.15.64:0/2042028416 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92f8067380 con 0x7f92f80ffe20 2023-09-13T03:29:54.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.877+0000 7f92f4ff9700 1 -- 172.21.15.64:0/2042028416 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 26) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f92ec0225e0 con 0x7f92f80ffe20 2023-09-13T03:29:54.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.878+0000 7f92f4ff9700 1 --2- 172.21.15.64:0/2042028416 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f92e00a0830 0x7f92e00a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-13T03:29:54.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.878+0000 7f92f4ff9700 1 -- 172.21.15.64:0/2042028416 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8345+0+0 (secure 0 0 0) 0x7f92ec0cd260 con 0x7f92f80ffe20 2023-09-13T03:29:54.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.879+0000 7f92f77fe700 1 --2- 172.21.15.64:0/2042028416 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f92e00a0830 0x7f92e00a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-13T03:29:54.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.879+0000 7f92f77fe700 1 --2- 172.21.15.64:0/2042028416 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f92e00a0830 0x7f92e00a2cf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f92e8009c60 tx=0x7f92e8009260 comp rx=0 tx=0).ready entity=mgr.15507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-13T03:29:54.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:54.882+0000 7f92f4ff9700 1 -- 172.21.15.64:0/2042028416 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f92ec096c50 con 0x7f92f80ffe20 2023-09-13T03:29:54.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:54 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/2592892323' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-13T03:29:54.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:54 smithi170 ceph-mon[134328]: pgmap v526: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 545 KiB/s wr, 82 op/s 2023-09-13T03:29:54.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:54 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2592892323' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-13T03:29:54.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:54 smithi064 ceph-mon[97675]: pgmap v526: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 545 KiB/s wr, 82 op/s 2023-09-13T03:29:54.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:54 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/2592892323' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-13T03:29:54.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:54 smithi142 ceph-mon[134082]: pgmap v526: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 545 KiB/s wr, 82 op/s 2023-09-13T03:29:55.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:55.105+0000 7f92fdfff700 1 -- 172.21.15.64:0/2042028416 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f92f819dfd0 con 0x7f92f80ffe20 2023-09-13T03:29:55.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:55.569+0000 7f92f4ff9700 1 -- 172.21.15.64:0/2042028416 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v109) v1 ==== 169+0+0 (secure 0 0 0) 0x7f92ec09a270 con 0x7f92f80ffe20 2023-09-13T03:29:55.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:55.574+0000 7f92fdfff700 1 -- 172.21.15.64:0/2042028416 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f92e00a0830 msgr2=0x7f92e00a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:55.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:55.574+0000 7f92fdfff700 1 --2- 172.21.15.64:0/2042028416 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f92e00a0830 0x7f92e00a2cf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f92e8009c60 tx=0x7f92e8009260 comp rx=0 tx=0).stop 2023-09-13T03:29:55.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:55.574+0000 7f92fdfff700 1 -- 172.21.15.64:0/2042028416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92f80ffe20 msgr2=0x7f92f81a3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-13T03:29:55.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:55.574+0000 7f92fdfff700 1 --2- 172.21.15.64:0/2042028416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92f80ffe20 0x7f92f81a3000 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f92ec005910 tx=0x7f92ec005720 comp rx=0 tx=0).stop 2023-09-13T03:29:55.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:55.575+0000 7f92fdfff700 1 -- 172.21.15.64:0/2042028416 shutdown_connections 2023-09-13T03:29:55.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:55.575+0000 7f92fdfff700 1 --2- 172.21.15.64:0/2042028416 >> [v2:172.21.15.64:6800/3860981465,v1:172.21.15.64:6801/3860981465] conn(0x7f92e00a0830 0x7f92e00a2cf0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:55.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:55.575+0000 7f92fdfff700 1 --2- 172.21.15.64:0/2042028416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92f8104900 0x7f92f819cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:55.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:55.575+0000 7f92fdfff700 1 --2- 172.21.15.64:0/2042028416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92f80ff500 0x7f92f81a2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:55.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:55.575+0000 7f92fdfff700 1 --2- 172.21.15.64:0/2042028416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92f80ffe20 0x7f92f81a3000 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-13T03:29:55.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:55.575+0000 7f92fdfff700 1 -- 172.21.15.64:0/2042028416 >> 172.21.15.64:0/2042028416 conn(0x7f92f80faf50 msgr2=0x7f92f8102400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-13T03:29:55.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:55.575+0000 7f92fdfff700 1 -- 172.21.15.64:0/2042028416 shutdown_connections 2023-09-13T03:29:55.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-09-13T03:29:55.576+0000 7f92fdfff700 1 -- 172.21.15.64:0/2042028416 wait complete. 2023-09-13T03:29:55.579 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' removed 2023-09-13T03:29:55.603 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-13T03:29:55.678 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-13T03:29:55.708 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-13T03:29:55.777 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-09-13T03:29:55.778 DEBUG:teuthology.orchestra.run.smithi064:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-09-13T03:29:55.824 DEBUG:teuthology.orchestra.run.smithi142:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-09-13T03:29:55.841 DEBUG:teuthology.orchestra.run.smithi170:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-09-13T03:29:55.869 INFO:tasks.cephadm:Stopping all daemons... 2023-09-13T03:29:55.870 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-09-13T03:29:55.870 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mon.a 2023-09-13T03:29:55.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:55 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/2592892323' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-09-13T03:29:55.910 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:55 smithi170 ceph-mon[134328]: osdmap e108: 12 total, 12 up, 12 in 2023-09-13T03:29:55.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:55 smithi170 ceph-mon[134328]: from='client.? 172.21.15.64:0/2042028416' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-13T03:29:55.911 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:55 smithi170 ceph-mon[134328]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-13T03:29:55.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:55 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2592892323' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-09-13T03:29:55.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:55 smithi064 ceph-mon[97675]: osdmap e108: 12 total, 12 up, 12 in 2023-09-13T03:29:55.918 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:55 smithi064 ceph-mon[97675]: from='client.? 172.21.15.64:0/2042028416' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-13T03:29:55.919 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:55 smithi064 ceph-mon[97675]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-13T03:29:55.995 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:55 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/2592892323' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-09-13T03:29:55.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:55 smithi142 ceph-mon[134082]: osdmap e108: 12 total, 12 up, 12 in 2023-09-13T03:29:55.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:55 smithi142 ceph-mon[134082]: from='client.? 172.21.15.64:0/2042028416' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-13T03:29:55.996 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:55 smithi142 ceph-mon[134082]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-13T03:29:56.395 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:55 smithi064 systemd[1]: Stopping Ceph mon.a for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:29:56.669 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:56 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mon-a[97651]: 2023-09-13T03:29:56.393+0000 7f9fd993f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T03:29:57.005 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:56 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mon-a[97651]: 2023-09-13T03:29:56.393+0000 7f9fd993f700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-09-13T03:29:57.168 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 13 03:29:56 smithi064 bash[180855]: ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mon-a 2023-09-13T03:29:57.338 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mon.a.service' 2023-09-13T03:29:57.452 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T03:29:57.453 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-09-13T03:29:57.453 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-09-13T03:29:57.454 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mon.b 2023-09-13T03:29:57.745 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:57 smithi142 systemd[1]: Stopping Ceph mon.b for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:29:58.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:58 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mon-b[134058]: 2023-09-13T03:29:58.124+0000 7fe8ac7bc700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T03:29:58.496 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:58 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mon-b[134058]: 2023-09-13T03:29:58.124+0000 7fe8ac7bc700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-09-13T03:29:59.063 INFO:journalctl@ceph.mon.b.smithi142.stdout:Sep 13 03:29:58 smithi142 bash[178614]: ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mon-b 2023-09-13T03:29:59.070 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mon.b.service' 2023-09-13T03:29:59.127 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T03:29:59.128 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-09-13T03:29:59.128 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-09-13T03:29:59.129 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mon.c 2023-09-13T03:29:59.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:59 smithi170 systemd[1]: Stopping Ceph mon.c for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:29:59.819 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:59 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mon-c[134294]: 2023-09-13T03:29:59.500+0000 7f5a01666700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T03:29:59.821 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:59 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mon-c[134294]: 2023-09-13T03:29:59.500+0000 7f5a01666700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-09-13T03:30:00.055 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mon.c.service' 2023-09-13T03:30:00.081 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:29:59 smithi170 bash[179616]: ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mon-c 2023-09-13T03:30:00.082 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:30:00 smithi170 systemd[1]: ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mon.c.service: Succeeded. 2023-09-13T03:30:00.082 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 13 03:30:00 smithi170 systemd[1]: Stopped Ceph mon.c for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da. 2023-09-13T03:30:00.101 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T03:30:00.101 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-09-13T03:30:00.102 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-09-13T03:30:00.103 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mgr.x 2023-09-13T03:30:00.418 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:30:00 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:30:00.155+0000 7fc934a66700 -1 client.25556: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2023-09-13T03:30:00.419 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:30:00 smithi064 systemd[1]: Stopping Ceph mgr.x for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:30:00.884 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:30:00 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:30:00.610+0000 7fc9d1b2b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T03:30:00.884 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:30:00 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x[97922]: 2023-09-13T03:30:00.610+0000 7fc9d1b2b700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-09-13T03:30:01.168 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 13 03:30:00 smithi064 bash[181005]: ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-x 2023-09-13T03:30:01.366 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mgr.x.service' 2023-09-13T03:30:01.458 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T03:30:01.459 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-09-13T03:30:01.459 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-09-13T03:30:01.460 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mgr.y 2023-09-13T03:30:01.745 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:30:01 smithi142 systemd[1]: Stopping Ceph mgr.y for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:30:02.226 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:30:02 smithi142 bash[178761]: ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-y 2023-09-13T03:30:02.227 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Sep 13 03:30:02 smithi142 systemd[1]: ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-09-13T03:30:02.354 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mgr.y.service' 2023-09-13T03:30:02.393 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T03:30:02.394 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-09-13T03:30:02.394 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-09-13T03:30:02.394 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mgr.z 2023-09-13T03:30:02.817 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:30:02 smithi170 systemd[1]: Stopping Ceph mgr.z for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:30:03.272 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:30:03 smithi170 bash[179761]: ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mgr-z 2023-09-13T03:30:03.272 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 13 03:30:03 smithi170 systemd[1]: ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-09-13T03:30:03.276 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mgr.z.service' 2023-09-13T03:30:03.314 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T03:30:03.315 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-09-13T03:30:03.315 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-09-13T03:30:03.316 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@osd.0 2023-09-13T03:30:03.668 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 13 03:30:03 smithi064 systemd[1]: Stopping Ceph osd.0 for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:30:04.168 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 13 03:30:03 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-0[120490]: 2023-09-13T03:30:03.817+0000 7f43eded6700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T03:30:04.169 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 13 03:30:03 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-0[120490]: 2023-09-13T03:30:03.817+0000 7f43eded6700 -1 osd.0 109 *** Got signal Terminated *** 2023-09-13T03:30:04.169 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 13 03:30:03 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-0[120490]: 2023-09-13T03:30:03.817+0000 7f43eded6700 -1 osd.0 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-13T03:30:09.820 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 13 03:30:09 smithi064 bash[181150]: ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-0 2023-09-13T03:30:11.458 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 13 03:30:11 smithi064 systemd[1]: ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@osd.0.service: Succeeded. 2023-09-13T03:30:11.458 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 13 03:30:11 smithi064 systemd[1]: Stopped Ceph osd.0 for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da. 2023-09-13T03:30:11.460 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@osd.0.service' 2023-09-13T03:30:11.552 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T03:30:11.553 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-09-13T03:30:11.553 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-09-13T03:30:11.554 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@osd.1 2023-09-13T03:30:11.919 INFO:journalctl@ceph.osd.1.smithi142.stdout:Sep 13 03:30:11 smithi142 systemd[1]: Stopping Ceph osd.1 for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:30:12.245 INFO:journalctl@ceph.osd.1.smithi142.stdout:Sep 13 03:30:11 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-1[139226]: 2023-09-13T03:30:11.917+0000 7f7d40877700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T03:30:12.245 INFO:journalctl@ceph.osd.1.smithi142.stdout:Sep 13 03:30:11 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-1[139226]: 2023-09-13T03:30:11.917+0000 7f7d40877700 -1 osd.1 109 *** Got signal Terminated *** 2023-09-13T03:30:12.246 INFO:journalctl@ceph.osd.1.smithi142.stdout:Sep 13 03:30:11 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-1[139226]: 2023-09-13T03:30:11.917+0000 7f7d40877700 -1 osd.1 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-13T03:30:17.791 INFO:journalctl@ceph.osd.1.smithi142.stdout:Sep 13 03:30:17 smithi142 bash[178917]: ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-1 2023-09-13T03:30:19.162 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@osd.1.service' 2023-09-13T03:30:19.202 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T03:30:19.202 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-09-13T03:30:19.203 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-09-13T03:30:19.203 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@osd.2 2023-09-13T03:30:19.560 INFO:journalctl@ceph.osd.2.smithi170.stdout:Sep 13 03:30:19 smithi170 systemd[1]: Stopping Ceph osd.2 for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:30:19.910 INFO:journalctl@ceph.osd.2.smithi170.stdout:Sep 13 03:30:19 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-2[140189]: 2023-09-13T03:30:19.558+0000 7f631a7c7700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T03:30:19.910 INFO:journalctl@ceph.osd.2.smithi170.stdout:Sep 13 03:30:19 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-2[140189]: 2023-09-13T03:30:19.558+0000 7f631a7c7700 -1 osd.2 109 *** Got signal Terminated *** 2023-09-13T03:30:19.910 INFO:journalctl@ceph.osd.2.smithi170.stdout:Sep 13 03:30:19 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-2[140189]: 2023-09-13T03:30:19.558+0000 7f631a7c7700 -1 osd.2 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-13T03:30:25.660 INFO:journalctl@ceph.osd.2.smithi170.stdout:Sep 13 03:30:25 smithi170 bash[179911]: ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-2 2023-09-13T03:30:26.758 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@osd.2.service' 2023-09-13T03:30:26.799 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T03:30:26.800 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-09-13T03:30:26.800 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-09-13T03:30:26.801 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@osd.3 2023-09-13T03:30:27.168 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 13 03:30:26 smithi064 systemd[1]: Stopping Ceph osd.3 for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:30:27.668 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 13 03:30:27 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-3[128330]: 2023-09-13T03:30:27.305+0000 7f86442fd700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T03:30:27.669 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 13 03:30:27 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-3[128330]: 2023-09-13T03:30:27.305+0000 7f86442fd700 -1 osd.3 109 *** Got signal Terminated *** 2023-09-13T03:30:27.669 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 13 03:30:27 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-3[128330]: 2023-09-13T03:30:27.305+0000 7f86442fd700 -1 osd.3 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-13T03:30:30.910 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 13 03:30:30 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-5[148087]: 2023-09-13T03:30:30.483+0000 7f1a47259700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:05.836805+0000 front 2023-09-13T03:30:05.836791+0000 (oldest deadline 2023-09-13T03:30:29.936553+0000) 2023-09-13T03:30:30.911 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:30 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:30.586+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:30:31.168 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 13 03:30:30 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-3[128330]: 2023-09-13T03:30:30.852+0000 7f86408f3700 -1 osd.3 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.614477+0000 front 2023-09-13T03:30:06.614602+0000 (oldest deadline 2023-09-13T03:30:30.714180+0000) 2023-09-13T03:30:31.745 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:31 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:31.267+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:30:31.910 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:31 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:31.578+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:30:31.910 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 13 03:30:31 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-5[148087]: 2023-09-13T03:30:31.449+0000 7f1a47259700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:05.836805+0000 front 2023-09-13T03:30:05.836791+0000 (oldest deadline 2023-09-13T03:30:29.936553+0000) 2023-09-13T03:30:32.168 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 13 03:30:31 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-3[128330]: 2023-09-13T03:30:31.855+0000 7f86408f3700 -1 osd.3 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.614477+0000 front 2023-09-13T03:30:06.614602+0000 (oldest deadline 2023-09-13T03:30:30.714180+0000) 2023-09-13T03:30:32.495 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:32 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:32.221+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:30:32.910 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 13 03:30:32 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-5[148087]: 2023-09-13T03:30:32.410+0000 7f1a47259700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:05.836805+0000 front 2023-09-13T03:30:05.836791+0000 (oldest deadline 2023-09-13T03:30:29.936553+0000) 2023-09-13T03:30:32.911 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:32 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:32.621+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:30:33.155 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:32 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:32.849+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.115759+0000 front 2023-09-13T03:30:07.115826+0000 (oldest deadline 2023-09-13T03:30:32.415918+0000) 2023-09-13T03:30:33.418 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 13 03:30:33 smithi064 bash[181432]: ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-3 2023-09-13T03:30:33.419 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:33 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:33.184+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:30:33.495 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:33 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:33.201+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:30:33.910 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 13 03:30:33 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-5[148087]: 2023-09-13T03:30:33.437+0000 7f1a47259700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:05.836805+0000 front 2023-09-13T03:30:05.836791+0000 (oldest deadline 2023-09-13T03:30:29.936553+0000) 2023-09-13T03:30:33.910 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:33 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:33.622+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:30:34.159 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:33 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:33.805+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.115759+0000 front 2023-09-13T03:30:07.115826+0000 (oldest deadline 2023-09-13T03:30:32.415918+0000) 2023-09-13T03:30:34.421 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:34 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:34.193+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:30:34.495 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:34 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:34.181+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:30:34.910 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 13 03:30:34 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-5[148087]: 2023-09-13T03:30:34.476+0000 7f1a47259700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:05.836805+0000 front 2023-09-13T03:30:05.836791+0000 (oldest deadline 2023-09-13T03:30:29.936553+0000) 2023-09-13T03:30:34.910 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:34 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:34.596+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:30:34.919 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:34 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:34.787+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.115759+0000 front 2023-09-13T03:30:07.115826+0000 (oldest deadline 2023-09-13T03:30:32.415918+0000) 2023-09-13T03:30:34.975 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@osd.3.service' 2023-09-13T03:30:35.068 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T03:30:35.069 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-09-13T03:30:35.069 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-09-13T03:30:35.070 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@osd.4 2023-09-13T03:30:35.431 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 13 03:30:35 smithi142 systemd[1]: Stopping Ceph osd.4 for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:30:35.432 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:35 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:35.225+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:30:35.668 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:35 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:35.224+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:30:35.745 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 13 03:30:35 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-4[147323]: 2023-09-13T03:30:35.430+0000 7f6d44d50700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T03:30:35.746 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 13 03:30:35 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-4[147323]: 2023-09-13T03:30:35.430+0000 7f6d44d50700 -1 osd.4 109 *** Got signal Terminated *** 2023-09-13T03:30:35.746 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 13 03:30:35 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-4[147323]: 2023-09-13T03:30:35.430+0000 7f6d44d50700 -1 osd.4 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-13T03:30:35.910 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:35 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:35.601+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:30:35.910 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 13 03:30:35 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-5[148087]: 2023-09-13T03:30:35.501+0000 7f1a47259700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:05.836805+0000 front 2023-09-13T03:30:05.836791+0000 (oldest deadline 2023-09-13T03:30:29.936553+0000) 2023-09-13T03:30:36.168 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:35 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:35.778+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.115759+0000 front 2023-09-13T03:30:07.115826+0000 (oldest deadline 2023-09-13T03:30:32.415918+0000) 2023-09-13T03:30:36.495 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:36 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:36.244+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:30:36.668 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:36 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:36.254+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:30:36.910 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:36 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:36.628+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:30:36.911 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 13 03:30:36 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-5[148087]: 2023-09-13T03:30:36.454+0000 7f1a47259700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:05.836805+0000 front 2023-09-13T03:30:05.836791+0000 (oldest deadline 2023-09-13T03:30:29.936553+0000) 2023-09-13T03:30:37.168 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:36 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:36.741+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.115759+0000 front 2023-09-13T03:30:07.115826+0000 (oldest deadline 2023-09-13T03:30:32.415918+0000) 2023-09-13T03:30:37.668 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:37 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:37.235+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:30:37.745 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:37 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:37.265+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:30:37.910 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 13 03:30:37 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-5[148087]: 2023-09-13T03:30:37.483+0000 7f1a47259700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:05.836805+0000 front 2023-09-13T03:30:05.836791+0000 (oldest deadline 2023-09-13T03:30:29.936553+0000) 2023-09-13T03:30:37.911 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:37 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:37.640+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:30:38.168 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:37 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:37.777+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.115759+0000 front 2023-09-13T03:30:07.115826+0000 (oldest deadline 2023-09-13T03:30:32.415918+0000) 2023-09-13T03:30:38.495 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:38 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:38.219+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:30:38.495 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:38 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:38.220+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:30:38.668 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:38 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:38.241+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:30:38.910 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 13 03:30:38 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-5[148087]: 2023-09-13T03:30:38.481+0000 7f1a47259700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:05.836805+0000 front 2023-09-13T03:30:05.836791+0000 (oldest deadline 2023-09-13T03:30:29.936553+0000) 2023-09-13T03:30:38.911 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:38 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:38.600+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:30:38.911 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:38 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:38.600+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:30:39.168 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:38 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:38.800+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.115759+0000 front 2023-09-13T03:30:07.115826+0000 (oldest deadline 2023-09-13T03:30:32.415918+0000) 2023-09-13T03:30:39.495 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:39 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:39.207+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:30:39.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:39 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:39.207+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:30:39.668 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:39 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:39.242+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:30:39.794 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 13 03:30:39 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-5[148087]: 2023-09-13T03:30:39.501+0000 7f1a47259700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:05.836805+0000 front 2023-09-13T03:30:05.836791+0000 (oldest deadline 2023-09-13T03:30:29.936553+0000) 2023-09-13T03:30:39.795 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:39 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:39.557+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:30:39.795 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:39 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:39.557+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:30:39.995 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 13 03:30:39 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-4[147323]: 2023-09-13T03:30:39.716+0000 7f6d40344700 -1 osd.4 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:14.331708+0000 front 2023-09-13T03:30:14.331743+0000 (oldest deadline 2023-09-13T03:30:39.031912+0000) 2023-09-13T03:30:40.160 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:30:39 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:30:39.793+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:14.436426+0000 front 2023-09-13T03:30:14.436375+0000 (oldest deadline 2023-09-13T03:30:39.736445+0000) 2023-09-13T03:30:40.168 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:39 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:39.795+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.115759+0000 front 2023-09-13T03:30:07.115826+0000 (oldest deadline 2023-09-13T03:30:32.415918+0000) 2023-09-13T03:30:40.495 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:40 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:40.236+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:30:40.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:40 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:40.236+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:30:40.668 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:40 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:40.255+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:30:40.761 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 13 03:30:40 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-5[148087]: 2023-09-13T03:30:40.463+0000 7f1a47259700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:05.836805+0000 front 2023-09-13T03:30:05.836791+0000 (oldest deadline 2023-09-13T03:30:29.936553+0000) 2023-09-13T03:30:40.762 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:40 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:40.577+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:30:40.762 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:40 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:40.577+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:30:41.160 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:30:40 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:30:40.760+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:14.436426+0000 front 2023-09-13T03:30:14.436375+0000 (oldest deadline 2023-09-13T03:30:39.736445+0000) 2023-09-13T03:30:41.168 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:40 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:40.747+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.115759+0000 front 2023-09-13T03:30:07.115826+0000 (oldest deadline 2023-09-13T03:30:32.415918+0000) 2023-09-13T03:30:41.245 INFO:journalctl@ceph.osd.4.smithi142.stdout:Sep 13 03:30:41 smithi142 bash[179178]: ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-4 2023-09-13T03:30:41.668 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:41 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:41.249+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:30:41.668 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:41 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:41.249+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.238945+0000 front 2023-09-13T03:30:15.239145+0000 (oldest deadline 2023-09-13T03:30:41.139111+0000) 2023-09-13T03:30:41.746 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:41 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:41.267+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:30:41.746 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:41 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:41.267+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:30:41.766 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:41 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:41.577+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:30:41.766 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:41 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:41.577+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:30:41.766 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 13 03:30:41 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-5[148087]: 2023-09-13T03:30:41.418+0000 7f1a47259700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:05.836805+0000 front 2023-09-13T03:30:05.836791+0000 (oldest deadline 2023-09-13T03:30:29.936553+0000) 2023-09-13T03:30:42.160 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:30:41 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:30:41.765+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:14.436426+0000 front 2023-09-13T03:30:14.436375+0000 (oldest deadline 2023-09-13T03:30:39.736445+0000) 2023-09-13T03:30:42.168 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:41 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:41.782+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.115759+0000 front 2023-09-13T03:30:07.115826+0000 (oldest deadline 2023-09-13T03:30:32.415918+0000) 2023-09-13T03:30:42.169 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:41 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:41.782+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.416578+0000 front 2023-09-13T03:30:12.416743+0000 (oldest deadline 2023-09-13T03:30:41.217924+0000) 2023-09-13T03:30:42.413 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:42 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:42.230+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:30:42.413 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:42 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:42.230+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:30:42.420 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@osd.4.service' 2023-09-13T03:30:42.459 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T03:30:42.460 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-09-13T03:30:42.460 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-09-13T03:30:42.460 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@osd.5 2023-09-13T03:30:42.668 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:42 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:42.298+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:30:42.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:42 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:42.298+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.238945+0000 front 2023-09-13T03:30:15.239145+0000 (oldest deadline 2023-09-13T03:30:41.139111+0000) 2023-09-13T03:30:42.776 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:42 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:42.576+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:30:42.776 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:42 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:42.576+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:30:42.777 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 13 03:30:42 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-5[148087]: 2023-09-13T03:30:42.420+0000 7f1a47259700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:05.836805+0000 front 2023-09-13T03:30:05.836791+0000 (oldest deadline 2023-09-13T03:30:29.936553+0000) 2023-09-13T03:30:42.777 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 13 03:30:42 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-5[148087]: 2023-09-13T03:30:42.420+0000 7f1a47259700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.638773+0000 front 2023-09-13T03:30:15.638875+0000 (oldest deadline 2023-09-13T03:30:41.538855+0000) 2023-09-13T03:30:42.777 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 13 03:30:42 smithi170 systemd[1]: Stopping Ceph osd.5 for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:30:43.160 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:30:42 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:30:42.775+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:14.436426+0000 front 2023-09-13T03:30:14.436375+0000 (oldest deadline 2023-09-13T03:30:39.736445+0000) 2023-09-13T03:30:43.160 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 13 03:30:42 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-5[148087]: 2023-09-13T03:30:42.818+0000 7f1a4d468700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T03:30:43.161 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 13 03:30:42 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-5[148087]: 2023-09-13T03:30:42.818+0000 7f1a4d468700 -1 osd.5 109 *** Got signal Terminated *** 2023-09-13T03:30:43.161 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 13 03:30:42 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-5[148087]: 2023-09-13T03:30:42.818+0000 7f1a4d468700 -1 osd.5 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-13T03:30:43.168 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:42 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:42.768+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.115759+0000 front 2023-09-13T03:30:07.115826+0000 (oldest deadline 2023-09-13T03:30:32.415918+0000) 2023-09-13T03:30:43.168 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:42 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:42.768+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.416578+0000 front 2023-09-13T03:30:12.416743+0000 (oldest deadline 2023-09-13T03:30:41.217924+0000) 2023-09-13T03:30:43.186 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:30:42 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:30:42.885+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:16.857617+0000 front 2023-09-13T03:30:16.857572+0000 (oldest deadline 2023-09-13T03:30:42.157731+0000) 2023-09-13T03:30:43.495 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:43 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:43.186+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:30:43.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:43 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:43.186+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:30:43.660 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 13 03:30:43 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-5[148087]: 2023-09-13T03:30:43.372+0000 7f1a47259700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:05.836805+0000 front 2023-09-13T03:30:05.836791+0000 (oldest deadline 2023-09-13T03:30:29.936553+0000) 2023-09-13T03:30:43.660 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 13 03:30:43 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-5[148087]: 2023-09-13T03:30:43.372+0000 7f1a47259700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.638773+0000 front 2023-09-13T03:30:15.638875+0000 (oldest deadline 2023-09-13T03:30:41.538855+0000) 2023-09-13T03:30:43.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:43 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:43.560+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:30:43.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:43 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:43.560+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:30:43.668 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:43 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:43.254+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:30:43.668 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:43 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:43.254+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.238945+0000 front 2023-09-13T03:30:15.239145+0000 (oldest deadline 2023-09-13T03:30:41.139111+0000) 2023-09-13T03:30:44.160 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:30:43 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:30:43.734+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:14.436426+0000 front 2023-09-13T03:30:14.436375+0000 (oldest deadline 2023-09-13T03:30:39.736445+0000) 2023-09-13T03:30:44.168 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:43 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:43.729+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.115759+0000 front 2023-09-13T03:30:07.115826+0000 (oldest deadline 2023-09-13T03:30:32.415918+0000) 2023-09-13T03:30:44.168 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:43 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:43.729+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.416578+0000 front 2023-09-13T03:30:12.416743+0000 (oldest deadline 2023-09-13T03:30:41.217924+0000) 2023-09-13T03:30:44.183 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:30:43 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:30:43.921+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:16.857617+0000 front 2023-09-13T03:30:16.857572+0000 (oldest deadline 2023-09-13T03:30:42.157731+0000) 2023-09-13T03:30:44.495 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:44 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:44.183+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:30:44.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:44 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:44.183+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:30:44.662 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 13 03:30:44 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-5[148087]: 2023-09-13T03:30:44.358+0000 7f1a47259700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:05.836805+0000 front 2023-09-13T03:30:05.836791+0000 (oldest deadline 2023-09-13T03:30:29.936553+0000) 2023-09-13T03:30:44.662 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 13 03:30:44 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-5[148087]: 2023-09-13T03:30:44.358+0000 7f1a47259700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.638773+0000 front 2023-09-13T03:30:15.638875+0000 (oldest deadline 2023-09-13T03:30:41.538855+0000) 2023-09-13T03:30:44.663 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:44 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:44.536+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:30:44.663 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:44 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:44.536+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:30:44.668 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:44 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:44.267+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:30:44.668 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:44 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:44.267+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.238945+0000 front 2023-09-13T03:30:15.239145+0000 (oldest deadline 2023-09-13T03:30:41.139111+0000) 2023-09-13T03:30:45.160 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:30:44 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:30:44.771+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:14.436426+0000 front 2023-09-13T03:30:14.436375+0000 (oldest deadline 2023-09-13T03:30:39.736445+0000) 2023-09-13T03:30:45.168 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:44 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:44.694+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.115759+0000 front 2023-09-13T03:30:07.115826+0000 (oldest deadline 2023-09-13T03:30:32.415918+0000) 2023-09-13T03:30:45.168 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:44 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:44.694+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.416578+0000 front 2023-09-13T03:30:12.416743+0000 (oldest deadline 2023-09-13T03:30:41.217924+0000) 2023-09-13T03:30:45.245 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:30:44 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:30:44.955+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:16.857617+0000 front 2023-09-13T03:30:16.857572+0000 (oldest deadline 2023-09-13T03:30:42.157731+0000) 2023-09-13T03:30:45.246 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:45 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:45.172+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:30:45.246 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:45 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:45.172+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:30:45.658 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:45 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:45.267+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:30:45.659 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:45 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:45.267+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.238945+0000 front 2023-09-13T03:30:15.239145+0000 (oldest deadline 2023-09-13T03:30:41.139111+0000) 2023-09-13T03:30:45.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:45 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:45.540+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:30:45.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:45 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:45.540+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:30:45.662 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 13 03:30:45 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-5[148087]: 2023-09-13T03:30:45.325+0000 7f1a47259700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:05.836805+0000 front 2023-09-13T03:30:05.836791+0000 (oldest deadline 2023-09-13T03:30:29.936553+0000) 2023-09-13T03:30:45.663 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 13 03:30:45 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-5[148087]: 2023-09-13T03:30:45.325+0000 7f1a47259700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.638773+0000 front 2023-09-13T03:30:15.638875+0000 (oldest deadline 2023-09-13T03:30:41.538855+0000) 2023-09-13T03:30:45.918 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:45 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:45.658+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.115759+0000 front 2023-09-13T03:30:07.115826+0000 (oldest deadline 2023-09-13T03:30:32.415918+0000) 2023-09-13T03:30:45.919 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:45 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:45.658+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.416578+0000 front 2023-09-13T03:30:12.416743+0000 (oldest deadline 2023-09-13T03:30:41.217924+0000) 2023-09-13T03:30:45.919 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:45 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:45.658+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.218666+0000 front 2023-09-13T03:30:21.218869+0000 (oldest deadline 2023-09-13T03:30:45.318921+0000) 2023-09-13T03:30:46.160 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:30:45 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:30:45.733+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:14.436426+0000 front 2023-09-13T03:30:14.436375+0000 (oldest deadline 2023-09-13T03:30:39.736445+0000) 2023-09-13T03:30:46.245 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:30:45 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:30:45.971+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:16.857617+0000 front 2023-09-13T03:30:16.857572+0000 (oldest deadline 2023-09-13T03:30:42.157731+0000) 2023-09-13T03:30:46.246 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:46 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:46.198+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:30:46.246 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:46 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:46.198+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:30:46.247 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:46 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:46.198+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.557932+0000 front 2023-09-13T03:30:21.558055+0000 (oldest deadline 2023-09-13T03:30:45.658031+0000) 2023-09-13T03:30:46.657 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:46 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:46.277+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:30:46.657 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:46 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:46.277+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.238945+0000 front 2023-09-13T03:30:15.239145+0000 (oldest deadline 2023-09-13T03:30:41.139111+0000) 2023-09-13T03:30:46.657 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:46 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:46.277+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.840328+0000 front 2023-09-13T03:30:22.840159+0000 (oldest deadline 2023-09-13T03:30:45.740226+0000) 2023-09-13T03:30:46.660 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 13 03:30:46 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-5[148087]: 2023-09-13T03:30:46.371+0000 7f1a47259700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:05.836805+0000 front 2023-09-13T03:30:05.836791+0000 (oldest deadline 2023-09-13T03:30:29.936553+0000) 2023-09-13T03:30:46.660 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 13 03:30:46 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-5[148087]: 2023-09-13T03:30:46.371+0000 7f1a47259700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.638773+0000 front 2023-09-13T03:30:15.638875+0000 (oldest deadline 2023-09-13T03:30:41.538855+0000) 2023-09-13T03:30:46.660 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 13 03:30:46 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-5[148087]: 2023-09-13T03:30:46.371+0000 7f1a47259700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.539234+0000 front 2023-09-13T03:30:21.539265+0000 (oldest deadline 2023-09-13T03:30:46.239437+0000) 2023-09-13T03:30:46.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:46 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:46.511+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:30:46.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:46 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:46.511+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:30:46.662 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:46 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:46.511+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:30:46.918 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:46 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:46.656+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.115759+0000 front 2023-09-13T03:30:07.115826+0000 (oldest deadline 2023-09-13T03:30:32.415918+0000) 2023-09-13T03:30:46.919 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:46 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:46.656+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.416578+0000 front 2023-09-13T03:30:12.416743+0000 (oldest deadline 2023-09-13T03:30:41.217924+0000) 2023-09-13T03:30:46.919 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:46 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:46.656+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.218666+0000 front 2023-09-13T03:30:21.218869+0000 (oldest deadline 2023-09-13T03:30:45.318921+0000) 2023-09-13T03:30:47.160 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:30:46 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:30:46.701+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:14.436426+0000 front 2023-09-13T03:30:14.436375+0000 (oldest deadline 2023-09-13T03:30:39.736445+0000) 2023-09-13T03:30:47.160 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:30:46 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:30:46.701+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.338362+0000 front 2023-09-13T03:30:21.338390+0000 (oldest deadline 2023-09-13T03:30:46.638525+0000) 2023-09-13T03:30:47.228 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:30:46 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:30:46.957+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:16.857617+0000 front 2023-09-13T03:30:16.857572+0000 (oldest deadline 2023-09-13T03:30:42.157731+0000) 2023-09-13T03:30:47.495 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:47 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:47.228+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:30:47.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:47 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:47.228+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:30:47.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:47 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:47.228+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.557932+0000 front 2023-09-13T03:30:21.558055+0000 (oldest deadline 2023-09-13T03:30:45.658031+0000) 2023-09-13T03:30:47.637 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:47 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:47.249+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:30:47.637 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:47 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:47.249+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.238945+0000 front 2023-09-13T03:30:15.239145+0000 (oldest deadline 2023-09-13T03:30:41.139111+0000) 2023-09-13T03:30:47.638 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:47 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:47.249+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.840328+0000 front 2023-09-13T03:30:22.840159+0000 (oldest deadline 2023-09-13T03:30:45.740226+0000) 2023-09-13T03:30:47.660 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 13 03:30:47 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-5[148087]: 2023-09-13T03:30:47.336+0000 7f1a47259700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:05.836805+0000 front 2023-09-13T03:30:05.836791+0000 (oldest deadline 2023-09-13T03:30:29.936553+0000) 2023-09-13T03:30:47.660 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 13 03:30:47 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-5[148087]: 2023-09-13T03:30:47.336+0000 7f1a47259700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.638773+0000 front 2023-09-13T03:30:15.638875+0000 (oldest deadline 2023-09-13T03:30:41.538855+0000) 2023-09-13T03:30:47.661 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 13 03:30:47 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-5[148087]: 2023-09-13T03:30:47.336+0000 7f1a47259700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.539234+0000 front 2023-09-13T03:30:21.539265+0000 (oldest deadline 2023-09-13T03:30:46.239437+0000) 2023-09-13T03:30:47.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:47 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:47.469+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:30:47.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:47 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:47.469+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:30:47.662 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:47 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:47.469+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:30:47.918 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:47 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:47.636+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.115759+0000 front 2023-09-13T03:30:07.115826+0000 (oldest deadline 2023-09-13T03:30:32.415918+0000) 2023-09-13T03:30:47.919 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:47 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:47.636+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.416578+0000 front 2023-09-13T03:30:12.416743+0000 (oldest deadline 2023-09-13T03:30:41.217924+0000) 2023-09-13T03:30:47.919 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:47 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:47.636+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.218666+0000 front 2023-09-13T03:30:21.218869+0000 (oldest deadline 2023-09-13T03:30:45.318921+0000) 2023-09-13T03:30:48.160 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:30:47 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:30:47.670+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:14.436426+0000 front 2023-09-13T03:30:14.436375+0000 (oldest deadline 2023-09-13T03:30:39.736445+0000) 2023-09-13T03:30:48.161 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:30:47 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:30:47.670+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.338362+0000 front 2023-09-13T03:30:21.338390+0000 (oldest deadline 2023-09-13T03:30:46.638525+0000) 2023-09-13T03:30:48.245 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:30:47 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:30:47.968+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:16.857617+0000 front 2023-09-13T03:30:16.857572+0000 (oldest deadline 2023-09-13T03:30:42.157731+0000) 2023-09-13T03:30:48.246 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:30:47 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:30:47.968+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.158371+0000 front 2023-09-13T03:30:22.158334+0000 (oldest deadline 2023-09-13T03:30:47.458424+0000) 2023-09-13T03:30:48.246 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:48 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:48.215+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:30:48.246 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:48 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:48.215+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:30:48.247 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:48 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:48.215+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.557932+0000 front 2023-09-13T03:30:21.558055+0000 (oldest deadline 2023-09-13T03:30:45.658031+0000) 2023-09-13T03:30:48.668 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:48 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:48.278+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:30:48.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:48 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:48.278+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.238945+0000 front 2023-09-13T03:30:15.239145+0000 (oldest deadline 2023-09-13T03:30:41.139111+0000) 2023-09-13T03:30:48.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:48 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:48.278+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.840328+0000 front 2023-09-13T03:30:22.840159+0000 (oldest deadline 2023-09-13T03:30:45.740226+0000) 2023-09-13T03:30:48.799 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 13 03:30:48 smithi170 bash[180178]: ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-5 2023-09-13T03:30:48.800 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:48 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:48.502+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:30:48.800 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:48 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:48.502+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:30:48.801 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:48 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:48.502+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:30:48.801 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:30:48 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:30:48.636+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:14.436426+0000 front 2023-09-13T03:30:14.436375+0000 (oldest deadline 2023-09-13T03:30:39.736445+0000) 2023-09-13T03:30:48.801 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:30:48 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:30:48.636+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.338362+0000 front 2023-09-13T03:30:21.338390+0000 (oldest deadline 2023-09-13T03:30:46.638525+0000) 2023-09-13T03:30:49.168 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:48 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:48.678+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.115759+0000 front 2023-09-13T03:30:07.115826+0000 (oldest deadline 2023-09-13T03:30:32.415918+0000) 2023-09-13T03:30:49.169 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:48 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:48.678+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.416578+0000 front 2023-09-13T03:30:12.416743+0000 (oldest deadline 2023-09-13T03:30:41.217924+0000) 2023-09-13T03:30:49.169 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:48 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:48.678+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.218666+0000 front 2023-09-13T03:30:21.218869+0000 (oldest deadline 2023-09-13T03:30:45.318921+0000) 2023-09-13T03:30:49.241 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:30:48 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:30:48.937+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:16.857617+0000 front 2023-09-13T03:30:16.857572+0000 (oldest deadline 2023-09-13T03:30:42.157731+0000) 2023-09-13T03:30:49.242 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:30:48 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:30:48.937+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.158371+0000 front 2023-09-13T03:30:22.158334+0000 (oldest deadline 2023-09-13T03:30:47.458424+0000) 2023-09-13T03:30:49.495 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:49 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:49.241+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:30:49.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:49 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:49.241+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:30:49.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:49 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:49.241+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.557932+0000 front 2023-09-13T03:30:21.558055+0000 (oldest deadline 2023-09-13T03:30:45.658031+0000) 2023-09-13T03:30:49.668 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:49 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:49.319+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:30:49.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:49 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:49.319+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.238945+0000 front 2023-09-13T03:30:15.239145+0000 (oldest deadline 2023-09-13T03:30:41.139111+0000) 2023-09-13T03:30:49.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:49 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:49.319+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.840328+0000 front 2023-09-13T03:30:22.840159+0000 (oldest deadline 2023-09-13T03:30:45.740226+0000) 2023-09-13T03:30:49.824 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:30:49 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:30:49.667+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:14.436426+0000 front 2023-09-13T03:30:14.436375+0000 (oldest deadline 2023-09-13T03:30:39.736445+0000) 2023-09-13T03:30:49.824 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:30:49 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:30:49.667+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.338362+0000 front 2023-09-13T03:30:21.338390+0000 (oldest deadline 2023-09-13T03:30:46.638525+0000) 2023-09-13T03:30:49.825 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:49 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:49.506+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:30:49.825 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:49 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:49.506+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:30:49.826 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:49 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:49.506+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:30:50.152 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@osd.5.service' 2023-09-13T03:30:50.168 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:49 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:49.695+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.115759+0000 front 2023-09-13T03:30:07.115826+0000 (oldest deadline 2023-09-13T03:30:32.415918+0000) 2023-09-13T03:30:50.169 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:49 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:49.695+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.416578+0000 front 2023-09-13T03:30:12.416743+0000 (oldest deadline 2023-09-13T03:30:41.217924+0000) 2023-09-13T03:30:50.169 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:49 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:49.695+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.218666+0000 front 2023-09-13T03:30:21.218869+0000 (oldest deadline 2023-09-13T03:30:45.318921+0000) 2023-09-13T03:30:50.194 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T03:30:50.195 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-09-13T03:30:50.196 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-09-13T03:30:50.196 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@osd.6 2023-09-13T03:30:50.217 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:30:49 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:30:49.933+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:16.857617+0000 front 2023-09-13T03:30:16.857572+0000 (oldest deadline 2023-09-13T03:30:42.157731+0000) 2023-09-13T03:30:50.217 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:30:49 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:30:49.933+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.158371+0000 front 2023-09-13T03:30:22.158334+0000 (oldest deadline 2023-09-13T03:30:47.458424+0000) 2023-09-13T03:30:50.495 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:50 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:50.217+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:30:50.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:50 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:50.217+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:30:50.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:50 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:50.217+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.557932+0000 front 2023-09-13T03:30:21.558055+0000 (oldest deadline 2023-09-13T03:30:45.658031+0000) 2023-09-13T03:30:50.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:50 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:50.363+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:30:50.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:50 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:50.363+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.238945+0000 front 2023-09-13T03:30:15.239145+0000 (oldest deadline 2023-09-13T03:30:41.139111+0000) 2023-09-13T03:30:50.670 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:50 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:50.363+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.840328+0000 front 2023-09-13T03:30:22.840159+0000 (oldest deadline 2023-09-13T03:30:45.740226+0000) 2023-09-13T03:30:50.670 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:50 smithi064 systemd[1]: Stopping Ceph osd.6 for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:30:50.910 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:30:50 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:30:50.700+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:14.436426+0000 front 2023-09-13T03:30:14.436375+0000 (oldest deadline 2023-09-13T03:30:39.736445+0000) 2023-09-13T03:30:50.910 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:30:50 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:30:50.700+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.338362+0000 front 2023-09-13T03:30:21.338390+0000 (oldest deadline 2023-09-13T03:30:46.638525+0000) 2023-09-13T03:30:50.911 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:50 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:50.498+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:30:50.911 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:50 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:50.498+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:30:50.911 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:50 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:50.498+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:30:51.168 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:50 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:50.702+0000 7fe5f3259700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T03:30:51.169 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:50 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:50.702+0000 7fe5f3259700 -1 osd.6 109 *** Got signal Terminated *** 2023-09-13T03:30:51.169 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:50 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:50.702+0000 7fe5f3259700 -1 osd.6 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-13T03:30:51.169 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:50 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:50.721+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.115759+0000 front 2023-09-13T03:30:07.115826+0000 (oldest deadline 2023-09-13T03:30:32.415918+0000) 2023-09-13T03:30:51.170 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:50 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:50.721+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.416578+0000 front 2023-09-13T03:30:12.416743+0000 (oldest deadline 2023-09-13T03:30:41.217924+0000) 2023-09-13T03:30:51.170 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:50 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:50.721+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.218666+0000 front 2023-09-13T03:30:21.218869+0000 (oldest deadline 2023-09-13T03:30:45.318921+0000) 2023-09-13T03:30:51.245 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:30:50 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:30:50.970+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:16.857617+0000 front 2023-09-13T03:30:16.857572+0000 (oldest deadline 2023-09-13T03:30:42.157731+0000) 2023-09-13T03:30:51.246 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:30:50 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:30:50.970+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.158371+0000 front 2023-09-13T03:30:22.158334+0000 (oldest deadline 2023-09-13T03:30:47.458424+0000) 2023-09-13T03:30:51.246 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:51 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:51.192+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:30:51.247 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:51 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:51.192+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:30:51.247 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:51 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:51.192+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.557932+0000 front 2023-09-13T03:30:21.558055+0000 (oldest deadline 2023-09-13T03:30:45.658031+0000) 2023-09-13T03:30:51.668 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:51 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:51.386+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:30:51.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:51 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:51.386+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.238945+0000 front 2023-09-13T03:30:15.239145+0000 (oldest deadline 2023-09-13T03:30:41.139111+0000) 2023-09-13T03:30:51.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:51 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:51.386+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.840328+0000 front 2023-09-13T03:30:22.840159+0000 (oldest deadline 2023-09-13T03:30:45.740226+0000) 2023-09-13T03:30:51.910 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:30:51 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:30:51.747+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:14.436426+0000 front 2023-09-13T03:30:14.436375+0000 (oldest deadline 2023-09-13T03:30:39.736445+0000) 2023-09-13T03:30:51.910 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:30:51 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:30:51.747+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.338362+0000 front 2023-09-13T03:30:21.338390+0000 (oldest deadline 2023-09-13T03:30:46.638525+0000) 2023-09-13T03:30:51.911 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:51 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:51.508+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:30:51.911 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:51 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:51.508+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:30:51.912 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:51 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:51.508+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:30:52.168 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:51 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:51.708+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.115759+0000 front 2023-09-13T03:30:07.115826+0000 (oldest deadline 2023-09-13T03:30:32.415918+0000) 2023-09-13T03:30:52.169 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:51 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:51.708+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.416578+0000 front 2023-09-13T03:30:12.416743+0000 (oldest deadline 2023-09-13T03:30:41.217924+0000) 2023-09-13T03:30:52.169 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:51 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:51.708+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.218666+0000 front 2023-09-13T03:30:21.218869+0000 (oldest deadline 2023-09-13T03:30:45.318921+0000) 2023-09-13T03:30:52.245 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:30:51 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:30:51.942+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:16.857617+0000 front 2023-09-13T03:30:16.857572+0000 (oldest deadline 2023-09-13T03:30:42.157731+0000) 2023-09-13T03:30:52.246 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:30:51 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:30:51.942+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.158371+0000 front 2023-09-13T03:30:22.158334+0000 (oldest deadline 2023-09-13T03:30:47.458424+0000) 2023-09-13T03:30:52.246 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:52 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:52.157+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:30:52.247 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:52 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:52.157+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:30:52.247 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:52 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:52.157+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.557932+0000 front 2023-09-13T03:30:21.558055+0000 (oldest deadline 2023-09-13T03:30:45.658031+0000) 2023-09-13T03:30:52.668 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:52 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:52.410+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:30:52.668 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:52 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:52.410+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.238945+0000 front 2023-09-13T03:30:15.239145+0000 (oldest deadline 2023-09-13T03:30:41.139111+0000) 2023-09-13T03:30:52.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:52 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:52.410+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.840328+0000 front 2023-09-13T03:30:22.840159+0000 (oldest deadline 2023-09-13T03:30:45.740226+0000) 2023-09-13T03:30:52.910 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:52 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:52.508+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:30:52.910 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:52 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:52.508+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:30:52.910 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:52 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:52.508+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:30:52.911 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:30:52 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:30:52.740+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:14.436426+0000 front 2023-09-13T03:30:14.436375+0000 (oldest deadline 2023-09-13T03:30:39.736445+0000) 2023-09-13T03:30:52.911 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:30:52 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:30:52.740+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.338362+0000 front 2023-09-13T03:30:21.338390+0000 (oldest deadline 2023-09-13T03:30:46.638525+0000) 2023-09-13T03:30:52.918 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:52 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:52.667+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.115759+0000 front 2023-09-13T03:30:07.115826+0000 (oldest deadline 2023-09-13T03:30:32.415918+0000) 2023-09-13T03:30:52.918 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:52 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:52.667+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.416578+0000 front 2023-09-13T03:30:12.416743+0000 (oldest deadline 2023-09-13T03:30:41.217924+0000) 2023-09-13T03:30:52.919 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:52 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:52.667+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.218666+0000 front 2023-09-13T03:30:21.218869+0000 (oldest deadline 2023-09-13T03:30:45.318921+0000) 2023-09-13T03:30:53.245 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:30:52 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:30:52.934+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:16.857617+0000 front 2023-09-13T03:30:16.857572+0000 (oldest deadline 2023-09-13T03:30:42.157731+0000) 2023-09-13T03:30:53.246 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:30:52 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:30:52.934+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.158371+0000 front 2023-09-13T03:30:22.158334+0000 (oldest deadline 2023-09-13T03:30:47.458424+0000) 2023-09-13T03:30:53.246 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:53 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:53.138+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:30:53.246 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:53 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:53.138+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:30:53.247 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:53 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:53.138+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.557932+0000 front 2023-09-13T03:30:21.558055+0000 (oldest deadline 2023-09-13T03:30:45.658031+0000) 2023-09-13T03:30:53.735 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:53 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:53.470+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:30:53.735 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:53 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:53.470+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:30:53.736 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:53 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:53.470+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:30:53.919 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:53 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:53.681+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.115759+0000 front 2023-09-13T03:30:07.115826+0000 (oldest deadline 2023-09-13T03:30:32.415918+0000) 2023-09-13T03:30:53.919 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:53 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:53.681+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.416578+0000 front 2023-09-13T03:30:12.416743+0000 (oldest deadline 2023-09-13T03:30:41.217924+0000) 2023-09-13T03:30:53.919 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:53 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:53.681+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.218666+0000 front 2023-09-13T03:30:21.218869+0000 (oldest deadline 2023-09-13T03:30:45.318921+0000) 2023-09-13T03:30:53.920 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:53 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:53.458+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:30:53.920 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:53 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:53.458+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.238945+0000 front 2023-09-13T03:30:15.239145+0000 (oldest deadline 2023-09-13T03:30:41.139111+0000) 2023-09-13T03:30:53.920 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:53 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:53.458+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.840328+0000 front 2023-09-13T03:30:22.840159+0000 (oldest deadline 2023-09-13T03:30:45.740226+0000) 2023-09-13T03:30:54.141 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:30:53 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:30:53.887+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:16.857617+0000 front 2023-09-13T03:30:16.857572+0000 (oldest deadline 2023-09-13T03:30:42.157731+0000) 2023-09-13T03:30:54.141 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:30:53 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:30:53.887+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.158371+0000 front 2023-09-13T03:30:22.158334+0000 (oldest deadline 2023-09-13T03:30:47.458424+0000) 2023-09-13T03:30:54.141 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:30:53 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:30:53.887+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:30.260261+0000 front 2023-09-13T03:30:30.260231+0000 (oldest deadline 2023-09-13T03:30:53.760185+0000) 2023-09-13T03:30:54.160 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:30:53 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:30:53.734+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:14.436426+0000 front 2023-09-13T03:30:14.436375+0000 (oldest deadline 2023-09-13T03:30:39.736445+0000) 2023-09-13T03:30:54.160 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:30:53 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:30:53.734+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.338362+0000 front 2023-09-13T03:30:21.338390+0000 (oldest deadline 2023-09-13T03:30:46.638525+0000) 2023-09-13T03:30:54.495 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:54 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:54.140+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:30:54.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:54 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:54.141+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:30:54.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:54 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:54.141+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.557932+0000 front 2023-09-13T03:30:21.558055+0000 (oldest deadline 2023-09-13T03:30:45.658031+0000) 2023-09-13T03:30:54.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:54 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:54.141+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:28.960471+0000 front 2023-09-13T03:30:28.960513+0000 (oldest deadline 2023-09-13T03:30:53.660585+0000) 2023-09-13T03:30:54.910 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:30:54 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:30:54.721+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:14.436426+0000 front 2023-09-13T03:30:14.436375+0000 (oldest deadline 2023-09-13T03:30:39.736445+0000) 2023-09-13T03:30:54.910 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:30:54 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:30:54.721+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.338362+0000 front 2023-09-13T03:30:21.338390+0000 (oldest deadline 2023-09-13T03:30:46.638525+0000) 2023-09-13T03:30:54.911 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:54 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:54.485+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:30:54.911 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:54 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:54.485+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:30:54.912 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:54 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:54.485+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:30:54.918 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:54 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:54.448+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:30:54.919 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:54 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:54.448+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.238945+0000 front 2023-09-13T03:30:15.239145+0000 (oldest deadline 2023-09-13T03:30:41.139111+0000) 2023-09-13T03:30:54.919 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:54 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:54.448+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.840328+0000 front 2023-09-13T03:30:22.840159+0000 (oldest deadline 2023-09-13T03:30:45.740226+0000) 2023-09-13T03:30:54.919 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:54 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:54.650+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.115759+0000 front 2023-09-13T03:30:07.115826+0000 (oldest deadline 2023-09-13T03:30:32.415918+0000) 2023-09-13T03:30:54.920 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:54 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:54.650+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.416578+0000 front 2023-09-13T03:30:12.416743+0000 (oldest deadline 2023-09-13T03:30:41.217924+0000) 2023-09-13T03:30:54.920 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:54 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:54.650+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.218666+0000 front 2023-09-13T03:30:21.218869+0000 (oldest deadline 2023-09-13T03:30:45.318921+0000) 2023-09-13T03:30:54.920 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:54 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:54.650+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.120747+0000 front 2023-09-13T03:30:31.120978+0000 (oldest deadline 2023-09-13T03:30:54.620793+0000) 2023-09-13T03:30:55.245 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:30:54 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:30:54.912+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:16.857617+0000 front 2023-09-13T03:30:16.857572+0000 (oldest deadline 2023-09-13T03:30:42.157731+0000) 2023-09-13T03:30:55.246 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:30:54 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:30:54.912+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.158371+0000 front 2023-09-13T03:30:22.158334+0000 (oldest deadline 2023-09-13T03:30:47.458424+0000) 2023-09-13T03:30:55.246 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:30:54 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:30:54.912+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:30.260261+0000 front 2023-09-13T03:30:30.260231+0000 (oldest deadline 2023-09-13T03:30:53.760185+0000) 2023-09-13T03:30:55.246 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:55 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:55.099+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:30:55.247 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:55 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:55.099+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:30:55.247 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:55 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:55.099+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.557932+0000 front 2023-09-13T03:30:21.558055+0000 (oldest deadline 2023-09-13T03:30:45.658031+0000) 2023-09-13T03:30:55.247 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:55 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:55.099+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:28.960471+0000 front 2023-09-13T03:30:28.960513+0000 (oldest deadline 2023-09-13T03:30:53.660585+0000) 2023-09-13T03:30:55.910 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:30:55 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:30:55.706+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:14.436426+0000 front 2023-09-13T03:30:14.436375+0000 (oldest deadline 2023-09-13T03:30:39.736445+0000) 2023-09-13T03:30:55.910 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:30:55 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:30:55.706+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.338362+0000 front 2023-09-13T03:30:21.338390+0000 (oldest deadline 2023-09-13T03:30:46.638525+0000) 2023-09-13T03:30:55.911 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:30:55 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:30:55.706+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.840430+0000 front 2023-09-13T03:30:31.840408+0000 (oldest deadline 2023-09-13T03:30:54.740429+0000) 2023-09-13T03:30:55.911 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:55 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:55.520+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:30:55.911 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:55 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:55.520+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:30:55.912 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:55 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:55.520+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:30:55.912 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:55 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:55.520+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.502766+0000 front 2023-09-13T03:30:31.502677+0000 (oldest deadline 2023-09-13T03:30:54.903625+0000) 2023-09-13T03:30:55.918 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:55 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:55.622+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.115759+0000 front 2023-09-13T03:30:07.115826+0000 (oldest deadline 2023-09-13T03:30:32.415918+0000) 2023-09-13T03:30:55.919 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:55 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:55.622+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.416578+0000 front 2023-09-13T03:30:12.416743+0000 (oldest deadline 2023-09-13T03:30:41.217924+0000) 2023-09-13T03:30:55.919 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:55 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:55.622+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.218666+0000 front 2023-09-13T03:30:21.218869+0000 (oldest deadline 2023-09-13T03:30:45.318921+0000) 2023-09-13T03:30:55.919 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:55 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6[136178]: 2023-09-13T03:30:55.622+0000 7fe5ee84d700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.120747+0000 front 2023-09-13T03:30:31.120978+0000 (oldest deadline 2023-09-13T03:30:54.620793+0000) 2023-09-13T03:30:55.920 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:55 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:55.456+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:30:55.920 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:55 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:55.456+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.238945+0000 front 2023-09-13T03:30:15.239145+0000 (oldest deadline 2023-09-13T03:30:41.139111+0000) 2023-09-13T03:30:55.920 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:55 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:55.456+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.840328+0000 front 2023-09-13T03:30:22.840159+0000 (oldest deadline 2023-09-13T03:30:45.740226+0000) 2023-09-13T03:30:55.920 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:55 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:55.456+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.641567+0000 front 2023-09-13T03:30:31.641612+0000 (oldest deadline 2023-09-13T03:30:55.141726+0000) 2023-09-13T03:30:56.245 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:30:55 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:30:55.911+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:16.857617+0000 front 2023-09-13T03:30:16.857572+0000 (oldest deadline 2023-09-13T03:30:42.157731+0000) 2023-09-13T03:30:56.246 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:30:55 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:30:55.911+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.158371+0000 front 2023-09-13T03:30:22.158334+0000 (oldest deadline 2023-09-13T03:30:47.458424+0000) 2023-09-13T03:30:56.246 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:30:55 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:30:55.911+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:30.260261+0000 front 2023-09-13T03:30:30.260231+0000 (oldest deadline 2023-09-13T03:30:53.760185+0000) 2023-09-13T03:30:56.247 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:56 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:56.095+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:30:56.247 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:56 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:56.095+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:30:56.247 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:56 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:56.095+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.557932+0000 front 2023-09-13T03:30:21.558055+0000 (oldest deadline 2023-09-13T03:30:45.658031+0000) 2023-09-13T03:30:56.247 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:56 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:56.095+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:28.960471+0000 front 2023-09-13T03:30:28.960513+0000 (oldest deadline 2023-09-13T03:30:53.660585+0000) 2023-09-13T03:30:56.500 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 13 03:30:56 smithi064 bash[181706]: ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-6 2023-09-13T03:30:56.501 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:56 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:56.467+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:30:56.501 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:56 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:56.467+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.238945+0000 front 2023-09-13T03:30:15.239145+0000 (oldest deadline 2023-09-13T03:30:41.139111+0000) 2023-09-13T03:30:56.501 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:56 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:56.467+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.840328+0000 front 2023-09-13T03:30:22.840159+0000 (oldest deadline 2023-09-13T03:30:45.740226+0000) 2023-09-13T03:30:56.502 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:56 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:56.467+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.641567+0000 front 2023-09-13T03:30:31.641612+0000 (oldest deadline 2023-09-13T03:30:55.141726+0000) 2023-09-13T03:30:56.910 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:30:56 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:30:56.667+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:14.436426+0000 front 2023-09-13T03:30:14.436375+0000 (oldest deadline 2023-09-13T03:30:39.736445+0000) 2023-09-13T03:30:56.910 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:30:56 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:30:56.667+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.338362+0000 front 2023-09-13T03:30:21.338390+0000 (oldest deadline 2023-09-13T03:30:46.638525+0000) 2023-09-13T03:30:56.911 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:30:56 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:30:56.667+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.840430+0000 front 2023-09-13T03:30:31.840408+0000 (oldest deadline 2023-09-13T03:30:54.740429+0000) 2023-09-13T03:30:56.911 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:56 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:56.483+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:30:56.911 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:56 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:56.483+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:30:56.912 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:56 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:56.483+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:30:56.912 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:56 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:56.483+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.502766+0000 front 2023-09-13T03:30:31.502677+0000 (oldest deadline 2023-09-13T03:30:54.903625+0000) 2023-09-13T03:30:57.245 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:30:56 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:30:56.899+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:16.857617+0000 front 2023-09-13T03:30:16.857572+0000 (oldest deadline 2023-09-13T03:30:42.157731+0000) 2023-09-13T03:30:57.246 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:30:56 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:30:56.899+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.158371+0000 front 2023-09-13T03:30:22.158334+0000 (oldest deadline 2023-09-13T03:30:47.458424+0000) 2023-09-13T03:30:57.246 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:30:56 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:30:56.899+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:30.260261+0000 front 2023-09-13T03:30:30.260231+0000 (oldest deadline 2023-09-13T03:30:53.760185+0000) 2023-09-13T03:30:57.247 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:57 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:57.122+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:30:57.247 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:57 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:57.122+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:30:57.247 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:57 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:57.122+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.557932+0000 front 2023-09-13T03:30:21.558055+0000 (oldest deadline 2023-09-13T03:30:45.658031+0000) 2023-09-13T03:30:57.248 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:57 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:57.122+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:28.960471+0000 front 2023-09-13T03:30:28.960513+0000 (oldest deadline 2023-09-13T03:30:53.660585+0000) 2023-09-13T03:30:57.668 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:57 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:57.517+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:30:57.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:57 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:57.517+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.238945+0000 front 2023-09-13T03:30:15.239145+0000 (oldest deadline 2023-09-13T03:30:41.139111+0000) 2023-09-13T03:30:57.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:57 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:57.517+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.840328+0000 front 2023-09-13T03:30:22.840159+0000 (oldest deadline 2023-09-13T03:30:45.740226+0000) 2023-09-13T03:30:57.670 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:57 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:57.517+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.641567+0000 front 2023-09-13T03:30:31.641612+0000 (oldest deadline 2023-09-13T03:30:55.141726+0000) 2023-09-13T03:30:57.910 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:57 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:57.497+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:30:57.911 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:57 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:57.497+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:30:57.911 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:57 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:57.497+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:30:57.911 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:57 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:57.497+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.502766+0000 front 2023-09-13T03:30:31.502677+0000 (oldest deadline 2023-09-13T03:30:54.903625+0000) 2023-09-13T03:30:57.912 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:30:57 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:30:57.628+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:14.436426+0000 front 2023-09-13T03:30:14.436375+0000 (oldest deadline 2023-09-13T03:30:39.736445+0000) 2023-09-13T03:30:57.912 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:30:57 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:30:57.628+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.338362+0000 front 2023-09-13T03:30:21.338390+0000 (oldest deadline 2023-09-13T03:30:46.638525+0000) 2023-09-13T03:30:57.912 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:30:57 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:30:57.628+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.840430+0000 front 2023-09-13T03:30:31.840408+0000 (oldest deadline 2023-09-13T03:30:54.740429+0000) 2023-09-13T03:30:58.245 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:30:57 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:30:57.886+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:16.857617+0000 front 2023-09-13T03:30:16.857572+0000 (oldest deadline 2023-09-13T03:30:42.157731+0000) 2023-09-13T03:30:58.246 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:30:57 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:30:57.886+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.158371+0000 front 2023-09-13T03:30:22.158334+0000 (oldest deadline 2023-09-13T03:30:47.458424+0000) 2023-09-13T03:30:58.246 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:30:57 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:30:57.886+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:30.260261+0000 front 2023-09-13T03:30:30.260231+0000 (oldest deadline 2023-09-13T03:30:53.760185+0000) 2023-09-13T03:30:58.247 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:58 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:58.074+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:30:58.247 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:58 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:58.074+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:30:58.247 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:58 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:58.074+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.557932+0000 front 2023-09-13T03:30:21.558055+0000 (oldest deadline 2023-09-13T03:30:45.658031+0000) 2023-09-13T03:30:58.248 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:58 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:58.074+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:28.960471+0000 front 2023-09-13T03:30:28.960513+0000 (oldest deadline 2023-09-13T03:30:53.660585+0000) 2023-09-13T03:30:58.257 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@osd.6.service' 2023-09-13T03:30:58.348 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T03:30:58.349 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-09-13T03:30:58.349 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-09-13T03:30:58.350 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@osd.7 2023-09-13T03:30:58.668 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:58 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:58.470+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:30:58.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:58 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:58.470+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.238945+0000 front 2023-09-13T03:30:15.239145+0000 (oldest deadline 2023-09-13T03:30:41.139111+0000) 2023-09-13T03:30:58.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:58 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:58.470+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.840328+0000 front 2023-09-13T03:30:22.840159+0000 (oldest deadline 2023-09-13T03:30:45.740226+0000) 2023-09-13T03:30:58.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:58 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:58.470+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.641567+0000 front 2023-09-13T03:30:31.641612+0000 (oldest deadline 2023-09-13T03:30:55.141726+0000) 2023-09-13T03:30:58.702 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:30:58 smithi142 systemd[1]: Stopping Ceph osd.7 for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:30:58.910 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:30:58 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:30:58.629+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:14.436426+0000 front 2023-09-13T03:30:14.436375+0000 (oldest deadline 2023-09-13T03:30:39.736445+0000) 2023-09-13T03:30:58.910 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:30:58 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:30:58.629+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.338362+0000 front 2023-09-13T03:30:21.338390+0000 (oldest deadline 2023-09-13T03:30:46.638525+0000) 2023-09-13T03:30:58.911 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:30:58 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:30:58.629+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.840430+0000 front 2023-09-13T03:30:31.840408+0000 (oldest deadline 2023-09-13T03:30:54.740429+0000) 2023-09-13T03:30:58.911 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:58 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:58.539+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:30:58.912 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:58 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:58.539+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:30:58.912 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:58 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:58.539+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:30:58.912 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:58 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:58.539+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.502766+0000 front 2023-09-13T03:30:31.502677+0000 (oldest deadline 2023-09-13T03:30:54.903625+0000) 2023-09-13T03:30:58.995 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:30:58 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:30:58.702+0000 7f435ae34700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T03:30:58.996 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:30:58 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:30:58.702+0000 7f435ae34700 -1 osd.7 109 *** Got signal Terminated *** 2023-09-13T03:30:58.996 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:30:58 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:30:58.702+0000 7f435ae34700 -1 osd.7 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-13T03:30:58.996 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:30:58 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:30:58.927+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:16.857617+0000 front 2023-09-13T03:30:16.857572+0000 (oldest deadline 2023-09-13T03:30:42.157731+0000) 2023-09-13T03:30:58.997 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:30:58 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:30:58.927+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.158371+0000 front 2023-09-13T03:30:22.158334+0000 (oldest deadline 2023-09-13T03:30:47.458424+0000) 2023-09-13T03:30:58.997 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:30:58 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:30:58.927+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:30.260261+0000 front 2023-09-13T03:30:30.260231+0000 (oldest deadline 2023-09-13T03:30:53.760185+0000) 2023-09-13T03:30:59.495 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:59 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:59.056+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:30:59.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:59 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:59.057+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:30:59.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:59 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:59.057+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.557932+0000 front 2023-09-13T03:30:21.558055+0000 (oldest deadline 2023-09-13T03:30:45.658031+0000) 2023-09-13T03:30:59.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:30:59 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:30:59.057+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:28.960471+0000 front 2023-09-13T03:30:28.960513+0000 (oldest deadline 2023-09-13T03:30:53.660585+0000) 2023-09-13T03:30:59.910 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:30:59 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:30:59.666+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:14.436426+0000 front 2023-09-13T03:30:14.436375+0000 (oldest deadline 2023-09-13T03:30:39.736445+0000) 2023-09-13T03:30:59.911 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:30:59 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:30:59.666+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.338362+0000 front 2023-09-13T03:30:21.338390+0000 (oldest deadline 2023-09-13T03:30:46.638525+0000) 2023-09-13T03:30:59.911 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:30:59 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:30:59.666+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.840430+0000 front 2023-09-13T03:30:31.840408+0000 (oldest deadline 2023-09-13T03:30:54.740429+0000) 2023-09-13T03:30:59.911 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:59 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:59.580+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:30:59.912 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:59 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:59.580+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:30:59.912 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:59 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:59.580+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:30:59.912 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:30:59 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:30:59.580+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.502766+0000 front 2023-09-13T03:30:31.502677+0000 (oldest deadline 2023-09-13T03:30:54.903625+0000) 2023-09-13T03:30:59.918 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:59 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:59.477+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:30:59.918 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:59 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:59.477+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.238945+0000 front 2023-09-13T03:30:15.239145+0000 (oldest deadline 2023-09-13T03:30:41.139111+0000) 2023-09-13T03:30:59.919 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:59 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:59.477+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.840328+0000 front 2023-09-13T03:30:22.840159+0000 (oldest deadline 2023-09-13T03:30:45.740226+0000) 2023-09-13T03:30:59.919 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:30:59 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:30:59.477+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.641567+0000 front 2023-09-13T03:30:31.641612+0000 (oldest deadline 2023-09-13T03:30:55.141726+0000) 2023-09-13T03:31:00.245 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:30:59 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:30:59.910+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:16.857617+0000 front 2023-09-13T03:30:16.857572+0000 (oldest deadline 2023-09-13T03:30:42.157731+0000) 2023-09-13T03:31:00.246 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:30:59 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:30:59.910+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.158371+0000 front 2023-09-13T03:30:22.158334+0000 (oldest deadline 2023-09-13T03:30:47.458424+0000) 2023-09-13T03:31:00.246 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:30:59 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:30:59.910+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:30.260261+0000 front 2023-09-13T03:30:30.260231+0000 (oldest deadline 2023-09-13T03:30:53.760185+0000) 2023-09-13T03:31:00.247 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:00 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:00.013+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:31:00.247 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:00 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:00.013+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:31:00.247 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:00 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:00.013+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.557932+0000 front 2023-09-13T03:30:21.558055+0000 (oldest deadline 2023-09-13T03:30:45.658031+0000) 2023-09-13T03:31:00.247 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:00 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:00.013+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:28.960471+0000 front 2023-09-13T03:30:28.960513+0000 (oldest deadline 2023-09-13T03:30:53.660585+0000) 2023-09-13T03:31:00.910 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:00 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:00.536+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:31:00.910 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:00 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:00.536+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:31:00.911 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:00 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:00.536+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:31:00.911 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:00 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:00.536+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.502766+0000 front 2023-09-13T03:30:31.502677+0000 (oldest deadline 2023-09-13T03:30:54.903625+0000) 2023-09-13T03:31:00.911 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:00 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:00.677+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:14.436426+0000 front 2023-09-13T03:30:14.436375+0000 (oldest deadline 2023-09-13T03:30:39.736445+0000) 2023-09-13T03:31:00.912 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:00 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:00.677+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.338362+0000 front 2023-09-13T03:30:21.338390+0000 (oldest deadline 2023-09-13T03:30:46.638525+0000) 2023-09-13T03:31:00.912 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:00 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:00.677+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.840430+0000 front 2023-09-13T03:30:31.840408+0000 (oldest deadline 2023-09-13T03:30:54.740429+0000) 2023-09-13T03:31:00.918 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:00 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:00.430+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:31:00.918 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:00 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:00.430+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.238945+0000 front 2023-09-13T03:30:15.239145+0000 (oldest deadline 2023-09-13T03:30:41.139111+0000) 2023-09-13T03:31:00.919 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:00 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:00.430+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.840328+0000 front 2023-09-13T03:30:22.840159+0000 (oldest deadline 2023-09-13T03:30:45.740226+0000) 2023-09-13T03:31:00.919 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:00 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:00.430+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.641567+0000 front 2023-09-13T03:30:31.641612+0000 (oldest deadline 2023-09-13T03:30:55.141726+0000) 2023-09-13T03:31:01.245 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:31:00 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:31:00.888+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:16.857617+0000 front 2023-09-13T03:30:16.857572+0000 (oldest deadline 2023-09-13T03:30:42.157731+0000) 2023-09-13T03:31:01.246 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:31:00 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:31:00.888+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.158371+0000 front 2023-09-13T03:30:22.158334+0000 (oldest deadline 2023-09-13T03:30:47.458424+0000) 2023-09-13T03:31:01.246 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:31:00 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:31:00.888+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:30.260261+0000 front 2023-09-13T03:30:30.260231+0000 (oldest deadline 2023-09-13T03:30:53.760185+0000) 2023-09-13T03:31:01.246 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:01 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:01.044+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:31:01.247 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:01 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:01.044+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:31:01.247 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:01 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:01.044+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.557932+0000 front 2023-09-13T03:30:21.558055+0000 (oldest deadline 2023-09-13T03:30:45.658031+0000) 2023-09-13T03:31:01.247 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:01 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:01.044+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:28.960471+0000 front 2023-09-13T03:30:28.960513+0000 (oldest deadline 2023-09-13T03:30:53.660585+0000) 2023-09-13T03:31:01.910 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:01 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:01.552+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:31:01.911 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:01 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:01.552+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:31:01.911 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:01 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:01.552+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:31:01.911 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:01 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:01.552+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.502766+0000 front 2023-09-13T03:30:31.502677+0000 (oldest deadline 2023-09-13T03:30:54.903625+0000) 2023-09-13T03:31:01.912 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:01 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:01.697+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:14.436426+0000 front 2023-09-13T03:30:14.436375+0000 (oldest deadline 2023-09-13T03:30:39.736445+0000) 2023-09-13T03:31:01.912 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:01 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:01.697+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.338362+0000 front 2023-09-13T03:30:21.338390+0000 (oldest deadline 2023-09-13T03:30:46.638525+0000) 2023-09-13T03:31:01.912 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:01 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:01.697+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.840430+0000 front 2023-09-13T03:30:31.840408+0000 (oldest deadline 2023-09-13T03:30:54.740429+0000) 2023-09-13T03:31:01.912 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:01 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:01.697+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:36.441959+0000 front 2023-09-13T03:30:36.441816+0000 (oldest deadline 2023-09-13T03:31:01.642671+0000) 2023-09-13T03:31:01.918 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:01 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:01.427+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:31:01.918 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:01 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:01.427+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.238945+0000 front 2023-09-13T03:30:15.239145+0000 (oldest deadline 2023-09-13T03:30:41.139111+0000) 2023-09-13T03:31:01.919 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:01 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:01.427+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.840328+0000 front 2023-09-13T03:30:22.840159+0000 (oldest deadline 2023-09-13T03:30:45.740226+0000) 2023-09-13T03:31:01.919 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:01 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:01.427+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.641567+0000 front 2023-09-13T03:30:31.641612+0000 (oldest deadline 2023-09-13T03:30:55.141726+0000) 2023-09-13T03:31:02.245 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:31:01 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:31:01.873+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:16.857617+0000 front 2023-09-13T03:30:16.857572+0000 (oldest deadline 2023-09-13T03:30:42.157731+0000) 2023-09-13T03:31:02.246 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:31:01 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:31:01.873+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.158371+0000 front 2023-09-13T03:30:22.158334+0000 (oldest deadline 2023-09-13T03:30:47.458424+0000) 2023-09-13T03:31:02.246 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:31:01 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:31:01.873+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:30.260261+0000 front 2023-09-13T03:30:30.260231+0000 (oldest deadline 2023-09-13T03:30:53.760185+0000) 2023-09-13T03:31:02.246 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:02 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:02.094+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:31:02.247 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:02 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:02.094+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:31:02.247 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:02 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:02.094+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.557932+0000 front 2023-09-13T03:30:21.558055+0000 (oldest deadline 2023-09-13T03:30:45.658031+0000) 2023-09-13T03:31:02.247 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:02 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:02.094+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:28.960471+0000 front 2023-09-13T03:30:28.960513+0000 (oldest deadline 2023-09-13T03:30:53.660585+0000) 2023-09-13T03:31:02.248 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:02 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:02.094+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:35.961436+0000 front 2023-09-13T03:30:35.961450+0000 (oldest deadline 2023-09-13T03:31:01.861557+0000) 2023-09-13T03:31:02.910 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:02 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:02.553+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:31:02.910 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:02 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:02.553+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:31:02.911 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:02 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:02.553+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:31:02.911 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:02 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:02.553+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.502766+0000 front 2023-09-13T03:30:31.502677+0000 (oldest deadline 2023-09-13T03:30:54.903625+0000) 2023-09-13T03:31:02.912 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:02 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:02.739+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:14.436426+0000 front 2023-09-13T03:30:14.436375+0000 (oldest deadline 2023-09-13T03:30:39.736445+0000) 2023-09-13T03:31:02.912 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:02 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:02.739+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.338362+0000 front 2023-09-13T03:30:21.338390+0000 (oldest deadline 2023-09-13T03:30:46.638525+0000) 2023-09-13T03:31:02.913 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:02 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:02.739+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.840430+0000 front 2023-09-13T03:30:31.840408+0000 (oldest deadline 2023-09-13T03:30:54.740429+0000) 2023-09-13T03:31:02.913 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:02 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:02.739+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:36.441959+0000 front 2023-09-13T03:30:36.441816+0000 (oldest deadline 2023-09-13T03:31:01.642671+0000) 2023-09-13T03:31:02.918 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:02 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:02.445+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:31:02.918 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:02 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:02.445+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.238945+0000 front 2023-09-13T03:30:15.239145+0000 (oldest deadline 2023-09-13T03:30:41.139111+0000) 2023-09-13T03:31:02.919 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:02 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:02.445+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.840328+0000 front 2023-09-13T03:30:22.840159+0000 (oldest deadline 2023-09-13T03:30:45.740226+0000) 2023-09-13T03:31:02.919 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:02 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:02.445+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.641567+0000 front 2023-09-13T03:30:31.641612+0000 (oldest deadline 2023-09-13T03:30:55.141726+0000) 2023-09-13T03:31:03.245 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:31:02 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:31:02.896+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:16.857617+0000 front 2023-09-13T03:30:16.857572+0000 (oldest deadline 2023-09-13T03:30:42.157731+0000) 2023-09-13T03:31:03.246 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:31:02 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:31:02.896+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.158371+0000 front 2023-09-13T03:30:22.158334+0000 (oldest deadline 2023-09-13T03:30:47.458424+0000) 2023-09-13T03:31:03.246 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:31:02 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:31:02.896+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:30.260261+0000 front 2023-09-13T03:30:30.260231+0000 (oldest deadline 2023-09-13T03:30:53.760185+0000) 2023-09-13T03:31:03.247 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:31:02 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7[155500]: 2023-09-13T03:31:02.896+0000 7f4355c27700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:37.762008+0000 front 2023-09-13T03:30:37.762047+0000 (oldest deadline 2023-09-13T03:31:02.462261+0000) 2023-09-13T03:31:03.247 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:03 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:03.057+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:31:03.247 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:03 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:03.057+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:31:03.248 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:03 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:03.057+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.557932+0000 front 2023-09-13T03:30:21.558055+0000 (oldest deadline 2023-09-13T03:30:45.658031+0000) 2023-09-13T03:31:03.248 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:03 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:03.057+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:28.960471+0000 front 2023-09-13T03:30:28.960513+0000 (oldest deadline 2023-09-13T03:30:53.660585+0000) 2023-09-13T03:31:03.248 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:03 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:03.057+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:35.961436+0000 front 2023-09-13T03:30:35.961450+0000 (oldest deadline 2023-09-13T03:31:01.861557+0000) 2023-09-13T03:31:03.668 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:03 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:03.402+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:31:03.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:03 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:03.402+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.238945+0000 front 2023-09-13T03:30:15.239145+0000 (oldest deadline 2023-09-13T03:30:41.139111+0000) 2023-09-13T03:31:03.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:03 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:03.402+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.840328+0000 front 2023-09-13T03:30:22.840159+0000 (oldest deadline 2023-09-13T03:30:45.740226+0000) 2023-09-13T03:31:03.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:03 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:03.402+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.641567+0000 front 2023-09-13T03:30:31.641612+0000 (oldest deadline 2023-09-13T03:30:55.141726+0000) 2023-09-13T03:31:03.789 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:03 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:03.508+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:31:03.789 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:03 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:03.508+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:31:03.790 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:03 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:03.508+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:31:03.790 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:03 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:03.508+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.502766+0000 front 2023-09-13T03:30:31.502677+0000 (oldest deadline 2023-09-13T03:30:54.903625+0000) 2023-09-13T03:31:04.160 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:03 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:03.789+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:14.436426+0000 front 2023-09-13T03:30:14.436375+0000 (oldest deadline 2023-09-13T03:30:39.736445+0000) 2023-09-13T03:31:04.161 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:03 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:03.789+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.338362+0000 front 2023-09-13T03:30:21.338390+0000 (oldest deadline 2023-09-13T03:30:46.638525+0000) 2023-09-13T03:31:04.161 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:03 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:03.789+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.840430+0000 front 2023-09-13T03:30:31.840408+0000 (oldest deadline 2023-09-13T03:30:54.740429+0000) 2023-09-13T03:31:04.161 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:03 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:03.789+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:36.441959+0000 front 2023-09-13T03:30:36.441816+0000 (oldest deadline 2023-09-13T03:31:01.642671+0000) 2023-09-13T03:31:04.484 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:04 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:04.079+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:31:04.484 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:04 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:04.079+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:31:04.485 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:04 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:04.079+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.557932+0000 front 2023-09-13T03:30:21.558055+0000 (oldest deadline 2023-09-13T03:30:45.658031+0000) 2023-09-13T03:31:04.485 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:04 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:04.079+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:28.960471+0000 front 2023-09-13T03:30:28.960513+0000 (oldest deadline 2023-09-13T03:30:53.660585+0000) 2023-09-13T03:31:04.485 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:04 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:04.079+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:35.961436+0000 front 2023-09-13T03:30:35.961450+0000 (oldest deadline 2023-09-13T03:31:01.861557+0000) 2023-09-13T03:31:04.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:04 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:04.376+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:31:04.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:04 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:04.376+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.238945+0000 front 2023-09-13T03:30:15.239145+0000 (oldest deadline 2023-09-13T03:30:41.139111+0000) 2023-09-13T03:31:04.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:04 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:04.376+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.840328+0000 front 2023-09-13T03:30:22.840159+0000 (oldest deadline 2023-09-13T03:30:45.740226+0000) 2023-09-13T03:31:04.670 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:04 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:04.376+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.641567+0000 front 2023-09-13T03:30:31.641612+0000 (oldest deadline 2023-09-13T03:30:55.141726+0000) 2023-09-13T03:31:04.745 INFO:journalctl@ceph.osd.7.smithi142.stdout:Sep 13 03:31:04 smithi142 bash[179447]: ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-7 2023-09-13T03:31:04.797 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:04 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:04.495+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:31:04.798 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:04 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:04.495+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:31:04.798 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:04 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:04.495+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:31:04.799 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:04 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:04.495+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.502766+0000 front 2023-09-13T03:30:31.502677+0000 (oldest deadline 2023-09-13T03:30:54.903625+0000) 2023-09-13T03:31:05.160 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:04 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:04.797+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:14.436426+0000 front 2023-09-13T03:30:14.436375+0000 (oldest deadline 2023-09-13T03:30:39.736445+0000) 2023-09-13T03:31:05.160 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:04 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:04.797+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.338362+0000 front 2023-09-13T03:30:21.338390+0000 (oldest deadline 2023-09-13T03:30:46.638525+0000) 2023-09-13T03:31:05.161 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:04 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:04.797+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.840430+0000 front 2023-09-13T03:30:31.840408+0000 (oldest deadline 2023-09-13T03:30:54.740429+0000) 2023-09-13T03:31:05.161 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:04 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:04.797+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:36.441959+0000 front 2023-09-13T03:30:36.441816+0000 (oldest deadline 2023-09-13T03:31:01.642671+0000) 2023-09-13T03:31:05.447 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:05 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:05.035+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:31:05.447 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:05 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:05.035+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:31:05.447 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:05 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:05.035+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.557932+0000 front 2023-09-13T03:30:21.558055+0000 (oldest deadline 2023-09-13T03:30:45.658031+0000) 2023-09-13T03:31:05.448 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:05 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:05.035+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:28.960471+0000 front 2023-09-13T03:30:28.960513+0000 (oldest deadline 2023-09-13T03:30:53.660585+0000) 2023-09-13T03:31:05.448 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:05 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:05.035+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:35.961436+0000 front 2023-09-13T03:30:35.961450+0000 (oldest deadline 2023-09-13T03:31:01.861557+0000) 2023-09-13T03:31:05.668 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:05 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:05.359+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:31:05.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:05 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:05.359+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.238945+0000 front 2023-09-13T03:30:15.239145+0000 (oldest deadline 2023-09-13T03:30:41.139111+0000) 2023-09-13T03:31:05.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:05 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:05.359+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.840328+0000 front 2023-09-13T03:30:22.840159+0000 (oldest deadline 2023-09-13T03:30:45.740226+0000) 2023-09-13T03:31:05.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:05 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:05.359+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.641567+0000 front 2023-09-13T03:30:31.641612+0000 (oldest deadline 2023-09-13T03:30:55.141726+0000) 2023-09-13T03:31:05.670 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:05 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:05.359+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:39.842954+0000 front 2023-09-13T03:30:39.842992+0000 (oldest deadline 2023-09-13T03:31:05.142850+0000) 2023-09-13T03:31:05.751 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@osd.7.service' 2023-09-13T03:31:05.792 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T03:31:05.792 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-09-13T03:31:05.793 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-09-13T03:31:05.793 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@osd.8 2023-09-13T03:31:05.825 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:05 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:05.449+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:31:05.826 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:05 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:05.449+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:31:05.826 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:05 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:05.449+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:31:05.826 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:05 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:05.449+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.502766+0000 front 2023-09-13T03:30:31.502677+0000 (oldest deadline 2023-09-13T03:30:54.903625+0000) 2023-09-13T03:31:05.827 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:05 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:05.449+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:37.105618+0000 front 2023-09-13T03:30:37.105683+0000 (oldest deadline 2023-09-13T03:31:05.306446+0000) 2023-09-13T03:31:06.151 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:05 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:05.832+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:14.436426+0000 front 2023-09-13T03:30:14.436375+0000 (oldest deadline 2023-09-13T03:30:39.736445+0000) 2023-09-13T03:31:06.152 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:05 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:05.832+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.338362+0000 front 2023-09-13T03:30:21.338390+0000 (oldest deadline 2023-09-13T03:30:46.638525+0000) 2023-09-13T03:31:06.152 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:05 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:05.832+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.840430+0000 front 2023-09-13T03:30:31.840408+0000 (oldest deadline 2023-09-13T03:30:54.740429+0000) 2023-09-13T03:31:06.152 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:05 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:05.832+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:36.441959+0000 front 2023-09-13T03:30:36.441816+0000 (oldest deadline 2023-09-13T03:31:01.642671+0000) 2023-09-13T03:31:06.153 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:05 smithi170 systemd[1]: Stopping Ceph osd.8 for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:31:06.153 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:06 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:06.151+0000 7fa0cd228700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T03:31:06.153 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:06 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:06.151+0000 7fa0cd228700 -1 osd.8 109 *** Got signal Terminated *** 2023-09-13T03:31:06.410 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:06 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:06.151+0000 7fa0cd228700 -1 osd.8 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-13T03:31:06.495 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:06 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:06.029+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:31:06.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:06 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:06.029+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:31:06.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:06 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:06.029+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.557932+0000 front 2023-09-13T03:30:21.558055+0000 (oldest deadline 2023-09-13T03:30:45.658031+0000) 2023-09-13T03:31:06.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:06 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:06.029+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:28.960471+0000 front 2023-09-13T03:30:28.960513+0000 (oldest deadline 2023-09-13T03:30:53.660585+0000) 2023-09-13T03:31:06.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:06 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:06.029+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:35.961436+0000 front 2023-09-13T03:30:35.961450+0000 (oldest deadline 2023-09-13T03:31:01.861557+0000) 2023-09-13T03:31:06.668 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:06 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:06.352+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:31:06.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:06 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:06.352+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.238945+0000 front 2023-09-13T03:30:15.239145+0000 (oldest deadline 2023-09-13T03:30:41.139111+0000) 2023-09-13T03:31:06.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:06 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:06.352+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.840328+0000 front 2023-09-13T03:30:22.840159+0000 (oldest deadline 2023-09-13T03:30:45.740226+0000) 2023-09-13T03:31:06.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:06 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:06.352+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.641567+0000 front 2023-09-13T03:30:31.641612+0000 (oldest deadline 2023-09-13T03:30:55.141726+0000) 2023-09-13T03:31:06.670 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:06 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:06.352+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:39.842954+0000 front 2023-09-13T03:30:39.842992+0000 (oldest deadline 2023-09-13T03:31:05.142850+0000) 2023-09-13T03:31:06.855 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:06 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:06.429+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:31:06.856 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:06 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:06.429+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:31:06.856 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:06 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:06.429+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:31:06.857 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:06 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:06.429+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.502766+0000 front 2023-09-13T03:30:31.502677+0000 (oldest deadline 2023-09-13T03:30:54.903625+0000) 2023-09-13T03:31:06.857 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:06 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:06.429+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:37.105618+0000 front 2023-09-13T03:30:37.105683+0000 (oldest deadline 2023-09-13T03:31:05.306446+0000) 2023-09-13T03:31:07.160 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:06 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:06.855+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:14.436426+0000 front 2023-09-13T03:30:14.436375+0000 (oldest deadline 2023-09-13T03:30:39.736445+0000) 2023-09-13T03:31:07.160 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:06 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:06.855+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.338362+0000 front 2023-09-13T03:30:21.338390+0000 (oldest deadline 2023-09-13T03:30:46.638525+0000) 2023-09-13T03:31:07.161 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:06 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:06.855+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.840430+0000 front 2023-09-13T03:30:31.840408+0000 (oldest deadline 2023-09-13T03:30:54.740429+0000) 2023-09-13T03:31:07.161 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:06 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:06.855+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:36.441959+0000 front 2023-09-13T03:30:36.441816+0000 (oldest deadline 2023-09-13T03:31:01.642671+0000) 2023-09-13T03:31:07.245 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:06 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:06.986+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:31:07.246 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:06 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:06.986+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:31:07.246 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:06 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:06.986+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.557932+0000 front 2023-09-13T03:30:21.558055+0000 (oldest deadline 2023-09-13T03:30:45.658031+0000) 2023-09-13T03:31:07.246 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:06 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:06.986+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:28.960471+0000 front 2023-09-13T03:30:28.960513+0000 (oldest deadline 2023-09-13T03:30:53.660585+0000) 2023-09-13T03:31:07.247 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:06 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:06.986+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:35.961436+0000 front 2023-09-13T03:30:35.961450+0000 (oldest deadline 2023-09-13T03:31:01.861557+0000) 2023-09-13T03:31:07.660 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:07 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:07.398+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:31:07.660 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:07 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:07.398+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:31:07.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:07 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:07.398+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:31:07.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:07 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:07.398+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.502766+0000 front 2023-09-13T03:30:31.502677+0000 (oldest deadline 2023-09-13T03:30:54.903625+0000) 2023-09-13T03:31:07.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:07 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:07.398+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:37.105618+0000 front 2023-09-13T03:30:37.105683+0000 (oldest deadline 2023-09-13T03:31:05.306446+0000) 2023-09-13T03:31:07.668 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:07 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:07.401+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:31:07.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:07 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:07.401+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.238945+0000 front 2023-09-13T03:30:15.239145+0000 (oldest deadline 2023-09-13T03:30:41.139111+0000) 2023-09-13T03:31:07.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:07 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:07.401+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.840328+0000 front 2023-09-13T03:30:22.840159+0000 (oldest deadline 2023-09-13T03:30:45.740226+0000) 2023-09-13T03:31:07.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:07 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:07.401+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.641567+0000 front 2023-09-13T03:30:31.641612+0000 (oldest deadline 2023-09-13T03:30:55.141726+0000) 2023-09-13T03:31:07.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:07 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:07.401+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:39.842954+0000 front 2023-09-13T03:30:39.842992+0000 (oldest deadline 2023-09-13T03:31:05.142850+0000) 2023-09-13T03:31:08.160 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:07 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:07.845+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:14.436426+0000 front 2023-09-13T03:30:14.436375+0000 (oldest deadline 2023-09-13T03:30:39.736445+0000) 2023-09-13T03:31:08.160 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:07 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:07.845+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.338362+0000 front 2023-09-13T03:30:21.338390+0000 (oldest deadline 2023-09-13T03:30:46.638525+0000) 2023-09-13T03:31:08.161 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:07 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:07.845+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.840430+0000 front 2023-09-13T03:30:31.840408+0000 (oldest deadline 2023-09-13T03:30:54.740429+0000) 2023-09-13T03:31:08.161 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:07 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:07.845+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:36.441959+0000 front 2023-09-13T03:30:36.441816+0000 (oldest deadline 2023-09-13T03:31:01.642671+0000) 2023-09-13T03:31:08.245 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:07 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:07.978+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:31:08.246 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:07 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:07.978+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:31:08.246 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:07 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:07.978+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.557932+0000 front 2023-09-13T03:30:21.558055+0000 (oldest deadline 2023-09-13T03:30:45.658031+0000) 2023-09-13T03:31:08.246 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:07 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:07.978+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:28.960471+0000 front 2023-09-13T03:30:28.960513+0000 (oldest deadline 2023-09-13T03:30:53.660585+0000) 2023-09-13T03:31:08.247 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:07 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:07.978+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:35.961436+0000 front 2023-09-13T03:30:35.961450+0000 (oldest deadline 2023-09-13T03:31:01.861557+0000) 2023-09-13T03:31:08.660 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:08 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:08.371+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:31:08.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:08 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:08.371+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:31:08.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:08 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:08.371+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:31:08.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:08 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:08.371+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.502766+0000 front 2023-09-13T03:30:31.502677+0000 (oldest deadline 2023-09-13T03:30:54.903625+0000) 2023-09-13T03:31:08.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:08 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:08.371+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:37.105618+0000 front 2023-09-13T03:30:37.105683+0000 (oldest deadline 2023-09-13T03:31:05.306446+0000) 2023-09-13T03:31:08.668 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:08 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:08.389+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:31:08.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:08 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:08.389+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.238945+0000 front 2023-09-13T03:30:15.239145+0000 (oldest deadline 2023-09-13T03:30:41.139111+0000) 2023-09-13T03:31:08.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:08 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:08.389+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.840328+0000 front 2023-09-13T03:30:22.840159+0000 (oldest deadline 2023-09-13T03:30:45.740226+0000) 2023-09-13T03:31:08.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:08 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:08.389+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.641567+0000 front 2023-09-13T03:30:31.641612+0000 (oldest deadline 2023-09-13T03:30:55.141726+0000) 2023-09-13T03:31:08.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:08 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:08.389+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:39.842954+0000 front 2023-09-13T03:30:39.842992+0000 (oldest deadline 2023-09-13T03:31:05.142850+0000) 2023-09-13T03:31:09.160 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:08 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:08.819+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:14.436426+0000 front 2023-09-13T03:30:14.436375+0000 (oldest deadline 2023-09-13T03:30:39.736445+0000) 2023-09-13T03:31:09.160 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:08 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:08.819+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.338362+0000 front 2023-09-13T03:30:21.338390+0000 (oldest deadline 2023-09-13T03:30:46.638525+0000) 2023-09-13T03:31:09.161 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:08 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:08.819+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.840430+0000 front 2023-09-13T03:30:31.840408+0000 (oldest deadline 2023-09-13T03:30:54.740429+0000) 2023-09-13T03:31:09.161 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:08 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:08.819+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:36.441959+0000 front 2023-09-13T03:30:36.441816+0000 (oldest deadline 2023-09-13T03:31:01.642671+0000) 2023-09-13T03:31:09.245 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:08 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:08.984+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:31:09.246 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:08 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:08.984+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:31:09.246 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:08 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:08.984+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.557932+0000 front 2023-09-13T03:30:21.558055+0000 (oldest deadline 2023-09-13T03:30:45.658031+0000) 2023-09-13T03:31:09.246 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:08 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:08.984+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:28.960471+0000 front 2023-09-13T03:30:28.960513+0000 (oldest deadline 2023-09-13T03:30:53.660585+0000) 2023-09-13T03:31:09.247 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:08 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:08.984+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:35.961436+0000 front 2023-09-13T03:30:35.961450+0000 (oldest deadline 2023-09-13T03:31:01.861557+0000) 2023-09-13T03:31:09.660 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:09 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:09.377+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:31:09.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:09 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:09.377+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:31:09.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:09 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:09.377+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:31:09.662 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:09 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:09.377+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.502766+0000 front 2023-09-13T03:30:31.502677+0000 (oldest deadline 2023-09-13T03:30:54.903625+0000) 2023-09-13T03:31:09.662 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:09 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:09.377+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:37.105618+0000 front 2023-09-13T03:30:37.105683+0000 (oldest deadline 2023-09-13T03:31:05.306446+0000) 2023-09-13T03:31:09.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:09 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:09.399+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:31:09.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:09 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:09.399+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.238945+0000 front 2023-09-13T03:30:15.239145+0000 (oldest deadline 2023-09-13T03:30:41.139111+0000) 2023-09-13T03:31:09.670 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:09 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:09.399+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.840328+0000 front 2023-09-13T03:30:22.840159+0000 (oldest deadline 2023-09-13T03:30:45.740226+0000) 2023-09-13T03:31:09.670 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:09 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:09.399+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.641567+0000 front 2023-09-13T03:30:31.641612+0000 (oldest deadline 2023-09-13T03:30:55.141726+0000) 2023-09-13T03:31:09.670 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:09 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:09.399+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:39.842954+0000 front 2023-09-13T03:30:39.842992+0000 (oldest deadline 2023-09-13T03:31:05.142850+0000) 2023-09-13T03:31:09.671 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:09 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:09.399+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:45.143359+0000 front 2023-09-13T03:30:45.143405+0000 (oldest deadline 2023-09-13T03:31:08.643469+0000) 2023-09-13T03:31:10.160 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:09 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:09.790+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:14.436426+0000 front 2023-09-13T03:30:14.436375+0000 (oldest deadline 2023-09-13T03:30:39.736445+0000) 2023-09-13T03:31:10.161 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:09 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:09.790+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.338362+0000 front 2023-09-13T03:30:21.338390+0000 (oldest deadline 2023-09-13T03:30:46.638525+0000) 2023-09-13T03:31:10.161 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:09 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:09.790+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.840430+0000 front 2023-09-13T03:30:31.840408+0000 (oldest deadline 2023-09-13T03:30:54.740429+0000) 2023-09-13T03:31:10.161 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:09 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:09.790+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:36.441959+0000 front 2023-09-13T03:30:36.441816+0000 (oldest deadline 2023-09-13T03:31:01.642671+0000) 2023-09-13T03:31:10.162 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:09 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:09.790+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:45.143669+0000 front 2023-09-13T03:30:45.143796+0000 (oldest deadline 2023-09-13T03:31:09.244616+0000) 2023-09-13T03:31:10.495 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:10 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:10.027+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:31:10.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:10 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:10.027+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:31:10.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:10 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:10.027+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.557932+0000 front 2023-09-13T03:30:21.558055+0000 (oldest deadline 2023-09-13T03:30:45.658031+0000) 2023-09-13T03:31:10.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:10 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:10.027+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:28.960471+0000 front 2023-09-13T03:30:28.960513+0000 (oldest deadline 2023-09-13T03:30:53.660585+0000) 2023-09-13T03:31:10.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:10 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:10.027+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:35.961436+0000 front 2023-09-13T03:30:35.961450+0000 (oldest deadline 2023-09-13T03:31:01.861557+0000) 2023-09-13T03:31:10.660 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:10 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:10.338+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:31:10.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:10 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:10.338+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:31:10.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:10 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:10.338+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:31:10.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:10 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:10.338+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.502766+0000 front 2023-09-13T03:30:31.502677+0000 (oldest deadline 2023-09-13T03:30:54.903625+0000) 2023-09-13T03:31:10.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:10 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:10.338+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:37.105618+0000 front 2023-09-13T03:30:37.105683+0000 (oldest deadline 2023-09-13T03:31:05.306446+0000) 2023-09-13T03:31:10.668 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:10 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:10.372+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:31:10.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:10 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:10.372+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.238945+0000 front 2023-09-13T03:30:15.239145+0000 (oldest deadline 2023-09-13T03:30:41.139111+0000) 2023-09-13T03:31:10.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:10 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:10.372+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.840328+0000 front 2023-09-13T03:30:22.840159+0000 (oldest deadline 2023-09-13T03:30:45.740226+0000) 2023-09-13T03:31:10.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:10 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:10.372+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.641567+0000 front 2023-09-13T03:30:31.641612+0000 (oldest deadline 2023-09-13T03:30:55.141726+0000) 2023-09-13T03:31:10.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:10 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:10.372+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:39.842954+0000 front 2023-09-13T03:30:39.842992+0000 (oldest deadline 2023-09-13T03:31:05.142850+0000) 2023-09-13T03:31:10.670 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:10 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:10.372+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:45.143359+0000 front 2023-09-13T03:30:45.143405+0000 (oldest deadline 2023-09-13T03:31:08.643469+0000) 2023-09-13T03:31:11.160 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:10 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:10.762+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:14.436426+0000 front 2023-09-13T03:30:14.436375+0000 (oldest deadline 2023-09-13T03:30:39.736445+0000) 2023-09-13T03:31:11.160 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:10 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:10.762+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.338362+0000 front 2023-09-13T03:30:21.338390+0000 (oldest deadline 2023-09-13T03:30:46.638525+0000) 2023-09-13T03:31:11.161 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:10 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:10.762+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.840430+0000 front 2023-09-13T03:30:31.840408+0000 (oldest deadline 2023-09-13T03:30:54.740429+0000) 2023-09-13T03:31:11.161 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:10 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:10.762+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:36.441959+0000 front 2023-09-13T03:30:36.441816+0000 (oldest deadline 2023-09-13T03:31:01.642671+0000) 2023-09-13T03:31:11.161 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:10 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8[156823]: 2023-09-13T03:31:10.762+0000 7fa0c781a700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:45.143669+0000 front 2023-09-13T03:30:45.143796+0000 (oldest deadline 2023-09-13T03:31:09.244616+0000) 2023-09-13T03:31:11.495 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:11 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:11.021+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:31:11.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:11 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:11.021+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:31:11.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:11 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:11.021+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.557932+0000 front 2023-09-13T03:30:21.558055+0000 (oldest deadline 2023-09-13T03:30:45.658031+0000) 2023-09-13T03:31:11.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:11 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:11.021+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:28.960471+0000 front 2023-09-13T03:30:28.960513+0000 (oldest deadline 2023-09-13T03:30:53.660585+0000) 2023-09-13T03:31:11.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:11 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:11.021+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:35.961436+0000 front 2023-09-13T03:30:35.961450+0000 (oldest deadline 2023-09-13T03:31:01.861557+0000) 2023-09-13T03:31:11.632 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:11 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:11.355+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:31:11.633 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:11 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:11.355+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:31:11.633 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:11 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:11.355+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:31:11.634 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:11 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:11.355+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.502766+0000 front 2023-09-13T03:30:31.502677+0000 (oldest deadline 2023-09-13T03:30:54.903625+0000) 2023-09-13T03:31:11.634 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:11 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:11.355+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:37.105618+0000 front 2023-09-13T03:30:37.105683+0000 (oldest deadline 2023-09-13T03:31:05.306446+0000) 2023-09-13T03:31:11.668 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:11 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:11.354+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:31:11.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:11 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:11.354+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.238945+0000 front 2023-09-13T03:30:15.239145+0000 (oldest deadline 2023-09-13T03:30:41.139111+0000) 2023-09-13T03:31:11.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:11 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:11.354+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.840328+0000 front 2023-09-13T03:30:22.840159+0000 (oldest deadline 2023-09-13T03:30:45.740226+0000) 2023-09-13T03:31:11.670 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:11 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:11.354+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.641567+0000 front 2023-09-13T03:30:31.641612+0000 (oldest deadline 2023-09-13T03:30:55.141726+0000) 2023-09-13T03:31:11.670 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:11 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:11.354+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:39.842954+0000 front 2023-09-13T03:30:39.842992+0000 (oldest deadline 2023-09-13T03:31:05.142850+0000) 2023-09-13T03:31:11.670 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:11 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:11.354+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:45.143359+0000 front 2023-09-13T03:30:45.143405+0000 (oldest deadline 2023-09-13T03:31:08.643469+0000) 2023-09-13T03:31:11.910 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 13 03:31:11 smithi170 bash[180457]: ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-8 2023-09-13T03:31:12.495 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:12 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:12.030+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:31:12.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:12 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:12.030+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:31:12.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:12 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:12.030+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.557932+0000 front 2023-09-13T03:30:21.558055+0000 (oldest deadline 2023-09-13T03:30:45.658031+0000) 2023-09-13T03:31:12.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:12 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:12.030+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:28.960471+0000 front 2023-09-13T03:30:28.960513+0000 (oldest deadline 2023-09-13T03:30:53.660585+0000) 2023-09-13T03:31:12.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:12 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:12.030+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:35.961436+0000 front 2023-09-13T03:30:35.961450+0000 (oldest deadline 2023-09-13T03:31:01.861557+0000) 2023-09-13T03:31:12.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:12 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:12.030+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:46.963876+0000 front 2023-09-13T03:30:46.964042+0000 (oldest deadline 2023-09-13T03:31:11.063941+0000) 2023-09-13T03:31:12.660 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:12 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:12.330+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:31:12.660 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:12 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:12.330+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:31:12.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:12 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:12.330+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:31:12.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:12 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:12.330+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.502766+0000 front 2023-09-13T03:30:31.502677+0000 (oldest deadline 2023-09-13T03:30:54.903625+0000) 2023-09-13T03:31:12.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:12 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:12.330+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:37.105618+0000 front 2023-09-13T03:30:37.105683+0000 (oldest deadline 2023-09-13T03:31:05.306446+0000) 2023-09-13T03:31:12.668 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:12 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:12.364+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:31:12.668 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:12 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:12.364+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.238945+0000 front 2023-09-13T03:30:15.239145+0000 (oldest deadline 2023-09-13T03:30:41.139111+0000) 2023-09-13T03:31:12.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:12 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:12.364+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.840328+0000 front 2023-09-13T03:30:22.840159+0000 (oldest deadline 2023-09-13T03:30:45.740226+0000) 2023-09-13T03:31:12.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:12 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:12.364+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.641567+0000 front 2023-09-13T03:30:31.641612+0000 (oldest deadline 2023-09-13T03:30:55.141726+0000) 2023-09-13T03:31:12.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:12 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:12.364+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:39.842954+0000 front 2023-09-13T03:30:39.842992+0000 (oldest deadline 2023-09-13T03:31:05.142850+0000) 2023-09-13T03:31:12.670 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:12 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:12.364+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:45.143359+0000 front 2023-09-13T03:30:45.143405+0000 (oldest deadline 2023-09-13T03:31:08.643469+0000) 2023-09-13T03:31:13.094 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@osd.8.service' 2023-09-13T03:31:13.175 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T03:31:13.176 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-09-13T03:31:13.176 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-09-13T03:31:13.177 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@osd.9 2023-09-13T03:31:13.410 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:13 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:13.282+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:31:13.410 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:13 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:13.282+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:31:13.411 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:13 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:13.282+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:31:13.411 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:13 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:13.282+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.502766+0000 front 2023-09-13T03:30:31.502677+0000 (oldest deadline 2023-09-13T03:30:54.903625+0000) 2023-09-13T03:31:13.411 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:13 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:13.282+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:37.105618+0000 front 2023-09-13T03:30:37.105683+0000 (oldest deadline 2023-09-13T03:31:05.306446+0000) 2023-09-13T03:31:13.412 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:13 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:13.282+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:47.607627+0000 front 2023-09-13T03:30:47.607604+0000 (oldest deadline 2023-09-13T03:31:12.907734+0000) 2023-09-13T03:31:13.495 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:13 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:13.026+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:31:13.495 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:13 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:13.026+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:31:13.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:13 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:13.026+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.557932+0000 front 2023-09-13T03:30:21.558055+0000 (oldest deadline 2023-09-13T03:30:45.658031+0000) 2023-09-13T03:31:13.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:13 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:13.026+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:28.960471+0000 front 2023-09-13T03:30:28.960513+0000 (oldest deadline 2023-09-13T03:30:53.660585+0000) 2023-09-13T03:31:13.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:13 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:13.026+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:35.961436+0000 front 2023-09-13T03:30:35.961450+0000 (oldest deadline 2023-09-13T03:31:01.861557+0000) 2023-09-13T03:31:13.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:13 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:13.026+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:46.963876+0000 front 2023-09-13T03:30:46.964042+0000 (oldest deadline 2023-09-13T03:31:11.063941+0000) 2023-09-13T03:31:13.668 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:13 smithi064 systemd[1]: Stopping Ceph osd.9 for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:31:13.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:13 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:13.365+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:31:13.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:13 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:13.365+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.238945+0000 front 2023-09-13T03:30:15.239145+0000 (oldest deadline 2023-09-13T03:30:41.139111+0000) 2023-09-13T03:31:13.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:13 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:13.365+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.840328+0000 front 2023-09-13T03:30:22.840159+0000 (oldest deadline 2023-09-13T03:30:45.740226+0000) 2023-09-13T03:31:13.670 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:13 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:13.365+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.641567+0000 front 2023-09-13T03:30:31.641612+0000 (oldest deadline 2023-09-13T03:30:55.141726+0000) 2023-09-13T03:31:13.670 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:13 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:13.365+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:39.842954+0000 front 2023-09-13T03:30:39.842992+0000 (oldest deadline 2023-09-13T03:31:05.142850+0000) 2023-09-13T03:31:13.670 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:13 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:13.365+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:45.143359+0000 front 2023-09-13T03:30:45.143405+0000 (oldest deadline 2023-09-13T03:31:08.643469+0000) 2023-09-13T03:31:14.168 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:13 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:13.682+0000 7f4082282700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T03:31:14.169 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:13 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:13.682+0000 7f4082282700 -1 osd.9 109 *** Got signal Terminated *** 2023-09-13T03:31:14.169 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:13 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:13.682+0000 7f4082282700 -1 osd.9 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-13T03:31:14.495 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:14 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:14.070+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:31:14.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:14 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:14.070+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:31:14.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:14 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:14.070+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.557932+0000 front 2023-09-13T03:30:21.558055+0000 (oldest deadline 2023-09-13T03:30:45.658031+0000) 2023-09-13T03:31:14.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:14 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:14.070+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:28.960471+0000 front 2023-09-13T03:30:28.960513+0000 (oldest deadline 2023-09-13T03:30:53.660585+0000) 2023-09-13T03:31:14.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:14 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:14.070+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:35.961436+0000 front 2023-09-13T03:30:35.961450+0000 (oldest deadline 2023-09-13T03:31:01.861557+0000) 2023-09-13T03:31:14.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:14 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:14.070+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:46.963876+0000 front 2023-09-13T03:30:46.964042+0000 (oldest deadline 2023-09-13T03:31:11.063941+0000) 2023-09-13T03:31:14.660 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:14 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:14.251+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:31:14.660 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:14 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:14.251+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:31:14.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:14 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:14.251+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:31:14.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:14 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:14.251+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.502766+0000 front 2023-09-13T03:30:31.502677+0000 (oldest deadline 2023-09-13T03:30:54.903625+0000) 2023-09-13T03:31:14.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:14 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:14.251+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:37.105618+0000 front 2023-09-13T03:30:37.105683+0000 (oldest deadline 2023-09-13T03:31:05.306446+0000) 2023-09-13T03:31:14.662 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:14 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:14.251+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:47.607627+0000 front 2023-09-13T03:30:47.607604+0000 (oldest deadline 2023-09-13T03:31:12.907734+0000) 2023-09-13T03:31:14.668 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:14 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:14.354+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:31:14.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:14 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:14.354+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.238945+0000 front 2023-09-13T03:30:15.239145+0000 (oldest deadline 2023-09-13T03:30:41.139111+0000) 2023-09-13T03:31:14.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:14 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:14.354+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.840328+0000 front 2023-09-13T03:30:22.840159+0000 (oldest deadline 2023-09-13T03:30:45.740226+0000) 2023-09-13T03:31:14.670 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:14 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:14.354+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.641567+0000 front 2023-09-13T03:30:31.641612+0000 (oldest deadline 2023-09-13T03:30:55.141726+0000) 2023-09-13T03:31:14.670 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:14 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:14.354+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:39.842954+0000 front 2023-09-13T03:30:39.842992+0000 (oldest deadline 2023-09-13T03:31:05.142850+0000) 2023-09-13T03:31:14.670 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:14 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:14.354+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:45.143359+0000 front 2023-09-13T03:30:45.143405+0000 (oldest deadline 2023-09-13T03:31:08.643469+0000) 2023-09-13T03:31:15.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:15 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:15.101+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:31:15.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:15 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:15.101+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:31:15.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:15 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:15.101+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.557932+0000 front 2023-09-13T03:30:21.558055+0000 (oldest deadline 2023-09-13T03:30:45.658031+0000) 2023-09-13T03:31:15.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:15 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:15.101+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:28.960471+0000 front 2023-09-13T03:30:28.960513+0000 (oldest deadline 2023-09-13T03:30:53.660585+0000) 2023-09-13T03:31:15.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:15 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:15.101+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:35.961436+0000 front 2023-09-13T03:30:35.961450+0000 (oldest deadline 2023-09-13T03:31:01.861557+0000) 2023-09-13T03:31:15.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:15 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:15.101+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:46.963876+0000 front 2023-09-13T03:30:46.964042+0000 (oldest deadline 2023-09-13T03:31:11.063941+0000) 2023-09-13T03:31:15.660 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:15 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:15.234+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:31:15.660 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:15 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:15.234+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:31:15.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:15 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:15.234+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:31:15.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:15 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:15.234+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.502766+0000 front 2023-09-13T03:30:31.502677+0000 (oldest deadline 2023-09-13T03:30:54.903625+0000) 2023-09-13T03:31:15.662 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:15 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:15.234+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:37.105618+0000 front 2023-09-13T03:30:37.105683+0000 (oldest deadline 2023-09-13T03:31:05.306446+0000) 2023-09-13T03:31:15.662 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:15 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:15.234+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:47.607627+0000 front 2023-09-13T03:30:47.607604+0000 (oldest deadline 2023-09-13T03:31:12.907734+0000) 2023-09-13T03:31:15.668 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:15 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:15.381+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:31:15.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:15 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:15.381+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.238945+0000 front 2023-09-13T03:30:15.239145+0000 (oldest deadline 2023-09-13T03:30:41.139111+0000) 2023-09-13T03:31:15.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:15 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:15.381+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.840328+0000 front 2023-09-13T03:30:22.840159+0000 (oldest deadline 2023-09-13T03:30:45.740226+0000) 2023-09-13T03:31:15.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:15 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:15.381+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.641567+0000 front 2023-09-13T03:30:31.641612+0000 (oldest deadline 2023-09-13T03:30:55.141726+0000) 2023-09-13T03:31:15.670 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:15 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:15.381+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:39.842954+0000 front 2023-09-13T03:30:39.842992+0000 (oldest deadline 2023-09-13T03:31:05.142850+0000) 2023-09-13T03:31:15.670 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:15 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:15.381+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:45.143359+0000 front 2023-09-13T03:30:45.143405+0000 (oldest deadline 2023-09-13T03:31:08.643469+0000) 2023-09-13T03:31:16.495 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:16 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:16.094+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:31:16.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:16 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:16.094+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:31:16.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:16 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:16.094+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.557932+0000 front 2023-09-13T03:30:21.558055+0000 (oldest deadline 2023-09-13T03:30:45.658031+0000) 2023-09-13T03:31:16.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:16 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:16.094+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:28.960471+0000 front 2023-09-13T03:30:28.960513+0000 (oldest deadline 2023-09-13T03:30:53.660585+0000) 2023-09-13T03:31:16.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:16 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:16.094+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:35.961436+0000 front 2023-09-13T03:30:35.961450+0000 (oldest deadline 2023-09-13T03:31:01.861557+0000) 2023-09-13T03:31:16.498 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:16 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:16.094+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:46.963876+0000 front 2023-09-13T03:30:46.964042+0000 (oldest deadline 2023-09-13T03:31:11.063941+0000) 2023-09-13T03:31:16.660 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:16 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:16.219+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:31:16.660 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:16 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:16.219+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:31:16.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:16 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:16.219+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:31:16.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:16 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:16.219+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.502766+0000 front 2023-09-13T03:30:31.502677+0000 (oldest deadline 2023-09-13T03:30:54.903625+0000) 2023-09-13T03:31:16.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:16 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:16.219+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:37.105618+0000 front 2023-09-13T03:30:37.105683+0000 (oldest deadline 2023-09-13T03:31:05.306446+0000) 2023-09-13T03:31:16.662 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:16 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:16.219+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:47.607627+0000 front 2023-09-13T03:30:47.607604+0000 (oldest deadline 2023-09-13T03:31:12.907734+0000) 2023-09-13T03:31:16.668 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:16 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:16.392+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:31:16.668 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:16 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:16.392+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.238945+0000 front 2023-09-13T03:30:15.239145+0000 (oldest deadline 2023-09-13T03:30:41.139111+0000) 2023-09-13T03:31:16.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:16 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:16.392+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.840328+0000 front 2023-09-13T03:30:22.840159+0000 (oldest deadline 2023-09-13T03:30:45.740226+0000) 2023-09-13T03:31:16.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:16 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:16.392+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.641567+0000 front 2023-09-13T03:30:31.641612+0000 (oldest deadline 2023-09-13T03:30:55.141726+0000) 2023-09-13T03:31:16.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:16 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:16.392+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:39.842954+0000 front 2023-09-13T03:30:39.842992+0000 (oldest deadline 2023-09-13T03:31:05.142850+0000) 2023-09-13T03:31:16.670 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:16 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:16.392+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:45.143359+0000 front 2023-09-13T03:30:45.143405+0000 (oldest deadline 2023-09-13T03:31:08.643469+0000) 2023-09-13T03:31:17.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:17 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:17.049+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:31:17.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:17 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:17.049+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:31:17.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:17 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:17.049+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.557932+0000 front 2023-09-13T03:30:21.558055+0000 (oldest deadline 2023-09-13T03:30:45.658031+0000) 2023-09-13T03:31:17.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:17 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:17.049+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:28.960471+0000 front 2023-09-13T03:30:28.960513+0000 (oldest deadline 2023-09-13T03:30:53.660585+0000) 2023-09-13T03:31:17.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:17 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:17.049+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:35.961436+0000 front 2023-09-13T03:30:35.961450+0000 (oldest deadline 2023-09-13T03:31:01.861557+0000) 2023-09-13T03:31:17.498 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:17 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:17.049+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:46.963876+0000 front 2023-09-13T03:30:46.964042+0000 (oldest deadline 2023-09-13T03:31:11.063941+0000) 2023-09-13T03:31:17.660 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:17 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:17.233+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:31:17.660 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:17 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:17.233+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:31:17.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:17 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:17.233+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:31:17.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:17 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:17.234+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.502766+0000 front 2023-09-13T03:30:31.502677+0000 (oldest deadline 2023-09-13T03:30:54.903625+0000) 2023-09-13T03:31:17.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:17 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:17.234+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:37.105618+0000 front 2023-09-13T03:30:37.105683+0000 (oldest deadline 2023-09-13T03:31:05.306446+0000) 2023-09-13T03:31:17.662 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:17 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:17.234+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:47.607627+0000 front 2023-09-13T03:30:47.607604+0000 (oldest deadline 2023-09-13T03:31:12.907734+0000) 2023-09-13T03:31:17.668 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:17 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:17.394+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:31:17.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:17 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:17.394+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.238945+0000 front 2023-09-13T03:30:15.239145+0000 (oldest deadline 2023-09-13T03:30:41.139111+0000) 2023-09-13T03:31:17.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:17 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:17.394+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.840328+0000 front 2023-09-13T03:30:22.840159+0000 (oldest deadline 2023-09-13T03:30:45.740226+0000) 2023-09-13T03:31:17.669 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:17 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:17.394+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.641567+0000 front 2023-09-13T03:30:31.641612+0000 (oldest deadline 2023-09-13T03:30:55.141726+0000) 2023-09-13T03:31:17.670 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:17 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:17.394+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:39.842954+0000 front 2023-09-13T03:30:39.842992+0000 (oldest deadline 2023-09-13T03:31:05.142850+0000) 2023-09-13T03:31:17.670 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:17 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:17.394+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:45.143359+0000 front 2023-09-13T03:30:45.143405+0000 (oldest deadline 2023-09-13T03:31:08.643469+0000) 2023-09-13T03:31:18.495 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:18 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:18.038+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:31:18.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:18 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:18.038+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:31:18.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:18 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:18.038+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.557932+0000 front 2023-09-13T03:30:21.558055+0000 (oldest deadline 2023-09-13T03:30:45.658031+0000) 2023-09-13T03:31:18.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:18 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:18.038+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:28.960471+0000 front 2023-09-13T03:30:28.960513+0000 (oldest deadline 2023-09-13T03:30:53.660585+0000) 2023-09-13T03:31:18.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:18 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:18.038+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:35.961436+0000 front 2023-09-13T03:30:35.961450+0000 (oldest deadline 2023-09-13T03:31:01.861557+0000) 2023-09-13T03:31:18.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:18 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:18.038+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:46.963876+0000 front 2023-09-13T03:30:46.964042+0000 (oldest deadline 2023-09-13T03:31:11.063941+0000) 2023-09-13T03:31:18.660 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:18 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:18.205+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:31:18.660 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:18 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:18.205+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:31:18.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:18 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:18.205+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:31:18.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:18 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:18.205+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.502766+0000 front 2023-09-13T03:30:31.502677+0000 (oldest deadline 2023-09-13T03:30:54.903625+0000) 2023-09-13T03:31:18.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:18 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:18.205+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:37.105618+0000 front 2023-09-13T03:30:37.105683+0000 (oldest deadline 2023-09-13T03:31:05.306446+0000) 2023-09-13T03:31:18.662 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:18 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:18.205+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:47.607627+0000 front 2023-09-13T03:30:47.607604+0000 (oldest deadline 2023-09-13T03:31:12.907734+0000) 2023-09-13T03:31:18.918 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:18 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:18.440+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:07.637379+0000 front 2023-09-13T03:30:07.637406+0000 (oldest deadline 2023-09-13T03:30:32.337671+0000) 2023-09-13T03:31:18.919 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:18 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:18.440+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:15.238945+0000 front 2023-09-13T03:30:15.239145+0000 (oldest deadline 2023-09-13T03:30:41.139111+0000) 2023-09-13T03:31:18.919 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:18 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:18.440+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.840328+0000 front 2023-09-13T03:30:22.840159+0000 (oldest deadline 2023-09-13T03:30:45.740226+0000) 2023-09-13T03:31:18.920 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:18 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:18.440+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.641567+0000 front 2023-09-13T03:30:31.641612+0000 (oldest deadline 2023-09-13T03:30:55.141726+0000) 2023-09-13T03:31:18.920 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:18 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:18.440+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:39.842954+0000 front 2023-09-13T03:30:39.842992+0000 (oldest deadline 2023-09-13T03:31:05.142850+0000) 2023-09-13T03:31:18.920 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:18 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9[144770]: 2023-09-13T03:31:18.440+0000 7f407d075700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:45.143359+0000 front 2023-09-13T03:30:45.143405+0000 (oldest deadline 2023-09-13T03:31:08.643469+0000) 2023-09-13T03:31:19.495 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:19 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:19.050+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:31:19.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:19 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:19.050+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:31:19.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:19 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:19.050+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.557932+0000 front 2023-09-13T03:30:21.558055+0000 (oldest deadline 2023-09-13T03:30:45.658031+0000) 2023-09-13T03:31:19.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:19 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:19.050+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:28.960471+0000 front 2023-09-13T03:30:28.960513+0000 (oldest deadline 2023-09-13T03:30:53.660585+0000) 2023-09-13T03:31:19.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:19 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:19.050+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:35.961436+0000 front 2023-09-13T03:30:35.961450+0000 (oldest deadline 2023-09-13T03:31:01.861557+0000) 2023-09-13T03:31:19.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:19 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:19.050+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:46.963876+0000 front 2023-09-13T03:30:46.964042+0000 (oldest deadline 2023-09-13T03:31:11.063941+0000) 2023-09-13T03:31:19.660 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:19 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:19.243+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:31:19.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:19 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:19.243+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:31:19.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:19 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:19.243+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:31:19.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:19 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:19.243+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.502766+0000 front 2023-09-13T03:30:31.502677+0000 (oldest deadline 2023-09-13T03:30:54.903625+0000) 2023-09-13T03:31:19.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:19 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:19.243+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:37.105618+0000 front 2023-09-13T03:30:37.105683+0000 (oldest deadline 2023-09-13T03:31:05.306446+0000) 2023-09-13T03:31:19.662 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:19 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:19.243+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:47.607627+0000 front 2023-09-13T03:30:47.607604+0000 (oldest deadline 2023-09-13T03:31:12.907734+0000) 2023-09-13T03:31:20.169 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 13 03:31:19 smithi064 bash[181980]: ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-9 2023-09-13T03:31:20.495 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:20 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:20.089+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:31:20.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:20 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:20.089+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:31:20.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:20 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:20.089+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.557932+0000 front 2023-09-13T03:30:21.558055+0000 (oldest deadline 2023-09-13T03:30:45.658031+0000) 2023-09-13T03:31:20.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:20 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:20.089+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:28.960471+0000 front 2023-09-13T03:30:28.960513+0000 (oldest deadline 2023-09-13T03:30:53.660585+0000) 2023-09-13T03:31:20.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:20 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:20.089+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:35.961436+0000 front 2023-09-13T03:30:35.961450+0000 (oldest deadline 2023-09-13T03:31:01.861557+0000) 2023-09-13T03:31:20.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:20 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:20.089+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:46.963876+0000 front 2023-09-13T03:30:46.964042+0000 (oldest deadline 2023-09-13T03:31:11.063941+0000) 2023-09-13T03:31:20.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:20 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:20.089+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2023-09-13T03:30:55.566394+0000 front 2023-09-13T03:30:55.566492+0000 (oldest deadline 2023-09-13T03:31:19.066404+0000) 2023-09-13T03:31:20.660 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:20 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:20.198+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:31:20.660 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:20 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:20.198+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:31:20.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:20 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:20.198+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:31:20.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:20 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:20.198+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.502766+0000 front 2023-09-13T03:30:31.502677+0000 (oldest deadline 2023-09-13T03:30:54.903625+0000) 2023-09-13T03:31:20.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:20 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:20.198+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:37.105618+0000 front 2023-09-13T03:30:37.105683+0000 (oldest deadline 2023-09-13T03:31:05.306446+0000) 2023-09-13T03:31:20.662 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:20 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:20.198+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:47.607627+0000 front 2023-09-13T03:30:47.607604+0000 (oldest deadline 2023-09-13T03:31:12.907734+0000) 2023-09-13T03:31:21.432 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@osd.9.service' 2023-09-13T03:31:21.495 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:21 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:21.124+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:31:21.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:21 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:21.124+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:31:21.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:21 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:21.124+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.557932+0000 front 2023-09-13T03:30:21.558055+0000 (oldest deadline 2023-09-13T03:30:45.658031+0000) 2023-09-13T03:31:21.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:21 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:21.124+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:28.960471+0000 front 2023-09-13T03:30:28.960513+0000 (oldest deadline 2023-09-13T03:30:53.660585+0000) 2023-09-13T03:31:21.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:21 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:21.124+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:35.961436+0000 front 2023-09-13T03:30:35.961450+0000 (oldest deadline 2023-09-13T03:31:01.861557+0000) 2023-09-13T03:31:21.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:21 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:21.124+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:46.963876+0000 front 2023-09-13T03:30:46.964042+0000 (oldest deadline 2023-09-13T03:31:11.063941+0000) 2023-09-13T03:31:21.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:21 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:21.124+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2023-09-13T03:30:55.566394+0000 front 2023-09-13T03:30:55.566492+0000 (oldest deadline 2023-09-13T03:31:19.066404+0000) 2023-09-13T03:31:21.521 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T03:31:21.522 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-09-13T03:31:21.522 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-09-13T03:31:21.523 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@osd.10 2023-09-13T03:31:21.660 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:21 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:21.230+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:31:21.660 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:21 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:21.230+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:31:21.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:21 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:21.230+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:31:21.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:21 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:21.230+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.502766+0000 front 2023-09-13T03:30:31.502677+0000 (oldest deadline 2023-09-13T03:30:54.903625+0000) 2023-09-13T03:31:21.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:21 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:21.230+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:37.105618+0000 front 2023-09-13T03:30:37.105683+0000 (oldest deadline 2023-09-13T03:31:05.306446+0000) 2023-09-13T03:31:21.662 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:21 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:21.230+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:47.607627+0000 front 2023-09-13T03:30:47.607604+0000 (oldest deadline 2023-09-13T03:31:12.907734+0000) 2023-09-13T03:31:21.899 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:21 smithi142 systemd[1]: Stopping Ceph osd.10 for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:31:22.158 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:21 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:21.897+0000 7f53e2f42700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T03:31:22.158 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:21 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:21.897+0000 7f53e2f42700 -1 osd.10 109 *** Got signal Terminated *** 2023-09-13T03:31:22.159 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:21 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:21.897+0000 7f53e2f42700 -1 osd.10 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-13T03:31:22.495 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:22 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:22.157+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:31:22.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:22 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:22.157+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:31:22.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:22 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:22.157+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.557932+0000 front 2023-09-13T03:30:21.558055+0000 (oldest deadline 2023-09-13T03:30:45.658031+0000) 2023-09-13T03:31:22.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:22 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:22.157+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:28.960471+0000 front 2023-09-13T03:30:28.960513+0000 (oldest deadline 2023-09-13T03:30:53.660585+0000) 2023-09-13T03:31:22.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:22 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:22.157+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:35.961436+0000 front 2023-09-13T03:30:35.961450+0000 (oldest deadline 2023-09-13T03:31:01.861557+0000) 2023-09-13T03:31:22.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:22 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:22.157+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:46.963876+0000 front 2023-09-13T03:30:46.964042+0000 (oldest deadline 2023-09-13T03:31:11.063941+0000) 2023-09-13T03:31:22.498 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:22 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:22.157+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2023-09-13T03:30:55.566394+0000 front 2023-09-13T03:30:55.566492+0000 (oldest deadline 2023-09-13T03:31:19.066404+0000) 2023-09-13T03:31:22.660 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:22 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:22.230+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:31:22.660 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:22 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:22.230+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:31:22.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:22 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:22.230+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:31:22.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:22 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:22.230+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.502766+0000 front 2023-09-13T03:30:31.502677+0000 (oldest deadline 2023-09-13T03:30:54.903625+0000) 2023-09-13T03:31:22.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:22 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:22.230+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:37.105618+0000 front 2023-09-13T03:30:37.105683+0000 (oldest deadline 2023-09-13T03:31:05.306446+0000) 2023-09-13T03:31:22.662 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:22 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:22.230+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:47.607627+0000 front 2023-09-13T03:30:47.607604+0000 (oldest deadline 2023-09-13T03:31:12.907734+0000) 2023-09-13T03:31:23.495 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:23 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:23.145+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:31:23.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:23 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:23.145+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:31:23.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:23 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:23.145+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.557932+0000 front 2023-09-13T03:30:21.558055+0000 (oldest deadline 2023-09-13T03:30:45.658031+0000) 2023-09-13T03:31:23.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:23 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:23.145+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:28.960471+0000 front 2023-09-13T03:30:28.960513+0000 (oldest deadline 2023-09-13T03:30:53.660585+0000) 2023-09-13T03:31:23.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:23 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:23.145+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:35.961436+0000 front 2023-09-13T03:30:35.961450+0000 (oldest deadline 2023-09-13T03:31:01.861557+0000) 2023-09-13T03:31:23.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:23 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:23.145+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:46.963876+0000 front 2023-09-13T03:30:46.964042+0000 (oldest deadline 2023-09-13T03:31:11.063941+0000) 2023-09-13T03:31:23.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:23 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:23.145+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2023-09-13T03:30:55.566394+0000 front 2023-09-13T03:30:55.566492+0000 (oldest deadline 2023-09-13T03:31:19.066404+0000) 2023-09-13T03:31:23.660 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:23 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:23.182+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:31:23.660 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:23 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:23.182+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:31:23.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:23 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:23.182+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:31:23.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:23 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:23.182+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.502766+0000 front 2023-09-13T03:30:31.502677+0000 (oldest deadline 2023-09-13T03:30:54.903625+0000) 2023-09-13T03:31:23.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:23 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:23.182+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:37.105618+0000 front 2023-09-13T03:30:37.105683+0000 (oldest deadline 2023-09-13T03:31:05.306446+0000) 2023-09-13T03:31:23.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:23 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:23.182+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:47.607627+0000 front 2023-09-13T03:30:47.607604+0000 (oldest deadline 2023-09-13T03:31:12.907734+0000) 2023-09-13T03:31:24.495 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:24 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:24.168+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:31:24.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:24 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:24.168+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:31:24.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:24 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:24.168+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.557932+0000 front 2023-09-13T03:30:21.558055+0000 (oldest deadline 2023-09-13T03:30:45.658031+0000) 2023-09-13T03:31:24.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:24 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:24.168+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:28.960471+0000 front 2023-09-13T03:30:28.960513+0000 (oldest deadline 2023-09-13T03:30:53.660585+0000) 2023-09-13T03:31:24.498 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:24 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:24.168+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:35.961436+0000 front 2023-09-13T03:30:35.961450+0000 (oldest deadline 2023-09-13T03:31:01.861557+0000) 2023-09-13T03:31:24.498 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:24 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:24.168+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:46.963876+0000 front 2023-09-13T03:30:46.964042+0000 (oldest deadline 2023-09-13T03:31:11.063941+0000) 2023-09-13T03:31:24.498 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:24 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:24.168+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2023-09-13T03:30:55.566394+0000 front 2023-09-13T03:30:55.566492+0000 (oldest deadline 2023-09-13T03:31:19.066404+0000) 2023-09-13T03:31:24.660 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:24 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:24.202+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:31:24.660 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:24 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:24.202+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:31:24.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:24 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:24.202+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:31:24.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:24 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:24.202+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.502766+0000 front 2023-09-13T03:30:31.502677+0000 (oldest deadline 2023-09-13T03:30:54.903625+0000) 2023-09-13T03:31:24.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:24 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:24.202+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:37.105618+0000 front 2023-09-13T03:30:37.105683+0000 (oldest deadline 2023-09-13T03:31:05.306446+0000) 2023-09-13T03:31:24.662 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:24 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:24.202+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:47.607627+0000 front 2023-09-13T03:30:47.607604+0000 (oldest deadline 2023-09-13T03:31:12.907734+0000) 2023-09-13T03:31:25.495 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:25 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:25.139+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:31:25.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:25 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:25.139+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:31:25.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:25 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:25.139+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.557932+0000 front 2023-09-13T03:30:21.558055+0000 (oldest deadline 2023-09-13T03:30:45.658031+0000) 2023-09-13T03:31:25.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:25 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:25.139+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:28.960471+0000 front 2023-09-13T03:30:28.960513+0000 (oldest deadline 2023-09-13T03:30:53.660585+0000) 2023-09-13T03:31:25.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:25 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:25.139+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:35.961436+0000 front 2023-09-13T03:30:35.961450+0000 (oldest deadline 2023-09-13T03:31:01.861557+0000) 2023-09-13T03:31:25.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:25 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:25.139+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:46.963876+0000 front 2023-09-13T03:30:46.964042+0000 (oldest deadline 2023-09-13T03:31:11.063941+0000) 2023-09-13T03:31:25.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:25 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:25.139+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2023-09-13T03:30:55.566394+0000 front 2023-09-13T03:30:55.566492+0000 (oldest deadline 2023-09-13T03:31:19.066404+0000) 2023-09-13T03:31:25.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:25 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:25.166+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:31:25.662 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:25 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:25.166+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:31:25.662 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:25 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:25.166+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:31:25.662 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:25 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:25.166+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.502766+0000 front 2023-09-13T03:30:31.502677+0000 (oldest deadline 2023-09-13T03:30:54.903625+0000) 2023-09-13T03:31:25.663 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:25 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:25.166+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:37.105618+0000 front 2023-09-13T03:30:37.105683+0000 (oldest deadline 2023-09-13T03:31:05.306446+0000) 2023-09-13T03:31:25.663 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:25 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:25.166+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:47.607627+0000 front 2023-09-13T03:30:47.607604+0000 (oldest deadline 2023-09-13T03:31:12.907734+0000) 2023-09-13T03:31:26.495 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:26 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:26.117+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.355710+0000 front 2023-09-13T03:30:06.355701+0000 (oldest deadline 2023-09-13T03:30:30.455480+0000) 2023-09-13T03:31:26.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:26 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:26.117+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:13.356618+0000 front 2023-09-13T03:30:13.356671+0000 (oldest deadline 2023-09-13T03:30:37.456778+0000) 2023-09-13T03:31:26.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:26 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:26.117+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:21.557932+0000 front 2023-09-13T03:30:21.558055+0000 (oldest deadline 2023-09-13T03:30:45.658031+0000) 2023-09-13T03:31:26.496 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:26 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:26.117+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:28.960471+0000 front 2023-09-13T03:30:28.960513+0000 (oldest deadline 2023-09-13T03:30:53.660585+0000) 2023-09-13T03:31:26.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:26 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:26.117+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:35.961436+0000 front 2023-09-13T03:30:35.961450+0000 (oldest deadline 2023-09-13T03:31:01.861557+0000) 2023-09-13T03:31:26.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:26 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:26.117+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:46.963876+0000 front 2023-09-13T03:30:46.964042+0000 (oldest deadline 2023-09-13T03:31:11.063941+0000) 2023-09-13T03:31:26.497 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:26 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10[163519]: 2023-09-13T03:31:26.117+0000 7f53de536700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2023-09-13T03:30:55.566394+0000 front 2023-09-13T03:30:55.566492+0000 (oldest deadline 2023-09-13T03:31:19.066404+0000) 2023-09-13T03:31:26.660 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:26 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:26.213+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:31:26.660 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:26 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:26.213+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:31:26.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:26 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:26.213+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:31:26.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:26 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:26.213+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.502766+0000 front 2023-09-13T03:30:31.502677+0000 (oldest deadline 2023-09-13T03:30:54.903625+0000) 2023-09-13T03:31:26.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:26 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:26.213+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:37.105618+0000 front 2023-09-13T03:30:37.105683+0000 (oldest deadline 2023-09-13T03:31:05.306446+0000) 2023-09-13T03:31:26.662 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:26 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:26.213+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:47.607627+0000 front 2023-09-13T03:30:47.607604+0000 (oldest deadline 2023-09-13T03:31:12.907734+0000) 2023-09-13T03:31:27.660 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:27 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:27.233+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:31:27.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:27 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:27.233+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:31:27.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:27 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:27.234+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:31:27.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:27 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:27.234+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.502766+0000 front 2023-09-13T03:30:31.502677+0000 (oldest deadline 2023-09-13T03:30:54.903625+0000) 2023-09-13T03:31:27.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:27 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:27.234+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:37.105618+0000 front 2023-09-13T03:30:37.105683+0000 (oldest deadline 2023-09-13T03:31:05.306446+0000) 2023-09-13T03:31:27.662 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:27 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:27.234+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:47.607627+0000 front 2023-09-13T03:30:47.607604+0000 (oldest deadline 2023-09-13T03:31:12.907734+0000) 2023-09-13T03:31:27.662 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:27 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:27.234+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2023-09-13T03:31:00.910358+0000 front 2023-09-13T03:31:00.910484+0000 (oldest deadline 2023-09-13T03:31:26.810226+0000) 2023-09-13T03:31:27.995 INFO:journalctl@ceph.osd.10.smithi142.stdout:Sep 13 03:31:27 smithi142 bash[179717]: ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-10 2023-09-13T03:31:28.660 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:28 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:28.255+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:31:28.660 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:28 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:28.255+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:31:28.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:28 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:28.255+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:31:28.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:28 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:28.255+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.502766+0000 front 2023-09-13T03:30:31.502677+0000 (oldest deadline 2023-09-13T03:30:54.903625+0000) 2023-09-13T03:31:28.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:28 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:28.255+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:37.105618+0000 front 2023-09-13T03:30:37.105683+0000 (oldest deadline 2023-09-13T03:31:05.306446+0000) 2023-09-13T03:31:28.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:28 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:28.255+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:47.607627+0000 front 2023-09-13T03:30:47.607604+0000 (oldest deadline 2023-09-13T03:31:12.907734+0000) 2023-09-13T03:31:28.662 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:28 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:28.255+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2023-09-13T03:31:00.910358+0000 front 2023-09-13T03:31:00.910484+0000 (oldest deadline 2023-09-13T03:31:26.810226+0000) 2023-09-13T03:31:29.138 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@osd.10.service' 2023-09-13T03:31:29.179 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T03:31:29.179 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-09-13T03:31:29.180 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-09-13T03:31:29.180 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@osd.11 2023-09-13T03:31:29.539 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:29 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:29.216+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:31:29.540 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:29 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:29.216+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:31:29.540 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:29 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:29.216+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:31:29.540 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:29 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:29.216+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.502766+0000 front 2023-09-13T03:30:31.502677+0000 (oldest deadline 2023-09-13T03:30:54.903625+0000) 2023-09-13T03:31:29.541 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:29 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:29.216+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:37.105618+0000 front 2023-09-13T03:30:37.105683+0000 (oldest deadline 2023-09-13T03:31:05.306446+0000) 2023-09-13T03:31:29.541 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:29 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:29.216+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:47.607627+0000 front 2023-09-13T03:30:47.607604+0000 (oldest deadline 2023-09-13T03:31:12.907734+0000) 2023-09-13T03:31:29.541 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:29 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:29.216+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2023-09-13T03:31:00.910358+0000 front 2023-09-13T03:31:00.910484+0000 (oldest deadline 2023-09-13T03:31:26.810226+0000) 2023-09-13T03:31:29.542 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:29 smithi170 systemd[1]: Stopping Ceph osd.11 for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:31:29.910 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:29 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:29.538+0000 7f89c09a6700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T03:31:29.911 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:29 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:29.538+0000 7f89c09a6700 -1 osd.11 109 *** Got signal Terminated *** 2023-09-13T03:31:29.911 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:29 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:29.538+0000 7f89c09a6700 -1 osd.11 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-13T03:31:30.660 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:30 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:30.182+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:31:30.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:30 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:30.182+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:31:30.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:30 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:30.182+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:31:30.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:30 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:30.182+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.502766+0000 front 2023-09-13T03:30:31.502677+0000 (oldest deadline 2023-09-13T03:30:54.903625+0000) 2023-09-13T03:31:30.662 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:30 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:30.182+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:37.105618+0000 front 2023-09-13T03:30:37.105683+0000 (oldest deadline 2023-09-13T03:31:05.306446+0000) 2023-09-13T03:31:30.662 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:30 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:30.182+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:47.607627+0000 front 2023-09-13T03:30:47.607604+0000 (oldest deadline 2023-09-13T03:31:12.907734+0000) 2023-09-13T03:31:30.662 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:30 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:30.182+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2023-09-13T03:31:00.910358+0000 front 2023-09-13T03:31:00.910484+0000 (oldest deadline 2023-09-13T03:31:26.810226+0000) 2023-09-13T03:31:31.660 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:31 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:31.184+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:31:31.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:31 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:31.184+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:31:31.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:31 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:31.184+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:31:31.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:31 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:31.184+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.502766+0000 front 2023-09-13T03:30:31.502677+0000 (oldest deadline 2023-09-13T03:30:54.903625+0000) 2023-09-13T03:31:31.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:31 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:31.184+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:37.105618+0000 front 2023-09-13T03:30:37.105683+0000 (oldest deadline 2023-09-13T03:31:05.306446+0000) 2023-09-13T03:31:31.662 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:31 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:31.184+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:47.607627+0000 front 2023-09-13T03:30:47.607604+0000 (oldest deadline 2023-09-13T03:31:12.907734+0000) 2023-09-13T03:31:31.662 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:31 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:31.184+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2023-09-13T03:31:00.910358+0000 front 2023-09-13T03:31:00.910484+0000 (oldest deadline 2023-09-13T03:31:26.810226+0000) 2023-09-13T03:31:32.660 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:32 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:32.198+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:31:32.660 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:32 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:32.198+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:31:32.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:32 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:32.198+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:31:32.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:32 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:32.198+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.502766+0000 front 2023-09-13T03:30:31.502677+0000 (oldest deadline 2023-09-13T03:30:54.903625+0000) 2023-09-13T03:31:32.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:32 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:32.198+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:37.105618+0000 front 2023-09-13T03:30:37.105683+0000 (oldest deadline 2023-09-13T03:31:05.306446+0000) 2023-09-13T03:31:32.662 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:32 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:32.198+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:47.607627+0000 front 2023-09-13T03:30:47.607604+0000 (oldest deadline 2023-09-13T03:31:12.907734+0000) 2023-09-13T03:31:32.662 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:32 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:32.198+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2023-09-13T03:31:00.910358+0000 front 2023-09-13T03:31:00.910484+0000 (oldest deadline 2023-09-13T03:31:26.810226+0000) 2023-09-13T03:31:33.660 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:33 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:33.218+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:31:33.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:33 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:33.218+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:31:33.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:33 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:33.218+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:31:33.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:33 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:33.218+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.502766+0000 front 2023-09-13T03:30:31.502677+0000 (oldest deadline 2023-09-13T03:30:54.903625+0000) 2023-09-13T03:31:33.662 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:33 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:33.218+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:37.105618+0000 front 2023-09-13T03:30:37.105683+0000 (oldest deadline 2023-09-13T03:31:05.306446+0000) 2023-09-13T03:31:33.662 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:33 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:33.218+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:47.607627+0000 front 2023-09-13T03:30:47.607604+0000 (oldest deadline 2023-09-13T03:31:12.907734+0000) 2023-09-13T03:31:33.663 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:33 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:33.218+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2023-09-13T03:31:00.910358+0000 front 2023-09-13T03:31:00.910484+0000 (oldest deadline 2023-09-13T03:31:26.810226+0000) 2023-09-13T03:31:34.660 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:34 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:34.257+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2023-09-13T03:30:06.398584+0000 front 2023-09-13T03:30:06.398494+0000 (oldest deadline 2023-09-13T03:30:29.898489+0000) 2023-09-13T03:31:34.660 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:34 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:34.257+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2023-09-13T03:30:12.799806+0000 front 2023-09-13T03:30:12.799847+0000 (oldest deadline 2023-09-13T03:30:38.099904+0000) 2023-09-13T03:31:34.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:34 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:34.257+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-09-13T03:30:22.101606+0000 front 2023-09-13T03:30:22.101641+0000 (oldest deadline 2023-09-13T03:30:45.601745+0000) 2023-09-13T03:31:34.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:34 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:34.257+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2023-09-13T03:30:31.502766+0000 front 2023-09-13T03:30:31.502677+0000 (oldest deadline 2023-09-13T03:30:54.903625+0000) 2023-09-13T03:31:34.661 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:34 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:34.257+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2023-09-13T03:30:37.105618+0000 front 2023-09-13T03:30:37.105683+0000 (oldest deadline 2023-09-13T03:31:05.306446+0000) 2023-09-13T03:31:34.662 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:34 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:34.257+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-09-13T03:30:47.607627+0000 front 2023-09-13T03:30:47.607604+0000 (oldest deadline 2023-09-13T03:31:12.907734+0000) 2023-09-13T03:31:34.662 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:34 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11[165020]: 2023-09-13T03:31:34.257+0000 7f89baf98700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2023-09-13T03:31:00.910358+0000 front 2023-09-13T03:31:00.910484+0000 (oldest deadline 2023-09-13T03:31:26.810226+0000) 2023-09-13T03:31:35.410 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 13 03:31:35 smithi170 bash[180724]: ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-osd-11 2023-09-13T03:31:36.431 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@osd.11.service' 2023-09-13T03:31:36.472 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T03:31:36.472 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-09-13T03:31:36.473 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-09-13T03:31:36.473 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mds.a 2023-09-13T03:31:36.919 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 13 03:31:36 smithi064 systemd[1]: Stopping Ceph mds.a for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:31:37.418 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 13 03:31:36 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-a[152587]: 2023-09-13T03:31:36.979+0000 7feb4eb1d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T03:31:37.419 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 13 03:31:36 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-a[152587]: 2023-09-13T03:31:36.979+0000 7feb4eb1d700 -1 mds.a *** got signal Terminated *** 2023-09-13T03:31:40.917 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 13 03:31:40 smithi064 bash[182247]: ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-a 2023-09-13T03:31:40.944 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mds.a.service' 2023-09-13T03:31:41.031 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T03:31:41.032 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-09-13T03:31:41.033 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-09-13T03:31:41.033 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mds.d 2023-09-13T03:31:41.418 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 13 03:31:41 smithi064 systemd[1]: Stopping Ceph mds.d for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:31:41.918 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 13 03:31:41 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-d[151588]: 2023-09-13T03:31:41.520+0000 7f8781b34700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T03:31:41.919 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 13 03:31:41 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-d[151588]: 2023-09-13T03:31:41.520+0000 7f8781b34700 -1 mds.d *** got signal Terminated *** 2023-09-13T03:31:44.668 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 13 03:31:44 smithi064 bash[182393]: ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-d 2023-09-13T03:31:44.746 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mds.d.service' 2023-09-13T03:31:44.829 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T03:31:44.830 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-09-13T03:31:44.830 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-09-13T03:31:44.831 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mds.g 2023-09-13T03:31:45.168 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 13 03:31:44 smithi064 systemd[1]: Stopping Ceph mds.g for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:31:45.668 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 13 03:31:45 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-g[152178]: 2023-09-13T03:31:45.361+0000 7f5ff3b71700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T03:31:45.669 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 13 03:31:45 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-g[152178]: 2023-09-13T03:31:45.361+0000 7f5ff3b71700 -1 mds.g *** got signal Terminated *** 2023-09-13T03:31:48.668 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 13 03:31:48 smithi064 bash[182538]: ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-g 2023-09-13T03:31:48.876 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mds.g.service' 2023-09-13T03:31:48.958 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T03:31:48.959 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-09-13T03:31:48.960 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-09-13T03:31:48.960 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mds.j 2023-09-13T03:31:49.168 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 13 03:31:49 smithi064 systemd[1]: Stopping Ceph mds.j for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:31:49.918 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 13 03:31:49 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-j[151200]: 2023-09-13T03:31:49.452+0000 7fcdf95ca700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T03:31:49.919 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 13 03:31:49 smithi064 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-j[151200]: 2023-09-13T03:31:49.452+0000 7fcdf95ca700 -1 mds.j *** got signal Terminated *** 2023-09-13T03:31:53.600 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 13 03:31:53 smithi064 bash[182683]: ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-j 2023-09-13T03:31:53.800 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mds.j.service' 2023-09-13T03:31:53.862 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 13 03:31:53 smithi064 systemd[1]: ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mds.j.service: Succeeded. 2023-09-13T03:31:53.862 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 13 03:31:53 smithi064 systemd[1]: Stopped Ceph mds.j for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da. 2023-09-13T03:31:53.886 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T03:31:53.887 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-09-13T03:31:53.887 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-09-13T03:31:53.887 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mds.b 2023-09-13T03:31:54.245 INFO:journalctl@ceph.mds.b.smithi142.stdout:Sep 13 03:31:53 smithi142 systemd[1]: Stopping Ceph mds.b for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:31:54.745 INFO:journalctl@ceph.mds.b.smithi142.stdout:Sep 13 03:31:54 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-b[170662]: 2023-09-13T03:31:54.251+0000 7f9529ad8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T03:31:54.746 INFO:journalctl@ceph.mds.b.smithi142.stdout:Sep 13 03:31:54 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-b[170662]: 2023-09-13T03:31:54.251+0000 7f9529ad8700 -1 mds.b *** got signal Terminated *** 2023-09-13T03:31:56.745 INFO:journalctl@ceph.mds.b.smithi142.stdout:Sep 13 03:31:56 smithi142 bash[179994]: ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-b 2023-09-13T03:31:56.825 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mds.b.service' 2023-09-13T03:31:56.862 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T03:31:56.863 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-09-13T03:31:56.864 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-09-13T03:31:56.864 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mds.e 2023-09-13T03:31:57.246 INFO:journalctl@ceph.mds.e.smithi142.stdout:Sep 13 03:31:56 smithi142 systemd[1]: Stopping Ceph mds.e for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:31:57.745 INFO:journalctl@ceph.mds.e.smithi142.stdout:Sep 13 03:31:57 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-e[169981]: 2023-09-13T03:31:57.297+0000 7f576fa69700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T03:31:57.746 INFO:journalctl@ceph.mds.e.smithi142.stdout:Sep 13 03:31:57 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-e[169981]: 2023-09-13T03:31:57.297+0000 7f576fa69700 -1 mds.e *** got signal Terminated *** 2023-09-13T03:32:00.689 INFO:journalctl@ceph.mds.e.smithi142.stdout:Sep 13 03:32:00 smithi142 bash[180139]: ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-e 2023-09-13T03:32:00.695 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mds.e.service' 2023-09-13T03:32:00.729 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T03:32:00.730 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-09-13T03:32:00.730 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-09-13T03:32:00.731 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mds.h 2023-09-13T03:32:00.995 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:32:00 smithi142 systemd[1]: Stopping Ceph mds.h for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:32:01.495 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:32:01 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-h[170323]: 2023-09-13T03:32:01.108+0000 7fbae825d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T03:32:01.496 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:32:01 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-h[170323]: 2023-09-13T03:32:01.108+0000 7fbae825d700 -1 mds.h *** got signal Terminated *** 2023-09-13T03:32:04.579 INFO:journalctl@ceph.mds.h.smithi142.stdout:Sep 13 03:32:04 smithi142 bash[180279]: ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-h 2023-09-13T03:32:04.764 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mds.h.service' 2023-09-13T03:32:04.797 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T03:32:04.801 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-09-13T03:32:04.802 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-09-13T03:32:04.802 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mds.k 2023-09-13T03:32:05.183 INFO:journalctl@ceph.mds.k.smithi142.stdout:Sep 13 03:32:04 smithi142 systemd[1]: Stopping Ceph mds.k for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:32:05.495 INFO:journalctl@ceph.mds.k.smithi142.stdout:Sep 13 03:32:05 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-k[169640]: 2023-09-13T03:32:05.182+0000 7f69572b5700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T03:32:05.496 INFO:journalctl@ceph.mds.k.smithi142.stdout:Sep 13 03:32:05 smithi142 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-k[169640]: 2023-09-13T03:32:05.182+0000 7f69572b5700 -1 mds.k *** got signal Terminated *** 2023-09-13T03:32:08.544 INFO:journalctl@ceph.mds.k.smithi142.stdout:Sep 13 03:32:08 smithi142 bash[180418]: ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-k 2023-09-13T03:32:08.743 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mds.k.service' 2023-09-13T03:32:08.777 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T03:32:08.777 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-09-13T03:32:08.778 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-09-13T03:32:08.778 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mds.c 2023-09-13T03:32:09.160 INFO:journalctl@ceph.mds.c.smithi170.stdout:Sep 13 03:32:08 smithi170 systemd[1]: Stopping Ceph mds.c for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:32:09.660 INFO:journalctl@ceph.mds.c.smithi170.stdout:Sep 13 03:32:09 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-c[171964]: 2023-09-13T03:32:09.270+0000 7f1f96236700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T03:32:09.660 INFO:journalctl@ceph.mds.c.smithi170.stdout:Sep 13 03:32:09 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-c[171964]: 2023-09-13T03:32:09.270+0000 7f1f96236700 -1 mds.c *** got signal Terminated *** 2023-09-13T03:32:12.660 INFO:journalctl@ceph.mds.c.smithi170.stdout:Sep 13 03:32:12 smithi170 bash[180999]: ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-c 2023-09-13T03:32:12.765 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mds.c.service' 2023-09-13T03:32:12.803 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T03:32:12.804 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-09-13T03:32:12.804 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-09-13T03:32:12.805 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mds.f 2023-09-13T03:32:13.160 INFO:journalctl@ceph.mds.f.smithi170.stdout:Sep 13 03:32:12 smithi170 systemd[1]: Stopping Ceph mds.f for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:32:13.660 INFO:journalctl@ceph.mds.f.smithi170.stdout:Sep 13 03:32:13 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-f[171621]: 2023-09-13T03:32:13.194+0000 7f322e5b6700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T03:32:13.661 INFO:journalctl@ceph.mds.f.smithi170.stdout:Sep 13 03:32:13 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-f[171621]: 2023-09-13T03:32:13.194+0000 7f322e5b6700 -1 mds.f *** got signal Terminated *** 2023-09-13T03:32:16.942 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mds.f.service' 2023-09-13T03:32:16.981 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T03:32:16.982 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-09-13T03:32:16.982 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-09-13T03:32:16.983 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mds.i 2023-09-13T03:32:17.356 INFO:journalctl@ceph.mds.i.smithi170.stdout:Sep 13 03:32:17 smithi170 systemd[1]: Stopping Ceph mds.i for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:32:17.660 INFO:journalctl@ceph.mds.i.smithi170.stdout:Sep 13 03:32:17 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-i[172301]: 2023-09-13T03:32:17.355+0000 7f6282484700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T03:32:17.660 INFO:journalctl@ceph.mds.i.smithi170.stdout:Sep 13 03:32:17 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-i[172301]: 2023-09-13T03:32:17.355+0000 7f6282484700 -1 mds.i *** got signal Terminated *** 2023-09-13T03:32:20.910 INFO:journalctl@ceph.mds.i.smithi170.stdout:Sep 13 03:32:20 smithi170 bash[181285]: ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-i 2023-09-13T03:32:20.990 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mds.i.service' 2023-09-13T03:32:21.025 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T03:32:21.025 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-09-13T03:32:21.026 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-09-13T03:32:21.026 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mds.l 2023-09-13T03:32:21.410 INFO:journalctl@ceph.mds.l.smithi170.stdout:Sep 13 03:32:21 smithi170 systemd[1]: Stopping Ceph mds.l for f8dbced4-51e1-11ee-9ab7-7b867c8bd7da... 2023-09-13T03:32:21.910 INFO:journalctl@ceph.mds.l.smithi170.stdout:Sep 13 03:32:21 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-l[171280]: 2023-09-13T03:32:21.435+0000 7f83bd5ef700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-13T03:32:21.910 INFO:journalctl@ceph.mds.l.smithi170.stdout:Sep 13 03:32:21 smithi170 ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-l[171280]: 2023-09-13T03:32:21.435+0000 7f83bd5ef700 -1 mds.l *** got signal Terminated *** 2023-09-13T03:32:24.660 INFO:journalctl@ceph.mds.l.smithi170.stdout:Sep 13 03:32:24 smithi170 bash[181435]: ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da-mds-l 2023-09-13T03:32:24.769 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8dbced4-51e1-11ee-9ab7-7b867c8bd7da@mds.l.service' 2023-09-13T03:32:24.803 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-13T03:32:24.804 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-09-13T03:32:24.804 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da --force --keep-logs 2023-09-13T03:32:25.209 INFO:teuthology.orchestra.run.smithi064.stdout:Deleting cluster with fsid: f8dbced4-51e1-11ee-9ab7-7b867c8bd7da 2023-09-13T03:32:28.609 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da --force --keep-logs 2023-09-13T03:32:28.912 INFO:teuthology.orchestra.run.smithi142.stdout:Deleting cluster with fsid: f8dbced4-51e1-11ee-9ab7-7b867c8bd7da 2023-09-13T03:32:30.469 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da --force --keep-logs 2023-09-13T03:32:30.768 INFO:teuthology.orchestra.run.smithi170.stdout:Deleting cluster with fsid: f8dbced4-51e1-11ee-9ab7-7b867c8bd7da 2023-09-13T03:32:32.676 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-13T03:32:32.758 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-13T03:32:32.795 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-13T03:32:32.826 INFO:tasks.cephadm:Archiving crash dumps... 2023-09-13T03:32:32.828 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/crash to /home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395256/remote/smithi064/crash 2023-09-13T03:32:32.829 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/crash -- . 2023-09-13T03:32:32.912 INFO:teuthology.orchestra.run.smithi064.stderr:tar: /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/crash: Cannot open: No such file or directory 2023-09-13T03:32:32.913 INFO:teuthology.orchestra.run.smithi064.stderr:tar: Error is not recoverable: exiting now 2023-09-13T03:32:32.916 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/crash to /home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395256/remote/smithi142/crash 2023-09-13T03:32:32.917 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/crash -- . 2023-09-13T03:32:32.963 INFO:teuthology.orchestra.run.smithi142.stderr:tar: /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/crash: Cannot open: No such file or directory 2023-09-13T03:32:32.964 INFO:teuthology.orchestra.run.smithi142.stderr:tar: Error is not recoverable: exiting now 2023-09-13T03:32:32.966 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/crash to /home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395256/remote/smithi170/crash 2023-09-13T03:32:32.967 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/crash -- . 2023-09-13T03:32:33.057 INFO:teuthology.orchestra.run.smithi170.stderr:tar: /var/lib/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/crash: Cannot open: No such file or directory 2023-09-13T03:32:33.058 INFO:teuthology.orchestra.run.smithi170.stderr:tar: Error is not recoverable: exiting now 2023-09-13T03:32:33.059 INFO:tasks.cephadm:Checking cluster log for badness... 2023-09-13T03:32:33.060 DEBUG:teuthology.orchestra.run.smithi064:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-09-13T03:32:33.141 INFO:teuthology.orchestra.run.smithi064.stderr:grep: /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph.log: No such file or directory 2023-09-13T03:32:33.146 INFO:tasks.cephadm:Compressing logs... 2023-09-13T03:32:33.150 DEBUG:teuthology.orchestra.run.smithi064:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-09-13T03:32:33.155 DEBUG:teuthology.orchestra.run.smithi142:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-09-13T03:32:33.157 DEBUG:teuthology.orchestra.run.smithi170:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-09-13T03:32:33.187 INFO:teuthology.orchestra.run.smithi142.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-09-13T03:32:33.188 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-osd.10.log 2023-09-13T03:32:33.189 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mds.h.log 2023-09-13T03:32:33.190 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mds.e.log 2023-09-13T03:32:33.191 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-osd.10.log: /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-osd.4.log 2023-09-13T03:32:33.191 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-osd.1.log 2023-09-13T03:32:33.191 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mds.k.log 2023-09-13T03:32:33.192 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-osd.1.log: /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-volume.log 2023-09-13T03:32:33.195 INFO:teuthology.orchestra.run.smithi170.stderr:find: ‘/var/log/rbd-target-api’gzip -5 : No such file or directory 2023-09-13T03:32:33.196 INFO:teuthology.orchestra.run.smithi170.stderr:--verbose -- /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-osd.5.log 2023-09-13T03:32:33.200 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-volume.log: 91.0% -- replaced with /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-volume.log.gz 2023-09-13T03:32:33.200 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-osd.7.log 2023-09-13T03:32:33.201 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mon.b.log 2023-09-13T03:32:33.204 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-osd.2.log 2023-09-13T03:32:33.204 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-osd.11.log 2023-09-13T03:32:33.205 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mgr.z.log 2023-09-13T03:32:33.205 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-osd.8.log 2023-09-13T03:32:33.205 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mds.f.log 2023-09-13T03:32:33.206 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mds.c.log 2023-09-13T03:32:33.206 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mds.i.log 2023-09-13T03:32:33.207 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mon.c.log 2023-09-13T03:32:33.208 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-volume.log 2023-09-13T03:32:33.208 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mds.i.log: /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mds.l.log 2023-09-13T03:32:33.208 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-09-13T03:32:33.209 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mds.l.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mgr.z.log.gz 2023-09-13T03:32:33.212 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mds.b.log 2023-09-13T03:32:33.218 INFO:teuthology.orchestra.run.smithi064.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-09-13T03:32:33.218 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-osd.0.log 2023-09-13T03:32:33.219 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mds.g.log 2023-09-13T03:32:33.220 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mds.a.log 2023-09-13T03:32:33.221 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-osd.6.log 2023-09-13T03:32:33.222 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-osd.3.log 2023-09-13T03:32:33.224 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mds.d.log 2023-09-13T03:32:33.225 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mgr.x.log 2023-09-13T03:32:33.227 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-osd.9.log 2023-09-13T03:32:33.229 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mgr.y.log 2023-09-13T03:32:33.236 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mgr.x.log: /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-osd.9.log: 94.5% -- replaced with /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mds.d.log.gz 2023-09-13T03:32:33.236 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-volume.log 2023-09-13T03:32:33.247 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mds.j.log 2023-09-13T03:32:33.247 INFO:teuthology.orchestra.run.smithi064.stderr: 91.0% -- replaced with /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-volume.log.gz 2023-09-13T03:32:33.251 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mgr.y.log: 92.1% -- replaced with /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mgr.y.log.gz 2023-09-13T03:32:33.252 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-09-13T03:32:33.258 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mon.a.log 2023-09-13T03:32:33.258 INFO:teuthology.orchestra.run.smithi064.stderr: 94.5% -- replaced with /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mds.j.log.gz 2023-09-13T03:32:33.269 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-09-13T03:32:33.279 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/cephadm.log: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2023-09-13T03:32:33.291 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/cephadm.log: 92.9% -- replaced with /var/log/ceph/cephadm.log.gz 2023-09-13T03:32:33.641 INFO:teuthology.orchestra.run.smithi064.stderr: 89.8% -- replaced with /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mgr.x.log.gz 2023-09-13T03:32:33.738 INFO:teuthology.orchestra.run.smithi170.stderr: 90.9% -- replaced with /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-volume.log.gz 2023-09-13T03:32:33.874 INFO:teuthology.orchestra.run.smithi142.stderr: 91.9% -- replaced with /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mon.b.log.gz 2023-09-13T03:32:34.554 INFO:teuthology.orchestra.run.smithi170.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2023-09-13T03:32:34.896 INFO:teuthology.orchestra.run.smithi064.stderr: 90.3% -- replaced with /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mon.a.log.gz 2023-09-13T03:32:35.037 INFO:teuthology.orchestra.run.smithi170.stderr: 93.1% -- replaced with /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mon.c.log.gz 2023-09-13T03:32:37.102 INFO:teuthology.orchestra.run.smithi064.stderr: 95.1% -- replaced with /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mds.g.log.gz 2023-09-13T03:32:37.608 INFO:teuthology.orchestra.run.smithi064.stderr: 95.2% -- replaced with /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mds.a.log.gz 2023-09-13T03:32:37.830 INFO:teuthology.orchestra.run.smithi142.stderr: 95.2% -- replaced with /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mds.k.log.gz 2023-09-13T03:32:38.682 INFO:teuthology.orchestra.run.smithi170.stderr: 95.1% -- replaced with /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mds.l.log.gz 2023-09-13T03:32:39.383 INFO:teuthology.orchestra.run.smithi142.stderr: 95.2% -- replaced with /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mds.e.log.gz 2023-09-13T03:32:41.153 INFO:teuthology.orchestra.run.smithi142.stderr: 94.1% -- replaced with /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-osd.4.log.gz 2023-09-13T03:32:43.057 INFO:teuthology.orchestra.run.smithi064.stderr: 93.8% -- replaced with /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-osd.0.log.gz 2023-09-13T03:32:43.411 INFO:teuthology.orchestra.run.smithi142.stderr: 93.8% -- replaced with /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-osd.1.log.gz 2023-09-13T03:32:43.980 INFO:teuthology.orchestra.run.smithi064.stderr: 93.8% -- replaced with /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-osd.9.log.gz 2023-09-13T03:32:44.242 INFO:teuthology.orchestra.run.smithi064.stderr: 93.8% -- replaced with /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-osd.6.log.gz 2023-09-13T03:32:45.116 INFO:teuthology.orchestra.run.smithi142.stderr: 93.9% -- replaced with /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-osd.10.log.gz 2023-09-13T03:32:46.572 INFO:teuthology.orchestra.run.smithi064.stderr: 93.7% -- replaced with /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-osd.3.log.gz 2023-09-13T03:32:46.577 INFO:teuthology.orchestra.run.smithi064.stderr: 2023-09-13T03:32:46.578 INFO:teuthology.orchestra.run.smithi064.stderr:real 0m13.385s 2023-09-13T03:32:46.578 INFO:teuthology.orchestra.run.smithi064.stderr:user 0m51.278s 2023-09-13T03:32:46.579 INFO:teuthology.orchestra.run.smithi064.stderr:sys 0m4.092s 2023-09-13T03:32:46.775 INFO:teuthology.orchestra.run.smithi170.stderr: 93.9% -- replaced with /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-osd.11.log.gz 2023-09-13T03:32:47.679 INFO:teuthology.orchestra.run.smithi170.stderr: 93.8% -- replaced with /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-osd.2.log.gz 2023-09-13T03:32:48.177 INFO:teuthology.orchestra.run.smithi142.stderr: 93.9% -- replaced with /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-osd.7.log.gz 2023-09-13T03:32:51.747 INFO:teuthology.orchestra.run.smithi170.stderr: 93.9% -- replaced with /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-osd.8.log.gz 2023-09-13T03:32:52.417 INFO:teuthology.orchestra.run.smithi170.stderr: 93.9% -- replaced with /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-osd.5.log.gz 2023-09-13T03:33:34.790 INFO:teuthology.orchestra.run.smithi142.stderr: 93.1% -- replaced with /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mds.h.log.gz 2023-09-13T03:33:46.357 INFO:teuthology.orchestra.run.smithi170.stderr: 93.0% -- replaced with /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mds.c.log.gz 2023-09-13T03:33:54.536 INFO:teuthology.orchestra.run.smithi170.stderr: 93.0% -- replaced with /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mds.i.log.gz 2023-09-13T03:33:56.674 INFO:teuthology.orchestra.run.smithi170.stderr: 93.1% -- replaced with /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mds.f.log.gz 2023-09-13T03:33:56.676 INFO:teuthology.orchestra.run.smithi170.stderr: 2023-09-13T03:33:56.676 INFO:teuthology.orchestra.run.smithi170.stderr:real 1m23.502s 2023-09-13T03:33:56.676 INFO:teuthology.orchestra.run.smithi170.stderr:user 3m59.625s 2023-09-13T03:33:56.677 INFO:teuthology.orchestra.run.smithi170.stderr:sys 0m8.696s 2023-09-13T03:33:57.274 INFO:teuthology.orchestra.run.smithi142.stderr: 93.3% -- replaced with /var/log/ceph/f8dbced4-51e1-11ee-9ab7-7b867c8bd7da/ceph-mds.b.log.gz 2023-09-13T03:33:57.275 INFO:teuthology.orchestra.run.smithi142.stderr: 2023-09-13T03:33:57.276 INFO:teuthology.orchestra.run.smithi142.stderr:real 1m24.102s 2023-09-13T03:33:57.276 INFO:teuthology.orchestra.run.smithi142.stderr:user 3m14.477s 2023-09-13T03:33:57.276 INFO:teuthology.orchestra.run.smithi142.stderr:sys 0m6.599s 2023-09-13T03:33:57.277 INFO:tasks.cephadm:Archiving logs... 2023-09-13T03:33:57.278 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395256/remote/smithi064/log 2023-09-13T03:33:57.284 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-13T03:34:03.757 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395256/remote/smithi142/log 2023-09-13T03:34:03.787 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-13T03:34:30.353 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395256/remote/smithi170/log 2023-09-13T03:34:30.366 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-13T03:35:05.094 INFO:tasks.cephadm:Removing cluster... 2023-09-13T03:35:05.095 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da --force 2023-09-13T03:35:05.512 INFO:teuthology.orchestra.run.smithi064.stdout:Deleting cluster with fsid: f8dbced4-51e1-11ee-9ab7-7b867c8bd7da 2023-09-13T03:35:06.385 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da --force 2023-09-13T03:35:06.697 INFO:teuthology.orchestra.run.smithi142.stdout:Deleting cluster with fsid: f8dbced4-51e1-11ee-9ab7-7b867c8bd7da 2023-09-13T03:35:07.404 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f8dbced4-51e1-11ee-9ab7-7b867c8bd7da --force 2023-09-13T03:35:07.720 INFO:teuthology.orchestra.run.smithi170.stdout:Deleting cluster with fsid: f8dbced4-51e1-11ee-9ab7-7b867c8bd7da 2023-09-13T03:35:08.495 INFO:tasks.cephadm:Removing cephadm ... 2023-09-13T03:35:08.497 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf /home/ubuntu/cephtest/cephadm 2023-09-13T03:35:08.546 DEBUG:teuthology.orchestra.run.smithi142:> rm -rf /home/ubuntu/cephtest/cephadm 2023-09-13T03:35:08.568 DEBUG:teuthology.orchestra.run.smithi170:> rm -rf /home/ubuntu/cephtest/cephadm 2023-09-13T03:35:08.742 INFO:tasks.cephadm:Teardown complete 2023-09-13T03:35:08.743 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-09-13T03:35:08.756 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-09-13T03:35:08.756 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-13T03:35:08.762 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-13T03:35:08.764 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-13T03:35:08.918 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-09-13T03:35:08.929 INFO:teuthology.task.clock:Checking final clock skew... 2023-09-13T03:35:08.929 DEBUG:teuthology.orchestra.run.smithi064:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-13T03:35:08.934 DEBUG:teuthology.orchestra.run.smithi142:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-13T03:35:08.937 DEBUG:teuthology.orchestra.run.smithi170:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-13T03:35:08.953 INFO:teuthology.orchestra.run.smithi170.stderr:bash: ntpq: command not found 2023-09-13T03:35:08.954 INFO:teuthology.orchestra.run.smithi142.stderr:bash: ntpq: command not found 2023-09-13T03:35:08.959 INFO:teuthology.orchestra.run.smithi142.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-13T03:35:08.959 INFO:teuthology.orchestra.run.smithi142.stdout:=============================================================================== 2023-09-13T03:35:08.959 INFO:teuthology.orchestra.run.smithi142.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 0 -251us[ -251us] +/- 90ms 2023-09-13T03:35:08.960 INFO:teuthology.orchestra.run.smithi142.stdout:^* hv02.front.sepia.ceph.com 3 6 377 60 -374us[ -370us] +/- 50ms 2023-09-13T03:35:08.960 INFO:teuthology.orchestra.run.smithi142.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 61 +1006us[+1010us] +/- 90ms 2023-09-13T03:35:08.960 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-09-13T03:35:08.973 INFO:teuthology.orchestra.run.smithi064.stderr:bash: ntpq: command not found 2023-09-13T03:35:08.978 INFO:teuthology.orchestra.run.smithi170.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-13T03:35:08.979 INFO:teuthology.orchestra.run.smithi170.stdout:=============================================================================== 2023-09-13T03:35:08.979 INFO:teuthology.orchestra.run.smithi170.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 61 -275us[ -269us] +/- 90ms 2023-09-13T03:35:08.979 INFO:teuthology.orchestra.run.smithi170.stdout:^* hv02.front.sepia.ceph.com 3 6 377 58 -402us[ -396us] +/- 50ms 2023-09-13T03:35:08.980 INFO:teuthology.orchestra.run.smithi170.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 0 +991us[ +991us] +/- 59ms 2023-09-13T03:35:08.980 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-09-13T03:35:08.981 INFO:teuthology.orchestra.run.smithi064.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-13T03:35:08.982 INFO:teuthology.orchestra.run.smithi064.stdout:=============================================================================== 2023-09-13T03:35:08.982 INFO:teuthology.orchestra.run.smithi064.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 59 -259us[ -259us] +/- 90ms 2023-09-13T03:35:08.982 INFO:teuthology.orchestra.run.smithi064.stdout:^* hv02.front.sepia.ceph.com 3 6 377 61 -385us[ -379us] +/- 50ms 2023-09-13T03:35:08.983 INFO:teuthology.orchestra.run.smithi064.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 60 +994us[ +994us] +/- 90ms 2023-09-13T03:35:08.983 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-09-13T03:35:08.984 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-09-13T03:35:08.995 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-09-13T03:35:08.995 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-09-13T03:35:09.029 DEBUG:teuthology.orchestra.run.smithi064:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-09-13T03:35:09.212 DEBUG:teuthology.orchestra.run.smithi142:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-09-13T03:35:09.315 DEBUG:teuthology.orchestra.run.smithi170:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-09-13T03:35:09.431 DEBUG:teuthology.orchestra.run.smithi064:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-13T03:35:09.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-13T03:35:09.509 DEBUG:teuthology.orchestra.run.smithi142:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-13T03:35:09.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-13T03:35:09.544 DEBUG:teuthology.orchestra.run.smithi170:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-13T03:35:09.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-13T03:35:09.577 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-09-13T03:35:09.589 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-09-13T03:35:09.637 INFO:teuthology.task.internal:Duration was 2870.957368 seconds 2023-09-13T03:35:09.638 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-09-13T03:35:09.688 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-09-13T03:35:09.689 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-13T03:35:09.694 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-13T03:35:09.696 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-13T03:35:09.744 INFO:teuthology.orchestra.run.smithi142.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-13T03:35:09.805 INFO:teuthology.orchestra.run.smithi170.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-13T03:35:09.809 INFO:teuthology.orchestra.run.smithi064.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-13T03:35:10.199 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-09-13T03:35:10.200 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi064.front.sepia.ceph.com 2023-09-13T03:35:10.200 DEBUG:teuthology.orchestra.run.smithi064:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-13T03:35:10.259 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi142.front.sepia.ceph.com 2023-09-13T03:35:10.260 DEBUG:teuthology.orchestra.run.smithi142:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-13T03:35:10.284 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi170.front.sepia.ceph.com 2023-09-13T03:35:10.284 DEBUG:teuthology.orchestra.run.smithi170:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-13T03:35:10.311 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-09-13T03:35:10.312 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-13T03:35:10.316 DEBUG:teuthology.orchestra.run.smithi142:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-13T03:35:10.326 DEBUG:teuthology.orchestra.run.smithi170:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-13T03:35:10.414 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-09-13T03:35:10.415 DEBUG:teuthology.orchestra.run.smithi064:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-13T03:35:10.420 DEBUG:teuthology.orchestra.run.smithi142:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-13T03:35:10.423 DEBUG:teuthology.orchestra.run.smithi170:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-13T03:35:10.588 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-09-13T03:35:10.600 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-09-13T03:35:10.601 DEBUG:teuthology.orchestra.run.smithi064:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-13T03:35:10.670 DEBUG:teuthology.orchestra.run.smithi142:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-13T03:35:10.697 DEBUG:teuthology.orchestra.run.smithi170:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-13T03:35:10.729 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-09-13T03:35:10.741 DEBUG:teuthology.orchestra.run.smithi064:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-13T03:35:10.746 DEBUG:teuthology.orchestra.run.smithi142:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-13T03:35:10.749 DEBUG:teuthology.orchestra.run.smithi170:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-13T03:35:10.772 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern = core 2023-09-13T03:35:10.796 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern = core 2023-09-13T03:35:10.810 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = core 2023-09-13T03:35:10.846 DEBUG:teuthology.orchestra.run.smithi064:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-13T03:35:10.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-13T03:35:10.888 DEBUG:teuthology.orchestra.run.smithi142:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-13T03:35:10.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-13T03:35:10.905 DEBUG:teuthology.orchestra.run.smithi170:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-13T03:35:10.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-13T03:35:10.922 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-09-13T03:35:10.934 INFO:teuthology.task.internal:Transferring archived files... 2023-09-13T03:35:10.936 DEBUG:teuthology.misc:Transferring archived files from smithi064:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395256/remote/smithi064 2023-09-13T03:35:10.936 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-09-13T03:35:11.019 DEBUG:teuthology.misc:Transferring archived files from smithi142:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395256/remote/smithi142 2023-09-13T03:35:11.019 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-09-13T03:35:11.061 DEBUG:teuthology.misc:Transferring archived files from smithi170:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395256/remote/smithi170 2023-09-13T03:35:11.061 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-09-13T03:35:11.101 INFO:teuthology.task.internal:Removing archive directory... 2023-09-13T03:35:11.101 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf -- /home/ubuntu/cephtest/archive 2023-09-13T03:35:11.106 DEBUG:teuthology.orchestra.run.smithi142:> rm -rf -- /home/ubuntu/cephtest/archive 2023-09-13T03:35:11.109 DEBUG:teuthology.orchestra.run.smithi170:> rm -rf -- /home/ubuntu/cephtest/archive 2023-09-13T03:35:11.151 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-09-13T03:35:11.165 INFO:teuthology.task.internal:Not uploading archives. 2023-09-13T03:35:11.165 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-09-13T03:35:11.230 INFO:teuthology.task.internal:Tidying up after the test... 2023-09-13T03:35:11.230 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-13T03:35:11.235 DEBUG:teuthology.orchestra.run.smithi142:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-13T03:35:11.237 DEBUG:teuthology.orchestra.run.smithi170:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-13T03:35:11.252 INFO:teuthology.orchestra.run.smithi142.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 03:35 /home/ubuntu/cephtest 2023-09-13T03:35:11.254 INFO:teuthology.orchestra.run.smithi170.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 03:35 /home/ubuntu/cephtest 2023-09-13T03:35:11.274 INFO:teuthology.orchestra.run.smithi064.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 03:35 /home/ubuntu/cephtest 2023-09-13T03:35:11.277 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-09-13T03:35:11.289 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-09-13T03:35:11.381 INFO:teuthology.nuke:Checking targets against current locks 2023-09-13T03:35:11.406 DEBUG:teuthology.nuke:shortname: smithi064 2023-09-13T03:35:11.407 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-13T03:35:11.435 DEBUG:teuthology.nuke:shortname: smithi142 2023-09-13T03:35:11.435 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-13T03:35:11.440 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi064.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395256', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-13 02:38:41.933685', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-13T03:35:11.462 DEBUG:teuthology.nuke:shortname: smithi170 2023-09-13T03:35:11.463 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-13T03:35:11.467 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi142.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395256', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-13 02:38:41.938062', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-13T03:35:11.500 INFO:teuthology.orchestra.console.smithi064:Power off 2023-09-13T03:35:11.501 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-13T03:35:11.510 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi170.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-09-12_14:07:50-fs-wip-batrick-testing-20230912.122437-distro-default-smithi/7395256', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-13 02:38:41.935152', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-13T03:35:11.523 DEBUG:teuthology.orchestra.console.smithi064:power off output: Chassis Power Control: Down/Off 2023-09-13T03:35:11.524 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T03:35:11.544 INFO:teuthology.orchestra.console.smithi142:Power off 2023-09-13T03:35:11.545 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-13T03:35:11.555 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is on 2023-09-13T03:35:11.581 INFO:teuthology.orchestra.console.smithi170:Power off 2023-09-13T03:35:11.581 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-13T03:35:11.592 DEBUG:teuthology.orchestra.console.smithi142:power off output: Chassis Power Control: Down/Off 2023-09-13T03:35:11.593 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T03:35:11.606 DEBUG:teuthology.orchestra.console.smithi170:power off output: Chassis Power Control: Down/Off 2023-09-13T03:35:11.607 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T03:35:11.616 DEBUG:teuthology.orchestra.console.smithi142:check power output: Chassis Power is on 2023-09-13T03:35:11.629 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is on 2023-09-13T03:35:15.556 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T03:35:15.617 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T03:35:15.630 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T03:35:15.666 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is on 2023-09-13T03:35:15.727 DEBUG:teuthology.orchestra.console.smithi142:check power output: Chassis Power is on 2023-09-13T03:35:15.740 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is on 2023-09-13T03:35:19.670 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T03:35:19.729 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T03:35:19.742 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T03:35:19.784 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is on 2023-09-13T03:35:19.844 DEBUG:teuthology.orchestra.console.smithi142:check power output: Chassis Power is on 2023-09-13T03:35:19.857 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is off 2023-09-13T03:35:19.958 INFO:teuthology.orchestra.console.smithi170:Power off completed 2023-09-13T03:35:23.789 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T03:35:23.845 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T03:35:23.900 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is off 2023-09-13T03:35:23.956 DEBUG:teuthology.orchestra.console.smithi142:check power output: Chassis Power is off 2023-09-13T03:35:24.002 INFO:teuthology.orchestra.console.smithi064:Power off completed 2023-09-13T03:35:24.058 INFO:teuthology.orchestra.console.smithi142:Power off completed 2023-09-13T03:35:24.165 INFO:teuthology.run:Summary data: client.0-kernel-sha1: fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} duration: 2870.957367658615 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi064 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3711e7dcbd7a68fe399334914fa2d5c17ccc9c2f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_pdonnell@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=9b5adaaf424f46be8df3f3ab259b8cb6 status: fail success: false 2023-09-13T03:35:24.165 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T03:35:24.244 INFO:teuthology.run:FAIL